diff options
Diffstat (limited to 'docs')
63 files changed, 1519 insertions, 817 deletions
diff --git a/docs/COLO-FT.txt b/docs/COLO-FT.txt index 8ec653f81c..2e760a4aee 100644 --- a/docs/COLO-FT.txt +++ b/docs/COLO-FT.txt @@ -210,6 +210,7 @@ children.0=childs0 \ 3. On Secondary VM's QEMU monitor, issue command {"execute":"qmp_capabilities"} +{"execute": "migrate-set-capabilities", "arguments": {"capabilities": [ {"capability": "x-colo", "state": true } ] } } {"execute": "nbd-server-start", "arguments": {"addr": {"type": "inet", "data": {"host": "0.0.0.0", "port": "9999"} } } } {"execute": "nbd-server-add", "arguments": {"device": "parent0", "writable": true } } diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst index 89cae5a6bb..0e2cb9e770 100644 --- a/docs/about/build-platforms.rst +++ b/docs/about/build-platforms.rst @@ -98,7 +98,7 @@ Python runtime option of the ``configure`` script to point QEMU to a supported version of the Python runtime. - As of QEMU |version|, the minimum supported version of Python is 3.6. + As of QEMU |version|, the minimum supported version of Python is 3.7. Python build dependencies Some of QEMU's build dependencies are written in Python. Usually these diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 1ca9dc33d6..0743459862 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -111,6 +111,27 @@ Use ``-machine acpi=off`` instead. The HAXM project has been retired (see https://github.com/intel/haxm#status). Use "whpx" (on Windows) or "hvf" (on macOS) instead. +``-async-teardown`` (since 8.1) +''''''''''''''''''''''''''''''' + +Use ``-run-with async-teardown=on`` instead. + +``-singlestep`` (since 8.1) +''''''''''''''''''''''''''' + +The ``-singlestep`` option has been turned into an accelerator property, +and given a name that better reflects what it actually does. +Use ``-accel tcg,one-insn-per-tb=on`` instead. + +User-mode emulator command line arguments +----------------------------------------- + +``-singlestep`` (since 8.1) +''''''''''''''''''''''''''' + +The ``-singlestep`` option has been given a name that better reflects +what it actually does. For both linux-user and bsd-user, use the +new ``-one-insn-per-tb`` option instead. QEMU Machine Protocol (QMP) commands ------------------------------------ @@ -183,6 +204,45 @@ accepted incorrect commands will return an error. Users should make sure that all arguments passed to ``device_add`` are consistent with the documented property types. +``StatusInfo`` member ``singlestep`` (since 8.1) +'''''''''''''''''''''''''''''''''''''''''''''''' + +The ``singlestep`` member of the ``StatusInfo`` returned from the +``query-status`` command is deprecated. This member has a confusing +name and it never did what the documentation claimed or what its name +suggests. We do not believe that anybody is actually using the +information provided in this member. + +The information it reports is whether the TCG JIT is in "one +instruction per translated block" mode (which can be set on the +command line or via the HMP, but not via QMP). The information remains +available via the HMP 'info jit' command. + +QEMU Machine Protocol (QMP) events +---------------------------------- + +``MEM_UNPLUG_ERROR`` (since 6.2) +'''''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Use the more generic event ``DEVICE_UNPLUG_GUEST_ERROR`` instead. + +``vcpu`` trace events (since 8.1) +''''''''''''''''''''''''''''''''' + +The ability to instrument QEMU helper functions with vCPU-aware trace +points was removed in 7.0. However QMP still exposed the vcpu +parameter. This argument has now been deprecated and the remaining +remaining trace points that used it are selected just by name. + +Human Monitor Protocol (HMP) commands +------------------------------------- + +``singlestep`` (since 8.1) +'''''''''''''''''''''''''' + +The ``singlestep`` command has been replaced by the ``one-insn-per-tb`` +command, which has the same behaviour but a less misleading name. + Host Architectures ------------------ @@ -207,20 +267,11 @@ it. Since all recent x86 hardware from the past >10 years is capable of the 64-bit x86 extensions, a corresponding 64-bit OS should be used instead. -QEMU API (QAPI) events ----------------------- - -``MEM_UNPLUG_ERROR`` (since 6.2) -'''''''''''''''''''''''''''''''''''''''''''''''''''''''' - -Use the more generic event ``DEVICE_UNPLUG_GUEST_ERROR`` instead. - - System emulator machines ------------------------ -Arm ``virt`` machine ``dtb-kaslr-seed`` property -'''''''''''''''''''''''''''''''''''''''''''''''' +Arm ``virt`` machine ``dtb-kaslr-seed`` property (since 7.1) +'''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' The ``dtb-kaslr-seed`` property on the ``virt`` board has been deprecated; use the new name ``dtb-randomness`` instead. The new name @@ -284,6 +335,14 @@ from Intel that was not properly allocated. Since version 5.2, the controller has used a properly allocated identifier. Deprecate the ``use-intel-id`` machine compatibility parameter. +``-device cxl-type3,memdev=xxxx`` (since 8.0) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``cxl-type3`` device initially only used a single memory backend. With +the addition of volatile memory support, it is now necessary to distinguish +between persistent and volatile memory backends. As such, memdev is deprecated +in favor of persistent-memdev. + Block device options '''''''''''''''''''' diff --git a/docs/about/emulation.rst b/docs/about/emulation.rst index b510a54418..0ad0b86f0d 100644 --- a/docs/about/emulation.rst +++ b/docs/about/emulation.rst @@ -99,7 +99,7 @@ depending on the guest architecture. - Yes - A configurable 32 bit soft core now owned by Cadence -A number of features are are only available when running under +A number of features are only available when running under emulation including :ref:`Record/Replay<replay>` and :ref:`TCG Plugins`. .. _Semihosting: diff --git a/docs/conf.py b/docs/conf.py index 00767b0e24..e84a95e71c 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -32,15 +32,6 @@ import sphinx from distutils.version import LooseVersion from sphinx.errors import ConfigError -# Make Sphinx fail cleanly if using an old Python, rather than obscurely -# failing because some code in one of our extensions doesn't work there. -# In newer versions of Sphinx this will display nicely; in older versions -# Sphinx will also produce a Python backtrace but at least the information -# gets printed... -if sys.version_info < (3,6): - raise ConfigError( - "QEMU requires a Sphinx that uses Python 3.6 or better\n") - # The per-manual conf.py will set qemu_docdir for a single-manual build; # otherwise set it here if this is an entire-manual-set build. # This is always the absolute path of the docs/ directory in the source tree. @@ -98,7 +89,7 @@ default_role = 'any' # General information about the project. project = u'QEMU' -copyright = u'2022, The QEMU Project Developers' +copyright = u'2023, The QEMU Project Developers' author = u'The QEMU Project Developers' # The version info for the project you're documenting, acts as replacement for diff --git a/docs/devel/acpi-bits.rst b/docs/devel/acpi-bits.rst index 9eb4b9e666..22e2580200 100644 --- a/docs/devel/acpi-bits.rst +++ b/docs/devel/acpi-bits.rst @@ -135,7 +135,7 @@ Under ``tests/avocado/`` as the root we have: (c) They need not be loaded by avocado framework when running tests. -Author: Ani Sinha <ani@anisinha.ca> +Author: Ani Sinha <anisinha@redhat.com> References: ----------- diff --git a/docs/devel/atomics.rst b/docs/devel/atomics.rst index 248076375b..ff9b5ee30c 100644 --- a/docs/devel/atomics.rst +++ b/docs/devel/atomics.rst @@ -102,28 +102,10 @@ Similar operations return the new value of ``*ptr``:: typeof(*ptr) qatomic_or_fetch(ptr, val) typeof(*ptr) qatomic_xor_fetch(ptr, val) -``qemu/atomic.h`` also provides loads and stores that cannot be reordered -with each other:: +``qemu/atomic.h`` also provides an optimized shortcut for +``qatomic_set`` followed by ``smp_mb``:: - typeof(*ptr) qatomic_mb_read(ptr) - void qatomic_mb_set(ptr, val) - -However these do not provide sequential consistency and, in particular, -they do not participate in the total ordering enforced by -sequentially-consistent operations. For this reason they are deprecated. -They should instead be replaced with any of the following (ordered from -easiest to hardest): - -- accesses inside a mutex or spinlock - -- lightweight synchronization primitives such as ``QemuEvent`` - -- RCU operations (``qatomic_rcu_read``, ``qatomic_rcu_set``) when publishing - or accessing a new version of a data structure - -- other atomic accesses: ``qatomic_read`` and ``qatomic_load_acquire`` for - loads, ``qatomic_set`` and ``qatomic_store_release`` for stores, ``smp_mb`` - to forbid reordering subsequent loads before a store. + void qatomic_set_mb(ptr, val) Weak atomic access and manual memory barriers @@ -523,8 +505,7 @@ and memory barriers, and the equivalents in QEMU: | :: | | | | a = qatomic_read(&x); | - | qatomic_set(&x, a + 2); | - | smp_mb(); | + | qatomic_set_mb(&x, a + 2); | | b = qatomic_read(&y); | +--------------------------------+ diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst index 1894721743..64efa26b90 100644 --- a/docs/devel/build-system.rst +++ b/docs/devel/build-system.rst @@ -4,30 +4,14 @@ The QEMU build system architecture This document aims to help developers understand the architecture of the QEMU build system. As with projects using GNU autotools, the QEMU build -system has two stages, first the developer runs the "configure" script +system has two stages; first the developer runs the "configure" script to determine the local build environment characteristics, then they run -"make" to build the project. There is about where the similarities with +"make" to build the project. This is about where the similarities with GNU autotools end, so try to forget what you know about them. +The two general ways to perform a build are as follows: -Stage 1: configure -================== - -The QEMU configure script is written directly in shell, and should be -compatible with any POSIX shell, hence it uses #!/bin/sh. An important -implication of this is that it is important to avoid using bash-isms on -development platforms where bash is the primary host. - -In contrast to autoconf scripts, QEMU's configure is expected to be -silent while it is checking for features. It will only display output -when an error occurs, or to show the final feature enablement summary -on completion. - -Because QEMU uses the Meson build system under the hood, only VPATH -builds are supported. There are two general ways to invoke configure & -perform a build: - - - VPATH, build artifacts outside of QEMU source tree entirely:: + - build artifacts outside of QEMU source tree entirely:: cd ../ mkdir build @@ -35,88 +19,122 @@ perform a build: ../qemu/configure make - - VPATH, build artifacts in a subdir of QEMU source tree:: + - build artifacts in a subdir of QEMU source tree:: mkdir build cd build ../configure make -The configure script automatically recognizes -command line options for which a same-named Meson option exists; -dashes in the command line are replaced with underscores. +Most of the actual build process uses Meson under the hood, therefore +build artifacts cannot be placed in the source tree itself. + + +Stage 1: configure +================== + +The configure script has five tasks: + + - detect the host architecture -Many checks on the compilation environment are still found in configure -rather than ``meson.build``, but new checks should be added directly to -``meson.build``. + - list the targets for which to build emulators; the list of + targets also affects which firmware binaries and tests to build + + - find the compilers (native and cross) used to build executables, + firmware and tests. The results are written as either Makefile + fragments (``config-host.mak``) or a Meson machine file + (``config-meson.cross``) + + - create a virtual environment in which all Python code runs during + the build, and possibly install packages into it from PyPI + + - invoke Meson in the virtual environment, to perform the actual + configuration step for the emulator build + +The configure script automatically recognizes command line options for +which a same-named Meson option exists; dashes in the command line are +replaced with underscores. + +Almost all QEMU developers that need to modify the build system will +only be concerned with Meson, and therefore can skip the rest of this +section. + + +Modifying ``configure`` +----------------------- + +``configure`` is a shell script; it uses ``#!/bin/sh`` and therefore +should be compatible with any POSIX shell. It is important to avoid +using bash-isms to avoid breaking development platforms where bash is +the primary host. + +The configure script provides a variety of functions to help writing +portable shell code and providing consistent behavior across architectures +and operating systems: + +``error_exit $MESSAGE $MORE...`` + Print $MESSAGE to stderr, followed by $MORE... and then exit from the + configure script with non-zero status. -Patches are also welcome to move existing checks from the configure -phase to ``meson.build``. When doing so, ensure that ``meson.build`` does -not use anymore the keys that you have removed from ``config-host.mak``. -Typically these will be replaced in ``meson.build`` by boolean variables, -``get_option('optname')`` invocations, or ``dep.found()`` expressions. -In general, the remaining checks have little or no interdependencies, -so they can be moved one by one. +``has $COMMAND`` + Determine if $COMMAND exists in the current environment, either as a + shell builtin, or executable binary, returning 0 on success. The + replacement in Meson is ``find_program()``. -Helper functions ----------------- +``probe_target_compiler $TARGET`` + Detect a cross compiler and cross tools for the QEMU target $TARGET (e.g., + ``$CPU-softmmu``, ``$CPU-linux-user``, ``$CPU-bsd-user``). If a working + compiler is present, return success and set variables ``$target_cc``, + ``$target_ar``, etc. to non-empty values. -The configure script provides a variety of helper functions to assist -developers in checking for system features: +``write_target_makefile`` + Write a Makefile fragment to stdout, exposing the result of the most + ``probe_target_compiler`` call as the usual Make variables (``CC``, + ``AR``, ``LD``, etc.). -``do_cc $ARGS...`` - Attempt to run the system C compiler passing it $ARGS... -``do_cxx $ARGS...`` - Attempt to run the system C++ compiler passing it $ARGS... +Configure does not generally perform tests for compiler options beyond +basic checks to detect the host platform and ensure the compiler is +functioning. These are performed using a few more helper functions: ``compile_object $CFLAGS`` Attempt to compile a test program with the system C compiler using $CFLAGS. The test program must have been previously written to a file - called $TMPC. The replacement in Meson is the compiler object ``cc``, - which has methods such as ``cc.compiles()``, - ``cc.check_header()``, ``cc.has_function()``. + called $TMPC. ``compile_prog $CFLAGS $LDFLAGS`` Attempt to compile a test program with the system C compiler using $CFLAGS and link it with the system linker using $LDFLAGS. The test program must have been previously written to a file called $TMPC. - The replacement in Meson is ``cc.find_library()`` and ``cc.links()``. - -``has $COMMAND`` - Determine if $COMMAND exists in the current environment, either as a - shell builtin, or executable binary, returning 0 on success. The - replacement in Meson is ``find_program()``. ``check_define $NAME`` - Determine if the macro $NAME is defined by the system C compiler + Determine if the macro $NAME is defined by the system C compiler. -``check_include $NAME`` - Determine if the include $NAME file is available to the system C - compiler. The replacement in Meson is ``cc.has_header()``. +``do_compiler $CC $ARGS...`` + Attempt to run the C compiler $CC, passing it $ARGS... This function + does not use flags passed via options such as ``--extra-cflags``, and + therefore can be used to check for cross compilers. However, most + such checks are done at ``make`` time instead (see for example the + ``cc-option`` macro in ``pc-bios/option-rom/Makefile``). ``write_c_skeleton`` Write a minimal C program main() function to the temporary file - indicated by $TMPC + indicated by $TMPC. -``error_exit $MESSAGE $MORE...`` - Print $MESSAGE to stderr, followed by $MORE... and then exit from the - configure script with non-zero status -``query_pkg_config $ARGS...`` - Run pkg-config passing it $ARGS. If QEMU is doing a static build, - then --static will be automatically added to $ARGS +Python virtual environments and the QEMU build system +----------------------------------------------------- +TBD Stage 2: Meson ============== -The Meson build system is currently used to describe the build -process for: +The Meson build system describes the build and install process for: 1) executables, which include: - - Tools - ``qemu-img``, ``qemu-nbd``, ``qga`` (guest agent), etc + - Tools - ``qemu-img``, ``qemu-nbd``, ``qemu-ga`` (guest agent), etc - System emulators - ``qemu-system-$ARCH`` @@ -126,7 +144,8 @@ process for: 2) documentation -3) ROMs, which can be either installed as binary blobs or compiled +3) ROMs, whether provided as binary blobs in the QEMU distributions + or cross compiled under the direction of the configure script 4) other data files, such as icons or desktop files @@ -164,9 +183,9 @@ Target-independent emulator sourcesets: This includes error handling infrastructure, standard data structures, platform portability wrapper functions, etc. - Target-independent code lives in the ``common_ss``, ``softmmu_ss`` and + Target-independent code lives in the ``common_ss``, ``system_ss`` and ``user_ss`` sourcesets. ``common_ss`` is linked into all emulators, - ``softmmu_ss`` only in system emulators, ``user_ss`` only in user-mode + ``system_ss`` only in system emulators, ``user_ss`` only in user-mode emulators. Target-independent sourcesets must exercise particular care when using @@ -178,11 +197,11 @@ Target-independent emulator sourcesets: symbol:: # Some targets have CONFIG_ACPI, some don't, so this is not enough - softmmu_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi.c'), + system_ss.add(when: 'CONFIG_ACPI', if_true: files('acpi.c'), if_false: files('acpi-stub.c')) # This is required as well: - softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('acpi-stub.c')) + system_ss.add(when: 'CONFIG_ALL', if_true: files('acpi-stub.c')) Target-dependent emulator sourcesets: In the target-dependent set lives CPU emulation, some device emulation and @@ -210,16 +229,16 @@ Target-dependent emulator sourcesets: for all emulators and for system emulators only. For example:: arm_ss = ss.source_set() - arm_softmmu_ss = ss.source_set() + arm_system_ss = ss.source_set() ... target_arch += {'arm': arm_ss} - target_softmmu_arch += {'arm': arm_softmmu_ss} + target_softmmu_arch += {'arm': arm_system_ss} Module sourcesets: There are two dictionaries for modules: ``modules`` is used for target-independent modules and ``target_modules`` is used for target-dependent modules. When modules are disabled the ``module`` - source sets are added to ``softmmu_ss`` and the ``target_modules`` + source sets are added to ``system_ss`` and the ``target_modules`` source sets are added to ``specific_ss``. Both dictionaries are nested. One dictionary is created per @@ -281,8 +300,7 @@ system/userspace emulation target Adding checks ------------- -New checks should be added to Meson. Compiler checks can be as simple as -the following:: +Compiler checks can be as simple as the following:: config_host_data.set('HAVE_BTRFS_H', cc.has_header('linux/btrfs.h')) @@ -311,8 +329,7 @@ dependency will be used:: sdl_image = not_found if not get_option('sdl_image').auto() or have_system sdl_image = dependency('SDL2_image', required: get_option('sdl_image'), - method: 'pkg-config', - static: enable_static) + method: 'pkg-config') endif This avoids warnings on static builds of user-mode emulators, for example. @@ -360,22 +377,30 @@ script, which may point to something other than the first python3 binary on the path. -Stage 3: makefiles -================== +Stage 3: Make +============= -The use of GNU make is required with the QEMU build system. +The next step in building QEMU is to invoke make. GNU Make is required +to build QEMU, and may be installed as ``gmake`` on some hosts. -The output of Meson is a build.ninja file, which is used with the Ninja -build system. QEMU uses a different approach, where Makefile rules are -synthesized from the build.ninja file. The main Makefile includes these -rules and wraps them so that e.g. submodules are built before QEMU. -The resulting build system is largely non-recursive in nature, in -contrast to common practices seen with automake. +The output of Meson is a ``build.ninja`` file, which is used with the +Ninja build tool. However, QEMU's build comprises other components than +just the emulators (namely firmware and the tests in ``tests/tcg``) which +need different cross compilers. The QEMU Makefile wraps both Ninja and +the smaller build systems for firmware and tests; it also takes care of +running ``configure`` again when the script changes. Apart from invoking +these sub-Makefiles, the resulting build is largely non-recursive. -Tests are also ran by the Makefile with the traditional ``make check`` -phony target, while benchmarks are run with ``make bench``. Meson test -suites such as ``unit`` can be ran with ``make check-unit`` too. It is also -possible to run tests defined in meson.build with ``meson test``. +Tests, whether defined in ``meson.build`` or not, are also ran by the +Makefile with the traditional ``make check`` phony target, while benchmarks +are run with ``make bench``. Meson test suites such as ``unit`` can be ran +with ``make check-unit``, and ``make check-tcg`` builds and runs "non-Meson" +tests for all targets. + +If desired, it is also possible to use ``ninja`` and ``meson test``, +respectively to build emulators and run tests defined in meson.build. +The main difference is that ``make`` needs the ``-jN`` flag in order to +enable parallel builds or tests. Useful make targets ------------------- @@ -387,6 +412,7 @@ Useful make targets Print the value of the variable VAR. Useful for debugging the build system. + Important files for the build system ==================================== @@ -400,8 +426,7 @@ number of dynamically created files listed later. ``Makefile`` The main entry point used when invoking make to build all the components of QEMU. The default 'all' target will naturally result in the build of - every component. Makefile takes care of recursively building submodules - directly via a non-recursive set of rules. + every component. ``*/meson.build`` The meson.build file in the root directory is the main entry point for the @@ -410,59 +435,92 @@ number of dynamically created files listed later. other meson.build files spread throughout the QEMU source tree. ``tests/Makefile.include`` - Rules for external test harnesses. These include the TCG tests, - ``qemu-iotests`` and the Avocado-based integration tests. + Rules for external test harnesses. These include the TCG tests + and the Avocado-based integration tests. ``tests/docker/Makefile.include`` - Rules for Docker tests. Like tests/Makefile, this file is included - directly by the top level Makefile, anything defined in this file will - influence the entire build system. + Rules for Docker tests. Like ``tests/Makefile.include``, this file is + included directly by the top level Makefile, anything defined in this + file will influence the entire build system. ``tests/vm/Makefile.include`` - Rules for VM-based tests. Like tests/Makefile, this file is included - directly by the top level Makefile, anything defined in this file will - influence the entire build system. + Rules for VM-based tests. Like ``tests/Makefile.include``, this file is + included directly by the top level Makefile, anything defined in this + file will influence the entire build system. Dynamically created files ------------------------- -The following files are generated dynamically by configure in order to -control the behaviour of the statically defined makefiles. This avoids -the need for QEMU makefiles to go through any pre-processing as seen -with autotools, where Makefile.am generates Makefile.in which generates -Makefile. +The following files are generated at run-time in order to control the +behaviour of the Makefiles. This avoids the need for QEMU makefiles to +go through any pre-processing as seen with autotools, where configure +generates ``Makefile`` from ``Makefile.in``. Built by configure: ``config-host.mak`` When configure has determined the characteristics of the build host it - will write a long list of variables to config-host.mak file. This - provides the various install directories, compiler / linker flags and a + will write them to this file for use in ``Makefile`` and to a smaller + extent ``meson.build``. These include the paths to various tools and a variety of ``CONFIG_*`` variables related to optionally enabled features. - This is imported by the top level Makefile and meson.build in order to - tailor the build output. - config-host.mak is also used as a dependency checking mechanism. If make + ``config-host.mak`` is also used as a dependency checking mechanism. If make sees that the modification timestamp on configure is newer than that on - config-host.mak, then configure will be re-run. + ``config-host.mak``, then configure will be re-run. + + The variables defined here apply to all QEMU + build outputs. + +``config-meson.cross`` + + A Meson "cross file" (or native file) used to communicate the paths to + the toolchain and other configuration options. + +``config.status`` - The variables defined here are those which are applicable to all QEMU - build outputs. Variables which are potentially different for each - emulator target are defined by the next file... + A small shell script that will invoke configure again with the same + environment variables that were set during the first run. It's used to + rerun configure after changes to the source code, but it can also be + inspected manually to check the contents of the environment. +``Makefile.prereqs`` + + A set of Makefile dependencies that order the build and execution of + firmware and tests after the container images and emulators that they + need. + +``pc-bios/*/config.mak``, ``tests/tcg/config-host.mak``, ``tests/tcg/*/config-target.mak`` + + Configuration variables used to build the firmware and TCG tests, + including paths to cross compilation toolchains. + +``pyvenv`` + + A Python virtual environment that is used for all Python code running + during the build. Using a virtual environment ensures that even code + that is run via ``sphinx-build``, ``meson`` etc. uses the same interpreter + and packages. Built by Meson: +``config-host.h`` + Used by C code to determine the properties of the build environment + and the set of enabled features for the entire build. + ``${TARGET-NAME}-config-devices.mak`` - TARGET-NAME is again the name of a system or userspace emulator. The - config-devices.mak file is automatically generated by make using the - scripts/make_device_config.sh program, feeding it the - default-configs/$TARGET-NAME file as input. - -``config-host.h``, ``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h`` - These files are used by source code to determine what features are - enabled. They are generated from the contents of the corresponding - ``*.mak`` files using Meson's ``configure_file()`` function. + TARGET-NAME is the name of a system emulator. The file is + generated by Meson using files under ``configs/devices`` as input. + +``${TARGET-NAME}-config-target.mak`` + TARGET-NAME is the name of a system or usermode emulator. The file is + generated by Meson using files under ``configs/targets`` as input. + +``$TARGET_NAME-config-target.h``, ``$TARGET_NAME-config-devices.h`` + Used by C code to determine the properties and enabled + features for each target. enabled. They are generated from + the contents of the corresponding ``*.mak`` files using Meson's + ``configure_file()`` function; each target can include them using + the ``CONFIG_TARGET`` and ``CONFIG_DEVICES`` macro respectively. ``build.ninja`` The build rules. diff --git a/docs/devel/decodetree.rst b/docs/devel/decodetree.rst index 49ea50c2a7..e3392aa705 100644 --- a/docs/devel/decodetree.rst +++ b/docs/devel/decodetree.rst @@ -23,22 +23,42 @@ Fields Syntax:: - field_def := '%' identifier ( unnamed_field )* ( !function=identifier )? + field_def := '%' identifier ( field )* ( !function=identifier )? + field := unnamed_field | named_field unnamed_field := number ':' ( 's' ) number + named_field := identifier ':' ( 's' ) number For *unnamed_field*, the first number is the least-significant bit position of the field and the second number is the length of the field. If the 's' is -present, the field is considered signed. If multiple ``unnamed_fields`` are -present, they are concatenated. In this way one can define disjoint fields. +present, the field is considered signed. + +A *named_field* refers to some other field in the instruction pattern +or format. Regardless of the length of the other field where it is +defined, it will be inserted into this field with the specified +signedness and bit width. + +Field definitions that involve loops (i.e. where a field is defined +directly or indirectly in terms of itself) are errors. + +A format can include fields that refer to named fields that are +defined in the instruction pattern(s) that use the format. +Conversely, an instruction pattern can include fields that refer to +named fields that are defined in the format it uses. However you +cannot currently do both at once (i.e. pattern P uses format F; F has +a field A that refers to a named field B that is defined in P, and P +has a field C that refers to a named field D that is defined in F). + +If multiple ``fields`` are present, they are concatenated. +In this way one can define disjoint fields. If ``!function`` is specified, the concatenated result is passed through the named function, taking and returning an integral value. -One may use ``!function`` with zero ``unnamed_fields``. This case is called +One may use ``!function`` with zero ``fields``. This case is called a *parameter*, and the named function is only passed the ``DisasContext`` and returns an integral value extracted from there. -A field with no ``unnamed_fields`` and no ``!function`` is in error. +A field with no ``fields`` and no ``!function`` is in error. Field examples: @@ -56,6 +76,9 @@ Field examples: | %shimm8 5:s8 13:1 | expand_shimm8(sextract(i, 5, 8) << 1 | | | !function=expand_shimm8 | extract(i, 13, 1)) | +---------------------------+---------------------------------------------+ +| %sz_imm 10:2 sz:3 | expand_sz_imm(extract(i, 10, 2) << 3 | | +| !function=expand_sz_imm | extract(a->sz, 0, 3)) | ++---------------------------+---------------------------------------------+ Argument Sets ============= diff --git a/docs/devel/index-api.rst b/docs/devel/index-api.rst index 60c0d7459d..7108821746 100644 --- a/docs/devel/index-api.rst +++ b/docs/devel/index-api.rst @@ -12,3 +12,4 @@ generated from in-code annotations to function prototypes. memory modules ui + zoned-storage diff --git a/docs/devel/kconfig.rst b/docs/devel/kconfig.rst index ac9453eba9..e3a544e463 100644 --- a/docs/devel/kconfig.rst +++ b/docs/devel/kconfig.rst @@ -282,9 +282,19 @@ want to change some lines in the first group, for example like this:: CONFIG_PCI_DEVICES=y #CONFIG_TEST_DEVICES=n -and/or pick a subset of the devices in those device groups. Right now -there is no single place that lists all the optional devices for -``CONFIG_PCI_DEVICES`` and ``CONFIG_TEST_DEVICES``. In the future, +and/or pick a subset of the devices in those device groups. Without +further modifications to ``configs/devices/``, a system emulator built +without default devices might not do much more than start an empty +machine, and even then only if ``--nodefaults`` is specified on the +command line. Starting a VM *without* ``--nodefaults`` is allowed to +fail, but should never abort. Failures in ``make check`` with +``--without-default-devices`` are considered bugs in the test code: +the tests should either use ``--nodefaults``, and should be skipped +if a necessary device is not present in the build. Such failures +should not be worked around with ``select`` directives. + +Right now there is no single place that lists all the optional devices +for ``CONFIG_PCI_DEVICES`` and ``CONFIG_TEST_DEVICES``. In the future, we expect that ``.mak`` files will be automatically generated, so that they will include all these symbols and some help text on what they do. diff --git a/docs/devel/loads-stores.rst b/docs/devel/loads-stores.rst index ad5dfe133e..d2cefc77a2 100644 --- a/docs/devel/loads-stores.rst +++ b/docs/devel/loads-stores.rst @@ -297,31 +297,20 @@ swap: ``translator_ld{sign}{size}_swap(env, ptr, swap)`` Regexes for git grep - ``\<translator_ld[us]\?[bwlq]\(_swap\)\?\>`` -``helper_*_{ld,st}*_mmu`` +``helper_{ld,st}*_mmu`` ~~~~~~~~~~~~~~~~~~~~~~~~~ These functions are intended primarily to be called by the code -generated by the TCG backend. They may also be called by target -CPU helper function code. Like the ``cpu_{ld,st}_mmuidx_ra`` functions -they perform accesses by guest virtual address, with a given ``mmuidx``. +generated by the TCG backend. Like the ``cpu_{ld,st}_mmu`` functions +they perform accesses by guest virtual address, with a given ``MemOpIdx``. -These functions specify an ``opindex`` parameter which encodes -(among other things) the mmu index to use for the access. This parameter -should be created by calling ``make_memop_idx()``. +They differ from ``cpu_{ld,st}_mmu`` in that they take the endianness +of the operation only from the MemOpIdx, and loads extend the return +value to the size of a host general register (``tcg_target_ulong``). -The ``retaddr`` parameter should be the result of GETPC() called directly -from the top level HELPER(foo) function (or 0 if no guest CPU state -unwinding is required). +load: ``helper_ld{sign}{size}_mmu(env, addr, opindex, retaddr)`` -**TODO** The names of these functions are a bit odd for historical -reasons because they were originally expected to be called only from -within generated code. We should rename them to bring them more in -line with the other memory access functions. The explicit endianness -is the only feature they have beyond ``*_mmuidx_ra``. - -load: ``helper_{endian}_ld{sign}{size}_mmu(env, addr, opindex, retaddr)`` - -store: ``helper_{endian}_st{size}_mmu(env, addr, val, opindex, retaddr)`` +store: ``helper_{size}_mmu(env, addr, val, opindex, retaddr)`` ``sign`` - (empty) : for 32 or 64 bit sizes @@ -334,14 +323,9 @@ store: ``helper_{endian}_st{size}_mmu(env, addr, val, opindex, retaddr)`` - ``l`` : 32 bits - ``q`` : 64 bits -``endian`` - - ``le`` : little endian - - ``be`` : big endian - - ``ret`` : target endianness - Regexes for git grep - - ``\<helper_\(le\|be\|ret\)_ld[us]\?[bwlq]_mmu\>`` - - ``\<helper_\(le\|be\|ret\)_st[bwlq]_mmu\>`` + - ``\<helper_ld[us]\?[bwlq]_mmu\>`` + - ``\<helper_st[bwlq]_mmu\>`` ``address_space_*`` ~~~~~~~~~~~~~~~~~~~ diff --git a/docs/devel/multiple-iothreads.txt b/docs/devel/multiple-iothreads.txt index 343120f2ef..a3e949f6b3 100644 --- a/docs/devel/multiple-iothreads.txt +++ b/docs/devel/multiple-iothreads.txt @@ -61,6 +61,7 @@ There are several old APIs that use the main loop AioContext: * LEGACY qemu_aio_set_event_notifier() - monitor an event notifier * LEGACY timer_new_ms() - create a timer * LEGACY qemu_bh_new() - create a BH + * LEGACY qemu_bh_new_guarded() - create a BH with a device re-entrancy guard * LEGACY qemu_aio_wait() - run an event loop iteration Since they implicitly work on the main loop they cannot be used in code that @@ -72,8 +73,14 @@ Instead, use the AioContext functions directly (see include/block/aio.h): * aio_set_event_notifier() - monitor an event notifier * aio_timer_new() - create a timer * aio_bh_new() - create a BH + * aio_bh_new_guarded() - create a BH with a device re-entrancy guard * aio_poll() - run an event loop iteration +The qemu_bh_new_guarded/aio_bh_new_guarded APIs accept a "MemReentrancyGuard" +argument, which is used to check for and prevent re-entrancy problems. For +BHs associated with devices, the reentrancy-guard is contained in the +corresponding DeviceState and named "mem_reentrancy_guard". + The AioContext can be obtained from the IOThread using iothread_get_aio_context() or for the main loop using qemu_get_aio_context(). Code that takes an AioContext argument works both in IOThreads or the main diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst index 23e7f2fb1c..7f78183cd4 100644 --- a/docs/devel/qapi-code-gen.rst +++ b/docs/devel/qapi-code-gen.rst @@ -545,7 +545,8 @@ Member 'allow-oob' declares whether the command supports out-of-band { 'command': 'migrate_recover', 'data': { 'uri': 'str' }, 'allow-oob': true } -See qmp-spec.txt for out-of-band execution syntax and semantics. +See the :doc:`/interop/qmp-spec` for out-of-band execution syntax +and semantics. Commands supporting out-of-band execution can still be executed in-band. @@ -805,9 +806,8 @@ gets its generated code guarded like this:: ... generated code ... #endif /* defined(HAVE_BAR) && defined(CONFIG_FOO) */ -Individual members of complex types, commands arguments, and -event-specific data can also be made conditional. This requires the -longhand form of MEMBER. +Individual members of complex types can also be made conditional. +This requires the longhand form of MEMBER. Example: a struct type with unconditional member 'foo' and conditional member 'bar' :: @@ -925,14 +925,17 @@ first character of the first line. The usual ****strong****, *\*emphasized\** and ````literal```` markup should be used. If you need a single literal ``*``, you will need to -backslash-escape it. As an extension beyond the usual rST syntax, you -can also use ``@foo`` to reference a name in the schema; this is rendered -the same way as ````foo````. +backslash-escape it. + +Use ``@foo`` to reference a name in the schema. This is an rST +extension. It is rendered the same way as ````foo````, but carries +additional meaning. Example:: ## # Some text foo with **bold** and *emphasis* + # # 1. with a list # 2. like that # @@ -945,6 +948,11 @@ Example:: # <- get that ## +For legibility, wrap text paragraphs so every line is at most 70 +characters long. + +Separate sentences with two spaces. + Definition documentation ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -961,57 +969,46 @@ commands and events), member (for structs and unions), branch (for alternates), or value (for enums), a description of each feature (if any), and finally optional tagged sections. -The description of an argument or feature 'name' starts with -'\@name:'. The description text can start on the line following the -'\@name:', in which case it must not be indented at all. It can also -start on the same line as the '\@name:'. In this case if it spans -multiple lines then second and subsequent lines must be indented to -line up with the first character of the first line of the -description:: - - # @argone: - # This is a two line description - # in the first style. - # - # @argtwo: This is a two line description - # in the second style. +Descriptions start with '\@name:'. The description text should be +indented like this:: -The number of spaces between the ':' and the text is not significant. + # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed + # do eiusmod tempor incididunt ut labore et dolore magna aliqua. -.. admonition:: FIXME +.. FIXME The parser accepts these things in almost any order. - The parser accepts these things in almost any order. - -.. admonition:: FIXME - - union branches should be described, too. +.. FIXME union branches should be described, too. Extensions added after the definition was first released carry a -'(since x.y.z)' comment. +"(since x.y.z)" comment. The feature descriptions must be preceded by a line "Features:", like this:: # Features: + # # @feature: Description text A tagged section starts with one of the following words: "Note:"/"Notes:", "Since:", "Example"/"Examples", "Returns:", "TODO:". The section ends with the start of a new section. -The text of a section can start on a new line, in -which case it must not be indented at all. It can also start -on the same line as the 'Note:', 'Returns:', etc tag. In this -case if it spans multiple lines then second and subsequent -lines must be indented to match the first, in the same way as -multiline argument descriptions. +The second and subsequent lines of sections other than +"Example"/"Examples" should be indented like this:: + + # Note: Ut enim ad minim veniam, quis nostrud exercitation ullamco + # laboris nisi ut aliquip ex ea commodo consequat. + # + # Duis aute irure dolor in reprehenderit in voluptate velit esse + # cillum dolore eu fugiat nulla pariatur. -A 'Since: x.y.z' tagged section lists the release that introduced the +A "Since: x.y.z" tagged section lists the release that introduced the definition. -An 'Example' or 'Examples' section is automatically rendered -entirely as literal fixed-width text. In other sections, -the text is formatted, and rST markup can be used. +An "Example" or "Examples" section is rendered entirely +as literal fixed-width text. "TODO" sections are not rendered at all +(they are for developers, not users of QMP). In other sections, the +text is formatted, and rST markup can be used. For example:: @@ -1021,7 +1018,7 @@ For example:: # Statistics of a virtual block device or a block backing device. # # @device: If the stats are for a virtual block device, the name - # corresponding to the virtual block device. + # corresponding to the virtual block device. # # @node-name: The node name of the device. (since 2.3) # @@ -1038,8 +1035,8 @@ For example:: # # Query the @BlockStats for all virtual block devices. # - # @query-nodes: If true, the command will query all the - # block nodes ... explain, explain ... (since 2.3) + # @query-nodes: If true, the command will query all the block nodes + # ... explain, explain ... (since 2.3) # # Returns: A list of @BlockStats for each virtual block devices. # @@ -1058,6 +1055,63 @@ For example:: 'returns': ['BlockStats'] } +Markup pitfalls +~~~~~~~~~~~~~~~ + +A blank line is required between list items and paragraphs. Without +it, the list may not be recognized, resulting in garbled output. Good +example:: + + # An event's state is modified if: + # + # - its name matches the @name pattern, and + # - if @vcpu is given, the event has the "vcpu" property. + +Without the blank line this would be a single paragraph. + +Indentation matters. Bad example:: + + # @none: None (no memory side cache in this proximity domain, + # or cache associativity unknown) + # (since 5.0) + +The last line's de-indent is wrong. The second and subsequent lines +need to line up with each other, like this:: + + # @none: None (no memory side cache in this proximity domain, + # or cache associativity unknown) + # (since 5.0) + +Section tags are case-sensitive and end with a colon. Good example:: + + # Since: 7.1 + +Bad examples (all ordinary paragraphs):: + + # since: 7.1 + + # Since 7.1 + + # Since : 7.1 + +Likewise, member descriptions require a colon. Good example:: + + # @interface-id: Interface ID + +Bad examples (all ordinary paragraphs):: + + # @interface-id Interface ID + + # @interface-id : Interface ID + +Undocumented members are not flagged, yet. Instead, the generated +documentation describes them as "Not documented". Think twice before +adding more undocumented members. + +When you change documentation comments, please check the generated +documentation comes out as intended! + + Client JSON Protocol introspection ================================== diff --git a/docs/devel/qom.rst b/docs/devel/qom.rst index 3e34b07c98..c9237950d0 100644 --- a/docs/devel/qom.rst +++ b/docs/devel/qom.rst @@ -1,3 +1,5 @@ +.. _qom: + =========================== The QEMU Object Model (QOM) =========================== diff --git a/docs/devel/style.rst b/docs/devel/style.rst index 68aa776930..aa5e083ff8 100644 --- a/docs/devel/style.rst +++ b/docs/devel/style.rst @@ -300,6 +300,20 @@ putting those into qemu/typedefs.h instead of including the header. Cyclic inclusion is forbidden. +Generative Includes +------------------- + +QEMU makes fairly extensive use of the macro pre-processor to +instantiate multiple similar functions. While such abuse of the macro +processor isn't discouraged it can make debugging and code navigation +harder. You should consider carefully if the same effect can be +achieved by making it easy for the compiler to constant fold or using +python scripting to generate grep friendly code. + +If you do use template header files they should be named with the +``.c.inc`` or ``.h.inc`` suffix to make it clear they are being +included for expansion. + C types ======= @@ -614,6 +628,97 @@ are still some caveats to beware of QEMU Specific Idioms ******************** +QEMU Object Model Declarations +============================== + +The QEMU Object Model (QOM) provides a framework for handling objects +in the base C language. The first declaration of a storage or class +structure should always be the parent and leave a visual space between +that declaration and the new code. It is also useful to separate +backing for properties (options driven by the user) and internal state +to make navigation easier. + +For a storage structure the first declaration should always be called +"parent_obj" and for a class structure the first member should always +be called "parent_class" as below: + +.. code-block:: c + + struct MyDeviceState { + DeviceState parent_obj; + + /* Properties */ + int prop_a; + char *prop_b; + /* Other stuff */ + int internal_state; + }; + + struct MyDeviceClass { + DeviceClass parent_class; + + void (*new_fn1)(void); + bool (*new_fn2)(CPUState *); + }; + +Note that there is no need to provide typedefs for QOM structures +since these are generated automatically by the QOM declaration macros. +See :ref:`qom` for more details. + +QEMU GUARD macros +================= + +QEMU provides a number of ``_GUARD`` macros intended to make the +handling of multiple exit paths easier. For example using +``QEMU_LOCK_GUARD`` to take a lock will ensure the lock is released on +exit from the function. + +.. code-block:: c + + static int my_critical_function(SomeState *s, void *data) + { + QEMU_LOCK_GUARD(&s->lock); + do_thing1(data); + if (check_state2(data)) { + return -1; + } + do_thing3(data); + return 0; + } + +will ensure s->lock is released however the function is exited. The +equivalent code without _GUARD macro makes us to carefully put +qemu_mutex_unlock() on all exit points: + +.. code-block:: c + + static int my_critical_function(SomeState *s, void *data) + { + qemu_mutex_lock(&s->lock); + do_thing1(data); + if (check_state2(data)) { + qemu_mutex_unlock(&s->lock); + return -1; + } + do_thing3(data); + qemu_mutex_unlock(&s->lock); + return 0; + } + +There are often ``WITH_`` forms of macros which more easily wrap +around a block inside a function. + +.. code-block:: c + + WITH_RCU_READ_LOCK_GUARD() { + QTAILQ_FOREACH_RCU(kid, &bus->children, sibling) { + err = do_the_thing(kid->child); + if (err < 0) { + return err; + } + } + } + Error handling and reporting ============================ diff --git a/docs/devel/tcg-ops.rst b/docs/devel/tcg-ops.rst index f3f451b77f..6a166c5665 100644 --- a/docs/devel/tcg-ops.rst +++ b/docs/devel/tcg-ops.rst @@ -672,19 +672,20 @@ QEMU specific operations | This operation is optional. If the TCG backend does not implement the goto_ptr opcode, emitting this op is equivalent to emitting exit_tb(0). - * - qemu_ld_i32/i64 *t0*, *t1*, *flags*, *memidx* + * - qemu_ld_i32/i64/i128 *t0*, *t1*, *flags*, *memidx* - qemu_st_i32/i64 *t0*, *t1*, *flags*, *memidx* + qemu_st_i32/i64/i128 *t0*, *t1*, *flags*, *memidx* qemu_st8_i32 *t0*, *t1*, *flags*, *memidx* - | Load data at the guest address *t1* into *t0*, or store data in *t0* at guest - address *t1*. The _i32/_i64 size applies to the size of the input/output + address *t1*. The _i32/_i64/_i128 size applies to the size of the input/output register *t0* only. The address *t1* is always sized according to the guest, and the width of the memory operation is controlled by *flags*. | | Both *t0* and *t1* may be split into little-endian ordered pairs of registers - if dealing with 64-bit quantities on a 32-bit host. + if dealing with 64-bit quantities on a 32-bit host, or 128-bit quantities on + a 64-bit host. | | The *memidx* selects the qemu tlb index to use (e.g. user or kernel access). The flags are the MemOp bits, selecting the sign, width, and endianness @@ -693,6 +694,8 @@ QEMU specific operations | For a 32-bit host, qemu_ld/st_i64 is guaranteed to only be used with a 64-bit memory access specified in *flags*. | + | For qemu_ld/st_i128, these are only supported for a 64-bit host. + | | For i386, qemu_st8_i32 is exactly like qemu_st_i32, except the size of the memory operation is known to be 8-bit. This allows the backend to provide a different set of register constraints. diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index 4071e72710..203facb417 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -479,6 +479,12 @@ first to contribute the mapping to the ``libvirt-ci`` project: contains the ``mappings.yml`` update. Then add the prerequisite and run ``make lcitool-refresh``. + * Please also trigger gitlab container generation pipelines on your change + for as many OS distros as practical to make sure that there are no + obvious breakages when adding the new pre-requisite. Please see + `CI <https://www.qemu.org/docs/master/devel/ci.html>`__ documentation + page on how to trigger gitlab CI pipelines on your change. + For enterprise distros that default to old, end-of-life versions of the Python runtime, QEMU uses a separate set of mappings that work with more recent versions. These can be found in ``tests/lcitool/mappings.yml``. diff --git a/docs/devel/zoned-storage.rst b/docs/devel/zoned-storage.rst new file mode 100644 index 0000000000..30296d3c85 --- /dev/null +++ b/docs/devel/zoned-storage.rst @@ -0,0 +1,62 @@ +============= +zoned-storage +============= + +Zoned Block Devices (ZBDs) divide the LBA space into block regions called zones +that are larger than the LBA size. They can only allow sequential writes, which +can reduce write amplification in SSDs, and potentially lead to higher +throughput and increased capacity. More details about ZBDs can be found at: + +https://zonedstorage.io/docs/introduction/zoned-storage + +1. Block layer APIs for zoned storage +------------------------------------- +QEMU block layer supports three zoned storage models: +- BLK_Z_HM: The host-managed zoned model only allows sequential writes access +to zones. It supports ZBD-specific I/O commands that can be used by a host to +manage the zones of a device. +- BLK_Z_HA: The host-aware zoned model allows random write operations in +zones, making it backward compatible with regular block devices. +- BLK_Z_NONE: The non-zoned model has no zones support. It includes both +regular and drive-managed ZBD devices. ZBD-specific I/O commands are not +supported. + +The block device information resides inside BlockDriverState. QEMU uses +BlockLimits struct(BlockDriverState::bl) that is continuously accessed by the +block layer while processing I/O requests. A BlockBackend has a root pointer to +a BlockDriverState graph(for example, raw format on top of file-posix). The +zoned storage information can be propagated from the leaf BlockDriverState all +the way up to the BlockBackend. If the zoned storage model in file-posix is +set to BLK_Z_HM, then block drivers will declare support for zoned host device. + +The block layer APIs support commands needed for zoned storage devices, +including report zones, four zone operations, and zone append. + +2. Emulating zoned storage controllers +-------------------------------------- +When the BlockBackend's BlockLimits model reports a zoned storage device, users +like the virtio-blk emulation or the qemu-io-cmds.c utility can use block layer +APIs for zoned storage emulation or testing. + +For example, to test zone_report on a null_blk device using qemu-io is:: + + $ path/to/qemu-io --image-opts -n driver=host_device,filename=/dev/nullb0 -c "zrp offset nr_zones" + +To expose the host's zoned block device through virtio-blk, the command line +can be (includes the -device parameter):: + + -blockdev node-name=drive0,driver=host_device,filename=/dev/nullb0,cache.direct=on \ + -device virtio-blk-pci,drive=drive0 + +Or only use the -drive parameter:: + + -driver driver=host_device,file=/dev/nullb0,if=virtio,cache.direct=on + +Additionally, QEMU has several ways of supporting zoned storage, including: +(1) Using virtio-scsi: --device scsi-block allows for the passing through of +SCSI ZBC devices, enabling the attachment of ZBC or ZAC HDDs to QEMU. +(2) PCI device pass-through: While NVMe ZNS emulation is available for testing +purposes, it cannot yet pass through a zoned device from the host. To pass on +the NVMe ZNS device to the guest, use VFIO PCI pass the entire NVMe PCI adapter +through to the guest. Likewise, an HDD HBA can be passed on to QEMU all HDDs +attached to the HBA. diff --git a/docs/interop/firmware.json b/docs/interop/firmware.json index 56814f02b3..cc8f869186 100644 --- a/docs/interop/firmware.json +++ b/docs/interop/firmware.json @@ -258,7 +258,7 @@ # # @mode: Describes how the firmware build handles code versus variable # storage. If not present, it must be treated as if it was -# configured with value ``split``. Since: 7.0.0 +# configured with value @split. Since: 7.0.0 # # @executable: Identifies the firmware executable. The @mode # indicates whether there will be an associated @@ -267,13 +267,13 @@ # -drive if=none,id=pflash0,readonly=on,file=@executable.@filename,format=@executable.@format # -machine pflash0=pflash0 # or equivalent -blockdev instead of -drive. When -# @mode is ``combined`` the executable must be +# @mode is @combined the executable must be # cloned before use and configured with readonly=off. # With QEMU versions older than 4.0, you have to use # -drive if=pflash,unit=0,readonly=on,file=@executable.@filename,format=@executable.@format # # @nvram-template: Identifies the NVRAM template compatible with -# @executable, when @mode is set to ``split``, +# @executable, when @mode is set to @split, # otherwise it should not be present. # Management software instantiates an # individual copy -- a specific NVRAM file -- from diff --git a/docs/interop/index.rst b/docs/interop/index.rst index 6351ff9ba6..ed65395bfb 100644 --- a/docs/interop/index.rst +++ b/docs/interop/index.rst @@ -15,6 +15,7 @@ are useful for making QEMU interoperate with other software. dbus-display live-block-operations pr-helper + qmp-spec qemu-ga qemu-ga-ref qemu-qmp-ref diff --git a/docs/interop/qcow2.txt b/docs/interop/qcow2.txt index f7dc304ff6..2c4618375a 100644 --- a/docs/interop/qcow2.txt +++ b/docs/interop/qcow2.txt @@ -214,14 +214,19 @@ version 2. type. If the incompatible bit "Compression type" is set: the field - must be present and non-zero (which means non-zlib + must be present and non-zero (which means non-deflate compression type). Otherwise, this field must not be present - or must be zero (which means zlib). + or must be zero (which means deflate). Available compression type values: - 0: zlib <https://www.zlib.net/> + 0: deflate <https://www.ietf.org/rfc/rfc1951.txt> 1: zstd <http://github.com/facebook/zstd> + The deflate compression type is called "zlib" + <https://www.zlib.net/> in QEMU. However, clusters with the + deflate compression type do not have zlib headers. + + 105 - 111: Padding, contents defined below. === Header padding === diff --git a/docs/interop/qmp-intro.txt b/docs/interop/qmp-intro.txt deleted file mode 100644 index 1c745a7af0..0000000000 --- a/docs/interop/qmp-intro.txt +++ /dev/null @@ -1,88 +0,0 @@ - QEMU Machine Protocol - ===================== - -Introduction ------------- - -The QEMU Machine Protocol (QMP) allows applications to operate a -QEMU instance. - -QMP is JSON[1] based and features the following: - -- Lightweight, text-based, easy to parse data format -- Asynchronous messages support (ie. events) -- Capabilities Negotiation - -For detailed information on QMP's usage, please, refer to the following files: - -o qmp-spec.txt QEMU Machine Protocol current specification -o qemu-qmp-ref.html QEMU QMP commands and events (auto-generated at build-time) - -[1] https://www.json.org - -Usage ------ - -You can use the -qmp option to enable QMP. For example, the following -makes QMP available on localhost port 4444: - -$ qemu [...] -qmp tcp:localhost:4444,server=on,wait=off - -However, for more flexibility and to make use of more options, the -mon -command-line option should be used. For instance, the following example -creates one HMP instance (human monitor) on stdio and one QMP instance -on localhost port 4444: - -$ qemu [...] -chardev stdio,id=mon0 -mon chardev=mon0,mode=readline \ - -chardev socket,id=mon1,host=localhost,port=4444,server=on,wait=off \ - -mon chardev=mon1,mode=control,pretty=on - -Please, refer to QEMU's manpage for more information. - -Simple Testing --------------- - -To manually test QMP one can connect with telnet and issue commands by hand: - -$ telnet localhost 4444 -Trying 127.0.0.1... -Connected to localhost. -Escape character is '^]'. -{ - "QMP": { - "version": { - "qemu": { - "micro": 0, - "minor": 0, - "major": 3 - }, - "package": "v3.0.0" - }, - "capabilities": [ - "oob" - ] - } -} - -{ "execute": "qmp_capabilities" } -{ - "return": { - } -} - -{ "execute": "query-status" } -{ - "return": { - "status": "prelaunch", - "singlestep": false, - "running": false - } -} - -Please refer to docs/interop/qemu-qmp-ref.* for a complete command -reference, generated from qapi/qapi-schema.json. - -QMP wiki page -------------- - -https://wiki.qemu.org/QMP diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.rst index b0e8351d5b..563344160e 100644 --- a/docs/interop/qmp-spec.txt +++ b/docs/interop/qmp-spec.rst @@ -1,24 +1,26 @@ - QEMU Machine Protocol Specification +.. + Copyright (C) 2009-2016 Red Hat, Inc. -0. About This Document -====================== - -Copyright (C) 2009-2016 Red Hat, Inc. + This work is licensed under the terms of the GNU GPL, version 2 or + later. See the COPYING file in the top-level directory. -This work is licensed under the terms of the GNU GPL, version 2 or -later. See the COPYING file in the top-level directory. -1. Introduction -=============== +=================================== +QEMU Machine Protocol Specification +=================================== -This document specifies the QEMU Machine Protocol (QMP), a JSON-based +The QEMU Machine Protocol (QMP) is a JSON-based protocol which is available for applications to operate QEMU at the machine-level. It is also in use by the QEMU Guest Agent (QGA), which is available for host applications to interact with the guest -operating system. +operating system. This page specifies the general format of +the protocol; details of the commands and data structures can +be found in the :doc:`qemu-qmp-ref` and the :doc:`qemu-ga-ref`. -2. Protocol Specification -========================= +.. contents:: + +Protocol Specification +====================== This section details the protocol format. For the purpose of this document, "Server" is either QEMU or the QEMU Guest Agent, and @@ -30,9 +32,7 @@ following format: json-DATA-STRUCTURE-NAME Where DATA-STRUCTURE-NAME is any valid JSON data structure, as defined -by the JSON standard: - -http://www.ietf.org/rfc/rfc8259.txt +by the `JSON standard <http://www.ietf.org/rfc/rfc8259.txt>`_. The server expects its input to be encoded in UTF-8, and sends its output encoded in ASCII. @@ -45,83 +45,89 @@ important unless specifically documented otherwise. Repeating a key within a json-object gives unpredictable results. Also for convenience, the server will accept an extension of -'single-quoted' strings in place of the usual "double-quoted" +``'single-quoted'`` strings in place of the usual ``"double-quoted"`` json-string, and both input forms of strings understand an additional -escape sequence of "\'" for a single quote. The server will only use +escape sequence of ``\'`` for a single quote. The server will only use double quoting on output. -2.1 General Definitions ------------------------ +General Definitions +------------------- -2.1.1 All interactions transmitted by the Server are json-objects, always - terminating with CRLF +All interactions transmitted by the Server are json-objects, always +terminating with CRLF. -2.1.2 All json-objects members are mandatory when not specified otherwise +All json-objects members are mandatory when not specified otherwise. -2.2 Server Greeting -------------------- +Server Greeting +--------------- Right when connected the Server will issue a greeting message, which signals that the connection has been successfully established and that the Server is ready for capabilities negotiation (for more information refer to section -'4. Capabilities Negotiation'). +`Capabilities Negotiation`_). The greeting message format is: -{ "QMP": { "version": json-object, "capabilities": json-array } } +:: + + { "QMP": { "version": json-object, "capabilities": json-array } } - Where, +Where: -- The "version" member contains the Server's version information (the format - is the same of the query-version command) -- The "capabilities" member specify the availability of features beyond the +- The ``version`` member contains the Server's version information (the format + is the same as for the query-version command). +- The ``capabilities`` member specifies the availability of features beyond the baseline specification; the order of elements in this array has no particular significance. -2.2.1 Capabilities ------------------- +Capabilities +------------ Currently supported capabilities are: -- "oob": the QMP server supports "out-of-band" (OOB) command - execution, as described in section "2.3.1 Out-of-band execution". +``oob`` + the QMP server supports "out-of-band" (OOB) command + execution, as described in section `Out-of-band execution`_. -2.3 Issuing Commands --------------------- +Issuing Commands +---------------- The format for command execution is: -{ "execute": json-string, "arguments": json-object, "id": json-value } +:: + + { "execute": json-string, "arguments": json-object, "id": json-value } or -{ "exec-oob": json-string, "arguments": json-object, "id": json-value } +:: - Where, + { "exec-oob": json-string, "arguments": json-object, "id": json-value } -- The "execute" or "exec-oob" member identifies the command to be +Where: + +- The ``execute`` or ``exec-oob`` member identifies the command to be executed by the server. The latter requests out-of-band execution. -- The "arguments" member is used to pass any arguments required for the +- The ``arguments`` member is used to pass any arguments required for the execution of the command, it is optional when no arguments are required. Each command documents what contents will be considered - valid when handling the json-argument -- The "id" member is a transaction identification associated with the + valid when handling the json-argument. +- The ``id`` member is a transaction identification associated with the command execution, it is optional and will be part of the response - if provided. The "id" member can be any json-value. A json-number + if provided. The ``id`` member can be any json-value. A json-number incremented for each successive command works fine. -The actual commands are documented in the QEMU QMP reference manual -docs/interop/qemu-qmp-ref.{7,html,info,pdf,txt}. +The actual commands are documented in the :doc:`qemu-qmp-ref`. -2.3.1 Out-of-band execution ---------------------------- +Out-of-band execution +--------------------- The server normally reads, executes and responds to one command after the other. The client therefore receives command responses in issue order. -With out-of-band execution enabled via capability negotiation (section -4.), the server reads and queues commands as they arrive. It executes +With out-of-band execution enabled via `capabilities negotiation`_, +the server reads and queues commands as they arrive. It executes commands from the queue one after the other. Commands executed out-of-band jump the queue: the command get executed right away, possibly overtaking prior in-band commands. The client may therefore @@ -129,8 +135,8 @@ receive such a command's response before responses from prior in-band commands. To be able to match responses back to their commands, the client needs -to pass "id" with out-of-band commands. Passing it with all commands -is recommended for clients that accept capability "oob". +to pass ``id`` with out-of-band commands. Passing it with all commands +is recommended for clients that accept capability ``oob``. If the client sends in-band commands faster than the server can execute them, the server will stop reading requests until the request @@ -140,57 +146,61 @@ To ensure commands to be executed out-of-band get read and executed, the client should have at most eight in-band commands in flight. Only a few commands support out-of-band execution. The ones that do -have "allow-oob": true in output of query-qmp-schema. +have ``"allow-oob": true`` in the output of ``query-qmp-schema``. -2.4 Commands Responses ----------------------- +Commands Responses +------------------ There are two possible responses which the Server will issue as the result of a command execution: success or error. -As long as the commands were issued with a proper "id" field, then the -same "id" field will be attached in the corresponding response message +As long as the commands were issued with a proper ``id`` field, then the +same ``id`` field will be attached in the corresponding response message so that requests and responses can match. Clients should drop all the -responses that have an unknown "id" field. +responses that have an unknown ``id`` field. -2.4.1 success -------------- +Success +------- The format of a success response is: -{ "return": json-value, "id": json-value } +:: + + { "return": json-value, "id": json-value } - Where, +Where: -- The "return" member contains the data returned by the command, which +- The ``return`` member contains the data returned by the command, which is defined on a per-command basis (usually a json-object or json-array of json-objects, but sometimes a json-number, json-string, or json-array of json-strings); it is an empty json-object if the - command does not return data -- The "id" member contains the transaction identification associated - with the command execution if issued by the Client + command does not return data. +- The ``id`` member contains the transaction identification associated + with the command execution if issued by the Client. -2.4.2 error ------------ +Error +----- The format of an error response is: -{ "error": { "class": json-string, "desc": json-string }, "id": json-value } +:: - Where, + { "error": { "class": json-string, "desc": json-string }, "id": json-value } -- The "class" member contains the error class name (eg. "GenericError") -- The "desc" member is a human-readable error message. Clients should +Where: + +- The ``class`` member contains the error class name (eg. ``"GenericError"``). +- The ``desc`` member is a human-readable error message. Clients should not attempt to parse this message. -- The "id" member contains the transaction identification associated with - the command execution if issued by the Client +- The ``id`` member contains the transaction identification associated with + the command execution if issued by the Client. -NOTE: Some errors can occur before the Server is able to read the "id" member, -in these cases the "id" member will not be part of the error response, even +NOTE: Some errors can occur before the Server is able to read the ``id`` member; +in these cases the ``id`` member will not be part of the error response, even if provided by the client. -2.5 Asynchronous events ------------------------ +Asynchronous events +------------------- As a result of state changes, the Server may send messages unilaterally to the Client at any time, when not in the middle of any other @@ -198,44 +208,45 @@ response. They are called "asynchronous events". The format of asynchronous events is: -{ "event": json-string, "data": json-object, - "timestamp": { "seconds": json-number, "microseconds": json-number } } +:: - Where, + { "event": json-string, "data": json-object, + "timestamp": { "seconds": json-number, "microseconds": json-number } } -- The "event" member contains the event's name -- The "data" member contains event specific data, which is defined in a - per-event basis, it is optional -- The "timestamp" member contains the exact time of when the event +Where: + +- The ``event`` member contains the event's name. +- The ``data`` member contains event specific data, which is defined in a + per-event basis. It is optional. +- The ``timestamp`` member contains the exact time of when the event occurred in the Server. It is a fixed json-object with time in seconds and microseconds relative to the Unix Epoch (1 Jan 1970); if there is a failure to retrieve host time, both members of the timestamp will be set to -1. -The actual asynchronous events are documented in the QEMU QMP -reference manual docs/interop/qemu-qmp-ref.{7,html,info,pdf,txt}. +The actual asynchronous events are documented in the :doc:`qemu-qmp-ref`. Some events are rate-limited to at most one per second. If additional "similar" events arrive within one second, all but the last one are dropped, and the last one is delayed. "Similar" normally means same event type. -2.6 Forcing the JSON parser into known-good state -------------------------------------------------- +Forcing the JSON parser into known-good state +--------------------------------------------- Incomplete or invalid input can leave the server's JSON parser in a state where it can't parse additional commands. To get it back into known-good state, the client should provoke a lexical error. The cleanest way to do that is sending an ASCII control character -other than '\t' (horizontal tab), '\r' (carriage return), or '\n' (new -line). +other than ``\t`` (horizontal tab), ``\r`` (carriage return), or +``\n`` (new line). Sadly, older versions of QEMU can fail to flag this as an error. If a client needs to deal with them, it should send a 0xFF byte. -2.7 QGA Synchronization ------------------------ +QGA Synchronization +------------------- When a client connects to QGA over a transport lacking proper connection semantics such as virtio-serial, QGA may have read partial @@ -243,86 +254,106 @@ input from a previous client. The client needs to force QGA's parser into known-good state using the previous section's technique. Moreover, the client may receive output a previous client didn't read. To help with skipping that output, QGA provides the -'guest-sync-delimited' command. Refer to its documentation for +``guest-sync-delimited`` command. Refer to its documentation for details. -3. QMP Examples -=============== +QMP Examples +============ This section provides some examples of real QMP usage, in all of them -"C" stands for "Client" and "S" stands for "Server". +``->`` marks text sent by the Client and ``<-`` marks replies by the Server. -3.1 Server greeting -------------------- +.. admonition:: Example -S: { "QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 3}, - "package": "v3.0.0"}, "capabilities": ["oob"] } } + Server greeting -3.2 Capabilities negotiation ----------------------------- + .. code-block:: QMP -C: { "execute": "qmp_capabilities", "arguments": { "enable": ["oob"] } } -S: { "return": {}} + <- { "QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 3}, + "package": "v3.0.0"}, "capabilities": ["oob"] } } -3.3 Simple 'stop' execution ---------------------------- +.. admonition:: Example -C: { "execute": "stop" } -S: { "return": {} } + Capabilities negotiation -3.4 KVM information -------------------- + .. code-block:: QMP -C: { "execute": "query-kvm", "id": "example" } -S: { "return": { "enabled": true, "present": true }, "id": "example"} + -> { "execute": "qmp_capabilities", "arguments": { "enable": ["oob"] } } + <- { "return": {}} -3.5 Parsing error ------------------- +.. admonition:: Example -C: { "execute": } -S: { "error": { "class": "GenericError", "desc": "Invalid JSON syntax" } } + Simple 'stop' execution -3.6 Powerdown event -------------------- + .. code-block:: QMP -S: { "timestamp": { "seconds": 1258551470, "microseconds": 802384 }, - "event": "POWERDOWN" } + -> { "execute": "stop" } + <- { "return": {} } -3.7 Out-of-band execution -------------------------- +.. admonition:: Example -C: { "exec-oob": "migrate-pause", "id": 42 } -S: { "id": 42, - "error": { "class": "GenericError", - "desc": "migrate-pause is currently only supported during postcopy-active state" } } + KVM information + .. code-block:: QMP -4. Capabilities Negotiation -=========================== + -> { "execute": "query-kvm", "id": "example" } + <- { "return": { "enabled": true, "present": true }, "id": "example"} + +.. admonition:: Example + + Parsing error + + .. code-block:: QMP + + -> { "execute": } + <- { "error": { "class": "GenericError", "desc": "JSON parse error, expecting value" } } + +.. admonition:: Example + + Powerdown event + + .. code-block:: QMP + + <- { "timestamp": { "seconds": 1258551470, "microseconds": 802384 }, + "event": "POWERDOWN" } + +.. admonition:: Example + + Out-of-band execution + + .. code-block:: QMP + + -> { "exec-oob": "migrate-pause", "id": 42 } + <- { "id": 42, + "error": { "class": "GenericError", + "desc": "migrate-pause is currently only supported during postcopy-active state" } } + + +Capabilities Negotiation +======================== When a Client successfully establishes a connection, the Server is in Capabilities Negotiation mode. -In this mode only the qmp_capabilities command is allowed to run, all -other commands will return the CommandNotFound error. Asynchronous +In this mode only the ``qmp_capabilities`` command is allowed to run; all +other commands will return the ``CommandNotFound`` error. Asynchronous messages are not delivered either. -Clients should use the qmp_capabilities command to enable capabilities -advertised in the Server's greeting (section '2.2 Server Greeting') they -support. +Clients should use the ``qmp_capabilities`` command to enable capabilities +advertised in the `Server Greeting`_ which they support. -When the qmp_capabilities command is issued, and if it does not return an -error, the Server enters in Command mode where capabilities changes take -effect, all commands (except qmp_capabilities) are allowed and asynchronous +When the ``qmp_capabilities`` command is issued, and if it does not return an +error, the Server enters Command mode where capabilities changes take +effect, all commands (except ``qmp_capabilities``) are allowed and asynchronous messages are delivered. -5 Compatibility Considerations -============================== +Compatibility Considerations +============================ All protocol changes or new features which modify the protocol format in an incompatible way are disabled by default and will be advertised by the -capabilities array (section '2.2 Server Greeting'). Thus, Clients can check +capabilities array (in the `Server Greeting`_). Thus, Clients can check that array and enable the capabilities they support. The QMP Server performs a type check on the arguments to a command. It @@ -337,12 +368,12 @@ However, Clients must not assume any particular: - Length of json-arrays - Size of json-objects; in particular, future versions of QEMU may add - new keys and Clients should be able to ignore them. + new keys and Clients should be able to ignore them - Order of json-object members or json-array elements - Amount of errors generated by a command, that is, new errors can be added to any existing command in newer versions of the Server -Any command or member name beginning with "x-" is deemed experimental, +Any command or member name beginning with ``x-`` is deemed experimental, and may be withdrawn or changed in an incompatible manner in a future release. @@ -350,8 +381,8 @@ Of course, the Server does guarantee to send valid JSON. But apart from this, a Client should be "conservative in what they send, and liberal in what they accept". -6. Downstream extension of QMP -============================== +Downstream extension of QMP +=========================== We recommend that downstream consumers of QEMU do *not* modify QMP. Management tools should be able to support both upstream and downstream @@ -363,23 +394,25 @@ avoid modifying QMP. Both upstream and downstream need to take care to preserve long-term compatibility and interoperability. To help with that, QMP reserves JSON object member names beginning with -'__' (double underscore) for downstream use ("downstream names"). This +``__`` (double underscore) for downstream use ("downstream names"). This means upstream will never use any downstream names for its commands, arguments, errors, asynchronous events, and so forth. -Any new names downstream wishes to add must begin with '__'. To +Any new names downstream wishes to add must begin with ``__``. To ensure compatibility with other downstreams, it is strongly -recommended that you prefix your downstream names with '__RFQDN_' where +recommended that you prefix your downstream names with ``__RFQDN_`` where RFQDN is a valid, reverse fully qualified domain name which you control. For example, a qemu-kvm specific monitor command would be: +:: + (qemu) __org.linux-kvm_enable_irqchip -Downstream must not change the server greeting (section 2.2) other than +Downstream must not change the `server greeting`_ other than to offer additional capabilities. But see below for why even that is discouraged. -Section '5 Compatibility Considerations' applies to downstream as well +The section `Compatibility Considerations`_ applies to downstream as well as to upstream, obviously. It follows that downstream must behave exactly like upstream for any input not containing members with downstream names ("downstream members"), except it may add members diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst index 8a5924ea75..5a070adbc1 100644 --- a/docs/interop/vhost-user.rst +++ b/docs/interop/vhost-user.rst @@ -130,18 +130,8 @@ A vring address description Note that a ring address is an IOVA if ``VIRTIO_F_IOMMU_PLATFORM`` has been negotiated. Otherwise it is a user address. -Memory regions description -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -+-------------+---------+---------+-----+---------+ -| num regions | padding | region0 | ... | region7 | -+-------------+---------+---------+-----+---------+ - -:num regions: a 32-bit number of regions - -:padding: 32-bit - -A region is: +Memory region description +^^^^^^^^^^^^^^^^^^^^^^^^^ +---------------+------+--------------+-------------+ | guest address | size | user address | mmap offset | @@ -155,22 +145,49 @@ A region is: :mmap offset: 64-bit offset where region starts in the mapped memory +When the ``VHOST_USER_PROTOCOL_F_XEN_MMAP`` protocol feature has been +successfully negotiated, the memory region description contains two extra +fields at the end. + ++---------------+------+--------------+-------------+----------------+-------+ +| guest address | size | user address | mmap offset | xen mmap flags | domid | ++---------------+------+--------------+-------------+----------------+-------+ + +:xen mmap flags: 32-bit bit field + +- Bit 0 is set for Xen foreign memory mapping. +- Bit 1 is set for Xen grant memory mapping. +- Bit 8 is set if the memory region can not be mapped in advance, and memory + areas within this region must be mapped / unmapped only when required by the + back-end. The back-end shouldn't try to map the entire region at once, as the + front-end may not allow it. The back-end should rather map only the required + amount of memory at once and unmap it after it is used. + +:domid: a 32-bit Xen hypervisor specific domain id. + Single memory region description ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -+---------+---------------+------+--------------+-------------+ -| padding | guest address | size | user address | mmap offset | -+---------+---------------+------+--------------+-------------+ ++---------+--------+ +| padding | region | ++---------+--------+ :padding: 64-bit -:guest address: a 64-bit guest address of the region +A region is represented by Memory region description. -:size: a 64-bit size +Multiple Memory regions description +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -:user address: a 64-bit user address ++-------------+---------+---------+-----+---------+ +| num regions | padding | region0 | ... | region7 | ++-------------+---------+---------+-----+---------+ -:mmap offset: 64-bit offset where region starts in the mapped memory +:num regions: a 32-bit number of regions + +:padding: 32-bit + +A region is represented by Memory region description. Log description ^^^^^^^^^^^^^^^ @@ -867,6 +884,7 @@ Protocol features #define VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS 14 #define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS 15 #define VHOST_USER_PROTOCOL_F_STATUS 16 + #define VHOST_USER_PROTOCOL_F_XEN_MMAP 17 Front-end message types ----------------------- @@ -952,8 +970,8 @@ Front-end message types ``VHOST_USER_SET_MEM_TABLE`` :id: 5 :equivalent ioctl: ``VHOST_SET_MEM_TABLE`` - :request payload: memory regions description - :reply payload: (postcopy only) memory regions description + :request payload: multiple memory regions description + :reply payload: (postcopy only) multiple memory regions description Sets the memory map regions on the back-end so it can translate the vring addresses. In the ancillary data there is an array of file diff --git a/docs/meson.build b/docs/meson.build index f220800e3e..9040f860ae 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -1,4 +1,4 @@ -sphinx_build = find_program(get_option('sphinx_build'), +sphinx_build = find_program(fs.parent(python.full_path()) / 'sphinx-build', required: get_option('docs')) # Check if tools are available to build documentation. @@ -10,6 +10,18 @@ if sphinx_build.found() SPHINX_ARGS += [ '-W', '-Dkerneldoc_werror=1' ] endif + sphinx_version = run_command(SPHINX_ARGS + ['--version'], + check: true).stdout().split()[1] + if sphinx_version.version_compare('>=1.7.0') + SPHINX_ARGS += ['-j', 'auto'] + else + nproc = find_program('nproc') + if nproc.found() + jobs = run_command(nproc, check: true).stdout() + SPHINX_ARGS += ['-j', jobs] + endif + endif + # This is a bit awkward but works: create a trivial document and # try to run it with our configuration file (which enforces a # version requirement). This will fail if sphinx-build is too old. diff --git a/docs/pcie_sriov.txt b/docs/pcie_sriov.txt index 11158dbf88..7eff7f2703 100644 --- a/docs/pcie_sriov.txt +++ b/docs/pcie_sriov.txt @@ -9,10 +9,7 @@ virtual functions (VFs) for the main purpose of eliminating software overhead in I/O from virtual machines. QEMU now implements the basic common functionality to enable an emulated device -to support SR/IOV. Yet no fully implemented devices exists in QEMU, but a -proof-of-concept hack of the Intel igb can be found here: - -git://github.com/knuto/qemu.git sriov_patches_v5 +to support SR/IOV. Implementation ============== diff --git a/docs/specs/index.rst b/docs/specs/index.rst index a58d9311cb..e58be38c41 100644 --- a/docs/specs/index.rst +++ b/docs/specs/index.rst @@ -8,6 +8,9 @@ guest hardware that is specific to QEMU. .. toctree:: :maxdepth: 2 + pci-ids + pci-serial + pci-testdev ppc-xive ppc-spapr-xive ppc-spapr-numa diff --git a/docs/specs/pci-ids.rst b/docs/specs/pci-ids.rst new file mode 100644 index 0000000000..e302bea484 --- /dev/null +++ b/docs/specs/pci-ids.rst @@ -0,0 +1,98 @@ +================ +PCI IDs for QEMU +================ + +Red Hat, Inc. donates a part of its device ID range to QEMU, to be used for +virtual devices. The vendor IDs are 1af4 (formerly Qumranet ID) and 1b36. + +Contact Gerd Hoffmann <kraxel@redhat.com> to get a device ID assigned +for your devices. + +1af4 vendor ID +-------------- + +The 1000 -> 10ff device ID range is used as follows for virtio-pci devices. +Note that this allocation is separate from the virtio device IDs, which are +maintained as part of the virtio specification. + +1af4:1000 + network device (legacy) +1af4:1001 + block device (legacy) +1af4:1002 + balloon device (legacy) +1af4:1003 + console device (legacy) +1af4:1004 + SCSI host bus adapter device (legacy) +1af4:1005 + entropy generator device (legacy) +1af4:1009 + 9p filesystem device (legacy) +1af4:1012 + vsock device (bug compatibility) + +1af4:1040 to 1af4:10ef + ID range for modern virtio devices. The PCI device + ID is calculated from the virtio device ID by adding the + 0x1040 offset. The virtio IDs are defined in the virtio + specification. The Linux kernel has a header file with + defines for all virtio IDs (``linux/virtio_ids.h``); QEMU has a + copy in ``include/standard-headers/``. + +1af4:10f0 to 1a4f:10ff + Available for experimental usage without registration. Must get + official ID when the code leaves the test lab (i.e. when seeking + upstream merge or shipping a distro/product) to avoid conflicts. + +1af4:1100 + Used as PCI Subsystem ID for existing hardware devices emulated + by QEMU. + +1af4:1110 + ivshmem device (shared memory, ``docs/specs/ivshmem-spec.txt``) + +All other device IDs are reserved. + +1b36 vendor ID +-------------- + +The 0000 -> 00ff device ID range is used as follows for QEMU-specific +PCI devices (other than virtio): + +1b36:0001 + PCI-PCI bridge +1b36:0002 + PCI serial port (16550A) adapter (:doc:`pci-serial`) +1b36:0003 + PCI Dual-port 16550A adapter (:doc:`pci-serial`) +1b36:0004 + PCI Quad-port 16550A adapter (:doc:`pci-serial`) +1b36:0005 + PCI test device (:doc:`pci-testdev`) +1b36:0006 + PCI Rocker Ethernet switch device +1b36:0007 + PCI SD Card Host Controller Interface (SDHCI) +1b36:0008 + PCIe host bridge +1b36:0009 + PCI Expander Bridge (-device pxb) +1b36:000a + PCI-PCI bridge (multiseat) +1b36:000b + PCIe Expander Bridge (-device pxb-pcie) +1b36:000d + PCI xhci usb host adapter +1b36:000f + mdpy (mdev sample device), ``linux/samples/vfio-mdev/mdpy.c`` +1b36:0010 + PCIe NVMe device (``-device nvme``) +1b36:0011 + PCI PVPanic device (``-device pvpanic-pci``) +1b36:0012 + PCI ACPI ERST device (``-device acpi-erst``) + +All these devices are documented in :doc:`index`. + +The 0100 device ID is used for the QXL video card device. diff --git a/docs/specs/pci-ids.txt b/docs/specs/pci-ids.txt deleted file mode 100644 index e463c4cb3a..0000000000 --- a/docs/specs/pci-ids.txt +++ /dev/null @@ -1,70 +0,0 @@ - -PCI IDs for qemu -================ - -Red Hat, Inc. donates a part of its device ID range to qemu, to be used for -virtual devices. The vendor IDs are 1af4 (formerly Qumranet ID) and 1b36. - -Contact Gerd Hoffmann <kraxel@redhat.com> to get a device ID assigned -for your devices. - -1af4 vendor ID --------------- - -The 1000 -> 10ff device ID range is used as follows for virtio-pci devices. -Note that this allocation separate from the virtio device IDs, which are -maintained as part of the virtio specification. - -1af4:1000 network device (legacy) -1af4:1001 block device (legacy) -1af4:1002 balloon device (legacy) -1af4:1003 console device (legacy) -1af4:1004 SCSI host bus adapter device (legacy) -1af4:1005 entropy generator device (legacy) -1af4:1009 9p filesystem device (legacy) -1af4:1012 vsock device (bug compatibility) - -1af4:1040 Start of ID range for modern virtio devices. The PCI device - to ID is calculated from the virtio device ID by adding the -1af4:10ef 0x1040 offset. The virtio IDs are defined in the virtio - specification. The Linux kernel has a header file with - defines for all virtio IDs (linux/virtio_ids.h), qemu has a - copy in include/standard-headers/. - -1af4:10f0 Available for experimental usage without registration. Must get - to official ID when the code leaves the test lab (i.e. when seeking -1af4:10ff upstream merge or shipping a distro/product) to avoid conflicts. - -1af4:1100 Used as PCI Subsystem ID for existing hardware devices emulated - by qemu. - -1af4:1110 ivshmem device (shared memory, docs/specs/ivshmem-spec.txt) - -All other device IDs are reserved. - -1b36 vendor ID --------------- - -The 0000 -> 00ff device ID range is used as follows for QEMU-specific -PCI devices (other than virtio): - -1b36:0001 PCI-PCI bridge -1b36:0002 PCI serial port (16550A) adapter (docs/specs/pci-serial.txt) -1b36:0003 PCI Dual-port 16550A adapter (docs/specs/pci-serial.txt) -1b36:0004 PCI Quad-port 16550A adapter (docs/specs/pci-serial.txt) -1b36:0005 PCI test device (docs/specs/pci-testdev.txt) -1b36:0006 PCI Rocker Ethernet switch device -1b36:0007 PCI SD Card Host Controller Interface (SDHCI) -1b36:0008 PCIe host bridge -1b36:0009 PCI Expander Bridge (-device pxb) -1b36:000a PCI-PCI bridge (multiseat) -1b36:000b PCIe Expander Bridge (-device pxb-pcie) -1b36:000d PCI xhci usb host adapter -1b36:000f mdpy (mdev sample device), linux/samples/vfio-mdev/mdpy.c -1b36:0010 PCIe NVMe device (-device nvme) -1b36:0011 PCI PVPanic device (-device pvpanic-pci) -1b36:0012 PCI ACPI ERST device (-device acpi-erst) - -All these devices are documented in docs/specs. - -The 0100 device ID is used for the QXL video card device. diff --git a/docs/specs/pci-serial.rst b/docs/specs/pci-serial.rst new file mode 100644 index 0000000000..8d916a3669 --- /dev/null +++ b/docs/specs/pci-serial.rst @@ -0,0 +1,37 @@ +======================= +QEMU PCI serial devices +======================= + +QEMU implements some PCI serial devices which are simple PCI +wrappers around one or more 16550 UARTs. + +There is one single-port variant and two multiport-variants. Linux +guests work out-of-the box with all cards. There is a Windows inf file +(``docs/qemupciserial.inf``) to set up the cards in Windows guests. + + +Single-port card +---------------- + +Name: + ``pci-serial`` +PCI ID: + 1b36:0002 +PCI Region 0: + IO bar, 8 bytes long, with the 16550 UART mapped to it. +Interrupt: + Wired to pin A. + + +Multiport cards +--------------- + +Name: + ``pci-serial-2x``, ``pci-serial-4x`` +PCI ID: + 1b36:0003 (``-2x``) and 1b36:0004 (``-4x``) +PCI Region 0: + IO bar, with two or four 16550 UARTs mapped after each other. + The first is at offset 0, the second at offset 8, and so on. +Interrupt: + Wired to pin A. diff --git a/docs/specs/pci-serial.txt b/docs/specs/pci-serial.txt deleted file mode 100644 index 66c761f2b4..0000000000 --- a/docs/specs/pci-serial.txt +++ /dev/null @@ -1,34 +0,0 @@ - -QEMU pci serial devices -======================= - -There is one single-port variant and two muliport-variants. Linux -guests out-of-the box with all cards. There is a Windows inf file -(docs/qemupciserial.inf) to setup the single-port card in Windows -guests. - - -single-port card ----------------- - -Name: pci-serial -PCI ID: 1b36:0002 - -PCI Region 0: - IO bar, 8 bytes long, with the 16550 uart mapped to it. - Interrupt is wired to pin A. - - -multiport cards ---------------- - -Name: pci-serial-2x -PCI ID: 1b36:0003 - -Name: pci-serial-4x -PCI ID: 1b36:0004 - -PCI Region 0: - IO bar, with two/four 16550 uart mapped after each other. - The first is at offset 0, second at offset 8, ... - Interrupt is wired to pin A. diff --git a/docs/specs/pci-testdev.rst b/docs/specs/pci-testdev.rst new file mode 100644 index 0000000000..4b6d36543b --- /dev/null +++ b/docs/specs/pci-testdev.rst @@ -0,0 +1,39 @@ +==================== +QEMU PCI test device +==================== + +``pci-testdev`` is a device used for testing low level IO. + +The device implements up to three BARs: BAR0, BAR1 and BAR2. +Each of BAR 0+1 can be memory or IO. Guests must detect +BAR types and act accordingly. + +BAR 0+1 size is up to 4K bytes each. +BAR 0+1 starts with the following header: + +.. code-block:: c + + typedef struct PCITestDevHdr { + uint8_t test; /* write-only, starts a given test number */ + uint8_t width_type; /* + * read-only, type and width of access for a given test. + * 1,2,4 for byte,word or long write. + * any other value if test not supported on this BAR + */ + uint8_t pad0[2]; + uint32_t offset; /* read-only, offset in this BAR for a given test */ + uint32_t data; /* read-only, data to use for a given test */ + uint32_t count; /* for debugging. number of writes detected. */ + uint8_t name[]; /* for debugging. 0-terminated ASCII string. */ + } PCITestDevHdr; + +All registers are little endian. + +The device is expected to always implement tests 0 to N on each BAR, and to add new +tests with higher numbers. In this way a guest can scan test numbers until it +detects an access type that it does not support on this BAR, then stop. + +BAR2 is a 64bit memory BAR, without backing storage. It is disabled +by default and can be enabled using the ``membar=<size>`` property. This +can be used to test whether guests handle PCI BARs of a specific +(possibly quite large) size correctly. diff --git a/docs/specs/pci-testdev.txt b/docs/specs/pci-testdev.txt deleted file mode 100644 index 4280a1e73c..0000000000 --- a/docs/specs/pci-testdev.txt +++ /dev/null @@ -1,31 +0,0 @@ -pci-test is a device used for testing low level IO - -device implements up to three BARs: BAR0, BAR1 and BAR2. -Each of BAR 0+1 can be memory or IO. Guests must detect -BAR types and act accordingly. - -BAR 0+1 size is up to 4K bytes each. -BAR 0+1 starts with the following header: - -typedef struct PCITestDevHdr { - uint8_t test; <- write-only, starts a given test number - uint8_t width_type; <- read-only, type and width of access for a given test. - 1,2,4 for byte,word or long write. - any other value if test not supported on this BAR - uint8_t pad0[2]; - uint32_t offset; <- read-only, offset in this BAR for a given test - uint32_t data; <- read-only, data to use for a given test - uint32_t count; <- for debugging. number of writes detected. - uint8_t name[]; <- for debugging. 0-terminated ASCII string. -} PCITestDevHdr; - -All registers are little endian. - -device is expected to always implement tests 0 to N on each BAR, and to add new -tests with higher numbers. In this way a guest can scan test numbers until it -detects an access type that it does not support on this BAR, then stop. - -BAR2 is a 64bit memory bar, without backing storage. It is disabled -by default and can be enabled using the membar=<size> property. This -can be used to test whether guests handle pci bars of a specific -(possibly quite large) size correctly. diff --git a/docs/sphinx/dbusdomain.py b/docs/sphinx/dbusdomain.py index 2ea95af623..9872fd5bf6 100644 --- a/docs/sphinx/dbusdomain.py +++ b/docs/sphinx/dbusdomain.py @@ -400,6 +400,10 @@ class DBusDomain(Domain): for refname, obj in self.objects.items(): yield (refname, refname, obj.objtype, obj.docname, obj.node_id, 1) + def merge_domaindata(self, docnames, otherdata): + for name, obj in otherdata['objects'].items(): + if obj.docname in docnames: + self.data['objects'][name] = obj def setup(app): app.add_domain(DBusDomain) diff --git a/docs/sphinx/fakedbusdoc.py b/docs/sphinx/fakedbusdoc.py index d2c5079046..2d2e6ef640 100644 --- a/docs/sphinx/fakedbusdoc.py +++ b/docs/sphinx/fakedbusdoc.py @@ -23,3 +23,8 @@ class FakeDBusDocDirective(Directive): def setup(app: Sphinx) -> Dict[str, Any]: """Register a fake dbus-doc directive with Sphinx""" app.add_directive("dbus-doc", FakeDBusDocDirective) + + return dict( + parallel_read_safe = True, + parallel_write_safe = True + ) diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py index d791b59492..8f3b9997a1 100644 --- a/docs/sphinx/qapidoc.py +++ b/docs/sphinx/qapidoc.py @@ -268,6 +268,9 @@ class QAPISchemaGenRSTVisitor(QAPISchemaVisitor): """Return list of doctree nodes for additional sections""" nodelist = [] for section in doc.sections: + if section.name and section.name == 'TODO': + # Hide TODO: sections + continue snode = self._make_section(section.name) if section.name and section.name.startswith('Example'): snode += self._nodes_for_example(section.text) diff --git a/docs/sphinx/qmp_lexer.py b/docs/sphinx/qmp_lexer.py index f7e4c0e198..a59de8a079 100644 --- a/docs/sphinx/qmp_lexer.py +++ b/docs/sphinx/qmp_lexer.py @@ -41,3 +41,8 @@ def setup(sphinx): sphinx.add_lexer('QMP', QMPExampleLexer) except errors.VersionRequirementError: sphinx.add_lexer('QMP', QMPExampleLexer()) + + return dict( + parallel_read_safe = True, + parallel_write_safe = True + ) diff --git a/docs/system/arm/aspeed.rst b/docs/system/arm/aspeed.rst index d4e293e7f9..80538422a1 100644 --- a/docs/system/arm/aspeed.rst +++ b/docs/system/arm/aspeed.rst @@ -122,6 +122,11 @@ Options specific to Aspeed machines are : * ``spi-model`` to change the SPI Flash model. + * ``bmc-console`` to change the default console device. Most of the + machines use the ``UART5`` device for a boot console, which is + mapped on ``/dev/ttyS4`` under Linux, but it is not always the + case. + For instance, to start the ``ast2500-evb`` machine with a different FMC chip and a bigger (64M) SPI chip, use : @@ -129,6 +134,12 @@ FMC chip and a bigger (64M) SPI chip, use : -M ast2500-evb,fmc-model=mx25l25635e,spi-model=mx66u51235f +To change the boot console and use device ``UART3`` (``/dev/ttyS2`` +under Linux), use : + +.. code-block:: bash + + -M ast2500-evb,bmc-console=uart3 Aspeed minibmc family boards (``ast1030-evb``) ================================================================== diff --git a/docs/system/arm/bananapi_m2u.rst b/docs/system/arm/bananapi_m2u.rst new file mode 100644 index 0000000000..b09ba5c548 --- /dev/null +++ b/docs/system/arm/bananapi_m2u.rst @@ -0,0 +1,139 @@ +Banana Pi BPI-M2U (``bpim2u``) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Banana Pi BPI-M2 Ultra is a quad-core mini single board computer built with +Allwinner A40i/R40/V40 SoC. It features 2GB of RAM and 8GB eMMC. It also +has onboard WiFi and BT. On the ports side, the BPI-M2 Ultra has 2 USB A +2.0 ports, 1 USB OTG port, 1 HDMI port, 1 audio jack, a DC power port, +and last but not least, a SATA port. + +Supported devices +""""""""""""""""" + +The Banana Pi M2U machine supports the following devices: + + * SMP (Quad Core Cortex-A7) + * Generic Interrupt Controller configuration + * SRAM mappings + * SDRAM controller + * Timer device (re-used from Allwinner A10) + * UART + * SD/MMC storage controller + * EMAC ethernet + * GMAC ethernet + * Clock Control Unit + * TWI (I2C) + +Limitations +""""""""""" + +Currently, Banana Pi M2U does *not* support the following features: + +- Graphical output via HDMI, GPU and/or the Display Engine +- Audio output +- Hardware Watchdog +- Real Time Clock +- USB 2.0 interfaces + +Also see the 'unimplemented' array in the Allwinner R40 SoC module +for a complete list of unimplemented I/O devices: ``./hw/arm/allwinner-r40.c`` + +Boot options +"""""""""""" + +The Banana Pi M2U machine can start using the standard -kernel functionality +for loading a Linux kernel or ELF executable. Additionally, the Banana Pi M2U +machine can also emulate the BootROM which is present on an actual Allwinner R40 +based SoC, which loads the bootloader from a SD card, specified via the -sd +argument to qemu-system-arm. + +Running mainline Linux +"""""""""""""""""""""" + +To build a Linux mainline kernel that can be booted by the Banana Pi M2U machine, +simply configure the kernel using the sunxi_defconfig configuration: + +.. code-block:: bash + + $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- make mrproper + $ ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- make sunxi_defconfig + +To boot the newly build linux kernel in QEMU with the Banana Pi M2U machine, use: + +.. code-block:: bash + + $ qemu-system-arm -M bpim2u -nographic \ + -kernel /path/to/linux/arch/arm/boot/zImage \ + -append 'console=ttyS0,115200' \ + -dtb /path/to/linux/arch/arm/boot/dts/sun8i-r40-bananapi-m2-ultra.dtb + +Banana Pi M2U images +"""""""""""""""""""" + +Note that the mainline kernel does not have a root filesystem. You can choose +to build you own image with buildroot using the bananapi_m2_ultra_defconfig. +Also see https://buildroot.org for more information. + +Another possibility is to run an OpenWrt image for Banana Pi M2U which +can be downloaded from: + + https://downloads.openwrt.org/releases/22.03.3/targets/sunxi/cortexa7/ + +When using an image as an SD card, it must be resized to a power of two. This can be +done with the ``qemu-img`` command. It is recommended to only increase the image size +instead of shrinking it to a power of two, to avoid loss of data. For example, +to prepare a downloaded Armbian image, first extract it and then increase +its size to one gigabyte as follows: + +.. code-block:: bash + + $ qemu-img resize \ + openwrt-22.03.3-sunxi-cortexa7-sinovoip_bananapi-m2-ultra-ext4-sdcard.img \ + 1G + +Instead of providing a custom Linux kernel via the -kernel command you may also +choose to let the Banana Pi M2U machine load the bootloader from SD card, just like +a real board would do using the BootROM. Simply pass the selected image via the -sd +argument and remove the -kernel, -append, -dbt and -initrd arguments: + +.. code-block:: bash + + $ qemu-system-arm -M bpim2u -nic user -nographic \ + -sd openwrt-22.03.3-sunxi-cortexa7-sinovoip_bananapi-m2-ultra-ext4-sdcard.img + +Running U-Boot +"""""""""""""" + +U-Boot mainline can be build and configured using the Bananapi_M2_Ultra_defconfig +using similar commands as describe above for Linux. Note that it is recommended +for development/testing to select the following configuration setting in U-Boot: + + Device Tree Control > Provider for DTB for DT Control > Embedded DTB + +The BootROM of allwinner R40 loading u-boot from the 8KiB offset of sdcard. +Let's create an bootable disk image: + +.. code-block:: bash + + $ dd if=/dev/zero of=sd.img bs=32M count=1 + $ dd if=u-boot-sunxi-with-spl.bin of=sd.img bs=1k seek=8 conv=notrunc + +And then boot it. + +.. code-block:: bash + + $ qemu-system-arm -M bpim2u -nographic -sd sd.img + +Banana Pi M2U integration tests +""""""""""""""""""""""""""""""" + +The Banana Pi M2U machine has several integration tests included. +To run the whole set of tests, build QEMU from source and simply +provide the following command: + +.. code-block:: bash + + $ cd qemu-build-dir + $ AVOCADO_ALLOW_LARGE_STORAGE=yes tests/venv/bin/avocado \ + --verbose --show=app,console run -t machine:bpim2u \ + ../tests/avocado/boot_linux_console.py diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst index 7338987875..ecbbd63adf 100644 --- a/docs/system/arm/emulation.rst +++ b/docs/system/arm/emulation.rst @@ -50,6 +50,7 @@ the following architecture extensions: - FEAT_LRCPC (Load-acquire RCpc instructions) - FEAT_LRCPC2 (Load-acquire RCpc instructions v2) - FEAT_LSE (Large System Extensions) +- FEAT_LSE2 (Large System Extensions v2) - FEAT_LVA (Large Virtual Address space) - FEAT_MTE (Memory Tagging Extension) - FEAT_MTE2 (Memory Tagging Extension) diff --git a/docs/system/arm/sbsa.rst b/docs/system/arm/sbsa.rst index b499d7e927..f571fe645e 100644 --- a/docs/system/arm/sbsa.rst +++ b/docs/system/arm/sbsa.rst @@ -6,12 +6,7 @@ any real hardware the ``sbsa-ref`` board intends to look like real hardware. The `Server Base System Architecture <https://developer.arm.com/documentation/den0029/latest>`_ defines a minimum base line of hardware support and importantly how the firmware -reports that to any operating system. It is a static system that -reports a very minimal DT to the firmware for non-discoverable -information about components affected by the qemu command line (i.e. -cpus and memory). As a result it must have a firmware specifically -built to expect a certain hardware layout (as you would in a real -machine). +reports that to any operating system. It is intended to be a machine for developing firmware and testing standards compliance with operating systems. @@ -19,7 +14,7 @@ standards compliance with operating systems. Supported devices """"""""""""""""" -The sbsa-ref board supports: +The ``sbsa-ref`` board supports: - A configurable number of AArch64 CPUs - GIC version 3 @@ -27,6 +22,35 @@ The sbsa-ref board supports: - System bus EHCI controller - CDROM and hard disc on AHCI bus - E1000E ethernet card on PCIe bus - - VGA display adaptor on PCIe bus + - Bochs display adapter on PCIe bus - A generic SBSA watchdog device + +Board to firmware interface +""""""""""""""""""""""""""" + +``sbsa-ref`` is a static system that reports a very minimal devicetree to the +firmware for non-discoverable information about system components. This +includes both internal hardware and parts affected by the qemu command line +(i.e. CPUs and memory). As a result it must have a firmware specifically built +to expect a certain hardware layout (as you would in a real machine). + +DeviceTree information +'''''''''''''''''''''' + +The devicetree provided by the board model to the firmware is not intended +to be a complete compliant DT. It currently reports: + + - CPUs + - memory + - platform version + - GIC addresses + +The platform version is only for informing platform firmware about +what kind of ``sbsa-ref`` board it is running on. It is neither +a QEMU versioned machine type nor a reflection of the level of the +SBSA/SystemReady SR support provided. + +The ``machine-version-major`` value is updated when changes breaking +fw compatibility are introduced. The ``machine-version-minor`` value +is updated when features are added that don't break fw compatibility. diff --git a/docs/system/arm/xenpvh.rst b/docs/system/arm/xenpvh.rst new file mode 100644 index 0000000000..e1655c7ab8 --- /dev/null +++ b/docs/system/arm/xenpvh.rst @@ -0,0 +1,34 @@ +XENPVH (``xenpvh``) +========================================= +This machine creates a IOREQ server to register/connect with Xen Hypervisor. + +When TPM is enabled, this machine also creates a tpm-tis-device at a user input +tpm base address, adds a TPM emulator and connects to a swtpm application +running on host machine via chardev socket. This enables xenpvh to support TPM +functionalities for a guest domain. + +More information about TPM use and installing swtpm linux application can be +found at: docs/specs/tpm.rst. + +Example for starting swtpm on host machine: +.. code-block:: console + + mkdir /tmp/vtpm2 + swtpm socket --tpmstate dir=/tmp/vtpm2 \ + --ctrl type=unixio,path=/tmp/vtpm2/swtpm-sock & + +Sample QEMU xenpvh commands for running and connecting with Xen: +.. code-block:: console + + qemu-system-aarch64 -xen-domid 1 \ + -chardev socket,id=libxl-cmd,path=qmp-libxl-1,server=on,wait=off \ + -mon chardev=libxl-cmd,mode=control \ + -chardev socket,id=libxenstat-cmd,path=qmp-libxenstat-1,server=on,wait=off \ + -mon chardev=libxenstat-cmd,mode=control \ + -xen-attach -name guest0 -vnc none -display none -nographic \ + -machine xenpvh -m 1301 \ + -chardev socket,id=chrtpm,path=tmp/vtpm2/swtpm-sock \ + -tpmdev emulator,id=tpm0,chardev=chrtpm -machine tpm-base-addr=0x0C000000 + +In above QEMU command, last two lines are for connecting xenpvh QEMU to swtpm +via chardev socket. diff --git a/docs/system/arm/xlnx-versal-virt.rst b/docs/system/arm/xlnx-versal-virt.rst index 92ad10d2da..d2d1b26692 100644 --- a/docs/system/arm/xlnx-versal-virt.rst +++ b/docs/system/arm/xlnx-versal-virt.rst @@ -34,6 +34,7 @@ Implemented devices: - DDR memory - BBRAM (36 bytes of Battery-backed RAM) - eFUSE (3072 bytes of one-time field-programmable bit array) +- 2 CANFDs QEMU does not yet model any other devices, including the PL and the AI Engine. @@ -224,3 +225,33 @@ To use a different index value, N, from default of 1, add: Better yet, do not use actual product data when running guest image on this Xilinx Versal Virt board. + +Using CANFDs for Versal Virt +"""""""""""""""""""""""""""" +Versal CANFD controller is developed based on SocketCAN and QEMU CAN bus +implementation. Bus connection and socketCAN connection for each CAN module +can be set through command lines. + +To connect both CANFD0 and CANFD1 on the same bus: + +.. code-block:: bash + + -object can-bus,id=canbus -machine canbus0=canbus -machine canbus1=canbus + +To connect CANFD0 and CANFD1 to separate buses: + +.. code-block:: bash + + -object can-bus,id=canbus0 -object can-bus,id=canbus1 \ + -machine canbus0=canbus0 -machine canbus1=canbus1 + +The SocketCAN interface can connect to a Physical or a Virtual CAN interfaces on +the host machine. Please check this document to learn about CAN interface on +Linux: docs/system/devices/can.rst + +To connect CANFD0 and CANFD1 to host machine's CAN interface can0: + +.. code-block:: bash + + -object can-bus,id=canbus -machine canbus0=canbus -machine canbus1=canbus + -object can-host-socketcan,id=canhost0,if=can0,canbus=canbus diff --git a/docs/system/device-emulation.rst b/docs/system/device-emulation.rst index c1b1934e3d..8d4a1821fa 100644 --- a/docs/system/device-emulation.rst +++ b/docs/system/device-emulation.rst @@ -93,4 +93,5 @@ Emulated Devices devices/virtio-pmem.rst devices/vhost-user-rng.rst devices/canokey.rst + devices/usb-u2f.rst devices/igb.rst diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst index 4c38223069..f12011e230 100644 --- a/docs/system/devices/cxl.rst +++ b/docs/system/devices/cxl.rst @@ -162,7 +162,7 @@ Example system Topology. x marks the match in each decoder level:: |<------------------SYSTEM PHYSICAL ADDRESS MAP (1)----------------->| | __________ __________________________________ __________ | | | | | | | | | - | | CFMW 0 | | CXL Fixed Memory Window 1 | | CFMW 1 | | + | | CFMW 0 | | CXL Fixed Memory Window 1 | | CFMW 2 | | | | HB0 only | | Configured to interleave memory | | HB1 only | | | | | | memory accesses across HB0/HB1 | | | | | |__________| |_____x____________________________| |__________| | @@ -208,8 +208,8 @@ Notes: (1) **3 CXL Fixed Memory Windows (CFMW)** corresponding to different ranges of the system physical address map. Each CFMW has particular interleave setup across the CXL Host Bridges (HB) - CFMW0 provides uninterleaved access to HB0, CFW2 provides - uninterleaved access to HB1. CFW1 provides interleaved memory access + CFMW0 provides uninterleaved access to HB0, CFMW2 provides + uninterleaved access to HB1. CFMW1 provides interleaved memory access across HB0 and HB1. (2) **Two CXL Host Bridges**. Each of these has 2 CXL Root Ports and @@ -247,7 +247,7 @@ Example topology involving a switch:: |<------------------SYSTEM PHYSICAL ADDRESS MAP (1)----------------->| | __________ __________________________________ __________ | | | | | | | | | - | | CFMW 0 | | CXL Fixed Memory Window 1 | | CFMW 1 | | + | | CFMW 0 | | CXL Fixed Memory Window 1 | | CFMW 2 | | | | HB0 only | | Configured to interleave memory | | HB1 only | | | | | | memory accesses across HB0/HB1 | | | | | |____x_____| |__________________________________| |__________| | @@ -300,22 +300,43 @@ Example topology involving a switch:: Example command lines --------------------- -A very simple setup with just one directly attached CXL Type 3 device:: +A very simple setup with just one directly attached CXL Type 3 Persistent Memory device:: - qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \ + qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \ ... -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \ -object memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=256M \ -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \ - -device cxl-type3,bus=root_port13,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \ + -device cxl-type3,bus=root_port13,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \ + -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G + +A very simple setup with just one directly attached CXL Type 3 Volatile Memory device:: + + qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \ + ... + -object memory-backend-ram,id=vmem0,share=on,size=256M \ + -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ + -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \ + -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,id=cxl-vmem0 \ + -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G + +The same volatile setup may optionally include an LSA region:: + + qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \ + ... + -object memory-backend-ram,id=vmem0,share=on,size=256M \ + -object memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa.raw,size=256M \ + -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ + -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \ + -device cxl-type3,bus=root_port13,volatile-memdev=vmem0,lsa=cxl-lsa0,id=cxl-vmem0 \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G A setup suitable for 4 way interleave. Only one fixed window provided, to enable 2 way interleave across 2 CXL host bridges. Each host bridge has 2 CXL Root Ports, with the CXL Type3 device directly attached (no switches).:: - qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \ + qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \ ... -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=256M \ -object memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=256M \ @@ -328,18 +349,18 @@ the CXL Type3 device directly attached (no switches).:: -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ -device pxb-cxl,bus_nr=222,bus=pcie.0,id=cxl.2 \ -device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \ - -device cxl-type3,bus=root_port13,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \ + -device cxl-type3,bus=root_port13,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem0 \ -device cxl-rp,port=1,bus=cxl.1,id=root_port14,chassis=0,slot=3 \ - -device cxl-type3,bus=root_port14,memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem1 \ + -device cxl-type3,bus=root_port14,persistent-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem1 \ -device cxl-rp,port=0,bus=cxl.2,id=root_port15,chassis=0,slot=5 \ - -device cxl-type3,bus=root_port15,memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem2 \ + -device cxl-type3,bus=root_port15,persistent-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem2 \ -device cxl-rp,port=1,bus=cxl.2,id=root_port16,chassis=0,slot=6 \ - -device cxl-type3,bus=root_port16,memdev=cxl-mem4,lsa=cxl-lsa4,id=cxl-pmem3 \ + -device cxl-type3,bus=root_port16,persistent-memdev=cxl-mem4,lsa=cxl-lsa4,id=cxl-pmem3 \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.targets.1=cxl.2,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=8k An example of 4 devices below a switch suitable for 1, 2 or 4 way interleave:: - qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 -cpu max \ + qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \ ... -object memory-backend-file,id=cxl-mem0,share=on,mem-path=/tmp/cxltest.raw,size=256M \ -object memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest1.raw,size=256M \ @@ -354,15 +375,23 @@ An example of 4 devices below a switch suitable for 1, 2 or 4 way interleave:: -device cxl-rp,port=1,bus=cxl.1,id=root_port1,chassis=0,slot=1 \ -device cxl-upstream,bus=root_port0,id=us0 \ -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \ - -device cxl-type3,bus=swport0,memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-pmem0,size=256M \ + -device cxl-type3,bus=swport0,persistent-memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-pmem0 \ -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \ - -device cxl-type3,bus=swport1,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem1,size=256M \ + -device cxl-type3,bus=swport1,persistent-memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem1 \ -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \ - -device cxl-type3,bus=swport2,memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem2,size=256M \ + -device cxl-type3,bus=swport2,persistent-memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem2 \ -device cxl-downstream,port=3,bus=us0,id=swport3,chassis=0,slot=7 \ - -device cxl-type3,bus=swport3,memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem3,size=256M \ + -device cxl-type3,bus=swport3,persistent-memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem3 \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k +Deprecations +------------ + +The Type 3 device [memdev] attribute has been deprecated in favor of the +[persistent-memdev] attributes. [memdev] will default to a persistent memory +device for backward compatibility and is incapable of being used in combination +with [persistent-memdev]. + Kernel Configuration Options ---------------------------- diff --git a/docs/system/devices/igb.rst b/docs/system/devices/igb.rst index 70edadd574..04e79dfe54 100644 --- a/docs/system/devices/igb.rst +++ b/docs/system/devices/igb.rst @@ -14,7 +14,8 @@ Limitations =========== This igb implementation was tested with Linux Test Project [2]_ and Windows HLK -[3]_ during the initial development. The command used when testing with LTP is: +[3]_ during the initial development. Later it was also tested with DPDK Test +Suite [4]_. The command used when testing with LTP is: .. code-block:: shell @@ -22,17 +23,17 @@ This igb implementation was tested with Linux Test Project [2]_ and Windows HLK Be aware that this implementation lacks many functionalities available with the actual hardware, and you may experience various failures if you try to use it -with a different operating system other than Linux and Windows or if you try -functionalities not covered by the tests. +with a different operating system other than DPDK, Linux, and Windows or if you +try functionalities not covered by the tests. Using igb ========= Using igb should be nothing different from using another network device. See -:ref:`pcsys_005fnetwork` in general. +:ref:`Network_emulation` in general. However, you may also need to perform additional steps to activate SR-IOV -feature on your guest. For Linux, refer to [4]_. +feature on your guest. For Linux, refer to [5]_. Developing igb ============== @@ -60,7 +61,7 @@ Avocado test and can be ran with the following command: .. code:: shell - make check-avocado AVOCADO_TESTS=tests/avocado/igb.py + make check-avocado AVOCADO_TESTS=tests/avocado/netdev-ethtool.py References ========== @@ -68,4 +69,5 @@ References .. [1] https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/82576eb-gigabit-ethernet-controller-datasheet.pdf .. [2] https://github.com/linux-test-project/ltp .. [3] https://learn.microsoft.com/en-us/windows-hardware/test/hlk/ -.. [4] https://docs.kernel.org/PCI/pci-iov-howto.html +.. [4] https://doc.dpdk.org/dts/gsg/ +.. [5] https://docs.kernel.org/PCI/pci-iov-howto.html diff --git a/docs/system/devices/ivshmem.rst b/docs/system/devices/ivshmem.rst index b03a48afa3..e7aaf34c20 100644 --- a/docs/system/devices/ivshmem.rst +++ b/docs/system/devices/ivshmem.rst @@ -1,5 +1,3 @@ -.. _pcsys_005fivshmem: - Inter-VM Shared Memory device ----------------------------- diff --git a/docs/system/devices/net.rst b/docs/system/devices/net.rst index 4b2640c448..2ab516d4b0 100644 --- a/docs/system/devices/net.rst +++ b/docs/system/devices/net.rst @@ -1,4 +1,4 @@ -.. _pcsys_005fnetwork: +.. _Network_Emulation: Network emulation ----------------- diff --git a/docs/system/devices/usb-u2f.rst b/docs/system/devices/usb-u2f.rst new file mode 100644 index 0000000000..4f57d5c8c3 --- /dev/null +++ b/docs/system/devices/usb-u2f.rst @@ -0,0 +1,93 @@ +Universal Second Factor (U2F) USB Key Device +============================================ + +U2F is an open authentication standard that enables relying parties +exposed to the internet to offer a strong second factor option for end +user authentication. + +The second factor is provided by a device implementing the U2F +protocol. In case of a USB U2F security key, it is a USB HID device +that implements the U2F protocol. + +QEMU supports both pass-through of a host U2F key device to a VM, +and software emulation of a U2F key. + +``u2f-passthru`` +---------------- + +The ``u2f-passthru`` device allows you to connect a real hardware +U2F key on your host to a guest VM. All requests made from the guest +are passed through to the physical security key connected to the +host machine and vice versa. + +In addition, the dedicated pass-through allows you to share a single +U2F security key with several guest VMs, which is not possible with a +simple host device assignment pass-through. + +You can specify the host U2F key to use with the ``hidraw`` +option, which takes the host path to a Linux ``/dev/hidrawN`` device: + +.. parsed-literal:: + |qemu_system| -usb -device u2f-passthru,hidraw=/dev/hidraw0 + +If you don't specify the device, the ``u2f-passthru`` device will +autoscan to take the first U2F device it finds on the host (this +requires a working libudev): + +.. parsed-literal:: + |qemu_system| -usb -device u2f-passthru + +``u2f-emulated`` +---------------- + +``u2f-emulated`` is a completely software emulated U2F device. +It uses `libu2f-emu <https://github.com/MattGorko/libu2f-emu>`__ +for the U2F key emulation. libu2f-emu +provides a complete implementation of the U2F protocol device part for +all specified transports given by the FIDO Alliance. + +To work, an emulated U2F device must have four elements: + + * ec x509 certificate + * ec private key + * counter (four bytes value) + * 48 bytes of entropy (random bits) + +To use this type of device, these have to be configured, and these +four elements must be passed one way or another. + +Assuming that you have a working libu2f-emu installed on the host, +there are three possible ways to configure the ``u2f-emulated`` device: + + * ephemeral + * setup directory + * manual + +Ephemeral is the simplest way to configure; it lets the device generate +all the elements it needs for a single use of the lifetime of the device. +It is the default if you do not pass any other options to the device. + +.. parsed-literal:: + |qemu_system| -usb -device u2f-emulated + +You can pass the device the path of a setup directory on the host +using the ``dir`` option; the directory must contain these four files: + + * ``certificate.pem``: ec x509 certificate + * ``private-key.pem``: ec private key + * ``counter``: counter value + * ``entropy``: 48 bytes of entropy + +.. parsed-literal:: + |qemu_system| -usb -device u2f-emulated,dir=$dir + +You can also manually pass the device the paths to each of these files, +if you don't want them all to be in the same directory, using the options + + * ``cert`` + * ``priv`` + * ``counter`` + * ``entropy`` + +.. parsed-literal:: + |qemu_system| -usb -device u2f-emulated,cert=$DIR1/$FILE1,priv=$DIR2/$FILE2,counter=$DIR3/$FILE3,entropy=$DIR4/$FILE4 diff --git a/docs/system/devices/usb.rst b/docs/system/devices/usb.rst index 37cb9b33ae..a6ca7b0c37 100644 --- a/docs/system/devices/usb.rst +++ b/docs/system/devices/usb.rst @@ -1,5 +1,3 @@ -.. _pcsys_005fusb: - USB emulation ------------- @@ -209,7 +207,7 @@ option or the ``device_add`` monitor command. Available devices are: USB audio device ``u2f-{emulated,passthru}`` - Universal Second Factor device + :doc:`usb-u2f` ``canokey`` An Open-source Secure Key implementing FIDO2, OpenPGP, PIV and more. diff --git a/docs/system/devices/vhost-user.rst b/docs/system/devices/vhost-user.rst index 86128114fa..a80e95a48a 100644 --- a/docs/system/devices/vhost-user.rst +++ b/docs/system/devices/vhost-user.rst @@ -38,13 +38,13 @@ system memory as defined by the ``-m`` argument. Example ======= -First start you daemon. +First start your daemon. .. parsed-literal:: $ virtio-foo --socket-path=/var/run/foo.sock $OTHER_ARGS -The you start your QEMU instance specifying the device, chardev and +Then you start your QEMU instance specifying the device, chardev and memory objects. .. parsed-literal:: diff --git a/docs/system/gdb.rst b/docs/system/gdb.rst index 453eb73f6c..7d3718deef 100644 --- a/docs/system/gdb.rst +++ b/docs/system/gdb.rst @@ -46,6 +46,28 @@ Here are some useful tips in order to use gdb on system code: 3. Use ``set architecture i8086`` to dump 16 bit code. Then use ``x/10i $cs*16+$eip`` to dump the code at the PC position. +Breakpoint and Watchpoint support +================================= + +While GDB can always fall back to inserting breakpoints into memory +(if writable) other features are very much dependent on support of the +accelerator. For TCG system emulation we advertise an infinite number +of hardware assisted breakpoints and watchpoints. For other +accelerators it will depend on if support has been added (see +supports_guest_debug and related hooks in AccelOpsClass). + +As TCG cannot track all memory accesses in user-mode there is no +support for watchpoints. + +Relocating code +--------------- + +On modern kernels confusion can be caused by code being relocated by +features such as address space layout randomisation. To avoid +confusion when debugging such things you either need to update gdb's +view of where things are in memory or perhaps more trivially disable +ASLR when booting the system. + Debugging multicore machines ============================ diff --git a/docs/system/guest-loader.rst b/docs/system/guest-loader.rst index 9ef9776bf0..304ee5d531 100644 --- a/docs/system/guest-loader.rst +++ b/docs/system/guest-loader.rst @@ -14,7 +14,7 @@ The guest loader does two things: - load blobs (kernels and initial ram disks) into memory - sets platform FDT data so hypervisors can find and boot them -This is what is typically done by a boot-loader like grub using it's +This is what is typically done by a boot-loader like grub using its multi-boot capability. A typical example would look like: .. parsed-literal:: @@ -25,9 +25,9 @@ multi-boot capability. A typical example would look like: -device guest-loader,addr=0x47000000,initrd=rootfs.cpio In the above example the Xen hypervisor is loaded by the -kernel -parameter and passed it's boot arguments via -append. The Dom0 guest +parameter and passed its boot arguments via -append. The Dom0 guest is loaded into the areas of memory. Each blob will get -``/chosen/module@<addr>`` entry in the FDT to indicate it's location and +``/chosen/module@<addr>`` entry in the FDT to indicate its location and size. Additional information can be passed with by using additional arguments. diff --git a/docs/system/keys.rst b/docs/system/keys.rst index e596ae6c4e..0fc17b994d 100644 --- a/docs/system/keys.rst +++ b/docs/system/keys.rst @@ -1,4 +1,4 @@ -.. _pcsys_005fkeys: +.. _GUI_keys: Keys in the graphical frontends ------------------------------- diff --git a/docs/system/linuxboot.rst b/docs/system/linuxboot.rst index 228650abc5..5db2e560dc 100644 --- a/docs/system/linuxboot.rst +++ b/docs/system/linuxboot.rst @@ -27,4 +27,4 @@ virtual serial port and the QEMU monitor to the console with the -append "root=/dev/hda console=ttyS0" -nographic Use Ctrl-a c to switch between the serial console and the monitor (see -:ref:`pcsys_005fkeys`). +:ref:`GUI_keys`). diff --git a/docs/system/multi-process.rst b/docs/system/multi-process.rst index 16f0352416..2008a67809 100644 --- a/docs/system/multi-process.rst +++ b/docs/system/multi-process.rst @@ -4,7 +4,7 @@ Multi-process QEMU ================== This document describes how to configure and use multi-process qemu. -For the design document refer to docs/devel/qemu-multiprocess. +For the design document refer to docs/devel/multi-process.rst. 1) Configuration ---------------- diff --git a/docs/system/qemu-block-drivers.rst.inc b/docs/system/qemu-block-drivers.rst.inc index dfe5d2293d..105cb9679c 100644 --- a/docs/system/qemu-block-drivers.rst.inc +++ b/docs/system/qemu-block-drivers.rst.inc @@ -430,6 +430,12 @@ Hard disks you may corrupt your host data (use the ``-snapshot`` command line option or modify the device permissions accordingly). +Zoned block devices + Zoned block devices can be passed through to the guest if the emulated storage + controller supports zoned storage. Use ``--blockdev host_device, + node-name=drive0,filename=/dev/nullb0,cache.direct=on`` to pass through + ``/dev/nullb0`` as ``drive0``. + Windows ^^^^^^^ diff --git a/docs/system/riscv/virt.rst b/docs/system/riscv/virt.rst index 4b16e41d7f..b33f45e5b3 100644 --- a/docs/system/riscv/virt.rst +++ b/docs/system/riscv/virt.rst @@ -53,6 +53,37 @@ with the default OpenSBI firmware image as the -bios. It also supports the recommended RISC-V bootflow: U-Boot SPL (M-mode) loads OpenSBI fw_dynamic firmware and U-Boot proper (S-mode), using the standard -bios functionality. +Using flash devices +------------------- + +By default, the first flash device (pflash0) is expected to contain +S-mode firmware code. It can be configured as read-only, with the +second flash device (pflash1) available to store configuration data. + +For example, booting edk2 looks like + +.. code-block:: bash + + $ qemu-system-riscv64 \ + -blockdev node-name=pflash0,driver=file,read-only=on,filename=<edk2_code> \ + -blockdev node-name=pflash1,driver=file,filename=<edk2_vars> \ + -M virt,pflash0=pflash0,pflash1=pflash1 \ + ... other args .... + +For TCG guests only, it is also possible to boot M-mode firmware from +the first flash device (pflash0) by additionally passing ``-bios +none``, as in + +.. code-block:: bash + + $ qemu-system-riscv64 \ + -bios none \ + -blockdev node-name=pflash0,driver=file,read-only=on,filename=<m_mode_code> \ + -M virt,pflash0=pflash0 \ + ... other args .... + +Firmware images used for pflash must be exactly 32 MiB in size. + Machine-specific options ------------------------ diff --git a/docs/system/target-arm.rst b/docs/system/target-arm.rst index 91ebc26c6d..790ac1b8a2 100644 --- a/docs/system/target-arm.rst +++ b/docs/system/target-arm.rst @@ -83,6 +83,7 @@ undocumented; you can get a complete list by running arm/versatile arm/vexpress arm/aspeed + arm/bananapi_m2u.rst arm/sabrelite arm/digic arm/cubieboard @@ -106,6 +107,7 @@ undocumented; you can get a complete list by running arm/stm32 arm/virt arm/xlnx-versal-virt + arm/xenpvh Emulated CPU architecture support ================================= diff --git a/docs/system/target-i386.rst b/docs/system/target-i386.rst index 77c2f3b979..1b8a1f248a 100644 --- a/docs/system/target-i386.rst +++ b/docs/system/target-i386.rst @@ -3,8 +3,6 @@ x86 System emulator ------------------- -.. _pcsys_005fdevices: - Board-specific documentation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -32,8 +30,6 @@ Architectural features i386/sgx i386/amd-memory-encryption -.. _pcsys_005freq: - OS requirements ~~~~~~~~~~~~~~~ diff --git a/docs/u2f.txt b/docs/u2f.txt deleted file mode 100644 index 7f5813a0b7..0000000000 --- a/docs/u2f.txt +++ /dev/null @@ -1,110 +0,0 @@ -QEMU U2F Key Device Documentation. - -Contents -1. USB U2F key device -2. Building -3. Using u2f-emulated -4. Using u2f-passthru -5. Libu2f-emu - -1. USB U2F key device - -U2F is an open authentication standard that enables relying parties -exposed to the internet to offer a strong second factor option for end -user authentication. - -The standard brings many advantages to both parties, client and server, -allowing to reduce over-reliance on passwords, it increases authentication -security and simplifies passwords. - -The second factor is materialized by a device implementing the U2F -protocol. In case of a USB U2F security key, it is a USB HID device -that implements the U2F protocol. - -In QEMU, the USB U2F key device offers a dedicated support of U2F, allowing -guest USB FIDO/U2F security keys operating in two possible modes: -pass-through and emulated. - -The pass-through mode consists of passing all requests made from the guest -to the physical security key connected to the host machine and vice versa. -In addition, the dedicated pass-through allows to have a U2F security key -shared on several guests which is not possible with a simple host device -assignment pass-through. - -The emulated mode consists of completely emulating the behavior of an -U2F device through software part. Libu2f-emu is used for that. - - -2. Building - -To ensure the build of the u2f-emulated device variant which depends -on libu2f-emu: configuring and building: - - ./configure --enable-u2f && make - -The pass-through mode is built by default on Linux. To take advantage -of the autoscan option it provides, make sure you have a working libudev -installed on the host. - - -3. Using u2f-emulated - -To work, an emulated U2F device must have four elements: - * ec x509 certificate - * ec private key - * counter (four bytes value) - * 48 bytes of entropy (random bits) - -To use this type of device, this one has to be configured, and these -four elements must be passed one way or another. - -Assuming that you have a working libu2f-emu installed on the host. -There are three possible ways of configurations: - * ephemeral - * setup directory - * manual - -Ephemeral is the simplest way to configure, it lets the device generate -all the elements it needs for a single use of the lifetime of the device. - - qemu -usb -device u2f-emulated - -Setup directory allows to configure the device from a directory containing -four files: - * certificate.pem: ec x509 certificate - * private-key.pem: ec private key - * counter: counter value - * entropy: 48 bytes of entropy - - qemu -usb -device u2f-emulated,dir=$dir - -Manual allows to configure the device more finely by specifying each -of the elements necessary for the device: - * cert - * priv - * counter - * entropy - - qemu -usb -device u2f-emulated,cert=$DIR1/$FILE1,priv=$DIR2/$FILE2,counter=$DIR3/$FILE3,entropy=$DIR4/$FILE4 - - -4. Using u2f-passthru - -On the host specify the u2f-passthru device with a suitable hidraw: - - qemu -usb -device u2f-passthru,hidraw=/dev/hidraw0 - -Alternately, the u2f-passthru device can autoscan to take the first -U2F device it finds on the host (this requires a working libudev): - - qemu -usb -device u2f-passthru - - -5. Libu2f-emu - -The u2f-emulated device uses libu2f-emu for the U2F key emulation. Libu2f-emu -implements completely the U2F protocol device part for all specified -transport given by the FIDO Alliance. - -For more information about libu2f-emu see this page: -https://github.com/MattGorko/libu2f-emu. diff --git a/docs/user/main.rst b/docs/user/main.rst index 6f2ffa080f..f478635396 100644 --- a/docs/user/main.rst +++ b/docs/user/main.rst @@ -93,8 +93,13 @@ Debug options: ``-g port`` Wait gdb connection to port +``-one-insn-per-tb`` + Run the emulation with one guest instruction per translation block. + This slows down emulation a lot, but can be useful in some situations, + such as when trying to analyse the logs produced by the ``-d`` option. + ``-singlestep`` - Run the emulation in single step mode. + This is a deprecated synonym for the ``-one-insn-per-tb`` option. Environment variables: @@ -242,5 +247,10 @@ Debug options: ``-p pagesize`` Act as if the host page size was 'pagesize' bytes +``-one-insn-per-tb`` + Run the emulation with one guest instruction per translation block. + This slows down emulation a lot, but can be useful in some situations, + such as when trying to analyse the logs produced by the ``-d`` option. + ``-singlestep`` - Run the emulation in single step mode. + This is a deprecated synonym for the ``-one-insn-per-tb`` option. |