diff options
Diffstat (limited to 'docs/devel')
| -rw-r--r-- | docs/devel/build-environment.rst | 118 | ||||
| -rw-r--r-- | docs/devel/codebase.rst | 220 | ||||
| -rw-r--r-- | docs/devel/control-flow-integrity.rst | 2 | ||||
| -rw-r--r-- | docs/devel/decodetree.rst | 2 | ||||
| -rw-r--r-- | docs/devel/ebpf_rss.rst | 2 | ||||
| -rw-r--r-- | docs/devel/index-build.rst | 1 | ||||
| -rw-r--r-- | docs/devel/index-internals.rst | 2 | ||||
| -rw-r--r-- | docs/devel/index-process.rst | 1 | ||||
| -rw-r--r-- | docs/devel/index.rst | 1 | ||||
| -rw-r--r-- | docs/devel/migration/CPR.rst | 184 | ||||
| -rw-r--r-- | docs/devel/migration/main.rst | 2 | ||||
| -rw-r--r-- | docs/devel/multi-thread-tcg.rst | 3 | ||||
| -rw-r--r-- | docs/devel/qapi-code-gen.rst | 5 | ||||
| -rw-r--r-- | docs/devel/rust.rst | 476 | ||||
| -rw-r--r-- | docs/devel/style.rst | 20 | ||||
| -rw-r--r-- | docs/devel/submitting-a-patch.rst | 39 | ||||
| -rw-r--r-- | docs/devel/tcg-ops.rst | 19 | ||||
| -rw-r--r-- | docs/devel/testing/functional.rst | 8 | ||||
| -rw-r--r-- | docs/devel/testing/main.rst | 9 | ||||
| -rw-r--r-- | docs/devel/testing/qtest.rst | 2 |
20 files changed, 1101 insertions, 15 deletions
diff --git a/docs/devel/build-environment.rst b/docs/devel/build-environment.rst new file mode 100644 index 0000000000..f133ef2e01 --- /dev/null +++ b/docs/devel/build-environment.rst @@ -0,0 +1,118 @@ + +.. _setup-build-env: + +Setup build environment +======================= + +QEMU uses a lot of dependencies on the host system. glib2 is used everywhere in +the code base, and most of the other dependencies are optional. + +We present here simple instructions to enable native builds on most popular +systems. + +You can find additional instructions on `QEMU wiki <https://wiki.qemu.org/>`_: + +- `Linux <https://wiki.qemu.org/Hosts/Linux>`_ +- `MacOS <https://wiki.qemu.org/Hosts/Mac>`_ +- `Windows <https://wiki.qemu.org/Hosts/W32>`_ +- `BSD <https://wiki.qemu.org/Hosts/BSD>`_ + +Note: Installing dependencies using your package manager build dependencies may +miss out on deps that have been newly introduced in qemu.git. In more, it misses +deps the distribution has decided to exclude. + +Linux +----- + +Fedora +++++++ + +:: + + sudo dnf update && sudo dnf builddep qemu + +Debian/Ubuntu ++++++++++++++ + +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_. +Then, use apt to install dependencies: + +:: + + sudo apt update && sudo apt build-dep qemu + +MacOS +----- + +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to +install dependencies: + +:: + + brew update && brew install $(brew deps --include-build qemu) + +Windows +------- + +You first need to install `MSYS2 <https://www.msys2.org/>`_. +MSYS2 offers `different environments <https://www.msys2.org/docs/environments/>`_. +x86_64 environments are based on GCC, while aarch64 is based on Clang. + +We recommend to use MINGW64 for windows-x86_64 and CLANGARM64 for windows-aarch64 +(only available on windows-aarch64 hosts). + +Then, you can open a windows shell, and enter msys2 env using: + +:: + + c:/msys64/msys2_shell.cmd -defterm -here -no-start -mingw64 + # Replace -ucrt64 by -clangarm64 or -ucrt64 for other environments. + +MSYS2 package manager does not offer a built-in way to install build +dependencies. You can start with this list of packages using pacman: + +Note: Dependencies need to be installed again if you use a different MSYS2 +environment. + +:: + + # update MSYS2 itself, you need to reopen your shell at the end. + pacman -Syu + pacman -S \ + base-devel binutils bison diffutils flex git grep make sed \ + ${MINGW_PACKAGE_PREFIX}-toolchain \ + ${MINGW_PACKAGE_PREFIX}-glib2 \ + ${MINGW_PACKAGE_PREFIX}-gtk3 \ + ${MINGW_PACKAGE_PREFIX}-libnfs \ + ${MINGW_PACKAGE_PREFIX}-libssh \ + ${MINGW_PACKAGE_PREFIX}-ninja \ + ${MINGW_PACKAGE_PREFIX}-pixman \ + ${MINGW_PACKAGE_PREFIX}-pkgconf \ + ${MINGW_PACKAGE_PREFIX}-python \ + ${MINGW_PACKAGE_PREFIX}-SDL2 \ + ${MINGW_PACKAGE_PREFIX}-zstd + +If you want to install all dependencies, it's possible to use recipe used to +build QEMU in MSYS2 itself. + +:: + + pacman -S wget + wget https://raw.githubusercontent.com/msys2/MINGW-packages/refs/heads/master/mingw-w64-qemu/PKGBUILD + # Some packages may be missing for your environment, installation will still + # be done though. + makepkg -s PKGBUILD || true + +Build on windows-aarch64 +++++++++++++++++++++++++ + +When trying to cross compile meson for x86_64 using UCRT64 or MINGW64 env, +configure will run into an error because the cpu detected is not correct. + +Meson detects x86_64 processes emulated, so you need to manually set the cpu, +and force a cross compilation (with empty prefix). + +:: + + ./configure --cpu=x86_64 --cross-prefix= + diff --git a/docs/devel/codebase.rst b/docs/devel/codebase.rst new file mode 100644 index 0000000000..4039875ee0 --- /dev/null +++ b/docs/devel/codebase.rst @@ -0,0 +1,220 @@ +======== +Codebase +======== + +This section presents the various parts of QEMU and how the codebase is +organized. + +Beyond giving succint descriptions, the goal is to offer links to various +parts of the documentation/codebase. + +Subsystems +---------- + +An exhaustive list of subsystems and associated files can be found in the +`MAINTAINERS <https://gitlab.com/qemu-project/qemu/-/blob/master/MAINTAINERS>`_ +file. + +Some of the main QEMU subsystems are: + +- `Accelerators<Accelerators>` +- Block devices and `disk images<disk images>` support +- `CI<ci>` and `Tests<testing>` +- `Devices<device-emulation>` & Board models +- `Documentation <documentation-root>` +- `GDB support<GDB usage>` +- `Migration<migration>` +- `Monitor<QEMU monitor>` +- :ref:`QOM (QEMU Object Model)<qom>` +- `System mode<System emulation>` +- :ref:`TCG (Tiny Code Generator)<tcg>` +- `User mode<user-mode>` (`Linux<linux-user-mode>` & `BSD<bsd-user-mode>`) +- User Interfaces + +More documentation on QEMU subsystems can be found on :ref:`internal-subsystem` +page. + +The Grand tour +-------------- + +We present briefly here what every folder in the top directory of the codebase +contains. Hop on! + +The folder name links here will take you to that folder in our gitlab +repository. Other links will take you to more detailed documentation for that +subsystem, where we have it. Unfortunately not every subsystem has documentation +yet, so sometimes the source code is all you have. + +* `accel <https://gitlab.com/qemu-project/qemu/-/tree/master/accel>`_: + Infrastructure and architecture agnostic code related to the various + `accelerators <Accelerators>` supported by QEMU + (TCG, KVM, hvf, whpx, xen, nvmm). + Contains interfaces for operations that will be implemented per + `target <https://gitlab.com/qemu-project/qemu/-/tree/master/target>`_. +* `audio <https://gitlab.com/qemu-project/qemu/-/tree/master/audio>`_: + Audio (host) support. +* `authz <https://gitlab.com/qemu-project/qemu/-/tree/master/authz>`_: + `QEMU Authorization framework<client authorization>`. +* `backends <https://gitlab.com/qemu-project/qemu/-/tree/master/backends>`_: + Various backends that are used to access resources on the host (e.g. for + random number generation, memory backing or cryptographic functions). +* `block <https://gitlab.com/qemu-project/qemu/-/tree/master/block>`_: + Block devices and `image formats<disk images>` implementation. +* `bsd-user <https://gitlab.com/qemu-project/qemu/-/tree/master/bsd-user>`_: + `BSD User mode<bsd-user-mode>`. +* build: Where the code built goes by default. You can tell the QEMU build + system to put the built code anywhere else you like. +* `chardev <https://gitlab.com/qemu-project/qemu/-/tree/master/chardev>`_: + Various backends used by char devices. +* `common-user <https://gitlab.com/qemu-project/qemu/-/tree/master/common-user>`_: + User-mode assembly code for dealing with signals occuring during syscalls. +* `configs <https://gitlab.com/qemu-project/qemu/-/tree/master/configs>`_: + Makefiles defining configurations to build QEMU. +* `contrib <https://gitlab.com/qemu-project/qemu/-/tree/master/contrib>`_: + Community contributed devices/plugins/tools. +* `crypto <https://gitlab.com/qemu-project/qemu/-/tree/master/crypto>`_: + Cryptographic algorithms used in QEMU. +* `disas <https://gitlab.com/qemu-project/qemu/-/tree/master/disas>`_: + Disassembly functions used by QEMU target code. +* `docs <https://gitlab.com/qemu-project/qemu/-/tree/master/docs>`_: + QEMU Documentation. +* `dump <https://gitlab.com/qemu-project/qemu/-/tree/master/dump>`_: + Code to dump memory of a running VM. +* `ebpf <https://gitlab.com/qemu-project/qemu/-/tree/master/ebpf>`_: + eBPF program support in QEMU. `virtio-net RSS<ebpf-rss>` uses it. +* `fpu <https://gitlab.com/qemu-project/qemu/-/tree/master/fpu>`_: + Floating-point software emulation. +* `fsdev <https://gitlab.com/qemu-project/qemu/-/tree/master/fsdev>`_: + `VirtFS <https://www.linux-kvm.org/page/VirtFS>`_ support. +* `gdbstub <https://gitlab.com/qemu-project/qemu/-/tree/master/gdbstub>`_: + `GDB <GDB usage>` support. +* `gdb-xml <https://gitlab.com/qemu-project/qemu/-/tree/master/gdb-xml>`_: + Set of XML files describing architectures and used by `gdbstub <GDB usage>`. +* `host <https://gitlab.com/qemu-project/qemu/-/tree/master/host>`_: + Various architecture specific header files (crypto, atomic, memory + operations). +* `linux-headers <https://gitlab.com/qemu-project/qemu/-/tree/master/linux-headers>`_: + A subset of headers imported from Linux kernel and used for implementing + KVM support and user-mode. +* `linux-user <https://gitlab.com/qemu-project/qemu/-/tree/master/linux-user>`_: + `User mode <user-mode>` implementation. Contains one folder per target + architecture. +* `.gitlab-ci.d <https://gitlab.com/qemu-project/qemu/-/tree/master/.gitlab-ci.d>`_: + `CI <ci>` yaml and scripts. +* `include <https://gitlab.com/qemu-project/qemu/-/tree/master/include>`_: + All headers associated to different subsystems in QEMU. The hierachy used + mirrors source code organization and naming. +* `hw <https://gitlab.com/qemu-project/qemu/-/tree/master/hw>`_: + `Devices <device-emulation>` and boards emulation. Devices are categorized by + type/protocol/architecture and located in associated subfolder. +* `io <https://gitlab.com/qemu-project/qemu/-/tree/master/io>`_: + QEMU `I/O channels <https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg04208.html>`_. +* `libdecnumber <https://gitlab.com/qemu-project/qemu/-/tree/master/libdecnumber>`_: + Import of gcc library, used to implement decimal number arithmetic. +* `migration <https://gitlab.com/qemu-project/qemu/-/tree/master/migration>`__: + `Migration framework <migration>`. +* `monitor <https://gitlab.com/qemu-project/qemu/-/tree/master/monitor>`_: + `Monitor <QEMU monitor>` implementation (HMP & QMP). +* `nbd <https://gitlab.com/qemu-project/qemu/-/tree/master/nbd>`_: + QEMU `NBD (Network Block Device) <nbd>` server. +* `net <https://gitlab.com/qemu-project/qemu/-/tree/master/net>`_: + Network (host) support. +* `pc-bios <https://gitlab.com/qemu-project/qemu/-/tree/master/pc-bios>`_: + Contains pre-built firmware binaries and boot images, ready to use in + QEMU without compilation. +* `plugins <https://gitlab.com/qemu-project/qemu/-/tree/master/plugins>`_: + :ref:`TCG plugins <tcg-plugins>` core implementation. Plugins can be found in + `tests <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/tcg/plugins>`__ + and `contrib <https://gitlab.com/qemu-project/qemu/-/tree/master/contrib/plugins>`__ + folders. +* `po <https://gitlab.com/qemu-project/qemu/-/tree/master/po>`_: + Translation files. +* `python <https://gitlab.com/qemu-project/qemu/-/tree/master/python>`_: + Python part of our build/test system. +* `qapi <https://gitlab.com/qemu-project/qemu/-/tree/master/qapi>`_: + `QAPI <qapi>` implementation. +* `qobject <https://gitlab.com/qemu-project/qemu/-/tree/master/qobject>`_: + QEMU Object implementation. +* `qga <https://gitlab.com/qemu-project/qemu/-/tree/master/qga>`_: + QEMU `Guest agent <qemu-ga>` implementation. +* `qom <https://gitlab.com/qemu-project/qemu/-/tree/master/qom>`_: + QEMU :ref:`Object model <qom>` implementation, with monitor associated commands. +* `replay <https://gitlab.com/qemu-project/qemu/-/tree/master/replay>`_: + QEMU :ref:`Record/replay <replay>` implementation. +* `roms <https://gitlab.com/qemu-project/qemu/-/tree/master/roms>`_: + Contains source code for various firmware and ROMs, which can be compiled if + custom or updated versions are needed. +* `rust <https://gitlab.com/qemu-project/qemu/-/tree/master/rust>`_: + Rust integration in QEMU. It contains the new interfaces defined and + associated devices using it. +* `scripts <https://gitlab.com/qemu-project/qemu/-/tree/master/scripts>`_: + Collection of scripts used in build and test systems, and various + tools for QEMU codebase and execution traces. +* `scsi <https://gitlab.com/qemu-project/qemu/-/tree/master/scsi>`_: + Code related to SCSI support, used by SCSI devices. +* `semihosting <https://gitlab.com/qemu-project/qemu/-/tree/master/semihosting>`_: + QEMU `Semihosting <Semihosting>` implementation. +* `stats <https://gitlab.com/qemu-project/qemu/-/tree/master/stats>`_: + `Monitor <QEMU monitor>` stats commands implementation. +* `storage-daemon <https://gitlab.com/qemu-project/qemu/-/tree/master/storage-daemon>`_: + QEMU `Storage daemon <storage-daemon>` implementation. +* `stubs <https://gitlab.com/qemu-project/qemu/-/tree/master/stubs>`_: + Various stubs (empty functions) used to compile QEMU with specific + configurations. +* `subprojects <https://gitlab.com/qemu-project/qemu/-/tree/master/subprojects>`_: + QEMU submodules used by QEMU build system. +* `system <https://gitlab.com/qemu-project/qemu/-/tree/master/system>`_: + QEMU `system mode <System emulation>` implementation (cpu, mmu, boot support). +* `target <https://gitlab.com/qemu-project/qemu/-/tree/master/target>`_: + Contains code for all target architectures supported (one subfolder + per arch). For every architecture, you can find accelerator specific + implementations. +* `tcg <https://gitlab.com/qemu-project/qemu/-/tree/master/tcg>`_: + :ref:`TCG <tcg>` related code. + Contains one subfolder per host supported architecture. +* `tests <https://gitlab.com/qemu-project/qemu/-/tree/master/tests>`_: + QEMU `test <testing>` suite + + - `avocado <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/avocado>`_: + Functional tests booting full VM using `Avocado framework <checkavocado-ref>`. + Those tests will be transformed and moved into + `tests/functional <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/functional>`_ + in the future. + - `data <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/data>`_: + Data for various tests. + - `decode <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/decode>`_: + Testsuite for :ref:`decodetree <decodetree>` implementation. + - `docker <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/docker>`_: + Code and scripts to create `containers <container-ref>` used in `CI <ci>`. + - `fp <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/fp>`_: + QEMU testsuite for soft float implementation. + - `functional <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/functional>`_: + `Functional tests <checkfunctional-ref>` (full VM boot). + - `lcitool <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/lcitool>`_: + Generate dockerfiles for CI containers. + - `migration <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/migration>`_: + Test scripts and data for `Migration framework <migration>`. + - `multiboot <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/multiboot>`_: + Test multiboot functionality for x86_64/i386. + - `qapi-schema <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/qapi-schema>`_: + Test scripts and data for `QAPI <qapi-tests>`. + - `qemu-iotests <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/qemu-iotests>`_: + `Disk image and block tests <qemu-iotests>`. + - `qtest <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/qtest>`_: + `Device emulation testing <qtest>`. + - `tcg <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/tcg>`__: + `TCG related tests <checktcg-ref>`. Contains code per architecture + (subfolder) and multiarch tests as well. + - `tsan <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/tsan>`_: + `Suppressions <tsan-suppressions>` for thread sanitizer. + - `uefi-test-tools <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/uefi-test-tools>`_: + Test tool for UEFI support. + - `unit <https://gitlab.com/qemu-project/qemu/-/tree/master/tests/unit>`_: + QEMU `Unit tests <unit-tests>`. +* `trace <https://gitlab.com/qemu-project/qemu/-/tree/master/trace>`_: + :ref:`Tracing framework <tracing>`. Used to print information associated to various + events during execution. +* `ui <https://gitlab.com/qemu-project/qemu/-/tree/master/ui>`_: + QEMU User interfaces. +* `util <https://gitlab.com/qemu-project/qemu/-/tree/master/util>`_: + Utility code used by other parts of QEMU. diff --git a/docs/devel/control-flow-integrity.rst b/docs/devel/control-flow-integrity.rst index e6b73a4fe1..3d5702fa4c 100644 --- a/docs/devel/control-flow-integrity.rst +++ b/docs/devel/control-flow-integrity.rst @@ -1,3 +1,5 @@ +.. _cfi: + ============================ Control-Flow Integrity (CFI) ============================ diff --git a/docs/devel/decodetree.rst b/docs/devel/decodetree.rst index e3392aa705..98ad33a487 100644 --- a/docs/devel/decodetree.rst +++ b/docs/devel/decodetree.rst @@ -1,3 +1,5 @@ +.. _decodetree: + ======================== Decodetree Specification ======================== diff --git a/docs/devel/ebpf_rss.rst b/docs/devel/ebpf_rss.rst index 4a68682b31..ed5d33767b 100644 --- a/docs/devel/ebpf_rss.rst +++ b/docs/devel/ebpf_rss.rst @@ -1,3 +1,5 @@ +.. _ebpf-rss: + =========================== eBPF RSS virtio-net support =========================== diff --git a/docs/devel/index-build.rst b/docs/devel/index-build.rst index 0023953be3..0745c81a26 100644 --- a/docs/devel/index-build.rst +++ b/docs/devel/index-build.rst @@ -8,6 +8,7 @@ some of the basics if you are adding new files and targets to the build. :maxdepth: 3 build-system + build-environment kconfig docs qapi-code-gen diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst index ab9fbc4482..bca597c658 100644 --- a/docs/devel/index-internals.rst +++ b/docs/devel/index-internals.rst @@ -1,3 +1,5 @@ +.. _internal-subsystem: + Internal Subsystem Information ------------------------------ diff --git a/docs/devel/index-process.rst b/docs/devel/index-process.rst index 362f97ee30..cb7c6640fd 100644 --- a/docs/devel/index-process.rst +++ b/docs/devel/index-process.rst @@ -17,3 +17,4 @@ Notes about how to interact with the community and how and where to submit patch stable-process submitting-a-pull-request secure-coding-practices + rust diff --git a/docs/devel/index.rst b/docs/devel/index.rst index a53f1bfda5..29f032d6a8 100644 --- a/docs/devel/index.rst +++ b/docs/devel/index.rst @@ -35,3 +35,4 @@ the :ref:`tcg_internals`. index-api index-internals index-tcg + codebase diff --git a/docs/devel/migration/CPR.rst b/docs/devel/migration/CPR.rst index 63c36470cf..7897873c86 100644 --- a/docs/devel/migration/CPR.rst +++ b/docs/devel/migration/CPR.rst @@ -5,7 +5,7 @@ CPR is the umbrella name for a set of migration modes in which the VM is migrated to a new QEMU instance on the same host. It is intended for use when the goal is to update host software components that run the VM, such as QEMU or even the host kernel. At this time, -cpr-reboot is the only available mode. +the cpr-reboot and cpr-transfer modes are available. Because QEMU is restarted on the same host, with access to the same local devices, CPR is allowed in certain cases where normal migration @@ -53,7 +53,7 @@ RAM is copied to the migration URI. Outgoing: * Set the migration mode parameter to ``cpr-reboot``. * Set the ``x-ignore-shared`` capability if desired. - * Issue the ``migrate`` command. It is recommended the the URI be a + * Issue the ``migrate`` command. It is recommended the URI be a ``file`` type, but one can use other types such as ``exec``, provided the command captures all the data from the outgoing side, and provides all the data to the incoming side. @@ -145,3 +145,183 @@ Caveats cpr-reboot mode may not be used with postcopy, background-snapshot, or COLO. + +cpr-transfer mode +----------------- + +This mode allows the user to transfer a guest to a new QEMU instance +on the same host with minimal guest pause time, by preserving guest +RAM in place, albeit with new virtual addresses in new QEMU. Devices +and their pinned memory pages will also be preserved in a future QEMU +release. + +The user starts new QEMU on the same host as old QEMU, with command- +line arguments to create the same machine, plus the ``-incoming`` +option for the main migration channel, like normal live migration. +In addition, the user adds a second -incoming option with channel +type ``cpr``. This CPR channel must support file descriptor transfer +with SCM_RIGHTS, i.e. it must be a UNIX domain socket. + +To initiate CPR, the user issues a migrate command to old QEMU, +adding a second migration channel of type ``cpr`` in the channels +argument. Old QEMU stops the VM, saves state to the migration +channels, and enters the postmigrate state. Execution resumes in +new QEMU. + +New QEMU reads the CPR channel before opening a monitor, hence +the CPR channel cannot be specified in the list of channels for a +migrate-incoming command. It may only be specified on the command +line. + +Usage +^^^^^ + +Memory backend objects must have the ``share=on`` attribute. + +The VM must be started with the ``-machine aux-ram-share=on`` +option. This causes implicit RAM blocks (those not described by +a memory-backend object) to be allocated by mmap'ing a memfd. +Examples include VGA and ROM. + +Outgoing: + * Set the migration mode parameter to ``cpr-transfer``. + * Issue the ``migrate`` command, containing a main channel and + a cpr channel. + +Incoming: + * Start new QEMU with two ``-incoming`` options. + * If the VM was running when the outgoing ``migrate`` command was + issued, then QEMU automatically resumes VM execution. + +Caveats +^^^^^^^ + +cpr-transfer mode may not be used with postcopy, background-snapshot, +or COLO. + +memory-backend-epc is not supported. + +The main incoming migration channel address cannot be a file type. + +If the main incoming channel address is an inet socket, then the port +cannot be 0 (meaning dynamically choose a port). + +When using ``-incoming defer``, you must issue the migrate command to +old QEMU before issuing any monitor commands to new QEMU, because new +QEMU blocks waiting to read from the cpr channel before starting its +monitor, and old QEMU does not write to the channel until the migrate +command is issued. However, new QEMU does not open and read the +main migration channel until you issue the migrate incoming command. + +Example 1: incoming channel +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In these examples, we simply restart the same version of QEMU, but +in a real scenario one would start new QEMU on the incoming side. +Note that new QEMU does not print the monitor prompt until old QEMU +has issued the migrate command. The outgoing side uses QMP because +HMP cannot specify a CPR channel. Some QMP responses are omitted for +brevity. + +:: + + Outgoing: Incoming: + + # qemu-kvm -qmp stdio + -object memory-backend-file,id=ram0,size=4G, + mem-path=/dev/shm/ram0,share=on -m 4G + -machine memory-backend=ram0 + -machine aux-ram-share=on + ... + # qemu-kvm -monitor stdio + -incoming tcp:0:44444 + -incoming '{"channel-type": "cpr", + "addr": { "transport": "socket", + "type": "unix", "path": "cpr.sock"}}' + ... + {"execute":"qmp_capabilities"} + + {"execute": "query-status"} + {"return": {"status": "running", + "running": true}} + + {"execute":"migrate-set-parameters", + "arguments":{"mode":"cpr-transfer"}} + + {"execute": "migrate", "arguments": { "channels": [ + {"channel-type": "main", + "addr": { "transport": "socket", "type": "inet", + "host": "0", "port": "44444" }}, + {"channel-type": "cpr", + "addr": { "transport": "socket", "type": "unix", + "path": "cpr.sock" }}]}} + + QEMU 10.0.50 monitor + (qemu) info status + VM status: running + + {"execute": "query-status"} + {"return": {"status": "postmigrate", + "running": false}} + +Example 2: incoming defer +^^^^^^^^^^^^^^^^^^^^^^^^^ + +This example uses ``-incoming defer`` to hot plug a device before +accepting the main migration channel. Again note you must issue the +migrate command to old QEMU before you can issue any monitor +commands to new QEMU. + + +:: + + Outgoing: Incoming: + + # qemu-kvm -monitor stdio + -object memory-backend-file,id=ram0,size=4G, + mem-path=/dev/shm/ram0,share=on -m 4G + -machine memory-backend=ram0 + -machine aux-ram-share=on + ... + # qemu-kvm -monitor stdio + -incoming defer + -incoming '{"channel-type": "cpr", + "addr": { "transport": "socket", + "type": "unix", "path": "cpr.sock"}}' + ... + {"execute":"qmp_capabilities"} + + {"execute": "device_add", + "arguments": {"driver": "pcie-root-port"}} + + {"execute":"migrate-set-parameters", + "arguments":{"mode":"cpr-transfer"}} + + {"execute": "migrate", "arguments": { "channels": [ + {"channel-type": "main", + "addr": { "transport": "socket", "type": "inet", + "host": "0", "port": "44444" }}, + {"channel-type": "cpr", + "addr": { "transport": "socket", "type": "unix", + "path": "cpr.sock" }}]}} + + QEMU 10.0.50 monitor + (qemu) info status + VM status: paused (inmigrate) + (qemu) device_add pcie-root-port + (qemu) migrate_incoming tcp:0:44444 + (qemu) info status + VM status: running + + {"execute": "query-status"} + {"return": {"status": "postmigrate", + "running": false}} + +Futures +^^^^^^^ + +cpr-transfer mode is based on a capability to transfer open file +descriptors from old to new QEMU. In the future, descriptors for +vfio, iommufd, vhost, and char devices could be transferred, +preserving those devices and their kernel state without interruption, +even if they do not explicitly support live migration. diff --git a/docs/devel/migration/main.rst b/docs/devel/migration/main.rst index c2857fc244..cdd4f4a6d7 100644 --- a/docs/devel/migration/main.rst +++ b/docs/devel/migration/main.rst @@ -1,3 +1,5 @@ +.. _migration: + =================== Migration framework =================== diff --git a/docs/devel/multi-thread-tcg.rst b/docs/devel/multi-thread-tcg.rst index d706c27ea7..b0f473961d 100644 --- a/docs/devel/multi-thread-tcg.rst +++ b/docs/devel/multi-thread-tcg.rst @@ -4,6 +4,8 @@ This work is licensed under the terms of the GNU GPL, version 2 or later. See the COPYING file in the top-level directory. +.. _mttcg: + ================== Multi-threaded TCG ================== @@ -35,7 +37,6 @@ if: * forced by --accel tcg,thread=single * enabling --icount mode -* 64 bit guests on 32 bit hosts (TCG_OVERSIZED_GUEST) In the general case of running translated code there should be no inter-vCPU dependencies and all vCPUs should be able to run at full diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst index 583207a8ec..9fa94251b0 100644 --- a/docs/devel/qapi-code-gen.rst +++ b/docs/devel/qapi-code-gen.rst @@ -9,6 +9,7 @@ How to use the QAPI code generator This work is licensed under the terms of the GNU GPL, version 2 or later. See the COPYING file in the top-level directory. +.. _qapi: Introduction ============ @@ -1854,7 +1855,7 @@ Example:: #ifndef EXAMPLE_QAPI_INIT_COMMANDS_H #define EXAMPLE_QAPI_INIT_COMMANDS_H - #include "qapi/qmp/dispatch.h" + #include "qapi/qmp-registry.h" void example_qmp_init_marshal(QmpCommandList *cmds); @@ -1985,7 +1986,7 @@ Example:: #ifndef EXAMPLE_QAPI_INTROSPECT_H #define EXAMPLE_QAPI_INTROSPECT_H - #include "qapi/qmp/qlit.h" + #include "qobject/qlit.h" extern const QLitObject example_qmp_schema_qlit; diff --git a/docs/devel/rust.rst b/docs/devel/rust.rst new file mode 100644 index 0000000000..90958e5a30 --- /dev/null +++ b/docs/devel/rust.rst @@ -0,0 +1,476 @@ +.. |msrv| replace:: 1.63.0 + +Rust in QEMU +============ + +Rust in QEMU is a project to enable using the Rust programming language +to add new functionality to QEMU. + +Right now, the focus is on making it possible to write devices that inherit +from ``SysBusDevice`` in `*safe*`__ Rust. Later, it may become possible +to write other kinds of devices (e.g. PCI devices that can do DMA), +complete boards, or backends (e.g. block device formats). + +__ https://doc.rust-lang.org/nomicon/meet-safe-and-unsafe.html + +Building the Rust in QEMU code +------------------------------ + +The Rust in QEMU code is included in the emulators via Meson. Meson +invokes rustc directly, building static libraries that are then linked +together with the C code. This is completely automatic when you run +``make`` or ``ninja``. + +However, QEMU's build system also tries to be easy to use for people who +are accustomed to the more "normal" Cargo-based development workflow. +In particular: + +* the set of warnings and lints that are used to build QEMU always + comes from the ``rust/Cargo.toml`` workspace file + +* it is also possible to use ``cargo`` for common Rust-specific coding + tasks, in particular to invoke ``clippy``, ``rustfmt`` and ``rustdoc``. + +To this end, QEMU includes a ``build.rs`` build script that picks up +generated sources from QEMU's build directory and puts it in Cargo's +output directory (typically ``rust/target/``). A vanilla invocation +of Cargo will complain that it cannot find the generated sources, +which can be fixed in different ways: + +* by using special shorthand targets in the QEMU build directory:: + + make clippy + make rustfmt + make rustdoc + +* by invoking ``cargo`` through the Meson `development environment`__ + feature:: + + pyvenv/bin/meson devenv -w ../rust cargo clippy --tests + pyvenv/bin/meson devenv -w ../rust cargo fmt + + If you are going to use ``cargo`` repeatedly, ``pyvenv/bin/meson devenv`` + will enter a shell where commands like ``cargo clippy`` just work. + +__ https://mesonbuild.com/Commands.html#devenv + +* by pointing the ``MESON_BUILD_ROOT`` to the top of your QEMU build + tree. This third method is useful if you are using ``rust-analyzer``; + you can set the environment variable through the + ``rust-analyzer.cargo.extraEnv`` setting. + +As shown above, you can use the ``--tests`` option as usual to operate on test +code. Note however that you cannot *build* or run tests via ``cargo``, because +they need support C code from QEMU that Cargo does not know about. Tests can +be run via ``meson test`` or ``make``:: + + make check-rust + +Building Rust code with ``--enable-modules`` is not supported yet. + +Supported tools +''''''''''''''' + +QEMU supports rustc version 1.63.0 and newer. Notably, the following features +are missing: + +* ``core::ffi`` (1.64.0). Use ``std::os::raw`` and ``std::ffi`` instead. + +* ``cast_mut()``/``cast_const()`` (1.65.0). Use ``as`` instead. + +* "let ... else" (1.65.0). Use ``if let`` instead. This is currently patched + in QEMU's vendored copy of the bilge crate. + +* Generic Associated Types (1.65.0) + +* ``CStr::from_bytes_with_nul()`` as a ``const`` function (1.72.0). + +* "Return position ``impl Trait`` in Traits" (1.75.0, blocker for including + the pinned-init create). + +* ``MaybeUninit::zeroed()`` as a ``const`` function (1.75.0). QEMU's + ``Zeroable`` trait can be implemented without ``MaybeUninit::zeroed()``, + so this would be just a cleanup. + +* ``c"" literals`` (stable in 1.77.0). QEMU provides a ``c_str!()`` macro + to define ``CStr`` constants easily + +* ``offset_of!`` (stable in 1.77.0). QEMU uses ``offset_of!()`` heavily; it + provides a replacement in the ``qemu_api`` crate, but it does not support + lifetime parameters and therefore ``&'a Something`` fields in the struct + may have to be replaced by ``NonNull<Something>``. *Nested* ``offset_of!`` + was only stabilized in Rust 1.82.0, but it is not used. + +* inline const expression (stable in 1.79.0), currently worked around with + associated constants in the ``FnCall`` trait. + +* associated constants have to be explicitly marked ``'static`` (`changed in + 1.81.0`__) + +* ``&raw`` (stable in 1.82.0). Use ``addr_of!`` and ``addr_of_mut!`` instead, + though hopefully the need for raw pointers will go down over time. + +* ``new_uninit`` (stable in 1.82.0). This is used internally by the ``pinned_init`` + crate, which is planned for inclusion in QEMU, but it can be easily patched + out. + +* referencing statics in constants (stable in 1.83.0). For now use a const + function; this is an important limitation for QEMU's migration stream + architecture (VMState). Right now, VMState lacks type safety because + it is hard to place the ``VMStateField`` definitions in traits. + +* associated const equality would be nice to have for some users of + ``callbacks::FnCall``, but is still experimental. ``ASSERT_IS_SOME`` + replaces it. + +__ https://github.com/rust-lang/rust/pull/125258 + +It is expected that QEMU will advance its minimum supported version of +rustc to 1.77.0 as soon as possible; as of January 2025, blockers +for that right now are Debian bookworm and 32-bit MIPS processors. +This unfortunately means that references to statics in constants will +remain an issue. + +QEMU also supports version 0.60.x of bindgen, which is missing option +``--generate-cstr``. This option requires version 0.66.x and will +be adopted as soon as supporting these older versions is not necessary +anymore. + +Writing Rust code in QEMU +------------------------- + +Right now QEMU includes three crates: + +* ``qemu_api`` for bindings to C code and useful functionality + +* ``qemu_api_macros`` defines several procedural macros that are useful when + writing C code + +* ``pl011`` (under ``rust/hw/char/pl011``) is the sample device that is being + used to further develop ``qemu_api`` and ``qemu_api_macros``. It is a functional + replacement for the ``hw/char/pl011.c`` file. + +This section explains how to work with them. + +Status +'''''' + +Modules of ``qemu_api`` can be defined as: + +- *complete*: ready for use in new devices; if applicable, the API supports the + full functionality available in C + +- *stable*: ready for production use, the API is safe and should not undergo + major changes + +- *proof of concept*: the API is subject to change but allows working with safe + Rust + +- *initial*: the API is in its initial stages; it requires large amount of + unsafe code; it might have soundness or type-safety issues + +The status of the modules is as follows: + +================ ====================== +module status +================ ====================== +``assertions`` stable +``bitops`` complete +``callbacks`` complete +``cell`` stable +``c_str`` complete +``irq`` complete +``memory`` stable +``module`` complete +``offset_of`` stable +``qdev`` stable +``qom`` stable +``sysbus`` stable +``timer`` stable +``vmstate`` proof of concept +``zeroable`` stable +================ ====================== + +.. note:: + API stability is not a promise, if anything because the C APIs are not a stable + interface either. Also, ``unsafe`` interfaces may be replaced by safe interfaces + later. + +Naming convention +''''''''''''''''' + +C function names usually are prefixed according to the data type that they +apply to, for example ``timer_mod`` or ``sysbus_connect_irq``. Furthermore, +both function and structs sometimes have a ``qemu_`` or ``QEMU`` prefix. +Generally speaking, these are all removed in the corresponding Rust functions: +``QEMUTimer`` becomes ``timer::Timer``, ``timer_mod`` becomes ``Timer::modify``, +``sysbus_connect_irq`` becomes ``SysBusDeviceMethods::connect_irq``. + +Sometimes however a name appears multiple times in the QOM class hierarchy, +and the only difference is in the prefix. An example is ``qdev_realize`` and +``sysbus_realize``. In such cases, whenever a name is not unique in +the hierarchy, always add the prefix to the classes that are lower in +the hierarchy; for the top class, decide on a case by case basis. + +For example: + +========================== ========================================= +``device_cold_reset()`` ``DeviceMethods::cold_reset()`` +``pci_device_reset()`` ``PciDeviceMethods::pci_device_reset()`` +``pci_bridge_reset()`` ``PciBridgeMethods::pci_bridge_reset()`` +========================== ========================================= + +Here, the name is not exactly the same, but nevertheless ``PciDeviceMethods`` +adds the prefix to avoid confusion, because the functionality of +``device_cold_reset()`` and ``pci_device_reset()`` is subtly different. + +In this case, however, no prefix is needed: + +========================== ========================================= +``device_realize()`` ``DeviceMethods::realize()`` +``sysbus_realize()`` ``SysbusDeviceMethods::sysbus_realize()`` +``pci_realize()`` ``PciDeviceMethods::pci_realize()`` +========================== ========================================= + +Here, the lower classes do not add any functionality, and mostly +provide extra compile-time checking; the basic *realize* functionality +is the same for all devices. Therefore, ``DeviceMethods`` does not +add the prefix. + +Whenever a name is unique in the hierarchy, instead, you should +always remove the class name prefix. + +Common pitfalls +''''''''''''''' + +Rust has very strict rules with respect to how you get an exclusive (``&mut``) +reference; failure to respect those rules is a source of undefined behavior. +In particular, even if a value is loaded from a raw mutable pointer (``*mut``), +it *cannot* be casted to ``&mut`` unless the value was stored to the ``*mut`` +from a mutable reference. Furthermore, it is undefined behavior if any +shared reference was created between the store to the ``*mut`` and the load:: + + let mut p: u32 = 42; + let p_mut = &mut p; // 1 + let p_raw = p_mut as *mut u32; // 2 + + // p_raw keeps the mutable reference "alive" + + let p_shared = &p; // 3 + println!("access from &u32: {}", *p_shared); + + // Bring back the mutable reference, its lifetime overlaps + // with that of a shared reference. + let p_mut = unsafe { &mut *p_raw }; // 4 + println!("access from &mut 32: {}", *p_mut); + + println!("access from &u32: {}", *p_shared); // 5 + +These rules can be tested with `MIRI`__, for example. + +__ https://github.com/rust-lang/miri + +Almost all Rust code in QEMU will involve QOM objects, and pointers to these +objects are *shared*, for example because they are part of the QOM composition +tree. This creates exactly the above scenario: + +1. a QOM object is created + +2. a ``*mut`` is created, for example as the opaque value for a ``MemoryRegion`` + +3. the QOM object is placed in the composition tree + +4. a memory access dereferences the opaque value to a ``&mut`` + +5. but the shared reference is still present in the composition tree + +Because of this, QOM objects should almost always use ``&self`` instead +of ``&mut self``; access to internal fields must use *interior mutability* +to go from a shared reference to a ``&mut``. + +Whenever C code provides you with an opaque ``void *``, avoid converting it +to a Rust mutable reference, and use a shared reference instead. Rust code +will then have to use QEMU's ``BqlRefCell`` and ``BqlCell`` type, which +enforce that locking rules for the "Big QEMU Lock" are respected. These cell +types are also known to the ``vmstate`` crate, which is able to "look inside" +them when building an in-memory representation of a ``struct``s layout. +Note that the same is not true of a ``RefCell`` or ``Mutex``. + +In the future, similar cell types might also be provided for ``AioContext``-based +locking as well. + +Writing bindings to C code +'''''''''''''''''''''''''' + +Here are some things to keep in mind when working on the ``qemu_api`` crate. + +**Look at existing code** + Very often, similar idioms in C code correspond to similar tricks in + Rust bindings. If the C code uses ``offsetof``, look at qdev properties + or ``vmstate``. If the C code has a complex const struct, look at + ``MemoryRegion``. Reuse existing patterns for handling lifetimes; + for example use ``&T`` for QOM objects that do not need a reference + count (including those that can be embedded in other objects) and + ``Owned<T>`` for those that need it. + +**Use the type system** + Bindings often will need access information that is specific to a type + (either a builtin one or a user-defined one) in order to pass it to C + functions. Put them in a trait and access it through generic parameters. + The ``vmstate`` module has examples of how to retrieve type information + for the fields of a Rust ``struct``. + +**Prefer unsafe traits to unsafe functions** + Unsafe traits are much easier to prove correct than unsafe functions. + They are an excellent place to store metadata that can later be accessed + by generic functions. C code usually places metadata in global variables; + in Rust, they can be stored in traits and then turned into ``static`` + variables. Often, unsafe traits can be generated by procedural macros. + +**Document limitations due to old Rust versions** + If you need to settle for an inferior solution because of the currently + supported set of Rust versions, document it in the source and in this + file. This ensures that it can be fixed when the minimum supported + version is bumped. + +**Keep locking in mind**. + When marking a type ``Sync``, be careful of whether it needs the big + QEMU lock. Use ``BqlCell`` and ``BqlRefCell`` for interior data, + or assert ``bql_locked()``. + +**Don't be afraid of complexity, but document and isolate it** + It's okay to be tricky; device code is written more often than bindings + code and it's important that it is idiomatic. However, you should strive + to isolate any tricks in a place (for example a ``struct``, a trait + or a macro) where it can be documented and tested. If needed, include + toy versions of the code in the documentation. + +Writing procedural macros +''''''''''''''''''''''''' + +By conventions, procedural macros are split in two functions, one +returning ``Result<proc_macro2::TokenStream, MacroError>` with the body of +the procedural macro, and the second returning ``proc_macro::TokenStream`` +which is the actual procedural macro. The former's name is the same as +the latter with the ``_or_error`` suffix. The code for the latter is more +or less fixed; it follows the following template, which is fixed apart +from the type after ``as`` in the invocation of ``parse_macro_input!``:: + + #[proc_macro_derive(Object)] + pub fn derive_object(input: TokenStream) -> TokenStream { + let input = parse_macro_input!(input as DeriveInput); + let expanded = derive_object_or_error(input).unwrap_or_else(Into::into); + + TokenStream::from(expanded) + } + +The ``qemu_api_macros`` crate has utility functions to examine a +``DeriveInput`` and perform common checks (e.g. looking for a struct +with named fields). These functions return ``Result<..., MacroError>`` +and can be used easily in the procedural macro function:: + + fn derive_object_or_error(input: DeriveInput) -> + Result<proc_macro2::TokenStream, MacroError> + { + is_c_repr(&input, "#[derive(Object)]")?; + + let name = &input.ident; + let parent = &get_fields(&input, "#[derive(Object)]")?[0].ident; + ... + } + +Use procedural macros with care. They are mostly useful for two purposes: + +* Performing consistency checks; for example ``#[derive(Object)]`` checks + that the structure has ``#[repr[C])`` and that the type of the first field + is consistent with the ``ObjectType`` declaration. + +* Extracting information from Rust source code into traits, typically based + on types and attributes. For example, ``#[derive(TryInto)]`` builds an + implementation of ``TryFrom``, and it uses the ``#[repr(...)]`` attribute + as the ``TryFrom`` source and error types. + +Procedural macros can be hard to debug and test; if the code generation +exceeds a few lines of code, it may be worthwhile to delegate work to +"regular" declarative (``macro_rules!``) macros and write unit tests for +those instead. + + +Coding style +'''''''''''' + +Code should pass clippy and be formatted with rustfmt. + +Right now, only the nightly version of ``rustfmt`` is supported. This +might change in the future. While CI checks for correct formatting via +``cargo fmt --check``, maintainers can fix this for you when applying patches. + +It is expected that ``qemu_api`` provides full ``rustdoc`` documentation for +bindings that are in their final shape or close. + +Adding dependencies +------------------- + +Generally, the set of dependent crates is kept small. Think twice before +adding a new external crate, especially if it comes with a large set of +dependencies itself. Sometimes QEMU only needs a small subset of the +functionality; see for example QEMU's ``assertions`` or ``c_str`` modules. + +On top of this recommendation, adding external crates to QEMU is a +slightly complicated process, mostly due to the need to teach Meson how +to build them. While Meson has initial support for parsing ``Cargo.lock`` +files, it is still highly experimental and is therefore not used. + +Therefore, external crates must be added as subprojects for Meson to +learn how to build them, as well as to the relevant ``Cargo.toml`` files. +The versions specified in ``rust/Cargo.lock`` must be the same as the +subprojects; note that the ``rust/`` directory forms a Cargo `workspace`__, +and therefore there is a single lock file for the whole build. + +__ https://doc.rust-lang.org/cargo/reference/workspaces.html#virtual-workspace + +Choose a version of the crate that works with QEMU's minimum supported +Rust version (|msrv|). + +Second, a new ``wrap`` file must be added to teach Meson how to download the +crate. The wrap file must be named ``NAME-SEMVER-rs.wrap``, where ``NAME`` +is the name of the crate and ``SEMVER`` is the version up to and including the +first non-zero number. For example, a crate with version ``0.2.3`` will use +``0.2`` for its ``SEMVER``, while a crate with version ``1.0.84`` will use ``1``. + +Third, the Meson rules to build the crate must be added at +``subprojects/NAME-SEMVER-rs/meson.build``. Generally this includes: + +* ``subproject`` and ``dependency`` lines for all dependent crates + +* a ``static_library`` or ``rust.proc_macro`` line to perform the actual build + +* ``declare_dependency`` and a ``meson.override_dependency`` lines to expose + the result to QEMU and to other subprojects + +Remember to add ``native: true`` to ``dependency``, ``static_library`` and +``meson.override_dependency`` for dependencies of procedural macros. +If a crate is needed in both procedural macros and QEMU binaries, everything +apart from ``subproject`` must be duplicated to build both native and +non-native versions of the crate. + +It's important to specify the right compiler options. These include: + +* the language edition (which can be found in the ``Cargo.toml`` file) + +* the ``--cfg`` (which have to be "reverse engineered" from the ``build.rs`` + file of the crate). + +* usually, a ``--cap-lints allow`` argument to hide warnings from rustc + or clippy. + +After every change to the ``meson.build`` file you have to update the patched +version with ``meson subprojects update --reset ``NAME-SEMVER-rs``. This might +be automated in the future. + +Also, after every change to the ``meson.build`` file it is strongly suggested to +do a dummy change to the ``.wrap`` file (for example adding a comment like +``# version 2``), which will help Meson notice that the subproject is out of date. + +As a last step, add the new subproject to ``scripts/archive-source.sh``, +``scripts/make-release`` and ``subprojects/.gitignore``. diff --git a/docs/devel/style.rst b/docs/devel/style.rst index 2f68b50079..d025933808 100644 --- a/docs/devel/style.rst +++ b/docs/devel/style.rst @@ -416,6 +416,26 @@ definitions instead of typedefs in headers and function prototypes; this avoids problems with duplicated typedefs and reduces the need to include headers from other headers. +Bitfields +--------- + +C bitfields can be a cause of non-portability issues, especially under windows +where `MSVC has a different way to lay them out than GCC +<https://gcc.gnu.org/onlinedocs/gcc/x86-Type-Attributes.html>`_, or where +endianness matters. + +For this reason, we disallow usage of bitfields in packed structures and in any +structures which are supposed to exactly match a specific layout in guest +memory. Some existing code may use it, and we carefully ensured the layout was +the one expected. + +We also suggest avoiding bitfields even in structures where the exact +layout does not matter, unless you can show that they provide a significant +usability benefit. + +We encourage the usage of ``include/hw/registerfields.h`` as a safe replacement +for bitfields. + Reserved namespaces in C and POSIX ---------------------------------- diff --git a/docs/devel/submitting-a-patch.rst b/docs/devel/submitting-a-patch.rst index 03b2ac298a..65c64078cb 100644 --- a/docs/devel/submitting-a-patch.rst +++ b/docs/devel/submitting-a-patch.rst @@ -235,6 +235,31 @@ to another list.) ``git send-email`` (`step-by-step setup guide works best for delivering the patch without mangling it, but attachments can be used as a last resort on a first-time submission. +.. _use_git_publish: + +Use git-publish +~~~~~~~~~~~~~~~ + +If you already configured git send-email, you can simply use `git-publish +<https://github.com/stefanha/git-publish>`__ to send series. + +:: + + $ git checkout master -b my-feature + $ # work on new commits, add your 'Signed-off-by' lines to each + $ git publish + $ ... more work, rebase on master, ... + $ git publish # will send a v2 + +Each time you post a series, git-publish will create a local tag with the format +``<branchname>-v<version>`` to record the patch series. + +When sending patch emails, 'git publish' will consult the output of +'scripts/get_maintainers.pl' and automatically CC anyone listed as maintainers +of the affected code. Generally you should accept the suggested CC list, but +there may sometimes be scenarios where it is appropriate to cut it down (eg on +certain large tree-wide cleanups), or augment it with other interested people. + .. _if_you_cannot_send_patch_emails: If you cannot send patch emails @@ -408,6 +433,20 @@ For more details on how QEMU's stable process works, refer to the .. _participating_in_code_review: +Retrieve an existing series +--------------------------- + +If you want to apply an existing series on top of your tree, you can simply use +`b4 <https://github.com/mricon/b4>`__. + +:: + + b4 shazam $msg-id + +The message id is related to the patch series that has been sent to the mailing +list. You need to retrieve the "Message-Id:" header from one of the patches. Any +of them can be used and b4 will apply the whole series. + Participating in Code Review ---------------------------- diff --git a/docs/devel/tcg-ops.rst b/docs/devel/tcg-ops.rst index d46b625e0e..688984fd39 100644 --- a/docs/devel/tcg-ops.rst +++ b/docs/devel/tcg-ops.rst @@ -712,10 +712,9 @@ QEMU specific operations Host vector operations ---------------------- -All of the vector ops have two parameters, ``TCGOP_VECL`` & ``TCGOP_VECE``. -The former specifies the length of the vector in log2 64-bit units; the -latter specifies the length of the element (if applicable) in log2 8-bit units. -E.g. VECL = 1 -> 64 << 1 -> v128, and VECE = 2 -> 1 << 2 -> i32. +All of the vector ops have two parameters, ``TCGOP_TYPE`` & ``TCGOP_VECE``. +The former specifies the length of the vector as a TCGType; the latter +specifies the length of the element (if applicable) in log2 8-bit units. .. list-table:: @@ -729,7 +728,7 @@ E.g. VECL = 1 -> 64 << 1 -> v128, and VECE = 2 -> 1 << 2 -> i32. * - dup_vec *v0*, *r1* - - | Duplicate the low N bits of *r1* into VECL/VECE copies across *v0*. + - | Duplicate the low N bits of *r1* into TYPE/VECE copies across *v0*. * - dupi_vec *v0*, *c* @@ -738,7 +737,7 @@ E.g. VECL = 1 -> 64 << 1 -> v128, and VECE = 2 -> 1 << 2 -> i32. * - dup2_vec *v0*, *r1*, *r2* - - | Duplicate *r2*:*r1* into VECL/64 copies across *v0*. This opcode is + - | Duplicate *r2*:*r1* into TYPE/64 copies across *v0*. This opcode is only present for 32-bit hosts. * - add_vec *v0*, *v1*, *v2* @@ -810,7 +809,7 @@ E.g. VECL = 1 -> 64 << 1 -> v128, and VECE = 2 -> 1 << 2 -> i32. .. code-block:: c - for (i = 0; i < VECL/VECE; ++i) { + for (i = 0; i < TYPE/VECE; ++i) { v0[i] = v1[i] << s2; } @@ -832,7 +831,7 @@ E.g. VECL = 1 -> 64 << 1 -> v128, and VECE = 2 -> 1 << 2 -> i32. .. code-block:: c - for (i = 0; i < VECL/VECE; ++i) { + for (i = 0; i < TYPE/VECE; ++i) { v0[i] = v1[i] << v2[i]; } @@ -928,7 +927,9 @@ operation uses a constant input constraint which does not allow all constants, it must also accept registers in order to have a fallback. The constraint '``i``' is defined generically to accept any constant. The constraint '``r``' is not defined generically, but is consistently -used by each backend to indicate all registers. +used by each backend to indicate all registers. If ``TCG_REG_ZERO`` +is defined by the backend, the constraint '``z``' is defined generically +to map constant 0 to the hardware zero register. The movi_i32 and movi_i64 operations must accept any constants. diff --git a/docs/devel/testing/functional.rst b/docs/devel/testing/functional.rst index ae238ed3fc..ecc738922b 100644 --- a/docs/devel/testing/functional.rst +++ b/docs/devel/testing/functional.rst @@ -351,5 +351,13 @@ the code snippet below: Tests should not live in this state forever and should either be fixed or eventually removed. +QEMU_TEST_ALLOW_SLOW +^^^^^^^^^^^^^^^^^^^^ +Tests that have a very long runtime and might run into timeout issues +e.g. if the QEMU binary has been compiled with debugging options enabled. +To avoid these timeout issues by default and to save some precious CPU +cycles during normal testing, such tests are disabled by default unless +the QEMU_TEST_ALLOW_SLOW environment variable has been set. + .. _unittest: https://docs.python.org/3/library/unittest.html diff --git a/docs/devel/testing/main.rst b/docs/devel/testing/main.rst index 91f4dc61fb..9869bcf034 100644 --- a/docs/devel/testing/main.rst +++ b/docs/devel/testing/main.rst @@ -39,6 +39,8 @@ Before running tests, it is best to build QEMU programs first. Some tests expect the executables to exist and will fail with obscure messages if they cannot find them. +.. _unit-tests: + Unit tests ~~~~~~~~~~ @@ -126,6 +128,8 @@ successfully on various hosts. The following list shows some best practices: #ifdef in the codes. If the whole test suite cannot run on Windows, disable the build in the meson.build file. +.. _qapi-tests: + QAPI schema tests ~~~~~~~~~~~~~~~~~ @@ -160,6 +164,8 @@ check-block are in the "auto" group). See the "QEMU iotests" section below for more information. +.. _qemu-iotests: + QEMU iotests ------------ @@ -679,6 +685,8 @@ The above exitcode=0 has TSan continue without error if any warnings are found. This allows for running the test and then checking the warnings afterwards. If you want TSan to stop and exit with error on warnings, use exitcode=66. +.. _tsan-suppressions: + TSan Suppressions ~~~~~~~~~~~~~~~~~ Keep in mind that for any data race warning, although there might be a data race @@ -901,7 +909,6 @@ You can run the avocado tests simply by executing: See :ref:`checkavocado-ref` for more details. - .. _checktcg-ref: Testing with "make check-tcg" diff --git a/docs/devel/testing/qtest.rst b/docs/devel/testing/qtest.rst index c5b8546b3e..73ef7702b7 100644 --- a/docs/devel/testing/qtest.rst +++ b/docs/devel/testing/qtest.rst @@ -1,3 +1,5 @@ +.. _qtest: + ======================================== QTest Device Emulation Testing Framework ======================================== |