diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/devel/clocks.rst | 71 | ||||
| -rw-r--r-- | docs/devel/index.rst | 1 | ||||
| -rw-r--r-- | docs/devel/qgraph.rst | 568 | ||||
| -rw-r--r-- | docs/devel/qtest.rst | 8 | ||||
| -rw-r--r-- | docs/generic-loader.txt | 92 | ||||
| -rw-r--r-- | docs/interop/parallels.txt | 28 | ||||
| -rw-r--r-- | docs/system/arm/aspeed.rst | 16 | ||||
| -rw-r--r-- | docs/system/arm/mps2.rst | 6 | ||||
| -rw-r--r-- | docs/system/deprecated.rst | 16 | ||||
| -rw-r--r-- | docs/system/generic-loader.rst | 117 | ||||
| -rw-r--r-- | docs/system/guest-loader.rst | 54 | ||||
| -rw-r--r-- | docs/system/index.rst | 2 | ||||
| -rw-r--r-- | docs/system/quickstart.rst | 8 | ||||
| -rw-r--r-- | docs/system/removed-features.rst | 14 | ||||
| -rw-r--r-- | docs/system/targets.rst | 2 | ||||
| -rw-r--r-- | docs/tools/qemu-storage-daemon.rst | 68 |
16 files changed, 936 insertions, 135 deletions
diff --git a/docs/devel/clocks.rst b/docs/devel/clocks.rst index c54bbb8240..956bd147ea 100644 --- a/docs/devel/clocks.rst +++ b/docs/devel/clocks.rst @@ -80,11 +80,12 @@ Adding clocks to a device must be done during the init method of the Device instance. To add an input clock to a device, the function ``qdev_init_clock_in()`` -must be used. It takes the name, a callback and an opaque parameter -for the callback (this will be explained in a following section). +must be used. It takes the name, a callback, an opaque parameter +for the callback and a mask of events when the callback should be +called (this will be explained in a following section). Output is simpler; only the name is required. Typically:: - qdev_init_clock_in(DEVICE(dev), "clk_in", clk_in_callback, dev); + qdev_init_clock_in(DEVICE(dev), "clk_in", clk_in_callback, dev, ClockUpdate); qdev_init_clock_out(DEVICE(dev), "clk_out"); Both functions return the created Clock pointer, which should be saved in the @@ -113,7 +114,7 @@ output. * callback for the input clock (see "Callback on input clock * change" section below for more information). */ - static void clk_in_callback(void *opaque); + static void clk_in_callback(void *opaque, ClockEvent event); /* * static array describing clocks: @@ -124,7 +125,7 @@ output. * the clk_out field of a MyDeviceState structure. */ static const ClockPortInitArray mydev_clocks = { - QDEV_CLOCK_IN(MyDeviceState, clk_in, clk_in_callback), + QDEV_CLOCK_IN(MyDeviceState, clk_in, clk_in_callback, ClockUpdate), QDEV_CLOCK_OUT(MyDeviceState, clk_out), QDEV_CLOCK_END }; @@ -153,6 +154,47 @@ nothing else to do. This value will be propagated to other clocks when connecting the clocks together and devices will fetch the right value during the first reset. +Clock callbacks +--------------- + +You can give a clock a callback function in several ways: + + * by passing it as an argument to ``qdev_init_clock_in()`` + * as an argument to the ``QDEV_CLOCK_IN()`` macro initializing an + array to be passed to ``qdev_init_clocks()`` + * by directly calling the ``clock_set_callback()`` function + +The callback function must be of this type: + +.. code-block:: c + + typedef void ClockCallback(void *opaque, ClockEvent event); + +The ``opaque`` argument is the pointer passed to ``qdev_init_clock_in()`` +or ``clock_set_callback()``; for ``qdev_init_clocks()`` it is the +``dev`` device pointer. + +The ``event`` argument specifies why the callback has been called. +When you register the callback you specify a mask of ClockEvent values +that you are interested in. The callback will only be called for those +events. + +The events currently supported are: + + * ``ClockPreUpdate`` : called when the input clock's period is about to + update. This is useful if the device needs to do some action for + which it needs to know the old value of the clock period. During + this callback, Clock API functions like ``clock_get()`` or + ``clock_ticks_to_ns()`` will use the old period. + * ``ClockUpdate`` : called after the input clock's period has changed. + During this callback, Clock API functions like ``clock_ticks_to_ns()`` + will use the new period. + +Note that a clock only has one callback: it is not possible to register +different functions for different events. You must register a single +callback which listens for all of the events you are interested in, +and use the ``event`` argument to identify which event has happened. + Retrieving clocks from a device ------------------------------- @@ -231,7 +273,7 @@ object during device instance init. For example: .. code-block:: c clk = qdev_init_clock_in(DEVICE(dev), "clk-in", clk_in_callback, - dev); + dev, ClockUpdate); /* set initial value to 10ns / 100MHz */ clock_set_ns(clk, 10); @@ -267,11 +309,12 @@ next lowest integer. This implies some inaccuracy due to the rounding, so be cautious about using it in calculations. It is also possible to register a callback on clock frequency changes. -Here is an example: +Here is an example, which assumes that ``clock_callback`` has been +specified as the callback for the ``ClockUpdate`` event: .. code-block:: c - void clock_callback(void *opaque) { + void clock_callback(void *opaque, ClockEvent event) { MyDeviceState *s = (MyDeviceState *) opaque; /* * 'opaque' is the argument passed to qdev_init_clock_in(); @@ -317,6 +360,18 @@ rather than simply passing it to a QEMUTimer function like ``timer_mod_ns()`` then you should be careful to avoid overflow in those calculations, of course.) +Obtaining tick counts +--------------------- + +For calculations where you need to know the number of ticks in +a given duration, use ``clock_ns_to_ticks()``. This function handles +possible non-whole-number-of-nanoseconds periods and avoids +potential rounding errors. It will return '0' if the clock is stopped +(i.e. it has period zero). If the inputs imply a tick count that +overflows a 64-bit value (a very long duration for a clock with a +very short period) the output value is truncated, so effectively +the 64-bit output wraps around. + Changing a clock period ----------------------- diff --git a/docs/devel/index.rst b/docs/devel/index.rst index ae664da00c..7c424ea6d7 100644 --- a/docs/devel/index.rst +++ b/docs/devel/index.rst @@ -12,6 +12,7 @@ Contents: .. toctree:: :maxdepth: 2 + :includehidden: build-system style diff --git a/docs/devel/qgraph.rst b/docs/devel/qgraph.rst new file mode 100644 index 0000000000..a9aff167ad --- /dev/null +++ b/docs/devel/qgraph.rst @@ -0,0 +1,568 @@ +.. _qgraph: + +======================================== +Qtest Driver Framework +======================================== + +In order to test a specific driver, plain libqos tests need to +take care of booting QEMU with the right machine and devices. +This makes each test "hardcoded" for a specific configuration, reducing +the possible coverage that it can reach. + +For example, the sdhci device is supported on both x86_64 and ARM boards, +therefore a generic sdhci test should test all machines and drivers that +support that device. +Using only libqos APIs, the test has to manually take care of +covering all the setups, and build the correct command line. + +This also introduces backward compability issues: if a device/driver command +line name is changed, all tests that use that will not work +properly anymore and need to be adjusted. + +The aim of qgraph is to create a graph of drivers, machines and tests such that +a test aimed to a certain driver does not have to care of +booting the right QEMU machine, pick the right device, build the command line +and so on. Instead, it only defines what type of device it is testing +(interface in qgraph terms) and the framework takes care of +covering all supported types of devices and machine architectures. + +Following the above example, an interface would be ``sdhci``, +so the sdhci-test should only care of linking its qgraph node with +that interface. In this way, if the command line of a sdhci driver +is changed, only the respective qgraph driver node has to be adjusted. + +The graph is composed by nodes that represent machines, drivers, tests +and edges that define the relationships between them (``CONSUMES``, ``PRODUCES``, and +``CONTAINS``). + + +Nodes +^^^^^^ + +A node can be of four types: + +- **QNODE_MACHINE**: for example ``arm/raspi2`` +- **QNODE_DRIVER**: for example ``generic-sdhci`` +- **QNODE_INTERFACE**: for example ``sdhci`` (interface for all ``-sdhci`` + drivers). + An interface is not explicitly created, it will be automatically + instantiated when a node consumes or produces it. + An interface is simply a struct that abstracts the various drivers + for the same type of device, and offers an API to the nodes that + use it ("consume" relation in qgraph terms) that is implemented/backed up by the drivers that implement it ("produce" relation in qgraph terms). +- **QNODE_TEST**: for example ``sdhci-test``. A test consumes an interface + and tests the functions provided by it. + +Notes for the nodes: + +- QNODE_MACHINE: each machine struct must have a ``QGuestAllocator`` and + implement ``get_driver()`` to return the allocator mapped to the interface + "memory". The function can also return ``NULL`` if the allocator + is not set. +- QNODE_DRIVER: driver names must be unique, and machines and nodes + planned to be "consumed" by other nodes must match QEMU + drivers name, otherwise they won't be discovered + +Edges +^^^^^^ + +An edge relation between two nodes (drivers or machines) `X` and `Y` can be: + +- ``X CONSUMES Y``: `Y` can be plugged into `X` +- ``X PRODUCES Y``: `X` provides the interface `Y` +- ``X CONTAINS Y``: `Y` is part of `X` component + +Execution steps +^^^^^^^^^^^^^^^ + +The basic framework steps are the following: + +- All nodes and edges are created in their respective + machine/driver/test files +- The framework starts QEMU and asks for a list of available devices + and machines (note that only machines and "consumed" nodes are mapped + 1:1 with QEMU devices) +- The framework walks the graph starting from the available machines and + performs a Depth First Search for tests +- Once a test is found, the path is walked again and all drivers are + allocated accordingly and the final interface is passed to the test +- The test is executed +- Unused objects are cleaned and the path discovery is continued + +Depending on the QEMU binary used, only some drivers/machines will be +available and only test that are reached by them will be executed. + +Creating a new driver and its interface +""""""""""""""""""""""""""""""""""""""""" + +Here we continue the ``sdhci`` use case, with the following scenario: + +- ``sdhci-test`` aims to test the ``read[q,w], writeq`` functions + offered by the ``sdhci`` drivers. +- The current ``sdhci`` device is supported by both ``x86_64/pc`` and ``ARM`` + (in this example we focus on the ``arm-raspi2``) machines. +- QEMU offers 2 types of drivers: ``QSDHCI_MemoryMapped`` for ``ARM`` and + ``QSDHCI_PCI`` for ``x86_64/pc``. Both implement the + ``read[q,w], writeq`` functions. + +In order to implement such scenario in qgraph, the test developer needs to: + +- Create the ``x86_64/pc`` machine node. This machine uses the + ``pci-bus`` architecture so it ``contains`` a PCI driver, + ``pci-bus-pc``. The actual path is + + ``x86_64/pc --contains--> 1440FX-pcihost --contains--> + pci-bus-pc --produces--> pci-bus``. + + For the sake of this example, + we do not focus on the PCI interface implementation. +- Create the ``sdhci-pci`` driver node, representing ``QSDHCI_PCI``. + The driver uses the PCI bus (and its API), + so it must ``consume`` the ``pci-bus`` generic interface (which abstracts + all the pci drivers available) + + ``sdhci-pci --consumes--> pci-bus`` +- Create an ``arm/raspi2`` machine node. This machine ``contains`` + a ``generic-sdhci`` memory mapped ``sdhci`` driver node, representing + ``QSDHCI_MemoryMapped``. + + ``arm/raspi2 --contains--> generic-sdhci`` +- Create the ``sdhci`` interface node. This interface offers the + functions that are shared by all ``sdhci`` devices. + The interface is produced by ``sdhci-pci`` and ``generic-sdhci``, + the available architecture-specific drivers. + + ``sdhci-pci --produces--> sdhci`` + + ``generic-sdhci --produces--> sdhci`` +- Create the ``sdhci-test`` test node. The test ``consumes`` the + ``sdhci`` interface, using its API. It doesn't need to look at + the supported machines or drivers. + + ``sdhci-test --consumes--> sdhci`` + +``arm-raspi2`` machine, simplified from +``tests/qtest/libqos/arm-raspi2-machine.c``:: + + #include "qgraph.h" + + struct QRaspi2Machine { + QOSGraphObject obj; + QGuestAllocator alloc; + QSDHCI_MemoryMapped sdhci; + }; + + static void *raspi2_get_driver(void *object, const char *interface) + { + QRaspi2Machine *machine = object; + if (!g_strcmp0(interface, "memory")) { + return &machine->alloc; + } + + fprintf(stderr, "%s not present in arm/raspi2\n", interface); + g_assert_not_reached(); + } + + static QOSGraphObject *raspi2_get_device(void *obj, + const char *device) + { + QRaspi2Machine *machine = obj; + if (!g_strcmp0(device, "generic-sdhci")) { + return &machine->sdhci.obj; + } + + fprintf(stderr, "%s not present in arm/raspi2\n", device); + g_assert_not_reached(); + } + + static void *qos_create_machine_arm_raspi2(QTestState *qts) + { + QRaspi2Machine *machine = g_new0(QRaspi2Machine, 1); + + alloc_init(&machine->alloc, ...); + + /* Get node(s) contained inside (CONTAINS) */ + machine->obj.get_device = raspi2_get_device; + + /* Get node(s) produced (PRODUCES) */ + machine->obj.get_driver = raspi2_get_driver; + + /* free the object */ + machine->obj.destructor = raspi2_destructor; + qos_init_sdhci_mm(&machine->sdhci, ...); + return &machine->obj; + } + + static void raspi2_register_nodes(void) + { + /* arm/raspi2 --contains--> generic-sdhci */ + qos_node_create_machine("arm/raspi2", + qos_create_machine_arm_raspi2); + qos_node_contains("arm/raspi2", "generic-sdhci", NULL); + } + + libqos_init(raspi2_register_nodes); + +``x86_64/pc`` machine, simplified from +``tests/qtest/libqos/x86_64_pc-machine.c``:: + + #include "qgraph.h" + + struct i440FX_pcihost { + QOSGraphObject obj; + QPCIBusPC pci; + }; + + struct QX86PCMachine { + QOSGraphObject obj; + QGuestAllocator alloc; + i440FX_pcihost bridge; + }; + + /* i440FX_pcihost */ + + static QOSGraphObject *i440FX_host_get_device(void *obj, + const char *device) + { + i440FX_pcihost *host = obj; + if (!g_strcmp0(device, "pci-bus-pc")) { + return &host->pci.obj; + } + fprintf(stderr, "%s not present in i440FX-pcihost\n", device); + g_assert_not_reached(); + } + + /* x86_64/pc machine */ + + static void *pc_get_driver(void *object, const char *interface) + { + QX86PCMachine *machine = object; + if (!g_strcmp0(interface, "memory")) { + return &machine->alloc; + } + + fprintf(stderr, "%s not present in x86_64/pc\n", interface); + g_assert_not_reached(); + } + + static QOSGraphObject *pc_get_device(void *obj, const char *device) + { + QX86PCMachine *machine = obj; + if (!g_strcmp0(device, "i440FX-pcihost")) { + return &machine->bridge.obj; + } + + fprintf(stderr, "%s not present in x86_64/pc\n", device); + g_assert_not_reached(); + } + + static void *qos_create_machine_pc(QTestState *qts) + { + QX86PCMachine *machine = g_new0(QX86PCMachine, 1); + + /* Get node(s) contained inside (CONTAINS) */ + machine->obj.get_device = pc_get_device; + + /* Get node(s) produced (PRODUCES) */ + machine->obj.get_driver = pc_get_driver; + + /* free the object */ + machine->obj.destructor = pc_destructor; + pc_alloc_init(&machine->alloc, qts, ALLOC_NO_FLAGS); + + /* Get node(s) contained inside (CONTAINS) */ + machine->bridge.obj.get_device = i440FX_host_get_device; + + return &machine->obj; + } + + static void pc_machine_register_nodes(void) + { + /* x86_64/pc --contains--> 1440FX-pcihost --contains--> + * pci-bus-pc [--produces--> pci-bus (in pci.h)] */ + qos_node_create_machine("x86_64/pc", qos_create_machine_pc); + qos_node_contains("x86_64/pc", "i440FX-pcihost", NULL); + + /* contained drivers don't need a constructor, + * they will be init by the parent */ + qos_node_create_driver("i440FX-pcihost", NULL); + qos_node_contains("i440FX-pcihost", "pci-bus-pc", NULL); + } + + libqos_init(pc_machine_register_nodes); + +``sdhci`` taken from ``tests/qtest/libqos/sdhci.c``:: + + /* Interface node, offers the sdhci API */ + struct QSDHCI { + uint16_t (*readw)(QSDHCI *s, uint32_t reg); + uint64_t (*readq)(QSDHCI *s, uint32_t reg); + void (*writeq)(QSDHCI *s, uint32_t reg, uint64_t val); + /* other fields */ + }; + + /* Memory Mapped implementation of QSDHCI */ + struct QSDHCI_MemoryMapped { + QOSGraphObject obj; + QSDHCI sdhci; + /* other driver-specific fields */ + }; + + /* PCI implementation of QSDHCI */ + struct QSDHCI_PCI { + QOSGraphObject obj; + QSDHCI sdhci; + /* other driver-specific fields */ + }; + + /* Memory mapped implementation of QSDHCI */ + + static void *sdhci_mm_get_driver(void *obj, const char *interface) + { + QSDHCI_MemoryMapped *smm = obj; + if (!g_strcmp0(interface, "sdhci")) { + return &smm->sdhci; + } + fprintf(stderr, "%s not present in generic-sdhci\n", interface); + g_assert_not_reached(); + } + + void qos_init_sdhci_mm(QSDHCI_MemoryMapped *sdhci, QTestState *qts, + uint32_t addr, QSDHCIProperties *common) + { + /* Get node contained inside (CONTAINS) */ + sdhci->obj.get_driver = sdhci_mm_get_driver; + + /* SDHCI interface API */ + sdhci->sdhci.readw = sdhci_mm_readw; + sdhci->sdhci.readq = sdhci_mm_readq; + sdhci->sdhci.writeq = sdhci_mm_writeq; + sdhci->qts = qts; + } + + /* PCI implementation of QSDHCI */ + + static void *sdhci_pci_get_driver(void *object, + const char *interface) + { + QSDHCI_PCI *spci = object; + if (!g_strcmp0(interface, "sdhci")) { + return &spci->sdhci; + } + + fprintf(stderr, "%s not present in sdhci-pci\n", interface); + g_assert_not_reached(); + } + + static void *sdhci_pci_create(void *pci_bus, + QGuestAllocator *alloc, + void *addr) + { + QSDHCI_PCI *spci = g_new0(QSDHCI_PCI, 1); + QPCIBus *bus = pci_bus; + uint64_t barsize; + + qpci_device_init(&spci->dev, bus, addr); + + /* SDHCI interface API */ + spci->sdhci.readw = sdhci_pci_readw; + spci->sdhci.readq = sdhci_pci_readq; + spci->sdhci.writeq = sdhci_pci_writeq; + + /* Get node(s) produced (PRODUCES) */ + spci->obj.get_driver = sdhci_pci_get_driver; + + spci->obj.start_hw = sdhci_pci_start_hw; + spci->obj.destructor = sdhci_destructor; + return &spci->obj; + } + + static void qsdhci_register_nodes(void) + { + QOSGraphEdgeOptions opts = { + .extra_device_opts = "addr=04.0", + }; + + /* generic-sdhci */ + /* generic-sdhci --produces--> sdhci */ + qos_node_create_driver("generic-sdhci", NULL); + qos_node_produces("generic-sdhci", "sdhci"); + + /* sdhci-pci */ + /* sdhci-pci --produces--> sdhci + * sdhci-pci --consumes--> pci-bus */ + qos_node_create_driver("sdhci-pci", sdhci_pci_create); + qos_node_produces("sdhci-pci", "sdhci"); + qos_node_consumes("sdhci-pci", "pci-bus", &opts); + } + + libqos_init(qsdhci_register_nodes); + +In the above example, all possible types of relations are created:: + + x86_64/pc --contains--> 1440FX-pcihost --contains--> pci-bus-pc + | + sdhci-pci --consumes--> pci-bus <--produces--+ + | + +--produces--+ + | + v + sdhci + ^ + | + +--produces-- + + | + arm/raspi2 --contains--> generic-sdhci + +or inverting the consumes edge in consumed_by:: + + x86_64/pc --contains--> 1440FX-pcihost --contains--> pci-bus-pc + | + sdhci-pci <--consumed by-- pci-bus <--produces--+ + | + +--produces--+ + | + v + sdhci + ^ + | + +--produces-- + + | + arm/raspi2 --contains--> generic-sdhci + +Adding a new test +""""""""""""""""" + +Given the above setup, adding a new test is very simple. +``sdhci-test``, taken from ``tests/qtest/sdhci-test.c``:: + + static void check_capab_sdma(QSDHCI *s, bool supported) + { + uint64_t capab, capab_sdma; + + capab = s->readq(s, SDHC_CAPAB); + capab_sdma = FIELD_EX64(capab, SDHC_CAPAB, SDMA); + g_assert_cmpuint(capab_sdma, ==, supported); + } + + static void test_registers(void *obj, void *data, + QGuestAllocator *alloc) + { + QSDHCI *s = obj; + + /* example test */ + check_capab_sdma(s, s->props.capab.sdma); + } + + static void register_sdhci_test(void) + { + /* sdhci-test --consumes--> sdhci */ + qos_add_test("registers", "sdhci", test_registers, NULL); + } + + libqos_init(register_sdhci_test); + +Here a new test is created, consuming ``sdhci`` interface node +and creating a valid path from both machines to a test. +Final graph will be like this:: + + x86_64/pc --contains--> 1440FX-pcihost --contains--> pci-bus-pc + | + sdhci-pci --consumes--> pci-bus <--produces--+ + | + +--produces--+ + | + v + sdhci <--consumes-- sdhci-test + ^ + | + +--produces-- + + | + arm/raspi2 --contains--> generic-sdhci + +or inverting the consumes edge in consumed_by:: + + x86_64/pc --contains--> 1440FX-pcihost --contains--> pci-bus-pc + | + sdhci-pci <--consumed by-- pci-bus <--produces--+ + | + +--produces--+ + | + v + sdhci --consumed by--> sdhci-test + ^ + | + +--produces-- + + | + arm/raspi2 --contains--> generic-sdhci + +Assuming there the binary is +``QTEST_QEMU_BINARY=./qemu-system-x86_64`` +a valid test path will be: +``/x86_64/pc/1440FX-pcihost/pci-bus-pc/pci-bus/sdhci-pc/sdhci/sdhci-test`` + +and for the binary ``QTEST_QEMU_BINARY=./qemu-system-arm``: + +``/arm/raspi2/generic-sdhci/sdhci/sdhci-test`` + +Additional examples are also in ``test-qgraph.c`` + +Command line: +"""""""""""""" + +Command line is built by using node names and optional arguments +passed by the user when building the edges. + +There are three types of command line arguments: + +- ``in node`` : created from the node name. For example, machines will + have ``-M <machine>`` to its command line, while devices + ``-device <device>``. It is automatically done by the framework. +- ``after node`` : added as additional argument to the node name. + This argument is added optionally when creating edges, + by setting the parameter ``after_cmd_line`` and + ``extra_edge_opts`` in ``QOSGraphEdgeOptions``. + The framework automatically adds + a comma before ``extra_edge_opts``, + because it is going to add attributes + after the destination node pointed by + the edge containing these options, and automatically + adds a space before ``after_cmd_line``, because it + adds an additional device, not an attribute. +- ``before node`` : added as additional argument to the node name. + This argument is added optionally when creating edges, + by setting the parameter ``before_cmd_line`` in + ``QOSGraphEdgeOptions``. This attribute + is going to add attributes before the destination node + pointed by the edge containing these options. It is + helpful to commands that are not node-representable, + such as ``-fdsev`` or ``-netdev``. + +While adding command line in edges is always used, not all nodes names are +used in every path walk: this is because the contained or produced ones +are already added by QEMU, so only nodes that "consumes" will be used to +build the command line. Also, nodes that will have ``{ "abstract" : true }`` +as QMP attribute will loose their command line, since they are not proper +devices to be added in QEMU. + +Example:: + + QOSGraphEdgeOptions opts = { + .before_cmd_line = "-drive id=drv0,if=none,file=null-co://," + "file.read-zeroes=on,format=raw", + .after_cmd_line = "-device scsi-hd,bus=vs0.0,drive=drv0", + + opts.extra_device_opts = "id=vs0"; + }; + + qos_node_create_driver("virtio-scsi-device", + virtio_scsi_device_create); + qos_node_consumes("virtio-scsi-device", "virtio-bus", &opts); + +Will produce the following command line: +``-drive id=drv0,if=none,file=null-co://, -device virtio-scsi-device,id=vs0 -device scsi-hd,bus=vs0.0,drive=drv0`` + +Qgraph API reference +^^^^^^^^^^^^^^^^^^^^ + +.. kernel-doc:: tests/qtest/libqos/qgraph.h diff --git a/docs/devel/qtest.rst b/docs/devel/qtest.rst index 97c5a75626..c3dceb6c8a 100644 --- a/docs/devel/qtest.rst +++ b/docs/devel/qtest.rst @@ -2,6 +2,11 @@ QTest Device Emulation Testing Framework ======================================== +.. toctree:: + :hidden: + + qgraph + QTest is a device emulation testing framework. It can be very useful to test device models; it could also control certain aspects of QEMU (such as virtual clock stepping), with a special purpose "qtest" protocol. Refer to @@ -24,6 +29,9 @@ On top of libqtest, a higher level library, ``libqos``, was created to encapsulate common tasks of device drivers, such as memory management and communicating with system buses or devices. Many virtual device tests use libqos instead of directly calling into libqtest. +Libqos also offers the Qgraph API to increase each test coverage and +automate QEMU command line arguments and devices setup. +Refer to :ref:`qgraph` for Qgraph explanation and API. Steps to add a new QTest case are: diff --git a/docs/generic-loader.txt b/docs/generic-loader.txt deleted file mode 100644 index a9603a2af7..0000000000 --- a/docs/generic-loader.txt +++ /dev/null @@ -1,92 +0,0 @@ -Copyright (c) 2016 Xilinx Inc. - -This work is licensed under the terms of the GNU GPL, version 2 or later. See -the COPYING file in the top-level directory. - - -The 'loader' device allows the user to load multiple images or values into -QEMU at startup. - -Loading Data into Memory Values -------------------------------- -The loader device allows memory values to be set from the command line. This -can be done by following the syntax below: - - -device loader,addr=<addr>,data=<data>,data-len=<data-len> - [,data-be=<data-be>][,cpu-num=<cpu-num>] - - <addr> - The address to store the data in. - <data> - The value to be written to the address. The maximum size of - the data is 8 bytes. - <data-len> - The length of the data in bytes. This argument must be - included if the data argument is. - <data-be> - Set to true if the data to be stored on the guest should be - written as big endian data. The default is to write little - endian data. - <cpu-num> - The number of the CPU's address space where the data should - be loaded. If not specified the address space of the first - CPU is used. - -All values are parsed using the standard QemuOps parsing. This allows the user -to specify any values in any format supported. By default the values -will be parsed as decimal. To use hex values the user should prefix the number -with a '0x'. - -An example of loading value 0x8000000e to address 0xfd1a0104 is: - -device loader,addr=0xfd1a0104,data=0x8000000e,data-len=4 - -Setting a CPU's Program Counter -------------------------------- -The loader device allows the CPU's PC to be set from the command line. This -can be done by following the syntax below: - - -device loader,addr=<addr>,cpu-num=<cpu-num> - - <addr> - The value to use as the CPU's PC. - <cpu-num> - The number of the CPU whose PC should be set to the - specified value. - -All values are parsed using the standard QemuOps parsing. This allows the user -to specify any values in any format supported. By default the values -will be parsed as decimal. To use hex values the user should prefix the number -with a '0x'. - -An example of setting CPU 0's PC to 0x8000 is: - -device loader,addr=0x8000,cpu-num=0 - -Loading Files -------------- -The loader device also allows files to be loaded into memory. It can load ELF, -U-Boot, and Intel HEX executable formats as well as raw images. The syntax is -shown below: - - -device loader,file=<file>[,addr=<addr>][,cpu-num=<cpu-num>][,force-raw=<raw>] - - <file> - A file to be loaded into memory - <addr> - The memory address where the file should be loaded. This is - required for raw images and ignored for non-raw files. - <cpu-num> - This specifies the CPU that should be used. This is an - optional argument and will cause the CPU's PC to be set to - the memory address where the raw file is loaded or the entry - point specified in the executable format header. This option - should only be used for the boot image. - This will also cause the image to be written to the specified - CPU's address space. If not specified, the default is CPU 0. - <force-raw> - Setting force-raw=on forces the file to be treated as a raw - image. This can be used to load supported executable formats - as if they were raw. - -All values are parsed using the standard QemuOps parsing. This allows the user -to specify any values in any format supported. By default the values -will be parsed as decimal. To use hex values the user should prefix the number -with a '0x'. - -An example of loading an ELF file which CPU0 will boot is shown below: - -device loader,file=./images/boot.elf,cpu-num=0 - -Restrictions and ToDos ----------------------- - - At the moment it is just assumed that if you specify a cpu-num then you - want to set the PC as well. This might not always be the case. In future - the internal state 'set_pc' (which exists in the generic loader now) should - be exposed to the user so that they can choose if the PC is set or not. diff --git a/docs/interop/parallels.txt b/docs/interop/parallels.txt index f15bf35bd1..bb3fadf369 100644 --- a/docs/interop/parallels.txt +++ b/docs/interop/parallels.txt @@ -208,21 +208,25 @@ of its data area are: 28 - 31: l1_size The number of entries in the L1 table of the bitmap. - variable: l1_table (8 * l1_size bytes) - L1 offset table (in bytes) + variable: L1 offset table (l1_table), size: 8 * l1_size bytes -A dirty bitmap is stored using a one-level structure for the mapping to host -clusters - an L1 table. +The dirty bitmap described by this feature extension is stored in a set of +clusters inside the Parallels image file. The offsets of these clusters are +saved in the L1 offset table specified by the feature extension. Each L1 table +entry is a 64 bit integer as described below: -Given an offset in bytes into the bitmap data, the offset in bytes into the -image file can be obtained as follows: +Given an offset in bytes into the bitmap data, corresponding L1 entry is - offset = l1_table[offset / cluster_size] + (offset % cluster_size) + l1_table[offset / cluster_size] -If an L1 table entry is 0, the corresponding cluster of the bitmap is assumed -to be zero. +If an L1 table entry is 0, all bits in the corresponding cluster of the bitmap +are assumed to be 0. -If an L1 table entry is 1, the corresponding cluster of the bitmap is assumed -to have all bits set. +If an L1 table entry is 1, all bits in the corresponding cluster of the bitmap +are assumed to be 1. -If an L1 table entry is not 0 or 1, it allocates a cluster from the data area. +If an L1 table entry is not 0 or 1, it contains the corresponding cluster +offset (in 512b sectors). Given an offset in bytes into the bitmap data the +offset in bytes into the image file can be obtained as follows: + + offset = l1_table[offset / cluster_size] * 512 + (offset % cluster_size) diff --git a/docs/system/arm/aspeed.rst b/docs/system/arm/aspeed.rst index 690bada784..d1fb8f25b3 100644 --- a/docs/system/arm/aspeed.rst +++ b/docs/system/arm/aspeed.rst @@ -48,6 +48,7 @@ Supported devices * UART * Ethernet controllers * Front LEDs (PCA9552 on I2C bus) + * LPC Peripheral Controller (a subset of subdevices are supported) Missing devices @@ -56,7 +57,6 @@ Missing devices * Coprocessor support * ADC (out of tree implementation) * PWM and Fan Controller - * LPC Bus Controller * Slave GPIO Controller * Super I/O Controller * Hash/Crypto Engine @@ -72,18 +72,22 @@ Missing devices Boot options ------------ -The Aspeed machines can be started using the -kernel option to load a -Linux kernel or from a firmware image which can be downloaded from the -OpenPOWER jenkins : +The Aspeed machines can be started using the ``-kernel`` option to +load a Linux kernel or from a firmware. Images can be downloaded from +the OpenBMC jenkins : - https://openpower.xyz/ + https://jenkins.openbmc.org/job/ci-openbmc/lastSuccessfulBuild/distro=ubuntu,label=docker-builder + +or directly from the OpenBMC GitHub release repository : + + https://github.com/openbmc/openbmc/releases The image should be attached as an MTD drive. Run : .. code-block:: bash $ qemu-system-arm -M romulus-bmc -nic user \ - -drive file=flash-romulus,format=raw,if=mtd -nographic + -drive file=obmc-phosphor-image-romulus.static.mtd,format=raw,if=mtd -nographic Options specific to Aspeed machines are : diff --git a/docs/system/arm/mps2.rst b/docs/system/arm/mps2.rst index 601ccea15c..f83b151787 100644 --- a/docs/system/arm/mps2.rst +++ b/docs/system/arm/mps2.rst @@ -1,5 +1,5 @@ -Arm MPS2 and MPS3 boards (``mps2-an385``, ``mps2-an386``, ``mps2-an500``, ``mps2-an505``, ``mps2-an511``, ``mps2-an521``, ``mps3-an524``) -========================================================================================================================================= +Arm MPS2 and MPS3 boards (``mps2-an385``, ``mps2-an386``, ``mps2-an500``, ``mps2-an505``, ``mps2-an511``, ``mps2-an521``, ``mps3-an524``, ``mps3-an547``) +========================================================================================================================================================= These board models all use Arm M-profile CPUs. @@ -27,6 +27,8 @@ QEMU models the following FPGA images: Dual Cortex-M33 as documented in Arm Application Note AN521 ``mps3-an524`` Dual Cortex-M33 on an MPS3, as documented in Arm Application Note AN524 +``mps3-an547`` + Cortex-M55 on an MPS3, as documented in Arm Application Note AN547 Differences between QEMU and real hardware: diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index cfabe69846..5e3a31c123 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -166,6 +166,14 @@ Using ``-M kernel-irqchip=off`` with x86 machine types that include a local APIC is deprecated. The ``split`` setting is supported, as is using ``-M kernel-irqchip=off`` with the ISA PC machine type. +hexadecimal sizes with scaling multipliers (since 6.0) +'''''''''''''''''''''''''''''''''''''''''''''''''''''' + +Input parameters that take a size value should only use a size suffix +(such as 'k' or 'M') when the base is written in decimal, and not when +the value is hexadecimal. That is, '0x20M' is deprecated, and should +be written either as '32M' or as '0x2000000'. + QEMU Machine Protocol (QMP) commands ------------------------------------ @@ -402,14 +410,6 @@ it out of sheepdog volumes into an alternative storage backend. linux-user mode CPUs -------------------- -``tilegx`` CPUs (since 5.1.0) -''''''''''''''''''''''''''''' - -The ``tilegx`` guest CPU support (which was only implemented in -linux-user mode) is deprecated and will be removed in a future version -of QEMU. Support for this CPU was removed from the upstream Linux -kernel in 2018, and has also been dropped from glibc. - ``ppc64abi32`` CPUs (since 5.2.0) ''''''''''''''''''''''''''''''''' diff --git a/docs/system/generic-loader.rst b/docs/system/generic-loader.rst new file mode 100644 index 0000000000..6bf8a4eb48 --- /dev/null +++ b/docs/system/generic-loader.rst @@ -0,0 +1,117 @@ +.. + Copyright (c) 2016, Xilinx Inc. + +This work is licensed under the terms of the GNU GPL, version 2 or later. See +the COPYING file in the top-level directory. + +Generic Loader +-------------- + +The 'loader' device allows the user to load multiple images or values into +QEMU at startup. + +Loading Data into Memory Values +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The loader device allows memory values to be set from the command line. This +can be done by following the syntax below:: + + -device loader,addr=<addr>,data=<data>,data-len=<data-len> \ + [,data-be=<data-be>][,cpu-num=<cpu-num>] + +``<addr>`` + The address to store the data in. + +``<data>`` + The value to be written to the address. The maximum size of the data + is 8 bytes. + +``<data-len>`` + The length of the data in bytes. This argument must be included if + the data argument is. + +``<data-be>`` + Set to true if the data to be stored on the guest should be written + as big endian data. The default is to write little endian data. + +``<cpu-num>`` + The number of the CPU's address space where the data should be + loaded. If not specified the address space of the first CPU is used. + +All values are parsed using the standard QemuOps parsing. This allows the user +to specify any values in any format supported. By default the values +will be parsed as decimal. To use hex values the user should prefix the number +with a '0x'. + +An example of loading value 0x8000000e to address 0xfd1a0104 is:: + + -device loader,addr=0xfd1a0104,data=0x8000000e,data-len=4 + +Setting a CPU's Program Counter +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The loader device allows the CPU's PC to be set from the command line. This +can be done by following the syntax below:: + + -device loader,addr=<addr>,cpu-num=<cpu-num> + +``<addr>`` + The value to use as the CPU's PC. + +``<cpu-num>`` + The number of the CPU whose PC should be set to the specified value. + +All values are parsed using the standard QemuOpts parsing. This allows the user +to specify any values in any format supported. By default the values +will be parsed as decimal. To use hex values the user should prefix the number +with a '0x'. + +An example of setting CPU 0's PC to 0x8000 is:: + + -device loader,addr=0x8000,cpu-num=0 + +Loading Files +^^^^^^^^^^^^^ + +The loader device also allows files to be loaded into memory. It can load ELF, +U-Boot, and Intel HEX executable formats as well as raw images. The syntax is +shown below: + + -device loader,file=<file>[,addr=<addr>][,cpu-num=<cpu-num>][,force-raw=<raw>] + +``<file>`` + A file to be loaded into memory + +``<addr>`` + The memory address where the file should be loaded. This is required + for raw images and ignored for non-raw files. + +``<cpu-num>`` + This specifies the CPU that should be used. This is an + optional argument and will cause the CPU's PC to be set to the + memory address where the raw file is loaded or the entry point + specified in the executable format header. This option should only + be used for the boot image. This will also cause the image to be + written to the specified CPU's address space. If not specified, the + default is CPU 0. <force-raw> - Setting force-raw=on forces the file + to be treated as a raw image. This can be used to load supported + executable formats as if they were raw. + +All values are parsed using the standard QemuOpts parsing. This allows the user +to specify any values in any format supported. By default the values +will be parsed as decimal. To use hex values the user should prefix the number +with a '0x'. + +An example of loading an ELF file which CPU0 will boot is shown below:: + + -device loader,file=./images/boot.elf,cpu-num=0 + +Restrictions and ToDos +^^^^^^^^^^^^^^^^^^^^^^ + +At the moment it is just assumed that if you specify a cpu-num then +you want to set the PC as well. This might not always be the case. In +future the internal state 'set_pc' (which exists in the generic loader +now) should be exposed to the user so that they can choose if the PC +is set or not. + + diff --git a/docs/system/guest-loader.rst b/docs/system/guest-loader.rst new file mode 100644 index 0000000000..37d03cbd89 --- /dev/null +++ b/docs/system/guest-loader.rst @@ -0,0 +1,54 @@ +.. + Copyright (c) 2020, Linaro + +Guest Loader +------------ + +The guest loader is similar to the `generic-loader` although it is +aimed at a particular use case of loading hypervisor guests. This is +useful for debugging hypervisors without having to jump through the +hoops of firmware and boot-loaders. + +The guest loader does two things: + + - load blobs (kernels and initial ram disks) into memory + - sets platform FDT data so hypervisors can find and boot them + +This is what is typically done by a boot-loader like grub using it's +multi-boot capability. A typical example would look like: + +.. parsed-literal:: + + |qemu_system| -kernel ~/xen.git/xen/xen \ + -append "dom0_mem=1G,max:1G loglvl=all guest_loglvl=all" \ + -device guest-loader,addr=0x42000000,kernel=Image,bootargs="root=/dev/sda2 ro console=hvc0 earlyprintk=xen" \ + -device guest-loader,addr=0x47000000,initrd=rootfs.cpio + +In the above example the Xen hypervisor is loaded by the -kernel +parameter and passed it's boot arguments via -append. The Dom0 guest +is loaded into the areas of memory. Each blob will get +`/chosen/module@<addr>` entry in the FDT to indicate it's location and +size. Additional information can be passed with by using additional +arguments. + +Currently the only supported machines which use FDT data to boot are +the ARM and RiscV `virt` machines. + +Arguments +^^^^^^^^^ + +The full syntax of the guest-loader is:: + + -device guest-loader,addr=<addr>[,kernel=<file>,[bootargs=<args>]][,initrd=<file>] + +``addr=<addr>`` + This is mandatory and indicates the start address of the blob. + +``kernel|initrd=<file>`` + Indicates the filename of the kernel or initrd blob. Both blobs will + have the "multiboot,module" compatibility string as well as + "multiboot,kernel" or "multiboot,ramdisk" as appropriate. + +``bootargs=<args>`` + This is an optional field for kernel blobs which will pass command + like via the `/chosen/module@<addr>/bootargs` node. diff --git a/docs/system/index.rst b/docs/system/index.rst index 625b494372..6ad9c93806 100644 --- a/docs/system/index.rst +++ b/docs/system/index.rst @@ -25,6 +25,8 @@ Contents: usb ivshmem linuxboot + generic-loader + guest-loader vnc-security tls gdb diff --git a/docs/system/quickstart.rst b/docs/system/quickstart.rst index 3a3acab5e7..681678c86e 100644 --- a/docs/system/quickstart.rst +++ b/docs/system/quickstart.rst @@ -11,3 +11,11 @@ Download and uncompress a PC hard disk image with Linux installed (e.g. |qemu_system| linux.img Linux should boot and give you a prompt. + +Users should be aware the above example elides a lot of the complexity +of setting up a VM with x86_64 specific defaults and assumes the +first non switch argument is a PC compatible disk image with a boot +sector. For a non-x86 system where we emulate a broad range of machine +types, the command lines are generally more explicit in defining the +machine and boot behaviour. You will find more example command lines +in the :ref:`system-targets-ref` section of the manual. diff --git a/docs/system/removed-features.rst b/docs/system/removed-features.rst index c8481cafbd..4dcf4f924c 100644 --- a/docs/system/removed-features.rst +++ b/docs/system/removed-features.rst @@ -142,6 +142,20 @@ This machine has been renamed ``fuloong2e``. These machine types were very old and likely could not be used for live migration from old QEMU versions anymore. Use a newer machine type instead. + +linux-user mode CPUs +-------------------- + +``tilegx`` CPUs (removed in 6.0) +'''''''''''''''''''''''''''''''' + +The ``tilegx`` guest CPU support has been removed without replacement. It was +only implemented in linux-user mode, but support for this CPU was removed from +the upstream Linux kernel in 2018, and it has also been dropped from glibc, so +there is no new Linux development taking place with this architecture. For +running the old binaries, you can use older versions of QEMU. + + Related binaries ---------------- diff --git a/docs/system/targets.rst b/docs/system/targets.rst index 75ed1087fd..9dcd95dd84 100644 --- a/docs/system/targets.rst +++ b/docs/system/targets.rst @@ -1,3 +1,5 @@ +.. _system-targets-ref: + QEMU System Emulator Targets ============================ diff --git a/docs/tools/qemu-storage-daemon.rst b/docs/tools/qemu-storage-daemon.rst index c05b3d3811..086493ebb3 100644 --- a/docs/tools/qemu-storage-daemon.rst +++ b/docs/tools/qemu-storage-daemon.rst @@ -69,7 +69,7 @@ Standard options: a description of character device properties. A common character device definition configures a UNIX domain socket:: - --chardev socket,id=char1,path=/tmp/qmp.sock,server=on,wait=off + --chardev socket,id=char1,path=/var/run/qsd-qmp.sock,server=on,wait=off .. option:: --export [type=]nbd,id=<id>,node-name=<node-name>[,name=<export-name>][,writable=on|off][,bitmap=<name>] --export [type=]vhost-user-blk,id=<id>,node-name=<node-name>,addr.type=unix,addr.path=<socket-path>[,writable=on|off][,logical-block-size=<block-size>][,num-queues=<num-queues>] @@ -80,8 +80,9 @@ Standard options: requests for modifying data (the default is off). The ``nbd`` export type requires ``--nbd-server`` (see below). ``name`` is - the NBD export name. ``bitmap`` is the name of a dirty bitmap reachable from - the block node, so the NBD client can use NBD_OPT_SET_META_CONTEXT with the + the NBD export name (if not specified, it defaults to the given + ``node-name``). ``bitmap`` is the name of a dirty bitmap reachable from the + block node, so the NBD client can use NBD_OPT_SET_META_CONTEXT with the metadata context name "qemu:dirty-bitmap:BITMAP" to inspect the bitmap. The ``vhost-user-blk`` export type takes a vhost-user socket address on which @@ -101,14 +102,17 @@ Standard options: .. option:: --nbd-server addr.type=inet,addr.host=<host>,addr.port=<port>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>] --nbd-server addr.type=unix,addr.path=<path>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>] + --nbd-server addr.type=fd,addr.str=<fd>[,tls-creds=<id>][,tls-authz=<id>][,max-connections=<n>] is a server for NBD exports. Both TCP and UNIX domain sockets are supported. - TLS encryption can be configured using ``--object`` tls-creds-* and authz-* - secrets (see below). + A listen socket can be provided via file descriptor passing (see Examples + below). TLS encryption can be configured using ``--object`` tls-creds-* and + authz-* secrets (see below). - To configure an NBD server on UNIX domain socket path ``/tmp/nbd.sock``:: + To configure an NBD server on UNIX domain socket path + ``/var/run/qsd-nbd.sock``:: - --nbd-server addr.type=unix,addr.path=/tmp/nbd.sock + --nbd-server addr.type=unix,addr.path=/var/run/qsd-nbd.sock .. option:: --object help --object <type>,help @@ -118,6 +122,20 @@ Standard options: List object properties with ``<type>,help``. See the :manpage:`qemu(1)` manual page for a description of the object properties. +.. option:: --pidfile PATH + + is the path to a file where the daemon writes its pid. This allows scripts to + stop the daemon by sending a signal:: + + $ kill -SIGTERM $(<path/to/qsd.pid) + + A file lock is applied to the file so only one instance of the daemon can run + with a given pid file path. The daemon unlinks its pid file when terminating. + + The pid file is written after chardevs, exports, and NBD servers have been + created but before accepting connections. The daemon has started successfully + when the pid file is written and clients may begin connecting. + Examples -------- Launch the daemon with QMP monitor socket ``qmp.sock`` so clients can execute @@ -127,6 +145,42 @@ QMP commands:: --chardev socket,path=qmp.sock,server=on,wait=off,id=char1 \ --monitor chardev=char1 +Launch the daemon from Python with a QMP monitor socket using file descriptor +passing so there is no need to busy wait for the QMP monitor to become +available:: + + #!/usr/bin/env python3 + import subprocess + import socket + + sock_path = '/var/run/qmp.sock' + + with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as listen_sock: + listen_sock.bind(sock_path) + listen_sock.listen() + + fd = listen_sock.fileno() + + subprocess.Popen( + ['qemu-storage-daemon', + '--chardev', f'socket,fd={fd},server=on,id=char1', + '--monitor', 'chardev=char1'], + pass_fds=[fd], + ) + + # listen_sock was automatically closed when leaving the 'with' statement + # body. If the daemon process terminated early then the following connect() + # will fail with "Connection refused" because no process has the listen + # socket open anymore. Launch errors can be detected this way. + + qmp_sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + qmp_sock.connect(sock_path) + ...QMP interaction... + +The same socket spawning approach also works with the ``--nbd-server +addr.type=fd,addr.str=<fd>`` and ``--export +type=vhost-user-blk,addr.type=fd,addr.str=<fd>`` options. + Export raw image file ``disk.img`` over NBD UNIX domain socket ``nbd.sock``:: $ qemu-storage-daemon \ |