summary refs log tree commit diff stats
path: root/python/qemu/machine.py (unfollow)
Commit message (Collapse)AuthorFilesLines
2019-07-19linux-user: fix to handle variably sized SIOCGSTAMP with new kernelsDaniel P. Berrangé4-38/+159
The SIOCGSTAMP symbol was previously defined in the asm-generic/sockios.h header file. QEMU sees that header indirectly via sys/socket.h In linux kernel commit 0768e17073dc527ccd18ed5f96ce85f9985e9115 the asm-generic/sockios.h header no longer defines SIOCGSTAMP. Instead it provides only SIOCGSTAMP_OLD, which only uses a 32-bit time_t on 32-bit architectures. The linux/sockios.h header then defines SIOCGSTAMP using either SIOCGSTAMP_OLD or SIOCGSTAMP_NEW as appropriate. If SIOCGSTAMP_NEW is used, then the tv_sec field is 64-bit even on 32-bit architectures To cope with this we must now convert the old and new type from the target to the host one. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Message-Id: <20190718130641.15294-1-laurent@vivier.eu> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-07-18linux-user: check valid address in access_ok()Rémi Denis-Courmont2-1/+7
Fix a crash with LTP testsuite and aarch64: tst_test.c:1015: INFO: Timeout per run is 0h 05m 00s qemu-aarch64: .../qemu/accel/tcg/translate-all.c:2522: page_check_range: Assertion `start < ((target_ulong)1 << L1_MAP_ADDR_SPACE_BITS)' failed. qemu:handle_cpu_signal received signal outside vCPU context @ pc=0x60001554 page_check_range() should never be called with address outside the guest address space. This patch adds a guest_addr_valid() check in access_ok() to only call page_check_range() with a valid address. Fixes: f6768aa1b4c6 ("target/arm: fix AArch64 virtual address space size") Signed-off-by: Rémi Denis-Courmont <remi@remlab.net> Signed-off-by: Laurent Vivier <lvivier@redhat.com> Message-Id: <20190704084115.24713-1-lvivier@redhat.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-07-16Update version for v4.1.0-rc1 releasePeter Maydell1-1/+1
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-07-16linux-user: Fix structure target_ucontext for MIPSAleksandar Markovic1-3/+2
Structure ucontext for MIPS is defined in the following way in Linux kernel: (arch/mips/include/uapi/asm/ucontext.h, lines 54-64) struct ucontext { /* Historic fields matching asm-generic */ unsigned long uc_flags; struct ucontext *uc_link; stack_t uc_stack; struct sigcontext uc_mcontext; sigset_t uc_sigmask; /* Extended context structures may follow ucontext */ unsigned long long uc_extcontext[0]; }; Fix the structure target_ucontext for MIPS to reflect the definition above, except the correction for field uc_extcontext, which will follow at some later time. Fixes: 94c5495d Reported-by: Dragan Mladjenovic <dmladjenovic@wavecomp.com> Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Laurent Vivier <laurent@vivier.eu> Message-Id: <1562931470-3700-2-git-send-email-aleksandar.markovic@rt-rk.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-07-16hw/block/pflash_cfi01: Start state machine as READY to accept commandsPhilippe Mathieu-Daudé1-1/+1
When the state machine is ready to accept command, the bit 7 of the status register (SR) is set to 1. The guest polls the status register and check this bit before writting command to the internal 'Write State Machine' (WSM). Set SR.7 bit to 1 when the device is created. There is no migration impact by this change. Reference: Read Array Flowchart "Common Flash Interface (CFI) and Command Sets" (Intel Application Note 646) Appendix B "Basic Command Set" Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Regression-tested-by: Laszlo Ersek <lersek@redhat.com> Message-Id: <20190715121338.20600-5-philmd@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-16hw/block/pflash_cfi02: Explicit switch fallthrough for ERASE commandsPhilippe Mathieu-Daudé1-0/+1
Previous to commit ddb6f2254, the DQ2 bit was incorrectly set during PROGRAM command (0xA0). The commit reordered the switch cases to only set the DQ2 bit for the ERASE commands using a fallthrough, but did not explicit the fallthrough is intentional. Mark the switch fallthrough with a comment interpretable by C preprocessors and static analysis tools. Reported-by: Coverity (CID 1403012) Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-Id: <20190711130759.27720-1-philmd@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-16s390x/cpumodel: change internal name of vxpdeh to match descriptionChristian Borntraeger2-3/+3
The internal macro name VECTOR_BCD_ENH does not match the actual description. Fix this. Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com> Message-Id: <20190715142304.215018-4-borntraeger@de.ibm.com> [CH: vxp->vxpdeh, as discussed] Reviewed-by: David Hildenbrand <david@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2019-07-16s390x/cpumodel: also change name of vxbehChristian Borntraeger1-1/+1
David suggested to keep everything in sync as 4.1 is not yet released. This patch fixes the name "vxbeh" into "vxpdeh". To simplify the backports this patch will not change VECTOR_BCD_ENH as this is just an internal name. That will be done by an extra patch that does not need to be backported. Suggested-by: David Hildenbrand <david@redhat.com> Fixes: d05be57ddc2e ("s390: cpumodel: fix description for the new vector facility") Fixes: 54d65de0b525 ("s390x/cpumodel: vector enhancements") Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com> Message-Id: <20190715142304.215018-3-borntraeger@de.ibm.com> [CH: vxp->vxpdeh, as discussed] Reviewed-by: David Hildenbrand <david@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2019-07-16s390x/cpumodel: remove esort from the default modelChristian Borntraeger1-1/+0
esort might not be available on all models. Fixes: caef62430fed6e73 ("s390x/cpumodel: add gen15 defintions") Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com> Message-Id: <20190715142304.215018-2-borntraeger@de.ibm.com> Reviewed-by: David Hildenbrand <david@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2019-07-16vl: make sure char-pty message displayed by moving setbuf to the beginningWei Yang1-2/+2
Recently we found a behavior change after commit 6ade45f2ac93611 ('char-pty: Print "char device redirected" message to stdout'). When we redirect output to a file, the message "char device redirected to PTY_NAME (label LABEL)" would not be seen at the beginning of the file. Instead, the message is displayed after QEMU quit. This will block test automation. The reason is this message is printed after we set line buffer mode. So move this to the beginning. Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15target/mips: Add missing 'break' for certain cases of MTTR handlingAleksandar Markovic1-0/+2
This was found by GCC 8.3 static analysis. Fixes: ead9360e2fb Reported-by: Stefan Weil <sw@weilnetz.de> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com> Message-Id: <1563220847-14630-5-git-send-email-aleksandar.markovic@rt-rk.com>
2019-07-15target/mips: Add missing 'break' for certain cases of MFTR handlingAleksandar Markovic1-0/+2
This was found by GCC 8.3 static analysis. Fixes: ead9360e2fb Reported-by: Stefan Weil <sw@weilnetz.de> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com> Message-Id: <1563220847-14630-4-git-send-email-aleksandar.markovic@rt-rk.com>
2019-07-15target/mips: Add missing 'break' for a case of MTHC0 handlingAleksandar Markovic1-0/+1
This was found by GCC 8.3 static analysis. Fixes: 5fb2dcd1792 Reported-by: Stefan Weil <sw@weilnetz.de> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com> Message-Id: <1563220847-14630-3-git-send-email-aleksandar.markovic@rt-rk.com>
2019-07-15qemu-tech: Fix dangling @menu entriesMarkus Armbruster1-3/+0
Recent commit 2f2c4e4731 "Convert "translator internals" docs to RST, move to devel manual" and commit 282d36b5e2 "qemu-tech.texi: Remove "QEMU compared to other emulators" section" removed @node, but left their @menu entries behind. This broke building qemu-doc.info (but not qemu-doc.{html,pdf,txt}; how odd). Bury the dead @menu entries. Reported-by: Philippe Mathieu-Daudé <philmd@redhat.com> Fixes: 2f2c4e4731449449a2b1aafcd73e4f9ae107d78b Fixes: 282d36b5e27ba86d42d0638430e439c2c257367b Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190715055736.15214-3-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
2019-07-15Makefile: Fix missing dependency of on qemu-tech.texiMarkus Armbruster1-1/+2
The qemu-doc.{html,info,pdf,txt} depend on qemu-doc.texi and its include files. Except qemu-tech.texi is missing. Has always been missing as far as I can see. Fix it. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190715055736.15214-2-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-15Makefile: Fix "make install" when "make all" needs workMarkus Armbruster1-0/+1
Until recently, target install used to recurse into target directories in its recipe: it ran make install in a for-loop. Since target install depends on target all, this trivially ensured we run the sub-make install only after completing target all. Commit 1338a4b "Makefile: Reuse all's recursion machinery for clean and install" moved the target recursion to dependencies. That's good (the commit message explains why), but I forgot to add dependencies to ensure make runs the sub-make install only after completing target all. Do that now. Fixes: 1338a4b72659ce08eacb9de0205fe16202a22d9c Reported-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> Reported-by: Guenter Roeck <linux@roeck-us.net> Tested-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190712055935.23061-1-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Tested-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
2019-07-15create_config: remove $(CONFIG_SOFTMMU) hackPaolo Bonzini2-2/+2
CONFIG_TPM is defined to a rather weird $(CONFIG_SOFTMMU) so that it expands to the right thing in hw/Makefile.objs. This however is not needed anymore and it has a corresponding hack in create_config to turn it into "#define CONFIG_TPM 1". Clean up. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15Makefile: do not repeat $(CONFIG_SOFTMMU) in hw/Makefile.objsPaolo Bonzini1-29/+32
The device directories must be included only for softmmu builds. Instead of repeating $(CONFIG_SOFTMMU), use an "if". Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15hw/usb/Kconfig: USB_XHCI_NEC requires USB_XHCIPhilippe Mathieu-Daudé1-1/+1
TYPE_NEC_XHCI is child of TYPE_XHCI. Add the missing Kconfig dependency. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15hw/usb/Kconfig: Add CONFIG_USB_EHCI_PCIPhilippe Mathieu-Daudé2-5/+9
The USB_EHCI entry currently include PCI code. Since the EHCI implementation is already split in sysbus/PCI, add a new USB_EHCI_PCI. There are no logical changes, but the Kconfig dependencies tree is cleaner. Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15target/i386: sev: Do not unpin ram device memory regionAlex Williamson1-0/+11
The commit referenced below skipped pinning ram device memory when ram blocks are added, we need to do the same when they're removed. Cc: Brijesh Singh <brijesh.singh@amd.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Fixes: cedc0ad539af ("target/i386: sev: Do not pin the ram device memory region") Signed-off-by: Alex Williamson <alex.williamson@redhat.com> Message-Id: <156320087103.2556.10983987500488190423.stgit@gimli.home> Reviewed-by: Singh, Brijesh <brijesh.singh@amd.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15checkpatch: detect doubly-encoded UTF-8Paolo Bonzini1-0/+16
Copy and pasting from Thunderbird's "view source" window results in double encoding of multibyte UTF-8 sequences. The appearance of those sequences is very peculiar, so detect it and give an error despite the (low) possibility of false positives. As the major offender, I am also adding the same check to my applypatch-msg and commit-msg hooks, but this will also cause patchew to croak loudly when this mistake happens. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <1558099140-53240-1-git-send-email-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15hw/lm32/Kconfig: Milkymist One provides a USB 1.1 ControllerPhilippe Mathieu-Daudé1-0/+1
The Milkymist SoftUSB block provides the OHCI USB standard (missed in 0858746b835). Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20190714124755.14356-1-philmd@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-07-15gluster: fix .bdrv_reopen_prepare when backing file is a JSON objectStefano Garzarella1-1/+11
When the backing_file is specified as a JSON object, the qemu_gluster_reopen_prepare() fails with this message: invalid URI json:{"server.0.host": ...} In this case, we should call qemu_gluster_init() using the QDict 'state->options' that contains the JSON parameters already parsed. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1542445 Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> Message-id: 20190715132844.506584-1-sgarzare@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15iotests: Add read-only test case to 030Max Reitz2-3/+30
This tests that the stream job exits cleanly (without abort) when the top node is read-only and cannot be reopened read/write. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190703172813.6868-12-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15iotests: Add new case to 030Max Reitz2-2/+27
We recently removed the dependency of the stream job on its base node. That makes it OK to use a commit filter node there. Test that. Signed-off-by: Max Reitz <mreitz@redhat.com> Tested-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 20190703172813.6868-11-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15iotests: Add @use_log to VM.run_job()Max Reitz1-5/+13
unittest-style tests generally do not use the log file, but VM.run_job() can still be useful to them. Add a parameter to it that hides its output from the log file. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190703172813.6868-10-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15iotests: Compare error messages in 030Max Reitz1-24/+42
Currently, 030 just compares the error class, which does not say anything. Before HEAD^ added throttling to test_overlapping_4, that test actually usually failed because node2 was already gone, not because it was the commit and stream job were not allowed to overlap. Prevent such problems in the future by comparing the error description instead. Signed-off-by: Max Reitz <mreitz@redhat.com> Tested-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 20190703172813.6868-9-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15iotests: Fix throttling in 030Max Reitz1-5/+27
Currently, TestParallelOps in 030 creates images that are too small for job throttling to be effective. This is reflected by the fact that it never undoes the throttling. Increase the image size and undo the throttling when the job should be completed. Also, add throttling in test_overlapping_4, or the jobs may not be so overlapping after all. In fact, the error usually emitted here is that node2 simply does not exist, not that overlapping jobs are not allowed -- the fact that this job ignores the exact error messages and just checks the error class is something that should be fixed in a follow-up patch. Signed-off-by: Max Reitz <mreitz@redhat.com> Tested-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 20190703172813.6868-8-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15block: Deep-clear inherits_fromMax Reitz1-10/+24
BDS.inherits_from does not always point to an immediate parent node. When launching a block job with a filter node, for example, the node directly below the filter will not point to the filter, but keep its old pointee (above the filter). If that pointee goes away while the job is still running, the node's inherits_from will not be updated and thus point to garbage. To fix this, bdrv_unref_child() has to check not only the parent node's immediate children for nodes whose inherits_from needs to be cleared, but its whole subtree. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190703172813.6868-7-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15block/stream: Swap backing file change orderMax Reitz1-1/+1
bdrv_change_backing_file() can result in yields. Therefore, @base may no longer be the the backing_bs() of s->bottom afterwards. Just swap the order of the two calls to fix this. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190703172813.6868-4-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15block/stream: Fix error pathMax Reitz1-1/+1
As of commit c624b015bf14fe01f1e6452a36e63b3ea1ae4998, the stream job only freezes the chain until the overlay of the base node. The error path must consider this. Fixes: c624b015bf14fe01f1e6452a36e63b3ea1ae4998 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190703172813.6868-3-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15block: Add BDS.never_freezeMax Reitz4-0/+19
The commit and the mirror block job must be able to drop their filter node at any point. However, this will not be possible if any of the BdrvChild links to them is frozen. Therefore, we need to prevent them from ever becoming frozen. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 20190703172813.6868-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15nvme: Set number of queues later in nvme_init()Michal Privoznik1-1/+1
When creating the admin queue in nvme_init() the variable that holds the number of queues created is modified before actual queue creation. This is a problem because if creating the queue fails then the variable is left in inconsistent state. This was actually observed when I tried to hotplug a nvme disk. The control got to nvme_file_open() which called nvme_init() which failed and thus nvme_close() was called which in turn called nvme_free_queue_pair() with queue being NULL. This lead to an instant crash: #0 0x000055d9507ec211 in nvme_free_queue_pair (bs=0x55d952ddb880, q=0x0) at block/nvme.c:164 #1 0x000055d9507ee180 in nvme_close (bs=0x55d952ddb880) at block/nvme.c:729 #2 0x000055d9507ee3d5 in nvme_file_open (bs=0x55d952ddb880, options=0x55d952bb1410, flags=147456, errp=0x7ffd8e19e200) at block/nvme.c:781 #3 0x000055d9507629f3 in bdrv_open_driver (bs=0x55d952ddb880, drv=0x55d95109c1e0 <bdrv_nvme>, node_name=0x0, options=0x55d952bb1410, open_flags=147456, errp=0x7ffd8e19e310) at block.c:1291 #4 0x000055d9507633d6 in bdrv_open_common (bs=0x55d952ddb880, file=0x0, options=0x55d952bb1410, errp=0x7ffd8e19e310) at block.c:1551 #5 0x000055d950766881 in bdrv_open_inherit (filename=0x0, reference=0x0, options=0x55d952bb1410, flags=32768, parent=0x55d9538ce420, child_role=0x55d950eaade0 <child_file>, errp=0x7ffd8e19e510) at block.c:3063 #6 0x000055d950765ae4 in bdrv_open_child_bs (filename=0x0, options=0x55d9541cdff0, bdref_key=0x55d950af33aa "file", parent=0x55d9538ce420, child_role=0x55d950eaade0 <child_file>, allow_none=true, errp=0x7ffd8e19e510) at block.c:2712 #7 0x000055d950766633 in bdrv_open_inherit (filename=0x0, reference=0x0, options=0x55d9541cdff0, flags=0, parent=0x0, child_role=0x0, errp=0x7ffd8e19e908) at block.c:3011 #8 0x000055d950766dba in bdrv_open (filename=0x0, reference=0x0, options=0x55d953d00390, flags=0, errp=0x7ffd8e19e908) at block.c:3156 #9 0x000055d9507cb635 in blk_new_open (filename=0x0, reference=0x0, options=0x55d953d00390, flags=0, errp=0x7ffd8e19e908) at block/block-backend.c:389 #10 0x000055d950465ec5 in blockdev_init (file=0x0, bs_opts=0x55d953d00390, errp=0x7ffd8e19e908) at blockdev.c:602 Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Message-id: 927aae40b617ba7d4b6c7ffe74e6d7a2595f8e86.1562770546.git.mprivozn@redhat.com Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-07-15migration: always initial RAMBlock.bmap to 1 for new migrationIvan Ren1-7/+8
Reproduce the problem: migrate migrate_cancel migrate Error happen for memory migration The reason as follows: 1. qemu start, ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION] all set to 1 by a series of cpu_physical_memory_set_dirty_range 2. migration start:ram_init_bitmaps - memory_global_dirty_log_start: begin log diry - memory_global_dirty_log_sync: sync dirty bitmap to ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION] - migration_bitmap_sync_range: sync ram_list. dirty_memory[DIRTY_MEMORY_MIGRATION] to RAMBlock.bmap and ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION] is set to zero 3. migration data... 4. migrate_cancel, will stop log dirty 5. migration start:ram_init_bitmaps - memory_global_dirty_log_start: begin log diry - memory_global_dirty_log_sync: sync dirty bitmap to ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION] - migration_bitmap_sync_range: sync ram_list. dirty_memory[DIRTY_MEMORY_MIGRATION] to RAMBlock.bmap and ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION] is set to zero Here RAMBlock.bmap only have new logged dirty pages, don't contain the whole guest pages. Signed-off-by: Ivan Ren <ivanren@tencent.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Message-Id: <1563115879-2715-1-git-send-email-ivanren@tencent.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15migration/postcopy: remove redundant cpu_synchronize_all_post_initWei Yang1-1/+0
cpu_synchronize_all_post_init() is called twice in loadvm_postcopy_handle_run_bh(), so remove one redundant call. Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Message-Id: <20190715080751.24304-1-richardw.yang@linux.intel.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15migration/postcopy: fix document of postcopy_send_discard_bm_ram()Wei Yang1-3/+2
Commit 6b6712efccd3 ('ram: Split dirty bitmap by RAMBlock') changes the parameter of postcopy_send_discard_bm_ram(), while left the document part untouched. This patch correct the document and fix two typo by hand. Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20190715020549.15018-1-richardw.yang@linux.intel.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15migration: allow private destination ram with x-ignore-sharedPeng Tao1-7/+0
By removing the share ram check, qemu is able to migrate to private destination ram when x-ignore-shared capability is on. Then we can create multiple destination VMs based on the same source VM. This changes the x-ignore-shared migration capability to work similar to Lai's original bypass-shared-memory work(https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg00003.html) which enables kata containers (https://katacontainers.io) to implement the VM templating feature. An example usage in kata containers(https://katacontainers.io): 1. Start the source VM: qemu-system-x86 -m 2G \ -object memory-backend-file,id=mem0,size=2G,share=on,mem-path=/tmpfs/template-memory \ -numa node,memdev=mem0 2. Stop the template VM, set migration x-ignore-shared capability, migrate "exec:cat>/tmpfs/state", quit it 3. Start target VM: qemu-system-x86 -m 2G \ -object memory-backend-file,id=mem0,size=2G,share=off,mem-path=/tmpfs/template-memory \ -numa node,memdev=mem0 \ -incoming defer 4. connect to target VM qmp, set migration x-ignore-shared capability, migrate_incoming "exec:cat /tmpfs/state" 5. create more target VMs repeating 3 and 4 Cc: Dr. David Alan Gilbert <dgilbert@redhat.com> Cc: Yury Kotov <yury-kotov@yandex-team.ru> Cc: Jiangshan Lai <laijs@hyper.sh> Cc: Xu Wang <xu@hyper.sh> Signed-off-by: Peng Tao <tao.peng@linux.alibaba.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <1560494113-1141-1-git-send-email-tao.peng@linux.alibaba.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15migration: Split log_clear() into smaller chunksPeter Xu5-2/+150
Currently we are doing log_clear() right after log_sync() which mostly keeps the old behavior when log_clear() was still part of log_sync(). This patch tries to further optimize the migration log_clear() code path to split huge log_clear()s into smaller chunks. We do this by spliting the whole guest memory region into memory chunks, whose size is decided by MigrationState.clear_bitmap_shift (an example will be given below). With that, we don't do the dirty bitmap clear operation on the remote node (e.g., KVM) when we fetch the dirty bitmap, instead we explicitly clear the dirty bitmap for the memory chunk for each of the first time we send a page in that chunk. Here comes an example. Assuming the guest has 64G memory, then before this patch the KVM ioctl KVM_CLEAR_DIRTY_LOG will be a single one covering 64G memory. If after the patch, let's assume when the clear bitmap shift is 18, then the memory chunk size on x86_64 will be 1UL<<18 * 4K = 1GB. Then instead of sending a big 64G ioctl, we'll send 64 small ioctls, each of the ioctl will cover 1G of the guest memory. For each of the 64 small ioctls, we'll only send if any of the page in that small chunk was going to be sent right away. Signed-off-by: Peter Xu <peterx@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20190603065056.25211-12-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15kvm: Support KVM_CLEAR_DIRTY_LOGPeter Xu2-0/+183
Firstly detect the interface using KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 and mark it. When failed to enable the new feature we'll fall back to the old sync. Provide the log_clear() hook for the memory listeners for both address spaces of KVM (normal system memory, and SMM) and deliever the clear message to kernel. Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20190603065056.25211-11-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15kvm: Introduce slots lock for memory listenerPeter Xu2-12/+48
Introduce KVMMemoryListener.slots_lock to protect the slots inside the kvm memory listener. Currently it is close to useless because all the KVM code path now is always protected by the BQL. But it'll start to make sense in follow up patches where we might do remote dirty bitmap clear and also we'll update the per-slot cached dirty bitmap even without the BQL. So let's prepare for it. We can also use per-slot lock for above reason but it seems to be an overkill. Let's just use this bigger one (which covers all the slots of a single address space) but anyway this lock is still much smaller than the BQL. Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20190603065056.25211-10-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15kvm: Persistent per kvmslot dirty bitmapPeter Xu2-3/+9
When synchronizing dirty bitmap from kernel KVM we do it in a per-kvmslot fashion and we allocate the userspace bitmap for each of the ioctl. This patch instead make the bitmap cache be persistent then we don't need to g_malloc0() every time. More importantly, the cached per-kvmslot dirty bitmap will be further used when we want to add support for the KVM_CLEAR_DIRTY_LOG and this cached bitmap will be used to guarantee we won't clear any unknown dirty bits otherwise that can be a severe data loss issue for migration code. Signed-off-by: Peter Xu <peterx@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20190603065056.25211-9-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15kvm: Update comments for sync_dirty_bitmapPeter Xu1-6/+6
It's obviously obsolete. Do some update. Signed-off-by: Peter Xu <peterx@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Message-Id: <20190603065056.25211-8-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15memory: Introduce memory listener hook log_clear()Peter Xu4-0/+83
Introduce a new memory region listener hook log_clear() to allow the listeners to hook onto the points where the dirty bitmap is cleared by the bitmap users. Previously log_sync() contains two operations: - dirty bitmap collection, and, - dirty bitmap clear on remote site. Let's take KVM as example - log_sync() for KVM will first copy the kernel dirty bitmap to userspace, and at the same time we'll clear the dirty bitmap there along with re-protecting all the guest pages again. We add this new log_clear() interface only to split the old log_sync() into two separated procedures: - use log_sync() to collect the collection only, and, - use log_clear() to clear the remote dirty bitmap. With the new interface, the memory listener users will still be able to decide how to implement the log synchronization procedure, e.g., they can still only provide log_sync() method only and put all the two procedures within log_sync() (that's how the old KVM works before KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 is introduced). However with this new interface the memory listener users will start to have a chance to postpone the log clear operation explicitly if the module supports. That can really benefit users like KVM at least for host kernels that support KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2. There are three places that can clear dirty bits in any one of the dirty bitmap in the ram_list.dirty_memory[3] array: cpu_physical_memory_snapshot_and_clear_dirty cpu_physical_memory_test_and_clear_dirty cpu_physical_memory_sync_dirty_bitmap Currently we hook directly into each of the functions to notify about the log_clear(). Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20190603065056.25211-7-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15memory: Pass mr into snapshot_and_clear_dirtyPeter Xu3-4/+4
Also we change the 2nd parameter of it to be the relative offset within the memory region. This is to be used in follow up patches. Signed-off-by: Peter Xu <peterx@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Message-Id: <20190603065056.25211-6-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15bitmap: Add bitmap_copy_with_{src|dst}_offset()Peter Xu4-0/+168
These helpers copy the source bitmap to destination bitmap with a shift either on the src or dst bitmap. Meanwhile, we never have bitmap tests but we should. This patch also introduces the initial test cases for utils/bitmap.c but it only tests the newly introduced functions. Signed-off-by: Peter Xu <peterx@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Message-Id: <20190603065056.25211-5-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com> --- Bitmap test used sizeof(unsigned long) instead of BITS_PER_LONG.
2019-07-15memory: Don't set migration bitmap when without migrationPeter Xu3-2/+14
Similar to 9460dee4b2 ("memory: do not touch code dirty bitmap unless TCG is enabled", 2015-06-05) but for the migration bitmap - we can skip the MIGRATION bitmap update if migration not enabled. Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20190603065056.25211-4-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15migration: No need to take rcu during sync_dirty_bitmapPeter Xu2-4/+2
cpu_physical_memory_sync_dirty_bitmap() has one RAMBlock* as parameter, which means that it must be with RCU read lock held already. Taking it again inside seems redundant. Removing it. Instead comment on the functions about the RCU read lock. Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20190603065056.25211-2-peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15migration/ram.c: reset complete_round when we gets a queued pageWei Yang1-0/+6
In case we gets a queued page, the order of block is interrupted. We may not rely on the complete_round flag to say we have already searched the whole blocks on the list. Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Message-Id: <20190605010828.6969-1-richardw.yang@linux.intel.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2019-07-15migration/multifd: sync packet_num after all thread are doneWei Yang1-4/+4
Notification from recv thread is not ordered, which means we may be notified by one MultiFDRecvParams but adjust packet_num for another. Move the adjustment after we are sure each recv thread are sync-ed. Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Message-Id: <20190604023540.26532-1-richardw.yang@linux.intel.com> Signed-off-by: Juan Quintela <quintela@redhat.com>