summary refs log tree commit diff stats
path: root/tests/qemu-iotests (follow)
Commit message (Collapse)AuthorAgeFilesLines
...
| * tests/qemu-iotests: validate NBD TLS with UNIX socketsDaniel P. Berrangé2022-03-072-0/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | This validates that connections to an NBD server running on a UNIX socket can use TLS, and require a TLS hostname override to pass certificate validation. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20220304193610.3293146-12-berrange@redhat.com> [eblake: squash in rebase fix] Tested-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * tests/qemu-iotests: validate NBD TLS with hostname mismatchDaniel P. Berrangé2022-03-073-3/+38
| | | | | | | | | | | | | | | | | | | | | | This validates that connections to an NBD server where the certificate hostname does not match will fail. It further validates that using the new 'tls-hostname' override option can solve the failure. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20220304193610.3293146-11-berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * tests/qemu-iotests: convert NBD TLS test to use standard filtersDaniel P. Berrangé2022-03-072-21/+16
| | | | | | | | | | | | | | | | | | Using standard filters is more future proof than rolling our own. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20220304193610.3293146-10-berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * tests/qemu-iotests: introduce filter for qemu-nbd export listDaniel P. Berrangé2022-03-073-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Introduce a filter for the output of qemu-nbd export list so it can be reused in multiple tests. The filter is a bit more permissive that what test 241 currently uses, as its allows printing of the export count, along with any possible error messages that might be emitted. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20220304193610.3293146-9-berrange@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * tests/qemu-iotests: expand _filter_nbd rulesDaniel P. Berrangé2022-03-071-0/+4
| | | | | | | | | | | | | | | | | | | | | | Some tests will want to use 'localhost' instead of '127.0.0.1', and some will use the image options syntax rather than the classic URI syntax. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20220304193610.3293146-8-berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
| * tests/qemu-iotests: add QEMU_IOTESTS_REGEN=1 to update reference fileDaniel P. Berrangé2022-03-071-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When developing an I/O test it is typical to add some logic to the test script, run it to view the output diff, and then apply the output diff to the reference file. This can be drastically simplified by letting the test runner update the reference file in place. By setting 'QEMU_IOTESTS_REGEN=1', the test runner will report the failure and show the diff, but at the same time update the reference file. So next time the I/O test is run it will succeed. Continuing to display the diff when updating the reference gives the developer a chance to review what was changed. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20220304193610.3293146-7-berrange@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
* | hw/isa/isa-bus: Remove isabus_dev_print()Bernhard Beschow2022-03-081-26/+0
|/ | | | | | | | | | | | | | | | | All isabus_dev_print() did was to print up to two IRQ numbers per device. This is redundant if the IRQ numbers are present as QOM properties (see e.g. the modified tests/qemu-iotests/172.out). Now that the last devices relying on isabus_dev_print() had their IRQ numbers QOM'ified, the contribution of this function ultimately became redundant. Remove it. Signed-off-by: Bernhard Beschow <shentey@gmail.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20220301220037.76555-5-shentey@gmail.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20220307134353.1950-12-philippe.mathieu.daude@gmail.com> Reviewed-by: Bernhard Beschow <shentey@gmail.com>
* iotests/image-fleecing: test push backup with fleecingVladimir Sementsov-Ogievskiy2022-03-072-32/+156
| | | | | | Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20220303194349.2304213-17-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests/image-fleecing: add test case with bitmapVladimir Sementsov-Ogievskiy2022-03-072-9/+113
| | | | | | | | | Note that reads zero areas (not dirty in the bitmap) fails, that's correct. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20220303194349.2304213-16-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests.py: add qemu_io_pipe_and_status()Vladimir Sementsov-Ogievskiy2022-03-071-0/+3
| | | | | | | | | Add helper that returns both status and output, to be used in the following commit Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20220303194349.2304213-15-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests/image-fleecing: add test-case for fleecing format nodeVladimir Sementsov-Ogievskiy2022-03-072-19/+119
| | | | | | | Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220303194349.2304213-14-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* block: copy-before-write: realize snapshot-access APIVladimir Sementsov-Ogievskiy2022-03-071-0/+224
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Current scheme of image fleecing looks like this: [guest] [NBD export] | | |root | root v v [copy-before-write] -----> [temp.qcow2] | target | |file |backing v | [active disk] <-------------+ - On guest writes copy-before-write filter copies old data from active disk to temp.qcow2. So fleecing client (NBD export) when reads changed regions from temp.qcow2 image and unchanged from active disk through backing link. This patch makes possible new image fleecing scheme: [guest] [NBD export] | | | root | root v file v [copy-before-write]<------[snapshot-access] | | | file | target v v [active-disk] [temp.img] - copy-before-write does CBW operations and also provides snapshot-access API. The API may be accessed through snapshot-access driver. Benefits of new scheme: 1. Access control: if remote client try to read data that not covered by original dirty bitmap used on copy-before-write open, client gets -EACCES. 2. Discard support: if remote client do DISCARD, this additionally to discarding data in temp.img informs block-copy process to not copy these clusters. Next read from discarded area will return -EACCES. This is significant thing: when fleecing user reads data that was not yet copied to temp.img, we can avoid copying it on further guest write. 3. Synchronisation between client reads and block-copy write is more efficient. In old scheme we just rely on BDRV_REQ_SERIALISING flag used for writes to temp.qcow2. New scheme is less blocking: - fleecing reads are never blocked: if data region is untouched or in-flight, we just read from active-disk, otherwise we read from temp.img - writes to temp.img are not blocked by fleecing reads - still, guest writes of-course are blocked by in-flight fleecing reads, that currently read from active-disk - it's the minimum necessary blocking 4. Temporary image may be of any format, as we don't rely on backing feature. 5. Permission relation are simplified. With old scheme we have to share write permission on target child of copy-before-write, otherwise backing link conflicts with copy-before-write file child write permissions. With new scheme we don't have backing link, and copy-before-write node may have unshared access to temporary node. (Not realized in this commit, will be in future). 6. Having control on fleecing reads we'll be able to implement alternative behavior on failed copy-before-write operations. Currently we just break guest request (that's a historical behavior of backup). But in some scenarios it's a bad behavior: better is to drop the backup as failed but don't break guest request. With new scheme we can simply unset some bits in a bitmap on CBW failure and further fleecing reads will -EACCES, or something like this. (Not implemented in this commit, will be in future) Additional application for this is implementing timeout for CBW operations. Iotest 257 output is updated, as two more bitmaps now live in copy-before-write filter. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20220303194349.2304213-13-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests: Write test output to TEST_DIRHanna Reitz2022-03-074-16/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Drop the use of OUTPUT_DIR (test/qemu-iotests under the build directory), and instead write test output files (.out.bad, .notrun, and .casenotrun) to TEST_DIR. With this, the same test can be run concurrently without the separate instances interfering, because they will need separate TEST_DIRs anyway. Running the same test separately is useful when running the iotests with various format/protocol combinations in parallel, or when you just want to aggressively exercise a single test (e.g. when it fails only sporadically). Putting this output into TEST_DIR means that it will stick around for inspection after the test run is done (though running the same test in the same TEST_DIR will overwrite it, just as it used to be); but given that TEST_DIR is a scratch directory, it should be clear that users can delete all of its content at any point. (And if TEST_DIR is on tmpfs, it will just disappear on shutdown.) Contrarily, alternative approaches that would put these output files into OUTPUT_DIR with some prefix to differentiate between separate test runs might easily lead to cluttering OUTPUT_DIR. (This change means OUTPUT_DIR is no longer written to by the iotests, so we can drop its usage altogether.) Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220221172909.762858-1-hreitz@redhat.com> [hreitz: Simplified `Path(os.path.join(x, y))` to `Path(x, y)`, as suggested by Vladimir; and rebased on 9086c7639822b6 ("tests/qemu-iotests: Rework the checks and spots using GNU sed")] Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* tests/qemu-iotests/testrunner: Quote "case not run" lines in TAP modeThomas Huth2022-03-071-1/+4
| | | | | | | | | | | | | | | | | | | In TAP mode, the stdout is reserved for the TAP protocol, so we have to make sure to mark other lines with a comment '#' character at the beginning to avoid that the TAP parser at the other end gets confused. To test this condition, run "configure" for example with: --block-drv-rw-whitelist=copy-before-write,qcow2,raw,file,host_device,blkdebug,null-co,copy-on-read so that iotest 041 will report that some tests are not run due to the missing "quorum" driver. Without this change, "make check-block" fails since the meson tap parser gets confused by these messages. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20220223124353.3273898-1-thuth@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* tests/qemu-iotests/040: Skip TestCommitWithFilters without 'throttle'Thomas Huth2022-03-071-0/+1
| | | | | | | | | | | | | iotest 040 already has some checks for the availability of the 'throttle' driver, but some new code has been added in the course of time that depends on 'throttle' but does not check for its availability. Add a check to the TestCommitWithFilters class so that this iotest now also passes again if 'throttle' has not been enabled in the QEMU binaries. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20220223123127.3206042-1-thuth@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* tests/qemu-iotests: Rework the checks and spots using GNU sedThomas Huth2022-03-043-55/+57
| | | | | | | | | | | | | | | | | | | | | Instead of failing the iotests if GNU sed is not available (or skipping them completely in the check-block.sh script), it would be better to simply skip the bash-based tests that rely on GNU sed, so that the other tests could still be run. Thus we now explicitely use "gsed" (either as direct program or as a wrapper around "sed" if it's the GNU version) in the spots that rely on the GNU sed behavior. Statements that use the "-r" parameter of sed have been switched to use "-E" instead, since this switch is supported by all sed versions on our supported build hosts (most also support "-r", but macOS' sed only supports "-E"). With all these changes in place, we then can also remove the sed checks from the check-block.sh script, so that "make check-block" can now be run on systems without GNU sed, too. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20220216125454.465041-1-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests/graph-changes-while-io: New testHanna Reitz2022-03-042-0/+96
| | | | | | | | | | | | | | | | | | | | | | | Test the following scenario: 1. Some block node (null-co) attached to a user (here: NBD server) that performs I/O and keeps the node in an I/O thread 2. Repeatedly run blockdev-add/blockdev-del to add/remove an overlay to/from that node Each blockdev-add triggers bdrv_refresh_limits(), and because blockdev-add runs in the main thread, it does not stop the I/O requests. I/O can thus happen while the limits are refreshed, and when such a request sees a temporarily invalid block limit (e.g. alignment is 0), this may easily crash qemu (or the storage daemon in this case). The block layer needs to ensure that I/O requests to a node are paused while that node's BlockLimits are refreshed. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20220216105355.30729-4-hreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests: Allow using QMP with the QSDHanna Reitz2022-03-041-1/+31
| | | | | | | | | | | Add a parameter to optionally open a QMP connection when creating a QemuStorageDaemon instance. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220216105355.30729-3-hreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests/185: Add post-READY quit testsHanna Reitz2022-03-042-1/+237
| | | | | | | | | | | | | | | 185 tests quitting qemu while a block job is active. It does not specifically test quitting qemu while a mirror or active commit job is in its READY phase. Add two test cases for this, where we respectively mirror or commit to an external QSD instance, which provides a throttled block device. qemu is supposed to cancel the job so that it can quit as soon as possible instead of waiting for the job to complete (which it did before 6.2). Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220303164814.284974-5-hreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* crypto: perform permission checks under BQLEmanuele Giuseppe Esposito2022-03-042-14/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move the permission API calls into driver-specific callbacks that always run under BQL. In this case, bdrv_crypto_luks needs to perform permission checks before and after qcrypto_block_amend_options(). The problem is that the caller, block_crypto_amend_options_generic_luks(), can also run in I/O from .bdrv_co_amend(). This does not comply with Global State-I/O API split, as permissions API must always run under BQL. Firstly, introduce .bdrv_amend_pre_run() and .bdrv_amend_clean() callbacks. These two callbacks are guaranteed to be invoked under BQL, respectively before and after .bdrv_co_amend(). They take care of performing the permission checks in the same way as they are currently done before and after qcrypto_block_amend_options(). These callbacks are in preparation for next patch, where we delete the original permission check. Right now they just add redundant control. Then, call .bdrv_amend_pre_run() before job_start in qmp_x_blockdev_amend(), so that it will be run before the job coroutine is created and stay in the main loop. As a cleanup, use JobDriver's .clean() callback to call .bdrv_amend_clean(), and run amend-specific cleanup callbacks under BQL. After this patch, permission failures occur early in the blockdev-amend job to update a LUKS volume's keys. iotest 296 must now expect them in x-blockdev-amend's QMP reply instead of waiting for the actual job to fail later. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Message-Id: <20220209105452.1694545-2-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220304153729.711387-6-hreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* configure, meson: move some default-disabled options to meson_options.txtPaolo Bonzini2022-02-211-1/+1
| | | | | | | These do not depend on --with-default-features, so they become booleans in meson too. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* tests/qemu-iotests/testrunner: Print diff to stderr in TAP modeThomas Huth2022-02-161-1/+4
| | | | | | | | | | When running in TAP mode, stdout is reserved for the TAP protocol. To see the "diff" of the failed test, we have to print it to stderr instead. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20220209101530.3442837-8-thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* Merge remote-tracking branch 'remotes/kwolf-gitlab/tags/for-upstream' into ↵Peter Maydell2022-02-142-5/+35
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | staging Block layer patches - Fix crash in blockdev-reopen with iothreads - fdc-isa: Respect QOM properties when building AML # gpg: Signature made Fri 11 Feb 2022 17:44:52 GMT # gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6 # gpg: issuer "kwolf@redhat.com" # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kwolf-gitlab/tags/for-upstream: hw/block/fdc-isa: Respect QOM properties when building AML iotests: Test blockdev-reopen with iothreads and throttling block: Lock AioContext for drain_end in blockdev-reopen Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
| * iotests: Test blockdev-reopen with iothreads and throttlingKevin Wolf2022-02-112-5/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | The 'throttle' block driver implements .bdrv_co_drain_end, so blockdev-reopen will have to wait for it to complete in the polling loop at the end of qmp_blockdev_reopen(). This makes AIO_WAIT_WHILE() release the AioContext lock, which causes a crash if the lock hasn't correctly been taken. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20220203140534.36522-3-kwolf@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* | iotests/281: Let NBD connection yield in iothreadHanna Reitz2022-02-112-5/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Put an NBD block device into an I/O thread, and then read data from it, hoping that the NBD connection will yield during that read. When it does, the coroutine must be reentered in the block device's I/O thread, which will only happen if the NBD block driver attaches the connection's QIOChannel to the new AioContext. It did not do that after 4ddb5d2fde ("block/nbd: drop connection_co") and prior to "block/nbd: Move s->ioc on AioContext change", which would cause an assertion failure. To improve our chances of yielding, the NBD server is throttled to reading 64 kB/s, and the NBD client reads 128 kB, so it should yield at some point. Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* | iotests/281: Test lingering timersHanna Reitz2022-02-112-4/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to "block/nbd: Delete reconnect delay timer when done" and "block/nbd: Delete open timer when done", both of those timers would remain scheduled even after successfully (re-)connecting to the server, and they would not even be deleted when the BDS is deleted. This test constructs exactly this situation: (1) Configure an @open-timeout, so the open timer is armed, and (2) Configure a @reconnect-delay and trigger a reconnect situation (which succeeds immediately), so the reconnect delay timer is armed. Then we immediately delete the BDS, and sleep for longer than the @open-timeout and @reconnect-delay. Prior to said patches, this caused one (or both) of the timer CBs to access already-freed data. Accessing freed data may or may not crash, so this test can produce false successes, but I do not know how to show the problem in a better or more reliable way. If you run this test on "block/nbd: Assert there are no timers when closed" and without the fix patches mentioned above, you should reliably see an assertion failure. (But all other tests that use the reconnect delay timer (264 and 277) will fail in that configuration, too; as will nbd-reconnect-on-open, which uses the open timer.) Remove this test from the quick group because of the two second sleep this patch introduces. (I decided to put this test case into 281, because the main bug this series addresses is in the interaction of the NBD block driver and I/O threads, which is precisely the scope of 281. The test case for that other bug will also be put into the test class added here. Also, excuse the test class's name, I couldn't come up with anything better. The "yield" part will make sense two patches from now.) Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* | iotests.py: Add QemuStorageDaemon classHanna Reitz2022-02-111-0/+40
|/ | | | | | | | | | | | | | | This is a rather simple class that allows creating a QSD instance running in the background and stopping it when no longer needed. The __del__ handler is a safety net for when something goes so wrong in a test that e.g. the tearDown() method is not called (e.g. setUp() launches the QSD, but then launching a VM fails). We do not want the QSD to continue running after the test has failed, so __del__() will take care to kill it. Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* python/machine: raise VMLaunchFailure exception from launch()John Snow2022-02-021-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | This allows us to pack in some extra information about the failure, which guarantees that if the caller did not *intentionally* cause a failure (by capturing this Exception), some pretty good clues will be printed at the bottom of the traceback information. This will help make failures in the event of a non-negative return code more obvious when they go unhandled; the current behavior in _post_shutdown() is to print a warning message only in the event of signal-based terminations (for negative return codes). (Note: In Python, catching BaseException instead of Exception catches a broader array of Exception events, including SystemExit and KeyboardInterrupt. We do not want to "wrap" such exceptions as a VMLaunchFailure, because that will 'downgrade' the exception from a BaseException to a regular Exception. We do, however, want to perform cleanup in either case, so catch on the broadest scope and wrap-and-re-raise only in the more targeted scope.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20220201041134.1237016-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/migration-permissions: New testHanna Reitz2022-02-012-0/+106
| | | | | | | This test checks that a raw image in use by a virtio-blk device does not share the WRITE permission both before and after migration. Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests: declare lack of support for compresion_type in IMGOPTSVladimir Sementsov-Ogievskiy2022-02-015-7/+14
| | | | | | | | | | compression_type can't be used if we want to create image with compat=0.10. So, skip these tests, not many of them. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-20-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotest 214: explicit compression typeVladimir Sementsov-Ogievskiy2022-02-011-1/+1
| | | | | | | | | | | The test-case "Corrupted size field in compressed cluster descriptor" heavily depends on zlib compression type. So, make it explicit. This way test passes with IMGOPTS='compression_type=zstd'. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-19-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests 60: more accurate set dirty bit in qcow2 headerVladimir Sementsov-Ogievskiy2022-02-011-1/+1
| | | | | | | | | | Don't touch other incompatible bits, like compression-type. This makes the test pass with IMGOPTS='compression_type=zstd'. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-18-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests: bash tests: filter compression typeVladimir Sementsov-Ogievskiy2022-02-017-21/+41
| | | | | | | | | | | | | | We want iotests pass with both the default zlib compression and with IMGOPTS='compression_type=zstd'. Actually the only test that is interested in real compression type in test output is 287 (test for qcow2 compression type), so implement specific option for it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-17-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotest 39: use _qcow2_dump_headerVladimir Sementsov-Ogievskiy2022-02-011-1/+1
| | | | | | | | | | _qcow2_dump_header has filter for compression type, so this change makes test pass with IMGOPTS='compression_type=zstd'. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-16-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests: massive use _qcow2_dump_headerVladimir Sementsov-Ogievskiy2022-02-017-49/+49
| | | | | | | | | | | | | | | | We are going to add filtering in _qcow2_dump_header and want all tests use it. The patch is generated by commands: cd tests/qemu-iotests sed -ie 's/$PYTHON qcow2.py "$TEST_IMG" dump-header\($\| \)/_qcow2_dump_header\1/' ??? tests/* (the difficulty is to avoid converting dump-header-exts) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-15-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests/common.rc: introduce _qcow2_dump_header helperVladimir Sementsov-Ogievskiy2022-02-011-0/+10
| | | | | | | | | | We'll use it in tests instead of explicit qcow2.py. Then we are going to add some filtering in _qcow2_dump_header. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-14-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotest 302: use img_info_log() helperVladimir Sementsov-Ogievskiy2022-02-012-5/+6
| | | | | | | | | | | | | | | | | | | | | | Instead of qemu_img_log("info", ..) use generic helper img_info_log(). img_info_log() has smarter logic. For example it use filter_img_info() to filter output, which in turns filter a compression type. So it will help us in future when we implement a possibility to use zstd compression by default (with help of some runtime config file or maybe build option). For now to test you should recompile qemu with a small addition into block/qcow2.c before "if (qcow2_opts->has_compression_type": if (!qcow2_opts->has_compression_type && version >= 3) { qcow2_opts->has_compression_type = true; qcow2_opts->compression_type = QCOW2_COMPRESSION_TYPE_ZSTD; } Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-12-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests.py: filter compression type outVladimir Sementsov-Ogievskiy2022-02-014-15/+17
| | | | | | | | | | | | | | | We want iotests pass with both the default zlib compression and with IMGOPTS='compression_type=zstd'. Actually the only test that is interested in real compression type in test output is 287 (test for qcow2 compression type) and it's in bash. So for now we can safely filter out compression type in all qcow2 tests. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-11-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests.py: filter out successful output of qemu-img createVladimir Sementsov-Ogievskiy2022-02-017-67/+11
| | | | | | | | | | The only "feature" of this "Formatting ..." line is that we have to update it every time we add new option. Let's drop it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-10-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotest 065: explicit compression typeVladimir Sementsov-Ogievskiy2022-02-011-8/+8
| | | | | | | | | | | | The test checks different options. It of course fails if set IMGOPTS='compression_type=zstd'. So, let's be explicit in what compression type we want and independent of IMGOPTS. Test both existing compression types. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-9-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotest 303: explicit compression typeVladimir Sementsov-Ogievskiy2022-02-012-10/+45
| | | | | | | | | | | The test prints qcow2 header fields which depends on chosen compression type. So, let's be explicit in what compression type we want and independent of IMGOPTS. Test both existing compression types. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-8-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests.py: rewrite default luks support in qemu_imgVladimir Sementsov-Ogievskiy2022-02-011-19/+17
| | | | | | | | | | | Move the logic to more generic qemu_img_pipe_and_status(). Also behave better when we have several -o options. And reuse argument parser of course. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-7-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests: drop qemu_img_verbose() helperVladimir Sementsov-Ogievskiy2022-02-015-13/+10
| | | | | | | | | | | | qemu_img_verbose() has a drawback of not going through generic qemu_img_pipe_and_status(). qemu_img_verbose() is not very popular, so update the only two users to qemu_img_log() and drop qemu_img_verbose() at all. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-6-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests.py: qemu_img*("create"): support IMGOPTS='compression_type=zstd'Vladimir Sementsov-Ogievskiy2022-02-011-1/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adding support of IMGOPTS (like in bash tests) allows user to pass a lot of different options. Still, some may require additional logic. Now we want compression_type option, so add some smart logic around it: ignore compression_type=zstd in IMGOPTS, if test want qcow2 in compatibility mode. As well, ignore compression_type for non-qcow2 formats. Note that we may instead add support only to qemu_img_create(), but that works bad: 1. We'll have to update a lot of tests to use qemu_img_create instead of qemu_img('create'). (still, we may want do it anyway, but no reason to create a dependancy between task of supporting IMGOPTS and updating a lot of tests) 2. Some tests use qemu_img_pipe('create', ..) - even more work on updating 3. Even if we update all tests to go through qemu_img_create, we'll need a way to avoid creating new tests using qemu_img*('create') - add assertions.. That doesn't seem good. So, let's add support of IMGOPTS to most generic qemu_img_pipe_and_status(). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-5-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests: specify some unsupported_imgopts for python iotestsVladimir Sementsov-Ogievskiy2022-02-0115-15/+30
| | | | | | | | | | | We are going to support IMGOPTS for python iotests. Still some iotests will not work with common IMGOPTS used with bash iotests like specifying refcount_bits and compat qcow2 options. So we should define corresponding unsupported_imgopts for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211223160144.1097696-4-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests.py: implement unsupported_imgoptsVladimir Sementsov-Ogievskiy2022-02-011-1/+14
| | | | | | | | | | | | | | | | We are going to support some addition IMGOPTS in python iotests like in bash iotests. Similarly to bash iotests, we want a way to skip some tests which can't work with specific IMGOPTS. Globally for python iotests we will not support things like 'data_file=$TEST_IMG.ext_data_file' in IMGOPTS, so, forbid this globally in iotests.py. Suggested-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-3-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests.py: img_info_log(): rename imgopts argumentVladimir Sementsov-Ogievskiy2022-02-012-6/+7
| | | | | | | | | | | | We are going to support IMGOPTS environment variable like in bash tests. Corresponding global variable in iotests.py should be called imgopts. So to not interfere with function argument, rename it in advance. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-2-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests/MRCE: Write data to sourceHanna Reitz2022-02-011-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This test assumes that mirror flushes the source when entering the READY state, and that the format level will pass that flush on to the protocol level (where we intercept it with blkdebug). However, apparently that does not happen when using a VMDK image with zeroed_grain=on, which actually is the default set by testenv.py. Right now, Python tests ignore IMGOPTS, though, so this has no effect; but Vladimir has a series that will change this, so we need to fix this test before that series lands. We can fix it by writing data to the source before we start the mirror job; apparently that makes the (VMDK) format layer change its mind and pass on the pre-READY flush to the protocol level, so the test passes again. (I presume, without any data written, mirror just does a 64M zero write on the target, which VMDK with zeroed_grain=on basically just ignores.) Without this, we do not get a flush, and so blkdebug only sees a single flush at the end of the job instead of two, and therefore does not inject an error, which makes the block job complete instead of raising an error. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223165308.103793-1-hreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* tests/qemu-iotests: Fix 051 for binaries without 'lsi53c895a'Thomas Huth2022-02-011-0/+4
| | | | | | | | | | | The lsi53c895a SCSI adaptor might not be enabled in each and every x86 QEMU binary, e.g. it's disabled in the RHEL/CentOS build. Thus let's add a check to the 051 test so that it does not fail if this device is not available. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20211206143404.247032-1-thuth@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests/block-status-cache: New testHanna Reitz2022-01-282-0/+144
| | | | | | | | | | | | | | | | | | | Add a new test to verify that want_zero=false block-status calls do not pollute the block-status cache for want_zero=true calls. We check want_zero=true calls and their results using `qemu-img map` (over NBD), and want_zero=false calls also using `qemu-img map` over NBD, but using the qemu:allocation-depth context. (This test case cannot be integrated into nbd-qemu-allocation, because that is a qcow2 test, and this is a raw test.) Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220118170000.49423-3-hreitz@redhat.com> Reviewed-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>