summary refs log tree commit diff stats
path: root/tests/qemu-iotests (follow)
Commit message (Collapse)AuthorAgeFilesLines
...
* check-block: replace -makecheck with TAP outputPaolo Bonzini2022-01-284-41/+45
| | | | | | | Let "meson test" take care of showing the results of the individual tests, consistently with other output from "make check V=1". Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* build: make check-block a meson testPaolo Bonzini2022-01-281-0/+29
| | | | | | | | | | | | "meson test" can be asked to run tests verbosely; this makes it usable also for qemu-iotests's own harness, and it lets "make check-block" reuse mtest2make.py's infrastructure to find and build test dependencies. Adjust check-block.sh to use the standard exit code that reports a test as skipped. Alternatively, in the future we could make it produce TAP output, which is consistent with all other "make check" tests. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* iotests/testrunner.py: refactor test_field_widthVladimir Sementsov-Ogievskiy2022-01-141-11/+10
| | | | | | | | | | | | | | | | A lot of Optional[] types doesn't make code beautiful. test_field_width defaults to 8, but that is never used in the code. More over, if we want some default behavior for single call of test_run(), it should just print the whole test name, not limiting or expanding its width, so 8 is bad default. So, just drop the default as unused for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211210201450.101576-1-vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* block: drop BLK_PERM_GRAPH_MODVladimir Sementsov-Ogievskiy2022-01-141-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | First, this permission never protected a node from being changed, as generic child-replacing functions don't check it. Second, it's a strange thing: it presents a permission of parent node to change its child. But generally, children are replaced by different mechanisms, like jobs or qmp commands, not by nodes. Graph-mod permission is hard to understand. All other permissions describe operations which done by parent node on its child: read, write, resize. Graph modification operations are something completely different. The only place where BLK_PERM_GRAPH_MOD is used as "perm" (not shared perm) is mirror_start_job, for s->target. Still modern code should use bdrv_freeze_backing_chain() to protect from graph modification, if we don't do it somewhere it may be considered as a bug. So, it's a bit risky to drop GRAPH_MOD, and analyzing of possible loss of protection is hard. But one day we should do it, let's do it now. One more bit of information is that locking the corresponding byte in file-posix doesn't make sense at all. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210902093754.2352-1-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* qemu-img: make is_allocated_sectors() more efficientVladimir Sementsov-Ogievskiy2022-01-141-6/+2
| | | | | | | | | | | | | | | | | | | Consider the case when the whole buffer is zero and end is unaligned. If i <= tail, we return 1 and do one unaligned WRITE, RMW happens. If i > tail, we do on aligned WRITE_ZERO (or skip if target is zeroed) and again one unaligned WRITE, RMW happens. Let's do better: don't fragment the whole-zero buffer and report it as ZERO: in case of zeroed target we just do nothing and avoid RMW. If target is not zeroes, one unaligned WRITE_ZERO should not be much worse than one unaligned WRITE. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211217164654.1184218-3-vsementsov@virtuozzo.com> Tested-by: Peter Lieven <pl@kamp.de> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests: Test qemu-img convert of zeroed data clusterKevin Wolf2022-01-142-2/+9
| | | | | | | | | | | | | | This demonstrates what happens when the block status changes in sub-min_sparse granularity, but all of the parts are zeroed out. The alignment logic in is_allocated_sectors() prevents that the target image remains fully sparse as expected, but turns it into a data cluster of explicit zeros. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211217164654.1184218-2-vsementsov@virtuozzo.com> Tested-by: Peter Lieven <pl@kamp.de> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests/308: Fix for CAP_DAC_OVERRIDEHanna Reitz2022-01-142-3/+24
| | | | | | | | | | | | | | | | | | | With CAP_DAC_OVERRIDE (which e.g. root generally has), permission checks will be bypassed when opening files. 308 in one instance tries to open a read-only file (FUSE export) with qemu-io as read/write, and expects this to fail. However, when running it as root, opening will succeed (thanks to CAP_DAC_OVERRIDE) and only the actual write operation will fail. Note this as "Case not run", but have the test pass in either case. Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Fixes: 2c7dd057aa7bd7a875e9b1a53975c220d6380bc4 ("export/fuse: Pass default_permissions for mount") Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220103120014.13061-1-hreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests/stream-error-on-reset: New testHanna Reitz2022-01-142-0/+145
| | | | | | | | | | | | | | | | | Test the following scenario: - Simple stream block in two-layer backing chain (base and top) - The job is drained via blk_drain(), then an error occurs while the job settles the ongoing request - And so the job completes while in blk_drain() This was reported as a segfault, but is fixed by "block-backend: prevent dangling BDS pointers across aio_poll()". Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2036178 Signed-off-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20220111153613.25453-3-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* blockjob: drop BlockJob.blk fieldVladimir Sementsov-Ogievskiy2021-12-283-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's unused now (except for permission handling)[*]. The only reasonable user of it was block-stream job, recently updated to use own blk. And other block jobs prefer to use own source node related objects. So, the arguments of dropping the field are: - block jobs prefer not to use it - block jobs usually has more then one node to operate on, and better to operate symmetrically (for example has both source and target blk's in specific block-job state structure) *: BlockJob.blk is used to keep some permissions. We simply move permissions to block-job child created in block_job_create() together with blk. In mirror, we just should not care anymore about restoring state of blk. Most probably this code could be dropped long ago, after dropping bs->job pointer. Now it finally goes away together with BlockJob.blk itself. iotest 141 output is updated, as "bdrv_has_blk(bs)" check in qmp_blockdev_del() doesn't fail (we don't have blk now). Still, new error message looks even better. In iotest 283 we need to add a job id, otherwise "Invalid job ID" happens now earlier than permission check (as permissions moved from blk to block-job node). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
* Merge tag 'pull-nbd-2021-12-22-v2' of ↵Richard Henderson2021-12-233-13/+106
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://src.openvz.org/scm/~vsementsov/qemu into staging nbd: reconnect-on-open feature v2: simple fix for mypy and pylint complains on patch 04 # gpg: Signature made Thu 23 Dec 2021 12:45:20 AM PST # gpg: using RSA key 8B9C26CDB2FD147C880E86A1561F24C1F19F79FB # gpg: Good signature from "Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>" [unknown] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: 8B9C 26CD B2FD 147C 880E 86A1 561F 24C1 F19F 79FB * tag 'pull-nbd-2021-12-22-v2' of https://src.openvz.org/scm/~vsementsov/qemu: iotests: add nbd-reconnect-on-open test iotests.py: add qemu_io_popen() iotests.py: add and use qemu_io_wrap_args() iotests.py: add qemu_tool_popen() nbd/client-connection: improve error message of cancelled attempt nbd/client-connection: nbd_co_establish_connection(): return real error nbd: allow reconnect on open, with corresponding new options Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
| * iotests: add nbd-reconnect-on-open testVladimir Sementsov-Ogievskiy2021-12-232-0/+82
| | | | | | | | | | Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
| * iotests.py: add qemu_io_popen()Vladimir Sementsov-Ogievskiy2021-12-231-0/+3
| | | | | | | | | | | | | | | | Add qemu-io Popen constructor wrapper. To be used in the following new test commit. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
| * iotests.py: add and use qemu_io_wrap_args()Vladimir Sementsov-Ogievskiy2021-12-231-10/+10
| | | | | | | | | | | | | | | | | | | | | | For qemu_io* functions support --image-opts argument, which conflicts with -f argument from qemu_io_args. For QemuIoInteractive use new wrapper as well, which allows relying on default format. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
| * iotests.py: add qemu_tool_popen()Vladimir Sementsov-Ogievskiy2021-12-231-3/+11
| | | | | | | | | | | | | | | | Split qemu_tool_popen() from qemu_tool_pipe_and_status() to be used separately. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
* | iotests: check: multiprocessing supportVladimir Sementsov-Ogievskiy2021-12-222-9/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add -j <JOBS> parameter, to run tests in several jobs simultaneously. For realization - simply utilize multiprocessing.Pool class. Notes: 1. Of course, tests can't run simultaneously in same TEST_DIR. So, use subdirectories TEST_DIR/testname/ and SOCK_DIR/testname/ instead of simply TEST_DIR and SOCK_DIR 2. multiprocessing.Pool.starmap function doesn't support passing context managers, so we can't simply pass "self". Happily, we need self only for read-only access, and it just works if it is defined in global space. So, add a temporary link TestRunner.shared_self during run_tests(). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211203122223.2780098-4-vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Tested-by: John Snow <jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* | iotests/testrunner.py: move updating last_elapsed to run_testsVladimir Sementsov-Ogievskiy2021-12-221-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are going to use do_run_test() in multiprocessing environment, where we'll not be able to change original runner object. Happily, the only thing we change is that last_elapsed and it's simple to do it in run_tests() instead. All other accesses to self in do_runt_test() and in run_test() are read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211203122223.2780098-3-vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Tested-by: John Snow <jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* | iotests/testrunner.py: add doc string for run_test()Vladimir Sementsov-Ogievskiy2021-12-221-0/+13
|/ | | | | | | | | | | We are going to modify these methods and will add more documentation in further commit. As a preparation add basic documentation. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211203122223.2780098-2-vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Tested-by: John Snow <jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests/149: Skip on unsupported ciphersHanna Reitz2021-11-231-5/+18
| | | | | | | | | | | | | | | | Whenever qemu-img or qemu-io report that some cipher is unsupported, skip the whole test, because that is probably because qemu has been configured with the gnutls crypto backend. We could taylor the algorithm list to what gnutls supports, but this is a test that is run rather rarely anyway (because it requires password-less sudo), and so it seems better and easier to skip it. When this test is intentionally run to check LUKS compatibility, it seems better not to limit the algorithms but keep the list extensive. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211117151707.52549-3-hreitz@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
* iotests: Use aes-128-cbcHanna Reitz2021-11-234-10/+10
| | | | | | | | | | | | | | | | Our gnutls crypto backend (which is the default as of 8bd0931f6) supports neither twofish-128 nor the CTR mode. CBC and aes-128 are supported by all of our backends (as far as I can tell), so use aes-128-cbc in our iotests. (We could also use e.g. aes-256-cbc, but the different key sizes would lead to different key slot offsets and so change the reference output more, which is why I went with aes-128.) Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211117151707.52549-2-hreitz@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com>
* file-posix: Fix alignment after reopen changing O_DIRECTKevin Wolf2021-11-162-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | At the end of a reopen, we already call bdrv_refresh_limits(), which should update bs->request_alignment according to the new file descriptor. However, raw_probe_alignment() relies on s->needs_alignment and just uses 1 if it isn't set. We neglected to update this field, so starting with cache=writeback and then reopening with cache=none means that we get an incorrect bs->request_alignment == 1 and unaligned requests fail instead of being automatically aligned. Fix this by recalculating s->needs_alignment in raw_refresh_limits() before calling raw_probe_alignment(). Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20211104113109.56336-1-kwolf@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20211115145409.176785-13-kwolf@redhat.com> [hreitz: Fix iotest 142 for block sizes greater than 512 by operating on a file with a size of 1 MB] Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211116101431.105252-1-hreitz@redhat.com>
* iotests/030: Unthrottle parallel jobs in reverseHanna Reitz2021-11-161-1/+10
| | | | | | | | | | See the comment for why this is necessary. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211111120829.81329-11-hreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20211115145409.176785-11-kwolf@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* Merge remote-tracking branch 'remotes/kwolf/tags/for-upstream' into stagingRichard Henderson2021-11-032-4/+60
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Block layer patches - Fail gracefully when blockdev-snapshot creates loops - ide: Fix IDENTIFY DEVICE for disks > 128 GiB - file-posix: Fix return value translation for AIO discards - file-posix: add 'aio-max-batch' option - rbd: implement bdrv_co_block_status - Code cleanups and build fixes # gpg: Signature made Tue 02 Nov 2021 12:04:02 PM EDT # gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6 # gpg: issuer "kwolf@redhat.com" # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] * remotes/kwolf/tags/for-upstream: block/nvme: Extract nvme_free_queue() from nvme_free_queue_pair() block/nvme: Display CQ/SQ pointer in nvme_free_queue_pair() block/nvme: Automatically free qemu_memalign() with QEMU_AUTO_VFREE block-backend: Silence clang -m32 compiler warning linux-aio: add `dev_max_batch` parameter to laio_io_unplug() linux-aio: add `dev_max_batch` parameter to laio_co_submit() file-posix: add `aio-max-batch` option block/export/fuse.c: fix musl build ide: Cap LBA28 capacity announcement to 2^28-1 block/rbd: implement bdrv_co_block_status block: Fail gracefully when blockdev-snapshot creates loops block/file-posix: Fix return value translation for AIO discards Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
| * block: Fail gracefully when blockdev-snapshot creates loopsKevin Wolf2021-11-022-4/+60
| | | | | | | | | | | | | | | | | | | | | | | | Using blockdev-snapshot to append a node as an overlay to itself, or to any of its parents, causes crashes. Catch the condition and return an error for these cases instead. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1824363 Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20211018134714.48438-1-kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* | pylint: fix errors and warnings generated by tests/qemu-iotests/297Emanuele Giuseppe Esposito2021-11-025-27/+29
|/ | | | | | | | | | | | Test 297 in tests/qemu-iotests currently fails: pylint has learned new things to check, or we simply missed them. All fixes in this patch are related to additional spaces used or wrong indentation. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Message-Id: <20211008062821.1010967-2-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
* iotests/300: avoid abnormal shutdown race conditionJohn Snow2021-11-011-8/+5
| | | | | | | | | | | | | | | | | | Wait for the destination VM to close itself instead of racing to shut it down first, which produces different error log messages from AQMP depending on precisely when we tried to shut it down. (For example: We may try to issue 'quit' immediately prior to the target VM closing its QMP socket, which will cause an ECONNRESET error to be logged. Waiting for the VM to exit itself avoids the race on shutdown behavior.) Reported-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211026175612.4127598-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests: Conditionally silence certain AQMP errorsJohn Snow2021-11-012-5/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | AQMP likes to be very chatty about errors it encounters. In general, this is good because it allows us to get good diagnostic information for otherwise complex async failures. For example, during a failed QMP connection attempt, we might see: +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Negotiation failed: EOFError +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Failed to establish session: EOFError This might be nice in iotests output, because failure scenarios involving the new QMP library will be spelled out plainly in the output diffs. For tests that are intentionally causing this scenario though, filtering that log output could be a hassle. For now, add a context manager that simply lets us toggle this output off during a critical region. (Additionally, a forthcoming patch allows the use of either legacy or async QMP to be toggled with an environment variable. In this circumstance, we can't amend the iotest output to just always expect the error message, either. Just suppress it for now. More rigorous log filtering can be investigated later if/when it is deemed safe to permanently replace the legacy QMP library.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20211026175612.4127598-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests: Accommodate async QMP Exception classesJohn Snow2021-11-011-2/+3
| | | | | | | | | | | | | | | | | | | | | | | (But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError, excluding 'AQMPError' and its derivatives. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20211026175612.4127598-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* python/machine: remove has_quit argumentJohn Snow2021-11-013-8/+3
| | | | | | | | | | | | | | | | | | | | | If we spy on the QMP commands instead, we don't need callers to remember to pass it. Seems like a fair trade-off. The one slightly weird bit is overloading this instance variable for wait(), where we use it to mean "don't issue the qmp 'quit' command". This means that wait() will "fail" if the QEMU process does not terminate of its own accord. In most cases, we probably did already actually issue quit -- some iotests do this -- but in some others, we may be waiting for QEMU to terminate for some other reason, such as a test wherein we tell the guest (directly) to shut down. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20211026175612.4127598-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/linters: Add workaround for mypy bug #9852John Snow2021-11-011-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This one is insidious: if you write an import as "from {namespace} import {subpackage}" as mirror-top-perms (now) does, mypy will fail on every-other invocation *if* the package being imported is a typed, installed, namespace-scoped package. Upsettingly, that's exactly what 'qemu.[aqmp|qmp|machine]' et al are in the context of Python CI tests. Now, I could just edit mirror-top-perms to avoid this invocation, but since I tripped on a landmine, I might as well head it off at the pass and make sure nobody else trips on that same landmine. It seems to have something to do with the order in which files are checked as well, meaning the random order in which set(os.listdir()) produces the list of files to test will cause problems intermittently and not just strictly "every other run". This will be fixed in mypy >= 0.920, which is not released yet. The workaround for now is to disable incremental checking, which avoids the issue. Note: This workaround is not applied when running iotest 297 directly, because the bug does not surface there! Given the nature of CI jobs not starting with any stale cache to begin with, this really only has a half-second impact on manual runs of the Python test suite when executed directly by a developer on their local machine. The workaround may be removed when the Python package requirements can stipulate mypy 0.920 or higher, which can happen as soon as it is released. (Barring any unforseen compatibility issues that 0.920 may bring with it.) See also: https://github.com/python/mypy/issues/11010 https://github.com/python/mypy/issues/9852 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-14-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/linters: Add entry point for linting via Python CIJohn Snow2021-11-011-0/+27
| | | | | | | | | | | | | | | | | | | | We need at least a tiny little shim here to join test file discovery with test invocation. This logic could conceivably be hosted somewhere in python/, but I felt it was strictly the least-rude thing to keep the test logic here in iotests/, even if this small function isn't itself an iotest. Note that we don't actually even need the executable bit here, we'll be relying on the ability to run this module as a script using Python CLI arguments. No chance it gets misunderstood as an actual iotest that way. (It's named, not in tests/, doesn't have the execute bit, and doesn't have an execution shebang.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-13-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests: split linters.py out from 297John Snow2021-11-012-61/+87
| | | | | | | | | | | | | Now, 297 is just the iotests-specific incantations and linters.py is as minimal as I can think to make it. The only remaining element in here that ought to be configuration and not code is the list of skip files, but they're still numerous enough that repeating them for mypy and pylint configurations both would be ... a hassle. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-12-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: split test into sub-casesJohn Snow2021-11-011-23/+38
| | | | | | | | | | | Take iotest 297's main() test function and split it into two sub-cases that can be skipped individually. We can also drop custom environment setup from the pylint test as it isn't needed. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-11-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: update tool availability checksJohn Snow2021-11-011-4/+5
| | | | | | | | | | | As mentioned in 'iotests/297: Don't rely on distro-specific linter binaries', these checks are overly strict. Update them to be in-line with how we actually invoke the linters themselves. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: Change run_linter() to raise an exception on failureJohn Snow2021-11-011-10/+14
| | | | | | | | | | | | | | | | Instead of using a process return code as the python function return value (or just not returning anything at all), allow run_linter() to raise an exception instead. The responsibility for printing output on error shifts from the function itself to the caller, who will know best how to present/format that information. (Also, "suppress_output" is now a lot more accurate of a parameter name.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-9-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: refactor run_[mypy|pylint] as generic execution shimJohn Snow2021-11-011-22/+24
| | | | | | | | | | There's virtually nothing special here anymore; we can combine these into a single, rather generic function. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: Split run_linters apart into run_pylint and run_mypyJohn Snow2021-11-011-14/+24
| | | | | | | | | | Move environment setup into main(), and split the actual linter execution into run_pylint and run_mypy, respectively. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: Don't rely on distro-specific linter binariesJohn Snow2021-11-011-2/+2
| | | | | | | | | | | | | | | | | | | 'pylint-3' is another Fedora-ism. Use "python3 -m pylint" or "python3 -m mypy" to access these scripts instead. This style of invocation will prefer the "correct" tool when run in a virtual environment. Note that we still check for "pylint-3" before the test begins -- this check is now "overly strict", but shouldn't cause anything that was already running correctly to start failing. This is addressed by a commit later in this series; 'iotests/297: update tool availability checks'. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: Create main() functionJohn Snow2021-11-011-4/+8
| | | | | | | | | | | | | | | Instead of running "run_linters" directly, create a main() function that will be responsible for environment setup, leaving run_linters() responsible only for execution of the linters. (That environment setup will be moved over in forthcoming commits.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: Add get_files() functionJohn Snow2021-11-011-2/+7
| | | | | | | | | | Split out file discovery into its own method to begin separating out configuration/setup and test execution. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: Split mypy configuration out into mypy.iniJohn Snow2021-11-012-13/+13
| | | | | | | | | More separation of code and configuration. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* iotests/297: Move pylint config into pylintrcJohn Snow2021-11-012-3/+17
| | | | | | | | | | | Move --score=n and --notes=XXX,FIXME into pylintrc. This pulls configuration out of code, which I think is probably a good thing in general. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* qapi: Improve input_type_enum()'s error messageMarkus Armbruster2021-10-276-10/+10
| | | | | | | | | | | | | | | | | | The error message claims the parameter is invalid: $ qemu-system-x86_64 -object qom-type=nonexistent qemu-system-x86_64: -object qom-type=nonexistent: Invalid parameter 'nonexistent' What's wrong is actually the *value* 'nonexistent'. Improve the message to qemu-system-x86_64: -object qom-type=nonexistent: Parameter 'qom-type' does not accept value 'nonexistent' Fixes: https://gitlab.com/qemu-project/qemu/-/issues/608 Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20211020180231.434071-1-armbru@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
* iotests/051: Fix typoKevin Wolf2021-10-152-3/+3
| | | | | | | | | | | | | The iothread isn't called 'iothread0', but 'thread0'. Depending on the order that properties are parsed, the error message may change from the expected one to another one saying that the iothread doesn't exist. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211008133442.141332-7-kwolf@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Tested-by: Peter Krempa <pkrempa@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* iotests/245: Fix type for iothread propertyKevin Wolf2021-10-151-2/+2
| | | | | | | | | | | | iothread is a string property, so None (= JSON null) is not a valid value for it. Pass the empty string instead to get the default iothread. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211008133442.141332-6-kwolf@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Tested-by: Peter Krempa <pkrempa@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
* qemu-iotests: flush after every testPaolo Bonzini2021-10-131-0/+1
| | | | | | | | | This makes it possible to see what is happening, even if the output of "make check-block" is not sent to a tty (for example if it is sent to grep or tee). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* python, iotests: remove socket_scm_helperJohn Snow2021-10-124-151/+1
| | | | | | | | | | It's not used anymore, now. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20210923004938.3999963-11-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
* Merge remote-tracking branch ↵Richard Henderson2021-10-074-37/+173
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'remotes/vsementsov/tags/pull-jobs-2021-10-07-v2' into staging mirror: Handle errors after READY cancel v2: add small fix by Stefano, Hanna's series fixed # gpg: Signature made Thu 07 Oct 2021 08:25:07 AM PDT # gpg: using RSA key 8B9C26CDB2FD147C880E86A1561F24C1F19F79FB # gpg: Good signature from "Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>" [unknown] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: 8B9C 26CD B2FD 147C 880E 86A1 561F 24C1 F19F 79FB * remotes/vsementsov/tags/pull-jobs-2021-10-07-v2: iotests: Add mirror-ready-cancel-error test mirror: Do not clear .cancelled mirror: Stop active mirroring after force-cancel mirror: Check job_is_cancelled() earlier mirror: Use job_is_cancelled() job: Add job_cancel_requested() job: Do not soft-cancel after a job is done jobs: Give Job.force_cancel more meaning job: @force parameter for job_cancel_sync() job: Force-cancel jobs in a failed transaction mirror: Drop s->synced mirror: Keep s->synced on error job: Context changes in job_completed_txn_abort() block/aio_task: assert `max_busy_tasks` is greater than 0 block/backup: avoid integer overflow of `max-workers` Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
| * iotests: Add mirror-ready-cancel-error testHanna Reitz2021-10-072-0/+148
| | | | | | | | | | | | | | | | | | | | | | Test what happens when there is an I/O error after a mirror job in the READY phase has been cancelled. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211006151940.214590-14-hreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
| * job: @force parameter for job_cancel_sync()Hanna Reitz2021-10-072-37/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Callers should be able to specify whether they want job_cancel_sync() to force-cancel the job or not. In fact, almost all invocations do not care about consistency of the result and just want the job to terminate as soon as possible, so they should pass force=true. The replication block driver is the exception, specifically the active commit job it runs. As for job_cancel_sync_all(), all callers want it to force-cancel all jobs, because that is the point of it: To cancel all remaining jobs as quickly as possible (generally on process termination). So make it invoke job_cancel_sync() with force=true. This changes some iotest outputs, because quitting qemu while a mirror job is active will now lead to it being cancelled instead of completed, which is what we want. (Cancelling a READY mirror job with force=false may take an indefinite amount of time, which we do not want when quitting. If users want consistent results, they must have all jobs be done before they quit qemu.) Buglink: https://gitlab.com/qemu-project/qemu/-/issues/462 Signed-off-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211006151940.214590-6-hreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
* | iotests: Update for pylint 2.11.1John Snow2021-10-062-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Ignore the new f-strings warning, we're not interested in doing a full conversion at this time. 2. Just mute the unbalanced-tuple-unpacking warning, it's not a real error in this case and muting the dozens of callsites is just not worth it. 3. Add encodings to read_text(). Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20210923180715.4168522-7-jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>