summary refs log tree commit diff stats
path: root/tests/functional/qemu_test/testcase.py (follow)
Commit message (Collapse)AuthorAgeFilesLines
* tests/functional: remove all class level fieldsDaniel P. Berrangé2025-03-191-6/+0
| | | | | | | | | | | | | | | | A number of fields are set at the class level on QemuBaseTest, even though the exact same named field is then set at the object level later in most cases. The 'self.logger' initialization in ACPI bits test needs to be removed since 'self.log' won't exist at that point in the flow. It already initialized 'self.logger' later in the setUp() method, so the __init__ method was redundant. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250317124300.84266-1-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: Allow running TCG plugins tests on non-Linux/BSD hostsPhilippe Mathieu-Daudé2025-03-101-1/+11
| | | | | | | | | | | | | | | | | Not all platforms use the '.so' suffix for shared libraries, which is how plugins are built. Use the recently introduced dso_suffix() helper to get the proper host suffix. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2804 Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Suggested-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20250220080215.49165-4-philmd@linaro.org> [AJB: moved plugin_file into testcase.py] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20250304222439.2035603-11-alex.bennee@linaro.org>
* tests/functional: set 'qemu_bin' as an object level fieldDaniel P. Berrangé2025-03-061-1/+1
| | | | | | | | | | | | | | The 'qemu_bin' field is currently set on the class, despite being accessed as if it were an object instance field with 'self.qemu_bin'. This is no obvious need to have it as a class field, so move it into the object instance. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250228102738.3064045-3-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: remove unused 'bin_prefix' variableDaniel P. Berrangé2025-03-061-3/+3
| | | | | | | | | | | This was copied over from avocado but has not been used in the new functional tests. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250228102738.3064045-2-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: skip tests if assets are not availableDaniel P. Berrangé2024-12-171-0/+11
| | | | | | | | | | If downloading of assets has been disabled, then skip running a test if the assets it has registered are not already downloaded. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-31-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: replace 'run_cmd' with subprocess helpersDaniel P. Berrangé2024-12-171-14/+17
| | | | | | | | | | | The 'run_cmd' helper is re-implementing a convenient helper that already exists in the form of the 'run' and 'check_call' methods provided by 'subprocess'. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-29-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add 'uncompress' to QemuBaseTestDaniel P. Berrangé2024-12-171-0/+25
| | | | | | | | | | | This helper wrappers utils.uncompress, forcing the use of the scratch directory, to ensure any uncompressed files are cleaned at test termination. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-26-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add 'archive_extract' to QemuBaseTestDaniel P. Berrangé2024-12-171-0/+32
| | | | | | | | | | | | This helper wrappers archive.archive_extract, forcing the use of the scratch directory, to ensure any extracted files are cleaned at test termination. If a specific member is requested, then the path to the extracted file is also returned. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-23-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: switch over to using self.build_file(...)Daniel P. Berrangé2024-12-171-2/+2
| | | | | | | | | This removes direct access of the 'BUILD_DIR' variable. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-13-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: switch over to using self.log_file(...)Daniel P. Berrangé2024-12-171-5/+4
| | | | | | | | | This removes direct access of the 'self.logdir' variable. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-12-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add helpers for building file pathsDaniel P. Berrangé2024-12-171-0/+95
| | | | | | | | | | | | | | | | | | | Add helper methods that construct paths for * log files - to be preserved at the end of a test * scratch files - to be purged at the end of a test * build files - anything relative to the build root * data files - anything relative to the functional test source root * socket files - a short temporary dir to avoid UNIX socket limits These are to be used instead of direct access to the self.workdir, or self.logdir variables, or any other place where paths are built manually. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-11-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: avoid accessing log_filename on earlier failuresDaniel P. Berrangé2024-11-251-4/+6
| | | | | | | | | | | | | | | If a failure occurs early in the QemuBaseTest constructor, the 'log_filename' object atttribute may not exist yet. This happens most notably if the QEMU_TEST_QEMU_BINARY is not set. We can't initialize 'log_filename' earlier as we use the binary to identify the architecture which is then used to build the path in which the logs are stored. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-19-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-19-alex.bennee@linaro.org>
* tests/functional: add a QMP backdoor for debugging stalled testsDaniel P. Berrangé2024-11-251-0/+7
| | | | | | | | | | | | | | | | | | | | | Support the QEMU_TEST_QMP_BACKDOOR=backdoor.sock env variable as a way to get a QMP backdoor for debugging a stalled QEMU test. Most typically this would be used if running the tests directly: $ QEMU_TEST_QMP_BACKDOOR=backdoor.sock \ QEMU_TEST_QEMU_BINARY=./build/qemu-system-arm \ PYTHONPATH=./python \ ./tests/functional/test_arm_tuxrun.py And then, when the test stalls, in a second shell run: $ ./scripts/qmp/qmp-shell backdoor.sock Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-18-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-18-alex.bennee@linaro.org>
* tests/functional: enable debug logging for QEMUMachineDaniel P. Berrangé2024-11-251-0/+6
| | | | | | | | | | | | | | | Set the 'qemu.machine' logger to 'DEBUG' level, to ensure we see log messages related to the QEMUMachine class. Most importantly this ensures we capture the full QEMU command line args for instances we spawn. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-12-berrange@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20241121165806.476008-12-alex.bennee@linaro.org>
* tests/functional: honour requested test VM name in QEMUMachineDaniel P. Berrangé2024-11-251-0/+1
| | | | | | | | | | | | | | The functional test case class is going to the trouble of passing around a machine name, but then fails to give this QEMUMachine. As a result, QEMUMachine will create a completely random name. Since log file names match the machine name, this results in log files accumulating over time. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-11-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-11-alex.bennee@linaro.org>
* tests/functional: put QEMUMachine logs in testcase log directoryDaniel P. Berrangé2024-11-251-2/+3
| | | | | | | | | | | | | | | | We are not passing the 'log_dir' parameter to QEMUMachine, so the QEMU stdout/err logs are being placed in a temp directory and thus deleted after execution. This makes them inaccessible as gitlab CI artifacts. Pass the testcase log directory path into QEMUMachine to make the logs persistent. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-10-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-10-alex.bennee@linaro.org>
* tests/functional: automatically clean up scratch files after testsDaniel P. Berrangé2024-11-251-5/+9
| | | | | | | | | | | | | | | | The build/tests/functional subdirectories are consuming huge amounts of disk space. Split the location for scratch files into a 'scratch' sub-directory, separate from log files, and delete it upon completion of each test. The new env variable QEMU_TEST_KEEP_SCRATCH can be set to preserve this scratch dir for debugging access if required. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-3-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-3-alex.bennee@linaro.org>
* tests/functional: Provide the user with hints where to find more log filesThomas Huth2024-11-081-5/+13
| | | | | | | | | | | | | | Since the base.log and console.log files are not referenced from the meson test logs yet, they might be hard to find for the casual users. Thus let's print some hints in case a test case failed. For this we have to run unittest.main() with exit=False to get the results of the testing. Then we can iterate through the failed test cases to print out the information accordingly. Message-ID: <20241105123849.359391-1-thuth@redhat.com> Reviewed-by: Cédric Le Goater <clg@redhat.com> Tested-by: Cédric Le Goater <clg@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: Add QemuUserTest classPhilippe Mathieu-Daudé2024-09-041-0/+17
| | | | | | | | | | | | | | | Per commit 5334df4822 ("tests/avocado: Introduce QemuUserTest base class"): Similarly to the 'System' Test base class with methods for testing system emulation, the QemuUserTest class contains methods useful to test user-mode emulation. Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20240822104238.75045-2-philmd@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-ID: <20240830133841.142644-38-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: enable pre-emptive caching of assetsDaniel P. Berrangé2024-09-041-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Many tests need to access assets stored on remote sites. We don't want to download these during test execution when run by meson, since this risks hitting test timeouts when data transfers are slow. Add support for pre-emptive caching of assets by setting the env var QEMU_TEST_PRECACHE to point to a timestamp file. When this is set, instead of running the test, the assets will be downloaded and saved to the cache, then the timestamp file created. A meson custom target is created as a dependency of each test suite to trigger the pre-emptive caching logic before the test runs. When run in caching mode, it will locate assets by looking for class level variables with a name prefix "ASSET_", and type "Asset". At the ninja level ninja test --suite functional will speculatively download any assets that are not already cached, so it is advisable to set a timeout multiplier. QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional will fail the test if a required asset is not already cached ninja precache-functional will download and cache all assets required by the functional tests At the make level, precaching is always done by make check-functional Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Tested-by: Richard Henderson <richard.henderson@linaro.org> [thuth: Remove the duplicated "path = os.path.basename(...)" line] Message-ID: <20240830133841.142644-16-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: Set up loggingThomas Huth2024-09-041-1/+26
| | | | | | | | | Create log files for each test separately, one file that contains the basic logging and one that contains the console output. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20240830133841.142644-10-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: Add base classes for the upcoming pytest-based testsThomas Huth2024-09-041-0/+153
The files are mostly a copy of the tests/avocado/avocado_qemu/__init__.py file with some adjustments to get rid of the Avocado dependencies (i.e. we also have to drop the LinuxSSHMixIn and LinuxTest for now). The emulator binary and build directory are now passed via environment variables that will be set via meson.build later. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20240830133841.142644-9-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>