summary refs log tree commit diff stats
path: root/tests/functional/qemu_test (follow)
Commit message (Collapse)AuthorAgeFilesLines
* tests/functional: remove all class level fieldsDaniel P. Berrangé2025-03-191-6/+0
| | | | | | | | | | | | | | | | A number of fields are set at the class level on QemuBaseTest, even though the exact same named field is then set at the object level later in most cases. The 'self.logger' initialization in ACPI bits test needs to be removed since 'self.log' won't exist at that point in the flow. It already initialized 'self.logger' later in the setUp() method, so the __init__ method was redundant. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250317124300.84266-1-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional/asset: Add AssetError exception classNicholas Piggin2025-03-121-15/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | Assets are uniquely identified by human-readable-ish url, so make an AssetError exception class that prints url with error message. A property 'transient' is used to capture whether the client may retry or try again later, or if it is a serious and likely permanent error. This is used to retain the existing behaviour of treating HTTP errors other than 404 as 'transient' and not causing precache step to fail. Additionally, partial-downloads and stale asset caches that fail to resolve after the retry limit are now treated as transient and do not cause precache step to fail. For background: The NetBSD archive is, at the time of writing, failing with short transfer. Retrying the fetch at that position (as wget does) results in a "503 backend unavailable" error. We would like to get that error code directly, but I have not found a way to do that with urllib, so treating the short-copy as a transient failure covers that case (and seems like a reasonable way to handle it in general). Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Message-ID: <20250312130002.945508-4-npiggin@gmail.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional/asset: Verify downloaded sizeNicholas Piggin2025-03-121-0/+14
| | | | | | | | | | | | If the server provides a Content-Length header, use that to verify the size of the downloaded file. This catches cases where the connection terminates early, and gives the opportunity to retry. Without this, the checksum will likely mismatch and fail without retry. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Message-ID: <20250312130002.945508-3-npiggin@gmail.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional/asset: Fail assert fetch when retries are exceededNicholas Piggin2025-03-121-0/+3
| | | | | | | | | | | | | | Currently the fetch code does not fail gracefully when retry limit is exceeded, it just falls through the loop with no file, which ends up hitting other errors. Add a check for non-existing file, which indicates the retry limit was exceeded. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Message-ID: <20250312130002.945508-2-npiggin@gmail.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: Allow running TCG plugins tests on non-Linux/BSD hostsPhilippe Mathieu-Daudé2025-03-101-1/+11
| | | | | | | | | | | | | | | | | Not all platforms use the '.so' suffix for shared libraries, which is how plugins are built. Use the recently introduced dso_suffix() helper to get the proper host suffix. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2804 Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Suggested-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20250220080215.49165-4-philmd@linaro.org> [AJB: moved plugin_file into testcase.py] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20250304222439.2035603-11-alex.bennee@linaro.org>
* tests/functional: Introduce the dso_suffix() helperPhilippe Mathieu-Daudé2025-03-102-1/+13
| | | | | | | | | | | | | | Introduce a helper to get the default shared library suffix used on the host. Suggested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20250220080215.49165-3-philmd@linaro.org> [AJB: dropped whitespace cmd.py damage] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20250304222439.2035603-10-alex.bennee@linaro.org>
* tests/functional: Move the code for testing HTTP downloads to a common functionThomas Huth2025-03-071-1/+25
| | | | | | | | We are going to use this code in other tests, too, so let's move it to the qemu_test module to be able to re-use it more easily. Message-ID: <20250227103915.19795-2-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: stop output from zstd command when uncompressingDaniel P. Berrangé2025-03-071-3/+3
| | | | | | | | | | | | | | | The zstd command will print incremental decompression progress to stderr when running. Fortunately it is not on stdout as that would confuse the TAP parsing, but we should still not have this printed. By switching from 'check_call' to 'run' with the check=True and capture_output=True we'll get the desired silence on success, and on failure the raised exception will automatically include stdout/stderr data for diagnosis purposes. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-ID: <20250228102738.3064045-8-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: drop unused 'get_tag' methodDaniel P. Berrangé2025-03-071-11/+0
| | | | | | | Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-ID: <20250228102738.3064045-7-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: set 'qemu_bin' as an object level fieldDaniel P. Berrangé2025-03-061-1/+1
| | | | | | | | | | | | | | The 'qemu_bin' field is currently set on the class, despite being accessed as if it were an object instance field with 'self.qemu_bin'. This is no obvious need to have it as a class field, so move it into the object instance. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250228102738.3064045-3-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: remove unused 'bin_prefix' variableDaniel P. Berrangé2025-03-061-3/+3
| | | | | | | | | | | This was copied over from avocado but has not been used in the new functional tests. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250228102738.3064045-2-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: Add a decorator for skipping long running testsThomas Huth2025-01-302-1/+15
| | | | | | | | | | | | Some tests have a very long runtime and might run into timeout issues e.g. when QEMU has been compiled with --enable-debug. Add a decorator for marking them more easily. Rename the corresponding environment variable to be more in sync with the other QEMU_TEST_ALLOW_* switches that we already have, and add a paragraph about it in the documentation. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250128152839.184599-2-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: Fix broken decorators with lamda functionsThomas Huth2025-01-301-23/+21
| | | | | | | | | | | | The decorators that use a lambda function are currently broken and do not properly skip the test if the condition is not met. Using "return skipUnless(lambda: ...)" does not work as expected. To fix it, rewrite the decorators without lambda, it's simpler that way anyway. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250122134315.1448794-3-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional/qemu_test/decorators: Fix bad check for importsThomas Huth2025-01-301-1/+2
| | | | | | | | | | | | | | skipIfMissingImports should use importlib.import_module() for checking whether a module with the name stored in the "impname" variable is available or not, otherwise the code tries to import a module with the name "impname" instead. (This bug hasn't been noticed before since there is another issue with this decorator that will be fixed by the next patch) Suggested-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250122134315.1448794-2-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: update tuxruntest to use uncompress utilityAlex Bennée2025-01-101-11/+1
| | | | | | | | Use the utility functions to reduce code duplication. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20250108121054.1126164-21-alex.bennee@linaro.org>
* tests/functional: add zstd support to uncompress utilityAlex Bennée2025-01-101-0/+24
| | | | | | | | | | | | | Rather than using the python library (which has a different API anyway) lets just call the binary. zstdtools is already in out qemu.yml so all test containers should have it around. Tests should still use @skipIfMissingCommands('zstd') to gracefully handle when only minimal dependencies have been installed. Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20250108121054.1126164-20-alex.bennee@linaro.org>
* tests/functional: Extract the find_free_ports() function into a helper fileThomas Huth2025-01-021-0/+56
| | | | | | | | | | | | | | | | We'll need this functionality in other functional tests, too, so let's extract it into the qemu_test module. Also add an __enter__ and __exit__ function that can be used for using this functionality in a locked context, so that tests that are running in parallel don't try to compete for the same ports later. Also make sure to only use ports in the "Dynamic Ports" range (see https://www.rfc-editor.org/rfc/rfc6335) and "randomize" the start of the probed range with the PID of the test process to further avoid possible clashes with other competing processes. Message-ID: <20241218131439.255841-5-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: Add a helper function for retrieving the hostfwd portThomas Huth2024-12-171-0/+7
| | | | | | | | | | It's just a wrapper around get_info_usernet_hostfwd_port from the qemu module that is also calling the right monitor command for retrieving the information from QEMU. Message-ID: <20241217121550.141072-2-thuth@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: ignore errors when caching assets, except for 404Daniel P. Berrangé2024-12-171-1/+13
| | | | | | | | | | | | | | | We see periodic errors caching assets due to a combination of transient networking and server problems. With the previous patch to skip running a test when it has missing assets, we can now treat most cache download errors as non-fatal. Only HTTP 404 is retained as fatal, since it is a strong indicator of a fully broken test rather than a transient error. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-32-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: skip tests if assets are not availableDaniel P. Berrangé2024-12-172-1/+18
| | | | | | | | | | If downloading of assets has been disabled, then skip running a test if the assets it has registered are not already downloaded. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-31-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: remove now unused 'run_cmd' helperDaniel P. Berrangé2024-12-172-12/+1
| | | | | | | | | All usage has been replaced by direct 'subprocess' helpers. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-30-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: replace 'run_cmd' with subprocess helpersDaniel P. Berrangé2024-12-174-26/+32
| | | | | | | | | | | The 'run_cmd' helper is re-implementing a convenient helper that already exists in the form of the 'run' and 'check_call' methods provided by 'subprocess'. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-29-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: drop back compat imports from utils.pyDaniel P. Berrangé2024-12-171-5/+0
| | | | | | | | | | Now that all tests are converted over to the higher level wrapper functions, the back compat imports from utils.py are redundant. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-28-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add 'uncompress' to QemuBaseTestDaniel P. Berrangé2024-12-171-0/+25
| | | | | | | | | | | This helper wrappers utils.uncompress, forcing the use of the scratch directory, to ensure any uncompressed files are cleaned at test termination. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-26-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add a generalized uncompress helperDaniel P. Berrangé2024-12-172-0/+48
| | | | | | | | | | | There are many types of compression that the tests deal with, and it makes sense to have a single helper 'uncompress' that can deal with all. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-25-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: convert tests to new archive_extract helperDaniel P. Berrangé2024-12-171-19/+0
| | | | | | | | | | Replace use of utils.archive_extract and extract_from_deb with the new archive_extract helper. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-24-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add 'archive_extract' to QemuBaseTestDaniel P. Berrangé2024-12-171-0/+32
| | | | | | | | | | | | This helper wrappers archive.archive_extract, forcing the use of the scratch directory, to ensure any extracted files are cleaned at test termination. If a specific member is requested, then the path to the extracted file is also returned. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-23-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add a generalized archive_extractDaniel P. Berrangé2024-12-172-0/+59
| | | | | | | | | | | | There are many types of archives that the tests deal with. Provide a generalized 'archive_extract' that can detect the format and delegate to the appropriate helper for extraction. This ensures that all archive extraction code follows the same design pattern. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-22-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: let cpio_extract accept filenamesDaniel P. Berrangé2024-12-171-5/+11
| | | | | | | | | | | Currently cpio_extract differs from tar_extract/zip_extract in that it only allows a file-like object as input. Adapt it to also support filenames. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-21-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add common deb_extract helperDaniel P. Berrangé2024-12-172-9/+17
| | | | | | | | | | This mirrors the existing archive_extract, cpio_extract and zip_extract helpers Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-20-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add common zip_extract helperDaniel P. Berrangé2024-12-171-0/+8
| | | | | | | | | This mirrors the existing archive_extract and cpio_extract helpers Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-19-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: move uncompress handling into new uncompress.py fileDaniel P. Berrangé2024-12-172-25/+38
| | | | | | | | | | | | | | | More uncompress related code will be added shortly, so having a separate file makes more sense. The utils.py imports the functions from archive.py, so that existing callers don't need to be modified. This avoids redundant code churn until later in the series when all calls will be adapted for other reasons. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-18-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: move archive handling into new archive.py fileDaniel P. Berrangé2024-12-172-20/+34
| | | | | | | | | | | | | | | More archive related code will be added shortly, so having a separate file makes more sense. The utils.py imports the functions from archive.py, so that existing callers don't need to be modified. This avoids redundant code churn until later in the series when all calls will be adapted for other reasons. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-17-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: switch over to using self.scratch_file()Daniel P. Berrangé2024-12-172-5/+4
| | | | | | | | | | | | | | | | | | Replace any instances of os.path.join(self.workdir, ".../...") self.workdir + "/.../..." with self.scratch_file("...", "...") which is more compact and portable Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-15-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: switch over to using self.build_file(...)Daniel P. Berrangé2024-12-172-5/+4
| | | | | | | | | This removes direct access of the 'BUILD_DIR' variable. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-13-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: switch over to using self.log_file(...)Daniel P. Berrangé2024-12-171-5/+4
| | | | | | | | | This removes direct access of the 'self.logdir' variable. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-12-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: add helpers for building file pathsDaniel P. Berrangé2024-12-171-0/+95
| | | | | | | | | | | | | | | | | | | Add helper methods that construct paths for * log files - to be preserved at the end of a test * scratch files - to be purged at the end of a test * build files - anything relative to the build root * data files - anything relative to the functional test source root * socket files - a short temporary dir to avoid UNIX socket limits These are to be used instead of direct access to the self.workdir, or self.logdir variables, or any other place where paths are built manually. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-11-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: drop 'has_cmd' and 'has_cmds' helpersDaniel P. Berrangé2024-12-173-57/+9
| | | | | | | | | | The 'which' helper is simpler, not depending on the external 'which' binary, and is sufficient for test needs. Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-10-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: introduce some helpful decoratorsDaniel P. Berrangé2024-12-172-0/+110
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reduce repeated boilerplate with some helper decorators: @skipIfNotPlatform("x86_64", "aarch64") => Skip unless the build host platform matches @skipIfMissingCommands("mkisofs", "losetup") => Skips unless all listed commands are found in $PATH @skipIfMissingImports("numpy", "cv2") => Skips unless all listed modules can be imported @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/NNN") => Skips unless env var requests flaky tests with the reason documented in the referenced gitlab bug @skipBigData => Skips unless env var permits tests creating big data files @skipUntrustedTest => Skips unless env var permits tests which are potentially dangerous to the host Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-8-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: drop 'tesseract_available' helperDaniel P. Berrangé2024-12-171-11/+1
| | | | | | | | | | Platforms we target have new enough tesseract that it suffices to merely check if the binary exists. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20241217155953.3950506-7-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: simplify 'which' implementationDaniel P. Berrangé2024-12-171-1/+1
| | | | | | | | | | The 'access' check implies the file exists. Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-6-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: remove duplicated 'which' function implDaniel P. Berrangé2024-12-172-1/+11
| | | | | | | | | Put the 'which' function into shared code. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-5-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: resolve str(Asset) to cache file pathDaniel P. Berrangé2024-12-171-0/+3
| | | | | | | | | | | Allow an Asset object to be used in place of a filename but making its string representation resolve to the cache file path. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20241217155953.3950506-4-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: remove many unused importsDaniel P. Berrangé2024-12-173-4/+1
| | | | | | | | | Identified using 'pylint --disable=all --enable=W0611' Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20241217155953.3950506-3-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
* tests/functional: avoid accessing log_filename on earlier failuresDaniel P. Berrangé2024-11-251-4/+6
| | | | | | | | | | | | | | | If a failure occurs early in the QemuBaseTest constructor, the 'log_filename' object atttribute may not exist yet. This happens most notably if the QEMU_TEST_QEMU_BINARY is not set. We can't initialize 'log_filename' earlier as we use the binary to identify the architecture which is then used to build the path in which the logs are stored. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-19-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-19-alex.bennee@linaro.org>
* tests/functional: add a QMP backdoor for debugging stalled testsDaniel P. Berrangé2024-11-251-0/+7
| | | | | | | | | | | | | | | | | | | | | Support the QEMU_TEST_QMP_BACKDOOR=backdoor.sock env variable as a way to get a QMP backdoor for debugging a stalled QEMU test. Most typically this would be used if running the tests directly: $ QEMU_TEST_QMP_BACKDOOR=backdoor.sock \ QEMU_TEST_QEMU_BINARY=./build/qemu-system-arm \ PYTHONPATH=./python \ ./tests/functional/test_arm_tuxrun.py And then, when the test stalls, in a second shell run: $ ./scripts/qmp/qmp-shell backdoor.sock Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-18-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-18-alex.bennee@linaro.org>
* tests/functional: remove time.sleep usage from tuxrun testsDaniel P. Berrangé2024-11-251-10/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | The tuxrun tests send a series of strings to the guest to login and then run commands. Since we have been unable to match on console output that isn't followed by a newline, the test used many time.sleep() statements to pretend to synchronize with the guest. This has proved to be unreliable for the aarch64be instance of the tuxrun tests, with the test often hanging. The hang is a very subtle timing problem, and it is suspected that some (otherwise apparently harmless) I/O error messages could be resulting in full FIFO buffers, stalling interaction with the guest. With the newly rewritten console interaction able to match strings that don't have a following newline, the tux run tests can now match directly on the login prompt, and/or shell PS1 prompt. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2689 Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-17-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-17-alex.bennee@linaro.org>
* tests/functional: rewrite console handling to be bytewiseDaniel P. Berrangé2024-11-251-15/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | The console interaction that waits for predicted strings uses readline(), and thus is only capable of waiting for strings that are followed by a newline. This is inconvenient when needing to match on some things, particularly login prompts, or shell prompts, causing tests to use time.sleep(...) instead, which is unreliable. Switch to reading the console 1 byte at a time, comparing against the success/failure messages until we see a match, regardless of whether a newline is encountered. The success/failure comparisons are done with the python bytes type, rather than strings, to avoid the problem of needing to decode partially received multibyte utf8 characters. Heavily inspired by a patch proposed by Cédric, but written again to work in bytes, rather than strings. Co-developed-by: Cédric Le Goater <clg@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-16-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-16-alex.bennee@linaro.org>
* tests/functional: require non-NULL success_message for console waitDaniel P. Berrangé2024-11-251-2/+7
| | | | | | | | | | | | | | | | | | | | | When waiting for expected output, the 'success_message' is a mandatory parameter, with 'failure_message' defaulting to None. The code has logic which indicates it was trying to cope with 'success_message' being None and 'failure_message' being non-None but it does not appear able to actually do anything useful. The check for 'success_message is None' will break out of the loop before any check for 'failure_message' has been performed. IOW, for practcal purposes 'success_message' must be non-None unless 'send_string' is set. Assert this expectation and simplify the loop logic. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-15-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-15-alex.bennee@linaro.org>
* tests/functional: logs details of console interaction operationsDaniel P. Berrangé2024-11-251-0/+3
| | | | | | | | | | | | | When functional tests go wrong, it will often be related to the console interaction wait state. By logging the messages that we're looking for, and data we're about to be sending, it'll be easier to diagnose where tests are getting stuck. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20241121154218.1423005-13-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20241121165806.476008-13-alex.bennee@linaro.org>