summaryrefslogtreecommitdiff
path: root/mesonbuild/mtest.py
Commit message (Collapse)AuthorAgeFilesLines
* rename exe_runner to exe_wrapper everywhereEli Schwartz2021-12-051-6/+6
| | | | I don't understand the purpose of this confusing API naming split.
* mtest: add an ASCII-only version of the spinnerPaolo Bonzini2021-11-161-4/+7
| | | | | | | | | While the horizontal line and the other pictograms in mtest have an ASCII-only version, the spinner does not. This causes mtest to fail with a UnicodeEncodeError exception on non-Unicode locales. Fixes: #9538 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: accept very long linesPaolo Bonzini2021-10-281-1/+8
| | | | | | | | | | | | | Unless parsing TAP output, there is no strict requirement for "meson test" to process test output one line at a time; it simply looks nicer to not print a partial line if it can be avoided. However, in the case of extremely long lines StreamReader.readline can fail with a ValueError. Use readuntil('\n') instead and just process whatever pieces of the line it returns. Fixes: #8591 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* fix various flake8 whitespace errorsEli Schwartz2021-10-271-2/+2
|
* mtest: limit "magic" CTRL+C behavior to process group leadersPaolo Bonzini2021-10-181-1/+4
| | | | | | | | | | | | If meson is not a process group leader, a SIGINT will be delivered also to its parent process (and possibly other processes). The parent process then will probably exit and mtest will continue running in the background, without any way to interrupt the run completely. To fix this, treat SIGINT and SIGTERM the same way unless mtest is a process group leader. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* fix incorrect porting from % formatting to .format methodEli Schwartz2021-10-041-1/+1
| | | | | | | | | (var,) is the correct way to pass values to a percent formatted string, but not to .format so we would end up printing something like: unexpected input at line (4,) Upgrade to an f-string and insert the correct value correctly.
* various python neatness cleanupsEli Schwartz2021-10-041-3/+3
| | | | | | | | | | | | | | | | All changes were created by running "pyupgrade --py3-only" and committing the results. Although this has been performed in the past, newer versions of pyupgrade can automatically catch more opportunities, notably list comprehensions can use generators instead, in the following cases: - unpacking into function arguments as function(*generator) - unpacking into assignments of the form x, y = generator - as the argument to some builtin functions such as min/max/sorted Also catch a few creeping cases of new code added using older styles.
* Revert "mtest: fix test output issues (in console)"Paolo Bonzini2021-09-291-209/+201
| | | | | | | | | | | | | | | | | | This reverts commit 5fcb0e6525e2044e0f82bda488a51350e0f7f29f. The commit is a massive change that should have been split in separate pieces, and it also removes a few features: * in verbose mode, subtests are not printed as they happen * in non-verbose mode the progress report does not include the number of subtests that have been run * in non-parallel mode, output is batched rather than printed as it happens Furthermore, none of these changes are not documented in the release notes. Revert so that each proposal can be tested, evaluated and documented individually.
* mtest: fix test output issues (in console)Hemmo Nieminen2021-09-251-201/+209
| | | | | | | | | | | | | | | This change set aims to fix various "issues" seen with the current implementation. The changes can be summarized with the following list: * Replace emojis and spinners with multiline status displaying the name and running time of each currently running test. * The test output (especially in verbose mode or when multiple failing tests' output gets printed out) can get confusing. Try to make the output easier to read and grasp. Most notable change here is the addition of the test number to the beginning of each printed line. * Print exit details (i.e. exit code) of the test in verbose mode. * Try to make the verbose "live" output from tests to match the look and feel of otherwise produced (verbose) test output.
* mtest: Allow gtest protocol test to fail more gracefullyDylan Baker2021-09-201-1/+8
| | | | | | | | | | Currently, if the test fails to produce XML (or valid XML) then the test fails with a backtrace. It's actually pretty easy to get into this situation, a total failure of the test will result in no XML being written (this can happen, for example, if rpaths to gtest are not correctly set up). If we can't read the test, go ahead and complete using `TestRunExitCode.complete()`, which will fail for the bad exit code.
* cleanup self.options.wdPaolo Bonzini2021-07-071-4/+2
| | | | It is never None and always an absolute path
* resolve symlinks passed to -CPaolo Bonzini2021-07-071-2/+3
| | | | | | | | | | | | "meson setup" is resolving symlinks for the build directory in validate_core_dirs. For consistency with it, do the same when the build directory is passed via -C to devenv, dist, init, install and test. This ensures for example that the path to test dependencies is computed correctly in "meson test". Fixes: #8765
* fix: Always explicitly set encoding for text files (fixes #8263)Daniel Mensinger2021-06-291-1/+1
|
* mtest: timeout if the write side of pipes does not closePaolo Bonzini2021-03-231-11/+39
| | | | | | | | | | | | If a test program forks a child, the pipes might remain open and "await stdo_task"/"await stde_task" will never complete in SingleTestRunner._run_cmd(). Instead, catch them in TestSubprocess.wait() so that the whole process group is killed. Fixes: #8533 Reported-by: Bastien Nocera <hadess@hadess.net>
* mtest: remove pointless try/except from try_wait_onePaolo Bonzini2021-03-231-5/+3
| | | | | asyncio.wait does not return an asyncio.TimeoutError, so there is no exception to catch.
* split program related classes and functions out of dependenciesDylan Baker2021-03-191-1/+1
| | | | | | Dependencies is already a large and complicated package without adding programs to the list. This also allows us to untangle a bit of spaghetti that we have.
* mass rewrite of string formatting to use f-strings everywhereEli Schwartz2021-03-041-28/+28
| | | | performed by running "pyupgrade --py36-plus" and committing the results
* various python neatness cleanupsEli Schwartz2021-03-041-6/+6
| | | | | | | | | | | | | | | | All changes were created by running "pyupgrade --py3-only --keep-percent-format" and committing the results. I have not touched string formatting for now. - use set literals - simplify .format() parameter naming - remove __future__ - remove default "r" mode for open() - use OSError rather than compatibility aliases - remove stray parentheses in function(generator) scopes
* mtest: create separate runners for multiple repeatsPaolo Bonzini2021-03-021-18/+21
| | | | | | | | | | | Reusing the runners for multiple repeats of the test run gets in the way of the progress report, which stores runners in an OrderedSet. Instead, create a separate SingleTestRunner object for each repeat. While at it, fix the "duplicate suite" assertion as it can fire with TAP tests and --repeat=N. Fixes: #8405
* [TAP] Fix TAP parser when test exits with statusLuke Drummond2021-02-191-0/+1
| | | | | | | | | | Some time between 0.56 and 0.57 the TAP parser broke when a test exits with a nonzero status. The TAP protocol does not specify this behaviour - giving latitude to implementers, and meson's previous behaviour was to report the exit status gracefully. This patch restores the old behaviour and adds a regression test
* mtest: use / as path separator for ninja targetsPaolo Bonzini2021-02-171-1/+7
| | | | | | | | | | os.path.relpath(f, wd) returns path with \ seperator on Windows, but ninja targets always uses / separator. See for example https://gitlab.freedesktop.org/ocrete/libnice/-/jobs/7348274. Analyzed-by: Xavier Claessens <xavier.claessens@collabora.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: include classname in <testcase> JUnit elementPaolo Bonzini2021-02-101-2/+4
| | | | | | | | It looks like GitLab ignores the suite name and actually uses the classname. Adjust the output accordingly. Fixes: #8316 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: cancel stdout/stderr tasks on timeoutPaolo Bonzini2021-02-101-18/+35
| | | | | | | | | | | | | | | | Avoid that the tasks linger and SingleTestRunner.run() never terminates. In order to do this, we need read_decode() and read_decode_lines() to be cancellable, and to handle the CancelledError gracefully while returning the output they have collected so far. For read_decode(), this means always operating on a line-by-line basis, even if console_mode is not ConsoleUser.STDOUT. For read_decode_lines(), instead, we cannot return an iterator. Rather, read_decode_lines() returns the output directly (similar to read_decode) and communication with the parser is mediated by an asyncio.Queue. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: clean up conditions on whether tests are run in parallelPaolo Bonzini2021-02-101-10/+13
| | | | | | | | | This makes non-parallel tests emit their output on the fly, similar to ninja console jobs. It also cleans up the code a bit, avoiding the repetition of "self.options.num_processes" tests. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: fix nits in printing test stdoutPaolo Bonzini2021-02-101-4/+5
| | | | | | | | | - Ensure the output is terminated with a \n even if the test does not include one. - Ensure that stdout is flushed for each reported result Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: TestSetup can have [] as an exe_wrapperPaolo Bonzini2021-02-091-2/+2
| | | | | | | Fix "meson test --wrapper foo --setup bar", it should work just fine if the setup does not define a wrapper. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: run the test output parser as a taskPaolo Bonzini2021-02-091-5/+6
| | | | | | | | | | | | Start the parsing of the output early; this avoids a deadlock if the test writes to stdout but no one reads from it. It also reports TAP or Rust subtest results as they happen, which was the intention all along. While at it, use a consistent naming conventions for coroutines vs tasks. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: hide infinite timeout from the progress reportPaolo Bonzini2021-02-091-3/+8
| | | | | | | Avoid printing something like "30/-1s" when tests are run without a timeout or with --timeout-multiplier 0. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* interpreter, mtest: introduce add_test_setup(exclude_suites: ...)Paolo Bonzini2021-02-021-11/+15
| | | | | | | | | | | | This new keyword argument makes it possible to run specific test setups only on a subset of the tests. For example, to mark some tests as slow and avoid running them by default: add_test_setup('quick', exclude_suites: ['slow'], is_default: true) add_test_setup('slow') It will then be possible to run the slow tests with either `meson test --setup slow` or `meson test --suite slow`.
* mtest: extract get_test_setup, rename merge_suite_optionsPaolo Bonzini2021-02-021-10/+13
| | | | | | | merge_suite_options is really about test setups, so rename accordingly. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
* mtest: load build data earlyPaolo Bonzini2021-02-021-23/+21
| | | | | | This will be needed to exclude testsuites from test setups (which are stored in the build data). While at it, since a chdir is needed simplify a bit the loading of tests and benchmarks.
* mtest: compute logfile name earlyPaolo Bonzini2021-02-021-15/+18
| | | | Do this before overriding self.options.setup with the default setup.
* mtest: remove useless argumentPaolo Bonzini2021-02-021-3/+3
|
* Merge pull request #8200 from bonzini/mtest-asyncio-logsJussi Pakkanen2021-01-301-116/+234
|\ | | | | mtest: improvements to logging
| * mtest: log individual TAP subtestsPaolo Bonzini2021-01-221-11/+34
| |
| * mtest: infrastructure to print subtest resultsPaolo Bonzini2021-01-221-7/+18
| |
| * mtest: move --print-errorlogs output during the test runPaolo Bonzini2021-01-221-18/+13
| | | | | | | | | | | | | | Print the (shortened) output of the failed tests as they happen. If neither --verbose nor --print-errorlogs was specified, omit the summary of failures, because it is pretty much the same as the earlier output of "meson test".
| * mtest: log test start in verbose modePaolo Bonzini2021-01-221-13/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | In non-parallel verbose mode the output of the test/benchmark is not buffered, therefore the command line is only printed by ConsoleLogger for failing tests and only after the test has run. Verbose mode is designed mostly for CI systems, where output must be human readable but is generally consumed from a browser with "Find" commands rather than from a terminal. With this usecase in mind, it is better to provide as much detail as possible, so add more output and just tell the user which tests have started. Do so, using the recently introduced TestResult.RUNNING state.
| * mtest: call TestLogger.start_test when TestRun is in the RUNNING statePaolo Bonzini2021-01-221-8/+13
| |
| * mtest: use buffered stdout/stderr in parallel modePaolo Bonzini2021-01-221-2/+5
| | | | | | | | | | Similar to ninja, buffer stdout/stderr even in verbose mode if more than one test is being run in parallel.
| * mtest: quote environment variable values and command line argumentsPaolo Bonzini2021-01-221-2/+9
| |
| * mtest: include full environment in the TestRun objectPaolo Bonzini2021-01-221-32/+34
| | | | | | | | | | | | Ensure that all the required modifications are included in the logs. This makes it possible for users to cut-and-paste from the logs when trying to reproduce failures outside Meson.
| * mtest: make log output more suitable for consolePaolo Bonzini2021-01-221-37/+74
| | | | | | | | | | | | | | Right now the same code is used to print the logs for both the console and the text log. Differentiating them lets the important bits of the console output stand out, and makes the console output a bit more readable.
| * mtest: small refactoring of log printingPaolo Bonzini2021-01-221-23/+34
| | | | | | | | | | Start moving console-specific code to ConsoleLogger, as well as moving code out of get_log().
| * mtest: log in verbose mode, but not in gdb modePaolo Bonzini2021-01-221-1/+1
| | | | | | | | | | This is now possible because all stdout/stderr goes through asyncio pipes. However, logs make little sense in gdb mode.
* | test: Make timeout <= 0 infinite duractionXavier Claessens2021-01-271-6/+8
|/
* mtest: collect stdout/stderr even in verbose modePaolo Bonzini2021-01-201-14/+27
| | | | | | | | | | | | | | | | | | | | | | | Using verbose mode dropped stdout/stderr from the logs, because it was not captured. Now that we can easily stick code in the middle of the reading of stdout/stderr, use that to print stdout and stderr on the fly while also capturing them for the logs. The output is line-buffered. As a side effect, this also fixes a possible deadlock due to not using ensure_future around stdo_task and stde_task. In particular: - the stdo_task coroutine would not terminate until the test closed stdo_task - the stde_task coroutine would not start until the stdo_task coroutine finished Therefore, the test could get stuck waiting for its parent to read the contents of stderr, but that would not happen because Meson was still in the stdo_task coroutine.
* mtest: move I/O handling to TestSubprocessPaolo Bonzini2021-01-151-36/+44
| | | | | | | | | | Move the logic to start the read/decode tasks to TestSubprocess and keep SingleTestRunner simple. The lines() inner function is tweaked to produce stdout as a future. This removes the nonlocal access (which is not possible anymore when the code is moved out of _run_cmd), and also lets _run_cmd use "await stdo_task" for both parsed and unparsed output.
* mtest: tweak the gathering of stdo_task/stde_task resultsPaolo Bonzini2021-01-151-9/+14
| | | | | | | After the next patch, we will need to complete parse_task before stdo_task (because parse_task will not set the "stdo" variable anymore but it will still collect stdout just like now). Do the change now to isolate the more complicated changes.
* mtest: disable the progress report in gdb modePaolo Bonzini2021-01-151-3/+5
| | | | Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
View Lorry Controller Status