summaryrefslogtreecommitdiff
path: root/selftest/subunithelper.py
Commit message (Collapse)AuthorAgeFilesLines
* selftest: subunithelper needs to follow the subunit spec more closelyDouglas Bagnall2018-08-231-1/+2
| | | | | | | | In particular allow ]\n without \n]\n as used by cmocka Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> (cherry picked from commit 7d79575de8e47a0ce03e30c3ea84176be696269f)
* selftest: use an additional directory of knownfail/flapping filesDouglas Bagnall2017-06-031-14/+27
| | | | | | | | | | | This makes it easier to add a temporary knownfail to cover a patch series. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Sat Jun 3 13:55:41 CEST 2017 on sn-devel-144
* make perftest: for performance testingDouglas Bagnall2016-08-311-0/+68
| | | | | | | | | | This runs a selection of subunit tests and reduces the output to only the time it takes to run each test. The tests are listed in selftest/perf_tests.py. Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* filter-subunit: default to empty affixes, saving verbose checksDouglas Bagnall2016-08-311-8/+1
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* subunithelper: use set for efficient inclusion testDouglas Bagnall2016-08-311-1/+5
| | | | | Signed-off-by: Douglas Bagnall <douglas.bagnall@catalyst.net.nz> Reviewed-by: Garming Sam <garming@catalyst.net.nz>
* Remove another call to addUnexpectedSuccess with too many arguments.Jelmer Vernooij2015-03-061-1/+1
| | | | | | Change-Id: I5f1917e74ffd57b5ae228231c27fbdf70de2992f Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* TestCase.addUnexpectedSuccess doesn't take an error.Jelmer Vernooij2015-03-061-3/+3
| | | | | | Change-Id: I90f7dd225d9ed3cbc515292de9a37a816ac0639f Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Drop support for failfast mode, rather than adding support for it everywhere.Jelmer Vernooij2015-03-061-4/+0
| | | | | | Change-Id: I4d6070a0e3b89d5e390f84754dddba9ec17ddf21 Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Set failfast property for test reporters that need it.Jelmer Vernooij2015-03-061-0/+4
| | | | | | Change-Id: Ibd632b9f569c23e60bcd13bcff805e367dd2e71c Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Support using third party iso8601 module if system doesn't provide one.Jelmer Vernooij2015-03-061-1/+1
| | | | | | Change-Id: I5d035738d244d66d33788636c8ee8b322c227a0e Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* subunithelper: Fix progress support.Jelmer Vernooij2015-03-061-1/+8
| | | | | | Change-Id: I5dd2ca0f3bc02821d5c9e1dc878bba577667d162 Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Use iso8601 from the system, rather than the one bundled with subunit.Jelmer Vernooij2015-03-061-1/+1
| | | | | | Change-Id: I681bf79eb9ebe45b6972b3783c8e799eb612400b Signed-Off-By: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Fix handling of unexpected failures in subunithelper.Jelmer Vernooij2015-03-061-1/+1
| | | | | | Change-Id: I0da9fcec4a54c43c171b76bb9015ea84389c9bc6 Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Use samba.subunit in selftest.subunithelper, except for iso8601.Jelmer Vernooij2015-03-061-44/+38
| | | | | | Change-Id: Iec43e0368db91dbe981b39ac8388b50c1eafee0c Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* Display total number of tests in format-subunit, not just number of test suites.Jelmer Vernooij2014-11-191-1/+7
| | | | | | Change-Id: Iae17df773f380b13d2b3f69e83593139c01c4a6d Signed-Off-By: Jelmer Vernooij <jelmer@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* subunit: report [X/Y at Zs] instead of [X/Y in Zs]Michael Adam2014-10-231-1/+1
| | | | | | | when running test X out of Y after Z secons have passed Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Matthieu Patou <mat@matws.net>
* selftest/subunithelper.py: correctly pass testsuite-uxsuccess to end_testsuite()Stefan Metzmacher2014-03-131-1/+4
| | | | | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Jeremy Allison <jra@samba.org> Autobuild-Date(master): Thu Mar 13 23:49:36 CET 2014 on sn-devel-104
* selftest/subunithelper.py: correctly handle fail_immediately in ↵Stefan Metzmacher2014-03-131-0/+6
| | | | | | | | | end_testsuite of FilterOps This way --fail-immediately also works if a command segfaults. Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* selftest/subunithelper.py: correctly handle unexpected success in FilterOpsStefan Metzmacher2014-03-131-1/+13
| | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* selftest: Place the test environment at the end of the failure linesAndrew Bartlett2012-04-271-4/+9
| | | | | | | This allows a knownfail entry to be added for only one environment, rather than all environments. Andrew Bartlett
* selftest: Display unexpected successes and expected failures.Jelmer Vernooij2011-12-051-13/+35
|
* selftest: Add --flapping argument to filter-subunit.Jelmer Vernooij2011-12-051-2/+26
|
* subunithelper: Exit with 1 if more than zero testsuites failed orJelmer Vernooij2010-11-211-5/+10
| | | | | | | errorred. Autobuild-User: Jelmer Vernooij <jelmer@samba.org> Autobuild-Date: Sun Nov 21 21:13:00 CET 2010 on sn-devel-104
* filter-subunit: Add --fail-on-empty argument.Jelmer Vernooij2010-11-031-0/+3
|
* subunithelper: Don't access 'message' attribute directly as it deprecatedKamen Mazdrashki2010-10-101-1/+1
| | | | | | | Signed-off-by: Jelmer Vernooij <jelmer@samba.org> Autobuild-User: Jelmer Vernooij <jelmer@samba.org> Autobuild-Date: Sun Oct 10 01:50:09 UTC 2010 on sn-devel-104
* subunithelper: Remove accidentally added line.Jelmer Vernooij2010-10-021-3/+0
|
* subunithelper: Fix format time.Jelmer Vernooij2010-10-021-1/+2
|
* subunithelper: Make filter options optional.Jelmer Vernooij2010-10-021-9/+9
|
* selftest: Move plain text formatter to subunithelper.Jelmer Vernooij2010-10-021-1/+203
|
* selftest: Avoid accessing deprecated BaseException.message.Jelmer Vernooij2010-09-301-1/+1
| | | | Thanks to Andreas for pointing this out.
* Add dedicated exception for immediate failure in filter-subunit, don't raise ↵Jelmer Vernooij2010-09-281-3/+10
| | | | it on known exceptions.
* filter-subunit: added a --fail-immediately optionAndrew Tridgell2010-09-281-1/+7
|
* subunithelper: Cope with errors outside of testsuites.Jelmer Vernooij2010-09-221-0/+1
|
* selftest: Let tests add prefix to tests by themselve.Jelmer Vernooij2010-09-221-1/+1
|
* subunit: Re-add outputmsg.Jelmer Vernooij2010-09-151-0/+3
|
* subunit: Use RemoteError when passing errors to upstream subunit.Jelmer Vernooij2010-09-141-16/+19
|
* subunit: Use standard subunit functions for reproducing subunit streams.Jelmer Vernooij2010-09-141-50/+23
|
* subunit: Remove unused methods.Jelmer Vernooij2010-09-141-14/+2
|
* subunit: Use standard functions for addSuccess, addExpectedFail,Jelmer Vernooij2010-09-141-27/+62
| | | | addFailure, addSkip.
* subunit: Use standard addError method implementation.Jelmer Vernooij2010-09-141-9/+11
|
* subunit: Pass TestCase objects to startTest rather than test name strings.Jelmer Vernooij2010-09-141-7/+4
|
* subunit: Use subunit standard functions for handling time and progress.Jelmer Vernooij2010-09-141-29/+9
|
* subunit: Use standard subunit test protocol client, use standard name for ↵Jelmer Vernooij2010-09-141-21/+23
| | | | startTest.
* subunit: Initial work on using the standard TestResult class.Jelmer Vernooij2010-09-141-1/+2
|
* selftest/subunit: Parse timestamps generated by upstream subunit.Jelmer Vernooij2010-09-051-2/+8
|
* subunit: Make sure "]" in failure reason ends up on its own line, so theJelmer Vernooij2010-08-261-2/+2
| | | | next subunit parser in the line parses it correctly.
* subunithelper: Pass along lines as normal output when ignoring them for not ↵Jelmer Vernooij2010-08-261-0/+4
| | | | containing commands.
* selftest: list.remove raises ValueError rather than KeyError.Jelmer Vernooij2010-05-301-4/+4
|
* subunit: Support formatting compatible with upstream subunit, for consistency.Jelmer Vernooij2010-04-111-13/+19
| | | | | | Upstream subunit makes a ":" after commands optional, so I've fixed any places where we might trigger commands accidently. I've filed a bug about this in subunit.
* selftest: Accept 'testing:' to introduce new tests.Jelmer Vernooij2010-04-111-1/+1
|