diff options
Diffstat (limited to 'testsuite/driver/README.md')
| -rw-r--r-- | testsuite/driver/README.md | 14 |
1 files changed, 9 insertions, 5 deletions
diff --git a/testsuite/driver/README.md b/testsuite/driver/README.md index 6ae7db6842..786f98a6f5 100644 --- a/testsuite/driver/README.md +++ b/testsuite/driver/README.md @@ -34,11 +34,15 @@ differences, and things such as that are not necessary to be considered by the test writer anymore. This is due to the fact that the test comparison relies entirely on locally collected metrics on the testing machine. -As such, it is perfectly sufficient to write `collect_stats('all',5)` to +As such, it is perfectly sufficient to write `collect_stats('all',20)` to measure the 3 potential stats that can be collected for that test and -automatically test them for regressions, failing if there is more than a 5% -change in any direction. In fact, even that is not necessary as -`collect_stats()` defaults to 'all', and 20% deviation allowed. See the +automatically test them for regressions, failing if there is more than a 20% +change in any direction. In fact, even that is not necessary as +`collect_stats()` defaults to 'all', and 20% deviation allowed. + +The function `collect_compiler_stats()` is completely equivalent in every way to +`collect_stats` except that it measures the performance of the compiler itself +rather than the performance of the code generated by the compiler. See the implementation of collect_stats in /driver/perf_notes.py for more information. If the performance of a test is improved so much that the test fails, the value @@ -138,7 +142,7 @@ you checkout that commit and run the testsuite (or test). * accumulate_metrics - Exists in testglobals.py - A list of strings in the form `'\t'.join('test_env','test','way', - 'metric','value')` + 'metric','value')` This is what is written to the git notes for the HEAD commit. As such, this 'value' /does/ represent the actual performance measured during the test-suite's run. |