summaryrefslogtreecommitdiff
path: root/render-test/render_test.cpp
Commit message (Collapse)AuthorAgeFilesLines
* [render-test] Make the manifest mandatoryThiago Marcos P. Santos2020-01-081-6/+4
| | | | At the moment we have an invalid path as default.
* [render-test] Remove positional argumentThiago Marcos P. Santos2020-01-081-11/+5
| | | | Not in use, for running a single test, use -f instead.
* [render-test] Effectively use the offline cache for fixturesThiago Marcos P. Santos2020-01-081-1/+5
|
* [render-test] Add flag to run tests onlineThiago Marcos P. Santos2020-01-081-3/+11
| | | | | And also make it not go to the network by setting the status explicitly when when the -o flag is not passed.
* [render-test] Cleanup on the render tests resultsThiago Marcos P. Santos2019-12-161-5/+9
| | | | | Simplify the code path and make it print both rendering error and metrics on fail.
* [render-test] Fix metrics passing when failingThiago Marcos P. Santos2019-12-161-4/+15
| | | | | | Metrics will now fail, but the tool will return 0 when *ONLY* metrics fail, to make the bot continue. We will also run ignored tests and check if they are ignored but passing.
* [render test] Fix probes failure reportsMikhail Pozdnyakov2019-12-131-5/+1
|
* [test-runner] Don't update nor create metrics baseline for ignored testAlexander Shalamov2019-12-121-1/+1
|
* [test-runner] Add desctiption for --update commandAlexander Shalamov2019-12-101-1/+4
|
* [test runner] Report absolute path to the results html fileMikhail Pozdnyakov2019-12-101-1/+1
|
* [test-runner] Add rebaseline flagAlexander Shalamov2019-12-091-1/+2
| | | | | Rebaseline flag allows to update or create new metric, in cases when metric check is failed or expected metric is missing.
* [test-runner] Simplify update option and remove obsolete ifdefsAlexander Shalamov2019-12-091-25/+12
|
* [test-runner] Allow to define filter inside manifest fileAlexander Shalamov2019-12-031-3/+0
| | | | | Command line option is still available and allows to override filter that is defined in a manifest file.
* [test runner] Enable test results update from cmd lineMikhail Pozdnyakov2019-11-271-3/+35
| | | | | | | | Added the following command line argument: ``` -u[update], --update=[update] Test results update mode. Supported values are: "default", "platform", "metrics" ```
* [test-runner] Move probe related command line args to manifestAlexander Shalamov2019-11-271-22/+8
| | | | | | | | | | | - Move probe related command line arguments to manifest, example: "probes": ["probeGFX", "probeNetwork", "probeMemory"], "metric_path": "metrics/linux" - Use manifest file name as a postfix for a result page - Move injected probes 'begin' section before map object creation - Generalize artifact storing shell script
* [test-runner] Add render test runner options for injecting probesAlexander Shalamov2019-11-221-4/+20
| | | | | This allows injection of memory, gfx and network probes before and after operations defined in render test are run.
* [render-test] Wrap test data inside RenderTestRunner App + Add a new test ↵zmiao2019-11-191-3/+5
| | | | | | | | | | | | app (#15887) * [render-test] Wrap test resources inside app * [render-test] Add test app * fix test app failure * [render-test]add callback + add javaObjectWrapper
* [render-test] Add manifest parser for render-test-runner (#15861)zmiao2019-10-291-25/+87
| | | | | | | | | | * [render-test] Add manifest parser for render-test-runner * [render-test] Refactory manifest parser * [render-test] Parse full manifest file through CLI * [render-test] Add linux probe manifest
* [render-test] Add configurable rootPath, ignorePath via CLI arguments (#15832)zmiao2019-10-231-6/+7
| | | | | | * [render-test] Add configurable path via CLI arguments * [render-test] Change ignore path config
* [render-test] Implement fps benchmarking tests (#15803)Mikko Pulkki2019-10-221-1/+2
|
* [test runner] Improve error messaging for memory metrics testsMikhail Pozdnyakov2019-10-211-3/+1
|
* [test runner] Adjust memory metrics test resultsMikhail Pozdnyakov2019-10-211-0/+3
|
* [test-runner] make render-test a shared lib (#15816)zmiao2019-10-151-0/+167
* [test-runner] make render-test a shared lib [test-runner] nit [test-runner] nit again * [test-runner] Remove redundant linked file in cmake