summaryrefslogtreecommitdiff
path: root/render-test
Commit message (Collapse)AuthorAgeFilesLines
...
* [test-runner] Simplify update option and remove obsolete ifdefsAlexander Shalamov2019-12-092-33/+12
|
* [metrics] Create gcc8-release metrics configurationAlexander Shalamov2019-12-091145-40096/+0
|
* [metrics] Update render-test-probe-unit-tests memory expectationsMikhail Pozdnyakov2019-12-094-16/+16
|
* [core] Refactoring transform state (#15956)zmiao2019-12-052-4/+4
| | | | | | | | | | * [core] Refactoring transform state class * [core] update matrix in transform, Fix precision * [core] Make matrix mutable so that we only update them when needed * [core] Add getters for matrices
* [test-runner] Make Android RenderTestRunner bot fail when test cases are not ↵zmiao2019-12-0428-14/+342
| | | | | | | | | | | | | | fully passed (#15994) * [test-runner] Fix failed tests, generate android specific expectations * [test-runner] Check render test runner testing result * [test-runner] Add error case to android ignore list * [test-runner] Update metrics * [test-runner] Move Android render tests with big diff(>0.006) to ignore list
* [test-runner] Allow to define filter inside manifest fileAlexander Shalamov2019-12-035-7/+17
| | | | | Command line option is still available and allows to override filter that is defined in a manifest file.
* [test runner] Update test commands for Layer::setProperty()Mikhail Pozdnyakov2019-12-031-20/+4
|
* [metrics] Add baselines for formatted-images and line-dasharray render testsAlexander Shalamov2019-12-029-0/+315
|
* [test-runner] Add android RenderTestRunner baseline metrics (#15979)zmiao2019-12-021136-3/+53232
| | | | | | | | | | * [test-runner] Enable probe-testing android render test runner * [test-runner] Add pixel4 baseline metrics * Add git ignored metrics * Make renderTestrunner run two times with different manifest config
* [test-runner] Log all 'probeFileSize' failuresAlexander Shalamov2019-12-021-5/+8
| | | | | If there are multiple file size probes in a test file, it is better to report all failures instead of first failed probe.
* [test-runner] Run render tests with metrics on linuxJuha Alanen2019-11-294-1/+25
|
* [test-runner] Add baseline metrics for linuxJuha Alanen2019-11-291134-0/+39762
|
* [render-test] Add support for measuring compressed file sizeThiago Marcos P. Santos2019-11-283-4/+30
| | | | | Useful because for platform such as Android, we want to measure the size of the library inside the APK, which is usually compressed.
* [metrics] Make the parsing of metrics even more strictThiago Marcos P. Santos2019-11-281-1/+1
| | | | File must be `metrics.json`.
* [render-test] Fix regex matching the directory nameThiago Marcos P. Santos2019-11-271-3/+3
| | | | | This code was matching `style.json` instead of `metrics.json` if the `style.json` was contained inside `metrics/.../style.json`.
* [test runner] Enable test results update from cmd lineMikhail Pozdnyakov2019-11-273-10/+45
| | | | | | | | Added the following command line argument: ``` -u[update], --update=[update] Test results update mode. Supported values are: "default", "platform", "metrics" ```
* [test-runner] Move probe related command line args to manifestAlexander Shalamov2019-11-275-38/+48
| | | | | | | | | | | - Move probe related command line arguments to manifest, example: "probes": ["probeGFX", "probeNetwork", "probeMemory"], "metric_path": "metrics/linux" - Use manifest file name as a postfix for a result page - Move injected probes 'begin' section before map object creation - Generalize artifact storing shell script
* [render-test] Add metric_paths into Manifest (#15973)zmiao2019-11-2711-197/+275
| | | | | | | | | | * [render-test]Add expected metrics path in manifest file * seperate metrics.json and expected.png * fix expected path parsing * keep expectation path if update is enabled
* [test-runner] Add render test runner options for injecting probesAlexander Shalamov2019-11-223-48/+177
| | | | | This allows injection of memory, gfx and network probes before and after operations defined in render test are run.
* [render-test] Adding mvt-fixture to app asset content (#15955)zmiao2019-11-211-0/+1
|
* [build] Build Android release when running render testsThiago Marcos P. Santos2019-11-211-3/+12
|
* [render-test] Config CircleCi to run RenderTestApp on Firebase with Pixel4 ↵zmiao2019-11-201-0/+11
| | | | | (#15951) This reverts commit d2413d4ffffd087b352b6e4edd6e6322a311bc07.
* [render-test] Wrap test data inside RenderTestRunner App + Add a new test ↵zmiao2019-11-1913-59/+96
| | | | | | | | | | | | app (#15887) * [render-test] Wrap test resources inside app * [render-test] Add test app * fix test app failure * [render-test]add callback + add javaObjectWrapper
* fix -Wfor-loop-analysis and -Wreorder warnings (#15901)Dane Springmeyer2019-11-122-4/+5
|
* [render-test] Fix render-test manifest confilicted local and member names ↵zmiao2019-11-051-8/+8
| | | | (#15888)
* [render-test]Fix regex match for testFilter input (#15886)zmiao2019-11-041-2/+2
|
* [render-test] Add gfx probe test casesMikko Pulkki2019-10-3031-1/+266
|
* [render-test] Implement gfx probe for tracking gpu resourcesMikko Pulkki2019-10-304-6/+237
|
* [render-test] Add manifest parser for render-test-runner (#15861)zmiao2019-10-2913-440/+621
| | | | | | | | | | * [render-test] Add manifest parser for render-test-runner * [render-test] Refactory manifest parser * [render-test] Parse full manifest file through CLI * [render-test] Add linux probe manifest
* [core,android,darwin,qt] Add fields related to feature-stateJuha Alanen2019-10-281-7/+13
| | | | | Move the fields from geometry.hpp/feature.hpp as they are not part of the GeoJSON specification.
* [render-test] Add configurable rootPath, ignorePath via CLI arguments (#15832)zmiao2019-10-235-106/+160
| | | | | | * [render-test] Add configurable path via CLI arguments * [render-test] Change ignore path config
* [test-runner] Run render test runner as a NativeActivity app on android ↵zmiao2019-10-2218-5/+388
| | | | | | | | | | | | device (#15827) * [test-runner]Add android render-test-runner NativeActivity app * [test-runner] Fix cmake for CI build failure * [test-runner] Update folder location, change app configureation * [test-runner] Add running instruction
* [render-test] Implement fps benchmarking tests (#15803)Mikko Pulkki2019-10-225-74/+275
|
* [test runner] Improve error messaging for memory metrics testsMikhail Pozdnyakov2019-10-212-15/+10
|
* [test runner] Memory metrics expectations for MACMikhail Pozdnyakov2019-10-212-0/+19
|
* [test runner] Command line option for the tests expectations pathMikhail Pozdnyakov2019-10-214-30/+38
| | | | | | -e[expectationsPath], --expectationsPath=[expectationsPath] Test expectations path.
* [test runner] Adjust memory metrics test resultsMikhail Pozdnyakov2019-10-215-5/+62
|
* [test runner] Disable memory metrics checks on sanitize buildsMikhail Pozdnyakov2019-10-211-1/+2
|
* [test runner] Tests for memory size metricsMikhail Pozdnyakov2019-10-2111-2/+163
|
* [test runner] Threshold for memory metricsMikhail Pozdnyakov2019-10-213-13/+30
|
* [render-test] Add network metrics test caseJuha Alanen2019-10-1613-1/+328
|
* [render-test] Add support for collecting network metricsJuha Alanen2019-10-165-1/+202
|
* [test-runner] make render-test a shared lib (#15816)zmiao2019-10-152-2/+16
| | | | | | | | | | * [test-runner] make render-test a shared lib [test-runner] nit [test-runner] nit again * [test-runner] Remove redundant linked file in cmake
* [render-test] Add test for file size toleranceThiago Marcos P. Santos2019-10-146-0/+92
|
* [render-test] Add tolerance to file-size testsThiago Marcos P. Santos2019-10-149-16/+27
|
* [render-test] No camelCase on the metrics.json tagThiago Marcos P. Santos2019-10-1116-12/+12
| | | | It is ugly.
* [render-test] Prettyfy metrics outputThiago Marcos P. Santos2019-10-115-5/+57
|
* [render-test] Add more probe tests for file sizeThiago Marcos P. Santos2019-10-1113-32/+50
| | | | Also make the tests simpler.
* [render-test] Fix UPDATE_METRICS=1 changing the path of file size testsThiago Marcos P. Santos2019-10-114-9/+11
| | | | Should use the exact same path as described on style.json
* [render-test] Only write metrics if they are availableThiago Marcos P. Santos2019-10-101-18/+20
| | | | Otherwise the file will have an empty tag.