diff options
| author | zmiao <miao.zhao@mapbox.com> | 2019-12-04 18:10:58 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2019-12-04 18:10:58 +0200 |
| commit | e619552a5df5fdfa3c95efa0959ff2da20eac452 (patch) | |
| tree | 3974202bcb6db6ed097141b3f5b153678a2a648b /render-test/ignores | |
| parent | 1e2cd1e086c3053f1cea075d7641b7f56cc7dfbf (diff) | |
| download | qtlocation-mapboxgl-e619552a5df5fdfa3c95efa0959ff2da20eac452.tar.gz | |
[test-runner] Make Android RenderTestRunner bot fail when test cases are not fully passed (#15994)
* [test-runner] Fix failed tests, generate android specific expectations
* [test-runner] Check render test runner testing result
* [test-runner] Add error case to android ignore list
* [test-runner] Update metrics
* [test-runner] Move Android render tests with big diff(>0.006) to ignore list
Diffstat (limited to 'render-test/ignores')
| -rw-r--r-- | render-test/ignores/android-temp-fail.json | 9 | ||||
| -rw-r--r-- | render-test/ignores/linux-ignores.json | 11 |
2 files changed, 20 insertions, 0 deletions
diff --git a/render-test/ignores/android-temp-fail.json b/render-test/ignores/android-temp-fail.json new file mode 100644 index 0000000000..655c697528 --- /dev/null +++ b/render-test/ignores/android-temp-fail.json @@ -0,0 +1,9 @@ +{ + "render-tests/icon-text-fit/both-text-anchor-1x-image-2x-screen": "Temporarily fails as android test app does not derefence symbolinks in sprites folder.", + "render-tests/icon-text-fit/both-text-anchor-2x-image-1x-screen": "Temporarily fails as android test app does not derefence symbolinks in sprites folder.", + "render-tests/regressions/mapbox-gl-native#9976": "Needs to be investigated and fixed.", + "render-tests/regressions/mapbox-gl-js#5911a": "Needs to be investigated and fixed.", + "render-tests/line-pattern/overscaled": "Needs to be investigated and fixed.", + "render-tests/runtime-styling/image-add-pattern": "Needs to be investigated and fixed.", + "render-tests/runtime-styling/image-update-pattern": "Needs to be investigated and fixed." +} diff --git a/render-test/ignores/linux-ignores.json b/render-test/ignores/linux-ignores.json new file mode 100644 index 0000000000..d5a1a85c6c --- /dev/null +++ b/render-test/ignores/linux-ignores.json @@ -0,0 +1,11 @@ +{ + "render-tests/regressions/mapbox-gl-js#7066": "Failing with mbgl-render-test", + "render-tests/regressions/mapbox-gl-js#5642": "Failing with mbgl-render-test", + "render-tests/line-pattern/opacity": "Flaky on Linux: https://github.com/mapbox/mapbox-gl-native/issues/15320", + "render-tests/fill-opacity/zoom-and-property-function-pattern": "Flaky on Linux: https://github.com/mapbox/mapbox-gl-native/issues/15322", + "render-tests/fill-pattern/literal": "Flaky on Linux: https://github.com/mapbox/mapbox-gl-native/issues/14423", + "render-tests/fill-pattern/opacity": "Flaky on Linux: https://github.com/mapbox/mapbox-gl-native/issues/14870", + "render-tests/fill-pattern/zoomed": "Flaky on Linux: https://github.com/mapbox/mapbox-gl-native/issues/14768", + "render-tests/line-translate/literal": "Flaky on Linux: https://github.com/mapbox/mapbox-gl-native/issues/14859" +} + |