diff options
| author | Juha Alanen <juha.alanen@mapbox.com> | 2020-04-11 15:58:48 +0300 |
|---|---|---|
| committer | Juha Alanen <juha.alanen@mapbox.com> | 2020-04-15 19:11:28 +0300 |
| commit | f3df304a20249bf83e3e635528994d8cda82966a (patch) | |
| tree | 640fcdb2cc4a8dec9fb453f0fc54e379523522fc | |
| parent | 3a91597c5d218235d57b3c822374a572d0339d41 (diff) | |
| download | qtlocation-mapboxgl-f3df304a20249bf83e3e635528994d8cda82966a.tar.gz | |
[build] Compare benchmark results to baseline
| -rw-r--r-- | circle.yml | 15 |
1 files changed, 15 insertions, 0 deletions
diff --git a/circle.yml b/circle.yml index c41cda627e..a604f51a14 100644 --- a/circle.yml +++ b/circle.yml @@ -291,6 +291,7 @@ commands: command: | brew install cmake ccache glfw ninja pkgconfig qt chargepoint/xcparse/xcparse brew cask install google-cloud-sdk + pip install ansi2html scipy config: parameters: config_params: @@ -559,6 +560,11 @@ jobs: cd ../.. && mkdir -p /tmp/tests/benchmark if ls benchmark-app/TestLogs/*.json 1> /dev/null 2>&1; then cp benchmark-app/TestLogs/*.json /tmp/tests/benchmark/results.json; fi fi + - run: + name: Compare result to baseline + when: always + command: | + scripts/check_benchmark_results.sh metrics/benchmark/ios/results.json /tmp/tests/benchmark/results.json /tmp/tests/benchmark/results_comparison.txt /tmp/tests/benchmark/results_comparison.html - save android-benchmark-runner: executor: ubuntu-disco @@ -589,6 +595,15 @@ jobs: testResult=$(gsutil ls gs://test-lab-186672a0qp5bq-ycr70axads3nc/benchmark-${CIRCLE_BUILD_NUM}/**/results/results.json) mkdir -p /tmp/tests/benchmark gsutil cp $testResult /tmp/tests/benchmark + - run: + name: Install extra packages for result comparison + command: | + apt install -y colorized-logs python-scipy + - run: + name: Compare result to baseline + when: always + command: | + scripts/check_benchmark_results.sh metrics/benchmark/android/results.json /tmp/tests/benchmark/results.json /tmp/tests/benchmark/results_comparison.txt /tmp/tests/benchmark/results_comparison.html - save android-api-breakage: executor: ubuntu-disco |