diff options
author | Allan Sandfeld Jensen <allan.jensen@qt.io> | 2020-10-12 14:27:29 +0200 |
---|---|---|
committer | Allan Sandfeld Jensen <allan.jensen@qt.io> | 2020-10-13 09:35:20 +0000 |
commit | c30a6232df03e1efbd9f3b226777b07e087a1122 (patch) | |
tree | e992f45784689f373bcc38d1b79a239ebe17ee23 /chromium/docs/testing/web_tests.md | |
parent | 7b5b123ac58f58ffde0f4f6e488bcd09aa4decd3 (diff) | |
download | qtwebengine-chromium-85-based.tar.gz |
BASELINE: Update Chromium to 85.0.4183.14085-based
Change-Id: Iaa42f4680837c57725b1344f108c0196741f6057
Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Diffstat (limited to 'chromium/docs/testing/web_tests.md')
-rw-r--r-- | chromium/docs/testing/web_tests.md | 57 |
1 files changed, 29 insertions, 28 deletions
diff --git a/chromium/docs/testing/web_tests.md b/chromium/docs/testing/web_tests.md index 5e4d188e293..faa428ca938 100644 --- a/chromium/docs/testing/web_tests.md +++ b/chromium/docs/testing/web_tests.md @@ -55,23 +55,26 @@ strip ./xcodebuild/{Debug,Release}/content_shell.app/Contents/MacOS/content_shel TODO: mention `testing/xvfb.py` -The test runner script is in -`third_party/blink/tools/run_web_tests.py`. +The test runner script is in `third_party/blink/tools/run_web_tests.py`. To specify which build directory to use (e.g. out/Default, out/Release, out/Debug) you should pass the `-t` or `--target` parameter. For example, to use the build in `out/Default`, use: ```bash -python third_party/blink/tools/run_web_tests.py -t Default +third_party/blink/tools/run_web_tests.py -t Default ``` For Android (if your build directory is `out/android`): ```bash -python third_party/blink/tools/run_web_tests.py -t android --android +third_party/blink/tools/run_web_tests.py -t android --android ``` +*** promo +Windows users need to use `third_party/blink/tools/run_web_tests.bat` instead. +*** + Tests marked as `[ Skip ]` in [TestExpectations](../../third_party/blink/web_tests/TestExpectations) won't be run by default, generally because they cause some intractable tool error. @@ -97,13 +100,13 @@ arguments to `run_web_tests.py` relative to the web test directory use: ```bash -python third_party/blink/tools/run_web_tests.py fast/forms +third_party/blink/tools/run_web_tests.py fast/forms ``` Or you could use the following shorthand: ```bash -python third_party/blink/tools/run_web_tests.py fast/fo\* +third_party/blink/tools/run_web_tests.py fast/fo\* ``` *** promo @@ -111,7 +114,7 @@ Example: To run the web tests with a debug build of `content_shell`, but only test the SVG tests and run pixel tests, you would run: ```bash -[python] third_party/blink/tools/run_web_tests.py -t Default svg +third_party/blink/tools/run_web_tests.py -t Default svg ``` *** @@ -143,7 +146,7 @@ for more details of running `content_shell`. To see a complete list of arguments supported, run: ```bash -python third_party/blink/tools/run_web_tests.py --help +third_party/blink/tools/run_web_tests.py --help ``` *** note @@ -217,7 +220,7 @@ There are two ways to run web tests with additional command-line arguments: * Using `--additional-driver-flag`: ```bash - python run_web_tests.py --additional-driver-flag=--blocking-repaint + third_party/blink/tools/run_web_tests.py --additional-driver-flag=--blocking-repaint ``` This tells the test harness to pass `--blocking-repaint` to the @@ -382,10 +385,10 @@ tips for finding the problem. spacing or box sizes are often unimportant, especially around fonts and form controls. Differences in wording of JS error messages are also usually acceptable. - * `python run_web_tests.py path/to/your/test.html` produces a page listing - all test results. Those which fail their expectations will include links - to the expected result, actual result, and diff. These results are saved - to `$root_build_dir/layout-test-results`. + * `third_party/blink/tools/run_web_tests.py path/to/your/test.html` produces + a page listing all test results. Those which fail their expectations will + include links to the expected result, actual result, and diff. These + results are saved to `$root_build_dir/layout-test-results`. * Alternatively the `--results-directory=path/for/output/` option allows you to specify an alternative directory for the output to be saved to. * If you're still sure it's correct, rebaseline the test (see below). @@ -428,8 +431,7 @@ tips for finding the problem. To run the server manually to reproduce/debug a failure: ```bash -cd src/third_party/blink/tools -python run_blink_httpd.py +third_party/blink/tools/run_blink_httpd.py ``` The web tests are served from `http://127.0.0.1:8000/`. For example, to @@ -473,18 +475,12 @@ machine? ### Debugging DevTools Tests -* Add `debug_devtools=true` to `args.gn` and compile: `autoninja -C out/Default devtools_frontend_resources` - > Debug DevTools lets you avoid having to recompile after every change to the DevTools front-end. * Do one of the following: * Option A) Run from the `chromium/src` folder: - `third_party/blink/tools/run_web_tests.sh - --additional-driver-flag='--debug-devtools' - --additional-driver-flag='--remote-debugging-port=9222' - --time-out-ms=6000000` + `third_party/blink/tools/run_web_tests.py --additional-driver-flag='--remote-debugging-port=9222' --additional-driver-flag='--debug-devtools' --time-out-ms=6000000` * Option B) If you need to debug an http/tests/inspector test, start httpd as described above. Then, run content_shell: - `out/Default/content_shell --debug-devtools --remote-debugging-port=9222 --run-web-tests - http://127.0.0.1:8000/path/to/test.html` + `out/Default/content_shell --remote-debugging-port=9222 --additional-driver-flag='--debug-devtools' --run-web-tests http://127.0.0.1:8000/path/to/test.html` * Open `http://localhost:9222` in a stable/beta/canary Chrome, click the single link to open the devtools with the test loaded. * In the loaded devtools, set any required breakpoints and execute `test()` in @@ -546,8 +542,7 @@ read on. *** ```bash -cd src/third_party/blink -python tools/run_web_tests.py --reset-results foo/bar/test.html +third_party/blink/tools/run_web_tests.py --reset-results foo/bar/test.html ``` If there are current expectation files for `web_tests/foo/bar/test.html`, @@ -569,12 +564,18 @@ Though we prefer the Rebaseline Tool to local rebaselining, the Rebaseline Tool doesn't support rebaselining flag-specific expectations. ```bash -cd src/third_party/blink -python tools/run_web_tests.py --additional-driver-flag=--enable-flag --reset-results foo/bar/test.html +third_party/blink/tools/run_web_tests.py --additional-driver-flag=--enable-flag --reset-results foo/bar/test.html ``` +*** promo +You can use `--flag-specific=config` as a shorthand of +`--additional-driver-flag=--enable-flag` if `config` is defined in +`web_tests/FlagSpecificConfig`. +*** New baselines will be created in the flag-specific baselines directory, e.g. -`web_tests/flag-specific/enable-flag/foo/bar/test-expected.{txt,png}`. +`web_tests/flag-specific/enable-flag/foo/bar/test-expected.{txt,png}` +or +`web_tests/flag-specific/config/foo/bar/test-expected.{txt,png}` Then you can commit the new baselines and upload the patch for review. |