summaryrefslogtreecommitdiff
path: root/chromium/docs/testing/web_tests.md
diff options
context:
space:
mode:
authorAllan Sandfeld Jensen <allan.jensen@qt.io>2020-10-12 14:27:29 +0200
committerAllan Sandfeld Jensen <allan.jensen@qt.io>2020-10-13 09:35:20 +0000
commitc30a6232df03e1efbd9f3b226777b07e087a1122 (patch)
treee992f45784689f373bcc38d1b79a239ebe17ee23 /chromium/docs/testing/web_tests.md
parent7b5b123ac58f58ffde0f4f6e488bcd09aa4decd3 (diff)
downloadqtwebengine-chromium-85-based.tar.gz
BASELINE: Update Chromium to 85.0.4183.14085-based
Change-Id: Iaa42f4680837c57725b1344f108c0196741f6057 Reviewed-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Diffstat (limited to 'chromium/docs/testing/web_tests.md')
-rw-r--r--chromium/docs/testing/web_tests.md57
1 files changed, 29 insertions, 28 deletions
diff --git a/chromium/docs/testing/web_tests.md b/chromium/docs/testing/web_tests.md
index 5e4d188e293..faa428ca938 100644
--- a/chromium/docs/testing/web_tests.md
+++ b/chromium/docs/testing/web_tests.md
@@ -55,23 +55,26 @@ strip ./xcodebuild/{Debug,Release}/content_shell.app/Contents/MacOS/content_shel
TODO: mention `testing/xvfb.py`
-The test runner script is in
-`third_party/blink/tools/run_web_tests.py`.
+The test runner script is in `third_party/blink/tools/run_web_tests.py`.
To specify which build directory to use (e.g. out/Default, out/Release,
out/Debug) you should pass the `-t` or `--target` parameter. For example, to
use the build in `out/Default`, use:
```bash
-python third_party/blink/tools/run_web_tests.py -t Default
+third_party/blink/tools/run_web_tests.py -t Default
```
For Android (if your build directory is `out/android`):
```bash
-python third_party/blink/tools/run_web_tests.py -t android --android
+third_party/blink/tools/run_web_tests.py -t android --android
```
+*** promo
+Windows users need to use `third_party/blink/tools/run_web_tests.bat` instead.
+***
+
Tests marked as `[ Skip ]` in
[TestExpectations](../../third_party/blink/web_tests/TestExpectations)
won't be run by default, generally because they cause some intractable tool error.
@@ -97,13 +100,13 @@ arguments to `run_web_tests.py` relative to the web test directory
use:
```bash
-python third_party/blink/tools/run_web_tests.py fast/forms
+third_party/blink/tools/run_web_tests.py fast/forms
```
Or you could use the following shorthand:
```bash
-python third_party/blink/tools/run_web_tests.py fast/fo\*
+third_party/blink/tools/run_web_tests.py fast/fo\*
```
*** promo
@@ -111,7 +114,7 @@ Example: To run the web tests with a debug build of `content_shell`, but only
test the SVG tests and run pixel tests, you would run:
```bash
-[python] third_party/blink/tools/run_web_tests.py -t Default svg
+third_party/blink/tools/run_web_tests.py -t Default svg
```
***
@@ -143,7 +146,7 @@ for more details of running `content_shell`.
To see a complete list of arguments supported, run:
```bash
-python third_party/blink/tools/run_web_tests.py --help
+third_party/blink/tools/run_web_tests.py --help
```
*** note
@@ -217,7 +220,7 @@ There are two ways to run web tests with additional command-line arguments:
* Using `--additional-driver-flag`:
```bash
- python run_web_tests.py --additional-driver-flag=--blocking-repaint
+ third_party/blink/tools/run_web_tests.py --additional-driver-flag=--blocking-repaint
```
This tells the test harness to pass `--blocking-repaint` to the
@@ -382,10 +385,10 @@ tips for finding the problem.
spacing or box sizes are often unimportant, especially around fonts and
form controls. Differences in wording of JS error messages are also
usually acceptable.
- * `python run_web_tests.py path/to/your/test.html` produces a page listing
- all test results. Those which fail their expectations will include links
- to the expected result, actual result, and diff. These results are saved
- to `$root_build_dir/layout-test-results`.
+ * `third_party/blink/tools/run_web_tests.py path/to/your/test.html` produces
+ a page listing all test results. Those which fail their expectations will
+ include links to the expected result, actual result, and diff. These
+ results are saved to `$root_build_dir/layout-test-results`.
* Alternatively the `--results-directory=path/for/output/` option allows
you to specify an alternative directory for the output to be saved to.
* If you're still sure it's correct, rebaseline the test (see below).
@@ -428,8 +431,7 @@ tips for finding the problem.
To run the server manually to reproduce/debug a failure:
```bash
-cd src/third_party/blink/tools
-python run_blink_httpd.py
+third_party/blink/tools/run_blink_httpd.py
```
The web tests are served from `http://127.0.0.1:8000/`. For example, to
@@ -473,18 +475,12 @@ machine?
### Debugging DevTools Tests
-* Add `debug_devtools=true` to `args.gn` and compile: `autoninja -C out/Default devtools_frontend_resources`
- > Debug DevTools lets you avoid having to recompile after every change to the DevTools front-end.
* Do one of the following:
* Option A) Run from the `chromium/src` folder:
- `third_party/blink/tools/run_web_tests.sh
- --additional-driver-flag='--debug-devtools'
- --additional-driver-flag='--remote-debugging-port=9222'
- --time-out-ms=6000000`
+ `third_party/blink/tools/run_web_tests.py --additional-driver-flag='--remote-debugging-port=9222' --additional-driver-flag='--debug-devtools' --time-out-ms=6000000`
* Option B) If you need to debug an http/tests/inspector test, start httpd
as described above. Then, run content_shell:
- `out/Default/content_shell --debug-devtools --remote-debugging-port=9222 --run-web-tests
- http://127.0.0.1:8000/path/to/test.html`
+ `out/Default/content_shell --remote-debugging-port=9222 --additional-driver-flag='--debug-devtools' --run-web-tests http://127.0.0.1:8000/path/to/test.html`
* Open `http://localhost:9222` in a stable/beta/canary Chrome, click the single
link to open the devtools with the test loaded.
* In the loaded devtools, set any required breakpoints and execute `test()` in
@@ -546,8 +542,7 @@ read on.
***
```bash
-cd src/third_party/blink
-python tools/run_web_tests.py --reset-results foo/bar/test.html
+third_party/blink/tools/run_web_tests.py --reset-results foo/bar/test.html
```
If there are current expectation files for `web_tests/foo/bar/test.html`,
@@ -569,12 +564,18 @@ Though we prefer the Rebaseline Tool to local rebaselining, the Rebaseline Tool
doesn't support rebaselining flag-specific expectations.
```bash
-cd src/third_party/blink
-python tools/run_web_tests.py --additional-driver-flag=--enable-flag --reset-results foo/bar/test.html
+third_party/blink/tools/run_web_tests.py --additional-driver-flag=--enable-flag --reset-results foo/bar/test.html
```
+*** promo
+You can use `--flag-specific=config` as a shorthand of
+`--additional-driver-flag=--enable-flag` if `config` is defined in
+`web_tests/FlagSpecificConfig`.
+***
New baselines will be created in the flag-specific baselines directory, e.g.
-`web_tests/flag-specific/enable-flag/foo/bar/test-expected.{txt,png}`.
+`web_tests/flag-specific/enable-flag/foo/bar/test-expected.{txt,png}`
+or
+`web_tests/flag-specific/config/foo/bar/test-expected.{txt,png}`
Then you can commit the new baselines and upload the patch for review.