zmake: Implement zmake test v2

As described in go/zmake-cli-v2

Change zmake test to call zmake configure --test, unless the
--no-rebuild flag is used, in which case just run tests on existing
binaries. It is an error to use --no-rebuild, if you never did a zmake
build.

BRANCH=None
BUG=b:217788621
TEST=make clobber
zmake -j8 build herobrine test-drivers
zmake -j8 test --no-rebuild herobrine
zmake -j8 test --no-rebuild herobrine test-drivers
zmake -j8 test --no-rebuild herobrine test-drivers test-tasks
zmake -j8 build test-tasks
zmake -j8 test --no-rebuild herobrine test-drivers test-tasks
make clobber
zmake -j8 build --coverage herobrine test-drivers
zmake -j8 test --coverage --no-rebuild herobrine
zmake -j8 test --coverage --no-rebuild herobrine test-drivers
zmake -j8 test --coverage --no-rebuild herobrine test-drivers test-tasks
zmake -j8 build --coverage test-tasks
zmake -j8 test --coverage --no-rebuild herobrine test-drivers test-tasks
zmake -j8 test --coverage herobrine brya
zmake -j8 test --coverage --no-rebuild herobrine brya
zmake -j8 testall
zmake -j8 test --all
zmake -j8 test --all --no-rebuild
zmake -j8 test --host-tests-only
zmake -j8 test --host-tests-only --no-rebuild
zmake -j8 test --host-tests-only --coverage
zmake -j8 test --host-tests-only --coverage

Signed-off-by: Jeremy Bettis <jbettis@google.com>
Change-Id: I90f23cd0061d97e42359e1d49f2331b71554e3a1
Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3472683
Reviewed-by: Jack Rosenthal <jrosenth@chromium.org>
Tested-by: Jeremy Bettis <jbettis@chromium.org>
Commit-Queue: Jeremy Bettis <jbettis@chromium.org>

3 files changed, 21 insertions, 13 deletions

diff --git a/docs/code_coverage.md b/docs/code_coverage.md
index 93637ac019..a15c61b358 100644
--- a/docs/code_coverage.md
+++ b/docs/code_coverage.md
@@ -38,20 +38,28 @@ appear to be caused in part by using relative paths instead of absolute paths.)
 
 To build the Zephyr unit tests for code coverage run:
 
-`zmake coverage build/ztest-coverage`
+`zmake test --host-tests-only --coverage`
+`genhtml -q -o build/zephyr/coverage_rpt/ build/zephyr/all_tests.info`
 
-This target will compile, without linking, all zephyr projects with
-`CONFIG_COVERAGE` Kconfig option enabled, run the tests, and then process the
-profiling data into a code coverage report using the `lcov` and `genhtml`
-tools. This requires the `HAS_COVERAGE_SUPPORT` option, which can only be
-selected in `Kconfig.board`.
+The coverage report top-level page is
+`build/zephyr/coverage_rpt/index.html`.
+
+However you probably want to merge that with a single board's coverage report
+also, so that you can include code that is not part of any test as well.
+
+```
+zmake build --coverage herobrine
+zmake test --host-tests-only --coverage
+genhtml -q -s --branch-coverage -o build/zephyr/coverage_rpt/ \
+  build/zephyr/all_tests.info build/zephyr/herobrine/output/zephyr.info
+```
 
 The coverage report top-level page is
-`build/ztest-coverage/coverage_rpt/index.html`.
+`build/zephyr/coverage_rpt/index.html`.
 
-For manual coverage report you can run:
-`zmake configure --test --coverage <TESTNAME>`
+For coverage report for a single test you can run:
+`zmake test --coverage <TESTNAME>`
 
 Example:
-`zmake configure --test --coverage test-drivers`
-`genhtml -q -o build/ztest-coverage/coverage_rpt/ build/zephyr/test-drivers/output/zephyr.info`
+`zmake test --coverage test-drivers`
+`genhtml -q -o build/zephyr/test-drivers/output/coverage_rpt/ build/zephyr/test-drivers/output/zephyr.info`
diff --git a/docs/zephyr/zephyr_shim.md b/docs/zephyr/zephyr_shim.md
index fac0383aff..3c6b48f209 100644
--- a/docs/zephyr/zephyr_shim.md
+++ b/docs/zephyr/zephyr_shim.md
@@ -347,7 +347,7 @@ Follow these steps:
 Unit tests, implemented using the Ztest framework, can be found in
 `zephyr/test`.
 
-To run all unit tests, you use `zmake testall`.
+To build all projects and run all unit tests, you use `zmake test --all`.
 
 ## Alternatives Considered
 
diff --git a/docs/zephyr/ztest.md b/docs/zephyr/ztest.md
index 4a33d0a103..6bb121045d 100644
--- a/docs/zephyr/ztest.md
+++ b/docs/zephyr/ztest.md
@@ -178,7 +178,7 @@ uses.
 Use `zmake` to build and run the test:
 
 ```
-(cr) $ zmake -l DEBUG configure --test -B build/ztest/base32 test-base32
+(cr) $ zmake -l DEBUG test test-base32
 ...
 UART_0 connected to pseudotty: /dev/pts/1
 *** Booting Zephyr OS build zephyr-v2.4.0-1-g63b2330a85cd  ***