summaryrefslogtreecommitdiff
path: root/zephyr/firmware_builder.py
Commit message (Collapse)AuthorAgeFilesLines
* cq: Exclude zephyr/main/arch/posix from coverageJeremy Bettis2022-11-141-0/+1
| | | | | | | | | | | | | | | | | | | | | The code in zephyr/main/arch/posix is essentially test code, since the posix arch is only used for tests. There have been intermittent coverage changes in zephyr/main/arch/posix/posix_core.c BRANCH=None BUG=None TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Ib52ad05f81f709f0d95b840095fd2e12798dc9f9 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4020817 Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Aaron Massey <aaronmassey@google.com> Commit-Queue: Aaron Massey <aaronmassey@google.com> Tested-by: Jeremy Bettis <jbettis@chromium.org> Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com>
* Revert "cq: Run verbose make for better debugging"Jeremy Bettis2022-11-091-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 23fd725b83abdcb50eb99f69e6a4b102489ba3f7. Reason for revert: The build output for EC is 109Mb. Too big. Original change's description: > cq: Run verbose make for better debugging > > There have been several mysterious failures in the CQ and it's hard to > see what is going on since make doesn't print the commands by default. > > BRANCH=None > BUG=b:257393779 > TEST=CQ > > Signed-off-by: Jeremy Bettis <jbettis@google.com> > Change-Id: I4e4db7ef328b01f52d820405db036ef881d06405 > Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4005654 > Commit-Queue: Jeremy Bettis <jbettis@chromium.org> > Tested-by: Jeremy Bettis <jbettis@chromium.org> > Auto-Submit: Jeremy Bettis <jbettis@chromium.org> > Reviewed-by: Al Semjonovs <asemjonovs@google.com> > Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com> > Commit-Queue: Al Semjonovs <asemjonovs@google.com> Bug: b:257393779 Change-Id: I9fbde348c12ec77b2447a264d7190c3d75423958 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4014811 Reviewed-by: Al Semjonovs <asemjonovs@google.com> Commit-Queue: Al Semjonovs <asemjonovs@google.com> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com>
* ec: Track coverage for rex.Jeremy Bettis2022-11-091-1/+1
| | | | | | | | | | | | | | BRANCH=None BUG=None TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I67a103199dbb07178c3f263e44eb6aeb7eb2af16 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4012963 Tested-by: Jeremy Bettis <jbettis@chromium.org> Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com> Reviewed-by: Abe Levkoy <alevkoy@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org>
* twister: Implement --gcc and --clang optionsJeremy Bettis2022-11-081-22/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In the CQ we call twister with quite a lot of flags, and those could be in the twister wrapper instead. Add twister arg --toolchain=xxx (with aliases --gcc, --llvm, --clang) which will set the gcov-tool, ZEPHYR_TOOLCHAIN_VARIANT env var, cmake vars CMAKE_C_COMPILER and CMAKE_CXX_COMPILER. Remove now surplus args from firmware_builder.py. BRANCH=None BUG=None TEST=./twister --clobber --gcc --coverage TEST=./twister --clobber --toolchain=host --coverage TEST=./twister --clobber --toolchain=llvm --coverage TEST=./twister --clobber --clang --coverage TEST=./twister --clobber --llvm --coverage Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Ida16653bbe24891fd1114d71cdd3445b4146757d Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4010583 Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com> Reviewed-by: Tristan Honscheid <honscheid@google.com>
* zephyr: rename projects folder to programKeith Short2022-11-041-1/+1
| | | | | | | | | | | | | | | | Renme the projects folder to program for consistency with the name scheme used by the boxster configuration. BUG=b:254097139 BRANCH=none TEST=zmake compare-builds -a TEST=twister Signed-off-by: Keith Short <keithshort@chromium.org> Change-Id: Ib56a57f1e5942e6dd0460e3be81722896eed72af Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3968444 Reviewed-by: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org>
* cq: Run tests with both gcc and clang.Jeremy Bettis2022-11-041-11/+68
| | | | | | | | | | | | | | | | | | | | To ensure compatibility with gitlab, run all the zephyr tests with both gcc and llvm/clang. This adds 6 minutes to the build time on my machine but adds 0.4% coverage overall, and 1.7% to herobrine, since it fixes all the uncovered switch case labels. BRANCH=None BUG=None TEST=Ran firmware_builder.py locally Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I5a7c1f2e8f226e18d642eb886fd90c778c7e77d2 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4004761 Reviewed-by: Al Semjonovs <asemjonovs@google.com> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com> Reviewed-by: Tristan Honscheid <honscheid@google.com>
* cq: Run verbose make for better debuggingJeremy Bettis2022-11-041-2/+2
| | | | | | | | | | | | | | | | | | | There have been several mysterious failures in the CQ and it's hard to see what is going on since make doesn't print the commands by default. BRANCH=None BUG=b:257393779 TEST=CQ Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I4e4db7ef328b01f52d820405db036ef881d06405 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/4005654 Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Al Semjonovs <asemjonovs@google.com> Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com> Commit-Queue: Al Semjonovs <asemjonovs@google.com>
* cq: Make sure build is cleanJeremy Bettis2022-11-021-0/+5
| | | | | | | | | | | | | | | | | | | Run a make clobber before make buildall_only just to be sure that the runner isn't reusing a dirty chroot. BRANCH=None BUG=b:256761895 TEST=CQ. Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I2d8322ff7093b60dff0fac57c318b38cf9b29558 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3995326 Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Al Semjonovs <asemjonovs@google.com> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Al Semjonovs <asemjonovs@google.com> Code-Coverage: Zoss <zoss-cl-coverage@prod.google.com>
* cq: Pass the compiler to twisterJeremy Bettis2022-11-011-0/+2
| | | | | | | | | | | | | | | | | | | | | | On some CQ runners /usr/bin/cc is a symlink to clang instead of gcc. To prevent that from affecting the build, always pass the compiler to twister. The native_posix tests still use the compiler specified by the toolchain cmake files, this only affects unit_testing builds. BRANCH=None BUG=None TEST=Ran the twister cmd. Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I749149d17ad9cd46c0179048209d0067aa4ef2ae Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3997617 Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Aaron Massey <aaronmassey@google.com> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Aaron Massey <aaronmassey@google.com>
* cq: Remove unnecessary lcov commandJeremy Bettis2022-10-181-20/+1
| | | | | | | | | | | | | | | | | | There is a lcov command that is merging 2 files that are already combined. Remove it, and change the dependent steps. BRANCH=None BUG=None TEST=Ran before and after and diffed the metrics Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Ifb78cbec97983198baf00077c3bb945ec31f350f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3964534 Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Abe Levkoy <alevkoy@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Abe Levkoy <alevkoy@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org>
* ec: Replace dir in path with *Jeremy Bettis2022-10-141-1/+1
| | | | | | | | | | | | | | | BRANCH=None BUG=b:253647573 TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I1f91b8123a2e44ea5a137f9644645ca1b4494aaa Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3956204 Reviewed-by: Bobby Casey <bobbycasey@google.com> Commit-Queue: Bobby Casey <bobbycasey@google.com> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* cq: Expand coverage exclusionsJeremy Bettis2022-10-141-1/+20
| | | | | | | | | | | | | | | | | | | | | There were some test files that were not getting excluded by the test filter. Most importantly, some tests are mixed into the same directories, but named _test.c. BRANCH=None BUG=None TEST=./firmware_builder.py --code-coverage test lcov --list-full-path --list build/zephyr/lcov_no_tests.info | \ sed -e 's|/mnt/host/source/src/||' | egrep 'mock|emul|test|host' Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I4d8335c4185ba373f93fbfe6bf2ed99aaa7a0ba0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3957418 Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Yuval Peress <peress@google.com> Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Yuval Peress <peress@google.com>
* ec: New tool to merge lcov filesJeremy Bettis2022-10-111-21/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added new tool to merge lcov files using one file as the template and including only lines that are present in the template file. The name is because you use the template file like a "stencil" and the matching coverage shows through. Not great, but it's not really a "set intersect" or a "merge" either. Switch firmware_builder.py and gitlab to use the new tool in place of the filename filtering. See https://jbettis.users.x20web.corp.google.com/www/herobrine_rpt/index.html for updated coverage output. BRANCH=None BUG=None TEST=Ran firmware_builder.py locally. Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I0e9672c346971b0df4602b0adce27fea6367c6b4 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3943261 Reviewed-by: Abe Levkoy <alevkoy@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Abe Levkoy <alevkoy@chromium.org>
* cq: Add coverage metric for kinglerJeremy Bettis2022-10-111-1/+1
| | | | | | | | | | | | | | | | | | Corsola want to track coverage of 2 models, so add kingler to the special boards list. BRANCH=None BUG=None TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I7135c453d2e22cff92939de59871a1af3c1f9ef1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3946718 Tested-by: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jack Rosenthal <jrosenth@chromium.org> Reviewed-by: Jack Rosenthal <jrosenth@chromium.org>
* cq: Add coverage builds for cq and gitlabJeremy Bettis2022-10-051-1/+1
| | | | | | | | | | | | | | | | | | | Add krabby, nivviks, and skyrim specific coverage builds to cq and gitlab. In cq this is just lcov commands, no new builds will actually be run. BRANCH=None BUG=None TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I59120b9a2b7cff7fbe5f7d24c92a4c877ccd79b8 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3936944 Tested-by: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Simon Glass <sjg@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Simon Glass <sjg@chromium.org>
* cq: Only run twister once in test phaseJeremy Bettis2022-09-281-3/+1
| | | | | | | | | | | | | | | | | | | | | Instead of running twister --build-only in the build phase and twister --test-only in the test phase, just run a normal clean twister build once in the test phase. I don't know that it happens in the CQ, but when running firmware_builder.py locally, the --test-only step is often failing. BRANCH=None BUG=None TEST=Ran locally. Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I43cf9f5c360cb7de43fa6ae59128948f15230092 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3922446 Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Tristan Honscheid <honscheid@google.com> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Tristan Honscheid <honscheid@google.com>
* zephyr: upload test results for local runsAl Semjonovs2022-09-151-0/+1
| | | | | | | | | | | | | | | Upload test results for local twister runs BUG=None BRANCH=NONE TEST=./twister -T zephyr/test --no-upload-cros-rdb; ./twister -T zephyr/test Signed-off-by: Al Semjonovs <asemjonovs@google.com> Change-Id: I1ce87ab44fcfcf5ad96f3eb7c6f097bc36643985 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3891682 Commit-Queue: Yuval Peress <peress@google.com> Reviewed-by: Yuval Peress <peress@google.com>
* Update license boilerplate text in source code filesMike Frysinger2022-09-121-1/+1
| | | | | | | | | | | | | | | Normally we don't do this, but enough changes have accumulated that we're doing a tree-wide one-off update of the name & style. BRANCH=none BUG=chromium:1098010 TEST=`repo upload` works Change-Id: Icd3a1723c20595356af83d190b2c6a9078b3013b Signed-off-by: Mike Frysinger <vapier@chromium.org> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3891203 Reviewed-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Jack Rosenthal <jrosenth@chromium.org>
* zephyr: Also exclude mock dirJeremy Bettis2022-08-291-0/+1
| | | | | | | | | | | | | | | | Mocks are basically emulators. BRANCH=None BUG=None TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I4049cd90b29cfd3b00adb84df7bcf2a3a998d957 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3860367 Commit-Queue: Al Semjonovs <asemjonovs@google.com> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Al Semjonovs <asemjonovs@google.com>
* cq: Filter out tests in mcutests alsoJeremy Bettis2022-08-261-52/+60
| | | | | | | | | | | | | | | | | | | | | Move command logging to a function, and flush after each print. Move lcov exclude file patterns to variables for reuse. Add ec/private/fingerprint/google-fpalg/mcutest/** to exclude list. The mcutest dir is all test code, so it has 100% coverage, but isn't real coverage. BRANCH=None BUG=b:242334829,b:231639771 TEST=ran locally Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I88b72097344b512afa60c92f6e0d64c976fb2900 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3858391 Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Tristan Honscheid <honscheid@google.com> Commit-Queue: Tristan Honscheid <honscheid@google.com> Auto-Submit: Jeremy Bettis <jbettis@chromium.org>
* cq: Save the html report for herobrine coverageJeremy Bettis2022-08-241-1/+20
| | | | | | | | | | | | | | | | | In addition to the all boards html coverage report, also include the herobrine html in the build artifacts. BRANCH=None BUG=None TEST=Did build, looked in artifacts dir Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Ida81797963cb1db700f7de5b4fb43acdcca490a4 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3854555 Reviewed-by: Yuval Peress <peress@google.com> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Yuval Peress <peress@google.com> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* zephyr: Fix filtering of zephyr subsysJeremy Bettis2022-08-191-2/+2
| | | | | | | | | | | | | | | | | The coverage data is incorrect, because it is including lines covered in ztest and emulators. Fix the exclude pattern. BRANCH=None BUG=b:231639771 TEST=Ran cmd locally Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I0d72e5226645d2b7df9e3e2ad209bcb80df9fcf4 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3842205 Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Aaron Massey <aaronmassey@google.com> Reviewed-by: Aaron Massey <aaronmassey@google.com> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* cq: Add new metric for coverage w/o testsJeremy Bettis2022-08-161-1/+28
| | | | | | | | | | | | | | | | | | | | It isn't very useful to compare cl coverage if you include test code as covered code. Add a new metric that excludes tests, but still upload the lcov info file with tests to gerrit, so that the lines of the test that actually run show in the UI. BRANCH=None BUG=None TEST=see README.md Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: If7ee53ae7e95bcfa7542bc0675c24a5d4d432747 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3833918 Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Simon Glass <sjg@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Simon Glass <sjg@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* zephyr: Run the upstream shell tests alsoJeremy Bettis2022-08-161-2/+4
| | | | | | | | | | | | | | | | | | | When running twister, run both the EC tests and the zephyr shell tests by default. This should provide for more consistent coverage of the shell code. BRANCH=None BUG=b:242108457 TEST=./twister Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I25078ee0a375de18afaab6a3bfc1ffc8b47b6b3a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3829060 Reviewed-by: Tristan Honscheid <honscheid@google.com> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Tristan Honscheid <honscheid@google.com> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* zephyr: firmware_builder: Log twister test failsAaron Massey2022-08-141-1/+7
| | | | | | | | | | | | | | | | | | The zephyr/firmware_builder.py did not write the entire test log when twister ran a failing test. When a test failed, there was no way to find the specific test that failed from the logs. Add verbose and inline log parameters to twister invocations in the zephyr/firmware_builder.py CI script. BRANCH=none BUG=b:242058113 TEST=Run zephyr/firmware_builder.py locally Signed-off-by: Aaron Massey <aaronmassey@google.com> Change-Id: I1931d84987091b87c20313614b26b19e31e25d04 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3824924 Reviewed-by: Tristan Honscheid <honscheid@google.com>
* cq: Use /dev/null as stdin in buildsJeremy Bettis2022-08-121-7/+25
| | | | | | | | | | | | | | | | | | | There is some inconsistency between the CQ build and local builds, and one cause is that in the CQ there is no stdin when the tests are running. Close stdin explicitly so that it is always consistent. BRANCH=None BUG=b:242334829 TEST=Ran build locally Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I1c95c85c62ceaec8d1bf15d981488add75adffb2 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3830146 Reviewed-by: Tristan Honscheid <honscheid@google.com> Commit-Queue: Tristan Honscheid <honscheid@google.com> Tested-by: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org>
* zephyr: Add herobrine only coverage metricJeremy Bettis2022-08-081-24/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In the metrics created by the CQ builder, include a coverage number for herobrine as BOARD_HEROBRINE. Other boards can be added easily. Print out most all sub-commands before running them. Run make test-coverage instead of make coverage for more compatibility with gitlab, and expose that as the metric EC_LEGACY_TESTS. Expose zephyr tests without boards as EC_ZEPHYR_TESTS. Expose new metric ALL_TESTS, that is just tests, no boards. Remove metric EC_LEGACY_MERGED. BRANCH=None BUG=b:231639771 TEST=See README.md for commands. Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Iad7e6f1b2adf8d41cff1e8a41199d41f386f78d5 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3818127 Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Abe Levkoy <alevkoy@chromium.org> Commit-Queue: Abe Levkoy <alevkoy@chromium.org>
* zephyr: tests: remove BUILD.py for Zephyr EC unit testsTristan Honscheid2022-08-051-8/+5
| | | | | | | | | | | | | | | | | | | | | | | We have completed transitioning tests to Twister and verified parity in coverage reports between this and the previous version. This CL removes the BUILD.py files from our tests that declare the zmake projects and removes the call to `zmake -D test -a --no-rebuild` from `zephyr/firmware-builder.py` The `EC_ZEPHYR_MERGED` metric is now derived from Twister's coverage data instead. Made corresponding changes to `.gitlab-ci.yml` This CL was tested by comparing the total coverage and list of covered files before and after the change. BUG=b:240093007 BRANCH=None TEST=zephyr/firmware-builder.py --code-coverage {build|test|bundle} Signed-off-by: Tristan Honscheid <honscheid@google.com> Change-Id: I865a22a44f56760790444499c7588f7aec299374 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3811956 Reviewed-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Yuval Peress <peress@google.com>
* cq: Upload coverage html report as dir, not tarJeremy Bettis2022-07-291-4/+3
| | | | | | | | | | | | | | | | | | Instead of uploading a coverage html tar file, upload a directory. This should make the report more viewable. BRANCH=None BUG=b:231639771 TEST=Check results in CQ dry-run Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I7ec57e6e2ed705aa8e1404fd53b9c57a260353c3 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3790240 Commit-Queue: Al Semjonovs <asemjonovs@google.com> Tested-by: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Al Semjonovs <asemjonovs@google.com>
* zephyr: twister: Specify llvm-cov tool in twister wrapper scriptTristan Honscheid2022-07-281-2/+0
| | | | | | | | | | | | | | | This is part of our toolchain and tedious to write out every time, so include it in the default args passed by the wrapper script BRANCH=None BUG=b:240577313 TEST=./twister --coverage -p native_posix Signed-off-by: Tristan Honscheid <honscheid@google.com> Change-Id: I451245b6df8048984d93aa55b42667cdb7309d90 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3792505 Reviewed-by: Yuval Peress <peress@google.com> Reviewed-by: Jeremy Bettis <jbettis@chromium.org>
* cq: Don't return 0 for success, we have raiseJeremy Bettis2022-07-271-7/+3
| | | | | | | | | | | | | | | | | No reason to check the return code on subprocess if check=True, so remove those. Also the top level functions never return non-zero codes so remove those return statements. BRANCH=None BUG=b:231639771 TEST=Ran firmware_builder.py locally Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Ie25a5ea1d63fbe9220d48735e561f070eccd4682 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3789839 Reviewed-by: Tristan Honscheid <honscheid@google.com> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* cq: Set explicit cwd on all subprocessesJeremy Bettis2022-07-271-20/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To make sure all paths in coverage files are consistent, set explicit working dirs on all subprocess commands. Exclude twister-out and also /usr/include from coverage. BRANCH=None BUG=b:231639771 TEST=rm -rf /tmp/artifact_bundles-cov \ /tmp/artifact_bundle_metadata-cov \ /mnt/host/source/src/platform/ec/build && \ cd ~/chromiumos/src/platform/ec/zephyr && \ ./firmware_builder.py --metrics /tmp/metrics \ --code-coverage build && \ ./firmware_builder.py --metrics /tmp/metrics \ --code-coverage test && \ ./firmware_builder.py --metrics /tmp/metrics \ --code-coverage --output-dir=/tmp/artifact_bundles-cov \ --metadata=/tmp/artifact_bundle_metadata-cov bundle && \ echo PASSED && \ grep SF: ../build/zephyr/lcov.info | sort -u | \ grep -v SF:/mnt/host/source/src Change-Id: Ic63e51039f182c773fc52e48f86fd70c49f47fc3 Signed-off-by: Jeremy Bettis <jbettis@google.com> Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3789833 Tested-by: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Tristan Honscheid <honscheid@google.com>
* ec: Switch black to 80 cols and reformat filesJeremy Bettis2022-07-261-6/+20
| | | | | | | | | | | | | | | | | | | | Add pyproject.toml config file to set black to 80 columns. Remove column length overrides from other config files. Reformat python files to 80 cols. BRANCH=None BUG=b:238434058 TEST=presubmit/CQ Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I870a68f1bb751f4bad97024045f6e3075489e80f Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3764071 Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Jack Rosenthal <jrosenth@chromium.org>
* zephyr: Switch Twister tests to LLVMTristan Honscheid2022-07-261-6/+16
| | | | | | | | | | | | | | | | | | | | | | Switch back to using the LLVM toolchain and use an existing wrapper script for calling `llvm-cov gcov`. The wrapper script is necessary due to `llvm-cov` requiring `gcov` as a subcommand and there is no way to pass that in to Twister. Also made two minor tweaks to `run_twister`: 1. Turn extra_args in to an optional list param so multiple extra args can be specified 2. Replace opt with a dedicated param for enabling code coverage BRANCH=None BUG=b:240093007 TEST=./twister; zephyr/firmware_builder.py Signed-off-by: Tristan Honscheid <honscheid@google.com> Change-Id: I9a4aa085eb71de1688d55b14b432b6c459d1a119 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3782907 Reviewed-by: Yuval Peress <peress@google.com> Commit-Queue: Yuval Peress <peress@google.com>
* zephyr: Split twister build and testAl Semjonovs2022-07-251-8/+18
| | | | | | | | | | | | | | Perform build only and test only twister commands per build phase. BUG=None BRANCH=None TEST=./twister Signed-off-by: Al Semjonovs <asemjonovs@google.com> Change-Id: I1b68d15eea7404d050c7e8cdcdbaaa4e72192c40 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3783230 Reviewed-by: Yuval Peress <peress@google.com>
* zephyr: Add wrapper script for calling twisterTristan Honscheid2022-07-221-1/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a wrapper script that helps invoke twister with the correct configuration and defaults for running EC tests. Developers can simply run `./twister` from `src/platform/ec` to run all Twister-compatible EC tests, or specify specific tests to run by adding `-T path/to/testroot` arguments. Other CLI args are passed straight through to Twister. This CL also adds a testcase.yaml file to the `math` test, allowing it to be run through Twister: ``` cd $CROS_WORKON_SRCROOT/src/platform/ec ./twister (or) ./twister -T zephyr/test/math ``` Twister's output directories, `twister-out` and `twister-out.N` have been added to the `.gitignore`. The `firmware_builder.py` CQ script now also calls Twister and can merge in coverage reports. BRANCH=None BUG=None TEST=./twister Signed-off-by: Tristan Honscheid <honscheid@google.com> Change-Id: If6ff10ffb76655912c96a66aec28d12dcebd1764 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3773766 Reviewed-by: Yuval Peress <peress@google.com>
* cq: Exclude build dir from coverageJeremy Bettis2022-07-191-1/+17
| | | | | | | | | | | | | | | | | | | The build dir contains a number of generated files, that creates some inconsistent results in the coverage reports. Because of this, and also because those files are generated and don't matter, exclude them from the coverage report. BRANCH=None BUG=b:213374060 TEST=Ran commands in README.md Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Ie1da42c3cc14faa917d2f0deca6c7acc9a9b7751 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3773934 Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Keith Short <keithshort@chromium.org> Commit-Queue: Keith Short <keithshort@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org>
* ec: Enforce black in presubmitJeremy Bettis2022-07-131-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | Enforce black, isort formatting in presubmit for all python files. Do not enforce flake8, because it has errors on most of the files, and it seems to be basically the same as pylint, which is already run on pre-submit. Do not enforce it in the zmake unit test script. Do enforce the formatting in the firmware_builder.py script, and do it early for faster failures in CQ runs. BRANCH=None BUG=b:238434058 TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: If3c42b2af41fd2e68accbe2867999dc931e88872 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3749243 Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Jack Rosenthal <jrosenth@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org>
* cq: Upload code coverage html report as artifactJeremy Bettis2022-07-121-0/+23
| | | | | | | | | | | | | | | | | Run genhtml to create a code coverage report and upload it as an artifact. BRANCH=None BUG=b:231639771 TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I2de2f9ac8fcd93227fe3ab55ebda301f087d02db Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3755964 Reviewed-by: Jack Rosenthal <jrosenth@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Yuval Peress <peress@google.com>
* ec: Format all python files with black and isortJeremy Bettis2022-07-121-101/+108
| | | | | | | | | | | | | | | | | | | find . \( -path ./private -prune \) -o -name '*.py' -print | xargs black find . \( -path ./private -prune \) -o -name '*.py' -print | xargs ~/chromiumos/chromite/scripts/isort --settings-file=.isort.cfg BRANCH=None BUG=b:238434058 TEST=None Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I63462d6f15d1eaf3db84eb20d1404ee976be8382 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3749242 Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Tom Hughes <tomhughes@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jack Rosenthal <jrosenth@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Jack Rosenthal <jrosenth@chromium.org>
* firmware_builder: Enforce clang-format in CQJack Rosenthal2022-07-081-2/+4
| | | | | | | | | | | | | | | | Add a new helper script to run clang-format on all files, and call it in the firmware builder scripts. BUG=b:236386294 BRANCH=none TEST=Ran util/check_clang_format.py ~> On ToT: no errors reported ~> Intentionally mess up a file: errors reported Signed-off-by: Jack Rosenthal <jrosenth@chromium.org> Change-Id: Ibde2a4993527fd6f8f30dc25378fdd5f48416d7a Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3751463 Reviewed-by: Jeremy Bettis <jbettis@chromium.org>
* cq: Merge in the legacy tests in the zephyr covJeremy Bettis2022-07-071-1/+27
| | | | | | | | | | | | | | | | Just like in gitlab, merge in the legacy coverage data, and put that into the ALL_MERGED metric. BRANCH=None BUG=b:231639771 TEST=Ran locally. Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I05128b0c07160ab6d5f3937216dd1797b2403f39 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3748852 Tested-by: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Jack Rosenthal <jrosenth@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org>
* zephyr: Don't convert paths to relative in infoJeremy Bettis2022-06-071-13/+1
| | | | | | | | | | | | | | | | | The CQ recipe will now convert absolute paths to relative paths as needed, so leave the paths as absolute in the lcov file. BRANCH=None BUG=b:231639771 TEST=Ran script by hand Cq-Depend: chromium:3683588 Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I1461d935ae42ad09d3617ea506911cf5b5acbb00 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3684370 Reviewed-by: Aaron Massey <aaronmassey@google.com> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* zephyr: Export coverage metrics in CQ buildJeremy Bettis2022-05-191-6/+25
| | | | | | | | | | | | | | | | | | When running tests for the zephyr coverage cq build, put coverage data into the FwTestMetricList proto. BRANCH=None BUG=b:231639771 TEST=firmware_builder.py --metrics /tmp/metrics --code-coverage test cat /tmp/metrics Cq-Depend: chromium:3642959 Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I07902138d2cb1b75d08bad49accf8fe2f2049b8e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3642541 Commit-Queue: Tristan Honscheid <honscheid@google.com> Reviewed-by: Tristan Honscheid <honscheid@google.com> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* zephyr: Fix lint errors in firmware_builder.pyJeremy Bettis2022-05-131-10/+13
| | | | | | | | | | | | | | | | Fix all the cros lint errors. BRANCH=None BUG=None TEST=cros lint zephyr/firmware_builder.py firmware_builder.py Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Id00701fc308a0d5530ca36fec4f756f9c08ab92e Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3635076 Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Keith Short <keithshort@chromium.org> Commit-Queue: Keith Short <keithshort@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* ec: Switch to gen_sdk importJeremy Bettis2022-05-101-4/+4
| | | | | | | | | | | | | | | | | Switch from importing chromite.api.gen.chromite.api to chromite.api.gen_sdk.chromite.api, to avoid the odd import order. BRANCH=None BUG=b:187795298 TEST=Ran firmware builders Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I21ba7856679d67bd889a2585b28cdb74ef5493d7 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3635075 Tested-by: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Mike Frysinger <vapier@chromium.org> Commit-Queue: Mike Frysinger <vapier@chromium.org>
* cq: Export binary and ram sizes in cq build metricJeremy Bettis2022-03-171-46/+103
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change firmware_builder.py to export metrics. Here is an example ``` { "targetName": "nereid", "platformName": "it8xxx2", "fwSection": [ { "region": "RO_ROM", "used": 298164, "total": 1048576 }, { "region": "RO_RAM", "used": 51764, "total": 57344 }, { "region": "RO_IDT_LIST", "total": 2048 }, { "region": "RW_ROM", "used": 298164, "total": 1048576 }, { "region": "RW_RAM", "used": 51764, "total": 57344 }, { "region": "RW_IDT_LIST", "total": 2048 } ] }, ``` Also formatted to make cros lint happy. BRANCH=None BUG=b:223846977 TEST=Ran commands in README. Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: Ic1c53d0ce2ba273af714b74f8fc057d9237b5b02 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3529669 Reviewed-by: Jett Rink <jettrink@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org>
* cq: Use relative paths in lcov.info fileJeremy Bettis2022-02-251-1/+15
| | | | | | | | | | | | | | | | | After merging the lcov files, strip any platform/ec prefixes from the filenames. Gerrit's zoss plugin wants relative paths. BRANCH=None BUG=b:156895937 TEST=Ran commands in README.md Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I9eb85362e6ffc32bdc3c8e21979cf847df2dd5a0 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3489263 Tested-by: Jeremy Bettis <jbettis@chromium.org> Auto-Submit: Jeremy Bettis <jbettis@chromium.org> Reviewed-by: Denis Brockus <dbrockus@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org>
* zmake: Implement zmake test v2Jeremy Bettis2022-02-231-34/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As described in go/zmake-cli-v2 Change zmake test to call zmake configure --test, unless the --no-rebuild flag is used, in which case just run tests on existing binaries. It is an error to use --no-rebuild, if you never did a zmake build. BRANCH=None BUG=b:217788621 TEST=make clobber zmake -j8 build herobrine test-drivers zmake -j8 test --no-rebuild herobrine zmake -j8 test --no-rebuild herobrine test-drivers zmake -j8 test --no-rebuild herobrine test-drivers test-tasks zmake -j8 build test-tasks zmake -j8 test --no-rebuild herobrine test-drivers test-tasks make clobber zmake -j8 build --coverage herobrine test-drivers zmake -j8 test --coverage --no-rebuild herobrine zmake -j8 test --coverage --no-rebuild herobrine test-drivers zmake -j8 test --coverage --no-rebuild herobrine test-drivers test-tasks zmake -j8 build --coverage test-tasks zmake -j8 test --coverage --no-rebuild herobrine test-drivers test-tasks zmake -j8 test --coverage herobrine brya zmake -j8 test --coverage --no-rebuild herobrine brya zmake -j8 testall zmake -j8 test --all zmake -j8 test --all --no-rebuild zmake -j8 test --host-tests-only zmake -j8 test --host-tests-only --no-rebuild zmake -j8 test --host-tests-only --coverage zmake -j8 test --host-tests-only --coverage Signed-off-by: Jeremy Bettis <jbettis@google.com> Change-Id: I90f23cd0061d97e42359e1d49f2331b71554e3a1 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3472683 Reviewed-by: Jack Rosenthal <jrosenth@chromium.org> Tested-by: Jeremy Bettis <jbettis@chromium.org> Commit-Queue: Jeremy Bettis <jbettis@chromium.org>
* zephyr: zmake: Remove the hard-coded list of projects in firmware_builder.pyJack Rosenthal2022-02-111-13/+3
| | | | | | | | | | | | | | | | | This is actually just duplicate work of what the test phase is going to end up doing, as testall will want to do the build anyway. For now (that is, until we improve the zmake CLI enough to distinctly separate the phases logically), let's just do nothing here. BUG=b:217788621,b:218868887 BRANCH=none TEST=CQ passes Signed-off-by: Jack Rosenthal <jrosenth@chromium.org> Change-Id: I91cbdf45b80a4eec1ead03a8342790c476b36585 Reviewed-on: https://chromium-review.googlesource.com/c/chromiumos/platform/ec/+/3452933 Reviewed-by: Jeremy Bettis <jbettis@chromium.org>
View Lorry Controller Status