diff options
Diffstat (limited to 'chromium/docs/website/site/chromium-os/build')
15 files changed, 0 insertions, 1135 deletions
diff --git a/chromium/docs/website/site/chromium-os/build/OWNERS b/chromium/docs/website/site/chromium-os/build/OWNERS deleted file mode 100644 index 995488cce5e..00000000000 --- a/chromium/docs/website/site/chromium-os/build/OWNERS +++ /dev/null @@ -1 +0,0 @@ -include chromiumos/chromite:/OWNERS.build diff --git a/chromium/docs/website/site/chromium-os/build/add-sdk-package/index.md b/chromium/docs/website/site/chromium-os/build/add-sdk-package/index.md deleted file mode 100644 index 39781ae2e09..00000000000 --- a/chromium/docs/website/site/chromium-os/build/add-sdk-package/index.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: add-sdk-package -title: Adding a Package to the SDK ---- - -[TOC] - -## Add the Package - -When adding a package to the SDK, for third-party packages not yet in the source -tree, start by following the -[New & Upgrade Package Process](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/portage/package_upgrade_process.md) -guide (short version: use `cros_portage_upgrade` to pull the package from -upstream). For new cros-workon packages, see -[Adding a New Package](/chromium-os/how-tos-and-troubleshooting/add-a-new-package). -Once the package is in place, add it as a dependency to the -[virtual/target-chromium-os-sdk](https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/HEAD/virtual/target-chromium-os-sdk/target-chromium-os-sdk-9999.ebuild) -ebuild to be automatically installed as part of the SDK. - -Do note that because virtual/target-chromium-os-sdk is a cros-workon package, -before adding the dependency, run: - -``` -cros-workon --host start virtual/target-chromium-os-sdk -``` - -## Test the Package - -To install the new package to test against (e.g. to run commands/tools, check -behavior, etc.): - -``` -sudo emerge <package name> -``` - -To test the new package is installed correctly as part of the SDK, update the -chroot: - -``` -update_chroot -``` - -### Unit testing - -To run the unit tests for the new package, run: - -``` -cros_run_unit_tests --host --packages <package name> -``` - -For cros-workon packages, as long as the package is in the -virtual/target-chromium-os-sdk dependency tree, the unit tests will -automatically be run on the SDK as part of the host-packages-cq builder in CQ -and are required to pass. For third-party packages, unit tests can be run with -the `cros_run_unit_tests` command above to get feedback locally, but they are -not run as part of CQ and are not required to pass as it is expected that -upstream releases have already been tested. - -### Test with the SDK Builder - -For more information on the SDK Builder, see -[Chromium OS SDK Creation](/chromium-os/build/sdk-creation). - -To test building the SDK itself with the new package, by running the entire SDK -builder process, launch an SDK builder tryjob: - -``` -cros tryjob -g <cl 1> [-g ...] chromiumos-sdk-tryjob -``` - -Or, to just build the SDK board locally: - -``` -~/chromiumos/src/scripts/build_sdk_board -``` diff --git a/chromium/docs/website/site/chromium-os/build/bypassing-tests-on-a-per-project-basis/index.md b/chromium/docs/website/site/chromium-os/build/bypassing-tests-on-a-per-project-basis/index.md deleted file mode 100644 index 6cebd4125f3..00000000000 --- a/chromium/docs/website/site/chromium-os/build/bypassing-tests-on-a-per-project-basis/index.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: bypassing-tests-on-a-per-project-basis -title: Per-repo and per-directory configuration of CQ and pre-CQ ---- - -Different chromeos repositories have different testing needs. Using per-repo or -per-directory configuration, it is possible to tailor the behavior of the -[Chromeos Commit -Queue](http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium-os/commit-queue-overview) -to suit the particular change being tested. - -This documentation needs updating for the new Parallel CQ infrastructure. The -older COMMIT-QUEUE.ini files are no longer read by anything. Contact -chromeos-continuous-integration-team (crbug components ChromeOS>Infra>CI) -with any questions.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/build/c-exception-support/index.md b/chromium/docs/website/site/chromium-os/build/c-exception-support/index.md deleted file mode 100644 index 2ddbb3aad67..00000000000 --- a/chromium/docs/website/site/chromium-os/build/c-exception-support/index.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: c-exception-support -title: C++ exception support ---- - -C++ exceptions are disabled by default for all C and C++ code. If you need them, -you can allow-list your ebuild. - -**Background** - -C++ exceptions are not allowed by the [Google C++ Style -Guide](http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml) and only -a few projects use them. This supports adds a some overhead to the binary file -on the .eh_frame and .eh_frame_hdr sections, even if you don't use -try/catch/throw, even if your program is in C. Nevertheless, they are enabled by -default by the compiler, since some exception support is required even if you -don't use them. For example, if function f() calls g() calls h(), and h() throws -an exception that f() is supposed to catch, then g() also needs to be compiled -with exception support, even if g() is a C function. - -The overhead is in the range of 5% to 10% of the binary size, depending of the -average size of your functions, so we disable it by default. - -**What to allow-list** - -If your application uses try, catch or throw, it needs to be allow-listed -because otherwise it won't compile. - -If your application calls a library that might throw an exception, but you don't -catch it, the application doesn't need to be compiled with exceptions support. -If the library throws an exception, the application will be terminated in either -case. - -If your application catches an exception that is thrown by a function called by -an intermediate library, the library also needs to be compiled with exceptions -support. For example, if you pass a callback to a library that might throw an -exception, then the library should be also compiled with exception support. -That's why glib and glibc are allow-listed. - -**Implementation** - -To disable the exception support, we pass "-fno-exceptions -fno-unwind-tables --fno-asynchronous-unwind-tables" in CFLAGS and CXXFLAGS by default. To -allow-list them, we filter these flags from CFLAGS and CXXFLAGS. We also define -the environment variable CXXEXCEPTIONS to 0 or 1 to determine whether the -exceptions are enabled. - -The flags are added on the chromiumos-overlay/chromeos/make.conf.\*-target -files. To re-enable this support, you can call the bash function -cros_enable_cxx_exceptions defined in -chromiumos-overlay/profiles/base/profile.bashrc. That function will filter out -the flags from CXXFLAGS and CFLAGS. - -This can be done on a per ebuild basis. For example, the dev-libs/dbus-c++ -ebuild has the following snippet on -chromiumos-overlay/chromeos/config/env/dev-libs/dbus-c++. - -> cros_pre_src_prepare_enable_cxx_exceptions() { - -> cros_enable_cxx_exceptions - -> }
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/build/chromite-parallel/index.md b/chromium/docs/website/site/chromium-os/build/chromite-parallel/index.md deleted file mode 100644 index 34206d174f6..00000000000 --- a/chromium/docs/website/site/chromium-os/build/chromite-parallel/index.md +++ /dev/null @@ -1,156 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: chromite-parallel -title: Writing multiprocess programs in Python ---- - -The -[chromite.lib.parallel](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/lib/parallel.py) -module is the preferred way to write multiprocess python programs in Chrome OS. - -It offers the following advantages over the multiprocessing module: - -* The multiprocessing module can be buggy, so it is difficult to avoid - hangs without substantial trial and error figuring out what features - of multiprocessing actually work. We have already done this for you - in the parallel library. -* The interface for the parallel library is simpler than the - multiprocessing interface and performs tasks that you would need to - perform manually if you were using multiprocessing. -* The parallel library automatically queues up output in temporary - files and prints them out serially. This means that you never need - to worry about interleaved output. -* The parallel library handles cases where the subprocess hangs and - prints out suitable errors. - -Here's the -[chromite.lib.parallel](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/lib/parallel.py) -function for running multiple steps in parallel in a very simple way. The -docstring should help explain how it works. - -def RunParallelSteps(steps, max_parallel=None, halt_on_error=False, - -return_values=False): - -"""Run a list of functions in parallel. - -This function blocks until all steps are completed. - -The output from the functions is saved to a temporary file and printed as if - -they were run in sequence. - -If exceptions occur in the steps, we join together the tracebacks and print - -them after all parallel tasks have finished running. Further, a - -BackgroundFailure is raised with full stack traces of all exceptions. - -Args: - -steps: A list of functions to run. - -max_parallel: The maximum number of simultaneous tasks to run in parallel. - -By default, run all tasks in parallel. - -halt_on_error: After the first exception occurs, halt any running steps, - -and squelch any further output, including any exceptions that might occur. - -return_values: If set to True, RunParallelSteps returns a list containing - -the return values of the steps. Defaults to False. - -Returns: - -If |return_values| is True, the function will return a list containing the - -return values of the steps. - -Example: - -# This snippet will execute in parallel: - -# somefunc() - -# anotherfunc() - -# funcfunc() - -steps = \[somefunc, anotherfunc, funcfunc\] - -RunParallelSteps(steps) - -# Blocks until all calls have completed. - -""" - -If you want to be able to use a worker-based model and push items to the -subprocess one by one, the BackgroundTaskRunner will do the job: - -@contextlib.contextmanager - -def BackgroundTaskRunner(task, \*args, \*\*kwargs): - -"""Run the specified task on each queued input in a pool of processes. - -This context manager starts a set of workers in the background, who each - -wait for input on the specified queue. For each input on the queue, these - -workers run task(\*args + \*input, \*\*kwargs). Note that certain kwargs will - -not pass through to the task (see Args below for the list). - -The output from these tasks is saved to a temporary file. When control - -returns to the context manager, the background output is printed in order, - -as if the tasks were run in sequence. - -If exceptions occur in the steps, we join together the tracebacks and print - -them after all parallel tasks have finished running. Further, a - -BackgroundFailure is raised with full stack traces of all exceptions. - -Example: - -# This will run somefunc(1, 'small', 'cow', foo='bar') in the background - -# as soon as data is added to the queue (i.e. queue.put() is called). - -def somefunc(arg1, arg2, arg3, foo=None): - -... - -with BackgroundTaskRunner(somefunc, 1, foo='bar') as queue: - -... do random stuff ... - -queue.put(\['small', 'cow'\]) - -... do more random stuff while somefunc() runs ... - -# Exiting the with statement will block until all calls have completed. - -Args: - -task: Function to run on each queued input. - -queue: A queue of tasks to run. Add tasks to this queue, and they will - -be run in the background. If None, one will be created on the fly. - -processes: Number of processes to launch. - -onexit: Function to run in each background process after all inputs are - -processed. - -"""
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/build/chroot_version_hooks/index.md b/chromium/docs/website/site/chromium-os/build/chroot_version_hooks/index.md deleted file mode 100644 index 416cf6eff9f..00000000000 --- a/chromium/docs/website/site/chromium-os/build/chroot_version_hooks/index.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: chroot_version_hooks -title: chroot_version_hooks ---- - -### Summary - -On occasion changes are made to some component of the Chromium OS source that -are not backwards-compatible with existing chroots. When this happens a "chroot -version hook" should be used to make whatever changes are necessary to an -existing chroot to make it compatible. Often this involves deleting some -previously compiled/generated component in order to force it to be -compiled/generated again. Another possible scenario is when there is a Chromium -OS source change that is dependent on a change coming in a future version of the -SDK, a version hook is needed to bridge that gap between SDK releases and make -that change directly in existing chroots (e.g. a config file change). - -### Details - -The version hooks all exist under -[src/scripts/chroot_version_hooks.d](https://chromium.googlesource.com/chromiumos/platform/crosutils/+/HEAD/chroot_version_hooks.d/). -Each hook script has a name that begins with "<num>_", where <num> -is an integer that represents the version number of a chroot. Whenever a new -chroot is entered (via "cros_sdk") or "build_packages" is run all upgrade hooks -with versions newer than the chroot version are run (in sequential order) to -upgrade the chroot to the latest version. - -Upgrade scripts can be arbitrary bash scripts. The best way to write a new -script is to look for existing examples in chroot_version_hooks.d. With that -said, the following example hook script will delete the build roots for all -boards and all Chrome artifacts. This is needed, for example, when a toolchain -uprev needs to be reverted. Everything built with the newer (bad) toolchain must -be deleted. - -```none -# This is to clear the build roots for all boards to cope with a -# toolchain revert. -for board_root in /build/* ; do - board=${board_root##*/} - emerge_board=$(type -P emerge-${board} 2>/dev/null || true) - if [[ -x "${emerge_board}" ]]; then - # It is a valid baord. - build="/build/${board}" - if [[ -d ${build} ]] ; then - # The board has a build root to clear. - info "Deleting ${board} build root at ${build}" - sudo rm -rf ${build} - info "Running setup_board --board=${board}" - ~/trunk/src/scripts/setup_board --board=${board} --skip_chroot_upgrade - fi - fi -done -# Delete Chrome artifacts -sudo rm -rf /var/cache/chromeos-chrome/* || true -``` diff --git a/chromium/docs/website/site/chromium-os/build/cros-deploy/index.md b/chromium/docs/website/site/chromium-os/build/cros-deploy/index.md deleted file mode 100644 index 6bd0a2b7195..00000000000 --- a/chromium/docs/website/site/chromium-os/build/cros-deploy/index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: cros-deploy -title: Cros Deploy ---- - -## Note: this page is deprecated. It has been ported to <https://chromium.googlesource.com/chromiumos/docs/+/main/cros_deploy.md>. Please fix any links pointing to this page to point to the new URL.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/build/cros-flash/index.md b/chromium/docs/website/site/chromium-os/build/cros-flash/index.md deleted file mode 100644 index b4bc038e35e..00000000000 --- a/chromium/docs/website/site/chromium-os/build/cros-flash/index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: cros-flash -title: Cros Flash ---- - -## Note: this page is deprecated. It has been ported to <https://chromium.googlesource.com/chromiumos/docs/+/HEAD/cros_flash.md>. Please fix any links pointing to this page to point to the new URL. diff --git a/chromium/docs/website/site/chromium-os/build/faq/index.md b/chromium/docs/website/site/chromium-os/build/faq/index.md deleted file mode 100644 index 92a3b405c95..00000000000 --- a/chromium/docs/website/site/chromium-os/build/faq/index.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: faq -title: Build FAQ ---- - -[TOC] - -# Google Storage - -## What bucket can I use for testing? - -If you are a Googler, you can use gs://chromeos-throw-away-bucket for testing. -Keep in mind that the contents can be read or deleted by anyone at Google. - -We also have another similar bucket named gs://chromeos-shared-team-bucket . -Probably using gs://chromeos-throw-away-bucket makes more sense since it is -clear that the contents there can be deleted by anyone at Google. - -If you are not a Googler, you should create your own bucket in Google Storage. - -# Portage/Emerge - -## How do I force emerge to use prebuilt binaries? - -```none -emerge -g --usepkgonly -(Note: The --getbinpkg only flag for emerge does not actually work) -``` - -## How do I check the USE flags for a package I am emerging? - -Emerge will tell you what USE flags are used when the **-pv** flags are provided -e.g. - -```none -emerge-$BOARD chromeos-chrome -pv -``` - -# Development - -FAQ items covering general development questions - -## What should I read first/Where do I start? - -<https://chromium.googlesource.com/chromiumos/docs/+/HEAD/developer_guide.md> - -## How can I make changes depend on each other with Cq-Depend? - -<https://chromium.googlesource.com/chromiumos/docs/+/HEAD/contributing.md#cq-depend> - -## How do I create a new bucket in gsutil? - -[gsutil -FAQ](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/gsutil.md#FAQ) - -## How do I port a new board to Chromium OS? - -<https://sites.google.com/a/chromium.org/dev/chromium-os/how-tos-and-troubleshooting/chromiumos-board-porting-guide> - -## Making changes to the Chromium browser on ChromiumOS (AKA Simple Chrome) - -<https://chromium.googlesource.com/chromiumos/docs/+/HEAD/simple_chrome_workflow.md> - -## How to install an image on a DUT via cros flash? - -<https://chromium.googlesource.com/chromiumos/docs/+/HEAD/cros_flash.md> diff --git a/chromium/docs/website/site/chromium-os/build/index.md b/chromium/docs/website/site/chromium-os/build/index.md deleted file mode 100644 index e858aed2c05..00000000000 --- a/chromium/docs/website/site/chromium-os/build/index.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -page_name: build -title: Chromium OS Build ---- - -<div class="two-column-container"> -<div class="column"> - -Developer/User documentation pertaining to the Chrome OS build system. - -Google-internal documentation can be found linked from the [internal build team -page](http://goto.google.com/cros-build). - -#### For Developers - -* [Developer - guide](http://www.chromium.org/chromium-os/developer-guide) -* Imaging your device with [Cros - Flash](https://chromium.googlesource.com/chromiumos/docs/+/master/cros_flash.md) -* Install packages to your device with [Cros - Deploy](https://chromium.googlesource.com/chromiumos/docs/+/master/cros_deploy.md) -* [Bypassing tests on a per-project - basis](/chromium-os/build/bypassing-tests-on-a-per-project-basis) -* [Adding a Package to the SDK](/chromium-os/build/add-sdk-package) - -#### For Sheriffs - -* <http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium-os> - -#### For Build Contributors and other resources - -* Help writing unittests using [python-mock](/chromium-os/python-mock) -* [Python style - guide](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/styleguide/python.md) -* [Chromium OS Developer - Guide](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/developer_guide.md) -* [Developing Chromium on Chromium - OS](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/simple_chrome_workflow.md) - ("Simple Chrome") -* Misc. [developer helper - scripts](http://www.chromium.org/chromium-os/how-tos-and-troubleshooting/helper-scripts) - -* [Licensing for Chromium OS Package - Owners](/chromium-os/licensing/licensing-for-chromiumos-package-owners) -* [Licensing for Chromium OS - Developers](/chromium-os/licensing/licensing-for-chromiumos-developers) - -</div> -<div class="column"> - -#### Build System Documentation - -* [Portage Build and - FAQ](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/portage/ebuild_faq.md) -* [Portage Package Upgrade - Initiative](http://www.chromium.org/chromium-os/obsolete/portage-package-status) - * [Portage Package Status - Spreadsheet](https://docs.google.com/a/chromium.org/spreadsheet/ccc?key=0AsXDKtaHikmcdEp1dVN1SG1yRU1xZEw1Yjhka2dCSUE#gid=0) -* Build hacking - * [Chroot versioning](/chromium-os/build/chroot_version_hooks) - (chroot version hooks) - * [Clearing all - binaries](https://sites.google.com/a/google.com/chromeos/for-team-members/build/clear_binaries) - (e.g. for a toolchain revert) - -* [cbuildbot - Overview](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/remote_trybots.md) -* Buildbot [Configure/Set - up](/developers/testing/chromium-build-infrastructure/getting-the-buildbot-source/configuring-your-buildbot) - (Chrome Infra guide) -* [Commit Queue overview](/system/errors/NodeNotFound) -* [Local Trybot](http://www.chromium.org/chromium-os/build/local-trybot-documentation) -* [Remote trybot](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/remote_trybots.md) - -#### Build labels - -* [Build-Tools](https://code.google.com/p/chromium/issues/list?can=2&q=Build%3DTools+OS%3DChrome&colspec=ID+Pri+M+Iteration+ReleaseBlock+Cr+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=tiles) -* [Build-Tools-Cbuildbot](https://code.google.com/p/chromium/issues/list?can=2&q=Build%3DTools-Cbuildbot&colspec=ID+Pri+M+Iteration+ReleaseBlock+Cr+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=tiles) - (file [new issue](http://goto.google.com/cros-cbuildbot-ticket)) -* [Build-Tools-Paygen](https://code.google.com/p/chromium/issues/list?can=2&q=Build%3DTools-Paygen&colspec=ID+Pri+M+Iteration+ReleaseBlock+Cr+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=tiles) - (file [new issue](http://goto.google.com/cros-paygen-ticket)) -* [Build-Tools-Portage](https://code.google.com/p/chromium/issues/list?can=2&q=Build%3DTools-Portage&colspec=ID+Pri+M+Iteration+ReleaseBlock+Cr+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=tiles) -* [Build-Tools-Pushlive](https://code.google.com/p/chromium/issues/list?can=2&q=Build%3DTools-Pushlive&colspec=ID+Pri+M+Iteration+ReleaseBlock+Cr+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=tiles) - (file [new issue](http://goto.google.com/cros-pushlive-ticket)) -* [Build-Tools-SimpleChrome](https://code.google.com/p/chromium/issues/list?can=2&q=Build%3DTools-SimpleChrome&colspec=ID+Pri+M+Iteration+ReleaseBlock+Cr+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=tiles) -* [Build-Tools-Trybot](https://code.google.com/p/chromium/issues/list?can=2&q=Build%3DTools-Trybot&colspec=ID+Pri+M+Iteration+ReleaseBlock+Cr+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=tiles) -* [Build-Tools - label=stats](http://goto.google.com/cros-build-stats-tickets) (file - [new issue](http://goto.google.com/cros-build-stats-ticket)) -* [Cr-OS-Packages](https://code.google.com/p/chromium/issues/list?can=2&q=Cr%3DOS-Packages&colspec=ID+Pri+M+Iteration+ReleaseBlock+Cr+Status+Owner+Summary+OS+Modified&x=m&y=releaseblock&cells=tiles) - -Or file a generic Build issue to be triaged by the build team at -[goto.google.com/cros-build-ticket](https://code.google.com/p/chromium/issues/entry?template=Build%20Infrastructure&labels=Build,OS-Chrome,Pri-2&summary=your%20words%20here). - -#### Other Resources - -* Build [FAQ](/chromium-os/build/faq) - -</div> -</div> diff --git a/chromium/docs/website/site/chromium-os/build/local-trybot-documentation/index.md b/chromium/docs/website/site/chromium-os/build/local-trybot-documentation/index.md deleted file mode 100644 index 8d9b93b11c8..00000000000 --- a/chromium/docs/website/site/chromium-os/build/local-trybot-documentation/index.md +++ /dev/null @@ -1,152 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: local-trybot-documentation -title: Chromium OS Local Trybots ---- - -[TOC] - -## Introduction - -The local trybot allows you to emulate a buildbot run on your local machine with -a set of your changes. The changes are patched to tip of tree (TOT). They should -be very similar to remote tryjobs in most regards. - -For Google developers, please also take a look at the [Remote Trybot -documentation](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/remote_trybots.md). - -NOTE: The first time you run the trybot it will sync down a fresh checkout of -the source, build a new chroot and board, which will make the initial run take -longer than subsequent runs. An incremental run with an existing board takes -30min-40min. - -**Pre-instructions (These are important!)** - -1. Make sure you’ve read through the [ChromiumOS Developer - Guide](https://chromium.googlesource.com/chromiumos/docs/+/main/developer_guide.md) - and can kick off a build properly. -2. [Modify your sudo - config](https://chromium.googlesource.com/chromiumos/docs/+/main/tips-and-tricks.md#How-to-make-sudo-a-little-more-permissive) - If you are at Google, you need to follow [these - instructions](http://go/cros-glinux-sudo#configuring-etcsudoers) - first (Under 'Tweaking with /etc/sudoers' section). -3. Run `sudo apt install qemu-kvm pbzip2 zip`. -4. Run `sudo apt upgrade`. -5. Run `gclient` from outside the chroot to update depot_tools. -6. If you haven’t rebooted your computer in a while, reboot it now - after updating. - -## **Verifying the Trybot** - -It’s good to first verify things are working properly with your system setup. - -To do so, run the following from within your repo source checkout: - -1. Run `repo sync` to get latest version of trybot. -2. Run `cros tryjob --local amd64-generic-full-tryjob` to do a test run with - the current TOT. Everything should pass. - -## **Instructions For Using the Trybot** - -Run the following from within your repo source checkout: - -1. Run `repo sync` to get latest version of cros trybot. -2. Run `cros tryjob --list` to see a list of configs to run with. Add - additional arguments to filter, such as `cros tryjob --list release` - -### To patch in a gerrit CL - -**3a.** Run - -```none -cros tryjob --local -g '[*]<cl_1> [*]<cl_2> .. [*]<cl_N>' config -``` - -Substitute <config> with your desired config. Prepend a '\*' to the CL ID -to indicate it's an internal CL. The CL ID can be a Gerrit Change-ID or a change -number. - -An example: - -```none -cros tryjob --local -g I5bed88effd9c4c26885f8c75da1ec2499c4b74c8 -g 12345 -g *4168 amd64-generic-full-tryjob -``` - -This example patches in three CLs: - -1. an external CL using Gerrit change-ID (unabbreviated commit hash) -2. an external CL using Gerrit CL number -3. internal CL using Gerrit CL number. In case a CL has several patchsets -associated with it, the latest patchset is used. - -To patch in multiple CLs, you can pass all CL numbers in a quoted, -space-delimited string, or specify the `-g` argument multiple times. - -### To patch in a local change - -**3c.** Run - -```none -cros tryjob -p '<project1>[:<branch1>]...<projectN>[:<branchN>]' config -``` - -Specify the name of the project (not the path) and optionally the project -branch. If no branch is specified the current branch of the project will be -used. - -NOTE: To get the project name of the project you're working on run repo list. - -NOTE: Do not do development within the trybot buildroot! Your changes will get -wiped out on the next trybot run. Develop within your source root, and patch in -changes. - -NOTE: Use the --nosync option to prevent the trybot from updating its source -checkout. See the Tips section below for more info. - -NOTE: Per the output of `cros tryjob --help`, the `-p` option is known to be -buggy and `-g` is preferred for specifying patches. Note that `-g` requires the -specified change to be uploaded to Gerrit. - -## **Sample Trybot Run** - -```shell -$ cros tryjob --local -g 'I5bed88effd9c4c26885f8c75da1ec2499c4b74c8 4168' -p 'chromiumos/chromite chromiumos/platform/crosutils' amd64-generic-asan-tryjob - -WARNING: Using default directory /usr/local/google/home/ldap/trybot as buildroot - -Do you want to continue (yes/NO)? yes - -INFO: Saving output to cbuildbot.log file - -[sudo] password: -``` - -NOTE: The output of the trybot is automatically saved to a cbuildbot.log file in -<trybot_root>/cbuildbot_logs. The log directory is printed out at the end -of a run. - -## **Overview of Trybot Operation** - -The first time you run the trybot, it will sync down its own checkout of the -source, and build its own chroot. The CL's you specify to patch are patched to -the trybot's own source tree. - -When you specify local changes to patch (by specifying a project and branch the -changes are on) the trybot will generate git patch files based on those changes -and apply the patch files to its private source checkout. - -When you specify a Gerrit changelist, the trybot will look up the changelist -info from Gerrit, fetch the ref of the change, and rebase it to TOT in its -private source checkout. - -## **Feedback** - -Thanks for using the trybot! Please contact -[chromium-os-dev@chromium.org](https://groups.google.com/a/chromium.org/forum/#!forum/chromium-os-dev) -for any issues you run into. Please report any bugs you find, and file feature -requests. There will be active development on the trybot, so please sync to the -latest version before you run it. diff --git a/chromium/docs/website/site/chromium-os/build/sdk-config-management/index.md b/chromium/docs/website/site/chromium-os/build/sdk-config-management/index.md deleted file mode 100644 index ee02457f675..00000000000 --- a/chromium/docs/website/site/chromium-os/build/sdk-config-management/index.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: sdk-config-management -title: SDK Config Management ---- - -## SDK Config Files Overview - -Portage and Chromite libraries use various make.conf files to store and retrieve -SDK configurations. - -* `/etc/make.conf` - * symlinked to - [src/third_party/chromiumos-overlay/chromeos/config/make.conf.amd64-host](https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/main/chromeos/config/make.conf.amd64-host) - * Overall make.conf for the SDK “board.” - * Sources the other `make.conf.*` files to pull in additional config. - * Treated by - [sysroot_lib](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/lib/sysroot_lib.py) - in Chromite as the source to read configuration from as it contains the - full picture. -* `/etc/make.conf.board_setup` - * Generated by `cros_sysroot_utils` with references to amd64-host sysroot - removed. - * Declares `BOARD_OVERLAY` and `CHOST`. - * Treated by - [sysroot_lib](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/lib/sysroot_lib.py) - as where to write additional configuration. -* `/etc/make.conf.host_setup` - * Created by - [update_chroot](https://chromium.googlesource.com/chromiumos/platform/crosutils/+/HEAD/update_chroot) - each time the SDK is updated. diff --git a/chromium/docs/website/site/chromium-os/build/sdk-creation/index.md b/chromium/docs/website/site/chromium-os/build/sdk-creation/index.md deleted file mode 100644 index d98b5d844fc..00000000000 --- a/chromium/docs/website/site/chromium-os/build/sdk-creation/index.md +++ /dev/null @@ -1,202 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: sdk-creation -title: Chromium OS SDK Creation ---- - -[TOC] - -## Introduction - -The Chromium OS project has an SDK that provides a standalone environment for -building the target system. When you boil it down, it's simply a Gentoo/Linux -chroot with a lot of build scripts to simplify and automate the overall build -process. We use a chroot as it ends up being much more portable -- we don't have -to worry what distro you've decided to run and whether you've got all the right -packages (and are generally up-to-date). Most things run inside of the chroot, -and we fully control that environment, so Chromium OS developers have a lot less -to worry about. It does mean that you need root on the system, but -unfortunately, that cannot be avoided. - -This document will cover how the SDK is actually created. It assumes you have a -full Chromium OS checkout already. - -## Prebuilt Flow - -When you run `cros_sdk` (found in -[chromite/bin/](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/scripts/cros_sdk.py)) -for the first time, it automatically downloads the last known good sdk version -and unpacks it into the chroot/ directory (in the root of the Chromium OS source -checkout). - -That version information is stored in -[src/third_party/chromiumos-overlay/chromeos/binhost/host/sdk_version.conf](https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/HEAD/chromeos/binhost/host/sdk_version.conf): - -``` -SDK_LATEST_VERSION="2022.03.17.105954" -``` - -This is used to look up the tarball in the -[chromiumos-sdk](https://storage.googleapis.com/chromiumos-sdk/) Google Storage -bucket. So with the information above, we know to fetch the file: - -<https://storage.googleapis.com/chromiumos-sdk/cros-sdk-2022.03.17.105954.tar.xz> - -## Bootstrap Flow - -The question might arise: How is the prebuilt SDK tarball created in the first -place? The chromiumos-sdk builder (a.k.a. the “SDK builder”) builds each -new version of the SDK tarball for developers and other builders to use. The SDK -builder runs in a continuous loop to include any recent commits and takes ~24 -hours for a successful run. - -For a bootstrap starting point, to avoid the case where the SDK itself may have -been broken by a commit, the builder uses a pinned known "good" version from -which to build the next SDK version. This version is manually moved forward as -needed. - -If the overall SDK generation fails, then the SDK is not refreshed and the -released files stay stable for developers. - -### Overview - -The overall process looks like: - -* Download bootstrap version of the SDK and setup chroot environment -* Build the SDK (amd64-host) board -* Build and install the cross-compiler toolchains -* Generate standalone copies of the cross-compilers -* Package the freshly built SDK creating and uploading a tarball of it -* Verify the SDK by building/testing other boards with it -* Upload binpkgs -* Uprev the SDK to point to this latest version - -### SDK bootstrap version - -We download a copy of the SDK tarball to bootstrap with and setup the chroot -environment. This is just using the standard `cros_sdk` command with its -`--bootstrap` option. - -As with the latest SDK version, the bootstrap version of the SDK to use is -stored in -[sdk_version.conf](https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/HEAD/chromeos/binhost/host/sdk_version.conf): - -``` -BOOTSTRAP_FROZEN_VERSION="2020.11.09.170314" -``` - -This is used to look up the tarball in the chromiumos-sdk Google Storage bucket. -So with the information above, `cros_sdk` knows to fetch the file: - -<https://storage.googleapis.com/chromiumos-sdk/cros-sdk-2020.11.09.170314.tar.xz> - -`cros_sdk` continues its normal process of running -[`update_chroot`](https://chromium.googlesource.com/chromiumos/platform/crosutils/+/HEAD/update_chroot) -and setting up the chroot SDK environment. - -### Build the SDK (amd64-host) board - -The first step of the creation of the new SDK is for the SDK builder to -[build the special amd64-host board](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/cbuildbot/stages/build_stages.py#554). -To leverage existing build scripts and infrastructure we treat building the SDK -as similar to a traditional board called "amd64-host." - -This step is performed by the -[src/scripts/build_sdk_board](https://chromium.googlesource.com/chromiumos/platform/crosutils/+/HEAD/build_sdk_board) -script and is accomplished by just running: - -``` -./build_sdk_board --board amd64-host -``` - -If you're used to the normal Chromium OS flow, be aware that this step is like -running setup_board and build_packages in one go. - -This means it'll take quite a long time for this to finish (as it has to build a -few hundred packages from source). Everything will be written to -`/board/amd64-host`. - -### Build and install the cross-compiler toolchains - -To -[build and install the toolchains](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/cbuildbot/stages/sdk_stages.py#75), -the cbuildbot process will then run -[`cros_setup_toolchains`](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/scripts/cros_setup_toolchains.py) -which generates all toolchains marked for inclusion in the SDK (see the -toolchain.conf file for more details). - -### Generate and upload standalone cross-compilers - -Since our cross-compilers are pretty cool & useful, we want to be able to use -them all by themselves. In other words, without the overhead of the full -Chromium OS source checkout and the SDK. With a few tricks, we can accomplish -exactly that. It copies all the host libraries the toolchain itself uses, -generates wrapper scripts so that the local copies of libraries can be found, -and then takes care of munging all the paths to make them standalone. - -The -[cros_setup_toolchains](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/scripts/cros_setup_toolchains.py) -script has all the logic to create the packages which are then uploaded to a -Google Storage bucket. - -### Package The SDK - -Now that the SDK is compiled and has everything we want, we -[create then upload the SDK tarball](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/cbuildbot/stages/sdk_stages.py#127) -to a Google Storage bucket. - -### Test The SDK - -We want to be sure that the SDK we just built is actually sane. This helps us -from releasing a toolchain update (like gcc) that is horribly broken (e.g. can't -properly compile or link things). -[To test this](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/cbuildbot/stages/sdk_stages.py#309), -we simply use the SDK tarball built earlier to create a new chroot. This also -verifies the normal developer workflow of creating a new chroot from this SDK. - -Inside of the new chroot, we do a normal build flow for a couple of boards. At -the moment, that means one for each major architecture (i.e. amd64-generic, -arm-generic). We only run setup_board+build_packages though; no unittests or -anything else (as current history as shown it to not be necessary). - -### Upload the binary packages - -Once all the tests pass, we go ahead and -[upload the binary packages](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/cbuildbot/stages/artifact_stages.py#645) -for the SDK itself. - -### Uprev the SDK - -Now that the new SDK version has been tested and the tarball and binary packages -uploaded, the SDK is ready to be -[upreved](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/cbuildbot/stages/sdk_stages.py#379) -to point to this latest version. This simply updates the `SDK_LATEST_VERSION` in -the -[sdk_version.conf](https://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/HEAD/chromeos/binhost/host/sdk_version.conf) -file mentioned earlier. - -The new SDK is now available for developers and builders to use. - -## Running the SDK builder as a developer - -Suppose as a developer you have changes (e.g. in -[chromite/lib](https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/lib/)) -that you wish to test with the SDK builder. To run the entire chromiumos-sdk -builder process described above, run as a tryjob: - -``` -cros tryjob -g <cl 1> [-g …] chromiumos-sdk-tryjob -``` - -where `<cl 1> .. <cl n>` are CLs to be run against. - -It's likely that just building the SDK board locally would be sufficent for most -cases. To do that, from `~/chromiumos/src/scripts` run: - -``` -./build_sdk_board -``` diff --git a/chromium/docs/website/site/chromium-os/build/tour-of-the-chromiumos-buildbot/index.md b/chromium/docs/website/site/chromium-os/build/tour-of-the-chromiumos-buildbot/index.md deleted file mode 100644 index fc3516c72c7..00000000000 --- a/chromium/docs/website/site/chromium-os/build/tour-of-the-chromiumos-buildbot/index.md +++ /dev/null @@ -1,153 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: tour-of-the-chromiumos-buildbot -title: Waterfall Tour ---- - -A ChromiumOS Waterfall hosts and lists groups of ChromiumOS builds. - -ChromiumOS uses [LUCI](https://chromium.googlesource.com/infra/luci/luci-go/) to -run continuous builds, integration (commit queue) and tests. It uses chromite to -centralize/abstract the process of building for various configurations. - -There are a number of builder groups that each present a waterfall view of their -status, but with chromiumos' use of ebuilds and manifests, many of the LUCI's -paradigms are not appropriate. -There is a column of changes, but these represent changes present in the -repositories, but does not represent which ones are included in the product by -the manifest references and the ebuilds used. - -[TOC] - -## Basics - -The LUCI scheduler watches the clock, git repositories and previous builds, -telling builders to start building and testing new revisions. LUCI's Milo -service serves the "waterfall" page that shows all the results. For the full and -incremental builders every time a new revision is discovered, the client -triggers builds. Each of these builders generally has a single bot, so it is -usually busy when this happens. When the bot becomes idle the most recent -trigger is chosen to represent all automatic builds to date and it is started. -The rest of the pending builds are presumed to be covered enough, and are -removed. The build itself for chromeos is a sequence of three steps: - -1. Update the builder recipes (Basically, the piece the Chromium - infrastructure people maintain to run the builders) -2. Update the chromite tools (Basically, the piece the ChromeOS build - people maintain to drive a ChromiumOS build) -3. Run the cbuildbot command with a configuration name to do the real - work. - -The third step involves many stages, and it is parsed and presented as -individual steps by the recipe engine, but it is all coming from the one -command. This command can be run on local checkouts, by developers, in the same -ways as the bots do in order to predict and reproduce failures, tests, etc. This -page is about the mechanics that get it run, not what it does once it is run. - -While each of these steps is executing Milo is collecting the output, and -presents the state on the web interface. There are other people and processes -that also monitor the state the client exports, or the side effects of the build -on git or google storage, to start further builds, send emails, etc. - -## Builders - -Let's take a look at the [waterfall -page](http://build.chromium.org/p/chromiumos/waterfall). For now, skip over the -header (green, red, yellow, etc.) and the box at the top with lots of links and -horizontal stripes (we'll come back to them later), and look down at the row of -grey boxes with "changes" at the left. Those boxes show one column per builder. -In ChromeOS we tend to name these by board/config name, and role. Each of these -builders is associated with one or more bots, which are machines willing to run -such builds. Each builder belongs to exactly one client (waterfall) but in some -waterfalls they can be willing to serve more than one builder. The title in this -row will link to a page that lists the recent builds completed for this -configuration, and the builders associated with it. This is a useful view to see -what recent runs have looked like, especially for comparing or contrasting. - -## Builds - -Once a build request gets assigned to a builder it begins a "Build". -Chromite/cbuildbot generally starts with clean up and an open sync to the state -of the tree (tip of branch in the branch case). Chromite/cbuildbot will tell you -what it is building in the sync stage, or provide references as links in the -build steps. Each build will have a number, a bot, and will run until completion -or fatal error. The waterfall view will present the exposed steps in reverse -chronological order in the column it belongs in. There is a list of times -(generally in Google Coordinated Time) in the leftmost column to record when -observation are made, and the boxes change colors to represent the current state -as they progress. When a bot is finished it may disconnect and locally clean -itself up (e.g. by rebooting) before asking for its next slug of work. - -## Current Activity - -In the row just above the titles for the grid there is a summary for each -builder of what is currently going on, and what is upcoming (active builds, -pending builds, delay to next time based decision). Beware that the ETA in this -box is computed using a very simple predictive model that assume each build is -roughly the same as the ones before. This goes very wrong when changing between -builds that work, and builds which break a little ways in. Beware of these -predictions in anything other than routine situations. - -## Last Build - -The top row of the grid part is a summary of how the previous build finished. We -often view this as the state of the builder. In a rapidly cycling build system -it is a close representative of what the state of the tree is for the type of -build represented. For a slow-cycling system, or build requests that don't -follow a smooth tree ordering this is less true and relevant. For ChromeOS -consider the type of builder before thinking about what the state (or color) of -the builder means. - -## The Announcement - -When present, we tend to use variations on a theme for the announcement box on -top of the waterfall (and some other) pages. We use the announcement to -abbreviate and present other interesting state information. The top bar and it's -color represents the -[state](http://www.chromium.org/developers/tree-sheriffs/sheriff-details-chromium-os#TOC-How-do-I-read-the-waterfall-) -of the tree, including the current -[message](http://chromiumos-status.appspot.com/). Under that there are 2 panes. - -* On the right there are usefully grouped lines of boxes that repeat - the last build (see above) status. The label on the left of each row - indicates what it is, and should link to a view of the waterfall - with just those builders for more detail. -* On the left there are useful links, lists of current rotations - (roles taken by different people each day or two), and links to - different views of the build data. - -## Different types of ChromeOS builders - -Almost all of these come from the workings of chromite, but are very useful to -have an idea about to understand the big picture: - -* Full - a build that cleans out the chroot, and builds from stable - ebuilds at the tip of tree -* Canary - an official build with a new saved version -* Incremental - a build that updates the repos, and builds from stable - ebuilds at the tip of tree -* Try - a build that updates the repos, applies a patch, updates - ebuilds and tries to build -* [ChromeOS ASAN information](/system/errors/NodeNotFound)ASAN - see -* SDK - try new toolsets and update the prepackaged information for - the chromeos SDK -* Toolchain - try newer compilers -* (chrome) - try building and testing chromeos with tip of tree chrome - version -* branch - pull a branch of the source code and try building there - (factory, firmware, releases) -* LKGM - distribute a successful and stable manifest to other - interested parties - -## When the columns aren't right - -Currently the set of columns and the allocations of bots to back them is managed -by the Chrome Infrastructure team, please file an -[Infrastructure](https://goto.google.com/cros-infra-bug) -bug OS-Chrome to start the process of correcting it. Please -describe the type of build, and the cbuildbot configuration it should be -running. diff --git a/chromium/docs/website/site/chromium-os/build/using-remote-trybots/index.md b/chromium/docs/website/site/chromium-os/build/using-remote-trybots/index.md deleted file mode 100644 index e5063985629..00000000000 --- a/chromium/docs/website/site/chromium-os/build/using-remote-trybots/index.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/build - - Chromium OS Build -page_name: using-remote-trybots -title: Chromium OS Remote Trybots ---- - -This page has moved to -<https://chromium.googlesource.com/chromiumos/docs/+/HEAD/remote_trybots.md>. -Please update the link that brought you here. |