diff options
Diffstat (limited to 'chromium/docs/website/site/chromium-os/chromiumos-design-docs')
.png.sha1?id=7d2c5d177e9813077a621df8d18c0deda73099b3){kind=link}
.png.sha1?id=7d2c5d177e9813077a621df8d18c0deda73099b3){kind=link}
.png.sha1?id=7d2c5d177e9813077a621df8d18c0deda73099b3){kind=link}
107 files changed, 0 insertions, 11120 deletions
diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/cycle_break.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/cycle_break.png.sha1 deleted file mode 100644 index 442f1daf557..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/cycle_break.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -48ce62b95ca1a8d64ac7cb306f49a8591b1478ee
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/example.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/example.png.sha1 deleted file mode 100644 index 4d48833b437..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/example.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -891c4899987a83870f1ac70cd237ea6bbbd92885
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/index.md deleted file mode 100644 index cc578b9da0c..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/autoupdate-details/index.md +++ /dev/null @@ -1,229 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: autoupdate-details -title: Autoupdate Details ---- - -## Abstract - -This document describes details about how the autoupdate system works. For a -high-level view of autoupdate, and information about the filesystem, see [File -System/Autoupdate](/chromium-os/chromiumos-design-docs/filesystem-autoupdate). - -The autoupdate system provides bit-for-bit-exact in-place filesystem updates -with file-level binary diffs. - -## **Goals** - -* Updates should be as small as we can reasonably make them. -* An update must result in a root-filesystem partition in which each - disk block is exactly (bit-for-bit) as specified by the OS vendor, - so that the update can be signed on the server (for verified boot). -* Updates must be applied in place, so that many delta updates can be - installed without rebooting. For example: if a user is booted into - version N, and N+1 is released, the user downloads the (N→N+1) - update and installs. If the user is still booted into N when N+2 - comes out, then the user downloads the (N+1→N+2) update and in-place - installs it over N+1. - -## **Overview of the install procedure** - -The client will have a partition (which we'll call the "install partition") on -which to install a delta update. This partition will already contain some -version of the operating system. - -The client will contact the autoupdate server to request an update, specifying -the version number of the system installed on the install partition. The server -may provide the client with a delta update which the client will download. - -The delta update file contains an ordered list of operations to perform on the -install partition that will take it from the existing version to the new -version. - -## **Overview of the delta update file format** - -Note: The destination partition is composed of 4K blocks.\[1\] - -The update file is an ordered list of operations to perform. Each operation -operates on specific blocks on the partition. For each operation, there may be -an optional data blob, also included in the update file. - -The types of install operations are: - -* Copy: copy some blocks in the install partition to other blocks in - the install partition. This can be used to move a block to a new - location, or to copy a block to temporary storage; for more - information, see the example below. -* Bsdiff: read some blocks into memory from the existing install - partition, perform a patch (binary diff) using the attached data - blob, and write the resulting updated blocks to specified blocks in - the install partition. (See below for more details.) -* Replace: write the attached data blob to specified blocks in the - install partition. -* Replace_bz: B-unzip the attached data blob and write the result to - specified blocks in the install partition. - -A traditional diff update to a file (using bsdiff) works like this: the old file -and the patch file are read into memory. Next, a patch operation is performed in -memory, resulting in the new file being in memory. Finally, the new file is -written to disk. -We modify that operation slightly. We tell the patch program that the old file -is the install partition. However, rather than have the program read the entire -partition into memory, we tell it which blocks to read. Then the patch operation -is performed in memory. Finally, the result is written directly to the install -partition, but not at the beginning of the device: we will tell the program -which blocks to write the result to. - -## **Generating delta update files** - -This section describes how the OS vender creates an update file. - -The update file format will be: - -1. Magic number ('CrAU') -2. Version number -3. Eight bytes for protobuf length -4. The protobuf -5. Collection of data blobs -6. EOF - -The protobuf is a series of instructions that the client must perform in order. - -Note: To specify a set of blocks, we use an extent, which is simply a contiguous -range of disk blocks. For example, rather than specify blocks {10, 11, 12, 13, -14, 15, 17, 18}, it can be simpler to specify { (10, 6), (17, 2) } (a list of -extents). -message Manifest { -message InstallOperation { -enum CompressionType { COPY = 0, // file is unchanged; just move data blocks -BSDIFF = 1, // Read source data blocks as old file, included binary blob is -diff, output to new blocks REPLACE = 2, // Output included binary blob to new -blocks REPLACE_BZ = 3 // Bunzip binary blob into new blocks } uint64 -blob_offset; // if present, the offset in the update image of the binary blob -for this file uint64 blob_length; // if present, the length of the binary blob -message extent { uint64 offset; // in blocksize uint64 length; // in blocksize } -repeated extent input_extents; repeated extent output_extents; } -repeated InstallOperation install_operations; -} - -To generate a delta update, we iterate over each regular file on the new -filesystem. We get an ordered list of all datablocks in the file, then store -those in a File struct: -struct Extent { -uint64 start; -uint64 length; -} -struct File { -string path; // path within the filesystem -vector<Extent> dst_extents; // ordered list of all extents on the new -filesystem -vector<Extent> src_extents; // Applies only for COPY and BSDIFF -enum CompressionType; // one of: COPY, BSDIFF, REPLACE, REPLACE_BZ -} - -Note: eventually, each File object will be converted into an InstallOperation -message in the protobuf. -For each file, we look for the optimal way to compress it. If the file has -changed, then we compare the sizes of the binary-diff, uncompressed, and bzip -options, and we pick whichever yields the smallest file size. - -We then create a vertex in a graph for each File object. Alongside the graph, we -also create a vector to represent each block in the install partition: -struct Block { -File\* reader; -File\* writer; -} -vector<Block> blocks; // length is the size of the install partition - -We then go through each block in each File object. For each block, we set the -reader and writer parameters of the blocks vector. - -Next, we iterate through the blocks array, and for each block with a different -reader and writer (which are both non-null), we create an edge in the graph from -the writer to the reader. An edge in the (directed) graph points to a file -operation that must complete before the edge's source file operation starts. -Thus, we are trying to ensure that if a block is both read and written by -different file operations, the block is read before it's written. The edge -represents blocks in the graph, so the edge's weight is the number of blocks. - -At this point, we are likely to have a graph with cycles. We must break the -cycles. We find the cycles with Johnson's circuit finding agorithm -([PDF](http://dutta.csc.ncsu.edu/csc791_spring07/wrap/circuits_johnson.pdf)) and -Tarjan's strongly connected components agorithm. For each cycle, we find the -lowest-weight edge and cut it\[2\]. We cut an edge as follows: create a new node -that represents an operation of copying some extents to scratch space. We then -make the edge's source node point to the new node. We also modify the cut edge's -destination node to read from the scratch space rather than from the blocks -represented by the edge we're cutting. - -Here's an example of cutting an edge to break a cycle. Operation A reads block 3 -to write block 4. Operation B reads block 4 to write block 10. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/autoupdate-details/cycle_break.png">](/chromium-os/chromiumos-design-docs/autoupdate-details/cycle_break.png) - -Once the cycles are broken, we can use a topological sort to order all the -nodes. Then we convert each node to an InstallOperation and add it to the -Manifest structure. If the client were to download and install the Manifest at -this point, all the blocks that contain filedata would be set correctly. -However, we also need non-filedata blocks to be set correctly. To handle -non-filedata blocks, we create a single final InstallOperation of type -REPLACE_BZ which writes to all extents that don't contain filedata. The attached -data blob contains the bzip2-compressed data to go into those blocks. In -practice, this takes up about 2 megabytes compressed. - -## **Example** - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/autoupdate-details/example.png">](/chromium-os/chromiumos-design-docs/autoupdate-details/example.png) - -## **Streaming** - -Because the protobuf (which lists all operations) occurs at the beginning of the -file, the update doesn't need to be saved to disk. It can be applied while -streaming from the server. - -We do need to make sure that the update is signed by the OS vendor. We can begin -to apply the update and not mark it bootable until after the delta update -signature is verified. - -## **Alternatives considered** - -The solution presented here is not the only one we considered, but it is the -only one we've found that gives the best compression ratio in practice. Other -solutions considered were: - -* Delta-compress the entire partition - Bsdiff can't delta-compress the entire partition because its memory - requirements are too high. During patching, it needs enough memory to store - the original and new files, which in our case could have been over 1 - gigabyte. - However, another delta compression program, Xdelta, uses a sliding window. - In practice, this resulted in poor compression. On a test corpus, the diff - was hundreds of megabytes. - -* rdiff - Rdiff works by storing only changed blocks in the delta file. It uses a - sliding window so that blocks don't need to be aligned. When pointed at the - testing corpus, an rdiff delta of the entire partition was 104M, well above - the roughly 10M the proposed algorithm takes for the same corpus. - In the future, we might consider using rdiff-style (that is, rsync-style) - delta compression at the file level. We could drop that in alongside bsdiff - in the future. - -## **Footnotes** - -\[1\] Some of these blocks contain file-data and some contain other data -(metadata, for example). Since we are using ext4, which (at the time of writing) -doesn't support fragments, we know that a block cannot contain file data for -more than one file: a block either contains file data for one file or no files. -\[2\] In our tests, with the greedy agorithm (cut an edge in each cycle as it's -found), we cut about 28 MB worth of edges, which seems reasonable. Enumerating -all cycles and cutting no edges, we found over 5,000,000,000,000,000 cycles, so -it's absolutely infeasible to consider all cycles before making cuts. To cut -cycles more efficiently, though, we might consider more than 1 cycle before -making cuts (perhaps 1000 cycles or so).
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/boot-design/boot_design_diagram.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/boot-design/boot_design_diagram.png.sha1 deleted file mode 100644 index d302a0ffa87..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/boot-design/boot_design_diagram.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -e95c849dc9380e4584b82b6569fef249b2c43d8c
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/boot-design/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/boot-design/index.md deleted file mode 100644 index c925930a1a9..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/boot-design/index.md +++ /dev/null @@ -1,1014 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: boot-design -title: Chrome OS User-Land Boot Design ---- - -[TOC] - -## Introduction - -This document gives an overview of the design of user-land boot processes for -Chrome OS based systems. It covers what happens after /sbin/init first starts -execution, until all services in the system are ready. - -Chrome OS Core uses -[Upstart](http://www.google.com/url?q=http%3A%2F%2Fen.wikipedia.org%2Fwiki%2FUpstart&sa=D&sntz=1&usg=AFQjCNEuUVRg10UCp5SZDMxMYm7uXUDFrQ) -for its /sbin/init package. Readers of this document should have some basic -familiarity with Upstart concepts, such as the syntax of [job configuration -files](http://manpages.ubuntu.com/manpages/trusty/man5/init.5.html), -and Upstart events, such as -[starting](http://manpages.ubuntu.com/manpages/trusty/man7/starting.7.html) -and -[started](http://manpages.ubuntu.com/manpages/trusty/man7/started.7.html). -Readers should also have some understanding of Linux file system management -concepts, like -[mounting](http://www.google.com/url?q=http%3A%2F%2Flinux.die.net%2Fman%2F8%2Fmount&sa=D&sntz=1&usg=AFQjCNHWgzSgniJkKUaQX0F6aoV6tub_Zw) -and -[creating](http://www.google.com/url?q=http%3A%2F%2Flinux.die.net%2Fman%2F8%2Fmkfs&sa=D&sntz=1&usg=AFQjCNHT-LvbDmluahoYo1skfkb28btuiA) -file systems. - -If you want to go deeper and read the source code, you’ll need some -understanding of shell programming. If you want to understand the initialization -of an individual package, you’ll need to be familiar with that package’s basic -operations. Neither of these topics is covered in this document. - -## Summary of Boot Flow - -### Phases of Boot - -*Boot est omnis divisa in partes tres.* - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/boot-design/boot_design_diagram.png">](/chromium-os/chromiumos-design-docs/boot-design/boot_design_diagram.png) - -The diagram above shows an outline of the Chrome OS Core boot flow. Boot divides -into three sequential phases with four publicly defined Upstart events: - -<table> -<tr> -Upstart Event(s) Phase Description </tr> -<tr> - -<td>startup</td> - -<td><a href="#h.zhrd95efyszv">Basic Services</a></td> - -<td>Initialization of basic services needed by the rest of the system.</td> - -</tr> -<tr> - -<td>started boot-services</td> - -<td><a href="#h.lpny9tz0qmud">System Application</a></td> - -<td>Initialization of the system application and other services critical to the product’s UX</td> - -</tr> -<tr> - -<td>started system-services</td> - -<td>started failsafe</td> - -<td><a href="#h.nm45bjel265c">System Services</a></td> - -<td>Initialization of all other services. The failsafe job is guaranteed to start even if the system application fails.</td> - -</tr> -</table> - -### Basic Services Startup - -“Basic services” are the indispensable services required by the system -application and the rest of the system. Most services cannot operate until these -basic services are running normally. - -At the end of basic services startup (that is, when Upstart emits started -boot-services), the following services are guaranteed available: - -* A file system generally conforming to the [Linux - FHS](http://www.google.com/url?q=http%3A%2F%2Fwww.pathname.com%2Ffhs%2F&sa=D&sntz=1&usg=AFQjCNHWdZjRwe5FA0JKIi0EVckcCzGvaQ). -* A logging service compatible with rsyslogd. -* Device hotplug detection. User input devices or other devices - necessary to the system application will be detected, with modules - loaded. Detection of devices deemed non-critical may be delayed - until after the system application starts. -* dbus-daemon. - -Note that jobs that run during this phase must by their nature work in an -environment where the services above may not be available. At the beginning of -this phase, there is no guarantee of writable storage; jobs cannot log messages, -write any files, or read files outside of the root filesystem. Additionally, -device availability is not guaranteed in all cases; devices may not be -initialized, and their drivers may not even be loaded. - -### System Application Startup - -Chrome OS Core is predicated on the existence of a single, dedicated application -that is central to system startup. For Chrome OS, that application is the Chrome -browser. The Chrome OS sources are structured to allow a platform to choose to -use a different system application. - -The system application starts in parallel with various critical services. -Typically, “critical” means that the system application expects the service to -be present, or that the user would perceive the system as not functional without -it. For example, in Chrome OS these services include: - -* A network. -* The cryptohome daemon. -* System power management. - -The system application receives special treatment in the boot flow. The -application is responsible for emitting the event which starts the boot-complete -job, which allows system services startup to begin. The system application is -responsible for this for two reasons: - -* Delaying non-critical services allows the system application to - start faster. -* If the application fails during startup, the absence of running - system services can be used to detect the failure. - -For Chrome OS, boot-complete starts when the Chrome browser has displayed the -initial login screen. - -### System Services Startup - -This phase of boot includes anything not required in the earlier phases of boot. -Generally, all jobs in this phase run in parallel. Because the system is -considered already booted, this phase is not performance critical. - -After the started system-services event, services can rely on the following: - -* All services required after started boot-services are available. -* The udev events for all devices that were present at boot have been - processed. -* The system application is available. - -By design, system services only start if the system application announces its -successful initialization. For most services, this is sufficient: without a -system application, the system is in a failed state and many services have no -useful purpose. However some services may require that they start eventually, -even if the system as a whole has failed. To support this requirement, a service -can depend on the started failsafe event. After started boot-services, the -failsafe-delay job starts a 30 second timer. The started failsafe event occurs -at the earlier of started system-services or the expiration of the timer. - -Failsafe jobs are commonly used in test and development images for services -necessary to debug or recover from failures. Some examples: - -* The openssh-server job, which allows logging in via ssh. -* The udev-trigger job, which can sometimes be needed to detect and - configure a network device. - -## Filesystem Initialization and Layout - -### Creating the Stateful File System (the chromeos_startup script) - -At startup, there is no writable file system storage available. During basic -services startup, the script chromeos_startup is responsible for mounting file -systems, and creating the basic directory hierarchy. The script is invoked from -the startup job, which starts with the startup event emitted by /sbin/init at -the start of the userland boot process. - -The script creates missing directories and mount points as necessary. This is -done to support re-creating the filesystem after [stateful -wipe](#h.no2bd2qbh52h), and also to protect the filesystem from inadvertent -damage. Below is a list of the principle directories, listed in the order -they’re processed. - -* /tmp - mounted as a tmpfs filesystem -* /sys/kernel/debug - mounted as a debugfs filesystem. -* /mnt/stateful_partition - mounted as an ext4 file system from GPT - partition #1 (but for factory images, a tmpfs file system is used - instead). -* /home and its subdirectories - created as necessary; /home itself is - a bind mount on /mnt/stateful_partition/home -* /var and /home/chronos - mounted as described below under [Encrypted - Stateful](#h.zdlxdrpbjmx9). -* Subdirectories of /var - created as called for by the Linux FHS. -* /usr/local - This directory is only mounted for developer and test - systems. The directory is a bind mount over - /mnt/stateful_partition/dev_image. - -### Encrypted Stateful - -As a security measure to protect private data, the portions of the stateful -partition under /var and /home/chronos are mounted from an encrypted blob stored -in the stateful partition. The encrypted storage is kept in -/mnt/stateful_partition/encrypted. Mounting and unmounting the encrypted data is -handled by the mount-encrypted command. - -Not all Chrome OS Core platforms use this feature: The feature requires a TPM to -be effective. Moreover, the feature presupposes some specific privacy -requirements. For Chrome OS, these requirements apply; they may not be -applicable on all platforms. When the feature is not used, /var and -/home/chronos are simple bind mounts. - -## Special Boot Flows - -Chrome OS Core supports a number of special boot flows, many of which are -designed to handle system behaviors across multiple boots. This includes special -cases for installation, recovery, and update. - -### Boot Alert Messages - -Some special flows involve presenting simple messages to the user prior during -basic services startup. All such flows are detected and handled from -chromeos_startup. The flows involve one of two use cases: - -* During [stateful wipe](#h.no2bd2qbh52h), a message specific to the - specific wipe use case will be presented. -* From time to time, an update may contain new firmware for the - device. These updates must be performed at the start of the first - boot with the new update. - -Because these messages are presented during basi services startup, a special -script called chromeos-boot-alert is used to accommodate the restricted -operating environment: - -* The X server is not running. The script uses ply-image to write - directly to the frame buffer. -* The chromeos-boot-alert script must wait for the boot splash screen - animation to complete in order to guarantee exclusive access to the - frame buffer. -* The script must select a localized message text based on the user’s - default locale. - -### Stateful Wipe (a.k.a. “Powerwash”) - -In certain boot cases, chromeos_startup must erase and re-create the stateful -partition. The script triggers stateful wipe for one of three conditions: - -* The stateful partition failed to mount, or a bind mount on the - stateful partition failed. -* Wipe requested at the previous shutdown. The script will wipe the - stateful partition if a file named factory_install_reset is present - in the root of the stateful filesystem at boot time. This method is - used for two use cases: - * Factory wipe. At the end of the manufacturing process, the - device wipes the stateful partition in preparation for boxing - and shipment. - * Power wash. The user can request a wipe for purposes of - restoring it to a clean state. -* Wipe after changing from dev mode to verified mode, or vice versa. A - file named .developer_mode in the root of the stateful partition - flags whether the system was previously in dev mode: - * If the file is absent and we’re currently in dev mode, we - trigger a wipe for the verified-to-dev mode switch. - * If the file is present and we’re currently in verified mode, we - trigger a wipe for the dev-to-verified mode switch. - -Principally, wiping the stateful partition means re-creating an empty filesystem -using mkfs. Prior to creating the file system, the wipe procedure will also -erase data in the partition in order to prevent leaking any private data that -was present prior to the wipe. - -If the wipe procedure is performed while in dev mode, the .developer_mode file -will be created after the wipe. This marks that the system has successfully -wiped stateful after the conversion to dev mode. - -After the wipe procedure is complete, the system reboots; on reboot the system -will [rebuild directories and mount points](#h.z4jhloholo6z) in the stateful -file system. - -## Interactions with Install, Recovery, and Update - -After new software is installed by chromeos-install (including device recovery), -or automatic update, a reboot is necessary for the new software to take effect. -The first reboot after installing new software triggers special handling, -depending on the specific installation use case. - -### Rollback Protection After Update - -If a catastrophic system failure manifests immediately after an automatic -update, Chrome OS Core can detect the failure and roll back to the previously -installed software. The mechanism relies on a three-way interaction with the -update_engine service, the firmware, and the system application. - -Each kernel GPT partition on the boot disk contains three special attributes -flags, which the firmware uses to select which kernel to boot. The flags are -designated “priority”, “successful”, and “tries”. The firmware selects the -kernel based on these flags; the full rules are described in the [design -document for the disk -format](http://www.chromium.org/chromium-os/chromiumos-design-docs/disk-format). -The update_engine service updates these flags with specific values after -applying an update, and again with different values after the system boots -without failure. - -After update_engine has finished downloading and installed a new image, the -update’s post-install phase marks the GPT flags for the new kernel to have the -highest priority for boot, with “tries” set to 5, and “successful” set to 0. In -this state, the partition will be selected by the firmware as the first kernel -to boot. - -On each boot, the firmware decrements the “tries” setting for the new kernel in -the GPT. If “tries” is decremented to 0 and “successful” is never set to 1, the -firmware will roll back to the previous working software. To prevent the -rollback, the system must mark the kernel as good by setting the “tries” count -to 0 and the “successful” flag to 1 sometime after booting. - -Chrome OS Core does not automatically mark the kernel good immediately after -boot. Instead, when the update_engine service starts, it sets a 45 second timer. -When the timer expires, update_engine invokes a script called -chromeos-setgoodkernel. That script marks the kernel as good, after which -rollback is no longer possible. - -The update_engine service depends on started system-services. Consequently, -rollback will be triggered if a newly updated image consistently fails in any of -the following ways: - -* The kernel panics before chromeos-setgoodkernel can run. -* The system hangs before chromeos-setgoodkernel can run. -* The system application crashes before triggering boot-complete. - -Note that the user will typically have to forcibly power cycle a unit multiple -times in any failure case other than a kernel panic. - -A consequence of this design is that if the system application is crashing at -startup, the system cannot receive new updates. This is a deliberate design -choice: It is not a bug. The rationale is that if a system is failing in this -way, rollback is by far the most preferable way to recover. Waiting for an -update is unlikely to be useful in most cases: - -* If users wait for an update and the next update contains the same - bug, that new update will have overwritten the working previous - version. In this case, rollback (by far the most preferable option) - will be foreclosed. -* It could be days before a problem of this sort can be recognized, - diagnosed, fixed, and released. Many users will be unwilling to wait - so long. -* Even if users are willing to wait, most of them will not be easily - able to determine when a fixed update is available. -* Recovery is likely to be faster than waiting for an update. - -### Installation and Recovery - -The chromeos-install script can be used to install an image from scratch on a -device. The end-user [recovery -procedure](https://www.google.com/chromeos/recovery) is a wrapper around this -script. Additionally, developers can run the script manually when they boot a -custom image from USB in dev mode. Although the UX in the two cases is quite -different, the system state after installation in either case is -indistinguishable. - -The installation process includes creating a stateful partition file system, -similar to the process for stateful wipe. As with stateful wipe, installation is -responsible for creating the .developer_mode file when installing in dev mode. -Also as with stateful wipe, after rebooting the system will [rebuild directories -and mount points](#h.z4jhloholo6z) in the stateful file system. - -### Stateful Update and Install - -For developer and test builds of Chrome OS, substantial portions of the software -are installed in /usr/local, a [bind mount](#h.gfhw9kpet5ar) over -/mnt/stateful_partition/dev_image. This content is not delivered via the -autoupdate mechanism. Instead, a script called stateful_update is used to update -it separately. - -The stateful update procedure operates in two stages: - -* The script downloads a tar file, and extracts new content for - /usr/local and /var into temporary locations in the stateful - partition. -* After reboot, chromeos_startup replaces the old directories with the - new ones prior to mounting /usr/local. - -## Chrome OS - Starting Chrome (the “ui” Job) - -The Chrome browser is the system application for Chrome OS. The ui job is -responsible for starting, managing, and restarting the Chrome OS session_manager -process. The session_manager process in turn is responsible for managing the -Chrome browser process. - -### Job Startup Flow - -The ui job is responsible for starting session_manager. That program is -responsible for the following: - -* Starting the X server process in the background. -* Determining Chrome command-line options. -* Initializing environment variables for the Chrome browser process. -* Initializing the system resources (e.g. directories, files, cgroups) - needed by the Chrome browser. -* Starting the Chrome browser. - -Once Chrome has successfully displayed the initial startup screen, it kicks off -the following sequence of events: - -* Chrome calls session_manager’s EmitLoginPromptVisible interface over - D-bus. -* session_manager emits the login-prompt-visible Upstart event. - -* The login-prompt-visible event triggers the boot-complete job. - -### Handling Chrome Shutdown, Crashes, and Restarts - -When a user logs out of Chrome, the browser process terminates normally. In -turn, session_manager terminates, which ends the ui job. In normal operation, -the ui job must respawn when this happens. However, in the event of a Chrome -crash, the automatic respawn logic is more complex than Upstart’s standard -respawn behavior. Instead, whenever the ui job stops, a separate ui-respawn job -determines what to do according to these rules: - -* If the termination was for system shutdown or reboot, don’t respawn - and allow the shutdown to proceed. -* If the termination was for a simple log out, respawn the browser - unconditionally. -* For most abnormal termination cases, try respawning the browser. -* If an abnormal termination was because session_manager tried to - restart Chrome too many times, try rebooting the system. - -Special handling for abnormal termination is subject to rate limitations: - -* Respawn is limited to no more than 6 times in one minute. -* Rebooting is only attempted if the particular error termination - happens more than once in 3 minutes; otherwise it just respawns. -* Reboot is limited to no more than once every 9 minutes. - -If a failure isn’t handled by respawning or rebooting, the ui job stops, but the -system stays up. The user can manually force power-off, if desired. - -### Upstart Events During Chrome Startup - -Below is a list of various Upstart events triggered by Chrome browser startup, -and when they happen: - -<table> -<tr> -Event When it happens </tr> -<tr> - -<td>starting ui</td> - -<td>This first occurs at boot after started boot-services. After boot, the event will re-occur every time after a user logs out, or any time Chrome restarts after a crash.</td> - -</tr> -<tr> - -<td>x-started</td> - -<td>session_manager emits this event after the X server finishes initializing. This happens after every Chrome restart.</td> - -</tr> -<tr> - -<td>login-prompt-visible</td> - -<td>session_manager emits this event after Chrome announces that has finished displaying its login screen. This happens after every Chrome restart.</td> - -</tr> -<tr> - -<td>started boot-complete</td> - -<td>This occurs the first time that login-prompt-visible is emitted after boot.</td> - -</tr> -<tr> - -<td>start-user-session</td> - -<td>session_manager emits this event after Chrome announces the start of a user session.</td> - -</tr> -<tr> - -<td>stopping ui</td> - -<td>This occurs whenever a user logs out, if the Chrome browser crashes, or when the system is shutting down.</td> - -</tr> -</table> - -## Performance Considerations - -### The Boot Critical Path - -Each phase of boot is dominated by one long running step. These steps must run -sequentially, so together they make up a critical path. For Chrome OS, these are -the longest running steps in the critical path: - -chromeos_startup - -X server startup until session_manager emits the x-started event. - -Chrome browser startup, until login-prompt-visible - -Adding time directly to the critical path will result in delaying boot by an -equivalent time. Whenever possible, initialization should run in parallel with -the critical path. - -### ureadahead - -System startup time is heavily influenced by the wait time required to read data -that isn’t yet in the file buffer cache. The -[ureadahead](http://www.google.com/url?q=http%3A%2F%2Fmanpages.ubuntu.com%2Fmanpages%2Fprecise%2Fman8%2Fureadahead.8.html&sa=D&sntz=1&usg=AFQjCNGq-lkbAxTEkiTp_Yc2moX8IPPY5g) -program can improve boot time by requesting data in advance of when it’s needed, -so that boot spends less time waiting for data from the boot device. Some key -facts about ureadahead: - -* Platforms that want to use this feature must depend on the - sys-apps/ureadahead package. -* Platforms that want to use ureadahead must configure some specific - kernel tracing features. -* The ureadahead program starts execution when chromeos_startup - finishes, and terminates when boot-complete starts. So, ureadahead - doesn’t provide any improvement after the system application has - finished starting. -* Auto-update removes the ureadahead pack file during its post-install - phase. This means that the first boot after an update will be slower - because ureadahead must regenerate the pack file. - -### Impact of Cached Data - -During boot, various processes may cache data under /var, in order to make the -next time’s startup faster. However, from time to time these cached data may -need to be invalidated (i.e. removed), resulting in a one-time slower boot. -Below is a list of known caches affecting boot time, and a summary of what -causes them to be invalidated. - -* The [ureadahead](#h.4nqngmn6a9r9) pack files - these live under - /var/lib/ureadahead. The auto-update post-install phase invalidates - the data whenever a new update is ready. -* The xkb cache files - these live under /var/lib/xkb. They’re created - as needed during X server startup. They’re removed on the first boot - after any update; this happens in src/install-completed.conf in - src/platform/installer. -* The VPD cache - this collection of files is created by dump_vpd_logs - during basic services startup; see the sources under - src/platform/vpd for more details. The cached data comes from - read-only firmware and never needs to be invalidated. However, after - powerwash, the read-only firmware must be re-read, and this can be - expensive on some platforms. - -### Measuring Performance - -There’s a [web -site](http://www.chromium.org/chromium-os/how-tos-and-troubleshooting/measuring-boot-time-performance) -for this. - -Boot performance is measured by capturing timestamps at specific moments during -the boot flow. These are some of the key events reported by the tools: - -<table> -<tr> -bootstat name keyval name Meaning </tr> -<tr> - -<td>pre-startup</td> - -<td>seconds_kernel_to_startup</td> - -<td>Start of chromeos_startup</td> - -</tr> -<tr> - -<td>post-startup</td> - -<td>seconds_kernel_to_startup_done</td> - -<td>End of chromeos_startup</td> - -</tr> -<tr> - -<td>x-started</td> - -<td>seconds_kernel_to_x_started</td> - -<td>X server startup complete</td> - -</tr> -<tr> - -<td>chrome-exec</td> - -<td>seconds_kernel_to_chrome_exec</td> - -<td>Session manager fork/exec of the Chrome browser process</td> - -</tr> -<tr> - -<td>boot-complete</td> - -<td>seconds_kernel_to_login</td> - -<td>Start of the boot-complete Upstart job</td> - -</tr> -</table> - -All of these events are part of the critical path. - -#### Kernel Performance Accounting Anomalies - -For kernel startup, capturing a time stamp at the exact moment of some key -transitions is inconvenient. The result is that certain events that ought to be -ascribed to one phase (e.g. kernel startup) are ascribed to the adjacent phase: - -* Kernel startup begins with decompression of the kernel, and starting - kernel time accounting. This work takes a few hundred milliseconds, - the bulk which is spent on decompression. This time isn’t recorded - as part of the seconds_kernel_to_login metric; to observe this time, - you have to look at firmware performance numbers. -* Kernel initialization is complete when it hands control to - /sbin/init. However, the userland boot flow can’t mark the start - time until chromeos_startup has mounted /tmp and /sys/kernel/debug. - That work requires a few dozen milliseconds, and is attributed to - kernel startup time. - -### What Matters, What Doesn’t Matter - -Not every change to the boot flow matters to boot performance. Changes may -happen to flows that run in parallel with the critical path without slowing it -down. Even if a change measurably affects boot time, software engineering -considerations such as maintainability or readability may matter more than -performance. Below are guidelines for evaluating whether a change might impact -boot time, and whether the impact is truly significant. - -#### The Critical Path Matters - -[As noted](#h.kb7w2ickc8bc), time added in sequence with the critical path is -time added directly to boot time. When work executes in parallel with the -critical path, the boot time impact will be much smaller. In the best case, work -in parallel will consume resources that would otherwise be idle, meaning boot -time is unaffected. Even if jobs in parallel compete for resources with the -critical path, the impact is typically smaller than the total resource -requirement of the extra work. - -#### Evaluating Significance - -Performance tuning frequently requires trade-offs between a simpler, more -maintainable design, and the best possible performance. Moreover, even when -tuning doesn’t have a maintenance impact, the cost to make and test a code -change may exceed the benefit of the performance improvement. Finally, the boot -time numbers tend to very noisy; small changes are likely not to be -statistically significant. - -Use these rules when evaluating what trade-offs may be necessary for -performance: - -* Unless you’re making firmware changes (or certain kernel changes), - only changes to the seconds_kernel_to_login (a.k.a. boot-complete) - event time matter. The intermediate events are there to diagnose - problems, not as performance metrics. -* Changes less than 100ms (.1s) are too small to matter to a typical - human and too small to be measured reliably. Below this threshold, - the only considerations are the basic software engineering - considerations of maintainable, easy-to-understood source code. -* Changes more than 250ms (.25s) are significant. Above this - threshold, going out of your way to improve performance is - justified. -* In between these two thresholds, exercise careful judgment. - -Note that these guidelines apply equally to improvements and regressions: Just -as 50 ms slower isn’t a regression, 50 ms faster isn’t an improvement. - -## Design Principles - -The principles in this section are meant to improve the maintainability of -packages that deliver Upstart jobs. - -### Put Jobs in the Right Package - -An Upstart job should ideally be associated with a specific service, live in the -source code repository for that service, and be delivered in the same package as -the service. This has multiple advantages: - -* Not all services are required on all platforms/products. Keeping the - job with the service means the job won’t be present in systems where - it doesn’t belong. -* If the service becomes obsolete, the job will go away when the - service’s package is deleted. -* The service and the job frequently change together. Keeping them - together reduces the number of repositories you have to change. -* Having the job live close to the program it starts makes it easier - to read and understand the service as a whole; you don’t have to - consult two source repositories at once. - -### Depend on the Public Events - -For Chrome OS Core, there are four public Upstart events with well-defined, -stable semantics. In order of preference, from most to least preferred, they are -started system-services, started failsafe, started boot-services, and startup. A -package that delivers an Upstart job should depend on the most preferred public -job that will meet its requirements. - -Note that a job should depend on only one of the four events; depending on more -than one event is redundant: - -* started system-services implies started failsafe. -* started failsafe implies started boot-services. - -Other than the four public events, Upstart jobs and events generated by the -chromeos-base/chromeos-init package should be considered private to the package, -and subject to change at any time. If your job depends on a private event, -changes to chromeos-init could cause your job to start at the wrong time, or not -start at all. - -If your package depends on chromeos-base/chromeos-login, it can also depend on -the standard [Chrome startup events](#h.bm3ca2v09tf2). - -If your package delivers more than one job, it is safe and reasonable for some -jobs to depend on others in the same package instead of depending on the public -events. - -Where possible, jobs should not depend jobs delivered by other packages, because -the inter-package coupling can create maintenance hazards. However, if two -packages are already coupled for sound technical reasons, it’s reasonable for -one of the package to depend on the other’s Upstart jobs. For examples, look at -the tree of jobs for starting the network (see init/shill.conf in -src/platform/shill) or cryptohome services (see init/cryptohomed.conf in -src/platform/cryptohome). - -### Depend on Jobs, not Events - -Generally, start and stop conditions for jobs should be based on job events like -started and stopped, rather than starting or stopping jobs with initctl emit, -start, or stop. Depending on a job makes it easier to find the sources of -events, and trace their flow through the source code. - -If you must use initctl emit, start, or stop, follow these guidelines to help -readability: - -* Comments in the source where initctl emit is called should name what - jobs are affected, and how. -* In jobs affected by initctl emit, start, or stop, there should be - comments detailing the callers that do the work. -* There should be comments at the call source and/or job destination - that explain why the problem has to be solved this way. -* Both the caller and the affected job should be in the same source - package. - -### Create Your Own Storage - -If your service requires writable storage (e.g. a directory in /var/lib), your -service is responsible for creating the files and directories. Typically, this -can happen in a pre-start script in the Upstart job for the service. Work to -create this storage should belong to the package that owns the service. The work -should not be handled in chromeos_startup, or an Upstart job not related to your -package. The work must happen at boot: In general, there is no way to initialize -writable storage at build time. - -The objective is that if your package isn’t part of a distribution, it shouldn’t -create vestiges in the file systems of distributions that don’t need it. - -Note that packages can and should rely on directories specified in the Linux -FHS. If a location specified in the standard in the standard is missing, code to -create it should be added to chromeos_startup. - -### Runtime Resource Limits - -Every service should limit their runtime resource usage whenever possible. -Limits are important to help prevent memory or resource leaks in services from -freezing the system. Upstart has directives to control these. - -Here are the important ones: - -* `oom score`: All init scripts need to define an [OOM - score](http://upstart.ubuntu.com/cookbook/#oom-score). See the [CrOS - OOM - document](/chromium-os/chromiumos-design-docs/out-of-memory-handling) - to pick the score. -* `limit as`: Set absolute memory limits. Don't try to pick a tight - limit -- this is meant to catch processes that have run out of - control, so setting a limit that is 5x-10x what is normal is OK. - -## Navigating the Implementation - -### Source Code and ebuilds - -The table below shows the package names and source code repository paths for key -behaviors described in this document. - -<table> -<tr> -Package Source Repo Description </tr> -<tr> -<td> chromeos-base/chromeos-init </td> -<td> src/platform2/init </td> -<td> <a href="#h.n56t7kdl7ejr">Basic boot flow</a> </td> -</tr> -<tr> -<td> chromeos-base/chromeos-login </td> -<td> src/platform2/login_manager </td> -<td> <a href="#h.vmi3dv1ygkj">Chrome startup</a> </td> -</tr> -<tr> -<td> chromeos-base/update_engine </td> -<td> src/platform/update_engine </td> -<td> <a href="#h.pgddr78r6iyt">Rollback support</a> </td> -</tr> -<tr> -<td> chromeos-base/chromeos-installer </td> -<td> src/platform2/installer </td> -<td> <a href="#h.xu4oy15jvst">Install/Recovery</a> </td> -</tr> -<tr> -<td> chromeos-base/chromeos-assets </td> -<td> src/platform/assets </td> -<td> <a href="#h.r32c28uwsj84">Boot message texts</a> </td> -</tr> -</table> - -### Simple Upstart Job Recipes - -#### Simple one-time script - -You need to run a short script once after boot. You don’t care about running the -command if the system application fails at startup. - -Pattern your job after this: - -> `start on started system-services` - -> ` script` - -> ` # … initialization commands ...` - -> ` end script` - -#### Simple daemon - -You need a daemon to start after boot. If the daemon dies it should restart. The -daemon stores working data in a directory under /var/lib. - -Pattern your job after this: - -> `start on started system-services` - -> ` stop on stopping system-services` - -> ` respawn` - -> ` pre-start script` - -> ` mkdir -p /var/lib/my-daemon` - -> ` end script` - -> ` exec my-daemon` - -Many daemons require expect fork or expect daemon in addition to respawn. -Consult your friendly neighborhood Upstart guru for advice. - -#### Failsafe service - -You have a service that’s needed for administrative or debug access to your -device. Your service can start after the system application, so that it won’t -slow down boot. However, if the system applications fails, your service is vital -to connecting to the failed unit in order to repair or debug it. - -Pattern your job after this: - -> `start on started failsafe` - -> `stop on stopping failsafe` - -> `respawn` - -> `exec my-important-administrative-daemon` - -#### Service Required by the System Application - -When your system application starts, it connects to a service provided by a -separate daemon that must be started in its own job. The system application -can’t finish initialization without this second service, and will wait until -it’s ready. - -Pattern your job after this: - -> `start on started boot-services` -> `stop on stopping boot-services` -> `respawn` -> `exec my-important-daemon` - -#### Initialization That Blocks Chrome Startup - -You have system-wide initialization that must finish before Chrome can start. No -other services have a dependency. - -Pattern your job after this: - -> `start on starting ui` - -> `task` - -> `script` - -> ` # perform your important initialization here` - -> `end script` - -#### File System Initialization Required by Multiple Services - -You have a service that provides a shared resource in the file system (e.g. a -directory, a named FIFO, etc.). You need to make sure that the file system -resource is created before any service depending on started boot-services. - -Pattern your job after this: - -> start on starting boot-services - -> `task` - -> `script` - -> ` create-my-important-resource` - -> `end script` - -## FAQ - -### Creating a New Chrome OS Core Platform - -Q: What Upstart jobs must my platform supply? - -A: The platform must supply a boot-complete job conforming to certain basic -requirements. - -Q: What are the requirements for the boot-complete job? - -A: The job must have a start stanza that will start once the system application -is up and providing service. Your platform can define “providing service” any -way it wants, but generally the job shouldn’t start until these criteria are -met: - -* Performance critical startup is complete. -* An ordinary user will perceive the device as fully functional. - -For reference, on Chrome OS, after stripping out comments and boilerplate, the -boot-complete job consists of just this one line: - -start on login-prompt-visible - -Q: How do I provide a boot-complete.conf for my board? - -A: boot-complete.conf is provided by virtual/chromeos-bootcomplete. You can -override this virtual in your board overlay. - -Q: How do I disable/enable the [encrypted stateful](#h.y4vr1c4hxx6o) feature? - -A: The feature is enabled by default. To disable the feature, set -USE=”-encrypted_stateful”. - -### Adding a New Service - -Q: What repository should hold my Upstart job? - -A: Ideally, the job should live in the source repository for the service it -starts. If there’s no such repository (e.g. your service is an upstream -package), create a new package to install the job. For an example, see the -chromeos-base/openssh-server-init package. - -Q: When should I use start on started system-services? - -A: This is the preferred way to start any job that should run once at boot time. -However, if your job is important to starting the system application, you may -need to depend on boot-services instead. - -Q: When should I use start on started failsafe? - -A: Use this for a job that can start after the system application, but that must -run eventually even if the system application never starts. This start condition -is most useful for services that are needed to debug or recover a failed unit. - -Q: When should I use start on started boot-services? - -A: Use this when the job must start in parallel with the system application. -This is typically required for one of these reasons: - -* The job is required before the system application can provide - service, or -* Until the job runs, the user may perceive the system as not - functional. - -Note that by running in parallel with the system application, the job may slow -down system boot. - -Q: Can my service just use start on startup? - -A: Most services should avoid doing this, because of the restrictions it -imposes: - -* There is no writable storage: directories like /tmp, /home and /var - will not be mounted. -* System logging is not available; the only way to debug is to write - messages to a TTY device. -* Not all devices will be available under /dev; no udev rules for any - devices will have run. - -## References - -Upstart reference: <http://upstart.ubuntu.com/cookbook/> - -The ureadahead man page: -<http://manpages.ubuntu.com/manpages/precise/man8/ureadahead.8.html>
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cbi-cros-board-info/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cbi-cros-board-info/index.md deleted file mode 100644 index 59e46419809..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cbi-cros-board-info/index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: cbi-cros-board-info -title: 'CBI: CrOS Board Info' ---- - -# This page has been moved to [here](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/design_docs/cros_board_info.md).
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-battery-life-overview/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-battery-life-overview/index.md deleted file mode 100644 index 20cd8870e2c..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-battery-life-overview/index.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: chrome-os-battery-life-overview -title: Chrome OS Battery Life Overview ---- - -We design Chromium OS with the philosophy that devices provide users “all-day -battery life.” Because the notion of all-day battery life can be subject to -interpretation, we use the philosophy to define a usage model for -battery-powered devices. This model helps us define user expectations, and -provide battery recommendations for Chromium OS devices. - -The Battery Usage Model - -We don’t have a say in how users use their device. Users can and will use their -devices in diverse situations, with variable access to charging power. Moreover, -as devices and their uses change, the power requirements for systems evolve. As -such, Chromium OS provides tests and diagnostics which allow both users and -device manufacturers to ensure that they can achieve our notion of all-day -battery life. - -Active use and standby-time are different. While some devices, notably mobile -phones, blend battery life usage between active and standby states, we believe -these states are separate, important elements of all-day battery life. We define -active use as situations in which a user is “working away from the plug.” We -define standby time as situations in which the user is “idle away from the -plug.” We expect a user’s typical day to include periods of both, but that the -blend of states may vary day to day. - -Typical active-use should leave reasonable standby available. Importantly, -adhering to our notion of all-day battery life means that a user should be able -to actively work for many hours and still have sufficient standby time to reach -a power source and recharge the device. Our belief is that standby power -consumption should be at least an order of magnitude less that active-use power -consumption. Thus, if a device has been used for 80% of active-use time, the -remaining 20% should translate into a substantial amount of standby time in -which the device can be recharged without forced shutdown. - -Shutdown provides longevity beyond typical use. While a device is shutdown, it -can conserve most of its battery life. While our notion of all-day battery life -does not assume shutdowns, we believe that the fast-booting nature of Chromium -OS should make it easy for users to strongly preserve battery life when they -desire. - -A Typical Use Case - -Our philosophy, and its derived battery usage model, is well illustrated by an -example. Consider a typical Chromium OS user, whose device achieves the average -10 hours of active-use battery life. Our usage model expects that this user: - - Can use their device in active mode for an 8-hour workday, and have 20% of - battery remaining. - - The remaining 20% of battery life should translate into more than a day of - standby time; roughly 30 hours. - -Battery Recommendations - -Given the philosophy and usage model, Chromium OS recommends that device -batteries be sized such that an average of 10 active hours can be achieved on -the device before recharging. Thus, if a device consumes 4 Watts for each hour -of active time, a 40 Watt-Hour battery is recommended.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-performance-overview/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-performance-overview/index.md deleted file mode 100644 index 7207b1c32dd..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-performance-overview/index.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: chrome-os-performance-overview -title: Chrome OS Performance Philosophy ---- - -## Speed is core to Chrome OS, as it is to [Chrome](/developers/core-principles). Speed is one of the most compelling features of Chrome OS - we get out of the way as quickly as possible so users can be productive and enjoy their content. Beyond Chrome’s commitment to make the fastest browser, Chrome OS pays attention to the speed of all user interactions, ranging from I/O devices like the keyboard and screen, to connectivity components such as WiFi. - -We measure the speed of user interactions along various dimensions, with the -goal that Chromebooks should always feel responsive. We ask ourselves questions -like: - -How long does it take to boot the device? -How quickly do websites load? - -Are the graphics smooth when browsing the web or playing games? - -Does the screen behave correctly when the device changes modes? - -Do performance improvements in Chrome manifest in Chrome OS? - -We collect these metrics for all Chromebooks over their entire supported -lifecycle to ensure the user experience does not deteriorate. Every year, we -review our performance targets and we often raise our bar. Many of our devices -actually get faster over time, even as we add new features like Android apps.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-volume-keys/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-volume-keys/index.md deleted file mode 100644 index 8caa1a361a3..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chrome-os-volume-keys/index.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: chrome-os-volume-keys -title: Chrome OS Volume Keys ---- - -**# **## Current, intended behavior (since mid-2011)**** - -**When the user presses one of the three volume-related keys (Mute, Volume Down, -or Volume Up), a bubble appears in the lower-right corner of the screen -displaying the current volume level.** - -**<img alt="image" src="https://lh5.googleusercontent.com/Nm9_2BQJ5T3Sqt3rD6ZYejyox3OcpxsoA61mc8NM9Aug_19_i4bnXDmZtO3BA0uFIXMXfBkCoMVvpdheGIIxHjwZpUWz1wAp811TBxJ_Ry79bd1X6hQ" height=116px; width=330px;>** - -**Non-muted and ~60% of maximum level** - -**At the left, an icon shows whether the system is muted or not, and if not -muted, a coarse approximation of the current volume level (via zero, one, or two -sound waves emanating from a speaker). To the right of the icon, a toggle button -displays the current muting status.** - -**<img alt="image" src="https://lh3.googleusercontent.com/aFZMhLBlyas-l9XSOCg8W-KMonwPhnf4J9pt0iK9MbVaBnlRANM0_gck1ZsDH9vLHZkoZ5xu842ZviGA84WVzhj5BuGU__9LWai_R9g5KekYmB6sNDs" height=116px; width=330px;>** - -**Muted with ~60% level saved** - -**Either the icon or the toggle button can be clicked to toggle the muting status. At the right side of the bubble, a horizontal slider displays the current volume level. If the system is muted, the slider is displayed in an inactive state; note that the previously-set volume level is still visible, though.** -**The same controls are present in the larger bubble that is displayed when the -user clicks the system / status tray in the lower-right corner of the screen:** - -**<img alt="image" src="https://lh4.googleusercontent.com/ibZ7w_Ljo-MURsOQhV0Dx-XZPpdDU5fLEKNKSOSLkh6BlyyiTvsk94XiNWQfLpVSpYLemRluuqywKIFvTKJDB_uUbLyl381r2Zby4kUUkkkpzSrdRJ8" height=362px; width=324px;>** -**The volume keys’ current behavior was initially proposed in [chromium-os bug 13618](https://code.google.com/p/chromium-os/issues/detail?id=13618). Briefly,** - -**### When not muted:** - -**Mute: mutes instantly** -**Volume Down: decreases the volume** -**Volume Up: increases the volume** - -**### When muted:** - -**Mute: does nothing** -**Volume Down: sets the saved volume to zero, keeping the system muted** -**Volume Up: unmutes and restores saved volume (increasing it slightly if zero)** - -**## Objections** - -**### Many other operating systems, A/V equipment, etc. provide a “toggle mute” button instead.** - -**When in an environment where sound is undesirable (e.g. library, shared office, etc.), the user is guaranteed that pressing the Mute key will instantly mute the system; there is no risk of toggling sound back on if the system was already muted. If the user wants to set the volume to the lowest audible level, only three key-presses are needed (Mute, Volume Down, and then Volume Up).** - -**### But the user can just hold the Volume Down key if they want to make sure the volume is at 0%.** - -**This is a poor substitute; decreasing the volume from 100% to 0% takes more than a second and the user is unable to easily restore the original volume level later.** - -**### The toggle button between the icon and the slider is confusing (see e.g. [this bug](https://code.google.com/p/chromium/issues/detail?id=170935)).** - -**The button was originally labeled, but the label’s visual appearance was sub-optimal ([chromium bug 143426](https://code.google.com/p/chromium/issues/detail?id=143426)). Per [chromium bug 152070](https://code.google.com/p/chromium/issues/detail?id=152070), the unlabeled button will be merged into a more-meaningful icon (as tracked in [chromium bug 137947](https://code.google.com/p/chromium/issues/detail?id=137947)).**
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-cgroups/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-cgroups/index.md deleted file mode 100644 index 959a14eef38..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-cgroups/index.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: chromium-os-cgroups -title: Chromium OS Cgroups ---- - -All cgroups are optional, but highly-recommended and if present will be used by -Chrome and Chrome OS - -top-level cgroups created by /etc/init/cgroups.conf - -* cpu - Uses CPU time accounting to set limits on certain tasks - * chrome_renderers - Set by Chrome code in <fixme> - * session_manager_container - set by session manager in - <fixme> - * update-engine - set by update-engine in <fixme> -* freezer - * group of threads which stop running when we suspend or resume - * this limits the amount of work that happens at resume - * done by <fixme> -* devices - * limits the devices available to a process - * done by <fixme> -* cpuacct - * keeps track of resources consumed by a given group - -There are also cgroups which are board-specific. - -* cpuset - -For big.LITTLE systems with different types of CPU cores, we set up cpusets for -background tasks called cpuset/chrome/non-urgent/ which will be pinned to little -CPUs and another for performance-sensitive tasks called cpuset/chrome/urgent/ - -It's up to Chrome to put the correct tasks (threads) into the {non-,}urgent -group as appropriate.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/dco/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/dco/index.md deleted file mode 100644 index 35406d7403a..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/dco/index.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -- - /chromium-os/chromiumos-design-docs/chromium-os-kernel - - Kernel Design -page_name: dco -title: Documented Certificate of Ownership ---- - -The DCO is the standard release for Linux kernel code, and is required to submit -all Linux kernel code upstream, and hence to Google. - -> `By making a contribution to this project, I certify that: ` -> ` (a) The contribution was created in whole or in part by me and I have the -> right to submit it under the open source license indicated in the file; or` -> ` (b) The contribution is based upon previous work that, to the best of my -> knowledge, is covered under an appropriate open source license and I have the -> right under that license to submit that work with modifications, whether -> created in whole or in part by me, under the same open source license (unless -> I am permitted to submit under a different license), as indicated in the file; -> or` -> ` (c) The contribution was provided directly to me by some other person who -> certified (a), (b) or (c) and I have not modified it; and` -> ` (d) In the case of each of (a), (b), or (c), I understand and agree that -> this project and the contribution are public and that a record of the -> contribution (including all personal information I submit with it, including -> my sign-off) is maintained indefinitely and may be redistributed consistent -> with this project or the open source license indicated in the file.`
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/index.md deleted file mode 100644 index b231c75b413..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/index.md +++ /dev/null @@ -1,191 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: chromium-os-kernel -title: Kernel Design ---- - -[TOC] - -## Abstract - -This document outlines the overall design of the kernel, as it relates to -Chromium OS. - -See also the [Frequently Asked Questions -page](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/kernel_faq.md) -for more day-to-day details. - -## Background - -Chromium OS uses the Linux Kernel. Historically we stayed on a 2.6.32 -Ubuntu-based for the first several releases, but have since then moved on to -track the upstream mainline kernel directly, applying our changes for the -features and stability we need on top of it. - -## Source code - -### Git repository - -The Chromium OS Linux kernel will be stored in a gerrit/git repository, hosted -on an externally accessible website. All platforms will build from **one** -single kernel source tree (though there will be multiple binary images). - -If necessary, we will create short-term private branches for specific vendor -projects that involve pre-production hardware. These branches will be merged -back into the main kernel as soon as possible, using one code review per CL so -that all code in our master branch is either from upstream or code reviewed. The -intent of these private branches is to enable code review and project status -monitoring of code that cannot be publicly released yet, due to NDA. The intent -is specifically **not** to use these branches to keep changes that are shipping -to customers. - -Any code submitted should be done as one Git commit per logical change, with a -good description. (See -[Documentation/SubmittingPatches](/chromium-os/chromiumos-design-docs/chromium-os-kernel/submitting-patches) -in the kernel tree.) We may create temporary public branches for the purpose of -working together on code review and integration. - -### Licensing - -All Linux kernel code, from Google and from partners, must be released under -GPLv2 and must be released under the [DCO (documented certificate of -ownership)](/chromium-os/chromiumos-design-docs/chromium-os-kernel/dco) as a -signoff of ownership. Each commit will contain the signoff of the code -author/committer and the ACK of the patch approver. Code should be submitted -under the Chromium contributor license agreement. - -### Changelog entries - -Here's an example of a changelog entry: - -```none -CHROMIUM: bibble: a patch to fix everything -Bug: 12345 -This patch fixes bug 12345 by checking for a NULL pointer before dereferencing ops->thingy -Signed-off-by: Author <author@example.net> -Signed-off-by: Committer <committer@example.net> -Acked-by: Approver <approver@example.net> -``` - -All kernel code (including backports from upstream) contributed by Google or -partners will include a classification tag in the first line of the commit log. -The classification will be useful when the maintainer needs to rebase to a newer -kernel; the tag is also needed for proper attribution. The tag is in -ALL_CAPS_UNDERSCORE, is followed by a colon, and is at the beginning of the -first line. The first line from each commit should be a summary. - -Whenever possible use the -x flag with git cherry-pick. This appends a line to -the commit message that says which commit this cherry-pick came from. Only do -this in the case that the commit was cherry-picked from a repository that is -easily accessible by the public. - -Code contributed by the Chromium community will use the tag CHROMIUM. For -example: - -```none -CHROMIUM: bibble: a patch to fix everything -``` - -Code backported from upstream (Linus's tree) will use the tag UPSTREAM. For -example: - -```none -UPSTREAM: bibble: a patch to fix everything -``` - -Code backported from an upstream maintainer tree will use two tags. For example: - -```none -UPSTREAM: WIRELESS: bibble: a patch to fix everything -``` - -Code ported from a Linux distribution tree or other non-upstream tree will also -use an appropriate tag. For example: - -```none -UBUNTU: bibble: a patch to fix everything -``` - -*or* - -```none -YOCTO: bibble: a patch to fix everything -``` - -Code ported from a patch will use the tag FROMLIST and should include a link to -the list the patch was obtained from. For example: - -```none -FROMLIST: bibble: a patch to fix everything -``` - -```none -``` - -```none -(am from https://patchwork.kernel.org/patch/0987654/) -``` - -Code backported that you had to change to make it run with an older kernel -version, including conflict resolutions, will use the tag BACKPORT. For example: - -```none -BACKPORT: bibble: a patch to fix everything -``` - -### Upgrades - -The kernel will be upgraded to a new version as soon as practical after a new -version of the upstream kernel is released. We will do this via a Git rebase, -this means we'll keep clean versions of our patches floated on top of the latest -tree. In practice, this will happen approximately every 3-6 months, and -approximately every other kernel versions from upstream. Other, smaller updates -will be done on a continuous basis, such as merging in the -stable kernel -updates. - -All third-party vendors are expected to supply updated versions of their code -against every new mainline kernel version (for example, at 3.2, 3.3, and so on) -within 14 days of its release, if the code is not already upstream. - -## Supported and unsupported features - -### Kernel modules and initial RAM disks - -We will support kernel modules, though this feature may be removed at some point -in the future. We will **not** support modules built outside of the main kernel -tree. - -We will not support initial RAM disks (initrd) for the general kernel, but will -need them on recovery image kernels as well as for the factory install flow. - -### Architectures - -Support for i386, ARMv7 and x86_64 is planned. - -The tradeoff for using 64 bits is additional power consumption and memory usage, -in return for greater performance. - -### Swap - -We do not plan to support swap in our initial release, but will conduct a -further feasibility investigation to review this. We are concerned about the -effect of swap on: - -* SSD write cycle lifetime -* maximum latency - -We realize that not having swap will limit how many tabs a user can keep open -and the amount of anonymous memory a process can allocate. Aggregate size of -anonymous memory in the system will be limited to the size of RAM. We may -revisit this decision in future releases. - -## Configuration files - -We use a custom split config to help keep settings unified across different -architectures. See the [Kernel Configuration -document](/chromium-os/how-tos-and-troubleshooting/kernel-configuration) for -more details.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/submitting-patches/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/submitting-patches/index.md deleted file mode 100644 index 6ed08844cfa..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-kernel/submitting-patches/index.md +++ /dev/null @@ -1,462 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -- - /chromium-os/chromiumos-design-docs/chromium-os-kernel - - Kernel Design -page_name: submitting-patches -title: Submitting Patches ---- - -[TOC] - -## How to Get Your Change Into the Linux Kernel - -or - -## Care And Operation Of Your Linus Torvalds - -For a person or company who wishes to submit a change to the Linux kernel, the -process can sometimes be daunting if you're not familiar with "the system." This -text is a collection of suggestions which can greatly increase the chances of -your change being accepted. Read Documentation/SubmitChecklist for a list of -items to check before submitting code. If you are submitting a driver, also read -Documentation/SubmittingDrivers. -------------------------------------------- - -### SECTION 1 - CREATING AND SENDING YOUR CHANGE - --------------------------------------------- - -#### 1) "diff -up" - ------------- Use "diff -up" or "diff -uprN" to create patches. All changes to -the Linux kernel occur in the form of patches, as generated by diff(1). When -creating your patch, make sure to create it in "unified diff" format, as -supplied by the '-u' argument to diff(1). Also, please use the '-p' argument -which shows which C function each change is in - that makes the resultant diff a -lot easier to read. Patches should be based in the root kernel source directory, -not in any lower subdirectory. To create a patch for a single file, it is often -sufficient to do: SRCTREE= linux-2.6 MYFILE= drivers/net/mydriver.c cd -$SRCTREE cp $MYFILE $MYFILE.orig vi $MYFILE # make your change -cd .. diff -up $SRCTREE/$MYFILE{.orig,} > /tmp/patch To create a patch for -multiple files, you should unpack a "vanilla", or unmodified kernel source tree, -and generate a diff against your own source tree. For example: MYSRC= -/devel/linux-2.6 tar xvfz linux-2.6.12.tar.gz mv linux-2.6.12 -linux-2.6.12-vanilla diff -uprN -X -linux-2.6.12-vanilla/Documentation/dontdiff \\ linux-2.6.12-vanilla -$MYSRC > /tmp/patch "dontdiff" is a list of files which are generated by the -kernel during the build process, and should be ignored in any diff(1)-generated -patch. The "dontdiff" file is included in the kernel tree in 2.6.12 and later. -Make sure your patch does not include any extra files which do not belong in a -patch submission. Make sure to review your patch -after- generated it with -diff(1), to ensure accuracy. If your changes produce a lot of deltas, you may -want to look into splitting them into individual patches which modify things in -logical stages. This will facilitate easier reviewing by other kernel -developers, very important if you want your patch accepted. There are a number -of scripts which can aid in this: Quilt: -http://savannah.nongnu.org/projects/quilt Andrew Morton's patch scripts: -http://userweb.kernel.org/~akpm/stuff/patch-scripts.tar.gz Instead of these -scripts, quilt is the recommended patch management tool (see above). - -#### 2) Describe your changes. - -Describe the technical detail of the change(s) your patch includes. Be as -specific as possible. The WORST descriptions possible include things like -"update driver X", "bug fix for driver X", or "this patch includes updates for -subsystem X. Please apply." The maintainer will thank you if you write your -patch description in a form which can be easily pulled into Linux's source code -management system, git, as a "commit log". See #15, below. If your description -starts to get long, that's a sign that you probably need to split up your patch. -See #3, next. When you submit or resubmit a patch or patch series, include the -complete patch description and justification for it. Don't just say that this is -version N of the patch (series). Don't expect the patch merger to refer back to -earlier patch versions or referenced URLs to find the patch description and put -that into the patch. I.e., the patch (series) and its description should be -self-contained. This benefits both the patch merger(s) and reviewers. Some -reviewers probably didn't even receive earlier versions of the patch. If the -patch fixes a logged bug entry, refer to that bug entry by number and URL. If -you want to refer to a specific commit, don't just refer to the SHA-1 ID of the -commit. Please also include the oneline summary of the commit, to make it easier -for reviewers to know what it is about. Example: Commit -e21d2170f36602ae2708 ("video: remove unnecessary -platform_set_drvdata()") removed the unnecessary platform_set_drvdata(), -but left the variable "dev" unused, delete it. - -#### 3) Separate your changes. - -Separate _logical changes_ into a single patch file. For example, if your -changes include both bug fixes and performance enhancements for a single driver, -separate those changes into two or more patches. If your changes include an API -update, and a new driver which uses that new API, separate those into two -patches. On the other hand, if you make a single change to numerous files, group -those changes into a single patch. Thus a single logical change is contained -within a single patch. If one patch depends on another patch in order for a -change to be complete, that is OK. Simply note "this patch depends on patch X" -in your patch description. If you cannot condense your patch set into a smaller -set of patches, then only post say 15 or so at a time and wait for review and -integration. - -#### 4) Style check your changes. - -Check your patch for basic style violations, details of which can be found in -Documentation/CodingStyle. Failure to do so simply wastes the reviewers time and -will get your patch rejected, probably without even being read. At a minimum you -should check your patches with the patch style checker prior to submission -(scripts/checkpatch.pl). You should be able to justify all violations that -remain in your patch. - -#### 5) Select e-mail destination. - -Look through the MAINTAINERS file and the source code, and determine if your -change applies to a specific subsystem of the kernel, with an assigned -maintainer. If so, e-mail that person. The script scripts/get_maintainer.pl can -be very useful at this step. If no maintainer is listed, or the maintainer does -not respond, send your patch to the primary Linux kernel developer's mailing -list, linux-kernel@vger.kernel.org. Most kernel developers monitor this e-mail -list, and can comment on your changes. Do not send more than 15 patches at once -to the vger mailing lists!!! Linus Torvalds is the final arbiter of all changes -accepted into the Linux kernel. His e-mail address is . He gets a lot of e-mail, -so typically you should do your best to -avoid- sending him e-mail. Patches -which are bug fixes, are "obvious" changes, or similarly require little -discussion should be sent or CC'd to Linus. Patches which require discussion or -do not have a clear advantage should usually be sent first to linux-kernel. Only -after the patch is discussed should the patch then be submitted to Linus. - -#### 6) Select your CC (e-mail carbon copy) list. - -Unless you have a reason NOT to do so, CC linux-kernel@vger.kernel.org. Other -kernel developers besides Linus need to be aware of your change, so that they -may comment on it and offer code review and suggestions. linux-kernel is the -primary Linux kernel developer mailing list. Other mailing lists are available -for specific subsystems, such as USB, framebuffer devices, the VFS, the SCSI -subsystem, etc. See the MAINTAINERS file for a mailing list that relates -specifically to your change. Majordomo lists of VGER.KERNEL.ORG at: If -changes affect userland-kernel interfaces, please send the MAN-PAGES maintainer -(as listed in the MAINTAINERS file) a man-pages patch, or at least a -notification of the change, so that some information makes its way into the -manual pages. Even if the maintainer did not respond in step #5, make sure to -ALWAYS copy the maintainer when you change their code. For small patches you may -want to CC the Trivial Patch Monkey trivial@kernel.org which collects "trivial" -patches. Have a look into the MAINTAINERS file for its current manager. Trivial -patches must qualify for one of the following rules: Spelling fixes in -documentation Spelling fixes which could break grep(1) Warning fixes (cluttering -with useless warnings is bad) Compilation fixes (only if they are actually -correct) Runtime fixes (only if they actually fix things) Removing use of -deprecated functions/macros (eg. check_region) Contact detail and documentation -fixes Non-portable code replaced by portable code (even in arch-specific, since -people copy, as long as it's trivial) Any fix by the author/maintainer of the -file (ie. patch monkey in re-transmission mode) - -#### 7) No MIME, no links, no compression, no attachments. Just plain text. - -Linus and other kernel developers need to be able to read and comment on the -changes you are submitting. It is important for a kernel developer to be able to -"quote" your changes, using standard e-mail tools, so that they may comment on -specific portions of your code. For this reason, all patches should be -submitting e-mail "inline". WARNING: Be wary of your editor's word-wrap -corrupting your patch, if you choose to cut-n-paste your patch. Do not attach -the patch as a MIME attachment, compressed or not. Many popular e-mail -applications will not always transmit a MIME attachment as plain text, making it -impossible to comment on your code. A MIME attachment also takes Linus a bit -more time to process, decreasing the likelihood of your MIME-attached change -being accepted. Exception: If your mailer is mangling patches then someone may -ask you to re-send them using MIME. See Documentation/email-clients.txt for -hints about configuring your e-mail client so that it sends your patches -untouched. - -#### 8) E-mail size. - -When sending patches to Linus, always follow step #7. Large changes are not -appropriate for mailing lists, and some maintainers. If your patch, -uncompressed, exceeds 300 kB in size, it is preferred that you store your patch -on an Internet-accessible server, and provide instead a URL (link) pointing to -your patch. - -#### 9) Name your kernel version. - -It is important to note, either in the subject line or in the patch description, -the kernel version to which this patch applies. If the patch does not apply -cleanly to the latest kernel version, Linus will not apply it. - -#### 10) Don't get discouraged. Re-submit. - -After you have submitted your change, be patient and wait. If Linus likes your -change and applies it, it will appear in the next version of the kernel that he -releases. However, if your change doesn't appear in the next version of the -kernel, there could be any number of reasons. It's YOUR job to narrow down those -reasons, correct what was wrong, and submit your updated change. It is quite -common for Linus to "drop" your patch without comment. That's the nature of the -system. If he drops your patch, it could be due to \* Your patch did not apply -cleanly to the latest kernel version. \* Your patch was not sufficiently -discussed on linux-kernel. \* A style issue (see section 2). \* An e-mail -formatting issue (re-read this section). \* A technical problem with your -change. \* He gets tons of e-mail, and yours got lost in the shuffle. \* You are -being annoying. When in doubt, solicit comments on linux-kernel mailing list. - -#### 11) Include PATCH in the subject - -Due to high e-mail traffic to Linus, and to linux-kernel, it is common -convention to prefix your subject line with \[PATCH\]. This lets Linus and other -kernel developers more easily distinguish patches from other e-mail discussions. - -#### 12) Sign your work - -To improve tracking of who did what, especially with patches that can percolate -to their final resting place in the kernel through several layers of -maintainers, we've introduced a "sign-off" procedure on patches that are being -emailed around. The sign-off is a simple line at the end of the explanation for -the patch, which certifies that you wrote it or otherwise have the right to pass -it on as an open-source patch. The rules are pretty simple: if you can certify -the below: Developer's Certificate of Origin 1.1 By making a contribution to -this project, I certify that: (a) The contribution was created in whole or in -part by me and I have the right to submit it under the open source license -indicated in the file; or (b) The contribution is based upon previous work that, -to the best of my knowledge, is covered under an appropriate open source license -and I have the right under that license to submit that work with modifications, -whether created in whole or in part by me, under the same open source license -(unless I am permitted to submit under a different license), as indicated in the -file; or (c) The contribution was provided directly to me by some other person -who certified (a), (b) or (c) and I have not modified it. (d) I understand -and agree that this project and the contribution are public and that a -record of the contribution (including all personal information I submit -with it, including my sign-off) is maintained indefinitely and may be -redistributed consistent with this project or the open source license(s) -involved. then you just add a line saying Signed-off-by: Random J Developer -using your real name (sorry, no pseudonyms or anonymous contributions.) Some -people also put extra tags at the end. They'll just be ignored for now, but you -can do this to mark internal company procedures or just point out some special -detail about the sign-off. If you are a subsystem or branch maintainer, -sometimes you need to slightly modify patches you receive in order to merge -them, because the code is not exactly the same in your tree and the submitters'. -If you stick strictly to rule (c), you should ask the submitter to rediff, but -this is a totally counter-productive waste of time and energy. Rule (b) allows -you to adjust the code, but then it is very impolite to change one submitter's -code and make him endorse your bugs. To solve this problem, it is recommended -that you add a line between the last Signed-off-by header and yours, indicating -the nature of your changes. While there is nothing mandatory about this, it -seems like prepending the description with your mail and/or name, all enclosed -in square brackets, is noticeable enough to make it obvious that you are -responsible for last-minute changes. Example : Signed-off-by: Random J -Developer \[lucky@maintainer.example.org: struct foo moved from foo.c to -foo.h\] Signed-off-by: Lucky K Maintainer This practise is particularly helpful -if you maintain a stable branch and want at the same time to credit the author, -track changes, merge the fix, and protect the submitter from complaints. Note -that under no circumstances can you change the author's identity (the From -header), as it is the one which appears in the changelog. Special note to -back-porters: It seems to be a common and useful practise to insert an -indication of the origin of a patch at the top of the commit message (just after -the subject line) to facilitate tracking. For instance, here's what we see in -2.6-stable : Date: Tue May 13 19:10:30 2008 +0000 SCSI: libiscsi regression in -2.6.25: fix nop timer handling commit 4cf1043593db6a337f10e006c23c69e5fc93e722 -upstream And here's what appears in 2.4 : Date: Tue May 13 22:12:27 2008 +0200 -wireless, airo: waitbusy() won't delay \[backport of 2.6 commit -b7acbdfbd1f277c1eb23f344f899cfa4cd0bf36a\] Whatever the format, this information -provides a valuable help to people tracking your trees, and to people trying to -trouble-shoot bugs in your tree. - -#### 13) When to use Acked-by: and Cc: - -The Signed-off-by: tag indicates that the signer was involved in the development -of the patch, or that he/she was in the patch's delivery path. If a person was -not directly involved in the preparation or handling of a patch but wishes to -signify and record their approval of it then they can arrange to have an -Acked-by: line added to the patch's changelog. Acked-by: is often used by the -maintainer of the affected code when that maintainer neither contributed to nor -forwarded the patch. Acked-by: is not as formal as Signed-off-by:. It is a -record that the acker has at least reviewed the patch and has indicated -acceptance. Hence patch mergers will sometimes manually convert an acker's "yep, -looks good to me" into an Acked-by:. Acked-by: does not necessarily indicate -acknowledgement of the entire patch. For example, if a patch affects multiple -subsystems and has an Acked-by: from one subsystem maintainer then this usually -indicates acknowledgement of just the part which affects that maintainer's code. -Judgement should be used here. When in doubt people should refer to the original -discussion in the mailing list archives. If a person has had the opportunity to -comment on a patch, but has not provided such comments, you may optionally add a -"Cc:" tag to the patch. This is the only tag which might be added without an -explicit action by the person it names. This tag documents that potentially -interested parties have been included in the discussion - -#### 14) Using Reported-by:, Tested-by:, Reviewed-by: and Suggested-by: - -If this patch fixes a problem reported by somebody else, consider adding a -Reported-by: tag to credit the reporter for their contribution. Please note that -this tag should not be added without the reporter's permission, especially if -the problem was not reported in a public forum. That said, if we diligently -credit our bug reporters, they will, hopefully, be inspired to help us again in -the future. A Tested-by: tag indicates that the patch has been successfully -tested (in some environment) by the person named. This tag informs maintainers -that some testing has been performed, provides a means to locate testers for -future patches, and ensures credit for the testers. Reviewed-by:, instead, -indicates that the patch has been reviewed and found acceptable according to the -Reviewer's Statement: Reviewer's statement of oversight By -offering my Reviewed-by: tag, I state that: (a) I have carried out a technical -review of this patch to evaluate its appropriateness and readiness for -inclusion into the mainline kernel. (b) Any problems, concerns, or -questions relating to the patch have been communicated back to the -submitter. I am satisfied with the submitter's response to my comments. (c) -While there may be things that could be improved with this submission, -I believe that it is, at this time, (1) a worthwhile modification to the -kernel, and (2) free of known issues which would argue against its inclusion. -(d) While I have reviewed the patch and believe it to be sound, I do not -(unless explicitly stated elsewhere) make any warranties or guarantees that -it will achieve its stated purpose or function properly in any given -situation. A Reviewed-by tag is a statement of opinion that the patch is an -appropriate modification of the kernel without any remaining serious technical -issues. Any interested reviewer (who has done the work) can offer a Reviewed-by -tag for a patch. This tag serves to give credit to reviewers and to inform -maintainers of the degree of review which has been done on the patch. -Reviewed-by: tags, when supplied by reviewers known to understand the subject -area and to perform thorough reviews, will normally increase the likelihood of -your patch getting into the kernel. A Suggested-by: tag indicates that the patch -idea is suggested by the person named and ensures credit to the person for the -idea. Please note that this tag should not be added without the reporter's -permission, especially if the idea was not posted in a public forum. That said, -if we diligently credit our idea reporters, they will, hopefully, be inspired to -help us again in the future. - -#### 15) The canonical patch format - -The canonical patch subject line is: Subject: \[PATCH 001/123\] subsystem: -summary phrase The canonical patch message body contains the following: - A -"from" line specifying the patch author. - An empty line. - The body of the -explanation, which will be copied to the permanent changelog to describe this -patch. - The "Signed-off-by:" lines, described above, which will also go in the -changelog. - A marker line containing simply "---". - Any additional comments -not suitable for the changelog. - The actual patch (diff output). The Subject -line format makes it very easy to sort the emails alphabetically by subject line -- pretty much any email reader will support that - since because the sequence -number is zero-padded, the numerical and alphabetic sort is the same. The -"subsystem" in the email's Subject should identify which area or subsystem of -the kernel is being patched. The "summary phrase" in the email's Subject should -concisely describe the patch which that email contains. The "summary phrase" -should not be a filename. Do not use the same "summary phrase" for every patch -in a whole patch series (where a "patch series" is an ordered sequence of -multiple, related patches). Bear in mind that the "summary phrase" of your email -becomes a globally-unique identifier for that patch. It propagates all the way -into the git changelog. The "summary phrase" may later be used in developer -discussions which refer to the patch. People will want to google for the -"summary phrase" to read discussion regarding that patch. It will also be the -only thing that people may quickly see when, two or three months later, they are -going through perhaps thousands of patches using tools such as "gitk" or "git -log --oneline". For these reasons, the "summary" must be no more than 70-75 -characters, and it must describe both what the patch changes, as well as why the -patch might be necessary. It is challenging to be both succinct and descriptive, -but that is what a well-written summary should do. The "summary phrase" may be -prefixed by tags enclosed in square brackets: "Subject: \[PATCH tag\] ". The -tags are not considered part of the summary phrase, but describe how the patch -should be treated. Common tags might include a version descriptor if the -multiple versions of the patch have been sent out in response to comments (i.e., -"v1, v2, v3"), or "RFC" to indicate a request for comments. If there are four -patches in a patch series the individual patches may be numbered like this: 1/4, -2/4, 3/4, 4/4. This assures that developers understand the order in which the -patches should be applied and that they have reviewed or applied all of the -patches in the patch series. A couple of example Subjects: Subject: \[patch -2/5\] ext2: improve scalability of bitmap searching Subject: \[PATCHv2 001/207\] -x86: fix eflags tracking The "from" line must be the very first line in the -message body, and has the form: From: Original Author The "from" line specifies -who will be credited as the author of the patch in the permanent changelog. If -the "from" line is missing, then the "From:" line from the email header will be -used to determine the patch author in the changelog. The explanation body will -be committed to the permanent source changelog, so should make sense to a -competent reader who has long since forgotten the immediate details of the -discussion that might have led to this patch. Including symptoms of the failure -which the patch addresses (kernel log messages, oops messages, etc.) is -especially useful for people who might be searching the commit logs looking for -the applicable patch. If a patch fixes a compile failure, it may not be -necessary to include _all_ of the compile failures; just enough that it is -likely that someone searching for the patch can find it. As in the "summary -phrase", it is important to be both succinct as well as descriptive. The "---" -marker line serves the essential purpose of marking for patch handling tools -where the changelog message ends. One good use for the additional comments after -the "---" marker is for a diffstat, to show what files have changed, and the -number of inserted and deleted lines per file. A diffstat is especially useful -on bigger patches. Other comments relevant only to the moment or the maintainer, -not suitable for the permanent changelog, should also go here. A good example of -such comments might be "patch changelogs" which describe what has changed -between the v1 and v2 version of the patch. If you are going to include a -diffstat after the "---" marker, please use diffstat options "-p 1 -w 70" so -that filenames are listed from the top of the kernel source tree and don't use -too much horizontal space (easily fit in 80 columns, maybe with some -indentation). See more details on the proper patch format in the following -references. - -#### 16) Sending "git pull" requests (from Linus emails) - -Please write the git repo address and branch name alone on the same line so that -I can't even by mistake pull from the wrong branch, and so that a triple-click -just selects the whole thing. So the proper format is something along the lines -of: "Please pull from -git://jdelvare.pck.nerim.net/jdelvare-2.6 i2c-for-linus to get these -changes:" so that I don't have to hunt-and-peck for the address and inevitably -get it wrong (actually, I've only gotten it wrong a few times, and checking -against the diffstat tells me when I get it wrong, but I'm just a lot more -comfortable when I don't have to "look for" the right thing to pull, and -double-check that I have the right branch-name). Please use "git diff -M --stat ---summary" to generate the diffstat: the -M enables rename detection, and the -summary enables a summary of new/deleted or renamed files. With rename -detection, the statistics are rather different \[...\] because git will notice -that a fair number of the changes are renames. ------------------------------------ - -### SECTION 2 - HINTS, TIPS, AND TRICKS - ------------------------------------ This section lists many of the common -"rules" associated with code submitted to the kernel. There are always -exceptions... but you must have a really good reason for doing so. You could -probably call this section Linus Computer Science 101. - -#### 1) Read Documentation/CodingStyle - -Nuff said. If your code deviates too much from this, it is likely to be rejected -without further review, and without comment. One significant exception is when -moving code from one file to another -- in this case you should not modify the -moved code at all in the same patch which moves it. This clearly delineates the -act of moving the code and your changes. This greatly aids review of the actual -differences and allows tools to better track the history of the code itself. -Check your patches with the patch style checker prior to submission -(scripts/checkpatch.pl). The style checker should be viewed as a guide not as -the final word. If your code looks better with a violation then its probably -best left alone. The checker reports at three levels: - ERROR: things that are -very likely to be wrong - WARNING: things requiring careful review - CHECK: -things requiring thought You should be able to justify all violations that -remain in your patch. - -#### 2) #ifdefs are ugly - -Code cluttered with ifdefs is difficult to read and maintain. Don't do it. -Instead, put your ifdefs in a header, and conditionally define 'static inline' -functions, or macros, which are used in the code. Let the compiler optimize away -the "no-op" case. Simple example, of poor code: dev = alloc_etherdev -(sizeof(struct funky_private)); if (!dev) return -ENODEV; -#ifdef CONFIG_NET_FUNKINESS init_funky_net(dev); #endif Cleaned-up -example: (in header) #ifndef CONFIG_NET_FUNKINESS static inline void -init_funky_net (struct net_device \*d) {} #endif (in the code itself) dev -= alloc_etherdev (sizeof(struct funky_private)); if (!dev) -return -ENODEV; init_funky_net(dev); - -#### 3) 'static inline' is better than a macro - -Static inline functions are greatly preferred over macros. They provide type -safety, have no length limitations, no formatting limitations, and under gcc -they are as cheap as macros. Macros should only be used for cases where a static -inline is clearly suboptimal \[there are a few, isolated cases of this in fast -paths\], or where it is impossible to use a static inline function \[such as -string-izing\]. 'static inline' is preferred over 'static __inline__', 'extern -inline', and 'extern __inline__'. - -#### 4) Don't over-design. - -Don't try to anticipate nebulous future cases which may or may not be useful: -"Make it as simple as you can, and no simpler." ---------------------- - -### SECTION 3 - REFERENCES - ----------------------- Andrew Morton, "The perfect patch" (tpp). Jeff Garzik, -"Linux kernel patch submission format". Greg Kroah-Hartman, "How to piss off a -kernel subsystem maintainer". NO!!!! No more huge patch bombs to -linux-kernel@vger.kernel.org people! Kernel Documentation/CodingStyle: Linus -Torvalds's mail on the canonical patch format: Andi Kleen, "On submitting kernel -patches" Some strategies to get difficult or controversial changes in. -http://halobates.de/on-submitting-patches.pdf --
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-libcros/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-libcros/index.md deleted file mode 100644 index 16a43c6ff5c..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-libcros/index.md +++ /dev/null @@ -1,322 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: chromium-os-libcros -title: Cros - How Chromium talks to Chromium OS ---- - -[TOC] - -**DEPRECATED** - -As of October 2011, the Cros library (libcros.so) is now in the process of -phasing out. The chromium browser talks to the background servers of Chromium OS -via IPC mechanism called D-Bus. The browser has ability to issue D-Bus method -calls and handle D-Bus signals hence no longer needs the Cros library layer. - -## Abstract - -Cros came about because we needed a way for Chromium UI code to talk to Chromium -OS code. For example, the Chromium networking UI needs to talk to Chromium OS -flimflam to get network status and to make connections to wifi/cellular -networks. Cros is set of APIs that is implemented on the Chromium OS side and -exposed to Chromium via a dynamically linked *libcros.so* file. We designed a -versioning system to make sure that Chromium OS will only boot up if we have the -correct version of *libcros.so*. If either the *libcros.so* version or the -Chromium version is too old, we catch that and disable login. - -## Objective - -The design behind Cros is based on these requirements: - -* Chromium OS code and Chromium code are in 2 different source trees. - When we do release builds, we may use different snapshots of these 2 - trees depending on how stable they are. From past experience, we - would release Chromium OS with a slightly older but stable version - of Chromium. So Chromium should be able to talk to Chromium OS even - if the *libcros.so* file in Chromium OS is slightly newer. -* We should be able to modify Cros implementation without having to - recompile Chromium. -* We should be able to add new Cros API methods without having to - recompile Chromium. -* When new Cros API methods are needed by Chromium, we should be able - to tell Chromium to bind to these new methods and use them in - Chromium. -* The LKGR build of Chromium OS should always work with the LKGR build - of Chromium. -* When Cros in Chromium OS is incompatible with that of Chromium, we - need fail fast and let the user know why. - -## Detailed Description - -### Cros Files - -<table> -<tr> -<td> <b>File</b></td> -<td><b>Description</b></td> -<td> <b>Used by Chromium OS</b></td> -<td> <b>Used by Chromium</b></td> -</tr> -<tr> -<td> *platform/cros/chromeos_cros_api.h*</td> -<td> Defines version number of Cros: `kCrosAPIMinVersion` and `kCrosAPIVersion`</td> -<td> Yes</td> -<td> Yes (only kCrosAPIVersion)</td> -</tr> -<tr> -<td> *platform/cros/load.cc*</td> -<td> This is the code that Chromium uses to bind methods to *libcros.so*</td> -<td> Yes</td> -</tr> -<tr> -<td> *platform/cros/version_check.cc*</td> -<td> Version checking code to make sure that version of *libcros.so* is compatible with Chromium</td> -<td> Yes</td> -</tr> -<tr> -<td> *platform/cros/chromeos_\*.h*</td> -<td> Defines the Cros API</td> -<td> Yes</td> -<td> Yes</td> -</tr> -<tr> -<td> *platform/cros/chromeos_\*.cc*</td> -<td> Cros Implementation</td> -<td> Yes</td> -</tr> -</table> - -### Workflow - -This is the workflow of how Cros is used in Chromium OS: - -1. Chromium OS is built and the Cros files listed above are built into - the *libcros.so* file. -2. Chromium build uses a DEPS file to fetch a specific version of Cros - to build against. -3. Chromium is built against this specific version of Cros API. -4. When Chromium OS starts up Chromium, we make sure that the version - of Cros that Chromium is built against is compatible with the - version of *libcros.so*. -5. If it's compatible, we binds the method calls from Chromium to the - implementation in *libcros.so*. -6. All Cros method calls from Chromium will now execute the - corresponding method in Chromium OS's *libcros.so*. - -### Cros DEPS - -In order to have Chromium build against a specific version of Cros, we use this -DEPS file: -<http://src.chromium.org/viewvc/chrome/trunk/src/tools/cros.DEPS/DEPS?view=markup> - -```none -vars = { - "chromium_git": "http://src.chromium.org/git", -} -deps = { - "src/third_party/cros": - Var("chromium_git") + "/cros.git@e9730f8a -", -} -``` - -The 7 characters after the @ is the first 7 characters of the git commit hash -for the last Cros change that we want to build Chromium against. - -For this example in this specific case, this is the change: -<http://git.chromium.org/gitweb/?p=cros.git;a=commit;h=e9730f8a8a689ae3e506d1e400c4799190d302d0> - -This DEPS file will need to be updated when there are new Cros APIs that are -needed by Chromium. - -### Version Checking - -The version checking code is in *version_check.cc*, and it gets run by Chromium -when it is started by Chromium OS. - -In chromeos-api.h, we define 2 version numbers: `kCrosAPIMinVersion` and -`kCrosAPIVersion`. We use these version numbers to decide if the version of Cros -in Chromium is compatible with the version of Cros in Chromium OS. Since both -Chromium and Chromium OS have their own copy of chromeos-api.h, we have 2 sets -of these 2 version numbers. Chromium OS's `kCrosAPIMinVersion` and -`kCrosAPIVersion` specify the min and max version of the Cros API that the -*libcros.so* supports. In other words, it is only compatible if kCrosAPIVersion -in Chromium is within the range \[`kCrosAPIMinVersion,` `kCrosAPIVersion`\] of -Chromium OS. - -This means that the version of Cros in Chromium OS can be newer than the version -of Cros in Chromium. In other words, *libcros.so* can have new methods that are -not used by an older version of Cros in Chromium, but not the other way around. -If Chromium's kCrosAPIVersion is greater than Chromium OS's kCrosAPIVersion, -then Chromium is likely depending on a new Cros API that's not implemented in -*libcros.so*. In this case, the version check fails, and we prevent user from -logging in and display an error like: Incompatible libcros version. Client: 51 -Min: 29 Max: 50 - -The kCrosAPIMinVersion version in *libcros.so* is used to specify the minimum -version of the Cros API that's supported. This number gets increased when we -remove methods from the Cros API. So if Chromium's kCrosAPIVersion is less than -Chromium OS's kCrosAPIMinVersion, then that means Chromium is likely using a -Cros API method that's no longer there. In this case, the version check fails, -and we prevent user from logging in and display an error like: `Incompatible -libcros version. Client: 28 Min: 29 Max: 50` - -### Binding *libcros.so* - -After version check passes, the code in *load.cc* runs to bind the Chromium -method calls to the implementation in *libcros.so*. If binding fails for any -reason, we prevent the user from logging and display an error like: `Couldn't -load: MethodName` - -Common Developer Workflows - -### Modifying method implementation - -To change a Cros method implementation, you just need to change the code on the -Chromium OS side and you don't need to do anything on the Chromium side. -Chromium will bind with the unchanged Cros API on the new libcros.so and -exercise your new code without any problems. - -### Adding methods to Cros API - -This is best described by a table showing what you need to do on the Chromium OS -side and on Chromium side: - -<table> -<tr> -<td> <b>Chromium OS</b></td> -<td> <b>Chromium</b></td> -</tr> -<tr> -<td> Check in new code:</td> -<td>*chromeos_\*.h* - Add new API to </td> -<td>**chromeos_\*.cc* - Add implementation to* </td> -<td> *- Add code to *load.cc* to declare and initialize new API*</td> -<td> *- Increment kCrosAPIVersion in *chromeos_cros_api.h**</td> -</tr> -<tr> -<td> Get your change's commit hash from:</td> -<td> <a href="http://git.chromium.org/gitweb/?p=chromiumos/platform/cros.git;a=commit;h=HEAD">http://git.chromium.org/gitweb/?p=chromiumos/platform/cros.git;a=commit;h=HEAD</a></td> -</tr> -<tr> -<td> Update DEPS:</td> -<td> - Put your commit hash in cros_deps/DEPS</td> -<td> - Run "gclient sync"</td> -<td> - Confirm third-party/cros includes your change</td> -<td> - Make sure build compiles</td> -<td> - Copy cros_DEPS/DEPS to tools/cros.DEPS/DEPS </td> -<td> - Check in tools/cros.DEPS/DEPS</td> -</tr> -<tr> -<td> Check in new code that utilizes the new API</td> -</tr> -</table> - -### Deleting methods from Cros API - -Deleting a method from the API is a complicated process that's prone to error. -So make sure you know what you are doing. - -It is recommended that you wait at least a week after removing the usage and -binding of the method before you actually remove the implementation. This is -because deleting a method will make things backwards incompatible. So we need to -make sure that all possible versions of Chromium that was use with Chromium OS -will have this new Cros that does not bind with the deprecated method. - -<table> -<tr> -<td> <b>Chromium OS</b></td> -<td> <b>Chromium</b></td> -</tr> -<tr> -<td> Remove calls to deprecated API</td> -</tr> -<tr> -<td> Deprecate method:</td> -<td> - Keep the method implementation</td> -<td> - Remove binding of method in *load.cc*</td> -<td> - Increment *kCrosAPIVersion* in *chromeos-api.h*</td> -</tr> -<tr> -<td> Get your change's commit hash from:</td> -<td> <a href="http://git.chromium.org/gitweb/?p=chromiumos/platform/cros.git;a=commit;h=HEAD">http://git.chromium.org/gitweb/?p=chromiumos/platform/cros.git;a=commit;h=HEAD</a> </td> -</tr> -<tr> -<td> Update DEPS:</td> -<td> - Put your commit hash in cros_deps/DEPS</td> -<td> - Run "gclient sync"</td> -<td> - Confirm third-party/cros includes your change</td> -<td> - Make sure build compiles</td> -<td> - Copy cros_DEPS/DEPS to tools/cros.DEPS/DEPS</td> -<td> - Check in tools/cros.DEPS/DEPS</td> -</tr> -<tr> -<td> Delete method:</td> -<td> - Delete implementation</td> -<td> - Set kCrosAPIMinVersion to *kCrosAPIVersion*</td> -<td> - Increment *kCrosAPIVersion* in *chromeos-api.h*</td> -</tr> -<tr> -<td> Get your change's commit hash from:</td> -<td> <a href="http://git.chromium.org/gitweb/?p=chromiumos/platform/cros.git;a=commit;h=HEAD">http://git.chromium.org/gitweb/?p=chromiumos/platform/cros.git;a=commit;h=HEAD</a></td> -</tr> -<tr> -<td> Update DEPS:</td> -<td> - Put your commit hash in cros_deps/DEPS</td> -<td> - Run "gclient sync"</td> -<td> - Confirm third-party/cros includes your change</td> -<td> - Make sure build compiles</td> -<td> - Copy cros_DEPS/DEPS to tools/cros.DEPS/DEPS</td> -<td> - Check in tools/cros.DEPS/DEPS</td> -</tr> -</table> - -### Modifying method signatures - -It is highly recommend against modifying method signatures. This is because if -you modify a method signature, you would need to check in a backwards -incompatible change. In other words, in order for Chromium to talk to Chromium -OS, it would need this exact version of Cros. So you would have to check in code -to both Chromium OS and Chromium at the same time. And if anyone tries to use -Chromium OS with a slightly older version of Chromium, they will be blocked from -logging in. - -Instead of modifying a method signature, it is better to just add a new method -in Cros, change it so that Chromium calls this new method. And after a while, -delete the old deprecated method. - -## FAQ - -### How do I locally test a Cros and corresponding Chromium change before I check anything in? - -After you have "git commit" the Cros code, you can use "git pull" to pull that -code to Chromium: - -```none -cd ~/chrome/src/third_party/cros -git pull ~/chromeos/src/platform/cros -``` - -And if you want to test Chromium with the new *libcros.so*, you need to copy it -to the chromeos directory: - -<pre><code> -emerge-x86-generic libcros <i>(in Chromium OS chroot)</i> -mkdir ~/chrome/src/out/Debug/chromeos -cp ~/chromeos/chroot/build/x86-generic/opt/google/chrome/chromeos/* ~/chrome/src/out/Debug/chromeos -</code></pre> - -### Why does Cros fail to load with message "Incompatible libcros version. Client:x Min:y Max: z" where x > z? - -This happens when the version of Cros in Chromium is newer than the version of -Cros in Chromium OS. To fix this, make sure you fetch the latest code from -*platform/cros* and then emerge libcros. - -### Why does Cros fail to load with message "Incompatible libcros version. Client:x Min:y Max: z" where x < y? - -This happens when the version of Cros in Chromium is too old for the version of -Cros in Chromium OS. To fix this, make sure you build the latest Chromium.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-printing-design/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-printing-design/index.md deleted file mode 100644 index 8e1adb262e9..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/chromium-os-printing-design/index.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: chromium-os-printing-design -title: Chromium OS Printing ---- - -[TOC] - -### Abstract - -This document describes the cloud printing workflow and how it interacts with -Google Cloud Print. Support for and interactions with Google Cloud Print dialogs -will also be documented here as they are determined. - -### **Chromium OS** - -When asked to print, Chromium OS checks to see if the user is logged in. (This -case should be rare and occur only if the user entered via the guest account.) -If the user is not logged in, a login dialog is presented. - -Once logged in, if the user has not yet set up cloud printing or has no -registered or shared printers, a server-hosted introductory wizard is presented. -In this case, printers that are universally shared (for example, "printing" a -PDF to Google Docs) do not count as shared with the user, as it's expected that -most users will be interested in printing to local printers. This wizard -outlines the steps necessary to set up cloud printing, including instructions -for setting up a proxy and registering printers. This should happen even if -there are printers shared globally with the user. - -Once the user is logged in and has registered printers or has dismissed the -wizard as completed, the cloud printing workflow begins. - -### Chromium-based browser - -How cloud printing integrates into a Chromium-based browser is still to be -determined. - -### Cloud printing workflow - -On entering the cloud printing workflow, the browser checks to see if the -current page/application provides its own printing workflow (how to do this -TBD). Currently, some web applications, such as the document editor in Google -Docs, trap the keyboard shortcut for printing (Ctrl-P for English) and have -their own print menu items. The goal here is to unify the browser's print menu -item into that workflow for consistency. Once in the application's own print -flow, it is up to that application to work with Google Cloud Print or provide an -alternate means of printing. The application can choose to present its own print -settings user interface and re-join the common printing workflow by using the -window.print() JavaScript request, or it can provide its own entirely separate -printing solution. - -If the application does not provide its own print workflow, a modal cloud print -common dialog is presented in a new popup window. This single dialog allows the -user to quickly print with some default settings but also allows for changing of -some page setup and advanced print settings without bringing up additional -dialogs. The contents of the dialog are hosted by Google Cloud Print, and the -browser provides calls to the scripts on the page in the dialog to change page -setup information (information needed for PDF generation) and for returning the -contents of the generated PDF file from the browser to the scripts for uploading -to the service. (This may be changed so that the browser could also provide the -upload service as a call, so that it can happen on the IO thread.) Any setting -of the page setup information will require re-generation of the PDF file, which -should happen in the renderer process that owns the tab being printed. - -Optionally, in the future, the browser can get a thumbnail rendering of a -particular page and call into the dialog scripts to show it. Additionally, -application-specific simple and advanced settings may be incorporated into the -dialog. - -After the user sets other post-PDF generation information (job settings, for -example, *N*-up printing), the scripts in the dialog upload the PDF the browser -provided and the job ticket containing the job settings. It then makes a call to -the browser to close the dialog. - -#### Notes - -This is a change in printing workflow from what is currently in Chromium OS. In -order to mix the page settings into a single simple print dialog, the print -settings can't be treated as final before the dialog comes up (which is how -printing in Chromium OS is currently working). But this also gives us an -opportunity to get a PDF generation progress meter somewhere in this process. -(There's a Chromium OS bug currently open on this issue.) - -Care needs to be taken to generate the PDF in the background asynchronously. The -dialog script code needs to disable or otherwise indicate that it hasn't yet -gotten back the PDF information from the browser any time it makes a change to -the page setup information. - -Uploading of the PDF and job settings happens in the background.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cras-chromeos-audio-server/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cras-chromeos-audio-server/index.md deleted file mode 100644 index 56b48044bbe..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cras-chromeos-audio-server/index.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: cras-chromeos-audio-server -title: 'CRAS: Chromium OS Audio Server' ---- - -## Why - -Chromium OS needs a way to route audio to different devices. Currently Chromium -OS only plays audio to the first ALSA device in the system. Devices attached -after boot are ignored, and no priority is given to the built in sound device. -The goal of the new server is to allow for sound to be routed dynamically to -newly attached audio-capable monitors (DisplayPort and HDMI), USB webcam, USB -speakers, bluetooth headsets, etc. And to do this in a way that requires as -little CPU as possible and that adds little or no latency. - -## How - -Put an audio server in between Chromium and ALSA that can change how audio flows -on the fly. Chromium will connect to this audio server, send audio data to it, -and the audio server will render this to the selected ALSA device through -alsa-lib. For input the process will be reversed, the server will get audio -samples from alsa and forward them to Chromium. - -## Details - -The design decisions were driven by three main requirements: Low Latency, Low -CPU usage, and dynamic audio routing. This will handle the streaming of PCM -only; audio decode will all be done in Chromium. - -The basic design combines the timer-driven wakeup scheme of pulseaudio with the -synchronous audio callbacks of JACK. This synchronous callback eliminates local -buffering of data in the server, allowing the server to wake up less for a given -latency. The server wakes up when a timer fires, sometime before there is an -ALSA over/underrun and calls back to Chromium to get/put audio data. This -minimizes latency as the only additional latency needed is that to exchange IPC -messages between the server and client. Because ALSA wakeups aren’t used, on -some devices the minimal latency will be lower than without the server; there is -no need to wake up at an ALSA period boundary. The timer can be armed to wake up -when there are an arbitrary number of samples left in the hardware buffer. ALSA -will always be configured with the largest possible buffer size, yet only some -fraction of that will be used at any given time. - -Audio data will be exchanged through shared memory so that no copying is needed -(except into the ALSA buffer). Communication will happen through UNIX domain -sockets. There will be separate communication connections for control (high -latency, lower priority) and audio data (low latency and higher priority). - -At each wakeup a timestamp will be updated that, for playback streams, will -indicate the exact time at which the next written sample will be rendered to the -DAC. For capture, the timestamp will reflect the actual time the sample was -captured at the ADC. - -The key to keeping system resource usage low is to do as little as possible in -the server when it wakes up and similar for the callback in the client. Less -than two percent of the CPU of a CR48 is used while playing back audio with 20ms -latency and waking up every 10ms. The minimum the server can do is read in the -streams, mix them, and write the result to ALSA. Other processing can take place -in the server; DSP blocks can be configured to process the buffers as they pass -through, this will add to the processing time and increase the minimum latency. - -The lowest latency stream will be used to drive the timer wakeup interval (and -will have the most accurate latency). Only clients that need to send more data -will be woken. - -### Device Enumeration - -The server keeps a separate list of inputs and outputs. The server will listen -to udev events that notify it of newly added or removed devices. Whenever a -device appears or disappears, the priority list of devices will be re-evaluated -and the highest priority device will be used. - -Devices will be given a priority based on the device type and the time they are -inserted. The priority is as follows (for both input and output): - -1. Headset jack and USB audio devices. -2. DisplayPort and HDMI outputs (when EDID indicates the sink is audio - capable). -3. Built-in speakers. - -If two devices of the same priority are attached, the device that was plugged -most recently wins. - -On device removal, all streams are removed from the device and the device is -closed. The streams will be moved to the next device in the list when this -happens. - -### Mixing - -The server is required to mix for the case where multiple streams are being -played at once. The mixing will be done immediately before the samples are sent -to the playback hardware. If a DSP block is enabled, it will be applied -post-mix. - -### Per-Board Configuration - -Without a config, the server will automatically discover devices and mute/volume -controls. This provides all that is needed for x86 most of the time. On x86 the -main use of config files is to set volume level mappings between the UI and -hardware, and to configure codec-specific features. - -The volume curve file is per-codec, per-device and the format is described in -the main README file. - -ALSA UCM is used to do low level codec config. On ARM systems, this setup is -necessary to get audio paths routed correctly when a headset is attached and to -initialize codec setup after start up. In addition to this, the UCM file is used -to map jacks to audio devices. - -### More documents - -* [Chrome OS Audio Software - Overview](https://docs.google.com/document/d/1pDdzlJlNacvOB8CS4Qxg-0aQvEuCK8lFKaWaqckSl3w/edit?usp=sharing) -* [CRAS ELC - 2014.pdf](https://drive.google.com/file/d/1WBYe-M_xFaHIajn-hfQRBrKhPkbgHNF9/view?usp=sharing) -* [Gain in - CRAS](https://docs.google.com/document/d/1FfTGylzC-uGPhzxnotfxXjDeBPRPIMxkzqdQbQWJ7aE/edit?usp=sharing) -* [UCM Usage - Guide](https://docs.google.com/document/d/1AcXZI9dvJBW0Vy6h9vraD7VP9X_MpwHx0eJ6hNc4G_s/edit?usp=sharing)
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/cellular-activation/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/cellular-activation/index.md deleted file mode 100644 index 29ae51fe310..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/cellular-activation/index.md +++ /dev/null @@ -1,159 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -- - /chromium-os/chromiumos-design-docs/cros-network - - 'CrOS network: notes on ChromiumOS networking' -page_name: cellular-activation -title: Cellular Activation (and Chrome/CrOS network management API) ---- - -ActivateCellular (network_menu.cc) - --> -ash::Shell::GetInstance()->delegate()->OpenMobileSetup(cellular->service_path()) - --> MobileSetupDialog::Show(service_path) \[if kEnableMobileSetupDialog\] - --> MobileSetupDialogDelegate::GetInstance()->ShowDialog(service_path) - --> MobileSetupDialogDelegate::GetDialogContentURL \[presumably\] - -// returns chrome::kChromeUIMobileSetupURL + service_path_ - --> browser->OpenURL(chrome::kChromeUIMobileSetupURL) \[otherwise; resolves -to chrome://mobilesetup\] - -MobileActivator::SetTransactionStatus (mobile_activator.cc) - --> StartOTASP - --> EvaluateCellularNetwork - -MobileActivator::OnNetworkManagerChanged (mobile_activator.cc) - --> EvaluateCellularNetwork - -MobileActivator::OnNetworkChanged (mobile_activator.cc) - --> EvaluateCellularNetwork - -MobileActivator::StartActivation - --> EvaluateCelluarNetwork - -Who calls OnNetworkManagerChanged? - -- NetworkLibraryImplBase::NotifyNetworkManagerChanged - --> NetworkLibraryImplBase::SignalNetworkManagerObservers - -- NetworkMessageObserver::NetworkMessageObserver - -- AshSystemTrayDelegate - -- NetworkMenuButton - -- NetworkScreen - -MobileActivator::ChangeState (mobile_activator.cc) - --> FOR_EACH_OBSERVER(Observer, observers_, OnActivationStateChanged(network, -state_, error_description)); - --> MobileSetupHandler::OnActivationStateChanged (mobile_seutp_ui.cc) -\[presumably\] - --> web_ui()->CallJavascriptFunction(kJsDeviceStatusChangedCallback, -device_dict) - --> mobile.MobileSetup.deviceStateChanged (mobile_setup.js) - --> updateDeviceStatus_ - --> changeState_ - --> stopSpinner_ \[if PLAN_ACTIVATION_DONE\] - ---- - -MobileSetupDialogDelegate::OnActivationStateChanged (mobile_setup_dialog.cc) - -// does nothing - -chromeos.connectionManager.setTransactionStatus (connection_manager.js) - --> reportTransactionStatus_ - --> postMessage(msg, 'chrome://mobilesetup') - -// seems to be billing related. see onMessageReceived_ in mobile_setup.js - -NetworkLoginObserver - -// The network login observer reshows a login dialog if there was an error. - -NetworkMessageObserver - -// The network message observer displays a system notification for network - -// messages. - -NetworkLibrary interface declarations - -- NetworkManagerObserver::OnNetworkManagerChanged - -- NetworkObserver::OnNetworkChanged - -- NetworkDeviceObserver::OnNetworkDeviceChanged - -- NetworkDeviceObserver::OnNetworkDeviceFoundNetworks - -- NetworkDeviceObserver::OnNetworkDeviceSimLockChanged - -- CellularDataPlanObserver::OnCellularDataPlanChanged - -- PinOperationObserver::OnPinOperationCompleted - -- UserActionObserver::OnConnectionInitiated - -NetworkLibraryImplCros::NetworkManagerStatusChangedHandler - -NetworkLibraryImplCros::NetworkManagerUpdate - --> NetworkManagerStatusChanged - --> NotifyNetworkManagerChanged \[ONLY for PROPERTY_INDEX_OFFLINE_MODE\] - -CrosMonitorNetworkManagerProperties - --> new NetworkManagerPropertiesWatcher - --> -DBusThreadManager::Get()->GetFlimflamManagerClient()->SetPropertyChangedHandler - -NetworkLibraryImplCros::UpdateNetworkStatus - --> NotifyNetworkManagerChanged - -NetworkLibraryImplCros::UpdateTechnologies - --> NotifyNetworkManagerChanged - -NetworkLibraryImplCros::ParseNetwork - --> NotifyNetworkManagerChanged - -NetworkLibraryImplCros::ParseNetworkDevice - --> NotifyNetworkManagerChanged - -AshSystemTrayDelegate: updates status icons - ---- - -Network::SetState (network_library.cc) - -// processes state change events from connection manager
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/chrome-network-debugging/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/chrome-network-debugging/index.md deleted file mode 100644 index 8a2447222b7..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/chrome-network-debugging/index.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -- - /chromium-os/chromiumos-design-docs/cros-network - - 'CrOS network: notes on ChromiumOS networking' -page_name: chrome-network-debugging -title: chrome network debugging ---- - -chrome has several debugging tools: - -* netlog (chrome://net-internals -> events). see [NetLog: Chrome’s - network logging - system](/developers/design-documents/network-stack/netlog), or [the - list of netlog events and their - meanings](http://src.chromium.org/viewvc/chrome/trunk/src/net/base/net_log_event_type_list.h) -* chrome://net-internals/#chromeos -> controls to enable chromeos - network debugging and to package up log files -* chrome://net-internals/#tests -> test loading a particular URL -* network debugger app (in development; contact ebeach@)
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fake-cromo/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fake-cromo/index.md deleted file mode 100644 index c572a4854ab..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fake-cromo/index.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -- - /chromium-os/chromiumos-design-docs/cros-network - - 'CrOS network: notes on ChromiumOS networking' -page_name: fake-cromo -title: fake-cromo ---- - -* Implemented in - [fake-cromo](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/flimflam/test/fake-cromo). -* Seems to provide a mock implementation of modem-manager (classic) - APIs, focusing on CDMA. -* Works with - [activation-server](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/flimflam/test/activation-server), - which provides emulation of carrier activation sequence. - -Usage: - -Start the web server that emulates the carrier activation portal. - -# /usr/local/lib/flimflam/test/activation-server & - -Rename eth0 to the name of the cellular device, as reported to shill - -by the fake modemmanager. (The fake modemmanager is implemented in - -[the Modem class in -flimflam_test](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/flimflam/test/flimflam_test.py&l=106).) - -This means that we'll route "cellular" data traffic out our Ethernet - -connection. Note, however, that fake-cromo let's us implement a - -"captive portal" like mode. (That's implemented using iptables.) - -Note further that the name of the cellular device must be pseudo-modem0, - -as that's [hard-coded in -fake-cromo](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/flimflam/test/fake-cromo&l=61) -(and also used in a [default -argument](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/flimflam/test/flimflam_test.py&l=128) - -in the Modem class.) - -# /usr/local/lib/flimflam/test/backchannel setup eth0 pseudo-modem0 - -Start fake-cromo - -# /usr/local/lib/flimflam/test/fake-cromo &
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fake-gsm-modem/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fake-gsm-modem/index.md deleted file mode 100644 index 414a7cd601e..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fake-gsm-modem/index.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -- - /chromium-os/chromiumos-design-docs/cros-network - - 'CrOS network: notes on ChromiumOS networking' -page_name: fake-gsm-modem -title: fake-gsm-modem ---- - -* Implemented in - [fake-gsm-modem](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/flimflam/test/fake-gsm-modem). -* Seems to serve as a replacement for modemmanager (classic). I.e., I - think it provides a mock implementation of the MM D-Bus APIs. In - contrast, - [fakemodem](/chromium-os/chromiumos-design-docs/cros-network/fakemodem) - provides a mock implementation of the AT command interface. -* For data forwarding, you have to use an existing ethernet interface. - (See comments at top of - [fake-gsm-modem](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/flimflam/test/fake-gsm-modem).) -* To test shill (as opposed to flimflam) against fake-gsm-modem, - follow these steps: - -```none -$ stop flimflam -$ stop cromo -$ /usr/local/lib/flimflam/test/veth setup pseudomodem0 172.16.1 -$ /usr/local/lib/flimflam/test/fake-gsm-modem -c tmobile --shill -$ start flimflam -``` - -T-Mobile should now appear as a connectable network in the 'Internet Connection' -Settings UI. Furthermore, running 'modem status' in crosh should display -information on TestModem. - -Once done with the tests, run the following for cleanup: - -```none -$ stop flimflam -$ # kill the fake-gsm-modem process -$ /usr/local/lib/flimflam/test/veth teardown pseudomodem0 -$ start cromo -$ start flimflam -``` - -shill will now interact with cromo. Bear in mind that, as of the writing of this -entry, the shill daemon is still referred to as flimflam, which is why we call -`start flimflam` and not `start shill`. This will be changed in the future. - -The above steps apply to Chromebooks using a Gobi modem. For any other modem, -replace `cromo` with the correct modem manager.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fakemodem/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fakemodem/index.md deleted file mode 100644 index 482760ac520..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/fakemodem/index.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -- - /chromium-os/chromiumos-design-docs/cros-network - - 'CrOS network: notes on ChromiumOS networking' -page_name: fakemodem -title: fakemodem (for testing celluar modems with AT command interfaces) ---- - -* Gives pre-programmed answers to AT commands. (Regexp patterns match - the command, and probably can be used in generating the response.) -* Implemented in - [fakemodem.c](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/autotest/files/client/deps/fakemodem/src/fakemodem.c). -* Used by [modemmanager SMS - autotest](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/autotest/files/client/site_tests/network_ModemManagerSMS/network_ModemManagerSMS.py). -* Example regexp tables: [Icera SMS signal test - patterns](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/autotest/files/client/site_tests/network_ModemManagerSMSSignal/src/fake-icera), - [generic GSM SMS Signal test - patterns](http://code.google.com/searchframe#wZuuyuB8jKQ/src/third_party/autotest/files/client/site_tests/network_ModemManagerSMSSignal/src/fake-gsm).
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/index.md deleted file mode 100644 index 99e67e0a9fe..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/cros-network/index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: cros-network -title: 'CrOS network: notes on ChromiumOS networking' ---- - -{% subpages collections.all %} diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/developer-mode/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/developer-mode/index.md deleted file mode 100644 index 1a8572009bf..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/developer-mode/index.md +++ /dev/null @@ -1,570 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: developer-mode -title: Developer Mode ---- - -[TOC] - -# Introduction - -Google Chrome OS devices must be: - -* Secure - users must be protected from attackers. This protection - should reasonably extend to inexperienced or naive users. -* Open - developers must be able to install their own builds of a - Chromium-based OS, or another operating system (e.g., Ubuntu). - Developers should not be significantly impaired from using the full - capabilities of the device. - -These two requirements are somewhat at odds. Locking down a device to protect -users makes it less useful for developers. Openings for developers to run -modified software easily can also be exploited by attackers. - -This document describes: - -* Vulnerabilities to which Google Chrome OS devices may be exposed. -* Use cases we must allow for normal users and developers. -* A proposed implementation of developer mode that provides both - openness for developers and security for users. -* How the implementation protects from vulnerabilities while allowing - required use cases. - -Note that this document is specifically about the developer mode in Google -Chrome OS devices, not about other Chromium OS-based devices. It's focused on -developers who want to install a different operating system on a device that -initially shipped with Google Chrome OS. - -# Vulnerabilities - -The following vulnerabilities have been considered in this design. - -## Attacker with complete physical access - -An attacker with physical possession of the device over an extended period. The -attacker has access to tools including a soldering iron. - -The attacker can remove the EEPROM containing our read-only firmware and replace -it with compromised firmware which does not perform signature checks. -Alternately, the attacker could simply replace the embedded controller's EEPROM -with code implementing a keylogger. - -Based on YouTube videos of speed disassembly and assembly of computers and -consumer electronics devices, we believe that a practical attack could be -mounted in as little as 5 minutes. - -This attack is unlikely to be made against a large number of devices. - -Subcases: - -* Evil maid: The attacker modifies a targeted user's netbook so that - it captures passwords, etc. The attack can be reversed by the - attacker given a second possession of the device, leaving no - evidence of the attack. -* Malicious return: The attacker modifies a device then returns it to - a store. It could act as a keylogger, phoning home at a later date - with credentials of the next purchaser. -* Device swap: The attacker purchases the same model of netbook as the - targeted user. When the targeted user is distracted, a replacement - netbook is swapped for the user's netbook. - -We cannot prevent this attack. - -## Attacker with casual physical access - -An attacker with short-term access to a device—for example, a netbook briefly -left unattended at a conference or coffee shop. - -The attacker may be able to insert a USB key and reboot the device, but cannot -make extensive physical modifications. The attacker may be able to unscrew a -panel and flip a switch. - -Subcases: - -* The attacker attempts to quickly install malicious software on a - temporarily-unattended machine. -* The attacker walks down a row of machines at a trade show or store, - installing malicious software on them (password sniffer, keylogger, - etc). -* A store employee attempts to install malicious software on a large - number of computers for sale. - -We should protect against this attack by making it as costly as the physical -attack, in terms of time required and noticeability to passers-by. - -## Attacker with no physical access - -An attacker with no physical access attempts to alter the device to persistently -run modified software. This is the most dangerous attack, due to its ability to -affect a large number of devices. - -We must prevent this attack. That is, it should not be possible to persistently -control a device without first having physical access to it. - -## Developer borrows a device - -A developer buys or borrows a device, installs their own software, then returns -the device without reinstalling Google Chrome OS. - -The custom software installed is unlikely to be intentionally malicious. The -primary risk is that developer images may not autoupdate, so the software on the -device will not be patched to close vulnerabilities. This risk has manifested -recently in smartphones with user-modified ('jailbroken') operating systems. - -Subcases: - -* Return to store: The store may boot the device to make sure it's not - broken, but will not generally open the case to check for hardware - modification or switch state. The next user buys the device and - starts using it. The first time the user boots the device, they're - likely to be right in front of it, so will notice (but may not - understand) a warning screen. -* Return to user: It may take several boots for the user to actually - be present at the device during boot, but it's very likely the user - eventually sees a warning screen. - -We should protect against these cases. - -## Other vulnerabilities - -There are other potential data and device vulnerabilities, including: - -* Data corruption caused by cosmic rays, aborted auto-updates, - defective drives, etc. -* Theft of a device. - -These are outside the scope of this document since they do not relate to -developer mode. - -# Use cases to allow - -The following use cases have been considered in this design. - -## Normal user - -A normal user generally runs only Google Chrome OS on their device. They are not -generally interested in running any other operating system on the device. - -When faced with recovery mode, many users are likely to return the device to the -store or call tech support. - -The most naive users may not understand the significance of warning screens. We -should not be asking the user to make uninformed or cryptic decisions. - -We must allow this use case. - -## Developer - -A developer builds Chromium OS or their own OS and wants to install it on a -device. - -The developer has access to developer tools, the target device, and any -documentation which came with the device. They are experienced enough to use a -screwdriver to open a panel, but most will not be willing to use a soldering -iron to modify the device or void their warranty. - -The developer expects similar performance while using their own OS as they have -while using Google Chrome OS. That is, boot time and the boot process should not -be significantly impaired from a normal device. - -It is acceptable for the initial install of a developer OS to be slow, as long -as it is not painfully slow. Subsequent installs should be fast, so that a -developer can do rapid development. - -The developer should be able to go back easily to running Google Chrome OS, at -which point they should have access to automatic updates. This case also covers -responsible developers who buy or borrow a device, try it out with their own OS, -then run recovery mode to restore the device before returning it. - -Subcases: - -* Chromium OS developer: They will use the device in a manner similar - to Google Chrome OS, perhaps with some added capability (shell - access, etc). This is a common and valuable developer use case; - these developers contribute back to the Chromium OS project. -* Interactive developer: The developer uses the device like a normal - laptop. -* Remote developer: The developer uses the device as a media server, - set-top box, or some other use case where they do not have direct - access to the keyboard and primary display. - -We must allow these use cases. If we don't, it is likely developers will hack -holes in verified boot to enable them. - -## User installs prebuilt OS distribution (other than Google Chrome OS) - -This is much the same as the developer use case. The difference is that the user -did not build the install. This is probably more common than developer installs, -particularly for use cases such as media servers or set-top boxes. - -We should allow this use case. - -## Manufacturer - -Normally, read-only firmware is fixed, and rewritable firmware is signed by -Google. This introduces additional signing steps and delay into board bringup -and development. - -A vendor involved in manufacturing a device (OEM, ODM, BIOS vendor, component -vendor) must be able to develop and debug firmware, including the -normally-read-only parts. For example, the vendor must be able to fix bugs in -recovery mode. - -The vendor is capable of disassembling a device and making hardware -modifications involving soldering - -It is acceptable for modifying a device to disable [write -protection](/chromium-os/firmware-porting-guide/firmware-ec-write-protection) to -be slow, as long as it is not painfully slow. Subsequent installs should be -fast, so that the vendor can do rapid development. - -It is not necessary for the modifications to be reversible easily. For example, -removing the modifications can require re-disassembling the device and -re-soldering components. - -We must allow this use case, to reduce the time and cost of producing devices -for Google Chrome OS. - -# Implementation - -The following sections describe the implementation we will use. This design -supports developers while protecting normal users, and requires no changes to -the typical device manufacturing process. - -## Developer switch - -The device will have an on/off developer switch which is not easily accessible -by a normal user. - -The switch is reversible (that is, it can be turned back off once turned on). It -is turned off by default. It must be turned on physically; there must be no way -to turn this switch on without physically manipulating the device. - -Approved implementations for this switch: - -* A switch underneath a screw cover. -* A switch underneath/behind the battery, such that the battery must - be removed to access the switch. -* Pressing Control+D at the recovery screen, where the device uses the - approved keyboard recovery mechanism \[1\]. -* Pressing Control+D at the recovery screen followed by pressing the - recovery button, where devices have a physical recovery button. - -Other implementations must be approved by Google in advance. - -When the developer switch is off, only code signed by Google will run on the -device. Any other code will cause the device to reboot into recovery mode. This -protects against remote attack. - -Note 1: The approved keyboard recovery mechanism contains a Google-designed -circuit that provides assurance that the executing firmware is the read-only -version, and that the user is physically present. - -## Mode transition wipe - -On the transition between dev and normal modes, there will be a 5 minute delay -during which the OS will wipe the stateful partition of the device. The delay -and wipe occur on the first boot after the dev switch is enabled. - -The goal of this delay is to increase the difficulty of drive-by attacks to the -same level as physically compromising the device. (We could optionally play an -audio track—perhaps something along the lines of "Help, help, I'm being -repressed!"—or require user interaction during this delay, to make it more -noticeable if an attacker is attempting to compromise a device in a public -place.) - -Note that the delay protects both normal users and developers; an attacker -trying to change the image on a developer's machine (for example, at a Linux -conference, where Chromium OS machines are likely to be common) will still incur -the delay. - -## Recovery button always triggers recovery mode - -The device will have a mechanism through which it always boots in [recovery -mode](http://www.chromium.org/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery), -regardless of the position of the developer mode switch. This provides a path -for users and developers to get back to Google Chrome OS, with all its safety -and autoupdates. - -Approved implementations for the recovery button are: - -* Physical recovery button that must be held down on boot. -* Google approved keyboard controlled recovery circuitry, whereby a - user can hold three keys to get to recovery. - -## Boot errors always trigger recovery mode - -The firmware always checks the kernel signature at boot time. If the signature -is invalid, then the firmware checks the backup kernel. If both kernels are -invalid, the firmware runs recovery mode. - -Google will provide a utility for developers to sign their kernels based on a -self-generated certificate. - -Developer kernel signatures are not tied to a single target device; that would -prevent users from installing prebuilt OS distributions. In developer mode, the -cryptographic signature is cannot verified since the user’s public key is not -contained in the firmware; only the signing structure and checksums are used to -“validate” a self-signed kernel. - -## Recovery boot image - -The recovery boot image is Google-signed software on a removable drive. The -recovery boot image is what copies new firmware and software from the removable -drive to the fixed drive when booted in [recovery -mode](http://www.chromium.org/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery). -Recovery mode firmware will load only Google-signed software from the removable -drive. Those kernels contain their own initramfs, which copies the recovery -image onto the fixed drive. - -If the developer switch is off, the recovery boot image will refuse to copy -anything but Google-signed software to the device. This protects against -drive-by attacks; the attacker must have time to get to the developer switch. -This also protects normal users from a remote attacker who stores a malicious -payload on the user's SD card and then attempts to reboot into recovery mode to -install that payload. - -If the developer switch is on, the behavior of the recovery image depends on the -keys used to sign the current firmware/kernel/image and the new -firmware/kernel/image to be installed. - -* If the new key is Google-generated, there is no delay. This is a - user trying to get back to Google Chrome OS. -* If the new key matches the current key, there is no delay. This is a - developer updating their own system. -* If the new key is not Google-signed, and is different than the - current key, there is a 5-minute delay before copying the payload to - the fixed disk. - -During the delay, the recovery image will display the developer agreement and -instructions on how to reinstall Google Chrome OS if you decide you don't like -the image you're installing. The goal of this delay is to increase the -difficulty of drive-by attacks to the same level as physically compromising the -device. (We could optionally play an audio track—perhaps something along the -lines of "Help, help, I'm being repressed!"—or require user interaction during -this delay, to make it more noticeable if an attacker is attempting to -compromise a device in a public place.) - -Note that the delay protects both normal users and developers; an attacker -trying to change the image on a developer's machine (for example, at a Linux -conference, where Chromium OS machines are likely to be common) will still incur -the delay. - -## Boot time warning screen - -When the device boots normally in developer mode (that is, not in recovery -mode), it displays a warning so that ordinary users won't accidentally end up in -the wrong operating system. - -At boot time, if the developer switch is on and the kernel is signed, but not by -Google, then a warning screen will be displayed by the firmware. The UI varies -slightly with the device, but may show a message something like this: - -**WARNING!** -**YOUR COMPUTER IS NOT RUNNING GOOGLE CHROME OS!** -**PRESS THE SPACE BAR TO FIX THIS.** - -This screen is intended to be a little scary; for example, it should use colors -and symbols designed to indicate danger / badness. It must support i18n. - -If the user follows the on-screen instructions, the device will either revert to -normal mode or reboot into recovery mode. The on-screen instructions typically -involve pressing SPACE or RETURN, since these are the most common keys users -press when trying to get past a screen without reading it. - -Upon seeing this screen, non-developer users will either return to normal mode, -run recovery mode, call tech support, or give up and return the device. Any of -these outcomes protects the user's data. - -If the user presses Control+D, the device will bypass the warning screen waiting -period and continue booting the developer-signed code. Note that there is no -indication on the warning screen that Control+D is a valid option, so naive -users won't be tempted to press it. The Control+D sequence will be documented on -the Chromium OS developer website (you're reading it now), where it is easy for -developers to discover. We use the Control modifier so that accidentally bumping -the keyboard will not trigger it. For interactive developers, this reduces the -pain threshold of the warning screen to a single keypress. - -If the user presses a key other than Control+D, space bar, Enter, or Esc, no -action is taken. This reduces the annoyance level for developers; accidentally -pressing D+Control or Control+S does not reboot into recovery mode. - -After 20 seconds of displaying the warning screen, the device should beep. This -will alert a nearby user to look at the screen and see the warning. Interactive -developers are likely to press Control+D before this time, so will not be forced -to endure the beep. - -If the user waits 30 seconds, the device will continue booting the -developer-signed code. This is necessary to support the remote developer subcase -above. It does mean that a naïve user could keep ignoring this screen. This is a -reasonable tradeoff, given that requiring a keypress invites developers to hack -verified boot. - -In addition, there are bios flags that can be set to enable a user to trigger -other interactions, including but not limited to: - -* dev_boot_usb - this enables a device to boot from USB when a user - presses the Control+U keys at the boot time warning screen. -* dev_boot_legacy - this enables a device to boot from SeaBIOS when a - user presses the Control+L keys at the boot time warning screen. - -## Manufacturer hardware modifications - -The device can be modified to disable the write protection on the firmware. This -modification should involve partial device disassembly and soldering, and will -void the device warranty. Disassembling the device far enough to make this -modification should require destroying a tamper-resistant seal, so that it's -obvious the device has been disassembled. - -Once this modification is performed, the manufacturer can reprogram all the -firmware. - -On a device where write protection is enabled in hardware (for example, an -EEPROM with a write protect enable line), this modification may take the form of -removing a resistor which pulls the write protect enable line active, and/or -adding a resistor to another set of pads to pull the write protect enable line -to the disabled state. - -On a device where write protection is enabled in software (for example, a NAND -flash with a write-once-per-boot instruction which prevents further write -instructions that boot), this modification may take the form of an additional -input GPIO whose state is controlled by a resistor. The boot stub firmware will -read the GPIO state before writing the boot instruction; if the GPIO is -asserted, the boot stub firmware will not issue the write instruction to the -NAND flash. - -Alternately, the EEPROM, which is normally soldered directly to the motherboard, -may be replaced with a socketed EEPROM to facilitate reprogramming via an -external programmer. - -To prevent devices with disabled write protection from being shipped -accidentally to consumers, final manufacturing tests should verify that the -modifications have not been made. That is, the tests should attempt to write to -the read-only portion of the firmware and/or verify that a write protection -GPIO, if any, is asserted. - -Note that a sufficiently advanced developer willing to void the warranty on -their device may be able to make the hardware modifications to disable write -protection and replace the normal Google Chrome OS firmware with their own -firmware. Such a developer would also be able to desolder, remove, and rewrite -the EEPROM chip with their own firmware, so including manufacturer support does -not represent a significant reduction in security. - -# How this design addresses vulnerabilities and use cases - -## Attacker with complete physical access - -We cannot prevent this attack. - -## Attacker with casual physical access - -The attacker must have sufficient control of the machine to turn on the -not-easily-accessible developer switch. - -Since the attacker's code is signed with a different key than the existing -software on the device, the recovery image will delay installing the new image. -The attacker must wait around near the device to interact with the machine -during this time. This raises the risk of discovery for the attacker. - -If the attacker has sufficient control of the device that they're not worried -about unscrewing a panel, removing the battery, and interacting with the device -for 5 minutes, that's tantamount to complete physical access. As with the -previous case, an attacker with that level of access cannot be prevented. - -## Attacker with no physical access - -If there is a vulnerability in Google Chrome, the attacker may be able to obtain -temporary control over the device. Users could potentially enter their -credentials into this compromised device and run modified/unsafe code. - -The attacker will not be able to maintain long-term persistent control over the -device. When the device next reboots, the verified boot process will detect the -modified software and trigger recovery mode (if the developer mode switch is -off) or a warning screen (if the dev mode switch is on). - -## Developer borrows a device - -If the developer is a nice developer (and they remember to restore the device), -they run recovery mode on the device before returning it. - -If the developer returns the device without running recovery mode or turning the -developer switch back off, the device will show a warning screen at boot time. - -Stores should be advised to boot returned Google Chrome OS machines. If they do, -they'll notice the warning screen and can refuse the return (at which point the -developer will turn off the developer switch and run recovery mode and then -successfully return the device). - -If the store does not check a returned device, it could be sold to a user while -still containing developer software. The first time a new user boots their -device they're likely right in front of it (after all, they'll want to see our -5-second boot time) and will see the warning screen. At that point they'll -likely either return/exchange the device, call tech support, or fix it -themselves. - -A user who lent their device to a developer will likely notice the warning -screen, or that the login screen looks different. At this point they'll complain -to the developer ("Hey sonny, what did you do to Grandma's computer?"), who can -fix it. - -Note that in all of these cases, the developer software is unlikely to be -harmful in itself; the risk is that since developer mode removes some of the -normal-mode protections, the user is more vulnerable to attacks. - -## Normal user - -With the developer mode switch off, a normal user can only run Google Chrome OS. -Anything else will behave like the accidental corruption case—the device will -either repair itself or trigger recovery mode. - -## Developer - -A developer can modify the software on the device, installing their own Chromium -OS or other OS. Opening a screw panel and flipping a switch is easy and does not -void the device warranty. - -To run an OS which requires legacy BIOS, they can replace the kernel with -software which sets up legacy interrupts and then runs something int19h-like to -load that OS. This is even easier since we're using GUID partition tables; the -legacy OS will only see the partitions described in the legacy MBR which -precedes the GUID partition table, so won't see the dedicated kernel partitions. - -A developer using the device interactively needs to press a single key at boot -time to dismiss the warning screen. This is minimally invasive; it delays boot -time by only a second. - -A developer using the device remotely will need to wait 30 seconds after reboot -for the warning screen to time out. This is not likely to be a significant -impediment for remote uses; systems like media servers or set top boxes are -rebooted infrequently. - -## User installs prebuilt OS distribution (other than Google Chrome OS) - -Same as Developer case. The modification to enable developer mode is easy, and -the device is comparably usable. - -## Manufacturer - -A manufacturer can modify the device hardware to allow reprogramming the entire -firmware. Once the modification has been made, firmware development can proceed -at a rate similar to other unprotected devices. - -# Revision History - -1.04 (1 May 2014) - Clean up several minor inaccuracies. - -1.03 (20 Mar 2014) - Generalized from invalid kernel to "boot errors" trigger -recovery mode. - -1.02 (3 Mar 2014) - Changed "unsigned" to "unverified" in kernel discussion. - -1.01 (16 Mar 2010) - Clarified language for disabling write protect (previously -known as 'manufacturer mode'). - -1.0 (15 Mar 2010) - Published to chromium.org. - -0.1 (12 Dec 2009) - Initial version.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/developer-shell-access/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/developer-shell-access/index.md deleted file mode 100644 index 2b1f6075b29..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/developer-shell-access/index.md +++ /dev/null @@ -1,167 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: developer-shell-access -title: Developer Shell Access ---- - -[TOC] - -## Introduction - -Developers often need shell access to their Chromium OS device in order to -sanely debug things in the system. Think opening crosh and typing "shell", and -then logging in as root with "sudo". Or logging into a VT console when the UI is -broken. Or logging in remotely via ssh. - -However, this system is at odds with providing a simple system that is as secure -as possible, so we need to analyze the trade offs. - -## Goals - -* Security should not be impacted when developer mode is turned off, - * both directly -- e.g. user sitting in front of the machine, - * or indirectly -- e.g. an exploit that manages to execute - programs outside of the browser as the same user. - * This means no shell, ssh, VT, etc... access. -* The system should be dynamic. - * Take a normal verified/secure image, put it into developer mode, - and shell access will start working. -* Passwords/ssh keys cannot be static at build time in release images. - * Users may opt to set their own passwords in their own personal - builds. - * Test images may include preset authentication (discussion of - this is ongoing, but out of scope of this doc). -* Users can enable password protection on the fly. - * Keeps people from sitting down, getting a shell, and then - running sudo on your system. - * Passwords protect all avenues of access equally. - -## Summary - -TODO(vapier) - -## Buildtime - -TODO(vapier) - -## Runtime - -### Developer Mode Disabled - -#### VT Switching - -When X is launched by the session manager, it is passed the -maxvt flag set to -0. This way X itself ignores the key combos. - -#### Sysrq Magic Key - -The hotkey-access.conf script will turn off all sysrq requests except for the -"x" key by updating `/proc/sys/kernel/sysrq`. - -#### Crosh Shell - -The crosh script is still available, but it does not allow access to the "shell" -command (among others). - -#### SSH - -The ssh sever is not included in the base image, so it will never autostart. If -it was started somehow, then the sections below apply (which is to say, it still -wouldn't allow logins). - -#### sudo/su - -These cannot be run directly (as no shell is available), but even then, access -is denied via pam. - -#### pam - -A custom pam stack ("chromeos-auth") is installed that handles authentication -for the "login" and "sudo" services. When developer mode is disabled, this stack -will skip itself and continue to the normal system stacks. - -For more details on pam, see [The Linux-PAM System Administrators' -Guide](http://www.linux-pam.org/Linux-PAM-html/Linux-PAM_SAG.html). - -#### groups - -The chronos account is not part of the admin groups that would implicitly grant -access (e.g. `wheel`). - -#### passwords - -The system password database (/etc/shaddow) is in the read-only rootfs and -cannot be modified. The default images will list accounts with passwords set to -"\*" (so that password authentication will fail). - -The user custom dev mode password is not checked at all (see the pam section -above). - -### Developer Mode Enabled - -#### VT Switching - -When X is launched by the session manager, it is passed the -maxvt flag set to -2. This allows access to the VT2 console. Access is controlled by pam. - -#### Sysrq Magic Key - -The hotkey-access.conf script will enable all sysrq requests. - -#### Crosh Shell - -The crosh script allows access to the "shell" command (among others). - -#### SSH - -If it is launched by hand, or using a test image that autolaunches it, access is -controlled by the sections below. - -#### sudo/su - -Access is controlled by pam. - -#### pam - -A custom pam stack ("chromeos-auth") is installed that handles authentication -for the "login" and "sudo" services. When developer mode is enabled, this stack -will: - -* If a custom devmode password is set - * That password is allowed for authentication -* If a custom devmode password is not set - * If the account is in /etc/shadow - * Access is not allowed - * If the account is in /etc/shadow but no password set yet - * Passwordless access is granted - * If the account is in /etc/shadow with a password set - * Access is not allowed - -Note that this only applies to this particular stack. Other pam stacks may -allow/deny independently. - -For more details on pam, see [The Linux-PAM System Administrators' -Guide](http://www.linux-pam.org/Linux-PAM-html/Linux-PAM_SAG.html). - -#### groups - -The chronos account is not part of the admin groups that would implicitly grant -access (e.g. `wheel`). - -#### passwords - -The system password database (/etc/shaddow) is in the read-only rootfs and -cannot be modified. The default images will list accounts with passwords set to -"\*" (so that password authentication will fail). - -The user may set a custom password at runtime with the `chromeos-setdevpasswd` -script which is checked by pam. - -## References - -* http://crbug.com/182572 -* http://crbug.com/217710
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/disk-format/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/disk-format/index.md deleted file mode 100644 index 379c29dc3ed..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/disk-format/index.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: disk-format -title: Disk Format ---- - -## This page has moved to <https://chromium.googlesource.com/chromiumos/docs/+/HEAD/disk_format.md>. Please update the link that brought you here. diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/EC-3PO- The EC console interpreter (1).png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/EC-3PO- The EC console interpreter (1).png.sha1 deleted file mode 100644 index 0ac01ce3963..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/EC-3PO- The EC console interpreter (1).png.sha1 +++ /dev/null @@ -1 +0,0 @@ -1ce2ef6046b34ac98e235d918c766bbe54334b96
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/EC-3PO- The EC console interpreter.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/EC-3PO- The EC console interpreter.png.sha1 deleted file mode 100644 index 138bcfac772..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/EC-3PO- The EC console interpreter.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -8c87302945ad93926c3af0d0c98db38e1282efc4
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/index.md deleted file mode 100644 index 3a827cf606a..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/ec-3po/index.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: ec-3po -title: 'EC-3PO: The EC console interpreter' ---- - -This page has been [moved to the new CrOS EC markdown -docs](https://chromium.googlesource.com/chromiumos/platform/ec/+/HEAD/docs/ec-3po-design.md); -please visit it there.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/bootloader_flowpng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/bootloader_flowpng.sha1 deleted file mode 100644 index 4e1910f7909..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/bootloader_flowpng.sha1 +++ /dev/null @@ -1 +0,0 @@ -78d314543ff53965f1408aace1d438f2155ca021
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/firmware_updaterpng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/firmware_updaterpng.sha1 deleted file mode 100644 index 32d76aceff9..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/firmware_updaterpng.sha1 +++ /dev/null @@ -1 +0,0 @@ -3c0e4c2bcebbf6260a9f7b1d1db66fc3441e4813
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/index.md deleted file mode 100644 index e0c2ce52ff0..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/index.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: filesystem-autoupdate-supplements -title: File System/Autoupdate Supplements ---- - -[TOC] - -This document provides a set of brief Chromium OS design supplements, covering -areas related to the file system and autoupdates. - -## Reinstallation - -Reinstallation requires a recovery image (on a USB drive or SD card) to be -attached to the system. When the system boots, the user may force reinstallation -by forcing the firmware into recovery mode by pressing a recovery button. The -exact details of how the firmware accomplishes this are outside the scope of -this document. -The recovery image performs the actions described in the following diagram: - - - -## Firmware updating - -The firmware discussed in this document references a firmware designed by Google -for Chromium OS-based devices. We recommend that devices that support the Google -firmware use that firmware; that lets them take advantage of the boot time, -security, and recovery features discussed in the [Firmware Boot and -Recovery](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery) and -[Verified Boot](/chromium-os/chromiumos-design-docs/verified-boot) documents. -The firmware is covered in more detail in the [Firmware Boot and -Recovery](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery) -document, but here is how it will work from the autoupdate perspective. -We plan to generally update the firmware after successful boot of a new image -that contains new firmware to install, but we will be able to force installation -of new firmware before an update is applied if necessary. -The firmware contains a stub that can boot into either of two on-chip boot -loaders (A or B). The stub first tries to boot from A. If A's checksum fails, it -boots B. If B's checksum also fails, it goes into recovery mode. -The firmware updater is a userspace program that runs on system boot. It -performs the following actions: - - - -## Bit rot evasion (corruption on the hard drive) - -Note: it's unclear how much we expect to be impacted by drive corruption. -In this scheme, we take advantage of the fact that we could have two identical -system partitions. -Note that this is an early design that may change over time based on feedback. - -We use the extra bits in the partition table as follows. Note that we have 8 -bits in each byte reserved for the bootable flag. This gives us 7 unused bits -per partition for the boot loader. - - - -0x80 Bootable flag: this partition can be booted. This bit being set implies -that it's a good partition to boot from; it's been vetted. 0x40 Tryboot flag: -this partition, which must be bootable, should be booted *this* time, in the -event of multiple bootable partitions being found. 0x20-0x04 Unused. 0x02-0x01 -Attempt boot counter, also known as the "remaining_attempts" flag. This takes -precedence over bootable/tryboot. For more information, see the [File -System/Autoupdate](/chromium-os/chromiumos-design-docs/filesystem-autoupdate) -document. - -The boot loader flow is then: - - - -Open issue: We could use a third system partition. This comes at a cost of -space, but lets us *always* have a backup partition ready, even midway through -an autoupdate. If a system partition is 200MB, we might do this. If it's 1 GB, -we might not want to. - -## Stacking updates - -Updates will be stackable. In other words: - -Updates can be applied, one after another as they come out, without a reboot. -So, for example, if you boot into version n, then update the other partition to -n+1 but don't reboot, you're still running n. Then, if n+2 comes out, we will -update from n+1 to n+2 in the other partition, while remaining booted into n. - -## Forced reboot - -We will be able to force a reboot should we need to apply an emergency security -update. diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/partition_extra_bitspng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/partition_extra_bitspng.sha1 deleted file mode 100644 index 50a3fefab2a..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/partition_extra_bitspng.sha1 +++ /dev/null @@ -1 +0,0 @@ -7c003fd220f32205a68c7be80583e170f582c8ef
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/recovery_imagepng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/recovery_imagepng.sha1 deleted file mode 100644 index 16b2d05588c..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements/recovery_imagepng.sha1 +++ /dev/null @@ -1 +0,0 @@ -db186b1b40fa1bab9c3f4d5671102fce435ce082
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate/ChromeOSFilesystemAutoupdate.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate/ChromeOSFilesystemAutoupdate.png.sha1 deleted file mode 100644 index af4ea6b5d64..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate/ChromeOSFilesystemAutoupdate.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -eec9fe00e6f6cff1a5fbf01e7c1a0d5b5573f349
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate/index.md deleted file mode 100644 index 0102a9aae50..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/filesystem-autoupdate/index.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: filesystem-autoupdate -title: File System/Autoupdate ---- - -[TOC] - -## Abstract - -This document describes the Chromium OS file system and the autoupdate system. - -* We expect regular updates to Chromium OS to add new functionality, - improve system performance, and enhance the user experience. -* The autoupdate mechanism aims to provide seamless and secure updates - to the latest version of Chromium OS. No user interaction is - required. -* Updates usually come in the form of deltas (i.e. only the parts of - the system that changed are downloaded) which are downloaded to a - backup boot partition. Upon reboot, the backup partition becomes the - primary. -* In case there is a problem with the update, the system can revert to - using the previous partition. -* One way in which we secure Chromium OS is by keeping the partition - containing the OS and other system files read-only. All temporary - user state (such as browser caches, cookies, etc) is stored on a - separate partition. - -## Goals - -The autoupdate system has the following goals: - -* Updates are delta-compressed over the wire. -* Delta updates are quick to apply. -* Root file system is mounted read-only (another mount point is - read/write for storing state). -* Device can automatically revert to previous installed version of the - OS if the newly installed OS fails to boot properly. -* Updates are automatic and silent. -* Updates should be small. -* System protects users from security exploits. -* System never requires more than one reboot for an update. -* There's a clear division between the storage locations on the drive - for system software and for user data. -* The new update must be set bit-for-bit from the server, since we - will be signing each physical block on the system partition. (For - more information, see the [Verified - Boot](/chromium-os/chromiumos-design-docs/verified-boot) design - document.) - -## Partitions - -A drive currently contains at least these partitions: - -* One partition for state resident on the drive (user's home - directory/Chromium profile, logs, etc.)—called the "stateful - partition." -* Two partitions for the root file system. - -### Root file system - -Only one of the two partitions designated for the root file system will be in -use at a given time. The other will be used for autoupdating and for a fallback -if the current partition fails to boot. -While a partition is in use as the boot partition, it's read-only until the next -boot. Not even the autoupdater will edit the currently-booted root file system. -We will mount the stateful partition read-write and use that for all state that -needs to be stored locally. - -During autoupdate, we will write to the other system partition. Only the updater -(and apps running as root) will have access to that partition. - -## **The update process** - -### Successful boot - -The update process relies partly on the concept of a "successful boot." At any -given point, we will be able to say one of the following things: - -* This system booted up correctly. -* This system booted up incorrectly. -* The system is currently attempting to boot, so we don't know yet. - -We consider a boot successful if the updater process can successfully launch. -Once a system has booted successfully, we consider the other root partition to -be available for overwriting with an autoupdate. - -### Limiting the number of boot attempts - -An updated partition can attempt to boot only a limited number of times; if it -doesn't boot successfully after a couple of attempts, then the system goes back -to booting from the other partition. The number of attempts is limited as -follows: -When a partition has successfully been updated, it's assigned a -remaining_attempts value, currently 6. This value will be stored in the -partition table next to the bootable flag (there are unused bits in the GPT that -the boot loader can use for its own purposes). The boot loader will examine all -partitions in the system; if it finds any partition that has a -remaining_attempts value > 0, it will decrement remaining_attempts and then -attempt to boot from that partition. If the boot fails, then this process -repeats. -If no partitions have a remaining_attempts value > 0, the boot loader will -boot from a partition marked bootable, as a traditional boot loader would. - -### Diagram - -Here's a diagram of the boot process: - -<img alt="image" -src="/chromium-os/chromiumos-design-docs/filesystem-autoupdate/ChromeOSFilesystemAutoupdate.png"> - -## Supplements - -For a detailed design into how delta updates are generated and applied, see: -[Autoupdate Details](/chromium-os/chromiumos-design-docs/autoupdate-details). -For supplementary information about related material, see the -[Filesystem/Autoupdate -supplements](/chromium-os/chromiumos-design-docs/filesystem-autoupdate-supplements) -document. - -## **Other notes** - -* All updates directly overwrite the new partition and require no - temporary space. -* We download the update over HTTPS, requiring a server certificate - signed by the one Certification Authority that issues OS provider's - SSL certificates. -* Updates are hashed and checksummed. The update servers may sign - updates as well.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/ChromeOSFirmwareBootandRecoveryPUBLISHED.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/ChromeOSFirmwareBootandRecoveryPUBLISHED.png.sha1 deleted file mode 100644 index 7756bbfff8f..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/ChromeOSFirmwareBootandRecoveryPUBLISHED.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -ce8a9f66d8478465edd3f8fbdd5093ca4042ea38
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/eeprom-mappng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/eeprom-mappng.sha1 deleted file mode 100644 index b7f1b07125a..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/eeprom-mappng.sha1 +++ /dev/null @@ -1 +0,0 @@ -c9cc47e20ceabee1edf8c845d25706f0188d1e29
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/flowchart1png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/flowchart1png.sha1 deleted file mode 100644 index a53bcccae45..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/flowchart1png.sha1 +++ /dev/null @@ -1 +0,0 @@ -b732e2208e02649607ff8c8acff9ce1beb10d2a6
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/index.md deleted file mode 100644 index a11718e3382..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/index.md +++ /dev/null @@ -1,695 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: firmware-boot-and-recovery -title: Firmware Boot and Recovery ---- - -[TOC] - -## Abstract - -* The layout and structure of firmware for Chromium OS is designed for - security (see [Verified - Boot](/chromium-os/chromiumos-design-docs/verified-boot) - documentation), recovery and development. -* All firmware will contain a recovery code path, which will restore - the machine to its original Chromium OS state. This recovery code - path will be initiated either when any chain in the boot path is not - verified or when a user manually triggers recovery mode, likely via - an explicit recovery button on the device. -* Chromium OS wants to support developers as well. Developers are - provided with a means of running alternate software. In the - alternate boot paths, the user is notified that they are not running - a boot path provided as part of Chromium OS. -* The boot and recovery procedures outlined will be implemented and - required for all Chromium OS platforms regardless of architecture - (ARM/Intel/etc...). - -This document describes the firmware boot process, including detection and -recovery of corrupted or hacked firmware/software. - -## Potential problems - -The firmware boot process must be able to detect the following problems and, if -possible, repair them. - -### Firmware - -1. Incomplete update: An update of the firmware is interrupted. This - leaves the portion of the firmware which was being updated in an - unknown or corrupt state. For example, if the update is interrupted - after a firmware block is erased but before it is reprogrammed, that - block is empty. -2. Attack: An attacker compromises the software and is able to - reprogram the firmware. For example, an exploit of an unpatched - kernel vulnerability. In this case, both the main and backup - firmware may be compromised. -3. Corruption: The EEPROM holding the firmware becomes corrupted in the - sectors containing writable/updatable firmware. - -**Software** - -1. Incomplete update: An update of the software on the drive is - interrupted. This leaves the rootfs partition in an unknown state. -2. Attack: An attacker compromises the software and is able to rewrite - the data on the drive (rootfs or partition table). -3. Malicious user: A malicious user installs developer mode onto the - device, leaving behind a trojan, then returns the device. -4. Corruption: The drive becomes corrupted in the partition table or - rootfs partition. -5. Crash: Device crashes on boot due to bad software. For example, the - device is updated with the wrong image. This prevents the normal - autoupdate process from running. - -**Out of scope** - -The following problems are outside the scope of this document in its current -form: - -1. An attacker with physical access to the device opens the device and - alters its innards. This includes: - * Jumpering the EEPROM to gain write access to the read-only - portion. - * Replacing the EEPROM. - * Otherwise altering the circuit board (adding piggyback chips, - etc). -2. Exploits involving the normally-writable data partition. - * Changing bookmarks to point to sites which download HTML5 - malware, etc. - * Malformed bookmarks file or image which causes a buffer overrun - when parsed - * Changing the preferred wireless network to a malicious one which - logs/alters packets -3. Exploits involving other processors (embedded controller, modem - processor, etc.) - -## Design decisions - -### Boot needs to start securely - -In order to attempt a secure boot, the initial boot stub needs to perform a -minimum level of initialization to verify the next piece of boot code before -handing off to that code. - -To prevent accidental or intentional corruption of the known-good boot stub, -this code must be in a portion of memory which is not field-writable. Many -EEPROM devices have an external pin (==WP==) which can be pulled low to write -protect the upper portion of the EEPROM. This has a number of benefits: - -* Devices are writable at time of manufacture (as opposed to true - ROMs, which are fixed at time of ROM manufacture). -* Devices can be made writable for firmware development by simple - hardware modification. -* Both readable and read-only ROM are provided by a single device. - Simpler board design, fewer parts, lower cost. -* Any attacker who can open the case and modify the hardware to write - to the protected upper portion of ROM could also simply replace a - true ROM with a reprogrammed part, so this isn't significantly less - secure than a true ROM. - -On ARM platforms, the initial boot ROM may be in the same package as the -processor. This is even more secure compared to a separate EEPROM. - -### Writable firmware should have a backup - -To protect against a failed firmware update, the writable portion of the -firmware (responsible for doing the remainder of chipset and storage setup and -then bootstrapping the kernel off the storage device) should exist in two -copies. In the event the first copy is corrupt, the device can boot normally off -the second copy. This is similar to the design for the [file -system](/chromium-os/chromiumos-design-docs/filesystem-autoupdate), which has -two copies of the root partition. - -### Recovery firmware must be read-only - -Recovery firmware must be able to take over the boot process if the boot stub -determines that the normal writable firmware is corrupt, or if the user manually -boots the device into recovery mode. - -To prevent the recovery firmware from being corrupted by a firmware update or a -software-based attack, it must be in the same read-only portion of the EEPROM as -the boot stub. - -### Recovery firmware does not need to access the network - -The recovery process should not require firmware-level network access by the -device being recovered. The recovery procedure can involve a second computer, -which is used to create recovery media (for example, a USB drive or SD card). It -is assumed that second computer has network access. - -This simplifies the recovery process, since the recovery firmware only needs to -bring up enough of the system to bootstrap a Linux image from local storage. -That image can then take care of reflashing the EEPROM and reimaging. - -It is not necessary to implement a full network stack with WiFi configuration in -the recovery firmware to support PXE boot. PXE boot introduces a number of -complications: - -* Need to initialize more hardware to bring up wireless, keyboard, - etc. -* Need to implement a full network stack. -* Makes recovery an interactive process, including the user entering - their SSID and WPA key, which the user may not know. -* Unlikely to work for public WiFi access points; these often redirect - http access to a login screen, navigation of which which would - necessitate a full browser in the recovery firmware. -* Unlikely to work for cellular networks (3G/4G/etc...), if that - requires more complicated drivers or configuration. - -All of these issues would need to be resolved, and the resulting firmware must -be correct at the time the device ships, because recovery mode firmware can't be -updated in the field. It is unacceptable to ship a mostly-working PXE solution, -assuming that the user can fall back on a second computer in the event PXE -recovery fails. The only time the user would discover PXE recovery didn't work -is when the user is relying on it to repair their computer. - -More information on wireless network boot is here: -<http://etherboot.org/wiki/wirelessboot>. - -### Recovery firmware should tell the user how to recover - -If the recovery firmware finds a USB drive/SD card with a good recovery image on -it, it should boot it and use that to recover. The software in that recovery -image will have its own user interface to guide the user through recovery. - -If the recovery firmware does not find a good recovery image, it needs to tell -the user how to use a second computer to build that recovery image. - -The preferred way to do this is to initialize the screen and show recovery -instructions to the user, including a URL to go to in that second computer's web -browser. Note that recovery instructions need to be displayed in the correct -language for the user. - -It is desirable for the recovery instructions and/or recovery URL to include a -code for the device model. This allows the destination website to: - -* Provide the proper recovery image for that device model. -* Describe the recovery procedure specific to that device model. For - example, if the device has a SD card but no USB port, the recovery - procedure would indicate that a SD card is necessary, and would not - discuss the possibility of recovering using USB. -* Display graphics appropriate for the device model. For example, - showing the location of the USB or SD card port. - -### Users must be able to manually trigger recovery mode - -If the writable firmware and/or rootfs have valid signatures but don't work (for -example, the user somehow managed to get an ARM kernel on an x86 device), the -user needs to be able to force recovery mode to run. - -This can be done by having a physical reset button somewhere on the device. When -this button is pressed during power-on, the device goes straight to the recovery -firmware without even looking at the writable firmware or file system. - -Some options for the recovery button: - -* It could be a button attached to a GPIO on the main processor. In - this case, the boot stub would initialize the GPIO and read its - state at boot time. -* It could be a button attached to a subprocessor such as the Embedded - Controller (EC). In this case, the boot stub would need to request - the button state from the EC at boot time. -* It could be one of the keys on the keyboard, though this creates the - undesirable possibility of accidentally entering recovery mode. - * This is undesirable if the only interface to the keyboard is - USB, because USB firmware is more complex and the USB hardware - interface can take a significant amount of time to initialize. - * Some devices use a subprocessor to read the keyboard. In this - case, initiating recovery mode is much like the previous option. - * The keyboard could bring out the press-state of one of its keys - to a separate I/O line, which could be attached to a GPIO on the - processor or to a subprocessor. - -Since the recoverability of Chromium OS is one of its key features, we seek to -have a dedicated "recovery mode" button. - -### Support developers / l33t users installing their own software - -To provide a relatively trustable boot for the majority of users, we need to -control all the read-only firmware at the beginning of the boot process. - -To support developers, at some point during the boot process, we need to hand -off to code self-signed by someone else. - -The initial devices will allow hand off at the point the kernel is loaded and -its embedded signature is checked. This can produce one of three results: - -* Trusted kernel: The kernel has a valid signature, and the signing - key is known to the firmware. This is the normal case for production - devices. -* Developer kernel: The kernel has a valid signature, but the key used - to sign the kernel is not known to the firmware. This is the case - when a developer builds and self-signs their own kernel. -* Corrupt kernel: The kernel fails its signature check. - -Once the chain of trust departs from the standard Chromium OS boot chain, we -need to indicate this clearly to the user of the device. This prevents malicious -attackers from giving users a modified version of Chromium OS without the user -knowing. We likely will need to show a warning screen which includes the -following elements: - -* A warning that the standard image of Chromium OS is not running -* A means of reverting back to the standard Chromium OS image -* A means of allowing the user/developer to proceed down the - "untrusted" path - -It is desirable for the warning screen to have a timeout, so that Chromium OS -devices with developer images can be used in unattended applications (for -example, as a media server). The timeout should be sufficiently long that a user -can read and respond to it - for example, at least 30 seconds. - -Since language settings will not be available at this stage of the boot process, -any messaging will likely need to be internationalized and displayed in all -possible languages. - -## EEPROM map - -Here is an early guess at sizes and layout of the EEPROM. These sizes may change -as we progress with implementation. - - - -## Firmware block descriptions - -### Boot stub - -Must be at the top of EEPROM, since most processors jump to the top of memory -(0xFFFFFFF0) after internal initialization. - -Contains the "root key" - the official public key used to verify the signature -of the next stage of firmware. The private key is not contained on the device, -and must be protected from all unauthorized access. To reduce exposure of the -private root key, the private root key will be used to sign a second -date-limited or numbered key stored in the rewritable firmware, which is then -used to sign that firmware. Validation of the date or key number could be done -via a TPM module. - -**Pseudocode** - -1. Initialize processor and RAM (and implicitly, those parts of the - north bridge necessary to initialize RAM), using conservative - timings. -2. If user-forced recovery mode, skip to attempt loading recovery - firmware. (The recovery button asserts an I/O line that can be - measured by the firmware.) -3. Check the non-volatile register or memory region for a recovery mode - cookie value. If this is set, some later stage of the boot process - must have failed and requested recovery mode, so skip to attempt - loading recovery firmware. -4. Attempt loading Firmware A. - 1. Copy Firmware A to RAM. - 2. Verify signature of Firmware A using public key stored in boot - stub. - 3. If signature is valid, jump to Firmware A Setup. -5. Firmware A is bad. Repeat for Firmware B. -6. Both A and B are bad. Attempt loading recovery firmware to RAM and - verify signature. If valid, jump to recovery firmware. -7. All recovery options have been exhausted. Catch fire / emit POST - code / etc. - -### Firmware (A/B) setup - -This firmware sets up a minimal set of hardware components so that the boot -loader can load the kernel from the normal boot drive. For example, the SATA or -eMMC controller. - -**Pseudocode** - -1. Initialize chipset / file system sufficiently to jump to Boot Loader - code. -2. Jump to Boot Loader code. - -### Firmware (A/B) boot loader - -The boot loader is only designed to load Chromium OS. We can go directly from -firmware bootstrap to the kernel in the disk. - -**Pseudocode** - -1. **Verify the partition table on the disk looks sane.** -2. **Load kernel A from the disk.** -3. **Verify signature of kernel.** -4. **If signature is invalid:** - 1. **If this is kernel A, retry with kernel B.** - 2. **Else this is kernel B. Both kernels are bad, so set the - recovery-mode cookie non-volatile register and reboot into - recovery firmware.** -5. **If kernel was signed with a public key not known to the boot - loader, this is a developer kernel:** - 1. **Initialize the display.** - 2. **Display scary developer mode warning to user. For example: - "Google Chrome OS is not installed. Press space bar to - repair."** - 3. **Wait for keypress or 30-second delay before continuing.** - 4. **If key pressed was Space bar, Enter, or Esc, jump to Recovery - Firmware.** - 5. **If key pressed was Control+D, dismiss screen.** - 6. **Ignore other key presses.** -6. **Continue booting the kernel.** - -### Boot log - -A boot log will be stored at the bottom of the writable section of firmware. -This will store the following types of events: - -* Each time recovery firmware is run, with information on what - triggered it (manual, bad firmware, bad root filesystem, etc.). -* When a recovery completes, including which actions were taken. - -It does not store information on successful boots. This removes the need to -support EEPROM writing in the normal boot process, and conserves space for log -entries dealing with real errors. - -The boot log can be uploaded as part of the autoupdate process. It can then be -cleared and reused for new log entries. - -### Recovery firmware - -This firmware attempts to recover from bad firmware or rootfs by loading a -recovery image off an external storage device. - -**Pseudocode** - -1. Write a log entry to the boot log indicating why recovery mode was - activated (manual, bad writable firmware, user triggers recovery - mode, etc.) -2. Initialize enough of the chipset to be able to access a storage - device (USB drive, SD card, etc.) -3. Check for an inserted storage device. If no storage device is - present, skip to displaying recovery instructions. -4. Verify the signature of the recovery image, using a public key - stored in the recovery firmware. If the signature is invalid, skip - to displaying recovery instructions. -5. Load and run the recovery image. -6. If we're still here, either no storage device was inserted, or the - storage device did not contain a valid image. Display recovery - instructions: - 1. Initialize the display. - 2. Display instructions. These instructions must be - internationalized and rendered in multiple languages. The - instructions must include: - * A warning that the computer is in recovery mode - * How to obtain a trusted recovery image (for example, - instructions to go to a second computer and browse to a - specified URL for further instructions) - * An indication if an already-inserted storage device does not - contain a valid recovery image (sorry, try again). -7. Wait for a storage device to be inserted, then go back to step 3. - -Most of the recovery work is left to the recovery image loaded from the storage -device. This allows for publishing updated recovery images and instructions. -Because the recovery firmware is etched in stone (well, as electrons in floating -gates) at the time the device is launched, it needs to be as simple and robust -as possible. - -## Recovery image - -This is the software loaded onto a storage device (USB drive, SD card, etc.) -which does the bulk of the recovery work. - -The recovery image will be available for download. - -We will need to provide a downloadable installer for users to use to install the -recovery image on the storage device. This installer should be supplied for all -popular operating systems (Chromium OS, Windows, Mac OS X, Ubuntu Linux). The -installer will: - -* Prompt the user to select a destination for the recovery image - * Ideally, the user will only be able to select destination - devices appropriate for recovery of the laptop model they are - attempting to recover. - * Also, the user should only be able to select removable devices. - We don't want them trashing their hard disk. -* Warn the user that this will erase whatever is on that destination, - and prompt for confirmation -* Reformat the destination storage device -* Install the recovery image on the storage device -* Prompt the user to insert the storage device into the Chromium OS - device - -Ideally, the installer would be able to back up the current data on the -destination device, before reformatting and writing the recovery image. When -recovery is complete, the user could re-insert the destination device and have -the original contents restored. - -The recovery image should contain an entire clean copy of the firmware and -rootfs. This way, a user can make a recovery device ahead of time. For example, -if a user is going to be somewhere with poor connectivity, they could make a -recovery USB drive at home and keep it in their bag. Alternately, the system -could come with a recovery device (though users might end up reformatting it and -filling it with images of kittens). - -The recovery image on the storage device would do something like the following: - -1. Initialize the display and tell the user what's going on. Reassure - them that their Chromium OS system is trying to recover. -2. If developer firmware is detected, give the user a choice of "JUST - FIX IT" or "Scary Settings for L33t H4x0rs". If the user picks the - latter: - 1. Give the developer more control over each stage in the rest of - the recovery. - 2. For example, a developer might not want to wipe and replace - rootfs or stateful data, or might want to make a backup of that - data. -3. Run system tests. It's possible that the boot problems are due to - bad hardware. Now's the time to detect that. - 1. If hardware is sufficiently bad that the rest of recovery can't - be run, show the user information on how to return it. -4. Verify both firmware A and firmware B are up-to-date and valid. Load - known good firmware as needed. -5. Verify rootfs-A and rootfs-B are up-to-date and valid. Load known - good rootfs images as needed. -6. Wipe the stateful data. Anything the user cares about should be in - the cloud anyway. If the user's been hacked, wiping the data will - put the device back in a known good state. - 1. Optionally, it can inform the user that data on the system will - be erased. - 2. This message should emphasize that most data is saved to the - cloud, so that the user is more likely to proceed with recovery. - 3. It is less desirable to give the user a choice whether or not to - delete the stateful data. Most users would pick the less scary - but also less secure option of NOT deleting data - but this - leaves them vulnerable to persistent hacks on the data - partition, such as manipulating their /etc/hosts file or - bookmarks. -7. If recovery mode was manually triggered, ask the user why they're - running it. - 1. Couldn't even log in; crash during boot - 2. Kept crashing after boot - 3. Worried my system was hacked - 4. Hey, what's this button do? -8. Write an entry to the boot log describing repairs performed. - -### Recovery via Chromium OS - -The recovery installer should run on a healthy Chromium OS system. That is, it -should be able to download the recovery image, reformat the storage device, and -copy the recovery image to it. - -* This enables a healthy Chromium OS system to create its own recovery - device in advance. - * Perhaps we should periodically advise users to update their - recovery device? -* It also enables one Chromium OS system to download a recovery image - for a different model. (A user has a corrupt system, so they go to - their friend's house and use the friend's Chromium OS system to make - a recovery image). - -### Making recovery images read-only - -If the recovery image is inserted into a powered-on and hacked system, the -hacked software could delete or corrupt it. This won't cause the recovery -firmware to load that corrupt image, because the corrupt image will fail the -signature check. It will be annoying to the user, who will need to reflash the -storage device. - -This is more of an issue if we want to include an internal recovery image (for -example, on an internal storage device). - -The SD standard specifies a physical write-protect notch for SD cards, similar -to those on a 3.5" floppy disk. Unfortunately, the implementation of the -write-protection is purely software, so pwned drivers can choose to ignore the -write-protect detect signal. - -Some USB drives have a write-protect switch on them. In this case, the -protection is handled by the USB drive itself. - -Some eMMC chips have a number of protection mechanisms including: - -* an external LOCK# pin which can be used to make the device read-only -* a write-once bit which makes the device read-only until reboot - -Since these chips come in sizes up to 2GB (~$10 at stores), they provide a -possible place to store a recovery image. Some system architectures may be able -to use the eMMC drive to hold the main firmware image also. - -If we have an internal storage device, it's possible we could wire it so it's -only enabled if the recovery button is pressed at boot time. For example, we -could use that button to latch up a circuit which powers that device, so that it -will remain powered during that boot only. - -### Using recovery mode to load developer mode firmware/software - -If the rootfs image on the SD card is signed by someone else, the recovery image -will display a screen similar to the Developer Mode screen. Instructions on this -screen should include: - -* Notice that the recovery image is signed by someone else -* How to obtain a trusted recovery mode image -* Instructions on installing a trusted recovery image -* A means of allowing the user/developer to proceed down the - "untrusted" path - -Note that this screen must also be internationalized / rendered in multiple -languages. This is potentially much easier, since by the time we're doing this -the recovery image has booted a full OS with GUI. - -## Boot flowchart - -The three main downward flows in this chart show: - -* Start of boot in read-only firmware -* Normal continuation of boot, loading kernel from internal drive -* Recovery mode - - - -## Other notes - -### Verification of the rest of the rootfs - -The firmware boot process above describes a way to verify all code from the -start of boot through hand off to the kernel. - -The kernel is responsible for verifying the rest of the data in the rootfs - for -example, user-mode drivers. If the kernel determines that the rootfs has been -compromised, it can force recovery mode to run by setting the recovery mode -cookie in a non-volatile register and rebooting into recovery mode. - -Alternately, a kernel which determines its rootfs is corrupt can commit suicide -by marking its partition as inactive and then rebooting. The next boot will do -one of the following: - -* If the other root partition is good, the boot loader will run the - other root partition. -* If neither root partition is good, the boot loader will trigger the - recovery mode firmware, which can put the rootfs back in a known - state. - -### When to verify the boot process - -We should develop under the initial assumption that we can verify every boot, -since this provides the most security. - -Signature-checking code does impact startup time. However, this is likely not an -issue for the firmware (< 1 MB) and kernel (< 10 MB) than it is for the -remainder of the rootfs. We assume that we will verify the entire firmware and -kernel every boot. - -### Handling returned systems - -When a Chromium OS system is returned to the store, the store should check to -make sure it still boots. - -* If it boots normally, the store can follow their standard process - for dealing with returns. -* If it displays the Scary Dev Mode screen, the store can either run - recovery mode or refuse the return (in which case the original owner - can go home and run recovery mode, then come back and returns the - system again). -* If it fails to boot, it gets shipped back to the manufacturer. - -Note that it's advisable for stores to run recovery mode on returned computers -anyway, to put them back into a clean state and destroy any user data still on -the device. - -### Testing - -The comprehensive test suite for Chromium OS should include tests of each stage -of the verification process and the recovery procedure, including all decision -points in the pseudocode/flowchart. This will require actual hardware for each -firmware image being tested. - -Testing should also attempt to write to the read-only portion of the EEPROM, to -ensure that it is in fact not writable. - -## Open issues - -### What else do we need to verify? - -This document describes verifying the following: - -* Firmware in the main EEPROM -* Partition table -* The kernel partition - -It explicitly does not discuss verifying: - -* The rootfs (anything after the hand off to the kernel) -* The stateful data partition - -Are there other locations for persistent storage we should be verifying? For -example: - -* Firmware in other components? - * File system - * NIC - * 3G modem - * Embedded controller - * Modem processor (on ARM systems) - * Keyboard ([this apparently is an attack surface on some - systems](http://www.engadget.com/2009/08/04/apple-keyboard-gets-hacked-like-a-ripe-papaya-perp-caught-on-vi)) - * (we disallow option ROMs in the hardware spec, so we don't need - to worry about those) -* Removable user media - -### Can we log firmware crashes? - -* Crashes in boot stub (i.e., north bridge / RAM init) likely aren't - loggable at all. Not enough of the system is running. -* Other crashes in the firmware could write a cookie into a particular - address in SDRAM. - * When the system boots next, it could check that address for a - valid crash cookie. - * If so, assume the current firmware is bad; try the backup - firmware, or go to recovery mode. - * Downside: Intermittent crashes which would be mildly annoying - now prevent the system from attempting to boot at all, since it - would tend to go into recovery mode. -* Crashes in the kernel can be logged to the stateful partition. - -### Chain of verification on ARM? - -Some ARM SOCs contain a modem processor and an application processor. On boot, -only the modem processor is running; its firmware then sets up and starts the -application processor. If the modem firmware is writable, we could be hacked -before the first instruction of our firmware executes. We need a way to secure -the modem firmware. (This is true regardless of the boot order, if the modem -firmware has access to the registers or RAM used by the application processor). - -Such systems could use a Memory Protection Unit or similar such hardware -solution, to prevent the modem and application processors from reading/writing -each other's firmware or memory space, except for specific portions of the -memory space used for inter-processor communication. We are still investigating -what solutions can be used for verified boot. - -### Support for Trusted Platform Module (TPM)? - -A TPM is not required for key verification for the firmware boot and recovery -process described in this document. There's no point in the verification key -being more secure than the code in ROM used to do the verification. If an -attacker can crack the ROM code, they can just make it bypass the TPM check. - -If the boot stub contains a secondary key which is date-limited, it is desirable -to use the TPM to verify the date-limited key is valid. Many TPM modules contain -their own clock/counter, so can be used to defend against turn-back-the-clock -attacks. Similarly, if we use numbered or sequential secondary keys, storing the -highest-seen key number in the TPM can protect against rollback attacks. - -There may be some firmware requirements to set up the TPM so that subsequent -activities (3G authentication, etc.) can get access to the keys stored in the -TPM. diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/s0hDprW1DevZ8KuyE5lsKIg_903.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/s0hDprW1DevZ8KuyE5lsKIg_903.png.sha1 deleted file mode 100644 index 545ec9c3a70..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery/s0hDprW1DevZ8KuyE5lsKIg_903.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -f498f539f31983d6987209a189b22013d83a7937
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-updates/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-updates/index.md deleted file mode 100644 index 254973b6649..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/firmware-updates/index.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: firmware-updates -title: Firmware Updates ---- - -Firmware Update Processes and Utilities - -**Flashrom** - -Chromium OS currently uses Flashrom as the firmware update tool of choice for -x86, and has cross-platform ports for ARM, MIPS, and others. Flashrom is an -open-source, cross-platform EEPROM programming utility with an active community -and a large combination of supported host chipsets, flash memory devices, and -external programmers. The code is generally adaptable to a wide variety of -chipsets and interfaces and is thus a good candidate for servicing the needs of -Chromium OS platforms. -Flashrom main URL: http://www.flashrom.org - -Flashrom @ Chromium.org documentation: -http://[goo.gl/rEAwz](http://goo.gl/rEAwz) \[ [Long -URL](https://docs.google.com/document/pub?id=1H8zZ3aEMZmfO4ZEsWbHooUdGOgxy5LTPJ19YbaDsxdQ) -\]
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/hardening-against-malicious-stateful-data/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/hardening-against-malicious-stateful-data/index.md deleted file mode 100644 index be02b1ed3c1..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/hardening-against-malicious-stateful-data/index.md +++ /dev/null @@ -1,256 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: hardening-against-malicious-stateful-data -title: Hardening against malicious stateful data ---- - -## Motivation - -Chrome OS has Verified Boot, which is designed to make sure that the system will -only run binaries that are trusted, starting at firmware and continuing all the -way through the boot process to Chrome. It turns out this is not sufficient, as -the data dependencies of code running during boot pull unverified data from the -stateful partition. This may be configuration data, device state indicators, -data caches etc. It turns out that data dependencies in the boot process pose a -security risk: Attackers that have a root exploit control the stateful -partition, allowing them to stage malicious file system state (file contents, -symlinks, directory layout, etc.) that will affect a subsequent boot in a way -that re-exploits the device, thus providing a vehicle for the attacker to get a -persistent exploit. - -[Both](https://bugs.chromium.org/p/chromium/issues/detail?id=351788) -[examples](https://bugs.chromium.org/p/chromium/issues/detail?id=648971) of -persistent Chrome OS exploits that have been reported to us up to now exploit -this (symlink attacks) as their persistence mechanism. Thus, we need to: - - Eliminate data dependencies on stateful data as much as possible from the - boot process. - - Mitigate any legit and required dependencies to move the bar to a point - where exploiting them becomes really hard, if not impossible. - -This document describes approaches to achieving these goals.. - -## Anatomy of a persistent attack - -Here's a quick summary of the usual steps involving a persistent exploit. This -is useful for reference when assessing mitigations: - - Attacker exploits the running system. This might be in the form of Chrome - exploit, a system daemon exploit, a full root exploit, or even a kernel - exploit. - - Attacker manipulates state carried over across reboots. Depending on what - privileges the attacker managed to acquire, they have different options - here. A full root or kernel exploit will allow to make arbitrary changes to - the stateful file system for example, as well as other stateful storage such - as TPM and VPD. If the attacker has "only" a Chrome exploit, they may still - manipulate the stateful file system subtrees that are writeable by user - chronos. Similarly, if the attacker controls a system daemon, they can - manipulate that process' state (assuming the system daemon is running within - a sandbox). Some approaches to manipulation: - - Store malicious data in a file that is read and parsed, interpreted etc. - after reboot. - - Manipulate the file system layout: Insert symlinks or hard links that - will redirect write actions during boot (such as mkdir, writing cache - files etc.). - - Manipulate file system state to trigger unintended execution paths - (example: make files unreadable / wrong type to trigger - untested/inappropriate error handling or fallback behavior). - - System reboots. - - Re-gaining code execution: Verified boot ensures that kernel and root file - system remain intact, i.e. the attacker can't change code there to - re-acquire control of the system. Instead, the attacker will have to arrange - for the regular boot flow to "take a wrong turn", i.e. trick legit code - running during boot to perform inadvertent actions that will give the - attacker code execution again. This is where the manipulated stateful data - comes in: It will be consumed by init scripts and system services and may - affect their behavior in a variety of ways. Some of the more obvious are: - - Trigger execution of shell scripts stored on stateful. - - Exploiting weaknesses in config parsing to gain code execution within a - system service. - - Manipulating user-installed code that the system will run automatically, - such as extensions installed in a user profile. - -## Mitigations - -The remainder of the document discusses hardening measures that will prevent -certain classes of malicious stateful data to have an adverse effect. Note that -there is no comprehensive solution - any useful device will have to store some -data somewhere (in the cloud if not locally) and users expect to be able to see -their previous state after a reboot. This inherently implies that we need to -carry over state, so some risk will always remain. It makes sense to prioritize -mitigation work to reduce impact of successful exploits. To that effect, we -should start with addressing stateful data dependencies within highly-privileged -code (such as init scripts, system daemons), then work our way towards less -privileged code. - -### Restricting symlink traversal - -[Both](https://bugs.chromium.org/p/chromium/issues/detail?id=351788) -[cases](https://bugs.chromium.org/p/chromium/issues/detail?id=648971) of -persistent Chrome OS exploits that have been reported via the Chrome -vulnerability reward program relied heavily on placing symlinks on the stateful -file system to re-exploit the system after a reboot. The idea is to place a -symlink somewhere on a path written to by a privileged process during boot to -redirect the write to a different location. In some cases, the data that gets -written can also be controlled by the attacker, e.g. when the code in question -is copying files from one location in the stateful file system to another. It -turns out that intentional usage of symlinks is actually very rare on the -stateful file system. Given that and the relative success of symlink attacks, it -makes sense to generally disallow symlink traversal. - -To prevent symlinks to be traversed on the stateful file system, there are a -variety of possible approaches: - -1. Change all code that accesses files to double-check it didn't follow - a symlink inadvertently. Note that the O_NOFOLLOW open flag and - symlink-aware version of system calls (such as lstat() in favor of - stat()) are not sufficient, since they only cover the last path - component. Symlinks in parent components are still followed. A - working way of implementing a symlink traversal check is via - readlink() on /proc/self/fd/N. Unfortunately, it's just not - practical to do this for all file access in shell scripts, 3rd-party - software etc., so while this approach is technically feasible, it - doesn't fly in practice. -2. Add a mount option in a kernel patch, so symlink traversal can be - disabled on a per-mount basis. The BSDs have nosymfollow which - implements this. We have [proposed the same solution for - Linux](https://patchwork.kernel.org/patch/9434587/) upstream, but it - was met with skepticism. Reality is that few Linux distributions be - able to use this meaningfully (it only makes sense when you have a - verified or at least read-only rootfs anyways). We could carry the - patch in the Chrome OS kernel tree indefinitely, but that'll cause - maintenance overhead as file system internals change over time. -3. Scrub symlinks after mount. This would require making a full pass - over the mounted file system and remove any unintended symlinks. - This will adversely affect boot times since we'd have to do this - before starting any init jobs that read stateful data. -4. Use SELinux to apply policy to prevent symlink traversal. This would - require the entire stateful file system to be re-labelled after - mount to make sure there aren't any labels present that would allow - symlink traversal in locations that shouldn't do so. This approach - suffers from the same boot time issue as the previous one. -5. Reject symlinks via the LSM inode_follow_link hook in the Chromium - OS LSM. Implement the logic in a non-invasive way in the kernel to - keep maintenance overhead low. - -After considering the pros and cons of the approaches listed above, we've chosen -to go with the LSM inode_follow_link approach. Design highlights: - -* The inode_follow_link hook can look at the dentry and inode of the - traversed symlink location. We thus need to hook up any information - require to make a decision in such a way that it is accessible from - the inode/dentry. Note that we can't simply fail all symlink - traversal, since there are important cases of legit symlink usage on - the root file system, e.g. the shared library symlinks in /usr/lib. - We also want to be able to selectively allow symlinks for specific - locations even though symlink traversal should generally be blocked - on the stateful file system. -* To achieve per-inode decisions, the code uses fsnotify - infrastructure, which allows to attach "inode marks" to an inode. - This is also used by other kernel subsystems (e.g. the audit code) - and allows callbacks to be invoked when something happens. We only - rely on the notification send on inode destruction so we can clean - up tracked state, but we can use the inode mark to hold symlink - traversal policy for the inode. -* Symlink traversal policy is evaluated for a path in question by - searching up the parent directory chain until an inode is found that - carries a specific traversal policy. This allows setting the policy - to block symlinks on the root inode, which will thus be inherited by - the entire file system. This also allows exceptions to be granted by - setting the policy to allow traversal for a specific directory, - which will grant an exception for the subtree rooted at the - directory. -* Symlink traversal policy can be configured via userspace via - securityfs. -* Whenever an attempt at traversing a symlink runs into a policy that - blocks the access, the system call wiil fail with EPERM and a kernel - warning will get logged. -* The kernel warning collector uploads symlink traversal blocking - warnings via the crash reporter, so we have some insight whether - devices hit restrictions inadvertently. -* The code uses only "public" kernel APIs, i.e. it is entirely - contained within the LSM module. No kernel changes elsewhere are - needed. This keeps maintenance overhead at the absolute minimum. - -To make use of the symlink traversal policy mechanism provided by the LSM, we'll -require a few userspace changes. chromeos_startup will be responsible to -configure symlink traversal to be blocked on the stateful and encrypted stateful -file systems. We'll need to allow a few exceptions: - -* /var/cache/vpd, /var/cache/echo: These directories contain backwards - compatibility symlinks set up by dump_vpd_log. We should ideally - clean up consumers of these symlinks and remove the exception. -* /var/lib/timezone: This directory contains a symlink to the actual - time zone file and is maintained by Chrome. This is a low-risk - exception as this path is generally only sees read-only access by - privileged code. -* ~~/var/log: The log directory contains a number of convenience - symlinks that point at the "latest" log file for ui, chrome, - power_manager etc. logs. This is somewhat higher risk since - privileged system daemons write to files in /var/log. We should - consider restricting this further, but will grant an exception for - now.~~ The risk of symlinks in /var/log being abused has been - [mitigated](https://bugs.chromium.org/p/chromium/issues/detail?id=916152). -* /home: When using ext4 encryption, encrypted user data actually - resides within the stateful file system and encryption is handled on - a per-file basis. To avoid blowing up scope considerably for initial - roll-out, we'll explicitly allow /home so user data maintained by - Chrome can still use symlinks. Note that this is low risk for the - most part since the majority of the access are happening within - chronos user context. There is some data that is access by - privileged daemons in the /home/chronos/root/ bind-mounts though. We - should eventually lock down symlink traversal either for the entire - user data subtree, or at least for the locations access by system - daemons. -* /mnt/stateful_partition/dev_image: This directory contains utilties - that are convenient for developers running in dev mode. We'll grant - an exception for this when setting up this location for developer - mode. -* /mnt/stateful_partition/unencrypted/art-data: ARC++ uses symlinks in - /mnt/stateful_partition/unencrypted/art-data to make host compiled - code available in ARC++ container without copying. - -Further exceptions for symlink traversal restriction can be added in justified -cases. - -### Enabling generic per-inode access control policies and blocking FIFO access - -Building upon the existing framework for adding an "inode mark" to enable -per-inode decisions about symlink traversal, we have extended the code to allow -for generic per-inode access control policies regarding other types of accesses -to the file system. In this way, we can support the use of additional hooks in -the Chromium OS LSM which consult the "inode mark" when making decisions about -other file system security policies. For example, one recent -[exploit](https://bugs.chromium.org/p/chromium/issues/detail?id=766253) modified -a file on the stateful file system to convert it from a normal file into a FIFO -in order to disable the execution progress of a program that opened the file for -reading. In light of this, we have added an additional policy to the "inode -mark" metadata that allows us to deny opening of FIFOs on the stateful file -system in addition to restricting symlink traversal. We use the file_open hook -in the LSM to check the inode metadata when a FIFO is being opened on the -system. All other details are the same as described above for restricting -symlink traversal. As FIFO usage is even more rare than usage of symlinks on the -stateful file system, the only exceptions to this policy are: - -* /mnt/stateful_partition/dev_image: Used for developers running in - dev mode. -* /mnt/stateful_partition/home: ARC++ /data directory is mounted under - this path, and uses FIFOs (e.g. java.io.tmpdir is expected to - support FIFOs, and is set by the Android framework to use - /data/user/0/<package_name>/cache/). - -As above, further exceptions for FIFO access can be added in justified cases.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/index.md deleted file mode 100644 index 60f13cf6cf4..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/index.md +++ /dev/null @@ -1,119 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -page_name: chromiumos-design-docs -title: Design Documents ---- - -This page has links to a preliminary set of Chromium OS design docs. (Also see -the [User Experience](/user-experience) pages.) - -**Note: These designs will evolve significantly, based on implementation -challenges, community feedback, and other factors. In many cases, these design -docs represent early planning and may change direction. We'll publish more -design docs in the coming months.** - -<div class="two-column-container"> -<div class="column"> - -### General - -* [Software - Architecture](/chromium-os/chromiumos-design-docs/software-architecture) -* [Source Code - Management](/chromium-os/chromiumos-design-docs/source-code-management) -* [Upstream First](/chromium-os/chromiumos-design-docs/upstream-first) - -### Security - -* [Security - Overview](/chromium-os/chromiumos-design-docs/security-overview) -* [Protecting Cached User - Data](/chromium-os/chromiumos-design-docs/protecting-cached-user-data) -* [System - Hardening](/chromium-os/chromiumos-design-docs/system-hardening) -* [Chaps Technical - Design](/developers/design-documents/chaps-technical-design) -* [TPM Usage](/developers/design-documents/tpm-usage) -* [Verified Boot - Presentation](https://docs.google.com/a/chromium.org/presentation/d/1HHf_0nKrceQr_NQYGMpVlYTIYF8ky-eNxP7W5Lxw94Y/present#slide=id.g341ad2000_020) - (2014) -* [Verified Boot - Presentation](https://docs.google.com/presentation/d/14haBMrbpc2zlgdWmiaTlp_iDG_A8t5PTTXFMz5kqHSM/present?slide=id.g11a5e5b4cf_0_140) - (2016) -* [Keyboard-Controlled Reset - Circuit](/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit) - -### Firmware - -* [CBI: CrOS Board - Info](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/design_docs/cros_board_info.md) -* [Firmware Boot and - Recovery](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery) -* [Disk Format](/chromium-os/chromiumos-design-docs/disk-format) - (including boot process) -* [Developer Mode](/chromium-os/chromiumos-design-docs/developer-mode) -* [EC-3PO](/chromium-os/chromiumos-design-docs/ec-3po) (EC console - interpreter) -* [Recovery Mode](/chromium-os/chromiumos-design-docs/recovery-mode) -* [SAFT](/for-testers/saft) (semi-automated firmware test) -* [Verified Boot](/chromium-os/chromiumos-design-docs/verified-boot) -* [Verified Boot Crypto - Specification](/chromium-os/chromiumos-design-docs/verified-boot-crypto) -* [Verified Boot Data - Structures](/chromium-os/chromiumos-design-docs/verified-boot-data-structures) - -</div> -<div class="column"> - -### Platform - -* [Cros](/chromium-os/chromiumos-design-docs/chromium-os-libcros) (how - Chromium talks to Chromium OS) -* [File - System/Autoupdate](/chromium-os/chromiumos-design-docs/filesystem-autoupdate) -* [Kernel](/chromium-os/chromiumos-design-docs/chromium-os-kernel) -* [User-Land Boot - Design](/chromium-os/chromiumos-design-docs/boot-design) -* [Library - optimization](/chromium-os/chromiumos-design-docs/library-optimization) -* [Login](/chromium-os/chromiumos-design-docs/login) -* [Lucid Sleep](/chromium-os/chromiumos-design-docs/lucid-sleep) -* [Out of memory - handling](/chromium-os/chromiumos-design-docs/out-of-memory-handling) -* [Partition - Resizing](/chromium-os/chromiumos-design-docs/partition-resizing) -* [Power - Manager](https://chromium.googlesource.com/chromiumos/platform2/+/HEAD/power_manager/README.md) -* [Battery Life - Overview](/chromium-os/chromiumos-design-docs/chrome-os-battery-life-overview) -* [Performance - Overview](/chromium-os/chromiumos-design-docs/chrome-os-performance-overview) -* [Printing](http://www.chromium.org/chromium-os/chromiumos-design-docs/chromium-os-printing-design) -* [Text Input](/chromium-os/chromiumos-design-docs/text-input) -* [User Accounts and - Management](/chromium-os/chromiumos-design-docs/user-accounts-and-management) -* [CRAS: ChromeOS Audio - Server](/chromium-os/chromiumos-design-docs/cras-chromeos-audio-server) -* [Chromium OS - Cgroups](/chromium-os/chromiumos-design-docs/chromium-os-cgroups) -* [Unified - Builds](https://docs.google.com/document/d/1zonifFp8UpE6ISxsnzUrUb9ikx1pXeqs0UzW08ce6aY/edit#heading=h.guvbepjyp0oj) - (Chrome OS config) - -### Stateless - -* *Coming soon* - -**UI** - -* [Immersive - fullscreen](/developers/design-documents/immersive-fullscreen) -* [Volume - keys](/chromium-os/chromiumos-design-docs/chrome-os-volume-keys) -* [system - notifications](/chromium-os/chromiumos-design-docs/system-notifications) - -</div> -</div>
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit/KbdReset.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit/KbdReset.png.sha1 deleted file mode 100644 index a3ab8179503..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit/KbdReset.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -d0b392dbbd2fabd1edde756b91ae66c048043a86
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit/index.md deleted file mode 100644 index c1bac48c696..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit/index.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: keyboard-controlled-reset-circuit -title: Keyboard-Controlled Reset Circuit ---- - -To enable users to securely reset a Chrome device, or enter recovery and -developer modes, a hardware circuit is required. This function can be -implemented with a Silego chip, or an equivalent combination of parts and logic. - -### When a user presses a combination of the power key and certain other keys, the hardware circuit monitoring the keyboard generates a reset signal to the embedded controller. The embedded controller controls power to the rest of the system, which is reset as well. The circuit contains a flip-flop that is can only be cleared by the reset signal. This flip-flop is set by a GPIO signal from the embedded controller that is sent before the embedded controller transitions from its read-only firmware to its rewriteable firmware. The output of the flip-flop goes to a GPIO on the main processor, which can use this signal to verify that the EC has been reset and is still running read-only (and hence trusted) code. - -### Reset - -### To reset the system, the user presses Power+F3 (Power+VolumeUp on convertible devices without a secure keyboard connection). When the embedded controller resets, it progresses from read-only firmware to rewritable firmware for operation in Normal Mode. - -### Recovery mode - -To enter Recovery Mode, the user holds Power+F3+ESC (Power+VolumeUp+VolumeDown -on convertible devices without a secure keyboard connection). -When the embedded controller resets in Recovery Mode, it remains in its -read-only firmware and tells the main processor to boot in Recovery Mode. - -### Developer mode - -Users can enter Developer Mode from Recovery Mode, usually by pressing Ctrl+D at -the recovery screen. Developer Mode can be accessed if and only if the flip-flop -indicates that the embedded controller is still in read-only mode. This -requirement ensures that the Recovery Mode firmware on the main processor can -trust the keyboard messages from the embedded controller (and thus ensures that -the keyboard messages are in fact sent by a physical user who is present at the -keyboard—not, for example, from a keyboard that has been hijacked by a remote -process). - -### Optional battery cutoff - -Systems with non-removable batteries may implement a battery cut-off. The -battery can be cut off by removing the power adapter while holding Power+F3 -(Power+VolumeUp on convertible devices without a secure keyboard connection) for -10 seconds or more. The preferred implementation is with a control wire into the -controller/gas-gauge inside the battery pack, but an alternative is to force off -FETs in the charger circuit. This requirement allows users under direction from -support staff to completely cut off power to the device. It also enables users -to recover from situations where the power supply has locked up. For example, an -out of specification USB device causing the main power regulator to latch a -fault and shutdown until power is cycled. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit/KbdReset.png">](/chromium-os/chromiumos-design-docs/keyboard-controlled-reset-circuit/KbdReset.png) - -<img alt="KeyboardControlledReset.png.1353360421277.png" -src="https://lh5.googleusercontent.com/fmWa009HCo9PNdjbjeK5rpUVqax0sbm4RMc0bZpvQOM1NmhXZN867EuEZOCAWm8FMHGsIEm7ryT9kB1TITIfIvMyV32fxxw4nX9teXVWJUQ08EL5-vpybhQ9_ZMr74c55g" -height=437px; width=661px;> - -This hardware circuit does the following: - -* When the Power button is pressed, it checks the keyboard matrix to - determine whether F3 or VolumeUp (on convertible devices without a - secure keyboard connection) is also pressed. If so, it causes a - hard-reset of the embedded controller. This action has the side - effect of resetting the entire system. -* It provides an EC_IN_RW indicator for whether the embedded - controller code has transitioned from read-only code to rewritable - code as part of verified boot. This indicator is set by the embedded - controller via the EC_ENTERING_RW GPIO and is reset whenever a hard - reset is triggered. EC_IN_RW should go to a PCH/SOC GPIO line that - can be read by the main processor. EC_IN_RW is an open drain signal - and needs to connect to a pin with an internal pullup or have a - pullup on the board. -* EC_RST_L resets the embedded controller (and the rest of the - system). This is an open drain signal and should be pulled up to the - appropriate power rail. - -The embedded controller boots into read-only (RO) code. It then normally -verifies and jumps to rewritable (RW) code. Before jumping to RW code, the RO -code must assert the EC_ENTERING_RW GPIO. This will set the flip-flop, so that -it asserts the EC_IN_RW signal to the main processor. -Once the flip-flop has been set, it can only be cleared by pressing the Reset -key combination. No action by the embedded controller can clear the flip-flop.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/library-optimization/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/library-optimization/index.md deleted file mode 100644 index 7675ba95c5a..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/library-optimization/index.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: library-optimization -title: Library Optimization ---- - -## Problem - -One of the optimizations proposed for Chrome OS was the shrinking of C libraries -by removing the unused symbols from them. -This document describes the work done in this direction. - -## Existing solution - -Prior work has been done in this field by Montavista, which released an open -source set of scripts called the [Library Optimizer -Tool](http://libraryopt.sourceforge.net/) (libopt). According to their site: - -The Library Optimizer examines the complete target file system, resolves all -shared library symbol references, and rebuilds the shared libraries with only -the object files required to satisfy the symbol references - -Sadly the script is not very well documented (both in the way someone could use -it without modifying it and the way it works internally). After examining the -contents of the script and the example build script for libc it seems that the -script does indeed return a list of .o files which should be included in the -library after analyzing symbol dependencies. It can also receive a strip -parameter which uses the strip command to strip the .notes and the .comments -sections from the library. -I have modified the script to print out a lot of debug information and I have -created my own simple library to test the script on. Sadly the script did not -work as expected, as it didn’t output the expected .o file names. - -## New solution - -Some of the members of the compiler team came up with a new solution, which -would still involve passing a list of symbols to the linker, but which is less -error-prone as the linker can automatically detect the dependencies which these -symbols introduce. -The binaries can apparently be stripped of unused symbols using the garbage -collector, however this is not done by default yet on all Chromium OS packages. -Stripping the libraries is the interesting part, but this cannot be done -automatically without a view of the whole system. The solution presented by the -compiler team involves manipulating the visibility of functions and using -function sections and garbage collection to eliminate the unused symbols from -the library. -The current problems with this approach are: - -1. Using -ffunction-sections is supposed to increase the library size - but is needed for the shrink, must investigate if how much space is - saved -2. dlopen()/dlsym() breaks this method so they will have to be allowed - somehow - -## Current work - -A script has been created which currently: - -1. Builds a list of all the undefined symbols used by executables. -2. See what symbols are not used anywhere and marks them as unused - -The next steps are: - -1. Relink the libraries so that the unused symbols get removed (start - off with a small number of libraries) -2. Make it work with dlopen/dlsym.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/login/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/login/index.md deleted file mode 100644 index 00efd9cab11..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/login/index.md +++ /dev/null @@ -1,145 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: login -title: Login ---- - -[TOC] - -## Abstract - -* The Chromium OS-based device login mechanism will provide a single - sign on (SSO) capability that users can use to streamline access to - cloud-based services -* This mechanism will be designed for security, privacy and - ease-of-use -* We want to ensure that people can fully use Chromium OS without - needing a Google login -* While the initial work on the login mechanism has been focused on - providing instant access to Google services for users with Google - accounts, we are investigating support for OpenID to allow people to - fully use the system without needing a Google login. - -## Objective - -Chromium OS devices are cloud devices. They sync users' data and preferences -with cloud-based services. To do so, they need some way to bundle all the user's -data together and to know which machines to sync it down to. To enable this -functionality, users currently need to log in to Chromium OS devices with a -Google account. Other authentication service providers may also support this -functionality in the future. - -Chromium OS devices will be able to: - -1. Authenticate the user against Google if possible, so that it always - uses the user's latest password -2. Enable the user to log in when offline (assuming the user has logged - in online at least once) -3. Enable an SSO experience for Google properties -4. Allow the user to opt-in to auto-login that still does SSO, but does - not cache the user's password - -We also plan to support alternative authentication systems: - -1. Give users an SSO experience at OpenID relying parties -2. Give users an SSO experience at sites for which they've already - typed in credentials on a Chromium OS device - -We are also currently investigating the technical issues involved with allowing -users to log in to a Chromium OS device using a non-Google OpenID provider. We -are investigating how to enable 3rd parties to provide interoperable sync -services. - -## Design highlights - -### Phase 1 (complete) - -#### Current implementation - -The initial design achieves the minimum requirements by using Chromium's HTTPS -stack to talk to existing Google Accounts [HTTP(s) -APIs](http://code.google.com/apis/accounts/docs/AuthForInstalledApps.html) to -authenticate the user and get the appropriate cookies to log the user in to all -Google services the instant the browser UI shows up. Since the login screen is -actually presented by the browser process itself, it is a simple matter to -ensure that all cookies acquired as a result of authentication wind up in the -user's cookie jar. Once the user has successfully authenticated online once, we -use a hash of her password and the Trusted Platform Module (TPM) to wrap a magic -string. Offline login is implemented by asking for the user's password again and -attempting to successfully unwrap this magic value, an operation that cannot be -performed without access to both the specific TPM chip in the user's device and -the user's password. In the offline case, obviously, no cookies can be acquired. -Many users, though, will have unexpired cookies from a previous authentication -already cached in their Chromium browser session, and these can be used once -they get back online. The UI for logging in is provided by a the Chromium -browser itself in a special mode. We allow the user to select a keyboard layout -on this login screen, and locale will either be set by the device's owner, or be -selectable on the fly. - -For our needs, the normal Google Accounts ClientLogin API is not enough, as it -is designed to provide cookies that allow a client application to authenticate -to a single Google service. We want the cookies your browser gets when you run -through a normal web-based login, so that we can get them into Chromium after -the user's session has begun. Therefore, we currently go through a three-step -process: - -1. https://www.google.com/accounts/ClientLogin, to get a Google cookie -2. https://www.google.com/accounts/IssueAuthToken, to get a one-time - use token (good for a couple of minutes) that will authenticate the - user to any service -3. https://www.google.com/accounts/TokenAuth, to exchange the token for - the full set of browser cookies we need to do SSO - -In the future, we're looking to deploy another API that would minimize the -number of round trips. - -#### Making sure we're talking to Google - -In order to remain somewhat resistant to network-level attacks, the only root -certificate that the login process is willing to trust to identify web servers -is the one that issues Google's SSL certs. Thus, if an attacker is going to -trick a registrar into giving him an SSL cert for www.google.com, at least he -has to trick Google's registrar. - -#### Auto-login - -We don't want to implement auto-login by caching the user's password and running -it through the normal login process on his behalf; doing so would expose the -user's password to unnecessary risk. As auto-login is opt-in, the user action -that sets it up will take place after login, during an active user session -- -which means we have the Chromium browser up and running. Thus, turning on -auto-login will be a web-based flow that results in the user getting an OAuth -übertoken that we cache. These tokens are revokable, which allows the user to -limit his risk if one gets compromised. - -We would store this token in the same encrypted-while-shutdown storage as we're -using for system settings. The login process would check for it, and then, upon -successfully exchanging it for Google cookies, log the user in. If the login -would need to proceed offline, then the presence of the token is deemed to be -enough to allow login to succeed. The user opted in to auto-login; there is only -so much we can do to protect him and his data. - -### Phase 2 - -#### More efficient, more flexible - -Support for web-based authentication flows, like OpenID or those used internally -by many enterprises, is the next area we hope to explore. This will allow our -users to authenticate against providers other than Google, though there are -challenges around supporting arbitrary web-bases flows and still being able to -manage per-user data encryption without asking users to set up a separate, local -credential specifically for this purpose -- a user experience that we find less -compelling than what we provide today. - -#### SSO beyond Google - -Efforts in this space have not proceeded much at this time. It's probable that, -if users have chosen to "Log in via Google" at OpenID relying parties, the -presence of Google authentication cookies in their browser will Just Work. This -remains to be verified. As for logging in to third-party sites for which the -user has entered their password already, such behavior will likely be tied to a -password sync system.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/lucid-sleep/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/lucid-sleep/index.md deleted file mode 100644 index c47a3ec1826..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/lucid-sleep/index.md +++ /dev/null @@ -1,257 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: lucid-sleep -title: Lucid Sleep ---- - -[TOC] - -# Overview - -Some recent Chrome devices have the ability to wake up from a suspended state to -perform work. Potential tasks include: - -* Associating with a known WiFi access point that has just come into - range -* Updating user data in response to a received push message -* Synchronizing with app servers at the request of an app -* Performing system maintenance like trimming SSDs or checking the - battery level - -To minimize power consumption while performing tasks and external indications -the device is performing work, the device uses a hybrid power state called Lucid -Sleep. Effectively using this state requires coordination between all parts of -the system, from the kernel to the system daemons to the Chrome browser. - -This page describes the interactions between the system components to coordinate -Lucid Sleep. It is intended to help with firmware and kernel implementations and -troubleshooting of this feature on Chrome devices that support it. - -# Kernel - -When a Chrome device is suspending, the kernel performs the following steps: - -1. Userspace processes are frozen. Every userspace process is removed - from the scheduler’s run queue and no more CPU cycles are allocated. -2. Select kernel threads are frozen, depending on the device. Exactly - which threads are frozen depends on the hardware configuration; in - some cases, no kernel threads are frozen. -3. Hardware components are suspended, for example the display panel is - turned off, the hard drive is stopped, and USB ports are powered - down. -4. All non-boot CPUs are disabled. -5. The kernel programs the underlying hardware with a list of interrupt - sources that are allowed to wake up the system, for example the - keyboard or power button. -6. The kernel puts the boot CPU into a low-power state so that no code - is running on that CPU. - -Resuming a Chrome device is essentially the reverse of the suspend sequence: - -1. The hardware determines that a wakeup event has occurred and - delivers an interrupt to the boot CPU. -2. The boot CPU starts running the kernel code. -3. Non-boot CPUs are re-enabled. -4. Hardware components are resumed. -5. Any frozen kernel threads are thawed. -6. Userspace processes are thawed and the system resumes normal - activity. - -To support Lucid Sleep, the kernel allows userspace to expand the list of -interrupt sources that can trigger wakeup events. It is also possible to specify -sources that trigger Lucid Sleep versus a full system resume. Hardware -components that can trigger wakeup events must run independently in a low-power -state without kernel support. Conversely, kernel drivers for components that do -not trigger wakeup events must not re-initialize the hardware during Lucid -Sleep. - -For example, a device receives a push message over the network. The power -manager adds the Network Interface Card (NIC) to the list of hardware components -that can trigger a wakeup interrupt. It also adds the display panel and USB -ports to the list of components that must not re-initialize during Lucid Sleep. -At suspend time the NIC driver puts the NIC into a low-power state rather than -turning it off. Other suspend operations proceed as normal. - -After a wakeup event is triggered and before hardware components are resumed, -the kernel determines if the wakeup event should trigger Lucid Sleep or a full -resume and saves this value. When the hardware components are resumed, the -device drivers query the kernel to determine whether the wakeup is for Lucid -Sleep. If so, and userspace has requested that that hardware remain off, the -driver skips the remainder of re-initialization steps. After userspace processes -thaw, a process can also query the kernel via the sysfs interface to determine -if the wakeup event triggered Lucid Sleep. - -# Power Manager - -The power manager provides a mechanism for notifying the system daemons and -Chrome when the device is in Lucid Sleep. This mirrors the mechanism used for -notifying the system daemons when the Chrome device is about to suspend. A -process can register with the power manager on start up. When the system enters -Lucid Sleep, the power manager notifies registered processes by sending out a -DarkSuspendImminent D-Bus signal. This signal is the only external indication -that the system is in Lucid Sleep. - -When a device enters Lucid Sleep, the power manager also detects that the wakeup -event was not user-triggered and it should suspend the device again as soon as -possible. The DarkSuspendImminent signal both announces Lucid Sleep and signals -that suspension is imminent. - -When a suspend operation successfully completes (and userspace processes have -been thawed by the kernel), the power manager queries the kernel to see if the -current resume operation is from Lucid Sleep. If so, it sends out a -DarkSuspendImminent signal and waits for all applications that have registered -to receive the signal to report on readiness to suspend. After all registered -applications have reported readiness or the maximum delay has passed, the power -manager again instructs the kernel to suspend the system. - -## Handling user activity - -Occasionally, a user may start to interact with the Chrome device while it is in -the middle of Lucid Sleep. To ensure that all hardware components are properly -resumed, in this case the power manager instructs the kernel to perform a test -suspend. The normal suspend sequence is interrupted after suspending hardware -components, and the kernel immediately jumps to the step of resuming hardware -components in the resume sequence. This ensures that all hardware components are -ready before the user starts interacting with the system. - -# Shill - -Shill, the connection manager, decides what WiFi-related actions—such as -programming the wireless NIC and setting RTC timers—to take before the system -goes into suspend, when the system wakes up in Lucid Sleep, and when the system -fully resumes. The primary goals of these actions are to maintain WiFi -connectivity while the system is suspended or in Lucid Sleep and to allow the -system to respond to important events (e.g. GCM push notifications). - -Before the system suspends from a fully awake or Lucid Sleep state, shill -verifies whether the system’s WiFi interface is connected to a network. If the -system is connected, shill programs the wireless NIC to wake the system if the -NIC disconnects from that network, and starts an RTC timer to wake the system in -Lucid Sleep to renew its IPv4 DHCP lease (or obtain a new IPv6 configuration) -when it is about to expire. If the system is not connected, shill programs the -wireless NIC to independently perform periodic scans while the system is -suspended, and wake the system if any of its preferred networks are found. If -the number of preferred networks is greater than the NIC whitelist capacity, a -separate RTC wake timer is started to wake the system in Lucid Sleep to perform -shill-initiated scans. Shill also programs the wireless NIC to wake the system -upon receiving packets from whitelisted IP addresses registered by Chrome (more -details in next section). - -When the system wakes up in Lucid Sleep, shill receives a wakeup reason from the -kernel. If the reason is a disconnect or detection of a preferred network, shill -triggers a passive scan and attempts to establish a connection to any networks -found. If the reason is the receipt of a packet from a whitelisted IP address -(i.e. a GCM push notification), shill does nothing and allows Chrome to handle -the packet. For any other reason, including the aforementioned RTC timers, shill -either renews DHCP leases if the system is already connected, or performs a scan -and if the system is not connected, attempts to establish a connection. - -When the system fully resumes, shill deprograms all wake on WiFi triggers on the -wireless NIC, and stops any RTC timers that shill might have started before -suspend or during Lucid Sleep. Shill will then scan for networks if the system -wakes up not connected, and will resume its normal functionality thereafter. - -# Chrome - -The Chrome browser software handles push messages that arrive over the network. -It registers with the power manager so that the power manager will wait for it -to report readiness before suspending from Lucid Sleep. The progress of all push -messages is tracked from arrival to completion. If Chrome receives a -DarkSuspendImminent signal while one or more push messages are active, it waits -until no more active push messages remain before informing the power manager -that it is ready to suspend. Additionally, any network requests started by a -message consumer during processing can further delay Chrome before it reports -readiness to the power manager. - -Chrome also tells the NIC which connections to watch for incoming messages. -Currently, connections to Google Cloud Messaging (GCM) servers are supported. -After Chrome establishes a connection to the GCM server, it informs the NIC of -its IP address. While the system is suspended the NIC continues listening for -network activity and when a packet arrives from the IP address of the GCM -server, the NIC sends an interrupt to the boot CPU to wake the system into Lucid -Sleep. - -To maintain a connection with the GCM servers, Chrome sends a heartbeat message -to the server at regular intervals. A specialized timer (the -[AlarmTimer](https://code.google.com/p/chromium/codesearch#chromium/src/components/timers/alarm_timer_chromeos.h)) -is used to set a Real Time Clock (RTC) alarm. The RTC is a hardware timer that -runs even while the system is suspended and sends an interrupt to the boot CPU -to wake the system into Lucid Sleep. - -## Freezing Renderers - -If improperly handled, frequently waking up the system can consume a significant -amount of battery resources. In addition to conserving power by not resuming -power-hungry components like the display panel, while in Lucid Sleep Chrome -minimizes its usage of the CPU. - -Chrome has a multi-process architecture (for more details see -[here](http://www.chromium.org/developers/design-documents/multi-process-architecture).) -All apps, extensions, and web pages run inside renderer processes, which -represent the biggest consumers of the CPU in the system. To minimize the power -they consume (for example by running the CPU at a high load), Chrome freezes -renderer processes at the initial suspend time and thaws them only after a full -resume. - -The Linux kernel mechanism of control groups -([cgroups](https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt)) is -used. Freezing processes in a cgroup works the same way as freezing userspace -processes at suspend time: the kernel simply removes the processes from the -scheduler’s run queue. Since Chrome freezes its renderer processes before the -system suspends, those processes remain frozen even after the system resumes and -are thawed only when Chrome explicitly thaws them. Processes stay frozen for the -duration of Lucid Sleep and CPU load is minimized. - -### Properly handling the lock screen - -Either users or an enterprise policy can set a Chrome preference that locks the -screen when the system suspends. Like most parts of the system user interface, -the lock screen runs inside a renderer process. Common side effects of freezing -the lock screen operation include a noticeable lag when the system fully resumes -and dropped keystrokes. In some cases, users may need to re-type their -passwords. - -To address this, the lock screen renderer is given special treatment. If a user -requests locking the screen when the system suspends, Chrome puts the lock -screen renderer process in a separate cgroup from the other renderers. The lock -screen renderer remains unfrozen and responsive when the system fully resumes. - -### Dealing with renderers that may receive push messages - -Since all web applications as well as Chrome apps and extensions run inside a -renderer, a push message cannot be fully handled if the destination renderer has -no CPU cycles. So some renderers do need to remain active during Lucid Sleep. - -Chrome decides whether or not to freeze a renderer process when the renderer is -first spawned. For example, for Chrome apps and extensions, Chrome looks at the -permissions that the app/extension has requested in its manifest. If it has -requested the “gcm” permission, Chrome keeps the renderer out of the group of -renderers to freeze at suspend time. This ensures that when a push message -arrives it can be handled as quickly as possible. - -## Interacting with the GPU process - -Chrome uses a dedicated process to interact with the GPU on the system. This -process is not frozen during Lucid Sleep and can continue to respond to requests -to render objects to the screen. Though the renderers themselves may be frozen -and unable to make requests to the GPU process, the main browser process remains -active and can issue updates. - -This has implications for the display panel component, which is off during Lucid -Sleep. Any attempts by the GPU process to update the contents of the screen -result in errors, causing loss of context and forced reallocation of resources. -In addition to affecting GPU performance, repeatedly losing and allocating a -context (often multiple times per second) add a significant load to the CPU, -increasing the amount of power consumed during Lucid Sleep and reducing the -system’s overall battery life. - -To avoid consuming unnecessary resources, Chrome stops its compositor from -updating the screen and prevents rendering requests from reaching the GPU -process. If the user has configured the screen to lock when the system suspends, -Chrome waits until the contents of the lock screen are visible before directing -the compositor to stop rendering. The delay allows the contents of the user’s -screen to remain invisible and secure when the system resumes.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine (1).png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine (1).png.sha1 deleted file mode 100644 index a850f9c2d95..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine (1).png.sha1 +++ /dev/null @@ -1 +0,0 @@ -42261ac08d8c57b1157b0c218dd313948b1fe41a
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine.png.sha1 deleted file mode 100644 index 6a50ad344c0..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -e13400fec6a3186eb07eca09cf2b0377a673f097
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine.svg.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine.svg.sha1 deleted file mode 100644 index 1e1438468c7..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine.svg.sha1 +++ /dev/null @@ -1 +0,0 @@ -b6c323103897d21a3c135929e109c5dd0700dc50
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/index.md deleted file mode 100644 index 44fd7db72f5..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/network-portal-detection/index.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: network-portal-detection -title: Network Portal Detection ---- - -[TOC] - -Shill, the connection manager for Chromium OS, attempts to detect services that -are within a captive portal whenever a service transitions to the ready state. -This determination of being in a *captive portal* or being *online* is done by -attempting to retrieve the webpage http://clients3.google.com/generate_204. This -well known URL is known to return an empty page with an HTTP status 204. If for -any reason the web page is not returned, or an HTTP response other than 204 is -received, then shill marks the service as being in the *portal* state. - -Many, or perhaps most, captive portals found in Hotels, Coffee Shops, Airports, -etc, either run their own DNS server which returns IP address for all queries -which point to their webserver, or they intercept all HTTP web traffic and -return a 302 (redirect) response. The captive portal detection works very -reliably with these types of portal to indicate that the service is not fully -online. - -Other captive portals, sometimes run by cellular carriers, provide absolutely no -IP connectivity other than to their own servers, but they use a standard DNS -server and do not intercept HTTP requests. When a ChromeBook connects to this -type of network, the HTTP requests fail because the TCP connection to -clients3.google.com can never be established. The portal code tries multiple -times for up to 10 seconds to connect to clients3.google.com. If it cannot -connect it marks the service as being in a captive portal. This determination is -somewhat unreliable because very high latency connections, lossy connections and -other network issues can also result in failure to connect to -clients3.google.com. All of these are indicative of a network that is not fully -functional, but they do not necessarily indicate that the machine is stuck in a -*captive portal.* - -### Shill Service State Machine - -[<img alt="Flimflam service state machine" -src="/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine.png">](/chromium-os/chromiumos-design-docs/network-portal-detection/FlimflamServiceStateMachine.png) - -### Reverse Path Filtering - -By default ChromeOS enables [reverse path -filtering](http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.kernel.rpf.html) in the -kernel which drops packets received over a network interface inconsistent with -the outbound routing tables. This can present a problem when trying to determine -portal state when connected to multiple networks (WiFi, Wired Ethernet and/or a -Mobile Broadband connection). To avoid dropped packets, shill disables reverse -path filtering globally and on a per device basis while it is running the portal -detection code. - -### Shill Implementation - -shill attempts to determine the portal state whenever a service transitions to -the *ready* state. It does the determination using libcurl. Several options are -set: - -* libcurl is instructed not to cache DNS entries, and not to allow - re-use of HTTP connections -* libcurl is given a 10 second timeout for connection and for the - entire transaction. -* libcurl is told to bind to a specific interface so all traffic is - done over that interface -* libcurl is explicitly told which name servers to use for name - resolution -* reverse path filtering is disabled (see above). - -Portal detection can be enabled or disabled on a per service basis, or if that -is not set (set to AUTO), the determination is done based on the default for the -technology. - -### Interpretation of Portal State - -Chrome and shill use portal state for a variety of reasons. - -For prioritizing connections shill always prioritizes an *online* connection -over one in a *portal* state. To the extent that connections using a web proxy -are never marked as *portal* but instead immediately marked as *online* this -inhibits shill's ability to choose the best connection/service. - -For cellular connections, and especially for Verizon, Chrome uses portal state -to determine when to put up *toast* which will prompt the user to buy more data. -It also controls the presence of a 'buy data' button on the network -settings/information tabs for the Verizon Wireless service. - -When at the OOBE pages and the login screen, Chrome uses the portal state to -prompt the user to check the network connection (more details needed from the -Chrome team). - -### Web Proxies - -Web proxies present multiple problems for portal state determination. libcurl -supports forwarding requests via a proxy, but this can only be done if the IP -address of the proxy is known. Shill does not currently know the IP address of -the web proxy, nor does it have the ability to evaluate the URL against the -JavaScript code that determines if the proxy should be used. - -Several services (e.g. auto update, etc) use Chrome as the engine to fetch URLs, -or use Chrome to evaluate the URL and find out which, if any, proxy to use. This -seems like an appealing solution, but, knowing the proxy is insufficient, -because one might also need to authenticate to the proxy. It is unclear if -Chrome exposes the proxy authentication information. I also do not believe that -Chrome can proper evaluate a URL relative to the proxy settings on a per -interface basis. Chrome almost certainly uses the default connection and the -proxy settings for the default connection when doing HTTP requests and when -evaluating a URL against the JavaScript configuration. - -### Future Ideas - -* Improve Chrome to be able to do what libcurl does --- route http - requests via a specific interface and use the DNS servers associated - with that interface. -* Improve Chrome so that it can authenticate to two different proxies - on two different interfaces simultaneously. -* Improve Chrome so that it can figure out automatic proxy - configuration based on DNS lookup of wpad over different names on - different interfaces. (etc....) -* Have the ChromeOS connection manager ask Chrome to fetch the URL - instead of libcurl. - -Or - -* Do much of the above, but have Chrome return proxy authentication as - well as IP address for a given URL, so that shill can pass the proxy - information to libcurl (or implement it all ourselves in shill).
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/onc/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/onc/index.md deleted file mode 100644 index 181bc5eafd4..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/onc/index.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: onc -title: onc ---- - -See [Open Network -Configuration](/chromium-os/chromiumos-design-docs/open-network-configuration)
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/open-network-configuration/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/open-network-configuration/index.md deleted file mode 100644 index 4bdf8ea61d4..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/open-network-configuration/index.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: open-network-configuration -title: Open Network Configuration ---- - -The Open Network Configuration specification describes a network and certificate -configuration format that could be used across operating systems. Chromium OS -will support it natively. - -The specification can be found here: - -<https://chromium.googlesource.com/chromium/src/+/HEAD/components/onc/docs/onc_spec.md>
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/index.md deleted file mode 100644 index e44bfcb9ca0..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/index.md +++ /dev/null @@ -1,339 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: out-of-memory-handling -title: Chrome OS Out of Memory Design ---- - -August 2011 \ -jamescook@, gspencer@, ups@ - -Chromebooks have a limited amount of physical RAM (1 - 2 GB) and do not use -swap. Currently, when the machine runs out of memory the kernel’s OOM killer -runs and kills a process. Usually this is a renderer process, which results in -one or more tabs being killed. If that renderer is running the frontmost tab, -the user sees a “He’s dead, Jim” page informing the user that the tab was -killed. If it is running background tabs, those tabs are reloaded when the user -switches to them. - -## Background - -See the “Where does our memory go?” appendix below for some discussion of our -current memory usage. - -## Proposal - -Ideally, the user should never have to worry about running out of memory. We -should be able to determine when we’re getting low on RAM and take actions in -userspace to reduce our footprint. We propose to continue closing tabs to free -up RAM, but to start with closing individual tabs rather than the whole set -owned by one renderer. We like this direction because: - -* The user experience is better than killing a process - one thing dies at a - time, rather than a seemingly random group. -* This strategy is commonly used on phones and tablets. -* We can close tabs quickly, so the user experience should be Chrome running - fast, dropping data, and continuing to run fast, rather than a gradual - slowdown we would anticipate from trying to compress/hibernate the state of - a tab. -* We can try to be intelligent about which tabs we close, preferring those - which are rarely or never used, or which are unlikely to have user-created - state. -* We know that closing a tab will rapidly return a large block of memory to - the OS, as opposed to flushing caches, running garbage collection, etc. - which return variable amount of RAM and may quickly result in the tab - reclaiming it. -* The strategy is relatively simple - we already know how to close tabs and - automatically reload them. - -We should kill renderer processes only as a last resort, and even then avoid -killing the process associated with the page the user is viewing. The user -should never see “He’s dead Jim” unless the page he is looking at is consuming -all available memory on the device. - -We hypothesize that if we can avoid showing “He’s dead Jim” the majority of -users will not notice the page kills. If users find this objectionable, we can -try other strategies, like swap-to-memory (zram) or blocking the creation of new -tabs when we’re out of RAM. - -## Technical Design - -We’re going to build a “double-wall” for low-memory states: A “soft wall” where -the machine is very low on memory, which triggers a user space attempt to free -memory, and a “hard wall” where the machine is out of memory, which triggers the -normal kernel OOM killer. In both cases we’ll attempt to minimize impact on the -user. - -### For the “soft wall” - -It takes memory to free memory. In particular, while chrome is trying to free up -memory processes will continue to consume it. So we need to trigger the “soft -wall” while there is still some free memory available. - -The kernel will reserve 10 MB of memory for the wall. It will provide a special -file descriptor to allow user space processes to monitor for a low-memory event. -The chrome browser process will monitor that file descriptor for the event. -(We’ll keep the size in a constant, and maintain UMA statistics on whether it -was a large enough buffer space.) - -When an allocation occurs that eats into this reserve, the kernel will -temporarily “move the wall back”, allowing a memory page to be used and the -allocation to succeed. It will send an event on the file descriptor and start a -timer for 200 ms. It will listen on the file descriptor for a “done” event from -chrome. - -When the soft wall is reached, Chrome will evaluate tabs to decide which one(s) -to close, and in what order, in order to free up memory. Note that closing a tab -in this mode is different from the user closing a tab because we can’t run the -“onclose” javascript handlers that we would normally run (they might do -anything: open a modal dialog, for instance). We are investigating how to -implement this - we may choose to navigate forward to about:blank, which frees -most of the memory used by the page and allows us to navigate back when the user -selects the tab. This saves their scroll position, form data, etc. - -The chrome browser process will iterate through its list of tabs to choose one -to close or blank out. The browser prefers closing tabs in this order: - -* A background tab which the user has opened, but never seen (e.g., a - control-click on a link) -* A background tab into which the user has never typed/clicked/scrolled -* Any background tab (ordered by most recently clicked in buckets of 10 min, - then by memory consumption) -* The foreground tab - -Within each group, tabs are ordered by the last time that the user clicked on -them, preferring to close tabs that haven't been clicked on in the longest -amount of time. If two tabs were clicked on within 10 minutes of each other, -then the one with the largest memory use is more likely to be closed. -After closing the tab, if the renderer is still running (it has other tabs open) -it tells tcmalloc to release freed memory back to the OS immediately. The -browser process then tells the kernel it is done. The kernel then resets its -memory reserve back to 10 MB. - -### The “hard wall” - -Our hard wall is the OOM process killer. We hit the hard wall in two situations: - -1. During chrome’s memory cleanup, we still run out of physical memory - (a fast leak) -2. Chrome doesn’t send the done event promptly - -If Chrome sends the done event, but didn’t free enough memory, we send the soft -notification again, and rely on the timeout to do the hard kill. Per ups@ this -is easier to implement in the kernel and more robust. - -We cannot do any userspace work when we hit the hard wall. However, we can use -the kernel’s oom_score_adj property on processes to give it guidance as to which -process to kill. We set this score at startup for OS-related processes to -1000. -For Chrome related processes, we set an initial score as outlined below. For -renderers, we start them with an initial score of 300 and adjust the score once -every 10 seconds (from within the browser process). - -A higher oom_score_adj makes a process more likely to be killed in OOM -situations: you can think of it as a percentage of memory. Setting a -oom_score_adj value of 500, for example, is roughly equivalent to allowing the -remainder of tasks sharing the same system, cpuset, mempolicy, or memory -controller resources to use at least 50% more memory. A value of -500, on the -other hand, would be roughly equivalent to discounting 50% of the task's allowed -memory from being considered as scoring against the task. A score of -1000 makes -a process unkillable by the OOM killer. - -The current Linux OOM killer algorithm only considers allowed memory usage, and -not run time or process start time in its “badness” score. Note that this -algorithm does not guarantee that the processes will be killed in the order -specified, it is only a hint. Please check out [this -description](https://www.kernel.org/doc/html/latest/filesystems/proc.html#chapter-3-per-process-parameters) -for more detailed information. - -By default all processes started by init on ChromeOS are set to an initial value -of -1000, which is inherited by sub-processes, so that only processes which we -explicitly set to higher values will be killed by the OOM killer. We score -processes as follows: - -| Score | Description | -| ----- | ----------- | -| -1000 | Processes are never killed (Upstart uses `oom score never`) | -| -1000 | Linux daemons and other processes | -| -1000 | CrOS daemons that are critical to the system (dbus) | -| -900 | CrOS daemons that are needed to auto-update, but can restart | -| -100 | CrOS daemons that can recover (shill, system metrics) | -| -100 | Android system processes | -| 0 | Chrome browser and zygote | -| 100 | Plugins, NaCl loader | -| 200 | Chrome GPU process, workers, plugin broker process | -| 300 | Chrome extensions | -| 300-1000 | Chrome renderers | -| 1000 | Processes that are killed first | - -The renderer scoring will follow the algorithm for tab closing above. However, -since each renderer process might be servicing multiple tabs, we select the -lowest score (least likely to be killed) of all the tabs for each renderer. This -ensures that the renderer showing the tab the user is viewing has a low score, -even if other tabs in the same renderer have a higher score. - -### “Why not just garbage collect?” - -Well, we tried that. In purging memory, we used the MemoryPurger, and even added -Webkit cache and font cache, and notification of V8 that we’re low on memory to -the list of things that the memory purger pushes out, and found that for a -browser with 5-10 “real world” tabs open, we save single-digit megabytes, and -they rapidly fill up again (for instance, after 10s, 30% of the memory recovered -appears to be used again). With the exception of V8, it takes only 20ms or so to -do the purge, but it’s not very effective. V8 adds another 700ms or so to the -time it takes to purge. - -## Appendix: “Where does our memory go?” - -Memory consumption is notoriously tricky to measure and reason about, as Evan -recently noted in his blog post “[Some things I’ve learned about -memory](http://neugierig.org/software/blog/2011/05/memory.html)”. For a release -image of Chrome OS with the stock extensions installed, after logging in and -loading `about:blank`, `/proc/meminfo` looks like: - -``` -MemTotal: 1940064 kB <---- ~2 GB minus kernel -MemFree: 1586544 kB -Buffers: 3640 kB -Cached: 248344 kB -Active: 110848 kB -Inactive: 210648 kB -Active(anon): 77216 kB \\____ ~120 MB dynamically allocated “anonymous memory” -Inactive(anon): 48744 kB / -Active(file): 33632 kB \\____ ~200 MB file-backed executables, mmap’d files, -etc. -Inactive(file): 161904 kB / some of which can be released -... -AnonPages: 69512 kB -Mapped: 85740 kB -Shmem: 56448 kB <---- ~50 MB shared memory, mostly from video driver -Slab: 15096 kB -SReclaimable: 6476 kB -SUnreclaim: 8620 kB -KernelStack: 1288 kB -PageTables: 3380 kB -After opening gmail.com, calendar.google.com, and -[espn.go.com](http://espn.go.com/) (image and Flash heavy), /proc/meminfo -changes as follows: -Active(anon): 277856 kB \\___ ~390 MB dynamically allocated, up from ~120 -Inactive(anon): 111440 kB / -Active(file): 50912 kB \\___ ~220 MB file-backed, up from ~200 -Inactive(file): 174260 kB / -... -Shmem: 140012 kB <--- ~140 MB, up from ~50 -``` - -File-backed memory doesn’t change much, because most of it is executables and -mmap’d resource files that don’t change. Renderers account for most of the -change in dynamically allocated memory, though the browser process grows a -little for each new renderer. Both Flash and WebGL content will cause the video -driver to consume additional memory. (Of note, R12 had a bug where video streams -would cause the video driver’s memory consumption to grow rapidly, which was the -primary cause of “He’s dead Jim” in the cases I investigated.) - -From a userspace perspective, most of the renderer’s memory consumption is -images and strings, depending on the source site. For example, tcmalloc heap -profiling of a renderer on gmail.com shows the top 5 consumers: - -``` -Total: 27.5 MB -15.7 57.0% 57.0% 15.7 57.0% WTF::fastMalloc -4.3 15.5% 72.5% 4.3 15.5% pixman_image_create_bits (inline) -4.1 15.0% 87.5% 4.1 15.0% std::string::_Rep::_S_create (inline) -0.8 2.9% 90.4% 0.8 2.9% v8::internal::Malloced::New -0.6 2.2% 92.6% 0.6 2.2% sk_malloc_flags -``` - -The graphical profile (see attachment "pprof.renderer.gmail.svg") shows -`WTF::fastMalloc` is allocating strings, mostly for `HTMLTreeBuilder`. -For `espn.go.com`, an image heavy site, the heap profile looks like: - -``` -Total: 26.8 MB -11.4 42.6% 42.6% 11.4 42.6% WTF::fastMalloc -10.7 40.0% 82.5% 10.7 40.0% sk_malloc_flags -2.8 10.3% 92.8% 2.8 10.3% v8::internal::Malloced::New -0.2 0.7% 93.5% 0.2 0.7% WebCore::Text::create -0.2 0.6% 94.1% 0.2 0.8% SkGlyphCache::VisitCache -``` - -The graphical profile (see attachment "pprof.renderer.espn.svg") shows -`WTF::fastMalloc` is still strings, but now `sk_malloc_flags` is allocating large -image buffers, from WebCore::PNGImageDecoder and JPEGImageDecoder. -Work is underway to reduce WebKit memory consumption for 8-bit strings, see -<https://bugs.webkit.org/show_bug.cgi?id=66161> - -Some WebKit fixes recently landed to make decoded image pruning more efficient: -<https://bugs.webkit.org/show_bug.cgi?id=65859> - -All that said, users will always be able to fill available memory by opening a -large number of tabs, or using web sites that bloat with DOM nodes or JS -objects. We will always need some mechanism to handle the out-of-memory case. - -## Items for engineering review (added September 6): - -A. We can take three general approaches in OOM situations: - -1. Stop the user from consuming more RAM. (Warn them with a butter-bar, - then eventually block them from opening more tabs, etc.) -2. Spend CPU to simulate having more RAM (zram memory compression, - process hibernation?) -3. Throw away data we can quickly/safely regenerate - -We’re afraid 1 is annoying and 2 may be slow. We’re proposing trying 3, and we -can easily fall back to 1 if it doesn’t work out. - -B. On more reflection, the soft wall should probably be more than “one tab’s -worth” of memory away, perhaps 25 or 50 MB. That will give us much more time to -do work. How do we figure out the right value? - -C. This design intentionally does not address Chrome’s baseline memory usage or -cases where Chrome might leak or bloat. We’ve got extra instrumentation in R14 -to help find end-user cases where Chrome on Chrome OS uses a lot of RAM. We’d -like to do something simple with the OOM case, and spend more time -finding/fixing specific cases of bloat. - -D. We could add some sort of subtle notification when Chrome hits the soft wall. -Glen proposed badging the wrench icon and providing a menu item like “Chrome -needs attention” that would take you to the task manager and show you processes -that are eating a lot of RAM or CPU. This might help power users find and report -cases of bloat. Don’t - log it and learn. - -E. We can probably do testing of the signals sent from the kernel with autotest. -The correctness of the kill-priority algorithm can be tested with unit and -browser tests. The discard/kill process itself can be tested with browser tests. - -## Notes from design review (added September 6): - -Overall, we want to build something simple, get it in the field, collect -log/histogram data and iterate to ensure we’re killing the right tabs. We don’t -want to add any UI - managing memory is our problem, not the user’s problem. - -Action: Implement an experiment for Chrome users where we test our killing -algorithm by testing which tab the user is most likely to close or never visit -again. - -Action: Log how much memory you get back from each close/kill so that you can -start to develop an algorithm for which tab might get you the most memory back. -GPU process can run out of aperture space and the GL calls can fail. Must -reclaim memory by closing GPU intensive tabs. Probably needs a GPU-specific -implementation, not part of this feature. - -Action: Add a global notification that we are low on memory, and different -components can respond appropriately. - -Perhaps it would be good to implement logging per user to determine what their -patterns are to feed into the algorithm. - -What to do about parent/child tabs (gmail tearoffs, etc.) Can we restore this -connection? - -Decision: delay implementing zram until we know more about whether we want it -for extending memory. - -Action: Implement an about: page that describes tab close/kill order at the -moment. Could also show historical kill/close information. - -Action: Design an experiment for the size of the “soft wall” memory padding. diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/pprof.renderer.espn.svg.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/pprof.renderer.espn.svg.sha1 deleted file mode 100644 index 09b4f70e7b7..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/pprof.renderer.espn.svg.sha1 +++ /dev/null @@ -1 +0,0 @@ -30e9b835e0fc99f2ffef6a995e419167b757244f
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/pprof.renderer.gmail.svg.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/pprof.renderer.gmail.svg.sha1 deleted file mode 100644 index 84eb422c9f8..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/out-of-memory-handling/pprof.renderer.gmail.svg.sha1 +++ /dev/null @@ -1 +0,0 @@ -eb214e2c72edda852f03ce6299f50e9c26acb418
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/partition-resizing/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/partition-resizing/index.md deleted file mode 100644 index 3a394795570..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/partition-resizing/index.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: partition-resizing -title: Partition Resizing ---- - -[TOC] - -**Note: This is a proposal that has never shipped.** - -## Abstract - -Chromium OS will be able to resize partitions without erasing them, easing -delivery of system updates. - -## Use case - -A big new operating system release is on the way, and it needs more system space -than the current system partitions have. We can push an update that makes the -system partitions bigger (by shrinking the stateful partition); then when the -big new release arrives, enough space is available on the disk. - -## Partition layout - -A drive currently contains up to four partitions, in the following order: - -1. One partition for state resident on the drive, called the "stateful - partition." Any space required in the stateful partition is - pre-allocated by the system. For example, a large file will be - allocated for the autoupdater to use to store downloaded updates. -2. An optional swap partition. -3. Two partitions for the root file system. For more information, see - the [File - System/Autoupdate](/chromium-os/chromiumos-design-docs/filesystem-autoupdate) - document. - -In the future, drives may be able to have more partitions, as needed. - -## Procedure for resizing partitions - -The following diagram shows all the steps that must occur to resize partitions -and leave them in the desired state. -Keep in mind that at any point the system may be unexpectedly rebooted, and the -OS must recover gracefully (in particular, it must not brick). Also, keep in -mind that the OS doesn't allow changing the contents or location of the -partition we booted into. -The following procedure assumes that at the start of the process, the system has -booted into the last partition on the disk. If, instead, the system is currently -booted into the second-to-last partition, then the partition-resizing process -usually waits to begin until the next update or reboot occurs naturally. -However, if the resizing is urgent, the system can copy the A partition into B -and then reboot before starting the illustrated procedure. -If there's a swap partition, it isn't resized. -Note that, at the end of the following process, the A and B partitions are -always the same size as each other. - - - -## When does resizing occur? - -Partition resizing is initiated by the system, not by the user. -Some notes: - -* The approach described in this document depends on ext4 being able - to make the stateful partition smaller. -* If the stateful partition contains a lot of data, it may not always - be possible to resize it down to the desired size; in that case the - system will have to delete state information before resizing. -* The resizing process currently assumes that the machine can't be - used while partitions are being resized. Therefore, during the time - while the system is shrinking the stateful partition, the user will - have to leave the machine turned on but won't be able to use it. The - system will display a "working...please wait" message on the screen. -* Open issue: Will ext4 support resizing partitions while the file - system is mounted? If it does, then we may be able to allow the user - to use the machine while resizing partitions. - -Note that, to support encrypted home directories, we will already have to have a -"working...please wait" message on the screen at logout when we re-sparsify the -home directory. This would be an appropriate time to resize the stateful -partition file system if needed. diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/partition-resizing/resize_partitionpng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/partition-resizing/resize_partitionpng.sha1 deleted file mode 100644 index 0f17f6bb37b..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/partition-resizing/resize_partitionpng.sha1 +++ /dev/null @@ -1 +0,0 @@ -22041343faca02165ec17b4b8189da114b28d66e
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/powerwash/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/powerwash/index.md deleted file mode 100644 index b580874edcb..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/powerwash/index.md +++ /dev/null @@ -1,135 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: powerwash -title: Powerwash ---- - -**Status**: v1 implemented in M23 - -**Goal** - -Provide users with a means to clear mutable system data as part of the base OS. - -**Background** - -There are a number of cases where it can be useful for users to have a safety -hatch that doesn't require knowledge of the developer switch or have a valid -recovery device (USB Stick, or SD card) handy: - -* Disk corruption: in the wrong place, it could impair the ability to - sign in to any account -* Bad update interaction: may impair sign-in or other behavior -* Too many accounts to delete one at a time: happens in lab - environments or other high traffic locations -* ... - -**Requirements** - -> v1: - - * Must be accessible from the normal UI - * Must be accessible before sign-in - * Must not be usable by a remote attacker with a single account - compromise - * Must not destroy unrecoverable data: Limited TPM state on disk, - Install-time Attributes file - -> v2: - - * Must clear the TPM, or - * Must preserve enterprise enrollment status - * May preserve enterprise enrollment token - -**Overview** - -Powerwash will provide a key combination that is accessible at the sign-in -screen (Ctrl+Alt+Shift+R) and a menu item in Settings, after sign-in, to trigger -a system wipe. The wipe itself will using an existing implementation and will be -performed at boot-time. - -**Detailed Design** - -Powerwash is simple in practice. It performs the following steps at boot-time: - -* Preserve any files desired by moving them to /tmp (a ramdisk) -* Perform a fast wipe of the stateful partition (using the same script - as the developer mode transition back to verified) -* Restore the files preserved in /tmp - -In order to trigger those steps, a request from Chrome must be made to the -session manager requesting a wipe. If no user has signed in to the system yet, -then the request will be granted, and a wipe will occur on the next boot. If a -user has signed in, then a request to wipe the system will result in a reboot -followed by a confirmation request at the sign-in screen. This is meant to limit -the risk of a single compromised account from being used to wipe a machine. - -**Interface to Chrome** - -Chrome will communicate with session_manager over DBus. A call to -StartDeviceWipe() will trigger the request. If no user has signed in, then -session_manager writes "fast safe" to a reset file: -/mnt/stateful_partition/factory_install_reset. This is already used for factory -wipes, so the changes are minimal. - -**Wipe mechanism** - -During boot, chromeos_startup checks the ownership (root) and existence of the -reset file. This will follow the normal factory reset wipe path except that it -will not use the factory wipe screen. clobber-state is then called with the -arguments in the reset file, as mentioned above. - -**Preserved Files** - -In order to preserve files, the "safe" argument is added to clobber-state. A -"safe" wipe will use the tar utility to preserve the path, ownership, and -permissions of any files that should survive the wipe. At present, the install -attributes file (lockbox.pb) and the TPM key blobs are preserved. The TPM is not -cleared by a power wash which means that the lockbox cannot be reinitialized nor -can some pre-TPM-ownership actions be taken. - -**Internationalization** - -The UI in Chrome will use the normal localized asset infrastructure. - -The wipe message will use chromeos-boot-alert's localized asset implementation. - -**Security considerations** - -The primary security considerations here are around whether a remote attacker -can wipe a system. While this is not necessarily the most malicious of attacks, -it would not be a pleasant user experience in the least. To limit this attack, -we've required that powerwash be triggered or confirmed prior to any active -accounts signing in. This means that an unprivileged attacker could request a -powerwash, but not persist across the reboot to confirm it (minus a small -chronos-owner attack surface at sign-in time). If an attacker had gained higher -privileges, then they could wipe the device without help. - -**Privacy considerations** - -This feature allows users to wipe all user data and no user data is generated in -the process. - -**Testing Plans** - -(detailed elsewhere) - -**Caveats** - -**Differences from a developer mode wipe:** - -* The TPM is not cleared which means: - * enterprise enrollment after a powerwash will fail - * shared-data disk encryption key will not have changed - * lockbox & tpm data must be manually preserved using "safe" - -**Constraints on v1/Additional work needed:** - -* Enterprise enrolled devices cannot be reset using this mechanism. - * The UI does not support it. - * The implementation may not allow re-enrollment after a wipe (if - the UI supported it). -* crossystem clear_tpm_owner_request is not in use.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/ChromeOSCryptohomeTPMusage.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/ChromeOSCryptohomeTPMusage.png.sha1 deleted file mode 100644 index df11a5652f3..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/ChromeOSCryptohomeTPMusage.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -15e4075066cebced07ca3d28e315a51757e473b2
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/ChromiumOS_TPM_Cryptohome.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/ChromiumOS_TPM_Cryptohome.png.sha1 deleted file mode 100644 index 989f6f53226..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/ChromiumOS_TPM_Cryptohome.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -7e33a2996c2cd9a470bef383ac9a2d76093ef03b
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-benchmarks--ecryptfs2bext3-1008hapdf.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-benchmarks--ecryptfs2bext3-1008hapdf.sha1 deleted file mode 100644 index 280c723de0a..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-benchmarks--ecryptfs2bext3-1008hapdf.sha1 +++ /dev/null @@ -1 +0,0 @@ -5b9b983601a06533c25c5fa694c232d92f44f34d
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-benchmarks--no-encryptionpdf.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-benchmarks--no-encryptionpdf.sha1 deleted file mode 100644 index 84dd0287602..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-benchmarks--no-encryptionpdf.sha1 +++ /dev/null @@ -1 +0,0 @@ -731233dadd3ea78e0a2eae90414ec66a40e5e2ac
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-on-chrome-os--dmcrypt2bext42b1008hapdf.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-on-chrome-os--dmcrypt2bext42b1008hapdf.sha1 deleted file mode 100644 index 15daf3b0eb3..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/disk-encryption-on-chrome-os--dmcrypt2bext42b1008hapdf.sha1 +++ /dev/null @@ -1 +0,0 @@ -79cc4244358bb340d31cd748d4da1b280d104cd2
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/index.md deleted file mode 100644 index 10ea553f3e9..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/protecting-cached-user-data/index.md +++ /dev/null @@ -1,496 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: protecting-cached-user-data -title: Protecting Cached User Data ---- - -[TOC] - -## Abstract - -* Chromium OS devices are intended to be both portable and safely - shared. As a result, privacy protection for user data stored on the - local disk is a requirement for a Chromium-based OS. -* Privacy protection for user data stored on a local disk is - accomplished via system-level encryption of users' home directories. -* Chromium OS uses the [eCryptfs](https://launchpad.net/ecryptfs) - stacked filesystem with per-user vault directories and keysets to - separate and protect each user’s cached data. -* Chromium OS devices may use the Trusted Platform Module (TPM) to - protect against brute-force attempts to recover a user’s keyset (and - therefore the data it protects). - -## Problem: Multiple users, portable devices, locally stored data - -Chromium OS should provide privacy protection for user data stored on the local -disk. In particular, some subset of a user's email, pictures, and even HTTP -cookies will be stored on the local drive to enhance the online experience and -ensure Chrome notebooks are useful even when an Internet connection isn't -available. Without this protection, anyone who has physical access to the -Chromium OS device would be able to access any data cached on it. - -Normally, data stored in the cloud is protected by privacy-guarding mechanisms -provided by the servers that host it. When data is pulled out of the cloud, it -moves outside of the reach of those server-provided protections. Even though -this is the case today with most computers, there are two areas that make this -even more important to Chromium OS: - -* With a Chromium OS device, users shouldn't have to worry about - whether other users of the same device can see their cached data. - For example, if my friend wants to log in to my device to check her - mail, it's not a big deal. She won't be able to access my data, and - even though she used my device, I won't be able to access her data. -* Chromium OS devices are portable. The risk with very portable - devices is that they are easy to misplace or leave behind. We don't - want the person who finds a device to instantly have access to every - users’ websites, emails, pictures, and so on. - -## Solution: Encryption - -Since Chromium OS is built on Linux, we can leverage existing solutions for -[encrypting](http://en.wikipedia.org/wiki/Disk_encryption_theory) the user's -data at the underlying operating system level and make sure it is automatic and -mandatory. Given the available implementation choices, many possible approaches -were considered (see [Appendix -B](/chromium-os/chromiumos-design-docs/protecting-cached-user-data#Appendix_D_Designs_considered) -for details). Of those approaches, we chose a solution that provides file -system-level encryption of [home -directories](http://en.wikipedia.org/wiki/Home_directory) for each user on a -device. - -Encrypting each user's home directory means that all of the data stored and -cached while browsing will be encrypted by default. In addition, data created -and maintained by plugins, like [Adobe -Flash](http://www.adobe.com/products/flash/), will be transparently encrypted -without developers or users needing to take any special action. The same goes -for any data stored by HTML5-enabled web applications. - -Existing Operating Systems rely on file ownership and access permissions to -prevent two users of a system from accessing each other’s files. However, the -root or administrative user typically can still access any of these files, and -the files are really only protected as long as that Operating System instance is -booted. Mounting the drive as a secondary disk would allow full access. -Encryption is a better solution to this problem--even when the device is in the -hands of a malicious individual, that person would still have to recover the -encryption keys to be able to access the cached data. - -## How encryption works - -In a nutshell, each user gets a unique “vault” directory and keyset that is -created at her first login. The vault is used as the underlying encrypted -storage for her data. The keyset is tied to her login credentials and is -required to allow the system to both retrieve and store information in the -vault. The vault is opened transparently to the user at login. On logout or -reboot, the user's data is locked away again. - -### Details - -Each user is allocated a vault directory that contains the encrypted backing -storage for their home directory. The vault is located in a directory specific -to the user using a hash of the user name with a salt value, and is mounted -using eCryptfs at login to the user directory. A user-specific file encryption -key (FEK) and file name encryption key (FNEK) allow eCryptfs to transparently -encrypt and decrypt while the vault is mounted. A successful mount of the vault -is a required step in the login process of Chromium OS. On first login, -/etc/skel/ is copied to the new home directory, and then it's ready for use. - -The FEK and FNEK (together, with some additional data, called the keyset) for a -user are created using the randomness generator provided by the kernel, which, -on Chromium OS devices, is given additional entropy from the TPM at boot. These -128-bit AES keys are used to encrypt the file contents and file names, -respectively. As a stacked filesystem, eCryptfs has a lower file (the encrypted, -persistent file located in the vault and stored on disk) for each upper file -(the plaintext file presented at the mount point, which is never stored directly -to disk). The keys exist in kernel memory for the lifetime of the mount. - -A user’s keyset never changes unless their account is removed from the Chromium -OS device, or if the system is erased, such as when a recovery device is used. -It must be protected from disclosure and only available with valid login -credentials. To achieve this, Chromium OS uses one of two methods to store the -keyset, depending on whether a TPM is available and initialized. The preferred -method involves using the TPM, with the fallback method using the -[Scrypt](http://www.tarsnap.com/scrypt.html) encryption library. Both methods -tie the user’s vault keyset to their Google Accounts password, and are described -further below. - -All of this is managed by one daemon, cryptohomed, which listens for incoming -requests from the login manager. If the request is to mount an encrypted vault, -the user name and weak password hash must be included. If the request is to -unmount the image, cryptohomed clears the keys stored by the kernel and unmounts -the vault. - -### Performance concerns - -Adding cryptography to a filesystem comes at a cost--encryption consumes both -CPU cycles and additional memory pages. The latter is primarily managed the the -Linux page cache, but the former can result in reads and writes being CPU-bound -rather than I/O-bound. Based on testing compared to I/O direct to the solid -state disk, we’ve found the current overhead to be acceptable, though there is -always room to be better. Chromium OS uses encryption to protect cached user -data, which does include some frequently used items such as the browser cache -and databases, but does not include shared Operating System files and programs. -So while there is a performance cost, it does not apply to all disk I/O. - -## Reclaiming lost space - -Being a stacked filesystem, eCryptfs has a side effect of having a single lower -file for every upper file. This allows for “passthrough”--where a file or -directory is created as plaintext in the underlying vault folder, and shows up -in the upper mount point. Generally, this is not a good idea for files (we -wouldn’t want to store plaintext data on disk), but for directories, it has a -nice benefit of allowing offline file cleanup. - -Specifically, passthrough directories allow a plaintext directory name to show -up in the vault, but the directory contents (files, subdirectories, etc.) are -still encrypted. Chromium OS can create passthrough directories for discardable -information, such as the browser cache. Since the name is a standard location, -reclaiming disk space when it is low is simply a matter of removing the known -cache directory names from each vault that exists on the system. Without -passthrough directories, each vault would have to be mounted (requiring each -user’s credentials) to be able to identify the cache directory and delete it. - -## Managing encryption keys - -As mentioned earlier, the user’s vault keyset is ultimately protected by a -partial cryptographic hash of the user's passphrase. A partial cryptographic -hash is used because the user's Google Accounts password must not be exposed to -offline attack. Instead, we're willing to lose some of the entropy supplied by -the passphrase by performing a SHA-256 digest of a user-specific salt -concatenated with the user's passphrase. The first 128 bits of the digest are -used as the user's "weak password hash." - -Often, user passwords will only contain anywhere from [18 bits to 30 bits of -entropy](http://en.wikipedia.org/wiki/Password_strength#Human-generated_passwords) -starting at 8 characters in length. Spreading that entropy over a hash and -halving it is not really that great. At the very least, adding a salt means that -it will be quite costly to precompute a dictionary of hashes, but that still -isn't perfect if an attacker has the time and access to the local salt in order -to attempt to brute force the local password. There are two other useful options -that can be pursued to help dissuade offline attacks. The first is that [key -strengthening](http://en.wikipedia.org/wiki/Key_strengthening) can be used to -increase the amount of time it takes to calculate a hash from a dictionary-based -brute force attack that uses repeated passes through a cryptographic hash. The -other option is to make use of a TPM device when it is present, such as in -Chromium OS devices. This allows the TPM to effectively rate limit brute force -attacks without the overhead of key strengthening. - -As mentioned above, both Scrypt-based and TPM-based protection may be employed. -We’ll describe the Scrypt method first, being the simpler of the two. The -benefit of using Scrypt is that it uses a unique method of password -strengthening to derive a key for the encryption that is both CPU- and -memory-bound, making a parallelized brute-force attack more costly than a scheme -such as repeated hash rounds. Scrypt is used to directly protect the vault -keyset using a key based on the strengthened user’s password (actually the -user’s weak password hash). The encrypted keyset is stored on disk, and is -unlocked for use at login when valid credentials allow for it to be decrypted. - -The preferred method of protecting the vault keyset involves the TPM to protect -against brute force attacks. The method takes advantage of the secure key -storage functionality of the TPM (and the relatively slow clock speed of the -chip) to protect the keyset. - -When using the TPM, Chromium OS creates a system-wide RSA key using OpenSSL and -then wraps it with the TPM’s Storage Root Key on first boot. The original key is -destroyed, and we are left with an RSA key whose private key is known only -on-chip on the TPM. This means that, short of a vulnerability in the TPM, in -order to use this RSA key, the private key operation must happen on this -specific TPM, so mechanisms that use it are better protected against brute-force -attacks. Furthermore, as current TPMs can take around half a second for a -private key operation, even attacks which use the TPM are effectively rate -limited. - -We use a system-wide key rather than per-user key for two important reasons: -first, key load into the TPM is costly (it must decrypt the private key -component). While 0.5s is a tolerable penalty to pay for the private key -operation, longer (which would be the case if we had to load a user-specific key -on each login attempt) is a noticable impact on user login. Using a system-wide -key allows one key to be loaded and generally left in the TPM while the system -is booted. Since it is protected by the chip, this is okay. Second, the typical -method of incorporating user passwords with keys on the TPM is to require -authentication for their use. This would be a good solution for Chromium OS, but -there is a downside. TPMs feature a vendor-specific dictionary attack defense -mode, which is easily entered into with too many failed TPM authentications. On -many chips, this mode is aggressive, and once entered, renders the chip useless -for some amount of time. Since failed passwords happen (both legitimately, and -by an attacker), too many failed passwords would leave the TPM disabled for a -time, and the vaults it protected unusable. Instead, we use a passwordless -system-wide key, but mix the user’s password in a unique way that achieves the -same benefits without the possibility of entering this mode. - -Specifically, when a vault keyset is encrypted, the following process takes -place: - -1. A 256-bit AES vault keyset key (VKK below) is created randomly and - used to encrypt the vault keyset. The encrypted vault keyset is - stored to disk. This intermediate key is used to allow the vault - keyset to be larger than a single RSA operation would allow for. -2. The VKK is encrypted with the system-wide RSA key previously - mentioned (TPM_CHK below), resulting in an intermediate encrypted - vault keyset key (IEVKK). -3. The final 128 bits of the encrypted blob from step 2 are encrypted - in place using the user’s weak password hash which has first been - hashed a keyset-specific salt. This is done using a single AES - operation with no padding, allowing for no verification on decrypt. - The result is the encrypted vault keyset key (EVKK), which is stored - to disk. - -Step 3 is critical to combining the user’s weak password hash (and therefore -preventing anyone with access to the TPM and system-wide RSA key from decrypting -the vault keyset). Unless the attacker acquires the user’s password (or guesses -it), the blob passed to the TPM to decrypt using the system-wide key is -effectively garbage--the TPM will fail to decrypt because is is effectively -corrupt until those final 128 bits are corrected. The TPM treats this as a -failed decrypt (rather than a failed authentication), and the vault keyset key -(and vault keyset it protects) cannot be recovered. This operation is -illustrated below: - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/protecting-cached-user-data/ChromiumOS_TPM_Cryptohome.png">](/chromium-os/chromiumos-design-docs/protecting-cached-user-data/ChromiumOS_TPM_Cryptohome.png) - -While these protection methods result in protecting the cached user data, what -they effectively address is preventing an attacker from offline brute-force -guessing a user’s Google Accounts password. - -## Hashing the user name - -The primary reason for hashing the user name is to make it safe for use on the -filesystem without writing a custom mapping of unsafe characters. A web-based -user name may contain characters that are not safe for use on the file system. -Using a strong hash will give a reasonably unique value that is safe for use on -the file system. In addition, if all references to user names outside of the -encrypted images are done using a HASH(salt||user@domain.com), then it becomes -quite difficult for any user, or attacker, to determine what user names are in -use on a given device without attempting a brute force attack. This is a nice -side effect, but not an explicit goal. - -## **Password change and offline login** - -Password change and offline login are both handled through cryptohomed. - -### **Password change** - -If the user changes her password from another browser, the vault keyset is still -tied to the previous password. Chromium OS allows password change only on a -successful online login. In that case, Google’s web service will authenticate -the user with her new credentials, returning a successful condition. Chromium OS -will attempt to mount her vault, but that will fail with a bad decrypt -condition. Since Chromium OS knows the password was in fact correct, it will -present the user with the option of entering her old password once (with which -it can recover the vault keyset and then re-protect with the new password), or -simply re-create the vault from scratch. - -### **Offline login** - -When a Chromium notebook is offline, it cannot authenticate a user with Google’s -web service. However, the encrypted vault keyset can be used for offline -credential verification. Put simply, if a given password results in a successful -recovery of the vault keyset, the password was correct. So, when Chromium OS -doesn’t detect a network connection, it will attempt to authenticate by mounting -the user’s vault directly (this only works if the user has previously logged in -while online, and so she has a vault present on the notebook). - -## Encrypting swap and temporary data - -At present, Chromium OS devices don't use a swap partition, and temporary data -is written to a tmpfs file system that is not backed by any permanent storage -(such as disk). If a swap device is needed in the future, we'll explore the -challenges associated with protecting that data satisfactorily. - -### zram swap - -While newer releases of Chrome OS have started including zram for swap purposes, -it is still not stored on disk. zram takes a chunk of memory and uses that for -swap. When pages are moved between that swap device, they are -compressed/uncompressed on the fly. So there still is no permanent storage in -play. - -## Error conditions - -You may be wondering what happens if an error occurs. Is the user locked out? If -a user authenticates successfully, but all attempts to create or access an -encrypted vault fail, then login may or may not fail. The answer is dependent on -the scenario. Chromium OS automatically detects unrecoverable failures, and -re-creates a user’s vault in the conditions. These conditions are typically -related to the TPM, such as happens if it is cleared (effectively making data -protected by it unrecoverable). Transient failures do not result in a fresh -vault, but both types of failures are rare and usually predictable (see our -document on TPM use for more information). - -Regardless, the user can always log in to the device as a guest. While it does -not provide access to the encrypted user data, and the session history is not -stored, it does always allow use of the system. - -## Disk speed and battery life - -Simple benchmarks indicate that eCryptfs under common browsing patterns does not -significantly alter battery life. It's when sustained reads and writes occur -that more and more CPU is used, and hence more battery draw. A test with a -heavy, sustained write resulted in similar battery discharge rate as the heavy -writing used without encryption, but the encrypted large write took about twice -as long to complete. - -In most use cases, disk encryption isn't noticeable. If AES acceleration reaches -additional processors, then the impact will be even lower. - -## Directory structure - -All metadata for this feature will live under the /home/.shadow directory. Each -user will have a subdirectory with a name based on the user name hash. That -directory will contain all data related to that user's image on the machine. For -example: - -/home/.shadow/da39a3ee5e6b4b0d3255bfef95601890afd80709/vault - -/home/.shadow/da39a3ee5e6b4b0d3255bfef95601890afd80709/master.0 - -The system-wide TPM RSA key is stored in: - -/home/.shadow/cryptohome.key - -## Appendix A: Formal requirements - -The primary objective is to protect sensitive end user data from exposure if the -device or drive is lost. - -* User data **MUST NOT** be accessible when the device is powered - down. -* User data **MUST** be protected when the disk has been removed from - a device. -* User data **MAY** be protected when the device is suspended. -* Google Accounts passphrases **MUST NOT** be exposed for direct - offline attack. -* User data **MUST** be available during offline login. -* User data **MUST** be recoverable after an out-of-band password - change. -* User data **MUST** **NOT** be exposed across users on a multiuser - device. -* User data swapped out of memory **MAY BE** protected (depending on - the requirement below). -* Power and performance overhead **SHOULD BE** minimized. (Some - concrete maximum acceptable overhead should be determined. It may - end up being device specific, however. In particular, boot time, - login time, and Chromium browser responsiveness issues are - important.) -* Users **MUST NOT** be able to opt out of encryption. - -Non-goals: - -* Offline credential storage behavior may be a side effect of the - chosen implementation. -* A TPM device may be used, but it is a non-goal to create a - TPM-dependent solution. -* Root partition encryption/protection is not a goal. -* It is a non-goal to provide a remote wipe mechanism via this - feature, but it may make such a design easier. - -## Appendix B: Designs considered - -* **Per-user block-based encryption**: This approach would use a - random key to encrypt a partition per-user that is protected by the - user's passphrase (and a Google-held backup key). This approach - would ensure that a user's data will be accessible only when the - user has logged in, and it allows for easy remote wipe (removing the - key for the partition). This does add more pain when it comes to - device management. Each data partition would need to be separated - into preallocated regions for each simultaneously cached user stored - on the system (that is, data_space / n = users_space). - *Note:* This should not be visible to the user unless he tries to log in - with the new passphrase while he is offline. -* **Per-user file system-based encryption**: This approach would make - use of encfs, cryptofs, or eCryptfs to add a layered encrypted file - system on top of a preconfigured block device. Barring eCryptfs, the - performance is quite painful with userland solutions like encfs and - cryptofs. eCryptfs provides good performance, and allows for an - approach similar to above (guaranteeing per-user privacy and offline - defenses) without requiring preallocated block devices per user. -* **Whole disk encryption with TPM**: This would use a TPM-protected - key to unlock a drive. This would mean that with the whole device, - the data would be exposed to OS-level attacks once booted, unless - some form of pre-boot authentication was used. This is not the user - experience we are looking for, and this would expose data trivially - to a physical attacker. The disk would only be protected if removed - from the device. -* **Whole data partition encryption**: This would simplify the space - utilization problem by putting all users on one encrypted partition. - Normal access controls could keep users out of each other's data, - but adding a new user to the encrypted drive without requiring - another user to "authorize" the new user at login time is a - challenge under this scheme. The reason this is an issue is because - each user's passphrase-derived key is used to armor the disk - encryption key. Even if a user authorizes another user to access the - drive, the new user would be required to type the original - passphrase immediately (or at next login), or a secondary key would - need to be in play to allow for the encryption key to be decrypted - and re-encrypted without the first user present. There are no clean - ways around this. - Another downside to this approach is that if any user is compromised and a - privilege escalation vulnerability exists in the system, all user data would - be compromised. - Additionally, this approach would complicate any remote-wipe scheme we might - implement. Whole-partition encryption would necessitate actual file removal - to implement a per-user wipe, instead of just reinitializing the - cryptographic file system or full data wipe. If we support only full data - wipe, then that would probably be restricted to device owners only, but a - per-user wipe would allow a user to clean his own data off a friend's - machine he used once. -* **No encryption**: Not only would this mean most enterprises can't - use the device for work, it also would mean that many users would be - unhappy. As is, most people don't want to lose their data if someone - steals their device. And regardless of policies, people will use - their personal devices at work. We'd like to think that providing - simple, effective encryption to the platform is a long-term benefit - to everyone. -* **Chromium browser-only encryption**: Pushing encryption into the - Chromium browser was considered. This approach would ensure that all - cached data and all downloaded/accessed data is encrypted. This - solution may be pursued in the long term. At present, providing the - encryption at the OS level means that it will catch any data - external to the Chromium browser (Adobe Flash local storage) and - developer data. -* **Other interesting side effects**: Both the whole data partition - encryption and the per-user encryption come with the benefit of a - free offline credential store. A weak hash of the password can be - used to protect the stores which would then be checked on mount. In - addition, the user can be mapped to a system account using this - information. If a user is password slot 1, then he will be mapped to - user_1, and so on. - -## Appendix C: Threats - -The following list includes the primary threats that are addressed by this -design: - -* An attacker acquires a powered down device with the drive in it and - attempts to access all local user home directories. -* An attacker acquires a device in suspend state with the drive in it - and attempts to extract the encryption key with a cold boot attack. - -* An attacker with login privileges attempts to access another user's - data using a known bug, vulnerability, or workaround. - -Many threats are not dealt with through this design: - -* An attacker with a remote compromise will be able to access the - currently logged in user's data. -* An attacker will be able to access all user data if autologin is - enabled and the device is in any state. -* An attacker will be able to access all user data if the machine is - suspended or logged in and screen locking is disabled. -* An attacker will be able to dump all RAM from a suspended machine - using a [cold boot - attack](http://en.wikipedia.org/wiki/Cold_boot_attack) exposing any - credentials in memory and the disk encryption key schedule until - that can be fixed. -* An attacker who steals a running machine with screen locking will be - able to perform screen lock, usb, network, and any other runtime - attacks to gain access unless the user is able to initiate a remote - wipe while the machine is still Internet-connected. -* An attacker who is able to modify the root partition will be able to - compromise the runtime environment leading to data exposure when a - user subsequently logs in. Our [verified boot - process](/chromium-os/chromiumos-design-docs/verified-boot) should - help combat this, however.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/index.md deleted file mode 100644 index 682f4e2737e..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/index.md +++ /dev/null @@ -1,383 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: recovery-mode -title: Recovery Mode ---- - -[TOC] - -## Overview - -This document describes how recovery mode works in a Chromium-based OS. It -assumes the firmware has already booted in recovery mode, and is now searching -removable media (a USB key or an SD card) for a valid recovery image. - -In this document, the term "recovery storage device" (or "RSD") refers to a -removable drive containing Chromium OS recovery software. In contexts other than -that phrase, the term "device" refers to a Chromium OS-based device such as a -netbook. - -For background and other relevant information, see the following documents: - -* [Firmware Boot and - Recovery](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery): - how the firmware gets into recovery mode. -* [Developer - Mode](/chromium-os/chromiumos-design-docs/developer-mode): how - developer mode works, including the developer switch. -* [File - System/Autoupdate](http://www.chromium.org/chromium-os/chromiumos-design-docs/filesystem-autoupdate): - how normal updates work. -* [Firmware Verified - Boot](/chromium-os/chromiumos-design-docs/verified-boot): how kernel - images are signed. -* [Disk Format](/chromium-os/chromiumos-design-docs/disk-format): disk - drive format, used for both the Chromium OS device and the recovery - storage device. -* Boot Process (forthcoming): how kernel images are loaded into RAM - and executed. - -## Design goals - -The recovery software can vary by use case to satisfy a variety of different -goals. - -For the average user: - -* Only need to support one device model. -* Download the minimum amount of data. -* Should be a way to build a recovery device ahead of time; using this - recovery device for a restore should not require the network. -* Minimal interaction required. - -For the developer: - -* Support loading other operating systems. - -For manufacturing or corporate settings: - -* Support a range of device models. -* Centralized management of images to be pushed; updating the image - should not require re-flashing many RSDs. - -Security: - -* Content stored locally on the storage device should be used in - preference to that on the network. -* Use secure transport to network image server. -* Images are signed (see the [Firmware Verified - Boot](/chromium-os/chromiumos-design-docs/verified-boot) document) - -Simplicity & Robustness: - -* Push complexity from the firmware to the recovery kernel+rootfs, - where a full Linux-like environment is available. - -## Entering recovery mode - -For detailed information about entering recovery mode, see the [Firmware Boot -and Recovery](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery) -document. This section is only a brief summary. - -Recovery mode can be triggered: - -* Automatically by the Chromium OS firmware or software, if it - determines that the system is corrupt and that the backup - firmware/software is also corrupt. -* Manually, by the user pressing a Recovery Mode button during boot. - -In either case, if a storage device is present and it contains a signed recovery -image, then the firmware doesn't display anything; it assumes the recovery image -will display relevant information and tell the user what is going on. - -If a storage device is not present, or does not contain a valid recovery image, -then the firmware needs to get the user started on the recovery process. To do -this, it will display a screen which conveys the following information to the -user: - -* Your netbook has booted into recovery mode. -* Don't panic! -* Go to a URL specific to the device model for more instructions: - (insert model-specific URL here; the exact URL is still under - development - for example, - http://chromeos.google.com/recovery/*hardware_model_ID* or - http://goo.gl/recovery/*hardware_model_ID*) - -That information needs to be accessible in many languages, since we won't know -ahead of time what language the user is speaking, and the firmware which -displays the screen is non-interactive (does not have access to the keyboard). -One way to do this is to use just a pictograph and a URL. It is crucial to -instill the need for the user to go to another computer to visit the URL -specified on the screen. - -## Recovery website - -For devices shipping with Google Chrome OS, the recovery website will be a -Google-hosted website which provides instructions and installers for creating -RSDs. - -Instructions will be customized for each device model. For example, if a -particular netbook model does not have wired networking, the instructions for -that model won't suggest plugging in a network cable. The types of valid -external storage devices and their physical locations on the netbook can also be -customized per model. - -The recovery website will ask the user to get an appropriate storage device, -then download a recovery installer (see below) to finish the recovery. It should -also download a small config file with the model number of the netbook to be -recovered, so that the recovery installer can use that to fetch the right -recovery data. - -## Recovery installer - -This is a small program which helps a user create a RSD. - -Since the installer needs to run on any computer, we need multiple versions of -it (Windows, Mac, Linux, Chromium OS). The Chromium OS installer will be part of -the default Chromium OS install image, so that any Chromium OS device can be -used to create a RSD for any other Chromium OS device. - -The installer will do the following: - -1. Prompt the user to choose a target storage device to use as the RSD. -2. Verify the storage device is large enough. -3. Download the appropriate recovery software (recovery kernel + - recovery root filesystem) and save it to the RSD. -4. Download the appropriate recovery data (Chromium OS firmware + - kernel + rootfs + recovery configuration data) and save it to the - RSD. -5. Prompt the user to remove the storage device and plug it into the - netbook. - -For more information about the recovery software and data, see the "Types of -Data" section elsewhere in this document. - -The following features for the recovery installer are optional for version 1.0, -and may be included in future versions: - -* If the target storage device is big enough for the recovery - software, but too small to fit both the recovery software and - recovery data, the installer could help the user set up network - recovery mode (see below). -* It could offer to back up the existing contents of the storage - device before creating the RSD, then restore those contents to the - storage device after the recovery is done. -* It could detect the language setting on the source computer, then - automatically use that language for all displays and prompts during - recovery. - -One potential starting point for a recovery installer is -[UNetbootin](http://unetbootin.sourceforge.net/). It already makes recovery / -installer images for several OSes, and runs on Windows and Linux. - -## Types of data on a recovery storage device - -The following types of data may be present on a RSD. For information on which -kinds of data appear on each type of RSD, see the "Types of recovery storage -device" section, below. - -For more information on GUID partition tables and Chromium OS partition types, -see [Disk Format](/chromium-os/chromiumos-design-docs/disk-format). - -### Recovery kernel - -This is a Chromium OS kernel partition, signed by the same authority as the -firmware on the device. It boots the netbook into a more usable / interactive -state for the rest of recovery mode. - -For Google Chrome OS devices, the kernel is signed by Google. The user will -download the recovery kernel from the Google recovery website. - -A different recovery kernel is needed for each processor architecture (x86, -ARM). - -Since boot speed is not important, we don't need to optimize the kernel for -variants of each processor architecture; we can compile for the lowest common -feature set. - -### Recovery root filesystem - -This is a Chromium OS rootfs partition, signed by the same authority as the -recovery kernel. - -A recovery root filesystem can support a range of models with compatible -processor architecture; drivers for multiple types of display and storage can be -included. Alternately, if over time the diversity of Chromium OS devices makes -the size of the filesystem prohibitively large, we can produce multiple images, -each applicable to a subset of devices. In the latter case, the model-specific -URL provided by the recovery mode firmware will point the user to the correct -image for that device. - -To determine and/or verify which set of recovery data (Chromium OS kernel, -rootfs and firmware) is appropriate for the netbook to be recovered, the -read-only firmware on the netbook will supply an API for the recovery image to -discover the netbook's model number. (See the (forthcoming) System SKU Reporting -Specification.) - -### Recovery data - -This data partition contains data for the recovery. It may contain rootfs -image(s), firmware image(s), and/or network configuration. - -#### Chromium OS kernel+rootfs - -This is a full Chromium OS kernel+rootfs image. - -All images are signed to protect against accidental or intentional corruption. - -* It may be a Google-signed image provided by the Google Chrome OS - update server. -* Alternately, it may be a developer-signed image. See Developer Mode - below. - -Images will have a header that provides the following information: - -* Chromium OS version -* Signing authority (Google or developer) - -If multiple versions of an image are available from the same location (for -example, the data partition on the RSD), the newer version should take -precedence. - -Images signed by the same authority as the firmware take precedence over images -signed with other authorities. On a Google Chrome OS device, if both a -Google-signed image and a developer-signed image are available from the same -location (for example, the data partition on the RSD), the Google-signed image -should take precedence. This protects normal users. The developer documentation -on the Chromium OS website will provide instructions for developers to construct -an RSD without a Google-signed image, so that they can load their own OS. - -#### Chromium OS firmware - -This is firmware for a Chromium OS device. - -It has the same information about versions and signing authority in the filename -and/or header as the rootfs, and the same rules of precedence apply if multiple -firmware files are present. - -The firmware is likely to be included inside the Chromium OS rootfs. - -#### Network configuration - -This is configuration data for downloading recovery data over the network. This -includes: - -* order in which to attempt network interfaces (default: wired, then - wireless) -* WiFi SSID and WPA key, to support loading recovery data over WiFi - (no default; if not present and attempting wireless, prompt user) -* network configuration (gateway, DNS, etc.) (default: use DHCP) -* hostname or IP address of recovery server (default: Google's update - server) -* ssh keys for recovery server, to prevent attackers from presenting a - simulated server (default: whatever keys we use to sign - communication with our update server) -* should the image downloaded from the server be stored to the RSD's - data partition for future re-use, or not? (default: yes) - -## Types of recovery storage device - -A recovery storage device is a USB key or SD card containing one or more -recovery kernels and recovery root filesystems, and data to support the -recovery. - -(Other kinds of external storage devices may be usable in the future.) - -Depending on the contents of the RSD: - -* It may support only one model of netbook, or multiple models. -* It may load recovery data (rootfs + firmware) from the RSD itself, - or over the network. - -The following subsections describe some typical types of RSDs. - -### Single recovery storage device - -This is the most common type of RSD. It contains a single bootable recovery -image, and a single set of recovery data. It is self-contained; using it to -recover a device does not require access to the network. This is the type of RSD -created by a recovery installer. It is the least flexible RSD, since it supports -only a single device model. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/recovery-mode/srjy2ox0dbMkVTv25qW2QdA.png">](/chromium-os/chromiumos-design-docs/recovery-mode/srjy2ox0dbMkVTv25qW2QdA.png) - -### Network recovery storage device - -This type of RSD loads the Chromium OS software and firmware from a server, -rather than storing it directly on the card. - -The recovery software identifies the target device, then downloads matching -recovery data from the server. This allows a single RSD to support multiple -target device models. - -The server may be the main Google Chrome OS server provided by Google, or a -local server. This is suitable for a larger corporate, commercial, or -manufacturing environment. Upgrading the recovery data for a netbook model only -requires updating the server, not multiple RSDs. A new model can also be -supported without upgrading the RSDs, as long as the network recovery software -supports the new model's hardware. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/recovery-mode/sk78RvI_UcGGddwNUBzsAcA.png">](/chromium-os/chromiumos-design-docs/recovery-mode/sk78RvI_UcGGddwNUBzsAcA.png) - -Future versions of this specification may allow a PXE boot style implementation, -where the PXE code lives in the recovery kernel instead of the firmware. - -## Developer mode - -A developer can use recovery mode to load their own OS image onto a Google -Chrome OS device. To do this, all they need to do is make a RSD which contains -developer-signed recovery data in place of the Google-signed recovery data. - -See the Developer Mode design document (forthcoming) for more details. Key -points: - -* The device's developer mode switch must be turned on before - developer-signed images can be loaded. -* If the kernel on the RSD is signed with a different key than the - kernel on the destination device, there is a 5-minute delay, during - which time the user must interact with the screen and keyboard, and - sound is played. This makes it more difficult to discreetly - reprogram a device, protecting both users and developers. - -For i18n, the developer mode screens must be displayed in the proper language. -If the language is not detected/set by the recovery installer, then recovery -needs to have a language selection screen during startup. - -## Network recovery - -If appropriate recovery data for the target netbook is not present on the RSD, -the recovery software on the RSD can attempt to download the recovery data from -a server on the network. - -To deter spoofing attacks, communication with the server should be done over a -secure channel (such as sftp). - -The default behavior is to contact the Google Chrome OS update server, using -wired networking in preference to wireless. This can be overridden via a network -configuration file on the RSD. - -Network recovery is useful during manufacturing, to load the initial rootfs onto -the drive. This procedure would look something like this: - -1. As each assembled netbook is ready for programming, a tech - 1. takes an RSD from a bucket and plugs it into the netbook; - 2. plugs the netbook into external power (and possibly wired - network); - 3. powers on the netbook. -2. Recovery mode launches, because the netbook's drive has no bootable - image. -3. The network configuration file on the RSD points to a manufacturing - server. The correct image for the netbook is loaded from the server. -4. When the netbook is done recovering, the tech unplugs the RSD and - tosses it back in the bucket. - -Network recovery is also useful in corporate environments. Tech support can -maintain a single server with all appropriate images; each tech stop location -only needs a few generic RSDs to support all the models of netbooks. To deploy a -new image, only the server needs to be changed.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/sk78RvI_UcGGddwNUBzsAcA.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/sk78RvI_UcGGddwNUBzsAcA.png.sha1 deleted file mode 100644 index 1dc42728213..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/sk78RvI_UcGGddwNUBzsAcA.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -0e96d377bf54d6e4ffe32c8f9e909a9668e5566c
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/srjy2ox0dbMkVTv25qW2QdA.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/srjy2ox0dbMkVTv25qW2QdA.png.sha1 deleted file mode 100644 index 282ad665953..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/recovery-mode/srjy2ox0dbMkVTv25qW2QdA.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -f25955c6130b191d655329f3e22fb0f46dca26fc
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/security-overview/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/security-overview/index.md deleted file mode 100644 index 7606fa2276b..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/security-overview/index.md +++ /dev/null @@ -1,492 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: security-overview -title: Security Overview ---- - -[TOC] - -## Abstract - -* Chromium OS has been designed from the ground up with security in - mind. -* Security is not a one-time effort, but rather an iterative process - that must be focused on for the life of the operating system. -* The goal is that, should either the operating system or the user - detect that the system has been compromised, an update can be - initiated, and—after a reboot—the system will have been returned to - a known good state. -* Chromium OS security strives to protect against an opportunistic - adversary through a combination of system hardening, process - isolation, continued web security improvements in Chromium, secure - autoupdate, verified boot, encryption, and intuitive account - management. - -We have made a concerted effort to provide users of Chromium OS-based devices -with a system that is both practically secure and easy to use. To do so, we've -followed a set of four guiding principles: - -* The perfect is the enemy of the good. -* Deploy defenses in depth. -* Make devices secure by default. -* Don't scapegoat our users. - -In the rest of this document, we first explain these principles and discuss some -expected use cases for Chromium OS devices. We then give a high-level overview -of the threat model against which we will endeavor to protect our users, while -still enabling them to make full use of their cloud device. - -## Guiding principles - -**The perfect is the enemy of the good.** No security solution is ever perfect. -Mistakes will be made, there will be unforeseen interactions between multiple -complex systems that create security holes, and there will be vulnerabilities -that aren't caught by pre-release testing. Thus, we must not allow our search -for some mythical perfect system to stop us from shipping something that is -still very good. -**Deploy defenses in depth.** In light of our first principle, we will deploy a -variety of defenses to act as a series of stumbling blocks for the attacker. We -will make it hard to get into the system, but assume that the attacker will. -We'll put another layer of defenses in place to make it difficult to turn a user -account compromise into root or a kernel exploit. Then, we'll also make it -difficult for an attacker to persist their presence on the system by preventing -them from adding an account, installing services, or re-compromising the system -after reboot. -**Make it secure by default.** Being safe is not an advanced or optional -feature. Until now, the security community has had to deploy solutions that cope -with arbitrary software running on users' machines; as a result, these solutions -have often cost the user in terms of system performance or ease-of-use. Since we -have the advantage of knowing which software should be running on the device at -all times, we should be better able to deploy solutions that leave the user's -machine humming along nicely. -**Don't scapegoat our users.** In real life, people assess their risk all the -time. The Web is really a huge set of intertwined, semi-compatible -implementations of overlapping standards. Unsurprisingly, it is difficult to -make accurate judgments about one's level of risk in the face of such -complexity, and that is *not* our users' fault. We're working to figure out the -right signals to send our users, so that we can keep them informed, ask fewer -questions, require them to make decisions only about things they comprehend, and -be sure that we fail-safe if they don't understand a choice and just want to -click and make it go away. - -### Use cases and requirements - -We are initially targeting the following use cases with Chromium OS devices: - -* Computing on the couch -* Use as a lightweight, secondary work computer -* Borrowing a device for use in coffee shops and libraries -* Sharing a second computer among family members - -Targeting these goals dictates several security-facing requirements: - -* The owner should be able to delegate login rights to users of their - choice. -* The user can manage their risk with respect to data loss, even in - the face of device loss or theft. -* A user's data can't be exposed due to the mistakes of other users on - the system. -* The system provides a multi-tiered defense against malicious - websites and other network-based attackers. -* Recovering from an attack that replaces or modifies system binaries - should be as simple as rebooting. -* In the event of a security bug, once an update is pushed, the user - can reboot and be safe. - -### Our threat model - -When designing security technology for Chromium OS systems, we consider two -different kinds of adversaries: - -* An **opportunistic adversary** -* A **dedicated adversary** - -The **opportunistic adversary** is just trying to compromise an individual -user's machine and/or data. They are not targeting a specific user or -enterprise, and they are not going to steal the user's machine to obtain the -user's data. The opportunistic adversary will, however, deploy attacks designed -to lure users on the web to sites that will compromise their machines or to web -apps that will try to gain unwarranted privileges (webcam access, mic access, -etc). If the opportunistic adversary *does* steal a device and the user's data -is in the clear, the opportunistic adversary may take it. - -The **dedicated adversary** *may* target a user or an enterprise specifically -for attack. They are willing to steal devices to recover data or account -credentials (not just to re-sell the device to make money). They are willing to -deploy DNS or other network-level attacks to attempt to subvert the Chromium OS -device login or update processes. They may also do anything that the -opportunistic adversary can do. - -For version 1.0, we are focusing on dangers posed by opportunistic adversaries. -We further subdivide the possible threats into two different classes of attacks: -**remote system compromise** and **device theft.** - -## Mitigating remote system compromise - -There are several vectors through which an adversary might try to compromise a -Chromium OS device remotely: an exploit that gives them control of one of the -Chromium-based browser processes, an exploit in a plugin, tricking the user into -giving a malicious web app unwarranted access to HTML5/Extension APIs, or trying -to subvert our autoupdate process in order to get some malicious code onto the -device. - -As in any good security strategy, we wish to provide defense in depth: -mechanisms that try to prevent these attacks and then several more layers of -protection that try to limit how much damage the adversary can do provided that -he's managed to execute one of these attacks. The architecture of Chromium -browsers provides us with some very nice process isolation already, but there is -likely more that we can do. - -### OS hardening - -The lowest level of our security strategy involves a combination of OS-level -protection mechanisms and exploit mitigation techniques. This combination limits -our attack surface, reduces the the likelihood of successful attack, and reduces -the usefulness of successful user-level exploits. These protections aid in -defending against both opportunistic and dedicated adversaries. The approach -designed relies on a number of independent techniques: - -* Process sandboxing - * Mandatory access control implementation that limits resource, - process, and kernel interactions - * Control group device filtering and resource abuse constraint - * Chrooting and process namespacing for reducing resource and - cross-process attack surfaces - * Media device interposition to reduce direct kernel interface - access from Chromium browser and plugin processes -* Toolchain hardening to limit exploit reliability and success - * NX, ASLR, stack cookies, etc -* Kernel hardening and configuration paring -* Additional file system restrictions - * Read-only root partition - * tmpfs-based /tmp - * User home directories that can't have executables, privileged - executables, or device nodes -* Longer term, additional system enhancements will be pursued, like - driver sandboxing - -Detailed discussion can be found in the [System -Hardening](/chromium-os/chromiumos-design-docs/system-hardening) design -document. - -### Making the browser more modular - -The more modular the browser is, the easier it is for the Chromium OS to -separate functionality and to sandbox different processes. Such increased -modularity would also drive more efficient IPC within Chromium. We welcome input -from the community here, both in terms of ideas and in code. Potential areas for -future work include: - -* Plugins - * All plugins should run as independent processes. We can then - apply OS-level sandboxing techniques to limit their abilities. - Approved plugins could even have Mandatory Access Control (MAC) - policies generated and ready for use. -* Standalone network stack - * Chromium browsers already sandbox media and HTML parsers. - * If the HTTP and SSL stacks were isolated in independent - processes, robustness issues with HTTP parsing or other behavior - could be isolated. In particular, if all SSL traffic used one - process while all plaintext traffic used another, we would have - some protection from unauthenticated attacks leading to full - information compromise. This can even be beneficial for cookie - isolation, as the HTTP-only stack should never get access to - cookies marked "secure." It would even be possible to run two - SSL/HTTP network stacks—one for known domains based on a large - whitelist and one for unknown domains. Alternately, it could be - separated based on whether a certificate exception is required - to finalize the connection. -* Per-domain local storage - * If it were possible to isolate renderer access per domain, then - access to local storage services could similarly by isolated—at - a process level. This would mean that a compromise of a renderer - that escapes the sandbox would still not be guaranteed access to - the other stored data unless the escape vector were a - kernel-level exploit. - -### Web app security - -As we enable web applications to provide richer functionality for users, we are -increasing the value of web-based exploits, whether the attacker tricks the -browser into giving up extra access or the user into giving up extra access. We -are working on multiple fronts to design a system that allows Chromium OS -devices to manage access to new APIs in a unified manner, providing the user -visibility into the behavior of web applications where appropriate and an -intuitive way to manage permissions granted to different applications where -necessary. - -* User experience - * The user experience should be orthogonal to the policy - enforcement mechanism inside the Chromium browser and, ideally, - work the same for HTML5 APIs and Google Chrome Extensions. -* HTML5/Open Web Platform APIs and Google Chrome Extensions - * We're hopeful that we can unify the policy enforcement - mechanisms/user preference storage mechanisms across all of the - above. -* Plugins - * We are developing a multi-tiered sandboxing strategy, leveraging - existing Chromium browser sandboxing technology and some of the - work we discuss in our [System - Hardening](/chromium-os/chromiumos-design-docs/system-hardening) - design document. - * Long term, we will work with other browser vendors to make - plugins more easily sandboxed. - * Full-screen mode in some plugins could allow an attacker to mock - out the entire user experience of a Chromium OS device. We are - investigating a variety of mitigation strategies in this space. - -As HTML5 features like persistent workers move through the standards process, we -must ensure that we watch for functionality creeping in that can poke holes in -our security model and take care to handle it appropriately. - -### Phishing, XSS, and other web vulnerabilities - -Phishing, XSS, and other web-based exploits are no more of an issue for Chromium -OS systems than they are for Chromium browsers on other platforms. The only -JavaScript APIs used in web applications on Chromium OS devices will be the same -HTML5 and Open Web Platform APIs that are being deployed in Chromium browsers -everywhere. As the browser goes, so will we. - -### Secure autoupdate - -Attacks against the autoupdate process are likely to be executed by a dedicated -adversary who would subvert networking infrastructure to inject a fake -autoupdate with malicious code inside it. That said, a well supported -opportunistic adversary could attempt to subvert the update process for many -users simultaneously, so we should address this possibility here. (For more on -this subject, also see the [File -System/Autoupdate](/chromium-os/chromiumos-design-docs/filesystem-autoupdate) -design document.) - -* Signed updates are downloaded over SSL. -* Version numbers of updates can't go backwards. -* The integrity of each update is verified on subsequent boot, using - our Verified Boot process, described below. - -### Verified boot - -Verified boot provides a means of getting cryptographic assurances that the -Linux kernel, non-volatile system memory, and the partition table are untampered -with when the system starts up. This approach is not "trusted boot" as it does -not depend on a TPM device or other specialized processor features. Instead, a -chain of trust is created using custom read-only firmware that performs -integrity checking on a writable firmware. The verified code in the writable -firmware then verifies the next component in the boot path, and so on. This -approach allows for more flexibility than traditional trusted boot systems and -avoids taking ownership away from the user. The design is broken down into two -stages: - -* Firmware-based verification - (for details, see the [Firmware Boot and - Recovery](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery) - design document) - * Read-only firmware checks writable firmware with a permanently - stored key. - * Writable firmware then checks any other non-volatile memory as - well as the bootloader and kernel. - * If verification fails, the user can either bypass checking or - boot to a safe recovery mode. -* Kernel-based verification - (for details, see the [Verified - Boot](/chromium-os/chromiumos-design-docs/verified-boot) design document) - * This approach extends authenticity and integrity guarantees to - files and metadata on the root file system. - * All access to the root file system device traverses a - transparent layer which ensure data block integrity. - * Block integrity is determined using cryptographic hashes stored - after the root file system on the system partition. - * All verification is done on-the-fly to avoid delaying system - startup. - * The implementation is not tied to the firmware-based - verification and may be compatible with any trusted kernel. - -When combined, the two verification systems will perform as follows: - -* Detects changes at boot-time - * Files, or read-write firmware, changed by an opportunistic - attacker with a bootable USB drive will be caught on reboot. - * Changes performed by a successful runtime attack will also be - detected on next reboot. -* Provides a secure recovery path so that new installs are safe from - past attacks. -* Doesn't protect against - * Dedicated attackers replacing the firmware. - * Run-time attacks: Only code loaded from the file system is - verified. Running code is not. - * Persistent attacks run by a compromised Chromium browser: It's not possible to verify the browser's configuration as safe using this technique. - -It's important to note that at no point is the system restricted to code from -the Chromium project; however, if Google Chrome OS is used, additional -hardware-supported integrity guarantees can be made. - -### Rendering pwned devices useless - -We do not intend to brick devices that we believe to be hacked. If we can -reliably detect this state on the client, we should just initiate an update and -reboot. We could try to leverage the abuse detection and mitigation mechanisms -in the Google services that people are using from their Chromium OS devices, but -it seems more scalable to allow each service to continue handling these problems -on its own. - -## Mitigating device theft - -A stolen device is likely to have a higher value to a dedicated adversary than -to an opportunistic adversary. An opportunistic adversary is more likely to -reset the device for resale, or try to log in to use the device for himself. - -The challenges here are myriad: - -* We want to protect user data while also enabling users to opt-in to - auto-login. -* We want to protect user data while also allowing users to share the - device. -* We especially want to protect user credentials without giving up - offline login, auto-login, and device sharing. -* Disk encryption can have real impact on battery life and performance - speed. -* The attacker can remove the hard drive to circumvent OS-level - protections. -* The attacker can boot the device from a USB device. - -### Data protection - -Users shouldn't need to worry about the privacy of their data if they forget -their device in a coffee shop or share it with their family members. The easiest -way to protect the data from opportunistic attackers is to ensure that it is -unreadable except when it is in use by its owner. - -The [Protecting Cached User -Data](/chromium-os/chromiumos-design-docs/protecting-cached-user-data) design -document provides details on data protection. Key requirements for protecting -cached user data (at rest) are as follows: - -* Each user has their own encrypted store. -* All user data stored by the operating system, browser, and any - plugins are encrypted. -* Users cannot access each other's data on a shared device. -* The system does not protect against attacks while a user is logged - in. -* The system will attempt to protect against memory extraction (cold - boot) attacks when additional hardware support arrives. -* The system does not protect against root file system tampering by a - dedicated attacker (verified boot helps there). - -### Account management - -Preventing the adversary from logging in to the system closes one easy pathway -to getting the machine to execute code on their behalf. That said, many want -this device to be just as sharable as a Google Doc. How can we balance these -questions, as well as take into account certain practical concerns? These issues -are discussed at length in the [User Accounts and -Management](/chromium-os/chromiumos-design-docs/user-accounts-and-management) -design document, with some highlights below. - -* What *are* the practical concerns? - * Whose wifi settings do you use? Jane's settings on Bob's device - in Bob's house don't work. - * But using Bob's settings no matter where the device is doesn't - work either. - * And if Bob's device is set up to use his enterprise's wifi, then - it's dangerous if someone steals it! -* "The owner" - * Each device has one and only one owner. - * *User preferences* are distinct from *system settings.* - * *System settings*, like wifi, follow the owner of a device. -* Whitelisting - * The owner can whitelist other users, who can then log in. - * The user experience for this feature should require only a few - clicks or keystrokes. -* Promiscuous mode - * The owner can opt in to a mode where anyone with a Google - account can log in. -* Guest mode - * Users can initiate a completely stateless session, which does - not sync or cache data. - * All system settings would be kept out of this session, including - networking config. - -### Login - -For design details, see the [Login](/chromium-os/chromiumos-design-docs/login) -design document. - -At a high level, here is how Chromium OS devices authenticate users: - -1. Try to reach Google Accounts online. -2. If the service can't be reached, attempt to unwrap the - locally-stored, TPM-wrapped keys we use for per-user data - encryption. Successful unwrap means successful login. -3. For every successful online login, use a salted hash of the password - to wrap a fresh encryption key using the TPM. - -There are a number of important convenience features around authentication that -we must provide for users, and some consequences of integrating with Google -Accounts we must deal with. These all have security consequences, and we discuss -these (and potential mitigation) here. Additionally, we are currently working -through the various tradeoffs of supporting non-Google OpenID providers as an -authentication backend. -**CAPTCHAs** - -Rather than strictly rate limiting failed authentication attempts, Google -Accounts APIs respond with CAPTCHAs if our servers believe an attack is -underway. We do not want users to face CAPTCHAs to log in to their device; if -the user has correctly provided their credentials, they should be successfully -logged in. -Furthermore, including HTML rendering code in our screen locker would introduce -more potential for crashing bugs, which would give an attacker an opportunity to -access the machine. That said, we cannot introduce a vector by which attackers -can brute force Google Accounts. -To work around this right now, we do offline credential checking when unlocking -the screen and ignore problems at login time, though we realize this is not -acceptable long-term and are considering a variety of ways to address this issue -in time for V1. - -**Single signon** - -As we discuss in the [Login](/chromium-os/chromiumos-design-docs/login) design -document, we want to provide an SSO experience on the web for our users. Upon -login, we decrypt the user's profile and perform a request for her Google -Accounts cookies in the background. As a result, her profile gets fresh cookies -and she is logged in to all her Google services. - -### Future work - -**Auto-login** - -When a user turns on auto-login, they are asserting that they wish this device -to be trusted as though it had the user's credentials at all times; however, we -don't want to store actual Google Account credentials on the device—doing so -would expose them to offline attack unnecessarily. We also don't want to rely on -an expiring cookie; auto-login would "only work sometimes," which is a poor user -experience. We would like to store a revokable credential on the device, one -that can be exchanged on-demand for cookies that will log the user in to all of -their Google services. We're considering using an OAuth token for this purpose. - -**Biometrics, smartcards and Bluetooth** - -We expect to keep an eye on biometric authentication technologies as they -continue to become cheaper and more reliable, but at this time we believe that -the cost/reliability tradeoff is not where it needs to be for our target users. -We expect these devices to be covered in our users' fingerprints, so a low-cost -fingerprint scanner could actually increase the likelihood of compromise. We -were able to break into one device that used facial recognition authentication -software just by holding it up to the user's photo. Bluetooth adds a whole new -software stack to our login/screenlocker code that could potentially be buggy, -and the security of the pairing protocol has been [criticized in the -past](http://www.schneier.com/blog/archives/2005/06/attack_on_the_b_1.html). -Smart cards and USB crypto tokens are an interesting technology, but we don't -want our users to have to keep track of a physically distinct item just to use -their device. - -**Single signon** - -For third-party sites, we would like to provide credential generation and -synced, cloud-based storage. - -## Wrapup - -In this document, we have aimed only to summarize the wide-ranging efforts we are undertaking to secure Chromium OS at all levels. For more detail, please read the rest of the [design documents.](/chromium-os/chromiumos-design-docs)
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/chromepng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/chromepng.sha1 deleted file mode 100644 index f6d11aba0c9..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/chromepng.sha1 +++ /dev/null @@ -1 +0,0 @@ -e04ebc2ef990fa84389c667654f37cfb853935da
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/firmwarepng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/firmwarepng.sha1 deleted file mode 100644 index 6b4167d171d..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/firmwarepng.sha1 +++ /dev/null @@ -1 +0,0 @@ -5f9656ed3823b61a75d5768e65d4f7af6657d197
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/index.md deleted file mode 100644 index 9e687ac6fc5..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/index.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: software-architecture -title: Software Architecture ---- - -[TOC] - -## Abstract - -Chromium OS consists of three major components: - -* The Chromium-based browser and the window manager -* System-level software and user-land services: the kernel, drivers, - connection manager, and so on -* Firmware - - - -## High-level design - -We'll look at each component, starting with the firmware. - -### Firmware - -The firmware plays a key part to make booting the OS faster and more secure. To -achieve this goal we are removing unnecessary components and adding support for -verifying each step in the boot process. We are also adding support for system -recovery into the firmware itself. We can avoid the complexity that's in most PC -firmware because we don't have to be backwards compatible with a large amount of -legacy hardware. For example, we don't have to probe for floppy drives. - -Our firmware will implement the following functionality: - -* [**System - recovery**](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery)**:** - The recovery firmware can re-install Chromium OS in the event that - the system has become corrupt or compromised. -* **[Verified - boot](/chromium-os/chromiumos-design-docs/verified-boot):** Each - time the system boots, Chromium OS verifies that the firmware, - kernel, and system image have not been tampered with or become - corrupt. This process starts in the firmware. -* [**Fast - boot**](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery)**:** - We have improved boot performance by removing a lot of complexity - that is normally found in PC firmware. - - - -### System-level and user-land software - -From here we bring in the Linux kernel, drivers, and user-land daemons. Our -kernel is mostly stock except for a handful of patches that we pull in to -improve boot performance. On the user-land side of things we have streamlined -the init process so that we're only running services that are critical. All of -the user-land services are managed by Upstart. By using Upstart we are able to -start services in parallel, re-spawn jobs that crash, and defer services to make -boot faster. - -Here's a quick list of things that we depend on: - -* **D-Bus:** The browser uses D-Bus to interact with the rest of the - system. Examples of this include the battery meter and network - picker. -* **Connection Manager:** Provides a common API for interacting with - the network devices, provides a DNS proxy, and manages network - services for 3G, wireless, and ethernet. -* **WPA Supplicant:** Used to connect to wireless networks. -* **Autoupdate:** Our autoupdate daemon silently installs new system - images. -* **Power Management:** (ACPI on Intel) Handles power management - events like closing the lid or pushing the power button. -* **Standard Linux services:** NTP, syslog, and cron. - -### Chromium and the window manager - -The window manager is responsible for handling the user's interaction with -multiple client windows. It does this in a manner similar to that of other X -window managers, by controlling window placement, assigning the input focus, and -exposing hotkeys that exist outside the scope of a single browser window. Parts -of the ICCCM (Inter-Client Communication Conventions Manual) and EWHM (Extended -Window Manager Hints) specifications are used for communication between clients -and the window manager where possible. -The window manager also uses the XComposite extension to redirect client windows -to offscreen pixmaps so that it can draw a final, composited image incorporating -their contents itself. This lets windows be transformed and blended together. -The window manager contains a compositor that animates these windows and renders -them via OpenGL or OpenGL|ES. - - diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/overviewpng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/overviewpng.sha1 deleted file mode 100644 index fc0fe8d130b..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/overviewpng.sha1 +++ /dev/null @@ -1 +0,0 @@ -9016299e3dfb1581eb85d3bc628fce62ddcb5a98
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/systempng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/systempng.sha1 deleted file mode 100644 index 497b9c4a361..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/software-architecture/systempng.sha1 +++ /dev/null @@ -1 +0,0 @@ -d5e2c9a6872ec53072f8069f6acb45e378dd6cd9
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/source-code-management/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/source-code-management/index.md deleted file mode 100644 index 25aecf56441..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/source-code-management/index.md +++ /dev/null @@ -1,220 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: source-code-management -title: Source Code Management ---- - -[TOC] - -## Abstract - -* Chromium OS is an open source project with the goal of keeping good - relationships and communication with upstream sources. Chromium OS - is thus structured to upstream code early and often. -* Chromium OS will use Git as its version control system, Rietveld for - code reviews and gclient for package checkout management. - -This document outlines the Chromium OS process for managing the upstream bits -that make up the Chromium OS Linux distribution. - -## Background - -Chromium OS is an open-source project, built for people who spend most of their -time on the web. Chromium OS is currently made up of 190 or so open-source -packages. - -## Requirements - -**Sources hosted by us.** Upstream mirrors can be rearranged: sources can be -moved, removed, or modified. We also need the sources locally since we need them -in order to build. -**One format.** The upstream sources may be stored in any number of version -control systems (VCS). Or they may not even be stored in a VCS and instead be -available as a tarball, .deb, or .rpm. For our hosted sources, we'd like to have -them stored in one VCS format. This may require import from a VCS into our VCS. -Alternatively, we could store sources in their native VCS but this would require -our developers to work with multiple VCSs. -**Avoid patch files.** Patch files are difficult to maintain and difficult to -keep in sync with upstream. We plan to keep a copy of the entire upstream source -tree in a VCS. - -**Simple upstreaming.** We want to upstream early and often to minimize the set -of changes we have to maintain. We also want a good partnership with upstream. -**Simple tracking.** It needs to be easy to track what bits we've added to -open-source components. It should also be easy to track the upstream status of -our changes: Has the patch been sent upstream? Has it been accepted, rejected, -or accepted in modified form? - -## Design objectives - -**Upstream package database.** We need a database that stores metadata about -each package we are using: homepage, upstream repository information, upstream -bugs database location, mailing list, license, upstream submission process, and -so on. Examples: -[fedora](https://admin.fedoraproject.org/pkgdb/acls/name/acpid), -[debian](http://packages.debian.org/source/lenny/acpid), -[ubuntu](https://launchpad.net/ubuntu/+source/acpid), -[gentoo](http://packages.gentoo.org/package/sys-power/acpid). - -**Distributed Version Control System (DVCS).** Without a -[DVCS](http://en.wikipedia.org/wiki/Distributed_revision_control), distributed -development (working with upstream) is difficult. With a DVCS, cloning, merging, -and pulling in upstream changes are common operations. For a traditional VCS, -cloning an upstream repository involves initially using rsync to copy the -upstream repository (assuming such access is given) and then special scripts to -handle future pulls. - -**Each package in its own repository.** Pulling multiple upstream trees into one -monolithic tree is difficult. It would also make it nearly impossible to use a -DVCS. We want to make it easy to work with upstream, easy to track ancestry of -our code, and easy to track what patches we've applied. This is hard to do with -a monolithic repository. With a repository per package, a simple invocation of -the VCS tool's diff will do. - -**One DVCS tool.** We could mirror each upstream repository locally in its -native VCS but then we wouldn't be able to use a DVCS where upstream is not. -Also, it would force our developers to have to deal with many different VCS -tools. We could write our own wrapper library but that's extra development -effort we should be able to avoid. Instead, we should just mirror in our one -DVCS format, sources which are managed upstream with a different DVCS. - -**A DVCS with good import/export capability.** Many of the most popular DVCS -tools support import and export to other backends. The DVCS will support checkin -and checkout of code to and from a different backend. Often the import/export is -seamless enough that the workflow for dealing with a different backend is not -much different from the normal workflow. - -**Commit log metadata.** For some modules, we will be merging in patches from -different sources. For such patches, we will annotate the commit log with -metadata which documents the original source. - -## Detailed design - -### Version control system - -Out of all the available DVCS tools, Git implements our design objectives best. -Also, of the projects we use which use a DVCS, most use Git. - -Our process for managing source repositories will be as follows: - -1. We will host a Git repository for all source components which we are - modifying.Whenever a new package or dependency is added, a new Git - repository will be created. -2. For upstream Git sources, our repository will be a clone of - upstream. -3. For upstream sources not stored in Git, we'll create an additional - Git repository. - * The import repository will be named *<project>*-import. - * For example, project foo would be: foo-import - * Our Git repository will be "based" on the import. - * We will automatically re-sync the import repository with - upstream on a regular basis (for example, daily). -4. For sources which are not hosted in a VCS upstream, we'll create an - additional Git repository. Typically this will come from a tarball, - deb package, or rpm package. - * The repository will contain the unpackaged source tree. - * Any patches held in the package will be applied as subsequent - changes in Git. - * Future upstream releases will be committed to this repository in - the same way. - * The repository will be named: *<project>*-import - * For example, project bar would be: bar-import - * Our repository will be "based" on the import. -5. We will create a branch off of a recent stable upstream release and - make our changes there. -6. To make future syncing easier, and to make it easier to separate out - our bits from upstream, we will - [rebase](http://www.gitready.com/intermediate/2009/01/31/intro-to-rebase.html). - Rebasing is a best-practice for managing upstream sources. For - example, the Ubuntu kernel is regularly rebased against upstream. - The alternative, merging commits from upstream, results in local - changes being interleaved with upstream. Rebasing creates a clean - history with local changes appearing after upstream commits. - -The diagram below shows the flow of code between repositories. Repositories -above the top dashed-line are upstream-hosted, the repositories between the -dashed-lines are Google-hosted, and the repositories below the lower dashed-line -are the user's local checkout. -Where upstream is non-Git, we'll create a local Git import tree: "bar svn -import" and "bash .deb import" above. The Google repository will be a Git -repository based on a particular upstream tag of the upstream Git repository (or -our Git import if upstream is not using Git). - - - -### Package database - -Initially, we will store a README.chromium file in the root directory of the -package which will contain the following fields in shell parseable format: - -1. DESCRIPTION: one-line description of the package -2. HOMEPAGE: URL to upstream page (possibly a freshmeat.net page) -3. UPSTREAM_REPO: URL for upstream repository -4. LOCAL_GIT_REPO: git url for our Git repository -5. UPSTREAM_BUGSDB: URL for upstream bug database -6. LOCAL_BUGSDB: URL for our bug database for this package -7. LICENSE: name of license (we'll need a license database somewhere) - -Eventually, we may move to a more sophisticated web-hosted package database like -[Launchpad](https://launchpad.net/). In a web-hosted package database, we could -also display derived information like: current Chromium OS version, upstream -latest version, and so on. - -### Sample README.chromium for busybox - -DESCRIPTION="Utilities for embeddded systems" -HOMEPAGE="http://www.busybox.net" -UPSTREAM_REPO="git://git.busybox.net/busybox" -LOCAL_GIT_REPO="https://chromium.googlesource.com/external/busybox" -UPSTREAM_BUGSDB="https://bugs.busybox.net/" -LOCAL_BUGSDB="http://tracker.chromium.org/busybox" -LICENSE="GPL-2" - -LICENSE_FILE="LICENSE" - -### Commit log metadata - -We want to track where our sources are coming from. The mechanism for doing this -will be to recommend that the following metadata be included as part of the -commit log: - -* Source: Where did this change come from? Google? Ubuntu? kernel.org? -* Commit ID: Git commit id / SVN log number / patch number in package -* Tested status: Leaning on people to describe how they tested a - checkin has proved to be extremely effective "social engineering" in - other projects -* Upstream status: Tracking upstream status is more difficult as the - status will evolve over time. This data may need to be in a separate - database, but tying the database into the DVCS will not be simple to - keep consistent over Git rebases (which change Git commit IDs). - -### Code review - -Our current code review tool, -[Rietveld](http://code.google.com/appengine/articles/rietveld.html), already -supports Git change uploads via -[update.py](http://code.google.com/p/rietveld/source/browse/trunk/static/upload.py). -For making it easier to do Git code reviews with Rietveld, we will be using -[git-cl](http://groups.google.com/group/codereview-discuss/browse_thread/thread/d9f65d04165e274f/b8740b9beab78e4c), -. There is also an open-source tool designed specifically for Git code reviews: -[gerrit](http://source.android.com/submit-patches/workflow). It is a [fork of -reitveld](http://code.google.com/p/gerrit/wiki/Background). - -### Checkout management - -We are using [gclient](http://code.google.com/p/gclient/) for package checkout -management. - -## References and links - -### Git tutorial - -<http://www-cs-students.stanford.edu/~blynn/gitmagic/> - -### Git documentation - -<http://git-scm.com/documentation> diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/source-code-management/repo-archpng.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/source-code-management/repo-archpng.sha1 deleted file mode 100644 index 3958e0716b9..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/source-code-management/repo-archpng.sha1 +++ /dev/null @@ -1 +0,0 @@ -443d64eddafce617287f2f2588f46c5c63c52275
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/system-hardening/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/system-hardening/index.md deleted file mode 100644 index e8362f4cdb4..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/system-hardening/index.md +++ /dev/null @@ -1,692 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: system-hardening -title: System Hardening ---- - -[TOC] - -Note: A practical guide for hardening individual services can be found -[here](https://chromium.googlesource.com/chromiumos/docs/+/HEAD/sandboxing.md). - -## Abstract - -* Chromium OS strives to make remote attacks more difficult by using - multiple techniques ranging from privilege minimization to - compile-time hardening. -* A phased approach to system hardening is proposed to iteratively - reduce the exposed attack surfaces and increase the number of - defensive layers. - -This document lays out a technical vision for making Chromium OS-based systems -difficult for remote attackers to compromise using various system-level -mechanisms. Three objectives guide this vision: - -* Reducing surface area exposed to attack -* Reducing ability to successfully and reliably exploit any exposed, - vulnerable software -* Reducing benefit of a successful exploitation - -Efforts to secure Linux environments tend to revolve around the [principle of -least privilege](http://web.mit.edu/Saltzer/www/publications/protection/) and -applying exploit mitigation tactics wherever possible. While the exploit -mitigation techniques are effective, they are never a perfect defense and often -the specific techniques deployed vary from distribution to distribution. In -addition, the principle of least privilege is excellent in a server environment -and for locking down system services on desktops. However, desktop systems are -meant to be general purpose. This makes it incredibly difficult to determine the -least privilege needed if a program has not ever been seen on the system before -(or was written since the system was installed!). The end result is that the -risks from interactively executed applications are addressed only using exploit -mitigations and not as comprehensively as desired. - -Chromium OS has an advantage. All native programs run by the end user are known -in advance since all general purpose applications are web applications. We use -this knowledge to apply comprehensive access control enforcement in addition to -the well-known exploit mitigation techniques. This combination allows Chromium -OS to benefit from the great work securing Linux in both end-user and server -enviroments! - -## Technology - -[Control Groups -(cgroups)](http://lxr.linux.no/linux+v2.6.30/Documentation/cgroups/) are a -somewhat recent addition to the Linux kernel. They are a hierarchical collection -of tasks that can be arbitrarily created. Once a task has been associated with a -hierarchy, it falls under runtime constraints ranging from limited device node -access to limited CPU and memory usage. cgroups restrictions can be specified in -terms of percentages or as constants which provide an intuitive means of -*"guaranteeing"* that processes operate within their given bounds. This feature -is great for constraining denial of service attacks and general robustness -issues (and combines well with rlimit). The device filtering is useful for -limiting /dev access in constrained namespaces. - -[Namespacing](http://lxr.linux.no/linux+v2.6.30/Documentation/namespaces/) -provide a means of isolating processes, and process trees, from other running -processes on a system. Their use has been driven largely by the [Linux -VServer](http://wiki.linux-vserver.org/). Namespaces can be created only when a -new process is started using clone(). The following namespaces are currently -available in the upstream kernel: - -* PID: The process called with a new PID namespace becomes pid=1 and - is treated as a local *init* for the purposes of reparenting - orphaned processes. When this process terminates, the namespace is - shut down. Processes in this namespace can only "see" other - processes in the namespace and a custom /proc can be mounted to - provide visibility. A custom mount should be paired with a VFS - namespace. -* UTS: UTS namespacing allows for a custom host and domain names to be - set for a given namespace. -* User: User namespacing remaps users to new, localized UIDs. At - present, this functionality is immature in the kernel as many - guarantees are not met, such as similar UIDs in different namespaces - sharing a global UID. -* IPC: This isolates SystemV IPC ids to the processes in this - namespace. This means that globally shared memory, semaphores, and - message queues are accessible only to other processes in the same - namespace. -* [VFS](http://www.ibm.com/developerworks/linux/library/l-mount-namespaces.html): - VFS namespacing allows for a custom view of the locally mounted file - systems. In addition, any file systems mounted in a VFS namespace - will automatically be unmounted when all the processes in the - namespace have terminated. In addition, mount namespacing also comes - with interesting new sharing attributes (make-private, - make-unbindable, make-slave, make-shared). -* [NET](http://lxc.sourceforge.net/network.php): This namespace - isolates the processes from the network interfaces, defaulting to - having access only to the loopback interface. - -In addition to namespacing and cgroups, Linux supports [parceling superuser -privileges using -capabilities](http://www.ibm.com/developerworks/aix/library/l-posixcap.html). -Privileges that were once limited to uid=0 are now available in a coarse-grained -fashion using runtime and file system (extended attribute)-based labeling. In -addition, process tree capabilities inheritance is possible with a [lightweight -kernel patch](http://marc.info/?l=linux-kernel&m=125026482525494&w=2). With file -system capabilities enabled, specific process trees can also disable uid=0 from -having any default privilege (other than that granted by file system -permissions) using the securebits SECURE_NOROOT and SECURE_NO_SETUID_FIXUP. All -processes in the subtree could then be locked into this pure capability-based -superuser privilege mode barring a kernel privilege escalation vulnerability. -All capabilities are broken into effective, permitted, and inherited sets, which -can be applied to a file or process. In addition, processes all have a starting -bounding set that places an upper limit on which capabilities can be used, even -if in one of the other sets. At present, we're not aware of any Linux -distributions that make heavy use of capabilities. - -[Linux Security Modules](http://lwn.net/Articles/154277/) (LSM) is a subsystem -of the Linux Kernel Modules that implements a number of kernel task-based hooks. -It allows for a security module to be implemented that enforces mandatory access -controls and can be stacked (if supported by the module) with other security -modules. [Tomoyo](http://tomoyo.sourceforge.jp/) 2.x, -[SELinux](http://www.nsa.gov/research/selinux/index.shtml), and -[SMACK](http://schaufler-ca.com/) are examples of these systems. - -[grsecurity](http://grsecurity.net/) is a standalone kernel patch that provides -role-based access controls, kernel hardening, and bug fixes, as well as -extensive detection functionality. - -The kernel supports a notification system to inform userland of process events: -fork, id changes, so on. The [process events](http://lwn.net/Articles/157150/) -subsystem allows for low-cost monitoring of task creation and removal as well as -other pertinent runtime information. Communication is handled over a netlink -connection provided by the CONFIG_CONNECTOR option. - -## Detailed design - -This project will be undertaken iteratively to increase both the amount of -process isolation and overall system security as the implementation proceeds. -To reduce the surface area exposed to attack, we aggressively isolate processes -to access only what they require to function. Applying the [principle of least -privilege](http://web.mit.edu/Saltzer/www/publications/protection/) reduces the -total amount of code exposed at any one point and, ideally, minimizes the risk -that the exposed code contains a vulnerability. -Since it's not practically possible to isolate any software so completely as to -guarantee that vulnerable code isn't exposed, we use runtime and compile-time -exploit mitigations to increase the difficulty of a successful attack. In -addition, the mitigations used should make viable exploits behave unreliably -across Chromium OS systems. -Again, we assume that a *perfect* defense is unattainable, but providing a very -good defense will be on-going, iterative work which we all must contribute to. -To that end, we want to make successful exploitation less beneficial for the -attacker. For successful user-level exploits, this is done using the same -isolation techniques used to minimize exposed kernel attack surfaces. Successful -kernel-level exploits may be partially contained through some kernel hardening -patches, but more extensive fault isolation approaches have not yet been -explored. - -### Containment options - -Two main approaches will be supported for containment: - -* minijail: a tiny, custom launcher that handles namespacing, control - groups, chroot'ing, and more. -* A mandatory access control mechanism with automated learning mode, - like Tomoyo or grsecurity. - -#### minijail - -*minijail* is a small executable that can be used by root and non-root users to -perform a range of behaviors when launching a new executable: - -* Dropping capabilities from the bounding set -* Enable/disable capabilities -* chroot -* Dropping root (uid+gid) -* Setting securebits (SECURE_NOROOT, SET_DUMPABLE) -* Setting rlimits -* Process namespacing: - * Will act as init for any pid namespaces - * Will mount /proc in any child vfs namespaces - * Will support uts namespacing (not required) - * Will support user namespacing (in the future) - * Will support IPC namespacing (binary decision) - * Will support net namespacing (for locking down network - interfaces w/veth) -* A generic [suid - sandbox](http://src.chromium.org/viewvc/chrome/trunk/src/sandbox/linux/suid/sandbox.c?revision=25019&view=markup) - (using the features above) - -The final implementation should include a standalone library, libminijail, and a -command-line tool that supports individual feature specification or the loading -of specific, preconfigured profiles. - -Regardless of the implementation, the binary will need the cap_setpcaps extended -attribute set (+ep) to be able to operate without root privileges. (If we move -to a root file system that doesn't support extended attributes, any specialized -binaries can be mounted off of an ext3 loopback, or the kernel-based inheritance -patch can be used.) - -#### Mandatory access controls (MAC) - -If we are able to use kernel hardening patches (discussed later), we should -seriously consider using grsecurity. If not, we will use the in-kernel solution -Tomoyo. grsecurity supplies an effective automatic rule generator and includes a -large number of kernel hardening features and bug fixes that will otherwise be -missed. -Tomoyo will provide mandatory access controls to ensure that processes don't -exceed their expected boundaries. Initially, the coverage will be largely -limited to additional file system enforcements. Protection around ptrace, ioctl, -and other areas are needed before this will be considered a good solution. -Whichever MAC solution we choose will be configured to run in enforcement mode -for all processes, except when running in development mode. In that case, the -development mode parent process runs in permissive mode to allow the developer -to take whatever actions he wishes. - -### Deployment - -System hardening will proceed in roughly three phases. The first phase applies -comprehensive userland isolation using the two main containment options. - -### Phase 1: Surface changes - -Phase 1 focuses largely on putting existing services (and user logins) in to -SECURE_NOROOT+namespaced jails with as little system impact as possible. This -can be done by tweaking initialization scripts and modifying file extended -attributes. In particular, this will be done in our custom upstart configuration -files and other boot scripts. - -#### Nobody puts user in a corner: Or, taking root from the user. - -It turns out that this is pretty easy as long as we don't focus on making all of -Xorg non-root-based. For example, there is a command run by the login manager -upon successful user authentication: - -/bin/bash --login /etc/X11/Xsessionrc %session - -which we could change to - -/sbin/capsh --secbits=0x2f --drop=\[all\] -- --login /etc/X11/Xsessionrc -%session - -in order to run the user's session in SECURE_NOROOT, if we were using the capsh -tool (capabilities-aware shell). - -However, since we have minijail, we can use it to do more than just run in -SECURE_NOROOT: - -/sbin/minijail --init --namespace=pid,vfs --cgroup=chromeuser --secbits=0x2f ---drop=\[all\] --exec=/bin/bash -- --login /etc/Xsessionrc %s - -This will dump the new process in its own namespace where minijail acts as pid -1. The entire X session will only be able to see a /proc that is related to its -pid namespace, and it will have access only to devices enabled for the cgroup: -chromeuser. No SUID binaries or binaries with any additional extended capability -attributes set will be executable in this session. - -The biggest impact is that when it comes time to perform screen unlocking, the -xscreensaver process will not be able to do anything privileged. - -**Note:** We have not chroot'd or net-namespaced the binaries. At present, /dev/ -is limited by cgroup filtering (discussed below) and by mounting a fresh /proc -with the namespace view. While we will have stripped power from any simple root -privilege escalation attack, a user running as uid=0 will still have normal -discretionary access controls to a crazy number of files and devices as the root -user. In Phase 2, we will look at further segmenting access. - -Big Note: Any privileged actions must be brokered by preconfigured, preexecuted -binaries. - -With this Big Note in mind, we can look back at our slim.conf. If we find that -we need to launch some processes with some privileges, we can tone down how -aggressive the first call to minijail is. It can set up the namespace and lock -down root, but it can leave the bounding set a bit wider; that way, we can -launch utilities that may need capabilities like pulseaudio. This can be done -inside the Xsessionrc by calling minijail on all subsequent binaries with -specific bounding set changes, etc. They can all, thankfully, live in their new -pid namespace unless we lock them down further (chroot, etc). Initially, we'll -start with the above configuration and tweak as problems are introduced. - -Guiding resource utilization with control groups - -Control groups (cgroups) will be used to segment the population with respect to -device access and resource utilization. To that end, we can preconfigure a few -control groups at start via a simple /etc/init.d/cgroups script: -mkdir /cgroup -chown root:root /cgroup -chmod 700 /cgroup -mount none /cgroups -t cgroup -o cpu,memory,dev -mkdir /cgroups/services/network -mkdir /cgroups/services/autoupdate -mkdir /cgroups/user/chrome -mkdir /cgroups/user/chrome/sandbox -mkdir /cgroups/user/chrome/plugins/flash -mkdir /cgroups/user/xorg -With that done, we can leave it to minijail invocations to add the pid to the -/cgroups/<cgroup>/task file. The biggest challenge will be nested cgroups -like chrome/sandbox, since we will not mount /cgroups in the chrome user -namespace and users will be unable to see the /cgroups file system. In Phase 1, -we'll just have to let renderers live in the chrome cgroup and hope for the -best. Segmenting chrome user processes from the system services should be enough -to guarantee that a Chromium browser CPU DoS won't peg the system too badly, but -we'll see. If it can tie up xorg, the user experience will be the same. However, -in Phase 2, we will introduce a cgroupsd daemon. This daemon will monitor new -process creation (via a TBD mechanism with _low_ power/cpu needs) and -automatically add them to the appropriate cgroup. - -Devices will be added quite simply with: - -echo 'c 1:3 mr' > /cgroups/1/devices.allow - -Memory per group can be determined based on system memory. Below, limit chrome -to using 80 percent of available memory: -total_mem=$(free -b | grep Mem | tr -s ' ' | cut -f2 -d' ') -echo $((total_mem / 5 \* 4)) > /cgroups/user/chrome/memory.limit_in_bytes -CPU usage can also be determined using the system total, which is available in -cpu.shares. Below, we give all chrome processes 80 percent of the CPU shares: -total_cpu=$(cat /cgroups/cpu.shares) -echo $((total_cpu / 5 \* 4)) > /cgroups/user/chrome/cpu.shares -Of course, we can tweak the total number of shares to make specific allocations. -The allocations should then be used for fair scheduling. -Longer term, we may be able to use 'freezer' support to freeze all processes -prior to suspend or use cpusets to ensure that the Chromium browser, or perhaps -even an [extension](http://code.google.com/chrome/extensions/), is privately -allocated an entire CPU core (using *cpusets*). In addition, if any of these -items imply too much overhead, it is possible to achieve similar (and even more -focused) results using RLIMITS. - -#### Locking down existing daemons with extended attributes and a bit of luck - -#### At present, there are a number of processes running as root: - -* SLiM and X: both run with privilege. SLiM starts Xorg, and Xorg - needs ioctl and ioperm access, which equates to root access in most - cases. We will explore a non-root Xorg in Phase 2. -* connman will need CAP_NET_ADMIN, CAP_NET_RAWIO at most in its - bounding set so that they may be used by properly annotated files: - ping, dhclient, wpa_supplicant. -* dhclient is used for acquiring a network address and configuring an - existing network device. It needs to broadcast UDP and change - network device parameters. -* wpa_supplicant is used for configuring the wireless device to - properly associate with wireless access points. -* acpid handles power management and other events: lid close, low - battery, etc. -* getty handles the standalone console, which will most likely be - disabled in nondeveloper installs. -* udev handles firmware loading and hot-pluggable device. - -The final goal is to move to a pure capability-based system which means that no -service should *need* root access. To this end, we'll need to modify the startup -process for these daemons. The easiest approach is to just wrap their -start-stop-daemon calls with calls to minijail, either in the control panel or -in /etc/init.d. Each one should get its own namespacing with chroot'ing if -possible. (If it seems difficult to chroot a specific binary, then we should -consider doing so with the Chromium OS project's -[LinuxSUIDSandbox](http://code.google.com/p/chromium/wiki/LinuxSUIDSandbox).) -Capabilities required will be determined using *strace | grep 'EPERM|EACCESS'* -while locking down the binaries. Each binary will have the capabilities it needs -added to its extended attributes. For example, dhclient needs -CAP_NET_BIND_SERVICE|CAP_NET_BROADCAST|CAP_NET_ADMIN: - -capset cap_net_bind_service,cap_net_broadcast,cap_net_admin=ep /sbin/dhclient3 - -Then, dhclient will be called with minijail dropping all capabilities except -those three (and cap_setcaps if needed). If dhclient is called from connman, -then that process can already be running with a restricted capability bounding -set. - -At present, our root file system supports extended attributes. If we move to -squashfs or another file system without xattr**, we will have to work around the -restriction. This can be done using the patch referenced in the Technology -section or by mounting a loopback file system with the desired binaries with the -appropriate xattrs.** - -#### Supporting development mode before we finish designing it - -Once we dump the user in SECURE_NOROOT land, sudo and su become useless. To -allow continued tinkering, we can use the secondary console if enabled or a -loopback ssh daemon. As long as we don't lock these alternative entry points the -same way as the primary user session, they will be perfectly sane ways to -implement a secure but useful development mode. - -#### Adding fine-grained controls over coarse-grained capabilities with mandatory access controls - -For our Mandatory Access Control (MAC) needs, we are considering the external -grsecurity kernel patch as well as the currently in-kernel Tomoyo module and -accompanying tomoyo-ccstools package. In either case, a similar approach -applies. Once installed, an initial policy can be configured using learning -mode. We can then enable enforcement on process trees which shouldn't change: -dhclient, wpa_supplicant, etc. The Chromium browser itself will likely run in a -permissive mode as we explore extensions and other changes. However, as we -approach releases, we can configure a final policy using learning mode both with -active users and through automated testing that exercises all the expected use -cases for the system. - -The Tomoyo tool for editing the policy is ccs-editpolicy. Initially, there -should be no need to perform any special customization other than converting -process trees from disabled to learning or permissive. In addition, we will need -to make sure that the development mode runs in permissive or disabled mode. - -#### Locking down the file system (discretionary access controls) - -Discretionary access controls have been satisfactory at implementing basic -security in Linux for years. An audit of the root file system will ensure that -no end user can read files or directories he has no need to read, execute files -he has no need to access, or chdir to directories he has no need to enter. This -effort will be further expanded by the Phase 2 efforts around changing file -ownership to limit root's discretionary access—even when its capabilities have -been revoked. -In addition to the normal file system, /dev and /proc can be dangerous for an -unprivileged root user. While we are filtering devices per-cgroup, we should -ensure that CONFIG_STRICT_DEVMEM is set to limit /dev/mem usefulness. If we -patch Xorg, we may be able to get rid of /dev/mem entirely. In addition, a -review of what is available in /proc and /dev for each group will be crucial. -Whenever we place a process tree in a new VFS, we can mount --bind in only the -files we want. This is harder with /proc, but doable if needed. /proc may be -remounted read-only at the very least. We may be able to make use of the Linux -VServer's 'setattr' tool to hide /proc entries on namespace mount. If so, this -would be done in the minijail code but would require that we support the vserver -kernel patches. However, namespacing and chroot'ing will hopefully cover a lot -of ground. -User home directories and the /home partition should be mounted nosuid, nodev, -and ideally, noexec. We should attempt to limit user access to included -scripting engines if possible to to aid in enforcing noexec (dash, bash, or any -others). - -#### Deploying the firewall - -The netfilter/iptables infrastructure provides a number of interesting -approaches for limiting network access, both inbound and outbound. While a basic -inbound-only policy will work for general TCP and UDP level attack protection, -it would be nice to limit the OUTPUT chain as well. We can also consider using -network namespaces and VETH devices, but for Phase 1, the added complexity and -potential robustness issues make it questionable (see Phase 2 for more detail). - -#### Xorg without the root user - -On a testing system, we have restricted Xorg to CAP_SYS_RAWIO and seen -everything work except ioctl() access to the graphics device. We believe there -to be patches that deal with this, but getting them working within our codebase -may require some work. In addition, we can't just drop privilege because that -will make returning from suspend quite painful. - -#### SLiM and pam_google with limited capabilities - -Any privileged behavior needed by pam_google will need to be moved into a -standalone daemon. For instance, any encrypted volume management will need to be -shifted into a daemon that handles it for the user. A simple daemon that checks -SO_PEERCRED and the current mount state would do the trick. - -### Phase 2: Diving deeper - -Phase 2 is where we start making changes that are farther reaching. - -#### cgroupsd - -cgroupsd is a simple daemon that will automagically add new processes to a -control group specified in a libcg-style configuration file. The only useful -design point is that it will do so by using CONFIG_CONNECTOR and -CONFIG_PROC_EVENTS to be notified by the kernel of new processes via netlink. It -will need access to the /cgroups mountpoint, but otherwise, will not need any -additional privileges. In particular, if cgroupsd is used *only* for Chromium -browser sandbox processes, it can run with privileges to modify only -/cgroups/user/chrome/sandbox/tasks. -cgroupsd will also be used to enforce device filtering and resource management -on plugins like Flash. - -#### Other local system management daemons (network, sound, removable storage, ?) - -The further we lock down the user's session, the more work we'll have to put in -to brokering access to system resources. The control panel approach is great for -system management brokering (via a network loopback), but we will need to make -sure that udev and hal access can be handled safely. -We're probably also going to see some pain with Adobe Flash and other binary -plugins unless we give them access. Reviewing and integrating plugins with this -design will be critical to avoiding introducing a trivial backdoor through the -protections. - -#### Put browser instances in their own namespaces - -Chromium browsers can make use of the measures deployed in Phase 1. If the -existing sandbox isn't isolating rendered processes in their own pid namespace, -then it should be done here. In addition, every browser process itself can be -dropped into its own namespace when launched. - -#### Applying net namespaces - -One possible approach to isolating processes further is putting them all in -their own [net namespace](http://lxc.sourceforge.net/network/configuration.php). -We can then expose to the system a virtual interface with a virtual, internal, -IP address. We could even optionally enforce userland proxy use (for truly dodgy -inmates). However, this may introduce robustness issues if a user is assigned a -physical address that is the same or in the same netmask as the virtual ones. -Given that we can't control the eth0 address, we have delayed pursuing this -until Phase 2. When we get there, it will be worth investigating and deploying -if possible to keep any process from being able to bind to an external port. - -#### Re-chowning the file system, or why root shouldn't own *everything*. - -Even in a SECURE_NOROOT environment, root still has discretionary access control -to a large number of files and devices, and that access may be used to escalate -privileges or make system changes. One possible approach is to create multiple -system accounts that are responsible for files along some logical domain. One -option would be to use a var user and a home user and a bin user for each of the -areas they may own. A privileged user can always override such discretionary -access control mechanisms, but it will stop any accidental root access from -being completely detrimental. -That change may be overkill or add more complexity than the gain. Another option -would be to add a new secure bit along the lines of SECURE_UNSAFE. If that -secure bit is set on a tree, then no process in the tree can change to UID/GID -0. - -#### Device interposition - -Given that Chromium browsers and plugins need access to the webcam and audio (in -and out), it's important to isolate the kernel device drivers from software that -may be attacker controlled. To this end, access to /dev/video0 will be brokered -via a userland daemon. The work may be based on -[GSTFakeVideo](http://code.google.com/p/gstfakevideo/). Not only will this avoid -direct attacks on random webcam drivers, it will also mean we can later offer an -interface for doing real-time video stream filtering: custom effects, etc. -In addition to /dev/video, we'll want to position userland code for audio -interception (e.g, /dev/dsp, etc). This can be done using something like esound -or pulseaudio. If we go with one of those daemons anyway, we can get this for -free. -After the audio/video experience, we're left with one major exposed surface for -plugins which require video card device access. Since we will want to support -accelerated 3D and other fast rendering, we'll be exposing (possibly -binary-only) video card drivers via X/DRI. This is a larger problem that will be -addressed in a more detailed design document on the issue. - -#### Monitoring - -If we can monitor our system for any clear signs of compromise without seriously -affecting battery life or performance, then we should! This area should be -branched out into a full standalone design document. But within the scope of -this document, it is worth considering a very simple system. -A single daemon can monitor process creation and uid changes via the proc events -kernel interface. If it sees any process become uid/euid==0, then we have -someone running a privilege escalation exploit. If we determine an -exceptional-event user interface or a reboot path that will notify the user to -put-a-paperclip-in to reset the device, then we can trigger it immediately. -While an exploit can target this behavior, it is just one more layer of defense. -The Linux Auditing Framework may be very useful for doing detection, but its -cost may outweigh the benefit. Since we are not expecting a huge number of -process creation events, we can monitor system calls ranging from ptrace to -clone(2) to fork. If we avoid high traffic system calls, we should be able to -enforce some basic system call detection without sandboxing explicitly. In -addition, if we don't use auditd, but instead a [custom -listener](http://people.redhat.com/sgrubb/audit/audit-rt-events.txt), we can -immediately react to an event—such as terminating the calling process or -triggering a reboot into the recovery system. - -### Phase 3: Don't forget your snorkel! - -Phase 3 is where we get to explore additional innovations in the security space -that will require the most long-term investment. We're excited about the -possibilities around integrating new kernel and user space hardening techniques -and figuring out how to properly isolate drivers in kernel-space. -Here are some of the ideas we have, but there is a lot of area to research: - -* Retrofit /sbin/init and remove root everywhere. -* Isolate all running Xorg windows in Xephyr transparently to mitigate - keystroke theft, etc. (is the benefit worth the extra code?) -* Custom Linux Security Module for doing nested runtime sandboxes. -* Device driver security: We need to analyze device drivers and - determine a plan for isolation and or robustness. - * Using [KLEE](http://hci.stanford.edu/cstr/reports/2008-03.pdf) - or other automated dynamic analysis suite - * Consider vt-d/trustzone approaches for driver isolation - (l4linux, 64-bit friendly KERNSEAL/KERNEXEC?, Nooks) -* Harden the kernel heap management. -* Chromium-browser-based isolation of user data and processes by site - domain. -* Chromium browser http and https network stack isolation (as - processes) to protect secure cookies from evil sites accessed over - HTTP. -* Further harden userland/kernel interfaces. - -### Designing and developing for security - -Future software written for and included in Chromium OS-based devices must not -work around or otherwise undermine any of the features implemented to ensure -system security. In addition, it is important that software used in Chromium OS -receive sufficient peer code reviews, manual and automated security testing, as -well as security code audits. Security testing and code review processes should -be discussed in more detail in another document. Obviously, we feel that all -Chromium OS devices should only run software that follows these guidelines, but -we can only ensure that this is so for our official builds. - -### Exploit mitigation - -In addition to the containment plan, there are steps we can take to hardening -our user and kernel toolchains and harden the kernel itself. Toolchain hardening -and kernel patching should be performed opportunistically and are not tied to -the phased implementation laid out above. - -#### Kernel toolchain and patches - -There are a huge number of potential changes to the kernel we can pursue. We'll -start with known approaches and then expand as permitted into newer areas as -possible: - -* [PaX](http://pax.grsecurity.net/docs/) provides a number of useful - exploit mitigation techniques as a standalone patch to the Linux - kernel. grsecurity and PaX are available in one bundled patch. If at - all possible, PaX and grsecurity should be applied to the kernel. -* Ensure that we don't optimize out NULL pointer checks as mmap(0) - tricks are the current trend in kernel exploitation. - -fno-delete-null-pointer-checks -* The kernel should be compiled so that it is relocatable: - CONFIG_RELOCATABLE=y -* Compile out the vdso mapping unless we are really using something - that needs it: - COMPAT_VDSO=n -* A minimum mmap address must be enforced (> NULL): - CONFIG_SECURITY_DEFAULT_MMAP_MIN_ADDR -* The kernel should be compiled with stack smashing detection support: - CONFIG_CC_STACKPROTECTOR=y CONFIG_CC_STACKPROTECTOR_ALL=y -* System call patching: we should patch out, or severely restrict, - system calls that are new or potentially risky (e.g., ptrace, - vmsplice). -* Rebooting on soft panic: Currently, the kernel will try to continue - after a soft panic. To avoid a soft panic being caused by a - less-than-robust compromise, it'd be better to just reboot—which - leads into the next point. -* [kexec-on-panic](http://lwn.net/Articles/108596/): While - [kdump](http://lxr.linux.no/#linux+v2.6.30/Documentation/kdump/kdump.txt) - is meant to aid in enterprise-grade kernel debugging, we can - automatically boot over to a standby kernel if something goes wrong - with our primary kernel. This means that we will gain the long term - option of uploading kernel crash dumps, but at the very least, we - could boot a kernel that displays "Awww, snap!" and safely - reboots—or calls the autoupdater. While an attacker could target - modifying the kexec backup kernel, it seems an unlikely target if - the attacker can modify the running kernel (but we expect someone - will do it for fun). Note, this is not an option for ARM. -* Additional kernel configuration audit: remove all unneeded features. -* Disable automodule loading after the system is brought up. One - approach that was suggested is running sysctl -q - kernel.modprobe="/usr/bin/logger" and then rmmod -a after the system - is up if we can't move to a module-less kernel build. -* Enforce that system calls can't be called directly from writable - memory segments, like the heap. - -#### Userland toolchain - -The userland toolchain should enforce a number of requirements (which will need -benchmarking): - -* -fno-delete-null-pointer-checks -* -fstack-protector: stack canary -* -pie: relocatable executables (and marked ET_DYN) -* -Wl,-z,relro: read-only relocations -* FORTIFY_SOURCE - -Longer term, we'll also look at: - -* Additional glibc heap hardening -* Extending stack protector to obscure return addresses on the stack - -### Quantifying effectiveness - -In order to quantify effectiveness, we should take the time to truly enumerate -the attack surfaces yielded by these hardening steps: - -* Available files -* Files with privileges (capabilities, suid, /proc, /dev, /sys) -* System calls -* User id separation - -In addition to enumeration of attack surfaces, we should also implement -proof-of-concept attacks for each layer of our defenses by placing vulnerable -code at various points and directly testing the security properties at that -point. To supplement explicit tests, we will also pull in new exploits for newly -vulnerable code and evaluated if it is successful, how reliable it is, and what -we can do to protect against something like it in the future. - -## Plan - -The phases as laid out will be followed with the end goal of having Phases 1 and -2 completed fully in a few months -- including toolchain hardening. -Attack-oriented testing may be performed as part of the functional testing -during deployment, but if not, it should be completed immediately after phase 2 -along with a full system review. In addition to retrospective analysis of the -security, as new attacks emerge for Linux and Chromium OS, we will need to -evaluate their behavior and determine how to continue to futureproof the system -from a security perspective!
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/system-notifications/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/system-notifications/index.md deleted file mode 100644 index 93aeae11896..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/system-notifications/index.md +++ /dev/null @@ -1,344 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: system-notifications -title: system notifications ---- - -## Summary - -ChromeOS has several types of system status and some of them should be notified -to the user. This document summarizes which types of notifications are currently -used and what kind of UI flow is necessary here. - -The notifications are usually generated through the internal C++ API for the -[desktop -notifications](/developers/design-documents/extensions/proposed-changes/apis-under-development/desktop-notification-api), -which will appear as the [rich -notification](http://blog.chromium.org/2013/05/rich-notifications-in-chrome.html) -as well. - -## Blocking and system notifications - -Sometimes normal notifications should be blocked due to the current system -status. For example, all notifications should be queued during the lock screen, -and they should appear when unlocked. It also should be blocked when fullscreen -state; notifications should be annoying when the user watches YouTube videos, or -has a presentation (note: the notifications **should** appear in [immersive -fullscreen](/developers/design-documents/immersive-fullscreen) mode instead, -because the shelf / message center UI is visible in such case). - -However, the system status could be so important that they should appear in such -cases. This does not mean all system notifications should appear; some could be -misleading. For example, the notification for taking screenshot would behave as -same as the normal notifications, so it should not appear in the lock screen. In -addition, screenshot notification has the message to 'click to view' but click -should not work in lock screen. On the other hand, connecting display would be -okay to be notified even in the lock screen, it just notifies the new system -status. - -See also the [notification blocking code -update](https://docs.google.com/document/d/1Ox0Gb659lE2eusk-Gwm-a_JkARva7LydQc3hZNJvDn0/edit?usp=sharing) -design doc. - -## List of the system notifications - -<table> -<tr> -<td>message center?</td> -<td>Component</td> -<td>Source</td> -<td>Message</td> -<td>Show Always? ("System")</td> -<td>Timeout</td> -<td>Secure? (Show on Lock Screen)</td> -<td>Customize?</td> -<td>(can be disabled)</td> -<td>Click Action</td> -<td>Button</td> -<td>Triggers</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>DisplayErrorObserver</td> -<td>Could not mirror displays...</td> -<td>No</td> -<td>Yes</td> -<td>Yes</td> -<td>No</td> -<td>None</td> -<td>display connection</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>DisplayErrorObserver</td> -<td>That monitor is not supported</td> -<td>No</td> -<td>Yes</td> -<td>Yes</td> -<td>No</td> -<td>None</td> -<td>display connection</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>ResolutionNotificationController</td> -<td>..resolution was changed to...</td> -<td>Yes</td> -<td>Both</td> -<td>No</td> -<td>No</td> -<td>Accept / Revert change</td> -<td>Accept / Revert change</td> -<td>settings change from chrome://settings/display</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>ScreenCaptureTrayItem</td> -<td>... is sharing your screen</td> -<td>Yes</td> -<td>No</td> -<td>No</td> -<td>None</td> -<td>Stop Capture</td> -<td>apps</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>ScreenShareTrayItem</td> -<td>Sharing control of your screen...</td> -<td>Yes</td> -<td>No</td> -<td>No</td> -<td>None</td> -<td>Stop Share</td> -<td>apps</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>ScreenshotTaker</td> -<td>Screenshot taken</td> -<td>No</td> -<td>Yes</td> -<td>No</td> -<td>Yes</td> -<td>Show FIles App</td> -<td>keyboard shortcut</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>TrayDisplay</td> -<td>Mirroring to...</td> -<td>No</td> -<td>Yes</td> -<td>Yes</td> -<td>No</td> -<td>Show Display Settings</td> -<td>settings change from chrome://settings/display, and keyboard shortcut</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>TrayDisplay</td> -<td>Extending screen to...</td> -<td>No</td> -<td>Yes</td> -<td>Yes</td> -<td>No</td> -<td>Show Display Settings</td> -<td>settings change from chrome://settings/display, and keyboard shortcut</td> -</tr> -<tr> -<td>Yes</td> -<td>display</td> -<td>TrayDisplay</td> -<td>Docked mode</td> -<td>No</td> -<td>Yes</td> -<td>Yes</td> -<td>No</td> -<td>Show Display Settings</td> -<td>close lid</td> -</tr> -<tr> -<td>Yes</td> -<td>language</td> -<td>LocaleNotificationController</td> -<td>The language has changed from...</td> -<td>No</td> -<td>No</td> -<td>No</td> -<td>No</td> -<td>Accept change</td> -<td>Revert change</td> -<td>login</td> -</tr> -<tr> -<td>No</td> -<td>language</td> -<td>TrayAccessability</td> -<td>Spoken feedback is enabled.</td> -<td>\*</td> -<td>Yes</td> -<td>No</td> -<td>None</td> -<td>keyboard shortcut</td> -</tr> -<tr> -<td>No</td> -<td>language</td> -<td>TrayCapsLock</td> -<td>CAPS LOCK is on</td> -<td>\*</td> -<td>Yes</td> -<td>No</td> -<td>None</td> -<td>keyboard shortcut</td> -</tr> -<tr> -<td>Yes</td> -<td>language</td> -<td>TrayIME</td> -<td>Your input method has changed...</td> -<td>No</td> -<td>Yes</td> -<td>No</td> -<td>No</td> -<td>Show IME detailed view</td> -<td>keyboard shortcut</td> -</tr> -<tr> -<td>Yes</td> -<td>network</td> -<td>DataPromoNotification</td> -<td>Google Chrome will use mobile data...</td> -<td>Yes</td> -<td>No</td> -<td>?</td> -<td>No</td> -<td>Promo URL or settings</td> -</tr> -<tr> -<td>Yes</td> -<td>network</td> -<td>network_connect</td> -<td>Activation of... requires a network connection</td> -<td>Yes</td> -<td>No</td> -<td>?</td> -<td>No</td> -<td>Show Settings</td> -</tr> -<tr> -<td>Yes</td> -<td>network</td> -<td>NetworkStateNotifier</td> -<td>...Your .. service has been activated</td> -<td>Yes</td> -<td>No</td> -<td>?</td> -<td>No</td> -<td>Show Settings</td> -</tr> -<tr> -<td>Yes</td> -<td>network</td> -<td>NetworkStateNotifier</td> -<td>You may have used up your mobile data...</td> -<td>Yes</td> -<td>No</td> -<td>?</td> -<td>No</td> -<td>Configure Network</td> -</tr> -<tr> -<td>Yes</td> -<td>network</td> -<td>NetworkStateNotifier</td> -<td>Failed to connect to network...</td> -<td>Yes</td> -<td>No</td> -<td>?</td> -<td>No</td> -<td>Show Settings</td> -</tr> -<tr> -<td>No</td> -<td>network</td> -<td>TraySms</td> -<td>SMS from ...</td> -<td>\*</td> -<td>No</td> -<td>No</td> -<td>Show SMS detailed view</td> -</tr> -<tr> -<td>No</td> -<td>power</td> -<td>TrayPower</td> -<td>Battery full OR %<X> remaining</td> -<td>\*</td> -<td>No / Never</td> -<td>Yes</td> -<td>No</td> -<td>None</td> -<td>low battery</td> -</tr> -<tr> -<td>Yes</td> -<td>power</td> -<td>TrayPower</td> -<td>Your Chromebook may not charge...</td> -<td>No</td> -<td>Yes</td> -<td>Yes</td> -<td>No</td> -<td>None</td> -<td>usb charger connected</td> -</tr> -<tr> -<td>Yes</td> -<td>user</td> -<td>TrayLocallyManagedUser</td> -<td>This user is supervised by...</td> -<td>Yes</td> -<td>No</td> -<td>No</td> -<td>No</td> -<td>None</td> -<td>login</td> -</tr> -<tr> -<td>Yes</td> -<td>user</td> -<td>TraySessionLengthLimit</td> -<td>This session will end in...</td> -<td>Yes</td> -<td>No</td> -<td>No</td> -<td>No</td> -<td>None</td> -<td>timeout set by policy / sync / remaining time change</td> -</tr> -<tr> -<td>No</td> -<td>user</td> -<td>TrayUser</td> -<td>You can only have up to three accounts in multiple sign-in</td> -<td>\*</td> -<td>\*</td> -<td>Yes</td> -<td>No</td> -<td>None</td> -</tr> -</table>
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/tab-discarding-and-reloading/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/tab-discarding-and-reloading/index.md deleted file mode 100644 index 39492bed9d0..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/tab-discarding-and-reloading/index.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: tab-discarding-and-reloading -title: Tab Discarding and Reloading ---- - -**Why are my tabs reloading?** - -Your device is out of memory. Like your Android phone or tablet, Chrome is -silently closing background tabs in order to make memory available. When you -click on one of those tabs it reloads. - -**What's taking up all my memory?** - -Open the Task manager (under Tools > Tools > Task manager, or hit -Shift-Esc) to see the usage of tabs and extensions. about:memory has more -detailed information. - -**How can I make it stop?** - -Close some tabs or uninstall extensions that take a lot of memory. If there's a -specific tab you don't want discarded, right-click on the tab and pin it. - -**How does Chrome choose which tab to discard?** - -You can go to about:discards to see the current ranking of your tabs. It -discards in this order: - -1. Internal pages like new tab page, bookmarks, etc. -2. Tabs selected a long time ago -3. Tabs selected recently -4. Tabs playing audio -5. Apps running in a window -6. Pinned tabs -7. The selected tab - -**How often are users affected by this?** - -The majority of Chrome OS users have only one or two tabs open. Even users with -more tabs open rarely run out of memory. Some users who run out of memory never -see a reload (they log out without ever looking at a discarded tab). We have -internal UMA metrics on all these conditions. Googlers tend to have large -numbers of tabs open to complex web sites and tend to hit reloads more often. - -**Why doesn't Chrome do this on Mac / Windows / Goobuntu?** - -Those machines swap memory out to disk when they get low on resources. Changing -tabs then slows down as the data is loaded from disk. - -**Why doesn't Chrome OS use swap?** - -This was a very early design decision. My understanding of the rationale is: - -\* Chrome OS doesn't want to wear out the SSD / flash chips by constantly -writing to them. - -\* To preserve device security it would have to encrypt the swap, which -increases the CPU and battery usage. - -\* Rather than slowing down by swapping, Chrome OS should run fast until it hits -a wall, then discard data and keep running fast. - -**Why doesn't Chrome OS tell me that I'm out of memory?** - -This is an intentional UI design decision. My understanding of the rationale is: - -\* Phones and tablets silently discard pages - -\* Users shouldn't have to worry about managing memory - -**Why do we discard tabs instead of doing something else?** - -Well, we used to do something worse. When the machine ran out of memory the -kernel out-of-memory killer would kill a renderer, which would kill a -semi-random set of tabs. Sometimes this included the selected tab, so the user -would see a sad tab page (specifically, He's dead, Jim.). The tab discarder only -drops one tab at a time and tries to be smart about what it discards. We tried -other approaches without much success. For example, forcing JavaScript garbage -collection is too slow. Dropping various graphics caches only frees memory for a -short amount of time. Part of the difficulty is that the response to low memory -conditions needs to be fast, reliable and lead to a persistent decrease in -memory consumption. - -**Is there a long-term plan for this issue?** - -Sometimes these issues are caused by memory leaks or bloat, which we fix. -davemoore@ has been particularly good at finding Chrome OS memory problems. We -also need to systematically reduce the amount of memory Chrome uses. People -across the browser, renderer and graphics teams are working on this. We're also -investigating zram (a kind of in-memory swap). At some point we may want to -revisit the design decisions above. - -**Where can I find more detailed information about what we do in low memory -situations?** - -See the [out of memory design -document](/chromium-os/chromiumos-design-docs/out-of-memory-handling).
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/basic-architecture.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/basic-architecture.png.sha1 deleted file mode 100644 index 242b50a3ee2..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/basic-architecture.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -d7a7897f467fa7ff807807dc7d92635af71e02c5
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/basic-architecture2.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/basic-architecture2.png.sha1 deleted file mode 100644 index f9be281c46c..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/basic-architecture2.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -4b4a723573c8a7ffa926ef5738efd5a95e6f1bd1
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/candidate_window.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/candidate_window.png.sha1 deleted file mode 100644 index 41e171c6067..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/candidate_window.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -6756ba713d0295c060ef2856eff6086fd4164628
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/candidate_window_demo.swf.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/candidate_window_demo.swf.sha1 deleted file mode 100644 index 548699663b2..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/candidate_window_demo.swf.sha1 +++ /dev/null @@ -1 +0,0 @@ -7deaaf4d3583be6dbc4f8aaaba6bb6b4e97c537d
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/index.md deleted file mode 100644 index ea53b0d8af5..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/index.md +++ /dev/null @@ -1,275 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: text-input -title: Text Input ---- - -[TOC] - -## OBSOLETE: The contents need to be updated - -## Abstract - -* Chromium OS will be internationalized and will support many - languages other than English -* Chromium OS will support a variety of text input methods to make - text entry as easy as possible for non-English languages -* End user language and input-method settings are synchronized with - the user's profile in the cloud to enable a consistent user - experience from any device - -## Objective - -Build text input support for Chromium OS to make it well internationalized. - -## Background - -Chromium OS will be supporting multiple languages other than English. Regardless -of the UI language (the language shown in menu items and any other strings in -Chromium OS), Chromium OS should allow you to type in many languages. We will -build text input support for Chromium OS on top of the existing text input -technologies available for Linux. - -## Requirements - -Chromium OS will provide users several ways to type in text: - -* **Simple Keyboard:** Users type in characters directly from the - keyboard. Examples: English, German -* **Switchable Keyboard:** Users can switch keyboard mappings (for - example, Latin/Cyrillic) and directly type in different character - sets based on the current mapping. For example, Russian users can - switch the keyboard mapping between Cyrillic and Latin to compose - bilingual text. -* **Simple IME:** Users type in most characters directly using a - simple Input Method Editor (IME) that does some minor tasks such as - validation, auto-correction, and composition. Examples: Korean, Thai -* **Transliteration IME:** Users type in English characters that the - Transliteration IME directly converts to native characters. Example: - Indic -* **Candidate IME:** Users type in all characters indirectly using an - IME. For example, Japanese has too many characters to directly map - on a keyboard. Using a Japanese IME, users type in the "sound" (for - example, "nihon") to the IME, which provides a set of candidates - (for example, "日本") that the user selects from. - -There are also Chromium OS-specific requirements, such as supporting both x86 -and ARM processors and following Chromium OS UI design guidelines. Meeting all -of these requirements is not trivial. We'll reuse existing input technologies as -much as possible. - -We will support the following languages by any of the ways listed above. - -<table> -<tr> -<td>Arabic</td> -<td> Bulgarian</td> -<td> Catalan</td> -<td> Croatian</td> -<td> Czech</td> -<td> Danish</td> -<td> Dutch</td> -<td> English (GB)</td> -<td> English (US)</td> -<td>Estonian</td> -<td>Filipino</td> -<td>Finnish</td> -<td>French</td> -<td> German</td> -<td> Greek</td> -<td> Hebrew</td> -<td> Hindi</td> -<td> Hungarian</td> -<td> Indonesian</td> -<td> Italian</td> -<td> Japanese</td> -<td> Korean</td> -<td>Latvian</td> -<td> Lithuanian </td> -<td> Norwegian</td> -<td> Polish</td> -<td> Portuguese (Brazil)</td> -<td> Portuguese (Portugal)</td> -<td> Romanian</td> -<td> Russian</td> -<td> Serbian</td> -<td>Simplified Chinese</td> -<td>Slovak</td> -<td>Slovenian</td> -<td> Spanish</td> -<td>Spanish in Latin America</td> -<td> Swedish</td> -<td> Thai</td> -<td> Traditional Chinese</td> -<td> Turkish</td> -<td> Ukrainian</td> -<td> Vietnamese</td> -</tr> -</table> - -## Design - -### Basic architecture - -For languages that can be supported by Simple Keyboard or Switchable Keyboard, -we'll use [XKB](http://en.wikipedia.org/wiki/X_keyboard_extension) (X Keyboard -Extension). For languages in other categories, we'll use -[IBus](http://en.wikipedia.org/wiki/Intelligent_Input_Bus) (Intelligent Input -Bus). The following diagram illustrates the basic architecture of text input -support for Chromium OS. - -> [<img alt="image" -> src="/chromium-os/chromiumos-design-docs/text-input/basic-architecture2.png">](/chromium-os/chromiumos-design-docs/text-input/basic-architecture2.png) - -### Keyboard layout: XKB - -We will create the XKB definition file for each keyboard layout. - -### Input method framework: IBus - -We will use IBus as the input method framework (IM framework). IBus is an -open-source input method framework that works as a middle layer between -applications and language-specific backend conversion engines. - -We decided to use IBus for the following reasons. - -* It's the default IM framework for Fedora 11 and Ubuntu 9.10. -* No rebooting is necessary (unlike older IM frameworks) when major - configurations are changed, such as when adding a new language. -* Apps should be more stable. Unlike older IM frameworks, crashes in - conversion engines won't affect applications. -* IBus is implemented on top of D-Bus, which is also used in other - components of Chromium OS. - -### Conversion engines - -Many conversion engines are available for IBus, such as ibus-pinyin for Chinese. -We'll use the following open source conversion engines. - -<table> -<tr> -<td><b>IBus engine</b> </td> -<td> ibus-pinyin</td> -<td><b> ibus-chewing</b> </td> -<td><b> ibus-table</b></td> -<td><b> ibus-hangul</b></td> -<td><b>Language</b> </td> -<td> Chinese (Simplified)</td> -<td> Chinese (Traditional - Chewing)</td> -<td> Chinese (Traditional - Bopomofo and Cangjie)</td> -<td> Korean</td> -</tr> -</table> -New: For Japanese, we plan to port [Google Japanese -Input](http://www.google.com/intl/ja/ime/) to Chromium OS. - -### User experience - -We'll make the text input experience be uniform with the user experience in -Chromium OS. To achieve this goal, we'll rewrite all user interfaces using -Chromium's UI toolkit library called Views. - -**Candidate window** - -Will be implemented as separate process. Communicates to IBus-daemon via D-Bus. - -New: Here's a screenshot of a work-in-progress version of the candidate window. -[A demo](/system/errors/NodeNotFound) is also available. - -> [<img alt="image" -> src="/chromium-os/chromiumos-design-docs/text-input/candidate_window.png">](/chromium-os/chromiumos-design-docs/text-input/candidate_window.png) - -**XKB configuration dialog** - -Will be part of Chromium's options dialog. Communicates to X server via libcros. - -**IBus configuration dialog** - -Will be part of Chromium's options dialog. Communicates to IBus-daemon via -libcros. - -**Conversion engine specific configuration dialogs** - -Will be part of Chromium's options dialog. Communicates to conversion engines -via libcros. - -**Input language selector** - -Will be part of Chromium's status area (near the time and the power indicator). -Communicates to IBus-daemon and X Server via libcros. We'll integrate the input -language selector with the keyboard layout selector, unlike traditional Linux -desktops. Here's a mockup of what it will look like: - -> [<img alt="image" -> src="/chromium-os/chromiumos-design-docs/text-input/input.png">](/chromium-os/chromiumos-design-docs/text-input/input.png) - -**Input language selector (on login manager)** - -Probably not required, because Google accounts (that is, email addresses) used -for login are all in ASCII characters, as are the passwords. We will implement -it once it becomes necessary. Internationalized keyboards can be supported by -XKB configurations. - -**Sync with the cloud** - -Text input-related data is synced with the cloud. We sync the user's choices of -languages, keyboard layouts, and input method extensions as described in -[Syncing Languages and Input -Methods](/chromium-os/chromiumos-design-docs/text-input/syncing-input-methods). - -In the future, we may sync: - -* User settings for XKB, IBus, each IME, and so on. -* User input history used for better conversions (optional) -* User-defined dictionaries (optional) - -Syncing configuration data is a must, but users can opt out of syncing other -user data. - -**Security considerations** - -An indirect input layer like IBus can be a security weak point, as it can be -exploited as a key logger. We plan to use -[GRSecurity](http://en.wikipedia.org/wiki/Grsecurity) to restrict resource -access from IBus and conversion engines. We also plan to review the security of -IBus. - -**Long-term goals** - -Although the following goals aren't targeted for the initial version, these are -some of our long-term goals: - -* Better integration with browsers, such as changing behavior - depending on the input form. -* Multi-modal input, such as hand writing and voice recognition. -* Virtual keyboard (aka software keyboard). Web-based virtual keyboard - can be used in the initial version, though. -* Multi-keypress input - -## Code location - -### IBus - -* chromium-os/src/third_party/ibus/ (Tip of tree of IBus. We'll use - the ibus-daemon, ibus-pinyin, ibus-chewing, ibus-table, and - ibus-hangul modules as-is. However, we won't use the ibus-setup or - ibus-panel modules.) - -### Configuration dialog and language selector - -* [chromium/src/chrome/browser/chromeos/options](http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/options/) - (configuration page) -* [chromium-os/src/platform/cros](http://src.chromium.org/cgi-bin/gitweb.cgi?p=cros.git;a=summary) - (libcros part, replacement for ibus-setup and ibus-panel) - * [chromeos_ime.cc](http://src.chromium.org/cgi-bin/gitweb.cgi?p=cros.git;a=blob;f=chromeos_ime.cc;hb=HEAD) - - APIs used for implementing the candidate window - * [chromeos_language.cc](http://src.chromium.org/cgi-bin/gitweb.cgi?p=cros.git;a=blob;f=chromeos_ime.cc;hb=HEAD) - - APIs used for manipulating input language status. - -### Candidate window - -* [chromium/src/chrome/browser/chromeos/text_input](http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/chromeos/text_input/)
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/input.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/input.png.sha1 deleted file mode 100644 index 4431ceec8ff..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/input.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -a79a2fd183fd588c3b91f27d10ae995c85fd5d75
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/syncing-input-methods/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/syncing-input-methods/index.md deleted file mode 100644 index aca8b455a78..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/text-input/syncing-input-methods/index.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -- - /chromium-os/chromiumos-design-docs/text-input - - Text Input -page_name: syncing-input-methods -title: Syncing Languages and Input Methods ---- - -## Background - -In M41, Chromium OS [started syncing -preferences](https://codereview.chromium.org/312023002) for languages, keyboard -layouts, and extension IMEs. Because devices may have different physical -keyboard layouts, as well as different available locales and input methods, -changes on one device should not automatically propagate to another device. -However, we still wanted to provide a convenient out-of-box experience for users -who go beyond their device's default input method when they Powerwash their -system or sign in to a new device. - -## Overview - -Instead of syncing input methods both ways, we send changes to input methods -from the device to the sync server. This lets us "remember" the most recent -settings without overwriting existing preferences on other machines. When a user -signs in to a device for the first time, we combine the input method used to -sign in with the information from the sync server, adding the user's saved input -method preferences to the local preferences without overwriting any input -methods already set up. - -For instance, a user should only have to manually add the Dvorak layout to a -device once. Then, we automatically add this layout to new devices for this -user, without affecting devices that have already been set up. If the layout -isn't available on a device, it's simply not added. In the worst case, we may -add an input method that is incompatible with the hardware keyboard, but the -user would still retain any input methods that they enabled before the first -sync. - -## Design - -### Preferences - -We kept the original unsyncable preferences, which describe the local settings: - -* **settings.language.preferred_languages:** enabled language IDs - (e.g., "en-US,fr,ko") -* **settings.language.preload_engines:** preloaded (active) component - input method IDs (e.g., "pinyin,mozc") -* **settings.language.enabled_extension_imes:** enabled extension IME - IDs - -We added three syncable preferences: - -* **settings.language.preferred_languages_syncable** -* **settings.language.preload_engines_syncable** -* **settings.language.enabled_extension_imes_syncable** - -During the first sync, the values of these preferences are merged with the local -preferences above. After that, these preferences are only used to keep track of -the user's latest settings. They never affect the settings on a device that has -already synced. In other words, they can be thought of as one-way, syncing from -the device to the sync server -- except during the first sync. - -A fourth preference was added to indicate whether the initial merging should -happen: **settings.language.merge_input_methods**. This is set to true during -OOBE. After the first sync, this is always false. This ensures that devices set -up before M41 won't have their input settings changed. - -### First Sync - -[chromeos::input_method::InputMethodSyncer](https://cs.chromium.org/chromium/src/chrome/browser/chromeos/input_method/input_method_syncer.h) -was created to handle sync logic for input methods. After the first ever sync, -MergeSyncedPrefs adds the IME settings from the syncable prefs to the local -prefs using the following algorithm for each pref: - -1. Append unique tokens from the syncable pref to the local pref. -2. Set the syncable pref equal to the merged list.\* -3. Remove tokens corresponding to values not available on this system. -4. Set the local pref equal to this validated list. - -MergeSyncedPrefs also sets merge_input_methods to false, ensuring that it won't -be called again. - -Notice that this solution doesn't assume the first sync happens right after -OOBE. For cases where users have set encryption passphrases, they could easily -add languages and input methods before enabling sync, so we support arbitrary -lists instead of single values during the merge. - -### Updates - -When a local input method pref changes, InputMethodSyncer updates the syncable -prefs to correspond to the local pref values.\* We update all three prefs at the -same time to ensure the settings are consistent -- since these will be the -values used the next time the user completes OOBE, the languages should -correspond to the input methods. - -\* For preload_engines_syncable (input method IDs), we transform the list to use -legacy component IDs so the preference can sync between Chrome OS and Chromium -OS. During first sync, we convert this back to locally valid component IDs.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/touch-firmware-updater/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/touch-firmware-updater/index.md deleted file mode 100644 index 54a957a94ce..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/touch-firmware-updater/index.md +++ /dev/null @@ -1,157 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: touch-firmware-updater -title: Touch Firmware Updater ---- - -[TOC] - -## Introduction - -Touchpads and touchscreens are among the most sensitive input devices integrated -into computing systems today. The complexity of touch controllers has increased -so that touch devices can be thought of as small embedded devices, with many -features and performance determined by upgradeable firmware. Throughout -development and the lifespan of a Chromebook, firmware updates of the touch -device may occur. This page provides an overview of how Chrome OS handles touch -device firmware updates. - -## Bus overview - -Current generation Chromebooks use I2C (and the SMBus variant) as the underlying -communication bus between the host system (where Chrome OS and the kernel driver -run) and the touch device controller (where the firmware runs). I2C was chosen -because it provides a good balance between power draw, complexity, and -bandwidth. The bus is also well-supported on ARM based platforms. In addition to -I2C for data, an interrupt line is routed from the trackpad to the host to -signal the kernel driver when it is time to query for new data from the device -during operation (and optionally signal state transitions during a firmware -update). - -## Kernel driver - -Most of the actual work of moving of firmware bits across the bus is done by the -vendor-specific kernel driver or user-space utility. The driver or utility sends -commands to the device to ready it to accept a new firmware payload. Some -trackpad vendors refer to this mode as a “bootloader” mode. The bootloader is a -last resort in case the more complex operational mode firmware is corrupted. A -trackpad without a valid firmware needs to at least be able to accept a working -payload via an update. - -The driver or utility sends the firmware binary over the i2c bus to the device’s -bootloader, using the vendor-specific protocol. - -While the protocol from the kernel driver to the touch device for the firmware -update is vendor-specific, [Chrome OS requires that the kernel driver use the -request_firmware hotplug -interface](https://www.kernel.org/doc/Documentation/firmware_class/README) to -expose the same interface to userspace: - -At a high level, request_firmware allows the kernel driver to access a file for -read operations at /lib/firmware in the root file system. - -The device driver must support the following sysfs attributes for version -management: - -* **firmware_version** or **fw_version** - identifies the current - version number of the firmware loaded on the device, in the form - <*major_version*>.<*minor_version*> -* **product_id** or **hw_version** - a unique string that can identify - the hardware version in the case that the same driver may be used by - multiple device variants. - -For example, a Chromebook Pixel has two Atmel mXT devices. The hw_version of the -mXT224SL trackpad is 130.1. The hw_version of the mXT1664S touchscreen is 162.0. - -Finally, one sysfs attribute must be provided to trigger a firmware update -programmatically from user space: - -* **update_fw** - should be writable by the root user. It should also - be able to accept a different filename in case more than one device - uses the driver on the system, and therefore, two different - firmwares are meant for different targets. - -The following example shows how the two Pixel touch devices share the same -driver, but appear as separate instances in sysfs : - -echo “maxtouch-tp.fw” > /sys/bus/i2c/devices/1-004b/update_fw echo -“maxtouch-ts.fw” > /sys/bus/i2c/devices/2-004a/update_fw - -For examples, see the -[cypress](https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-3.4/drivers/input/mouse/cyapa.c) -and -[atmel](https://chromium.googlesource.com/chromiumos/third_party/kernel/+/refs/heads/chromeos-3.4/drivers/input/touchscreen/atmel_mxt_ts.c) -drivers in the chromeos-kernel tree. - -**Userspace updater** - -If the vendor uses I2C-HID kernel driver, they may use a user-space utility to -update the device. - -## Userspace organization and scripts - -The touch firmware updater consists of a set of scripts and firmware files. The -firmware update script is available in a [Chromium OS source -tree](http://git.chromium.org/gitweb/?p=chromiumos/platform/touch_updater.git;a=tree;f=scripts). - -The files used by the touch firmware updater (as seen on the filesystem of a -target system) are organized as follows: - -/opt/google/touch/scripts/chromeos-touch-firmware-update.sh -/opt/google/touch/firmware/CYTRA-116002-00_2.9.bin -/lib/firmware/cyapa.bin->/opt/google/touchpad/firmware/CYTRA-116002-00_2.9.bin - -Note that `/lib/firmware/cyapa.bin` is a symlink that links to the current -latest version of firmware elsewhere in the rootfs. - -On a production Chrome OS system, the rootfs is protected as a part of Chrome OS -verified boot, so neither the scripts nor the firmware binary can be changed by -the user. Each version of Chrome OS ships with a firmware binary for every touch -device in the system installed in the rootfs. Following an auto-update, on boot -the touch firmware update script probes the device's current firmware version -and may trigger a firmware update if the current version is not compatible. - -The name of the firmware binary is used to identify the version of the firmware -binary by the chromeos-touch-firmwareupdate.sh. It should be in the format : -<*product_id*>_<*firmware_version*>.bin - -In the above example, the product_id is CYTRA-116002-00, and the version is 2.9. - -## Firmware update sequence - -The touch firmware update runs after the kernel device driver has successfully -probed the touch device. - -The chromeos-touch-update upstart job runs before the Chrome OS UI has started. -It blocks the UI job from starting until a firmware version check and potential -update are completed. The UI job is blocked because the touchpad or touchscreen -are nonresponsive for the duration of the update, and during this process a user -should not be able to interact with the device. - -For details, see the -[chromeos-touch-update.conf](http://git.chromium.org/gitweb/?p=chromiumos/platform/touch_updater.git;a=blob;f=scripts/chromeos-touch-update.conf;hb=HEAD) -job. - -The updater also runs as a part of the recovery process from a recovery image -booted via USB. - -## Error recovery - -A typical update requires between 8 and 18 seconds. The firmware update runs -before the Chrome OS UI starts (the user sees the splash screen) to prevent the -user from inadvertently causing the update to fail. - -However, it is still possible that a firmware update might not complete properly -if the system shuts down due to low battery. In this case, the operational mode -firmware will be corrupted. The next time the device boots, it will not be -possible to probe the device and have a functional touch device. - -For error recovery, it is a requirement for the kernel device driver to -recognize this condition and probe successfully into a bootloader-only mode. -From this state, the driver must be able to perform a forced update to a known -good firmware. This means the update_fw sysfs property must exist and perform -the same task in this error condition as during a normal firmware update.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/upstream-first/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/upstream-first/index.md deleted file mode 100644 index 2f0a71f542b..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/upstream-first/index.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: upstream-first -title: Upstream First ---- - -## Abstract - -This document outlines a process for managing the flow of kernel source patches -between Chromium OS and hardware partners. - -## Goals - -Our goal with the "Upstream First" policy is to eliminate kernel version -fragmentation. - -We know that hardware partners support other devices as well as Chromium -OS-based devices. So rather than having partners target multiple kernels for -multiple devices, we'd like to encourage them to target one kernel: the upstream -kernel. - -By having everyone target a single kernel, we hope to avoid duplication of -effort and get everyone (our hardware partners, their partners, and the upstream -community) working together on improving a single set of patches. By working -together, we can build a better kernel and avoid later having to rework changes -that are incompatible with the upstream kernel. - -## Accepting patches - -A device driver patch must be accepted upstream before it can be accepted into -the Chromium OS kernel. Ideally, the patch will be in Linus Torvalds's kernel -tree before it is accepted into the Chromium OS kernel. However, other options -are also available. - -The following list shows options for getting a patch accepted into the Chromium -OS kernel. We encourage all partners to strive to attain option 1. - -**Patch options, ordered from high to low preference** - -1. The patch is accepted into Linus Torvalds's upstream tree. -2. The patch is accepted into the subsystem maintainer tree (such as - Dave Miller's netdev tree). -3. The patch is accepted into the the device maintainer tree (such as - samsung or msm) and a pull request has been sent to Linus Torvalds. - -After the patch is accepted into an upstream tree, Google will cherry-pick it -into the Chromium OS kernel. If the patch requires backport work, Google will -work with the vendor to do the backport. - -Exemptions are granted on a case-by-case basis.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/user-accounts-and-management/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/user-accounts-and-management/index.md deleted file mode 100644 index 2b2e0429723..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/user-accounts-and-management/index.md +++ /dev/null @@ -1,219 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: user-accounts-and-management -title: User Accounts and Management ---- - -[TOC] - -## Abstract - -* End users must already have, or be willing to create, an account - that is used to store preferences and settings. Along with Google - Accounts, we want to support other authentication systems, - particularly OpenID. -* Storing preferences and settings with the account in the cloud - allows end users to maintain a consistent user experience while - switching among Chromium OS devices. -* Chromium OS devices are designed to be easily shared among users, - while still providing a high level of security for the owner of the - device and other authorized users. -* Owners of Chromium OS devices can choose to manage a list of users - who are allowed to log in to a device, providing a higher level of - security for the device. -* Chromium OS also provides a Guest mode, allowing anyone to browse - with the device without preserving any data, preferences, or other - information after the session ends. The owner of the device can - choose to disable this mode. - -## Objective - -For V1, at least, we are asserting the following about logging in to Chromium OS -devices: - -1. We want the device to be as shareable as a Google Doc. -2. The first time a given user logs into the device, we require - connectivity for the initial authentication. -3. We believe users want to control who can log in to their machine. -4. We consider traditional user account management to be onerous. - -Our goal is to enable use cases like the following: -> I have a Chromium OS device that I use and have configured to work on my home -> WiFi network, which has some Network Attached Storage (a NAS). A friend asks -> if she can borrow my device to do some work. I log out and hand her the -> device. She logs in. The device should remain associated with my WiFi network, -> though her preferences (address bar state, installed apps, privacy settings, -> etc) should sync down and be applied for her account. As a byproduct, my -> friend has access to my NAS. That's okay. - -without also enabling attacks like this: -> I have a Chromium OS device that I use and have configured to work on my home -> WiFi network, which has a NAS. Someone steals my device. The thief has a -> Google Account. He logs in, and not only does he have a local account on my -> machine, which removes a significant obstacle to compromise, but he also has -> access to my NAS and everything else on my home network. - -At a high level, what we're saying is that, for certain settings, the right -thing to do is have a per-device **system setting** override a **user -preference**—but that some per-device settings could be sensitive. This insight -dictates several requirements: - -* For users who log in to the system, we must come up with a sane way - to overlay user preferences on system settings. -* There must be some mechanism to allow users to manage who can log in - to the device, including allowing anyone (who has a Google Account) - to log in to the device. -* We must not drive users toward having a single, shared account; - common multi-user use cases should fit well within our design. - -It's worth noting that we also have some thoughts about supporting users without -a Google Account; see the [Login -document](/chromium-os/chromiumos-design-docs/login), and the discussion of -"Guest" mode later in this document. - -## Design highlights - -We've shown above that we must, at least, deal with the wrinkle where we need to -access some available, but secured, wireless network to log a new user in. After -that, ideally, we would like to apply all of a user's cloud-cached preferences -locally so that their computing environment can follow them seamlessly from -device to device. In addition to the issue of wireless networks, consider also -the case of keyboard layouts: -> Antoine is French and has set up his Chromium OS device with a US keyboard to -> have some key mappings that will handle accents nicely for him. When he goes -> to France and logs into his sister's Chromium OS device with a French -> keyboard, it doesn't make sense for his mapping from home to apply. In fact, -> it should fail, as the machine has a different keyboard. When I log in to -> Antoine's machine, though, my default US keyboard mapping can be applied, and -> should be. - -The specifics of trying to apply a given user preference and having it fail vary -depending on the preference in question. We define **system settings** as any -setting we need to be able to apply before a user has logged in. -Currently, we break down **system settings** and **user preferences** as -follows: - -<table> -<tr> -System Settings User Preferences </tr> -<tr> -<td>Locale</td> -<td> Wifi networks</td> -<td> Described in this doc:</td> - -<td>Owner</td> -<td> Whitelist</td> -<td> Guest mode</td> - -<td>Proxies</td> - -<td>Bookmarks</td> -<td> New Tab page</td> -<td> Preferences (browser settings)</td> -<td> Apps</td> - -<td>Extensions</td> -<td> Themes</td> -<td> Pinned tabs</td> -<td> Notifications</td> -<td> Printers list</td> -<td>Thumbnails</td> -<td> Autofill data</td> - -</tr> -</table> - -For at least the "current WiFi" setting, we need a user to have done the -configuration first. In other cases, we will have sensible defaults that we will -use to initially configure the owner's account. -These features dictate several requirements: - -1. Settings deemed system settings need to be accessible to all - authenticated users, but not to just anyone who pulls the hard - drive. -2. User preferences should be stored so that only the user in question - can access them. -3. The client side of the sync engine needs to be able to distinguish - system settings from user preferences, so that it can store them - separately. - -### Syncing mechanism - -Out of scope for this document. All that matters from our point of view is that -the client side can distinguish between system settings and user preferences. - -### Caching and applying config info - -The Linux system services that rely on these settings and preferences access -them in a variety of ways, though mostly through configuration files. We must -design a system that can use the preferences and settings we sync down from the -cloud and apply them at the appropriate time. This is to-be-designed, but must -have the following characteristics: - -1. User preferences will be synced down by a Chromium-based browser and - cached in the user's data partition, which is encrypted by default. - The system should be capable of applying these as the user's session - is being brought up. -2. System settings must be usable during the boot process; otherwise, - we won't be able to connect to a network for online login. We would - like, however, to at least protect these settings while the device - is off. -3. The system settings should be writable only by the Owner; we can't - cryptographically enforce this, but we can at least use some system - daemon to enforce it. -4. We need to enable configuration to be federated; multiple sources of - config data (settings and prefs) need to be sensibly overlaid on one - another. The precise semantics depend on the setting in question. - -### Keeping track of the Owner and the whitelist - -The point of keeping track of who the Owner is and who's on the whitelist is to -keep the attacker from getting an account on the device. If they've already -managed to root the box, or to exercise a kernel vuln, they can work around -Ownership and whitelisting anyway (replace the session manager or the chrome -binary, or something else). So, designing a mechanism that is robust against -root seems ... futile (we're working hard to take root away from all userland -processes regardless, in our [system hardening -efforts](/chromium-os/chromiumos-design-docs/system-hardening)). - -If the owner has opted out of whitelisting, we allow every user with a valid -Google Account to log in. - -### Guest mode - -We are considering logically extending the notion of "Incognito" mode found in -Chromium-based browsers to Chromium OS-based devices: a stateless session that -requires no login and that caches data only until the session is over. *No* -state would sync down from the cloud, and *no* state from this session would -persist past termination. The option to start such a session will be on by -default, though the Owner can disable it when she is logged in. We create a -tmpfs on which to store the Chromium browser's session data, and open up an -instance of the browser in incognito mode for the user. Once that instance is -exited, the session is over and we return to the login screen. -By default, a Chromium OS device will be **opted out of whitelisting** and -**opted-in to Guest mode**. We will provide UI that allows a logged-in Owner to -explicitly add users, and provide the Owner an option at that time to opt-in to -whitelisting and Guest mode. - -### Managing write access to system settings - -Only the Owner should be able to change system settings. We don't want random -users able to add to the whitelist or enable guest mode inadvertently. As a -root/kernel exploit means the game is over anyway, it's not worth trying to -design anything that is robust against that kind of compromise; the attacker -could just work around our security measures. So, we will have a settings -daemon, settingsd, that manages write access to the system settings. -When the first user logs in, we generate an RSA keypair and store the private -half in his encrypted home directory, exporting the public half via settingsd. -All requests to update the signed settings store (which includes the whitelist) -must contain a valid signature generated with the owner's private key. When -settingsd receives a request for the value of a system setting, it returns the -signature over that value along with the requested data. The requesting process -then verifies this signature, ensuring that only values which are correctly -signed by the Owner are respected. Of course, a root-level exploit could replace -the exported Owner public key. As stated above, though, the game is already over -in this case.
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/userland-boot/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/userland-boot/index.md deleted file mode 100644 index 973a91eb74a..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/userland-boot/index.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: userland-boot -title: Userland Boot in Chrome OS ---- - diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-crypto/cryptospecs.webp.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-crypto/cryptospecs.webp.sha1 deleted file mode 100644 index 0ab09f97e0c..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-crypto/cryptospecs.webp.sha1 +++ /dev/null @@ -1 +0,0 @@ -64392c17e208c4082a8188684652a04a9a44d66d
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-crypto/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-crypto/index.md deleted file mode 100644 index 6b1c8e72a37..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-crypto/index.md +++ /dev/null @@ -1,440 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: verified-boot-crypto -title: Firmware Verified Boot Crypto Specification ---- - -[TOC] - -### Abstract - -This document describes the cryptographic primitives and building blocks used -for implementing verified boot in the firmware. The document is primarily -targeted at Google Chrome OS-based devices running Google-signed firmware and -software. Chromium OS devices may use similar verified boot schemes with their -own (non-Google-signed) chain of trust. We want to emphasize this does not -prevent devices based on the open source Chromium OS project from having their -own firmware. The goal here is to assure customers of Google Chrome OS devices -that they are actually running Google Chrome OS. - -### Design principles and pitfalls - -* Since we are in firmware land, the cost of changing the architecture - after an implementation is high! Cost varies from "unfixable" for - the boot stub to "need a firmware update" for the writable firmware - and the algorithm for verifying the kernel. - * Choose a conservative and simple design. - * Have the design reviewed by experts. (Review first, implement - later.) -* Security doesn't come for free. We want to minimize the performance - overhead as much as possible. - * In the context of this document, the main performance concern is - minimizing time to boot, or more specifically, the time taken to - pass control to a trusted Chrome OS kernel. -* Use well understood cryptographic building blocks instead of - designing new ones. - * Cryptography is hard to understand, even harder to get right. - * Use well known and studied crypto primitives (RSA, SHA, AES, - etc.) and protocols (public-key signatures). - -### Background - -The firmware is the lowest level of software that runs on a system. In Chrome OS -devices, the firmware is responsible for initializing the main application -processor along with RAM, embedded controllers, chipsets, etc. In certain -architectures, e.g. ARM, a separate modem processor is responsible for -initializing the main processor. Parts of the firmware are *read-only* (at the -lower layers). This code is baked into the firmware during production and can't -be changed. The rest of the firmware is field upgradable. The proposed -mechanisms for such updates are described in a separate document. At the -boundary of *read-only* to *read-write* firmware, the verified boot mechanism -(firmware-land) kicks in. It is responsible for verifying that the next piece of -code is signed by Google for Google Chrome OS devices. The goal here is to -provide a boot path to end users which provides assurancesthat they are running -known good code. - -**Important note:** To reemphasize the point made in the [Verified -Boot](http://www.chromium.org/chromium-os/chromiumos-design-docs/verified-boot) -design document, we are **not** *preventing* users from running *any* operating -system they please, only to provide a mechanism that guarantees that a Chrome OS -device is running the authentic Google Chrome OS. - -## **How cryptographic signature verification works** - -This section presents a small primer on how signature verification is designed -to work in our context. We use public-key cryptography: each key is actually a -keypair with public and private key components. Data encrypted by the private -key can be decrypted by the public key and vice versa. In all cases, the private -key component is known only to Google. The public key is used by the firmware -for performing the verification. A simple implementation would be for Google to -encrypt an entire r/w firmware component with the private key, and for the -firmware to use the corresponding pre-baked public key to decrypt the firmware -and continue execution (if decryption is successful). However, public-key -cryptography is sloooow, and moreover, we are not interested in the -confidentiality of the firmware component, only that any code we execute in the -verified boot path is authenticated. Therefore, we use the concept of -cryptographic signatures. Public-key signatures combine message digests (such as -SHA-1) with public-key cryptography. - -A message digest of a message *m* maps the message to a fixed size hash value -*H(m)*. A cryptographic message digest has two essential properties: - -* Collision resistance: Two different messages are highly unlikely to - map to the same hash value. -* Irreversibility: Given a hash value, it's computationally infeasible - to generate a message *m* which hashes to that value. - -Here are the steps for firmware verification: - -1. For a firmware component blob *F*, Google generates a signature - *S(F)* = *RSA_privatekey(H(F))*, where *H* is the message digest - function. -2. The firmware verification routine takes *F* and *S(F)* as input, - computes *H(F)* and verifies that *H(F)* == *RSA_publickey(S(F)).* - -The security of the above scheme depends on two key factors: - -* The firmware verification routine uses the *correct* public key on - the attached signature to get the expected hash value. -* Only Google can generate a signature that successfully verifies - using the "correct" public key. - -We ensure that the correct public key is used by putting the public key in the -read only part of the firmware. It doesn't matter if it can be read by an -attacker, since it only allows for verifying an existing signature. We ensure -that the signature is generated by Google by keeping the private key known only -to Google, so only Google can generate a valid signature on a blob of data. - -Here's a table listing various hashing and signature verification algorithms, -including their speed as measured by invoking "openssl speed" on a typical -netbook. -<table> -<tr> -<td><b>Algorithm</b></td> -<td><b>Speed</b></td> -<td><b>Note</b></td> -</tr> -<tr> -<td>RSA-1024</td> -<td>0.4ms/verification</td> -<td>NIST recommends not to be used after 2010</td> -</tr> -<tr> -<td>RSA-2048</td> -<td>1.28ms/verification</td> -<td>NIST recommended for use after 2010</td> -</tr> -<tr> -<td>RSA-4096</td> -<td>4.5ms/verification</td> -<td>NIST recommended for use after 2010</td> -</tr> -<tr> -<td>MD5</td> -<td>256MB/s\*</td> -<td>Considered Unsafe</td> -</tr> -<tr> -<td>SHA-1</td> -<td>111MB/s\*</td> -<td>Recent attacks suggest its collision resistance is lesser than previously thought (80-bits)</td> -</tr> -<tr> -<td>SHA-256</td> -<td>26MB/s\*</td> -<td>NIST recommended for use after 2010</td> -</tr> -<tr> -<td>SHA-512</td> -<td>27MB/s\*</td> -</tr> -</table> -\* - 8192 byte blocks. - -### Root keys and signing keys - -Since signature verification comes with an additional overhead, we can't simply -choose to use the strongest available verification algorithm for the whole -verified boot process. However, we need to start with a strong enough algorithm -to protect Chrome OS devices for their expected lifetime. For long-term use, -NIST recommends using *at least* [RSA-2048 and -SHA-256](http://csrc.nist.gov/publications/nistpubs/800-57/sp800-57_part1_rev3_general.pdf). -Therefore, we plan on using a combination of a stronger root key and weaker -signing keys. We will use a strongest available algorithm (within practical -speed constraints) to sign an initial key. Subsequent stages of the verified -boot will use the weaker (but faster) algorithms for verification. - -If one of the weaker algorithms or signing keys used for verification of these -subsequent stages is cracked/broken, a new firmware or software update can be -issued that uses a different signing key and/or a stronger algorithm. - -#### **Preventing rollback attacks** - -To prevent rollback attacks, where an attacker may try to to update a Google -Chrome OS device with an older signed firmware that has a known vulnerability, -each signing key has a strictly increasing associated **Key Version#**. The -read-only firmware writes the highest **Key Version#** so far into an NVRAM -location, in the TPM chip. This NVRAM location is lockable, such that only the -read-only firmware can write to it, and can't be overwritten by any other -firmware component or the operating system. During the verified boot process, -the read-only firmware refuses to pass control to a signed firmware block that -uses a signing key with an index smaller than the highest seen so far. In -addition, since we have two firmware copies, this index can be updated whenever -min(Key#A, Key#B) changes. This allows the device to survive a failed firmware -update. On successful boot with the new firmware, the previous copy of the -firmware is overwritten by the new one, the highest signing key index is -updated, and any further rollbacks are prevented. - -In addition to the **Key Version#** associated with each key, each firmware -update has an associated **Firmware Version#**. The R/O firmware also keeps -track of this in a non-volatile location similar to the Key Version# to prevent -rollbacks to an older R/W firmware. Note that these two numbers serve different -purposes: - -* Key Version# prevents key rollbacks (use of an older invalidated - signing key that might have been compromised to sign a firmware) -* Firmware Version# prevents firmware rollbacks (use of an older - signed firmware update that might have known vulnerabilities) - -#### **Speed and bottleneck considerations** - -Although public key signature verification is an expensive operation, we expect -the main bottleneck to be hash computation of blocks of data that need to be -verified. In addition, verification will likely happen just a few times for each -final hash value that is computed. Given these tradeoffs, it's likely that we'll -still be able to use the stronger verification algorithm (for example, RSA-2048 -below) once we switch to using the signing key for verification, but we'll need -to switch to a weaker hashing algorithm (SHA-1), that is, start with -RSA-4096+SHA-512, then switch to RSA-2048+SHA-1. This tradeoff can be revisited -and analyzed in more detail once we have some hard numbers from an actual -verified boot implementation. - -### Putting it all together: Crypto Architecture - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-crypto/cryptospecs.webp">](/chromium-os/chromiumos-design-docs/verified-boot-crypto/cryptospecs.webp) - -For **Header H:** - -* **Length**, is a fixed-width field and equals the length of H -* **Algorithm** is a fixed-width field integer that specifies the key - size and hashing algorithm to use for verifying signatures using the - signing key. -* **Signing Key** is the signing key used for the rest of the - firmware. -* **Key Version#** is the rollback index associated with the **Signing - Key** -* **Crypto Hash** is a fixed-width field that contains the - cryptographic hash of the rest of the header **H(Len || Algorithm || - Signing Key || Key Version#).** - -Signature **S_root = RSA_Sign(Root Key, H)** - -The signature for the R/W firmware is split into two: - -* Signature **S_signing#1** = **RSA_Sign**(**Signing Key**, **Firmware - Version# | Len | Preamble**) -* Signature **S_signing#2** = **RSA_Sign**(**Signing Key**, **R/W - Firmware Data**). - -This signature helps in verifying the length field and the Firmware rollback -early in the verification process. -For rollback protection, we have a pair of indices: the **Key Version# (K#)** -and the **Firmware Version# (F#)**. K# is updated whenever the signing key is -changed—either if it's past its lifetime (*Y* months) or if it has been -compromised. F# is incremented with each update. Logically, the combination of -K# and F# uniquely identifies the version of an update. The update process will -**refuse to update if either of the following conditions is TRUE**: - -1. K# == stored K#, but F# < stored F# -2. K# < stored K# - -Having separate key version and firmware version rollback prevention indices -allows for generating new firmware updates without the need to generate a new -signing key each time (which in turn requires getting the root key out of secure -storage). This way, firmware updates are protected against rollbacks even if the -signing key stays the same. - -The steps performed by the read-only firmware are as follows: - -1. Verify the signing key signature is valid - - If RSA_Verify(**Root Key**, **H**) fails, Go to **Step 7** (Failure). -2. Verify that this is not a **key rollback** - - If K# < Stored K#, go to **Step 7** (Failure). - - else, Stored K# = K#, lock Stored Key# to prevent write access -3. Verify that this is not a **firmware rollback** - **-** If F# < Stored F#, go to **Step 7** (Failure). - - else, Stored F# = F#, lock Stored F# to prevent write access -4. Determine signing key algorithm to use from "Algorithm." For now, - let's assume we decide to go with RSA-1024 and SHA-1. -5. From here on, the rest of the verification is performed using the - signing key and algorithm - - For example, To verify a firmware blob F, do RSA_Verify(**Signing Key**, - **F**). If NOT, Go to **Step 7.** -6. Success - Continue with **kernel level verification** -7. Failure -If this is firmware-A, repeat above steps with firmware-B. - If firmware-B is bad, go to recovery!! - -Note that, for both the root key and signing key, only the public part of the -key is exposed. This ensures that only Google can generate S_root and S_signing. - -### **Kernel-level verification** - -Once the R/W firmware has been verified, the next step is for the R/W firmware -to verify the kernel before passing control to it. There are a few possible -options here: - -1. Use a different signing key just for kernel signing. This key must - be verified by the R/W firmware signing key. -2. Use the R/W firmware signing key for verifying the kernel. - -The main disadvantage of option 1 is the extra verification step required for -verifying the kernel signing key. In addition, a separate rollback key index -must stored for the kernel signing key. - -The main disadvantage of option 2is that now any kernel update must be -accompanied by a firmware update with a new signing key (for preventing -rollback). - -As a firmware update is much more time consuming and disruptive than a kernel -update, we chose option 1 (use a different signing key just for kernel signing). - -In addition, the kernel is stored on the disk and it needs to be retrieved by -parsing the partition table. Currently, there is no cryptographic verification -of the partition table. However, the firmware code does a strict checking of the -partition table (e.g. ensuring there are no overlapping partitions) and outright -rejects partition tables that do not look right. - -Note also that since this happens in the R/W firmware, the verification -mechanism can change over time and with autoupdates. - -### Lifetime of the root key - -Since the root key is baked into the read-only firmware at manufacture, it can't -be changed once a Chrome OS device leaves the factory. Since any update to this -key requires changes during manufacture, these will be changed once every four -years. Since the root keys have a much larger lifetime than the signing keys, -they will be of higher strength (4096 bit RSA keys or higher). The signing key -to use for pushing an update to a device can be determined via the SKU. Newer -SKUs will use the newer root key chain of trust. - -### What crypto implementations need to be a part of the R/O firmware - -Ideally, due to firmware size limitations, goals for minimal complexity, and the -inability to update in the field, we'd like to limit the functionality -implemented in the the read-only firmware to the bare essentials. - -At the very least, the read-only firmware needs to implement the following: - -1. RSA Signature Verification (for combinations of various 1024-8192 - bit key sizes and SHA-1/256/512 message digest algorithms) -2. SHA-1/256/512 Hashing -3. A way to write and lock a portion of the NVRAM such that it can't be - written to by any other component - -Items 1 and 2 are required to verify the signing key. Item 3 is required to -[prevent rollback -attacks](https://docs.google.com/a/google.com/Doc?docid=0AWwzXTHj7UwzZGhtOHZwNHJfMTNnaHc1cHNmNA&hl=en#Preventing_Rollback_Attacks_32). - -The weaker (relatively) crypto algorithms used by the signing keys **may** be -implemented in the R/W firmware. However, there are risks to consider here: - -1. The algorithm implementations on the R/W firmware also needs to be - verified by the R/O firmware, a task that results in additional - overhead. -2. Additional complexity is required in the R/O firmware for memory - mapping these algorithms for use by the rest of the firmware. - -To avoid the additional complexity involved in having the algorithm -implementations in the R/W firmware, we currently plan on having the R/O -firmware implement all of SHA-1/256/512 and RSA-1024/2048/4096/8192. Although -R/O firmware size may be a constraint, the variants within each class of -algorithms (SHA-256 and SHA-512, for example) are similar, and there should be -some opportunity for code reuse between them. - -### Other considerations - -#### What key size is enough for the root key and signing key? - -NIST -[recommends](http://csrc.nist.gov/publications/nistpubs/800-57/sp800-57_part1_rev3_general.pdf) -the use of RSA 2048/SHA-256 or higher as the signature algorithm for use after -12/31/2010. Our plan is to use a 8192-bit RSA key with SHA-512 for the root key -signatures. - -The security of RSA derives from the computational difficulty of factoring very -large numbers into its prime factors. The best known current algorithm for -factoring large numbers is the Number Field Sieve (NFS). The runtime of NFS -scales as e*c* *n*^1/3 (log2 *n*)^2/3, where *n* is the size of the number in -bits and *c* is typically 1.5-2. - -For the root key, we plan to use a 8192-bit RSA key. Note that, based on the -runtime scaling of the Number Field Sieve, it takes at least 1.429206 x 1024 -more computational effort to factor a 4096-bit number than a 1024-bit number. As -a reference, the general-purpose factoring record is for the 664-bit RSA200 -number which took the equivalent of 55 2.2GHz Opteron CPU years for the sieving -step of NFS and an additional 3 months on an 80 2.2GHz Opteron CPU cluster for -the matrix steps. - -For signing keys, we have considerable more flexibility, since the key size or -the keys themselves can be changed with future system update. Since these keys -will change/rotate often, we plan to use 1024-bit RSA keys which provides a good -speed/performance trade-off. - -#### RSA padding - -RSA operations during signature operations are not directly performed on the raw -cryptographic hashes but rather on a padded/extended version of the -cryptographic hash. We intend to use the standard padding/extension function for -this. There are two padding schemes specified by PKCS#1 v2.1 standard - PKCS1.5 -and PSS. Although PKCS#1 recommends phasing out PKCS1.5 (certain attacks were -discovered against faulty implementations) in favor of PSS, PSS is more complex -to implement. PSS requires the use of a PRNG. So if PSS padding is used, a -crypto-grade PRNG will also need to be implemented as part of the read-only -firmware. Currently, we plan on using PKCS1.5 padding since its implementation -is much simpler. - -#### ****What is the TPM used for?**** - -For verified boot, we do not use the TPM for performing cryptographic -operations, since TPM chips are not necessarily crypto-accelerator devices, and -there is no speed advantage. We use the TPM as secure non-volatile storage for -preventing key rollback attacks. The TCG TPM standard (v1.2 rev103) includes two -mechanisms that we may use for storing monotonically increasing integers in a -runtime, tamper-resistant fashion: - -* NVRAM regions that may be locked and written to prior to TPM - ownership assertion -* Monotonically increasing counters - -We use NVRAM regions in the TPM and write the counters to that space. This area -is only writable at firmware read-only boot time and during recovery mode. - -### **Functional tests and hardware qualifications** - -For firmware vendors, we have a suite of functional tests to test their -implementations. This includes testing crypto blocks like RSA, SHA, as well as -crypto primitives like signature checking. - -For hardware qualification tests, here are examples of tests we use: - -1. **Root Key Verification Test:** Provide firmware where the signing - key cannot be verified by using the root key (S_root is invalid). - **Expected Output:** Device should refuse to use the firmware, jump to - Recovery. -2. **Rollback Prevention Test:** Provide firmware where the signing - key# is less or equal to S_high. - **Expected Output:** Device should refuse to use old firmware, jump to - Recovery. -3. **Signing Key Verification Test:** Provide firmware where the - verification of F using the signing key fails (S_signing is - invalid). - **Expected Output:** Device should refuse to use the firmware, jump to - Recovery. - -More details can be found in the documentation for the Semi-automated Firmware -Testing Framework ([SAFT](/for-testers/saft)).
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/VerifiedBootDataStructures (1).png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/VerifiedBootDataStructures (1).png.sha1 deleted file mode 100644 index 50e68b1b9ef..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/VerifiedBootDataStructures (1).png.sha1 +++ /dev/null @@ -1 +0,0 @@ -0e1731f1c5c183407748c36944402dfbf20b102d
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/VerifiedBootDataStructures.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/VerifiedBootDataStructures.png.sha1 deleted file mode 100644 index 616c6c61bf9..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/VerifiedBootDataStructures.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -cd988dbb44f756d214a6f48d246241850144ab80
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/firmware.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/firmware.png.sha1 deleted file mode 100644 index 3bcc718cc30..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/firmware.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -5728f478912e2c02af89d3c2a366a97c2aba8e0f
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/index.md deleted file mode 100644 index dde2a9f6ec4..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/index.md +++ /dev/null @@ -1,174 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: verified-boot-data-structures -title: Verified Boot Data Structures ---- - -[TOC] - -## Introduction - -As the [verified boot -library](https://chromium.googlesource.com/chromiumos/platform/vboot_reference/) -has evolved to meet the needs of firmware, kernel boot, and security, we’ve -discovered some parts of it can be made easier to work with. This document -describes the refactoring of the verified boot library to address these issues. - -## Basic Guidelines - -### Structures are versioned - -Versioning allows readers to check for older/newer structures. -This feature lets us add fields to the structures without necessarily breaking -old readers and provides an upgrade path for newer readers to handle old -structures. Versioning structures will resolve the issue where kernel and -firmware must be updated in unison. -Structures have a major and minor version number. - -* ReaderMajor != StructureMajor - fundamental change in struct; do not - attempt to parse -* ReaderMinor == StructureMinor - all is well -* ReaderMinor < StructureMinor - reading a newer file; fields have - been added to the struct, but the existing reader will be able to - ignore them and still function -* ReaderMinor > StructureMinor - reading an older file; reader - won’t find the fields added since version StructureMinor, and should - use default values for those fields - -### Key structures keep track of their own key version and algorithm - -While public keys generally have some internal structure, it’s not always -apparent. We can use various sizes and algorithms for different keys, so we’ve -defined a `**VbPublicKey**` structure that simply encapsulates the public key -data along with its size and algorithm: - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbpublickey.png">](/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbpublickey.png) - -`**VbSignature**` structures are very similar, but instead of the algorithm and -version they only remember the length of data they signed. A signature data blob -is usually a hash that has been encrypted with a private key. The corresponding -public key must be used to extract the original hash. In some cases we may also -use a VbSignature structure for a nonencrypted hash. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbsignature.png">](/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbsignature.png) - -### The public key and signature headers know how to find their data - -Given a pointer to the `VbPublicKey` or `VbSignature` header, you can find the -corresponding data, which may not be immediately following the header itself. -This lets us put the header structs as sub-structs in the major structures -(preambles, etc.), followed by the data, without needing to pass in data -pointers separately. -Functions should take header pointers rather than raw data wherever possible. - -### Common structures are reused - -Keys, signatures, and key blocks are used by firmware and kernel algorithms. -Common code for generating and parsing is reused. - -## Major structures - -### Key block - -At the beginning of a firmware or kernel image is a **key block** that contains -the subkey used to sign the rest of the data in the image. The subkey itself is -signed by some higher key, which is presumably more secure or robust. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-data-structures/keyblock.png">](/chromium-os/chromiumos-design-docs/verified-boot-data-structures/keyblock.png) - -The key block itself is either signed (by Google) or simply checksummed (for -self-signed developer images), so only one of the two VbSignatures is used to -validate it. The reason we have two VbSignatures in the key block header is so -that we can build one image that can be validated in either way, which makes -testing and development much simpler. -This key block structure is identical for firmware and kernel images, so the -code to create and verify it can be shared. - -### Firmware image - -A firmware image consists of a key block, a firmware **preamble**, and the -firmware body. The key block, preamble, and body do not directly point to each -other, so they can be read into separate blocks of memory. The firmware preamble -is signed by the key block, and contains the signature of the firmware body plus -another subkey which will be used to validate the kernel image. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-data-structures/firmware.png">](/chromium-os/chromiumos-design-docs/verified-boot-data-structures/firmware.png) - -### Kernel image - -A kernel image consists of a key block, a kernel preamble, and the kernel body. -The key block, preamble, and body do not directly point to each other, so they -can be read into separate blocks of memory. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-data-structures/kernel.png">](/chromium-os/chromiumos-design-docs/verified-boot-data-structures/kernel.png) - -## Root of trust - -When the device is first powered on, it begins execution from a read-only -section of flash memory. In addition to (hopefully) secure code, that read-only -region contains the primary root keys needed to validate all the rest of the -signed images. The structure that locates the root keys and various other -read-only components is the GoogleBinaryBlockHeader, or “**GBB**”. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-data-structures/readonly.png">](/chromium-os/chromiumos-design-docs/verified-boot-data-structures/readonly.png) - -There are two primary keys in the GBB. The **root key** is used in the normal -boot sequence to sign the read/write firmware, which contains subkeys to sign -the kernel, and so forth. In recovery mode, only the read-only firmware is -executed (because the read/write firmware may have been erased), so the kernel -image on the removable drive is signed by the **recovery key**. -There are a total of six separate keypairs involved in the complete boot process -(see diagrams below): - -### Normal boot - -1. Read-only firmware looks in the GBB, finds the root key in read-only - memory. -2. The root key validates the read/write firmware’s keyblock, which - contains the **firmware data key**. -3. The firmware data key validates the firmware preamble (which - contains the **kernel subkey**) and the read/write firmware body. -4. The read/write firmware begins execution. -5. The read/write firmware uses the kernel subkey to validate the - kernel’s keyblock, which contains the **kernel data key**. -6. The kernel data key validates the kernel preamble and the kernel - blob. -7. The kernel begins execution. -8. The kernel image contains the hash of hashes, which is used to - validate the bundle of hashes used by the block-based rootfs - validation scheme. -9. The filesystem driver uses the bundle of hashes to validate each - block of the rootfs as is read off the disk. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-data-structures/VerifiedBootDataStructures.png">](/chromium-os/chromiumos-design-docs/verified-boot-data-structures/VerifiedBootDataStructures.png) - -### Recovery boot - -1. Read-only firmware looks in the GBB, finds the recovery key in - read-only memory. -2. The recovery key validates the kernel’s keyblock, which contains the - **recovery kernel data key**. Although it serves the same purpose as - the kernel subkey in normal boot, it uses a larger key because the - read/write firmware is not used as an intermediate step. -3. The recovery kernel data key validates the kernel preamble and the - kernel blob. -4. The kernel begins execution. -5. The kernel image contains the hash of hashes, which is used to - validate the bundle of hashes used by the block-based rootfs - validation scheme. -6. The filesystem driver uses the bundle of hashes to validate each - block of the rootfs as it is read from the removable media. - -[<img alt="image" -src="/chromium-os/chromiumos-design-docs/verified-boot-data-structures/recovery.png">](/chromium-os/chromiumos-design-docs/verified-boot-data-structures/recovery.png)
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/kernel.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/kernel.png.sha1 deleted file mode 100644 index d24c1bf840b..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/kernel.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -a385193878c4608aba1d2a1a237ce6e384553022
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/keyblock.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/keyblock.png.sha1 deleted file mode 100644 index ce17911eab0..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/keyblock.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -69b1e17fc6bb547c81d69143e3526ed098e3981c
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/normal.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/normal.png.sha1 deleted file mode 100644 index 316cb5b33ba..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/normal.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -4c446d7c1478eb54167852db6477f2244d78f10d
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/readonly.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/readonly.png.sha1 deleted file mode 100644 index 432fc85b54e..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/readonly.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -d1fc51a18cc906e7befab970eb6466ad67173421
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/recovery.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/recovery.png.sha1 deleted file mode 100644 index d3bb5fa4d7f..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/recovery.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -855665e23c4a09f8a33afdc6994d05c16051bb18
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbpublickey.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbpublickey.png.sha1 deleted file mode 100644 index 6169d347d37..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbpublickey.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -5df42dbdedfcd67dbdaf7190777af007645357f3
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbsignature.png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbsignature.png.sha1 deleted file mode 100644 index eefea7574f6..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot-data-structures/vbsignature.png.sha1 +++ /dev/null @@ -1 +0,0 @@ -8f34c65d7bd02f3c88f3e671f034fa7b9f0a32fe
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot/diag2png.sha1 b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot/diag2png.sha1 deleted file mode 100644 index ab8c7ba5fc9..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot/diag2png.sha1 +++ /dev/null @@ -1 +0,0 @@ -42977d11082047463a5c6910208d0bee9e239066
\ No newline at end of file diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot/index.md deleted file mode 100644 index f527fc5a155..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verified-boot/index.md +++ /dev/null @@ -1,382 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: verified-boot -title: Verified Boot ---- - -[TOC] - -## Abstract - -* The Chromium OS team is implementing a verified boot solution that - strives to ensure that users feel secure when logging into a - Chromium OS device. Verified boot starts with a read-only portion of - firmware, which only executes the next chunk of boot code after - verification. -* Verified boot strives to ensure that all executed code comes from - the Chromium OS source tree, rather than from an attacker or - corruption. -* Verified boot is focused on stopping the opportunistic attacker. - While verified boot is not expected to detect every attack, the goal - is to be a significant deterrent which will be improved upon - iteratively. -* Verification during boot is performed on-the-fly to avoid delaying - system start up. It uses stored cryptographic hashes and may be - compatible with any trusted kernel. - -This document extends and expands on the [Firmware Boot and -Recovery](/chromium-os/chromiumos-design-docs/firmware-boot-and-recovery), -[Verified Boot -Crypto](/chromium-os/chromiumos-design-docs/verified-boot-crypto), and [Verified -Boot Data -Structures](/chromium-os/chromiumos-design-docs/verified-boot-data-structures) -documents. - -Verified Boot should provide a mechanism that aids the user in detecting when -their system is in need of recovery due to boot path changes. In particular, it -should meet these requirements: - -* Detect non-volatile memory changes from expected state (rw firmware) -* Detect file system changes relevant to system boot (kernel, init, - modules, fs metadata, policies) -* Support functionality upgrades in the field - -It is important to note that restraining the boot path to only -Chromium-project-supplied code is not a goal. The focus is to ensure that when -code is run that is not provided for or maintained by upstream, that the user -will have the option to immediately reset the device to a known-good state. -Along these lines, there is no dependence on remote attestation or other -external authorization. Users will always own their computers. - -## Goals of verified boot - -The primary attacker in this model is an opportunistic attacker. This means that -the attacker has accessed the system using: - -* a remote vector, such as the Chromium-based browser or a browser - plugin -* a local vector, such as booting to a USB drive and changing files - (but **not** by replacing the write-protected firmware) - -If we assume attackers access the system via a remote vector and bypass all -run-time defenses, then they will have access to modify the root partition -(kernel, modules, browser, ...), update read-write firmware, inject code into -SMRAM, and so on. In addition, the attacker can now access any data in the -currently-logged-in user's account such as locally stored media and website -cookies. The attacker may collect passwords when typed by the user into the -Chromium-based browser or the screenlocker. - -An opportunistic *local* attacker will have a completely different level of -access. Access will be achieved using a USB boot drive, or other out-of-band -bootable material supported by the firmware. Once the system is running the -attacker's operating system, she will be able to modify the root file system and -encrypted user-data blobs. She won't have any visibility into the encrypted -information but may copy it or modify it. - -While Chromium OS does as much as possible to guard against such remote and -local breaches, no software system is impervious to successful attacks. -Therefore, it is important that the attacker cannot continue to "own" a machine -through permanent, local changes. To that end, on boot, the firmware and other -accessible regions of the system internals are verified to be in a known good -state. If they aren't, then the firmware recovery process will be initiated (or -the user can request permission to proceed, which would make sense in the case -of a development install, for example). - -The important factor to consider with the attackers considered above is that if -an attacker gains access via the Chromium browser, they can presumably modify -the Chromium browser's startup (or bookmarks or server-side settings) to -re-attack the machine at next reboot. This is why it is important to be able to -ensure that a safe recovery/reinstall is possible outside of what can be done by -an attacker on the machine. (Obviously, this is no deterrent for a -**determined** attacker willing to modify the system physically.) - -## Getting to the kernel safely - -As outlined in the Firmware Boot and Recovery document, verification will occur -in several places. Initially, the small read-only stub code will compute a SHA-2 -hash (either with internal code or using a provided SHA-2 accelerator) of the -read-write portion of the firmware. An RSA signature of this hash will then be -verified using a permanently stored public key (of, -[ideally](http://csrc.nist.gov/publications/nistpubs/800-57/sp800-57-Part1-revised2_Mar08-2007.pdf), -2048 bits or more). - -The read-write firmware is then responsible for computing hashes of any other -non-volatile memory and the kernel that will be executed. It will contain its -own *subkey* and a list of cryptographic hashes for the data to be verified: -kernel, initrd, master boot record, and so on. These additional hashes will be -signed by the subkey so that they may be updated without requiring the -write-protected key to be used for every update. (Note, the kernel+initrd signed -hashes may be stored with the kernel+initrd on disk to avoid needing a firmware -update when they change.) - -Once we're in the kernel, we've successfully performed a verified boot. - -## Extending verification from the kernel on upward - -In general, once we're running, integrity measurements become less useful. We -can ensure that the Chromium browser that we execute has not been tampered with, -but we can't guarantee that the same attack that compromised it the first time -won't compromise it a second time without updating. - -To ensure that an update is possible, the executables, modules, and -configuration files that allow the system to receive updates must be authentic -and untampered with. Getting to that point requires network access and a running -autoupdater. Given that Chromium OS keeps a very minimal root file system, it's -easier to just verify everything on it. - -While that sounds great in theory, in practice it is hard to guarantee an intact -file system without paying the cost for upfront checks. If the read-write -firmware were to verify the entire root partition before proceeding to boot, it -would add at least 5 seconds to the boot-time on current netbooks. This delay is -untenable. - -Instead of performing full file system verification in advance, it can be done -on demand from a verified kernel. A transparent block device will be layered -between the run-time system and all running processes. It will be configured -during kernel startup using either in-kernel code changes or from a -firmware-verified initial ramdisk image. Each block that is accessed via the -transparent block device layer will be checked against a cryptographic hash -which is stored in a central collection of hashes for the verifiable block -device. This may be in a standalone partition or trailing the filesystem on the -verifiable block device. - -Initially, blocks will be 4KB in size. For a root file system of roughly ~75MB, -there will be roughly 19,200 SHA-1 hashes. On current x86 and ARM based systems, -computing the SHA-1 hash of 4KB takes between 0.2ms and 0.5ms. There will be -additional overhead (TBD) incurred by accessing the correct block hash and -comparing the cryptographic digests. Once verified, blocks should live in the -page cache above the block layer. This will mean that verification does not -occur on every read event. To further amortize time-costs, the block hashes will -be segmented into logical bundles of block hashes. Each bundle will be hashed. -The subsequent list of bundle hashes will then be hashed. This layering can be -repeated as needed to build a tree. The final, single hash will be hard coded -into the kernel or supplied through a device interface from a trusted initial -RAM disk. - - - -Note that SHA-1 is considered to be [unsafe after 2010 by -NIST](http://csrc.nist.gov/groups/ST/toolkit/secure_hashing.html) for general -use. The biggest risk here is that specific block collisions can be found and -made such that they provide an alternate execution path. We could use any -hashing algorithm supported by the Linux kernel in our implementation. SHA-1 is -just a specific example. - -## Known weaknesses of verified boot - -While verified boot can ensure that the system image (i.e. firmware, kernel, -root file system) are protected against tampering by attackers, it can't protect -data that must inherently be modifiable by a running system. This includes user -data, but also system-wide state such as system configuration (network, time -zone, keyboard layout, etc.), cached data maintained by the system (VPD -contents, metrics and crash reporting data, etc.). This state is generally kept -on the writable stateful file system. In some cases, it is consumed by the boot -process and may affect the behavior of the (verified) software. If an attacker -manages to place malicious data on the stateful file system that will cause the -verified code to "take a wrong turn", they may cause inadvertent side effects -that may ultimately lead to the system getting exploited and thus defeating -verified boot. The [Hardening against malicious stateful -data](/chromium-os/chromiumos-design-docs/hardening-against-malicious-stateful-data) -document discusses details and mitigations. - -## Mitigating potential bottlenecks - -Loading hashes off the same disk as the data for each block would affect -performance during a read. Right now, the plan is to read in signed bundles of -hashes as blocks in that bundle range are accessed. Once a bundle is loaded into -memory, it is kept there. If we assume that we're looking at something like -20k-hashes, then that will require around 400KB of memory. Even if the needed -hashes grow to twice that, allocating 1MB of kernel memory doesn't seem too -onerous. In addition, keeping the block hashes in memory will provide for easy -linear addressing of the hashes since they will be in block-order. - -## Handling updates - -For Chromium OS, the autoupdater will update the collection of hashes specific -to the partition it is updating. In general, the complete collection of hashes -for a specific partition will be generated by running a lightweight utility -directly on the filesystem image. It will walk the origin block device and emit -an image file that contains the properly formatted hash collection. In addition, -it will emit the SHA-1 hash of the bundle hashes. This will be the authoritative -hash that will need to be either signed or stored in a signed/verified location, -such as in the kernel. The resulting file can either be appended to the -filesystem image or stored in a standalone partition (hash partition). On -update, a direct difference of the new hash collection can be taken using bsdiff -against the last versions. However, it may be that more efficient difference -generation approaches may be used as long as the end result is the same. - -## The implementation - -Post-firmware verified boot will most likely be implemented as a device mapper -target. It will provide the transparent, verifying block layer. Initially, it -will be assumed that there will be a verified initrd that can be used to set up -the root partition using the dm device. A simple utility tool will be written -that will directly hash a given block device and emit a compatible binary blob -that contains the collection of hashes. It will take the format: - -``` -hashblock_1 . . . hashblock_n -hashbundle_1. . . hashbundle_m -``` - -This data will live either in its own partition or be appended to the verified -partition (aligned on a block boundary). Its location, the hash algorithm used, -and the hash of bundle hashes should be passed in as arguments to the device -mapper setup process (either using dmsetup from an initrd or directly in the -kernel). - -Living implementation documentation can be found -[here](https://chromium.googlesource.com/chromiumos/third_party/kernel/+/HEAD/Documentation/device-mapper/dm-verity.txt) -and -[here](https://chromium.googlesource.com/chromiumos/third_party/kernel/+/HEAD/Documentation/device-mapper/dm-bht.txt). - -## Other issues, ideas, and notes - -* All verification performed after the kernel is running should be - independent of the firmware verification. This allows for developers - to run their own builds as well as for the boot-time verification to - be compatible with a TCG-style boot. -* The partition table should be validated to some extent by the - read-write firmware, but if there are any kernel/firmware partition - parsing bugs, we may be able to catch them with audits as well. -* Verified Boot can play nicely with the TPM. Once a signed kernel is - up, it can initialize the TPM's PCR registers and use those for - measurement tracking. In particular, we have the option of using it - (Linux-IMA style) to perform disk encryption key protection and - possibly even other pre-login state protection where the key becomes - available only if we've booted without modification. -* The hash partition will be subject to replay attack unless the - kernel version that pairs with it is included in the file and the - kernel is upgraded when the hashes are. However, the kernel+initrd - will suffer the same attack for the next level up. Avoiding - replay/rollback attacks is non-trivial since the firmware can't - guarantee the local clock is not changed. A local TPM tickstamp blob - would need to be included in the signed hash payloads to solve the - problem to some degree. If autoupdates were customized per-download, - this may be possible, but at present, this is not planned. -* Key management is of utmost importance for the key used to sign the - read-only firmware. That key should only be used on the R/W firmware - which should be updated much less frequently than the rest of the - system. If possible, this key should never be exposed on a - network-enabled machine, but that is out of scope for this design. -* Having a fully tamper-evident root file system means that if - desired, a manifest of service-specific public keys/certificates - could be stored on the root partition. These keys could then be used - to verify the authenticity and integrity any data stored on the - stateful partition that was signed by a remote server (Google or - another provider). -* There is no plan to support any remote inspection of whether a - Chromium OS installation is using a 'verified boot' or a development - version. -* The autoupdate process currently has a file-centric view. This means - that it could be possible for file system layout to diverge across - machines. If this is the case, block hashes may still be used, but a - more file-centric view may be needed. If the updater moves towards - file system image differencing, this design will work as is. - -## Attack cases - -This section only discusses the current threat model, but many of the attacks -can be generalized to other attack vectors. In addition, these scenarios ignore -all other attack mitigation techniques not included in the document above. In -reality, various system-level and Chromium-level defenses should aid in making -run-time attacks difficult and unreliable. - -*Vector*: Opportunistic local attacker with a USB stick or bootable SD card. -*Scenario*: Attacker boots the system off of an external boot device. The -attacker then changes files and copies the entire system. -*Coverage*: Verified Boot will detect this tampering. Encrypted user data will -still be protected. -*Exposure:* None. User will need to recover their system. - -*Scenario*: Attacker boots the system off of an external boot device and leaves -the system running a "fake" Chromium OS to phish user data. -*Coverage*: Verified Boot will not impact this *unless*the user reboots the -system before logging in. -*Exposure:* None to complete depending on if the user reboots prior to logging -in. If the user left the machine at the screenlocker, a fake screenlocker could -be used in the phishing OS since it is unlikely a user will reboot before -unlocking. This may be addressed in the future with clever authentication use -(PCR+TPM, ?). However, a paranoid user that left their machine in an unsafe -place may just want to reboot to be safe. - -*Vector*: Determined local attacker with a USB drive and a toolkit -*Scenario*: Attacker opens the system up and enables the write-pin on the -write-protected firmware. The attacker then boots the system off of an external -boot device. The root file system is changed along with the formerly -write-protected and read-write firmware. -*Coverage*: Verified Boot will operate normally and will not detect any -variance. -*Exposure:* Complete; a determined attacker that will physically modify the -machine cannot be easily stopped. They may also install hardware keyboard -sniffers, microphones, cameras, etc. - -*Vector*: Chromium or a Chromium plugin -*Scenario*: No superuser privileges are gained, but the attacker can modify -Chromium data. The attacker changes bookmarks, starting pages, marks their pages -as ok for popups, and disables safe browsing. In addition, cookies and stored -passwords are harvested and posted to a remote server. -*Coverage*: Verified Boot has no impact. -*Exposure:* Only the initially compromised user is exposed. - -*Scenario*: Attacker gains superuser privileges. The attacker remounts root -partition read-write directly. The attacker then replaces -/usr/bin/chromeos-chrome with their own build of Chromium that includes -malware/illegal ad revenue and password/credit card sniffing. -*Coverage*: Verified Boot will detect after the next reboot (not after a suspend -to RAM). -*Exposure:* Until reboot, any user that logs in is exposed (password, cookies, -encrypted data). - -*Scenario*: Attacker gains superuser privileges. The attacker remounts root -partition read-write directly. The attacker then adds a kernel module in the -form of a rootkit for the system to load on next reboot. -*Coverage*: Verified Boot will detect after the next reboot. -*Exposure:* Until reboot, any user that logs in is exposed (password, cookies, -encrypted data). - -*Scenario*: Attacker gains superuser privileges. The attacker then modifies the -encrypted file system metadata which exploits a file system bug in the kernel on -next login. -*Coverage*: Verified Boot has no direct impact. However, if the autoupdater runs -before next login, the vulnerability may be patched. -*Exposure*: On next login, the tampered with encrypted file system metadata -attack will trigger. - -*Scenario*: Attacker gains superuser privileges. The attacker then replaces the -hash-partition with an older version and replaces system image with one that has -known vulnerabilities (which may be easier to exploit reliably than the vector -used for attack). The attacker will then change the current user's configuration -to auto-open an attack URL to re-exploit the system immediately after reboot. If -the attacker can gain superuser privileges repeatedly, then it will be difficult -for autoupdate to repair. -*Coverage*: Verified Boot will not be able to detect hash-partition replay -attacks easily. It may be possible to retroactively detect then by the -autoupdater after the network is up, but an attacker will always be able to the -system appear to just be out-of-date. -*Exposure*: Any user that logs in is exposed across reboots. -*Notes*: Downgrade/replay attacks of this nature will be less dangerous if -autoupdater is able to run prior to Chromium starting, but there will most -likely be a race between the two. It may make sense to include a version check -early in the Chromium startup process to detect seriously out-of-date -browsers/systems prior to their opening dangerous pages. - -*Exposure summary:* - -* Run-time attacks against unsignable data: Chromium configuration -* Persisted attacks by defaulting Chromium to launch the attack site - on next start -* Logged-in user data is exposed immediately after compromise. - * With partial signing, a Chromium replacement would result in - cross-user exposure after reboot. - * With full signing, other system users would be notified prior to - exposure -* Metadata attacks against the kernel will not be caught by signing - per-file. -* Downgraded manifest file attacks are difficult to detect since there - is no current way to encoded tamperproof system time into manifest - files. diff --git a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verify-prebuilts-using-content-hashing/index.md b/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verify-prebuilts-using-content-hashing/index.md deleted file mode 100644 index b1df6dedd38..00000000000 --- a/chromium/docs/website/site/chromium-os/chromiumos-design-docs/verify-prebuilts-using-content-hashing/index.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -breadcrumbs: -- - /chromium-os - - Chromium OS -- - /chromium-os/chromiumos-design-docs - - Design Documents -page_name: verify-prebuilts-using-content-hashing -title: 'Design proposal: Verify prebuilts using content hashing' ---- - -# Verify prebuilts using content hashing - -Proposal status: Implemented and in production - -Author: David James <davidjames@> - -## Summary - -I'd like to propose a small change to the way we verify prebuilts. This change -is intended to ensure that developers do not need to build packages from source -if they have not modified them locally. This should shave significant time off -developer builds (e.g., up to 20 minutes, if the developer syncs at a busy -time). - -## What needs to be changed? - -Right now, we only use prebuilts for builds if the SHA1 of the commit that we -tested exactly matches the SHA1 in the remote repository. This is needed for -correctness, as explained below in the “Why is prebuilt verification needed at -all?” section. - -Unfortunately, in practice, the SHA1 of the commit we tested often differs from -the SHA1 in the remote repository. This happens because when the commit queue -submits a change, Gerrit adds in new metadata to the commit message, causing the -prebuilts to become invalid. - -Currently, the commit queue accounts for this problem by just rebuilding any -packages with incorrect SHA1s on the next run. This doesn’t help, though, if -developers sync during the day, when many changes are being pushed by the commit -queue. In that case, this problem causes developer builds to slow down -significantly, as they need to rebuild many packages from source. - -## Proposed solution - -We should use the SHA1 of the contents of the source tree to validate the -prebuilts. Unlike the commit hash, this SHA1 is unaffected by the history of the -repository, or by commit messages. This ensures that users always have correct -prebuilts, without causing unnecessary rebuilds. - -### Implementation - -1. First, add the SHA1 of the contents of the source tree into the - ebuild as `CROS_WORKON_TREE`. This can be calculated instantly using - the following command: - `git log -1 --format=%T` -2. Next, add `CROS_WORKON_TREE_$CROS_WORKON_TREE` to `IUSE` in - `cros-workon.eclass`. -3. Finally, remove `CROS_WORKON_COMMIT_$CROS_WORKON_COMMIT` from `IUSE` - from `cros-workon.eclass`. - -## Why is prebuilt verification needed at all? - -Besides the above proposal, another option for speeding up builds would be to -eliminate verification of SHA1s in ebuilds entirely. This would speed up builds, -but would open up developers to race conditions where they might have incorrect -builds. - -Example: If one developer sends a change through the commit queue, and another -developer bypasses the commit queue at the same time, the ebuild will be marked -with the latest SHA1 (including both change), but the prebuilt may only include -the first change. This results in developers (and bots that use binaries) never -picking up the ‘chump’ change. - -There are also other race conditions. Example: - -1. A chump change is pushed to power_manager. -2. The incremental builder kicks off. -3. Another chump change is pushed to power_manager. -4. The commit queue kicks off and marks both changes as stable. -5. The incremental builder kicks off again, and doesn’t notice any - changes, because its local prebuilt has the same version number as - the one that was pushed by the commit queue. - -Based on the above problems, I think that we should not eliminate prebuilt -verification of SHA1s, and instead use the proposed solution above.
\ No newline at end of file |
.png.sha1?id=c5dbcb143405a38088d78b4b760d64aaff5157ab){kind=link}
.png.sha1?id=c5dbcb143405a38088d78b4b760d64aaff5157ab){kind=link}
.png.sha1?id=c5dbcb143405a38088d78b4b760d64aaff5157ab){kind=link}
