diff options
| author | Allan Sandfeld Jensen <allan.jensen@qt.io> | 2017-04-05 17:15:33 +0200 |
|---|---|---|
| committer | Allan Sandfeld Jensen <allan.jensen@qt.io> | 2017-04-11 07:47:18 +0000 |
| commit | 7324afb043a0b1e623d8e8eb906cdc53bdeb4685 (patch) | |
| tree | a3fe2d74ea9c9e142c390dac4ca0e219382ace46 /chromium/docs | |
| parent | 6a4cabb866f66d4128a97cdc6d9d08ce074f1247 (diff) | |
| download | qtwebengine-chromium-7324afb043a0b1e623d8e8eb906cdc53bdeb4685.tar.gz | |
BASELINE: Update Chromium to 58.0.3029.54
Change-Id: I67f57065a7afdc8e4614adb5c0230281428df4d1
Reviewed-by: Peter Varga <pvarga@inf.u-szeged.hu>
Diffstat (limited to 'chromium/docs')
48 files changed, 3917 insertions, 1094 deletions
diff --git a/chromium/docs/accessibility.md b/chromium/docs/accessibility.md index c0c57797e44..eef1344fe6c 100644 --- a/chromium/docs/accessibility.md +++ b/chromium/docs/accessibility.md @@ -1,515 +1,11 @@ -# Accessibility Overview +# Accessibility -Accessibility means ensuring that all users, including users with disabilities, -have equal access to software. One piece of this involves basic design -principles such as using appropriate font sizes and color contrast, -avoiding using color to convey important information, and providing keyboard -alternatives for anything that is normally accomplished with a pointing device. -However, when you see the word "accessibility" in a directory name in Chromium, -that code's purpose is to provide full access to Chromium's UI via external -accessibility APIs that are utilized by assistive technology. +* [Accessibility Overview](accessibility/overview.md) +* [Accessibility Tests](accessibility/tests.md) -**Assistive technology** here refers to software or hardware which -makes use of these APIs to create an alternative interface for the user to -accommodate some specific needs, for example: +## Chrome OS -Assistive technology includes: - -* Screen readers for blind users that describe the screen using - synthesized speech or braille -* Voice control applications that let you speak to the computer, -* Switch access that lets you control the computer with a small number - of physical switches, -* Magnifiers that magnify a portion of the screen, and often highlight the - cursor and caret for easier viewing, and -* Assistive learning and literacy software that helps users who have a hard - time reading print, by highlighting and/or speaking selected text - -In addition, because accessibility APIs provide a convenient and universal -way to explore and control applications, they're often used for automated -testing scripts, and UI automation software like password managers. - -Web browsers play an important role in this ecosystem because they need -to not only provide access to their own UI, but also provide access to -all of the content of the web. - -Each operating system has its own native accessibility API. While the -core APIs tend to be well-documented, it's unfortunately common for -screen readers in particular to depend on additional undocumented or -vendor-specific APIs in order to fully function, especially with web -browsers, because the standard APIs are insufficient to handle the -complexity of the web. - -Chromium needs to support all of these operating system and -vendor-specific accessibility APIs in order to be usable with the full -ecosystem of assistive technology on all platforms. Just like Chromium -sometimes mimics the quirks and bugs of older browsers, Chromium often -needs to mimic the quirks and bugs of other browsers' implementation -of accessibility APIs, too. - -## Concepts - -While each operating system and vendor accessibility API is different, -there are some concepts all of them share. - -1. The *tree*, which models the entire interface as a tree of objects, exposed - to assistive technology via accessibility APIs; -2. *Events*, which let assistive technology know that a part of the tree has - changed somehow; -3. *Actions*, which come from assistive technology and ask the interface to - change. - -Consider the following small HTML file: - -``` -<html> -<head> - <title>How old are you?</title> -</head> -<body> - <label for="age">Age</label> - <input id="age" type="number" name="age" value="42"> - <div> - <button>Back</button> - <button>Next</button> - </div> -</body> -</html> -``` - -### The Accessibility Tree and Accessibility Attributes - -Internally, Chromium represents the accessibility tree for that web page -using a data structure something like this: - -``` -id=1 role=WebArea name="How old are you?" - id=2 role=Label name="Age" - id=3 role=TextField labelledByIds=[2] value="42" - id=4 role=Group - id=5 role=Button name="Back" - id=6 role=Button name="Next" -``` - -Note that the tree structure closely resembles the structure of the -HTML elements, but slightly simplified. Each node in the accessibility -tree has an ID and a role. Many have a name. The text field has a value, -and instead of a name it has labelledByIds, which indicates that its -accessible name comes from another node in the tree, the label node -with id=2. - -On a particular platform, each node in the accessibility tree is implemented -by an object that conforms to a particular protocol. - -On Windows, the root node implements the IAccessible protocol and -if you call IAccessible::get_accRole, it returns ROLE_SYSTEM_DOCUMENT, -and if you call IAccessible::get_accName, it returns "How old are you?". -Other methods let you walk the tree. - -On macOS, the root node implements the NSAccessibility protocol and -if you call [NSAccessibility accessibilityRole], it returns @"AXWebArea", -and if you call [NSAccessibility accessibilityLabel], it returns -"How old are you?". - -The Linux accessibility API, ATK, is more similar to the Windows APIs; -they were developed together. (Chrome's support for desktop Linux -accessibility is unfinished.) - -The Android accessibility API is of course based on Java. The main -data structure is AccessibilityNodeInfo. It doesn't have a role, but -if you call AccessibilityNodeInfo.getClassName() on the root node -it returns "android.webkit.WebView", and if you call -AccessibilityNodeInfo.getContentDescription() it returns "How old are you?". - -On Chrome OS, we use our own accessibility API that closely maps to -Chrome's internal accessibility API. - -So while the details of the interface vary, the underlying concepts are -similar. Both IAccessible and NSAccessibility have a concept of a role, -but IAccessible uses a role of "document" for a web page, while NSAccessibility -uses a role of "web area". Both IAccessible and NSAccessibility have a -concept of the primary accessible text for a node, but IAccessible calls -it the "name" while NSAccessibility calls it the "label", and Android -calls it a "content description". - -**Historical note:** The internal names of roles and attributes in -Chrome often tend to most closely match the macOS accessibility API -because Chromium was originally based on WebKit, where most of the -accessibility code was written by Apple. Over time we're slowly -migrating internal names to match what those roles and attributes are -called in web accessibility standards, like ARIA. - -### Accessibility Events - -In Chromium's internal terminology, an Accessibility Event always represents -communication from the app to the assistive technology, indicating that the -accessibility tree changed in some way. - -As an example, if the user were to press the Tab key and the text -field from the example above became focused, Chromium would fire a -"focus" accessibility event that assistive technology could listen -to. A screen reader might then announce the name and current value of -the text field. A magnifier might zoom the screen to its bounding -box. If the user types some text into the text field, Chromium would -fire a "value changed" accessibility event. - -As with nodes in the accessibility tree, each platform has a slightly different -API for accessibility events. On Windows we'd fire EVENT_OBJECT_FOCUS for -a focus change, and on Mac we'd fire @"AXFocusedUIElementChanged". -Those are pretty similar. Sometimes they're quite different - to support -live regions (notifications that certain key parts of a web page have changed), -on Mac we simply fire @"AXLiveRegionChanged", but on Windows we need to -fire IA2_EVENT_TEXT_INSERTED and IA2_EVENT_TEXT_REMOVED events individually -on each affected node within the changed region, with additional attributes -like "container-live:polite" to indicate that the affected node was part of -a live region. This discussion is not meant to explain all of the technical -details but just to illustrate that the concepts are similar, -but the details of notifying software on each platform about changes can -vary quite a bit. - -### Accessibility Actions - -Each native object that implements a platform's native accessibility API -supports a number of actions, which are requests from the assistive -technology to control or change the UI. This is the opposite of events, -which are messages from Chromium to the assistive technology. - -For example, if the user had a voice control application running, such as -Voice Access on Android, the user could just speak the name of one of the -buttons on the page, like "Next". Upon recognizing that text and finding -that it matches one of the UI elements on the page, the voice control -app executes the action to click the button id=6 in Chromium's accessibility -tree. Internally we call that action "do default" rather than click, since -it represents the default action for any type of control. - -Other examples of actions include setting focus, changing the value of -a control, and scrolling the page. - -### Parameterized attributes - -In addition to accessibility attributes, events, and actions, native -accessibility APIs often have so-called "parameterized attributes". -The most common example of this is for text - for example there may be -a function to retrieve the bounding box for a range of text, or a -function to retrieve the text properties (font family, font size, -weight, etc.) at a specific character position. - -Parameterized attributes are particularly tricky to implement because -of Chromium's multi-process architecture. More on this in the next section. - -## Chromium's multi-process architecture - -Native accessibility APIs tend to have a *functional* interface, where -Chromium implements an interface for a canonical accessible object that -includes methods to return various attributes, walk the tree, or perform -an action like click(), focus(), or setValue(...). - -In contrast, the web has a largely *declarative* interface. The shape -of the accessibility tree is determined by the DOM tree (occasionally -influenced by CSS), and the accessible semantics of a DOM element can -be modified by adding ARIA attributes. - -One important complication is that all of these native accessibility APIs -are *synchronous*, while Chromium is multi-process, with the contents of -each web page living in a different process than the process that -implements Chromium's UI and the native accessibility APIs. Furthermore, -the renderer processes are *sandboxed*, so they can't implement -operating system APIs directly. - -If you're unfamiliar with Chrome's multi-process architecture, see -[this blog post introducing the concept]( -https://blog.chromium.org/2008/09/multi-process-architecture.html) or -[the design doc on chromium.org]( -https://www.chromium.org/developers/design-documents/multi-process-architecture) -for an intro. - -Chromium's multi-process architecture means that we can't implement -accessibility APIs the same way that a single-process browser can - -namely, by calling directly into the DOM to compute the result of each -API call. For example, on some operating systems there might be an API -to get the bounding box for a particular range of characters on the -page. In other browsers, this might be implemented by creating a DOM -selection object and asking for its bounding box. - -That implementation would be impossible in Chromium because it'd require -blocking the main thread while waiting for a response from the renderer -process that implements that web page's DOM. (Not only is blocking the -main thread strictly disallowed, but the latency of doing this for every -API call makes it prohibitively slow anyway.) Instead, Chromium takes an -approach where a representation of the entire accessibility tree is -cached in the main process. Great care needs to be taken to ensure that -this representation is as concise as possible. - -In Chromium, we build a data structure representing all of the -information for a web page's accessibility tree, send the data -structure from the renderer process to the main browser process, cache -it in the main browser process, and implement native accessibility -APIs using solely the information in that cache. - -As the accessibility tree changes, tree updates and accessibility events -get sent from the renderer process to the browser process. The browser -cache is updated atomically in the main thread, so whenever an external -client (like assistive technology) calls an accessibility API function, -we're always returning something from a complete and consistent snapshot -of the accessibility tree. From time to time, the cache may lag what's -in the renderer process by a fraction of a second. - -Here are some of the specific challenges faced by this approach and -how we've addressed them. - -### Sparse data - -There are a *lot* of possible accessibility attributes for any given -node in an accessibility tree. For example, there are more than 150 -unique accessibility API methods that Chrome implements on the Windows -platform alone. We need to implement all of those APIs, many of which -request rather rare or obscure attributes, but storing all possible -attribute values in a single struct would be quite wasteful. - -To avoid each accessible node object containing hundreds of fields the -data for each accessibility node is stored in a relatively compact -data structure, ui::AXNodeData. Every AXNodeData has an integer ID, a -role enum, and a couple of other mandatory fields, but everything else -is stored in attribute arrays, one for each major data type. - -``` -struct AXNodeData { - int32_t id; - AXRole role; - ... - std::vector<std::pair<AXStringAttribute, std::string>> string_attributes; - std::vector<std::pair<AXIntAttribute, int32_t>> int_attributes; - ... -} -``` - -So if a text field has a placeholder attribute, we can store -that by adding an entry to `string_attributes` with an attribute -of ui::AX_ATTR_PLACEHOLDER and the placeholder string as the value. - -### Incremental tree updates - -Web pages change frequently. It'd be terribly inefficient to send a -new copy of the accessibility tree every time any part of it changes. -However, the accessibility tree can change shape in complicated ways - -for example, whole subtrees can be reparented dynamically. - -Rather than writing code to deal with every possible way the -accessibility tree could be modified, Chromium has a general-purpose -tree serializer class that's designed to send small incremental -updates of a tree from one process to another. The tree serializer has -just a few requirements: - -* Every node in the tree must have a unique integer ID. -* The tree must be acyclic. -* The tree serializer must be notified when a node's data changes. -* The tree serializer must be notified when the list of child IDs of a - node changes. - -The tree serializer doesn't know anything about accessibility attributes. -It keeps track of the previous state of the tree, and every time the tree -structure changes (based on notifications of a node changing or a node's -children changing), it walks the tree and builds up an incremental tree -update that serializes as few nodes as possible. - -In the other process, the Unserialization code applies the incremental -tree update atomically. - -### Text bounding boxes - -One challenge faced by Chromium is that accessibility clients want to be -able to query the bounding box of an arbitrary range of text - not necessarily -just the current cursor position or selection. As discussed above, it's -not possible to block Chromium's main browser process while waiting for this -information from Blink, so instead we cache enough information to satisfy these -queries in the accessibility tree. - -To compactly store the bounding box of every character on the page, we -split the text into *inline text boxes*, sometimes called *text runs*. -For example, in a typical paragraph, each line of text would be its own -inline text box. In general, an inline text box or text run contians a -sequence of text characters that are all oriented in the same direction, -in a line, with the same font, size, and style. - -Each inline text box stores its own bounding box, and then the relative -x-coordinate of each character in its text (assuming left-to-right). -From that it's possible to compute the bounding box -of any individual character. - -The inline text boxes are part of Chromium's internal accessibility tree. -They're used purely internally and aren't ever exposed directly via any -native accessibility APIs. - -For example, suppose that a document contains a text field with the text -"Hello world", but the field is narrow, so "Hello" is on the first line and -"World" is on the second line. Internally Chromium's accessibility tree -might look like this: - -``` -staticText location=(8, 8) size=(38, 36) name='Hello world' - inlineTextBox location=(0, 0) size=(36, 18) name='Hello ' characterOffsets=12,19,23,28,36 - inlineTextBox location=(0, 18) size=(38, 18) name='world' characterOffsets=12,20,25,29,37 -``` - -### Scrolling, transformations, and animation - -Native accessibility APIs typically want the bounding box of every element in the -tree, either in window coordinates or global screen coordinates. If we -stored the global screen coordinates for every node, we'd be constantly -re-serializing the whole tree every time the user scrolls or drags the -window. - -Instead, we store the bounding box of each node in the accessibility tree -relative to its *offset container*, which can be any ancestor. If no offset -container is specified, it's assumed to be the root of the tree. - -In addition, any offset container can contain scroll offsets, which can be -used to scroll the bounding boxes of anything in that subtree. - -Finally, any offset container can also include an arbitrary 4x4 transformation -matrix, which can be used to represent arbitrary 3-D rotations, translations, and -scaling, and more. The transformation matrix applies to the whole subtree. - -Storing coordinates this way means that any time an object scrolls, moves, or -animates its position and scale, only the root of the scrolling or animation -needs to post updates to the accessibility tree. Everything in the subtree -remains valid relative to that offset container. - -Computing the global screen coordinates for an object in the accessibility -tree just means walking up its ancestor chain and applying offsets and -occasionally multiplying by a 4x4 matrix. - -### Site isolation / out-of-process iframes - -At one point in time, all of the content of a single Tab or other web view -was contained in the same Blink process, and it was possible to serialize -the accessibility tree for a whole frame tree in a single pass. - -Today the situation is a bit more complicated, as Chromium supports -out-of-process iframes. (It also supports "browser plugins" such as -the `<webview>` tag in Chrome packaged apps, which embeds a whole -browser inside a browser, but for the purposes of accessibility this -is handled the same as frames.) - -Rather than a mix of in-process and out-of-process frames that are handled -differently, Chromium builds a separate independent accessibility tree -for each frame. Each frame gets its own tree ID, and it keeps track of -the tree ID of its parent frame (if any) and any child frames. - -In Chrome's main browser process, the accessibility trees for each frame -are cached separately, and when an accessibility client (assistive -technology) walks the accessibility tree, Chromium dynamically composes -all of the frames into a single virtual accessibility tree on the fly, -using those aforementioned tree IDs. - -The node IDs for accessibility trees only need to be unique within a -single frame. Where necessary, separate unique IDs are used within -Chrome's main browser process. In Chromium accessibility, a "node ID" -always means that ID that's only unique within a frame, and a "unique ID" -means an ID that's globally unique. - -## Blink - -Blink constructs an accessibility tree (a hierarchy of [WebAXObject]s) from the -page it is rendering. WebAXObject is the public API wrapper around [AXObject], -which is the core class of Blink's accessibility tree. AXObject is an abstract -class; the most commonly used concrete subclass of it is [AXNodeObject], which -wraps a [Node]. In turn, most AXNodeObjects are actually [AXLayoutObject]s, -which wrap both a [Node] and a [LayoutObject]. Access to the LayoutObject is -important because some elements are only in the AXObject tree depending on their -visibility, geometry, linewrapping, and so on. There are some subclasses of -AXLayoutObject that implement special-case logic for specific types of Node. -There are also other subclasses of AXObject, which are mostly used for testing. - -Note that not all AXLayoutObjects correspond to actual Nodes; some are synthetic -layout objects which group related inline elements or similar. - -The central class responsible for dealing with accessibility events in Blink is -[AXObjectCacheImpl], which is responsible for caching the corresponding -AXObjects for Nodes or LayoutObjects. This class has many methods named -`handleFoo`, which are called throughout Blink to notify the AXObjectCacheImpl -that it may need to update its tree. Since this class is already aware of all -accessibility events in Blink, it is also responsible for relaying accessibility -events from Blink to the embedding content layer. - -## The content layer - -The content layer lives on both sides of the renderer/browser split. The content -layer translates WebAXObjects into [AXContentNodeData], which is a subclass of -[ui::AXNodeData]. The ui::AXNodeData class and related classes are Chromium's -cross-platform accessibility tree. The translation is implemented in -[BlinkAXTreeSource]. This translation happens on the renderer side, so the -ui::AXNodeData tree now needs to be sent to the browser, which is done by -sending [AccessibilityHostMsg_EventParams] with the payload being serialized -delta-updates to the tree, so that changes that happen on the renderer side can -be reflected on the browser side. - -On the browser side, these IPCs are received by [RenderFrameHostImpl], and then -usually forwarded to [BrowserAccessibilityManager] which is responsible for: - -1. Merging AXNodeData trees into one tree of [BrowserAccessibility] objects, - by linking to other BrowserAccessibilityManagers. This is important because - each page has its own accessibility tree, but each Chromium *window* must - have only one accessibility tree, so trees from multiple pages need to be - combined (possibly also with trees from Views UI). -2. Dispatching outgoing accessibility events to the platform's accessibility - APIs. This is done in the platform-specific subclasses of - BrowserAccessibilityManager, in a method named `NotifyAccessibilityEvent`. -3. Dispatching incoming accessibility actions to the appropriate recipient, via - [BrowserAccessibilityDelegate]. For messages destined for a renderer, - [RenderFrameHostImpl], which is a BrowserAccessibilityDelegate, is - responsible for sending appropriate `AccessibilityMsg_Foo` IPCs to the - renderer, where they will be received by [RenderAccessibilityImpl]. - -On Chrome OS, RenderFrameHostImpl does not route events to -BrowserAccessibilityManager at all, since there is no platform screenreader -outside Chromium to integrate with. - -## Views - -Views generates a [NativeViewAccessibility] for each View, which is used as the -delegate for an [AXPlatformNode] representing that View. This part is relatively -straightforward, but then the generated tree must be combined with the web -accessibility tree, which is handled by BrowserAccessibilityManager. - -## WebUI - -Since WebUI surfaces have renderer processes as normal, WebUI accessibility goes -through the blink-to-content-to-platform pipeline described above. Accessibility -for WebUI is largely implemented in JavaScript in [webui-js]; these classes take -care of adding ARIA attributes and so on to DOM nodes as needed. - -## The Chrome OS layer - -The accessibility tree is also exposed via the [chrome.automation API], which -gives extension JavaScript access to the accessibility tree, events, and -actions. This API is implemented in C++ by [AutomationInternalCustomBindings], -which is renderer-side code, and in JavaScript by the [automation API]. The API -is defined by [automation.idl], which must be kept synchronized with -[ax_enums.idl]. - -[AccessibilityHostMsg_EventParams]: https://cs.chromium.org/chromium/src/content/common/accessibility_messages.h?sq=package:chromium&l=75 -[AutomationInternalCustomBindings]: https://cs.chromium.org/chromium/src/chrome/renderer/extensions/automation_internal_custom_bindings.h -[AXContentNodeData]: https://cs.chromium.org/chromium/src/content/common/ax_content_node_data.h -[AXLayoutObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/accessibility/AXLayoutObject.h -[AXNodeObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/accessibility/AXNodeObject.h -[AXObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/accessibility/AXObject.h -[AXObjectCacheImpl]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/accessibility/AXObjectCacheImpl.h -[AXPlatformNode]: https://cs.chromium.org/chromium/src/ui/accessibility/platform/ax_platform_node.h -[AXTreeSerializer]: https://cs.chromium.org/chromium/src/ui/accessibility/ax_tree_serializer.h -[BlinkAXTreeSource]: https://cs.chromium.org/chromium/src/content/renderer/accessibility/blink_ax_tree_source.h -[BrowserAccessibility]: https://cs.chromium.org/chromium/src/content/browser/accessibility/browser_accessibility.h -[BrowserAccessibilityDelegate]: https://cs.chromium.org/chromium/src/content/browser/accessibility/browser_accessibility_manager.h?sq=package:chromium&l=64 -[BrowserAccessibilityManager]: https://cs.chromium.org/chromium/src/content/browser/accessibility/browser_accessibility_manager.h -[LayoutObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/layout/LayoutObject.h -[NativeViewAccessibility]: https://cs.chromium.org/chromium/src/ui/views/accessibility/native_view_accessibility.h -[Node]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/dom/Node.h -[RenderAccessibilityImpl]: https://cs.chromium.org/chromium/src/content/renderer/accessibility/render_accessibility_impl.h -[RenderFrameHostImpl]: https://cs.chromium.org/chromium/src/content/browser/frame_host/render_frame_host_impl.h -[ui::AXNodeData]: https://cs.chromium.org/chromium/src/ui/accessibility/ax_node_data.h -[WebAXObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/public/web/WebAXObject.h -[automation API]: https://cs.chromium.org/chromium/src/chrome/renderer/resources/extensions/automation -[automation.idl]: https://cs.chromium.org/chromium/src/chrome/common/extensions/api/automation.idl -[ax_enums.idl]: https://cs.chromium.org/chromium/src/ui/accessibility/ax_enums.idl -[chrome.automation API]: https://developer.chrome.com/extensions/automation -[webui-js]: https://cs.chromium.org/chromium/src/ui/webui/resources/js/cr/ui/ +* [ChromeVox for Developers](accessibility/chromevox.md) +* [ChromeVox on Desktop Linux](accessibility/chromevox_on_desktop_linux.md) +* [Updating brltty braille drivers](accessibility/brltty.md) +* [Updating the patts speech synthesis engine](accessibility/patts.md) diff --git a/chromium/docs/accessibility/brltty.md b/chromium/docs/accessibility/brltty.md new file mode 100644 index 00000000000..94fd260ff3e --- /dev/null +++ b/chromium/docs/accessibility/brltty.md @@ -0,0 +1,65 @@ +# BRLTTY in Chrome OS + +Chrome OS uses the open-source [BRLTTY](http://mielke.cc/brltty/) +library to provide support for refreshable braille displays. + +We typically ship with a stable release build of BRLTTY plus some +cherry-picked patches. + +## Updating BRLTTY or adding a patch + +First, follow the public +[Chromium OS Developer Guide](http://www.chromium.org/chromium-os/developer-guide) to check out the source. +At a minimum you'll need to create a chroot. +You do not need to build everything from source. +You do need to start the devserver. + +Next, flash your device to a very recent test build. Internally at Google +you can do this with the following command when the dev server is running, +where CHROMEBOOK_IP_ADDRESS is the IP address of your Chromebook already +in developer mode, and $BOARD is your Chromebook's board name. + +```cros flash ssh://CHROMEBOOK_IP_ADDRESS xbuddy://remote/$BOARD/latest-dev/test``` + +The BRLTTY files can be found in this directory: + +```third_party/chromiumos-overlay/app-accessibility/brltty``` + +The first thing you'll need to do is edit the ebuild symlink to change the +revision number. The real file is something like brltty-5.4.ebuild, +but the revision will be something like brltty-5.4-r5.ebuild. You'll need +to increment it. + +To increment it from r5 to r6, you'd do something like this: + +``` +rm brltty-5.4-r5.ebuild +ln -s brltty-5.4.ebuild brltty-5.4-r6.ebuild +git add brltty-5.4-r6.ebuild +``` + +The changes we make are all patches against a stable release of brltty. +To add a new patch, put it in the files/ directory and reference it in +brltty.bashrc + +Once you're done adding patches or making other changes, flash it to your +device like this: + +``` +emerge-$BOARD brltty +cros deploy CHROMEBOOK_IP_ADDRESS brltty +``` + +After that, reboot your Chromebook and verify that brltty works. + +To upload a change, use repo, something like this: + +``` +repo start <branch_name> . +git commit -a + BUG=chromium:12345 + TEST=Write what you tested here +repo upload . +``` + +Note that you shouldn't need to run cros_workon. diff --git a/chromium/docs/accessibility/chromevox.md b/chromium/docs/accessibility/chromevox.md new file mode 100644 index 00000000000..ebc6f6f5ac0 --- /dev/null +++ b/chromium/docs/accessibility/chromevox.md @@ -0,0 +1,131 @@ +# ChromeVox (for developers) + +ChromeVox is the built-in screen reader on Chrome OS. It was originally +developed as a separate extension but now the code lives inside of the Chromium +tree and it's built as part of Chrome OS. + +To start or stop ChromeVox on Chrome OS, press Ctrl+Alt+Z at any time. + +## Developer Info + +Code location: ```chrome/browser/resources/chromeos/chromevox``` + +Ninja target: it's built as part of "chrome", but you can build and run +chromevox_tests to test it (Chrome OS target only - you must have target_os = +"chromeos" in your GN args first). + +## Developing On Linux + +ChromeVox for Chrome OS development is done on Linux. + +See [ChromeVox on Desktop Linux](chromevox_on_desktop_linux.md) +for more information. + +## ChromeVox Next + +ChromeVox Next is the code name we use for a major new rewrite to ChromeVox that +uses the automation API instead of content scripts. The code is part of +ChromeVox (unique ChromeVox Next code is found in +chrome/browser/resources/chromeos/chromevox/cvox2). + +ChromeVox contains all of the classic and next code in the same codebase, it +switches its behavior dynamically based on the mode: + +* Next: as of version 56 of Chrome/Chrome OS, this is default. ChromeVox uses new key/braille bindings, earcons, speech/braille output style, the Next engine (Automation API), and other major/minor improvements +* Next Compat: in order to maintain compatibility with some clients of the ChromeVox Classic js APIs, some sites have been whitelisted for this mode. ChromeVox will inject classic content scripts, but expose a Next-like user experience (like above) +* Classic: as of version 56 of Chrome/Chrome OS, this mode gets enabled via a keyboard toggle Search+Q. Once enabled, ChromeVox will behave like it did in the past including keyboard bindings, earcons, speech/braille output style, and the underlying engine (content scripts). +* Classic compat for some sites that require Next, while running in Classic, ChromeVox will use the Next engine but expose a Classic user experience (like above) + +Once it's ready, the plan is to retire everything other than Next mode. + +## ChromeVox Next + +To test ChromeVox Next, click on the Gear icon in the upper-right of the screen +to open the ChromeVox options (or press the keyboard shortcut Search+Shift+O, O) +and then click the box to opt into ChromeVox Next. + +If you are running m56 or later, you already have ChromeVox Next on by +default. To switch back to Classic, press Search+Q. + +## Debugging ChromeVox + +There are options available that may assist in debugging ChromeVox. Here are a +few use cases. + +### Feature development + +When developing a new feature, it may be helpful to save time by not having to +go through a compile cycle. This can be achieved by setting +```chromevox_compress_js``` to 0 in +chrome/browser/resources/chromeos/chromevox/BUILD.gn, or by using a debug build. + +In a debug build or with chromevox_compress_js off, the unflattened files in the +Chrome out directory (e.g. out/Release/resources/chromeos/chromevox/). Now you +can hack directly on the copy of ChromeVox in out/ and toggle ChromeVox to pick +up your changes (via Ctrl+Alt+Z). + +### Fixing bugs + +The easiest way to debug ChromeVox is from an external browser. Start Chrome +with this command-line flag: + +```out/Release/chrome --remote-debugging-port=9222``` + +Now open http://localhost:9222 in a separate instance of the browser, and debug the ChromeVox extension background page from there. + +Another option is to use emacs jade (available through -mx +package-list-packages). + +It also talks to localhost:9222 but integrates more tightly into emacs instead. + +Another option is to use the built-in developer console. Go to the +ChromeVox options page with Search+Shift+o, o; then, substitute the +“options.html” path with “background.html”, and then open up the +inspector. + +### Running tests + +Build the chromevox_tests target. To run +lots of tests in parallel, run it like this: + +```out/Release/chromevox_tests --test-launcher-jobs=20``` + +Use a test filter if you only want to run some of the tests from a +particular test suite - for example, most of the ChromeVox Next tests +have "E2E" in them (for "end-to-end"), so to only run those: + +```out/Release/chromevox_tests --test-launcher-jobs=20 --gtest_filter="*E2E*"``` + +## ChromeVox for other platforms + +ChromeVox can be run as an installable extension, separate from a +linux Chrome OS build. + +### From source + +chrome/browser/resources/chromeos/chromevox/tools has the required scripts that pack ChromeVox as an extension and make any necessary manifest changes. + +### From Webstore + +Alternatively, the webstore has the stable version of ChromeVox. + +To install without interacting with the webstore UI, place the +following json block in +/opt/google/chrome-unstable/extensions/kgejglhpjiefppelpmljglcjbhoiplfn.json + +``` +{ +"external_update_url": "https://clients2.google.com/service/update2/crx" +} +``` + +If you're using the desktop Linux version of Chrome, we recommend you +use Voxin for speech. Run chrome with: “google-chrome +--enable-speech-dispatcher” and select a voice provided by the speechd +package from the ChromeVox options page (ChromeVox+o, o). As of the +latest revision of Chrome 44, speechd support has become stable enough +to use with ChromeVox, but still requires the flag. + +In the ChromeVox options page, select the flat keymap and use sticky +mode (double press quickly of insert) to emulate a modal screen +reader. diff --git a/chromium/docs/accessibility/chromevox_on_desktop_linux.md b/chromium/docs/accessibility/chromevox_on_desktop_linux.md new file mode 100644 index 00000000000..3dc128588d6 --- /dev/null +++ b/chromium/docs/accessibility/chromevox_on_desktop_linux.md @@ -0,0 +1,115 @@ +# ChromeVox on Desktop Linux + +## Starting ChromeVox + +On Chrome OS, you can enable spoken feedback (ChromeVox) by pressing Ctrl+Alt+Z. + +If you have a Chromebook, this gives you speech support built-in. If you're +building Chrome from source and running it on desktop Linux, speech and braille +won't be included by default. Here's how to enable it. + +## Compiling the Chrome OS version of Chrome + +First follow the public instructions for +[Chrome checkout and build](https://www.chromium.org/developers/how-tos/get-the-code). + +Create a GN configuration with "chromeos" as the target OS, for example: + +``` +gn args out/cros +``` + +...in editor, add this line: + +``` +target_os = "chromeos" +is_component_build = true +is_debug = false +``` + +Note: Only ```target_os = "chromeos"``` is required, the others are recommended +for a good experience but you can configure Chrome however you like otherwise. +Note that Native Client is required, so do not put `enable_nacl = false` in +your file anywhere! + +Now build Chrome as usual, e.g.: + +``` +ninja -C out/cros chrome +``` + +And run it as usual to see a mostly-complete Chrome OS desktop inside +of a window: + +``` +out/cros/chrome +``` + +By default you'll be logged in as the default user. If you want to +simulate the login manager too, run it like this: + +``` +out/cros/chrome --login-manager +``` + +You can run any of the above under it’s own X session (avoiding any window +manager key combo conflicts) by doing something like + +``` +startx out/cros/chrome +``` + +## Speech + +If you want speech, you just need to copy the speech synthesis data files to +/usr/share like it would be on a Chrome OS device: + +``` +sudo git clone https://chromium.googlesource.com/chromiumos/platform/assets /usr/share/chromeos-assets +``` + +Change the permissions: + +``` +sudo find /usr/share/chromeos-assets -type f -exec chmod 644 {} \; +sudo find /usr/share/chromeos-assets -type d -exec chmod 755 {} \; +``` + +Next, unzip the NaCl executables. You only need to do the one for your host +architecture: + +``` +PATTS_DIR=/usr/share/chromeos-assets/speech_synthesis/patts +sudo unzip $PATTS_DIR/tts_service_x86-64.nexe.zip -d $PATTS_DIR +``` + +**Be sure to check permissions of /usr/share/chromeos-assets, some users report +they need to chmod or chown too, it really depends on your system.** + +After you do that, just run "chrome" as above (e.g. out/cros/chrome) and press +Ctrl+Alt+Z, and you should hear it speak! If not, check the logs. + +## Braille + +ChromeVox uses extension APIs to deliver braille to Brltty through libbrlapi +and uses Liblouis to perform translation and backtranslation. + +Once built, Chrome and ChromeVox will use your machine’s running Brltty +daemon to display braille if ChromeVox is running. Simply ensure you have a +display connected before running Chrome and that Brltty is running. + +Testing against the latest releases of Brltty (e.g. 5.4 at time of writing) is +encouraged. + +For more general information, see [ChromeVox](chromevox.md) + +# Using ChromeVox + +ChromeVox keyboard shortcuts use Search. On Linux that's usually your Windows +key. If some shortcuts don't work, you may need to remove Gnome keyboard +shortcut bindings, or use "startx", as suggested above, or remap it. + +* Search+Space: Click +* Search+Left/Right: navigate linearly +* Search+Period: Open ChromeVox menus +* Search+H: jump to next heading on page diff --git a/chromium/docs/accessibility/overview.md b/chromium/docs/accessibility/overview.md new file mode 100644 index 00000000000..c0c57797e44 --- /dev/null +++ b/chromium/docs/accessibility/overview.md @@ -0,0 +1,515 @@ +# Accessibility Overview + +Accessibility means ensuring that all users, including users with disabilities, +have equal access to software. One piece of this involves basic design +principles such as using appropriate font sizes and color contrast, +avoiding using color to convey important information, and providing keyboard +alternatives for anything that is normally accomplished with a pointing device. +However, when you see the word "accessibility" in a directory name in Chromium, +that code's purpose is to provide full access to Chromium's UI via external +accessibility APIs that are utilized by assistive technology. + +**Assistive technology** here refers to software or hardware which +makes use of these APIs to create an alternative interface for the user to +accommodate some specific needs, for example: + +Assistive technology includes: + +* Screen readers for blind users that describe the screen using + synthesized speech or braille +* Voice control applications that let you speak to the computer, +* Switch access that lets you control the computer with a small number + of physical switches, +* Magnifiers that magnify a portion of the screen, and often highlight the + cursor and caret for easier viewing, and +* Assistive learning and literacy software that helps users who have a hard + time reading print, by highlighting and/or speaking selected text + +In addition, because accessibility APIs provide a convenient and universal +way to explore and control applications, they're often used for automated +testing scripts, and UI automation software like password managers. + +Web browsers play an important role in this ecosystem because they need +to not only provide access to their own UI, but also provide access to +all of the content of the web. + +Each operating system has its own native accessibility API. While the +core APIs tend to be well-documented, it's unfortunately common for +screen readers in particular to depend on additional undocumented or +vendor-specific APIs in order to fully function, especially with web +browsers, because the standard APIs are insufficient to handle the +complexity of the web. + +Chromium needs to support all of these operating system and +vendor-specific accessibility APIs in order to be usable with the full +ecosystem of assistive technology on all platforms. Just like Chromium +sometimes mimics the quirks and bugs of older browsers, Chromium often +needs to mimic the quirks and bugs of other browsers' implementation +of accessibility APIs, too. + +## Concepts + +While each operating system and vendor accessibility API is different, +there are some concepts all of them share. + +1. The *tree*, which models the entire interface as a tree of objects, exposed + to assistive technology via accessibility APIs; +2. *Events*, which let assistive technology know that a part of the tree has + changed somehow; +3. *Actions*, which come from assistive technology and ask the interface to + change. + +Consider the following small HTML file: + +``` +<html> +<head> + <title>How old are you?</title> +</head> +<body> + <label for="age">Age</label> + <input id="age" type="number" name="age" value="42"> + <div> + <button>Back</button> + <button>Next</button> + </div> +</body> +</html> +``` + +### The Accessibility Tree and Accessibility Attributes + +Internally, Chromium represents the accessibility tree for that web page +using a data structure something like this: + +``` +id=1 role=WebArea name="How old are you?" + id=2 role=Label name="Age" + id=3 role=TextField labelledByIds=[2] value="42" + id=4 role=Group + id=5 role=Button name="Back" + id=6 role=Button name="Next" +``` + +Note that the tree structure closely resembles the structure of the +HTML elements, but slightly simplified. Each node in the accessibility +tree has an ID and a role. Many have a name. The text field has a value, +and instead of a name it has labelledByIds, which indicates that its +accessible name comes from another node in the tree, the label node +with id=2. + +On a particular platform, each node in the accessibility tree is implemented +by an object that conforms to a particular protocol. + +On Windows, the root node implements the IAccessible protocol and +if you call IAccessible::get_accRole, it returns ROLE_SYSTEM_DOCUMENT, +and if you call IAccessible::get_accName, it returns "How old are you?". +Other methods let you walk the tree. + +On macOS, the root node implements the NSAccessibility protocol and +if you call [NSAccessibility accessibilityRole], it returns @"AXWebArea", +and if you call [NSAccessibility accessibilityLabel], it returns +"How old are you?". + +The Linux accessibility API, ATK, is more similar to the Windows APIs; +they were developed together. (Chrome's support for desktop Linux +accessibility is unfinished.) + +The Android accessibility API is of course based on Java. The main +data structure is AccessibilityNodeInfo. It doesn't have a role, but +if you call AccessibilityNodeInfo.getClassName() on the root node +it returns "android.webkit.WebView", and if you call +AccessibilityNodeInfo.getContentDescription() it returns "How old are you?". + +On Chrome OS, we use our own accessibility API that closely maps to +Chrome's internal accessibility API. + +So while the details of the interface vary, the underlying concepts are +similar. Both IAccessible and NSAccessibility have a concept of a role, +but IAccessible uses a role of "document" for a web page, while NSAccessibility +uses a role of "web area". Both IAccessible and NSAccessibility have a +concept of the primary accessible text for a node, but IAccessible calls +it the "name" while NSAccessibility calls it the "label", and Android +calls it a "content description". + +**Historical note:** The internal names of roles and attributes in +Chrome often tend to most closely match the macOS accessibility API +because Chromium was originally based on WebKit, where most of the +accessibility code was written by Apple. Over time we're slowly +migrating internal names to match what those roles and attributes are +called in web accessibility standards, like ARIA. + +### Accessibility Events + +In Chromium's internal terminology, an Accessibility Event always represents +communication from the app to the assistive technology, indicating that the +accessibility tree changed in some way. + +As an example, if the user were to press the Tab key and the text +field from the example above became focused, Chromium would fire a +"focus" accessibility event that assistive technology could listen +to. A screen reader might then announce the name and current value of +the text field. A magnifier might zoom the screen to its bounding +box. If the user types some text into the text field, Chromium would +fire a "value changed" accessibility event. + +As with nodes in the accessibility tree, each platform has a slightly different +API for accessibility events. On Windows we'd fire EVENT_OBJECT_FOCUS for +a focus change, and on Mac we'd fire @"AXFocusedUIElementChanged". +Those are pretty similar. Sometimes they're quite different - to support +live regions (notifications that certain key parts of a web page have changed), +on Mac we simply fire @"AXLiveRegionChanged", but on Windows we need to +fire IA2_EVENT_TEXT_INSERTED and IA2_EVENT_TEXT_REMOVED events individually +on each affected node within the changed region, with additional attributes +like "container-live:polite" to indicate that the affected node was part of +a live region. This discussion is not meant to explain all of the technical +details but just to illustrate that the concepts are similar, +but the details of notifying software on each platform about changes can +vary quite a bit. + +### Accessibility Actions + +Each native object that implements a platform's native accessibility API +supports a number of actions, which are requests from the assistive +technology to control or change the UI. This is the opposite of events, +which are messages from Chromium to the assistive technology. + +For example, if the user had a voice control application running, such as +Voice Access on Android, the user could just speak the name of one of the +buttons on the page, like "Next". Upon recognizing that text and finding +that it matches one of the UI elements on the page, the voice control +app executes the action to click the button id=6 in Chromium's accessibility +tree. Internally we call that action "do default" rather than click, since +it represents the default action for any type of control. + +Other examples of actions include setting focus, changing the value of +a control, and scrolling the page. + +### Parameterized attributes + +In addition to accessibility attributes, events, and actions, native +accessibility APIs often have so-called "parameterized attributes". +The most common example of this is for text - for example there may be +a function to retrieve the bounding box for a range of text, or a +function to retrieve the text properties (font family, font size, +weight, etc.) at a specific character position. + +Parameterized attributes are particularly tricky to implement because +of Chromium's multi-process architecture. More on this in the next section. + +## Chromium's multi-process architecture + +Native accessibility APIs tend to have a *functional* interface, where +Chromium implements an interface for a canonical accessible object that +includes methods to return various attributes, walk the tree, or perform +an action like click(), focus(), or setValue(...). + +In contrast, the web has a largely *declarative* interface. The shape +of the accessibility tree is determined by the DOM tree (occasionally +influenced by CSS), and the accessible semantics of a DOM element can +be modified by adding ARIA attributes. + +One important complication is that all of these native accessibility APIs +are *synchronous*, while Chromium is multi-process, with the contents of +each web page living in a different process than the process that +implements Chromium's UI and the native accessibility APIs. Furthermore, +the renderer processes are *sandboxed*, so they can't implement +operating system APIs directly. + +If you're unfamiliar with Chrome's multi-process architecture, see +[this blog post introducing the concept]( +https://blog.chromium.org/2008/09/multi-process-architecture.html) or +[the design doc on chromium.org]( +https://www.chromium.org/developers/design-documents/multi-process-architecture) +for an intro. + +Chromium's multi-process architecture means that we can't implement +accessibility APIs the same way that a single-process browser can - +namely, by calling directly into the DOM to compute the result of each +API call. For example, on some operating systems there might be an API +to get the bounding box for a particular range of characters on the +page. In other browsers, this might be implemented by creating a DOM +selection object and asking for its bounding box. + +That implementation would be impossible in Chromium because it'd require +blocking the main thread while waiting for a response from the renderer +process that implements that web page's DOM. (Not only is blocking the +main thread strictly disallowed, but the latency of doing this for every +API call makes it prohibitively slow anyway.) Instead, Chromium takes an +approach where a representation of the entire accessibility tree is +cached in the main process. Great care needs to be taken to ensure that +this representation is as concise as possible. + +In Chromium, we build a data structure representing all of the +information for a web page's accessibility tree, send the data +structure from the renderer process to the main browser process, cache +it in the main browser process, and implement native accessibility +APIs using solely the information in that cache. + +As the accessibility tree changes, tree updates and accessibility events +get sent from the renderer process to the browser process. The browser +cache is updated atomically in the main thread, so whenever an external +client (like assistive technology) calls an accessibility API function, +we're always returning something from a complete and consistent snapshot +of the accessibility tree. From time to time, the cache may lag what's +in the renderer process by a fraction of a second. + +Here are some of the specific challenges faced by this approach and +how we've addressed them. + +### Sparse data + +There are a *lot* of possible accessibility attributes for any given +node in an accessibility tree. For example, there are more than 150 +unique accessibility API methods that Chrome implements on the Windows +platform alone. We need to implement all of those APIs, many of which +request rather rare or obscure attributes, but storing all possible +attribute values in a single struct would be quite wasteful. + +To avoid each accessible node object containing hundreds of fields the +data for each accessibility node is stored in a relatively compact +data structure, ui::AXNodeData. Every AXNodeData has an integer ID, a +role enum, and a couple of other mandatory fields, but everything else +is stored in attribute arrays, one for each major data type. + +``` +struct AXNodeData { + int32_t id; + AXRole role; + ... + std::vector<std::pair<AXStringAttribute, std::string>> string_attributes; + std::vector<std::pair<AXIntAttribute, int32_t>> int_attributes; + ... +} +``` + +So if a text field has a placeholder attribute, we can store +that by adding an entry to `string_attributes` with an attribute +of ui::AX_ATTR_PLACEHOLDER and the placeholder string as the value. + +### Incremental tree updates + +Web pages change frequently. It'd be terribly inefficient to send a +new copy of the accessibility tree every time any part of it changes. +However, the accessibility tree can change shape in complicated ways - +for example, whole subtrees can be reparented dynamically. + +Rather than writing code to deal with every possible way the +accessibility tree could be modified, Chromium has a general-purpose +tree serializer class that's designed to send small incremental +updates of a tree from one process to another. The tree serializer has +just a few requirements: + +* Every node in the tree must have a unique integer ID. +* The tree must be acyclic. +* The tree serializer must be notified when a node's data changes. +* The tree serializer must be notified when the list of child IDs of a + node changes. + +The tree serializer doesn't know anything about accessibility attributes. +It keeps track of the previous state of the tree, and every time the tree +structure changes (based on notifications of a node changing or a node's +children changing), it walks the tree and builds up an incremental tree +update that serializes as few nodes as possible. + +In the other process, the Unserialization code applies the incremental +tree update atomically. + +### Text bounding boxes + +One challenge faced by Chromium is that accessibility clients want to be +able to query the bounding box of an arbitrary range of text - not necessarily +just the current cursor position or selection. As discussed above, it's +not possible to block Chromium's main browser process while waiting for this +information from Blink, so instead we cache enough information to satisfy these +queries in the accessibility tree. + +To compactly store the bounding box of every character on the page, we +split the text into *inline text boxes*, sometimes called *text runs*. +For example, in a typical paragraph, each line of text would be its own +inline text box. In general, an inline text box or text run contians a +sequence of text characters that are all oriented in the same direction, +in a line, with the same font, size, and style. + +Each inline text box stores its own bounding box, and then the relative +x-coordinate of each character in its text (assuming left-to-right). +From that it's possible to compute the bounding box +of any individual character. + +The inline text boxes are part of Chromium's internal accessibility tree. +They're used purely internally and aren't ever exposed directly via any +native accessibility APIs. + +For example, suppose that a document contains a text field with the text +"Hello world", but the field is narrow, so "Hello" is on the first line and +"World" is on the second line. Internally Chromium's accessibility tree +might look like this: + +``` +staticText location=(8, 8) size=(38, 36) name='Hello world' + inlineTextBox location=(0, 0) size=(36, 18) name='Hello ' characterOffsets=12,19,23,28,36 + inlineTextBox location=(0, 18) size=(38, 18) name='world' characterOffsets=12,20,25,29,37 +``` + +### Scrolling, transformations, and animation + +Native accessibility APIs typically want the bounding box of every element in the +tree, either in window coordinates or global screen coordinates. If we +stored the global screen coordinates for every node, we'd be constantly +re-serializing the whole tree every time the user scrolls or drags the +window. + +Instead, we store the bounding box of each node in the accessibility tree +relative to its *offset container*, which can be any ancestor. If no offset +container is specified, it's assumed to be the root of the tree. + +In addition, any offset container can contain scroll offsets, which can be +used to scroll the bounding boxes of anything in that subtree. + +Finally, any offset container can also include an arbitrary 4x4 transformation +matrix, which can be used to represent arbitrary 3-D rotations, translations, and +scaling, and more. The transformation matrix applies to the whole subtree. + +Storing coordinates this way means that any time an object scrolls, moves, or +animates its position and scale, only the root of the scrolling or animation +needs to post updates to the accessibility tree. Everything in the subtree +remains valid relative to that offset container. + +Computing the global screen coordinates for an object in the accessibility +tree just means walking up its ancestor chain and applying offsets and +occasionally multiplying by a 4x4 matrix. + +### Site isolation / out-of-process iframes + +At one point in time, all of the content of a single Tab or other web view +was contained in the same Blink process, and it was possible to serialize +the accessibility tree for a whole frame tree in a single pass. + +Today the situation is a bit more complicated, as Chromium supports +out-of-process iframes. (It also supports "browser plugins" such as +the `<webview>` tag in Chrome packaged apps, which embeds a whole +browser inside a browser, but for the purposes of accessibility this +is handled the same as frames.) + +Rather than a mix of in-process and out-of-process frames that are handled +differently, Chromium builds a separate independent accessibility tree +for each frame. Each frame gets its own tree ID, and it keeps track of +the tree ID of its parent frame (if any) and any child frames. + +In Chrome's main browser process, the accessibility trees for each frame +are cached separately, and when an accessibility client (assistive +technology) walks the accessibility tree, Chromium dynamically composes +all of the frames into a single virtual accessibility tree on the fly, +using those aforementioned tree IDs. + +The node IDs for accessibility trees only need to be unique within a +single frame. Where necessary, separate unique IDs are used within +Chrome's main browser process. In Chromium accessibility, a "node ID" +always means that ID that's only unique within a frame, and a "unique ID" +means an ID that's globally unique. + +## Blink + +Blink constructs an accessibility tree (a hierarchy of [WebAXObject]s) from the +page it is rendering. WebAXObject is the public API wrapper around [AXObject], +which is the core class of Blink's accessibility tree. AXObject is an abstract +class; the most commonly used concrete subclass of it is [AXNodeObject], which +wraps a [Node]. In turn, most AXNodeObjects are actually [AXLayoutObject]s, +which wrap both a [Node] and a [LayoutObject]. Access to the LayoutObject is +important because some elements are only in the AXObject tree depending on their +visibility, geometry, linewrapping, and so on. There are some subclasses of +AXLayoutObject that implement special-case logic for specific types of Node. +There are also other subclasses of AXObject, which are mostly used for testing. + +Note that not all AXLayoutObjects correspond to actual Nodes; some are synthetic +layout objects which group related inline elements or similar. + +The central class responsible for dealing with accessibility events in Blink is +[AXObjectCacheImpl], which is responsible for caching the corresponding +AXObjects for Nodes or LayoutObjects. This class has many methods named +`handleFoo`, which are called throughout Blink to notify the AXObjectCacheImpl +that it may need to update its tree. Since this class is already aware of all +accessibility events in Blink, it is also responsible for relaying accessibility +events from Blink to the embedding content layer. + +## The content layer + +The content layer lives on both sides of the renderer/browser split. The content +layer translates WebAXObjects into [AXContentNodeData], which is a subclass of +[ui::AXNodeData]. The ui::AXNodeData class and related classes are Chromium's +cross-platform accessibility tree. The translation is implemented in +[BlinkAXTreeSource]. This translation happens on the renderer side, so the +ui::AXNodeData tree now needs to be sent to the browser, which is done by +sending [AccessibilityHostMsg_EventParams] with the payload being serialized +delta-updates to the tree, so that changes that happen on the renderer side can +be reflected on the browser side. + +On the browser side, these IPCs are received by [RenderFrameHostImpl], and then +usually forwarded to [BrowserAccessibilityManager] which is responsible for: + +1. Merging AXNodeData trees into one tree of [BrowserAccessibility] objects, + by linking to other BrowserAccessibilityManagers. This is important because + each page has its own accessibility tree, but each Chromium *window* must + have only one accessibility tree, so trees from multiple pages need to be + combined (possibly also with trees from Views UI). +2. Dispatching outgoing accessibility events to the platform's accessibility + APIs. This is done in the platform-specific subclasses of + BrowserAccessibilityManager, in a method named `NotifyAccessibilityEvent`. +3. Dispatching incoming accessibility actions to the appropriate recipient, via + [BrowserAccessibilityDelegate]. For messages destined for a renderer, + [RenderFrameHostImpl], which is a BrowserAccessibilityDelegate, is + responsible for sending appropriate `AccessibilityMsg_Foo` IPCs to the + renderer, where they will be received by [RenderAccessibilityImpl]. + +On Chrome OS, RenderFrameHostImpl does not route events to +BrowserAccessibilityManager at all, since there is no platform screenreader +outside Chromium to integrate with. + +## Views + +Views generates a [NativeViewAccessibility] for each View, which is used as the +delegate for an [AXPlatformNode] representing that View. This part is relatively +straightforward, but then the generated tree must be combined with the web +accessibility tree, which is handled by BrowserAccessibilityManager. + +## WebUI + +Since WebUI surfaces have renderer processes as normal, WebUI accessibility goes +through the blink-to-content-to-platform pipeline described above. Accessibility +for WebUI is largely implemented in JavaScript in [webui-js]; these classes take +care of adding ARIA attributes and so on to DOM nodes as needed. + +## The Chrome OS layer + +The accessibility tree is also exposed via the [chrome.automation API], which +gives extension JavaScript access to the accessibility tree, events, and +actions. This API is implemented in C++ by [AutomationInternalCustomBindings], +which is renderer-side code, and in JavaScript by the [automation API]. The API +is defined by [automation.idl], which must be kept synchronized with +[ax_enums.idl]. + +[AccessibilityHostMsg_EventParams]: https://cs.chromium.org/chromium/src/content/common/accessibility_messages.h?sq=package:chromium&l=75 +[AutomationInternalCustomBindings]: https://cs.chromium.org/chromium/src/chrome/renderer/extensions/automation_internal_custom_bindings.h +[AXContentNodeData]: https://cs.chromium.org/chromium/src/content/common/ax_content_node_data.h +[AXLayoutObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/accessibility/AXLayoutObject.h +[AXNodeObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/accessibility/AXNodeObject.h +[AXObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/accessibility/AXObject.h +[AXObjectCacheImpl]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/accessibility/AXObjectCacheImpl.h +[AXPlatformNode]: https://cs.chromium.org/chromium/src/ui/accessibility/platform/ax_platform_node.h +[AXTreeSerializer]: https://cs.chromium.org/chromium/src/ui/accessibility/ax_tree_serializer.h +[BlinkAXTreeSource]: https://cs.chromium.org/chromium/src/content/renderer/accessibility/blink_ax_tree_source.h +[BrowserAccessibility]: https://cs.chromium.org/chromium/src/content/browser/accessibility/browser_accessibility.h +[BrowserAccessibilityDelegate]: https://cs.chromium.org/chromium/src/content/browser/accessibility/browser_accessibility_manager.h?sq=package:chromium&l=64 +[BrowserAccessibilityManager]: https://cs.chromium.org/chromium/src/content/browser/accessibility/browser_accessibility_manager.h +[LayoutObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/layout/LayoutObject.h +[NativeViewAccessibility]: https://cs.chromium.org/chromium/src/ui/views/accessibility/native_view_accessibility.h +[Node]: https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/dom/Node.h +[RenderAccessibilityImpl]: https://cs.chromium.org/chromium/src/content/renderer/accessibility/render_accessibility_impl.h +[RenderFrameHostImpl]: https://cs.chromium.org/chromium/src/content/browser/frame_host/render_frame_host_impl.h +[ui::AXNodeData]: https://cs.chromium.org/chromium/src/ui/accessibility/ax_node_data.h +[WebAXObject]: https://cs.chromium.org/chromium/src/third_party/WebKit/public/web/WebAXObject.h +[automation API]: https://cs.chromium.org/chromium/src/chrome/renderer/resources/extensions/automation +[automation.idl]: https://cs.chromium.org/chromium/src/chrome/common/extensions/api/automation.idl +[ax_enums.idl]: https://cs.chromium.org/chromium/src/ui/accessibility/ax_enums.idl +[chrome.automation API]: https://developer.chrome.com/extensions/automation +[webui-js]: https://cs.chromium.org/chromium/src/ui/webui/resources/js/cr/ui/ diff --git a/chromium/docs/accessibility/patts.md b/chromium/docs/accessibility/patts.md new file mode 100644 index 00000000000..7d67ee8580b --- /dev/null +++ b/chromium/docs/accessibility/patts.md @@ -0,0 +1,112 @@ +# The Chrome OS PATTS speech synthesis engine + +Chrome OS comes with a speech synthesis engine developed internally at Google +called PATTS. It's based on the same engine that ships with all Android devices. + +## Building from source + +This is for Googlers only. + +Visit [http://go/chrome-tts-blaze](http://go/chrome-tts-blaze) +for instructions on how to build the engine from source and get the +latest voice files. + +When debugging, start Chrome from the command-line and set the +NACL_PLUGIN_DEBUG environment variable to 1 to print log messages to stdout. + +## Updating + +First, follow the public +[Chromium OS Developer Guide](http://www.chromium.org/chromium-os/developer-guide) to check out the source. +At a minimum you'll need to create a chroot. +You do not need to build everything from source. +You do need to start the devserver. + +Next, flash your device to a very recent test build. Internally at Google +you can do this with the following command when the dev server is running, +where CHROMEBOOK_IP_ADDRESS is the IP address of your Chromebook already +in developer mode, and $BOARD is your Chromebook's board name. + +```cros flash ssh://CHROMEBOOK_IP_ADDRESS xbuddy://remote/$BOARD/latest-dev/test``` + +Before you can make changes to PATTS, the first thing you need to run +(from the chroot) is call cros_workon with two relevant ebuilds: + +``` +cros_workon --board=$BOARD start chromiumos-assets +cros_workon --board=$BOARD start common-assets +``` + +From outside the root, from anywhere under your top-level ```<repo-dir>```, pull down the relevant sources: + +``` +repo sync +``` + +Again, outside the root, make sure you're in the ```<repo-dir>/src/platform/assets``` directory and run +```repo start``` to create a branch: + +``` +cd src/platform/assets +repo start <branch_name> . +``` + + +The PATTS data files can be found in this directory: + +```src/platform/assets/speech_synthesis/patts``` + +When updating the files, the native client files (nexe) need to be zipped. + +Replace all of the files you need to update, commit them using git, +then from the chroot, run: + +``` +emerge-$BOARD common-assets +cros deploy CHROMEBOOK_IP_ADDRESS common-assets +``` + +Note that you need to call cros_workon on both chromeos-assets and +common-assets. You will be changing files in chromeos-assets, but +to flash it onto your device, you need to emerge and deploy +common-assets. + +After that, reboot your Chromebook and verify that speech works. + +To upload the change, use repo upload, something like this: + +``` +git commit -a + BUG=chromium:12345 + TEST=Write what you tested here +repo upload . +``` + +## Ebuild + +Note that sometimes you'll have to update the ebuild file that +takes the patts data files and installs them, unzipping the .nexe +files in the process. + +For example, you'll need to edit the ebuild if you add or remove +a language code, or if you add or remove a file that needs to be +installed as part of the extension. + +To update the ebuild, edit this file: + +``` +/third_party/chromiumos-overlay/chromeos-base/common-assets/common-assets-9999.ebuild +``` + +If you need to land changes to both common-assets and chromiumos-assets, +upload the changes separately and then make them depend on one another +using this syntax in the changelog: + +``` +CQ-DEPEND=CL:12345 +``` + +Note that you can (and often should) have two changes depend on one another +so they'll land atomically. + + diff --git a/chromium/docs/accessibility/tests.md b/chromium/docs/accessibility/tests.md new file mode 100644 index 00000000000..f1a74207885 --- /dev/null +++ b/chromium/docs/accessibility/tests.md @@ -0,0 +1,123 @@ +# Accessibility + +Here's a quick overview of all of the locations in the codebase where +you'll find accessibility tests, and a brief overview of the purpose of +all of them. + +## Layout Tests + +This is the primary place where we test accessibility code in Blink. This +code should have no platform-specific code. Use this to test anything +where there's any interesting web platform logic, or where you need to be +able to query things synchronously from the renderer thread to test them. + +Don't add tests for trivial features like ARIA attributes that we just +expose directly to the next layer up. In those cases the Blink tests are +trivial and it's more valuable to test these features at a higher level +where we can ensure they actually work. + +Note that many of these tests are inherited from WebKit and the coding style +has evolved a lot since then. Look for more recent tests as a guide if writing +a new one. + +Test files: +``` +third_party/WebKit/LayoutTests/accessibility +``` + +Source code to AccessibilityController and WebAXObjectProxy: +``` +content/shell/test_runner +``` + +To run all accessibility LayoutTests: +``` +ninja -C out/release blink_tests +third_party/WebKit/Tools/Scripts/run-webkit-tests --build-directory=out --target=release accessibility/ +``` + +To run just one test by itself without the script: +``` +ninja -C out/release blink_tests +out/release/content_shell --run-layout-test third_party/WebKit/LayoutTests/accessibility/name-calc-inputs.html +``` + +## DumpAccessibilityTree tests + +This is the best way to write both cross-platform and platform-specific tests +using only an input HTML file, some magic strings to describe what attributes +you're interested in, and one or more expectation files to enable checking +whether the resulting accessibility tree is correct or not. In particular, +almost no test code is required. + +[More documentation on DumpAccessibilityTree](../../content/test/data/accessibility/readme.md) + +Test files: +``` +content/test/data/accessibility +``` + +Test runner: +``` +content/browser/accessibility/dump_accessibility_tree_browsertest.cc +``` + +To run all tests: +``` +ninja -C out/release content_browsertests +out/release/content_browsertests --gtest_filter="DumpAccessibilityTree*" +``` + +## Other content_browsertests + +There are many other tests in content/ that relate to accessibility. +All of these tests work by launching a full multi-process browser shell, +loading a web page in a renderer, then accessing the resulting accessibility +tree from the browser process, and running some test from there. + +To run all tests: +``` +ninja -C out/release content_browsertests +out/release/content_browsertests --gtest_filter="*ccessib*" +``` + +## Accessibility unittests + +This tests the core accessibility code that's shared by both web and non-web +accessibility infrastructure. + +Code location: +``` +ui/accessibility +``` + +To run all tests: +``` +ninja -C out/release accessibility_unittests +out/release/accessibility_unittests +``` + +## ChromeVox tests + +You must build with ```target_os = "chromeos"``` in your GN args. + +To run all tests: +``` +ninja -C out/release chromevox_tests +out/release/chromevox_tests --test-launcher-jobs=10 +``` + +## Other locations of accessibility tests: + +Even this isn't a complete list. The tests described above cover more +than 90% of the accessibility tests, and the remainder are scattered +throughout the codebase. Here are a few other locations to check: + +``` +chrome/android/javatests/src/org/chromium/chrome/browser/accessibility +chrome/browser/accessibility +chrome/browser/chromeos/accessibility/ +ui/chromeos +ui/views/accessibility +``` + diff --git a/chromium/docs/adding_to_third_party.md b/chromium/docs/adding_to_third_party.md index 89b9ff19e02..745e7598cd7 100644 --- a/chromium/docs/adding_to_third_party.md +++ b/chromium/docs/adding_to_third_party.md @@ -82,7 +82,7 @@ license is not included in about:credits page. ### Modify DEPS If the code is applicable and will be compiled on all supported Chromium -platforms (Windows, Mac, Linux, ChromeOS, iOS, Android), check it in to +platforms (Windows, Mac, Linux, Chrome OS, iOS, Android), check it in to [src/third_party](http://src.chromium.org/viewvc/chrome/trunk/src/third_party/). If the code is only applicable to certain platforms, check it in to @@ -126,12 +126,14 @@ All third party additions and substantive changes like re-licensing need the following sign-offs. Some of these are accessible to Googlers only. Non-Googlers can email one of the people in third_party/OWNERS for help. -* Chrome Eng Review. Googlers should see go/chrome-eng-review +* Chrome Eng Review. Googlers should see go/chrome-eng-review (please include information about the additional checkout size, build times, and binary sizes. Please also make sure that the motivation for your project is clear, e.g., a design doc has been circulated). * open-source-third-party-reviews@google.com (ping the list with relevant details and a link to the CL). * security@chromium.org (ping the list with relevant details and a link to the CL). +Please send separate emails to the three lists. + Third party code is a hot spot for security vulnerabilities. When adding a new package that could potentially carry security risk, make sure to highlight risk to security@chromium.org. You may be asked to add a README.security or, in diff --git a/chromium/docs/android_build_instructions.md b/chromium/docs/android_build_instructions.md index 2e2cf1bfd23..be30dfa87bf 100644 --- a/chromium/docs/android_build_instructions.md +++ b/chromium/docs/android_build_instructions.md @@ -134,7 +134,7 @@ configurations. To create a build directory which builds Chrome for Android, run: ```shell -$ gn gen '--args="target_os="android"' out/Default +$ gn gen --args='target_os="android"' out/Default ``` * You only have to run this once for each new build directory, Ninja will diff --git a/chromium/docs/android_studio.md b/chromium/docs/android_studio.md index 2a10aa7a8c8..3752122574c 100644 --- a/chromium/docs/android_studio.md +++ b/chromium/docs/android_studio.md @@ -4,7 +4,8 @@ ## Usage -Make sure you have followed [android build instructions](android_build_instructions.md) already. +Make sure you have followed +[android build instructions](android_build_instructions.md) already. ```shell build/android/gradle/generate_gradle.py @@ -20,24 +21,28 @@ By default, only common targets are generated. To customize the list of targets to generate projects for: ```shell -build/android/gradle/generate_gradle.py --target //some:target_apk --target //some/other:target_apk +build/android/gradle/generate_gradle.py --target //chrome/android:chrome_public_apk --target //android_webview/test:android_webview_apk ``` For first-time Android Studio users: * Avoid running the setup wizard. - * The wizard will force you to download unwanted SDK components to `//third_party/android_tools`. + * The wizard will force you to download unwanted SDK components to + `//third_party/android_tools`. * To skip it, select "Cancel" when it comes up. To import the project: -* Use "Import Project", and select the directory containing the generated project, by default `out-gn/Debug/gradle`. +* Use "Import Project", and select the directory containing the generated + project, by default `out/Debug/gradle`. You need to re-run `generate_gradle.py` whenever `BUILD.gn` files change. -* After regenerating, Android Studio should prompt you to "Sync". If it doesn't, use: +* After regenerating, Android Studio should prompt you to "Sync". If it + doesn't, use: + * Button with two arrows on the right side of the top strip. * Help -> Find Action -> "Sync Project with Gradle Files" - + * After `gn clean` you may need to restart Android Studio. ## How it Works @@ -45,17 +50,13 @@ Android Studio integration works by generating `build.gradle` files based on GN targets. Each `android_apk` and `android_library` target produces a separate Gradle sub-project. -### Symlinks and .srcjars +### Excluded files and .srcjars Gradle supports source directories but not source files. However, some -`java/src/` directories in Chromium are split amonst multiple GN targets. To -accommodate this, the script detects such targets and creates a `symlinked-java/` -directory to point gradle at. Be warned that creating new files from Android -Studio within these symlink-based projects will cause new files to be created in -the generated `symlinked-java/` rather than the source tree where you want it. - -*** note -** TLDR:** Always create new files outside of Android Studio. +directories in Chromium are split amonst multiple GN targets. To accommodate +this, the script detects such targets and creates exclude patterns to exclude +files not in the current target. You may still see them when editing, but they +are excluded in gradle tasks. *** Most generated .java files in GN are stored as `.srcjars`. Android Studio does @@ -69,12 +70,15 @@ includes `R.java`). ## Android Studio Tips -* Configuration instructions can be found [here](http://tools.android.com/tech-docs/configuration). One suggestions: - * Launch it with more RAM: `STUDIO_VM_OPTIONS=-Xmx2048m /opt/android-studio-stable/bin/studio-launcher.sh` +* Configuration instructions can be found + [here](http://tools.android.com/tech-docs/configuration). One suggestions: + * Launch it with more RAM: + `STUDIO_VM_OPTIONS=-Xmx2048m /opt/android-studio-stable/bin/studio-launcher.sh` * If you ever need to reset it: `rm -r ~/.AndroidStudio*/` * Import Android style settings: - * Help -> Find Action -> "Code Style" (settings) -> Java -> Manage -> Import - * Select `third_party/android_platform/development/ide/intellij/codestyles/AndroidStyle.xml` + * Help -> Find Action -> "Code Style" (settings) -> Java -> + Manage -> Import + * Select `tools/android/android_studio/ChromiumStyle.xml` * Turn on automatic import: * Help -> Find Action -> "Auto Import" * Tick all the boxes under "Java" and change the dropdown to "All". @@ -97,19 +101,22 @@ includes `R.java`). ### Building from the Command Line -Gradle builds can be done from the command-line after importing the project into -Android Studio (importing into the IDE causes the Gradle wrapper to be added). -This wrapper can also be used to invoke gradle commands. +Gradle builds can be done from the command-line after importing the project +into Android Studio (importing into the IDE causes the Gradle wrapper to be +added). This wrapper can also be used to invoke gradle commands. cd $GRADLE_PROJECT_DIR && bash gradlew The resulting artifacts are not terribly useful. They are missing assets, resources, native libraries, etc. -* Use a [gradle daemon](https://docs.gradle.org/2.14.1/userguide/gradle_daemon.html) to speed up builds: - * Add the line `org.gradle.daemon=true` to `~/.gradle/gradle.properties`, creating it if necessary. +* Use a + [gradle daemon](https://docs.gradle.org/2.14.1/userguide/gradle_daemon.html) + to speed up builds using the gradlew script: + * Add the line `org.gradle.daemon=true` to `~/.gradle/gradle.properties`, + creating it if necessary. -## Status (as of Jan 19, 2017) +## Status (as of Feb 7th, 2017) ### What works @@ -117,10 +124,12 @@ resources, native libraries, etc. * Java editing and gradle compile works. * Instrumentation tests included as androidTest. * Symlinks to existing .so files in jniLibs (doesn't generate them). +* Editing resource xml files. ### What doesn't work (yet) ([crbug](https://bugs.chromium.org/p/chromium/issues/detail?id=620034)) -* Make gradle aware of resources and assets +* Make gradle aware of assets +* Layout editor * Add a mode in which gradle is responsible for generating `R.java` * Add support for native code editing * Make the "Make Project" button work correctly diff --git a/chromium/docs/android_test_instructions.md b/chromium/docs/android_test_instructions.md index 5bd33bca79d..b26130cc6f4 100644 --- a/chromium/docs/android_test_instructions.md +++ b/chromium/docs/android_test_instructions.md @@ -206,13 +206,13 @@ run the test. ```shell # Build the test suite. -ninja -C out/Release chrome_junit_tests +ninja -C out/my_build chrome_junit_tests # Run the test suite. -build/android/test_runner.py junit -s chrome_junit_tests --release -vvv +BUILDTYPE=my_build build/android/test_runner.py junit -s chrome_junit_tests -vvv # Run a subset of tests. You might need to pass the package name for some tests. -build/android/test_runner.py junit -s chrome_junit_tests --release -vvv +BUILDTYPE=my_build build/android/test_runner.py junit -s chrome_junit_tests -vvv -f "org.chromium.chrome.browser.media.*" ``` diff --git a/chromium/docs/chrome_settings.md b/chromium/docs/chrome_settings.md index 69893c5b8f7..5628d56c922 100644 --- a/chromium/docs/chrome_settings.md +++ b/chromium/docs/chrome_settings.md @@ -1,8 +1,28 @@ -# What is chrome://settings? +# Deprecation + +This doc refers to "options", the soon-to-be-deprecated version of +chrome://settings being replaced by Material Design Settings. + +**Any new features and bug fixes should be contributed to MD Settings, found in +the /settings/ directories (as opposed to the /options/ directories).** + +See: https://www.chromium.org/developers/updating-webui-for-material-design + +Note: As the soft launch begins, chrome://settings may start surfacing the +Material Design settings page. The deprecated "options" page will be available +during this time at chrome://settings-frame, but will eventually be removed. + +# Outdated + +The remainder of this doc has not been substantially updated in some time, but +is generally correct. Contact /options/ OWNERS if you have questions, but see +the deprecation notice above. + +## What is chrome://settings? Chrome (version 10 and above) uses WebUI settings by default for all platforms. Access it via the wrench menu ("Preferences" on Mac and Linux; "Options" on -Windows and ChromeOS), or by typing chrome://settings into the address bar. +Windows and Chrome OS), or by typing chrome://settings into the address bar. One advantage of chrome://settings over platform-native dialogs is that it is shared by all platforms; therefore, it is easier to add new options UI and to diff --git a/chromium/docs/clang_static_analyzer.md b/chromium/docs/clang_static_analyzer.md index ed9feab99e5..dd79d0fc225 100644 --- a/chromium/docs/clang_static_analyzer.md +++ b/chromium/docs/clang_static_analyzer.md @@ -3,71 +3,20 @@ See the [official clang static analyzer page](http://clang-analyzer.llvm.org/) for background. -We don't run this regularly (because the analyzer's -[support for C++ isn't great yet](http://clang-analyzer.llvm.org/dev_cxx.html)), -so everything on this page is likely broken. The last time I checked, the -analyzer reported mostly uninteresting things. This assumes you're -[building chromium with clang](clang.md). +As of early 2017, we have experimental support for the Clang static analysis +tool in the Chrome build. Warnings generated by the analysis tool are logged +to stderr along with other compiler errors at build time. -You need an llvm checkout to get `scan-build` and `scan-view`; the easiest way -to get that is to run +To enable static analysis for your build, add the following line to your +output directory's `args.gn`: -```shell -tools/clang/scripts/update.py --force-local-build --without-android ``` - -## With make - -To build base, if you use the make build: - -``` -builddir_name=out_analyze \ -PATH=$PWD/third_party/llvm-build/Release+Asserts/bin:$PATH \ -third_party/llvm/tools/clang/tools/scan-build/scan-build \ - --keep-going --use-cc clang --use-c++ clang++ \ - make -j8 base -``` - -(`builddir_name` is set to force a clobber build.) - -Once that's done, run `third_party/llvm/tools/clang/tools/scan-view/scan-view` -to see the results; pass in the pass that `scan-build` outputs. - -## With ninja - -scan-build does its stuff by mucking with $CC/$CXX, which ninja ignores. gyp -does look at $CC/$CXX however, so you need to first run gyp\_chromium under -scan-build: - -```shell -time GYP_GENERATORS=ninja \ -GYP_DEFINES='component=shared_library clang_use_chrome_plugins=0 \ - mac_strip_release=0 dcheck_always_on=1' \ -third_party/llvm/tools/clang/tools/scan-build/scan-build \ - --use-analyzer $PWD/third_party/llvm-build/Release+Asserts/bin/clang \ - build/gyp_chromium -Goutput_dir=out_analyze -``` - -You then need to run the build under scan-build too, to get a HTML report: - -```shell -time third_party/llvm/tools/clang/tools/scan-build/scan-build \ - --use-analyzer $PWD/third_party/llvm-build/Release+Asserts/bin/clang \ - ninja -C out_analyze/Release/ base +use_clang_static_analyzer = true ``` -Then run `scan-view` as described above. - -## Known False Positives - -* http://llvm.org/bugs/show_bug.cgi?id=11425 - -## Stuff found by the static analyzer +The next time you rebuild, you should see static analysis warnings appear inline +with the usual Clang build warnings and errors. -* https://code.google.com/p/skia/issues/detail?id=399 -* https://code.google.com/p/skia/issues/detail?id=400 -* https://codereview.chromium.org/8308008/ -* https://codereview.chromium.org/8313008/ -* https://codereview.chromium.org/8308009/ -* https://codereview.chromium.org/10031018/ -* https://codereview.chromium.org/12390058/ +## Future plans/potential issues +* Support for running under GOMA is untested, but will be added shortly if + feasible. diff --git a/chromium/docs/code_reviews.md b/chromium/docs/code_reviews.md new file mode 100644 index 00000000000..6efe906166b --- /dev/null +++ b/chromium/docs/code_reviews.md @@ -0,0 +1,201 @@ +# Code Reviews + +Code reviews are a central part of developing high-quality code for Chromium. +All changes must be reviewed. + +The bigger patch-upload-and-land process is covered in more detail the +[contributing code](https://www.chromium.org/developers/contributing-code) +page. + +# Code review policies + +Ideally the reviewer is someone who is familiar with the area of code you are +touching. Any committer can review code, but an owner must provide a review +for each directory you are touching. If you have doubts, look at the git blame +for the file and the `OWNERS` files (see below). + +To indicate a positive review, the reviewer types "LGTM" (case insensitive) +into a comment on the code review. This stands for "Looks Good To Me." The text +"not LGTM" will cancel out a previous positive review. + +If you have multiple reviewers, make it clear in the message you send +requesting review what you expect from each reviewer. Otherwise people might +assume their input is not required or waste time with redundant reviews. + +#### Expectations for all reviewers + + * Aim to provide some kind of actionable response within 24 hours of receipt + (not counting weekends and holidays). This doesn't mean you have to have + done a complete review, but you should be able to give some initial + feedback, request more time, or suggest another reviewer. + + * It can be nice to indicate if you're away in your name in the code review + tool. If you do this, indicate when you'll be back. + + * Don't generally discourage people from sending you code reviews. This + includes writing a blanket ("slow") after your name in the review tool. + +## OWNERS files + +In various directories there are files named `OWNERS` that list the email +addresses of people qualified to review changes in that directory. You must +get a positive review from an owner of each directory your change touches. + +Owners files are recursive, so each file also applies to its subdirectories. +It's generally best to pick more specific owners. People listed in higher-level +directories may have less experience with the code in question. More detail on +the owners file format is provided in the "More information" section below. + +*Tip:* The `git cl owners` command can help find owners. + +While owners must approve all patches, any committer can contribute to the +review. In some directories the owners can be overloaded or there might be +people not listed as owners who are more familiar with the low-level code in +question. In these cases it's common to request a low-level review from an +appropriate person, and then request a high-level owner review once that's +complete. As always, be clear what you expect of each reviewer to avoid +duplicated work. + +Owners do not have to pick other owners for reviews. Since they should already +be familiar with the code in question, a thorough review from any appropriate +committer is sufficient. + +#### Expectations of owners + +The existing owners of a directory approve additions to the list. It is +preferrable to have many directories, each with a smaller number of specific +owners rather than large directories with many owners. Owners must: + + * Demonstrate excellent judgment, teamwork and ability to uphold Chrome + development principles. + + * Be already acting as an owner, providing high-quality reviews and design + feedback + + * Be a Chromium project member with full commit access of at least 6 + months tenure. + + * Have submitted a substantial number of non-trivial changes to the affected + directory. + + * Have committed or reviewed substantial work to the affected directory + within the last 90 days. + + * Have the bandwidth to contribute to reviews in a timely manner. If the load + is unsustainable, work to expand the number of owners. Don't try to + discourage people from sending reviews, including writing "slow" or + "emeritus" after your name. + +Seldom-updated directories may have exceptions. Directories in `third_party` +should list those most familiar with the library. + +## TBR ("To Be Reviewed") + +"TBR" is our mechanism for post-commit review. It should be used rarely and +only in cases where a review is unnecessary or as described below. The most +common use of TBR is to revert patches that broke the build. + +TBR does not mean "no review." A reviewer TBR-ed on a change should still +review the change. If there comments after landing the author is obligated to +address them in a followup patch. + +Do not use TBR just because a change is urgent or the reviewer is being slow. +Contact the reviewer directly or find somebody. + +To send a change TBR, annotate the description and send email like normal. +Otherwise the reviewer won't know to review the patch. + + * Add the reviewer's email address in the code review tool's reviewer field + like normal. + + * Add a line "TBR=<reviewer's email>" to the bottom of the change list + description. + + * Push the "send mail" button. + +### TBR-ing certain types of mechanical changes + +Sometimes you might do something that affects many callers in different +directories. For example, adding a parameter to a common function in //base. +If the updates to the callers is mechanical, you can: + + * Get a normal owner of the lower-level directory you're changing (in this + example, `//base`) to do a proper review of those changes. + + * Get _somebody_ to review the downstream changes. This is often the same + person from the previous step but could be somebody else. + + * Add the owners of the affected downstream directories as TBR. + +This process ensures that all code is reviewed prior to checkin and that the +concept of the change is reviewed by a qualified person, but you don't have to +track down many individual owners for trivial changes to their directories. + +### TBR-ing documentation updates + +You can TBR documentation updates. Documentation means markdown files, text +documents, and high-level comments in code. At finer levels of detail, comments +in source files become more like code and should be reviewed normally (not +using TBR). Non-TBR-able stuff includes things like function contracts and most +comments inside functions. + + * Use good judgement. If you're changing something very important, tricky, + or something you may not be very familiar with, ask for the code review + up-front. + + * Don't TBR changes to policy documents like the style guide or this document. + + * Don't mix unrelated documentation updates with code changes. + + * Be sure to actually send out the email for the code review. If you get one, + please actually read the changes. + +## More information + +### OWNERS file details + +Refer to the [source code](https://chromium.googlesource.com/chromium/tools/depot_tools/+/master/owners.py) +for all details on the file format. + +This example indicates that two people are owners, in addition to any owners +from the parent directory. `git cl owners` will list the comment after an +owner address, so this is a good place to include restrictions or special +instructions. +``` +# You can include comments like this. +a@chromium.org +b@chromium.org # Only for the frobinator. +``` + +A `*` indicates that all committers are owners: +``` +* +``` + +The text `set noparent` will stop owner propagation from parent directories. +This is used for specialized code. In this example, only the two listed people +are owners: +``` +set noparent +a@chromium.org +b@chromium.org +``` + +The `per-file` directive allows owners to be added that apply only to files +matching a pattern. In this example, owners from the parent directiory +apply, plus one person for some classes of files, and all committers are +owners for the readme: +``` +per-file foo_bar.cc=a@chromium.org +per-file foo.*=a@chromium.org + +per-file readme.txt=* +``` + +Other `OWNERS` files can be included by reference by listing the path to the +file with `file://...`. This example indicates that only the people listed in +`//ipc/SECURITY_OWNERS` can review the messages files: +``` +per-file *_messages*.h=set noparent +per-file *_messages*.h=file://ipc/SECURITY_OWNERS +``` diff --git a/chromium/docs/component_build.md b/chromium/docs/component_build.md index 06341c21cd3..650cd83fe5a 100644 --- a/chromium/docs/component_build.md +++ b/chromium/docs/component_build.md @@ -253,3 +253,12 @@ the object file might not be brought into the link. Even if it is brought in today, it might not be brought in due to completely unrelated changes in the future. The result will be undefined symbol errors from other components. Use source sets if your component is made up of more than one target. + +### Exporting functions and classes implemented in headers + +When you implement a symbol in a header the compiler will put that in every +necessary translation unit and the linker will pick one. If the symbol is never +referenced by code in the shared library it's supposed exported from, it will +never be instantiated and never exported. The result will be undefined external +symbol errors when linking. Exported symbols should be declared in a header but +always implemented in a .cc file. diff --git a/chromium/docs/documentation_best_practices.md b/chromium/docs/documentation_best_practices.md index 328a98f7f41..4e4df45b945 100644 --- a/chromium/docs/documentation_best_practices.md +++ b/chromium/docs/documentation_best_practices.md @@ -97,7 +97,7 @@ to detailed prose: design. However, once the code is implemented, design docs should serve as archives of these decisions, not as half-correct docs (they are often misused). See - [Implementation state](#implementation_state_determines_document_location) + [Implementation state](#Implementation-state-determines-document-repository) below. ## Implementation state determines document repository diff --git a/chromium/docs/documentation_guidelines.md b/chromium/docs/documentation_guidelines.md new file mode 100644 index 00000000000..14f2d4dd0ff --- /dev/null +++ b/chromium/docs/documentation_guidelines.md @@ -0,0 +1,90 @@ +# **Chromium Documentation Guidelines** + +Chromium's code base is large. Very large. Like most large places, it can be hard to find your way around if you don't already know the way. It also changes a lot. Lots of people work on Chromium and refactoring, componentization, addition +or removal of layers, etc. means that knowledge one does have can quickly get out of date. + +As a result, it can be hard to figure out how things are supposed to fit together. Documentation on [dev.chromium.org](http://dev.chromium.org/developers) can be hard to navigate and harder to keep up to date. The [starter guide](https://sites.google.com/a/chromium.org/dev/developers/how-tos/getting-around-the-chrome-source-code) is helpful, but stops at a very high-level overview. Ultimately [codesearch.chromium.org](http://codesearch.chromium.org) is the way most people explore. + +This guide attempts to lay out some practices that will make it easier for newcomers to find their way around Chromium's code and for old hands to keep up with a constantly evolving code base. It works from the principle that all documentation is wrong and out of date, but in-code documentation is less so and valuable. + +## Guidelines + +In-code documentation can be categorized into three tiers: + +* **Module / component documentation**: This is documentation that tends to be found in README.md files at the root level directory of a component. The purpose of this type of documentation is to explain what a given set of related classes are for, roughly how they are structured and *what* the component is expected to be used for. +* **Interface / class documentation**: This is documentation most often found in header files near class definitions. This may describe what a class or small set of classes are for and *how* they are intended to be used. It may also provide examples of usage of the class where this isn't obvious. +* **Implementation documentation**: This type of documentation is found next to implementations. In C++, this will tend to be embedded in .cc files. It is meant to help a reader understand *how the code works* or explain an implementation path that might not be obvious from the code itself. + +Let's dig in a bit to how to use these types of documentation in Chromium. + +### Module / component documentation + +Ok, so you're working on something big and important. Maybe it's a chunk of code that merits living in its own component in [src/components/](https://codesearch.chromium.org/chromium/src/components/) or even directly under [src/](https://codesearch.chromium.org/chromium/src/) itself. One day someone's going to come along and wonder what it's for and they'll discover that awesome-directory-name (breakpad, courgette, mojo, rutabaga, quick: which one of those was made up?) doesn't by itself shed a lot of light on what the code does. + +***Any self contained unit of code that merits a container directory should have a README.md file that describes what that component is, how it is expected to be used and provides rough outline of the code's structure.*** + +The README.md file should be located at the root of the component directory and should be in [markdown format](https://www.chromium.org/developers/markdown-documentation) according to the markdown [style guide](https://github.com/google/styleguide/blob/gh-pages/docguide/style.md). + +The first thing in the README.md file should be a description of the component in sufficient detail that someone unfamiliar with the code will understand why it exists. It should answer the questions "what does this component do?" and "is there any in particular I should know about it?". Some larger components (e.g. v8, mojo, etc.) may have additional up-to-date documentation on [dev.chromium.org](http://dev.chromium.org) or elsewhere makes sense too. + +The second thing in the README.md file should be an outline of how it is expected to be used. For components of small to moderate size (e.g. under [src/components/](https://codesearch.chromium.org/chromium/src/components/)), this can mean what are the main classes to care about. For larger components this can describe how to go about using the component, possibly in the form of links to external documentation (e.g.: [src/v8/](https://codesearch.chromium.org/chromium/src/v8/README.md)). + +The third thing should be a lay of the land of the component, sufficient breadcrumbs for a newcomer to be able to orient themselves. This usually means at least an explanation of the subdirectories of the component and possibly a description of the major interfaces a consumer of the component will need to care about. Again for larger components this can live in external kept-up-to-date documentation where needed. + +The fourth thing should be historical links to design docs for the component, including early ones. These are super handy for people looking to understand why the component exists in its current form. + +A great example of a README.md file is [src/native_client/README.md](https://codesearch.chromium.org/chromium/src/native_client/README.md). There are many more examples that are waiting to be made great. + +### Interface / class documentation + +Class and function declarations in header files describe to the world how a piece of code should be used. The public interface of a class or a header file containing function declarations describe the API to the code but don't necessarily say much about how it is expected to be used. As such: + +***Non-trivial classes or sets of function declarations in header files should include comments with a brief description of the purpose of the class or set of functions as well as examples of expected usage. *** + +(Good) examples: + +https://codesearch.chromium.org/chromium/src/base/callback_list.h + +https://codesearch.chromium.org/chromium/src/base/feature_list.h + +### Implementation documentation + +Code should be as self documenting as possible, but sometimes that isn't enough. Maybe there's a tricky bit of implementation that isn't obvious. Maybe there's a clever algorithm that has a good reason it needs to be clever. A good rule of thumb is that if a code reviewer had a clarifying question about something during review and the code couldn't be simplified to answer the question then a comment is probably appropriate. Don't go nuts here though, often if code is so complicated that it needs lots of comments, then it could probably be simplified first. + +## Language + +Be concise. + +Use correct grammar as much as possible. + +Avoid fancy or confusing language. + +Don't assume that readers know everything you currently know. + +## Working with existing code + +If you got this far and have some experience with Chromium's code, you'll have figured out that these guidelines are aspirational more than what the world looks like today. So what do we do when working with existing code. + +First off: ***[Documentation changes can be TBRed](https://chromium.googlesource.com/chromium/src/+/master/docs/code_reviews.md#TBR_ing-documentation-updates)***. Even in-code changes. If you have discovered something that isn't documented, have figured out how it works and would like to pay it forward, feel free to write something down and check it in. + +At the component level, if you are the owner of a component that isn't documented, please add a README.md with content as per the above. + +If you don't own a component and figure out how it works, consider adding some notes to the component's README.md file. You don't need to be an expert to share what you do know with those who come afterwards. + +The same applies to class level comments and implementation comments. If you find yourself modifying code and realizing that it is… under-documented.. feel free to add some notes nearby. + +In general, use your judgment. Changes will NOT be blocked on requiring people to add missing documentation, but adding missing docs is a great way to make our code base healthier. + +## FAQ + +#### Are these hard rules? + +[Well, no](https://www.youtube.com/watch?v=jl0hMfqNQ-g). This said, following these guidelines will make our code base healthier, more consistent and much easier to understand. Over time improving in-code documentation will make the whole team faster. But if a small fix is genuinely needed (it almost always isn't), then don't block on writing detailed documentation. As always: use your judgment. + +#### I'm not an expert on this undocumented code, should I still add documentation? + +Yes! Partial documentation is much better than no documentation. + +#### I hate writing documentation, it will slow me down! + +Chromium is big enough and complicated enough that a newcomer has to read a lot of code to figure out how things fit together. Documentation provides breadcrumbs to speed up understanding, which over time will make the whole team work more quickly. Short term execution speed is far less important than long term team velocity. diff --git a/chromium/docs/get_the_code.md b/chromium/docs/get_the_code.md index 0308a2dad4a..cc834b5622e 100644 --- a/chromium/docs/get_the_code.md +++ b/chromium/docs/get_the_code.md @@ -11,7 +11,7 @@ you might want to build: * [Android](android_build_instructions.md) * [Cast](old_cast_build_instructions.md) -* [ChromeOS](old_chromeos_build_instructions.md) +* [Chrome OS](old_chromeos_build_instructions.md) * [iOS](ios_build_instructions.md) * [Linux](linux_build_instructions.md) * [Mac](mac_build_instructions.md) diff --git a/chromium/docs/ios_build_instructions.md b/chromium/docs/ios_build_instructions.md index b23138d1a7f..2d7a013236f 100644 --- a/chromium/docs/ios_build_instructions.md +++ b/chromium/docs/ios_build_instructions.md @@ -1,6 +1,6 @@ # Checking out and building Chromium for iOS -There are instructions for other platforms linked from the +There are instructions for other platforms linked from the [get the code](get_the_code.md) page. ## Instructions for Google Employees @@ -16,10 +16,10 @@ Are you a Google employee? See * [Xcode](https://developer.apple.com/xcode) 8.0+. * The OS X 10.10 SDK. Run - ```shell + ```shell $ ls `xcode-select -p`/Platforms/MacOSX.platform/Developer/SDKs ``` - + to check whether you have it. Building with the 10.11 SDK works too, but the releases currently use the 10.10 SDK. * The current version of the JDK (required for the Closure compiler). @@ -81,7 +81,7 @@ development and testing purposes. Since the iOS build is a bit more complicated than a desktop build, we provide `ios/build/tools/setup-gn.py`, which will create four appropriately configured build directories under `out` for Release and Debug device and simulator -builds, and generates an appropriate Xcode workspace as well. +builds, and generates an appropriate Xcode workspace as well. This script is run automatically by fetch (as part of `gclient runhooks`). @@ -101,10 +101,129 @@ Note: you need to run `setup-gn.py` script every time one of the `BUILD.gn` file is updated (either by you or after rebasing). If you forget to run it, the list of targets and files in the Xcode solution may be stale. -You can also follow the manual instructions on the +You can also follow the manual instructions on the [Mac page](mac_build_instructions.md), but make sure you set the GN arg `target_os="ios"`. +## Building for device + +To be able to build and run Chromium and the tests for devices, you need to +have an Apple developer account (a free one will work) and the appropriate +provisioning profiles, then configure the build to use them. + +### Code signing identity + +Please refer to the Apple documentation on how to get a code signing identity +and certificates. You can check that you have a code signing identity correctly +installed by running the following command. + +```shell +$ xcrun security find-identity -v -p codesigning + 1) 0123456789ABCDEF0123456789ABCDEF01234567 "iPhone Developer: someone@example.com (XXXXXXXXXX)" + 1 valid identities found +``` + +If the command output says you have zero valid identities, then you do not +have a code signing identity installed and need to get one from Apple. If +you have more than one identity, the build system may select the wrong one +automatically, and you can use the `ios_code_signing_identity` gn variable +to control which one to use by setting it to the identity hash, e.g. to +`"0123456789ABCDEF0123456789ABCDEF01234567"`. + +### Mobile provisioning profiles + +Once you have the code signing identity, you need to decide on a prefix +for the application bundle identifier. This is controlled by the gn variable +`ios_app_bundle_id_prefix` and usually corresponds to a reversed domain name +(the default value is `"org.chromium"`). + +You then need to request provisioning profiles from Apple for your devices +for the following bundle identifiers to build and run Chromium with these +application extensions: + +- `${prefix}.chrome.ios.herebedragons` +- `${prefix}.chrome.ios.herebedragons.ShareExtension` +- `${prefix}.chrome.ios.herebedragons.TodayExtension` + +All these certificates need to have the "App Groups" +(`com.apple.security.application-groups`) capability enabled for +the following groups: + +- `group.${prefix}.chrome` +- `group.${prefix}.common` + +The `group.${prefix}.chrome` is only shared by Chromium and its extensions +to share files and configurations while the `group.${prefix}.common` is shared +with Chromium and other applications from the same organisation and can be used +to send commands to Chromium. + +### Mobile provisioning profiles for tests + +In addition to that, you need provisioning profiles for the individual test +suites that you want to run. Their bundle identifier depends on whether the +gn variable `ios_automatically_manage_certs` is set to true (the default) +or false. + +If set to true, then you just need a provisioning profile for the bundle +identifier `${prefix}.gtest.generic-unit-test` but you can only have a +single test application installed on the device (all the test application +will share the same bundle identifier). + +If set to false, then you need a different provisioning profile for each +test application. Those provisioning profile will have a bundle identifier +matching the following pattern `${prefix}.gtest.${test-suite-name}` where +`${test-suite-name}` is the name of the test suite with underscores changed +to dashes (e.g. `base_unittests` app will use `${prefix}.gest.base-unittests` +as bundle identifier). + +To be able to run the EarlGrey tests on a device, you'll need two provisioning +profiles for EarlGrey and OCHamcrest frameworks: + +- `${prefix}.test.OCHamcrest` +- `${prefix}.test.EarlGrey` + +In addition to that, then you'll need one additional provisioning profile for +the XCTest module too. This module bundle identifier depends on whether the +gn variable `ios_automatically_manage_certs` is set to true or false. If set +to true, then `${prefix}.test.gtest.generic-unit-test.generic-unit-test-module` +will be used, otherwise it will match the following pattern: +`${prefix}.test.${test-suite-name}.${test-suite-name}-module`. + +### Other applications + +Other applications like `ios_web_shell` usually will require mobile provisioning +profiles with bundle identifiers that may usually match the following pattern +`${prefix}.${application-name}` and may require specific capabilities. + +Generally, if the mobile provisioning profile is missing then the code signing +step will fail and will print the bundle identifier of the bundle that could not +be signed on the command line, e.g.: + +```shell +$ ninja -C out/Debug-iphoneos ios_web_shell +ninja: Entering directory `out/Debug-iphoneos' +FAILED: ios_web_shell.app/ios_web_shell ios_web_shell.app/_CodeSignature/CodeResources ios_web_shell.app/embedded.mobileprovision +python ../../build/config/ios/codesign.py code-sign-bundle -t=iphoneos -i=0123456789ABCDEF0123456789ABCDEF01234567 -e=../../build/config/ios/entitlements.plist -b=obj/ios/web/shell/ios_web_shell ios_web_shell.app +Error: no mobile provisioning profile found for "org.chromium.ios-web-shell". +ninja: build stopped: subcommand failed. +``` + +Here, the build is failing because there are no mobile provisioning profiles +installed that could sign the `ios_web_shell.app` bundle with the identity +`0123456789ABCDEF0123456789ABCDEF01234567`. To fix the build, you'll need to +request such a mobile provisioning profile from Apple. + +You can inspect the file passed via the `-e` flag to the `codesign.py` script +to check which capabilites are required for the mobile provisioning profile +(e.g. `src/build/config/ios/entitlements.plist` for the above build error, +remember that the paths are relative to the build directory, not to the source +directory). + +If the required capabilities are not enabled on the mobile provisioning profile, +then it will be impossible to install the application on a device (Xcode will +display an error stating that "The application was signed with invalid +entitlements"). + ## Running apps from the commandline Any target that is built and runs on the bots (see [below](#Troubleshooting)) diff --git a/chromium/docs/ios_infra.md b/chromium/docs/ios_infra.md new file mode 100644 index 00000000000..3ab0d7f299c --- /dev/null +++ b/chromium/docs/ios_infra.md @@ -0,0 +1,367 @@ +# Continuous build and test infrastructure for Chromium for iOS + +See the [instructions] for how to check out and build Chromium for iOS. + +The Chromium projects use buildbot for continuous integration. This doc starts +with an overview of the system, then gives detailed explanations about each +part. + +[TOC] + +## Overview + +Commits are made using the [commit queue], which triggers a series of try jobs +to compile and test the proposed patch against Chromium tip of tree before +actually making the commit. If the try jobs succeed the patch is committed. A +newly committed change triggers the builders (or "bots") to compile and test +the change again. + +## Bots + +Bots are slaves attached to a buildbot master (or "waterfall"). A buildbot +master is a server which polls for commits to a repository and triggers workers +to compile and test new commits whenever they are detected. [chromium.mac] is +the main waterfall for Mac desktop and iOS. [tryserver.chromium.mac] serves +as the try server for Mac desktop and iOS. + +The bots know how to check out a given revision of Chromium, compile, and test. + +### Code location + +#### Master configs + +The masters are configured in [tools/build], a separate repository which +contains various infra-related scripts. + +#### Pollers + +[chromium.mac] uses a `GitilesPoller` which polls the Chromium repository for +new commits using the [gitiles] interface. When a new commit is detected, the +bots are triggered. + +#### Recipes + +The bots run [recipes], which are scripts that specify their sequence of steps +located in [tools/build]. An iOS-specific [recipe module] contains common +functionality that the various [iOS recipes] use. + +#### Configs + +Because the recipes live in another repository, changes to the recipes don't +go through the Chromium [commit queue] and aren't tested on the [try server]. +In order to allow bot changes to be tested by the commit queue, the recipes +for iOS are generic instead of bot-specific, and rely on configuration files +which live in master-specific JSON config files located in [src/ios/build/bots]. +These configs define the `gn_args` to use during compilation as well as the +tests to run. + +#### Scripts + +The [test runner] is the script which installs and runs the tests, interprets +the results, and collects any files emitted by the test ("test data"). It can +be found in [src/ios/build/bots/scripts], which means changes to the test runner +can be tested on the [try server]. + +### Compiling with goma + +Goma is the distributed build system used by Chromium. It reduces compilation +time by avoiding recompilation of objects which have already been compiled +elsewhere. + +### Testing with swarming + +Tests run on [swarming], a distributed test system used by Chromium. After +compilation, configured tests will be zipped up along with their necessary +dependencies ("isolated") and sent to the [swarming server] for execution. The +server issues tasks to its attached workers for execution. The bots themselves +don't run any tests, they trigger tests to be run remotely on the swarming +server, then wait and display the results. This allows multiple tests to be +executed in parallel. + +## Try bots + +Try bots are bots which test proposed patches which are not yet committed. + +Request [try job access] in order to trigger try jobs against your patch. The +relevant try bots for an iOS patch are `ios-device`, `ios-device-xcode-clang`, +`ios-simulator`, and `ios-simulator-xcode-clang`. These bots can be found on +the Mac-specific [try server]. A try job is said to succeed when the build +passes (i.e. when the bot successfully compiles and tests the patch). + +`ios-device` and `ios-device-xcode-clang` both compile for the iOS device +architecture (ARM), and neither run any tests. A build is considered successful +so long as compilation is successful. + +`ios-simulator` and `ios-simulator-xcode-clang` both compile for the iOS +simulator architecture (x86), and run tests in the iOS [simulator]. A build is +considered successful when both compilation and all configured test succeed. + +`ios-device` and `ios-simulator` both compile using the version of [clang] +defined by the `CLANG_REVISION` in the Chromium tree. + +`ios-device-xcode-clang` and `ios-simulator-xcode-clang` both compile using the +version of clang that ships with Xcode. + +### Scheduling try jobs using buildbucket + +Triggering a try job and collecting its results is accomplished using +[buildbucket]. The service allows for build requests to be put into buckets. A +request in this context is a set of properties indicating things such as where +to get the patch. The try bots are set up to poll a particular bucket for build +requests which they execute and post the results of. + +### Compiling with the analyzer + +In addition to goma, the try bots use another time-saving mechanism called the +[analyzer] to determine the subset of compilation targets affected by the patch +that need to be compiled in order to run the affected tests. If a patch is +determined not to affect a certain test target, compilation and execution of the +test target will be skipped. + +## Configuring the bots + +See the [configs code location](#Configs) for where to find the config files for +the bots. The config files are JSON which describe how the bot should compile +and which tests it should run. The config files are located in the configs +directory. The configs directory contains a named directory for each master. For +example: +```shell +$ ls ios/build/bots +OWNERS scripts tests chromium.fyi chromium.mac +``` +In this case, configs are defined for iOS bots on [chromium.fyi] and +[chromium.mac]. Inside each master-specific directory are JSON config files +named after each bot. For example: +```shell +$ ls ios/build/bots/chromium.mac +ios-device.json ios-simulator.json +``` +The `ios-device` bot on [chromium.mac] will read its configuration from +`chromium.mac/ios-device.json` in the configs directory. + +### Example + +```json +{ + "comments": [ + "Sample config for a bot." + ], + "gn_args": [ + "is_debug=true", + "target_cpu=\"x64\"" + ], + "tests": [ + { + "app": "ios_chrome_unittests", + "device type": "iPhone 5s", + "os": "10.0", + "xcode version": "8.0" + } + ] +} +``` +The `comments` key is optional and defines a list of strings which can be used +to annotate the config. You may want to explain why the bot exists and what it's +doing, particularly if there are extensive and atypical `gn_args`. + +The `gn_args` key is a required list of arguments to pass to [GN] to generate +the build files. Two GN args are required, `is_debug` and `target_cpu`. Use +`is_debug` to define whether to compile for Debug or Release, and `target_cpu` +to define whether to compile for x86, x64, arm, or arm64. The iOS bots typically +perform Debug builds for x86 and x64, and Release builds for arm and arm64. An +x86/x64 build can only be tested on the [iOS simulator], while an arm/arm64 +build can only be tested on a physical device. + +Since Chromium for iOS is shipped as a [universal binary], it's also fairly +common to set `additional_target_cpus`. For simulator builds, we typically set: +```json +"gn_args": [ + "additional_target_cpus=[\"x86\"]", + "is_debug=true", + "target_cpu=\"x64\"" +] +``` +This builds universal binaries which run in 32-bit mode on 32-bit simulators and +64-bit mode on 64-bit simulators. For device builds we typically set: +```json +"gn_args": [ + "additional_target_cpus=[\"arm\"]", + "is_debug=false", + "target_cpu=\"arm64\"" +] +``` +In order to build universal binaries which run in 32-bit mode on 32-bit devices +and 64-bit mode on 64-bit devices. + +The `tests` key is an optional list of dictionaries defining tests to run. There +are two types of test dictionary, `app` and `include`. An `app` dict defines a +specific compiled app to run, for example: +```json +"tests": [ + { + "app": "ios_chrome_unittests", + "device type": "iPhone 5s", + "os": "10.0", + "xcode version": "8.0" + } +] +``` +This dict says to run `ios_chrome_unittests` on an `iPhone 5s` running iOS +`10.0` using Xcode `8.0`. A test dict may optionally define a list of `test +args`, which are arguments to pass directly to the test on the command line, +and it may define a boolean value `xctest` to indicate whether the test is an +[xctest] \(default if unspecified is `false`\). For example: +```json +"tests": [ + { + "app": "ios_chrome_unittests", + "device type": "iPhone 5s", + "os": "10.0", + "test args": [ + "--foo", + "--bar" + ], + "xcode version": "8.0" + }, + { + "app": "ios_chrome_integration_egtests", + "device type": "iPhone 5s", + "os": "10.0", + "xcode version": "8.0", + "xctest": true + } +] +``` +This defines two tests to run, first `ios_chrome_unittests` will be run with +`--foo` and `--bar` passed directly to the test on the command line. Next, +`ios_chrome_integration_egtests` will be run as an xctest. `"xctest": true` +must be specified for all xctests, it is an error to try and launch an xctest as +a regular test. + +An `include` dict defines a list of tests to import from the `tests` +subdirectory in the configs directory. For example: +```json +"tests": [ + { + "include": "common_tests.json", + "device type": "iPhone 5s", + "os": "10.0", + "xcode version": "8.0" + } +] +``` +This dict says to import the list of tests from the `tests` subdirectory and run +each one on an `iPhone 5s` running iOS `10.0` using Xcode `8.0`. Here's what +`common_tests.json` might look like: +```json +"tests": [ + { + "app": "ios_chrome_unittests" + }, + { + "app": "ios_net_unittests" + }, + { + "app": "ios_web_unittests" + }, +] +``` +Includes may contain other keys besides `app` which can then be omitted in the +bot config. For example if `common_tests.json` specifies: +```json +"tests": [ + { + "app": "ios_chrome_integration_egtests", + "xctest": true, + "xcode version": "8.0" + } +] +``` +Then the bot config may omit the `xctest` or `xcode version` keys, for example: +```json +{ + "comments": [ + "Sample config for a bot." + ], + "gn_args": [ + "is_debug=true", + "target_cpu=\"x64\"" + ], + "tests": [ + { + "include": "common_tests.json", + "device type": "iPhone 5s", + "os": "10.0" + } + ] +} +``` +Includes are not recursive, so `common_tests.json` may not itself include any +`include` dicts. + +### Uploading compiled artifacts from a bot + +A bot may be configured to upload compiled artifacts. This is defined by the +`upload` key. For example: +```json +{ + "comments": [ + "Sample config for a bot which uploads artifacts." + ], + "gn_args": [ + "is_debug=true", + "target_cpu=\"x64\"" + ], + "upload": [ + { + "artifact": "Chromium.breakpad", + "bucket": "my-gcs-bucket", + }, + { + "artifact": "Chromium.app", + "bucket": "my-gcs-bucket", + "compress": true, + }, + { + "artifact": "Chromium.breakpad", + "symupload": true, + } + ] +} +``` +After compilation, the bot will upload three artifacts. First the +`Chromium.breakpad` symbols will be uploaded to +`gs://my-gcs-bucket/<buildername>/<buildnumber>/Chromium.breakpad`. Next +`Chromium.app` will be tarred, gzipped, and uploaded to +`gs://my-gcs-bucket/<buildername>/<buildnumber>/Chromium.tar.gz`. Finally +the `Chromium.breakpad` symbols will be uploaded to the [breakpad] crash +reporting server where they can be used to symbolicate stack traces. + +If `artifact` is a directory, you must specify `"compress": true`. + +[analyzer]: ../tools/mb +[breakpad]: https://chromium.googlesource.com/breakpad/breakpad +[buildbucket]: https://cr-buildbucket.appspot.com +[chromium.fyi]: https://build.chromium.org/p/chromium.fyi/waterfall +[chromium.mac]: https://build.chromium.org/p/chromium.mac +[clang]: ../tools/clang +[commit queue]: https://dev.chromium.org/developers/testing/commit-queue +[gitiles]: https://gerrit.googlesource.com/gitiles +[GN]: ../tools/gn +[instructions]: ./ios_build_instructions.md +[iOS recipes]: https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/recipes/ios +[iOS simulator]: ../testing/iossim +[recipe module]: https://chromium.googlesource.com/chromium/tools/build/+/master/scripts/slave/recipe_modules/ios +[recipes]: https://chromium.googlesource.com/infra/infra/+/HEAD/doc/users/recipes.md +[simulator]: https://developer.apple.com/library/content/documentation/IDEs/Conceptual/iOS_Simulator_Guide/Introduction/Introduction.html +[src/ios/build/bots]: ../ios/build/bots +[src/ios/build/bots/scripts]: ../ios/build/bots/scripts +[swarming]: https://github.com/luci/luci-py/tree/master/appengine/swarming +[swarming server]: https://chromium-swarm.appspot.com +[test runner]: ../ios/build/bots/scripts/test_runner.py +[tools/build]: https://chromium.googlesource.com/chromium/tools/build +[try job access]: https://www.chromium.org/getting-involved/become-a-committer#TOC-Try-job-access +[try server]: https://build.chromium.org/p/tryserver.chromium.mac/waterfall +[tryserver.chromium.mac]: https://build.chromium.org/p/tryserver.chromium.mac/waterfall +[universal binary]: https://en.wikipedia.org/wiki/Universal_binary +[xctest]: https://developer.apple.com/reference/xctest diff --git a/chromium/docs/ios_voiceover.md b/chromium/docs/ios_voiceover.md new file mode 100644 index 00000000000..3f8a5cdf798 --- /dev/null +++ b/chromium/docs/ios_voiceover.md @@ -0,0 +1,48 @@ +# VoiceOver + +VoiceOver is an Accessibility feature available on Apple platforms. It provides +an interface between your app and your blind or low-vision users. In short, it's +a screen reader. + +## Overview + +[Activating VoiceOver](#Activate-VoiceOver) fundamentally changes the touch +interface of an iOS device, so when you activate it, you need to know these +basic commands to navigate: + +- VoiceOver focuses an item on the screen and speaks it to the user. +- To navigate between items, you can either flick left/right to go to the + previous/next item, or you can drag your finger around the screen. +- Double tapping anywhere on the screen activates the item (no need to double + tap exactly on the item on the screen). +- Items have a label (describing the item) and can have among others: + - traits (describing the item: a header, a button, selection state, etc.); + - a hint (describing how to interact with the item). + +The following video is a great summary. The content is succinct, high quality +and up-to-date, even though the video quality is low and the demo is on an +iPhone 3GS. + +[![Video screenshot]][Video link] + +## Activate VoiceOver + +Go to `Settings > General > Accessibility > VoiceOver` and toggle the top +switch. + +Alternatively, if you are activating/deactivating VoiceOver often, I'd advise +to set it as the Accessibility Shortcut. When enabled, a triple tap on the Home +button will toggle VoiceOver ON and OFF from anywhere. +``` +Settings > General > Accessibility > Accessibility Shortcut +``` + +## Additional Resources + +- [Apple presentation of VoiceOver] +- [AppleVis intro to VoiceOver] + +[Apple presentation of VoiceOver]: http://www.apple.com/accessibility/osx/voiceover/ +[AppleVis intro to VoiceOver]: http://www.applevis.com/guides/ios-miscellaneous-voiceover/intro-ios-accessibility-blind-and-low-vision-users +[Video screenshot]: https://img.youtube.com/vi/WxQ2qKShvmc/0.jpg +[Video link]: https://www.youtube.com/watch?v=WxQ2qKShvmc "iPhone VoiceOver Function For People With Disabilities" diff --git a/chromium/docs/linux_gtk_theme_integration.md b/chromium/docs/linux_gtk_theme_integration.md index 1fe912a26f4..caf0941993c 100644 --- a/chromium/docs/linux_gtk_theme_integration.md +++ b/chromium/docs/linux_gtk_theme_integration.md @@ -16,24 +16,18 @@ mostly a list of about 80 colors (see //src/ui/native_theme/native_theme.h) overridden by the theme. Chromium usually doesn't use GTK to render entire widgets, but instead tries to determine colors from them. -There are three types of colors Chromium needs from widgets: - -* Foreground color: determined by the CSS "color" property -* Background color: determined by the CSS "background-color" and "background-image" properties -* Border color: determined by the "border-color", "border-image", - "border-style", and "border-width" properties - -Backgrounds and borders are complicated because in general they might have -multiple gradients or images. To get the color, Chromium uses GTK to render the -background or border into a single pixel and uses the resulting color for -theming. This mostly gives reasonable results, but in case theme authors do not -like the resulting color, they have the option to theme Chromium widgets -specially. +Chromium needs foreground, background and border colors from widgets. The +foreground color is simply taken from the CSS "color" property. Backgrounds and +borders are complicated because in general they might have multiple gradients or +images. To get the color, Chromium uses GTK to render the background or border +into a 24x24 bitmap and uses the average color for theming. This mostly gives +reasonable results, but in case theme authors do not like the resulting color, +they have the option to theme Chromium widgets specially. ## Note to GTK theme authors: How to theme Chromium widgets Every widget Chromium uses will have a "chromium" style class added to it. For -example, a texfield selector might look like: +example, a textfield selector might look like: ``` .window.background.chromium .entry.chromium @@ -59,11 +53,6 @@ they might use: For GTK3.20 or later, themes will as usual have to replace ".entry" with "entry". -Additional requirements for border colors to be picked up: - -* Must have a border-style that is not none. -* Must have a border-width that is nonzero. - The list of CSS selectors that Chromium uses to determine its colors is in //src/chrome/browser/ui/libgtkui/native_theme_gtk3.cc. diff --git a/chromium/docs/linux_minidump_to_core.md b/chromium/docs/linux_minidump_to_core.md index 930d905743d..7f25c88d449 100644 --- a/chromium/docs/linux_minidump_to_core.md +++ b/chromium/docs/linux_minidump_to_core.md @@ -104,5 +104,5 @@ Then use gdb as normal. For more discussion on this process see [Debugging a Minidump](https://www.chromium.org/chromium-os/how-tos-and-troubleshooting/crash-reporting/debugging-a-minidump). -This page discusses the same process in the context of ChromeOS and many of the +This page discusses the same process in the context of Chrome OS and many of the concepts and techniques overlap. diff --git a/chromium/docs/linux_sandboxing.md b/chromium/docs/linux_sandboxing.md index dfbdf3ac210..2bd09a2459a 100644 --- a/chromium/docs/linux_sandboxing.md +++ b/chromium/docs/linux_sandboxing.md @@ -41,11 +41,11 @@ network, something that is difficult to perform at the layer-2. | **Name** | **Layer and process** | **Linux flavors where available** | **State** | |:---------|:----------------------|:----------------------------------|:----------| -| [Setuid sandbox](#The_setuid_sandbox.md) | Layer-1 in Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient), some utility processes) | Linux distributions and Chrome OS | Enabled by default (old kernels) and maintained | -| [User namespaces sandbox](#User_namespaces_sandbox.md) | Modern alternative to the setuid sandbox. Layer-1 in Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient), some utility processes) | Linux distributions and Chrome OS (kernel >= 3.8) | Enabled by default (modern kernels) and actively developed | -| [Seccomp-BPF](#The_seccomp-bpf_sandbox.md) | Layer-2 in some Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient)), Layer-1 + Layer-2 in GPU process | Linux kernel >= 3.5, Chrome OS and Ubuntu | Enabled by default and actively developed | -| [Seccomp-legacy](#The_seccomp_sandbox.md) | Layer-2 in renderers | All | [Deprecated](https://src.chromium.org/viewvc/chrome?revision=197301&view=revision) | -| [SELinux](#SELinux.md) | Layer-1 in Zygote processes (renderers, PPAPI) | SELinux distributions | [Deprecated](https://src.chromium.org/viewvc/chrome?revision=200838&view=revision) | +| [Setuid sandbox](#The-setuid-sandbox) | Layer-1 in Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient), some utility processes) | Linux distributions and Chrome OS | Enabled by default (old kernels) and maintained | +| [User namespaces sandbox](#User-namespaces-sandbox) | Modern alternative to the setuid sandbox. Layer-1 in Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient), some utility processes) | Linux distributions and Chrome OS (kernel >= 3.8) | Enabled by default (modern kernels) and actively developed | +| [Seccomp-BPF](#The-sandbox-1) | Layer-2 in some Zygote processes (renderers, PPAPI, [NaCl](https://www.chromium.org/nativeclient)), Layer-1 + Layer-2 in GPU process | Linux kernel >= 3.5, Chrome OS and Ubuntu | Enabled by default and actively developed | +| [Seccomp-legacy](#The-sandbox-2) | Layer-2 in renderers | All | [Deprecated](https://src.chromium.org/viewvc/chrome?revision=197301&view=revision) | +| [SELinux](#SELinux) | Layer-1 in Zygote processes (renderers, PPAPI) | SELinux distributions | [Deprecated](https://src.chromium.org/viewvc/chrome?revision=200838&view=revision) | | AppArmor | Outer layer-1 in Zygote processes (renderers, PPAPI) | Not used | Deprecated | ## The setuid sandbox diff --git a/chromium/docs/linux_sublime_dev.md b/chromium/docs/linux_sublime_dev.md index e80aa2533b8..1bbe4ffbbda 100644 --- a/chromium/docs/linux_sublime_dev.md +++ b/chromium/docs/linux_sublime_dev.md @@ -249,6 +249,19 @@ and hence is not useful on third_party projects that use another style. 2. Select some text and press `Ctrl + Shift + C` to format, or select no text to format the entire file +## CodeSearch Integration with Chromium X-Refs + +With [Chromium X-Refs](https://github.com/karlinjf/ChromiumXRefs/) you can +perform [https://cs.chromium.org](https://cs.chromium.org) cross-reference +searches in your editor. This gives you the call graph, overrides, references, +declaration, and definition of most of the code. The results are as fresh as +the search engine's index so uncomitted changes won't be reflected. + +More information on Chromium X-Ref's functionality (including keyboard and +mouse shortcuts) can be found on the [Chromium X-Refs +page](https://github.com/karlinjf/ChromiumXRefs/). + + ## Code Completion with SublimeClang (Linux Only) SublimeClang is a powerful autocompletion plugin for Sublime that uses the Clang diff --git a/chromium/docs/linux_zygote.md b/chromium/docs/linux_zygote.md index 5504115d04c..da7c841460f 100644 --- a/chromium/docs/linux_zygote.md +++ b/chromium/docs/linux_zygote.md @@ -3,10 +3,20 @@ and forks itself in response. Generally they are used because forking a process after some expensive setup has been performed can save time and share extra memory pages. -On Linux, for Chromium, this is not the point, and measurements suggest that the -time and memory savings are minimal or negative. +More specifically, on Linux, it allows to: + * Amortize the runtime and memory cost of the dynamic loader's relocations, + which is respectively ~6 MB and 60 ms/GHz per process. + See [Appendix A](#appendix-a-runtime-impact-of-relocations) and + [Appendix B](#appendix-b-memory-impact-of-relocations). + * Amortize the runtime and memory cost for initializing common + libraries, such as ICU, NSS, the V8 snapshot and anything else in + `ContentMainRunnerImpl::Initialize()`. With the above, this saves + up to ~8 MB per process. See [Appendix C](#appendix-c-overall-memory-impact). -We use it because it's the only reasonable way to keep a reference to a binary +Security-wise, the Zygote is responsible for setting up and bookkeeping the +[namespace sandbox](linux_sandboxing.md). + +Furthermore it is the only reasonable way to keep a reference to a binary and a set of shared libraries that can be exec'ed. In the model used on Windows and Mac, renderers are exec'ed as needed from the chrome binary. However, if the chrome binary, or any of its shared libraries are updated while Chrome is @@ -34,3 +44,37 @@ Signaling the zygote for a new renderer happens in You can use the `--zygote-cmd-prefix` flag to debug the zygote process. If you use `--renderer-cmd-prefix` then the zygote will be bypassed and renderers will be exec'ed afresh every time. + +## Appendix A: Runtime impact of relocations +Measured on a Z620: + + $ LD_DEBUG=statistics /opt/google/chrome-beta/chrome --help + runtime linker statistics: + total startup time in dynamic loader: 73899158 clock cycles + time needed for relocation: 56836478 clock cycles (76.9%) + number of relocations: 4271 + number of relocations from cache: 11347 + number of relative relocations: 502740 + time needed to load objects: 15789844 clock cycles (21.3%) + +56836478 clock cycles -> ~56 ms/GHz + +## Appendix B: Memory impact of relocations + + $ readelf -WS /opt/google/chrome-beta/chrome + [Nr] Name Type Address Off Size ES Flg Lk Inf Al + ... + [25] .data.rel.ro PROGBITS 0000000006a8b590 6a8a590 5b5500 00 WA 0 0 16 + ... + Note: 0x5b5500 -> 5.98 MB + +Actual impact in terms of memory pages that get shared due to CoW: + + $ cat /proc/.../smaps + 7fbdd1c81000-7fbdd2233000 r--p 06a5d000 fc:00 665771 /opt/google/chrome-unstable/chrome + ... + Shared_Dirty: 5796 kB + +## Appendix C: Overall memory impact + $ cat /proc/$PID_OF_ZYGOTE/smaps | grep Shared_Dirty | awk '{TOTAL += $2} END {print TOTAL}' + 8092 # KB for dirty pages shared with other processes (mostly forked child processes). diff --git a/chromium/docs/mac_build_instructions.md b/chromium/docs/mac_build_instructions.md index 5acb6c9f45a..646b64b6332 100644 --- a/chromium/docs/mac_build_instructions.md +++ b/chromium/docs/mac_build_instructions.md @@ -12,7 +12,7 @@ Are you a Google employee? See ## System requirements -* A 64-bit Mac running 10.9+. +* A 64-bit Mac running 10.11+. * [Xcode](https://developer.apple.com/xcode) 7.3+. * The OS X 10.10 SDK. Run diff --git a/chromium/docs/old_chromeos_build_instructions.md b/chromium/docs/old_chromeos_build_instructions.md index 45f936c4d90..5baa5150bfd 100644 --- a/chromium/docs/old_chromeos_build_instructions.md +++ b/chromium/docs/old_chromeos_build_instructions.md @@ -1,4 +1,4 @@ -# ChromeOS Build Instructions (Chromium OS on Linux) +# Chrome OS Build Instructions (Chromium OS on Linux) Chromium on Chromium OS is built from a mix of code sourced from Chromium on Linux and Chromium on Windows. Much of the user interface code is diff --git a/chromium/docs/optimizing_web_uis.md b/chromium/docs/optimizing_web_uis.md new file mode 100644 index 00000000000..d1b0c5dcbde --- /dev/null +++ b/chromium/docs/optimizing_web_uis.md @@ -0,0 +1,126 @@ +# Optimizing Chrome Web UIs + +## How do I do it? + +In order to build with a fast configuration, try setting these options in your +GN args: + +``` +use_vulcanize = true +is_debug = false +``` + +## How is the code optimized? + +### Resource combination + +[HTML imports](https://www.html5rocks.com/en/tutorials/webcomponents/imports/) +are a swell technology, but can be used is slow ways. Each import may also +contain additional imports, which must be satisfied before certain things can +continue (i.e. script execution may be paused). + +```html +<!-- If a.html contains more imports... --> +<link rel="import" href="a.html"> +<!-- This script is blocked until done. --> +<script> startThePageUp(); </script> +``` + +To reduce this latency, Chrome uses a tool created by the Polymer project named +[vulcanize](https://github.com/Polymer/polymer-bundler/tree/1.x). It processes +a page starting from a URL entry point and inlines resources the first time +they're encountered. This greatly decreases latency due to HTML imports. + +```html +<!-- Contents of a.html and all its dependencies. --> +<script> startThePageUp(); </script> +``` + +### CSS @apply to --var transformation + +We also use +[polymer-css-build](https://github.com/PolymerLabs/polymer-css-build) to +transform CSS @apply mixins (which are not yet natively supported) into faster +--css-variables. This turns something like this: + +```css +:host { + --mixin-name: { + color: red; + display: block; + }; +} +/* In a different place */ +.red-thing { + @apply(--mixin-name); +} +``` + +into the more performant: + +```css +:host { + --mixin-name_-_color: red; + --mixin-name_-_display: block; +} +/* In a different place */ +.red-thing { + color: var(--mixin-name_-_color); + display: var(--mixin-name_-_display); +} +``` + +### JavaScript Minification + +In order to minimize disk size, we run +[uglifyjs](https://github.com/mishoo/UglifyJS2) on all combined JavaScript. This +reduces installer and the size of resources required to load to show a UI. + +Code like this: + +```js +function fizzBuzz() { + for (var i = 1; i <= 100; i++) { + var fizz = i % 3 == 0 ? 'fizz' : ''; + var buzz = i % 5 == 0 ? 'buzz' : ''; + console.log(fizz + buzz || i); + } +} +fizzBuzz(); +``` + +would be minified to: + +```js +function fizzBuzz(){for(var z=1;100>=z;z++){var f=z%3==0?"fizz":"",o=z%5==0?"buzz":"";console.log(f+o||z)}}fizzBuzz(); +``` + +If you'd like to more easily debug minified code, click the "{}" prettify button +in Chrome's developer tools, which will beautify the code and allow setting +breakpoints on the un-minified version. + +### Gzip compression of web resources + +In certain cases, it might be preferable to leave web resources compressed on +disk and inflate them when needed (i.e. when a user wants to see a page). + +In this case, you can run `gzip --rsyncable` on a resource before it's put into +a .pak file via GRIT with this syntax: + +```xml +<include name="IDR_MY_PAGE" file="my/page.html" type="BINDATA" compress="gzip" /> +``` + +Gzip is currently set up to apply to a whole WebUI's data source, though it's +possible to exclude specific paths for things like dynamically generated content +(i.e. many pages load translations dynamically from a path named "strings.js"). + +To mark a WebUI's resources compressed, you'll need to do something like: + +```c++ +WebUIDataSource* data_source = WebUIDataSource::Create(...); +data_source->SetDefaultResource(IDR_MY_PAGE); +std::unordered_set<std::string> exclusions; +exclusions.insert("strings.js"); // if required +data_source->UseGzip(exclusions); +``` diff --git a/chromium/docs/origin_trials_integration.md b/chromium/docs/origin_trials_integration.md new file mode 100644 index 00000000000..d56431a8fb0 --- /dev/null +++ b/chromium/docs/origin_trials_integration.md @@ -0,0 +1,180 @@ +# Integrating a feature with the Origin Trials framework + +To expose your feature via the origin trials framework, there are a few code +changes required. + +[TOC] + +## Code Changes + +### Runtime Enabled Features + +First, you’ll need to configure [RuntimeEnabledFeatures.json5]. This is +explained in the file, but you use `origin_trial_feature_name` to associate your +runtime feature flag with a name for your origin trial. The name can be the +same as your runtime feature flag, or different. Eventually, this configured +name will be used in the Origin Trials developer console (still under +development). You can have both `status: experimental` and +`origin_trial_feature_name` if you want your feature to be enabled either by +using the `--enable-experimental-web-platform-features` flag **or** the origin +trial. + +You may have a feature that is not available on all platforms, or need to limit +the trial to specific platforms. Use `origin_trial_os: [list]` to specify which +platforms will allow the trial to be enabled. The list values are case- +insensitive, but must match one of the defined `OS_<platform>` macros (see +[build_config.h]). + +#### Examples + +Flag name and trial name are the same: +``` +{ + name: "MyFeature", + origin_trial_feature_name: "MyFeature", + status: "experimental", +}, +``` +Flag name and trial name are different: +``` +{ + name: "MyFeature", + origin_trial_feature_name: "MyFeatureTrial", + status: "experimental", +}, +``` +Trial limited to specific platform: +``` json +{ + name: "MyFeature", + origin_trial_feature_name: "MyFeature", + origin_trial_os: ["android"], + status: "experimental", +}, +``` + +### Gating Access + +Once configured, there are two mechanisms to gate access to your feature behind +an origin trial. You can use either mechanism, or both, as appropriate to your +feature implementation. + +1. A native C++ method that you can call in Blink code at runtime to expose your + feature: `bool OriginTrials::myFeatureEnabled()` +2. An IDL attribute \[[OriginTrialEnabled]\] that you can use to automatically + expose and hide JavaScript methods/attributes/objects. This attribute works + very similar to \[RuntimeEnabled\]. +``` +[OriginTrialEnabled=MyFeature] +partial interface Navigator { + readonly attribute MyFeatureManager myFeature; +} +``` + +**NOTE:** Your feature implementation must not persist the result of the enabled +check. Your code should simply call `OriginTrials::myFeatureEnabled()` as often +as necessary to gate access to your feature. + +### IDL Bindings + +When using the \[OriginTrialEnabled\] IDL attribute, you'll need to manually +install the appropriate methods in the V8 bindings code. Based on the +\[OriginTrialEnabled\] attribute, there will be one or more `installMyFeature` +methods generated in the bindings code. These methods must be manually +installed: + +- Find the relevant methods by doing a code search for `installMyFeature`: + - Search within the generated bindings code, e.g. 'f:gen/blink/bindings' + - Example search results: `V8WindowPartial::installMyFeature`, + `V8NavigatorPartial::installMyFeature`, `V8<type>::installMyFeature` +- Determine which bindings code needs to be updated: + - [ConditionalFeatures.cpp]: Your feature lives in `core` (i.e. generated + methods are found under .../bindings/core/...) + - [ConditionalFeaturesForModules.cpp]: Your feature lives under `modules` + (i.e. generated methods are found under .../bindings/modules/...) +- Update `installConditionalFeatures[Core|ForModules]()`: + - These methods are broken down by type. + - Add/update the logic for each type to call the corresponding + `V8<type>::installMyFeature()` methods. +- Update `installPendingConditionalFeature[Core|ForModules]()`: + - These methods are broken down by trial/feature. + - Add/update the logic for each feature to call all of the + `installMyFeature()` methods. + +Eventually, the V8 bindings code will be generated automatically (See +[crbug.com/615060]). + + +## Limitations + +What you can't do, because of the nature of these Origin Trials, is know at +either browser or renderer startup time whether your feature is going to be used +in the current page/context. This means that if you require lots of expensive +processing to begin (say you index the user's hard drive, or scan an entire city +for interesting weather patterns,) that you will have to either do it on browser +startup for *all* users, just in case it's used, or do it on first access. (If +you go with first access, then only people trying the experiment will notice the +delay, and hopefully only the first time they use it.). We are investigating +providing a method like `OriginTrials::myFeatureShouldInitialize()` that will +hint if you should do startup initialization. For example, this could include +checks for trials that have been revoked (or throttled) due to usage, if the +entire origin trials framework has been disabled, etc. The method would be +conservative and assume initialization is required, but it could avoid expensive +startup in some known scenarios. + +Similarly, if you need to know in the browser process whether a feature should +be enabled, then you will have to either have the renderer inform it at runtime, +or else just assume that it's always enabled, and gate access to the feature +from the renderer. + +## Testing + +If you want to test your code's interactions with the framework, you'll need to +generate some tokens of your own. To generate your own tokens, use +[generate_token.py]. You can generate signed tokens for localhost, or for +127.0.0.1, or for any origin that you need to help you test. For example: + +``` +tools/origin_trials/generate_token.py http://localhost:8000 MyFeature +``` + +The file `tools/origin_trials/eftest.key` is used by default as the private key +for the test keypair used by Origin Trials unit tests and layout tests (i.e. in +content shell). Tokens generated with this key will **not** work in the browser +by default (see the [Developer Guide] for instructions on creating real tokens). +To use a test token with the browser, run Chrome with the command-line flag: + +``` +--origin-trial-public-key=dRCs+TocuKkocNKa0AtZ4awrt9XKH2SQCI6o4FY6BNA= +``` + +This is the base64 encoding of the public key associated with `eftest.key`. If +it doesn't work, see [trial_token_unittest.cc]. If you cannot set command-line +switches (e.g., on Chrome OS), you can also directly modify +[chrome_origin_trial_policy.cc]. + +### Layout Tests +When using the \[OriginTrialEnabled\] IDL attribute, you should add layout tests +to verify that you've correctly updated the V8 bindings code. Depending on how +your feature is exposed, you'll want tests for the exposed interfaces, as well +as tests for script-added tokens. For examples, refer to the existing tests in +[origin_trials/webexposed]. + +In addition, you'll need to update the command line for the +`origin-trials-runtimeflags-disabled` virtual suite, in [VirtualTestSuites]. +Add your feature name to the list of explicitly disabled features. This test +suite ensures that features are actually enabled by origin trial, not overridden +by the runtime flag. + +[build_config.h]: /build/build_config.h +[chrome_origin_trial_policy.cc]: /chrome/common/origin_trials/chrome_origin_trial_policy.cc +[crbug.com/615060]: https://bugs.chromium.org/p/chromium/issues/detail?id=615060 +[ConditionalFeatures.cpp]: /third_party/WebKit/Source/bindings/core/v8/ConditionalFeatures.cpp +[ConditionalFeaturesForModules.cpp]: /third_party/WebKit/Source/bindings/modules/v8/ConditionalFeaturesForModules.cpp +[generate_token.py]: /tools/origin_trials/generate_token.py +[Developer Guide]: https://github.com/jpchase/OriginTrials/blob/gh-pages/developer-guide.md +[OriginTrialEnabled]: /third_party/WebKit/Source/bindings/IDLExtendedAttributes.md#_OriginTrialEnabled_i_m_a_c_ +[origin_trials/webexposed]: /third_party/WebKit/LayoutTests/http/tests/origin_trials/webexposed/ +[RuntimeEnabledFeatures.json5]: /third_party/WebKit/Source/platform/RuntimeEnabledFeatures.json5 +[trial_token_unittest.cc]: /content/common/origin_trials/trial_token_unittest.cc +[VirtualTestSuites]: /third_party/WebKit/LayoutTests/VirtualTestSuites diff --git a/chromium/docs/ozone_overview.md b/chromium/docs/ozone_overview.md index 3d070392075..2ba7f29f388 100644 --- a/chromium/docs/ozone_overview.md +++ b/chromium/docs/ozone_overview.md @@ -118,7 +118,7 @@ through creating a new ozone platform called `foo`. ## Building with Ozone -### ChromeOS - ([waterfall](https://build.chromium.org/p/chromium.chromiumos/waterfall?builder=Linux+ChromiumOS+Ozone+Builder&builder=Linux+ChromiumOS+Ozone+Tests+%281%29&builder=Linux+ChromiumOS+Ozone+Tests+%282%29&reload=none)) +### Chrome OS - ([waterfall](https://build.chromium.org/p/chromium.chromiumos/waterfall?builder=Linux+ChromiumOS+Ozone+Builder&builder=Linux+ChromiumOS+Ozone+Tests+%281%29&builder=Linux+ChromiumOS+Ozone+Tests+%282%29&reload=none)) To build `chrome`, do this from the `src` directory: @@ -224,9 +224,9 @@ content_shell --ozone-platform=headless \ This is Linux direct rending with acceleration via mesa GBM & linux DRM/KMS (EGL/GLES2 accelerated rendering & modesetting in GPU process) and is in -production use on [ChromeOS](https://www.chromium.org/chromium-os). +production use on [Chrome OS](https://www.chromium.org/chromium-os). -Note that all ChromeOS builds of Chrome will compile and attempt to use this. +Note that all Chrome OS builds of Chrome will compile and attempt to use this. See [Building Chromium for Chromium OS](https://www.chromium.org/chromium-os/how-tos-and-troubleshooting/building-chromium-browser) for build instructions. ### Cast diff --git a/chromium/docs/qtcreator.md b/chromium/docs/qtcreator.md new file mode 100644 index 00000000000..932cc093c3f --- /dev/null +++ b/chromium/docs/qtcreator.md @@ -0,0 +1,120 @@ +# Use Qt Creator as IDE or GUI Debugger + +[Qt Creator](https://www.qt.io/ide/) +([Wiki](https://en.wikipedia.org/wiki/Qt_Creator)) is a cross-platform C++ IDE. + +You can use Qt Creator as a daily IDE or just as a GDB frontend and that does +not require project configuration. + +[TOC] + +## IDE + +### Workflow + +1. `ctrl+k` Activate Locator, you can open file(not support sublime-like-search) or type `. ` go to symbol. +2. `ctrl+r` Build and Run, `F5` Debug. +3. `F4` switch between header file and cpp file. +4. `ctrl+shift+r` rename symbol under cursor. +5. Code completion is built-in. And you can add your snippets. + +### Setup as IDE + +1. Install latest Qt Creator +2. under chromium/src `gn gen out/Default --ide=qtcreator` +3. qtcreator out/Default/qtcreator_project/all.creator + +It takes 3 minutes to parsing C++ files in my workstation!!! And It will not +block you while parsing. + +#### Code Style + +1. Help - About Plugins enable Beautifier. +2. Tools - Options - Beautifier - Clang Format, select use predefined style: +chromium. You can also set a keyboard shortcut for it. +3. Tools - Options - Code Style import this xml file + +``` +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE QtCreatorCodeStyle> +<!-- Written by QtCreator 4.2.1, 2017-02-08T19:07:34. --> +<qtcreator> + <data> + <variable>CodeStyleData</variable> + <valuemap type="QVariantMap"> + <value type="bool" key="AlignAssignments">true</value> + <value type="bool" key="AutoSpacesForTabs">false</value> + <value type="bool" key="BindStarToIdentifier">false</value> + <value type="bool" key="BindStarToLeftSpecifier">false</value> + <value type="bool" key="BindStarToRightSpecifier">false</value> + <value type="bool" key="BindStarToTypeName">true</value> + <value type="bool" + key="ExtraPaddingForConditionsIfConfusingAlign">true</value> + <value type="bool" key="IndentAccessSpecifiers">true</value> + <value type="bool" key="IndentBlockBody">true</value> + <value type="bool" key="IndentBlockBraces">false</value> + <value type="bool" key="IndentBlocksRelativeToSwitchLabels">false</value> + <value type="bool" key="IndentClassBraces">false</value> + <value type="bool" key="IndentControlFlowRelativeToSwitchLabels">true</value> + <value type="bool" + key="IndentDeclarationsRelativeToAccessSpecifiers">false</value> + <value type="bool" key="IndentEnumBraces">false</value> + <value type="bool" key="IndentFunctionBody">true</value> + <value type="bool" key="IndentFunctionBraces">false</value> + <value type="bool" key="IndentNamespaceBody">false</value> + <value type="bool" key="IndentNamespaceBraces">false</value> + <value type="int" key="IndentSize">2</value> + <value type="bool" key="IndentStatementsRelativeToSwitchLabels">true</value> + <value type="bool" key="IndentSwitchLabels">false</value> + <value type="int" key="PaddingMode">2</value> + <value type="bool" key="ShortGetterName">true</value> + <value type="bool" key="SpacesForTabs">true</value> + <value type="int" key="TabSize">2</value> + </valuemap> + </data> + <data> + <variable>DisplayName</variable> + <value type="QString">chrome</value> + </data> +</qtcreator> +``` + +#### Build & Run + +In left panel, projects - setup the ninja command in build and clean step and +executable chrome path in run. + +## Debugger + +**You can skip the project settings and use QtCreator as a single file +standalone GDB frontend. ** + +1. Tools - Options - Build & Run - Debuggers, make sure GDB is set. +2. Tools - Options - Kits, change the Desktop kit to GDB(LLDB doesnot work in + Linux). +3. Open file you want to debug. +4. Debug - Start Debugging - Attach to running Application, you may need to + open chrome's task manager to find the process number. + +### Tips, tricks, and troubleshooting + +#### Make GDB start fast + +Add `gdb_index = true` to `gn args`. + +#### Debugger shows start then finish + +``` +$ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope +``` + +Ensure yama allow you to attach another process. + +#### Debugger do not stop in break point + +Ensure you are using GDB not LLDB in Linux. + +#### More + +See +https://chromium.googlesource.com/chromium/src/+/master/docs/linux_debugging.md
\ No newline at end of file diff --git a/chromium/docs/sync/model_api.md b/chromium/docs/sync/model_api.md new file mode 100644 index 00000000000..eebf34d8a09 --- /dev/null +++ b/chromium/docs/sync/model_api.md @@ -0,0 +1,297 @@ +# Chrome Sync's Model API + +Chrome Sync operates on discrete, explicitly defined model types (bookmarks, +preferences, tabs, etc). These model types are individually responsible for +implementing their own local storage and responding to remote changes. This +guide is for developers interested in syncing data for their model type to the +cloud using Chrome Sync. It describes the newest version of the API, known as +Unified Sync and Storage (USS). There is also the deprecated [SyncableService +API] (aka Directory), which as of early 2016 is still used by most model types. + +[SyncableService API]: https://www.chromium.org/developers/design-documents/sync/syncable-service-api + +[TOC] + +## Overview + +To correctly sync data, USS requires that sync metadata be stored alongside your +model data in a way such that they are written together atomically. **This is +very important!** Sync must be able to update the metadata for any local data +changes as part of the same write to disk. If you attempt to write data to disk +and only notify sync afterwards, a crash in between the two writes can result in +changes being dropped and never synced to the server, or data being duplicated +due to being committed more than once. + +[`ModelTypeSyncBridge`][Bridge] is the interface the model code must implement. +The bridge tends to be either a [`KeyedService`][KeyedService] or owned by one. +The correct place for the bridge generally lies as close to where your model +data is stored as possible, as the bridge needs to be able to inject metadata +updates into any local data changes that occur. + +The bridge has access to a [`ModelTypeChangeProcessor`][MTCP] object, which it +uses to communicate local changes to sync using the `Put` and `Delete` methods. +The processor will communicate remote changes from sync to the bridge using the +`ApplySyncChanges` method. [`MetadataChangeList`][MCL] is the way sync will +communicate metadata changes to the storage mechanism. Note that it is typically +implemented on a per-storage basis, not a per-type basis. + +[Bridge]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_sync_bridge.h +[KeyedService]: https://cs.chromium.org/chromium/src/components/keyed_service/core/keyed_service.h +[MTCP]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_change_processor.h +[MCL]: https://cs.chromium.org/chromium/src/components/sync/model/metadata_change_list.h + +## Data + +### Specifics + +Model types will define a proto that contains the necessary fields of the +corresponding native type (e.g. [`TypedUrlSpecifics`][TypedUrlSpecifics] +contains a URL and a list of visit timestamps) and include it as a field in the +generic [`EntitySpecifics`][EntitySpecifics] proto. This is the form that all +communications with sync will use. This proto form of the model data is referred +to as the specifics. + +[TypedUrlSpecifics]: https://cs.chromium.org/chromium/src/components/sync/protocol/typed_url_specifics.proto +[EntitySpecifics]: https://cs.chromium.org/search/?q="message+EntitySpecifics"+file:sync.proto + +### Identifiers + +There are two primary identifiers for entities: **storage key** and **client +tag**. The bridge will need to take an [`EntityData`][EntityData] object (which +contains the specifics) and be able generate both of these from it. For +non-legacy types without significant performance concerns, these will generally +be the same. + +The storage key is used to uniquely identify entities locally within a client. +It’s what’s used to refer to entities most of the time and, as its name implies, +the bridge needs to be able to look up local data and metadata entries in the +store using it. Because it is a local identifier, it can change as part of +database migrations, etc. This may be desirable for efficiency reasons. + +The client tag is used to generate the **client tag hash**, which will identify +entities **across clients**. This means that its implementation can **never +change** once entities have begun to sync, without risking massive duplication +of entities. This means it must be generated using only immutable data in the +specifics. If your type does not have any immutable fields to use, you will need +to add one (e.g. a GUID, though be wary as they have the potential to conflict). +While the hash gets written to disk as part of the metadata, the tag itself is +never persisted locally. + +[EntityData]: https://cs.chromium.org/chromium/src/components/sync/model/entity_data.h + +## Storage + +A crucial requirement of USS is that the model must add support for keeping +sync’s metadata in the same storage as its normal data. The metadata consists of +one [`EntityMetadata`][EntityMetadata] proto for each data entity, and one +[`ModelTypeState`][ModelTypeState] proto containing metadata pertaining to the +state of the entire type (the progress marker, for example). This typically +requires two extra tables in a database to do (one for each type of proto). + +Since the processor doesn’t know anything about the store, the bridge provides +it with an implementation of the [`MetadataChangeList`][MCL] interface. The +change processor writes metadata through this interface when changes occur, and +the bridge simply has to ensure it gets passed along to the store and written +along with the data changes. + +[EntityMetadata]: https://cs.chromium.org/chromium/src/components/sync/protocol/entity_metadata.proto +[ModelTypeState]: https://cs.chromium.org/chromium/src/components/sync/protocol/model_type_state.proto + +### ModelTypeStore + +While the model type may store its data however it chooses, many types use +[`ModelTypeStore`][Store], which was created specifically to provide a +convenient persistence solution. It’s backed by a [LevelDB] to store serialized +protos to disk. `ModelTypeStore` provides two `MetadataChangeList` +implementations for convenience; both accessed via +[`ModelTypeStore::WriteBatch`][WriteBatch]. One passes metadata changes directly +into an existing `WriteBatch` and another caches them in memory until a +`WriteBatch` exists to consume them. + +The store interface abstracts away the type and will handle setting up tables +for the type’s data, so multiple `ModelTypeStore` objects for different types +can share the same LevelDB backend just by specifying the same path and task +runner. Sync already has a backend it uses for DeviceInfo that can be shared by +other types via the +[`ProfileSyncService::GetModelTypeStoreFactory`][StoreFactory] method. + +[Store]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_store.h +[LevelDB]: http://leveldb.org/ +[WriteBatch]: https://cs.chromium.org/search/?q="class+WriteBatch"+file:model_type_store.h +[StoreFactory]: https://cs.chromium.org/search/?q=GetModelTypeStoreFactory+file:profile_sync_service.h + +## Implementing ModelTypeSyncBridge + +### Initialization + +The bridge is required to load all of the metadata for its type from storage and +provide it to the processor via the [`ModelReadyToSync`][ModelReadyToSync] +method **before any local changes occur**. This can be tricky if the thread the +bridge runs on is different from the storage mechanism. No data will be synced +with the server if the processor is never informed that the model is ready. + +Since the tracking of changes and updating of metadata is completely +independent, there is no need to wait for the sync engine to start before +changes can be made. This prevents the need for an expensive association step in +the initialization. + +[ModelReadyToSync]: https://cs.chromium.org/search/?q=ModelReadyToSync+file:/model_type_change_processor.h + +### MergeSyncData + +This method is called only once, when a type is first enabled. Sync will +download all the data it has for the type from the server and provide it to the +bridge using this method. Sync filters out any tombstones for this call, so +`EntityData::is_deleted()` will never be true for the provided entities. The +bridge must then examine the sync data and the local data and merge them +together: + +* Any remote entities that don’t exist locally must be be written to local + storage. +* Any local entities that don’t exist remotely must be provided to sync via + [`ModelTypeChangeProcessor::Put`][Put]. +* Any entities that appear in both sets must be merged and the model and sync + informed accordingly. Decide which copy of the data to use (or a merged + version or neither) and update the local store and sync as necessary to + reflect the decision. How the decision is made can vary by model type. + +The [`MetadataChangeList`][MCL] passed into the function is already populated +with metadata for all the data passed in (note that neither the data nor the +metadata have been committed to storage yet at this point). It must be given to +the processor for any `Put` or `Delete` calls so the relevant metadata can be +added/updated/deleted, and then passed to the store for persisting along with +the data. + +Note that if sync gets disabled and the metadata cleared, entities that +originated from other clients will exist as “local” entities the next time sync +starts and merge is called. Since tombstones are not provided for merge, this +can result in reviving the entity if it had been deleted on another client in +the meantime. + +[Put]: https://cs.chromium.org/search/?q=Put+file:/model_type_change_processor.h + +### ApplySyncChanges + +While `MergeSyncData` provides the state of sync data using `EntityData` +objects, `ApplySyncChanges` provides changes to the state using +[`EntityChange`][EntityChange] objects. These changes must be applied to the +local state. + +Here’s an example implementation of a type using `ModelTypeStore`: + +```cpp +base::Optional<ModelError> DeviceInfoSyncBridge::ApplySyncChanges( + std::unique_ptr<MetadataChangeList> metadata_change_list, + EntityChangeList entity_changes) { + std::unique_ptr<WriteBatch> batch = store_->CreateWriteBatch(); + for (const EntityChange& change : entity_changes) { + if (change.type() == EntityChange::ACTION_DELETE) { + batch->DeleteData(change.storage_key()); + } else { + batch->WriteData(change.storage_key(), + change.data().specifics.your_type().SerializeAsString()); + } + } + + batch->TransferMetadataChanges(std::move(metadata_change_list)); + store_->CommitWriteBatch(std::move(batch), base::Bind(...)); + NotifyModelOfChanges(); + return {}; +} +``` + +A conflict can occur when an entity has a pending local commit when an update +for the same entity comes from another client. In this case, the bridge’s +[`ResolveConflict`][ResolveConflict] method will have been called prior to the +`ApplySyncChanges` call in order to determine what should happen. This method +defaults to having the remote version overwrite the local version unless the +remote version is a tombstone, in which case the local version wins. + +[EntityChange]: https://cs.chromium.org/chromium/src/components/sync/model/entity_change.h +[ResolveConflict]: https://cs.chromium.org/search/?q=ResolveConflict+file:/model_type_sync_bridge.h + +### Local changes + +The [`ModelTypeChangeProcessor`][MTCP] must be informed of any local changes via +its `Put` and `Delete` methods. Since the processor cannot do any useful +metadata tracking until `MergeSyncData` is called, the `IsTrackingMetadata` +method is provided. It can be checked as an optimization to prevent unnecessary +processing preparing the parameters to a `Put` or `Delete` call. + +Here’s an example of handling a local write using `ModelTypeStore`: + +```cpp +void WriteLocalChange(std::string key, ModelData data) { + std::unique_ptr<WriteBatch> batch = store_->CreateWriteBatch(); + if (change_processor()->IsTrackingMetadata()) { + change_processor()->Put(key, ModelToEntityData(data), + batch->GetMetadataChangeList()); + } + batch->WriteData(key, specifics->SerializeAsString()); + store_->CommitWriteBatch(std::move(batch), base::Bind(...)); +} +``` + +## Error handling + +If any errors occur during store operations that could compromise the +consistency of the data and metadata, the processor’s +[`ReportError`][ReportError] method should be called. The only exception to this +is errors during `MergeSyncData` or `ApplySyncChanges`, which should just return +a [`ModelError`][ModelError]. + +This will inform sync of the error, which will stop all communications with the +server so bad data doesn’t get synced. Since the metadata might no longer be +valid, the bridge will asynchronously receive a `DisableSync` call (this is +implemented by the abstract base class; subclasses don’t need to do anything). +All the metadata will be cleared from the store (if possible), and the type will +be started again from scratch on the next client restart. + +[ReportError]: https://cs.chromium.org/search/?q=ReportError+file:/model_type_change_processor.h +[ModelError]: https://cs.chromium.org/chromium/src/components/sync/model/model_error.h + +## Sync Integration Checklist + +* Define your specifics proto in [`//components/sync/protocol/`][protocol]. +* Add a field for it to [`EntitySpecifics`][EntitySpecifics]. +* Add it to the [`ModelType`][ModelType] enum and + [`kModelTypeInfoMap`][info_map]. +* Add it to the [proto value conversions][conversions] files. +* Register a [`ModelTypeController`][ModelTypeController] for your type in + [`ProfileSyncComponentsFactoryImpl::RegisterDataTypes`][RegisterDataTypes]. +* Tell sync how to access your `ModelTypeSyncBridge` in + [`ChromeSyncClient::GetSyncBridgeForModelType`][GetSyncBridge]. +* Add to the [start order list][kStartOrder]. +* Add an field for encrypted data to [`NigoriSpecifics`][NigoriSpecifics]. +* Add to two encrypted types translation functions in + [`nigori_util.cc`][nigori_util]. +* Add a [preference][pref_names] for tracking whether your type is enabled. +* Map your type to the pref in [`GetPrefNameForDataType`][GetPrefName]. +* Check whether you should be part of a [pref group][RegisterPrefGroup]. +* Add to the `SyncModelTypes` enum and `SyncModelType` suffix in + [`histograms.xml`][histograms]. +* Add to the [`SYNC_DATA_TYPE_HISTOGRAM`][DataTypeHistogram] macro. + +[protocol]: https://cs.chromium.org/chromium/src/components/sync/protocol/ +[ModelType]: https://cs.chromium.org/chromium/src/components/sync/base/model_type.h +[info_map]: https://cs.chromium.org/search/?q="kModelTypeInfoMap%5B%5D"+file:model_type.cc +[conversions]: https://cs.chromium.org/chromium/src/components/sync/protocol/proto_value_conversions.h +[ModelTypeController]: https://cs.chromium.org/chromium/src/components/sync/driver/model_type_controller.h +[RegisterDataTypes]: https://cs.chromium.org/search/?q="ProfileSyncComponentsFactoryImpl::RegisterDataTypes" +[GetSyncBridge]: https://cs.chromium.org/search/?q=GetSyncBridgeForModelType+file:chrome_sync_client.cc +[kStartOrder]: https://cs.chromium.org/search/?q="kStartOrder[]" +[NigoriSpecifics]: https://cs.chromium.org/chromium/src/components/sync/protocol/nigori_specifics.proto +[nigori_util]: https://cs.chromium.org/chromium/src/components/sync/syncable/nigori_util.cc +[pref_names]: https://cs.chromium.org/chromium/src/components/sync/base/pref_names.h +[GetPrefName]: https://cs.chromium.org/search/?q=::GetPrefNameForDataType+file:sync_prefs.cc +[RegisterPrefGroup]: https://cs.chromium.org/search/?q=::RegisterPrefGroups+file:sync_prefs.cc +[histograms]: https://cs.chromium.org/chromium/src/tools/metrics/histograms/histograms.xml +[DataTypeHistogram]: https://cs.chromium.org/chromium/src/components/sync/base/data_type_histogram.h + +## Testing + +The [`TwoClientUssSyncTest`][UssTest] suite is probably a good place to start +for integration testing. Especially note the use of a `StatusChangeChecker` to +wait for events to happen. + +[UssTest]: https://cs.chromium.org/chromium/src/chrome/browser/sync/test/integration/two_client_uss_sync_test.cc diff --git a/chromium/docs/sync/uss/shared_model_type_processor.md b/chromium/docs/sync/uss/shared_model_type_processor.md new file mode 100644 index 00000000000..46bc5a1114e --- /dev/null +++ b/chromium/docs/sync/uss/shared_model_type_processor.md @@ -0,0 +1,114 @@ +# SharedModelTypeProcessor + +The [`SharedModelTypeProcessor`][SMTP] is a crucial piece of the USS codepath. +It lives on the model thread and performs the tracking of sync metadata for the +[`ModelTypeSyncBridge`][MTSB] that owns it by implementing the +[`ModelTypeChangeProcessor`][MTCP] interface, as well as sending commit requests +to the [`ModelTypeWorker`][MTW] on the sync thread via the [`CommitQueue`][CQ] +interface and receiving updates from the same worker via the +[`ModelTypeProcessor`][MTP] interface. + +[SMTP]: https://cs.chromium.org/chromium/src/components/sync/model_impl/shared_model_type_processor.h +[MTSB]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_sync_bridge.h +[MTCP]: https://cs.chromium.org/chromium/src/components/sync/model/model_type_change_processor.h +[MTW]: https://cs.chromium.org/chromium/src/components/sync/engine_impl/model_type_worker.h +[CQ]: https://cs.chromium.org/chromium/src/components/sync/engine/commit_queue.h +[MTP]: https://cs.chromium.org/chromium/src/components/sync/engine/model_type_processor.h + +[TOC] + +## Lifetime + +The bridge owns a processor object at all times and operates on the same thread +as it. If sync is disabled, the processor is destroyed but a new one is +immediately created to replace it. + +## Processor State Machines + +The processor sits between the model bridge and the sync engine. It has +knowledge of what state each is in based on the calls it has receieved and +performed. The states are not stored explicitly, but are implicit based on +state stored in the processor. Here are the states of each, with notes on their +transitions and how to determine them. + +### Model States + +* `UNREADY` + * Waiting for `ModelReadyToStart` to be called. + * Determined by: `waiting_for_metadata_ && !model_error_` +* `NEEDS_DATA` + * Waiting for data for pending commits to be loaded. + * This state is skipped if there are no pending commits. + * Determined by: `waiting_for_pending_data_ && !model_error_` +* `READY` + * The model is completely ready to sync. + * Determined by: `!waiting_for_metadata_ && !waiting_for_pending_data && + !model_error` +* `ERROR` + * Something in the model or storage broke. + * This state is permanent until DisableSync destroys the object. + * Determined by: `!!model_error_` + +### Sync States + +* `DISCONNECTED` + * Sync for this type has not started. + * This state can be re-entered from any other state if Disconnect is + called. + * Determined by: `!error_handler_`. +* `STARTED` + * Sync has started but the model is not yet `READY` (or `ERROR`). + * This state is skipped if the model is ready before sync is. + * Determined by: `error_handler_ && start_callback_` +* `CONNECT_PENDING` + * Both the model and sync are ready. The start callback has been called + and we're waiting to connect to the sync thread. + * If the model was `ERROR`, the error is passed along and the callback is + cleared; we're really waiting for DisableSync instead of connect. + * Determined by: `error_handler_ && !start_callback_` +* `CONNECTED` + * We have a [`CommitQueue`][CQ] that passes changes to the + [`ModelTypeWorker`][MTW] on the sync thread. + * Determined by: `!!worker_` + +### Processor States + +Based on the interplay of the model and sync states, the processor effectively +progresses through 3 states worth noting: + +* `UNINITIALIZED` + * Metadata isn't loaded so we have no knowledge of entities. + * `Put` and `Delete` calls are not allowed in this state (will DCHECK). +* `NOT_TRACKING` + * Indicates that not metadata is being tracked and that `Put` and `Delete` + calls will be ignored. + * This state is entered if the loaded metadata shows an initial merge + hasn't happened (`ModelTypeState::initial_sync_done` is false). + * Exposed via `IsTrackingMetadata` for optimization, not correctness. +* `TRACKING` + * Indicates that metadata is being tracked and `Put` and `Delete` calls + must happen for entity changes. + * This state is entered if the loaded metadata shows an initial merge + has happened (`ModelTypeState::initial_sync_done` is true). +* `SYNCING` + * Indicates that commits can be sent and updates can be received from the + sync server. This is a superstate of `TRACKING`. + * If the processor was in `TRACKING`, it progresses to this state as soon + as it gets connected to the worker. + * If the processor was in `NOT_TRACKING`, it progresses to this state + after `MergeSyncData` is called and the metadata is initialized. + +## Entity Tracker + +The [`ProcessorEntityTracker`][PET] tracks the state of individual entities for +the processor. It keeps the [`EntityMetadata`][EM] proto in memory, as well as +any pending commit data until it gets acked by the server. It also stores the +special `commit_requested_sequence_number_`, which tracks the sequence number of +the last version that's been sent to the server. + +The tracker holds the metadata in memory forever, which is needed so we know +what to update the on-disk memory with when we get a new local or remote change. +Changing this would require being able to handle updates asynchronously. + +[PET]: https://cs.chromium.org/chromium/src/components/sync/model_impl/processor_entity_tracker.h +[EM]: https://cs.chromium.org/chromium/src/components/sync/protocol/entity_metadata.proto diff --git a/chromium/docs/testing/layout_test_expectations.md b/chromium/docs/testing/layout_test_expectations.md index 746cadbc784..53545442206 100644 --- a/chromium/docs/testing/layout_test_expectations.md +++ b/chromium/docs/testing/layout_test_expectations.md @@ -163,11 +163,6 @@ removes the `[ NeedsRebaseline ]` entry from TestExpectations. since they're clearly not getting fixed anytime soon. * [W3CImportExpectations](../../third_party/WebKit/LayoutTests/W3CImportExpectations): A record of which W3C tests should be imported or skipped. -* [WPTServeExpectations](../../third_party/WebKit/LayoutTests/WPTServeExpectations): - Expectations for tests that fail differently when run under the W3C's wptserve - HTTP server with the `--enable-wptserve flag`. This is an experimental feature - at this time. - ### Flag-specific expectations files diff --git a/chromium/docs/testing/layout_tests.md b/chromium/docs/testing/layout_tests.md index d4be954bded..c6d57e271e3 100644 --- a/chromium/docs/testing/layout_tests.md +++ b/chromium/docs/testing/layout_tests.md @@ -184,10 +184,8 @@ A row of dots in the script's output indicates one or more tests that passed. ## Test expectations The -[TestExpectations](../../WebKit/LayoutTests/TestExpectations) file (and related -files, including -[skia_test_expectations.txt](../../skia/skia_test_expectations.txt)) -contains the list of all known layout test failures. See the +[TestExpectations](../../third_party/WebKit/LayoutTests/TestExpectations) file (and related +files) contains the list of all known layout test failures. See the [Layout Test Expectations documentation](./layout_test_expectations.md) for more on this. @@ -209,7 +207,7 @@ There are two ways to run layout tests with additional command-line arguments: suppressions in this file override the main TestExpectations file. * Using a *virtual test suite* defined in - [LayoutTests/VirtualTestSuites](https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/LayoutTests/VirtualTestSuites). + [LayoutTests/VirtualTestSuites](../../third_party/WebKit/LayoutTests/VirtualTestSuites). A virtual test suite runs a subset of layout tests under a specific path with additional flags. For example, you could test a (hypothetical) new mode for repainting using the following virtual test suite: @@ -324,10 +322,10 @@ finding the problem. * If you're lucky, your test is one that runs properly when you navigate to it in content_shell normally. In that case, build the Debug content_shell project, fire it up in your favorite debugger, and load the test file either - from a file:// URL. + from a `file:` URL. * You'll probably be starting and stopping the content_shell a lot. In VS, to save navigating to the test every time, you can set the URL to your - test (file: or http:) as the command argument in the Debugging section of + test (`file:` or `http:`) as the command argument in the Debugging section of the content_shell project Properties. * If your test contains a JS call, DOM manipulation, or other distinctive piece of code that you think is failing, search for that in the Chrome @@ -338,7 +336,7 @@ finding the problem. * If your test only works in full layout-test mode, or if you find it simpler to debug without all the overhead of an interactive session, start the content_shell with the command-line flag `--run-layout-test`, followed by the - URL (file: or http:) to your test. More information about running layout tests + URL (`file:` or `http:`) to your test. More information about running layout tests in content_shell can be found [here](./layout_tests_in_content_shell.md). * In VS, you can do this in the Debugging section of the content_shell project Properties. @@ -424,7 +422,7 @@ machine? * Open `http://localhost:9222` in a stable/beta/canary Chrome, click the single link to open the devtools with the test loaded. * You may need to replace devtools.html with inspector.html in your URL (or you - can use local chrome inspection of content_shell from chrome://inspect + can use local chrome inspection of content_shell from `chrome://inspect` instead) * In the loaded devtools, set any required breakpoints and execute `test()` in the console to actually start the test. @@ -464,11 +462,10 @@ test is being re-baselined. If this is a special case (i.e., something we've decided to be different with upstream), please put a README file next to the new expected output explaining the difference. -## W3C Tests +## web-platform-tests -In addition to layout tests developed and run just by the Blink team, there are -also W3C conformance tests. For more info, see -[Importing the W3C Tests](https://www.chromium.org/blink/importing-the-w3c-tests). +In addition to layout tests developed and run just by the Blink team, there is +also a shared test suite, see [web-platform-tests](./web_platform_tests.md). ## Known Issues diff --git a/chromium/docs/testing/layout_tests_tips.md b/chromium/docs/testing/layout_tests_tips.md new file mode 100644 index 00000000000..cd50bcc60e6 --- /dev/null +++ b/chromium/docs/testing/layout_tests_tips.md @@ -0,0 +1,301 @@ +# Layout Tests Tips + +The recommendations here are intended to help you write new tests that go +through code review with a minimal number of round trips, remain useful as Blink +evolves, and serve as an asset (rather than a liability) for the team. + +While reading existing layout tests, please keep in mind that they represent +snapshots taken over many years of an ever-evolving collective opinion of what +good Web pages and solid tests should look like. Thus, it should not come as a +surprise that most existing layout tests are not consistent with these +recommendations, and are not even consistent with each other. + +*** note +This document intentionally uses _should_ a lot more than _must_, as defined in +[RFC 2119](https://www.ietf.org/rfc/rfc2119.txt). Writing layout tests is a +careful act of balancing many concerns, and this humble document cannot possibly +capture the context that rests in the head of an experienced Blink engineer. +*** + +## General Principles + +This section contains guidelines adopted from +[web-platform-tests documentation](http://web-platform-tests.org/writing-tests/general-guidelines.html) +and +[WebKit's Wiki page on Writing good test cases](https://trac.webkit.org/wiki/Writing%20Layout%20Tests%20for%20DumpRenderTree), +with Blink-specific flavoring. + +### Concise + +Tests should be **concise**, without compromising on the principles below. Every +element and piece of code on the page should be necessary and relevant to what +is being tested. For example, don't build a fully functional signup form if you +only need a text field or a button. + +Content needed to satisfy the principles below is considered necessary. For +example, it is acceptable and desirable to add elements that make the test +self-describing (see below), and to add code that makes the test more reliable +(see below). + +Content that makes test failures easier to debug is considered necessary (to +maintaining a good development speed), and is both acceptable and desirable. + +*** promo +Conciseness is particularly important for reference tests and pixel tests, as +the test pages are rendered in an 800x600px viewport. Having content outside the +viewport is undesirable because the outside content does not get compared, and +because the resulting scrollbars are platform-specific UI widgets, making the +test results less reliable. +*** + +### Fast + +Tests should be as **fast** as possible, without compromising on the principles +below. Blink has several thousand layout tests that are run in parallel, and +avoiding unnecessary delays is crucial to keeping our Commit Queue in good +shape. + +Avoid +[window.setTimeout](https://developer.mozilla.org/en-US/docs/Web/API/WindowTimers/setTimeout), +as it wastes time on the testing infrastructure. Instead, use specific event +handlers, such as +[window.onload](https://developer.mozilla.org/en-US/docs/Web/API/GlobalEventHandlers/onload), +to decide when to advance to the next step in a test. + +### Reliable + +Tests should be **reliable** and yield consistent results for a given +implementation. Flaky tests slow down your fellow developers' debugging efforts +and the Commit Queue. + +`window.setTimeout` is again a primary offender here. Asides from wasting time +on a fast system, tests that rely on fixed timeouts can fail when on systems +that are slower than expected. + +When adding or significantly modifying a layout test, use the command below to +assess its flakiness. While not foolproof, this approach gives you some +confidence, and giving up CPU cycles for mental energy is a pretty good trade. + +```bash +third_party/WebKit/Tools/Scripts/run-webkit-tests path/to/test.html --repeat-each=100 +``` + +The +[PSA on writing reliable layout tests](https://docs.google.com/document/d/1Yl4SnTLBWmY1O99_BTtQvuoffP8YM9HZx2YPkEsaduQ/edit). +also has good guidelines for writing reliable tests. + +### Self-Describing + +Tests should be **self-describing**, so that a project member can recognize +whether a test passes or fails without having to read the specification of the +feature being tested. + +`testharness.js` makes a test self-describing when used correctly. Other types +of tests, such as reference tests and +[tests with manual fallback](./layout_tests_with_manual_fallback.md), +[must be carefully designed](http://web-platform-tests.org/writing-tests/manual.html#requirements-for-a-manual-test) +to be self-describing. + +### Minimal + +Tests should require a **minimal** amount of cognitive effort to read and +maintain. + +Avoid depending on edge case behavior of features that aren't explicitly covered +by the test. For example, except where testing parsing, tests should contain +valid markup (no parsing errors). + +Tests should provide as much relevant information as possible when failing. +`testharness.js` tests should prefer +[rich assert_ functions](https://github.com/w3c/testharness.js/blob/master/docs/api.md#list-of-assertions) +to combining `assert_true()` with a boolean operator. Using appropriate +`assert_` functions results in better diagnostic output when the assertion +fails. + +### Cross-Platform + +Tests should be as **cross-platform** as reasonably possible. Avoid assumptions +about device type, screen resolution, etc. Unavoidable assumptions should be +documented. + +When possible, tests should only use Web platform features, as specified +in the relevant standards. When the Web platform's APIs are insufficient, +tests should prefer to use WPT extended testing APIs, such as +`wpt_automation`, over Blink-specific testing APIs. + +Test pages should use the HTML5 doctype (`<!doctype html>`) unless they +specifically cover +[quirks mode](https://developer.mozilla.org/docs/Quirks_Mode_and_Standards_Mode) +behavior. + +Tests should avoid using features that haven't been shipped by the +actively-developed major rendering engines (Blink, WebKit, Gecko, Edge). When +unsure, check [caniuse.com](http://caniuse.com/). By necessity, this +recommendation does not apply to the feature targeted by the test. + +*** note +It may be tempting have a test for a bleeding-edge feature X depend on feature +Y, which has only shipped in beta / development versions of various browsers. +The reasoning would be that all browsers that implement X will have implemented +Y. Please keep in mind that Chrome has un-shipped features that made it to the +Beta channel in the past. +*** + +*** aside +[ES2015](http://benmccormick.org/2015/09/14/es5-es6-es2016-es-next-whats-going-on-with-javascript-versioning/) +is shipped by all major browsers under active development (except for modules), +so using ES2015 features is acceptable. + +At the time of this writing, ES2016 is not fully shipped in all major browsers. +*** + +### Self-Contained + +Tests must be **self-contained** and not depend on external network resources. + +Unless used by multiple test files, CSS and JavaScript should be inlined using +`<style>` and `<script>` tags. Content shared by multiple tests should be +placed in a `resources/` directory near the tests that share it. See below for +using multiple origins in a test. + +### File Names + +Test **file names** should describe what is being tested. + +File names should use `snake-case`, but preserve the case of any embedded API +names. For example, prefer `document-createElement.html` to +`document-create-element.html`. + +### Character Encoding + +Tests should use the UTF-8 **character encoding**, which should be declared by +`<meta charset=utf-8>`. A `<meta>` tag is not required (but is acceptable) for +tests that only contain ASCII characters. This guideline does not apply when +specifically testing encodings. + +The `<meta>` tag must be the first child of the document's `<head>` element. In +documents that do not have an explicit `<head>`, the `<meta>` tag must follow +the doctype. + +## Coding Style + +No coding style is enforced for layout tests. This section highlights coding +style aspects that are not consistent across our layout tests, and suggests some +defaults for unopinionated developers. When writing layout tests for a new part +of the codebase, you can minimize review latency by taking a look at existing +tests, and pay particular attention to these issues. Also beware of per-project +style guides, such as the +[ServiceWorker Tests Style guide](https://www.chromium.org/blink/serviceworker/testing). + +### Baseline + +[Google's JavaScript Style Guide](https://google.github.io/styleguide/jsguide.html) +and +[Google's HTML/CSS Style Guide](https://google.github.io/styleguide/htmlcssguide.xml) +are a reasonable baseline for coding style defaults, with the caveat that layout +tests do not use Google Closure or JSDoc. + +### == vs === + +JavaScript's +[== operator](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Operators/Comparison_Operators#Equality_()) +performs some +[type conversion](http://www.ecma-international.org/ecma-262/6.0/#sec-abstract-equality-comparison). +on its arguments, which might be surprising to readers whose experience centers +around C++ or Java. The +[=== operator](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Operators/Comparison_Operators#Identity_strict_equality_()) +is much more similar to `==` in C++. + +Using `===` everywhere is an easy default that saves you, your reviewer, and any +colleague that might have to debug test failures, from having to reason about +[special cases for ==](http://dorey.github.io/JavaScript-Equality-Table/). At +the same time, some developers consider `===` to add unnecessary noise when `==` +would suffice. While `===` should be universally accepted, be flexible if your +reviewer expresses a strong preference for `==`. + +### Let and Const vs Var + +JavaScript variable declarations introduced by +[var](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Statements/var) +are hoisted to the beginning of their containing function, which may be +surprising to C++ and Java developers. By contrast, +[const](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Statements/const) +and +[let](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Statements/let) +declarations are block-scoped, just like in C++ and Java, and have the added +benefit of expressing mutability intent. + +For the reasons above, a reasonable default is to prefer `const` and `let` over +`var`, with the same caveat as above. + +### Strict Mode + +JavaScript's +[strict mode](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Strict_mode), +activated by adding `'use strict';` to the very top of a script, helps catch +some errors, such as mistyping a variable name, forgetting to declare a +variable, or attempting to change a read-only property. + +Given that strict mode gives some of the benefits of using a compiler, adding it +to every test is a good default. This does not apply when specifically testing +sloppy mode behavior. + +Some developers argue that adding the `'use strict';` boilerplate can be +difficult to remember, weighs down smaller tests, and in many cases running a +test case is sufficient to discover any mistyped variable names. + +### Promises + +[Promises](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise) +are a mechanism for structuring asynchronous code. When used correctly, Promises +avoid some of the +[issues of callbacks](http://colintoh.com/blog/staying-sane-with-asynchronous-programming-promises-and-generators). +For these reasons, a good default is to prefer promises over other asynchronous +code structures. + +When using promises, be aware of the +[execution order subtleties](https://jakearchibald.com/2015/tasks-microtasks-queues-and-schedules/) +associated with them. Here is a quick summary. + +* The function passed to `Promise.new` is executed synchronously, so it finishes + before the Promise is created and returned. +* The functions passed to `then` and `catch` are executed in + _separate microtasks_, so they will be executed after the code that resolved + or rejected the promise finishes, but before any other event handler. + +### Classes + +[Classes](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Classes) +are syntactic sugar for JavaScript's +[prototypal inheritance](https://developer.mozilla.org/docs/Web/JavaScript/Inheritance_and_the_prototype_chain). +Compared to manipulating prototypes directly, classes offer a syntax that is +more familiar to developers coming from other programming languages. + +A good default is to prefer classes over other OOP constructs, as they will make +the code easier to read for many of your fellow Chrome developers. At the same +time, most layout tests are simple enough that OOP is not justified. + +### Character Encoding + +When HTML pages do not explicitly declare a character encoding, browsers +determine the encoding using an +[encoding sniffing algorithm](https://html.spec.whatwg.org/multipage/syntax.html#determining-the-character-encoding) +that will surprise most modern Web developers. Highlights include a default +encoding that depends on the user's locale, and non-standardized +browser-specific heuristics. + +The easiest way to not have to think about any of this is to add +`<meta charset="utf-8">` to all your tests. This is easier to remember if you +use a template for your layout tests, rather than writing them from scratch. + +## Tests with Manual Feedback + +Tests that rely on the testing APIs exposed by WPT or Blink will not work when +loaded in a standard browser environment. When writing such tests, default to +having the tests gracefully degrade to manual tests in the absence of the +testing APIs. + +The +[document on layout tests with manual feedback](./layout_tests_with_manual_fallback.md) +describes the approach in detail and highlights the trade-off between added test +weight and ease of debugging. diff --git a/chromium/docs/testing/layout_tests_with_manual_fallback.md b/chromium/docs/testing/layout_tests_with_manual_fallback.md new file mode 100644 index 00000000000..0d8e3a69b9e --- /dev/null +++ b/chromium/docs/testing/layout_tests_with_manual_fallback.md @@ -0,0 +1,119 @@ +# Layout Tests with Manual Fallback + +Some Blink features cannot be automatically tested using the Web Platform. Prime +examples are the APIs that require +[user activation](https://html.spec.whatwg.org/multipage/interaction.html#triggered-by-user-activation) +(also known as _a user gesture_), such as [Full Screen](https://developer.mozilla.org/en-US/docs/Web/API/Fullscreen_API). +Automated tests for these Blink features must rely on special APIs, which are +only exposed in testing environments, and are therefore not available in a +normal browser session. + +A popular pattern used in these tests is to rely on the user to perform some +manual steps in order to run the test case in a normal browser session. These +tests are effectively +[manual tests](http://web-platform-tests.org/writing-tests/manual.html), with +additional JavaScript code that automatically performs the desired manual steps, +when loaded in an environment that exposes the needed testing APIs. + +## Motivation + +Layout tests that degrade to manual tests in the absence of testing APIs have +the following benefits. + +* The manual test component can be debugged in a normal browser session, using + the rich [developer tools](https://developer.chrome.com/devtools). Tests + without a manual fallback can only be debugged in the test runner. +* The manual tests can run in other browsers, making it easy to check whether + our behavior matches other browsers. +* The layout tests can form the basis for manual tests that are contributed to + [web-platform-tests](./web_platform_tests.md). + +Therefore, the desirability of adding a manual fallback to a test heavily +depends on whether the feature under test is a Web Platform feature or a +Blink-only feature, and on the developer's working style. The benefits above +should be weighed against the added design effort needed to build a manual test, +and the size and complexity introduced by the manual fallback. + +## Development Tips + +A natural workflow for writing a layout test that gracefully degrades to a +manual test is to first develop the manual test in a browser, and then add code +that feature-checks for testing APIs, and uses them to automate the test's +manual steps. + +Manual tests should minimize the chance of user error. This implies keeping the +manual steps to a minimum, and having simple and clear instructions that +describe all the configuration changes and user gestures that match the effect +of the Blink-specific APIs used by the test. + +## Example + +Below is an example of a fairly minimal test that uses a Blink-Specific API +(`window.eventSender`), and gracefully degrades to a manual test. + +```html +<!doctype html> +<meta charset="utf-8"> +<title>DOM: Event.isTrusted for UI events</title> +<link rel="help" href="https://dom.spec.whatwg.org/#dom-event-istrusted"> +<link rel="help" href="https://dom.spec.whatwg.org/#constructing-events"> +<meta name="assert" + content="Event.isTrusted is true for events generated by user interaction"> +<script src="../../resources/testharness.js"></script> +<script src="../../resources/testharnessreport.js"></script> + +<p>Please click on the button below.</p> +<button>Click Me!</button> + +<script> +'use strict'; + +setup({ explicit_timeout: true }); + +promise_test(() => { + const button = document.querySelector('button'); + return new Promise((resolve, reject) => { + const button = document.querySelector('button'); + button.addEventListener('click', (event) => { + resolve(event); + }); + + if (window.eventSender) { + eventSender.mouseMoveTo(button.offsetLeft, button.offsetTop); + eventSender.mouseDown(); + eventSender.mouseUp(); + } + }).then((clickEvent) => { + assert_true(clickEvent.isTrusted); + }); + +}, 'Click generated by user interaction'); + +</script> +``` + +The test exhibits the following desirable features: + +* It has a second specification URL (`<link rel="help">`), because the paragraph + that documents the tested feature (referenced by the primary URL) is not very + informative on its own. +* It links to the + [WHATWG Living Standard](https://wiki.whatwg.org/wiki/FAQ#What_does_.22Living_Standard.22_mean.3F), + rather than to a frozen version of the specification. +* It contains clear instructions for manually triggering the test conditions. + The test starts with a paragraph (`<p>`) that tells the tester exactly what to + do, and the `<button>` that needs to be clicked is clearly labeled. +* It disables the timeout mechanism built into `testharness.js` by calling + `setup({ explicit_timeout: true });` +* It checks for the presence of the Blink-specific testing APIs + (`window.eventSender`) before invoking them. The test does not automatically + fail when the APIs are not present. +* It uses [Promises](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise) + to separate the test setup from the assertions. This is particularly helpful + for manual tests that depend on a sequence of events to occur, as Promises + offer a composable way to express waiting for asynchronous events that avoids + [callback hell](http://stackabuse.com/avoiding-callback-hell-in-node-js/). + +Notice that the test is pretty heavy compared to a minimal JavaScript test that +does not rely on testing APIs. Only use testing APIs when the desired testing +conditions cannot be set up using Web Platform APIs. diff --git a/chromium/docs/testing/test_browser_dialog.md b/chromium/docs/testing/test_browser_dialog.md new file mode 100644 index 00000000000..44cf92ad147 --- /dev/null +++ b/chromium/docs/testing/test_browser_dialog.md @@ -0,0 +1,279 @@ +# Testing Chrome browser dialogs with TestBrowserDialog + +\#include "[chrome/browser/ui/test/test_browser_dialog.h]" + +`TestBrowserDialog` provides a way to register an `InProcessBrowserTest` testing +harness with a framework that invokes Chrome browser dialogs in a consistent +way. It optionally provides a way to invoke dialogs "interactively". This allows +screenshots to be generated easily, with the same test data, to assist with UI +review. It also provides a registry of dialogs so they can be systematically +checked for subtle changes and regressions. + +[TOC] + +## How to register a dialog + +If registering an existing dialog, there's a chance it already has a test +harness inheriting, using, or with `typedef InProcessBrowserTest` (or a +descendant of it). If so, using `TestBrowserDialog` is straightforward. Assume +the existing InProcessBrowserTest is in `foo_dialog_browsertest.cc`: + + class FooDialogTest : public InProcessBrowserTest { ... + +Change this to inherit from DialogBrowserTest, and override +`ShowDialog(std::string)`. See [Advanced Usage](#Advanced-Usage) for details. + +```cpp +class FooDialogTest : public DialogBrowserTest { + public: + .. + // DialogBrowserTest: + void ShowDialog(const std::string& name) override { + /* Show dialog attached to browser() and leave it open. */ + } + .. +}; +``` + +Then add test invocations using the usual GTest macros, in +`foo_dialog_browsertest.cc`: + +```cpp +IN_PROC_BROWSER_TEST_F(FooDialogTest, InvokeDialog_default) { + RunDialog(); +} +``` + +Notes: + +* The body of the test is always just "`RunDialog();`". +* "`default`" is the `std::string` passed to `ShowDialog()` and can be + customized. See + [Testing additional dialog "styles"](#Testing-additional-dialog-styles). +* The text before `default` (in this case) must always be "`InvokeDialog_`". + +### Concrete examples + +* [chrome/browser/ui/ask_google_for_suggestions_dialog_browsertest.cc] +* [chrome/browser/ui/autofill/card_unmask_prompt_view_browsertest.cc] +* [chrome/browser/ui/collected_cookies_browsertest.cc] +* [chrome/browser/ui/update_chrome_dialog_browsertest.cc] + +## Running the tests + +List the available dialogs with + + $ ./browser_tests --gtest_filter=BrowserDialogTest.Invoke + +E.g. `FooDialogTest.InvokeDialog_default` should be listed. To show the dialog +interactively, run + + $ ./browser_tests --gtest_filter=BrowserDialogTest.Invoke --interactive \ + --dialog=FooDialogTest.InvokeDialog_default + +### Implementation + +`BrowserDialogTest.Invoke` searches for gtests that have "`InvokeDialog_`" in +their name, so they can be collected in a list. Providing a `--dialog` argument +will invoke that test case in a subprocess. Including `--interactive` will set +up an environment for that subprocess that allows interactivity, e.g., to take +screenshots. The test ends once the dialog is dismissed. + +The `FooDialogTest.InvokeDialog_default` test case **will still be run in the +usual browser_tests test suite**. Ensure it passes, and isn’t flaky. This will +give your dialog some regression test coverage. `RunDialog()` checks to ensure a +dialog is actually created when it invokes `ShowDialog("default")`. + +### BrowserDialogTest.Invoke + +This is also run in browser_tests but, when run that way, the test case just +lists the registered test harnesses (it does *not* iterate over them). A +subprocess is never created unless --dialog is passed on the command line. + +## Advanced Usage + +If your test harness inherits from a descendant of `InProcessBrowserTest` (one +example: [ExtensionBrowserTest]) then the `SupportsTestDialog<>` template is +provided. E.g. + +```cpp +class ExtensionInstallDialogViewTestBase : public ExtensionBrowserTest { ... +``` + +becomes + +```cpp +class ExtensionInstallDialogViewTestBase : + public SupportsTestDialog<ExtensionBrowserTest> { ... +``` + +### Testing additional dialog "styles" + +Add additional test cases, with a different string after "`InvokeDialog_`". +Example: + +```cpp +IN_PROC_BROWSER_TEST_F(CardUnmaskViewBrowserTest, InvokeDialog_expired) { + RunDialog(); +} + +IN_PROC_BROWSER_TEST_F(CardUnmaskViewBrowserTest, InvokeDialog_valid) { + RunDialog(); +} +``` + +The strings "`expired`" or “`valid`” will be given as arguments to +`ShowDialog(std::string)`. + +## Rationale + +Bug reference: [Issue 654151](http://crbug.com/654151). + +Chrome has a lot of browser dialogs; often for obscure use-cases and often hard +to invoke. It has traditionally been difficult to be systematic while checking +dialogs for possible regressions. For example, to investigate changes to shared +layout parameters which are testable only with visual inspection. + +For Chrome UI review, screenshots need to be taken. Iterating over all the +"styles" that a dialog may appear with is fiddly. E.g. a login or particular web +server setup may be required. It’s important to provide a consistent “look” for +UI review (e.g. same test data, same browser size, anchoring position, etc.). + +Some dialogs lack tests. Some dialogs have zero coverage on the bots. Dialogs +can have tricky lifetimes and common mistakes are repeated. TestBrowserDialog +runs simple "Show dialog" regression tests and can be extended to do more. + +Even discovering the full set of dialogs present for each platform in Chrome is +[difficult](http://crbug.com/686239). + +### Why browser_tests? + +* `browser_tests` already provides a `browser()->window()` of a consistent + size that can be used as a dialog anchor and to take screenshots for UI + review. + * UI review have requested that screenshots be provided with the entire + browser window so that the relative size of the dialog/change under + review can be assessed. + +* Some dialogs already have a test harness with appropriate setup (e.g. test + data) running in browser_tests. + * Supporting `BrowserDialogTest` should require minimal setup and minimal + ongoing maintenance. + +* An alternative is to maintain a working end-to-end build target executable + to do this, but this has additional costs (and is hard). + * E.g. setup/teardown of low-level functions ( + `InitializeGLOneOffPlatform()`, + `MaterialDesignController::Initialize()`, etc.). + +* Why not chrome.exe? + * E.g. a scrappy chrome:// page with links to invoke dialogs would be + great! + * But... + * A dialog may have test data (e.g. credit card info) which shouldn’t + be in the release build. + * A dialog may use EmbeddedTestServer. + * Higher maintenance cost - can’t leverage existing test harnesses. + +## Future Work + +* Opt in more dialogs! + * Eventually, all of them. + * A `BrowserDialogTest` for every descendant of `views::DialogDelegate`. + +* Automatically generate screenshots (for each platform, in various languages) + * Build upon [CL 2008283002](https://codereview.chromium.org/2008283002/) + +* (maybe) Try removing the subprocess + * Probably requires altering the browser_test suite code directly rather + than just adding a test case as in the current approach + +* An automated test suite for dialogs + * Test various ways to dismiss or hide a dialog + * e.g. native close (via taskbar?) + * close parent window (possibly via task bar) + * close parent tab + * switch tabs + * close via `DialogClientView::AcceptWindow` (and `CancelWindow`) + * close via `Widget::Close` + * close via `Widget::CloseNow` + * Drag tab off browser into a new window + * Fullscreen that may create a new window/parent + +* Find obscure workflows for invoking dialogs that have no test coverage and + cause crashes (e.g. [http://crrev.com/426302](http://crrev.com/426302)) + * Supporting window-modal dialogs with a null parent window. + +* Find memory leaks, e.g. [http://crrev.com/432320](http://crrev.com/432320) + * "Fix memory leak for extension uninstall dialog". + +## Appendix: Sample output + +**$ ./out/gn_Debug/browser_tests --gtest_filter=BrowserDialogTest.Invoke** +``` +Note: Google Test filter = BrowserDialogTest.Invoke +[==========] Running 1 test from 1 test case. +[----------] Global test environment set-up. +[----------] 1 test from BrowserDialogTest +[ RUN ] BrowserDialogTest.Invoke +[26879:775:0207/134949.118352:30434675...:INFO:browser_dialog_browsertest.cc(46) +Pass one of the following after --dialog= + AskGoogleForSuggestionsDialogTest.InvokeDialog_default + CardUnmaskPromptViewBrowserTest.InvokeDialog_expired + CardUnmaskPromptViewBrowserTest.InvokeDialog_valid + CollectedCookiesTestMd.InvokeDialog_default + UpdateRecommendedDialogTest.InvokeDialog_default +/* lots more will eventually be listed here */ +[ OK ] BrowserDialogTest.Invoke (0 ms) +[----------] 1 test from BrowserDialogTest (0 ms total) +[----------] Global test environment tear-down +[==========] 1 test from 1 test case ran. (1 ms total) +[ PASSED ] 1 test. +[1/1] BrowserDialogTest.Invoke (334 ms) +SUCCESS: all tests passed. +``` + +**$ ./out/gn_Debug/browser_tests --gtest_filter=BrowserDialogTest.Invoke +--dialog=CardUnmaskPromptViewBrowserTest.InvokeDialog_expired** + +``` +Note: Google Test filter = BrowserDialogTest.Invoke +[==========] Running 1 test from 1 test case. +[----------] Global test environment set-up. +[----------] 1 test from BrowserDialogTest +[ RUN ] BrowserDialogTest.Invoke +Note: Google Test filter = CardUnmaskPromptViewBrowserTest.InvokeDefault +[==========] Running 1 test from 1 test case. +[----------] Global test environment set-up. +[----------] 1 test from CardUnmaskPromptViewBrowserTest, where TypeParam = +[ RUN ] CardUnmaskPromptViewBrowserTest.InvokeDialog_expired +/* 7 lines of uninteresting log spam */ +[ OK ] CardUnmaskPromptViewBrowserTest.InvokeDialog_expired (1324 ms) +[----------] 1 test from CardUnmaskPromptViewBrowserTest (1324 ms total) +[----------] Global test environment tear-down +[==========] 1 test from 1 test case ran. (1325 ms total) +[ PASSED ] 1 test. +[ OK ] BrowserDialogTest.Invoke (1642 ms) +[----------] 1 test from BrowserDialogTest (1642 ms total) +[----------] Global test environment tear-down +[==========] 1 test from 1 test case ran. (1642 ms total) +[ PASSED ] 1 test. +[1/1] BrowserDialogTest.Invoke (2111 ms) +SUCCESS: all tests passed. +``` + +**$ ./out/gn_Debug/browser_tests --gtest_filter=BrowserDialogTest.Invoke +--dialog=CardUnmaskPromptViewBrowserTest.InvokeDialog_expired --interactive** +``` +/* + * Output as above, except the test are not interleaved, and the browser window + * should remain open until the dialog is dismissed + */ +``` + +[chrome/browser/ui/test/test_browser_dialog.h]: https://cs.chromium.org/chromium/src/chrome/browser/ui/test/test_browser_dialog.h +[chrome/browser/ui/autofill/card_unmask_prompt_view_browsertest.cc]: https://cs.chromium.org/chromium/src/chrome/browser/ui/autofill/card_unmask_prompt_view_browsertest.cc?l=104&q=ShowDialog +[chrome/browser/ui/collected_cookies_browsertest.cc]: https://cs.chromium.org/chromium/src/chrome/browser/ui/collected_cookies_browsertest.cc?l=26&q=ShowDialog +[chrome/browser/ui/ask_google_for_suggestions_dialog_browsertest.cc]: https://cs.chromium.org/chromium/src/chrome/browser/ui/ask_google_for_suggestions_dialog_browsertest.cc?l=18&q=ShowDialog +[chrome/browser/ui/update_chrome_dialog_browsertest.cc]: https://cs.chromium.org/chromium/src/chrome/browser/ui/update_chrome_dialog_browsertest.cc?l=15&q=ShowDialog +[ExtensionBrowserTest]: https://cs.chromium.org/chromium/src/chrome/browser/extensions/extension_browsertest.h?q=extensionbrowsertest&l=40 diff --git a/chromium/docs/testing/using_breakpad_with_content_shell.md b/chromium/docs/testing/using_breakpad_with_content_shell.md index 0fcfadfbbfd..7579deb9858 100644 --- a/chromium/docs/testing/using_breakpad_with_content_shell.md +++ b/chromium/docs/testing/using_breakpad_with_content_shell.md @@ -1,7 +1,7 @@ # Using breakpad with content shell When running layout tests, it is possible to use -[breakpad](../../breakpad/breakpad/) to capture stack traces on crashes while +[breakpad](../../breakpad/) to capture stack traces on crashes while running without a debugger attached and with the sandbox enabled. ## Setup diff --git a/chromium/docs/testing/web_platform_tests.md b/chromium/docs/testing/web_platform_tests.md new file mode 100644 index 00000000000..063ca3e368c --- /dev/null +++ b/chromium/docs/testing/web_platform_tests.md @@ -0,0 +1,171 @@ +# web-platform-tests + +Interoperability between browsers is +[critical](https://www.chromium.org/blink/platform-predictability) to Chromium's +mission of improving the web. We believe that leveraging and contributing to a +shared test suite is one of the most important tools in achieving +interoperability between browsers. The [web-platform-tests +repository](https://github.com/w3c/web-platform-tests) is the primary shared +test suite where all browser engines are collaborating. There's also a +[csswg-test repository](https://github.com/w3c/csswg-test) for CSS tests, but +that will [soon be merged into +web-platform-tests](https://github.com/w3c/csswg-test/issues/1102). + +Chromium has 2-way import/export process with the upstream web-platform-tests +repository, where tests are imported into +[LayoutTests/external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) +and any changes to the imported tests are also exported to web-platform-tests. + +See http://web-platform-tests.org/ for general documentation on +web-platform-tests, including tips for writing and reviewing tests. + +[TOC] + +## Importing tests + +Chromium has mirrors +([web-platform-tests](https://chromium.googlesource.com/external/w3c/web-platform-tests/), +[csswg-test](https://chromium.googlesource.com/external/w3c/csswg-test/)) of the +GitHub repos, and periodically imports a subset of the tests so that they are +run as part of the regular Blink layout test testing process. + +The goal of this process are to be able to run web-platform-tests unmodified +locally just as easily as we can run the Blink tests, and ensure that we are +tracking tip-of-tree in the web-platform-tests repository as closely as +possible, and running as many of the tests as possible. + +### Automatic import process + +There is an automatic process for updating the Chromium copy of +web-platform-tests. The import is done by the builder [w3c-test-autoroller +builder](https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller). + +The easiest way to check the status of recent imports is to look at: + +- Recent logs on Buildbot for [w3c-test-autoroller + builder](https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller) +- Recent CLs created by + [blink-w3c-test-autoroller@chromium.org](https://codereview.chromium.org/search?owner=blink-w3c-test-autoroller%40chromium.org). + +Automatic imports are intended to run at least once every 24 hours. + +### Skipped tests + +We control which tests are imported via a file called +[W3CImportExpectations](../../third_party/WebKit/LayoutTests/W3CImportExpectations), +which has a list of directories to skip while importing. + +In addition to the directories and tests explicitly skipped there, tests may +also be skipped for a couple other reasons, e.g. if the file path is too long +for Windows. To check what files are skipped in import, check the recent logs +for [w3c-test-autoroller +builder](https://build.chromium.org/p/chromium.infra.cron/builders/w3c-test-autoroller). + +### Manual import + +To pull the latest versions of the tests that are currently being imported, you +can also directly invoke the +[wpt-import](../../third_party/WebKit/Tools/Scripts/wpt-import) script. + +That script will pull the latest version of the tests from our mirrors of the +upstream repositories. If any new versions of tests are found, they will be +committed locally to your local repository. You may then upload the changes. + +### Enabling import for a new directory + +If you wish to add more tests (by un-skipping some of the directories currently +skipped in `W3CImportExpectations`), you can modify that file locally and commit +it, and on the next auto-import, the new tests should be imported. + +If you want to import immediately (in order to try the tests out locally, etc) +you can also run `wpt-import --allow-local-commits`, but this is not required. + +## Writing tests + +To contribute changes to web-platform-tests, just commit your changes directly +to [LayoutTests/external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) +and the changes will be automatically upstreamed within 24 hours. + +Changes involving adding, removing or modifying tests can all be upstreamed. +Any changes outside of +[external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) will not be +upstreamed, and any changes `*-expected.txt`, `OWNERS`, and `MANIFEST.json`, +will also not be upstreamed. + +Running the layout tests will automatically regenerate MANIFEST.json to pick up +any local modifications. + +Most tests are written using testharness.js, see +[Writing Layout Tests](./writing_layout_tests.md) and +[Layout Tests Tips](./layout_tests_tips.md) for general guidelines. + +### Write tests against specifications + +Tests in web-platform-tests are expected to match behavior defined by the +relevant specification. In other words, all assertions that a test makes +should be derived from a specification's normative requirements, and not go +beyond them. It is often necessary to change the specification to clarify what +is and isn't required. + +When the standards discussion is still ongoing or blocked on some implementation +successfully shipping the hoped-for behavior, write the tests outside of +web-platform-tests and upstream them when the specification is finally updated. +Optionally, it may be possible to write deliberately failing tests against the +current specification and later update them. + +### Tests that require testing APIs + +Tests that depend on `internals.*`, `eventSender.*` or other internal testing +APIs cannot yet be written as part of web-platform-tests. + +An alternative is to write manual tests that are automated with scripts from +[wpt_automation](../../third_party/WebKit/LayoutTests/external/wpt_automation). +Such tests still require case-by-case automation to run for other browser +engines, but are more valuable than purely manual tests. + +*** note +TODO(foolip): Figure out and document a more scalable test automation solution. +*** + +### Adding new top-level directories + +Entirely new top-level directories should generally be added upstream, since +that's the only way to add an OWNERS file upstream. After adding a new top-level +directory upstream, you should add a line for it in `W3CImportExpectations`. + +Adding the new directory (and `W3CImportExpectations` entry) in Chromium and +later adding an OWNERS file upstream also works. + +### Will the exported commits be linked to my GitHub profile? + +The email you commit with in Chromium will be the author of the commit on +GitHub. You can [add it as a secondary address on your GitHub +account](https://help.github.com/articles/adding-an-email-address-to-your-github-account/) +to link your exported commits to your GitHub profile. + +### What if there are conflicts? + +This cannot be avoided entirely as the two repositories are independent, but +should be rare with frequent imports and exports. When it does happen, manual +intervention will be needed and in non-trivial cases you may be asked to help +resolve the conflict. + +### Direct pull requests + +It's still possible to make direct pull requests to web-platform-tests, see +http://web-platform-tests.org/appendix/github-intro.html. + +## Reviewing tests + +Anyone who can review code and tests in Chromium can also review changes in +[external/wpt](../../third_party/WebKit/LayoutTests/external/wpt) +that will be automatically upstreamed. There will be no additional review in +web-platform-tests as part of the export process. + +If upstream reviewers have feedback on the changes, discuss on the pull request +created during export, and if necessary work on a new pull request to iterate +until everyone is satisfied. + +When reviewing tests, check that they match the relevant specification, which +may not fully match the implementation. See also +[Write tests against specifications](#Write-tests-against-specifications). diff --git a/chromium/docs/testing/writing_layout_tests.md b/chromium/docs/testing/writing_layout_tests.md index 37e5e2ab7af..81146831e66 100644 --- a/chromium/docs/testing/writing_layout_tests.md +++ b/chromium/docs/testing/writing_layout_tests.md @@ -13,8 +13,7 @@ or XHTML) and lives in Layout tests should be used to accomplish one of the following goals: 1. The entire surface of Blink that is exposed to the Web should be covered by - tests that we contribute to the - [Web Platform Tests Project](https://github.com/w3c/web-platform-tests) + tests that we contribute to [web-platform-tests](./web_platform_tests.md) (WPT). This helps us avoid regressions, and helps us identify Web Platform areas where the major browsers don't have interoperable implementations. Furthermore, by contributing to projects such as WPT, we share the burden of @@ -68,246 +67,24 @@ There are four broad types of layout tests, listed in the order of preference. Layout tree tests are used as a last resort to test the internal quirks of the implementation, and they should be avoided in favor of one of the earlier options. -## General Principles - -The principles below are adapted from -[Test the Web Forward's Test Format Guidelines](http://testthewebforward.org/docs/test-format-guidelines.html) -and -[WebKit's Wiki page on Writing good test cases](https://trac.webkit.org/wiki/Writing%20Layout%20Tests%20for%20DumpRenderTree). - -*** note -This document intentionally uses _should_ a lot more than _must_, as defined in -[RFC 2119](https://www.ietf.org/rfc/rfc2119.txt). Writing layout tests is a -careful act of balancing many concerns, and this humble document cannot possibly -capture the context that rests in the head of an experienced Blink engineer. -*** - -### Concise - -Tests should be **concise**, without compromising on the principles below. Every -element and piece of code on the page should be necessary and relevant to what -is being tested. For example, don't build a fully functional signup form if you -only need a text field or a button. - -Content needed to satisfy the principles below is considered necessary. For -example, it is acceptable and desirable to add elements that make the test -self-describing (see below), and to add code that makes the test more reliable -(see below). - -Content that makes test failures easier to debug is considered necessary (to -maintaining a good development speed), and is both acceptable and desirable. - -*** promo -Conciseness is particularly important for reference tests and pixel tests, as -the test pages are rendered in an 800x600px viewport. Having content outside the -viewport is undesirable because the outside content does not get compared, and -because the resulting scrollbars are platform-specific UI widgets, making the -test results less reliable. -*** - -### Fast - -Tests should be as **fast** as possible, without compromising on the principles -below. Blink has several thousand layout tests that are run in parallel, and -avoiding unnecessary delays is crucial to keeping our Commit Queue in good -shape. - -Avoid -[window.setTimeout](https://developer.mozilla.org/en-US/docs/Web/API/WindowTimers/setTimeout), -as it wastes time on the testing infrastructure. Instead, use specific event -handlers, such as -[window.onload](https://developer.mozilla.org/en-US/docs/Web/API/GlobalEventHandlers/onload), -to decide when to advance to the next step in a test. - -### Reliable - -Tests should be **reliable** and yield consistent results for a given -implementation. Flaky tests slow down your fellow developers' debugging efforts -and the Commit Queue. - -`window.setTimeout` is again a primary offender here. Asides from wasting time -on a fast system, tests that rely on fixed timeouts can fail when on systems -that are slower than expected. - -Follow the guidelines in this -[PSA on writing reliable layout tests](https://docs.google.com/document/d/1Yl4SnTLBWmY1O99_BTtQvuoffP8YM9HZx2YPkEsaduQ/edit). - -### Self-Describing - -Tests should be **self-describing**, so that a project member can recognize -whether a test passes or fails without having to read the specification of the -feature being tested. `testharness.js` makes a test self-describing when used -correctly, but tests that degrade to manual tests -[must be carefully designed](http://testthewebforward.org/docs/test-style-guidelines.html) -to be self-describing. - -### Minimal - -Tests should require a **minimal** amount of cognitive effort to read and -maintain. - -Avoid depending on edge case behavior of features that aren't explicitly covered -by the test. For example, except where testing parsing, tests should contain -valid markup (no parsing errors). -Tests should provide as much relevant information as possible when failing. -`testharness.js` tests should prefer -[rich assert_ functions](https://github.com/w3c/testharness.js/blob/master/docs/api.md#list-of-assertions) -to combining `assert_true()` with a boolean operator. Using appropriate -`assert_` functions results in better diagnostic output when the assertion -fails. - -🚧 Prefer JavaScript's -[===](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Operators/Comparison_Operators#Identity_strict_equality_()) -operator to -[==](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Operators/Comparison_Operators#Equality_()) -so that readers don't have to reason about -[type conversion](http://www.ecma-international.org/ecma-262/6.0/#sec-abstract-equality-comparison). - -*** promo -The === vs == recommendation is still being discussed on -[blink-dev](https://groups.google.com/a/chromium.org/d/topic/blink-dev/XsR6PKRrS1E/discussion). -However, please keep in mind that a fellow developer who has to debug a failing -test may have to consider the -[special cases for ==](http://dorey.github.io/JavaScript-Equality-Table/) when -the types of the two values being compared aren't immediately obvious. -*** - -### Cross-Platform - -Tests should be as **cross-platform** as reasonably possible. Avoid assumptions -about device type, screen resolution, etc. Unavoidable assumptions should be -documented. - -When possible, tests should only use Web platform features, as specified -in the relevant standards. When the Web platform's APIs are insufficient, -tests should prefer to use WPT extended testing APIs, such as -`wpt_automation`, over Blink-specific testing APIs. - -🚧 Tests that use testing APIs should feature-test for the presence of -those APIs, and gracefully degrade to manual tests (see below) when the testing -APIs are not available. +## General Principles -Test pages should use the HTML5 doctype (`<!doctype html>`) unless they -specifically cover -[quirks mode](https://developer.mozilla.org/docs/Quirks_Mode_and_Standards_Mode) -behavior. Tests should be written under the assumption that they will be upstreamed -to the WPT project. For example, tests should follow the -[WPT guidelines](http://testthewebforward.org/docs/writing-tests.html). - -Tests should avoid using features that haven't been shipped by the -actively-developed major rendering engines (Blink, WebKit, Gecko, Edge). When -unsure, check [caniuse.com](http://caniuse.com/). By necessity, this -recommendation does not apply to the feature targeted by the test. - -*** note -It may be tempting have a test for a bleeding-edge feature X depend on feature -Y, which has only shipped in beta / development versions of various browsers. -The reasoning would be that all browsers that implement X will have implemented -Y. Please keep in mind that Chrome has un-shipped features that made it to the -Beta channel in the past. -*** - -Tests that use Blink-specific testing APIs should feature-test for the -presence of the testing APIs and degrade to -[manual tests](http://testthewebforward.org/docs/manual-test.html) when -the testing APIs are not present. - -*** promo -The recommendation to degrade to manual tests is still being discussed on -[blink-dev](https://groups.google.com/a/chromium.org/d/topic/blink-dev/XsR6PKRrS1E/discussion). -However, please keep in mind that a manual test can be debugged in the browser, -whereas a test that does not degrade gracefully can only be debugged in the test -runner. Fellow project members and future you will thank you for having your -test work as a manual test. -*** - -### Self-Contained - -Tests must be **self-contained** and not depend on external network resources. - -Unless used by multiple test files, CSS and JavaScript should be inlined using -`<style>` and `<script>` tags. Content shared by multiple tests should be -placed in a `resources/` directory near the tests that share it. See below for -using multiple origins in a test. - -### File Names - -Test **file names** should describe what is being tested. +to the WPT project. To this end, tests should follow the +[WPT guidelines](http://web-platform-tests.org/writing-tests/). -File names should use `snake-case`, but preserve the case of any embedded API -names. For example, prefer `document-createElement.html` to -`document-create-element.html`. -### Modern Features +There is no style guide that applies to all layout tests. However, some projects +have adopted style guides, such as the +[ServiceWorker Tests Style guide](https://www.chromium.org/blink/serviceworker/testing). -Tests should prefer **modern features** in JavaScript and in the Web Platform, -provided that they meet the recommendations above for cross-platform tests. - -Tests should use -[strict mode](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Strict_mode) -for all JavaScript, except when specifically testing sloppy mode behavior. -Strict mode flags deprecated features and helps catch some errors, such as -forgetting to declare variables. - -🚧 JavaScript code should prefer -[const](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Statements/const) -and -[let](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Statements/let) -over `var`, -[classes](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Classes) -over other OOP constructs, and -[Promises](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise) -over other mechanisms for structuring asynchronous code. - -*** promo -The recommendation to prefer `const` and `let` over `var` is currently being -discussed on -[blink-dev](https://groups.google.com/a/chromium.org/d/topic/blink-dev/XsR6PKRrS1E/discussion). -*** - -### Character Encoding - -🚧 Tests should use the UTF-8 **character encoding**, which should be -declared by `<meta charset=utf-8>`. This does not apply when specifically -testing encodings. - -The `<meta>` tag must be the first child of the document's `<head>` element. In -documents that do not have an explicit `<head>`, the `<meta>` tag must follow -the doctype. - -When HTML pages do not explicitly declare a character encoding, browsers -determine the encoding using an -[encoding sniffing algorithm](https://html.spec.whatwg.org/multipage/syntax.html#determining-the-character-encoding) -that will surprise most modern Web developers. Highlights include a default -encoding that depends on the user's locale, and non-standardized -browser-specific heuristics. - -*** promo -The WPT guidelines state that test files that only contain ASCII characters may -omit the `<meta>` tag. This exception is currently discussed on -[blink-dev](https://groups.google.com/a/chromium.org/d/topic/blink-dev/XsR6PKRrS1E/discussion). -If taking that route, please keep in mind that Firefox currently issues a -[development tools](https://developer.mozilla.org/en-US/docs/Tools) warning for -pages without a declared encoding. -*** - -### Coding Style - -Tests should aim to have a **coding style** that is consistent with -[Google's JavaScript Style Guide](https://google.github.io/styleguide/jsguide.html), -and -[Google's HTML/CSS Style Guide](https://google.github.io/styleguide/htmlcssguide.xml), -with the following exceptions. - -* Rules related to Google Closure and JSDoc do not apply. -* Modern Web Platform and JavaScript features should be preferred to legacy - constructs that target old browsers. -* Per the JavaScript guide, new tests should also follow any per-project - style guide, such as the - [ServiceWorker Tests Style guide](https://www.chromium.org/blink/serviceworker/testing). +Our [document on layout tests tips](./layout_tests_tips.md) summarizes the most +important WPT guidelines and highlights some JavaScript concepts that are worth +paying attention to when trying to infer style rules from existing tests. If +you're unopinionated and looking for a style guide to follow, the document also +suggests some defaults. ## JavaScript Tests @@ -322,12 +99,12 @@ framework is used by the tests in the which is shared with all the other browser vendors, so `testharness.js` tests are more accessible to browser developers. -As a shared framework, `testharness.js` enjoys high-quality documentation, such -as [a tutorial](http://testthewebforward.org/docs/testharness-tutorial.html) and -[API documentation](https://github.com/w3c/testharness.js/blob/master/docs/api.md). -Layout tests should follow the recommendations of the above documents. +See the [API documentation](https://github.com/w3c/testharness.js/blob/master/docs/api.md) +for a thorough introduction to `testharness.js`. + +Layout tests should follow the recommendations of the above documentation. Furthermore, layout tests should include relevant -[metadata](http://testthewebforward.org/docs/css-metadata.html). The +[metadata](http://web-platform-tests.org/writing-tests/css-metadata.html). The specification URL (in `<link rel="help">`) is almost always relevant, and is incredibly helpful to a developer who needs to understand the test quickly. @@ -337,10 +114,8 @@ in order to follow the minimality guideline, the test omits the tags `<html>`, ```html <!doctype html> -<meta charset="utf-8"> -<title>JavaScript: the true literal</title> +<title>JavaScript: the true literal is immutable and equal to itself</title> <link rel="help" href="https://tc39.github.io/ecma262/#sec-boolean-literals"> -<meta name="assert" value="The true literal is equal to itself and immutable"> <script src="/resources/testharness.js"></script> <script src="/resources/testharnessreport.js"></script> <script> @@ -375,9 +150,6 @@ promise_test(() => { Some points that are not immediately obvious from the example: -* The `<meta name="assert">` describes the purpose of the entire file, and - is not redundant to `<title>`. Don't add a `<meta name="assert">` when the - information in the `<title>` is sufficient. * When calling an `assert_` function that compares two values, the first argument is the actual value (produced by the functionality being tested), and the second argument is the expected value (known good, golden). The order @@ -417,7 +189,7 @@ and [/resources/testharnessreport.js](../../third_party/WebKit/LayoutTests/resources/testharnessreport.js). This is contrary to the WPT guidelines, which call for absolute paths. This limitation does not apply to the tests in `LayoutTests/http`, which rely on -an HTTP server, or to the tests in `LayoutTests/imported/wpt`, which are +an HTTP server, or to the tests in `LayoutTests/external/wpt`, which are imported from the [WPT repository](https://github.com/w3c/web-platform-tests). *** @@ -428,16 +200,17 @@ some tests that require a user to perform a gesture, such as a mouse click, cannot be implemented using Web APIs. The WPT project covers some of these cases via supplemental testing APIs. +When writing tests that rely on supplemental testing APIs, please consider the +cost and benefits of having the tests +[gracefully degrade to manual tests](./layout_tests_with_manual_fallback.md) in +the absence of the testing APIs. + *** promo In many cases, the user gesture is not actually necessary. For example, many event handling tests can use [synthetic events](https://developer.mozilla.org/docs/Web/Guide/Events/Creating_and_triggering_events). *** -*** note -TODO: document wpt_automation. Manual tests might end up moving here. -*** - ### Relying on Blink-Specific Testing APIs Tests that cannot be expressed using the Web Platform APIs or WPT's testing APIs @@ -453,7 +226,7 @@ For example, the most popular Blink-specific API is `testRunner`, which is implemented in [components/test_runner/test_runner.h](../../components/test_runner/test_runner.h) and -[components/test_runner/test_runner.cpp](../../components/test_runner/test_runner.cpp). +[components/test_runner/test_runner.cc](../../components/test_runner/test_runner.cc). By skimming the `TestRunnerBindings::Install` method, we learn that the testRunner API is presented by the `window.testRunner` and `window.layoutTestsController` objects, which are synonyms. Reading the @@ -480,7 +253,7 @@ See the [components/test_runner/](../../components/test_runner/) directory and for other useful APIs. For example, `window.eventSender` ([components/test_runner/event_sender.h](../../components/test_runner/event_sender.h) and -[components/test_runner/event_sender.cpp](../../components/test_runner/event_sender.cpp)) +[components/test_runner/event_sender.cc](../../components/test_runner/event_sender.cc)) has methods that simulate events input such as keyboard / mouse input and drag-and-drop. @@ -488,98 +261,6 @@ Here is a UML diagram of how the `testRunner` bindings fit into Chromium. [](https://docs.google.com/drawings/d/1KNRNjlxK0Q3Tp8rKxuuM5mpWf4OJQZmvm9_kpwu_Wwg/edit) -### Manual Tests - -🚧 Whenever possible, tests that rely on (WPT's or Blink's) testing APIs -should also be usable as -[manual tests](http://testthewebforward.org/docs/manual-test.html). This makes -it easy to debug the test, and to check whether our behavior matches other -browsers. - -*** promo -The recommendation to degrade to manual tests is still being discussed on -[blink-dev](https://groups.google.com/a/chromium.org/d/topic/blink-dev/XsR6PKRrS1E/discussion). -However, please keep in mind that a manual test can be debugged in the browser, -whereas a test that does not degrade gracefully can only be debugged in the test -runner. Fellow project members and future you will thank you for having your -test work as a manual test. -*** - -Manual tests should minimize the chance of user error. This implies keeping the -manual steps to a minimum, and having simple and clear instructions that -describe all the configuration changes and user gestures that match the effect -of the Blink-specific APIs used by the test. - -Below is an example of a fairly minimal test that uses a Blink-Specific API -(`window.eventSender`), and gracefully degrades to a manual test. - -```html -<!doctype html> -<meta charset="utf-8"> -<title>DOM: Event.isTrusted for UI events</title> -<link rel="help" href="https://dom.spec.whatwg.org/#dom-event-istrusted"> -<link rel="help" href="https://dom.spec.whatwg.org/#constructing-events"> -<meta name="assert" - content="Event.isTrusted is true for events generated by user interaction"> -<script src="../../resources/testharness.js"></script> -<script src="../../resources/testharnessreport.js"></script> - -<p>Please click on the button below.</p> -<button>Click Me!</button> - -<script> -'use strict'; - -setup({ explicit_timeout: true }); - -promise_test(() => { - const button = document.querySelector('button'); - return new Promise((resolve, reject) => { - const button = document.querySelector('button'); - button.addEventListener('click', (event) => { - resolve(event); - }); - - if (window.eventSender) { - eventSender.mouseMoveTo(button.offsetLeft, button.offsetTop); - eventSender.mouseDown(); - eventSender.mouseUp(); - } - }).then((clickEvent) => { - assert_true(clickEvent.isTrusted); - }); - -}, 'Click generated by user interaction'); - -</script> -``` - -The test exhibits the following desirable features: - -* It has a second specification URL (`<link rel="help">`), because the paragraph - that documents the tested feature (referenced by the primary URL) is not very - informative on its own. -* It links to the - [WHATWG Living Standard](https://wiki.whatwg.org/wiki/FAQ#What_does_.22Living_Standard.22_mean.3F), - rather than to a frozen version of the specification. -* It contains clear instructions for manually triggering the test conditions. - The test starts with a paragraph (`<p>`) that tells the tester exactly what to - do, and the `<button>` that needs to be clicked is clearly labeled. -* It disables the timeout mechanism built into `testharness.js` by calling - `setup({ explicit_timeout: true });` -* It checks for the presence of the Blink-specific testing APIs - (`window.eventSender`) before invoking them. The test does not automatically - fail when the APIs are not present. -* It uses [Promises](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise) - to separate the test setup from the assertions. This is particularly helpful - for manual tests that depend on a sequence of events to occur, as Promises - offer a composable way to express waiting for asynchronous events that avoids - [callback hell](http://stackabuse.com/avoiding-callback-hell-in-node-js/). - -Notice that the test is pretty heavy compared to a minimal JavaScript test that -does not rely on testing APIs. Only use testing APIs when the desired testing -conditions cannot be set up using Web Platform APIs. - ### Text Test Baselines By default, all the test cases in a file that uses `testharness.js` are expected @@ -671,8 +352,8 @@ be slower as well. Therefore, they should only be used for functionality that cannot be covered by JavaScript tests. New reference tests should follow the -[WPT reftests guidelines](http://testthewebforward.org/docs/reftests.html). The -most important points are summarized below. +[WPT reftests guidelines](http://web-platform-tests.org/writing-tests/reftests.html). +The most important points are summarized below. * 🚧 The test page declares the reference page using a `<link rel="match">` or `<link rel="mismatch">`, depending on whether the test @@ -704,7 +385,6 @@ The example assumes that the test page is named `ol-reversed.html`. ```html <!doctype html> -<meta charset="utf-8"> <link rel="match" href="ol-reversed-expected.html"> <ol reversed> @@ -718,7 +398,6 @@ The reference page, which must be named `ol-reversed-expected.html`, is below. ```html <!doctype html> -<meta charset="utf-8"> <ol> <li value="3">A</li> @@ -746,7 +425,7 @@ tests**. Pixel tests should still follow the principles laid out above. Pixel tests pose unique challenges to the desire to have *self-describing* and *cross-platform* tests. The -[WPT test style guidelines](http://testthewebforward.org/docs/test-style-guidelines.html) +[WPT rendering test guidelines](http://web-platform-tests.org/writing-tests/rendering.html) contain useful guidance. The most relevant pieces of advice are below. * Whenever possible, use a green paragraph / page / square to indicate success. @@ -787,7 +466,7 @@ body { ``` *** promo -Tests outside `LayoutTests/http` and `LayoutTests/imported/wpt` currently need +Tests outside `LayoutTests/http` and `LayoutTests/external/wpt` currently need to use a relative path to [/third_party/WebKit/LayoutTests/resources/ahem.js](../../third_party/WebKit/LayoutTests/resources/ahem.js) *** @@ -844,7 +523,6 @@ The following page is an example of a layout tree test. ```html <!doctype html> -<meta charset="utf-8"> <style> body { font: 10px Ahem; } span::after { diff --git a/chromium/docs/updating_clang_format_binaries.md b/chromium/docs/updating_clang_format_binaries.md index 1915152799c..95ef812f8a0 100644 --- a/chromium/docs/updating_clang_format_binaries.md +++ b/chromium/docs/updating_clang_format_binaries.md @@ -28,7 +28,7 @@ Windows step-by-step: # [double check you have the tools you need] where cmake.exe # You need to install this. where svn.exe # Maybe fix with: set PATH=%PATH%;D:\src\depot_tools\svn_bin -"c:\Program Files (x86)\Microsoft Visual Studio 12.0\vc\vcvarsall.bat" amd64_x86 +"c:\Program Files (x86)\Microsoft Visual Studio 14.0\vc\vcvarsall.bat" amd64_x86 set CLANG_REV=198831 # You must change this value (see above) @@ -43,8 +43,8 @@ svn co http://llvm.org/svn/llvm-project/cfe/trunk@%CLANG_REV% clang cd ..\..\llvm-build set CC=cl set CXX=cl -cmake -G Ninja ..\llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_CRT_RELEASE=MT \ - -DLLVM_ENABLE_ASSERTIONS=NO -DLLVM_ENABLE_THREADS=NO \ +cmake -G Ninja ..\llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_CRT_RELEASE=MT ^ + -DLLVM_ENABLE_ASSERTIONS=NO -DLLVM_ENABLE_THREADS=NO ^ -DPYTHON_EXECUTABLE=d:\src\depot_tools\python276_bin\python.exe ninja clang-format bin\clang-format.exe --version @@ -65,7 +65,7 @@ svn co http://llvm.org/svn/llvm-project/cfe/trunk@$CLANG_REV clang cd ../../llvm-build # Option 1: with cmake -MACOSX_DEPLOYMENT_TARGET=10.9 cmake -G Ninja -DCMAKE_BUILD_TYPE=Release \ +MACOSX_DEPLOYMENT_TARGET=10.9 cmake -G Ninja -DCMAKE_BUILD_TYPE=Release \ -DLLVM_ENABLE_ASSERTIONS=NO -DLLVM_ENABLE_THREADS=NO ../llvm/ time caffeinate ninja clang-format strip bin/clang-format @@ -91,11 +91,11 @@ Platform specific notes: Copy the binaries into your chromium checkout (under `src/buildtools/(win|linux64|mac)/clang-format(.exe?)`). For each binary, you'll need to run upload_to_google_storage.py according to the instructions in -[README.txt](/buildtools/clang_format/README.txt). This will upload the binary -into a publicly accessible google storage bucket, and update `.sha1` file in -your Chrome checkout. You'll check in the `.sha1` file (but NOT the clang-format -binary) into source control. In order to be able to upload, you'll need write -permission to the bucket -- see the prerequisites. +[README.txt](https://chromium.googlesource.com/chromium/buildtools/+/master/clang_format/README.txt). +This will upload the binary into a publicly accessible google storage bucket, +and update `.sha1` file in your Chrome checkout. You'll check in the `.sha1` +file (but NOT the clang-format binary) into source control. In order to be able +to upload, you'll need write permission to the bucket -- see the prerequisites. ## Copy the helper scripts and update README.chromium diff --git a/chromium/docs/useful_urls.md b/chromium/docs/useful_urls.md index 2ec7621abd6..fb2bbf3bd57 100644 --- a/chromium/docs/useful_urls.md +++ b/chromium/docs/useful_urls.md @@ -10,19 +10,19 @@ This page aims to be a repository of useful links that people may find useful. * [List of the last 100 potential LKGRs](http://chromium-status.appspot.com/revisions) * [Status dashboard for LKGR](https://build.chromium.org/p/chromium/lkgr-status/) * https://build.chromium.org/p/tryserver.chromium/waterfall?committer=developer@chromium.org : Try bot runs, by developer -* [Tree uptime stats](http://chromium-status.appspot.com/status_viewer) -* [Commit queue status](http://chromium-cq-status.appspot.com) +* [Tree uptime stats](https://chromium-status.appspot.com/status_viewer) +* [Commit queue status](https://chromium-cq-status.appspot.com) * [Pending commit queue jobs](https://codereview.chromium.org/search?closed=3&commit=2&limit=50) -* [Search for historical test failures by test name](http://chromium-build-logs.appspot.com/) -* [Filterable list of most recent build logs](http://chromium-build-logs.appspot.com/list) +* [Search for historical test failures by test name](https://chromium-build-logs.appspot.com/) +* [Filterable list of most recent build logs](https://chromium-build-logs.appspot.com/list) ## For Sheriffs * https://build.chromium.org/p/chromium.chromiumos/waterfall?show_events=true&reload=120&failures_only=true : List of failing bots for a waterfall(chromium.chromiumos as an example) * https://build.chromium.org/p/chromium.linux/waterfall?show_events=true&reload=120&builder=Linux%20Builder%20x64&builder=Linux%20Builder%20(dbg) : Monitor one or multiple bots(Linux Builder x64 and Linux Builder (dbg) on chromium.linux as an example) * https://build.chromium.org/p/chromium.win/waterfall/help : Customize the waterfall view for a waterfall(using chromium.win as an example) -* [Alternate waterfall view that helps with test failure triage](http://chromium-sheriffing.appspot.com) -* [Lists historical test results for the bots](http://test-results.appspot.com/dashboards/flakiness_dashboard.html) +* [Alternate waterfall view that helps with test failure triage](https://chromium-sheriffing.appspot.com) +* [Lists historical test results for the bots](https://test-results.appspot.com/dashboards/flakiness_dashboard.html) ## Release Information diff --git a/chromium/docs/user_handle_mapping.md b/chromium/docs/user_handle_mapping.md index 38c8608f11d..a155b3a334b 100644 --- a/chromium/docs/user_handle_mapping.md +++ b/chromium/docs/user_handle_mapping.md @@ -25,6 +25,7 @@ For Chromium contributors that have different nicks on other domains. | darin, fishd | fishd | darin | | dbeam | danbeam | dbeam | | dmurph | dmurph | dmurph | +| dougt | dougt | dft | | eaugusti | eriq | eaugustine | | erg | eglaysher | erg | | eroman | eroman | ericroman | diff --git a/chromium/docs/using_a_linux_chroot.md b/chromium/docs/using_a_linux_chroot.md index 96fadfd3654..da37cfe279b 100644 --- a/chromium/docs/using_a_linux_chroot.md +++ b/chromium/docs/using_a_linux_chroot.md @@ -80,7 +80,7 @@ file:///path/to/results.html, check the following conditions. 1. Make sure `DISPLAY` is set. See the - [Running X apps](#Running_X_apps) section above. + [Running X apps](#Running-X-apps) section above. 1. Install `xdg-utils`, which includes `xdg-open`, a utility for finding the right application to open a file or URL with. 1. Install [Chrome](https://www.google.com/intl/en/chrome/browser/). diff --git a/chromium/docs/vscode.md b/chromium/docs/vscode.md new file mode 100644 index 00000000000..474de314aa2 --- /dev/null +++ b/chromium/docs/vscode.md @@ -0,0 +1,87 @@ +# Use Visual Studio Code on Chromium code base + +[TOC] + +[Visual Studio Code](http://code.visualstudio.com/) +([Wikipedia](https://en.wikipedia.org/wiki/Visual_Studio_Code)) is a +multi-platform code editor that is itself based on Electron which is based on +Chromium. Visual Studio Code has a growing community and base of installable +extensions and themes. It works without too much setup. + +## Install extensions + +`ctrl+p` paste `ext install cpptools you-complete-me clang-format` then enter. +For more extensions: https://marketplace.visualstudio.com/search?target=vscode + +Highly recommend you also install your favorite keymap. + +An Example to install eclipse keymaps `ext install vscode-eclipse-keybindings`. +You can search keymaps here. +https://marketplace.visualstudio.com/search?target=vscode&category=Keymaps + + +## Settings + +Open Settings `File/Code - Preferences - Settings` and add the following +settings. + +``` +{ + "editor.tabSize": 2, + "editor.rulers": [80], + // Exclude + "files.exclude": { + "**/.git": true, + "**/.svn": true, + "**/.hg": true, + "**/.DS_Store": true, + "**/out": true + }, + // YCM + "ycmd.path": "<your_ycmd_path>", + "ycmd.global_extra_config": + "<your_chromium_path>/src/tools/vim/chromium.ycm_extra_conf.py", + "ycmd.confirm_extra_conf": false, + "ycmd.use_imprecise_get_type": true, + // clang-format + "clang-format.style": "Chromium", + "editor.formatOnSave": true +} +``` + +### Install auto-completion engine(ycmd) + +``` +$ git clone https://github.com/Valloric/ycmd.git ~/.ycmd +$ cd ~/.ycmd +$ ./build.py --clang-completer +``` + +## Work flow + +1. `ctrl+p` open file. +2. `ctrl+o` goto symbol. `ctrl+l` goto line. +3. <code>ctrl+`</code> toggle terminal. + +## Tips + +### On laptop + +Because we use ycmd to enable auto completion. We can disable CPP autocomplete +and index to save battery. + +``` +"C_Cpp.autocomplete": "Disabled", +"C_Cpp.addWorkspaceRootToIncludePath": false +``` + +### Enable Sublime-like minimap + +``` +"editor.minimap.enabled": true, +"editor.minimap.renderCharacters": false +``` + +### More + +https://github.com/Microsoft/vscode-tips-and-tricks/blob/master/README.md
\ No newline at end of file diff --git a/chromium/docs/vulcanize.md b/chromium/docs/vulcanize.md deleted file mode 100644 index bf0d877d6bb..00000000000 --- a/chromium/docs/vulcanize.md +++ /dev/null @@ -1,59 +0,0 @@ -# Vulcanizing Chrome Polymer UIs - -`vulcanize` is an npm module used to combine resources. In order to make the -Material Design downloads and history pages sufficiently fast, we run vulcanize -on the source files to combine them and reduce blocking load/parse time. - -## Required software - -Vulcanization currently requires: - -- node.js: >= v4.4.2 (can be found with `node --version`) -- npm: >= 1.3.10 (can be found with `npm --version`) -- vulcanize: 1.14.8 (can be found with `vulcanize --version`) -- crisper: 2.0.1 (can be found with `npm list -g crisper`) -- uglifyjs: 2.4.10 (can be found with `uglifyjs --version`) -- polymer-css-build: 0.0.6 (can be found with `npm list -g polymer-css-build`) - -## Installing required software - -For instructions on installing node and npm, see -[here](https://docs.npmjs.com/getting-started/installing-node). - -We recommend telling npm where to store downloaded modules: - -```bash -$ npm config set -g prefix "$HOME/node_modules" -``` - -Then install the required modules: - -```bash -$ npm install -g crisper@2.0.1 vulcanize@1.14.8 uglifyjs@2.4.10 \ - polymer-css-build@0.0.6 -``` - -Ultimately, all that is required to run this script is that the node binaries -listed above are on your $PATH. - -## Combining resources with vulcanize - -To combine all the CSS/HTML/JS for all pages which use vulcanize, making them -production fast, you can run the command: - -```bash -$ chrome/browser/resources/vulcanize.py # from src/ -``` - -This should overwrite the following files: - -- chrome/browser/resources/md_downloads/ - - vulcanized.html (all <link rel=import> and stylesheets inlined) - - crisper.js (all JavaScript, extracted from vulcanized.html) -- chrome/browser/resources/md_history/ - - app.vulcanized.html - - app.crisper.js - -## Testing downloads without vulcanizing - -Build with `use_vulcanize = false` in your gn args to build without vulcanizing. |