diff options
author | Allan Sandfeld Jensen <allan.jensen@theqtcompany.com> | 2016-05-09 14:22:11 +0200 |
---|---|---|
committer | Allan Sandfeld Jensen <allan.jensen@qt.io> | 2016-05-09 15:11:45 +0000 |
commit | 2ddb2d3e14eef3de7dbd0cef553d669b9ac2361c (patch) | |
tree | e75f511546c5fd1a173e87c1f9fb11d7ac8d1af3 /chromium/components/tracing/docs | |
parent | a4f3d46271c57e8155ba912df46a05559d14726e (diff) | |
download | qtwebengine-chromium-2ddb2d3e14eef3de7dbd0cef553d669b9ac2361c.tar.gz |
BASELINE: Update Chromium to 51.0.2704.41
Also adds in all smaller components by reversing logic for exclusion.
Change-Id: Ibf90b506e7da088ea2f65dcf23f2b0992c504422
Reviewed-by: Joerg Bornemann <joerg.bornemann@theqtcompany.com>
Diffstat (limited to 'chromium/components/tracing/docs')
6 files changed, 814 insertions, 0 deletions
diff --git a/chromium/components/tracing/docs/adding_memory_infra_tracing.md b/chromium/components/tracing/docs/adding_memory_infra_tracing.md new file mode 100644 index 00000000000..f60f65e7f5a --- /dev/null +++ b/chromium/components/tracing/docs/adding_memory_infra_tracing.md @@ -0,0 +1,178 @@ +# Adding MemoryInfra Tracing to a Component + +If you have a component that manages memory allocations, you should be +registering and tracking those allocations with Chrome's MemoryInfra system. +This lets you: + + * See an overview of your allocations, giving insight into total size and + breakdown. + * Understand how your allocations change over time and how they are impacted by + other parts of Chrome. + * Catch regressions in your component's allocations size by setting up + telemetry tests which monitor your allocation sizes under certain + circumstances. + +Some existing components that use MemoryInfra: + + * **Discardable Memory**: Tracks usage of discardable memory throughout Chrome. + * **GPU**: Tracks OpenGL and other GPU object allocations. + * **V8**: Tracks the heap size for JS. + +[TOC] + +## Overview + +In order to hook into Chrome's MemoryInfra system, your component needs to do +two things: + + 1. Create a [`MemoryDumpProvider`][mdp] for your component. + 2. Register and unregister you dump provider with the + [`MemoryDumpManager`][mdm]. + +[mdp]: https://chromium.googlesource.com/chromium/src/+/master/base/trace_event/memory_dump_provider.h +[mdm]: https://chromium.googlesource.com/chromium/src/+/master/base/trace_event/memory_dump_manager.h + +## Creating a Memory Dump Provider + +You can implement a [`MemoryDumpProvider`][mdp] as a stand-alone class, or as an +additional interface on an existing class. For example, this interface is +frequently implemented on classes which manage a pool of allocations (see +[`cc::ResourcePool`][resource-pool] for an example). + +A `MemoryDumpProvider` has one basic job, to implement `OnMemoryDump`. This +function is responsible for iterating over the resources allocated or tracked by +your component, and creating a [`MemoryAllocatorDump`][mem-alloc-dump] for each +using [`ProcessMemoryDump::CreateAllocatorDump`][pmd]. A simple example: + +```cpp +bool MyComponent::OnMemoryDump(const MemoryDumpArgs& args, + ProcessMemoryDump* process_memory_dump) { + for (const auto& allocation : my_allocations_) { + auto* dump = process_memory_dump->CreateAllocatorDump( + "path/to/my/component/allocation_" + allocation.id().ToString()); + dump->AddScalar(base::trace_event::MemoryAllocatorDump::kNameSize, + base::trace_event::MemoryAllocatorDump::kUnitsBytes, + allocation.size_bytes()); + + // While you will typically have a kNameSize entry, you can add additional + // entries to your dump with free-form names. In this example we also dump + // an object's "free_size", assuming the object may not be entirely in use. + dump->AddScalar("free_size", + base::trace_event::MemoryAllocatorDump::kUnitsBytes, + allocation.free_size_bytes()); + } +} +``` + +For many components, this may be all that is needed. See +[Handling Shared Memory Allocations](#Handling-Shared-Memory-Allocations) and +[Suballocations](#Suballocations) for information on more complex use cases. + +[resource-pool]: https://chromium.googlesource.com/chromium/src/+/master/cc/resources/resource_pool.h +[mem-alloc-dump]: https://chromium.googlesource.com/chromium/src/+/master/base/trace_event/memory_allocator_dump.h +[pmd]: https://chromium.googlesource.com/chromium/src/+/master/base/trace_event/process_memory_dump.h + +## Registering a Memory Dump Provider + +Once you have created a [`MemoryDumpProvider`][mdp], you need to register it +with the [`MemoryDumpManager`][mdm] before the system can start polling it for +memory information. Registration is generally straightforward, and involves +calling `MemoryDumpManager::RegisterDumpProvider`: + +```cpp +// Each process uses a singleton |MemoryDumpManager|. +base::trace_event::MemoryDumpManager::GetInstance()->RegisterDumpProvider( + my_memory_dump_provider_, my_single_thread_task_runner_); +``` + +In the above code, `my_memory_dump_provider_` is the `MemoryDumpProvider` +outlined in the previous section. `my_single_thread_task_runner_` is more +complex and may be a number of things: + + * Most commonly, if your component is always used from the main message loop, + `my_single_thread_task_runner_` may just be + [`base::ThreadTaskRunnerHandle::Get()`][task-runner-handle]. + * If your component already uses a custom `base::SingleThreadTaskRunner` for + executing tasks on a specific thread, you should likely use this runner. + +[task-runner-handle]: https://chromium.googlesource.com/chromium/src/+/master/base/thread_task_runner_handle.h + +## Unregistration + +Unregistration must happen on the thread belonging to the +`SingleThreadTaskRunner` provided at registration time. Unregistering on another +thread can lead to race conditions if tracing is active when the provider is +unregistered. + +```cpp +base::trace_event::MemoryDumpManager::GetInstance()->UnregisterDumpProvider( + my_memory_dump_provider_); +``` + +## Handling Shared Memory Allocations + +When an allocation is shared between two components, it may be useful to dump +the allocation in both components, but you also want to avoid double-counting +the allocation. This can be achieved using the concept of _ownership edges_. +An ownership edge represents that the _source_ memory allocator dump owns a +_target_ memory allocator dump. If multiple source dumps own a single target, +then the cost of that target allocation will be split between the sources. +Additionally, importance can be added to a specific ownership edge, allowing +the highest importance source of that edge to claim the entire cost of the +target. + +In the typical case, you will use [`ProcessMemoryDump`][pmd] to create a shared +global allocator dump. This dump will act as the target of all +component-specific dumps of a specific resource: + +```cpp +// Component 1 is going to create a dump, source_mad, for an allocation, +// alloc_, which may be shared with other components / processes. +MyAllocationType* alloc_; +base::trace_event::MemoryAllocatorDump* source_mad; + +// Component 1 creates and populates source_mad; +... + +// In addition to creating a source dump, we must create a global shared +// target dump. This dump should be created with a unique global ID which can be +// generated any place the allocation is used. I recommend adding a global ID +// generation function to the allocation type. +base::trace_event::MemoryAllocatorDumpGUID guid(alloc_->GetGUIDString()); + +// From this global ID we can generate the parent allocator dump. +base::trace_event::MemoryAllocatorDump* target_mad = + process_memory_dump->CreateSharedGlobalAllocatorDump(guid); + +// We now create an ownership edge from the source dump to the target dump. +// When creating an edge, you can assign an importance to this edge. If all +// edges have the same importance, the size of the allocation will be split +// between all sources which create a dump for the allocation. If one +// edge has higher importance than the others, its soruce will be assigned the +// full size of the allocation. +const int kImportance = 1; +process_memory_dump->AddOwnershipEdge( + source_mad->guid(), target_mad->guid(), kImportance); +``` + +If an allocation is being shared across process boundaries, it may be useful to +generate a global ID which incorporates the ID of the local process, preventing +two processes from generating colliding IDs. As it is not recommended to pass a +process ID between processes for security reasons, a function +`MemoryDumpManager::GetTracingProcessId` is provided which generates a unique ID +per process that can be passed with the resource without security concerns. +Frequently this ID is used to generate a global ID that is based on the +allocated resource's ID combined with the allocating process' tracing ID. + +## Suballocations + +Another advanced use case involves tracking sub-allocations of a larger +allocation. For instance, this is used in +[`gpu::gles2::TextureManager`][texture-manager] to dump both the suballocations +which make up a texture. To create a suballocation, instead of calling +[`ProcessMemoryDump::CreateAllocatorDump`][pmd] to create a +[`MemoryAllocatorDump`][mem-alloc-dump], you call +[`ProcessMemoryDump::AddSubAllocation`][pmd], providing the ID of the parent +allocation as the first parameter. + +[texture-manager]: https://chromium.googlesource.com/chromium/src/+/master/gpu/command_buffer/service/texture_manager.cc diff --git a/chromium/components/tracing/docs/heap_profiler.md b/chromium/components/tracing/docs/heap_profiler.md new file mode 100644 index 00000000000..bb46dc9bd83 --- /dev/null +++ b/chromium/components/tracing/docs/heap_profiler.md @@ -0,0 +1,114 @@ +# Heap Profiling with MemoryInfra + +As of Chrome 48, MemoryInfra supports heap profiling. The core principle is +a solution that JustWorks™ on all platforms without patching or rebuilding, +intergrated with the chrome://tracing ecosystem. + +[TOC] + +## How to Use + + 1. Start Chrome with the `--enable-heap-profiling` switch. This will make + Chrome keep track of all allocations. + + 2. Grab a [MemoryInfra][memory-infra] trace. For best results, start tracing + first, and _then_ open a new tab that you want to trace. Furthermore, + enabling more categories (besides memory-infra) will yield more detailed + information in the heap profiler backtraces. + + 3. When the trace has been collected, select a heavy memory dump indicated by + a purple ![M][m-purple] dot. Heap dumps are only included in heavy memory + dumps. + + 4. In the analysis view, cells marked with a triple bar icon (☰) contain heap + dumps. Select such a cell. + + ![Cells containing a heap dump][cells-heap-dump] + + 5. Scroll down all the way to _Heap Details_. + + 6. Pinpoint the memory bug and live happily ever after. + +[memory-infra]: memory_infra.md +[m-purple]: https://storage.googleapis.com/chromium-docs.appspot.com/d7bdf4d16204c293688be2e5a0bcb2bf463dbbc3 +[cells-heap-dump]: https://storage.googleapis.com/chromium-docs.appspot.com/a24d80d6a08da088e2e9c8b2b64daa215be4dacb + +## Heap Details + +The heap details view contains a tree that represents the heap. The size of the +root node corresponds to the selected allocator cell. + +*** aside +The size value in the heap details view will not match the value in the selected +analysis view cell exactly. There are three reasons for this. First, the heap +profiler reports the memory that _the program requested_, whereas the allocator +reports the memory that it _actually allocated_ plus its own bookkeeping +overhead. Second, allocations that happen early --- before Chrome knows that +heap profiling is enabled --- are not captured by the heap profiler, but they +are reported by the allocator. Third, tracing overhead is not discounted by the +heap profiler. +*** + +The heap can be broken down in two ways: by _backtrace_ (marked with an ƒ), and +by _type_ (marked with a Ⓣ). When tracing is enabled, Chrome records trace +events, most of which appear in the flame chart in timeline view. At every +point in time these trace events form a pseudo stack, and a vertical slice +through the flame chart is like a backtrace. This corresponds to the ƒ nodes in +the heap details view. Hence enabling more tracing categories will give a more +detailed breakdown of the heap. + +The other way to break down the heap is by object type. At the moment this is +only supported for PartitionAlloc. + +*** aside +In official builds, only the most common type names are included due to binary +size concerns. Development builds have full type information. +*** + +To keep the trace log small, uninteresting information is omitted from heap +dumps. The long tail of small nodes is not dumped, but grouped in an `<other>` +node instead. Note that altough these small nodes are insignificant on their +own, together they can be responsible for a significant portion of the heap. The +`<other>` node is large in that case. + +## Example + +In the trace below, `ParseAuthorStyleSheet` is called at some point. + +![ParseAuthorStyleSheet pseudo stack][pseudo-stack] + +The pseudo stack of trace events corresponds to the tree of ƒ nodes below. Of +the 23.5 MiB of memory allocated with PartitionAlloc, 1.9 MiB was allocated +inside `ParseAuthorStyleSheet`, either directly, or at a deeper level (like +`CSSParserImpl::parseStyleSheet`). + +![Memory Allocated in ParseAuthorStyleSheet][break-down-by-backtrace] + +By expanding `ParseAuthorStyleSheet`, we can see which types were allocated +there. Of the 1.9 MiB, 371 KiB was spent on `ImmutableStylePropertySet`s, and +238 KiB was spent on `StringImpl`s. + +![ParseAuthorStyleSheet broken down by type][break-down-by-type] + +It is also possible to break down by type first, and then by backtrace. Below +we see that of the 23.5 MiB allocated with PartitionAlloc, 1 MiB is spent on +`Node`s, and about half of the memory spent on nodes was allocated in +`HTMLDocumentParser`. + +![The PartitionAlloc heap broken down by type first and then by backtrace][type-then-backtrace] + +Heap dump diffs are fully supported by trace viewer. Select a heavy memory dump +(a purple dot), then with the control key select a heavy memory dump earlier in +time. Below is a diff of theverge.com before and in the middle of loading ads. +We can see that 4 MiB were allocated when parsing the documents in all those +iframes, almost a megabyte of which was due to JavaScript. (Note that this is +memory allocated by PartitionAlloc alone, the total renderer memory increase was +around 72 MiB.) + +![Diff of The Verge before and after loading ads][diff] + +[pseudo-stack]: https://storage.googleapis.com/chromium-docs.appspot.com/058e50350836f55724e100d4dbbddf4b9803f550 +[break-down-by-backtrace]: https://storage.googleapis.com/chromium-docs.appspot.com/ec61c5f15705f5bcf3ca83a155ed647a0538bbe1 +[break-down-by-type]: https://storage.googleapis.com/chromium-docs.appspot.com/2236e61021922c0813908c6745136953fa20a37b +[type-then-backtrace]: https://storage.googleapis.com/chromium-docs.appspot.com/c5367dde11476bdbf2d5a1c51674148915573d11 +[diff]: https://storage.googleapis.com/chromium-docs.appspot.com/802141906869cd533bb613da5f91bd0b071ceb24 diff --git a/chromium/components/tracing/docs/heap_profiler_internals.md b/chromium/components/tracing/docs/heap_profiler_internals.md new file mode 100644 index 00000000000..d1019c8d212 --- /dev/null +++ b/chromium/components/tracing/docs/heap_profiler_internals.md @@ -0,0 +1,184 @@ +# Heap Profiler Internals + +This document describes how the heap profiler works and how to add heap +profiling support to your allocator. If you just want to know how to use it, +see [Heap Profiling with MemoryInfra](heap_profiler.md) + +[TOC] + +## Overview + +The heap profiler consists of tree main components: + + * **The Context Tracker**: Responsible for providing context (pseudo stack + backtrace) when an allocation occurs. + * **The Allocation Register**: A specialized hash table that stores allocation + details by address. + * **The Heap Dump Writer**: Extracts the most important information from a set + of recorded allocations and converts it into a format that can be dumped into + the trace log. + +These components are designed to work well together, but to be usable +independently as well. + +When there is a way to get notified of all allocations and frees, this is the +normal flow: + + 1. When an allocation occurs, call + [`AllocationContextTracker::GetInstanceForCurrentThread()->GetContextSnapshot()`][context-tracker] + to get an [`AllocationContext`][alloc-context]. + 2. Insert that context together with the address and size into an + [`AllocationRegister`][alloc-register] by calling `Insert()`. + 3. When memory is freed, remove it from the register with `Remove()`. + 4. On memory dump, collect the allocations from the register, call + [`ExportHeapDump()`][export-heap-dump], and add the generated heap dump to + the memory dump. + +[context-tracker]: https://chromium.googlesource.com/chromium/src/+/master/base/trace_event/heap_profiler_allocation_context_tracker.h +[alloc-context]: https://chromium.googlesource.com/chromium/src/+/master/base/trace_event/heap_profiler_allocation_context.h +[alloc-register]: https://chromium.googlesource.com/chromium/src/+/master/base/trace_event/heap_profiler_allocation_register.h +[export-heap-dump]: https://chromium.googlesource.com/chromium/src/+/master/base/trace_event/heap_profiler_heap_dump_writer.h + +*** aside +An allocator can skip step 2 and 3 if it is able to store the context itself, +and if it is able to enumerate all allocations for step 4. +*** + +When heap profiling is enabled (the `--enable-heap-profiling` flag is passed), +the memory dump manager calls `OnHeapProfilingEnabled()` on every +`MemoryDumpProvider` as early as possible, so allocators can start recording +allocations. This should be done even when tracing has not been started, +because these allocations might still be around when a heap dump happens during +tracing. + +## Context Tracker + +The [`AllocationContextTracker`][context-tracker] is a thread-local object. Its +main purpose is to keep track of a pseudo stack of trace events. Chrome has +been instrumented with lots of `TRACE_EVENT` macros. These trace events push +their name to a thread-local stack when they go into scope, and pop when they +go out of scope, if all of the following conditions have been met: + + * A trace is being recorded. + * The category of the event is enabled in the trace config. + * Heap profiling is enabled (with the `--enable-heap-profiling` flag). + +This means that allocations that occur before tracing is started will not have +backtrace information in their context. + +A thread-local instance of the context tracker is initialized lazily when it is +first accessed. This might be because a trace event pushed or popped, or because +`GetContextSnapshot()` was called when an allocation occurred. + +[`AllocationContext`][alloc-context] is what is used to group and break down +allocations. Currently `AllocationContext` has the following fields: + + * Backtrace: filled by the context tracker, obtained from the thread-local + pseudo stack. + * Type name: to be filled in at a point where the type of a pointer is known, + set to _[unknown]_ by default. + +It is possible to modify this context after insertion into the register, for +instance to set the type name if it was not known at the time of allocation. + +## Allocation Register + +The [`AllocationRegister`][alloc-register] is a hash table specialized for +storing `(size, AllocationContext)` pairs by address. It has been optimized for +Chrome's typical number of unfreed allocations, and it is backed by `mmap` +memory directly so there are no reentrancy issues when using it to record +`malloc` allocations. + +The allocation register is threading-agnostic. Access must be synchronised +properly. + +## Heap Dump Writer + +Dumping every single allocation in the allocation register straight into the +trace log is not an option due to the sheer volume (~300k unfreed allocations). +The role of the [`ExportHeapDump()`][export-heap-dump] function is to group +allocations, striking a balance between trace log size and detail. + +See the [Heap Dump Format][heap-dump-format] document for more details about the +structure of the heap dump in the trace log. + +[heap-dump-format]: https://docs.google.com/document/d/1NqBg1MzVnuMsnvV1AKLdKaPSPGpd81NaMPVk5stYanQ + +## Instrumenting an Allocator + +Below is an example of adding heap profiling support to an allocator that has +an existing memory dump provider. + +```cpp +class FooDumpProvider : public MemoryDumpProvider { + + // Kept as pointer because |AllocationRegister| allocates a lot of virtual + // address space when constructed, so only construct it when heap profiling is + // enabled. + scoped_ptr<AllocationRegister> allocation_register_; + Lock allocation_register_lock_; + + static FooDumpProvider* GetInstance(); + + void InsertAllocation(void* address, size_t size) { + AllocationContext context = AllocationContextTracker::GetInstanceForCurrentThread()->GetContextSnapshot(); + AutoLock lock(allocation_register_lock_); + allocation_register_->Insert(address, size, context); + } + + void RemoveAllocation(void* address) { + AutoLock lock(allocation_register_lock_); + allocation_register_->Remove(address); + } + + // Will be called as early as possible by the memory dump manager. + void OnHeapProfilingEnabled(bool enabled) override { + AutoLock lock(allocation_register_lock_); + allocation_register_.reset(new AllocationRegister()); + + // At this point, make sure that from now on, for every allocation and + // free, |FooDumpProvider::GetInstance()->InsertAllocation()| and + // |RemoveAllocation| are called. + } + + bool OnMemoryDump(const MemoryDumpArgs& args, + ProcessMemoryDump& pmd) override { + // Do regular dumping here. + + // Dump the heap only for detailed dumps. + if (args.level_of_detail == MemoryDumpLevelOfDetail::DETAILED) { + TraceEventMemoryOverhead overhead; + hash_map<AllocationContext, size_t> bytes_by_context; + + { + AutoLock lock(allocation_register_lock_); + if (allocation_register_) { + // Group allocations in the register into |bytes_by_context|, but do + // no additional processing inside the lock. + for (const auto& alloc_size : *allocation_register_) + bytes_by_context[alloc_size.context] += alloc_size.size; + + allocation_register_->EstimateTraceMemoryOverhead(&overhead); + } + } + + if (!bytes_by_context.empty()) { + scoped_refptr<TracedValue> heap_dump = ExportHeapDump( + bytes_by_context, + pmd->session_state()->stack_frame_deduplicator(), + pmb->session_state()->type_name_deduplicator()); + pmd->AddHeapDump("foo_allocator", heap_dump); + overhead.DumpInto("tracing/heap_profiler", pmd); + } + } + + return true; + } +}; + +``` + +*** aside +The implementation for `malloc` is more complicated because it needs to deal +with reentrancy. +*** diff --git a/chromium/components/tracing/docs/memory_infra.md b/chromium/components/tracing/docs/memory_infra.md new file mode 100644 index 00000000000..085f4515139 --- /dev/null +++ b/chromium/components/tracing/docs/memory_infra.md @@ -0,0 +1,171 @@ +# MemoryInfra + +MemoryInfra is a timeline-based profiling system integrated in chrome://tracing. +It aims at creating Chrome-scale memory measurement tooling so that on any +Chrome in the world --- desktop, mobile, Chrome OS or any other --- with the +click of a button you can understand where memory is being used in your system. + +[TOC] + +## Getting Started + + 1. Get a bleeding-edge or tip-of-tree build of Chrome. + + 2. [Record a trace as usual][record-trace]: open [chrome://tracing][tracing] + on Desktop Chrome or [chrome://inspect?tracing][inspect-tracing] to trace + Chrome for Android. + + 3. Make sure to enable the **memory-infra** category on the right. + + ![Tick the memory-infra checkbox when recording a trace.][memory-infra-box] + + 4. For now, some subsystems only work if Chrome is started with the + `--no-sandbox` flag. + <!-- TODO(primiano) TODO(ssid): https://crbug.com/461788 --> + +[record-trace]: https://sites.google.com/a/chromium.org/dev/developers/how-tos/trace-event-profiling-tool/recording-tracing-runs +[tracing]: chrome://tracing +[inspect-tracing]: chrome://inspect?tracing +[memory-infra-box]: https://storage.googleapis.com/chromium-docs.appspot.com/1c6d1886584e7cc6ffed0d377f32023f8da53e02 + +![Timeline View and Analysis View][tracing-views] + +After recording a trace, you will see the **timeline view**. Timeline view +shows: + + * Total resident memory grouped by process (at the top). + * Total resident memory grouped by subsystem (at the top). + * Allocated memory per subsystem for every process. + +Click one of the ![M][m-blue] dots to bring up the **analysis view**. Click +on a cell in analysis view to reveal more information about its subsystem. +PartitionAlloc for instance, has more details about its partitions. + +![Component details for PartitionAlloc][partalloc-details] + +The purple ![M][m-purple] dots represent heavy dumps. In these dumps, components +can provide more details than in the regular dumps. The full details of the +MemoryInfra UI are explained in its [design doc][mi-ui-doc]. + +[tracing-views]: https://storage.googleapis.com/chromium-docs.appspot.com/db12015bd262385f0f8bd69133330978a99da1ca +[m-blue]: https://storage.googleapis.com/chromium-docs.appspot.com/b60f342e38ff3a3767bbe4c8640d96a2d8bc864b +[partalloc-details]: https://storage.googleapis.com/chromium-docs.appspot.com/02eade61d57c83f8ef8227965513456555fc3324 +[m-purple]: https://storage.googleapis.com/chromium-docs.appspot.com/d7bdf4d16204c293688be2e5a0bcb2bf463dbbc3 +[mi-ui-doc]: https://docs.google.com/document/d/1b5BSBEd1oB-3zj_CBAQWiQZ0cmI0HmjmXG-5iNveLqw/edit + +## Columns + +**Columns in blue** reflect the amount of actual physical memory used by the +process. This is what exerts memory pressure on the system. + + * **Total Resident**: (TODO: document this). + * **Peak Total Resident**: (TODO: document this). + * **PSS**: (TODO: document this). + * **Private Dirty**: (TODO: document this). + * **Swapped**: (TODO: document this). + +**Columns in black** reflect a best estimation of the the amount of physical +memory used by various subsystems of Chrome. + + * **Blink GC**: Memory used by [Oilpan][oilpan]. + * **CC**: Memory used by the compositor. + See [cc/memory][cc-memory] for the full details. + * **Discardable**: (TODO: document this). + * **Font Caches**: (TODO: document this). + * **GPU** and **GPU Memory Buffer**: GPU memory and RAM used for GPU purposes. + See [GPU Memory Tracing][gpu-memory]. + * **LevelDB**: (TODO: document this). + * **Malloc**: Memory allocated by calls to `malloc`, or `new` for most + non-Blink objects. + * **PartitionAlloc**: Memory allocated via [PartitionAlloc][partalloc]. + Blink objects that are not managed by Oilpan are allocated with + PartitionAlloc. + * **Skia**: (TODO: document this). + * **SQLite**: (TODO: document this). + * **V8**: (TODO: document this). + * **Web Cache**: (TODO: document this). + +The **tracing column in gray** reports memory that is used to collect all of the +above information. This memory would not be used if tracing were not enabled, +and it is discounted from malloc and the blue columns. + +<!-- TODO(primiano): Improve this. https://crbug.com/??? --> + +[oilpan]: /third_party/WebKit/Source/platform/heap/BlinkGCDesign.md +[cc-memory]: /cc/memory.md +[gpu-memory]: memory_infra_gpu.md +[partalloc]: /third_party/WebKit/Source/wtf/PartitionAlloc.md + +## Related Pages + + * [Adding MemoryInfra Tracing to a Component](adding_memory_infra_tracing.md) + * [GPU Memory Tracing](memory_infra_gpu.md) + * [Heap Profiler Internals](heap_profiler_internals.md) + * [Heap Profiling with MemoryInfra](heap_profiler.md) + * [Startup Tracing with MemoryInfra](memory_infra_startup_tracing.md) + +## Rationale + +Another memory profiler? What is wrong with tool X? +Most of the existing tools: + + * Are hard to get working with Chrome. (Massive symbols, require OS-specific + tricks.) + * Lack Chrome-related context. + * Don't deal with multi-process scenarios. + +MemoryInfra leverages the existing tracing infrastructure in Chrome and provides +contextual data: + + * **It speaks Chrome slang.** + The Chromium codebase is instrumented. Its memory subsystems (allocators, + caches, etc.) uniformly report their stats into the trace in a way that can + be understood by Chrome developers. No more + `__gnu_cxx::new_allocator< std::_Rb_tree_node< std::pair< std::string const, base::Value*>>> ::allocate`. + * **Timeline data that can be correlated with other events.** + Did memory suddenly increase during a specific Blink / V8 / HTML parsing + event? Which subsystem increased? Did memory not go down as expected after + closing a tab? Which other threads were active during a bloat? + * **Works out of the box on desktop and mobile.** + No recompilations with unmaintained `GYP_DEFINES`, no time-consuming + symbolizations stages. All the logic is already into Chrome, ready to dump at + any time. + * **The same technology is used for telemetry and the ChromePerf dashboard.** + See [the slides][chromeperf-slides] and take a look at + [some ChromePerf dashboards][chromeperf] and + [telemetry documentation][telemetry]. + +[chromeperf-slides]: https://docs.google.com/presentation/d/1OyxyT1sfg50lA36A7ibZ7-bBRXI1kVlvCW0W9qAmM_0/present?slide=id.gde150139b_0_137 +[chromeperf]: https://chromeperf.appspot.com/report?sid=3b54e60c9951656574e19252fadeca846813afe04453c98a49136af4c8820b8d +[telemetry]: https://catapult.gsrc.io/telemetry + +## Development + +MemoryInfra is based on a simple and extensible architecture. See +[the slides][dp-slides] on how to get your subsystem reported in MemoryInfra, +or take a look at one of the existing examples such as +[malloc_dump_provider.cc][malloc-dp]. The crbug label is +[Hotlist-MemoryInfra][hotlist]. Don't hesitate to contact +[tracing@chromium.org][mailtracing] for questions and support. + +[dp-slides]: https://docs.google.com/presentation/d/1GI3HY3Mm5-Mvp6eZyVB0JiaJ-u3L1MMJeKHJg4lxjEI/present?slide=id.g995514d5c_1_45 +[malloc-dp]: https://chromium.googlesource.com/chromium/src.git/+/master/base/trace_event/malloc_dump_provider.cc +[hotlist]: https://code.google.com/p/chromium/issues/list?q=label:Hotlist-MemoryInfra +[mailtracing]: mailto:tracing@chromium.org + +## Design documents + +Architectural: + +<iframe width="100%" height="300px" src="https://docs.google.com/a/google.com/embeddedfolderview?id=0B3KuDeqD-lVJfmp0cW1VcE5XVWNxZndxelV5T19kT2NFSndYZlNFbkFpc3pSa2VDN0hlMm8"> +</iframe> + +Chrome-side design docs: + +<iframe width="100%" height="300px" src="https://docs.google.com/a/google.com/embeddedfolderview?id=0B3KuDeqD-lVJfndSa2dleUQtMnZDeWpPZk1JV0QtbVM5STkwWms4YThzQ0pGTmU1QU9kNVk"> +</iframe> + +Catapult-side design docs: + +<iframe width="100%" height="300px" src="https://docs.google.com/a/google.com/embeddedfolderview?id=0B3KuDeqD-lVJfm10bXd5YmRNWUpKOElOWS0xdU1tMmV1S3F4aHo0ZDJLTmtGRy1qVnQtVWM"> +</iframe> diff --git a/chromium/components/tracing/docs/memory_infra_gpu.md b/chromium/components/tracing/docs/memory_infra_gpu.md new file mode 100644 index 00000000000..57f49a48d54 --- /dev/null +++ b/chromium/components/tracing/docs/memory_infra_gpu.md @@ -0,0 +1,93 @@ +# GPU Memory Tracing + +This is an overview of the GPU column in [MemoryInfra][memory-infra]. + +[TOC] + +## Quick Start + +If you want an overview of total GPU memory usage, select the GPU process' GPU +category and look at the _size_ column. (Not _effective size_.) + +![Look at the size column for total GPU memory][gpu-size-column] + +[memory-infra]: memory_infra.md +[gpu-size-column]: https://storage.googleapis.com/chromium-docs.appspot.com/c7d632c18d90d99e393ad0ade929f96e7d8243fe + +## In Depth + +GPU Memory in Chrome involves several different types of allocations. These +include, but are not limited to: + + * **Raw OpenGL Objects**: These objects are allocated by Chrome using the + OpenGL API. Chrome itself has handles to these objects, but the actual + backing memory may live in a variety of places (CPU side in the GPU process, + CPU side in the kernel, GPU side). Because most OpenGL operations occur over + IPC, communicating with Chrome's GPU process, these allocations are almost + always shared between a renderer or browser process and the GPU process. + * **GPU Memory Buffers**: These objects provide a chunk of writable memory + which can be handed off cross-process. While GPUMemoryBuffers represent a + platform-independent way to access this memory, they have a number of + possible platform-specific implementations (EGL surfaces on Linux, + IOSurfaces on Mac, or CPU side shared memory). Because of their cross + process use case, these objects will almost always be shared between a + renderer or browser process and the GPU process. + * **GLImages**: GLImages are a platform-independent abstraction around GPU + memory, similar to GPU Memory Buffers. In many cases, GLImages are created + from GPUMemoryBuffers. The primary difference is that GLImages are designed + to be bound to an OpenGL texture using the image extension. + +GPU Memory can be found across a number of different processes, in a few +different categories. + +Renderer or browser process: + + * **CC Category**: The CC category contains all resource allocations used in + the Chrome Compositor. When GPU rasterization is enabled, these resource + allocations will be GPU allocations as well. See also + [cc/memory][cc-memory]. + * **Skia/gpu_resources Category**: All GPU resources used by Skia. + * **GPUMemoryBuffer Category**: All GPUMemoryBuffers in use in the current + process. + +GPU process: + + * **GPU Category**: All GPU allocations, many shared with other processes. + * **GPUMemoryBuffer Category**: All GPUMemoryBuffers. + +## Example + +Many of the objects listed above are shared between multiple processes. +Consider a GL texture used by CC --- this texture is shared between a renderer +and the GPU process. Additionally, the texture may be backed by a GLImage which +was created from a GPUMemoryBuffer, which is also shared between the renderer +and GPU process. This means that the single texture may show up in the memory +logs of two different processes multiple times. + +To make things easier to understand, each GPU allocation is only ever "owned" +by a single process and category. For instance, in the above example, the +texture would be owned by the CC category of the renderer process. Each +allocation has (at least) two sizes recorded --- _size_ and _effective size_. +In the owning allocation, these two numbers will match: + +![Matching size and effective size][owner-size] + +Note that the allocation also gives information on what other processes it is +shared with (seen by hovering over the green arrow). If we navigate to the +other allocation (in this case, gpu/gl/textures/client_25/texture_216) we will +see a non-owning allocation. In this allocation the size is the same, but the +_effective size_ is 0: + +![Effective size of zero][non-owner-size] + +Other types, such as GPUMemoryBuffers and GLImages have similar sharing +patterns. + +When trying to get an overview of the absolute memory usage tied to the GPU, +you can look at the size column (not effective size) of just the GPU process' +GPU category. This will show all GPU allocations, whether or not they are owned +by another process. + +[cc-memory]: /cc/memory.md +[owner-size]: https://storage.googleapis.com/chromium-docs.appspot.com/a325c4426422e53394a322d31b652cfa34231189 +[non-owner-size]: https://storage.googleapis.com/chromium-docs.appspot.com/b8cf464636940d0925f29a102e99aabb9af40b13 diff --git a/chromium/components/tracing/docs/memory_infra_startup_tracing.md b/chromium/components/tracing/docs/memory_infra_startup_tracing.md new file mode 100644 index 00000000000..138be30933c --- /dev/null +++ b/chromium/components/tracing/docs/memory_infra_startup_tracing.md @@ -0,0 +1,74 @@ +# Startup Tracing with MemoryInfra + +[MemoryInfra](memory_infra.md) supports startup tracing. + +## The Simple Way + +Start Chrome as follows: + + $ chrome --no-sandbox \ + --trace-startup=-*,disabled-by-default-memory-infra \ + --trace-startup-file=/tmp/trace.json \ + --trace-startup-duration=7 + +On Android, enable startup tracing and start Chrome as follows: + + $ build/android/adb_chrome_public_command_line \ + --trace-startup=-*,disabled-by-default-memory-infra \ + --trace-startup-file=/sdcard/Download/trace.json \ + --trace-startup-duration=7 + + $ build/android/adb_run_chrome_public + + $ adb pull /sdcard/Download/trace.json # After tracing. + +Note that startup tracing will be enabled upon every Chrome launch until you +delete the command-line flags: + + $ build/android/adb_chrome_public_command_line "" + +This will use the default configuration: one memory dump every 250 ms with a +detailed dump ever two seconds. + +## The Advanced Way + +If you need more control over the granularity of the memory dumps, you can +specify a custom trace config file as follows: + + $ cat > /tmp/trace.config + { + "startup_duration": 4, + "result_file": "/tmp/trace.json", + "trace_config": { + "included_categories": ["disabled-by-default-memory-infra"], + "excluded_categories": ["*"], + "memory_dump_config": { + "triggers": [ + { "mode": "light", "periodic_interval_ms": 50 }, + { "mode": "detailed", "periodic_interval_ms": 1000 } + ] + } + } + } + + $ chrome --no-sandbox --trace-config-file=/tmp/trace.config + +On Android, the config file has to be pushed to a fixed file location: + + $ adb root + $ adb push /tmp/trace.config /data/local/chrome-trace-config.json + + $ build/android/adb_run_chrome_public + + $ adb pull /sdcard/Download/trace.json # After tracing. + +Make sure that the "result_file" location is writable by the Chrome process on +Android (e.g. "/sdcard/Download/trace.json"). Note that startup tracing will be +enabled upon every Chrome launch until you delete the config file: + + $ adb shell rm /data/local/chrome-trace-config.json + +## Related Pages + + * [General information about startup tracing](https://sites.google.com/a/chromium.org/dev/developers/how-tos/trace-event-profiling-tool/recording-tracing-runs) + * [Memory tracing with MemoryInfra](memory_infra.md) |