summaryrefslogtreecommitdiff
path: root/.gitlab-ci/tracie/README.md
diff options
context:
space:
mode:
Diffstat (limited to '.gitlab-ci/tracie/README.md')
-rw-r--r--.gitlab-ci/tracie/README.md126
1 files changed, 126 insertions, 0 deletions
diff --git a/.gitlab-ci/tracie/README.md b/.gitlab-ci/tracie/README.md
new file mode 100644
index 00000000000..5c1f8a881ac
--- /dev/null
+++ b/.gitlab-ci/tracie/README.md
@@ -0,0 +1,126 @@
+Tracie - Mesa Traces Continuous Integration System
+==================================================
+
+Home of the Mesa trace testing effort.
+
+### Traces definition file
+
+The trace definition file contains information about the git repo/commit to get
+the traces from, and a list of the traces to run along with their expected image
+checksums on each device. An example:
+
+```yaml
+traces-db:
+ repo: https://gitlab.freedesktop.org/gfx-ci/tracie/traces-db
+ commit: master
+
+traces:
+ - path: glmark2/jellyfish.rdc
+ expectations:
+ - device: intel-0x3185
+ checksum: 58359ea4caf6ad44c6b65526881bbd17
+ - device: vmware-llvmpipe
+ checksum: d82267c25a0decdad7b563c56bb81106
+ - path: supertuxkart/supertuxkart-antediluvian-abyss.rdc
+ expectations:
+ - device: intel-0x3185
+ checksum: ff827f7eb069afd87cc305a422cba939
+```
+
+The traces-db entry can be absent, in which case it is assumed that the
+current directory is the traces-db directory.
+
+Traces that don't have an expectation for the current device are skipped
+during trace replay.
+
+Adding a new trace to the list involves commiting the trace to the git repo and
+adding an entry to the `traces` list. The reference checksums can be calculated
+with the [image_checksum.py](.gitlab-ci/tracie/image_checksum.py) script.
+Alternatively, an arbitrary checksum can be used, and during replay (see below)
+the scripts will report the mismatch and expected checksum.
+
+### Trace-db repos
+
+The trace-db repos are assumed to be git repositories using LFS for their trace
+files. This is so that trace files can be potentially checked out and replayed
+individually, thus reducing storage requirements during CI runs.
+
+### Enabling trace testing on a new device
+
+To enable trace testing on a new device:
+
+1. Create a new job in .gitlab-ci.yml. The job will need to be tagged
+ to run on runners with the appropriate hardware. Use the `.traces-test`
+ template job as a base, and make sure you set a unique value for the
+ `DEVICE_NAME` variable:
+
+ ```yaml
+ my-hardware-traces:
+ variables:
+ DEVICE_NAME: "myhardware"
+ extends: .traces-test
+ ```
+
+2. Update the .gitlab-ci/traces.yml file with expectations for the new device.
+ Ensure that the device name used in the expectations matches the one
+ set in the job. For more information, and tips about how to calculate
+ the checksums, see the section describing the trace definition files.
+
+### Trace files
+
+Tracie supports both renderdoc (.rdc) and apitrace (.trace) files. Trace files
+need to have the correct extension so that tracie can detect them properly.
+
+The trace files that are contained in public traces-db repositories must be
+legally redistributable. This is typically true for FOSS games and
+applications. Traces for proprietary games and application are typically not
+redistributable, unless specific redistribution rights have been granted by the
+publisher.
+
+### Replaying traces
+
+Mesa traces CI uses a set of scripts to replay traces and check the output
+against reference checksums.
+
+The high level script [tracie.sh](.gitlab-ci/tracie/tracie.sh) accepts
+a traces definition file and the type of traces (apitrace/renderdoc) to run:
+
+ tracie.sh .gitlab-ci/traces.yml renderdoc
+
+tracie.sh copies produced artifacts to the `$CI_PROJECT_DIR/result`
+directory. By default, created images from traces are only stored in case of a
+checksum mismatch. The `TRACIE_STORE_IMAGES` CI/environment variable can be set
+to `1` to force storing images, e.g., to get a complete set of reference
+images.
+
+The `tracie.sh` script requires that the environment variable `DEVICE_NAME` is
+properly set for the target machine, and matches the `device` field of the
+relevant trace expectations in the used `traces.yml` file.
+
+At a lower level the
+[dump_trace_images.py](.gitlab-ci/tracie/dump_trace_images.py) script is
+called, which replays a trace, dumping a set of images in the process. By
+default only the image corresponding to the last frame of the trace is dumped,
+but this can be changed with the `--calls` parameter. The dumped images are
+stored in a subdirectory `test/<device-name>` next to the trace file itself,
+with names of the form `tracefilename-callnum.png`. The full log of any
+commands used while dumping the images is also saved in a file in the
+'test/<device-name>' subdirectory, named after the trace name with '.log'
+appended.
+
+Examples:
+
+ python3 dump_traces_images.py --device-name=vmware-llvmpipe mytrace.trace
+ python3 dump_traces_images.py --device-name=vmware-llvmpipe --calls=2075,3300 mytrace.trace
+
+### Running the replay scripts locally
+
+It's often useful, especially during development, to be able to run the scripts
+locally. The scripts require a recent version of apitrace being in the path,
+and also the renderdoc python module being available.
+
+To ensure python3 can find the renderdoc python module you need to set
+`PYTHONPATH` to point to the location of `renderdoc.so` (binary python modules)
+and `LD_LIBRARY_PATH` to point to the location of `librenderdoc.so`. In the
+renderdoc build tree, both of these are in `renderdoc/<builddir>/lib`. Note
+that renderdoc doesn't install the `renderdoc.so` python module.