CONTRIBUTING.rst: Updated to reflect running tests using tox.

1 files changed, 40 insertions, 12 deletions

diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 12f61fc5f..e80fd3af4 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -1478,48 +1478,76 @@ Don't get lost in the docs if you don't need to, follow existing examples instea
 
 Running tests
 ~~~~~~~~~~~~~
-To run the tests, just type::
+We use `tox <https://tox.readthedocs.org/>`_ as a frontend run the tests which
+are implemented using `pytest <https://pytest.org/>`_. To run the tests, simply
+navigate to the toplevel directory of your buildstream checkout and run::
 
-  ./setup.py test
+  tox
 
-At the toplevel.
+By default, the test suite will be run against every supported python version
+found on your host. If you have multiple python versions installed, you may
+want to run tests against only one version and you can do that using the ``-e``
+option when running tox::
 
-When debugging a test, it can be desirable to see the stdout
-and stderr generated by a test, to do this use the ``--addopts``
-function to feed arguments to pytest as such::
+  tox -e py37
 
-  ./setup.py test --addopts -s
+The output of all failing tests will always be printed in the summary, but
+if you want to observe the stdout and stderr generated by a passing test,
+you can pass the ``-s`` option to pytest as such::
+
+  tox -- -s
+
+.. tip::
+
+   The ``-s`` option is `a pytest option <https://docs.pytest.org/latest/usage.html>`_.
+
+   Any options specified before the ``--`` separator are consumed by ``tox``,
+   and any options after the ``--`` separator will be passed along to pytest.
 
 You can always abort on the first failure by running::
 
-  ./setup.py test --addopts -x
+  tox -- -x
 
 If you want to run a specific test or a group of tests, you
 can specify a prefix to match. E.g. if you want to run all of
 the frontend tests you can do::
 
-  ./setup.py test --addopts 'tests/frontend/'
+  tox -- tests/frontend/
 
 Specific tests can be chosen by using the :: delimeter after the test module.
 If you wanted to run the test_build_track test within frontend/buildtrack.py you could do::
 
-  ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track'
+  tox -- tests/frontend/buildtrack.py::test_build_track
 
 We also have a set of slow integration tests that are disabled by
 default - you will notice most of them marked with SKIP in the pytest
 output. To run them, you can use::
 
-  ./setup.py test --addopts '--integration'
+  tox -- --integration
 
 By default, buildstream also runs pylint on all files. Should you want
 to run just pylint (these checks are a lot faster), you can do so
 with::
 
-  ./setup.py test --addopts '-m pylint'
+  tox -- -m pylint
 
 Alternatively, any IDE plugin that uses pytest should automatically
 detect the ``.pylintrc`` in the project's root directory.
 
+.. note::
+
+   While using ``tox`` is practical for developers running tests in
+   more predictable execution environments, it is still possible to
+   execute the test suite against a specific installation environment
+   using pytest directly::
+
+     ./setup.py test
+
+   Specific options can be passed to ``pytest`` using the ``--addopts``
+   option::
+
+     ./setup.py test --addopts 'tests/frontend/buildtrack.py::test_build_track'
+
 
 Adding tests
 ~~~~~~~~~~~~