summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorRobert Collins <robertc@robertcollins.net>2012-12-19 15:28:22 +1300
committerRobert Collins <robertc@robertcollins.net>2012-12-19 15:28:22 +1300
commitb4fdb43317ad5611ea3816e94a5c9b1cc838f762 (patch)
tree6cbc53ed57cd93f4624a0c66f230899d29233a68 /doc
parent8b20441776e21a089d4d6dade3a8d5f08b108d39 (diff)
downloadtestrepository-b4fdb43317ad5611ea3816e94a5c9b1cc838f762.tar.gz
Document overview.
Diffstat (limited to 'doc')
-rw-r--r--doc/MANUAL.txt55
1 files changed, 45 insertions, 10 deletions
diff --git a/doc/MANUAL.txt b/doc/MANUAL.txt
index 11f396b..5cd1fed 100644
--- a/doc/MANUAL.txt
+++ b/doc/MANUAL.txt
@@ -208,11 +208,44 @@ Remote or isolated test environments
A common problem with parallel test running is test runners that use global
resources such as well known ports, well known database names or predictable
-directories on disk. Test repository has an optional interface where it can ask
-for environments to run your tests in, and then enumerate and execute the tests
-in that environment for you, allowing you to isolate your concurrent test
-processes, or to have testr coordinate tests running on remote machines / in
-virtual machines.
+directories on disk.
+
+One way to solve this is to setup isolated environments such as chroots,
+containers or even separate machines. Such environments typically require
+some coordination when being used to run tests, so testr provides an explicit
+model for working with them.
+
+The model testr has is intended to support both developers working
+incrementally on a change and CI systems running tests in a one-off setup,
+for both statically and dynamically provisioned environments.
+
+The process testr follows is:
+
+# The user should perform any one-time or once-per-session setup. For instance,
+ checking out source code, creating a template container, sourcing your cloud
+ credentials.
+
+# Execute testr run.
+
+# testr queries for concurrency.
+
+# testr will make a callout request to provision that many instances.
+ The provisioning callout needs to synchronise source code and do any other
+ per-instance setup at this stage.
+
+# testr will make callouts to execute tests, supplying files that should be
+ copied into the execution environment. Note that instances may be used for
+ more than one command execution.
+
+# testr will callout to dispose of the instances after the test run completes.
+
+Instances may be expensive to create and dispose of. testr does not perform
+any caching, but the callout pattern is intended to facilitate external
+caching - the provisioning callout can be used to pull environments out of
+a cache, and the dispose to just return it to the cache.
+
+Configuring environment support
+-------------------------------
There are three callouts that testrepository depends on - configured in
.testr.conf as usual. For instance::
@@ -228,11 +261,13 @@ These should operate as follows:
that testr should supply to the dispose and execute commands. There should
be no other output on stdout (stderr is entirely up for grabs). An exit code
of non-zero will cause testr to consider the command to have failed. A
- provisioned instance should be able to execute the list tests and execute
- tests commands that testr will run via the instance_execute callout. Its
- possible to lazy-provision things if you desire - testr doesn't care - but
- to reduce latency we suggest performing any rsync or other code
- synchronisation steps during the provision step.
+ provisioned instance should be able to execute the list tests command and
+ execute tests commands that testr will run via the instance_execute callout.
+ Its possible to lazy-provision things if you desire - testr doesn't care -
+ but to reduce latency we suggest performing any rsync or other code
+ synchronisation steps during the provision step, as testr may make multiple
+ calls to one environment, and re-doing costly operations on each command
+ execution would impair performance.
* instance_dispose should take a list of instance ids and get rid of them
this might mean putting them back in a pool of instances, or powering them