Merge artifact splitting work

Rationale
=========

This patch series implements the concept of stratum splitting.

For a long time we've had code to split a chunk into multiple artifacts,
however there's not been a way to split strata up, or systems select a
subset of the produced stratum artifacts to be included in the system.

This patch series implements the ability to split strata and have systems
include them in a way which still has the same behaviour if no rules
are specified, but with default rules that split chunk artifacts up into
various components, strata into runtime and development versions and has
systems include everything by default, but can be told to include less.

The default rules have chunk foo split up into -bins, -libs, -devel,
-doc, -locale and -misc.

These rules can be overridden in the chunk morphology by adding the new
'products' field, which lists match rules like the following:

    products:
    - artifact: libudev
      include:
      - (usr/)?lib(32|64)?/lubg?udev\..*
    - artifact: udev
      include:
      - (usr/)?s?bin/udev*
      - (usr/)?lib(32|64|exec)?/systemd/systemd-udevd

Strata are by default split into -runtime and -devel. -devel by default
contains chunks ending with -devel and -doc, -runtime contains everything
else.

Extra match rules can be added to a stratum similarly to chunks, but
instead of matching file names, they match artifact names.

    products:
    - artifact: core-python
      include:
      - "cpython-.*" # lazy shortcut to put all of cpython in this stratum
      - "python-.*" # lazy shortcut to include all python chunks in

Additionally, in chunk specs, chunk artifacts may be assigned to stratum
artifacts, this takes precedence over products match rules in the stratum
and the default match rules. Assigning the chunk to `null` will discard
the chunk.

    chunks:
    ...
    - name: systemd
      ...
      artifacts:
          libudev: foundation-runtime
          udev: foundation-runtime
          systemd-doc: null

By default a system includes every produced artifact of every stratum
listed. Instead a subset can be specified in the stratum spec as follows:

    name: tiny-system
    strata:
    - name: build-essential
      ...
      artifacts:
      - build-essential-runtime

1 files changed, 21 insertions, 11 deletions

diff --git a/morphlib/buildcommand.py b/morphlib/buildcommand.py
index 4b3b2108..6485f510 100644
--- a/morphlib/buildcommand.py
+++ b/morphlib/buildcommand.py
@@ -1,4 +1,4 @@
-# Copyright (C) 2011-2013  Codethink Limited
+# Copyright (C) 2011-2014  Codethink Limited
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -22,6 +22,14 @@ import tempfile
 import morphlib
 
 
+class MultipleRootArtifactsError(morphlib.Error):
+
+    def __init__(self, artifacts):
+        self.msg = ('System build has multiple root artifacts: %r'
+                    % [a.name for a in artifacts])
+        self.artifacts = artifacts
+
+
 class BuildCommand(object):
 
     '''High level logic for building.
@@ -142,7 +150,15 @@ class BuildCommand(object):
         artifacts = ar.resolve_artifacts(srcpool)
 
         self.app.status(msg='Computing build order', chatty=True)
-        root_artifact = self._find_root_artifact(artifacts)
+        root_artifacts = self._find_root_artifacts(artifacts)
+        if len(root_artifacts) > 1:
+            # Validate root artifacts, since validation covers errors
+            # such as trying to build a chunk or stratum directly,
+            # and this is one cause for having multiple root artifacts
+            for root_artifact in root_artifacts:
+                self._validate_root_artifact(root_artifact)
+            raise MultipleRootArtifactsError(root_artifacts)
+        root_artifact = root_artifacts[0]
 
         # Validate the root artifact here, since it's a costly function
         # to finalise it, so any pre finalisation validation is better
@@ -231,8 +247,8 @@ class BuildCommand(object):
                          other.morphology['kind'],
                          wanted))
 
-    def _find_root_artifact(self, artifacts):
-        '''Find the root artifact among a set of artifacts in a DAG.
+    def _find_root_artifacts(self, artifacts):
+        '''Find all the root artifacts among a set of artifacts in a DAG.
         
         It would be nice if the ArtifactResolver would return its results in a
         more useful order to save us from needing to do this -- the root object
@@ -240,13 +256,7 @@ class BuildCommand(object):
         
         '''
 
-        maybe = set(artifacts)
-        for a in artifacts:
-            for dep in a.dependencies:
-                if dep in maybe:
-                    maybe.remove(dep)
-        assert len(maybe) == 1
-        return maybe.pop()
+        return [a for a in artifacts if not a.dependents]
 
     def build_in_order(self, root_artifact):
         '''Build everything specified in a build order.'''