Initial import.

22 files changed, 1618 insertions, 0 deletions

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 00000000..0d20b648
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1 @@
+*.pyc
diff --git a/README b/README
new file mode 100644
index 00000000..05a49b99
--- /dev/null
+++ b/README
@@ -0,0 +1,238 @@
+README for morph
+================
+
+> **NOTA BENE:** This document is very much work-in-progress, and anything
+> and everything may and will change at little or no notice. If you see
+> problems, mail lars.wirzenius@codethink.co.uk (for now).
+
+`morph` builds binaries for [Baserock](http://www.baserock.org/),
+an embedded Linux solution. Some important points:
+
+* everything is built from **source in git**, not release tarballs
+* a binary is called a **stratum**, and is a collection of software that forms 
+  a whole, e.g., the core system for Baserock, the essential build tools 
+  for Baserock, or the GNOME platform libraries
+  - later there will be support for **erratics**, for individual software
+    packages, which will be isolated from each other
+* a stratum is atomic: it cannot be split into smaller parts
+* parts of a stratum that correspond to individual upstream projects
+  (e.g., busybox for the core stratum, or a particular library in the GNOME
+  platform) can be built separately, but the result (a **chunk**) cannot
+  be installed on its own: it needs to be combined with other chunks to
+  form a complete stratum
+
+In other words:
+
+* an individual upstream project is built into a chunk
+* chunks are combined into strata
+* strata are installed completely, or not at all
+
+The build of a chunk or a stratum is controlled by a
+**morphology**, which consists of:
+
+* a name
+* type of result: chunk or stratum
+* for a chunk, a triple of git repository, branch, and commit reference
+* for a stratum, one or more such triplets
+* the commands for configuring, building, testing, and installing the project
+
+JSON is used for the morphology syntax. For example, to build a chunk:
+
+    {
+        "name": "busybox",
+        "kind": "chunk",
+        "source": {
+            "repo": "git://git.baserock.org/busybox/",
+            "ref": "HEAD",
+        },
+        "configure-commands": [
+            "./configure --prefix=$PREFIX",
+        ],
+        "build-commands": [
+            "make",
+        ],
+        "test-commands": [
+            "make check",
+        ],
+        "install-commands": [
+            "make DESTDIR=$DESTDIR install",
+        ]
+    }
+
+(Later, there will be defaults and things to make the morph files shorter.)
+
+To build a stratum:
+
+    {
+        "name": "core",
+        "kind": "stratum",
+        "source": [
+            {
+                "repo": "git://git.baserock.org/busybox/",
+                "ref": "DEADBEEF",
+            },
+            {
+                "repo": "git://git.baserock.org/gzip/",
+                "ref": "CAFEBABE",
+            },
+        ],
+    }
+
+To use morph, create the relevant morphology files (`*.morph`),
+then run a command like the following:
+
+    morph build core.morph
+
+This will build the Baserock core stratum. It will recursively build
+any chunks that also need to be built.
+
+
+Morphology spec
+---------------
+
+A morphology is a JSON file with a single object (dict), with the
+following keys:
+
+* `name`: a string
+* `kind`: either `chunk` or `stratum`
+  - question: could this be deduced automatically?
+* `source`: either a single dict (for chunks), or a list of dicts
+   (for strata), with the following keys:
+  - `repo`: URL to git repository
+    - the URL may be relative to the value given to the
+      `--git-base-url` option
+  - `ref`: a reference to a commit (either a commit id, or `HEAD`)
+* chunks may also have the following keys:
+  - `configure-commands`: a list of strings giving shell commands
+    that should be executed to configure the software being built
+    (can also be a single string instead of a list)
+  - `build-commands`: similarly, commands for building the software
+  - `test-commands`: similarly, commands for running automatic tests
+    on the built (but uninstalled) software
+  - `install-commands` similarly, commands for installing the software
+
+Unknown keys are errors. Known keys with the wrong kind of values
+result in an error.
+
+Commands run during the building of a chunk are passed on to the shell 
+for execution, and may refer to the following extra environment variables:
+
+* `WORKAREA`: the temporary directory where all actual building occurs
+  - commands must avoid touching anything in the actual source tree,
+    and must modify files only in the temporary directory
+* `DESTDIR`: to be prefixed to install paths during install
+
+
+Build process
+-------------
+
+You give morph one or more morphologies (`*.morph`) in files, and
+it builds them in order. Built chunks are stored in a central
+cache directory (see `morph --cachedir`). Built strata are stored
+in the current working directory.
+
+During the build of a chunk, morph goes through the following steps:
+
+* clone the git repository into a fresh location
+  - note: later a way to cache the clones will be added, e.g., to use
+    any locally available git clones
+* export the files from git to a temporary location, so the build happens
+  in a clean directory
+* configure, build, and test the software
+* create a temporary directory into which the software is installed
+* install the software there
+* create a chunk file of the contents of the temporary directory
+* clean up by removing temporary stuff
+* put chunk and build log and other deliverables in their places
+
+For strata, morph does this instead:
+
+* create a temporary directory
+* unpack all the chunks into the temporary directory
+* create a stratum file from the temporary directory
+* clean up
+* put stratum file and build log and other deliverables in their places
+
+For the first minimal, "hello world" version of morph, building a
+stratum does not build any missing chunks automatically. You have
+to build them manually.
+
+
+File formats
+------------
+
+Both chunk and stratum files use the same file format, for simplicity.
+The file is a tar file, to be unpacked at the filesystem root, and
+all permission bits set exactly as they should be at the final install.
+The metadata is stored in a directory `BASEROCK` at the root of the
+directory tree, with the following files (where `foo` is the name
+of the chunk or stratum):
+
+* `foo.json`: a JSON file with the metadata of the chunk or stratum
+* in the future, there may be preinst, postinst, etc, scripts as well,
+  but as far as possible, we will try to do without
+
+No two chunks that are put into the same stratum may contain the
+same files, and no two strata installed on the same Baserock system
+can contain the same files.
+
+Note that this file format is preliminary, and may well change in
+the future. It is chosen for simplicity of implementation, not
+any other reason.
+
+Any tar variant that busybox tar and Python's tar library both can
+unpack is acceptable.
+
+
+Hacking morph
+-------------
+
+You can run `morph` from the unpacked source tree:
+
+    ./morph build foo.morph
+    
+To run unit tests:
+
+    nosetests
+
+Alternatively (and preferably), install CoverageTestRunner
+(from <http://liw.fi/coverage-test-runner/>) and run this:
+
+    python setup.py check
+
+To run black box tests, get cmdtest (from <http://liw.fi/cmdtest/>)
+and run this:
+
+    cmdtest tests -c ./morph --log cmdtest.log
+
+You should probably run the automatic tests before submitting a
+bug report. A patch that includes a unit test or black box test is
+most welcome.
+
+
+Open questions and things to consider
+-------------------------------------
+
+* When morph starts building missing chunks automatically, how will it
+  find the `*.morph` files?
+  - from the specified git?
+  - from some default git?
+  - from some other central location?
+  - from all of the above, in some order?
+  - maybe allow specifying the morph file in the "source" part of a
+    stratum's morphology?
+* Build dependencies will need to be specified in some way. They should
+  be on whole strata, not individual software.
+* There may be build dependencies between the projects that go into the
+  same chunk: app foo may build-depend on libfoobar in the same stratum.
+  We need to deal with this. Possibly put the things into `source` in
+  build order?
+* Build dependency loops should be detected, and if found, they should
+  fail the build.
+* We need ways to make use of git repositories that already local:
+  - if developer has cloned a repo, use that, instead of accessing the
+    central copy, for speed, and also so that we can build the developer's
+    changes without pushing them to a server
+  - also need a way to use mirrored repos, e.g., mirrored into
+    the Codethink office from a public server, for performance reasons
+
diff --git a/morph b/morph
new file mode 100755
index 00000000..95c6d212
--- /dev/null
+++ b/morph
@@ -0,0 +1,59 @@
+#!/usr/bin/python
+#
+# WARNING: THIS IS HIGHLY EXPERIMENTAL CODE RIGHT NOW. JUST PROOF OF CONCEPT.
+# DO NOT RUN UNTIL YOU KNOW WHAT YOU ARE DOING.
+# 
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import cliapp
+import json
+import logging
+import os
+import shutil
+import tempfile
+
+import morphlib
+
+
+class Morph(cliapp.Application):
+
+    def add_settings(self):
+        self.settings.boolean(['verbose', 'v'], 'show what is happening')
+        self.settings.string(['git-base-url'],
+                             'prepend URL to git repos that are not URLs',
+                             metavar='URL')
+
+    def cmd_build(self, morph_filenames):
+        tempdir = morphlib.tempdir.Tempdir()
+        builder = morphlib.builder.Builder(tempdir, self.msg)
+        for name in morph_filenames:
+            self.msg('Building morphology %s' % name)
+            with self.open_input(name, 'r') as f:
+                morph = morphlib.morphology.Morphology(f, 
+                                    baseurl=self.settings['git-base-url'])
+                builder.build(morph)
+        tempdir.remove()
+
+    def msg(self, msg):
+        '''Show a message to the user about what is going on.'''
+        logging.debug(msg)
+        if self.settings['verbose']:
+            self.output.write('%s\n' % msg)
+
+
+if __name__ == '__main__':
+    Morph().run()
diff --git a/morphlib/__init__.py b/morphlib/__init__.py
new file mode 100644
index 00000000..bd318978
--- /dev/null
+++ b/morphlib/__init__.py
@@ -0,0 +1,27 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+'''Baserock library.'''
+
+
+__version__ = '0.0'
+
+
+import builder
+import execute
+import morphology
+import tempdir
+import util
diff --git a/morphlib/builder.py b/morphlib/builder.py
new file mode 100644
index 00000000..befe8485
--- /dev/null
+++ b/morphlib/builder.py
@@ -0,0 +1,117 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import logging
+import os
+
+import morphlib
+
+
+class Builder(object):
+
+    '''Build binary objects for Baserock.
+    
+    The objects may be chunks or strata.'''
+    
+    def __init__(self, tempdir, msg):
+        self.tempdir = tempdir
+        self.msg = msg
+
+    def build(self, morph):
+        '''Build a binary based on a morphology.'''
+        if morph.kind == 'chunk':
+            self.build_chunk(morph)
+        elif morph.kind == 'stratum':
+            self.build_stratum(morph)
+        else:
+            raise Exception('Unknown kind of morphology: %s' % morph.kind)
+
+    def build_chunk(self, morph):
+        '''Build a chunk from a morphology.'''
+        logging.debug('Building chunk')
+        self.ex = morphlib.execute.Execute(self._build, self.msg)
+        self.ex.env['WORKAREA'] = self.tempdir.dirname
+        self.ex.env['DESTDIR'] = self._inst + '/'
+        self.create_build_tree(morph)
+        self.ex.run(morph.configure_commands)
+        self.ex.run(morph.build_commands)
+        self.ex.run(morph.test_commands)
+        self.ex.run(morph.install_commands)
+        self.create_chunk(morph)
+        self.tempdir.clear()
+        
+    def create_build_tree(self, morph):
+        '''Export sources from git into the ``self._build`` directory.'''
+
+        logging.debug('Creating build tree at %s' % self._build)
+        tarball = self.tempdir.join('sources.tar')
+        self.ex.runv(['git', 'archive',
+                      '--output', tarball,
+                      '--remote', morph.source['repo'],
+                      morph.source['ref']])
+        os.mkdir(self._build)
+        self.ex.runv(['tar', '-C', self._build, '-xf', tarball])
+        os.remove(tarball)
+
+    def create_chunk(self, morph):
+        '''Create a Baserock chunk from the ``self._inst`` directory.
+        
+        The directory must be filled in with all the relevant files already.
+        
+        '''
+
+        dirname = os.path.dirname(morph.filename)
+        filename = os.path.join(dirname, '%s.chunk' % morph.name)
+        logging.debug('Creating chunk %s at %s' % (morph.name, filename))
+        self.ex.runv(['tar', '-C', self._inst, '-czf', filename, '.'])
+
+    def build_stratum(self, morph):
+        '''Build a stratum from a morphology.'''
+        os.mkdir(self._inst)
+        self.ex = morphlib.execute.Execute(self.tempdir.dirname, self.msg)
+        for chunk_name in morph.sources:
+            filename = self._chunk_filename(morph, chunk_name)
+            self.unpack_chunk(filename)
+        self.create_stratum(morph)
+        self.tempdir.clear()
+
+    def unpack_chunk(self, filename):
+        self.ex.runv(['tar', '-C', self._inst, '-xf', filename])
+
+    def create_stratum(self, morph):
+        '''Create a Baserock stratum from the ``self._inst`` directory.
+        
+        The directory must be filled in with all the relevant files already.
+        
+        '''
+
+        dirname = os.path.dirname(morph.filename)
+        filename = os.path.join(dirname, '%s.stratum' % morph.name)
+        logging.debug('Creating stratum %s at %s' % (morph.name, filename))
+        self.ex.runv(['tar', '-C', self._inst, '-czf', filename, '.'])
+
+    @property
+    def _build(self):
+        return self.tempdir.join('build')
+
+    @property
+    def _inst(self):
+        return self.tempdir.join('inst')
+
+    def _chunk_filename(self, morph, chunk_name):
+        dirname = os.path.dirname(morph.filename)
+        return os.path.join(dirname, '%s.chunk' % chunk_name)
+
diff --git a/morphlib/execute.py b/morphlib/execute.py
new file mode 100644
index 00000000..9a279591
--- /dev/null
+++ b/morphlib/execute.py
@@ -0,0 +1,87 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import logging
+import os
+import subprocess
+
+import morphlib
+
+
+class CommandFailure(Exception):
+
+    pass
+
+
+class Execute(object):
+
+    '''Execute commands for morph.'''
+    
+    def __init__(self, dirname, msg):
+        self._setup_env()
+        self.dirname = dirname
+        self.msg = msg
+
+    def _setup_env(self):
+        self.env = dict(os.environ)
+
+    def run(self, commands):
+        '''Execute a list of commands.
+        
+        If a command fails (returns non-zero exit code), the rest are
+        not run, and CommandFailure is returned.
+        
+        '''
+
+        stdouts = []
+        for command in commands:
+            self.msg('# %s' % command)
+            p = subprocess.Popen([command], shell=True,
+                                 stdout=subprocess.PIPE,
+                                 stderr=subprocess.PIPE,
+                                 env=self.env,
+                                 cwd=self.dirname)
+            out, err = p.communicate()
+            logging.debug('Exit code: %d' % p.returncode)
+            logging.debug('Standard output:\n%s' % morphlib.util.indent(out))
+            logging.debug('Standard error:\n%s' % morphlib.util.indent(err))
+            if p.returncode != 0:
+                raise CommandFailure('Command failed: %s\n%s' % 
+                                      (command, morphlib.util.indent(err)))
+            stdouts.append(out)
+        return stdouts
+
+    def runv(self, argv):
+        '''Run a command given as a list of argv elements.
+        
+        Return standard output. Raise ``CommandFailure`` if the command
+        fails. Log standard output and error in any case.
+        
+        '''
+
+        self.msg('# %s' % ' '.join(argv))
+        p = subprocess.Popen(argv, stdout=subprocess.PIPE, 
+                             stderr=subprocess.PIPE)
+        out, err = p.communicate()
+        
+        logging.debug('Exit code: %d' % p.returncode)
+        logging.debug('Standard output:\n%s' % morphlib.util.indent(out))
+        logging.debug('Standard error:\n%s' % morphlib.util.indent(err))
+        if p.returncode != 0:
+            raise CommandFailure('Command failed: %s\n%s' % 
+                                  (argv, morphlib.util.indent(err)))
+        return out
+
diff --git a/morphlib/execute_tests.py b/morphlib/execute_tests.py
new file mode 100644
index 00000000..721d0807
--- /dev/null
+++ b/morphlib/execute_tests.py
@@ -0,0 +1,51 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import os
+import unittest
+
+import morphlib
+
+
+class ExecuteTests(unittest.TestCase):
+
+    def setUp(self):
+        self.e = morphlib.execute.Execute('/')
+
+    def test_has_same_path_as_environment(self):
+        self.assertEqual(self.e.env['PATH'], os.environ['PATH'])
+
+    def test_executes_true_ok(self):
+        self.assertEqual(self.e.run(['true']), [''])
+
+    def test_raises_commandfailure_for_false(self):
+        self.assertRaises(morphlib.execute.CommandFailure,
+                          self.e.run, ['false'])
+
+    def test_returns_stdout_from_all_commands(self):
+        self.assertEqual(self.e.run(['echo foo', 'echo bar']),
+                         ['foo\n', 'bar\n'])
+
+    def test_sets_working_directory(self):
+        self.assertEqual(self.e.run(['pwd']), ['/\n'])
+
+    def test_executes_argv(self):
+        self.assertEqual(self.e.runv(['echo', 'foo']), 'foo\n')
+
+    def test_raises_error_when_argv_fails(self):
+        self.assertRaises(morphlib.execute.CommandFailure,
+                          self.e.runv, ['false'])
+
diff --git a/morphlib/morphology.py b/morphlib/morphology.py
new file mode 100644
index 00000000..d31a000c
--- /dev/null
+++ b/morphlib/morphology.py
@@ -0,0 +1,196 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import json
+import logging
+
+
+class SchemaError(Exception):
+
+    pass
+
+
+class Morphology(object):
+
+    '''Represent a morphology: description of how to build binaries.'''
+    
+    def __init__(self, fp, baseurl=None):
+        self._fp = fp
+        self._baseurl = baseurl or ''
+        self._load()
+
+    def _load(self):
+        logging.debug('Loading morphology %s' % self._fp.name)
+        self._dict = json.load(self._fp)
+
+        if 'name' not in self._dict:
+            raise self._error('must contain "name"')
+            
+        if not self.name:
+            raise self._error('"name" must not be empty')
+
+        if 'kind' not in self._dict:
+            raise self._error('must contain "kind"')
+
+        if self.kind == 'chunk':
+            self._validate_chunk()
+            self.source['repo'] = self._join_with_baseurl(self.source['repo'])
+        elif self.kind == 'stratum':
+            self._validate_stratum()
+            for source in self.sources.itervalues():
+                source['repo'] = self._join_with_baseurl(source['repo'])
+        else:
+            raise self._error('kind must be chunk or stratum, not %s' %
+                                self.kind)
+
+        self.filename = self._fp.name
+
+    def _validate_chunk(self):
+        valid_toplevel_keys = ['name', 'kind', 'source', 'configure-commands',
+                               'build-commands', 'test-commands',
+                               'install-commands']
+
+        if 'source' not in self._dict:
+            raise self._error('chunks must have "source" field')
+
+        if type(self.source) != dict:
+            raise self._error('"source" must be a dictionary')
+
+        if len(self.source) == 0:
+            raise self._error('"source" must not be empty')
+            
+        if 'repo' not in self.source:
+            raise self._error('"source" must contain "repo"')
+            
+        if not self.source['repo']:
+            raise self._error('"source" must contain non-empty "repo"')
+            
+        if 'ref' not in self.source:
+            raise self._error('"source" must contain "ref"')
+            
+        if not self.source['ref']:
+            raise self._error('"source" must contain non-empty "ref"')
+
+        for key in self.source.keys():
+            if key not in ('repo', 'ref'):
+                raise self._error('unknown key "%s" in "source"' % key)
+
+        cmdlists = [
+            (self.configure_commands, 'configure-commands'),
+            (self.build_commands, 'build-commands'),
+            (self.test_commands, 'test-commands'),
+            (self.install_commands, 'install-commands'),
+        ]
+        for value, name in cmdlists:
+            if type(value) != list:
+                raise self._error('"%s" must be a list' % name)
+            for x in value:
+                if type(x) != unicode:
+                    raise self._error('"%s" must contain strings' % name)
+
+        for key in self._dict.keys():
+            if key not in valid_toplevel_keys:
+                raise self._error('unknown key "%s"' % key)
+
+    def _validate_stratum(self):
+        valid_toplevel_keys = ['name', 'kind', 'sources']
+
+        if 'sources' not in self._dict:
+            raise self._error('stratum must contain "sources"')
+
+        if type(self.sources) != dict:
+            raise self._error('"sources" must be a dict')
+
+        if len(self.sources) == 0:
+            raise self._error('"sources" must not be empty')
+
+        for name, source in self.sources.iteritems():
+            if type(source) != dict:
+                raise self._error('"sources" must contain dicts')
+            if 'repo' not in source:
+                raise self._error('sources must have "repo"')
+            if type(source['repo']) != unicode:
+                raise self._error('"repo" must be a string')
+            if not source['repo']:
+                raise self._error('"repo" must be a non-empty string')
+            if 'ref' not in source:
+                raise self._error('sources must have "ref"')
+            if type(source['ref']) != unicode:
+                raise self._error('"ref" must be a string')
+            if not source['ref']:
+                raise self._error('"ref" must be a non-empty string')
+            for key in source:
+                if key not in ('repo', 'ref'):
+                    raise self._error('unknown key "%s" in sources' % key)
+
+        for key in self._dict.keys():
+            if key not in valid_toplevel_keys:
+                raise self._error('unknown key "%s"' % key)
+
+    @property
+    def name(self):
+        return self._dict['name']
+
+    @property
+    def kind(self):
+        return self._dict['kind']
+
+    @property
+    def source(self):
+        return self._dict['source']
+
+    @property
+    def sources(self):
+        return self._dict['sources']
+
+    @property
+    def configure_commands(self):
+        return self._dict.get('configure-commands', [])
+
+    @property
+    def build_commands(self):
+        return self._dict.get('build-commands', [])
+
+    @property
+    def test_commands(self):
+        return self._dict.get('test-commands', [])
+
+    @property
+    def install_commands(self):
+        return self._dict.get('install-commands', [])
+
+    @property
+    def manifest(self):
+        if self.kind == 'chunk':
+            return [(self.source['repo'], self.source['ref'])]
+        else:
+            return [(source['repo'], source['ref'])
+                     for source in self.sources.itervalues()]
+
+    def _join_with_baseurl(self, url):
+        is_relative = (':' not in url or
+                       '/' not in url or
+                       url.find('/') < url.find(':'))
+        if is_relative:
+            if not url.endswith('/'):
+                url += '/'
+            return self._baseurl + url
+        else:
+            return url
+
+    def _error(self, msg):
+        return SchemaError('Morphology %s: %s' % (self._fp.name, msg))
+
diff --git a/morphlib/morphology_tests.py b/morphlib/morphology_tests.py
new file mode 100644
index 00000000..7a38bff4
--- /dev/null
+++ b/morphlib/morphology_tests.py
@@ -0,0 +1,458 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import json
+import StringIO
+import unittest
+
+import morphlib
+
+
+class MockFile(StringIO.StringIO):
+
+    def __init__(self, *args, **kwargs):
+        StringIO.StringIO.__init__(self, *args, **kwargs)
+        self.name = 'mockfile'
+
+
+class MorphologyTests(unittest.TestCase):
+
+    def assertRaisesSchemaError(self, morph_dict):
+        f = MockFile(json.dumps(morph_dict))
+        self.assertRaises(morphlib.morphology.SchemaError,
+                          morphlib.morphology.Morphology, f)
+
+    def test_raises_exception_for_empty_file(self):
+        self.assertRaises(ValueError, 
+                          morphlib.morphology.Morphology,
+                          MockFile())
+
+    def test_raises_exception_for_file_without_kind_field(self):
+        self.assertRaisesSchemaError({})
+
+    def test_raises_exception_for_chunk_with_unknown_keys_only(self):
+        self.assertRaisesSchemaError({ 'x': 'y' })
+
+    def test_raises_exception_if_name_only(self):
+        self.assertRaisesSchemaError({ 'name': 'hello' })
+
+    def test_raises_exception_if_name_is_empty(self):
+        self.assertRaisesSchemaError({ 'name': '', 'kind': 'chunk',
+                                       'sources': { 'repo': 'x', 'ref': 'y' }})
+
+    def test_raises_exception_if_kind_only(self):
+        self.assertRaisesSchemaError({ 'kind': 'chunk' })
+
+    def test_raises_exception_for_kind_that_has_unknown_kind(self):
+        self.assertRaisesSchemaError({ 'name': 'hello', 'kind': 'x' })
+
+    def test_raises_exception_for_chunk_without_source(self):
+        self.assertRaisesSchemaError({ 'name': 'hello', 'kind': 'chunk' })
+
+    def test_raises_exception_for_chunk_with_nondict_source(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': [],
+        })
+
+    def test_raises_exception_for_chunk_with_empty_source(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {},
+        })
+
+    def test_raises_exception_for_chunk_without_repo_in_source(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'x': 'y'
+            },
+        })
+
+    def test_raises_exception_for_chunk_with_empty_repo_in_source(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': '',
+                'ref': 'master'
+            },
+        })
+
+    def test_raises_exception_for_chunk_without_ref_in_source(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+            },
+        })
+
+    def test_raises_exception_for_chunk_with_empty_ref_in_source(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': ''
+            },
+        })
+
+    def test_raises_exception_for_chunk_with_unknown_keys_in_source(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master',
+                'x': 'y'
+            },
+        })
+
+    def test_raises_exception_for_chunk_with_unknown_keys(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'x': 'y'
+        })
+
+    def test_raises_exception_for_nonlist_configure_commands(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'configure-commands': 0,
+        })
+
+    def test_raises_exception_for_list_of_nonstring_configure_commands(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'configure-commands': [0],
+        })
+
+    def test_raises_exception_for_nonlist_build_commands(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'build-commands': 0,
+        })
+
+    def test_raises_exception_for_list_of_nonstring_build_commands(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'build-commands': [0],
+        })
+
+    def test_raises_exception_for_nonlist_test_commands(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'test-commands': 0,
+        })
+
+    def test_raises_exception_for_list_of_nonstring_test_commands(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'build-commands': [0],
+        })
+
+    def test_raises_exception_for_nonlist_install_commands(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'install-commands': 0,
+        })
+
+    def test_raises_exception_for_list_of_nonstring_install_commands(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'chunk',
+            'source': {
+                'repo': 'foo',
+                'ref': 'master'
+            },
+            'install-commands': [0],
+        })
+
+    def test_accepts_valid_chunk_morphology(self):
+        chunk = morphlib.morphology.Morphology(
+                          MockFile('''
+                            {
+                                "name": "hello",
+                                "kind": "chunk", 
+                                "source": 
+                                    {
+                                        "repo": "foo",
+                                        "ref": "ref"
+                                    },
+                                "configure-commands": ["./configure"],
+                                "build-commands": ["make"],
+                                "test-commands": ["make check"],
+                                "install-commands": ["make install"]
+                            }'''))
+        self.assertEqual(chunk.kind, 'chunk')
+        self.assertEqual(chunk.filename, 'mockfile')
+
+    def test_raises_exception_for_stratum_without_sources(self):
+        self.assertRaisesSchemaError({ 'name': 'hello', 'kind': 'stratum' })
+
+    def test_raises_exception_for_stratum_with_nondict_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': [],
+        })
+
+    def test_raises_exception_for_stratum_with_empty_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {},
+        })
+        
+    def test_raises_exception_for_stratum_with_bad_children_in_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': 0,
+            },
+        })
+
+    def test_raises_exception_for_stratum_without_repo_in_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': {
+                    'ref': 'master'
+                }
+            },
+        })
+
+    def test_raises_exception_for_stratum_with_empty_repo_in_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': {
+                    'repo': '',
+                    'ref': 'master'
+                }
+            },
+        })
+
+    def test_raises_exception_for_stratum_with_nonstring_repo_in_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': {
+                    'repo': 0,
+                    'ref': 'master'
+                }
+            },
+        })
+
+    def test_raises_exception_for_stratum_without_ref_in_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': {
+                    'repo': 'foo',
+                }
+            },
+        })
+
+    def test_raises_exception_for_stratum_with_empty_ref_in_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': {
+                    'repo': 'foo',
+                    'ref': ''
+                }
+            },
+        })
+
+    def test_raises_exception_for_stratum_with_nonstring_ref_in_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': {
+                    'repo': 'foo',
+                    'ref': 0
+                }
+            },
+        })
+
+    def test_raises_exception_for_stratum_with_unknown_keys_in_sources(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': {
+                    'repo': 'foo',
+                    'ref': 'master',
+                    'x': 'y'
+                }
+            },
+        })
+
+    def test_raises_exception_for_stratum_with_unknown_keys(self):
+        self.assertRaisesSchemaError({
+            'name': 'hello', 
+            'kind': 'stratum',
+            'sources': {
+                'foo': {
+                    'repo': 'foo',
+                    'ref': 'master'
+                }
+            },
+            'x': 'y'
+        })
+
+    def test_accepts_valid_stratum_morphology(self):
+        morph = morphlib.morphology.Morphology(
+                          MockFile('''
+                            {
+                                "name": "hello",
+                                "kind": "stratum", 
+                                "sources": 
+                                    {
+                                        "foo": {
+                                            "repo": "foo",
+                                            "ref": "ref"
+                                        }
+                                    }
+                            }'''))
+        self.assertEqual(morph.kind, 'stratum')
+        self.assertEqual(morph.filename, 'mockfile')
+
+
+class ChunkRepoTests(unittest.TestCase):
+
+    def chunk(self, repo):
+        return morphlib.morphology.Morphology(
+                          MockFile('''
+                            {
+                                "name": "hello",
+                                "kind": "chunk", 
+                                "source": 
+                                    {
+                                        "repo": "%s",
+                                        "ref": "HEAD"
+                                    },
+                                "configure-commands": ["./configure"],
+                                "build-commands": ["make"],
+                                "test-commands": ["make check"],
+                                "install-commands": ["make install"]
+                            }''' % repo),
+                            baseurl='git://git.baserock.org/')
+
+    def test_returns_repo_with_schema_as_is(self):
+        self.assertEqual(self.chunk('git://git.baserock.org/foo/').manifest,
+                         [('git://git.baserock.org/foo/', 'HEAD')])
+
+    def test_prepends_baseurl_to_repo_without_schema(self):
+        self.assertEqual(self.chunk('foo').manifest,
+                         [('git://git.baserock.org/foo/', 'HEAD')])
+
+    def test_leaves_absolute_repo_in_source_dict_as_is(self):
+        chunk = self.chunk('git://git.baserock.org/foo/')
+        self.assertEqual(chunk.source['repo'], 'git://git.baserock.org/foo/')
+
+    def test_makes_relative_repo_url_absolute_in_source_dict(self):
+        chunk = self.chunk('foo')
+        self.assertEqual(chunk.source['repo'], 'git://git.baserock.org/foo/')
+
+
+class StratumRepoTests(unittest.TestCase):
+
+    def stratum(self, repo):
+        return morphlib.morphology.Morphology(
+                          MockFile('''
+                            {
+                                "name": "hello",
+                                "kind": "stratum", 
+                                "sources": 
+                                    {
+                                        "foo": {
+                                            "repo": "%s",
+                                            "ref": "HEAD"
+                                        }
+                                    }
+                            }''' % repo),
+                            baseurl='git://git.baserock.org/')
+
+    def test_returns_repo_with_schema_as_is(self):
+        self.assertEqual(self.stratum('git://git.baserock.org/foo/').manifest,
+                         [('git://git.baserock.org/foo/', 'HEAD')])
+
+    def test_prepends_baseurl_to_repo_without_schema(self):
+        self.assertEqual(self.stratum('foo').manifest,
+                         [('git://git.baserock.org/foo/', 'HEAD')])
+
+    def test_leaves_absolute_repo_in_source_dict_as_is(self):
+        stratum = self.stratum('git://git.baserock.org/foo/')
+        self.assertEqual(stratum.sources['foo']['repo'], 
+                         'git://git.baserock.org/foo/')
+
+    def test_makes_relative_repo_url_absolute_in_source_dict(self):
+        stratum = self.stratum('foo')
+        self.assertEqual(stratum.sources['foo']['repo'], 
+                         'git://git.baserock.org/foo/')
+
diff --git a/morphlib/tempdir.py b/morphlib/tempdir.py
new file mode 100644
index 00000000..7f58cdc3
--- /dev/null
+++ b/morphlib/tempdir.py
@@ -0,0 +1,58 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import logging
+import os
+import shutil
+import tempfile
+
+
+class Tempdir(object):
+
+    '''Temporary file handling for morph.'''
+
+    def __init__(self, parent=None):
+        self.dirname = tempfile.mkdtemp(dir=parent)
+        logging.debug('Created temporary directory %s' % self.dirname)
+
+    def remove(self):
+        '''Remove the temporary directory.'''
+        logging.debug('Removing temporary directory %s' % self.dirname)
+        shutil.rmtree(self.dirname)
+        self.dirname = None
+
+    def clear(self):
+        '''Clear temporary directory of everything.'''
+        for x in os.listdir(self.dirname):
+            filename = self.join(x)
+            if os.path.isdir(filename):
+                shutil.rmtree(filename)
+            else:
+                os.remove(filename)
+
+    def join(self, relative):
+        '''Return full path to file in temporary directory.
+        
+        The relative path is given appended to the name of the
+        temporary directory. If the relative path is actually absolute,
+        it is forced to become relative.
+        
+        The returned path is normalized.
+        
+        '''
+        
+        return os.path.normpath(os.path.join(self.dirname, './' + relative))
+
diff --git a/morphlib/tempdir_tests.py b/morphlib/tempdir_tests.py
new file mode 100644
index 00000000..6572b70a
--- /dev/null
+++ b/morphlib/tempdir_tests.py
@@ -0,0 +1,69 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import os
+import shutil
+import unittest
+
+import morphlib
+
+
+class TempdirTests(unittest.TestCase):
+
+    def setUp(self):
+        self.parent = os.path.abspath('unittest-tempdir')
+        os.mkdir(self.parent)
+        self.tempdir = morphlib.tempdir.Tempdir(parent=self.parent)
+
+    def tearDown(self):
+        shutil.rmtree(self.parent)
+
+    def test_creates_the_directory(self):
+        self.assert_(os.path.isdir(self.tempdir.dirname))
+
+    def test_creates_subdirectory_of_parent(self):
+        self.assert_(self.tempdir.dirname.startswith(self.parent + '/'))
+
+    def test_uses_default_if_parent_not_specified(self):
+        t = morphlib.tempdir.Tempdir()
+        shutil.rmtree(t.dirname)
+        self.assertNotEqual(t.dirname, None)
+
+    def test_removes_itself(self):
+        dirname = self.tempdir.dirname
+        self.tempdir.remove()
+        self.assertEqual(self.tempdir.dirname, None)
+        self.assertFalse(os.path.exists(dirname))
+
+    def test_joins_filename(self):
+        self.assertEqual(self.tempdir.join('foo'), 
+                         os.path.join(self.tempdir.dirname, 'foo'))
+
+    def test_joins_absolute_filename(self):
+        self.assertEqual(self.tempdir.join('/foo'), 
+                         os.path.join(self.tempdir.dirname, 'foo'))
+
+    def test_clears_when_empty(self):
+        self.tempdir.clear()
+        self.assertEqual(os.listdir(self.tempdir.dirname), [])
+
+    def test_clears_when_not_empty(self):
+        os.mkdir(self.tempdir.join('foo'))
+        with open(self.tempdir.join('bar'), 'w') as f:
+            f.write('bar')
+        self.tempdir.clear()
+        self.assertEqual(os.listdir(self.tempdir.dirname), [])
+
diff --git a/morphlib/util.py b/morphlib/util.py
new file mode 100644
index 00000000..a181ff29
--- /dev/null
+++ b/morphlib/util.py
@@ -0,0 +1,32 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+'''Utility functions for morph.'''
+
+
+def indent(string, spaces=4):
+    '''Return ``string`` indented by ``spaces`` spaces.
+    
+    The final line is not terminated by a newline. This makes it easy
+    to use this function for indenting long text for logging: the
+    logging library adds a newline, so not including it in the indented
+    text avoids a spurious empty line in the log file.
+    
+    '''
+
+    return '\n'.join('%*s%s' % (spaces, '', line) 
+                      for line in string.splitlines())
+
diff --git a/morphlib/util_tests.py b/morphlib/util_tests.py
new file mode 100644
index 00000000..d27de771
--- /dev/null
+++ b/morphlib/util_tests.py
@@ -0,0 +1,36 @@
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+import unittest
+
+import morphlib
+
+
+class IndentTests(unittest.TestCase):
+
+    def test_returns_empty_string_for_empty_string(self):
+        self.assertEqual(morphlib.util.indent(''), '')
+        
+    def test_indents_single_line(self):
+        self.assertEqual(morphlib.util.indent('foo'), '    foo')
+
+    def test_obeys_spaces_setting(self):
+        self.assertEqual(morphlib.util.indent('foo', spaces=2), '  foo')
+
+    def test_indents_multiple_lines(self):
+        self.assertEqual(morphlib.util.indent('foo\nbar\n'), 
+                         '    foo\n    bar')
+
diff --git a/setup.py b/setup.py
new file mode 100644
index 00000000..8b80b7bb
--- /dev/null
+++ b/setup.py
@@ -0,0 +1,103 @@
+#!/usr/bin/python
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+
+'''Setup.py for morph.'''
+
+
+from distutils.core import setup
+from distutils.cmd import Command
+from distutils.command.clean import clean
+import glob
+import os
+import shutil
+import subprocess
+
+import morphlib
+
+
+class Clean(clean):
+
+    clean_files = [
+        '.coverage',
+        'build',
+        'unittest-tempdir',
+    ]
+    clean_globs = [
+        '*/*.py[co]',
+    ]
+
+    def run(self):
+        clean.run(self)
+        itemses = ([self.clean_files] + 
+                   [glob.glob(x) for x in self.clean_globs])
+        for items in itemses:
+            for filename in items:
+                if os.path.isdir(filename):
+                    shutil.rmtree(filename)
+                elif os.path.exists(filename):
+                    os.remove(filename)
+
+
+class Check(Command):
+
+    user_options = [
+    ]
+
+    def initialize_options(self):
+        pass
+
+    def finalize_options(self):
+        pass
+
+    def run(self):
+        subprocess.check_call(['python', '-m', 'CoverageTestRunner',
+                               '--ignore-missing-from=without-test-modules',
+                               'morphlib'])
+        os.remove('.coverage')
+
+
+
+setup(name='morph',
+      version=morphlib.__version__,
+      description='FIXME',
+      long_description='''\
+FIXME
+''',
+      classifiers=[
+        'Development Status :: 2 - Pre-Alpha',
+        'Environment :: Console',
+        'Intended Audience :: Developers',
+        'License :: OSI Approved :: GNU General Public License (GPL)',
+        'Operating System :: POSIX :: Linux',
+        'Programming Language :: Python',
+        'Topic :: Software Development :: Build Tools',
+        'Topic :: Software Development :: Embedded Systems',
+        'Topic :: System :: Archiving :: Packaging',
+        'Topic :: System :: Software Distribution',
+      ],
+      author='Lars Wirzenius',
+      author_email='lars.wirzenius@codethink.co.uk',
+      url='http://www.baserock.org/',
+      scripts=['morph'],
+      packages=['morphlib'],
+      data_files=[('share/man/man1', glob.glob('*.[1-8]'))],
+      cmdclass={
+        'check': Check,
+        'clean': Clean,
+      },
+     )
+
diff --git a/tests/hello-chunk.morph b/tests/hello-chunk.morph
new file mode 100644
index 00000000..739239ff
--- /dev/null
+++ b/tests/hello-chunk.morph
@@ -0,0 +1,17 @@
+{
+    "name": "hello",
+    "kind": "chunk",
+    "source": {
+        "repo": "hello",
+        "ref": "master"
+    },
+    "configure-commands": [
+        "./configure --prefix=/usr"
+    ],
+    "build-commands": [
+        "make"
+    ],
+    "install-commands": [
+        "make DESTDIR=\"$DESTDIR\" install"
+    ]
+}
diff --git a/tests/hello-chunk.script b/tests/hello-chunk.script
new file mode 100755
index 00000000..52e7bc68
--- /dev/null
+++ b/tests/hello-chunk.script
@@ -0,0 +1,23 @@
+#!/bin/sh
+#
+# Test building a chunk for GNU Hello.
+#
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+tar -C "$DATADIR" -xf tests/hello-chunk.tar.gz
+cp tests/hello-chunk.morph "$DATADIR/hello-chunk.morph"
+./morph build "$DATADIR/hello-chunk.morph" --git-base-url "file://$DATADIR/"
+tar -tf "$DATADIR/hello.chunk" | LC_ALL=C sort
diff --git a/tests/hello-chunk.stdout b/tests/hello-chunk.stdout
new file mode 100644
index 00000000..3049f010
--- /dev/null
+++ b/tests/hello-chunk.stdout
@@ -0,0 +1,5 @@
+./
+./usr/
+./usr/local/
+./usr/local/bin/
+./usr/local/bin/greeter
diff --git a/tests/hello-chunk.tar.gz b/tests/hello-chunk.tar.gz
new file mode 100644
index 00000000..7ecb02ec
--- /dev/null
+++ b/tests/hello-chunk.tar.gz
diff --git a/tests/hello-stratum.morph b/tests/hello-stratum.morph
new file mode 100644
index 00000000..11ac4e1c
--- /dev/null
+++ b/tests/hello-stratum.morph
@@ -0,0 +1,10 @@
+{
+    "name": "hello",
+    "kind": "stratum",
+    "sources": {
+        "hello": {
+            "repo": "hello",
+            "ref": "master"
+        }
+    }
+}
diff --git a/tests/hello-stratum.script b/tests/hello-stratum.script
new file mode 100755
index 00000000..ec9d6929
--- /dev/null
+++ b/tests/hello-stratum.script
@@ -0,0 +1,24 @@
+#!/bin/sh
+#
+# Test building a stratum for GNU Hello.
+#
+# Copyright (C) 2011  Codethink Limited
+# 
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License.
+# 
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+tar -C "$DATADIR" -xf tests/hello-chunk.tar.gz
+cp tests/hello-chunk.morph tests/hello-stratum.morph "$DATADIR"
+./morph build "$DATADIR/hello-chunk.morph" --git-base-url "file://$DATADIR/"
+./morph build "$DATADIR/hello-stratum.morph" --git-base-url "file://$DATADIR/"
+tar -tf "$DATADIR/hello.stratum" | LC_ALL=C sort
diff --git a/tests/hello-stratum.stdout b/tests/hello-stratum.stdout
new file mode 100644
index 00000000..3049f010
--- /dev/null
+++ b/tests/hello-stratum.stdout
@@ -0,0 +1,5 @@
+./
+./usr/
+./usr/local/
+./usr/local/bin/
+./usr/local/bin/greeter
diff --git a/without-test-modules b/without-test-modules
new file mode 100644
index 00000000..227bb836
--- /dev/null
+++ b/without-test-modules
@@ -0,0 +1,2 @@
+morphlib/__init__.py
+morphlib/builder.py