summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Add validation-specific assert() helper functions to morphloader_testsbaserock/samthursfield/morphloader-test-cleanupSam Thursfield2014-01-141-40/+44
|
* Move morphloader errors into separate module for easy accessSam Thursfield2014-01-144-183/+200
| | | | | | We can now import the errors into the local namespace of any module we want (such as the unit tests) which saves having to use 40-character identifiers to refer to the exceptions.
* Remove duplication in morphloader_tests.py, part 1Sam Thursfield2014-01-141-234/+129
|
* yarns: Add tests for building systems with splittingRichard Maw2014-01-132-0/+138
| | | | | | | This includes tests for systems with the default splits and a system that selects only one of the produced stratum artifacts to go into the system artifact, since this is roughly the expected use-case for the tiny system morphologies.
* cmdtests: Take into account new constructed artifactsRichard Maw2014-01-1329-215/+1809
|
* Split Stratum artifacts according to new rulesRichard Maw2014-01-131-35/+43
| | | | | | | | | | The logic is roughly the same as before, it just does it for every artifact that a Stratum produces now. NOTE: cmdtest tests do not pass, since they list files and artifacts produced, which has changed since the new default splitting rules were added. The next patch fixes this, but was kept as a separate commit for readability.
* Unit tests: Fix up for changes to chunk construct apiRichard Maw2014-01-131-24/+15
|
* Split chunk morphologies according to new rulesRichard Maw2014-01-132-85/+49
| | | | | | | | | | | | | | | | | | Filenames are now matched before chunks are constructed, so bins.create_chunk now takes a list of relative file names. bins.chunk_contents is gone, since this is now handled by passing source.split_rules.partition the file names. We now don't consider it to be a problem for directories to remain in the DESTDIR after artifacts have been removed, since we need to handle file matches implying their parent directories, and explicit matches against directories. NOTE: The bins_tests were broken in this patch, and are fixed in the next. This was done to try and aid readability of the patch series. Full functionality is still broken until stratum splitting is fixed.
* ArtifactResolver: Generate dependencies from split rulesRichard Maw2014-01-132-527/+256
| | | | | | | | | | | | | | | | | | One important change is that the builds_artifacts field of Morphologies is not used any more, since the split rules provide this information. Another important change is that the ArtifactResolver now only returns aritfacts that are required to build the root artifact, rather than every artifact in the build. Previously there was no distinction. This is required because when artifact splitting is in effect, some artifacts may be produced, but not depended on by anything. This confuses the BuildCommand, which expects to be able to find a single root artifact. NOTE: This change breaks artifact construction until "Split chunk morphologies according to new rules" and "Split Stratum artifacts according to new rules", since systems and strata depend on artifacts that weren't created.
* CacheKeyComputer: Include split rules in computationRichard Maw2014-01-132-2/+4
| | | | | | | | | | | | Chunk artifacts need the [(artifact_name, [regular_expression])] so that if the default split rules change, or the blending rule changes, then an extra version field doesn't need to be added to the cache key computer. This is for future plans to allow the split rules to be configurable and allow us to more easily change them. System and Stratum artifact computations don't need this, since those splitting rules are already expressed in the dependencies information.
* Add split rules to sourcesRichard Maw2014-01-134-1/+298
| | | | | | | | | This introduces a new artifactsplitrule module, which tries to provide a nice abstraction over matching a sequence of things to a bunch of outputs, to be used by both chunks splitting, for separating files out into chunk artifacts, the stratum splitting, where chunks are aggregated into stratum artifacts, and systems selecting the right strata to go into the artifact.
* MorphologyFactory: validate new fields using loader codeRichard Maw2014-01-101-0/+2
|
* MorphologyLoader: Validate new fieldsRichard Maw2014-01-102-6/+183
|
* Rename chunk 'chunks' field to 'products'Richard Maw2014-01-107-19/+25
| | | | | It's confusing for a chunk to specify chunks. Splits and artifacts weren't satisfactory names either, so they're now called 'products'.
* ArtifactResolver: Use Source.get_artifactRichard Maw2014-01-101-20/+9
|
* source: Memoize artifact creation in Source objectRichard Maw2014-01-101-1/+28
| | | | | | This can simplify logic in the ArtifactResolver, since it no longer needs its own cache, and it allows you to go from a Source to its Artifacts later.
* morph2.Morphology: add trivial .get methodRichard Maw2014-01-101-1/+9
| | | | | This it convenient, as it allows the new validation code to validate the old morphology class during the transition period.
* BuildCommand.find_root_artifact: Simplify root artifact location algorithmRichard Maw2014-01-101-4/+3
| | | | | It's way simpler to pick things that have no dependents than go through all the artifacts and remove any that are depended on.
* BuildCommand: Validate multiple root morphologiesRichard Maw2014-01-101-3/+18
| | | | | | | | | | | | | | When you attempt to build a stratum or chunk, the ArtifactResolver can return multiple root artifacts, since the root source produces multiple artifacts. Rather than having the BuildCommand complain that there's mutiple root artifacts, it now validates all the produced artifacts too, since that will validate the kinds of artifacts produced, and give a more useful error message, that you're trying to build a Stratum or Chunk directly. If all the produced artifacts validate, then an Exception is raised to signal that it got multiple artifacts, when it only expected one.
* unit tests: Fix invalid morphologiesRichard Maw2014-01-104-13/+15
|
* yarns: Add a deploy testRichard Maw2014-01-102-3/+49
|
* yarns: Make test chunk create a bunch of filesRichard Maw2014-01-091-2/+81
| | | | This is useful for splitting tests
* yarns: Remove redundant repo in IMPLEMENTS THENRichard Maw2014-01-093-4/+4
| | | | | morph build the system... doesn't need the repository to be specified, just the branch; morph is able to work it out for itself.
* Merge branch 'liw/update-NEWS-for-BR13'baserock-13Lars Wirzenius2014-01-061-1/+43
|\ | | | | | | | | Reviewed-by: Dan Firth Reviewed-by: Richard Maw
| * Update NEWS based on feedbackLars Wirzenius2014-01-061-4/+5
| | | | | | | | Feedback by Richard Maw.
| * Update NEWS for Baserock 13 releaseLars Wirzenius2014-01-061-1/+42
|/
* Merge branch 'benbrown/RT44'Ben Brown2013-12-2013-551/+149
|\
| * LocalArtifactCache now takes a an FS objectDaniel Firth2013-12-204-53/+40
| |
| * Removed unused CacheDir classDaniel Firth2013-12-203-215/+0
| |
| * Replaces Tempdir with fs.tempfs.TempFSDaniel Firth2013-12-205-152/+50
| |
| * Refactored localrepocacheDaniel Firth2013-12-202-131/+59
|/
* Merge branch 'baserock/pedroalvarez/fix-cross-bootstrap6'Pedro Alvarez2013-12-205-4/+88
|\ | | | | | | Reviewed-by: Richard Maw
| * Add yarn test for cross-bootstrapPedro Alvarez2013-12-202-3/+69
| |
| * Fix old cross-bootstrap testPedro Alvarez2013-12-201-1/+2
| |
| * Deploy and cross-bootstrap commands no longer validate host architectureBen Brown2013-12-202-0/+17
|/
* Merge branch 'benbrown/quote-filename-and-ref'Ben Brown2013-12-182-10/+15
|\
| * Quote ref and filenameBen Brown2013-12-182-10/+15
|/ | | | | | Previous fix only quoted URLs, which fixed petrify --no-git-update, but not the whole problem, quoting ref and filename prevents other problems that may be caused by non URL friendly characters.
* Merge branch 'benbrown/S9850/strip-morph-extension'Ben Brown2013-12-165-5/+38
|\
| * Modify morph to strip .morph extensions from parametersBen Brown2013-12-165-5/+38
|/
* Merge branch 'benbrown/S9823/petrify-no-update'Ben Brown2013-12-112-7/+9
|\
| * urllib: Convert URLs with non URL friendly characters into a valid format.Ben Brown2013-12-112-7/+9
|/
* Merge remote-tracking branch ↵Lars Wirzenius2013-11-2919-205/+972
|\ | | | | | | 'remotes/origin/baserock/richardmaw/S9475/refactor-build-cmd-v3'
| * plugins: Convert deploy to new classes.Richard Maw2013-11-291-85/+70
| | | | | | | | | | | | | | | | | | It now does not push branches as this is not necessary to locate the artifact. It still makes temporary build branches, since it is assumed that if you have changes in your workspace, it's preferable for the deploy to fail, rather than think you've deployed something you haven't.
| * yarns: Change expected result of building uncommitted morphologiesRichard Maw2013-11-291-3/+1
| | | | | | | | | | The new build code uses `git update-index --add`, which means it can use morphologies that haven't previously been added.
| * plugins: Use new build command as defaultRichard Maw2013-11-292-2/+2
| | | | | | | | The old build is still around for comparison.
| * plugins: Add new build commandRichard Maw2013-11-291-0/+92
| | | | | | | | | | | | This uses all the new APIs, so the code is shared across morphlib and unit tested rather than everything being in one massive plugin that is only black-box tested.
| * morphlib: Add BuildBranch abstractionRichard Maw2013-11-293-0/+263
| | | | | | | | | | | | | | This is an abstraction on top of SystemBranchDirectories, providing the ability to add uncommitted changes to the temporary build branch, push temporary build branches and retrieve the correct repository URI and ref to build the system.
| * branchmanager: Allow deferred and optional cleanup on success.Richard Maw2013-11-292-14/+179
| | | | | | | | | | | | | | | | | | | | | | | | Now it will optionally clean up on success based on a constructor parameter. It can be later cleaned up explicitly by calling close(). It is called close, rather than something more obvious, like cleanup(), since it means the manager can be re-used with contextlib.closing(). This now means that using the Managers without a context manager is less ugly, since you can explicitly call .close() in a finally block.
| * sysbranchdir: Move load_all_morphologies helper hereRichard Maw2013-11-293-10/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was previously a private method of the branch and merge plugin, but it's useful to other plugins, so has been moved to the SystemBranchDirectory class, where everything else can get to it. It has an unpleasant amount of coupling to other classes, but in a *good* object oriented design it would either be a tiny module on its own, or not exist and leave all its users to re-implement the same logic multiple ways, so we've opted for a less clean, but more useful design. It is left un-covered by the unit tests, since it requires a great deal of instrumentation to test, at which point it may be best to leave it to integration tests.
| * yarns: un-parameterise architecture in system morphologiesRichard Maw2013-11-293-10/+18
| | | | | | | | | | | | It doesn't make sense to be able to specify an architecture from the IMPLEMENTS name, since you either need your architecture for something to build, or testarch for something that consistently doesn't build.
View Lorry Controller Status