summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Add write and configuration extensions to help.baserock/markdoffman/s10382/add-help-option-v2Mark Doffman2014-02-211-38/+51
| | | | | | | Add a command 'help-extensions' to list all extensions. Add the ability to find help on an extension by calling 'morph help [extension name]'. This will then call the extension with the '--help' option to obtain help text.
* Add utilities for listing and finding extensions.Mark Doffman2014-02-214-40/+156
|
* Make '--help' and help subcommand the same.Mark Doffman2014-02-211-16/+53
| | | | | '--help' when used with a subcommand will show the subcommand help. Do not reflow the help text by using a custom formatter.
* Merge branch 'baserock/richardmaw/S10407/field-order'Richard Maw2014-02-1813-204/+176
|\ | | | | | | Reviewed-by: Lars Wirzenius
| * cmdtests: Use new morphology field orders in tag testsRichard Maw2014-02-172-8/+8
| |
| * Legacy morph code: Also dump in specified orderRichard Maw2014-02-174-125/+12
| | | | | | | | | | This rips out any remaining order-preserving code and instead uses the yaml dumper from morphloader.
| * cmdtests: Update to use new orderingRichard Maw2014-02-177-74/+71
| |
| * morphloader: Dump dicts with a nicer key orderRichard Maw2014-02-172-5/+93
|/
* Merge remote-tracking branch ↵Richard Maw2014-02-173-6/+36
|\ | | | | | | | | | | | | | | 'origin/baserock/richardmaw/10312/ensure-not-add-repo-ref' This includes a typo fix of Unpertify -> Unpetrify Reviewed-by: Lars Wirzenius
| * yarns: Ensure machine altered morphologies don't include repo/refRichard Maw2014-02-143-1/+32
| |
| * yarns: Remove code to check refs to strataRichard Maw2014-02-141-5/+4
|/ | | | | | | | It should no longer be possible to set these fields. Either we create the morphologies with them in a GIVEN, at which point we already know it's there, so checking whether it is there is pointless; or we check that morph doesn't create them, but the current implementation would not allow it, since yarn doesn't have THEN NOT.
* Merge branch 'baserock/richardmaw/S10310/invalid-repo-ref-v2'Richard Maw2014-02-1439-767/+222
|\ | | | | | | Reviewed-by: Sam Thursfield
| * Warn and default to null if repo or ref are givenRichard Maw2014-02-139-63/+158
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This now means that the system morphology is not altered when chunks are altered, so some tests had to change. Since this uses the python warnings API, these warnings can be ignored by running python -W ignore:"stratum morphology" \ -W ignore:"system morphology" \ "$(which morph)" ...` or turned into errors with python -W error:"stratum morphology" \ -W error:"system morphology" \ "$(which morph)" ...`
| * Remove tests for working with null refsRichard Maw2014-02-137-180/+1
| | | | | | | | | | Refs should be completely omitted, and this is now the standard behaviour, so there's little value in testing the behaviour separately.
| * Don't use repo or ref in test suiteRichard Maw2014-02-1315-79/+15
| |
| * Allow omitting repo and ref when referring to strataRichard Maw2014-02-136-34/+35
| |
| * Allow tagging branches with null refsRichard Maw2014-02-131-27/+13
| | | | | | | | | | | | | | This includes various changes allowing the repo and ref fields to be missing; but also a change to the the component_key function, so that instead of generating a string, it returns a tuple, since it's only required to be a consistently hashable index.
| * Remove edit-updates-stratum-build-depends testRichard Maw2014-02-122-95/+0
| | | | | | | | | | | | | | | | | | | | This test stopped being fit for purpose after an accidental edit made it no longer check the stratum build depends. However, rather than fixing it, it should be removed, since we're changing morph's behaviour to not support morphologies in different repositories, which means that updating these refs is no longer necessary.
| * Remove test for tagging with multiple morphs reposRichard Maw2014-02-122-289/+0
| | | | | | | | | | We don't support this any more, it makes morphologies complicated to implement and reason about.
* | Update copyright in READMESam Thursfield2014-02-141-1/+1
| |
* | Add note to README about running Morph from a Git checkoutSam Thursfield2014-02-131-0/+4
|/
* Update Copyright and wrap lines longer than 79 charsPedro Alvarez2014-02-041-2/+3
|
* Improve help for gc commandrichardipsum/gc_plugin_improve_helpRichard Ipsum2014-01-301-6/+9
|
* Fix copyright year in previous commitRichard Maw2014-01-281-1/+1
|
* Merge branch 'baserock/richardmaw/S10140/vbox-3.4.0-deploy'Richard Maw2014-01-281-1/+19
|\ | | | | | | Reviewed-by: Pedro Alvarez, Lars Wirzenius and Rob Taylor
| * virtualbox-ssh: Work around change in VBox optionsRichard Maw2014-01-281-1/+19
|/ | | | | | | | | | | | VirtualBox changed a command line option in 4.3 incompatibly, so we now have to check the version number and change an option from --sataportcount to --portcount if the version of VirtualBox running on the target is at least 4.3 This turns the version into a tuple and compares it against another, since it's more reliable than comparing strings, which will count '1.10' as earlier than '1.2', and more convenient than comparing the digits individually.
* Merge branch 'benbrown/S9933/validate-chunk-refs'Ben Brown2014-01-222-0/+69
|\
| * Add validation for chunk refs in strata to be (non-empty) stringsBen Brown2014-01-222-0/+69
|/
* Merge branch ↵Richard Maw2014-01-224-17/+95
|\ | | | | | | | | | | 'baserock/richardmaw/S10166/test-key-collision-with-artifact-names-v2' of git://git.baserock.org/baserock/baserock/morph Reviewed-by: Lars Wirzenius
| * yarns: Add regression test for cache-key clashRichard Maw2014-01-223-0/+52
| |
| * yarns: Move system artifact inclusion implementationRichard Maw2014-01-223-16/+42
| | | | | | | | | | | | This moves the GIVEN system $system uses $artifacts from $source to the generic implements section, and the Python implementation into the edit-morph helper script.
| * yarns: install misc files in /etc, not /usr/etcRichard Maw2014-01-221-1/+1
|/
* Merge branch 'baserock/richardmaw/S10135/fixups-v2' of ↵Richard Maw2014-01-224-6/+102
|\ | | | | | | | | | | git://git.baserock.org/baserock/baserock/morph Reviewed-by: Sam Thursfield
| * cache-key: Fix obscure cache key collisionRichard Maw2014-01-223-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If two systems with the same name (e.g. different repo/ref) depend on the same strata, then it will collide with systems which depend on different artifacts from that stratum, but the same number of artifacts. For example, if you checkout an existing branch and change the artifacts used by one of its strata, then your local changes won't be built. This is because the 'kids' field lists artifacts it depends on by their cache-key, which is now no longer sufficient to uniquely identify artifacts. The same number of artifacts issue is from it listing cache keys multiple times. The fix for this is to include the artifact name, so the 'kids' field is now a list of dicts, with artifact name and cache key. This is a dict rather than a tuple so that the generated /baserock metadata is more readable.
| * cache-key: Don't reduntantly hash products fieldRichard Maw2014-01-221-2/+6
| | | | | | | | | | | | For chunks the products field doesn't need to be hashed, since the split-rules field is used instead, and includes the default rule set, and for strata and systems it is handled by hashing the dependencies.
| * yarns: Add test for not building unnecessary chunksRichard Maw2014-01-221-0/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's an unintended, but possibly not unwelcome feature, that if you have a system that includes stratum artifacts that don't require building the whole stratum, that those chunks are not built. It's debatable whether the confusion of being able to have chunks defined that aren't built outweighs reduced build-times, indeed the test works by having a chunk that can never successfully build, which shows that one pitfall is the possibility of not noticing some chunks fail to build. However, whether it's a good thing or not, we should document this behaviour with a test, so that we don't accidentally break behaviour that someone depends on.
| * yarns: Add commentary for splitting scenariosRichard Maw2014-01-221-0/+36
|/
* Merge branch 'baserock/pedroalvarez/armhf'Pedro Alvarez2014-01-222-2/+24
|\
| * Add armv7lhf detectionRichard Ipsum2014-01-211-1/+22
| |
| * Add armv7lhf to list of valid archsRichard Ipsum2014-01-211-1/+2
|/
* Merge artifact splitting workRichard Maw2014-01-1755-930/+3058
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rationale ========= This patch series implements the concept of stratum splitting. For a long time we've had code to split a chunk into multiple artifacts, however there's not been a way to split strata up, or systems select a subset of the produced stratum artifacts to be included in the system. This patch series implements the ability to split strata and have systems include them in a way which still has the same behaviour if no rules are specified, but with default rules that split chunk artifacts up into various components, strata into runtime and development versions and has systems include everything by default, but can be told to include less. The default rules have chunk foo split up into -bins, -libs, -devel, -doc, -locale and -misc. These rules can be overridden in the chunk morphology by adding the new 'products' field, which lists match rules like the following: products: - artifact: libudev include: - (usr/)?lib(32|64)?/lubg?udev\..* - artifact: udev include: - (usr/)?s?bin/udev* - (usr/)?lib(32|64|exec)?/systemd/systemd-udevd Strata are by default split into -runtime and -devel. -devel by default contains chunks ending with -devel and -doc, -runtime contains everything else. Extra match rules can be added to a stratum similarly to chunks, but instead of matching file names, they match artifact names. products: - artifact: core-python include: - "cpython-.*" # lazy shortcut to put all of cpython in this stratum - "python-.*" # lazy shortcut to include all python chunks in Additionally, in chunk specs, chunk artifacts may be assigned to stratum artifacts, this takes precedence over products match rules in the stratum and the default match rules. Assigning the chunk to `null` will discard the chunk. chunks: ... - name: systemd ... artifacts: libudev: foundation-runtime udev: foundation-runtime systemd-doc: null By default a system includes every produced artifact of every stratum listed. Instead a subset can be specified in the stratum spec as follows: name: tiny-system strata: - name: build-essential ... artifacts: - build-essential-runtime
| * yarns: Add tests for building systems with splittingRichard Maw2014-01-162-0/+138
| | | | | | | | | | | | | | This includes tests for systems with the default splits and a system that selects only one of the produced stratum artifacts to go into the system artifact, since this is roughly the expected use-case for the tiny system morphologies.
| * cmdtests: Take into account new constructed artifactsRichard Maw2014-01-1629-215/+1809
| |
| * Use given artifact when constructing StrataRichard Maw2014-01-161-8/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | For some reason we used to create a new Artifact object with the same name as the Stratum morphology for our cache key. This is non-sensical, since we already have an Artifact object and it breaks splitting strata. NOTE: cmdtest tests do not pass, since they list files and artifacts produced, which has changed since the new default splitting rules were added. The next patch fixes this, but was kept as a separate commit for readability.
| * Unit tests: Fix up for changes to chunk construct apiRichard Maw2014-01-161-24/+15
| |
| * Split chunk morphologies according to new rulesRichard Maw2014-01-162-85/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Filenames are now matched before chunks are constructed, so bins.create_chunk now takes a list of relative file names. bins.chunk_contents is gone, since this is now handled by passing source.split_rules.partition the file names. We now don't consider it to be a problem for directories to remain in the DESTDIR after artifacts have been removed, since we need to handle file matches implying their parent directories, and explicit matches against directories. NOTE: The bins_tests were broken in this patch, and are fixed in the next. This was done to try and aid readability of the patch series. Full functionality is still broken until stratum splitting is fixed.
| * ArtifactResolver: Generate dependencies from split rulesRichard Maw2014-01-162-527/+256
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One important change is that the builds_artifacts field of Morphologies is not used any more, since the split rules provide this information. Another important change is that the ArtifactResolver now only returns aritfacts that are required to build the root artifact, rather than every artifact in the build. Previously there was no distinction. This is required because when artifact splitting is in effect, some artifacts may be produced, but not depended on by anything. This confuses the BuildCommand, which expects to be able to find a single root artifact. NOTE: This change breaks artifact construction until "Split chunk morphologies according to new rules" and "Split Stratum artifacts according to new rules", since systems and strata depend on artifacts that weren't created.
| * CacheKeyComputer: Include split rules in computationRichard Maw2014-01-162-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | Chunk artifacts need the [(artifact_name, [regular_expression])] so that if the default split rules change, or the blending rule changes, then an extra version field doesn't need to be added to the cache key computer. This is for future plans to allow the split rules to be configurable and allow us to more easily change them. System and Stratum artifact computations don't need this, since those splitting rules are already expressed in the dependencies information.
| * Add split rules to sourcesRichard Maw2014-01-164-2/+299
| | | | | | | | | | | | | | | | | | This introduces a new artifactsplitrule module, which tries to provide a nice abstraction over matching a sequence of things to a bunch of outputs, to be used by both chunks splitting, for separating files out into chunk artifacts, the stratum splitting, where chunks are aggregated into stratum artifacts, and systems selecting the right strata to go into the artifact.
| * MorphologyFactory: validate new fields using loader codeRichard Maw2014-01-161-0/+2
| |
View Lorry Controller Status