summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| * | Rename method to MorphologyLoader.parse_morphology_textLars Wirzenius2013-08-152-6/+8
| | | | | | | | | | | | | | | | | | | | | The new name is more descriptive, since it turns out that the method doesn't just parse strings, but also open file handles. Suggested by Richard Maw.
| * | Drop use of explicit StringIO with the yaml moduleLars Wirzenius2013-08-151-5/+2
| | | | | | | | | | | | | | | | | | | | | It turns out the yaml module can handle strings directly, and does not need explicit file handles. Suggested by Richard Maw.
| * | Disable tests broken by "morph edit"Lars Wirzenius2013-08-1410-2/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To the best of my ability to understand the old "morph build" code, these tests fail with this error: Conflicting versions of stratum 'hello-stratum' appear in the build. because of how the old "morph build" code creates temporary build refs, and gets them wrong. I believe the new "morph edit" is correct, but have not been able to find out what the root problem in "morph build" is, because the code is beyond my ability to understand it within two hours. Since I have a deadline for "morph edit", I reluctantly disable these tests, temporarily, rather than back out of all "morph edit" changes.
| * | Disable merge testsLars Wirzenius2013-08-141-1/+1
| | | | | | | | | | | | | | | morph merge works badly enough that we don't really support it right now, and I don't have time to tweak "morph edit" to make the tests pass.
| * | Disable test cases (lack of time, may not need feature)Lars Wirzenius2013-08-143-2/+19
| | | | | | | | | | | | | | | | | | | | | We don't know if we want to support keeping system and stratum morphologies in different git repositories, and adding the support is too much work for this deadline. I'm keeping the tests, but disabling them, in case we want to enable it again.
| * | Fix test suite to patch YAML instead of JSONLars Wirzenius2013-08-1415-444/+361
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I shouldn't be doing this in the middle of refactoring, but trying to get the new code to emit exactly the right kind of JSON is too much pain, and these tests need to be rewritten to use YAML in the near future anyway (we'll drop JSON support eventually), so I don't want to spend more time on this than necessary. The changes in this commit convert morphologies in JSON to YAML, and fix test code that modifies the morphologies to work with the YAML. There is probably more JSON lurking about.
| * | Fix test morphology to be a valid oneLars Wirzenius2013-08-141-0/+3
| | | | | | | | | | | | | | | | | | The MorphologyLoader validates morphologies stricter than old code, so adjust the test morphology accordingly: add arch to a system morphology, and build-mode and build-depends to a stratum one.
| * | Re-implement "morph edit" using new infrastructureLars Wirzenius2013-08-142-2/+206
| | | | | | | | | | | | | | | | | | Test suite currently fails because MorphologyLoader validates differently from the old MorphologyFactory code, and because MorphSet changes refs more correctly (including, it seems, build-dependency refs).
| * | Add a MorphologySet class, for changing many morphologiesLars Wirzenius2013-08-143-0/+339
| | | | | | | | | | | | | | | | | | | | | Various parts of Morph need to change a set of morphologies at once, particularly for petrification and unpetrification. This is easiest done by loading all the morphologies into memory at once, and changing them there, then saving again.
| * | Load morphologies using the new mechanisms, not the oldLars Wirzenius2013-08-141-4/+4
| | |
| * | Add new morphology abstraction and morphology loading/savingLars Wirzenius2013-08-145-0/+912
| | | | | | | | | | | | | | | | | | | | | | | | | | | The old code is somewhat weird. The new code is meant to be cleaner and more straightforward to understand and to use. For example, the old code has setting of defaults in both the Morphology and MorphologyFactory classes. The new code has a minimally simple Morphology class, and has all the logic to validate and set defaults in the MorphologyLoader class. Further, the new code makes it possible to load an invalid morphology, which will be useful later.
| * | Make SystemBranchDirectory remember an absolute pathLars Wirzenius2013-08-141-1/+1
| | | | | | | | | | | | | | | | | | | | | This is necessary because a relative pathname may become invalid, if the current working directory changes, either in Morph itself, of when Morph invokes a subprocess and gives the pathname to the system branch root directory to it.
| * | Add SystemBranchDirectory.get_filename methodLars Wirzenius2013-08-142-0/+18
| | |
| * | Remove unused argument from SystemBranchDirectory.clone_cached_repoLars Wirzenius2013-08-143-7/+5
| |/ | | | | | | | | For some reason, there was an unused argument in the method. Remove it from the definition and all call sites.
* | Merge branch 'jonathan/disable-network'Jonathan Maw2013-08-151-0/+1
|\ \ | |/ |/| | | | | Reviewed-by: Lars Wirzenius <lars.wirzenius@codethink.co.uk> Reviewed-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * Disable network during buildsJonathan Maw2013-08-131-0/+1
|/
* Merge remote-tracking branch ↵Lars Wirzenius2013-08-133-55/+106
|\ | | | | | | | | | | | | 'origin/baserock/richardmaw/S8544/deploy-cleanup-v3' Reviewed-by: Lars Wirzenius Reviewed-by: Daniel Silverstone
| * deploy: Always cleanup deploy tempdir on failureRichard Maw2013-08-131-45/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a configuration extension or write extension fails, then it should abort then, not continue to run other configuration extensions. There was a period where this didn't happen, due to a missing feature of cliapp that was assumed to be there, so failure to run these extensions was not noticed. This has since been fixed, but this would cause deploy to fail to clean up its temporary directories. Now it will cleanup the contents of the temporary directory after any failures after it has been created. A small amount of re-ordering was performed to make this easier.
| * deploy: Parse arguments before unpackRichard Maw2013-08-131-3/+3
| | | | | | | | | | | | | | | | It's a waste of time to unpack the rootfs, only to have to clean it up again when you find out that you messed up the command line arguments. This also has the benefit of reducing the amount of resources that have to be considered for cleanup.
| * deploy: refactor environment argument parsingRichard Maw2013-08-133-7/+51
| | | | | | | | It is now a tested helper function in morphlib.util
| * deploy plugin: Clean up deployments' tempdirsRichard Maw2013-08-091-7/+7
|/ | | | | | | | | We can't assume an extension cleans up after itself, as they can be arbitrary shell scripts, and the best shell has to offer for cleanup is `trap`, which is difficult to use. So now, anything created with `mktemp` will get automatically cleaned up by morph.
* Merge commit '7819677343bb40e899fa2cd252c203013911fede'Lars Wirzenius2013-08-093-1/+56
|\
| * yarns: add a scenario for checking out a chunkRichard Maw2013-08-082-0/+30
| |
| * yarns: allow attempt checkout to sometimes workRichard Maw2013-08-081-1/+1
| | | | | | | | | | | | | | | | | | Checkout needs the branch repository and ref. It was previously only getting the ref. This was not noticed, since the implementation was only used in cases where it was expected to fail, and the nature of the error was not being checked.
| * workflow-tools: check branch roots have systemsRichard Maw2013-08-081-0/+25
| | | | | | | | | | | | | | | | It's rather easy to accidentally try to create a system branch of the `morph` repository rather than the `morphs` repository. This commit verifies that a checked out branch contains system morphologies, otherwise it aborts and cleans up.
* | Merge remote-tracking branch ↵Lars Wirzenius2013-08-095-15/+137
|\ \ | |/ |/| | | 'origin/baserock/richardmaw/S8532/validate-stratum-build-depends'
| * morphologyfactory: validate stratum build-dependsRichard Maw2013-08-072-0/+55
| | | | | | | | | | | | | | | | | | | | | | | | A stratum morphology must either have stratum build-depends, or have some bootstrap chunks, otherwise there's no way to have the required set of commands to be able to build chunks. A concession has been made to also allow strata that contain chunks built in test mode, but this opens a reproducibility hole. Unit tests for these failures have been added, and the stratum used by other unit tests has been fixed.
| * black-box-tests: Set build-mode: test in strataRichard Maw2013-08-072-0/+25
| | | | | | | | | | | | This is already needed elsewhere for tests that perform builds, however in future, validation will be more strict, so now these dummy strata need to be marked as tests as well.
| * morphologyfactory: move empty stratum check hereRichard Maw2013-08-073-10/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is better to test whether a straum is empty here, since it will be noticed earlier in the build, as soon as the morphologies are loaded, rather than after they have all been parsed. It is also conceptually nicer to put it here, since the morphologyfactory was written to perform this kind of validation. On a more practical note, the validation is moved here so that the test for this error isn't masked by the test for no build dependencies. To ensure tests still pass, we alter the stratum morphology used by other unit tests to no longer be empty, and add an empty one to test.
| * morphologyfactory: refactor chunk bdeps exceptionsRichard Maw2013-08-072-6/+30
| | | | | | | | | | | | | | | | | | This creates a small exception hierarchy for failures to validate stratum morphologies. This is currently a line rather than a tree, but it will be expanded later in the patch series. This also adds test coverage for chunk build dependencies being omitted.
* | workflow-tools: Fix typo in `morph help branch`Richard Maw2013-08-071-1/+1
|/ | | | Reviewed-by: Lars Wirzenius and Jonathan Maw on IRC
* Merge branch 'liw/refactor-checkout-etc-v2'Lars Wirzenius2013-08-0712-114/+985
|\ | | | | | | Reviewed-by: various
| * Re-implement "morph branch" using new infrastructureLars Wirzenius2013-08-062-42/+73
| |
| * Re-implement "morph show-branch-root" using new infrastructureLars Wirzenius2013-08-062-20/+18
| |
| * Re-implement "morph show-system-branch" using new infrastructureLars Wirzenius2013-08-062-8/+9
| |
| * Re-implement "morph checkout" using new infrastructureLars Wirzenius2013-08-062-37/+99
| |
| * Add SystemBranchDirectory classLars Wirzenius2013-08-063-0/+433
| |
| * Add GitDirectory classLars Wirzenius2013-08-063-0/+198
| |
| * Add LocalRepoCache.get_updated_repo methodLars Wirzenius2013-08-061-1/+13
| | | | | | | | | | This is a helper to avoid having the same code in other places. Have it in the one place it belongs.
| * Add Workspace.create_system_branch_directory methodLars Wirzenius2013-08-062-0/+27
| |
| * Add Workspace.get_default_system_branch_directory_name methodLars Wirzenius2013-08-062-0/+23
| | | | | | | | | | This will be useful later, when we start creating system branch directories.
| * Add morphlib.util.find_root and find_leaf functionsLars Wirzenius2013-08-062-6/+92
| | | | | | | | | | | | | | These will be used to find workspace and system branch root directories. Also accidentally removed some whitespace from ends of lines. Too lazy to split that into a separate commit.
* | Merge remote-tracking branch ↵Lars Wirzenius2013-08-064-12/+39
|\ \ | |/ |/| | | 'origin/baserock/richardmaw/S8530/atomic-hl-cache-creation'
| * stagingarea: move to failed on construction failRichard Maw2013-08-063-6/+21
| | | | | | | | | | | | | | | | | | This uses the same logic as when a build fails, so it's been consolidated into `StagingArea.abort()`. You could argue that if a build fails before any commands are run, then there's nothing interesting to see, but it will be useful if the hardlink/tarball extract algorithm fails in some corner case.
| * stagingarea: Construct hardlink cache atomicallyRichard Maw2013-08-061-6/+14
| | | | | | | | | | | | | | | | | | | | | | | | Paul had file system problems which led to a partially constructed chunk hardlink cache, which caused later builds to fail, since they got a partially extracted chunk. This patch fixes the case where the failed extract caused unreproducible builds, but it's possible to corrupt the hardlink cache in other ways. Read-only btrfs subvolumes would fix this, but either tie us further to btrfs, or complicates the codebase with fallback logic.
| * stagingarea tests: create chunks subdirectoryRichard Maw2013-08-061-0/+4
|/ | | | | | | | It is expected that the Application creates directories that will be used, to keep the code for using them cleaner. The FakeApplication wasn't updated to do this, as it was not yet needed, but changes for atomicity require it to exist.
* Merge commit '95a4918a471ad1dc7110d3d64eb7e6a89dea34a5'Lars Wirzenius2013-08-011-3/+9
|\
| * Set build-ref-prefix to be based on trove-id if the setting existsTiago Gomes2013-07-311-3/+9
| |
* | Merge branch 'liw/refactor-init-workspace'Lars Wirzenius2013-07-316-52/+264
|\ \ | |/ |/| | | Reviewed-by: Daniel Silverstone
| * Re-implement "morph workspace" using the Workspace classLars Wirzenius2013-07-313-8/+11
| | | | | | | | | | | | | | | | | | Put new implementation into new branch and merge plugin, and remove old implementation from the old plugin. Also change the error message for the NotInWorkspace exception, so that it matches what the test suite expects. It's a crappy error message, I think, but I don't want to change external behaviour during refactoring.
View Lorry Controller Status