summaryrefslogtreecommitdiff
path: root/morphlib
Commit message (Collapse)AuthorAgeFilesLines
...
| * Refere to .write.help file for documentationPete Fotheringham2014-11-282-26/+11
| |
| * Improve readability by adding blank linesPete Fotheringham2014-11-281-0/+6
| |
| * Add more information on existing parameters.Pete Fotheringham2014-11-261-8/+16
|/ | | | Add AUTOSTART parameter
* Document the write extension in the write.help filePete Fotheringham2014-11-262-23/+26
| | | | | - Move docstring from .write to .write.help - Rework the content and formatting of the help information
* Whitespace and line endings?Pete Fotheringham2014-11-251-11/+10
|
* Merge branch 'sam/ensure-git-directory-exists'Sam Thursfield2014-11-172-12/+31
|\ | | | | | | | | Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk> Reviewed-By: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
| * Check that directory passed to GitDirectory is a git repoSam Thursfield2014-11-172-12/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, creating a GitDirectory object for something that wasn't a Git repository would succeed, but most operations would raise a cliapp.AppException with the following error message: ERROR: Command failed: git config -z core.bare Now, the constructor will raise a NoGitRepoError if the directory isn't a Git repo, which (unless handled) gives the user the following sort of message: ERROR: Directory /src/ws/definitions is not a Git repository
* | Merge branch 'sam/fix-list-artifacts'Sam Thursfield2014-11-141-1/+1
|\ \ | | | | | | | | | Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
| * | Fix list-artifacts commandSam Thursfield2014-11-141-1/+1
| | | | | | | | | | | | | | | Silly mistake caused this to be broken, and there are no tests for this command inside the morph.git source tree.
* | | Unrevert "Make system-integration commands use containerised_cmdline"baserock/richardmaw/fixup-ro-integrationsRichard Maw2014-11-121-55/+5
| | | | | | | | | | | | | | | The bug in the generated command lines has been fixed, so we can have this cleanup back now.
* | | Make invert_paths work more reliably for writable-allRichard Maw2014-11-122-41/+51
|/ /
* | Revert "Make system-integration commands use containerised_cmdline"Richard Maw2014-11-111-5/+55
| | | | | | | | | | | | | | | | | | | | | | This reverts commit 203f34b2b19b1d3dc51a956ae9329d064ea294dd. The path inversion logic is still buggy for the case of wanting to keep the top-level directory writable. This escaped detection in the test suite because it resulted in directories two levels down being made read-only, and I was only testing one level down.
* | Allow `morph deploy` to cache missing Git repos if it needs toSam Thursfield2014-11-111-1/+0
|/ | | | | | | | | | | | | | | | | | | | | | | | This reverts commit c63f4e61f69820c71c2d8e9b96ce9bdec0d476a1. This change was introduced back when the Baserock system definitions used named Git refs rather than exact SHA1s. Morph would thus need to update each repo involved in a build and resolve each ref before it could build. Now that the system definitions use SHA1s, the 'updating gits' phase is much less slow. The problem with forcing 'no-git-update' is that it makes `morph deploy` depend on the remote repo cache. In the case that all repos involved in the build are avaiable in the remote repo cache (Trove), there is no need for them to have been cached locally just to construct a build graph. But if the network connection to the Trove is unreliable Morph will need to fall back to the local repo cache. Unless Morph has done a full local build of the system being built, it might not have every repo cached locally. Often, cached artifacts from the Trove or distbuild may be used instead of locally building everything. In this case the user sees `morph deploy` failing with errors such as: Repository upstream:binutils-redhat is not cached yet
* Add docstring to SourceResolver class.Sam Thursfield2014-11-101-1/+22
|
* Set the 'update' flag in the SourceResolver constructorSam Thursfield2014-11-101-12/+14
|
* Move create_source_pool code into new 'sourceresolver' moduleSam Thursfield2014-11-105-136/+185
| | | | | | | | This code is an essential part of 'morph build'. It's quite complex and really shouldn't be mixed in with the base Application class. Given a dedicated class we can store some state in the object and avoid functions with seven parameters, too.
* Remove one place where we pass the Application instance aroundSam Thursfield2014-11-063-14/+7
| | | | | | The MorphologyFactory class only uses the status() function of the morphlib.Application instance that it gets passed. So make it require only a status callback.
* Merge branch 'sam/gitdir-fixes-v2'Sam Thursfield2014-11-064-14/+39
|\ | | | | | | | | Reviewed-By: Pedro Alvarez <pedro.alvarez@codethink.co.uk> Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk>
| * Improve documentation of GitDirectory classSam Thursfield2014-11-051-5/+13
| | | | | | | | Clarify that bare repositories are supported, and other fixes.
| * Only search for repository root in GitDirectory constructor if told toSam Thursfield2014-11-054-9/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The GitDirectory() constructor, if passed a 'dirname' that doesn't contain a '.git' subdirectory, can search upwards to find the real root of the repository. This is used by the `add-binary`, `push`, and `pull` commands. This causes very confusing behaviour in the case that 'dirname' points to a directory that should be a Git repository, but isn't, and that directory is a path inside the working tree of another Git repository. Rather than raising an error, in this case the GitDirectory class would perform operations on a different repository to the one the caller expected. This 'search_for_root' behaviour is now opt-in, to avoid confusion.
* | Merge branch 'sam/gitdir-ref-exists'Sam Thursfield2014-11-062-0/+14
|\ \ | | | | | | | | | | | | Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk> Reviewed-By: Jim MacArtuhur <jim.macarthur@codethink.co.uk>
| * | Add GitDirectory.ref_exists() methodSam Thursfield2014-11-052-0/+14
| |/ | | | | | | | | This allows checking if a ref exists without requiring the caller to use a try:, except: block.
* | Use the default symlink when creating the orig subvolume.Pedro Alvarez2014-11-052-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch solves the issue caused by upgrading a system without a factory version. Currently we are only using the factory version to snapshot its orig subvolume to make faster the transfer of the new content (rsync won't have to send everything). The default symlink may not be present, but it can't be deleted easily using system-version-manager. This is a quick fix, but in the future we may want to not harcode the path from where we snapshot the orig subvolume. Or improve system-version-manager to make sure that the default symlink is always present.
* | Check for commit object in submodule pathbaserock/franred/check-for-commit-object-in-submodule-pathFrancisco Redondo Marchena2014-11-041-1/+1
|/ | | | | Fix checking for commit object in submodules which their path and name are not the same.
* Merge remote-tracking branch 'origin/sam/transfer-status'Sam Thursfield2014-10-314-15/+45
|\ | | | | | | | | Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk> Reviewed-By: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * Echo stderr of subcommands that do network IO when --verbose is usedSam Thursfield2014-10-294-9/+42
| | | | | | | | | | | | | | | | | | | | | | | | Morph can appear to hang in situations where it is actually waiting on a slow network operation. This change gives users a way to see the output of the subcommands that are doing the network IO (either 'wget', 'git clone' or 'git remote update'). The status information goes onto stderr, because that is where the subcommands write it. Morph tends to put its status output on stdout, but (a) some commands are machine-parsed, such as `serialise-artifact` and (b) it's tricky to get Git to put status output on stdout.
| * Remove workaround for an old version of GitSam Thursfield2014-10-291-6/+2
| |
| * Don't say 'Updating git repository xxx' unless we actually areSam Thursfield2014-10-291-1/+2
| | | | | | | | | | | | | | This fixes an issue where various branch-and-merge commands appeared to be updating Git repos even when --no-git-update was specified. The flag was actually honoured but the message made it seem as if it was being ignored.
* | stagingarea: Remove vestigial pre-command mounting logicRichard Maw2014-10-301-22/+2
| |
* | Make build commands use containerised_cmdlineRichard Maw2014-10-301-31/+14
| |
* | Make deployment extensions use unshared_cmdlineRichard Maw2014-10-301-4/+1
| |
* | Make system-integration commands use containerised_cmdlineRichard Maw2014-10-301-55/+5
| |
* | Move unsharing and containerising logic to utilRichard Maw2014-10-301-0/+114
| | | | | | | | | | This way the build commands, system integration commands and deployment extension commands can all share the logic.
* | Allow invert_paths to skip the top level directoryRichard Maw2014-10-291-0/+5
|/ | | | | | We're going to be passing it "$CHROOT/" as a writable directory by default, so it needs to be able to tell that it means to leave everything writable.
* Fix lines longer than 79 charactersSam Thursfield2014-10-291-2/+2
|
* Merge remote-tracking branch 'origin/sam/build-fix-3'Sam Thursfield2014-10-292-11/+17
|\ | | | | | | | | Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk> Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
| * Fix distbuild to allow passing a commit instead of a named ref to be builtSam Thursfield2014-10-272-11/+17
| | | | | | | | | | | | | | | | | | | | | | The recent changes to the BuildCommand.build() function caused distbuild to break, because I didn't make the same change to the InitiatorBuildCommand.build() function but did change how it was called. This commit adds the ability to have optional fields in distbuild messages. This is used to add an optional 'original_ref' field, which will get passed to `morph serialise-artifact` by new distbuild controllers, and will be ignored by older ones.
* | Make pushed_build_branch always return (url, commit, ref)Richard Ipsum2014-10-281-2/+7
|/
* Fix confusing messagebaserock/ps/message-fixPaul Sherwood2014-10-271-1/+1
|
* build: Resolve the ref being built in the user's local definitions.gitSam Thursfield2014-10-243-7/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | Most usefully, this patch means that Morph no longer updates its cached copy of definitions.git every time you run `morph build`. Also, it prevents confusion in the following situation. Imagine I have run: morph checkout baserock:baserock/definitions master I then wait a while, during which time someone pushes to 'master' in the definitions.git repo that I cloned from. Now I run: cd master morph build systems/whatever.morph Which commit does it build, the local head of 'master' or the remote head of 'master'? The answer, both before and after this patch, is that it builds the local version of master. But previously, this only happened because of the magic that we have to detect local changes. With this patch, the local change detection could be disabled and `morph build` would still build what the user had checked out as 'master' locally, not whatever 'master' pointed to in the remote repo.
* build: Allow passing in the name of the original ref to create_source_pool()Sam Thursfield2014-10-242-7/+15
| | | | | This means that we can force the building of a specific commit without losing the original branch name in the metadata of the resulting system.
* build: Restructure how BuildCommand.build() is calledSam Thursfield2014-10-242-15/+19
| | | | | | | | | Rather than take a list of triplets to build, the BuildCommand.build() function now takes a single repo/ref/morph triplet. Iterating through multiple sets of triplets is now done in the build plugin. There are a couple of cosmetic changes to the status output at the start and end of a build as a result.
* Merge branch 'baserock/richardmaw/parallelism-improvements'Richard Maw2014-10-243-11/+23
|\ | | | | | | | | | | | | | | This stripped out the commit from the patch series on the mailing list to remove /dev/shm. Reviewed-by: Sam Thursfield Reviewed-by: Daniel Silverstone
| * stagingarea: Mount proc and ccache inside the namespaceRichard Maw2014-10-242-10/+18
| | | | | | | | | | | | | | | | | | | | | | This works towards allowing multiple concurrent builds in the same system, which has the same problem as deployments. This is the easy bit, since linux-user-chroot has support for bind mounts and /proc mounts. We also need to get rid of the /dev/shm mount to be able to build in parallel though.
| * deploy extensions: Don't crash if someone builds at the same timeRichard Maw2014-10-241-1/+3
|/ | | | | | | | | | | | | | If a build happens, it creates a new network namespace, and if this happens while you have a disk image mounted, then you can't remove the mount-point, because the other namespace is using it. We can avoid the other namespace keeping this mount-point open by creating the disk image in a private mount namespace, so it never sees it. The nicest way to do this is to have every extension run in a private mount namespace, since you'd have to have extensions re-exec themselves, since the appropriate system calls aren't exposed very well.
* Run system-integrations in a namespaceRichard Maw2014-10-241-23/+53
| | | | | | | | | | | | | | | All temporary mounts should be done inside a namespace, so that they don't interfere with other namespaces. system-integrations may need capabilities that regular builds don't have, so they're chrooted, rather than linux-user-chrooted, which means it's more complicated to do namespaces. In the absence of a better command for it, we can do this with an in-lined shell script. This also stops us using the run-parts inside the system, and executes the integrations directly.
* Fix synopsis of `morph edit` commandSam Thursfield2014-10-221-1/+1
| | | | | Previously it was incorrect in the `morph help` output and in Morph's man page.
* Give clearer error when the same 'name' is used by two strataSam Thursfield2014-10-161-5/+4
|
* Don't print the ref of each stratum in the buildSam Thursfield2014-10-161-3/+0
| | | | | | | There's no need because all strata live in the same Git repository now. This message was added back when there could be multiple versions of strata coming from different repos or different refs in the same repo and life was very confusing.
* Merge branch 'baserock/richardipsum/arfix'Richard Maw2014-10-131-74/+51
|\ | | | | | | | | | | Reviewed-by: Richard Maw Reviewed-by: Richard Ipsum Reviewed-by: Sam Thursfield