summaryrefslogtreecommitdiff
path: root/morphlib/plugins/build_plugin.py
Commit message (Collapse)AuthorAgeFilesLines
* WIP Add support for multiple sources per chunkTiago Gomes2016-03-251-0/+1
| | | | | | | | | TODO: - Add API to the cache server to retrieve a submodule commit from a given path. - Fix cross-bootstrap command. Change-Id: I3475c2bcb648a272fee33bc878a521f79d4e6581
* Remove (dist)?build-morphology commandsTiago Gomes2015-12-021-79/+0
| | | | Change-Id: Ifb2bc2e2084806fe70b6db96828a390d3082288f
* Prepare the removal of workspace and system branches codeTiago Gomes2015-11-251-10/+5
| | | | Change-Id: Id470c7a77a47c89118a5d9d0d23b2206d8a839e4
* Fix crash in `morph distbuild` commandSam Thursfield2015-06-231-1/+1
| | | | | | | | This was causing: UnboundLocalError: local variable 'original_ref' referenced before assignment Change-Id: I46e4b5d527d9ac5480a3d1fe5e6d631f0b7279dc
* Determine whether to include_local_changes from app.settingsSam Thursfield2015-06-221-3/+1
| | | | | | | Previously callers to DefinitionsRepoWithApp.source_pool() could override the user's value, but that's not actually used anywhere. Change-Id: I43b53e7b9fc937886c8e6e95947e5e2b6776d085
* Use DefinitionsRepo in build commandSam Thursfield2015-06-161-143/+74
| | | | | | | | | This allows using `morph build`, `morph distbuild` and `morph distbuild-start` from any Git checkout of a definitions.git repo, so nobody needs to use `morph checkout` or `morph branch` if they don't want to. Change-Id: I5fdfae0f8bec1953893e26f0d227e289da11fa84
* Fix prototype of create_source_pool()Sam Thursfield2015-05-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This fixes an issue in `morph list-artifacts` and probably other commands: ERROR: Command failed: morph list-artifacts --quiet file:///ws/master/git.baserock.org/baserock/baserock/definitions HEAD systems/build-system-armv7lhf-jetson.morph systems/weston-system-armv7lhf-jetson.morph systems/genivi-baseline-system-armv7lhf-jetson.morph Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 190, in _run self.process_args(args) File "/usr/lib/python2.7/site-packages/morphlib/app.py", line 290, in process_args cliapp.Application.process_args(self, args) File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 539, in process_args method(args[1:]) File "/usr/lib/python2.7/site-packages/morphlib/plugins/list_artifacts_plugin.py", line 67, in list_artifacts repo, ref, system_filename) File "/usr/lib/python2.7/site-packages/morphlib/plugins/list_artifacts_plugin.py", line 91, in list_artifacts_for_system status_cb=self.app.status) File "/usr/lib/python2.7/site-packages/morphlib/sourceresolver.py", line 641, in create_source_pool definitions_original_ref=original_ref) File "/usr/lib/python2.7/site-packages/morphlib/sourceresolver.py", line 583, in traverse_morphs definitions_absref, definitions_tree, visit) File "/usr/lib/python2.7/site-packages/morphlib/sourceresolver.py", line 408, in _process_definitions_with_children definitions_queue = collections.deque(system_filenames) TypeError: 'NoneType' object is not iterable The issue is that the sourceresolver.create_source_pool() function did not work correctly if passed a single filename. This was a regression from commit 4cc75039a78bd8aef9ef464bc0eb6c3ff16809d0. In order to have a logical function prototype, the create_source_pool() function now only takes a list of systems, instead of taking either a single system or a list. Change-Id: I8f5ec8859a7e26d8107ed1c268fe925818d080a5
* Fix distbuild-morphologyAdam Coldrick2015-05-071-1/+1
| | | | | | This fixes an error caused by not enough parameters being given to the InitiatorBuildCommand constructor in distbuild-morphology. Change-Id: I133bd2f267fd06cfe88a1cbf4711cc79ad00d209
* Check number of args given to build commandsRichard Ipsum2015-04-301-5/+38
| | | | Change-Id: I501632c4d7acde06e391df3752275a3afc334ba6
* distbuild: Add distbuild start and cancel functionalityLauren Perry2015-04-291-1/+30
| | | | | | | | | | | Add command for distbuild-start to build_plugin in morphlib, and create a boolean parameter to inform the initiator whether to disconnect the controller and leave the build running remotely. Add distbuild-cancel command to parse currently-running distbuild build-request IDs and cancel the one matching the given argument Change-Id: I458a5767bb768ceb2b4d8876adf1c86075d452bd
* Fix regression that caused original_ref to be lost in some buildsSam Thursfield2015-04-271-2/+3
| | | | | | | | | | Commit bd788c7219d8b ("Implement partial builds") caused the original_ref metadata to be lost in some cases. This meant that the metadata for built systems would have original_ref set to a SHA1 hex string, instead of the original named ref that the system was built from. Change-Id: Id31654ad60b411206ccc7ee70b46f005baa209ef
* distbuild: Don't check if there is free disk space on the initiatorSam Thursfield2015-04-221-6/+7
| | | | | | | This check is intended for local builds, not distbuilds. Much less disk space is needed to distbuild. Change-Id: Iaff77805119129cbf6584de418ef97c7f1b82ad8
* Implement partial distbuildsAdam Coldrick2015-04-021-4/+20
| | | | | | | | | | | | | | In addition to partial builds we also want to be able to do partial distbuilds, and distbuild uses a different codepath. This commit updates the distbuild code to know what to do if a partial build is requested. It only builds up to the latest chunk/stratum that was requested, and displays where to find the artifacts for each of the chunks/strata requested upon completion of the build. The usage is the same as for local builds. Change-Id: I0537f74e2e65c7aefe5e71795f17999e2415fce5
* Implement partial buildsAdam Coldrick2015-04-011-20/+88
| | | | | | | | | | | | | | | | | | This commit allows you to build only a single chunk or stratum and its dependencies, instead of a whole system. This will be useful for developers who want to quickly check the build commands of a chunk for example. You can give more than one chunk or stratum to the command, and the build will run up to the latest one, building the dependencies of both. An example of the usage: morph build systems/devel-system-x86_64-generic.morph \ strata/build-essential.morph strata/core/curl.morph This will only build up to curl in the core stratum. Change-Id: Id55b8335fc61e8fdf847f5e383cfedd13d07e6d2
* build: Clearer messages when creating temporary build branchesSam Thursfield2015-03-311-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | The temporary build branch (--local-changes=include) feature can be convenient, but it is also slow, and causes the resulting system to be 'unreproducible' (built from temporary commits). Before: $ morph build systems/devel-system-x86_64-generic.morph 2015-03-31 10:47:18 Starting build 53546647f92a4283bf96e33ea93e767e 2015-03-31 10:47:18 Collecting morphologies involved in building systems/devel-system-x86_64-generic.morph from master 2015-03-31 10:47:25 Building file:///src/ws/master/baserock/baserock/definitions 7735ec20d13490058a3d7758f79471637ce1ae71 systems/devel-system-x86_64-generic.morph 2015-03-31 10:47:25 Deciding on task order ... After: $ morph build systems/devel-system-x86_64-generic.morph 2015-03-31 10:46:21 Looking for uncommitted changes (pass --local-changes=ignore to skip) 2015-03-31 10:46:21 Creating temporary branch in /src/ws/master/baserock/baserock/definitions named refs/heads/baserock/builds/f0b21fe240b244edb7e4142b6e201658/8df11f234ab24d22a9616ce911542332 2015-03-31 10:46:28 Building file:///src/ws/master/baserock/baserock/definitions 7735ec20d13490058a3d7758f79471637ce1ae71 systems/devel-system-x86_64-generic.morph 2015-03-31 10:46:28 Deciding on task order ... Change-Id: I21a3e65c29adf0b07f743289c6b3f0f4dddc34be
* Use the modern way of the GPL copyright header: URL instead real addressJavier Jardón2015-03-161-2/+1
| | | | Change-Id: I992dc0c1d40f563ade56a833162d409b02be90a0
* Treat path given to build cmd as relative to cwdRichard Ipsum2015-01-231-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the build commands treat the system argument as a path relative to the root repo. This means that regardless of your working directory you must run morph build systems/foo-system.morph This behaviour can be confusing, for example when your working directory is $systembranch/definitions/systems you might expect to be able to run morph build foo-system.morph especially since most shells would tab-complete the filename for you. At the moment running the above command from $systembranch/definitions/systems would result in an error, because morph would look for $systembranch/definitions/foo-system.morph rather than $systembranch/definitions/systems/foo-system.morph This behaviour also means you can't give the morph build commands an absolute path to a system morph. This patch changes the treatment of the system arg so that it is interpreted relative to the current working directory.
* Make temporary build branches optionalSam Thursfield2015-01-061-8/+54
| | | | | | | | | | | | | | | Not everyone is a fan of the `morph build` magic that collects up your changes and puts them in a temporary branch. Now you can disable it by setting 'local-changes=ignore' in your morph.conf file. This speeds up `morph build` and `morph deploy` by 5-10 seconds on my machine. I looked an option to make `morph build` warn if there are uncommitted changes. I found that with a cold cache, it takes about 5 seconds on my machine to verify that there are no uncommitted changes to a checkout of definitions.git. That defeats the main purpose of this patch for me, so I didn't include the option.
* build: Resolve the ref being built in the user's local definitions.gitSam Thursfield2014-10-241-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Most usefully, this patch means that Morph no longer updates its cached copy of definitions.git every time you run `morph build`. Also, it prevents confusion in the following situation. Imagine I have run: morph checkout baserock:baserock/definitions master I then wait a while, during which time someone pushes to 'master' in the definitions.git repo that I cloned from. Now I run: cd master morph build systems/whatever.morph Which commit does it build, the local head of 'master' or the remote head of 'master'? The answer, both before and after this patch, is that it builds the local version of master. But previously, this only happened because of the magic that we have to detect local changes. With this patch, the local change detection could be disabled and `morph build` would still build what the user had checked out as 'master' locally, not whatever 'master' pointed to in the remote repo.
* build: Restructure how BuildCommand.build() is calledSam Thursfield2014-10-241-3/+5
| | | | | | | | | Rather than take a list of triplets to build, the BuildCommand.build() function now takes a single repo/ref/morph triplet. Iterating through multiple sets of triplets is now done in the build plugin. There are a couple of cosmetic changes to the status output at the start and end of a build as a result.
* Avoid creating and pushing temporary build branches when they aren't necessary.Richard Maw2014-08-121-32/+7
| | | | | | | | | | | | | | | | | | | | | Sorry about the big lump, I can split it into a nicer set of changes, but they didn't naturally emerge in a nice series. This creates a pushed_build_branch context manager, to eliminate code duplication between build and deploy. Rather than the build branch being constructed knowing whether it needs to push the branch, it infers that from the state of the repositories, and whether a local build would be possible. If there are no uncommitted changes and all local branches are pushed, then it doesn't create temporary branches or push them, and instead uses what it already has. It will currently create and use temporary build branches even for chunks that have no local changes, but it's pretty cheap, and doesn't require re-working the build-ref injection code to check whether there are local changes.
* Turn BuildBranch methods into regular functionsRichard Maw2014-08-121-5/+9
| | | | | | | | | | | | | | | | | | | | | | | | Previously they were generator functions, which yielded interesting context at interesting times so that the caller could respond by printing status messages. The only benefits this had over callbacks were: 1. 1 fewer function scope to worry about. I don't have data on the amount of memory used for a function scope vs a generator, but it could be less cognitive load for determining which variables are defined in the callback's body. 2. It is possible to yield in the caller, so you could make that into a coroutine too, however this wasn't required in this case, as the yielded value was intended to be informational. The downsides to this are: 1. It's a rather peculiar construct, so can be difficult to understand what's going on, and the implications, which led to 2. If you call the function, but don't use the iterator it returned, then it won't do anything, which is very confusing indeed, if you're not used to how generator functions work.
* Use exact filenames to refer to morphology filesRichard Maw2014-07-101-7/+8
| | | | | | | | | | | | | | | | | | | Rather than repeatedly stripping and appending an optional .morph extension morphology names, instead always use the file path of the morphology relative to the definitions repository. This is an inversion of the previous logic, which would strip the .morph extension and use the "name" internally. The exception to this rule of always using the filename, is that `morph edit CHUNK` uses the name of the morphology as-defined in the stratum. This is based off Adam Coldrick's inital patch, but this version will allow the old style of providing the "name" by converting it into a path if it does not have either a / or a . in it. An unfortunate consequence of this change is that the show-dependencies command's output changed, so the test needed updating.
* Add distbuild-morphology command.Sam Thursfield2014-05-141-0/+23
| | | | | | | | | This is useful to build releases using distbuild. It avoids having the SHA1 fields in the artifact metadata files pointing to commits that exist only on temporary build branches. It also avoids file:// URLs in the repo fields. Note that the repo URL still points to the Trove used by the distbuild network, rather than being an upstream URL pointing to git.baserock.org.
* UI change: distbuild has its own subcommandRichard Ipsum2014-03-261-7/+39
|
* Modify morph to strip .morph extensions from parametersBen Brown2013-12-161-1/+1
|
* plugins: Use new build command as defaultRichard Maw2013-11-291-1/+1
| | | | The old build is still around for comparison.
* plugins: Add new build commandRichard Maw2013-11-291-0/+92
| | | | | | This uses all the new APIs, so the code is shared across morphlib and unit tested rather than everything being in one massive plugin that is only black-box tested.
* Improve docstring for "morph build-morphology"Lars Wirzenius2013-07-051-3/+20
|
* S7904: Add disk space checks before build & deployRichard Maw2013-06-051-1/+8
| | | | | | | The same check that cachedir and tempdir are large enough is used for both build and build-morphology. Deploy only checks for tempdir being large enough.
* Improve command descriptions in --helpSam Thursfield2012-10-261-8/+3
| | | | | Make sure all commands have one line of description, and reduce the size of some which had large amounts of text.
* Rename 'morph build' command to 'morph build-morphology'Jannis Pohlmann2012-09-071-2/+2
| | | | | | | | | | This is needed because we are replacing the current build command by a new one that builds from a local system branch. Since that will be the more common scenario, we want the new implementaiton to go by the name 'morph build'. This commit therefore renames 'morph build' and updates all the tests to use 'morph build-morphology' instead.
* Move cmd_build to a separate pluginRichard Maw2012-08-011-0/+47
It's such a small amount of code, it's possibly not worth it, but now all commands are in plugins.
View Lorry Controller Status