summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| * | | | Show a message when the build is startingAdam Coldrick2015-02-181-1/+1
| | | | |
| * | | | Add tests for sourceresolverAdam Coldrick2015-02-183-14/+341
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This only adds tests for the bits which were moved from morphologyfactory into sourceresolver, namely detection of build systems and the '_get_morphology()' function. These are just the morphologyfactory tests reworked slightly to work properly with the modified API.
| * | | | Read files from a local clone of definitions where possibleSam Thursfield2015-02-181-1/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most morphologies involved in a build are in the definitions repo these days. Currently we read each of them using `git cat-file`, which is slow. It's quicker to check out all the files in one go to a temporary directory and then read them from there. With the current workflow users often have definitions.git checked out on disk. It seems strange to not just read the files from there. There are two reasons why I don't want to do that yet: - there are commands which don't run inside a system branch, which would be broken if we expected to always be in a system branch - there may be local changes in the checked-out repo, and it takes around 5 seconds on each build to check if there aren't any local changes. It actually seems faster to just check out a known clean version from the cache.
| * | | | Add a mechanism for extracting all files from a given commit to a dirSam Thursfield2015-02-184-6/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is nice because it's fast. We don't have to copy all the Git history along with it like we do with a clone. And it doesn't touch any files in the cached repo.
| * | | | Split up traverse_morphs to improve readabilitySam Thursfield2015-02-181-61/+79
| | | | |
| * | | | Add an LRU cache for detected build-systemsSam Thursfield2015-02-181-19/+101
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This will speed up builds of chunks which don't have a chunk morph. It won't have much (if any) effect on the speed of the first build, but subsequent builds will be much faster as we won't have to query the git cache.
| * | | | Add an LRU cache for resolved tree refsSam Thursfield2015-02-183-30/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This uses the PyLRU module, from: <https://pypi.python.org/pypi/pylru/1.0.6>. Python 3.2 and newer provide a built-in LRU cache, but this is specifically for in-memory use. See <http://bugs.python.org/issue17528>. Git commits are immutable, so caching information about their contents is fairly easy and trouble-free. There's no danger of the cache becoming stale.
| * | | | Add a cache manager utilising pylru and pickleAdam Coldrick2015-02-181-2/+52
| | | | |
| * | | | Move MorphologyFactory into SourceResolverSam Thursfield2015-02-184-381/+69
| | | | | | | | | | | | | | | | | | | | | | | | | There's no need for this stuff to be in a separate class. This allows integrating it with the caching in the SourceResolver class.
| * | | | sourceresolver: Add comments and factor out common functionSam Thursfield2015-02-181-13/+15
| | | | |
| * | | | Convert the empty-stratum test into a yarnAdam Coldrick2015-02-185-38/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We will use MorphologyLoader.load_from_file() to load morphologies from a temporary file, so the output will be unpredictable. That will break this test, so we may as well convert it into a yarn to fix it.
| * | | | Fix some yarns to shut down their git servers when finishedAdam Coldrick2015-02-183-0/+3
|/ / / / | | | | | | | | | | | | | | | | | | | | This commit fixes an issue where running morph's yarn tests would leave behind a number of scripts/git-daemon-wrap processes due to some tests not stopping their git server properly.
* | | | Merge branch 'sam/distbuild-errors'Sam Thursfield2015-02-134-71/+81
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | Reviewed-By: Pedro Alvarez <pedro.alvarez@codethink.co.uk> Reviewed-By: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
| * | | | Fix copyright yearsSam Thursfield2015-02-114-4/+4
| | | | |
| * | | | distbuild: Refuse to start controller if there are no workers listedSam Thursfield2015-02-111-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is another situation where builds could hang forever if the server is misconfigured. Longer term, workers should be able to come and go dynamically without needing to reconfigure and restart the controller process.
| * | | | distbuild: Give more detail when requests to cache-server failSam Thursfield2015-02-111-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Let the end-user see the URL that distbuild was attempting to talk to, so they can more easily spot configuration errors. It's kind of silly to say 'HTTP request failed' without saying where the request was going.
| * | | | distbuild: Simplify error when computing build graph failsSam Thursfield2015-02-111-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The previous error looked like this by the time it had reached the initiator's console: ERROR: Failed to build baserock:baserock/definitions c7292b7c81cdd7e5b9e85722406371748453c44f systems/base-system-x86_64-generic.morph.frodsham: Failed to compute build graph. Problem with serialise-artifact: ERROR: Couldn't find morphology: systems/base-system-x86_64-generic.morph.frodsham New message is at least a bit simpler: ERROR: Failed to build baserock:baserock/definitions c7292b7c81cdd7e5b9e85722406371748453c44f systems/base-system-x86_64-generic.morph.frodsham: ERROR: Couldn't find morphology: systems/base-system-x86_64-generic.morph.frodsham
| * | | | distbuild: Fix case where 'computing build graph' would hang foreverSam Thursfield2015-02-112-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If there's no distbuild-helper process running on the controller then the controller would hang forever. This situation is unlikely, but it's important to give the user feedback instead of silently hanging forever.
| * | | | distbuild: Simplify failure cases in BuildControllerSam Thursfield2015-02-111-43/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's no need to handle failure differently at each stage of the build. Simpler to use the BuildFailed message for all errors. This then allows us to have a single self.fail() function that can be used everywhere.
| * | | | distbuild: Rearrange code that sends exec-request messageSam Thursfield2015-02-111-8/+13
| | | | |
| * | | | distbuild: Write name of build worker to build-step-xx.log filesSam Thursfield2015-02-111-15/+33
|/ / / / | | | | | | | | | | | | | | | | | | | | | | | | Knowing which worker built something is useful for debugging, and right now that information is only present on the initiator's console. It's good to have it in the build-step-xx.log file too so the information doesn't get lost.
* | | | Merge branch 'sam/builder-cleanups'Sam Thursfield2015-02-111-21/+0
|\ \ \ \ | |_|/ / |/| | | | | | | | | | | Reviewed-By: Pedro Alvarez <pedro.alvarez@codethink.co.uk> Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk>
| * | | Remove two unused functionsSam Thursfield2015-02-111-17/+0
| | | |
| * | | Remove some ununsed importsSam Thursfield2015-02-111-4/+0
|/ / /
* | | Fix failing coverage testSam Thursfield2015-02-031-2/+2
| | |
* | | Update copyright years so ./check passesSam Thursfield2015-02-032-2/+2
| | |
* | | Merge branch 'baserock/tiagogomes/armv8'Tiago Gomes2015-02-031-1/+5
|\ \ \ | |/ / |/| | | | | | | | | | | Reviewed by * Pedro Alvarez * Sam Thursfield
| * | Add logic for contructing the correct GNU tripet for ARMv8Tiago Gomes2015-02-031-1/+5
| | |
* | | Merge branch 'jmac/remove-eth0-eth1-check'Pedro Alvarez2015-01-271-14/+0
|\ \ \ | |_|/ |/| | | | | | | | Reviewed-by: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk> Reviewed-by: Pedro Alvarez <pedro.alvarez@codethink.co.uk>
| * | Remove checks for NETWORK_CONFIG and eth0 and eth1 in itJim MacArthur2015-01-271-14/+0
|/ / | | | | | | | | | | | | | | network_config isn't used anywhere in this function. The purpose of this function (getting the name of an appropriate host-only network interface) doesn't seem to depend on it either. eth0 and eth1 won't always be present (several Baserock systems will have enp0s3, etc). So I think these checks should be removed.
* | Merge branch 'baserock/richardipsum/build-command-tweak-v3'Richard Ipsum2015-01-236-34/+124
|\ \ | |/ |/| | | | | | | Reviewed by: Adam Coldrick <adam.coldrick@codethink.co.uk> Richard Maw <richard.maw@gmail.com>
| * Fix copyrightRichard Ipsum2015-01-231-1/+1
| |
| * Fix an existing build yarnRichard Ipsum2015-01-231-1/+2
| | | | | | | | | | | | | | Fix "attempting to build a system morphology which has never been committed" use WHEN ... THEN morph succeeded The old yarn worked just as well, this change is just for consistency.
| * Make some existing deployment scenarios simplerRichard Ipsum2015-01-231-14/+4
| | | | | | | | | | Simplify 'deploying a cluster morphology as a tarfile' and 'attempting to upgrade a tarfile deployment'
| * Quote variables and use = instead of ==Richard Ipsum2015-01-231-3/+3
| |
| * Add yarns to test deploying with relative pathsRichard Ipsum2015-01-232-6/+58
| |
| * Treat path given to deploy cmd as relative to cwdRichard Ipsum2015-01-231-2/+4
| |
| * Add yarns to test building with relative pathsRichard Ipsum2015-01-232-0/+35
| |
| * Update tests to work with modified build commandsRichard Ipsum2015-01-231-6/+6
| | | | | | | | | | | | | | Existing tests generally request something like morph build systems/foo-system.morph, now that the system arg is treated relative to the working directory we must change into the directory that contains the morphs to run morph build systems/foo-system.morph
| * Treat path given to build cmd as relative to cwdRichard Ipsum2015-01-231-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the build commands treat the system argument as a path relative to the root repo. This means that regardless of your working directory you must run morph build systems/foo-system.morph This behaviour can be confusing, for example when your working directory is $systembranch/definitions/systems you might expect to be able to run morph build foo-system.morph especially since most shells would tab-complete the filename for you. At the moment running the above command from $systembranch/definitions/systems would result in an error, because morph would look for $systembranch/definitions/foo-system.morph rather than $systembranch/definitions/systems/foo-system.morph This behaviour also means you can't give the morph build commands an absolute path to a system morph. This patch changes the treatment of the system arg so that it is interpreted relative to the current working directory.
| * Add relative_to_root_repo to sysbranchdirRichard Ipsum2015-01-231-0/+6
|/
* Merge branch 'baserock/jmalk/cross-bootstrap-aarch64-v2'Sam Thursfield2015-01-212-5/+5
|\ | | | | | | | | | | | | | | Reviewed-By: Pedro Alvarez <pedro.alvarez@codethink.co.uk> Reviewed-By: Rob Kendrick <rob.kendrick@codethink.co.uk> Reviewed-By: Paul Sherwood <paul.sherwood@codethink.co.uk> Reviewed-By: Sam Thursfield <sam.thursfield@codethink.co.uk> Reviewed-By: Zara Zaimeche <zara.zaimeche@codethink.co.uk>
| * Add armv8l64 and armv8b64 to list of valid architecturesJosh Malkinson2015-01-212-5/+5
|/
* Fix line > 79 characters that was breaking ./checkSam Thursfield2015-01-191-1/+2
|
* Merge branch 'sam/optional-temporary-build-branches'Sam Thursfield2015-01-193-26/+95
|\ | | | | | | | | Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk> Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
| * Make temporary build branches optionalSam Thursfield2015-01-063-26/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Not everyone is a fan of the `morph build` magic that collects up your changes and puts them in a temporary branch. Now you can disable it by setting 'local-changes=ignore' in your morph.conf file. This speeds up `morph build` and `morph deploy` by 5-10 seconds on my machine. I looked an option to make `morph build` warn if there are uncommitted changes. I found that with a cold cache, it takes about 5 seconds on my machine to verify that there are no uncommitted changes to a checkout of definitions.git. That defeats the main purpose of this patch for me, so I didn't include the option.
* | Update CopyrightRichard Ipsum2015-01-191-1/+1
| |
* | Correct resolution of error_message_for_containerised_commandline method.Craig Griffiths2015-01-161-1/+1
| |
* | Fix copyright yearsSam Thursfield2015-01-122-2/+2
| |
* | Merge branch 'sam/cached-repo-cleanup'Sam Thursfield2015-01-1217-318/+214
|\ \ | |/ |/| | | | | Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk> Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
View Lorry Controller Status