summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* yarns: Use a more realistic simulation of a git serverbaserock/richardmaw/yarn-git-daemonRichard Maw2014-08-287-11/+62
| | | | | | | | | | | Previously we would use file: URIs to point to the git repositories. This was fast and simple, but had the drawback that it bypassed all the git cache logic, so changes to the git cache weren't adequately covered by the test suite. Now we spool up a simulated git server per scenario, and shut it down at the end.
* Add script for launching git daemon on random portRichard Maw2014-08-281-0/+46
| | | | | | | This starts the git daemon procvess in inetd mode, so we can bind to an ephemeral port, and still be able to report which port was used, so we can construct git URIs using a non-standard port to talk to simulated git servers.
* Add yarn for building uncommitted changesRichard Maw2014-08-282-56/+33
| | | | This replaces an old cmdtest.
* yarns: Use run_morph for pushing branchesRichard Maw2014-08-281-1/+2
| | | | We're using the wrong morph if we don't use run_morph
* Remove cmdtests deprecated by yarnsRichard Maw2014-08-282-82/+0
| | | | | "morph makes temporary build branches for uncommitted changes when necessary" makes these redundant.
* Merge remote-tracking branch 'origin/baserock/richardmaw/bugfix/http-fail'Richard Maw2014-08-262-5/+12
|\ | | | | | | | | Reviewed-by: Francisco Redondo Marchena Reviewed-by: Sam Thursfield
| * Only autodetect morphology when result is 404baserock/richardmaw/bugfix/http-failRichard Maw2014-08-192-5/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The MorphologyFactory class will use a RemoteRepoCache to see if a morphology file exists, and if it doesn't, uses a file listing to see if it can detect what build-system is uses, hence what the default morphology should be. However, it was overly generic in what error cases it would accept as the morphology not being found, so if the RemoteRepoCache was suddenly un-resolvable for a brief period, then it would assume the morphology didn't exist, and use the default one. This happened to a user, and the result was a full rebuild. So we now fix this by only raising the exception that means the file didn't exist, if we got a HTTP 404.
* | Fix `morph edit` for non-file URIsRichard Maw2014-08-261-1/+1
| | | | | | | | | | | | | | | | | | The clone_into function is non-functional when you pass it a sha1 ref. If you have a file:// URI then this doesn't get used, which is how it slipped past the tests. Reviewed-by: Lars Wirzenius +2
* | Prevent git-replace refs affecting git operationsRichard Maw2014-08-2111-139/+171
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We assumed that the sha1 of the tree of the commit of the ref we care about was sufficient to cache, but `git replace` means that you need to know the state of other branches too, since anything in refs/replace can completely change what the tree you check-out is. This behaviour can be disabled globally by setting GIT_NO_REPLACE_OBJECTS, so we're going to do that. If we need to integrate a project that uses git-replace to change the contents of their git trees then we could support that by: if any(refs/replace/*): potentially_replacable_objects = [ `git rev-parse HEAD`, `git rev-parse HEAD^{commit}`, `git rev-parse HEAD^{tree}`] potentially_replacable_objects.extend( `git ls-tree -r HEAD | awk '{print $3}'`) # NOTE: doesn't handle submodules, and you'd need to expand this # set whenever you process a replacement for object in refs/replace/*: if basename(object) not in potentially_replacable_objects: continue cache_key['replacements'][basename(object)] = `git rev-parse $object` If we were to support this would need to traverse the tree anyway, doing replacements, so we may as well use libgit to do the checkout anyway, and list which replacements were used. However, since the expected use-case of `git replace` is as a better way to do history grafting, we're unlikely to need it, as it would only have any effect if it replaced the commit we were using with a different one. Rubber-stamped-by: Daniel Silverstone
* | deploy: Check correct usage of --upgrade for rawdisk deploymentsSam Thursfield2014-08-191-0/+21
| | | | | | | | | | | | | | | | | | | | This avoids confusion when the user expected to be doing an initial deployment and wasn't aware that a file with the same name as the target already existed. Previously rawdisk.write would try to mount the file and upgrade it. Now we require the user to pass '--upgrade' when they intend to upgrade, as with other deployment extensions.
* | Cache configuration values in GitDirectory.Daniel Silverstone2014-08-191-2/+6
|/ | | | | | | | This reduces the vast number of 'git config -z core.bare' which we used to get a lot of, to one or two per run. It doesn't save much time, but it does make logs less full of confusion. Reviewed-By: Richard maw <richard.maw@codethink.co.uk> +2
* Merge branch 'baserock/straycat/distbuild-trivia'Sam Thursfield2014-08-181-30/+9
|\ | | | | | | | | Reviewed-By: Adam Coldrick <adam.coldrick@codethink.co.uk> Reviewed-By: Sam Thursfield <sam.thursfield@codethink.co.uk>
| * Rename for consistencyRichard Ipsum2014-08-141-10/+9
| |
| * Remove dead functionsRichard Ipsum2014-08-141-20/+0
| |
* | Merge branch 'baserock/franred/fixes-needed-for-organize-definitions'Francisco Redondo Marchena2014-08-152-26/+40
|\ \ | |/ |/| | | | | Reviewed by: Richard Maw <richard.maw@codethink.co.uk> Merged by: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
| * Fix morphloader to unset_defaults for chunksFrancisco Redondo Marchena2014-08-151-2/+13
| |
| * Update morphloader_test.pyFrancisco Redondo Marchena2014-08-151-12/+13
| |
| * Set chunk static default to Nonebaserock/franred/fixes-needed-for-organize-definitionsFrancisco Redondo Marchena2014-08-151-12/+12
| |
| * Add system-integration to chunk _static_defaultsFrancisco Redondo Marchena2014-08-151-0/+1
| |
| * Add deploy-defaults before deploy in the MorphologyDumper keyorderFrancisco Redondo Marchena2014-08-151-0/+1
|/
* Merge branch 'baserock/richardmaw/james/writeexts_support_jetson'James Thomas2014-08-142-22/+83
|\ | | | | | | | | Reviewed by: Richard Maw <richard.maw@codethink.co.uk> Merged by: James Thomas <james.thomas@codethink.co.uk>
| * Merge remote-tracking branch 'origin/baserock/james/writeexts_support_jetson'baserock/richardmaw/james/writeexts_support_jetsonRichard Maw2014-08-122-22/+83
| |\
| | * Add support for a device tree to be set using DTB_PATHbaserock/james/writeexts_support_jetsonJames Thomas2014-08-031-0/+21
| | |
| | * Make bootloader config/install more genericJames Thomas2014-08-031-17/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the BOOTLOADER environment variable and instead favour BOOTLOADER_CONFIG_FORMAT to set the desired bootloader format, and BOOTLOADER_INSTALL to set the type of bootloader to install. For example, since u-boot can boot using extlinux.conf files, it's conceivable that someone might want to do CONFIG_FORMAT=extlinux.conf, INSTALL=u-boot. However, for most platforms you would want to set INSTALL to "none"
| | * Support setting a different root device using ROOT_DEVICEJames Thomas2014-08-031-2/+5
| |/
* | Merge branch 'baserock/adamcoldrick/remove-morph2-tweaked'Adam Coldrick2014-08-1450-2212/+1016
|\ \ | | | | | | | | | | | | Reviewed-by: Richard Maw
| * | Rename morph3 to morphologyAdam Coldrick2014-08-146-57/+57
| | | | | | | | | | | | | | | | | | Instead of leaving morph3 with a potentially confusing name, rename it to `morphology` since it is now the only implementation of the Morphology class.
| * | Revert "Ensure that none of the _orig_* fields are in the cache key"Adam Coldrick2014-08-141-4/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 4124c50b8dc3dfb0ffb933153d0fe6385edf389c. This commit is no longer needed now that morph2 is gone. Conflicts: morphlib/cachekeycomputer.py
| * | Remove morph2 and its testsAdam Coldrick2014-08-143-705/+0
| | | | | | | | | | | | This commit removes the now unneeded morph2 and its associated tests.
| * | edit-morph: Use morphloader rather than morph2Adam Coldrick2014-08-141-52/+8
| | | | | | | | | | | | | | | | | | Update the edit-morph script used in the test suite to use morphloader for saving/loading morphologies rather than morph2. Also remove some unused code.
| * | cmdtests: Use YAML not JSON to represent morphologiesAdam Coldrick2014-08-1421-867/+549
| | |
| * | cmdtests: Change the error messages to be detectedAdam Coldrick2014-08-144-4/+4
| | |
| * | unittests: Make the unittests use morphloaderAdam Coldrick2014-08-146-297/+254
| | | | | | | | | | | | | | | This commit removes all use of morph2 from the unittests, replacing it with morphloader/morph3. It also converts the test morphologies to YAML.
| * | buildsystem: Generate a Morphology not textAdam Coldrick2014-08-144-30/+29
| | | | | | | | | | | | | | | Rather than generating the text of a morphology which is later loaded, generate a Morphology object and return that.
| * | morphologyfactory: Use morphloader not morph2Adam Coldrick2014-08-142-166/+75
| | | | | | | | | | | | | | | | | | | | | | | | This commit reworks morphologyfactory to stop using morph2. It removes the validation that is already handled by morphloader, and uses morphloader to load the morphology rather than using morph2. The unit tests are also changed to turn the example morphologies into YAML, and also to check for the correct exceptions.
| * | morphloader: Get commands when loading morphologyAdam Coldrick2014-08-144-3/+14
| | | | | | | | | | | | | | | | | | Rather than having a `get_commands` method to obtain missing commands from the build system when they are needed, get the commands when loading a morphology.
| * | morphloader: Add and remove some default valuesAdam Coldrick2014-08-142-12/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | This commit stops morphloader setting the `morph` field in chunk specs, and also makes it set defaults for the prefix and build-mode fields. Not setting the `morph` field is necessary as its presence in chunk specs is used by `traverse_morphs` to mean that the morphology file is in the definitions repository, not the chunk source repository. If we set a default value here, we end up looking for files which do not exist.
| * | morphset: Don't expect to be able to find `morph` or `name`Adam Coldrick2014-08-141-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | Currently, morphloader sets the morph field in chunk specs by default. A later commit will remove this behaviour, so morphset needs to stop expecting the `morph` field to exist in morphology specs. However, stratum specs don't have a `name` field so we can't expect to be able to find that either.
| * | show-dependencies: Use sanitise_morphology_path instead of stripping `.morph`Adam Coldrick2014-08-142-3/+3
| | |
| * | cachedrepo: Remove unused load_morphology methodAdam Coldrick2014-08-142-13/+0
|/ /
* | Merge branch 'baserock/richardmaw/misc-fixups'baserock/richardmaw/tmpRichard Maw2014-08-1225-848/+105
|\ \ | | | | | | | | | | | | Reviewed-by: Daniel Silverstone Reviewed-by: Pedro Alvarez
| * | Fix `morph edit` when repo has the same ref as system branchRichard Maw2014-08-123-6/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There was a check in it to see whether it needed to do the git branch and git checkout based on whether the name of the branch matched that in the morphology. This had a couple of problems: 1. Now that we aren't always building from HEAD, we need to be able to roll its commit back, so using the existing branch isn't always the best idea. 2. It only checks the "ref" field, not "unpetrify-ref", so even though we clone the right ref in there, it's checking the commit id against the system branch name, so would always try to re-create the branch, and fail when it already exists. So now, we remove the original ref and re-create it with our preferred HEAD. A better solution might be to change the clone logic to not automatically checkout HEAD, and instead require an explicit branch then checkout, but the initial clone logic is shared with build, and I didn't feel like tracking down all the different places that it was used.
| * | Make morph show-branch-root print the pathRichard Maw2014-08-1210-114/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | The help for the show-branch-root command said it returns a path, but the command and the yarns just showed the aliased url it was cloned from. Given I found myself needing the path in some scripts, not the repo url, I think it's more useful to reconcile the difference this way.
| * | yarns: Start from petrified refsRichard Maw2014-08-124-30/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | This is closer to our current workflow, where we are always petrified and update the ref field when we need to update a component. This required rearranging the operations to create the chunk repository before the definitions repository, and remove a check that assumes we weren't already petrified.
| * | Remove petrify and unpetrify commandsRichard Maw2014-08-1215-698/+1
|/ / | | | | | | | | | | | | We don't use this any more, and instead prefer to always keep definitions.git petrified, and update the refs ourselves. branch-from-image still uses some of the remaining petrify code.
* | Merge remote-tracking branch ↵Richard Maw2014-08-1210-133/+403
|\ \ | |/ |/| | | | | | | 'origin/baserock/richardmaw/S11416/no-unnecessary-temp-branches' Reviewed-by: Daniel Silverstone
| * Add yarns for checking the state of temporary build branchesbaserock/richardmaw/S11416/no-unnecessary-temp-branchesRichard Maw2014-08-122-0/+239
| |
| * Yarns: Preserve the output of morph foo in out-fooRichard Maw2014-08-121-2/+3
| |
| * Yarns: implement pushing a whole system branchRichard Maw2014-08-121-2/+1
| |
| * Yarns: misc fixesRichard Maw2014-08-122-3/+3
| |