path: root/scripts
Commit message (Collapse)AuthorAgeFilesLines
* scripts: Fix release-build to not depend on a system branchbaserock-15.34baserock/release/baserock-15.34Pedro Alvarez2015-08-191-3/+16
| | | | | Signed-off-by: Sam Thursfield <> Change-Id: I1c8d2ed5d9c06466bdaac1c1e914f5f9e3969e11
* sync with upstreamJavier Jardón2015-03-171-10/+18
| | | | Change-Id: Id3b868db36554541334dae3bd6363d5adc0289b8
* Include rsync as GPL3 chunkPedro Alvarez2015-03-111-0/+1
* - allow user to specify target system version namePaul Sherwood2014-11-071-10/+21
* Move os-init-script out of /rootAdam Coldrick2014-10-271-1/+1
* Stop assuming that known_hosts exists, and fix a typoAdam Coldrick2014-10-272-1/+5
* Mason: Provide moving parts needed for testing on OpenStackMichael Drake2014-10-271-0/+524
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Testing currently involves building a Baserock devel system, deploying it as a VM and testing that that deployed VM can build Baserock successfully. Originally Mason used scripts/release-test to do testing on a kvm host. In this patch the bits needed to do testing on an OpenStack host are provided. The new scripts/release-test-os script is based on the old scripts/release-test, and it uses `nova` to boot/delete/etc images and instances on OpenStack. The mason script, `` is updated to optionally run either scripts/release-test-os or scripts/release-test, depending on whether TEST_INFRASTRUCTURE_TYPE is set to 'openstack' or 'kvmhost' in the Mason's mason.conf. The `os.conf` file is sourced by ``, and should be updated to contain the relevant credentials and details for the OpenStack tenancy to be used for test deployments. When Mason creates a test OpenStack instance, there is potential for a race condition depending on whether ssh comes up before the cloud-init has finished resizing the instance's disc. If morph running on the test instance tries to build before the disc size is increased, it will fail complaining of insufficient free space. To eliminate this race, the cloud init script `os-init-script` is passed to `nova boot`. This touches a file after the disc is resized, which Mason checks for before it runs a `morph build`. The `os.conf` and `os-init-script` files must both be placed in the Mason system's `/root/` directory before the system is deployed. This should happen in the `mason.configure` configuration extension. The `mason.configure` configuration extension should also be updated to handle adding two extra variables to the `mason.conf` file. These are the aforementioned TEST_INFRASTRUCTURE_TYPE and OPENSTACK_NETWORK_ID, which is the ID for the configured OpenStack network that test instances should use.
* Merge remote-tracking branch ↵Richard Maw2014-10-101-5/+12
|\ | | | | | | | | | | 'origin/baserock/ps/cycle-refuses-to-delete-TEST-system' Reviewed-by: Richard Maw (+2)
| * Tidy up a bit, and don't let user run this in TEST systemPaul Sherwood2014-10-091-4/+11
* Strip GNU patch from genivi baseline systemsPedro Alvarez2014-10-031-0/+1
| | | | | Reviewed-by: Paul Sherwood Reviewed-by: Sam Thursfield
* Add cycle script for build+deploy to selfPaul Sherwood2014-10-021-0/+43
* Deploy Masons to use source on upstream trovesRichard Maw2014-08-281-2/+9
| | | | | | | | | | | | | | | | | The per-mason trove only needs to worry about being an artifact cache, so we can prevent it populating itself from the upstream trove by making it use the SSH protocol for fetching sources, and not registering its ssh key with the upstream trove. The MASON_UPSTREAM_TROVE_ADDRESS option has been removed, as this is now the TROVE_HOST. The distbuild network is now configured to use the upstream trove for sources, and the local trove for artifacts, with the ARTIFACT_CACHE_SERVER option. mason.configure now uses ARTIFACT_CACHE_SERVER to tell deploy commands which server to fetch artifacts from.
* Update to use the new definitions tree.Pedro Alvarez2014-08-201-14/+12
* Use file: URI for list-artifacts in release-uploadRichard Maw2014-08-191-1/+1
| | | | | | This requires the script be run in the top of the definitions repository, but will actually try to upload the changes that were tested, rather than the current HEAD.
* Remove all VM storage together in one commandRichard Maw2014-08-191-4/+1
* Never remember hosts when running ssh commands on instancesRichard Maw2014-08-191-4/+2
| | | | | The rest of the ssh commands are to the VM host, which we can't change all of, since some are run as part of the deployment extension.
* Rename wait_online to wait_until_onlineRichard Maw2014-08-191-2/+2
* Rename Instance -> DeployedSystemInstanceRichard Maw2014-08-191-2/+3
| | | | This name requires less context to understand its use.
* Various style fixes to release-testRichard Maw2014-08-191-4/+7
* Implement load_cluster_systems in terms of load_morphologyRichard Maw2014-08-191-2/+1
* Rename MorphologyFrobber -> MorphologyHelperRichard Maw2014-08-191-13/+12
| | | | It was a sucky placeholder name that wasn't replaced by anything better.
* Add a script to organize morpholgies into definitionsbaserock/franred/organize-morphologies-script_v2Francisco Redondo Marchena2014-08-181-0/+266
* Add script for testing releasesRichard Maw2014-08-051-0/+402
| | | | | This will deploy and run build tests on systems listed in the morphology passed as its first argument.
* Pass cluster morphology to release upload script.baserock/michaeldrake/release-upload-clusterMichael Drake2014-08-011-9/+12
| | | | This makes the release-upload script more versatile.
* Add --upload-build-artifacts settingLars Wirzenius2014-07-301-1/+13
* Join short lines into one line, for readabilityLars Wirzenius2014-07-301-2/+1
* Avoid running rsync if source file list is emptyLars Wirzenius2014-07-301-1/+4
* Refactor long method into smaller onesLars Wirzenius2014-07-301-19/+37
| | | | For comprehensibility.
* Add markers around list in debug logLars Wirzenius2014-07-301-0/+2
* Simplify logic and conditionLars Wirzenius2014-07-301-3/+2
| | | | | Avoiding a condition that has a negation tends to be a bit simpler for humans to understand.
* Refactor process_args to be clearerLars Wirzenius2014-07-301-9/+14
| | | | | Move stuff into new methods to make overall logic clearer and to avoid stuffing too much into each method.
* Add --upload-release-artifacts settingLars Wirzenius2014-07-301-13/+9
* Remove unnecessary leading _ from methodLars Wirzenius2014-07-301-3/+3
* Allow release deployments with system name ≠ deployment namebaserock/michaeldrake/ci-release-scriptsMichael Drake2014-07-301-24/+21
| | | | | There used to be a check that prevented deployments with names different to the system. I don't know why this was, but I don't think we need it.
* Fix releasing when no changes to the cache are requiredMichael Drake2014-07-301-6/+9
| | | | | Without this change the rsync and xargs commands will wait forever for input that will never arrive.
* Allow release scripts to be configured to only handle a subset of architecturesMichael Drake2014-07-302-1/+33
| | | | | | | | | | | We currently build all architectures at once during the release process, however for our CD pipeline we operate with one CD pipeline per architecture. This is not just useful for the CD pipeline work though, as it allows one organisation to handle releases for x86, where the infrastructure may be located in the cloud, and one organisation to handle ARM systems, which may be located in an office.
* Allow release-upload to be configured not to upload release imagesMichael Drake2014-07-301-3/+21
| | | | | | | | | For continuous artifact cache population, we don't care so much about the large disk images that we make available at release time. This patch allows omitting any of the configuration required to upload the release images to mean that we didn't want to upload them, and continue without doing so.
* Allow release-build to build subsystems of defined systemsMichael Drake2014-07-301-1/+8
* Write release images into release subdirectoryMichael Drake2014-07-301-2/+1
| | | | | | | | The script used to chdir into the release directory before running morph deploy. Unfortunately, this didn't work because deployments are run from the top of the definitons repository. So now the release directory is included in the path to be deployed.
* Remove old release scriptsLars Wirzenius2014-07-242-687/+0
* Chmod uploaded filesLars Wirzenius2014-07-241-1/+18
| | | | Suggested-by: Sam Thursfield
* Add new scripts for building, uploading releaseLars Wirzenius2014-07-244-0/+547
| | | | | | | | | | | | | | | | | | | | | | These scripts are a rewrite of scripts/ and scripts/distbuild-cluster. The biggest difference is that they split the tasks of building the things that are to be released, and uploading them to /, where combines both (and distbuild-cluster only builds chunk/stratum/system artifacts, not the release images). The new scripts are also configurable using command line options or a configuration file rather than requiring editing of the source. These changes will allow, for example, a CI job that builds a release, but doesn't upload it to The new scripts are coupled with a change to the release process, which will be documented as a change to the release process page on The 14.29 release of Baserock was done with slightly different versions of these scripts to make it feasible to upload things over multiple network connections.
* Tidy up the bits that the user is meant to edit.baserock/michaeldrake/fix-release-branchMichael Drake2014-07-141-3/+3
| | | | | Fix string quoting. Put all stuff that needs changed in angle brackets.
* Fix to use correct release branch.Michael Drake2014-07-141-2/+3
| | | | Replaces references to `master` with release tag name.
* Merge branch 'baserock/michaeldrake/distbuilt-runner-cli'Michael Drake2014-07-101-14/+43
|\ | | | | | | | | Reviewed-By: Richard Maw <> Reviewed-By: Lars Wirzenius <>
| * Allow use without editing the script, by passing params on cli.Michael Drake2014-07-101-14/+43
| | | | | | | | | | | | | | - Adds ability to pass cluster, ref and distbuild controller hosts on the command line. - Adds --help, and usage. - Prints the specified parameters out.
* | do-release: 3 small cleanupsSam Thursfield2014-07-081-4/+2
|/ | | | | Remove a comment that is no longer true, fix a formatting error, and add a docstring to a class that lacked one.
* do-release: take advantage of partial deploymentSam Thursfield2014-07-071-54/+33
| | | | | | | | | | | | | | | | | | The 'morph deploy' command now lets us deploy system images one at a time, so let's do that. This means that if all but one image is deployed successfully, on the next run the user just needs to deploy one further image. Also, since each deployment has a unique name in release.morph now, we can override the location and VERSION_LABEL fields instead of requiring the user to update them manually before each release. The release.morph cluster should now specify the *basename* of the image in the location field only. By basename, I mean the system name plus the appropriate extension (normally .tar or .img). The do-release script will then prepend the image path and the version label to get a filename. The release.morph cluster has been updated accordingly.
* do-release: Explain why xargs is not used in a commentSam Thursfield2014-07-041-1/+4
| | | | | This was suggested by Richard Maw and I forgot to fix up the branch 'sam/auto-release' before merging it.
* do-release: Make use of str.rstrip()Sam Thursfield2014-07-041-3/+1
| | | | | This was a review comment from Richard Maw which I forgot to include before merging branch 'sam/auto-release'