summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* strip-gplv3: Force `busybox find` to use posix-basic regular expressions.baserock/pedroalvarez/genivi-kronosPedro Alvarez2015-08-131-1/+1
| | | | | | | We could also set the '-regextype posix-basic' option to do this in GNU find, but that would fail with busybox. Change-Id: Ideaa9950efc23d8351e2796b7c6f06988c8c0176
* Upgrade node-health-monitor to 1.3.5Pedro Alvarez2015-08-121-2/+2
| | | | Change-Id: Ided9707f30fe9f1f532af36430ee827403bf4195
* Upgrade audiomanager to 7.0Pedro Alvarez2015-08-121-2/+2
| | | | Change-Id: I8d8dce474d64fb2961d98a2945ccdc12fce7e228
* Upgrade to Common API 3.1.2Pedro Alvarez2015-08-123-10/+6
| | | | Change-Id: Ia664c7cbb8343aac98384bc3707dcf59a8e8dbe2
* mason: Improve mason-report.sh to give more feedback when building.Pedro Alvarez2015-08-101-6/+45
| | | | | | | It will now show a row when a build has started, with a link to the current build log. Change-Id: Iec01529a80b45b2a61a6cf454a0ac8300c73e139
* Move GNU File into core to fix MIPS buildPaul Martin2015-08-072-6/+16
| | | | | | | | | | | | | Recent autoconf (and autoconf generated ./configure files) uses the "file" program to work out what binary format is currently being produced by the compiler when the architecture is MIPS. If "file" doesn't exist, or it fails to identify the binary format, autoconf/configure silently disables the building of shared libraries, which causes later chunks to fail to build at link stage due to missing .so files. Change-Id: Ia98a22d121fc8cb95d36bcb1d3ae2ce44cc6ddf5
* Update libsoup referencePaul Martin2015-08-031-2/+2
| | | | | | This later version will be needed by rygel and its dependents. Change-Id: I5fd1f14108d722b5cb76f8fb74954fbe248ba2a3
* cpython: Use system libffi and libexpat librariesJavier Jardón2015-08-032-9/+15
| | | | Change-Id: I1e96327e30051079968e39db0032b98e0c81d352
* writeexts.py: Capture stderr so that errors are capturedEdward Cragg2015-07-301-1/+1
| | | | | | | | The conversion from cliapp to the direct use of subprocess left a call which did not capture stderr, so `stderr=subprocess.STDOUT` is added to maintain the expected behaviour. Change-Id: I3ee0c064f9813d15004234d4daea81dc8219fc42
* Update Morph build tool in reference systemsSam Thursfield2015-07-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This includes the following changes: Add support for Baserock definitions version 6 Remove support for Baserock definitions format versions 0, 1 and 2 Don't provide default strip-commands for 'manual' build system Extract downloaded tarball as current user Add bitbucket repo alias to default aliases sourceresolver_tests: Remove bogus test sourceresolver_tests: Fix invalid backup definition distbuild: Fix cache status message distbuild: Fix crash when worker disconnects distbuild: Fix partial distbuilding distbuild: Add __str__() and __repr__() to ArtifactReference distbuild: Hide a log message distbuild: Check cache status each time we enqueue new artifacts distbuild: Add docstrings to BuildController state machine Fix crash in `morph distbuild` command Make `morph show-build-log` look in local repo cache for build logs Make more commands work outside a system-branch/workspace Determine whether to include_local_changes from app.settings Raise exception if show-build-log doesn't find the build log Fix `morph show-build-log` for chunks of a different architecture localrepocache: Count freshly cloned repositories as updated Fix anchor yarn Add script to run a distbuild network on the local machine Don't ignore the 'cwd' parameter Expose reset_workdir in GitDirectory class Swap repr() for str() in ref add errmsg Improve error when local HEAD of definitions repo is not found in remote Use DefinitionsRepo class in deploy plugin Use DefinitionsRepo in build command Add DefinitionsRepo class Treat True and False as actual commands, rather than booleans. buildsystem: Comment strip command Change-Id: Ic55e12bfe5e98f4a1a4df875c23ad3d447f7038c
* Set migrations/006-specify-build-system.py to be executableSam Thursfield2015-07-291-0/+0
| | | | This was an oversight in the previous commit.
* Add migration for definitions version 6Sam Thursfield2015-07-281-0/+329
| | | | Change-Id: I80ce9eee253b25689f9a360047dc9b3e9b1cb12a
* migrations: Don't run 'indent' as a migrationSam Thursfield2015-07-281-1/+2
|
* Add migration scripts for historical versions of the definitions formatSam Thursfield2015-07-2811-4/+849
| | | | | | | | | | | | | | | | | | | | | | See README for more information on how the migrations are intended work. These migrations are probably not widely useful, as our definitions have already been migrated manually. However, I want to come up with a good pattern for writing migration scripts, and actually doing it seems like the best way. There is a 'migrations/indent' tool, which reformats a set of definitions according to how the ruamel.yaml program writes them out. This tool is nice if you like everything to have consistent indent and line wrapping, and you can run it before running the migrations to ensure that the migrations don't do any reformatting when writing the .moprh files back to disk. The migration scripts require the ruamel.yaml Python library. I have sent a separate change to add this to the 'build' and 'devel' reference systems. Change-Id: Ibd62ba140d3f7e8e638beed6d714f671405bdc79
* Remove completely xml-catalog from strata/virtualization.morphPedro Alvarez2015-07-271-1/+0
| | | | | | | | It was moved to strata/docutils.morph in the commit 490f87c9ccc4046caeb3ab41a067a9a5ad3ad95c and we missed this build depenency. Change-Id: Ic40ca21f2d066d5fe859e44c647fe5b42faaf9b3
* Upgrade perl to v5.22Richard Ipsum2015-07-2711-25/+232
| | | | Change-Id: I309f2ee3d9ff0569907c51ee9984b365b31cdbf5
* Add a simple samba stratumPaul Martin2015-07-272-0/+139
| | | | Change-Id: I46ca2add3ec07be66053bdf8b2a9956c680855b8
* Remove duplicate bsp from the build systemRichard Ipsum2015-07-212-4/+0
| | | | Change-Id: I40f9bfc413da1dc3ea5b8110fa7fc2ab7cb2275b
* strata/devtools.morph: Change wget's gnulib submodule locationJavier Jardón2015-07-191-2/+2
| | | | | | And upgrade from 1.16 to 1.16.3 at the same time Change-Id: I592f49b671ad0c4ea93f4be9a8231a7f87560a29
* Reformat all definitions according to a consistent styleSam Thursfield2015-07-1965-455/+489
| | | | | | | | | | | | | | | | | This was done using the 'indent' tool, which uses a fork of PyYAML named 'ruamel.yaml' to rewrite YAML files without losing comments, ordering, or certain elements of formatting. My aim with doing this is to open the door to automated editing of the reference system definitions using the 'ruamel.yaml' library. This can be used to implement automated migration when we want to make changes to the YAML format that we use to represent Baserock system definitions. Although this looks drastic, remember that it's actually only altered 65 out of 608 .morph files -- the vast majority already pass unchanged through my version of ruamel.yaml. Change-Id: I95ec978714b5bd1c02c90183336a9fbb846cb692
* Add ruamel.yaml library to 'build' and 'devel' systemsSam Thursfield2015-07-191-0/+12
| | | | | | | | | | | | This is currently done by adding it to the 'morph-utils' stratum, because it is used as a tool for working with .morph files. The ruamel.yaml library allows writing YAML files while preserving comments, ordering and some formatting form the input files. This makes it more practical for us to write automated migration scripts for Baserock definitions. Change-Id: I886232dad4d64bcf1803f52e94ae25b0fa2d9969
* Update cpython Python interpreter to latest stable versionsSam Thursfield2015-07-182-4/+4
| | | | Change-Id: I1faa4db9e3bb5c2e8e3ef80d88b3ef3e5cf2ae26
* Add Python 3 to all systems that contain Python 2 (cpython interpreter)Sam Thursfield2015-07-1837-0/+74
| | | | Change-Id: Ib66f3f56b60cc5dc78d08e28e281d120d83a7b9d
* Add wgetRichard Ipsum2015-07-172-0/+15
| | | | Change-Id: I2cd784c45a4e96f59b124ca9b9bb06993b9e3f12
* Update lorry to gain two new featuresRichard Ipsum2015-07-161-1/+1
| | | | | | | * "reproducible" tar lorries * zip support Change-Id: Id12e868c511996d7e44651bfb4b4c885cf9748c4
* Fix chunk name mismatches in 'build-depends' listsPedro Alvarez2015-07-152-2/+2
| | | | | | We forgot about this in 4a81346961031aecc50c657423bd8d68250fc40d Change-Id: I60c73295f622abff4bafeaebfa830db0d7efcd9a
* Fix 'name' field mismatchesSam Thursfield2015-07-0711-11/+11
| | | | | | | | | | | | | The 'name' field is set in both a chunk's definition, and in the chunk reference point in each stratum where that chunk is included. Some of them didn't agree on the name of the chunk. In all cases I've made 'name' match the filename of the chunk .morph file, either by fixing the chunk .morph file or the stratum .morph file (and in one case, the filename). This should have no effect on behaviour, it's just a tidyup. Change-Id: I90bcd3fa382e24c8aa2d370ea5f545491169d1db
* Make ssh-rsync.check check VERSION_LABELRichard Ipsum2015-07-041-0/+18
| | | | | | | | | As well as checking whether it's defined, we make sure there isn't already a system named $VERSION_LABEL Note, this depends on the new json-list feature of system-version-manager. Change-Id: I0dee5253ba95a002b493f4179c2acda734afc37a
* babel: Don't include the date in the version numberAdam Coldrick2015-07-031-3/+4
| | | | | | | | A patch to fix this has been sent upstream, but the upstream repository seems inactive. Pull request is here: https://github.com/mitsuhiko/babel/pull/164 Change-Id: I1170e8b66c3975459dfed8ffdf66f053c2c477a5
* ca-certificates: This chunk doesnt use autotools build systemJavier Jardón2015-06-291-4/+5
| | | | | | But a simple Makefile instead Change-Id: I0d73e2cf44fc97ba9f6452ea417c1b317b68efa6
* essential-files: create symlink /etc/os-release -> /usr/lib/os-releasebaserock/pedroalvarez/improve-mason-reportJavier Jardón2015-06-233-6/+8
| | | | | | | | | | | | | | | systemd is moving towards having /usr to be vendor operating system only, whilst /etc will be used for configuration only (the contents of this directory could be erased to do a factory reset) [1]. /usr/lib is the recommended place to place this file [2] [1] http://0pointer.net/blog/projects/stateless.html [2] http://www.freedesktop.org/software/systemd/man/os-release.html This has been reported by Tiago Gomes Change-Id: I58dd711fac162751a71e8a7d9c65ce9583a23cc7
* attr: separate bootstrap and configure commandsJavier Jardón2015-06-232-2/+2
| | | | | | Also attr doesnt depend on automake to build Change-Id: Ied3ba4af7991c44ed080362a8dae5d00092e5514
* More fixes for various deployment extensionsAdam Coldrick2015-06-237-6/+10
| | | | | | | | | Some of the extensions had extraneous imports, and a couple didn't import things they used. The same bug as in the kvm deployment extension existed in the virtualbox deployment extension, and is fixed in this commit. Change-Id: I2bd8169818ce5444af522c135a9e0326c1212ce2
* Fix kvm deploymentAdam Coldrick2015-06-222-4/+8
| | | | | | | | Deployment to kvm got broken by the patch to remove dependencies on morphlib and cliapp. This commit fixes the issues that were introduced. Change-Id: Iface1b38e638be4d84cf86b434ee30e1fffc34a0
* essential-files: Install brpaste by defaultPedro Alvarez2015-06-222-0/+31
| | | | | | This script helps developers to share logs via paste.baserock.org Change-Id: I22b255c90e0b42a4b65dea41ec196ae92577a7f8
* OpenStack: add configuration for scenario testsTiago Gomes2015-06-222-8/+17
| | | | Change-Id: Ifaaced5256ce7aeca786118a2075c991b3fa103f
* OpenStack: integrate the OpenStack unified CLI clientTiago Gomes2015-06-227-0/+54
| | | | | | | | | "OpenStackClient (aka OSC) is a command-line client for OpenStack that brings the command set for Compute, Identity, Image, Object Store and Volume APIs together in a single shell with a uniform command structure." Change-Id: Icf3200e99100f46e5d8044844232025e7bada75d
* Set default hostnames in release.morphAdam Coldrick2015-06-221-0/+7
| | | | Change-Id: Iffce1a4c25965ca4d7e635534fab16d0e8ccb39b
* Merge branch 'baserock/adamcoldrick/remove-dependencies-v3'Adam Coldrick2015-06-1928-350/+1420
|\ | | | | | | | | | | | | | | This stops the deployment extensions from depending on morphlib and cliapp, as well as setting the definitions version to 5. Reviewed-by: Sam Thursfield <sam.thursfield@codethink.co.uk> Reviewed-by: Paul Sherwood <paul.sherwood@codethink.co.uk>
| * Remove dependencies on morphlib and cliapp from deployment extensionsbaserock/adamcoldrick/remove-dependencies-v3Adam Coldrick2015-06-1126-406/+546
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is done by either copying some utility functions from morph into writeexts.py, and using the `subprocess` module rather than cliapp's runcmd and ssh_runcmd. Note that this means that these extensions will require "$definitions_checkout/extensions" in PYTHONPATH when they are run. This commit also updates VERSION to 5, since the PYTHONPATH requirement means that this change is incompatible with old versions of morph. Change-Id: Iec6fa7e3c7219619ce55e18493e5c37c36e97816
| * Move xfer-hole and recv-hole into extensions/baserock/adamcoldrick/writeexts-in-definitions-v2Adam Coldrick2015-06-102-0/+0
| | | | | | | | Change-Id: I46ed5f3ec85f9662bebac592eff7a6eb6d628f28
| * Put recv-hole and xfer-hole into definitionsAdam Coldrick2015-06-102-0/+295
| |\ | | | | | | | | | This merge commit adds the history of the two scripts as well.
| | * Use the modern way of the GPL copyright header: URL instead real addressJavier Jardón2015-03-162-6/+4
| | | | | | | | | | | | Change-Id: I992dc0c1d40f563ade56a833162d409b02be90a0
| | * xfer-hole: Fix bug in copy_slice_from_file.Lars Wirzenius2014-10-021-2/+8
| | | | | | | | | | | | | | | os.read is limited to an int in size. copy_slice_from_file was trying to os.read more than that causing an OverflowError.
| | * Use $((...))) instead of $(... | bc)Lars Wirzenius2014-09-051-4/+4
| | |
| | * Document why VBoxManage is run in recv-hole instead of callerLars Wirzenius2014-09-051-0/+25
| | |
| | * Transfer sparse files faster for kvm, vbox deploymentLars Wirzenius2014-09-052-0/+266
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The KVM and VirtualBox deployments use sparse files for raw disk images. This means they can store a large disk (say, tens or hundreds of gigabytes) without using more disk space than is required for the actual content (e.g., a gigabyte or so for the files in the root filesystem). The kernel and filesystem make the unwritten parts of the disk image look as if they are filled with zero bytes. This is good. However, during deployment those sparse files get transferred as if there really are a lot of zeroes. Those zeroes take a lot of time to transfer. rsync, for example, does not handle large holes efficiently. This change introduces a couple of helper tools (morphlib/xfer-hole and morphlib/recv-hole), which transfer the holes more efficiently. The xfer-hole program reads a file and outputs records like these: DATA 123 binary data (exaclyt 123 bytes and no newline at the end) HOLE 3245 xfer-hole can do this efficiently, without having to read through all the zeroes in the holes, using the SEEK_DATA and SEEK_HOLE arguments to lseek. Using this, the holes take only take a few bytes each, making it possible to transfer a disk image faster. In my benchmarks, transferring a 100G byte disk image took about 100 seconds for KVM, and 220 seconds for VirtualBox (which needs to more work at the receiver to convert the raw disk to a VDI). Both benchmarks were from a VM on my laptop to the laptop itself. The interesting bit here is that the receiver (recv-hole) is simple enough that it can be implemented in a bit of shell script, and the text of the shell script can be run on the remote end by giving it to ssh as a command line argument. This means there is no need to install any special tools on the receiver, which makes using this improvement much simpler.
| * Stop writeexts.py depending on morphlibAdam Coldrick2015-06-101-6/+17
| | | | | | | | Change-Id: I7f3702e80678aeee89dd22116510a6d8d7e04841
| * Put writeexts.py in extensions subdirectoryAdam Coldrick2015-06-101-0/+0
| | | | | | | | Change-Id: I6eb8fe69416bbf483ffa8c1c317c8f8cea56ef0e
| * Put writeexts.py from morphlib in definitionsAdam Coldrick2015-06-101-0/+624
| |\ | | | | | | | | | | | | | | | This is only used by write extensions, all of which are now in definitions. This merge commit also adds all the history of the writeexts.py file from morphlib.
View Lorry Controller Status