summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Improve help for '--local-changes' optionPedro Alvarez2016-07-271-3/+4
| | | | Change-Id: Ie905724b0e77a7f9b3a2ea1af348aa0f53ae26b8
* Add global '--ref' and '--repo' optionsPedro Alvarez2016-07-271-0/+12
| | | | | | | | These new options will be used by subcommands that use a morphology file to allow them to use remote files without having to clone the repository. Change-Id: I0f35068627b7f7ba766ac015f0e1debff94abc57
* Rename get-repo's option '--ref' to '--checkout-ref'Pedro Alvarez2016-07-261-1/+1
| | | | | | | The option '--ref' is going to be used for other things in the tool. Change-Id: Icbee89c4693ca53618b9c039120fbb0c6636285d
* Fix generate-manifest-csv options helpPedro Alvarez2016-07-261-1/+1
| | | | Change-Id: I3a87c82b0c50a648edd7c6487e0269116da35c20
* worker_build_scheduler: Consider active jobs when creating/cancellingPedro Alvarez2016-07-251-5/+11
| | | | | | | | | | | | | In WorkerBuildQueuer._handle_request we were only considering running jobs when creating a new one to not create duplicates. This was making `morph distbuild` build some components more than one time. In this commit we also start considering active jobs when cancelling them. Change-Id: Ib0a7296d453ccd0b8c636c7506d9f1da82acc462
* Make clear how long a command took to failPedro Alvarez2016-06-031-0/+2
| | | | | | | | | | | | | | This is done by adding an status update when a command fails. An example of the new output is: 2016-06-03 10:35:04 [Build 5/133] [stage2-libstdc++] Creating staging area 2016-06-03 10:35:04 [Build 5/133] [stage2-libstdc++] Starting actual build: stage2-libstdc++ fc4868e 2016-06-03 10:35:04 [Build 5/133] [stage2-libstdc++] Extracting /src/gcc-tarball into /src/tmp/staging/tmpE5b84y/stage2-libstdc++.build 2016-06-03 10:35:38 [Build 5/133] [stage2-libstdc++] Running configure-commands 2016-06-03 10:35:55 [Build 5/133] [stage2-libstdc++] Running build-commands 2016-06-03 10:36:14 [Build 5/133] [stage2-libstdc++] Error when running build-commands Change-Id: Iac94794311e649d13c534f453d3473ac8003e8f4
* Make `morph list-artifacts` work from definitions checkoutPedro Alvarez2016-05-173-10/+39
| | | | Change-Id: I711953b829c786911c46e413d2b7af3427b7ba26
* Make `morph cross-bootstrap` work from definitions checkoutPedro Alvarez2016-05-173-31/+39
| | | | Change-Id: I9ae5a846e20c0d32047a3d4fd5892e2f632f0cbe
* build: Give a helpful error when the file to be built is not foundSam Thursfield2016-04-271-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recently something regressed which meant that when building a missing file, the result was this long unhelpful traceback: 2016-04-27 12:23:36 Deciding on task order Traceback (most recent call last): ... File "/src/morph/morphlib/sourceresolver.py", line 506, in create_source_pool definitions_original_ref=original_ref) File "/src/morph/morphlib/sourceresolver.py", line 440, in add_morphs_to_source_pool add_to_pool, predefined_split_rules) File "/src/morph/morphlib/sourceresolver.py", line 287, in _process_definitions_with_children morphology = get_morphology(filename) File "/src/morph/morphlib/sourceresolver.py", line 282, in get_morphology filename) File "/src/morph/morphlib/sourceresolver.py", line 255, in _get_morphology morph = morph_loader.load_from_string(text, filename) File "/src/morph/morphlib/morphloader.py", line 395, in load_from_string obj = yaml.safe_load(string) ... File "/usr/lib/python2.7/site-packages/yaml/reader.py", line 178, in update_raw data = self.stream.read(size) AttributeError: 'NoneType' object has no attribute 'read' Now the error is: ERROR: Couldn't find definition file to build: missing-file.morph Change-Id: Iabb315c9d0ac5c7c50003db36a1b21e86fb23223
* Honour --quietRichard Ipsum2016-04-251-2/+3
| | | | Change-Id: Ifec7f4274079d9c8f2a75bdedc8a253568a6ec78
* yarns: Stop git server when we use it.Pedro Alvarez2016-04-183-0/+8
| | | | | | Fixes https://storyboard.baserock.org/#!/story/83 Change-Id: Iae3f99b4189f513fa9b02d904c4fcdc0f52bbfb6
* Make `morph get-repo` use submodules from definitionsPedro Alvarez2016-04-123-17/+69
| | | | Change-Id: I8d9330eb5f72a8f3f106aae4387eb8321f6126e5
* morphlib/git: Fix update_submodulesPedro Alvarez2016-04-061-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It seems to be broken since d58d8e8f7a4ec03ff14021a4515c8283dad52573 and it's making `morph get-repo` fail with: /src/definitions # morph get-repo ansible 2016-04-06 11:36:31 Loading in all morphologies 2016-04-06 11:36:32 Checking out ref 0a7124541247cc613352054f4bc0c3e116e0b657 of ansible in ansible stratum Traceback (most recent call last): File "cliapp/app.py", line 190, in _run self.process_args(args) File "/src/morph/morphlib/app.py", line 299, in process_args cliapp.Application.process_args(self, args) File "cliapp/app.py", line 539, in process_args method(args[1:]) File "/src/morph/morphlib/plugins/get_repo_plugin.py", line 142, in get_repo chunk_dirname = checkout_chunk(morph, chunk) File "/src/morph/morphlib/plugins/get_repo_plugin.py", line 110, in checkout_chunk ref or chunk_spec['ref']) File "/src/morph/morphlib/plugins/get_repo_plugin.py", line 60, in _clone_repo gd.update_submodules(self.app) File "/src/morph/morphlib/gitdir.py", line 527, in update_submodules morphlib.git.update_submodules(app, self.dirname) File "/src/morph/morphlib/git.py", line 150, in update_submodules submodules.load() File "/src/morph/morphlib/git.py", line 69, in load content = self._read_gitmodules_file() File "/src/morph/morphlib/git.py", line 82, in _read_gitmodules_file ignore_fail=True) File "/src/morph/morphlib/git.py", line 306, in gitcmd return runcmd(cmdline, **kwargs) TypeError: 'str' object is not callable Change-Id: I8d08eb71a2b0a773bc4322ed46222bea0adbbee1
* Convert the output of unpushed branch check to lower caseBen Brown2016-04-041-3/+3
| | | | | | | | | | In commit 3a429d0 of git, the casing of certain error messages was changed, convert the message to lower case before comparison. ERROR: Command failed: git rev-parse --abbrev-ref benbrown/foo@{upstream} fatal: no upstream configured for branch 'benbrown/foo' Change-Id: I5e0659b69f3be84820592b7fe26e50203272da22
* morphlib/gitdir: Recover CheckoutErrorPedro Alvarez2016-04-041-0/+8
| | | | | | It was removed by mistake in 014a029ade9a045a839ca86c35690b218098ea33 Change-Id: I766f39f09444a66234ce3bdcb9134e592d5363ce
* morphlib/git: Ignore not valid submodulesPedro Alvarez2016-04-041-1/+8
| | | | | | | | | | | | Some repositories may have submodules listed in .gitmodules that are not pointing to a commit object in the tree. This was causing the following error: ERROR: /srv/distbuild/gits/git___git_foo_bar_git:c36652c7dd9f1687c87ce04ad3d4af2beb52dc25:.gitmodules: No commit object found for submodule "codegen" Instead of raising that error, capture it and ignore the submodule. Change-Id: Iba0428feea7605f797dc78e3e250f3d66c003670
* README: Add Baserock-independent build instructionsEdward Cragg2016-04-011-1/+45
| | | | | | | | Update README with details on how to run morph independently from Baserock. https://storyboard.baserock.org/#!/story/88 Change-Id: Ic1cb58dddb43599017b41fa1126a49edb243a89a
* Fix issues running tests outside a Baserock systemSam Thursfield2016-03-312-8/+16
| | | | | | | | Warn if 'Yarn' isn't available, since this means most tests don't run. Also fix some issue with missing .coverage file that I hit. Change-Id: I675354799bbdefe1ced0f5a0c5221b103de29a1c
* Add local cliapp moduleEdward Cragg2016-03-3111-2/+2459
| | | | | | | | | | | | | | Add a copy of the specific version of the cliapp module required by morph, a known version of which can't otherwise be guaranteed to be available on general Linux systems, or via PyPi. Including the module directly allows for continued development in line with morph, until a time when it may become desirable to move away from dependence on this module. With a local copy of cliapp added to morph, it has been shown that morph is capable of building Baserock images without depending on an existing Baserock system. Change-Id: Iab1f7037c7afff054a2404c0552d9b5e4d2d141f
* Add support for definitions version 8Pedro Alvarez2016-03-2918-149/+350
| | | | | | | | | This code is a rework from changes done by: - Tiago Gomes <tiago.gomes@codethink.co.uk> https://storyboard.baserock.org/#!/story/86 Change-Id: I3475c2bcb648a272fee33bc878a521f79d4e6581
* Remove support for definitions version 6Tiago Gomes2016-03-2510-354/+18
| | | | Change-Id: I603b5172902b5249874ea3bb90de59db5a908185
* Adapt integration test to definitions version 7Tiago Gomes2016-03-253-1683/+303
| | | | | | | | With version 7, less dependencies are shown as the DEFAULTS file used for testing has a reduced set of artifact splitting rules, compared with the built-in split rules. Change-Id: I1e9d5390126bad007d39729faabf83cfb60a5092
* morphloader: simplify APITiago Gomes2016-03-253-41/+31
| | | | | | | Use an optional parameter to control whether the defaults should be set instead of a separate function. Change-Id: Idbbd5a08e8b16d8e01bb9539274092978b64f6f0
* Unify local and remote repo cache modulesSam Thursfield2016-03-1623-957/+976
| | | | | | | | | | | | | | | | | | | | | There's not really any reason you'd want to use the RemoteRepoCache class except as a workaround for the slow speed of some LocalRepoCache operations, so I can't see this ruining anyone's day. The main reason for doing this is so we can simply the sourceresolver code. One reason that the sourceresolver class is so hopelessly complicated is that it right now has to use two incompatible interfaces for Git repo caches. I've taken the opportunity to detangle the RepoCache class from the App class. Now all of the configuration for the RepoCache class is passed into the constructor explicitly. This makes the class usable from outside Morph: resolver = morphlib.repoaliasresolver.RepoAliasResolver(aliases=[]) repo_cache = morphlib.repocache.RepoCache('/src/cache/gits', resolver) Change-Id: I596c81d7645b67504c88e555172a8c238f4f8a66
* Get rid of the CachedRepo class (almost)Sam Thursfield2016-03-0316-586/+154
| | | | | | | | | | | | | | | | | | For a long time the CachedRepo class has basically been a wrapper around the GitDir class, but with a few extra methods that don't really even belong there. It is now a tiny class in the localrepocache module which just keeps track of a few extra attributes. All other functionality is provided by the gitdir module. This commit also removes the `git clone` approach for copying repos out of the cache. The alternative approach implemented by git.copy_repository() was slightly faster when I tested, so for now we should use that everywhere. Longer term we should find out why this is quicker than `git clone`, and fix Git itself to be fast. Change-Id: I1686ab43253d44c3903d9a0bad8bb75528e9cf75
* Hide get_repo() and cache_repo() functions, always use get_updated_repo()Sam Thursfield2016-03-038-73/+33
| | | | | | | | | | This allows us to simplify a couple of places. I cannot think of a single situation where'd you want to get the cached copy of a repo, but not update it. Think about it -- the repo might be *years* behind the upstream remote. Change-Id: I60340c7fb33e7bfe871ad30c0a9322a7202548e2
* Fix some test failuresSam Thursfield2016-03-032-6/+9
| | | | | | These seem to have been introduced by commit fb6dd955cbc7081f47dc4391. Change-Id: I2fb0be5f8829722264f3509f4db9473d19ea2939
* Fix copyright year of morphlib/morphloader.pybaserock/pedroalvarez/wip-gearPedro Alvarez2016-02-271-1/+1
| | | | | | File was modified in 472dbbbbfa07e67c02a98e9f4ee50fc86ad32b7e Change-Id: If0ffa0aae76ff9f0093e5eb7ff7d74d66c26119d
* Error when different names in system/strataPedro Alvarez2016-02-251-6/+25
| | | | | | | | | | | | | | Now is an error when the name of a stratum in a system is different from the name of the stratum in the stratum file. Example of the error message: ERROR: Name 'build-essential-foo' doesn't match 'build-essential' in morphology: strata/build-essential.morph We are forcing the error here instead of a warning because this problem makes Morph create broken systems. https://storyboard.baserock.org/#!/story/15 Change-Id: Ifbe8d98e64a768e4e238d2213ffaf575789dd9e2
* Warning when different names in strata/chunksPedro Alvarez2016-02-252-1/+5
| | | | | | | | | Now is an warning when the name of a chunk in a stratum is different from the name of the chunk in the chunk file. Example of the warning message: WARNING: Name 'binutils' doesn't match 'binutils-foo' in morphology: strata/build-essential/binutils.morph Change-Id: I38f0ccff57fc25db12c572fdc8a01dbd6434f01c
* Validate empty configure/build/test/install commandsEdward Cragg2016-02-251-0/+13
| | | | | | | | | | | | | If a command in a chunk is evaluated to None when a morphology is loaded, this would previously lead to the subprocess module throwing a 'Coercing to Unicode' error when morph attempts to run the command during the chunk build. This change throws a more helpful error if any commands are not valid strings when the chunk is validated, for example: ERROR: Field configure-commands[217] expected type <type 'str'>, got <type 'NoneType'> in morphology linux-jetson-tk1 Change-Id: I45220e7deab8c7cd9351507bc998f7ff12797442
* Avoid stack overflow in cachekeycomputerSam Thursfield2016-02-241-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I managed to trigger a stack overflow in Morph, by having two chunks with the same name and same build instructions in different strata (by mistake, obviously). The error before was this: .... File "/src/morph/morphlib/cachekeycomputer.py", line 77, in get_cache_id cacheid = self._calculate(source) File "/src/morph/morphlib/cachekeycomputer.py", line 86, in _calculate for a in source.dependencies], File "/src/morph/morphlib/cachekeycomputer.py", line 38, in compute_key ret = self._hash_id(self.get_cache_id(source)) File "/src/morph/morphlib/cachekeycomputer.py", line 77, in get_cache_id cacheid = self._calculate(source) File "/src/morph/morphlib/cachekeycomputer.py", line 83, in _calculate 'env': self._filterenv(self._build_env.env), File "/src/morph/morphlib/cachekeycomputer.py", line 32, in _filterenv return dict([(k, env[k]) for k in keys]) RuntimeError: maximum recursion depth exceeded while calling a Python object The error afterwards is this: 2016-02-18 13:53:14 Deciding on task order ERROR: There are multiple versions of component 'cython' Change-Id: I96a41cabe4d9aa81bddc5186c5248a636a0843c2
* Avoid UnicodeDecodeError when writing to log filesPedro Alvarez2016-02-231-5/+11
| | | | | | | This is the counterpart fix to b3ecd02236e58386ac4d7566ef70e751ff0d7e26, which had broken the Morph test suite, as it turns out. Change-Id: I5392c2c762c733d7d88cd20898970ec314525d89
* deploy: Fix an unlikely crash at deploy-timeSam Thursfield2016-02-191-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For some reason I had a deployment .morph file that specified: deploy-defaults: This resulted in the deploy_defaults variable having value None, which lead to a crash further down: 2016-02-19 12:03:52 Deciding on task order Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 190, in _run self.process_args(args) File "/src/morph/morphlib/app.py", line 299, in process_args cliapp.Application.process_args(self, args) File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 539, in process_args method(args[1:]) File "/src/morph/morphlib/plugins/deploy_plugin.py", line 574, in upgrade self.deploy(args) File "/src/morph/morphlib/plugins/deploy_plugin.py", line 370, in deploy env_vars, deployments) File "/src/morph/morphlib/plugins/deploy_plugin.py", line 403, in deploy_cluster parent_location='') File "/src/morph/morphlib/plugins/deploy_plugin.py", line 462, in deploy_system env_vars, deployment_filter, parent_location) File "/src/morph/morphlib/plugins/deploy_plugin.py", line 483, in deploy_system_with_source_pool system_id, env_vars, deploy_defaults, deploy_params) File "/src/morph/morphlib/plugins/deploy_plugin.py", line 59, in configuration_for_system deploy_params.items() + AttributeError: 'NoneType' object has no attribute 'items' This change makes the code work as expected. Change-Id: I667f4142667be31797cac7c8994d35a404119cca
* sourceresolver: Use real name for sources without chunk filePedro Alvarez2016-02-162-1647/+1648
| | | | | | | | | | | | | | | | | | The filename was being used instead. I noticed this bug after seeing cache artifacts named like: de31c7bcac0240315bc4e0994b34f3757d37d35e4179cdbe0a8b2e60e26449f8.chunk.xorg-proto-x11proto.morph-bins de7b9f6f7a10b8dc04b52a08e74987df7da310b59dc25c6cf083b0381726e15f.chunk.networkx.morph-bins def910ed756be94c626d59d73f0770a445d62278a2eb46e031af42563691c6ed.chunk.pv.morph-bins e003da7f8c697286fa4d550299920160a6e76df449b7e434b270cd760c0c9304.chunk.cython.morph-bins e193576471c957387c9684c385a8c03db1e70618890210f6272c362ccc3e259b.chunk.xorg-proto-xcmiscproto.morph-bins e3b0b0dbb7b3de4bb8c72cd064408956296e290fc4b56acbbee241e2d78bf533.chunk.greenlet.morph-bins e3f09aa095e5839aa9bf7774720951c8b03d78300b056f7bbdeacdf4c5d5944d.chunk.libvorbis.morph-bins e437810247200713da17937181cc914e080ac7d6030451ccfaa86ddccf7a395b.chunk.python-requests.morph-bins e4b7de417c5691794f4ad6a6481cacc16242af072db39bcadede6ca2bf48241e.chunk.flac.morph-bins e4f1e6dbaeed869b445f9b45b98033b6e6a2952674b76b8465e40a67e675333c.chunk.Sub-Name.morph-bins Change-Id: I8a8697a12cf7e4c7891c93e4f774819fe2f8ea86
* Fix unused 'bs' var in builderRichard Ipsum2016-02-151-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's no longer any need to look up the buildsystem in builder, since the morphloader now sets commands directly in the morphology. Removing this unused variable also fixes a bug where morph can crash if the chunk being built is being built with a buildsystem that is not hardcoded in morph, error below: 2016-02-15 15:29:46 Starting build of devel-system-x86_64-generic 2016-02-15 15:29:46 [Build 70/407] [cython.morph] Building chunk cython.morph 2016-02-15 15:29:46 [Build 70/407] [cython.morph] Creating staging area 2016-02-15 15:29:47 [Build 70/407] [cython.morph] Starting actual build: cython.morph 4dd8e76 2016-02-15 15:29:47 [Build 70/407] [cython.morph] Extracting upstream:cython into /src/tmp/staging/tmpLI9TUj/cython.morph.build Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 190, in _run self.process_args(args) File "/src/morph/morphlib/app.py", line 299, in process_args cliapp.Application.process_args(self, args) File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 539, in process_args method(args[1:]) File "/src/morph/morphlib/plugins/build_plugin.py", line 208, in build self._build(source_pool, filename, component_names=component_names) File "/src/morph/morphlib/plugins/build_plugin.py", line 237, in _build bc.build_in_order(component) File "/src/morph/morphlib/buildcommand.py", line 297, in build_in_order self.cache_or_build_source(s, build_env) File "/src/morph/morphlib/buildcommand.py", line 317, in cache_or_build_source self.build_source(source, build_env) File "/src/morph/morphlib/buildcommand.py", line 374, in build_source self.build_and_cache(staging_area, source, setup_mounts) File "/src/morph/morphlib/buildcommand.py", line 545, in build_and_cache return builder.build_and_cache(source) File "/src/morph/morphlib/builder.py", line 748, in build_and_cache built_artifacts = o.build_and_cache() File "/src/morph/morphlib/builder.py", line 288, in build_and_cache self.run_commands(temppath, stdout) File "/src/morph/morphlib/builder.py", line 314, in run_commands bs = morphlib.buildsystem.lookup_build_system(m['build-system']) File "/src/morph/morphlib/buildsystem.py", line 300, in lookup_build_system raise KeyError('Unknown build system: %s' % name) KeyError: 'Unknown build system: python3-distutils' Change-Id: Iddf651b408c612c7c80296012cf1c97ab99e8ccc
* check-copyright-year: Check author date instead of commiter datePedro Alvarez2016-02-131-2/+2
| | | | | | | Now if a patch is sent to review, on year 20xx, and then merged on year 20xx+1, the script will check the former instead of the latter. Change-Id: I5bb9ac05f1fb3fa4a51a051351e754c524897ea1
* Fix breakage when new build systems are added to DEFAULTSSam Thursfield2016-02-111-4/+8
| | | | | | | | | | | | | It turns out that one part of Morph was still using the predefined build systems that were built into the 'morphlib.buildsystem' Python module, instead of using the ones from the DEFAULTS file. This meant that Morph wasn't actually implementing Baserock definitions version 7 properly. https://storyboard.baserock.org/#!/story/72 Change-Id: I440a8f7455874a1b8491b48b0b122792aa8189ae
* More robust creation of tempdirsPedro Alvarez2016-02-051-2/+9
| | | | | | | | | | | | | | | | | | | This commit addresses the same issue that we tried to solve in fa431c3ce38204f5237337fd2045ad6269413258 but this time with temp dirs: + echo INFO: Mason building: master at fe64c00f17a4922c7499f9bc48f672cca293fc65 INFO: Mason building: master at fe64c00f17a4922c7499f9bc48f672cca293fc65 + scripts/release-build --no-default-configs --trove-host 192.168.222.58 --artifact-cache-server http://192.168.222.14:8080/ --controllers x86_64:mason-x86-64 clusters/ci.morph ERROR: /srv/distbuild/tmp/chunks: File exists ERROR: /srv/distbuild/tmp/chunks: File exists This has been happening in Mason since we remove every folder used by Morph after Mason finishes. Change-Id: I5a5bfd3c01b6fd0da4b913c5bb8eac77a3233d1e
* distbuild: Avoid UnicodeEncodeError when writing build output to log filesSam Thursfield2016-01-261-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Text encoding in Python 2 is a total mess so I can only pretend to understand what's going on. The 'stdout' and 'stderr' messages are Python 'unicode' instances, which isn't really important, but I know that because when we try to write them the log file, and they contain non-ASCII data, we see this error: File "/usr/lib/python2.7/site-packages/distbuild/initiator.py", line 231, in _handle_step_output_message f.write(msg['stdout']) UnicodeEncodeError: 'ascii' codec can't encode character u'\u2022' in position 29: ordinal not in range(128) Who said anything about encoding 'unicode' to 'ascii'? It turns out that when you write to a file, Python implicitly tries to encode the data to the 'default encoding', which happens to be 'ascii'. You lose! The only way to fix this is to tell Python that the file has a different encoding, a nice one like UTF-8. (I tried opening the file with 'b' mode, that doesn't seem to help). UTF-8 can only encode valid Unicode data, of course, so we need to make sure the data we write is valid UTF-8. You can to this by calling decode('unicode-escape'), which converts *from* Unicode *to* Unicode, but replacing any invalid characters with escape codes so that we don't get any UnicodeDecodeErrors during the conversion, or when we try to write it to the UTF-8 file. See this presentation for more info: http://farmdev.com/talks/unicode/ Or just use Python 3. Change-Id: I6316d346f5cca2c75f198b48ec9878ac647ae7e5
* More robust creation of cache dirsPedro Alvarez2016-01-221-3/+18
| | | | | | | | | | | | | | | | | | When starting various distbuilds at the same time we were having randomly errors like: 2016-01-06 12:14:03 Starting distributed build 2016-01-06 12:14:03 Connecting to mason-x86-64:7878 2016-01-06 12:14:03 Requesting build of git://192.168.222.58/baserock/baserock/definitions 28b92192c00a36395acd6a960959d3b4468f9894 systems/openstack-system-x86_64.morph ERROR: /srv/distbuild/artifacts: File exists ERROR: /srv/distbuild/artifacts: File exists ERROR: /srv/distbuild/artifacts: File exists ERROR: /srv/distbuild/artifacts: File exists This has been happening in Mason since we remove every folder used by Morph after Mason finishes. Change-Id: I175de7282302d9c1e2fb1b0872f7eb72c742f28e
* Don't show progress bars if stderr is being redirected to a fileSam Thursfield2015-12-211-7/+8
| | | | | | | | | | | | | | | | | | Currently the logs of our continuous builder (https://mason-x86-64.baserock.org) are filled with lots of this sort of crap: stage1-binutils-misc[ ] 2.4/73.0 MB stage1-binutils-misc[ ] 2.4/73.0 MB stage1-binutils-misc[ ] 2.4/73.0 MB stage1-binutils-misc[ ] 2.4/73.0 MB stage1-binutils-misc[# ] 2.4/73.0 MB stage1-binutils-misc[# ] 2.5/73.0 MB stage1-binutils-misc[# ] 2.5/73.0 MB ... Change-Id: I0b7a4c8421c3ecdd141fc8323d3001ae4fb44d9b
* morphloader: remove aliasTiago Gomes2015-12-211-6/+5
| | | | | | | The field 'alias' is not defined in the schema for definitions version 7. Change-Id: Id595562258ebe4c3f228cf3f04456e7f69ce63ae
* morphloader: remove unset functionsTiago Gomes2015-12-212-150/+0
| | | | Change-Id: I2a9177245e8cd4eac54ef8a2079eaed2e98e88e7
* morphloader: remove arch normalizationTiago Gomes2015-12-212-16/+0
| | | | | | | It is a non-documented secret sauce and having to type an extra letter for the ARMv7 little-endian architecture does not justify it. Change-Id: I733d1b5a5a5a53ed4dd9815d16b946f9b0a980ec
* morphloader: remove obsolete fieldsTiago Gomes2015-12-212-160/+2
| | | | Change-Id: If7bf7af93eb404be48f752ed64e8b038a3073668
* Remove non-existing files from without-test-modulesTiago Gomes2015-12-151-6/+0
| | | | Change-Id: I0399dbfe3f9f71ca9895fef435bb604edb224928
* Cleanup buildbranch.py moduleTiago Gomes2015-12-112-178/+139
| | | | | | | | | | | | | | | | Now that we don't support system branches, we don't need to iterate over a list of repos to create temporary build branches, commit their local changes and push those branches. We only need to do this for the definitions repository itself. A bug is also fixed where the local build branch was not being deleted due a missing call to _register_cleanup() when commiting the changes. This commit also renames some functions to more clear names, and moves the logic on pushed_build_branch() to the definitions_repo module, the only place where it is used. Change-Id: Id86240d0c189245bed36bc46355be13d46498dbc
* Remove (dist)?build-morphology commandsTiago Gomes2015-12-021-79/+0
| | | | Change-Id: Ifb2bc2e2084806fe70b6db96828a390d3082288f
* Convert last cmdtests to yarnsTiago Gomes2015-12-0235-2801/+2253
| | | | | | | The conversion was made so that the build command was used instead of the build-morphology command. Change-Id: I67c43d765ed603ecdd806bc649815526243b7b87
View Lorry Controller Status