summaryrefslogtreecommitdiff
path: root/morphlib/util.py
Commit message (Collapse)AuthorAgeFilesLines
* Use exact filenames to refer to morphology filesRichard Maw2014-07-101-7/+16
| | | | | | | | | | | | | | | | | | | Rather than repeatedly stripping and appending an optional .morph extension morphology names, instead always use the file path of the morphology relative to the definitions repository. This is an inversion of the previous logic, which would strip the .morph extension and use the "name" internally. The exception to this rule of always using the filename, is that `morph edit CHUNK` uses the name of the morphology as-defined in the stratum. This is based off Adam Coldrick's inital patch, but this version will allow the old style of providing the "name" by converting it into a path if it does not have either a / or a . in it. An unfortunate consequence of this change is that the show-dependencies command's output changed, so the test needed updating.
* Add armv7lhf detectionRichard Ipsum2014-01-211-1/+22
|
* LocalArtifactCache now takes a an FS objectDaniel Firth2013-12-201-3/+5
|
* Modify morph to strip .morph extensions from parametersBen Brown2013-12-161-0/+9
|
* sysbranchdir: Move load_all_morphologies helper hereRichard Maw2013-11-291-1/+0
| | | | | | | | | | | | | | | This was previously a private method of the branch and merge plugin, but it's useful to other plugins, so has been moved to the SystemBranchDirectory class, where everything else can get to it. It has an unpleasant amount of coupling to other classes, but in a *good* object oriented design it would either be a tiny module on its own, or not exist and leave all its users to re-implement the same logic multiple ways, so we've opted for a less clean, but more useful design. It is left un-covered by the unit tests, since it requires a great deal of instrumentation to test, at which point it may be best to leave it to integration tests.
* Make morph able to use ppc64 architecture (POWER PC 64 bits)Pedro Alvarez2013-11-271-0/+1
|
* util: Add helper for splitting up huge iterablesRichard Maw2013-11-221-0/+10
| | | | | | | | This is used to split a large iterable into more manageable chunks. This is important when a consumer can only handle so much input at once e.g. A program can take a large, but finite number of arguments, xargs exists to handle this for shell programs.
* Add --artifact-cache-server and --git-resolve-cache-serverLars Wirzenius2013-10-291-2/+18
|
* Tidy up debug output of morph buildDaniel Firth2013-10-281-6/+5
|
* Fix app.status call in log_dict_diffLars Wirzenius2013-10-241-2/+4
| | | | | | | | | | | | | | | | This bug was triggered by the fix to CachedRepo.resolve_ref and without this fix, the resolve_ref fix will break the test suite. The bug is that log_dict_diff calls the status method with an msg keyword argument that may contain percentage characters. status interprets the value of msg as a format string, and the percentage characters trigger formatting to happen. The fix for that is to not interpolate the value of key and dictA[key] and dictB[key] into msg before calling status, but letting status do that. Thus the msg values are changed to reference %(key)s instead and passing in a value for key as a separate argument. Ditto for dictA[key] and dictB[key].
* Reduced debug output for environment propertiesDan Firth2013-10-141-1/+2
|
* Added chunkname prefix to some elements of the debug log.Dan Firth2013-10-111-4/+4
|
* util: add find_leaves() supplimenting find_leaf()Richard Maw2013-09-171-13/+22
| | | | | | | | | | | | | | | The workspace needs to be able to list all its system branches, and the system branches need to be able to list all their git repositories. This is broadly the same thing, just with a different directory to look out for, so provide that utility in morphlib.util. find_leaf() is rewritten to use find_leaves(), this is less efficient since it waits until every leaf is found. I felt it was better to reduce the code than maintain a slightly more optimal algorithm. The find_leaf() algorithm could become more optimal if it could lazily check for at least one result in a generator.
* Utility function to convert all values in a dictionary to stringsTiago Gomes2013-08-161-1/+5
| | | | | | | | | This will be useful in the next commit, where we want to construct an environment to run a command from a dictionary. The cliapp runcmd method expects that the values in this dictionary are strings, so we need to convert them before.
* Add "morph print-architecture" subcommandLars Wirzenius2013-08-151-0/+22
|
* deploy: refactor environment argument parsingRichard Maw2013-08-131-0/+32
| | | | It is now a tested helper function in morphlib.util
* Add morphlib.util.find_root and find_leaf functionsLars Wirzenius2013-08-061-6/+53
| | | | | | | These will be used to find workspace and system branch root directories. Also accidentally removed some whitespace from ends of lines. Too lazy to split that into a separate commit.
* Merge remote-tracking branch ↵Daniel Silverstone2013-06-191-0/+4
|\ | | | | | | | | | | 'origin/baserock/richarddale/check-repo-alias-option' Reviewed-By: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
| * Check that that the repo-alias option matches the expected RERichard Dale2013-06-171-0/+4
| |
* | util: suggest how to deal with insufficient spaceRichard Maw2013-06-071-1/+7
| |
* | util: split out space unify logicRichard Maw2013-06-071-6/+19
|/
* S7904: method for checking disk space is availableRichard Maw2013-06-051-0/+21
|
* S7904: method for testing paths are on the same fsRichard Maw2013-06-051-0/+10
|
* S7904: add method for finding bytes free in pathRichard Maw2013-06-051-0/+7
| | | | | This has no tests, as the ability to run them requires root privilidges or a more recent kernel and user namespace support.
* util: Fix a missing import in combine_aliasesRichard Maw2013-06-041-1/+1
| | | | ciapp is not imported, but morphlib.Error is an appropriate exception
* UTIL: Update trove-id processing for hyphensDaniel Silverstone2013-06-031-2/+2
| | | | | Ensure that trove-id meets the same requirements as repo aliases in terms of naming.
* rename trove-prefix and trove_prefix to trove_idPaul Sherwood2013-04-131-9/+9
|
* One 'arch' to rule them allSam Thursfield2013-03-151-15/+0
| | | | | | | | | | | | | Define a specific set of 4 architectures that Morph supports, and only expose that value to morphologies. Since GNU triplets are very common we also expose a GNU triplet. Other morphologies should work out their configuration based on MORPH_ARCH. This commit also removes the morphlib.util.arch() function, which detected the machine Morph is running on via 'uname -m'. Morph's architecture names do not necessarily map to the output of 'uname -m' so we should not rely on it anywhere.
* Set environment variables defining target for build-essentialSam Thursfield2013-03-131-0/+11
| | | | | | In the future we will allow this to be modified to provide a cross-bootstrap mode, but for now we use the same target as the host compiler.
* Refactor build processSam Thursfield2013-03-131-1/+0
| | | | | | | | | | | | | | | | | | Reorganise the build_artifact() and build_artifacts() functions to allow more complex work when setting up chunk builds in build_artifact(). The staging area now holds the BuildEnvironment object (the environment variables that should be set during build). This makes sense because all build commands should be run inside the staging area and therefore through the StagingArea object. The BuildEnvironment object is now considered immutable after it is created. The environment is used in cache key computation when computing what artifacts are required; if it changes after that point we risk either computing different artifact keys for the same artifact or missing data in the cache key that should be included in the hash. Better to force changes into a separate 'extra_env' variable.
* Merge remote-tracking branch ↵Lars Wirzenius2013-03-111-0/+17
|\ | | | | | | | | | | 'origin/baserock/richardholland/environment-logging' into staging Added a "pragma: no cover" to make the test suite pass.
| * Change morph to reduce loogign of the environmentRic Holland2013-03-081-0/+17
|/ | | | | | | | Added function log_dict_diff to identify and record changes in dicts to the debug log This new function was then implemented in app.py to log changes in the environment
* Tidy up create_cachedir()Sam Thursfield2013-02-061-13/+10
|
* Move OrderedDict and json loading to morphlib.utilRichard Maw2013-01-221-1/+11
| | | | | | | | This way everyone who needs json or an OrderedDict doesn't have to worry about it. Currently morph2 is the only one that needs this complex behaviour, other users of the json module don't need object_pairs_hook.
* Add a sparse-aware (for writing) copyfileobj.Daniel Silverstone2012-10-091-0/+39
|
* Use tarballs instead of bundlesDaniel Silverstone2012-09-251-2/+2
| | | | | | | This makes a non-backward-compatible change to morph which switches it to using tarballs instead of bundles when initialising cached git repositories. This is faster because it doesn't require index-pack --fix-thin operations on the machine running morph.
* Add support for Trove hosted content to morph configuration.Daniel Silverstone2012-09-121-0/+45
| | | | | | | | | | | Trove has a fixed structure for repositories and prefixes. By taking advantage of this, we can simplify the repo-alias configuration in morph somewhat and thus make it easier for others to add their own aliases and prefixes later, without worrying that they'll miss out on default ones created by the Baserock team. We override process_args in the application class to achieve this because cliapp's setup() hook is not quite right for our purposes.
* Move utility functions for creating cachesRichard Maw2012-08-011-0/+56
| | | | | | | | | | | Code for creating caches is repeated through the codebase because there wasn't a useful function for creating caches. Now the functions in BuildCommand have been moved into morphlib.util, where they will be more usefully reachable. Stubs have been left in BuildCommand so they can be overridden by subclasses of BuildCommand and not break external plugins.
* python scripts: pep8ize codebaseRichard Maw2012-08-011-13/+10
| | | | | | | | | This was done with the aid of the pep8 script, available by running `easy_install pep8`. It may be worth making this part of ./check, but that will require putting pep8 into the development tools stratum. This should be easy, given pep8 has no external dependencies.
* Bump copyright years, get rid of long lines and unused imports.Jannis Pohlmann2012-01-231-1/+1
|
* Fix coverageLars Wirzenius2011-11-151-1/+1
|
* Fix reference to cpu_count in actual codeLars Wirzenius2011-11-151-1/+1
|
* Handle non-functional multiprocessingLars Wirzenius2011-11-151-1/+4
| | | | This can happen when we run morph in a chroot without /dev/urandom
* Use make_concurrency to set MAKEFLAGS when buidling.Lars Wirzenius2011-10-191-0/+1
|
* Add method for deciding how many concurrent make jobs to run.Lars Wirzenius2011-10-191-0/+15
|
* Move Builder.arch into util.arch.Lars Wirzenius2011-10-171-0/+9
| | | | This makes it easy to add a test case.
* Handle unicode in indented error messages.Lars Wirzenius2011-10-081-2/+7
|
* Fix GPL version sentence grammar.Lars Wirzenius2011-10-061-1/+1
|
* Initial import.Lars Wirzenius2011-09-291-0/+32