summaryrefslogtreecommitdiff
path: root/morphlib/bins.py
Commit message (Collapse)AuthorAgeFilesLines
* Use the modern way of the GPL copyright header: URL instead real addressJavier Jardón2015-03-161-3/+2
| | | | Change-Id: I992dc0c1d40f563ade56a833162d409b02be90a0
* Fix tarfile.chown for versions of python smaller than 2.7.3baserock/franred/bugfix/python-tarfile-chownFrancisco Redondo Marchena2014-09-121-9/+38
| | | | | | | | This workaround fix http://bugs.python.org/issue12841 The code added in this patch is from tarfile.py from python 2.7.3. Also only use the workaround for tarfile.makefile when the ptyhon is smaller than 2.7.4
* Split chunk morphologies according to new rulesRichard Maw2014-01-161-62/+8
| | | | | | | | | | | | | | | | | | Filenames are now matched before chunks are constructed, so bins.create_chunk now takes a list of relative file names. bins.chunk_contents is gone, since this is now handled by passing source.split_rules.partition the file names. We now don't consider it to be a problem for directories to remain in the DESTDIR after artifacts have been removed, since we need to handle file matches implying their parent directories, and explicit matches against directories. NOTE: The bins_tests were broken in this patch, and are fixed in the next. This was done to try and aid readability of the patch series. Full functionality is still broken until stratum splitting is fixed.
* Make morphlib.bins.chunk_filenames() a private methodRichard Dale2013-05-241-3/+3
|
* Remove excessive debugging messagesRichard Dale2013-05-241-4/+0
|
* Add contents list to chunk and stratum metadataRichard Dale2013-05-241-15/+40
|
* Do not compress chunk artifacts upon creationLars Wirzenius2013-03-111-1/+1
| | | | | | | | | We already silently, automatically decompress if the artifact is compressed. It is thus safe to turn off compression. Compression saves disk space, but on ARM it takes up a lot extra time, and we have lots of disk space, so for speed, not compressing is a good idea.
* Add ExtractedTarball class and method to extract/mount an artifactJannis Pohlmann2013-02-211-1/+28
| | | | | | | | | | | | | ExtractedTarball is more or less the equivalent to MountableImage for artifacts that are not mountable images. So in order to inspect root file system tarballs, ExtractedTarball can be used, for disk images, MountableImage can be used. The morphlib.bins.call_in_artifact_directory() method combines these two classes and provides a way to extract/mount an artifact and call a callback with the temporary directory / mount point as its first argument. Using this, a plugin that runs a command relative to an artifact's root directory can be written easily.
* Override Python tarfile module to close file handles on exceptionSam Thursfield2012-11-151-0/+15
| | | | Reported upstream as: http://bugs.python.org/issue16477
* Close file handles correctly on exceptions when unpacking strataSam Thursfield2012-11-151-5/+6
| | | | | | | This prevents us from leaving file handles open when code throws an exception. When the file handle is on a loopback mount, this is a real problem because Morph then cannot unmount the image in its exception handler. In most cases 'with' is the best option.
* Use standard library function for relative pathsRichard Maw2012-10-091-9/+3
| | | | | os.path.relpath does approximately the same as mkrel, with the exception of asserting.
* Fix error reporting from tar extract errorsLars Wirzenius2012-08-311-7/+6
| | | | | | | | | | | | | | | | | | | | | | Mutliple issues here: * tarfile can throw IOErrors in addition to OSErrors, and both may need the filename attribute set * tarfile needs to be told to raise exceptions for errors, rather than ignoring them (errorlevel=1 to the constructor) * tarfile needs to be told to raise exceptions for tar format corruptions and not just I/O errors (errorlevel=2) * the exception should be re-raised with a plain "raise" instead of "raise e" to preserve the stack trace correctly; it's OK to modify the exception before re-raising it Also moved the creating of tf to the rest of the code that sets it up, for clarity. I've tested these changes manually, but not written a test case, since a test case is tricky to write for this. Originally found by Joe Burmeister.
* python scripts: pep8ize codebaseRichard Maw2012-08-011-27/+28
| | | | | | | | | This was done with the aid of the pep8 script, available by running `easy_install pep8`. It may be worth making this part of ./check, but that will require putting pep8 into the development tools stratum. This should be easy, given pep8 has no external dependencies.
* unit tests: add tests for tarball extractionRichard Maw2012-07-051-1/+1
|
* Normalize timestamps without filesystem operationsLars Wirzenius2012-06-291-4/+9
| | | | | | | This changes the timestamp in the Python tarfile.TarInfo data structure rather than setting the timestamp in the filesystem. Suggested by Richard Maw.
* Normalize timestamps of files in chunk artifactsLars Wirzenius2012-06-281-0/+9
| | | | | | | This avoids problems with clock skew between the machine that built an artifact and the machine the uses it. I ran into this problem during a test build of other changes in this patch series. We have seen it before, now it is fixed.
* morphlib.bins: remove create_stratumRichard Maw2012-06-141-8/+0
| | | | create_stratum is no longer used, since strata are now chunk lists
* morph: remove dead code and replace Execute with app.runcmdRichard Maw2012-05-301-2/+2
|
* bins: move errno from stagingareaRichard Maw2012-05-151-0/+1
| | | | | This import was forgotten when the artifact extract code was moved into bins, so it could be used in builder
* builder: use the same tar extract logic for all artifactsRichard Maw2012-05-141-10/+81
| | | | | Before only the staging area had symbolic links handled, now strata and systems will have the same logic.
* Fix minor problemsLars Wirzenius2012-04-181-1/+1
|
* Add a way to unpack a tar file from an open file handleLars Wirzenius2012-04-181-0/+12
|
* tar: add -h option to extractionRichard Maw2012-03-161-1/+1
| | | | | | This should stop tar from replacing symbolic links with directories and instead follow the symbolic links, extracting the files into the linked directories
* Test for and include dirnames in chunks the same as filenamesLars Wirzenius2012-03-061-3/+1
|
* Create chunks, strata in cache via temporary filesLars Wirzenius2012-02-271-8/+10
| | | | | This avoids problems with files with the right names but partial content, if morph is killed in the middle of writing the file.
* Bump copyright years, get rid of long lines and unused imports.Jannis Pohlmann2012-01-231-4/+1
|
* stop morph from using fakeroot and sudoLars Wirzenius2012-01-111-7/+3
| | | | | | | | | | | | | Instead, assume the whole build will be run, by the user, under fakeroot or sudo (the latter for system image builds). This allows us to run all non-system-image-build tests without root access at all. We now always create the cache directory, if missing, even if we're running as root. We no longer run ldconfig if ld.so.conf is missing. It is missing during our tests, but is (now) created by fhs-dirs for real builds.
* use tarfile againLars Wirzenius2011-12-141-10/+5
| | | | | Busybox tar is incapable of doing this, so we have to use Python's tarlib, at least for now.
* Made some sudo commands into fakeroot commandsRichard Maw2011-12-081-14/+14
| | | | | | | | | | | | | | | | | | | | Some commands can remain as fakeroot if they share the same fakeroot This can be done with the -i and -s parameters to fakeroot Now the Execute objects contain a reference to the file that stores their state, so fake commands' fake state is recognised by later commands The creating a system image definitely needs to be sudo though. Morph needs to create some files as part of the process, so unfortunately the equivalent to 'echo foo | sudo tee bar >/dev/null' is being used. They could be created with sufficient permissions, written then have better permissions set, but it seemed neater this way A better solution may be to have a subprogram that does the system image creation Putting files into the staging area for building other chunks is currently sudo, but shouldn't have to be. Rather than having a fakeroot per Execute object, it may be better to have a global one, though this would prevent concurrent use of fakeroot.
* Make the install parts of morphs run as sudoRichard Maw2011-12-081-3/+6
| | | | | | | | Then fix all the places where this broke things because they didn't have the required permissions. This is a potential security risk of course, it would be preferable if the install commands were run in a fakeroot session and only the final image building was run as root, but it needs to work first
* use external tar toolLars Wirzenius2011-12-061-10/+13
| | | | This saves a bunch of memory use.
* add memory profiling to create_chunkLars Wirzenius2011-12-061-2/+8
|
* use system tar instead of Python's tarfile library to extract tarballsLars Wirzenius2011-11-301-3/+2
|
* Fix chunk building to deal correctly with symlinks to directoriesLars Wirzenius2011-11-141-4/+10
|
* Combine unpack_chunk and unpack_stratum into unpack_binaryLars Wirzenius2011-11-031-17/+4
|
* Output multiple chunks when requestedLars Wirzenius2011-11-031-1/+2
|
* Include only files matching regexps, removes them from after adding themLars Wirzenius2011-11-031-4/+53
|
* Add tests for create_chunk's glob handlingLars Wirzenius2011-11-031-1/+1
|
* Add functions for creating and unpacking strata.Lars Wirzenius2011-10-171-0/+22
|
* Add functions for creating and unpacking binary chunks.Lars Wirzenius2011-10-171-0/+49
View Lorry Controller Status