summaryrefslogtreecommitdiff
path: root/morphlib
Commit message (Collapse)AuthorAgeFilesLines
* SystemBuilder: mount rootfs rw initiallyRichard Maw2012-07-092-2/+2
| | | | | | | | | | This is required for systemd's journald to start. This is probably a bad dependency in systemd, trying to start the journal before it has mounted everything properly. This required a compat change, it is a string to make it more noticeable that it's a temporary version.
* Merge branch 'master' of roadtrain.codethink.co.uk:baserock/morphLars Wirzenius2012-07-061-0/+4
|\
| * Add a hook for creating a BuildCommand instanceLars Wirzenius2012-07-051-0/+4
| | | | | | | | | | This will allow plugins to override the BuildCommand with their own instance.
* | Merge remote branch 'origin/baserock/feature/S2995-write-build-logs-to-cache'Lars Wirzenius2012-07-061-3/+26
|\ \ | |/ |/|
| * ChunkBuilder: runcmd, redirect err to outRichard Maw2012-07-061-1/+2
| | | | | | | | | | Data buffers can get confused if two file handles are writing to the same file, python's subprocess module has subprocess.STDOUT for this.
| * ChunkBuilder: dump build log on errorRichard Maw2012-07-061-4/+20
| |
| * ChunkBuilder: write output of build to cacheRichard Maw2012-07-061-3/+9
| |
* | bins_tests: fix tabsRichard Maw2012-07-051-3/+3
| |
* | unit tests: add tests for tarball extractionRichard Maw2012-07-052-2/+75
|/
* tests: fix up for change in artifact nameRichard Maw2012-07-032-6/+30
| | | | | This fixes references to a system artifact without the -rootfs appended and adds a unit test for resolving an arm system artifact.
* ArtifactResolver: systems define multiple artifactsRichard Maw2012-07-031-16/+22
| | | | | Instead of a System source producing one System artifact, it can produce multiple System artifacts, dependent on if it is an ARM System.
* Merge branch 'master' of roadtrain.codethink.co.uk:baserock/morphLars Wirzenius2012-07-031-5/+3
|\
| * morph branching: use git-config to set pushInsteadOfRichard Maw2012-07-031-5/+3
| | | | | | | | | | | | | | This is the standard way of altering config, it shouldn't be done by modifying the config file. This may allow better forward compatibility if the config keys or storage format change.
* | Merge remote branch 'origin/baserock/bugfix/S3215-bootstrap'Lars Wirzenius2012-07-031-2/+16
|\ \ | |/ |/|
| * BuildCommand: make get_recursive_deps iterativeRichard Maw2012-07-031-7/+6
| | | | | | | | This is less code and can't blow the stack.
| * build: install recursive deps in staging areaRichard Maw2012-07-031-2/+14
| | | | | | | | | | | | All of the dependencies need to be installed, so generate the transitive closiure to pass to cache_artifacts_locally and install_chunk_artifacts.
| * build: re-implement bootstrap build logic.Richard Maw2012-07-021-0/+3
| |
* | Merge remote branch 'origin/baserock/bugfix/S2528-remount-rw'Lars Wirzenius2012-07-022-2/+4
|\ \ | |/ |/|
| * CacheKeyComputer: increment compat-versionRichard Maw2012-07-021-0/+2
| | | | | | | | | | | | | | | | | | This will prevent systems that don't have the right mount options being cache hits. Having fstab be in some System configuration morphology, which is also included in the cache key would prevent this being needed if the fstab format changes again.
| * SystemBuilder: fix mounting optionsRichard Maw2012-06-251-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | fstab had errors=remount-ro, this is an ext option, btrfs doesn't support it. However it should be mounted noatime, to prevent the metadata trees being duplicated because they were read. extlinux.conf also has its mounting options changed so that the rootfs is mounted read-only initially. This should prevent the metadata trees being updated by reading them until it is remounted with noatime.
* | Normalize timestamps without filesystem operationsLars Wirzenius2012-06-291-4/+9
| | | | | | | | | | | | | | This changes the timestamp in the Python tarfile.TarInfo data structure rather than setting the timestamp in the filesystem. Suggested by Richard Maw.
* | Improve error message for missing build-depsLars Wirzenius2012-06-291-2/+4
| | | | | | | | Suggested by Richard Maw.
* | Normalize timestamps of files in chunk artifactsLars Wirzenius2012-06-282-1/+10
| | | | | | | | | | | | | | This avoids problems with clock skew between the machine that built an artifact and the machine the uses it. I ran into this problem during a test build of other changes in this patch series. We have seen it before, now it is fixed.
* | Add check for missing build depends in strataLars Wirzenius2012-06-281-0/+11
| | | | | | | | | | | | | | This will make a build fail if the morphology is missing explicit build dependencies. Also fix test causes so that ./check passes.
* | Improve status messages for clarityLars Wirzenius2012-06-271-13/+29
| | | | | | | | | | | | | | Suggested by Richard Maw. This commit also includes a bug fix, since I'm too lazy to detangle changes in git commits today.
* | Remove unnecessary variableLars Wirzenius2012-06-271-1/+1
| | | | | | | | Suggested by Richard Maw.
* | Rename build_order method to build_in_orderLars Wirzenius2012-06-271-3/+3
| | | | | | | | Suggested by Richard Maw.
* | Create class for build logic and fix the logicLars Wirzenius2012-06-271-206/+280
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The build logic was spread all across the Morph class. This patch gathers it into one class. Having it in one place is useful for general code quality reasons, but also makes it feasible to experiment with the build logic, and override only parts of it. This patch also changes how the build logic works. Earlier we kept one staging area for staging builds, and were unpacking everything in a build group at a time. This meant that the staging area would contain things that were not strictly build dependencies. This has a possiblity of changing what gets built. The new logic creates a new, fresh, clean staging area for each build. This makes building a bit slower, but we'll fix that by making it really fast to create a staging area and filling it with build dependencies.
* | Fix test suite to deal with morph output changesLars Wirzenius2012-06-261-7/+9
| | | | | | | | | | | | | | | | Also, some bug fixes. Also, when handling a BaseException, log the exception (with traceback) that we're handling, in case there is an error while handling it, because the second error will otherwise mask the first one.
* | Fix problems found by Richard in reviewLars Wirzenius2012-06-262-14/+9
| |
* | Be more usefully verbose for chunk buildingLars Wirzenius2012-06-251-2/+6
| |
* | Add status message for unpacking chunksLars Wirzenius2012-06-252-4/+5
| |
* | Replace Morph.msg with Morph.statusLars Wirzenius2012-06-252-55/+134
| | | | | | | | | | | | | | | | | | The new method takes a list of keyword arguments. This is more useful than the old way of giving just a string, since now the presentation layer may transmogrify the status update. For example, it can usefully translate the message to another language. Add --verbose option to allow more control over what the user sees.
* | Remove optional msg argument from Submodule classLars Wirzenius2012-06-251-3/+2
| | | | | | | | | | Nobody was using this, so it is obviously useless. Instead, log at the warning level.
* | Merge remote branch ↵Lars Wirzenius2012-06-252-9/+10
|\ \ | |/ |/| | | 'remotes/origin/baserock/feature/S2938-system-artifact-splitting'
| * make-patch: fix artifact name to include suffixRichard Maw2012-06-211-1/+4
| |
| * SystemBuilder: create a new artifact for kernelRichard Maw2012-06-211-8/+6
| | | | | | | | | | | | | | The kernel should be considered a full artifact, not just metadata. The rootfs is now suffixed by -rootfs and the kernel is named $name-kernel. This is similar to how chunk splitting works.
* | Use shutil.copyfileobj to avoid excessive memory useLars Wirzenius2012-06-221-1/+1
| | | | | | | | | | | | Thanks to Richard Maw for pointing this out! I need to get out of my ancient habits and use the modern Python stdlib and rember that files in these degenerate modern times can be larger than 64 KiB.
* | Handle only the urllib2.URLError exceptionsLars Wirzenius2012-06-222-4/+9
| | | | | | | | | | | | | | | | Handling all exceptions in a way that assumes they're all related to the URL fetching hides unrelated errors, such as missing imports, or bad types, or such. Also, add more logging for debugging.
* | Add logging, for debuggingLars Wirzenius2012-06-221-0/+1
| |
* | Prettify str(RemoteArtifactCache) to be the server URLLars Wirzenius2012-06-221-0/+3
| | | | | | | | | | This makes error messages that print the artifact cache object be useful to the reader.
* | Include cache id in error messageLars Wirzenius2012-06-221-2/+3
| | | | | | | | For usefulness when tracing why artifact isn't being found.
* | Change install_artifacts to fetch missing artifactsLars Wirzenius2012-06-221-3/+9
|/ | | | | | | | | | | | This is a behavioral change, and is useful when building and having access to an artifact cache server. Previously, nothing was actually fetching artifacts from the remote server when the staging area was being crafted. Morph could do with a bit of cleanup where we separate more cleanly the acts of fetching stuff to the local artifact cache from the remote one, when possible, and then using artifacts from the local cache. Currently this stuff is spread around a bit too much. I did not do that in this patch, however, to keep undiscussed changes small.
* ChunkBuilder: mount /dev/shm as wellRichard Maw2012-06-201-19/+24
| | | | | | This required refactoring mount_proc and unmount_proc into more generic mounting functions. do_mounts returns the paths that were mounted instead of just one
* app: rename setup_proc to setup_mountsRichard Maw2012-06-202-13/+13
| | | | | This change is purely cosmetic, but reflects the fact that we need to mount more than just proc sometimes.
* Remove dead code (dependencies in Source objects)Lars Wirzenius2012-06-182-34/+0
|
* Return list of built artifacts from builderLars Wirzenius2012-06-151-2/+9
|
* Merge branch 'richardmaw/strata-chunk-lists'Richard Maw2012-06-147-88/+190
|\
| * fix tabs and line widths so ./check passesRichard Maw2012-06-141-11/+11
| |
| * morphlib.builder2: check in local cache firstRichard Maw2012-06-141-2/+2
| | | | | | | | | | | | It's possible to work without a remote artifact cache, sadly there is not an explicit check for a None remote artifact cache to prevent it downloading dependencies.
View Lorry Controller Status