summaryrefslogtreecommitdiff
path: root/morphlib/builder2.py
Commit message (Collapse)AuthorAgeFilesLines
...
* Consistency fixSam Thursfield2012-11-151-1/+1
|
* Write /etc/os-release in system imagesSam Thursfield2012-10-261-1/+14
| | | | | | This is intended to for human consumption, with the full set of metadata in /baserock/system-artifact-name.meta. It currently lists system name, the symbolic ref of the system morphology and build date.
* Write metadata for the rootfs itself into the rootfsSam Thursfield2012-10-261-2/+3
| | | | | | | | This makes it easier to identify what version of a system we are running. Tests are updated to check inside the contents of the rootfs we created, and the code to mount the rootfs as a loopback device was extracted out into tests.as-root/lib
* Fix use of GzipFile to not assume "with" protocolLars Wirzenius2012-10-121-3/+4
| | | | | This fixes a bunch of tests so they pass on squeeze, which has a version of Python whose GzipFile doesn't support the "with" protocol.
* Correct a few long lines.Daniel Silverstone2012-10-091-3/+6
|
* Compress system disk imagesDaniel Silverstone2012-10-081-29/+45
| | | | | | | | | | Since tarball rootfs images are compressed with gzip -1, we should do the same for disk images. This means they'll need decompressing before they can be used as disk images, but it means we'll be shunting around something closer to their true data size rather than the disk image size. For example, a 2G development system image compresses down to around 380M which is a lot faster to transfer and doesn't risk de-sparseification.
* Rework git caches to be bare mirrors of the repos.Daniel Silverstone2012-09-141-11/+9
| | | | | | | | This reworks the code for managing and using the git caches in morph to treat the caches as bare repositories which are mirrors of where we clone from. In addition we correctly prune the branches during updates, so that we don't end up accumulating pointless branches over and over. This is even more important with branch-and-merge generating temporary build refs for things.
* Fix code format for unicode fixRichard Maw2012-09-131-1/+2
|
* Encode the root path for os.walk()Daniel Silverstone2012-09-121-1/+1
| | | | | | | | | Unfortunately we were running into issues with unicode filenames in repositories causing set_mtime_recursively() to explode. This fixes the problem. Reviewed-By: Lars Wirzenius (In real life)
* Warning instead of error for empty strata, temporarilySam Thursfield2012-09-031-1/+1
| | | | | | | | | This kind of reverts commit: c4facff97b27968f00a75ffb3d49cea2080ad9a4 Currently our bootstrap depends on building an empty stratum; this should be changed so that it builds a tarball system image, but this is currently broken too. This commit should be reverted when bootstrap is fixed to not depend on building an empty stratum.
* Log a clear error if creating an empty stratumSam Thursfield2012-08-301-0/+2
| | | | | This is mostly useful for diagnosing Morph bugs. Currently empty strata just cause confusing errors when building systems.
* Fix test coverage for squeeze coverage.pyLars Wirzenius2012-08-151-3/+3
| | | | | | | | The squeeze version of coverage.py seems to have problems with methods that consist of a single pass statement. This patch replaces those statements with placeholder docstrings so that ./check passes on squeeze. Reviewed-By: Daniel Silverstone (on irc)
* Initial non-syslinux disk image kindDaniel Silverstone2012-08-141-0/+136
| | | | | | | This adds a disk system image kind which does everything the syslinux-disk one does, but without syslinux. It deliberately carries stubbed bootloader operations so that we can later make syslinux-disk inherit from this one and override it.
* python scripts: pep8ize codebaseRichard Maw2012-08-011-75/+83
| | | | | | | | | This was done with the aid of the pep8 script, available by running `easy_install pep8`. It may be worth making this part of ./check, but that will require putting pep8 into the development tools stratum. This should be easy, given pep8 has no external dependencies.
* Refactor: move kernel artifact creation into base classLars Wirzenius2012-07-261-0/+19
|
* Move fstab creation into base classLars Wirzenius2012-07-261-0/+20
|
* Move strata unpacking into a base classLars Wirzenius2012-07-261-0/+54
|
* Start a tarball systembuilderLars Wirzenius2012-07-261-0/+2
| | | | | Also, rename the syslinux-disk builder plugin file to make it clear it's a system builder plugin.
* Move syslinux-disk system building into pluginLars Wirzenius2012-07-261-237/+1
|
* Refactor system building to use syskind specific classLars Wirzenius2012-07-261-2/+50
|
* Expose the real filename for SaveFileLars Wirzenius2012-07-171-1/+1
|
* Log to morph's log output of a failed buildLars Wirzenius2012-07-171-0/+7
| | | | | | Normally, the build log would go into the artifact cache, but it's tedious to fetch from there, so we log it into the normal morph log as well if things fail.
* SystemBuild: remove check for if arch == NoneRichard Maw2012-07-111-4/+4
| | | | | | This should never happen any more, so the check is now redundant. This may have a negligible performance benefit, but it prevents the code implying that None is a valid architecture.
* SystemBuilder: mount rootfs rw initiallyRichard Maw2012-07-091-1/+1
| | | | | | | | | | This is required for systemd's journald to start. This is probably a bad dependency in systemd, trying to start the journal before it has mounted everything properly. This required a compat change, it is a string to make it more noticeable that it's a temporary version.
* ChunkBuilder: runcmd, redirect err to outRichard Maw2012-07-061-1/+2
| | | | | Data buffers can get confused if two file handles are writing to the same file, python's subprocess module has subprocess.STDOUT for this.
* ChunkBuilder: dump build log on errorRichard Maw2012-07-061-4/+20
|
* ChunkBuilder: write output of build to cacheRichard Maw2012-07-061-3/+9
|
* Merge remote branch 'origin/baserock/bugfix/S2528-remount-rw'Lars Wirzenius2012-07-021-2/+2
|\
| * SystemBuilder: fix mounting optionsRichard Maw2012-06-251-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | fstab had errors=remount-ro, this is an ext option, btrfs doesn't support it. However it should be mounted noatime, to prevent the metadata trees being duplicated because they were read. extlinux.conf also has its mounting options changed so that the rootfs is mounted read-only initially. This should prevent the metadata trees being updated by reading them until it is remounted with noatime.
* | Fix test suite to deal with morph output changesLars Wirzenius2012-06-261-7/+9
| | | | | | | | | | | | | | | | Also, some bug fixes. Also, when handling a BaseException, log the exception (with traceback) that we're handling, in case there is an error while handling it, because the second error will otherwise mask the first one.
* | Fix problems found by Richard in reviewLars Wirzenius2012-06-261-3/+3
| |
* | Be more usefully verbose for chunk buildingLars Wirzenius2012-06-251-2/+6
| |
* | Add status message for unpacking chunksLars Wirzenius2012-06-251-1/+3
| |
* | Replace Morph.msg with Morph.statusLars Wirzenius2012-06-251-22/+48
| | | | | | | | | | | | | | | | | | The new method takes a list of keyword arguments. This is more useful than the old way of giving just a string, since now the presentation layer may transmogrify the status update. For example, it can usefully translate the message to another language. Add --verbose option to allow more control over what the user sees.
* | SystemBuilder: create a new artifact for kernelRichard Maw2012-06-211-8/+6
|/ | | | | | | The kernel should be considered a full artifact, not just metadata. The rootfs is now suffixed by -rootfs and the kernel is named $name-kernel. This is similar to how chunk splitting works.
* ChunkBuilder: mount /dev/shm as wellRichard Maw2012-06-201-19/+24
| | | | | | This required refactoring mount_proc and unmount_proc into more generic mounting functions. do_mounts returns the paths that were mounted instead of just one
* app: rename setup_proc to setup_mountsRichard Maw2012-06-201-7/+7
| | | | | This change is purely cosmetic, but reflects the fact that we need to mount more than just proc sometimes.
* Return list of built artifacts from builderLars Wirzenius2012-06-151-2/+9
|
* fix tabs and line widths so ./check passesRichard Maw2012-06-141-11/+11
|
* morphlib.builder2: check in local cache firstRichard Maw2012-06-141-2/+2
| | | | | | It's possible to work without a remote artifact cache, sadly there is not an explicit check for a None remote artifact cache to prevent it downloading dependencies.
* morphlib.builder2: write stratum metadataRichard Maw2012-06-141-5/+25
| | | | | | Stratum metadata was getting lost because it is not part of the chunks. Now metadata is written to the cache, then when the system builder assembles the strata it copies the metadata in.
* morphlib.builder2: fix unit testsRichard Maw2012-06-141-4/+4
|
* morphlib.builder2: write overlaps to cacheRichard Maw2012-06-141-15/+38
| | | | Rather than just log the overlaps, write them to the cache as well
* builder2: write json to cache instead of tarballRichard Maw2012-06-141-33/+53
| | | | | | | | | | | | | | Note that this has no way to deal with an old cached stratum being a tarball yet. Strata are now json files listing basenames, which get converted into ArtifactCacheReferences. SystemBuilders now have to ensure they download the chunks as well, since having the strata doesn't imply that the chunks are also available. Overlap checking is slightly more complicated by strata not being tarballs any more, but this is dealt with by looking at the chunks' tarballs.
* Merge branch 'richardmaw/arm-hack'Richard Maw2012-06-131-14/+35
|\
| * SystemBuilder ARM: write kernel as System metadataRichard Maw2012-06-131-0/+13
| | | | | | | | | | This should really be an artifact, but it is assumed that systems only generate one artifact, so this may be difficult.
| * SystemBuilder: pass missing parameter to install_boot_filesRichard Maw2012-06-131-1/+1
| |
| * SystemBuilder: __getitem__ is for indexing not propertiesRichard Maw2012-06-131-1/+1
| | | | | | | | Note to self, remember python is not lua
| * morphlib: add 'arch' field to morphologiesRichard Maw2012-06-131-13/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an ugly, ugly way to do this, but time is pressing. SystemBuilder checks what arch is defined in the morphology, if it is an x86 (or None for compatibility) then it will do the syslinux install stuff. This hack is needed because syslinux is x86 specific and arm often has different requirements for where the kernel must be loaded from, sometimes it is flash, sometimes it is a different partition. This will likely become board specific, but for a qemu-system-arm, the kernel should be a separate file, to be passed on the command line. Having a different 'kind' for each architecture would be a nicer way, but would require more changes, since there are various checks for morphology['kind'] == 'system'
* | Add setup_proc as initializer argument to BuilderLars Wirzenius2012-06-111-2/+3
|/ | | | | Setting it via direct attribute access, when everything else is set via initializer arguments, is just strange.
View Lorry Controller Status