summaryrefslogtreecommitdiff
path: root/extensions
Commit message (Collapse)AuthorAgeFilesLines
* Be explicit about extensions that need python2Daniel Firth2016-11-1722-21/+22
|
* Revert "Add ostree.write deployment extension"Javier Jardón2016-10-125-297/+0
| | | | | | | | Deployment of the minimal system broke after applying this This reverts commit 5bca5fc3e8215601d1e2d33ce31f6a41c4bcfcc9. Change-Id: Ibbc7cd24e10adec207ccb9848985c301974dca6b
* Add ostree.write deployment extensionSam Thursfield2016-10-075-0/+297
| | | | | | | | | | | | | | | | | | | | | | | | This is a simple deployment extension to write the results of a Baserock build to an OSTree repo. See ostree.write.help for more information. The Baserock reference systems contain static device nodes, which are not supported by OSTree, and are obsolete on Linux systems. To work around this problem, I added a strip-device-nodes.configure extension. The systems should still work the same as devtmpfs will create the device nodes. I've tested this by deploying minimal-system-x86_64 to an OSTree repo, then checking it out into a temporary directory, creating a read-only bind-mount of the temporary directory and chrooting then into it. This is only going to be useful for containers systems and chroots for the time being. We would need to make some quite major changes to the Baserock reference systems to enable deployment to VMs or bare metal using OSTree. Change-Id: I8556139d118da759360cdc1ce998590b1f50b644
* == syntax doesn't work for a real /bin/shPaul Sherwood2016-07-121-1/+1
| | | | Change-Id: I6a563d7268df670495db792912e5ef8434d91d93
* Create /etc in the crazy circumstance it doesn't existBen Brown2016-06-231-0/+1
| | | | Change-Id: Ie47d1c1d59f208287dd4e4170a195cc4e8a79da4
* extensions/strip-gplv3.configure: Fix to work with YBDEdward Cragg2016-05-041-21/+10
| | | | | | | | Add an object and functions allowing metadata to be read more easily from multiple sources, either morph metadata, or with a conversion function for YBD formatted metadata. Change-Id: I94ea2425edecb4a659968134b0900081e22913d3
* cloud-init.configure: Fix use of MORPH_LOG_FDSam Thursfield2016-03-071-1/+7
| | | | | | | | | | | | | | MORPH_LOG_FD exists so that deployment extensions can write detailed log info to a file. This extension doesn't output much detailed log info, but some of the Python ones do & it's really useful for debugging. With Morph, this extension seems to have been writing logs to a file named '3' rather than to the actual deployment log. With YBD, this extension crashed because MORPH_LOG_FD was unset. Extra logs will now go to stdout when deploying with YBD. Change-Id: I46ead092d0e071970e355c8539823836f28dd97b
* Update licensecheck.py and strip-gplv3.configurePedro Alvarez2016-03-011-1/+3
| | | | Change-Id: I3149413c2126194c538105d26b93fde9a8298ed0
* mason: Use 'find' to clean the cachePedro Alvarez2016-02-251-2/+2
| | | | | | | | | | | Previous approach is not working anymore: # mv /srv/distbuild/!(remove) /srv/distbuild/remove -sh: syntax error: unexpected "(" Also use `find -delete` instead of `rm -r` because is faster. Change-Id: I0b500869281ce44ce565b8cca051c91961a6613d
* writeexts: Fix some errors the `pep8` tool foundSam Thursfield2016-02-251-7/+6
| | | | | | | | | | This includes a crash if DISK_SIZE isn't specified (due to missing 'writeexts' import). There are still various indent/formatting issues that `pep8` complains about. Change-Id: Ida805ad829eeee62768ac54313417d4c9a4a94d9
* writeexts: Clarify a few docstringsSam Thursfield2016-02-251-3/+19
| | | | Change-Id: I742825726280656d5834817c8a0b496fa6aedaae
* Opearate on the temp root when looking for menu.c32James Thomas2016-02-251-4/+4
| | | | | | | | It's possible we'll be generating the bootloader configuration on a partition that doesn't have the root filing system, so grap this file from the extracted rootfs instead Change-Id: I7a599654c2686f554e54aee069dd38200ab9e915
* Correctly setup /boot partition if availibleJames Thomas2016-02-251-10/+45
| | | | | | | | | | | | | | | Treat the prescence of a /boot partition as the partition where the bootloader configuration should live. By maintaining the baserock system layout, and installing the bootloader configuration to /boot in this manner, we can allow boot syslinux and u-boot systems to be upgraded Additionally move the installation of the MBR for x86 systems to where the partitioned system is created, since we only need to do it when deploying a partitioned system Change-Id: I39c90f73f22da209eefc1c408bd6d6c0ab6fd048
* Move extlinux.conf to /extlinux/James Thomas2016-02-251-1/+9
| | | | | | | | U-boot's sysboot command generally looks for the config in /extlinux rather than /, so move it there and create a symlink to maintain comptability with syslinux Change-Id: I19173e284993f364cc8892ba44b8f5d7bc394ec3
* Split btrfs filesystem functions in layout and rootfs creationJames Thomas2016-02-251-8/+13
| | | | | | | This will allow us to mirror the baserock btrfs layout on a non btrfs partition Change-Id: If5dbba133a187eeb26eae671da7dd71495127c8e
* Split creation of the bootloader config/install into separate functionJames Thomas2016-02-251-28/+46
| | | | | | | This will allow us to use the existing btfs layout function on other partitions (namely /boot) Change-Id: I6d43e6eb25c263f7c18eb06b96fa6076483f8e67
* Fix get_partition_by_mountpoint to actually return non '/' mount pointsJames Thomas2016-02-251-3/+6
| | | | Change-Id: Ia11f37eef9aa927414f8b487eba3a276cc91b2aa
* mason-report: sync with mason.sh to get the SHA1Pedro Alvarez2016-02-241-1/+1
| | | | Change-Id: I64f0d1de7d7e7e0329a61fb028cd7efdf418b13f
* mason: Move Mason definitions clone to /wsPedro Alvarez2016-02-211-1/+3
| | | | | | | It's being created in /srv/distbuild at the moment, and mason-report.sh is cleaning that folder after every run. Change-Id: If589b07149bbaa71aacfd33efe68050deafa3620
* mason: Fix scripts to not use morph init/checkoutPedro Alvarez2016-02-151-7/+2
| | | | | | https://storyboard.baserock.org/#!/story/71 Change-Id: Iaed10e9edb66f92fd35d61601c06f496c2bea5f3
* mason: Show build logs correctly on failureSam Thursfield2015-12-261-1/+1
| | | | | | This is more fall-out from my fixes to scripts/release-build Change-Id: I986b384b9cda07d0f9a9778f6decca080c094155
* trove.configure: Add support for installing SSL certificatesPedro Alvarez2015-11-192-0/+32
| | | | Change-Id: I892b1b0a99c7103fbe2a4ab49b273b76397b3feb
* Fix bug in fstab layout generationRichard Ipsum2015-11-021-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This fixes a bug in our fstab layout generation, currently we fail to generate the complete fstab layout, so /etc/fstab in any newly deployed or upgraded baserocks contains just: # Morph default system layout UUID=<uuid> / btrfs defaults,rw,noatime 0 1 This is actually a pretty bad problem because it breaks system-version-manager, since systemd creates a btrfs subvolume and mounts it to /var/lib/machines if /var/lib/machines doesn't already exist[1]. As a result system-version-manager when asked to remove an image will fail to do so, since it will fail to remove the /run subvolume because of the subvolume created at /run/var/lib/machines. The result is a partially removed image that cannot be removed with system-version-manager. The fstab layout we use currently: UUID=<uuid> / btrfs defaults,rw,noatime 0 1 UUID=<uuid> /var btrfs subvol=/state/var,defaults,rw,noatime 0 2 UUID=<uuid> /opt btrfs subvol=/state/opt,defaults,rw,noatime 0 2 UUID=<uuid> /srv btrfs subvol=/state/srv,defaults,rw,noatime 0 2 UUID=<uuid> /root btrfs subvol=/state/root,defaults,rw,noatime 0 2 UUID=<uuid> /home btrfs subvol=/state/home,defaults,rw,noatime 0 2 ensures that anything created under /var goes into a shared /state/var subvolume, so when systemd creates its 'machines' subvolume it will be under state/ which doesn't cause any problems when we try to remove a system. To reproduce, deploy an upgrade to your machine, reboot into the upgraded system, when you cat /etc/fstab you should see only one entry, mounting /. Now switch back to the previous image with, system-version-manager set-default <previous image name> reboot remove the newly deployed image with, system-version-manager remove <upgraded image name> after some time this will fail with an error message similar to this: Removing system: faultylayout Delete subvolume (no-commit): '/tmp/tmp1gdIL2/systems/faultylayout/run' ERROR: cannot delete '/tmp/tmp1gdIL2/systems/faultylayout/run' - Directory not empty Delete subvolume (no-commit): '/tmp/tmp1gdIL2/systems/faultylayout/orig' Traceback (most recent call last): File "/usr/bin/system-version-manager", line 421, in <module> SystemVersionManager(sys.argv, mount_dir).run() File "/usr/bin/system-version-manager", line 401, in run self.cmd_remove(args.system_name) File "/usr/bin/system-version-manager", line 363, in cmd_remove shutil.rmtree(system_root) File "/usr/lib/python2.7/shutil.py", line 247, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib/python2.7/shutil.py", line 247, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib/python2.7/shutil.py", line 247, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib/python2.7/shutil.py", line 247, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib/python2.7/shutil.py", line 256, in rmtree onerror(os.rmdir, path, sys.exc_info()) File "/usr/lib/python2.7/shutil.py", line 254, in rmtree os.rmdir(path) OSError: [Errno 1] Operation not permitted: '/tmp/tmp1gdIL2/systems/faultylayout/run/var/lib/machines [1]: http://cgit.freedesktop.org/systemd/systemd/commit/?id=113b3fc1a8061f4a24dd0db74e9a3cd0083b2251 Change-Id: I9c160d1fdd75fbbf9a3336d1ce35e4ce8ed7787d
* extensions: Add genivi.configure and use it in GENIVI systemsGENIVI-K1.0.1baserock/pedroalvarez/gdp-rebase3Pedro Alvarez2015-10-302-0/+149
| | | | | | | | | This extension allow us to do at configure time: - Configure different weston.ini files (for GENIVI baseline and GDP) - Enable weston.service - Set different backends in weston.service Change-Id: Idfdb8b3d0e881d2da43eeefe86e42ca6876bb790
* mason: clean-up all the cache after every runPedro Alvarez2015-10-251-0/+6
| | | | | | | | | | | This way we never fill up the disk. We don't care about the cache given that the distbiuld network uploads everything to the artifacts server. The solution is not very clever, but will help us to avoid Mason going red just because of disk space. Also, I don't think Mason deserves more attention. Change-Id: Ifbdaac0e992fe9cc4736010fd26dc9c9e7beb7f4
* Rawdisk partitioning v2: Add documentationEdward Cragg2015-10-242-0/+189
| | | | Change-Id: Icf965499380522d8f80cc07f17678179dedeee1a
* Rawdisk partitioning v2: Add partitioning functionsEdward Cragg2015-10-244-71/+490
| | | | | | Add partitioning functions to the rawdisk.write deployment extension Change-Id: I45bcabc191951d086b8f4ae028a248f95a5e6f2e
* Rawdisk partitioning v2: Add pyfdisk.py moduleEdward Cragg2015-10-231-0/+766
| | | | | | | This module is a GPL2 licenced python wrapper for fdisk, used to provide partitioning capabilities Change-Id: I5a28310f081045dd65d37bd5da4356ff3e6410f9
* install-files: Allow definition of manifests in multiple variablesRichard Maw2015-09-183-6/+28
| | | | | | | | | | | | | | | | | | | install-files was initially designed such that if you wanted to install multiple manifests you should put multiple whitespace separated paths in the INSTALL_FILES variable. This is clunky if you want to extend the set of files to install, particularly in cluster deployments with multiple systems, as otherwise you could make use of YAML merge keys. So now, like fstab.configure you can set multple options, by using environment variables that start with INSTALL_FILES. It's also easier to extend the set of install files on the command-line or in the environment this way too, since you don't need to parse the YAML to see what it originally had. Change-Id: I298db7ae9790a536e5a5f32b069d0908012695af
* Revert "Make ssh-rsync.check check VERSION_LABEL"Sam Thursfield2015-09-151-18/+0
| | | | | | | | | | | | This reverts commit 51b71cc42eb9f1d6c35823a490fd73b6bee63eac. This breaks in the case where system-version-manager doesn't have the --json flag, which it wasn't supposed to do. I don't have time to fix right now. See http://listmaster.pepperfish.net/pipermail/baserock-dev-baserock.org/2015-September/013263.html Change-Id: Icbfd7cc0cd8f4dc84d61d2a7f793ad566b67b3d8
* Update ceph.configure to facilitate cluster deploymentsPatrick Darley2015-08-231-86/+164
| | | | | | | | | | | | | | The changes include: - Updating the osd script for osd activation on first boot. - Allow the systemd units to run on boots subsequent to the first. - Adjust the disk location in the osd deployment script. - Add option to add client.admin.keyring at deploy time. Being deployed with key allows each node of the cluster administrative privelage from firstboot. - Allow OSD storage device location to be set at deploy time. Change-Id: Ibfd4db24b0ad946c551a8bdfe7d60d10a9ea687f
* openstack: use baremetal filtersTiago Gomes2015-08-191-0/+3
| | | | | | | | | Use baremetal filters when we are scheduling baremetal, as suggested on the Ironic install guide [1]. [1]: http://docs.openstack.org/developer/ironic/deploy/install-guide.html Change-Id: Ic6a82812748a98397b705e2428b06f77841d7437
* openstack: install chain.c32Tiago Gomes2015-08-191-0/+2
| | | | | | | This allows to deploy images that contain a bootloader inside with Ironic. Change-Id: I3c48ea4bd0e972f704ff21d1a21219da02be7d57
* strip-gplv3: Force `busybox find` to use posix-basic regular expressions.Pedro Alvarez2015-08-151-1/+1
| | | | | | | We could also set the '-regextype posix-basic' option to do this in GNU find, but that would fail with busybox. Change-Id: Ideaa9950efc23d8351e2796b7c6f06988c8c0176
* mason: Improve mason-report.sh to give more feedback when building.Pedro Alvarez2015-08-101-6/+45
| | | | | | | It will now show a row when a build has started, with a link to the current build log. Change-Id: Iec01529a80b45b2a61a6cf454a0ac8300c73e139
* writeexts.py: Capture stderr so that errors are capturedEdward Cragg2015-07-301-1/+1
| | | | | | | | The conversion from cliapp to the direct use of subprocess left a call which did not capture stderr, so `stderr=subprocess.STDOUT` is added to maintain the expected behaviour. Change-Id: I3ee0c064f9813d15004234d4daea81dc8219fc42
* Make ssh-rsync.check check VERSION_LABELRichard Ipsum2015-07-041-0/+18
| | | | | | | | | As well as checking whether it's defined, we make sure there isn't already a system named $VERSION_LABEL Note, this depends on the new json-list feature of system-version-manager. Change-Id: I0dee5253ba95a002b493f4179c2acda734afc37a
* More fixes for various deployment extensionsAdam Coldrick2015-06-237-6/+10
| | | | | | | | | Some of the extensions had extraneous imports, and a couple didn't import things they used. The same bug as in the kvm deployment extension existed in the virtualbox deployment extension, and is fixed in this commit. Change-Id: I2bd8169818ce5444af522c135a9e0326c1212ce2
* Fix kvm deploymentAdam Coldrick2015-06-222-4/+8
| | | | | | | | Deployment to kvm got broken by the patch to remove dependencies on morphlib and cliapp. This commit fixes the issues that were introduced. Change-Id: Iface1b38e638be4d84cf86b434ee30e1fffc34a0
* Merge branch 'baserock/adamcoldrick/remove-dependencies-v3'Adam Coldrick2015-06-1927-349/+1419
|\ | | | | | | | | | | | | | | This stops the deployment extensions from depending on morphlib and cliapp, as well as setting the definitions version to 5. Reviewed-by: Sam Thursfield <sam.thursfield@codethink.co.uk> Reviewed-by: Paul Sherwood <paul.sherwood@codethink.co.uk>
| * Remove dependencies on morphlib and cliapp from deployment extensionsbaserock/adamcoldrick/remove-dependencies-v3Adam Coldrick2015-06-1125-405/+545
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is done by either copying some utility functions from morph into writeexts.py, and using the `subprocess` module rather than cliapp's runcmd and ssh_runcmd. Note that this means that these extensions will require "$definitions_checkout/extensions" in PYTHONPATH when they are run. This commit also updates VERSION to 5, since the PYTHONPATH requirement means that this change is incompatible with old versions of morph. Change-Id: Iec6fa7e3c7219619ce55e18493e5c37c36e97816
| * Move xfer-hole and recv-hole into extensions/baserock/adamcoldrick/writeexts-in-definitions-v2Adam Coldrick2015-06-102-0/+295
| | | | | | | | Change-Id: I46ed5f3ec85f9662bebac592eff7a6eb6d628f28
| * Stop writeexts.py depending on morphlibAdam Coldrick2015-06-101-6/+17
| | | | | | | | Change-Id: I7f3702e80678aeee89dd22116510a6d8d7e04841
| * Put writeexts.py in extensions subdirectoryAdam Coldrick2015-06-101-0/+624
| | | | | | | | Change-Id: I6eb8fe69416bbf483ffa8c1c317c8f8cea56ef0e
* | OpenStack: avoid running OpenVSwitch in the ControllerTiago Gomes2015-06-171-0/+30
| | | | | | | | | | | | | | Avoid running OpenVSwitch in the Controller node for a three-node OpenStack system. Change-Id: I7a2da7f1d3ff1ed3dce5060675560d0a04996563
* | OpenStack: request time from the ControllerTiago Gomes2015-06-171-0/+61
| | | | | | | | | | | | | | | | Configure the Networking and Compute machines to ntpd from the Controller machine instead of other lower strata NTP servers. This is the recommended by the OpenStack install guide. Change-Id: I6911a50ee4bcebd2626859626b7236935994de9b
* | OpenStack: Add custom configuration for the componentsPedro Alvarez2015-06-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add custom configuration for Keystone, Nova, Neutron, Cinder, Glance, Ironic, Swift, Ceilometer, Horizon and Tempest for Openstack Kilo release. Notes: - Horizon configuration has change in order to be less error prone and to be close to the default configuration, the following changes has been made to achive this purpose: - Install the openstack_dashboard django project in /var/lib/horizon avoiding: - to write in /usr/lib/python2.7/site-packages/static - to maintain a copy of local_settings.py - to create a symlink as workaround to make django.wsgi to work with the previous configuration. - Change the path of django.wsgi file - Remove the 'horizon/' suffix to access Horizon via URL. - Grant permissions to access to /var/lib/horizon/static Signed-off-by: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk> Signed-off-by: Patrick Darley <patrick.darley@codethink.co.uk> Signed-off-by: Tiago Gomes <tiago.gomes@codethink.co.uk> Change-Id: I1fda4f85eb334b1b5bfec54dfcd9780a6112ea3b
* | openstack-nova.configure: Don't do anyting with libvirt default network.Pedro Alvarez2015-06-111-8/+0
|/ | | | | | | This network is not going to autostart anymore, there is no need to do this anymore. Change-Id: Ic166a04a705255aac32a5dabc416b87a42e5f971
* Fix install-essential-filesAdam Coldrick2015-06-031-2/+1
| | | | Change-Id: I6ace782d08aaa476e9da07d295724734e41b9837
* Move all the directories used for install-files into a subdirectorybaserock/adamcoldrick/all-exts-in-definitions-v2Adam Coldrick2015-06-021-1/+1
| | | | Change-Id: I309c183ce8b9ff9d0f5ac4807244547f2cc4ddf5
View Lorry Controller Status