| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| | |/
| |
| |
| | |
I meant to do this as part of the previous merge.
|
| | |\
| | |
| | |
| | |
| | | |
Reviewed-By: Richard Ipsum <richard.ipsum@codethink.co.uk>
Reviewed-By: Francisco Redondo Marchena <francisco.marchena@codethink.co.uk>
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
In order to check the user's credentials at the start of deployment, we
try to run `glance image-list`. I found a situation where this command
failed despite my credentials being correct. Morph outputted a
misleading error message that said 'Wrong OpenStack credentials'
The code now checks that the error returned by 'glance' does indeed
look like a credentials error. If it doesn't, the full error output is
displayed. The error I encountered now gets a message like this:
ERROR: openstack.check failed with code 1: ERROR: Failed to connect
to OpenStack instance at https://example.com:5000/v2.0:
[('SSL routines', 'SSL3_GET_SERVER_CERTIFICATE', 'certificate verify
failed')]
(If you are curious, I fixed this by running `update-ca-certificates`.)
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
The user should be aware of this because if they aren't building
baserock:baserock/definitions or a repo forked from it, those extensions
won't be available.
Also fix some long lines that I seem to have failed to commit already.
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | |/ |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| | |
This patch also modifies rawdisk.write to use the UPGRADE environment
variable to figure out when is doing an upgrade or a fresh deployment.
This change is important because os.path.isfile doesn't work with
devices.
|
| | |
| |
| |
| | |
Reviewed-by: Richard Maw
|
| | |
| |
| |
| |
| | |
- Document different ways of calling parameters
- Allowed values for boolean parameters
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| | |
Add AUTOSTART parameter
|
| | |
| |
| |
| |
| | |
- Move docstring from .write to .write.help
- Rework the content and formatting of the help information
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This patch solves the issue caused by upgrading a system without
a factory version. Currently we are only using the factory
version to snapshot its orig subvolume to make faster the transfer
of the new content (rsync won't have to send everything).
The default symlink may not be present, but it can't be deleted
easily using system-version-manager.
This is a quick fix, but in the future we may want to not harcode
the path from where we snapshot the orig subvolume. Or improve
system-version-manager to make sure that the default symlink
is always present.
|
| | |
| |
| |
| |
| |
| | |
Reviewed-by: Sam Thursfield
Reviewed-by: Jim MacArthur
Reviewed-by: Richard Ipsum
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
A quirk in the resource cleanup code meant that if you gave the same
version label when deploying a new version, then it would fail, then
remove the old version, as it had assumed that it was the one to create
those directories.
This patch fixes this issue by making short context managers for all the
resource allocation, so cleanup is done by walking up the context
managers, so only the mount and the temporary directory need to be
cleaned up if the `mkdir "$VERSION_ROOT"` fails.
I've tested this with a deploy of a version that doesn't already exist,
and the version I'm currently running, so I can conclusively say it's
fixed that problem.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
It now gives an error message.
Previously it would fail with a backtrace like this:
2014-10-08 09:51:37 [systems/genivi-baseline-system-armv7lhf-jetson.morph][self]Removing temporary mounts
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cliapp/app.py", line 190, in _run
self.process_args(args)
File "/src/morph/morphlib/exts/ssh-rsync.write", line 54, in process_args
self.upgrade_remote_system(location, temp_root)
File "/src/morph/morphlib/exts/ssh-rsync.write", line 107, in upgrade_remote_system
location, ['btrfs', 'subvolume', 'delete', orig_dir])
UnboundLocalError: local variable 'orig_dir' referenced before assignment
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
With this patch, the fstab of the system to be deployed
as an upgrade will be conifgured using the UUID of
the disk.
Now when doing an upgrade is not needed to specify the
ROOT_DEVICE.
|
| | |
| |
| |
| |
| |
| | |
The openstack.write extension was calling a nonexistent method
'check_location'. This metod was moved to openstack.check
in the commit ba7d1d1ed3bad002ce36e5d4adf4e3794625091a.
|
| | | |
|
| | |\
| | |
| | |
| | |
| | | |
Reviewed-by: Sam Thursfield
Reviewed-by: Lars Wirzenius
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
If the credentials are wrong, then morph will fail before
attempting the OpenStack deployment.
To achieve that openstack.check will attempt to run
`glance image-list`.
|
| | | |
| | |
| | |
| | | |
Found by Richard Maw.
|
| | |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The KVM and VirtualBox deployments use sparse files for raw disk
images. This means they can store a large disk (say, tens or hundreds
of gigabytes) without using more disk space than is required for the
actual content (e.g., a gigabyte or so for the files in the root
filesystem). The kernel and filesystem make the unwritten parts of the
disk image look as if they are filled with zero bytes. This is good.
However, during deployment those sparse files get transferred as if
there really are a lot of zeroes. Those zeroes take a lot of time to
transfer. rsync, for example, does not handle large holes efficiently.
This change introduces a couple of helper tools (morphlib/xfer-hole
and morphlib/recv-hole), which transfer the holes more efficiently.
The xfer-hole program reads a file and outputs records like these:
DATA
123
binary data (exaclyt 123 bytes and no newline at the end)
HOLE
3245
xfer-hole can do this efficiently, without having to read through all
the zeroes in the holes, using the SEEK_DATA and SEEK_HOLE arguments
to lseek.
Using this, the holes take only take a few bytes each, making it
possible to transfer a disk image faster. In my benchmarks,
transferring a 100G byte disk image took about 100 seconds for KVM,
and 220 seconds for VirtualBox (which needs to more work at the
receiver to convert the raw disk to a VDI). Both benchmarks were from
a VM on my laptop to the laptop itself.
The interesting bit here is that the receiver (recv-hole) is simple
enough that it can be implemented in a bit of shell script, and the
text of the shell script can be run on the remote end by giving it to
ssh as a command line argument. This means there is no need to install
any special tools on the receiver, which makes using this improvement
much simpler.
|
| | |
| |
| |
| |
| |
| | |
The arguments to `morph deploy` can get quite long, any way we can make it
shorter and clearer is useful. We can also avoid having the strange
--no-upgrade flag in future.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This avoids confusion when the user expected to be doing an initial
deployment and wasn't aware that a file with the same name as the
target already existed. Previously rawdisk.write would try to mount
the file and upgrade it.
Now we require the user to pass '--upgrade' when they intend to
upgrade, as with other deployment extensions.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If btrfs is not present in the kernel we end up with strange output like
this:
Error creating disk image2014-06-10 16:00:40
[devel-system-x86_64-generic][my-raw-disk-image][rawdisk.write]Failure
to create disk image at /src/tmp/testdev.img
ERROR: Command failed: mount -o loop /src/tmp/testdev.img
/src/tmp/deployments/tmpQ7wXO1/tmp4lVDcu/tmpvHSzDE
mount: mounting /dev/loop0 on
/src/tmp/deployments/tmpQ7wXO1/tmp4lVDcu/tmpvHSzDE failed: Device or
resource busy
To avoid this confusing error, Morph should explicitly check first.
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
If INITRAMFS_PATH is specified and the file exists, then the produced
kernel command line will use root=UUID=$uuid_of_created_disk rather than
root=/dev/sda, which may be incorrect.
Help files have been updated to mention the new option.
This leads to an unfortunate duplication of the path to the initramfs,
in both the location field of the nested deployment and the
INITRAMFS_PATH of the disk image creation.
However, an initramfs could be produced by a chunk and put in the same
place, so it doesn't make sense to couple the rawdisk and initramfs
write extensions to remove this duplication.
Similarly, there may be multiple valid initramfs in the rootfs e.g.
extlinux loads a hypervisor, which is Linux + initramfs, and the
initramfs then boots a guest Linux system, which uses a different
initramfs.
This makes it important to explicitly let the rootfs write extensions
know which to use, or not as the case may be.
util-linux's blkid is required, since the busybox version ignores the
options to filter its output, and parsing the output is undesirable.
Because syslinux's btrfs subvolume support is limited to being able to
use a non-0 default subvolume, the initramfs has to be copied out of
the run-time rootfs subvolume and into the boot subvolume.
This pushed the required disk space of a minimal system over the 512M
threshold because we do not have the userland tooling support to be able
to do a btrfs file contents clone.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This creates a gzipped cpio archive that may be used as an initramfs.
It is hard-coded to use gzip to compress the initramfs, since it's the
most common way to do it.
This is unfortunate, since the busybox gzip utility only allows maximum
compression, which is rather slow and doesn't give progress reporting,
so you can easily think it's gotten stuck.
It's possible to use other compression formats, but they need the kernel
to be built with them supported, and in the case of lz4, unusual userland
tools to create it, since the version of lz4 supported in the kernel is
not what the standard lz4 tools produce.
|
| | |\
| | |
| | |
| | |
| | | |
Reviewed-By: Richard Maw <richard.maw@codethink.co.uk>
Reviewed-By: Lars Wirzenius <lars.wirzenius@codethink.co.uk>
|
| | | | |
|
| | | | |
|
| | |/
| |
| |
| |
| |
| |
| | |
Add support to the VirtualBox write extension to notice if we are
doing a Vagrant Basebox installation and not do the clever network
setup we normally do to allow machines to talk to one another since
this confuses Vagrant quite a bit if it is left in.
|
| | |
| |
| |
| |
| | |
Move the help out of the comment and into a help file,
and add a clearer example.
|
| | | |
|
| | |
| |
| |
| |
| | |
Test that a VM with the given name does not already exist, and check
that the files specified in ATTACH_DISKS do already exist.
|
| | |
| |
| |
| |
| | |
Also, change it to log the real error message in morph.log before
raising a more general exception to the user.
|
| | |
| |
| |
| |
| |
| | |
Slight duplication is necessary, but it's only a few lines. We could
move the duplicated code into the base class in 'morphlib.writeexts' if
there was more duplication.
|
| | |
| |
| |
| |
| |
| | |
Begin adding help documentation for configuration
and write extensions starting with nfsboot.write,
rawdisk.write and tar.write.
|
| | | |
|
