| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
| |
- Move docstring from .write to .write.help
- Rework the content and formatting of the help information
|
| |
|
|
|
|
| |
Found by Richard Maw.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The KVM and VirtualBox deployments use sparse files for raw disk
images. This means they can store a large disk (say, tens or hundreds
of gigabytes) without using more disk space than is required for the
actual content (e.g., a gigabyte or so for the files in the root
filesystem). The kernel and filesystem make the unwritten parts of the
disk image look as if they are filled with zero bytes. This is good.
However, during deployment those sparse files get transferred as if
there really are a lot of zeroes. Those zeroes take a lot of time to
transfer. rsync, for example, does not handle large holes efficiently.
This change introduces a couple of helper tools (morphlib/xfer-hole
and morphlib/recv-hole), which transfer the holes more efficiently.
The xfer-hole program reads a file and outputs records like these:
DATA
123
binary data (exaclyt 123 bytes and no newline at the end)
HOLE
3245
xfer-hole can do this efficiently, without having to read through all
the zeroes in the holes, using the SEEK_DATA and SEEK_HOLE arguments
to lseek.
Using this, the holes take only take a few bytes each, making it
possible to transfer a disk image faster. In my benchmarks,
transferring a 100G byte disk image took about 100 seconds for KVM,
and 220 seconds for VirtualBox (which needs to more work at the
receiver to convert the raw disk to a VDI). Both benchmarks were from
a VM on my laptop to the laptop itself.
The interesting bit here is that the receiver (recv-hole) is simple
enough that it can be implemented in a bit of shell script, and the
text of the shell script can be run on the remote end by giving it to
ssh as a command line argument. This means there is no need to install
any special tools on the receiver, which makes using this improvement
much simpler.
|
|
|
|
|
|
| |
Slight duplication is necessary, but it's only a few lines. We could
move the duplicated code into the base class in 'morphlib.writeexts' if
there was more duplication.
|
|
|
|
|
| |
Also, be more flexible when parsing environment booleans -- convert to
lower case and match 0/1 and true/false as well as yes/no.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
-s, or --protect-args prevents the file path components of destination
or source paths being interpreted by the remote shell.
This is for wildcards or other shell features, but it breaks when paths
have whitespace.
We tend to always use absolute paths, so all uses of rsync now pass -s.
kvm.write needs it, since the disk can be written to a path with spaces.
Nfsboot and ssh-rsync need it because version labels are used, which may
have spaces, and temporary directories are used, which could have spaces
in weird TMPDIR configurations.
|
| |
|
| |
|
| |
|
| |
|
|
|
|
|
|
| |
If AUTOSTART is 'yes' then the VM will be started once it is
created. If it is 'no' or undefined, then it will need to be manually
started. If it is any other value, then an exception is raised.
|
| |
|
| |
|
|
|
|
| |
Suggested-By: Richard Maw
|
|
|
|
|
|
|
|
|
|
| |
This allows code sharing amongst all the places that create a
system in a raw disk image. This also adds the creation of
a factory-run subvolume, and fixes error messages for errors
that happen during a disk image creation.
Suggested-By: Richard Maw
Suggested-By: Sam Thursfield
|
|
|
|
| |
Suggested-By: Richard Maw
|
|
|