summaryrefslogtreecommitdiff
path: root/morphlib/exts
diff options
context:
space:
mode:
authorAdam Coldrick <adam.coldrick@codethink.co.uk>2015-03-19 09:34:58 +0000
committerMorph (on behalf of Adam Coldrick) <adam.coldrick@codethink.co.uk>2015-03-19 09:34:58 +0000
commit7db4ee53fb5398dd8f4ae8f56778735fe6531178 (patch)
tree01513d77326acd03b2da356ec2cd7f4761901b6b /morphlib/exts
parent211d6317d22bace089da58875d280ae5e54d5d54 (diff)
downloadmorph-7db4ee53fb5398dd8f4ae8f56778735fe6531178.tar.gz
Morph build 2ee8190abe87461992f5b7ed85fe2ee9
System branch: master
Diffstat (limited to 'morphlib/exts')
-rwxr-xr-xmorphlib/exts/add-config-files.configure5
-rwxr-xr-xmorphlib/exts/fstab.configure5
-rwxr-xr-xmorphlib/exts/initramfs.write5
-rw-r--r--morphlib/exts/initramfs.write.help20
-rwxr-xr-xmorphlib/exts/install-files.configure5
-rw-r--r--morphlib/exts/install-files.configure.help14
-rwxr-xr-xmorphlib/exts/kvm.check74
-rwxr-xr-xmorphlib/exts/kvm.write40
-rw-r--r--morphlib/exts/kvm.write.help92
-rwxr-xr-xmorphlib/exts/nfsboot.check5
-rwxr-xr-xmorphlib/exts/nfsboot.configure5
-rwxr-xr-xmorphlib/exts/nfsboot.write18
-rw-r--r--morphlib/exts/nfsboot.write.help23
-rwxr-xr-xmorphlib/exts/openstack.check20
-rwxr-xr-xmorphlib/exts/openstack.write46
-rw-r--r--morphlib/exts/openstack.write.help51
-rwxr-xr-xmorphlib/exts/rawdisk.check23
-rwxr-xr-xmorphlib/exts/rawdisk.write92
-rw-r--r--morphlib/exts/rawdisk.write.help87
-rwxr-xr-xmorphlib/exts/set-hostname.configure5
-rwxr-xr-xmorphlib/exts/simple-network.configure151
-rwxr-xr-xmorphlib/exts/ssh-rsync.check5
-rwxr-xr-xmorphlib/exts/ssh-rsync.write15
-rw-r--r--morphlib/exts/ssh-rsync.write.help50
-rwxr-xr-xmorphlib/exts/sysroot.check29
-rwxr-xr-xmorphlib/exts/sysroot.write7
-rwxr-xr-xmorphlib/exts/tar.check5
-rwxr-xr-xmorphlib/exts/tar.write5
-rw-r--r--morphlib/exts/tar.write.help16
-rwxr-xr-xmorphlib/exts/vdaboot.configure5
-rwxr-xr-xmorphlib/exts/virtualbox-ssh.check5
-rwxr-xr-xmorphlib/exts/virtualbox-ssh.write50
-rw-r--r--morphlib/exts/virtualbox-ssh.write.help137
33 files changed, 847 insertions, 268 deletions
diff --git a/morphlib/exts/add-config-files.configure b/morphlib/exts/add-config-files.configure
index 0094cf6b..2cf96fd1 100755
--- a/morphlib/exts/add-config-files.configure
+++ b/morphlib/exts/add-config-files.configure
@@ -1,5 +1,5 @@
#!/bin/sh
-# Copyright (C) 2013 Codethink Limited
+# Copyright (C) 2013,2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
# Copy all files located in $SRC_CONFIG_DIR to the image /etc.
diff --git a/morphlib/exts/fstab.configure b/morphlib/exts/fstab.configure
index a1287ea4..3bbc9102 100755
--- a/morphlib/exts/fstab.configure
+++ b/morphlib/exts/fstab.configure
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2013 Codethink Limited
+# Copyright (C) 2013,2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
#
# =*= License: GPL-2 =*=
diff --git a/morphlib/exts/initramfs.write b/morphlib/exts/initramfs.write
index f8af6d84..1059defa 100755
--- a/morphlib/exts/initramfs.write
+++ b/morphlib/exts/initramfs.write
@@ -1,5 +1,5 @@
#!/bin/sh
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
#
# =*= License: GPL-2 =*=
diff --git a/morphlib/exts/initramfs.write.help b/morphlib/exts/initramfs.write.help
index 29a9d266..54d3ae8c 100644
--- a/morphlib/exts/initramfs.write.help
+++ b/morphlib/exts/initramfs.write.help
@@ -1,4 +1,19 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
help: |
+
Create an initramfs for a system by taking an existing system and
converting it to the appropriate format.
@@ -33,3 +48,8 @@ help: |
initramfs:
type: initramfs
location: boot/initramfs.gz
+
+ Parameters:
+
+ * location: the path where the initramfs will be installed (e.g.
+ `boot/initramfs.gz`) in the above example
diff --git a/morphlib/exts/install-files.configure b/morphlib/exts/install-files.configure
index 04dc5f18..58cf373a 100755
--- a/morphlib/exts/install-files.configure
+++ b/morphlib/exts/install-files.configure
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2013-2014 Codethink Limited
+# Copyright (C) 2013-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
''' A Morph configuration extension for adding arbitrary files to a system
diff --git a/morphlib/exts/install-files.configure.help b/morphlib/exts/install-files.configure.help
index eb3aab0c..991c26c8 100644
--- a/morphlib/exts/install-files.configure.help
+++ b/morphlib/exts/install-files.configure.help
@@ -1,3 +1,17 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
help: |
Install a set of files onto a system
diff --git a/morphlib/exts/kvm.check b/morphlib/exts/kvm.check
index 1bb4007a..62d76453 100755
--- a/morphlib/exts/kvm.check
+++ b/morphlib/exts/kvm.check
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,12 +11,12 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''Preparatory checks for Morph 'kvm' write extension'''
import cliapp
+import os
import re
import urlparse
@@ -43,8 +43,10 @@ class KvmPlusSshCheckExtension(morphlib.writeexts.WriteExtension):
ssh_host, vm_name, vm_path = self.check_and_parse_location(location)
self.check_ssh_connectivity(ssh_host)
+ self.check_can_create_file_at_given_path(ssh_host, vm_path)
self.check_no_existing_libvirt_vm(ssh_host, vm_name)
self.check_extra_disks_exist(ssh_host, self.parse_attach_disks())
+ self.check_virtual_networks_are_started(ssh_host)
def check_and_parse_location(self, location):
'''Check and parse the location argument to get relevant data.'''
@@ -73,6 +75,26 @@ class KvmPlusSshCheckExtension(morphlib.writeexts.WriteExtension):
'write extension to deploy upgrades to existing machines.' %
(ssh_host, vm_name))
+ def check_can_create_file_at_given_path(self, ssh_host, vm_path):
+
+ def check_can_write_to_given_path():
+ try:
+ cliapp.ssh_runcmd(ssh_host, ['touch', vm_path])
+ except cliapp.AppException as e:
+ raise cliapp.AppException("Can't write to location %s on %s"
+ % (vm_path, ssh_host))
+ else:
+ cliapp.ssh_runcmd(ssh_host, ['rm', vm_path])
+
+ try:
+ cliapp.ssh_runcmd(ssh_host, ['test', '-e', vm_path])
+ except cliapp.AppException as e:
+ # vm_path doesn't already exist, so let's test we can write
+ check_can_write_to_given_path()
+ else:
+ raise cliapp.AppException('%s already exists on %s'
+ % (vm_path, ssh_host))
+
def check_extra_disks_exist(self, ssh_host, filename_list):
for filename in filename_list:
try:
@@ -81,4 +103,50 @@ class KvmPlusSshCheckExtension(morphlib.writeexts.WriteExtension):
raise cliapp.AppException('Did not find file %s on host %s' %
(filename, ssh_host))
+ def check_virtual_networks_are_started(self, ssh_host):
+
+ def check_virtual_network_is_started(network_name):
+ cmd = ['virsh', '-c', 'qemu:///system', 'net-info', network_name]
+ net_info = cliapp.ssh_runcmd(ssh_host, cmd).split('\n')
+
+ def pretty_concat(lines):
+ return '\n'.join(['\t%s' % line for line in lines])
+
+ for line in net_info:
+ m = re.match('^Active:\W*(\w+)\W*', line)
+ if m:
+ break
+ else:
+ raise cliapp.AppException(
+ "Got unexpected output parsing output of `%s':\n%s"
+ % (' '.join(cmd), pretty_concat(net_info)))
+
+ network_active = m.group(1) == 'yes'
+
+ if not network_active:
+ raise cliapp.AppException("Network '%s' is not started"
+ % network_name)
+
+ def name(nic_entry):
+ if ',' in nic_entry:
+ # NETWORK_NAME,mac=12:34,model=e1000...
+ return nic_entry[:nic_entry.find(',')]
+ else:
+ return nic_entry # NETWORK_NAME
+
+ if 'NIC_CONFIG' in os.environ:
+ nics = os.environ['NIC_CONFIG'].split()
+
+ # --network bridge= is used to specify a bridge
+ # --network user is used to specify a form of NAT
+ # (see the virt-install(1) man page)
+ networks = [name(n) for n in nics if not n.startswith('bridge=')
+ and not n.startswith('user')]
+ else:
+ networks = ['default']
+
+ for network in networks:
+ check_virtual_network_is_started(network)
+
+
KvmPlusSshCheckExtension().run()
diff --git a/morphlib/exts/kvm.write b/morphlib/exts/kvm.write
index 16f188b5..0d0c095b 100755
--- a/morphlib/exts/kvm.write
+++ b/morphlib/exts/kvm.write
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2012-2014 Codethink Limited
+# Copyright (C) 2012-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,11 +11,14 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
-'''A Morph deployment write extension for deploying to KVM+libvirt.'''
+'''A Morph deployment write extension for deploying to KVM+libvirt.
+
+See file kvm.write.help for documentation
+
+'''
import cliapp
@@ -30,40 +33,20 @@ import morphlib.writeexts
class KvmPlusSshWriteExtension(morphlib.writeexts.WriteExtension):
- '''Create a KVM/LibVirt virtual machine during Morph's deployment.
-
- The location command line argument is the pathname of the disk image
- to be created. The user is expected to provide the location argument
- using the following syntax:
-
- kvm+ssh://HOST/GUEST/PATH
-
- where:
-
- * HOST is the host on which KVM/LibVirt is running
- * GUEST is the name of the guest virtual machine on that host
- * PATH is the path to the disk image that should be created,
- on that host
-
- The extension will connect to HOST via ssh to run libvirt's
- command line management tools.
-
- '''
-
location_pattern = '^/(?P<guest>[^/]+)(?P<path>/.+)$'
def process_args(self, args):
if len(args) != 2:
raise cliapp.AppException('Wrong number of command line args')
-
+
temp_root, location = args
ssh_host, vm_name, vm_path = self.parse_location(location)
autostart = self.get_environment_boolean('AUTOSTART')
-
+
fd, raw_disk = tempfile.mkstemp()
os.close(fd)
self.create_local_system(temp_root, raw_disk)
-
+
try:
self.transfer(raw_disk, ssh_host, vm_path)
self.create_libvirt_guest(ssh_host, vm_name, vm_path, autostart)
@@ -105,7 +88,7 @@ class KvmPlusSshWriteExtension(morphlib.writeexts.WriteExtension):
def create_libvirt_guest(self, ssh_host, vm_name, vm_path, autostart):
'''Create the libvirt virtual machine.'''
-
+
self.status(msg='Creating libvirt/kvm virtual machine')
attach_disks = self.parse_attach_disks()
@@ -135,4 +118,3 @@ class KvmPlusSshWriteExtension(morphlib.writeexts.WriteExtension):
['virsh', '--connect', 'qemu:///system', 'autostart', vm_name])
KvmPlusSshWriteExtension().run()
-
diff --git a/morphlib/exts/kvm.write.help b/morphlib/exts/kvm.write.help
index 8b5053a5..812a5309 100644
--- a/morphlib/exts/kvm.write.help
+++ b/morphlib/exts/kvm.write.help
@@ -1,4 +1,90 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
help: |
- The INITRAMFS_PATH option can be used to specify the location of an
- initramfs for syslinux to tell Linux to use, rather than booting
- the rootfs directly.
+
+ Deploy a Baserock system as a *new* KVM/LibVirt virtual machine.
+
+ Use the `ssh-rsync` write extension to deploy upgrades to an *existing* VM
+
+ Parameters:
+
+ * location: a custom URL scheme of the form `kvm+ssh://HOST/GUEST/PATH`,
+ where:
+ * HOST is the name of the host on which KVM/LibVirt is running
+ * GUEST is the name of the guest VM on that host
+ * PATH is the path to the disk image that should be created,
+ on that host. For example,
+ `kvm+ssh://alice@192.168.122.1/testsys/home/alice/testys.img` where
+ * `alice@192.168.122.1` is the target host as given to ssh,
+ **from within the development host** (which may be
+ different from the target host's normal address);
+ * `testsys` is the name of the new guest VM';
+ * `/home/alice/testys.img` is the pathname of the disk image files
+ on the target host.
+
+ * HOSTNAME=name: the hostname of the **guest** VM within the network into
+ which it is being deployed
+
+ * DISK_SIZE=X: the size of the VM's primary virtual hard disk. `X` should
+ use a suffix of `K`, `M`, or `G` (in upper or lower case) to indicate
+ kilo-, mega-, or gigabytes. For example, `DISK_SIZE=100G` would create a
+ 100 gigabyte disk image. **This parameter is mandatory**.
+
+ * RAM_SIZE=X: The amount of RAM that the virtual machine should allocate
+ for itself from the host. `X` is interpreted in the same was as for
+ DISK_SIZE`, and defaults to `1G`
+
+ * VCPUS=n: the number of virtual CPUs for the VM. Allowed values 1-32. Do
+ not use more CPU cores than you have available physically (real cores, no
+ hyperthreads)
+
+ * INITRAMFS_PATH=path: the location of an initramfs for the bootloader to
+ tell Linux to use, rather than booting the rootfs directly.
+
+ * AUTOSTART=<VALUE>` - boolean. If it is set, the VM will be started when
+ it has been deployed.
+
+ * DTB_PATH=path: **(MANDATORY)** for systems that require a device tree
+ binary - Give the full path (without a leading /) to the location of the
+ DTB in the built system image . The deployment will fail if `path` does
+ not exist.
+
+ * BOOTLOADER_INSTALL=value: the bootloader to be installed
+ **(MANDATORY)** for non-x86 systems
+
+ allowed values =
+ - 'extlinux' (default) - the extlinux bootloader will
+ be installed
+ - 'none' - no bootloader will be installed by `morph deploy`. A
+ bootloader must be installed manually. This value must be used when
+ deploying non-x86 systems such as ARM.
+
+ * BOOTLOADER_CONFIG_FORMAT=value: the bootloader format to be used.
+ If not specified for x86-32 and x86-64 systems, 'extlinux' will be used
+
+ allowed values =
+ - 'extlinux'
+
+ * KERNEL_ARGS=args: optional additional kernel command-line parameters to
+ be appended to the default set. The default set is:
+
+ 'rw init=/sbin/init rootfstype=btrfs \
+ rootflags=subvol=systems/default/run \
+ root=[name or UUID of root filesystem]'
+
+ (See https://www.kernel.org/doc/Documentation/kernel-parameters.txt)
+
+ (See `morph help deploy` for details of how to pass parameters to write
+ extensions)
diff --git a/morphlib/exts/nfsboot.check b/morphlib/exts/nfsboot.check
index 806e560a..e273f61c 100755
--- a/morphlib/exts/nfsboot.check
+++ b/morphlib/exts/nfsboot.check
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''Preparatory checks for Morph 'nfsboot' write extension'''
diff --git a/morphlib/exts/nfsboot.configure b/morphlib/exts/nfsboot.configure
index 660d9c39..6a68dc48 100755
--- a/morphlib/exts/nfsboot.configure
+++ b/morphlib/exts/nfsboot.configure
@@ -1,5 +1,5 @@
#!/bin/sh
-# Copyright (C) 2013-2014 Codethink Limited
+# Copyright (C) 2013-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
# Remove all networking interfaces. On nfsboot systems, eth0 is set up
diff --git a/morphlib/exts/nfsboot.write b/morphlib/exts/nfsboot.write
index 8d3d6df7..d928775e 100755
--- a/morphlib/exts/nfsboot.write
+++ b/morphlib/exts/nfsboot.write
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2013-2014 Codethink Limited
+# Copyright (C) 2013-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,12 +11,21 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''A Morph deployment write extension for deploying to an nfsboot server
+*** DO NOT USE ***
+- This was written before 'proper' deployment mechanisms were in place
+It is unlikely to work at all and will not work correctly
+
+Use the pxeboot write extension instead
+
+***
+
+
+
An nfsboot server is defined as a baserock system that has tftp and nfs
servers running, the tftp server is exporting the contents of
/srv/nfsboot/tftp/ and the user has sufficient permissions to create nfs roots
@@ -125,7 +134,7 @@ class NFSBootWriteExtension(morphlib.writeexts.WriteExtension):
self.status(msg='Creating destination directories')
try:
- cliapp.ssh_runcmd('root@%s' % location,
+ cliapp.ssh_runcmd('root@%s' % location,
['mkdir', '-p', orig_path, run_path])
except cliapp.AppException:
raise cliapp.AppException('Could not create dirs %s and %s on %s'
@@ -191,4 +200,3 @@ mv "$temp" "$target"
NFSBootWriteExtension().run()
-
diff --git a/morphlib/exts/nfsboot.write.help b/morphlib/exts/nfsboot.write.help
index 598b1b23..186c479a 100644
--- a/morphlib/exts/nfsboot.write.help
+++ b/morphlib/exts/nfsboot.write.help
@@ -1,6 +1,27 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
help: |
+ *** DO NOT USE ***
+ - This was written before 'proper' deployment mechanisms were in place.
+ It is unlikely to work at all, and will not work correctly.
+
+ Use the pxeboot write extension instead
+
+ ***
Deploy a system image and kernel to an nfsboot server.
-
+
An nfsboot server is defined as a baserock system that has
tftp and nfs servers running, the tftp server is exporting
the contents of /srv/nfsboot/tftp/ and the user has sufficient
diff --git a/morphlib/exts/openstack.check b/morphlib/exts/openstack.check
index edc37cc1..4c21b604 100755
--- a/morphlib/exts/openstack.check
+++ b/morphlib/exts/openstack.check
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''Preparatory checks for Morph 'openstack' write extension'''
@@ -77,9 +76,16 @@ class OpenStackCheckExtension(morphlib.writeexts.WriteExtension):
'--os-password', password,
'--os-auth-url', auth_url,
'image-list']
- try:
- cliapp.runcmd(cmdline)
- except cliapp.AppException:
- raise cliapp.AppException('Wrong OpenStack credentals.')
+
+ exit, out, err = cliapp.runcmd_unchecked(cmdline)
+
+ if exit != 0:
+ if err.startswith('The request you have made requires '
+ 'authentication. (HTTP 401)'):
+ raise cliapp.AppException('Invalid OpenStack credentials.')
+ else:
+ raise cliapp.AppException(
+ 'Failed to connect to OpenStack instance at %s: %s' %
+ (auth_url, err))
OpenStackCheckExtension().run()
diff --git a/morphlib/exts/openstack.write b/morphlib/exts/openstack.write
index 516fe367..67e07c18 100755
--- a/morphlib/exts/openstack.write
+++ b/morphlib/exts/openstack.write
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2013 - 2014 Codethink Limited
+# Copyright (C) 2013-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''A Morph deployment write extension for deploying to OpenStack.'''
@@ -28,40 +27,12 @@ import morphlib.writeexts
class OpenStackWriteExtension(morphlib.writeexts.WriteExtension):
- '''Configure a raw disk image into an OpenStack host.
-
- The raw disk image is created during Morph's deployment and the
- image is deployed in OpenStack using python-glanceclient.
-
- The location command line argument is the authentification url
- of the OpenStack server using the following syntax:
-
- http://HOST:PORT/VERSION
-
- where
-
- * HOST is the host running OpenStack
- * PORT is the port which is using OpenStack for authentifications.
- * VERSION is the authentification version of OpenStack (Only v2.0
- supported)
-
- This extension needs in the environment the following variables:
-
- * OPENSTACK_USER is the username to use in the deployment.
- * OPENSTACK_TENANT is the project name to use in the deployment.
- * OPENSTACK_IMAGENAME is the name of the image to create.
- * OPENSTACK_PASSWORD is the password of the user.
-
-
- The extension will connect to OpenStack using python-glanceclient
- to configure a raw image.
-
- '''
+ '''See openstack.write.help for documentation'''
def process_args(self, args):
if len(args) != 2:
raise cliapp.AppException('Wrong number of command line args')
-
+
temp_root, location = args
os_params = self.get_openstack_parameters()
@@ -69,7 +40,7 @@ class OpenStackWriteExtension(morphlib.writeexts.WriteExtension):
fd, raw_disk = tempfile.mkstemp()
os.close(fd)
self.create_local_system(temp_root, raw_disk)
- self.status(msg='Temporary disk image has been created at %s'
+ self.status(msg='Temporary disk image has been created at %s'
% raw_disk)
self.set_extlinux_root_to_virtio(raw_disk)
@@ -79,8 +50,7 @@ class OpenStackWriteExtension(morphlib.writeexts.WriteExtension):
def set_extlinux_root_to_virtio(self, raw_disk):
'''Re-configures extlinux to use virtio disks'''
self.status(msg='Updating extlinux.conf')
- mp = self.mount(raw_disk)
- try:
+ with self.mount(raw_disk) as mp:
path = os.path.join(mp, 'extlinux.conf')
with open(path) as f:
@@ -91,9 +61,6 @@ class OpenStackWriteExtension(morphlib.writeexts.WriteExtension):
with open(path, "w") as f:
f.write(extlinux_conf)
- finally:
- self.unmount(mp)
-
def get_openstack_parameters(self):
'''Get the environment variables needed.
@@ -124,4 +91,3 @@ class OpenStackWriteExtension(morphlib.writeexts.WriteExtension):
self.status(msg='Image configured.')
OpenStackWriteExtension().run()
-
diff --git a/morphlib/exts/openstack.write.help b/morphlib/exts/openstack.write.help
new file mode 100644
index 00000000..26983060
--- /dev/null
+++ b/morphlib/exts/openstack.write.help
@@ -0,0 +1,51 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
+help: |
+
+ Deploy a Baserock system as a *new* OpenStack virtual machine.
+ (Use the `ssh-rsync` write extension to deploy upgrades to an *existing*
+ VM)
+
+ Deploys the system to the OpenStack host using python-glanceclient.
+
+ Parameters:
+
+ * location: the authentication url of the OpenStack server using the
+ following syntax:
+
+ http://HOST:PORT/VERSION
+
+ where
+
+ * HOST is the host running OpenStack
+ * PORT is the port which is using OpenStack for authentications.
+ * VERSION is the authentication version of OpenStack (Only v2.0
+ supported)
+
+ * OPENSTACK_USER=username: the username to use in the `--os-username`
+ argument to `glance`.
+
+ * OPENSTACK_TENANT=tenant: the project name to use in the
+ `--os-tenant-name` argument to `glance`.
+
+ * OPENSTACK_IMAGENAME=imagename: the name of the image to use in the
+ `--name` argument to `glance`.
+
+ * OPENSTACK_PASSWORD=password: the password of the OpenStack user. (We
+ recommend passing this on the command-line, rather than setting an
+ environment variable or storing it in a cluster cluster definition file.)
+
+ (See `morph help deploy` for details of how to pass parameters to write
+ extensions)
diff --git a/morphlib/exts/rawdisk.check b/morphlib/exts/rawdisk.check
index acdc4de1..9be0ce91 100755
--- a/morphlib/exts/rawdisk.check
+++ b/morphlib/exts/rawdisk.check
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''Preparatory checks for Morph 'rawdisk' write extension'''
@@ -33,10 +32,11 @@ class RawdiskCheckExtension(morphlib.writeexts.WriteExtension):
location = args[0]
upgrade = self.get_environment_boolean('UPGRADE')
if upgrade:
- if not os.path.isfile(location):
- raise cliapp.AppException(
- 'Cannot upgrade %s: it is not an existing disk image' %
- location)
+ if not self.is_device(location):
+ if not os.path.isfile(location):
+ raise cliapp.AppException(
+ 'Cannot upgrade %s: it is not an existing disk image' %
+ location)
version_label = os.environ.get('VERSION_LABEL')
if version_label is None:
@@ -44,9 +44,10 @@ class RawdiskCheckExtension(morphlib.writeexts.WriteExtension):
'VERSION_LABEL was not given. It is required when '
'upgrading an existing system.')
else:
- if os.path.exists(location):
- raise cliapp.AppException(
- 'Target %s already exists. Use `morph upgrade` if you '
- 'want to update an existing image.' % location)
+ if not self.is_device(location):
+ if os.path.exists(location):
+ raise cliapp.AppException(
+ 'Target %s already exists. Use `morph upgrade` if you '
+ 'want to update an existing image.' % location)
RawdiskCheckExtension().run()
diff --git a/morphlib/exts/rawdisk.write b/morphlib/exts/rawdisk.write
index 12db4398..6f2d45ba 100755
--- a/morphlib/exts/rawdisk.write
+++ b/morphlib/exts/rawdisk.write
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2012-2014 Codethink Limited
+# Copyright (C) 2012-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''A Morph deployment write extension for raw disk images.'''
@@ -29,81 +28,77 @@ import morphlib.writeexts
class RawDiskWriteExtension(morphlib.writeexts.WriteExtension):
- '''Create a raw disk image during Morph's deployment.
-
- If the image already exists, it is upgraded.
-
- The location command line argument is the pathname of the disk image
- to be created/upgraded.
-
- '''
+ '''See rawdisk.write.help for documentation'''
def process_args(self, args):
if len(args) != 2:
raise cliapp.AppException('Wrong number of command line args')
-
+
temp_root, location = args
- if os.path.isfile(location):
+ upgrade = self.get_environment_boolean('UPGRADE')
+
+ if upgrade:
self.upgrade_local_system(location, temp_root)
else:
try:
- self.create_local_system(temp_root, location)
- self.status(msg='Disk image has been created at %s' % location)
+ if not self.is_device(location):
+ with self.created_disk_image(location):
+ self.format_btrfs(location)
+ self.create_system(temp_root, location)
+ self.status(msg='Disk image has been created at %s' %
+ location)
+ else:
+ self.format_btrfs(location)
+ self.create_system(temp_root, location)
+ self.status(msg='System deployed to %s' % location)
except Exception:
- self.status(msg='Failure to create disk image at %s' %
+ self.status(msg='Failure to deploy system to %s' %
location)
- if os.path.exists(location):
- os.remove(location)
raise
def upgrade_local_system(self, raw_disk, temp_root):
self.complete_fstab_for_btrfs_layout(temp_root)
- mp = self.mount(raw_disk)
+ with self.mount(raw_disk) as mp:
+ version_label = self.get_version_label(mp)
+ self.status(msg='Updating image to a new version with label %s' %
+ version_label)
- version_label = self.get_version_label(mp)
- self.status(msg='Updating image to a new version with label %s' %
- version_label)
+ version_root = os.path.join(mp, 'systems', version_label)
+ os.mkdir(version_root)
- version_root = os.path.join(mp, 'systems', version_label)
- os.mkdir(version_root)
+ old_orig = os.path.join(mp, 'systems', 'default', 'orig')
+ new_orig = os.path.join(version_root, 'orig')
+ cliapp.runcmd(
+ ['btrfs', 'subvolume', 'snapshot', old_orig, new_orig])
- old_orig = os.path.join(mp, 'systems', 'default', 'orig')
- new_orig = os.path.join(version_root, 'orig')
- cliapp.runcmd(
- ['btrfs', 'subvolume', 'snapshot', old_orig, new_orig])
+ cliapp.runcmd(
+ ['rsync', '-a', '--checksum', '--numeric-ids', '--delete',
+ temp_root + os.path.sep, new_orig])
- cliapp.runcmd(
- ['rsync', '-a', '--checksum', '--numeric-ids', '--delete',
- temp_root + os.path.sep, new_orig])
+ self.create_run(version_root)
- self.create_run(version_root)
+ default_path = os.path.join(mp, 'systems', 'default')
+ if os.path.exists(default_path):
+ os.remove(default_path)
+ else:
+ # we are upgrading and old system that does
+ # not have an updated extlinux config file
+ if self.bootloader_config_is_wanted():
+ self.generate_bootloader_config(mp)
+ self.install_bootloader(mp)
+ os.symlink(version_label, default_path)
- default_path = os.path.join(mp, 'systems', 'default')
- if os.path.exists(default_path):
- os.remove(default_path)
- else:
- # we are upgrading and old system that does
- # not have an updated extlinux config file
if self.bootloader_config_is_wanted():
- self.generate_bootloader_config(mp)
- self.install_bootloader(mp)
- os.symlink(version_label, default_path)
-
- if self.bootloader_config_is_wanted():
- self.install_kernel(version_root, temp_root)
-
- self.unmount(mp)
+ self.install_kernel(version_root, temp_root)
def get_version_label(self, mp):
version_label = os.environ.get('VERSION_LABEL')
if version_label is None:
- self.unmount(mp)
raise cliapp.AppException('VERSION_LABEL was not given')
if os.path.exists(os.path.join(mp, 'systems', version_label)):
- self.unmount(mp)
raise cliapp.AppException('VERSION_LABEL %s already exists'
% version_label)
@@ -111,4 +106,3 @@ class RawDiskWriteExtension(morphlib.writeexts.WriteExtension):
RawDiskWriteExtension().run()
-
diff --git a/morphlib/exts/rawdisk.write.help b/morphlib/exts/rawdisk.write.help
index 298d441c..52ed73fb 100644
--- a/morphlib/exts/rawdisk.write.help
+++ b/morphlib/exts/rawdisk.write.help
@@ -1,11 +1,82 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
help: |
- Create a raw disk image during Morph's deployment.
-
- If the image already exists, it is upgraded.
- The `location` argument is a pathname to the image to be
- created or upgraded.
+ Write a system produced by Morph to a physical disk, or to a file that can
+ be used as a virtual disk. The target will be formatted as a single Btrfs
+ partition, with the system image written to a subvolume in /systems, and
+ other subvolumes created for /home, /opt, /root, /srv and /var.
+
+ When written to a physical drive, the drive can be used as the boot device
+ for a 'real' machine.
+
+ When written to a file, the file can be used independently of `morph` to
+ create virtual machines with KVM / libvirt, OpenStack or, after converting
+ it to VDI format, VirtualBox.
+
+ `morph deploy` will fail if the file specified by `location` already
+ exists.
+
+ If used in `morph upgrade`, the rootfs produced by 'morph build' is added
+ to the existing raw disk image or device as an additional btrfs sub-volume.
+ `morph upgrade` will fail if the file specified by `location` does not
+ exist, or is not a Baserock raw disk image. (Most users are unlikely to
+ need or use this functionality: it is useful mainly for developers working
+ on the Baserock tools.)
+
+ Parameters:
+
+ * location: the pathname of the disk image to be created/upgraded, or the
+ path to the physical device.
+
+ * VERSION_LABEL=label - should contain only alpha-numeric
+ characters and the '-' (hyphen) character. Mandatory if being used with
+ `morph update`
+
+ * INITRAMFS_PATH=path: the location of an initramfs for the bootloader to
+ tell Linux to use, rather than booting the rootfs directly.
+
+ * DTB_PATH=path: **(MANDATORY)** for systems that require a device tree
+ binary - Give the full path (without a leading /) to the location of the
+ DTB in the built system image . The deployment will fail if `path` does
+ not exist.
+
+ * BOOTLOADER_INSTALL=value: the bootloader to be installed
+ **(MANDATORY)** for non-x86 systems
+
+ allowed values =
+ - 'extlinux' (default) - the extlinux bootloader will
+ be installed
+ - 'none' - no bootloader will be installed by `morph deploy`. A
+ bootloader must be installed manually. This value must be used when
+ deploying non-x86 systems such as ARM.
+
+ * BOOTLOADER_CONFIG_FORMAT=value: the bootloader format to be used.
+ If not specified for x86-32 and x86-64 systems, 'extlinux' will be used
+
+ allowed values =
+ - 'extlinux'
+
+ * KERNEL_ARGS=args: optional additional kernel command-line parameters to
+ be appended to the default set. The default set is:
+
+ 'rw init=/sbin/init rootfstype=btrfs \
+ rootflags=subvol=systems/default/run \
+ root=[name or UUID of root filesystem]'
+
+ (See https://www.kernel.org/doc/Documentation/kernel-parameters.txt)
- The INITRAMFS_PATH option can be used to specify the location of an
- initramfs for syslinux to tell Linux to use, rather than booting
- the rootfs directly.
+ (See `morph help deploy` for details of how to pass parameters to write
+ extensions)
diff --git a/morphlib/exts/set-hostname.configure b/morphlib/exts/set-hostname.configure
index e44c5d56..4b2424d8 100755
--- a/morphlib/exts/set-hostname.configure
+++ b/morphlib/exts/set-hostname.configure
@@ -1,5 +1,5 @@
#!/bin/sh
-# Copyright (C) 2013 Codethink Limited
+# Copyright (C) 2013,2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
# Set hostname on system from HOSTNAME.
diff --git a/morphlib/exts/simple-network.configure b/morphlib/exts/simple-network.configure
index b98b202c..61113325 100755
--- a/morphlib/exts/simple-network.configure
+++ b/morphlib/exts/simple-network.configure
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2013 Codethink Limited
+# Copyright (C) 2013,2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,16 +11,17 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
-'''A Morph deployment configuration extension to handle /etc/network/interfaces
+'''A Morph deployment configuration extension to handle network configutation
-This extension prepares /etc/network/interfaces with the interfaces specified
-during deployment.
+This extension prepares /etc/network/interfaces and networkd .network files
+in /etc/systemd/network/ with the interfaces specified during deployment.
If no network configuration is provided, eth0 will be configured for DHCP
-with the hostname of the system.
+with the hostname of the system in the case of /etc/network/interfaces.
+In the case of networkd, any interface starting by e* will be configured
+for DHCP
'''
@@ -37,20 +38,74 @@ class SimpleNetworkError(morphlib.Error):
class SimpleNetworkConfigurationExtension(cliapp.Application):
- '''Configure /etc/network/interfaces
+ '''Configure /etc/network/interfaces and generate networkd .network files
- Reading NETWORK_CONFIG, this extension sets up /etc/network/interfaces.
+ Reading NETWORK_CONFIG, this extension sets up /etc/network/interfaces
+ and .network files in /etc/systemd/network/.
'''
def process_args(self, args):
- network_config = os.environ.get(
- "NETWORK_CONFIG", "lo:loopback;eth0:dhcp,hostname=$(hostname)")
+ network_config = os.environ.get("NETWORK_CONFIG")
- self.status(msg="Processing NETWORK_CONFIG=%(nc)s", nc=network_config)
+ self.rename_networkd_chunk_file(args)
- stanzas = self.parse_network_stanzas(network_config)
- iface_file = self.generate_iface_file(stanzas)
+ if network_config is None:
+ self.generate_default_network_config(args)
+ else:
+ self.status(msg="Processing NETWORK_CONFIG=%(nc)s",
+ nc=network_config)
+
+ stanzas = self.parse_network_stanzas(network_config)
+
+ self.generate_interfaces_file(args, stanzas)
+ self.generate_networkd_files(args, stanzas)
+
+ def rename_networkd_chunk_file(self, args):
+ """Rename the 10-dchp.network file generated in the systemd chunk
+ The systemd chunk will place something in 10-dhcp.network, which will
+ have higher precedence than anything added in this extension (we
+ start at 50-*).
+
+ We should check for that file and rename it instead remove it in
+ case the file is being used by the user.
+
+ Until both the following happen, we should continue to rename that
+ default config file:
+
+ 1. simple-network.configure is always run when systemd is included
+ 2. We've been building systems without systemd including that default
+ networkd config for long enough that nobody should be including
+ that config file.
+ """
+ file_path = os.path.join(args[0], "etc", "systemd", "network",
+ "10-dhcp.network")
+ try:
+ os.rename(file_path, file_path + ".morph")
+ self.status(msg="Renaming networkd file from systemd chunk: %(f)s \
+ to %(f)s.morph", f=file_path)
+ except OSError:
+ pass
+
+ def generate_default_network_config(self, args):
+ """Generate default network config: DHCP in all the interfaces"""
+
+ default_network_config_interfaces = "lo:loopback;" \
+ "eth0:dhcp,hostname=$(hostname)"
+ default_network_config_networkd = "e*:dhcp"
+
+ stanzas_interfaces = self.parse_network_stanzas(
+ default_network_config_interfaces)
+ stanzas_networkd = self.parse_network_stanzas(
+ default_network_config_networkd)
+
+ self.generate_interfaces_file(args, stanzas_interfaces)
+ self.generate_networkd_files(args, stanzas_networkd)
+
+ def generate_interfaces_file(self, args, stanzas):
+ """Generate /etc/network/interfaces file"""
+
+ iface_file = self.generate_iface_file(stanzas)
with open(os.path.join(args[0], "etc/network/interfaces"), "w") as f:
f.write(iface_file)
@@ -83,6 +138,74 @@ class SimpleNetworkConfigurationExtension(cliapp.Application):
lines += [""]
return "\n".join(lines)
+ def generate_networkd_files(self, args, stanzas):
+ """Generate .network files"""
+
+ for i, stanza in enumerate(stanzas, 50):
+ iface_file = self.generate_networkd_file(stanza)
+
+ if iface_file is None:
+ continue
+
+ path = os.path.join(args[0], "etc", "systemd", "network",
+ "%s-%s.network" % (i, stanza['name']))
+
+ with open(path, "w") as f:
+ f.write(iface_file)
+
+ def generate_networkd_file(self, stanza):
+ """Generate an .network file from the provided data."""
+
+ name = stanza['name']
+ itype = stanza['type']
+ pairs = stanza['args'].items()
+
+ if itype == "loopback":
+ return
+
+ lines = ["[Match]"]
+ lines += ["Name=%s\n" % name]
+ lines += ["[Network]"]
+ if itype == "dhcp":
+ lines += ["DHCP=yes"]
+ else:
+ lines += self.generate_networkd_entries(pairs)
+
+ return "\n".join(lines)
+
+ def generate_networkd_entries(self, pairs):
+ """Generate networkd configuration entries with the other parameters"""
+
+ address = None
+ netmask = None
+ gateway = None
+ lines = []
+ for pair in pairs:
+ if pair[0] == 'address':
+ address = pair[1]
+ elif pair[0] == 'netmask':
+ netmask = pair[1]
+ elif pair[0] == 'gateway':
+ gateway = pair[1]
+
+ if address and netmask:
+ network_suffix = self.convert_net_mask_to_cidr_suffix (netmask);
+ address_line = address + '/' + str(network_suffix)
+ lines += ["Address=%s" % address_line]
+ elif address or netmask:
+ raise Exception('address and netmask must be specified together')
+
+ if gateway is not None:
+ lines += ["Gateway=%s" % gateway]
+
+ return lines
+
+ def convert_net_mask_to_cidr_suffix(self, mask):
+ """Convert dotted decimal form of a subnet mask to CIDR suffix notation
+
+ For example: 255.255.255.0 -> 24
+ """
+ return sum(bin(int(x)).count('1') for x in mask.split('.'))
def parse_network_stanzas(self, config):
"""Parse a network config environment variable into stanzas.
diff --git a/morphlib/exts/ssh-rsync.check b/morphlib/exts/ssh-rsync.check
index 11446c28..c3bdfd29 100755
--- a/morphlib/exts/ssh-rsync.check
+++ b/morphlib/exts/ssh-rsync.check
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''Preparatory checks for Morph 'ssh-rsync' write extension'''
diff --git a/morphlib/exts/ssh-rsync.write b/morphlib/exts/ssh-rsync.write
index 2d7258ba..6d596500 100755
--- a/morphlib/exts/ssh-rsync.write
+++ b/morphlib/exts/ssh-rsync.write
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2013-2014 Codethink Limited
+# Copyright (C) 2013-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''A Morph deployment write extension for upgrading systems over ssh.'''
@@ -37,14 +36,8 @@ def ssh_runcmd_ignore_failure(location, command, **kwargs):
class SshRsyncWriteExtension(morphlib.writeexts.WriteExtension):
- '''Upgrade a running baserock system with ssh and rsync.
-
- It assumes the system is baserock-based and has a btrfs partition.
-
- The location command line argument is the 'user@hostname' string
- that will be passed to ssh and rsync
-
- '''
+ '''See ssh-rsync.write.help for documentation'''
+
def find_root_disk(self, location):
'''Read /proc/mounts on location to find which device contains "/"'''
diff --git a/morphlib/exts/ssh-rsync.write.help b/morphlib/exts/ssh-rsync.write.help
new file mode 100644
index 00000000..f3f79ed5
--- /dev/null
+++ b/morphlib/exts/ssh-rsync.write.help
@@ -0,0 +1,50 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
+help: |
+
+ Upgrade a Baserock system which is already deployed:
+ - as a KVM/LibVirt, OpenStack or vbox-ssh virtual machine;
+ - on a Jetson board.
+
+ Copies a binary delta over to the target system and arranges for it
+ to be bootable.
+
+ The recommended way to use this extension is by calling `morph upgrade`.
+ Using `morph deploy --upgrade` is deprecated.
+
+ The upgrade will fail if:
+ - no VM is deployed and running at `location`;
+ - the target system is not a Baserock system;
+ - the target's filesystem and its layout are not compatible with that
+ created by `morph deploy`."
+
+ See also the 'Upgrading a Baserock installation' section of the 'Using
+ Baserock` page at wiki.baserock.org
+ http://wiki.baserock.org/devel-with/#index8h2
+
+ Parameters:
+
+ * location: the 'user@hostname' string that will be used by ssh and rsync.
+ 'user' will always be `root` and `hostname` the hostname or address of
+ the system being upgraded.
+
+ * VERSION_LABEL=label - **(MANDATORY)** should contain only alpha-numeric
+ characters and the '-' (hyphen) character.
+
+ * AUTOSTART=<VALUE>` - boolean. If it is set, the VM will be started when
+ it has been deployed.
+
+ (See `morph help deploy` for details of how to pass parameters to write
+ extensions)
diff --git a/morphlib/exts/sysroot.check b/morphlib/exts/sysroot.check
new file mode 100755
index 00000000..8ed965bd
--- /dev/null
+++ b/morphlib/exts/sysroot.check
@@ -0,0 +1,29 @@
+#!/bin/sh
+# Copyright (C) 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Preparatory checks for Morph 'sysroot' write extension
+
+set -eu
+
+location="$1"
+if [ -d "$location" ]; then
+ echo >&2 "ERROR: Deployment directory already exists: $location"
+ exit 1
+fi
+
+if [ "$UPGRADE" == "yes" ]; then
+ echo >&2 "ERROR: Cannot upgrade a sysroot deployment"
+ exit 1
+fi
diff --git a/morphlib/exts/sysroot.write b/morphlib/exts/sysroot.write
index 1ae4864f..0ad8d630 100755
--- a/morphlib/exts/sysroot.write
+++ b/morphlib/exts/sysroot.write
@@ -1,5 +1,5 @@
#!/bin/sh
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014,2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,16 +11,13 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
# A Morph write extension to deploy to another directory
set -eu
-# Ensure the target is an empty directory
mkdir -p "$2"
-find "$2" -mindepth 1 -delete
# Move the contents of our source directory to our target
# Previously we would (cd "$1" && find -print0 | cpio -0pumd "$absolute_path")
diff --git a/morphlib/exts/tar.check b/morphlib/exts/tar.check
index cbeaf163..f2304d46 100755
--- a/morphlib/exts/tar.check
+++ b/morphlib/exts/tar.check
@@ -1,5 +1,5 @@
#!/bin/sh
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
# Preparatory checks for Morph 'tar' write extension
diff --git a/morphlib/exts/tar.write b/morphlib/exts/tar.write
index 333626b5..01b545b4 100755
--- a/morphlib/exts/tar.write
+++ b/morphlib/exts/tar.write
@@ -1,5 +1,5 @@
#!/bin/sh
-# Copyright (C) 2013 Codethink Limited
+# Copyright (C) 2013,2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
# A Morph write extension to deploy to a .tar file
diff --git a/morphlib/exts/tar.write.help b/morphlib/exts/tar.write.help
index f052ac03..b45c61fa 100644
--- a/morphlib/exts/tar.write.help
+++ b/morphlib/exts/tar.write.help
@@ -1,5 +1,19 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
help: |
Create a .tar file of the deployed system.
-
+
The `location` argument is a pathname to the .tar file to be
created.
diff --git a/morphlib/exts/vdaboot.configure b/morphlib/exts/vdaboot.configure
index b88eb3a8..60de925b 100755
--- a/morphlib/exts/vdaboot.configure
+++ b/morphlib/exts/vdaboot.configure
@@ -1,5 +1,5 @@
#!/bin/sh
-# Copyright (C) 2013 Codethink Limited
+# Copyright (C) 2013,2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
# Change the "/" mount point to /dev/vda to use virtio disks.
diff --git a/morphlib/exts/virtualbox-ssh.check b/morphlib/exts/virtualbox-ssh.check
index 57d54db1..a97f3294 100755
--- a/morphlib/exts/virtualbox-ssh.check
+++ b/morphlib/exts/virtualbox-ssh.check
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2014 Codethink Limited
+# Copyright (C) 2014-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''Preparatory checks for Morph 'virtualbox-ssh' write extension'''
diff --git a/morphlib/exts/virtualbox-ssh.write b/morphlib/exts/virtualbox-ssh.write
index 39ea8f86..774f2b4f 100755
--- a/morphlib/exts/virtualbox-ssh.write
+++ b/morphlib/exts/virtualbox-ssh.write
@@ -1,5 +1,5 @@
#!/usr/bin/python
-# Copyright (C) 2012-2014 Codethink Limited
+# Copyright (C) 2012-2015 Codethink Limited
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -11,8 +11,7 @@
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
-# with this program; if not, write to the Free Software Foundation, Inc.,
-# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+# with this program. If not, see <http://www.gnu.org/licenses/>.
'''A Morph deployment write extension for deploying to VirtualBox via ssh.
@@ -20,6 +19,8 @@
VirtualBox is assumed to be running on a remote machine, which is
accessed over ssh. The machine gets created, but not started.
+See file virtualbox-ssh.write.help for documentation
+
'''
@@ -36,30 +37,10 @@ import morphlib.writeexts
class VirtualBoxPlusSshWriteExtension(morphlib.writeexts.WriteExtension):
- '''Create a VirtualBox virtual machine during Morph's deployment.
-
- The location command line argument is the pathname of the disk image
- to be created. The user is expected to provide the location argument
- using the following syntax:
-
- vbox+ssh://HOST/GUEST/PATH
-
- where:
-
- * HOST is the host on which VirtualBox is running
- * GUEST is the name of the guest virtual machine on that host
- * PATH is the path to the disk image that should be created,
- on that host
-
- The extension will connect to HOST via ssh to run VirtualBox's
- command line management tools.
-
- '''
-
def process_args(self, args):
if len(args) != 2:
raise cliapp.AppException('Wrong number of command line args')
-
+
temp_root, location = args
ssh_host, vm_name, vdi_path = self.parse_location(location)
autostart = self.get_environment_boolean('AUTOSTART')
@@ -88,7 +69,7 @@ class VirtualBoxPlusSshWriteExtension(morphlib.writeexts.WriteExtension):
def parse_location(self, location):
'''Parse the location argument to get relevant data.'''
-
+
x = urlparse.urlparse(location)
if x.scheme != 'vbox+ssh':
raise cliapp.AppException(
@@ -169,11 +150,11 @@ class VirtualBoxPlusSshWriteExtension(morphlib.writeexts.WriteExtension):
attach_disks = self.parse_attach_disks()
for device_no, disk in enumerate(attach_disks, 1):
- cmd = ['storageattach', vm_name,
+ cmd = ['storageattach', vm_name,
'--storagectl', 'SATA Controller',
'--port', str(device_no),
'--device', '0',
- '--type', 'hdd',
+ '--type', 'hdd',
'--medium', disk]
commands.append(cmd)
@@ -187,20 +168,6 @@ class VirtualBoxPlusSshWriteExtension(morphlib.writeexts.WriteExtension):
def get_host_interface(self, ssh_host):
host_ipaddr = os.environ.get('HOST_IPADDR')
netmask = os.environ.get('NETMASK')
- network_config = os.environ.get("NETWORK_CONFIG")
-
- if network_config is None:
- raise cliapp.AppException('NETWORK_CONFIG was not given')
-
- if "eth0:" not in network_config:
- raise cliapp.AppException(
- 'NETWORK_CONFIG does not contain '
- 'the eth0 configuration')
-
- if "eth1:" not in network_config:
- raise cliapp.AppException(
- 'NETWORK_CONFIG does not contain '
- 'the eth1 configuration')
if host_ipaddr is None:
raise cliapp.AppException('HOST_IPADDR was not given')
@@ -242,4 +209,3 @@ class VirtualBoxPlusSshWriteExtension(morphlib.writeexts.WriteExtension):
return iface
VirtualBoxPlusSshWriteExtension().run()
-
diff --git a/morphlib/exts/virtualbox-ssh.write.help b/morphlib/exts/virtualbox-ssh.write.help
index 8b5053a5..2dbf988c 100644
--- a/morphlib/exts/virtualbox-ssh.write.help
+++ b/morphlib/exts/virtualbox-ssh.write.help
@@ -1,4 +1,135 @@
+# Copyright (C) 2014, 2015 Codethink Limited
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, see <http://www.gnu.org/licenses/>.
+
help: |
- The INITRAMFS_PATH option can be used to specify the location of an
- initramfs for syslinux to tell Linux to use, rather than booting
- the rootfs directly.
+
+ Deploy a Baserock system as a *new* VirtualBox virtual machine.
+ (Use the `ssh-rsync` write extension to deploy upgrades to an *existing*
+ VM)
+
+ Connects to HOST via ssh to run VirtualBox's command line management tools.
+
+ Parameters:
+
+ * location: a custom URL scheme of the form `vbox+ssh://HOST/GUEST/PATH`,
+ where:
+ * HOST is the name of the host on which VirtualBox is running
+ * GUEST is the name of the guest VM on that host
+ * PATH is the path to the disk image that should be created,
+ on that host. For example,
+ `vbox+ssh://alice@192.168.122.1/testsys/home/alice/testys.img` where
+ * `alice@192.168.122.1` is the target host as given to ssh,
+ **from within the development host** (which may be
+ different from the target host's normal address);
+ * `testsys` is the name of the new guest VM';
+ * `/home/alice/testys.img` is the pathname of the disk image files
+ on the target host.
+
+ * HOSTNAME=name: the hostname of the **guest** VM within the network into
+ which it is being deployed.
+
+ * DISK_SIZE=X: **(MANDATORY)** the size of the VM's primary virtual hard
+ disk. `X` should use a suffix of `K`, `M`, or `G` (in upper or lower
+ case) to indicate kilo-, mega-, or gigabytes. For example,
+ `DISK_SIZE=100G` would create a 100 gigabyte virtual hard disk.
+
+ * RAM_SIZE=X: The amount of RAM that the virtual machine should allocate
+ for itself from the host. `X` is interpreted in the same as for
+ DISK_SIZE, and defaults to `1G`.
+
+ * VCPUS=n: the number of virtual CPUs for the VM. Allowed values 1-32. Do
+ not use more CPU cores than you have available physically (real cores,
+ no hyperthreads).
+
+ * INITRAMFS_PATH=path: the location of an initramfs for the bootloader to
+ tell Linux to use, rather than booting the rootfs directly.
+
+ * DTB_PATH=path: **(MANDATORY)** for systems that require a device tree
+ binary - Give the full path (without a leading /) to the location of the
+ DTB in the built system image . The deployment will fail if `path` does
+ not exist.
+
+ * BOOTLOADER_INSTALL=value: the bootloader to be installed
+ **(MANDATORY)** for non-x86 systems
+
+ allowed values =
+ - 'extlinux' (default) - the extlinux bootloader will
+ be installed
+ - 'none' - no bootloader will be installed by `morph deploy`. A
+ bootloader must be installed manually. This value must be used when
+ deploying non-x86 systems such as ARM.
+
+ * BOOTLOADER_CONFIG_FORMAT=value: the bootloader format to be used.
+ If not specified for x86-32 and x86-64 systems, 'extlinux' will be used
+
+ allowed values =
+ - 'extlinux'
+
+ * KERNEL_ARGS=args: optional additional kernel command-line parameters to
+ be appended to the default set. The default set is:
+
+ 'rw init=/sbin/init rootfstype=btrfs \
+ rootflags=subvol=systems/default/run \
+ root=[name or UUID of root filesystem]'
+
+ (See https://www.kernel.org/doc/Documentation/kernel-parameters.txt)
+
+ * AUTOSTART=<VALUE> - boolean. If it is set, the VM will be started when
+ it has been deployed.
+
+ * VAGRANT=<VALUE> - boolean. If it is set, then networking is configured
+ so that the VM will work with Vagrant. Otherwise networking is
+ configured to run directly in VirtualBox.
+
+ * HOST_IPADDR=<ip_address> - the IP address of the VM host.
+
+ * NETMASK=<netmask> - the netmask of the VM host.
+
+ * NETWORK_CONFIG=<net_config> - `net_config` is used to set up the VM's
+ network interfaces. It is a string containing semi-colon separated
+ 'stanzas' where each stanza provides information about a network
+ interface. Each stanza is of the form name:type[,arg=value] e.g.
+
+ lo:loopback
+ eth0:dhcp
+ eth1:static,address=10.0.0.1,netmask=255.255.0.0
+
+ An example of the NETWORK_CONFIG parameter (It should be in one line)
+
+ `"lo:loopback;eth0:static,address=192.168.100.2,netmask=255.255.255.0;
+ eth1:dhcp,hostname=$(hostname)"`
+
+ It is useful to configure one interface to use NAT to give the VM access
+ to the outside world and another interface to use the Virtual Box host
+ adapter to allow you to access the Trove from the host machine.
+
+ The NAT interface eth1 is set up to use dhcp, the host-only adapter
+ interface is configured statically.
+
+ Note: you must give the host-only adapter interface an address that lies
+ **on the same network** as the host adapter. So if the host adapter has
+ an IP of 192.168.100.1 eth0 should have an address such as
+ 192.168.100.42.
+
+ The settings of the host adapter, including its IP can be changed either
+ in the VirtualBox manager UI
+ (https://www.virtualbox.org/manual/ch03.html#settings-network)
+ or via the VBoxManage command line
+ (https://www.virtualbox.org/manual/ch08.html#idp57572192)
+
+ See Chapter 6 of the VirtualBox User Manual for more information about
+ virtual networking (https://www.virtualbox.org/manual/ch06.html)
+
+ (See `morph help deploy` for details of how to pass parameters to write
+ extensions)