1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
|
Morph Deployment Tests
======================
SCENARIO deploying a non-cluster morphology
GIVEN a workspace
AND a git server
WHEN the user checks out the system branch called master
AND the user attempts to deploy the system systems/test-system.morph in branch master
THEN morph failed
AND the deploy error message includes the string "morph deploy is only supported for cluster morphologies"
SCENARIO deploying a cluster morphology as a tarfile
GIVEN a workspace
AND a git server
WHEN the user checks out the system branch called master
GIVEN a cluster called test-cluster.morph in system branch master
AND a system in cluster test-cluster.morph in branch master called test-system
AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph
AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar
AND system test-system in cluster test-cluster.morph in branch master has deployment location: test.tar
WHEN the user builds the system systems/test-system.morph in branch master
AND the user attempts to deploy the cluster test-cluster.morph in branch master
THEN morph succeeded
Some deployment types support upgrades, but some do not and Morph needs to make
this clear.
SCENARIO attempting to upgrade a tarfile deployment
GIVEN a workspace
AND a git server
WHEN the user checks out the system branch called master
GIVEN a cluster called test-cluster.morph in system branch master
AND a system in cluster test-cluster.morph in branch master called test-system
AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph
AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar
AND system test-system in cluster test-cluster.morph in branch master has deployment location: test.tar
WHEN the user builds the system systems/test-system.morph in branch master
AND the user attempts to upgrade the cluster test-cluster.morph in branch master
THEN morph failed
The rawdisk write extension supports both initial deployment and subsequent
upgrades. Note that the rawdisk upgrade code needs bringing up to date to use
the new Baserock OS version manager tool. Also, the test deploys an identical
base OS as an upgrade. While pointless, this is permitted and does exercise
the same code paths as a real upgrade.
SCENARIO deploying a cluster morphology as rawdisk and then upgrading it
GIVEN a workspace
AND a git server
WHEN the user checks out the system branch called master
GIVEN a cluster called test-cluster.morph in system branch master
AND a system in cluster test-cluster.morph in branch master called test-system
AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph
AND system test-system in cluster test-cluster.morph in branch master has deployment type: rawdisk
AND system test-system in cluster test-cluster.morph in branch master has deployment location: test.tar
WHEN the user builds the system systems/test-system.morph in branch master
AND the user attempts to deploy the cluster test-cluster.morph in branch master with options test-system.DISK_SIZE=20M test-system.VERSION_LABEL=test1
THEN morph succeeded
WHEN the user attempts to upgrade the cluster test-cluster.morph in branch master with options test-system.VERSION_LABEL=test2
THEN morph succeeded
Nested deployments
==================
For the use-cases of:
1. Installer CD/USB
2. NFS/VM host
3. System with multiple containerised applications
4. System with a toolchain targetting the sysroot of another target
5. Any nested combination of the above
It is convenient to be able to deploy one system inside another.
SCENARIO deploying a cluster morphology with nested systems
GIVEN a workspace
AND a git server
WHEN the user checks out the system branch called master
GIVEN a cluster called test-cluster.morph in system branch master
AND a system in cluster test-cluster.morph in branch master called test-system
AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph
AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar
After the usual setup, we also add a subsystem to the cluster.
GIVEN a subsystem in cluster test-cluster.morph in branch master called test-system.sysroot
AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master builds systems/test-system.morph
AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment type: sysroot
We specify the location as a file path, this is relative to the parent
system's extracted rootfs, before it is configured.
AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment location: var/lib/sysroots/test-system
WHEN the user builds the system systems/test-system.morph in branch master
AND the user attempts to deploy the cluster test-cluster.morph in branch master with options test-system.location="$DATADIR/test.tar"
THEN morph succeeded
Morph succeeding alone is not sufficient to check whether it actually
worked, since if it ignored the subsystems field, or got the location
wrong for the subsystem. To actually test it, we have to check that our
deployed system contains the other. Since the baserock directory is in
every system, we can check for that.
AND tarball test.tar contains var/lib/sysroots/test-system/baserock
Initramfs deployments
=====================
There's a few ways of creating an initramfs. We could:
1. Build a sysroot and:
1. Have a chunk turn that into a cpio archive, written into /boot.
2. Embed it in the Linux kernel image, having the initramfs as part
of the BSP.
2. Deploy an existing system as a cpio archive
1. As a stand-alone system, without a rootfs
2. Nested inside another system
1.1 and 1.2 require system engineering work, so won't be mentioned here.
SCENARIO deploying a system with an initramfs
ASSUMING there is space for 5 512M disk images
GIVEN a workspace
AND a git server
WHEN the user checks out the system branch called master
GIVEN a cluster called C.morph in system branch master
AND a system in cluster C.morph in branch master called S
2.2 needs a nested system that is deployed with the initramfs write
extension.
GIVEN a subsystem in cluster C.morph in branch master called S.I
AND subsystem S.I in cluster C.morph in branch master builds systems/test-system.morph
AND subsystem S.I in cluster C.morph in branch master has deployment type: initramfs
The nested system needs to be placed somewhere in the parent. The
traditional place for an initramfs is `/boot`.
AND subsystem S.I in cluster C.morph in branch master has deployment location: boot/initramfs.gz
1.1 and 2.2 need the write extension to configure the boot-loader to
use the produced initramfs. Only write extensions that involve creating a disk image care, so we'll use `rawdisk.write`.
GIVEN system S in cluster C.morph in branch master builds systems/test-system.morph
AND system S in cluster C.morph in branch master has deployment type: rawdisk
AND system S in cluster C.morph in branch master has deployment location: test.img
AND system S in cluster C.morph in branch master has deployment variable: DISK_SIZE=512M
Initramfs support is triggered by the `INITRAMFS_PATH` variable. It could have been made automatic, triggering the behaviour if `/boot/initramfs.gz` exists, but:
1. There are a bunch of possible names, some of which imply different formats.
2. If we decide on one specific name, how do we pick.
3. If we allow multiple possible names, how do we handle multiple being possible.
4. We may need to pick a non-standard name: e.g. We have a deployment
where the system loads a kernel and initramfs from a disk, then boots
the target in KVM, so the bootloader we want to use for the guest is
`initramfs.gz`, while the host's initramfs is `hyp-initramfs.gz`.
5. We may have the initramfs come from a chunk the system built, but
for speed, we want this particular deployment not to use an initramfs,
even though we have a generic image that may support one.
For all these reasons, despite there being redundancy in some cases,
we're going to set `INITRAMFS_PATH` to the same as the nested deployment's
location.
GIVEN system S in cluster C.morph in branch master has deployment variable: INITRAMFS_PATH=boot/initramfs.gz
Fully testing that the system is bootable requires a lot more time,
infrastructure and dependencies, so we're just going to build it and
inspect the result of the deployment.
WHEN the user builds the system systems/test-system.morph in branch master
AND the user attempts to deploy the cluster C.morph in branch master
THEN morph succeeded
AND file workspace/master/test/morphs/test.img exists
If the initramfs write extension works, the rootfs image should contain
`boot/initramfs.gz`.
WHEN disk image workspace/master/test/morphs/test.img is mounted at mnt
THEN file mnt/systems/default/run/boot/initramfs.gz exists
If the `rawdisk` write extension worked, then the bootloader config file
will mention the initramfs, and the UUID of the disk.
AND file mnt/extlinux.conf matches initramfs
AND file mnt/extlinux.conf matches root=UUID=
FINALLY mnt is unmounted
Partial deployments
===================
Deploy part of a cluster
------------------------
Starting from the well-defined position of having a cluster morphology
with only one definition.
SCENARIO partially deploying a cluster morphology
GIVEN a workspace
AND a git server
WHEN the user checks out the system branch called master
AND the user builds the system systems/test-system.morph in branch master
GIVEN a cluster called test-cluster.morph in system branch master
AND a system in cluster test-cluster.morph in branch master called test-system
AND system test-system in cluster test-cluster.morph in branch master builds systems/test-system.morph
AND system test-system in cluster test-cluster.morph in branch master has deployment type: tar
AND system test-system in cluster test-cluster.morph in branch master has deployment location: test-system.tar
It is useful to group related deployments together, so we support adding
another deployment to the same cluster morphology.
GIVEN a system in cluster test-cluster.morph in branch master called second-system
AND system second-system in cluster test-cluster.morph in branch master builds systems/test-system.morph
AND system second-system in cluster test-cluster.morph in branch master has deployment type: tar
AND system second-system in cluster test-cluster.morph in branch master has deployment location: second-system.tar
When we don't tell `morph deploy` which system we want to deploy, all
of the systems in the cluster are deployed. Here a successful deployment
will have morph exit sucessfully and in the case of tarball deployments,
the tarballs for both the systems will be created.
WHEN the user attempts to deploy the cluster test-cluster.morph in branch master
THEN morph succeeded
AND file workspace/master/test/morphs/test-system.tar exists
AND file workspace/master/test/morphs/second-system.tar exists
However, we don't need to deploy every system defined in a cluster at
once. This is useful for cases such as having a cluster morphology for
deploying a whole distbuild network, and re-deploying only nodes that
have failed.
GIVEN the files workspace/master/test/morphs/test-system.tar and workspace/master/test/morphs/second-system.tar are removed
WHEN the user attempts to deploy test-system from cluster test-cluster.morph in branch master
A successful deployment will have morph exit successfully, and in the
case of tarball deployments, only the tarball for the system we asked
for will be created.
THEN morph succeeded
AND file workspace/master/test/morphs/test-system.tar exists
AND file workspace/master/test/morphs/second-system.tar does not exist
Cluster morphs can contain "nested systems", i.e. systems which have
subsystems to deploy as part of them.
We need to add a subsystem to the cluster to test this.
GIVEN a subsystem in cluster test-cluster.morph in branch master called test-system.sysroot
AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master builds systems/test-system.morph
AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment type: sysroot
We specify the location as a file path, this is relative to the parent
system's extracted rootfs, before it is configured.
AND subsystem test-system.sysroot in cluster test-cluster.morph in branch master has deployment location: var/lib/sysroots/test-system
The system which contains a nested system is deployed the same as
before, we don't need to mention the nested deployment.
AND the file workspace/master/test/morphs/test-system.tar is removed
WHEN the user attempts to deploy test-system from cluster test-cluster.morph in branch master
THEN morph succeeded
AND file workspace/master/test/morphs/test-system.tar exists
AND tarball workspace/master/test/morphs/test-system.tar contains var/lib/sysroots/test-system/baserock
Morph will abort deployment if the system to deploy that is specified
on the command line is not defined in the morphology.
WHEN the user attempts to deploy not-a-system from cluster test-cluster.morph in branch master
THEN morph failed
It is not valid to deploy a nested system on its own. If it becomes
desirable to deploy a system that is identical to a system that already
exists but is nested in another, it should be redefined as a top-level
deployment.
WHEN the user attempts to deploy test-system.sysroot from cluster test-cluster.morph in branch master
THEN morph failed
|