1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
|
Dom0less
========
"Dom0less" is a set of Xen features that enable the deployment of a Xen
system without an control domain (often referred to as "dom0"). Each
feature can be used independently from the others, unless otherwise
stated.
Booting Multiple Domains from Device Tree
-----------------------------------------
This feature enables Xen to create a set of DomUs at boot time.
Information about the DomUs to be created by Xen is passed to the
hypervisor via Device Tree. Specifically, the existing Device Tree based
Multiboot specification has been extended to allow for multiple domains
to be passed to Xen. See docs/misc/arm/device-tree/booting.txt for more
information about the Multiboot specification and how to use it.
Currently, a control domain ("dom0") is still required to manage the DomU
domains, but the system can start also without dom0 if the Device Tree
doesn't specify the dom0 kernel and it declares one or more domUs.
Instead of waiting for the control domain (when declared) to be fully
booted and the Xen tools to become available, domains created by Xen
this way are started right away in parallel. Hence, their boot time is
typically much shorter.
Configuration
-------------
### Loading binaries into memory ###
U-Boot needs to load not just Xen, the device tree binary, the dom0 kernel and
ramdisk. It also needs to load the kernel and ramdisk of any additional domains
to boot. For example if this is the bootcmd for Xen and Dom0:
tftpb 0x1280000 xen.dtb
tftpb 0x0x80000 xen-Image
tftpb 0x1400000 xen.ub
tftpb 0x9000000 xen-rootfs.cpio.gz.u-boot
bootm 0x1400000 0x9000000 0x1280000
If we want to add one DomU with Image-DomU as the DomU kernel
and ramdisk-DomU as DomU ramdisk:
tftpb 0x1280000 xen.dtb
tftpb 0x80000 xen-Image
tftpb 0x1400000 xen.ub
tftpb 0x9000000 xen-rootfs.cpio.gz.u-boot
tftpb 0x2000000 Image-DomU
tftpb 0x3000000 ramdisk-DomU
bootm 0x1400000 0x9000000 0x1280000
### Device Tree configuration ###
In addition to loading the necessary binaries, we also need to advertise
the presence of the additional VM and its configuration. It is done via
device tree adding a node under /chosen as follows:
domU1 {
#address-cells = <1>;
#size-cells = <1>;
compatible = "xen,domain";
memory = <0 0x20000>;
cpus = <1>;
vpl011;
module@2000000 {
compatible = "multiboot,kernel", "multiboot,module";
reg = <0x2000000 0xffffff>;
bootargs = "console=ttyAMA0";
};
module@30000000 {
compatible = "multiboot,ramdisk", "multiboot,module";
reg = <0x3000000 0xffffff>;
};
};
Where memory is the memory of the VM in KBs, cpus is the number of
cpus. module@2000000 and module@3000000 advertise where the kernel and
ramdisk are in memory.
Note: the size specified should exactly match the size of the Kernel/initramfs.
Otherwise, they may be unusable in Xen (for instance if they are compressed).
See docs/misc/arm/device-tree/booting.txt for more information.
PV Drivers
----------
It is possible to use PV drivers with dom0less guests with some
restrictions:
- dom0less domUs that want to use PV drivers support should have the
"xen,enhanced" property set under their device tree nodes (see
docs/misc/arm/device-tree/booting.txt)
- a dom0 must be present (or another domain with enough privileges to
run the toolstack)
- after dom0 is booted, the utility "init-dom0less" must be run
- do not run "init-dom0less" while creating other guests with xl
After the execution of init-dom0less, it is possible to use "xl" to
hotplug PV drivers to dom0less guests. E.g. xl network-attach domU.
The implementation works as follows:
- Xen allocates the xenstore event channel for each dom0less domU that
has the "xen,enhanced" property, and sets HVM_PARAM_STORE_EVTCHN
- Xen does *not* allocate the xenstore page and sets HVM_PARAM_STORE_PFN
to ~0ULL (invalid)
- Dom0less domU kernels check that HVM_PARAM_STORE_PFN is set to invalid
- Old kernels will continue without xenstore support (Note: some old
buggy kernels might crash because they don't check the validity of
HVM_PARAM_STORE_PFN before using it! Disable "xen,enhanced" in
those cases)
- New kernels will wait for a notification on the xenstore event
channel (HVM_PARAM_STORE_EVTCHN) before continuing with the
initialization
- Once dom0 is booted, init-dom0less is executed:
- it allocates the xenstore shared page and sets HVM_PARAM_STORE_PFN
- it calls xs_introduce_domain
- Xenstored notices the new domain, initializes interfaces as usual, and
sends an event channel notification to the domain using the xenstore
event channel (HVM_PARAM_STORE_EVTCHN)
- The Linux domU kernel receives the event channel notification, checks
HVM_PARAM_STORE_PFN again and continue with the initialization
Limitations
-----------
Domains started by Xen at boot time currently have the following
limitations:
- They cannot be properly shutdown or rebooted using xl. If one of them
crashes, the whole platform should be rebooted.
- Some xl operations might not work as expected. xl is meant to be used
with domains that have been created by it. Using xl with domains
started by Xen at boot might not work as expected.
- The GIC version is the native version. In absence of other
information, the GIC version exposed to the domains started by Xen at
boot is the same as the native GIC version.
- Pinning vCPUs of domains started by Xen at boot can be
done from the control domain, using `xl vcpu-pin` as usual. It is not
currently possible to configure vCPU pinning without a control domain.
However, the NULL scheduler can be selected by passing `sched=null` to
the Xen command line. The NULL scheduler automatically assigns and
pins vCPUs to pCPUs, but the vCPU-pCPU assignments cannot be
configured.
Notes
-----
- 'xl console' command will not attach to the domain's console in case
of dom0less. DomU are domains created by Xen (similar to Dom0) and
therefore they are all managed by Xen and some of the commands may not work.
A user is allowed to configure the key sequence to switch input.
Pressing the Xen "conswitch" (Ctrl-A by default) three times
switches input in case of dom0less mode.
- Domains created by Xen will have no name at boot. Domain-0 has a name
thanks to the helper xen-init-dom0 called at boot by the initscript.
If you want to setup DomU name, then you will have to create the xenstore
node associated. By default DomU names are shown as '(null)' in the
xl domains list.
|
