1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
|
=====================
Resource Lock Manager
=====================
.. contents::
This page describes the design of the resource lock manager that is used for
locking disk images, to ensure exclusive access to content.
Goals
-----
The high level goal is to prevent the same disk image being used by more than
one QEMU instance at a time (unless the disk is marked as shareable, or
readonly). The scenarios to be prevented are thus:
#. Two different guests running configured to point at the same disk image.
#. One guest being started more than once on two different machines due to admin
mistake
#. One guest being started more than once on a single machine due to libvirt
driver bug on a single machine.
Requirements
------------
The high level goal leads to a set of requirements for the lock manager design
#. A lock must be held on a disk whenever a QEMU process has the disk open
#. The lock scheme must allow QEMU to be configured with readonly, shared write,
or exclusive writable disks
#. A lock handover must be performed during the migration process where 2 QEMU
processes will have the same disk open concurrently.
#. The lock manager must be able to identify and kill the process accessing the
resource if the lock is revoked.
#. Locks can be acquired for arbitrary VM related resources, as determined by
the management application.
Design
------
Within a lock manager the following series of operations will need to be
supported.
- **Register object** Register the identity of an object against which locks
will be acquired
- **Add resource** Associate a resource with an object for future lock
acquisition / release
- **Acquire locks** Acquire the locks for all resources associated with the
object
- **Release locks** Release the locks for all resources associated with the
object
- **Inquire locks** Get a representation of the state of the locks for all
resources associated with the object
Plugin Implementations
----------------------
Lock manager implementations are provided as LGPLv2+ licensed, dlopen()able
library modules. The plugins will be loadable from the following location:
::
/usr/{lib,lib64}/libvirt/lock_manager/$NAME.so
The lock manager plugin must export a single ELF symbol named
``virLockDriverImpl``, which is a static instance of the ``virLockDriver``
struct. The struct is defined in the header file
::
#include <libvirt/plugins/lock_manager.h>
All callbacks in the struct must be initialized to non-NULL pointers. The
semantics of each callback are defined in the API docs embedded in the
previously mentioned header file
QEMU Driver integration
-----------------------
With the QEMU driver, the lock plugin will be set in the
``/etc/libvirt/qemu.conf`` configuration file by specifying the lock manager
name.
::
lockManager="sanlock"
By default the lock manager will be a 'no op' implementation for backwards
compatibility
Lock usage patterns
-------------------
The following pseudo code illustrates the common patterns of operations invoked
on the lock manager plugin callbacks.
Lock acquisition
~~~~~~~~~~~~~~~~
Initial lock acquisition will be performed from the process that is to own the
lock. This is typically the QEMU child process, in between the fork+exec
pairing. When adding further resources on the fly, to an existing object holding
locks, this will be done from the libvirtd process.
::
virLockManagerParam params[] = {
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
.key = "uuid",
},
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
.key = "name",
.value = { .str = dom->def->name },
},
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
.key = "id",
.value = { .i = dom->def->id },
},
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
.key = "pid",
.value = { .i = dom->pid },
},
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
.key = "uri",
.value = { .cstr = driver->uri },
},
};
mgr = virLockManagerNew(lockPlugin,
VIR_LOCK_MANAGER_TYPE_DOMAIN,
G_N_ELEMENTS(params),
params,
0)));
foreach (initial disks)
virLockManagerAddResource(mgr,
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
$path, 0, NULL, $flags);
if (virLockManagerAcquire(lock, NULL, 0) < 0);
...abort...
Lock release
~~~~~~~~~~~~
The locks are all implicitly released when the process that acquired them exits,
however, a process may voluntarily give up the lock by running
::
char *state = NULL;
virLockManagerParam params[] = {
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
.key = "uuid",
},
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
.key = "name",
.value = { .str = dom->def->name },
},
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
.key = "id",
.value = { .i = dom->def->id },
},
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
.key = "pid",
.value = { .i = dom->pid },
},
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
.key = "uri",
.value = { .cstr = driver->uri },
},
};
mgr = virLockManagerNew(lockPlugin,
VIR_LOCK_MANAGER_TYPE_DOMAIN,
G_N_ELEMENTS(params),
params,
0)));
foreach (initial disks)
virLockManagerAddResource(mgr,
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
$path, 0, NULL, $flags);
virLockManagerRelease(mgr, & state, 0);
The returned state string can be passed to the ``virLockManagerAcquire`` method
to later re-acquire the exact same locks. This state transfer is commonly used
when performing live migration of virtual machines. By validating the state the
lock manager can ensure no other VM has re-acquire the same locks on a different
host. The state can also be obtained without releasing the locks, by calling the
``virLockManagerInquire`` method.
|
