2013-05-03 14:25:37 +00:00
|
|
|
<?xml version="1.0" encoding="UTF-8"?>
|
2017-07-26 17:01:25 +00:00
|
|
|
<!DOCTYPE html>
|
2013-05-03 14:25:37 +00:00
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
2010-11-22 16:44:13 +00:00
|
|
|
<body>
|
|
|
|
|
<h1>Resource Lock Manager</h1>
|
|
|
|
|
|
|
|
|
|
<ul id="toc"></ul>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
This page describes the design of the resource lock manager
|
|
|
|
|
that is used for locking disk images, to ensure exclusive
|
|
|
|
|
access to content.
|
|
|
|
|
</p>
|
|
|
|
|
|
2017-07-26 14:52:42 +00:00
|
|
|
<h2><a id="goals">Goals</a></h2>
|
2010-11-22 16:44:13 +00:00
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The high level goal is to prevent the same disk image being
|
|
|
|
|
used by more than one QEMU instance at a time (unless the
|
2011-07-15 18:08:59 +00:00
|
|
|
disk is marked as shareable, or readonly). The scenarios
|
2010-11-22 16:44:13 +00:00
|
|
|
to be prevented are thus:
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<ol>
|
|
|
|
|
<li>
|
|
|
|
|
Two different guests running configured to point at the
|
|
|
|
|
same disk image.
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
One guest being started more than once on two different
|
|
|
|
|
machines due to admin mistake
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
One guest being started more than once on a single machine
|
|
|
|
|
due to libvirt driver bug on a single machine.
|
|
|
|
|
</li>
|
|
|
|
|
</ol>
|
|
|
|
|
|
2017-07-26 14:52:42 +00:00
|
|
|
<h2><a id="requirement">Requirements</a></h2>
|
2010-11-22 16:44:13 +00:00
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The high level goal leads to a set of requirements
|
|
|
|
|
for the lock manager design
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<ol>
|
|
|
|
|
<li>
|
|
|
|
|
A lock must be held on a disk whenever a QEMU process
|
|
|
|
|
has the disk open
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
The lock scheme must allow QEMU to be configured with
|
|
|
|
|
readonly, shared write, or exclusive writable disks
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
A lock handover must be performed during the migration
|
|
|
|
|
process where 2 QEMU processes will have the same disk
|
|
|
|
|
open concurrently.
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
The lock manager must be able to identify and kill the
|
|
|
|
|
process accessing the resource if the lock is revoked.
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
Locks can be acquired for arbitrary VM related resources,
|
|
|
|
|
as determined by the management application.
|
|
|
|
|
</li>
|
|
|
|
|
</ol>
|
|
|
|
|
|
2017-07-26 14:52:42 +00:00
|
|
|
<h2><a id="design">Design</a></h2>
|
2010-11-22 16:44:13 +00:00
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Within a lock manager the following series of operations
|
|
|
|
|
will need to be supported.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<ul>
|
|
|
|
|
<li>
|
|
|
|
|
<strong>Register object</strong>
|
|
|
|
|
Register the identity of an object against which
|
|
|
|
|
locks will be acquired
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<strong>Add resource</strong>
|
|
|
|
|
Associate a resource with an object for future
|
|
|
|
|
lock acquisition / release
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<strong>Acquire locks</strong>
|
|
|
|
|
Acquire the locks for all resources associated
|
|
|
|
|
with the object
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<strong>Release locks</strong>
|
|
|
|
|
Release the locks for all resources associated
|
|
|
|
|
with the object
|
|
|
|
|
</li>
|
|
|
|
|
<li>
|
|
|
|
|
<strong>Inquire locks</strong>
|
|
|
|
|
Get a representation of the state of the locks
|
|
|
|
|
for all resources associated with the object
|
|
|
|
|
</li>
|
|
|
|
|
</ul>
|
|
|
|
|
|
2017-07-26 14:52:42 +00:00
|
|
|
<h2><a id="impl">Plugin Implementations</a></h2>
|
2010-11-22 16:44:13 +00:00
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Lock manager implementations are provided as LGPLv2+
|
|
|
|
|
licensed, dlopen()able library modules. The plugins
|
|
|
|
|
will be loadable from the following location:
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
|
|
|
|
/usr/{lib,lib64}/libvirt/lock_manager/$NAME.so
|
|
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The lock manager plugin must export a single ELF
|
|
|
|
|
symbol named <code>virLockDriverImpl</code>, which is
|
|
|
|
|
a static instance of the <code>virLockDriver</code>
|
|
|
|
|
struct. The struct is defined in the header file
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
#include <libvirt/plugins/lock_manager.h>
|
2010-11-22 16:44:13 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
All callbacks in the struct must be initialized
|
|
|
|
|
to non-NULL pointers. The semantics of each
|
|
|
|
|
callback are defined in the API docs embedded
|
|
|
|
|
in the previously mentioned header file
|
|
|
|
|
</p>
|
|
|
|
|
|
2017-07-26 14:52:42 +00:00
|
|
|
<h2><a id="qemuIntegrate">QEMU Driver integration</a></h2>
|
2010-11-22 16:44:13 +00:00
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
With the QEMU driver, the lock plugin will be set
|
|
|
|
|
in the <code>/etc/libvirt/qemu.conf</code> configuration
|
|
|
|
|
file by specifying the lock manager name.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
lockManager="sanlock"
|
2010-11-22 16:44:13 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
By default the lock manager will be a 'no op' implementation
|
|
|
|
|
for backwards compatibility
|
|
|
|
|
</p>
|
|
|
|
|
|
2017-07-26 14:52:42 +00:00
|
|
|
<h2><a id="usagePatterns">Lock usage patterns</a></h2>
|
2010-11-22 16:44:13 +00:00
|
|
|
|
|
|
|
|
<p>
|
2012-10-11 16:31:20 +00:00
|
|
|
The following pseudo code illustrates the common
|
2010-11-22 16:44:13 +00:00
|
|
|
patterns of operations invoked on the lock
|
|
|
|
|
manager plugin callbacks.
|
|
|
|
|
</p>
|
|
|
|
|
|
2017-07-26 14:52:42 +00:00
|
|
|
<h3><a id="usageLockAcquire">Lock acquisition</a></h3>
|
2010-11-22 16:44:13 +00:00
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Initial lock acquisition will be performed from the
|
|
|
|
|
process that is to own the lock. This is typically
|
|
|
|
|
the QEMU child process, in between the fork+exec
|
|
|
|
|
pairing. When adding further resources on the fly,
|
|
|
|
|
to an existing object holding locks, this will be
|
|
|
|
|
done from the libvirtd process.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
virLockManagerParam params[] = {
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
|
|
|
|
.key = "uuid",
|
|
|
|
|
},
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
|
|
|
|
.key = "name",
|
|
|
|
|
.value = { .str = dom->def->name },
|
|
|
|
|
},
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
|
|
|
|
.key = "id",
|
|
|
|
|
.value = { .i = dom->def->id },
|
|
|
|
|
},
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
|
|
|
|
.key = "pid",
|
|
|
|
|
.value = { .i = dom->pid },
|
|
|
|
|
},
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
|
|
|
|
.key = "uri",
|
|
|
|
|
.value = { .cstr = driver->uri },
|
|
|
|
|
},
|
|
|
|
|
};
|
|
|
|
|
mgr = virLockManagerNew(lockPlugin,
|
|
|
|
|
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
|
|
|
|
ARRAY_CARDINALITY(params),
|
|
|
|
|
params,
|
|
|
|
|
0)));
|
|
|
|
|
|
|
|
|
|
foreach (initial disks)
|
|
|
|
|
virLockManagerAddResource(mgr,
|
|
|
|
|
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
|
|
|
|
$path, 0, NULL, $flags);
|
|
|
|
|
|
|
|
|
|
if (virLockManagerAcquire(lock, NULL, 0) < 0);
|
|
|
|
|
...abort...
|
2010-11-22 16:44:13 +00:00
|
|
|
</pre>
|
|
|
|
|
|
2017-07-26 14:52:42 +00:00
|
|
|
<h3><a id="usageLockAttach">Lock release</a></h3>
|
2010-11-22 16:44:13 +00:00
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The locks are all implicitly released when the process
|
|
|
|
|
that acquired them exits, however, a process may
|
|
|
|
|
voluntarily give up the lock by running
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
char *state = NULL;
|
|
|
|
|
virLockManagerParam params[] = {
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
|
|
|
|
.key = "uuid",
|
|
|
|
|
},
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
|
|
|
|
.key = "name",
|
|
|
|
|
.value = { .str = dom->def->name },
|
|
|
|
|
},
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
|
|
|
|
.key = "id",
|
|
|
|
|
.value = { .i = dom->def->id },
|
|
|
|
|
},
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
|
|
|
|
.key = "pid",
|
|
|
|
|
.value = { .i = dom->pid },
|
|
|
|
|
},
|
|
|
|
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
|
|
|
|
.key = "uri",
|
|
|
|
|
.value = { .cstr = driver->uri },
|
|
|
|
|
},
|
|
|
|
|
};
|
|
|
|
|
mgr = virLockManagerNew(lockPlugin,
|
|
|
|
|
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
|
|
|
|
ARRAY_CARDINALITY(params),
|
|
|
|
|
params,
|
|
|
|
|
0)));
|
|
|
|
|
|
|
|
|
|
foreach (initial disks)
|
|
|
|
|
virLockManagerAddResource(mgr,
|
|
|
|
|
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
|
|
|
|
$path, 0, NULL, $flags);
|
|
|
|
|
|
|
|
|
|
virLockManagerRelease(mgr, & state, 0);
|
2010-11-22 16:44:13 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The returned state string can be passed to the
|
|
|
|
|
<code>virLockManagerAcquire</code> method to
|
|
|
|
|
later re-acquire the exact same locks. This
|
|
|
|
|
state transfer is commonly used when performing
|
|
|
|
|
live migration of virtual machines. By validating
|
|
|
|
|
the state the lock manager can ensure no other
|
|
|
|
|
VM has re-acquire the same locks on a different
|
|
|
|
|
host. The state can also be obtained without
|
|
|
|
|
releasing the locks, by calling the
|
|
|
|
|
<code>virLockManagerInquire</code> method.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
