mirror of
https://gitlab.com/libvirt/libvirt.git
synced 2024-12-22 13:45:38 +00:00
docs: Convert 'internals/locking' page to rst and move it to 'kbase/internals'
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This commit is contained in:
Peter Krempa
parent
dc57ae6fe1
commit
d838439794
@ -157,9 +157,6 @@ Project development
|
||||
`RPC protocol & APIs <internals/rpc.html>`__
|
||||
RPC protocol information and API / dispatch guide
|
||||
|
||||
`Lock managers <internals/locking.html>`__
|
||||
Use lock managers to protect disk content
|
||||
|
||||
`Functional testing <testsuites.html>`__
|
||||
Testing libvirt with
|
||||
`TCK test suite <testtck.html>`__ and
|
||||
|
||||
@ -1,267 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE html>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml">
|
||||
<body>
|
||||
<h1>Resource Lock Manager</h1>
|
||||
|
||||
<ul id="toc"></ul>
|
||||
|
||||
<p>
|
||||
This page describes the design of the resource lock manager
|
||||
that is used for locking disk images, to ensure exclusive
|
||||
access to content.
|
||||
</p>
|
||||
|
||||
<h2><a id="goals">Goals</a></h2>
|
||||
|
||||
<p>
|
||||
The high level goal is to prevent the same disk image being
|
||||
used by more than one QEMU instance at a time (unless the
|
||||
disk is marked as shareable, or readonly). The scenarios
|
||||
to be prevented are thus:
|
||||
</p>
|
||||
|
||||
<ol>
|
||||
<li>
|
||||
Two different guests running configured to point at the
|
||||
same disk image.
|
||||
</li>
|
||||
<li>
|
||||
One guest being started more than once on two different
|
||||
machines due to admin mistake
|
||||
</li>
|
||||
<li>
|
||||
One guest being started more than once on a single machine
|
||||
due to libvirt driver bug on a single machine.
|
||||
</li>
|
||||
</ol>
|
||||
|
||||
<h2><a id="requirement">Requirements</a></h2>
|
||||
|
||||
<p>
|
||||
The high level goal leads to a set of requirements
|
||||
for the lock manager design
|
||||
</p>
|
||||
|
||||
<ol>
|
||||
<li>
|
||||
A lock must be held on a disk whenever a QEMU process
|
||||
has the disk open
|
||||
</li>
|
||||
<li>
|
||||
The lock scheme must allow QEMU to be configured with
|
||||
readonly, shared write, or exclusive writable disks
|
||||
</li>
|
||||
<li>
|
||||
A lock handover must be performed during the migration
|
||||
process where 2 QEMU processes will have the same disk
|
||||
open concurrently.
|
||||
</li>
|
||||
<li>
|
||||
The lock manager must be able to identify and kill the
|
||||
process accessing the resource if the lock is revoked.
|
||||
</li>
|
||||
<li>
|
||||
Locks can be acquired for arbitrary VM related resources,
|
||||
as determined by the management application.
|
||||
</li>
|
||||
</ol>
|
||||
|
||||
<h2><a id="design">Design</a></h2>
|
||||
|
||||
<p>
|
||||
Within a lock manager the following series of operations
|
||||
will need to be supported.
|
||||
</p>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
<strong>Register object</strong>
|
||||
Register the identity of an object against which
|
||||
locks will be acquired
|
||||
</li>
|
||||
<li>
|
||||
<strong>Add resource</strong>
|
||||
Associate a resource with an object for future
|
||||
lock acquisition / release
|
||||
</li>
|
||||
<li>
|
||||
<strong>Acquire locks</strong>
|
||||
Acquire the locks for all resources associated
|
||||
with the object
|
||||
</li>
|
||||
<li>
|
||||
<strong>Release locks</strong>
|
||||
Release the locks for all resources associated
|
||||
with the object
|
||||
</li>
|
||||
<li>
|
||||
<strong>Inquire locks</strong>
|
||||
Get a representation of the state of the locks
|
||||
for all resources associated with the object
|
||||
</li>
|
||||
</ul>
|
||||
|
||||
<h2><a id="impl">Plugin Implementations</a></h2>
|
||||
|
||||
<p>
|
||||
Lock manager implementations are provided as LGPLv2+
|
||||
licensed, dlopen()able library modules. The plugins
|
||||
will be loadable from the following location:
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
/usr/{lib,lib64}/libvirt/lock_manager/$NAME.so
|
||||
</pre>
|
||||
|
||||
<p>
|
||||
The lock manager plugin must export a single ELF
|
||||
symbol named <code>virLockDriverImpl</code>, which is
|
||||
a static instance of the <code>virLockDriver</code>
|
||||
struct. The struct is defined in the header file
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
#include <libvirt/plugins/lock_manager.h>
|
||||
</pre>
|
||||
|
||||
<p>
|
||||
All callbacks in the struct must be initialized
|
||||
to non-NULL pointers. The semantics of each
|
||||
callback are defined in the API docs embedded
|
||||
in the previously mentioned header file
|
||||
</p>
|
||||
|
||||
<h2><a id="qemuIntegrate">QEMU Driver integration</a></h2>
|
||||
|
||||
<p>
|
||||
With the QEMU driver, the lock plugin will be set
|
||||
in the <code>/etc/libvirt/qemu.conf</code> configuration
|
||||
file by specifying the lock manager name.
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
lockManager="sanlock"
|
||||
</pre>
|
||||
|
||||
<p>
|
||||
By default the lock manager will be a 'no op' implementation
|
||||
for backwards compatibility
|
||||
</p>
|
||||
|
||||
<h2><a id="usagePatterns">Lock usage patterns</a></h2>
|
||||
|
||||
<p>
|
||||
The following pseudo code illustrates the common
|
||||
patterns of operations invoked on the lock
|
||||
manager plugin callbacks.
|
||||
</p>
|
||||
|
||||
<h3><a id="usageLockAcquire">Lock acquisition</a></h3>
|
||||
|
||||
<p>
|
||||
Initial lock acquisition will be performed from the
|
||||
process that is to own the lock. This is typically
|
||||
the QEMU child process, in between the fork+exec
|
||||
pairing. When adding further resources on the fly,
|
||||
to an existing object holding locks, this will be
|
||||
done from the libvirtd process.
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
virLockManagerParam params[] = {
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
||||
.key = "uuid",
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
||||
.key = "name",
|
||||
.value = { .str = dom->def->name },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||
.key = "id",
|
||||
.value = { .i = dom->def->id },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||
.key = "pid",
|
||||
.value = { .i = dom->pid },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
||||
.key = "uri",
|
||||
.value = { .cstr = driver->uri },
|
||||
},
|
||||
};
|
||||
mgr = virLockManagerNew(lockPlugin,
|
||||
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
||||
ARRAY_CARDINALITY(params),
|
||||
params,
|
||||
0)));
|
||||
|
||||
foreach (initial disks)
|
||||
virLockManagerAddResource(mgr,
|
||||
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
||||
$path, 0, NULL, $flags);
|
||||
|
||||
if (virLockManagerAcquire(lock, NULL, 0) < 0);
|
||||
...abort...
|
||||
</pre>
|
||||
|
||||
<h3><a id="usageLockAttach">Lock release</a></h3>
|
||||
|
||||
<p>
|
||||
The locks are all implicitly released when the process
|
||||
that acquired them exits, however, a process may
|
||||
voluntarily give up the lock by running
|
||||
</p>
|
||||
|
||||
<pre>
|
||||
char *state = NULL;
|
||||
virLockManagerParam params[] = {
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
||||
.key = "uuid",
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
||||
.key = "name",
|
||||
.value = { .str = dom->def->name },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||
.key = "id",
|
||||
.value = { .i = dom->def->id },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||
.key = "pid",
|
||||
.value = { .i = dom->pid },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
||||
.key = "uri",
|
||||
.value = { .cstr = driver->uri },
|
||||
},
|
||||
};
|
||||
mgr = virLockManagerNew(lockPlugin,
|
||||
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
||||
ARRAY_CARDINALITY(params),
|
||||
params,
|
||||
0)));
|
||||
|
||||
foreach (initial disks)
|
||||
virLockManagerAddResource(mgr,
|
||||
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
||||
$path, 0, NULL, $flags);
|
||||
|
||||
virLockManagerRelease(mgr, & state, 0);
|
||||
</pre>
|
||||
|
||||
<p>
|
||||
The returned state string can be passed to the
|
||||
<code>virLockManagerAcquire</code> method to
|
||||
later re-acquire the exact same locks. This
|
||||
state transfer is commonly used when performing
|
||||
live migration of virtual machines. By validating
|
||||
the state the lock manager can ensure no other
|
||||
VM has re-acquire the same locks on a different
|
||||
host. The state can also be obtained without
|
||||
releasing the locks, by calling the
|
||||
<code>virLockManagerInquire</code> method.
|
||||
</p>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
@ -1,5 +1,4 @@
|
||||
internals_in_files = [
|
||||
'locking',
|
||||
'rpc',
|
||||
]
|
||||
|
||||
|
||||
@ -91,3 +91,6 @@ Internals
|
||||
|
||||
`Event loop and worker pool <internals/eventloop.html>`__
|
||||
Libvirt's event loop and worker pool mode
|
||||
|
||||
`Lock managers <internals/locking.html>`__
|
||||
Use lock managers to protect disk content
|
||||
|
||||
190
docs/kbase/internals/locking.rst
Normal file
190
docs/kbase/internals/locking.rst
Normal file
@ -0,0 +1,190 @@
|
||||
=====================
|
||||
Resource Lock Manager
|
||||
=====================
|
||||
|
||||
.. contents::
|
||||
|
||||
This page describes the design of the resource lock manager that is used for
|
||||
locking disk images, to ensure exclusive access to content.
|
||||
|
||||
Goals
|
||||
-----
|
||||
|
||||
The high level goal is to prevent the same disk image being used by more than
|
||||
one QEMU instance at a time (unless the disk is marked as shareable, or
|
||||
readonly). The scenarios to be prevented are thus:
|
||||
|
||||
#. Two different guests running configured to point at the same disk image.
|
||||
#. One guest being started more than once on two different machines due to admin
|
||||
mistake
|
||||
#. One guest being started more than once on a single machine due to libvirt
|
||||
driver bug on a single machine.
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
||||
The high level goal leads to a set of requirements for the lock manager design
|
||||
|
||||
#. A lock must be held on a disk whenever a QEMU process has the disk open
|
||||
#. The lock scheme must allow QEMU to be configured with readonly, shared write,
|
||||
or exclusive writable disks
|
||||
#. A lock handover must be performed during the migration process where 2 QEMU
|
||||
processes will have the same disk open concurrently.
|
||||
#. The lock manager must be able to identify and kill the process accessing the
|
||||
resource if the lock is revoked.
|
||||
#. Locks can be acquired for arbitrary VM related resources, as determined by
|
||||
the management application.
|
||||
|
||||
Design
|
||||
------
|
||||
|
||||
Within a lock manager the following series of operations will need to be
|
||||
supported.
|
||||
|
||||
- **Register object** Register the identity of an object against which locks
|
||||
will be acquired
|
||||
- **Add resource** Associate a resource with an object for future lock
|
||||
acquisition / release
|
||||
- **Acquire locks** Acquire the locks for all resources associated with the
|
||||
object
|
||||
- **Release locks** Release the locks for all resources associated with the
|
||||
object
|
||||
- **Inquire locks** Get a representation of the state of the locks for all
|
||||
resources associated with the object
|
||||
|
||||
Plugin Implementations
|
||||
----------------------
|
||||
|
||||
Lock manager implementations are provided as LGPLv2+ licensed, dlopen()able
|
||||
library modules. The plugins will be loadable from the following location:
|
||||
|
||||
::
|
||||
|
||||
/usr/{lib,lib64}/libvirt/lock_manager/$NAME.so
|
||||
|
||||
The lock manager plugin must export a single ELF symbol named
|
||||
``virLockDriverImpl``, which is a static instance of the ``virLockDriver``
|
||||
struct. The struct is defined in the header file
|
||||
|
||||
::
|
||||
|
||||
#include <libvirt/plugins/lock_manager.h>
|
||||
|
||||
All callbacks in the struct must be initialized to non-NULL pointers. The
|
||||
semantics of each callback are defined in the API docs embedded in the
|
||||
previously mentioned header file
|
||||
|
||||
QEMU Driver integration
|
||||
-----------------------
|
||||
|
||||
With the QEMU driver, the lock plugin will be set in the
|
||||
``/etc/libvirt/qemu.conf`` configuration file by specifying the lock manager
|
||||
name.
|
||||
|
||||
::
|
||||
|
||||
lockManager="sanlock"
|
||||
|
||||
By default the lock manager will be a 'no op' implementation for backwards
|
||||
compatibility
|
||||
|
||||
Lock usage patterns
|
||||
-------------------
|
||||
|
||||
The following pseudo code illustrates the common patterns of operations invoked
|
||||
on the lock manager plugin callbacks.
|
||||
|
||||
Lock acquisition
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
Initial lock acquisition will be performed from the process that is to own the
|
||||
lock. This is typically the QEMU child process, in between the fork+exec
|
||||
pairing. When adding further resources on the fly, to an existing object holding
|
||||
locks, this will be done from the libvirtd process.
|
||||
|
||||
::
|
||||
|
||||
virLockManagerParam params[] = {
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
||||
.key = "uuid",
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
||||
.key = "name",
|
||||
.value = { .str = dom->def->name },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||
.key = "id",
|
||||
.value = { .i = dom->def->id },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||
.key = "pid",
|
||||
.value = { .i = dom->pid },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
||||
.key = "uri",
|
||||
.value = { .cstr = driver->uri },
|
||||
},
|
||||
};
|
||||
mgr = virLockManagerNew(lockPlugin,
|
||||
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
||||
ARRAY_CARDINALITY(params),
|
||||
params,
|
||||
0)));
|
||||
|
||||
foreach (initial disks)
|
||||
virLockManagerAddResource(mgr,
|
||||
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
||||
$path, 0, NULL, $flags);
|
||||
|
||||
if (virLockManagerAcquire(lock, NULL, 0) < 0);
|
||||
...abort...
|
||||
|
||||
Lock release
|
||||
~~~~~~~~~~~~
|
||||
|
||||
The locks are all implicitly released when the process that acquired them exits,
|
||||
however, a process may voluntarily give up the lock by running
|
||||
|
||||
::
|
||||
|
||||
char *state = NULL;
|
||||
virLockManagerParam params[] = {
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
||||
.key = "uuid",
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
||||
.key = "name",
|
||||
.value = { .str = dom->def->name },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||
.key = "id",
|
||||
.value = { .i = dom->def->id },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||
.key = "pid",
|
||||
.value = { .i = dom->pid },
|
||||
},
|
||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
||||
.key = "uri",
|
||||
.value = { .cstr = driver->uri },
|
||||
},
|
||||
};
|
||||
mgr = virLockManagerNew(lockPlugin,
|
||||
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
||||
ARRAY_CARDINALITY(params),
|
||||
params,
|
||||
0)));
|
||||
|
||||
foreach (initial disks)
|
||||
virLockManagerAddResource(mgr,
|
||||
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
||||
$path, 0, NULL, $flags);
|
||||
|
||||
virLockManagerRelease(mgr, & state, 0);
|
||||
|
||||
The returned state string can be passed to the ``virLockManagerAcquire`` method
|
||||
to later re-acquire the exact same locks. This state transfer is commonly used
|
||||
when performing live migration of virtual machines. By validating the state the
|
||||
lock manager can ensure no other VM has re-acquire the same locks on a different
|
||||
host. The state can also be obtained without releasing the locks, by calling the
|
||||
``virLockManagerInquire`` method.
|
||||
@ -2,6 +2,7 @@ docs_kbase_internals_files = [
|
||||
'command',
|
||||
'eventloop',
|
||||
'incremental-backup',
|
||||
'locking',
|
||||
'migration',
|
||||
]
|
||||
|
||||
|
||||
@ -36,7 +36,6 @@ foreach name : docs_kbase_files
|
||||
}
|
||||
endforeach
|
||||
|
||||
# keep the XSLT processing code block in sync with docs/meson.build
|
||||
|
||||
# --- begin of XSLT processing ---
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user