mirror of
https://gitlab.com/libvirt/libvirt.git
synced 2024-12-22 13:45:38 +00:00
docs: Convert 'internals/locking' page to rst and move it to 'kbase/internals'
Signed-off-by: Peter Krempa <pkrempa@redhat.com> Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This commit is contained in:
Peter Krempa
parent
dc57ae6fe1
commit
d838439794
@ -157,9 +157,6 @@ Project development
|
|||||||
`RPC protocol & APIs <internals/rpc.html>`__
|
`RPC protocol & APIs <internals/rpc.html>`__
|
||||||
RPC protocol information and API / dispatch guide
|
RPC protocol information and API / dispatch guide
|
||||||
|
|
||||||
`Lock managers <internals/locking.html>`__
|
|
||||||
Use lock managers to protect disk content
|
|
||||||
|
|
||||||
`Functional testing <testsuites.html>`__
|
`Functional testing <testsuites.html>`__
|
||||||
Testing libvirt with
|
Testing libvirt with
|
||||||
`TCK test suite <testtck.html>`__ and
|
`TCK test suite <testtck.html>`__ and
|
||||||
|
|||||||
@ -1,267 +0,0 @@
|
|||||||
<?xml version="1.0" encoding="UTF-8"?>
|
|
||||||
<!DOCTYPE html>
|
|
||||||
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
||||||
<body>
|
|
||||||
<h1>Resource Lock Manager</h1>
|
|
||||||
|
|
||||||
<ul id="toc"></ul>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
This page describes the design of the resource lock manager
|
|
||||||
that is used for locking disk images, to ensure exclusive
|
|
||||||
access to content.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<h2><a id="goals">Goals</a></h2>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
The high level goal is to prevent the same disk image being
|
|
||||||
used by more than one QEMU instance at a time (unless the
|
|
||||||
disk is marked as shareable, or readonly). The scenarios
|
|
||||||
to be prevented are thus:
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<ol>
|
|
||||||
<li>
|
|
||||||
Two different guests running configured to point at the
|
|
||||||
same disk image.
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
One guest being started more than once on two different
|
|
||||||
machines due to admin mistake
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
One guest being started more than once on a single machine
|
|
||||||
due to libvirt driver bug on a single machine.
|
|
||||||
</li>
|
|
||||||
</ol>
|
|
||||||
|
|
||||||
<h2><a id="requirement">Requirements</a></h2>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
The high level goal leads to a set of requirements
|
|
||||||
for the lock manager design
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<ol>
|
|
||||||
<li>
|
|
||||||
A lock must be held on a disk whenever a QEMU process
|
|
||||||
has the disk open
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
The lock scheme must allow QEMU to be configured with
|
|
||||||
readonly, shared write, or exclusive writable disks
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
A lock handover must be performed during the migration
|
|
||||||
process where 2 QEMU processes will have the same disk
|
|
||||||
open concurrently.
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
The lock manager must be able to identify and kill the
|
|
||||||
process accessing the resource if the lock is revoked.
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
Locks can be acquired for arbitrary VM related resources,
|
|
||||||
as determined by the management application.
|
|
||||||
</li>
|
|
||||||
</ol>
|
|
||||||
|
|
||||||
<h2><a id="design">Design</a></h2>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
Within a lock manager the following series of operations
|
|
||||||
will need to be supported.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<ul>
|
|
||||||
<li>
|
|
||||||
<strong>Register object</strong>
|
|
||||||
Register the identity of an object against which
|
|
||||||
locks will be acquired
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<strong>Add resource</strong>
|
|
||||||
Associate a resource with an object for future
|
|
||||||
lock acquisition / release
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<strong>Acquire locks</strong>
|
|
||||||
Acquire the locks for all resources associated
|
|
||||||
with the object
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<strong>Release locks</strong>
|
|
||||||
Release the locks for all resources associated
|
|
||||||
with the object
|
|
||||||
</li>
|
|
||||||
<li>
|
|
||||||
<strong>Inquire locks</strong>
|
|
||||||
Get a representation of the state of the locks
|
|
||||||
for all resources associated with the object
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
|
|
||||||
<h2><a id="impl">Plugin Implementations</a></h2>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
Lock manager implementations are provided as LGPLv2+
|
|
||||||
licensed, dlopen()able library modules. The plugins
|
|
||||||
will be loadable from the following location:
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<pre>
|
|
||||||
/usr/{lib,lib64}/libvirt/lock_manager/$NAME.so
|
|
||||||
</pre>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
The lock manager plugin must export a single ELF
|
|
||||||
symbol named <code>virLockDriverImpl</code>, which is
|
|
||||||
a static instance of the <code>virLockDriver</code>
|
|
||||||
struct. The struct is defined in the header file
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<pre>
|
|
||||||
#include <libvirt/plugins/lock_manager.h>
|
|
||||||
</pre>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
All callbacks in the struct must be initialized
|
|
||||||
to non-NULL pointers. The semantics of each
|
|
||||||
callback are defined in the API docs embedded
|
|
||||||
in the previously mentioned header file
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<h2><a id="qemuIntegrate">QEMU Driver integration</a></h2>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
With the QEMU driver, the lock plugin will be set
|
|
||||||
in the <code>/etc/libvirt/qemu.conf</code> configuration
|
|
||||||
file by specifying the lock manager name.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<pre>
|
|
||||||
lockManager="sanlock"
|
|
||||||
</pre>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
By default the lock manager will be a 'no op' implementation
|
|
||||||
for backwards compatibility
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<h2><a id="usagePatterns">Lock usage patterns</a></h2>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
The following pseudo code illustrates the common
|
|
||||||
patterns of operations invoked on the lock
|
|
||||||
manager plugin callbacks.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<h3><a id="usageLockAcquire">Lock acquisition</a></h3>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
Initial lock acquisition will be performed from the
|
|
||||||
process that is to own the lock. This is typically
|
|
||||||
the QEMU child process, in between the fork+exec
|
|
||||||
pairing. When adding further resources on the fly,
|
|
||||||
to an existing object holding locks, this will be
|
|
||||||
done from the libvirtd process.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<pre>
|
|
||||||
virLockManagerParam params[] = {
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
|
||||||
.key = "uuid",
|
|
||||||
},
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
|
||||||
.key = "name",
|
|
||||||
.value = { .str = dom->def->name },
|
|
||||||
},
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
|
||||||
.key = "id",
|
|
||||||
.value = { .i = dom->def->id },
|
|
||||||
},
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
|
||||||
.key = "pid",
|
|
||||||
.value = { .i = dom->pid },
|
|
||||||
},
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
|
||||||
.key = "uri",
|
|
||||||
.value = { .cstr = driver->uri },
|
|
||||||
},
|
|
||||||
};
|
|
||||||
mgr = virLockManagerNew(lockPlugin,
|
|
||||||
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
|
||||||
ARRAY_CARDINALITY(params),
|
|
||||||
params,
|
|
||||||
0)));
|
|
||||||
|
|
||||||
foreach (initial disks)
|
|
||||||
virLockManagerAddResource(mgr,
|
|
||||||
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
|
||||||
$path, 0, NULL, $flags);
|
|
||||||
|
|
||||||
if (virLockManagerAcquire(lock, NULL, 0) < 0);
|
|
||||||
...abort...
|
|
||||||
</pre>
|
|
||||||
|
|
||||||
<h3><a id="usageLockAttach">Lock release</a></h3>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
The locks are all implicitly released when the process
|
|
||||||
that acquired them exits, however, a process may
|
|
||||||
voluntarily give up the lock by running
|
|
||||||
</p>
|
|
||||||
|
|
||||||
<pre>
|
|
||||||
char *state = NULL;
|
|
||||||
virLockManagerParam params[] = {
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
|
||||||
.key = "uuid",
|
|
||||||
},
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
|
||||||
.key = "name",
|
|
||||||
.value = { .str = dom->def->name },
|
|
||||||
},
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
|
||||||
.key = "id",
|
|
||||||
.value = { .i = dom->def->id },
|
|
||||||
},
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
|
||||||
.key = "pid",
|
|
||||||
.value = { .i = dom->pid },
|
|
||||||
},
|
|
||||||
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
|
||||||
.key = "uri",
|
|
||||||
.value = { .cstr = driver->uri },
|
|
||||||
},
|
|
||||||
};
|
|
||||||
mgr = virLockManagerNew(lockPlugin,
|
|
||||||
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
|
||||||
ARRAY_CARDINALITY(params),
|
|
||||||
params,
|
|
||||||
0)));
|
|
||||||
|
|
||||||
foreach (initial disks)
|
|
||||||
virLockManagerAddResource(mgr,
|
|
||||||
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
|
||||||
$path, 0, NULL, $flags);
|
|
||||||
|
|
||||||
virLockManagerRelease(mgr, & state, 0);
|
|
||||||
</pre>
|
|
||||||
|
|
||||||
<p>
|
|
||||||
The returned state string can be passed to the
|
|
||||||
<code>virLockManagerAcquire</code> method to
|
|
||||||
later re-acquire the exact same locks. This
|
|
||||||
state transfer is commonly used when performing
|
|
||||||
live migration of virtual machines. By validating
|
|
||||||
the state the lock manager can ensure no other
|
|
||||||
VM has re-acquire the same locks on a different
|
|
||||||
host. The state can also be obtained without
|
|
||||||
releasing the locks, by calling the
|
|
||||||
<code>virLockManagerInquire</code> method.
|
|
||||||
</p>
|
|
||||||
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@ -1,5 +1,4 @@
|
|||||||
internals_in_files = [
|
internals_in_files = [
|
||||||
'locking',
|
|
||||||
'rpc',
|
'rpc',
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|||||||
@ -91,3 +91,6 @@ Internals
|
|||||||
|
|
||||||
`Event loop and worker pool <internals/eventloop.html>`__
|
`Event loop and worker pool <internals/eventloop.html>`__
|
||||||
Libvirt's event loop and worker pool mode
|
Libvirt's event loop and worker pool mode
|
||||||
|
|
||||||
|
`Lock managers <internals/locking.html>`__
|
||||||
|
Use lock managers to protect disk content
|
||||||
|
|||||||
190
docs/kbase/internals/locking.rst
Normal file
190
docs/kbase/internals/locking.rst
Normal file
@ -0,0 +1,190 @@
|
|||||||
|
=====================
|
||||||
|
Resource Lock Manager
|
||||||
|
=====================
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
|
||||||
|
This page describes the design of the resource lock manager that is used for
|
||||||
|
locking disk images, to ensure exclusive access to content.
|
||||||
|
|
||||||
|
Goals
|
||||||
|
-----
|
||||||
|
|
||||||
|
The high level goal is to prevent the same disk image being used by more than
|
||||||
|
one QEMU instance at a time (unless the disk is marked as shareable, or
|
||||||
|
readonly). The scenarios to be prevented are thus:
|
||||||
|
|
||||||
|
#. Two different guests running configured to point at the same disk image.
|
||||||
|
#. One guest being started more than once on two different machines due to admin
|
||||||
|
mistake
|
||||||
|
#. One guest being started more than once on a single machine due to libvirt
|
||||||
|
driver bug on a single machine.
|
||||||
|
|
||||||
|
Requirements
|
||||||
|
------------
|
||||||
|
|
||||||
|
The high level goal leads to a set of requirements for the lock manager design
|
||||||
|
|
||||||
|
#. A lock must be held on a disk whenever a QEMU process has the disk open
|
||||||
|
#. The lock scheme must allow QEMU to be configured with readonly, shared write,
|
||||||
|
or exclusive writable disks
|
||||||
|
#. A lock handover must be performed during the migration process where 2 QEMU
|
||||||
|
processes will have the same disk open concurrently.
|
||||||
|
#. The lock manager must be able to identify and kill the process accessing the
|
||||||
|
resource if the lock is revoked.
|
||||||
|
#. Locks can be acquired for arbitrary VM related resources, as determined by
|
||||||
|
the management application.
|
||||||
|
|
||||||
|
Design
|
||||||
|
------
|
||||||
|
|
||||||
|
Within a lock manager the following series of operations will need to be
|
||||||
|
supported.
|
||||||
|
|
||||||
|
- **Register object** Register the identity of an object against which locks
|
||||||
|
will be acquired
|
||||||
|
- **Add resource** Associate a resource with an object for future lock
|
||||||
|
acquisition / release
|
||||||
|
- **Acquire locks** Acquire the locks for all resources associated with the
|
||||||
|
object
|
||||||
|
- **Release locks** Release the locks for all resources associated with the
|
||||||
|
object
|
||||||
|
- **Inquire locks** Get a representation of the state of the locks for all
|
||||||
|
resources associated with the object
|
||||||
|
|
||||||
|
Plugin Implementations
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
Lock manager implementations are provided as LGPLv2+ licensed, dlopen()able
|
||||||
|
library modules. The plugins will be loadable from the following location:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
/usr/{lib,lib64}/libvirt/lock_manager/$NAME.so
|
||||||
|
|
||||||
|
The lock manager plugin must export a single ELF symbol named
|
||||||
|
``virLockDriverImpl``, which is a static instance of the ``virLockDriver``
|
||||||
|
struct. The struct is defined in the header file
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
#include <libvirt/plugins/lock_manager.h>
|
||||||
|
|
||||||
|
All callbacks in the struct must be initialized to non-NULL pointers. The
|
||||||
|
semantics of each callback are defined in the API docs embedded in the
|
||||||
|
previously mentioned header file
|
||||||
|
|
||||||
|
QEMU Driver integration
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
With the QEMU driver, the lock plugin will be set in the
|
||||||
|
``/etc/libvirt/qemu.conf`` configuration file by specifying the lock manager
|
||||||
|
name.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
lockManager="sanlock"
|
||||||
|
|
||||||
|
By default the lock manager will be a 'no op' implementation for backwards
|
||||||
|
compatibility
|
||||||
|
|
||||||
|
Lock usage patterns
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
The following pseudo code illustrates the common patterns of operations invoked
|
||||||
|
on the lock manager plugin callbacks.
|
||||||
|
|
||||||
|
Lock acquisition
|
||||||
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Initial lock acquisition will be performed from the process that is to own the
|
||||||
|
lock. This is typically the QEMU child process, in between the fork+exec
|
||||||
|
pairing. When adding further resources on the fly, to an existing object holding
|
||||||
|
locks, this will be done from the libvirtd process.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
virLockManagerParam params[] = {
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
||||||
|
.key = "uuid",
|
||||||
|
},
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
||||||
|
.key = "name",
|
||||||
|
.value = { .str = dom->def->name },
|
||||||
|
},
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||||
|
.key = "id",
|
||||||
|
.value = { .i = dom->def->id },
|
||||||
|
},
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||||
|
.key = "pid",
|
||||||
|
.value = { .i = dom->pid },
|
||||||
|
},
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
||||||
|
.key = "uri",
|
||||||
|
.value = { .cstr = driver->uri },
|
||||||
|
},
|
||||||
|
};
|
||||||
|
mgr = virLockManagerNew(lockPlugin,
|
||||||
|
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
||||||
|
ARRAY_CARDINALITY(params),
|
||||||
|
params,
|
||||||
|
0)));
|
||||||
|
|
||||||
|
foreach (initial disks)
|
||||||
|
virLockManagerAddResource(mgr,
|
||||||
|
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
||||||
|
$path, 0, NULL, $flags);
|
||||||
|
|
||||||
|
if (virLockManagerAcquire(lock, NULL, 0) < 0);
|
||||||
|
...abort...
|
||||||
|
|
||||||
|
Lock release
|
||||||
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
|
The locks are all implicitly released when the process that acquired them exits,
|
||||||
|
however, a process may voluntarily give up the lock by running
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
char *state = NULL;
|
||||||
|
virLockManagerParam params[] = {
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UUID,
|
||||||
|
.key = "uuid",
|
||||||
|
},
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_STRING,
|
||||||
|
.key = "name",
|
||||||
|
.value = { .str = dom->def->name },
|
||||||
|
},
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||||
|
.key = "id",
|
||||||
|
.value = { .i = dom->def->id },
|
||||||
|
},
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_UINT,
|
||||||
|
.key = "pid",
|
||||||
|
.value = { .i = dom->pid },
|
||||||
|
},
|
||||||
|
{ .type = VIR_LOCK_MANAGER_PARAM_TYPE_CSTRING,
|
||||||
|
.key = "uri",
|
||||||
|
.value = { .cstr = driver->uri },
|
||||||
|
},
|
||||||
|
};
|
||||||
|
mgr = virLockManagerNew(lockPlugin,
|
||||||
|
VIR_LOCK_MANAGER_TYPE_DOMAIN,
|
||||||
|
ARRAY_CARDINALITY(params),
|
||||||
|
params,
|
||||||
|
0)));
|
||||||
|
|
||||||
|
foreach (initial disks)
|
||||||
|
virLockManagerAddResource(mgr,
|
||||||
|
VIR_LOCK_MANAGER_RESOURCE_TYPE_DISK,
|
||||||
|
$path, 0, NULL, $flags);
|
||||||
|
|
||||||
|
virLockManagerRelease(mgr, & state, 0);
|
||||||
|
|
||||||
|
The returned state string can be passed to the ``virLockManagerAcquire`` method
|
||||||
|
to later re-acquire the exact same locks. This state transfer is commonly used
|
||||||
|
when performing live migration of virtual machines. By validating the state the
|
||||||
|
lock manager can ensure no other VM has re-acquire the same locks on a different
|
||||||
|
host. The state can also be obtained without releasing the locks, by calling the
|
||||||
|
``virLockManagerInquire`` method.
|
||||||
@ -2,6 +2,7 @@ docs_kbase_internals_files = [
|
|||||||
'command',
|
'command',
|
||||||
'eventloop',
|
'eventloop',
|
||||||
'incremental-backup',
|
'incremental-backup',
|
||||||
|
'locking',
|
||||||
'migration',
|
'migration',
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|||||||
@ -36,7 +36,6 @@ foreach name : docs_kbase_files
|
|||||||
}
|
}
|
||||||
endforeach
|
endforeach
|
||||||
|
|
||||||
# keep the XSLT processing code block in sync with docs/meson.build
|
|
||||||
|
|
||||||
# --- begin of XSLT processing ---
|
# --- begin of XSLT processing ---
|
||||||
|
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user