2015-02-16 11:58:32 +00:00
|
|
|
<?xml version="1.0" encoding="UTF-8"?>
|
|
|
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
|
|
|
<body>
|
|
|
|
|
<h1>Virtual machine lock manager, virtlockd plugin</h1>
|
|
|
|
|
|
|
|
|
|
<ul id="toc"></ul>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
This page describes use of the <code>virtlockd</code>
|
|
|
|
|
service as a <a href="locking.html">lock driver</a>
|
|
|
|
|
plugin for virtual machine disk mutual exclusion.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h2><a name="background">virtlockd background</a></h2>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The virtlockd daemon is a single purpose binary which
|
|
|
|
|
focuses exclusively on the task of acquiring and holding
|
|
|
|
|
locks on behalf of running virtual machines. It is
|
|
|
|
|
designed to offer a low overhead, portable locking
|
|
|
|
|
scheme can be used out of the box on virtualization
|
|
|
|
|
hosts with minimal configuration overheads. It makes
|
|
|
|
|
use of the POSIX fcntl advisory locking capability
|
|
|
|
|
to hold locks, which is supported by the majority of
|
|
|
|
|
commonly used filesystems.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h2><a name="sanlock">virtlockd daemon setup</a></h2>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
In most OS, the virtlockd daemon itself will not require
|
|
|
|
|
any upfront configuration work. It is installed by default
|
|
|
|
|
when libvirtd is present, and a systemd socket unit is
|
|
|
|
|
registered such that the daemon will be automatically
|
|
|
|
|
started when first required. With OS that predate systemd
|
|
|
|
|
though, it will be necessary to start it at boot time,
|
|
|
|
|
prior to libvirtd being started. On RHEL/Fedora distros,
|
|
|
|
|
this can be achieved as follows
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
# chkconfig virtlockd on
|
|
|
|
|
# service virtlockd start
|
2015-02-16 11:58:32 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The above instructions apply to the instance of virtlockd
|
|
|
|
|
that runs privileged, and is used by the libvirtd daemon
|
|
|
|
|
that runs privileged. If running libvirtd as an unprivileged
|
|
|
|
|
user, it will always automatically spawn an instance of
|
|
|
|
|
the virtlockd daemon unprivileged too. This requires no
|
|
|
|
|
setup at all.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h2><a name="lockdplugin">libvirt lockd plugin configuration</a></h2>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Once the virtlockd daemon is running, or setup to autostart,
|
|
|
|
|
the next step is to configure the libvirt lockd plugin.
|
|
|
|
|
There is a separate configuration file for each libvirt
|
|
|
|
|
driver that is using virtlockd. For QEMU, we will edit
|
|
|
|
|
<code>/etc/libvirt/qemu-lockd.conf</code>
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The default behaviour of the lockd plugin is to acquire locks
|
|
|
|
|
directly on the virtual disk images associated with the guest
|
|
|
|
|
<disk> elements. This ensures it can run out of the box
|
|
|
|
|
with no configuration, providing locking for disk images on
|
|
|
|
|
shared filesystems such as NFS. It does not provide any cross
|
|
|
|
|
host protection for storage that is backed by block devices,
|
|
|
|
|
since locks acquired on device nodes in /dev only apply within
|
|
|
|
|
the host. It may also be the case that the filesystem holding
|
|
|
|
|
the disk images is not capable of supporting fcntl locks.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
To address these problems it is possible to tell lockd to
|
|
|
|
|
acquire locks on an indirect file. Essentially lockd will
|
|
|
|
|
calculate the SHA256 checksum of the fully qualified path,
|
|
|
|
|
and create a zero length file in a given directory whose
|
|
|
|
|
filename is the checksum. It will then acquire a lock on
|
|
|
|
|
that file. Assuming the block devices assigned to the guest
|
|
|
|
|
are using stable paths (eg /dev/disk/by-path/XXXXXXX) then
|
|
|
|
|
this will allow for locks to apply across hosts. This
|
|
|
|
|
feature can be enabled by setting a configuration setting
|
|
|
|
|
that specifies the directory in which to create the lock
|
|
|
|
|
files. The directory referred to should of course be
|
|
|
|
|
placed on a shared filesystem (eg NFS) that is accessible
|
|
|
|
|
to all hosts which can see the shared block devices.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
$ su - root
|
|
|
|
|
# augtool -s set \
|
|
|
|
|
/files/etc/libvirt/qemu-lockd.conf/file_lockspace_dir \
|
|
|
|
|
"/var/lib/libvirt/lockd/files"
|
2015-02-16 11:58:32 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
If the guests are using either LVM and SCSI block devices
|
|
|
|
|
for their virtual disks, there is a unique identifier
|
|
|
|
|
associated with each device. It is possible to tell lockd
|
|
|
|
|
to use this UUID as the basis for acquiring locks, rather
|
|
|
|
|
than the SHA256 sum of the filename. The benefit of this
|
|
|
|
|
is that the locking protection will work even if the file
|
|
|
|
|
paths to the given block device are different on each
|
|
|
|
|
host.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
$ su - root
|
|
|
|
|
# augtool -s set \
|
|
|
|
|
/files/etc/libvirt/qemu-lockd.conf/scsi_lockspace_dir \
|
|
|
|
|
"/var/lib/libvirt/lockd/scsi"
|
|
|
|
|
# augtool -s set \
|
|
|
|
|
/files/etc/libvirt/qemu-lockd.conf/lvm_lockspace_dir \
|
|
|
|
|
"/var/lib/libvirt/lockd/lvm"
|
2015-02-16 11:58:32 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
It is important to remember that the changes made to the
|
|
|
|
|
<code>/etc/libvirt/qemu-lockd.conf</code> file must be
|
|
|
|
|
propagated to all hosts before any virtual machines are
|
|
|
|
|
launched on them. This ensures that all hosts are using
|
|
|
|
|
the same locking mechanism
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h2><a name="qemuconfig">QEMU/KVM driver configuration</a></h2>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The QEMU driver is capable of using the virtlockd plugin
|
|
|
|
|
since the release <span>1.0.2</span>.
|
|
|
|
|
The out of the box configuration, however, currently
|
|
|
|
|
uses the <strong>nop</strong> lock manager plugin.
|
|
|
|
|
To get protection for disks, it is thus necessary
|
|
|
|
|
to reconfigure QEMU to activate the <strong>lockd</strong>
|
|
|
|
|
driver. This is achieved by editing the QEMU driver
|
|
|
|
|
configuration file (<code>/etc/libvirt/qemu.conf</code>)
|
|
|
|
|
and changing the <code>lock_manager</code> configuration
|
|
|
|
|
tunable.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
$ su - root
|
|
|
|
|
# augtool -s set /files/etc/libvirt/qemu.conf/lock_manager lockd
|
|
|
|
|
# service libvirtd restart
|
2015-02-16 11:58:32 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Every time you start a guest, the virtlockd daemon will acquire
|
|
|
|
|
locks on the disk files directly, or in one of the configured
|
|
|
|
|
lookaside directories based on SHA256 sum. To check that locks
|
|
|
|
|
are being acquired as expected, the <code>lslocks</code> tool
|
|
|
|
|
can be run.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
