backup: Document new XML for backups

Prepare for new backup APIs by describing the XML that will represent
a backup.  The XML resembles snapshots and checkpoints in being able
to select actions for a set of disks, but has other differences.  It
can support both push model (the hypervisor does the backup directly
into the destination file) and pull model (the hypervisor exposes an
access port for a third party to grab what is necessary).  Add
testsuite coverage for some minimal uses of the XML.

The <disk> element within <domainbackup> tries to model the same
elements as a <disk> under <domain>, but sharing the RNG grammar
proved to be hairy. That is in part because while <domain> use
<source> to describe a host resource in use by the guest, a backup job
is using a host resource that is not visible to the guest: a push
backup action is instead describing a <target> (which ultimately could
be a remote network resource, but for simplicity the RNG just
validates a local file for now), and a pull backup action is instead
describing a temporary local file <scratch> (which probably should not
be a remote resource).  A future refactoring may thus introduce some
way to parameterize RNG to accept <disk type='FOO'>...</disk> so that
the name of the subelement can be <source> for domain, or <target> or
<scratch> as needed for backups. Future patches may improve this area
of code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This commit is contained in:
Eric Blake 2019-08-21 20:42:41 -05:00 committed by Peter Krempa
parent c3e18c13ca
commit 252958ee16
15 changed files with 470 additions and 8 deletions

View File

@ -85,7 +85,8 @@
<a href="formatnode.html">node devices</a>,
<a href="formatsecret.html">secrets</a>,
<a href="formatsnapshot.html">snapshots</a>,
<a href="formatcheckpoint.html">checkpoints</a></dd>
<a href="formatcheckpoint.html">checkpoints</a>,
<a href="formatbackup.html">backup jobs</a></dd>
<dt><a href="uri.html">URI format</a></dt>
<dd>The URI formats used for connecting to libvirt</dd>

View File

@ -27,6 +27,7 @@
<li><a href="formatsecret.html">Secrets</a></li>
<li><a href="formatsnapshot.html">Snapshots</a></li>
<li><a href="formatcheckpoint.html">Checkpoints</a></li>
<li><a href="formatbackup.html">Backup jobs</a></li>
</ul>
<h2>Command line validation</h2>

175
docs/formatbackup.html.in Normal file
View File

@ -0,0 +1,175 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<body>
<h1>Backup XML format</h1>
<ul id="toc"></ul>
<h2><a id="BackupAttributes">Backup XML</a></h2>
<p>
Creating a backup, whether full or incremental, is done
via <code>virDomainBackupBegin()</code>, which takes an XML
description of the actions to perform, as well as an optional
second XML document <a href="formatcheckpoint.html">describing a
checkpoint</a> to create at the same point in time. See
also <a href="domainstatecapture.html">a comparison</a> between
the various state capture APIs.
</p>
<p>
There are two general modes for backups: a push mode (where the
hypervisor writes out the data to the destination file, which
may be local or remote), and a pull mode (where the hypervisor
creates an NBD server that a third-party client can then read as
needed, and which requires the use of temporary storage,
typically local, until the backup is complete).
</p>
<p>
The instructions for beginning a backup job are provided as
attributes and elements of the
top-level <code>domainbackup</code> element. This element
includes an optional attribute <code>mode</code> which can be
either "push" or "pull" (default
push). <code>virDomainBackupGetXMLDesc()</code> can be used to
see the actual values selected for elements omitted during
creation (for example, learning which port the NBD server is
using in the pull model or what file names libvirt generated
when none were supplied). The following child elements and attributes
are supported:
</p>
<dl>
<dt><code>incremental</code></dt>
<dd>An optional element giving the name of an existing
checkpoint of the domain, which will be used to make this
backup an incremental one. In the push model, only changes
since the named checkpoint are written to the destination. In
the pull model, the NBD server uses the
NBD_OPT_SET_META_CONTEXT extension to advertise to the client
which portions of the export contain changes since the named
checkpoint. If omitted, a full backup is performed.
</dd>
<dt><code>server</code></dt>
<dd>Present only for a pull mode backup. Contains the same
attributes as
the <a href="formatdomain.html#elementsDisks"><code>protocol</code>
element of a disk</a> attached via NBD in the domain (such as
transport, socket, name, port, or tls), necessary to set up an
NBD server that exposes the content of each disk at the time
the backup is started.
</dd>
<dt><code>disks</code></dt>
<dd>An optional listing of instructions for disks participating
in the backup (if omitted, all disks participate and libvirt
attempts to generate filenames by appending the current
timestamp as a suffix). If the entire element was omitted on
input, then all disks participate in the backup, otherwise,
only the disks explicitly listed which do not also
use <code>backup='no'</code> will participate. On output, this
is the state of each of the domain's disk in relation to the
backup operation.
<dl>
<dt><code>disk</code></dt>
<dd>This sub-element describes the backup properties of a
specific disk, with the following attributes and child
elements:
<dl>
<dt><code>name</code></dt>
<dd>A mandatory attribute which must match
the <code>&lt;target dev='name'/&gt;</code>
of one of
the <a href="formatdomain.html#elementsDisks">disk
devices</a> specified for the domain at the time of
the checkpoint.</dd>
<dt><code>backup</code></dt>
<dd>Setting this attribute to <code>yes</code>(default) specifies
that the disk should take part in the backup and using
<code>no</code> excludes the disk from the backup.</dd>
<dt><code>type</code></dt>
<dd>A mandatory attribute to describe the type of the
disk, except when <code>backup='no'</code> is
used. Valid values include <code>file</code>,
<code>block</code>, or <code>network</code>.
Similar to a disk declaration for a domain, the choice of type
controls what additional sub-elements are needed to describe
the destination (such as <code>protocol</code> for a
network destination).</dd>
<dt><code>target</code></dt>
<dd>Valid only for push mode backups, this is the
primary sub-element that describes the file name of
the backup destination, similar to
the <code>source</code> sub-element of a domain
disk. An optional sub-element <code>driver</code> can
also be used, with an attribute <code>type</code> to
specify a destination format different from
qcow2. </dd>
<dt><code>scratch</code></dt>
<dd>Valid only for pull mode backups, this is the
primary sub-element that describes the file name of
the local scratch file to be used in facilitating the
backup, and is similar to the <code>source</code>
sub-element of a domain disk. Currently only <code>file</code>
and <code>block</code> scratch storage is supported. The
<code>file</code> scratch file is created and deleted by
libvirt in the given location. A <code>block</code> scratch
device must exist prior to starting the backup and is formatted.
The block device must have enough space for the corresponding
disk data including format overhead.
If <code>VIR_DOMAIN_BACKUP_BEGIN_REUSE_EXTERNAL</code> flag is
used the file for a scratch of <code>file</code> type must
exist with the correct format and size to hold the copy and is
used without modification. The file is not deleted after the
backup but the contents of the file don't make sense outside
of the backup. The same applies for the block device which
must be formatted appropriately.</dd>
</dl>
</dd>
</dl>
</dd>
</dl>
<h2><a id="example">Examples</a></h2>
<p>Use <code>virDomainBackupBegin()</code> to perform a full
backup using push mode. The example lets libvirt pick the
destination and format for 'vda', fully specifies that we want a
raw backup of 'vdb', and omits 'vdc' from the operation.
</p>
<pre>
&lt;domainbackup&gt;
&lt;disks&gt;
&lt;disk name='vda' backup='yes'/&gt;
&lt;disk name='vdb' type='file'&gt;
&lt;target file='/path/to/vdb.backup'/&gt;
&lt;driver type='raw'/&gt;
&lt;/disk&gt;
&lt;disk name='vdc' backup='no'/&gt;
&lt;/disks&gt;
&lt;/domainbackup&gt;
</pre>
<p>If the previous full backup also passed a parameter describing
<a href="formatcheckpoint.html">checkpoint XML</a> that resulted
in a checkpoint named <code>1525889631</code>, we can make
another call to <code>virDomainBackupBegin()</code> to perform
an incremental backup of just the data changed since that
checkpoint, this time using the following XML to start a pull
model export of the 'vda' and 'vdb' disks, where a third-party
NBD client connecting to '/path/to/server' completes the backup
(omitting 'vdc' from the explicit list has the same effect as
the backup='no' from the previous example):
</p>
<pre>
&lt;domainbackup mode="pull"&gt;
&lt;incremental&gt;1525889631&lt;/incremental&gt;
&lt;server transport="unix" socket="/path/to/server"/&gt;
&lt;disks&gt;
&lt;disk name='vda' backup='yes' type='file'&gt;
&lt;scratch file='/path/to/file1.scratch'/&gt;
&lt;/disk&gt;
&lt;/disks&gt;
&lt;/domainbackup&gt;
</pre>
</body>
</html>

View File

@ -28,12 +28,12 @@
first checkpoint and the second backup operation), it is
possible to do an offline reconstruction of the state of the
disk at the time of the second backup without having to copy as
much data as a second full backup would require. Future API
additions will make it possible to create checkpoints in
conjunction with a backup
via <code>virDomainBackupBegin()</code> or with an external
snapshot via <code>virDomainSnapshotCreateXML2</code>; but for
now, libvirt exposes enough support to create disk checkpoints
much data as a second full backup would require. Most disk
checkpoints are created in conjunction with a backup
via <code>virDomainBackupBegin()</code>, although a future API
addition of <code>virDomainSnapshotCreateXML2()</code> will also
make this possible when creating external snapshots; however,
libvirt also exposes enough support to create disk checkpoints
independently from a backup operation
via <code>virDomainCheckpointCreateXML()</code> <span class="since">since
5.6.0</span>. Likewise, the creation of checkpoints when

View File

@ -59,7 +59,8 @@
<a href="formatnode.html">node devices</a>,
<a href="formatsecret.html">secrets</a>,
<a href="formatsnapshot.html">snapshots</a>,
<a href="formatcheckpoint.html">checkpoints</a></dd>
<a href="formatcheckpoint.html">checkpoints</a>,
<a href="formatbackup.html">backup jobs</a></dd>
<dt><a href="http://wiki.libvirt.org">Wiki</a></dt>
<dd>Read further community contributed content</dd>
</dl>

View File

@ -0,0 +1,223 @@
<?xml version="1.0"?>
<!-- A Relax NG schema for the libvirt domain backup properties XML format -->
<grammar xmlns="http://relaxng.org/ns/structure/1.0">
<start>
<ref name='domainbackup'/>
</start>
<include href='domaincommon.rng'/>
<define name='domainbackup'>
<element name='domainbackup'>
<interleave>
<optional>
<element name='incremental'>
<text/>
</element>
</optional>
<choice>
<group>
<optional>
<attribute name='mode'>
<value>push</value>
</attribute>
</optional>
<ref name='backupDisksPush'/>
</group>
<group>
<attribute name='mode'>
<value>pull</value>
</attribute>
<interleave>
<element name='server'>
<choice>
<group>
<optional>
<attribute name='transport'>
<value>tcp</value>
</attribute>
</optional>
<attribute name='name'>
<choice>
<ref name='dnsName'/>
<ref name='ipAddr'/>
</choice>
</attribute>
<optional>
<attribute name='port'>
<ref name='unsignedInt'/>
</attribute>
</optional>
<!-- add tls? -->
</group>
<group>
<attribute name='transport'>
<value>unix</value>
</attribute>
<attribute name='socket'>
<ref name='absFilePath'/>
</attribute>
</group>
</choice>
</element>
<ref name='backupDisksPull'/>
</interleave>
</group>
</choice>
</interleave>
</element>
</define>
<define name='backupPushDriver'>
<optional>
<element name='driver'>
<attribute name='type'>
<ref name='storageFormat'/>
</attribute>
</element>
</optional>
</define>
<define name='backupPullDriver'>
<optional>
<element name='driver'>
<attribute name='type'>
<value>qcow2</value>
</attribute>
</element>
</optional>
</define>
<define name='backupAttr'>
<optional>
<attribute name='backup'>
<choice>
<value>yes</value>
</choice>
</attribute>
</optional>
</define>
<define name='backupDisksPush'>
<optional>
<element name='disks'>
<oneOrMore>
<element name='disk'>
<attribute name='name'>
<choice>
<ref name='diskTarget'/>
</choice>
</attribute>
<choice>
<group>
<attribute name='backup'>
<value>no</value>
</attribute>
</group>
<group>
<ref name='backupAttr'/>
<attribute name='type'>
<value>file</value>
</attribute>
<interleave>
<optional>
<element name='target'>
<attribute name='file'>
<ref name='absFilePath'/>
</attribute>
<zeroOrMore>
<ref name='devSeclabel'/>
</zeroOrMore>
</element>
</optional>
<ref name='backupPushDriver'/>
</interleave>
</group>
<group>
<ref name='backupAttr'/>
<attribute name='type'>
<value>block</value>
</attribute>
<interleave>
<optional>
<element name='target'>
<attribute name='dev'>
<ref name='absFilePath'/>
</attribute>
<zeroOrMore>
<ref name='devSeclabel'/>
</zeroOrMore>
</element>
</optional>
<ref name='backupPushDriver'/>
</interleave>
</group>
</choice>
</element>
</oneOrMore>
</element>
</optional>
</define>
<define name='backupDisksPull'>
<optional>
<element name='disks'>
<oneOrMore>
<element name='disk'>
<attribute name='name'>
<choice>
<ref name='diskTarget'/>
</choice>
</attribute>
<choice>
<group>
<attribute name='backup'>
<value>no</value>
</attribute>
</group>
<group>
<optional>
<ref name='backupAttr'/>
<attribute name='type'>
<value>file</value>
</attribute>
</optional>
<optional>
<interleave>
<element name='scratch'>
<attribute name='file'>
<ref name='absFilePath'/>
</attribute>
<zeroOrMore>
<ref name='devSeclabel'/>
</zeroOrMore>
</element>
<ref name='backupPullDriver'/>
</interleave>
</optional>
</group>
<group>
<ref name='backupAttr'/>
<attribute name='type'>
<value>block</value>
</attribute>
<interleave>
<element name='scratch'>
<attribute name='dev'>
<ref name='absFilePath'/>
</attribute>
<zeroOrMore>
<ref name='devSeclabel'/>
</zeroOrMore>
</element>
<ref name='backupPullDriver'/>
</interleave>
</group>
</choice>
</element>
</oneOrMore>
</element>
</optional>
</define>
</grammar>

View File

@ -1886,6 +1886,7 @@ exit 0
%{_datadir}/libvirt/schemas/capability.rng
%{_datadir}/libvirt/schemas/cputypes.rng
%{_datadir}/libvirt/schemas/domain.rng
%{_datadir}/libvirt/schemas/domainbackup.rng
%{_datadir}/libvirt/schemas/domaincaps.rng
%{_datadir}/libvirt/schemas/domaincheckpoint.rng
%{_datadir}/libvirt/schemas/domaincommon.rng

View File

@ -238,6 +238,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh
%{mingw32_datadir}/libvirt/schemas/capability.rng
%{mingw32_datadir}/libvirt/schemas/cputypes.rng
%{mingw32_datadir}/libvirt/schemas/domain.rng
%{mingw32_datadir}/libvirt/schemas/domainbackup.rng
%{mingw32_datadir}/libvirt/schemas/domaincaps.rng
%{mingw32_datadir}/libvirt/schemas/domaincheckpoint.rng
%{mingw32_datadir}/libvirt/schemas/domaincommon.rng
@ -329,6 +330,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh
%{mingw64_datadir}/libvirt/schemas/capability.rng
%{mingw64_datadir}/libvirt/schemas/cputypes.rng
%{mingw64_datadir}/libvirt/schemas/domain.rng
%{mingw64_datadir}/libvirt/schemas/domainbackup.rng
%{mingw64_datadir}/libvirt/schemas/domaincaps.rng
%{mingw64_datadir}/libvirt/schemas/domaincheckpoint.rng
%{mingw64_datadir}/libvirt/schemas/domaincommon.rng

View File

@ -91,6 +91,7 @@ EXTRA_DIST = \
commanddata \
cputestdata \
domaincapsdata \
domainbackupxml2xmlin \
domainconfdata \
domainschemadata \
fchostdata \

View File

@ -0,0 +1,18 @@
<domainbackup mode="pull">
<incremental>1525889631</incremental>
<server transport='tcp' name='localhost' port='10809'/>
<disks>
<disk name='vda' type='file'>
<driver type='qcow2'/>
<scratch file='/path/to/file'>
<seclabel model='dac' relabel='no'/>
</scratch>
</disk>
<disk name='vdb' type='block'>
<driver type='qcow2'/>
<scratch dev='/dev/block'>
<seclabel model='dac' relabel='no'/>
</scratch>
</disk>
</disks>
</domainbackup>

View File

@ -0,0 +1,10 @@
<domainbackup mode="pull">
<incremental>1525889631</incremental>
<server transport='tcp' name='localhost' port='10809'/>
<disks>
<disk name='vda' type='file'>
<scratch file='/path/to/file'/>
</disk>
<disk name='hda' backup='no'/>
</disks>
</domainbackup>

View File

@ -0,0 +1,17 @@
<domainbackup mode="push">
<incremental>1525889631</incremental>
<disks>
<disk name='vda' type='file'>
<driver type='raw'/>
<target file='/path/to/file'>
<seclabel model='dac' relabel='no'/>
</target>
</disk>
<disk name='vdb' type='block'>
<driver type='qcow2'/>
<target dev='/dev/block'>
<seclabel model='dac' relabel='no'/>
</target>
</disk>
</disks>
</domainbackup>

View File

@ -0,0 +1,10 @@
<domainbackup mode="push">
<incremental>1525889631</incremental>
<disks>
<disk name='vda' type='file'>
<driver type='raw'/>
<target file='/path/to/file'/>
</disk>
<disk name='hda' backup='no'/>
</disks>
</domainbackup>

View File

@ -0,0 +1 @@
<domainbackup/>

View File

@ -205,6 +205,7 @@ mymain(void)
"genericxml2xmloutdata", "xlconfigdata", "libxlxml2domconfigdata",
"qemuhotplugtestdomains");
DO_TEST_DIR("domaincaps.rng", "domaincapsdata");
DO_TEST_DIR("domainbackup.rng", "domainbackupxml2xmlin");
DO_TEST_DIR("domaincheckpoint.rng", "qemudomaincheckpointxml2xmlin",
"qemudomaincheckpointxml2xmlout");
DO_TEST_DIR("domainsnapshot.rng", "qemudomainsnapshotxml2xmlin",