From d85842cfd7442ac0cc1f5a1f05c3e2a6921356a8 Mon Sep 17 00:00:00 2001 From: Eric Blake Date: Mon, 11 Jun 2018 22:12:21 -0500 Subject: [PATCH] backup: Document new XML for checkpoints MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Prepare for new checkpoint APIs by describing the XML that will represent a checkpoint. The checkpoint XML is modeled heavily after virDomainSnapshotPtr. See the docs for more details. Add testsuite coverage for some minimal uses of the XML (bare minimum, the sample from html, and a full dumpxml, and some counter-examples that should fail schema validation). Although use of the REDEFINE flag will require the subelement to be present, it is easier for most of the tests to provide counterpart output produced with the NO_DOMAIN flag (particularly since synthesizing a valid during testing is not trivial). Signed-off-by: Eric Blake Reviewed-by: Daniel P. Berrangé --- docs/docs.html.in | 3 +- docs/format.html.in | 1 + docs/formatcheckpoint.html.in | 194 ++++++++++++++++++ docs/index.html.in | 3 +- docs/schemas/domaincheckpoint.rng | 94 +++++++++ libvirt.spec.in | 1 + mingw-libvirt.spec.in | 2 + tests/Makefile.am | 2 + .../disk-default.xml | 7 + .../disk-invalid.xml | 8 + tests/qemudomaincheckpointxml2xmlin/empty.xml | 1 + .../name-invalid.xml | 4 + .../qemudomaincheckpointxml2xmlin/sample.xml | 7 + tests/qemudomaincheckpointxml2xmlin/size.xml | 4 + .../disk-default.xml | 11 + .../qemudomaincheckpointxml2xmlout/empty.xml | 7 + .../redefine.xml | 63 ++++++ .../qemudomaincheckpointxml2xmlout/sample.xml | 12 ++ tests/qemudomaincheckpointxml2xmlout/size.xml | 11 + tests/virschematest.c | 2 + 20 files changed, 435 insertions(+), 2 deletions(-) create mode 100644 docs/formatcheckpoint.html.in create mode 100644 docs/schemas/domaincheckpoint.rng create mode 100644 tests/qemudomaincheckpointxml2xmlin/disk-default.xml create mode 100644 tests/qemudomaincheckpointxml2xmlin/disk-invalid.xml create mode 100644 tests/qemudomaincheckpointxml2xmlin/empty.xml create mode 100644 tests/qemudomaincheckpointxml2xmlin/name-invalid.xml create mode 100644 tests/qemudomaincheckpointxml2xmlin/sample.xml create mode 100644 tests/qemudomaincheckpointxml2xmlin/size.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/disk-default.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/empty.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/redefine.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/sample.xml create mode 100644 tests/qemudomaincheckpointxml2xmlout/size.xml diff --git a/docs/docs.html.in b/docs/docs.html.in index 9e1ef6be41..74af38002e 100644 --- a/docs/docs.html.in +++ b/docs/docs.html.in @@ -81,7 +81,8 @@ storage pool capabilities, node devices, secrets, - snapshots + snapshots, + checkpoints
URI format
The URI formats used for connecting to libvirt
diff --git a/docs/format.html.in b/docs/format.html.in index b488f7b38f..3be2237663 100644 --- a/docs/format.html.in +++ b/docs/format.html.in @@ -26,6 +26,7 @@
  • Node devices
  • Secrets
  • Snapshots
    • +
  • Checkpoints

    • Command line validation

    diff --git a/docs/formatcheckpoint.html.in b/docs/formatcheckpoint.html.in new file mode 100644 index 0000000000..50dd4d284c --- /dev/null +++ b/docs/formatcheckpoint.html.in @@ -0,0 +1,194 @@ + + + + +

    Checkpoint XML format

    + +
      + +

      Checkpoint XML

      + +

      + One method of capturing domain disk backups is via the use of + incremental backups. Right now, incremental backups are only + supported for the QEMU hypervisor when using qcow2 disks at the + active layer; if other disk formats are in use, capturing disk + backups requires different libvirt APIs. +

      +

      + Libvirt is able to facilitate incremental backups by tracking + disk checkpoints, which are points in time against which it is + easy to compute which portion of the disk has changed. Given a + full backup (a backup created from the creation of the disk to a + given point in time), coupled with the creation of a disk + checkpoint at that time, and an incremental backup (a backup + created from just the dirty portion of the disk between the + first checkpoint and the second backup operation), it is + possible to do an offline reconstruction of the state of the + disk at the time of the second backup without having to copy as + much data as a second full backup would require. Future API + additions will make it possible to create checkpoints in + conjunction with a backup + via virDomainBackupBegin() or with an external + snapshot via virDomainSnapshotCreateXML2; but for + now, libvirt exposes enough support to create disk checkpoints + independently from a backup operation + via virDomainCheckpointCreateXML() since + 5.6.0. +

      +

      + Attributes of libvirt checkpoints are stored as child elements + of the domaincheckpoint element. At checkpoint + creation time, normally only + the name, description, + and disks elements are settable. The rest of the + fields are ignored on creation and will be filled in by libvirt + in for informational purposes + by virDomainCheckpointGetXMLDesc(). However, when + redefining a checkpoint, with + the VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE flag + of virDomainCheckpointCreateXML(), all of the XML + fields described here are relevant on input, even the fields + that are normally described as readonly for output. +

      +

      + The top-level domaincheckpoint element may contain + the following elements: +

      +
      +
      name
      +
      The optional name for this checkpoint. If the name is + omitted, libvirt will create a name based on the time of the + creation. +
      +
      description
      +
      An optional human-readable description of the checkpoint. + If the description is omitted when initially creating the + checkpoint, then this field will be empty. +
      +
      disks
      +
      On input, this is an optional listing of specific + instructions for disk checkpoints; it is needed when making a + checkpoint on only a subset of the disks associated with a + domain. In particular, since QEMU checkpoints require qcow2 + disks, this element may be needed on input for excluding guest + disks that are not in qcow2 format. If the entire element was + omitted on input, then all disks participate in the + checkpoint, otherwise, only the disks explicitly listed which + do not also use checkpoint='no' will + participate. On output, this is the checkpoint state of each + of the domain's disks. +
      +
      disk
      +
      This sub-element describes the checkpoint properties of + a specific disk with the following attributes: +
      +
      name
      +
      A mandatory attribute which must match either + the <target dev='name'/> or an + unambiguous <source file='name'/> + of one of + the disk + devices specified for the domain at the time of + the checkpoint.
      +
      checkpoint
      +
      An optional attribute; possible values + are no when the disk does not participate + in this checkpoint; or bitmap if the disk + will track all changes since the creation of this + checkpoint via a bitmap.
      +
      bitmap
      +
      The attribute bitmap is only valid + if checkpoint='bitmap'; it describes the + name of the tracking bitmap (defaulting to the + checkpoint name).
      +
      size
      +
      The attribute size is ignored on input; + on output, it is only present if + the VIR_DOMAIN_CHECKPOINT_XML_SIZE flag + was used to perform a dynamic query of the estimated + size in bytes of the changes made since the checkpoint + was created.
      +
      +
      +
      +
      +
      creationTime
      +
      A readonly representation of the time this checkpoint was + created. The time is specified in seconds since the Epoch, + UTC (i.e. Unix time). +
      +
      parent
      +
      Readonly, present if this checkpoint has a parent. The + parent name is given by the sub-element name. The + parent relationship allows tracking a list of related checkpoints. +
      +
      domain
      +
      A readonly representation of the + inactive domain configuration + at the time the checkpoint was created. This element may be + omitted for output brevity by supplying + the VIR_DOMAIN_CHECKPOINT_XML_NO_DOMAIN flag, but + the resulting XML is no longer viable for use with + the VIR_DOMAIN_CHECKPOINT_CREATE_REDEFINE flag + of virDomainCheckpointCreateXML(). The domain + will have security-sensitive information omitted unless the + flag VIR_DOMAIN_CHECKPOINT_XML_SECURE is provided + on a read-write connection. +
      +
      + +

      Examples

      + +

      Using this XML to create a checkpoint of just vda on a qemu + domain with two disks and a prior checkpoint:

      + 
      +<domaincheckpoint>
+  <description>Completion of updates after OS install</description>
+  <disks>
+    <disk name='vda' checkpoint='bitmap'/>
+    <disk name='vdb' checkpoint='no'/>
+  </disks>
+</domaincheckpoint>
      + +

      will result in XML similar to this from + virDomainCheckpointGetXMLDesc():

      + 
      +<domaincheckpoint>
+  <name>1525889631</name>
+  <description>Completion of updates after OS install</description>
+  <parent>
+    <name>1525111885</name>
+  </parent>
+  <creationTime>1525889631</creationTime>
+  <disks>
+    <disk name='vda' checkpoint='bitmap' bitmap='1525889631'/>
+    <disk name='vdb' checkpoint='no'/>
+  </disks>
+  <domain type='qemu'>
+    <name>fedora</name>
+    <uuid>93a5c045-6457-2c09-e56c-927cdf34e178</uuid>
+    <memory>1048576</memory>
+    ...
+    <devices>
+      <disk type='file' device='disk'>
+        <driver name='qemu' type='qcow2'/>
+        <source file='/path/to/file1'/>
+        <target dev='vda' bus='virtio'/>
+      </disk>
+      <disk type='file' device='disk' snapshot='external'>
+        <driver name='qemu' type='raw'/>
+        <source file='/path/to/file2'/>
+        <target dev='vdb' bus='virtio'/>
+      </disk>
+      ...
+    </devices>
+  </domain>
+</domaincheckpoint>
      + +

      With that checkpoint created, the qcow2 image is now tracking + all changes that occur in the image since the checkpoint via + the persistent bitmap named 1525889631. +

      + + diff --git a/docs/index.html.in b/docs/index.html.in index 2bd647f8dd..7d0ab650e3 100644 --- a/docs/index.html.in +++ b/docs/index.html.in @@ -58,7 +58,8 @@ storage pool capabilities, node devices, secrets, - snapshots + snapshots, + checkpoints
      Wiki
      Read further community contributed content
      diff --git a/docs/schemas/domaincheckpoint.rng b/docs/schemas/domaincheckpoint.rng new file mode 100644 index 0000000000..4399eca7fc --- /dev/null +++ b/docs/schemas/domaincheckpoint.rng @@ -0,0 +1,94 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + no + + + + + bitmap + + + + + + + + + + + + + + + + + + + + + [a-zA-Z0-9_\-][a-zA-Z0-9_\-.]* + + + + diff --git a/libvirt.spec.in b/libvirt.spec.in index b13b863928..4dee2d8ac2 100644 --- a/libvirt.spec.in +++ b/libvirt.spec.in @@ -1802,6 +1802,7 @@ exit 0 %{_datadir}/libvirt/schemas/cputypes.rng %{_datadir}/libvirt/schemas/domain.rng %{_datadir}/libvirt/schemas/domaincaps.rng +%{_datadir}/libvirt/schemas/domaincheckpoint.rng %{_datadir}/libvirt/schemas/domaincommon.rng %{_datadir}/libvirt/schemas/domainsnapshot.rng %{_datadir}/libvirt/schemas/interface.rng diff --git a/mingw-libvirt.spec.in b/mingw-libvirt.spec.in index ad6d6e952e..df89d9f8b3 100644 --- a/mingw-libvirt.spec.in +++ b/mingw-libvirt.spec.in @@ -232,6 +232,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh %{mingw32_datadir}/libvirt/schemas/cputypes.rng %{mingw32_datadir}/libvirt/schemas/domain.rng %{mingw32_datadir}/libvirt/schemas/domaincaps.rng +%{mingw32_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw32_datadir}/libvirt/schemas/domaincommon.rng %{mingw32_datadir}/libvirt/schemas/domainsnapshot.rng %{mingw32_datadir}/libvirt/schemas/interface.rng @@ -321,6 +322,7 @@ rm -rf $RPM_BUILD_ROOT%{mingw64_libexecdir}/libvirt-guests.sh %{mingw64_datadir}/libvirt/schemas/cputypes.rng %{mingw64_datadir}/libvirt/schemas/domain.rng %{mingw64_datadir}/libvirt/schemas/domaincaps.rng +%{mingw64_datadir}/libvirt/schemas/domaincheckpoint.rng %{mingw64_datadir}/libvirt/schemas/domaincommon.rng %{mingw64_datadir}/libvirt/schemas/domainsnapshot.rng %{mingw64_datadir}/libvirt/schemas/interface.rng diff --git a/tests/Makefile.am b/tests/Makefile.am index f480e68e7d..9e2e57459e 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -110,6 +110,8 @@ EXTRA_DIST = \ qemublocktestdata \ qemucapabilitiesdata \ qemucaps2xmloutdata \ + qemudomaincheckpointxml2xmlin \ + qemudomaincheckpointxml2xmlout \ qemudomainsnapshotxml2xmlin \ qemudomainsnapshotxml2xmlout \ qemuhotplugtestcpus \ diff --git a/tests/qemudomaincheckpointxml2xmlin/disk-default.xml b/tests/qemudomaincheckpointxml2xmlin/disk-default.xml new file mode 100644 index 0000000000..706a9a8fd5 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/disk-default.xml @@ -0,0 +1,7 @@ + + Completion of updates after OS install + + + + + diff --git a/tests/qemudomaincheckpointxml2xmlin/disk-invalid.xml b/tests/qemudomaincheckpointxml2xmlin/disk-invalid.xml new file mode 100644 index 0000000000..c57c8f6490 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/disk-invalid.xml @@ -0,0 +1,8 @@ + + bogus + + + + + diff --git a/tests/qemudomaincheckpointxml2xmlin/empty.xml b/tests/qemudomaincheckpointxml2xmlin/empty.xml new file mode 100644 index 0000000000..dc36449142 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/empty.xml @@ -0,0 +1 @@ + diff --git a/tests/qemudomaincheckpointxml2xmlin/name-invalid.xml b/tests/qemudomaincheckpointxml2xmlin/name-invalid.xml new file mode 100644 index 0000000000..b13d7b09e9 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/name-invalid.xml @@ -0,0 +1,4 @@ + + + ../escape + diff --git a/tests/qemudomaincheckpointxml2xmlin/sample.xml b/tests/qemudomaincheckpointxml2xmlin/sample.xml new file mode 100644 index 0000000000..70ed964e1e --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/sample.xml @@ -0,0 +1,7 @@ + + Completion of updates after OS install + + + + + diff --git a/tests/qemudomaincheckpointxml2xmlin/size.xml b/tests/qemudomaincheckpointxml2xmlin/size.xml new file mode 100644 index 0000000000..99ce84c7a0 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlin/size.xml @@ -0,0 +1,4 @@ + + c2 + something fun + diff --git a/tests/qemudomaincheckpointxml2xmlout/disk-default.xml b/tests/qemudomaincheckpointxml2xmlout/disk-default.xml new file mode 100644 index 0000000000..ea18252c17 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/disk-default.xml @@ -0,0 +1,11 @@ + + 1525889631 + Completion of updates after OS install + + 1525111885 + + 1525889631 + + + + diff --git a/tests/qemudomaincheckpointxml2xmlout/empty.xml b/tests/qemudomaincheckpointxml2xmlout/empty.xml new file mode 100644 index 0000000000..6fe6124ce4 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/empty.xml @@ -0,0 +1,7 @@ + + 1525889631 + 1525889631 + + + + diff --git a/tests/qemudomaincheckpointxml2xmlout/redefine.xml b/tests/qemudomaincheckpointxml2xmlout/redefine.xml new file mode 100644 index 0000000000..823ae1ee2d --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/redefine.xml @@ -0,0 +1,63 @@ + + c1 + 1553647812 + + + + + + + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + 1 + + hvm + + + + destroy + restart + destroy + + /usr/bin/qemu-system-i686 + + + + +
      + + + + + + +
      + + + + + +
      + + + + + +
      + + +
      + + +
      + + + + + + + + diff --git a/tests/qemudomaincheckpointxml2xmlout/sample.xml b/tests/qemudomaincheckpointxml2xmlout/sample.xml new file mode 100644 index 0000000000..91cfac27df --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/sample.xml @@ -0,0 +1,12 @@ + + 1525889631 + Completion of updates after OS install + + 1525111885 + + 1525889631 + + + + + diff --git a/tests/qemudomaincheckpointxml2xmlout/size.xml b/tests/qemudomaincheckpointxml2xmlout/size.xml new file mode 100644 index 0000000000..4f11cfa680 --- /dev/null +++ b/tests/qemudomaincheckpointxml2xmlout/size.xml @@ -0,0 +1,11 @@ + + c2 + something fun + + 1525111885 + + 1553648510 + + + + diff --git a/tests/virschematest.c b/tests/virschematest.c index 56ee6ec77e..d5a56d3f96 100644 --- a/tests/virschematest.c +++ b/tests/virschematest.c @@ -222,6 +222,8 @@ mymain(void) "genericxml2xmloutdata", "xlconfigdata", "libxlxml2domconfigdata", "qemuhotplugtestdomains"); DO_TEST_DIR("domaincaps.rng", "domaincapsschemadata"); + DO_TEST_DIR("domaincheckpoint.rng", "qemudomaincheckpointxml2xmlin", + "qemudomaincheckpointxml2xmlout"); DO_TEST_DIR("domainsnapshot.rng", "qemudomainsnapshotxml2xmlin", "qemudomainsnapshotxml2xmlout"); DO_TEST_DIR("interface.rng", "interfaceschemadata");