Earlier patches mentioned that the initial implementation will prevent
snapshots and checkpoints from being used on the same domain at once.
However, the actual restriction is done in this separate patch to make
it easier to lift that restriction via a revert, when we are finally
ready to tackle that integration in the future.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Time to actually issue the QMP transactions that create and delete
persistent checkpoints, resolving TODOs intentionally left earlier in
the series. For create, we only need one transaction: inside, we
visit all disks affected by the checkpoint, and create a new enabled
bitmap, as well as disabling the bitmap of the first ancestor
checkpoint (if any) that also had a bitmap. For deletion, we need
multiple QMP calls: for each disk, if there is an ancestor checkpoint
with a bitmap, then the bitmap must be merged (including activating
the ancestor bitmap if the leaf node changes), all before deleting the
bitmap from the checkpoint being removed.
Signed-off-by: Eric Blake <eblake@redhat.com>
Qemu bitmap operations require knowing the node name associated with
the format layer (the qcow2 file); as upcoming patches will be
grabbing that information frequently, make a helper function to access
it.
Another potential benefit of this function is that we have a single
place where we could insert a QMP node-name scraping call if we don't
currently know the node name, when -blockdev is not supported;
however, the goal is that we hopefully don't ever have to do that
because we instead scrape node names only at the point where they
change.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
A lot of this work heavily copies from the existing snapshot APIs.
What's more, this patch is (intentionally) very similar to the
checkpoint code just added in the test driver, to the point that qemu
checkpoints are not fully usable in this patch, but it at least
bisects and builds cleanly. The separation between patches is done
because the grunt work of saving and restoring XML and tracking
relations between checkpoints is common to the test driver, while the
later patch adding integration with QMP is specific to qemu.
Also note that the interlocking to prevent checkpoints and snapshots
from existing at the same time will be a separate patch, to make it
easier to revert that restriction when we finally round out the design
for supporting interaction between the two concepts.
Signed-off-by: Eric Blake <eblake@redhat.com>
This is similar to the existing directory for snapshots; the domain
will save one xml file per checkpoint, for reloading on the next
libvirtd restart. Fortunately, since checkpoints mandate RNG
validation, we are assured that the checkpoint name will be usable as
a file name (no abuse of '../escape' as a checkpoint name, for
example).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Introduce the handler for finalizing a block commit and active bloc
commit job which will allow to use it with blockdev.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Introduce the handler for finalizing a block pull job which will allow
to use it with blockdev.
This patch also contains some additional machinery which is required to
store all the relevant job data in the status XML which will also be
reused with other block job types.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
In case of an incoming migration we do not need to run swtpm_setup
with all the parameters but only want to get the benefit of it
creating a TPM state file for us that we can then label with an
SELinux label. The actual state will be overwritten by the in-
coming state. So we have to pass an indicator for incomingMigration
all the way to the command line parameter generation for swtpm_setup.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Similar to virsh-snapshots. Provides decent coverage of the checkpoint
API, the test driver implementation, and the virsh access to the API.
A later patch will worry about testing that snapshots and checkpoints
are mutually exclusive (in part so it is easier to revert that when we
finally implement the interaction and lift that restriction).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
A lot of this work heavily copies from the existing snapshot APIs.
The test driver doesn't really have to do anything more than just
expose an interface into libvirt metadata, making it possible to test
saving and restoring XML, and tracking relations between multiple
checkpoints.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Introduce a bunch of new virsh commands for managing checkpoints in
isolation. More commands are needed for performing incremental
backups, but these commands were easy to implement by modeling heavily
after virsh-snapshot.c. There is no need for checkpoint-revert or
checkpoint-current since those snapshot APIs have no checkpoint
counterpart. Similarly, it is not necessary to change which
checkpoint is current when redefining from XML, since until we
integrate checkpoints with snapshots, there is only a linear chain
(and you can deduce the current checkpoint by instead using
'checkpoint-list --leaves'). Other aspects of checkpoint-list are
also a bit simpler than the snapshot counterpart, in part because we
don't have to cater to back-compat to older API.
Upcoming patches will test these interfaces once the test driver
supports checkpoints.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The remote code generator had to be taught about the new
virDomainCheckpointPtr type, at which point the remote driver code for
checkpoints can be generated.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Creating a checkpoint does not modify guest-visible state,
but does modify host resources. Rather than reuse existing
domain:write, domain:block_write, or domain:snapshot access
controls, it seems better to introduce a new access control
specific to tasks related to checkpoints and incremental
backups of guest disk state.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Wire up the use of a checkpoint list into each domain, similar to the
existing snapshot list. This includes adding a function for checking
that a redefine operation fits in with the existing list, as well as
various filtering capabilities over the list contents.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Create a new file for managing a list of checkpoint objects, borrowing
heavily from existing virDomainSnapshotObjList paradigms.
Note that while snapshots definitely have a use case for multiple
children to a single parent (create a base snapshot, create a child
snapshot, revert to the base, then create another child snapshot),
it's harder to predict how checkpoints will play out with reverting to
prior points in time. Thus, in initial use, given a list of
checkpoints, you never have more than one child, and we can treat the
most-recent leaf node as the parent of the next node creation, without
having to expose a notion of a current node in XML or public API.
However, as the snapshot machinery is already generic, it is easier to
reuse the generic machinery that tracks relations between domain
moments than it is to open-code a new list-management scheme just for
checkpoints (hence, we still have internal functions related to a
current checkpoint, even though that has no observable effect
externally, as well as the addition of a function to easily find the
lone leaf in the list to use as the current checkpoint).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Add a new file checkpoint_conf.c that performs the translation to and
from new XML describing a checkpoint. The code shares a common base
class with snapshots, since a checkpoint similarly represents the
domain state at a moment in time. Add some basic testing of round trip
XML handling through the new code.
Of note - this code intentionally differs from snapshots in that XML
schema validation is unconditional, rather than based on a public API
flag. We have many existing interfaces that still need to add a flag
for opt-in schema validation, but those interfaces have existing
clients that may not have been producing strictly-compliant XML, or we
may still uncover bugs where our RNG grammar is inconsistent with our
code (where omitting the opt-in flag allows existing apps to keep
working while waiting for an RNG patch). But since checkpoints are
brand-new, it's easier to ensure the code matches the schema by always
using the schema. If needed, a later patch could extend the API and
add a flag to turn on to request schema validation, rather than having
it forced (possibly just the validation of the <domain> sub-element
during REDEFINE) - but if a user encounters XML that looks like it
should be good but fails to validate with our RNG schema, they would
either have to upgrade to a new libvirt that adds the new flag, or
upgrade to a new libvirt that fixes the RNG schema, which implies
adding such a flag won't help much.
Also, the redefine flag requires the <domain> sub-element to be
present, rather than catering to historical back-compat to older
versions.
Signed-off-by: Eric Blake <eblake@redhat.com>
Now that various new API have been added or are coming soon, it is
worth a landing page that gives an overview of capturing various
pieces of guest state, and which APIs are best suited to which tasks.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Introduce a bunch of new public APIs related to backup checkpoints.
Checkpoints are modeled heavily after virDomainSnapshotPtr (both
represent a point in time of the guest), although a snapshot exists
with the intent of rolling back to that state, while a checkpoint
exists to make it possible to create an incremental backup at a later
time. We may have a future hypervisor that can completely manage
checkpoints without libvirt metadata, but the first two planned
hypervisors (qemu and test) both always use libvirt for tracking
metadata relations between checkpoints, so for now, I've deferred
the counterpart of virDomainSnapshotHasMetadata for a separate
API addition at a later date if there is ever a need for it.
Note that until we allow snapshots and checkpoints to exist
simultaneously on the same domain (although the actual prevention of
this will be in a separate patch for the sake of an easier revert down
the road), that it is not possible to branch out to create more than
one checkpoint child to a given parent, although it may become
possible later when we revert to a snapshot that coincides with a
checkpoint. This also means that for now, the decision of which
checkpoint becomes the parent of a newly created one is the only
checkpoint with no child (so while there are APIs for dealing with a
current snapshot, we do not need those for checkpoints). We may end
up exposing a notion of a current checkpoint later, but it's easier to
add stuff when proven needed than to blindly support it now and wish
we hadn't exposed it.
The following map shows the API relations to snapshots, with new APIs
on the right:
Operate on a domain object to create/redefine a child:
virDomainSnapshotCreateXML virDomainCheckpointCreateXML
Operate on a child object for lifetime management:
virDomainSnapshotDelete virDomainCheckpointDelete
virDomainSnapshotFree virDomainCheckpointFree
virDomainSnapshotRef virDomainCheckpointRef
Operate on a child object to learn more about it:
virDomainSnapshotGetXMLDesc virDomainCheckpointGetXMLDesc
virDomainSnapshotGetConnect virDomainCheckpointGetConnect
virDomainSnapshotGetDomain virDomainCheckpointGetDomain
virDomainSnapshotGetName virDomainCheckpiontGetName
virDomainSnapshotGetParent virDomainCheckpiontGetParent
virDomainSnapshotHasMetadata (deferred for later)
virDomainSnapshotIsCurrent (no counterpart, see note above)
Operate on a domain object to list all children:
virDomainSnapshotNum (no counterparts, these are the old
virDomainSnapshotListNames racy interfaces)
virDomainSnapshotListAllSnapshots virDomainListAllCheckpoints
Operate on a child object to list descendents:
virDomainSnapshotNumChildren (no counterparts, these are the old
virDomainSnapshotListChildrenNames racy interfaces)
virDomainSnapshotListAllChildren virDomainCheckpointListAllChildren
Operate on a domain to locate a particular child:
virDomainSnapshotLookupByName virDomainCheckpointLookupByName
virDomainSnapshotCurrent (no counterpart, see note above)
virDomainHasCurrentSnapshot (no counterpart, old racy interface)
Operate on a snapshot to roll back to earlier state:
virDomainSnapshotRevert (no counterpart, instead checkpoints
are used in incremental backups via
XML to virDomainBackupBegin)
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Prepare for new checkpoint APIs by describing the XML that will
represent a checkpoint. The checkpoint XML is modeled heavily after
virDomainSnapshotPtr. See the docs for more details.
Add testsuite coverage for some minimal uses of the XML (bare minimum,
the sample from html, and a full dumpxml, and some counter-examples
that should fail schema validation). Although use of the REDEFINE flag
will require the <domain> subelement to be present, it is easier for
most of the tests to provide counterpart output produced with the
NO_DOMAIN flag (particularly since synthesizing a valid <domain>
during testing is not trivial).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
If we are using -blockdev, then node names are always available
(because we set them). But when not using it, we have to scrape node
names from QMP, and want to do so as infrequently as possible. We
were scraping node names after reconnecting a new libvirtd to an
existing guest (see qemuProcessReconnect), and after any block job
that may have changed the set of node names we care about (legacy
block jobs), but forgot to scrape the names when first starting a
guest. Do so now in order to allow the checkpoint code to always have
access to a node name without having to repeat a node name scrape
itself.
Future patches may need to clean up qemuDomainSetBlockThreshold (if
node names are always available, then it doesn't need to repeat a
scrape) and/or hotplug and media changes (if the addition of new nodes
can result in a null node name, then scraping at that point in time
would be appropriate). But for now, this patch addresses only the
most common instance of a missing node name.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Since we are checking the 2nd parameter in the function for NULL,
we need to remove ATTRIBUTE_NONNULL(2) from the prototype.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Message-Id: <20190726205633.2041912-5-stefanb@linux.vnet.ibm.com>
Remove the ATTRIBUTE_NONNULL(1) from virCommandSetSendBuffer()
prototype since we are checking for '!cmd' and move the initialization
if 'i' after the test for '!cmd'.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Message-Id: <20190726205633.2041912-4-stefanb@linux.vnet.ibm.com>
Fix a potential memory leak by calling virCommandFree() in the cleanup
section.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Message-Id: <20190726205633.2041912-3-stefanb@linux.vnet.ibm.com>
Use the existing variables rather then calling virTPMSwtpmXYZ().
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Message-Id: <20190726205633.2041912-2-stefanb@linux.vnet.ibm.com>
Create an empty log file if the log file was removed, otherwise the
transaction to set the security labels on the file will fail.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20190726210706.24440-3-stefanb@linux.ibm.com>
Set the transactionStarted to false if the commit failed. If this is not
done, then the failure path will report 'no transaction is set' and hide
more useful error reports.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20190726210706.24440-2-stefanb@linux.ibm.com>
Creating an 'exp' output file, but never comparing it against the
actual output, does not actually constitute testing the output. :)
Fixes: 280a2b41e
Signed-off-by: Eric Blake <eblake@redhat.com>
Starting with QEMU 4.1 qemuMonitorCPUModelInfo structure in virQEMUCaps
stores only canonical feature names which may differ from the name used
by libvirt. We need translate these canonical names into libvirt names
for further consumption.
This fixes a bug in qemuConnectBaselineHypervisorCPU which would remove
all features for which libvirt's spelling differs from the QEMU's
preferred name. For example, the following result of
qemuConnectBaselineHypervisorCPU on my host with QEMU 4.1 is wrong:
<cpu mode='custom' match='exact'>
<model fallback='forbid'>Skylake-Client</model>
<vendor>Intel</vendor>
<feature policy='require' name='ss'/>
<feature policy='require' name='vmx'/>
<feature policy='require' name='hypervisor'/>
<feature policy='require' name='clflushopt'/>
<feature policy='require' name='umip'/>
<feature policy='require' name='arch-capabilities'/>
<feature policy='require' name='xsaves'/>
<feature policy='require' name='pdpe1gb'/>
<feature policy='require' name='invtsc'/>
<feature policy='disable' name='pclmuldq'/>
<feature policy='disable' name='lahf_lm'/>
</cpu>
The 'pclmuldq' and 'lahf_lm' should not be disabled in the baseline CPU
as they are supported by QEMU on this host.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Describe the encryption element in the TPM's domain XML.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Extend the Secret XML documentation with vtpm usage type.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Since swtpm does not support getting started without password
once it was created with encryption enabled, we don't allow
encryption to be removed. Similarly, we do not allow encryption
to be added once swtpm has run. We also prevent chaning the type
of the TPM backend since the encrypted state is still around and
the next time one was to switch back to the emulator backend
and forgot the encryption the TPM would not work.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This patch now passes the passphrase as a migration key to swtpm.
This now encrypts the state of the TPM while a VM is migrated between
hosts or when suspended into a file. Since the migration key secret
is the same as the state encryption secret, this now requires that
the migration destination host has the same secret value.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Allow vTPM state encryption when swtpm_setup and swtpm support
passing a passphrase using a file descriptor.
This patch enables the encryption of the vTPM state only. It does
not encrypt the state during migration, so the destination secret
does not need to have the same password at this point.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Add a test case to commandtest.c to test the transfer of data to a
process who received the read-end of pipes' file descriptors. Transfer
large (128 kb) byte streams.
Extend the commandhelper.c with support for --readfd <fd> command line
parameter and convert the data receive loop to use poll and receive data
on multiple file descriptors (up to 3) and read data into distinct buffers
that we grow while adding more (string) data.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Extend virCommandProcessIO to include the send buffers in the poll
loop.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Mark a virCommand's inpipe (write-end of pipe) as non-blocking so that it
will never block when we were to try to write too many bytes to it while
it doesn't have the capacity to hold them.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Convert the struct pollfd *fds to be allocated rather than residing
on the stack. This prepares it for the next patch where the size of
the array of fds becomes dynamic.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Implement virCommandSetSendBuffer() that allows the caller to pass a
file descriptor and buffer to virCommand. virCommand will write the
buffer into the file descriptor. That file descriptor could be the
write end of a pipe or one of the file descriptors of a socketpair.
The other file descriptor should be passed to the launched process to
read the data from.
Only implement the function to allocate memory for send buffers
and to free them later on.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Run 'swtpm socket --print-capabilities' and
'swtpm_setup --print-capabilities' to get the JSON object of the
features the programs are supporting and parse them into a bitmap.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Check whether previously found executables were updated and if
so look for them again. This helps to use updated features of
swtpm and its tools upon updating them.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Refactor virTPMEmulatorInit to use a loop with parameters. This allows
for easier extension later on.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Move qemuTPMEmulatorInit to virTPMEmulatorInit in virtpm.c and introduce
a few functions to query the executables needed for virCommands.
Add locking to protect the tool paths and return a copy of the tool paths
to callers wanting to access them so that we can run the initialization
function multiples time later on and detect when the executable gets updated.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The QEMU command line does not change when TPM state is encrypted
compared to when it is plain.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Add a test case for the TPM XML encryption parser and formatter.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Extend the TPM device XML parser and XML generator with emulator
state encryption support.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Add an already existing test case tpm-emulator-tpm2 to qemuxml2xmltest.c
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Add support for usage type vTPM to secret.
Extend the schema for the Secret to support the vTPM usage type
and add a test case for parsing the Secret with usage type vTPM.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
