readline-devel is an optional build dependency; when it is not
present, the output of 'virsh <<EOF ... EOF' is different in that the
input provided by the user is not echoed, and prompts become
interleaved on the same line as actual output, which in turn causes
the sed doing prompt filtering to mess up:
| ./virsh-snapshot
| --- exp 2019-07-31 18:42:31.107399428 -0300
| +++ out.cooked 2019-07-31 18:42:31.108399437 -0300
| @@ -1,8 +1,3 @@
| -
| -
| -Domain snapshot s3 created from 's3.xml'
| -Domain snapshot s2 created from 's2.xml'
| -Name: s2
| Domain: test
| Current: yes
| State: running
Maybe we should fix virsh in interactive mode to echo regardless of
whether readline-devel was used, but the quicker fix is to make the
test use 'virsh "..."' rather than reading its input from stdin.
Reported-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Checkpoints are definitely a news-worthy addition, even if the
virDomainBackup API is not going to make it until a later release.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
I messed up formatting during conflict resolution across rebasing
while preparing my checkpoint patches :)
Signed-off-by: Eric Blake <eblake@redhat.com>
hv-spinlocks is not a CPUID feature and should not be checked as such.
While starting a domain with hv-spinlocks enabled, we would report a
warning about unsupported hyperv spinlocks feature even though it was
set properly.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Introduced by: commit e9528f41c6
'msrs' is a feature unrelated to Hyper-V Enlightenments, the commit message
which added it and the test have it right:
<features>
...
<msrs unknown='ignore'>
...
</features>
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
SynIC stands for 'Synthetic Interrupt Controller', it is not a NIC. Fix the
spelling in accordance with Hypervisor Top Level Functional Specification.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
The example XML we have contains all other Hyper-V Enlightenments but
'stimer' is missing.
Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
CI flagged a failing mingw build, due to:
In file included from ../../src/conf/checkpoint_conf.c:24:
../gnulib/lib/configmake.h:8:17: error: expected identifier or '(' before string constant
8 | #define DATADIR "/usr/i686-w64-mingw32/sys-root/mingw/share"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As previously learned in commits bd205a90 and 976abdf6, gnulib's
configmake.h header does #define DATADIR "string...", while mingw's
<winsock2.h> expects to declare a type named DATADIR. As long as the
mingw system header is included first before configmake.h, the two
uses do not conflict, but until gnulib is patched to make configmake.h
automatically work around the issue, our immediate fix is the
workaround of rearranging our include order to insure no conflict.
Copy the paradigm used in domain_conf.c of using <unistd.h> to trigger
the indirect inclusion of <winsock2.h> on mingw.
Fixes: 1a4df34a
Signed-off-by: Eric Blake <eblake@redhat.com>
Turning a NULL URI instead the empty string is very misleading when
reading the debug logs as the distinction between the two is
functionally important.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The recently added test27 spawns commandhelper. This is fine,
except, one of the things that commandhelper does is it records
arguments it was spawn with into commandhelper.log. Other test
cases then use checkoutput() to compare the arguments against the
expected ones and also unlink() the log file. However, test27()
is not doing that and thus it leaves the file behind. This
breaks distcheck.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
These test cases will start failing once the test driver provides
implementation for the virDomainGetCPUStats API.
Signed-off-by: Ilias Stamatis <stamatis.iliass@gmail.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The returned array of qemuMonitorJobInfo structs must be freed.
164 (16 direct, 148 indirect) bytes in 1 blocks are definitely lost in loss record 64 of 84
at 0x4A3568B: realloc (vg_replace_malloc.c:826)
by 0x4D888BD: virReallocN (viralloc.c:244)
by 0x4D889B3: virExpandN (viralloc.c:293)
by 0x4D88C87: virInsertElementsN (viralloc.c:435)
by 0x214004: qemuMonitorJSONGetJobInfo (qemu_monitor_json.c:9185)
by 0x148B3F: testQueryJobs (qemumonitorjsontest.c:2979)
by 0x14C192: virTestRun (testutils.c:174)
by 0x14BF36: mymain (qemumonitorjsontest.c:3286)
by 0x14E256: virTestMain (testutils.c:1096)
by 0x14BFD9: main (qemumonitorjsontest.c:3298)
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
ACKed-by: Peter Krempa <pkrempa@redhat.com>
Originally the names of the KVM CPU features were only used internally
for looking up their CPUID bits. So we used "__kvm_" prefix for them to
make sure the names do not collide with normal CPU features stored in
our CPU map.
But with QEMU 4.1 we check which features were enabled or disabled by a
freshly started QEMU process using their names rather than their CPUID
bits (mostly because of MSR features). Thus we need to change our made
up internal names into the actual names used by QEMU.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Tested-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Most of the internally defined KVM CPUID features are not actually used
by libvirt. The QEMU driver may enable or disable them on the command
line, but we don't check for the associated CPU properties or CPUID
bits. They would be useless with QEMU 4.1 anyway since their names were
only remotely similar to the actual feature names.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Tested-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
All the features are hyperv features even though they are provided by
KVM with QEMU. The "KVM" part in the macro names does not make a lot of
sense.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Tested-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Starting with QEMU 4.1, we're using the canonical feature names on the
command line and avoid aliases to prepare for possible deprecation of
all aliases in QEMU. But we do so only for features from our CPU map,
hyperv features defined in the code were unchanged and this patch fixes
it. Some features use "hv-" prefix unconditionally because they were
introduced recently enough to always support spelling with a dash.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Tested-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Originally the names of the hyperv CPU features were only used
internally for looking up their CPUID bits. So we used "__kvm_hv_"
prefix for them to make sure the names do not collide with normal CPU
features stored in our CPU map.
But with QEMU 4.1 we check which features were enabled or disabled by a
freshly started QEMU process using their names rather than their CPUID
bits (mostly because of MSR features). Thus we need to change our made
up internal names into the actual names used by QEMU. Most of the names
are only used with QEMU 4.1 and newer and the reset was introduced with
QEMU recently enough to already support spelling with "-". Thus we don't
need to define them as "hv_*" with a translation to "hv-*" for new QEMU.
Without this patch libvirt would mistakenly report all hyperv features
as unavailable and refuse to start any domain using them with QEMU 4.1.
Reported-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Tested-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Earlier patches mentioned that the initial implementation will prevent
snapshots and checkpoints from being used on the same domain at once.
However, the actual restriction is done in this separate patch to make
it easier to lift that restriction via a revert, when we are finally
ready to tackle that integration in the future.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Time to actually issue the QMP transactions that create and delete
persistent checkpoints, resolving TODOs intentionally left earlier in
the series. For create, we only need one transaction: inside, we
visit all disks affected by the checkpoint, and create a new enabled
bitmap, as well as disabling the bitmap of the first ancestor
checkpoint (if any) that also had a bitmap. For deletion, we need
multiple QMP calls: for each disk, if there is an ancestor checkpoint
with a bitmap, then the bitmap must be merged (including activating
the ancestor bitmap if the leaf node changes), all before deleting the
bitmap from the checkpoint being removed.
Signed-off-by: Eric Blake <eblake@redhat.com>
Qemu bitmap operations require knowing the node name associated with
the format layer (the qcow2 file); as upcoming patches will be
grabbing that information frequently, make a helper function to access
it.
Another potential benefit of this function is that we have a single
place where we could insert a QMP node-name scraping call if we don't
currently know the node name, when -blockdev is not supported;
however, the goal is that we hopefully don't ever have to do that
because we instead scrape node names only at the point where they
change.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
A lot of this work heavily copies from the existing snapshot APIs.
What's more, this patch is (intentionally) very similar to the
checkpoint code just added in the test driver, to the point that qemu
checkpoints are not fully usable in this patch, but it at least
bisects and builds cleanly. The separation between patches is done
because the grunt work of saving and restoring XML and tracking
relations between checkpoints is common to the test driver, while the
later patch adding integration with QMP is specific to qemu.
Also note that the interlocking to prevent checkpoints and snapshots
from existing at the same time will be a separate patch, to make it
easier to revert that restriction when we finally round out the design
for supporting interaction between the two concepts.
Signed-off-by: Eric Blake <eblake@redhat.com>
This is similar to the existing directory for snapshots; the domain
will save one xml file per checkpoint, for reloading on the next
libvirtd restart. Fortunately, since checkpoints mandate RNG
validation, we are assured that the checkpoint name will be usable as
a file name (no abuse of '../escape' as a checkpoint name, for
example).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Introduce the handler for finalizing a block commit and active bloc
commit job which will allow to use it with blockdev.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Introduce the handler for finalizing a block pull job which will allow
to use it with blockdev.
This patch also contains some additional machinery which is required to
store all the relevant job data in the status XML which will also be
reused with other block job types.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
In case of an incoming migration we do not need to run swtpm_setup
with all the parameters but only want to get the benefit of it
creating a TPM state file for us that we can then label with an
SELinux label. The actual state will be overwritten by the in-
coming state. So we have to pass an indicator for incomingMigration
all the way to the command line parameter generation for swtpm_setup.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Similar to virsh-snapshots. Provides decent coverage of the checkpoint
API, the test driver implementation, and the virsh access to the API.
A later patch will worry about testing that snapshots and checkpoints
are mutually exclusive (in part so it is easier to revert that when we
finally implement the interaction and lift that restriction).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
A lot of this work heavily copies from the existing snapshot APIs.
The test driver doesn't really have to do anything more than just
expose an interface into libvirt metadata, making it possible to test
saving and restoring XML, and tracking relations between multiple
checkpoints.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Introduce a bunch of new virsh commands for managing checkpoints in
isolation. More commands are needed for performing incremental
backups, but these commands were easy to implement by modeling heavily
after virsh-snapshot.c. There is no need for checkpoint-revert or
checkpoint-current since those snapshot APIs have no checkpoint
counterpart. Similarly, it is not necessary to change which
checkpoint is current when redefining from XML, since until we
integrate checkpoints with snapshots, there is only a linear chain
(and you can deduce the current checkpoint by instead using
'checkpoint-list --leaves'). Other aspects of checkpoint-list are
also a bit simpler than the snapshot counterpart, in part because we
don't have to cater to back-compat to older API.
Upcoming patches will test these interfaces once the test driver
supports checkpoints.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The remote code generator had to be taught about the new
virDomainCheckpointPtr type, at which point the remote driver code for
checkpoints can be generated.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Creating a checkpoint does not modify guest-visible state,
but does modify host resources. Rather than reuse existing
domain:write, domain:block_write, or domain:snapshot access
controls, it seems better to introduce a new access control
specific to tasks related to checkpoints and incremental
backups of guest disk state.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Wire up the use of a checkpoint list into each domain, similar to the
existing snapshot list. This includes adding a function for checking
that a redefine operation fits in with the existing list, as well as
various filtering capabilities over the list contents.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Create a new file for managing a list of checkpoint objects, borrowing
heavily from existing virDomainSnapshotObjList paradigms.
Note that while snapshots definitely have a use case for multiple
children to a single parent (create a base snapshot, create a child
snapshot, revert to the base, then create another child snapshot),
it's harder to predict how checkpoints will play out with reverting to
prior points in time. Thus, in initial use, given a list of
checkpoints, you never have more than one child, and we can treat the
most-recent leaf node as the parent of the next node creation, without
having to expose a notion of a current node in XML or public API.
However, as the snapshot machinery is already generic, it is easier to
reuse the generic machinery that tracks relations between domain
moments than it is to open-code a new list-management scheme just for
checkpoints (hence, we still have internal functions related to a
current checkpoint, even though that has no observable effect
externally, as well as the addition of a function to easily find the
lone leaf in the list to use as the current checkpoint).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Add a new file checkpoint_conf.c that performs the translation to and
from new XML describing a checkpoint. The code shares a common base
class with snapshots, since a checkpoint similarly represents the
domain state at a moment in time. Add some basic testing of round trip
XML handling through the new code.
Of note - this code intentionally differs from snapshots in that XML
schema validation is unconditional, rather than based on a public API
flag. We have many existing interfaces that still need to add a flag
for opt-in schema validation, but those interfaces have existing
clients that may not have been producing strictly-compliant XML, or we
may still uncover bugs where our RNG grammar is inconsistent with our
code (where omitting the opt-in flag allows existing apps to keep
working while waiting for an RNG patch). But since checkpoints are
brand-new, it's easier to ensure the code matches the schema by always
using the schema. If needed, a later patch could extend the API and
add a flag to turn on to request schema validation, rather than having
it forced (possibly just the validation of the <domain> sub-element
during REDEFINE) - but if a user encounters XML that looks like it
should be good but fails to validate with our RNG schema, they would
either have to upgrade to a new libvirt that adds the new flag, or
upgrade to a new libvirt that fixes the RNG schema, which implies
adding such a flag won't help much.
Also, the redefine flag requires the <domain> sub-element to be
present, rather than catering to historical back-compat to older
versions.
Signed-off-by: Eric Blake <eblake@redhat.com>
Now that various new API have been added or are coming soon, it is
worth a landing page that gives an overview of capturing various
pieces of guest state, and which APIs are best suited to which tasks.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Introduce a bunch of new public APIs related to backup checkpoints.
Checkpoints are modeled heavily after virDomainSnapshotPtr (both
represent a point in time of the guest), although a snapshot exists
with the intent of rolling back to that state, while a checkpoint
exists to make it possible to create an incremental backup at a later
time. We may have a future hypervisor that can completely manage
checkpoints without libvirt metadata, but the first two planned
hypervisors (qemu and test) both always use libvirt for tracking
metadata relations between checkpoints, so for now, I've deferred
the counterpart of virDomainSnapshotHasMetadata for a separate
API addition at a later date if there is ever a need for it.
Note that until we allow snapshots and checkpoints to exist
simultaneously on the same domain (although the actual prevention of
this will be in a separate patch for the sake of an easier revert down
the road), that it is not possible to branch out to create more than
one checkpoint child to a given parent, although it may become
possible later when we revert to a snapshot that coincides with a
checkpoint. This also means that for now, the decision of which
checkpoint becomes the parent of a newly created one is the only
checkpoint with no child (so while there are APIs for dealing with a
current snapshot, we do not need those for checkpoints). We may end
up exposing a notion of a current checkpoint later, but it's easier to
add stuff when proven needed than to blindly support it now and wish
we hadn't exposed it.
The following map shows the API relations to snapshots, with new APIs
on the right:
Operate on a domain object to create/redefine a child:
virDomainSnapshotCreateXML virDomainCheckpointCreateXML
Operate on a child object for lifetime management:
virDomainSnapshotDelete virDomainCheckpointDelete
virDomainSnapshotFree virDomainCheckpointFree
virDomainSnapshotRef virDomainCheckpointRef
Operate on a child object to learn more about it:
virDomainSnapshotGetXMLDesc virDomainCheckpointGetXMLDesc
virDomainSnapshotGetConnect virDomainCheckpointGetConnect
virDomainSnapshotGetDomain virDomainCheckpointGetDomain
virDomainSnapshotGetName virDomainCheckpiontGetName
virDomainSnapshotGetParent virDomainCheckpiontGetParent
virDomainSnapshotHasMetadata (deferred for later)
virDomainSnapshotIsCurrent (no counterpart, see note above)
Operate on a domain object to list all children:
virDomainSnapshotNum (no counterparts, these are the old
virDomainSnapshotListNames racy interfaces)
virDomainSnapshotListAllSnapshots virDomainListAllCheckpoints
Operate on a child object to list descendents:
virDomainSnapshotNumChildren (no counterparts, these are the old
virDomainSnapshotListChildrenNames racy interfaces)
virDomainSnapshotListAllChildren virDomainCheckpointListAllChildren
Operate on a domain to locate a particular child:
virDomainSnapshotLookupByName virDomainCheckpointLookupByName
virDomainSnapshotCurrent (no counterpart, see note above)
virDomainHasCurrentSnapshot (no counterpart, old racy interface)
Operate on a snapshot to roll back to earlier state:
virDomainSnapshotRevert (no counterpart, instead checkpoints
are used in incremental backups via
XML to virDomainBackupBegin)
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Prepare for new checkpoint APIs by describing the XML that will
represent a checkpoint. The checkpoint XML is modeled heavily after
virDomainSnapshotPtr. See the docs for more details.
Add testsuite coverage for some minimal uses of the XML (bare minimum,
the sample from html, and a full dumpxml, and some counter-examples
that should fail schema validation). Although use of the REDEFINE flag
will require the <domain> subelement to be present, it is easier for
most of the tests to provide counterpart output produced with the
NO_DOMAIN flag (particularly since synthesizing a valid <domain>
during testing is not trivial).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
If we are using -blockdev, then node names are always available
(because we set them). But when not using it, we have to scrape node
names from QMP, and want to do so as infrequently as possible. We
were scraping node names after reconnecting a new libvirtd to an
existing guest (see qemuProcessReconnect), and after any block job
that may have changed the set of node names we care about (legacy
block jobs), but forgot to scrape the names when first starting a
guest. Do so now in order to allow the checkpoint code to always have
access to a node name without having to repeat a node name scrape
itself.
Future patches may need to clean up qemuDomainSetBlockThreshold (if
node names are always available, then it doesn't need to repeat a
scrape) and/or hotplug and media changes (if the addition of new nodes
can result in a null node name, then scraping at that point in time
would be appropriate). But for now, this patch addresses only the
most common instance of a missing node name.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Since we are checking the 2nd parameter in the function for NULL,
we need to remove ATTRIBUTE_NONNULL(2) from the prototype.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Message-Id: <20190726205633.2041912-5-stefanb@linux.vnet.ibm.com>
Remove the ATTRIBUTE_NONNULL(1) from virCommandSetSendBuffer()
prototype since we are checking for '!cmd' and move the initialization
if 'i' after the test for '!cmd'.
Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Message-Id: <20190726205633.2041912-4-stefanb@linux.vnet.ibm.com>
