Commit Graph

26234 Commits

Author SHA1 Message Date
Pavel Hrdina
c1024416f4 use g_ascii_islower instead of c_islower from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
a405834a5d use g_ascii_isdigit instead of c_isdigit frum gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
b10dd7d443 use g_ascii_isalpha instead of c_isalpha from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
c275ea93db use g_ascii_isalnum instead of c_isalnum from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:23 +01:00
Pavel Hrdina
d07cb0bcd3 virkeyfile: define IS_ASCII instead c_isascii from gnulib
GLib doesn't provide alternative to c_isascii and this is the only usage
of that macro so define a replacement ourselves.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:23 +01:00
Pavel Hrdina
243dbf5494 util: define IS_BLANK instead of using c_isblank from gnulib
The same way how we have IS_EOL in two files where we actually need it
defince IS_BLANK so we can drop usage of c_isblank.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:23 +01:00
Peter Krempa
1a2934d61c qemu: Add support for VIR_DOMAIN_CAPS_FEATURE_BACKUP
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
528191c754 conf: domaincaps: Add 'backup' feature flag
This flag will allow figuring out whether the hypervisor supports the
incremental backup and checkpoint features.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
9400302477 qemu: blockjob: Implement concluded blockjob handler for backup blockjobs
After the individual sub-blockjobs of a backup libvirt job finish we
must detect it and notify the parent job, so that it can be properly
terminated.

Since we update job information to determine success of a blockjob we
can directly report back also statistics of the blockjob.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
e0daa03a35 qemu: driver: Allow cancellation of the backup job
Use the helper which cancels all blockjobs to perform the backup job
cancellation in qemuDomainAbortJob.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
5ea6cec9ef qemu: backup: Implement stats gathering while the job is running
We can use the output of 'query-jobs' to figure out some useful
information about a backup job. That is progress in case of a push job
and scratch file use in case of a pull job.

Add a worker which will total up the data and call it from
qemuDomainGetJobStatsInternal.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
a1521f84a5 qemu: Implement backup job APIs and qemu handling
This allows to start and manage the backup job.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
e8ae2ddbb1 doc: Document quirk of getting block job info for a 'backup' blockjob
The stats reported for a blockjob which is member of a domain pull
backup refer to the utilization of the scratch file rather than the
progress of the backup as the progress of the backup depends on the
client. Note this quirk in the docs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
86d2866a90 conf: backup: Add fields for tracking stats of completed sub-jobs
We need a place to store stats of completed sub-jobs so that we can
later report accurate stats.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
2c59f0083e qemu: blockjob: Track internal data for 'backup' blockjob
A backup blockjob needs to be able to notify the parent backup job as
well as track all data to be able to clean up the bitmap and blockdev
used for the backup.

Add the data structure, job allocation function and status XML formatter
and parser.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
9ac4b7db8d qemu: domain: Track backup job data in the status XML
Store the data of a backup job along with the index counter for new
backup jobs in the status XML. Currently we will support only one
backup job and thus there's no necessity to add arrays of jobs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
e87120307a qemu: monitor: Add support for blockdev-backup via 'transaction'
Implement the transaction actions generator for blockdev-backup.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
08b810053a Add 'backup' block job type
A backup job may consist of many backup sub-blockjobs. Add the new
blockjob type and add all type converter strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
3656bb0a13 qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type
We will want to use the async job infrastructure along with all the APIs
and event for the backup job so add the backup job as a new async job
type.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
fed2cb487b qemu: Add infrastructure for statistics of a backup job
Introduce QEMU_DOMAIN_JOB_STATS_TYPE_BACKUP and the convertors and other
plumbing to be able to report statistics for the backup job.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Eric Blake
02f790ffbe backup: Parse and output backup XML
Accept XML describing a generic block job, and output it again as
needed. This may still need a few tweaks to match the documented XML
and RNG schema.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:56 +01:00
Eric Blake
42adc45937 backup: Implement backup APIs for remote driver
This one is fairly straightforward - the generator already does what
we need.

Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:56 +01:00
Eric Blake
74ca70507a backup: Introduce virDomainBackup APIs
Introduce a few new public APIs related to incremental backups.  This
builds on the previous notion of a checkpoint (without an existing
checkpoint, the new API is a full backup, differing from
virDomainBlockCopy in the point of time chosen and in operation on
multiple disks at once); and also allows creation of a new checkpoint
at the same time as starting the backup (after all, an incremental
backup is only useful if it covers the state since the previous
backup).

A backup job also affects filtering a listing of domains, as well as
adding event reporting for signaling when a push model backup
completes (where the hypervisor creates the backup); note that the
pull model does not have an event (starting the backup lets a third
party access the data, and only the third party knows when it is
finished).

The full list of new APIs:
        virDomainBackupBegin;
        virDomainBackupGetXMLDesc;

Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:56 +01:00
Peter Krempa
1e320c9d83 qemu: domain: Export qemuDomainGetImageIds
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:56 +01:00
Cole Robinson
97cafa610e qemu: domain: raise an error when no emulator is found
$ cat f | grep -e arch -e emulator
    <type arch='mipsel'>hvm</type>
$ sudo virsh define f
error: Failed to define domain from f
error: An error occurred, but the cause is unknown

After:
$ sudo virsh define f
error: Failed to define domain from f
error: unsupported configuration: No emulator found for arch 'mipsel'

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-12-09 14:40:43 -05:00
Daniel P. Berrangé
66d4fd7009 vz: drop virCapsPtr param from many API domain parse calls
This fixes commit bf9d812956

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 14:46:14 +00:00
Daniel P. Berrangé
ef7526cc19 vz: declare missing driver variable
This fixes commit 61bff77bf9

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 14:46:14 +00:00
Daniel P. Berrangé
92fdbaf97a vz: pass pointer to vz capabilities
This fixes commit 4a4132b462

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 14:46:13 +00:00
Daniel P. Berrangé
eb2952c0f5 vz: pass virDomainXMLOptionPtr to virDomainVideoDefNew
The virDomainVideoDefNew requires the xml options to be
provided since

  commit 3dbf3941ad
  Author: Marc-André Lureau <marcandre.lureau@redhat.com>
  Date:   Mon Sep 23 14:44:35 2019 +0400

    conf: add privateData to virDomainVideoDef

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 14:46:13 +00:00
Cole Robinson
4ce7e743df qemu: checkpoint: fix NULL dereference at create time
'cfg' is never initialized here, which causes a crash
later in qemuCheckpointCreateFinalize

Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-12-09 08:57:40 -05:00
Peter Krempa
6f6a1763a1 qemu: snapshot: Mark file becoming backingStore as read-only
For any backing file we set 'read-only' to true, but didn't do this when
modifying the recorded backing store when creating external snapshots.

This meant that qemu would attempt to open the backing-file read-write.
This would fail for example when selinux is used as qemu doesn't have
write permission for the backing file.

https://bugzilla.redhat.com/show_bug.cgi?id=1781079

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-09 14:07:24 +01:00
Peter Krempa
35614b6022 xen: Remove unused 'cfg'
Refactoring of the XML parser left few instances of the
libxlDriverConfig object unused. Remove them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2019-12-09 14:06:15 +01:00
Daniel P. Berrangé
3dd91af01f qemu: stop creating capabilities at driver startup
Now that nearly all internal APIs use the QEMU capabilities or other
QEMU driver data directly, there's no compelling benefit to create
virCapsPtr at driver startup.

Skipping this means we don't probe capabilities for all 30 system
emulator targets at startup, only those emulators which are referenced
by an XML doc. This massively improves libvirtd startup time when the
capabilities cache is not populated. It even improves startup time
when the cache is up to date, as we don't bother to load files from
the cache until we need them.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
7be5fe66cd qemu: directly create virResctrlInfo ignoring capabilities
We always refresh the capabilities object when using virResctrlInfo
during process startup. This is undesirable overhead, because we can
just directly create a virResctrlInfo instead.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
adf009b48f qemu: use host CPU object directly
Avoid grabbing the whole virCapsPtr object when we only need the
host CPU information.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
8506afff7b conf: make virCPUDef into a ref counted struct
Annoyingly there was no existing constructor, and identifying all the
places which do a VIR_ALLOC(cpu) is a bit error prone. Hopefully this
has found & converted them all.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
1a1d848694 qemu: use NUMA capabilities object directly
Avoid grabbing the whole virCapsPtr object when we only need the
NUMA information.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
6cc992bd1a conf: move NUMA capabilities into self contained object
The NUMA cells are stored directly in the virCapsHostPtr
struct. This moves them into their own struct allowing
them to be stored independantly of the rest of the host
capabilities. The change is used as an excuse to switch
the representation to use a GPtrArray too.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
bc1676fc2f qemu: drop virCapsPtr param & vars from many APIs
Now that the domain XML APIs don't use virCapsPtr we can stop passing it
around many QEMU driver methods.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
575d9d2504 conf: drop virCapsPtr param from snapshot & checkpoint APIs
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
1cd2b2d545 conf: drop virCapsPtr param from domain obj list APIs
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
78d8228eec conf: drop virCapsPtr param from APIs for saving domains
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
24d87d2e88 conf: drop virCapsPtr param from domain APIs for copying config
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
bf9d812956 conf: drop virCapsPtr param from domain parse APIs
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
b5f591cdb4 conf: drop virCapsPtr param from domain post parse & validate APIs
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
61bff77bf9 conf: drop virCapsPtr param from domain formatting APIs
This parameter is now unused and can be removed entirely.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
04c960dd64 conf: drop virCapsPtr param from domain validate callback
None of the impls of this callback require the virCapsPtr param.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
33b62676f8 conf: drop virCapsPtr param from device post parse callback
None of the impls of this callback require the virCapsPtr param.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
c919336288 conf: drop virCapsPtr param from post parse callback
No impl of this callback requires the virCapsPtr anymore.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
227a0503e2 conf: drop virCapsPtr param from post parse data alloc callback
The only user of this callback did not require the virCapsPtr parameter.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:27 +00:00
Daniel P. Berrangé
74fb858f7d conf: drop virCapsPtr param from basic post parse callback
The QEMU impl of the callback can directly use the QEMU capabilities
cache to resolve the emulator binary name, allowing virCapsPtr to be
dropped.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:17:19 +00:00
Daniel P. Berrangé
a7ef72a8fd conf: drop virCapsPtr param from assign addresses callback
The virCapsPtr param is not used by any of the virt drivers providing
this callback.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
638ffa2228 conf: pass default sec model in parser config
Instead of using the virCapsPtr to get the default security model,
pass this in via the parser config.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
99a949ffc4 conf: move seclabel validation into post-parse phase
Currently the disk and chardev seclabels are validated immediately at
the time their data is parsed. This forces the parser to fill in the
top level secmodel at time of parsing which is an undesirable thing.
This validation conceptually should be done in the post-parse phase
instead.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
a7b6e49d00 conf: pass netprefix in the domain parser config struct
Instead of using the virCapsPtr information, pass the driver specific
netprefix in the domain parser struct. This eliminates one more use of
virCapsPtr from the XML parsing/formatting code.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
4a4132b462 conf: don't use passed in caps in post parse method
To enable the virCapsPtr parameter to the post parse method to be
eliminated, the drivers must fetch the virCapsPtr from their own
driver via the opaque parameter, or use an alternative approach
to validate the parsed data.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
2578d74aee conf: move virt type / os type / arch validation to post-parse
The XML parser currently calls virCapabilitiesDomainDataLookup during
parsing to find the domain capabilities matching the triple

  (virt type, os type, arch)

This is, however, bogus with the QEMU driver as it assumes that there
is an emulator known to the default driver capabilities that matches
this triple. It is entirely possible for the driver to be parsing an
XML file with a custom emulator path specified pointing to a binary
that doesn't exist in the default driver capabilities.  This will,
for example be the case on a RHEL host which only installs the host
native emulator to /usr/bin. The user can have built a custom QEMU
for non-native arches into $HOME and wish to use that.

Aside from validation, this call is also used to fill in a machine type
for the guest if not otherwise specified. Again, this data may be
incorrect for the QEMU driver because it is not taking account of
the emulator binary that is referenced.

To start fixing this, move the validation to the post-parse callbacks
where more intelligent driver specific logic can be applied.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
6430c00552 conf: pass in default architecture via domain XML options
When parsing the guest XML we must fill in the default guest arch if it
is not already present because later parts of the parsing process need
this information.

If no arch is specified we lookup the first guest in the capabilities
data matching the os type and virt type. In most cases this will result
in picking the host architecture but there are some exceptions...

 - The test driver is hardcoded to always use i686 arch
 - The VMWare/ESX drivers will always place i686 guests ahead
   of x86_64 guests in capabilities, so effectively they always
   use i686
 - The QEMU driver can potentially return any arch at all
   depending on what combination of QEMU binaries are installed.

The domain XML hardware configurations are inherently architecture
specific in many places. As a result whomever/whatever created the
domain XML will have had a particular architecture in mind when
specifying the config. In pretty much any sensible case this arch
will have been the native host architecture. i686 on x86_64 is
the only sensible divergance because both these archs are
compatible from a domaain XML config POV.

IOW, although the QEMU driver can pick an almost arbitrary arch as its
default, in the real world no application or user is likely to be
relying on this default arch being anything other than native.

With all this in mind, it is reasonable to change the XML parser to
allow the default architecture to be passed via the domain XML options
struct. If no info is explicitly given then it is safe & sane to pick
the host native architecture as the default for the guest.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
92d412149c conf: sanitize virDomainObjFormat & virDomainDefFormat* APIs
Moving their instance parameter to be the first one, and give consistent
ordering of other parameters across all functions. Ensure that the xml
options are passed into both functions in prep for future work.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
908701c64a conf: sanitize virDomainSaveStatus & virDomainSaveConfig APIs
Our normal practice is for the object type to be the name prefix, and
the object instance be the first parameter passed in.

Rename these to virDomainObjSave and virDomainDefSave moving their
primary parameter to be the first one. Ensure that the xml options
are passed into both functions in prep for future work.

Finally enforce checking of the return type and mark all parameters
as non-NULL.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Daniel P. Berrangé
5cedfbd100 qemu: add machines & arch to fake test capabilities cache
Currently the virQEMUCapsPtr objects are just empty. Future patches are
going to expect them to contain real data. Start off by populating the
machine types and arch information.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:15 +00:00
Daniel P. Berrangé
bce3b0807e qemu: cache host arch separately from virCapsPtr
As part of a goal to eliminate the need to use virCapsPtr for anything
other than the virConnectGetCapabilies() API impl, cache the host arch
against the QEMU driver struct and use that field directly.

In the tests we move virArchFromHost() globally in testutils.c so that
every test runs with a fixed default architecture reported.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:15 +00:00
Jiri Denemark
c11706cc25 qemu: Implement VIR_MIGRATE_PARAM_TLS_DESTINATION
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2019-12-09 10:11:58 +01:00
Jiri Denemark
cc023b33bb qemu: Add support for setting string migration params
The functions for converting migration typed parameters to QEMU
migration parameters and back were only implemented for integer types.
This patch adds support for string parameters.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2019-12-09 10:11:58 +01:00
Peter Krempa
f5259ba402 qemu: migration: Properly setup mirror for blockdev configurations
With blockdev we need to refer to the nodename of the disk source image
as the source argument for the blockdev-mirror operation while still
keeping the old job name. With blockdev we must also persist the job in
qemu.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
3e35156bd1 qemu: migration: Mention disk target rather than the drive name in debug msg
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
69abc80f5e qemu: migration: Split out setup of the migration target
Separate out allocation of the virStorageSource corresponding to the
target NBD export of the migration.

As part of the splitout we allocate the export name explicitly as that
one must not change regardless whether blockdev is used or not to
provide compatibility.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
0d0b490a32 qemu: blockjob: Allow NULL 'mirror' for block copy jobs due to migration
The non-shared-storage migration tracks the storage source used
explicitly in the migration data so we must allow for processing of the
block job which has NULL mirror as the mirror will not be populated.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
e1910a1f3b qemu: migration: Simplify cleanup in qemuMigrationSrcNBDCopyCancelOne
Now that the cleanup section does not exist remove the label.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
37e801340a qemu: migration: Access job name from job struct
qemuMigrationSrcNBDCopyCancelOne uses the block job data structure but
generated it's own job name rather than taking it from the block job
data.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
5339f57a0b qemu: migration: Properly export backend for NBD storage migration
With -blockdev we must use the nodename as the export but we must keep
the name of the export as it was before to ensure compatiblity.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
83137f5d37 qemu: migration: Simplify handling of 'diskAlias' when adding NBD exports
Declare the variable inside the loop with automatic clearing.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
d179acf4ad qemu: driver: Use appropriate job name when setting blockjob speed
qemuDomainBlockJobSetSpeed was not converted to get the job name from
the block job data. This means that after enabling blockdev the API call
would fail as we wouldn't use the appropriate name.

https://bugzilla.redhat.com/show_bug.cgi?id=1780497

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:24 +01:00
Peter Krempa
e3faa84819 qemu: driver: Use qemuDomainDiskByName instead of virDomainDiskByName
Where appropriate replace the open coded call with the qemu wrapper
which already reports the error.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:23 +01:00
Peter Krempa
ca67d02fbe qemu: domain: Mention searched disk in error of qemuDomainDiskByName
Mention the argument used if the disk can't be located.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-09 07:11:23 +01:00
Jidong Xia
863c0d8e8d qemu: cold-unplug of sound
With this patch users can cold unplug some sound devices.
use "virsh detach-device vm sound.xml --config" command.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Jidong Xia <xiajidong@cmss.chinamobile.com>
2019-12-08 19:41:34 -05:00
Pavel Mores
7b2a6e8cf7 qemu: remove nested branching to enhance readability
This is a follow-up to patch series posted in

https://www.redhat.com/archives/libvir-list/2019-November/msg01180.html

It implements a suggestion made by Cole in

https://www.redhat.com/archives/libvir-list/2019-November/msg01207.html

and discussed in follow-up messages as there were no objections to the
change.

The aim is to make the code more readable by replacing nested branching
with a flat structure.

Signed-off-by: Pavel Mores <pmores@redhat.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-06 14:42:56 +01:00
Michal Privoznik
4c53267b70 qemu_monitor_text: Drop unused variable and avoid crash
In v5.8.0-rc1~122 we've removed the only use of @safename in
qemuMonitorTextLoadSnapshot(). What we are left with is an
declared but not initialized variable that is passed to
VIR_FREE().

Caught by libvirt-php test suite.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-06 10:29:46 +01:00
Michal Privoznik
516b867685 qemuProcessStop: Remove image metadata only when allowed
In v5.9.0-370-g8fa0374c5b I've tried to fix a bug by removing
some stale XATTRs in qemuProcessStop(). However, I forgot to
do nothing when the VIR_QEMU_PROCESS_STOP_NO_RELABEL flag was
specified.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2019-12-05 15:08:28 +01:00
Peter Krempa
6865dc295e qemu: Convert aborting of a domain job to a switch statement
Until now we only really aborted migration via qemuDomainAbortJob. This
will change with the upcoming addition of the backup job. Additionally
there were a bunch of if statements checking various aspects of the
current job.

To make it more obvious convert qemuDomainAbortJob to use a switch
statement and move the individual conditions to the appropriate job
type.

Every job type has now it's own case despite multiple job types just
plainly cancelling the job for clarity and future extension.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2019-12-05 13:53:15 +01:00
Peter Krempa
4ef3401eb8 qemu: driver: Split out cancellation of migration from qemuDomainAbortJob
Following patch will refactor qemuDomainAbortJob to use a per-job-type
switch where we will need to abort a migration job in various branches.

Save some code duplication by introducing a helper.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2019-12-05 13:53:15 +01:00
Daniel P. Berrangé
e0314b6855 src: rewrite remote protocol checker in Python
As part of a goal to eliminate Perl from libvirt build tools,
rewrite the pdwtags processing script in Python.

The original inline shell and perl code was completely
unintelligible. The new python code is a manual conversion
that attempts todo basically the same thing.

Tested-by: Cole Robinson <crobinso@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 13:44:15 +00:00
Han Han
0e37372291 conf: fix logic error for scsi units
Introduced in c8007fdc5d, it should use 'greater than max' instead of
'equal or greater than max' for the condition of checking invalid scsi
unit.

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-04 13:20:57 +01:00
Daniel P. Berrangé
1fbd80c42a tests: stop static linking to libvirt code in tests
If we static link to libvirt_util.la then we can't override functions in
this file by simply implementing them in the test code. Any tests should
dynamic link to the main libvirt.la and ensure symbols are exported.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 12:08:21 +00:00
Daniel P. Berrangé
d57d8d6e92 docs: move hyperv_wmi_generator.py to the scripts/ directory
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 11:42:28 +00:00
Daniel P. Berrangé
74f270095a docs: move esx_vi_generator.py to the scripts/ directory
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-04 11:42:16 +00:00
Peter Krempa
cd1e6fd478 qemu: process: Re-process qemu capability lockout in qemuProcessPrepareQEMUCaps
We clear some capabilities here so the lockouts need to be
re-evaluated.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
be1a201533 qemu: capabilities: Lock out incremental backup capability without blockdev
Blockdev is required to do incremental backups properly. Add a helper
function for locking out capabilities and export it to allow re-doing
the processing if a different code path modifies capabilities.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
a64265f440 qemu: caps: Don't check capability before clearing it
Checking whether a qemu capability set right before clearing it without
any other logic doesn't make sense.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
78c2a8b934 qemu: process: Move handling of qemu capability overrides
Do all post-processing of capabilities in qemuProcessPrepareQEMUCaps.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
97c9ece79b qemu: process: Move clearing of QEMU_CAPS_CHARDEV_FD_PASS to qemuProcessPrepareQEMUCaps
Move the post-processing of the QEMU_CAPS_CHARDEV_FD_PASS flag to the
new function.

The clearing of the capability is based on the presence of
VIR_QEMU_PROCESS_START_STANDALONE so we must also pass in the process
start flags.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
3a075524d9 qemu: process: Move clearing of the BLOCKDEV capability to qemuProcessPrepareQEMUCaps
Start aggregating all capability post-processing code in one place.

The comment was modified while moving it as it was mentioning floppies
which are no longer clearing the blockdev capability.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
dbbc9a3c40 qemu: Move and rename qemuDomainUpdateQEMUCaps
The function is now used only in qemu_process.c so move it there and
name it 'qemuProcessPrepareQEMUCaps' which is more appropriate to what
it's doing.

The reworded comment now mentions that it will also post-process the
caps for VM startup.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
530d7a73f4 qemu: process: Don't try to redetect missing qemuCaps on reconnect
The redetection was originally added in 43c01d3838 as a way to recover
from libvirtd upgrade from the time when we didn't persist the qemu
capabilities in the status XML. Also this the oldest supported qemu by
more than two years.

Even if somebody would have a running VM running at least qemu 1.5 with
such an old libvirt we certainly wouldn't do the right thing by
redetecting the capabilities and then trying to communicate with qemu.

For now it will be the best to just stop considering this scenario any
more and error out for such VM.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
2504dbeb5d qemu: process: Make it obvious that virDomainDefPostParse is called with NULL opaque
Commit c90fb5a828 added explicit use of the private copy of the qemu
capabilities to various places. The change to qemuProcessInit was bogus
though as at the point where we re-initiate the post parse callbacks
priv->qemuCaps is still NULL as we clear it after shutdown of the VM and
don't initiate it until a later point.

Using the value from priv->qemuCaps might mislead readers of the code
into thinking that something useful is being passed at that point so go
with an explicit NULL instead.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
ab163144eb qemu: Always reset @info in qemuDomainGetJobInfo
qemuDomainGetJobInfo didn't always reset the return data in @info.
Thankfully this wouldn't be a problem as the RPC layer does it but we
should do it anyways.

Since we reset the struct we don't have to set the type to
VIR_DOMAIN_JOB_NONE as the value is 0.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:54 +01:00
Peter Krempa
2dae916de4 qemu: Implement VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Peter Krempa
d4a4933a0d api: Allow keeping completed domain job stats when reading them
virDomainGetJobStats destroys the completed statistics on the first
read. Give the user possibility to keep them around if they wish so.

Add a flag VIR_DOMAIN_JOB_STATS_KEEP_COMPLETED which will read the stats
without destroying them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-03 15:26:53 +01:00
Daniel P. Berrangé
8aaed287f5 qemu: make 'xz' image compression viable by using -3
For managed save we can choose between various compression
methods. I randomly tested the 'xz' program on a 8 GB guest
and was surprised to have to wait > 50 minutes for it to
finish compressing, with 'xz' burning 100% cpu for the
entire time. Despite the impressive compression, this is
completely useless in the real world as it is far too long
to wait to save the VM.

The 'xz' binary defaults to '-6' optimization level which
aims for high compression, with moderate memory usage,
at the expense of speed.

This change switches it to use the '-3' optimization level
which is documented as being the one that optimizes speed
at expense of compression. Even with this, it will still
outperform all the other options in terms of compression
level. It is a little less than x4 faster than '-6' which
means it starts to be a viable choice to use 'xz' for
people who really want best compression.

The test results on a 1 GB, fairly freshly booted VM are
as follows

  format | save  | restore  size
  =======+=======+=============
  raw    |   05s |    1s  | 428 MB
  lzop   |   05s |    3s  | 160 MB
  gzip   |   29s |    5s  | 118 MB
  bz2    |   54s |   22s  | 114 MB
  xz     | 4m37s |   13s  |  86 MB
  xz -3  | 1m20s |   12s  |  95 MB

Based on this we can say

 * For moderate compression with no noticable loss in speed

       => use lzop

 * For high compression with moderate loss in speed

       => use gzip

 * For best compression with significant loss in speed

       => use xz

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-03 13:43:45 +00:00
Erik Skultety
d7efba9711 storage: backend_disk: Implement the deletePool function
This is a very simple and straightforward implementation of the opposite
what buildPool does for the disk backend.
The background for this change comes from an existing test case in TCK
which does use the delete method for a pool of type disk, but it
truly could not have ever worked since the implementation simply
wasn't there for the pool of type disk.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-03 09:31:10 +01:00