26463 Commits

Author SHA1 Message Date
Ján Tomko
610963cd04 util: remove VIR_STRDUP and VIR_STRNDUP
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-12 14:24:35 +01:00
Ján Tomko
93de3025b4 Remove the rest of VIR_STRNDUP
Replace all the uses passing a single parameter as the length.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-12 14:24:35 +01:00
Ján Tomko
b315a2873b Remove all the uses that use subtraction in their length argument
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-12 14:24:35 +01:00
Ján Tomko
7318bff551 Remove VIR_STRNDUP usage with checked pointers
Remove the usage where sanity of the length argument is verified
by other conditions not matching the previous patches.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-12 14:24:34 +01:00
Ján Tomko
99114a6259 Remove VIR_STRDUP usage that sneaked in in the meantime
My hesitation to remove VIR_STRDUP without VIR_STRNDUP resulted
in these being able to sneak in.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-12 14:24:34 +01:00
Daniel P. Berrangé
766c8ae963 Revert "qemu: directly create virResctrlInfo ignoring capabilities"
This reverts commit 7be5fe66cd024b9ffba7c52cdbf5effedeac2c0d.

This commit broke resctrl, because it missed the fact that the
virResctrlInfoGetCache() has side-effects causing it to actually
change the virResctrlInfo parameter, not merely get data from
it.

This code will need some refactoring before we can try separating
it from virCapabilities again.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-12 11:16:44 +00:00
Pavel Mores
d75f865fb9 qemu: fix concurrency crash bug in snapshot revert
This commit aims to fix

https://bugzilla.redhat.com/show_bug.cgi?id=1610207

The cause was apparently incorrect handling of jobs in snapshot
revert code which allowed a thread executing snapshot delete to
begin job while snapshot revert was still running on another
thread. The snapshot delete thread then waited on a condition
variable in qemuMonitorSend() while the revert thread finished,
changing (and effectively corrupting) the qemuMonitor structure
under the delete thread which led to its crash.

The incorrect handling of jobs in revert code was due to the fact
that although qemuDomainRevertToSnapshot() correctly begins a job
at the start, the job was implicitly ended when qemuProcessStop()
was called because the job lives in the QEMU driver's private
data (qemuDomainObjPrivate) that was purged during
qemuProcessStop().

This fix prevents qemuProcessStop() from clearing jobs as the
idea of qemuProcessStop() clearing jobs seems wrong in the first
place. It was (inadvertently) introduced in commit
888aa4b6b9db65e3db273341e79846, which is effectively reverted by
the second hunk of this commit. To preserve the desired effects
of the faulty commit, the first hunk is included as suggested by
Michal.

Signed-off-by: Pavel Mores <pmores@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-12 11:26:19 +01:00
Daniel P. Berrangé
1902356231 qemu: keep capabilities when running QEMU as root
When QEMU uid/gid is set to non-root this is pointless as if we just
used a regular setuid/setgid call, the process will have all its
capabilities cleared anyway by the kernel.

When QEMU uid/gid is set to root, this is almost (always?) never
what people actually want. People make QEMU run as root in order
to access some privileged resource that libvirt doesn't support
yet and this often requires capabilities. As a result they have
to go find the qemu.conf param to turn this off. This is not
viable for libguestfs - they want to control everything via the
XML security label to request running as root regardless of the
qemu.conf settings for user/group.

Clearing capabilities was implemented originally because there
was a proposal in Fedora to change permissions such that root,
with no capabilities would not be able to compromise the system.
ie a locked down root account. This never went anywhere though,
and as a result clearing capabilities when running as root does
not really get us any security benefit AFAICT. The root user
can easily do something like create a cronjob, which will then
faithfully be run with full capabilities, trivially bypassing
the restriction we place.

IOW, our clearing of capabilities is both useless from a security
POV, and breaks valid use cases when people need to run as root.

This removes the clear_emulator_capabilities configuration
option from qemu.conf, and always runs QEMU with capabilities
when root.  The behaviour when non-root is unchanged.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-11 16:01:20 +00:00
Daniel P. Berrangé
e8dfddfa21 docs: remove build recipes related to pod2man usage
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-11 14:28:41 +00:00
Daniel P. Berrangé
dc0d07278c docs: convert virkeycode*/virkeyname* man pages from pod to rst
The keycodemap tool is told to generate docs in rst format now
instead of pod.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-11 14:28:41 +00:00
Daniel P. Berrangé
f09c4d7899 docs: convert virtlogd man page from pod to rst
This was a semi-automated conversion. First it was run through pod2rst,
and then it was manually editted to use a rst structure that matches
expectations of rst2man.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-11 14:28:41 +00:00
Daniel P. Berrangé
a318683c5b docs: convert virtlockd man page from pod to rst
This was a semi-automated conversion. First it was run through pod2rst,
and then it was manually editted to use a rst structure that matches
expectations of rst2man.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-11 14:28:41 +00:00
Daniel P. Berrangé
b8aa1846a0 docs: convert libvirtd man page from pod to rst
This was a semi-automated conversion. First it was run through pod2rst,
and then it was manually editted to use a rst structure that matches
expectations of rst2man.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-11 14:28:41 +00:00
Daniel P. Berrangé
2979b88609 src: update keycodemapdb submodule
Pull in changes which support use of RST for docs output format
instead of POD.

The generator tool has changed its command line arg handling
so all args must be after the command name. The docs title and
subtitle must be specified separately too.

Reviewed-by: Cole Robinson <crobinso@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-11 14:28:40 +00:00
Pavel Mores
7d484ede20 qemu: block: enable the snapshot image deletion feature
With all plumbing in place, we can now enable the new functionality.

Signed-off-by: Pavel Mores <pmores@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2019-12-11 08:40:19 +01:00
Pavel Mores
73532dadd2 qemu: block: store the delete flag in libvirtd's status XML
Since blockcommit is asynchronous, libvirtd can be restarted while the
operation runs.  To ensure the information necessary to finish up the job
is not lost, serialisation to and deserialisation from the status XML is
added.

To unittest this, the new element was only added to the active commit test,
the non-active commit test doesn't have the new element so as to test its
absence.

Signed-off-by: Pavel Mores <pmores@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2019-12-11 08:40:18 +01:00
Pavel Mores
9e5c98e84f qemu: block: use the delete flag to delete snapshot images if requested
When blockcommit finishes successfully, one of the
qemuBlockJobProcessEventCompletedCommit() and
qemuBlockJobProcessEventCompletedActiveCommit() event handlers is called.
This is where the delete flag (stored in qemuBlockJobCommitData since the
previous commit) can actually be used to delete the committed snapshot
images if requested.

We use virFileRemove() instead of a simple unlink() to cover the case where
the image to be removed is on an NFS volume.

Signed-off-by: Pavel Mores <pmores@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2019-12-11 08:40:18 +01:00
Pavel Mores
cb03fd9340 qemu: block: propagate the delete flag to where it can actually be used
Propagate the delete flag from qemuDomainBlockCommit() (which was just
ignoring it until now) to qemuBlockJobDiskNewCommit() where it can be
stored in the qemuBlockJobCommitData structure which holds information
necessary to finish the job asynchronously.

In the actual qemuBlockJobDiskNewCommit() in this commit, we temporarily
pass a literal 'false' to preserve the current behaviour until the whole
implementation of the feature is in place.

Signed-off-by: Pavel Mores <pmores@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2019-12-11 08:40:18 +01:00
Cole Robinson
b9a055a409 security: apparmor: Label externalDataStore
Teach virt-aa-helper how to label a qcow2 data_file, tracked internally
as externalDataStore. It should be treated the same as its sibling
disk image

Reviewed-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 10:14:58 -05:00
Cole Robinson
22e7997d0c lxc: Refresh capabilities if they have never been initalized
Adjust virLXCDriverGetCapabilities to fill in driver->caps if it is
empty, regardless of the passed 'refresh' value. This matches the
pattern used in virQEMUDriverGetCapabilities

This fixes LXC XML startup parsing for me

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 10:12:48 -05:00
Pavel Hrdina
cd9492a98a virkeyfile: fix compilation error with clang
Clang complains about condition being always true:

src/util/virkeyfile.c:113:23: error: result of comparison of constant 128 with expression of type 'const char' is always true [-Werror,-Wtautological-constant-out-of-range-compare]
    while (!IS_EOF && IS_ASCII(CUR) && CUR != ']')
                      ^~~~~~~~~~~~~
src/util/virkeyfile.c:80:26: note: expanded from macro 'IS_ASCII'
                     ~~~ ^ ~~~

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 16:03:39 +01:00
Cole Robinson
24cbacfe7a Fix src/ pkg-config API XML paths
The .pc files in src/ are intended for use with the ./run script,
to ease building bindings against an uninstalled libvirt build.
The pointer to the API XML files is incorrect though, it needs to
point into the build tree.

This fixes use of the run script for building libvirt-python, ex:

  /path/to/libvirt.git/run ./setup.py build

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 09:47:26 -05:00
Pavel Hrdina
c061fc2603 use g_ascii_strncasecmp instead of c_strncasecmp from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 14:08:51 +01:00
Pavel Hrdina
c89e792d17 use g_ascii_strcasecmp instead of c_strcasecmp from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 14:08:46 +01:00
Pavel Hrdina
5d98c513c5 use g_ascii_toupper instead of c_toupper from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
cc0e7541ad use g_ascii_tolower instead of c_tolower from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
51c2bc4ba9 use g_ascii_isxdigit instead of c_isxdigit from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
caab1fbd67 use g_ascii_isspace instead of c_isspace from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
04d267d347 use g_ascii_isprint instead of c_isprint from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
c1024416f4 use g_ascii_islower instead of c_islower from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
a405834a5d use g_ascii_isdigit instead of c_isdigit frum gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
b10dd7d443 use g_ascii_isalpha instead of c_isalpha from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:24 +01:00
Pavel Hrdina
c275ea93db use g_ascii_isalnum instead of c_isalnum from gnulib
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:23 +01:00
Pavel Hrdina
d07cb0bcd3 virkeyfile: define IS_ASCII instead c_isascii from gnulib
GLib doesn't provide alternative to c_isascii and this is the only usage
of that macro so define a replacement ourselves.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:23 +01:00
Pavel Hrdina
243dbf5494 util: define IS_BLANK instead of using c_isblank from gnulib
The same way how we have IS_EOL in two files where we actually need it
defince IS_BLANK so we can drop usage of c_isblank.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-10 13:49:23 +01:00
Peter Krempa
1a2934d61c qemu: Add support for VIR_DOMAIN_CAPS_FEATURE_BACKUP
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
528191c754 conf: domaincaps: Add 'backup' feature flag
This flag will allow figuring out whether the hypervisor supports the
incremental backup and checkpoint features.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
9400302477 qemu: blockjob: Implement concluded blockjob handler for backup blockjobs
After the individual sub-blockjobs of a backup libvirt job finish we
must detect it and notify the parent job, so that it can be properly
terminated.

Since we update job information to determine success of a blockjob we
can directly report back also statistics of the blockjob.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
e0daa03a35 qemu: driver: Allow cancellation of the backup job
Use the helper which cancels all blockjobs to perform the backup job
cancellation in qemuDomainAbortJob.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
5ea6cec9ef qemu: backup: Implement stats gathering while the job is running
We can use the output of 'query-jobs' to figure out some useful
information about a backup job. That is progress in case of a push job
and scratch file use in case of a pull job.

Add a worker which will total up the data and call it from
qemuDomainGetJobStatsInternal.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
a1521f84a5 qemu: Implement backup job APIs and qemu handling
This allows to start and manage the backup job.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
e8ae2ddbb1 doc: Document quirk of getting block job info for a 'backup' blockjob
The stats reported for a blockjob which is member of a domain pull
backup refer to the utilization of the scratch file rather than the
progress of the backup as the progress of the backup depends on the
client. Note this quirk in the docs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
86d2866a90 conf: backup: Add fields for tracking stats of completed sub-jobs
We need a place to store stats of completed sub-jobs so that we can
later report accurate stats.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:58 +01:00
Peter Krempa
2c59f0083e qemu: blockjob: Track internal data for 'backup' blockjob
A backup blockjob needs to be able to notify the parent backup job as
well as track all data to be able to clean up the bitmap and blockdev
used for the backup.

Add the data structure, job allocation function and status XML formatter
and parser.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
9ac4b7db8d qemu: domain: Track backup job data in the status XML
Store the data of a backup job along with the index counter for new
backup jobs in the status XML. Currently we will support only one
backup job and thus there's no necessity to add arrays of jobs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
e87120307a qemu: monitor: Add support for blockdev-backup via 'transaction'
Implement the transaction actions generator for blockdev-backup.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
08b810053a Add 'backup' block job type
A backup job may consist of many backup sub-blockjobs. Add the new
blockjob type and add all type converter strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
3656bb0a13 qemu: domain: Introduce QEMU_ASYNC_JOB_BACKUP async job type
We will want to use the async job infrastructure along with all the APIs
and event for the backup job so add the backup job as a new async job
type.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Peter Krempa
fed2cb487b qemu: Add infrastructure for statistics of a backup job
Introduce QEMU_DOMAIN_JOB_STATS_TYPE_BACKUP and the convertors and other
plumbing to be able to report statistics for the backup job.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:57 +01:00
Eric Blake
02f790ffbe backup: Parse and output backup XML
Accept XML describing a generic block job, and output it again as
needed. This may still need a few tweaks to match the documented XML
and RNG schema.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-12-10 12:41:56 +01:00