Commit Graph

24827 Commits

Author SHA1 Message Date
Peter Krempa
8e2a5c3a4c qemu: process: Don't trigger BLOCK_JOB* events with -blockdev
With blockdev we'll need to use the JOB_STATUS_CHANGE so gate the old
events by the blockdev capability.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
e55d64818d qemu: blockjob: Add 'concluded' state for a block job
This new state is entered when qemu finished the job but libvirt does
not know whether it was successful or not.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
3b6161a5f2 qemu: driver: Remove unnecessary saving of status XML
Now that the blockjob handling code deals with the status XML we don't
need to save it explicitly when starting blockjobs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
4cc4357f3e qemu: blockjob: Save status XML when modifying job state
Now that block job data is stored in the status XML portion we need to
make sure that everything which changes the state also saves the status
XML. The job registering function is used while parsing the status XML
so in that case we need to skip the XML saving.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
d3158852fa qemu: domain: Store blockjob data in the status XML
We need to store the block job state in the status XML so that we can
properly recover any data when reconnecting after startup and also in
the end to be able to do any transition of the backing chain that
happened while libvirt was not connected to the monitor.

First step is to note the name, type, state and corresponding disk into
the status XML.

We also need to make sure that a broken blockjob does not make libvirt
lose the VM, thus many of the errors just mark the job as invalid.
Later on we'll cancel all invalid jobs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
61e9066a69 qemu: blockjob: Add flag for invalid block job data
The job data saved in the XML may be partially invalid e.g. if something
is missing. To prevent losing a domain with such a job add a flag to the
job data so that job APIs can ignore such a job and we can just cancel
it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
3dc496e098 qemu: blockjob: Export functions for allocating and registering job data
When parsing the status XML we need to register all existing jobs.
Export the functions so that they are usable in other modules.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
8d82e6d98a qemu: blockjob: Add string convertors for blockjob type and state enums
Later on we'll format these values into the status XML so the from/to
string functions will come handy. The implementation also notes that
these will be used in the status XML to avoid somebody changing the
values.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
5ff46aaa7f qemu: blockjob: Register new and running blockjobs in the global table
Add the job structure to the table when instantiating a new job and
remove it when it terminates/fails.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
acff582915 qemu: domain: Add global table of blockjobs
Block jobs currently belong to disks only so we can look up the block
job data for them in the corresponding disks. This won't be the case
when using blockdev as certain jobs don't even correspond to a disk and
most of them can run on a part of the backing chain.

Add a global table of blockjobs which can be used to look up the data
for the blockjobs when the job events need to be processed.

The table is a hash table organized by job name and has a reference to
the job. New and running jobs will later be added to this table.
Reference counting will allow to reap job state for synchronous callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
9dd12d4ecf qemu: blockjob: Update new job state earlier in qemuBlockJobEventProcessLegacy
The legacy job handler does not look at the old job state so we can
update it earlier.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
b6316435e4 qemu: blockjob: Save config only in qemuBlockJobEventProcessLegacyCompleted
There's no need to do it if the job is not completed. The new helper
allows to do this with much less hassle in the correct place.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
dbdda6aca0 qemu: blockjob: Separate and unify block job (un)registration
Rename and move qemuBlockJobTerminate to qemuBlockJobUnregister and
separate bits from qemuBlockJobDiskNew which register the job with the
disk. This creates an unified interface for other APIs to use.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
0610aa51c4 qemu: blockjob: Use VIR_AUTOUNREF in qemuBlockJobDataNew
Simplify error paths.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
a3b0e09242 qemu: domain: Add helper for saving config XML
Similarly to qemuDomainSaveStatus add a helper to save the config XML
named qemuDomainSaveConfig.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
997746b2df qemu: domain: Repurpose and export helper for saving domain status XML
Rename qemuDomainObjSaveJob and create a wrapper for it which does not
require 'driver' to be passed and export it so that other palces can
easily save the status XML without having to invoke virDomainSaveStatus
which has unpleasing parameters.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
50e1e053a5 conf: snapshot: Provide a function to free virDomainSnapshotDiskDefPtr
Tests will need to parse such a definition so it also needs to be freed.
Provide a function for it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
3b8135b03e conf: snapshot: Export virDomainSnapshotDiskDefParseXML
Allow using it from the tests.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
b84c09f41a qemu: domain: Export qemuDomainPrepareStorageSourceBlockdev
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
ed812441b3 qemu: block: Add generator for creating storage with blockdev-create
QEMU allows us to create storage on certain network protocols which
allow image creation through their API. Wire up the generator for using
it with libvirt as well as for local files.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
2d593705c8 qemu: block: Add generator for image format creation properties
'blockdev-add' allows us to use qemu to format images to our desired
format. This patch implements helpers which convert a
virStorageSourcePtr into JSON objects describing the required
configuration.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
2c4c347c4b qemu: block: Use 'auto-read-only' instead of 'read-only' for backing chain
To allow using -blockdev with blockjobs QEMU needs to reopen files in
read-write mode when modifying the backing chain. To achieve this we
need to use 'auto-read-only' for the backing files rather than the
normal 'read-only' property. That way qemu knows that the files need to
be reopened.

Note that the format drivers (e.g. qcow2) are still opened with the
read-only property enabled when being a member of the backing chain
since they are supposed to be immutable unless a block job is started.

QEMU v4.0 (since commit 23dece19da4) allows also dynamic behaviour for
auto-read-only which allows us to use sVirt as we only grant write
permissions to files when doing a blockjob.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
8e0b1f8a9e qemu: block: Extract formatting of 'driver' attribute from child formatters
To allow reusing the formatters in the code for creating JSON properties
for 'blockdev-create' we need to create everything except the 'driver'
attribute.

Use the new helper virJSONValueObjectPrependString to put the driver at
the same place so that we don't change any output.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
12a330b50c util: json: Introduce helper to prepend string into a virJSONValueObject
Libvirt treats the JSON objects as lists thus the values appear in the
order they were added. To avoid too much changes introduce a helper
which allows to prepend a string which will allow to keep certain
outputs in order.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
423b4f6625 qemu: block: Allow skipping non-target related data when formating disk JSON
When formatting new qcow2 images we need to provide the backing store
string which should not contain any authentication or irrelevant data.

Add a flag for qemuBlockStorageSourceGetBackendProps which allows to
skip the irrelevant data.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
aa7d73134f qemu: monitor: Add APIs for 'blockdev-create'
The 'blockdev-create' starts a job which creates a storage volume using
the given protocol or formats an existing (added) volume with one of the
supported storage formats.

This patch adds the monitor interaction bits.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
587c0ed12a qemu: monitor: Implement support for 'JOB_STATUS_CHANGE' event
This new event is a superset of the BLOCK_JOB* events and also covers
jobs which don't bind to a VM disk.

In this patch the monitor part is implemented.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
ed56851f1b qemu: monitor: Add infrastructure for 'query-jobs'
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
93de886b10 qemu: monitor: Add support for 'job-complete' command
This belongs to the new job management API which can manage also
non-block based jobs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
1d2e044302 qemu: monitor: Add support for 'job-cancel' command
This belongs to the new job management API which can manage also
non-block based jobs. Since we'll need to be able to attempt to cancel
jobs which potentially were not started (during reconnect) the 'quiet'
flag allows to suppress errors reported.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
190e66ea5d qemu: monitor: Add support for 'job-dismiss' command
This belongs to the new job management API for generic jobs.

The dismiss command is meant to remove a concluded job after we were
able to get the final status.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
e53adccebd qemu: monitor: Add new fields for 'blockdev-mirror' command
Allow using the delayed dismiss of the job so that we can reap the state
even if libvirtd was not running when qemu emitted the job completion
event.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
ac6c579af3 qemu: monitor: Add new fields for 'block-commit' command
Allow using the node name to specify the base and top of the 'commit'
operation, allow specifying explicit job name and add support for
delayed dismiss of the job so that we can reap the state even if
libvirtd was not running when qemu emitted the job completion event.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
a4f10a6821 qemu: monitor: Add new fields for 'block-stream' command
Allow using the node name to specify the base of the 'stream' operation,
allow specifying explicit job name and add support for delayed dismiss
of the job so that we can reap the state even if libvirtd was not
running when qemu emitted the job completion event.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:33 +02:00
Peter Krempa
f0430d069a util: storage: Don't leak metadata on repeated calls of virStorageFileGetMetadata
When querying storage metadata after a block job we re-run
virStorageFileGetMetadata on the top level storage file. This means that
the workers (virStorageFileGetMetadataInternal) must not overwrite any
pointers without freeing them.

This was not considered for src->compat and src->features. Fix it and
add a comment mentioning that.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-07-18 17:54:22 +02:00
Peter Krempa
5b8e64f0bc util: storage: Clean up label use in virStorageFileGetMetadataInternal
The function does not do any cleanup, so replace the 'cleanup' label
with return of -1 and the 'done' label with return of 0.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-07-18 17:54:22 +02:00
Michal Privoznik
112174b205 network: Link with libxml2
Since fb9f6ce625 we are including a libxml header file in the
network driver but never link with it. This hasn't caused an
immediate problem because in the end the network driver links
with libvirt.la. But apparently, it's causing a build issue on
old Ubuntu.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2019-07-18 16:57:02 +02:00
Peter Krempa
7283439837 util: command: Ignore bitmap errors when enumerating file descriptors to close
virCommandMassCloseGetFDsLinux fails when running libvird on valgrind
with the following message:

libvirt:  error : internal error: unable to set FD as open: 1024

This is because valgrind opens few file descriptors beyond the limit:

65701125 lr-x------. 1 root root 64 Jul 18 14:48 1024 -> /home/pipo/build/libvirt/gcc/src/.libs/libvirtd
65701126 lrwx------. 1 root root 64 Jul 18 14:48 1025 -> '/tmp/valgrind_proc_3849_cmdline_186612e3 (deleted)'
65701127 lrwx------. 1 root root 64 Jul 18 14:48 1026 -> '/tmp/valgrind_proc_3849_auxv_186612e3 (deleted)'
65701128 lrwx------. 1 root root 64 Jul 18 14:48 1027 -> /dev/pts/11
65701129 lr-x------. 1 root root 64 Jul 18 14:48 1028 -> 'pipe:[65689522]'
65701130 l-wx------. 1 root root 64 Jul 18 14:48 1029 -> 'pipe:[65689522]'
65701131 lr-x------. 1 root root 64 Jul 18 14:48 1030 -> /tmp/vgdb-pipe-from-vgdb-to-3849-by-root-on-angien

Ignore bitmap errors in this case since we'd leak those FD's anyways in
the previous scenario.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-07-18 15:55:16 +02:00
Michal Privoznik
9bc01ad87c libvirt_private.syms: Properly expose virPCI* function from virpci.h
There are couple of functions that are meant to be exposed but
are missing syms file adjustment.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2019-07-18 15:35:47 +02:00
Michal Privoznik
c012e0f7fa virSecurityManagerMetadataLock: Skip over duplicate paths
If there are two paths on the list that are the same we need to
lock it only once. Because when we try to lock it the second time
then open() fails. And if it didn't, locking it the second time
would fail for sure. After all, it is sufficient to lock all
paths just once satisfy the caller.

Reported-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-07-18 15:13:13 +02:00
Michal Privoznik
ee6501ab05 virSecurityManagerMetadataLock: Expand the comment on deadlocks
Document why we need to sort paths while it's still fresh in my
memory.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-07-18 15:13:05 +02:00
Julio Faracco
48df09874b util: Fix uninitalized variable to avoid garbage value.
This commit is similar with 596aa144. It fixes an uninitialized
variable to avoid garbage value. This case, it uses time 't' 0 if
an error occurs with virTimeMillisNowRaw.

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-07-18 11:31:19 +02:00
Cole Robinson
fb9f6ce625 network: wire up dnsmasq option xmlns
This maps to XML like:

  <network xmlns:dnsmasq='http://libvirt.org/schemas/network/dnsmasq/1.0'>
    ...
    <dnsmasq:options>
      <dnsmasq:option value="foo=bar"/>
      <dnsmasq:option value="cname=*.foo.example.com,master.example.com"/>
    </dnsmasq:options>
  </network>

To dnsmasq config options

  ...
  foo=bar
  cname=*.foo.example.com,master.example.com

Reviewed-by: Laine Stump <laine@laine.org>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-07-17 17:18:56 -04:00
Cole Robinson
2dde2dbba1 conf: Add virNetworkXMLNamespace
Just the plumbing, no real implementation yet

Reviewed-by: Laine Stump <laine@laine.org>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-07-17 17:18:56 -04:00
Cole Robinson
8911d843f3 conf: Add network xmlopt argument
Pass an xmlopt argument through all the needed network conf
functions, like is done for domain XML handling. No functional
change for now

Reviewed-by: Laine Stump <laine@laine.org>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-07-17 17:18:56 -04:00
Cole Robinson
dc2c4f8435 conf: Add virNetworkXMLOption
Just a stub for now that is unused. Add init+cleanup plumbing and
demostrate it in bridge_driver.c

Reviewed-by: Laine Stump <laine@laine.org>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
2019-07-17 17:18:55 -04:00
Ján Tomko
898821cce8 qemu: command: remove qemuDomainFSDriver
Having a translation enum full of empty strings seems excessive.
Now that the validiation is performed in qemuDomainDeviceDefValidateFS,
remove it completely and open-code the two allowed cases.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
2019-07-16 17:00:37 +02:00
Ján Tomko
e6e7c41f84 qemu: command: use VIR_AUTOCLEAN in qemuBuildFS*
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
2019-07-16 17:00:37 +02:00
Ján Tomko
acef350080 qemu: introduce qemuDomainDeviceDefValidateFS
Move validation of the filesystem device out of qemu_command.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
2019-07-16 17:00:37 +02:00
Ján Tomko
77570e2600 qemu: command: use VIR_AUTOFREE in qemuBuildFSDevCommandLine
Introduce two separate variables instead of reusing the same one
for clarity.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
2019-07-16 17:00:37 +02:00
Ján Tomko
da0f5aab3e qemu: command: re-introduce qemuBuildFSDevCommandLine
This time it only builds one device.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
2019-07-16 17:00:36 +02:00
Ján Tomko
d8f8f1d172 qemu: command: rename qemuBuildFSDevCommandLine
This function iterates over all filesystems, not just -fsdevs.

Rename it to free the name for a function that actually builds fsdevs.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
2019-07-16 17:00:36 +02:00
Ján Tomko
628709245b qemu: address: remove useless comment
Commit b27375a9b8 omitted one zero.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
2019-07-16 17:00:36 +02:00
Michal Privoznik
432faf259b virCommand: use procfs to learn opened FDs
When spawning a child process, between fork() and exec() we close
all file descriptors and keep only those the caller wants us to
pass onto the child. The problem is how we do that. Currently, we
get the limit of opened files and then iterate through each one
of them and either close() it or make it survive exec(). This
approach is suboptimal (although, not that much in default
configurations where the limit is pretty low - 1024). We have
/proc where we can learn what FDs we hold open and thus we can
selectively close only those.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-16 13:47:49 +02:00
Michal Privoznik
c1a9bfbbba vircommand: Separate mass FD closing into a function
I will optimize this code a bit in the next commit. But for that
it is better if the code lives in a separate function.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-16 13:43:43 +02:00
Michal Privoznik
cc34260f5a test: Introduce virnetdevopenvswitchtest
Test if our parsing of interface stats as returned by ovs-vsctl
works as expected. To achieve this without having to mock
virCommand* I'm separating parsing of stats into a separate
function.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-16 13:42:39 +02:00
Michal Privoznik
c297eab525 virNetDevOpenvswitchInterfaceStats: Optimize for speed
We run 'ovs-vsctl' nine times (first to find if interface is
there and then eight times = for each stats member separately).
This is very inefficient. I've found a way to run it once and
with a bit of help from virJSON module we can parse out stats
we need.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-16 13:39:16 +02:00
Ilias Stamatis
cbfc84f7bc test_driver: implement virDomainSetMemoryParameters
Signed-off-by: Ilias Stamatis <stamatis.iliass@gmail.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-07-15 15:24:53 +02:00
Daniel P. Berrangé
8486611548 interface: fix driver name in state directory path
Typo meant we use 'nodedev' instead of 'interface'. This doesn't hurt
libvirtd because if a process tries to acquire a lock it already holds
it will succeed. It fails when nodedev & interface drivers are in
separate daemons though.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-15 13:36:45 +01:00
Daniel P. Berrangé
cb1938eb58 all: don't wait for driver lock during startup
When the drivers acquire their pidfile lock we don't want to wait if the
lock is already held. We need the driver to immediately report error,
causing the daemon to exit.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-15 13:36:45 +01:00
Ilias Stamatis
38e9e7e0d7 test_driver: implement virDomainSetUserPassword
Signed-off-by: Ilias Stamatis <stamatis.iliass@gmail.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-07-15 14:26:29 +02:00
Christian Ehrhardt
bc85411c49
security: aa-helper: allow virt-aa-helper to read .vhd images
VHD images can be used as any other, so we should add them to the list
of types that virt-aa-helper can read when creating the per-guest rules
for backing files.

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Acked-by: Jamie Strandboge <jamie@canonical.com>
2019-07-15 13:43:51 +02:00
Michal Privoznik
7711a7346a qemu: Relax os.loader->type check when validating domain
When validating a domain among all the checks there are two that
concern VIR_DOMAIN_LOADER_TYPE_PFLASH specifically. The first
check ensures that on x86 ACPI is enabled when UEFI is requested,
the second ensures that UEFI is used when ACPI is requested on
aarch64. However, check for UEFI is done by plain comparison of
def->os.loader->type which is insufficient because we have
def->os.firmware too.

NB, this wouldn't be a problem for active domain, because on
startup process def->os.loader->type gets filled by
qemuFirmwareEnableFeatures(), but that's not the case for
inactive domains.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1729604

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2019-07-15 13:24:09 +02:00
Peter Krempa
35e3069ce4 qemu: block: Split up qemuBlockStorageSourceAttachApply
Split up the addition of a storage source into the following sub-steps:
1) storage access dependencies (TLS transport, persistent reservation)
2) storage acccess node (file/gluster/nbd...)
3) format driver dependencies (encryption secret)
4) format driver node (qcow2, raw, ...)

The functions split out will be later reused when implementing support
for 'blockdev-create' as we'll need the dependencies plugged in first,
then blockdev-create will be called and after that successfully finishes
blockdev-add will be added.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-15 10:26:23 +02:00
Peter Krempa
16ca234b56 qemu: Refactor variables for extracting flags in qemuDomainBlockCopyCommon
Add separate booleans for extracting VIR_DOMAIN_BLOCK_COPY_REUSE_EXT and
VIR_DOMAIN_BLOCK_COPY_SHALLOW from '@flags' and also change 'reuse' into
'existing'.

qemuMonitorDriveMirror requires the unmodified state of the flags to
pass to qemu and also we use the value a few times internally. Extract
it separately now.

The 'reuse' flag did not indicate reusing of the file as much as the
fact that the storage is existing and thus should not be created, so
modify the name to reflect this.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-15 10:26:23 +02:00
Peter Krempa
0db5912617 qemu: blockjob: Don't emit traditional disk events for jobs without disk
With -blockdev it will be possible that a block job loses the disk that
was used to start it to a guest-initiated hot-unplug. Don't emit the
block job events in that case as we can't report the top level source or
disk target for an unplugged (and potentially replugged with different
source) disk.

Eventually when we add machinery for tracking jobs globally for a VM the
event will be reinstated via the domain job event.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-15 10:26:23 +02:00
Peter Krempa
3149e00ab1 qemu: blockjob: Don't reset state when entering sync blockjob
job->newstate is now used internally all the time so there's no need to
clear it as it already has correct value.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-15 10:26:23 +02:00
Jonathon Jongsma
e579f5300b qemu: add 'bochs' video display type
Update schema and configuration to allow specifying new video type of
'bochs'. Add implementation and tests for qemu.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2019-07-15 10:21:21 +02:00
Jonathon Jongsma
086c19d699 qemu: Add bochs-display capability
Check whether qemu supports the bochs-display device and set a
capability. Update tests.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2019-07-15 10:21:21 +02:00
Han Han
3efdffc450 storage: Add iscsi-direct pool list type flag
Add pool list type flag VIR_CONNECT_LIST_STORAGE_POOLS_ISCSI_DIRECT,
which was forgotten when introducing iscsi-direct pool at f0bf1be3.

https://bugzilla.redhat.com/show_bug.cgi?id=1726609

Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-07-13 17:40:09 +02:00
Han Han
8462467cf7 storage: Add missing pool type flags in comment
Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-07-13 17:39:06 +02:00
Eric Blake
e3989ce3ed snapshot: Factor out redefine cycle validation
The code to check whether a redefined snapshot/checkpoint XML is
attempting to create a cycle in the list of moments is lengthy, and
common between the two types of list. Therefore, it belongs in the
shared base file.

Signed-off-by: Eric Blake <eblake@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
2019-07-12 15:12:29 -05:00
Daniel P. Berrangé
597bded48d rpc: remove unused typedef for auto shutdown function callback
The use of the virNetServerAutoShutdownFunc typedef was removed in

  commit 79b8a56995
  Author: Daniel P. Berrange <berrange@redhat.com>
  Date:   Wed Oct 31 19:03:55 2012 +0000

    Replace polling for active VMs with signalling by drivers

This unused typedef was then copied into the virNetDaemon object
when that was split off from virNetServer, resulting in a typedef
virNetDaemonAutoShutdownFunc that has never been needed.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
a1dd97cd8d util: remove unused helper for getting UNIX socket path
The new systemd activation APIs mean there is no longer a need to get
the UNIX socket path associated with a plain FD.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
2879315d39 util: move code for getting listen FDs into systemd module
The virGetListenFDs method no longer needs to be called directly, so it
can be a static function internal to the systemd code.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
7ac64b5c86 logging: convert log daemon to use systemd activation APIs
Using the new system activation APIs allows for simpler code setting up
the network services.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
26ae4e1a92 locking: convert lock daemon to use systemd activation APIs
Using the new system activation APIs allows for simpler code setting up
the network services.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
2dfacbffea util: remove code spawning with systemd activation env vars
The only use of this code was removed by:

  commit be78814ae0
  Author: Michal Privoznik <mprivozn@redhat.com>
  Date:   Thu Apr 2 14:41:17 2015 +0200

    virNetSocketNewConnectUNIX: Use flocks when spawning a daemon

less than a year after it was first introduced in

  commit 1b807f92db
  Author: Martin Kletzander <mkletzan@redhat.com>
  Date:   Wed Jul 16 08:00:19 2014 +0200

    rpc: pass listen FD to the daemon being started

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
ee04bfaf9d remote: update config files to note usage wrt systemd socket activation
Certain libvirtd.conf settings are not honoured when using systemd
socket activation.

Certain systemd unit file settings must match those defined in
libvirtd.conf for systemd socket activation to work with systemd
version < 227, otherwise libvirtd cannot determine which inherited
FD to use for which service.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
4304222f50 remote: make system libvirtd exit when idle via timeout
Since we have socket activation available now, we can let the system
libvirtd exit when it is idle. This allows it to still do autostart
when the host boots up, but when nothing was started it will quickly
exit again until some mgmt app connects to the socket.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
2c3e08b0e3 remote: add systemd socket units for UNIX/TCP sockets
We don't do socket activation of libvirtd, since we need to
unconditionally start libvirtd in order to perform autostart. This
doesn't mean we can't have systemd socket units. Some use cases will
not need libvirt's autostart & are thus free to use activation.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
03fd51051e rpc: remove unused API for creating services from FDs
The virNetServerServiceNewFDOrUNIX method cannot be correctly used when
dealing with systemd activation of a service which can receive more than
one socket FD as there is not guaranteed ordering of FDs.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
e2f012b8f0 remote: fix handling of systemd activation wrt socket ordering
The current libvirtd code for systemd socket activation assumes socket
FDs are passed in the order unix-rw, unix-ro, unix-admin.  There is in
fact no ordering guarantee made by systemd. Applications are expected
to check the address or name associated with each FD to figure out its
identity.

This rewrites libvirtd to make use of the new systemd activation APIs
to make it robust wrt socket ordering changes.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
e9095c328a remote: simplify libvirtd code for deciding if SASL auth is needed
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
49fa9e64ca rpc: add API for checking whether an auth scheme is in use on a server
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:40 +01:00
Daniel P. Berrangé
9692fe10eb rpc: add helper APIs for adding services with systemd activation
Currently code has to first create the service and then separately
register it with the server. If the socket associated with a particular
service is not passed from systemd we want to skip creating the service
altogether. This means we can't put the systemd activation logic into
the constructors for virNetServerService.

This patch thus creates some helper methods against virNetServer which
combine systemd activation, service creation and service registration
into one single operation. This operation is automatically a no-op if
systemd activation is present and no sockets were passed in.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:39 +01:00
Daniel P. Berrangé
1519c55dc8 rpc: avoid unlinking sockets passed in from systemd
Currently the socket code will unlink any UNIX socket path which is
associated with a server socket. This is not fine grained enough, as we
need to avoid unlinking server sockets we were passed by systemd.

To deal with this we must explicitly track whether each socket needs to
be unlinked when closed, separately of the client vs server state.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:39 +01:00
Daniel P. Berrangé
9f3d1c5c8f rpc: allow creating RPC service from an array of FDs
The virNetServerServiceNewFD API only accepts a single FD, but it is
easily changed to allow for an array of FDs to be passed in.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:39 +01:00
Daniel P. Berrangé
3b6bfde089 rpc: refactor RPC service constructors to share more code
Introduce a virNetServerServiceNewSocket API that allows the various
constructors to share more code.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:39 +01:00
Daniel P. Berrangé
5b8569dd6e util: add APIs for facilitating use of systemd activation FDs
When receiving multiple FDs from systemd during service activation it is
neccessary to identify which purpose each FD is used for. While this
could be inferred by looking for the specific IP ports or UNIX socket
paths, this requires the systemd config to always match what is expected
by the code. Using systemd FD names we can remove this restriction and
simply identify FDs based on an arbitrary name.

The FD names are passed by systemd in the LISTEN_FDNAMES env variable
which is populated with the socket unit file names, unless overriden
by using the FileDescriptorName setting.

This is supported since the system 227 release and unfortunately RHEL7
lacks this version. Thus the code has some back compat support whereby
we look at the TCP ports or the UNIX socket paths to identify what
socket maps to which name. This back compat code is written such that
is it easly deleted when we are able to mandate newer systemd.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:39 +01:00
Daniel P. Berrangé
f8b3905edf rpc: ensure all sockets bind to same port when service is NULL
When the service passed to getaddrinfo is NULL the kernel will choose a
free port to bind to. In a dual stack though we will get separate
sockets for IPv4 and IPv6 and we need them to bind to the same port
number. Thus once the kerel has auto-selected a port for the first
socket, we must disable auto-select for subsequent IP sockets and force
reuse of the first port.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 16:55:39 +01:00
Ján Tomko
b632e00ffb storage: rbd: do not attempt to dereference a non-pointer
My commit 9b7c4048fa was too blind
and my librbd was not new enough to actually compile this part.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2019-07-12 17:51:39 +02:00
Ján Tomko
5d74619329 Revert "conf: Remove volOptions for VIR_STORAGE_POOL_RBD"
This reverts commit 035db37394

Even though we only allow using RBD with raw volumes,
removing the options and the default format causes our
parser not to fill out the volume format and the backend code
rejects creating a non-raw volume.

Re-introduce the volume options to fix volume creation while
erroring out on requests to use non-raw formats.

https://bugzilla.redhat.com/show_bug.cgi?id=1724065

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 17:09:43 +02:00
Ján Tomko
9b7c4048fa storage: rbd: actually index the array when iterating over it
https://bugzilla.redhat.com/show_bug.cgi?id=1729292

Fixes: 3aa190f2a4
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 17:09:42 +02:00
Ján Tomko
d43bc53edd storage: rbd: use VIR_REALLOC in the loop
If there are more than 16 images, the memory allocated in images
might be leaked on subsequent execution(s).

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 17:09:42 +02:00
Jonathon Jongsma
062b4a4cd7 qemu: minor refactor of video device string handling
In preparation for adding the bochs display device, refactor the logic
so that each branch handles a single device type and checks its
parameters within that branch. In this case VGA and VMVGA are still
grouped into the same branch since they share device-specific parameter
names.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2019-07-12 16:54:32 +02:00
Michal Privoznik
87050bcede virnetworkobj: Drop needless cleanup label in virNetworkObjDeletePort
The cleanup label in virNetworkObjDeletePort() function serves no
purpose. Drop it and thus simplify the function a bit.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-07-12 16:26:09 +02:00
Michal Privoznik
6246400329 virnetworkobj: Drop needless cleanup label in virNetworkObjAddPort
The cleanup label in virNetworkObjAddPort() function serves no
purpose. Drop it and thus simplify the function a bit.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-07-12 16:25:57 +02:00
Michal Privoznik
37d8d6b98d virnetworkobj: Free retval of virNetworkObjGetPortStatusDir()
The virNetworkObjGetPortStatusDir() function allocates a memory
to construct a path. None of the callers free it leading to a
memleak.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-07-12 16:19:43 +02:00
Daniel P. Berrangé
975b004d07 logging: ensure virtlogd rollover takes priority over logrotate
The virtlogd config is set to rollover logs every 2 MB.

Normally a logrotate config file is also installed to handle cases where
virtlogd is disabled. This is set to rollover weekly with no size
constraint.

As a result logrotate can interfere with virtlogd's, rolling over files
that virtlogd has already taken care of.

This changes logrotate configs to rollover based on a max size
constraint of 2 MB + 1 byte. When virtlogd is running the log files will
never get this large, making logrotate a no-op.

If the user changes the size in virtlogd's config to something larger,
they are responsible for also changing the logrotate config suitably.

The LXC/libxl drivers don't use virtlogd, but there logrotate config is
altered to match the QEMU driver config, for the sake of consistency.

Reviewed-by: Jim Fehlig <jfehlig@suse.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-07-12 12:44:59 +01:00