Commit Graph

5992 Commits

Author SHA1 Message Date
Michal Privoznik
c799d150d5 virprocess: Passthru error from virProcessRunInForkHelper
When running a function in a forked child, so far the only thing
we could report is exit status of the child and the error
message. However, it may be beneficial to the caller to know the
actual error that happened in the child.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Mores <pmores@redhat.com>
2020-03-20 16:42:45 +01:00
Peter Krempa
096e94cef6 tests: virstoragetest: validate that array deflattening works for gluster
Validate that we are able to parse back the dotted syntax arrays we were
generating in the pre-blockdev era.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-20 09:47:16 +01:00
Peter Krempa
cdf82b60fd jsontest: Add test cases for deflattening of arrays
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-20 09:47:16 +01:00
Peter Krempa
3ceb6951bd virBitmapNewEmpty: Use g_new0 to allocate and remove error checking
virBitmapNewEmpty can't fail now so we can make it obvious and fix all
callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-20 09:47:16 +01:00
Peter Krempa
1a86e3b24a qemu: block: Split up formatting of JSON props for 'raw' and 'luks' drivers
qemuBlockStorageSourceGetFormatRawProps aggregated both formats but
since we now have props specific for either of those formats it's
unwanted to aggregate the code such way. Split out the 'luks' props
formatter into qemuBlockStorageSourceGetFormatLUKSProps.

The wrong separation demonstrates istself on formatting of the 'size'
and 'offset' attributes for the 'luks' driver which does not conform
to the qapi schema.

https://bugzilla.redhat.com/show_bug.cgi?id=1814975

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-20 09:47:16 +01:00
Peter Krempa
7393be3bbc qemuBlockStorageSourceNeedsStorageSliceLayer: Deal with 'luks' files
The 'luks' driver in qemu is as any other non-raw format driver and thus
doesn't support the properties for 'slice'. Since libvirt considers
luks files to be raw+encryption we need to special case them when
dealing with the slice.

https://bugzilla.redhat.com/show_bug.cgi?id=1814975

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-20 09:47:16 +01:00
Peter Krempa
2e3d17d7a3 qemuxml2argvdata/disk-slices: Add test case for 'luks' encryption
Since libvirt handles the luks encryption in a weird special way
(raw+encryption) we should really test that case with slices as well.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-20 09:47:16 +01:00
Pino Toscano
77696510e2 tests: switch away from HAVE_SOCKETPAIR
Since the removal of gnulib, HAVE_SOCKETPAIR is no more defined, making
these two tests effectively skipped.

Use the same strategy used in other generic library bits, i.e. exclude
the socketpair usage on Windows.

Semi-related change in virnetdaemontest.c to make it build: since
virutil.h does not include unistd.h anymore, we need to include it.

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-19 14:39:38 +01:00
Pino Toscano
c5ee737bc5 vmx: make 'fileName' optional for CD-ROMs
It seems like CD-ROMs may have no 'fileName' property specified in case
there is nothing configured as attachment for the drive. Hence, make
sure that virVMXParseDisk() do not consider it mandatory anymore,
considering it an empty block cdrom device. Sadly virVMXParseDisk() is
used also to parse disk and floppies, so make sure that a NULL fileName
is handled in cdrom- and floppy-related paths.

https://bugzilla.redhat.com/show_bug.cgi?id=1808610

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
2020-03-19 11:25:33 +01:00
Michal Privoznik
c9bd08ee35 conf: Don't generate clashing machine names for embed driver
So far, when using the qemu:///embed driver, management
applications can't chose whether they want to register their
domains in machined or not. While having that option is certainly
desired, it will require more work. What we can do meanwhile is
to generate names that include part of hash of the root
directory. This is to ensure that if two applications using
different roots but the same domain name (and ID) start the
domain no clashing name for machined is generated.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2020-03-18 15:52:08 +01:00
Peter Krempa
cab3622119 qemublocktest: Add tests for re-enabling of bitmaps after commit
Some branches were not covered and thus we didn't catch that the bitmaps
are not re-enabled if nothing is merged into them. Two bitmaps are
necessary to reliably test the case due to hash table ordering.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-17 16:32:13 +01:00
Peter Krempa
4e9bb10cf3 qemublocktest: Add tests of broken bitmap chain handling during block-commit
Use the 'snapshots-synthetic-broken' test data for block-commit.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
8f096cd880 qemublocktest: Add more tests for block-commit bitmap handling with snapshots
Test handling of more complex cases of merging bitmaps accross
snapshots.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
77b9d574b4 qemublocktest: Add tests for handling of bitmaps during block-commit
Add code for testing the two necessary steps of handling bitmaps during
block commit and exercise the code on the test data which we have for
bitmap handling.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
f8389505aa qemublocktest: Fix and optimize fake image chain
Set the 'id' field of the backing chain properly so that we can look
up images, and initialize 6 images instead of 10 as we don't use more
currently.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
41de7230ab qemu: blockjob: Store list of bitmaps disabled prior to commit
Starting a commit job will require disabling bitmaps in the base image
so that they are not dirtied by the commit job. We need to store a list
of the bitmaps so that we can later re-enable them.

Add a field and status XML handling code as well as a test.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Daniel P. Berrangé
c34b8cbf1c tests: validate parsing of CPUs with dies > 1
Add sample data files for validating handling of a QEMU guest started
with:

  -smp 7,maxcpus=16,sockets=2,dies=2,cores=2,threads=2

Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-16 16:00:27 +00:00
Peter Krempa
d6db013c6e qemu: Pass through arguments of 'ssh' block driver used by libguestfs
We currently don't model the 'ssh' protocol properties properly and
since it seems impossible for now (agent path passed via environment
variable). To allow libguestfs to work as it used in pre-blockdev era we
must carry the properties over to the command line. For this instance we
just store it internally and format it back.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
d089234110 qemublocktest: Add JSON->JSON test cases for block device backends
Add testing of the interpretation of the JSON pseudo-protocol backing
store into JSON structs for blockdev. This will be used to test JSON
pseudo-URIs used by libguestfs while actually also validating the output
against the QMP schema. Since libguestfs uses obsolete/undocumented
values the outputs will differ and a benefit is that modern output is
used now.

The example test case covers the fields and values used by libguestfs
when using the https driver.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
7f8d0ca56a qemublocktest: XMLjsonXML: Test formatting/parsing of modern JSON
The test was invoking the JSON formatter with the 'legacy' flag thus
formatting bunch of obsolete JSON blockdev definitions. We also should
test the modern ones. Add a boolean and re-run all the tests in both
cases.

Additionally for any modern invocation we should also validate that the
output conforms to the QAPI schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
218ce53069 qemublocktest: Extract schema root for blockdev-add validation
Move lookup of the schema root earlier so that multiple functions
can use it for validation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
1262cdede4 qemublocktest: Load QMP schema earlier
Multiple tests require the schema. Extract the loading into a separate
variable to avoid issues with ownership of the pointer.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
5179cc6b08 virStorageSourceParseBackingJSONUri: Handle undocumented value 'off' for sslverify
libguestfs abuses a quirk of qemu's parser to accept also other variants
of the 'sslverify' field which would be valid on the command line but
are not documented in the QMP schema.

If we encounter the 'off' string instead of an boolean handle it rather
than erroring out to continue support of pre-blockdev configurations.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
77194db01c virstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout'
Add support for parsing the recently added fields from backing file
pseudo-protocol strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
9cac141cd6 qemu: block: Implement readahead and timeout properties for 'curl' driver
Pass in the correct fields.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
06d3e8d539 qemu: block: Add support for HTTP cookies
Pass the alias of the secret object holding the cookie data as
'cookie-secret' to qemu.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
4e8faa5cdc qemu: domain: Store data for 'secret' object representing http cookies
The http cookies can have potentially sensitive values and thus should
not be leaked into the command line. This means that we'll need to
instantiate a 'secret' object in qemu to pass the value encrypted.

This patch adds infrastructure for storing of the alias in the status
XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
304da9376c qemu: block: Implement ssl verification configuration
Allow disabling of SSL certificate validation for HTTPS and FTPS drives
in qemu.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
3dd7952f6f qemuxml2argvtest: Add test case for disks with http(s) source
Upcoming patches will implement the support for sslverify, cookies,
readahead, and timeout properties. Add a test file which will collect
the cases.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
63fd461773 conf: Add support for setting timeout and readahead size for network disks
Some disk backends support configuring the readahead buffer or timeout
for requests. Add the knobs to the XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
3b076391be conf: Add support for cookies for HTTP based disks
Add possibility to specify one or more cookies for http based disks.
This patch adds the config parser, storage and validation of the
cookies.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
25481e25b1 conf: Add support for modifying ssl validation for https/ftps disks
To allow turning off verification of SSL cerificates add a new element
<ssl> to the disk source XML which will allow configuring the validation
process using the 'verify' attribute.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
70d2758a9c qemuDomainSecretStorageSourcePrepare: Change aliases for disk secrets
Originally there was only the secret for authentication so we didn't use
any suffix to tell it apart. With the introduction of encryption we
added a 'luks' suffix for the encryption secrets. Since encryption is
really generic and authentication is not the only secret modify the
aliases for the secrets to better describe what they are used for.

This is possible as we store the disk secrets in the status XML thus
only new machines will use the new secrets.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
d6498be165 qemu: capabilities: Introduce QEMU_CAPS_BLOCKDEV_SNAPSHOT_ALLOW_WRITE_ONLY
The capability is based on qemu's support of using blockdev-snapshot to
install backing chain also for images which are in use by a block-copy
job.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-13 13:09:12 +01:00
Peter Krempa
7c7fda15f3 qemu: capabilities: Update qemu-5.0.0 capabilities for x86_64
Update to v4.2.0-2265-g67923a7ea6 to pick up recent addition of
'allow-write-only-overlay' feature of 'blockdev-snapshot' command.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-13 13:09:12 +01:00
Daniel P. Berrangé
46e16b553d tests: fix double unlock of monitor in hotplug test
The qemuMonitorTestNew() function returns with the monitor object
locked, and expects it to still be locked when qemuMonitorTestFree
is called.  The qemuhotplug test, however, explicitly unlocks the
monitor, but then forgets to lock it again. As a result the
qemuMonitorTestFree function is unlocking a mutex it doesn't own.

This bug has existed forever, but since we use normal POSIX mutexes
and don't check the return value of pthread_mutex_lock/unlock we
didn't see the error. It was harmless until the switch to the per
monitor event loop which requires the thread synchronization to
work reliably, whereupon it started crashing.

Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-13 11:27:57 +00:00
Michal Privoznik
2695191a44 conf: Don't generate machine names with a dot
According to the linked BZ, machined expects either valid
hostname or valid FQDN (see systemd commit
v239-3092-gd65652f1f2). While in case of multiple dots, a
trailing one doesn't violate FQDN, it does violate the rule in
case of something simple, like "domain.". But it's safe to remove
it in both cases.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1808499
Fixes: 45464db8ba

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-13 11:59:55 +01:00
Nikolay Shirokovskiy
b47e3b9b5c qemu: agent: sync once if qemu has serial port event
Sync was introduced in [1] to check for ga presence. This
check is racy but in the era before serial events are available
there was not better solution I guess.

In case we have the events the sync function is different. It allows us
to flush stateless ga channel from remnants of previous communications.
But we need to do it only once. Until we get timeout on issued command
channel state is ok.

[1] qemu_agent: Issue guest-sync prior to every command

Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-12 18:07:50 +01:00
Michal Privoznik
a30078cb83 qemu: Create multipath targets for PRs
If a disk has persistent reservations enabled, qemu-pr-helper
might open not only /dev/mapper/control but also individual
targets of the multipath device. We are already querying for them
in CGroups, but now we have to create them in the namespace too.
This was brought up in [1].

1: https://bugzilla.redhat.com/show_bug.cgi?id=1711045#c61

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Tested-by: Lin Ma <LMa@suse.com>
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
2020-03-12 08:04:40 +01:00
Daniel P. Berrangé
a18f2c52ac qemu: convert agent to use the per-VM event loop
This converts the QEMU agent APIs to use the per-VM
event loop, which involves switching from virEvent APIs
to GMainContext / GSource APIs.

A GSocket is used as a convenient way to create a GSource
for a socket, but is not yet used for actual I/O.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-11 14:45:01 +00:00
Daniel P. Berrangé
436a56e37d qemu: convert monitor to use the per-VM event loop
This converts the QEMU monitor APIs to use the per-VM
event loop, which involves switching from virEvent APIs
to GMainContext / GSource APIs.

A GSocket is used as a convenient way to create a GSource
for a socket, but is not yet used for actual I/O.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-11 14:44:55 +00:00
Daniel P. Berrangé
ba906ab1c0 tests: start/stop an event thread for QEMU monitor/agent tests
Tests which are using the QEMU monitor / agent need to have an
event thread running a private GMainContext.

There is already a thread running the main libvirt event loop
but this can't be eliminated yet as it is used for more than
just the monitor client I/O.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-03-11 14:44:51 +00:00
Jiri Denemark
62a50628ff cputest: Add data for Intel(R) Core(TM) i7-8550U CPU without TSX
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2020-03-09 16:17:34 +01:00
Ján Tomko
181a945fd8 qemumonitorjsontest: GetCPUModelBaseline: use g_auto
Use g_autoptr for the virCPUDef variables.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-09 15:46:43 +01:00
Ján Tomko
4bc5a32e6f qemumonitorjsontest: GetCPUModelComparison: use g_auto
Use g_autoptr for the virCPUDef variables and get rid
of the cleanup label.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-09 15:46:43 +01:00
Ján Tomko
7b6308b725 virsystemdtest: do not leak socket path
Use an autofree'd helper variable to store the socket path
and free it after the function finishes.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Fixes: 5b8569dd6e
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-09 15:46:43 +01:00
Ján Tomko
9aa2426795 qemumonitorjsontest: use virCPUDefNew()
virCPUDefPtr uses refcounting internally and must be allocated
using virCPUDefNew, otherwise virCPUDefFree would be a no-op.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Fixes: fa2404bf4f
Fixes: eee09435ee
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-09 15:46:43 +01:00
Ján Tomko
01e68455b0 qemumonitorjsontest: do not leak qapiData.schema
Free the x86_64 schema before overwriting it with s390x schema.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Fixes: eee09435ee
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-09 15:46:43 +01:00
Ján Tomko
fc7981610c tests: valgrind: do not trace system binaries
Add /usr/bin/* to -trace-children-skip

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-09 15:46:42 +01:00
Ján Tomko
cb6ea86f6f tests: valgrind.supp: suppress g_type_register_static leaks
When a type is registered, it holds allocated memory until
the program exits.

Add an exception to valgrind.supp to make the output of
  make -C tests valgrind
more readable.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-09 15:46:42 +01:00