Commit Graph

103 Commits

Author SHA1 Message Date
Peter Krempa
4fcea23e6c testQEMUSchemaValidate(Command): Allow skipping validation of deprecated fields
Some test cases are used to validate interactions with old qemu. We need
to skip validation of deprecation with those once it will be added.

In case of commands which were already replaced by code based on
capabilities we can skip the full validation once the command is
removed.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-05-20 08:53:29 +02:00
Peter Krempa
6bde2a1e20 conf: Sanitize handling of <auth> and <encryption> placement for disks
Modern way to store <auth> and <encryption> of a <disk> is under
<source>. This was added to mirror how <backingStore> handles these and
in fact they are relevant to the source rather than to any other part of
the disk. Historically we allowed them to be directly under <disk> and
we need to keep compatibility.

This wasn't a problem until introduction of -blockdev in qemu using of
<auth> or <encryption> plainly wouldn't work with backing chains.

Now that it works in backing chains and can be moved back and forth
using snapshots/block-commit we need to ensure that the original
placement is properly kept even if the source changes.

To achieve the above semantics we need to store the preferred placement
with the disk definition rather than the storage source definitions and
also ensure that the modern way is chosen when the VM started with
<source/encryption> only in the backing store.

https://bugzilla.redhat.com/show_bug.cgi?id=1822878

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-05-12 06:55:00 +02:00
Peter Krempa
961ba2b969 qemuDomainValidateStorageSource: Allow masking out blockdev support
In case of 'sd' cards we'll use pre-blockdev code also if qemu supports
blockdev. In that specific case we'll need to mask out blockdev support
for 'sd' disks. Plumb in a boolean to allow it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-05-12 06:55:00 +02:00
Peter Krempa
514822dbbd qemuCheckDiskConfig: Remove and untangle callers
Remove the function and passing of 'def' through the callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-05-12 06:54:59 +02:00
Peter Krempa
ca03369c99 qemu: validate: Validate blkdeviotune settings in the validator
Move the code from qemuCheckDiskConfigBlkdeviotune in
src/qemu/qemu_commandline.c to
qemuValidateDomainDeviceDefDiskBlkdeviotune.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-05-12 06:54:59 +02:00
Han Han
94eaba0aef tests: Tests for io mode io_uring
Signed-off-by: Han Han <hhan@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2020-04-24 09:17:06 +02:00
Daniel Henrique Barboza
b8f1f037ed qemu: move qemuDomainDeviceDefValidateDisk() to qemu_validate.c
This function will remain public due to its usage in qemublocktest.c
even after moving qemuDomainDeviceDefValidate(). The position of its
header in qemu_validate.h is no accident.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2020-04-03 16:25:30 +02:00
Peter Krempa
1b84dd190c storage: Parse 'nvme' disk source properties from json:{} pseudo-uri
Our code allows snapshots of NVMe based disks which means we create
overlay file with a 'json:{}' pseudo-uri refering to the NVME device.
Our parser code doesn't handle them though. Add the parser and test it
via the XML->json->XML round-trip and reference data.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:48 +01:00
Peter Krempa
3b06103e69 qemuBlockGetBackingStoreString: Properly handle 'http/s' with cookies and others
Format cookies into the backing store string without encryption as they
will not be visible on the command line when formatting a 'target' only
string. In cases when cookies or other options are used we must use the
JSON format rather than pure URI.

Add tests to validate the scenario.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:48 +01:00
Peter Krempa
5f3b4a3727 qemublocktest: Test backing store strings
With -blockdev libvirt provides the string which is recorded  as
'backing store' property of an image to qemu. Add testing for
qemuBlockGetBackingStoreString which generates these strings as there's
logic which determines which format to use.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:47 +01:00
Peter Krempa
93171b63c3 testQemuDiskXMLToProps: Store all per-image data in one structure
We had two non-syncrhonized arrays holding the individual data. This was
a lazy way to do it when I was adding new tests recently. Since it's
hard to extend with new data to test refactor the storage of test data
to use a new struct where all per-image data are kept and can be
extended easily.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:47 +01:00
Peter Krempa
fed97cb435 testQemuDiskXMLToPropsValidateFileSrcOnly: Move together with rest of xml->json code
The function was misplaced. Group it together with other helper
functions for testing disk XML to qemu JSON props conversion.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:47 +01:00
Peter Krempa
4a9f355535 qemublocktest: xml->json: Refactor cleanup in test case functions
Use automatic variable clearing and remove the cleanup sections of
testQemuDiskXMLToProps, testQemuDiskXMLToPropsValidateSchema and
testQemuDiskXMLToPropsValidateFile.

testQemuDiskXMLToPropsValidateFileSrcOnly already uses new helpers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:47 +01:00
Peter Krempa
2337dbfdd1 qemublocktest: xml->json: Add test for NVMe
Based on the configuration from the only qemuxml2argv test.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:47 +01:00
Peter Krempa
4e9bb10cf3 qemublocktest: Add tests of broken bitmap chain handling during block-commit
Use the 'snapshots-synthetic-broken' test data for block-commit.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
8f096cd880 qemublocktest: Add more tests for block-commit bitmap handling with snapshots
Test handling of more complex cases of merging bitmaps accross
snapshots.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
77b9d574b4 qemublocktest: Add tests for handling of bitmaps during block-commit
Add code for testing the two necessary steps of handling bitmaps during
block commit and exercise the code on the test data which we have for
bitmap handling.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
f8389505aa qemublocktest: Fix and optimize fake image chain
Set the 'id' field of the backing chain properly so that we can look
up images, and initialize 6 images instead of 10 as we don't use more
currently.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
d6db013c6e qemu: Pass through arguments of 'ssh' block driver used by libguestfs
We currently don't model the 'ssh' protocol properties properly and
since it seems impossible for now (agent path passed via environment
variable). To allow libguestfs to work as it used in pre-blockdev era we
must carry the properties over to the command line. For this instance we
just store it internally and format it back.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
d089234110 qemublocktest: Add JSON->JSON test cases for block device backends
Add testing of the interpretation of the JSON pseudo-protocol backing
store into JSON structs for blockdev. This will be used to test JSON
pseudo-URIs used by libguestfs while actually also validating the output
against the QMP schema. Since libguestfs uses obsolete/undocumented
values the outputs will differ and a benefit is that modern output is
used now.

The example test case covers the fields and values used by libguestfs
when using the https driver.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
7f8d0ca56a qemublocktest: XMLjsonXML: Test formatting/parsing of modern JSON
The test was invoking the JSON formatter with the 'legacy' flag thus
formatting bunch of obsolete JSON blockdev definitions. We also should
test the modern ones. Add a boolean and re-run all the tests in both
cases.

Additionally for any modern invocation we should also validate that the
output conforms to the QAPI schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
218ce53069 qemublocktest: Extract schema root for blockdev-add validation
Move lookup of the schema root earlier so that multiple functions
can use it for validation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
1262cdede4 qemublocktest: Load QMP schema earlier
Multiple tests require the schema. Extract the loading into a separate
variable to avoid issues with ownership of the pointer.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
77194db01c virstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout'
Add support for parsing the recently added fields from backing file
pseudo-protocol strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Collin Walling
6523a7ea5d qemumonitorjsontest: load schema based on specified arch
There are some architectures that support capabilities that others
do not (e.g. s390x supports cpu comparison and baseline via QEMU).

Let's make testQEMUSchemaLoad accept a string to specify the schema
to load based on the specified arch.

Signed-off-by: Collin Walling <walling@linux.ibm.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-21 10:50:10 +01:00
Peter Krempa
9b804ef5ef tests: qemublock: Add cases for creating image overlays on top of disks with <slice>
Add a set of test data to see whether the backing store strings are
formatted reasonably. Note that we don't support direct creation of such
images so those tests are not enabled.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa
9b13af73ac tests: qemublock: Add tests for qemuBlockBitmapsHandleBlockcopy
Use some of the existing bitmap data to add tests for
qemuBlockBitmapsHandleBlockcopy.

As the output depends on the ordering in the hash table we must also
install the "virdeterministichash" mock preload library.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:33 +01:00
Peter Krempa
8e29a8b151 tests: qemublocktest: Add another synthetic test case for broken bitmaps
Add a case where a bitmap spanning multiple images is missing one of the
intermediate components.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:33 +01:00
Peter Krempa
41c7e5c2a6 qemu: block: Add validator for bitmap chains accross backing chains
Add a validator which checks that a bitmap spanning multiple backing
chain members doesn't look broken. The current rules are that no
intermediate birmaps are missing (unfortunately it's hard to know
whether the topmost or bottommost bitmap is missing) and none of the
components is inconsistent.

We can obviously improve it over time.

The validator is also tested against the existing bitmap data we have
for the backup merging test as well as some of the existing broken
bitmap synthetic test cases.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:33 +01:00
Peter Krempa
d69470a18a virJSONValueNewArray: Use g_new0 to allocate and remove NULL checks from callers
Use the glib allocation function that never returns NULL and remove the
now dead-code checks from all callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:33 +01:00
Peter Krempa
8e94e29010 qemu: checkpoint: Track and relabel images for bitmap merging
Allow qemu access to modify backing files in case when we want to delete
a checkpoint.

This patch adds tracking of which images need to be relabelled when
calculating the transaction, the code to relabel them and rollback.

To verify that stuff works we also output the list of images to relabel
into the test case output files in qemublocktest.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
065e548ebf tests: qemublock: Add checkpoint deletion tests for some special cases
Use the synthetic test data to verify that the algorithm correctly picks
bitmaps to merge when the bitmap is changed along with the image itself.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
3c1c35bada tests: qemublock: Add checkpoint deletion test for deep backing chain
Add test cases for merging various pairs of bitmaps when snapshots were
created together with checkpoints.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
30bc426071 qemu: checkpoint: Introduce support for deleting checkpoints accross snapshots
Allow deleting of checkpoints when snapshots were created along. The
code tracks and modifies the checkpoint list so that backups can still
be taken with such a backing chain. This unfortunately requires to
rename few bitmaps (by copying and deleting them) in some cases.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
d7d97e87af tests: qemublock: Add synthetic snapshot+checkpoint test data
Add a faked qemu output which would simulate scenario where libvirt
would take a snapshot and checkpoint simultaneously. This is visible in
libvirt-2-format node where bitmap 'c' appears, but bitmap 'b' which is
active in the previous layer is not present.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
0cf33ab8f1 tests: qemublock: Add few more test cases for checkpoint deletion
Add all intermediate steps and deletion of the current checkpoint on a
flat (single-image) disk image.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
25f7489972 tests: qemublock: Add test for checkpoint deletion bitmap merge
Add test infrastructure and a basic test for bitmap deletion.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Ján Tomko
0a125c7144 virBufferTrim: do not accept len
Always trim the full specified suffix.

All of the callers outside of tests were passing either
strlen or the actual length of the string.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2020-02-03 19:44:38 +01:00
Nikolay Shirokovskiy
dd94f36ffb qemu: check iotune params same for all disk in group
Currently it is possible to start a domain which have disks
in same iotune group and at the same time having different iotune
params. Both params set are passed to qemu in command line and the one
that is passed later down command line is get actually set.
Let's prohibit such configurations.

Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-01-29 11:46:51 +01:00
Peter Krempa
3f2d167d9c conf: Always format storage source auth and encryption under <source> for backing files
Historically there are two places where we format authentication and
encryption for a disk. The logich which formats it for backing files was
flawed though and didn't format it at all. This worked if the image
became a backing file through the means of a snapshot but not directly.

Force formatting of the source and encryption for any non-disk case to
fix the issue.

This caused problems in many places as we use the formatter to copy the
definition. Effectively any copy lost the secret definition.

https://bugzilla.redhat.com/show_bug.cgi?id=1789310
https://bugzilla.redhat.com/show_bug.cgi?id=1788898

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2020-01-13 12:53:58 +01:00
Ján Tomko
20071c9f99 Remove redundant usage of WITH_YAJL
As of commit 2a00ef6e71 which
was released in v5.2.0, we require YAJL to build the QEMU driver.

Remove the checks from code that requires the QEMU driver
or checks that also check for WITH_QEMU.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-13 14:57:32 +01:00
Peter Krempa
9851a0f390 tests: qemublock: Add tests for cross-snapshot incremental backups
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Peter Krempa
7dfd5d8d81 tests: qemublock: Add testing of bitmap merging for incremental backups
Add test code which will crawl a fake internal list of checkpoints and
generate the list of bitmaps for merging to gather the final bitmap for
the backup.

The initial tests cover the basic case of all bitmaps being present in
the top layer of the backing chain.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Peter Krempa
0b27b655b1 tests: qemublock: Add test case for detecting bitmaps as we create snapshots
Add test data gathered from a run of qemu after creating bitmaps and
snapshots together in various combinations.

The following sequence of commands was used to achieve the
configuration:

virsh checkpoint-create-as VM --name a
virsh snapshot-create-as VM --disk-only
virsh snapshot-create-as VM --disk-only
virsh checkpoint-create-as VM --name b
virsh checkpoint-create-as VM --name c
virsh snapshot-create-as VM --disk-only
virsh checkpoint-create-as VM --name d
virsh snapshot-create-as VM --disk-only
virsh checkpoint-create-as VM --name current

Note that VM was restarted after these operations to allow renumbering
of the bitmaps in a more human-readable way.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Peter Krempa
1326fd1ce6 tests: qemublocktest: Add a synthetic test case for bitmap detection
The real data gathered for the 'basic' test case don't exercise some
fields. Add a copy with a few values modified manually.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Peter Krempa
9aac9d5bda tests: qemublock: Add test for bitmap detection
Test the extraction of data about changed block tracking bitmaps. The
first test case adds a simple scenario of multiple bitmaps in one layer.

The test data will be also later reused for testing the code that
determines which bitmaps to merge for an incremental backup.

The sequence of bitmaps was created by the libvirt checkpoint API with
the following sequence of commands:

virsh checkpoint-create-as VM --name a
virsh checkpoint-create-as VM --name b
virsh checkpoint-create-as VM --name c
virsh checkpoint-create-as VM --name d
virsh checkpoint-create-as VM --name current

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Daniel P. Berrangé
99a949ffc4 conf: move seclabel validation into post-parse phase
Currently the disk and chardev seclabels are validated immediately at
the time their data is parsed. This forces the parser to fill in the
top level secmodel at time of parsing which is an undesirable thing.
This validation conceptually should be done in the post-parse phase
instead.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Michal Privoznik
4fa804c0c7 tests: Use g_strdup_printf() instead of virAsprintf()
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-11-12 16:15:59 +01:00
Peter Krempa
0967708b81 util: buffer: Remove virBufferCheckError
The function now does not return an error so we can drop it fully.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-10-24 19:35:34 +02:00
Peter Krempa
08e0ffe8f5 qemu: domain: Remove pointless return value in qemuDomainPrepareDiskSourceData
The function does not do anything that could fail. Remove the return
value.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-10-21 16:29:05 +02:00