Commit Graph

136 Commits

Author SHA1 Message Date
Peter Krempa
f8389505aa qemublocktest: Fix and optimize fake image chain
Set the 'id' field of the backing chain properly so that we can look
up images, and initialize 6 images instead of 10 as we don't use more
currently.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-16 17:33:08 +01:00
Peter Krempa
d6db013c6e qemu: Pass through arguments of 'ssh' block driver used by libguestfs
We currently don't model the 'ssh' protocol properties properly and
since it seems impossible for now (agent path passed via environment
variable). To allow libguestfs to work as it used in pre-blockdev era we
must carry the properties over to the command line. For this instance we
just store it internally and format it back.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
d089234110 qemublocktest: Add JSON->JSON test cases for block device backends
Add testing of the interpretation of the JSON pseudo-protocol backing
store into JSON structs for blockdev. This will be used to test JSON
pseudo-URIs used by libguestfs while actually also validating the output
against the QMP schema. Since libguestfs uses obsolete/undocumented
values the outputs will differ and a benefit is that modern output is
used now.

The example test case covers the fields and values used by libguestfs
when using the https driver.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
7f8d0ca56a qemublocktest: XMLjsonXML: Test formatting/parsing of modern JSON
The test was invoking the JSON formatter with the 'legacy' flag thus
formatting bunch of obsolete JSON blockdev definitions. We also should
test the modern ones. Add a boolean and re-run all the tests in both
cases.

Additionally for any modern invocation we should also validate that the
output conforms to the QAPI schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
218ce53069 qemublocktest: Extract schema root for blockdev-add validation
Move lookup of the schema root earlier so that multiple functions
can use it for validation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
1262cdede4 qemublocktest: Load QMP schema earlier
Multiple tests require the schema. Extract the loading into a separate
variable to avoid issues with ownership of the pointer.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
77194db01c virstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout'
Add support for parsing the recently added fields from backing file
pseudo-protocol strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Collin Walling
6523a7ea5d qemumonitorjsontest: load schema based on specified arch
There are some architectures that support capabilities that others
do not (e.g. s390x supports cpu comparison and baseline via QEMU).

Let's make testQEMUSchemaLoad accept a string to specify the schema
to load based on the specified arch.

Signed-off-by: Collin Walling <walling@linux.ibm.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-21 10:50:10 +01:00
Peter Krempa
9b804ef5ef tests: qemublock: Add cases for creating image overlays on top of disks with <slice>
Add a set of test data to see whether the backing store strings are
formatted reasonably. Note that we don't support direct creation of such
images so those tests are not enabled.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa
9b13af73ac tests: qemublock: Add tests for qemuBlockBitmapsHandleBlockcopy
Use some of the existing bitmap data to add tests for
qemuBlockBitmapsHandleBlockcopy.

As the output depends on the ordering in the hash table we must also
install the "virdeterministichash" mock preload library.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:33 +01:00
Peter Krempa
8e29a8b151 tests: qemublocktest: Add another synthetic test case for broken bitmaps
Add a case where a bitmap spanning multiple images is missing one of the
intermediate components.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:33 +01:00
Peter Krempa
41c7e5c2a6 qemu: block: Add validator for bitmap chains accross backing chains
Add a validator which checks that a bitmap spanning multiple backing
chain members doesn't look broken. The current rules are that no
intermediate birmaps are missing (unfortunately it's hard to know
whether the topmost or bottommost bitmap is missing) and none of the
components is inconsistent.

We can obviously improve it over time.

The validator is also tested against the existing bitmap data we have
for the backup merging test as well as some of the existing broken
bitmap synthetic test cases.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:33 +01:00
Peter Krempa
d69470a18a virJSONValueNewArray: Use g_new0 to allocate and remove NULL checks from callers
Use the glib allocation function that never returns NULL and remove the
now dead-code checks from all callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:33 +01:00
Peter Krempa
8e94e29010 qemu: checkpoint: Track and relabel images for bitmap merging
Allow qemu access to modify backing files in case when we want to delete
a checkpoint.

This patch adds tracking of which images need to be relabelled when
calculating the transaction, the code to relabel them and rollback.

To verify that stuff works we also output the list of images to relabel
into the test case output files in qemublocktest.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
065e548ebf tests: qemublock: Add checkpoint deletion tests for some special cases
Use the synthetic test data to verify that the algorithm correctly picks
bitmaps to merge when the bitmap is changed along with the image itself.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
3c1c35bada tests: qemublock: Add checkpoint deletion test for deep backing chain
Add test cases for merging various pairs of bitmaps when snapshots were
created together with checkpoints.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
30bc426071 qemu: checkpoint: Introduce support for deleting checkpoints accross snapshots
Allow deleting of checkpoints when snapshots were created along. The
code tracks and modifies the checkpoint list so that backups can still
be taken with such a backing chain. This unfortunately requires to
rename few bitmaps (by copying and deleting them) in some cases.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
d7d97e87af tests: qemublock: Add synthetic snapshot+checkpoint test data
Add a faked qemu output which would simulate scenario where libvirt
would take a snapshot and checkpoint simultaneously. This is visible in
libvirt-2-format node where bitmap 'c' appears, but bitmap 'b' which is
active in the previous layer is not present.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
0cf33ab8f1 tests: qemublock: Add few more test cases for checkpoint deletion
Add all intermediate steps and deletion of the current checkpoint on a
flat (single-image) disk image.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Peter Krempa
25f7489972 tests: qemublock: Add test for checkpoint deletion bitmap merge
Add test infrastructure and a basic test for bitmap deletion.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-04 13:45:32 +01:00
Ján Tomko
0a125c7144 virBufferTrim: do not accept len
Always trim the full specified suffix.

All of the callers outside of tests were passing either
strlen or the actual length of the string.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2020-02-03 19:44:38 +01:00
Nikolay Shirokovskiy
dd94f36ffb qemu: check iotune params same for all disk in group
Currently it is possible to start a domain which have disks
in same iotune group and at the same time having different iotune
params. Both params set are passed to qemu in command line and the one
that is passed later down command line is get actually set.
Let's prohibit such configurations.

Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-01-29 11:46:51 +01:00
Peter Krempa
3f2d167d9c conf: Always format storage source auth and encryption under <source> for backing files
Historically there are two places where we format authentication and
encryption for a disk. The logich which formats it for backing files was
flawed though and didn't format it at all. This worked if the image
became a backing file through the means of a snapshot but not directly.

Force formatting of the source and encryption for any non-disk case to
fix the issue.

This caused problems in many places as we use the formatter to copy the
definition. Effectively any copy lost the secret definition.

https://bugzilla.redhat.com/show_bug.cgi?id=1789310
https://bugzilla.redhat.com/show_bug.cgi?id=1788898

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2020-01-13 12:53:58 +01:00
Ján Tomko
20071c9f99 Remove redundant usage of WITH_YAJL
As of commit 2a00ef6e71 which
was released in v5.2.0, we require YAJL to build the QEMU driver.

Remove the checks from code that requires the QEMU driver
or checks that also check for WITH_QEMU.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-13 14:57:32 +01:00
Peter Krempa
9851a0f390 tests: qemublock: Add tests for cross-snapshot incremental backups
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Peter Krempa
7dfd5d8d81 tests: qemublock: Add testing of bitmap merging for incremental backups
Add test code which will crawl a fake internal list of checkpoints and
generate the list of bitmaps for merging to gather the final bitmap for
the backup.

The initial tests cover the basic case of all bitmaps being present in
the top layer of the backing chain.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Peter Krempa
0b27b655b1 tests: qemublock: Add test case for detecting bitmaps as we create snapshots
Add test data gathered from a run of qemu after creating bitmaps and
snapshots together in various combinations.

The following sequence of commands was used to achieve the
configuration:

virsh checkpoint-create-as VM --name a
virsh snapshot-create-as VM --disk-only
virsh snapshot-create-as VM --disk-only
virsh checkpoint-create-as VM --name b
virsh checkpoint-create-as VM --name c
virsh snapshot-create-as VM --disk-only
virsh checkpoint-create-as VM --name d
virsh snapshot-create-as VM --disk-only
virsh checkpoint-create-as VM --name current

Note that VM was restarted after these operations to allow renumbering
of the bitmaps in a more human-readable way.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Peter Krempa
1326fd1ce6 tests: qemublocktest: Add a synthetic test case for bitmap detection
The real data gathered for the 'basic' test case don't exercise some
fields. Add a copy with a few values modified manually.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Peter Krempa
9aac9d5bda tests: qemublock: Add test for bitmap detection
Test the extraction of data about changed block tracking bitmaps. The
first test case adds a simple scenario of multiple bitmaps in one layer.

The test data will be also later reused for testing the code that
determines which bitmaps to merge for an incremental backup.

The sequence of bitmaps was created by the libvirt checkpoint API with
the following sequence of commands:

virsh checkpoint-create-as VM --name a
virsh checkpoint-create-as VM --name b
virsh checkpoint-create-as VM --name c
virsh checkpoint-create-as VM --name d
virsh checkpoint-create-as VM --name current

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-12-13 13:22:55 +01:00
Daniel P. Berrangé
99a949ffc4 conf: move seclabel validation into post-parse phase
Currently the disk and chardev seclabels are validated immediately at
the time their data is parsed. This forces the parser to fill in the
top level secmodel at time of parsing which is an undesirable thing.
This validation conceptually should be done in the post-parse phase
instead.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-09 10:15:16 +00:00
Michal Privoznik
4fa804c0c7 tests: Use g_strdup_printf() instead of virAsprintf()
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-11-12 16:15:59 +01:00
Peter Krempa
0967708b81 util: buffer: Remove virBufferCheckError
The function now does not return an error so we can drop it fully.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-10-24 19:35:34 +02:00
Peter Krempa
08e0ffe8f5 qemu: domain: Remove pointless return value in qemuDomainPrepareDiskSourceData
The function does not do anything that could fail. Remove the return
value.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-10-21 16:29:05 +02:00
Peter Krempa
aef87271be qemu: domain: Split out setup of virStorageSource from qemu driver config
qemuDomainPrepareDiskSourceData historically prepared everything but
we've split out the majority of the functionality so that it sets up
predominately only according to the configuration of the disk. There
was one leftover bit of setting the gluster debug level from the config.

Split this out into a separate function so that
qemuDomainPrepareDiskSourceData only prepares based on the disk.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
ACKed-by: Eric Blake <eblake@redhat.com>
2019-10-21 16:28:02 +02:00
Ján Tomko
29b1e859e3 tests: use g_strdup instead of VIR_STRDUP
Replace all occurrences of
  if (VIR_STRDUP(a, b) < 0)
     /* effectively dead code */
with:
  a = g_strdup(b);

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-21 12:51:59 +02:00
Ján Tomko
b6108a04ea Use g_steal_pointer instead of VIR_STEAL_PTR everywhere
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 15:59:42 +02:00
Ján Tomko
2b390b97b4 Use g_autoptr instead of VIR_AUTOUNREF
Now that all the types using VIR_AUTOUNREF have a cleanup func defined
to virObjectUnref, use g_autoptr instead of VIR_AUTOUNREF.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:44 +02:00
Ján Tomko
45678bd70a Use g_autoptr instead of VIR_AUTOPTR
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOPTR aliases to g_autoptr. Replace all of its use by the GLib
macro version.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Ján Tomko
1e2ae2e311 Use g_autofree instead of VIR_AUTOFREE
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOFREE is just an alias for g_autofree. Use the GLib macros
directly instead of our custom aliases.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Ján Tomko
2b2c67b401 virbuffer: use g_auto directly for virBuffer
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOCLEAN is just an alias for g_auto. Use the GLib macros
directly instead of our custom aliases.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Peter Krempa
a7643da82e qemu: block: Explicitly specify backingStore when creating format layer props
Pass in backing store explicitly to qemuBlockStorageSourceGetBlockdevProps
and fix the callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-09-06 08:12:21 +02:00
Peter Krempa
5dc567f80a tests: qemublock: Use bigger numbers as dummy capacity/physical
Actually test that the full range is available.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-09-04 09:13:49 +02:00
Michal Privoznik
e3511ee5ec tests: Always put '\n' at the end of VIR_TEST_VERBOSE
Similarly to the previous commit, VIR_TEST_VERBOSE should put
'\n' at the end of each call so that the output is not broken.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-27 15:56:17 +02:00
Peter Krempa
5265743daa util: storagefile: Modify arguments of virStorageSourceNewFromBackingAbsolue
Return the parsed storage source via an pointer in arguments and return
an integer from the function. Describe the semantics with a comment for
the function and adjust callers to the new semantics.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-26 13:49:17 +02:00
Peter Krempa
c1a1975e49 tests: qemublock: Add testing of 'blockdev-create' generators
Test the output against the schema and also against what we expect.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
2c4c347c4b qemu: block: Use 'auto-read-only' instead of 'read-only' for backing chain
To allow using -blockdev with blockjobs QEMU needs to reopen files in
read-write mode when modifying the backing chain. To achieve this we
need to use 'auto-read-only' for the backing files rather than the
normal 'read-only' property. That way qemu knows that the files need to
be reopened.

Note that the format drivers (e.g. qcow2) are still opened with the
read-only property enabled when being a member of the backing chain
since they are supposed to be immutable unless a block job is started.

QEMU v4.0 (since commit 23dece19da4) allows also dynamic behaviour for
auto-read-only which allows us to use sVirt as we only grant write
permissions to files when doing a blockjob.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
13f59e6f40 tests: qemublock: Add testing of pure disk source specification JSON generator
Add testing of the host specification part so that we can be sure that
no image/host specific data will be present.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Peter Krempa
423b4f6625 qemu: block: Allow skipping non-target related data when formating disk JSON
When formatting new qcow2 images we need to provide the backing store
string which should not contain any authentication or irrelevant data.

Add a flag for qemuBlockStorageSourceGetBackendProps which allows to
skip the irrelevant data.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-07-18 17:59:34 +02:00
Eric Blake
1ec3e39742 conf: Add parameter to virDomainDiskSourceFormat
Commits 4bc42986 and 218c81ea removed virDomainStorageSourceFormat on
the grounds that there were no external callers; however, the upcoming
backup code wants to output a <target> (push mode) or <scratch> (pull
mode) element that is in all other respects identical to a domain's
<source> element, where the previous virDomainStorageSourceFormat fit
the bill nicely. But rather than reverting the commits, it's easier to
just add an additional parameter for the element name to use, and
update all callers.

Signed-off-by: Eric Blake <eblake@redhat.com>
2019-05-06 18:05:17 -05:00
Andrea Bolognani
3fe020a038 tests: Drop dirname argument from testQemuGetLatestCapsForArch()
As evidenced by all existing callers, the only directory it makes
sense to use is TEST_QEMU_CAPS_PATH, so let's just bake that into
the function.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-04-17 08:30:55 +02:00
Peter Krempa
dfaf170df5 conf: Replace virDomainDiskSourceParse by virDomainStorageSourceParse
virDomainDiskSourceParse was now just a thin wrapper without any extra
value. Replace all usage of it by the function it calls and remove the
function.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-04-03 11:58:09 +02:00
Peter Krempa
5f6f803ca1 conf: Merge virDomainDiskSourceFormatInternal into virDomainDiskSourceFormat
Remove the wrapper and fix callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-04-03 11:58:09 +02:00
Peter Krempa
4885e9fdd9 tests: Refactor control flow in testBackingXMLjsonXML
Get rid of the 'cleanup' label.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-03-20 08:17:05 +01:00
Peter Krempa
5f13df4cf4 tests: Use full force of our VIR_AUTO* machinery in testBackingXMLjsonXML
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-03-20 08:17:05 +01:00
Peter Krempa
e1c01b2252 util: Remove the AUTOPTR func for virStorageSource
Since virStorageSource is now a subclass of virObject, we can use
VIR_AUTOUNREF instead.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-02-18 10:31:21 +01:00
Peter Krempa
dcda2bf4c1 util: Introduce function for allocating virStorageSource
Add virStorageSourceNew and refactor places allocating that structure to
use the helper.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-02-18 10:27:30 +01:00
John Ferlan
bf688a0067 util: Introduce VIR_DEFINE_AUTOPTR_FUNC for virStorageSource
Let's make use of the auto __cleanup capabilities cleaning up any
now unnecessary goto paths.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-02-12 13:19:26 -05:00
Peter Krempa
6e7e965dcd util: storage: Properly parse URIs with missing trailing slash
The URI parser used by libvirt does not populate uri->path if the
trailing slash is missing. The code virStorageSourceParseBackingURI
would then not populate src->path.

As only NBD network disks are allowed to have the 'name' field in the
XML defining the disk source omitted we'd generate an invalid XML which
we'd not parse again.

Fix it by populating src->path with an empty string if the uri is
lacking slash.

As pointed out above NBD is special in this case since we actually allow
it being NULL. The URI path is used as export name. Since an empty
export does not make sense the new approach clears the src->path if the
trailing slash is present but nothing else.

Add test cases now to cover all the various cases for NBD and non-NBD
uris as there was to time only 1 test abusing the quirk witout slash for
NBD and all other URIs contained the slash or in case of NBD also the
export name.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2018-10-12 16:41:48 +02:00
Erik Skultety
5165ff0971 src: More cleanup of some system headers already contained in internal.h
All of the ones being removed are pulled in by internal.h. The only
exception is sanlock which expects the application to include <stdint.h>
before sanlock's headers, because sanlock prototypes use fixed width
int, but they don't include stdint.h themselves, so we have to leave
that one in place.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 10:16:39 +02:00
Daniel P. Berrangé
dfd282f620 tests: skip qemublocktest if building without YAJL
If no JSON parser is available qemublocktest fails, so skip its execution.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-09-04 12:00:14 +01:00
Ján Tomko
d99a89592d Revert "Switch from yajl to Jansson"
This reverts commit 9cf38263d0.

Jansson cannot parse QEMU's quirky JSON.
Revert back to yajl.

https://bugzilla.redhat.com/show_bug.cgi?id=1614569

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-08-13 15:50:01 +02:00
Ján Tomko
6f99de3148 Revert "m4: Introduce STABLE_ORDERING_JANSSON"
This reverts commit 4dd6054000.

Jansson cannot parse QEMU's quirky JSON.
Revert back to yajl.

https://bugzilla.redhat.com/show_bug.cgi?id=1614569

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-08-13 15:50:00 +02:00
Ján Tomko
4dd6054000 m4: Introduce STABLE_ORDERING_JANSSON
Add a second check for Jansson >= 2.8, which includes
fixes to preserve ordering of object keys.

Use this constant to guard tests that depend on stable ordering.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-07-18 17:54:18 +02:00
Ján Tomko
9cf38263d0 Switch from yajl to Jansson
Yajl has not seen much activity upstream recently.
Switch to using Jansson >= 2.5.

All the platforms we target on https://libvirt.org/platforms.html
have a version >= 2.7 listed on the sites below:
https://repology.org/metapackage/jansson/versions
https://build.opensuse.org/package/show/devel:libraries:c_c++/libjansson

Additionally, Ubuntu 14.04 on Travis-CI has 2.5. Set the requirement
to 2.5 since we don't use anything from newer versions.

Implement virJSONValue{From,To}String using Jansson, delete the yajl
code (and the related virJSONParser structure) and report an error
if someone explicitly specifies --with-yajl.

Also adjust the test data to account for Jansson's different whitespace
usage for empty arrays and tune up the specfile to keep 'make rpm'
working when bisecting.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-07-18 17:54:18 +02:00
Peter Krempa
3875459235 tests: qemublock: Test NBD with TLS in the JSON generator
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-05 08:14:00 +02:00
Peter Krempa
f18771d41c qemu: block: Add support for 'pr-manager' in qemuBlockStorageSourceGetFileProps
To keep feature parity, we need to be able to format the PR manager
alias when using blockdev.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-05 08:13:59 +02:00
Peter Krempa
88860bb9d9 qemu: domain: Split validation and setup of the virStorageSource
Remove the call to the validating function from the function which sets
stuff up.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-05 08:13:58 +02:00
Peter Krempa
9b6a636d49 qemu: domain: don't loop through images in qemuDomainPrepareDiskSourceChain
Convert the function to just prepare data for the disk. Callers need to
do the looping since there's more to do than just copy the data around.

The code path in qemuDomainPrepareDiskSource doesn't need to loop over
the chain yet, since there currently is no chain at this point. This
will be addressed later in the blockdev series where we will setup much
more stuff.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-05 08:13:58 +02:00
Peter Krempa
b8240fe704 qemu: block: Don't nest storage layer properties into format layer
Reference the storage via node name rather than inlining it. This is
the approach that will be used with -blockdev/blockdev-add since it
allows more control and is more future proof.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-23 13:18:23 +02:00
Peter Krempa
55111f3508 tests: qemublock: Test handling of block devices
Make sure that 'host_device' is generated for type='block'.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:12:01 +02:00
Peter Krempa
ca916d7709 tests: qemublock: Test handling of all cache modes
The test cases would correspond to the following -drive command lines:

dir-fat-cache.xml:
-drive file=fat:/var/somefiles,if=none,id=drive-dummy,readonly=on,cache=directsync
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=off

file-backing_basic-cache-directsync.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=directsync
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=off

file-backing_basic-cache-none.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=none
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=on

file-backing_basic-cache-unsafe.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=unsafe
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=on

file-backing_basic-cache-writeback.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=writeback
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=on

file-backing_basic-cache-writethrough.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=writethrough
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=off

network-qcow2-backing-chain-cache-unsafe.xml:
-drive file=rbd:rbdpool/rbdimg:id=testuser-rbd:auth_supported=cephx\;none:
        mon_host=host1.example.com\;host2.example.com,
        file.password-secret=node-a-s-secalias,format=qcow2,
        if=none,id=drive-dummy,cache=directsync
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=off

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:09:30 +02:00
Peter Krempa
24db3d9993 tests: qemublock: Test handling of 'unmap' and 'detect-zeroes' options
The test cases would correspond to the following -drive command lines:

file-backing_basic-detect.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,detect-zeroes=on
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-backing_basic-unmap-detect.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,discard=unmap,detect-zeroes=unmap
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-backing_basic-unmap-ignore.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,discard=ignore,detect-zeroes=on
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-backing_basic-unmap.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,discard=unmap
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:05:31 +02:00
Peter Krempa
18458e8fd1 tests: qemublock: Add test combining authentication and encryption
iscsi and rbd support authentication of the connection. Combine it with
encryption of qcow2.

The top level disk image would generate the following '-drive' cmdline:

-drive file=rbd:rbdpool/rbdimg:id=testuser-rbd:auth_supported=cephx\;none:
            mon_host=host1.example.com\;host2.example.com,
            file.password-secret=node-a-s-secalias,encrypt.format=luks,
            encrypt.key-secret=node-b-f-encalias,format=qcow2,
            if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:03:46 +02:00
Peter Krempa
2c71edcf90 tests: qemublock: basic qcow2 tests
Add tests for backing chain handling, including a very long chain which
is fully specified in the XML and an unterminated chain.

The top level disk image would generate the following '-drive':

file-qcow2-backing-chain-encryption.xml:
-drive file=/var/lib/libvirt/images/a,encrypt.format=luks,
    encrypt.key-secret=node-b-f-encalias,format=qcow2,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-qcow2-backing-chain-noopts.xml:
-drive file=/var/lib/libvirt/images/rhel7.3.1507297895,format=qcow2,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-qcow2-backing-chain-unterminated.xml:
-drive file=/var/lib/libvirt/images/rhel7.3.1507297895,format=qcow2,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:01:09 +02:00
Peter Krempa
23a7f94893 tests: qemublock: Add test for raw luks disk format
Apart from adding test data add a function which sets up fake secrets
for the test.

The top level disk image would generate the following '-drive' cmdline:

-drive file=/path/luks.img,key-secret=test1-encalias,format=luks,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:58:51 +02:00
Peter Krempa
ac71f4e6d5 tests: qemublock: Add test cases for 'aio' options of 'file' storage
Test that the 'aio' option is applied correctly for the 'file' protocol
backend and across the backing chain.

The top level disk image would generate the following '-drive' cmdline:

file-backing_basic-aio_threads:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,aio=threads
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-raw-aio_native:
-drive file=/path/to/i.img,format=raw,if=none,id=drive-dummy,cache=none,aio=native
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=on

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:57:52 +02:00
Peter Krempa
7e441e2e2c tests: qemublock: Add test-case for the 'vvfat' driver in qemu
Test mapping of the 'FAT' disk format to 'vvfat' in qemu.

The top level disk image would generate the following '-drive' cmdline:

dir-fat-readonly.xml:
-drive file=fat:/var/somefiles,if=none,id=drive-dummy,readonly=on
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

dir-fat-floppy.xml
-drive file=fat:floppy:/var/somefiles,if=none,id=drive-dummy,readonly=on

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:55:16 +02:00
Peter Krempa
b54af513d6 tests: qemublock: Add tests for basic backing chain formats
Formats supporting backing chain such as qed, vmdk, don't have any other
parameters than the backing store and 'qcow' has only encryption params
which will be tested extra. Add this test case so they are covered since
any further test cases will mainly care about 'qcow2' and 'raw'.

The top level disk image would generate the following '-drive' cmdline:

-drive file=/var/lib/libvirt/images/a,format=qed,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:53:57 +02:00
Peter Krempa
c4f0e16f5a tests: qemublock: Add tests for all other format without special options
Similarly to the 'raw' case add tests for bochs, cloop, dmg, ploop, vdi
vhd, and vpc. Covering all supported non-backing formats.

Note that the JSON name for 'ploop' maps to 'parallels' and 'vhd' maps
to 'vhdx'.

Files added here would result in the followint configs:

file-bochs-noopts.xml:
-drive file=/path/to/i.img,format=bochs,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-cloop-noopts.xml:
-drive file=/path/to/i.img,format=cloop,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-dmg-noopts.xml:
-drive file=/path/to/i.img,format=dmg,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-ploop-noopts.xml:
-drive file=/path/to/i.img,format=ploop,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-vdi-noopts.xml:
-drive file=/path/to/i.img,format=vdi,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-vhd-noopts.xml:
-drive file=/path/to/i.img,format=vhd,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-vpc-noopts.xml:
-drive file=/path/to/i.img,format=vpc,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:53:12 +02:00
Peter Krempa
5ce01b15e1 tests: qemublock: Add basic 'raw' file test
Test the JSON props generator with a very simple 'raw' image with no
other options. The node-names for the image are 31 bytes long so that we
validate our node name detector.

The top level disk image would generate the following '-drive' cmdline:

-drive file=/var/lib/libvirt/images/i.img,format=raw,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:51:45 +02:00
Peter Krempa
8f60653de3 tests: qemublock: Add testing of blockdev JSON property generator
Add a test infrastructure that will allow testing the JSON object
generator used for generating data to use with blockdev-add.

The resulting disk including the backing chain is validated to conform
to the QAPI schema and the expected output files.

The first test cases make sure that libvirt will not allow nodenames
exceeding 31 chars.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:48:10 +02:00
Peter Krempa
21498975d1 tests: qemublock: Rename variables in anticipation of new tests
New tests will add new data structures so rename the 'data' structure.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:38:59 +02:00
Peter Krempa
d94aa38ffb qemu: block: Propagate 'legacy' parameter when formatting disk backing
The gluster protocol in qemu uses two styles, one of which is legacy and
not covered by the QAPI schema.

To allow using of the new style in the blockdev-add code, add a
parameter for qemuBlockStorageSourceGetBackendProps which will switch
between the two modes.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:31:48 +02:00
Peter Krempa
ea520f6b67 util: json: Fix freeing of objects appended to virJSONValue
It was not possible to determine whether virJSONValueObjectAddVArgs and
the functions using it would consume a virJSONValue or not when used
with the 'a' or 'A' modifier depending on when the loop failed.

Fix this by passing in a pointer to the pointer so that it can be
cleared once it's successfully consumed and the callers don't have to
second-guess leaving a chance of leaking or double freeing the value
depending on the ordering.

Fix all callers to pass a double pointer too.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2018-04-03 13:34:33 +02:00
Peter Krempa
aed3d038a6 conf: Add infrastructure for disk source private data XML
VM drivers may need to store additional private data to the status XML
so that it can be restored after libvirtd restart. Since not everything
is needed add a callback infrastructure, where VM drivers can add only
stuff they need.

Note that the private data is formatted as a <privateData> sub-element
of the <disk> or <backingStore> <source> sub-element. This is done since
storing it out of band (in the VM private data) would require a complex
matching process to allow to put the data into correct place.
2017-12-14 10:24:36 +01:00
Peter Krempa
96be3e72e8 tests: Add testing of storage backend JSON props formatter
Add a new test program called 'qemublocktest' to test the block layer
related stuff and test storage source to JSON generator by comparing it
to the JSON parser.
2017-11-07 16:18:14 +01:00