Commit Graph

128 Commits

Author SHA1 Message Date
Erik Skultety
5165ff0971 src: More cleanup of some system headers already contained in internal.h
All of the ones being removed are pulled in by internal.h. The only
exception is sanlock which expects the application to include <stdint.h>
before sanlock's headers, because sanlock prototypes use fixed width
int, but they don't include stdint.h themselves, so we have to leave
that one in place.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 10:16:39 +02:00
Daniel P. Berrangé
dfd282f620 tests: skip qemublocktest if building without YAJL
If no JSON parser is available qemublocktest fails, so skip its execution.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-09-04 12:00:14 +01:00
Ján Tomko
d99a89592d Revert "Switch from yajl to Jansson"
This reverts commit 9cf38263d0.

Jansson cannot parse QEMU's quirky JSON.
Revert back to yajl.

https://bugzilla.redhat.com/show_bug.cgi?id=1614569

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-08-13 15:50:01 +02:00
Ján Tomko
6f99de3148 Revert "m4: Introduce STABLE_ORDERING_JANSSON"
This reverts commit 4dd6054000.

Jansson cannot parse QEMU's quirky JSON.
Revert back to yajl.

https://bugzilla.redhat.com/show_bug.cgi?id=1614569

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-08-13 15:50:00 +02:00
Ján Tomko
4dd6054000 m4: Introduce STABLE_ORDERING_JANSSON
Add a second check for Jansson >= 2.8, which includes
fixes to preserve ordering of object keys.

Use this constant to guard tests that depend on stable ordering.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-07-18 17:54:18 +02:00
Ján Tomko
9cf38263d0 Switch from yajl to Jansson
Yajl has not seen much activity upstream recently.
Switch to using Jansson >= 2.5.

All the platforms we target on https://libvirt.org/platforms.html
have a version >= 2.7 listed on the sites below:
https://repology.org/metapackage/jansson/versions
https://build.opensuse.org/package/show/devel:libraries:c_c++/libjansson

Additionally, Ubuntu 14.04 on Travis-CI has 2.5. Set the requirement
to 2.5 since we don't use anything from newer versions.

Implement virJSONValue{From,To}String using Jansson, delete the yajl
code (and the related virJSONParser structure) and report an error
if someone explicitly specifies --with-yajl.

Also adjust the test data to account for Jansson's different whitespace
usage for empty arrays and tune up the specfile to keep 'make rpm'
working when bisecting.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-07-18 17:54:18 +02:00
Peter Krempa
3875459235 tests: qemublock: Test NBD with TLS in the JSON generator
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-05 08:14:00 +02:00
Peter Krempa
f18771d41c qemu: block: Add support for 'pr-manager' in qemuBlockStorageSourceGetFileProps
To keep feature parity, we need to be able to format the PR manager
alias when using blockdev.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-05 08:13:59 +02:00
Peter Krempa
88860bb9d9 qemu: domain: Split validation and setup of the virStorageSource
Remove the call to the validating function from the function which sets
stuff up.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-05 08:13:58 +02:00
Peter Krempa
9b6a636d49 qemu: domain: don't loop through images in qemuDomainPrepareDiskSourceChain
Convert the function to just prepare data for the disk. Callers need to
do the looping since there's more to do than just copy the data around.

The code path in qemuDomainPrepareDiskSource doesn't need to loop over
the chain yet, since there currently is no chain at this point. This
will be addressed later in the blockdev series where we will setup much
more stuff.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-05 08:13:58 +02:00
Peter Krempa
b8240fe704 qemu: block: Don't nest storage layer properties into format layer
Reference the storage via node name rather than inlining it. This is
the approach that will be used with -blockdev/blockdev-add since it
allows more control and is more future proof.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-23 13:18:23 +02:00
Peter Krempa
55111f3508 tests: qemublock: Test handling of block devices
Make sure that 'host_device' is generated for type='block'.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:12:01 +02:00
Peter Krempa
ca916d7709 tests: qemublock: Test handling of all cache modes
The test cases would correspond to the following -drive command lines:

dir-fat-cache.xml:
-drive file=fat:/var/somefiles,if=none,id=drive-dummy,readonly=on,cache=directsync
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=off

file-backing_basic-cache-directsync.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=directsync
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=off

file-backing_basic-cache-none.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=none
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=on

file-backing_basic-cache-unsafe.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=unsafe
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=on

file-backing_basic-cache-writeback.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=writeback
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=on

file-backing_basic-cache-writethrough.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow2,if=none,id=drive-dummy,cache=writethrough
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=off

network-qcow2-backing-chain-cache-unsafe.xml:
-drive file=rbd:rbdpool/rbdimg:id=testuser-rbd:auth_supported=cephx\;none:
        mon_host=host1.example.com\;host2.example.com,
        file.password-secret=node-a-s-secalias,format=qcow2,
        if=none,id=drive-dummy,cache=directsync
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=off

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:09:30 +02:00
Peter Krempa
24db3d9993 tests: qemublock: Test handling of 'unmap' and 'detect-zeroes' options
The test cases would correspond to the following -drive command lines:

file-backing_basic-detect.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,detect-zeroes=on
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-backing_basic-unmap-detect.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,discard=unmap,detect-zeroes=unmap
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-backing_basic-unmap-ignore.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,discard=ignore,detect-zeroes=on
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-backing_basic-unmap.xml:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,discard=unmap
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:05:31 +02:00
Peter Krempa
18458e8fd1 tests: qemublock: Add test combining authentication and encryption
iscsi and rbd support authentication of the connection. Combine it with
encryption of qcow2.

The top level disk image would generate the following '-drive' cmdline:

-drive file=rbd:rbdpool/rbdimg:id=testuser-rbd:auth_supported=cephx\;none:
            mon_host=host1.example.com\;host2.example.com,
            file.password-secret=node-a-s-secalias,encrypt.format=luks,
            encrypt.key-secret=node-b-f-encalias,format=qcow2,
            if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:03:46 +02:00
Peter Krempa
2c71edcf90 tests: qemublock: basic qcow2 tests
Add tests for backing chain handling, including a very long chain which
is fully specified in the XML and an unterminated chain.

The top level disk image would generate the following '-drive':

file-qcow2-backing-chain-encryption.xml:
-drive file=/var/lib/libvirt/images/a,encrypt.format=luks,
    encrypt.key-secret=node-b-f-encalias,format=qcow2,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-qcow2-backing-chain-noopts.xml:
-drive file=/var/lib/libvirt/images/rhel7.3.1507297895,format=qcow2,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-qcow2-backing-chain-unterminated.xml:
-drive file=/var/lib/libvirt/images/rhel7.3.1507297895,format=qcow2,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 16:01:09 +02:00
Peter Krempa
23a7f94893 tests: qemublock: Add test for raw luks disk format
Apart from adding test data add a function which sets up fake secrets
for the test.

The top level disk image would generate the following '-drive' cmdline:

-drive file=/path/luks.img,key-secret=test1-encalias,format=luks,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:58:51 +02:00
Peter Krempa
ac71f4e6d5 tests: qemublock: Add test cases for 'aio' options of 'file' storage
Test that the 'aio' option is applied correctly for the 'file' protocol
backend and across the backing chain.

The top level disk image would generate the following '-drive' cmdline:

file-backing_basic-aio_threads:
-drive file=/var/lib/libvirt/images/a,format=qcow,if=none,id=drive-dummy,aio=threads
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-raw-aio_native:
-drive file=/path/to/i.img,format=raw,if=none,id=drive-dummy,cache=none,aio=native
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy,write-cache=on

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:57:52 +02:00
Peter Krempa
7e441e2e2c tests: qemublock: Add test-case for the 'vvfat' driver in qemu
Test mapping of the 'FAT' disk format to 'vvfat' in qemu.

The top level disk image would generate the following '-drive' cmdline:

dir-fat-readonly.xml:
-drive file=fat:/var/somefiles,if=none,id=drive-dummy,readonly=on
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

dir-fat-floppy.xml
-drive file=fat:floppy:/var/somefiles,if=none,id=drive-dummy,readonly=on

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:55:16 +02:00
Peter Krempa
b54af513d6 tests: qemublock: Add tests for basic backing chain formats
Formats supporting backing chain such as qed, vmdk, don't have any other
parameters than the backing store and 'qcow' has only encryption params
which will be tested extra. Add this test case so they are covered since
any further test cases will mainly care about 'qcow2' and 'raw'.

The top level disk image would generate the following '-drive' cmdline:

-drive file=/var/lib/libvirt/images/a,format=qed,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:53:57 +02:00
Peter Krempa
c4f0e16f5a tests: qemublock: Add tests for all other format without special options
Similarly to the 'raw' case add tests for bochs, cloop, dmg, ploop, vdi
vhd, and vpc. Covering all supported non-backing formats.

Note that the JSON name for 'ploop' maps to 'parallels' and 'vhd' maps
to 'vhdx'.

Files added here would result in the followint configs:

file-bochs-noopts.xml:
-drive file=/path/to/i.img,format=bochs,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-cloop-noopts.xml:
-drive file=/path/to/i.img,format=cloop,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-dmg-noopts.xml:
-drive file=/path/to/i.img,format=dmg,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-ploop-noopts.xml:
-drive file=/path/to/i.img,format=ploop,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-vdi-noopts.xml:
-drive file=/path/to/i.img,format=vdi,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-vhd-noopts.xml:
-drive file=/path/to/i.img,format=vhd,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

file-vpc-noopts.xml:
-drive file=/path/to/i.img,format=vpc,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:53:12 +02:00
Peter Krempa
5ce01b15e1 tests: qemublock: Add basic 'raw' file test
Test the JSON props generator with a very simple 'raw' image with no
other options. The node-names for the image are 31 bytes long so that we
validate our node name detector.

The top level disk image would generate the following '-drive' cmdline:

-drive file=/var/lib/libvirt/images/i.img,format=raw,if=none,id=drive-dummy
-device virtio-blk-pci,scsi=off,drive=drive-dummy,id=dummy

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:51:45 +02:00
Peter Krempa
8f60653de3 tests: qemublock: Add testing of blockdev JSON property generator
Add a test infrastructure that will allow testing the JSON object
generator used for generating data to use with blockdev-add.

The resulting disk including the backing chain is validated to conform
to the QAPI schema and the expected output files.

The first test cases make sure that libvirt will not allow nodenames
exceeding 31 chars.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:48:10 +02:00
Peter Krempa
21498975d1 tests: qemublock: Rename variables in anticipation of new tests
New tests will add new data structures so rename the 'data' structure.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:38:59 +02:00
Peter Krempa
d94aa38ffb qemu: block: Propagate 'legacy' parameter when formatting disk backing
The gluster protocol in qemu uses two styles, one of which is legacy and
not covered by the QAPI schema.

To allow using of the new style in the blockdev-add code, add a
parameter for qemuBlockStorageSourceGetBackendProps which will switch
between the two modes.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 15:31:48 +02:00
Peter Krempa
ea520f6b67 util: json: Fix freeing of objects appended to virJSONValue
It was not possible to determine whether virJSONValueObjectAddVArgs and
the functions using it would consume a virJSONValue or not when used
with the 'a' or 'A' modifier depending on when the loop failed.

Fix this by passing in a pointer to the pointer so that it can be
cleared once it's successfully consumed and the callers don't have to
second-guess leaving a chance of leaking or double freeing the value
depending on the ordering.

Fix all callers to pass a double pointer too.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2018-04-03 13:34:33 +02:00
Peter Krempa
aed3d038a6 conf: Add infrastructure for disk source private data XML
VM drivers may need to store additional private data to the status XML
so that it can be restored after libvirtd restart. Since not everything
is needed add a callback infrastructure, where VM drivers can add only
stuff they need.

Note that the private data is formatted as a <privateData> sub-element
of the <disk> or <backingStore> <source> sub-element. This is done since
storing it out of band (in the VM private data) would require a complex
matching process to allow to put the data into correct place.
2017-12-14 10:24:36 +01:00
Peter Krempa
96be3e72e8 tests: Add testing of storage backend JSON props formatter
Add a new test program called 'qemublocktest' to test the block layer
related stuff and test storage source to JSON generator by comparing it
to the JSON parser.
2017-11-07 16:18:14 +01:00