Commit Graph

31 Commits

Author SHA1 Message Date
Abhiram Tilak
f432114d9c storage: Upgrade default qcow2 verion to 1.1
Change the default to modern qcow2 as it's supported by all qemu
versions supported by libvirt and in fact 'qemu-img' already defaults to
the new format for a long time.

Some Unittests require changes to pass, now that version 1.1 is default.
Unittests like `qcow2-1.1.argv` may not be relevant anymore, but this
patch doesn't affect them.

Closes: https://gitlab.com/libvirt/libvirt/-/issues/602
Signed-off-by: Abhiram Tilak <atp.exp@gmail.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2024-03-12 10:34:43 +01:00
Peter Krempa
66566e84b8 storage: Introduce 'extended_l2' feature for storage volume
QCOW2 images now support 'extended_l2' which splits the default clusters
into 32 subcluster allocation units. This allows the allocation units to
be smaller without increasing the size of L2 table too much and thus also
the cache requirements for holding the full L2 table in memory.

Unfortunately it's incompatible with qemu versions older than 5.2 thus
can't be used as default.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-12-21 13:23:09 +01:00
Pavel Hrdina
3e1d2c93a3 storage: add support for QCOW2 cluster_size option
The default value hard-coded in QEMU (64KiB) is not always the ideal.
Having a possibility to set the cluster_size by user may in specific
use-cases improve performance for QCOW2 images.

QEMU internally has some limits, the value has to be between 512B and
2048KiB and must by power of two, except when the image has Extended L2
Entries the minimal value has to be 16KiB.

Since qemu-img ensures the value is correct and the limit is not always
the same libvirt will not duplicate any of these checks as the error
message from qemu-img is good enough:

    Cluster size must be a power of two between 512 and 2048k

Resolves: https://gitlab.com/libvirt/libvirt/-/issues/154

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2021-05-21 14:00:43 +02:00
Peter Krempa
19d7f60133 storagevolxml2argvtest: Use internal wrapping of command line arguments
virCommandToString has the possibility to return an already wrapped
string with better format than what we get from the test wrapper script.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2021-04-12 15:55:10 +02:00
Peter Krempa
4079144836 storagevolxml2argvdata: Rewrap all output files
Use scripts/test-wrap-argv.py to rewrap the output files so that any
further changes don't introduce churn since we are rewrapping the output
automatically now.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2021-02-16 12:25:30 +01:00
Daniel P. Berrangé
ecfc4094d8 storage: add support for qcow2 LUKS encryption
The storage driver was wired up to support creating raw volumes in LUKS
format, but was never adapted to support LUKS-in-qcow2. This is trivial
as it merely requires the encryption properties to be prefixed with
the "encrypt." prefix, and "encrypt.format=luks" when creating the
volume.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-09-18 11:22:28 +01:00
Daniel P. Berrangé
093ee8b3e2 tests: remove redundant LUKS volume data files
The two removed files have exactly the same config as other LUKS volume
data files, simply with different file names. Consolidate down to just
two LUKS volume data files as that's all that we need for the test
coverage.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-09-18 11:22:26 +01:00
Daniel P. Berrangé
912c6b22fc util: require command args to be non-NULL
The virCommand APIs do not expect to be given a NULL value for an arg
name or value. Such a mistake can lead to execution of the wrong
command, as the NULL may prematurely terminate the list of args.
Detect this and report suitable error messages.

This identified a flaw in the storage test which was passing a NULL
instead of the volume path. This flaw was then validated by an incorrect
set of qemu-img args as expected data.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-12-17 14:43:33 +00:00
John Ferlan
b975afc725 storage: Allow inputvol to be encrypted
https://bugzilla.redhat.com/show_bug.cgi?id=1613737

When processing the inputvol for encryption, we need to handle
the case where the inputvol is encrypted. This then allows for
the encrypted inputvol to be used either for an output encrypted
volume or an output volume of some XML provided type.

Add tests to show the various conversion options when either input
or output is encrypted. This includes when both are encrypted.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-12 07:24:04 -04:00
John Ferlan
8041471858 storage: Allow for inputvol to have any format for encryption
Commit 39cef12a9 altered/fixed the inputvol processing to create
a multistep process when using an inputvol to create an encrypted
output volume; however, it unnecessarily assumed/restricted the
inputvol to be of 'raw' format only.

Modify the processing code to allow the inputvol format to be checked
and used in order to create the encrypted volume.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-12 07:24:04 -04:00
John Ferlan
39cef12a95 storage: Add support for using inputvol for encryption
Starting with QEMU 2.9, encryption convert processing requires
a multi-step process in order to generate an encrypted image from
some non encrypted raw image.

Processing requires to first create an encrypted image using the
sizing parameters from the input source and second to use the
--image-opts, -n, and --target-image-opts options along with inline
driver options to describe the input and output files, generating
two commands such as:

  $ qemu-img create -f luks \
      --object secret,id=demo.img_encrypt0,file=/path/to/secretFile \
      -o key-secret=demo.img_encrypt0 \
      demo.img 500K
  Formatting 'demo.img', fmt=luks size=512000 key-secret=demo.img_encrypt0
  $ qemu-img convert --image-opts -n --target-image-opts \
      --object secret,id=demo.img_encrypt0,file=/path/to/secretFile \
      driver=raw,file.filename=sparse.img \
      driver=luks,file.filename=demo.img,key-secret=demo.img_encrypt0
  $

This patch handles the convert processing by running the processing
in a do..while loop essentially reusing the existing create logic and
arguments to create the target vol from the inputvol and then converting
the inputvol using new arguments.

This then allows the following virsh command to work properly:

  virsh vol-create-from default encrypt1-luks.xml data.img --inputpool default

where encrypt1-luks.xml would provided the path and secret for
the new image, while data.img would be the source image.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
John Ferlan
a02d879858 tests: Remove qcow2 encryption from storagevol tests
We're about to disallow creation of a qcow2 encrypted storage
volume, so let's remove the qcow encryption element from the
tests which are testing whether other format='qcow2' related
features work properly.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
John Ferlan
23e9aa7297 tests: Add luks creation examples to storagevolxml2argvtest
Add the storagevolxml2xmltest "luks" and "luks-cipher" tests
to the storagevolxml2argvtest.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
Ján Tomko
74abb43aca tests: assume FMT_COMPAT for qemu-img tests
No point in testing outdated command lines.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 10:21:59 +02:00
Ján Tomko
3590ebdfd2 tests: delete most qemu-img test cases assuming FMT_OPTIONS
We have two leftover "capabilites" for qemu-img:
QEMU_IMG_BACKING_FORMAT_OPTIONS
QEMU_IMG_BACKING_FORMAT_OPTIONS_COMPAT

The former says we are able to specify the backing format via -o
(which has been the case for a long time now) and the second one
says we can use -o compat to specify the qcow2 version.

Since we require QEMU 1.5.0, we can always assume -o compat,
which was introduced in QEMU 1.1.

Drop the test cases using FMT_OPTIONS which have a FMT_COMPAT
counterpart to prepare for deprecating FMT_OPTIONS (and these flags)
completely.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 10:21:39 +02:00
Wim ten Have
c9ec7088c7 storage: extend preallocation flags support for qemu-img
This patch adds support to qcow2 formatted filesystem object storage by
instructing qemu-img to build them with preallocation=falloc whenever the
XML described storage <allocation> matches its <capacity>.  For all other
cases the filesystem stored objects are built with preallocation=metadata.

Signed-off-by: Wim ten Have <wim.ten.have@oracle.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-04-11 10:15:58 +02:00
Cole Robinson
0de1066fb9 tests: Add storagevol tests for format=iso
Demonstrates the bug fix in commit 0e5db76262
2017-03-08 13:22:33 -05:00
John Ferlan
f6a92f8e20 storage: Adjust qemu-img switches check
Since we support QEMU 0.12 and later, checking for support of specific flags
added prior to that isn't necessary.

Thus start with the base of having the "-o options" available for the
qemu-img create option and then determine whether we have the compat
option for qcow2 files (which would be necessary up through qemu 2.0
where the default changes to compat 0.11).

Adjust test to no long check for NONE and FLAG options as well was removing
results of tests that would use that option.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-21 14:31:19 -04:00
Chris J Arges
be6c35e4ac tests: add vol-qcow2-zerocapacity test to storagevolxml2argvtest
Add a testcase for the previous change to ensure zero capacity volumes can be
defined without a backing store.

Signed-off-by: Chris J Arges <chris.j.arges@canonical.com>
2015-07-24 11:23:45 -04:00
Ján Tomko
155ca616eb Allow creating volumes with a backing store but no capacity
The tool creating the image can get the capacity from the backing
storage. Just refresh the volume afterwards.

https://bugzilla.redhat.com/show_bug.cgi?id=958510
2015-03-02 08:07:11 +01:00
Ján Tomko
e3f1d2a820 Allow cloning volumes with no capacity specified
In virStorageVolCreateXML, add VIR_VOL_XML_PARSE_NO_CAPACITY
to the call parsing the XML of the new volume to make the capacity
optional.

If the capacity is omitted, use the capacity of the old volume.
We already do that for values that are less than the original
volume capacity.
2015-03-02 08:07:11 +01:00
Ján Tomko
be3cbecd0e Fix a crash when cloning a volume with no backing store
Introduced by commit 15213d1, not yet released.
2014-08-01 15:42:28 +02:00
Chunyan Liu
0b0c641b66 add nocow test case
Add file in storagevolxml2xmlin and storagevolxml2xmlout, let
storagevolxml2xmltest and storagevolschematest cover 'nocow'.
Add test case to storagevolxml2argvtest to cover 'nocow'.

Signed-off-by: Chunyan Liu <cyliu@suse.com>
2014-07-16 13:35:26 +02:00
Ján Tomko
bab2eda6ad Always specify qcow2 compat level on qemu-img command line
qemu-img is going to switch the default for QCOW2
to QCOW2v3 (compat=1.1)

Extend the probing for qemu-img command line options to check
if -o compat is supported. If the volume definition specifies
the qcow2 format but no compat level and -o compat is supported,
specify -o compat=0.10 to create a QCOW2v2 image.

https://bugzilla.redhat.com/show_bug.cgi?id=997977
2013-08-27 12:45:58 +02:00
Ján Tomko
601d465254 Add inputpool to storagevolxml2argvtest
This allows testing the command line for cloning file-based
volumes into logical volumes and vice versa.
2013-07-25 13:27:41 +02:00
Ján Tomko
3a75fca289 Move volume XMLs out of storagevolxml2argvdata
Reuse the XML files in storagevolxml2xmlin.

(This requires changing a few backing files to /dev/null,
since virStorageBackendCreateQemuImgCmd checks for its
presence)
2013-07-25 13:26:41 +02:00
Ján Tomko
823778829e Move pool XML out of storagevolxml2argvdata
Reuse the pool definition from storagepoolxml2xmlin.
2013-07-25 13:26:40 +02:00
Eric Blake
684c90bfbc tests: split long lines
Long lines are harder to read and harder to diff; in fact, if lines get
too long (> 1000 bytes), it starts causing issues where git send-email
refuses to send patches for the file.  I've cleaned up the tests
directory in the past (see commits bd6c46f, 3b750d1), but new long
lines have been introduced in the meantime.

Why 90 instead of 80? Because there were too many tests on the fringe
edge, and I didn't want to edit that many files.

Add a syntax check to prevent future long lines.

* cfg.mk (sc_prohibit_long_lines): New rule.
* tests/qemuxml2argvdata/qemuxml2argv-*.args: Split lines of any
file with content longer than 90 columns.
* tests/storagevolxml2argvdata/*.argv: Likewise.

Signed-off-by: Eric Blake <eblake@redhat.com>
2013-07-18 10:08:04 -06:00
Ján Tomko
6298f74d9a storage: add support for creating qcow2 images with extensions
Add -o compat= and -o lazy_refcounts options for qemu-img.
2013-06-21 13:25:30 +02:00
Ján Tomko
81738cd014 storage: move qemu-img options before positional arguments
Modify the expected output of storagevolxml2argv tests as well.
2013-02-24 18:11:40 +01:00
Ján Tomko
56e1b4c30f storage: add test for qemu-img command line generation 2013-02-24 18:11:40 +01:00