Introduce a set of helpers to call blockdev-reopen in certain scenarios
Libvirt will use the QMP command to turn certain members of the backing
chain read-write for bitmap manipulation and we'll also want to use it
to replace/install the backing chain of a qcow2 format node.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
We currently don't model the 'ssh' protocol properties properly and
since it seems impossible for now (agent path passed via environment
variable). To allow libguestfs to work as it used in pre-blockdev era we
must carry the properties over to the command line. For this instance we
just store it internally and format it back.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Pass the alias of the secret object holding the cookie data as
'cookie-secret' to qemu.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Implement both commandline support and hotplug by adding the http cookie
handling to 'qemuBlockStorageSourceAttachData' handling functions for
it.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Allow disabling of SSL certificate validation for HTTPS and FTPS drives
in qemu.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Use the 'flat' flag for 'query-named-block-nodes' if qemu supports
QEMU_CAPS_QMP_QUERY_NAMED_BLOCK_NODES_FLAT in qemuBlockGetNamedNodeData.
We don't need the data so plumb in whether qemu supports the
'flat' output.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Implement support for the slice of type 'storage' which allows to set
the offset and size which modifies where qemu should look for the start
of the format container inside the image.
Since slicing is done using the 'raw' driver we need to add another
layer into the blockdev tree if there's any non-raw image format driver
used to access the data.
This patch adds the blockdev integration and setup of the image data so
that we can use the slices for any backing image.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
When creating overlay images e.g. for snapshots or when merging
snapshots we often specify the backing store string to use. Make the
formatter aware of backing chain entries which have a <slice>
configured so that we record it properly. Otherwise such images
would not work without the XML (when detecting the backing chain).
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Specifically creating such images via libvirt during blockjobs would
be much more hassle than it's worth. Just forbid them for now.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
If we have a 'format' type slice for a raw driver we can directly format
the values.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Modify qemuMonitorBlockdevAdd so that it takes a double pointer for the
@props argument so that it's cleared inside the call. This allows
writing cleaner callers.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Few switch cases returned failure but didn't report an error. For a
situation when the backingStore type='volume' was not translated the
following error would occur:
$ virsh start VM
error: Failed to start domain VM
error: An error occurred, but the cause is unknown
After this patch:
$ virsh start VM
error: Failed to start domain VM
error: internal error: storage source pool 'tmp' volume 'pull3.qcow2' is not translated
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Add a function calculating which bitmaps to copy to the mirror during
a block-copy operation.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Add a validator which checks that a bitmap spanning multiple backing
chain members doesn't look broken. The current rules are that no
intermediate birmaps are missing (unfortunately it's hard to know
whether the topmost or bottommost bitmap is missing) and none of the
components is inconsistent.
We can obviously improve it over time.
The validator is also tested against the existing bitmap data we have
for the backup merging test as well as some of the existing broken
bitmap synthetic test cases.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Use the glib allocation function that never returns NULL and remove the
now dead-code checks from all callers.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Create a wrapper for qemuBlockGetNamedNodeData named
qemuBlockGetNamedNodeData. The purpose of the wrapper is to integrate
the monitor handling functionality and in the future possible
qemuCaps-based flags.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
libvirt treats 'luks' images as raw+encryption. The logic in
qemuBlockStorageSourceCreateFormat skipped the creation if the requested
image was raw but didn't take into account the encryption.
This manifested itself e.g. when attempting to do a virsh blockcopy with
the following XML:
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/tmp/enccpy'>
<encryption format='luks'>
<secret type='passphrase' uuid='0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f'/>
</encryption>
</source>
</disk>
Where qemu would report the following error:
unable to execute QEMU command 'blockdev-add': Volume is not in LUKS format
rather than actually formatting the image first.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The waiting loop used QEMU_ASYNC_JOB_NONE rather than 'asyncJob' passed
from the caller.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Now, that we have everything prepared, we can generate command
line for NVMe disks.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
To simplify implementation, some restrictions are added. For
instance, an NVMe disk can't go to any bus but virtio and has to
be type of 'disk' and can't have startupPolicy set.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
This function looks up a named bitmap for a virStorageSource in the data
returned from query-named-block-nodes.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
This previous commit introduced a simpler free callback for
hash data with only 1 arg, the value to free:
commit 49288fac96
Author: Peter Krempa <pkrempa@redhat.com>
Date: Wed Oct 9 15:26:37 2019 +0200
util: hash: Add possibility to use simpler data free function in virHash
It missed two functions in the hash table code which need
to call the alternate data free function, virHashRemoveEntry
and virHashRemoveSet.
After the previous patch though, there is no code that
makes functional use of the 2nd key arg in the data
free function. There is merely one log message that can
be dropped.
We can thus purge the current virHashDataFree callback
entirely, and rename virHashDataFreeSimple to replace
it.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
There are four places where we remove image XATTRs and in all of
them we have the same for() loop with the same body. Move it into
a separate function because I'm about to introduce fifth place
where the same needs to be done.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Calling the monitor was convenient for the implementation in
qemuDomainBlockCopyCommon, but causes the snapshot code to call
query-named-block-nodes for every disk.
Fix this by removing the monitor call from
qemuBlockStorageSourceCreateDetectSize so that the data can be reused in
loops.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
ACKed-by: Eric Blake <eblake@redhat.com>
Replace all occurrences of
if (VIR_STRDUP(a, b) < 0)
/* effectively dead code */
with:
a = g_strdup(b);
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Replace all the occurrences of
ignore_value(VIR_STRDUP(a, b));
with
a = g_strdup(b);
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Since commit 44e7f02915
util: rewrite auto cleanup macros to use glib's equivalent
VIR_AUTOPTR aliases to g_autoptr. Replace all of its use by the GLib
macro version.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Since commit 44e7f02915
util: rewrite auto cleanup macros to use glib's equivalent
VIR_AUTOPTR aliases to g_autoptr. Replace all uses of VIR_DEFINE_AUTOPTR_FUNC
with G_DEFINE_AUTOPTR_CLEANUP_FUNC in preparation for replacing the
rest.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Since commit 44e7f02915
util: rewrite auto cleanup macros to use glib's equivalent
VIR_AUTOFREE is just an alias for g_autofree. Use the GLib macros
directly instead of our custom aliases.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Use G_GNUC_UNUSED from GLib instead of ATTRIBUTE_UNUSED.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Use the new generator residing in the monitor code rather than directly
using qemuMonitorJSONTransactionAdd.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Pass backing store as an argument rather than extracting it locally and
fix the callers.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Pass in backing store explicitly to qemuBlockStorageSourceGetBlockdevProps
and fix the callers.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Move all bits of the formatting of the 'backing' attribute to a single
condition and make it use a single extracted copy of the backing store.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The 'u' modifier creates an unsigned int JSON attribute but the disk size
and capacity fields are unsigned long long. If the size of the created
image would be more than 4GiB we'd overflow and create sub-4G image.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
QEMU finally exposes an interface which allows us to instruct it to
format or create arbitrary images. This is required for blockdev
integration of block copy and snapshots as we need to pre-format images
prior to use with blockdev-add.
This path introduces job handling and also helpers for formatting and
attaching a whole image described by a virStorageSource.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
In case when the backing store can be represented with something
simpler such as a URI we can use it rather than falling back to the
json: pseudo-protocol.
In cases when it's not worth it (e.g. with the old ugly NBD or RBD
strings) let's switch to json.
The function is exported as we'll need it when overwriting the ugly
strings qemu would come up with during blockjobs.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
For the modern use cases we are going to use 'blockdev-snapshot' instead
of 'blockdev-snapshot-sync'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
QEMU allows us to create storage on certain network protocols which
allow image creation through their API. Wire up the generator for using
it with libvirt as well as for local files.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
'blockdev-add' allows us to use qemu to format images to our desired
format. This patch implements helpers which convert a
virStorageSourcePtr into JSON objects describing the required
configuration.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
To allow using -blockdev with blockjobs QEMU needs to reopen files in
read-write mode when modifying the backing chain. To achieve this we
need to use 'auto-read-only' for the backing files rather than the
normal 'read-only' property. That way qemu knows that the files need to
be reopened.
Note that the format drivers (e.g. qcow2) are still opened with the
read-only property enabled when being a member of the backing chain
since they are supposed to be immutable unless a block job is started.
QEMU v4.0 (since commit 23dece19da4) allows also dynamic behaviour for
auto-read-only which allows us to use sVirt as we only grant write
permissions to files when doing a blockjob.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
To allow reusing the formatters in the code for creating JSON properties
for 'blockdev-create' we need to create everything except the 'driver'
attribute.
Use the new helper virJSONValueObjectPrependString to put the driver at
the same place so that we don't change any output.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
When formatting new qcow2 images we need to provide the backing store
string which should not contain any authentication or irrelevant data.
Add a flag for qemuBlockStorageSourceGetBackendProps which allows to
skip the irrelevant data.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Split up the addition of a storage source into the following sub-steps:
1) storage access dependencies (TLS transport, persistent reservation)
2) storage acccess node (file/gluster/nbd...)
3) format driver dependencies (encryption secret)
4) format driver node (qcow2, raw, ...)
The functions split out will be later reused when implementing support
for 'blockdev-create' as we'll need the dependencies plugged in first,
then blockdev-create will be called and after that successfully finishes
blockdev-add will be added.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
These are meant to replace the ad-hoc helpers qemuHotplugDiskSourceAttach...
and the open-coded version in qemu_command.c for use in command line
generation.
The functions for preparing for attach of chains unfortunately need to
be in qemu_command.c as they use function defined by that file and
inclusion hierarchy.
In this patch new functions are introduced and subsequent patches then
refactor individual parts to use them.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Move it to qemu_block.c and call it qemuBlockStorageSourceDetachPrepare.
It will be reused in other parts as well.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Demonstrate how VIR_RETURN_PTR is used by refactoring qemu_block.c
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This is a locally used helper struct but we can make use of automatic
freeing for it.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The copy on read functionality is done using a separate layer in the
backing chain. Add function to generate properties for it.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Turns out that 'query-nodes' is not what we want and the
'query-blockstats' command was in fact buggy. Revert the new field since
it's not needed.
This reverts commit 50edca1331.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The 'query-blockstats' command does not return statistics for the
explicitly named nodes unless the new argument is specified. Add
infrastrucuture that will allow us to use the new approach if desired.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
With 'transaction' support we don't need to keep around the multipurpose
code which would create the snapshot if 'transaction' is not supported.
To simplify this add a new helper that just wraps the arguments for
'blockdev-snapshot-sync' operation in 'transaction' and use it instead
of qemuBlockSnapshotAddLegacy.
Additionally this allows to format the arguments prior to creating the
file for simpler cleanup.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
To allow using -blockdev with RBD we need to support the recently added
RBD authentication.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Introduce a new setup function for all the related configuration and
move the setup and attachment of the PR code.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Create a new "Prepare" function and move the drive add code into the new
helpers. This will eventually allow to simplify and unify the attaching
code for use with blockdev at the same time as providing compatibility
with older qemus.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
To keep feature parity, we need to be able to format the PR manager
alias when using blockdev.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
These helpers add infrastructure which simplifies adding and rolling
back virStorageSources to a running qemu instance. Using of the helper
structure and separate functions allows for a much cleaner code in the
section dealing with the monitor.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Reference the storage via node name rather than inlining it. This is
the approach that will be used with -blockdev/blockdev-add since it
allows more control and is more future proof.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
When using blockdev-add and friends, libvirt will need to create also
properties for the qcow2/raw/... format handler in qemu. This patch adds
the infrastructure and implements all formats known to libvirt including
all properties which are expressed at the format level in qemu.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Enabling discard for the storage node allows the format drivers to
discard snapshots and other things, while configuration of the format
layer actually decides whether to actually discard data on request from
the host.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
This will be required when doing blockdev-add to conform with the
approach qemu choses to create the disks.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
When used directly with blockdev-add/-blockdev the cache mode will need
to be specified directly for every image rather than just for the disk
itself. This implements the backing options 'direct' and 'no-flush'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
qemu declares node-name as a 32 byte buffer and silently truncates
anything longer than that. This is unacceptable for libvirt, so we need
to make sure that we won't ever supply a node-name exceeding 31 chars.
Add a function which will do the validation and use it to validate
storage-protocol node names.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
The gluster protocol in qemu uses two styles, one of which is legacy and
not covered by the QAPI schema.
To allow using of the new style in the blockdev-add code, add a
parameter for qemuBlockStorageSourceGetBackendProps which will switch
between the two modes.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Block storage should actually be passed to qemu via 'host_device' or
'host_cdrom' according to the device type. There were no users of this
behaviour so we thankfully can change it.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Handle VIR_STORAGE_TYPE_DIR in qemuBlockStorageSourceGetBackendProps so
that a 'vvfat' driver is used, which emulates a FAT filesystem
containing the folders.
qemu requires us to add it as a storage layer, since a 'raw' layer is
usually put on top of it.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
'file' backend in qemu supports few more options than the current
implementation. Extract it so that changes don't pollute the code.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
It was not possible to determine whether virJSONValueObjectAddVArgs and
the functions using it would consume a virJSONValue or not when used
with the 'a' or 'A' modifier depending on when the loop failed.
Fix this by passing in a pointer to the pointer so that it can be
cleared once it's successfully consumed and the callers don't have to
second-guess leaving a chance of leaking or double freeing the value
depending on the ordering.
Fix all callers to pass a double pointer too.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
The documentation for the JSON/qapi type 'UnixSocketAddress' states that
the unix socket path field is named 'path'. Unfortunately qemu uses
'socket' in case of the gluster driver (despite documented otherwise).
Add logic which will format the correct fields while keeping support of
the old spelling.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1544325
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
The array indexes are formatted if the JSON->commandline translator is
translating an array type. It does not at all depend on this function.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Storage source format backing a shared device (e.g. running a cluster
filesystem) needs to support the sharing so that metadata are not
corrupted. Add a central function for checking this.
If creation of the main JSON object containing the storage portion of a
virStorageSource would fail but we'd allocate the server structure we'd
leak it. Found by coverity.
Return NULL right away in qemuBlockStorageSourceGetBackendProps when an
invalid storage source is presented so that virJSONValueObjectAdd isn't
called with a NULL argument.
Found by coverity.
Libvirt historically stores storage source path including the volume as
one string in the XML, but that is not really flexible enough when
dealing with the fields in the code. Previously we'd store the slash
separating the two as part of the image name. This was fine for gluster
but it's not necessary and does not scale well when converting other
protocols.
Don't store the slash as part of the path. The resulting change from
absolute to relative path within the gluster driver should be okay,
as the root directory is the default when accessing gluster.
