Commit Graph

406 Commits

Author SHA1 Message Date
Michal Privoznik
d3a7479c0b Use more of VIR_XPATH_NODE_AUTORESTORE
This is convenience macro, use it more. This commit was generated
using the following spatch:

  @@
  symbol node;
  identifier old;
  identifier ctxt;
  type xmlNodePtr;
  @@
  - xmlNodePtr old;
  + VIR_XPATH_NODE_AUTORESTORE(ctxt);
    ...
  - old = ctxt->node;
    ... when != old
  - ctxt->node = old;

  @@
  symbol node;
  identifier old;
  identifier ctxt;
  type xmlNodePtr;
  @@
  - xmlNodePtr old = ctxt->node;
  + VIR_XPATH_NODE_AUTORESTORE(ctxt);
    ... when != old
  - ctxt->node = old;

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-06-03 21:06:02 +02:00
Peter Krempa
6bde2a1e20 conf: Sanitize handling of <auth> and <encryption> placement for disks
Modern way to store <auth> and <encryption> of a <disk> is under
<source>. This was added to mirror how <backingStore> handles these and
in fact they are relevant to the source rather than to any other part of
the disk. Historically we allowed them to be directly under <disk> and
we need to keep compatibility.

This wasn't a problem until introduction of -blockdev in qemu using of
<auth> or <encryption> plainly wouldn't work with backing chains.

Now that it works in backing chains and can be moved back and forth
using snapshots/block-commit we need to ensure that the original
placement is properly kept even if the source changes.

To achieve the above semantics we need to store the preferred placement
with the disk definition rather than the storage source definitions and
also ensure that the modern way is chosen when the VM started with
<source/encryption> only in the backing store.

https://bugzilla.redhat.com/show_bug.cgi?id=1822878

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-05-12 06:55:00 +02:00
Peter Krempa
606fb3979a util: qcow2GetExtensions: Remove support for 'data file' extension
The implementation was never finished in libvirt. Remove it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-04-27 10:31:40 +02:00
Peter Krempa
38757cb142 util: Remove 'externalDataStoreRaw' field from virStorageSource
It's not used for anything so we don't need to extract it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-04-27 10:31:40 +02:00
Peter Krempa
94d1cb25a0 util: Remove 'externalDataStore' field from virStorageSource
It's not used for anything so we don't need to extract it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-04-27 10:31:40 +02:00
Peter Krempa
f8d6b319a6 virStorageSourceParseNBDColonString: Rewrite to match what qemu does
Our implementation wasn't quite able to parse everything that qemu does.
This patch rewrites the parser to a code that semantically resembles the
combination of 'nbd_parse_filename' and 'inet_parse' methods in qemu to
be able to parse the strings in an equivalent manner.

The only thing that libvirt doesn't do is to check the lengths of
various components in the nbd string in places where qemu uses constant
size buffers.

The test cases validate that some of the corner cases involving colons
are parsed properly.

https://bugzilla.redhat.com/show_bug.cgi?id=1826652

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-04-27 08:05:49 +02:00
Peter Krempa
544ef82d05 virStorageSourceParseBackingURI: Preserve query string of URI for http(s)
For http/https URIs we need to preserve the query part as it may be
important to refer to the image.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-30 16:30:34 +02:00
Peter Krempa
5636812472 conf: Add support for http(s) query strings
Add a new attribute for holding the query part for http(s) disks.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-30 16:30:34 +02:00
Peter Krempa
b9166baebe virStorageSourceNetCookieValidate: Accept quoted cookie value
The quotes are forbidden only inside the value, but the value itself may
be enclosed in quotes. Fix the RNG schema and validator and add a test
case.

https://bugzilla.redhat.com/show_bug.cgi?id=1804750

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-27 15:46:52 +01:00
Peter Krempa
1b84dd190c storage: Parse 'nvme' disk source properties from json:{} pseudo-uri
Our code allows snapshots of NVMe based disks which means we create
overlay file with a 'json:{}' pseudo-uri refering to the NVME device.
Our parser code doesn't handle them though. Add the parser and test it
via the XML->json->XML round-trip and reference data.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:48 +01:00
Peter Krempa
5a70f1048f storage: Implement backing store support for "fat:" prefix
qemublocktest showed that we don't add the "fat:" prefix for directory
storage when formatting the backing store string. While it's unlikely to
be used it's simple enough to actually implement the support rather than
trying to forbid it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:47 +01:00
Peter Krempa
5793b8baa7 conf: rename 'namespace' property of struct _virStorageSourceNVMeDef
While 'namespace' is not a reserved word in C, it is in C++. Our
compilers are happy with it but syntax-hilighting in some editors
hilights is as a keyword. Rename it to prevent confusion.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-24 14:17:47 +01:00
Peter Krempa
02f909b8a6 virSecretLookupDefCopy: Remove return value
The function always returns succes so there's no need for a return
value.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-03-20 09:47:16 +01:00
Peter Krempa
d6db013c6e qemu: Pass through arguments of 'ssh' block driver used by libguestfs
We currently don't model the 'ssh' protocol properties properly and
since it seems impossible for now (agent path passed via environment
variable). To allow libguestfs to work as it used in pre-blockdev era we
must carry the properties over to the command line. For this instance we
just store it internally and format it back.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
5179cc6b08 virStorageSourceParseBackingJSONUri: Handle undocumented value 'off' for sslverify
libguestfs abuses a quirk of qemu's parser to accept also other variants
of the 'sslverify' field which would be valid on the command line but
are not documented in the QMP schema.

If we encounter the 'off' string instead of an boolean handle it rather
than erroring out to continue support of pre-blockdev configurations.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
77194db01c virstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout'
Add support for parsing the recently added fields from backing file
pseudo-protocol strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
63fd461773 conf: Add support for setting timeout and readahead size for network disks
Some disk backends support configuring the readahead buffer or timeout
for requests. Add the knobs to the XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
3b076391be conf: Add support for cookies for HTTP based disks
Add possibility to specify one or more cookies for http based disks.
This patch adds the config parser, storage and validation of the
cookies.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
25481e25b1 conf: Add support for modifying ssl validation for https/ftps disks
To allow turning off verification of SSL cerificates add a new element
<ssl> to the disk source XML which will allow configuring the validation
process using the 'verify' attribute.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa
ae9e6c2a2b virStorageFileGetMetadataRecurse: Allow format probing under special circumstances
Allow format probing to work around lazy clients which did not specify
their format in the overlay. Format probing will be allowed only, if we
are able to probe the image, the probing result was successful and the
probed image does not have any backing or data file.

This relaxes the restrictions which were imposed in commit 3615e8b39b
in cases when we know that the image probing will not result in security
issues or data corruption.

We perform the image format detection and in the case that we were able
to probe the format and the format does not specify a backing store (or
doesn't support backing store) we can use this format.

With pre-blockdev configurations this will restore the previous
behaviour for the images mentioned above as qemu would probe the format
anyways. It also improves error reporting compared to the old state as
we now report that the backing chain will be broken in case when there
is a backing file.

In blockdev configurations this ensures that libvirt will not cause data
corruption by ending the chain prematurely without notifying the user,
but still allows the old semantics when the users forgot to specify the
format.

Users thus don't have to re-invent when image format detection is safe
to do.

The price for this is that libvirt will need to keep the image format
detector still current and working or replace it by invocation of
qemu-img.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-25 21:14:27 +01:00
Daniel P. Berrangé
6a38acb3c0 src: add virutil.h to more source files for geteuid() compat
The virutil.h header defines a geteuid() macro for Windows platforms.
This fixes a few missed cases from:

  commit b11e8cccdd
  Author: Ján Tomko <jtomko@redhat.com>
  Date:   Sun Feb 16 23:09:15 2020 +0100

    Remove virutil.h from all header files

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-25 10:02:15 +00:00
Ján Tomko
f331a9ef64 Remove virutil.h where possible
Historically, this file was a dump for most of our helper
functions and needed almost everywhere.
With the introduction of virfile.h and virstring.h,
and more importantly, virenum.h and the introduction
of GLib, that is no longer true.

Remove its include from C files that don't even use it.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-24 23:15:49 +01:00
Peter Krempa
c95656c995 virStorageFileGetMetadataFromFD: Remove unused 'backingFormat' argument
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa
57df35aead virStorageFileGetMetadataFromBuf: Remove 'backingFormat' argument
None of the callers actually use it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa
264b79c63a virStorageSourceNewFromBacking: Also transfer the format
When we create the new virStorageSource from the definitions stored in
the parent we should also use the 'backingStoreRawFormat' field to
populate the format.

Callers which use virStorageSourceNewFromBacking are also fixed to stop
setting the format manually.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa
62539c5f7d util: storage: Store backing store format in virStorageSource
We store the backing file string in the structure so we should also
store the format so that callers can be simplified.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa
35d1f5bd14 virStorageSourceUpdateCapacity: Drop 'probe' argument
Both callers pass false. Since we frown upon format probing, remove the
unused possibility to do the probing.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa
a570dc6767 virStorageFileGetMetadataRecurse: Remove 'cleanup' label
There's nothing to clean up. Make it obvious what is returned.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:09 +01:00
Peter Krempa
01adad0932 virStorageFileGetMetadataRecurse: Extract storage access
Extract the code that directly deals with storage. This allows further
simplification and clarification of virStorageFileGetMetadataRecurse.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:09 +01:00
Peter Krempa
e3960f4b6d virStorageFileGetMetadataRecurse: Use virHashHasEntry instead of fake pointers
Replacing virHashLookup by virHashHasEntry allows us to use NULL as the
payload of the hash table rather than putting a fake '1' pointer into
the table.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:09 +01:00
Peter Krempa
157b8722cb virStorageFileGetMetadataRecurse: Expect NULL src->path
The path can be NULL e.g. for NBD disks. Use NULLSTR to prevent use of
NULL in %s.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:08 +01:00
Peter Krempa
b347e5c7dd virStorageFileGetMetadataRecurse: Shuffle around assignment of backing chain depth
Move the assignment to a place where we know that the backing store is
present rather than having to check in the cleanup section.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:08 +01:00
Peter Krempa
84df98f29e virStorageFileGetMetadataRecurse: Remove impossible error report
We call virStorageFileSupportsBackingChainTraversal which already checks
that the 'storageFileRead' callback is non-NULL, which in turn means
that virStorageFileRead will not return -2.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:08 +01:00
Peter Krempa
181fccc2ed util: storagefile: Drop image format probing by file suffix
Probing by file suffix was meant to be a last resort if probing by
contents fails or is not supported. For most formats we never specified
any suffix. There's a few formats implementing both magic bytes and
suffix and finally DMG which had only suffix probing. Since suffix
probing is nowhere reliable and only one format depends on in which has a
comment that qemu doesn't do the probing either drop the whole
infrastructure.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:08 +01:00
Peter Krempa
e8a819e87f virStorageSourceParseBackingJSONRaw: Parse 'offset' and 'size' attributes
If the parsed 'raw' format JSON string has 'offset' or 'size' attributes
parse them as the format slice.

https://bugzilla.redhat.com/show_bug.cgi?id=1791788

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa
6efa046165 util: virstoragefile: Add data structure for storing storage source slices
Introduce virStorageSourceSlice which will store the 'offset' and 'size'
of a virStorageSource and declare it as 'sliceStorage' and 'sliceFormat'
attributes of a virStorageSource.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa
fd70f1b4d3 virStorageSourceParseBackingJSON: Prevent arbitrary nesting with format drivers
Since we parse attributes for 'raw' which is a format driver and thus
has nested 'file' structure we must prevent that this isn't nested
arbitrarily.

Add a flag for the function which allows parsing of 'format' type
drivers only on the first pass.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
f8e097570e virStorageSourceParseBackingJSON: Allow 'json:' pseudo URIs without 'file' wrapper
There are two possibilities:
1) json:{"file":{"driver":...}}
2) json:{"driver":...}

Our code didn't work properly with the second one as it was expecting
the 'file' wrapper. Conditionalize the removal to only the situation
when the top level doesn't have "driver".

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
7e13ff8dc0 virStorageSourceJSONDriverParser: annotate 'format' drivers
The parser was originally designed only for protocol parsers. Since
we already have 'raw' format driver in the list we'll need to be able
to parse it too. In later patches this will be used to prevent parsing
nested format drivers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
aadb34be34 virStorageSourceParseBackingJSON: Move deflattening of json: URIs out of recursion
Originally virStorageSourceParseBackingJSON didn't recurse, but when
the 'raw' driver support was added we need to parse it's information
which contains nested 'file' object.

Since the deflattening helper recurses already there's no need to call
it again. Move it one level up to the entry point.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
4a6bc568cd virStorageSourceParseBackingJSON: Pass around original backing file string
There are a few error messages which might want to report the original
backing store string. Pass it around rather than trying to re-generate
it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
5f2fa393f7 util: storagefile: Properly set transport type when parsing NBD strings
When parsing legacy NBD backing file strings such as
'nbd:unix:/tmp/sock:exportname=/' we'd fail to set the transport to
VIR_STORAGE_NET_HOST_TRANS_UNIX. This started to be a problem once we
actually started to generate config of the backing store on the command
line with -blockdev as the JSON code would try to format it as TCP and
fail with:

 internal error: argument key 'host' must not have null value

Set the type properly and add a test.

This bug was found by the libguestfs test suite in:

https://bugzilla.redhat.com/show_bug.cgi?id=1791614

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reported-by: Ming Xie <mxie@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
2020-01-17 12:34:10 +01:00
Daniel P. Berrangé
fa434739a0 src: replace verify(expr) with G_STATIC_ASSERT(expr)
G_STATIC_ASSERT() is a drop-in functional equivalent of
the GNULIB verify() macro.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-17 10:02:01 +00:00
Peter Krempa
30867fb098 util: storage: Link to knowledge base when reporting missing image backing format
Mention the knowledge base article which has tips how to fix the backing
chain to work with current libvirt.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-10 12:49:03 +01:00
Daniel Henrique Barboza
a3645a4747 util: remove unneeded labels
Remove unneeded, easy to remove goto labels (cleanup|error|done|...).

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-01-07 16:40:41 +01:00
Daniel P. Berrangé
bf7d2a26a3 src: replace mdir_name() with g_path_get_dirname()
Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-03 15:42:13 +00:00
Peter Krempa
3615e8b39b util: storage: Don't treat files with missing backing store format as 'raw'
Assuming that the backing image format is raw is wrong when doing image
detection:

1) In -drive mode qemu will still probe the image format of the backing
   image. This means it will try to open a backing file of the image
   which will fail if a more advanced security model is in use.

2) In blockdev mode the image will be opened as raw actually which is
   wrong since it might be qcow. Not opening the backing images will
   also end up in the guest seeing corrupted data.

Rather than attempt to solve various corner cases when us assuming the
storage file being raw and actually being right forbid startup when the
guest image doesn't have the format specified in the metadata.

https://bugzilla.redhat.com/show_bug.cgi?id=1588373

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-18 09:36:48 +01:00
Michal Privoznik
a88eef7c48 virstoragefile: Introduce virStorageSourceChainHasNVMe
This function will return true if there's a storage source of
type VIR_STORAGE_TYPE_NVME, or false otherwise.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
ACKed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-17 10:04:43 +01:00
Michal Privoznik
8cd7196974 conf: Format and parse NVMe type disk
To simplify implementation, some restrictions are added. For
instance, an NVMe disk can't go to any bus but virtio and has to
be type of 'disk' and can't have startupPolicy set.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2019-12-17 10:04:43 +01:00
Ján Tomko
93de3025b4 Remove the rest of VIR_STRNDUP
Replace all the uses passing a single parameter as the length.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-12 14:24:35 +01:00