Commit Graph

208 Commits

Author SHA1 Message Date
Peter Krempa
2775aada16 tests: virstorage: Add tests for NBD URI style syntax over UNIX
Add few test cases for nbd+unix style URIs with few corner cases.

The NBD URI syntax is documented at
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-01-17 12:34:10 +01:00
Peter Krempa
5f2fa393f7 util: storagefile: Properly set transport type when parsing NBD strings
When parsing legacy NBD backing file strings such as
'nbd:unix:/tmp/sock:exportname=/' we'd fail to set the transport to
VIR_STORAGE_NET_HOST_TRANS_UNIX. This started to be a problem once we
actually started to generate config of the backing store on the command
line with -blockdev as the JSON code would try to format it as TCP and
fail with:

 internal error: argument key 'host' must not have null value

Set the type properly and add a test.

This bug was found by the libguestfs test suite in:

https://bugzilla.redhat.com/show_bug.cgi?id=1791614

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reported-by: Ming Xie <mxie@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Richard W.M. Jones <rjones@redhat.com>
2020-01-17 12:34:10 +01:00
Peter Krempa
3f2d167d9c conf: Always format storage source auth and encryption under <source> for backing files
Historically there are two places where we format authentication and
encryption for a disk. The logich which formats it for backing files was
flawed though and didn't format it at all. This worked if the image
became a backing file through the means of a snapshot but not directly.

Force formatting of the source and encryption for any non-disk case to
fix the issue.

This caused problems in many places as we use the formatter to copy the
definition. Effectively any copy lost the secret definition.

https://bugzilla.redhat.com/show_bug.cgi?id=1789310
https://bugzilla.redhat.com/show_bug.cgi?id=1788898

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2020-01-13 12:53:58 +01:00
Daniel P. Berrangé
8812163124 src: remove unused imports of dirname.h
A few places were importing dirname.h without actually using it.

Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-03 15:42:13 +00:00
Peter Krempa
3615e8b39b util: storage: Don't treat files with missing backing store format as 'raw'
Assuming that the backing image format is raw is wrong when doing image
detection:

1) In -drive mode qemu will still probe the image format of the backing
   image. This means it will try to open a backing file of the image
   which will fail if a more advanced security model is in use.

2) In blockdev mode the image will be opened as raw actually which is
   wrong since it might be qcow. Not opening the backing images will
   also end up in the guest seeing corrupted data.

Rather than attempt to solve various corner cases when us assuming the
storage file being raw and actually being right forbid startup when the
guest image doesn't have the format specified in the metadata.

https://bugzilla.redhat.com/show_bug.cgi?id=1588373

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-18 09:36:48 +01:00
Peter Krempa
a649369480 tests: storage: Remove unused test modes
EXP_WARN and ALLOW_PROBE flags for the testStorageChain cases are no
longer used so we can remove them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-18 09:36:48 +01:00
Peter Krempa
7e582fe995 tests: storage: Use strict version of virStorageFileGetMetadata
Pass in 'true' as '@report_broken' of virStorageFileGetMetadata to make
it fail in the tests. The most important code paths (when starting the
VM) expect this function to fail rather than silently return partial
data. Switch the test to exercise this more important code path.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-12-18 09:36:48 +01:00
Michal Privoznik
4fa804c0c7 tests: Use g_strdup_printf() instead of virAsprintf()
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-11-12 16:15:59 +01:00
Michal Privoznik
3b4df5d350 Drop needless ret variable
In few places we have the following code pattern:

  int ret;
  ... /* @ret is not accessed here */
  ret = f(...);
  return ret;

This pattern can be written less verbose:

  ...
  return f(...);

This patch was generated with following coccinelle spatch:

  @@
  type T;
  constant C;
  expression f;
  identifier ret;
  @@
  -T ret = C;
   ... when != ret
  -ret = f;
  -return ret;
  +return f;

Afterwards I needed to fix a few places, e.g. comment in
virDomainNetIPParseXML() was removed too because coccinelle
thinks it refers to @ret while in fact it doesn't. Also in few
places it replaced @ret declaration with a few spaces instead of
removing the line. But nothing terribly wrong.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2019-10-24 08:10:37 +02:00
Ján Tomko
29b1e859e3 tests: use g_strdup instead of VIR_STRDUP
Replace all occurrences of
  if (VIR_STRDUP(a, b) < 0)
     /* effectively dead code */
with:
  a = g_strdup(b);

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-21 12:51:59 +02:00
Ján Tomko
b6108a04ea Use g_steal_pointer instead of VIR_STEAL_PTR everywhere
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 15:59:42 +02:00
Ján Tomko
2b390b97b4 Use g_autoptr instead of VIR_AUTOUNREF
Now that all the types using VIR_AUTOUNREF have a cleanup func defined
to virObjectUnref, use g_autoptr instead of VIR_AUTOUNREF.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:44 +02:00
Ján Tomko
45678bd70a Use g_autoptr instead of VIR_AUTOPTR
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOPTR aliases to g_autoptr. Replace all of its use by the GLib
macro version.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Ján Tomko
1e2ae2e311 Use g_autofree instead of VIR_AUTOFREE
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOFREE is just an alias for g_autofree. Use the GLib macros
directly instead of our custom aliases.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Ján Tomko
2b2c67b401 virbuffer: use g_auto directly for virBuffer
Since commit 44e7f02915
    util: rewrite auto cleanup macros to use glib's equivalent

VIR_AUTOCLEAN is just an alias for g_auto. Use the GLib macros
directly instead of our custom aliases.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-16 12:06:43 +02:00
Ján Tomko
67e72053c1 Use G_N_ELEMENTS instead of ARRAY_CARDINALITY
Prefer the GLib version of the macro.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 16:14:19 +02:00
Ján Tomko
0d94f02455 tests: use G_GNUC_UNUSED
Use G_GNUC_UNUSED from GLib instead of ATTRIBUTE_UNUSED.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2019-10-15 11:25:25 +02:00
Peter Krempa
6ff9241058 util: storagefile: Flag backing store strings with authentication
Using inline authentication for storage volumes will not work properly
as libvirt requires use of the secret driver for the auth data and
thus would not be able to represent the passwords stored in the backing
store string.

Make sure that the backing store parsers return 1 which is a sign for
the caller to not use the file in certain cases.

The test data include iscsi via a json pseudo-protocol string and URIs
with the userinfo part being present.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-26 13:49:17 +02:00
Peter Krempa
e1189ae5f3 tests: virstorage: Allow testing return value of virStorageSourceNewFromBackingAbsolute
Modify testBackingParse to allow testing other return values of the
backing store string parser.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-26 13:49:17 +02:00
Peter Krempa
5265743daa util: storagefile: Modify arguments of virStorageSourceNewFromBackingAbsolue
Return the parsed storage source via an pointer in arguments and return
an integer from the function. Describe the semantics with a comment for
the function and adjust callers to the new semantics.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-26 13:49:17 +02:00
Peter Krempa
ffabad7572 tests: storage: Refactor cleanup in testBackingParse
Automatically clean the temporary buffer and get rid of the cleanup
label.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-26 13:49:16 +02:00
Peter Krempa
96f0a17ead util: storage: Fix parsing of 'exportname' from legacy NBD strings
If the nbd export name contains a colon, our parser would not parse it
properly as we split the string by colons. Modify the code to look up
the exportname and copy any trailing characters as the export name is
supposed to be at the end of the string.

https://bugzilla.redhat.com/show_bug.cgi?id=1733044

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2019-08-16 13:16:40 +02:00
Eric Blake
1ec3e39742 conf: Add parameter to virDomainDiskSourceFormat
Commits 4bc42986 and 218c81ea removed virDomainStorageSourceFormat on
the grounds that there were no external callers; however, the upcoming
backup code wants to output a <target> (push mode) or <scratch> (pull
mode) element that is in all other respects identical to a domain's
<source> element, where the previous virDomainStorageSourceFormat fit
the bill nicely. But rather than reverting the commits, it's easier to
just add an additional parameter for the element name to use, and
update all callers.

Signed-off-by: Eric Blake <eblake@redhat.com>
2019-05-06 18:05:17 -05:00
Peter Krempa
f785318187 Revert "Include unistd.h directly by files using it"
This reverts commit a5e1602090.

Getting rid of unistd.h from our headers will require more work than
just fixing the broken mingw build. Revert it until I have a more
complete proposal.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2019-04-10 12:26:32 +02:00
Peter Krempa
a5e1602090 Include unistd.h directly by files using it
util/virutil.h bogously included unistd.h. Drop it and replace it by
including it directly where needed.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-04-10 09:12:04 +02:00
Peter Krempa
5f6f803ca1 conf: Merge virDomainDiskSourceFormatInternal into virDomainDiskSourceFormat
Remove the wrapper and fix callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-04-03 11:58:09 +02:00
Andrea Bolognani
35d76db2b7 tests: Don't use canonical paths in virstoragetest
The layout of my home directory is somewhat peculiar: I store
all git repositories in ~/src/upstream, but since I spend
almost all of my time hacking on libvirt, I also have a
convenience symlink ~/src/libvirt -> ~/src/upstream/libvirt
that I use to access that specific git repository.

The above setup has served me well for years; however, ever
since commit ca1471622d dropped our own custom definitions
for abs_{,top_}{src,build}dir and started using the ones
provided by autotools, virstoragetest has started reliably
failing with errors such as

   2) Storage backing chain 2 ...
  Offset 0
  Expect [chain member: 0
  path:/home/abologna/src/upstream/libvirt/tests/virstoragedata/raw
  backingStoreRaw: <null>
  capacity: 0
  encryption: 0
  relPath:<null>
  type:1
  format:1
  protocol:none
  hostname:<null>
  ]
  Actual [chain member: 0
  path:/home/abologna/src/libvirt/tests/virstoragedata/raw
  backingStoreRaw: <null>
  capacity: 0
  encryption: 0
  relPath:<null>
  type:1
  format:1
  protocol:none
  hostname:<null>
  ]
                              ... FAILED

Using abolute paths instead of canonical ones in the tests makes
the problem go away.

Note that all tests that are specifically designed to test path
canonicalization via TEST_PATH_CANONICALIZE() were passing even
before this patch and are not touched by it.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
2019-04-03 09:46:51 +02:00
Peter Krempa
44601a0e96 util: Replace virStorageSourceFree with virObjectUnref
Now that virStorageSource is a subclass of virObject we can use
virObjectUnref and remove virStorageSourceFree which was a thin wrapper.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-02-18 10:31:21 +01:00
Peter Krempa
e1c01b2252 util: Remove the AUTOPTR func for virStorageSource
Since virStorageSource is now a subclass of virObject, we can use
VIR_AUTOUNREF instead.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-02-18 10:31:21 +01:00
Peter Krempa
dcda2bf4c1 util: Introduce function for allocating virStorageSource
Add virStorageSourceNew and refactor places allocating that structure to
use the helper.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-02-18 10:27:30 +01:00
John Ferlan
bf688a0067 util: Introduce VIR_DEFINE_AUTOPTR_FUNC for virStorageSource
Let's make use of the auto __cleanup capabilities cleaning up any
now unnecessary goto paths.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-02-12 13:19:26 -05:00
John Ferlan
9916f2a3c8 tests: Rename variable in testStorageFileGetMetadata
To prepare for subsequent change to use VIR_AUTOPTR logic rename
the @ret to @def.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-02-12 08:51:23 -05:00
John Ferlan
5f02df444b tests: Use VIR_AUTOFREE for various storage tests
Let's make use of the auto __cleanup capabilities cleaning up any
now unnecessary goto paths.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-02-12 08:51:23 -05:00
John Ferlan
9a4199304f storage: Use VIR_AUTOPTR(virCommand)
Let's make use of the auto __cleanup capabilities cleaning up any
now unnecessary goto paths.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-02-12 08:51:23 -05:00
Daniel P. Berrangé
600462834f Remove all Author(s): lines from source file headers
In many files there are header comments that contain an Author:
statement, supposedly reflecting who originally wrote the code.
In a large collaborative project like libvirt, any non-trivial
file will have been modified by a large number of different
contributors. IOW, the Author: comments are quickly out of date,
omitting people who have made significant contribitions.

In some places Author: lines have been added despite the person
merely being responsible for creating the file by moving existing
code out of another file. IOW, the Author: lines give an incorrect
record of authorship.

With this all in mind, the comments are useless as a means to identify
who to talk to about code in a particular file. Contributors will always
be better off using 'git log' and 'git blame' if they need to  find the
author of a particular bit of code.

This commit thus deletes all Author: comments from the source and adds
a rule to prevent them reappearing.

The Copyright headers are similarly misleading and inaccurate, however,
we cannot delete these as they have legal meaning, despite being largely
inaccurate. In addition only the copyright holder is permitted to change
their respective copyright statement.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-12-13 16:08:38 +00:00
Peter Krempa
6e7e965dcd util: storage: Properly parse URIs with missing trailing slash
The URI parser used by libvirt does not populate uri->path if the
trailing slash is missing. The code virStorageSourceParseBackingURI
would then not populate src->path.

As only NBD network disks are allowed to have the 'name' field in the
XML defining the disk source omitted we'd generate an invalid XML which
we'd not parse again.

Fix it by populating src->path with an empty string if the uri is
lacking slash.

As pointed out above NBD is special in this case since we actually allow
it being NULL. The URI path is used as export name. Since an empty
export does not make sense the new approach clears the src->path if the
trailing slash is present but nothing else.

Add test cases now to cover all the various cases for NBD and non-NBD
uris as there was to time only 1 test abusing the quirk witout slash for
NBD and all other URIs contained the slash or in case of NBD also the
export name.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2018-10-12 16:41:48 +02:00
Erik Skultety
5165ff0971 src: More cleanup of some system headers already contained in internal.h
All of the ones being removed are pulled in by internal.h. The only
exception is sanlock which expects the application to include <stdint.h>
before sanlock's headers, because sanlock prototypes use fixed width
int, but they don't include stdint.h themselves, so we have to leave
that one in place.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 10:16:39 +02:00
Ján Tomko
d99a89592d Revert "Switch from yajl to Jansson"
This reverts commit 9cf38263d0.

Jansson cannot parse QEMU's quirky JSON.
Revert back to yajl.

https://bugzilla.redhat.com/show_bug.cgi?id=1614569

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-08-13 15:50:01 +02:00
Ján Tomko
9cf38263d0 Switch from yajl to Jansson
Yajl has not seen much activity upstream recently.
Switch to using Jansson >= 2.5.

All the platforms we target on https://libvirt.org/platforms.html
have a version >= 2.7 listed on the sites below:
https://repology.org/metapackage/jansson/versions
https://build.opensuse.org/package/show/devel:libraries:c_c++/libjansson

Additionally, Ubuntu 14.04 on Travis-CI has 2.5. Set the requirement
to 2.5 since we don't use anything from newer versions.

Implement virJSONValue{From,To}String using Jansson, delete the yajl
code (and the related virJSONParser structure) and report an error
if someone explicitly specifies --with-yajl.

Also adjust the test data to account for Jansson's different whitespace
usage for empty arrays and tune up the specfile to keep 'make rpm'
working when bisecting.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-07-18 17:54:18 +02:00
Peter Krempa
e1e8d0a9b7 util: storage: remove 'allow_probe' from virStorageFileGetMetadata
All callers pass 'false' now so it's no longer needed.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-08 09:26:38 +02:00
Peter Krempa
e0f8839b88 tests: storage: Drop duplicate tests with TEST_CHAIN
Now that all test cases with TEST_CHAIN were testing the same thing
twice drop one of them. Note that some of the cases were duplicate even
before dropping the image format probing tests.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-08 09:26:38 +02:00
Peter Krempa
655b95a01d tests: storage: Drop format probing from tests
Storage drivers now don't allow it so there's no need to test it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-08 09:26:38 +02:00
Peter Krempa
67d7d41615 tests: storage: Absorb ALLOW_PROBE flag into the TEST_CHAIN macro
The second set of arguments for TEST_CHAIN always specifies the
'ALLOW_PROBE' flag. Make it part of the macro.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-08 09:26:38 +02:00
Peter Krempa
300515b414 tests: virstoragetest: Add complementary test case for QED format
We have a test case for QED disk image with autodetection but not with
the format explicitly specified.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-06-08 09:26:38 +02:00
ramyelkest
2b6667abbf all: Replace virGetLastError with virGetLastErrorCode where we can
Replace instances where we previously called virGetLastError just to
either get the code or to check if an error exists with
virGetLastErrorCode to avoid a validity pre-check.

Signed-off-by: Ramy Elkest <ramyelkest@gmail.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2018-06-05 18:44:05 +02:00
Andrea Bolognani
4267393198 all: Use virFileCanonicalizePath() instead of canonicalize_file_name()
The latter is impossible to mock on platforms that use the
gnulib implementation, such as FreeBSD, while the former
doesn't suffer from this limitation.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-03 18:23:30 +02:00
Peter Krempa
28a36fe2ce util: storage: Remove detected authentication data for backing chains
We can't really detect all the authentication data in a sane manner for
disk backing chains. Since the old RBD parser parses it in some cases as
the argv->XML convertor requires it, we can't just drop it.

Instead clear any detected authentication data in the code paths related
to disk backing chain lookup and fix the tests to cope with the change.

https://bugzilla.redhat.com/show_bug.cgi?id=1544659

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2018-02-14 16:04:46 +01:00
Peter Krempa
d70d07eef8 virstoragetest: Add test case for NBD over unix socket with new syntax
Use the new syntax which uses the 'UnixSocket' type in qemu.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2018-02-14 15:58:04 +01:00
Daniel P. Berrangé
064fec69be storage: move storage file backend framework into util directory
The QEMU driver loadable module needs to be able to resolve all ELF
symbols it references against libvirt.so. Some of its symbols can only
be resolved against the storage_driver.so loadable module which creates
a hard dependancy between them. By moving the storage file backend
framework into the util directory, this gets included directly in the
libvirt.so library. The actual backend implementations are still done as
loadable modules, so this doesn't re-add deps on gluster libraries.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2018-02-09 11:05:10 +00:00
Peter Krempa
f46d6e22f2 util: storage: Parse 'lun' for iSCSI protocol from JSON as string or number
While the QEMU QAPI schema describes 'lun' as a number, the code dealing
with JSON strings does not strictly adhere to this schema and thus
formats the number back as a string. Use the new helper to retrieve both
possibilities.

Note that the formatting code is okay and qemu will accept it as an int.

Tweak also one of the test strings to verify that both formats work
with libvirt.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1540290
2018-01-31 12:22:25 +01:00
Peter Krempa
aed3d038a6 conf: Add infrastructure for disk source private data XML
VM drivers may need to store additional private data to the status XML
so that it can be restored after libvirtd restart. Since not everything
is needed add a callback infrastructure, where VM drivers can add only
stuff they need.

Note that the private data is formatted as a <privateData> sub-element
of the <disk> or <backingStore> <source> sub-element. This is done since
storing it out of band (in the VM private data) would require a complex
matching process to allow to put the data into correct place.
2017-12-14 10:24:36 +01:00
Peter Krempa
0a3bae7d22 util: storage: Fix parsing of IPv6 portal address for iSCSI
Split on the last colon and avoid parsing port if the split remainder
contains the closing square bracket, so that IPv6 addresses are
interpreted correctly.
2017-11-08 13:10:40 +01:00
Peter Krempa
bd37213786 qemu: block: Use proper type for servers for VxHS disks
Original implementation used 'SocketAddress' equivalent from qemu for
the disk server field, while qemu documentation specifies
'InetSocketAddress'. The backing store parser uses the correct parsing
function but the formatter used the incorrect one (and also with the
legacy mode enabled which was wrong).
2017-11-07 14:43:12 +01:00
Andrea Bolognani
3e7db8d3e8 Remove backslash alignment attempts
Right-aligning backslashes when defining macros or using complex
commands in Makefiles looks cute, but as soon as any changes is
required to the code you end up with either distractingly broken
alignment or unnecessarily big diffs where most of the changes
are just pushing all backslashes a few characters to one side.

Generated using

  $ git grep -El '[[:blank:]][[:blank:]]\\$' | \
    grep -E '*\.([chx]|am|mk)$$' | \
    while read f; do \
      sed -Ei 's/[[:blank:]]*[[:blank:]]\\$/ \\/g' "$f"; \
    done

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-11-03 13:24:12 +01:00
John Ferlan
8002d3cb1b conf: Add/Allow parsing the auth in the disk source
Since the virStorageAuthDefPtr auth; is a member of _virStorageSource
it really should be allowed to be a subelement of the disk <source>
for the RBD and iSCSI prototcols. That way we can set up to allow
the <auth> element to be formatted within the disk source.

Since we've allowed the <auth> to be a child of <disk>, we'll need
to keep track of how it was read so that when writing out we'll know
whether to format as child of <disk> or <source>. For the argv2xml
parsing, let's format under <source> as a preference. Do not allow
<auth> to be both a child of <disk> and <source>.

Modify the qemuxml2argvtest to add a parse failure when there is an
<auth> as a child of <disk> *and* an <auth> as a child of <source>.

Add tests to validate that if the <auth> was found in <source>, then
the resulting xml2xml and xml2arg works just fine.  The two new .args
file are exact copies of the non "-source" version of the file.

The virschematest will read the new test files and validate from a
RNG viewpoint things are fine

Update the virstoragefile, virstoragetest, and args2xml file to show
the "preference" to place <auth> as a child of <source>.
2017-10-19 15:26:49 -04:00
Peter Krempa
0a294a8e28 util: storagefile: Add helpers to check presence of backing store
Add helpers that will simplify checking if a backing file is valid or
whether it has backing store. The helper virStorageSourceIsBacking
returns true if the given virStorageSource is a valid backing store
member. virStorageSourceHasBacking returns true if the virStorageSource
has a backing store child.

Adding these functions creates a central points for further refactors.
2017-10-17 06:19:18 +02:00
Peter Krempa
8fdeefe1df test: set 'type' field of virStorageSource
Set the type so that the iterators will work after upcoming
modification.
2017-10-17 06:19:18 +02:00
Ashish Mittal
2a48252bb5 util: storage: Add JSON backing volume parse for VxHS
Add the backing parse and a test case to verify parsing of VxHS
backing storage.

Signed-off-by: Ashish Mittal <Ashish.Mittal@veritas.com>
Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-09-19 21:10:21 -04:00
Martin Kletzander
3401e208ab qemu: Don't mangle the storage format for type='dir'
Our backing probing code handles directory file types properly in
virStorageFileGetMetadataRecurse(), by that I mean it leaves them
alone.  However its caller, the virStorageFileGetMetadata() resets the
type to raw before probing, without even checking the type.  We need
to special-case TYPE_DIR in order to achieve desired results.

Also, in order to properly test this, we need to stop resetting format
of volumes in tests for TYPE_DIR (probably the reason why we didn't
catch that and why the test data didn't need to be modified).

Partially-resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1443434

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-08-29 16:30:04 +02:00
Peter Krempa
8444419f8c util: storage: fill in default ports when parsing backing chain
Similarly to when parsing XML we need to fill in default ports for the
backing chain. This was missed in commit 5bda835466
2017-07-24 10:55:43 +02:00
Peter Krempa
9506bd25a3 storage: Split out virStorageSource accessors to separate file
The helper methods for actually accessing the storage objects don't
really belong to the main storage driver implementation file. Split them
out.
2017-07-11 17:07:04 +02:00
Peter Krempa
2117d42c7c tests: storage: Fully register storage driver
Use the full storage driver registration method that also fails if one
of the storage backends is not present. This makes the test fail if a
submodule fails registration, which is useful for testing.

Additionally return EXIT_FAILURE as usual in tests rather than -1.
2017-07-11 17:07:04 +02:00
Peter Krempa
ffdf532328 tests: Validate that JSON deflattening fixed nested json pseudo-protocol strings
Sheepdog and possibly others use nested objects for network server and
thus could be specified in a way that libvirt would not parse.

Validates that https://bugzilla.redhat.com/show_bug.cgi?id=1464821
is fixed properly.
2017-07-11 14:24:17 +02:00
Cole Robinson
8a129b75e2 tests: virstoragetest: fix --without-yajl
Recently added JSON tests should be skipped if compiled --without-yajl

https://bugzilla.redhat.com/show_bug.cgi?id=1463435
2017-06-21 12:12:26 -04:00
Peter Krempa
b16133b114 util: storage: adapt to changes in JSON format for sheepdog
Since qemu 2.9 the options changed from a monolithic string into fine
grained options for the json pseudo-protocol object.
2017-06-20 08:40:18 +02:00
Peter Krempa
ea2c418ac3 util: storage: adapt to changes in JSON format for ssh
Since qemu 2.9 the options changed from a monolithic string into fine
grained options for the json pseudo-protocol object.
2017-06-20 08:40:18 +02:00
Peter Krempa
4fac5a1935 util: storage: adapt to changes in JSON format for ceph/rbd
Since qemu 2.9 the options changed from a monolithic string into fine
grained options for the json pseudo-protocol object.
2017-06-20 08:40:18 +02:00
Peter Krempa
35d23f90b2 util: storage: adapt to changes in JSON format for NBD
Since 2.9 the host and port for NBD are no longer directly under the
json pseudo-protocol object, but rather belong to a sub-object called
'server'.
2017-06-20 08:40:18 +02:00
Peter Krempa
b24bc54080 util: storage: Add JSON parser for new options in iSCSI protocol
Starting from qemu 2.9, more granular options are supported. Add parser
for the relevant bits.

With this patch libvirt is able to parse the host and target IQN of from
the JSON pseudo-protocol specification.

This corresponds to BlockdevOptionsIscsi in qemu qapi.
2017-06-20 08:40:18 +02:00
Peter Krempa
1f915d40a2 util: storage: Add support for type 'inet' in virStorageSourceParseBackingJSONSocketAddress
'SocketAddress' structure was changed to contain 'inet' instead of
'tcp' since qemu commit c5f1ae3ae7b. Existing entries have a backward
compatibility layer.

Libvirt will parse 'inet' and 'tcp' as equivalents.
2017-06-20 08:40:18 +02:00
Andrea Bolognani
4ceac4bf29 tests: Rename VIRT_TEST_* macros to VIR_TEST_*
We use the "vir" prefix pretty consistently in our
APIs, both external and internal, which made these
macros stood out.
2017-04-04 17:30:03 +02:00
Tomáš Golembiovský
e4c6d4ae55 util: storage: add JSON backing volume parser 'raw' block driver
The 'raw' block driver in Qemu is not directly interesting from
libvirt's perspective, but it can be layered above some other block
drivers and this may be interesting for the user.

The patch adds support for the 'raw' block driver. The driver is treated
simply as a pass-through and child driver in JSON is queried to get the
necessary information.

Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com>
2017-02-22 10:39:57 +01:00
Peter Krempa
93a3f516ee tests: drivermodule: Make sure that all compiled storage backends can be loaded
Add a new storage driver registration function that will force the
backend code to fail if any of the storage backend modules can't be
loaded. This will make sure that they work and are present.
2017-02-22 09:31:33 +01:00
Peter Krempa
f813fe810f storage: backend: Refactor registration of the backend drivers
Add APIs that allow to dynamically register driver backends so that the
list of available drivers does not need to be known during compile time.

This will allow us to modularize the storage driver on runtime.
2017-02-21 09:34:30 +01:00
Peter Krempa
3de7da9448 util: storage: Add json pseudo protocol support for legacy RBD strings
RBD in qemu still uses only the legacy 'filename' syntax.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1371758
2016-09-06 14:01:41 +02:00
Peter Krempa
b7a650c97c util: storage: Properly set protocol type when parsing gluster json string
Commit 2ed772cd forgot to set proper protocol. This was also present in
the test data.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1372251
2016-09-06 14:01:41 +02:00
Ján Tomko
f9785569de cfg.mk: join not_streq and not_strneq tests
The marginally nicer error message is not worth the extra lines in
cfg.mk.

Also drop the excludes since there was only one offender in the tests.
2016-08-18 13:37:08 +02:00
Peter Krempa
c2e12b01ba utils: storage: Fix JSON field name for uri based storage
qemu uses 'url' instead of 'uri'. They unfortunately look very similar.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1367260
2016-08-16 13:30:45 +02:00
Peter Krempa
bc225b1b5f util: storage: Add JSON backing volume parser for 'ssh' protocol 2016-07-27 13:24:20 +02:00
Peter Krempa
a1674fd9d9 util: storage: Add JSON backing volume parser for 'nbd' protocol 2016-07-27 13:24:20 +02:00
Peter Krempa
2ed772cd63 util: storage: Add json pseudo protocol support for gluster volumes
Along with the legacy URI based syntax add support for the brand-new
fully object based syntax.
2016-07-27 13:24:20 +02:00
Peter Krempa
ba05b5b7e7 util: storage: Add support for URI based backing volumes in qemu's JSON pseudo-protocol
http(s), ftp(s) and tftp use URIs for volume definitions in the JSON
pseudo protocol so it's pretty straightforward to add support for them.
2016-07-27 13:24:20 +02:00
Peter Krempa
47f292dd35 util: storage: Add support for host device backing specified via JSON
JSON pseudo protocol for qemu allows to explicitly specify devices.
Add convertor to the internal type.
2016-07-27 13:24:20 +02:00
Peter Krempa
e91f767c74 util: storage: Add parser for qemu's json backing pseudo-protocol
Add a modular parser that will allow to parse 'json' backing definitions
that are supported by qemu. The initial implementation adds support for
the 'file' driver.

Due to the approach qemu took to implement the JSON backing strings it's
possible to specify them in two approaches.

The object approach:
    json:{ "file" : { "driver":"file",
                      "filename":"/path/to/file"
                    }
         }

And a partially flattened approach:
    json:{"file.driver":"file"
          "file.filename":"/path/to/file"
         }

Both of the above are supported by qemu and by the code added in this
commit. The current implementation de-flattens the first level ('file.')
if possible and required. Other handling may be added later but
currently only one level was possible anyways.
2016-07-27 13:24:20 +02:00
Peter Krempa
4e3dbfa2a5 tests: Add testing of backing store string parser
As we already test that the extraction of the backing store string works
well additional tests for the backing store string parser can be made
simpler.

Export virStorageSourceNewFromBackingAbsolute and use it to parse the
backing store strings, format them using virDomainDiskSourceFormat and
match them against expected XMLs.
2016-07-27 09:39:33 +02:00
Tomáš Ryšavý
f5444742b0 tests: Rename virtTestCounterNext to virTestCounterNext.
This function doesn't follow our convention of naming functions.
2016-06-08 11:23:13 -04:00
Tomáš Ryšavý
327b844341 tests: Rename virtTestCounterReset to virTestCounterReset.
This function doesn't follow our convention of naming functions.
2016-06-08 11:23:12 -04:00
Tomáš Ryšavý
062ea148a0 Rename virtTestDifference to virTestDifference.
This function doesn't follow our convention of naming functions.
2016-06-08 11:23:12 -04:00
Tomáš Ryšavý
cd7dd1508d tests: Rename virtTestRun to virTestRun.
This function doesn't follow our convention of naming functions.
2016-06-08 11:23:12 -04:00
Cole Robinson
487d211d20 storage: remove support for /usr/bin/kvm-img
This an ubuntu/debian packaging convention. At one point it may have
been an actually different binary, but at least as of ubuntu precise
(the oldest supported ubuntu distro, released april 2012) kvm-img is
just a symlink to qemu-img for back compat.

I think it's safe to drop support for it
2016-04-20 08:55:36 -04:00
Peter Krempa
fdb80ed4f6 util: storage: Fix parsing of nbd:// URI without path
If a storage file would be backed with a NBD device without path
(nbd://localhost) libvirt would crash when parsing the backing path for
the disk as the URI structure's path element is NULL in such case but
the NBD parser would access it shamelessly.
2015-02-04 08:38:25 +01:00
Peter Krempa
df04741cd3 tests: virstoragetest: Switch backing chain test to use automatic numbering
I'm going to add a few test cases so it's the best time to convert the
test to automatic numbering.
2015-02-04 08:31:36 +01:00
Peter Krempa
930b77598b storage: Allow parsing of RBD backing strings when building backing chain
As we now have a common function to parse backing store string for RBD
backing store we can reuse it in the backing store walker so that we
don't fail on files backed by RBD storage.

This patch also adds a few tests to verify that the parsing works as
expected.
2014-11-21 14:37:02 +01:00
Peter Krempa
e650f30b93 test: virstoragetest: Add testing of network disk details
To be able to fully test parsing of networked storage strings we need to
add a few fields for: hostname, protocol and auth string.
2014-11-21 14:37:01 +01:00
Peter Krempa
98784369fd storage: Fix crash when parsing backing store URI with schema
The code that parses the schema from the URI touches the "hosts[0]"
member of the storage file source structure in case the URI contains a
schema. The hosts array was not yet allocated at the point in the code
where the transport protocol was parsed and set. This lead to a crash of
libvirtd.

Fix the code by allocating the "hosts" array upfront and add a test case
to verify this scenario. (Unfortunately this requires shuffling the test
case numbers too).

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1156288
2014-10-29 17:10:42 +01:00
Peter Krempa
b8549877a1 util: storage: Allow metadata crawler to report useful errors
Add a new parameter to virStorageFileGetMetadata that will break the
backing chain detection process and report useful error message rather
than having to use virStorageFileChainGetBroken.

This patch just introduces the option, usage will be provided
separately.
2014-09-24 09:28:29 +02:00
Peter Krempa
750177104d util: storage: Return complete parent info from virStorageFileChainLookup
Instead of just returning the parent path, return the complete parent
source structure.
2014-07-09 11:41:34 +02:00
Peter Krempa
9a39f50420 storage: Don't store parent directory of an image explicitly
The parent directory doesn't necessarily need to be stored after we
don't mangle the path stored in the image. Remove it and tweak the code
to avoid using it.
2014-06-25 10:05:56 +02:00
Peter Krempa
e71437fff2 storage: Don't canonicalize paths unnecessarily
Store backing chain paths as non-canonical. The canonicalization step
will be already taken. This will allow to avoid storing unnecessary
amounts of data.
2014-06-25 10:02:59 +02:00
Peter Krempa
4074ad2d74 tests: virstoragetest: Remove unneeded relative test plumbing
After we don't test relative paths, remove even more unnecessary cruft
from the test code.
2014-06-25 10:02:02 +02:00