Commit Graph

1276 Commits

Author SHA1 Message Date
John Ferlan
3d3647e14f storage: Add the nfsvers to the command line
If protocolVer present, add the -o nfsvers=# to the command
line for the NFS Storage Pool

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-29 19:16:07 -05:00
John Ferlan
f00cde7f11 storage: Add default mount options for fs/netfs storage pools
https://bugzilla.redhat.com/show_bug.cgi?id=1584663

Modify the command generation to add some default options to the
fs/netfs storage pools based on the OS type. For Linux, it'll be
the "nodev, nosuid, noexec". For FreeBSD, it'll be "nosuid, noexec".
For others, just leave the options alone.

Modify the storagepoolxml2argvtest to handle the fact that the
same input XML could generate different output XML based on whether
Linux, FreeBSD, or other was being built.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-01-29 19:15:20 -05:00
Michal Privoznik
5772885d28 lib: Use more of VIR_STEAL_PTR()
We have this very handy macro called VIR_STEAL_PTR() which steals
one pointer into the other and sets the other to NULL. The
following coccinelle patch was used to create this commit:

  @ rule1 @
  identifier a, b;
  @@

  - b = a;
    ...
  - a = NULL;
  + VIR_STEAL_PTR(b, a);

Some places were clean up afterwards to make syntax-check happy
(e.g. some curly braces were removed where the body become a one
liner).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2019-01-28 14:46:58 +01:00
Daniel P. Berrangé
568a417224 Enforce a standard header file guard symbol name
Require that all headers are guarded by a symbol named

  LIBVIRT_$FILENAME

where $FILENAME is the uppercased filename, with all characters
outside a-z changed into '_'.

Note we do not use a leading __ because that is technically a
namespace reserved for the toolchain.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-12-14 10:47:13 +00:00
Daniel P. Berrangé
4cfd709021 Fix many mistakes & inconsistencies in header file layout
This introduces a syntax-check script that validates header files use a
common layout:

  /*
   ...copyright header...
   */
  <one blank line>
  #ifndef SYMBOL
  # define SYMBOL
  ....content....
  #endif /* SYMBOL */

For any file ending priv.h, before the #ifndef, we will require a
guard to prevent bogus imports:

  #ifndef SYMBOL_ALLOW
  # error ....
  #endif /* SYMBOL_ALLOW */
  <one blank line>

The many mistakes this script identifies are then fixed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-12-14 10:46:53 +00:00
John Ferlan
e6f53e7a4b storage: Fix build issue with MOUNT and VGCHANGE commands
Turns out there some build platforms that must not define MOUNT
or VGCHANGE in config.h... So moving the commands from the storage
backend specific module into a common storage_util module causes
issues for those platforms.

So instead of assuming they are there, let's just pass the command
string to the storage util API's from the storage backend specific
code (as would have been successful before).  Also modify the test
to determine whether the MOUNT and/or VGCHANGE doesn't exist and
just define it to (for example) what Fedora has for the path. Could
have just used "mount" and "vgchange" in the call, but that defeats
the purpose of adding the call to virTestClearCommandPath.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-12-13 12:42:36 -05:00
Daniel P. Berrangé
600462834f Remove all Author(s): lines from source file headers
In many files there are header comments that contain an Author:
statement, supposedly reflecting who originally wrote the code.
In a large collaborative project like libvirt, any non-trivial
file will have been modified by a large number of different
contributors. IOW, the Author: comments are quickly out of date,
omitting people who have made significant contribitions.

In some places Author: lines have been added despite the person
merely being responsible for creating the file by moving existing
code out of another file. IOW, the Author: lines give an incorrect
record of authorship.

With this all in mind, the comments are useless as a means to identify
who to talk to about code in a particular file. Contributors will always
be better off using 'git log' and 'git blame' if they need to  find the
author of a particular bit of code.

This commit thus deletes all Author: comments from the source and adds
a rule to prevent them reappearing.

The Copyright headers are similarly misleading and inaccurate, however,
we cannot delete these as they have legal meaning, despite being largely
inaccurate. In addition only the copyright holder is permitted to change
their respective copyright statement.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-12-13 16:08:38 +00:00
John Ferlan
a15fe1247d storage: Add tests for logical backend startup
Add the logical storage pool startup validation (xml2argv) tests.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-12-13 07:09:42 -05:00
John Ferlan
df1a9c7380 logical: Fix @on argument type
It's only pass as 0 or 1 and used as a bool, let's just use a bool

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-12-13 07:09:42 -05:00
John Ferlan
b663d4329c storage: Rework virStorageBackendFileSystemMountCmd
Let's create helpers for each style of command line created. This
primarily is easier on the eyes rather than the large multi line
if-then-else-else clause used, but may also be useful if in the
future any particular pool needs to add to the command line based
on pool xml format.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-12-13 07:09:42 -05:00
John Ferlan
1f6ca26d72 storage: Move virStorageBackendFileSystemGetPoolSource
Move into storage_util for reuse by test harness

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-12-13 07:09:42 -05:00
John Ferlan
728b9ed61a storage: Move FS backend mount creation command helper
Move virStorageBackendFileSystemMountCmd to storage_util so that
it can be used by the test harness.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-12-13 07:09:42 -05:00
John Ferlan
1bebb904fe storage: Extract out mount command creation for FS Backend
Extract out the code that is used to create the MOUNT command
for starting the pool. We can use this for Storage Pool XML
to Argv testing to ensure code changes don't alter how a
storage pool is started.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-12-13 07:09:42 -05:00
Yuri Chornoivan
e5c1fbca24 Fix minor typos in messages and docs
Signed-off-by: Yuri Chornoivan <yurchor@ukr.net>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-12-05 10:39:54 +01:00
Erik Skultety
f1e8d2f09a conf: Add new module node_device_util
There's a lot of stuff going on in src/conf/nodedev_conf which is
sometimes not directly related to config and we're not really consistent
with putting only parser/formatter related stuff here, e.g. like we do
for domains. So, let's start simply by adding a new module
node_device_util containing some of the helpers. Unfortunately, even
though these helpers tend to open a secondary driver connection and would
be much therefore better suited as a nodedev driver module, we can't do
that without pulling headers from the driver into conf/ and that's wrong
because we want conf/ to stay driver-agnostic.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
2018-11-12 16:23:35 +01:00
John Ferlan
e3a42028af Remove ignore_value or void from unlink calls
There seems to be no need to add the ignore_value wrapper or
caste with (void) to the unlink() calls, so let's just remove
them. I assume at one point in time Coverity complained. So,
let's just be consistent - those that care to check the return
status can and those that don't can just have the naked unlink.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2018-09-20 13:45:56 -04:00
John Ferlan
5309b6cb64 storage: Save error during refresh failure processing
https://bugzilla.redhat.com/show_bug.cgi?id=1614283

Save the error from the refresh failure because the stopPool
processing may overwrite the error or even worse clear it
due to calling an external libvirt API that resets the last
error such as is the case with the SCSI pool which may call
virGetConnectNodeDev (see commit decaeb288) in order to
process deleting an NPIV vport.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 07:49:49 -04:00
John Ferlan
5745f08cea storage: Introduce storagePoolRefreshFailCleanup
Create a common pool refresh failure handling method as the
same code is repeated multiple times.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 07:49:43 -04:00
John Ferlan
1ff45609d6 storage: Create error label path for storagePoolCreateXML
Rather than duplicate the error code, let's create an error
label to keep code common.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 07:49:38 -04:00
John Ferlan
4a7abc67d5 storage: Clean up storagePoolUpdateStateCallback processing
Alter the code path to remove the need to to go cleanup and thus
remove the label completely.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 07:49:33 -04:00
John Ferlan
49c322145b storage: Clean up stateFile if refreshPool fails
If the virStoragePoolRefresh fails and we call stopPool, the
code neglected to clean up the state file leading to the next
libvirtd restart attempting to start the pool. For a transient
pool this could make it unexpectedly reappear.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 07:49:19 -04:00
Erik Skultety
5165ff0971 src: More cleanup of some system headers already contained in internal.h
All of the ones being removed are pulled in by internal.h. The only
exception is sanlock which expects the application to include <stdint.h>
before sanlock's headers, because sanlock prototypes use fixed width
int, but they don't include stdint.h themselves, so we have to leave
that one in place.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 10:16:39 +02:00
Erik Skultety
9403b63102 internal: Move <stdio.h> include to internal.h
It doesn't really make sense for us to have stdlib.h and string.h but
not stdio.h in the internal.h header.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-20 10:16:38 +02:00
Shi Lei
c9ed87a610 src: remove blank first line in function body
Signed-off-by: Shi Lei <shi_lei@massclouds.com>
2018-09-17 13:29:01 +02:00
John Ferlan
b975afc725 storage: Allow inputvol to be encrypted
https://bugzilla.redhat.com/show_bug.cgi?id=1613737

When processing the inputvol for encryption, we need to handle
the case where the inputvol is encrypted. This then allows for
the encrypted inputvol to be used either for an output encrypted
volume or an output volume of some XML provided type.

Add tests to show the various conversion options when either input
or output is encrypted. This includes when both are encrypted.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-12 07:24:04 -04:00
John Ferlan
8041471858 storage: Allow for inputvol to have any format for encryption
Commit 39cef12a9 altered/fixed the inputvol processing to create
a multistep process when using an inputvol to create an encrypted
output volume; however, it unnecessarily assumed/restricted the
inputvol to be of 'raw' format only.

Modify the processing code to allow the inputvol format to be checked
and used in order to create the encrypted volume.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-12 07:24:04 -04:00
John Ferlan
f6aa977fab storage: Remove secretPath from _virStorageBackendQemuImgInfo
There's really no need for it to be there since it's only ever
used inside virStorageBackendCreateQemuImgCmdFromVol

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-09-12 07:24:04 -04:00
Michal Privoznik
ca0ab9cdd2 storage_driver: Release pool object lock for some long running jobs
As advertised in previous commit, there are three APIs that might
run for quite some time (because they read/write data from/to a
volume) and these three are: downloadVol, uploadVol, wipeVol.
Release pool object lock and reacquire it later to allow more
concurrency.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-09-04 10:11:40 +02:00
Michal Privoznik
f1ae8ecc90 storage_driver: Mark volume as 'in use' for some operations
There are few operations in the storage driver that read/write
data onto volumes. Such operations can take very long time to
finish. During that time the storage pool object is locked which
has bad performance impacts (other threads can't fetch its XML
for instance). This commit prepares the storage driver for
releasing the lock during those operations (downloadVol,
uploadVol, wipeVol).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-09-04 10:11:40 +02:00
Michal Privoznik
bc9a80161a virstorageobj: Check for source duplicates from virStoragePoolObjAssignDef
Just like a few commits earlier, checking for pool source
duplicates and unlocking pools list afterwards is a buggy
pattern. The check must go into virStoragePoolObjAssignDef.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-09-04 10:11:40 +02:00
Michal Privoznik
4f426ce4ba virStoragePoolObjSourceFindDuplicate: Drop @conn argument
The @conn argument is needed only to do some source matching in
case of iSCSI source. Anyway, it's used just for node device
driver and as such can be replaced with virGetConnectNodeDev().

At the same time, the @conn struct member is dropped from
_virStoragePoolObjFindDuplicateData.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-09-04 10:11:40 +02:00
Michal Privoznik
4391b5222f virstorageobj: Check for duplicates from virStoragePoolObjAssignDef
Even though we do some checking it is not as thorough as it
should be. We already have virStoragePoolObjIsDuplicate but the
way we use it is a typical TOCTOU. Imagine two threads trying to
define two pools with the same name but different UUIDs. With the
current code neither of them finds a duplicate and thus proceed
to virStoragePoolObjAssignDef where only names are compared.
Therefore both threads succeed which is obviously wrong.

We should check for duplicates where we care for them.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-09-04 10:11:40 +02:00
Michal Privoznik
16f5abb2f8 storage_backend_rbd: Drop ATTRIBUTE_UNUSED for arguments that are used
In two places the passed pool object argument is marked as
ATTRIBUTE_UNUSED even though it's used right away.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-09-04 10:11:40 +02:00
John Ferlan
b04d1b6a35 storage: Add --shrink to qemu-img command when shrinking vol
https://bugzilla.redhat.com/show_bug.cgi?id=1613746

When shrinking the capacity of a qcow2 or luks volume using
the qemu-img program, the --shrink qualifier must be added.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-08-24 09:14:13 -04:00
Shivaprasad G Bhat
1a94cbcd16 storage: fix the error message when encrypted raw volume resize
The vol-dumpxml shows the volume target format type as raw for
encrypted volumes. The error message when attempting to resize
with prealloc is confusing here.

Signed-off-by: Shivaprasad G Bhat <sbhat@linux.vnet.ibm.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-08-23 12:14:31 -04:00
Michal Privoznik
bfd91dc0c4 storage: Properly terminate secrets
The virSecretGetSecretString() helper looks up a secret for given
pool and returns its value in @secret_value and its length in
@secret_value_size. However, the trailing '\0' is not included in
either of the variables. This is because usually the value of the
secret is passed to some encoder (usually base64 encoder) where
the trailing zero must not be accounted for.

However, in two places we actually want the string as we don't
process it any further.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-08-15 17:11:16 +02:00
Clementine Hayat
e41bfae562 storage: add wipeVol to iscsi-direct storage backend
Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-08-13 08:44:34 +02:00
Clementine Hayat
e68426f0ad storage: add SetConnection to iscsi-direct backend
The code to set the connection and connect using libiscsi will always be
the same. Add function to avoid code duplication.

Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-08-13 08:20:06 +02:00
Clementine Hayat
14bfe9fa84 storage: refactor volume capacity in iscsi-direct backend
The size of blocks inside a volume and the number of blocks are needed
later. Having a function that return thoses information will avoid code
duplication.

Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-08-13 08:09:03 +02:00
Daniel P. Berrangé
f72c5945e1 storage: tweak error message when skipping file
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-08-09 11:19:15 +01:00
Clementine Hayat
01dd26811a storage: add findPoolSources to iscsi_direct pool backend
Change the SetContext function to be able to take the session type in
argument.
Took the function findPoolSources of iscsi backend and wired it to my
function since the formatting is essentially the same.

Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-08-06 12:24:18 +02:00
Clementine Hayat
91a3d88a61 storage: Implement iscsi_direct pool backend
We need here libiscsi for the storgae pool backend.
For the iscsi-direct storage pool, only checkPool and refreshPool should
be necessary for basic support.
The pool is state-less and just need the informations within the volume
to work.

Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-08-06 12:23:55 +02:00
Clementine Hayat
f0bf1be3e7 storage: Introduce iscsi_direct pool type
Introducing the pool as a noop. Integration inside the build
system. Implementation will be in the following commits.

Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-08-06 12:23:55 +02:00
Michal Privoznik
adfcbdff91 virISCSIScanTargets: Allow making targets persistent
After a new iSCSI interface is successfully set up, we issue a
sendtargets command. However, after 56057900dc we don't
update the host config which in turn makes login fail because
iscsiadm is unable to find any matching record for the interface.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-07-25 07:11:13 +02:00
Michal Privoznik
2e5ad5881b virISCSIScanTargets: Honour iSCSI interface
When scanning for targets, iSCSI might give different results
depending on the interface used. This is basically just name of
config file under /etc/iscsi/ifaces to use. The file contains
initiator IQN thus different results claim.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-07-25 07:11:13 +02:00
Michal Privoznik
fa18da2d81 storage_util: Prefer generic FICLONE over btrfs/xfs defines
After my change to the original patch that resulted in commit
8ed874b39b it was brought to my attention that all three defines
are the same: FICLONE = BTRFS_IOC_CLONE = XFS_IOC_CLONE (as
documented in ioctl_ficlone(2)). Therefore we should prefer
generic FICLONE over 'specific' defines for btrfs/xfs.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2018-07-24 15:15:34 +02:00
Shichangkuo
a94bb4ebdf storagePoolDefineXML: prefer using newDef to save config file
When re-defining an active storage pool, due to a bug the config
file on disk is not changed. This is because we are passing old
definition instead of new one to virStoragePoolObjSaveDef.
This issue was introduced by bfcd8fc9,

Signed-off-by: Changkuo Shi <shi.changkuo@h3c.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-07-11 11:22:26 +02:00
Julio Faracco
8ed874b39b storage: Rename btrfsCloneFile to support other filesystems.
This commit renames and adds other macros to support aother filesystems
when a reflink is performed. After that, XFS filesystems (and others)
with reflink support will be able to clone.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1565004

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-07-06 16:56:56 +02:00
John Ferlan
39cef12a95 storage: Add support for using inputvol for encryption
Starting with QEMU 2.9, encryption convert processing requires
a multi-step process in order to generate an encrypted image from
some non encrypted raw image.

Processing requires to first create an encrypted image using the
sizing parameters from the input source and second to use the
--image-opts, -n, and --target-image-opts options along with inline
driver options to describe the input and output files, generating
two commands such as:

  $ qemu-img create -f luks \
      --object secret,id=demo.img_encrypt0,file=/path/to/secretFile \
      -o key-secret=demo.img_encrypt0 \
      demo.img 500K
  Formatting 'demo.img', fmt=luks size=512000 key-secret=demo.img_encrypt0
  $ qemu-img convert --image-opts -n --target-image-opts \
      --object secret,id=demo.img_encrypt0,file=/path/to/secretFile \
      driver=raw,file.filename=sparse.img \
      driver=luks,file.filename=demo.img,key-secret=demo.img_encrypt0
  $

This patch handles the convert processing by running the processing
in a do..while loop essentially reusing the existing create logic and
arguments to create the target vol from the inputvol and then converting
the inputvol using new arguments.

This then allows the following virsh command to work properly:

  virsh vol-create-from default encrypt1-luks.xml data.img --inputpool default

where encrypt1-luks.xml would provided the path and secret for
the new image, while data.img would be the source image.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
John Ferlan
40f0e0348d storage: Remove storageBackendGenerateSecretData
Since we no longer support creating qcow2 encryption format
volumes, we no longer have to possibly create some secret and
have no real need for the function, so move the remaining
functionality to build the secret path back into the caller
storageBackendCreateQemuImg.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
John Ferlan
9b753116f0 storage: Clean up storageBackendCreateQemuImgOpts
Since we only generate the @encinfo when there's a secret object
and thus we need to reference it in the options,

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
John Ferlan
9fe597dd09 storage: Clean up storageBackendCreateQemuImgCheckEncryption
Remove the checks for qcow encryption since both callers (create
and resize) would have already disallowed usage.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
John Ferlan
8f83af6823 storage: Disallow create/resize of qcow2 encrypted images
https://bugzilla.redhat.com/show_bug.cgi?id=1526382

Since commit c4eedd793 disallowed qcow2 encrypted images to be
used for domains, it no longer makes sense to allow a qcow2
encrypted volume to be created or resized.

Add a test that will exhibit the failure of creation as well
as the xml2xml validation of the format still being correct.

Update the documentation to note the removal of the capability
to create and use qcow/default encrypted volumes.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
John Ferlan
af0e6580cd storage: Rename encryption info variable for clarity
Change from @enc to @encinfo leaving @enc for the vol->target.encryption
in the storageBackendCreateQemuImgSetOptions code path.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
John Ferlan
b056e09b28 storage: Don't allow encryption secretPath to be NULL
Allowing a NULL @secretPath for virStorageBackendCreateQemuImgCmdFromVol
would result in a generated command line with a dangling "file=" output.
So let's make sure the @secretPath exists before processing.

This means we should pass a dummy path from the storage test.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-06-26 14:02:43 -04:00
Anya Harter
031eb8f6dc events: add NULL check in virObjectEventStateQueue
And remove NULL checking from all callers.

Signed-off-by: Anya Harter <aharter@redhat.com>
2018-06-12 07:28:18 +02:00
Katerina Koukiou
fab2e49d3c storage: fix crash in luks encrypted volume creation
Fix the case when creating a luks encrypted volume
via an xml file without 'secret' element.
libvirtd was receiving SIGSEGV, now proper error is reported for
the missing element.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1468422

Signed-off-by: Katerina Koukiou <kkoukiou@redhat.com>
2018-06-06 18:02:59 +02:00
John Ferlan
9b837963c5 storage: Add specific check for LUKS encryption support
Modify virStorageBackendLogicalLVCreate to ensure if encryption
is requested that only type LUKS is supported; otherwise, error.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2018-06-05 07:42:00 -04:00
John Ferlan
57d6df39bd storage: Add capability to use LUKS encryption for disk backend
https://bugzilla.redhat.com/show_bug.cgi?id=1560946

Similar to the the Logical backend, use qemu-img on the created
disk partition device to set up for LUKS encryption. Secret mgmt
for the device can be complicated by a reboot possibly changing
the path to the device if the infrastructure changes.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Peter Krempa <pkrempa@redhat.com>
2018-05-29 10:48:51 -04:00
John Ferlan
e288080ae0 disk: Allow settling to occur after reading partitions
https://bugzilla.redhat.com/show_bug.cgi?id=1400475

In order to avoid a possible error as a result of kernel interactions
with the partition helper, let's use virWaitForDevices to force things
to settle down before attempting to open and read the partition. This
is related to https://bugzilla.redhat.com/show_bug.cgi?id=1264719.

Although perhaps overkill to have too many places to settle, since
we know that the act of reading the partitions via libvirt_parthelper
will cause udev activity/events - we just need to ensure udev has
been settled before proceding with usage of the device.

Signed-off-by: John Ferlan <jferlan@redhat.com>
ACKed-by: Michal Privoznik <mprivozn@redhat.com>
2018-05-25 08:26:41 -04:00
John Ferlan
352e549450 storage_util: Introduce storageBackendDoCreateQemuImg
Extract out command line setup and run from storageBackendCreateQemuImg
as we'll need to run it twice soon.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:57:24 -04:00
John Ferlan
e1e1fcf769 storage_util: Introduce storageBackendCreateQemuImgSetInput
Split up virStorageBackendCreateQemuImgCmdFromVol into two parts.
It's too long anyway and virStorageBackendCreateQemuImgCmdFromVol
should just handle the command line processing.

NB: Requires changing info.* into info->* references.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:57:24 -04:00
John Ferlan
4b9055c930 storage_util: Move @type into _virStorageBackendQemuImgInfo
We're about to split up the code a bit more, so we'll need this
to be in the local struct.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:57:24 -04:00
John Ferlan
b5c8c547b7 storage_util: Split preallocate set in storageBackendCreateQemuImgOpts
The only way preallocate could be set is if the info->format was
not RAW (see storageBackendCreateQemuImgSetBacking), so let's just
extract it from the if/else surrounding the application of the
encryption options.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:57:24 -04:00
John Ferlan
c46fc572be storage_util: Split backing_fmt set in storageBackendCreateQemuImgOpts
The only way backing_fmts could be set is if the info->format was
not RAW (see storageBackendCreateQemuImgSetBacking), so let's just
extract it from the if/else surrounding the application of the
encryption options.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:57:24 -04:00
John Ferlan
3d14408c17 storage_util: Remove luks distinction from secret path and alias
Remove the "luks" distinction as the code is about to become more
generic and be able to support qcow encryption as well.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:57:24 -04:00
John Ferlan
9118e23f6d storage_util: Move secretPath generation
Move generation of secretPath to storageBackendGenerateSecretData
and simplify a bit since we know vol->target.encryption is set plus
we have a local @enc.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:57:24 -04:00
John Ferlan
3d1db851ca storage_util: Generate the qcow secret earlier
Rather than having storageBackendCreateQemuImgCheckEncryption
perform the virStorageGenerateQcowEncryption, let's just do that
earlier during storageBackendCreateQemuImg so that the check
helper is just a check helper rather doing something different
based on whether the format is qcow[2] or raw based encryption.

This fixes an issue in the storageBackendResizeQemuImg processing
for qcow encryption where if a secret was not available for a
volume, a new secret will not be generated and instead an error
message will be generated.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:55:09 -04:00
John Ferlan
9629fe258f storage_util: Rename virQEMUBuildLuksOpts
Rename to storageBackendCreateQemuImgOpts - which is what it's doing.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:54:58 -04:00
John Ferlan
8fc9949caa storage_util: Remove unnecessary check
Commit id 'a48c71411' altered the logic a bit and didn't
remove an unnecessary check as info.encryption is true when
vol->target.encryption != NULL, so if we enter the if segment
with info.format == VIR_STORAGE_FILE_RAW && vol->target.encryption
!= NULL, then there's no way info.encryption could be false.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:54:58 -04:00
John Ferlan
f78c814c5c storage_util: Cleanup usage of target.encryption
Remove the != NULL checks, use !! for setting info.encryption.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:54:58 -04:00
John Ferlan
9cfd507367 storage_util: Some code cleanup
Perform some code cleanup in areas that are about to be altered.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2018-05-15 16:54:58 -04:00
Ján Tomko
e5261d8fe3 storage: remove qemu-img help scraping
We have been checking whether qemu-img supports the -o compat
option by scraping the -help output.

Since we require QEMU 1.5.0 now and this option was introduced in 1.1,
assume we support it and ditch the help parsing code along with the
extra qemu-img invocation.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-07 10:23:20 +02:00
Martin Kletzander
d80b84c68b storage/: Remove spaces after casts
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-03 22:31:37 +02:00
Andrea Bolognani
4267393198 all: Use virFileCanonicalizePath() instead of canonicalize_file_name()
The latter is impossible to mock on platforms that use the
gnulib implementation, such as FreeBSD, while the former
doesn't suffer from this limitation.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-03 18:23:30 +02:00
Daniel P. Berrangé
01888af0e2 storage: create separate loadable modules for storage file drivers
The storage file drivers are currently loaded as a side effect of
loading the storage driver. This is a bogus dependancy because the
storage file code has no interaction with the storage drivers, and
even ultimately be running in a completely separate daemon.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-03 13:17:59 +01:00
Daniel P. Berrangé
dbf28572c8 storage: split fs storage file code from storage driver backend
The storage file code needs to be run in the hypervisor drivers, while
the storage backend code needs to be run in the storage driver. Split
the source code as a preparatory step for creating separate loadable
modules.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-03 12:51:38 +01:00
Daniel P. Berrangé
f78cbe9424 storage: remove unused imports of libxml headers
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-03 12:49:57 +01:00
Daniel P. Berrangé
7568fc730f storage: split gluster storage file code from storage driver backend
The storage file code needs to be run in the hypervisor drivers, while
the storage backend code needs to be run in the storage driver. Split
the source code as a preparatory step for creating separate loadable
modules.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-03 12:49:22 +01:00
Daniel P. Berrangé
8b0f721f2e util: create new virmodule.{c,h} files for dlopen support code
The driver.{c,h} files are primarily targetted at loading hypervisor
drivers and some helper functions in that area. It also, however,
contains a generically useful function for loading extension modules
that is called by the storage driver. Split that functionality off
into a new virmodule.{c,h} file to isolate it.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-03 12:47:43 +01:00
Daniel P. Berrangé
8a062f5fe3 driver: add option to make missing drivers a fatal problem
Currently the driver module loading code does not report an error if the
driver module is physically missing on disk. This is useful for distro
packaging optional pieces. When the daemons are split up into one daemon
per driver, we will expect module loading to always succeed. If a driver
is not desired, the entire daemon should not be installed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-04-24 17:01:48 +01:00
Daniel P. Berrangé
2902b76472 driver: fix handling of error return from finding resource
The virFileFindResource method merely builds up the expected fully
qualified path to the resource. It does not actually check if it exists
on disk. The loadable module callers were mistakenly thinking a NULL
indicates the file doesn't exist on disk, whereas it in fact indicates
an out of memory error.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-04-24 17:00:49 +01:00
Daniel P. Berrangé
96a72f3444 driver: don't keep a pointer to the loaded library handle
Now that we've activated two hacks to prevent unloading of modules,
there is no point passing back a pointer to the loaded library handle.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-04-24 17:00:47 +01:00
Daniel P. Berrangé
4c8574c85c driver: ensure NULL URI isn't passed to drivers with whitelisted URIs
Ensuring that we don't call the virDrvConnectOpen method with a NULL URI
means that the drivers can drop various checks for NULL URIs. These were
not needed anymore since the probe functionality was split

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-04-12 16:52:02 +01:00
Daniel P. Berrangé
8e4f9a2773 driver: declare supported URI schemes in virConnectDriver struct
Declare what URI schemes a driver supports in its virConnectDriver
struct. This allows us to skip trying to open the driver entirely
if the URI scheme doesn't match.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-04-12 16:52:02 +01:00
Daniel P. Berrangé
3714cc952d driver: allow drivers to indicate if they permit remote connections
Add a localOnly flag to the virConnectDriver struct which allows a
driver to indicate whether it is local-only, or permits remote
connections. Stateful drivers running inside libvirtd are generally
local only. This allows us to remote the check for uri->server != NULL
from most drivers.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-04-12 16:52:02 +01:00
Wim ten Have
c9ec7088c7 storage: extend preallocation flags support for qemu-img
This patch adds support to qcow2 formatted filesystem object storage by
instructing qemu-img to build them with preallocation=falloc whenever the
XML described storage <allocation> matches its <capacity>.  For all other
cases the filesystem stored objects are built with preallocation=metadata.

Signed-off-by: Wim ten Have <wim.ten.have@oracle.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-04-11 10:15:58 +02:00
Michal Privoznik
7b4964479d storageDriverAutostart: Drop unused @conn
This variable is not actually used in the function. Especially
after 31869efe2a.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-04-06 14:57:34 +02:00
Michal Privoznik
36e7ce761b storageVolLookupData: Drop conn struct member
Since its introduction in 5d5c732d74 it has been never used.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-04-06 14:57:34 +02:00
Daniel P. Berrangé
d9da08b759 make: split storage driver build rules into storage/Makefile.inc.am
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-05 17:11:02 +00:00
Daniel P. Berrangé
7aa0e8c0cb storage: export virStoragePoolLookupByTargetPath as a public API
The storagePoolLookupByTargetPath() method in the storage driver is used
by the QEMU driver during block migration. If there's a valid use case
for this in the QEMU driver, then external apps likely have similar
needs. Exposing it in the public API removes the direct dependancy from
the QEMU driver to the storage driver.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-02-09 11:05:10 +00:00
Daniel P. Berrangé
fee840cc96 conf: move virStorageTranslateDiskSourcePool into domain conf
The virStorageTranslateDiskSourcePool method modifies a virDomainDiskDef
to resolve any storage pool reference. For some reason this was added
into the storage driver code, despite working entirely in terms of the
public APIs. Move it into the domain conf file and rename it to match the
object it modifies.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-02-09 11:05:10 +00:00
Daniel P. Berrangé
064fec69be storage: move storage file backend framework into util directory
The QEMU driver loadable module needs to be able to resolve all ELF
symbols it references against libvirt.so. Some of its symbols can only
be resolved against the storage_driver.so loadable module which creates
a hard dependancy between them. By moving the storage file backend
framework into the util directory, this gets included directly in the
libvirt.so library. The actual backend implementations are still done as
loadable modules, so this doesn't re-add deps on gluster libraries.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2018-02-09 11:05:10 +00:00
Daniel P. Berrangé
3be2d1684a storage: extract storage file backend from main storage driver backend
The storage driver backends are serving the public storage pools API,
while the storage file backends are serving the internal QEMU driver and
/ or libvirt utility code.

To prep for moving this storage file backend framework into the utility
code, split out the backend definitions.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2018-02-09 11:05:10 +00:00
Daniel P. Berrangé
31869efe2a storage: remove virConnectPtr from all backend functions
Now that we can open connections to the secondary drivers on demand,
there is no need to pass a virConnectPtr into all the backend
functions.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-01-31 18:34:48 +00:00
Daniel P. Berrangé
decaeb2882 storage: open nodedev driver connection at time of use
Instead of passing around a virConnectPtr object, just open a connection
to the nodedev driver at time of use. Opening connections on demand will
be beneficial when the nodedev driver is in a separate daemon. It also
solves the problem that a number of callers just pass in a NULL
connection today which prevents nodedev lookup working at all.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-01-31 18:34:48 +00:00
Daniel P. Berrangé
a494f7fd4f storage: open secret driver connection at time of use
Instead of passing around a virConnectPtr object, just open a connection
to the secret driver at time of use. Opening connections on demand will
be beneficial when the secret driver is in a separate daemon. It also
solves the problem that a number of callers just pass in a NULL
connection today which prevents secret lookup working at all.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-01-31 18:34:48 +00:00
Daniel P. Berrangé
b8eb6194da storage: allow opening with storage:///system and storage:///session URIs
Allow the possibility of opening a connection to only the storage
driver, by defining storage:///system and storage:///session URIs
and registering a fake hypervisor driver that supports them.

The hypervisor drivers can now directly open a storage driver
connection at time of need, instead of having to pass around a
virConnectPtr through many functions. This will facilitate the later
change to support separate daemons for each driver.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-01-31 17:44:43 +00:00
Daniel P. Berrangé
d8924a6408 storage: move driver registration back to end of the file
By convention the last thing in the driver.c files should be the driver
callback table and function to register it.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-01-31 17:40:32 +00:00
Peter Krempa
b2c5f28a72 storage: util: Properly ignore errors when backing volume is inaccessible
Commit 000e950455 tried to fix improper bracketing when refreshing disk
volume stats for a backing volume. Unfortunately the condition is still
wrong as in cases as the backing store being inaccessible
storageBackendUpdateVolTargetInfo returns -2 if instructed to ignore
errors. The condition does not take this into account.

Dumping XML of a volume which has inacessible backing store would then
result into:

 # virsh vol-dumpxml http.img --pool default
 error: An error occurred, but the cause is unknown

Properly ignore -2 for backing volumes.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1540022
2018-01-30 11:41:59 +01:00
John Ferlan
71d80c9726 storage: Modify virStorageBackendDiskMakeDataVol logic
Alter the logic such that we only add the volume to the pool once
we've filled in all the information and cause failure to go to a
common error: label. Patches to place the @vol into a few hash tables
will soon "require" that at least the keys (name, target.path, and key)
be populated with valid data.
2018-01-10 08:10:23 -05:00
John Ferlan
ec24d2905b storage: When delete volume avoid disk backend removal
For a disk backend, the deleteVol code will clear all the
volumes in the pool and perform a pool refresh, thus the
storageVolDeleteInternal should not use access @voldef
after deleteVol succeeds.
2018-01-10 08:10:23 -05:00
John Ferlan
607524af4f conf: Use bool for @check_active parameter
Use a bool as that's how the variable is used in the function.
2018-01-04 10:54:08 -05:00
Julio Faracco
d0307a9a76 storage: Fixing missing 'backingStore' tag from volume XML dumps.
After commit a693fdb 'vol-dumpxml' missed the ability to show backingStore
information. This commit adds a volume type for files that fixes this
problem.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1529663

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
2018-01-03 16:04:24 +01:00
John Ferlan
052ecaee3a storage: Reduce need for using storageDriverLock
Now that the storage pools are self locking, we can reduce the number
of places that need to take the big hammer storage driver lock
2017-12-13 15:06:42 -05:00
John Ferlan
4b2e0ed6e3 storage: Convert virStoragePoolObjList to use virObjectRWLockable
Now that we have a private storage pool list, we can take the next
step and convert to using objects. In this case, we're going to use
RWLockable objects (just like every other driver) with two hash
tables for lookup by UUID or Name.

Along the way the ForEach and Search API's will be adjusted to use
the related Hash API's and the various FindBy functions altered and
augmented to allow for HashLookup w/ and w/o the pool lock already
taken.

After virStoragePoolObjRemove we will need to virObjectUnref(obj)
after to indicate the caller is "done" with it's reference. The
Unlock occurs during the Remove.

The NumOf, GetNames, and Export functions all have their own callback
functions to return the required data and the FindDuplicate code
can use the HashSearch function callbacks.
2017-12-13 15:05:10 -05:00
John Ferlan
90e65353a2 storage: Privatize virStoragePoolObjListPtr
Move the structure into virstorageobj.c.

Use the virStoragePoolObjListNew allocator to fill in the @pools for
the storage driver and test driver.
2017-12-13 14:51:30 -05:00
John Ferlan
d4f80d439e storage: Fix path check in storagePoolLookupByTargetPath
Commit id '5ab746b8' introduced the function as perhaps a copy
of storageVolLookupByPath; however, it did not use the @cleanpath
variable even though it used the virFileSanitizePath. So in essance
the only "check" being done for failure is whether it was possible
to strdup the path.

Looking at the virStoragePoolDefParseXML one will note that the
target.path is stored using the result of virFileSanitizePath.
Therefore, this function should sanitize and use the input @path
for the argument to storagePoolLookupByTargetPathCallback which
is comparing against stored target.path values.

Additionally, if there was an error we should use the proper error
of VIR_ERR_NO_STORAGE_POOL (instead of VIR_ERR_NO_STORAGE_VOL).
2017-12-13 14:51:30 -05:00
Peter Krempa
2d07f1f0eb storage: Don't dereference driver object if virStorageSource is not initialized
virStorageFileReportBrokenChain uses data from the driver private data
pointer to print the user and group. This would lead to a crash in call
paths where we did not initialize the storage backend as recently added
in commit 24e47ee2b9 to qemuDomainDetermineDiskChain.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1522682
2017-12-07 13:03:09 +01:00
John Ferlan
6caf67f834 storage: Fix broken storage_driver build
Commit id '5d5c732d7' had an incorrect assignment and was found
by travis build:

storage/storage_driver.c:1668:14: error: equality comparison with extraneous
      parentheses [-Werror,-Wparentheses-equality]
    if ((obj == virStoragePoolObjListSearch(&driver->pools,

         ~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2017-11-24 09:42:07 -05:00
John Ferlan
dcb5d8bb13 storage: Convert virStoragePoolObj into virObjectLockable
Now that we're moved the object into virstorageobj, let's make the
code use the lockable object.
2017-11-24 08:08:36 -05:00
John Ferlan
5d5c732d74 storage: Introduce virStoragePoolObjListSearch
Create an API to search through the storage pool objects looking for
a specific truism from a callback API in order to return the specific
storage pool object that is desired.
2017-11-24 08:08:32 -05:00
John Ferlan
d0258dd9d6 storage: Introduce virStoragePoolObjListForEach
Create an API to walk the pools->objs[] list in order to perform a
callback function for each element of the objs array that doesn't care
about whether the action succeeds or fails as the desire is to run the
code over every element in the array rather than fail as soon as or if
one fails.
2017-11-24 07:26:06 -05:00
John Ferlan
770aa08e48 storage: Introduce virStoragePoolObjEndAPI
For now it'll just call the virStoragePoolObjUnlock, but a future
adjustment will do something different. Since the new API will check
for a NULL object before the Unlock call, callers no longer need to
check for NULL before calling.

The virStoragePoolObjUnlock is now private/static to virstorageobj.c
with a short term forward reference.
2017-11-24 07:26:06 -05:00
John Ferlan
2dd70901db storage: Resolve storage driver crash
Resolve a storage driver crash as a result of a long running
storageVolCreateXML when the virStorageVolPoolRefreshThread is
run as a result of when a storageVolUpload completed and ran the
virStoragePoolObjClearVols without checking if the creation
code was currently processing a buildVol after incrementing
the driver->asyncjob count.

The refreshThread will now check the pool asyncjob count before
attempting to pursue the pool refresh. Adjust the documentation
to describe the condition.

Crash from valgrind is as follows (with a bit of editing):

==21309== Invalid read of size 8
==21309==    at 0x153E47AF: storageBackendUpdateVolTargetInfo
==21309==    by 0x153E4C30: virStorageBackendUpdateVolInfo
==21309==    by 0x153E52DE: virStorageBackendVolRefreshLocal
==21309==    by 0x153DE29E: storageVolCreateXML
==21309==    by 0x562035B: virStorageVolCreateXML
==21309==    by 0x147366: remoteDispatchStorageVolCreateXML
...
==21309==  Address 0x2590a720 is 64 bytes inside a block of size 336 free'd
==21309==    at 0x4C2F2BB: free
==21309==    by 0x54CB9FA: virFree
==21309==    by 0x55BC800: virStorageVolDefFree
==21309==    by 0x55BF1D8: virStoragePoolObjClearVols
==21309==    by 0x153D967E: virStorageVolPoolRefreshThread
...
==21309==  Block was alloc'd at
==21309==    at 0x4C300A5: calloc
==21309==    by 0x54CB483: virAlloc
==21309==    by 0x55BDC1F: virStorageVolDefParseXML
==21309==    by 0x55BDC1F: virStorageVolDefParseNode
==21309==    by 0x55BE5A4: virStorageVolDefParse
==21309==    by 0x153DDFF1: storageVolCreateXML
==21309==    by 0x562035B: virStorageVolCreateXML
==21309==    by 0x147366: remoteDispatchStorageVolCreateXML
...
2017-11-16 11:34:26 -05:00
John Ferlan
ba6575e2f6 storage: Use virStoragePoolObjGetDef accessor for new driver events
Missed from merge from commit id 'b0652192' into commit id 'bfcd8fc92'
were a couple of obj->def-> references.
2017-11-07 14:12:52 -05:00
John Ferlan
121cf746b2 storage: Use virStoragePoolObjGetDef accessor for ZFS backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
159f691fae storage: Use virStoragePoolObjGetDef accessor for VSTORAGE backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
d823466bc2 storage: Use virStoragePoolObjGetDef accessor for SCSI backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
4d95a14faf storage: Use virStoragePoolObjGetDef accessor for RBD backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
1d995fbebc storage: Use virStoragePoolObjGetDef accessor for MPATH backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
John Ferlan
dc48de256e storage: Use virStoragePoolObjGetDef accessor for iSCSI backend
In preparation for privatizing the object, use the accessor.
2017-11-07 14:12:52 -05:00
Andrea Bolognani
3e7db8d3e8 Remove backslash alignment attempts
Right-aligning backslashes when defining macros or using complex
commands in Makefiles looks cute, but as soon as any changes is
required to the code you end up with either distractingly broken
alignment or unnecessarily big diffs where most of the changes
are just pushing all backslashes a few characters to one side.

Generated using

  $ git grep -El '[[:blank:]][[:blank:]]\\$' | \
    grep -E '*\.([chx]|am|mk)$$' | \
    while read f; do \
      sed -Ei 's/[[:blank:]]*[[:blank:]]\\$/ \\/g' "$f"; \
    done

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-11-03 13:24:12 +01:00
Peter Krempa
b4daf6af9a storage: Extract error reporting for broken chains
Simplify reporting the error if backing chain is broken for further
callers by extracting it into a separate function.
2017-11-03 09:15:41 +01:00
Peter Krempa
a93d750a7e storage: Add feature check for storage file backend supporting access check
When the user provides backing chain, we don't need the full support for
traversing the backing chain. This patch adds a feature check for the
virStorageSourceAccess API.
2017-11-03 09:15:41 +01:00
Peter Krempa
8c6b6684a8 storage: Extract common code to retrieve driver backend for support check
The 'file access' module of the storage driver has few feature checks to
determine whether libvirt supports given storage driver method. The code
to retrieve the driver struct needed for the check is the same so it can
be extracted.
2017-11-03 09:15:41 +01:00
John Ferlan
2518fd3b6a storage: Allow creation of a LUKS using logical volume
https://bugzilla.redhat.com/show_bug.cgi?id=1427049

Use virStorageBackendCreateVolUsingQemuImg to apply the LUKS information
to the logical volume just created.  As part of the processing of the
lvcreate command add 2MB to the capacity to account for the LUKS header
when it's determined that the volume desires to use encryption.
2017-10-27 05:46:35 -04:00
John Ferlan
29c5c7d27e storage: Introduce virStorageBackendCreateVolUsingQemuImg
Create a shim that will allow other backends to make use of qemu-img
functionality to create or possibly modify the volume.
2017-10-27 05:46:35 -04:00
John Ferlan
e32a9ef420 storage: Extract out the LVCREATE
Refactor to extract out the LVCREATE command.  This also removes the
need for the local @created since the error path can now only be reached
after the creation of the logical volume.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-10-27 05:46:35 -04:00
John Ferlan
becb383a63 storage: Properly resize a local volume using LUKS
https://bugzilla.redhat.com/show_bug.cgi?id=1490279

Turns out the virStorageBackendVolResizeLocal did not differentiate
whether the target volume was a LUKS volume or not and just blindly
did the ftruncate() on the target volume.

Follow the volume creation logic (in general) and create a qemu-img
resize command to resize the target volume for LUKS ensuring that
the --object secret is provided as well as the '--image-opts' used
by the qemu-img resize logic to describe the path and secret ensuring
that it's using the luks driver on the volume of course.
2017-10-27 05:34:36 -04:00
John Ferlan
03984ae543 storage: Alter storageBackendCreateQemuImgSecretObject args
Since all that was really needed was a couple of fields and building
the object can be more generic, let's alter the args a bit. This will
be useful shortly for adding the secret object for a volume resize
operation on a luks volume that will need a secret object.
2017-10-27 05:31:19 -04:00
John Ferlan
07731f9917 storage: Add error path for virStorageBackendCreateQemuImgCmdFromVol
Rather than inline the various free's and return NULL, just create
an error label.
2017-10-27 05:31:19 -04:00
John Ferlan
ae94084b76 storage: Alter args to storageBackendResizeQemuImg
Rather than passing just the path, pass the virStorageVolDefPtr as we're
going to need it shortly.

Also fix the order of code and stack variables in the calling function
virStorageBackendVolResizeLocal.
2017-10-27 05:31:19 -04:00
Peter Krempa
a693fdba01 Terminate backing chains explicitly
Express a properly terminated backing chain by putting a
virStorageSource of type VIR_STORAGE_TYPE_NONE in the chain. The newly
used helpers simplify this greatly.

The change fixes a bug as formatting an incomplete backing chain and
parsing it back would end up in expressing a terminated chain since
src->backingStoreRaw was not populated. By relying on the terminator
object this can be now processed appropriately.
2017-10-17 06:19:18 +02:00
Peter Krempa
0a294a8e28 util: storagefile: Add helpers to check presence of backing store
Add helpers that will simplify checking if a backing file is valid or
whether it has backing store. The helper virStorageSourceIsBacking
returns true if the given virStorageSource is a valid backing store
member. virStorageSourceHasBacking returns true if the virStorageSource
has a backing store child.

Adding these functions creates a central points for further refactors.
2017-10-17 06:19:18 +02:00
Peter Krempa
ee4d2df87d storage: Fill in 'type' field for virStorageSource in storage driver
Storage driver uses virStorageSource only partially to store it's
configuration but fully when parsing backing files of storage volumes.
This patch sets the 'type' field to a value other than
VIR_STORAGE_TYPE_NONE so that further patches can add a terminator
element to backing chains without breaking iteration.
2017-10-17 06:19:18 +02:00
Peter Krempa
c6231a6a6c util: storage: Store backing chain index in virStorageSource
The backing store indexes were not bound to the storage sources in any
way. To allow us to bind a given alias to a given storage source we need
to save the index in virStorageSource. The backing store ids are now
generated when detecting the backing chain.

Since we don't re-detect the backing chain after snapshots, the
numbering needs to be fixed there.
2017-10-17 06:19:18 +02:00
John Ferlan
746a18dbe8 storage: Use virStoragePoolObjGetDef accessor for Gluster backend
In preparation for privatizing the object, use the accessor.
2017-10-06 07:09:47 -04:00
John Ferlan
759fbdd00b storage: Use virStoragePoolObjGetDef accessor for FS backend
In preparation for privatizing the object, use the accessor.
2017-10-06 07:09:47 -04:00
John Ferlan
2bbb66c4b7 storage: Use virStoragePoolObjGetDef accessor for Sheepdog backend
In preparation for privatizing the object, use the accessor.
2017-10-06 07:09:47 -04:00
John Ferlan
7920d97ab9 storage: Use virStoragePoolObjGetDef accessor for Logical backend
In preparation for privatizing the object, use the accessor.
2017-10-06 07:09:47 -04:00
John Ferlan
cd3eecd839 storage: Use virStoragePoolObjGetDef accessor for Disk backend
In preparation for privatizing the object, use the accessor.
2017-10-06 07:09:47 -04:00
John Ferlan
c0682800cd storage: Use virStoragePoolObjGetDef accessor for storage_util
In preparation for privatizing the object, use the accessor.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-10-06 07:09:47 -04:00
John Ferlan
bfcd8fc924 storage: Use virStoragePoolObjGetDef accessor for driver
In preparation for privatizing the object, use the accessor to fetch
the obj->def instead of the direct reference.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-10-06 06:53:05 -04:00
Pavel Hrdina
000e950455 storage: Fix incorrect parenthesis placement
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1498528

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-10-04 18:52:54 +02:00
Ján Tomko
3f702f5ab1 virStorageFileResize: fallocate the whole capacity
We have been trying to implement the ALLOCATE flag to mean
"the volume should be fully allocated after the resize".

Since commit b0579ed9 we do not allocate from the existing
capacity, but from the existing allocation value.
However this value is a total of all the allocated bytes,
not an offset.

For a sparsely allocated file:
$ perl -e 'print "x"x8192;' > vol1
$ fallocate -p -o 0 -l 4096 vol1
$ virsh vol-info vol1 default
Capacity:       8.00 KiB
Allocation:     4.00 KiB

Treating allocation as an offset would result in an incompletely
allocated file:
$ virsh vol-resize vol1 --pool default 16384 --allocate
Capacity:       16.00 KiB
Allocation:     12.00 KiB

Call fallocate from zero on the whole requested capacity to fully
allocate the file. After that, the volume is fully allocated
after the resize:
$ virsh vol-resize vol1 --pool default 16384 --allocate
$ virsh vol-info vol1 default
Capacity:       16.00 KiB
Allocation:     16.00 KiB
2017-09-27 14:40:44 +02:00
John Ferlan
2dd024754e util: Move virSecretUsageType to virsecret.h
Move the virSecretUsageType into the util.
2017-09-21 15:46:48 -04:00
Julio Faracco
b06521928c storage: Add new events for *PoolBuild() and *PoolDelete().
This commit adds new events for two methods and operations: *PoolBuild() and
*PoolDelete(). Using the event-test and the commands set below we have the
following outputs:

$ sudo ./event-test
Registering event callbacks
myStoragePoolEventCallback EVENT: Storage pool test Defined 0
myStoragePoolEventCallback EVENT: Storage pool test Created 0
myStoragePoolEventCallback EVENT: Storage pool test Started 0
myStoragePoolEventCallback EVENT: Storage pool test Stopped 0
myStoragePoolEventCallback EVENT: Storage pool test Deleted 0
myStoragePoolEventCallback EVENT: Storage pool test Undefined 0

Another terminal:
$ sudo virsh pool-define test.xml
Pool test defined from test.xml

$ sudo virsh pool-build test
Pool test built

$ sudo virsh pool-start test
Pool test started

$ sudo virsh pool-destroy test
Pool test destroyed

$ sudo virsh pool-delete test
Pool test deleted

$ sudo virsh pool-undefine test
Pool test has been undefined

This commits can be a solution for RHBZ #1475227.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1475227

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-09-20 11:52:56 +02:00
John Ferlan
d085197ce2 storage: Use virStoragePoolObjDefUseNewDef
Use the new accessor API for storage_driver.
2017-09-19 08:31:56 -04:00
John Ferlan
67129bb435 storage: Use virStoragePoolObj{Get|Incr|Decr}Asyncjobs
Use the new accessor APIs for storage_driver.
2017-09-19 08:31:05 -04:00
John Ferlan
ccc8c311b2 storage: Internally represent @autostart as bool
Since it's been used that way anyway, let's just convert it to a bool
and only make the external representation be an int.
2017-09-19 08:30:37 -04:00
John Ferlan
bb15e65af2 storage: Use virStoragePoolObj{Is|Set}Autostart
Use the new accessor APIs for storage_driver and test_driver.
2017-09-19 08:30:19 -04:00
John Ferlan
0147f72741 storage: Use virStoragePoolObj{Is|Set}Active
Use the new accessor APIs for storage_driver, test_driver, and
gluster backend.
2017-09-19 08:30:19 -04:00
John Ferlan
1bd4349671 storage: Use virStoragePoolObjGetAutostartLink
Use the new accessor API for storage_driver.
2017-09-19 08:28:50 -04:00
John Ferlan
8603d848a3 storage: Use virStoragePoolObj{Get|Set}ConfigFile
Use the new accessor APIs for storage_driver and test_driver.
2017-09-19 08:28:50 -04:00
John Ferlan
5bf9b65501 storage: Introduce APIs to search/scan storage pool volumes list
Introduce virStoragePoolObjForEachVolume to scan each volume
calling the passed callback function until all volumes have been
processed in the storage pool volume list, unless the callback
function returns an error.

Introduce virStoragePoolObjSearchVolume to search each volume
calling the passed callback function until it returns true
indicating that the desired volume was found.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-09-19 08:28:50 -04:00
John Ferlan
40630a8e45 storage: Introduce storage volume add, delete, count APIs
Create/use virStoragePoolObjAddVol in order to add volumes onto list.

Create/use virStoragePoolObjRemoveVol in order to remove volumes from list.

Create/use virStoragePoolObjGetVolumesCount to get count of volumes on list.

For the storage driver, the logic alters when the volumes.obj list grows
to after we've fetched the volobj. This is an optimization of sorts, but
also doesn't "needlessly" grow the volumes.objs list and then just decr
the count if the virGetStorageVol fails.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-09-19 08:28:50 -04:00
John Ferlan
239781e03a storage: Adjust expected format for Disk startup processing
https://bugzilla.redhat.com/show_bug.cgi?id=1464313

If a Disk pool was defined/created using XML that either didn't
specify a specific format or specified format type='unknown', then
restarting a pool after an initial disk backend build with overwrite
would fail after a libvirtd restart for a non-autostarted pool.

This is because the persistent pool data is not updated during pool
build w/ overwrite processing to have the VIR_STORAGE_POOL_DISK_DOS
default format.

So in addition to the alteration done during disk build processing,
alter the default expectation for disk startup to be DOS if nothing
has been defined yet. That will either succeed if the pool had been
successfully built previously using the default DOS format or fail
with a message indicating the format is something else that does not
match the expect format 'dos'.
2017-09-12 10:52:06 -04:00
John Ferlan
d16f803d78 storage: Use virStorageBackendRefreshVolTargetUpdate after wipeVol
https://bugzilla.redhat.com/show_bug.cgi?id=1437797

Rather than using refreshVol which essentially only updates the
allocation, capacity, and permissions for the volume, but not
the format which does get updated in a pool refresh - let's use
the same helper that pool refresh uses in order to update the
volume target.
2017-08-30 15:32:13 -04:00
John Ferlan
7c2945b854 storage: Introduce virStorageBackendRefreshVolTargetUpdate
Create a separate function to handle the volume target update
via probe processing.
2017-08-30 15:32:13 -04:00
Martin Kletzander
3401e208ab qemu: Don't mangle the storage format for type='dir'
Our backing probing code handles directory file types properly in
virStorageFileGetMetadataRecurse(), by that I mean it leaves them
alone.  However its caller, the virStorageFileGetMetadata() resets the
type to raw before probing, without even checking the type.  We need
to special-case TYPE_DIR in order to achieve desired results.

Also, in order to properly test this, we need to stop resetting format
of volumes in tests for TYPE_DIR (probably the reason why we didn't
catch that and why the test data didn't need to be modified).

Partially-resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1443434

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-08-29 16:30:04 +02:00
Peter Krempa
5aec02dc37 make: Drop building without driver modules
Driver modules proved to be reliable for a long time. Since support for
not building modules complicates the code and makefiles drop it.

Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2017-07-27 12:00:35 +02:00
John Ferlan
43e6686c7f storage: Disallow usage of the HBA for a fc_host backing
Disallow providing the wwnn/wwpn of the HBA in the adapter XML:

  <adapter type='fc_host' [parent='scsi_hostN'] wwnn='HBA_wwnn'
    wwpn='HBA_wwpn'/>

This should be considered a configuration error since a vHBA
would not be created. In order to use the HBA as the backing the
following XML should be used:

  <adapter type='scsi_host' name='scsi_hostN'/>

So add a check prior to the checkParent call to validate that
the provided wwnn/wwpn resolves to a vHBA and not an HBA.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-26 15:28:06 -04:00
John Ferlan
f7237d63e8 storage: Check if provided parent is vHBA capable
https://bugzilla.redhat.com/show_bug.cgi?id=1458708

If the parent provided for the storage pool adapter is not vHBA
capable, then issue a configuration error even though the provided
wwnn/wwpn were found.

It is a configuration error to provide a mismatched parent to
the wwnn/wwpn. The @parent is optional and is used as a means to
perform duplicate pool source checks.
2017-07-24 12:27:41 -04:00
John Ferlan
214a353c02 storage: Remove @conn from virNodeDeviceCreateVport
It's no longer needed since the checkParent code moved back to
storage_backend_scsi.c

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-24 12:27:41 -04:00
John Ferlan
c4030331c8 storage: Fix existing parent check for vHBA creation
https://bugzilla.redhat.com/show_bug.cgi?id=1472277

Commit id '106930aaa' altered the order of checking for an existing
vHBA (e.g something created via nodedev-create functionality outside
of the storage pool logic) which inadvertantly broke the code to
decide whether to alter/force the fchost->managed field to be 'yes'
because the storage pool will be managing the created vHBA in order
to ensure when the storage pool is destroyed that the vHBA is also
destroyed.

This patch moves the check (and checkParent helper) for an existing
vHBA back into the createVport in storage_backend_scsi. It also
adjusts the checkParent logic to more closely follow the intentions
prior to commit id '79ab0935'. The changes made by commit id '08c0ea16f'
are only necessary to run the virStoragePoolFCRefreshThread when
a vHBA was really created because there's a timing lag such that
the refreshPool call made after a startPool from storagePoolCreate*
wouldn't necessarily find LUNs, but the thread would. For an already
existing vHBA, using the thread is unnecessary since the vHBA already
exists and the lag to configure the LUNs wouldn't exist.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-24 12:27:41 -04:00
Peter Krempa
97ea8da183 virStorageNetHostDef: Turn @port into integer
Currently, @port is type of string. Well, that's overkill and
waste of memory. Port is always an integer. Use it as such.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-07-24 10:55:44 +02:00
John Ferlan
f36f2e463f storage: Fix editing mistake in storagePoolSetAutostart
Commit id '905f1024b' had a rogue editing mistake that inadvertently
dropped a goto cleanup in storagePoolSetAutostart, but Coverity noted it.
2017-07-22 07:11:28 -04:00
John Ferlan
7cc30e0ed7 storage: Alter volume num, name, and export API's to just take obj
Alter the virStoragePoolObjNumOfVolumes, virStoragePoolObjVolumeGetNames,
and virStoragePoolObjVolumeListExport APIs to take a virStoragePoolObjPtr
instead of the &obj->volumes and obj->def.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-21 14:51:47 -04:00
John Ferlan
905f1024bd storage: Use consistent variable names for driver
A virStoragePoolObjPtr will be an 'obj'.

A virStoragePoolPtr will be a 'pool'.

A virStorageVolPtr will be a 'vol'.

A virStorageVolDefPtr will be a 'voldef'.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-21 14:51:47 -04:00
John Ferlan
d062dfd9d9 storage: Fix return value checks for virAsprintf
Use the < 0 rather than == -1 (consistently) for virAsprintf errors.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-07-21 14:51:47 -04:00
ZhiPeng Lu
8c1f25438e mpath: Fix memory leak in virStorageBackendCreateVols
@map_device, allocated by virAsprintf in virStorageBackendCreateVols,
was not freed and leaked.

Signed-off-by: Zhipeng Lu <lu.zhipeng@zte.com.cn>
2017-07-19 16:47:10 +02:00
Peter Krempa
204f373a91 storage: Make virStorageFileReadHeader more universal
Allow specifying offset to read an arbitrary position in the file. This
warrants a rename to virStorageFileRead.
2017-07-11 17:07:04 +02:00
Peter Krempa
9506bd25a3 storage: Split out virStorageSource accessors to separate file
The helper methods for actually accessing the storage objects don't
really belong to the main storage driver implementation file. Split them
out.
2017-07-11 17:07:04 +02:00
John Ferlan
2065499b60 events: Avoid double free possibility on remote call failure
If a remote call fails during event registration (more than likely from
a network failure or remote libvirtd restart timed just right), then when
calling the virObjectEventStateDeregisterID we don't want to call the
registered @freecb function because that breaks our contract that we
would only call it after succesfully returning.  If the @freecb routine
were called, it could result in a double free from properly coded
applications that free their opaque data on failure to register, as seen
in the following details:

    Program terminated with signal 6, Aborted.
    #0  0x00007fc45cba15d7 in raise
    #1  0x00007fc45cba2cc8 in abort
    #2  0x00007fc45cbe12f7 in __libc_message
    #3  0x00007fc45cbe86d3 in _int_free
    #4  0x00007fc45d8d292c in PyDict_Fini
    #5  0x00007fc45d94f46a in Py_Finalize
    #6  0x00007fc45d960735 in Py_Main
    #7  0x00007fc45cb8daf5 in __libc_start_main
    #8  0x0000000000400721 in _start

The double dereference of 'pyobj_cbData' is triggered in the following way:

    (1) libvirt_virConnectDomainEventRegisterAny is invoked.
    (2) the event is successfully added to the event callback list
        (virDomainEventStateRegisterClient in
        remoteConnectDomainEventRegisterAny returns 1 which means ok).
    (3) when function remoteConnectDomainEventRegisterAny is hit,
        network connection disconnected coincidently (or libvirtd is
        restarted) in the context of function 'call' then the connection
        is lost and the function 'call' failed, the branch
        virObjectEventStateDeregisterID is therefore taken.
    (4) 'pyobj_conn' is dereferenced the 1st time in
        libvirt_virConnectDomainEventFreeFunc.
    (5) 'pyobj_cbData' (refered to pyobj_conn) is dereferenced the
         2nd time in libvirt_virConnectDomainEventRegisterAny.
    (6) the double free error is triggered.

Resolve this by adding a @doFreeCb boolean in order to avoid calling the
freeCb in virObjectEventStateDeregisterID for any remote call failure in
a remoteConnect*EventRegister* API. For remoteConnect*EventDeregister* calls,
the passed value would be true indicating they should run the freecb if it
exists; whereas, it's false for the remote call failure path.

Patch based on the investigation and initial patch posted by
fangying <fangying1@huawei.com>.
2017-06-25 08:16:04 -04:00
Peter Krempa
d97cfdc891 storage: Add helper to retrieve the backing store string of a storage volume
It is necessary for some parts of the code to refresh just data
based on the based on the backing store string. Add a convenience
function that will retrieve this data.
2017-06-20 13:25:55 +02:00
Marc Hartmayer
adf846d3c9 Use ATTRIBUTE_FALLTHROUGH
Use ATTRIBUTE_FALLTHROUGH, introduced by commit
5d84f5961b, instead of comments to
indicate that the fall through is an intentional behavior.

Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Reviewed-by: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Reviewed-by: Bjoern Walk <bwalk@linux.vnet.ibm.com>
2017-06-12 19:11:30 -04:00
Michal Privoznik
1f43aa67c5 Introduce virStorageVol{Download,Upload}Flags
These flags to APIs will tell if caller wants to use sparse
stream for storage transfer. At the same time, it's safe to
enable them in storage driver frontend and rely on our backends
checking the flags. This way we can enable specific flags only on
some specific backends, e.g. enable
VIR_STORAGE_VOL_DOWNLOAD_SPARSE_STREAM for filesystem backend but
not iSCSI backend.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
895479647b fdstream: Implement sparse stream
Basically, what is needed here is to introduce new message type
for the messages passed between the event loop callbacks and the
worker thread that does all the I/O. The idea is that instead of
a queue of read buffers we will have a queue where "hole of size
X" messages appear. That way the event loop callbacks can just
check the head of the queue and see if the worker thread is in
data or a hole section and how long the section is.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Serge Hallyn
756ef0c353 storage: use 0711 as the default perms for dirs
There should be no need to make dir based pools world/group readable.
So use 0711, not 0755, as the default perms for storage dirs.

Updates in v2:
 - adapt commit wording to mention dropping group readable as well

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2017-05-15 12:09:24 +01:00
John Ferlan
3c4f2e3fb7 disk: Use virStorageBackendZeroPartitionTable
https://bugzilla.redhat.com/show_bug.cgi?id=1439132

During 'matrix' testing of all possible combinations I found that if
device is formated with "gpt" first, then an attempt is made to format
using "mac", a startup will fail.

Deeper analysis by Peter Krempa indicates that the "mac" table fits
into the first block on the disk. Since the GPT disklabel is stored
at LBA address 1 it is not overwritten at all. Thus it's apparent that
the (blkid) detection tool then prefers GPT over a older disklabel.

The GPT disklabel has also a secondary copy at the last LBA of the disk.

So, follow the same logic as the logical pool in clearing a 1MB swath
at the beginning and end of the device to avoid potential issues with
larger sector sizes for the device.

Also fixed a minor formatting nit in virStorageBackendDeviceIsEmpty call.
2017-04-26 07:28:08 -04:00
John Ferlan
d942bf6e9e logical: Increase the size of the data to wipe
Since a sector size may be larger than 512 bytes, let's just increase
the size to wipe to 1MB rather than 2KB
2017-04-26 07:28:08 -04:00
John Ferlan
c6aa81c65a logical: Use virStorageBackendZeroPartitionTable
Rather than open code it, use the new function which uses the wipe algorithm
in order to zero the front and tail of the partition.
2017-04-26 07:28:08 -04:00
John Ferlan
e8b0212458 storage: Introduce virStorageBackendZeroPartitionTable
Create a wrapper/helper that can be used to call the storage backend
wipe helper - storageBackendVolWipeLocalFile for future use by logical
and disk backends to clear out the partition table rather than having
each open code the same algorithm.
2017-04-26 07:28:08 -04:00
John Ferlan
859a2d162a storage: Modify storageBackendWipeLocal to allow zero from end of device
Add bool 'zero_end' and logic that would allow a caller to wipe specific
portions of a target device either from the beginning (the default) or
from the end when zero_end is true.

This will allow for this code to wipe out partition table information
from a device.
2017-04-26 07:28:08 -04:00
John Ferlan
4a5d191e2a storage: Create helpers to perform FindByUUID and FindByName
Create a couple of helpers that will perform the same call sequence.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-11 08:49:51 -04:00
John Ferlan
babf148a94 storage: Pass driver arg by ref
Alter virStoragePoolObjListExport in order to pass the drivers->pools
by reference

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-11 08:49:51 -04:00
John Ferlan
50e6d4e8e1 storage: Introduce virStoragePoolObjGetNames
Mostly code motion to move storageConnectList[Defined]StoragePools
and similar test driver code into virstorageobj.c and rename to
virStoragePoolObjGetNames.

Also includes a couple of variable name adjustments to keep code consistent
with other drivers.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-11 08:49:51 -04:00
John Ferlan
2fae7c7fb2 storage: Introduce virStoragePoolObjNumOfStoragePools
Unify the NumOf[Defined]StoragePools API into virstorageobj.c from
storage_driver and test_driver.  The only real difference between the
two is the test driver doesn't call using the aclfilter API.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-11 08:49:51 -04:00
John Ferlan
96155c6994 storage: Introduce virStoragePoolObjVolumeListExport
Essentially code motion to move the storage/test driver ListAllVolumes
logic into virstorageobj.c

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-11 08:49:51 -04:00
John Ferlan
7e94830f07 storage: Introduce virStoragePoolObjVolumeGetNames
Mostly code motion to move storagePoolListVolumes code into virstorageobj.c
and rename to virStoragePoolObjVolumeGetNames.

Also includes a couple of variable name adjustments to keep code consistent
with other drivers.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-11 08:49:51 -04:00
John Ferlan
4a440e4366 storage: Introduce virStoragePoolObjNumOfVolumes
Unify the NumOfVolumes API into virstorageobj.c from storage_driver and
test_driver.  The only real difference between the two is the test driver
doesn't call using the aclfilter API.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-11 08:49:51 -04:00
John Ferlan
98f424d503 disk: Force usage of parted when checking disk format for "bsd"
https://bugzilla.redhat.com/show_bug.cgi?id=1439132

Add "bsd" to the list of format types to not checked during blkid
processing even though it supposedly knows the format - for some
(now unknown) reason it's returning partition table not found. So
let's just let PARTED handle "bsd" too.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-07 11:58:36 -04:00
John Ferlan
f2a1232031 disk: Resolve issues with disk partition build/start checks
https://bugzilla.redhat.com/show_bug.cgi?id=1439132

Commit id 'a48c674fb' added a check for format types "dvh" and "pc98"
to use the parted print processing instead of using blkid processing
in order to validate the label on the disk was what is expected for
disk pool startup. However, commit id 'a4cb4a74f' really messed things
up by missing an else condition causing PARTEDFindLabel to always
return DIFFERENT.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-04-07 11:58:36 -04:00
Andrea Bolognani
611ddefc16 storage: Avoid leak in virStorageUtilGlusterExtractPoolSources()
The contents of volname would be leaked if the function were
to be passed an invalid pooltype by the caller.

Make sure the memory is released instead.
2017-04-06 10:03:26 +02:00
Peter Krempa
a200ebbc6f storage: gluster: Implement 'checkPool' method so that state is restored
After restart of libvirtd the 'checkPool' method is supposed to validate
that the pool is online. Since libvirt then refreshes the pool contents
anyways just return whether the pool was supposed to be online so that
the code can be reached. This is necessary since if a pool does not
implement the method it's automatically considered as inactive.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1436065
2017-04-05 12:21:17 +02:00
Peter Krempa
dff04e0af0 storage: gluster: Use volume name as "<name>" field in the XML
For native gluster pools the <dir> field denotes a directory inside the
pool. For the actual pool name the <name> field has to be used.
2017-04-04 16:36:15 +02:00
Peter Krempa
5df6992e1c storage: Fix XPath for looking up gluster volume name
Use the relative lookup specifier rather than the global one. Otherwise
only the first name would be looked up. Add a test case to cover the
scenario.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1436574
2017-04-04 16:36:15 +02:00
Peter Krempa
e238bfa6d4 storage: util: Split out the gluster volume extraction code into new function
To allow testing of the algorithm, split out the extractor into a
separate helper.
2017-04-04 16:30:45 +02:00
Peter Krempa
a92160dbd5 storage: util: Pass pool type to virStorageBackendFindGlusterPoolSources
The native gluster pool source list data differs from the data used for
attaching gluster volumes as netfs pools. Currently the only difference
was the format. Since native pools don't use it and later there will be
more differences add a more deterministic way to switch between the
types instead.
2017-04-04 16:30:45 +02:00