Commit Graph

932 Commits

Author SHA1 Message Date
John Ferlan
5e46d7d6b6 storage: Add support to create a luks volume
Partially resolves:
https://bugzilla.redhat.com/show_bug.cgi?id=1301021

If the volume xml was looking to create a luks volume take the necessary
steps in order to make that happen.

The processing will be:
 1. create a temporary file (virStorageBackendCreateQemuImgSecretPath)
   1a. use the storage driver state dir path that uses the pool and
       volume name as a base.

 2. create a secret object (virStorageBackendCreateQemuImgSecretObject)
   2a. use an alias combinding the volume name and "_luks0"
   2b. add the file to the object

 3. create/add luks options to the commandline (virQEMUBuildLuksOpts)
   3a. at the very least a "key-secret=%s" using the secret object alias
   3b. if found in the XML the various "cipher" and "ivgen" options

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-07-19 09:40:01 -04:00
Olga Krishtal
3dd50be7ca vz: support filesystem type volume
Vz containers are able to use ploop volumes from storage pools
to work upon.

To use filesystem type volume, pool name and volume name should be
specifaed in <source> :
   <filesystem type='volume' accessmode='passthrough'>
      <driver type='ploop' format='ploop'/>
      <source pool='guest_images' volume='TEST_POOL_CT'/>
      <target dir='/'/>
   </filesystem>

The information about pool and volume is stored in ct dom configuration:
<StorageURL>libvirt://localhost/pool_name/vol_name</StorageURL>
and can be easily obtained via PrlVmDevHd_GetStorageURL sdk call.

The only shorcoming: if storage pool is moved somewhere the ct
should be redefined in order to refresh the information aboot path
to root.hdd

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
2016-07-18 23:39:57 +03:00
Olga Krishtal
88c61785b2 storage: dir: adapts .wipeVol for ploop volumes
The modification of .volWipe callback wipes ploop volume using one of
given wiping algorithm: dod, nnsa, etc.
However, in case of ploop volume we need to reinitialize root.hds and DiskDescriptor.xml.

v2:
- added check on ploop tools presens
- virCommandAddArgFormat changed to virCommandAddArg

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
2016-07-12 13:16:13 +02:00
John Ferlan
9bbf0d7e64 encryption: Add luks parsing for storageencryption
Add parse and format of the luks/passphrase secret including tests for
volume XML parsing.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-07-01 15:46:52 -04:00
John Ferlan
47e88b33be util: Add 'usage' for encryption
In order to use more common code and set up for a future type, modify the
encryption secret to allow the "usage" attribute or the "uuid" attribute
to define the secret. The "usage" in the case of a volume secret would be
the path to the volume as dictated by the backwards compatibility brought
on by virStorageGenerateQcowEncryption where it set up the usage field as
the vol->target.path and didn't allow someone to provide it. This carries
into virSecretObjListFindByUsageLocked which takes the secret usage attribute
value from from the domain disk definition and compares it against the
usage type from the secret definition. Since none of the code dealing
with qcow/qcow2 encryption secrets uses usage for lookup, it's a mostly
cosmetic change. The real usage comes in a future path where the encryption
is expanded to be a luks volume and the secret will allow definition of
the usage field.

This code will make use of the virSecretLookup{Parse|Format}Secret common code.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-07-01 15:46:24 -04:00
Michal Privoznik
ca5d51df27 virStorageTranslateDiskSourcePool: Avoid double free
https://bugzilla.redhat.com/show_bug.cgi?id=1316370

Consider the following disk for a domain:

    <disk type='volume' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <auth username='libvirt'>
        <secret type='iscsi' usage='libvirtiscsi'/>
      </auth>
      <source pool='iscsi-secret-pool' volume='unit:0:0:0' mode='direct' startupPolicy='optional'/>
      <target dev='sda' bus='scsi'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>

Now, startupPolicy is currently not allowed for iscsi disks, so
one would expect an error message to be thrown. But what a
surprise is waiting for users if they try to start up such
domain:

==15724== Invalid free() / delete / delete[] / realloc()
==15724==    at 0x4C2B1F0: free (vg_replace_malloc.c:473)
==15724==    by 0x54B7A69: virFree (viralloc.c:582)
==15724==    by 0x552DC90: virStorageAuthDefFree (virstoragefile.c:1549)
==15724==    by 0x552F023: virStorageSourceClear (virstoragefile.c:2055)
==15724==    by 0x552F054: virStorageSourceFree (virstoragefile.c:2067)
==15724==    by 0x55556AA: virDomainDiskDefFree (domain_conf.c:1562)
==15724==    by 0x5557ABE: virDomainDefFree (domain_conf.c:2547)
==15724==    by 0x1B43CC42: qemuProcessStop (qemu_process.c:5918)
==15724==    by 0x1B43BA2E: qemuProcessStart (qemu_process.c:5511)
==15724==    by 0x1B48993E: qemuDomainObjStart (qemu_driver.c:7050)
==15724==    by 0x1B489B9A: qemuDomainCreateWithFlags (qemu_driver.c:7104)
==15724==    by 0x1B489C01: qemuDomainCreate (qemu_driver.c:7122)
==15724==  Address 0x21cfbb90 is 0 bytes inside a block of size 48 free'd
==15724==    at 0x4C2B1F0: free (vg_replace_malloc.c:473)
==15724==    by 0x54B7A69: virFree (viralloc.c:582)
==15724==    by 0x552DC90: virStorageAuthDefFree (virstoragefile.c:1549)
==15724==    by 0x12D1C8D4: virStorageTranslateDiskSourcePool (storage_driver.c:3475)
==15724==    by 0x1B4396E4: qemuProcessPrepareDomain (qemu_process.c:4896)
==15724==    by 0x1B43B880: qemuProcessStart (qemu_process.c:5466)
==15724==    by 0x1B48993E: qemuDomainObjStart (qemu_driver.c:7050)
==15724==    by 0x1B489B9A: qemuDomainCreateWithFlags (qemu_driver.c:7104)
==15724==    by 0x1B489C01: qemuDomainCreate (qemu_driver.c:7122)
==15724==    by 0x561CA97: virDomainCreate (libvirt-domain.c:6787)
==15724==    by 0x12B6FD: remoteDispatchDomainCreate (remote_dispatch.h:4116)
==15724==    by 0x12B61A: remoteDispatchDomainCreateHelper (remote_dispatch.h:4092)

The problem is, in virStorageTranslateDiskSourcePool disk
def->src->auth is freed, but the pointer is not set to NULL. So
later, when qemuProcessStop starts to free the domain definition,
virStorageAuthDefFree() tries to free the memory again, instead
of jumping out immediately.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-06-28 15:02:16 +02:00
John Ferlan
01f4a4a070 storage: Introduce virStoragePoolObjBuildTempFilePath
Create a function to return a temporary file path to be used in a mkostemp
type call using the path to the stateDir + pool->def->name + vol->name

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-24 13:42:38 -04:00
Daniel P. Berrange
0330848207 Promote storage pool refresh lifecycle event to top level event
The VIR_STORAGE_POOL_EVENT_REFRESHED constant does not
reflect any change in the lifecycle of the storage pool.

It should thus not be part of the storage pool lifecycle
event set, but rather be a top level event in its own
right. Thus we introduce VIR_STORAGE_POOL_EVENT_ID_REFRESH
to replace it.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-06-24 18:26:11 +01:00
Ján Tomko
83066f99a9 Fix error detection in virStorageBackendISCSIGetHostNumber
In the unlikely case the iSCSI session path exists, but does not
contain an entry starting with "target", we would silently use
an initialized value.

Rewrite the function to correctly report errors.
2016-06-24 16:30:55 +02:00
Ján Tomko
0f79480b9f Replace some uses STREQLEN with STRPREFIX
Do not call it with a magic constant matching the length
of the pattern.
2016-06-24 16:30:55 +02:00
Ján Tomko
290f2adf46 virStorageBackendISCSIGetHostNumber: correctly use virDirOpen
Incorrect conflict resolution in my commit e81de04c1 broke this.
2016-06-24 14:51:35 +02:00
Ján Tomko
994b024624 Use virDirOpenQuiet
Remove all the remaining usage of opendir.
2016-06-24 14:20:57 +02:00
Ján Tomko
e81de04c10 Use virDirOpen
Switch from opendir to virDirOpen everywhere we need to report an error.
2016-06-24 14:20:57 +02:00
Cole Robinson
bdb868101b storage: Fix coverity warning
After commit e808d3f227 cbdata is always available here, so the
check is pointless
2016-06-24 07:31:47 -04:00
Ján Tomko
dad2f010b0 Do not skip hidden entries when looking for a stable path
The device names are unlikely to start with a dot.
'.' and '..' are already skipped by virDirRead.
2016-06-23 21:58:38 +02:00
Ján Tomko
70a033ab42 Do not ignore hidden files in /sys and /proc
The directories we iterate over are unlikely to contain any entries
starting with a dot, other than '.' and '..' which is already skipped
by virDirRead.
2016-06-23 21:58:38 +02:00
Ján Tomko
852cd39830 Fix comment in virStorageBackendFileSystemRefresh
'.' and '..' are now skipped by virDirRead, there's no need to mention
them in the comment.
2016-06-23 21:58:38 +02:00
Ján Tomko
a4e6f1eb9c Introduce VIR_DIR_CLOSE
Introduce a helper that only calls closedir if DIR* is non-NULL
and sets it to NULL afterwards.
2016-06-23 21:58:33 +02:00
John Ferlan
1eca5f6581 secret: Move virStorageSecretType and rename
Move the enum into a new src/util/virsecret.h, rename it to be
virSecretLookupType. Add a src/util/virsecret.h in order to perform
a couple of simple operations on the secret XML and virSecretLookupTypeDef
for clearing and copying.

This includes quite a bit of collateral damage, but the goal is to remove
the "virStorage*" and replace with the virSecretLookupType so that it's
easier to to add new lookups that aren't necessarily storage pool related.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-23 12:30:27 -04:00
Cole Robinson
e808d3f227 storage: Remove redundant refreshPool check
Every driver provides a refreshPool impl, and many other critical
places in the code unconditionally call it without checking if
it exists, so this check is pointless
2016-06-23 09:29:54 -04:00
John Ferlan
35f6abef6b storage: Use virSecretGetSecretString
Rather than inline code secret lookup for rbd/iscsi, use the common function.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-21 14:31:19 -04:00
John Ferlan
7df631b66a storage: Create helper to set options for CreateQemuImg code
Create a helper virStorageBackendCreateQemuImgSetOptions to set either
the qemu-img -o options or the previous mechanism using -F

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-21 14:31:19 -04:00
John Ferlan
d12a64f310 storage: Create helper to set backing for CreateQemuImg code
Create a helper virStorageBackendCreateQemuImgSetBacking to perform the
backing store set

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-21 14:31:19 -04:00
John Ferlan
f6a92f8e20 storage: Adjust qemu-img switches check
Since we support QEMU 0.12 and later, checking for support of specific flags
added prior to that isn't necessary.

Thus start with the base of having the "-o options" available for the
qemu-img create option and then determine whether we have the compat
option for qcow2 files (which would be necessary up through qemu 2.0
where the default changes to compat 0.11).

Adjust test to no long check for NONE and FLAG options as well was removing
results of tests that would use that option.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-21 14:31:19 -04:00
Jovanka Gulicoska
41b2f108d5 storage: implement storage lifecycle event APIs
Implement storage pool event callbacks for START, STOP, DEFINE, UNDEFINED
and REFRESHED in functions when a storage pool is created/started/stopped
etc. accordingly
2016-06-16 12:22:11 -04:00
John Ferlan
77ad76b615 storage: Create helper to set input for CreateQemuImg code
Create helper virStorageBackendCreateQemuImgSetInput to set the input

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-06 12:15:43 -04:00
John Ferlan
4c6038a35e storage: Split out a helper for encryption checks
Split out a helper from virStorageBackendCreateQemuImgCmdFromVol
to check the encryption - soon a new encryption sheriff will be
patroling and that'll mean all sorts of new checks.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-06 11:41:45 -04:00
John Ferlan
a2a7f7ede8 storage: Split out setting default secret for encryption
Split the qcow setting of encryption secrets into a helper

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-06-06 11:41:45 -04:00
Jovanka Gulicoska
580dbf06a4 storage: Replace VIR_ERROR with standard vir*Error in state driver init
Replace VIR_ERROR with virReportError and virReportSystemError
2016-05-23 15:42:46 -04:00
Ján Tomko
21fdb4fe70 storage: do not clear vols before volume upload
Commit 5e54361c added virStoragePoolObjClearVols before refreshPool
to prevent duplicate volume entries.

However it is not needed here because we're not refreshing the pool yet,
just checking for the existence of the refresh callback.

The actual refresh is done via virStorageVolFDStreamCloseCb
in virStorageVolPoolRefreshThread, which already calls
virStoragePoolObjClearVols.
2016-05-23 10:47:32 +02:00
Jovanka Gulicoska
b29e08dbe3 More usage of virGetLastErrorMessage
Convert to virGetLastErrorMessage() in the rest of the code
2016-05-19 15:17:03 -04:00
John Ferlan
027986f5bf iscsi: Remove initiatoriqn from virISCSIScanTargets
No longer necessary to have it, so remove it.
2016-05-18 08:29:24 -04:00
John Ferlan
8b10494733 util: Add exitstatus parameter to virCommandRunRegex
Rather than have virCommandRun just spit out the error, allow callers
to decide to pass the exitstatus so the caller can make intelligent
decisions based on the error.
2016-05-18 08:29:24 -04:00
Peter Krempa
cb2e3e50ee util: string: Introduce virStringEncodeBase64
Add a new helper that sanitizes error semantics of base64_encode_alloc.
2016-05-16 12:58:48 +02:00
John Ferlan
8cdff0b93f storage: Fix virStorageBackendDiskDeleteVol for device mapper
Commit id 'df1011ca8' modified virStorageBackendDiskDeleteVol to use
"dmsetup remove --force" to remove the volume, but left things in an
inconsistent state since the partition still existed on the disk and
only the device mapper device (/dev/dm-#) was removed.

Prior to commit '1895b421' (or '1ffd82bb' and '471e1c4e'), this could
go unnoticed since virStorageBackendDiskRefreshPool wasn't called.
However, the pool would be unusable since the /dev/dm-# device would
be removed even though the partition was not removed unless a multipathd
restart reset the link. That would of course make the volume appear again
in the pool after a refresh or pool start after libvirt reload.

This patch removes the 'dmsetup' logic and re-implements the partition
deletion logic for device mapper devices. The removal of the partition
via 'parted rm --script #' will cause udev device change logic to allow
multipathd to handle removing the dm-* device associated with the partition.
2016-05-11 09:23:31 -04:00
John Ferlan
e7bde8d319 storage: Fix algorithm generating path names for devmapper
https://bugzilla.redhat.com/show_bug.cgi?id=1265694

Commit id '020135dc' didn't quite get the algorithm correct when a
device mapper source ended with a non numeric value (e.g. ends with
an alphabet value).

This patch modifies the 'part_separator' logic to add the "p" separator
to the attempted target path name only when specified as part_separator='yes'.

For a source name that already ends with a number, the logic doesn't change
as the part separator would need to be there.

For a source name that ends with something other than a number, this allows
the possibility that a "p" separator can be added. The default for one of
these source devices is to not add the separator.

The key for device mapper and the need for a partition separator "p" is
the presence of a number in the last character of the device name link
in /dev/mapper.  A name such as "/dev/mapper/mpatha1" would generate
a "/dev/mapper/mpatha1p1" partition, while "/dev/mapper/mpatha" would
generate partition "/dev/mapper/mpatha1". Similarly for a device
mapper entry not using friendly names or an alias, a device such as
"/dev/mapper/3600a0b80005b10ca00005ad656fd8d93" would generate a
paritition "/dev/mapper/3600a0b80005b10ca00005ad656fd8d93p1", while
a device such as "/dev/mapper/3600a0b80005b10ca00005e115729093f" would
generate a partition "/dev/mapper/3600a0b80005b10ca00005e115729093f1".
The long number is the WWID of the device. It's also possible to assign
an alias for a device mapper entry, that alias follows the same rules
with respect to ending with a number or not when adding a "p" to create
the target device path.
2016-05-11 09:23:31 -04:00
John Ferlan
5e54361c9d storage: Need to clear pool prior to calling the refreshPool
Prior to calling the 'refreshPool' during CreatePool or UploadPool
operations, we need to clear the pool; otherwise, the pool will
have duplicated entries.
2016-05-11 09:23:31 -04:00
John Ferlan
2c52ec43aa storage: Fix regression cloning volume into a logical pool
https://bugzilla.redhat.com/show_bug.cgi?id=1318993

Commit id 'dd519a294' caused a regression cloning a volume into a
logical pool by removing just the 'allocation' adjustment during
storageVolCreateXMLFrom. Combined with the change to not require the
new volume input XML to have a capacity listed (commit id 'e3f1d2a8')
left the possibility that a zero allocation value (e.g., not provided)
would create a thin/sparse logical volume. When a thin lv becomes fully
populated, then LVM sets the partition 'inactive' and the subsequent
fdatasync() fails.

Add a new 'has_allocation' flag to be set at XML parse time to indicate
that allocation was provided. This is done so that if it's not provided
the create-from code uses the capacity value since we document that if
omitted, the volume will be fully allocated at time of creation.

For a logical backend, that creation time is 'createVol', while for a
file backend, creation doesn't set the size, but the 'createRaw' called
during buildVolFrom will decide whether the file is sparse or not based
on the provided capacity and allocation value.

For volume clones that provide different allocation and capacity values
to allow for sparse files, there is no change.
2016-05-11 09:06:26 -04:00
Martin Kletzander
c36b1f7b6a Change virDevicePCIAddress to virPCIDeviceAddress
We had both and the only difference was that the latter also included
information about multifunction setting.  The problem with that was that
we couldn't use functions made for only one of the structs (e.g.
parsing).  To consolidate those two structs, use the one in virpci.h,
include that in domain_conf.h and add the multifunction member in it.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-05-02 15:46:23 +02:00
Yuri Chornoivan
50fc4b4bdd Fix minor typos in messages
Signed-off-by: Yuri Chornoivan <yurchor@ukr.net>
2016-04-30 15:37:31 +02:00
John Ferlan
662bf30c0f secret: Change virSecretDef variable names
Change 'ephemeral' to 'isephemeral' and 'private' to 'isprivate' since
both are bools.
2016-04-25 15:45:29 -04:00
Cole Robinson
272c622475 storage: drop the plumbing needed for kvm-img/qcow-create
Remove all the plumbing needed for the different qcow-create/kvm-img
non-raw file creation.

We can drop the error messages because CreateQemuImg will thrown an
error for us but with slightly less fidelity (unable to find qemu-img),
which I think is acceptable given the unlikeliness of that error in
practice.
2016-04-20 08:59:57 -04:00
Cole Robinson
487d211d20 storage: remove support for /usr/bin/kvm-img
This an ubuntu/debian packaging convention. At one point it may have
been an actually different binary, but at least as of ubuntu precise
(the oldest supported ubuntu distro, released april 2012) kvm-img is
just a symlink to qemu-img for back compat.

I think it's safe to drop support for it
2016-04-20 08:55:36 -04:00
Cole Robinson
1196fed2e3 storage: remove support for /usr/bin/qcow-create
qcow-create was a crippled qemu-img impl that shipped with xen. I
think supporting this was only relevant for really old distros
that didn't have a proper qemu package, like early RHEL5. I think
it's fair to drop support
2016-04-20 08:55:36 -04:00
Richard Laager
c81bba4f6f ZFS: Support sparse volumes
By default, `zfs create -V ...` reserves space for the entire volsize,
plus some extra (which attempts to account for overhead).

If `zfs create -s -V ...` is used instead, zvols are (fully) sparse.

A middle ground (partial allocation) can be achieved with
`zfs create -s -o refreservation=... -V ...`.  Both libvirt and ZFS
support this approach, so the ZFS storage backend should support it.

Signed-off-by: Richard Laager <rlaager@wiktel.com>
2016-04-17 07:32:27 +03:00
Jiri Denemark
00307b5d82 ploop: Fix build with gluster
Recent patches addiing support for ploop volumes did not properly update
gluster backend.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
2016-04-15 18:09:18 +02:00
Olga Krishtal
03e750f35d storage: dir: adapt .uploadVol .dowloadVol for ploop volume
In case of ploop volume, target path of the volume is the path to the
directory that contains image file named root.hds and DiskDescriptor.xml.
While using uploadVol and downloadVol callbacks we need to open root.hds
itself.
Upload or download operations with ploop volume are only allowed when
images do not have snapshots. Otherwise operation fails.

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2016-04-15 17:27:32 +02:00
Olga Krishtal
ea94be4703 storage: dir: adapt .refreshVol and .refreshPool for ploop volumes
Refreshes meta-information such as allocation, capacity, format, etc.
Ploop volumes differ from other volume types. Path to volume is the path
to directory with image file root.hds and DiskDescriptor.xml.
https://openvz.org/Ploop/format
Due to this fact, operations of opening the volume have to be done once
again. get the information.

To decide whether the given volume is ploops one, it is necessary to check
the presence of root.hds and DiskDescriptor.xml files in volumes' directory.
Only in this case the volume can be manipulated as the ploops one.
Such strategy helps us to resolve problems that might occure, when we
upload some other volume type from ploop source.

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2016-04-15 17:27:32 +02:00
Olga Krishtal
0927fb3ea8 storage: dir: .wipeVol is left unsupported for ploop volume
Returns error in case of vol-wipe cmd for a ploop volume

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
2016-04-15 17:27:32 +02:00
Olga Krishtal
d957ba8deb storage: dir: .resizeVol callback for ploop volume
Changes the size of given ploop volume via ploop resize tool.

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
2016-04-15 17:27:32 +02:00
Olga Krishtal
02d1e45654 storage: dir: .deleteVol callback for ploop volume
Recursively deletes whole directory of a ploop volume.
To delete ploop image it has to be unmounted.

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
2016-04-15 17:27:32 +02:00
Olga Krishtal
cff2138b71 storage: dir: .buildVol and .buildVolFrom callbacks for ploop
These callbacks let us to create ploop volumes in dir, fs and etc. pools.
If a ploop volume was created via buildVol callback, then this volume
is an empty ploop device with DiskDescriptor.xml.
If the volume was created via .buildFrom - then its content is similar to
input volume content.

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2016-04-15 17:27:32 +02:00
Olga Krishtal
ee36975597 storage: add ploop volume type
Ploop image consists of directory with two files: ploop image itself,
called root.hds and DiskDescriptor.xml that contains information about
ploop device: https://openvz.org/Ploop/format.
Such volume are difficult to manipulate in terms of existing volume types
because they are neither a single files nor a directory.
This patch introduces new volume type - ploop. This volume type is used
by ploop volume's exclusively.

Signed-off-by: Olga Krishtal <okrishtal@virtuozzo.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2016-04-15 17:27:32 +02:00
Cole Robinson
e7db227810 util: Add virGettextInitialize, convert the code
Take setlocale/gettext error handling pattern from tools/virsh-*
and use it for all standalone binaries via a new shared
virGettextInitialize routine. The virsh* pattern differed slightly
from other callers. All users now consistently:

* Ignore setlocale errors. virsh has done this forever, presumably for
  good reason. This has been partially responsible for some bug reports:

  https://bugzilla.redhat.com/show_bug.cgi?id=1312688
  https://bugzilla.redhat.com/show_bug.cgi?id=1026514
  https://bugzilla.redhat.com/show_bug.cgi?id=1016158

* Report the failed function name
* Report strerror
2016-04-14 13:22:40 -04:00
Cole Robinson
8f8c0feb11 storage: mpath: Don't error on target_type=NULL
We use device-mapper to enumerate all dm devices, and filter out
the list of multipath devices by checking the target_type string
name. The code however cancels all scanning if we encounter
target_type=NULL

I don't know how to reproduce that situation, but a user was hitting
it in their setup, and inspecting the lvm2/device-mapper code shows
many places where !target_type is explicitly ignored and processing
continues on to the next device. So I think we should do the same

https://bugzilla.redhat.com/show_bug.cgi?id=1069317
2016-04-14 12:52:45 -04:00
Martin Kletzander
fb6ec0ed3d Fix various shadowed declarations
I tried compiling libvirt with older gcc and probably because I used
different configure options I got some shadowed declarations.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-04-14 08:55:08 +02:00
Nitesh Konkar
3e19b5d53d storage: Initialize pool size parameters for refresh thread
If the pool creation thread happens to detect the luns in
the scsi target, the size parameters will be calculated as
part of the refreshPool called from storagePoolCreate().

This means the virStoragePoolFCRefreshThread (commit id
'512b874') waiting to run and "refresh" the pool will
essentially double the allocation and capacity values.
A separate refresh would correct the values.

To avoid this, the FCRefreshThread needs to reinitialize
the pool size values prior to calling virStorageBackendSCSIFindLUs
which eventually calls virStorageBackendSCSINewLun and
updates the size values for each volume found.
2016-03-29 07:28:47 -04:00
Peter Krempa
98033a8b94 storage: rbd: Fix build
After the recent commits the build didn't work for me. Fix it by
using size_t as the callback argument is using and the correct
formatter. The attempted fixup to use %llu as a formatter was wrong.
2016-03-29 08:51:33 +02:00
Roman Bogorodskiy
b77cec09db Revert "zfs: Only raw volumes are supported"
This reverts commit bb5f2dc91f.

The "if (vol->target.format != VIR_STORAGE_FILE_RAW)" check in the
createVol backend. This check is bogus because virStorageVolDefParseXML()
in conf/storage_conf.c sets target.format only if volOptions in
virStoragePoolTypeInfo has formatFromString set, and that's not the
case the zfs backend.

So the check always fails and breaks volume creation.
2016-03-27 11:11:04 -04:00
Roman Bogorodskiy
139a319794 Revert "logical: Only raw volumes are supported"
This reverts commit 6682d6219d.

The "if (vol->target.format != VIR_STORAGE_FILE_RAW)" check in the
createVol backend. This check is bogus because virStorageVolDefParseXML()
in conf/storage_conf.c sets target.format only if volOptions in
virStoragePoolTypeInfo has formatFromString set, and that's not the
case the logical backend.

So the check always fails and breaks volume creation.
2016-03-27 11:09:53 -04:00
Christophe Fergeau
7114c5ff25 storage/rbd: Use correct printf-modifier for uint64
%zu is for size_t variables, not uint64 ones. This causes a warning when building on
a 32 bit linux.
2016-03-25 09:04:46 -04:00
Richard Laager
0c7245994f zfs: Only unencrypted volumes are supported 2016-03-21 08:47:05 +03:00
Richard Laager
bb5f2dc91f zfs: Only raw volumes are supported 2016-03-21 08:47:03 +03:00
Richard Laager
6682d6219d logical: Only raw volumes are supported 2016-03-21 08:47:01 +03:00
Richard Laager
98ee86e76c storage: Improve code consistency between backends
This improves the code consistency around freeing vol->target.path in
createVol implementations.
2016-03-21 08:46:57 +03:00
Richard Laager
ed0221d6b3 sheepdog: Use a consistent error message
This also reduces the number of strings to translate.
2016-03-21 08:46:54 +03:00
Richard Laager
e7d5c4e877 rbd: Use proper error type 2016-03-21 08:46:49 +03:00
Michal Privoznik
bde6e002b5 Initialize couple of variables.
While trying to build with -Os couple of compile errors showed
up.

conf/domain_conf.c: In function 'virDomainChrRemove':
conf/domain_conf.c:13666:24: error: 'ret' may be used uninitialized in this function [-Werror=maybe-uninitialized]
     virDomainChrDefPtr ret, **arrPtr = NULL;
                        ^
Compiler fails to see that @ret is used only if set in the loop,
but whatever, there's no harm in initializing the variable.

In vboxAttachDrivesNew and _vboxAttachDrivesOld compiler thinks
that @rc may be used uninitialized. Well, not directly, but maybe
after some optimization. Yet again, no harm in initializing a
variable.

In file included from ./util/virthread.h:26:0,
                 from ./datatypes.h:28,
                 from vbox/vbox_tmpl.c:43,
                 from vbox/vbox_V3_1.c:37:
vbox/vbox_tmpl.c: In function '_vboxAttachDrivesOld':
./util/virerror.h:181:5: error: 'rc' may be used uninitialized in this function [-Werror=maybe-uninitialized]
     virReportErrorHelper(VIR_FROM_THIS, code, __FILE__,              \
     ^
In file included from vbox/vbox_V3_1.c:37:0:
vbox/vbox_tmpl.c:1041:14: note: 'rc' was declared here
     nsresult rc;
              ^
Yet again, one uninitialized variable:

qemu/qemu_driver.c: In function 'qemuDomainBlockCommit':
qemu/qemu_driver.c:17194:9: error: 'baseSource' may be used uninitialized in this function [-Werror=maybe-uninitialized]
         qemuDomainPrepareDiskChainElement(driver, vm, baseSource,
         ^

And another one:

storage/storage_backend_logical.c: In function 'virStorageBackendLogicalMatchPoolSource.isra.2':
storage/storage_backend_logical.c:618:33: error: 'thisSource' may be used uninitialized in this function [-Werror=maybe-uninitialized]
                       thisSource->devices[j].path))
                                 ^

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-03-03 14:39:57 +01:00
John Ferlan
ee67069c73 storage: Fix error path in storagePoolDefineXML
Found by inspection - after calling virStoragePoolObjAssignDef the
pool is part of the driver->pools.objs list and the failure path
for the virStoragePoolObjSaveDef will use virStoragePoolObjRemove
to remove the pool from the objs list which will unlock and free
the pool pointer (as pools->objs[i] during the loop). Since the call
doesn't clear the pool address from the callee, we need to set it
to NULL; otherwise, the virStoragePoolObjUnlock in the cleanup: code
will fail miserably.
2016-02-26 07:23:05 -05:00
John Ferlan
5430ee3aa6 storage: No need to check ret after VIR_APPEND_ELEMENT
Generates a false positive for Coverity, but it turns out there's no need
to check ret == -1 since if VIR_APPEND_ELEMENT is successful, the local
vol pointer is cleared anyway.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-02-25 11:52:49 -05:00
John Ferlan
4e87164306 zfs: Resolve RESOURCE_LEAK
Found by my Coverity checker - virCheckFlags call could return -1, but
not virCommandFree(destroy_cmd).

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-02-25 11:52:49 -05:00
Eric Blake
5a5c2837c8 rbd: fix 32-bit build
%zu is not always synonymous with uint64_t; on 32-bit machines,
size_t is only 32 bits.  Prefer "%lld"/'unsigned long long' when
the variable is under our control, and "%"PRIu64 when we are
stuck with 'uint64_t' from RBD.

Fixes errors such as:

../../src/storage/storage_backend_rbd.c: In function 'virStorageBackendRBDVolWipe':
../../src/storage/storage_backend_rbd.c:1281:15: error: format '%zu' expects argument of type 'size_t', but argument 8 has type 'uint64_t {aka long long unsigned int}' [-Werror=format=]
     VIR_DEBUG("Need to wipe %zu bytes from RBD image %s/%s",
               ^
../../src/util/virlog.h:90:73: note: in definition of macro 'VIR_DEBUG_INT'
     virLogMessage(src, VIR_LOG_DEBUG, filename, linenr, funcname, NULL, __VA_ARGS__)
                                                                         ^
../../src/storage/storage_backend_rbd.c:1281:5: note: in expansion of macro 'VIR_DEBUG'
     VIR_DEBUG("Need to wipe %zu bytes from RBD image %s/%s",
     ^

Signed-off-by: Eric Blake <eblake@redhat.com>
2016-02-23 16:54:35 -07:00
Martin Kletzander
457ff97fa2 Miscellaneous for-loop syntax clean-ups
Checking whether x > 0 before looping over [0..x] items doesn't make
sense and multi-line body must have curly brackets around it.

Best viewed with '-w'.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-02-22 11:29:59 +01:00
Ján Tomko
cdb757c970 Revert "storageVolCreateXMLFrom: Check if backend knows how to createVol"
This reverts commit 611a278fa4.

According to the original commit message, this is dead code:

  It is highly unlikely that a backend will know how to create a
  volume from a different volume (buildVolFrom) and not know how to
  create an empty volume (createVol).
2016-02-17 13:29:41 +01:00
Wido den Hollander
98782f8899 rbd: Use RBD fast-diff for querying actual volume allocation
Since Ceph version Infernalis (9.2.0) the new fast-diff mechanism
of RBD allows for querying actual volume usage.

Prior to this version there was no easy and fast way to query how
much allocation a RBD volume had inside a Ceph cluster.

To use the fast-diff feature it needs to be enabled per RBD image
and is only supported by Ceph cluster running version Infernalis
(9.2.0) or newer.

Without the fast-diff feature enabled libvirt will report an allocation
identical to the image capacity. This is how libvirt behaves currently.

'virsh vol-info rbd/image2' might output for example:

  Name:           image2
  Type:           network
  Capacity:       1,00 GiB
  Allocation:     124,00 MiB

Newly created volumes will have the fast-diff feature enabled if the
backing Ceph cluster supports it.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-02-12 16:02:05 -05:00
Wido den Hollander
ab342e99f6 rbd: rbd_diff_iterate2() is available in librbd since 266
In commit 0b15f920 there is a #ifdef which requires LIBRBD_VERSION_CODE
266 or newer for rbd_diff_iterate2()

rbd_diff_iterate2() is available since 266, so this if-statement should
require anything newer than 265.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-02-12 15:51:37 -05:00
Wido den Hollander
b61871c06f rbd: Add volStorageBackendRBDGetFeatures() for internal calls
As more and more features are added to RBD volumes we will need to
call this method more often.

By moving it into a internal function we can re-use code inside the
storage backend.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-02-12 15:51:37 -05:00
Michal Privoznik
611a278fa4 storageVolCreateXMLFrom: Check if backend knows how to createVol
It is highly unlikely that a backend will know how to create a
volume from a different volume (buildVolFrom) and not know how to
create an empty volume (createVol). But:
1) we call the function without any prior check so if that's the
case we would SIGSEGV immediatelly
2) it's better to be safe than sorry.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-02-12 16:16:58 +01:00
Michal Privoznik
78490acc39 storageVolCreateXML: Swap order of two operations
Firstly, we realloc internal list to hold new item (=volume that
will be potentially created) and then we check whether we
actually know how to create it. If we don't we consume more
memory than we really need for no good reason.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-02-12 16:16:46 +01:00
Ján Tomko
2542eb75bd Clean up usage of 'ret' variable
Do not store the return value of called functions in the same variable
as the (future) return value of the current function.

This makes tracking the origin of the value easier and reduces
the chance of introducing a new point of exit without resetting
the return value back to -1.
2016-02-11 08:05:16 +01:00
Ján Tomko
28e5655de3 Prohibit verbose strcat
Using strcat directly is more readable than passing strlen
of the copied string to strncat.
2016-02-11 08:05:16 +01:00
Michal Privoznik
d1a7102389 virStringListLength: Ensure const correctness
The virStringListLength function does not ever modify the passed
string list. It merely counts the items in it. Make sure that we
reflect this bit in the function header.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>

(crobinso: fix up spacing and squash in sheepdog bit suggested
 by Andrea)
2016-02-09 15:44:58 -05:00
Wido den Hollander
2aed051d0d rbd: Use %zu for uint64_t instead of casting to unsigned long long
This was only used in debugging messages and not in any real code.

Ceph/RBD uses uint64_t for sizes internally and they can be printed
with %zu without any need for casting.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-02-05 14:29:24 -05:00
Wido den Hollander
f4981ebf5d rbd: Code styling cleanup
Through the years the RBD storage pool code hasn't maintained the
same or correct coding standard which applies to libvirt.

This patch doesn't change any logic in the code, it only applies
the proper coding standards to the code where possible without
making large changes.

This way the code style used in this storage pool is consistent
throughout the whole file.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-02-05 14:29:24 -05:00
John Ferlan
7de8b442ff logical: Clarify pieces of lvs regex
Rather than have a unwieldy regex string - split it up into its components
each having it's own #define and then combine in a different #define

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-02-04 10:15:30 -05:00
Roman Bogorodskiy
c94f6d4dff storage: zfs: flexible use of 'volmode' option
There are slight differences in various ZFS implementations.
Specifically, ZFS on FreeBSD requires to set value of 'volmode'
option to 'dev' to expose volumes as raw disk device (that's what
we need) rather than geom provides, for example.

With ZFS on Linux, however, such option is not available and
volumes exposed like we need by default.

To make our implementation more flexible, only pass 'volmode'
when it's supported. Support is checked by parsing usage
information of the 'zfs get' command.
2016-02-04 03:16:50 +03:00
John Ferlan
6ec319b84f logical: Clean up allocation when building regex on the fly
Rather than a loop reallocating space to build the regex, just allocate
it once up front, then if there's more than 1 nextent, append a comma and
another regex_unit string.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-02-02 13:13:05 -05:00
John Ferlan
c6d526f33f logical: Use 'stripes' value for mirror/raid segtype
The 'stripes' value is described as the "Number of stripes or mirrors in
a logical volume". So add "mirror" and anything that starts with "raid"
to the list of segtypes that can have an 'nextents' value greater than one.
Use of raid segtypes (raid1, raid4, raid5*, raid6*, and raid10) is favored
over mirror in more recent lvm code.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-02-02 13:13:01 -05:00
John Ferlan
69267756d0 logical: Use VIR_APPEND_ELEMENT instead of VIR_REALLOC_N
Rather than preallocating a set number of elements, then walking through
the extents and adjusting the specific element in place, use the APPEND
macros to handle that chore.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-02-02 13:12:57 -05:00
John Ferlan
63e15ad5e0 logical: Create helper virStorageBackendLogicalParseVolExtents
Create a helper routine in order to parse any extents information
including the extent size, length, and the device string contained
within the generated 'lvs' output string.

A future patch would then be able to avoid the code more cleanly

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-01-29 14:13:14 -05:00
Wido den Hollander
84678267e4 rbd: Open in Read-Only mode when refreshing a volume
By opening a RBD volume in Read-Only we do not register a
watcher on the header object inside the Ceph cluster.

Refreshing a volume only calls rbd_stat() which is a operation
which does not write to a RBD image.

This allows us to use a cephx user which has no write
permissions if we would want to use the libvirt storage pool
for informational purposes only.

It also saves us a write into the Ceph cluster which should
speed up refreshing a RBD pool.

rbd_open_read_only() is available in all librbd versions which
also support rbd_open().

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-29 14:09:34 -05:00
Wido den Hollander
0b15f92032 rbd: Implement buildVolFrom using RBD cloning
RBD supports cloning by creating a snapshot, protecting it and create
a child image based on that snapshot afterwards.

The RBD storage driver will try to find a snapshot with zero deltas between
the current state of the original volume and the snapshot.

If such a snapshot is found a clone/child image will be created using
the rbd_clone2() function from librbd.

rbd_clone2() is available in librbd since Ceph version Dumpling (0.67) which
dates back to August 2013.

It will use the same features, strip size and stripe count as the parent image.

This implementation will only create a single snapshot on the parent image if
never changes. This reduces the amount of snapshots created for that RBD image
which benefits the performance of the Ceph cluster.

During build the decision will be made to use either rbd_diff_iterate() or
rbd_diff_iterate2().

The latter is faster, but only available on Ceph versions after 0.94 (Hammer).

Cloning is only supported if RBD format 2 is used. All images created by libvirt
are already format 2.

If a RBD format 1 image is used as the original volume the backend will report
a VIR_ERR_OPERATION_UNSUPPORTED error.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-29 11:11:51 -05:00
Wido den Hollander
34872ca461 rbd: Add support for wiping RBD volumes using TRIM.
Using VIR_STORAGE_VOL_WIPE_ALG_TRIM a RBD volume can be trimmed down
to 0 bytes using rbd_discard()

Effectively all the data on the volume will be lost/gone, but the volume
remains available for use afterwards.

Starting at offset 0 the storage pool will call rbd_discard() in stripe
size * count increments which is usually 4MB. Stripe size being 4MB and
count 1.

rbd_discard() is available since Ceph version Dumpling (0.67) which dates
back to August 2013.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-29 11:11:32 -05:00
Wido den Hollander
63cdc92f04 storage: Add TRIM algorithm to storage volume API
This new algorithm adds support for wiping volumes using TRIM.

It does not overwrite all the data in a volume, but it tells the
backing storage pool/driver that all bytes in a volume can be
discarded.

It depends on the backing storage pool how this is handled.

A SCSI backend might send UNMAP commands to remove all data present
on a LUN.

A Ceph backend might use rbd_discard() to instruct the Ceph cluster
that all data on that RBD volume can be discarded.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-29 11:09:14 -05:00
Wido den Hollander
f226ecbfbb rbd: Add support for wiping RBD volumes
When wiping the RBD image will be filled with zeros started
at offset 0 and until the end of the volume.

This will result in the RBD volume growing to it's full allocation
on the Ceph cluster. All data on the volume will be overwritten
however, making it unavailable.

It does NOT take any RBD snapshots into account. The original data
might still be in a snapshot of that RBD volume.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-29 10:42:36 -05:00
Wido den Hollander
69535c6124 storage: Adjust fix virStorageBackendVolWipeLocal switch
Use the cast of (virStorageVolWipeAlgorithm) adding the missing case:'s
(VIR_STORAGE_VOL_WIPE_ALG_ZERO and VIR_STORAGE_VOL_WIPE_ALG_LAST).

Additionally, the old code would also still run the SCRUB command on
default since it didn't go to cleanup when a invalid flag was supplied.
We now go to cleanup and exit if a invalid flag would be provided.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-29 10:24:20 -05:00
John Ferlan
680030c42b logical: Fix comment examples for virStorageBackendLogicalFindLVs
When commit id '82c1740a' made changes to the output format (changing from
using a ',' separator to '#'), the examples in the lvs output from the
comments weren't changed.

Additionally, the two new fields added ('segtype' and 'stripes') were
not included in the output, leaving it well confusing.

This patch fixes the sample output, adds a 'striped' example, and makes
other comment related adjustments for long line and spacing between followup
'NB' remarks (while I'm there).

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-01-28 16:50:46 -05:00
John Ferlan
020135dc85 storage: Add new flag for libvirt_parthelper
https://bugzilla.redhat.com/show_bug.cgi?id=1265694

In order to be able to process disk storage pool's using a multipath
device to handle the partitions, libvirt_parthelper will need a way to
not automatically add a partition separator "p" to the generated device
name for each partition found. This is designed to mimic the multipath
features known as 'user_friendly_names' and custom 'alias' name.

If the part_separator attribute is set to "no", then generation of the
multipath partition name will not include the "p" partition separator
unless the source device path name ends with a number. The generated
partition names that get passed back to libvirt are processed in order
to find the device mapper multipath (dm-#) path device.

For example, device path "/dev/mapper/mpatha" would create partitions
"/dev/mapper/mpatha1", "/dev/mapper/mpatha2", etc. instead of
"/dev/mapper/mpathap1", "/dev/mapper/mpathap2", etc. If the device
path ends with a number "/dev/mapper/mpatha1", then the algorithm
to generate names "/dev/mapper/mpatha1p1", "/dev/mapper/mpatha1p2", etc.
would be utilized.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-01-19 13:02:59 -05:00
Wido den Hollander
a5a383adc1 rbd: Set r variable so it can be returned should an error occur
This was reported in bug #1298024 where r would be filled with the
return code of rbd_open().

Should rbd_snap_unprotect() fail for any reason the virReportSystemError
call would return 'Success' since rbd_open() succeeded.

https://bugzilla.redhat.com/show_bug.cgi?id=1298024
Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-18 14:06:24 +01:00
Wido den Hollander
6343018fac rbd: Do not append Ceph monitor port number 6789 if not provided
If no port number was provided for a storage pool libvirt defaults to
port 6789; however, librbd/librados already default to 6789 when no port
number is provided.

In the future Ceph will switch to a new port for the Ceph monitors since
port 6789 is already assigned to a different application by IANA.

Port 6789 is assigned to SMC-HTTPS and Ceph now has port 3300 assigned as
the 'Ceph monitor' port.

In this case it is the best solution to not hardcode any port number into
libvirt and let librados handle the connection.

Only if a user specifies a different port number we pass it down to librados,
otherwise we leave it blank.

Signed-off-by: Wido den Hollander <wido@widodh.nl>

merge
2016-01-06 08:13:50 -05:00
Wido den Hollander
f46d137e33 rbd: Do not error out on a single image during pool refresh
It could happen that rbd_list() returns X names, but that while
refreshing the pool one of those RBD images is removed from Ceph
through a different route then libvirt.

We do not need to error out in such case, we can simply ignore the
volume and continue.

  error : volStorageBackendRBDRefreshVolInfo:289 :
    failed to open the RBD image 'vol-998': No such file or directory

It could also be that one or more Placement Groups (PGs) inside Ceph
are inactive due to a system failure.

If that happens it could be that some RBD images can not be refreshed
and a timeout will be raised by librados.

  error : volStorageBackendRBDRefreshVolInfo:289 :
    failed to open the RBD image 'vol-893': Connection timed out

Ignore the error and continue to refresh the rest of the pool's
contents.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-06 07:46:18 -05:00
Wido den Hollander
10028a9d58 rbd: Only close RBD image if it has been opened
It could be that we error out while the RBD image has not been
opened yet. This would cause us to call rbd_close() on pointer
which has not been initialized.

Set it to NULL by default and only close if it is not NULL.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-06 07:46:17 -05:00
John Ferlan
dc77344a8e storage: Clean up error path for create buildPool failure
Commit id 'aeb1078ab' added a buildPool option and failure path which
calls virStoragePoolObjRemove, which unlocks the pool, clears the 'pool'
variable, and goto cleanup.  However, at cleanup virStoragePoolObjUnlock
is called without check if pool is non NULL.
2016-01-05 09:08:02 -05:00
Wido den Hollander
688623b5b0 rbd: Return VIR_STORAGE_FILE_RAW as format for RBD volumes
This used to return 'unkown' and that was not correct.

A vol-dumpxml now returns:

<volume type='network'>
  <name>image3</name>
  <key>libvirt/image3</key>
  <source>
  </source>
  <capacity unit='bytes'>10737418240</capacity>
  <allocation unit='bytes'>10737418240</allocation>
  <target>
    <path>libvirt/image3</path>
    <format type='raw'/>
  </target>
</volume>

The RBD driver will now error out if a different format than RAW
is provided when creating a volume.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2016-01-04 15:19:06 +01:00
Michael Chapman
c494db8fd6 storage: do not leak storage pool XML filename
Valgrind complained:

==28277== 38 bytes in 1 blocks are definitely lost in loss record 298 of 957
==28277==    at 0x4A06A2E: malloc (vg_replace_malloc.c:270)
==28277==    by 0x82D7F57: __vasprintf_chk (in /lib64/libc-2.12.so)
==28277==    by 0x52EF16A: virVasprintfInternal (stdio2.h:199)
==28277==    by 0x52EF25C: virAsprintfInternal (virstring.c:514)
==28277==    by 0x52B1FA9: virFileBuildPath (virfile.c:2831)
==28277==    by 0x19B1947C: storageDriverAutostart (storage_driver.c:191)
==28277==    by 0x19B196A7: storageStateAutoStart (storage_driver.c:307)
==28277==    by 0x538527E: virStateInitialize (libvirt.c:793)
==28277==    by 0x11D7CF: daemonRunStateInit (libvirtd.c:947)
==28277==    by 0x52F4694: virThreadHelper (virthread.c:206)
==28277==    by 0x6E08A50: start_thread (in /lib64/libpthread-2.12.so)
==28277==    by 0x82BE93C: clone (in /lib64/libc-2.12.so)

Signed-off-by: Michael Chapman <mike@very.puzzling.org>
2016-01-04 14:54:23 +01:00
John Ferlan
be783825af storage: Add virCheckFlags to virStorageBackendRBDDeleteVol
The initial commit '74951eade' did not include the proper check for whether
any flags are supported by the driver.

Even though the driver doesn't support VIR_STORAGE_VOL_DELETE_ZEROED,
it still checks and allows the processing to continue

Also add the new VIR_STORAGE_VOL_DELETE_WITH_SNAPSHOTS since it is handled
as of commit id '3c7590e0a'.
2015-12-18 10:51:08 -05:00
John Ferlan
aeb1078ab5 storage: Add flags to allow building pool during create processing
https://bugzilla.redhat.com/show_bug.cgi?id=830056

Add flags handling to the virStoragePoolCreate and virStoragePoolCreateXML
API's which will allow the caller to provide the capability for the storage
pool create API's to also perform a pool build during creation rather than
requiring the additional buildPool step. This will allow transient pools
to be defined, built, and started.

The new flags are:

    * VIR_STORAGE_POOL_CREATE_WITH_BUILD
      Perform buildPool without any flags passed.

    * VIR_STORAGE_POOL_CREATE_WITH_BUILD_OVERWRITE
      Perform buildPool using VIR_STORAGE_POOL_BUILD_OVERWRITE flag.

    * VIR_STORAGE_POOL_CREATE_WITH_BUILD_NO_OVERWRITE
      Perform buildPool using VIR_STORAGE_POOL_BUILD_NO_OVERWRITE flag.

It is up to the backend to handle the processing of build flags. The
overwrite and no-overwrite flags are mutually exclusive.

NB:
This patch is loosely based upon code originally authored by Osier
Yang that were not reviewed and pushed, see:

https://www.redhat.com/archives/libvir-list/2012-July/msg01328.html
2015-12-17 11:56:18 -05:00
John Ferlan
8c865052b9 storage: Fix startup issue for logical pool
Commit id '71b803ac' assumed that the storage pool source device path
was required for a 'logical' pool. This resulted in a failure to start
a pool without any device path defined.

So, adjust the virStorageBackendLogicalMatchPoolSource logic to
return success if at least the pool name matches the vgs output
when no pool source device path is/are provided.
2015-12-17 08:20:22 -05:00
John Ferlan
80ca86e54d storage: Attempt to refresh volume after successful wipe volume
https://bugzilla.redhat.com/show_bug.cgi?id=1270709

When a volume wipe is successful, perform a volume refresh afterwards to
update any volume data that may be used in future volume commands, such as
volume resize.  For a raw file volume, a wipe could truncate the file and
a followup volume resize the capacity may fail because the volume target
allocation isn't updated to reflect the wipe activity.
2015-12-17 07:30:03 -05:00
Ján Tomko
f61770a169 virStorageBackendWipeLocal: remove bytes_wiped argument
It is not used by the caller.
2015-12-17 12:44:35 +01:00
Ján Tomko
c3f7371c5e storage: drop 'Extent' from virStorageBackendWipeExtentLocal
The only caller always passes 0 for the extent start.
Drop the 'extent_start' parameter, as well as the mention of extents
from the function name.

Change off_t extent_length to unsigned long long wipe_len, as well as the
'remain' variable.
2015-12-17 12:44:35 +01:00
Ján Tomko
4bccdf0ceb storage: move buffer allocation inside virStorageBackendWipeExtentLocal
We do not need to pass a zero-filled buffer as an argument,
the function can allocate its own.
2015-12-17 12:44:35 +01:00
Ján Tomko
09cbfc0481 storage: fix return values of virStorageBackendWipeExtentLocal
Return -1:
* on all failures of fdatasync. Instead of propagating -errno
  all the way up to the virStorageVolWipe API, which is documented
  to return 0 or -1.
* after a partial wipe. If safewrite failed, we would re-use the
  non-negative return value of lseek (which should be 0 in this case,
  because that's the only offset we seek to).
2015-12-17 12:44:02 +01:00
John Ferlan
71b803ac9a storage: Add helper to compare logical pool def against pvs output
https://bugzilla.redhat.com/show_bug.cgi?id=1025230

Add a new helper virStorageBackendLogicalMatchPoolSource to compare the
pool's source name against the output from a 'pvs' command to list all
volume group physical volume data on the host.  In addition, compare the
pool's source device list against the particular volume group's device
list to ensure the source device(s) listed for the pool match what the
was listed for the volume group.

Then for pool startup or check API's we need to call this new API in
order to ensure that the pool we're about to start or declare active
during checkPool has a valid definition vs. the running host.
2015-12-15 14:33:05 -05:00
John Ferlan
ae5519f7f8 storage: Create helper for virStorageBackendLogicalFindPoolSources
Rework virStorageBackendLogicalFindPoolSources a bit to create a
helper virStorageBackendLogicalGetPoolSources that will make the
pvs call in order to generate a list of associated pv_name and vg_name's.

A future patch will make use of this for start/check processing to
ensure the storage pool source definition matches expectations.
2015-12-15 14:33:04 -05:00
John Ferlan
dae7007d6e storage: Check FS pool source during virStorageBackendFileSystemIsMounted
https://bugzilla.redhat.com/show_bug.cgi?id=1025230

When determining whether a FS pool is mounted, rather than assuming that
the FS pool is mounted just because the target.path is in the mount list,
let's make sure that the FS pool source matches what is mounted
2015-12-15 14:33:04 -05:00
John Ferlan
61c29fe56f storage: Refactor virStorageBackendFileSystemGetPoolSource
Refactor code to use standard return functioning with respect to setting
a ret value and going to cleanup.
2015-12-15 14:33:04 -05:00
John Ferlan
1d1330f37e storage: Create helper to generate FS pool source value
Refactor the code that builds the pool source string during the FS
storage pool mount to be a separate helper.

A future patch will use the helper in order to validate the mounted
FS matches the pool's expectation during poolCheck processing
2015-12-15 14:33:00 -05:00
Eric Blake
034e47c338 CVE-2015-5313: storage: don't allow '/' in filesystem volume names
The libvirt file system storage driver determines what file to
act on by concatenating the pool location with the volume name.
If a user is able to pick names like "../../../etc/passwd", then
they can escape the bounds of the pool.  For that matter,
virStoragePoolListVolumes() doesn't descend into subdirectories,
so a user really shouldn't use a name with a slash.

Normally, only privileged users can coerce libvirt into creating
or opening existing files using the virStorageVol APIs; and such
users already have full privilege to create any domain XML (so it
is not an escalation of privilege).  But in the case of
fine-grained ACLs, it is feasible that a user can be granted
storage_vol:create but not domain:write, and it violates
assumptions if such a user can abuse libvirt to access files
outside of the storage pool.

Therefore, prevent all use of volume names that contain "/",
whether or not such a name is actually attempting to escape the
pool.

This changes things from:

$ virsh vol-create-as default ../../../../../../etc/haha --capacity 128
Vol ../../../../../../etc/haha created
$ rm /etc/haha

to:

$ virsh vol-create-as default ../../../../../../etc/haha --capacity 128
error: Failed to create vol ../../../../../../etc/haha
error: Requested operation is not valid: volume name '../../../../../../etc/haha' cannot contain '/'

Signed-off-by: Eric Blake <eblake@redhat.com>
2015-12-11 16:34:53 -07:00
John Ferlan
a523770c32 storage: Ignore block devices that fail format detection
https://bugzilla.redhat.com/show_bug.cgi?id=1276198

Prior to commit id '98322052' failure to saferead the block device would
cause an error to be logged and the device to be skipped while attempting
to discover/create a stable target path for a new LUN (NPIV).

This was because virStorageBackendSCSIFindLUs ignored errors from
processLU and virStorageBackendSCSINewLun.

Ignoring the failure allowed a multipath device with an "active" and
"ghost" to be present on the host with the "ghost" block device being
ignored. This patch will return a -2 to the caller indicating the desire
to ignore the block device since it cannot be used directly rather than
fail the pool startup.
2015-12-09 16:31:15 -05:00
John Ferlan
b3df72c4dd storage: Add debug message
I found this useful while processing a volume that wouldn't end up
showing up in the resulting list of block volumes. In this case, the
partition type wasn't found in the disk_types table.
2015-12-09 16:31:14 -05:00
John Ferlan
1bc84b0a08 storage: Handle readflags errors
Similar to the openflags VIR_STORAGE_VOL_OPEN_NOERROR processing, if some
read processing operation fails, check the readflags for the corresponding
error flag being set. If so, rather then causing an error - use VIR_WARN
to flag the error, but return -2 which some callers can use to perform
specific actions. Use a new VIR_STORAGE_VOL_READ_NOERROR flag in a new
VolReadErrorMode enum.
2015-12-09 16:31:14 -05:00
John Ferlan
1edfce9b18 storage: Set ret = -1 on failures in virStorageBackendUpdateVolTargetInfo
While processing the volume for lseek, virFileReadHeaderFD, and
virStorageFileGetMetadataFromBuf - failure would cause an error,
but ret would not be set. That would result in an error message being
sent, but successful status being returned.
2015-12-09 16:31:14 -05:00
John Ferlan
af4028dccd storage: Add comments for backend APIs
Just so it's clearer what to expect upon input and what types of return
values could be generated.  These were loosely copied from existing
virStorageBackendUpdateVolTargetInfoFD.
2015-12-09 16:31:14 -05:00
John Ferlan
22346003dc storage: Add readflags for backend error processing
Similar to the openflags which allow VIR_STORAGE_VOL_OPEN_NOERROR to be
passed to avoid open errors, add a 'readflags' variable so that in the
future read failures could also be ignored.
2015-12-09 16:31:14 -05:00
John Ferlan
5e3ad0b775 storage: Change virStorageBackendVolOpen to use virFileOpenAs
https://bugzilla.redhat.com/show_bug.cgi?id=1282288

Rather than using just open on the path, allow for the possibility that
the path to be opened resides on an NFS root-squash target and was created
under a different uid/gid.

Without using virFileOpenAs an attempt to get the volume size data may fail
if the current user doesn't have permissions to read the volume, such as
would be the case if mode wasn't supplied in the volume XML and the default
VIR_STORAGE_DEFAULT_VOL_PERM_MODE (e.g. 0600) was used. Under this scenario
the owner/group is not root:root, thus this path run under root would fail
to open/read the volume.

NB: The virFileOpenAs code using OPEN_FORK will only work when the failure
is not EACESS/EPERM and the path resolves to a shared file system.
2015-11-20 17:07:13 -05:00
John Ferlan
ce6506b0bc storage: Really fix setting mode for backend exec in NFS root-squash env
https://bugzilla.redhat.com/show_bug.cgi?id=1282288

Although commit id '77346f27' resolves part of the problem regarding creating
a qemu-img image in an NFS root-squash environment, it really didn't fix the
entire problem. Unfortunately it only masked the problem. It seems qemu-img
must open/create the image using 0644, which if used by target.perms would
result in the chmod not being called since the mode desired and set match.

Although qemu-img could conceivably ignore the mode when creating, libvirt
has more knowledge of the environment and can make the adjustment to the
mode far more easily by using virFileOpenAs with VIR_FILE_OPEN_FORCE_MODE.
If that's successful, then we know on return the file will have the right
owner and mode, so we can declare success
2015-11-20 17:07:13 -05:00
John Ferlan
d3fa510a75 storage: Don't assume storage pool exists for FC/SCSI refresh thread
https://bugzilla.redhat.com/show_bug.cgi?id=1277781

The virStoragePoolFCRefreshThread had passed a pointer to the pool obj
in the virStoragePoolFCRefreshInfoPtr; however, we cannot assume that
the pool exists still since we don't keep the pool lock throughout
the duration of the thread.

Therefore, instead of passing the pool obj pointer, pass the UUID of
the pool and perform a lookup.  If found, then we can perform the
refresh using the locked pool obj pointer; otherwise, we just exit
the thread since the pool is now gone.
2015-11-12 06:30:33 -05:00
John Ferlan
c3afa6a9a3 storage: Introduce virStoragePoolObjFindPoolByUUID
Add a new API to search the currently defined pool list for a pool with
a matching UUID and return the locked pool object pointer.
2015-11-12 06:30:32 -05:00
John Ferlan
27432ba70c storage: Change cbdata scsi refresh thread field name
Change the field name from 'name' to 'fchost_name' to better id it.
2015-11-12 06:30:32 -05:00
John Ferlan
0c7a9b994c storage: Make active boolean
Since we treat it like a boolean, let's store it that way. At least one
path had already treated as true/false anyway.
2015-11-12 06:30:32 -05:00
John Ferlan
4cd7d220c9 storage: On 'buildVol' failure don't delete the volume
https://bugzilla.redhat.com/show_bug.cgi?id=1233003

Commit id 'fdda3760' only managed a symptom where it was possible to
create a file in a pool without libvirt's knowledge, so it was reverted.

The real fix is to have all the createVol API's which actually create
a volume (disk, logical, zfs) and the buildVol API's which handle the
real creation of some volume file (fs, rbd, sheepdog) manage deleting
any volume which they create when there is some sort of error in
processing the volume.

This way the onus isn't left up to the storage_driver to determine whether
the buildVol failure was due to some failure as a result of adjustments
made to the volume after creation such as getting sizes, changing ownership,
changing volume protections, etc. or simple a failure in creation.

Without needing to consider that the volume has to be removed, the
buildVol failure path only needs to remove the volume from the pool.
This way if a creation failed due to duplicate name, libvirt wouldn't
remove a volume that it didn't create in the pool target.
2015-11-04 07:21:11 -05:00
John Ferlan
0a6e709c95 Revert "storage: Prior to creating a volume, refresh the pool"
This reverts commit fdda37608a.

This commit only manages a symptom of finding a buildRet failure
where a volume was not listed in the pool, but someone created the
volume outside of libvirt in the pool being managed by libvirt.
2015-11-04 07:21:11 -05:00
John Ferlan
a1703557fd storage: Pull volume removal from pool in storageVolDeleteInternal
Create a helper function to remove volume from the pool.
2015-11-04 07:21:11 -05:00
John Ferlan
2265e7dd14 storage: Cleanup failures in virStorageBackendCreateRaw
After successfully returning from virFileOpenAs, if subsequent calls fail,
then we need to remove the file since our caller expects that failures after
creation will remove the created file.
2015-11-04 07:21:11 -05:00
John Ferlan
9345c2bfcf storage: Cleanup failures virStorageBackendCreateExecCommand
After a successful qemu-img/qcow-create of the backing file, if we
fail to stat the file, change it owner/group, or mode, then the
cleanup path should remove the file.
2015-11-04 07:21:11 -05:00
John Ferlan
77346f2787 storage: Fix setting mode in virStorageBackendCreateExecCommand
Currently the code does not handle the NFS root squash environment
properly since if the file gets created, then the subsequent chmod
will fail in a root squash environment where we're creating a file
in the pool with qemu tools, such as seen via:

   $ virsh vol-create-from $pool $file.xml file.img --inputpool $pool

assuming $file.xml is creating a file of "<format type='qcow2'"> from
an existing file.img in the pool of "<format type='raw'>".

This patch will utilize the virCommandSetUmask when creating the file
in the NETFS pool. The virCommandSetUmask API was added in commit id
'0e1a1a8c4', which was after the original code was developed in commit
id 'e1f27784' to attempt to handle the root squash environment.

Also, rather than blindly attempting to chmod, check to see if the
st_mode bits from the stat match what we're trying to set and only
make the chmod if they don't.

Also, a slight adjustment to the fallback algorithm to move the
virCommandSetUID/virCommandSetGID inside the if (!filecreated) since
they're only useful if we need to attempt to create the file again.
2015-11-04 07:21:11 -05:00
Wido den Hollander
3c7590e0a4 rbd: Remove snapshots if the DELETE_WITH_SNAPSHOTS flag has been provided
When a RBD volume has snapshots it can not be removed.

This patch introduces a new flag to force volume removal,
VIR_STORAGE_VOL_DELETE_WITH_SNAPSHOTS.

With this flag any existing snapshots will be removed prior to
removing the volume.

No existing mechanism in libvirt allowed us to pass such information,
so that's why a new flag was introduced.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-10-27 16:12:12 -04:00
Ishmanpreet Kaur Khera
32cee5b2f0 Avoid using !STREQ and !STRNEQ
We have macros for both positive and negative string matching.
Therefore there is no need to use !STREQ or !STRNEQ. At the same
time as we are dropping this, new syntax-check rule is
introduced to make sure we won't introduce it again.

Signed-off-by: Ishmanpreet Kaur Khera <khera.ishman@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2015-10-21 15:03:35 +02:00
John Ferlan
1059c48180 storage: Rework error paths for virStorageBackendCreateExecCommand
Rework the code in order to use the "ret = -1;" and goto cleanup;
coding style.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2015-10-13 18:03:55 -04:00
John Ferlan
c4dd2a1faf storage: Track successful creation of LV for removal
https://bugzilla.redhat.com/show_bug.cgi?id=1233003

Track when the logical volume was successfully created in order to
properly handle the call to virStorageBackendLogicalDeleteVol. It's
possible that the failure to create was because someone created an
LV in the pool outside of libvirt's knowledge. In this case, we don't
want to delete that LV.  A subsequent or future refresh of the pool
will find the volume and cause an earlier failure

Signed-off-by: John Ferlan <jferlan@redhat.com>
2015-10-13 18:03:55 -04:00
John Ferlan
27d2d99fe7 storage: Fix a resource leak in storageVolCreateXML
Commit id '1b5685da' refactored the code to move buildvoldef inside
the buildVol conditional; however, the VIR_FREE of the memory was
left only when 'buildret' failed, thus we're leaking memory.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2015-10-13 18:03:55 -04:00
John Ferlan
6ab98a68b7 storage: Remove duplicitous refreshVol in Sheepdog buildVol
As of commit id '155ca616' a 'refreshVol' is called after a buildVol
succeeds in storageVolCreateXML, thus a volStorageBackendSheepdogRefreshVolInfo
call in virStorageBackendSheepdogBuildVol is no longer necessary.

Additionally, the 'conn' parameter becomes unused.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2015-10-13 18:03:55 -04:00
John Ferlan
9cb36def82 storage: Remove duplicitous refreshVol in RBD buildVol
As of commit id '155ca616' a 'refreshVol' is called after the buildVol
succeeds in storageVolCreateXML, thus the volStorageBackendRBDRefreshVolInfo
call in virStorageBackendRBDBuildVol is no longer necessary.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2015-10-13 18:03:55 -04:00
John Ferlan
5275c0f4a1 storage: Fix incorrect format for <disk> <auth> XML
https://bugzilla.redhat.com/show_bug.cgi?id=1256999

After creating a copy of the 'authdef' in a pool -> disk translation,
unconditionally clear the 'authType' in the resulting disk auth def
structure since that's used for a storage pool and not a disk.  This
ensures virStorageAuthDefFormat will properly format the <auth> XML
for a <disk> (e.g. it won't have a <auth type='%s'.../>).
2015-10-12 09:46:59 -04:00
John Ferlan
4ee3d4c285 storage: Perform some cleanup of calls
Cleanup calls to virStorageBackendCopyToFD a bit.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2015-10-08 07:06:51 -04:00
John Ferlan
1895b42114 storage: Adjust calculation of alloc/capacity for disk
https://bugzilla.redhat.com/show_bug.cgi?id=1247987

Calculation of the extended and logical partition values for the disk
pool is complex. As the bz points out an extended partition should have
it's allocation initialized to 0 (zero) and keep the capacity as the size
dictated by the extents read.  Then for each logical partition found,
adjust the allocation of the extended partition.

Finally, previous logic tried to avoid recalculating things if a logical
partition was deleted; however, since we now have special logic to handle
the allocation of the extended partition, just make life easier by reading
the partition table again - rather than doing the reverse adjustment.
2015-10-05 08:14:44 -04:00
John Ferlan
657f3bea8d storage: Introduce virStorageBackendDiskStartPool
https://bugzilla.redhat.com/show_bug.cgi?id=1251461

When 'starting' up a disk pool, we need to make sure the label on the
device is valid; otherwise, the followup refreshPool will assume the
disk has been properly formatted for use. If we don't find the valid
label, then refuse the start and give a proper reason.
2015-10-05 08:14:44 -04:00
John Ferlan
fba2076f43 storage: Add additional errors/checks for disk label
Let's check to ensure we can find the Partition Table in the label
and that libvirt actually recognizes that type; otherwise, when we
go to read the partitions during a refresh operation we may not be
reading what we expect.

This will expand upon the types of errors or reason that a build
would fail, so we can create more direct error messages.
2015-10-05 08:14:44 -04:00
John Ferlan
05c46f5c22 storage: Add param to check whether we can write a disk label
Modify virStorageBackendDiskValidLabel to add a 'writelabel' parameter.
While initially for the purpose of determining whether the label should
be written during DiskBuild, a future use during DiskStart could determine
whether the pool should be started using the label found. Augment the
error messages also to give a hint as to what someone may need to do
or why the command failed.
2015-10-05 08:14:44 -04:00