Lets say I got a volume with '1G' allocation and '10G' capacity. The
available space in the parent pool is '5G'. With the current check for
overcapacity, I can only try to resize to <= '6G'. You see the problem?
Currently, if scrub (used for wiping algorithms) is not present
at compile time, we don't support any other wiping algorithms than
zeroing, even if it was installed later. Switch to runtime detection
instead.
Input to the volume cloning code is a source volume and an XML
descriptor for the new volume. It is possible for the new volume
to have a greater size than source volume, at which point libvirt
will just stick 0s on the end of the new image (for raw format
anyways).
Unfortunately a logic error messed up our tracking of the of the
excess amount that needed to be written: end result is that sparse
clones were made very much non-sparse, and cloning regular disk
images could end up excessively sized (though data unaltered).
Drop the 'remain' variable entriely here since it's redundant, and
track actual allocation directly against the desired 'total'.
virFileOpenAs previously would only try opening a file as the current
user, or as a different user, but wouldn't try both methods in a
single call. This made it cumbersome to use as a replacement for
open(2). Additionally, it had a lot of historical baggage that led to
it being difficult to understand.
This patch refactors virFileOpenAs in the following ways:
* reorganize the code so that everything dealing with both the parent
and child sides of the "fork+setuid+setgid+open" method are in a
separate function. This makes the public function easier to understand.
* Allow a single call to virFileOpenAs() to first attempt the open as
the current user, and if that fails to automatically re-try after
doing fork+setuid (if deemed appropriate, i.e. errno indicates it
would now be successful, and the file is on a networkFS). This makes
it possible (in many, but possibly not all, cases) to drop-in
virFileOpenAs() as a replacement for open(2).
(NB: currently qemuOpenFile() calls virFileOpenAs() twice, once
without forking, then again with forking. That unfortunately can't
be changed without at least some discussion of the ramifications,
because the requested file permissions are different in each case,
which is something that a single call to virFileOpenAs() can't deal
with.)
* Add a flag so that any fchown() of the file to a different uid:gid
is explicitly requested when the function is called, rather than it
being implied by the presence of the O_CREAT flag. This just makes
for less subtle surprises to consumers. (Commit
b1643dc15c added the check for O_CREAT
before forcing ownership. This patch just makes that restriction
more explicit.)
* If either the uid or gid is specified as "-1", virFileOpenAs will
interpret this to mean "the current [gu]id".
All current consumers of virFileOpenAs should retain their present
behavior (after a few minor changes to their setup code and
arguments).
The old virRandom() API was not generating good random numbers.
Replace it with a new API virRandomBits which instead of being
told the upper limit, gets told the number of bits of randomness
required.
* src/util/virrandom.c, src/util/virrandom.h: Add virRandomBits,
and move virRandomInitialize
* src/util/util.h, src/util/util.c: Delete virRandom and
virRandomInitialize
* src/libvirt.c, src/security/security_selinux.c,
src/test/test_driver.c, src/util/iohelper.c: Update for
changes from virRandom to virRandomBits
* src/storage/storage_backend_iscsi.c: Remove bogus call
to virRandomInitialize & convert to virRandomBits
Currently, we support only filling a volume with zeroes on wiping.
However, it is not enough as data might still be readable by
experienced and equipped attacker. Many technical papers have been
written, therefore we should support other wiping algorithms.
On F16 at least, empty volume groups don't have a directory under /dev.
The directory only appears once a logical volume is created.
This tickles some behavior in BackendStablePath which ends with
libvirt sleeping for 5 seconds while waiting for the directory to appear.
This causes all sorts of problems for the virStorageVolLookupByPath API
which virtinst uses, even if trying to resolve a path that is independent
of the logical pool.
In reality we don't even need to do that checking since logical pools
always have a stable target path. Short circuit the polling in that
case.
Fixes bug 782261
If the vol object is newly created, it increases the volumes count,
but doesn't decrease the volumes count when do cleanup. It can
cause libvirtd to crash when one trying to free the volume objects
like:
for (i = 0; i < pool->volumes.count; i++)
virStorageVolDefFree(pool->volumes.objs[i]);
It's more reliable if we add the newly created vol object in the
end.
Current "-ay | -an" has problems on pool starting/refreshing if
the volumes are clustered. Rommer has posted a patch to list 2
months ago.
https://www.redhat.com/archives/libvir-list/2011-October/msg01116.html
But IMO we shouldn't skip the inactived vols. So this is a squashed
patch by Rommer.
Signed-off-by: Rommer <rommer@active.by>
https://bugzilla.redhat.com/show_bug.cgi?id=648855 mentioned a
misuse of 'an' where 'a' is proper; that has since been fixed,
but a search found other problems (some were a spelling error for
'and', while most were fixed by 'a').
* daemon/stream.c: Fix grammar.
* src/conf/domain_conf.c: Likewise.
* src/conf/domain_event.c: Likewise.
* src/esx/esx_driver.c: Likewise.
* src/esx/esx_vi.c: Likewise.
* src/rpc/virnetclient.c: Likewise.
* src/rpc/virnetserverprogram.c: Likewise.
* src/storage/storage_backend_fs.c: Likewise.
* src/util/conf.c: Likewise.
* src/util/dnsmasq.c: Likewise.
* src/util/iptables.c: Likewise.
* src/xen/xen_hypervisor.c: Likewise.
* src/xen/xend_internal.c: Likewise.
* src/xen/xs_internal.c: Likewise.
* tools/virsh.c: Likewise.
This partly reverts my previous patch f88de3eb. We need to
get file status after open, as given path could have been symlink,
so fstat() will operate on different file than lstat().
virStorageBackendLogicalDeleteVol() could not remove the lv with error
"could not remove open logical volume" sometimes. Generally it's caused
by the volume is still active, even if lvremove tries to remove it with
option "--force".
This patch is to fix it by disbale the lv first using "lvchange -aln"
and "lvremove -f" afterwards if the direct "lvremove -f" failed.
lvs outputs "[$lvname_vorigin]" for the virtual snapshot lv
(created with "--virtualsize"), and the original device pointed
by "$lvname_vorigin" is just for lvm internal use, one should
never use it.
Per lvm's nameing rules, "[" is not valid as part of the vg/lv name.
(man 8 lvm).
<quote>
VALID NAMES
The following characters are valid for VG and LV names: a-z A-Z 0-9 + _
. -
VG and LV names cannot begin with a hyphen. There are also various
reserved names that are used internally by lvm that can not be used as
LV or VG names. A VG cannot be called anything that exists in /dev/ at
the time of creation, nor can it be called '.' or '..'. A LV cannot be
called '.' '..' 'snapshot' or 'pvmove'. The LV name may also not con‐
tain the strings '_mlog' or '_mimage'
</quote>
So we can skip the set the lv's backingStore by checking if the name
begins with a "[".
which would blow away all volumes. Honor VIR_STORAGE_POOL_BUILD_OVERWRITE
to force a rebuild.
This was caught by libvirt-tck's storage/110-disk-pool.t.
Detected by Coverity. Only possible if qemu-img gives bogus output,
but we might as well be robust.
* src/storage/storage_backend.c
(virStorageBackendQEMUImgBackingFormat): Check for strstr failure.
Splitting into two functions allows the user to call the right
function, rather than having to remember that a *Free function is
an exception to the rule.
* src/conf/storage_conf.h (virStoragePoolSourceClear): New function.
* src/libvirt_private.syms (storage_conf.h): Export it.
* src/conf/storage_conf.c (virStoragePoolSourceFree): Split...
(virStoragePoolSourceClear): ...into new function.
(virStoragePoolDefFree, virStoragePoolDefParseSourceString):
Update callers.
* src/test/test_driver.c (testStorageFindPoolSources): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemNetFindPoolSourcesFunc)
(virStorageBackendFileSystemNetFindPoolSources): Likewise.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalFindPoolSources): Likewise.
Detected by Coverity. virStoragePoolSourceFree does not free the
actual passed-in pointer. A bigger patch would be to rename it
virStoragePoolSourceClear to match behavior, or even split it into
two functions depending on needed behavior; but this is the minimal
fix to the one location out of eight that leaked memory.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Free memory.
* src/storage/storage_backend_logical.c:
If a logical vol is created as striped. (e.g. --stripes 3),
the "device" field of lvs output will have multiple fileds which are
seperated by comma. Thus the RE we write in the codes will not
work well anymore. E.g. (lvs output for a stripped vol, uses "#" as
seperator here):
test_stripes##fSLSZH-zAS2-yAIb-n4mV-Al9u-HA3V-oo9K1B#\
/dev/sdc1(10240),/dev/sdd1(0)#42949672960#4194304
The RE we use:
const char *regexes[] = {
"^\\s*(\\S+),(\\S*),(\\S+),(\\S+)\\((\\S+)\\),(\\S+),([0-9]+),?\\s*$"
};
Also the RE doesn't match the "devices" field of striped vol properly,
it contains multiple "device path" and "offset".
This patch mainly does:
1) Change the seperator into "#"
2) Change the RE for "devices" field from "(\\S+)\\((\\S+)\\)"
into "(\\S+)".
3) Add two new options for lvs command, (segtype, stripes)
4) Extend the RE to match the value for the two new fields.
5) Parse the "devices" field seperately in virStorageBackendLogicalMakeVol,
multiple "extents" info are generated if the vol is striped. The
number of "extents" is equal to the stripes number of the striped vol.
A incidental fix: (virStorageBackendLogicalMakeVol)
Free "vol" if it's new created and there is error.
Demo on striped vol with the patch applied:
% virsh vol-dumpxml /dev/test_vg/vol_striped2
<volume>
<name>vol_striped2</name>
<key>QuWqmn-kIkZ-IATt-67rc-OWEP-1PHX-Cl2ICs</key>
<source>
<device path='/dev/sda5'>
<extent start='79691776' end='88080384'/>
</device>
<device path='/dev/sda6'>
<extent start='62914560' end='71303168'/>
</device>
</source>
<capacity>8388608</capacity>
<allocation>8388608</allocation>
<target>
<path>/dev/test_vg/vol_striped2</path>
<permissions>
<mode>0660</mode>
<owner>0</owner>
<group>6</group>
<label>system_u:object_r:fixed_disk_device_t:s0</label>
</permissions>
</target>
</volume>
RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=727474
If the regexes supported (?:pvs)?, then we could handle this by
optionally matching but not returning the initial command name. But it
doesn't. So add a new char* argument to
virStorageBackendRunProgRegex(). If that argument is NULL then we act
as usual. Otherwise, if the string at that argument is found at the
start of a returned line, we drop that before running the regex.
With this patch, virt-manager shows me lvs with command_names 1 or 0.
The definitions of PVS_BASE etc may want to be moved into the configure
scripts (though given how PVS is found, IIUC that could only happen if
pvs was a link to pvs_real), but in any case no sense dealing with that
until we're sure this is an ok way to handle it.
Signed-off-by: Serge Hallyn <serge.hallyn@canonical.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
* src/storage/storage_driver.c: As virStorageVolLookupByPath lookups
all the pool objs of the drivers, breaking when failing on getting
the stable path of the pool will just breaks the whole lookup process,
it can cause the API fails even if the vol exists indeed. It won't get
any benefit. This patch is to fix it.
Related #BZ: https://bugzilla.redhat.com/show_bug.cgi?id=702260.
There are two problems described in the BZ:
1) "Can't remove open logical volume".
2) "Unable to deactivate logical volume "foo""
This patch just intends to fix 2), as 1) is expected if the vol
is still used by something, and you never known if "lvchange -an"
will fail or not either (sometime, it will succeed, sometimes not).
We'd better not look for trouble, :-)
For 2), that's caused by race between lvremove and udev event handling,
the only workable way now is to wait the events handling are finished,
though it might introduce latencies, as "udevadmin settle" exits
after *all* events are handled, it's the only way we can fix
the racing in libvirt layer.
See https://bugzilla.redhat.com/show_bug.cgi?id=570359 for more
details.
Mac OS X 10.6. Snow Leopard and probably other do not provide a mkfs
command to create filesystems. Macro MKFS then remained undefined and
did not provide any substitute, so that build failed on a missing
argument.
Struct virStoragePoolProbeResult was compiled in conditionaly, but
virStorageBackendFileSystemProbe used it unconditionaly. This patch
exempts the struct from conditional include.
Fix bug #611823 storage driver should prohibit pools with duplicate
underlying storage.
Add internal API virStoragePoolSourceFindDuplicate() to do uniqueness
check based on source location infomation for pool type.
* AUTHORS: add Lei Li
This patch adds the ability to make the filesystem for a filesystem
pool during a pool build.
The patch adds two new flags, no overwrite and overwrite, to control
when mkfs gets executed. By default, the patch preserves the
current behavior, i.e., if no flags are specified, pool build on a
filesystem pool only makes the directory on which the filesystem
will be mounted.
If the no overwrite flag is specified, the target device is checked
to determine if a filesystem of the type specified in the pool is
present. If a filesystem of that type is already present, mkfs is
not executed and the build call returns an error. Otherwise, mkfs
is executed and any data present on the device is overwritten.
If the overwrite flag is specified, mkfs is always executed, and any
existing data on the target device is overwritten unconditionally.
Parted does not report disk size in 512 byte units, but
rather the disks' logical sector size, which with modern
drives might be 4k.
* src/storage/parthelper.c: Remove hardcoded 512 byte sector
size
Although we are flushing cache after some critical writes (e.g.
volume creation), after some others we do not (e.g. volume cloning).
This patch fix this issue. That is for volume cloning, writing
header of logical volume, and storage wipe.
Revert 6a1f5f568f. Now that libvirt_iohelper takes fds by
inheritance rather than by open() (commit 1eb66479), there is
no longer a race where the parent can unlink() a file prior to
the iohelper open()ing the same file. From there, it makes
more sense to have the callers both create and unlink, rather
than the caller create and the stream unlink, since the latter
was only needed when iohelper had to do the unlink.
* src/fdstream.h (virFDStreamOpenFile, virFDStreamCreateFile):
Callers are responsible for deletion.
* src/fdstream.c (virFDStreamOpenFileInternal): Don't leak created
file on failure.
(virFDStreamOpenFile, virFDStreamCreateFile): Drop parameter.
* src/lxc/lxc_driver.c (lxcDomainOpenConsole): Update callers.
* src/qemu/qemu_driver.c (qemuDomainScreenshot)
(qemuDomainOpenConsole): Likewise.
* src/storage/storage_driver.c (storageVolumeDownload)
(storageVolumeUpload): Likewise.
* src/uml/uml_driver.c (umlDomainOpenConsole): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainScreenshot): Likewise.
* src/xen/xen_driver.c (xenUnifiedDomainOpenConsole): Likewise.
Many volume operations will fail if the volume in question is being
allocated. These operations were returning VIR_ERR_INTERNAL_ERROR
when they should be returning VIR_ERR_OPERATION_INVALID.
Getting metadata on storage allocates a memory (path) which need to
be freed after use otherwise it gets leaked. This means after use of
virStorageFileGetMetadataFromFD or virStorageFileGetMetadata one
must call virStorageFileFreeMetadata to free it. This function frees
structure internals and structure itself.
No caller was using the flags argument, and this function is internal
only, so we might as well skip it.
* src/util/util.h (safezero): Update signature.
* src/util/util.c (safezero): Update function.
* src/locking/lock_driver_sanlock.c
(virLockManagerSanlockSetupLockspace)
(virLockManagerSanlockCreateLease): Update all callers.
* src/storage/storage_backend.c (createRawFile): Likewise.
Some callers expected virFileMakePath to set errno, some expected
it to return an errno value. Unify this to return 0 on success and
-1 on error. Set errno to report detailed error information.
Also optimize virFileMakePath if stat fails with an errno different
from ENOENT.
virStorageBackendCreateRaw: createRawFile already reported the
exact error.
Before the fix:
error: Failed to create vol vol-create.img
error: cannot create path '/var/lib/libvirt/images/vol-create.img': Unknown error 18446744073709551597
After the fix:
error: Failed to create vol vol-create.img
error: cannot fill file '/var/lib/libvirt/images/vol-create.img': No space left on device
