We had incomplete RelaxNG support for storage formats listed
in virstoragefile.h: commit 027bf2e added 'vdi' but forgot
to update the <volume> and <domain> xml lists; the <volume>
list was also missing 'fat' and 'vhd'. Maintaining two lists
is a recipe for them getting out of sync, so make the list
common so that both contexts benefit the next time we add a
format in a single location.
* docs/schemas/domaincommon.rng (storageFormat): Move...
* docs/schemas/storagecommon.rng: ...here, and add vdi.
* docs/schemas/storagevol.rng (formatfile): Use common list.
Signed-off-by: Eric Blake <eblake@redhat.com>
In general, we try to make virt-xml-validate tolerant of input
elements in any order when possible. However, as written, the
RNG grammar did not permit <source> unless there was an explicit
type= attribute (even though the C code manages just fine by
defaulting to type='file'). After making the attribute optional
on the 'file' branch, I noticed that the use of diskspec was now
redundant with the branch when no <source> was supplied.
View this patch with 'git diff -b' for a better picture of the
schema change.
* docs/schemas/domaincommon.rng (disk): Hoist 'diskspec' out of
choice, make type='file' default, and still preserve interleave.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-source-pool.xml:
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-drive-discard.xml:
New files.
* tests/qemuxml2argvdata/qemuxml2argv-disk-source-pool.xml:
* tests/qemuxml2argvdata/qemuxml2argv-disk-drive-discard.xml:
Reorder XML.
* tests/qemuxml2xmltest.c (mymain): Cover new files.
Signed-off-by: Eric Blake <eblake@redhat.com>
Commit 4897698 fixed the build without dbus by only building
the virSystemdPMSupportTarget with SYSTEMD_DAEMON.
Introduce a virDBusMessageUnref wrapper for dbus_message_unref
to let virsystemd.c build without dbus, while still allowing
virsystemdtest to run without SYSTEM_DAEMON.
In order to do that, virNodeSuspendSupportsTargetPMUtils() and
virSystemdPMSupportTarget() are created even when pm-utils and dbus
are compiled out, respectively, but in that case returning -2 meaning
"unavailable" (this return code was already used for unavailability
before). Error is reported in virNodeSuspendSupportsTarget() only if
both functions returned -2, otherwise the error (or success) is properly
propagated up the stack.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Having two tiny files with a couple definitions didn't make
as much sense as one common file, especially since I plan to
add more definitions and use it in more places.
* docs/schemas/storageencryption.rng: Merge this...
* docs/schemas/storagefilefeatures.rng: ...and this, into...
* docs/schemas/storagecommon.rng: ...this new file.
* docs/schemas/Makefile.am (schema_DATA): Reflect renames.
* docs/schemas/storagevol.rng: Likewise.
* docs/schemas/domaincommon.rng: Likewise.
* libvirt.spec.in: Likewise.
* mingw-libvirt.spec.in: Likewise.
Signed-off-by: Eric Blake <eblake@redhat.com>
When passing migration bandwidth to QEMU, we multiply it by 1024 * 1024
to convert the speed to B/s and the result still needs to fit in
int64_t.
https://bugzilla.redhat.com/show_bug.cgi?id=1083483
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Another field no longer needed, getting us one step closer to
merging virStorageFileMetadata and virStorageSource.
* src/util/virstoragefile.h (_virStorageFileMetadata): Drop
field.
* src/util/virstoragefile.c (virStorageFileGetMetadataInternal)
(virStorageFileGetMetadataFromFDInternal): Alter signature.
(virStorageFileFreeMetadata, virStorageFileGetMetadataFromBuf)
(virStorageFileGetMetadataFromFD): Adjust clients.
* tests/virstoragetest.c (_testFileData, testStorageChain)
(mymain): Simplify test.
Signed-off-by: Eric Blake <eblake@redhat.com>
Thanks to the testsuite, I feel quite confident that this rewrite
is correct; it gives the same results for all cases except for one.
I can make the argument that _that_ case was a pre-existing bug:
when looking up relative names, the lookup is supposed to be
pegged to the directory that contains the parent qcow2 file. Thus,
this resolves the fixme first mentioned in commit 367cd69 (even
though I accidentally removed the fixme comment early in 74430fe).
* src/util/virstoragefile.c (virStorageFileChainLookup): Depend on
new rather than old fields.
* tests/virstoragetest.c (mymain): Adjust test to match fix.
Signed-off-by: Eric Blake <eblake@redhat.com>
The original chain lookup code had to pass in the starting name,
because it was not available in the chain. But now that we have
added fields to the struct, this parameter is redundant.
* src/util/virstoragefile.h (virStorageFileChainLookup): Alter
signature.
* src/util/virstoragefile.c (virStorageFileChainLookup): Adjust
handling of top of chain.
* src/qemu/qemu_driver.c (qemuDomainBlockCommit): Adjust caller.
* tests/virstoragetest.c (testStorageLookup, mymain): Likewise.
Signed-off-by: Eric Blake <eblake@redhat.com>
The chain lookup function was inconsistent on whether it left
a message in the log when looking up a name that is not found
on the chain (leaving a message for OOM or if name was
relative but not part of the chain), and could litter the log
even when successful (when name was relative but deep in the
chain, use of virFindBackingFile early in the chain would complain
about a file not found). It's easier to make the function
consistently emit a message exactly once on failure, and to let
all callers rely on the clean semantics.
* src/util/virstoragefile.c (virStorageFileChainLookup): Always
report error on failure. Simplify relative lookups.
* src/qemu/qemu_driver.c (qemuDomainBlockCommit): Avoid
overwriting error.
Signed-off-by: Eric Blake <eblake@redhat.com>
When checking if two filenames point to the same inode (whether
by hardlink or symlink), sometimes one of the names might be
relative. This convenience function makes it easier to check.
* src/util/virfile.h (virFileRelLinkPointsTo): New prototype.
* src/util/virfile.c (virFileRelLinkPointsTo): New function.
* src/libvirt_private.syms (virfile.h): Export it.
* src/xen/xm_internal.c (xenXMDomainGetAutostart): Use it.
Signed-off-by: Eric Blake <eblake@redhat.com>
I realized that we had no good test coverage of looking up a
name from within a backing chain, even though code like
block-commit is relying on it.
* tests/virstoragetest.c (testStorageLookup): New function.
(mymain): New tests.
Signed-off-by: Eric Blake <eblake@redhat.com>
Noticed while tweaking the RelaxNG grammar for <disk> elements.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-numad-static-vcpu-no-numatune.xml:
* tests/qemuxml2xmloutdata/qemuxml2xmlout-disk-cdrom-empty.xml:
Drop unused files.
* tests/qemuargv2xmltest.c (testInfo, DO_TEST_FULL): Drop unused
field.
Signed-off-by: Eric Blake <eblake@redhat.com>
Drop another redundant field from virStorageFileMetadata.
* src/util/virstoragefile.h (_virStorageFileMetadata): Drop
field.
* src/util/virstoragefile.c
(virStorageFileGetMetadataFromFDInternal)
(virStorageFileGetMetadataFromFD)
(virStorageFileGetMetadataRecurse): Adjust callers.
* tests/virstoragetest.c (_testFileData, testStorageChain)
(mymain): Simplify test.
Signed-off-by: Eric Blake <eblake@redhat.com>
A couple pieces of virStorageFileMetadata are used only while
collecting information about the chain, and don't need to
live permanently in the struct. This patch refactors external
callers to collect the information separately, so that the
next patch can remove the fields.
* src/util/virstoragefile.h (virStorageFileGetMetadataFromBuf):
Alter signature.
* src/util/virstoragefile.c (virStorageFileGetMetadataInternal):
Likewise.
(virStorageFileGetMetadataFromFDInternal): Adjust callers.
* src/storage/storage_backend_fs.c (virStorageBackendProbeTarget):
Likewise.
* src/storage/storage_backend_gluster.c
(virStorageBackendGlusterRefreshVol): Likewise.
Signed-off-by: Eric Blake <eblake@redhat.com>
Finally starting to prune away some of the old fields that have
been made redundant by the new fields, on my way towards directly
reusing virStorageSource.
* src/util/virstoragefile.h (_virStorageFileMetadata): Drop
field.
* src/util/virstoragefile.c (virStorageFileGetMetadataInternal)
(virStorageFileChainLookup): Adjust callers.
* tests/virstoragetest.c (_testFileData, testStorageChain)
(mymain): Simplify test.
Signed-off-by: Eric Blake <eblake@redhat.com>
Deciding if a user string represents a local file instead of a
network path is an operation worth exposing directly, particularly
since the next patch will be removing a redundant variable that
was caching the information.
* src/util/virstoragefile.h (virStorageIsFile): New declaration.
* src/util/virstoragefile.c (virBackingStoreIsFile): Rename...
(virStorageIsFile): ...export, and allow NULL input.
(virStorageFileGetMetadataInternal)
(virStorageFileGetMetadataRecurse, virStorageFileGetMetadata):
Update callers.
* src/conf/domain_conf.c (virDomainDiskDefForeachPath): Use it.
* src/storage/storage_backend_fs.c (virStorageBackendProbeTarget):
Likewise.
* src/libvirt_private.syms (virstoragefile.h): Export function.
Signed-off-by: Eric Blake <eblake@redhat.com>
So far, my work has been merely preserving the status quo of
backing file analysis. But this patch starts to tread in the
territory of making the backing chain code more powerful - we
will eventually support network storage containing non-raw
formats. Here, we expose metadata information about a network
backing store, even if that information is still hardcoded to
a raw format for now.
* src/util/virstoragefile.c (virStorageFileGetMetadataRecurse):
Also populate struct for non-file backing.
(virStorageFileGetMetadata, virStorageFileGetMetadatainternal):
Recognize non-file top image.
(virFindBackingFile): Add comment.
(virStorageFileChainGetBroken): Adjust comment, ensure output
is set.
* tests/virstoragetest.c (mymain): Update test to reflect it.
Signed-off-by: Eric Blake <eblake@redhat.com>
I'm tired of alternating between test failures due to bugs in
my refactoring work, vs. test failures due to leftovers in
the file system from the previous test. This patch has no
impact when the testsuite is successful, but doeesn't hurt either.
* tests/virstoragetest.c (testPrepImages): Clean up from prior
failed test.
Signed-off-by: Eric Blake <eblake@redhat.com>
Most of the APIs in CPU driver do not expect to get NULL for input
parameters. Let's mark them with ATTRIBUTE_NONNULL and also check for
some members of virCPUDef when the APIs expect them have some specific
values.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Every test that makes use of virmock.h (only virsystemdtest as of now)
needs to be linked with -export-dynamic to make sure the LD_PRELOADed
mock library can access its wrap_* symbols. Normally,
DRIVER_MODULE_LDFLAGS variable contains -export-dynamic but when
--without-driver-modules configure option is used, DRIVER_MODULE_LDFLAGS
is empty.
This patch turns on -export-dynamic for all tests unconditionally
regardless on --without-driver-modules. This fixes virsystemdtest and
all future users of virmock.h.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
The original comment of vshCmdInfo:
"name" - command name
Actually it's 'help' and the short description
of command, not the command name.
Signed-off-by: Li Yang <liyang.fnst@cn.fujitsu.com>
The iterator is checked for being less than or equal to need_cpus.
The 'n' variable is incremented need_cpus + 1 times.
Simplify the computation of need_cpus and make its value one larger,
to let it be used instead of 'n' and compared without the equal sign
in loop conditions.
Just index the sum_cpu_time array instead of using a helper variable.
Start the loop at start_cpu instead of continuing for all lower values.
total_cpus is the total number of CPUs on the host
need_cpus is the number of CPUs we need to look at
(need_cpus can be larger than ncpus, because we need to look
at CPUs before the startcpu too, even if we aren't reporting
their stats)
Currently, virCgroupGetPercpuStats is only used by the LXC driver,
filling out the CPUTIME stats. qemuDomainGetPercpuStats does this
and also filles out VCPUTIME stats.
Extend virCgroupGetPercpuStats to also report VCPUTIME stats if
nvcpupids is non-zero. In the LXC driver, we don't have cpupids.
In the QEMU driver, there is at least one cpupid for a running domain,
so the behavior shouldn't change for QEMU either.
Also rename getSumVcpuPercpuStats to virCgroupGetPercpuVcpuSum.
Validate that all the new fields are getting set to desired values.
* tests/virstoragetest.c (_testFileData, testStorageChain): Check
for more fields.
(mymain): Populate additional fields.
Signed-off-by: Eric Blake <eblake@redhat.com>
The testsuite is absolutely essential to feeling comfortable
about swapping the backing chain structure over to a new format.
This patch tests the path settings, and demonstrates that the
correct short name is being passed to the child.
* tests/virstoragetest.c (testStorageChain): Test path.
(mymain): Update expected data.
Signed-off-by: Eric Blake <eblake@redhat.com>
The current use of virStorageFileMetadata is awkward; to learn
some of the information about a child node, you have to read
fields in the parent node. This does not lend itself well to
modifying backing chains (whether inserting a new node in the
chain, or consolidating existing nodes); better would be to
learn about a child node directly in that node. This patch
sets up some new fields which contain redundant information,
although not necessarily in the final desired state for the
new fields (see the next patch for actual tests of what is there
now). Then later patches will do any refactoring necessary to
get the fields to their desired states, and update clients to
get the information from the new fields, so we can finally
delete the fields that are tracking information about the wrong
node.
More concretely, compare these three example backing chains:
good <- one
missing <- two
gluster://server/vol/img <- three
Pre-patch, querying the chains gives:
{ .backingStore = "/path/to/good",
.backingStoreRaw = "good",
.backingStoreIsFile = true,
.backingStoreFormat = VIR_STORAGE_FILE_RAW,
.backingMeta = {
.backingStore = NULL,
.backingStoreRaw = NULL,
.backingStoreIsFile = false,
.backingMeta = NULL,
}
}
{ .backingStore = NULL,
.backingStoreRaw = "missing",
.backingStoreIsFile = false,
.backingStoreFormat = VIR_STORAGE_FILE_NONE,
.backingMeta = NULL,
}
{ .backingStore = "gluster://server/vol/img",
.backingStoreRaw = NULL,
.backingStoreIsFile = false,
.backingStoreFormat = VIR_STORAGE_FILE_RAW,
.backingMeta = NULL,
}
Deciding whether to ignore a missing backing file (as in virsh
vol-dumpxml) or report an error (as in security manager sVirt
labeling) requires reading multiple fields. Plus, the format
is hard-coded to treat all network protocols as end-of-the-chain,
as if they were raw. By the end of this patch series, the goal
is to instead represent these three situations as:
{ .path = "one",
.canonPath = "/path/to/one",
.type = VIR_STORAGE_TYPE_FILE,
.format = VIR_STORAGE_FILE_QCOW2,
.backingStoreRaw = "good",
.backingMeta = {
.path = "good",
.canonPath = "/path/to/good",
.type = VIR_STORAGE_TYPE_FILE,
.format = VIR_STORAGE_FILE_RAW,
.backingStoreRaw = NULL,
.backingMeta = NULL,
}
}
{ .path = "two",
.canonPath = "/path/to/two",
.type = VIR_STORAGE_TYPE_FILE,
.format = VIR_STORAGE_FILE_QCOW2,
.backingStoreRaw = "missing",
.backingMeta = NULL,
}
{ .path = "three",
.canonPath = "/path/to/three",
.type = VIR_STORAGE_TYPE_FILE,
.format = VIR_STORAGE_FILE_QCOW2,
.backingStoreRaw = "gluster://server/vol/img",
.backingMeta = {
.path = "gluster://server/vol/img",
.canonPath = "gluster://server/vol/img",
.type = VIR_STORAGE_TYPE_NETWORK,
.format = VIR_STORAGE_FILE_RAW,
.backingStoreRaw = NULL,
.backingMeta = NULL,
}
}
or, for the second file, maybe also allowing:
{ .path = "two",
.canonPath = "/path/to/two",
.type = VIR_STORAGE_TYPE_FILE,
.format = VIR_STORAGE_FILE_QCOW2,
.backingStoreRaw = "missing",
.backingMeta = {
.path = "missing",
.canonPath = NULL,
.type = VIR_STORAGE_TYPE_NONE,
.format = VIR_STORAGE_FILE_NONE,
.backingStoreRaw = NULL,
.backingMeta = NULL,
}
}
* src/util/virstoragefile.h (_virStorageFileMetadata): Add
path, canonPath, relDir, type, and format fields. Reorder
existing fields, and add lots of comments.
* src/util/virstoragefile.c (virStorageFileFreeMetadata): Clean
new fields.
(virStorageFileGetMetadataInternal)
(virStorageFileGetMetadataFromFDInternal): Start populating new
fields.
Signed-off-by: Eric Blake <eblake@redhat.com>
A later patch will be adding some new fields to
virStorageFileMetadata; to minimize confusion, renaming the
test fields now will make it more obvious which fields are
being tested later.
* tests/virstoragetest.c (_testFileData): Alter names.
(testStorageChain, mymain): Adjust clients.
Signed-off-by: Eric Blake <eblake@redhat.com>
Right now, we are allocating virStorageFileMetadata near the bottom
of the callchain, only after we have identified that we are visiting
a file (and not a network resource). I'm hoping to eventually
support parsing the backing chain from XML, where the backing chain
crawl then validates what was parsed rather than allocating a fresh
structure. Likewise, I'm working towards a setup where we have a
backing element even for networks. Both of these use cases are
easier to code if the allocation is hoisted earlier.
* src/util/virstoragefile.c (virStorageFileGetMetadataInternal)
(virStorageFileGetMetadataFromFDInternal): Change signature.
(virStorageFileGetMetadataFromBuf)
(virStorageFileGetMetadataRecurse, virStorageFileGetMetadata):
Update callers.
Signed-off-by: Eric Blake <eblake@redhat.com>
The previous patch started a separation of error messages
reported against the user-specified name, vs. tracking the
canonical path that was actually opened. This patch extends
that notion, by hoisting directory detection up front, passing
the canonical path through the entire call chain, and
simplifying lower-level functions that can now assume that
a canonical path and directory have been supplied.
* src/util/virstoragefile.c
(virStorageFileGetMetadataFromFDInternal)
(virStorageFileGetMetadataInternal): Add parameter, require
directory.
(virFindBackingFile): Require directory.
(virStorageFileGetMetadataFromFD): Pass canonical path.
(virStorageFileGetMetadataFromBuf): Likewise.
(virStorageFileGetMetadata): Determine initial directory.
Signed-off-by: Eric Blake <eblake@redhat.com>
Refactor the function to avoid multiple wrappers splitting identical
fields from the now common metadata struct.
The refactor is done by folding in the wrapper used for disk sources
which allows us to lookup secrets via the secret driver. This may allow
using stored secrets for snapshot disk images too in the future.
Now that we store all metadata about a storage image in a
virStorageSource struct let's use it also to store information needed by
the storage driver to access and do operations on the files.
https://bugzilla.redhat.com/show_bug.cgi?id=1024159
If adding a volume to a storage pool fails during the CreateXML or
CreateXMLFrom API's, we don't want to adjust the available and
allocation values for the storage pool during storageVolDelete
since we haven't adjusted the values for the create.
Refactor storageVolDelete() a bit to create a storageVolDeleteInternal()
which will handle the primary deletion activities. Add a parameter
updateMeta which will signify whether to update the values or not.
Adjust the calls from CreateXML and CreateXMLFrom to directly call the
DeleteInternal with the pool lock held. This does bypass the call
to virStorageVolDeleteEnsureACL().
Introduced in commit d1e55de3.
virstoragetest.c: In function ‘testStorageChain’:
virstoragetest.c:249:10: warning: declaration of ‘abs’ shadows a global
declaration [-Wshadow]
Another reduction in the number of structs I have to modify
when I start tracking new fields in virStorageFileMetadata.
* tests/virstoragetest.c (_testFileData): Add fields.
(testStorageChain): Select between fields based on flag.
(mymain): Record both absolute and relative expectations in one
struct.
Signed-off-by: Eric Blake <eblake@redhat.com>
As I add more tests, it's getting harder to follow the split between
a struct in one place and a test using the struct in another.
Interleaving the tests makes changes more localized, and also makes
debugging easier when a test goes wrong during my refactoring work.
* tests/virstoragetest.c (mymain): Modify structs as we go, rather
than up-front.
(testStorageChain): Make failure debugging easier.
Signed-off-by: Eric Blake <eblake@redhat.com>
Part of the upcoming refactoring will change how broken chains
are detected; it makes sense to test that this works. In
particular, test the just-fixed infinite loop detection bug.
Also, make sure that detection of directories is sane.
* tests/virstoragetest.c (testStorageChain): Enhance test.
(mymain): Add more tests.
(testCleanupImages, testPrepImages): Populate a directory.
Signed-off-by: Eric Blake <eblake@redhat.com>
While trying to refactor the backing file chain, I noticed that
if you have a self-referential qcow2 file via a relative name:
qemu-img create -f qcow2 loop 10M
qemu-img rebase -u -f qcow2 -F qcow2 -b loop loop
then libvirt was creating a chain 2 deep before realizing it
had hit a loop; furthermore, virStorageFileChainCheckBroken
was not identifying the chain as broken. With this patch,
the loop is detected when the chain is only 1 deep; still
enough for storage volume XML to display the file, but now
with a proper error report about where the loop was found.
This patch adds a parameter to virStorageFileGetMetadataRecurse,
so that errors at the top of the chain remain unchanged; messages
issued for backing files now use the name provided by the user
instead of the canonical name (for VDSM, which uses relative
symlinks to device mapper block devices, this is actually more
useful).
* src/util/virstoragefile.c (virStorageFileGetMetadataRecurse):
Add parameter, require canonical path up front. Mark chain
broken on OOM or loop detection.
(virStorageFileGetMetadata): Pass in canonical name.
Signed-off-by: Eric Blake <eblake@redhat.com>
