The previous optimizations lead to some follow-on cleanups.
* src/conf/domain_conf.c (virDomainSnapshotForEachChild)
(virDomainSnapshotForEachDescendant): Drop dead parameter.
(virDomainSnapshotActOnDescendant)
(virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): Update callers.
* src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren)
(qemuDomainSnapshotListChildrenNames, qemuDomainSnapshotDelete):
Likewise.
* src/conf/domain_conf.h: Update prototypes.
Among other improvements, virDomainSnapshotForEachDescendant is
changed from iterative O(n^2) to recursive O(n). A bit better
than the O(n^3) implementation in virsh snapshot-list!
* src/conf/domain_conf.c (virDomainSnapshotObjListNum)
(virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjeListGetNames, virDomainSnapshotForEachChild)
(virDomainSnapshotForEachDescendant): Optimize.
(virDomainSnapshotActOnDescendant): Tweak.
(virDomainSnapshotActOnChild, virDomainSnapshotMarkDescendant):
Delete, now that they are unused.
Maintain the parent/child relationships of all qemu snapshots.
* src/qemu/qemu_driver.c (qemuDomainSnapshotLoad): Populate
relationships after loading.
(qemuDomainSnapshotCreateXML): Set relations on creation; tweak
redefinition to reuse existing object.
(qemuDomainSnapshotReparentChildren, qemuDomainSnapshotDelete):
Clear relations on delete.
No one was using virDomainSnapshotHasChildren, but that was an
O(n) function. Exposing and tracking a bit more metadata for each
snapshot will allow the same query to be made with an O(1) query
of the member field. For single snapshot operations (create,
delete), callers can be trusted to maintain the metadata themselves,
but for reloading, we can't compute parents as we go since there
is no guarantee that parents were parsed before children, so we also
provide a function to refresh the relationships, and which can
be used to detect if the user has ignored our warnings and been
directly modifying files in /var/lib/libvirt/qemu/snapshot. This
patch only adds metadata; later patches will actually use it.
This layout intentionally hardcodes the size of each snapshot struct,
by tracking sibling pointers, rather than having to deal with the
headache of yet more memory management by directly sticking a
dynamically sized child[] on each parent.
* src/conf/domain_conf.h (_virDomainSnapshotObj)
(_virDomainSnapshotObjList): Add members.
(virDomainSnapshotUpdateRelations, virDomainSnapshotDropParent):
New prototypes.
(virDomainSnapshotHasChildren): Delete.
* src/conf/domain_conf.c (virDomainSnapshotSetRelations)
(virDomainSnapshotUpdateRelations, virDomainSnapshotDropParent):
New functions.
(virDomainSnapshotHasChildren): Drop unused function.
* src/libvirt_private.syms (domain_conf): Update exports.
To date, JSON disk snapshots worked by accident, as they were always
using hmp fallback due to a typo in commit e702b5b not picking up
on the (intentional) difference in command names between the two
monitor protocols.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONDiskSnapshot):
Spell QMP command correctly.
Reported by Luiz Capitulino.
Rather than having to do:
$ virsh snapshot-revert dom $(virsh snapshot-current dom --name)
I thought it would be nice to do:
$ virsh snapshot-revert dom --current
I didn't add 'virsh snapshot-dumpxml --current' since we already have
'virsh snapshot-current' for the same task. snapshot-list accepted
a name but did not require it, and that remains the case, with
--current serving in place of that name. For all other commands,
name used to be required, and can now be replaced by --current;
I intentionally made it so that omitting both --current and a name
is an error (having the absence of a name imply --current seems
just a bit too magic, so --current must be explicit). I also had
to keep snapshot-edit backwards-compatible, as the only command
that already had a --current argument alongside a name, which still
works to both edit a named snapshot and make it current.
* tools/virsh.c (vshLookupSnapshot): New helper function.
(cmdSnapshotEdit, cmdSnapshotList, cmdSnapshotParent)
(cmdSnapshotDelete, cmdDomainSnapshotRevert): Use it, adding an
option where needed.
* tools/virsh.pod (snapshot-delete, snapshot-edit)
(snapshot-list, snapshot-parent, snapshot-revert): Document
use of --current.
(snapshot-dumpxml): Mention alternative.
Detected by autogen.sh on a cross-mingw build:
Creating library file: .libs/libvirt.dll.a
Cannot export virNetSASLContextCheckIdentity: symbol not defined
Cannot export virNetSASLContextNewServer: symbol not defined
...
* src/libvirt_private.syms (virnetsaslcontext.h): Move symbols...
* src/libvirt_sasl.syms: ...to new file.
* src/Makefile.am (USED_SYM_FILES) [HAVE_SASL]: Use new file.
(EXTRA_DIST): Ship it.
I got these distcheck failures with sanlock enabled:
ERROR: files left in build directory after distclean:
./tools/virt-sanlock-cleanup
./src/locking/qemu-sanlock.conf
* src/Makefile.am (DISTCLEANFILES) [HAVE_SANLOCK]: Clean built
file.
* tools/Makefile.am (DISTCLEANFILES): Likewise.
when building libvirt in Fedora/s390x I've found that xenlight needs to
be explicitly disabled in the spec file. Configure properly sets the
library as non-existent, but the %files section still wants to package
the 3 /var/*/libvirt/libxl directories. See also
https://bugzilla.redhat.com/show_bug.cgi?id=745020
Building on Linux with dtrace enabled was failing 'make check':
CCLD nodeinfotest
../src/.libs/libvirt_test.a(libvirt_net_rpc_client_la-virnetclient.o): In function `virNetClientNew':
/home/remote/eblake/libvirt/src/rpc/virnetclient.c:162: undefined reference to `libvirt_rpc_client_new_semaphore'
On looking further, I see some earlier warnings emitted from libtool:
*** Warning: Linking the shared library libvirt.la against the non-libtool
*** objects probes.o is not portable!
Since src/probes.o is only built on Linux, and even then, only when
dtrace is enabled, this failure does not affect other platforms, and
despite libtool warning that it is not generally portable, it is not
a problem for our use-case in libvirt.la. But it turns out that while
libtool is willing to jam raw .o files into an installed shared
library (libvirt.la becomes libvirt.so), it is NOT willing to jam
the same .o file into the convenience library libvirt_test.la.
Perhaps this is a bug in libtool, but even if we get libtool fixed,
libvirt must continue to build on platforms with older libtool. So,
the fix is the same as we are already using for the libvirt_lxc
executable - don't rely on the .o file being in the convenience
library, but instead use LDADD to pull it in directly.
* tests/Makefile.am (PROBES_O): New macro.
(LDADDS): Use it to fix link errors.
This removes the old example for legacy probes and adds two
new scripts demonstrating many of the new probe point facilities.
The rpc-monitor.stp script will print out friendly details of all
RPC traffic between a libvirt client/server. This is incredibly
useful in seeing what RPC calls are being made, and also debugging
problems in the RPC protocol code
The events.stp script will print out lots of info about the poll
event loop, which is useful for debugging event handling problems
* examples/systemtap/events.stp, examples/systemtap/rpc-monitor.stp:
New examples
* examples/systemtap/client.stp: Remove obsolete example
The libvirtd daemon had a few crude system tap probes. Some of
these were broken during the RPC rewrite. The new modular RPC
code is structured in a way that allows much more effective
tracing. Instead of trying to hook up the original probes,
define a new set of probes for the RPC and event code.
The master probes file is now src/probes.d. This contains
probes for virNetServerClientPtr, virNetClientPtr, virSocketPtr
virNetTLSContextPtr and virNetTLSSessionPtr modules. Also add
probes for the poll event loop.
The src/dtrace2systemtap.pl script can convert the probes.d
file into a libvirt_probes.stp file to make use from systemtap
much simpler.
The src/rpc/gensystemtap.pl script can generate a set of
systemtap functions for translating RPC enum values into
printable strings. This works for all RPC header enums (program,
type, status, procedure) and also the authentication enum
The PROBE macro will automatically generate a VIR_DEBUG
statement, so any place with a PROBE can remove any existing
manual DEBUG statements.
* daemon/libvirtd.stp, daemon/probes.d: Remove obsolete probing
* daemon/libvirtd.h: Remove probe macros
* daemon/Makefile.am: Remove all probe buildings/install
* daemon/remote.c: Update authentication probes
* src/dtrace2systemtap.pl, src/rpc/gensystemtap.pl: Scripts
to generate STP files
* src/internal.h: Add probe macros
* src/probes.d: Master list of probes
* src/rpc/virnetclient.c, src/rpc/virnetserverclient.c,
src/rpc/virnetsocket.c, src/rpc/virnettlscontext.c,
src/util/event_poll.c: Insert probe points, removing any
DEBUG statements that duplicate the info
Pull the call to gnutls_x509_crt_get_dn up into a higher function
so that the 'dname' variable will be available for probe points
* src/rpc/virnettlscontext.c: Pull gnutls_x509_crt_get_dn up
one level
If we receive an error on the stream, set the EOF marker so
that any further (bogus) incoming data is dropped.
* src/rpc/virnetclientstream.c: Set EOF on stream
To avoid static linking libvirtd to the RPC server code, which
then prevents sane introduction of DTrace probes, put it all
in the libvirt.so, and export it
* daemon/Makefile.am: Don't link to RPC libraries
* src/Makefile.am: Link all RPC libraries to libvirt.so
* src/libvirt_private.syms: Export all RPC functions
It was fairly trivial to return snapshot listing based on a
point in the hierarchy, rather than starting at all roots.
* src/esx/esx_driver.c (esxDomainSnapshotNumChildren)
(esxDomainSnapshotListChildrenNames): New functions.
Not too hard to wire up. The trickiest part is realizing that
listing children of a snapshot cannot use SNAPSHOT_LIST_ROOTS,
and that we overloaded that bit to also mean SNAPSHOT_LIST_DESCENDANTS;
we use that bit to decide which iteration to use, but don't want
the existing counting/listing functions to see that bit.
* src/conf/domain_conf.h (virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): New prototypes.
* src/conf/domain_conf.c (virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): New functions.
* src/libvirt_private.syms (domain_conf.h): Export them.
* src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren)
(qemuDomainSnapshotListChildrenNames): New functions.
Very mechanical. I'm so glad we've automated the generation of things,
compared to what it was in 0.8.x days, where this would be much longer.
* src/remote/remote_protocol.x
(REMOTE_PROC_DOMAIN_SNAPSHOT_NUM_CHILDREN)
(REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_CHILDREN_NAMES): New rpcs.
(remote_domain_snapshot_num_children_args)
(remote_domain_snapshot_num_children_ret)
(remote_domain_snapshot_list_children_names_args)
(remote_domain_snapshot_list_children_names_ret): New structs.
* src/remote/remote_driver.c (remote_driver): Use it.
* src/remote_protocol-structs: Update.
Given a list of snapshots and their parents, finding all descendants
requires a hairy traversal. This code is O(n^3); it could maybe be
made to scale O(n^2) with the use of a hash table, but that costs more
memory. Hopefully there aren't too many people with a hierarchy
so large as to approach REMOTE_DOMAIN_SNAPSHOT_LIST_NAMES_MAX (1024).
* tools/virsh.c (cmdSnapshotList): Add final fallback.
Iterating over one level of children requires parsing all snapshots
and their parents; a bit of code shuffling makes it pretty easy
to do this as well.
* tools/virsh.c (cmdSnapshotList): Add another fallback.
Emulating --from requires grabbing the entire list of snapshots
and their parents, and recursively iterating over the list from
the point of interest - but we already do that for --tree. This
turns on emulation for that situation.
* tools/virsh.c (__vshControl): Rename member.
(vshReconnect, cmdConnect, vshGetSnapshotParent): Update clients.
(cmdSnapshotList): Add fallback.
Sometimes, we only care about one branch of the snapshot hierarchy.
Make it easier to list a single branch, by using the new APIs.
Technically, I could emulate these new virsh options on old servers
by doing a complete dump, then scraping xml to filter out just the
snapshots that I care about, but I didn't want to do that in this patch.
* tools/virsh.c (cmdSnapshotList): Add --from, --descendants.
* tools/virsh.pod (snapshot-list): Document them.
The previous API addition allowed traversal up the hierarchy;
this one makes it easier to traverse down the hierarchy.
In the python bindings, virDomainSnapshotNumChildren can be
generated, but virDomainSnapshotListChildrenNames had to copy
from the hand-written example of virDomainSnapshotListNames.
* include/libvirt/libvirt.h.in (virDomainSnapshotNumChildren)
(virDomainSnapshotListChildrenNames): New prototypes.
(VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS): New flag alias.
* src/libvirt.c (virDomainSnapshotNumChildren)
(virDomainSnapshotListChildrenNames): New functions.
* src/libvirt_public.syms: Export them.
* src/driver.h (virDrvDomainSnapshotNumChildren)
(virDrvDomainSnapshotListChildrenNames): New callbacks.
* python/generator.py (skip_impl, nameFixup): Update lists.
* python/libvirt-override-api.xml: Likewise.
* python/libvirt-override.c
(libvirt_virDomainSnapshotListChildrenNames): New wrapper function.
On xen 4.1 I observed configurations that look like:
(image
(hvm
(kernel '')
(loader '/foo/bar')
))
The kernel element is there but unset. This leads to an empty <kernel/>
element in the XML and even worse makes us skip the boot order parsing
and therefore not emit a <boot device='$dev>'/> element which breaks CD
booting.
otherwise a missing UUID in a domain config just shows:
error: An error occurred, but the cause is unknown
Now we have:
error: configuration file syntax error: config value uuid was missing
I was a bit surprised that 'virsh snapshot-edit dom name' silently
allowed me to clone things, while still telling me the old name,
especially since other commands like 'virsh edit dom' reject rename
attempts (*). This fixes things to be more explicit (**).
(*) Technically, 'virsh edit dom' relies on virDomainDefineXML
behavior, which rejects attempts to mix a new name with existing
uuid or new uuid with existing name, but you can create a new
domain by changing both uuid and name. On the other hand, while
snapshot-edit --clone is a true clone, creating a new domain
would also have to decide whether to clone snapshot metadata,
managed save, and any other secondary data related to the domain.
Domain renames are not trivial either.
(**) Renaming or creating a clone is still a risky proposition -
for offline snapshots and system checkpoints, if the new name
does not match an actual name recorded in the qcow2 internal
snapshots, then you cannot revert to the new checkpoint. But it
is assumed that anyone using the new virsh flags knows what they
are doing, and can deal with the fallout caused by a rename/clone;
that is, we can't completely prevent a user from shooting
themselves in the foot, so much as we are making the default
action less risky.
* tools/virsh.c (cmdSnapshotEdit): Add --rename, --clone.
* tools/virsh.pod (snapshot-edit): Document them.
* src/storage/storage_backend_logical.c:
If a logical vol is created as striped. (e.g. --stripes 3),
the "device" field of lvs output will have multiple fileds which are
seperated by comma. Thus the RE we write in the codes will not
work well anymore. E.g. (lvs output for a stripped vol, uses "#" as
seperator here):
test_stripes##fSLSZH-zAS2-yAIb-n4mV-Al9u-HA3V-oo9K1B#\
/dev/sdc1(10240),/dev/sdd1(0)#42949672960#4194304
The RE we use:
const char *regexes[] = {
"^\\s*(\\S+),(\\S*),(\\S+),(\\S+)\\((\\S+)\\),(\\S+),([0-9]+),?\\s*$"
};
Also the RE doesn't match the "devices" field of striped vol properly,
it contains multiple "device path" and "offset".
This patch mainly does:
1) Change the seperator into "#"
2) Change the RE for "devices" field from "(\\S+)\\((\\S+)\\)"
into "(\\S+)".
3) Add two new options for lvs command, (segtype, stripes)
4) Extend the RE to match the value for the two new fields.
5) Parse the "devices" field seperately in virStorageBackendLogicalMakeVol,
multiple "extents" info are generated if the vol is striped. The
number of "extents" is equal to the stripes number of the striped vol.
A incidental fix: (virStorageBackendLogicalMakeVol)
Free "vol" if it's new created and there is error.
Demo on striped vol with the patch applied:
% virsh vol-dumpxml /dev/test_vg/vol_striped2
<volume>
<name>vol_striped2</name>
<key>QuWqmn-kIkZ-IATt-67rc-OWEP-1PHX-Cl2ICs</key>
<source>
<device path='/dev/sda5'>
<extent start='79691776' end='88080384'/>
</device>
<device path='/dev/sda6'>
<extent start='62914560' end='71303168'/>
</device>
</source>
<capacity>8388608</capacity>
<allocation>8388608</allocation>
<target>
<path>/dev/test_vg/vol_striped2</path>
<permissions>
<mode>0660</mode>
<owner>0</owner>
<group>6</group>
<label>system_u:object_r:fixed_disk_device_t:s0</label>
</permissions>
</target>
</volume>
RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=727474
If parsing qemu command line fails (e.g. because of non-existing
process number supplied), we jump to cleanup label where we free
pidfile. Therefore it needs to be initialized. Otherwise we free
random pointer.
Coverity complained that 4 out of 5 callers to virJSONValueObjectGetBoolean
checked for errors. But we documented that we don't care in this case.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONGetBlockInfo): Use
ignore_value.
Detected by Coverity. We want to increment the size_t counter,
not the pointer to the counter. Bug present since 5f5c6fde (0.9.5).
* src/lxc/lxc_controller.c (lxcSetupLoopDevices): Use correct
precedence.
virsh undefine command can now undefine an active guest, but the help information is still the old.
This patch modifies it and make it coincident to the manpage of virsh.
Signed-off-by: tangchen <tangchen@cn.fujitsu.com>
If we send back an unknown program error for async messages,
we will confuse the client because they only expect replies
for method calls. Just log & drop any invalid async messages
* src/rpc/virnetserver.c: Don't send error for async messages
Commit 597fe3cee6 accidentally
introduced a deadlock when reporting an unknown RPC program.
The virNetServerDispatchNewMessage method is called with
the client locked, and must therefore not attempt to send
any RPC messages back to the client. Only once the incoming
message is passed off to the virNetServerHandleJob worker
is it safe to start sending messages back
* src/rpc/virnetserver.c: Delay checking for unknown RPC
program until in worker thread
Redefining disk-only snapshot xml should work even if the user
did not explicitly pass VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY;
the flag is only required for conditions where the <state>
subelement is not already present in parsing (that is, defining
a new snapshot).
Also, fix the error code of some user-visible errors (the remaining
VIR_ERR_INTERNAL_ERROR should not be user-visible, since parsing
of <active> is only done from internal code).
* src/conf/domain_conf.c (virDomainSnapshotDefParseString): Allow
disks during redefinition of disk snapshot.
It was impossible for 'virsh snapshot-current dom name' to set name
as the current snapshot, if name is a disk-only snapshot.
Using strstr rather than full-blown xml parsing is safe, since the
xml is assumed to be well-formed coming from libvirtd rather than
arbitrary text coming from the user.
* tools/virsh.c (cmdSnapshotCurrent, cmdSnapshotEdit): Pass
disk_only flag when redefining a disk snapshot.
I am getting this failure with 'make distcheck':
GEN ../../src/remote_protocol-structs
/bin/sh: ../../src/remote_protocol-structs-t: Permission denied
make[4]: *** [../../src/remote_protocol-structs] Error 1
since it attempts a sub-run of a VPATH 'make check' where $(srcdir)
is intentionally read-only. I'm not sure which commit introduced
the problem, although I suspect it was around 62dee6f when I
refactored protocol struct checking to be more powerful.
$(@F) is required by POSIX, and although it is not yet portable
to all make implementations, we already require GNU make.
* src/Makefile.am (PDWTAGS): Generate temp file into current
directory, since $(srcdir) is read-only during distcheck.
Previously libvirt's disk device XML only had a single attribute,
error_policy, to control both read and write error policy, but qemu
has separate options for controlling read and write. In one case
(enospc) a policy is allowed for write errors but not read errors.
This patch adds a separate attribute that sets only the read error
policy. If just error_policy is set, it will apply to both read and
write error policy (previous behavior), but if the new rerror_policy
attribute is set, it will override error_policy for read errors only.
Possible values for rerror_policy are "stop", "report", and "ignore"
("report" is the qemu-controlled default for rerror_policy when
error_policy isn't specified).
For consistency, the value "report" has been added to the possible
values for error_policy as well.
commit 12062ab set rerror=ignore when error_policy="enospace" was
selected (since the rerror option in qemu doesn't accept "enospc", as
the werror option does).
After that patch was already pushed, Paolo Bonzini noticed it and
commented that leaving rerror at the default ("report") would be a
better choice. This patch corrects the problem - if error_policy =
"enospace" is given, rerror is left off the qemu commandline,
effectively setting it to "report". For other values, rerror is still
set to match werror.
Additionally, the parsing of error_policy was changed to no longer
erroneously allow "default" as a choice - as with most other
attributes, if you want the default setting, just don't specify an
error_policy.
Finally, two ommissions in the first patch were corrected - a
long-dormant qemuxml2argv test for enospace was enabled, and fixed to
pass, and the argv2xml parser in qemu_command.c was updated to
recognize the different spelling on the qemu commandline.
