The migration protocol has support for a 'cookie' parameter which
is an opaque array of bytes as far as libvirt is concerned. Drivers
may use this for passing around arbitrary extra data they might
need during migration. The QEMU driver needs to do a few things:
- Pass hostname/uuid to allow strict protection against localhost
migration attempts
- Pass SPICE/VNC server port from the target back to the source to
allow seamless relocation of client sessions
- Pass lock driver state from source to destination
This patch introduces the basic glue for handling cookies
but only includes the host/guest UUID & name.
* src/libvirt_private.syms: Export virXMLParseStrHelper
* src/qemu/qemu_migration.c, src/qemu/qemu_migration.h: Parsing
and formatting of migration cookies
* src/qemu/qemu_driver.c: Pass in cookie parameters where possible
* src/remote/remote_protocol.h, src/remote/remote_protocol.x: Change
cookie max length to 16384 bytes
Migration just seems to go from bad to worse. We already had to
introduce a second migration protocol when adding the QEMU driver,
since the one from Xen was insufficiently flexible to cope with
passing the data the QEMU driver required.
It turns out that this protocol still has some flaws that we
need to address. The current sequence is
* Src: DumpXML
- Generate XML to pass to dst
* Dst: Prepare
- Get ready to accept incoming VM
- Generate optional cookie to pass to src
* Src: Perform
- Start migration and wait for send completion
- Kill off VM if successful, resume if failed
* Dst: Finish
- Wait for recv completion and check status
- Kill off VM if unsuccessful
The problems with this are:
- Since the first step is a generic 'DumpXML' call, we can't
add in other migration specific data. eg, we can't include
any VM lease data from lock manager plugins
- Since the first step is a generic 'DumpXML' call, we can't
emit any 'migration begin' event on the source, or have
any hook that runs right at the start of the process
- Since there is no final step on the source, if the Finish
method fails to receive all migration data & has to kill
the VM, then there's no way to resume the original VM
on the source
This patch attempts to introduce a version 3 that uses the
improved 5 step sequence
* Src: Begin
- Generate XML to pass to dst
- Generate optional cookie to pass to dst
* Dst: Prepare
- Get ready to accept incoming VM
- Generate optional cookie to pass to src
* Src: Perform
- Start migration and wait for send completion
- Generate optional cookie to pass to dst
* Dst: Finish
- Wait for recv completion and check status
- Kill off VM if failed, resume if success
- Generate optional cookie to pass to src
* Src: Confirm
- Kill off VM if success, resume if failed
The API is designed to allow both input and output cookies
in all methods where applicable. This lets us pass around
arbitrary extra driver specific data between src & dst during
migration. Combined with the extra 'Begin' method this lets
us pass lease information from source to dst at the start of
migration
Moving the killing of the source VM out of Perform and
into Confirm, means we can now recover if the dst host
can't successfully Finish receiving migration data.
Only in drivers which use virDomainObj, drivers that query hypervisor
for domain status need to be updated separately in case their hypervisor
supports this functionality.
The reason is also saved into domain state XML so if a domain is not
running (i.e., no state XML exists) the reason will be lost by libvirtd
restart. I think this is an acceptable limitation.
The same code for queueing, flushing, and deregistering events exists
in multiple drivers, which will soon use these common functions.
v2:
Adjust libvirt_private.syms
isDispatching bool fixes
NONNULL tagging
v3:
Add requireTimer parameter to virDomainEventStateNew
This structure will be used to unify lots of duplicated event handling code
across the state drivers.
v2:
Check for state == NULL in StateFree
Add NONNULL tagging
Use bool for isDispatching
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Actually execs the argv/env we've generated, replacing the current process.
Kind of has a limited usage, but allows us to use virCommand in LXC
driver to launch the 'init' process
Users often edit XML file stored in configuration directory
thinking of modifying a domain/network/pool/etc. Thus it is wise
to let them know they are using the wrong way and give them hint.
Some configuration setups for guests are allowed, but strongly
discouraged and unsupportable in production systems. Introduce
a concept of 'tainting' to virDomainObjPtr to allow such setups
to be identified. Drivers can then log warnings at suitable
times
* src/conf/domain_conf.c, src/conf/domain_conf.h: Declare taint
flags and add parsing/formatting of domain status XML
Match the fact that we have virAsprintf and virVasprintf.
* src/util/buf.h (virBufferVasprintf): New prototype.
* src/util/buf.c (virBufferAsprintf): Move guts...
(virBufferVasprintf): ...to new function.
* src/libvirt_private.syms (buf.h): Export it.
* bootstrap.conf (gnulib_modules): Add stdarg, for va_copy.
We already have virAsprintf, so picking a similar name helps for
seeing a similar purpose. Furthermore, the prefix V before printf
generally implies 'va_list', even though this variant was '...', and
the old name got in the way of adding a new va_list version.
global rename performed with:
$ git grep -l virBufferVSprintf \
| xargs -L1 sed -i 's/virBufferVSprintf/virBufferAsprintf/g'
then revert the changes in ChangeLog-old.
Support changes of disks by MODIFY_CONFIG for qemu.
This patch includes patches for qemu's disk to support
virDomainAt(De)tachDeviceFlags with VIR_DOMAIN_DEVICE_MODIFY_CONFIG.
Other devices can be added incrementally.
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
* /src/conf/domain_conf.c
(virDomainDiskIndexByName): returns array index of disk in vmdef.
(virDomainDiskRemoveByName): removes a disk which has the name in vmdef.
* src/qemu/qemu_driver.c
(qemuDomainAttachDeviceConfig): add support for Disks.
(qemuDomainDetachDeviceConfig): add support for Disks.
This patch adds functions for modify domain's persistent definition.
To do error recovery in easy way, we use a copy of vmdef and update it.
The whole sequence will be:
make a copy of domain definition.
if (flags & MODIFY_CONFIG)
update copied domain definition
if (flags & MODIF_LIVE)
do hotplug.
if (no error)
save copied one to the file and update cached definition.
else
discard copied definition.
This patch is mixuture of Eric Blake's work and mine.
From: Eric Blake <eblake@redhat.com>
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
(virDomainObjCopyPersistentDef): make a copy of persistent vm definition
(qemuDomainAttach/Detach/UpdateDeviceConfig) : callbacks. now empty
(qemuDomainModifyDeviceFlags): add support for MODIFY_CONFIG and MODIFY_CURRENT
This adds several tests for remaining hash APIs (custom
hasher/comparator functions are not covered yet, though).
All tests pass both before and after the "Simplify hash implementation".
mingw lacks the counterpart to PTHREAD_MUTEX_INITIALIZER, so the
best we can do is portably expose once-only runtime initialization.
* src/util/threads.h (virOnceControlPtr): New opaque type.
(virOnceFunc): New callback type.
(virOnce): New prototype.
* src/util/threads-pthread.h (virOnceControl): Declare.
(VIR_ONCE_CONTROL_INITIALIZER): Define.
* src/util/threads-win32.h (virOnceControl)
(VIR_ONCE_CONTROL_INITIALIZER): Likewise.
* src/util/threads-pthread.c (virOnce): Implement in pthreads.
* src/util/threads-win32.c (virOnce): Implement in WIN32.
* src/libvirt_private.syms: Export it.
Make: passed
Make check: passed
Make syntax-check: passed
this is the commit to introduce the function to create new character
device definition for the domain as advised by Cole Robinson
<crobinso@redhat.com>.
The function is used on the relevant places and also new tests has
been added.
Signed-off-by: Michal Novotny <minovotn@redhat.com>
This extends the SPICE XML to allow variable compression settings for audio,
images and streaming:
<graphics type='spice' port='5901' tlsPort='-1' autoport='yes'>
<image compression='auto_glz'/>
<jpeg compression='auto'/>
<zlib compression='auto'/>
<playback compression='on'/>
</graphics>
All new elements are optional.
This patch adds support for the evaluation of TCP flags in nwfilters.
It adds documentation to the web page and extends the tests as well.
Also, the nwfilter schema is extended.
The following are some example for rules using the tcp flags:
<rule action='accept' direction='in'>
<tcp state='NONE' flags='SYN/ALL' dsptportstart='80'/>
</rule>
<rule action='drop' direction='in'>
<tcp state='NONE' flags='SYN/ALL'/>
</rule>
Also related new functions' declaration, and expose the new introduced
functions in libvirt_private.syms.
v1 - v2:
Don't expose "virAllocVar" in libvirt_private.syms
* src/Makefile.am src/libvirt_private.syms configure.ac: share and
reuse the sexpr routines from sexpr.h of the old xen driver
* src/libxl/libxl_driver.c: implements libxlDomainXMLFromNative and
libxlDomainXMLToNative
This patch intentionally doesn't change indentation, in order to
make it easier to review the real changes.
* src/util/util.h (VIR_FILE_OP_RETURN_FD, virFileOperationHook):
Delete.
(virFileOperation): Rename...
(virFileOpenAs): ...and reduce parameters.
* src/util/util.c (virFileOperationNoFork, virFileOperation):
Rename and simplify.
* src/qemu/qemu_driver.c (qemudDomainSaveFlag): Adjust caller.
* src/storage/storage_backend.c (virStorageBackendCreateRaw):
Likewise.
* src/libvirt_private.syms: Reflect rename.
Sometimes, an asynchronous helper is started (such as a compressor
or iohelper program), but a later error means that we want to
abort that child. Make this easier.
Note that since daemons and virCommandRunAsync can't mix, the only
time virCommandFree can reap a process is if someone did
virCommandRunAsync for a non-daemon and didn't stash the pid.
* src/util/command.h (virCommandAbort): New prototype.
* src/util/command.c (_virCommand): Add new field.
(virCommandRunAsync, virCommandWait): Track whether pid was used.
(virCommandFree): Reap child if caller did not request pid.
(virCommandAbort): New function.
* src/libvirt_private.syms (command.h): Export it.
* tests/commandtest.c (test19): New test.
Child processes don't always reach _exit(); if they die from a
signal, then any messages should still be accurate. Most users
either expect a 0 status (thankfully, if status==0, then
WIFEXITED(status) is true and WEXITSTATUS(status)==0 for all
known platforms) or were filtering on WIFEXITED before printing
a status, but a few were missing this check. Additionally,
nwfilter_ebiptables_driver was making an assumption that works
on Linux (where WEXITSTATUS shifts and WTERMSIG just masks)
but fails on other platforms (where WEXITSTATUS just masks and
WTERMSIG shifts).
* src/util/command.h (virCommandTranslateStatus): New helper.
* src/libvirt_private.syms (command.h): Export it.
* src/util/command.c (virCommandTranslateStatus): New function.
(virCommandWait): Use it to also diagnose status from signals.
* src/security/security_apparmor.c (load_profile): Likewise.
* src/storage/storage_backend.c
(virStorageBackendQEMUImgBackingFormat): Likewise.
* src/util/util.c (virExecDaemonize, virRunWithHook)
(virFileOperation, virDirCreate): Likewise.
* daemon/remote.c (remoteDispatchAuthPolkit): Likewise.
* src/nwfilter/nwfilter_ebiptables_driver.c (ebiptablesExecCLI):
Likewise.
The ref count was assigned to 1 at creation, then never modified again
until it was decremented just before freeing the object.
* src/conf/domain_conf.h (_virDomainSnapshotObj): Delete unused
field.
(virDomainSnapshotObjUnref): Delete unused prototype.
* src/libvirt_private.syms: Likewise.
* src/conf/domain_conf.c (virDomainSnapshotObjNew)
(virDomainSnapshotObjListDataFree): Update users.
(virDomainSnapshotObjUnref): Delete.
* Correct the documentation for cgroup: the swap_hard_limit indicates
mem+swap_hard_limit.
* Change cgroup private apis to: virCgroupGet/SetMemSwapHardLimit
Signed-off-by: Nikunj A. Dadhania <nikunj@linux.vnet.ibm.com>
The virSetNonBlock() API only allows enabling non-blocking
operations. It doesn't allow turning blocking back on. Add
a new API to allow arbitrary toggling.
* src/libvirt_private.syms, src/util/util.h
src/util/util.c: Add virSetBlocking
This is the part allowing to dynamically resize the debug log
buffer from it's default 64kB size. The buffer is now dynamically
allocated.
It adds a new API virLogSetBufferSize() which resizes the buffer
If passed a zero size, the buffer is deallocated and we do the small
optimization of not formatting messages which are not output anymore.
On the daemon side, it just adds a new option log_buffer_size to
libvirtd.conf and call virLogSetBufferSize() if needed
* src/util/logging.h src/util/logging.c src/libvirt_private.syms:
make buffer dynamic and add virLogSetBufferSize() internal API
* daemon/libvirtd.conf: document the new log_buffer_size option
* daemon/libvirtd.c: read and use the new log_buffer_size option
Although the cgroup device ACL controller path can be worked out
by researching the code, it is more efficient to include that
information directly in the audit message.
* src/util/cgroup.h (virCgroupPathOfController): New prototype.
* src/util/cgroup.c (virCgroupPathOfController): Export.
* src/libvirt_private.syms: Likewise.
* src/qemu/qemu_audit.c (qemuAuditCgroup): Use it.
Not all applications have an existing event loop they need
to integrate with. Forcing them to implement the libvirt
event loop integration APIs is an undue burden. This just
exposes our simple poll() based implementation for apps
to use. So instead of calling
virEventRegister(....callbacks...)
The app would call
virEventRegisterDefaultImpl()
And then have a thread somewhere calling
static bool quit = false;
....
while (!quit)
virEventRunDefaultImpl()
* daemon/libvirtd.c, tools/console.c,
tools/virsh.c: Convert to public event loop APIs
* include/libvirt/libvirt.h.in, src/libvirt_private.syms: Add
virEventRegisterDefaultImpl and virEventRunDefaultImpl
* src/util/event.c: Implement virEventRegisterDefaultImpl
and virEventRunDefaultImpl using poll() event loop
* src/util/event_poll.c: Add full error reporting
* src/util/virterror.c, include/libvirt/virterror.h: Add
VIR_FROM_EVENTS
The event loop implementation is used by more than just the
daemon, so move it into the shared area.
* daemon/event.c, src/util/event_poll.c: Renamed
* daemon/event.h, src/util/event_poll.h: Renamed
* tools/Makefile.am, tools/console.c, tools/virsh.c: Update
to use new virEventPoll APIs
* daemon/mdns.c, daemon/mdns.c, daemon/Makefile.am: Update
to use new virEventPoll APIs
virLogEmergencyDumpAll() allows to dump the content of the
debug buffer from within a signal handler. It saves to all
log file or stderr if none is found
* src/util/logging.h src/util/logging.c: add the new API
and cleanup the old virLogDump code
* src/libvirt_private.syms: exports it as a private symbol
The virCgroupKill method kills all PIDs found in a cgroup
The virCgroupKillRecursively method does this recursively
for child cgroups.
The virCgroupKillPainfully method does a recursive kill
several times in a row until everything has really died
Since the deallocator is passed into the constructor of
a hash table it is not desirable to pass it into each
function again. Remove it from all functions, but provide
a virHashSteal to allow a item to be removed from a hash
table without deleteing it.
* src/util/hash.c, src/util/hash.h: Remove deallocator
param from all functions. Add virHashSteal
* src/libvirt_private.syms: Add virHashSteal
* src/conf/domain_conf.c, src/conf/nwfilter_params.c,
src/nwfilter/nwfilter_learnipaddr.c,
src/qemu/qemu_command.c, src/xen/xm_internal.c: Update
for changed hash API
This fixes https://bugzilla.redhat.com/show_bug.cgi?id=609463
The problem was that, since a bridge always acquires the MAC address
of the connected interface with the numerically lowest MAC, as guests
are started and stopped, it was possible for the MAC address to change
over time, and this change in the network was being detected by
Windows 7 (it sees the MAC of the default route change), so on each
reboot it would bring up a dialog box asking about this "new network".
The solution is to create a dummy tap interface with a MAC guaranteed
to be lower than any guest interface's MAC, and attach that tap to the
bridge as soon as it's created. Since all guest MAC addresses start
with 0xFE, we can just generate a MAC with the standard "0x52, 0x54,
0" prefix, and it's guaranteed to always win (physical interfaces are
never connected to these bridges, so we don't need to worry about
competing numerically with them).
Note that the dummy tap is never set to IFF_UP state - that's not
necessary in order for the bridge to take its MAC, and not setting it
to UP eliminates the clutter of having an (eg) "virbr0-nic" displayed
in the output of the ifconfig command.
I chose to not auto-generate the MAC address in the network XML
parser, as there are likely to be consumers of that API that don't
need or want to have a MAC address associated with the
bridge.
Instead, in bridge_driver.c when the network is being defined, if
there is no MAC, one is generated. To account for virtual network
configs that already exist when upgrading from an older version of
libvirt, I've added a %post script to the specfile that searches for
all network definitions in both the config directory
(/etc/libvirt/qemu/networks) and the state directory
(/var/lib/libvirt/network) that are missing a mac address, generates a
random address, and adds it to the config (and a matching address to
the state file, if there is one).
docs/formatnetwork.html.in: document <mac address.../>
docs/schemas/network.rng: add nac address to schema
libvirt.spec.in: %post script to update existing networks
src/conf/network_conf.[ch]: parse and format <mac address.../>
src/libvirt_private.syms: export a couple private symbols we need
src/network/bridge_driver.c:
auto-generate mac address when needed,
create dummy interface if mac address is present.
tests/networkxml2xmlin/isolated-network.xml
tests/networkxml2xmlin/routed-network.xml
tests/networkxml2xmlout/isolated-network.xml
tests/networkxml2xmlout/routed-network.xml: add mac address to some tests
The name convention of device mapper disk is different, and 'parted'
can't be used to delete a device mapper disk partition. e.g.
Name Path
-----------------------------------------
3600a0b80005ad1d7000093604cae912fp1 /dev/mapper/3600a0b80005ad1d7000093604cae912fp1
Error: Expecting a partition number.
This patch introduces 'dmsetup' to fix it.
Changes:
- New function "virIsDevMapperDevice" in "src/utils/utils.c"
- remove "is_dm_device" in "src/storage/parthelper.c", use
"virIsDevMapperDevice" instead.
- Requires "device-mapper" for 'with-storage-disk" in "libvirt.spec.in"
- Check "dmsetup" in 'configure.ac' for "with-storage-disk"
- Changes on "src/Makefile.am" to link against libdevmapper
- New entry for "virIsDevMapperDevice" in "src/libvirt_private.syms"
Changes from v1 to v3:
- s/virIsDeviceMapperDevice/virIsDevMapperDevice/g
- replace "virRun" with "virCommand"
- sort the list of util functions in "libvirt_private.syms"
- ATTRIBUTE_NONNULL(1) for virIsDevMapperDevice declaration.
e.g.
Name Path
-----------------------------------------
3600a0b80005ad1d7000093604cae912fp1 /dev/mapper/3600a0b80005ad1d7000093604cae912fp1
Vol /dev/mapper/3600a0b80005ad1d7000093604cae912fp1 deleted
Name Path
-----------------------------------------
Adds <smartcard mode='passthrough' type='spicevmc'/>, which uses the
new <channel name='smartcard'/> of <graphics type='spice'>.
* docs/schemas/domain.rng: Support new XML.
* docs/formatdomain.html.in: Document it.
* src/conf/domain_conf.h (virDomainGraphicsSpiceChannelName): New
enum value.
(virDomainChrSpicevmcName): New enum.
(virDomainChrSourceDef): Distinguish spicevmc types.
* src/conf/domain_conf.c (virDomainGraphicsSpiceChannelName): Add
smartcard.
(virDomainSmartcardDefParseXML): Parse it.
(virDomainChrDefParseXML, virDomainSmartcardDefParseXML): Set
spicevmc name.
(virDomainChrSpicevmc): New enum conversion functions.
* src/libvirt_private.syms: Export new functions.
* src/qemu/qemu_command.c (qemuBuildChrChardevStr): Conditionalize
name.
* tests/qemuxml2argvtest.c (domain): New test.
* tests/qemuxml2argvdata/qemuxml2argv-smartcard-passthrough-spicevmc.args:
New file.
* tests/qemuxml2argvdata/qemuxml2argv-smartcard-passthrough-spicevmc.xml:
Likewise.
* src/conf/domain_conf.h (virDomainSmartcardType): New enum.
(virDomainSmartcardDef, virDomainDeviceCcidAddress): New structs.
(virDomainDef): Include smartcards.
(virDomainSmartcardDefIterator): New typedef.
(virDomainSmartcardDefFree, virDomainSmartcardDefForeach): New
prototypes.
(virDomainControllerType, virDomainDeviceAddressType): Add ccid
enum values.
(virDomainDeviceInfo): Add ccid address type.
* src/conf/domain_conf.c (virDomainSmartcard): Convert between
enum and string.
(virDomainSmartcardDefParseXML, virDomainSmartcardDefFormat)
(virDomainSmartcardDefFree, virDomainDeviceCcidAddressParseXML)
(virDomainDefMaybeAddSmartcardController): New functions.
(virDomainDefParseXML): Parse the new XML.
(virDomainDefFormat): Convert back to XML.
(virDomainDefFree): Clean up.
(virDomainDeviceInfoIterate): Iterate over passthrough aliases.
(virDomainController, virDomainDeviceAddress)
(virDomainDeviceInfoParseXML, virDomainDeviceInfoFormat)
(virDomainDefAddImplicitControllers): Support new values.
* src/libvirt_private.syms (domain_conf.h): New exports.
* cfg.mk (useless_free_options): List new function.
Add a hook to the error reporting APIs to allow specific
error messages to be filtered out. Wire up libvirtd to
remove VIR_ERR_NO_DOMAIN & similar error codes from the
logs. They are still logged at DEBUG level.
* daemon/libvirtd.c: Filter VIR_ERR_NO_DOMAIN and friends
* src/libvirt_private.syms, src/util/virterror.c,
src/util/virterror_internal.h: Hook for changing error
reporting level
This reverts the additions in commit
abff683f78ffd01df5005fb7a457c0b38b8eb6e1
taking us back to state where all errors are fully logged
in both libvirtd and normal clients.
THe intent was to stop VIR_ERR_NO_DOMAIN (No such domain
with UUID XXXX) messages from client apps polluting syslog
The change affected all error codes, but more seriously,
it also impacted errors from internal libvirtd infrastructure
For example guest autostart no longer logged errors. The
libvirtd network code no longer logged some errors. This
makes debugging incredibly hard
* daemon/libvirtd.c: Remove error log priority filter
* src/util/virterror.c, src/util/virterror_internal.h: Remove
callback for overriding log priority
A need was found to set the SELinux context label on an open fd (a
pipe, as a matter of fact). This patch adds a function to the security
driver API that will set the label on an open fd to secdef.label. For
all drivers other than the SELinux driver, it's a NOP. For the SElinux
driver, it calls fsetfilecon().
If the return is a failure, it only returns error up to the caller if
1) the desired label is different from the existing label, 2) the
destination fd is of a type that supports setting the selinux context,
and 3) selinux is in enforcing mode. Otherwise it will return
success. This follows the pattern of the existing function
SELinuxSetFilecon().
The public object is called NWFilter but the corresponding private
object is called NWFilterPool. I don't see compelling reasons for this
Pool suffix. One might argue that an NWFilter is a "pool" of rules, etc.
Remove the Pool suffix from NWFilterPool. No functional change included.
It was awkward having only int conversion in the virStrToLong family,
but only long conversion in the virXPath family. Make both families
support both types.
* src/util/util.h (virStrToLong_l, virStrToLong_ul): New
prototypes.
* src/util/xml.h (virXPathInt, virXPathUInt): Likewise.
* src/util/util.c (virStrToLong_l, virStrToLong_ul): New
functions.
* src/util/xml.c (virXPathInt, virXPathUInt): Likewise.
* src/libvirt_private.syms (util.h, xml.h): Export them.
