As pointed out by jtomko in his review of the IOThreads pinning code:
http://www.redhat.com/archives/libvir-list/2015-March/msg00495.html
there are some comments sprinkled in indicating IOThreads were using
the same structure as the VcpuPin code...
This is the first patch of a few that will change the virDomainVcpuPin*
structures and code to just virDomainPin* - starting with the data
structure naming...
During his review of the iothreads pin setting code, Pavel noted that
there was a potential memory leak with respect to how the newVcpuPin
is handled and the goto endjob's in failure paths which would not free
the memory. For reference, See:
http://www.redhat.com/archives/libvir-list/2015-March/msg00415.html
Now that the size of guest's memory can be inferred from the NUMA
configuration (if present) make it optional to specify <memory>
explicitly.
To make sure that memory is specified add a check that some form of
memory size was specified. One side effect of this change is that it is
no longer possible to specify 0KiB as memory size for the VM, but I
don't think it would be any useful to do so. (I can imagine embedded
systems without memory, just registers, but that's far from what libvirt
is usually doing).
Forbidding 0 memory for guests also fixes a few corner cases where 0 was
not interpreted correctly and caused failures. (Arguments for numad when
using automatic placement, size of the balloon). This fixes problems
described in https://bugzilla.redhat.com/show_bug.cgi?id=1161461
Test case changes are added to verify that the schema change and code
behave correctly.
Use the NUMA total instead of the configured size both in XML and for
uses in the code once NUMA is enabled for a domain.
One test case change is necessary as the rounding of the individual cell
sizes was not matching the rounding of the total size.
The memory sizes in qemu are aligned up to 1 MiB boundaries. There are
two places where this was done once for the total size and then for
individual NUMA cell sizes.
Add a function that will align the sizes in one place so that it's clear
where the sizes are aligned.
As there are two possible approaches to define a domain's memory size -
one used with legacy, non-NUMA VMs configured in the <memory> element
and per-node based approach on NUMA machines - the user needs to make
sure that both are specified correctly in the NUMA case.
To avoid this burden on the user I'd like to replace the NUMA case with
automatic totaling of the memory size. To achieve this I need to replace
direct access to the virDomainMemtune's 'max_balloon' field with
two separate getters depending on the desired size.
The two sizes are needed as:
1) Startup memory size doesn't include memory modules in some
hypervisors.
2) After startup these count as the usable memory size.
Note that the comments for the functions are future aware and document
state that will be present after a few later patches.
Surprisingly we did not grab a VM job when a block job finished and we'd
happily rewrite the backing chain data. This made it possible to crash
libvirt when queueing two backing chains tightly and other badness.
To fix it, add yet another handler to the helper thread that handles
monitor events that require a job.
We interpret port values as signed int (convert them from char *),
so if a negative value is provided in network disk's configuration,
we accept it as valid, however there's an 'unknown cause' error raised later.
This error is only accidental because we return the port value in the return code.
This patch adds just a minor tweak to the already existing check so we
reject negative values the same way as we reject non-numerical strings.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1163553
Valgrind detected a leak:
==17820== 102 (56 direct, 46 indirect) bytes in 1 blocks are definitely lost in loss record 479 of 646
==17820== at 0x4A08946: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==17820== by 0x508521A: virAllocVar (viralloc.c:560)
==17820== by 0x50D9FCA: virObjectNew (virobject.c:193)
==17820== by 0x50A4FD9: dnsmasqCapsNewEmpty (virdnsmasq.c:784)
==17820== by 0x50A514E: dnsmasqCapsNewFromBinary (virdnsmasq.c:830)
==17820== by 0x1B508287: networkStateInitialize (bridge_driver.c:666)
It looks like commit 172acef introduced the problem, because
networkGetDnsmasqCaps() increments the reference count but an
early exit never does a matching decrement.
* src/network/bridge_driver.c (networkStateCleanup): Plug leak.
Signed-off-by: Eric Blake <eblake@redhat.com>
Valgrind complained:
==3770== Syscall param ioctl(SIOCETHTOOL) points to uninitialised byte(s)
==3770== at 0x919D407: ioctl (syscall-template.S:81)
==3770== by 0x530FE7E: rpl_ioctl (ioctl.c:42)
==3770== by 0x50CB433: virNetDevFeatureAvailable (virnetdev.c:2764)
==3770== by 0x50CB6A7: virNetDevGetFeatures (virnetdev.c:2830)
==3770== by 0x1F0E5347: udevProcessNetworkInterface (node_device_udev.c:722)
==3770== by 0x1F0E689F: udevGetDeviceDetails (node_device_udev.c:1300)
==3770== by 0x1F0E6E06: udevAddOneDevice (node_device_udev.c:1422)
==3770== by 0x1F0E6FB8: udevProcessDeviceListEntry (node_device_udev.c:1464)
==3770== by 0x1F0E70CF: udevEnumerateDevices (node_device_udev.c:1494)
==3770== by 0x1F0E7BB4: nodeStateInitialize (node_device_udev.c:1806)
==3770== by 0x51B4303: virStateInitialize (libvirt.c:777)
==3770== by 0x11DEE7: daemonRunStateInit (libvirtd.c:906)
==3770== Address 0x228e38d4 is on thread 12's stack
==3770== in frame #2, created by virNetDevFeatureAvailable (virnetdev.c:2750)
* src/util/virnetdev.c (virNetDevFeatureAvailable): Initialize all
bytes of ifr.
Signed-off-by: Eric Blake <eblake@redhat.com>
Introduced by f6a2f97e
Problem Description:
After multiple times of migrating a domain, which has an ovs interface with no portData set,
with non-shared disk, nbd ports got overflowed.
The steps to reproduce the problem:
1 define and start a domain with its network configured as:
<interface type='bridge'>
<source bridge='br0'/>
<virtualport type='openvswitch'>
</virtualport>
<model type='virtio'/>
<driver name='vhost' queues='4'/>
</interface>
2 do not set the network's portData.
3 migrate(ToURI2) it with flag 91(1011011), which means:
VIR_MIGRATE_LIVE
VIR_MIGRATE_PEER2PEER
VIR_MIGRATE_PERSIST_DEST
VIR_MIGRATE_UNDEFINE_SOURCE
VIR_MIGRATE_NON_SHARED_DISK
4 migrate success, but we got an error log in libvirtd.log:
error : virCommandWait:2423 : internal error: Child process (ovs-vsctl --timeout=5 get Interface
vnet1 external_ids:PortData) unexpected exit status 1: ovs-vsctl: no key "PortData" in Interface
record "vnet1" column external_ids
5 migrate it back, migrate it , migrate it back, .......
6 nbd port got overflowed.
The reasons for the problem is :
1 virNetDevOpenvswitchGetMigrateData() takes it as wrong if no portData is available for the ovs
interface of a domain. (We think it's not appropriate, as portData is just OPTIONAL)
2 in func qemuMigrationBakeCookie(), it fails in qemuMigrationCookieAddNetwork(), and returns with -1.
qemuMigrationCookieAddNBD() is not called thereafter, and mig->nbd is still NULL.
3 However, qemuMigrationRun() just *WARN* if qemuMigrationBakeCookie() fails, migration still successes.
cookie is NULL, it's not baked on the src side.
4 On the destination side, it would alloc a port first and then free the nbd port in COOKIE.
But the cookie is NULL due to qemuMigrationCookieAddNetwork() failure at src side. thus the nbd port
is not freed.
In this patch, we add "--if-exists" option to make ovs-vsctl not raise error if there's no portData available.
Further more, because portData may be NULL in the cookie at the dest side, check it before setting portData.
Signed-off-by: Zhou Yimin <zhouyimin@huawei.com>
Signed-off-by: Zhang Bo <oscar.zhangbo@huawei.com>
It will not be possible to detach such device later. Also improve
logging in such cases.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
virDomainNetFindIdx no longer returns info whether device was not found,
or there was multiple matches. Additionally it already handle error
reporting. Introduce virDomainHasNet which does a simple task, without
implicit error reporting.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
In Xen>=4.3, libxl supports new syntax for USB devices:
usbdevice=[ "DEVICE", "DEVICE", ... ]
Add support for that in xenconfig driver. When only one device is
defined, keep using old syntax for backward compatibility.
Adjust tests for changed options order.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Now that the network driver lock is ash heap of history,
we can use more of networkObjFromNetwork().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
While in previous commits there were some places that relied on
the big lock, in this file there's no such place and the big
driver lock can be dropped completely. Yay!
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Well, if 'everywhere' is defined as that part of the driver code
that serves virNetwork* APIs. Again, we lower layers already have
their locks, so there's no point doing big lock.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Now that we have fine grained locks, there's no need to
lock the whole driver. We can rely on self-locking APIs.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
In order to drop network driver lock, lets annotate which
structure items are immutable, which have self-locking
APIs and so on.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This is not an immutable pointer and can change during lifetime.
Therefore, in order to drop network driver lock, we must use an
internal accessor which does not lock the network driver yet, but
it will soon. Now it merely returns an referenced object.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Well, network driver code has the driver accessible as a global
variable. This makes any rework hard, as it's unclear where the
variable is accessed and/or modified. Lets just pass the driver
as a parameter to all functions where needed.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
A helper that never returns an error and treats bits out of bitmap range
as false.
Use it everywhere we use ignore_value on virBitmapGetBit, or loop over
the bitmap size.
For VBOX it's most likely that the connection is vbox:///session and it
runs with local non-root account. This caused permission denied when
LOCALSTATEDIR was used to create temp file. This patch makes use of the
virGetUserCacheDirectory to address this problem for non-root users.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Commit c9027d8f added a detection of NIC HW features, but some of them
are not available in old kernel. Very old kernels lack enum
ethtool_flags and even if this enum is present, not all values are
available for all kernels. To be sure that we have everything in kernel
that we need, we must check for existence of most of that flags, because
only few of them were defined at first.
Also to successfully build libvirt with older kernel we need to include
<linux/types.h> before <linux/ethtool.h> to have __u32 and friends
defined.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Coverity notes in xenapiDomainGetXMLDesc that 'vms' is dereferenced
a few times before a "if (vms) xen_vm_set_free(vms);" call is made.
Since we'd exit out much sooner if the fetch of the vms failed, just
remove the unnecessary "if (vms)" check.
Coverity complains that "net_set" is compared to NULL before calling
xen_network_set_free, but used rather liberally before that. While
I was looking at the code I also noted that if the virAsprintfQuiet
fails, then we leak our structures - so I added those too.
Coverity points out that the return from virDomainDefParseString is
not checked in xenapiDomainCreateXML like it should be which could
end up in a NULL pointer dereference
Since inception. Coverity complains that the code checks "(record ==
NULL && !session->ok)", but doesn't check (record != NULL) before
dereferencing at "record->is_a_template"
https://bugzilla.redhat.com/show_bug.cgi?id=1135491
More or less a virtual copy of the existing virDomainVcpuPin{Add|Del} API's.
NB: The IOThreads implementation "reused" the virDomainVcpuPinDefPtr
since it provided everything necessary - an "id" and a "map" for each
thread id configured.
Add virDomainPinIOThread to allow setting the CPU affinity for a specific
IOThread based on the output generated from virDomainGetIOThreadsInfo
The API supports updating both the --live domain and the --config data
This patch turns both virNetworkObjFindByUUID() and
virNetworkObjFindByName() to return an referenced object so that
even if caller unlocks it, it's for sure that object won't
disappear meanwhile. Especially if the object (in general) is
locked and unlocked during the caller run.
Moreover, this commit is nicely small, since the object unrefing
can be done in virNetworkObjEndAPI().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Every API that touches internal structure of the object must lock
the object first. Not every API that has the object as an
argument needs to do that though. Some APIs just pass the object
to lower layers which, however, must lock the object then. Look
at the code, you'll get my meaning soon.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This is going to be needed later, when some functions already
have the virNetworkObjList object already locked and need to
lookup a object to work on. As an example of such function is
virNetworkAssignDef(). The other use case might be in
virNetworkObjListForEach() callback.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Later we can turn APIs to lock the object if needed instead of
relying on caller to mutually exclude itself (probably done by
locking a big lock anyway).
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
So far, this is pure code replacement. But once we introduce
reference counting to virNetworkObj this will be more handy as
there'll be only one function to change: virNetworkObjEndAPI().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
So far, this is pure code replacement. But once we introduce
reference counting to virNetworkObj this will be more handy as
there'll be only one function to change: virNetworkObjEndAPI().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
So far, this is pure code replacement. But once we introduce
reference counting to virNetworkObj this will be more handy as
there'll be only one function to change: virNetworkObjEndAPI().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This is practically copy of qemuDomObjEndAPI. The reason why is
it so widely available is to avoid code duplication, since the
function is going to be called from our bridge driver, test
driver and parallels driver too.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
So far it's just a structure which happens to have 'Obj' in its
name, but otherwise it not related to virObject at all. No
reference counting, not virObjectLock(), nothing.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Now that qemuDomainBlocksStatsGather provides functions of both
qemuMonitorGetBlockStatsParamsNumber and qemuMonitorGetBlockStatsInfo we
can reuse it and kill a lot of code.
Additionally as a bonus qemuDomainBlockStatsFlags will now support
summary statistics so add a statement to the virsh man page about that.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1142636
In the LXC driver, if the disk path is not provided the API returns
total statistics for all disks of the domain. With the new text monitor
implementation this can be now done in the qemu driver too.
Add code that wil total the stats for all disks if the path is not
provided.
Extract the code to look up the disk alias and return the block stats
struct so that it can be reused later in qemuDomainBlockStatsFlags.
The function uses qemuMonitorGetAllBlockStatsInfo instead of
qemuMonitorGetBlockStatsInfo.
Our virDomainBlockStatsFlags API uses the old approach where, when it's
called without the typed parameter array, returns the count of parameters
supported by qemu.
The supported parameter count is obtained via separate monitor calls
which is a waste since we can calculate it when gathering the data.
This patch adds code to the qemuMonitorGetAllBlockStatsInfo workers that
allows to track the count of supported fields reported by qemu and will
allow to remove the old duplicate code.
The function that is extracting block stats data from the QMP monitor
reply contains a lot of repeated code. Since I'd be changing each of the
copies in the next patch, lets convert it to a macro right away.
Add a different version of parser for "info blockstats" that basically
parses the same information as the existing copy of the function.
This will allow us to remove the single device version
qemuMonitorGetBlockStatsInfo in the future.
The new implementation uses few new helpers so it should be more
understandable and provides a test case to verify that it works.
Allocate the hash table in the monitor wrapper function instead of the
worker itself so that the text monitor impl that will be added in the
next patch doesn't have to duplicate it.
If a domain object is being removed and looked up concurrently we must
ensure we unlock the object before unreferencing it, since the latter
might free the object.
The flaw was introduced in commit feb1a4d792.
Signed-off-by: Michael Chapman <mike@very.puzzling.org>
Undefining a running, autostarted domain removes the autostart link, but
dom->autostart is not cleared. If the domain is subsequently redefined,
libvirt thinks it is already autostarted and will not create the link
even if requested:
# virsh dominfo example | grep Autostart
Autostart: enable
# ls /etc/libvirt/qemu/autostart/example.xml
/etc/libvirt/qemu/autostart/example.xml
# virsh undefine example
Domain example has been undefined
# virsh define example.xml
Domain example defined from example.xml
# virsh dominfo example | grep Autostart
Autostart: enable
# virsh autostart example
Domain example marked as autostarted
# ls /etc/libvirt/qemu/autostart/example.xml
ls: cannot access /etc/libvirt/qemu/autostart/example.xml: No such file or directory
This commit ensures dom->autostart is cleared whenever the config and
autostart link (if present) are removed.
The bridge network driver cleared this flag itself in networkUndefine.
This commit moves this into virNetworkDeleteConfig for symmetry with
virDomainDeleteConfig, and to ensure it is not missed in future network
drivers.
Signed-off-by: Michael Chapman <mike@very.puzzling.org>
Error messages are already set in all code paths returning -1 from
networkGetNetworkAddress, so we don't want to overwrite them.
Signed-off-by: Luyao Huang <lhuang@redhat.com>
Signed-off-by: John Ferlan <jferlan@redhat.com>
When creating qemu capabilities, a dummy virDomainObj is created just
because our monitor code expects that. However, the object is created
locked already. Then, under cleanup label, we simply unref the object
which results in whole domain object to be disposed. The object lock
is destroyed subsequently, but hey - it's still locked:
==24845== Thread #14's call to pthread_mutex_destroy failed
==24845== with error code 16 (EBUSY: Device or resource busy)
==24845== at 0x4C3024E: pthread_mutex_destroy (in /usr/lib64/valgrind/vgpreload_helgrind-amd64-linux.so)
==24845== by 0x531F72E: virMutexDestroy (virthread.c:83)
==24845== by 0x5302977: virObjectLockableDispose (virobject.c:237)
==24845== by 0x5302A89: virObjectUnref (virobject.c:265)
==24845== by 0x1DD37866: virQEMUCapsInitQMP (qemu_capabilities.c:3397)
==24845== by 0x1DD37CC6: virQEMUCapsNewForBinary (qemu_capabilities.c:3481)
==24845== by 0x1DD381E2: virQEMUCapsCacheLookup (qemu_capabilities.c:3609)
==24845== by 0x1DD30F8A: virQEMUCapsInitGuest (qemu_capabilities.c:744)
==24845== by 0x1DD31889: virQEMUCapsInit (qemu_capabilities.c:1020)
==24845== by 0x1DD7DD36: virQEMUDriverCreateCapabilities (qemu_conf.c:888)
==24845== by 0x1DDC57C0: qemuStateInitialize (qemu_driver.c:803)
==24845== by 0x53DC743: virStateInitialize (libvirt.c:777)
==24845==
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
in virDomainFSInfoFree(), don't free the virDomainFSInfo data.
==10670== 80 bytes in 2 blocks are definitely lost in loss record 576 of 793
==10670== at 0x4A06BC3: calloc (vg_replace_malloc.c:618)
==10670== by 0x509DEBD: virAlloc (viralloc.c:144)
==10670== by 0x19FBD558: qemuAgentGetFSInfo (qemu_agent.c:1837)
==10670== by 0x1A03CF91: qemuDomainGetFSInfo (qemu_driver.c:19238)
Signed-off-by: Chen Fan <chen.fan.fnst@cn.fujitsu.com>
Commit 4bbe1029f fixed a problem in commit f7afeddc by moving the call
to virNetDevGetIndex() to a location common to all interface types (so
that the nicindex array would be filled in for macvtap as well as tap
interfaces), but the location was *too* common, as the original call
to virNetDevGetIndex() had been in a section qualified by "if
(cfg->privileged)". The result was that the "fixed" libvirtd would try
to call virNetDevGetIndex() even for session mode libvirtd, and end up
failing with the log message:
Unable to open control socket: Operation not permitted
To remedy that, this patch qualifies the call to virNetDevGetIndex()
in its new location with cfg->privileged.
This resolves https://bugzilla.redhat.com/show_bug.cgi?id=1198244
As of bba93d40 all of our RPC objects are derived from
virObjectLockable. However, during rewrite some errors sneaked
in. For instance, the dispose functions to virNetClient and
virNetServerClient objects were not only freeing allocated
memory, but unlocking themselves. This is wrong. Object should
never disappear while locked.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Well, one day this will be self-locking object, but not today.
But lets prepare the code for that! Moreover,
virNetworkObjListFree() is no longer needed, so turn it into
virNetworkObjListDispose().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This API will be used in the future to call passed callback over
each network object in the list. It's slightly different to its
virDomainObjListForEach counterpart, because virDomainObjList
uses a hash table to store domain object, while virNetworkObjList
uses an array.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Commit v1.2.4-52-gda879e5 fixed issues with domains started before
sanlock driver was enabled by checking whether a running domain is
registered with sanlock and if it's not, sanlock driver is basically
ignored for the domain.
However, it was checking this even for domain which has just been
started and no sanlock_* API was called for them yet. This results in
cmd 9 target pid 2135544 not found
error messages to appear in sanlock.log whenever we start a new domain.
This patch avoids this useless check for freshly started domains.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
virLockManager*New APIs are never called with
VIR_LOCK_MANAGER_USES_STATE. Moreover, lockd driver does not maintain
any state that would need to be transferred during migration and thus it
should not mention VIR_LOCK_MANAGER_USES_STATE at all.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
By adding a call and check of return of virBitmapToData to the
IOThreads code, my Coverity checker lets me know qemuDomainHelperGetVcpus
also needs to check the status...
We have something like pvpanic device. However, in some cases it does
not have any address assigned, in which case we produce this ugly XML
(still valid though):
<devices>
<emulator>/usr/bin/qemu</emulator>
...
<panic>
</panic>
</devices>
Lets format "<panic/>" instead.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Depending on the flags passed, either attempt to return the active/live
IOThread data for the domain or the config data.
The active/live path will call into the Monitor in order to get the
IOThread data and then correlate the thread_id's returned from the
monitor to the currently running system/threads in order to ascertain
the affinity for each iothread_id.
The config path will map each of the configured IOThreads and return
any configured iothreadspin data
Signed-off-by: John Ferlan <jferlan@redhat.com>
