Commit Graph

3212 Commits

Author SHA1 Message Date
Martin Kletzander
7c4b4f8905 util: Make virResctrlGetCacheControlType() behave like other functions
That means that returning negative values means error and non-negative
values differ in meaning, but are all successful.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-08-14 10:01:12 +02:00
Martin Kletzander
af4270400a Move resctrl-related code from conf/capabilities to util/virresctrl
It doesn't access anything from conf/ and ti will be needed to use
from other util/ places.  This split makes the separation clearer.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-08-14 10:01:12 +02:00
Martin Kletzander
62146d8532 virxml: Fix indentation
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-08-14 10:01:12 +02:00
Laine Stump
f5bc8b5436 util: eliminate superfluous saveVlan check in virNetDevSetNetConfig()
Commit 81fb440b further qualified an if statement by adding the
boolean saveVlan to the condition. Coverity pointed out that this
change in the logic eliminated the need to check saveVlan in an
argument to virAsprintf().
2017-08-13 23:07:13 -04:00
Laine Stump
83074cc917 util: fix improper assignment of return value in virHostdevReadNetConfig()
Commit 9a94af6d restructured virHostdevReadNetConfig() so that it
would manually set ret = 0 after successfully reading the device's
config, but Coverity pointed out that "ret = 0" was erroneously placed
outside of an "else" clause, meaning that the the value of ret set in
the "if" clause was unnecessarily and incorrectly overwritten.

This patch moves ret = 0 into the else clause, which should silence
Coverity.
2017-08-13 23:07:13 -04:00
Laine Stump
489a937eb4 util: check for PF online status earlier in guest startup
When using a VF from an SRIOV-capable network card in a guest (either
in macvtap passthrough mode, or via VFIO PCI device assignment), The
associated PF netdev must be online in order for the VF to be usable
by the guest. The guest, however, is not able to change the state of
the PF. And libvirt *could* set the PF online as needed, but that
could lead to the host receiving unexpected IPv6 traffic (since the
default for an unconfigured interface is to participate in IPv6
autoconf). For this reason, before assigning a VF to a guest, libvirt
verifies that the related PF netdev is online - if it isn't, then we
log an error and don't allow the guest startup to continue.

Until now, this check was done during virNetDevSetNetConfig(). This
works nicely because the same function is called both for macvtap
passthrough and for VFIO device assignment. But in the case of VFIO,
the VF has already been unbound from its netdev driver by the time we
get to virNetDevSetNetConfig(), and in the case of dual port Mellanox
NICs that have their VFs setup in single port mode, the *only* way to
determine the proper PF netdev to query for online status is via the
"phys_port_id" file that is in the VF netdev's sysfs directory. *BUT*
if we've unbound the VF from the netdev driver, then it doesn't *have*
a netdev sysfs directory.

So, in order to check the correct PF netdev for online status, this
patch moved the check earlier in the setup, into
virNetDevSaveNetConfig(), which is called *before* unbinding the VF
from its netdev driver.

(Note that this implies that if you are using VFIO device assignment
for the VFs of a Mellanox NIC that has the VFs programmed in single
port mode, you must let the VFs be bound to their net driver and use
"managed='yes'" in the device definition. To be more specific, this is
only true if the VFs in single port mode are using port *2* of the PF
- if the VFs are using only port 1, then the correct PF netdev will be
arrived at by default/chance))

  This resolves: https://bugzilla.redhat.com/267191
2017-08-11 19:13:33 -04:00
Laine Stump
9a08168301 util: restructure virNetDevReadNetConfig() to eliminate false error logs
virHostdevRestoreNetConfig() calls virNetDevReadNetConfig() to try and
read the "original config" of a netdev, and if that fails, it tries
again with a different directory/netdev name. This achieves the
desired effect (we end up finding the config wherever it may be), but
for each failure, virNetDevReadNetConfig() places a nice error message
in the system logs. Experience has shown that false-positive error
logs like this lead to erroneous bug reports, and can often mislead
those searching for *real* bugs.

This patch changes virNetDevReadNetConfig() to explicitly check if the
file exists before calling virFileReadAll(); if it doesn't exist,
virNetDevReadNetConfig() returns a success, but leaves all the
variables holding the results as NULL. (This makes sense if you define
the purpose of the function as "read a netdev's config from its config
file *if that file exists*).

To take advantage of that change, the caller,
virHostdevRestoreNetConfig() is modified to fail immediately if
virNetDevReadNetConfig() returns an error, and otherwise to try the
different directory/netdev name if adminMAC & vlan & MAC are all NULL
after the preceding attempt.
2017-08-11 19:09:49 -04:00
Laine Stump
b67eaa6351 util: save the correct VF's info when using a dual port SRIOV NIC in single port mode
Mellanox ConnectX-3 dual port SRIOV NICs present a bit of a challenge
when assigning one of their VFs to a guest using VFIO device
assignment.

These NICs have only a single PCI PF device, and that single PF has
two netdevs sharing the single PCI address - one for port 1 and one
for port 2. When a VF is created it can also have 2 netdevs, or it can
be setup in "single port" mode, where the VF has only a single netdev,
and that netdev is connected either to port 1 or to port 2.

When the VF is created in dual port mode, you get/set the MAC
address/vlan tag for the port 1 VF by sending a netlink message to the
PF's port1 netdev, and you get/set the MAC address/vlan tag for the
port 2 VF by sending a netlink message to the PF's port 2 netdev. (Of
course libvirt doesn't have any way to describe MAC/vlan info for 2
ports in a single hostdev interface, so that's a bit of a moot point)

When the VF is created in single port mode, you can *set* the MAC/vlan
info by sending a netlink message to *either* PF netdev - the driver
is smart enough to understand that there's only a single netdev, and
set the MAC/vlan for that netdev. When you want to *get* it, however,
the driver is more accurate - it will return 00:00:00:00:00:00 for the
MAC if you request it from the port 1 PF netdev when the VF was
configured to be single port on port 2, or if you request if from the
port 2 PF netdev when the VF was configured to be single port on port
1.

Based on this information, when *getting* the MAC/vlan info (to save
the original setting prior to assignment), we determine the correct PF
netdev by matching phys_port_id between VF and PF.

(IMPORTANT NOTE: this implies that to do PCI device assignment of the
VFs on dual port Mellanox cards using <interface type='hostdev'>
(i.e. if you want the MAC address/vlan tag to be set), not only must
the VFs be configured in single port mode, but also the VFs *must* be
bound to the host VF net driver, and libvirt must use managed='yes')

By the time libvirt is ready to set the new MAC/vlan tag, the VF has
already been unbound from the host net driver and bound to
vfio-pci. This isn't problematic though because, as stated earlier,
when a VF is created in single port mode, commands to configure it can
be sent to either the port 1 PF netdev or the port 2 PF netdev.

When it is time to restore the original MAC/vlan tag, again the VF
will *not* be bound to a host net driver, so it won't be possible to
learn from sysfs whether to use the port 1 or port 2 PF netdev for the
netlink commands. And again, it doesn't matter which netdev you
use. However, we must keep in mind that we saved the original settings
to a file called "${PF}_${VFNUM}". To solve this problem, we just
check for the existence of ${PF1}_${VFNUM} and ${PF2}_${VFNUM}, and
use whichever one we find (since we know that only one can be there)
2017-08-11 19:05:20 -04:00
Laine Stump
39d136b67b util: match phys_port_id when converting PF-netdev to/from VF-netdev
This patch updates functions in netdev.c to pay attention to
phys_port_id. It uses the new function virNetDevGetPhysPortID() to
learn the phys_port_id of a VF or PF, then sends that info to
virPCIGetNetName(), which has newly been modified to take an optional
phys_port_id.
2017-08-11 18:55:25 -04:00
Laine Stump
b3b5aa75ed util: make virPCIGetNetName() more versatile
A single PCI device may have multiple netdevs associated with it. Each
of those netdevs will have a different phys_port_id entry in
sysfs. This patch modifies virPCIGetNetName() to allow selecting one
of the potential many netdevs in two different ways:

1) by setting the "idx" argument, the caller can select the 1st (0),
2nd (1), etc. netdev from the PCI device's net subdirectory.

2) If the physPortID arg is set (to a null-terminated string) then
virPCIGetNetName() returns the netdev that has that phys_port_id in
the sysfs file of the same name in the netdev's directory.
2017-08-11 18:35:09 -04:00
Laine Stump
0dc67e6d2d util: Fix const'ness of 1st arg to virPCIGetNetName()
The first arg isn't modified in the function, so it should be const.
2017-08-11 18:30:14 -04:00
Laine Stump
48f33bb5df util: new function virNetDevGetPhysPortID()
On Linux each network device *can* (but not necessarily *does*) have
an attribute called phys_port_id which can be read from the file of
that name in the netdev's sysfs directory. The examples I've seen have
been a many-digit hexadecimal number (as an ASCII string).

This value can be useful when a single PCI device is associated with
multiple netdevs (e.g a dual port Mellanox SR-IOV NIC - this card has
a single PCI Physical Function (PF), and that PF has two netdevs
associated with it (the "net" subdirectory of the PF in sysfs has two
links rather than the usual single link to a netdev directory). Each
of the PF netdevs has a different phys_port_id. The Virtual Functions
(VF) are similar - the PF (a PCI device) has "n" VFs (also each of
these is a PCI device), each VF has two netdevs, and each of the VF
netdevs points back to the VF PCI device (with the "device" entry in
its sysfs directory) as well as having a phys_port_id matching the PF
netdev it is associated with.

virNetDevGetPhysPortID() simply attempts to read the phys_port_id for
the given netdev and return it to the caller. If this particular
netdev driver doesn't support phys_port_id, it returns NULL (*not* a
NULL-terminated string, but a NULL pointer) but still counts it as a
success.
2017-08-11 18:25:00 -04:00
Ján Tomko
e9f3222705 introduce virConfReadString
Rewrite virConfReadMem to take a null-terminated string.
All the callers were calling strlen on it anyway.
2017-08-08 12:19:17 +02:00
Peter Krempa
0b1ecf7b53 util: hash: Make virHashCodeGen mockable
Export the function from the util module so that dynamic linking can
override it.
2017-08-03 09:49:15 +02:00
Peter Krempa
8982f3ab20 util: hash: Include stdbool.h in the header file
The functions declared in virhash.h return bool, but stdbool.h was not
included.
2017-08-03 09:49:15 +02:00
Michal Privoznik
3e609bf4e4 virCgroupValidateMachineGroup: Don't free @machinename
We are given a string in @machinename, we never allocate it, just
merely use it for reading. We should not free it otherwise it
leads to double free:

==32191== Thread 17:
==32191== Invalid free() / delete / delete[] / realloc()
==32191==    at 0x4C2D1A0: free (vg_replace_malloc.c:530)
==32191==    by 0x54BBB84: virFree (viralloc.c:582)
==32191==    by 0x2BC04499: qemuProcessStop (qemu_process.c:6313)
==32191==    by 0x2BC500FF: processMonitorEOFEvent (qemu_driver.c:4724)
==32191==    by 0x2BC502FC: qemuProcessEventHandler (qemu_driver.c:4769)
==32191==    by 0x5550640: virThreadPoolWorker (virthreadpool.c:167)
==32191==    by 0x554FBCF: virThreadHelper (virthread.c:206)
==32191==    by 0x8F913D3: start_thread (in /lib64/libpthread-2.23.so)
==32191==    by 0x928DE3C: clone (in /lib64/libc-2.23.so)
==32191==  Address 0x31893d70 is 0 bytes inside a block of size 1,100 free'd
==32191==    at 0x4C2D1A0: free (vg_replace_malloc.c:530)
==32191==    by 0x54BBB84: virFree (viralloc.c:582)
==32191==    by 0x54C1936: virCgroupValidateMachineGroup (vircgroup.c:343)
==32191==    by 0x54C4B29: virCgroupNewDetectMachine (vircgroup.c:1550)
==32191==    by 0x2BBDDA29: qemuConnectCgroup (qemu_cgroup.c:972)
==32191==    by 0x2BC05DA7: qemuProcessReconnect (qemu_process.c:6822)
==32191==    by 0x554FBCF: virThreadHelper (virthread.c:206)
==32191==    by 0x8F913D3: start_thread (in /lib64/libpthread-2.23.so)
==32191==    by 0x928DE3C: clone (in /lib64/libc-2.23.so)
==32191==  Block was alloc'd at
==32191==    at 0x4C2BE80: malloc (vg_replace_malloc.c:298)
==32191==    by 0x4C2E35F: realloc (vg_replace_malloc.c:785)
==32191==    by 0x54BB492: virReallocN (viralloc.c:245)
==32191==    by 0x54BEDF2: virBufferGrow (virbuffer.c:150)
==32191==    by 0x54BF3B9: virBufferVasprintf (virbuffer.c:408)
==32191==    by 0x54BF324: virBufferAsprintf (virbuffer.c:381)
==32191==    by 0x55BB271: virDomainGenerateMachineName (domain_conf.c:27078)
==32191==    by 0x2BBD5B8F: qemuDomainGetMachineName (qemu_domain.c:9595)
==32191==    by 0x2BBDD9B4: qemuConnectCgroup (qemu_cgroup.c:966)
==32191==    by 0x2BC05DA7: qemuProcessReconnect (qemu_process.c:6822)
==32191==    by 0x554FBCF: virThreadHelper (virthread.c:206)
==32191==    by 0x8F913D3: start_thread (in /lib64/libpthread-2.23.so)

Moreover, make the @machinename 'const char *' to mark it
explicitly that we are not changing the passed string.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-08-01 11:37:51 +02:00
Peter Krempa
c61d169327 util: storagefile: rename 'nodebacking' to 'nodestorage' in virStorageSource
Make it less confusing by naming the field which refers to the storage
object as 'nodestorage'.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:44:05 +02:00
Peter Krempa
3c60388591 util: buffer: Add virBufferStrcatVArgs
Split out the worker loop into a separate function and export it.

Reviewed-by: Eric Blake <eblake@redhat.com>
2017-07-27 09:31:14 +02:00
Pavel Hrdina
ac3eb2ab24 util: introduce virFileCache
The new virFileCache will nicely handle the caching logic for any data
that we would like to cache.  For each type of data we will just need
to implement few handlers that will take care of creating, validating,
loading and saving the cached data.

The cached data must be an instance of virObject.

Currently we cache QEMU capabilities which will start using
virFileCache.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-26 15:31:25 +02:00
Martin Kletzander
eaf2c9f891 Move machineName generation from virsystemd into domain_conf
It is more related to a domain as we might use it even when there is
no systemd and it does not use any dbus/systemd functions.  In order
not to use code from conf/ in util/ pass machineName in cgroups code
as a parameter.  That also fixes a leak of machineName in the lxc
driver and cleans up and de-duplicates some code.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-25 17:02:27 +02:00
Michal Privoznik
2d3c7122c8 Revert "virthread: Introduce virRWLockInitPreferWriter"
This reverts commit 328bd24443.

As it turns out, this is not portable and very Linux & glibc
specific. Worse, this may lead to not starving writers on Linux
but everywhere else. Revert this and if the starvation occurs
resolve it.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
2017-07-25 10:56:03 +02:00
Michal Privoznik
77f4593b09 virobject: Introduce virObjectRWLockable
Up until now we only had virObjectLockable which uses mutexes for
mutually excluding each other in critical section. Well, this is
not enough. Future work will require RW locks so we might as well
have virObjectRWLockable which is introduced here.

Moreover, polymorphism is introduced to our code for the first
time. Yay! More specifically, virObjectLock will grab a write
lock, virObjectLockRead will grab a read lock then (what a
surprise right?). This has great advantage that an object can be
made derived from virObjectRWLockable in a single line and still
continue functioning properly (mutexes can be viewed as grabbing
write locks only). Then just those critical sections that can
grab a read lock need fixing. Therefore the resulting change is
going to be way smaller.

In order to avoid writer starvation, the object initializes RW
lock that prefers writers.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2017-07-24 15:54:06 +02:00
Michal Privoznik
328bd24443 virthread: Introduce virRWLockInitPreferWriter
We already have virRWLockInit. But this uses pthread defaults
which prefer reader to initialize the RW lock. This may lead to
writer starvation. Therefore we need to have the counterpart that
prefers writers. Now, according to the
pthread_rwlockattr_setkind_np() man page setting
PTHREAD_RWLOCK_PREFER_WRITER_NP attribute is no-op. Therefore we
need to use PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP
attribute. So much for good enum value names.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2017-07-24 15:54:06 +02:00
Peter Krempa
97ea8da183 virStorageNetHostDef: Turn @port into integer
Currently, @port is type of string. Well, that's overkill and
waste of memory. Port is always an integer. Use it as such.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-07-24 10:55:44 +02:00
Peter Krempa
8444419f8c util: storage: fill in default ports when parsing backing chain
Similarly to when parsing XML we need to fill in default ports for the
backing chain. This was missed in commit 5bda835466
2017-07-24 10:55:43 +02:00
Peter Krempa
1f920b9f02 util: uri: Convert port number to unsigned integer
Negative ports don't make sense so use a unsigned integer.
2017-07-24 10:55:43 +02:00
Peter Krempa
e8b69016b1 qemu: command: Rename and move qemuNetworkDriveGetPort
Move it to virstring.c and improve it to parse and validate ports. New
name is virStringParsePort.
2017-07-24 10:55:20 +02:00
Peter Krempa
a908e9e45e util: bitmap: Modify virBitmapSubtract to virBitmapIntersect
Since virBitmapSubtract is unused modify it to perform bitmap
intersection.
2017-07-20 16:14:50 +02:00
Antoine Millet
e484cb3eca Handle hotplug change on VLAN configuration using OVS
A new function virNetDevOpenvswitchUpdateVlan has been created to instruct
OVS of the changes. qemuDomainChangeNet has been modified to handle the
update of the VLAN configuration for a running guest and rely on
virNetDevOpenvswitchUpdateVlan to do the actual update if needed.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-07-20 15:15:03 +02:00
Antoine Millet
695611f99e virnetdevopenvswitch: Move OVS VLAN configuration to a separate function
This piece of code is going to be reused. So move it out to a
separate function.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-07-20 15:15:03 +02:00
Pavel Hrdina
38e516a524 util/virhash: add name parameter to virHashSearch
While searching for an element using a function it may be
desirable to know the element key for future operation.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2017-07-20 14:02:14 +02:00
Michal Privoznik
8ae82e676a virFileInData: Report an error if unable to reposition file
The purpose of this function is to tell if the current position
in given FD is in data section or a hole and how much bytes there
is remaining until the end of the section. This is achieved by
couple of lseeks(). The most important part is that we reposition
the FD back, so that the position is unchanged from the caller
POV. And until now the final lseek() back to the original
position was done with no check for errors. And I was convinced
that that's okay since nothing can go wrong. However, review
feedback from a related series persuaded me, that it's better to
be safe than sorry. Therefore, lets check if the final lseek()
succeeded and if it doesn't report an error.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2017-07-19 09:46:17 +02:00
Peter Krempa
9756884d14 conf: Pre-fill default ports when parsing network disk sources
Fill them in right away rather than having to figure out at runtime
whether they are necessary or not.

virStorageSourceNetworkDefaultPort does not need to be exported any
more.
2017-07-14 16:05:46 +02:00
Peter Krempa
5bda835466 util: storage: Fill in default ports for gluster and iscsi
Our documentation provides them, so the helper should return them.
2017-07-14 16:05:46 +02:00
Peter Krempa
34ffc2ff41 util: Extract helper to retrieve default port for network protocol
Make the stuff hardcoded in qemu a global helper so that other parts of
the code can determine the default port too.
2017-07-14 16:05:46 +02:00
Daniel P. Berrange
407a281a8e Revert "Prevent more compiler optimization of mockable functions"
This reverts commit e4b980c853.

When a binary links against a .a archive (as opposed to a shared library),
any symbols which are marked as 'weak' get silently dropped. As a result
when the binary later runs, those 'weak' functions have an address of
0x0 and thus crash when run.

This happened with virtlogd and virtlockd because they don't link to
libvirt.so, but instead just libvirt_util.a and libvirt_rpc.a. The
virRandomBits symbols was weak and so left out of the virtlogd &
virtlockd binaries, despite being required by virHashTable functions.

Various other binaries like libvirt_lxc, libvirt_iohelper, etc also
link directly to .a files instead of libvirt.so, so are potentially
at risk of dropping symbols leading to a later runtime crash.

This is normal linker behaviour because a weak symbol is not treated
as undefined, so nothing forces it to be pulled in from the .a You
have to force the linker to pull in weak symbols using -u$SYMNAME
which is not a practical approach.

This risk is silent bad linkage that affects runtime behaviour is
not acceptable for a fix that was merely trying to fix the test
suite. So stop using __weak__ again.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-07-13 13:07:06 +01:00
Martin Kletzander
1701ba6fdc util: Don't leak linksrc in vircgroup
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-07-13 13:14:23 +02:00
Juan Hernandez
dacd160d74 Avoid hidden cgroup mount points
Currently the scan of the /proc/mounts file used to find cgroup mount
points doesn't take into account that mount points may hidden by other
mount points. For, example in certain Kubernetes environments the
/proc/mounts contains the following lines:

  cgroup /sys/fs/cgroup/net_prio,net_cls cgroup ...
  tmpfs /sys/fs/cgroup tmpfs ...
  cgroup /sys/fs/cgroup/net_cls,net_prio cgroup ...

In this particular environment the first mount point is hidden by the
second one. The correct mount point is the third one, but libvirt will
never process it because it only checks the first mount point for each
controller (net_cls in this case). So libvirt will try to use the first
mount point, which doesn't actually exist, and the complete detection
process will fail.

To avoid that issue this patch changes the virCgroupDetectMountsFromFile
function so that when there are duplicates it takes the information from
the last line in /proc/mounts. This requires removing the previous
explicit condition to skip duplicates, and adding code to free the
memory used by the processing of duplicated lines.

Related-To: https://bugzilla.redhat.com/1468214
Related-To: https://github.com/kubevirt/libvirt/issues/4
Signed-off-by: Juan Hernandez <jhernand@redhat.com>
2017-07-13 09:37:52 +02:00
Daniel P. Berrange
e4b980c853 Prevent more compiler optimization of mockable functions
Currently all mockable functions are annotated with the 'noinline'
attribute. This is insufficient to guarantee that a function can
be reliably mocked with an LD_PRELOAD. The C language spec allows
the compiler to assume there is only a single implementation of
each function. It can thus do things like propagating constant
return values into the caller at compile time, or creating
multiple specialized copies of the function body each optimized
for a different caller. To prevent these optimizations we must
also set the 'noclone' and 'weak' attributes.

This fixes the test suite when libvirt.so is built with CLang
with optimization enabled.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-07-11 13:57:12 +01:00
Daniel P. Berrange
d8f8c7a83d Remove network constants out of internal.h
The HOST_NAME_MAX, INET_ADDRSTRLEN and VIR_LOOPBACK_IPV4_ADDR
constants are only used by a handful of files, so are better
kept in virsocketaddr.h or the source file that uses them.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-07-11 13:57:11 +01:00
Peter Krempa
6d7cdec63d util: storage: Always deflatten JSON pseudo-protocol objects
Now that the JSON deflattener is working sanely we can always attempt
the deflattening so that we can then parse the tree as expected.
2017-07-11 14:23:08 +02:00
Peter Krempa
428d175206 util: json: Recursively deflatten objects virJSONValueObjectDeflatten
If a value of the first level object contains more objects needing
deflattening which would be wrapped in an actual object the function
would not recurse into them.

By this simple addition we can fully deflatten the objects.
2017-07-11 14:20:05 +02:00
Peter Krempa
d40f4b3e67 util: json: Properly implement JSON deflattening
As it turns out sometimes users pass in an arbitrarily nested structure
e.g. for the qemu backing chains JSON pseudo protocol. This new
implementation deflattens now a single object fully even with nested
keys.

Additionally it's not necessary now to stick with the "file." prefix for
the properties.
2017-07-11 14:13:35 +02:00
Peter Krempa
f43b7d60d8 util: json: Don't remove the 'file' subobject when deflattening
Currently the function would deflatten the object by dropping the 'file'
prefix from the attributes. This does not really scale well or adhere to
the documentation.

Until we refactor the worker to properly deflatten everything we at
least simulate it by adding the "file" wrapper object back.
2017-07-11 14:10:31 +02:00
Peter Krempa
de75de7c97 util: Move JSON object deflattening code to json utility file
The code will become more universal so it makes more sense for it to
live with the rest of the JSON functions.
2017-07-11 14:02:28 +02:00
Peter Krempa
cadd96b3ea util: json: Add virJSONValueIsObject
Allows testing whether a virJSONValue is an object.
2017-07-11 14:02:28 +02:00
Cédric Bosdonnat
0980764dee util: share code between virExec and virCommandExec
virCommand is a version of virExec that doesn't fork, however it is
just calling execve and doesn't honors setting uid/gid and pwd.

This commit extrac those pieces from virExec() to a virExecCommon()
function that is called from both virExec() and virCommandExec().
2017-07-11 10:41:24 +02:00
Michal Privoznik
0fe4aa149f fdstream: Report error from the I/O thread
Problem with our error reporting is that the error object is a
thread local variable. That means if there's an error reported
within the I/O thread it gets logged and everything, but later
when the event loop aborts the stream it doesn't see the original
error. So we are left with some generic error. We can do better
if we copy the error message between the threads.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2017-07-11 08:41:01 +02:00
Michal Privoznik
3a2ca2fbe4 virfdstream: Check for thread error more frequently
When the I/O thread quits (e.g. due to an I/O error, lseek()
error, whatever), any subsequent virFDStream API should return
error too. Moreover, when invoking stream event callback, we must
set the VIR_STREAM_EVENT_ERROR flag so that the callback knows
something bad happened.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2017-07-11 08:40:13 +02:00
Peter Krempa
d077fbc221 util: netdevbridge: Refactor error handling in virNetDevBridgeCreate
Replace the switch statement with a simpler if statement. This also
removes the fallthrough path that coverity was complaining about.
2017-06-28 15:27:17 +02:00
Martin Kletzander
8110b4e073 util: Extract locale-related fixes into separate functions
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2017-06-26 09:20:08 +02:00
John Ferlan
5431055d2b util: Force reading of meta data to get encryption capacity value
https://bugzilla.redhat.com/show_bug.cgi?id=1371892

As it turns out the volume create, build, and refresh path was not peeking
at the meta data, so immediately after a create operation the value displayed
for capacity was still incorrect. However, if a pool refresh was done the
correct value was fetched as a result of a meta data peek.

The reason is it seems historically if the file type is RAW then peeking
at the file just took the physical value for the capacity. However, since
we know if it's an encrypted file, then peeking at the meta data will be
required in order to get a true capacity value.

So check for encryption in the source and if present, use the meta data
in order to fill in the capacity value and set the payload_offset.
2017-06-24 06:43:25 -04:00
Michal Privoznik
edaf135657 virNetDevOpenvswitchInterfaceStats: Be more forgiving when fetching stats
https://bugzilla.redhat.com/show_bug.cgi?id=1461270

When fetching stats for a vhost-user type of interface, we run
couple of ovs-vsctl commands and parse their output. However, not
all stats exist at all times, for instance "rx_dropped" or
"tx_errors" can be missing. Thing is, we ask for a bulk of
statistics and if one of them is missing an error is reported
instead of returning the rest. Since we ignore errors, we fail to
set statistics. Fix this by asking for each piece alone.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-06-23 11:51:23 +02:00
John Ferlan
10c2bb2b19 util: Introduce virObjectGetLockableObj
Split out the object fetch in virObject{Lock|Unlock} into a helper

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-06-22 09:34:40 -04:00
John Ferlan
209a95e354 util: Formatting cleanups to virobject API
Alter to use more recent formatting guidelines

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-06-22 09:34:40 -04:00
Martin Kletzander
c9d1e5951c util: Move locale.h include from virutil to virstring
Commit 5c54d29aae forgot to do that when moving the only function
using it and it broke the build on some platforms.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-06-22 14:30:27 +02:00
Julio Faracco
96a9b9a7f0 util: fix locale problem with virStrToDouble().
This commit fixes a locale problem with locales that use comma as a mantissa
separator. Example: 12.34 en_US = 12,34 pt_BR. Since strtod() is a non-safe
function, virStrToDouble() will have problems to parse double numbers from
kernel settings and other double numbers from static files (XMLs, JSONs, etc).

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1457634
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1457481

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
2017-06-22 11:30:20 +02:00
Julio Faracco
5c54d29aae util: moving virDoubleToStr() from virutil to virstring.
The function virDoubleToStr() is defined in virutil.* and virStrToDouble() is
defined in virstring.*. Joining both functions into the same file makes more
sense.

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
2017-06-22 11:30:20 +02:00
Peter Krempa
296a53313f util: storage: Make @backingFormat optional in virStorageFileGetMetadataInternal
Some callers don't need to know the backing format. Make the argument
optional by using a dummy int if NULL is passed.
2017-06-20 16:50:26 +02:00
Peter Krempa
e4c3eff70e util: storage: Export virStorageIsRelative 2017-06-20 13:25:55 +02:00
Peter Krempa
b16133b114 util: storage: adapt to changes in JSON format for sheepdog
Since qemu 2.9 the options changed from a monolithic string into fine
grained options for the json pseudo-protocol object.
2017-06-20 08:40:18 +02:00
Peter Krempa
ea2c418ac3 util: storage: adapt to changes in JSON format for ssh
Since qemu 2.9 the options changed from a monolithic string into fine
grained options for the json pseudo-protocol object.
2017-06-20 08:40:18 +02:00
Peter Krempa
4fac5a1935 util: storage: adapt to changes in JSON format for ceph/rbd
Since qemu 2.9 the options changed from a monolithic string into fine
grained options for the json pseudo-protocol object.
2017-06-20 08:40:18 +02:00
Peter Krempa
35d23f90b2 util: storage: adapt to changes in JSON format for NBD
Since 2.9 the host and port for NBD are no longer directly under the
json pseudo-protocol object, but rather belong to a sub-object called
'server'.
2017-06-20 08:40:18 +02:00
Peter Krempa
b24bc54080 util: storage: Add JSON parser for new options in iSCSI protocol
Starting from qemu 2.9, more granular options are supported. Add parser
for the relevant bits.

With this patch libvirt is able to parse the host and target IQN of from
the JSON pseudo-protocol specification.

This corresponds to BlockdevOptionsIscsi in qemu qapi.
2017-06-20 08:40:18 +02:00
Peter Krempa
299aff7e0c util: storage: Report errors when source host data is missing
Merge the reporting of the missing source host data into the parser
functions so that callers don't have to do it separately.
2017-06-20 08:40:18 +02:00
Peter Krempa
49ed98a457 util: storage: Split out parsing of TCP network host from JSON pseudoprotocol
Few backing protocols support only TCP. Split out the function which
will correspond to parsing qemu's InetSocketAddressBase.
2017-06-20 08:40:18 +02:00
Peter Krempa
1f915d40a2 util: storage: Add support for type 'inet' in virStorageSourceParseBackingJSONSocketAddress
'SocketAddress' structure was changed to contain 'inet' instead of
'tcp' since qemu commit c5f1ae3ae7b. Existing entries have a backward
compatibility layer.

Libvirt will parse 'inet' and 'tcp' as equivalents.
2017-06-20 08:40:18 +02:00
Peter Krempa
6402f402d4 util: storage: make virStorageSourceParseBackingJSONGlusterHost universal
The same json strucutre is used for NBD and sheepdog volumes for
specifying of the host. Rename the function and fix up error messages to
be more universal.
2017-06-19 15:50:29 +02:00
Peter Krempa
506b80c84e util: storage: Add missing return to virStorageSourceParseBackingJSONGluster
If the number of servers is not expected the code would report an error
but would not return failure.
2017-06-19 15:50:29 +02:00
Peter Krempa
236e1f7e8c util: storage: Output parsed network backing store string to debug log 2017-06-19 15:50:29 +02:00
Martin Kletzander
307a205e25 qemu: Allow live-updates of coalesce settings
Change the settings from qemuDomainUpdateDeviceLive() as otherwise the
call would succeed even though nothing has changed.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1414627

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-06-16 10:18:35 +02:00
Daniel P. Berrange
5e9ca5508d Use sys/uio.h for writev()
With glibc >= 2.25.90 writev() is only available if you explicitly
include sys/uio.h. This matches the documented requirements, but
older glibc and other *NIX pulled in writev indirectly so the bug
wasn't noticed previously.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-06-14 15:01:42 +01:00
Marc Hartmayer
adf846d3c9 Use ATTRIBUTE_FALLTHROUGH
Use ATTRIBUTE_FALLTHROUGH, introduced by commit
5d84f5961b, instead of comments to
indicate that the fall through is an intentional behavior.

Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Reviewed-by: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Reviewed-by: Bjoern Walk <bwalk@linux.vnet.ibm.com>
2017-06-12 19:11:30 -04:00
Pavel Hrdina
9fd816ed33 Revert "util: virqemu: introduce virQEMUBuildBufferEscape"
This reverts commit 22b02f4492.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2017-06-12 12:45:42 +02:00
Michal Privoznik
f3908d8557 virNetDevOpenvswitchGetVhostuserIfname: Fix off by one error
https://bugzilla.redhat.com/show_bug.cgi?id=1459091

We try to get the last element of the passed path by calling
strrch(path, '/'). However, the pointer that strrchr() returns
points at the slash, We want string that starts right after that.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-06-08 15:02:23 +02:00
Michal Privoznik
5004f121bc virFDStreamThread: Make sure we won't exceed @length
There's a problem with current streams after I switched them from
iohelper to thread implementation. Previously, iohelper made sure
not to exceed specified @length resulting in the pipe EOF
appearing at the exact right moment (the pipe was used to tunnel
the data from the iohelper to the daemon). Anyway, when switching
to thread I had to write the I/O code from scratch. Whilst doing
that I took an inspiration from the iohelper code, but since the
usage of pipe switched to slightly different meaning, there was
no 1:1 relationship between the codes.

Moreover, after introducing VIR_FDSTREAM_MSG_TYPE_HOLE, the
condition that should made sure we won't exceed @length was
completely wrong.

The fix is to:

a) account for holes for @length
b) cap not just data sections but holes too (if @length would be
exceeded)

For this purpose, the condition needs to be brought closer to the
code that handles holes and data sections.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-06-05 17:00:48 +02:00
Julio Faracco
4fd5c2fbce util: remove dead code inside virstoragefile
The host address or the socket path have already been checked at the
begining of the function virStorageSourceParseNBDColonString(). So,
when the parameter is not a unix socket, there is no reason to check
the address again because if it does not exists, the logic will fail
in the first IF conditional.

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
2017-06-02 09:39:10 +02:00
Julio Faracco
54aee01d87 util: fix wrong comparison inside virStoragePermsCopy()
VIR_STRDUP returns -1 if the string copy was not successful. So, the
current comparison/logic is throwing an error when VIR_STRDUP() returns
1. Only when source is NULL, it is considering as a success which is
not right.

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
2017-06-02 09:37:44 +02:00
Nitesh Konkar
4ae0f65669 util: hostcpu: Correctly report total number of vcpus in virHostCPUGetMap
Callers expect the return value to be the total number of vcpus in the
host (including offline vcpus). The refactor in c67e04e25f
broke this assumption by using virHostCPUGetOnlineBitmap which only
creates a bitmap long enough to hold the last online vcpu.

Report the full number of host vcpus by returning value from
virHostCPUGetCount().

Signed-off-by: Nitesh Konkar <nitkon12@linux.vnet.ibm.com>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2017-05-30 10:42:28 +02:00
Roman Bogorodskiy
981e2c7097 util: fix virfcp build on non-Linux
- Include virerror.h for virReportSystemError
 - Rename stub functions to match original function names
2017-05-26 20:00:51 +04:00
Bjoern Walk
c47a3b130d util: helper functions for fibre channel devices
We will need some convenient helper functions for managing sysfs-entries
for fibre channel-backed devices. Let's implement them and make them
available in the private API.

Signed-off-by: Bjoern Walk <bwalk@linux.vnet.ibm.com>
2017-05-26 10:44:05 -04:00
Yi Wang
9f4c39f309 util: fix memory leak in virSocketAddrFormatFull
The @ipv6_host allocated in virAsprintf may be lost when virAsprintf
addrstr failed.

Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
2017-05-26 12:15:13 +02:00
Chen Hanxiao
a495e3f9ea util: display leading zeros of USB vendor/product id's in log messages
Many vendor id's and product id's have leading zeros.  We should show
them in the logs.

Signed-off-by: Chen Hanxiao <chenhanxiao@gmail.com>
Reviewed-by: Laine Stump <laine@laine.org>
2017-05-24 12:23:59 -04:00
Jim Fehlig
975ea20f85 maint: define a macro for IPv4 loopback address
Use a macro instead of hardcoding "127.0.0.1" throughout the
sources.
2017-05-22 10:20:27 -06:00
Wang King
c4a4c01e6e util: Don't leak @reply in virSystemdGetMachineNameByPID
@reply is a DBusMessage object returned by virDBusCallMethod in
get machine object call path, dereference it before calling
virDBusCallMethod again to get machine name.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-22 06:48:01 +02:00
Michal Privoznik
ab26790f07 virfile: Provide stub for virFileInData
Some older systems (such as RHEL6) lack SEEK_HOLE and SEEK_DATA
which virFileInData relies on. Provide a stub for these systems.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-19 14:02:37 +02:00
Michal Privoznik
0da4a635bc virStream: Forbid negative seeks
Currently, we don't assign any meaning to that. Our current view
on virStream is that it's merely a pipe. And pipes don't support
seeking.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 15:05:18 +02:00
Wang King
506acac87f util: Do not leak @handles in stop netlink event service
Commit e3ba4025 introduced srv->handles and VIR_RESIZE_N to allocate
@handles as necessary, but did not free the handles during when calling
virNetlinkEventServiceStop.
2017-05-18 07:26:05 -04:00
Wang King
c886b5d153 util: Deduplicate code in virNetlinkEventServiceStopAll
Commit 15a71e60 introduced the virNetlinkEventServiceStopAll function, and
the code in virNetlinkEventServiceStop is copied to this function, so just
call virNetlinkEventServiceStop instead.
2017-05-18 07:26:05 -04:00
Erik Skultety
3a2a2a7401 mdev: Pass a uuidstr rather than an mdev object to some util functions
Namely, this patch is about virMediatedDeviceGetIOMMUGroup{Dev,Num}
functions. There's no compelling reason why these functions should take
an object, on the contrary, having to create an object every time one
needs to query the IOMMU group number, discarding the object afterwards,
seems odd.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2017-05-18 12:20:15 +02:00
Marek Marczykowski-Górecki
1128769f9e pci: fix link maximum speed detection
Commit 8e09663 "pci: recognize/report GEN4 (PCIe 4.0) card 16GT/s Link
speed" introduced another speed into enum, but mistakenly also altered
field width, so one bit of link width was included there.

Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
2017-05-18 10:45:58 +02:00
Michal Privoznik
895479647b fdstream: Implement sparse stream
Basically, what is needed here is to introduce new message type
for the messages passed between the event loop callbacks and the
worker thread that does all the I/O. The idea is that instead of
a queue of read buffers we will have a queue where "hole of size
X" messages appear. That way the event loop callbacks can just
check the head of the queue and see if the worker thread is in
data or a hole section and how long the section is.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
b928d140f4 util: Introduce virFileInData
This function takes a FD and determines whether the current
position is in data section or in a hole. In addition to that,
it also determines how much bytes are there remaining till the
current section ends.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
07c2399c01 virfdstream: Use messages instead of pipe
One big downside of using the pipe to transfer the data is that
we can really transfer just bare data. No metadata can be carried
through unless some formatted messages are introduced. That would
be quite painful to achieve so let's use a message queue. It's
fairly easy to exchange info between threads now that iohelper is
no longer used.

The reason why we cannot use the FD for plain files directly is
that despite us setting noblock flag on the FD, any
read()/write() blocks regardless (which is a show stopper since
those parts of the code are run from the event loop) and poll()
reports such FD as always readable/writable - even though the
subsequent operation might block.

The pipe is still not gone though. It is used to signal the event
loop that an event occurred (e.g. data is available for reading
in the queue, or vice versa).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Andrea Bolognani
5645badd1f gic: Remove VIR_GIC_VERSION_DEFAULT
The QEMU default is GICv2, and some of the code in libvirt
relies on the exact value. Stop pretending that's not the
case and use GICv2 explicitly where needed.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-05-16 16:48:30 +02:00
Peter Krempa
14789b7ea8 util: conf: Don't log when adding commented out lines
virConfAddEntry spams debug logs even for fully commented out lines.
Skip such messages to avoid:

2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
...

This also fixes NULL passed to printf.
2017-05-15 13:51:25 +02:00
Daniel P. Berrange
3213e369c3 Detect VMDK version 3 files
The metadata libvirt cares about is identical for version 3
as for previous versions, so we merely need list the new
version number.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-05-15 12:09:04 +01:00
Pavel Hrdina
0918b84968 util: introduce virBufferEscapeRegex
Add a helper to escape all possible meta-characters used for
POSIX extended regular expressions.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-05-12 16:54:33 +02:00
Pavel Hrdina
f15f155403 util: introduce virStringMatch
Simply tries to match the provided regex on a string and returns
the result.  Useful if caller don't care about the matched substring
and want to just test if some pattern patches a string.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-05-12 16:51:18 +02:00