Commit Graph

29525 Commits

Author SHA1 Message Date
Ján Tomko
799011bbe7 vircrypto: Rely on GnuTLS for hash functions
Ditch the use of gnulib's digest functions in favor of GnuTLS,
which might be more likely to get FIPS-certified.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 14:05:21 +02:00
Ján Tomko
c038a3cfed esx: Use VIR_CRYPTO_HASH_SIZE_MD5
Do not rely on gnulib's MD5_DIGEST_SIZE from md5.h.

Include vircrypto.h and use VIR_CRYPTO_HASH_SIZE_MD5.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 14:05:21 +02:00
Ján Tomko
3ec34fd7a3 esx: use virCryptoHashBuf
Instead of using md5_buffer from gnulib directly.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 14:05:21 +02:00
Ján Tomko
0eeedd61a9 Introduce virCryptoHashBuf
A function that keeps the hash in binary form instead of converting
it to human-readable hexadecimal form.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 13:57:45 +02:00
Ján Tomko
95ba1c2f6e vircrypto: provide constants for hash sizes
The callers needing to know the size of the resulting digest
rely on _DIGEST_SIZE constants from gnulib.

Introduce VIR_CRYPTO_HASH_SIZE_ constants to remove the dependency.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 13:55:47 +02:00
Andrea Bolognani
57b8df9219 travis: Uninstall packages before upgrade
numpy (needed by cgal) started having the same issue with
linking as python, which makes upgrade and thus the entire
build fail on macOS.

Instead of playing more tricks with linking/unlinking, just
uninstall the problematic packages (and those dragging them
in) before doing anything else.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-14 12:33:04 +02:00
Michal Privoznik
2c4affd57e qemu: Implement memoryBacking/discard
https://bugzilla.redhat.com/show_bug.cgi?id=1480668

QEMU has this new feature memory-backend-file.discard-data=yes
which is a nifty optimization. Basically, when qemu is quitting
or on memory hotplug it calls munmap() and close() on the file
that is backing the memory. However, this does not mean kernel
won't stop touching that part of memory. It still might. With
this feature enabled we tell kernel: "we don't need this memory
nor data stored in it". This makes kernel drop the memory
immediately without trying to sync memory with the mapped file.

Unfortunately, this cannot be turned on by default because we
can't be sure when users really don't care about what happens to
data after qemu dies. So it has to be opt-in. As usual, there are
three places where one can configure memory attributes. This
patch adds the feature to all of them.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 09:42:21 +02:00
Michal Privoznik
2300c92fe0 conf: Introduce memoryBacking/discard
QEMU has possibility to call madvise(.., MADV_REMOVE) in some
cases. Expose this feature to users by new element/attribute
discard.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 09:42:20 +02:00
Michal Privoznik
0329075733 conf: Move virDomainMemtune formatting into a separate function
At the same time convert the code to use virXMLFormatElement.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 09:42:20 +02:00
Michal Privoznik
72c1770aa0 qemu_capabilities: Introduce QEMU_CAPS_OBJECT_MEMORY_FILE_DISCARD
This capability tracks if memory-backend-file has discard-data
attribute or not.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 09:42:20 +02:00
Michal Privoznik
8a94501e8c qemu_capabilities: Introduce QEMU_CAPS_QOM_LIST_PROPERTIES
This capability tracks if qemu has "qom-list-properties" monitor
command.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 09:42:20 +02:00
Michal Privoznik
d81cf424ca qemu_monitor: Introduce qemuMonitorGetObjectProps
Now that we've gotten rid of misleading names we can introduce
qemuMonitorGetObjectProps() function which queries -object
properties. Again, some parts of code can be reused.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 09:42:20 +02:00
Michal Privoznik
036120209b qemuMonitorJSONGetDeviceProps: Separate props processing
The code that processes list of device properties is going to be
reused. Therefore put it into a separate function.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 09:42:20 +02:00
Clementine Hayat
920a7d3336 bhyve: start using virDomainObjCheckActive
Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 08:58:33 +02:00
Clementine Hayat
3c311d7e9b lxc: start using virDomainObjCheckActive
Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 08:58:33 +02:00
Clementine Hayat
48d44efcc0 qemu: start using virDomainObjCheckActive
Signed-off-by: Clementine Hayat <clem@lse.epita.fr>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 08:58:33 +02:00
Ján Tomko
e8c0558414 virutil.c: do not include stdarg.h
Last functions using it were moved to virfile.c in commit <bfe7721>.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 08:41:58 +02:00
Ján Tomko
d80881db3f virutil.c: do not include ioctl.h
Added by commit 61674cc, but we have started using
set_nonblocking_flag from gnulib in commit da3c4714

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-14 08:41:49 +02:00
Fabian Freyer
51c030f839 bhyve: document support for wiring guest memory
Signed-off-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Reviewed-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
2018-05-13 13:38:47 +04:00
Fabian Freyer
cb434b442a bhyve: add tests for wiring memory
Signed-off-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Reviewed-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
2018-05-13 13:38:31 +04:00
Fabian Freyer
b181e44d9f bhyve: add support for wiring memory
The <memoryBacking><locked/></memoryBacking> element will now pass the
wired (-S) flag to the bhyve command.

Signed-off-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Reviewed-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
2018-05-13 13:38:23 +04:00
Eric Blake
65dd2ac92d maint: typo fix in VIR_MIGRATE_PARAM_URI
s/filed/field/

Signed-off-by: Eric Blake <eblake@redhat.com>
2018-05-11 15:34:23 -05:00
Daniel P. Berrangé
8ccee910f5 log: update docs for daemons to improve user understanding
Strongly recommend against use of the log_levels setting since it
creates overly verbose logs and has a serious performance impact.

Describe the log filter syntax better and mention use of shell
glob syntax. Also provide more realistic example of good settings
to use. The libvirtd example is biased towards QEMU, but when the
drivers split off each daemon can get its own more appropriate
example.

Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-11 17:11:46 +01:00
Daniel P. Berrangé
cbb0fd3cfd log: support logging using shell wildcard syntax
Rather than specialcasing handling of the '*' character, use fnmatch()
to get normal shell wildcard syntax, as described in 'man glob(7)'.

To get an indication of the performance impact of using globs instead
of plain string matches, a test program was written. The list of all
260 log categories was extracted from the source. Then a typical log
filters setup was picked by creating an array of the strings "qemu",
"security", "util", "cgroup", "event", "object". Every filter string
was matched against every log category. Timing information showed that
using strstr() this took 8 microseconds, while fnmatch() took 114
microseconds.

IOW, fnmatch is 14 times slower than our existing strstr check. These
numbers show a worst case scenario that will never be hit, because it
is rare that every log category would have data output. The log category
matches are cached, so each category is only checked once no matter how
many log statements are emitted. IOW despite being slower, this will
be lost in the noise and have no consequence on real world logging
performance.

Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-11 17:08:06 +01:00
Daniel P. Berrangé
4a239d1509 log: rename virLogFlags to virLogFilterFlags to match docs
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-11 17:06:55 +01:00
Michal Privoznik
ebad530e9f virbuffer: Set child buffer indent properly
There's this macro virBufferSetChildIndent which sets offset of
child buffer from given parent buffer. However, it is calling
virBufferAdjustIndent() which only adds adjustment instead of
calling virBufferSetIndent() which clears out any adjustment
previously set.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-11 17:04:12 +02:00
Ján Tomko
b3f75d9980 tests: replace references to yajl
Use "libvirt not compiled with JSON support" instead of mentioning
yajl specifically.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-11 13:28:35 +02:00
Ján Tomko
50b4e9afe2 virJSONValue: remove unused 'protect' property
The last usage was removed by commit <167028e>

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-11 13:23:06 +02:00
Lin Ma
f33d02c11f virsh: Don't skip the first entry when the typed command share same prefix
Signed-off-by: Lin Ma <lma@suse.com>
2018-05-11 13:13:22 +02:00
Julio Faracco
c9da6cbec9 rpc: replacing ssh_get_publickey() by ssh_get_server_publickey().
After version 0.7.5, libssh deprecated the function scope
ssh_get_publickey() and moved to ssh_get_server_publickey(). So, Libvirt
is failing to compile using this new function name.

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2018-05-11 10:38:17 +02:00
Julio Faracco
463fa9c762 m4: checking if ssh_get_server_publickey() exists.
This commit adds some checks inside libssh m4 checking to verify if
ssh_get_server_publickey is available. This new function scope replaces
the old ssh_get_publickey() from libssh 0.7.5 and below. Assuming that
some distros are not showing the right version of libssh. This is a
simple way to check which function is available.

Signed-off-by: Julio Faracco <jcfaracco@gmail.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2018-05-11 10:38:08 +02:00
Michal Privoznik
b0cd8045f0 qemu: Detect pr-manager-helper capability
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:26:47 +02:00
Michal Privoznik
eba6467fed qemu_hotplug: Hotunplug of reservations
If we are the last one to use pr-manager object we need to remove
it and also kill the qemu-pr-helper process.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:26:47 +02:00
Michal Privoznik
3f968fda7b qemu_hotplug: Hotplug of reservations
When attaching a disk that requires pr-manager we might need to
plug the pr-manager object and start the pr-helper process.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:26:47 +02:00
Michal Privoznik
053d9e30e7 qemu: Start PR daemon on domain startup
Before we exec() qemu we have to spawn pr-helper processes for
all managed reservations (well, technically there can only one).
The only caveat there is that we should place the process into
the same namespace and cgroup as qemu (so that it shares the same
view of the system). But we can do that only after we've forked.
That means calling the setup function between fork() and exec().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:26:38 +02:00
Michal Privoznik
8be74af168 qemu: Introduce pr_helper to qemu.conf
Just like we allow users overriding path to bridge-helper
detected at compile time we can allow them to override path to
qemu-pr-helper.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:02:56 +02:00
Michal Privoznik
d13179fe8d qemu_cgroup: Allow /dev/mapper/control for PR
Just like in previous commit, qemu-pr-helper might want to open
/dev/mapper/control under certain circumstances. Therefore we
have to allow it in cgroups.

The change virdevmapper.c might look spurious but it isn't. After
6dd84f6850 any path that we're allowing in deivces CGroup is
subject to virDevMapperGetTargets() inspection. And libdevmapper
returns ENXIO for the path from subject.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:02:56 +02:00
Michal Privoznik
5bf89434ff qemu_ns: Allow /dev/mapper/control for PR
If qemu-pr-helper is compiled with multipath support the first
thing it does is open /dev/mapper/control. Since we're going
to be running it inside qemu namespace we need to create it
there. Unfortunately, we don't know if it was compiled with or
without multipath so we have to create it anyway.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:02:56 +02:00
Michal Privoznik
13fe558fb4 qemu: Generate pr cmd line at startup
For command line we need two things:

1) -object pr-manager-helper,id=$alias,path=$socketPath
2) -drive file.pr-manager=$alias

In -object pr-manager-helper we tell qemu which socket to connect
to, then in -drive file-pr-manager we just reference the object
the drive in question should use.

For managed PR helper the alias is always "pr-helper0" and socket
path "${vm->priv->libDir}/pr-helper0.sock".

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:02:56 +02:00
Michal Privoznik
3c28602759 qemu: Introduce pr-manager-helper capability
The capability tracks if qemu has pr-manager-helper object. At
this time don't actually detect if qemu has the capability. Not
just yet. Only after the code is written the feature will be
enabled.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:02:56 +02:00
Michal Privoznik
c7c9dea0a0 qemuDomainDiskChangeSupported: Deny changing reservations
Couple of reasons for that:

a) there's no monitor command to change path where the pr-helper
connects to, or
b) there's no monitor command to introduce a new pr-helper for a
disk that already exists.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:02:56 +02:00
Michal Privoznik
687730540e virstoragefile: Introduce virStoragePRDef
This is a definition that holds information on SCSI persistent
reservation settings. The XML part looks like this:

  <reservations enabled='yes' managed='no'>
    <source type='unix' path='/path/to/qemu-pr-helper.sock' mode='client'/>
  </reservations>

If @managed is set to 'yes' then the <source/> is not parsed.
This design was agreed on here:

https://www.redhat.com/archives/libvir-list/2017-November/msg01005.html

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
2018-05-11 09:02:56 +02:00
John Ferlan
4a3d6ed5ee util: Clean up consumers of virJSONValueArraySize
Rather than have virJSONValueArraySize return a -1 when the input
is not an array and then splat an error message, let's check for
an array before calling and then change the return to be a size_t
instead of ssize_t.

That means using the helper virJSONValueIsArray as well as using a
more generic error message such as "Malformed <something> array".
In some cases we can remove stack variables and when we cannot,
those variables should be size_t not ssize_t. Alter a few references
of if (!value) to be if (value == 0) instead as well.

Some callers can already assume an array is being worked on based
on the previous call, so there's less to do.

Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2018-05-10 14:59:15 -04:00
Roland Schulz
91234b0573 qemu: Remove parameter 'driver' from qemuBlockJobUpdate
The pointer to the qemu driver is already included in domain object's
private data, so does not need to be passed as yet another parameter
when the domain object is already passed.

Also removes parameter 'driver' from functions which had it just because of
qemuBlockJobUpdate.

Signed-off-by: Roland Schulz <schullzroll@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
2018-05-10 16:43:24 +02:00
Lin Ma
919bb45b56 virsh: Apply macro for current VSH_OT_ARGV "domain" options
Signed-off-by: Lin Ma <lma@suse.com>
2018-05-10 10:01:15 +02:00
Lin Ma
90396d795a virsh: Create macros for VSH_OT_ARGV "domain" option
Signed-off-by: Lin Ma <lma@suse.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-05-10 10:01:15 +02:00
Lin Ma
0d1c1a74b7 vshReadlineParse: Ignore vshReadlineOptionsGenerator for VSH_OT_ARGV options
Currently the VSH_OT_ARGV options don't support complete, But some of
VSH_OT_ARGV options are gonna support complete in upcoming patches.

Once applied the upcoming completion patches for VSH_OT_ARGV options, If
we don't ignore VSH_OT_ARGV here, The vshReadlineOptionsGenerator will
be called, Hence complete output will consist of the result by command
completer + the result by option completer, It's confusing.
e.g.
    $ virsh domstats --domain <TAB><TAB>
    --backing     --interface      --list-paused      --perf      --vcpu
    --balloon     leap42.3         --list-persistent  --raw       win10
    --block       --list-active    --list-running     sles12sp3
    --cpu-total   --list-inactive  --list-shutoff     sles15
    --enforce     --list-other     --list-transient   --state

After this patch and the upcoming completion patches:
    $ virsh domstats --domain <TAB><TAB>
    leap42.3    sles12sp3    sles15    win10

Signed-off-by: Lin Ma <lma@suse.com>
2018-05-10 10:01:15 +02:00
Lin Ma
ee6113aaea vshReadlineOptionsGenerator: Add already provided VSH_OT_ARGV options to list
It's helpful for users while they type certain kind of VSH_OT_ARGV options.
e.g.

$ virsh domstats --domain sles12sp3 --d<TAB>

Signed-off-by: Lin Ma <lma@suse.com>
2018-05-10 10:01:15 +02:00
Lin Ma
d86531daf2 virsh: Apply macro for current VSH_OT_STRING "domain" options
These VSH_OT_STRING "domain" options support domain name completion now.

Signed-off-by: Lin Ma <lma@suse.com>
2018-05-10 09:21:13 +02:00
Lin Ma
4be3b50dcc virsh: Create macros for VSH_OT_STRING "domain" option
Signed-off-by: Lin Ma <lma@suse.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2018-05-10 09:21:04 +02:00