It was reported that the performance of tunnelled migration and
volume upload/download regressed in 6.9.0, when the virt-ssh-helper
is used for remote SSH tunnelling instead of netcat.
When seeing data available to read from stdin, or the socket,
the current code will allocate at most 1k of extra space in
the buffer it has.
After writing data to the socket, or stdout, if more than 1k
of extra space is in the buffer, it will reallocate to free
up that space.
This results in a huge number of mallocs when doing I/O, as
well as a huge number of syscalls since at most 1k of data
will be read/written at a time.
Also if writing blocks for some reason, it will continue to
read data with no memory bound which is bad.
This changes the code to use a 1 MB fixed size buffer in each
direction. If that buffer becomes full, it will update the
watches to stop reading more data. It will never reallocate
the buffer at runtime.
This increases the performance by orders of magnitude.
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Tested-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
When the host is shutting down then we get PrepareForShutdown
signal on DBus to which we react by creating a thread which
runs virStateStop() and thus qemuStateStop(). But if scheduling
the thread is delayed just a but it may happen that we receive
SIGTERM (sent by systemd) to which we respond by quitting our
event loop and cleaning up everything (including drivers). And
only after that the thread gets to run only to find qemu_driver
being NULL.
What we can do is to delay exiting event loop and join the thread
that's executing virStateStop(). If the join doesn't happen in
given timeout (currently 30 seconds) then libvirtd shuts down
forcefully anyways (see virNetDaemonRun()).
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1895359
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1739564
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
When introducing the API I've mistakenly used 'int' type for
@nkeys argument which does nothing more than tells the API how
many items there are in @keys array. Obviously, negative values
are not expected and therefore 'unsigned int' should have been
used.
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Since both APIs accept/return an array of strings we can't have
client/server dispatch code generated. But implementation is
fairly trivial, although verbose.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
When restarting libvirt services and sockets *and* libvirt-guests.service
is running, the latter will sometimes hang when trying to connect to
libvirtd. Even though libvirt-guests has 'Wants=libvirtd.service' and
'After=libvirtd.service', we can see via journalctl that it is not
shutdown before libvirtd when executing something like
systemctl try-restart libvirtd.service libvirtd.socket \
libvirtd-ro.socket virtlockd.service virtlockd.socket \
virtlogd.service virtlogd.socket virt-guest-shutdown.target
Oct 28 15:53:31 systemd[1]: Stopping Virtualization daemon...
Oct 28 15:53:31 systemd[1]: libvirtd.service: Succeeded.
Oct 28 15:53:31 systemd[1]: Stopped Virtualization daemon.
Oct 28 15:53:31 systemd[1]: libvirtd-admin.socket: Succeeded.
Oct 28 15:53:31 systemd[1]: Closed Libvirt admin socket.
Oct 28 15:53:31 systemd[1]: Stopping Libvirt admin socket.
Oct 28 15:53:31 systemd[1]: libvirtd-ro.socket: Succeeded.
Oct 28 15:53:31 systemd[1]: Closed Libvirt local read-only socket.
Oct 28 15:53:31 systemd[1]: Stopping Libvirt local read-only socket.
Oct 28 15:53:31 systemd[1]: libvirtd.socket: Succeeded.
Oct 28 15:53:31 systemd[1]: Closed Libvirt local socket.
Oct 28 15:53:31 systemd[1]: Stopping Libvirt local socket.
Oct 28 15:53:31 systemd[1]: Listening on Libvirt local socket.
Oct 28 15:53:31 systemd[1]: Listening on Libvirt admin socket.
Oct 28 15:53:31 systemd[1]: Listening on Libvirt local read-only socket.
Oct 28 15:53:31 systemd[1]: virtlockd.socket: Succeeded.
Oct 28 15:53:31 systemd[1]: Closed Virtual machine lock manager socket.
Oct 28 15:53:31 systemd[1]: Stopping Virtual machine lock manager socket.
Oct 28 15:53:31 systemd[1]: Listening on Virtual machine lock manager socket.
Oct 28 15:53:31 systemd[1]: virtlogd.socket: Succeeded.
Oct 28 15:53:31 systemd[1]: Closed Virtual machine log manager socket.
Oct 28 15:53:31 systemd[1]: Stopping Virtual machine log manager socket.
Oct 28 15:53:31 systemd[1]: Listening on Virtual machine log manager socket.
Oct 28 15:53:31 systemd[1]: Stopping Suspend/Resume Running libvirt Guests...
In this case, the try-restart command hung and libvirt-guests was stuck
trying to connect to libvirtd. In the following case, the try-restart
worked since libvirtd was started again before libvirt-guests was stopped!
Oct 28 15:19:02 systemd[1]: Stopping Virtualization daemon...
Oct 28 15:19:02 systemd[1]: Stopped Virtualization daemon.
Oct 28 15:19:02 systemd[1]: Closed Libvirt admin socket.
Oct 28 15:19:02 systemd[1]: Stopping Libvirt admin socket.
Oct 28 15:19:02 systemd[1]: Closed Virtual machine lock manager socket.
Oct 28 15:19:02 systemd[1]: Stopping Virtual machine lock manager socket.
Oct 28 15:19:02 systemd[1]: Listening on Virtual machine lock manager socket.
Oct 28 15:19:02 systemd[1]: Closed Libvirt local read-only socket.
Oct 28 15:19:02 systemd[1]: Stopping Libvirt local read-only socket.
Oct 28 15:19:02 systemd[1]: Closed Libvirt local socket.
Oct 28 15:19:02 systemd[1]: Stopping Libvirt local socket.
Oct 28 15:19:02 systemd[1]: Listening on Libvirt local socket.
Oct 28 15:19:02 systemd[1]: Listening on Libvirt admin socket.
Oct 28 15:19:02 systemd[1]: Listening on Libvirt local read-only socket.
Oct 28 15:19:02 systemd[1]: Closed Virtual machine log manager socket.
Oct 28 15:19:02 systemd[1]: Stopping Virtual machine log manager socket.
Oct 28 15:19:02 systemd[1]: Listening on Virtual machine log manager socket.
Oct 28 15:19:02 systemd[1]: Starting Virtualization daemon...
Oct 28 15:19:02 systemd[1]: Stopping Suspend/Resume Running libvirt Guests...
Oct 28 15:19:02 systemd[1]: Started Virtualization daemon.
Oct 28 15:19:02 libvirt-guests.sh[4912]: Running guests on default URI: no running guests.
Oct 28 15:19:02 systemd[1]: Stopped Suspend/Resume Running libvirt Guests.
Oct 28 15:19:02 systemd[1]: Stopped target Libvirt guests shutdown.
Oct 28 15:19:02 systemd[1]: Stopping Libvirt guests shutdown.
Oct 28 15:19:02 systemd[1]: Reached target Libvirt guests shutdown.
Oct 28 15:19:02 systemd[1]: Starting Suspend/Resume Running libvirt Guests...
Oct 28 15:19:02 systemd[1]: Started Suspend/Resume Running libvirt Guests.
Adding 'Requires=libvirtd.service' to virt-guest-shutdown.target results
in expected behavior
Oct 28 15:40:00 systemd[1]: Stopping Suspend/Resume Running libvirt Guests...
Oct 28 15:40:00 libvirt-guests.sh[5245]: Running guests on default URI: no running guests.
Oct 28 15:40:00 systemd[1]: Stopped Suspend/Resume Running libvirt Guests.
Oct 28 15:40:00 systemd[1]: Stopped target Libvirt guests shutdown.
Oct 28 15:40:00 systemd[1]: Stopping Libvirt guests shutdown.
Oct 28 15:40:00 systemd[1]: Stopping Virtualization daemon...
Oct 28 15:40:00 systemd[1]: Stopped Virtualization daemon.
Oct 28 15:40:00 systemd[1]: Closed Virtual machine log manager socket.
Oct 28 15:40:00 systemd[1]: Stopping Virtual machine log manager socket.
Oct 28 15:40:00 systemd[1]: Listening on Virtual machine log manager socket.
Oct 28 15:40:00 systemd[1]: Closed Libvirt admin socket.
Oct 28 15:40:00 systemd[1]: Stopping Libvirt admin socket.
Oct 28 15:40:00 systemd[1]: Closed Libvirt local read-only socket.
Oct 28 15:40:00 systemd[1]: Stopping Libvirt local read-only socket.
Oct 28 15:40:00 systemd[1]: Closed Libvirt local socket.
Oct 28 15:40:00 systemd[1]: Stopping Libvirt local socket.
Oct 28 15:40:00 systemd[1]: Listening on Libvirt local socket.
Oct 28 15:40:00 systemd[1]: Listening on Libvirt admin socket.
Oct 28 15:40:00 systemd[1]: Listening on Libvirt local read-only socket.
Oct 28 15:40:00 systemd[1]: Closed Virtual machine lock manager socket.
Oct 28 15:40:00 systemd[1]: Stopping Virtual machine lock manager socket.
Oct 28 15:40:00 systemd[1]: Listening on Virtual machine lock manager socket.
Oct 28 15:40:00 systemd[1]: Starting Virtualization daemon...
Oct 28 15:40:00 systemd[1]: Started Virtualization daemon.
Oct 28 15:40:00 systemd[1]: Reached target Libvirt guests shutdown.
Oct 28 15:40:00 systemd[1]: Starting Suspend/Resume Running libvirt Guests...
Oct 28 15:40:00 systemd[1]: Started Suspend/Resume Running libvirt Guests.
Signed-off-by: Jim Fehlig <jfehlig@suse.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Introduce memory failure event. Libvirt should monitor domain's
event, then posts it to uplayer. According to the hardware memory
corrupted message, a cloud scheduler could migrate domain to another
health physical server.
Several changes in this patch:
public API:
include/*
src/conf/*
src/remote/*
src/remote_protocol-structs
client:
examples/c/misc/event-test.c
tools/virsh-domain.c
With this patch, each driver could implement its own method to run
this new event.
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reported-by: John Ferlan <jferlan@redhat.com>
Fixes: 8487595bee
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Use g_new0 for allocation and remove all the temporary
variables.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
CVE-2020-25637
Add a requirement for domain:write if source is set to
VIR_DOMAIN_INTERFACE_ADDRESSES_SRC_AGENT.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reported-by: Ilja Van Sprundel <ivansprundel@ioactive.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
CVE-2020-25637
Add a new field to @acl annotations for filtering by
unsigned int parameters.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
The current debug message reports the "mode" after selection has
completed, however, the "mode" value can be changed by the selection
logic. It is thus beneficial to report most values upfront, and only
report newly changed values at the end.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This wires up support for using the new virt-ssh-helper binary with the ssh,
libssh and libssh2 protocols.
The new binary will be used preferentially if it is available in $PATH,
otherwise we fall back to traditional netcat.
The "proxy" URI parameter can be used to force use of netcat e.g.
qemu+ssh://host/system?proxy=netcat
or the disable fallback e.g.
qemu+ssh://host/system?proxy=native
With use of virt-ssh-helper, we can now support remote session URIs
qemu+ssh://host/session
and this will only use virt-ssh-helper, with no fallback. This also lets
the libvirtd process be auto-started, and connect directly to the
modular daemons, avoiding use of virtproxyd back-compat tunnelling.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Switch keyfile and netcat parameters, since the netcat path and
socket path are a logical pair that belong together. This patches
the other constructors.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
When accessing libvirtd over a SSH tunnel, the remote driver needs a way
to proxy the SSH input/output stream to a suitable libvirt daemon. This
is currently done by spawning netcat, pointing it to the libvirtd socket
path. This is problematic for a number of reasons:
- The socket path varies according to the --prefix chosen at build
time. The remote client is seeing the local prefix, but what we
need is the remote prefix
- The socket path varies according to remote env variables, such as
the XDG_RUNTIME_DIR location. Again we see the local XDG_RUNTIME_DIR
value, but what we need is the remote value (if any)
- The remote driver doesn't know whether it must connect to the legacy
libvirtd or the modular daemons, so must always assume legacy
libvirtd for back-compat. This means we'll always end up using the
virtproxyd daemon adding an extra hop in the RPC layer.
- We can not able to autospawn the libvirtd daemon for session mode
access
To address these problems this patch introduces the 'virtd-ssh-helper'
program which takes the URI for the remote driver as a CLI parameter.
It then figures out which daemon to connect to and its socket path,
using the same code that the remote driver client would on the remote
host's build of libvirt.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We'll shortly want to reuse code for determining whether to connect to
the system or session daemon from places outside the remote driver
client. Pulling it out into a self contained function facilitates reuse.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The remoteGetUNIXSocketHelper method will be needed by source files
beyond the remote driver client.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We delay converting the remote transport string to enum form until
fairly late. As a result we're doing string comparisons when we
could be just doing enum comparisons.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The remoteSplitURISCheme method will be needed by source files beyond
the remote driver client.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The remoteDriverTransport and remoteDriverMode enums are going to be
needed by source files beyond the remote driver client.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We don't want to repeat the choice of default netcat binary setting in
three different places. This will also make it possible to do better
error reporting in the helper.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Currently we have issues like [1] on libvirtd shutdown as we cleanup while RPC
and other threads are still running. Let's finish all threads other then main
before cleanup.
The approach to finish threads is suggested in [2]. In order to finish RPC
threads serving API calls we let the event loop run but stop accepting new API
calls and block processing any pending API calls. We also inform all drivers of
shutdown so they can prepare for shutdown too. Then we wait for all RPC threads
and driver's background thread to finish. If finishing takes more then 15s we
just exit as we can't safely cleanup in time.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1828207
[2] https://www.redhat.com/archives/libvir-list/2020-April/msg01328.html
Signed-off-by: Nikolay Shirokovskiy <nshirokovskiy@virtuozzo.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Local socket connections were outright disabled because there was no "server"
part in the URI. However, given how requirements and usage scenarios are
evolving, some management apps might need the source libvirt daemon to connect
to the destination daemon over a UNIX socket for peer2peer migration. Since we
cannot know where the socket leads (whether the same daemon or not) let's decide
that based on whether the socket path is non-standard, or rather explicitly
specified in the URI. Checking non-standard path would require to ask the
daemon for configuration and the only misuse that it would prevent would be a
pretty weird one. And that's not worth it. The assumption is that whenever
someone uses explicit UNIX socket paths in the URI for migration they better
know what they are doing.
Partially resolves: https://bugzilla.redhat.com/1638889
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
We need to use @default_auth@ in the augeas test case to match
its use in the main libvirtd.conf.in file.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The systemd .socket unit files we ship for libvirt daemons use
SocketMode=0666 on the assumption that libvirt is built with
polkit which provides access control.
Some people, however, may have explicitly turned off polkit at
build time and not realize that leaves them insecure unless
they also change the SocketMode. This addresses that problem
by making the SocketMode default to 0600 when polkit is
disabled at compile time.
Note we cannot automatically fix the case where the user
compiles polkit, but then overrides the libvirtd.conf defaults
to disable polkit. This is what lead to CVE-2020-15708 in
Ubuntu 20.10. We can at least improve the inline comments
in the config file to give a clearer warning though, which
may have helped avoid the mistaken config.
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The list archives, people.redhat.com and bugzilla all support
https.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Many of our functions start with a DEBUG statement.
Move the statements after declarations to appease
our coding style.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
During the switch to meson, one of the patches mistakenly changed the
runtime socket prefix for {libvirtd, virtproxyd} to "libvirtd-" from
the original "libvirt-". Not to be mistaken with the systemd unit name
which actually follows the daemon name, IOW the systemd unit name
remains as e.g. "libvirtd.socket", but the actual unix socket created
on the filesystem that the daemon binds to must be named "libvirt-sock"
and not "libvirtd-sock".
Fixes: dd4f2c73ad
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Both accept a NULL value gracefully and virStringFreeList
does not zero the pointer afterwards, so a straight replace
is safe.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
