libvirtd can spawn threads/tasks when creating new domains for
some hypervisors such as Xen's libxl driver, quickly reaching
the cgroups pids controller default TasksMax setting of 512. When
the limit is reached, attempting to create additional domains
results in an error from the cgroups pids controller, e.g.
kernel: [71282.213347] cgroup: fork rejected by pids controller in
/system.slice/libvirtd.service
Depending on domain type and configuration, anywhere from 4-7
threads/tasks may be created by libxl when starting a domain.
In order to support 4096 domains, similar to commit 27cd763500,
increase the TasksMax setting in libvirtd.service to
4096 * 8 = 32768 tasks.
Linux still defaults to a 1024 open file handle limit. This causes
scalability problems for libvirtd / virtlockd / virtlogd on large
hosts which might want > 1024 guest to be running. In fact if each
guest needs > 1 FD, we can't even get to 500 guests. This is not
good enough when we see machines with 100's of physical cores and
TBs of RAM.
In comparison to other memory requirements of libvirtd & related
daemons, the resource usage associated with open file handles
is essentially line noise. It is thus reasonable to increase the
limits unconditionally for all installs.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
After deploying virtlogd by default we identified a number of
mistakes in the systemd unit file. virtlockd's relationship
to libvirtd is the same as virtlogd, so we must apply the
same unit file fixes to virtlockd
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
We already guarantee that virtlogd.socket is enabled/disabled
along with libvirtd.service, but if libvirtd.service has just
been installed and is started before rebooting, then
virtlogd.socket will not be running and guest startup will
fail.
Add Requires=virtlogd.socket to libvirtd.service to make sure
virtlogd.socket is always started along with libvirtd.service,
and add Before=libvirtd.service to both virtlogd.socket and
virtlogd.service so that virtlogd never disappears before
libvirtd has exited.
Also add PartOf=libvirtd.service to both virtlogd.socket and
virtlogd.service, so that virtlogd can be shut down when not
needed.
Resolves: https://bugzilla.redhat.com/1372576
This reverts commit 1e9808d3a1.
We shouldn't advertise libvirtd.socket activation, since currently
it means VM/network/... autostart won't work as expected.
We tried to find a middle ground by installing the config file without
an [Install] section, since systemd won't allow .socket to be enabled
without one... or at least it did do that; presently on f24 it allows
activating the socket quite happily. This also caused user confusion[1]
Just remove the socket file. I've filed a new RFE to track coming up
with a solution to the autostart problem[2], we can point users at that
if there's more confusion:
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1279348
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1326136
When someone does 'systemctl enable libvirtd.service' we should
also enable virtlockd.socket/virtlogd.socket, so that they can
be auto-activated if libvirtd tries to access the sockets.
Without this, people have to manually enable the units themselves
via 'systemctl enable virtdlogd.socket'.
This also ensures that if distros uses 'systemctl preset' for
enabling 'libvirtd.service', then the virtdlogd.socket gets
enabled without having to wait for the distro to update their
presets file.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
When images are on mounted filesystems, there is no guarantee libvirtd
will start before they are mounted. Since we want to start after both
local and remote filesystems are mounted, we need to add both local-fs
and remote-fs as targets that should precede starting libvirtd service.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Systemd does not forget about the cases, where client service needs to
wait for daemon service to initialize and start accepting new clients.
Setting a dependency in client is not enough as systemd doesn't know
when the daemon has initialized itself and started accepting new
clients. However, it offers a mechanism to solve this. The daemon needs
to call a special systemd function by which the daemon tells "I'm ready
to accept new clients". This is exactly what we need with
libvirtd-guests (client) and libvirtd (daemon). So now, with this
change, libvirt-guests.service is invoked not any sooner than
libvirtd.service calls the systemd notify function.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
There's a quite old bug entry here:
https://bugzilla.redhat.com/show_bug.cgi?id=700010
I just stumbled over that very issue on F18. Doing a little bit
debugging of the shutdown sequence, it turns out that - at least on my
F18 installation - libvirtd is shutdown *after* iscsid, which makes it
impossible for libvirt to perform the logout of the iscsi session properly.
This patch simply adds another startup dependancy on iscsid.service
which in turn delays iscsid shutdown until after libvirtd has stopped.
Having that applied, the system shuts down properly again.
Signed-off-by: Eric Blake <eblake@redhat.com>
https://bugzilla.redhat.com/show_bug.cgi?id=830201
The initscript and upstart services depend on dbus starting
before libvirtd. When we first wrote the systemd script, we
tried to do the same, but we depended on dbus.target (which
does not exist) in comparison to network.target (which does
exist), so we removed that in commit 4c7973e. But we still
need dbus up and running first, especially now that we want
to support shutdown inhibition via dbus (whereas we originally
needed dbus only for firewall control).
http://www.freedesktop.org/software/systemd/man/systemd.target.html
explains how a target (such as network.target) is just a collection
of common services bundled together, and why we want network.target
but dbus.service.
* daemon/libvirtd.service.in (Unit): Depend on dbus starting
first.
Domains configured with autostart may fail to start if the host
network stack has not been started. E.g. when using bridged
networking autostarting a domain can fail with
libvirtd[1403]: 2012-06-20 13:23:49.833+0000: 1485: error :
qemuAutostartDomain:177 : Failed to autostart VM 'test': Cannot get
interface MTU on 'br0': No such device
Adding syslog.target is obsolete, avahi.target does not
exist and dbus.target is also obsolete
Reported-by: Lennart Poettering <lpoetter@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
There is no 'udev.target' unit in systemd (only 'udev.service')
yet libvirtd's unit file had a dep on one. There's no compelling
reason for a dep on udev, so remove it altogether.
Reported-by: Avi Kivity <avi@redhat.com>
Currently the libvirt.service unit file for systemd does not
specify any kill mode. So systemd kills off every process
inside its cgroup. ie all dnsmasq processes, all virtual
machines. This obviously not what we want. Set KillMode=process
so that it only kills the top level process of libvirtd
* daemon/libvirtd.service.in: Add KillMode=process
Reported-By: Mark McLoughlin <markmc@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
One of my latest patches (d8db0f9690) created support for setting
the limit for the maximum of opened files by qemu user. However,
since libvirtd keeps one FD opened per domain (well, for qemu at least)
it will likely hit this limit on huge scenarios.
This patch adds support for a systemd init service for libvirtd
and libvirt-guests. The libvirtd.service is *not* written to use
socket activation, since we want libvirtd to start on boot so it
can do guest auto-start.
The libvirt-guests.service is pretty lame, just exec'ing the
original init script for now. Ideally we would factor out the
functionality, into some shared tool.
Instead of
./configure --with-init-script=redhat
You can now do
./configure --with-init-script=systemd
Or better still:
./configure --with-init-script=systemd+redhat
We can also now support install of the upstart init script
* configure.ac: Add systemd, and systemd+redhat options to
--with-init-script option
* daemon/Makefile.am: Install systemd services
* daemon/libvirtd.sysconf: Add note about unused env variable
with systemd
* daemon/libvirtd.service.in: libvirtd systemd service unit
* libvirt.spec.in: Add scripts to installing systemd services
and migrating from legacy init scripts
* tools/Makefile.am: Install systemd services
* tools/libvirt-guests.init.sh: Rename to tools/libvirt-guests.init.in
* tools/libvirt-guests.service.in: systemd service unit
