Libvirt provides a portable, long term stable C API for managing the virtualization technologies provided by many operating systems. It includes support for QEMU, KVM, Xen, LXC, bhyve, Virtuozzo, VMware vCenter and ESX, VMware Desktop, Hyper-V, VirtualBox and the POWER Hypervisor.
Go to file
Michal Privoznik 4e5f0dd2d3 virLXCMonitorClose: Unlock domain while closing monitor
There's a race in lxc driver causing a deadlock. If a domain is
destroyed immediately after started, the deadlock can occur. When domain
is started, the even loop tries to connect to the monitor. If the
connecting succeeds, virLXCProcessMonitorInitNotify() is called with
@mon->client locked. The first thing that callee does, is
virObjectLock(vm). So the order of locking is: 1) @mon->client, 2) @vm.

However, if there's another thread executing virDomainDestroy on the
very same domain, the first thing done here is locking the @vm. Then,
the corresponding libvirt_lxc process is killed and monitor is closed
via calling virLXCMonitorClose(). This callee tries to lock @mon->client
too. So the order is reversed to the first case. This situation results
in deadlock and unresponsive libvirtd (since the eventloop is involved).

The proper solution is to unlock the @vm in virLXCMonitorClose prior
entering virNetClientClose(). See the backtrace as follows:

Thread 25 (Thread 0x7f1b7c9b8700 (LWP 16312)):
0  0x00007f1b80539714 in __lll_lock_wait () from /lib64/libpthread.so.0
1  0x00007f1b8053516c in _L_lock_516 () from /lib64/libpthread.so.0
2  0x00007f1b80534fbb in pthread_mutex_lock () from /lib64/libpthread.so.0
3  0x00007f1b82a637cf in virMutexLock (m=0x7f1b3c0038d0) at util/virthreadpthread.c:85
4  0x00007f1b82a4ccf2 in virObjectLock (anyobj=0x7f1b3c0038c0) at util/virobject.c:320
5  0x00007f1b82b861f6 in virNetClientCloseInternal (client=0x7f1b3c0038c0, reason=3) at rpc/virnetclient.c:696
6  0x00007f1b82b862f5 in virNetClientClose (client=0x7f1b3c0038c0) at rpc/virnetclient.c:721
7  0x00007f1b6ee12500 in virLXCMonitorClose (mon=0x7f1b3c007210) at lxc/lxc_monitor.c:216
8  0x00007f1b6ee129f0 in virLXCProcessCleanup (driver=0x7f1b68100240, vm=0x7f1b680ceb70, reason=VIR_DOMAIN_SHUTOFF_DESTROYED) at lxc/lxc_process.c:174
9  0x00007f1b6ee14106 in virLXCProcessStop (driver=0x7f1b68100240, vm=0x7f1b680ceb70, reason=VIR_DOMAIN_SHUTOFF_DESTROYED) at lxc/lxc_process.c:710
10 0x00007f1b6ee1aa36 in lxcDomainDestroyFlags (dom=0x7f1b5c002560, flags=0) at lxc/lxc_driver.c:1291
11 0x00007f1b6ee1ab1a in lxcDomainDestroy (dom=0x7f1b5c002560) at lxc/lxc_driver.c:1321
12 0x00007f1b82b05be5 in virDomainDestroy (domain=0x7f1b5c002560) at libvirt.c:2303
13 0x00007f1b835a7e85 in remoteDispatchDomainDestroy (server=0x7f1b857419d0, client=0x7f1b8574ae40, msg=0x7f1b8574acf0, rerr=0x7f1b7c9b7c30, args=0x7f1b5c004a50) at remote_dispatch.h:3143
14 0x00007f1b835a7d78 in remoteDispatchDomainDestroyHelper (server=0x7f1b857419d0, client=0x7f1b8574ae40, msg=0x7f1b8574acf0, rerr=0x7f1b7c9b7c30, args=0x7f1b5c004a50, ret=0x7f1b5c0029e0) at remote_dispatch.h:3121
15 0x00007f1b82b93704 in virNetServerProgramDispatchCall (prog=0x7f1b8573af90, server=0x7f1b857419d0, client=0x7f1b8574ae40, msg=0x7f1b8574acf0) at rpc/virnetserverprogram.c:435
16 0x00007f1b82b93263 in virNetServerProgramDispatch (prog=0x7f1b8573af90, server=0x7f1b857419d0, client=0x7f1b8574ae40, msg=0x7f1b8574acf0) at rpc/virnetserverprogram.c:305
17 0x00007f1b82b8c0f6 in virNetServerProcessMsg (srv=0x7f1b857419d0, client=0x7f1b8574ae40, prog=0x7f1b8573af90, msg=0x7f1b8574acf0) at rpc/virnetserver.c:163
18 0x00007f1b82b8c1da in virNetServerHandleJob (jobOpaque=0x7f1b8574dca0, opaque=0x7f1b857419d0) at rpc/virnetserver.c:184
19 0x00007f1b82a64158 in virThreadPoolWorker (opaque=0x7f1b8573cb10) at util/virthreadpool.c:144
20 0x00007f1b82a63ae5 in virThreadHelper (data=0x7f1b8574b9f0) at util/virthreadpthread.c:161
21 0x00007f1b80532f4a in start_thread () from /lib64/libpthread.so.0
22 0x00007f1b7fc4f20d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f1b83546740 (LWP 16297)):
0  0x00007f1b80539714 in __lll_lock_wait () from /lib64/libpthread.so.0
1  0x00007f1b8053516c in _L_lock_516 () from /lib64/libpthread.so.0
2  0x00007f1b80534fbb in pthread_mutex_lock () from /lib64/libpthread.so.0
3  0x00007f1b82a637cf in virMutexLock (m=0x7f1b680ceb80) at util/virthreadpthread.c:85
4  0x00007f1b82a4ccf2 in virObjectLock (anyobj=0x7f1b680ceb70) at util/virobject.c:320
5  0x00007f1b6ee13bd7 in virLXCProcessMonitorInitNotify (mon=0x7f1b3c007210, initpid=4832, vm=0x7f1b680ceb70) at lxc/lxc_process.c:601
6  0x00007f1b6ee11fd3 in virLXCMonitorHandleEventInit (prog=0x7f1b3c001f10, client=0x7f1b3c0038c0, evdata=0x7f1b8574a7d0, opaque=0x7f1b3c007210) at lxc/lxc_monitor.c:109
7  0x00007f1b82b8a196 in virNetClientProgramDispatch (prog=0x7f1b3c001f10, client=0x7f1b3c0038c0, msg=0x7f1b3c003928) at rpc/virnetclientprogram.c:259
8  0x00007f1b82b87030 in virNetClientCallDispatchMessage (client=0x7f1b3c0038c0) at rpc/virnetclient.c:1019
9  0x00007f1b82b876bb in virNetClientCallDispatch (client=0x7f1b3c0038c0) at rpc/virnetclient.c:1140
10 0x00007f1b82b87d41 in virNetClientIOHandleInput (client=0x7f1b3c0038c0) at rpc/virnetclient.c:1312
11 0x00007f1b82b88f51 in virNetClientIncomingEvent (sock=0x7f1b3c0044e0, events=1, opaque=0x7f1b3c0038c0) at rpc/virnetclient.c:1832
12 0x00007f1b82b9e1c8 in virNetSocketEventHandle (watch=3321, fd=54, events=1, opaque=0x7f1b3c0044e0) at rpc/virnetsocket.c:1695
13 0x00007f1b82a272cf in virEventPollDispatchHandles (nfds=21, fds=0x7f1b8574ded0) at util/vireventpoll.c:498
14 0x00007f1b82a27af2 in virEventPollRunOnce () at util/vireventpoll.c:645
15 0x00007f1b82a25a61 in virEventRunDefaultImpl () at util/virevent.c:273
16 0x00007f1b82b8e97e in virNetServerRun (srv=0x7f1b857419d0) at rpc/virnetserver.c:1097
17 0x00007f1b8359db6b in main (argc=2, argv=0x7ffff98dbaa8) at libvirtd.c:1512
2013-07-24 17:53:00 +02:00
.gnulib@644c40496c maint: update to latest gnulib 2013-07-19 05:53:02 -06:00
build-aux Add API for calling systemd-machined's DBus API 2013-07-22 13:09:58 +01:00
daemon maint: split long lines in Makefiles 2013-07-19 05:25:35 -06:00
docs conf: Introduce new XML tag "mode" for disk source 2013-07-22 14:01:04 -04:00
examples examples: Handle VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED event 2013-07-18 15:28:45 +02:00
gnulib maint: use LGPL correctly 2013-05-20 14:03:48 -06:00
include Add API for calling systemd-machined's DBus API 2013-07-22 13:09:58 +01:00
m4 Fix build with clang 2013-07-04 11:35:59 +02:00
po caps: Add helpers to convert NUMA nodes to corresponding CPUs 2013-07-18 14:41:01 +02:00
python maint: split long lines in Makefiles 2013-07-19 05:25:35 -06:00
src virLXCMonitorClose: Unlock domain while closing monitor 2013-07-24 17:53:00 +02:00
tests qemu: set/validate slot/connection type when assigning slots for PCI devices 2013-07-24 06:45:07 -04:00
tools virsh: fix change-media bug on disk block type 2013-07-23 14:20:56 +08:00
.ctags maint: Make ctags work out of the box 2013-07-18 08:47:21 +02:00
.dir-locals.el build: avoid tabs that failed syntax-check 2012-09-06 09:43:46 -06:00
.gitignore Add API for calling systemd-machined's DBus API 2013-07-22 13:09:58 +01:00
.gitmodules make .gnulib a submodule 2009-07-08 16:17:51 +02:00
.mailmap Autogenerate AUTHORS 2012-10-19 12:44:56 -04:00
AUTHORS.in Add John Ferlan to the committers list 2013-02-05 10:59:32 -05:00
autobuild.sh Set PKG_CONFIG_LIBDIR in autobuild.sh 2013-05-17 17:09:29 +01:00
autogen.sh autogen: Handle case when libvirt's submodule 2013-07-19 13:45:22 +02:00
bootstrap maint: update to latest gnulib 2013-07-19 05:53:02 -06:00
bootstrap.conf maint: update to latest gnulib 2013-07-19 05:53:02 -06:00
cfg.mk maint: split long lines in Makefiles 2013-07-19 05:25:35 -06:00
ChangeLog-old virterror.c: Fix several spelling mistakes 2012-02-03 11:32:51 -07:00
configure.ac Fix link_addr detection 2013-07-24 15:31:34 +02:00
COPYING maint: follow recommended practice for using LGPL 2013-05-20 14:15:21 -06:00
COPYING.LESSER maint: follow recommended practice for using LGPL 2013-05-20 14:15:21 -06:00
HACKING Adapt to VIR_ALLOC and virAsprintf in docs/ 2013-07-10 11:07:31 +02:00
libvirt.pc.in build: silence warning from autoconf 2012-05-30 09:22:02 -06:00
libvirt.spec.in Release of libvirt-1.1.0 2013-07-01 17:22:49 +08:00
Makefile.am maint: split long lines in Makefiles 2013-07-19 05:25:35 -06:00
Makefile.nonreentrant maint: use LGPL correctly 2013-05-20 14:03:48 -06:00
mingw-libvirt.spec.in conf: add features to volume target XML 2013-06-21 13:25:30 +02:00
README Correct typos in the documentation (Atsushi SAKAI) 2008-01-24 10:15:13 +00:00
README-hacking maint: relax git minimum version 2010-02-24 14:29:27 -05:00
run.in run: license as LGPL 2013-02-23 14:03:19 -07:00
TODO Update todo list file to point at bugzilla/website 2010-10-13 16:45:26 +01:00

         LibVirt : simple API for virtualization

  Libvirt is a C toolkit to interact with the virtualization capabilities
of recent versions of Linux (and other OSes). It is free software
available under the GNU Lesser General Public License. Virtualization of
the Linux Operating System means the ability to run multiple instances of
Operating Systems concurrently on a single hardware system where the basic
resources are driven by a Linux instance. The library aim at providing
long term stable C API initially for the Xen paravirtualization but
should be able to integrate other virtualization mechanisms if needed.

Daniel Veillard <veillard@redhat.com>