Jim Fehlig 5c5df5310f libxl: Search for virDomainObj in event handler threads
libxl can deliver events and invoke callbacks on any application thread
calling into libxl. This can cause deadlock in the libvirt libxl driver

Thread 19 (Thread 0x7f31411ec700 (LWP 14068) "libvirtd"):
#0  0x00007f318520cc7d in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x00007f3185205ed5 in pthread_mutex_lock () from /lib64/libpthread.so.0
#2  0x00007f3189488015 in virMutexLock (m=<optimized out>) at ../../src/util/virthread.c:79
#3  0x00007f3189463f3b in virObjectLock (anyobj=<optimized out>) at ../../src/util/virobject.c:433
#4  0x00007f31894f2f41 in virDomainObjListSearchID (payload=0x7f317400a6d0, name=<optimized out>, data=0x7f31411eaeac) at ../../src/conf/virdomainobjlist.c:105
#5  0x00007f3189437ac5 in virHashSearch (ctable=0x7f3124025a30, iter=iter@entry=0x7f31894f2f30 <virDomainObjListSearchID>, data=data@entry=0x7f31411eaeac, name=name@entry=0x0) at ../../src/util/virhash.c:745
#6  0x00007f31894f3919 in virDomainObjListFindByID (doms=0x7f3124025430, id=<optimized out>) at ../../src/conf/virdomainobjlist.c:121
#7  0x00007f3152f292e5 in libxlDomainEventHandler (data=0x7f3124023d80, event=0x7f310c010ae0) at ../../src/libxl/libxl_domain.c:660
#8  0x00007f3152c6ff5d in egc_run_callbacks (egc=egc@entry=0x7f31411eaf50) at libxl_event.c:1427
#9  0x00007f3152c718bd in libxl__egc_cleanup (egc=0x7f31411eaf50) at libxl_event.c:1458
#10 libxl__ao_inprogress (ao=ao@entry=0x7f310c00b8a0, file=file@entry=0x7f3152cce987 "libxl_domain.c", line=line@entry=730, func=func@entry=0x7f3152ccf750 <__func__.22238> "libxl_domain_unpause") at libxl_event.c:2047
#11 0x00007f3152c8c5b8 in libxl_domain_unpause (ctx=0x7f3124015a40, domid=<optimized out>, ao_how=ao_how@entry=0x0) at libxl_domain.c:730
#12 0x00007f3152f2a584 in libxl_domain_unpause_0x041200 (domid=<optimized out>, ctx=<optimized out>) at /usr/include/libxl.h:1756
#13 libxlDomainStart (driver=driver@entry=0x7f3124023d80, vm=vm@entry=0x7f317400a6d0, start_paused=start_paused@entry=false, restore_fd=restore_fd@entry=-1, restore_ver=<optimized out>, restore_ver@entry=2) at ../../src/libxl/libxl_domain.c:1482
#14 0x00007f3152f2a6e3 in libxlDomainStartNew (driver=driver@entry=0x7f3124023d80, vm=vm@entry=0x7f317400a6d0, start_paused=start_paused@entry=false) at ../../src/libxl/libxl_domain.c:1545
#15 0x00007f3152f2a789 in libxlDomainShutdownHandleRestart (driver=0x7f3124023d80, vm=0x7f317400a6d0) at ../../src/libxl/libxl_domain.c:464
#16 0x00007f3152f2a9e4 in libxlDomainShutdownThread (opaque=<optimized out>) at ../../src/libxl/libxl_domain.c:559
#17 0x00007f3189487ee2 in virThreadHelper (data=<optimized out>) at ../../src/util/virthread.c:196
#18 0x00007f3185203539 in start_thread () from /lib64/libpthread.so.0
#19 0x00007f3184f3becf in clone () from /lib64/libc.so.6

Frame 16 runs a thread created to handle domain shutdown processing for
domid 28712. In this case the event contained the reboot reason, so the
old domain is destroyed and a new one is created by libxlDomainStart new.
After starting the domain, it is unpaused by calling libxl_domain_unpause
in frame 12. While the thread is running within libxl, libxl takes the
opportunity to deliver a pending domain shutdown event for unrelated domid
28710. While searching for the associated virDomainObj by ID, a deadlock is
encountered when attempting to lock the virDomainObj for domid 28712, which
is already locked since this thread is processing its shutdown event.

The deadlock can be avoided by moving the search for a virDomainObj
associated with the event domid to the shutdown thread. The same is done
for the death thread.

Signed-off-by: Jim Fehlig <jfehlig@suse.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-12-02 11:21:10 -07:00
2019-05-31 17:54:28 +02:00
2021-11-23 16:44:18 +00:00
2021-11-27 16:16:17 +01:00
2021-12-01 13:53:12 +01:00
2019-09-06 12:47:46 +02:00
2020-01-16 13:04:11 +00:00
2020-08-03 09:26:48 +02:00
2019-10-18 17:32:52 +02:00
2021-12-01 10:53:20 +01:00
2021-12-01 10:53:20 +01:00
2020-08-03 15:08:28 +02:00

GitLab CI Build Status

CII Best Practices

Translation status

Libvirt API for virtualization

Libvirt provides a portable, long term stable C API for managing the virtualization technologies provided by many operating systems. It includes support for QEMU, KVM, Xen, LXC, bhyve, Virtuozzo, VMware vCenter and ESX, VMware Desktop, Hyper-V, VirtualBox and the POWER Hypervisor.

For some of these hypervisors, it provides a stateful management daemon which runs on the virtualization host allowing access to the API both by non-privileged local users and remote users.

Layered packages provide bindings of the libvirt C API into other languages including Python, Perl, PHP, Go, Java, OCaml, as well as mappings into object systems such as GObject, CIM and SNMP.

Further information about the libvirt project can be found on the website:

https://libvirt.org

License

The libvirt C API is distributed under the terms of GNU Lesser General Public License, version 2.1 (or later). Some parts of the code that are not part of the C library may have the more restrictive GNU General Public License, version 2.0 (or later). See the files COPYING.LESSER and COPYING for full license terms & conditions.

Installation

Instructions on building and installing libvirt can be found on the website:

https://libvirt.org/compiling.html

Contributing

The libvirt project welcomes contributions in many ways. For most components the best way to contribute is to send patches to the primary development mailing list. Further guidance on this can be found on the website:

https://libvirt.org/contribute.html

Contact

The libvirt project has two primary mailing lists:

Further details on contacting the project are available on the website:

https://libvirt.org/contact.html

Description
Libvirt provides a portable, long term stable C API for managing the virtualization technologies provided by many operating systems. It includes support for QEMU, KVM, Xen, LXC, bhyve, Virtuozzo, VMware vCenter and ESX, VMware Desktop, Hyper-V, VirtualBox and the POWER Hypervisor.
Readme 901 MiB
Languages
C 94.8%
Python 2%
Meson 0.9%
Shell 0.8%
Dockerfile 0.6%
Other 0.8%