mirror of
https://gitlab.com/libvirt/libvirt.git
synced 2025-01-18 18:45:16 +00:00
9a47afb1cd
There were multiple race conditions that could lead to segmentation faults. The first precondition for this is qemuProcessLaunch must fail sometime shortly after starting the new QEMU process. The second precondition for the segmentation faults is that the new QEMU process dies - or to be more precise the QEMU monitor has to be closed irregularly. If both happens during qemuProcessStart (starting a domain) there are race windows between the thread with the event loop (T1) and the thread that is starting the domain (T2). First segmentation fault scenario: If qemuProcessLaunch fails during qemuProcessStart the code branches to the 'stop' path where 'qemuMonitorSetDomainLog(priv->mon, NULL, NULL, NULL)' will set the log function of the monitor to NULL (done in T2). In the meantime the event loop of T1 will wake up with an EOF event for the QEMU monitor because the QEMU process has died. The crash occurs if T1 has checked 'mon->logFunc != NULL' in qemuMonitorIO just before the logFunc was set to NULL by T2. If this situation occurs T1 will try to call mon->logFunc which leads to the segmentation fault. Solution: Require the monitor lock for setting the log function. Backtrace: 0 0x0000000000000000 in ?? () 1 0x000003ffe9e45316 in qemuMonitorIO (watch=<optimized out>, fd=<optimized out>, events=<optimized out>, opaque=0x3ffe08aa860) at ../../src/qemu/qemu_monitor.c:727 2 0x000003fffda2e1a4 in virEventPollDispatchHandles (nfds=<optimized out>, fds=0x2aa000fd980) at ../../src/util/vireventpoll.c:508 3 0x000003fffda2e398 in virEventPollRunOnce () at ../../src/util/vireventpoll.c:657 4 0x000003fffda2ca10 in virEventRunDefaultImpl () at ../../src/util/virevent.c:314 5 0x000003fffdba9366 in virNetDaemonRun (dmn=0x2aa000cc550) at ../../src/rpc/virnetdaemon.c:818 6 0x000002aa00024668 in main (argc=<optimized out>, argv=<optimized out>) at ../../daemon/libvirtd.c:1541 Second segmentation fault scenario: If qemuProcessLaunch fails it will unref the log context and with invoking qemuMonitorSetDomainLog(priv->mon, NULL, NULL, NULL) qemuDomainLogContextFree() will be invoked. qemuDomainLogContextFree() invokes virNetClientClose() to close the client and cleans everything up (including unref of _virLogManager.client) when virNetClientClose() returns. When T1 is now trying to report 'qemu unexpectedly closed the monitor' libvirtd will crash because the client has already been freed. Solution: As the critical section in qemuMonitorIO is protected with the monitor lock we can use the same solution as proposed for the first segmentation fault. Backtrace: 0 virClassIsDerivedFrom (klass=0x3100979797979797, parent=0x2aa000d92f0) at ../../src/util/virobject.c:169 1 0x000003fffda659e6 in virObjectIsClass (anyobj=<optimized out>, klass=<optimized out>) at ../../src/util/virobject.c:365 2 0x000003fffda65a24 in virObjectLock (anyobj=0x3ffe08c1db0) at ../../src/util/virobject.c:317 3 0x000003fffdba4688 in virNetClientIOEventLoop (client=client@entry=0x3ffe08c1db0, thiscall=thiscall@entry=0x2aa000fbfa0) at ../../src/rpc/virnetclient.c:1668 4 0x000003fffdba4b4c in virNetClientIO (client=client@entry=0x3ffe08c1db0, thiscall=0x2aa000fbfa0) at ../../src/rpc/virnetclient.c:1944 5 0x000003fffdba4d42 in virNetClientSendInternal (client=client@entry=0x3ffe08c1db0, msg=msg@entry=0x2aa000cc710, expectReply=expectReply@entry=true, nonBlock=nonBlock@entry=false) at ../../src/rpc/virnetclient.c:2116 6 0x000003fffdba6268 in virNetClientSendWithReply (client=0x3ffe08c1db0, msg=0x2aa000cc710) at ../../src/rpc/virnetclient.c:2144 7 0x000003fffdba6e8e in virNetClientProgramCall (prog=0x3ffe08c1120, client=<optimized out>, serial=<optimized out>, proc=<optimized out>, noutfds=<optimized out>, outfds=0x0, ninfds=0x0, infds=0x0, args_filter=0x3fffdb64440 <xdr_virLogManagerProtocolDomainReadLogFileArgs>, args=0x3ffffffe010, ret_filter=0x3fffdb644c0 <xdr_virLogManagerProtocolDomainReadLogFileRet>, ret=0x3ffffffe008) at ../../src/rpc/virnetclientprogram.c:329 8 0x000003fffdb64042 in virLogManagerDomainReadLogFile (mgr=<optimized out>, path=<optimized out>, inode=<optimized out>, offset=<optimized out>, maxlen=<optimized out>, flags=0) at ../../src/logging/log_manager.c:272 9 0x000003ffe9e0315c in qemuDomainLogContextRead (ctxt=0x3ffe08c2980, msg=0x3ffffffe1c0) at ../../src/qemu/qemu_domain.c:4422 10 0x000003ffe9e280a8 in qemuProcessReadLog (logCtxt=<optimized out>, msg=msg@entry=0x3ffffffe288) at ../../src/qemu/qemu_process.c:1800 11 0x000003ffe9e28206 in qemuProcessReportLogError (logCtxt=<optimized out>, msgprefix=0x3ffe9ec276a "qemu unexpectedly closed the monitor") at ../../src/qemu/qemu_process.c:1836 12 0x000003ffe9e28306 in qemuProcessMonitorReportLogError (mon=mon@entry=0x3ffe085cf10, msg=<optimized out>, opaque=<optimized out>) at ../../src/qemu/qemu_process.c:1856 13 0x000003ffe9e452b6 in qemuMonitorIO (watch=<optimized out>, fd=<optimized out>, events=<optimized out>, opaque=0x3ffe085cf10) at ../../src/qemu/qemu_monitor.c:726 14 0x000003fffda2e1a4 in virEventPollDispatchHandles (nfds=<optimized out>, fds=0x2aa000fd980) at ../../src/util/vireventpoll.c:508 15 0x000003fffda2e398 in virEventPollRunOnce () at ../../src/util/vireventpoll.c:657 16 0x000003fffda2ca10 in virEventRunDefaultImpl () at ../../src/util/virevent.c:314 17 0x000003fffdba9366 in virNetDaemonRun (dmn=0x2aa000cc550) at ../../src/rpc/virnetdaemon.c:818 18 0x000002aa00024668 in main (argc=<optimized out>, argv=<optimized out>) at ../../daemon/libvirtd.c:1541 Other code parts where the same problem was possible to occur are fixed as well (qemuMigrationFinish, qemuProcessStart, and qemuDomainSaveImageStartVM). Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com> Reported-by: Sascha Silbe <silbe@linux.vnet.ibm.com> (cherry picked from commit 20e95cb7c8653b02016ab9a7118c6de8c9866ea9)
libvirt library code README
===========================
The directory provides the bulk of the libvirt codebase. Everything
except for the libvirtd daemon and client tools. The build uses a
large number of libtool convenience libraries - one for each child
directory, and then links them together for the final libvirt.so,
although some bits get linked directly to libvirtd daemon instead.
The files directly in this directory are supporting the public API
entry points & data structures.
There are two core shared modules to be aware of:
* util/ - a collection of shared APIs that can be used by any
code. This directory is always in the include path
for all things built
* conf/ - APIs for parsing / manipulating all the official XML
files used by the public API. This directory is only
in the include path for driver implementation modules
* vmx/ - VMware VMX config handling (used by esx/ and vmware/)
Then there are the hypervisor implementations:
* bhyve - bhyve - The BSD Hypervisor
* esx/ - VMware ESX and GSX support using vSphere API over SOAP
* hyperv/ - Microsoft Hyper-V support using WinRM
* lxc/ - Linux Native Containers
* openvz/ - OpenVZ containers using cli tools
* phyp/ - IBM Power Hypervisor using CLI tools over SSH
* qemu/ - QEMU / KVM using qemu CLI/monitor
* remote/ - Generic libvirt native RPC client
* test/ - A "mock" driver for testing
* uml/ - User Mode Linux
* vbox/ - Virtual Box using native API
* vmware/ - VMware Workstation and Player using the vmrun tool
* xen/ - Xen using hypercalls, XenD SEXPR & XenStore
* xenapi/ - Xen using libxenserver
Finally some secondary drivers that are shared for several HVs.
Currently these are used by LXC, OpenVZ, QEMU, UML and Xen drivers.
The ESX, Hyper-V, Power Hypervisor, Remote, Test & VirtualBox drivers all
implement the secondary drivers directly
* cpu/ - CPU feature management
* interface/ - Host network interface management
* network/ - Virtual NAT networking
* nwfilter/ - Network traffic filtering rules
* node_device/ - Host device enumeration
* secret/ - Secret management
* security/ - Mandatory access control drivers
* storage/ - Storage management drivers
Since both the hypervisor and secondary drivers can be built as
dlopen()able modules, it is *FORBIDDEN* to have build dependencies
between these directories. Drivers are only allowed to depend on
the public API, and the internal APIs in the util/ and conf/
directories