Commit 94c465d0 refactored the logging setup phase but introduced an
issue, where the daemon ignores verbose mode when there are no outputs
defined and the default must be used. The problem is that the default
output was determined too early, thus ignoring the potential '--verbose'
option taking effect. This patch postpones the creation of the default
output to the very last moment when nothing else can change. Since the
default output is only created during the init phase, it's safe to leave
the pointer as NULL for a while, but it will be set eventually, thus not
affecting runtime.
Patch also adjusts both the other daemons.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1442947
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Since its introduction in f61341173b it was never
implemented nor there are plans to implement it. Drop it.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Driver modules proved to be reliable for a long time. Since support for
not building modules complicates the code and makefiles drop it.
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Problem with our error reporting is that the error object is a
thread local variable. That means if there's an error reported
within the I/O thread it gets logged and everything, but later
when the event loop aborts the stream it doesn't see the original
error. So we are left with some generic error. We can do better
if we copy the error message between the threads.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
The DBus conditional was renamed way back:
commit da77f04ed5
Author: Daniel P. Berrange <berrange@redhat.com>
Date: Thu Sep 20 15:05:39 2012 +0100
Convert HAVE_DBUS to WITH_DBUS
but the shutdown inhibit code was not updated. Thus libvirt
was never inhibiting shutdown by a logged in user when VMs
are running.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The default conf files, for example libvirtd.conf, virtlockd.conf, and
virtlogd.conf, should be located under the directory "/etc/libvirt" when
root as root, rather than "/etc". When run as non-root, the configuration
files should be located under "$XDG_CONFIG_HOME/libvirt/", rather than
"XDG_CONFIG_HOME".
Signed-off-by: Lily Zhu <lizhu@redhat.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Right now, there is a lot of exit points from the function.
Depending on their position they need to copy the same free
calls. This goes against our style where we usually have just one
exit point from the function which also does the necessary free.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
@rundir, allocated by virGetUserRuntimeDirectory, is leaked in case
virFileMakePath fails.
Signed-off-by: Xi Xu <xu.xi8@zte.com.cn>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
SASL context would be initialized even if the corresponding TCP or TLS
sockets are not enabled.
fe772f24a6 attempted to fix the symptom by commenting out the settings,
but that did not fix the root cause. 3c647ee4bb later reverted those
changes so that the more secure algorithm is used.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1450095
Whenever client is able to receive some data from stream
daemonStreamHandleRead is called. But now the behaviour of this
function needs to be changed a bit. Previously it just read data
from underlying file (of chardev or whatever) and sent those
through the stream to client. This model will not work any longer
because it does not differentiate whether underlying file is in
data or hole section. Therefore, at the beginning of this
function add code that checks this situation and acts
accordingly.
So after the this, when wanting to send some data we always check
whether we are not in a hole and if so, skip it an inform client
about its size.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Basically, whenever the new type of stream packet arrives to the
daemon call this function that decodes it and calls
virStreamSendHole(). Otherwise a regular data stream packet has
arrived and therefore continue its processing.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This is a special type of stream packet, that is bidirectional
and contains information regarding how many bytes each side will
be skipping in the stream.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Add a new argument to daemonCreateClientStream in order to allow for
future expansion to mark that a specific stream can be used to skip
data, such as the case with sparsely populated files. The new flag will
be the eventual decision point between client/server to decide whether
both ends can support and want to use sparse streams.
A new bool 'allowSkip' is added to both _virNetClientStream and
daemonClientStream in order to perform the tracking.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Based upon an idea and some research by Wang King <king.wang@huawei.com>
and xinhua.Cao <caoxinhua@huawei.com>.
Since we're assigning the 'client' to our callback event lookaside list,
it's imperative that we grab a reference to the object; otherwise, when
the object is unref'd during virNetServerProcessClients when it's determined
that the virNetServerClientIsClosed and the memory is free'd before perhaps
the object event state callbacks are run. When a virObjectLock() is run,
before sending the message the following trace occurs;
#0 0x00007fda223d66d8 in virClassIsDerivedFrom
(klass=0xdeadbeef, parent=0x7fda24c81b40)
at util/virobject.c:169
#1 0x00007fda223d6a1e in virObjectIsClass
(anyobj=anyobj@entry=0x7fd9e575b400, klass=<optimized out>)
at util/virobject.c:365
#2 0x00007fda223d6a44 in virObjectLock
(anyobj=0x7fd9e575b400)
at util/virobject.c:317
#3 0x00007fda22507f71 in virNetServerClientSendMessage
(client=client@entry=0x7fd9e575b400, msg=msg@entry=0x7fd9ec30de90)
at rpc/virnetserverclient.c:1422
#4 0x00007fda230d714d in remoteDispatchObjectEventSend
(client=0x7fd9e575b400, program=0x7fda24c844e0, procnr=348,
proc=0x7fda2310e5e0 <xdr_remote_domain_event_callback_tunable_msg>,
data=0x7ffc3857fdb0)
at remote.c:3803
#5 0x00007fda230dd71b in remoteRelayDomainEventTunable
(conn=<optimized out>, dom=0x7fda27cd7660, params=0x7fda27f3aae0,
nparams=1,opaque=0x7fd9e6c99e00)
at remote.c:1033
#6 0x00007fda224484cb in virDomainEventDispatchDefaultFunc
(conn=0x7fda27cd0120, event=0x7fda2736ea00, cb=0x7fda230dd610
<remoteRelayDomainEventTunable>, cbopaque=0x7fd9e6c99e00)
at conf/domain_event.c:1910
#7 0x00007fda22446871 in virObjectEventStateDispatchCallbacks
(callbacks=<optimized out>, callbacks=<optimized out>,
event=0x7fda2736ea00,state=0x7fda24ca3960)
at conf/object_event.c:722
#8 virObjectEventStateQueueDispatch
(callbacks=0x7fda24c65800, queue=0x7ffc3857fe90, state=0x7fda24ca3960)
at conf/object_event.c:736
#9 virObjectEventStateFlush (state=0x7fda24ca3960)
at conf/object_event.c:814
#10 virObjectEventTimer (timer=<optimized out>, opaque=0x7fda24ca3960)
at conf/object_event.c:560
#11 0x00007fda223ae8b9 in virEventPollDispatchTimeouts ()
at util/vireventpoll.c:458
#12 virEventPollRunOnce ()
at util/vireventpoll.c:654
#13 0x00007fda223ad1d2 in virEventRunDefaultImpl ()
at util/virevent.c:314
#14 0x00007fda225046cd in virNetDaemonRun (dmn=0x7fda24c775c0)
at rpc/virnetdaemon.c:818
#15 0x00007fda230d6351 in main (argc=<optimized out>, argv=<optimized out>)
at libvirtd.c:1623
Signed-off-by: John Ferlan <jferlan@redhat.com>
Rather than 'n' repetitive code segments, let's create a single macro
which will make the code easier to read.
Signed-off-by: John Ferlan <jferlan@redhat.com>
libvirtd can spawn threads/tasks when creating new domains for
some hypervisors such as Xen's libxl driver, quickly reaching
the cgroups pids controller default TasksMax setting of 512. When
the limit is reached, attempting to create additional domains
results in an error from the cgroups pids controller, e.g.
kernel: [71282.213347] cgroup: fork rejected by pids controller in
/system.slice/libvirtd.service
Depending on domain type and configuration, anywhere from 4-7
threads/tasks may be created by libxl when starting a domain.
In order to support 4096 domains, similar to commit 27cd763500,
increase the TasksMax setting in libvirtd.service to
4096 * 8 = 32768 tasks.
Introduce STRICT_FRAME_LIMIT_CFLAGS that will be used for
production code and RELAXED_FRAME_LIMIT_CFLAGS for tests.
Raising the limit for tests allows building them with clang
with optimizations disabled.
So far our code is full of the following pattern:
dom = virGetDomain(conn, name, uuid)
if (dom)
dom->id = 42;
There is no reasong why it couldn't be just:
dom = virGetDomain(conn, name, uuid, id);
After all, client domain representation consists of tuple (name,
uuid, id).
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
When using thin provisioning, management tools need to resize the disk
in certain cases. To avoid having them to poll disk usage introduce an
event which will be fired when a given offset of the storage is written
by the hypervisor. Together with the API which will be added later, it
will allow registering thresholds for given storage backing volumes and
this event will then notify management if the threshold is exceeded.
Currently 'virsh perf domain' errors out as the perf nparams is
incorrectly compared against REMOTE_DOMAIN_MEMORY_PARAMETERS_MAX
instead of REMOTE_DOMAIN_PERF_EVENTS_MAX.
Signed-off-by: Nitesh Konkar <nitkon12@linux.vnet.ibm.com>
When providing explicit x509 cert/key paths in libvirtd.conf,
the user must provide all three. If one or more is missed,
this leads to obscure errors at runtime when negotiating
the TLS session
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Linux still defaults to a 1024 open file handle limit. This causes
scalability problems for libvirtd / virtlockd / virtlogd on large
hosts which might want > 1024 guest to be running. In fact if each
guest needs > 1 FD, we can't even get to 500 guests. This is not
good enough when we see machines with 100's of physical cores and
TBs of RAM.
In comparison to other memory requirements of libvirtd & related
daemons, the resource usage associated with open file handles
is essentially line noise. It is thus reasonable to increase the
limits unconditionally for all installs.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
RFC 6331 documents a number of serious security weaknesses in
the SASL DIGEST-MD5 mechanism. As such, libvirtd should not
by using it as a default mechanism. GSSAPI is the only other
viable SASL mechanism that can provide secure session encryption
so enable that by defalt as the replacement.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Pass the registration function name to virDriverLoadModule so that we
can later call specific functions if necessary (e.g. for testing
purposes). This gets rid of the rather ugly automatic name generator and
unifies the code to load/initialize the modules.
It's also clear which registration function gets called.
Since a successful completion of the calls to openvswitch is expected
a longer timeout should be able to be chosen to account for loaded systems.
Therefore this patch provides the ability to specify the timeout value for
openvswitch calls in the libvirtd configuration file.
Signed-off-by: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Reviewed-by: Bjoern Walk <bwalk@linux.vnet.ibm.com>
Provide the ability to specify a default timeout value for
successful completion of openvswitch calls in the libvirtd
configuration file.
Signed-off-by: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Reviewed-by: Bjoern Walk <bwalk@linux.vnet.ibm.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
After deploying virtlogd by default we identified a number of
mistakes in the systemd unit file. virtlockd's relationship
to libvirtd is the same as virtlogd, so we must apply the
same unit file fixes to virtlockd
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
When changing the metadata via virDomainSetMetadata, we now
emit an event to notify the app of changes. This is useful
when co-ordinating different applications read/write of
custom metadata.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
https://bugzilla.redhat.com/show_bug.cgi?id=1332019
This function will essentially be a wrapper to virStorageVolInfo in order
to provide a mechanism to have the "physical" size of the volume returned
instead of the "allocation" size. This will provide similar capabilities to
the virDomainBlockInfo which can return both allocation and physical of a
domain storage volume.
NB: Since we're reusing the _virStorageVolInfo and not creating a new
_virStorageVolInfoFlags structure, we'll need to generate the rpc APIs
remoteStorageVolGetInfoFlags and remoteDispatchStorageVolGetInfoFlags
(although both were originally created from gendispatch.pl and then
just copied into daemon/remote.c and src/remote/remote_driver.c).
The new API will allow the usage of a VIR_STORAGE_VOL_GET_PHYSICAL flag
and will make the decision to return the physical or allocation value
into the allocation field.
In order to get that physical value, virStorageBackendUpdateVolTargetInfoFD
adds logic to fill in physical value matching logic in qemuStorageLimitsRefresh
used by virDomainBlockInfo when the domain is inactive.
Signed-off-by: John Ferlan <jferlan@redhat.com>
Enable libvirt users to modify daemon's logging output settings from outside.
If either an empty string or NULL is passed, a default logging output will be
used the same way as it would be in case writing an empty string to the
libvirtd.conf
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Now that virLog{Get,Set}DefaultOutput routines are introduced we can wire them
up to the daemon's logging initialization code. Also, change the order of
operations a bit so that we still strictly honor our precedence of settings:
cmdline > env > config now that outputs and filters are not appended anymore.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Along with an empty string, it should also be possible for users to pass
NULL to the public APIs which in turn would trigger a routine(future
work) responsible for defining an appropriate default logging output
given the current circumstances.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
When trying to install libvirtd from sources I've noticed the
following failure:
/usr/bin/install: cannot stat 'virt-guest-shutdown.target': No such file or directory
Makefile:2792: recipe for target 'install-init-systemd' failed
make[3]: *** [install-init-systemd] Error 1
make[3]: *** Waiting for unfinished jobs....
The problem is that while other files around that location in
Makefile are firstly generated into the builddir and only after
that installed, virt-guest-shutdown.target file is not generated
at all and should be installed from the srcdir.
This was introduced in 01079727.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
We have couple of functions that operate over NULL terminated
lits of strings. However, our naming sucks:
virStringJoin
virStringFreeList
virStringFreeListCount
virStringArrayHasString
virStringGetFirstWithPrefix
We can do better:
virStringListJoin
virStringListFree
virStringListFreeCount
virStringListHasString
virStringListGetFirstWithPrefix
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
It is already discussed in "[RFC] daemon: remove hardcode dep on libvirt-guests" [1].
Mgmt can use means to save/restore domains on system shutdown/boot other than
libvirt-guests.service. Thus we need to specify appropriate ordering dependency between
libvirtd, domains and save/restore service. This patch takes approach suggested
in RFC and introduces a systemd target, so that ordering can be built next way:
libvirtd -> domain -> virt-guest-shutdown.target -> save-restore.service.
This way domains are decoupled from specific shutdown service via intermediate
target.
[1] https://www.redhat.com/archives/libvir-list/2016-September/msg01353.html
Do not dereference the 'dmn' until after the virStateCleanup is completed.
During initialization, virStateInitialize requires/uses the "dmn" as the
argument to/for the daemonInhibitCallback functions. Thus, cleanup cannot
dereference 'dmn' until after calling the virStateCleanup which calls the
the daemonInhibitCallback using 'dmn'; otherwise, the following crash occurs:
backtrace (shortened a bit)
1 0x00007fd3a791b2e6 in virCondWait (c=<optimized out>, m=<optimized out>)
at util/virthread.c:154
2 0x00007fd3a791bcb0 in virThreadPoolFree (pool=0x7fd38024ee00)
at util/virthreadpool.c:266
3 0x00007fd38edaa00e in qemuStateCleanup () at qemu/qemu_driver.c:1116
4 0x00007fd3a79abfeb in virStateCleanup () at libvirt.c:808
5 0x00007fd3a85f2c9e in main (argc=<optimized out>, argv=<optimized out>)
at libvirtd.c:1660
Thread 1 (Thread 0x7fd38722d700 (LWP 32256)):
0 0x00007fd3a7900910 in virClassIsDerivedFrom
(klass=0xdfd36058d4853, parent=0x7fd3a8f394d0) at util/virobject.c:169
1 0x00007fd3a7900c4e in virObjectIsClass
(anyobj=anyobj@entry=0x7fd3a8f2f850, klass=<optimized out>)
at util/virobject.c:365
2 0x00007fd3a7900c74 in virObjectLock (anyobj=0x7fd3a8f2f850)
at util/virobject.c:317
3 0x00007fd3a7a24d5d in virNetDaemonRemoveShutdownInhibition
(dmn=0x7fd3a8f2f850) at rpc/virnetdaemon.c:547
4 0x00007fd38ed722cf in qemuProcessStop
(driver=driver@entry=0x7fd380103810, vm=vm@entry=0x7fd38025b6d0,
reason=reason@entry=VIR_DOMAIN_SHUTOFF_SHUTDOWN,
asyncJob=asyncJob@entry=QEMU_ASYNC_JOB_NONE, flags=flags@entry=0)
at qemu/qemu_process.c:5786
5 0x00007fd38edd9428 in processMonitorEOFEvent
(vm=0x7fd38025b6d0, driver=0x7fd380103810) at qemu/qemu_driver.c:4588
6 qemuProcessEventHandler (data=<optimized out>, opaque=0x7fd380103810)
at qemu/qemu_driver.c:4632
7 0x00007fd3a791bb55 in virThreadPoolWorker
(opaque=opaque@entry=0x7fd3a8f1e4c0) at util/virthreadpool.c:145
Similar to outputs, parser should do parsing only, thus the 'define' logic
is going to be stripped from virLogParseAndDefineFilters by replacing calls to
this method to virLogSetFilters instead.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Since virLogParseAndDefineOutputs is going to be stripped from 'output defining'
logic, replace all relevant occurrences with virLogSetOutputs call to make the
change transparent to all original callers (daemons mostly).
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Right now virLogParse* functions are doing both parsing and defining of filters
and outputs which should be two separate operations. Since the naming is
apparently a bit poor this patch renames these functions to
virLogParseAndDefine* which eventually will be replaced by virLogSet*.
Additionally, virLogParse{Filter,Output} will be later (after the split) reused,
so that these functions do exactly what the their name suggests.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
