Despite the misleading name, these were supposed to be used
with a System V style init; however, none of the platforms we
target is using that kind of init anymore: almost all Linux
distributions have switched to systemd, those that haven't
(such as Gentoo and Alpine) are mostly using OpenRC with
custom init scripts, and the BSDs have been doing their own
thing all along.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
According to the official documentation for autoconf[1], the
correct names for these variables are abs_top_{src,build}dir
rather than abs_top{src,build}dir; in fact, we're already
using the correct names in various places, so let's just make
everything nice and consistent.
[1] https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Preset-Output-Variables.html
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Missing semicolon at the end of macros can confuse some analyzers
(like cppcheck <filename>). VIR_ONCE_GLOBAL_INIT is almost
exclusively called without an ending semicolon, but let's
standardize on using one like the other macros.
Add a dummy struct definition at the end of the macro, so
the compiler will require callers to add a semicolon.
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Missing semicolon at the end of macros can confuse some analyzers
(like cppcheck <filename>), and we have a mix of semicolon and
non-semicolon usage through the code. Let's standardize on using
a semicolon for VIR_ENUM_IMPL calls.
Move the verify() statement to the end of the macro and drop
the semicolon, so the compiler will require callers to add a
semicolon.
While we are touching these call sites, standardize on putting
the closing parenth on its own line, as discussed here:
https://www.redhat.com/archives/libvir-list/2019-January/msg00750.html
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Missing semicolon at the end of macros can confuse some analyzers
(like cppcheck <filename>), and we have a mix of semicolon and
non-semicolon usage through the code. Let's standardize on using
a semicolon for VIR_ENUM_DECL calls.
Drop the semicolon from the final statement of the macro, so
the compiler will require callers to add a semicolon.
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
The virtualization driver has two connections to the virtlogd daemon,
one pipe fd for writing to the log file, and one socket fd for making
RPC calls. The typical sequence is to write some data to the pipe fd and
then make an RPC call to determine the current log file offset.
Unfortunately these two operations are not guaranteed to be handling in
order by virtlogd. The event loop for virtlogd may identify an incoming
event on both the pipe fd and socket fd in the same iteration of the
event loop. It is then entirely possible that it will process the socket
fd RPC call before reading the pending log data from the pipe fd.
As a result the virtualization driver will get an outdated log file
offset reported back.
This can be seen with the QEMU driver where, when a guest fails to
start, it will randomly include too much data in the error message it
has fetched from the log file.
The solution is to ensure we have drained all pending data from the pipe
fd before reporting the log file offset. The pipe fd is always in
blocking mode, so cares needs to be taken to avoid blocking. When
draining this is taken care of by using poll(). The extra complication
is that they might already be an event loop dispatch pending on the pipe
fd. If we have just drained the pipe this pending event will be invalid
so must be discarded.
See also https://bugzilla.redhat.com/show_bug.cgi?id=1356108
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Require that all headers are guarded by a symbol named
LIBVIRT_$FILENAME
where $FILENAME is the uppercased filename, with all characters
outside a-z changed into '_'.
Note we do not use a leading __ because that is technically a
namespace reserved for the toolchain.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This introduces a syntax-check script that validates header files use a
common layout:
/*
...copyright header...
*/
<one blank line>
#ifndef SYMBOL
# define SYMBOL
....content....
#endif /* SYMBOL */
For any file ending priv.h, before the #ifndef, we will require a
guard to prevent bogus imports:
#ifndef SYMBOL_ALLOW
# error ....
#endif /* SYMBOL_ALLOW */
<one blank line>
The many mistakes this script identifies are then fixed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
In many files there are header comments that contain an Author:
statement, supposedly reflecting who originally wrote the code.
In a large collaborative project like libvirt, any non-trivial
file will have been modified by a large number of different
contributors. IOW, the Author: comments are quickly out of date,
omitting people who have made significant contribitions.
In some places Author: lines have been added despite the person
merely being responsible for creating the file by moving existing
code out of another file. IOW, the Author: lines give an incorrect
record of authorship.
With this all in mind, the comments are useless as a means to identify
who to talk to about code in a particular file. Contributors will always
be better off using 'git log' and 'git blame' if they need to find the
author of a particular bit of code.
This commit thus deletes all Author: comments from the source and adds
a rule to prevent them reappearing.
The Copyright headers are similarly misleading and inaccurate, however,
we cannot delete these as they have legal meaning, despite being largely
inaccurate. In addition only the copyright holder is permitted to change
their respective copyright statement.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
All of the ones being removed are pulled in by internal.h. The only
exception is sanlock which expects the application to include <stdint.h>
before sanlock's headers, because sanlock prototypes use fixed width
int, but they don't include stdint.h themselves, so we have to leave
that one in place.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
Now that GnuTLS is a requirement, we can drop a lot of
conditionally built code. However, not all ifdef-s can go because
we still want libvirt_setuid to build without gnutls.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Strongly recommend against use of the log_levels setting since it
creates overly verbose logs and has a serious performance impact.
Describe the log filter syntax better and mention use of shell
glob syntax. Also provide more realistic example of good settings
to use. The libvirtd example is biased towards QEMU, but when the
drivers split off each daemon can get its own more appropriate
example.
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Rather than have virJSONValueArraySize return a -1 when the input
is not an array and then splat an error message, let's check for
an array before calling and then change the return to be a size_t
instead of ssize_t.
That means using the helper virJSONValueIsArray as well as using a
more generic error message such as "Malformed <something> array".
In some cases we can remove stack variables and when we cannot,
those variables should be size_t not ssize_t. Alter a few references
of if (!value) to be if (value == 0) instead as well.
Some callers can already assume an array is being worked on based
on the previous call, so there's less to do.
Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
That is a job of libvirtd and virtlogd has a dependency on it, so that will
prevent it properly. Doing it one extra time in virtlogd might also cause AVC
denials because it is not allowed to call that dbus method.
Caused by commit df34363d58.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1547250
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
So far we are repeating the following lines over and over:
if (!(virSomeObjectClass = virClassNew(virClassForObject(),
"virSomeObject",
sizeof(virSomeObject),
virSomeObjectDispose)))
return -1;
While this works, it is impossible to do some checking. Firstly,
the class name (the 2nd argument) doesn't match the name in the
code in all cases (the 3rd argument). Secondly, the current style
is needlessly verbose. This commit turns example into following:
if (!(VIR_CLASS_NEW(virSomeObject,
virClassForObject)))
return -1;
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Most of the augeas test files use ::CONFIG:: to pull in the master
config file for testing. This ensures that entries added to the config
file are actually tested by augeas.
This identified the missing admin_max_clients example in the virtlogd
config file, which in turn prompted a change in description of the
max_clients parameter, since these daemons don't have separate
readonly & readwrite sockets.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The global log buffer feature was deleted in:
commit c0c8c1d7bb
Author: Daniel P. Berrange <berrange@redhat.com>
Date: Mon Mar 3 14:54:33 2014 +0000
Remove global log buffer feature entirely
A earlier commit changed the global log buffer so that it only
records messages that are explicitly requested via the log
filters setting. This removes the performance burden, and
improves the signal/noise ratio for messages in the global
buffer. At the same time though, it is somewhat pointless, since
all the recorded log messages are already going to be sent to an
explicit log output like syslog, stderr or the journal. The
global log buffer is thus just duplicating this data on stderr
upon crash.
The log_buffer_size config parameter is left in the augeas
lens to prevent breakage for users on upgrade. It is however
completely ignored hereafter.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
This was in the 1.2.3 release, and 4 years is sufficient time for a
graceful upgrade path for augeas, so all remaining traces are now
removed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Currently both virtlogd and virtlockd use a single worker thread for
dispatching RPC messages. Even this is overkill and their RPC message
handling callbacks all run in short, finite time and so blocking the
main loop is not an issue like you'd see in libvirtd with long running
QEMU commands.
By setting max_workers==0, we can turn off the worker thread and run
these daemons single threaded. This in turn fixes a serious problem in
the virtlockd daemon whereby it loses all fcntl() locks at re-exec due
to multiple threads existing. fcntl() locks only get preserved if the
process is single threaded at time of exec().
Reviewed-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Add a virtlogd-admin-sock can serves the admin protocol for the virtlogd
daemon and define a virtlogd:///{system,session} URI scheme for
connecting to it.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
With the current code it is neccessary to call
virNetDaemonNewPostExecRestart()
and then for each server that needs restarting you are supposed
to call
virNetDaemonAddSeverPostExecRestart()
This is fine if there's only ever one server, but as soon as you
have two servers it is impossible to use this design. The code
has no idea which servers were recorded in the JSON state doc,
nor in which order the hash table serialized its keys.
So this patch changes things so that we only call
virNetDaemonNewPostExecRestart()
passing in a callback, which is invoked once for each server
found int he JSON state doc.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
We put the server into a hash table as we do with the other daemons,
there is no compelling reason why it should have another pointer
dedicated just to the server. Besides, the locking daemon doesn't have
it and virtlogd is essentially a copy paste of virtlockd.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
The initial assumption was ~2 files per guest, but some common setups
like Openstack drive up to 4 files per guest.
E.g. on Arm where the following XML leads to 4 file handles:
<serial type='file'>
<source path='/var/lib/nova/instances/7c0dcd78-.../console.log'/>
<target port='0'/>
<alias name='serial0'/>
</serial>
<console type='file'>
<source path='/var/lib/nova/instances/7c0dcd78-.../console.log'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
With that in mind and the target to support 4k guests by default we
should raise the limit to 16k.
Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Commit 94c465d0 refactored the logging setup phase but introduced an
issue, where the daemon ignores verbose mode when there are no outputs
defined and the default must be used. The problem is that the default
output was determined too early, thus ignoring the potential '--verbose'
option taking effect. This patch postpones the creation of the default
output to the very last moment when nothing else can change. Since the
default output is only created during the init phase, it's safe to leave
the pointer as NULL for a while, but it will be set eventually, thus not
affecting runtime.
Patch also adjusts both the other daemons.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1442947
Signed-off-by: Erik Skultety <eskultet@redhat.com>
The default conf files, for example libvirtd.conf, virtlockd.conf, and
virtlogd.conf, should be located under the directory "/etc/libvirt" when
root as root, rather than "/etc". When run as non-root, the configuration
files should be located under "$XDG_CONFIG_HOME/libvirt/", rather than
"XDG_CONFIG_HOME".
Signed-off-by: Lily Zhu <lizhu@redhat.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
The log and lock protocol don't have an extra handshake to close the
connection. Instead they just close the socket. Unfortunately that
resulted into a lot of spurious garbage logged to the system log files:
2017-03-17 14:00:09.730+0000: 4714: error : virNetSocketReadWire:1800 : End of file while reading data: Input/output error
or in the journal as:
Mar 13 16:19:33 xxxx virtlogd[32360]: End of file while reading data: Input/output error
Use the new facility in the netserverclient to suppress the IO error
report from the virNetSocket layer.
Linux still defaults to a 1024 open file handle limit. This causes
scalability problems for libvirtd / virtlockd / virtlogd on large
hosts which might want > 1024 guest to be running. In fact if each
guest needs > 1 FD, we can't even get to 500 guests. This is not
good enough when we see machines with 100's of physical cores and
TBs of RAM.
In comparison to other memory requirements of libvirtd & related
daemons, the resource usage associated with open file handles
is essentially line noise. It is thus reasonable to increase the
limits unconditionally for all installs.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Now that virLog{Get,Set}DefaultOutput routines are introduced we can wire them
up to the daemon's logging initialization code. Also, change the order of
operations a bit so that we still strictly honor our precedence of settings:
cmdline > env > config now that outputs and filters are not appended anymore.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Along with an empty string, it should also be possible for users to pass
NULL to the public APIs which in turn would trigger a routine(future
work) responsible for defining an appropriate default logging output
given the current circumstances.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Similar to outputs, parser should do parsing only, thus the 'define' logic
is going to be stripped from virLogParseAndDefineFilters by replacing calls to
this method to virLogSetFilters instead.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Since virLogParseAndDefineOutputs is going to be stripped from 'output defining'
logic, replace all relevant occurrences with virLogSetOutputs call to make the
change transparent to all original callers (daemons mostly).
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Right now virLogParse* functions are doing both parsing and defining of filters
and outputs which should be two separate operations. Since the naming is
apparently a bit poor this patch renames these functions to
virLogParseAndDefine* which eventually will be replaced by virLogSet*.
Additionally, virLogParse{Filter,Output} will be later (after the split) reused,
so that these functions do exactly what the their name suggests.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Commit 839a060 tied the lifecycle of virtlogd more
closely to that of libvirtd. Unfortunately, while starting
virtlogd when libvirtd is started is definitely a good idea,
restarting virtlogd or shutting it down at any time outside
of system poweroff is not.
Revert part of that commit by removing the PartOf= lines,
meaning that only startup requests will be propagated from
libvirtd to virtlogd.
Resolves: https://bugzilla.redhat.com/1372576
We already guarantee that virtlogd.socket is enabled/disabled
along with libvirtd.service, but if libvirtd.service has just
been installed and is started before rebooting, then
virtlogd.socket will not be running and guest startup will
fail.
Add Requires=virtlogd.socket to libvirtd.service to make sure
virtlogd.socket is always started along with libvirtd.service,
and add Before=libvirtd.service to both virtlogd.socket and
virtlogd.service so that virtlogd never disappears before
libvirtd has exited.
Also add PartOf=libvirtd.service to both virtlogd.socket and
virtlogd.service, so that virtlogd can be shut down when not
needed.
Resolves: https://bugzilla.redhat.com/1372576
People debugging guest OS boot processes and reported that
the default 128 KB size is too small to capture an entire
boot up sequence. Increase the default size to 2 MB which
should allow capturing a full boot up even with verbose
debugging.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Currently virtlogd has a hardcoded max file size of 128kb
and max of 3 backups. This adds two new config parameters
to /etc/libvirt/virtlogd.conf to let these be customized.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
For logging one-shot entries to the VM log file it's quite a waste to
hold open the file descriptor for logging that is provided by the
current API.
This new API will be ideal for logging one-shot entries to the file
e.g. at the point when we shut the VM down rather than having to add the
whole file-descriptor infrastructure.
Additionally this will allow to add the messages even after restart of
libvirtd since virtlogd doesn't allow to obtain a regular context with
filedescriptors while the VM is still active.
Admin API needs a way of addressing specific clients. Unlike servers, which we
are happy to address by names both because its name reflects its purpose (to
some extent) and we only have two of them (so far), naming clients doesn't make
any sense, since a) each client is an anonymous, i.e. not recognized after a
disconnect followed by a reconnect, b) we can't predict what kind of requests
it's going to send to daemon, and c) the are loads of them comming and going,
so the only viable option is to use an ID which is of a reasonably wide data
type.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
After this commit, all man pages are generated using the same two
steps:
1. Process a source $command.pod file with pod2man(1) to obtain
a valid man page in $command.$section.in
2. Process $command.$section.in with sed(1) to obtain the final
man page in $command.$section
Since servers know their name, there is no need to supply such
information twice. Also defeats inconsistencies.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
At first I did not want to do this, but after trying to implement some
newer feaures in the admin API I realized we need that to make our lives
easier. On the other hand they are not saved redundantly and the
virNetServer objects are still kept in a hash table.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
