This affects more than src/Makefile.am as the rule to generate source
files for protocols is generic for all sub-directories.
Affected files are:
src/admin/admin_protocol.{h,c}
src/locking/lock_protocol.{h,c}
src/logging/log_protocol.{h,c}
src/lxc/lxc_monitor_protocol.{h,c}
src/remote/{lxc,qemu,remote}_protocol.{h,c}
src/rpc/{virkeepalive,virnet}protocol.{h,c}
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Follow the same pattern as for other sub-directories where we create a
static library that is linked into libvirt.so.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Replace all occurrences of
if (VIR_STRDUP(a, b) < 0)
/* effectively dead code */
with:
a = g_strdup(b);
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The log filters have supported the use of a "+" before the source match
string to request that a stack trace be emitted for every log message:
commit 548563956e
Author: Daniel P. Berrange <berrange@redhat.com>
Date: Wed May 9 15:18:56 2012 +0100
Allow stack traces to be included with log messages
Sometimes it is useful to see the callpath for log messages.
This change enhances the log filter syntax so that stack traces
can be show by setting '1:+NAME' instead of '1:NAME'.
With the huge & ever increasing number of logging statements per file,
this will be incredibly verbose and have a major performance penalty.
This makes the feature impractical to use widely and as such it is not
worth the code maint cost.
Removing this seldom used feature allows us to drop the 'execinfo'
module in gnulib which provides the backtrace() function which doesn't
exist on non-Linux.
Users who want to get stack traces of parts of libvirt can use GDB,
or systemtap for live tracing with minimal perf impact.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The use of $(AUG_GENTEST) as a dependency in the makefiles is
a problem because this was assumed to be the filename of the
script, but is in fact a full shell command line.
Split it into two variables, so it can be correctly used for
dependencies.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Since commit 44e7f02915
util: rewrite auto cleanup macros to use glib's equivalent
VIR_AUTOPTR aliases to g_autoptr. Replace all of its use by the GLib
macro version.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Use G_GNUC_UNUSED from GLib instead of ATTRIBUTE_UNUSED.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Add the main glib.h to internal.h so that all common code can use it.
Historically glib allowed applications to register an alternative
memory allocator, so mixing g_malloc/g_free with malloc/free was not
safe.
This was feature was dropped in 2.46.0 with:
commit 3be6ed60aa58095691bd697344765e715a327fc1
Author: Alexander Larsson <alexl@redhat.com>
Date: Sat Jun 27 18:38:42 2015 +0200
Deprecate and drop support for memory vtables
Applications are still encourged to match g_malloc/g_free, but it is no
longer a mandatory requirement for correctness, just stylistic. This is
explicitly clarified in
commit 1f24b36607bf708f037396014b2cdbc08d67b275
Author: Daniel P. Berrangé <berrange@redhat.com>
Date: Thu Sep 5 14:37:54 2019 +0100
gmem: clarify that g_malloc always uses the system allocator
Applications can still use custom allocators in general, but they must
do this by linking to a library that replaces the core malloc/free
implemenentation entirely, instead of via a glib specific call.
This means that libvirt does not need to be concerned about use of
g_malloc/g_free causing an ABI change in the public libary, and can
avoid memory copying when talking to external libraries.
This patch probes for glib, which provides the foundation layer with
a collection of data structures, helper APIs, and platform portability
logic.
Later patches will introduce linkage to gobject which provides the
object type system, built on glib, and gio which providing objects
for various interesting tasks, most notably including DBus client
and server support and portable sockets APIs, but much more too.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Neither virThreadInitialize or virThreadOnExit do anything since we
dropped the Win32 threads impl, in favour of win-pthreads with:
commit 0240d94c36
Author: Daniel P. Berrange <berrange@redhat.com>
Date: Wed Jan 22 16:17:10 2014 +0000
Remove windows thread implementation in favour of pthreads
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The Perl bindings for libvirt use the test driver for unit tests. This
tries to load the cpu_map/index.xml file, and when run from an
uninstalled build will fail.
The problem is that virFileActivateDirOverride is called by our various
binaries like libvirtd, virsh, but is not called when a 3rd party app
uses libvirt.so
To deal with this we allow the LIBVIRT_DIR_OVERRIDE=1 env variable to be
set and make virInitialize look for this. The 'run' script will set it,
so now build using this script to run against an uninstalled tree we
will correctly resolve files to the source tree.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
All code using LOCALSTATEDIR "/run" is updated to use RUNSTATEDIR
instead. The exception is the remote driver client which still
uses LOCALSTATEDIR "/run". The client needs to connect to remote
machines which may not be using /run, so /var/run is more portable
due to the /var/run -> /run symlink.
Some duplicate paths in the apparmor code are also purged.
There's no functional change by default yet since both expressions
expand to the same value.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
If a systemd socket uses /var/run in its path, systemd prints a warning
at runtime
[ 15.139976] systemd[1]: /usr/lib/systemd/system/virtlockd.socket:5:
ListenStream= references a path below legacy directory /var/run/,
updating /var/run/libvirt/virtlockd-sock → /run/libvirt/virtlockd-sock;
please update the unit file accordingly.
This minimal change updates the socket unit files to honour the
$runstatedir path.
There's no functional change by default yet since both expressions
expand to the same value.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The make logic assumes that the SYSTEMD_UNIT_FILES var can be built from
SYSTEMD_UNIT_FILES_IN by simply dropping the directory prefix and the
.in suffix.
This won't work in future when a single .in unit file can be used to
generate multiple different units.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Substitute in the @sysconfigdir@ value instead of /etc.
Reviewed-by: Christophe de Dinechin <dinechin@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Using @VARNAME@ is a normal style of automake, so lets match that.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Instead of each subdir containing its own custom rule for checking the
augeas tests, use common rule for all.
The new rule searches both src + build dirs for include files, since
some augeas files will be auto-generated very shortly.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The current make rules are inconsistent about which directory the
augeas test files are created in. Put them all in the same dir as
their source.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We already have a variable that lists all augeas test files, so we can
add everything to CLEANFILES at once.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The augeas-gentest.pl program merges a config file into a augeas
file, saving the output to a new file. It is going to be useful
to further process the output file, and it would be easier if this can
be done with a pipeline, so change augeas-gentest.pl to write to stdout
instead of a file.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Instead of having each caller pass in the desired logfile name, pass in
the binary name instead. The logging code can then just derive a logfile
name by appending ".log".
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Using the new system activation APIs allows for simpler code setting up
the network services.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Currently the socket code will unlink any UNIX socket path which is
associated with a server socket. This is not fine grained enough, as we
need to avoid unlinking server sockets we were passed by systemd.
To deal with this we must explicitly track whether each socket needs to
be unlinked when closed, separately of the client vs server state.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The virNetServerServiceNewFD API only accepts a single FD, but it is
easily changed to allow for an array of FDs to be passed in.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The virtlogd config is set to rollover logs every 2 MB.
Normally a logrotate config file is also installed to handle cases where
virtlogd is disabled. This is set to rollover weekly with no size
constraint.
As a result logrotate can interfere with virtlogd's, rolling over files
that virtlogd has already taken care of.
This changes logrotate configs to rollover based on a max size
constraint of 2 MB + 1 byte. When virtlogd is running the log files will
never get this large, making logrotate a no-op.
If the user changes the size in virtlogd's config to something larger,
they are responsible for also changing the logrotate config suitably.
The LXC/libxl drivers don't use virtlogd, but there logrotate config is
altered to match the QEMU driver config, for the sake of consistency.
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
It doesn't make sense to have the admin socket active if the main
socket is not running, so bind their lifecycle together.
This ensures that if primary socket is stopped, the corresponding
admin socket is also stopped.
In the reverse, starting the admin socket will also automatically
start the primary socket.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Libvirtd has long had integration with avahi for advertising libvirtd
using mDNS when TCP/TLS listening is enabled. For a long time the
virt-manager application had support for auto-detecting libvirtds
on the local network using mDNS, but this was removed last year
commit fc8f8d5d7e3ba80a0771df19cf20e84a05ed2422
Author: Cole Robinson <crobinso@redhat.com>
Date: Sat Oct 6 20:55:31 2018 -0400
connect: Drop avahi support
Libvirtd can advertise itself over avahi. The feature is disabled by
default though and in practice I hear of no one actually using it
and frankly I don't think it's all that useful
The 'Open Connection' wizard has a disproportionate amount of code
devoted to this feature, but I don't think it's useful or worth
maintaining, so let's drop it
I've never heard of any other applications having support for using
mDNS to detect libvirtd instances. Though it is theoretically possible
something exists out there, it is clearly going to be a niche use case
in the virt ecosystem as a whole.
By removing avahi integration we can cut down the dependency chain for
the basic libvirtd install and reduce our code maint burden.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The virtlogd daemon's only intended client is the libvirtd daemon. As
such it should never allow clients from other user accounts to connect.
The code already enforces this and drops clients from other UIDs, but
we can get earlier (and thus stronger) protection against DoS by setting
the socket permissions to 0600
Fixes CVE-2019-10132
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Various binaries are statically linking to libvirt_util.la and
other intermediate libraries we build. These intermediate libs
all get built into the main libvirt.so shared library eventually,
so we can dynamically link to that instead and reduce the on disk
footprint.
In libvirt-daemon RPM:
virtlockd: 1.6 MB -> 153 KB
virtlogd: 1.6 MB -> 157 KB
libvirt_iohelper: 937 KB -> 23 KB
In libvirt-daemon-driver-network RPM:
libvirt_leaseshelper: 940 KB -> 26 KB
In libvirt-daemon-driver-storage-core RPM:
libvirt_parthelper: 926 KB -> 21 KB
IOW, about 5.6 MB total space saving in a build done on Fedora 30
x86_64 architecture.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Vim has trouble figuring out the filetype automatically because
the name doesn't follow existing conventions; annotations like
the ones we already have in Makefile.ci help it out.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
A bunch of files include src/rpc/virnetsaslcontext.h, which
in turn includes <sasl/sasl.h>, and without the corresponding
CFLAGS the compiler can't locate the latter if it happens to
be installed outside of the default include path as is the
case, for example, on FreeBSD.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Standardize on putting the _LAST enum value on the second line
of VIR_ENUM_IMPL invocations. Later patches that add string labels
to VIR_ENUM_IMPL will push most of these to the second line anyways,
so this saves some noise.
Signed-off-by: Cole Robinson <crobinso@redhat.com>
This reverts commit a5e1602090.
Getting rid of unistd.h from our headers will require more work than
just fixing the broken mingw build. Revert it until I have a more
complete proposal.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
util/virutil.h bogously included unistd.h. Drop it and replace it by
including it directly where needed.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
virutil.(c|h) is a very gross collection of random code. Remove the enum
handlers from there so we can limit the scope where virtutil.h is used.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Despite the misleading name, these were supposed to be used
with a System V style init; however, none of the platforms we
target is using that kind of init anymore: almost all Linux
distributions have switched to systemd, those that haven't
(such as Gentoo and Alpine) are mostly using OpenRC with
custom init scripts, and the BSDs have been doing their own
thing all along.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
According to the official documentation for autoconf[1], the
correct names for these variables are abs_top_{src,build}dir
rather than abs_top{src,build}dir; in fact, we're already
using the correct names in various places, so let's just make
everything nice and consistent.
[1] https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Preset-Output-Variables.html
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Missing semicolon at the end of macros can confuse some analyzers
(like cppcheck <filename>). VIR_ONCE_GLOBAL_INIT is almost
exclusively called without an ending semicolon, but let's
standardize on using one like the other macros.
Add a dummy struct definition at the end of the macro, so
the compiler will require callers to add a semicolon.
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Missing semicolon at the end of macros can confuse some analyzers
(like cppcheck <filename>), and we have a mix of semicolon and
non-semicolon usage through the code. Let's standardize on using
a semicolon for VIR_ENUM_IMPL calls.
Move the verify() statement to the end of the macro and drop
the semicolon, so the compiler will require callers to add a
semicolon.
While we are touching these call sites, standardize on putting
the closing parenth on its own line, as discussed here:
https://www.redhat.com/archives/libvir-list/2019-January/msg00750.html
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
Missing semicolon at the end of macros can confuse some analyzers
(like cppcheck <filename>), and we have a mix of semicolon and
non-semicolon usage through the code. Let's standardize on using
a semicolon for VIR_ENUM_DECL calls.
Drop the semicolon from the final statement of the macro, so
the compiler will require callers to add a semicolon.
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Cole Robinson <crobinso@redhat.com>
The virtualization driver has two connections to the virtlogd daemon,
one pipe fd for writing to the log file, and one socket fd for making
RPC calls. The typical sequence is to write some data to the pipe fd and
then make an RPC call to determine the current log file offset.
Unfortunately these two operations are not guaranteed to be handling in
order by virtlogd. The event loop for virtlogd may identify an incoming
event on both the pipe fd and socket fd in the same iteration of the
event loop. It is then entirely possible that it will process the socket
fd RPC call before reading the pending log data from the pipe fd.
As a result the virtualization driver will get an outdated log file
offset reported back.
This can be seen with the QEMU driver where, when a guest fails to
start, it will randomly include too much data in the error message it
has fetched from the log file.
The solution is to ensure we have drained all pending data from the pipe
fd before reporting the log file offset. The pipe fd is always in
blocking mode, so cares needs to be taken to avoid blocking. When
draining this is taken care of by using poll(). The extra complication
is that they might already be an event loop dispatch pending on the pipe
fd. If we have just drained the pipe this pending event will be invalid
so must be discarded.
See also https://bugzilla.redhat.com/show_bug.cgi?id=1356108
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Require that all headers are guarded by a symbol named
LIBVIRT_$FILENAME
where $FILENAME is the uppercased filename, with all characters
outside a-z changed into '_'.
Note we do not use a leading __ because that is technically a
namespace reserved for the toolchain.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This introduces a syntax-check script that validates header files use a
common layout:
/*
...copyright header...
*/
<one blank line>
#ifndef SYMBOL
# define SYMBOL
....content....
#endif /* SYMBOL */
For any file ending priv.h, before the #ifndef, we will require a
guard to prevent bogus imports:
#ifndef SYMBOL_ALLOW
# error ....
#endif /* SYMBOL_ALLOW */
<one blank line>
The many mistakes this script identifies are then fixed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
In many files there are header comments that contain an Author:
statement, supposedly reflecting who originally wrote the code.
In a large collaborative project like libvirt, any non-trivial
file will have been modified by a large number of different
contributors. IOW, the Author: comments are quickly out of date,
omitting people who have made significant contribitions.
In some places Author: lines have been added despite the person
merely being responsible for creating the file by moving existing
code out of another file. IOW, the Author: lines give an incorrect
record of authorship.
With this all in mind, the comments are useless as a means to identify
who to talk to about code in a particular file. Contributors will always
be better off using 'git log' and 'git blame' if they need to find the
author of a particular bit of code.
This commit thus deletes all Author: comments from the source and adds
a rule to prevent them reappearing.
The Copyright headers are similarly misleading and inaccurate, however,
we cannot delete these as they have legal meaning, despite being largely
inaccurate. In addition only the copyright holder is permitted to change
their respective copyright statement.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
All of the ones being removed are pulled in by internal.h. The only
exception is sanlock which expects the application to include <stdint.h>
before sanlock's headers, because sanlock prototypes use fixed width
int, but they don't include stdint.h themselves, so we have to leave
that one in place.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Michal Privoznik <mprivozn@redhat.com>
Now that GnuTLS is a requirement, we can drop a lot of
conditionally built code. However, not all ifdef-s can go because
we still want libvirt_setuid to build without gnutls.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Strongly recommend against use of the log_levels setting since it
creates overly verbose logs and has a serious performance impact.
Describe the log filter syntax better and mention use of shell
glob syntax. Also provide more realistic example of good settings
to use. The libvirtd example is biased towards QEMU, but when the
drivers split off each daemon can get its own more appropriate
example.
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Rather than have virJSONValueArraySize return a -1 when the input
is not an array and then splat an error message, let's check for
an array before calling and then change the return to be a size_t
instead of ssize_t.
That means using the helper virJSONValueIsArray as well as using a
more generic error message such as "Malformed <something> array".
In some cases we can remove stack variables and when we cannot,
those variables should be size_t not ssize_t. Alter a few references
of if (!value) to be if (value == 0) instead as well.
Some callers can already assume an array is being worked on based
on the previous call, so there's less to do.
Signed-off-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
That is a job of libvirtd and virtlogd has a dependency on it, so that will
prevent it properly. Doing it one extra time in virtlogd might also cause AVC
denials because it is not allowed to call that dbus method.
Caused by commit df34363d58.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1547250
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
So far we are repeating the following lines over and over:
if (!(virSomeObjectClass = virClassNew(virClassForObject(),
"virSomeObject",
sizeof(virSomeObject),
virSomeObjectDispose)))
return -1;
While this works, it is impossible to do some checking. Firstly,
the class name (the 2nd argument) doesn't match the name in the
code in all cases (the 3rd argument). Secondly, the current style
is needlessly verbose. This commit turns example into following:
if (!(VIR_CLASS_NEW(virSomeObject,
virClassForObject)))
return -1;
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Most of the augeas test files use ::CONFIG:: to pull in the master
config file for testing. This ensures that entries added to the config
file are actually tested by augeas.
This identified the missing admin_max_clients example in the virtlogd
config file, which in turn prompted a change in description of the
max_clients parameter, since these daemons don't have separate
readonly & readwrite sockets.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The global log buffer feature was deleted in:
commit c0c8c1d7bb
Author: Daniel P. Berrange <berrange@redhat.com>
Date: Mon Mar 3 14:54:33 2014 +0000
Remove global log buffer feature entirely
A earlier commit changed the global log buffer so that it only
records messages that are explicitly requested via the log
filters setting. This removes the performance burden, and
improves the signal/noise ratio for messages in the global
buffer. At the same time though, it is somewhat pointless, since
all the recorded log messages are already going to be sent to an
explicit log output like syslog, stderr or the journal. The
global log buffer is thus just duplicating this data on stderr
upon crash.
The log_buffer_size config parameter is left in the augeas
lens to prevent breakage for users on upgrade. It is however
completely ignored hereafter.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
This was in the 1.2.3 release, and 4 years is sufficient time for a
graceful upgrade path for augeas, so all remaining traces are now
removed.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Currently both virtlogd and virtlockd use a single worker thread for
dispatching RPC messages. Even this is overkill and their RPC message
handling callbacks all run in short, finite time and so blocking the
main loop is not an issue like you'd see in libvirtd with long running
QEMU commands.
By setting max_workers==0, we can turn off the worker thread and run
these daemons single threaded. This in turn fixes a serious problem in
the virtlockd daemon whereby it loses all fcntl() locks at re-exec due
to multiple threads existing. fcntl() locks only get preserved if the
process is single threaded at time of exec().
Reviewed-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Add a virtlogd-admin-sock can serves the admin protocol for the virtlogd
daemon and define a virtlogd:///{system,session} URI scheme for
connecting to it.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
With the current code it is neccessary to call
virNetDaemonNewPostExecRestart()
and then for each server that needs restarting you are supposed
to call
virNetDaemonAddSeverPostExecRestart()
This is fine if there's only ever one server, but as soon as you
have two servers it is impossible to use this design. The code
has no idea which servers were recorded in the JSON state doc,
nor in which order the hash table serialized its keys.
So this patch changes things so that we only call
virNetDaemonNewPostExecRestart()
passing in a callback, which is invoked once for each server
found int he JSON state doc.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
We put the server into a hash table as we do with the other daemons,
there is no compelling reason why it should have another pointer
dedicated just to the server. Besides, the locking daemon doesn't have
it and virtlogd is essentially a copy paste of virtlockd.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
The initial assumption was ~2 files per guest, but some common setups
like Openstack drive up to 4 files per guest.
E.g. on Arm where the following XML leads to 4 file handles:
<serial type='file'>
<source path='/var/lib/nova/instances/7c0dcd78-.../console.log'/>
<target port='0'/>
<alias name='serial0'/>
</serial>
<console type='file'>
<source path='/var/lib/nova/instances/7c0dcd78-.../console.log'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
With that in mind and the target to support 4k guests by default we
should raise the limit to 16k.
Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Commit 94c465d0 refactored the logging setup phase but introduced an
issue, where the daemon ignores verbose mode when there are no outputs
defined and the default must be used. The problem is that the default
output was determined too early, thus ignoring the potential '--verbose'
option taking effect. This patch postpones the creation of the default
output to the very last moment when nothing else can change. Since the
default output is only created during the init phase, it's safe to leave
the pointer as NULL for a while, but it will be set eventually, thus not
affecting runtime.
Patch also adjusts both the other daemons.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1442947
Signed-off-by: Erik Skultety <eskultet@redhat.com>
The default conf files, for example libvirtd.conf, virtlockd.conf, and
virtlogd.conf, should be located under the directory "/etc/libvirt" when
root as root, rather than "/etc". When run as non-root, the configuration
files should be located under "$XDG_CONFIG_HOME/libvirt/", rather than
"XDG_CONFIG_HOME".
Signed-off-by: Lily Zhu <lizhu@redhat.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
The log and lock protocol don't have an extra handshake to close the
connection. Instead they just close the socket. Unfortunately that
resulted into a lot of spurious garbage logged to the system log files:
2017-03-17 14:00:09.730+0000: 4714: error : virNetSocketReadWire:1800 : End of file while reading data: Input/output error
or in the journal as:
Mar 13 16:19:33 xxxx virtlogd[32360]: End of file while reading data: Input/output error
Use the new facility in the netserverclient to suppress the IO error
report from the virNetSocket layer.
Linux still defaults to a 1024 open file handle limit. This causes
scalability problems for libvirtd / virtlockd / virtlogd on large
hosts which might want > 1024 guest to be running. In fact if each
guest needs > 1 FD, we can't even get to 500 guests. This is not
good enough when we see machines with 100's of physical cores and
TBs of RAM.
In comparison to other memory requirements of libvirtd & related
daemons, the resource usage associated with open file handles
is essentially line noise. It is thus reasonable to increase the
limits unconditionally for all installs.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Now that virLog{Get,Set}DefaultOutput routines are introduced we can wire them
up to the daemon's logging initialization code. Also, change the order of
operations a bit so that we still strictly honor our precedence of settings:
cmdline > env > config now that outputs and filters are not appended anymore.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Along with an empty string, it should also be possible for users to pass
NULL to the public APIs which in turn would trigger a routine(future
work) responsible for defining an appropriate default logging output
given the current circumstances.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Similar to outputs, parser should do parsing only, thus the 'define' logic
is going to be stripped from virLogParseAndDefineFilters by replacing calls to
this method to virLogSetFilters instead.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Since virLogParseAndDefineOutputs is going to be stripped from 'output defining'
logic, replace all relevant occurrences with virLogSetOutputs call to make the
change transparent to all original callers (daemons mostly).
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Right now virLogParse* functions are doing both parsing and defining of filters
and outputs which should be two separate operations. Since the naming is
apparently a bit poor this patch renames these functions to
virLogParseAndDefine* which eventually will be replaced by virLogSet*.
Additionally, virLogParse{Filter,Output} will be later (after the split) reused,
so that these functions do exactly what the their name suggests.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Commit 839a060 tied the lifecycle of virtlogd more
closely to that of libvirtd. Unfortunately, while starting
virtlogd when libvirtd is started is definitely a good idea,
restarting virtlogd or shutting it down at any time outside
of system poweroff is not.
Revert part of that commit by removing the PartOf= lines,
meaning that only startup requests will be propagated from
libvirtd to virtlogd.
Resolves: https://bugzilla.redhat.com/1372576
We already guarantee that virtlogd.socket is enabled/disabled
along with libvirtd.service, but if libvirtd.service has just
been installed and is started before rebooting, then
virtlogd.socket will not be running and guest startup will
fail.
Add Requires=virtlogd.socket to libvirtd.service to make sure
virtlogd.socket is always started along with libvirtd.service,
and add Before=libvirtd.service to both virtlogd.socket and
virtlogd.service so that virtlogd never disappears before
libvirtd has exited.
Also add PartOf=libvirtd.service to both virtlogd.socket and
virtlogd.service, so that virtlogd can be shut down when not
needed.
Resolves: https://bugzilla.redhat.com/1372576
People debugging guest OS boot processes and reported that
the default 128 KB size is too small to capture an entire
boot up sequence. Increase the default size to 2 MB which
should allow capturing a full boot up even with verbose
debugging.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Currently virtlogd has a hardcoded max file size of 128kb
and max of 3 backups. This adds two new config parameters
to /etc/libvirt/virtlogd.conf to let these be customized.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
For logging one-shot entries to the VM log file it's quite a waste to
hold open the file descriptor for logging that is provided by the
current API.
This new API will be ideal for logging one-shot entries to the file
e.g. at the point when we shut the VM down rather than having to add the
whole file-descriptor infrastructure.
Additionally this will allow to add the messages even after restart of
libvirtd since virtlogd doesn't allow to obtain a regular context with
filedescriptors while the VM is still active.
Admin API needs a way of addressing specific clients. Unlike servers, which we
are happy to address by names both because its name reflects its purpose (to
some extent) and we only have two of them (so far), naming clients doesn't make
any sense, since a) each client is an anonymous, i.e. not recognized after a
disconnect followed by a reconnect, b) we can't predict what kind of requests
it's going to send to daemon, and c) the are loads of them comming and going,
so the only viable option is to use an ID which is of a reasonably wide data
type.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
After this commit, all man pages are generated using the same two
steps:
1. Process a source $command.pod file with pod2man(1) to obtain
a valid man page in $command.$section.in
2. Process $command.$section.in with sed(1) to obtain the final
man page in $command.$section
Since servers know their name, there is no need to supply such
information twice. Also defeats inconsistencies.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
At first I did not want to do this, but after trying to implement some
newer feaures in the admin API I realized we need that to make our lives
easier. On the other hand they are not saved redundantly and the
virNetServer objects are still kept in a hash table.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
The virtlogd daemon currently opens all files for append, but
in some cases the user may wish to discard existing data. Define
a new flag to indicate that logfiles should be truncated when
opening.
Not all callers of virLogManagerDomainOpenLogFile will
care about getting the current inode/offset, so we should
allow those parameters to be NULL
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Since the daemon can manage and add (at fresh start) multiple servers,
we also should be able to add them from a JSON state file in case of a
daemon restart, so post exec restart support for multiple servers is also
provided. Patch also updates virnetdaemontest accordingly.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Since we pass dummy variables @fdout and @fdoutlen into
virNetClientProgramCall() we make it alloc @fdout array (even
though it's an array of 0 elements since vitlogd can hardly pass
us some FDs at this stage). Nevertheless, it's an allocation not
followed by free():
==29385== 0 bytes in 60 blocks are definitely lost in loss record 2 of 1,009
==29385== at 0x4C2C070: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==29385== by 0x54B99EF: virAllocN (viralloc.c:191)
==29385== by 0x56821B1: virNetClientProgramCall (virnetclientprogram.c:359)
==29385== by 0x563B304: virLogManagerDomainReadLogFile (log_manager.c:272)
==29385== by 0x217CD613: qemuDomainLogContextRead (qemu_domain.c:2485)
==29385== by 0x217EDC76: qemuProcessReadLog (qemu_process.c:1660)
==29385== by 0x217EDE1D: qemuProcessReportLogError (qemu_process.c:1696)
==29385== by 0x217EE8C1: qemuProcessWaitForMonitor (qemu_process.c:1957)
==29385== by 0x217F6636: qemuProcessLaunch (qemu_process.c:4955)
==29385== by 0x217F71A4: qemuProcessStart (qemu_process.c:5152)
==29385== by 0x21846582: qemuDomainObjStart (qemu_driver.c:7396)
==29385== by 0x218467DE: qemuDomainCreateWithFlags (qemu_driver.c:7450)
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Commit 2b6f6ad introduced the virxdrdefs.h header with
common definitions to be included in the protocol files,
but logging/log_protocol.x was missed, so add it there as well.
Hopefully this fixes build on OS X.
Commit b22344f328 mistakenly reordered
Default-* lines. Thanks to that I noticed that we are very inconsistent
with our init scripts, so I took the liberty of synchronizing them,
updating them and making them all look shiny and new. So apart from
fixing the LSB requirements, I also fixed the ordering, specified
runlevels and fix the link to the reference specification.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Currently pkg build of master branch fails:
[ 300s] + /usr/lib/rpm/brp-boot-scripts
[ 300s] E: File `virtlogd' is missing `Required-Start', please add even if empty!
[ 300s] W: File `virtlogd' is missing `Required-Stop', please add even if empty!
[ 300s] E: File `virtlogd' has empty `Default-Start', please specify default runlevel(s)!
[ 300s] ERROR: found one or more broken init or boot scripts, please fix them.
[ 300s] For more information about LSB headers please read the manual
[ 300s] page of of insserv by executing the command `man 8 insserv'.
[ 300s] If you don't understand this, mailto=werner@suse.de
[ 300s] error: Bad exit status from /var/tmp/rpm-tmp.44965 (%install)
Add the required tags, fix the existing tags.
Use soft dependency "Should-Start" because virtlogd may work without network.
Signed-off-by: Olaf Hering <olaf@aepfle.de>
The virtlogd initscript's lock file should go in /var/lock/subsys/, not
(the nonexistent) /var/log/subsys/.
Signed-off-by: Michael Chapman <mike@very.puzzling.org>
The virtlogd RPC messages all have a flags parameter. For
sake of future error reporting we should be verifying
these are all 0 for now.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The current virtlogd RPC protocol provides the ability to
handle log files associated with QEMU stdout/err. The log
protocol messages take the virt driver, domain name and
use that to form a log file path. This is quite restrictive
as it prevents us re-using the same RPC protocol messages
for logging to char device backends where the filename
can be arbitrarily user specified. It is also bad because
it means we have 2 separate locations which have to decide
on logfile name.
This change alters the RPC protocol so that we pass the
desired log file path along when opening the log file
initially. Now the virt driver is exclusively in charge
of deciding the log filename
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
