Commit Graph

44 Commits

Author SHA1 Message Date
Erik Skultety
b988f794e3 daemon: logging: Fix --verbose option being ignored by the daemon
Commit 94c465d0 refactored the logging setup phase but introduced an
issue, where the daemon ignores verbose mode when there are no outputs
defined and the default must be used. The problem is that the default
output was determined too early, thus ignoring the potential '--verbose'
option taking effect. This patch postpones the creation of the default
output to the very last moment when nothing else can change. Since the
default output is only created during the init phase, it's safe to leave
the pointer as NULL for a while, but it will be set eventually, thus not
affecting runtime.
Patch also adjusts both the other daemons.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1442947

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2017-08-28 16:42:13 +02:00
Lily Zhu
0f0c0921e6 man: Fix documentation errors about the paths of the config files
The default conf files, for example libvirtd.conf, virtlockd.conf, and
virtlogd.conf, should be located under the directory "/etc/libvirt" when
root as root, rather than "/etc". When run as non-root, the configuration
files should be located under "$XDG_CONFIG_HOME/libvirt/", rather than
"XDG_CONFIG_HOME".

Signed-off-by: Lily Zhu <lizhu@redhat.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
2017-06-28 11:07:08 +02:00
Peter Krempa
f0803dae93 (log|lock)daemon: Don't spam logs with IO error messages after client disconnects
The log and lock protocol don't have an extra handshake to close the
connection. Instead they just close the socket. Unfortunately that
resulted into a lot of spurious garbage logged to the system log files:

2017-03-17 14:00:09.730+0000: 4714: error : virNetSocketReadWire:1800 : End of file while reading data: Input/output error

or in the journal as:

Mar 13 16:19:33 xxxx virtlogd[32360]: End of file while reading data: Input/output error

Use the new facility in the netserverclient to suppress the IO error
report from the virNetSocket layer.
2017-03-17 17:20:22 +01:00
Daniel P. Berrange
27cd763500 Increase default file handle limits for daemons
Linux still defaults to a 1024 open file handle limit. This causes
scalability problems for libvirtd / virtlockd / virtlogd on large
hosts which might want > 1024 guest to be running. In fact if each
guest needs > 1 FD, we can't even get to 500 guests. This is not
good enough when we see machines with 100's of physical cores and
TBs of RAM.

In comparison to other memory requirements of libvirtd & related
daemons, the resource usage associated with open file handles
is essentially line noise. It is thus reasonable to increase the
limits unconditionally for all installs.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-03-16 10:48:09 +00:00
Erik Skultety
94c465d0eb daemon: Hook up the virLog{Get,Set}DefaultOutput to the daemon's init routine
Now that virLog{Get,Set}DefaultOutput routines are introduced we can wire them
up to the daemon's logging initialization code. Also, change the order of
operations a bit so that we still strictly honor our precedence of settings:
cmdline > env > config now that outputs and filters are not appended anymore.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-12-15 10:36:23 +01:00
Erik Skultety
0d6cf32721 admin: Allow passing NULL to virLogSetOutputs
Along with an empty string, it should also be possible for users to pass
NULL to the public APIs which in turn would trigger a routine(future
work) responsible for defining an appropriate default logging output
given the current circumstances.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-12-15 10:36:23 +01:00
Peter Krempa
15398e6a4c log: Fix loading of conf file for log daemon
'log_outputs' would be read into the variable for log_filters
2016-12-14 07:24:24 +01:00
Erik Skultety
30b650b2ba daemon: Split filter parsing and filter defining
Similar to outputs, parser should do parsing only, thus the 'define' logic
is going to be stripped from virLogParseAndDefineFilters by replacing calls to
this method to virLogSetFilters instead.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-10-10 08:27:25 +02:00
Erik Skultety
c9279169a1 daemon: Split output parsing and output defining
Since virLogParseAndDefineOutputs is going to be stripped from 'output defining'
logic, replace all relevant occurrences with virLogSetOutputs call to make the
change transparent to all original callers (daemons mostly).

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-10-10 08:27:25 +02:00
Erik Skultety
b8c370a96e virlog: Rename virLogParse* to virLogParseAndDefine*
Right now virLogParse* functions are doing both parsing and defining of filters
and outputs which should be two separate operations. Since the naming is
apparently a bit poor this patch renames these functions to
virLogParseAndDefine* which eventually will be replaced by virLogSet*.
Additionally, virLogParse{Filter,Output} will be later (after the split) reused,
so that these functions do exactly what the their name suggests.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-10-10 08:27:24 +02:00
Andrea Bolognani
f496ce1df3 virtlogd: Don't stop or restart along with libvirtd
Commit 839a060 tied the lifecycle of virtlogd more
closely to that of libvirtd. Unfortunately, while starting
virtlogd when libvirtd is started is definitely a good idea,
restarting virtlogd or shutting it down at any time outside
of system poweroff is not.

Revert part of that commit by removing the PartOf= lines,
meaning that only startup requests will be propagated from
libvirtd to virtlogd.

Resolves: https://bugzilla.redhat.com/1372576
2016-09-20 15:22:04 +02:00
Andrea Bolognani
839a060890 virtlogd.socket: Tie lifecycle to libvirtd.service
We already guarantee that virtlogd.socket is enabled/disabled
along with libvirtd.service, but if libvirtd.service has just
been installed and is started before rebooting, then
virtlogd.socket will not be running and guest startup will
fail.

Add Requires=virtlogd.socket to libvirtd.service to make sure
virtlogd.socket is always started along with libvirtd.service,
and add Before=libvirtd.service to both virtlogd.socket and
virtlogd.service so that virtlogd never disappears before
libvirtd has exited.

Also add PartOf=libvirtd.service to both virtlogd.socket and
virtlogd.service, so that virtlogd can be shut down when not
needed.

Resolves: https://bugzilla.redhat.com/1372576
2016-09-06 16:05:20 +02:00
Daniel P. Berrange
46dbc97e65 virtlogd: convert to typedef virConf accessors
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-07-12 09:58:21 +01:00
Daniel P. Berrange
8f390596be virtlogd: increase max file size to 2 MB
People debugging guest OS boot processes and reported that
the default 128 KB size is too small to capture an entire
boot up sequence. Increase the default size to 2 MB which
should allow capturing a full boot up even with verbose
debugging.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-07-07 15:08:54 +01:00
Daniel P. Berrange
24aacfa8e8 virtlogd: make max file size & number of backups configurable
Currently virtlogd has a hardcoded max file size of 128kb
and max of 3 backups. This adds two new config parameters
to /etc/libvirt/virtlogd.conf to let these be customized.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-07-07 15:08:54 +01:00
Jaroslav Suchanek
c59b9e1483 logging: fixing log level initialization from cmdline
Reorder code for setting default log level from cmdline prior
initialization of log outputs. Thus the --verbose option is reflected.

This resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1325072
2016-06-28 13:52:34 +02:00
Peter Krempa
78b9b85c06 log: daemon: Add remote protocol handling for the log appending API
Implement the RPC dispatcher and caller for the new API.
2016-06-07 18:10:29 +02:00
Peter Krempa
5e6143fbcc log: handler: Add new API to append to logging files
For logging one-shot entries to the VM log file it's quite a waste to
hold open the file descriptor for logging that is provided by the
current API.

This new API will be ideal for logging one-shot entries to the file
e.g. at the point when we shut the VM down rather than having to add the
whole file-descriptor infrastructure.

Additionally this will allow to add the messages even after restart of
libvirtd since virtlogd doesn't allow to obtain a regular context with
filedescriptors while the VM is still active.
2016-06-07 18:10:29 +02:00
Jovanka Gulicoska
b29e08dbe3 More usage of virGetLastErrorMessage
Convert to virGetLastErrorMessage() in the rest of the code
2016-05-19 15:17:03 -04:00
Erik Skultety
5841d64d25 rpc: virnetserverclient: Identify clients by an integer ID
Admin API needs a way of addressing specific clients. Unlike servers, which we
are happy to address by names both because its name reflects its purpose (to
some extent) and we only have two of them (so far), naming clients doesn't make
any sense, since a) each client is an anonymous, i.e. not recognized after a
disconnect followed by a reconnect, b) we can't predict what kind of requests
it's going to send to daemon, and c) the are loads of them comming and going,
so the only viable option is to use an ID which is of a reasonably wide data
type.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-02 22:25:51 +02:00
Andrea Bolognani
92b2c047d6 man: Fix SYNOPSIS section
Format the text properly.
2016-04-25 15:40:44 +02:00
Andrea Bolognani
5b479f3b5b build: Standardize on .pod -> .x.in -> .x
After this commit, all man pages are generated using the same two
steps:

  1. Process a source $command.pod file with pod2man(1) to obtain
     a valid man page in $command.$section.in

  2. Process $command.$section.in with sed(1) to obtain the final
     man page in $command.$section
2016-04-25 15:40:44 +02:00
Cole Robinson
e7db227810 util: Add virGettextInitialize, convert the code
Take setlocale/gettext error handling pattern from tools/virsh-*
and use it for all standalone binaries via a new shared
virGettextInitialize routine. The virsh* pattern differed slightly
from other callers. All users now consistently:

* Ignore setlocale errors. virsh has done this forever, presumably for
  good reason. This has been partially responsible for some bug reports:

  https://bugzilla.redhat.com/show_bug.cgi?id=1312688
  https://bugzilla.redhat.com/show_bug.cgi?id=1026514
  https://bugzilla.redhat.com/show_bug.cgi?id=1016158

* Report the failed function name
* Report strerror
2016-04-14 13:22:40 -04:00
Cole Robinson
cdced0128a virtlogd: Fix a couple minor memory leaks
https://bugzilla.redhat.com/show_bug.cgi?id=1303885
2016-04-11 07:32:57 -04:00
Martin Kletzander
fd4f278e1a daemon: Get server name from the server itself
Since servers know their name, there is no need to supply such
information twice.  Also defeats inconsistencies.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
dad3b07814 server: Store server name in server object
At first I did not want to do this, but after trying to implement some
newer feaures in the admin API I realized we need that to make our lives
easier.  On the other hand they are not saved redundantly and the
virNetServer objects are still kept in a hash table.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Daniel P. Berrange
3f68f33c36 logging: support truncation of logfiles when opening
The virtlogd daemon currently opens all files for append, but
in some cases the user may wish to discard existing data. Define
a new flag to indicate that logfiles should be truncated when
opening.
2016-03-10 15:41:52 +00:00
Daniel P. Berrange
f12bfc5ed5 logging: allow inode/offset params to be NULL
Not all callers of virLogManagerDomainOpenLogFile will
care about getting the current inode/offset, so we should
allow those parameters to be NULL

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-03-10 15:33:16 +00:00
Erik Skultety
252610f7dd virnetdaemon: Store servers in a hash table
Since the daemon can manage and add (at fresh start) multiple servers,
we also should be able to add them from a JSON state file in case of a
daemon restart, so post exec restart support for multiple servers is also
provided. Patch also updates virnetdaemontest accordingly.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-02-17 12:46:34 +01:00
Michal Privoznik
c03fbecc7c virLogManagerDomainReadLogFile: Don't do dummy allocs
Since we pass dummy variables @fdout and @fdoutlen into
virNetClientProgramCall() we make it alloc @fdout array (even
though it's an array of 0 elements since vitlogd can hardly pass
us some FDs at this stage). Nevertheless, it's an allocation not
followed by free():

==29385== 0 bytes in 60 blocks are definitely lost in loss record 2 of 1,009
==29385==    at 0x4C2C070: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==29385==    by 0x54B99EF: virAllocN (viralloc.c:191)
==29385==    by 0x56821B1: virNetClientProgramCall (virnetclientprogram.c:359)
==29385==    by 0x563B304: virLogManagerDomainReadLogFile (log_manager.c:272)
==29385==    by 0x217CD613: qemuDomainLogContextRead (qemu_domain.c:2485)
==29385==    by 0x217EDC76: qemuProcessReadLog (qemu_process.c:1660)
==29385==    by 0x217EDE1D: qemuProcessReportLogError (qemu_process.c:1696)
==29385==    by 0x217EE8C1: qemuProcessWaitForMonitor (qemu_process.c:1957)
==29385==    by 0x217F6636: qemuProcessLaunch (qemu_process.c:4955)
==29385==    by 0x217F71A4: qemuProcessStart (qemu_process.c:5152)
==29385==    by 0x21846582: qemuDomainObjStart (qemu_driver.c:7396)
==29385==    by 0x218467DE: qemuDomainCreateWithFlags (qemu_driver.c:7450)

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-01-18 17:14:16 +01:00
Roman Bogorodskiy
bc451c4980 Add missing virxdrdefs.h include to log_protocol
Commit 2b6f6ad introduced the virxdrdefs.h header with
common definitions to be included in the protocol files,
but logging/log_protocol.x was missed, so add it there as well.

Hopefully this fixes build on OS X.
2016-01-12 18:15:09 +03:00
Martin Kletzander
6dc0e4f171 Fix LSB requirements in service script and sync them
Commit b22344f328 mistakenly reordered
Default-* lines.  Thanks to that I noticed that we are very inconsistent
with our init scripts, so I took the liberty of synchronizing them,
updating them and making them all look shiny and new.  So apart from
fixing the LSB requirements, I also fixed the ordering, specified
runlevels and fix the link to the reference specification.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-01-11 15:49:13 +01:00
Olaf Hering
b22344f328 fix LSB part of virtlogd runlevel script
Currently pkg build of master branch fails:

[  300s] + /usr/lib/rpm/brp-boot-scripts
[  300s] E: File `virtlogd' is missing `Required-Start', please add even if empty!
[  300s] W: File `virtlogd' is missing `Required-Stop', please add even if empty!
[  300s] E: File `virtlogd' has empty `Default-Start', please specify default runlevel(s)!
[  300s] ERROR: found one or more broken init or boot scripts, please fix them.
[  300s]        For more information about LSB headers please read the manual
[  300s]        page of of insserv by executing the command `man 8 insserv'.
[  300s]        If you don't understand this, mailto=werner@suse.de
[  300s] error: Bad exit status from /var/tmp/rpm-tmp.44965 (%install)

Add the required tags, fix the existing tags.
Use soft dependency "Should-Start" because virtlogd may work without network.

Signed-off-by: Olaf Hering <olaf@aepfle.de>
2016-01-06 10:59:53 +01:00
Michael Chapman
44b4a4f6d7 virtlogd: fix lock file path in initscript
The virtlogd initscript's lock file should go in /var/lock/subsys/, not
(the nonexistent) /var/log/subsys/.

Signed-off-by: Michael Chapman <mike@very.puzzling.org>
2016-01-05 18:48:50 -05:00
Daniel P. Berrange
b6cbabc551 logging: validate flags passed from client in virtlogd
The virtlogd RPC messages all have a flags parameter. For
sake of future error reporting we should be verifying
these are all 0 for now.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-07 17:30:07 +00:00
Daniel P. Berrange
50896b2804 logging: change log protocol to be more reusable
The current virtlogd RPC protocol provides the ability to
handle log files associated with QEMU stdout/err. The log
protocol messages take the virt driver, domain name and
use that to form a log file path. This is quite restrictive
as it prevents us re-using the same RPC protocol messages
for logging to char device backends where the filename
can be arbitrarily user specified. It is also bad because
it means we have 2 separate locations which have to decide
on logfile name.

This change alters the RPC protocol so that we pass the
desired log file path along when opening the log file
initially. Now the virt driver is exclusively in charge
of deciding the log filename

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-07 17:30:07 +00:00
Daniel P. Berrange
d4abb09d80 logging: preserve driver, dom name & uuid against log file
The virt driver, dom name and uuid associated with a log
file are important pieces of metadata to keep around for
sake of future enhancements to virtlogd. Currently we
discard them after opening the log file, but we should
preserve them, even across restarts.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-12-07 17:30:07 +00:00
Michal Privoznik
bd205a9036 log_manager: Include configmake.h last
The problem is that in some mingw header DATADIR is used but
gnulib defines it too. This leads to the following compile error:

  CC       locking/libvirt_driver_la-lock_manager.lo
In file included from /usr/i686-w64-mingw32/sys-root/mingw/include/objbase.h:66:0,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/ole2.h:17,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/wtypes.h:12,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/winscard.h:10,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/windows.h:97,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/winsock2.h:23,
                 from ../gnulib/lib/unistd.h:48,
                 from ../../src/util/virutil.h:29,
                 from ../../src/logging/log_manager.c:30:
/usr/i686-w64-mingw32/sys-root/mingw/include/objidl.h:12275:2: error: expected identifier or '(' before string constant
 } DATADIR;
  ^
Makefile:7888: recipe for target 'logging/libvirt_driver_la-log_manager.lo' failed

The fix is to include configmake.h at the end of includes.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2015-12-01 16:43:56 +01:00
Daniel P. Berrange
753346c5dd logging: avoid variables called 'daemon' due to function clash
With some versions of GLibC / GCC, a variable called 'daemon'
will result in a warning about clashing with the function also
named 'daemon'. Rename it to 'dmn' to avoid the clash.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-11-27 09:27:53 +00:00
Guido Günther
35b2a2d99a virtlogd: use %llu to print 64bit types
Otherwise we fail on 32bit with:

CC     logging/virtlogd-log_daemon_dispatch.o
logging/log_daemon_dispatch.c: In function 'virLogManagerProtocolDispatchDomainReadLogFile':
logging/log_daemon_dispatch.c:120:9: error: format '%zu' expects argument of type 'size_t', but argument 7 has type 'uint64_t' [-Werror=format]
2015-11-26 19:37:37 +01:00
Daniel P. Berrange
df34363d58 logging: inhibit virtlogd shutdown while log files are open
The virtlogd daemon is launched with a 30 second timeout for
unprivileged users. Unfortunately the timeout is only inhibited
while RPC clients are connected, and they only connect for a
short while to open the log file descriptor. We need to hold
an inhibition for as long as the log file descriptor itself
is open.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-11-26 14:30:16 +00:00
Daniel P. Berrange
37ed422404 logging: add client for virtlogd daemon
Add the virLogManager API which allows for communication with
the virtlogd daemon to RPC program. This provides the client
side API to open log files for guest domains.

The virtlogd daemon is setup to auto-spawn on first use when
running unprivileged. For privileged usage, systemd socket
activation is used instead.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-11-26 14:30:13 +00:00
Daniel P. Berrange
19e5db4ae2 logging: introduce log handling protocol
Define a new RPC protocol for the virtlogd daemon that provides
for handling of logs. The initial RPC method defined allows a
client to obtain a file handle to use for writing to a log
file for a guest domain. The file handle passed back will not
actually refer to the log file, but rather an anonymous pipe.
The virtlogd daemon will forward I/O between them, ensuring
file rotation happens when required.

Initially the log setup is hardcoded to cap log files at
128 KB, and keep 3 backups when rolling over, which gives
a max usage of 512 KB per guest.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-11-26 14:28:55 +00:00
Daniel P. Berrange
323a329b26 Import stripped down virtlockd code as basis of virtlogd
Copy the virtlockd codebase across to form the initial virlogd
code. Simple search & replace of s/lock/log/ and gut the remote
protocol & dispatcher. This gives us a daemon that starts up
and listens for connections, but does nothing with them.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2015-11-26 14:28:55 +00:00