Commit Graph

36451 Commits

Author SHA1 Message Date
Ján Tomko
cf17015fde virshtest: use virCommand instead of custom impl
Our virCommand helper API already has the ability to capture
program output, there's no need to open-code it.

Apart from simplifying the code, the test is marginally faster
due to recent improvements in virCommandMassClose.

Until now, both stderr and stdout were stored in the same buffer.
This change stores stderr separately and expects it to be empty
for all the tests we currently run.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-11 16:30:18 +01:00
Ján Tomko
8bacdde9d5 virshtest: refactor testCompareOutputLit
Use g_autofree and get rid of the cleanup label.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-11 16:30:18 +01:00
Ján Tomko
994688e0df testutils: remove unnecessary labels
The cleanups made some labels redundant.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-11 16:30:18 +01:00
Ján Tomko
f11e9abe88 testutils: use g_autoptr
Use g_autoptr where possible.

virTestCapsBuildNUMATopology is not converted completely,
because while the VIR_FREE call on cell_cpus is technically
wrong, neither VIR_ALLOC_N nor virBitmapNew can return
an allocation error now so it is effectively dead code.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-11 16:30:18 +01:00
Ján Tomko
40656ff758 testutils: use g_autofree
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-11 16:30:18 +01:00
Ján Tomko
879e9db194 testutils: check return value of g_setenv
The function returns gboolean.
Compare against the FALSE value from GLib.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Fixes: 2c33532423
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-11 16:30:18 +01:00
Ján Tomko
665c5dfb71 syntax-check: remove some exception mechanisms
Do not look for exception patterns in ${srcdir}./x-$@
nor the VC_LIST_EXCEPT_DEFAULT variable.

This also removes the default exception for ChangeLog.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
a275ce2ab7 syntax-check: remove README
This exception is no longer useful since README is just a symlink
to README.md, which is a subject to this check already.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
9d8d332dbd syntax-check: exclude: remove virstring
We no longer implement a wrapper over strdup - g_strdup
is preferred.

The use of strncpy was removed in:
commit 7d70a63b94
    util: Improve virStrncpy() implementation

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
7a86524ddb syntax-check: exclude: remove deleted files
Both xen/xend_internal and bootstrap.conf have been deleted from git.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
73c35a3083 syntax-check: drop update-NEWS-hash
NEWS was replaced by docs/news.html, so the pre-requisite for this
rule is not fulfilled.

Also, PREV_VERSION_REGEXP does not seem to be defined anywhere.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
1a97ebae1a syntax-check: drop CVS keyword expansion check
$ date +%Y
  2020

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
b491528604 syntax-check: drop vulnerable Makefile checks
As foretold, fixed automake is so common nowadays even Ubuntu 16.04
and Debian 9 have 1.11.6 as the oldest available version.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
c642199fdf syntax-check: fix sc-prohibit-cross-inclusion
Using '^' in in_vc_files assumes the build is happening
in srcdir, which is no longer possible as of:
commit f96395e78e

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:08 +01:00
Ján Tomko
04cef3b2ed syntax-check: do not enforce ChangeLog syntax
Our last conforming ChangeLog was removed from git in 2009 by:
commit 27b175b9a2
    generate ChangeLog from git logs into distribution tarball

Ten years later, it reappeared in:
commit ce97c33a79
    maint: Stop generating ChangeLog from git
which does not have a single line complying to this syntax
check rule.

At some point between the two commits VC_LIST_EXCEPT gained
a default exception for ChangeLog and it did not seem to be
overriden anywhere.

Just remove the rule.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-02-11 11:48:07 +01:00
Peter Krempa
0d0d60ddc5 tests: virstorage: Add test cases for "json:" pseudo-URI without 'file' wrapper
Add few cases that prove the second format of "json:" pseudo-URIs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
fd70f1b4d3 virStorageSourceParseBackingJSON: Prevent arbitrary nesting with format drivers
Since we parse attributes for 'raw' which is a format driver and thus
has nested 'file' structure we must prevent that this isn't nested
arbitrarily.

Add a flag for the function which allows parsing of 'format' type
drivers only on the first pass.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
f8e097570e virStorageSourceParseBackingJSON: Allow 'json:' pseudo URIs without 'file' wrapper
There are two possibilities:
1) json:{"file":{"driver":...}}
2) json:{"driver":...}

Our code didn't work properly with the second one as it was expecting
the 'file' wrapper. Conditionalize the removal to only the situation
when the top level doesn't have "driver".

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
7e13ff8dc0 virStorageSourceJSONDriverParser: annotate 'format' drivers
The parser was originally designed only for protocol parsers. Since
we already have 'raw' format driver in the list we'll need to be able
to parse it too. In later patches this will be used to prevent parsing
nested format drivers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
aadb34be34 virStorageSourceParseBackingJSON: Move deflattening of json: URIs out of recursion
Originally virStorageSourceParseBackingJSON didn't recurse, but when
the 'raw' driver support was added we need to parse it's information
which contains nested 'file' object.

Since the deflattening helper recurses already there's no need to call
it again. Move it one level up to the entry point.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
4a6bc568cd virStorageSourceParseBackingJSON: Pass around original backing file string
There are a few error messages which might want to report the original
backing store string. Pass it around rather than trying to re-generate
it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
b71cf8726c qemu: hotplug: Fix handling of the 'copy-on-read' layer with blockdev
My original implementation was completely broken because it attempted to
use object-add/del instead of blockdev-add/del.

https://bugzilla.redhat.com/show_bug.cgi?id=1798366

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
db57e9daf5 qemuMonitorBlockdevAdd: Take double pointer argument
Modify qemuMonitorBlockdevAdd so that it takes a double pointer for the
@props argument so that it's cleared inside the call. This allows
writing cleaner callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
a592d589aa qemuMonitorJSONBlockdevDel: Refactor cleanup
Use automatic variable freeing and get rid of the cleanup section.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
643294110c qemuMonitorJSONBlockdevAdd: Refactor cleanup
Use automatic variable freeing and get rid of the cleanup section.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
37f01262ee virDomainDiskTranslateSourcePool: Translate 'volume' disks in whole backing chain
Now that we accept full backing chains on input nothing should prevent
users from also using disk type 'VOLUME' for specifying the backing
images.

Do the translation for the whole backing chain.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
63469116cc virDomainDiskTranslateSourcePool: split code to setup one storage source
Extract all the code setting up one storage source from the rest which
sets up the whole disk. This will allow us to prepare the whole backing
chain.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa
bc42d88ffd virDomainDiskTranslateISCSIDirect: Take virStorageSourcePtr instead of virDomainDiskDefPtr
Only 'def->src' was ever used in this function. Use the source directly.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
4fca8299c5 virDomainDiskTranslateSourcePoolAuth: Take virStorageSourcePtr instead of virDomainDiskDefPtr
Only 'def->src' was ever used in this function. Use the source directly.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
47cad72553 virDomainDiskAddISCSIPoolSourceHost: Take virStorageSourcePtr instead of virDomainDiskDefPtr
Only 'def->src' was ever used in this function. Use the source directly.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
e20c5b1703 virDomainDiskAddISCSIPoolSourceHost: Remove ternary operator
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
4d5093ef75 virDomainDiskAddISCSIPoolSourceHost: Remove 'cleanup' label
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
5eda34f2e1 virDomainDiskAddISCSIPoolSourceHost: use g_new0 instead of VIR_ALLOC_N
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
c5b1c14379 virDomainDiskAddISCSIPoolSourceHost: Sanitize handling of string list
Use virStringSplitCount instead of virStringSplit so that we can drop
the call to virStringListLength and use VIR_AUTOSTRINGLIST to declare
it and allow removal of the cleanup section.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
4e3e69fed4 qemuBlockStorageSourceGetBackendProps: Report errors on all switch cases
Few switch cases returned failure but didn't report an error. For a
situation when the backingStore type='volume' was not translated the
following error would occur:

 $ virsh start VM
 error: Failed to start domain VM
 error: An error occurred, but the cause is unknown

After this patch:

 $ virsh start VM
 error: Failed to start domain VM
 error: internal error: storage source pool 'tmp' volume 'pull3.qcow2' is not translated

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
41345f6201 virsh: Allow extracting 'return' section of QMP command in 'qemu-monitor-command'
Simplify gathering the actual return value from a passed-through QMP
command when using 'qemu-monitor-command' by adding '--return-value'
switch which just extracts the 'return' section and alternatively
reports an error if the section is not present.

This simplifies gathering of some test data where the full reply would
need to be trimmed just for the actual return value.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-10 17:26:26 +01:00
Peter Krempa
9860d00004 docs: virsh: Modernize docs for qemu-monitor-command
Mention that we actually by default use QMP and clarify the rest
of the documentation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-10 17:26:26 +01:00
Ján Tomko
457b0e7488 qemu: preserve error on bandwidth rollback
We call APIs that reset the error in the rollback code.
Preserve the error from the original call that failed.

This turns the boringly cryptic:
  error: Unable to set interface parameters
  error: An error occurred, but the cause is unknown
to the unexpectedly anarchist:
  error: internal error: Child process (/usr/sbin/tc filter add
  dev vnet1 parent ffff: protocol all u32 match u32 0 0 police
  rate 4294968kbps burst 4294968kb mtu 64kb drop flowid :1)
  unexpected exit status 1: Illegal "rate"
  Illegal "police"

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Fixes: f02e21cb33
https://bugzilla.redhat.com/show_bug.cgi?id=1800505
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-10 16:41:52 +01:00
Ján Tomko
bd622e2a21 qemu: do not revert to NULL bandwidth
Otherwise an attempt to set an invalid value:
  virsh domiftune rhel8.2 vnet0 --outbound 4294968
on an interface with no bandwidth set crashes.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Fixes: f02e21cb33
https://bugzilla.redhat.com/show_bug.cgi?id=1800505
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-10 16:41:51 +01:00
Ján Tomko
cebb468ef5 testutils: print a helpful summary of failed tests
When debugging test failures in seven independent test
cases, it might be helpful to only gather the debug output
of the failing cases.

Record the indexes of the tests that fail and print them
in the VIR_TEST_RANGE of the command line that will result
in only those tests being run.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-09 02:11:07 +01:00
Daniel P. Berrangé
2621d48f00 gnulib: delete all gnulib integration
This deletes all trace of gnulib from libvirt. We still
have the keycodemapdb submodule to deal with. The simple
solution taken was to update it when running autogen.sh.

Previously gnulib could auto-trigger refresh when running
'make' too. We could figure out a solution for this, but
with the pending meson rewrite it isn't worth worrying
about, given how infrequently keycodemapdb changes.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:54 +00:00
Daniel P. Berrangé
55fe8110af util: delete the poll() based event loop impl
It is no longer require since switching to the GLib based
event loop impl.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:54 +00:00
Daniel P. Berrangé
946a25274c util: switch to use the GLib event loop impl
This sets the GLib event loop as the impl when calling
virEventRegisterDefaultImpl(). This remains a private
impl detail of libvirt, so applications must *NOT*
assume that a call to virEventRegisterDefaultImpl()
results in a GLib based event loop.

They should continue to use the libvirt-glib API
gvir_event_register() if they explicitly want to guarantee
a GLib event loop.

This follows the general principal that the libvirt public
API should not expose the fact that GLib is being used
internally.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:53 +00:00
Daniel P. Berrangé
f8ab47cb44 util: import an event loop impl based on GMainContext
The libvirt-glib project has provided a GMainContext based
event loop impl for applications. This imports it and sets
it up for use by libvirt as the primary event loop. This
remains a private impl detail of libvirt.

IOW, applications must *NOT* assume that a call to
"virEventRegisterDefaultImpl" results in a GLib based
event loop. They should continue to use the libvirt-glib
API gvir_event_register() if they explicitly want to
guarantee a GLib event loop.

This follows the general principle that the libvirt public
API should not expose the fact that GLib is being used
internally.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:53 +00:00
Daniel P. Berrangé
7d4350bcac rpc: convert RPC client to use GMainLoop instead of poll
To eliminate the dependancy on GNULIB's poll impl, we need
to change the RPC client code to use GMainLoop. We don't
really want to use GIOChannel, but it provides the most
convenient way to do socket event watches with Windows
portability. The other alternative would be to use GSocket
but that is a much more complex change affecting libvirt
more broadly.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:53 +00:00
Daniel P. Berrangé
5de317b8e7 src: introduce helper API for creating GSource for socket
We need to be able to create event loop watches using the
GSource API for sockets. GIOChannel is able todo this, but
we don't want to use the GIOChannel APIs for reading/writing,
and testing shows just using its GSource APIs is unreliable
on Windows.

This patch thus creates a standalone helper API for creating
a GSource for a socket file descriptor. This impl is derived
from code in QEMU's io/channel-watch.c file that was written
by myself & Paolo Bonzini & thus under Red Hat copyright.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:53 +00:00
Daniel P. Berrangé
dc0771cfa2 tools: rewrite interactive job monitoring logic
For long running jobs (save, managed save, dump & live migrate)
virsh runs a background thread for executing the job and then
has the main thread catch Ctrl-C for graceful shutdown, as well
as displaying progress info.

The monitoring code is written using poll, with a pipe used
to get the completion status from the thread. Using a pipe
and poll is problematic for Windows portability. This rewrites
the code to use a GMainLoop instance for monitoring stdin and
doing progress updates. The use of a pipe is entirely eliminated,
instead there is just a shared variable between both threads
containing the job completion status.

No mutex locking is used because the background thread writes
to the variable only when the main loop is still running,
while the foreground thread only reads it after the main loop
has exited.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:53 +00:00
Daniel P. Berrangé
29c4a3c753 tests: avoid referencing stale readdir pointer
The contents of 'struct dirent' are only valid until the next call to
readdir() or closedir(). It is thus invalid to save a pointer to the
'd_name' field. Somehow this hasn't affected the test suite until
recently when FreeBSD 12 started showing use of uninitialized memory
resulting in test failures.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 14:57:59 +00:00
zhenwei pi
26badd13e8 qemu: support Panic Crashloaded event handling
Pvpanic device supports bit 1 as crashloaded event, it means that
guest actually panicked and run kexec to handle error by guest side.

Handle crashloaded as a lifecyle event in libvirt.

Test case:
Guest side:
before testing, we need make sure kdump is enabled,
1, build new pvpanic driver (with commit from upstream
   e0b9a42735f2672ca2764cfbea6e55a81098d5ba
   191941692a3d1b6a9614502b279be062926b70f5)
2, insmod new kmod
3, enable crash_kexec_post_notifiers,
  # echo 1 > /sys/module/kernel/parameters/crash_kexec_post_notifiers
4, trigger kernel panic
  # echo 1 > /proc/sys/kernel/sysrq
  # echo c > /proc/sysrq-trigger

Host side:
1, build new qemu with pvpanic patches (with commit from upstream
   600d7b47e8f5085919fd1d1157f25950ea8dbc11
   7dc58deea79a343ac3adc5cadb97215086054c86)
2, build libvirt with this patch
3, handle lifecycle event and trigger guest side panic
  # virsh event stretch --event lifecycle
  event 'lifecycle' for domain stretch: Crashed Crashloaded
  events received: 1

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-02-07 14:05:25 +00:00
Daniel P. Berrangé
039787c71a docs: render class="literal" with monospace font
When using ``....`` in RST, this results in <span class="literal">...</span>
instead of <code>...</code>. We thus need an extra rule to render it
with a monospace font. Colouring a light gray also helps the text
stand out a little more and matches background of <pre> blocks.

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 10:18:16 +00:00