Let's generate prealloc-threads property onto the cmd line if
domain configuration requests so.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Only fairly new QEMUs are capable of user provided number of
preallocation threads. Validate this assumption.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
The prealloc-threads is property of memory-backend class which is
parent to the other three classes memory-backend-{ram,file,memfd}.
Therefore the property is present for all, or none if QEMU is
older than v5.0.0-rc0~75^2~1^2~3 which introduced the property.
Anyway, the .reserve property is the same story, and we chose
memory-backend-file to detect it, so stick with our earlier
decision and use the same backend to detect this new property.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Since its v5.0.0 release QEMU is capable of specifying number of
threads used to allocate memory. It defaults to 1, which may be
too low for humongous guests with gigantic pages.
In general, on QEMU cmd line level it is possible to use
different number of threads per each memory-backend-* object, in
practical terms it's not useful. Therefore, use <memoryBacking/>
to set guest wide value and let all memory devices 'inherit' it,
silently. IOW, don't introduce per device knob because that would
only complicate things for a little or no benefit.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
This brings in all the fixes made since April 2020.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
By default, stdout/stderr Avocado test log files do not have any file
extension which confuses GitLab's web UI to mangle the MIME type for
these and so the browser will never offer the option to open such file
from in a text editor rather than dowloading it.
Since GitLab sets a proper MIME for .txt and .log file extensions,
rename all Avocado log files without an extension to *.log . This pairs
nicely with the coredumpctl info file which we already name as
'coredumpctl.txt' because of this.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Some Red Hat-like distros have cores limited with a soft limit of 0
which means that neither a stack trace nor a core file will be
available. Since we want the stack trace we need to set the core limit
with systemd globally to unlimited/infinity.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Custom runners are private to a project, so naturally forks cannot run
any workloads on these. The integration test suite which requires
access to our custom runner is naturally disabled on forks and can be
enabled by setting LIBVIRT_CI_INTEGRATION=1.
The problem is that the current integration jobs definitions have tags
statically defined as 'redhat-vm-host'. If users are going to supply
their own private runners for their forks, they can define whatever
tags they want with it and so unless they add 'redhat-vm-host' to their
own runner's tags, the pipeline won't run.
To solve this, define the integration job tag using a variable. The
repo config will use the value defined in the job for the variable
while users can override the value easily on a project/pipeline level
thanks to GitLab's CI variable precedence [1].
[1] https://docs.gitlab.com/ee/ci/variables/#cicd-variable-precedence
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The updateLock is a R/W lock held by anything which needs to read or
modify the rules associated with an NWFilter.
APIs for defining/undefining NW filters rules hold a write lock on
updateLock.
APIs for creating/deleting NW filter bindings hold a read lock on
updateLock, which prevents define/undefine taking place concurrently.
The problems arise when we attempt to creating two NW filter bindings in
parallel.
Thread 1 can acquire the mutex for filter A
Thread 2 can acquire the mutex for filter B
Consider if filters A and B both reference filters C and D, but in
different orders:
Filter A
-> filter C
-> filter D
Filter B
-> filter D
-> filter C
Thread 1 will try to acquire locks in order A, C, D while thread 1 will
try to acquire in order A, D, C. Deadlock can still occur.
Think we can sort the list of filters before acquiring locks on all of
them ? Nope, we allow arbitrary recursion:
Filter A
-> filter C
-> filter E
-> filter F
-> filter H
-> filter K
-> filter D
-> filter G
-> filter I
So we can't tell from looking at 'A' which filters we're going to
need to lock. We can only see the first level of filters references
and we need to lock those before we can see the second level of
filters, etc.
We could probably come up with some cleverness to address this but
it isn't worth the time investment. It is simpler to just keep the
process of creating NW filter bindings totally serialized.
Using two separate locks for this serialization though is pointless.
Every code path which gets a read(updateLock) will go on to hold
updateMutex. It is simpler to just hold write(updateLock) and
get rid of updateMutex. At that point we don't need updateLock
to be a R/W lock, it can be a plain mutex.
Thus this patch gets rid of the current updateLock and updateMutex
and introduces a new top level updateMutex.
This has a secondary benefit of introducing fairness into the
locking. With a POSIX R/W lock, you get writer starvation if
you have lots of readers. IOW, if we call virNWFilterBIndingCreate
and virNWFilterBindingDelete in a tight loop from a couple of
threads, we can prevent virNWFilterDefine from ever acquiring
a write lock.
Getting rid of the R/W lock gives us FIFO lock acquisition
preventing starvation of any API call servicing.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
schemas are used for more than just documentation,
virsh edit fails if schemas are not available.
Therefore, fix the no-docs build by moving schemas/
to the parsing code inside src/conf/.
Signed-off-by: Claudio Fontana <cfontana@suse.de>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
In case we are snapshotting at least one 'manual' disk we will pause the
VM and keep it paused.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1866400
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The idea of the manual mode is to allow a synchronized snapshot in cases
when the storage is outsourced to an unmanaged storage provider which
requires cooperation with snapshotting.
The mode will instruct the hypervisor to pause along when the other
components are snapshotted and the 'manual' disk can be snapshotted
along. This increases latency of the snapshot but allows them in
otherwise impossible situations.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The code parsing thue query-cpu-definitions response will short-circuit
the for loop in the case where usable=yes, resulting in us failing to
parse the CPU deprecation flag.
IOW, we only reported deprecations in domain capabilities for CPU models
which were not runnable on the host.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
In the not so distant past, the lock ordering in
virNWFilterLockIface() was as follows: global mutex ifaceMapLock
was acquired, then internal representation of given interface was
looked up in a hash table (or created brand new if none was
found), the global lock was released and the lock of the
interface was acquired.
But this was mistakenly changed as the function was rewritten to
use automatic mutexes, because now the global lock is held
throughout the whole run of the function and thus the interface
specific lock is acquired with the global lock held. This results
in a deadlock.
Fixes: dd8150c48d
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Tested-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The qemuProcessQMPStop() function is intended to kill this dummy
QEMU process we started only for querying capabilities.
Nevertheless, it may be not plain QEMU binary we executed, but
in fact it may be a memcheck tool (e.g. valgrind) that executes
QEMU later. By switching to virProcessKillPainfully() we allow
this wrapper tool to exit gracefully.
Another up side is that virProcessKillPainfully() reports an
error so no need for us to VIR_ERROR() ourselves.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Right now the jobs have no rules so they will always be created in
a pipeline. If the user's fork has no runner configured, then the
jobs will never be able to execute and the pipeline will not finish.
Even on upstream, there might be times the runner has to be taken
offline for maint work, or unexpectedly fail. We need a quick way
to disable the integration tests if we decide we don't want to
have pipelines queued until the runner comes back online.
Both these problems can be addressed by requiring a environment
variable to be set
LIBVIRT_CI_INTEGRATION=1
This can be done in the GitLab repo CI settings for permanent
enablement. Alternatively it can be set for individual
scheduled jobs, or using a push option
git push -o ci.variable=LIBVIRT_CI_INTEGRATION=1
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Although we split out jobs across many files, the template / job
namespace is global, so we should use something more specific
than '.tests' as the template name.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
While its name would suggest that virNWFilterObj is an actual
virObject it is not. It's a plain structure (with virMutex as its
first member). Therefore, when locking the struct
virObjectLockGuard() can' be used and virLockGuardLock() must be
used instead.
Spotted-by: Martin Kletzander <mkletzan@redhat.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
It does not make sense to have both of these, since one of them
is only a wrapper for the other one. I decided to preserve the
more general one, which requires only virDomainObj and rewrote it
a bit, so that it pulls the qemu driver from privateData.
Signed-off-by: Kristina Hanicova <khanicov@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
This allows nwfilterStateCleanupLocked to be used in
nwfilterStateInitialize in a later patch.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This enables a later patch to simplify locking during initialization
and cleanup of virNWFilterDriverState.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
virHashTableForEach unhelpfully has payload/key args in
its callback reversed compared to g_hash_table_foreach.
When converting from one to the other the semantics
change but you don't get a compile error
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
A wrong reordering caused "priv" to be derefenced before the NULL-check
in esxStreamSend and esxStreamRecvFlags.
Fixes: 12e19f172d
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This stage will download build artifacts from both the libvirt and
libvirt-perl (multi-project CI) builds, install all them on the custom
runners and configures libvirt debug logging on the runners prior to
executing the actual test suite. In case of a failure, libvirt and
Avocado logs will be saved and published as pipeline artifacts.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We're already building libvirt in the containers already, if we publish
the build in form of, say, RPMs, later stages of the pipeline can
consume the RPMs instead of re-building libvirt from scratch.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This makes it possible to reduce the number of cases we have to
consider, because 'sles' declares itself to be like 'suse' and
both 'rhel' and 'centos' declare themselves to be like 'fedora'.
We have to move the check for Ubuntu before the one for Debian,
however, because 'ubuntu' declares itself to be like 'debian'
and it would end up with the wrong defaults otherwise.
Suggested-by: Olaf Hering <olaf@aepfle.de>
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The documentation included in these manual pages is mostly useful
to users of the 'send-key' virsh command, and the virsh manual
page refers to them, so it makes more sense to install them along
with virsh instead of libvirtd.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
I introduced support for these vim plugins several years ago
but have since moved away from them. These days developers
are likely better served by lsp-based tooling, which doesn't
require additional per-project configuration.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The files marked as export-ignore here are not going to be
included in the tarball produced by 'meson dist' when using
meson >= 0.60.
Older versions of meson excluded a small subset of these files
automatically, but since we have more control now we can be
more aggressive and leave out anything that doesn't make sense
in a release tarball.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The current use of an array for nwfilter objects requires
the caller to iterate over all elements to find a filter,
and also requires locking each filter.
Switching to a pair of hash tables enables O(1) lookups
both by name and uuid, with no locking required.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The comment against the 'updateMutex' refers to a problem with
lock ordering when looking up filters in the virNWFilterObjList
which uses an array. That problem does indeed exist.
Unfortunately it claims that switching to a hash table would
solve the lock ordering problems during instantiation. That
is not correct because there is a second lock ordering
problem related to how we traverse related filters when
instantiating filters. Consider a set of filters:
Filter A:
Reference Filter C
Reference Filter D
Filter B:
Reference Filter D
Reference Filter C
In one example, we lock A, C, D, in the other example
we lock A, D, C.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The virNWFilterObjListNumOfNWFilters method iterates over the
driver->nwfilters, accessing virNWFilterObj instances. As such
it needs to be protected against concurrent modification of
the driver->nwfilters object.
This API allows unprivileged users to connect, so users with
read-only access to libvirt can cause a denial of service
crash if they are able to race with a call of virNWFilterUndefine.
Since network filters are usually statically defined, this is
considered a low severity problem.
This is assigned CVE-2022-0897.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
When nodedev objects are added and removed if possible check if mdev-types is
supported by the object and trigger a mdev device definition update to correct
the associated parent nodedevs.
Signed-off-by: Boris Fiuczynski <fiuczy@linux.ibm.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Jonathon Jongsma <jjongsma@redhat.com>
