Currently when we build with nbdkit support, libvirt will always try to
use nbdkit to access remote disk sources when it is available. But
without an up-to-date selinux policy allowing this, it will fail.
because the required selinux policies are not yet widely available, we
have disabled nbdkit support on rpm builds for all distributions before
Fedora 40.
Unfortunately, this makes it more difficult to test nbdkit support.
After someone updates to the necessary selinux policies, they would also
need to rebuild libvirt to enable nbdkit support. By introducing a
configure option (nbdkit_config_default), we can build packages with
nbdkit support but have it disabled by default.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Suggested-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
We only use it at runtime, not during the build process.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
We only use it at runtime, not during the build process.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
We only use it at runtime, not during the build process.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Xcode 15, which provides the compiler toolchain for building libvirt
on macOS has switched to a new linker that warns about duplicated
"-lblah" options on the ld commandline. In practice this is impossible
to prevent in a large project, and also harmless.
Fortunately the new ld command also has an option,
-no_warn_duplicate_libraries, that supresses this harmless/pointless
warning, meson has a simple way to check if that option is supported,
and libvirt's meson.build files already have examples of adding an
option to the ld commandline if it's available.
Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This replaces use of 'rpcgen' with our new python impl of
the RPC code generator. Since the new impl generates code
that matches our style/coding rules, and does not contain
long standing bugs, we no longer need to post-process the
output.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This adds a lexer capable of handling the XDR protocol files.
The lexical rquirements are detailed in
https://www.rfc-editor.org/rfc/rfc4506#section-6.2
pytest is introduced as a build dependancy for testing python
code.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The 'black' tool is intended to be an opinionated formatting
tool for python code. It is complementary to flake8 which
validates coding bad practices, but (mostly) ignores code
layout issues.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Given that this variable now controls not just whether C tests
are built, but also whether any test at all is executed, the new
name is more appropriate.
Update the description for the corresponding meson option
accordingly.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Currently, passing -Dtests=disabled only disables a subset of
tests: those that are written in C and thus require compilation.
Other tests, such as the syntax-check ones and those that are
implemented as scripts, are always enabled.
There's a potentially dangerous consequence of this behavior:
when tests are disabled, 'meson test' will succeed as if they
had been enabled. No indication of this will be shown, so the
user will likely make the reasonable assumption that everything
is fine when in fact the significantly reduced coverage might
be hiding failures.
To solve this issues, disable *all* tests when asked to do so,
and inject an intentionally failing test to ensure that 'meson
test' doesn't succeed.
Best viewed with 'git show -w'.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
It only makes sense to enable expensive tests when tests are
enabled. Disallow invalid configurations.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
There are some cases in which we automatically disable tests when
using Clang as the compiler. If the user has explicitly asked for
tests to be enabled, however, we should error out instead of
silently disabling things.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
This will make future patches nicer.
Note that we need to handle these somewhat late because of the
dependency on information about the compiler and the flags it
supports.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
The situation is the same as Linux: since glibc no
longer includes the RPC functionality, libtirpc must
be used to complement it.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Currently, nbdkit support will automatically be enabled as long as
the pidfd_open(2) syscall is available. Optionally, libnbd is used
to generate more user-friendly error messages.
In theory this is all good, since use of nbdkit is supposed to be
transparent to the user. In practice, however, there is a problem:
if support for it is enabled at build time and the necessary
runtime components are installed, nbdkit will always be preferred,
with no way for the user to opt out.
This will arguably be fine in the long run, but right now none of
the platforms that we target ships with a SELinux policy that
allows libvirt to launch nbdkit, and the AppArmor policy that we
maintain ourselves hasn't been updated either.
So, in practice, as of today having nbdkit installed on the host
makes network disks completely unusable unless you're willing to
compromise the overall security of the system by disabling
SELinux/AppArmor.
In order to make the transition smoother, provide a convenient
way for users and distro packagers to disable nbdkit support at
compile time until SELinux and AppArmor are ready.
In the process, detection is completely overhauled. libnbd is
made mandatory when nbdkit support is enabled, since availability
across operating systems is comparable and offering users the
option to make error messages worse doesn't make a lot of sense;
we also make sure that an explicit request from the user to
enable/disable nbdkit support is either complied with, or results
in a build failure when that's not possible. Last but not least,
we avoid linking against libnbd when nbdkit support is disabled.
At the RPM level, we disable the feature when building against
anything older than Fedora 40, which still doesn't have the
necessary SELinux bits but will hopefully gain them by the time
it's released. We also allow nbdkit support to be disabled at
build time the same way as other optional features, that is, by
passing "--define '_without_nbdkit 1'" to rpmbuild. Finally, if
nbdkit support has been disabled, installing libvirt will no
longer drag it in as a (weak) dependency.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Jonathon Jongsma <jjongsma@redhat.com>
When using nbdkit to serve a network disk source, the nbdkit process
will start and wait for an nbd connection before actually attempting to
connect to the (remote) disk location. Because of this, nbdkit will not
report an error until after qemu is launched and tries to read from the
disk. This results in a fairly user-unfriendly error saying that qemu
was unable to start because "Requested export not available".
Ideally we'd like to be able to tell the user *why* the export is not
available, but this sort of information is only available to nbdkit, not
qemu. It could be because the url was incorrect, or because of an
authentication failure, or one of many other possibilities.
To make this friendlier for users and easier to detect
misconfigurations, try to connect to nbdkit immediately after starting
nbdkit and before we try to start qemu. This requires adding a
dependency on libnbd. If an error occurs when connecting to nbdkit, read
back from the nbdkit error log and provide that information in the error
report from qemuNbdkitProcessStart().
User-visible change demonstrated below:
Previous error:
$ virsh start nbdkit-test
2023-01-18 19:47:45.778+0000: 30895: error : virNetClientProgramDispatchError:172 : internal
error: process exited while connecting to monitor: 2023-01-18T19:47:45.704658Z
qemu-system-x86_64: -blockdev {"driver":"nbd","server":{"type":"unix",
"path":"/var/lib/libvirt/qemu/domain-1-nbdkit-test/nbdkit-libvirt-1-storage.socket"},
"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Requested export not
available
error: Failed to start domain 'nbdkit-test'
error: internal error: process exited while connecting to monitor: 2023-01-18T19:47:45.704658Z
qemu-system-x86_64: -blockdev {"driver":"nbd","server":{"type":"unix",
"path":"/var/lib/libvirt/qemu/domain-1-nbdkit-test/nbdkit-libvirt-1-storage.socket"},
"node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Requested export not
available
After this change:
$ virsh start nbdkit-test
2023-01-18 19:44:36.242+0000: 30895: error : virNetClientProgramDispatchError:172 : internal
error: Failed to connect to nbdkit for 'http://localhost:8888/nonexistent.iso': nbdkit: curl[1]:
error: problem doing HEAD request to fetch size of URL [http://localhost:8888/nonexistent.iso]:
HTTP response code said error: The requested URL returned error: 404
error: Failed to start domain 'nbdkit-test'
error: internal error: Failed to connect to nbdkit for 'http://localhost:8888/nonexistent.iso]:
error: problem doing HEAD request to fetch size of URL [http://localhost:8888/nonexistent.iso]:
HTTP response code said error: The requested URL returned error: 404
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Adds the ability to monitor the nbdkit process so that we can take
action in case the child exits unexpectedly.
When the nbdkit process exits, we pause the vm, restart nbdkit, and then
resume the vm. This allows the vm to continue working in the event of a
nbdkit failure.
Eventually we may want to generalize this functionality since we may
need something similar for e.g. qemu-storage-daemon, etc.
The process is monitored with the pidfd_open() syscall if it exists
(since linux 5.3). Otherwise it resorts to checking whether the process
is alive once a second. The one-second time period was chosen somewhat
arbitrarily.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
The conversion from ternary to a 'if' clause was wrong and thus didn't
properly increase the stack size where needed but only where not
actually needed.
Fixes: b68faa99d9
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Instead of an assignment into the 'stack_frame_size' variable when
sanitizers are enabled I've accidentally compared the value against the
requested size.
Fix the typo.
Fixes: b68faa99d9
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
When building without optimization on clang, certain big functions trip
the stack size limit despite not actually reaching it. Relax the stack
limit size for clang without optimization.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
After recent cleanups we can now restrict the maximum stack frame size
to 2k.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Kristina Hanicova <khanicov@redhat.com>
openSUSE Leap 15.{4,5} are supported under libvirt's distro support
statement, but they only contain attr version 2.4.47.
Reverts: dffeef89ef
Signed-off-by: Jim Fehlig <jfehlig@suse.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
We will soon need to base some decisions on whether AppArmor 3.x
or 2.x is present on the system.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
So far this change alone doesn't make much sense, but prepares
code for upcoming change. Unfortunately, some conf.has()
statements have to stay, because there's no corresponding
dependency(). But that's okay.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The pkg-config file to libnuma was introduced in 2.0.12 release
(though the comment mistakenly claims 2.0.14 version). Every
supported distro ships at least this version, and thus we can
switch meson detection to dependency().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The pkg-config file to libattr was introduced in 2.4.48 release.
Now that every supported distro ships at least this version, we
can switch meson detection to dependency().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The pkg-config file to libacl was introduced in 2.2.53 release.
Now that every supported distro ships at least this version, we
can switch meson detection to dependency().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Now that we're performing the lookup at runtime, doing it at
build time is no longer necessary.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Now that we're performing the lookup at runtime, doing it at
build time is no longer necessary.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Keep /etc/sysconfig as the fallback, but pick more suitable
values for various Linux distros.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
We're about to introduce another user of the value in a
different scope.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Right now we expect the configuration files for init scripts
to live in /etc/sysconfig, but that location is only used by
RHEL- and SUSE-derived distros.
This means that packagers for other distros have to patch
things as part of the build process, while people building
from source will get wonky integration.
This new option will provide a convenient way to override
the default location at build time that is usable by distro
packagers and people building from source alike.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
The current values might have been accurate at the time
when the logic was introduced, but these days Arch is
using the same ones as Debian.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat>
This fixes cross-building in some scenarios.
Specifically, when building for armv7l on x86_64, has_header()
will see the x86_64 version of the linux/kmv.h header and
consider it to be usable. Later, when an attempt is made to
actually include it, the compiler will quickly realize that
things can't quite work.
The reason why we haven't hit this in our CI is that we only ever
install the foreign version of header files. When building the
Debian package, however, some of the Debian-specific tooling will
bring in the native version of the Linux headers in addition to
the foreign one, causing meson to misreport the header's
availability status.
Checking for actual usability, as opposed to mere presence, of
headers is enough to make things work correctly in all cases.
The meson documentation recommends using has_header() instead of
check_header() whenever possible for performance reasons, but
while testing this change on fairly old and underpowered hardware
I haven't been able to measure any meaningful slowdown.
https://bugs.debian.org/1024504
Suggested-by: Helmut Grohne <helmut@subdivi.de>
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In our meson scripts, we use configure_file(copy:true) to copy
files from srcdir into builddir. However, as of meson-0.64.0,
this is deprecated [1] in favor of using:
fs = import('fs')
fs.copyfile(in, out)
Except, the submodule's new method wasn't introduced until
0.64.0. And since we can't bump the minimal meson version we
require, we have to work with both: new and old versions.
Now, the fun part: fs.copyfile() is not a drop in replacement as
it returns different type (a custom_target object). This is
incompatible with places where we store the configure_file()
retval in a variable to process it further.
While we could just replace 'copy:true' with a dummy
'configuration:...' (say 'configuration: configmake_conf') we
can't do that for binary files (like src/fonts/ or src/images/).
Therefore, places where we are not interested in the retval can
be switched to fs.copyfile() and places where we are interested
in the retval will just use a dummy 'configuration:'.
Except, src/network/meson.build. In here we not just copy the
file but also specify alternative install dir and that's not
something that fs.copyfile() can handle. Yet, using 'copy: true'
is viewed wrong [2].
1: https://mesonbuild.com/Release-notes-for-0-64-0.html#fscopyfile-to-replace-configure_filecopy-true
2: https://github.com/mesonbuild/meson/pull/10042
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Follow better meson build system conventions. This allows to find
keymap-gen or CSV without explicitly setting the paths.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Or meson will complain with:
../meson.build:770:2: ERROR: Search directory /sbin is not an absolute path.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
There are some CLang versions that do not support
-fsemantic-interposition. If that's the case, the code is
optimized so much that our mocking no longer works.
Therefore, disable tests and produce a warning.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
With its version 16.0, the LLVM's linker turned on
--no-undefined-version by default [1]. This breaks how we detect
--version-script= detection, because at the compile time there's
no library built yet that we can use to make --version-script=
happy. Unfortunately, meson does not provide a way to detect this
either [2].
But there's not much sense in detecting the argument either. We
already special case some systems (windows, darwin) and do the
check for others, which are expected to support versioned
symbols, because of ELF. Worst case scenario - the error is
reported during compile time rather than configure time.
1: https://reviews.llvm.org/D135402
2: https://github.com/mesonbuild/meson/issues/3047
Resolves: https://bugs.gentoo.org/902211
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The virNumaNodeIsAvailable function is stubbed out when building
without libnuma, such that it just returns a constant value. When
CLang is optimizing, it does inter-procedural analysis across
function calls. When it sees that the call to virNumaNodeIsAvailable
returns a fixed constant, it elides the conditional check for errors
in the callers such as virNumaNodesetIsAvailable.
This is a valid optimization as the C standard declares that there
must only be one implementation of each function in a binary. This
is normally the case, but ELF allows for function overrides when
linking or at runtime with LD_PRELOAD, which is technically outside
the mandated C language behaviour.
So while CLang's optimization works fine at runtime, it breaks in our
test suite which aims to mock the virNumaNodeIsAvailable function so
that it has specific semantics regardless of whether libnuma is built
or not. The return value check optimization though means our mock
override won't have the right effect. The mock will be invoked, but
its return value is not used.
Potentially the same problem could be exhibited with GCC if certain
combinations of optimizations are enabled, though thus far we've
not seen it.
To be robust on both CLang and GCC we need to make it more explicit
that we want to be able to replace functions and thus optimization
of calls must be limited. Currently we rely on 'noinline' which
does successfully prevent inlining of the function, but it cannot
stop the eliding of checks based on the constant return value.
Thus we need a bigger hammer.
There are a couple of options to disable this optimization:
* Annotate a symbol as 'weak'. This is tells the compiler
that the symbol is intended to be overridable at linktime
or runtime, and thus it will avoid doing inter-procedural
analysis for optimizations. This was tried previously but
have to be reverted as it had unintended consequences
when linking .a files into our final .so, resulting in all
the weak symbol impls being lost. See commit
407a281a8e
* Annotate a symbol with 'noipa'. This tells the compiler
to avoid inter-procedural analysis for calls to just this
function. This would be ideal match for our scenario, but
unfortunately it is only implemented for GCC currently:
https://reviews.llvm.org/D101011
* The '-fsemantic-interposition' argument tells the optimizer
that any functions may be replaced with alternative
implementations that have different semantics. It thus
blocks any optimizations across function calls. This is
quite a harsh block on the optimizer, but it appears to be
the only one that is viable with CLang.
Out of those choices option (3) is the only viable option for
CLang. We don't want todo it for GCC though as it is such a
big hammer. Probably we should apply (2) for GCC, should we
experiance a problem in future.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
