Commit Graph

49120 Commits

Author SHA1 Message Date
Erik Skultety
78862b67e1 ci: Drop the now unused Makefile
All the functionality this script provided has been incorporated either
in the Python ci/helper tool or lcitool directly.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
3bcf326150 ci: helper: Drop the _make_run method
We've successfully migrated over to lcitool to take care of the
container workload execution, so dropping this 'make' prep code is a
prerequisite of finally getting rid of the ci/Makefile script.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
1e77c9c834 ci: helper: Drop the --meson-args/--ninja-args CLI options
These originally allowed customizing the ci/Makefile script which was
the core of the local container executions. The problem was that
however flexible this may have been, it never mirrored what was being
done as part of the GitLab jobs. Motivated by the effort of mirroring
GitLab jobs locally, these would only ever make sense to be set/used in
interactive shell container sessions where the developer is perfectly
capable of using the right meson/ninja CLI options directly without
going through another shell variable indirection as it was the case
with these ci/helper options.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
0f7e2b7468 ci: helper: Drop original actions
Previous patches added a single 'run' command parametrized with GitLab
job specs via '--job' that cover all of these original actions, adding
some more in the process. Drop the original actions as we don't need
them anymore.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
eb41e45630 ci: helper: Add a job argparse subparser
The idea behind this subcommand is to follow whatever build job we have
defined in the GitLab CI pipeline, so that we only have a single source
of truth for the recipes. Adds 'shell' as an extra option for
interactive container build debugging.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
e4dc7deaaa ci: helper: Add an action to run the container workload via lcitool
Just like with the other CLI sub-commands, add an action to run a
GitLab spec job in a local container via lcitool.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
04b9118fe5 ci: helper: Rework _lcitool_run method logic
This method wasn't even utilized before this patch. This patch adds all
the necessary logic to successfully execute a container workload via
lcitool (which will later allow us to ditch ci/Makefile). Because
container executions via lcitool creates the following inside the
container:

    $ ls
    script datadir

where 'datadir' is the workload directory (in this case a local git
repo clone) and 'script' is the code that runs whatever the workload is
over 'datadir'.

In order to satisfy the ^above, our helper generates a trivial
temporary 'script' that will source ci/build.sh and run whatever was
specified as --job essentially to simulate the exact steps a GitLab
pipeline job would go through.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
4ad2f89244 ci: helper: Add a helper to create a local repo clone Pythonic way
A proper Python equivalent of 'git clone --local'.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
24319b2cca ci: helper: Add Python code hangling git clones
This helper will be utilized by a future patch which will add the
lcitool container execution logic. The reason why the required_deps
decorator isn't being used here is because this is a property.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
05d65767cf ci: helper: Add a required_deps higher order helper/decorator
Since we'll depend on GitPython for repo cloning, we need to make sure
to emit a user friendly error if the module is not installed. This
patch introduces a helper which future patches will use as a decorator.
Inspiration for this helper has been taken out of lcitool where we use
an identical helper for this purpose.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
a35b4e4e97 ci: helper: Add --lcitool-path CLI option
We'll soon be relying solely on lcitool so we need to be able to run it
from a user-provided location if it's not installed in a known
location.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
f6f930b435 ci: helper: Don't make ':' literal a static part of the image tag
':' is just a connecting character, we can add it to the appropriate
place later in the Python script later, but it doesn't make sense to be
part of the image 'tag' string.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
5f78628fb6 ci: helper: Drop _lcitool_get_targets method
This method unused anywhere, so drop it.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
2471a3b232 .gitlab-ci.yml: Convert the potfile job to the build.sh usage
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
7f2adaa3d8 .gitlab-ci.yml: Convert the codestyle job to the build.sh usage
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
6283e99edb .gitlab-ci.yml: Convert the website build job to the build.sh usage
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
8a2866a4f6 .gitlab-ci.yml: Convert the cross build job to the build.sh usage
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
f688a53a3e .gitlab-ci.yml: Convert the native build job to the build.sh usage
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
3104dee434 .gitlab-ci.yml: Add 'after_script' stage to prep for artifact collection
This is one of the preparation steps that if not done would otherwise
collide with local container executions where we:
1) don't collect artifacts
2) are not limited by GitLab's environment and hence moving build
   artifacts to unusual places would only cause confusion when doing
   local build inspection in an interactive container shell session

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
ed7aa04de4 ci: Rename build.sh -> jobs.sh
After the recent changes, this script no longer executes any logic
anymore, it merely defines the jobs run in the GitLab environment. In
order to use it, one has to source the file in the environment and then
run one of the job "functions". For that, the 'build.sh' name is no
longer descriptive enough and 'jobs.sh' feels more suitable and less
misleading.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
5a2f2eb221 ci: build.sh: Drop MESON_ARGS definition from global level
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
1460572b66 ci: build.sh: Drop direct invocation of meson/ninja commands
We've moved all invocations to the respective helper function which
we'll execute both from gitlab CI jobs and local environments so we
don't need to have them on the global level as it would also not work
with "sourcing" this file to populate the environment with function
definitions.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
56b37fe381 ci: build.sh: Drop changing working directory to CI_CONT_DIR
Firstly, this would mangle with "sourcing" this file in either
execution environment later down the road. Secondly, we won't need this
as future ci/helper patches will generate a throwaway script that will
take care of a correct execution of a build job in a similar fashion as
if the job ran in a GitLab environment.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
fe9f788812 ci: build.sh: Add a wrapper function over the 'website' job
This helper is a shell function transcript of its original GitLab CI
counterpart.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
da0a354476 ci: build.sh: Add a wrapper function over the 'rpmbuild' job
This helper is a shell function transcript of its original GitLab CI
counterpart.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
07a8556076 ci: build.sh: Add a wrapper function over the 'potfile' job
This helper is a shell function transcript of its original GitLab CI
counterpart. There's one notable difference such that we pass '-j1' to
the meson compile command otherwise we'd have to execute the 'run_build'
function twice, passing 'libvirt-pot-dep' and 'libvirt-pot' targets
in a serial manner.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
1b5f2d6b09 ci: build.sh: Add a wrapper function over the 'codestyle' job
This helper is a shell function transcript of its original GitLab CI
counterpart.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
8e660c5286 ci: build.sh: Add a wrapper function over the 'test' job
This helper is a shell function transcript of its original GitLab CI
counterpart.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
387553db48 ci: build.sh: Add a helper function to create the dist tarball
This helper function does not correspond to a particular GitLab job, it
just logically separates the necessary step of creating a dist tarball
from the RPM build job that takes over.
One notable change here is the need to update git's file index which
causes issues in local container executions which rely on a shallow
copy of the libvirt repo created as:

    $ git clone --local

Even if all changes have been committed, git often complained
otherwise. Updating the index in a GitLab environment is a NOP.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
5452ca9b1c ci: build.sh: Add a wrapper function over the 'build' job
This helper is a shell function transcript of its original GitLab CI
counterpart.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
5ee2851481 ci: build.sh: Add a wrapper function over meson's setup
The reason for this wrapper is that all job functions introduced in
future patches will refer to this one instead of open-coding the same
'meson setup' invocation N times. It also prevents 'setup' to be called
multiple times as some future job functions might actually do just that
in a transitive manner.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
6214ae55f6 ci: build.sh: Add a wrapper function executing 'shell' commands
This would normally be not needed at all, but the problem here is the
Shell-in-YAML which GitLab interprets. It outputs every command that
appears as a line in the 'script' segment in a color-coded fashion for
easy identification of problems. Well, that useful feature is lost when
there's indirection and one script calls into another in which case it
would only output the respective script name which would make failure
investigation harder. This simple helper tackles that by echoing the
command to be run by any script/function with a color escape sequence
so that we don't lose track of the *actual* shell commands being run as
part of the GitLab job pipelines. An example of what the output then
might look like:
    [RUN COMMAND]: 'meson compile -C build install-web'

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:35:58 +02:00
Erik Skultety
d3c6fb6a28 ci: build.sh: Don't mention that MESON_ARGS are available via CLI
Previous patches have removed the code that allowed injecting arbitrary
meson arguments, same for ninja args.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 10:36:13 +02:00
Erik Skultety
e61241d7f4 ci: build.sh: Add GIT_ROOT env helper variable
We'll use this one in many of the job functions future patches will
introduce, it's a neat shortcut to avoid using relative paths.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 10:36:13 +02:00
Erik Skultety
470596b850 ci: build.sh: Add variables from .gitlab-ci.yml
These are common variables we wish to use in containerized environments
both in GitLab and locally. Having these defined in a single place
rather than twice is highly preferable.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 10:36:04 +02:00
Peter Krempa
ca40725a21 manpage: virsh: Note that 'virsh undefine --remove-all-storage' doesn't remove backing images
Removing a backing image could break other image chains as it's
theoretically possible to share backing chains.

As --storage/--remove-all-storage is fully implemented in virsh as a
helper option, which enumerates and deletes VM's volumes. We do not plan
to make it any more complicated.

Document that backing chains are not removed.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-11 13:15:01 +02:00
Michal Privoznik
c96b3f5b5d virutil: Check retval of capng_apply()
Inside of virSetUIDGIDWithCaps() there's a naked call to
capng_apply(), i.e. without any retval check. This is potentially
dangerous as capng_apply() may fail. Do the check and report an
error.

This also fixes the build on bleeding edge distros - like Fedora
rawhide - where the function is declared with 'warn unused
result' [1].

1: a0743c335c

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2023-09-11 12:34:11 +02:00
Michal Privoznik
3222c9ca67 lxc_container: Check retval of capng_get_caps_process()
Added in v0.6.5~14 the call to capng_get_caps_process() inside of
lxcContainerDropCapabilities() is not really explained in the
commit message. But looking into the libcap-ng sources it's to
initialize the internal state of the library.

But with recent libcap-ng commit [1] (which some bleeding edge
distros - like Fedora rawhide - already picked up) the function
has been marked as 'warn unused result'. Well, check for its
retval then.

1: a0743c335c

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2023-09-11 12:34:03 +02:00
Praveen K Paladugu
efeaf5589c ch: Fix cloud-hypervisor version processing
Refactor the version processing logic in ch driver to support versions
from non-release cloud-hypervisor binaries. This version also supports
versions with branch prefixes in them.

Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2023-09-11 10:40:53 +02:00
Michal Privoznik
16ad37c119 virnetdevopenvswitch: Propagate OVS error messages
When configuring OVS interfaces/bridges we spawn 'ovs-vsctl' with
appropriate arguments and if it exited with a non-zero status we
report a generic error message, like "Unable to add port vnet0 to
OVS bridge ovsbr0". This is all cool, but the real reason why
operation failed is hidden in (debug) logs because that's where
virCommandRun() reports it unless caller requested otherwise.

This is a bit clumsy because then we have to ask users to turn on
debug logs and reproduce the problem again, e.g. [1].

Therefore, in cases where an error is reported to the user - just
read ovs-vsctl's stderr and include it in the error message. For
other cases (like VIR_DEBUG/VIR_WARN) - well they are meant to
end up in (debug) logs anyway.

1: https://mail.openvswitch.org/pipermail/ovs-discuss/2023-September/052640.html
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2023-09-08 13:42:25 +02:00
김인수
32613cb695 Translated using Weblate (Korean)
Currently translated at 100.0% (10411 of 10411 strings)

Translation: libvirt/libvirt
Translate-URL: https://translate.fedoraproject.org/projects/libvirt/libvirt/ko/

Co-authored-by: 김인수 <simmon@nplob.com>
Signed-off-by: 김인수 <simmon@nplob.com>
2023-09-08 05:21:16 +02:00
Michal Privoznik
67c93cfae9 virsh-snapshot: Avoid passing NULL to qsort() in virshSnapshotListCollect()
If a domain has no snapshots and 'virsh snapshot-list' is called,
this gets all the way down to virshSnapshotListCollect() which
then collects all snapshots (none), and passes them to qsort()
which doesn't like being called with NULL:

  extern void qsort (void *__base, size_t __nmemb, size_t __size,
                     __compar_fn_t __compar) __nonnull ((1, 4));

Resolves: https://gitlab.com/libvirt/libvirt/-/issues/533
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-07 11:37:54 +02:00
Tim Wiederhake
bfe53e9145 cpu_map: Add cpu model EPYC Genoa
This was added in qemu commit 166b174188.
No additional features had to be added to libvirt.

Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2023-09-06 20:35:45 +02:00
Tim Wiederhake
1504d677c2 cpu_map: Ignore all vmx-* features in sync_qemu_features_i386.py
The script that synchronizes cpu models from qemu,
sync_qemu_models_i386.py, ignores all features that begin with
"vmx-". Do the same for synchronizing cpu features so we do not
have to track irrelevant features individually.

Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2023-09-06 20:35:34 +02:00
Andrea Bolognani
65c5c4db3d rpm: Install client-qemu with main package
The libvirt package is a sort of catch-all that brings in all
daemon drivers as well as the client package, so it makes sense
for it do drag in the QEMU-specific clients as well.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-06 15:03:47 +02:00
Peter Krempa
58d9fe7202 qemu: capabilities: Remove 'virQEMUCapsAddCPUDefinitions'
The function was used only to fill the cpu models into fake
capabilities, whic no longer exists.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-06 14:09:46 +02:00
Peter Krempa
14106524cc qemuxml2argvtest: Remove cpu model faking for fake-caps tests
Now that all fake-caps testing was removed we can also remove the
filling of the fake caps by cpu models.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-06 14:09:46 +02:00
Peter Krempa
33f27cc978 qemu: capabilities: Unexport 'virQEMUCapsAddMachine'
It's no longer needed in tests as we are no longer adding fake machines.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-06 14:09:46 +02:00
Peter Krempa
6c31cddd58 testutilsqemu: Unexport 'testQemuCapsInit'
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-06 14:09:46 +02:00
Peter Krempa
fd1c066de0 testutilsqemu: Drop infrastructure for fake-caps testing
Remove all the code for adding fake machines into the testing capability
cache as we no longer have any machines in it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-06 14:09:46 +02:00