Commit Graph

285 Commits

Author SHA1 Message Date
Ján Tomko
b1524a3efc ci: update OpenSUSE Leap to 15.6
It was released on June 12, 2024.

The update means we no longer have to care about json-c 0.13
present in Leap 15.5, which solves some whitespace issues in
tests.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2024-08-16 13:11:57 +02:00
Michal Privoznik
fcf6beaf3d ci: Refresh generated files
This is supposed to unstuck FreeBSD as it switched to
Python-3.11.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2024-07-15 15:20:50 +02:00
Daniel P. Berrangé
ffa29eeb11 ci: fix CI package list and refresh with 'lcitool manifest'
The ci/manifest.yml file references a package 'libclang-rt-dev' that
does not exist in libvirt-ci mappings.yml. The latest refresh in

  commit 0759cf3fa6
  Author: Michal Prívozník <mprivozn@redhat.com>
  Date:   Fri May 3 15:58:20 2024 +0200

    ci: Introduce Ubuntu 24.04

was presumably done against a local change to libvirt-ci.git that
had not yet been merged, as the clang packages now appear on many
more build envs.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2024-06-18 11:13:51 +01:00
Michal Privoznik
0759cf3fa6 ci: Introduce Ubuntu 24.04
Ubuntu 24.04 was released recently. Add it to our CI. Also, to be
able to run ASAN/UBSAN builds on Ubuntu 24.04 libclang-rt-dev
needs to be installed (because clang's runtime was moved into a
separate package). Hence so many seemingly unrelated changes.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2024-05-14 15:17:23 +02:00
Michal Privoznik
a50f870da6 ci: Drop Ubuntu 20.04
It's now more than two years since Ubuntu 22.04 was released and
per our support policy, Ubuntu 20.04 (the previous major release)
is now not supported. Remove it from our CI testing.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2024-05-14 15:13:20 +02:00
Michal Privoznik
dd085b1a16 ci: Switch from Fedora 38 to Fedora 40
Since Fedora 40 was released recently, Fedora 38 is now
unsupported. Drop Fedora 38 and introduce Fedora 40 to our CI.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2024-05-14 15:13:12 +02:00
Michal Privoznik
24a3d3975f ci: Switch from AlmaLinux 8 to AlmaLinux 9
By the time of release, it's going to be more than two years
since AlmaLinux 9 was released and per our support policy,
AlmaLinux 8 (the previous major release) will be not supported.
Switch from AlmaLinux 8 to AlmaLinux 9.

This also means the website_job which depends on AlmaLinux 8
needs to be moved to newer AlmaLinux.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2024-05-14 15:13:01 +02:00
Daniel P. Berrangé
d754500e5f ci: drop CentOS 8 Stream and refresh
This drops the CentOS 8 Stream distro target, since that is going EOL
at the end of May, at which point it will cease to be installable
due to package repos being archived.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2024-05-02 13:52:30 +01:00
Daniel P. Berrangé
8132d38120 ci: refresh with latest lcitool manifest
This brings in a fix to the job rules which solves a problem with
jobs getting skipped in merge requests in some scenarios. It also
changes the way Cirrus CI vars are set, which involves a weak to
the way $PATH is set in build.yml.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2024-05-02 13:52:17 +01:00
Daniel P. Berrangé
ba3a5604c9 ci: refresh with lcitool manifest
Picks up the switch from FreeBSD 13.2 to 13.3

Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2024-03-05 10:25:12 +00:00
Michal Privoznik
cb3f1d5b29 ci: regenerate
Regenerate the ci files using the latest libvirt-ci:

  commit face9746f9729699ae8525ffac4ee19be82c1ba5
  ci: drop update-alternatives for opensuse tumbleweed

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2024-02-27 12:04:13 +01:00
Peter Krempa
743730a83b ci: cirrus: Use --break-system-packages with 'pip install'
Cirrus tests on macos started to fail when installing 'pytest'. Override
the pip lock.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
2024-02-26 11:25:09 +01:00
Ján Tomko
a883602c9a ci: regenerate
Regenerate the ci files using the latest libvirt-ci:
commit 5b9b11261fa28cae964fd91638056318f270e300
    examples: illustrate use of remote project reference

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2024-02-09 13:23:06 +01:00
Ján Tomko
aeadf4f9dc ci: lcitool: sync dependencies with libvirt-ci repo
libvirt commit 120a674f25
    ci: lcitool: Maintain project package deps lists here

added an override file for lcitool in August, but nobody regenerating
libvirt's ci files from the manifest seemed to use the override dir.

libvirt-ci commit 1f4184edfdd541964a187810b34ac4c7702b6577
    commandline: set --extra-data-dir default path
from January made $PWD/ci/lcitool the default.

Reflect the changes made in libvirt-ci's repo here too.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2024-02-09 13:23:06 +01:00
Peter Krempa
7a1285216b ci: Regenerate with latest 'lcitool' and fix manual job definitions
The latest lcitool merged the 'prebuilt-env' and 'local-env' jobs into
one which use variables to pick up the right environment and steps
rather than duplicating everything.

Regenerate the generated job definitions, fix the helper definitions
and also fix the manually defined jobs (website-job).

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2024-01-19 14:48:21 +01:00
Andrea Bolognani
10042f0253 ci: Do more as part of .qemu-build-template
Entering $SCRATCH_DIR, going back to the original directory and
setting SELinux labels for the newly-installed QEMU binaries
are all steps that logically belong to this template rather
than its callers.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2024-01-12 14:28:43 +01:00
Andrea Bolognani
9cf0faac48 ci: Fix .integration_tests_upstream_qemu
We enter $SCRATCH_DIR before going through the process of
cloning QEMU's upstream repo and building it, but once we're
done we don't get back to libvirt's sources, so the very next
step fails with

  /tmp/script.: line 188: ci/jobs.sh: No such file or directory

Use pushd/popd to ensure that we're back to the correct place
once QEMU has been built and installed.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2024-01-12 14:28:38 +01:00
Andrea Bolognani
f913c7a77b ci: Add notes for integration jobs
These are special in that, when a new target is introduced, some
preparation is needed before the changes can be merged. Since
this only happens every six months or so, it's unsurprising that
we keep messing it up and forgetting some steps. Having notes
right in the file will hopefully help going forward.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2024-01-09 09:45:49 +01:00
Andrea Bolognani
88169d35dd ci: Move upstream-qemu job to Fedora 39
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2024-01-09 09:45:45 +01:00
Andrea Bolognani
c65862fbac ci: Fix upstream-qemu job definitions
These are jobs are supposed to be running tests using a QEMU
binary built from the latest upstream sources, but right now
they're just doing the same thing as the other jobs for the
target. Use the correct job templates.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2024-01-09 09:45:36 +01:00
Michal Privoznik
392897d9b0 ci: Update Alpine and Fedora and regenerate
New Alpine and Fedora releases were added to libvirt-ci (3.19 and
39, respectively) and old ones were removed. Update the manifest
file and regenerate the rest.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2024-01-03 17:24:11 +01:00
Michal Privoznik
c330890f63 ci: integration: Switch upstream integration tests to Fedora 39
Currently, Fedora 37 and 38 is used. The former is now EOL since
there's new release. Switch 37 to 39 then.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2024-01-03 17:23:40 +01:00
Michal Privoznik
8a7b3ded60 ci: Switch from FreeBSD 12.0 to FreeBSD 14.0
FreeBSD 12.0 is no longer supported since 14.0 is out. Change the
CI manifest and refresh the rest.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2024-01-03 10:53:13 +01:00
Andrea Bolognani
fa14b4a1a6 ci: Stop passing --nodeps to rpmbuild
All BuildRequires should now be part of the build environment,
so it's no longer needed.

This ensures that, if we forget to add some BuildRequires to the
build environment in the future, the mistake will be quickly
detected.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2023-12-05 11:50:52 +01:00
Andrea Bolognani
417bd41cba ci: Refresh generated files
Add black, drop scrub.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2023-12-05 11:50:50 +01:00
Andrea Bolognani
10b354f913 ci: Refresh generated files
This adds pytest everywhere.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-12-01 13:50:16 +01:00
Andrea Bolognani
717790adde ci: Build RPMs on MinGW
Now that the spec file supports selectively disabling the native,
mingw32 and mingw64 parts, we can add coverage for the MinGW RPM
builds.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2023-11-23 18:32:03 +01:00
Andrea Bolognani
b6eab588b5 ci: Drop macOS 12
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-06 16:48:57 +01:00
Andrea Bolognani
cff934754a ci: Add macOS 14
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-06 16:48:56 +01:00
Andrea Bolognani
c1b78e1162 ci: Add macOS 13
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-06 16:48:54 +01:00
Andrea Bolognani
8895d59688 ci: Drop rpcgen override from macOS $PATH
We no longer use rpcgen.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-06 16:48:53 +01:00
Andrea Bolognani
2fd2385d9d ci: Update generated files
Gain native gettext on MinGW, lose glusterfs on 32-bit
architectures and rpcgen everywhere.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-06 16:48:47 +01:00
Daniel P. Berrangé
bbb2332f7e ci: workaround broken opensuse leap 15.5 tirpc
Temporarily disable '-Wmissing-include-dirs' becuase the
libtirpc pkg-config file has a bogus include dir. The
headers can fortunately still be found since they are
in the system include dir.

Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-06 04:25:51 -05:00
Daniel P. Berrangé
a1d92b93c5 ci: drop mipsel on Debian Sid
This port was discontinued and purged from the archives:

https://lists.debian.org/debian-devel-announce/2023/09/msg00000.html

Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2023-11-06 04:25:26 -05:00
Andrea Bolognani
5d95617ed9 ci: Disable optimizations on macOS
Clang can be too aggressive at optimizations, which can end up
breaking our test suite. See f9f5ab5718 for details.

As a result of this, since 7944700b40 we are automatically
disabling tests when Clang is used unless it supports the
-fsemantic-interposition compiler flag.

Since the version of Clang included in macOS doesn't support that
compiler flag, we end up always disabling the test suite on that
platform.

This is already far from ideal, considering that it was just last
year when we finally managed to get the test suite to successfully
pass on macOS, and it would be a real shame if the situation
regressed again.

With the upcoming changes, which will turn running 'meson test'
into a hard failure if tests are disabled, this behavior will
result in every single pipeline failing.

Work around the problem the only way we can: disabling
optimizations entirely for the macOS CI jobs.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2023-10-26 11:31:12 +02:00
Jonathon Jongsma
b74fd210b3 ci: add libnbd to build
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2023-09-19 14:28:50 -05:00
Erik Skultety
4096a59e6e ci: jobs.sh: Define and create SCRATCH_DIR for local executions
Running outside of GitLab will likely not have the variable set and
hence the execution would fail. To make sure we always start with a
clean scratch dir (which may or may not be the best thing), create it
with 'mktemp'. The main reason for a temporary directory is to ensure a
clean environment for the job every time run_integration function is
run. For repeated interactive use case, it is imperative that the
developer takes care of their environment.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
a2c3a3e038 ci: jobs.sh: run_integration: Print DAEMONS variable for debugging
One advantage that GitLab's YAML has with Shell commands is that every
single line is printed out as is, including control structures. In
order to see whether the logic did the same thing and the tests are
going to operate on the right set of daemons (monolithic vs modular),
lets print the DAEMONS variable that we set depending on the distro we
execute the tests on.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
0c9603c858 ci: jobs.sh: integration: Execute commands via 'run_cmd[_quiet]' helpers
Unfortunately, once we go down the line of running our own scripts as
part of GitLab CI jobs rather than open coding Shell in YAML, we lose
the benefit of seeing each line the script executes. The downside of
the default YAML however is that we have to maintain the same piece of
code on 2 places in that case. Let's adopt what we use with other
container jobs and prefix each shell command with 'run_cmd' or
'run_cmd_quiet' which will dump it in the logs before executing.
Flow control expressions and structures are a problem though in this
regard, so let's just print some important values for debugging
purposes.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
e004458616 ci: jobs.sh: Introduce a quiet version of run_cmd
We've started using the run_cmd helper function to log what kind of
command is being executed as well as actually executing the command.
The problem however is doing I/O redirections for commands which we
don't wish to see any output for whatever reason. Now, if the
redirection is applied at parameter passing to run_cmd it's going to be
applied to the debug print as well. Let's introduce another helper,
run_cmd_quiet which takes care of the I/O redirection and executes the
command completely silently.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
a91e6f69f4 ci: jobs.sh: run_integration: Make POSIX-compliant
Neither '&>' nor 'source' are defined in POSIX.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
a6b85e03d4 ci: jobs.sh: run_integration: Add/Rewrite/Reformat commentaries
Because of the nature of writing inline shell commands to YAML, most of
the commentaries where inlined with the command not to hinder YAML
readability any further. Since we moved the logic to a standalone
script, we can now do whatever formatting & readability adjustments we
want.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
5c9dfa2091 ci: jobs.sh: integration: Use --quiet with virsh
We've not been interested in any extra output from the command at all
since we always redirected both stdout and stderr to /dev/null. Future
patch will change that slightly, so --quiet will start making sense.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
9847d3eda4 ci: jobs.sh: Drop comment about the need for Avocado 98.0
We needed v98.0 in commit c9a65eb8 due to a bug in Avocado in the past
and have been installing the latest Avocado for a while since commit
91774931, yet we kept the comment by a mistake.

Besides, looks like v98.0 ignores the avocado.config file in the TCK
repo instructing it to run the test suite sequentially leading to test
stability issues, so abandoning the v98.0 in commit 91774931 was a good
thing in the end.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Erik Skultety
b0061e77c8 ci: integration: Drop the 'install-deps' hidden job and reference
Since the section now only consists of a single command, we can happily
move the command to the main integration template job body.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Erik Skultety
82f4bfc34c ci: integration: Adjust the check for CentOS Stream version
All supported versions of Fedora and CentOS Stream 9 default to modular
setup, it's probably better if we cosmetically adjust the CentOS Stream
version check to make it explicit that monolithic daemon services ought
to be started only on Stream 8.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Erik Skultety
6e9bd600d7 ci: integration: Extract the integration CI main recipe to jobs.sh
Follow what's been done to other jobs in .gitlab-ci.yml and extract the
shell logic from YAML to a function in ci/jobs.sh

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Erik Skultety
1f85f0967b ci: jobs.sh: Add back '--no-suite syntax-check --print-errorlogs'
Commit f688a53a converted .gitlab-ci.yml to the usage of ci/jobs.sh
functions, but in doing that our test options
    '--no-suite syntax-check --print-errorlogs'
got lost in the process and since commit 8e660c52 didn't introduce them
in the first place, it caused a behavioral regression. This patch adds
them back.

Fixes: 8e660c5286

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-13 14:17:26 +02:00
Erik Skultety
78862b67e1 ci: Drop the now unused Makefile
All the functionality this script provided has been incorporated either
in the Python ci/helper tool or lcitool directly.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00
Erik Skultety
3bcf326150 ci: helper: Drop the _make_run method
We've successfully migrated over to lcitool to take care of the
container workload execution, so dropping this 'make' prep code is a
prerequisite of finally getting rid of the ci/Makefile script.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-12 11:36:03 +02:00