Regenerate the ci files using the latest libvirt-ci:
commit 5b9b11261fa28cae964fd91638056318f270e300
examples: illustrate use of remote project reference
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
libvirt commit 120a674f25
ci: lcitool: Maintain project package deps lists here
added an override file for lcitool in August, but nobody regenerating
libvirt's ci files from the manifest seemed to use the override dir.
libvirt-ci commit 1f4184edfdd541964a187810b34ac4c7702b6577
commandline: set --extra-data-dir default path
from January made $PWD/ci/lcitool the default.
Reflect the changes made in libvirt-ci's repo here too.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
The latest lcitool merged the 'prebuilt-env' and 'local-env' jobs into
one which use variables to pick up the right environment and steps
rather than duplicating everything.
Regenerate the generated job definitions, fix the helper definitions
and also fix the manually defined jobs (website-job).
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Entering $SCRATCH_DIR, going back to the original directory and
setting SELinux labels for the newly-installed QEMU binaries
are all steps that logically belong to this template rather
than its callers.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We enter $SCRATCH_DIR before going through the process of
cloning QEMU's upstream repo and building it, but once we're
done we don't get back to libvirt's sources, so the very next
step fails with
/tmp/script.: line 188: ci/jobs.sh: No such file or directory
Use pushd/popd to ensure that we're back to the correct place
once QEMU has been built and installed.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
These are special in that, when a new target is introduced, some
preparation is needed before the changes can be merged. Since
this only happens every six months or so, it's unsurprising that
we keep messing it up and forgetting some steps. Having notes
right in the file will hopefully help going forward.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
These are jobs are supposed to be running tests using a QEMU
binary built from the latest upstream sources, but right now
they're just doing the same thing as the other jobs for the
target. Use the correct job templates.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
New Alpine and Fedora releases were added to libvirt-ci (3.19 and
39, respectively) and old ones were removed. Update the manifest
file and regenerate the rest.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Currently, Fedora 37 and 38 is used. The former is now EOL since
there's new release. Switch 37 to 39 then.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
FreeBSD 12.0 is no longer supported since 14.0 is out. Change the
CI manifest and refresh the rest.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
All BuildRequires should now be part of the build environment,
so it's no longer needed.
This ensures that, if we forget to add some BuildRequires to the
build environment in the future, the mistake will be quickly
detected.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Now that the spec file supports selectively disabling the native,
mingw32 and mingw64 parts, we can add coverage for the MinGW RPM
builds.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Gain native gettext on MinGW, lose glusterfs on 32-bit
architectures and rpcgen everywhere.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Temporarily disable '-Wmissing-include-dirs' becuase the
libtirpc pkg-config file has a bogus include dir. The
headers can fortunately still be found since they are
in the system include dir.
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Clang can be too aggressive at optimizations, which can end up
breaking our test suite. See f9f5ab5718 for details.
As a result of this, since 7944700b40 we are automatically
disabling tests when Clang is used unless it supports the
-fsemantic-interposition compiler flag.
Since the version of Clang included in macOS doesn't support that
compiler flag, we end up always disabling the test suite on that
platform.
This is already far from ideal, considering that it was just last
year when we finally managed to get the test suite to successfully
pass on macOS, and it would be a real shame if the situation
regressed again.
With the upcoming changes, which will turn running 'meson test'
into a hard failure if tests are disabled, this behavior will
result in every single pipeline failing.
Work around the problem the only way we can: disabling
optimizations entirely for the macOS CI jobs.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Running outside of GitLab will likely not have the variable set and
hence the execution would fail. To make sure we always start with a
clean scratch dir (which may or may not be the best thing), create it
with 'mktemp'. The main reason for a temporary directory is to ensure a
clean environment for the job every time run_integration function is
run. For repeated interactive use case, it is imperative that the
developer takes care of their environment.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
One advantage that GitLab's YAML has with Shell commands is that every
single line is printed out as is, including control structures. In
order to see whether the logic did the same thing and the tests are
going to operate on the right set of daemons (monolithic vs modular),
lets print the DAEMONS variable that we set depending on the distro we
execute the tests on.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Unfortunately, once we go down the line of running our own scripts as
part of GitLab CI jobs rather than open coding Shell in YAML, we lose
the benefit of seeing each line the script executes. The downside of
the default YAML however is that we have to maintain the same piece of
code on 2 places in that case. Let's adopt what we use with other
container jobs and prefix each shell command with 'run_cmd' or
'run_cmd_quiet' which will dump it in the logs before executing.
Flow control expressions and structures are a problem though in this
regard, so let's just print some important values for debugging
purposes.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
We've started using the run_cmd helper function to log what kind of
command is being executed as well as actually executing the command.
The problem however is doing I/O redirections for commands which we
don't wish to see any output for whatever reason. Now, if the
redirection is applied at parameter passing to run_cmd it's going to be
applied to the debug print as well. Let's introduce another helper,
run_cmd_quiet which takes care of the I/O redirection and executes the
command completely silently.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Neither '&>' nor 'source' are defined in POSIX.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Because of the nature of writing inline shell commands to YAML, most of
the commentaries where inlined with the command not to hinder YAML
readability any further. Since we moved the logic to a standalone
script, we can now do whatever formatting & readability adjustments we
want.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
We've not been interested in any extra output from the command at all
since we always redirected both stdout and stderr to /dev/null. Future
patch will change that slightly, so --quiet will start making sense.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
We needed v98.0 in commit c9a65eb8 due to a bug in Avocado in the past
and have been installing the latest Avocado for a while since commit
91774931, yet we kept the comment by a mistake.
Besides, looks like v98.0 ignores the avocado.config file in the TCK
repo instructing it to run the test suite sequentially leading to test
stability issues, so abandoning the v98.0 in commit 91774931 was a good
thing in the end.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Since the section now only consists of a single command, we can happily
move the command to the main integration template job body.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
All supported versions of Fedora and CentOS Stream 9 default to modular
setup, it's probably better if we cosmetically adjust the CentOS Stream
version check to make it explicit that monolithic daemon services ought
to be started only on Stream 8.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Follow what's been done to other jobs in .gitlab-ci.yml and extract the
shell logic from YAML to a function in ci/jobs.sh
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Commit f688a53a converted .gitlab-ci.yml to the usage of ci/jobs.sh
functions, but in doing that our test options
'--no-suite syntax-check --print-errorlogs'
got lost in the process and since commit 8e660c52 didn't introduce them
in the first place, it caused a behavioral regression. This patch adds
them back.
Fixes: 8e660c5286
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
All the functionality this script provided has been incorporated either
in the Python ci/helper tool or lcitool directly.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
We've successfully migrated over to lcitool to take care of the
container workload execution, so dropping this 'make' prep code is a
prerequisite of finally getting rid of the ci/Makefile script.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
These originally allowed customizing the ci/Makefile script which was
the core of the local container executions. The problem was that
however flexible this may have been, it never mirrored what was being
done as part of the GitLab jobs. Motivated by the effort of mirroring
GitLab jobs locally, these would only ever make sense to be set/used in
interactive shell container sessions where the developer is perfectly
capable of using the right meson/ninja CLI options directly without
going through another shell variable indirection as it was the case
with these ci/helper options.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Previous patches added a single 'run' command parametrized with GitLab
job specs via '--job' that cover all of these original actions, adding
some more in the process. Drop the original actions as we don't need
them anymore.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The idea behind this subcommand is to follow whatever build job we have
defined in the GitLab CI pipeline, so that we only have a single source
of truth for the recipes. Adds 'shell' as an extra option for
interactive container build debugging.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Just like with the other CLI sub-commands, add an action to run a
GitLab spec job in a local container via lcitool.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This method wasn't even utilized before this patch. This patch adds all
the necessary logic to successfully execute a container workload via
lcitool (which will later allow us to ditch ci/Makefile). Because
container executions via lcitool creates the following inside the
container:
$ ls
script datadir
where 'datadir' is the workload directory (in this case a local git
repo clone) and 'script' is the code that runs whatever the workload is
over 'datadir'.
In order to satisfy the ^above, our helper generates a trivial
temporary 'script' that will source ci/build.sh and run whatever was
specified as --job essentially to simulate the exact steps a GitLab
pipeline job would go through.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
A proper Python equivalent of 'git clone --local'.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This helper will be utilized by a future patch which will add the
lcitool container execution logic. The reason why the required_deps
decorator isn't being used here is because this is a property.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Since we'll depend on GitPython for repo cloning, we need to make sure
to emit a user friendly error if the module is not installed. This
patch introduces a helper which future patches will use as a decorator.
Inspiration for this helper has been taken out of lcitool where we use
an identical helper for this purpose.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
We'll soon be relying solely on lcitool so we need to be able to run it
from a user-provided location if it's not installed in a known
location.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
':' is just a connecting character, we can add it to the appropriate
place later in the Python script later, but it doesn't make sense to be
part of the image 'tag' string.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
After the recent changes, this script no longer executes any logic
anymore, it merely defines the jobs run in the GitLab environment. In
order to use it, one has to source the file in the environment and then
run one of the job "functions". For that, the 'build.sh' name is no
longer descriptive enough and 'jobs.sh' feels more suitable and less
misleading.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
