The previous fix:
commit b069efe29c
Author: Daniel P. Berrangé <berrange@redhat.com>
Date: Fri Jun 14 19:57:06 2024 +0100
gitlab: fix codestyle CI job
was incomplete, as the job inheritance was also
broken.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Jobs whose names start with a '.' as treated as templates, so
not actually run in a pipeline.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
When the 'pages' job is configured to run 'on_success' it's skipped if
any other pipeline fails. This is bad in cases such as if an external
service runs out of CI minutes as the web stops being updated.
Since the 'artifacts' of the 'website_job' are generated only if that
phase succeeds this will update the web when the web part is buildable.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
It's currently running against AlmaLinux 8 which went out of
support.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
It's currently running against AlmaLinux 8 which went out of
support.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
By the time of release, it's going to be more than two years
since AlmaLinux 9 was released and per our support policy,
AlmaLinux 8 (the previous major release) will be not supported.
Switch from AlmaLinux 8 to AlmaLinux 9.
This also means the website_job which depends on AlmaLinux 8
needs to be moved to newer AlmaLinux.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Adapt the configuration of redirects from the server hosting libvirt.org
The redirect to the 'libvirt-console-proxy' Golang module is not adapted
as it doesn't exist on the current server.
NOTES:
- The redirects are currently configured for hosting via the
'gitlab.io/libvirt' URI. For hosting via custom domain it will need
to be rewritten to drop the '/libvirt' prefixes.
- gitlab pages doesn't currently support redirects to outside content,
thus most of the redirects don't actually work:
https://gitlab.com/gitlab-org/gitlab-pages/-/issues/601
- The redirects file is only installed in the webpage job but is not
actually distributed.
- The validity of the redirects can be checked by visiting:
https://libvirt.gitlab.io/_redirects
Having them installed allows us to validate them before we'll be
switching to use gitlab pages completely.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
On pushes to master publish the webpage (built in the 'website_job' job)
via gitlab pages. The 'pages' job uses the default image that gitlab
assumes as it's consuming artifacts from an existing job and naming
them properly.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The latest lcitool merged the 'prebuilt-env' and 'local-env' jobs into
one which use variables to pick up the right environment and steps
rather than duplicating everything.
Regenerate the generated job definitions, fix the helper definitions
and also fix the manually defined jobs (website-job).
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Now that the spec file supports selectively disabling the native,
mingw32 and mingw64 parts, we can add coverage for the MinGW RPM
builds.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
The URL to the Coverity tools download has changed; the old one points
to an obsolete version that is not supported anymore. Adjust to point
to the correct and supported tools.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Individual shell command executions are replaced by respective
functions in the ci/build.sh base script. This will make sure we use
the same recipes in GitLab jobs as well as in local executions.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This is one of the preparation steps that if not done would otherwise
collide with local container executions where we:
1) don't collect artifacts
2) are not limited by GitLab's environment and hence moving build
artifacts to unusual places would only cause confusion when doing
local build inspection in an interactive container shell session
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This is continuation of what commit b56e2be68e started. If we stick to
only calling meson commands directly, we can achieve much better
consistency in passing arguments to meson especially if we unify the
recipes run in gitlab CI and what we can currently run locally in
containers using docker/podman.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Commit a3cc0e9ceb forgot to tweak the codestyle job so that we refer
to Leap 15.4 as Leap 15 (with the recent change in lcitool). However,
it was easy to miss as that job is not managed by the manifest.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
The potfile job will fail unless all format strings are permutable
(checked by meson compile libvirt-pot-check).
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The '15.3' version is EOL now:
https://get.opensuse.org/leap/15.3/
Also switch the 'codestyle' job to the appropriate container image.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The 'potfile' job is lacking the 'variables' section which defines the
'NAME' variable used by the 'image' property resulting in the 'potfile'
job failing with:
Pulling docker image registry.gitlab.com/libvirt/libvirt/ci-:latest ...
WARNING: Failed to pull image with policy "always": invalid reference format (manager.go:235:0s)
ERROR: Job failed: failed to pull image "registry.gitlab.com/libvirt/libvirt/ci-:latest" with specified policies [always]: invalid reference format (manager.go:235:0s)
Fixes: 491d918502
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This refresh switches the CI for contributors to be triggered by merge
requests. Pushing to a branch in a fork will no longer run CI pipelines,
in order to avoid consuming CI minutes. To regain the original behaviour
contributors can opt-in to a pipeline on push
git push <remote> -o ci.variable=RUN_PIPELINE=1
This variable can also be set globally on the repository, through the
web UI options Settings -> CI/CD -> Variables, though this is not
recommended. Upstream repo pushes to branches will run CI.
The use of containers has changed in this update, with only the upstream
repo creating containers, in order to avoid consuming contributors'
limited storage quotas. A fork with existing container images may delete
them. Containers will be rebuilt upstream when pushing commits with CI
changes to the default branch. Any other scenario with CI changes will
simply install build pre-requisite packages in a throaway environment,
using the ci/buildenv/ scripts. These scripts may also be used on a
contributor's local machines.
With pipelines triggered by merge requests, it is also now possible to
workaround the inability of contributors to run pipelines if they have
run out of CI quota. A project member can trigger a pipeline from the
merge request, which will run in context of upstream, however, note
this should only be done after reviewing the code for any malicious
CI changes.
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Right now we're setting the prefix to a custom path, which
results in paths like
/builds/libvirt/libvirt/vroot/etc/libvirt/virtqemud.conf
ending up in the generated HTML. In order to avoid that,
set the prefix and other installation paths to reasonable
default values by passing
-Dsystem=true
and then take advantage of $DESTDIR support to still be able
to write the HTML files without requiring root privileges.
Reported-by: Martin Kletzander <mkletzan@redhat.com>
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
One specfile containing both native and mingw builds is the
new best practice for Fedora. This reduces the maint burden
and ensures the mingw packages don't fall behind.
Note this adds many more BuildRequires for anyone building
on Fedora, which will now need installing.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The container jobs are all optional in upstream now, only
running if dockerfile changes were made. Thus any build
jobs which reference the container jobs need to mark their
dependency as optional too.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This stage will download build artifacts from both the libvirt and
libvirt-perl (multi-project CI) builds, install all them on the custom
runners and configures libvirt debug logging on the runners prior to
executing the actual test suite. In case of a failure, libvirt and
Avocado logs will be saved and published as pipeline artifacts.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We're already building libvirt in the containers already, if we publish
the build in form of, say, RPMs, later stages of the pipeline can
consume the RPMs instead of re-building libvirt from scratch.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Upstream lcitool suggests that as a solution to 'centos-8' being
removed.
Move also the website and other jobs to depend on
'x86_64-almalinux-8-container'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
When a build fails it is helpful to know what packages were installed,
because by the time we look at the build job output, the original
container image might have changed.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We previously had a 'rules:' entry that caused a job to be skipped if
the variable "TEMPORARILY_DISABLED" was set. This is no longer needed
since we can set a similar flag in ci/manifest.yml and re-generate
to temporarily skip a job.
Unfortunately the 'rules:' entry had an unexpected side-effect on
the pipelines that was never previously noticed. Instead of only
running pipelines on push, the mere existance of the 'rules:' entry
caused triggering of pipelines on merge requests too.
The newly auto-generated ci/gitlab.yml file does not have a 'rules:'
for the container job template, and thus only runs on git push.
The result is that build jobs try to run on merge requests and the
container jobs they depend on don't exist. This breaks the entire
pipeline with a message that the config is invalid due to broken
job dependencies.
This fixes a regression introduced in
commit ccc7a44adb
Author: Daniel P. Berrangé <berrange@redhat.com>
Date: Thu Sep 9 14:49:01 2021 +0100
ci: re-generate containers/gitlab config from manifest
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This uses the command "lcitool manifest ci/manifest.yml" to re-generate
all existing dockerfiles and gitlab CI config.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The sanitizer jobs run in ubuntu 20.04 containers and thus overlap with
testing already done for the regular ubuntu 20.04 build job. Fold the
sanitizer run for GCC into the regular build job and add a second
ubuntu 20.04 build job for CLang sanitizers.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
It was in the build stage previously to let it run in parallel with
other build jobs, but with the "needs" clause this is not required.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The current docker:dind container has broken default seccomp filter that
results in clone3 being blocked, which in turn breaks Fedora 35 rawhide.
This custom image has a workaround that causes the seccomp filter to
return ENOSYS for clone3 instad of EPERM, thus triggering glibc to
fallback to clone correctly.
Reviewed-by: Erik Skultety <eskultet@redhat>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The nature of Fedora rawhide means that it will inevitably have failures
periodically. Currently it is failing to even update packages due to
glibc switching to use of clone3 syscall, which is mistakenly blocked by
seccomp in container runtimes using EPERM instead of ENOSYS.
When we fail to build the rawhide containers, it is quite likely that we
still have the previous build available in the gitlab registry, so it is
reasonable to allow the container job to fail and try the build job
anyway.
Ideally we would ignore the container build failure if-and-only-if the
previous container was build with the same list of packages. We don't
record the original dependency package list though, so that's not
currently possible.
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The undefined behaviour sanitizer (UBSAN) defaults to merely printing an
error message if it detects undefined behaviour. These error messages often
end up in captured output and do not fail the tests, effectively hiding
the warning. Make the test cases fail to make the issues visible.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The base OS image might include outdated contents, and we don't
want to get spurious failures caused by bugs that have already been
fixed in the respective packages.
This is particularly important on macOS, because 'brew install foo'
will fail if 'foo' is already installed but outdated: upgrading all
packages first ensures we never run into this scenario.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Docs are not sanitized, thus there's no point in building them.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Tim Wiederhake <twiederh@redhat.com>
meson supports the following sanitizers: "address" (e.g. out-of-bounds
memory access, use-after-free, etc.), "thread" (data races), "undefined"
(e.g. signed integer overflow), and "memory" (use of uninitialized
memory). Note that not all sanitizers are supported by all compilers,
and that more sanitizers exist.
Not all sanitizers can be enabled at the same time, but "address" and
"undefined" can. Both thread and memory sanitizers require an instrumented
build of all dependencies, including libc.
gcc and clang use different implementations of these sanitizers and
have proven to find different issues. Create CI jobs for both.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In the distros using RPMs, we build libvirt once as a side effect
of running "ninja dist", and once via rpmbuild.
In addition "ninja dist" will run all tests including the "syntax-check"
suite, despite use having a separate "codestyle" job for for that.
There is no way to pass "--no-suite" when creating the dist, but if we
switch to invoking "meson dist", we can skip the build+test part
entirely using "--no-tests".
When doing this we then run explicit "meson compile" and "meson test"
commands for the distros that don't build the RPMs, and in the latter
case we can now skip the "syntax-check" suite.
The RPM builds already skipped the "syntax-check" suite.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The "dist" and "test" targets in ninja end up calling back into
the equivalent meson commands. The meson commands support various
arguments that are not accessible when invoked via ninja, so it
is preferrable to use meson directly.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The "DIST=skip" flag as used in CentOS 7 jobs to workaround a problem
with older git versions. This is no longer required since
commit d35003aee7
Author: Andrea Bolognani <abologna@redhat.com>
Date: Tue May 4 10:45:29 2021 +0200
ci: Drop CentOS 7
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
"meson test" will only print a list of which tests fail by default,
so we were sending the full test log to stdout on failure. This makes
it really hard to see the errors though as the test log has all
succcesful tests too.
"ninja test" will print the same as "meson test", following by details
of each failure.
It does this using the "--print-errorlog" flag, so lets use that in
the codestyle job.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
When CI jobs fail on the test suite, we don't see much helpful
information by default:
stderr:
TEST: bhyvexml2argvtest
......!!.............!......!........... 40
........................!...... 71 FAIL
Some tests failed. Run them using:
VIR_TEST_DEBUG=1 VIR_TEST_RANGE=7-8,22,29,65
/tmp/cirrus-ci-build/build/meson-private/dist-build/tests/bhyvexml2argvtest
Following the instructions to re-run the test with VIR_TEST_DEBUG=1 is
quite unfriendly when we could have had that set for CI already.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
