After addition of the new libvirt-client-qemu sub-package which is using
python bindings (thus creating a circular dependency between the libvirt
and libvirt-python projects) the integration jobs fail with:
Error:
Problem: conflicting requests
- nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
The libvirt-python project now provides the RPMs in artifacts:
https://gitlab.com/libvirt/libvirt-python/-/merge_requests/96
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This refresh switches the CI for contributors to be triggered by merge
requests. Pushing to a branch in a fork will no longer run CI pipelines,
in order to avoid consuming CI minutes. To regain the original behaviour
contributors can opt-in to a pipeline on push
git push <remote> -o ci.variable=RUN_PIPELINE=1
This variable can also be set globally on the repository, through the
web UI options Settings -> CI/CD -> Variables, though this is not
recommended. Upstream repo pushes to branches will run CI.
The use of containers has changed in this update, with only the upstream
repo creating containers, in order to avoid consuming contributors'
limited storage quotas. A fork with existing container images may delete
them. Containers will be rebuilt upstream when pushing commits with CI
changes to the default branch. Any other scenario with CI changes will
simply install build pre-requisite packages in a throaway environment,
using the ci/buildenv/ scripts. These scripts may also be used on a
contributor's local machines.
With pipelines triggered by merge requests, it is also now possible to
workaround the inability of contributors to run pipelines if they have
run out of CI quota. A project member can trigger a pipeline from the
merge request, which will run in context of upstream, however, note
this should only be done after reviewing the code for any malicious
CI changes.
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
libvirt-derived repos recently changed the way how and when CI
containers are built and for that a different naming scheme was adopted
to differentiate between the 2. Update the integration pipeline config
to reflect this change.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This updates the FreeBSD 13 image to 13.1 which should fix the
symbol lookup errors seen in CI recently.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
After support for the sheepdog storage driver backend was removed we
don't need to install it any longer in the containers.
Regenerate the dockerfiles after:
https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/314
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Debian 10 reaches EOL in August of 2022.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Debian 10 will reach EOL in august of 2022 and thus libvirt will no
longer target it. Move CI jobs over to Debian-11.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
'lcitool' dropped alpine-314
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Apart from other changes this fixes failures with builds on FreeBSD.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This requires publishing the RPMs as artifacts from the regular
build job.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Since a fix for CVE-2022-24765 was released every git command is now
checked against the context repo in which it's supposed to run
resulting in a fatal error if the repo is owned by other user than the
one running the git command.
This means that in order to be able to do 'sudo make install', we have
to set the 'safe.directory' for the root user. This is because QEMU
runs 'git submodule update' automatically on 'make install'.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
We currently build cross-arch containers for all three Debian
releases (10, 11, Sid), and do libvirt builds covering each
arch. This is overkill in terms of the number of problems it
identifies. The most important aspect of cross arch builds is
to find problems with 32-bit builds and problems with big
endian builds.
With this in mind the cross arch jobs are altered as follows
- Debian 10
- build the containers by default
- build armv7 (32-bit) & s390x (big endian)
- other arch builds manual
- Debian 11 / Sid
- container builds all optional
- arch builds all optional
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
We currently build on Ubuntu 22.04 twice, for GCC and CLang
with santizers turned on. Moving the GCC santizers build
to 20.04 lets us cull one of the 22.04 jobs
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Currently we do native builds on all distros that are covered
by the support matrix. This reduces that such that we mostly
only run builds on the newest (ie bleeding edge non-released)
version and the oldest version. The effect is that cut out
builds on the newest release version. This is acceptable,
because that version is sandwiched between two versions we
do still test, so unlikely to have failures not already
identified by other jobs.
This has the effect of disabling:
- AlmaLinux 8 GCC - still has a CLang build
and CentOS 8 Stream also gives coverage
- Debian 11 - still has a Debian 10 and Sid
build
- Alpine 3.15 - still has a Alpine 3.14 and Edge
build
Ideally Fedora 35 would be disabled too, but we rely on that
for the integration tests.
The Ubuntu jobs will be handled in the next patch.
The containers are still built since this is cheap-ish.
The build jobs can also be triggered manually if desired.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This refreshes the containers bringing in new behaviour when
builds/containers are disabled.
Instead of deleting the job entirely, the job still exists
but is set to be a manual job. It won't affect the pipeline
result, but can be triggered by the developer if they wish
to test a specific scenario.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
It's a stable distro, so we expect all jobs to succeed.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The target is intentionally not added to the integration tests
at this time, because the corresponding VM template is not yet
available on the runner. A later patch will take care of that.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Notable changes:
* 'lcitool manifest' now generates GitLab CI rules spread
across a bunch of files;
* container images are built less frequently for the main
repository.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The MinGW-w64 project has effectively replaced the original
MinGW project, and distributions such as Fedora have been shipping
packages based on the former for years now.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Basically all files in the repository are already passing the
check, except for syntax-check.mk itself. Fix that, and stop
limiting the files on which the test is performed.
These changes have been generated by running
$ sed -Ei 's/[ '$'\t'']+\\$/ \\/g' $(git grep -El '[ '$'\t'']+\\$')
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
YAML anchors don't work with Shell condition structures, so we cannot
simply reference the QEMU build template YAML anchor conditionally and
hence have everything as part of a single job template.
Instead, we have to "subclass" the .integration_tests template and
inject the QEMU building bits explicitly.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This was heavily inspired by QEMU's upstream CI buildtest-template.yml.
Rather than referencing QEMU's template directly (which GitLab can do),
this patch resorts to hard-coding the build steps ourselves, solely
because there's no guarantee QEMU will keep either the template file
name or the template name from which the build steps were mostly copied
from.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
There's quite a lot happening in the .integration_tests template
already even without adding upstream QEMU build into the mix.
Let's break the template into more pieces which can then reference
in the .integration_tests template when putting all the pieces back
together using YAML anchors.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Future patches will do more code extraction from the existing template
using YAML anchors so it'd be better that the templates would live
separately from job definitions.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Using an updated toolchain means we're likely to catch more
issues.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Notable changes:
* use the most recent point release for FreeBSD 12.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Mostly the tests should fail cleanly, but sometimes the test might crash
or abort. In this case we'll need to know the stack traces in order to
debug the problem. Fortunately on the Cirrus CI macOS instances, the
crash reporting service is active and saving crashes to the directory
~/Library/Logs/DiagnosticReports/
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We don't have access to the 'testlog.txt' file, so we need meson to
print the failures for any broken tests directly.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Using the extended definition format is only necessary when
the defaults are not good enough, but that's not the case here.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
All build jobs for the target are explicitly disabled, so
there's no point in keeping the variables file around and we
can simply not mention it in the manifest at all.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
In this batch:
- dnsmasq is dropped as build dependency
- Alpine Edge rpcgen package collision fix
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
As of April 23 2022, Ubuntu 20.04 will be out for two years, which per
our platform support policy means we no longer have to support
Ubuntu 18.04.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
By default, stdout/stderr Avocado test log files do not have any file
extension which confuses GitLab's web UI to mangle the MIME type for
these and so the browser will never offer the option to open such file
from in a text editor rather than dowloading it.
Since GitLab sets a proper MIME for .txt and .log file extensions,
rename all Avocado log files without an extension to *.log . This pairs
nicely with the coredumpctl info file which we already name as
'coredumpctl.txt' because of this.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Some Red Hat-like distros have cores limited with a soft limit of 0
which means that neither a stack trace nor a core file will be
available. Since we want the stack trace we need to set the core limit
with systemd globally to unlimited/infinity.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Custom runners are private to a project, so naturally forks cannot run
any workloads on these. The integration test suite which requires
access to our custom runner is naturally disabled on forks and can be
enabled by setting LIBVIRT_CI_INTEGRATION=1.
The problem is that the current integration jobs definitions have tags
statically defined as 'redhat-vm-host'. If users are going to supply
their own private runners for their forks, they can define whatever
tags they want with it and so unless they add 'redhat-vm-host' to their
own runner's tags, the pipeline won't run.
To solve this, define the integration job tag using a variable. The
repo config will use the value defined in the job for the variable
while users can override the value easily on a project/pipeline level
thanks to GitLab's CI variable precedence [1].
[1] https://docs.gitlab.com/ee/ci/variables/#cicd-variable-precedence
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Right now the jobs have no rules so they will always be created in
a pipeline. If the user's fork has no runner configured, then the
jobs will never be able to execute and the pipeline will not finish.
Even on upstream, there might be times the runner has to be taken
offline for maint work, or unexpectedly fail. We need a quick way
to disable the integration tests if we decide we don't want to
have pipelines queued until the runner comes back online.
Both these problems can be addressed by requiring a environment
variable to be set
LIBVIRT_CI_INTEGRATION=1
This can be done in the GitLab repo CI settings for permanent
enablement. Alternatively it can be set for individual
scheduled jobs, or using a push option
git push -o ci.variable=LIBVIRT_CI_INTEGRATION=1
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
