The default expiry time is 30 days. Since the RPM artifacts coming from
the previous pipeline stages are set to expire in 1 day we can set the
failed integration job log artifacts to the same value. The sentiment
here is that if an integration job legitimately failed (i.e. not with
an infrastructure failure) unless it was fixed in the meantime it will
fail the next day with the scheduled pipeline again, meaning, that even
if the older log artifacts are removed, they'll be immediately
replaced with fresh ones.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Avocado 99.0 causes the TCK test suite to fail with the nwfilter tests
(which is another Bash framework underneath). Until the culprit is
identified and fixed in Avocado, let's lock the version to 98.0 which
worked with the test suite just fine.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Two notable changes:
* the macOS platform has switched from x86_64 to aarch64
* if a new pipeline starts before a previous one finishes,
jobs marked 'interruptible: true' will be auto-cancelled
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Latest versions of Avocado create 'by-status' symlink shortcuts to test
results, IOW:
# this is the main test results directory containing all data
$ ls <path>/avocado/job-results/latest/test-results/
01-scripts_networks_050-transient-lifecycle.t
02-scripts_networks_051-transient-autostart.t
...
22-scripts_networks_400-guest-bandwidth.t
by_status/
# list only the failed tests
$ ls -l <path>/avocado/job-results/latest/test-results/by-status/FAIL
19-scripts_networks_360-guest-network-vepa.t ->
<path>/avocado/job-results/latest/test-results/19-scripts_networks_360-guest-network-vepa.t
Therefore, let's bundle only the failed ones, it's going to make the
log artifacts more obvious when looking for libvirt errors.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Don't create an avocado directory in the resulting log artifacts
if Avocado didn't even run (e.g. libvirt errored out on service
restart).
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
All 'script' blocks are defined as 'set -e' and so a single failed
return value means we won't collect some of the logs. Because of
the nature of the original job's failure some of the log sources
might not be available, but that's fine, however, the gitlab
after_script job cannot finish prematurely.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
It could be quite confusing looking at the job log artifacts and having
an empty coredump log in there, IOW it doesn't really give much
confidence that the reporting mechanism actually works.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
It's a directory, so -d should be used with 'test'.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Both log filters and log outputs expect string values, however, augeas
apparently requires an extra level of quotes apart from the ones we
pass via shell (see comment [1]) to work properly, otherwise augeas
ignores the value and returns 0.
Without this fix we don't set libvirt's log level to debug, we don't
set logging to a file and hence we don't include the logs in CI
artifacts in case the test suite fails.
[1] https://github.com/hercules-team/augeas/issues/301#issuecomment-143699880
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
It was missing from the set. While at it, order the daemon set
alphabetically.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
After addition of the new libvirt-client-qemu sub-package which is using
python bindings (thus creating a circular dependency between the libvirt
and libvirt-python projects) the integration jobs fail with:
Error:
Problem: conflicting requests
- nothing provides python3-libvirt >= 8.9.0-1.el9 needed by libvirt-client-qemu-8.9.0-1.el9.x86_64
The libvirt-python project now provides the RPMs in artifacts:
https://gitlab.com/libvirt/libvirt-python/-/merge_requests/96
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This refresh switches the CI for contributors to be triggered by merge
requests. Pushing to a branch in a fork will no longer run CI pipelines,
in order to avoid consuming CI minutes. To regain the original behaviour
contributors can opt-in to a pipeline on push
git push <remote> -o ci.variable=RUN_PIPELINE=1
This variable can also be set globally on the repository, through the
web UI options Settings -> CI/CD -> Variables, though this is not
recommended. Upstream repo pushes to branches will run CI.
The use of containers has changed in this update, with only the upstream
repo creating containers, in order to avoid consuming contributors'
limited storage quotas. A fork with existing container images may delete
them. Containers will be rebuilt upstream when pushing commits with CI
changes to the default branch. Any other scenario with CI changes will
simply install build pre-requisite packages in a throaway environment,
using the ci/buildenv/ scripts. These scripts may also be used on a
contributor's local machines.
With pipelines triggered by merge requests, it is also now possible to
workaround the inability of contributors to run pipelines if they have
run out of CI quota. A project member can trigger a pipeline from the
merge request, which will run in context of upstream, however, note
this should only be done after reviewing the code for any malicious
CI changes.
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
libvirt-derived repos recently changed the way how and when CI
containers are built and for that a different naming scheme was adopted
to differentiate between the 2. Update the integration pipeline config
to reflect this change.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
This updates the FreeBSD 13 image to 13.1 which should fix the
symbol lookup errors seen in CI recently.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
After support for the sheepdog storage driver backend was removed we
don't need to install it any longer in the containers.
Regenerate the dockerfiles after:
https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/314
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
Debian 10 reaches EOL in August of 2022.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Debian 10 will reach EOL in august of 2022 and thus libvirt will no
longer target it. Move CI jobs over to Debian-11.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
'lcitool' dropped alpine-314
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Apart from other changes this fixes failures with builds on FreeBSD.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This requires publishing the RPMs as artifacts from the regular
build job.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Since a fix for CVE-2022-24765 was released every git command is now
checked against the context repo in which it's supposed to run
resulting in a fatal error if the repo is owned by other user than the
one running the git command.
This means that in order to be able to do 'sudo make install', we have
to set the 'safe.directory' for the root user. This is because QEMU
runs 'git submodule update' automatically on 'make install'.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
We currently build cross-arch containers for all three Debian
releases (10, 11, Sid), and do libvirt builds covering each
arch. This is overkill in terms of the number of problems it
identifies. The most important aspect of cross arch builds is
to find problems with 32-bit builds and problems with big
endian builds.
With this in mind the cross arch jobs are altered as follows
- Debian 10
- build the containers by default
- build armv7 (32-bit) & s390x (big endian)
- other arch builds manual
- Debian 11 / Sid
- container builds all optional
- arch builds all optional
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
We currently build on Ubuntu 22.04 twice, for GCC and CLang
with santizers turned on. Moving the GCC santizers build
to 20.04 lets us cull one of the 22.04 jobs
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Currently we do native builds on all distros that are covered
by the support matrix. This reduces that such that we mostly
only run builds on the newest (ie bleeding edge non-released)
version and the oldest version. The effect is that cut out
builds on the newest release version. This is acceptable,
because that version is sandwiched between two versions we
do still test, so unlikely to have failures not already
identified by other jobs.
This has the effect of disabling:
- AlmaLinux 8 GCC - still has a CLang build
and CentOS 8 Stream also gives coverage
- Debian 11 - still has a Debian 10 and Sid
build
- Alpine 3.15 - still has a Alpine 3.14 and Edge
build
Ideally Fedora 35 would be disabled too, but we rely on that
for the integration tests.
The Ubuntu jobs will be handled in the next patch.
The containers are still built since this is cheap-ish.
The build jobs can also be triggered manually if desired.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This refreshes the containers bringing in new behaviour when
builds/containers are disabled.
Instead of deleting the job entirely, the job still exists
but is set to be a manual job. It won't affect the pipeline
result, but can be triggered by the developer if they wish
to test a specific scenario.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
It's a stable distro, so we expect all jobs to succeed.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The target is intentionally not added to the integration tests
at this time, because the corresponding VM template is not yet
available on the runner. A later patch will take care of that.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Notable changes:
* 'lcitool manifest' now generates GitLab CI rules spread
across a bunch of files;
* container images are built less frequently for the main
repository.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The MinGW-w64 project has effectively replaced the original
MinGW project, and distributions such as Fedora have been shipping
packages based on the former for years now.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Basically all files in the repository are already passing the
check, except for syntax-check.mk itself. Fix that, and stop
limiting the files on which the test is performed.
These changes have been generated by running
$ sed -Ei 's/[ '$'\t'']+\\$/ \\/g' $(git grep -El '[ '$'\t'']+\\$')
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
YAML anchors don't work with Shell condition structures, so we cannot
simply reference the QEMU build template YAML anchor conditionally and
hence have everything as part of a single job template.
Instead, we have to "subclass" the .integration_tests template and
inject the QEMU building bits explicitly.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
This was heavily inspired by QEMU's upstream CI buildtest-template.yml.
Rather than referencing QEMU's template directly (which GitLab can do),
this patch resorts to hard-coding the build steps ourselves, solely
because there's no guarantee QEMU will keep either the template file
name or the template name from which the build steps were mostly copied
from.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
There's quite a lot happening in the .integration_tests template
already even without adding upstream QEMU build into the mix.
Let's break the template into more pieces which can then reference
in the .integration_tests template when putting all the pieces back
together using YAML anchors.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Future patches will do more code extraction from the existing template
using YAML anchors so it'd be better that the templates would live
separately from job definitions.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Using an updated toolchain means we're likely to catch more
issues.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Notable changes:
* use the most recent point release for FreeBSD 12.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Mostly the tests should fail cleanly, but sometimes the test might crash
or abort. In this case we'll need to know the stack traces in order to
debug the problem. Fortunately on the Cirrus CI macOS instances, the
crash reporting service is active and saving crashes to the directory
~/Library/Logs/DiagnosticReports/
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We don't have access to the 'testlog.txt' file, so we need meson to
print the failures for any broken tests directly.
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Using the extended definition format is only necessary when
the defaults are not good enough, but that's not the case here.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
All build jobs for the target are explicitly disabled, so
there's no point in keeping the variables file around and we
can simply not mention it in the manifest at all.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
In this batch:
- dnsmasq is dropped as build dependency
- Alpine Edge rpcgen package collision fix
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
As of April 23 2022, Ubuntu 20.04 will be out for two years, which per
our platform support policy means we no longer have to support
Ubuntu 18.04.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
By default, stdout/stderr Avocado test log files do not have any file
extension which confuses GitLab's web UI to mangle the MIME type for
these and so the browser will never offer the option to open such file
from in a text editor rather than dowloading it.
Since GitLab sets a proper MIME for .txt and .log file extensions,
rename all Avocado log files without an extension to *.log . This pairs
nicely with the coredumpctl info file which we already name as
'coredumpctl.txt' because of this.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Some Red Hat-like distros have cores limited with a soft limit of 0
which means that neither a stack trace nor a core file will be
available. Since we want the stack trace we need to set the core limit
with systemd globally to unlimited/infinity.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Custom runners are private to a project, so naturally forks cannot run
any workloads on these. The integration test suite which requires
access to our custom runner is naturally disabled on forks and can be
enabled by setting LIBVIRT_CI_INTEGRATION=1.
The problem is that the current integration jobs definitions have tags
statically defined as 'redhat-vm-host'. If users are going to supply
their own private runners for their forks, they can define whatever
tags they want with it and so unless they add 'redhat-vm-host' to their
own runner's tags, the pipeline won't run.
To solve this, define the integration job tag using a variable. The
repo config will use the value defined in the job for the variable
while users can override the value easily on a project/pipeline level
thanks to GitLab's CI variable precedence [1].
[1] https://docs.gitlab.com/ee/ci/variables/#cicd-variable-precedence
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Right now the jobs have no rules so they will always be created in
a pipeline. If the user's fork has no runner configured, then the
jobs will never be able to execute and the pipeline will not finish.
Even on upstream, there might be times the runner has to be taken
offline for maint work, or unexpectedly fail. We need a quick way
to disable the integration tests if we decide we don't want to
have pipelines queued until the runner comes back online.
Both these problems can be addressed by requiring a environment
variable to be set
LIBVIRT_CI_INTEGRATION=1
This can be done in the GitLab repo CI settings for permanent
enablement. Alternatively it can be set for individual
scheduled jobs, or using a push option
git push -o ci.variable=LIBVIRT_CI_INTEGRATION=1
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Although we split out jobs across many files, the template / job
namespace is global, so we should use something more specific
than '.tests' as the template name.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This stage will download build artifacts from both the libvirt and
libvirt-perl (multi-project CI) builds, install all them on the custom
runners and configures libvirt debug logging on the runners prior to
executing the actual test suite. In case of a failure, libvirt and
Avocado logs will be saved and published as pipeline artifacts.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We're already building libvirt in the containers already, if we publish
the build in form of, say, RPMs, later stages of the pipeline can
consume the RPMs instead of re-building libvirt from scratch.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Regenerate with lcitool as of:
commit f83b916d5efa4bd33fbf4b7ea41bf6d535cc63fb
Author: Alex Bennée <alex.bennee@linaro.org>
Date: Fri Feb 11 09:39:30 2022 +0000
mappings: skip multipath-tools for cross Debian
This package is both a mix of library files, headers and native
binaries so cannot be installed in a cross environment. For now skip
it for cross targets.
See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1005323
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Upstream lcitool suggests that as a solution to 'centos-8' being
removed.
Move also the website and other jobs to depend on
'x86_64-almalinux-8-container'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Re-generate our CI infra with 'lcitool' as of:
commit b346752e98bd12395233ebba8c9312e08212b639 (HEAD)
Author: Erik Skultety <eskultet@redhat.com>
Date: Tue Feb 1 10:48:53 2022 +0100
tests: Replace CentOS 8 with AlmaLinux 8 in test scenarios
Switch the test target before actually dropping CentOS 8.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
This is prior to upstream lcitool dropping 'centos-8' support to
minimize the differences.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The meson version provided by the package managing system satisfies our
minimum requirement.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Apart from some churn, the important is the removal of 'netcf-devel'
from the fedora rawhide container.
Update to state as of 174fe4999204afcae (libvirt-ci).
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Both of the current mingw jobs are marked as 'allow_failure' because
they are running against Fedora rawhide which is an unstable distro.
We need at least one mingw job to be gating to more reliably detect
problems.
This introduces dockerfiles for both mingw variants on Fedora 35
and sets the mingw64 build to run on Fedora 34, and mingw32 on
Fedora rawhide.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
- The Cirrus CI variables are now sorted
- The dockerfiles update commands changed for some distros
- Meson in CentOS is now new enough to use
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This removes the libnetcf-dev package from Debian Sid, as it is no
longer available in that distro stream.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The non-released distros have reasonably frequent package installation
failures that can last for days at a time. This makes them unsuitable
for use as gating CI jobs.
This ensures all of the jobs in Debian Sid, Fedora Rawhide, openSUSE
Tumbleweed and FreeBSD Current are marked "allow-failure: true".
This means the jobs still run, but any failure will not be considered
fatal to the pipeline.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The layering of the cross containers is fixed to move arch specific
ccache setup out of the common base layer.
A missing Cirrus CI variable substitution is added, though this is
irrelevant given libvirt's package list.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The armv7l and ppc64le cross-builds as well as the Clang build
are adopted from Debian 10, while the mips64el build is adopted
from Debian sid. As always, the way jobs are distributed across
Debian versions is fairly arbitrary.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The value 3 is the length of the "ci-" prefix, which is present
in the items returned by get_registry_images() but not in those
returned by get_dockerfiles().
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
These were removed along with the outdated information on how
to regenerate the Dockerfiles contained in the repository, but
this part is still relevant.
Reverts: 30856d2865 (partially)
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We now use lcitool's manifest feature to generate files. The logic
for checking for stale containers in the registry, however, is still
relevant so that is propagated to a standalone command.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
We now use lcitool's manifest feature to generate files.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This uses the command "lcitool manifest ci/manifest.yml" to re-generate
all existing dockerfiles and gitlab CI config.
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
lcitool now uses the term "target" instead of "host" to refer to
the various operating systems it supports, and we need to adapt
our helper script so that it works with the new command line
interface.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
All the python packages got renamed from py37- to py38-
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The base OS image might include outdated contents, and we don't
want to get spurious failures caused by bugs that have already been
fixed in the respective packages.
This is particularly important on macOS, because 'brew install foo'
will fail if 'foo' is already installed but outdated: upgrading all
packages first ensures we never run into this scenario.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
In the distros using RPMs, we build libvirt once as a side effect
of running "ninja dist", and once via rpmbuild.
In addition "ninja dist" will run all tests including the "syntax-check"
suite, despite use having a separate "codestyle" job for for that.
There is no way to pass "--no-suite" when creating the dist, but if we
switch to invoking "meson dist", we can skip the build+test part
entirely using "--no-tests".
When doing this we then run explicit "meson compile" and "meson test"
commands for the distros that don't build the RPMs, and in the latter
case we can now skip the "syntax-check" suite.
The RPM builds already skipped the "syntax-check" suite.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The "dist" and "test" targets in ninja end up calling back into
the equivalent meson commands. The meson commands support various
arguments that are not accessible when invoked via ninja, so it
is preferrable to use meson directly.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
When CI jobs fail on the test suite, we don't see much helpful
information by default:
stderr:
TEST: bhyvexml2argvtest
......!!.............!......!........... 40
........................!...... 71 FAIL
Some tests failed. Run them using:
VIR_TEST_DEBUG=1 VIR_TEST_RANGE=7-8,22,29,65
/tmp/cirrus-ci-build/build/meson-private/dist-build/tests/bhyvexml2argvtest
Following the instructions to re-run the test with VIR_TEST_DEBUG=1 is
quite unfriendly when we could have had that set for CI already.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Notable changes:
* the CentOS Stream 8 container is now using a proper base
image instead of starting from a CentOS 8 image and then
adding the CentOS Stream 8 repositories on top;
* distributions that have a perl-base package are now using
that one instead of the regular perl package, which
contains a bunch of features we don't need, resulting in
smaller containers.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The names have been recently changed in libvirt-ci to be more
accurate, so we should follow along.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Notable changes:
* cross-building container images are smaller because they
no longer include the native compilers;
* ccache is enabled for clang builds.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Unless you create such an commit, cirrus-ci.com will not pick up the
github project and cirrus-run will fail.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Notable changes:
* HAL is no longer installed on FreeBSD;
* the native version of libwsman is no longer installed in
containers intended for cross-compilation;
* Meson 0.55 rather than 0.54 is requested when installing
it from PyPI;
* GNU sed and GNU grep are installed explicitly everywhere.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
FreeBSD 12 was released in December 2018, so according to our
platform support policy we can now drop support for the previous
major release. It would be going EOL in September anyway.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Since the string "ci" is already contained in the path, it
seems unnecessary to include it into the filename too: in fact,
we only do that for Dockerfiles and not for files in ci/cirrus,
even though those are generated the very same way.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
As documented at [1], the common practice with respect to private
attributes/methods naming is to prefix them with an underscore.
[1] https://docs.python.org/3/tutorial/classes.html#private-variables
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This function checks whether there are any stale Docker images in the
registry that can be purged. Since we're pulling available container
images from our GitLab registry with the 'list-images' action, it
could happen that we'd list old (already unsupported) images and make
them available for the user to consume and run a build in them.
Naturally, the build will most likely fail leaving the user confused.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
The corresponding Bash script is dropped.
After this patch's rewrite, the Makefile's original image listing
target remains intact only to notify the user to use the Python helper
instead.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
With the gradual rewrite of the Makefile to the 'helper' script will
require helper functions that would better live in a separate util
module.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Offer an option to silence all output to stdout coming out of the
dockerfiles/varsfiles generating code.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This help formatter class reports the defaults we use for options
taking an argument.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
We now wrap all its important functionality with the much more
user-friendly ci/helper script, and the long term plan is for
the Makefile to disappear completely.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The functionality is now available in the ci/helper script.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This simply calls the underlying Makefile target, but allows
additional arguments to be specified in a more convenient and
discoverable way.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This simply calls the underlying Makefile target, but allows
additional arguments to be specified in a more convenient and
discoverable way.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This simply calls the underlying Makefile target, but allows
additional arguments to be specified in a more convenient and
discoverable way.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This simply calls the underlying Makefile target.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This provides the same functionality as the two refresh scripts
that are currently in the repository, with the following
advantages:
* all files are refreshed with a single command;
* if lcitool is present in the user's $PATH, it will be
discovered and used automatically;
* some output is produced, so the user can follow along with
the progress of the operation.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This is intended to be perform a number of CI-related operations
that are currently implemented in various different scripts
written in various different programming languages.
Eventually, all existing functionality will be reimplemented in
Python and made available through this single entry point; for
now, let's start with a very basic skeleton.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The target was renamed when moving to Meson, but the help text
was not updated accordingly.
Fixes: 1a0af38ae7
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Using locally built images is a useful feature; our commentaries even
mention overriding them may be useful in some scenarios. Expose the
variables in the help to let users know they can use the feature.
Formatting would definitely break, so this patch adds more spacing for
proper alignment.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
In commit 321293e2 I dropped the prepare.sh script, but forgot to
remove the corresponding variable from the Makefile.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
macOS builder capacity on Cirrus CI is quite limited, and so we
can't afford to keep the old build job around after adding the
new one like we do for FreeBSD.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
While pkgng on FreeBSD updates the package list automatically
when it's run, homebrew on macOS doesn't do the same thing, which
can result in stale packages being installed. Explicitly call
'brew update' before 'brew install' to avoid that scenario.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
More often than not I find myself debugging in the containers which
means that I need to have root inside, but without manually tweaking
the Makefile each time the execution would simply fail thanks to the
uid/gid mapping we do. What if we expose the CI_USER_LOGIN variable, so
that when needed, the root can be simply passed with this variable and
voila - you have a root shell inside the container with CWD=~root.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
The purpose of this script was to prepare a customized environment in
the container, but was actually never used and it required the usage of
sudo to switch the environment from root's context to a regular user's
one.
The thing is that once someone needs a custom script they would very
likely to debug something and would also benefit from root privileges
in general, so the usage of 'sudo' in such case was a bit cumbersome.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
The prepare.sh script isn't currently used and forces us to make use
of sudo to switch the user inside the container from root to $USER
which created a problem on our Debian Slim-based containers which don't
have the 'sudo' package installed.
This patch removes the sudo invocation and instead runs the CMD
directly with podman.
Summary of the changes:
- move the corresponding env variables which we need to be set in the
environment from the sudo invocation to the podman invocation
- pass --workdir to podman to retain the original behaviour we had with
sudo spawning a login shell.
- MESON_OPTS env variable doesn't need to propagated to the execution
environment anymore (like we had to do with sudo), because it's
defined in the Dockerfile
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
This is necessary for the follow up patch, because the default
entrypoint for a Dockerfile is exec.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
dtrace is now also installed when cross-building.
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Leap 15.1 will reach EOL on January 31st 2021, so we should not test on
it during the current development cycle ending on March 1st.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
In this refresh CentOS 7 now uses docker.io registry and the PowerTools
repo name regression was fixed for CentOS Stream this time.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
In this refresh the PIP variable is renamed to PIP3 and the
PYPI_PKGS variable disappears since we (currently) don't have
any need for it.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The "libvirt-" prefix was removed from hostnames in libvirt-ci.
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Contains changes utilizing "nosync" and "eatmydata" for speedup as well
as fixes for CentOS-8 repoid regression.
ci-commit: b098ec6631a85880f818f2dd25c437d509e53680
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
This makes the dockerfile name match the output container name
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This introduces Fedora 33 and removes some redundant packages.
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
There's no need to have different CI process between macOS and FreeBSD
as test suite has been fixed on macOS.
Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Add meson required bits to the ci logic in the repo to be able to run
a meson build in a container.
This patch also drops several environment variables we don't need with
meson anymore.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Introduce new job to make a coverity build and upload coverity data to
scan.coverity.com where the analysis is then executed.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The corresponding lcitool change is
00d736ea99
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The previous 11.3 image provided by Cirrus did not boot, but they have
now provided a working 11.4 image
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Update the remaining 'make check' references after the
switch to meson/ninja.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
