Commit Graph

72 Commits

Author SHA1 Message Date
Andrea Bolognani
8e2b814ead ci: Call meson consistently
We should always pass --werror and display the contents of the
log file in case of failure.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2021-04-01 14:09:47 +02:00
Andrea Bolognani
552a501d55 ci: Drop FreeBSD 11 build
FreeBSD 12 was released in December 2018, so according to our
platform support policy we can now drop support for the previous
major release. It would be going EOL in September anyway.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
2021-03-23 13:41:11 +01:00
Andrea Bolognani
792843c05e ci: Drop prefix from Dockerfiles
Since the string "ci" is already contained in the path, it
seems unnecessary to include it into the filename too: in fact,
we only do that for Dockerfiles and not for files in ci/cirrus,
even though those are generated the very same way.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2021-03-22 11:40:57 +01:00
Andrea Bolognani
3ab9b399bc ci: Build on macOS 11 instead of macOS 10.15
macOS builder capacity on Cirrus CI is quite limited, and so we
can't afford to keep the old build job around after adding the
new one like we do for FreeBSD.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-02-16 11:35:44 +01:00
Andrea Bolognani
cb5defccb1 ci: Update package list on Cirrus CI
While pkgng on FreeBSD updates the package list automatically
when it's run, homebrew on macOS doesn't do the same thing, which
can result in stale packages being installed. Explicitly call
'brew update' before 'brew install' to avoid that scenario.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-02-16 11:35:38 +01:00
Andrea Bolognani
ee095c2312 ci: Build on FreeBSD 12.2
The FreeBSD 12.1 image on Cirrus CI is currently broken, but
that's okay because a FreeBSD 12.2 image is also available and
we'd rather build on the more up-to-date target anyway.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2021-02-15 12:11:28 +01:00
Andrea Bolognani
025ac65ada ci: Move ppc64le build from Debian sid to Debian 10
Debian sid is currently broken on ppc64le, so move the build to
Debian 10; do the opposite for the aarch64 and mips64el builds to
try and restore the 10/sid balance.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-02-12 16:35:15 +01:00
Andrea Bolognani
c56dac1c54 ci: Mark container build jobs as required/optional correctly
Whether a container build job is considered required depends on
whether the corresponding cross-build job exists, and in a few
cases the two got out of sync over time.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-02-12 16:35:09 +01:00
Andrea Bolognani
52a6cd5f9e ci: Shuffle cross-building jobs around
Keep them ordered by architecture, the same way the corresponding
container jobs are, to make it easier to jump between the two
sections and compare them.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-02-12 16:34:39 +01:00
Andrea Bolognani
65edb00539 gitlab: Fix build container dependencies for build jobs
A couple of these were wrong, resulting in the build job not
waiting for the correct container to be built before starting.

Fixes: 77296c807b

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2021-02-09 10:28:50 +01:00
Daniel P. Berrangé
e603efb6ec gitlab: force dwarf4 format for debuginfo in Fedora rawhide
Fedora 34 rawhide has pulled in a new GCC 11 build which now
defaults to dwarf5 format. This format is not compatible with
the pdwtags program used in our test suite to validate the
RPC files.

We have no need for debuginfo in CI except for pdwtags,
so the simplest short term fix is to force the older dwarf
version in the hope that a fixed dwarves release will
arrive before Fedora 34 is released, or GCC 11 becomes more
widespread. Eventually we might need to figure out a way to
probe for compatibility but for now, we'll hope that any
distro with GCC 11 will be able to have a fixed dwarves too.

https://bugzilla.redhat.com/show_bug.cgi?id=1919965
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-26 12:33:31 +00:00
Erik Skultety
a79ac43f04 ci: Drop openSUSE Leap 15.1
Leap 15.1 will reach EOL on January 31st 2021, so we should not test on
it during the current development cycle ending on March 1st.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-20 11:20:26 +01:00
Erik Skultety
3114cb4bfd ci: Add openSUSE Leap 15.2
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-20 11:20:26 +01:00
Erik Skultety
c86aedf60f gitlab-ci.yml: Add an explicit env variable to mark a job as broken
Thanks to the 'rules' syntax, this will translate to
'allow_failure:true' and let the job fail but will not affect the rest
of the pipeline.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2021-01-15 10:44:39 +01:00
Erik Skultety
db44db77e7 gitlab-ci.yml: Convert only/except to the rules syntax
'rules' syntax replaces the only/except syntax with which it is
mutually exclusive. In some cases the 'rules' syntax is more readable
than the 'only/except' equivalent, in some cases it is not.
The idea behind this conversion is to introduce an explicit env variable
controlling the 'allow_failure' attribute which would then be attached
to a broken build job which would in turn result in a soft failure.
Such behaviour is not possible to achieve with the older 'only/except'
syntax.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2021-01-15 10:44:33 +01:00
Erik Skultety
b974873887 gitlab-ci.yml: Replace template anchors with extends
'extends' is slightly more readable and definitely more flexible in
terms of allowing includes of templates.
The main reason for this patch though is that the next patch converts
the 'only/except' syntax to the new (preferable) 'rules' syntax.
Variable anchors are still kept intact because the use case there is
different from regular template anchors.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2021-01-15 10:44:12 +01:00
Daniel P. Berrangé
d855f12945 ci: refresh cirrus vars files
In this refresh the PIP variable is renamed to PIP3 and the
PYPI_PKGS variable disappears since we (currently) don't have
any need for it.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-11 16:14:12 +00:00
Daniel P. Berrangé
855399e382 ci: rename the cirrus vars file to remove the "libvirt-" prefix
The "libvirt-" prefix was removed from hostnames in libvirt-ci.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-11 16:14:06 +00:00
Daniel P. Berrangé
4e750e932a gitlab: replace "libvirt-" prefix with "ci-" in dockerfiles
This makes the dockerfile name match the output container name

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-12-08 16:04:03 +00:00
Daniel P. Berrangé
0fbb495e75 gitlab: re-generate container images from lcitool
This introduces Fedora 33 and removes some redundant packages.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-12-08 16:03:54 +00:00
Pavel Hrdina
d9096ce998 gitlab-ci: add coverity job
Introduce new job to make a coverity build and upload coverity data to
scan.coverity.com where the analysis is then executed.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-11-19 14:38:13 +01:00
Daniel P. Berrangé
d63764f8bc gitlab: run unit tests in i686 cross build
In most cross builds we can't run tests since we can't assume QEMU user
mode emulators are loaded. i686 is special though because x86_64 can run
i686 natively, so we should run unit tests there.

The key benefit is that this gives us 32-bit unit test coverage in CI.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-11-11 11:13:25 +00:00
Andrea Bolognani
0f4dbd6160 ci: Start building RPMs
We lost this coverage during the move from CentOS CI to GitLab CI,
and it's high time we brought it back.

Building RPMs is currently skipped for

  * openSUSE, which is not supported by our spec file;

  * clang builds, where rpmbuild fails with

    meson.build:1:0: ERROR: Unable to determine dynamic linker

  * targets where we install Meson from PyPi, because that doesn't
    bring in the necessary RPM macros.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-09 10:14:42 +02:00
Andrea Bolognani
71c97400a3 ci: Allow skipping dist
Instead of having an ad-hoc build script for CentOS 7, follow the
pattern established in other repositories under the libvirt group
and allow selectively disabling that specific part of the build.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-09 10:14:42 +02:00
Daniel P. Berrangé
417e556147 Add FreeBSD 11.4 CI job on Cirrus
The previous 11.3 image provided by Cirrus did not boot, but they have
now provided a working 11.4 image

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-09-10 13:11:46 +01:00
Andrea Bolognani
9b5622d31f ci: Temporarily enable verbose mode for cirrus-run
We've hit issues with GitLab CI jobs reporting a failure despite
the corresponding Cirrus CI job finishing successfully: this is
apparently caused by the underlying VM being rescheduled.

A workaround for this issue has been implemented as of

  5299874bbb

which will be included in the upcoming 0.3.0 release; however, in
order to validate that this workaround is effective it would be
useful to have more data.

Based on the conversation in

  https://github.com/sio/cirrus-run/issues/4

enabling verbose mode allows to collect this data while not having
any impact on performance, so let's enable it temporarily and then
disable it again once cirrus-run 0.3.0 is out.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-08-04 13:29:34 +02:00
Andrea Bolognani
d8f13661e0 ci: Show build log for Cirrus CI jobs
This feature has been implemented as of

  b9b2278f1d

and will be included in the upcoming 0.3.0 release.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-08-04 13:29:34 +02:00
Ján Tomko
77296c807b ci: specify containter dependencies for all the jobs
Write down the container job needed for each build job in the
'needs' keyword to allow greater parallelism.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 09:46:20 +02:00
Pavel Hrdina
644d9ddb1a meson: .gitlab-ci.yml: switch armv7l build to Debian 10
On Debian Sid there are broken headers for cross compilation. There is
no <asm/kvm.h> header unconditionally included by <linux/kvm.h>.

With autotools the build works correctly because the header detection
tries to compile simple code that includes the header but with Meson
the detection works differently. We check <linux/kvm.h> which is present
so it should be safe to include it.

We could workaround it in libvirt by checking for presence of
<asm/kvm.h> as well but I don't believe that's a correct approach.
IMHO it should be fixed in Debian.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 09:27:09 +02:00
Pavel Hrdina
4be700fed9 meson: update .gitlab-ci.yml file
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 09:27:09 +02:00
Pavel Hrdina
9aea94b385 meson: ci: increase git clone depth to 1000
Having limit set to 100 is in most cases perfect but sometimes there can
be a larger series that will have more than 100 patches and it will make
the check-dco job fail.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 09:25:54 +02:00
Ján Tomko
a82e1d4e57 ci: run container jobs right away
The jobs building containers do not depend on anything.
Make this explicit by including the 'needs' keyword in case
we need to shuffle around the stages in the future.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 08:36:15 +02:00
Ján Tomko
09464e6929 ci: run Cirrus-based builds sooner
Jobs using cirrus-run don't need to wait for any jobs from
the container stage. Run them as soon as possible.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 08:35:22 +02:00
Ján Tomko
14ef032e20 ci: add clang jobs
Run the build with clang on Fedora Rawhide, Debian 10 and CentOS 8.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 08:35:22 +02:00
Ján Tomko
31c69e188c ci: move sanity_checks after other stages
Be less hostile to developers and run the build jobs even
if the dco-check failed. That way they can test their own
"private" branches without a sign-off.

Also specify empty `needs` for it, since it does not depend
on any of the jobs in the container phase and can be run
right away.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-08-03 07:23:45 +02:00
Andrea Bolognani
8a4f807536 ci: Use correct name for armv7l architecture
Turns out 'ldebian' was not a typo, but rather the result of
mistakenly shifting the dash in 'armv7l-debian' one spot to the
left.

Fixes: 821100e812
Reported-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-07-29 10:55:01 +02:00
Andrea Bolognani
821100e812 ci: Fix typo 'ldebian'
Fixes: 95abbdc432
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-07-29 10:21:24 +02:00
Andrea Bolognani
fc832369d0 ci: Drop Debian 9 jobs
The existing cross-compilation jobs are carefully redistributed
among Debian 10 and Debian sid to ensure we don't use the latter
for aarch64, mipsel or mips64el, since those architectures are
currently broken.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-07-15 12:06:26 +02:00
Andrea Bolognani
fb91290131 cirrus: Generate jobs dynamically
Instead of having static job definitions for FreeBSD and macOS,
use a generic template for both and fill in the details that are
actually different, such as the list of packages to install, in
the GitLab CI job, right before calling cirrus-run.

The target-specific information are provided by lcitool, so that
keeping them up to date is just a matter of running the refresh
script when necessary.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-07-03 11:00:22 +02:00
Andrea Bolognani
4268e18753 ci: Run all jobs, for all branches, all the time
After recent changes (increasing the parallelism of the pipeline
by reducing the number of stages, introducing FreeBSD builds that
take longer than any other job), the difference between running
the full pipeline or a reduced one has basically disappeared: in
both cases, the completion time is around 25-35 minutes depending
on whether containers need to be rebuilt and how many shared
runners are available.

Reduce the complexity of our .gitlab-ci.yml and make things
simpler for contributors by simply always running all jobs.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-29 17:43:28 +02:00
Jonathon Jongsma
d66181c84e ci: Use openSUSE image for code style checking
CentOS does not have the cppi package, so some code style checks are
skipped. Switch to a openSUSE image to do the code style checks.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-26 09:25:45 +02:00
Andrea Bolognani
60a7c78348 ci: Drop mips container for Debian sid
The build job for this container has been failing every single
time, and as it turns out the explanation for that is very simple:
Debian is just not going to support the mips architecture going
forward.

Reported-by: Pino Toscano <ptoscano@redhat.com>
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-23 12:14:00 +02:00
Andrea Bolognani
63d08bec0b ci: Swap mipsel and ppc64le builds
Debian sid is currently broken on mipsel, so use Debian 10 for
that architecture; at the same time, move the ppc64le build from
Debian 10 to Debian sid to keep things balanced.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-11 20:08:32 +02:00
Andrea Bolognani
95abbdc432 ci: Use GitLab container registry
Instead of using pre-built containers hosted on Quay, build
containers as part of the GitLab CI pipeline and upload them to the
GitLab container registry for later use.

This will not significantly slow down builds, because containers are
only rebuilt when the corresponding Dockerfile has been modified.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-11 19:06:51 +02:00
Andrea Bolognani
7ef1324284 ci: Use variables to build image names
This removes a lot of repetition and makes the configuration much
easier to read.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-11 19:06:49 +02:00
Andrea Bolognani
0d009ca646 ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.

This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.

Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.

Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].

[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 19:33:09 +02:00
Andrea Bolognani
6190c14151 ci: Enable Cirrus CI integration
We use cirrus-run to trigger Cirrus CI jobs from GitLab CI jobs,
making it possible to extend our platform coverage to include
FreeBSD without having to maintain our own runners; additionally,
we'll be able to ditch Travis CI and, since results for Cirrus CI
jobs are reflected back to the GitLab CI jobs that triggered them,
we will be able to get all information from a single dashboard.

The FreeBSD and macOS job definitions can be improved further: for
example, we will want to enable caching to speed up builds, and
ultimately we should figure out a way to generate at least part of
them, notably the list of packages to be installed, using lcitool.
All of that will happen in later patches: for now, this is good
enough to start using Cirrus CI.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-10 10:30:56 +02:00
Daniel P. Berrangé
3cdda5d5e8 gitlab: fix potfile job for new location
Since we're storing the libvirt.pot file in git now, we should pull from
the source dir, not the build dir.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-04 18:53:21 +01:00
Daniel P. Berrangé
019b71dead gitlab: move some jobs onto CentOS 8
So that we don't have to chase frequent Fedora releases, move the
non-build related jobs onto the long life CentOS 8 distro.

Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-06 13:16:58 +01:00
Daniel P. Berrangé
57e9f3a7e0 gitlab: update list of distros to use latest versions
Replace Fedora 30 with Fedora 32
Replace Ubuntu 16.04 with Ubuntu 20.04

Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-04 17:20:08 +01:00