Commit Graph

102 Commits

Author SHA1 Message Date
Erik Skultety
07103a0cae ci: Drop the CI_PREPARE_SCRIPT variable
In commit 321293e2 I dropped the prepare.sh script, but forgot to
remove the corresponding variable from the Makefile.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2021-02-17 11:05:42 +01:00
Andrea Bolognani
cf325da1f8 ci: Remove Debian 9 special case
We no longer target this platform.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2021-02-17 09:53:32 +01:00
Andrea Bolognani
3ab9b399bc ci: Build on macOS 11 instead of macOS 10.15
macOS builder capacity on Cirrus CI is quite limited, and so we
can't afford to keep the old build job around after adding the
new one like we do for FreeBSD.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-02-16 11:35:44 +01:00
Andrea Bolognani
cb5defccb1 ci: Update package list on Cirrus CI
While pkgng on FreeBSD updates the package list automatically
when it's run, homebrew on macOS doesn't do the same thing, which
can result in stale packages being installed. Explicitly call
'brew update' before 'brew install' to avoid that scenario.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2021-02-16 11:35:38 +01:00
Andrea Bolognani
2d92970d8f ci: Refresh Dockerfiles
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2021-02-15 12:11:23 +01:00
Erik Skultety
f28a652a32 ci: Makefile: Expose the CI_USER_LOGIN variable for users to use
More often than not I find myself debugging in the containers which
means that I need to have root inside, but without manually tweaking
the Makefile each time the execution would simply fail thanks to the
uid/gid mapping we do. What if we expose the CI_USER_LOGIN variable, so
that when needed, the root can be simply passed with this variable and
voila - you have a root shell inside the container with CWD=~root.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2021-02-12 17:01:41 +01:00
Erik Skultety
321293e2a3 ci: Drop the prepare.sh script
The purpose of this script was to prepare a customized environment in
the container, but was actually never used and it required the usage of
sudo to switch the environment from root's context to a regular user's
one.
The thing is that once someone needs a custom script they would very
likely to debug something and would also benefit from root privileges
in general, so the usage of 'sudo' in such case was a bit cumbersome.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2021-02-12 17:01:41 +01:00
Erik Skultety
ee07bffacc ci: Run podman command directly without wrapping it with prepare.sh
The prepare.sh script isn't currently used and forces us to make use
of sudo to switch the user inside the container from root to $USER
which created a problem on our Debian Slim-based containers which don't
have the 'sudo' package installed.
This patch removes the sudo invocation and instead runs the CMD
directly with podman.

Summary of the changes:
- move the corresponding env variables which we need to be set in the
  environment from the sudo invocation to the podman invocation
- pass --workdir to podman to retain the original behaviour we had with
  sudo spawning a login shell.
- MESON_OPTS env variable doesn't need to propagated to the execution
  environment anymore (like we had to do with sudo), because it's
  defined in the Dockerfile

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2021-02-12 17:01:41 +01:00
Erik Skultety
3ca7299a00 ci: Specify the shebang sequence for build.sh
This is necessary for the follow up patch, because the default
entrypoint for a Dockerfile is exec.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2021-02-12 17:01:41 +01:00
Andrea Bolognani
0947902d87 ci: Refresh dockerfiles
dtrace is now also installed when cross-building.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-26 11:01:55 +01:00
Erik Skultety
a79ac43f04 ci: Drop openSUSE Leap 15.1
Leap 15.1 will reach EOL on January 31st 2021, so we should not test on
it during the current development cycle ending on March 1st.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-20 11:20:26 +01:00
Erik Skultety
3114cb4bfd ci: Add openSUSE Leap 15.2
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-20 11:20:26 +01:00
Erik Skultety
6315d923c2 ci: Refresh Dockerfiles
In this refresh CentOS 7 now uses docker.io registry and the PowerTools
repo name regression was fixed for CentOS Stream this time.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-20 11:20:26 +01:00
Daniel P. Berrangé
d855f12945 ci: refresh cirrus vars files
In this refresh the PIP variable is renamed to PIP3 and the
PYPI_PKGS variable disappears since we (currently) don't have
any need for it.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-11 16:14:12 +00:00
Daniel P. Berrangé
855399e382 ci: rename the cirrus vars file to remove the "libvirt-" prefix
The "libvirt-" prefix was removed from hostnames in libvirt-ci.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-11 16:14:06 +00:00
Erik Skultety
555b880aae ci: containers: Refresh the Dockerfiles
Contains changes utilizing "nosync" and "eatmydata" for speedup as well
as fixes for CentOS-8 repoid regression.
ci-commit: b098ec6631a85880f818f2dd25c437d509e53680

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
2020-12-15 09:52:14 +00:00
Daniel P. Berrangé
4e750e932a gitlab: replace "libvirt-" prefix with "ci-" in dockerfiles
This makes the dockerfile name match the output container name

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-12-08 16:04:03 +00:00
Daniel P. Berrangé
82a7f760df gitlab: refresh containers with lcitool for fully minimized base
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-12-08 16:03:59 +00:00
Daniel P. Berrangé
0fbb495e75 gitlab: re-generate container images from lcitool
This introduces Fedora 33 and removes some redundant packages.

Reviewed-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-12-08 16:03:54 +00:00
Roman Bolshakov
82256eaa4a ci: Run test suite on macOS
There's no need to have different CI process between macOS and FreeBSD
as test suite has been fixed on macOS.

Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-11-27 16:31:30 +01:00
Erik Skultety
1a0af38ae7 ci: Switch to meson build system
Add meson required bits to the ci logic in the repo to be able to run
a meson build in a container.
This patch also drops several environment variables we don't need with
meson anymore.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2020-11-27 15:14:35 +01:00
Pavel Hrdina
d9096ce998 gitlab-ci: add coverity job
Introduce new job to make a coverity build and upload coverity data to
scan.coverity.com where the analysis is then executed.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-11-19 14:38:13 +01:00
Andrea Bolognani
81e1df6acb ci: Refresh Dockerfiles
The corresponding lcitool change is

  00d736ea99

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-09 10:14:42 +02:00
Daniel P. Berrangé
417e556147 Add FreeBSD 11.4 CI job on Cirrus
The previous 11.3 image provided by Cirrus did not boot, but they have
now provided a working 11.4 image

Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-09-10 13:11:46 +01:00
Daniel Henrique Barboza
3252742bab ci/Makefile: get rid of 'make check' references
Update the remaining 'make check' references after the
switch to meson/ninja.

Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-08-17 08:42:47 +02:00
Pavel Hrdina
4be700fed9 meson: update .gitlab-ci.yml file
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 09:27:09 +02:00
Ján Tomko
34f62f8ecb ci: refresh Dockerfiles
The corresponding libvirt-ci commit is:
commit 512b1011559165e3f37eae675dd43a431b95baa3
    projects: install clang on Linux

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
2020-08-03 08:35:13 +02:00
Andrea Bolognani
a167e95d17 ci: Run 'make distcheck' on FreeBSD
The Cirrus CI integration was modeled after the Travis CI jobs,
but those were limited to macOS where the test suite is currently
still broken. FreeBSD can run the full distcheck just fine, so
let's do that.

Fixes: 6190c14151
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-07-23 15:03:11 +02:00
Andrea Bolognani
2b23d194d2 ci: Drop Debian 9 containers
The corresponding libvirt-ci commit is 5abf5e7e2326.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-07-15 12:06:34 +02:00
Andrea Bolognani
31939c80cd ci: Refresh Dockerfiles
All targets get cpanm, which is now part of the base system, and
xz is now installed explicitly instead of relying on it being either
present by default or dragged in indirectly.

The corresponding libvirt-ci commit is 8920e8f408ba.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-07-14 12:02:59 +02:00
Andrea Bolognani
4541d3a918 ci: Exclude macOS when generating Dockerfiles
Just like FreeBSD, macOS is not supported by the Dockerfile
generator.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-07-14 12:00:12 +02:00
Andrea Bolognani
fb91290131 cirrus: Generate jobs dynamically
Instead of having static job definitions for FreeBSD and macOS,
use a generic template for both and fill in the details that are
actually different, such as the list of packages to install, in
the GitLab CI job, right before calling cirrus-run.

The target-specific information are provided by lcitool, so that
keeping them up to date is just a matter of running the refresh
script when necessary.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-07-03 11:00:22 +02:00
Andrea Bolognani
d95eeb97a3 ci: Refresh Dockerfiles
All targets get pip3, which is now part of the base system, and
the number of native packages included in the MinGW containers is
significantly reduced.

The corresponding libvirt-ci commit is 4ff697ba0b5d.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-06-25 14:28:25 +02:00
Andrea Bolognani
60a7c78348 ci: Drop mips container for Debian sid
The build job for this container has been failing every single
time, and as it turns out the explanation for that is very simple:
Debian is just not going to support the mips architecture going
forward.

Reported-by: Pino Toscano <ptoscano@redhat.com>
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-23 12:14:00 +02:00
Erik Skultety
4e6e1e2a85 ci: Makefile: Mention CI_MAKE_ARGS and CI_CONFIGURE_ARGS in ci-help
Document the CI_MAKE_ARGS and CI_CONFIGURE_ARGS so that users don't have
to skim through the Makefile to be able to pass arbitrary recognized
make targets to the build system.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2020-06-18 09:19:21 +02:00
Erik Skultety
32a5083cad ci: Makefile: Fix 'overridde' typo
Signed-off-by: Erik Skultety <eskultet@redhat.com>
2020-06-17 11:47:02 +02:00
Andrea Bolognani
2250a0b56f ci: Update build system integration
The ci-* targets need to know where our container images are stored
and how they are called to work, so now that we use the GitLab
container registry instead of Quay some changes are necessary.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-11 19:06:53 +02:00
Andrea Bolognani
95abbdc432 ci: Use GitLab container registry
Instead of using pre-built containers hosted on Quay, build
containers as part of the GitLab CI pipeline and upload them to the
GitLab container registry for later use.

This will not significantly slow down builds, because containers are
only rebuilt when the corresponding Dockerfile has been modified.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-11 19:06:51 +02:00
Andrea Bolognani
6190c14151 ci: Enable Cirrus CI integration
We use cirrus-run to trigger Cirrus CI jobs from GitLab CI jobs,
making it possible to extend our platform coverage to include
FreeBSD without having to maintain our own runners; additionally,
we'll be able to ditch Travis CI and, since results for Cirrus CI
jobs are reflected back to the GitLab CI jobs that triggered them,
we will be able to get all information from a single dashboard.

The FreeBSD and macOS job definitions can be improved further: for
example, we will want to enable caching to speed up builds, and
ultimately we should figure out a way to generate at least part of
them, notably the list of packages to be installed, using lcitool.
All of that will happen in later patches: for now, this is good
enough to start using Cirrus CI.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-10 10:30:56 +02:00
Andrea Bolognani
507724c355 ci: Drop handling of $PKG_CONFIG_LIBDIR
As of libvirt-jenkins-ci commit e41e341f0d8f, we no longer bake
this environment variable into our container images.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-04 11:24:10 +01:00
Andrea Bolognani
058b30b382 ci: Fix handling of $PKG_CONFIG_LIBDIR
There are two environment variables that are baked into our
cross-compilation container images at build time, $CONFIGURE_OPTS
and $PKG_CONFIG_LIBDIR: the former contain the options necessary
to convince configure to perform a cross build rather than a
native one, and the latter is necessary so that pkg-config will
locate the .pc files for MinGW libraries. Container images that
are not intended for cross-compilation will not have either one
defined.

The problem is that, while an empty $CONFIGURE_OPTS is completely
harmless, setting $PKG_CONFIG_LIBDIR to an emtpy value will
result in pkg-config not looking in its default search path, thus
not finding any library, and subsequently breaking native builds.

To work around this issue, only pass $PKG_CONFIG_LIBDIR to sudo
when the value is set in the calling environment.

Fixes: 71517ae4db
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-02-24 15:40:55 +01:00
Andrea Bolognani
71517ae4db ci: Make container environment available to scripts
For container images targeted at cross-building, we bake a small
amount of architecture-specific information in the environment so
that builds can work as expected without requiring additional work
from the user; unfortunately this information got lost as soon as
we called sudo. Explicitly allow it.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-02-24 13:41:58 +01:00
Daniel P. Berrangé
2621d48f00 gnulib: delete all gnulib integration
This deletes all trace of gnulib from libvirt. We still
have the keycodemapdb submodule to deal with. The simple
solution taken was to update it when running autogen.sh.

Previously gnulib could auto-trigger refresh when running
'make' too. We could figure out a solution for this, but
with the pending meson rewrite it isn't worth worrying
about, given how infrequently keycodemapdb changes.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-07 15:03:54 +00:00
Andrea Bolognani
3e3cad5238 ci: Fetch list of available container images dynamically
Any static list of images is destined to become outdated eventually,
so let's start generating it dynamically instead.

Unfortunately there doesn't seem to be a straightforward way to get
Podman/Docker to list all repositories under quay.io/libvirt, so we
have to resort to searching and filtering manually; and since the
two tools behave slightly differently in that regard, it's more
sane to have the logic in a separate shell script than it would be
to keep it inline in the Makefile with all the annoying escaping
doing so would entail.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
2020-01-06 16:43:46 +01:00
Andrea Bolognani
e672f827d9 ci: Stop using --workdir
Now that we're using sudo, the initial work directory is no
longer relevant since the user will find themselves in their
home directory when they get control anyway.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-21 18:58:34 +02:00
Andrea Bolognani
4c39e54ca9 ci: Run $(CI_PREPARE_SCRIPT) as root
In order for the prepare script to be really useful, it needs
to be able to perform privileged operations such as installing
additional packages or setting up custom mount points.

In order to achieve that, we now run the container as root,
run the prepare script with full privilege, and only then
switch to the unprivileged account with sudo.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-21 18:58:31 +02:00
Andrea Bolognani
2ce3274ea5 ci: Introduce $(CI_PREPARE_SCRIPT)
This script is run before $(CI_BUILD_SCRIPT) and can be used
to tweak the environment as necessary before the build starts.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-21 18:58:27 +02:00
Andrea Bolognani
82c311013a ci: Generalize running commands inside the container
Both for ci-build and ci-shell we want to execute basically
the same setup and cleanup logic, the only difference being
that for the former we then run the build script and with the
latter a shell.

Rework the targets so that they both call the generic
ci-run-command rule passing an appropriate $(CI_COMMAND).

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-21 18:58:25 +02:00
Andrea Bolognani
0d1aecdd6a ci: Introduce $(CI_BUILD_SCRIPT)
Instead of hardcoding build instructions into the Makefile,
move them to a separate script that's mounted into the
container.

This gives us a couple of advantages: we no longer have to
deal with the awkward quoting required when embedding shell
code in a Makefile, and we also provide the users with a way
to override the default build instructions with their own.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-21 18:58:22 +02:00
Andrea Bolognani
a79ae3b9a9 ci: Move source directory under $(CI_USER_HOME)
Now that we have a home directory for the user, storing the
source there rather than in a custom top-level directory is
the obvious choice.

Later on we're also going to add some more files related to
builds, and storing everything in the user's home directory
will keep things nice and tidy.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-21 18:58:19 +02:00
Andrea Bolognani
e14bfc97b7 ci: Create user's home directory in the container
Some applications expect the user's home directory to be
present on the system and require workarounds when that's not
the case. Creating the home directory along with everything
else is easy enough for us, so let's just do that.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-21 18:58:16 +02:00
Andrea Bolognani
ffad19f94c ci: Move everything to a separate directory
We're going to have a few more CI-related files in a second, and
it makes sense to have a separate directory for them rather than
littering the root directory.

$(CI_SCRATCHDIR) can now also be created inside the CI directory,
and as a bonus the make rune necessary to start CI builds without
running configure first becomes shorter.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2019-08-21 18:58:13 +02:00