Commit Graph

37505 Commits

Author SHA1 Message Date
Andrea Bolognani
95abbdc432 ci: Use GitLab container registry
Instead of using pre-built containers hosted on Quay, build
containers as part of the GitLab CI pipeline and upload them to the
GitLab container registry for later use.

This will not significantly slow down builds, because containers are
only rebuilt when the corresponding Dockerfile has been modified.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-11 19:06:51 +02:00
Andrea Bolognani
7ef1324284 ci: Use variables to build image names
This removes a lot of repetition and makes the configuration much
easier to read.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-11 19:06:49 +02:00
Andrea Bolognani
9170b0ee6f docs: Document CIRRUS_GITHUB_REPO variable
This needs to be set for every repository for Cirrus CI integration
to work.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-11 12:03:41 +02:00
Yi Li
414aee194a conf: snapshot: Drop unused variable 'creation'
Signed-off-by: Yi Li <yili@winhong.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-11 09:30:18 +02:00
Andrea Bolognani
0d009ca646 ci: Reduce number of stages
Right now we're dividing the jobs into three stages: prebuild, which
includes DCO checking as well as building artifacts such as the
website, and native_build/cross_build, which do exactly what you'd
expect based on their names.

This organization is nice from the logical point of view, but results
in poor utilization of the available CI resources: in particular, the
fact that cross_build jobs can only start after all native_build jobs
have finished means that if even a single one of the latter takes a
bit longer the pipeline will stall, and with native builds taking
anywhere from less than 10 minutes to more than 20, this happens all
the time.

Building artifacts in a separate pipeline stage also doesn't have any
advantages, and only delays further stages by a couple of minutes.
The only job that really makes sense in its own stage is the DCO
check, because it's extremely fast (less than 1 minute) and, if that
fails, we can avoid kicking off all other jobs.

Reducing the number of stages results in significant speedups:
specifically, going from three stages to two stages reduces the
overall completion time for a full CI pipeline from ~45 minutes[1]
to ~30 minutes[2].

[1] https://gitlab.com/abologna/libvirt/-/pipelines/154751893
[2] https://gitlab.com/abologna/libvirt/-/pipelines/154771173

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 19:33:09 +02:00
Michal Privoznik
2072c39541 news: Document -fw_cfg
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
14c32cd10f qemu: Generate command line for -fw_cfg
This is pretty straightforward and self explanatory.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1837990

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
d024a7da7a secdrivers: Relabel firmware config files
For the case where -fw_cfg uses a file, we need to set the
seclabels on it to allow QEMU the access. While QEMU allows
writing into the file (if specified on the command line), so far
we are enabling reading only and thus we can use read only label
(in case of SELinux).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
9ce32b0935 qemu: Introduce fw_cfg capability
This capability tracks whether QEMU supports -fw_cfg command line
option, more specifically whether it allows specifying filename.

There are some releases of QEMU which support -fw_cfg but not
filename. If this is ever a problem we can refine the capability
later on.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
b5f8f04989 qemu: Validate firmware blob configuration
There are recommendations and limitations to the name of the
config blobs we need to follow [1].

We don't want users to change any value only add new blobs. This
means, that the name must have "opt/" prefix and at the same time
must not begin with "opt/ovmf" nor "opt/org.qemu" as these are
reserved for OVMF or QEMU respectively.

1: docs/specs/fw_cfg.txt from qemu.git

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
3dda889a44 conf: Add firmware blob configuration
QEMU has -fw_cfg which allows users to tweak how firmware
configures itself and/or provide new configuration blobs.
Introduce new <sysinfo/> type "fwcfg" that will hold these
new blobs.

It's possible to either specify new value as a string or
provide a filename which contents then serve as the value.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
b44898dd31 virsysinfo: Parse OEM strings
Setting OEM strings for a domain was introduced in
v4.1.0-rc1~315. However, any application that wanted to use them
(e.g. to point to an URL where a config file is stored) had to
'dmidecode -u --oem-string N' (where N is index of the string).
Well, we can expose them under our <sysinfo/> XML and if the
domain is running Libvirt inside it can be obtained using
virConnectGetSysinfo() API.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
994e56ba42 virsysinfo: Drop global @sysinfoDmidecode
Since nobody sets custom dmidecode path anymore, we can drop all
code that exists only because of that.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
d954fbc74e sysinfotest: Move from custom dmidecode scripts to virCommandSetDryRun()
Problem with custom dmidecode scripts is that they are hard to
modify, especially if we will want them to act differently based
on passed arguments. So far, we have two scripts which do no more
than 'cat $sysinfo' where $sysinfo is saved dmidecode output.

The virCommandSetDryRun() can be used to trick
virSysinfoReadDMI() thinking it executed real dmidecode.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
2ee837a68c sysinfotest: Dissolve sysinfotest_run() in testSysinfo()
There is no real need to have two separate functions. They can be
merged together which not only saves couple of lines, but
prepares the structure of the code for future expansion. See next
commits.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
fc4364a59a testSysinfo: Use more g_auto*()
Some variables defined in the function can be freed
automatically when going out of scope. This renders @result
variable and cleanup label needless.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
f603b99ad9 virSysinfoReadDMI: Drop needless virFindFileInPath()
When trying to decode DMI table, just before constructing
virCommand() the decoder is looked for in PATH using
virFindFileInPath(). Well, this is not necessary because
virCommandRun() will do this too (in virExec()).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Michal Privoznik
30ccd55338 virSysinfoReadDMI: Use more g_auto*()
Virtually every variable defined in the function can be freed
automatically when going out of scope.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-10 14:00:38 +02:00
Andrea Bolognani
c66ccb65f6 ci: Remove Travis CI
Since we now use Cirrus CI for macOS jobs, we no longer need to
keep Travis CI around.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-10 10:31:12 +02:00
Andrea Bolognani
6190c14151 ci: Enable Cirrus CI integration
We use cirrus-run to trigger Cirrus CI jobs from GitLab CI jobs,
making it possible to extend our platform coverage to include
FreeBSD without having to maintain our own runners; additionally,
we'll be able to ditch Travis CI and, since results for Cirrus CI
jobs are reflected back to the GitLab CI jobs that triggered them,
we will be able to get all information from a single dashboard.

The FreeBSD and macOS job definitions can be improved further: for
example, we will want to enable caching to speed up builds, and
ultimately we should figure out a way to generate at least part of
them, notably the list of packages to be installed, using lcitool.
All of that will happen in later patches: for now, this is good
enough to start using Cirrus CI.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
2020-06-10 10:30:56 +02:00
Jiri Denemark
cfdceb9754 qemu: Pass migratable=on|off property for -cpu host
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Jiri Denemark
201bd5db63 qemu: Fill default value in //cpu/@migratable attribute
Before QEMU introduced migratable CPU property, "-cpu host" included all
features that could be enabled on the host, even those which would block
migration. In other words, the default was equivalent to migratable=off.
When the migratable property was introduced, the default changed to
migratable=on. Let's record the default in domain XML.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Jiri Denemark
19926477cb qemu: Advertise migratable attribute for CPU in domcaps
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Jiri Denemark
120ac100fd conf: Advertise migratable attribute for CPU in domcaps
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Jiri Denemark
524f5f00e7 conf: Introduce migratable attribute for the <cpu> element
The attribute is only allowed for host-passthrough CPUs and it can be
used to request only migratable or all supported features to be enabled
in the virtual CPU.

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Jiri Denemark
f48bdbdfe4 qemu: Avoid probing unsupported migratable CPU expansion
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Jiri Denemark
8662b34f18 qemu: Probe for migrtability support in CPU expansion
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Jiri Denemark
2bfa2fea02 qemu: Probe for .migratable property of a CPU
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Jiri Denemark
a08669c31b conf: Use g_auto* in virCPUDefParseXML
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 20:32:50 +02:00
Michal Privoznik
b934d5f42f virSysinfoParseXML: Mangle ctxt inside virSysinfoParseXML()
There's no need to set ctxt->node outside of the function. The
function can set it itself - it has all the info needed.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-09 15:20:16 +02:00
Michal Privoznik
5dd52ecbcf domain_conf: Format NS always last
I think that since <qemu:commandline/> is kind of a hack, it
doesn't deserve place in the front row.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-09 15:20:12 +02:00
Bihong Yu
2a372a5ad5 Fix some wrong usage of ATTRIBUTE_NONNULL()
The virStateInitialize() function has ATTRIBUTE_NONNULL()
referring to @root argument (incorrectly anyway) but in
daemonRunStateInit() NULL is passed in anyway.

Then there is virCommandAddArgPair() which also has
ATTRIBUTE_NONNULL() for one of its arguments and then checks the
argument for being NULL anyways.

Signed-off-by:Bihong Yu <yubihong@huawei.com>
Reviewed-by:Chuan Zheng <zhengchuan@huawei.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-09 11:39:07 +02:00
Peter Krempa
f225f37a8a qemu: blockcommit: Fix placement of qemuDomainDiskBlockJobIsSupported
Commit b50a8354f6 added call to qemuDomainDiskBlockJobIsSupported prior
to filling the 'disk' variable resulting in a crash when attempting a
block commit.

https://gitlab.com/libvirt/libvirt/-/issues/31

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-06-09 09:28:12 +02:00
Laine Stump
2f82fe467f network: add private chains only if there are networks adding iptables rules
Juan Quintela noticed that when he restarted libvirt he was getting
extra iptables rules added by libvirt even though he didn't have any
libvirt networks that used iptables rules. It turns out this also
happens if the firewalld service is restarted. The extra rules are
just the private chains, and they're sometimes being added
unnecessarily because they are added separately in a global
networkPreReloadFirewallRules() that does the init if there are any
active networks, regardless of whether or not any of those networks
will actually add rules to the host firewall.

The fix is to change the check for "any active networks" to instead
check for "any active networks that add firewall rules".

(NB: although the timing seems suspicious, this isn't a new regression
caused by the recently pushed f5418b427 (which forces recreation of
private chains when firewalld is restarted); it was an existing bug
since iptables rules were first put into private chains, even after
commit c6cbe18771 delayed creation of the private chains. The
implication is that any downstream based on v5.1.0 or later that cares
about these extraneous (but harmless) private chains would want to
backport this patch (along with the other two if they aren't already
there))

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-06-08 23:06:36 -04:00
Daniel P. Berrangé
392331b198 rpm: adapt docs filelist to NEWS -> NEWS.rst rename
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-08 18:52:58 +01:00
Daniel P. Berrangé
21597d3caa po: fix update-po target to take account of libvirt.pot in srcdir
Now that we're storing libvirt.pot in git, it will be in srcdir instead
of builddir. Weblate is responsible for running msgmerge when the .pot
file changes, so add a warning that this target is not for general usage.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-08 16:39:09 +01:00
Daniel P. Berrangé
a484779392 po: remove obsolete zanata config file
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-08 16:39:03 +01:00
Daniel P. Berrangé
29f0603c48 po: remove obsolete zanata push/pull rules
We're no longer using Zanata, so remove the old push/pull rules.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-08 16:39:01 +01:00
Daniel P. Berrangé
2c31e5b9bc po: update docs to refer to Weblate instead of Zanata
The old information about managing PO files was outdated, as we're
managing files in a different way with Weblate. This also introduces a
badge showing the translation progress across languages.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-08 16:38:47 +01:00
Andrea Bolognani
f45735786a news: Add information about old releases
Until libvirt 2.5.0 we didn't have a real process for release
notes in place, and we just published the list of commits that
had made it into each release, dividing them into categories that
mostly matched the sections we use today. Those documents haven't
been relevant for years, but they're still in the git repository
and collectively take up almost 2 MiB of disk space.

Let's import the only valuable piece of information they contain,
the release date for each libvirt versions, into the current
document and then drop them for good.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-06-05 16:27:33 +02:00
Andrea Bolognani
dcd14c88a1 news: Convert to reStructuredText
Instead of storing release notes as XML and then converting them
to HTML and ASCII at build time using XSLT and a custom script,
we can use reStructuredText as both the source and ASCII
representation and generate HTML from it using the same tooling
we already use for the rest of the documentation.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-06-05 16:27:33 +02:00
Andrea Bolognani
0ccfcd0361 news: Output reStructuredText for the ASCII version
The ASCII output our scripts produce is already very close to
reStructuredText, and with just a few extra tweaks we can get
almost all of the way there.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-06-05 16:27:33 +02:00
Ján Tomko
1482f2148c docs: backing chains: fix qemu-img invocation
The example was missing the 'create' command.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reported-by: Roland Clobus <rclobus@rclobus.nl>
2020-06-05 16:01:55 +02:00
Daniel P. Berrangé
5263160f9b qemu: fix typo s/readaehad/readahead/
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-05 13:33:00 +01:00
Laine Stump
bc85c34ea9 qemu: don't reject interface update when switching to/from bridged network
If virDomainUpdateDeviceFlags() was used to update an <interface>, and
the interface type changed from type='network' where the network was
an unmanaged bridge (so actualType == bridge) to type='bridge'
(i.e. actualType *also* == bridge), the update would fail due to the
perceived change in type.

In practice it is okay to switch between any interface types that end
up using a tap device, since libvirt just needs to attach the device
to a new bridge. But in this case we were erroneously rejecting it due
to a conditional that was too restrictive. This is what the code was doing:

  if (old->type != new->type)
     [allow update]
  else
     if ((oldActual == bridge and newActual == network)
         || (oldActual == network and newActual == bridge)) {
         [allow update]
     else
         [error]

In the case described above though, old->type and new->type don't match,
but oldActual and newActual are both 'bridge', so we get an error.

This patch changes the inner conditional so that any combination of
'network' and 'bridge' for oldActual and newActual, since they both
use a tap device connected to a bridge.

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-06-04 22:46:26 -04:00
Weblate
dc9692dcdc Update translation files
Updated by "Update PO files to match POT (msgmerge)" hook in Weblate.

Translation: libvirt/libvirt
Translate-URL: https://translate.fedoraproject.org/projects/libvirt/libvirt/

Signed-off-by: Fedora Weblate Translation <i18n@lists.fedoraproject.org>
2020-06-04 23:32:25 +02:00
Yuri Chornoivan
a444dee124 Translated using Weblate (Ukrainian)
Currently translated at 100.0% (9803 of 9803 strings)

Translation: libvirt/libvirt
Translate-URL: https://translate.fedoraproject.org/projects/libvirt/libvirt/uk/

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-04 20:53:55 +01:00
Daniel P. Berrangé
5a92e912ef po: dedupe authors in translations
Weblate gets confused if the same email address is mentioned multiple
times in the translation headers. Dedupe authors so that each author
is mentioned only once, with a range of years listed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-04 20:50:49 +01:00
Daniel P. Berrangé
3cdda5d5e8 gitlab: fix potfile job for new location
Since we're storing the libvirt.pot file in git now, we should pull from
the source dir, not the build dir.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-04 18:53:21 +01:00
Daniel P. Berrangé
611e03127f po: go back to storing the .pot file in git
To integrate with weblate the only practical option currently is to
store the .pot file in git. This is required so that it can add new
languages by cloning the .pot file. It also enables weblate to run
msgmerge on the languages whenever pulling in new changes from git.

The pot file will have to be the full content including the source
locations, so this is going to result in unpleasant diffs when it
is updated periodically.

Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-06-04 11:18:35 +01:00