Commit Graph

39498 Commits

Author SHA1 Message Date
zhenwei pi
f76848a7c1 util: rename virNetDevParseVfConfig to virNetDevParseVfInfo
virNetDevParseVfConfig has became a multifunctional helper function,
rename it to virNetDevParseVfInfo.

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Reviewed-by: Laine Stump <laine@redhat.com>
2020-10-20 17:29:48 -04:00
zhenwei pi
b295f06da4 util: support device stats collection for <interface type='hostdev'>
libvirt can retrieve traffic stats for emulated interfaces that are
backed by tap or macvtap devices, but this information wasn't
available for hostdev interfaces (those that are implemented by
assigning an SR-IOV VF device to a guest using vfio):

  #virsh domifstat instance --interface=52:54:00:2d:b2:35
  error: Failed to get interface stats instance 52:54:00:2d:b2:35
  error: internal error: Interface name not provided

For some SR-IOV VF devices this information is available via the
netlink VFINFO_LIST request/response, and that is what this patch uses
to implement stats retrieval for VF. Not that this is dependent on
support in the PF driver - for example, the Mellanox ConnectX-4 Lx
(mlx5) driver reports usable stats, while Intel 82599 (ixgbe) and
82576 (igb) just report all stats as 0.  (this is the same result as
"ip -s link show").

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Reviewed-by: Laine Stump <laine@redhat.com>
2020-10-20 17:29:29 -04:00
Jonathon Jongsma
b79abf9c3c qemu: support hotplug of vdpa devices
By using the new qemu monitor functions to handle passing and removing
file descriptors, we can support hotplug of vdpa devices.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
2020-10-20 15:15:06 -04:00
Jonathon Jongsma
b8998cc670 qemu: add monitor functions for handling file descriptors
add-fd, remove-fd, and query-fdsets provide functionality that can be
used for passing fds to qemu and closing fdsets that are no longer
necessary.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
2020-10-20 15:14:37 -04:00
Jonathon Jongsma
0b1a05ffb5 qemu: add vdpa support
Enable <interface type='vdpa'> for qemu domains. This provides basic
support and does not support hotplug or migration.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
2020-10-20 14:48:02 -04:00
Jonathon Jongsma
9825f71b53 qemu: add vhost-vdpa capability
Recent versions of qemu added the -netdev vhost-vdpa device. This
capability allows libvirt to know whether this is supported.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
2020-10-20 14:47:20 -04:00
Jonathon Jongsma
08f8fd8413 conf: Add support for vDPA network devices
This patch adds new schema and adds support for parsing and formatting
domain configurations that include vdpa devices.

vDPA network devices allow high-performance networking in a virtual
machine by providing a wire-speed data path. These devices require a
vendor-specific host driver but the data path follows the virtio
specification.

When a device on the host is bound to an appropriate vendor-specific
driver, it will create a chardev on the host at e.g.  /dev/vhost-vdpa-0.
That chardev path can then be used to define a new interface with
type='vdpa'.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
2020-10-20 14:46:52 -04:00
Daniel P. Berrangé
c7fa462a87 meson: remove non-pkg-config fallback for devmapper
The fallback for distros which lack pkg-config support for devmapper
references an undefined variable "tmp". It appears non of our supported
build platforms are triggering this bug and so the fallback code can be
removed entirely rather than fixed.

Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-10-20 17:50:38 +01:00
Ján Tomko
92ac61ea5e qemu: agent: fix array access
My code movement changed the type of ifaces_ret from
virDomainInterfacePtr * to virDomainInterfacePtr **,
but failed to adjust the condition or dereference the
array correctly.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
Fixes: 6ddb1f803e
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 15:53:52 +02:00
Peter Krempa
b6fea3f5d8 qemuxml2argvmock: Remove mocking of 'virSCSIDeviceGetSgName'
Hostdev setup code no longer resolves hostdev name in the commandline
formatter but we mock it directly in the monitor code.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 15:08:22 +02:00
Peter Krempa
7b0ced89e7 qemu: Prepare hostdev data which depends on the host state separately
SCSI hostdev setup requires querying the host os for the actual path of
the configured hostdev. This was historically done in the command line
formatter. Our new approach is to split out this part into
'qemuProcessPrepareHost' which is designed to be skipped in tests.

Refactor the hostdev code to use this new semantics, and add appropriate
handlers filling in the data for tests and the qemuConnectDomainXMLToNative
users.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 15:08:22 +02:00
Peter Krempa
609497876c qemu: hostdev: Prepare definition bits in qemuDomainPrepareHostdev
qemuBuildHostdevSCSIAttachPrepare is supposed to prepare the data
structure used for attaching the hostdev not preparing the hostdev
definition itself. Move the corresponding bits to qemuDomainPrepareHostdev

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 15:08:22 +02:00
Peter Krempa
9ff3ad9058 qemuProcessCreatePretendCmd: Split up preparation and command building
Host preparation steps which are deliberately skipped when
pretend-creating a commandline are normally executed after VM object
preparation. In the test code we are faking some of the host
preparation steps, but we were doing that prior to the call to
qemuProcessPrepareDomain embedded in qemuProcessCreatePretendCmd.

By splitting up qemuProcessCreatePretendCmd into two functions we can
ensure that the ordering of the prepare steps stays consistent.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 15:08:22 +02:00
Michal Privoznik
d8960bff71 hyverv: hypervCreateEmbeddedParam: Rework items counting
It's not necessarily clear, why we need to create the hash table
as big as number of fields we want to store, but nevertheless,
the code can be written a bit better. The @count should be type
of size_t and could be used directly in the loop that counts the
fields.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Matt Coleman <matt@datto.com>
2020-10-20 12:12:21 +02:00
Michal Privoznik
7f238274bb hyperv: Drop needless error label in hypervCreateEmbeddedParam()
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Matt Coleman <matt@datto.com>
2020-10-20 12:09:57 +02:00
Michal Privoznik
28065881f7 hyperv: Use g_auto() for virHashTable in hypervCreateEmbeddedParam
This will allow us to drop 'error' label later.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 12:09:12 +02:00
Michal Privoznik
b54b229e74 hyperv: Accept const @value in hypervSetEmbeddedProperty()
The hypervSetEmbeddedProperty() function is used to update a
value for given property in a list of properties created by
hypervCreateEmbeddedParam(). The list is nothing fancy - it's a
virHashTable that has NULL as dataFree callback => the table does
not own the value. This is not that obvious since
hypervSetEmbeddedProperty() accepts a non-const pointer. This
fact makes it unnecessary hard to consume, e.g. if we wanted to
pass a stack allocated string.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Matt Coleman <matt@datto.com>
2020-10-20 12:05:50 +02:00
Peter Krempa
e257493441 virschematest: Validate schema in tests/cputestdata
We now have a schema file for the 'cpu' elements. Use it to validate
files in 'tests/cputestdata'

Unfortunately the files in the directory are too disorganised and not
easy to split up to do something more straightforward.

The -baseline- input files are tested by the test internally and the
rest of the files are internal data feeding the tests so they don't
need validation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:11 +02:00
Peter Krempa
f3bbaf3169 virschematest: Validate more XMLs
We weren't validating certain directories containing nwfilter, network
and capability XML test files.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
67214de5bb virschematest: Add directories containing domain XMLs
Validate additional XML documents we use for internal testing.
Specifically there's a lot of them belonging to the vmx and bhyve test
suite which were not validated.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
d504297993 tests: vmx: Add <name> element for all domain XMLs
<name> is mandatory for a domain XML. Add 'displayName' for all the test
cases which were missing them so that <name> is parsed correctly.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pino Toscano <ptoscano@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
6c7e6caf43 schema: domain: Allow space in XML schema for bridge source
vmware's network names can contain space and they are used as bridge
source. Modify the schema to allow it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
590423ca0f schema: domain: Add definition for the 'vmware' private namespace
The 'vmware' private namespace wasn't present in our schema definition
making all XMLs having the <datacenterpath> element invalid.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
37208d9d17 nwfilterxml2xmltest: Rename cases with XMLs not conforming to schema
There's quite a few negative tests. In anticipation of schema testing of
the 'nwfilterxml2xmlin' directory rename all negative/non-conformant
XMLs with the -invalid suffix.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
f4aa549ea8 nwfilterxml2firewall: Fix schema compilance of 'tcp' test case
Fix the 'flags' of the last rule to conform to the RNG schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
6ecd479d01 schema: nwfiter: Allow leading/trailing whitespace in 'variable-name-type'
The reference string parser tolerates some leading/trailing whitespace
for the reference strings as witnessed by
tests/nwfilterxml2xmlin/iter-test3.xml

Allow them in the schema so that the test passes schema validation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
4d57aee9ad schema: nwfilter: Allow all accepted values for 'ipsetflags'
The parser for the 'ipsetflags' accepts the 'src' and 'dst' values
stripping case. Express the same in the schema to pass validation of any
accepted string.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
59d847b578 domainconfdata/getfilesystem.xml: Remove <init> element
The element is not needed for the test and doesn't conform to the domain
XML schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
2dce8109f9 virschematest: Fix testing of 'nwfilter' xmls
The files are under 'src/nwfilter/xml'

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
57f1d5bb3b virschematest: Add coverage for 'examples/xml/test'
Add all appropriate file from our test driver example XML directory.

Note that the two 'node.*' files are actually custom for the test driver
to load full state. We don't have a schema for them.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
933b4b9980 virschematest: Add coverage for 'examples/xml/storage'
Match the correct subsets of the files via the 'dirRegex' property.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
11acb6f4eb examples: xml: Fix 'vol-raw.xml' and 'vol-sparse.xml'
Our schema forces a <target/> element which was not present in the
files.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
fb6944d0ab examples: xml: Fix 'pool-netfs.xml' example
Schema mandates a '<dir>' element, not '<directory>'.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
873d484190 virschematest: Add regex filtering for directory contents
In some cases we have directories with mixed XML files in the test
suite. Adding regex filtering will allow testing subsets of the XML
files against schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Peter Krempa
837633afb9 virschematest: Rewrite internals to allow increasing XML test coverage
To allow greater variablitity of XML schema validation tests without
needlessly reparsing the schema we need to refactor the internals to
pass in structs rather than just paths to directory.

This allows to directly implement testing of single files and will
simplify further additions such as filtering of the list of XML files in
a directory.

The list of tested paths is directly ported for now and will be improved
in follow-up patches.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-20 09:05:10 +02:00
Boris Fiuczynski
07d0a64ddb spec: Fix numad check
This caused

DEBUG: meson.build:2149:2: ERROR: Problem encountered: You must have numactl enabled for numad support.

on s390x.

Fixes: 974dc0a4c6
Signed-off-by: Boris Fiuczynski <fiuczy@linux.ibm.com>
Reviewed-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
2020-10-19 15:25:08 +02:00
Peter Krempa
0e83c12c68 util: xml: Add autoptr cleanup for virXMLValidator
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-10-19 14:02:56 +02:00
Peter Krempa
8c67e389d6 qemuBlockExportAddNBD: Use 'block-export-add' when available
Switch to the new QMP command once it becomes available. Since the code
was refactored to have just one central location to do this we can
contain the ugly bits to just this one function.

Since we now use the replacement for 'nbd-server-add' mark the test case
as being OK with removal of the command.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-19 13:36:58 +02:00
Peter Krempa
f70e9c6cb8 qemu: Add infrastructure for 'block-export-add' to export NBD
Add the monitor code, corresponding generator of properties for NBD and
tests validating it against the schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-19 13:36:58 +02:00
Peter Krempa
adb9f7123a qemu: capabilities: Add QEMU_CAPS_BLOCK_EXPORT_ADD
The 'block-export-add' QMP command is a replacement for 'nbd-server-add'
and will allow greater flexibility. Add a capability so that we can
switch to it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-19 13:36:58 +02:00
Peter Krempa
2f451d8114 tests: qemucapabilities: Update capabilities for qemu-5.2 dev cycle
Update to commit v5.1.0-2207-g96292515c0

Recent changes include deprecation of 'nbd-server-add' and addition of
'block-export-add'.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-19 13:36:58 +02:00
Peter Krempa
917d8a5390 qemumonitorjsontest: Allow deprecation of 'nbd-server-add' QMP command
qemu is going to deprecate this command in the next release. Allow this
as later patches will implement the use of replacement.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-19 13:36:58 +02:00
Peter Krempa
7c38a0dc9a qemu: block: Extract code for adding NBD exports to 'qemuBlockExportAddNBD'
Centralize the logic deciding which arguments to use when exporting a
block backend via NBD to a single place so that it can be centrally
fixed in upcoming commits to support the new export method via
'block-export-add'.

Additionally this allows simplification of the caller from migration as
the logic deciding which arguments to use is extracted too.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2020-10-19 13:36:58 +02:00
Peter Krempa
b1bac48b0b scripts: meson-html-gen: Remove
The script was obscuring what's happening and not reporting errors
properly. Remove it since it's no longer used now.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2020-10-19 12:40:51 +02:00
Peter Krempa
73cdeed502 docs/manpages/meson.build: Use template code for XSLT processing
Replace the reimplementation of the XSLT processing custom target with
an identical copy form docs/meson.build and a comment to keep them in
sync.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2020-10-19 12:40:51 +02:00
Peter Krempa
9ce9d794f5 docs/kbase/meson.build: Use template code for XSLT processing
Replace the reimplementation of the XSLT processing custom target with
an identical copy form docs/meson.build and a comment to keep them in
sync.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2020-10-19 12:40:51 +02:00
Peter Krempa
80c927b6f9 docs/internals/meson.build: Use template code for XSLT processing
Replace the reimplementation of the XSLT processing custom target with
an identical copy form docs/meson.build and a comment to keep them in
sync.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2020-10-19 12:40:51 +02:00
Peter Krempa
4ced77a309 docs: meson.build: Prepare for use of identical code for XSLT processing of htmls
Meson unfortunately doesn't give us any means to share the code using
xsltproc to output HTMLs processed by our template. This means we will
have to resort to copy&paste engineering.

To make things simpler, let's use the same block of code in
docs/meson.build but also any of the subdirs which generate htmls.

This will be achieved by making it configurable and wrapping it in a
comment that instructs anybody editing it to keep it identical.

We need to be able to configure the template file used and installation
directory. The rest of the processing is same as we do in
docs/meson.build.

This code will then be copied to subdirs to refactor the current
approach used there.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2020-10-19 12:40:50 +02:00
Peter Krempa
fa84e3c46a docs: meson.build: Generate HTML files directly by meson
Since we no longer reformat the XSLT-transformed files, there's no need
to use an external script any more.

Unfortunately this hid errors from 'xsltproc' as return value was not
checked and the stderr was piped into xmllints stdin. The result was
that any invalid input file would result into an empty output file.

Since the script's only purpose was to prevent additional temporary
files at the time we were reformatting the output in a pipeline we no
longer need this.

Moving the generation directly into the meson definition makes it more
obvious what's happening and saves readers from having to parse what's
going on. A free bonus is that errors are now properly caught and
reported.

This patch converts the main docs/ directory for now with cleanup of
other comming later.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2020-10-19 12:40:50 +02:00
Peter Krempa
46b5e9a92e docs: meson.build: Limit html files depending on 'aclperms.htmlinc'
Only 'acl.html' output file includes that file so there's no need to
make everything depend on it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
2020-10-19 12:40:50 +02:00