Commit Graph

49382 Commits

Author SHA1 Message Date
Jonathon Jongsma
b658b1a27e qemu: Extract qemuDomainLogContext into a new file
This will allow us to use it for nbdkit logging in upcoming commits.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:50 -05:00
Jonathon Jongsma
abdc4f2092 Generalize qemuDomainLogContextNew()
Allow to specify a basename for the log file so that
qemuDomainLogContextNew() can be used to create log contexts for
secondary loggers.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:50 -05:00
Jonathon Jongsma
8d73929e04 qemu: add functions to start and stop nbdkit
Add some helper functions to build a virCommand object and run the
nbdkit process for a given virStorageSource.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:50 -05:00
Jonathon Jongsma
4a632de2ac qemu: query nbdkit module dir from binary
Rather than hard-coding the nbdkit module directory, query the nbdkit
binary for the location to these directories. nbdkit provides a
--dump-config optiont that outputs this information and can be easily
parsed. We can also get the version from this output rather than
executing `nbdkit --version` separately.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:50 -05:00
Jonathon Jongsma
1475c8c0d7 qemu: Add qemuNbdkitProcess
An object for storing information about a nbdkit process that is serving
a specific virStorageSource. At the moment, this information is just
stored in the private data of virStorageSource and not used at all.
Future commits will use this data to actually start a nbdkit process.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:50 -05:00
Jonathon Jongsma
0c00059a72 qemu: use file cache for nbdkit caps
Add the virFileCache implementation for nbdkit capabilities to the qemu
driver. This allows us to determine whether nbdkit is installed and
which plugins are supported. it also has persistent caching and the
capabilities are re-queried whenever something changes.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:50 -05:00
Jonathon Jongsma
d0fd04556c qemu: implement persistent file cache for nbdkit caps
Implement the loadFile and saveFile virFileCacheHandlers callbacks so
that nbdkit capabilities are cached perstistently across daemon
restarts. The format and implementation is modeled on the qemu
capabilities, but simplified slightly.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:50 -05:00
Jonathon Jongsma
9c0b83a9ed qemu: implement basic virFileCache for nbdkit caps
Preparatory step for caching nbdkit capabilities. This patch implements
the newData and isValid virFileCacheHandlers callback functions.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:49 -05:00
Jonathon Jongsma
b7e7ac14d0 util: Allow virFileCache data to be any GObject
Since the libvirt documentation suggests to prefer GObject over
virObject, and since virObject is a GObject, change virFileCache to
allow GObjects as data.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:49 -05:00
Jonathon Jongsma
03afd10cd3 qemu: expand nbdkit capabilities
In order to add caching of the nbdkit capabilities, we will need to
compare against file modification times, etc. So look up this
information when creating the nbdkit caps.

Add a nbdkit_moddir build option to allow the builder to specify the
location to look for nbdkit plugins and filters.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:49 -05:00
Jonathon Jongsma
6b7e6b73a9 qemu: Add functions for determining nbdkit availability
In future commits, we will optionally use nbdkit to serve some remote
disk sources. This patch queries to see whether nbdkit is installed on
the host and queries it for capabilities. The data will be used in later
commits.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:49 -05:00
Jonathon Jongsma
7c5a5366ff schema: allow 'ssh' as a protocol for network disks
There was support in the code for parsing protocol='ssh' on network disk
sources, but it was not present in the xml schema. Add this to the
schema.

Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2023-09-19 14:28:49 -05:00
Jim Fehlig
417197a38f libxl: Fix Domain-0 ballooning logic
When Domain-0 autoballooning is enabled, it's possible that memory may
need to be ballooned down in Domain-0 to accommodate the needs of another
virtual machine. libxlDomainFreeMemory handles this task, but due to a
logic bug is underflowing the variable containing Domain-0 new
target memory. The resulting huge numbers are filtered by
libxlSetMemoryTargetWrapper and memory is not changed.

Under the covers, libxlDomainFreeMemory uses Xen's libxl_set_memory_target
API, which includes a 'relative' parameter for specifying how to set the
target. If true, the target is an increment/decrement value over the
current memory, otherwise target is taken as an absolute value.
libxlDomainFreeMemory sets 'relative' to true, but never allows for
negative values by declaring the target memory variable as an unsigned.
Fix by declaring the variable as signed, which also requried adjusting
libxlSetMemoryTargetWrapper.

Signed-off-by: Jim Fehlig <jfehlig@suse.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-19 09:56:29 -06:00
Erik Skultety
4096a59e6e ci: jobs.sh: Define and create SCRATCH_DIR for local executions
Running outside of GitLab will likely not have the variable set and
hence the execution would fail. To make sure we always start with a
clean scratch dir (which may or may not be the best thing), create it
with 'mktemp'. The main reason for a temporary directory is to ensure a
clean environment for the job every time run_integration function is
run. For repeated interactive use case, it is imperative that the
developer takes care of their environment.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
a2c3a3e038 ci: jobs.sh: run_integration: Print DAEMONS variable for debugging
One advantage that GitLab's YAML has with Shell commands is that every
single line is printed out as is, including control structures. In
order to see whether the logic did the same thing and the tests are
going to operate on the right set of daemons (monolithic vs modular),
lets print the DAEMONS variable that we set depending on the distro we
execute the tests on.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
0c9603c858 ci: jobs.sh: integration: Execute commands via 'run_cmd[_quiet]' helpers
Unfortunately, once we go down the line of running our own scripts as
part of GitLab CI jobs rather than open coding Shell in YAML, we lose
the benefit of seeing each line the script executes. The downside of
the default YAML however is that we have to maintain the same piece of
code on 2 places in that case. Let's adopt what we use with other
container jobs and prefix each shell command with 'run_cmd' or
'run_cmd_quiet' which will dump it in the logs before executing.
Flow control expressions and structures are a problem though in this
regard, so let's just print some important values for debugging
purposes.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
e004458616 ci: jobs.sh: Introduce a quiet version of run_cmd
We've started using the run_cmd helper function to log what kind of
command is being executed as well as actually executing the command.
The problem however is doing I/O redirections for commands which we
don't wish to see any output for whatever reason. Now, if the
redirection is applied at parameter passing to run_cmd it's going to be
applied to the debug print as well. Let's introduce another helper,
run_cmd_quiet which takes care of the I/O redirection and executes the
command completely silently.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
a91e6f69f4 ci: jobs.sh: run_integration: Make POSIX-compliant
Neither '&>' nor 'source' are defined in POSIX.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
a6b85e03d4 ci: jobs.sh: run_integration: Add/Rewrite/Reformat commentaries
Because of the nature of writing inline shell commands to YAML, most of
the commentaries where inlined with the command not to hinder YAML
readability any further. Since we moved the logic to a standalone
script, we can now do whatever formatting & readability adjustments we
want.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
5c9dfa2091 ci: jobs.sh: integration: Use --quiet with virsh
We've not been interested in any extra output from the command at all
since we always redirected both stdout and stderr to /dev/null. Future
patch will change that slightly, so --quiet will start making sense.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:59 +02:00
Erik Skultety
9847d3eda4 ci: jobs.sh: Drop comment about the need for Avocado 98.0
We needed v98.0 in commit c9a65eb8 due to a bug in Avocado in the past
and have been installing the latest Avocado for a while since commit
91774931, yet we kept the comment by a mistake.

Besides, looks like v98.0 ignores the avocado.config file in the TCK
repo instructing it to run the test suite sequentially leading to test
stability issues, so abandoning the v98.0 in commit 91774931 was a good
thing in the end.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Erik Skultety
b0061e77c8 ci: integration: Drop the 'install-deps' hidden job and reference
Since the section now only consists of a single command, we can happily
move the command to the main integration template job body.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Erik Skultety
82f4bfc34c ci: integration: Adjust the check for CentOS Stream version
All supported versions of Fedora and CentOS Stream 9 default to modular
setup, it's probably better if we cosmetically adjust the CentOS Stream
version check to make it explicit that monolithic daemon services ought
to be started only on Stream 8.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Erik Skultety
6e9bd600d7 ci: integration: Extract the integration CI main recipe to jobs.sh
Follow what's been done to other jobs in .gitlab-ci.yml and extract the
shell logic from YAML to a function in ci/jobs.sh

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Erik Skultety
c222e37264 syntax-check: Drop the shell's 'check for minus' rule
Apparently we've only had it because the -[ao] options weren't portable
at the time, but according to
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html

both are defined in POSIX.1-2017 revision which is old enough for all
our supported platforms to have adopted it already, so we can drop the
check. However, the above has also marked -[ao] as obsolescent stating
that:
   "[OB] Obsolescent

   The functionality described may be removed in a future version of
   this volume of POSIX.1-2017. Strictly Conforming POSIX Applications
   and Strictly Conforming XSI Applications shall not use obsolescent
   features."

It is however unlikely that the shell implementations would drop
support for -[ao] despite POSIX potentially removing them.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2023-09-19 13:35:58 +02:00
Michal Privoznik
732c5f4270 src: Avoid needless checks before calling g_strdup()
There are few places where the following pattern occurs:

  if (var)
      other = g_strdup(var);

where @other wasn't initialized before g_strdup(). Checking for
var != NULL is useless in this case, as that's exactly what
g_strdup() does (in which case it returns NULL).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-18 14:46:24 +02:00
K Shiva Kiran
65558eb5cb NEWS: Documnet network metadata change event
Signed-off-by: K Shiva Kiran <shiva_kr@riseup.net>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-18 11:24:59 +02:00
K Shiva Kiran
e9ffe9ad80 examples: Report network metadata change event
Signed-off-by: K Shiva Kiran <shiva_kr@riseup.net>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-18 11:24:55 +02:00
K Shiva Kiran
f7091b0c01 network: Emit Emit network metadata change event
Signed-off-by: K Shiva Kiran <shiva_kr@riseup.net>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-18 11:24:53 +02:00
K Shiva Kiran
601a931c46 test_driver: Emit network metadata change event
Signed-off-by: K Shiva Kiran <shiva_kr@riseup.net>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-18 11:24:52 +02:00
K Shiva Kiran
faaa9c0065 remote: Implement client side of network netadata change event
Signed-off-by: K Shiva Kiran <shiva_kr@riseup.net>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-18 11:24:50 +02:00
K Shiva Kiran
880274aace conf: Add methods to create Metadata change events
Adds two new private methods to create metadata change events:
- virNetworkEventMetadataChangeNewFromNet()
- virNetworkEventMetadataChangeNewFromObj()

Signed-off-by: K Shiva Kiran <shiva_kr@riseup.net>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-18 11:24:49 +02:00
K Shiva Kiran
95ee416a75 Define Network Metadata change event
When changing the metadata via virNetworkSetMetadata(), we can
now emit an event to notify the app of changes. This is useful
when co-ordinating different applications read/write of custom
metadata.

Signed-off-by: K Shiva Kiran <shiva_kr@riseup.net>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-18 11:24:47 +02:00
Michal Privoznik
47230842aa bhyveBuildControllerArgStr: Fill in missing cases to switch() statement
When making 'type' member of virDomainControllerDef to be of
virDomainControllerType rather than an int I forgot to update
bhyve_command.c.

Fixes: 27a653b893
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2023-09-15 13:26:40 +02:00
Michal Privoznik
5bddeb8e37 virDomainDiskSourceNVMeParse: Use virXMLPropULongLong()
Inside of virDomainDiskSourceNVMeParse() we have
virXMLPropString() + virStrToLong_ull() combo. Switch to
virXMLPropULongLong() which does the same thing.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:22 +02:00
Michal Privoznik
452695926d virStorageNetHostDef: Convert 'transport' field to proper enum type
Convert the field and adjust the XML parsers to use
virXMLPropEnumDefault().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:20 +02:00
Michal Privoznik
2dd6369477 virDomainDeviceInfo: Convert 'type' field to proper enum type
Convert the field, adjust the XML parsers to use virXMLPropEnum()
and fill in missing cases to switch() statements.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:17 +02:00
Michal Privoznik
cc4bb25852 virDomainWatchdogDef: Convert 'action' field to proper enum type
Convert the field and adjust the XML parser to use
virXMLPropEnumDefault().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:15 +02:00
Michal Privoznik
a6eadf00d2 virDomainWatchdogDef: Convert 'model' field to proper enum type
Convert the field and adjust the XML parsers to use
virXMLPropEnum().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:13 +02:00
Michal Privoznik
b691d249d1 virDomainRNGDef: Convert 'backend' field to proper enum type
Convert the field and adjust the XML parser to use
virXMLPropEnum().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:11 +02:00
Michal Privoznik
53edfa23f1 virDomainRNGDef: Convert 'model' field to proper enum type
Convert the field and adjust the XML parser to use
virXMLPropEnum().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:09 +02:00
Michal Privoznik
27a653b893 virDomainControllerDef: Convert 'type' field to proper enum type
Convert the field and fill in missing cases to switch()
statements.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:06 +02:00
Michal Privoznik
44f0371c8c virDomainHostdevSubsys: Convert 'type' field to proper enum type
Convert the field, and fill in missing cases to switch()
statements.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:04 +02:00
Michal Privoznik
c1afc835b0 virDomainHostdevCaps: Convert 'type' field to proper enum type
Convert the field, adjust the XML parsers to use virXMLPropEnum()
and fill in missing cases to switch() statements.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:35:01 +02:00
Michal Privoznik
7bdab79e40 virDomainHostdevDef: Convert 'mode' field to proper enum type
Convert the field, adjust the XML parser to use
virXMLPropEnumDefault() and fill in missing cases to switch()
statements.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:34:59 +02:00
Michal Privoznik
2630d222ab virDomainHostdevSubsysSCSIVHost: Convert 'protocol' field to proper enum type
Convert the field and adjust the XML parser to use
virXMLPropEnum().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:34:56 +02:00
Michal Privoznik
d537f203b9 virDomainHostdevSubsysSCSI: Convert 'protocol' field to proper enum type
Convert the field and adjust the XML parser to use
virXMLPropEnum().

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:34:54 +02:00
Michal Privoznik
f7534c2573 src: Drop needless typecast to virDomainDiskBus
The 'bus' member of _virDomainDiskDef is already declared of
virDomainDiskModel type. Hence, there is no need to typecast the
variable when passing to switch() statements.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:34:52 +02:00
Michal Privoznik
0041723233 qemu_domain_address: Drop needless typecast to virDomainDiskModel
The 'mode' member of _virDomainDiskDef is already declared of
virDomainDiskModel type. Hence, there is no need to typecast the
variable when passing to switch() statements.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:34:50 +02:00
Michal Privoznik
75cdb664d0 src: Drop needless typecast to virDomainDeviceType
The 'type' member of _virDomainDeviceDef is already declared of
virDomainDeviceType type. Hence, there is no need to typecast the
variable when passing to switch() statements.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2023-09-15 10:34:48 +02:00