Now that we've switched to the simple handling, the first thing that can
be massively simplified is checkpoint deletion. We now need to only go
through the backing chain and find the appropriately named bitmaps and
delete them, no complex lookups or merging.
Note that compared to other functions this deletes the bitmap in all
layers compared to others where we expect only exactly 1 bitmap of a
name in the backing chain to prevent potential problems.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Based on the 'snapshots' example with manual tweaks to introduce
inactive, transient, inconsistent and duplicate bitmaps in various parts
of the chain to exercise detection and new validation code.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Now that we've updated both the test data and the validator to new
semantics we can start testing again.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reject duplicates and other problematic bitmaps according to the new
semantics of bitmap use in libvirt.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Use test data which conforms to the new semantics which changed in the
previous patch.
The test data was created by the same set of commands as originally in
commit 0b27b655b1
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Use test data which conforms to the new semantics which changed in the
previous patch.
The test data was created by the same set of commands as originally in
commit 9aac9d5bda
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Chaining bitmaps for checkpoints (disabling the active one and creating
a new) severely overcomplicated all operations in regards to bitmaps.
Specifically it requires us re-matching the on-disk state to the
internal metadata and in case of merging during block jobs it makes it
almost impossible to cover all corner cases.
Since the checkpoints and incremental backups were not yet enabled,
let's change the design to keep one bitmap per checkpoint. In case of
layered snapshots this will be filled in by using dirty-bitmap-populate.
Finally the main reason for this unnecessary complexity was the fear
that qemu's performance could degrade. In the end I think that
addressing the performance issue will be better done in qemu (e.g by
keeping an internal bitmap updated with changes and merging it
periodically back to the real bitmaps. QEMU writes out changes to disk
at shutdown so consistency is not a problem).
Removing the relationships between bitmaps frees us from complex
handling and also makes all the surrounding code more robust as one
broken bitmap doesn't necessarily invalidate whole chains of backups.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
There will be multiple places where we'll need to print nodenames from a
GSList of virStorageSource for testing purposes. Extract the code into a
function.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
They will be replaced by a different set which will test scenarios
relevant for the new semantics.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Upcoming patches are going to rewrite and semantically modify how
bitmaps are handled during blockjobs. This is possible as incremental
backup is not yet fully enabled.
As the changes are going to be incompatible with any current test data
remove all test cases for bitmap handling during checkpoint deletion,
incremental backups, block commit, block copy, and bitmap validation
operations.
The tests will be gradually added back later after the code and
test-data is refactored.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Use the new test data for checkpoint deletion testing. This test also
requires modification of the internals to allow checking for test
failure.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Use the new test data when calculating incremental backup operations. As
incremental backup fails with no bitmap the test code is modified to
allow testing this case too.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Fetch the checkpoint list for every disk specifically based on the new
per-disk 'incremental' field.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
In preparation to allow heterogenous backups store the 'incremental'
field per-disk and fill it by default from the per-backup field.
Having this will be important once we'll want to allow incremental
backup working while hotplugging a new disk.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
If a disk is not captured by one of the intermediate checkpoints the
code would fail, but we can easily calculate the bitmaps to merge
correctly by skipping over checkpoints which don't describe the disk.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The algorithm is getting quite complex. Split out the lookup of range of
backing chain storage sources and bitmaps contained in them which
correspond to one checkpoint.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The CPUID data in *-{disabled,enabled}.xml convert feature names from
the corresponding *.json file into raw CPUID and MSR data and thus some
of them may need to be updated when new features are added into the CPU
map.
Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Rather than using the AM_LDFLAGS_MOD_NOUNDEF options with the noinstall
library that will come out of libtool from
libvirt_driver_nodedev_impl_la, use it with the installed version
libvirt_driver_nodedev_la.
Broken-by-commit: c44bffb9
Fixes: https://ci.centos.org/job/libvirt-rpm/systems=libvirt-fedora-32/1155/
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
As noted by Erik Skultety, we use the same XML schema to report
existing devices and to define new devices. However, some schema
elements are "read-only". In other words, they are used to report
information from the node device driver and cannot be used to define a
new device. Note these in the documentation.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Test that we run 'mdevctl' with the proper arguments when we destroy
mediated devices with virNodeDeviceDestroy()
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Add the ability to destroy mdev node devices via the mdevctl utility.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Test that we run 'mdevctl' with the proper arguments when creating new
mediated devices with virNodeDeviceCreateXML().
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In order to test the nodedev driver, we need to link against a
non-loadable module. Similar to other loadable modules already in the
repository, create an _impl library that can be linked against the unit
tests and then create a loadable module from that.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
With recent additions to the node device xml schema, an xml schema can
now describe a mdev device sufficiently for libvirt to create and start
the device using the mdevctl utility.
Note that some of the the configuration for a mediated device must be
passed to mdevctl as a JSON-formatted file. In order to avoid creating
and cleaning up temporary files, the JSON is instead fed to stdin and we
pass the filename /dev/stdin to mdevctl. While this may not be portable,
neither are mediated devices, so I don't believe it should cause any
problems.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In order to allow libvirt to create and start new mediated devices, we
need to be able to verify that the device has been started. In order to
do this, we'll need to save the UUID of newly-discovered devices within
the virNodeDevCapMdev structure. This allows us to search the device
list by UUID and verify whether the expected device has been started.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
In preparation for creating mediated devices in libvirt, we will need to
wait for new mediated devices to be created as well. Refactor
nodeDeviceFindNewDevice() so that we can re-use the main logic from this
function to wait for different device types by passing a different
'find' function.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Mediated devices support arbitrary vendor-specific attributes that can
be attached to a mediated device. These attributes are ordered, and are
written to sysfs in order after a device is created. This patch adds
support for these attributes to the mdev data types and XML schema.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Currently nodeDeviceCreateXML() and nodeDeviceDestroy() only support
NPIV HBAs, but we want to be able to create mdev devices as well. This
is a first step to enabling that support.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
When parsing a nodedev xml file, the iommuGroup element should be
optional. This element should be read-only and is determined by the
device driver. While this is a change to existing behavior, it doesn't
break backwards-compatibility because it makes the parser less strict.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Some less commonly used drivers were omitted when we switched
the allocator from a plain VIR_ALLOC to virDomainFSDefNew.
https://bugzilla.redhat.com/show_bug.cgi?id=1846450
Fixes: da665fbd48
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Introduced by commit 72ab0b6dc8 which
added some code depending on libvirt's log format string into
qemuProcessReadLogOutput. This function was deleted by commit
932534e85f later.
Drop the comment.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Closes: https://gitlab.com/libvirt/libvirt/-/issues/35
In a few cases we might set seclabels on a path outside of
namespaces. For instance, when restoring a domain from a file,
the file is opened, relabelled and only then the namespace is
created and the FD is passed to QEMU (see v6.3.0-rc1~108 for more
info). Therefore, when restoring the label on the restore file,
we must ignore domain namespaces and restore the label directly
in the host.
This bug demonstrates itself when restoring a domain from a block
device. We don't create the block device inside the domain
namespace and thus the following error is reported at the end of
(otherwise successful) restore:
error : virProcessRunInFork:1236 : internal error: child reported (status=125): unable to stat: /dev/sda: No such file or directory
error : virProcessRunInFork:1240 : unable to stat: /dev/sda: No such file or directory
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The function calls virSecurityManagerDomainRestorePathLabel()
after all.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The new name is virSecurityManagerDomainRestorePathLabel().
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
After previous commit this function is used no more.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
After previous commit this function is used no more.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
There are two places within qemu driver that misuse
qemuSecuritySetSavedStateLabel() to set seclabels on tempfiles
that are not state files: qemuDomainScreenshot() and
qemuDomainMemoryPeek(). They are doing so because of lack of
qemuSecurityDomainSetPathLabel() at the time of their
introduction.
In all three secdrivers (well, four if you count NOP driver) the
implementation of .domainSetSavedStateLabel and
.domainSetPathLabel callbacks is the same anyway.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Although this can be considered a new feature, from the user
standpoint is more of a QoL improvement.
Suggested-by: Michal Privoznik <mprivozn@redhat.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We're not mentioning that we're replicating QEMU behavior on purpose.
First because QEMU will one day, maybe, change the behavior and
start to refuse incomplete NUMA setups, and then our documentation
is now deprecated. Second, auto filling the CPUs in the first
cell will work regardless of QEMU changes in the future.
The idea is to encourage the user to provide a complete NUMA CPU topology,
not relying on the CPU auto fill mechanic.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Add a unit test to verify the NUMA vcpus autocomplete implemented
in the previous patch.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Libvirt allows the user to define an incomplete NUMA topology, where
the sum of all CPUs in each cell is less than the total of VCPUs.
What ends up happening is that QEMU allocates the non-enumerated CPUs
in the first NUMA node. This behavior is being flagged as 'to be
deprecated' at least since QEMU commit ec78f8114bc4 ("numa: use
possible_cpus for not mapped CPUs check").
In [1], Maxiwell suggested that we forbid the user to define such
topologies. In his review [2], Peter Krempa pointed out that we can't
break existing guests, and suggested that Libvirt should emulate the
QEMU behavior of putting the remaining vCPUs in the first NUMA node
in these cases.
This patch implements Peter Krempa's suggestion. Since we're going
to most likely end up with disjointed NUMA configuration in node 0
after the auto-fill, we're making auto-fill dependent on QEMU_CAPS_NUMA.
A following patch will update the documentation not just to inform
about the auto-fill mechanic with incomplete NUMA topologies, but also
to discourage the user to create such topologies in the future. This
approach also makes Libvirt independent of whether QEMU changes
its current behavior since we're either auto-filling the CPUs in
node 0 or the user (hopefully) is aware that incomplete topologies,
although supported in Libvirt, are to be avoided.
[1] https://www.redhat.com/archives/libvir-list/2019-June/msg00224.html
[2] https://www.redhat.com/archives/libvir-list/2019-June/msg00263.html
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
These helpers will be used in an auto-fill feature for incomplete
NUMA topologies in the next patch.
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
