These per-command generator functions were only exposed in the header to
allow the commandline generation to be tested. Now that we have a
generic mdevctl command generator, we can get rid of the per-command
wrappers and reduce the noise in the header.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
Currently there are dedicated wrappers to construct mdevctl command.
These are mostly fine except for the one that translates both "start"
and "define" commands, only because mdevctl takes the same set of
arguments. Instead, keep the wrappers, but let them call a single
global translator that handles all the mdevctl command differences and
commonalities.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
This is not a 1:1 mapping to mdevctl commands because mdevctl doesn't
support a separate 'create' command. mdevctl uses 'start' for both
starting a pre-defined device as well as for creating and starting a new
transient device. The libvirt code will be more readable if we treat
these as separate commands. When we need to actually execute mdevctl,
the 'create' command will be translated into the appropriate 'mdevctl
start' command.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Reviewed-by: Jonathon Jongsma <jjongsma@redhat.com>
rather than using short opentions (e.g. "-p 0000:00:02.0"), use long
options everywhere (e.g. "--parent=0000:00:02.0")
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
"start" in libvirt means - "take this object and create an
instance out of it"
"create" in libvirt most of the time means - "take and XML description,
make an object out of it and use it to create an instance"
This gets confusing with mdevctl which uses "start" for both. So, this
patch proposes to use virMdevctlStart in cases where from libvirt's POV
we're starting a defined device (unlike mdevctl). Similarly, use
virMdevctlCreate in scenarios where XML description is passed to
libvirt and a transient device is supposed to be created.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
These errors are demoted to debug statements[1] since they're only
intended to be used as return values for public APIs. This makes it
difficult to debug the problem when something goes wrong since no error
message is logged. Switch instead to VIR_ERR_INTERNAL_ERROR so that the
error is logged as expected.
[1] See the implementation of daemonErrorLogFilter() for details:
e2f82a3704/src/remote/remote_daemon.c (L89)
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The calling function will log the error. Just return NULL if a device
cannot be found.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Coverity reported that this function can return NULL, so it should be
handled properly.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
After all devices were enumerated, the enumeration thread call
nodeDeviceUpdateMediatedDevices() to refresh the state of
mediated devices. This means that 'mdevctl' will be executed. But
it may be missing on some systems (e.g. mine) in which case we
should just skip the update of mdevs instead of failing whole
device enumeration.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
To speed up nodedev driver initialization, the device enumeration
is done in a separate thread. Once finished, the thread sets a
boolean variable that allows public APIs to be called (instead of
waiting for the thread to finish).
However, if there's an error in the device enumeration thread
then the control jumps over at the 'error' label and the boolean
is never set. This means, that any virNodeDev*() API is stuck
forever. Mark the initialization as complete (the thread is
quitting anyway) and let the APIs proceed.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Although I have not experienced this in real life, there is a
possible race condition when creating new device, getting its XML
or parent or listing its capabilities. If the nodedev driver is
still enumerating devices (in a separate thread) and one of
virNodeDeviceGetXMLDesc(), virNodeDeviceGetParent(),
virNodeDeviceNumOfCaps(), virNodeDeviceListCaps() or
virNodeDeviceCreate() is called then it can lead to spurious
results because the device enumeration thread is removing devices
from or adding them to the internal list of devices (among with
their states).
Therefore, wait for things to settle down before proceeding with
any of the APIs.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This is more academic dispute than a real bug, but this is taken
from pthread_cond_broadcast(3p) man:
The pthread_cond_broadcast() or pthread_cond_signal() functions
may be called by a thread whether or not it currently owns the
mutex that threads calling pthread_cond_wait() or
pthread_cond_timedwait() have associated with the condition
variable during their waits; however, if predictable scheduling
behavior is required, then that mutex shall be locked by the
thread calling pthread_cond_broadcast() or
pthread_cond_signal().
Therefore, broadcast the initCond while the nodedev driver is
still locked.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
The consensus is to put the verb last. Therefore, the new name is
nodeDeviceInitWait(). This allows us to introduce new function
(done later in a separate commit) that will "complete" the device
initialization.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Historically, we declared pointer type to our types:
typedef struct _virXXX virXXX;
typedef virXXX *virXXXPtr;
But usefulness of such declaration is questionable, at best.
Unfortunately, we can't drop every such declaration - we have to
carry some over, because they are part of public API (e.g.
virDomainPtr). But for internal types - we can do drop them and
use what every other C project uses 'virXXX *'.
This change was generated by a very ugly shell script that
generated sed script which was then called over each file in the
repository. For the shell script refer to the cover letter:
https://listman.redhat.com/archives/libvir-list/2021-March/msg00537.html
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
During its initialization, the nodedev driver tries to set up
monitors for /etc/mdevctl.d directory, so that it can register
mdevs as they come and go. However, if the file doesn't exist
there is nothing to monitor and therefore we can exit early. In
fact, we have to otherwise monitorFileRecursively() fails and
whole driver initialization fails with it.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
During the nodedev driver initialization two threads are created:
one for listening on udev events (like device plug/unplug) and
the other for enumerating devices (so that the main thread doing
the driver init is not blocked). If something goes wrong at any
point then nodeStateCleanup() is called which joins those two
threads (possibly) created before. But it tries to join them even
they weren't created which is undefined behaviour (and it just so
happens that it crashes on my system).
If those two virThread variables are turned into pointers then we
can use comparison against NULL to detect whether threads were
created.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
The nodedev driver private data object @priv is created by
calling udevEventDataNew(). After that, driver->privateData
pointer is set to the freshly allocated object and only a few
lines after all of this the object is locked. Technically it is
safe because there should not be any other thread at this point,
but defensive style of programming says it's better if the object
is locked before driver's privateData is set.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
If initialization of priv->mdevctlMonitors fails, then the
control jumps over to cleanup label where nodeStateCleanup() is
called which tries to lock @priv. But since @priv was already
locked before taking the jump a deadlock occurs. The solution is
to jump onto @unlock label, just like the code around is doing.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Follow best practices and add a unsigned int flags parameter to these
new APIs that have not been in a release yet.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The comments mistakenly say 7.2.0, when they were actually merged during
the 7.3 development cycle.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
When calling virNodeDeviceDefineXML() to define a new mediated device,
we call virMdevctlDefine() and then wait for the new device to appear in
the driver's device list before returning. This caused long delays due
to the behavior of nodeDeviceFindNewMediatedDevice(). This function
checks to see if the device is in the list and then waits for 5s before
checking again.
Because mdevctl is relatively slow to query the list of defined
devices[0], the newly-defined device was generally not in the device
list when we first checked. This results in libvirt almost always taking
at least 5s to complete this API call for mediated devices, which is
unacceptable.
In order to avoid this long delay, we resort to a workaround. If the
call to virMdevctlDefine() was successful, we can assume that this new
device will exist the next time we query mdevctl for new devices. So we
simply add this provisional device definition directly to the nodedev
driver's device list and return from the function. At some point in the
future, the mdevctl handler will run and the "official" device will be
processed, which will update the provisional device if any new details
need to be added.
The reason that this is not necessary for virNodeDeviceCreateXML() is
because detecting newly-created (not defined) mdevs happens through
udev instead of mdevctl. And nodeDeviceFindNewMediatedDevice() always
calls 'udevadm settle' before checking to see whether the device is in
the list. This allows us to wait just long enough for all udev events to
be processed, so the device is almost always in the list the first time
we check and so we almost never end up hitting the 5s sleep.
[0] on my machine, 'mdevctl list --defined' took around 0.8s to
complete for only 3 defined mdevs.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
To accomodate re-use of this functionality in a following patch, split
out the processing of an individual mdev definition into a separate
function.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Calling `mdevctl stop` for a mediated device that is in use by an active
domain will block until that vm exits (or the vm closes the device).
Since the nodedev driver cannot query the hypervisor driver to see
whether any active domains are using the device, we resort to a
workaround that relies on the fact that a vfio group can only be opened
by one user at a time. If we get an EBUSY error when attempting to open
the group file, we assume the device is in use and refuse to try to
destroy that device.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Use the new <uuid> element in the mdev caps to define and start devices
with a specific UUID.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This new API function provides a way to start a persistently-defined
mediate device that was defined by virNodeDeviceDefineXML() (or one that
was defined externally via mdevctl)
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This interface allows you to undefine a persistently defined (but
inactive) mediated devices. It is implemented via 'mdevctl'
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
With mediated devices, we can now define persistent node devices that
can be started and stopped. In order to take advantage of this, we need
an API to define new node devices.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Abstract out the function used to generate the commandline for 'mdevctl
start' since they take the same arguments. Add tests to ensure that
we're generating the command properly.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
We need to query mdevctl for changes to device definitions since an
administrator can define new devices by executing mdevctl outside of
libvirt.
In the future, mdevctl may add a way to signal device add/remove via
events, but for now we resort to a bit of a workaround: monitoring the
mdevctl config directory for changes to files. When a change is
detected, we query mdevctl and update our device list. The mdevctl
querying is handled in a throwaway thread, and these threads are
synchronized with a mutex.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
mdevctl does not currently provide any events when the list of defined
devices changes, so we will need to poll mdevctl for the list of defined
devices periodically. When a mediated device no longer exists from one
iteration to the next, we need to treat it as an "undefine" event.
When we get such an event, we remove the device from the list if it's
not active. Otherwise, we simply mark it as non-persistent.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
At startup, query devices that are defined by 'mdevctl' and add them to
the node device list.
This adds a complication: we now have two potential sources of
information for a node device:
- udev for all devices and for activated mediated devices
- mdevctl for persistent mediated devices
Unfortunately, neither backend returns full information for a mediated
device. For example, if a persistent mediated device in the list (with
information provided from mdevctl) is 'started', that same device will
now be detected by udev. If we simply overwrite the existing device
definition with the new one provided by the udev backend, we will lose
extra information that was provided by mdevctl (e.g. attributes, etc).
To avoid this, make sure to copy the extra information into the new
device definition.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This adds an internal API to query for persistent mediated devices
that are defined by mdevctl. Upcoming commits will make use of this
information.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
This function will parse the list of mediated devices that are returned
by mdevctl and convert it into our internal node device representation.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Expose a helper function that can be used by udev and mdevctl to
generate device names for node devices.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Add two flag values for virConnectListAllNodeDevices() so that we can
list only node devices that are active or inactive.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
we will be able to define mediated devices that can be started or
stopped, so we need to be able to indicate whether the device is active
or not, similar to other resources (storage pools, domains, etc.)
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
When an mdevctl command fails, there is not much information available
to the user about why it failed. This is partly because we were not
making use of the error message that mdevctl itself prints upon failure.
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
Since the functions provided by libpciaccess are not thread-safe,
when the udev-event and nodedev-init threads of libvirt call the
pci_get_strings function provided by libpaciaccess at the same
time the following can happen:
nodedev-init thread:
nodeStateInitializeEnumerate ->
udevEnumerateDevices->
udevProcessDeviceListEntry ->
udevAddOneDevice ->
udevGetDeviceDetails->
udevProcessPCI ->
udevTranslatePCIIds ->
pci_get_strings -> (libpciaccess)
find_device_name ->
populate_vendor ->
d = realloc( vend->devices, (vend->num_devices + 1), * sizeof( struct pci_device_leaf ) );
vend->num_devices++;
udev-event thread:
udevEventHandleThread ->
udevHandleOneDevice ->
udevAddOneDevice->
udevGetDeviceDetails->
udevProcessPCI ->
udevTranslatePCIIds ->
pci_get_strings -> (libpciaccess)
find_device_name ->
populate_vendor ->
d = realloc( vend->devices, (vend->num_devices + 1), * sizeof( struct pci_device_leaf ) );
vend->num_devices++;
Signed-off-by: WangJian <wangjian161@huawei.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Generated by the following spatch:
@@
expression a, b;
@@
+ b = g_steal_pointer(&a);
- b = a;
... when != a
- a = NULL;
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
If libvirtd is terminated before the node driver finishes
initialization, it can crash with a backtrace similar to the following:
Stack trace of thread 1922933:
#0 0x00007f8515178774 g_hash_table_find (libglib-2.0.so.0)
#1 0x00007f851593ea98 virHashSearch (libvirt.so.0)
#2 0x00007f8515a1dd83 virNodeDeviceObjListSearch (libvirt.so.0)
#3 0x00007f84cceb40a1 udevAddOneDevice (libvirt_driver_nodedev.so)
#4 0x00007f84cceb5fae nodeStateInitializeEnumerate (libvirt_driver_nodedev.so)
#5 0x00007f85159840cb virThreadHelper (libvirt.so.0)
#6 0x00007f8511c7d14a start_thread (libpthread.so.0)
#7 0x00007f851442bdb3 __clone (libc.so.6)
Stack trace of thread 1922863:
#0 0x00007f851442651d syscall (libc.so.6)
#1 0x00007f85159842d4 virThreadSelfID (libvirt.so.0)
#2 0x00007f851594e240 virLogFormatString (libvirt.so.0)
#3 0x00007f851596635d vir_object_finalize (libvirt.so.0)
#4 0x00007f8514efe8e9 g_object_unref (libgobject-2.0.so.0)
#5 0x00007f85159667f8 virObjectUnref (libvirt.so.0)
#6 0x00007f851517755f g_hash_table_remove_all_nodes.part.0 (libglib-2.0.so.0)
#7 0x00007f8515177e62 g_hash_table_unref (libglib-2.0.so.0)
#8 0x00007f851596637e vir_object_finalize (libvirt.so.0)
#9 0x00007f8514efe8e9 g_object_unref (libgobject-2.0.so.0)
#10 0x00007f85159667f8 virObjectUnref (libvirt.so.0)
#11 0x00007f84cceb2b42 nodeStateCleanup (libvirt_driver_nodedev.so)
#12 0x00007f8515b37950 virStateCleanup (libvirt.so.0)
#13 0x00005648085348e8 main (libvirtd)
#14 0x00007f8514352493 __libc_start_main (libc.so.6)
#15 0x00005648085350fe _start (libvirtd)
This is because the initial population of the device list is done in a
separate initialization thread. If we attempt to exit libvirtd before
this init thread has completed, we'll try to free the device list while
accessing it from the other thread. In order to guarantee that this
init thread is not accessing the device list when we're cleaning up the
nodedev driver, make it joinable and wait for it to finish before
proceding with the cleanup. This is similar to how we handle the udev
event handler thread.
The separate initialization thread was added in commit
9f0ae0b1.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1836865
Signed-off-by: Jonathon Jongsma <jjongsma@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
These functions are identical. Made using this spatch:
@@
expression path, mode;
@@
- virFileMakePathWithMode(path, mode)
+ g_mkdir_with_parents(path, mode)
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The parent array takes ownership of the inserted value once all checks
pass. Don't make the callers second-guess when that happens and modify
the function to take a double pointer so that it can be cleared once the
ownership is taken.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The parent object takes ownership of the inserted value once all checks
pass. Don't make the callers second-guess when that happens and modify
the function to take a double pointer so that it can be cleared once the
ownership is taken.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The 'conflict' key in a virt_daemon_unit dictionary is not used when
generating systemd service and socket files. The comment associated
with the key claims the default is 'true', and a few build files
needlessly set it to 'true' when defining their virt_daemon_unit.
Remove the 'conflict' key and its use in the affect build files.
Signed-off-by: Jim Fehlig <jfehlig@suse.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
`udevGetIntSysfsAttr` does not necessarily write to the third parameter,
even when it returns 0.
This was found by clang-tidy's
"clang-analyzer-core.UndefinedBinaryOperatorResult" check.
Signed-off-by: Tim Wiederhake <twiederh@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
The current virPCIDeviceNew() signature, receiving 4 uints in sequence
(domain, bus, slot, function), is not neat.
We already have a way to represent a PCI address in virPCIDeviceAddress
that is used in the code. Aside from the test files, most of
virPCIDeviceNew() callers have access to a virPCIDeviceAddress reference,
but then we need to retrieve the 4 required uints (addr.domain, addr.bus,
addr.slot, addr.function) to satisfy virPCIDeviceNew(). The result is
that we have extra verbosity/boilerplate to retrieve an information that
is already available in virPCIDeviceAddress.
A better way is presented by virNVMEDeviceNew(), where the caller just
supplies a virPCIDeviceAddress pointer and the function handles the
details internally.
This patch changes virPCIDeviceNew() to receive a virPCIDeviceAddress
pointer instead of 4 uints.
Reviewed-by: Laine Stump <laine@redhat.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
With commit 09364608b4 node_device: refactor address retrieval of node device
"if-else if" was replaced by "switch".
The contained break statement now is no longer in context of the for loop
but instead of the switch causing the legitimate grumpiness of coverity.
Signed-off-by: Boris Fiuczynski <fiuczy@linux.ibm.com>
Suggested-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Allow mdev devices to be created on the matrix device.
Signed-off-by: Shalini Chellathurai Saroja <shalini@linux.ibm.com>
Reviewed-by: Bjoern Walk <bwalk@linux.ibm.com>
Reviewed-by: Boris Fiuczynski <fiuczy@linux.ibm.com>
Reviewed-by: Erik Skultety <eskultet@redhat.com>
