From 8b93d78c8325f1fba5db98848350f3db43f5e7d5 Mon Sep 17 00:00:00 2001 From: Laine Stump Date: Thu, 4 Jan 2024 20:12:51 -0500 Subject: [PATCH] conf: support manually specifying VFIO variant driver in XML This patch makes it possible to manually specify which VFIO variant driver to use for PCI hostdev device assignment, so that, e.g. you could force use of a VFIO "variant" driver, with e.g. or alternately to force use of the generic vfio-pci driver with when libvirt would have normally (after applying a subsequent patch) found a "better match" for a device in the active kernel's modules.alias file. (The main potential use of this manual override would probably be to work around a bug in a new VFIO variant driver by temporarily not using that driver). Signed-off-by: Laine Stump Reviewed-by: Peter Krempa --- docs/formatdomain.rst | 44 ++++++++++++++----- docs/formatnetwork.rst | 19 ++++---- src/conf/device_conf.c | 10 +++++ src/conf/device_conf.h | 4 ++ src/conf/domain_conf.c | 3 ++ src/conf/network_conf.c | 2 + src/conf/schemas/basictypes.rng | 21 ++++++--- src/conf/virnetworkportdef.c | 1 + src/network/bridge_driver.c | 1 + .../hostdev-pf-driver-model.xml | 8 ++++ .../hostdev-pf-driver-model.xml | 8 ++++ tests/networkxml2xmltest.c | 2 + .../hostdev-vfio.x86_64-latest.args | 5 ++- tests/qemuxml2argvdata/hostdev-vfio.xml | 18 ++++++++ .../hostdev-vfio.x86_64-latest.xml | 23 +++++++++- .../plug-hostdev-pci-unmanaged.xml | 1 - .../plug-hostdev-pci.xml | 1 - 17 files changed, 138 insertions(+), 33 deletions(-) create mode 100644 tests/networkxml2xmlin/hostdev-pf-driver-model.xml create mode 100644 tests/networkxml2xmlout/hostdev-pf-driver-model.xml diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst index 252ba7c17c..9698b96ce9 100644 --- a/docs/formatdomain.rst +++ b/docs/formatdomain.rst @@ -4508,17 +4508,39 @@ or: an error. See the `Device Addresses`_ section for more details on the address element. ``driver`` - PCI devices can have an optional ``driver`` subelement that specifies which - backend driver to use for PCI device assignment. Use the ``name`` attribute - to select either "vfio" (for the new VFIO device assignment backend, which is - compatible with UEFI SecureBoot) or "kvm" (the legacy device assignment - handled directly by the KVM kernel module) :since:`Since 1.0.5 (QEMU and KVM - only, requires kernel 3.6 or newer)` . When specified, device assignment will - fail if the requested method of device assignment isn't available on the - host. When not specified, the default is "vfio" on systems where the VFIO - driver is available and loaded, and "kvm" on older systems, or those where - the VFIO driver hasn't been loaded :since:`Since 1.1.3` (prior to that the - default was always "kvm"). + PCI hostdev devices can have an optional ``driver`` subelement that + specifies which host driver to bind to the device when preparing it + for assignment to a guest. :since:`Since 10.0.0 (useful for QEMU and + KVM only)`. This is done by setting the ```` element's ``model`` + attribute, for example:: + + ... + + + ... + + tells libvirt to bind the driver "vfio-pci-igb" to the device on + the host before handing it off to QEMU for assignment to the + guest. Normally libvirt will bind the device to the "best match" + VFIO-type driver that it finds in the kernel's modules.alias file + (based on matching the corresponding fields of the device's + modalias file in sysfs) or to the generic "vfio-pci" driver if no + better match is found (vfio-pci is always used prior to libvirt + 10.0.0), but in cases when the correct driver isn't listed in + modules.alias then the desired device-specific driver can be forced + by setting driver name, or if the device-specific driver that is + found is "problematic" in some way, the generic vfio-pci driver + similarly be forced. + + (Note: :since:`Since 1.0.5, the ``name`` attribute has been + described to be used to select the type of PCI device assignment + ("vfio", "kvm", or "xen"), but those values have been mostly + useless, since the type of device assignment is actually determined + by which hypservisor is in use. This means that you may + occasionally see ```` or ```` in a domain's status XML, or more rarely in config, + but those specific values are essentially ignored.) + ``readonly`` Indicates that the device is readonly, only supported by SCSI host device now. :since:`Since 1.0.6 (QEMU and KVM only)` diff --git a/docs/formatnetwork.rst b/docs/formatnetwork.rst index 5d300a035e..5335da5e11 100644 --- a/docs/formatnetwork.rst +++ b/docs/formatnetwork.rst @@ -315,17 +315,14 @@ to the physical LAN (if at all). guest, use the traditional ```` device definition. :since:` Since 0.10.0` - To force use of a particular type of device assignment, a interface can have an optional ``driver`` sub-element with - a ``name`` attribute set to either "vfio" (VFIO is a new method of device - assignment that is compatible with UEFI Secure Boot) or "kvm" (the legacy - device assignment handled directly by the KVM kernel module) :since:`Since - 1.0.5 (QEMU and KVM only, requires kernel 3.6 or newer)` . When specified, - device assignment will fail if the requested method of device assignment - isn't available on the host. When not specified, the default is "vfio" on - systems where the VFIO driver is available and loaded, and "kvm" on older - systems, or those where the VFIO driver hasn't been loaded :since:`Since - 1.1.3` (prior to that the default was always "kvm"). + To force use of a particular device-specific VFIO driver when + assigning the devices to a guest, a + interface can have an optional ``driver`` sub-element with a + ``model`` attribute set to the name of the driver to use + :since:`Since 10.0.0 (QEMU only)`. When not specified, libvirt + will attempt to find a suitable VFIO variant driver for the + device, and if not found it will use the generic driver + "vfio-pci". Note that this "intelligent passthrough" of network devices is very similar to the functionality of a standard ```` device, the diff --git a/src/conf/device_conf.c b/src/conf/device_conf.c index 68a8c7690a..f840efc1b5 100644 --- a/src/conf/device_conf.c +++ b/src/conf/device_conf.c @@ -67,6 +67,7 @@ virDeviceHostdevPCIDriverInfoParseXML(xmlNodePtr node, return -1; } + driver->model = virXMLPropString(node, "model"); return 0; } @@ -90,11 +91,20 @@ virDeviceHostdevPCIDriverInfoFormat(virBuffer *buf, virBufferAsprintf(&driverAttrBuf, " name='%s'", driverName); } + virBufferEscapeString(&driverAttrBuf, " model='%s'", driver->model); + virXMLFormatElement(buf, "driver", &driverAttrBuf, NULL); return 0; } +void +virDeviceHostdevPCIDriverInfoClear(virDeviceHostdevPCIDriverInfo *driver) +{ + VIR_FREE(driver->model); +} + + static int virZPCIDeviceAddressParseXML(xmlNodePtr node, virPCIDeviceAddress *addr) diff --git a/src/conf/device_conf.h b/src/conf/device_conf.h index 0b3f17a3aa..2d674ecd85 100644 --- a/src/conf/device_conf.h +++ b/src/conf/device_conf.h @@ -46,6 +46,7 @@ VIR_ENUM_DECL(virDeviceHostdevPCIDriverName); struct _virDeviceHostdevPCIDriverInfo { virDeviceHostdevPCIDriverName name; + char *model; }; typedef enum { @@ -192,6 +193,9 @@ int virDeviceHostdevPCIDriverInfoParseXML(xmlNodePtr node, int virDeviceHostdevPCIDriverInfoFormat(virBuffer *buf, const virDeviceHostdevPCIDriverInfo *driver); +void virDeviceHostdevPCIDriverInfoPostParse(virDeviceHostdevPCIDriverInfo *driver); +void virDeviceHostdevPCIDriverInfoClear(virDeviceHostdevPCIDriverInfo *driver); + void virDomainDeviceInfoClear(virDomainDeviceInfo *info); void virDomainDeviceInfoFree(virDomainDeviceInfo *info); diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index c11129c7ff..0ce2586a6c 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -2638,6 +2638,7 @@ virDomainHostdevDefClear(virDomainHostdevDef *def) VIR_FREE(def->source.subsys.u.scsi_host.wwpn); break; case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI: + virDeviceHostdevPCIDriverInfoClear(&def->source.subsys.u.pci.driver); g_clear_pointer(&def->source.subsys.u.pci.origstates, virBitmapFree); break; case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_USB: @@ -29949,6 +29950,7 @@ virDomainNetDefActualFromNetworkPort(virDomainNetDef *iface, actual->data.hostdev.def.source.subsys.type = VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI; actual->data.hostdev.def.source.subsys.u.pci.addr = port->plug.hostdevpci.addr; actual->data.hostdev.def.source.subsys.u.pci.driver.name = port->plug.hostdevpci.driver.name; + actual->data.hostdev.def.source.subsys.u.pci.driver.model = g_strdup(port->plug.hostdevpci.driver.model); break; case VIR_NETWORK_PORT_PLUG_TYPE_LAST: @@ -30050,6 +30052,7 @@ virDomainNetDefActualToNetworkPort(virDomainDef *dom, port->plug.hostdevpci.managed = virTristateBoolFromBool(actual->data.hostdev.def.managed); port->plug.hostdevpci.addr = actual->data.hostdev.def.source.subsys.u.pci.addr; port->plug.hostdevpci.driver.name = actual->data.hostdev.def.source.subsys.u.pci.driver.name; + port->plug.hostdevpci.driver.model = g_strdup(actual->data.hostdev.def.source.subsys.u.pci.driver.model); break; case VIR_DOMAIN_NET_TYPE_CLIENT: diff --git a/src/conf/network_conf.c b/src/conf/network_conf.c index 890c16b3b1..5c781d06af 100644 --- a/src/conf/network_conf.c +++ b/src/conf/network_conf.c @@ -229,6 +229,8 @@ virNetworkForwardDefClear(virNetworkForwardDef *def) { size_t i; + virDeviceHostdevPCIDriverInfoClear(&def->driver); + for (i = 0; i < def->npfs && def->pfs; i++) virNetworkForwardPfDefClear(&def->pfs[i]); VIR_FREE(def->pfs); diff --git a/src/conf/schemas/basictypes.rng b/src/conf/schemas/basictypes.rng index 8d5f4475ca..b65d210091 100644 --- a/src/conf/schemas/basictypes.rng +++ b/src/conf/schemas/basictypes.rng @@ -658,13 +658,20 @@ - - - kvm - vfio - xen - - + + + + kvm + vfio + xen + + + + + + + + diff --git a/src/conf/virnetworkportdef.c b/src/conf/virnetworkportdef.c index 49d00b2ea6..64db63ae66 100644 --- a/src/conf/virnetworkportdef.c +++ b/src/conf/virnetworkportdef.c @@ -64,6 +64,7 @@ virNetworkPortDefFree(virNetworkPortDef *def) break; case VIR_NETWORK_PORT_PLUG_TYPE_HOSTDEV_PCI: + virDeviceHostdevPCIDriverInfoClear(&def->plug.hostdevpci.driver); break; case VIR_NETWORK_PORT_PLUG_TYPE_LAST: diff --git a/src/network/bridge_driver.c b/src/network/bridge_driver.c index d156333626..9921c7cd14 100644 --- a/src/network/bridge_driver.c +++ b/src/network/bridge_driver.c @@ -3931,6 +3931,7 @@ networkAllocatePort(virNetworkObj *obj, } port->plug.hostdevpci.addr = dev->device.pci; port->plug.hostdevpci.driver.name = netdef->forward.driver.name; + port->plug.hostdevpci.driver.model = g_strdup(netdef->forward.driver.model); port->plug.hostdevpci.managed = virTristateBoolFromBool(netdef->forward.managed); if (port->virtPortProfile) { diff --git a/tests/networkxml2xmlin/hostdev-pf-driver-model.xml b/tests/networkxml2xmlin/hostdev-pf-driver-model.xml new file mode 100644 index 0000000000..848a720d76 --- /dev/null +++ b/tests/networkxml2xmlin/hostdev-pf-driver-model.xml @@ -0,0 +1,8 @@ + + hostdev + 81ff0d90-c91e-6742-64da-4a736edb9a9b + + + + + diff --git a/tests/networkxml2xmlout/hostdev-pf-driver-model.xml b/tests/networkxml2xmlout/hostdev-pf-driver-model.xml new file mode 100644 index 0000000000..848a720d76 --- /dev/null +++ b/tests/networkxml2xmlout/hostdev-pf-driver-model.xml @@ -0,0 +1,8 @@ + + hostdev + 81ff0d90-c91e-6742-64da-4a736edb9a9b + + + + + diff --git a/tests/networkxml2xmltest.c b/tests/networkxml2xmltest.c index b0814c7529..0783d84915 100644 --- a/tests/networkxml2xmltest.c +++ b/tests/networkxml2xmltest.c @@ -146,6 +146,8 @@ mymain(void) DO_TEST_FLAGS("passthrough-pf", VIR_NETWORK_XML_INACTIVE); DO_TEST("hostdev"); DO_TEST_FLAGS("hostdev-pf", VIR_NETWORK_XML_INACTIVE); + DO_TEST_FLAGS("hostdev-pf-driver-model", VIR_NETWORK_XML_INACTIVE); + DO_TEST("passthrough-address-crash"); DO_TEST("nat-network-explicit-flood"); DO_TEST("host-bridge-no-flood"); diff --git a/tests/qemuxml2argvdata/hostdev-vfio.x86_64-latest.args b/tests/qemuxml2argvdata/hostdev-vfio.x86_64-latest.args index c1f9258844..8529cde269 100644 --- a/tests/qemuxml2argvdata/hostdev-vfio.x86_64-latest.args +++ b/tests/qemuxml2argvdata/hostdev-vfio.x86_64-latest.args @@ -32,6 +32,9 @@ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest2/.config \ -device '{"driver":"ide-hd","bus":"ide.0","unit":0,"drive":"libvirt-1-format","id":"ide0-0-0","bootindex":1}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -device '{"driver":"vfio-pci","host":"0000:06:12.1","id":"hostdev0","bus":"pci.0","addr":"0x2"}' \ --device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","addr":"0x3"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:12.2","id":"hostdev1","bus":"pci.0","addr":"0x3"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:12.3","id":"hostdev2","bus":"pci.0","addr":"0x4"}' \ +-device '{"driver":"vfio-pci","host":"0000:06:12.4","id":"hostdev3","bus":"pci.0","addr":"0x5"}' \ +-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","addr":"0x6"}' \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on diff --git a/tests/qemuxml2argvdata/hostdev-vfio.xml b/tests/qemuxml2argvdata/hostdev-vfio.xml index a03870f6e0..812bac2cfd 100644 --- a/tests/qemuxml2argvdata/hostdev-vfio.xml +++ b/tests/qemuxml2argvdata/hostdev-vfio.xml @@ -29,6 +29,24 @@
+ + + +
+ + + + + +
+ + + + + +
+ + diff --git a/tests/qemuxml2xmloutdata/hostdev-vfio.x86_64-latest.xml b/tests/qemuxml2xmloutdata/hostdev-vfio.x86_64-latest.xml index 3915b515f2..2042ba6c16 100644 --- a/tests/qemuxml2xmloutdata/hostdev-vfio.x86_64-latest.xml +++ b/tests/qemuxml2xmloutdata/hostdev-vfio.x86_64-latest.xml @@ -39,8 +39,29 @@
- + + + +
+
+ + + + +
+ +
+ + + + +
+ +
+ + +
diff --git a/tests/virnetworkportxml2xmldata/plug-hostdev-pci-unmanaged.xml b/tests/virnetworkportxml2xmldata/plug-hostdev-pci-unmanaged.xml index da5f568031..fa974affee 100644 --- a/tests/virnetworkportxml2xmldata/plug-hostdev-pci-unmanaged.xml +++ b/tests/virnetworkportxml2xmldata/plug-hostdev-pci-unmanaged.xml @@ -6,7 +6,6 @@ -
diff --git a/tests/virnetworkportxml2xmldata/plug-hostdev-pci.xml b/tests/virnetworkportxml2xmldata/plug-hostdev-pci.xml index cc4419f3fd..7354e1d48c 100644 --- a/tests/virnetworkportxml2xmldata/plug-hostdev-pci.xml +++ b/tests/virnetworkportxml2xmldata/plug-hostdev-pci.xml @@ -6,7 +6,6 @@ -