conf: support manually specifying VFIO variant driver in <hostdev> XML

This patch makes it possible to manually specify which VFIO variant
driver to use for PCI hostdev device assignment, so that, e.g. you
could force use of a VFIO "variant" driver, with e.g.

  <driver model='mlx5_vfio_pci'/>

or alternately to force use of the generic vfio-pci driver with

  <driver model='vfio-pci'/>

when libvirt would have normally (after applying a subsequent patch)
found a "better match" for a device in the active kernel's
modules.alias file. (The main potential use of this manual override
would probably be to work around a bug in a new VFIO variant driver by
temporarily not using that driver).

Signed-off-by: Laine Stump <laine@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>

docs/formatdomain.rst

@@ -4508,17 +4508,39 @@ or:
    an error. See the `Device Addresses`_ section for more details on the address
    element.
 ``driver``
-   PCI devices can have an optional ``driver`` subelement that specifies which
+   PCI hostdev devices can have an optional ``driver`` subelement that
-   backend driver to use for PCI device assignment. Use the ``name`` attribute
+   specifies which host driver to bind to the device when preparing it
-   to select either "vfio" (for the new VFIO device assignment backend, which is
+   for assignment to a guest. :since:`Since 10.0.0 (useful for QEMU and
-   compatible with UEFI SecureBoot) or "kvm" (the legacy device assignment
+   KVM only)`. This is done by setting the ``<driver>`` element's ``model``
-   handled directly by the KVM kernel module) :since:`Since 1.0.5 (QEMU and KVM
+   attribute, for example::
-   only, requires kernel 3.6 or newer)` . When specified, device assignment will
+
-   fail if the requested method of device assignment isn't available on the
+     ...
-   host. When not specified, the default is "vfio" on systems where the VFIO
+       <hostdev mode='subsystem' type='pci' managed='yes'>
-   driver is available and loaded, and "kvm" on older systems, or those where
+         <driver model='vfio-pci-igb'/>
-   the VFIO driver hasn't been loaded :since:`Since 1.1.3` (prior to that the
+     ...
-   default was always "kvm").
+
+   tells libvirt to bind the driver "vfio-pci-igb" to the device on
+   the host before handing it off to QEMU for assignment to the
+   guest. Normally libvirt will bind the device to the "best match"
+   VFIO-type driver that it finds in the kernel's modules.alias file
+   (based on matching the corresponding fields of the device's
+   modalias file in sysfs) or to the generic "vfio-pci" driver if no
+   better match is found (vfio-pci is always used prior to libvirt
+   10.0.0), but in cases when the correct driver isn't listed in
+   modules.alias then the desired device-specific driver can be forced
+   by setting driver name, or if the device-specific driver that is
+   found is "problematic" in some way, the generic vfio-pci driver
+   similarly be forced.
+
+   (Note: :since:`Since 1.0.5, the ``name`` attribute has been
+   described to be used to select the type of PCI device assignment
+   ("vfio", "kvm", or "xen"), but those values have been mostly
+   useless, since the type of device assignment is actually determined
+   by which hypservisor is in use. This means that you may
+   occasionally see ``<driver name='vfio'/>`` or ``<driver
+   name='xen'/>`` in a domain's status XML, or more rarely in config,
+   but those specific values are essentially ignored.)
+
 ``readonly``
    Indicates that the device is readonly, only supported by SCSI host device
    now. :since:`Since 1.0.6 (QEMU and KVM only)`

docs/formatnetwork.rst

@@ -315,17 +315,14 @@ to the physical LAN (if at all).
       guest, use the traditional ``<hostdev>`` device definition. :since:` Since
       0.10.0`
 
-      To force use of a particular type of device assignment, a <forward
+      To force use of a particular device-specific VFIO driver when
-      type='hostdev'> interface can have an optional ``driver`` sub-element with
+      assigning the devices to a guest, a <forward type='hostdev'>
-      a ``name`` attribute set to either "vfio" (VFIO is a new method of device
+      interface can have an optional ``driver`` sub-element with a
-      assignment that is compatible with UEFI Secure Boot) or "kvm" (the legacy
+      ``model`` attribute set to the name of the driver to use
-      device assignment handled directly by the KVM kernel module) :since:`Since
+      :since:`Since 10.0.0 (QEMU only)`. When not specified, libvirt
-      1.0.5 (QEMU and KVM only, requires kernel 3.6 or newer)` . When specified,
+      will attempt to find a suitable VFIO variant driver for the
-      device assignment will fail if the requested method of device assignment
+      device, and if not found it will use the generic driver
-      isn't available on the host. When not specified, the default is "vfio" on
+      "vfio-pci".
-      systems where the VFIO driver is available and loaded, and "kvm" on older
-      systems, or those where the VFIO driver hasn't been loaded :since:`Since
-      1.1.3` (prior to that the default was always "kvm").
 
       Note that this "intelligent passthrough" of network devices is very
       similar to the functionality of a standard ``<hostdev>`` device, the

src/conf/device_conf.c

@@ -67,6 +67,7 @@ virDeviceHostdevPCIDriverInfoParseXML(xmlNodePtr node,
         return -1;
     }
 
+    driver->model = virXMLPropString(node, "model");
     return 0;
 }
 
@@ -90,11 +91,20 @@ virDeviceHostdevPCIDriverInfoFormat(virBuffer *buf,
         virBufferAsprintf(&driverAttrBuf, " name='%s'", driverName);
     }
 
+    virBufferEscapeString(&driverAttrBuf, " model='%s'", driver->model);
+
     virXMLFormatElement(buf, "driver", &driverAttrBuf, NULL);
     return 0;
 }
 
 
+void
+virDeviceHostdevPCIDriverInfoClear(virDeviceHostdevPCIDriverInfo *driver)
+{
+    VIR_FREE(driver->model);
+}
+
+
 static int
 virZPCIDeviceAddressParseXML(xmlNodePtr node,
                              virPCIDeviceAddress *addr)

src/conf/device_conf.h

@@ -46,6 +46,7 @@ VIR_ENUM_DECL(virDeviceHostdevPCIDriverName);
 
 struct _virDeviceHostdevPCIDriverInfo {
     virDeviceHostdevPCIDriverName name;
+    char *model;
 };
 
 typedef enum {
@@ -192,6 +193,9 @@ int virDeviceHostdevPCIDriverInfoParseXML(xmlNodePtr node,
 int virDeviceHostdevPCIDriverInfoFormat(virBuffer *buf,
                                         const virDeviceHostdevPCIDriverInfo *driver);
 
+void virDeviceHostdevPCIDriverInfoPostParse(virDeviceHostdevPCIDriverInfo *driver);
+void virDeviceHostdevPCIDriverInfoClear(virDeviceHostdevPCIDriverInfo *driver);
+
 void virDomainDeviceInfoClear(virDomainDeviceInfo *info);
 void virDomainDeviceInfoFree(virDomainDeviceInfo *info);
 

src/conf/domain_conf.c

@@ -2638,6 +2638,7 @@ virDomainHostdevDefClear(virDomainHostdevDef *def)
             VIR_FREE(def->source.subsys.u.scsi_host.wwpn);
             break;
         case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI:
+            virDeviceHostdevPCIDriverInfoClear(&def->source.subsys.u.pci.driver);
             g_clear_pointer(&def->source.subsys.u.pci.origstates, virBitmapFree);
             break;
         case VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_USB:
@@ -29949,6 +29950,7 @@ virDomainNetDefActualFromNetworkPort(virDomainNetDef *iface,
         actual->data.hostdev.def.source.subsys.type = VIR_DOMAIN_HOSTDEV_SUBSYS_TYPE_PCI;
         actual->data.hostdev.def.source.subsys.u.pci.addr = port->plug.hostdevpci.addr;
         actual->data.hostdev.def.source.subsys.u.pci.driver.name = port->plug.hostdevpci.driver.name;
+        actual->data.hostdev.def.source.subsys.u.pci.driver.model = g_strdup(port->plug.hostdevpci.driver.model);
         break;
 
     case VIR_NETWORK_PORT_PLUG_TYPE_LAST:
@@ -30050,6 +30052,7 @@ virDomainNetDefActualToNetworkPort(virDomainDef *dom,
         port->plug.hostdevpci.managed = virTristateBoolFromBool(actual->data.hostdev.def.managed);
         port->plug.hostdevpci.addr = actual->data.hostdev.def.source.subsys.u.pci.addr;
         port->plug.hostdevpci.driver.name = actual->data.hostdev.def.source.subsys.u.pci.driver.name;
+        port->plug.hostdevpci.driver.model = g_strdup(actual->data.hostdev.def.source.subsys.u.pci.driver.model);
         break;
 
     case VIR_DOMAIN_NET_TYPE_CLIENT:

src/conf/network_conf.c

@@ -229,6 +229,8 @@ virNetworkForwardDefClear(virNetworkForwardDef *def)
 {
     size_t i;
 
+    virDeviceHostdevPCIDriverInfoClear(&def->driver);
+
     for (i = 0; i < def->npfs && def->pfs; i++)
         virNetworkForwardPfDefClear(&def->pfs[i]);
     VIR_FREE(def->pfs);

src/conf/schemas/basictypes.rng

@@ -658,13 +658,20 @@
 
   <define name="hostdevDriver">
     <element name="driver">
-      <attribute name="name">
+      <optional>
-        <choice>
+        <attribute name="name">
-          <value>kvm</value>
+          <choice>
-          <value>vfio</value>
+            <value>kvm</value>
-          <value>xen</value>
+            <value>vfio</value>
-        </choice>
+            <value>xen</value>
-      </attribute>
+          </choice>
+        </attribute>
+      </optional>
+      <optional>
+        <attribute name="model">
+          <ref name="genericName"/>
+        </attribute>
+      </optional>
       <empty/>
     </element>
   </define>

src/conf/virnetworkportdef.c

@@ -64,6 +64,7 @@ virNetworkPortDefFree(virNetworkPortDef *def)
         break;
 
     case VIR_NETWORK_PORT_PLUG_TYPE_HOSTDEV_PCI:
+        virDeviceHostdevPCIDriverInfoClear(&def->plug.hostdevpci.driver);
         break;
 
     case VIR_NETWORK_PORT_PLUG_TYPE_LAST:

src/network/bridge_driver.c

@@ -3931,6 +3931,7 @@ networkAllocatePort(virNetworkObj *obj,
         }
         port->plug.hostdevpci.addr = dev->device.pci;
         port->plug.hostdevpci.driver.name = netdef->forward.driver.name;
+        port->plug.hostdevpci.driver.model = g_strdup(netdef->forward.driver.model);
         port->plug.hostdevpci.managed = virTristateBoolFromBool(netdef->forward.managed);
 
         if (port->virtPortProfile) {

tests/networkxml2xmlin/hostdev-pf-driver-model.xml

@@ -0,0 +1,8 @@
+<network>
+  <name>hostdev</name>
+  <uuid>81ff0d90-c91e-6742-64da-4a736edb9a9b</uuid>
+  <forward mode='hostdev' managed='yes'>
+    <driver model='vfio-pci-igb'/>
+    <pf dev='eth2'/>
+  </forward>
+</network>

tests/networkxml2xmlout/hostdev-pf-driver-model.xml

@@ -0,0 +1,8 @@
+<network>
+  <name>hostdev</name>
+  <uuid>81ff0d90-c91e-6742-64da-4a736edb9a9b</uuid>
+  <forward mode='hostdev' managed='yes'>
+    <driver model='vfio-pci-igb'/>
+    <pf dev='eth2'/>
+  </forward>
+</network>

tests/networkxml2xmltest.c

@@ -146,6 +146,8 @@ mymain(void)
     DO_TEST_FLAGS("passthrough-pf", VIR_NETWORK_XML_INACTIVE);
     DO_TEST("hostdev");
     DO_TEST_FLAGS("hostdev-pf", VIR_NETWORK_XML_INACTIVE);
+    DO_TEST_FLAGS("hostdev-pf-driver-model", VIR_NETWORK_XML_INACTIVE);
+
     DO_TEST("passthrough-address-crash");
     DO_TEST("nat-network-explicit-flood");
     DO_TEST("host-bridge-no-flood");

tests/qemuxml2argvdata/hostdev-vfio.x86_64-latest.args

@@ -32,6 +32,9 @@ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain--1-QEMUGuest2/.config \
 -device '{"driver":"ide-hd","bus":"ide.0","unit":0,"drive":"libvirt-1-format","id":"ide0-0-0","bootindex":1}' \
 -audiodev '{"id":"audio1","driver":"none"}' \
 -device '{"driver":"vfio-pci","host":"0000:06:12.1","id":"hostdev0","bus":"pci.0","addr":"0x2"}' \
--device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","addr":"0x3"}' \
+-device '{"driver":"vfio-pci","host":"0000:06:12.2","id":"hostdev1","bus":"pci.0","addr":"0x3"}' \
+-device '{"driver":"vfio-pci","host":"0000:06:12.3","id":"hostdev2","bus":"pci.0","addr":"0x4"}' \
+-device '{"driver":"vfio-pci","host":"0000:06:12.4","id":"hostdev3","bus":"pci.0","addr":"0x5"}' \
+-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","addr":"0x6"}' \
 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
 -msg timestamp=on

tests/qemuxml2argvdata/hostdev-vfio.xml

@@ -29,6 +29,24 @@
         <address domain='0x0000' bus='0x06' slot='0x12' function='0x1'/>
       </source>
     </hostdev>
+    <hostdev mode='subsystem' type='pci' managed='yes'>
+      <driver name='vfio'/>
+      <source>
+        <address domain='0x0000' bus='0x06' slot='0x12' function='0x2'/>
+      </source>
+    </hostdev>
+    <hostdev mode='subsystem' type='pci' managed='yes'>
+      <driver model='vfio-pci-igb'/>
+      <source>
+        <address domain='0x0000' bus='0x06' slot='0x12' function='0x3'/>
+      </source>
+    </hostdev>
+    <hostdev mode='subsystem' type='pci' managed='yes'>
+      <driver name='vfio' model='vfio-pci-igb'/>
+      <source>
+        <address domain='0x0000' bus='0x06' slot='0x12' function='0x4'/>
+      </source>
+    </hostdev>
     <memballoon model='virtio'/>
   </devices>
 </domain>

tests/qemuxml2xmloutdata/hostdev-vfio.x86_64-latest.xml

@@ -39,8 +39,29 @@
       </source>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
     </hostdev>
-    <memballoon model='virtio'>
+    <hostdev mode='subsystem' type='pci' managed='yes'>
+      <driver name='vfio'/>
+      <source>
+        <address domain='0x0000' bus='0x06' slot='0x12' function='0x2'/>
+      </source>
       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
+    </hostdev>
+    <hostdev mode='subsystem' type='pci' managed='yes'>
+      <driver model='vfio-pci-igb'/>
+      <source>
+        <address domain='0x0000' bus='0x06' slot='0x12' function='0x3'/>
+      </source>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
+    </hostdev>
+    <hostdev mode='subsystem' type='pci' managed='yes'>
+      <driver name='vfio' model='vfio-pci-igb'/>
+      <source>
+        <address domain='0x0000' bus='0x06' slot='0x12' function='0x4'/>
+      </source>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
+    </hostdev>
+    <memballoon model='virtio'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
     </memballoon>
   </devices>
 </domain>

tests/virnetworkportxml2xmldata/plug-hostdev-pci-unmanaged.xml

@@ -6,7 +6,6 @@
   </owner>
   <mac address='52:54:00:7b:35:93'/>
   <plug type='hostdev-pci' managed='no'>
-    <driver name='vfio'/>
     <address domain='0x0001' bus='0x02' slot='0x03' function='0x4'/>
   </plug>
 </networkport>

tests/virnetworkportxml2xmldata/plug-hostdev-pci.xml

@@ -6,7 +6,6 @@
   </owner>
   <mac address='52:54:00:7b:35:93'/>
   <plug type='hostdev-pci' managed='yes'>
-    <driver name='vfio'/>
     <address domain='0x0001' bus='0x02' slot='0x03' function='0x4'/>
   </plug>
 </networkport>