roots/wiki
1
0
Fork 0
Code Pull Requests Activity

docs: update gofurther/vfio-pci

Browse Source
This commit is contained in:
lukas 2025-04-02 16:07:44 +00:00
parent 8dc26a28af
commit ed08eb1029
1 changed files with 51 additions and 60 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

111
gofurther/vfio-pci.md
View File

@ -2,7 +2,7 @@
title: Virtual Function I/O passthrough (vfio-pci) title: Virtual Function I/O passthrough (vfio-pci)
description: Pass a physical device to a guest sysstem description: Pass a physical device to a guest sysstem
published: true published: true
date: 2025-04-01T18:01:48.788Z date: 2025-04-02T16:07:41.270Z
tags: tags:
editor: markdown editor: markdown
dateCreated: 2025-04-01T11:18:43.924Z dateCreated: 2025-04-01T11:18:43.924Z
@ -15,12 +15,12 @@ dateCreated: 2025-04-01T11:18:43.924Z
## Preparation ## Preparation
* [Install](/deploy/install) any version of *Phyllome OS* * [Install](/deploy/install) any version of Phyllome OS
* Make sure the GRUB has been updated after the first boot: * Make sure the GRUB has been updated after the first boot:
`# grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg` `# grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg`
## Isolate the physical device ## Collect information on the physical device to share
* List IOMMU groups and their associated devices (script courtesy of the [Arch Linux wiki](https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#Ensuring_that_the_groups_are_valid): * List IOMMU groups and their associated devices (script courtesy of the [Arch Linux wiki](https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#Ensuring_that_the_groups_are_valid):
@ -65,58 +65,65 @@ IOMMU Group 16:
[...] [...]
``` ```
In the above example, most devices are well isolated, at the exception to the USB controller and the HD audio controller. In the above example, most devices are well isolated in their own IOMMU group, at the exception to the USB controller and the HD audio controller.
> In general, but not always, devices associated to a particular IOMMU group have to be passed through a guest system together > In general, but not always, devices associated to a particular IOMMU group have to be passed through a guest system together
{.is-info} {.is-info}
In our example, Phyllome OS is installed on the Intel SSD 660P NVMe SSD. This device should not be shared with a guest system.
As Phyllome OS is installed on the Intel SSD 660P NVMe SSD, this device should not be shared with a guest system. | IOMMU Group | Bus | Slot | Function | Device | ID |
| --- | --- | --- | --- | --- | --- |
| 0 | 0x00 | 0x02 | 0x0 | Intel Corporation Iris Pro Graphics 580 | 8086:193b |
| 2 | 0x00 | 0x08 | 0x0 | Intel Gaussian Mixture Model - Neural Network Accelerator | 8086:1911 |
| **3** | 0x00 | 0x14 | 0x0 | USB 3.0 xHCI Controller | 8086:a12f |
| **3** | 0x00 | 0x14 | 0x2 | Thermal Subsystem | 8086:a131 |
| 4 | 0x00 | 0x16 | 0x0 | MEI Controller | 8086:a13a |
| **11** | 0x00 | 0x1f | 0x0 | M170 Chipset LPC/eSPI Controller | 8086:a14e |
| **11** | 0x00 | 0x1f | 0x2 | Power Management Controller | 8086:a121 |
| **11** | 0x00 | 0x1f | 0x3 | HD Audio Controller | 8086:a170 |
| **11** | 0x00 | 0x1f | 0x4 | SMBus | 8086:a123 |
| 12 | 0x00 | 0x1f | 0x4 | Ethernet Connection I219-LM | 8086:15b7 |
| 13 | 0x02 | 0x00 | 0x0 | SD/MMC Card Reader Controller | 1217:8621 |
| 14 | 0x03 | 0x00 | 0x0 | Intel Corporation Wireless 8260 | 8086:24f3 |
| 15 | 0x3d | 0x00 | 0x0 | NVMe SSD Controller SM951/PM951 | 144d:a802 |
| IOMMU Group | Bus | Device | ID | Let's focus on sharing the wireless controller, which is conveniently located on a IOMMU group of its own.
| --- | --- | --- | --- |
| 0 | 0x3d | Intel Corporation Iris Pro Graphics 580 | 8086:193b |
| 2 | | Intel Gaussian Mixture Model - Neural Network Accelerator | 8086:1911 |
| **3** | | USB 3.0 xHCI Controller | 8086:a12f |
| **3** | | Thermal Subsystem | 8086:a131 |
| 4 | | MEI Controller | 8086:a13a |
| **11** | | HM170 Chipset LPC/eSPI Controller | 8086:a14e |
| **11** | | Power Management Controller | 8086:a121 |
| **11** | | HD Audio Controller | 8086:a170 |
| **11** | 0x1f | SMBus | 8086:a123 |
| 12 | 0x1d | Ethernet Connection I219-LM | 8086:15b7 |
| 13 | 0x02 | SD/MMC Card Reader Controller | 1217:8621 |
| 14 | 0x03 | Intel Corporation Wireless 8260 | 8086:24f3 |
| 15 | 0x3d | NVMe SSD Controller SM951/PM951 | 144d:a802 |
| 16 | 0x3e | Intel Corporation SSD 660P Series | 8086:f1a8 |
``` ```
<hostdev mode="subsystem" type="pci" managed="yes"> IOMMU Group 14:
<source> 03:00.0 Network controller [0280]: Intel Corporation Wireless 8260 [8086:24f3] (rev 3a)
<address domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
</source>
</hostdev>
<hostdev mode="subsystem" type="pci" managed="yes">
<source>
<address domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
</source>
</hostdev>
<hostdev mode="subsystem" type="pci" managed="yes">
<source>
<address domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
</source>
</hostdev>
<hostdev mode="subsystem" type="pci" managed="yes">
<source>
<address domain="0x0000" bus="0x3d" slot="0x00" function="0x0"/>
</source>
</hostdev>
``` ```
- Let's have a look inside the guest system that has the physical devices attached to it: The first digits corresponds to the bus, slot and function associated to the device.
| IOMMU Group | Bus | Slot | Function | Device | ID |
| --- | --- | --- | --- | --- | --- |
| 14 | 0x03 | 0x00 | 0x0 | Intel Corporation Wireless 8260 | 8086:24f3 |
In order to share such wireless card with a guest, edit the XML file associated to the domain and add the following:
```
<domain type="kvm">
[...]
<devices>
[...]
<hostdev mode="subsystem" type="pci" managed="yes">
<source>
<address domain="0x0000" bus="0x00" slot="0x14" function="0x0"/>
</source>
</hostdev>
[...]
</devices>
[...]
</domain>
```
- Start the domain and have a look inside the guest system that has the physical devices attached to it:
``` ```
$ lspci $ lspci
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller 00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
00:01.0 VGA compatible controller: Red Hat, Inc. Virtio 1.0 GPU (rev 01) 00:01.0 VGA compatible controller: Red Hat, Inc. Virtio 1.0 GPU (rev 01)
00:02.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port 00:02.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
@ -127,26 +134,10 @@ $ lspci
09:00.0 Network controller: Intel Corporation Wireless 8260 (rev 3a) 09:00.0 Network controller: Intel Corporation Wireless 8260 (rev 3a)
0a:00.0 SD Host controller: O2 Micro, Inc. SD/MMC Card Reader Controller (rev 01) 0a:00.0 SD Host controller: O2 Micro, Inc. SD/MMC Card Reader Controller (rev 01)
0b:00.0 PCI bridge: Red Hat, Inc. Device 000e 0b:00.0 PCI bridge: Red Hat, Inc. Device 000e
0c:01.0 System peripheral: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model 09:00.0 Network controller: Intel Corporation Wireless 8260 (rev 3a)
0d:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 (rev 01)
``` ```
## Add a display device The wireless controller has been attached to bus 9 of the virtual chipset.
## Troubleshooting
### Low number of available vGPU instances
### Not all vGPUs are marked as active
## Resources
* Official page for vfio-mdev: https://www.kernel.org/doc/html/latest/driver-api/vfio-mediated-device.html
* Arch Linux's *must-read entry* on Intel GVT-g: https://wiki.archlinux.org/title/Intel_GVT-g
* DMA-BUF Linux documentation: https://www.kernel.org/doc/html/latest/driver-api/dma-buf.html
--- ---