docs: Add documentation for vDPA

Explain the reason why using vDPA might be interesting and how to use it
with Cloud Hypervisor.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>

docs/vdpa.md

@@ -0,0 +1,154 @@
+# Virtio Data Path Acceleration
+
+vDPA aims at achieving bare-metal performance for devices passed into a virtual
+machine. It is an alternative to VFIO, as it provides a simpler solution for
+achieving migration.
+
+It is a kernel framework introduced recently to handle devices complying with
+the VIRTIO specification on their data-path, while the control path is vendor
+specific. In practice, virtqueues are accessed directly through DMA mechanism
+between the hardware and the guest. The control path is accessed through the
+vDPA framework, being exposed through the vhost interface as a vhost-vdpa
+device.
+
+Because DMA accesses between device and guest are going through virtqueues,
+migration can be achieved without requiring device's driver to implement any
+specific migration support. In case of VFIO, each vendor is expected to provide
+an implementation of the VFIO migration framework, complicating things as it
+must be done for each and every device's driver.
+
+The official [website](https://vdpa-dev.gitlab.io/) contains some extensive
+documentation on the topic.
+
+## Usage
+
+`VdpaConfig` (known as `--vdpa` from the CLI perspective) contains the list of
+parameters available for the vDPA device.
+
+```rust
+struct VdpaConfig {
+    path: PathBuf,
+    num_queues: usize,
+    id: Option<String>,
+    pci_segment: u16,
+}
+```
+
+```
+--vdpa <vdpa>	vDPA device "path=<device_path>,num_queues=<number_of_queues>,iommu=on|off,id=<device_id>,pci_segment=<segment_id>"
+```
+
+### `path`
+
+Path of the vDPA device. Usually `/dev/vhost-vdpa-X`.
+
+This parameter is mandatory.
+
+Value is a string.
+
+_Example_
+
+```
+--vdpa path=/dev/vhost-vdpa-0
+```
+
+### `num_queues`
+
+Number of virtqueues supported by the vDPA device.
+
+This parameter is optional.
+
+Value is an unsigned integer set to `1` by default.
+
+_Example_
+
+```
+--vdpa path=/dev/vhost-vdpa-0,num_queues=2
+```
+
+### `id`
+
+Identifier of the vDPA device.
+
+This parameter is optional. If provided, it must be unique across the entire
+virtual machine.
+
+Value is a string.
+
+_Example_
+
+```
+--vdpa path=/dev/vhost-vdpa-0,id=vdpa0
+```
+
+### `pci_segment`
+
+PCI segment number to which the vDPA device should be attached to.
+
+This parameter is optional.
+
+Value is an unsigned integer of 16 bits set to `0` by default.
+
+_Example_
+
+```
+--vdpa path=/dev/vhost-vdpa-0,pci_segment=1
+```
+
+## Example with vDPA block simulator
+
+The vDPA framework provides a simulator with both `virtio-block` and
+`virtio-net` implementations. This is very useful for testing vDPA when we
+don't have access to the specific hardware.
+
+Given the host kernel has the appropriate modules available, let's load them
+all:
+
+```
+sudo modprobe vdpa
+sudo modprobe vhost_vdpa
+sudo modprobe vdpa_sim
+sudo modprobe vdpa_sim_blk
+```
+
+Given you have the `iproute2/vdpa` tool installed, let's now create the
+`virtio-block` vDPA device:
+
+```sh
+sudo vdpa dev add name vdpa-blk1 mgmtdev vdpasim_blk
+sudo chown $USER:$USER /dev/vhost-vdpa-0
+sudo chmod 660 /dev/vhost-vdpa-0
+```
+
+Increase the maximum locked memory to ensure setting up IOMMU mappings will
+succeed:
+
+```sh
+ulimit -l unlimited
+```
+
+Start Cloud Hypervisor:
+
+```sh
+cloud-hypervisor \
+    --cpus boot=1 \
+    --memory size=1G,hugepages=on \
+    --disk path=focal-server-cloudimg-amd64.raw \
+    --kernel vmlinux \
+    --cmdline "root=/dev/vda1 console=hvc0" \
+    --vdpa path=/dev/vhost-vdpa-0,num_queues=1
+```
+
+The `virtio-block` device backed by the vDPA simulator can be found as
+`/dev/vdb` in the guest:
+
+```
+cloud@cloud:~$ lsblk
+NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
+nullb0  252:0    0  250G  0 disk 
+vda     254:0    0  2.2G  0 disk 
+├─vda1  254:1    0  2.1G  0 part /
+├─vda14 254:14   0    4M  0 part 
+└─vda15 254:15   0  106M  0 part /boot/efi
+vdb     254:16   0  128M  0 disk
+```