cloud-hypervisor/docs/vhost-user-blk-testing.md
Yang Zhong 9c7215d620 docs: Add the vhost-user-blk test doc
This doc describe how to use vhost-user-blk in Cloud Hypervisor
with SPDK. With this doc, we can setup simple test environment on
Ubuntu release.

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
2020-04-22 20:18:23 +02:00

4.0 KiB

How to test vhost-user-blk with SPDK

The purpose of this document is to illustrate how to test vhost-user-blk in Cloud Hypervisor with SPDK as the backend.

Framework

It's a simple test to validate the block read/write between VM and block backend.

             +----+----------+          +-------------+-----------+
             |    |          |          |             |           |
             |    |vhost-user|----------| vhost-user  |    dpdk   |
             |    |blk device|          | port 1      |           |
             |    |          |          |             |           |
             |    +----------+          +-------------+-----------+
             |               |          |                         |
             |      vm       |          |           spdk          |
             |               |          |                         |
          +--+----------------------------------------------------+--+
          |  |                  hugepages                         |  |
          |  +----------------------------------------------------+  |
          |                                                          |
          |                       host                               |
          |                                                          |
          +----------------------------------------------------------+

Prerequisites

Prior to running the test, the following steps need to be performed.

  • Enable hugepages
  • Install SPDK

Here are some good references for detailing them.

Test environment

The below test environment is based on ubuntu release(16.04.1 LTS), as for other system, please check related document. The test runs with multiple queue (MQ) support enabled, using 4 queues defined for both SPDK and the virtual machine. Here are the details on how the test can be run.

The hugepages settings in host linux

Add "default_hugepagesz=1G hugepagesz=1G hugepages=2" into host linux cmdline. As for how to change Ubuntu linux cmdline in grub file, please ref below link: https://www.ostechnix.com/configure-grub-2-boot-loader-settings-ubuntu-16-04/ reboot Ubuntu sudo mount -t hugetlbfs -o pagesize=1G none /dev/hugepages

Download the SPDK code

git clone https://github.com/spdk/spdk cd spdk git submodule update --init

Create the build dep

./scripts/pkgdep.sh

Build spdk

./configure make

Set the SPDk environment

sudo HUGEMEM=2048 scripts/setup.sh sudo ./app/vhost/vhost -S /var/tmp -s 1024 -m 0x3 &

Create 512M block device

sudo scripts/rpc.py bdev_malloc_create 512 512 -b Malloc0 sudo scripts/rpc.py vhost_create_blk_controller --cpumask 0x1 vhost.1 Malloc0

Launch the VM

VMs run in client mode. They connect to the socket created by the dpdkvhostuser in the SPDK backend.

# From the test terminal. We need to create one vhost-user-blk device for the --disk.
./cloud-hypervisor \
        --cpus boot=4 \
        --memory size=1024M,file=/dev/hugepages \
        --kernel linux/arch/x86/boot/compressed/vmlinux.bin \
        --cmdline "console=ttyS0 reboot=k panic=1 nomodules i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd root=/dev/vda3 iommu=off" \
        --disk "path=images/clear-kvm.img" "num_queues=4,queue_size=128,vhost_user=true,socket=/var/tmp/vhost.1,wce=true" \
        --console off \
        --serial tty \
        --rng
# How to test the vhost-user-blk device with SPDK backend
login in guest

# Use lsblk command to find out vhost-user-blk device
lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda    253:0    0  8.5G  0 disk
├─vda1 253:1    0  511M  0 part
├─vda2 253:2    0   32M  0 part [SWAP]
└─vda3 253:3    0    8G  0 part /
vdb    253:16   0  512M  0 disk

The vhost-user-blk device is /dev/vdb

# How to do simple read/write test
dd if=/dev/vdb of=/dev/null bs=2M iflag=direct
dd of=/dev/vdb if=/dev/zero bs=2M oflag=direct count=256

If you want to do fio test, please install fio binary into guest. The detailed info is not listed here.