mirror of
https://github.com/cloud-hypervisor/cloud-hypervisor.git
synced 2024-07-02 16:02:37 +00:00
fa22cb0be5
Reverts commit a0922930b1
and update to latest changes.
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
123 lines
4.1 KiB
Markdown
123 lines
4.1 KiB
Markdown
# How to test vhost-user-blk with SPDK
|
|
|
|
The purpose of this document is to illustrate how to test vhost-user-blk in Cloud Hypervisor with SPDK as the backend.
|
|
|
|
## Framework
|
|
|
|
It's a simple test to validate the block read/write between VM and block backend.
|
|
```
|
|
+----+----------+ +-------------+-----------+
|
|
| | | | | |
|
|
| |vhost-user|----------| vhost-user | dpdk |
|
|
| |blk device| | port 1 | |
|
|
| | | | | |
|
|
| +----------+ +-------------+-----------+
|
|
| | | |
|
|
| vm | | spdk |
|
|
| | | |
|
|
+--+----------------------------------------------------+--+
|
|
| | hugepages | |
|
|
| +----------------------------------------------------+ |
|
|
| |
|
|
| host |
|
|
| |
|
|
+----------------------------------------------------------+
|
|
```
|
|
## Prerequisites
|
|
|
|
Prior to running the test, the following steps need to be performed.
|
|
- Enable hugepages
|
|
- Install SPDK
|
|
|
|
Here are some good references for detailing them.
|
|
- spdk
|
|
* https://spdk.io/doc/
|
|
|
|
## Test environment
|
|
|
|
The below test environment is based on Ubuntu release (20.04.1 LTS), as for other system, please check related documents.
|
|
This test runs with multiple queue (MQ) support enabled, using 4 queues defined for both SPDK and the virtual machine.
|
|
Here are the detailed instructions.
|
|
|
|
### The hugepages settings on the host
|
|
Allocate enough persistent huge pages (e.g. 2GiB) on the host:
|
|
|
|
```bash
|
|
# Assume the default hugepage size is 2MiB
|
|
echo 1024 | sudo tee /proc/sys/vm/nr_hugepages
|
|
# Confirm the value of "Hugepagesize" and "HugePages_Total"
|
|
cat /proc/meminfo | grep -i huge
|
|
```
|
|
|
|
### Download the SPDK code
|
|
```bash
|
|
git clone https://github.com/spdk/spdk
|
|
cd spdk
|
|
git submodule update --init
|
|
```
|
|
|
|
### Create the build dep
|
|
|
|
```bash
|
|
./scripts/pkgdep.sh
|
|
```
|
|
|
|
### Build SPDK
|
|
```bash
|
|
./configure
|
|
make
|
|
```
|
|
|
|
### Set the SPDK environment
|
|
|
|
Run SPDK vhost target with 2 CPU cores, i.e., core 0 and 1.
|
|
```bash
|
|
sudo HUGEMEM=2048 scripts/setup.sh
|
|
sudo ./build/bin/vhost -S /var/tmp -s 1024 -m 0x3 &
|
|
```
|
|
|
|
### Create 512M block device
|
|
|
|
Create a 512M (first parameter) block device with 512 bytes (second parameter) block size.
|
|
```bash
|
|
sudo scripts/rpc.py bdev_malloc_create 512 512 -b Malloc0
|
|
sudo scripts/rpc.py vhost_create_blk_controller --cpumask 0x1 vhost.1 Malloc0
|
|
```
|
|
|
|
### Launch the VM
|
|
|
|
VMs run in client mode. They connect to the socket created by the `dpdkvhostuser` in the SPDK backend.
|
|
```bash
|
|
# From the test terminal. We need to create one vhost-user-blk device for the --disk.
|
|
./cloud-hypervisor \
|
|
--cpus boot=4 \
|
|
--memory size=1024M,hugepages=on,shared=true \
|
|
--kernel linux/arch/x86/boot/compressed/vmlinux.bin \
|
|
--cmdline "console=ttyS0 root=/dev/vda1 rw iommu=off" \
|
|
--disk path=images/focal-server-cloudimg-amd64.raw vhost_user=true,socket=/var/tmp/vhost.1,num_queues=4,queue_size=128 \
|
|
--console off \
|
|
--serial tty \
|
|
--rng
|
|
```
|
|
|
|
```bash
|
|
# How to test the vhost-user-blk device with SPDK backend
|
|
login in guest
|
|
|
|
# Use lsblk command to find out vhost-user-blk device
|
|
lsblk
|
|
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
|
|
vda 252:0 0 2.2G 0 disk
|
|
├─vda1 252:1 0 2.1G 0 part /
|
|
├─vda14 252:14 0 4M 0 part
|
|
└─vda15 252:15 0 106M 0 part /boot/efi
|
|
vdb 253:16 0 512M 0 disk
|
|
|
|
The vhost-user-blk device is /dev/vdb
|
|
|
|
# How to do simple read/write test
|
|
dd if=/dev/vdb of=/dev/null bs=2M iflag=direct
|
|
dd of=/dev/vdb if=/dev/zero bs=2M oflag=direct count=256
|
|
|
|
If you want to do fio test, please install fio binary into guest. The detailed info is not listed here.
|