Commit Graph

87 Commits

Author SHA1 Message Date
Rob Bradford
bde4f735ab ch-remote: Refactor HTTP response handling
Extract HTTP response handling into its own function.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-11 08:22:09 +01:00
Rob Bradford
ba8cd4d55a bin: Introduce "ch-remote" for controlling VMM
This commit introduces a basic implementation of a remote control of a
running VMM implementing a subset of the API.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-09 15:03:01 +00:00
Cathy Zhang
6341736286 vhost_user_net: Provide tap option for vhost_user_net backend
Provide vhost_user_net backend with the tap option, it allows to
use the existing tap interface.

Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
2020-03-05 15:09:20 +00:00
Sergio Lopez
531f4ff6b0 vhost_user_fs: Remove an unneeded unwrap in handle_event
Remove an unneeded unwrap in handle_event.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-02-26 12:08:12 +01:00
Sergio Lopez
e52129efb4 vhost_user_fs: Process events from HIPRIO queue
We weren't processing events arriving at the HIPRIO queue, which
implied ignoring FUSE_INTERRUPT, FUSE_FORGET, and FUSE_BATCH_FORGET
requests.

One effect of this issue was that file descriptors weren't closed on
the server, so it eventually hits RLIMIT_NOFILE. Additionally, the
guest OS may hang while attempting to unmount the filesystem.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-02-26 12:08:12 +01:00
Sergio Lopez
1c5562b656 vhost_user_fs: Add support for EVENT_IDX
Now that Queue supports EVENT_IDX, expose the feature and add support
for it in vhost_user_fs.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-02-25 11:12:50 +00:00
Sergio Lopez
eae4f1d249 vhost_user_fs: Add support for indirect descriptors
Now that Queue supports indirect descriptors, expose the feature and
support them in vhost_user_fs too.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-02-25 11:12:50 +00:00
Sergio Lopez
ea0bc240fd vhost_user_fs: Be honest about protocol supported features
vhost_user_fs doesn't really support all vhost protocol features, just
MQ and SLAVE_REQ, so return that in protocol_features().

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-02-25 11:12:50 +00:00
Sergio Lopez
5c06b7f862 vhost_user_block: Implement optional static polling
Actively polling the virtqueue significantly reduces the latency of
each I/O operation, at the expense of using more CPU time. This
features is specially useful when using low-latency devices (SSD,
NVMe) as the backend.

This change implements static polling. When a request arrives after
being idle, vhost_user_block will keep checking the virtqueue for new
requests, until POLL_QUEUE_US (50us) has passed without finding one.

POLL_QUEUE_US is defined to be 50us, based on the current latency of
enterprise SSDs (< 30us) and the overhead of the emulation.

This feature is enabled by default, and can be disabled by using the
"poll_queue" parameter of "block-backend".

This is a test using null_blk as a backend for the image, with the
following parameters:

 - null_blk gb=20 nr_devices=1 irqmode=2 completion_nsec=0 no_sched=1

With "poll_queue=false":

fio --ioengine=sync --bs=4k --rw randread --name randread --direct=1
--filename=/dev/vdb --time_based --runtime=10

randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=sync, iodepth=1
fio-3.14
Starting 1 process
Jobs: 1 (f=1): [r(1)][100.0%][r=169MiB/s][r=43.2k IOPS][eta 00m:00s]
randread: (groupid=0, jobs=1): err= 0: pid=433: Tue Feb 18 11:12:59 2020
  read: IOPS=43.2k, BW=169MiB/s (177MB/s)(1688MiB/10001msec)
    clat (usec): min=17, max=836, avg=21.64, stdev= 3.81
     lat (usec): min=17, max=836, avg=21.77, stdev= 3.81
    clat percentiles (nsec):
     |  1.00th=[19328],  5.00th=[19840], 10.00th=[20352], 20.00th=[21120],
     | 30.00th=[21376], 40.00th=[21376], 50.00th=[21376], 60.00th=[21632],
     | 70.00th=[21632], 80.00th=[21888], 90.00th=[22144], 95.00th=[22912],
     | 99.00th=[28544], 99.50th=[30336], 99.90th=[39168], 99.95th=[42752],
     | 99.99th=[71168]
   bw (  KiB/s): min=168440, max=188496, per=100.00%, avg=172912.00, stdev=3975.63, samples=19
   iops        : min=42110, max=47124, avg=43228.00, stdev=993.91, samples=19
  lat (usec)   : 20=5.90%, 50=94.08%, 100=0.02%, 250=0.01%, 500=0.01%
  lat (usec)   : 750=0.01%, 1000=0.01%
  cpu          : usr=10.35%, sys=25.82%, ctx=432417, majf=0, minf=10
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=432220,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=169MiB/s (177MB/s), 169MiB/s-169MiB/s (177MB/s-177MB/s), io=1688MiB (1770MB), run=10001-10001msec

Disk stats (read/write):
  vdb: ios=427867/0, merge=0/0, ticks=7346/0, in_queue=0, util=99.04%

With "poll_queue=true" (default):

fio --ioengine=sync --bs=4k --rw randread --name randread --direct=1
--filename=/dev/vdb --time_based --runtime=10

randread: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=sync, iodepth=1
fio-3.14
Starting 1 process
Jobs: 1 (f=1): [r(1)][100.0%][r=260MiB/s][r=66.7k IOPS][eta 00m:00s]
randread: (groupid=0, jobs=1): err= 0: pid=422: Tue Feb 18 11:14:47 2020
  read: IOPS=68.5k, BW=267MiB/s (280MB/s)(2674MiB/10001msec)
    clat (usec): min=10, max=966, avg=13.60, stdev= 3.49
     lat (usec): min=10, max=966, avg=13.70, stdev= 3.50
    clat percentiles (nsec):
     |  1.00th=[11200],  5.00th=[11968], 10.00th=[11968], 20.00th=[12224],
     | 30.00th=[12992], 40.00th=[13504], 50.00th=[13760], 60.00th=[13888],
     | 70.00th=[14016], 80.00th=[14144], 90.00th=[14272], 95.00th=[14656],
     | 99.00th=[20352], 99.50th=[23936], 99.90th=[35072], 99.95th=[36096],
     | 99.99th=[47872]
   bw (  KiB/s): min=265456, max=296456, per=100.00%, avg=274229.05, stdev=13048.14, samples=19
   iops        : min=66364, max=74114, avg=68557.26, stdev=3262.03, samples=19
  lat (usec)   : 20=98.84%, 50=1.15%, 100=0.01%, 250=0.01%, 500=0.01%
  lat (usec)   : 750=0.01%, 1000=0.01%
  cpu          : usr=8.24%, sys=21.15%, ctx=684669, majf=0, minf=10
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=684611,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=267MiB/s (280MB/s), 267MiB/s-267MiB/s (280MB/s-280MB/s), io=2674MiB (2804MB), run=10001-10001msec

Disk stats (read/write):
  vdb: ios=677855/0, merge=0/0, ticks=7026/0, in_queue=0, util=99.04%

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-02-19 17:13:47 +00:00
Sergio Lopez
1ef6996207 vhost_user_backend: Add helpers for EVENT_IDX
Add helpers to Vring and VhostUserSlaveReqHandler for EVENT_IDX, so
consumers of this crate can make use of this feature.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-02-19 17:13:47 +00:00
Liu Bo
59958f0a61 vhost_user_fs: add the ability to set slave req fd
This adds the missing part of supporting virtiofs dax on the slave end,
that is, receiving a socket pair fd from the master end to set up a
communication channel for sending setupmapping & removemapping messages.

Signed-off-by: Liu Bo <bo.liu@linux.alibaba.com>
2020-02-19 07:52:50 +00:00
Rob Bradford
da7f31d4bc bin: vhost_user_fs: Port to new exit event strategy
Rather than handling the KILL_EVENT in the event handler itself use the
newly added support in VhostUserBackend for providing a kill event
framework.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-11 15:21:07 +01:00
Rob Bradford
7f032c8bb3 bin: vhost_user_fs: Shutdown worker thread on exit
This will ensure a clean shutdown of the backend

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-10 09:16:49 +01:00
Rob Bradford
99cb8dc0a4 bin: vhost_user_fs use error! macro logging for consistency
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-10 09:16:49 +01:00
Yang Zhong
99da1dff90 vhost-user-blk: Add MQ support in backend
Adding the num_queues parameter for vhost-user-blk backend, which
can enable MQ support in the backend.

This patch has enabled the MQ support from handle_event, and the
vhost-user-backend crate will enable multiple threads to call this
handle_event to handle read/write operations.

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
2020-02-03 09:49:27 +01:00
Sebastien Boeuf
aa5ae7616e src: Fix map_err losing the inner error
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-01-24 12:42:09 +01:00
Rob Bradford
7f73eebbdb vhost_user_block: Split launching backend into its own function
Split the basic launching functionality into its own function in the
newly added vhost_user_block crate.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-01-23 10:30:06 +00:00
Rob Bradford
9fd187c9fe vhost_user_net: Split launching backend into its own function
Split the basic launching functionality into its own function in the
newly added vhost_user_net crate.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-01-23 10:30:06 +00:00
Rob Bradford
1dd2451895 vhost_user_block: Refactor vhost_user_block backend code into a new crate
Extract the majority of the code that provides the vhost-user-block
backend into its own crate and port the binary to use it.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-01-23 10:30:06 +00:00
Rob Bradford
fb763c4043 vhost_user_net: Refactor vhost_user_net backend code into a new crate
Extract the majority of the code that provides the vhost-user-net
backend into its own crate and port the binary to use it.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-01-23 10:30:06 +00:00
Rob Bradford
e12179a3e6 bin: vhost_user_net: Rename "--backend" to "--net-backend"
This will prevent it from conflicting when it is aggregated into the
cloud-hypervisor binary.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-01-23 10:30:06 +00:00
Rob Bradford
3ede2dc53a bin: vhost_user_blk: Rename "--backend" to "--block-backend"
This will prevent it from conflicting when it is aggregated into the
cloud-hypervisor binary.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-01-23 10:30:06 +00:00
Sergio Lopez
e0a8da2f46 vhost_user_blk: Add missing WCE property support
Add missing WCE (write-cache enable) property support. This not only
an enhancement, but also a fix for a bug.

Right now, when vhost_user_blk uses a qcow2 image, it doesn't write
the QCOW2 metadata until the guest explicitly requests a flush. In
practice, this is equivalent to the write back semantic.

Without WCE, the guest assumes write through for the virtio_blk
device, and doesn't send those flush requests. Adding support for WCE,
and enabling it by default, we ensure the guest does send said
requests.

Supporting "WCE = false" would require updating our qcow2
implementation to ensure that, when required, it honors the write
through semantics by not deferring the updates to QCOW2 metadata.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-01-17 17:28:44 +00:00
Sergio Lopez
c7e9056c1e vhost_user_blk: implement support for direct (O_DIRECT) mode
Add support for opening the disk images with O_DIRECT. This allows
bypassing the host's file system cache, which is useful to avoid
polluting its cache and for better data integrity.

This mode of operation can be enabled by adding the "direct=<bool>"
parameter to the "backend" argument:

./target/debug/vhost_user_blk --backend image=test.raw,sock=/tmp/vhostblk,direct=true

The "direct" parameter defaults to "false", to preserve the original
behavior.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-01-17 17:28:44 +00:00
Sergio Lopez
a14aee9213 qcow: Use RawFile as backend instead of File
Use RawFile as backend instead of File. This allows us to abstract
the access to the actual image with a specialized layer, so we have a
place where we can deal with the low-level peculiarities.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-01-17 17:28:44 +00:00
Sergio Lopez
c5a656c9dc vm-virtio: block: Add support for alignment restrictions
Doing I/O on an image opened with O_DIRECT requires to adhere to
certain restrictions, requiring the following elements to be aligned:

 - Address of the source/destination memory buffer.
 - File offset.
 - Length of the data to be read/written.

The actual alignment value depends on various elements, and according
to open(2) "(...) there is currently no filesystem-independent
interface for an application to discover these restrictions (...)".

To discover such value, we iterate through a list of alignments
(currently, 512 and 4096) calling pread() with each one and checking
if the operation succeeded.

We also extend RawFile so it can be used as a backend for QcowFile,
so the later can be easily adapted to support O_DIRECT too.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-01-17 17:28:44 +00:00
Cathy Zhang
652e7b9b8a vm-virtio: Implement multiple queue support for net devices
Update the common part in net_util.rs under vm-virtio to add mq
support, meanwhile enable mq for virtio-net device, vhost-user-net
device and vhost-user-net backend. Multiple threads will be created,
one thread will be responsible to handle one queue pair separately.
To gain the better performance, it requires to have the same amount
of vcpus as queue pair numbers defined for the net device, due to
the cpu affinity.

Multiple thread support is not added for vhost-user-net backend
currently, it will be added in future.

Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
2020-01-17 12:06:19 +01:00
Cathy Zhang
1ae7deb393 vm-virtio: Implement refactor for net devices and backend
Since the common parts are put into net_util.rs under vm-virtio,
refactoring code for virtio-net device, vhost-user-net device
and backend to shrink the code size and improve readability
meanwhile.

Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
2020-01-17 12:06:19 +01:00
Yang Zhong
cee01edb97 vhost-user-blk backend: add readonly support
The current backend only support rw, and we also need
add readonly support.

The new command:
vhost_user_blk \
  --backend "image=/home/test.img, \
            sock=/home/path/vhost.socket, \
            readonly=true"

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
2019-12-18 09:45:11 +01:00
Sebastien Boeuf
50b0e58c88 vhost_user_fs: Allow specific shared directory to be specified
Because the vhost_user_backend crate needs some changes to support
moving the process to a different mount namespace and perform a pivot
root, it is not possible to change '/' to the given shared directory.

This commit, as a temporary measure, let the code point at the given
shared directory.

The long term solution is to perform the mount namespace change and the
pivot root as this will provide greater security.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
ba17758ac0 src: Add vhost-user-fs daemon
This patch implements a vhost-user-fs daemon based on Rust. It only
supports communicating through the virtqueues. The support for the
shared memory region associated with DAX will be added later.

It relies on all the code copied over from the crosvm repository, based
on the commit 961461350c0b6824e5f20655031bf6c6bf6b7c30.

It also relies on the vhost_user_backend crate, limiting the amount of
code needed to get this daemon up and running.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sergio Lopez
5870452d25 src: add vhost-user-blk backend
Create a vhost-user-blk backend using vhost-user-backend and following
the conventions established by the existing vhost-user-net
implementation.

This backend is based on https://github.com/slp/vhost-user-backend,
but a bit simplified, making it closer to the original implementation
in Firecracker. The main features missing are EVENT_IDX, support for
asynchronous I/O and multiqueue, but it's still fully functional and
provides a good starting point for evolving it into a more complete
implementation.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2019-11-07 10:36:30 +00:00
Sergio Lopez
08bebaae4f vhost_user_backend: move protocol_features to the backend
Extend VhostUserBackend trait with protocol_features(), so device
backend implementations can freely define which protocol features they
want to support.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2019-11-07 10:36:30 +00:00
Rob Bradford
cc72ed1cc9 vhost_user_net: Propagate errors correctly
Clean up the error handling and ensure that where possible errors are
propagated. Make use of std::convert::From in order to translate error
types.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-10-15 09:27:01 -07:00
Rob Bradford
8663b429b3 vhost_user_net: Remove unnecessary checks for unconfigured memory
Simplify the check for the unusual situation where the memory is not
configured by using .ok_or() on the option to convert it to a result.
This cleans up a bunch of extra indentation.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-10-15 09:27:01 -07:00
Rob Bradford
df336ade57 vhost_user_net: Remove debugging println! messages
Remove messages that are left over from the development of the project
that represent normal operation for the backend. This cleans up the
console output and improves performance.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-10-15 09:27:01 -07:00
Cathy Zhang
f6d1a9d9b8 src: Add vhost-user-net backend
Create vhost-user-net backend with Tap interface, to offload network
transaction from cloud-hypervisor. The goal is to provide flexibility
about the backend being in use, but also more security as it will allow
users to isolate the backend with different security profiles since it
will run as a dedicated process on the host.

Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-09-30 13:06:00 -07:00