Commit Graph

46 Commits

Author SHA1 Message Date
Yong He
0149e65081 vm-device: support batch update interrupt source group GSI
Split interrupt source group restore into two steps, first restore
the irqfd for each interrupt source entry, and second restore the
GSI routing of the entire interrupt source group.

This patch will reduce restore latency of interrupt source group,
and in a 200-concurrent restore test, the patch reduced the
average IOAPIC restore time from 15ms to 1ms.

Signed-off-by: Yong He <alexyonghe@tencent.com>
2023-08-03 15:58:36 +01:00
Yu Li
447cad3861 block: merge qcow, vhdx and block_util into block crate
This commit merges crates `qcow`, `vhdx` and `block_util` into the
crate `block`, which can allow `qcow` to use functions from `block_util`
without introducing a circular crate dependency.

This commit is based on crosvm implementation:
f2eecc4152

Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
2023-07-19 13:52:43 +01:00
Yong He
3494080e2f vmm: add configuration for network offloading features
Add new configuration for offloading features, including
Checksum/TSO/UFO, and set these offloading features as
enabled by default.

Fixes: #4792.

Signed-off-by: Yong He <alexyonghe@tencent.com>
2023-01-12 09:05:45 +00:00
Bo Chen
51307dd509 fuzz: Add fuzzer for 'linux loader' cmdline
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-12-12 13:50:28 +00:00
Bo Chen
32ded2c72b fuzz: Add fuzzer for 'linux loader'
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-12-12 13:50:28 +00:00
Bo Chen
e2e02c8f69 fuzz: Add fuzzer for virtio-net
To synthesize the interactions between the virtio-net device and the tap
interface, the fuzzer utilizes a pair of unix domain sockets: one socket
(e.g. the dummy tap frontend) is used to construct the 'net_util::Tap'
instance for creating a virtio-net device; the other socket (e.g. the
dummy tap backend) is used in a epoll loop for handling the tx and rx
requests from the virtio-net device.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-11-30 12:13:14 +00:00
Sebastien Boeuf
0bd910e8b0 devices, vmm: Move Serial to new restore design
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-11-25 17:37:29 +00:00
Rob Bradford
f603afc46e vmm: Make Transparent Huge Pages controllable (default on)
Add MemoryConfig::thp and `--memory thp=on|off` to allow control of
Transparent Huge Pages.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-11-09 16:51:21 +00:00
Bo Chen
ef8fb9bd25 fuzz: Add fuzzer for virtio-console
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-11-03 09:10:41 -07:00
Sebastien Boeuf
1f0e5eb66a vmm: virtio-devices: Restore every VirtioDevice upon creation
Following the new design proposal to improve the restore codepath when
migrating a VM, all virtio devices are supplied with an optional state
they can use to restore from. The restore() implementation every device
was providing has been removed in order to prevent from going through
the restoration twice.

Here is the list of devices now following the new restore design:

- Block (virtio-block)
- Net (virtio-net)
- Rng (virtio-rng)
- Fs (vhost-user-fs)
- Blk (vhost-user-block)
- Net (vhost-user-net)
- Pmem (virtio-pmem)
- Vsock (virtio-vsock)
- Mem (virtio-mem)
- Balloon (virtio-balloon)
- Watchdog (virtio-watchdog)
- Vdpa (vDPA)
- Console (virtio-console)
- Iommu (virtio-iommu)

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-10-24 14:17:08 +02:00
Bo Chen
802f489e4d fuzz: Add fuzzer for virtio-iommu
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-10-21 14:21:42 +01:00
Bo Chen
7b31871a36 fuzz: mem: Avoid using hugepages
The kernel will trigger a SIGBUS upon hugetlb page faults when there is
no huge pages available. We neither have a way to ensure enough huge
pages available on the host system, nor have a way to gracefully report
the lack of huge pages in advance from Cloud Hypervisor. For these
reasons, we have to avoid using huge pages from the virtio-mem fuzzer to
avoid SIGBUS errors.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-10-18 09:09:35 +01:00
Bo Chen
342851c88c fuzz: Add fuzzer for virtio-mem
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-10-12 16:09:27 +01:00
Bo Chen
4fea40f008 fuzz: Balloon: Reduce the guest memory size and queue size
With the guest memory size of 1MB, a valid descriptor size can be close
to the guest memory size (e.g. 1MB) and can contain close to 256k
valid pfn entries (each entry is 4 bytes). Multiplying the queue
size (e.g. 256), there can be close to 64 millions pfn entries to
process in a single request. This is why the oss-fuzz reported a
timeout (with a limit of 60s).

By reducing the guest memory size and the queue size, the worst-case now
is 8 million pfn entries for fuzzing, which can be finished in around 20
seconds according to my local experiment.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-10-04 09:40:12 +01:00
Bo Chen
ef603fde4c fuzz: Reduce the guest memory size for balloon fuzzer
As the virt queues are initialized with random bytes from the fuzzing
engine, a descriptor buffer for the available ring can have a very large
length (e.g. up to 4GB). This means there can be up to 1 billion
entries (e.g. page frame number) for virtio-balloon to process a signal
available descriptor (given each entry is 4 bytes). This is the reason
why oss-fuzz reported a hanging issue for this fuzzer, where the
generated descriptor buffer length is 4,278,321,152.

We can avoid this kind of long execution by reducing the size of guest
memory. For example, with 1MB of guest memory, the number of descriptor
entries for processing is limited ~256K.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-23 08:28:07 +01:00
Rob Bradford
194b59f44b fuzz: Don't overload meaning of reset()
This function is for really for the transport layer to trigger a device
reset. Instead name it appropriately for the fuzzing specific use case.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-09-22 11:01:41 -07:00
Bo Chen
ab0b3f1b7b fuzz: Add fuzzer for virtio-balloon
The fuzzer exercises the inflate, deflate and reporting events of
virtio-balloon via creating three queues and kicking three events.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-20 11:05:53 +02:00
Bo Chen
e1b483fc88 fuzz: Add fuzzer for virtio-rng
To make the fuzzer faster and more effective, the guest memory is
setup with a much smaller size (comparing with other virtio device
fuzzers) and  a hole between the memory for holding virtio queue and
the rest of guest data. It brings two benefits: 1) avoid writing large
chunk of data from 'urandom' into the available descriptor chain (which
makes the fuzzer faster); 2) reduce substantial amount of overwrites to
the virtio queue data by the data from 'urandom (which makes the fuzzer
more deterministic and hence effective).

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-20 11:05:53 +02:00
Bo Chen
f815fcbb5d fuzz: Add fuzzer for virtio-watchdog
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-20 11:05:53 +02:00
Bo Chen
91b8b00f95 fuzz: Add fuzzer for virtio-pmem
The fuzzer is focusing on the virtio-pmem code that processes guest
inputs (e.g. virt queues). Given 'flush' is the only virtio-pmem
request, the fuzzer is essentially testing the code for parsing and
error handling.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-15 09:08:11 +01:00
Bo Chen
67a89f4538 fuzz: Setup virt queue with proper addresses
To make the fuzzers more focused and more efficient, we now provide
default addresses for the descriptor table, available ring, and used
ring, which ensures the virt-queue has a valid memory layout (e.g. no
overlapping between descriptor tables, available ring, and used ring).

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-15 09:08:11 +01:00
Bo Chen
742d6858f7 fuzz: block: Setup the virt queue based on the fuzzed input bytes
Instead of always fuzzing virt-queues with default values (mostly 0s),
the fuzzer now initializes the virt-queue based on the fuzzed input
bytes, such as the tail position of the available ring, queue size
selected by driver, descriptor table address, available ring address,
used ring address, etc. In this way, the fuzzer can explore the
virtio-block code path with various virt-queue setup.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-01 08:39:28 +02:00
Bo Chen
6cb214f15c fuzz: block: Rely on custom EpollHelper::run and VirtioCommon:reset
This commit also extends the copyright header.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-08-30 14:01:33 -07:00
Bo Chen
0b182be65e fuzz: block: Remove meaningless setup to the virt-queue
The current fuzzer defines a 'format' for the random input 'bytes' from
libfuzzer, but this 'format' failed to improve the fuzzing
efficiency. Instead, the 'format' parsing process obfuscates the fuzzer and
makes the fuzzing engine much harder to focus on the actual fuzzing
target (e.g. virtio-block queue event handling). It is actually worse than
simply using the random inputs as the virt queue content for fuzzing.

We can later introduce a different 'format' to the input 'bytes' for
better fuzzing, say focusing more on virito-block fuzzing through
ensuring the virt queue content always has a valid 'available'
descriptor chain to process.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-08-11 09:35:46 +02:00
Bo Chen
fbec4a070d fuzz: block: Ensure the virtio-block thread is killed and joined
This also ensures that the 'queue_evt' is fully processed, as we enforce
the main thread is waiting for the virtio-block thread to process the
'kill_evt' which is after the 'queue_evt' processing.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-08-11 09:35:46 +02:00
Bo Chen
5ba3b80e83 fuzz: block: Ensure a queue event is properly processed
Currently the main thread returns immediately after sending a 'queue'
event which is rarely received and processed by the virtio-block
thread (unless system is in high workload). In this way, the fuzzer is
mostly doing nothing and is unable to reproduce its behavior
deterministically (from the same inputs). This patch relies on a
'level-triggered' epoll to ensure a 'queue' event is properly processed
before return from the main thread.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-08-11 09:35:46 +02:00
Bo Chen
1125fd2667 vmm: api: Use 'BTreeMap' for 'HttpRoutes'
In this way, we get the values sorted by its key by default, which is
useful for the 'http_api' fuzzer.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-08-03 10:18:24 +01:00
Bo Chen
e5155bab62 fuzz: Add fuzzer for HTTP API
Fuzz the HTTP API handling code with a minimum HTTP API
receiver (e.g. mocking the behavior of our "vmm" thread).

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-08-03 10:18:24 +01:00
Sebastien Boeuf
a423bf13ad virtio: Port codebase to the latest virtio-queue version
The new virtio-queue version introduced some breaking changes which need
to be addressed so that Cloud Hypervisor can still work with this
version.

The most important change is about removing a handle to the guest memory
from the Queue, meaning the caller has to provide the guest memory
handle for multiple methods from the QueueT trait.

One interesting aspect is that QueueT has been widely extended to
provide every getter and setter we need to access and update the Queue
structure without having direct access to its internal fields.

This patch ports all the virtio and vhost-user devices to this new crate
definition. It also updates both vhost-user-block and vhost-user-net
backends based on the updated vhost-user-backend crate. It also updates
the fuzz directory.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-07-29 17:41:32 +01:00
Rob Bradford
a330c531b0 fuzz: Add new fuzzer for emulated cmos device
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-07-27 18:12:56 +01:00
Rob Bradford
e4211272ad fuzz: Add new fuzzer for emulated serial device
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-07-27 18:12:56 +01:00
Sebastien Boeuf
3f62a172b2 virtio-devices: Pass a list of tuples for virtqueues
Instead of passing separately a list of Queues and the equivalent list
of EventFds, we consolidate these two through a tuple along with the
queue index.

The queue index can be very useful if looking for the actual index
related to the queue, no matter if other queues have been enabled or
not.

It's also convenient to have the EventFd associated with the Queue so
that we don't have to carry two lists with the same amount of items.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-07-21 14:28:41 +02:00
Sebastien Boeuf
de3e003e3e virtio-devices: Handle virtio queues interrupts from transport layer
Instead of relying on the virtio-queue crate to store the information
about the MSI-X vectors for each queue, we handle this directly from the
PCI transport layer.

This is the first step in getting closer to the upstream version of
virtio-queue so that we can eventually move fully to the upstream
version.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-01-25 12:01:12 +01:00
Sebastien Boeuf
0162d73ed8 virtio-queue: Update crate based on latest rust-vmm/vm-virtio
This crate contains up to date definition of the Queue, AvailIter,
DescriptorChain and Descriptor structures forked from the upstream
crate rust-vmm/vm-virtio 27b18af01ee2d9564626e084a758a2b496d2c618.

The following patches have been applied on top of this base in order to
make it work correctly with Cloud Hypervisor requirements:

- Add MSI vector field to the Queue

  In order to help with MSI/MSI-X support, it is convenient to store the
  value of the interrupt vector inside the Queue directly.

- Handle address translations

  For devices with access to data in memory being translated, we add to
  the Queue the ability to translate the address stored in the
  descriptor.
  It is very helpful as it performs the translation right after the
  untranslated address is read from memory, avoiding any errors from
  happening from the consumer's crate perspective. It also allows the
  consumer to reduce greatly the amount of duplicated code for applying
  the translation in many different places.

- Add helpers for Queue structure

  They are meant to help crate's consumers getting/setting information
  about the Queue.

These patches can be found on the 'ch' branch from the Cloud Hypervisor
fork: https://github.com/cloud-hypervisor/vm-virtio.git

This patch takes care of updating the Cloud Hypervisor code in
virtio-devices and vm-virtio to build correctly with the latest version
of virtio-queue.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-01-06 10:02:40 +00:00
Sebastien Boeuf
0249e8641a Move Cloud Hypervisor to virtio-queue crate
Relying on the vm-virtio/virtio-queue crate from rust-vmm which has been
copied inside the Cloud Hypervisor tree, the entire codebase is moved to
the new definition of a Queue and other related structures.

The reason for this move is to follow the upstream until we get some
agreement for the patches that we need on top of that to make it
properly work with Cloud Hypervisor.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-10-22 11:38:55 +02:00
Rob Bradford
687d646c60 virtio-devices, vmm: Shutdown VMM on virtio thread panic
Shutdown the VMM in the virtio (or VMM side of vhost-user) thread
panics.

See: #3031

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-09-08 09:40:36 +01:00
Fazla Mehrab
98fc38c465 fuzz: fuzz testing for VHDx block device is added
The fuzzer needs to take a larger input for the whole disk image to
be most useful. Since the file is small we can test by reading and
writing over the whole file.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Signed-off-by: Fazla Mehrab <akm.fazla.mehrab@intel.com>
2021-08-19 11:43:19 +02:00
Bo Chen
2d2463ce04 fuzz: Move to the seccompiler crate
Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-08-18 10:42:19 +02:00
Sebastien Boeuf
d278e9f39b fuzz: block: Test a RAW file instead QCOW
Instead of running the generic block fuzzer with QCOW, it's better to
use a RAW file since it's less complex and it will focus on virtqueues.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-08-11 08:55:54 -07:00
Bo Chen
169d6f6756 fuzz: Update the 'block' fuzz target with the rate limiter support
Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-03-12 09:35:03 +01:00
Sebastien Boeuf
2824642e80 virtio-devices: Rename BlockIoUring to Block
Now that BlockIoUring is the only implementation of virtio-block,
handling both synchronous and asynchronous backends based on the
AsyncIo trait, we can rename it to Block.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-01-22 16:10:34 +00:00
Sebastien Boeuf
41cfdb50cd virtio-devices: Remove virtio-block synchronous implementation
Now that both synchronous and asynchronous backends rely on the
asynchronous version of virtio-block (namely BlockIoUring), we can
get rid of the synchronous version (namely Block).

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-01-22 16:10:34 +00:00
Bo Chen
dda1df0a58 fuzz: Fix the building errors with recent changes
This patch adds two required dependencies to fuzz/Cargo.toml, and fixes
the building error on the 'block' fuzzer.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-01 11:16:35 +01:00
Rob Bradford
6e8c8991d9 fuzz: qcow: Remove unnecessary "use libc"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-07-20 20:22:47 +02:00
Rob Bradford
191f64028a fuzz: Add a fuzzer for virtio-devices::Block
This is based on the block fuzzer from crosvm and ported to Cloud
Hypervisor.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-07-20 20:22:47 +02:00
Rob Bradford
4d878418e5 fuzz: Add fuzzing infrastructure and QCOW fuzzer
Add the basic infrastructure for fuzzing along with a qcow fuzzer ported
from crosvm and adapted to our code.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-07-17 08:49:35 +02:00