Commit Graph

1939 Commits

Author SHA1 Message Date
Rob Bradford
3012975c17 tests: Enhance vsock integration test to support hotplug
Test basic vsock functionality works with hotplugged vsock device.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Rob Bradford
6c2bca5f1b bin: ch-remote: Add support for adding vsock devices
Add support for adding a vsock device using the HTTP API.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Rob Bradford
8de7448d44 vmm: api: Add "add-vsock" API entry point
This allows the hotplugging of vsock devices.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Rob Bradford
bf09a1e695 openapi: Add "id" field to VsockConfig
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Rob Bradford
a76cf0865f vmm: vm: Remove vsock device from config
When doing device unplug remove the vsock device from the configuration
if present.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Rob Bradford
99422324a7 vmm: vm: Add "add_vsock()"
Add the vsock device to the device manager and patch the config to add
the new vsock device.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Rob Bradford
1d61c476a1 vmm: device_manager: Add support for hotplugging virtio-vsock devices
Create a new VirtioVsock device and add it to the PCI bus upon hotplug.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Rob Bradford
f8501a3bd3 vmm: config: Move --vsock syntax to VsockConfig
This means it can be reused with ch-remote.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Sebastien Boeuf
6e049e0da1 vmm: Add an identifier to the --vsock device
It's possible to have multiple vsock devices so in preparation for
hotplug/unplug it is important to be able to have a unique identifier
for each device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 12:44:49 +01:00
Rob Bradford
10348f73e4 vmm, main: Support only zero or one vsock devices
The Linux kernel does not support multiple virtio-vsock devices.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-28 20:07:18 +02:00
Rob Bradford
9d1f95a3cc openapi: Add missing "id" field
NetConfig/DiskConfig/PmemConfig/FsConfig were all missing the id field
in the API yaml file.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-28 18:27:45 +02:00
dependabot-preview[bot]
30e2e51588 build(deps): bump serde_json from 1.0.51 to 1.0.52
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.51 to 1.0.52.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.51...v1.0.52)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-28 14:09:57 +02:00
dependabot-preview[bot]
dd9d0d0440 build(deps): bump micro_http from 0d87a94 to c9ffb90
Bumps [micro_http](https://github.com/firecracker-microvm/micro-http) from `0d87a94` to `c9ffb90`.
- [Release notes](https://github.com/firecracker-microvm/micro-http/releases)
- [Commits](0d87a94c8e...c9ffb90aeb)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-28 10:21:55 +01:00
dependabot-preview[bot]
cdc8493a05 build(deps): bump thiserror from 1.0.15 to 1.0.16
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.15 to 1.0.16.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.15...1.0.16)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-28 08:01:55 +02:00
dependabot-preview[bot]
f5debc4bc0 build(deps): bump libssh2-sys from 0.2.16 to 0.2.17
Bumps [libssh2-sys](https://github.com/alexcrichton/ssh2-rs) from 0.2.16 to 0.2.17.
- [Release notes](https://github.com/alexcrichton/ssh2-rs/releases)
- [Commits](https://github.com/alexcrichton/ssh2-rs/compare/libssh2-sys-0.2.16...libssh2-sys-0.2.17)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-27 09:10:14 +02:00
dependabot-preview[bot]
37dfb4cbb3 build(deps): bump hermit-abi from 0.1.11 to 0.1.12
Bumps [hermit-abi](https://github.com/hermitcore/rusty-hermit) from 0.1.11 to 0.1.12.
- [Release notes](https://github.com/hermitcore/rusty-hermit/releases)
- [Commits](https://github.com/hermitcore/rusty-hermit/compare/hermit-abi-0.1.11...hermit-abi-0.1.12)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-27 09:10:02 +02:00
Muminul Islam
e1a07ce3c4 vmm: vm: Unpark the threads before shutdown when the current state is paused
If the current state is paused that means most of the handles got killed by pthread_kill
We need to unpark those threads to make the shutdown worked. Otherwise
The shutdown API hangs and the API is not responding afterwards. So
before the shutdown call we need to resume the VM make it succeed.

Fixes: #817

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-04-27 09:09:12 +02:00
Rob Bradford
1df38daf74 vmm, tests: Make specifying a size optional for virtio-pmem
If a size is specified use it (in particular this is required if the
destination is a directory) otherwise seek in the file to get the size
of the file.

Add a new check that the size is a multiple of 2MiB otherwise the kernel
will reject it.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-24 18:30:05 +01:00
Rob Bradford
7481e4d959 vmm: config: Validate that shared memory is enabled if using vhost-user
Check that if any device using vhost-user (net & disk with
vhost_user=true) or virtio-fs is enabled then check shared memory is
also enabled.

Fixes: #848

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-24 16:01:49 +01:00
Bo Chen
2ac6971a8b vmm: MemoryManager: Cleanup the usage of std::ffi/io/result
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-04-23 21:39:51 +02:00
Bo Chen
3f42f86d81 vmm: Add the 'shared' and 'hugepages' controls to MemoryConfig
The new 'shared' and 'hugepages' controls aim to replace the 'file'
option in MemoryConfig. This patch also updated all related integration
tests to use the new controls (instead of providing explicit paths to
"/dev/shm" or "/dev/hugepages").

Fixes: #1011

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-04-23 21:39:51 +02:00
dependabot-preview[bot]
d6aa717913 build(deps): bump syn from 1.0.17 to 1.0.18
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.17 to 1.0.18.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.17...1.0.18)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-23 21:32:53 +02:00
Jose Carlos Venegas Munoz
3eaeba4b55 vm-virtio: Fix FS_IO callback for virtio-fs
FS_IO is part of the actions a vhost-user-fs daemon can ask the VMM to
perform on its behalf. It is meant to read/write the content from a file
descriptor directly into a guest memory region. This region can either
be a RAM region or the dedicated cache region for virtio-fs.

The way FS_IO was implemented, it was only expecting the guest physical
address provided through the "cache_offset" field to refer to the cache
region. Unfortunately, this was only implementing FS_IO partially.

This patch extends the existing FS_IO implementation by checking the GPA
against the cache region as a first step, but if it is not part of the
cache region address range, then we fallback onto searching for a RAM
region that could match. If there is a matching RAM region, we retrieve
the corresponding host address to let the VMM read/write from/to it.

Fixes: #1054

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2020-04-23 15:01:28 +01:00
dependabot-preview[bot]
df14a68e87 build(deps): bump smallvec from 1.3.0 to 1.4.0
Bumps [smallvec](https://github.com/servo/rust-smallvec) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/servo/rust-smallvec/releases)
- [Commits](https://github.com/servo/rust-smallvec/compare/v1.3.0...v1.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-23 14:38:39 +02:00
Samuel Ortiz
e685854f16 gh: Separate the build and release jobs
The release one is always skipped for PRs which can be confusing for
e.g. dependabot.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-04-23 13:45:12 +02:00
Rob Bradford
c790bba905 tests: Migrate from Ubuntu Eoan to Focal
This is the latest LTS release.

Fixes: #989

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-23 13:01:02 +02:00
dependabot-preview[bot]
e525af7a61 build(deps): bump ryu from 1.0.3 to 1.0.4
Bumps [ryu](https://github.com/dtolnay/ryu) from 1.0.3 to 1.0.4.
- [Release notes](https://github.com/dtolnay/ryu/releases)
- [Commits](https://github.com/dtolnay/ryu/compare/1.0.3...1.0.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-23 12:29:14 +02:00
Sebastien Boeuf
3e8a6ba0c0 ci: Ignore test_snapshot_restore
The newly added integration test "test_snapshot_restore" is very
unstable, which causes our CI to fail on most pull requests, which is
not acceptable. That's why we ignore this test until we can fix the
stability issue.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-23 11:38:47 +02:00
dependabot-preview[bot]
9ebf052538 build(deps): bump cc from 1.0.51 to 1.0.52
Bumps [cc](https://github.com/alexcrichton/cc-rs) from 1.0.51 to 1.0.52.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases)
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.51...1.0.52)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-22 22:19:53 +02:00
Sebastien Boeuf
f6b150a34d ci: Add integration test for VM migration
Now that we have multiple virtio devices supporting snapshot and restore
operations, we can add a new integration test to validate the migration
feature works as expected.

The important part is virtio-net as it is used to ssh into the VM to
verify the VM has been restored in the proper state.

This test only works for virtio-mmio for now. Further support for
testing virtio-pci will be added once the virtio-pci transport layer
will support migration.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-22 22:19:31 +02:00
dependabot-preview[bot]
9f08f537e0 build(deps): bump pin-utils from 0.1.0-alpha.4 to 0.1.0
Bumps [pin-utils](https://github.com/rust-lang-nursery/pin-utils) from 0.1.0-alpha.4 to 0.1.0.
- [Release notes](https://github.com/rust-lang-nursery/pin-utils/releases)
- [Commits](https://github.com/rust-lang-nursery/pin-utils/compare/0.1.0-alpha.4...0.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-22 20:18:42 +02:00
Yang Zhong
9c7215d620 docs: Add the vhost-user-blk test doc
This doc describe how to use vhost-user-blk in Cloud Hypervisor
with SPDK. With this doc, we can setup simple test environment on
Ubuntu release.

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
2020-04-22 20:18:23 +02:00
dependabot-preview[bot]
35744375e0 build(deps): bump cc from 1.0.50 to 1.0.51
Bumps [cc](https://github.com/alexcrichton/cc-rs) from 1.0.50 to 1.0.51.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases)
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.50...1.0.51)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-22 17:10:21 +02:00
Yi Sun
4fc75cf2b0 vm-virtio: Implement Snapshottable trait for Console
This patch implements the Snapshottable trait for virtio-console, which
enables migration support for it. A VM with a virtio-console device
attached can be snapshot and then restored without issues.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
2020-04-22 14:45:16 +02:00
Yi Sun
d41ce909a2 vm-virtio: Implement Snapshottable trait for Pmem
This brings the migration support to virtio-pmem device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
2020-04-22 14:45:02 +02:00
dependabot-preview[bot]
f626bd60c5 build(deps): bump parking_lot_core from 0.7.1 to 0.7.2
Bumps [parking_lot_core](https://github.com/Amanieu/parking_lot) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/Amanieu/parking_lot/releases)
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.7.1...core-0.7.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-22 09:12:08 +02:00
Martin Xu
5a380a6918 vmm: memory_manager: Support non-power-of-2 block sizes
Replace alignment calculation of start address with functionally
equivalent version that does not assume that the block size is a power
of two.

Signed-off-by: Martin Xu <martin.xu@intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-22 09:11:51 +02:00
dependabot-preview[bot]
f8ee89a514 build(deps): bump arc-swap from 0.4.5 to 0.4.6
Bumps [arc-swap](https://github.com/vorner/arc-swap) from 0.4.5 to 0.4.6.
- [Release notes](https://github.com/vorner/arc-swap/releases)
- [Changelog](https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/arc-swap/compare/v0.4.5...v0.4.6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-21 22:08:17 +02:00
Sebastien Boeuf
49322c5ebe vm-virtio: Implement the Snapshottable trait for Net
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Signed-off-by: Yi Sun <yi.y.sun@linux.intel.com>
2020-04-21 21:25:03 +02:00
Sebastien Boeuf
24c2b67aa4 vm-virtio: Improve virtio-net rx queue processing
The frame buffer must be updated depending on the amount read from it,
which depends on the number and depth of descriptors available at the
time of the processing.

This patch handles this buffer update, and allow for large buffers to be
correctly processed in multiple rounds.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 21:25:03 +02:00
Sebastien Boeuf
03dd24978e vm-virtio: Restore queues based on used index
On the restore path, using the available and used indexes read from
memory to fill the Queue structure was a mistake. Indeed, the available
index is written from the guest and it reflects the last available index
in the descriptor table. But the driver might have queued a lot of
buffers which have not yet been used by the device. This leads to a
situation where the next_avail from Queue is completely different from
the one we can read from memory.

Instead, the right way to determine the next_avail index that should be
used by the device is by relying on the used index from the memory. This
index represents the correct information we're looking for as it has
been updated before the snapshot to let the guest know the next index to
process.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 21:25:03 +02:00
Sebastien Boeuf
cf707da1a8 vm-virtio: Extend Queue helpers
First, this modifies the existing helpers on how to get indexes for
available and used rings from memory. Instead of updating the queue
through each helper, they are now used as simple getters.

Based on these new getters, we could create a new helper to determine if
the queue has some available descriptors already queued from the driver
side. This helper is going to be particularly helpful when trying to
determine from a virtio thread if a queue is already loaded with some
available buffers that can be used to send information to the guest.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 21:25:03 +02:00
Sebastien Boeuf
c22fd39170 vmm: Remove virtio device's userspace mapping on hot-unplug
When a virtio device is dynamically removed from the VM through the
hot-unplug mechanism, every mapping associated with it must be properly
removed.

Based on the previous patches letting a VirtioDevice expose the list of
userspace mappings associated with it, this patch can now remove all the
KVM userspace memory regions through the MemoryManager.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 10:02:21 +01:00
Sebastien Boeuf
0a97c25464 vmm: Extend MemoryManager to remove userspace mappings
The same way we added a helper for creating userspace memory mappings
from the MemoryManager, this patch adds a new helper to remove some
previously added mappings.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 10:02:21 +01:00
Sebastien Boeuf
b2de1cd523 vm-virtio: Implement shutdown() for virtio-fs
Since the virtio-fs device is backed by a vhost-user process, it is
important to implement the proper shutdown() function from the
VirtioDevice trait, as vhost-user-blk and vhost-user-net do.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 10:02:21 +01:00
Sebastien Boeuf
fbcf3a7a7a vm-virtio: Implement userspace_mappings() for virtio-pmem
When hot-unplugging the virtio-pmem from the VM, we don't remove the
associated userspace mapping. This patch will let us fix this in a
following patch. For now, it simply adapts the code so that the Pmem
device knows about the mapping associated with it. By knowing about it,
it can expose it to the caller through the new userspace_mappings()
function.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 10:02:21 +01:00
Sebastien Boeuf
b0353992d6 vm-virtio: Implement userspace_mappings() for virtio-fs
This will help when we will implement the hot-unplug of the virtio-fs
device, as we will have to remove correctly the userspace mappings
associated with the device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 10:02:21 +01:00
Sebastien Boeuf
3fb0a02fa2 vm-virtio: Get userspace mappings from VirtioDevice
Introduce new getter function to the VirtioDevice trait, as it will
allow the caller to retrieve the list of userspace mappings associated
with the device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-21 10:02:21 +01:00
dependabot-preview[bot]
8b823e5e34 build(deps): bump backtrace-sys from 0.1.35 to 0.1.36
Bumps [backtrace-sys](https://github.com/alexcrichton/backtrace-rs) from 0.1.35 to 0.1.36.
- [Release notes](https://github.com/alexcrichton/backtrace-rs/releases)
- [Commits](https://github.com/alexcrichton/backtrace-rs/compare/backtrace-sys-0.1.35...backtrace-sys-0.1.36)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-04-20 21:49:34 +00:00
Sebastien Boeuf
c23b48888d ci: Factorize virtio-fs hotplug integration tests
There is some duplication between regular and hotplug virtio-fs tests
that can be factorized by adding a simple hotplug flag to choose if each
test should run with or without hotplugging the device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-04-20 20:36:26 +02:00