Commit Graph

1318 Commits

Author SHA1 Message Date
Sebastien Boeuf
793d4e7b8d vmm: Move codebase to GuestMemoryAtomic from vm-memory
Relying on the latest vm-memory version, including the freshly
introduced structure GuestMemoryAtomic, this patch replaces every
occurrence of Arc<ArcSwap<GuestMemoryMmap> with
GuestMemoryAtomic<GuestMemoryMmap>.

The point is to rely on the common RCU-like implementation from
vm-memory so that we don't have to do it from Cloud-Hypervisor.

Fixes #735

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-02-19 13:48:19 +00:00
Sebastien Boeuf
ddf6caf955 ci: Improve test_memory_mergeable_on stability
The integration test test_memory_mergeable_on has been fairly unstable
for quite some time now. Because it can take some time for the VM to be
spawned and to be able to perform a correct measure of the PSS, this
commit simply increases the time before such measure is done.
This should return more accurate PSS results, which should help
stabilize the test.

Fixes #781

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-02-19 12:36:28 +00:00
dependabot-preview[bot]
af621be509 build(deps): bump micro_http from 57ac9df to 6fd1545
Bumps [micro_http](https://github.com/firecracker-microvm/firecracker) from `57ac9df` to `6fd1545`.
- [Release notes](https://github.com/firecracker-microvm/firecracker/releases)
- [Commits](57ac9df0ad...6fd1545222)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-19 07:53:09 +00:00
Liu Bo
4970e2f703 vhost-user-fs: add dax tests for vhost_user_fs rust daemon
Now that vhost_user_fs rust daemon supports virtiofs's dax mode, this adds
the two dax tests accordingly.

Signed-off-by: Liu Bo <bo.liu@linux.alibaba.com>
2020-02-19 07:52:50 +00:00
Liu Bo
59958f0a61 vhost_user_fs: add the ability to set slave req fd
This adds the missing part of supporting virtiofs dax on the slave end,
that is, receiving a socket pair fd from the master end to set up a
communication channel for sending setupmapping & removemapping messages.

Signed-off-by: Liu Bo <bo.liu@linux.alibaba.com>
2020-02-19 07:52:50 +00:00
Liu Bo
3f09eff6c5 vhost_user_fs: add fs cache request operations
This introduces setupmapping and removemapping methods to server.rs,
passthrough.rs and filesystem.rs in order to support virtiofs dax mode
inside guest.

Since we don't really want the server.rs to know that it is dealing with
vhost-user specifically, this is making it more generic by adding a new
trait which has three functions map()/unmap()/sync() corresponding to
fs_slave_{map, unmap, sync}, server.rs will take anything that implements
the trait.

Signed-off-by: Liu Bo <bo.liu@linux.alibaba.com>
2020-02-19 07:52:50 +00:00
Liu Bo
956a84f73a vhost_user_fs: add necessary structs for map/unmap requests
This is adding some structures with which we will talk with guest kernel on
map/unmap requests.

Signed-off-by: Liu Bo <bo.liu@linux.alibaba.com>
2020-02-19 07:52:50 +00:00
Liu Bo
269d660b0a vhost_user_fs: add SlaveFsCacheReq to handle map/unmap
This introduces SlaveFsCacheReq which implements
VhostUserMasterReqHandler to handle map/unmap requests.

Signed-off-by: Liu Bo <bo.liu@linux.alibaba.com>
2020-02-19 07:52:50 +00:00
Sebastien Boeuf
be78c6da49 vhost_rs: Fix unit test race condition
The unit tests are run from cargo test through multiple threads of the
same process. For this reason, all these threads share their file
descriptors (because that's how this works on Linux), which means that
any of them can close a file descriptor opened from another thread.

In the context of create_listener() and accept_connection() tests, they
can run concurrently and this generates some failure when the file
descriptor create_listener() is binding to is being closed from the
accept_connection() test.

In order to avoid such race condition, this patch simply removes the
part of the unit test performing an explicit and unsafe file descriptor
closure.

Fixes #759

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-02-18 16:59:13 +01:00
Rob Bradford
f7378bc092 tests: Add self spawning vhost-user-block test
Also rename the net self spawning test to differentiate it from the the
block one.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-18 08:43:47 +00:00
Rob Bradford
1f6cbad01a vmm: Add support for spawning vhost-user-block backend
If no socket is supplied when enabling "vhost_user=true" on "--disk"
follow the "exe" path in the /proc entry for this process and launch the
network backend (via the vmm_path field.)

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-18 08:43:47 +00:00
Rob Bradford
4d60ef59bc vm-virtio: vhost_user: block: On shutdown() drop the socket
This causes the vhost-user-block backend to shutdown.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-18 08:43:47 +00:00
Sebastien Boeuf
7fabca3548 ci: Don't run unit tests in a privileged container
The unit tests require some specific Linux capabilities and also to have
access to /dev/kvm device. This commit makes sure we enable only what's
necessary instead of blindly enable full priviliges with --privileged
option.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-02-18 08:53:03 +01:00
dependabot-preview[bot]
27247164b5 build(deps): bump micro_http from 4827569 to 57ac9df
Bumps [micro_http](https://github.com/firecracker-microvm/firecracker) from `4827569` to `57ac9df`.
- [Release notes](https://github.com/firecracker-microvm/firecracker/releases)
- [Commits](482756925c...57ac9df0ad)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-18 06:42:39 +00:00
Rob Bradford
08a68f2f0f build: Run unit tests on worker node
The unit tests require higher privileges so for safety run on the worker
nodes.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-17 15:41:06 +00:00
Samuel Ortiz
f21cd31b94 scripts: dev_cli: Add more privileges for the integration tests
We need the host IPC for sharing eventfds with KVM, and the host network
for VFIO.

We also enforce the no-seccomp setting on the container, to overcome any
potential filtering set by our container's Ubuntu base.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-02-17 15:06:04 +00:00
Rob Bradford
a94887e051 build: Use dev container for integration tests
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-17 15:06:04 +00:00
Sebastien Boeuf
3edc2bd6ab vmm: Prevent memory overcommitment through virtio-fs shared regions
When a virtio-fs device is created with a dedicated shared region, by
default the region should be mapped as PROT_NONE so that no pages can be
faulted in.

It's only when the guest performs the mount of the virtiofs filesystem
that we can expect the VMM, on behalf of the backend, to perform some
new mappings in the reserved shared window, using PROT_READ and/or
PROT_WRITE.

Fixes #763

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-02-17 15:03:47 +01:00
dependabot-preview[bot]
968c90a6ce build(deps): bump hermit-abi from 0.1.6 to 0.1.7
Bumps [hermit-abi](https://github.com/hermitcore/rusty-hermit) from 0.1.6 to 0.1.7.
- [Release notes](https://github.com/hermitcore/rusty-hermit/releases)
- [Commits](https://github.com/hermitcore/rusty-hermit/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-17 11:17:39 +00:00
Rob Bradford
7485a0c1f7 Revert "build: Don't fail build on test_vfio failure"
This reverts commit 014844d0da.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-17 08:23:02 +00:00
dependabot-preview[bot]
cbc0ac3267 build(deps): bump micro_http from 7a23e54 to 4827569
Bumps [micro_http](https://github.com/firecracker-microvm/firecracker) from `7a23e54` to `4827569`.
- [Release notes](https://github.com/firecracker-microvm/firecracker/releases)
- [Commits](7a23e54775...482756925c)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-17 07:41:05 +00:00
dependabot-preview[bot]
7fdb5ae4c4 build(deps): bump vm-memory from eb2fc0b to f615b19
Bumps [vm-memory](https://github.com/rust-vmm/vm-memory) from `eb2fc0b` to `f615b19`.
- [Release notes](https://github.com/rust-vmm/vm-memory/releases)
- [Commits](eb2fc0b6d9...f615b19469)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-17 07:40:49 +00:00
dependabot-preview[bot]
0d748c5c2b build(deps): bump scopeguard from 1.0.0 to 1.1.0
Bumps [scopeguard](https://github.com/bluss/scopeguard) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/bluss/scopeguard/releases)
- [Commits](https://github.com/bluss/scopeguard/compare/1.0.0...v1.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-17 07:40:24 +00:00
dependabot-preview[bot]
6692fa60e9 build(deps): bump thiserror from 1.0.10 to 1.0.11
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.10 to 1.0.11.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.10...1.0.11)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-15 11:05:06 +00:00
Rob Bradford
f03602a4c9 tests: Add self spawning vhost-user-net test
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-14 17:32:49 +00:00
Rob Bradford
bc75c1b4e1 vmm: Add support for spawning vhost-user-net backend
If no socket is supplied when enabling "vhost_user=true" on "--net"
follow the "exe" path in the /proc entry for this process and launch the
network backend (via the vmm_path field.)

Currently this only supports creating a new tap interface as the network
backend also only supports that.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-14 17:32:49 +00:00
Rob Bradford
d054dddcb3 vm-virtio: Retry connections to vhost-user backends
If the connection to the backend fails, sleep and retry up to a count.
This failure could happen because the backend is not yet ready to handle
the connection which is a common case when working with a spawned
backend.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-14 17:32:49 +00:00
Rob Bradford
b04eb4770b vmm: Follow the "exe" symlink from the PID directory in /proc
It is necessary to do this at the start of the VMM execution rather than
later as it must be done in the main thread in order to satisfy the
checks required by PTRACE_MODE_READ_FSCREDS (see proc(5) and
ptrace(2))

The alternative is to run as CAP_SYS_PTRACE but that has its
disadvantages.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-14 17:32:49 +00:00
Rob Bradford
503887843f vm-virtio: vhost_user: net: On shutdown() drop the socket
This causes the vhost-user-net backend to shutdown.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-14 17:32:49 +00:00
Rob Bradford
7c9e8b103f vmm: device_manager: Shutdown all virtio devices
When the DeviceManager is dropped explicitly shutdown() all virtio
devices.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-14 17:32:49 +00:00
Rob Bradford
545ea9ea33 vm-virtio: Add shutdown method to VirtioDevice trait
This allows the VMM to explicitly shutdown devices as part of the VM
shutdown ahead of what Drop::drop() would do.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-14 17:32:49 +00:00
Bo Chen
ebd83699dc main: Display git commit hash with the '--version' option
Add a build-script to propagate the git commit hash to other crates at
compile time through environment variables, and display the hash along
with the '--version' option.

Fixes #729

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-02-14 10:00:14 +01:00
dependabot-preview[bot]
bdb92f9ace build(deps): bump micro_http from 7fb2e46 to 7a23e54
Bumps [micro_http](https://github.com/firecracker-microvm/firecracker) from `7fb2e46` to `7a23e54`.
- [Release notes](https://github.com/firecracker-microvm/firecracker/releases)
- [Commits](7fb2e4658a...7a23e54775)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-14 07:27:28 +00:00
Rob Bradford
2061f0d8b4 tests: Always create shared VFIO directory from scratch
This ensures that any changes to the contents will not affect subsequent
tests runs.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-13 17:55:21 +01:00
Rob Bradford
e8e4f43d52 tests: Use hugepages for test_vfio
Using hugepages with VFIO and virtiofs can improve the performance of
the test_vfio test.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-13 17:55:21 +01:00
Samuel Ortiz
296ada9441 scripts: dev_cli: Fix post build permissions for the whole tree
Fixes: #752

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-02-13 17:52:30 +01:00
Rob Bradford
287897d702 tests: Run test_vfio with PCI binary
The placement of the explicit run of "test_vfio" meant it was run with
MMIO rather than PCI which meant it always failed.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-13 13:37:58 +00:00
dependabot-preview[bot]
16614447eb build(deps): bump serde_json from 1.0.47 to 1.0.48
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.47 to 1.0.48.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.47...v1.0.48)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-13 09:03:24 +00:00
dependabot-preview[bot]
96479dabe6 build(deps): bump vm-memory from f3d1c27 to eb2fc0b
Bumps [vm-memory](https://github.com/rust-vmm/vm-memory) from `f3d1c27` to `eb2fc0b`.
- [Release notes](https://github.com/rust-vmm/vm-memory/releases)
- [Commits](f3d1c2775c...eb2fc0b6d9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-13 07:20:51 +00:00
dependabot-preview[bot]
88c1683029 build(deps): bump memchr from 2.3.1 to 2.3.2
Bumps [memchr](https://github.com/BurntSushi/rust-memchr) from 2.3.1 to 2.3.2.
- [Release notes](https://github.com/BurntSushi/rust-memchr/releases)
- [Commits](https://github.com/BurntSushi/rust-memchr/compare/2.3.1...2.3.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-13 07:20:34 +00:00
dependabot-preview[bot]
8d3e4f989d build(deps): bump micro_http from c9e900c to 7fb2e46
Bumps [micro_http](https://github.com/firecracker-microvm/firecracker) from `c9e900c` to `7fb2e46`.
- [Release notes](https://github.com/firecracker-microvm/firecracker/releases)
- [Commits](c9e900c181...7fb2e4658a)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-12 18:12:43 +00:00
Sebastien Boeuf
53481aabe5 docs: Update documentation related to multiqueue network
Just updating the documentation regarding the pre-creation of a tap
interface supporting multiqueue, by adding what the cloud-hypervisor
--net parameter should look like.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-02-12 18:05:42 +00:00
Sebastien Boeuf
4dd16c2686 vm-virtio: Detect if a tap interface supports multiqueue
By detecting if an existing tap interface supports multiqueue, we now
have the information to determine if the command line parameters
regarding the number of queues is correct.

Fixes #738

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-02-12 18:05:42 +00:00
Sebastien Boeuf
8627656f55 net_util: Provide more accurate error messages
In order to identify better the problem whenever an existing TAP is
provided, this patch enhances the error returned for an easier
diagnosis of the problem.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-02-12 18:05:42 +00:00
dependabot-preview[bot]
6e5338d28f build(deps): bump memchr from 2.3.0 to 2.3.1
Bumps [memchr](https://github.com/BurntSushi/rust-memchr) from 2.3.0 to 2.3.1.
- [Release notes](https://github.com/BurntSushi/rust-memchr/releases)
- [Commits](https://github.com/BurntSushi/rust-memchr/compare/2.3.0...2.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-12 17:29:03 +00:00
Rob Bradford
014844d0da build: Don't fail build on test_vfio failure
test_vfio has been failing consistently on the CI so mark it with
a "#[ignore]" and then forceably build it again but ignore the build
result.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-12 15:37:47 +00:00
dependabot-preview[bot]
779cbfe8bd build(deps): bump backtrace from 0.3.43 to 0.3.44
Bumps [backtrace](https://github.com/rust-lang/backtrace-rs) from 0.3.43 to 0.3.44.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases)
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.43...0.3.44)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-02-11 18:10:45 +00:00
Rob Bradford
700df9eb16 vhost_user_net: Port to new exit event strategy
Implement the exit_event() method on the VhostUserBackend trait. It is
necessary to specify a custom exit event id in this case as the loop is
also used for handling activity on the tap file descriptors.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-11 15:21:07 +01:00
Rob Bradford
c33c38bd96 vhost_user_block: Port to new exit event strategy
Implement the newly added exit_event() method on the VhostUserBackend
trait to allow the backend to provide an EventFD for triggering an exit
of the worker thread.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-11 15:21:07 +01:00
Rob Bradford
da7f31d4bc bin: vhost_user_fs: Port to new exit event strategy
Rather than handling the KILL_EVENT in the event handler itself use the
newly added support in VhostUserBackend for providing a kill event
framework.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-11 15:21:07 +01:00