Commit Graph

3040 Commits

Author SHA1 Message Date
Sebastien Boeuf
a8a9e61c3d vmm: memory_manager: Allow host NUMA for RAM backed files
Let's narrow down the limitation related to mbind() by allowing shared
mappings backed by a file backed by RAM. This leaves the restriction on
only for mappings backed by a regular file.

With this patch, host NUMA node can be specified even if using
vhost-user devices.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-27 08:39:38 -07:00
Sebastien Boeuf
1b4591aecc vmm: memory_manager: Apply NUMA policy to memory zones
Relying on the new option 'host_numa_node' from the 'memory-zone'
parameter, the user can now define which NUMA node from the host
should be used to back the current memory zone.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-27 08:39:38 -07:00
Sebastien Boeuf
e6f585a31c vmm: Add 'host_numa_nodes' option to memory zones
Since memory zones have been introduced, it is now possible for a user
to specify multiple backends for the guest RAM. By adding a new option
'host_numa_node' to the 'memory-zone' parameter, we allow the guest RAM
to be backed by memory that might come from a specific NUMA node on the
host.

The option expects a node identifier, specifying which NUMA node should
be used to allocate the memory associated with a specific memory zone.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-27 08:39:38 -07:00
Sebastien Boeuf
ad5d0e4713 vmm: Remove 'mergeable' from memory zones
The flag 'mergeable' should only apply to the entire guest RAM, which is
why it is removed from the MemoryZoneConfig as it is defined as a global
parameter at the MemoryConfig level.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-27 07:26:49 +02:00
dependabot-preview[bot]
be8f40f2b6 build(deps): bump redox_users from 0.3.4 to 0.3.5
Bumps redox_users from 0.3.4 to 0.3.5.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-26 21:22:08 +00:00
Sebastien Boeuf
89e7774b96 vmm: openapi: Don't expect cmdline to always be there
The 'cmdline' parameter should not be required as it is not needed when
the 'kernel' parameter is the rust-hypervisor-fw, which means the kernel
and the associated command line will be found from the EFI partition.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:49:05 +02:00
Sebastien Boeuf
e8149380b7 vmm: memory_manager: Factorize memory regions creation
Factorize the codepath between simple memory and multiple memory zones.
This simplifies the way regions are memory mapped, as everything relies
on the same codepath. This is performed by creating a memory zone on the
fly for the specific use case where --memory is used with size being
different from 0. Internally, the code can rely on memory zones to
create the memory regions forming the guest memory.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
c58dd761f4 vmm: Remove 'file' option from MemoryConfig
After the introduction of user defined memory zones, we can now remove
the deprecated 'file' option from --memory parameter. This makes this
parameter simpler, letting more advanced users define their own custom
memory zones through the dedicated parameter.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
5bf7113768 vmm: memory_manager: Remove restrictions about snapshot/restore
User defined memory regions can now support being snapshot and restored,
therefore this commit removes the restrictions that were applied through
earlier commit.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
2583d572fc vmm: memory_manager: Simplify how to restore memory regions
By factorizing a lot of code into create_ram_region(), this commit
achieves the simplification of the restore codepath. Additionally, it
makes user defined memory zones compatible with snapshot/restore.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
b14c861c6f vmm: memory_manager: Store memory regions content only when necessary
First thing, this patch introduces a new function to identify if a file
descriptor is linked to any hard link on the system. This can let the
VMM know if the file can be accessed by the user, or if the file will
be destroyed as soon as the VMM releases the file descriptor.

Based on this information, and associated with the knowledge about the
region being MAP_SHARED or not, the VMM can now decide to skip the copy
of the memory region content. If the user has access to the file from
the filesystem, and if the file has been mapped as MAP_SHARED, we can
consider the guest memory region content to be present in this file at
any point in time. That's why in this specific case, there's no need for
performing the copy of the memory region content into a dedicated file.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
d1ce52f3a8 vmm: memory_manager: Make backing file from snapshot optional
Let's not assume that a backing file is going to be the result from
a snapshot for each memory region. These regions might be backed by
a file on the host filesystem (not a temporary file in host RAM), which
means they don't need to be copied and stored into dedicated files.

That's why this commit prepares for further changes by introducing an
optional PathBuf associated with the snapshot of each memory region.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
871138d5cc vm-migration: Make snapshot() mutable
There will be some cases where the implementation of the snapshot()
function from the Snapshottable trait will require to modify some
internal data, therefore we make this possible by updating the trait
definition with snapshot(&mut self).

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
7b898285d5 ci: Extend integration tests with user defined memory zones
Adding a small test to validate that user defined memory zones work as
expected when using --memory-zone option.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
c13721fdbd vmm: memory_manager: Handle user defined memory zones
In case the memory size is 0, this means the user defined memory
zones are used as a way to specify how to back the guest memory.

This is the first step in supporting complex use cases where the user
can define exactly which type of memory from the host should back the
memory from the guest.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
7cd3867e2c vmm: memory_manager: Provide file offset through create_ram_region()
In anticipation for the need to map part of a file with the function
create_ram_region(), it is extended to accept a file offset as argument.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
59d4a56ab7 vmm: memory_manager: Don't truncate backing file
In case the provided backing file is an actual file and not a directory,
we should not truncate it, as we expect the file to already be the right
size.

This change will be important once we try to map the same file through
multiple memory mappings. We can't let the file be truncated as the
second mapping wouldn't work properly.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
be475ddc22 main, vmm: Let the user define distincts memory zones
Introducing a new CLI option --memory-zone letting the user specify
custom memory zones. When this option is present, the --memory size
must be explicitly set to 0.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Sebastien Boeuf
d25ec66bb6 vmm: memory_manager: Simplify start_addr()
Small simplification for the function calculating the start address.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-25 16:43:10 +02:00
Anatol Belski
12212d2966 pci: device_manager: Remove hardcoded I/O port assignment
It is otherwise seems to be able to cause resource conflicts with
Windows APCI_HAL. The OS might do a better job on assigning resources
to this device, withouth them to be requested explicitly. 0xcf8 and
0xcfc are only what is certainly needed for the PCI device enumeration.

Signed-off-by: Anatol Belski <anatol.belski@microsoft.com>
2020-08-25 09:00:06 +02:00
Michael Zhao
a4222afd7c scripts: Remove the workaround for "with-serde" in unit tests on AArch64
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-08-24 10:59:08 +02:00
Michael Zhao
afc98a5ec9 vmm: Fix AArch64 clippy warnings of vmm and other crates
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-08-24 10:59:08 +02:00
Michael Zhao
46b8f38987 arch: Fix AArch64 clippy warnings of arch crate
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-08-24 10:59:08 +02:00
Michael Zhao
527e8e7b1d build: Add quality checks workflow for AArch64
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-08-24 10:59:08 +02:00
Amey Narkhede
dc9c12518e docs/iommu: Update cpu and memory option parameters
Use of backing file is deprecated hence use the `hugepages` field.
Also use the `boot` field for specifying number of cpus

Signed-off-by: Amey Narkhede <ameynarkhede02@gmail.com>
2020-08-24 08:49:47 +02:00
dependabot-preview[bot]
0c33bf0044 build(deps): bump syn from 1.0.38 to 1.0.39
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.38 to 1.0.39.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.38...1.0.39)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-24 06:48:33 +00:00
Muminul Islam
92b4499c1e vmm, hypervisor: Add vmstate to snapshot and restore path
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-08-24 08:48:15 +02:00
Muminul Islam
77e901a602 hypervisor: Introduce VM state to Vm hypervisor trait
We may need to store hypervisor speciific data to the VM. This support is
needed for Microsoft hyperv implementations. This patch introduces two
new definitions to Vm trait and implements for KVM.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-08-24 08:48:15 +02:00
dependabot-preview[bot]
8962e25268 build(deps): bump miniz_oxide from 0.4.0 to 0.4.1
Bumps [miniz_oxide](https://github.com/Frommi/miniz_oxide) from 0.4.0 to 0.4.1.
- [Release notes](https://github.com/Frommi/miniz_oxide/releases)
- [Changelog](https://github.com/Frommi/miniz_oxide/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Frommi/miniz_oxide/compare/0.4.0...0.4.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-24 04:00:04 +00:00
Wei Liu
47e8f5475e pci/msix: remove reference to KVM from a comment
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-08-21 16:23:41 +02:00
Wei Liu
e6849699d2 vfio: remove KVM-ism from comments and error messages
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-08-21 16:23:41 +02:00
Wei Liu
571c368528 vfio: fix comment for map_mmio_regions
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-08-21 16:23:41 +02:00
dependabot-preview[bot]
57ff608be9 build(deps): bump libc from 0.2.74 to 0.2.76
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.74 to 0.2.76.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.74...0.2.76)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-21 07:08:35 +00:00
dependabot-preview[bot]
da72d30f53 build(deps): bump ppv-lite86 from 0.2.8 to 0.2.9
Bumps [ppv-lite86](https://github.com/cryptocorrosion/cryptocorrosion) from 0.2.8 to 0.2.9.
- [Release notes](https://github.com/cryptocorrosion/cryptocorrosion/releases)
- [Commits](https://github.com/cryptocorrosion/cryptocorrosion/compare/ppv-lite86-0.2.8...ppv-lite86-0.2.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-21 06:17:47 +00:00
dependabot-preview[bot]
1c9c0fa0e5 build(deps): bump linux-loader from 726360b to 3cf96c5
Bumps [linux-loader](https://github.com/rust-vmm/linux-loader) from `726360b` to `3cf96c5`.
- [Release notes](https://github.com/rust-vmm/linux-loader/releases)
- [Commits](726360bb2c...3cf96c53ed)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-21 06:17:35 +00:00
dependabot-preview[bot]
421f1c9d8b build(deps): bump autocfg from 1.0.0 to 1.0.1
Bumps [autocfg](https://github.com/cuviper/autocfg) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/cuviper/autocfg/releases)
- [Commits](https://github.com/cuviper/autocfg/compare/1.0.0...1.0.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-21 04:14:36 +00:00
Sebastien Boeuf
d751f419de Jenkinsfile: Force the cleanup after stages are aborted
Problem with the previous solution was that Cleanup stage was not
executed when previous stages failed. We fix this by adding a post
section that executes always after the Aarch64 build completed, no
matter if it has failed, succeeded or been aborted.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-19 09:26:36 -07:00
Michael Zhao
cd0b8ed8f8 virtio-devices: Allowing SYS_write syscall for virtio-net-ctl thread
"debug!" marco is used in virtio-devices/src/epoll_helper.rs. When"-vvv"
and "--log-file" option was specified, the missing "SYS_write" rule
caused a "bad system call" crash.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-08-19 14:26:07 +02:00
Sebastien Boeuf
bcbc098c66 Jenkinsfile: Add a cleanup stage
Cleanup of the Aarch64 machine can't be done as part of the parallel
stage as this is often skipped. When the build is aborted because
another parallel stage failed, the post actions are simply not
performed. That's why we need a dedicated stage, out of the parallel
ones, to cleanup the Aarch64 machine.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-08-19 11:23:06 +02:00
Amey Narkhede
9304b92219 docs/virtio-fs: Update --memory option parameters
Use updated --memory option parameters. The field `size` needs
M/G suffix. Without the suffix cloud-hypervisor panics at
src/main.rs:353
Also the use of backing file is deprecated so use `shared` field

Signed-off-by: Amey Narkhede <ameynarkhede02@gmail.com>
2020-08-19 10:43:04 +02:00
Bo Chen
02d87833f0 virtio-devices: seccomp: Add seccomp filters for vhost_blk thread
This patch enables the seccomp filters for the vhost_blk worker thread.

Partially fixes: #925

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-19 08:33:58 +02:00
Bo Chen
4e0ea15075 virtio-devices: seccomp: Add seccomp filter for vhost_net thread
This patch enables the seccomp filters for the vhost_net worker thread.

Partially fixes: #925

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-19 08:33:58 +02:00
Bo Chen
896b9a1d4b virtio-devices: seccomp: Add seccomp filter for vhost_net_ctl thread
This patch enables the seccomp filters for the vhost_net_ctl worker thread.

Partially fixes: #925

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-19 08:33:58 +02:00
Bo Chen
02d63149fe virtio-devices: seccomp: Add seccomp filters for vhost_fs thread
This patch enables the seccomp filters for the vhost_fs worker thread.

Partially fixes: #925

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-19 08:33:58 +02:00
Bo Chen
c82ded8afa virtio-devices: seccomp: Add seccomp filters for balloon thread
This patch enables the seccomp filters for the balloon worker thread.

Partially fixes: #925

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-19 08:33:58 +02:00
Bo Chen
c460178723 virtio-devices: seccomp: Add seccomp filters for mem thread
This patch enables the seccomp filters for the mem worker thread.

Partially fixes: #925

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-19 08:33:58 +02:00
dependabot-preview[bot]
c30ff62a7a build(deps): bump libssh2-sys from 0.2.18 to 0.2.19
Bumps [libssh2-sys](https://github.com/alexcrichton/ssh2-rs) from 0.2.18 to 0.2.19.
- [Release notes](https://github.com/alexcrichton/ssh2-rs/releases)
- [Commits](https://github.com/alexcrichton/ssh2-rs/compare/libssh2-sys-0.2.18...libssh2-sys-0.2.19)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-19 06:24:42 +00:00
dependabot-preview[bot]
58d0aec510 build(deps): bump cc from 1.0.58 to 1.0.59
Bumps [cc](https://github.com/alexcrichton/cc-rs) from 1.0.58 to 1.0.59.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases)
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.58...1.0.59)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-08-19 06:24:29 +00:00
Michael Zhao
24fb5c8387 tests: Enable test case test_counters on AArch64
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-08-18 08:10:04 +02:00
Bo Chen
de3b17d948 virtio-devices: Print out worker thread errors
As we never join the spawned virtio-devices worker threads, the error
returned from each worker thread is lost. For now, we simply print out
the error from each worker thread.

Fixes: #1551

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-18 07:57:11 +02:00