Commit Graph

2812 Commits

Author SHA1 Message Date
Bo Chen
9ab2a34b40 vmm: Remove reserved 256M gaps for hotplugging memory with ACPI
We are now reserving a 256M gap in the guest address space each time
when hotplugging memory with ACPI, which prevents users from hotplugging
memory to the maximum size they requested. We confirm that there is no
need to reserve this gap.

This patch removes the 'reserved gaps'. It also refactors the
'MemoryManager::start_addr' so that it is rounding-up to 128M alignment
when hotplugged memory is allowed with ACPI.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-13 12:09:22 +02:00
Henry Wang
8bf0bac5a7 dev_cli: Fix TARGETARCH inconsistancy
In previous dev_cli.sh, the `uname -m` command will generate
either `x86_64` or `aarch64`, which is inconsistent with the
architectures in the Dockerfile, namely `amd64` and `arm64`.

This will cause some dependancy missing in the docker container
when the docker image is built locally.

This commit fixes this inconsistancy.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-10-13 11:57:01 +02:00
dependabot-preview[bot]
ef1e5fd953 build(deps): bump serde_json from 1.0.58 to 1.0.59
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.58 to 1.0.59.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.58...v1.0.59)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-13 06:57:48 +00:00
dependabot-preview[bot]
a06a6e4696 build(deps): bump aho-corasick from 0.7.13 to 0.7.14
Bumps [aho-corasick](https://github.com/BurntSushi/aho-corasick) from 0.7.13 to 0.7.14.
- [Release notes](https://github.com/BurntSushi/aho-corasick/releases)
- [Commits](https://github.com/BurntSushi/aho-corasick/compare/0.7.13...0.7.14)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-13 06:57:37 +00:00
Bo Chen
10f380f95b vmm: Report no error when resizing to current memory size with ACPI
We now try to create a ram region of size 0 when the requested memory
size is the same as current memory size. It results in an error of
`GuestMemoryRegion(Mmap(Os { code: 22, kind: InvalidInput, message:
"Invalid argument" }))`. This error is not meaningful to users and we
should not report it.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-12 08:46:38 +02:00
dependabot-preview[bot]
299e83bf43 build(deps): bump regex from 1.3.9 to 1.4.0
Bumps [regex](https://github.com/rust-lang/regex) from 1.3.9 to 1.4.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.3.9...1.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-12 05:17:07 +00:00
dependabot-preview[bot]
54d2298a93 build(deps): bump regex-syntax from 0.6.18 to 0.6.19
Bumps [regex-syntax](https://github.com/rust-lang/regex) from 0.6.18 to 0.6.19.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-12 05:16:46 +00:00
dependabot-preview[bot]
74627d2de5 build(deps): bump backtrace from 0.3.52 to 0.3.53
Bumps [backtrace](https://github.com/rust-lang/backtrace-rs) from 0.3.52 to 0.3.53.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases)
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.52...0.3.53)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-12 05:16:17 +00:00
dependabot-preview[bot]
818dc72e5f build(deps): bump pin-project from 0.4.26 to 0.4.27
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 0.4.26 to 0.4.27.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/master/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v0.4.26...v0.4.27)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-12 05:12:45 +00:00
dependabot-preview[bot]
a4be05ac40 build(deps): bump syn from 1.0.43 to 1.0.44
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.43 to 1.0.44.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.43...1.0.44)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-12 05:12:12 +00:00
Muminul Islam
029a6d8ab9 README: Add steps to install prerequisites
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-10-10 09:52:06 +02:00
Muminul Islam
0f48b61265 README: Syntax cleanups
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-10-10 09:52:06 +02:00
Bo Chen
789ee7b3e4 vmm: Support resizing memory up to and including hotplug size
The start address after the hottplugged memory can be the start address of
device area.

Fixes: #1803

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-10 09:51:32 +02:00
dependabot-preview[bot]
f639a8a0fd build(deps): bump syn from 1.0.42 to 1.0.43
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.42 to 1.0.43.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.42...1.0.43)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-09 17:56:30 +00:00
dependabot-preview[bot]
eabfd642e0 build(deps): bump cc from 1.0.60 to 1.0.61
Bumps [cc](https://github.com/alexcrichton/cc-rs) from 1.0.60 to 1.0.61.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases)
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.60...1.0.61)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-09 10:23:32 +00:00
dependabot-preview[bot]
ab32375666 build(deps): bump backtrace from 0.3.51 to 0.3.52
Bumps [backtrace](https://github.com/rust-lang/backtrace-rs) from 0.3.51 to 0.3.52.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases)
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.51...0.3.52)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-09 09:33:32 +00:00
Michael Zhao
2a7b1d78ff build: Fix AArch64 musl cross build failure
A failure appeared in AArch64 musl cross build, after upgrading rust
to v1.47.0. A symbol "strrchr" was missing while linking against
static libfdt.a.

The issue could be caused by missing symbol(s) in new rust toolchain.

This fix pins the rust version in this cross build action to a stable-
enough version. Further upgrade will be done manually after testing.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-10-09 10:49:54 +02:00
Rob Bradford
573a5c63cf hypervisor: kvm: Use unstable_sort() to keep clippy happy
"Using a stable sort consumes more memory and cpu cycles. Because values
which compare equal are identical, preserving their relative order (the
guarantee that a stable sort provides) means nothing, while the extra
costs still apply."

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-09 10:49:54 +02:00
Rob Bradford
bb5b9584d2 pci, ch-remote, vmm: Replace simple match blocks with matches!
This is a new clippy check introduced in 1.47 which requires the use of
the matches!() macro for simple match blocks that return a boolean.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-09 10:49:54 +02:00
Sebastien Boeuf
2fac41f154 deps: Patch kvm-ioctls and kvm-bindings crates
Instead of having the hypervisor crate embedding Cloud-Hypervisor forks
from the rust-vmm project, it's more appropriate to leave the rust-vmm
references in the hypervisor crate, and have the root Cargo.toml being
patched.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-08 18:45:42 +02:00
Anatol Belski
095ea049e2 doc: Add UEFI and Windows consumer pages
This changeset extends the documentation with the UEFI and Windows
related info. The focus is on providing consumer with a minimum
necessary and proper piece of the information to enter the features
quickly. While UEFI is a cross platform topic, it is a required
prerequisite for the Windows usage.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2020-10-08 15:45:30 +01:00
dependabot-preview[bot]
ff0d2431d0 build(deps): bump rustc-demangle from 0.1.16 to 0.1.17
Bumps [rustc-demangle](https://github.com/alexcrichton/rustc-demangle) from 0.1.16 to 0.1.17.
- [Release notes](https://github.com/alexcrichton/rustc-demangle/releases)
- [Commits](https://github.com/alexcrichton/rustc-demangle/compare/0.1.16...0.1.17)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-08 08:59:34 +00:00
Wei Liu
ed1fdd1f7d hypervisor, arch: rename "OneRegister" and relevant code
The OneRegister literally means "one (arbitrary) register". Just call it
"Register" instead. There is no need to inherit KVM's naming scheme in
the hypervisor agnostic code.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-10-08 08:55:10 +02:00
Wei Liu
9ad14e6b3a aarch64: Add OneReg to the list required extensions for KVM
Without that capability save / restore for aarch64 won't work.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-10-08 08:54:38 +02:00
Sebastien Boeuf
30e1162d7c virtio: Replace some repr(C, packed) with repr(C)
The goal here is to replace anywhere possible a virtio structure
with a "C, packed" representation by a "C" representation. Some
virtio structures are not expected to be packed, therefore there's
no reason for using the more restrictive "C, packed" representation.

This is important since "packed" representation can still cause
undefined behaviors with Rust 2018.

By removing the need for "packed" representation, we can simplify a
bit of code by deriving the Serialize trait without writing the
implementation ourselves.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-07 22:11:36 +02:00
dependabot-preview[bot]
2333c5e233 build(deps): bump miniz_oxide from 0.4.2 to 0.4.3
Bumps [miniz_oxide](https://github.com/Frommi/miniz_oxide) from 0.4.2 to 0.4.3.
- [Release notes](https://github.com/Frommi/miniz_oxide/releases)
- [Changelog](https://github.com/Frommi/miniz_oxide/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Frommi/miniz_oxide/compare/0.4.2...0.4.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-07 15:55:10 +00:00
Sebastien Boeuf
1e3a6cb450 vmm: Simplify some of the io_uring code
Small patch creating a dedicated `block_io_uring_is_supported()`
function for the non-io_uring case, so that we can simplify the
code in the DeviceManager.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-07 14:26:49 +02:00
Wei Liu
ac32b1e35f ci: use cargo check to validate building commits
The purpose of that step is to make sure each commit builds. The `check`
command is much faster for that purpose.

On my 32-core machine `cargo check --all` takes around 25 seconds while
`cargo build --all` takes around 35 seconds, so that's quite a bit of
time saving there.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-10-07 14:26:32 +02:00
Sebastien Boeuf
67025948dc ci: Simplify test scripts
Since all unit and integration tests are run inside containers because
they are called from dev_cli.sh, they always run as root. That's why
both unit and integration scripts can be simplified as they don't need
to apply specific capabilities and run cargo tests in a dedicated 'kvm'
group.

Fixes #1683

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-07 14:26:13 +02:00
dependabot-preview[bot]
a5d3604272 build(deps): bump anyhow from 1.0.32 to 1.0.33
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.32 to 1.0.33.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.32...1.0.33)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-07 06:58:14 +00:00
dependabot-preview[bot]
f48398083c build(deps): bump thiserror from 1.0.20 to 1.0.21
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.20 to 1.0.21.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.20...1.0.21)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-07 05:38:10 +00:00
Rob Bradford
00c0b9e170 devices: ioapic: Tolerate Windows writing zero to version register
Windows will write zero to the IOAPIC version register as part of
probing if the device is there.

Fixes: #1791

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-06 18:13:33 +01:00
Sebastien Boeuf
162789a2d1 ci: Extend virtio-blk hotplug integration test
The existing virtio-blk hotplug test is extended by removing and
re-adding the virtio-blk device. This ensures the unplug/re-plug
feature is properly tested.

Fixes #1809

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-06 17:22:29 +01:00
Sebastien Boeuf
14163c5200 ci: Extend virtio-net hotplug integration test
The existing virtio-net hotplug test is extended by removing and
re-adding the virtio-net device. This ensures the unplug/re-plug
feature is properly tested.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-06 17:22:29 +01:00
Rob Bradford
8824da27cb docs: Add documentation about the log levels and when to use them
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-06 16:52:29 +01:00
Rob Bradford
03f7d39ce5 main: Set default log level to warn!() equivalent.
Using our standard configuration and default kernel we trigger no
messages at this level.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-06 16:52:29 +01:00
Sebastien Boeuf
c02a02edfc vmm: Allow unlink syscall for vCPU threads
Without the unlink(2) syscall being allowed, Cloud-Hypervisor crashes
when we remove a virtio-vsock device that has been previously added.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-06 16:05:59 +01:00
Sebastien Boeuf
cc7a5aca6c ci: Validate each commit can be built
It's important to ensure Cloud-Hypervisor's Git tree is bisectable, and
that's why this commit extends the existing build.yaml Github action.
This will validate that each commit from an incoming pull request can be
built.

Fixes #1808

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-06 14:28:54 +01:00
Sebastien Boeuf
b33969896b virtio-devices: Rely on vhost crate from Cloud-Hypervisor fork
The Cloud-Hypervisor fork of the vhost crate contains one small
additional patch compared to the rust-vmm upstream version, meant for
increasing the connection timeout.

This patch is intended to be merged in order to check if it helps our CI
fixing the vhost-user-blk flakes that we've been observing recently.

If it fixes it, we'll submit a similar patch upstream and switch back to
the upstream vhost crate, otherwise we'll simply switch back to the
upstream crate, discarding this patch.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-06 12:56:33 +02:00
Sebastien Boeuf
6aa5e21212 vmm: device_manager: Fix PCI device unplug issues
Because of the PCI refactoring that happened in the previous commit
d793cc4da3, the ability to fully remove a
PCI device was altered.

The refactoring was correct, but the usage of a generic function to pass
the same reference for both BusDevice, PciDevice and Any + Send + Sync
causes the Arc::ptr_eq() function to behave differently than expected,
as it does not match the references later in the code. That means we
were not able to remove the device reference from the MMIO and/or PIO
buses, which was leading to some bus range overlapping error once we
were trying to add a device again to the previous range that should have
been removed.

Fixes #1802

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-06 12:56:17 +02:00
dependabot-preview[bot]
a32bd2fae1 build(deps): bump hermit-abi from 0.1.16 to 0.1.17
Bumps [hermit-abi](https://github.com/hermitcore/libhermit-rs) from 0.1.16 to 0.1.17.
- [Release notes](https://github.com/hermitcore/libhermit-rs/releases)
- [Commits](https://github.com/hermitcore/libhermit-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-06 09:25:31 +00:00
dependabot-preview[bot]
e553c053af build(deps): bump futures from 0.3.5 to 0.3.6
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-06 06:34:18 +00:00
dependabot-preview[bot]
83f6aa47c9 build(deps): bump futures-core from 0.3.5 to 0.3.6
Bumps [futures-core](https://github.com/rust-lang/futures-rs) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-05 20:58:50 +00:00
dependabot-preview[bot]
a4ebb6399a build(deps): bump futures-task from 0.3.5 to 0.3.6
Bumps [futures-task](https://github.com/rust-lang/futures-rs) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-05 20:58:24 +00:00
dependabot-preview[bot]
0c25b4d9fb build(deps): bump futures-io from 0.3.5 to 0.3.6
Bumps [futures-io](https://github.com/rust-lang/futures-rs) from 0.3.5 to 0.3.6.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-05 20:57:51 +00:00
dependabot-preview[bot]
c2cc26fc82 build(deps): bump libc from 0.2.78 to 0.2.79
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.78 to 0.2.79.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.78...0.2.79)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-05 07:02:05 +00:00
dependabot-preview[bot]
03eebcede2 build(deps): bump pin-project from 0.4.25 to 0.4.26
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 0.4.25 to 0.4.26.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/v0.4.26/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v0.4.25...v0.4.26)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-04 21:22:01 +00:00
Praveen Paladugu
71c435ce91 hypervisor, vmm: Introduce VmmOps trait
Run loop in hypervisor needs a callback mechanism to access resources
like guest memory, mmio, pio etc.

VmmOps trait is introduced here, which is implemented by vmm module.
While handling vcpuexits in run loop, this trait allows hypervisor
module access to the above mentioned resources via callbacks.

Signed-off-by: Praveen Paladugu <prapal@microsoft.com>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-02 16:42:55 +01:00
Rob Bradford
6a9934d933 build: Fix vm-memory bump build error
A new version of vm-memory was released upstream which resulted in some
components pulling in that new version. Update the version number used
to point to the latest version but continue to use our patched version
due to the fix for #1258

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-02 16:38:02 +01:00
Rob Bradford
2d457ab974 vmm: device_manager: Make PMEM "discard_writes" mode true CoW
The PMEM support has an option called "discard_writes" which when true
will prevent changes to the device from hitting the backing file. This
is trying to be the equivalent of "readonly" support of the block
device.

Previously the memory of the device was marked as KVM_READONLY. This
resulted in a trap when the guest attempted to write to it resulting a
VM exit (and recently a warning). This has a very detrimental effect on
the performance so instead make "discard_writes" truly CoW by mapping
the memory as `PROT_READ | PROT_WRITE` and using `MAP_PRIVATE` to
establish the CoW mapping.

Fixes: #1795

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-02 14:26:15 +02:00