Commit Graph

2948 Commits

Author SHA1 Message Date
Michael Zhao
69394c9c35 vmm: Handle hypervisor VCPU run result from Vcpu to VcpuManager
Now Vcpu::run() returns a boolean value to VcpuManager, indicating
whether the VM is going to reboot (false) or just continue (true).
Moving the handling of hypervisor VCPU run result from Vcpu to
VcpuManager gives us the flexibility to handle more scenarios like
shutting down on AArch64.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-10-30 17:14:44 +00:00
Michael Zhao
0f5e5d9e6d tests: Fix some build warnings in integration test on AArch64
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-10-30 17:14:12 +00:00
Michael Zhao
4606f0f28a tests: Make migration test case X86 only
Set the test case test_snapshot_restore X86 only, instead of excluding
it from test command line.

The command line option was added because we used to support migration
with Virtio-MMIO, but not Virtio-PCI.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-10-30 17:14:12 +00:00
Rob Bradford
cb88ceeae8 vmm: memory_manager: Move the restoration of guest memory later
Rather than filling the guest memory from a file at the point of the the
guest memory region being created instead fill from the file later. This
simplifies the region creation code but also adds flexibility for
sourcing the guest memory from a source other than an on disk file.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-30 12:31:47 +01:00
Rob Bradford
7dcd3aff05 build, release-notes.md: Document 0.11.0 release
Update release notes and version number for the new release.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-29 17:36:02 +00:00
Bo Chen
8c268e351c tests: Port more tests to use wait_vm_boot
Tests not ported include 1) the ones that start guest VMs without
network (e.g. test_net_hotplug, test_initramfs), 2) test_vfio that
involves l2 guest. Also, some tests that use bionic guest image are
given extended timeout (120s) for 'wait_vm_boot'.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-29 15:30:17 +00:00
Bo Chen
a79f058a90 tests: Accept custom timeout for wait_vm_boot
This allows known slow tests to have a longer/customized timeout while
keeping the default timeout short.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-29 15:30:17 +00:00
Bo Chen
26783fea89 tests: Wait explicitly for vm shutdown w/ the 'wait-timeout' crate
Instead of waiting blindly with fixed amount of sleeping time, we can
use the `wait-timeout` crate to explicitly wait VM shutdown (with a
timeout). It can reduces the execution time of some tests
substantially. Also, this patch increases the `shutdown` timeout for
'test_reboot', which should fix the recent sporadic failures on this
test.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-29 15:30:17 +00:00
Bo Chen
cd1c2ed31e build: Add the 'v' prefix when using the crate version
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-29 08:19:25 -07:00
dependabot-preview[bot]
b947320feb build(deps): bump memchr from 2.3.3 to 2.3.4
Bumps [memchr](https://github.com/BurntSushi/rust-memchr) from 2.3.3 to 2.3.4.
- [Release notes](https://github.com/BurntSushi/rust-memchr/releases)
- [Commits](https://github.com/BurntSushi/rust-memchr/compare/2.3.3...2.3.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-29 09:24:32 +00:00
Rob Bradford
d54544046c build: Use stable virtio-fs QEMU branch
Address build failure from activity in the development virtio-fs branch
by using the stable fork.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-29 08:52:25 +00:00
Bo Chen
8c92d1dbdc tests: Wait explicitly for the guest vm to boot
Instead of blindly waiting for 20-40s for the guest VM to boot, this
patch waits the notification from the guest VM explicitly by using a
simple TcpListener on the host and a custom systemd service in the
guest.

This patch also ported few tests to use this new machanism, while more
tests are to be ported.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-10-28 11:27:25 -07:00
dependabot-preview[bot]
cc67a05474 build(deps): bump proc-macro-hack from 0.5.18 to 0.5.19
Bumps [proc-macro-hack](https://github.com/dtolnay/proc-macro-hack) from 0.5.18 to 0.5.19.
- [Release notes](https://github.com/dtolnay/proc-macro-hack/releases)
- [Commits](https://github.com/dtolnay/proc-macro-hack/compare/0.5.18...0.5.19)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-28 08:04:11 +00:00
Rob Bradford
21db6f53c8 vmm: memory_manager: Write all guest region to disk
As a mirror of bdbea19e23 which ensured
that GuestMemoryMmap::read_exact_from() was used to read all the file to
the region ensure that all the guest memory region is written to disk.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-27 12:11:31 -07:00
Rob Bradford
be1b6bc1e1 main: Remove API socket when exiting
When exiting remove the API socket from the filesystem.

Fixes: #1241

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-27 13:27:23 +00:00
Rob Bradford
c22b788b47 main: Simplify error and return handling in start_vmm
Use a Result<> type with an error to simplify the code in start_vmm().
This will also make it easier to add cleanup funtionality.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-27 13:27:23 +00:00
Rob Bradford
dfd21cbfc5 vmm: Use thiserror/anyhow for vmm::Error
This gives a nicer user experience and this error can now be used as the
source for other errors based off this.

See: #1910

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-27 13:27:23 +00:00
Rob Bradford
6d4656c68f virtio-devices: seccomp_filters: Add fsync to block io_uring filter
This is required when booting with hypervisor-fw.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-27 10:53:54 +00:00
Rob Bradford
338521c700 build: Enable io_uring feature by default
Now that we that our CI is running with a kernel that is new enough to
support io_uring we can turn this feature on by default.

See: #1561

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-27 10:53:54 +00:00
Muminul Islam
2e1866a5db scripts: dev_cli: Add an option to dev_cli to export volumes
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-10-27 09:52:35 +00:00
Rob Bradford
8bd3fdea8f build: Move on-demand workers from bionic to groovy
This is an interim step to be able to test io_uring on our CI system.

See: #1561

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-26 14:27:20 -07:00
dependabot-preview[bot]
f0d0d8ccaf build(deps): bump libc from 0.2.79 to 0.2.80
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.79 to 0.2.80.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.79...0.2.80)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-26 09:31:03 +00:00
dependabot-preview[bot]
0415146018 build(deps): bump syn from 1.0.46 to 1.0.48
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.46 to 1.0.48.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.46...1.0.48)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-24 21:58:39 +00:00
dependabot-preview[bot]
499fbd0367 build(deps): bump futures from 0.3.6 to 0.3.7
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.6...0.3.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-24 00:08:41 +00:00
dependabot-preview[bot]
e70674943f build(deps): bump futures-sink from 0.3.6 to 0.3.7
Bumps [futures-sink](https://github.com/rust-lang/futures-rs) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.6...0.3.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-23 20:00:07 +00:00
Sebastien Boeuf
7e127df415 vmm: memory_manager: Replace 'ext_region' by 'saved_region'
Any occurrence of of a variable containing `ext_region` is replaced with
the less confusing name `saved_region`. The point is to clearly identify
the memory regions that might have been saved during a snapshot, while
the `ext` standing for `external` was pretty unclear.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-23 21:59:52 +02:00
Sebastien Boeuf
c0e8e5b53f vmm: memory_manager: Replace 'backing_file' variable names
In the context of saving the memory regions content through snapshot,
using the term "backing file" brings confusion with the actual backing
file that might back the memory mapping.

To avoid such conflicting naming, the 'backing_file' field from the
MemoryRegion structure gets replaced with 'content', as this is
designating the potential file containing the memory region data.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-23 21:59:52 +02:00
dependabot-preview[bot]
1409b36cad build(deps): bump futures-task from 0.3.6 to 0.3.7
Bumps [futures-task](https://github.com/rust-lang/futures-rs) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.6...0.3.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-23 18:58:11 +00:00
dependabot-preview[bot]
cf067d94c3 build(deps): bump futures-io from 0.3.6 to 0.3.7
Bumps [futures-io](https://github.com/rust-lang/futures-rs) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.6...0.3.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-23 18:57:45 +00:00
dependabot-preview[bot]
c377c1398f build(deps): bump futures-macro from 0.3.6 to 0.3.7
Bumps [futures-macro](https://github.com/rust-lang/futures-rs) from 0.3.6 to 0.3.7.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.6...0.3.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-23 18:57:26 +00:00
dependabot-preview[bot]
b64a48adb8 build(deps): bump arrayvec from 0.5.1 to 0.5.2
Bumps [arrayvec](https://github.com/bluss/arrayvec) from 0.5.1 to 0.5.2.
- [Release notes](https://github.com/bluss/arrayvec/releases)
- [Changelog](https://github.com/bluss/arrayvec/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bluss/arrayvec/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-23 17:27:29 +00:00
Rob Bradford
bdbea19e23 vmm: memory_manager: Completely fill guest ram from snapshot
Use GuestRegionMmap::read_exact_from() to ensure that all of the file is
read into the guest. This addresses an issue where
GuestRegionMmap::read_from() was only copying the first 2GiB of the
memory and so lead to snapshot-restore was failing when the guest RAM
was 2GiB or greater.

This change also propagates any error from the copying upwards.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-23 17:56:19 +01:00
Rob Bradford
8de3bd728c ch-remote, api_client: Split HTTP/API client code into new crate
Split out the HTTP request handling code from ch-remote into a new
crate which can be used in other places where talking to the API server
by HTTP is necessary.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-23 14:50:36 +02:00
Rob Bradford
654846f433 bin: ch-remote: Generalise functions that work on a HTTP socket
Use the trait for Read/Write rather than specifying the concrete type.
This allows for the functionality to be used for different socket types.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-23 14:50:36 +02:00
Rob Bradford
a60b437f89 vmm: memory_manager: Always copy anonymous RAM regions from disk
When restoring if a region of RAM is backed by anonymous memory i.e from
memfd_create() then copy the contents of the ram from the file that has
been saved to disk.

Previously the code would map the memory from that file into the guest
using a MAP_PRIVATE mapping. This has the effect of
minimising the restore time but provides an issue where the restored VM
does not have the same structure as the snapshotted VM, in particular
memory is backed by files in the restored VM that were anonymously
backed in the original.

This creates two problems:

* The snapshot data is mapped from files for the pages of the guest
  which prevents the storage from being reclaimed.
* When snapshotting again the guest memory will not be correctly saved
  as it will have looked like it was backed by a file so it will not be
  written to disk but as it is a MAP_PRIVATE mapping the changes will
  never be written to the disk again. This results in incorrect
  behaviour.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-23 12:34:32 +02:00
Wei Liu
255dbd29ef tests: factor out windows_auth
This avoid repetition once we have more Windows tests.

No functional change.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-10-22 18:23:08 +02:00
Wei Liu
2902a96ae6 tests: add Windows guest snapshot test
We also need to restrict the number of test threads to 1 to avoid tests
interfere with each other.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-10-22 18:23:08 +02:00
Sebastien Boeuf
89b3d7b49b ci: Update integration tests with latest balloon changes
Now that virtio-balloon is not declared as part of the --memory
parameter, the integration tests are updated to keep the correct
behavior.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-22 16:33:16 +02:00
Sebastien Boeuf
f4e391922f vmm: Remove balloon options from --memory parameter
The standalone `--balloon` parameter being fully functional at this
point, we can get rid of the balloon options from the --memory
parameter.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-22 16:33:16 +02:00
Sebastien Boeuf
3594685279 vmm: Move balloon code from MemoryManager to DeviceManager
Now that we have a new dedicated way of asking for a balloon through the
CLI and the REST API, we can move all the balloon code to the device
manager. This allows us to simplify the memory manager, which is already
quite complex.

It also simplifies the behavior of the balloon resizing command. Instead
of providing the expected size for the RAM, which is complex when memory
zones are involved, it now expects the balloon size. This is a much more
straightforward behavior as it really resizes the balloon to the desired
size. Additionally to the simplication, the benefit of this approach is
that it does not need to be tied to the memory manager at all.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-22 16:33:16 +02:00
Sebastien Boeuf
1d479e5e08 vmm: Introduce new --balloon parameter
This introduces a new way of defining the virtio-balloon device. Instead
of going through the --memory parameter, the idea is to consider balloon
as a standalone virtio device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-22 16:33:16 +02:00
dependabot-preview[bot]
ae296d2229 build(deps): bump syn from 1.0.45 to 1.0.46
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.45 to 1.0.46.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.45...1.0.46)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-22 06:53:06 +00:00
dependabot-preview[bot]
555610181b build(deps): bump rustc-demangle from 0.1.17 to 0.1.18
Bumps [rustc-demangle](https://github.com/alexcrichton/rustc-demangle) from 0.1.17 to 0.1.18.
- [Release notes](https://github.com/alexcrichton/rustc-demangle/releases)
- [Commits](https://github.com/alexcrichton/rustc-demangle/compare/0.1.17...0.1.18)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-10-21 21:26:57 +00:00
Sebastien Boeuf
28e12e9f3a vmm, hypervisor: Fix snapshot/restore for Windows guest
The snasphot/restore feature is not working because some CPU states are
not properly saved, which means they can't be restored later on.

First thing, we ensure the CPUID is stored so that it can be properly
restored later. The code is simplified and pushed down to the hypervisor
crate.

Second thing, we identify for each vCPU if the Hyper-V SynIC device is
emulated or not. In case it is, that means some specific MSRs will be
set by the guest. These MSRs must be saved in order to properly restore
the VM.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-21 19:11:03 +01:00
Rob Bradford
700f63fad8 tests: Add integration test for virtio-watchdog
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-21 16:02:39 +01:00
Rob Bradford
7ee63134d2 resources: Update Linux config to enable virtio-watchdog
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-21 16:02:39 +01:00
Rob Bradford
885ee9567b vmm: Add support for creating virtio-watchdog
The watchdog device is created through the "--watchdog" parameter. At
most a single watchdog can be created per VM.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-21 16:02:39 +01:00
Rob Bradford
d2c7645731 virtio-devices: Add simple virtio-watchdog device
This device operates a single virtq. When the driver offers a descriptor
to the device it is interpreted as a "ping" to indicate that the guest
is alive. A periodic timer fires and if when the timer is fired there
has not been a "ping" from the guest then the device will reset the VM.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-21 16:02:39 +01:00
Rob Bradford
ae375434cd vm-virtio: Add (temporary) number allocation for virtio-watchdog
The next available number for allocation is 35.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-21 16:02:39 +01:00
Michael Zhao
0b0596ef30 arch: Simplify PCI space address handling in AArch64 FDT
Before Virtio-mmio was removed, we passed an optional PCI space address
parameter to AArch64 code for generating FDT. The address is none if the
transport is MMIO.
Now Virtio-PCI is the only option, the parameter is mandatory.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-10-21 12:20:30 +01:00