Commit Graph

4106 Commits

Author SHA1 Message Date
Fei Li
2c576ab312 test: Introduce test_virtio_balloon() function
The new test_virtio_balloon() is to verify if the 'deflate_on_oom'
parameter works. Its testing result is as follows:

1. Start a 4G guest with 2G balloon, check memory once starts up.
total_mem    is 4294967296 bytes
actual_mem   is 2147483648 bytes
orig_balloon is 2147483648 bytes
       total      used      free    shared    buff/cache    available
Mem:   3.8Gi     2.1Gi     1.6Gi     0.0Ki         140Mi        1.6Gi
Swap:     0B        0B        0B

2. Run a command in guest to eat up 25G memory, and check again.
total_mem        is 4294967296 bytes
actual_mem       is 3121610752 bytes
deflated_balloon is 1173356544 bytes
       total      used      free    shared    buff/cache    available
Mem:   3.8Gi     1.2Gi     2.6Gi     0.0Ki          49Mi        2.5Gi
Swap:     0B        0B        0B

From above, we can notice the balloon memory indeed deflates from
2147483648 bytes to 1173356544 bytes once an oom is going to be
triggered.

Signed-off-by: Fei Li <lifei.shirley@bytedance.com>
2021-06-16 09:55:22 +02:00
Li Hangjing
3fed07419a virtio-balloon: use fallocate when balloon inflated
For vhost-user devices, memory should be shared between CLH and
vhost-user backend. However, madvise DONTNEED doesn't working in
this case. So, let's use fallocate PUNCH_HOLE to discard those
memory regions instead.

Signed-off-by: Li Hangjing <lihangjing@bytedance.com>
2021-06-16 09:55:22 +02:00
Fei Li
aa27f0e743 virtio-balloon: add deflate_on_oom support
Sometimes we need balloon deflate automatically to give memory
back to guest, especially for some low priority guest processes
under memory pressure. Enable deflate_on_oom to support this.

Usage: --balloon "size=0,deflate_on_oom=on" \

Signed-off-by: Fei Li <lifei.shirley@bytedance.com>
2021-06-16 09:55:22 +02:00
Sebastien Boeuf
a6fe4aa7e9 virtio-devices, vmm: Update virtio-iommu to rely on VIOT
Since using the VIRTIO configuration to expose the virtual IOMMU
topology has been deprecated, the virtio-iommu implementation must be
updated.

In order to follow the latest patchset that is about to be merged in the
upstream Linux kernel, it must rely on ACPI, and in particular the newly
introduced VIOT table to expose the information about the list of PCI
devices attached to the virtual IOMMU.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-15 17:05:59 +02:00
Fei Li
de7cfd61a0 tests: Clean up unused workload_path
Signed-off-by: Fei Li <lifei.shirley@bytedance.com>
2021-06-15 15:21:26 +02:00
dependabot[bot]
1ff4d4e3ae build: bump vm-fdt from 2e4ebde to c89583f
Bumps [vm-fdt](https://github.com/rust-vmm/vm-fdt) from `2e4ebde` to `c89583f`.
- [Release notes](https://github.com/rust-vmm/vm-fdt/releases)
- [Commits](2e4ebdeb27...c89583fa49)

---
updated-dependencies:
- dependency-name: vm-fdt
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-15 07:23:20 +00:00
Sebastien Boeuf
744e9d06e5 virtio-devices: vhost_user: Fix wrong naming regarding reconnection
Since the reconnection thread took on the responsibility to handle
backend initiated requests as well, the variable naming should reflect
this by avoiding the "reconnect" prefix.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-14 08:25:15 -07:00
dependabot[bot]
428c637506 build: bump libc from 0.2.96 to 0.2.97
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.96 to 0.2.97.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.96...0.2.97)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-14 09:50:38 +00:00
dependabot[bot]
f67ae9b47e build: bump object from 0.25.2 to 0.25.3
Bumps [object](https://github.com/gimli-rs/object) from 0.25.2 to 0.25.3.
- [Release notes](https://github.com/gimli-rs/object/releases)
- [Changelog](https://github.com/gimli-rs/object/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gimli-rs/object/compare/0.25.2...0.25.3)

---
updated-dependencies:
- dependency-name: object
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-14 09:00:25 +00:00
dependabot[bot]
251708b908 build: bump env_logger from 0.8.3 to 0.8.4
Bumps [env_logger](https://github.com/env-logger-rs/env_logger) from 0.8.3 to 0.8.4.
- [Release notes](https://github.com/env-logger-rs/env_logger/releases)
- [Changelog](https://github.com/env-logger-rs/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/env-logger-rs/env_logger/compare/v0.8.3...v0.8.4)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-14 08:50:26 +00:00
dependabot[bot]
40f73e4fa1 build: bump once_cell from 1.7.2 to 1.8.0 in /fuzz
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.7.2 to 1.8.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.7.2...v1.8.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-14 08:34:35 +00:00
dependabot[bot]
62e1721c52 build: bump libc from 0.2.96 to 0.2.97 in /fuzz
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.96 to 0.2.97.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.96...0.2.97)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-14 08:28:16 +00:00
Anatol Belski
57f70c0761 tests: Simplify Windows guest boot and DHCP setup
A new method has been introduced for WindowsGuest, that would poll on
the SSH connection and try to execute a test command. When successfull,
the polling stops and the guest is considered finished boot. The
downside of this method is that the network setup is needed always even
if the test doesn't use network, however it complies with the behavior
of other tests. I also observe a bit more stability on the local test
run, however it still appears to be a resource issue so some sporadic
fails are possible on a slower machine.

The hardcoded timeouts for guest boot and DHCP setup have been removed.

The dnsmasq invocation uses `--bind-dynamic` so then the daemon can be
started while the target interface could be down yet.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Co-authored-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-14 09:38:34 +02:00
Jiachen Zhang
0699dc1000 virtio-devices: vhost_user: fs: Enable inflight I/O tracking
Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
2021-06-11 17:13:29 +02:00
Jiachen Zhang
2e3e64a4cf virtio-devices: vhost_user: net: Enable inflight I/O tracking
Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
2021-06-11 17:13:29 +02:00
Jiachen Zhang
36d336841c virtio-devices: vhost_user: blk: Enable inflight I/O tracking
Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
2021-06-11 17:13:29 +02:00
Jiachen Zhang
bfd4aa2fed virtio-devices: vhost_user: Support inflight I/O tracking
Vhost user INFLIGHT_SHMFD protocol feature supports inflight I/O
tracking, this commit implement the vhost-user device (master) support
of the feature. Till this commit, specific vhost-user devices (blk, fs,
or net) have not enable this feature.

Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
2021-06-11 17:13:29 +02:00
Michael Zhao
f060ecd705 tests: Unblock some vhost-user tests
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-06-11 11:03:08 +02:00
dependabot[bot]
5363cee847 build: bump anyhow from 1.0.40 to 1.0.41 in /fuzz
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.40 to 1.0.41.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.40...1.0.41)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-11 07:58:29 +00:00
Henry Wang
31e4d80caf tests: Enable AArch64 API pause/resume test case
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-06-11 09:05:54 +02:00
dependabot[bot]
c7f69cc0df build: bump anyhow from 1.0.40 to 1.0.41
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.40 to 1.0.41.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.40...1.0.41)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-11 07:04:57 +00:00
dependabot[bot]
ee118152ff build: bump iced-x86 from 1.11.3 to 1.12.0
Bumps [iced-x86](https://github.com/icedland/iced) from 1.11.3 to 1.12.0.
- [Release notes](https://github.com/icedland/iced/releases)
- [Commits](https://github.com/icedland/iced/compare/v1.11.3...v1.12.0)

---
updated-dependencies:
- dependency-name: iced-x86
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-11 07:04:46 +00:00
Rob Bradford
5ed8e01faf build: Release v16.0
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-06-10 16:59:52 +01:00
Sebastien Boeuf
1a5c6631a5 virtio-devices: vhost_user: Reconnection for slave request handler
Add the support for reconnecting the backend request handler after a
disconnection/crash happened.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-10 16:17:23 +02:00
Sebastien Boeuf
acec7e34fc virtio-devices: vhost_user: Factorize slave request handler
Since the slave request handler is common to all vhost-user devices, the
same way the reconnection is, it makes sense to handle the requests from
the backend through the same thread.

The reconnection thread now handles both a reconnection as well as any
request coming from the backend.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-10 16:17:23 +02:00
Jiachen Zhang
deca570544 virtio-devices: vhost_user: fs: Support socket reconnection handling
This commit enables socket reconnection for vhost-user-fs backends. Note
that, till this commit:

- The re-establish of the slave communication channel is no supported. So
the socket reconnection does not support virtiofsd with DAX enabled.

- Inflight I/O tracking and restoring is not supported. Therefore, only
virtio-fs daemons that are not processing inflight requests can work
normally after reconnection.

- To make the restarted virtiofsd work normally after reconnection, the
internal status of virtiofsd should also be recovered. This is not the
work of cloud-hypervisor. If the virtio-fs daemon does not support
saving or restoring its internal status, then a re-mount in guest after
socket reconnection should be performed.

Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
2021-06-10 16:17:23 +02:00
Michael Zhao
7cfb2139d0 tests: Enable pty test case on AArch64
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-06-10 15:05:25 +02:00
Michael Zhao
1a7a76511f vmm: Enable pty console on AArch64
Allowed syscall "SYS_readlinkat" on AArch64 which is required by pty.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-06-10 15:05:25 +02:00
Bo Chen
8a22b2987f docs: Add documentation for live migration
Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-06-10 12:07:14 +02:00
Henry Wang
1eb8a4671f arch: aarch64: Remove hardcoded host IPA size
With the ability of getting host IPA size in `hypervisor` crate,
we can query the host IPA size through ioctl instead of hardcoding
a maximum IPA size. Therefore this commit removes the hardcoded
maximum host IPA size.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-06-10 12:06:17 +02:00
Henry Wang
2fe3586eba hypervisor: support AArch64 get_host_ipa_limit
Not all AArch64 platforms support IPAs up to 40 bits. Since the
kvm-ioctl crate now supports `get_host_ipa_limit` for AArch64,
when creating the VM, it is better to get the IPA size from the
host and use that to create the VM.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-06-10 12:06:17 +02:00
Henry Wang
805cb303d5 hypervisor: Add get_host_ipa_limit for AArch64
This commit adds a helper `get_host_ipa_limit` to the AArch64
`KvmHypervisor` struct. This helper can be used to get the
`Host_IPA_Limit`, which is the maximum possible value for
IPA_Bits on the host and is dependent on the CPU capability
and the kernel configuration.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-06-10 12:06:17 +02:00
Michael Zhao
97307a7fc1 tests: Remove test_aarch64_pe_boot case
test_aarch64_pe_boot was added at the very beginning of AArch64 support.
Now it can be combined to test_vmlinux_boot.

Renamed test_vmlinux_boot to test_direct_kernel_boot for generality.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-06-10 10:10:53 +02:00
Michael Zhao
6ed1f83a87 tests: Enable the boot_noacpi case on AArch64
Enabled test case test_vmlinux_boot_noacpi on AArch64 and renamed it
test_direct_kernel_boot_noacpi for generality.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-06-10 10:10:53 +02:00
Michael Zhao
04aff2e2f9 tests: Enable test_console_file case on AArch64
This test case could have been ready on AArch64 since the serial issue
was fixed.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-06-10 10:10:53 +02:00
dependabot[bot]
0268089ac6 build: bump syn from 1.0.72 to 1.0.73 in /fuzz
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.72...1.0.73)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-10 08:10:14 +00:00
dependabot[bot]
5d5d338e10 build: bump syn from 1.0.72 to 1.0.73
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.72...1.0.73)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-10 10:09:57 +02:00
Sebastien Boeuf
8f1404eeef deps: Update Cargo.lock to include uuid crate
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-09 14:35:07 +01:00
Sebastien Boeuf
9dba097224 ci: Update guest cloud image by installing "stress" utility
Some new integration tests will require the "stress" binary to be
present in the guest in order to run correctly.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-09 14:34:55 +01:00
Michael Zhao
ce5c5833cb tests: Add a simple test case for ACPI on AArch64
In the test case, the guest boots from EDK2.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-06-09 18:36:59 +08:00
Henry Wang
3cb2b72e92 tests: Build EDK2 for AArch64 integration test
This commit added support for building the edk2 binary dedicated
for the Cloud Hypervisor (CLOUDHV_EFI.fd).

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-06-09 18:36:59 +08:00
Michael Zhao
88fda7c305 aarch64, acpi: Change PCIe high space for EDK2
EDK2 requires the beginning of PCIe high space above 4G address.
In CLH the space follows the RAM. If the RAM space is small, the PCIe
high space could fall bellow 4G.
Here we put it above 512G in FDT to workaround the EDK2 check only when
ACPI is enabled, because EDK2 collects PCIe information from FDT.
The address written in ACPI is not impacted.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-06-09 18:36:59 +08:00
Jianyong Wu
b8b5dccfd8 aarch64: Enable UEFI image loading
Implemented an architecture specific function for loading UEFI binary.

Changed the logic of loading kernel image:
1. First try to load the image as kernel in PE format;
2. If failed, try again to load it as formatless UEFI binary.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-06-09 18:36:59 +08:00
Jianyong Wu
6880692a78 vmm, acpi: Add DSM method to ACPI
_DSM (Device Specific Method) is a control method that enables devices
to provide device specific control functions. Linux kernel will evaluate
this device then initialize preserve_config in acpi pci initialization.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-06-09 18:36:59 +08:00
dependabot[bot]
76911fbeb4 build: bump libc from 0.2.95 to 0.2.96 in /fuzz
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.95 to 0.2.96.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.95...0.2.96)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-09 07:28:02 +00:00
dependabot[bot]
0eb245e177 build: bump vhost from e4ba734 to 30ba3e7
Bumps [vhost](https://github.com/rust-vmm/vhost) from `e4ba734` to `30ba3e7`.
- [Release notes](https://github.com/rust-vmm/vhost/releases)
- [Commits](e4ba734144...30ba3e7bbe)

---
updated-dependencies:
- dependency-name: vhost
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-09 07:27:50 +00:00
dependabot[bot]
c866ccb3a1 build: bump libc from 0.2.95 to 0.2.96
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.95 to 0.2.96.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.95...0.2.96)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-09 07:27:37 +00:00
Sebastien Boeuf
bb68e0bc3b tests: Re-enable virtiofsd tests
Re-enable virtiofsd testing now that issues with capstone repository
have been resolved.

This reverts commit a14c70019a.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-08 20:13:41 +02:00
dependabot[bot]
f81f4490b0 build: bump vfio-ioctls from 5cb8550 to 0f7eb6f
Bumps [vfio-ioctls](https://github.com/rust-vmm/vfio-ioctls) from `5cb8550` to `0f7eb6f`.
- [Release notes](https://github.com/rust-vmm/vfio-ioctls/releases)
- [Commits](5cb8550556...0f7eb6f855)

---
updated-dependencies:
- dependency-name: vfio-ioctls
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-08 07:52:19 +00:00
Wei Liu
c419d3286b hypervisor: emulator: emulate OR instruction
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2021-06-07 11:54:57 -07:00