Commit Graph

120 Commits

Author SHA1 Message Date
Rob Bradford
bbc223b93f build: Update version of toolchain in container
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-10-26 10:22:56 +01:00
Bo Chen
27f7b2c2a1 Dockerfile: Build and install SPDK NVMe to the docker image
We now build SPDK-NVMe inside the container only for the x86_64
platform, as the cross-platform build with 'docker buildx' does not work
for SPDK. For aarch64 the platform, we will build it as a part of the CI
workflow (which is running on the bare-metal machine).

Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-10-21 14:29:27 +01:00
Rob Bradford
690bdc3448 Revert "Dockerfile: Build and install SPDK NVMe to the docker image"
This reverts commit a1d8b63d86.

Reverting as this does not successfully cross build.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-10-15 10:17:33 +01:00
Bo Chen
a1d8b63d86 Dockerfile: Build and install SPDK NVMe to the docker image
Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-10-13 14:20:19 +01:00
Jianyong Wu
a9f50dda08 resources: Enable rtc_drv_efi in arm64 kernel config
rtc-efi is used when system boot from UEFI thus the related kernel
driver should be enabled.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-29 16:32:51 +01:00
Henry Wang
6037c83585 resources: Add autotools and texinfo for AArch64
These packages will be used to compile `stress` from source, and
the `stress` will be used by the virtio-balloon integration test.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-09-27 17:34:29 +08:00
Michael Zhao
56c26b3d9c resource: Install libguestfs-tools in Docker image
Installed `libguestfs-tools` to replace kernel file in cloud image.
Installed a kernel as `libguestfs-tools` requires.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-09-23 08:59:58 +01:00
Bo Chen
ae68c802bd Dockerfile: Build and install virtiofsd to the docker image
Given the 'virtiofsd' executable is used in multiple CI workers,
installing them directly to the docker image is more efficient and can
save CI time.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-09-20 13:15:52 +01:00
Rob Bradford
82ace6e327 build: Update version of toolchain in container
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-09-10 10:10:11 -07:00
Jianyong Wu
a2e72774a9 bump kernel version to 5.14 for arm
Bump kernel version and update kernel config file accordingly.
Especially, enable virtio-mem and MEMORY_HOTPLUG_DEFAULT_ONLINE here.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2021-09-07 09:34:22 +01:00
Sebastien Boeuf
f3793c08b5 resources: Update to guest kernel 5.14
Moving to the latest kernel 5.14 simplifies greatly the list of patches
that we need to carry. Since virtio-iommu has been merged as part of the
5.14 release, the only remaining patches are virtio-watchdog and the EFI
reboot.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-08-30 10:23:57 -07:00
Rob Bradford
7935430f6b resources: Add GPIO and NVMe devices needed for vfio-user testing
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-08-10 16:01:00 +01:00
Rob Bradford
c597655d06 resources: Update version of Rust in dev container
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-08-03 17:49:37 +01:00
Muminul Islam
4a3cec8c1f resources: Enable CONFIG_HYPERV in the kernel config
Enabling CONFIG_HYPERV in the Linux Kernel allows
guest to use hyperv clock source. This changes
improves guest performance. Without this changes
we saw slowness in the guest on MSHV.

Signed-off-by: Vineeth Pillai <viremana@linux.microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-07-09 09:23:29 +02:00
Sebastien Boeuf
a6fe4aa7e9 virtio-devices, vmm: Update virtio-iommu to rely on VIOT
Since using the VIRTIO configuration to expose the virtual IOMMU
topology has been deprecated, the virtio-iommu implementation must be
updated.

In order to follow the latest patchset that is about to be merged in the
upstream Linux kernel, it must rely on ACPI, and in particular the newly
introduced VIOT table to expose the information about the list of PCI
devices attached to the virtual IOMMU.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-06-15 17:05:59 +02:00
Henry Wang
3cb2b72e92 tests: Build EDK2 for AArch64 integration test
This commit added support for building the edk2 binary dedicated
for the Cloud Hypervisor (CLOUDHV_EFI.fd).

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-06-09 18:36:59 +08:00
Sebastien Boeuf
fbaceea4c0 resources: Add ovs-dpdk install to Dockerfile
The Dockerfile is extended with some new instructions to install the
OVS-DPDK packages. This will be useful for writing new integration tests
and make sure OVS-DPDK is working properly.

This also updates the Rust toolchain from 1.51.0 to 1.52.1.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-05-27 13:51:06 +01:00
Anatol Belski
65429cebae docker: Add ntfs-3g to the package list
This will make `mkfs.ntfs` available in the next image update, so then
it can be used in the tests.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2021-05-18 17:07:19 +01:00
Henry Wang
1f63350eba resources: Remove libfdt in docker image
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-05-07 18:49:17 +02:00
Rob Bradford
b5cf1f5a9e resources, scripts: Update x86_64 kernel to 5.12
The 5.12 kernel contains a fix for issues observed with nested snapshot
& restore.

Fixes: #2535

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-04-28 09:02:01 +02:00
Rob Bradford
5d1abec565 resources, scripts: Update aarch64 kernel to 5.12
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-04-27 11:31:17 +02:00
Sebastien Boeuf
3d06657f06 resources: Update Dockerfile to Ubuntu Focal Fossa
In order to avoid possible issues with Bionic Beaver (18.04) as it's
getting old, we move the CI container to Focal Fossa (20.04).

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-04-21 14:51:23 +02:00
Rob Bradford
b18ae72d76 Dockerfile: Update for latest stable Rust release
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-03-26 14:25:05 +00:00
Henry Wang
c853ed3fdc resources: Enable GPIO PL061 driver in kernel
This commit enables the kernel config of PL061 GPIO controller.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-03-16 20:27:15 +08:00
Henry Wang
3965a8505b resources: AArch64: Enable PL011 driver in kernel
This commit enables the kernel config of PL011 UART controller.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-03-16 11:53:51 +08:00
Michael Zhao
c3a75dafc5 tests: Enable serial test cases for AArch64
Enabled all "ttyS0" related test cases:
- test_serial_off
- test_serial_tty
- test_serial_file

Enabled mandatory guest kernel driver for "ns16550a" on AArch64.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2021-03-15 20:59:50 +08:00
Rob Bradford
a59b361058 resources: Use Rust 1.49.0 in container
This is the latest stable version.

Fixes: #2092

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-24 17:05:07 +00:00
Rob Bradford
f6d6495b93 resources: Add CONFIG_ACPI_BUTTON to linux-config-x86_64
Thise is needed to enable clean shutdown of the guest via an ACPI power
button.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-13 17:00:39 +00:00
Rob Bradford
852bf0b2ce scripts, resources: aarch64: Update to Linux kernel v5.10 base
This allows the removal of most of the custom patches in particular all
the virtio-fs ones have been integrated.

Fixes: #2070

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-11 12:56:05 +01:00
Rob Bradford
539708e45e scripts, resources: x86_64: Update to Linux kernel v5.10 base
This allows the removal of most of the custom patches in particular all
the virtio-fs ones have been integrated.

Partially fixes: #2070

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-04 22:44:40 +00:00
Michael Zhao
52f60188ca resources: Enable watchdog in guest driver
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-11-02 08:19:07 +00:00
Rob Bradford
7ee63134d2 resources: Update Linux config to enable virtio-watchdog
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-21 16:02:39 +01:00
Sebastien Boeuf
1e5d35d13a resources: Finalize automated generation of the Docker image
By fixing the Dockerfile, we have now finalized the automated generation
of the Docker images for both architectures (amd64 and arm64).

Fixes #953

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-30 10:03:06 +01:00
Sebastien Boeuf
db5d42ad41 resources: Fix Dockerfile to support multi-architecture
In order to support both amd64 and arm64, we rely on the TARGETARCH
variable that is passed from the docker buildx command, based on the
platform used to build the container image.

There is no way to rely directly on $(uname -m) to assign a variable
with the correct x86_64 or aarch64 values we're looking for. Both ENV
and ARG don't evaluate the command, which means they see it as a simple
string. Using RUN is the only way to evaluate a command.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-29 09:29:36 +02:00
Sebastien Boeuf
77e9fcdebd resources: Update Dockerfile to install missing Python packages
In order to build virtiofsd from the latest build system, the Python
package python3-setuptools is required.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-28 09:28:37 +02:00
Rob Bradford
f4ec915c5d resources: Remove unused PPS features from kernel config
In particular this removes the annoying PPS messages that fill up the
dmesg log.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-09-25 14:07:41 +02:00
Sebastien Boeuf
ae44e9c076 resources: Reduce x86_64 kernel configuration to fix warnings
Removing the ISA DMA configurations prevents the kernel from accessing
the port I/O 0x87, which was generating the following warning:

WARN:vmm/src/cpu.rs:378 -- Guest PIO read to unregistered address 0x87

Removing the TELCLOCK configuration prevents the kernel from accessing
the port I/O reserved for the memory manager, which was causing the
following warning:

WARN:vmm/src/memory_manager.rs:289 -- Unexpected offset for accessing
memory manager device: 15

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-25 10:22:28 +01:00
Sebastien Boeuf
87bb041101 resources: x86_64: Enable auto onlining of memory blocks
In order to facilitate the memory hotplug for our users, let's enable
the automatic memory onlining in our guest kernel by activating the
kernel config option 'CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE'.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-16 19:20:04 +02:00
Henry Wang
77ba041362 ci: AArch64: Update custom linux kernel to v5.8-rc4
This commit updates the AArch64 kernel config file and integration
test script to v5.8-rc4, and this update keeps the aarch64 guest
kernel in sync with the x86_64 one.

Fixes: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/1516

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-31 14:06:55 +02:00
Michael Zhao
8e520d2415 resource: Enable PCI controller driver in AArch64 kernel config
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-07-30 09:52:12 +02:00
Sebastien Boeuf
2721a7004f ci: Update guest kernel to v5.8-rc4
Updating the kernel from 5.6-rc4 to 5.8-rc4 allows us to remove the
dependency on both virtio-vsock and virtio-mem patches as they are now
part of the upstream kernel. We're still carrying virtio-iommu and
virtio-fs patches.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-07-13 12:19:24 +02:00
Henry Wang
2ce231370d tests: Preparation of enabling integration tests for AArch64
This commit adds supporting components and code for enabling the
AArch64 integration tests, including:

1. A Linux kernel config file to build kernel on AArch64 machines.

2. Refactoring the `run_integration_test.sh` to architecture
specific scripts for readability.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-07 13:55:39 +01:00
Henry Wang
b70e254700 resources: Dockerfile: Update Dockerfile for AArch64
This update of Dockerfile will add support to the AArch64
integration tests and musl building, including:

1. Installation of the missing `setcap` binary on AArch64.
On AArch64, `setcap` binary should be installed via `libcap2-bin`.
This binary is needed in the integration test.

2. Related support for the cloud-hypervisor binary building by
musl toolchain.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-01 09:51:15 +01:00
Sebastien Boeuf
96a5e22bd6 resources: kernel: Enable 5 levels of page table
Some CPUs might support up to 52 bits of addressable space, that's the
reason why we need the guest kernel to support 5 levels of page table.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-06-09 18:08:09 +01:00
Rob Bradford
c471ae944c Dockerfile: Update to latest Rust toolchain: 1.43.0
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-05-27 19:32:12 +02:00
Michael Zhao
0090ec2dda build: Updated development utilities for AArch64
Updated Dockerfile to work with multiple architectures.
Updated dev_cli.sh to:
1. Build container image before AArch64 image is ready in public.
2. Adjust default feature collection on AArch64.
3. Workaround a build problem with musl on AArch64.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-05-21 11:56:26 +01:00
Samuel Ortiz
33b0e15804 resources: Add musl tools and toolchain to the Dockerfile
And fix the libssl install for building it with musl.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-04-29 17:57:01 +01:00
Damjan Georgievski
5157ba10d1 resources: enable initramfs in kernel config
so that we can use the kernel compiled for the integration tests

Signed-off-by: Damjan Georgievski <gdamjan@gmail.com>
2020-03-26 11:59:03 +01:00
Samuel Ortiz
4b462a54ee Dockerfile: Add cpio and bsdtar to the container image
To be able to build initrd images.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-03-23 19:54:40 +01:00
Samuel Ortiz
6ea85ca86c resources: Dockerfile: Update Rust toolchain
To the latest stable version: 1.42.0

Fixes: #909

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-03-13 12:26:15 +00:00
Sebastien Boeuf
df2570a472 resources: Simplify kernel config filename
Since we only keep one single version of the kernel config file in our
repository, there is no reason to keep the filename complex.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-12 14:07:11 +01:00
Sebastien Boeuf
9ab648b8f7 resources: Enable VIRTIO_MEM support
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-12 14:07:11 +01:00
Sebastien Boeuf
03398532dc ci: Bump to kernel 5.6-rc4
The kernel version is updated from 5.5-rc1 to 5.6-rc4, including the
updated kernel config file.

The kernel branch contains virtio-fs, virtio-iommu and virtio-mem
patches that are not upstream yet. It also contains one fix for
virtio-vsock which will be merged upstream in the next release.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-12 14:07:11 +01:00
Sebastien Boeuf
3f396d87ef resources: Enable ACPI PCI hotplug in the kernel config
The PCI hotplug through ACPI was not enabled by default.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-04 12:06:02 +00:00
Rob Bradford
77ee331be0 resources: Enable KASLR in kernel config
This option improves the security of the guest by randomising the start
address of the kernel in physical memory. We should turn this on so as
to ensure all our functionality such as memory hotplug and kernel
loading works as this is an option used widely in production.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-02-24 15:56:37 +00:00
Samuel Ortiz
db6f894e5e scripts: Add container based development script
The script is a development tool that runs all commands in a dedicated
container. This allows for containerized, isolated and reproducible
builds and CI runs.

The script supports the following command:

* build: Build Cloud Hypervisor binaries (debug and release)
* build-container: Build the container used by the script
* tests: Run unit, cargo and integration tests

$ ./scripts/dev_cli.sh help

Cloud Hypervisor dev_cli.sh
Usage: dev_cli.sh <command> [<command args>]

Available commands:

    build [--debug|--release] [-- [<cargo args>]]
        Build the Cloud Hypervisor binaries.
        --debug               Build the debug binaries. This is the default.
        --release             Build the release binaries.

    tests [--unit|--cargo|--all]
        Run the Cloud Hypervisor tests.
        --unit               Run the unit tests.
        --cargo              Run the cargo tests.
	--integration        Run the integration tests.
        --all                Run all tests.

    build-container [--type]
        Build the Cloud Hypervisor container.
        --dev                Build dev container. This is the default.

    help
        Display this help message.

Fixes: #682
Fixes: #684

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-01-31 11:42:20 +00:00
Sebastien Boeuf
aa4efdd7ec ci: Update kernel related to virtio-iommu
Because of the new set of patches related to virtio-iommu allowing only
for the topology to be described through virtio configuration, this
patch updates the kernel branch and the kernel configuration our CI
relies on.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-01-30 10:37:40 +01:00
Rob Bradford
d5d4053795 README.md: Update kernel instructions to use branch
Update the kernel build instructions to use the configuration and branch
that we test and develop against.

Fixes: #521

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-01-23 12:56:52 +01:00
Rob Bradford
351058ab0f resources: Add memory hotplug support to the kernel configuration
Add the config option for enabling memory hotplug to our recommended
kenrel configuration.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-01-15 20:21:22 +01:00
Sebastien Boeuf
4dbb33a1fa resources: Update kernel config
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-01-08 09:27:07 +01:00
Sebastien Boeuf
eff78f746a resources: Prevent kernel config interactive shell from showing up
Following the recent addition of CONFIG_EXPERT=y, the kernel config now
asks for several options through interactive shell. This is not
convenient when locally running the integration tests script as the
shell asks for input.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-10 06:24:11 +01:00
Qiu Wenbo
ce1bd9c7ff resources: turn on CONFIG_ACPI_REDUCED_HARDWARE_ONLY
We need turn on CONFIG_ACPI_REDUCED_HARDWARE_ONLY to support cpu hotplug
feature.

Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn>
2019-12-09 10:49:17 +01:00
Sebastien Boeuf
cb59f826ad scripts: Use virtio-fs-virtio-iommu branch as custom kernel
Because we want both early support for virtio-fs and virtio-iommu, our
custom kernel is now based on the kernel branch virtio-fs-virtio-iommu.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-10-10 16:07:50 +02:00
Sebastien Boeuf
688ec0eb2d resources: Update kernel config
In order to support nested virtualization and nested device passthrough
from our CI tests, we need some extra kernel configuration options to be
enabled.

CONFIG_KVM and CONFIG_VIRTUALIZATION for nested virtualization.
CONFIG_VFIO for nested device passthrough.
CONFIG_VIRTIO_IOMMU and CONFIG_ACPI_IORT for virtio-iommu support.

With all these new options applied, we can leverage virtio-iommu to
attach some VFIO devices to it and pass them through a second layer of
virtualization.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-10-10 16:07:50 +02:00
Sebastien Boeuf
af9a72eab6 tests: Add virtio-fs tests with dax=on and dax=off
The existing integration tests are extended to support both use cases
where dax=on and dax=off.

In order to support DAX, the kernel configuration needs to be updated to
include CONFIG_FS_DAX and CONFIG_ZONE_DEVICE.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
021e8d9e13 tests: Move to new kernel 5.3-rc3
The last kernel we were using included some manual porting of the
virtio-pmem and virtio-fs patches. By moving to 5.3-rc3, we are now
closer to upstream since the virtio-pmem patches are part of the linux
kernel now. Additionally, this includes the latest patches from
virtio-fs maintainers, which works with the latest version of virtiofsd.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
0fcca3ed74 tests: Add virtio-fs testing
This commit introduces the testing of the --fs option based on the
virtio-fs implementation. This does not simply add a test, but also
updates the integration script by generating a new kernel embedding
the virtio-fs patches and by downloading the virtiofsd daemon.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00
Rob Bradford
445b484881 fixup! resources: Shrink 5.0 kernel config 2019-06-10 16:49:34 +01:00
Rob Bradford
fa0f1c8ab8 resources: Shrink 5.0 kernel config
Remove some of the kernel configuration options that are not necessary
for manual testing and for testing with the CI in order to reduce the
kernel build time.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-06-10 16:49:34 +01:00
Sebastien Boeuf
a9ed8fa499 resources: Add a 5.0 kernel config
This is helpful and actually needed to ensure everyone is testing
from the same kernel reference.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-05-10 16:32:22 +02:00