Commit Graph

163 Commits

Author SHA1 Message Date
Muminul Islam
c4fb0f36bd scripts: Use hypervisor variable instead of kvm
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-01-28 09:11:02 +00:00
Muminul Islam
9cdcbb5121 script: Modify unit tests to support mshv
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-01-28 09:11:02 +00:00
Muminul Islam
b339aa6b88 scripts: Export devices(/dev/kvm or /dev/mshv) based on hypervisor
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-01-28 09:11:02 +00:00
Rob Bradford
b8342ebeb4 scripts: Don't download container on help invocations
Delay downloading the container until it is being used.

Fixes: #2065

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-26 13:29:05 +01:00
Rob Bradford
c2acea5156 tests: Support filtering integration tests
e.g.

scripts/dev_cli.sh tests --integration -- --test-filter test_watchdog

This used to be supported by passing "$@" but was broken when multiple
hypervisor support was added.

Fixes: #2182

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-25 17:34:10 +00:00
Sebastien Boeuf
3b43551d98 ci: Use device mapper to avoid copying Windows image
The Windows image is quite large (about 20GiB), hence it takes some time
to copy it for every test in order to avoid potential corruption.

One way to mitigate that without compromising on safety between each
test is by using device mapper. By creating a read-only base, we ensure
the image won't be modified by any of the tests, and by creating one
snapshot for each test, we avoid copying the entire image each time.
A dedicated Copy On Write disk image is created to handle any change
that might be performed on the base image, letting the tests behave as
expected.

Fixes #2155

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-01-13 10:48:06 +00:00
Rob Bradford
5881809126 tests: Update integration tests to use new image without pollinate
This image does not have the pollinate service which can sometimes fail
and prevent SSH from starting as it marks itself as a prerequisite. This
service will never fully succeed as it tries to make a network
connection which will fail inside our test VMs.

Fixes: #2113

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-11 16:38:07 +00:00
Rob Bradford
852bf0b2ce scripts, resources: aarch64: Update to Linux kernel v5.10 base
This allows the removal of most of the custom patches in particular all
the virtio-fs ones have been integrated.

Fixes: #2070

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-11 12:56:05 +01:00
Rob Bradford
642a1a610f scripts: Update to Linux fork using latest virtio-iommu patches
The ch-5.10.6 branch is rebased on top of Linux stable and includes the
latest virtio-iommu support patches.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-11 12:56:05 +01:00
Sebastien Boeuf
7889fc9207 scripts: Run test container with the default bridge network
Using --net=host is not necessary for any of the integration tests, so
let's use the default network option called "bridge".

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-01-11 11:18:48 +00:00
Rob Bradford
539708e45e scripts, resources: x86_64: Update to Linux kernel v5.10 base
This allows the removal of most of the custom patches in particular all
the virtio-fs ones have been integrated.

Partially fixes: #2070

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-04 22:44:40 +00:00
Muminul Islam
d79594aa3f scripts: dev_cli: Remove unnecessary spaces
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-12-05 00:01:41 +01:00
Muminul Islam
27b5f8d7e3 scripts: Add an option to accept hypervisor argument
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-12-05 00:01:41 +01:00
Rob Bradford
1908f488c3 tests: Move tests from vhost_user_fs to virtiofsd-rs
Download and build virtiofsd-rs and then use that in the integration
test suite.

Fixes: #2013

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-12-01 11:13:21 +01:00
Muminul Islam
465ef3f7ad scripts: dev_cli: Export volumes for windows test as well
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-11-09 14:58:26 -08:00
Michael Zhao
5566992422 scripts: Fix the failure to fetch latest guest kernel code on AArch64
The integration script failed to obtain the new guest kernel commit when
the code folder had been existing on CI machine.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-11-04 16:00:12 +00:00
Michael Zhao
4606f0f28a tests: Make migration test case X86 only
Set the test case test_snapshot_restore X86 only, instead of excluding
it from test command line.

The command line option was added because we used to support migration
with Virtio-MMIO, but not Virtio-PCI.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-10-30 17:14:12 +00:00
Rob Bradford
d54544046c build: Use stable virtio-fs QEMU branch
Address build failure from activity in the development virtio-fs branch
by using the stable fork.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-29 08:52:25 +00:00
Muminul Islam
2e1866a5db scripts: dev_cli: Add an option to dev_cli to export volumes
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-10-27 09:52:35 +00:00
Wei Liu
2902a96ae6 tests: add Windows guest snapshot test
We also need to restrict the number of test threads to 1 to avoid tests
interfere with each other.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-10-22 18:23:08 +02:00
Sebastien Boeuf
f73a345107 scripts: Remove all references to 'mmio' feature
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-19 14:58:48 +01:00
Wei Liu
57f81d0375 scripts: dev_cli: clarify which integration test is running
This makes it easier to grep. No functional change obviously.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-10-14 14:23:11 +02:00
Rob Bradford
934f992536 scripts: dev_cli: Add support for Windows guest integration tests
Now that we've written Windows integration tests and the associated
script to launch them, this patch enables the support for Windows tests
in dev_cli.sh, so that we can run it in our Cloud Hypervisor container.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-14 11:19:29 +02:00
Rob Bradford
76c04878fa scripts: Add wrapper script to run Windows guest integration tests
Adding a dedicated script to avoid confusion with other Linux
integration tests scripts.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-14 11:19:29 +02:00
Henry Wang
8bf0bac5a7 dev_cli: Fix TARGETARCH inconsistancy
In previous dev_cli.sh, the `uname -m` command will generate
either `x86_64` or `aarch64`, which is inconsistent with the
architectures in the Dockerfile, namely `amd64` and `arm64`.

This will cause some dependancy missing in the docker container
when the docker image is built locally.

This commit fixes this inconsistancy.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-10-13 11:57:01 +02:00
Sebastien Boeuf
67025948dc ci: Simplify test scripts
Since all unit and integration tests are run inside containers because
they are called from dev_cli.sh, they always run as root. That's why
both unit and integration scripts can be simplified as they don't need
to apply specific capabilities and run cargo tests in a dedicated 'kvm'
group.

Fixes #1683

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-07 14:26:13 +02:00
Sebastien Boeuf
532b3063be ci: No need to build Docker image locally for arm64
Now that Docker images are automatically generated for both amd64 and
arm64 architectures, there's no need to generate the arm64 image locally
on the ARM CI during a CI run. The image should be available from
DockerHub instead.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-01 10:04:32 +01:00
Sebastien Boeuf
8a11ef5ba6 ci: Move back to latest virtiofsd version
This commit moves back to the branch "virtio-fs-dev" from virtiofsd, as
we figured the changes needed to use this branch and the requirements
from the new meson build from QEMU.

It updates the container version to ensure the dev_cli.sh script will
rely on the latest container which contains the needed packages.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-30 13:10:23 +01:00
Sebastien Boeuf
827810dbd5 ci: Fix virtiofsd build by staying on older branch
While we figure out the details on how to correctly build virtiofsd from
the latest rebase from the branch "virtio-fs-dev" (which now relies on
QEMU's new build system), let's fix the CI by relying on an older branch
which still relies on the previous build system.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-23 17:06:34 +01:00
Henry Wang
c85c1f0d76 ci: AArch64: enable snapshot/restore integration test case
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-09-23 12:37:25 +01:00
Josh Soref
5c3f4dbe6f ch: Fix various misspelled words
Misspellings were identified by https://github.com/marketplace/actions/check-spelling
* Initial corrections suggested by Google Sheets
* Additional corrections by Google Chrome auto-suggest
* Some manual corrections

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-09-23 08:59:31 +01:00
Sebastien Boeuf
56b0c85578 ci: Introduce SGX integration testing
Extending the Cloud-Hypervisor CI to allow for testing SGX on a
dedicated machine where special image and kernels are ready.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-09 15:33:49 +02:00
Michael Zhao
a4222afd7c scripts: Remove the workaround for "with-serde" in unit tests on AArch64
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-08-24 10:59:08 +02:00
Rob Bradford
4ef702ea76 tests: Use a disk image for test_vfio rather than virtio-fs
This reduces the complexity of the test slightly. The PCI BDFs in the L1
needed changing as the block devices come before the network ones.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-08-12 12:19:34 +02:00
Rob Bradford
5da55f0f2e tests: Remove "--nocapture" from tests
`cargo test` runs every test in its own process and by default will
collect the output from the test and only print it if the test fails.
`--nocapture` instead prints any output from the test immediately and
does not collate it - this leads to interleaving.

`--nocapture` might be useful for local test debugging but is not
something that is helpful, and is instead unhelpful for our CI as a
whole.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-08-11 12:57:18 +02:00
Bo Chen
f65372ea4a dev_cli: Remove the use of undefined '$libc' for the 'shell' command
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-06 08:30:40 +02:00
Michael Zhao
fc2d9c6e31 ci: Remove unused image in sha1sums-aarch64
We no longer download focal-server-cloudimg-arm64.img, checking it
lead to a failure if "work_loads" folder is new.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-08-05 08:15:18 +01:00
Henry Wang
a47da51539 ci: AArch64: Enable basic virtio_vsock test case for AArch64
This commit enables the test case for testing the basic function
of virtio_vsock (i.e. without the hotplug).

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-08-04 13:16:14 +01:00
Henry Wang
5807a91f33 scripts: AArch64: Fix abnormal integration script behaviour
PR#1511 introduced a `flock` command in order to let AArch64 CI
can be executed with multiple executors. However the command
```
(
    echo "try to lock $WORKLOADS_DIR folder and update"
    flock -x 12 && update_workloads
)
```
will introduce an abnormal behavior: If any error happened in
function `updated_workloads`, the sub-shell opened by the pair of
parentheses will be killed instead of the main shell, which is not
right.

This commit fixes this abnormal execution behaviour.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-31 14:08:15 +02:00
Henry Wang
77ba041362 ci: AArch64: Update custom linux kernel to v5.8-rc4
This commit updates the AArch64 kernel config file and integration
test script to v5.8-rc4, and this update keeps the aarch64 guest
kernel in sync with the x86_64 one.

Fixes: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/1516

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-31 14:06:55 +02:00
Michael Zhao
44eccbe1af scripts: Remove the workaround for "with-serde" build error on AArch64
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-07-31 12:35:17 +01:00
Michael Zhao
fec54f71b0 tests: Enable PCI integration test cases on AArch64
Enabled a minimum set of PCI test case.
More cases are to be adapted or debugged.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-07-30 09:52:12 +02:00
Michael Zhao
cf1b5156f4 scripts: Change AArch64 container network type BRIDGE
Changed the container network type to BRIDGE to seprate the networking
of parallel containers.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-07-29 18:32:44 +01:00
Michael Zhao
7e3cbf04de scripts: Improve AArch64 CI for parallel executors
Lock "work_loads" folder when one job is syncing files. If another job
arrives, wait until the lock is released.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-07-29 18:32:44 +01:00
Sebastien Boeuf
8ad0e4d715 ci: Move to latest virtiofsd version
In order to follow the virtio-fs development, we rely on the latest
development branch regarding the virtio-fs daemon. It will be now build
from virtio-fs-dev branch from the virtio-fs maintainers repository.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-07-27 14:03:05 +01:00
Henry Wang
f449aec78e ci: AArch64: Enable AArch64 mmio-related integration test cases
This commit enables some mmio-related integration test cases on
AArch64, including:
* some vhost_user test cases
* virtio-blk test cases
* pmem test cases

Also this commit contains a bug fix in creating virtio-blk device.
Previously, when creating the FDT, the virtio-blk device was
labeled in the reverse order of address allocation.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-20 11:45:47 +01:00
Henry Wang
5a2ff98917 ci: AArch64: Add a build stage for musl toolchain
This commit adds required environment configurations to the
`dev_cli.sh` and a Jenkins stage to enable AArch64 binary
building using musl toolchain.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-15 08:19:15 +02:00
Henry Wang
e81402e76e ci: AArch64: Refactor custom linux kernel building strategy
As the current AArch64 integration test builds kernel every time,
which would take unnecessary time in CI and therefore not ideal.

This commit refactors the AArch64 kernel building strategy to:
1. Keep the Linux kernel source code directory instead of deleting
it everytime after the kernel is built.

2. In integration test script, check if the Linux kernel source
code directory exists. If so use `git fetch && git checkout -f` to
keep the source code always updated, else run `git clone` to get
the source code.

3. Copy config file in and then compile the kernel.

Fixes: https://github.com/cloud-hypervisor/cloud-hypervisor/issues/1444

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-15 08:10:20 +02:00
Sebastien Boeuf
2721a7004f ci: Update guest kernel to v5.8-rc4
Updating the kernel from 5.6-rc4 to 5.8-rc4 allows us to remove the
dependency on both virtio-vsock and virtio-mem patches as they are now
part of the upstream kernel. We're still carrying virtio-iommu and
virtio-fs patches.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-07-13 12:19:24 +02:00
Rob Bradford
f47f221f24 build: Remove obsolete feature differentiated vhost_user_net builds
Now that vhost_user_net crate does not depend on the virtio-devices
crate it is no longer compiled differently based on the mmio or pci
features.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-07-08 16:11:50 +01:00