Commit Graph

63 Commits

Author SHA1 Message Date
Fabiano Fidêncio
5343e09e7b scripts: dev_cli: Add --features option
Let's officially have a way to pass the features used to build
cloud-hypervisor to the dev_cli.sh script.

This doesn't invalidate the previous commit, as we still don't what the
features_build variable to be quoted, otherwise we face the following
issue:
```
error: Found argument '--features tdx' which wasn't expected, or isn't valid in this context
	Did you mean --features?

USAGE:
    cargo build --all --features <FEATURES>...
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-25 15:28:49 +00:00
Fabiano Fidêncio
7dc9259c6c scripts: dev_cli: Don't quote $features_build
2805e7b1dc quoted enclose variables to
prevent globbing or incorrect splitting.  However, by doing with with
$features_build it broke the capability to call the script as:
```
$ ./scripts/dev_cli.sh build --release --libc musl -- --features tdx

```

Before 2805e7b1dc it simply worked, after,
the result is:
```
docker run --user 1000:1000 --workdir /cloud-hypervisor --rm --volume /dev/kvm --volume /home/ffidenci/go/src/github.com/cloud-hypervisor/cloud-hypervisor:/cloud-hypervisor --env RUSTFLAGS= cloudhypervisor/dev:20220223-0 cargo build --all '' --target-dir /cloud-hypervisor/build/cargo_target --features tdx --release --target x86_64-unknown-linux-musl
error: Found argument '' which wasn't expected, or isn't valid in this context

USAGE:
    cargo build --all

For more information try --help
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-25 15:28:49 +00:00
Bo Chen
4552d07a80 scripts: Support custom arguments to the test binary with dev_cli.sh
The dev container interface script (e.g. 'dev_cli.sh') now supports the
following arguments syntax for running tests:

`tests [--unit|--cargo|--all] [--libc musl|gnu] [-- [<test scripts args>] [-- [<test binary args>]]] `

In this way, we can pass custom arguments to the test binary (either
"cargo test" or "performance-metrics") with our dev container script.
For example:

`$ ./dev_cli.sh tests --metrics -- -- --report-file /tmp/metrics.json --test-filter latency`
`$ ./dev_cli.sh tests --integration -- --test-filter "test_serial"  -- --nocapture --test-threads=1`

Fixes: #3739

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-24 16:34:32 +01:00
Rob Bradford
c35220b1ca scripts: dev_cli: Bump container version to use
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-23 16:30:22 +01:00
Rob Bradford
2861330d09 Dockerfile: Remove classic virtiofsd from container
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
55b8a2182b scripts: dev_cli.sh Support building container before use
If `--local` is provided or if the version is not available then build
the container before use. This allows combining updates to the
Dockerfile with a full CI run.

Drop the "--dev" parameter as we only support one container type for
simplicity.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
b1f61d7286 scipts: dev_cli.sh: Remove "ensure_latest_ctr" from build-container
It makes no sense to check for a container when building it.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
7e22786b2c scripts: dev_cli.sh: Remove unused --unattended
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
eade9d8b87 scripts: dev_cli.sh: Don't surround $exported_volumes
This leads to '' which docker doesn't handle.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
1a5b94ee0e scipts: dev_cli.sh: Reformat script for consistency
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
2805e7b1dc scripts: dev_cli.sh: Ensure correct quoting in bash script
Use quotes to correctly enclose variables for prevent globbing or
incorrect splitting.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
5d55070ceb scripts: Use date tagged docker container in dev_cli.sh
This allows more predictable use of the container.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-18 14:09:41 +00:00
Bo Chen
1025783973 scripts: Remove obsoleted env variable "cflags" and "target_cc"
To run unit tests correctly on musl target, We don't need to provide
specific "CFLAGS" or "TARGET_CC", as long as we use the correct build
target `*-linux-musl` with the "cargo test" command.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
1cf73c83e2 scripts: Generate performance metrics with the dev container
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Muminul Islam
ca4857b5e6 scripts: check hypervisor before running the test
Add two more checks if the hypervisor value is correct.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-09-29 15:02:50 +01:00
Bo Chen
a1a0bc8592 scripts: Add entry for live-migration integration tests
Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-09-02 14:57:31 +01:00
Henry Wang
e1f35628e2 dev_cli: Enable pulling latest dev container on Arm
Currently the latest cloudhypervisor/dev docker container is the
multi-arch image. We can pull the arm image directly.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-08-26 14:45:19 +01:00
Wei Liu
972e96ea68 scripts: honour --volumes in cmd_shell
It is useful to mount custom volumes sometimes.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2021-07-19 09:52:51 +02:00
Sebastien Boeuf
04dc496808 ci: Enable baremetal VFIO integration tests
Relying on a NVIDIA Tesla T4 card present in the SGX machine, this patch
enables baremetal VFIO testing, validated by running several NVIDIA
tools in the guest. The guest image has been prepared to include all the
software needed to run these tests.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-03-16 18:59:46 +00:00
Muminul Islam
c4fb0f36bd scripts: Use hypervisor variable instead of kvm
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-01-28 09:11:02 +00:00
Muminul Islam
9cdcbb5121 script: Modify unit tests to support mshv
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-01-28 09:11:02 +00:00
Muminul Islam
b339aa6b88 scripts: Export devices(/dev/kvm or /dev/mshv) based on hypervisor
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-01-28 09:11:02 +00:00
Rob Bradford
b8342ebeb4 scripts: Don't download container on help invocations
Delay downloading the container until it is being used.

Fixes: #2065

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-01-26 13:29:05 +01:00
Sebastien Boeuf
7889fc9207 scripts: Run test container with the default bridge network
Using --net=host is not necessary for any of the integration tests, so
let's use the default network option called "bridge".

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-01-11 11:18:48 +00:00
Muminul Islam
d79594aa3f scripts: dev_cli: Remove unnecessary spaces
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-12-05 00:01:41 +01:00
Muminul Islam
27b5f8d7e3 scripts: Add an option to accept hypervisor argument
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-12-05 00:01:41 +01:00
Muminul Islam
465ef3f7ad scripts: dev_cli: Export volumes for windows test as well
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-11-09 14:58:26 -08:00
Muminul Islam
2e1866a5db scripts: dev_cli: Add an option to dev_cli to export volumes
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2020-10-27 09:52:35 +00:00
Sebastien Boeuf
f73a345107 scripts: Remove all references to 'mmio' feature
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-19 14:58:48 +01:00
Wei Liu
57f81d0375 scripts: dev_cli: clarify which integration test is running
This makes it easier to grep. No functional change obviously.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-10-14 14:23:11 +02:00
Rob Bradford
934f992536 scripts: dev_cli: Add support for Windows guest integration tests
Now that we've written Windows integration tests and the associated
script to launch them, this patch enables the support for Windows tests
in dev_cli.sh, so that we can run it in our Cloud Hypervisor container.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-10-14 11:19:29 +02:00
Henry Wang
8bf0bac5a7 dev_cli: Fix TARGETARCH inconsistancy
In previous dev_cli.sh, the `uname -m` command will generate
either `x86_64` or `aarch64`, which is inconsistent with the
architectures in the Dockerfile, namely `amd64` and `arm64`.

This will cause some dependancy missing in the docker container
when the docker image is built locally.

This commit fixes this inconsistancy.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-10-13 11:57:01 +02:00
Sebastien Boeuf
532b3063be ci: No need to build Docker image locally for arm64
Now that Docker images are automatically generated for both amd64 and
arm64 architectures, there's no need to generate the arm64 image locally
on the ARM CI during a CI run. The image should be available from
DockerHub instead.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-10-01 10:04:32 +01:00
Sebastien Boeuf
8a11ef5ba6 ci: Move back to latest virtiofsd version
This commit moves back to the branch "virtio-fs-dev" from virtiofsd, as
we figured the changes needed to use this branch and the requirements
from the new meson build from QEMU.

It updates the container version to ensure the dev_cli.sh script will
rely on the latest container which contains the needed packages.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-30 13:10:23 +01:00
Sebastien Boeuf
56b0c85578 ci: Introduce SGX integration testing
Extending the Cloud-Hypervisor CI to allow for testing SGX on a
dedicated machine where special image and kernels are ready.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-09-09 15:33:49 +02:00
Bo Chen
f65372ea4a dev_cli: Remove the use of undefined '$libc' for the 'shell' command
Signed-off-by: Bo Chen <chen.bo@intel.com>
2020-08-06 08:30:40 +02:00
Michael Zhao
44eccbe1af scripts: Remove the workaround for "with-serde" build error on AArch64
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-07-31 12:35:17 +01:00
Michael Zhao
cf1b5156f4 scripts: Change AArch64 container network type BRIDGE
Changed the container network type to BRIDGE to seprate the networking
of parallel containers.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-07-29 18:32:44 +01:00
Henry Wang
5a2ff98917 ci: AArch64: Add a build stage for musl toolchain
This commit adds required environment configurations to the
`dev_cli.sh` and a Jenkins stage to enable AArch64 binary
building using musl toolchain.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-15 08:19:15 +02:00
Wei Liu
5bfac796b3 build: add a default feature KVM
It gets bubbled all the way up from hypervsior crate to top-level
Cargo.toml.

Cloud Hypervisor can't function without KVM at this point, so make it
a default feature.

Fix all scripts that use --no-default-features.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2020-07-08 11:07:15 +01:00
Henry Wang
2ce231370d tests: Preparation of enabling integration tests for AArch64
This commit adds supporting components and code for enabling the
AArch64 integration tests, including:

1. A Linux kernel config file to build kernel on AArch64 machines.

2. Refactoring the `run_integration_test.sh` to architecture
specific scripts for readability.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-07-07 13:55:39 +01:00
Rob Bradford
6f59cdbb55 build: dev_cli: Prepare for splitting binaries by building all binaries
In preparation for splitting the binaries into their own crates start
building all the binaries in the workspace when running the build
command inside the container.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-07-06 10:56:10 +02:00
Samuel Ortiz
5a6b8d6323 dev_cli: Add a shell command
And drop the caller into a privileged root shell.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-06-29 18:17:39 +01:00
Henry Wang
462c58d58b tests: Enable AArch64 Jenkins CI with unit tests for GNU
This commit enables the AArch64 Jenkins CI with build and running
unit tests for GNU toolchain.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2020-06-29 18:00:42 +01:00
Michael Zhao
3f18f93f40 docs: Add a guide for testing on AArch64
The support of AArch64 is in very early stage. The steps in building and
runing on X86 and AArch64 can not align well yet. Adding AArch64 content
to README.md would produce much divergence.
Adding a guide in docs/ folder could be a better way to start now.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-06-11 15:00:17 +01:00
Rob Bradford
dc034eb3b0 scripts: Only use musl for the Rust components
Don't use the musl toolchain for the virtiofsd build as it does not
work.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-06-10 10:16:57 +01:00
Rob Bradford
4c2e6054c0 build: Update to latest version of container
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-05-27 19:32:12 +02:00
Michael Zhao
0090ec2dda build: Updated development utilities for AArch64
Updated Dockerfile to work with multiple architectures.
Updated dev_cli.sh to:
1. Build container image before AArch64 image is ready in public.
2. Adjust default feature collection on AArch64.
3. Workaround a build problem with musl on AArch64.

Signed-off-by: Michael Zhao <michael.zhao@arm.com>
2020-05-21 11:56:26 +01:00
Samuel Ortiz
50134969b9 Jenkins: Run musl unit and integration tests on master branch
And use a bumped up container image for that.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-04-29 17:57:01 +01:00
Samuel Ortiz
ad9374bd68 dev_cli: Add --libc to the build and test commands
We only support musl and gnu, default is gnu.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-04-29 17:57:01 +01:00