Commit Graph

93 Commits

Author SHA1 Message Date
Rob Bradford
4817578ce9 build: Bump version used into container for clap crate
See: #5925

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
2023-11-08 08:13:45 -08:00
Rob Bradford
99a2551046 scripts: Propagate AUTH_DOWNLOAD_TOKEN into container
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
2023-11-07 08:44:56 -08:00
Wei Liu
8ba5682e3b scripts: propagate RUSTFLAGS and TARGET_CC to test scripts
There is no need to set them in the test scripts while the main script
already has them.

The consolidates how things are done.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-10-20 10:35:13 -07:00
Wei Liu
c7e51e51e5 scripts: pass BUILD_TARGET directly to test scripts
There is no need to reconstruct it from within the scripts since the
main script already constructed it once.

Drop the previously useless setting of BUILD_TARGET from various
scripts. The value was always overwritten at a later point.

No functional change intended.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-10-20 10:35:13 -07:00
Wei Liu
9d42f48f49 scripts: dev_cli.sh: respect preset RUSTFLAGS
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-10-20 10:35:13 -07:00
dom.song
7f47a030e7 tests: Update spdk code to support Ampere's new CPU
When performing integration testing on the Ampere One server, an error occurred when compiling spdk and not recognizing the CPU ID.
The latest spdk already contains fixes.

Signed-off-by: dom.song <dom.song@amperecomputing.com>
2023-10-12 11:25:38 -07:00
Anatol Belski
376b676910 Dockerfile: Build SPDK for skylake
As the container image can be used on both Intel and AMD,
ensure the SPDK binaries are compatible. This implies -march=skylake
passed to the underlaying toolchain.

Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
2023-08-04 13:50:34 +01:00
Ruslan Mstoi
70cfd1be67 scripts: dev_cli: add possibly to use different container registry
Currently if container registry is inaccessible the image will be built
locally and that takes time. This patch adds support to use mirror
registry. To use a different registry CTR_IMAGE environment variable
must be set. For example:

CTR_IMAGE="registry/cloud-hypervisor" scripts/dev_cli.sh

or

export CTR_IMAGE="registry/cloud-hypervisor"
scripts/dev_cli.sh

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2023-08-04 08:12:22 +08:00
Ruslan Mstoi
7d0aa1fd02 scripts: dev_cli: Bump container version
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2023-06-20 12:39:18 -07:00
Omer Faruk Bayram
a7aecb5eee tests: refactor test_api* to support the D-Bus API and add a new test
Implemented a `TargetApi` enum to make the process of implementing
tests for the D-Bus and HTTP API more convenient.

Refactored `test_api_{create_boot, shutdown, pause_resume, delete}` tests
with the `TargetApi` enum to also implement tests for the D-Bus API.

Added a new test named `test_api_dbus_and_http_interleaved` that uses
both the HTTP and D-Bus API at the same time.

Modified integration test scripts to enable the `dbus_api` feature when
compiling and start a dbus-session when integration tests are run.

Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
2023-06-06 10:18:26 -07:00
smit-gardhariya
b56da6ee24 Export RUST_BACKTRACE when it is not set explicitly
Currently we are overwriting the RUST_BACKTRACE if set
explicitly by user while running performance metrics tests
using dev_cli.sh

This change will allow user to set the RUST_BACKTRACE
while running performance metrics tests with dev_cli.sh
which invokes run_metrics.sh to run the performance binary.
We will set RUST_BACKTRACE to 1 if not set explicitly.

Signed-off-by: smit-gardhariya <gardhariya.smit@gmail.com>
2023-04-10 14:08:21 -07:00
Omer Faruk Bayram
06a9c3b839 scripts: dev_cli: mention the --local flag in the help string
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
2023-03-28 12:37:35 +01:00
Rob Bradford
a062401be6 scripts: dev_cli: Bump container version
Unfortunately the build had a transient error and so needed to be
restarted which generated a build with a different date tag.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
2023-03-16 08:36:25 +00:00
Rob Bradford
ca96ff601b build: Migrate from DockerHub to ghcr
It is necessary to transition away from DockerHub as the small
organisation plan in being EOLed.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
2023-03-15 22:41:23 +00:00
Bo Chen
85962099c7 build: Update development container and release action to Rust 1.67
This latest stable Rust toolchain can help reduce the final binary size
around ~300k when comparing with building with Rust 1.62. Specifically,
the binary size now is 3.3M (previously 3.6M) with default feature, and
4.0M (previously 4.2M) with "kvm+mshv" feature (e.g. our release action
build) .

Signed-off-by: Bo Chen <chen.bo@intel.com>
2023-03-01 15:16:23 +00:00
Ruslan Mstoi
2e94a86b31 scripts: dev_cli.sh: print help if command unspecified
To improve user friendliness, print help text when no command is given.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2023-02-22 11:29:54 -08:00
Muminul Islam
e436b382cc scripts: check if /dev/{mshv,kvm} exists before test run
Right now integration test fails during the test run if
/dev/mshv or /dev/kvm does not exist. We should not
progress and exit early if not present.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-01-26 07:58:13 +00:00
Praveen K Paladugu
a4ef6e57a7 ci: install swtpm in Docker container
Add steps to build and install swtpm and its dependencies in
ci docker container.

Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
2023-01-23 10:10:19 -08:00
Rob Bradford
5034bea8c4 Docker: Update Rust toolchain in dev container
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-16 16:43:49 +00:00
Bo Chen
ef1983ee60 scripts: Add script to run rate limiter integration tests
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-14 15:01:43 +01:00
Anatol Belski
944d09208e ci: Initial Windows image integration
This enables the Windows test module. One basic test is enabled,
while all others are disabled yet for aarch64. Jenkins file is
extended with the corresponding step for aarch64.

installAzureCli() is parametrized.

It seems that transferring a 30GB image would take >= 15 minutes. An
optimization here is having a gzip'ed image to 10GB which would unpack
in 3 minutes. Expect to be quicker than transferring an uncompressed
image while on another network.

Signed-off-by: Anatol Belski <ab@php.net>
2022-08-12 15:04:06 +01:00
Rob Bradford
5dfc49c9fa build: Update development container to Rust 1.62
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-07-05 11:50:46 +01:00
Rob Bradford
3b10699743 scripts: Use correct date for container image
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-05-25 09:38:20 +01:00
Rob Bradford
ffd1411e7c build: Bump minimum Rust version to 1.60
Some dependencies (e.g. pnet 0.30.0) require it.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-05-24 16:02:08 +01:00
Vincent Batts
88ed85247e scripts: dev_cli.sh: add option to specify container runtime
For example:

```shell
./scripts/dev_cli.sh build --release --libc musl --runtime "sudo nerdctl"
```

works. And presumably podman as well.

Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2022-04-20 17:57:16 +01:00
Bo Chen
044b57c927 scripts: Bump container version and SPDK version for aarch64
In this way, CI jobs on both `x86_64` and `aarch64` will start to use
the latest version of SPDK.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-04-05 13:04:12 +01:00
Rob Bradford
fa7487629f scripts: Fix feature build for aarch64 and mshv builds
These were erroneously skipping features for the unit tests and the
"build" target for dev_cli.sh

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-28 09:18:29 -07:00
Rob Bradford
6eb47bdb03 scripts: Remove unused "cargo" testing script
These are never run by the CI and is inconsistent with the way we build
test which is specified inside the .github workflows.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-28 09:18:29 -07:00
Rob Bradford
7936bfe680 Dockerfile, scripts: Update container Rust to latest stable
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-03 12:35:41 +00:00
Sebastien Boeuf
b0324f8557 scripts: Fix musl build on aarch64
Adding the missing TARGET_CC environment variable to get the build to
complete correctly.

Fixes #3776

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-03-01 13:29:40 +08:00
Fabiano Fidêncio
5343e09e7b scripts: dev_cli: Add --features option
Let's officially have a way to pass the features used to build
cloud-hypervisor to the dev_cli.sh script.

This doesn't invalidate the previous commit, as we still don't what the
features_build variable to be quoted, otherwise we face the following
issue:
```
error: Found argument '--features tdx' which wasn't expected, or isn't valid in this context
	Did you mean --features?

USAGE:
    cargo build --all --features <FEATURES>...
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-25 15:28:49 +00:00
Fabiano Fidêncio
7dc9259c6c scripts: dev_cli: Don't quote $features_build
2805e7b1dc quoted enclose variables to
prevent globbing or incorrect splitting.  However, by doing with with
$features_build it broke the capability to call the script as:
```
$ ./scripts/dev_cli.sh build --release --libc musl -- --features tdx

```

Before 2805e7b1dc it simply worked, after,
the result is:
```
docker run --user 1000:1000 --workdir /cloud-hypervisor --rm --volume /dev/kvm --volume /home/ffidenci/go/src/github.com/cloud-hypervisor/cloud-hypervisor:/cloud-hypervisor --env RUSTFLAGS= cloudhypervisor/dev:20220223-0 cargo build --all '' --target-dir /cloud-hypervisor/build/cargo_target --features tdx --release --target x86_64-unknown-linux-musl
error: Found argument '' which wasn't expected, or isn't valid in this context

USAGE:
    cargo build --all

For more information try --help
```

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-25 15:28:49 +00:00
Bo Chen
4552d07a80 scripts: Support custom arguments to the test binary with dev_cli.sh
The dev container interface script (e.g. 'dev_cli.sh') now supports the
following arguments syntax for running tests:

`tests [--unit|--cargo|--all] [--libc musl|gnu] [-- [<test scripts args>] [-- [<test binary args>]]] `

In this way, we can pass custom arguments to the test binary (either
"cargo test" or "performance-metrics") with our dev container script.
For example:

`$ ./dev_cli.sh tests --metrics -- -- --report-file /tmp/metrics.json --test-filter latency`
`$ ./dev_cli.sh tests --integration -- --test-filter "test_serial"  -- --nocapture --test-threads=1`

Fixes: #3739

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-24 16:34:32 +01:00
Rob Bradford
c35220b1ca scripts: dev_cli: Bump container version to use
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-23 16:30:22 +01:00
Rob Bradford
2861330d09 Dockerfile: Remove classic virtiofsd from container
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
55b8a2182b scripts: dev_cli.sh Support building container before use
If `--local` is provided or if the version is not available then build
the container before use. This allows combining updates to the
Dockerfile with a full CI run.

Drop the "--dev" parameter as we only support one container type for
simplicity.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
b1f61d7286 scipts: dev_cli.sh: Remove "ensure_latest_ctr" from build-container
It makes no sense to check for a container when building it.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
7e22786b2c scripts: dev_cli.sh: Remove unused --unattended
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
eade9d8b87 scripts: dev_cli.sh: Don't surround $exported_volumes
This leads to '' which docker doesn't handle.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
1a5b94ee0e scipts: dev_cli.sh: Reformat script for consistency
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
2805e7b1dc scripts: dev_cli.sh: Ensure correct quoting in bash script
Use quotes to correctly enclose variables for prevent globbing or
incorrect splitting.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-21 16:05:53 +00:00
Rob Bradford
5d55070ceb scripts: Use date tagged docker container in dev_cli.sh
This allows more predictable use of the container.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-18 14:09:41 +00:00
Bo Chen
1025783973 scripts: Remove obsoleted env variable "cflags" and "target_cc"
To run unit tests correctly on musl target, We don't need to provide
specific "CFLAGS" or "TARGET_CC", as long as we use the correct build
target `*-linux-musl` with the "cargo test" command.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
1cf73c83e2 scripts: Generate performance metrics with the dev container
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Muminul Islam
ca4857b5e6 scripts: check hypervisor before running the test
Add two more checks if the hypervisor value is correct.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-09-29 15:02:50 +01:00
Bo Chen
a1a0bc8592 scripts: Add entry for live-migration integration tests
Signed-off-by: Bo Chen <chen.bo@intel.com>
2021-09-02 14:57:31 +01:00
Henry Wang
e1f35628e2 dev_cli: Enable pulling latest dev container on Arm
Currently the latest cloudhypervisor/dev docker container is the
multi-arch image. We can pull the arm image directly.

Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2021-08-26 14:45:19 +01:00
Wei Liu
972e96ea68 scripts: honour --volumes in cmd_shell
It is useful to mount custom volumes sometimes.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2021-07-19 09:52:51 +02:00
Sebastien Boeuf
04dc496808 ci: Enable baremetal VFIO integration tests
Relying on a NVIDIA Tesla T4 card present in the SGX machine, this patch
enables baremetal VFIO testing, validated by running several NVIDIA
tools in the guest. The guest image has been prepared to include all the
software needed to run these tests.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2021-03-16 18:59:46 +00:00
Muminul Islam
c4fb0f36bd scripts: Use hypervisor variable instead of kvm
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2021-01-28 09:11:02 +00:00