7531 Commits

Author SHA1 Message Date
Rob Bradford
c588138187 build: Use authentication token to avoid GitHub rate limit
The workers share a common public IP address and often GitHub will
reject attempts to access the API due to exceeding the anonymous rate
limit threshold.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 0f71956d6df9c241b205ae3c68e38dfd98f73d68)
2024-03-14 19:52:57 -07:00
Bo Chen
4b72e5a886 build: Allow 'cancel-in-progress' for bare-metal workers
Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 46c9b9693c2b28e64af103d1549278c455b97706)
2024-03-14 19:52:57 -07:00
Rob Bradford
51febbb7fe build: Add SGX, VFIO and rate limit testing to MQ
Run these workflows as part of the merge queue to help improve testing
coverage.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit cdafe5344d09b5f7fd731dec90657ed0c1a5b5f8)
2024-03-14 19:52:57 -07:00
Bo Chen
551d36e502 build: Add a step to fix workspace permissions on bare-metal workers
When a bare-metal worker is canceled, its workspace can be left with
files owned by the root user as a result of running tests from our
container. This patch add a step to fix workspace permissions for such
case before checking out code.

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit f48942ce3f12f507ea5530b926aaf631d914dadd)
2024-03-14 19:52:57 -07:00
Bo Chen
6a5a2ac83d tests: Fix test_snapshot_restore_hotplug_virtiomem on 16 cores VM
It takes longer time to restore a VM on a VM with 16 cores comparing
with ones with 64 cores.

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 071806785187e28d3567d6f2471de07fdad07c76)
2024-03-14 19:52:57 -07:00
Bo Chen
37666f842d build: Run integration tests on smaller VMs
Signed-off-by: Bo Chen <chen.bo@intel.com>
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 7d60ab70e6d1d061f20524b85318cd650f88995f)
2024-03-14 19:52:57 -07:00
Ravi kumar Veeramally
a09d828713 scripts: Update Azure storage location to access images
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
(cherry picked from commit 05ec6190da2154074da186aca0076e7f0769586a)
2024-03-14 19:52:57 -07:00
Rob Bradford
67904a90fc build: Cancel in progress actions on update
If the PR updated cancel outstanding jobs to conserve resources.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 1db30405e13995172bd45386a49a54e7c7a5f621)
2024-03-14 19:52:57 -07:00
Rob Bradford
196e653a50 build: Only run bisectability check on PRs
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 3e355298427bfbc13b2c822a1bb68b3d80783e6f)
2024-03-14 19:52:57 -07:00
Rob Bradford
355148c3d6 build: Only check DCO on PRs
The DCO tool doesn't understand merge_groups but we still need to have a
valid status check to allow the merge group to proceed.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 96cc1ba76c620f0648a64c025111309c967a3f79)
2024-03-14 19:52:57 -07:00
Rob Bradford
1dff2503a6 build: Skip release check on pull requests
This takes a long time and duplicates existing checks on the pull
requests.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 022f375ef86f6b099b68144c9a85dcecc95492ef)
2024-03-14 19:52:57 -07:00
Rob Bradford
f4c85aef89 build: Only run Intel + glibc on PR builds for x86-64 tests
Run all the tests on the merge queue.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 81b95023c47123799413d1220c06fa0c5885cea6)
2024-03-14 19:52:57 -07:00
Rob Bradford
0131a408bf build: Make the Windows Guest Test always pass on PR builds
When running with the merge queue the tests will be fully executed.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit f15ca1aec398c8180fc0602b67f6e10e9abeab0f)
2024-03-14 19:52:57 -07:00
Rob Bradford
bd506500d7 build: Remove unnecessary if event checks from vfio/sgx workflows
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit cb8a728dfbeb10b0d50e8a38c0242576cabb3cc0)
2024-03-14 19:52:57 -07:00
Rob Bradford
80724b1662 build: Use a nicer name for DCO check step
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 80aa91f24c32c532ca1a73e03d04c9b3de872224)
2024-03-14 19:52:57 -07:00
Rob Bradford
1f6b43db49 build: Ensure all required checks run on merge_group
And clean up some of the whitespace formatting so that the "name" and
"on" are grouped away from the "jobs".

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit d9f48505fec88940025e6844cc541e84ffbff6dd)
2024-03-14 19:52:57 -07:00
Bo Chen
878c2275a2 ci: Remove Jenkinsfile
Most of our CI workers are now running form GitHub actions, so we are
ready to disable Jenkins CI workers.

See: #6231

Signed-off-by: Bo Chen <chen.bo@intel.com>
(cherry picked from commit 1d098949b9f3ed7965d3ff0d4fc1fcb348f33506)
2024-03-14 19:52:57 -07:00
Ravi kumar Veeramally
5fc018abdd build: Add GitHub action for Windows guest integration tests
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit ba6bfee4fffa892a4f8e9a31b65a108786a65261)
2024-03-14 19:52:57 -07:00
Ravi kumar Veeramally
690e10eef4 build: Add GitHub action for Rate Limiter integration tests
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
(cherry picked from commit 57fb97e41f7dbeb5c7b588646940d53b2bae2cc8)
2024-03-14 19:52:57 -07:00
Ravi kumar Veeramally
ea12024793 build: Add GitHub action for VFIO integration tests
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
(cherry picked from commit b765acd6081c0d2e3e88f2d2f1a4db38661c3d37)
2024-03-14 19:52:57 -07:00
Ravi kumar Veeramally
e082ed23ed build: Add GitHub action for SGX integration tests
Signed-off-by: Ravi kumar Veeramally <ravikumar.veeramally@intel.com>
(cherry picked from commit 4fb86e99150eb428d0c35e44cd83b020c10bb7a6)
2024-03-14 19:52:57 -07:00
Rob Bradford
d1953633e2 build: Add some timeouts to integration test workflow
Add top-level timeout for the jobs and also more agressive per step
timeouts.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 1fe2771a0ddb05d5e952eb67a18a34d656efe3a7)
2024-03-14 19:52:57 -07:00
Rob Bradford
23f1490667 build: Add libc to matrix for x86-64 tests
To reduce issues caused by flaky tests split the musl and glibc jobs
into separate jobs. This means fewer jobs will need to be restarted for
flaky tests. This will also increase CI throughput since the musl builds
account for ~40% of the total CI time when run together with glibc.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 2e4079becb785e1b948b9cd4ae97ca3ab846a9ef)
2024-03-14 19:52:57 -07:00
Rob Bradford
b312a970ef build: Disable "fail fast" on x86-64 GitHub action
This will help handle flakiness in the builds by requiring the minimum
number of restarts.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit d32de07be7a1fb2e5dea1a09906578107ed9b5df)
2024-03-14 19:52:57 -07:00
Rob Bradford
6e544d0a30 build: Switch GitHub action ARM64 builds to musl
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 6ec83c7d8ee17e7db00a8e0be70484145d7171aa)
2024-03-14 19:52:57 -07:00
Rob Bradford
bc84ac4699 build: Extend x86-64 GitHub action to AMD runner
Use the matrix to add a build runnind on the AMD variant of the garm
runner.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 84a6da5e93f15b1df4e088255f51e2840b15c041)
2024-03-14 19:52:57 -07:00
Rob Bradford
78f0f30751 tests: Remove download of unused bionic image for aarch64
The bionic image was being downloaded and converted but no test uses
this image any longer.

Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 6930370a03f8764b0054925515420fd20f3169df)
2024-03-14 19:52:57 -07:00
Rob Bradford
8b0d43e2fe build: Add GitHub action for ARM64 integration tests
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 89f2a4882ebb6f5416d531e72577399b55aca4d3)
2024-03-14 19:52:57 -07:00
Rob Bradford
61430fb345 build: Add GitHub action for unit/integration testing
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
(cherry picked from commit 307a0166c5c0bd623f26f5701d9ee60f3b6641be)
2024-03-14 19:52:57 -07:00
Bo Chen
5e702dcd5e build: Release v37.0
Signed-off-by: Bo Chen <chen.bo@intel.com>
v37.0
2023-12-14 09:49:25 -08:00
Bo Chen
602d704558 tests: Stabilize 'test_vfio_user' with retries to run host commands
The 'test_vfio_user' is prone to fail when the system is under high
workloads with errors:

```
Error while connecting to /var/tmp/spdk.sock
Is SPDK application running?
Error details: Invalid or non-existing address: '/var/tmp/spdk.sock'
```

This is because SPDK is not fully functional before we request to
create a nvme device using the vfio_user protocol. This patch stabilize
this test with allowing retires to execute host commands.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2023-12-14 07:12:04 -08:00
Bo Chen
38a2808d85 arch: x86_64: Refactor the way to generate e820 RAM maps
This patch defines a new function 'generate_ram_ranges', to generate
usable physical memory ranges for the guest based on the existing guest
memory managed by VMM. This function is also made public, so that it can
be reused, say by the IGVM loader in the future [1].

No functional change.

See: #6020

Signed-off-by: Bo Chen <chen.bo@intel.com>
2023-12-14 07:11:53 -08:00
dependabot[bot]
5e2f218832 build: Bump async-executor from 1.5.1 to 1.8.0
Bumps [async-executor](https://github.com/smol-rs/async-executor) from 1.5.1 to 1.8.0.
- [Release notes](https://github.com/smol-rs/async-executor/releases)
- [Changelog](https://github.com/smol-rs/async-executor/blob/master/CHANGELOG.md)
- [Commits](https://github.com/smol-rs/async-executor/compare/v1.5.1...v1.8.0)

---
updated-dependencies:
- dependency-name: async-executor
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-14 00:46:10 +00:00
dependabot[bot]
2fb8854ff6 build: Bump once_cell from 1.18.0 to 1.19.0 in /fuzz
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.18.0 to 1.19.0.
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.18.0...v1.19.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-13 23:54:27 +00:00
dependabot[bot]
883bb22b65 build: Bump mshv-ioctls from 6901f9c to 0dd4d34
Bumps [mshv-ioctls](https://github.com/rust-vmm/mshv) from `6901f9c` to `0dd4d34`.
- [Commits](6901f9cbd3...0dd4d3452a)

---
updated-dependencies:
- dependency-name: mshv-ioctls
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-13 23:24:43 +00:00
dependabot[bot]
442fe6afcb build: Bump anstyle-query from 1.0.0 to 1.0.2 in /fuzz
Bumps [anstyle-query](https://github.com/rust-cli/anstyle) from 1.0.0 to 1.0.2.
- [Commits](https://github.com/rust-cli/anstyle/compare/anstyle-query-v1.0.0...anstyle-query-v1.0.2)

---
updated-dependencies:
- dependency-name: anstyle-query
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-12 23:44:37 +00:00
Bo Chen
e64b66054e vmm: tdx: Error out early for TD migration
Signed-off-by: Bo Chen <chen.bo@intel.com>
2023-12-12 11:45:40 -08:00
Bo Chen
ceb1be9f50 vmm: Fix a typo from send_migration()
Signed-off-by: Bo Chen <chen.bo@intel.com>
2023-12-12 11:45:40 -08:00
Muminul Islam
7d5ea5ca37 hypervisor: fix few typos and cosmetic issues
This patch adds missing new lines after functions,
fixes few typos in the comments, adds few missing
comments to SNP related functions.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-12-12 14:42:22 +00:00
dependabot[bot]
d5839fe03c build: Bump pin-project from 1.1.2 to 1.1.3
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 1.1.2 to 1.1.3.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.2...v1.1.3)

---
updated-dependencies:
- dependency-name: pin-project
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-12 00:21:51 +00:00
dependabot[bot]
de1b7f5e83 build: Bump ryu from 1.0.15 to 1.0.16 in /fuzz
Bumps [ryu](https://github.com/dtolnay/ryu) from 1.0.15 to 1.0.16.
- [Release notes](https://github.com/dtolnay/ryu/releases)
- [Commits](https://github.com/dtolnay/ryu/compare/1.0.15...1.0.16)

---
updated-dependencies:
- dependency-name: ryu
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-11 23:59:23 +00:00
Jinank Jain
cf8a348238 docs: Add a missing backslash
There is missing backslash in the qemu command to launch windows guest.

Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
2023-12-11 13:30:24 +00:00
Jinank Jain
2197989782 vmm: igvm: Remove redundant copy_from_slice
There is no requirement to call copy_from_slice, since all the member
variables are identical and we can directly assign them value.

Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
2023-12-11 13:30:05 +00:00
Jinank Jain
5d4fe8efb9 github: ci: Move to action/checkout v4
There is a mix of v2, v3 and v4 in the codebase. Let's move to v4
everywhere because v2 seems to be using a deprecated version of nodejs.
This is throwing warnings when the Github action CI is running.

Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
2023-12-11 13:29:11 +00:00
Jinank Jain
638e29bdcc hypervisor: vmm: Fix warnings in Cargo.toml
Currently there are some inconsistencies in Cargo.toml which is causing
the following warnings during the build process:

Error parsing Cargo.toml manifest, fallback to caching entire file:
Invalid TOML document: expected key-value, found comma

Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
2023-12-11 13:29:11 +00:00
dependabot[bot]
d1560b4223 build: Bump futures-core from 0.3.28 to 0.3.29
Bumps [futures-core](https://github.com/rust-lang/futures-rs) from 0.3.28 to 0.3.29.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.28...0.3.29)

---
updated-dependencies:
- dependency-name: futures-core
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-09 00:45:57 +00:00
dependabot[bot]
6d7224dfee build: Bump anstream from 0.6.4 to 0.6.5 in /fuzz
Bumps [anstream](https://github.com/rust-cli/anstyle) from 0.6.4 to 0.6.5.
- [Commits](https://github.com/rust-cli/anstyle/compare/anstream-v0.6.4...anstream-v0.6.5)

---
updated-dependencies:
- dependency-name: anstream
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-08 23:41:32 +00:00
Muminul Islam
f38adfa166 vmm: Pass IGVM file to the loader
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-12-08 09:22:42 -08:00
Muminul Islam
7030b15e63 vmm: Add igvm module and loader module
vmm: Add igvm module and loader module

Add a separate module named igvm to the vmm crate
with definitions to parse and load igvm to the guest memory.

Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-12-08 09:22:42 -08:00
Muminul Islam
ec79820b3f hypervisor: Add api to retrieve CPUID leaf
Add necessary API to retrieve cpuid leaf on MSHV.
This API is used to update cpuid information
during the parsing of the igvm file.

Microsoft hypervisor does not provide common
CpuID like KVM. That's why we need to call this API
during the IGVM parsing.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-12-08 09:22:42 -08:00