Commit Graph

5055 Commits

Author SHA1 Message Date
Bo Chen
1a17564ee9 performance-metrics: Add a binary target for performance metrics
This commit creates a new binary target for generating performance metrics,
and also adds a set of performance tests for network throughput/latency,
block I/O performance, and boot time.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
c9c031a425 test_infra: Fix clippy issue on aarch64
error: unused variable: `fw_path`
   --> test_infra/src/lib.rs:837:9
    |
837 |         fw_path: &str,
    |         ^^^^^^^ help: if this is intentional, prefix it with an underscore: `_fw_path`
    |
    = note: `-D unused-variables` implied by `-D warnings`

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
0ab4097606 test_infra: Enable cross-build for "musl" and "aarch64" targets
With enabling the `vendered-openssl` feature, we can now cross-build the
`test_infra` crate for  "musl" and "aarch64" targets. In this way, we
can remove the `test_infra` crate from the "exclude" list, so that this
crate can be checked and processed by "cargo clippy" and "cargo fmt".

More details can be found: https://docs.rs/openssl/latest/openssl/#vendored

As 'musl-gcc' is required, this commit also installs the `musl-tools`
package for our "build" github action on the musl target [1].

[1] https://github.com/actions-rs/toolchain/issues/102

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
f1151cb55e test_infra: Allow to not print commandline when spawning guest
keep it as default to print out commandline when spawning guest in favor
of integration tests for debugging purpose, while allowing to disable
it when needed (e.g. for performance tests).

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
9b0bd71154 test_infra: Add helper function for performing scp
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
1603841f91 Dockerfile: Install iperf3 and ethr for performance tests
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Rob Bradford
9c6e7c4a4b pci: Support DWORD/4-byte writes to the MSI-X control register
The PCI spec does not specify that the access has to be of a specific
size.

Fixes: #3714

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-17 14:13:45 +00:00
Bo Chen
0ee8ead53d ch-remote, main, vhost_user: Fix deprecated APIs for clap v3.1.0
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 01:56:54 +00:00
dependabot[bot]
5fd49927f2 build: bump clap from 3.0.14 to 3.1.0
Bumps [clap](https://github.com/clap-rs/clap) from 3.0.14 to 3.1.0.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.0.14...clap_complete-v3.1.0)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-17 01:56:54 +00:00
dependabot[bot]
80c34bd756 build: bump cc from 1.0.72 to 1.0.73 in /fuzz
Bumps [cc](https://github.com/alexcrichton/cc-rs) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases)
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.72...1.0.73)

---
updated-dependencies:
- dependency-name: cc
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-17 01:13:38 +00:00
dependabot[bot]
76d3d3d505 build: bump clap from 3.0.14 to 3.1.0 in /fuzz
Bumps [clap](https://github.com/clap-rs/clap) from 3.0.14 to 3.1.0.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v3.0.14...clap_complete-v3.1.0)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-17 00:09:38 +00:00
dependabot[bot]
f16fe56e33 build: bump cc from 1.0.72 to 1.0.73
Bumps [cc](https://github.com/alexcrichton/cc-rs) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases)
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.72...1.0.73)

---
updated-dependencies:
- dependency-name: cc
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-17 00:08:47 +00:00
Rob Bradford
d9eff12ba3 pci: Only allow writes to RW bits in MSI-X register
The PCI spec specifies that only the following bits are writable:

16: MSI Enable
20,21,22: Multiple Message Enable
26: Extended Message Data Enable

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-16 22:48:54 +00:00
Sebastien Boeuf
42b5d4a2f7 pci, vmm: Update DeviceNode to store PciBdf instead of u32
By having the DeviceNode storing a PciBdf, we simplify the internal code
as well as allow for custom Serialize/Deserialize implementation for the
PciBdf structure. These custom implementations let us display the PCI
s/b/d/f in a human readable format.

Fixes #3711

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-16 11:57:23 +00:00
dependabot[bot]
200b13517b build: bump vfio-ioctls from 4630612 to f75a77c
Bumps [vfio-ioctls](https://github.com/rust-vmm/vfio) from `4630612` to `f75a77c`.
- [Release notes](https://github.com/rust-vmm/vfio/releases)
- [Commits](4630612f2f...f75a77c1ab)

---
updated-dependencies:
- dependency-name: vfio-ioctls
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-16 10:06:25 +00:00
dependabot[bot]
07d78e6a12 build: bump libc from 0.2.117 to 0.2.118
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.117 to 0.2.118.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.117...0.2.118)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-16 09:28:41 +00:00
dependabot[bot]
1659a0cb6f build: bump libc from 0.2.117 to 0.2.118 in /fuzz
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.117 to 0.2.118.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.117...0.2.118)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-15 23:43:59 +00:00
dependabot[bot]
6b44f0d12a build: bump virtio-queue from f2a46b3 to 15effce in /fuzz
Bumps [virtio-queue](https://github.com/rust-vmm/vm-virtio) from `f2a46b3` to `15effce`.
- [Release notes](https://github.com/rust-vmm/vm-virtio/releases)
- [Commits](f2a46b3f42...15effcedd4)

---
updated-dependencies:
- dependency-name: virtio-queue
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-15 23:35:16 +00:00
Fabiano Fidêncio
5752a2a4fb openapi: Add the 204 response to vm-add-* actions
As we've added support for cold adding devices to a VM that was created
but not already started, we should propagate the `204` response
generated on those cases to the yaml file, so openapi-generator can
produce the correct client code on the go side, to handle both `200` and
`204` successful results.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-15 11:07:26 -08:00
Fabiano Fidêncio
5d2db68f67 vmm: lib: Allow config changes before the VM is booted
Instead of erroring out when trying to change the configuration of the
VM somewhere between the VM was created but not yet booted, let's allow
users to change that without any issue, as long as the VM has already
been created.

Fixes: #3639

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-15 11:07:26 -08:00
Fabiano Fidêncio
b780a916bb vmm: lib: Add unit tests
Let's add very basic unit for the vm_add_$device() functions, so we can
easily expand those when changing its behaviour in the coming commits.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-15 11:07:26 -08:00
Fabiano Fidêncio
16782e8c6d vmm: lib: Do the config validation in the Vmm
Instead of doing the validation of the configuration change as part of
the vm, let's do this in the uper layer, in the Vmm.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-15 11:07:26 -08:00
Fabiano Fidêncio
bd024bffb1 vmm: config: Move add_to_config to config.rs
Let's move add_to_config to config.rs so it can be used from both inside
and outside of the vm.rs file.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-15 11:07:26 -08:00
Rob Bradford
ddef99a0ff README: Use MAINTAINERS.md for security issues
The GitHub security reporting link only works for those who are
repository owners.

Fixes: #3701

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-15 08:35:05 -08:00
dependabot[bot]
1591cae440 build: bump serde_json from 1.0.78 to 1.0.79 in /fuzz
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.78...v1.0.79)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-15 08:51:04 +00:00
dependabot[bot]
ea4ab9a4cc build: bump serde_json from 1.0.78 to 1.0.79
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.78...v1.0.79)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-14 23:38:59 +00:00
Fabiano Fidêncio
55479a64d2 openapi: Expose TDx configuration
TDx support is already present on the project for quite some time, but
the TDx configuration was not yet exposed to the ones using CH via the
OpenAPI auto generated code.

Signed-off-by: Fabiano Fidêncio <fabiano.fidencio@intel.com>
2022-02-14 11:12:12 +01:00
Rob Bradford
d0789871a5 tests: Add test_iommu_segments integration test
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-11 11:20:04 +00:00
Rob Bradford
57184f110a openapi: Add PlatformConfig to OpenAPI spec
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-11 11:20:04 +00:00
Rob Bradford
20b9f95afd vmm: Attach all devices from specified segments to the IOMMU
Since the devices behind the IOMMU cannot be changed at runtime we offer
the ability to place all devices on user chosen segments behind the
IOMMU. This allows the hotplugging of devices behind the IOMMU provided
that they are assigned to a segment that is located behind the iommu.

Fixes: #911

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-11 11:20:04 +00:00
Rob Bradford
6994b33a24 vmm: Add "iommu_segments" to --platform
This provides a list of segments on which all devices will be placed
behind the IOMMU.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-11 11:20:04 +00:00
Sebastien Boeuf
da20fa74c6 tests: Add new integration test for free page reporting
Introducing a new integration test relying on the virtio-balloon ability
to free host pages that have been reported as freed by the guest.

This test checks that after consuming a lot of RAM in the guest, the VMM
process is able to releases the pages reported by the guest. Simply done
by checking the RSS associated with the VMM's process follows the memory
trend in the guest.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-11 12:10:07 +01:00
Sebastien Boeuf
384752647a virtio-devices: balloon: Implement free page reporting
Implement the VIRTIO_BALLOON_F_REPORTING feature, indicating to the
guest it can report set of free pages. A new virtqueue dedicated for
receiving the information about the free pages is created. The VMM
releases the memory by punching holes with fallocate() if the guest
memory is backed by a file, and madvise() the host about the ranges of
memory that shouldn't be needed anymore.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-11 12:10:07 +01:00
Sebastien Boeuf
052f38fa96 vmm: Enable guest to report free pages through virtio-balloon
Adding a new parameter free_page_reporting=on|off to the balloon device
so that we can enable the corresponding feature from virtio-balloon.

Running a VM with a balloon device where this feature is enabled allows
the guest to report pages that are free from guest's perspective. This
information is used by the VMM to release the corresponding pages on the
host.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-11 12:10:07 +01:00
Sebastien Boeuf
414e8e068b virtio-devices: balloon: Factorization and cleanup
Improving the existing code for better readability and in anticipation
for adding an additional virtqueue for the free page reporting feature.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-11 12:10:07 +01:00
Sebastien Boeuf
c325aa3297 Jenkinsfile: Filter out pull requests containing only documentation
No need to run the Jenkins CI on pull request for which exclusively
markdown files have been modified.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-10 14:48:30 +00:00
Sebastien Boeuf
db5d74cb26 Jenkinsfile: Remove unnecessary environment variables
Since AZURE_CONNECTION_STRING is only useful for the Windows build,
let's remove it from other builds where it's not invoked.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-10 14:48:30 +00:00
dependabot[bot]
3982a2af0f build: bump arbitrary from 1.0.3 to 1.1.0 in /fuzz
Bumps [arbitrary](https://github.com/rust-fuzz/arbitrary) from 1.0.3 to 1.1.0.
- [Release notes](https://github.com/rust-fuzz/arbitrary/releases)
- [Changelog](https://github.com/rust-fuzz/arbitrary/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-fuzz/arbitrary/commits)

---
updated-dependencies:
- dependency-name: arbitrary
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-10 00:58:10 +00:00
Sebastien Boeuf
072114d7c5 docs: Add TDShim to the Intel TDX documentation
Adding a bit of documentation and links about the TDShim and how to run
it with Cloud Hypervisor.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-09 13:50:45 +00:00
Rob Bradford
5e19422fcf vmm: config: Fix PCI segment validation error format string
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-09 13:50:36 +00:00
Rob Bradford
26d1a76ad9 vmm: config: Validate balloon size is less than RAM size
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-09 13:50:36 +00:00
Rob Bradford
e864e35c29 virtio-devices: Print error on notification BAR read/write
This should not occur as ioeventfd is used for notification. Such an
error message would have made the discovery of the underlying cause of

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-09 11:45:08 +01:00
dependabot[bot]
8c170570c4 build: bump virtio-queue from 1ec72eb to f2a46b3 in /fuzz
Bumps [virtio-queue](https://github.com/rust-vmm/vm-virtio) from `1ec72eb` to `f2a46b3`.
- [Release notes](https://github.com/rust-vmm/vm-virtio/releases)
- [Commits](1ec72eb067...f2a46b3f42)

---
updated-dependencies:
- dependency-name: virtio-queue
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-08 23:38:09 +00:00
Sebastien Boeuf
10676b74dc vmm: Split VM config and VM state for snapshot/restore
In order to allow for human readable output for the VM configuration, we
pull it out of the snapshot, which becomes effectively the list of
states from the VM. The configuration is stored through a dedicated file
in JSON format (not including any binary output).

Having the ability to read and modify the VM configuration manually
between the snapshot and restore phases makes debugging easier, as well
as empowers users for extending the use cases relying on the
snapshot/restore feature.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-02-08 15:06:49 +00:00
dependabot[bot]
6717ac0cf2 build: bump autocfg from 1.0.1 to 1.1.0 in /fuzz
Bumps [autocfg](https://github.com/cuviper/autocfg) from 1.0.1 to 1.1.0.
- [Release notes](https://github.com/cuviper/autocfg/releases)
- [Commits](https://github.com/cuviper/autocfg/compare/1.0.1...1.1.0)

---
updated-dependencies:
- dependency-name: autocfg
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-08 01:38:49 +00:00
dependabot[bot]
df38ef56be build: bump semver from 1.0.4 to 1.0.5 in /fuzz
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.4...1.0.5)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-08 00:38:02 +00:00
dependabot[bot]
747003d33c build: bump autocfg from 1.0.1 to 1.1.0
Bumps [autocfg](https://github.com/cuviper/autocfg) from 1.0.1 to 1.1.0.
- [Release notes](https://github.com/cuviper/autocfg/releases)
- [Commits](https://github.com/cuviper/autocfg/compare/1.0.1...1.1.0)

---
updated-dependencies:
- dependency-name: autocfg
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-08 00:07:13 +00:00
dependabot[bot]
5ec693aa2a build: bump virtio-queue from 2ca74a6 to 1ec72eb in /fuzz
Bumps [virtio-queue](https://github.com/rust-vmm/vm-virtio) from `2ca74a6` to `1ec72eb`.
- [Release notes](https://github.com/rust-vmm/vm-virtio/releases)
- [Commits](2ca74a6d77...1ec72eb067)

---
updated-dependencies:
- dependency-name: virtio-queue
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-07 23:49:01 +00:00
dependabot[bot]
52e2286916 build: bump semver from 1.0.4 to 1.0.5
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.4 to 1.0.5.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.4...1.0.5)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-07 23:40:23 +00:00
Rob Bradford
507912385a vmm: Ensure that PIO and MMIO exits complete before pausing
As per this kernel documentation:

      For KVM_EXIT_IO, KVM_EXIT_MMIO, KVM_EXIT_OSI, KVM_EXIT_PAPR, KVM_EXIT_XEN,
      KVM_EXIT_EPR, KVM_EXIT_X86_RDMSR and KVM_EXIT_X86_WRMSR the corresponding
      operations are complete (and guest state is consistent) only after userspace
      has re-entered the kernel with KVM_RUN.  The kernel side will first finish
      incomplete operations and then check for pending signals.

      The pending state of the operation is not preserved in state which is
      visible to userspace, thus userspace should ensure that the operation is
      completed before performing a live migration.  Userspace can re-enter the
      guest with an unmasked signal pending or with the immediate_exit field set
      to complete pending operations without allowing any further instructions
      to be executed.

Since we capture the state as part of the pause and override it as part
of the resume we must ensure the state is consistent otherwise we will
lose the results of the MMIO or PIO operation that caused the exit from
which we paused.

Fixes: #3658

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-07 15:26:22 +00:00