Commit Graph

6805 Commits

Author SHA1 Message Date
Rob Bradford
22cf8c97e5 build: Release v29.0
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 17:10:35 +00:00
Wei Liu
9e6301be70 Reland: build: drop clap from dev-dependencies
Some crates don't need it at all.

Some crates are using it for a simple functionality which can be
replaced easily.

Signed-off-by: Wei Liu <liuwe@microsoft.com>

This was causing issues with the release build process but we now have a
fix to clean residual build assets.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 16:44:59 +00:00
Rob Bradford
547230bb77 .github: Clean source tree before cross building release assets
This address issues with leaking symbols into the cross build.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 16:44:59 +00:00
Rob Bradford
ef7e177df2 .github: Run release style builds on all PRs
Adjust the release workflow to move the conditional check on the tag
creation into the steps that create the release/upload the assets.

This allows us to ensure we're always in a releaseable state.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 14:38:15 +00:00
Rob Bradford
375e094c77 Revert "build: drop clap from dev-dependencies"
This reverts commit 998fb48ff2.

This is breaking the release build process (cross build for aarch64
musl) so temporarily reverting until we can identify the cause to allow
the release proceeed.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 14:38:15 +00:00
dependabot[bot]
3df82337f1 build: Bump thiserror from 1.0.37 to 1.0.38
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.37 to 1.0.38.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.37...1.0.38)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-13 01:34:59 +00:00
dependabot[bot]
0e273e8b0d build: Bump once_cell from 1.16.0 to 1.17.0 in /fuzz
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.16.0...v1.17.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-13 00:34:13 +00:00
Philipp Schuster
ad6c0ee52b virtio-devices: properly join all threads on Drop
This change is important to do a proper resource cleanup. We decided
to do this repetitive approach as VirtioCommon can't implement Drop
without major changes to the corresponding code. Also, devices such as
Net can't easily use the epoll_threads-abstraction from VirtioCommon as
it has multiple threads with different semantics.

Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
2023-01-12 18:03:33 +00:00
dependabot[bot]
b1f6a59579 build: Bump semver from 1.0.14 to 1.0.16
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.14 to 1.0.16.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.14...1.0.16)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-12 12:54:36 +00:00
Yong He
3494080e2f vmm: add configuration for network offloading features
Add new configuration for offloading features, including
Checksum/TSO/UFO, and set these offloading features as
enabled by default.

Fixes: #4792.

Signed-off-by: Yong He <alexyonghe@tencent.com>
2023-01-12 09:05:45 +00:00
Muminul Islam
06e583c9ab vmm: move kvm feature gate right before the if condition
This change uses the kvm feature gate cleaner way
in the handling of PIO/MMIO exits.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-01-12 09:03:28 +01:00
Muminul Islam
7d8f795430 hypervisor: remove unnecessary derive of HypervisorType
There was an unnecessary change in previous PR #5077.
This is the follow-up clean up patch.

Right now there is no use case of the drive of
Eq and PartialEq.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-01-12 09:03:28 +01:00
dependabot[bot]
f8c6d7fe37 build: Bump micro_http from 4b18a04 to fbef706 in /fuzz
Bumps [micro_http](https://github.com/firecracker-microvm/micro-http) from `4b18a04` to `fbef706`.
- [Release notes](https://github.com/firecracker-microvm/micro-http/releases)
- [Commits](4b18a043e9...fbef706e28)

---
updated-dependencies:
- dependency-name: micro_http
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-11 23:19:57 +00:00
Yong He
0dc122a9a9 virto-device: add latency account for virtio-block
Add new latency counters for virtio-block device, including
minimal latency, maximal latency, and average latency for block
read and write.

The average latency is calculated based on cumulative average.

Signed-off-by: Yong He <alexyonghe@tencent.com>
2023-01-11 17:38:42 +00:00
Yong He
e2ea9e2821 block_util: add timestamp in request
Add timestamp per request, so that we could account
block process latency base request.

Signed-off-by: Yong He <alexyonghe@tencent.com>
2023-01-11 17:38:42 +00:00
Muminul Islam
4e3bc20f2c vmm: Ensure PIO/MMIO exits complete before pausing only for KVM
MSHV does not require to ensure MMIO/PIO exits complete
before pausing. This patch makes sure the above requirement
by checking the hypervisor type run-time.

Fixes #5037

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-01-11 17:15:56 +00:00
Rob Bradford
e661139e1e arch: Print details of host hypervisor status & address space size
e.g. on QEMU on KVM:

cloud-hypervisor: 17.079406ms: <vmm> INFO:arch/src/x86_64/mod.rs:565 -- Running under nested virtualisation. Hypervisor string: KVMKVMKVM

Or under Azure:

cloud-hypervisor: 3.881263ms: <vmm> INFO:arch/src/x86_64/mod.rs:565 -- Running under nested virtualisation. Hypervisor string: Microsoft Hv

Fixes: #5067

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-11 14:38:22 +00:00
dependabot[bot]
c99a5fc7a5 build: Bump io-uring from 0.5.9 to 0.5.11
Bumps [io-uring](https://github.com/tokio-rs/io-uring) from 0.5.9 to 0.5.11.
- [Release notes](https://github.com/tokio-rs/io-uring/releases)
- [Commits](https://github.com/tokio-rs/io-uring/commits)

---
updated-dependencies:
- dependency-name: io-uring
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 23:41:07 +00:00
dependabot[bot]
5b2adb398a build: Bump serde_with_macros from 2.1.0 to 2.2.0 in /fuzz
Bumps [serde_with_macros](https://github.com/jonasbb/serde_with) from 2.1.0 to 2.2.0.
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v2.1.0...v2.2.0)

---
updated-dependencies:
- dependency-name: serde_with_macros
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 23:18:05 +00:00
Rob Bradford
db5582f7d1 block_util: Preserve ordering of sync file backend completions
For the synchronous backends efficiently preserve the order for
completion requests through the use of VecDequeue. Preserving the order
is not required but is beneficial as it matches the existing
optimisation that looks to match completions and requests.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 17:30:25 +00:00
Rob Bradford
ce51755109 block_util: Avoid intermediate completion queue allocation
Rather than aggregate the completion list into an intermediate vector
instead adjust the API to provide one completion item at a time.

With DHAT this shows the number of heap allocations has decreased.

Before:

    dhat: Total:     623,852 bytes in 8,157 blocks

After:

    dhat: Total:     380,444 bytes in 3,469 blocks

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 17:30:25 +00:00
Rob Bradford
67ff0d7819 block_util: Use SmallVec for I/O requests
The number of buffers in the request is usually one so by using SmallVec
the number of heap allocations can be significantly reduced.

DHAT reports:

Before:

dhat: Total:     1,166,412 bytes in 40,383 blocks

After:

dhat: Total:     623,852 bytes in 8,157 blocks

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 17:30:25 +00:00
Rob Bradford
e7a51cde78 block_util: Pass a reference to the slice of iovecs
Rather than passing the vector of iovecs for the I/O to act on pass a
reference to the slice of values inside them. This removes the explicit
container type from the API allowing the use of e.g. SmallVec.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 17:30:25 +00:00
Rob Bradford
9ba06ce5f5 vmm: memory_manager: Deprecate directory backing for memory
This functionality has been obsoleted by our native support for
hugepages and shared memory.

See: #5082

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 15:08:34 +00:00
Sebastien Boeuf
0e7d5d2761 qcow: Fix number of refcount table entries
The number of entries in the refcount table was incorrectly calculated
given there was no need for dividing the number of refblock clusters.
The number of refblock clusters is the number of entries in the refcount
table.

Suggested-by: lv_mz <lv.mengzhao@zte.com.cn>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2023-01-10 12:05:15 +00:00
Rob Bradford
ba9554389b virtio-devices: block: Replace use of HashMap for inflight requests
During analysis of the asynchrous block I/O handling it was observed
that the majority of the time the completion events occur in the same
order as submissions. Further the maximum number of inflight requests
during the boot time is much lower than the size of the queue.

Through the use of a double ended queue (VecDequeue) with a reasonable
pre-allocation capacity we can have O(1) allocation free addition of
items to the list of inflight requests and mostly O(1) matching of
completed requests to submissions.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 10:41:24 +00:00
Rob Bradford
c89b8e061f Revert "vmm: Deprecate MemoryZoneConfig::file"
This reverts commit 9fb0274479.

A user was identified of this functionality.

See: #4837
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 10:31:18 +00:00
Sebastien Boeuf
c2450bd055 tests: Remove unnecessary filename
Since SGX testing doesn't rely on a custom guest image anymore, there's
no need to keep the custom filename around as it's already not in use.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2023-01-10 09:22:59 +00:00
dependabot[bot]
bae9799259 build: Bump mshv-bindings from dda85d3 to fb19bb3
Bumps [mshv-bindings](https://github.com/rust-vmm/mshv) from `dda85d3` to `fb19bb3`.
- [Release notes](https://github.com/rust-vmm/mshv/releases)
- [Commits](dda85d32b9...fb19bb3011)

---
updated-dependencies:
- dependency-name: mshv-bindings
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-09 23:45:35 +00:00
dependabot[bot]
3dd487c2cf build: Bump syn from 1.0.105 to 1.0.107 in /fuzz
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.105 to 1.0.107.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.105...1.0.107)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-09 23:24:11 +00:00
Rob Bradford
eb87538100 block_util: Avoid reallocation of Vec on push
When using DHAT[1] for analysis the amount heap allocations change from:

dhat: Total:     3,186,536 bytes in 41,452 blocks

 to

dhat: Total:     1,059,816 bytes in 34,747 blocks

When running against virtio-block; this still more allocations than
virtio-pmem but a significant improvement without any cost.

[1] https://docs.rs/dhat/latest/dhat/

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-09 10:41:56 -08:00
Philipp Schuster
9e8296b696 log: align log message timestamp
With this change, all log messages will have the same width for the
timestamp. The number of ms is rounded to 6 decimal places.

Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
2023-01-09 16:38:29 +01:00
Philipp Schuster
0dc3282ae0 lifecycle: align "KILL_EVENT received" messages across code-base
To align the logging messages with the rest of the code, this
message should be aligned with another similar occurrence in
epoll_helper.rs

Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
2023-01-09 15:16:48 +00:00
dependabot[bot]
67340c5295 build: Bump clap from 4.0.29 to 4.0.32
Bumps [clap](https://github.com/clap-rs/clap) from 4.0.29 to 4.0.32.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.29...v4.0.32)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-06 23:41:55 +00:00
dependabot[bot]
160ea00d6b build: Bump ryu from 1.0.11 to 1.0.12 in /fuzz
Bumps [ryu](https://github.com/dtolnay/ryu) from 1.0.11 to 1.0.12.
- [Release notes](https://github.com/dtolnay/ryu/releases)
- [Commits](https://github.com/dtolnay/ryu/compare/1.0.11...1.0.12)

---
updated-dependencies:
- dependency-name: ryu
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-06 23:14:20 +00:00
Wei Liu
0389190c64 build: drop the need to import macros from Clap
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 13:06:16 -08:00
Wei Liu
998fb48ff2 build: drop clap from dev-dependencies
Some crates don't need it at all.

Some crates are using it for a simple functionality which can be
replaced easily.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 13:06:16 -08:00
Wei Liu
03d8a3ebcf performance-metrics: add metrics for UDP PPS
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00
Wei Liu
08e4d1f481 test_infra: make measure_virtio_net_throughput & co measure PPS too
While measuring UDP PPS, we saturate the link, so there are packets
lost. We only account for the packets that are not lost.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00
Wei Liu
d50a4cbb6b test_infra: do not output interval values for iperf3
We only care about the end result.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00
dependabot[bot]
28eeb8a492 build: Bump libc from 0.2.138 to 0.2.139
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.138 to 0.2.139.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.138...0.2.139)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-05 23:41:28 +00:00
dependabot[bot]
ec3520d8b2 build: Bump semver from 1.0.14 to 1.0.16 in /fuzz
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.14 to 1.0.16.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.14...1.0.16)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-05 23:14:59 +00:00
Rob Bradford
19661d36af build: Re-enable Jenkins bare metal workers
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-05 17:55:03 +00:00
dependabot[bot]
9747cd1059 build: Bump remain from 0.2.5 to 0.2.6
Bumps [remain](https://github.com/dtolnay/remain) from 0.2.5 to 0.2.6.
- [Release notes](https://github.com/dtolnay/remain/releases)
- [Commits](https://github.com/dtolnay/remain/compare/0.2.5...0.2.6)

---
updated-dependencies:
- dependency-name: remain
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-04 23:40:40 +00:00
dependabot[bot]
0bdc7dcc1c build: Bump proc-macro2 from 1.0.47 to 1.0.49 in /fuzz
Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.47 to 1.0.49.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](https://github.com/dtolnay/proc-macro2/compare/1.0.47...1.0.49)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-04 23:18:36 +00:00
Wei Liu
e16817ea14 performance-metrics: add IOPS tests for FIO
Change fio_ops to fio_control and add a new field to indicate whether it
should report bandwidth or IOPS.

All existing tests are bandwidth tests. Adapt the code accordingly. Add
a set of new tests to report IOPS.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-04 09:47:44 +00:00
dependabot[bot]
3de0a6d401 build: Bump iced-x86 from 1.17.0 to 1.18.0
Bumps [iced-x86](https://github.com/icedland/iced) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/icedland/iced/releases)
- [Commits](https://github.com/icedland/iced/compare/v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: iced-x86
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-03 23:41:28 +00:00
dependabot[bot]
bb9c5e5faf build: Bump serde from 1.0.151 to 1.0.152 in /fuzz
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.151 to 1.0.152.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.151...v1.0.152)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-03 23:12:47 +00:00
Rob Bradford
0036302fc9 tests: Remove support for Ubuntu Bionic
This OS is EOL this year and is well tested by the Rust Hypervisor
Firmware CI so there is no need to duplicate this effort.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-03 14:42:24 +01:00
dependabot[bot]
b4ec59cb0c build: Bump openssl-sys from 0.9.79 to 0.9.80
Bumps [openssl-sys](https://github.com/sfackler/rust-openssl) from 0.9.79 to 0.9.80.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-sys-v0.9.79...openssl-sys-v0.9.80)

---
updated-dependencies:
- dependency-name: openssl-sys
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-03 12:20:54 +00:00