Commit Graph

6665 Commits

Author SHA1 Message Date
Wei Liu
a0922930b1 docs: update command line options
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
d1ec65873e vhost_user_net: switch to argh
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
54e9ed9932 vhost_user_blk: switch to argh
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
111225a2a5 main: switch to argh
A few breaking changes:

1. `-vvv` needs to be written as `-v -v -v`.
2. `--disk D1 D2` and others need to be written as `--disk D1 --disk D2`.
3. `--option=value` needs to be written as `--option value`

Change integration tests to adapt to the breaking changes.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
fe49056129 main: split out a few functions
Switching to `argh` requires individual default functions.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
d5558aea2a ch-remote: switch to argh
Since argh does not support `--option=value`, we need to change the
integration test code to become `--option value`.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
1ba995d952 performance-metrics: switch to argh
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
dependabot[bot]
c6c081099f build: Bump proc-macro2 from 1.0.47 to 1.0.49
Bumps [proc-macro2](https://github.com/dtolnay/proc-macro2) from 1.0.47 to 1.0.49.
- [Release notes](https://github.com/dtolnay/proc-macro2/releases)
- [Commits](https://github.com/dtolnay/proc-macro2/compare/1.0.47...1.0.49)

---
updated-dependencies:
- dependency-name: proc-macro2
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-16 09:22:06 +01:00
dependabot[bot]
3a1254a9b2 build: Bump clap from 4.0.32 to 4.1.0 in /fuzz
Bumps [clap](https://github.com/clap-rs/clap) from 4.0.32 to 4.1.0.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.32...clap_complete-v4.1.0)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-16 09:21:57 +01:00
Rob Bradford
6e9172bf6f .github: Re-order release steps to ensure binaries are available
Since we run "cargo clean" before running the aarch64 build we need to
create the release and upload the x86-64 assets before the clean.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-14 12:12:11 +00:00
Rob Bradford
22cf8c97e5 build: Release v29.0
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 17:10:35 +00:00
Wei Liu
9e6301be70 Reland: build: drop clap from dev-dependencies
Some crates don't need it at all.

Some crates are using it for a simple functionality which can be
replaced easily.

Signed-off-by: Wei Liu <liuwe@microsoft.com>

This was causing issues with the release build process but we now have a
fix to clean residual build assets.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 16:44:59 +00:00
Rob Bradford
547230bb77 .github: Clean source tree before cross building release assets
This address issues with leaking symbols into the cross build.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 16:44:59 +00:00
Rob Bradford
ef7e177df2 .github: Run release style builds on all PRs
Adjust the release workflow to move the conditional check on the tag
creation into the steps that create the release/upload the assets.

This allows us to ensure we're always in a releaseable state.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 14:38:15 +00:00
Rob Bradford
375e094c77 Revert "build: drop clap from dev-dependencies"
This reverts commit 998fb48ff2.

This is breaking the release build process (cross build for aarch64
musl) so temporarily reverting until we can identify the cause to allow
the release proceeed.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-13 14:38:15 +00:00
dependabot[bot]
3df82337f1 build: Bump thiserror from 1.0.37 to 1.0.38
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.37 to 1.0.38.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.37...1.0.38)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-13 01:34:59 +00:00
dependabot[bot]
0e273e8b0d build: Bump once_cell from 1.16.0 to 1.17.0 in /fuzz
Bumps [once_cell](https://github.com/matklad/once_cell) from 1.16.0 to 1.17.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.16.0...v1.17.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-13 00:34:13 +00:00
Philipp Schuster
ad6c0ee52b virtio-devices: properly join all threads on Drop
This change is important to do a proper resource cleanup. We decided
to do this repetitive approach as VirtioCommon can't implement Drop
without major changes to the corresponding code. Also, devices such as
Net can't easily use the epoll_threads-abstraction from VirtioCommon as
it has multiple threads with different semantics.

Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
2023-01-12 18:03:33 +00:00
dependabot[bot]
b1f6a59579 build: Bump semver from 1.0.14 to 1.0.16
Bumps [semver](https://github.com/dtolnay/semver) from 1.0.14 to 1.0.16.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.14...1.0.16)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-12 12:54:36 +00:00
Yong He
3494080e2f vmm: add configuration for network offloading features
Add new configuration for offloading features, including
Checksum/TSO/UFO, and set these offloading features as
enabled by default.

Fixes: #4792.

Signed-off-by: Yong He <alexyonghe@tencent.com>
2023-01-12 09:05:45 +00:00
Muminul Islam
06e583c9ab vmm: move kvm feature gate right before the if condition
This change uses the kvm feature gate cleaner way
in the handling of PIO/MMIO exits.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-01-12 09:03:28 +01:00
Muminul Islam
7d8f795430 hypervisor: remove unnecessary derive of HypervisorType
There was an unnecessary change in previous PR #5077.
This is the follow-up clean up patch.

Right now there is no use case of the drive of
Eq and PartialEq.

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-01-12 09:03:28 +01:00
dependabot[bot]
f8c6d7fe37 build: Bump micro_http from 4b18a04 to fbef706 in /fuzz
Bumps [micro_http](https://github.com/firecracker-microvm/micro-http) from `4b18a04` to `fbef706`.
- [Release notes](https://github.com/firecracker-microvm/micro-http/releases)
- [Commits](4b18a043e9...fbef706e28)

---
updated-dependencies:
- dependency-name: micro_http
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-11 23:19:57 +00:00
Yong He
0dc122a9a9 virto-device: add latency account for virtio-block
Add new latency counters for virtio-block device, including
minimal latency, maximal latency, and average latency for block
read and write.

The average latency is calculated based on cumulative average.

Signed-off-by: Yong He <alexyonghe@tencent.com>
2023-01-11 17:38:42 +00:00
Yong He
e2ea9e2821 block_util: add timestamp in request
Add timestamp per request, so that we could account
block process latency base request.

Signed-off-by: Yong He <alexyonghe@tencent.com>
2023-01-11 17:38:42 +00:00
Muminul Islam
4e3bc20f2c vmm: Ensure PIO/MMIO exits complete before pausing only for KVM
MSHV does not require to ensure MMIO/PIO exits complete
before pausing. This patch makes sure the above requirement
by checking the hypervisor type run-time.

Fixes #5037

Signed-off-by: Muminul Islam <muislam@microsoft.com>
2023-01-11 17:15:56 +00:00
Rob Bradford
e661139e1e arch: Print details of host hypervisor status & address space size
e.g. on QEMU on KVM:

cloud-hypervisor: 17.079406ms: <vmm> INFO:arch/src/x86_64/mod.rs:565 -- Running under nested virtualisation. Hypervisor string: KVMKVMKVM

Or under Azure:

cloud-hypervisor: 3.881263ms: <vmm> INFO:arch/src/x86_64/mod.rs:565 -- Running under nested virtualisation. Hypervisor string: Microsoft Hv

Fixes: #5067

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-11 14:38:22 +00:00
dependabot[bot]
c99a5fc7a5 build: Bump io-uring from 0.5.9 to 0.5.11
Bumps [io-uring](https://github.com/tokio-rs/io-uring) from 0.5.9 to 0.5.11.
- [Release notes](https://github.com/tokio-rs/io-uring/releases)
- [Commits](https://github.com/tokio-rs/io-uring/commits)

---
updated-dependencies:
- dependency-name: io-uring
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 23:41:07 +00:00
dependabot[bot]
5b2adb398a build: Bump serde_with_macros from 2.1.0 to 2.2.0 in /fuzz
Bumps [serde_with_macros](https://github.com/jonasbb/serde_with) from 2.1.0 to 2.2.0.
- [Release notes](https://github.com/jonasbb/serde_with/releases)
- [Commits](https://github.com/jonasbb/serde_with/compare/v2.1.0...v2.2.0)

---
updated-dependencies:
- dependency-name: serde_with_macros
  dependency-type: indirect
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-10 23:18:05 +00:00
Rob Bradford
db5582f7d1 block_util: Preserve ordering of sync file backend completions
For the synchronous backends efficiently preserve the order for
completion requests through the use of VecDequeue. Preserving the order
is not required but is beneficial as it matches the existing
optimisation that looks to match completions and requests.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 17:30:25 +00:00
Rob Bradford
ce51755109 block_util: Avoid intermediate completion queue allocation
Rather than aggregate the completion list into an intermediate vector
instead adjust the API to provide one completion item at a time.

With DHAT this shows the number of heap allocations has decreased.

Before:

    dhat: Total:     623,852 bytes in 8,157 blocks

After:

    dhat: Total:     380,444 bytes in 3,469 blocks

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 17:30:25 +00:00
Rob Bradford
67ff0d7819 block_util: Use SmallVec for I/O requests
The number of buffers in the request is usually one so by using SmallVec
the number of heap allocations can be significantly reduced.

DHAT reports:

Before:

dhat: Total:     1,166,412 bytes in 40,383 blocks

After:

dhat: Total:     623,852 bytes in 8,157 blocks

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 17:30:25 +00:00
Rob Bradford
e7a51cde78 block_util: Pass a reference to the slice of iovecs
Rather than passing the vector of iovecs for the I/O to act on pass a
reference to the slice of values inside them. This removes the explicit
container type from the API allowing the use of e.g. SmallVec.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 17:30:25 +00:00
Rob Bradford
9ba06ce5f5 vmm: memory_manager: Deprecate directory backing for memory
This functionality has been obsoleted by our native support for
hugepages and shared memory.

See: #5082

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 15:08:34 +00:00
Sebastien Boeuf
0e7d5d2761 qcow: Fix number of refcount table entries
The number of entries in the refcount table was incorrectly calculated
given there was no need for dividing the number of refblock clusters.
The number of refblock clusters is the number of entries in the refcount
table.

Suggested-by: lv_mz <lv.mengzhao@zte.com.cn>
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2023-01-10 12:05:15 +00:00
Rob Bradford
ba9554389b virtio-devices: block: Replace use of HashMap for inflight requests
During analysis of the asynchrous block I/O handling it was observed
that the majority of the time the completion events occur in the same
order as submissions. Further the maximum number of inflight requests
during the boot time is much lower than the size of the queue.

Through the use of a double ended queue (VecDequeue) with a reasonable
pre-allocation capacity we can have O(1) allocation free addition of
items to the list of inflight requests and mostly O(1) matching of
completed requests to submissions.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 10:41:24 +00:00
Rob Bradford
c89b8e061f Revert "vmm: Deprecate MemoryZoneConfig::file"
This reverts commit 9fb0274479.

A user was identified of this functionality.

See: #4837
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-10 10:31:18 +00:00
Sebastien Boeuf
c2450bd055 tests: Remove unnecessary filename
Since SGX testing doesn't rely on a custom guest image anymore, there's
no need to keep the custom filename around as it's already not in use.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2023-01-10 09:22:59 +00:00
dependabot[bot]
bae9799259 build: Bump mshv-bindings from dda85d3 to fb19bb3
Bumps [mshv-bindings](https://github.com/rust-vmm/mshv) from `dda85d3` to `fb19bb3`.
- [Release notes](https://github.com/rust-vmm/mshv/releases)
- [Commits](dda85d32b9...fb19bb3011)

---
updated-dependencies:
- dependency-name: mshv-bindings
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-09 23:45:35 +00:00
dependabot[bot]
3dd487c2cf build: Bump syn from 1.0.105 to 1.0.107 in /fuzz
Bumps [syn](https://github.com/dtolnay/syn) from 1.0.105 to 1.0.107.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/1.0.105...1.0.107)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-09 23:24:11 +00:00
Rob Bradford
eb87538100 block_util: Avoid reallocation of Vec on push
When using DHAT[1] for analysis the amount heap allocations change from:

dhat: Total:     3,186,536 bytes in 41,452 blocks

 to

dhat: Total:     1,059,816 bytes in 34,747 blocks

When running against virtio-block; this still more allocations than
virtio-pmem but a significant improvement without any cost.

[1] https://docs.rs/dhat/latest/dhat/

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2023-01-09 10:41:56 -08:00
Philipp Schuster
9e8296b696 log: align log message timestamp
With this change, all log messages will have the same width for the
timestamp. The number of ms is rounded to 6 decimal places.

Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
2023-01-09 16:38:29 +01:00
Philipp Schuster
0dc3282ae0 lifecycle: align "KILL_EVENT received" messages across code-base
To align the logging messages with the rest of the code, this
message should be aligned with another similar occurrence in
epoll_helper.rs

Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
2023-01-09 15:16:48 +00:00
dependabot[bot]
67340c5295 build: Bump clap from 4.0.29 to 4.0.32
Bumps [clap](https://github.com/clap-rs/clap) from 4.0.29 to 4.0.32.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.29...v4.0.32)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-06 23:41:55 +00:00
dependabot[bot]
160ea00d6b build: Bump ryu from 1.0.11 to 1.0.12 in /fuzz
Bumps [ryu](https://github.com/dtolnay/ryu) from 1.0.11 to 1.0.12.
- [Release notes](https://github.com/dtolnay/ryu/releases)
- [Commits](https://github.com/dtolnay/ryu/compare/1.0.11...1.0.12)

---
updated-dependencies:
- dependency-name: ryu
  dependency-type: indirect
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-01-06 23:14:20 +00:00
Wei Liu
0389190c64 build: drop the need to import macros from Clap
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 13:06:16 -08:00
Wei Liu
998fb48ff2 build: drop clap from dev-dependencies
Some crates don't need it at all.

Some crates are using it for a simple functionality which can be
replaced easily.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 13:06:16 -08:00
Wei Liu
03d8a3ebcf performance-metrics: add metrics for UDP PPS
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00
Wei Liu
08e4d1f481 test_infra: make measure_virtio_net_throughput & co measure PPS too
While measuring UDP PPS, we saturate the link, so there are packets
lost. We only account for the packets that are not lost.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00
Wei Liu
d50a4cbb6b test_infra: do not output interval values for iperf3
We only care about the end result.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00