Commit Graph

59 Commits

Author SHA1 Message Date
Wei Liu
11ef495e6b performance-metrics: share build.rs from project root
No need to duplicate the same content in two places.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-30 21:10:02 +00:00
Bo Chen
574576c8e9 misc: Automatically fix cargo clippy issues added in 1.68 (beta)
Signed-off-by: Bo Chen <chen.bo@intel.com>
2023-01-26 08:58:37 -08:00
Wei Liu
111225a2a5 main: switch to argh
A few breaking changes:

1. `-vvv` needs to be written as `-v -v -v`.
2. `--disk D1 D2` and others need to be written as `--disk D1 --disk D2`.
3. `--option=value` needs to be written as `--option value`

Change integration tests to adapt to the breaking changes.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
1ba995d952 performance-metrics: switch to argh
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-16 16:39:03 +00:00
Wei Liu
0389190c64 build: drop the need to import macros from Clap
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 13:06:16 -08:00
Wei Liu
03d8a3ebcf performance-metrics: add metrics for UDP PPS
Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00
Wei Liu
08e4d1f481 test_infra: make measure_virtio_net_throughput & co measure PPS too
While measuring UDP PPS, we saturate the link, so there are packets
lost. We only account for the packets that are not lost.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-06 11:03:08 +00:00
Wei Liu
e16817ea14 performance-metrics: add IOPS tests for FIO
Change fio_ops to fio_control and add a new field to indicate whether it
should report bandwidth or IOPS.

All existing tests are bandwidth tests. Adapt the code accordingly. Add
a set of new tests to report IOPS.

Signed-off-by: Wei Liu <liuwe@microsoft.com>
2023-01-04 09:47:44 +00:00
Rob Bradford
5e52729453 misc: Automatically fix cargo clippy issues added in 1.65 (stable)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-12-14 14:27:19 +00:00
Sebastien Boeuf
89677c3181 build: Bump clap from 3.2.22 to 4.0.9
Bumps [clap](https://github.com/clap-rs/clap) from 3.2.22 to 4.0.9.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](clap-rs/clap@v3.2.22...v4.0.9)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Moving to the major version 4 introduced some breaking changes which had
to be handled manually.

Fixes #4709

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-10-05 12:59:14 +01:00
Rob Bradford
f32487f8e8 misc: Automatic beta clippy fixes
e.g. cargo clippy --all --tests --all-targets --fix --features=..

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-09-20 10:59:48 +01:00
Bo Chen
d8a6725995 performance-metrics: Move some helper functions to test_infra
The helper functions for measuring and parsing the performance of
virtio-net and virtio-block devices are moved to the `test_infra` crate
so that they be reused for integration tests of rate limiter.

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-09-14 15:01:43 +01:00
Sebastien Boeuf
474125db12 performance-metrics: Improve reliability of virtio_net_latency_us
The test virtio_net_latency_us recently failed because of the ethr tool
couldn't complete the latency measurement with the following error
message:

    Error dialing the latency connection: dial tcp4
    :0-\u003e172.19.0.2:8888: connect: connection refused

The "connection refused" error comes from the Golang Dial function which
couldn't complete as expected because there was no server listening on
the specified port at the specified destination.

In other words, the ethr server wasn't ready when the ethr client tried
to connect to it.

The easy way to avoid this problem is simply by increasing the sleep
time between the command spawning the ethr server in the guest and the
command running the ethr client from the host.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-08-04 17:52:29 +02:00
Rob Bradford
cac42301f8 performance-metrics: Produce some error messages if git commands fail
It is reasonable for these to fail as it the tool could be run outside
of a git repository but by not giving any error message we cannot see
issues when we expect the report to have the git details.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-06-10 17:20:13 +01:00
Rob Bradford
025447e8cc performance-metrics: Add ability to override test iterations
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-05-23 11:14:53 +01:00
Rob Bradford
d582121143 performance-metrics: Reduce verbosity for performance tests
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-05-23 11:14:53 +01:00
Maksym Pavlenko
3a0429c998 cargo: Clean up serde dependencies
There is no need to include serde_derive separately,
as it can be specified as serde feature instead.

Signed-off-by: Maksym Pavlenko <pavlenko.maksym@gmail.com>
2022-05-18 08:21:19 +02:00
Rob Bradford
3c0817a1b7 performance-metrics: Add boot time metrics using multiple vCPUs
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-05-10 13:10:11 +01:00
Henry Wang
8dda052c7d performance-metrics: disable net latency test on AArch64
As ethr is not stable on AArch64, disable it until we find other tool.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2022-03-23 18:25:48 +08:00
Jianyong Wu
1b494fb061 performance-metrics: enable fio test on AArch64
fio test depends on hot-add disk which needs uefi on AArch64, that
will lead to lots of performance issues. So, it's better to use
cold-plug disk.

Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
Signed-off-by: Henry Wang <Henry.Wang@arm.com>
2022-03-23 18:25:48 +08:00
Rob Bradford
c922bf23f6 performance-metrics: Use non-zero exit code on report file issues
This removes the need to wrap an error that is never used.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
f99c09fab7 performance-metrics: Successful output should be on stdout not stderr
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
a6d86b9496 performance-metrics: Use Write::write_all() for report
Write::write() is not guaranteed to write all the contents.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
1004f870e1 performance-metrics: Refactor report file opening code
It should be closer to the use of the opened file.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
d3be855b4b performance-metrics: Exit binary with non-zero exit code on test failure
This ensures that partial reports are not generated.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
0cbdbc8122 performance-metrics: Remove unnecessary prelude on --list-tests
The user knows they have requested the list of tests.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
959be92189 performance-metrics: Fix typo in function name
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
5934319953 performance-metrics: Remove obsolete TODO
We upload from the file written to disk.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-14 16:53:49 +01:00
Rob Bradford
638c4a5781 performance-metrics: Use friendlier metric units
* Boot time in ms
* Network throughput in Gbps
* Block throughput in MiBps

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-03 11:39:55 -08:00
Rob Bradford
9592accb46 performance-metrics: Rename virtio_net_latency_ns->virtio_net_latency_us
The unit from ethr is microseconds.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-03-01 08:44:57 -08:00
Rob Bradford
8458696094 performance-metrics: Rename git_committer_date to git_commit_date
The former implies a person rather than a thing.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
4cc44f8cc7 performance-metrics: Trim whitespace from git command output
This removes trailing "\n"s from the strings saved in the JSON file.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
846bd3a504 performance-metrics: Always derive core count from number of queues
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
c11d430651 performance-metrics: Don't use multipler for vCPU count on block test
This looks like it was copy and pasted from the network test which
required 2 vCPUs per queue pair but has since been resolved.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
9978aac402 performance-metrics: Consistently rename queue_num to num_queues
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
082872423f performance-metrics: Fix test label to remove duplicated "bps"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 12:52:27 -08:00
Rob Bradford
acafda6739 performance-metrics: Consistently use "test_timeout" over "test_time"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
6db3d89837 performance-metrics: Remove "performance_" prefix from report results
We know that these are all performance results.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
7c965538a6 performance-metrics: Use default queue sizes
Using the default queue sizes provides more realistic data about what
our users are testing. Extra metrics can be added later that also
modify the queue size but overloading the existing metrics is confusing.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
f254f11b48 performance-metrics: Specify queue details for latency test
The latency is the most sensitive to queue size changes.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
18d51d3bd1 performance-metrics: Be consistent with num_queues
Don't treat it as the number of pairs. Instead calculate the number of
pairs later when necessary.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
70a3564a5e performance-metrics: Suffix tests with unit
Also be clear that the block tests are in bytes per second (Bps) vs
network that is in bits per second (bps).

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
128d0984a2 performance-metrics: Add missing copyright & SPDX header
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
5e3d72671b performance-metrics: Rename queue_num to num_queues
The former is ambiguous as it could be considered the ordinal (and is
used elsewhere in the codebase for that).

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
60d805771b performance-metrics: Rename test_time to test_timeout
This clarifies its usage vs the more ambiguous name.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Rob Bradford
3f4cbce900 performance-metrics: Set default number of iterations to 5
Many of the tests already amortize their results over a longer time
period/sample so it is not necessary to run 30 iterations.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2022-02-25 16:28:27 +00:00
Bo Chen
ec7d4e727a performance-metrics: Store the test list in a const array
Fixes: #3740

Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-24 16:34:32 +01:00
Bo Chen
5bd305fa4f performance-metrics: Log the git information from the working folder
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
d650c684f1 performance-metrics: Add option "--report-file"
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00
Bo Chen
458ae2c5be performance-metrics: Add git committer date to the report
Signed-off-by: Bo Chen <chen.bo@intel.com>
2022-02-17 14:14:38 +00:00