cloud-hypervisor/docs/performance_metrics.md
Songqian Li e18d32bac2 performance-metrics: add restore latency performance metric
This patch calculates the recovery latency from the start of
recovery program to the end of restore.

Signed-off-by: Songqian Li <sionli@tencent.com>
2024-08-02 08:29:03 +00:00

5.5 KiB

Performance Metrics

Cloud Hypervisor provides a performance metrics binary for users to generate metrics data from their own environment. This document describes how to generate metrics data quickly by using Cloud Hypervisor's development script, e.g. dev_cli.sh. The only prerequisite is Docker installation. Please note that upon its first invocation, this script will pull a fairly large container image.

Run the Performance Tests

To generate metrics data for all available performance tests (including boot time, block I/O throughput, and network throughput & latency) and output the result into a json file:

$ ./scripts/dev_cli.sh tests --metrics -- -- --report-file /tmp/metrics.json

To get a list of available performance tests:

$ ./scripts/dev_cli.sh tests --metrics -- -- --list-tests

To generate metrics data for selected performance tests, e.g. boot time only:

$ ./scripts/dev_cli.sh tests --metrics -- -- --report-file /tmp/metrics.json --test-filter boot_time

To set custom timeout or test iterations for all performance tests:

$ ./scripts/dev_cli.sh tests --metrics -- -- --timeout 5 --iterations 10

Performance Tests Details

The following table lists the supported performance tests with default timeout and number of iterations. The timeout defines the maximum execution time of each test for each iteration. The iteration defines how many times a test needs to be executed to generate the final metrics data.

Type Metric Timeout(s) Iterations
Boot Time boot_time_ms 2 10
boot_time_pmem_ms 2 10
boot_time_16_vcpus_ms 2 10
boot_time_16_vcpus_pmem_ms 2 10
Virtio Net virtio_net_latency_us 10 5
virtio_net_throughput_single_queue_rx_gbps 10 5
virtio_net_throughput_single_queue_tx_gbps 10 5
virtio_net_throughput_multi_queue_rx_gbps 10 5
virtio_net_throughput_multi_queue_tx_gbps 10 5
Block block_read_MiBps 10 5
block_write_MiBps 10 5
block_random_read_MiBps 10 5
block_random_write_MiBps 10 5
block_multi_queue_read_MiBps 10 5
block_multi_queue_write_MiBps 10 5
block_multi_queue_random_read_MiBps 10 5
block_multi_queue_random_write_MiBps 10 5
block_read_IOPS 10 5
block_write_IOPS 10 5
block_random_read_IOPS 10 5
block_random_write_IOPS 10 5
block_multi_queue_read_IOPS 10 5
block_multi_queue_write_IOPS 10 5
block_multi_queue_random_read_IOPS 10 5
block_multi_queue_random_write_IOPS 10 5
Other restore_latency_time_ms 2 10

Output Format

Performance-metrics output the result into a json file if report-file param is set. The fields included in JSON include:

Field Name Content
git_human_readable Recent tag information of git repository
git_revision Commit id of HEAD
git_commit_date Commit date of HEAD
date Date for executing the program
results A list of metrics

Example

Here is an example of generating metrics data for the boot time using pmem:

$ ./scripts/dev_cli.sh tests --metrics -- -- --test-filter boot_time_pmem_ms

Here is a sample output:

{
  "git_human_readable": "v40.0",
  "git_revision": "e9b263975786abbf895469b93dfc00f21ce39a88",
  "git_commit_date": "Fri Jun 21 08:40:44 2024 +0000",
  "date": "Tue Jul 16 16:35:29 UTC 2024",
  "results": [
    {
      "name": "boot_time_pmem_ms",
      "mean": 105.9461,
      "std_dev": 7.140993312558129,
      "max": 120.01499999999999,
      "min": 92.37600000000002
    }
  ]
}

Note that the metrics data above is for illustration purpose only and does not represent the actual performance of Cloud Hypervisor on your system.