cloud-hypervisor/docs/cpu.md

CPU

Cloud Hypervisor has many options when it comes to the creation of virtual CPUs. This document aims to explain what Cloud Hypervisor is capable of and how it can be used to meet the needs of very different use cases.

Options

CpusConfig or what is known as --cpus from the CLI perspective is the way to set vCPUs options for Cloud Hypervisor.

struct CpusConfig {
    boot_vcpus: u8,
    max_vcpus: u8,
    topology: Option<CpuTopology>,
    kvm_hyperv: bool,
    max_phys_bits: u8,
    affinity: Option<Vec<CpuAffinity>>,
    features: CpuFeatures,
}

--cpus boot=<boot_vcpus>,max=<max_vcpus>,topology=<threads_per_core>:<cores_per_die>:<dies_per_package>:<packages>,kvm_hyperv=on|off,max_phys_bits=<maximum_number_of_physical_bits>,affinity=<list_of_vcpus_with_their_associated_cpuset>,features=<list_of_features_to_enable>

boot

Number of vCPUs present at boot time.

This option allows to define a specific number of vCPUs to be present at the time the VM is started. This option is mandatory when using the --cpus parameter. If --cpus is not specified, this option takes the default value of 1, starting the VM with a single vCPU.

Value is an unsigned integer of 8 bits.

Example

--cpus boot=2

max

Maximum number of vCPUs.

This option defines the maximum number of vCPUs that can be assigned to the VM. In particular, this option is used when looking for CPU hotplug as it lets the provide an indication about how many vCPUs might be needed later during the runtime of the VM. For instance, if booting the VM with 2 vCPUs and a maximum of 6 vCPUs, it means up to 4 vCPUs can be added later at runtime by resizing the VM.

The value must be greater than or equal to the number of boot vCPUs. The value is an unsigned integer of 8 bits.

By default this option takes the value of boot, meaning vCPU hotplug is not expected and can't be performed.

Example

--cpus max=3

topology

Topology of the guest platform.

This option gives the user a way to describe the exact topology that should be exposed to the guest. It can be useful to describe to the guest the same topology found on the host as it allows for proper usage of the resources and is a way to achieve better performances.

The topology is described through the following structure:

struct CpuTopology {
    threads_per_core: u8,
    cores_per_die: u8,
    dies_per_package: u8,
    packages: u8,
}

or the following syntax through the CLI:

topology=<threads_per_core>:<cores_per_die>:<dies_per_package>:<packages>

By default the topology will be 1:1:1:1.

Example

--cpus boot=2,topology=1:1:2:1

kvm_hyperv

Enable KVM Hyper-V emulation.

When turned on, this option relies on KVM to emulate the synthetic interrupt controller (SynIC) along with synthetic timers expected by a Windows guest. A Windows guest usually runs on top of Microsoft Hyper-V, therefore expects these synthetic devices to be present. That's why KVM provides a way to emulate them and avoids failures running a Windows guest with Cloud Hypervisor.

By default this option is turned off.

Example

--cpus kvm_hyperv=on

max_phys_bits

Maximum size for guest's addressable space.

This option defines the maximum number of physical bits for all vCPUs, which sets a limit for the size of the guest's addressable space. This is mainly useful for debug purpose.

The value is an unsigned integer of 8 bits.

Example

--cpus max_phys_bits=40

affinity

Affinity of each vCPU.

This option gives the user a way to provide the host CPU set associated with each vCPU. It is useful for achieving CPU pinning, ensuring multiple VMs won't affect the performance of each other. It might also be used in the context of NUMA as it is way of making sure the VM can run on a specific host NUMA node. In general, this option is used to increase the performances of a VM depending on the host platform and the type of workload running in the guest.

The affinity is described through the following structure:

struct CpuAffinity {
    vcpu: u8,
    host_cpus: Vec<u8>,
}

or the following syntax through the CLI:

affinity=[<vcpu_id1>@[<host_cpu_id1>, <host_cpu_id2>], <vcpu_id2>@[<host_cpu_id3>, <host_cpu_id4>]]

The outer brackets define the list of vCPUs. And for each vCPU, the inner brackets attached to @ define the list of host CPUs the vCPU is allowed to run onto.

Multiple values can be provided to define each list. Each value is an unsigned integer of 8 bits.

For instance, if one needs to run vCPU 0 on host CPUs from 0 to 4, the syntax using - will help define a contiguous range with affinity=0@[0-4]. The same example could also be described with affinity=0@[0,1,2,3,4].

A combination of both - and , separators is useful when one might need to describe a list containing host CPUs from 0 to 99 and the host CPU 255, as it could simply be described with affinity=0@[0-99,255].

As soon as one tries to describe a list of values, [ and ] must be used to demarcate the list.

By default each vCPU runs on the entire host CPU set.

Example

--cpus boot=3,affinity=[0@[2,3],1@[0,1]]

In this example, assuming the host has 4 CPUs, vCPU 0 will run exclusively on host CPUs 2 and 3, while vCPU 1 will run exclusively on host CPUs 0 and 1. Because nothing is defined for vCPU 2, it can run on any of the 4 host CPUs.

features

Set of CPU features to enable.

This option allows the user to enable a set of CPU features that are disabled by default otherwise.

The currently available feature set is: amx.

The amx feature will enable the x86 extension adding hardware units for matrix operations (int and float dot products). The goal of the extension is to provide performance enhancements for these common operations.

Example

--cpus features=amx

In this example the amx CPU feature will be enabled for the VMM.