The repository about the Rust hypervsior
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
dependabot[bot] ca6d338ffa build: Bump clap from 4.0.27 to 4.0.29 23 hours ago
.github .github: Ensure target used for clippy tests 2 days ago
acpi_tables build: Bulk update dependencies 5 days ago
api_client build: Bulk update dependencies 5 days ago
arch vmm: Seperate the CPUID setup from the CpuManager::new() 2 days ago
block_util build: Bulk update dependencies 5 days ago
devices devices: gic: Pass slice rather than &Vec 2 days ago
docs docs: Fix a typo on the doc for tpm 6 days ago
event_monitor event_monitor: add safety comments 2 weeks ago
fuzz build: Bump io-lifetimes from 1.0.2 to 1.0.3 in /fuzz 24 hours ago
hypervisor aarch64: Remove unnecessary casts (beta clippy check) 2 days ago
net_gen build: Bulk update dependencies 5 days ago
net_util virtio-devices: net: Provide custom functions for fuzzing 3 days ago
option_parser option_parser: For tuple list type strip brackets only once 4 months ago
pci pci: Remove unused restore() implementations 4 days ago
performance-metrics build: Bump clap from 4.0.27 to 4.0.29 23 hours ago
qcow build: Bulk update dependencies 5 days ago
rate_limiter build: Bulk update dependencies 5 days ago
resources build: Update development container to Rust 1.62 5 months ago
scripts tests: Enable VFIO integration tests 1 week ago
serial_buffer vmm: Move SerialBuffer to its own crate 3 months ago
src main: Allow the use of `let` ahead and immediate return 2 days ago
test_data/cloud-init/ubuntu test_data: Use bash TCP support instead of netcat for boot check 2 months ago
test_infra build: Bulk update dependencies 5 days ago
tests tests: Disable live_upgrade test on AArch64 2 days ago
tpm build: Bulk update dependencies 5 days ago
tracer tracer: add safety comments 2 weeks ago
vfio_user build: Bulk update dependencies 5 days ago
vhdx vhdx: add some safety comments 2 weeks ago
vhost_user_block build: Bump clap from 4.0.27 to 4.0.29 23 hours ago
vhost_user_net build: Bump clap from 4.0.27 to 4.0.29 23 hours ago
virtio-devices virtio-devices, vmm: Always restore virtio devices in paused state 3 days ago
vm-allocator build: Bulk update dependencies 5 days ago
vm-device build: Bulk update dependencies 5 days ago
vm-migration vmm: Move Vm to the new restore design 2 days ago
vm-virtio build: Bulk update dependencies 5 days ago
vmm build: Bump clap from 4.0.27 to 4.0.29 23 hours ago
.gitignore rpm: Extend spec to support aarch64 and more 6 months ago
.rustfmt.toml build: migrate to Rust 2021 edition 8 months ago
CODEOWNERS ci: Adding a CODEOWNERS file 7 months ago
CODE_OF_CONDUCT.md cloud-hypervisor: Adopt the Contributor Covenant code of conduct 4 years ago
CONTRIBUTING.md CONTRIBUTING.md: Add sample pre-commit hook 4 months ago
CREDITS.md cloud-hypervisor: Add CREDITS 4 years ago
Cargo.lock build: Bump clap from 4.0.27 to 4.0.29 23 hours ago
Cargo.toml build: Bump clap from 4.0.27 to 4.0.29 23 hours ago
Jenkinsfile tests: Enable VFIO integration tests 1 week ago
LICENSE-APACHE cloud-hypervisor: Add proper licensing 4 years ago
LICENSE-BSD-3-Clause cloud-hypervisor: Add proper licensing 4 years ago
MAINTAINERS.md docs: Update MAINTAINERS.md 1 year ago
README.md README: Update version of Rust Hypervisor Firmware 2 weeks ago
build.rs misc: Automatic beta clippy fixes 2 months ago
release-notes.md build: Release v28.0 2 weeks ago

README.md

1. What is Cloud Hypervisor?

Cloud Hypervisor is an open source Virtual Machine Monitor (VMM) that runs on top of the KVM hypervisor and the Microsoft Hypervisor (MSHV).

The project focuses on running modern, Cloud Workloads, on specific, common, hardware architectures. In this case Cloud Workloads refers to those that are run by customers inside a Cloud Service Provider. This means modern operating systems with most I/O handled by paravirtualised devices (e.g. virtio), no requirement for legacy devices, and 64-bit CPUs.

Cloud Hypervisor is implemented in Rust and is based on the Rust VMM crates.

Objectives

High Level

  • Runs on KVM or MSHV
  • Minimal emulation
  • Low latency
  • Low memory footprint
  • Low complexity
  • High performance
  • Small attack surface
  • 64-bit support only
  • CPU, memory, PCI hotplug
  • Machine to machine migration

Architectures

Cloud Hypervisor supports the x86-64 and AArch64 architectures. There are minor differences in functionality between the two architectures (see #1125).

Guest OS

Cloud Hypervisor supports 64-bit Linux and Windows 10/Windows Server 2019.

2. Getting Started

The following sections describe how to build and run Cloud Hypervisor on the x86-64 platform. For getting started on the AArch64 platform, please refer to the AArch64 documentation.

Host OS

For required KVM functionality the minimum host kernel version is 4.11. For adequate performance the minimum recommended host kernel version is 5.6. The majority of the CI currently tests with kernel version 5.15.

Use Pre-built Binaries

The recommended approach to getting started with Cloud Hypervisor is by using a pre-built binary. Binaries are available for the latest release. Use cloud-hypervisor-static for x86-64 or cloud-hypervisor-static-aarch64 for AArch64 platform.

Packages

For convenience, packages are also available targeting some popular Linux distributions. This is thanks to the Open Build Service. The OBS README explains how to enable the repository in a supported Linux distribution and install Cloud Hypervisor and accompanying packages. Please report any packaging issues in the obs-packaging repository.

Building from Source

Please see the instructions for building from source if you do not wish to use the pre-built binaries.

Booting Linux

The instructions below are for the x86-64 platform. For AArch64 please see the AArch64 specific documentation.

Cloud Hypervisor supports direct kernel boot (if the kernel is built with PVH support) or booting via a firmware (either Rust Hypervisor Firmware or an edk2 UEFI firmware called CLOUDHV.)

Binary builds of the firmware files are available for the latest release of Rust Hyperivor Firmware and our edk2 repository

The choice of firmware depends on your guest OS choice; some experimentation may be required.

Firmware Booting

Cloud Hypervisor supports booting disk images containing all needed components to run cloud workloads, a.k.a. cloud images.

The following sample commands will download an Ubuntu Cloud image, converting it into a format that Cloud Hypervisor can use and a firmware to boot the image with.

$ wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img
$ qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-amd64.img focal-server-cloudimg-amd64.raw
$ wget https://github.com/cloud-hypervisor/rust-hypervisor-firmware/releases/download/0.4.2/hypervisor-fw

The Ubuntu cloud images do not ship with a default password so it necessary to use a cloud-init disk image to customise the image on the first boot. A basic cloud-init image is generated by this script. This seeds the image with a default username/password of cloud/cloud123. It is only necessary to add this disk image on the first boot.

$ sudo setcap cap_net_admin+ep ./cloud-hypervisor
$ ./create-cloud-init.sh
$ ./cloud-hypervisor \
	--kernel ./hypervisor-fw \
	--disk path=focal-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=,ip=,mask="

If access to the firmware messages or interaction with the boot loader (e.g. GRUB) is required then it necessary to switch to the serial console instead of virtio-console.

$ ./cloud-hypervisor \
	--kernel ./hypervisor-fw \
	--disk path=focal-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=,ip=,mask=" \
	--serial tty \
	--console off

Custom Kernel and Disk Image

Building your Kernel

Cloud Hypervisor also supports direct kernel boot into a vmlinux ELF kernel (compiled with PVH support). In order to support development there is a custom branch; however provided the required options are enabled any recent kernel will suffice.

To build the kernel:

# Clone the Cloud Hypervisor Linux branch
$ git clone --depth 1 https://github.com/cloud-hypervisor/linux.git -b ch-5.15.12 linux-cloud-hypervisor
$ pushd linux-cloud-hypervisor
# Use the cloud-hypervisor kernel config to build your kernel
$ wget https://raw.githubusercontent.com/cloud-hypervisor/cloud-hypervisor/main/resources/linux-config-x86_64
$ cp linux-config-x86_64 .config
$ KCFLAGS="-Wa,-mx86-used-note=no" make bzImage -j `nproc`
$ popd

The vmlinux kernel image will then be located at linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin.

Disk image

For the disk image the same Ubuntu image as before can be used. This contains an ext4 root filesystem.

$ wget https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64.img
$ qemu-img convert -p -f qcow2 -O raw focal-server-cloudimg-amd64.img focal-server-cloudimg-amd64.raw

Booting the guest VM

These sample commands boot the disk image using the custom kernel whilst also supplying the desired kernel command line.

$ sudo setcap cap_net_admin+ep ./cloud-hypervisor
$ ./create-cloud-init.sh
$ ./cloud-hypervisor \
	--kernel ./linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin \
	--disk path=focal-server-cloudimg-amd64.raw path=/tmp/ubuntu-cloudinit.img \
	--cmdline "console=hvc0 root=/dev/vda1 rw" \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=,ip=,mask="

If earlier kernel messages are required the serial console should be used instead of virtio-console.

	--kernel ./linux-cloud-hypervisor/arch/x86/boot/compressed/vmlinux.bin \
	--console off \
	--serial tty \
	--disk path=focal-server-cloudimg-amd64.raw \
	--cmdline "console=ttyS0 root=/dev/vda1 rw" \
	--cpus boot=4 \
	--memory size=1024M \
	--net "tap=,mac=,ip=,mask="

3. Status

Cloud Hypervisor is under active development. The following stability guarantees are currently made:

  • The API (including command line options) will not be removed or changed in a breaking way without a minimum of 2 major releases notice. Where possible warnings will be given about the use of deprecated functionality and the deprecations will be documented in the release notes.

  • Point releases will be made between individual releases where there are substantial bug fixes or security issues that need to be fixed. These point releases will only include bug fixes.

Currently the following items are not guaranteed across updates:

  • Snapshot/restore is not supported across different versions
  • Live migration is not supported across different versions
  • The following features are considered experimental and may change substantially between releases: TDX, vfio-user, vDPA.

Further details can be found in the release documentation.

As of 2022-10-13, the following cloud images are supported:

Direct kernel boot to userspace should work with a rootfs from most distributions although you may need to enable exotic filesystem types in the reference kernel configuration (e.g. XFS or btrfs.)

Hot Plug

Cloud Hypervisor supports hotplug of CPUs, passthrough devices (VFIO), virtio-{net,block,pmem,fs,vsock} and memory resizing. This document details how to add devices to a running VM.

Device Model

Details of the device model can be found in this documentation.

Roadmap

The project roadmap is tracked through a GitHub project.

4. Relationship with Rust VMM Project

In order to satisfy the design goal of having a high-performance, security-focused hypervisor the decision was made to use the Rust programming language. The language's strong focus on memory and thread safety makes it an ideal candidate for implementing VMMs.

Instead of implementing the VMM components from scratch, Cloud Hypervisor is importing the Rust VMM crates, and sharing code and architecture together with other VMMs like e.g. Amazon's Firecracker and Google's crosvm.

Cloud Hypervisor embraces the Rust VMM project's goals, which is to be able to share and re-use as many virtualization crates as possible.

Differences with Firecracker and crosvm

A large part of the Cloud Hypervisor code is based on either the Firecracker or the crosvm project's implementations. Both of these are VMMs written in Rust with a focus on safety and security, like Cloud Hypervisor.

The goal of the Cloud Hypervisor project differs from the aforementioned projects in that it aims to be a general purpose VMM for Cloud Workloads and not limited to container/serverless or client workloads.

The Cloud Hypervisor community thanks the communities of both the Firecracker and crosvm projects for their excellent work.

5. Community

The Cloud Hypervisor project follows the governance, and community guidelines described in the Community repository.

Contribute

The project strongly believes in building a global, diverse and collaborative community around the Cloud Hypervisor project. Anyone who is interested in contributing to the project is welcome to participate.

Contributing to a open source project like Cloud Hypervisor covers a lot more than just sending code. Testing, documentation, pull request reviews, bug reports, feature requests, project improvement suggestions, etc, are all equal and welcome means of contribution. See the CONTRIBUTING document for more details.

Slack

Get an invite to our Slack channel and join us on Slack.

Mailing list

Please report bugs using the GitHub issue tracker but for broader community discussions you may use our mailing list.

Security issues

Please contact the maintainers listed in the MAINTAINERS.md file with security issues.