16 KiB
- Cloud Hypervisor API
Cloud Hypervisor API
The Cloud Hypervisor API is made of 2 distinct interfaces:
- The external API. This is the user facing API. Users and operators can control and manage Cloud Hypervisor through either a REST API or a Command Line Interface (CLI).
- The internal API, based on rust's Multi-Producer, Single-Consumer (MPSC) module. This API is used internally by the Cloud Hypervisor threads to communicate between each others.
The goal of this document is to describe the Cloud Hypervisor API as a whole, and to outline how the internal and external APIs are architecturally related.
External API
REST API
The Cloud Hypervisor REST API triggers VM and VMM specific actions, and as such it is designed as a collection of RPC-style, static methods.
The API is OpenAPI 3.0 compliant. Please consult the Cloud Hypervisor API document for more details about the API payloads and responses.
Location and availability
The REST API is available as soon as the Cloud Hypervisor binary is started,
through a local UNIX socket.
By default, it is located at /run/user/{user ID}/cloud-hypervisor.{Cloud Hypervisor PID}
.
For example, if you launched Cloud Hypervisor as user ID 1000 and its PID is
123456, the Cloud Hypervisor REST API will be available at /run/user/1000/cloud-hypervisor.123456
.
The REST API default URL can be overridden through the Cloud Hypervisor
option --api-socket
:
$ ./target/debug/cloud-hypervisor --api-socket /tmp/cloud-hypervisor.sock
Cloud Hypervisor Guest
API server: /tmp/cloud-hypervisor.sock
vCPUs: 1
Memory: 512 MB
Kernel: None
Kernel cmdline:
Disk(s): None
Endpoints
The Cloud Hypervisor API exposes the following actions through its endpoints:
Virtual Machine Manager (VMM) Actions
Action | Endpoint | Request Body | Response Body | Prerequisites |
---|---|---|---|---|
Check for the REST API availability | /vmm.ping |
N/A | /schemas/VmmPingResponse |
N/A |
Shut the VMM down | /vmm.shutdown |
N/A | N/A | The VMM is running |
Virtual Machine (VM) Actions
Action | Endpoint | Request Body | Response Body | Prerequisites |
---|---|---|---|---|
Create the VM | /vm.create |
/schemas/VmConfig |
N/A | The VM is not created yet |
Delete the VM | /vm.delete |
N/A | N/A | The VM is created but not booted |
Boot the VM | /vm.boot |
N/A | N/A | The VM is created |
Shut the VM down | /vm.shutdown |
N/A | N/A | The VM is booted |
Reboot the VM | /vm.reboot |
N/A | N/A | The VM is booted |
Pause the VM | /vm.pause |
N/A | N/A | The VM is booted |
Resume the VM | /vm.resume |
N/A | N/A | The VM is paused |
Add/remove CPUs to/from the VM | /vm.resize |
/schemas/VmResize |
N/A | The VM is booted |
Remove memory from the VM | /vm.resize |
/schemas/VmResize |
N/A | The VM is booted |
Dump the VM information | /vm.info |
N/A | /schemas/VmInfo |
The VM is created |
Add VFIO PCI device to the VM | /vm.add-device |
/schemas/VmAddDevice |
N/A | The VM is booted |
Remove VFIO PCI device from the VM | /vm.remove-device |
/schemas/VmRemoveDevice |
N/A | The VM is booted |
Add disk device to the VM | /vm.add-disk |
/schemas/DiskConfig |
N/A | The VM is booted |
REST API Examples
For the following set of examples, we assume Cloud Hypervisor is started with
the REST API available at /tmp/cloud-hypervisor.sock
:
$ ./target/debug/cloud-hypervisor --api-socket /tmp/cloud-hypervisor.sock
Cloud Hypervisor Guest
API server: /tmp/cloud-hypervisor.sock
vCPUs: 1
Memory: 512 MB
Kernel: None
Kernel cmdline:
Disk(s): None
Create a Virtual Machine
We want to create a virtual machine with the following characteristics:
- 4 vCPUs
- 1 GB of RAM
- 1 virtio based networking interface
- Direct kernel boot from a custom 5.5.0 Linux kernel located at
/opt/clh/kernel/vmlinux-virtio-fs-virtio-iommu
- Using a Clear Linux image as its root filesystem, located at
/opt/clh/images/clear-30080-kvm.img
#!/bin/bash
curl --unix-socket /tmp/cloud-hypervisor.sock -i \
-X PUT 'http://localhost/api/v1/vm.create' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"cpus":{"boot_vcpus": 4, "max_vcpus": 4},
"kernel":{"path":"/opt/clh/kernel/vmlinux-virtio-fs-virtio-iommu"},
"cmdline":{"args":"console=hvc0 reboot=k panic=1 nomodules i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd root=/dev/vda3"},
"disks":[{"path":"/opt/clh/images/clear-30080-kvm.img"}],
"rng":{"src":"/dev/urandom"},
"net":[{"ip":"192.168.10.10", "mask":"255.255.255.0", "mac":"12:34:56:78:90:01"}]
}'
Boot a Virtual Machine
Once the VM is created, we can boot it:
#!/bin/bash
curl --unix-socket /tmp/cloud-hypervisor.sock -i -X PUT 'http://localhost/api/v1/vm.boot'
Dump a Virtual Machine Information
We can fetch information about any VM, as soon as it's created:
#!/bin/bash
curl --unix-socket /tmp/cloud-hypervisor.sock -i \
-X GET 'http://localhost/api/v1/vm.info' \
-H 'Accept: application/json'
Reboot a Virtual Machine
We can reboot a VM that's already booted:
#!/bin/bash
curl --unix-socket /tmp/cloud-hypervisor.sock -i -X PUT 'http://localhost/api/v1/vm.reboot'
Shut a Virtual Machine Down
Once booted, we can shut a VM down from the REST API:
#!/bin/bash
curl --unix-socket /tmp/cloud-hypervisor.sock -i -X PUT 'http://localhost/api/v1/vm.shutdown'
Command Line Interface
The Cloud Hypervisor Command Line Interface (CLI) can only be used for launching the Cloud Hypervisor binary, i.e. it can not be used for controlling the VMM or the launched VM once they're up and running.
If you want to inspect the VMM, or control the VM after launching Cloud Hypervisor from the CLI, you must use the REST API.
From the CLI, one can either:
- Create and boot a complete virtual machine by using the CLI options to build
the VM config. Run
cloud-hypervisor --help
for a complete list of CLI options. As soon as thecloud-hypervisor
binary is launched, the REST API is available for controlling and managing the VM. - Start the REST API server only, by not passing any VM configuration options. The VM can then be asynchronously created and booted by sending HTTP commands to the REST API. Check the REST API examples section for more details.
REST API and CLI Architectural Relationship
The REST API and the CLI both rely on a common, internal API.
The CLI options are parsed by the clap crate and then translated into internal API commands.
The REST API is processed by an HTTP thread using the
Firecracker's micro_http
crate. As with the CLI, the HTTP requests eventually get translated into
internal API commands.
As a summary, the REST API and the CLI are essentially frontends for the internal API:
+------------------+
REST API | |
+--------->+ micro_http +--------+
| | | |
| +------------------+ |
| | +------------------------+
| | | |
+------------+ | | | |
| | | | | +--------------+ |
| User +---------+ +------> | Internal API | |
| | | | | +--------------+ |
+------------+ | | | |
| | | |
| | +------------------------+
| +----------+ | VMM
| CLI | | |
+----------->+ clap +--------------+
| |
+----------+
Internal API
The Cloud Hypervisor internal API, as its name suggests, is used internally by the different Cloud Hypervisor threads (VMM, HTTP, control loop, etc) to send commands and responses to each others.
It is based on rust's Multi-Producer, Single-Consumer (MPSC), and the single consumer (a.k.a. the API receiver) is the Cloud Hypervisor control loop.
API producers are the HTTP thread handling the REST API and the main thread that initially parses the CLI.
Goals and Design
The internal API is designed for controlling, managing and inspecting a Cloud Hypervisor VMM and its guest. It is a backend for handling external, user visible requests through either the REST API or the CLI interfaces.
The API follows a command-response scheme that closely maps the REST API. Any command must be replied to with a response.
Commands are MPSC based messages and are received and processed by the VMM control loop.
In order for the VMM control loop to respond to any internal API command, it must be able to send a response back to the MPSC sender. For that purpose, all internal API command payload carry the Sender end of an MPSC channel.
The sender of any internal API command is therefore responsible for:
- Creating an MPSC response channel.
- Passing the Sender end of the response channel as part of the internal API command payload.
- Waiting for the internal API command's response on the Receiver end of the response channel.
End to End Example
In order to further understand how the external and internal Cloud Hypervisor APIs work together, let's look at a complete VM creation flow, from the REST API call, to the reply the external user will receive:
- A user or operator sends an HTTP request to the Cloud Hypervisor
REST API in order to creates a virtual machine:
shell #!/bin/bash curl --unix-socket /tmp/cloud-hypervisor.sock -i \ -X PUT 'http://localhost/api/v1/vm.create' \ -H 'Accept: application/json' \ -H 'Content-Type: application/json' \ -d '{ "cpus":{"boot_vcpus": 4, "max_vcpus": 4}, "kernel":{"path":"/opt/clh/kernel/vmlinux-virtio-fs-virtio-iommu"}, "cmdline":{"args":"console=hvc0 reboot=k panic=1 nomodules i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd root=/dev/vda3"}, "disks":[{"path":"/opt/clh/images/clear-30080-kvm.img"}], "rng":{"src":"/dev/urandom"}, "net":[{"ip":"192.168.10.10", "mask":"255.255.255.0", "mac":"12:34:56:78:90:01"}] }'
- The Cloud Hypervisor HTTP thread processes the request and de-serializes the
HTTP request JSON body into an internal
VmConfig
structure. - The Cloud Hypervisor HTTP thread creates an MPSC channel for the internal API server to send its response back.
- The Cloud Hypervisor HTTP thread prepares an internal API command for creating a
virtual machine. The command's payload is made of the de-serialized
VmConfig
structure and the response channel:VmCreate(Arc<Mutex<VmConfig>>, Sender<ApiResponse>)
- The Cloud Hypervisor HTTP thread sends the internal API command, and waits
for the response:
// Send the VM creation request. api_sender .send(ApiRequest::VmCreate(config, response_sender)) .map_err(ApiError::RequestSend)?; api_evt.write(1).map_err(ApiError::EventFdWrite)?; response_receiver.recv().map_err(ApiError::ResponseRecv)??;
- The Cloud Hypervisor control loop receives the command, as it listens on the
internal API MPSC channel:
// Read from the API receiver channel let api_request = api_receiver.recv().map_err(Error::ApiRequestRecv)?;
- The Cloud Hypervisor control loop matches the received internal API against
the
VmCreate
payload, and extracts both theVmConfig
structure and the Sender from the command payload. It stores theVmConfig
structure and replies back to the sender ((The HTTP thread):match api_request { ApiRequest::VmCreate(config, sender) => { // We only store the passed VM config. // The VM will be created when being asked to boot it. let response = if self.vm_config.is_none() { self.vm_config = Some(config); Ok(ApiResponsePayload::Empty) } else { Err(ApiError::VmAlreadyCreated) }; sender.send(response).map_err(Error::ApiResponseSend)?; }
- The Cloud Hypervisor HTTP thread receives the internal API command response
as the return value from its
VmCreate
HTTP handler. Depending on the control loop internal API response, it generates the appropriate HTTP response:// Call vm_create() match vm_create(api_notifier, api_sender, Arc::new(Mutex::new(vm_config))) .map_err(HttpError::VmCreate) { Ok(_) => Response::new(Version::Http11, StatusCode::NoContent), Err(e) => error_response(e, StatusCode::InternalServerError), }
- The Cloud Hypervisor HTTP thread sends the formed HTTP response back to the user. This is abstracted by the micro_http crate.