2019-09-04 13:55:14 +00:00
|
|
|
// Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
|
|
|
|
//
|
|
|
|
// Portions Copyright 2017 The Chromium OS Authors. All rights reserved.
|
|
|
|
// Use of this source code is governed by a BSD-style license that can be
|
|
|
|
// found in the LICENSE-BSD-3-Clause file.
|
|
|
|
//
|
|
|
|
// Copyright © 2019 Intel Corporation
|
|
|
|
//
|
|
|
|
// SPDX-License-Identifier: Apache-2.0 AND BSD-3-Clause
|
|
|
|
//
|
|
|
|
|
2021-05-04 08:33:35 +00:00
|
|
|
use crate::config::{
|
2021-06-09 13:43:36 +00:00
|
|
|
ConsoleOutputMode, DeviceConfig, DiskConfig, FsConfig, NetConfig, PmemConfig, UserDeviceConfig,
|
2022-03-11 11:31:25 +00:00
|
|
|
VdpaConfig, VhostMode, VmConfig, VsockConfig,
|
2021-05-04 08:33:35 +00:00
|
|
|
};
|
2022-11-03 15:10:18 +00:00
|
|
|
use crate::cpu::{CpuManager, CPU_MANAGER_ACPI_SIZE};
|
2020-05-05 10:19:43 +00:00
|
|
|
use crate::device_tree::{DeviceNode, DeviceTree};
|
2020-07-29 12:09:52 +00:00
|
|
|
use crate::interrupt::LegacyUserspaceInterruptManager;
|
2022-05-10 16:00:17 +00:00
|
|
|
use crate::interrupt::MsiInterruptManager;
|
2022-06-08 14:41:24 +00:00
|
|
|
use crate::memory_manager::{Error as MemoryManagerError, MemoryManager, MEMORY_MANAGER_ACPI_SIZE};
|
2021-10-04 15:08:43 +00:00
|
|
|
use crate::pci_segment::PciSegment;
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
use crate::seccomp_filters::{get_seccomp_filter, Thread};
|
2021-09-16 19:47:36 +00:00
|
|
|
use crate::serial_manager::{Error as SerialManagerError, SerialManager};
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
use crate::sigwinch_listener::start_sigwinch_listener;
|
2021-06-02 19:08:04 +00:00
|
|
|
use crate::GuestRegionMmap;
|
2020-06-11 14:48:25 +00:00
|
|
|
use crate::PciDeviceInfo;
|
2020-05-05 10:19:43 +00:00
|
|
|
use crate::{device_node, DEVICE_MANAGER_SNAPSHOT_ID};
|
2022-07-25 12:36:31 +00:00
|
|
|
use acpi_tables::sdt::GenericAddress;
|
2023-03-02 08:01:33 +00:00
|
|
|
use acpi_tables::{aml, Aml};
|
2019-12-02 21:50:38 +00:00
|
|
|
use anyhow::anyhow;
|
2019-12-06 16:14:32 +00:00
|
|
|
use arch::layout;
|
2020-05-12 09:49:12 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2019-12-31 10:49:11 +00:00
|
|
|
use arch::layout::{APIC_START, IOAPIC_SIZE, IOAPIC_START};
|
2021-08-06 23:28:42 +00:00
|
|
|
use arch::NumaNodes;
|
2020-05-25 08:59:09 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
2021-03-26 17:58:17 +00:00
|
|
|
use arch::{DeviceType, MmioDeviceInfo};
|
2023-07-12 02:24:28 +00:00
|
|
|
use block::{
|
2023-10-19 02:59:22 +00:00
|
|
|
async_io::DiskFile, block_aio_is_supported, block_io_uring_is_supported, detect_image_type,
|
|
|
|
fixed_vhd_sync::FixedVhdDiskSync, qcow, qcow_sync::QcowDiskSync, raw_async_aio::RawFileDiskAio,
|
|
|
|
raw_sync::RawFileDiskSync, vhdx, vhdx_sync::VhdxDiskSync, ImageType,
|
2021-01-20 18:03:59 +00:00
|
|
|
};
|
2023-07-10 09:55:55 +00:00
|
|
|
#[cfg(feature = "io_uring")]
|
2023-07-12 02:24:28 +00:00
|
|
|
use block::{fixed_vhd_async::FixedVhdDiskAsync, raw_async::RawFileDisk};
|
2024-01-17 21:12:43 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
use devices::debug_console::DebugConsole;
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
2020-05-25 08:59:09 +00:00
|
|
|
use devices::gic;
|
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
use devices::ioapic;
|
2021-03-25 17:01:21 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
use devices::legacy::Pl011;
|
2020-05-25 08:27:08 +00:00
|
|
|
use devices::{
|
2021-03-12 05:18:52 +00:00
|
|
|
interrupt_controller, interrupt_controller::InterruptController, AcpiNotificationFlags,
|
2020-05-25 08:27:08 +00:00
|
|
|
};
|
2022-08-30 12:01:09 +00:00
|
|
|
use hypervisor::{HypervisorType, IoEventAddress};
|
2021-01-14 03:03:53 +00:00
|
|
|
use libc::{
|
2021-11-17 14:03:18 +00:00
|
|
|
cfmakeraw, isatty, tcgetattr, tcsetattr, termios, MAP_NORESERVE, MAP_PRIVATE, MAP_SHARED,
|
|
|
|
O_TMPFILE, PROT_READ, PROT_WRITE, TCSANOW,
|
2021-01-14 03:03:53 +00:00
|
|
|
};
|
2019-09-04 13:55:14 +00:00
|
|
|
use pci::{
|
2021-10-18 16:29:42 +00:00
|
|
|
DeviceRelocation, PciBarRegionType, PciBdf, PciDevice, VfioPciDevice, VfioUserDmaMapping,
|
2021-10-04 15:08:43 +00:00
|
|
|
VfioUserPciDevice, VfioUserPciDeviceError,
|
2019-09-04 13:55:14 +00:00
|
|
|
};
|
2023-12-07 19:45:08 +00:00
|
|
|
use rate_limiter::group::RateLimiterGroup;
|
2021-08-17 03:40:11 +00:00
|
|
|
use seccompiler::SeccompAction;
|
2022-05-17 21:04:38 +00:00
|
|
|
use serde::{Deserialize, Serialize};
|
2024-01-25 00:37:47 +00:00
|
|
|
use std::collections::{BTreeMap, BTreeSet, HashMap};
|
2021-01-14 03:03:53 +00:00
|
|
|
use std::fs::{read_link, File, OpenOptions};
|
2021-09-02 15:56:37 +00:00
|
|
|
use std::io::{self, stdout, Seek, SeekFrom};
|
2021-01-14 03:03:53 +00:00
|
|
|
use std::mem::zeroed;
|
2020-03-06 10:34:24 +00:00
|
|
|
use std::num::Wrapping;
|
2019-09-04 13:55:14 +00:00
|
|
|
use std::os::unix::fs::OpenOptionsExt;
|
2021-01-14 03:03:53 +00:00
|
|
|
use std::os::unix::io::{AsRawFd, FromRawFd, RawFd};
|
|
|
|
use std::path::PathBuf;
|
2019-09-04 13:55:14 +00:00
|
|
|
use std::result;
|
2021-10-08 09:49:26 +00:00
|
|
|
use std::sync::{Arc, Mutex};
|
2022-05-04 15:36:57 +00:00
|
|
|
use std::time::Instant;
|
2022-09-16 15:56:36 +00:00
|
|
|
use tracer::trace_scoped;
|
2022-07-21 13:15:15 +00:00
|
|
|
use vfio_ioctls::{VfioContainer, VfioDevice, VfioDeviceFd};
|
2020-07-02 12:25:19 +00:00
|
|
|
use virtio_devices::transport::VirtioTransport;
|
2022-05-17 14:33:33 +00:00
|
|
|
use virtio_devices::transport::{VirtioPciDevice, VirtioPciDeviceActivator};
|
2020-07-02 12:25:19 +00:00
|
|
|
use virtio_devices::vhost_user::VhostUserConfig;
|
2022-05-17 14:33:33 +00:00
|
|
|
use virtio_devices::{
|
|
|
|
AccessPlatformMapping, ActivateError, VdpaDmaMapping, VirtioMemMappingSource,
|
|
|
|
};
|
2021-10-21 10:41:16 +00:00
|
|
|
use virtio_devices::{Endpoint, IommuMapping};
|
2021-10-15 10:38:06 +00:00
|
|
|
use vm_allocator::{AddressAllocator, SystemAllocator};
|
2021-02-23 13:57:10 +00:00
|
|
|
use vm_device::dma_mapping::vfio::VfioDmaMapping;
|
2022-03-10 13:16:08 +00:00
|
|
|
use vm_device::dma_mapping::ExternalDmaMapping;
|
2020-02-04 11:04:10 +00:00
|
|
|
use vm_device::interrupt::{
|
|
|
|
InterruptIndex, InterruptManager, LegacyIrqGroupConfig, MsiIrqGroupConfig,
|
|
|
|
};
|
2020-09-09 14:30:31 +00:00
|
|
|
use vm_device::{Bus, BusDevice, Resource};
|
2020-01-24 14:47:15 +00:00
|
|
|
use vm_memory::guest_memory::FileOffset;
|
2021-03-17 17:26:01 +00:00
|
|
|
use vm_memory::GuestMemoryRegion;
|
2021-06-02 19:08:04 +00:00
|
|
|
use vm_memory::{Address, GuestAddress, GuestUsize, MmapRegion};
|
2022-03-29 13:00:29 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2021-03-05 13:44:54 +00:00
|
|
|
use vm_memory::{GuestAddressSpace, GuestMemory};
|
2020-04-28 10:08:51 +00:00
|
|
|
use vm_migration::{
|
2022-10-18 15:14:43 +00:00
|
|
|
protocol::MemoryRangeTable, snapshot_from_id, versioned_state_from_id, Migratable,
|
2022-12-02 13:49:22 +00:00
|
|
|
MigratableError, Pausable, Snapshot, SnapshotData, Snapshottable, Transportable,
|
2020-04-28 10:08:51 +00:00
|
|
|
};
|
2022-01-26 16:33:36 +00:00
|
|
|
use vm_virtio::AccessPlatform;
|
2021-10-21 10:41:16 +00:00
|
|
|
use vm_virtio::VirtioDeviceType;
|
2019-09-04 13:55:14 +00:00
|
|
|
use vmm_sys_util::eventfd::EventFd;
|
2024-01-17 21:12:43 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
use {devices::debug_console, devices::legacy::Serial};
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-10-15 15:34:35 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
2019-09-11 16:25:07 +00:00
|
|
|
const MMIO_LEN: u64 = 0x1000;
|
|
|
|
|
2022-05-04 09:36:26 +00:00
|
|
|
// Singleton devices / devices the user cannot name
|
2020-05-12 09:49:12 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2022-05-04 09:36:26 +00:00
|
|
|
const IOAPIC_DEVICE_NAME: &str = "__ioapic";
|
|
|
|
const SERIAL_DEVICE_NAME: &str = "__serial";
|
2024-01-17 21:12:43 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
const DEBUGCON_DEVICE_NAME: &str = "__debug_console";
|
2021-03-07 12:11:07 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
2022-05-04 09:36:26 +00:00
|
|
|
const GPIO_DEVICE_NAME: &str = "__gpio";
|
|
|
|
const RNG_DEVICE_NAME: &str = "__rng";
|
|
|
|
const IOMMU_DEVICE_NAME: &str = "__iommu";
|
|
|
|
const BALLOON_DEVICE_NAME: &str = "__balloon";
|
|
|
|
const CONSOLE_DEVICE_NAME: &str = "__console";
|
2023-06-15 04:02:45 +00:00
|
|
|
const PVPANIC_DEVICE_NAME: &str = "__pvpanic";
|
2022-05-04 09:36:26 +00:00
|
|
|
|
|
|
|
// Devices that the user may name and for which we generate
|
|
|
|
// identifiers if the user doesn't give one
|
2020-04-29 17:40:17 +00:00
|
|
|
const DISK_DEVICE_NAME_PREFIX: &str = "_disk";
|
|
|
|
const FS_DEVICE_NAME_PREFIX: &str = "_fs";
|
|
|
|
const NET_DEVICE_NAME_PREFIX: &str = "_net";
|
|
|
|
const PMEM_DEVICE_NAME_PREFIX: &str = "_pmem";
|
2022-03-11 11:31:25 +00:00
|
|
|
const VDPA_DEVICE_NAME_PREFIX: &str = "_vdpa";
|
2020-04-29 17:40:17 +00:00
|
|
|
const VSOCK_DEVICE_NAME_PREFIX: &str = "_vsock";
|
2022-05-04 09:36:26 +00:00
|
|
|
const WATCHDOG_DEVICE_NAME: &str = "__watchdog";
|
|
|
|
const VFIO_DEVICE_NAME_PREFIX: &str = "_vfio";
|
|
|
|
const VFIO_USER_DEVICE_NAME_PREFIX: &str = "_vfio_user";
|
2020-04-29 17:40:17 +00:00
|
|
|
const VIRTIO_PCI_DEVICE_NAME_PREFIX: &str = "_virtio-pci";
|
2020-04-27 14:05:04 +00:00
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
/// Errors associated with device manager
|
|
|
|
#[derive(Debug)]
|
|
|
|
pub enum DeviceManagerError {
|
|
|
|
/// Cannot create EventFd.
|
|
|
|
EventFd(io::Error),
|
|
|
|
|
|
|
|
/// Cannot open disk path
|
|
|
|
Disk(io::Error),
|
|
|
|
|
|
|
|
/// Cannot create vhost-user-net device
|
2020-07-02 12:25:19 +00:00
|
|
|
CreateVhostUserNet(virtio_devices::vhost_user::Error),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
/// Cannot create virtio-blk device
|
|
|
|
CreateVirtioBlock(io::Error),
|
|
|
|
|
|
|
|
/// Cannot create virtio-net device
|
2020-07-02 12:25:19 +00:00
|
|
|
CreateVirtioNet(virtio_devices::net::Error),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
/// Cannot create virtio-console device
|
|
|
|
CreateVirtioConsole(io::Error),
|
|
|
|
|
|
|
|
/// Cannot create virtio-rng device
|
|
|
|
CreateVirtioRng(io::Error),
|
|
|
|
|
|
|
|
/// Cannot create virtio-fs device
|
2020-07-02 12:25:19 +00:00
|
|
|
CreateVirtioFs(virtio_devices::vhost_user::Error),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-06-04 19:19:24 +00:00
|
|
|
/// Virtio-fs device was created without a socket.
|
2020-04-14 09:21:24 +00:00
|
|
|
NoVirtioFsSock,
|
|
|
|
|
2019-09-11 03:16:41 +00:00
|
|
|
/// Cannot create vhost-user-blk device
|
2020-07-02 12:25:19 +00:00
|
|
|
CreateVhostUserBlk(virtio_devices::vhost_user::Error),
|
2019-09-11 03:16:41 +00:00
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
/// Cannot create virtio-pmem device
|
|
|
|
CreateVirtioPmem(io::Error),
|
|
|
|
|
2022-03-11 11:31:25 +00:00
|
|
|
/// Cannot create vDPA device
|
|
|
|
CreateVdpa(virtio_devices::vdpa::Error),
|
|
|
|
|
2019-09-04 18:14:54 +00:00
|
|
|
/// Cannot create virtio-vsock device
|
|
|
|
CreateVirtioVsock(io::Error),
|
|
|
|
|
2022-08-15 15:34:54 +00:00
|
|
|
/// Cannot create tpm device
|
|
|
|
CreateTpmDevice(anyhow::Error),
|
|
|
|
|
2022-03-11 11:31:25 +00:00
|
|
|
/// Failed to convert Path to &str for the vDPA device.
|
|
|
|
CreateVdpaConvertPath,
|
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to convert Path to &str for the virtio-vsock device.
|
2019-09-04 21:19:16 +00:00
|
|
|
CreateVsockConvertPath,
|
|
|
|
|
|
|
|
/// Cannot create virtio-vsock backend
|
2020-07-02 12:25:19 +00:00
|
|
|
CreateVsockBackend(virtio_devices::vsock::VsockUnixError),
|
2019-09-04 21:19:16 +00:00
|
|
|
|
2019-10-02 20:57:20 +00:00
|
|
|
/// Cannot create virtio-iommu device
|
|
|
|
CreateVirtioIommu(io::Error),
|
|
|
|
|
2020-03-20 03:43:37 +00:00
|
|
|
/// Cannot create virtio-balloon device
|
|
|
|
CreateVirtioBalloon(io::Error),
|
|
|
|
|
2020-09-25 09:39:45 +00:00
|
|
|
/// Cannot create virtio-watchdog device
|
|
|
|
CreateVirtioWatchdog(io::Error),
|
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to parse disk image format
|
2021-01-28 08:11:25 +00:00
|
|
|
DetectImageType(io::Error),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
/// Cannot open qcow disk path
|
|
|
|
QcowDeviceCreate(qcow::Error),
|
|
|
|
|
2021-09-16 19:47:36 +00:00
|
|
|
/// Cannot create serial manager
|
|
|
|
CreateSerialManager(SerialManagerError),
|
|
|
|
|
|
|
|
/// Cannot spawn the serial manager thread
|
|
|
|
SpawnSerialManager(SerialManagerError),
|
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
/// Cannot open tap interface
|
|
|
|
OpenTap(net_util::TapError),
|
|
|
|
|
|
|
|
/// Cannot allocate IRQ.
|
|
|
|
AllocateIrq,
|
|
|
|
|
|
|
|
/// Cannot configure the IRQ.
|
2020-12-04 23:35:29 +00:00
|
|
|
Irq(vmm_sys_util::errno::Error),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
/// Cannot allocate PCI BARs
|
|
|
|
AllocateBars(pci::PciDeviceError),
|
|
|
|
|
2020-03-11 09:05:37 +00:00
|
|
|
/// Could not free the BARs associated with a PCI device.
|
|
|
|
FreePciBars(pci::PciDeviceError),
|
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
/// Cannot register ioevent.
|
2020-06-02 02:29:54 +00:00
|
|
|
RegisterIoevent(anyhow::Error),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-04-08 14:39:50 +00:00
|
|
|
/// Cannot unregister ioevent.
|
2020-06-02 02:29:54 +00:00
|
|
|
UnRegisterIoevent(anyhow::Error),
|
2020-04-08 14:39:50 +00:00
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
/// Cannot create virtio device
|
2022-11-23 14:01:16 +00:00
|
|
|
VirtioDevice(virtio_devices::transport::VirtioPciDeviceError),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
/// Cannot add PCI device
|
|
|
|
AddPciDevice(pci::PciRootError),
|
|
|
|
|
|
|
|
/// Cannot open persistent memory file
|
|
|
|
PmemFileOpen(io::Error),
|
|
|
|
|
|
|
|
/// Cannot set persistent memory file size
|
|
|
|
PmemFileSetLen(io::Error),
|
|
|
|
|
|
|
|
/// Cannot find a memory range for persistent memory
|
|
|
|
PmemRangeAllocation,
|
|
|
|
|
|
|
|
/// Cannot find a memory range for virtio-fs
|
|
|
|
FsRangeAllocation,
|
|
|
|
|
|
|
|
/// Error creating serial output file
|
|
|
|
SerialOutputFileOpen(io::Error),
|
|
|
|
|
2024-01-17 21:12:43 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
/// Error creating debug-console output file
|
|
|
|
DebugconOutputFileOpen(io::Error),
|
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
/// Error creating console output file
|
|
|
|
ConsoleOutputFileOpen(io::Error),
|
|
|
|
|
2021-01-14 03:03:53 +00:00
|
|
|
/// Error creating serial pty
|
|
|
|
SerialPtyOpen(io::Error),
|
|
|
|
|
|
|
|
/// Error creating console pty
|
|
|
|
ConsolePtyOpen(io::Error),
|
|
|
|
|
2024-01-17 21:12:43 +00:00
|
|
|
/// Error creating console pty
|
|
|
|
DebugconPtyOpen(io::Error),
|
|
|
|
|
2021-01-14 03:03:53 +00:00
|
|
|
/// Error setting pty raw mode
|
|
|
|
SetPtyRaw(vmm_sys_util::errno::Error),
|
|
|
|
|
|
|
|
/// Error getting pty peer
|
|
|
|
GetPtyPeer(vmm_sys_util::errno::Error),
|
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
/// Cannot create a VFIO device
|
2019-11-08 15:27:05 +00:00
|
|
|
VfioCreate(vfio_ioctls::VfioError),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
/// Cannot create a VFIO PCI device
|
2019-11-08 14:50:39 +00:00
|
|
|
VfioPciCreate(pci::VfioPciError),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
/// Failed to map VFIO MMIO region.
|
2019-11-08 14:50:39 +00:00
|
|
|
VfioMapRegion(pci::VfioPciError),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2021-02-26 10:53:02 +00:00
|
|
|
/// Failed to DMA map VFIO device.
|
2021-09-14 09:02:10 +00:00
|
|
|
VfioDmaMap(vfio_ioctls::VfioError),
|
2021-02-26 10:53:02 +00:00
|
|
|
|
|
|
|
/// Failed to DMA unmap VFIO device.
|
|
|
|
VfioDmaUnmap(pci::VfioPciError),
|
|
|
|
|
2020-07-17 15:20:47 +00:00
|
|
|
/// Failed to create the passthrough device.
|
|
|
|
CreatePassthroughDevice(anyhow::Error),
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
/// Failed to memory map.
|
|
|
|
Mmap(io::Error),
|
|
|
|
|
|
|
|
/// Cannot add legacy device to Bus.
|
2020-09-09 14:30:31 +00:00
|
|
|
BusError(vm_device::BusError),
|
2019-11-05 09:37:34 +00:00
|
|
|
|
|
|
|
/// Failed to allocate IO port
|
2021-03-25 17:01:21 +00:00
|
|
|
AllocateIoPort,
|
2019-11-27 15:28:22 +00:00
|
|
|
|
2021-01-20 15:32:10 +00:00
|
|
|
/// Failed to allocate MMIO address
|
2021-03-25 17:01:21 +00:00
|
|
|
AllocateMmioAddress,
|
2021-01-20 15:32:10 +00:00
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to make hotplug notification
|
2019-11-27 15:28:22 +00:00
|
|
|
HotPlugNotification(io::Error),
|
2019-12-20 16:11:30 +00:00
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Error from a memory manager operation
|
2019-12-20 16:11:30 +00:00
|
|
|
MemoryManager(MemoryManagerError),
|
2020-01-20 14:07:08 +00:00
|
|
|
|
|
|
|
/// Failed to create new interrupt source group.
|
|
|
|
CreateInterruptGroup(io::Error),
|
|
|
|
|
|
|
|
/// Failed to update interrupt source group.
|
|
|
|
UpdateInterruptGroup(io::Error),
|
2020-01-22 21:55:02 +00:00
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to create interrupt controller.
|
2020-05-25 08:27:08 +00:00
|
|
|
CreateInterruptController(interrupt_controller::Error),
|
2020-01-24 14:47:15 +00:00
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to create a new MmapRegion instance.
|
2020-01-24 14:47:15 +00:00
|
|
|
NewMmapRegion(vm_memory::mmap::MmapRegionError),
|
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to clone a File.
|
2020-01-24 14:47:15 +00:00
|
|
|
CloneFile(io::Error),
|
2020-02-04 16:44:12 +00:00
|
|
|
|
|
|
|
/// Failed to create socket file
|
|
|
|
CreateSocketFile(io::Error),
|
|
|
|
|
|
|
|
/// Failed to spawn the network backend
|
|
|
|
SpawnNetBackend(io::Error),
|
2020-02-04 16:44:12 +00:00
|
|
|
|
|
|
|
/// Failed to spawn the block backend
|
|
|
|
SpawnBlockBackend(io::Error),
|
2020-02-28 11:29:43 +00:00
|
|
|
|
|
|
|
/// Missing PCI bus.
|
|
|
|
NoPciBus,
|
2020-03-06 10:34:24 +00:00
|
|
|
|
2020-04-15 16:58:49 +00:00
|
|
|
/// Could not find an available device name.
|
|
|
|
NoAvailableDeviceName,
|
2020-03-06 16:52:40 +00:00
|
|
|
|
|
|
|
/// Missing PCI device.
|
|
|
|
MissingPciDevice,
|
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to remove a PCI device from the PCI bus.
|
2020-03-06 16:52:40 +00:00
|
|
|
RemoveDeviceFromPciBus(pci::PciRootError),
|
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to remove a bus device from the IO bus.
|
2020-09-09 14:30:31 +00:00
|
|
|
RemoveDeviceFromIoBus(vm_device::BusError),
|
2020-03-06 16:52:40 +00:00
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to remove a bus device from the MMIO bus.
|
2020-09-09 14:30:31 +00:00
|
|
|
RemoveDeviceFromMmioBus(vm_device::BusError),
|
2020-03-09 10:49:15 +00:00
|
|
|
|
2020-04-27 14:55:52 +00:00
|
|
|
/// Failed to find the device corresponding to a specific PCI b/d/f.
|
|
|
|
UnknownPciBdf(u32),
|
|
|
|
|
|
|
|
/// Not allowed to remove this type of device from the VM.
|
|
|
|
RemovalNotAllowed(vm_virtio::VirtioDeviceType),
|
|
|
|
|
|
|
|
/// Failed to find device corresponding to the given identifier.
|
|
|
|
UnknownDeviceId(String),
|
2020-03-09 15:09:11 +00:00
|
|
|
|
|
|
|
/// Failed to find an available PCI device ID.
|
|
|
|
NextPciDeviceId(pci::PciRootError),
|
2020-03-09 15:32:27 +00:00
|
|
|
|
2020-05-11 16:09:00 +00:00
|
|
|
/// Could not reserve the PCI device ID.
|
|
|
|
GetPciDeviceId(pci::PciRootError),
|
|
|
|
|
2020-03-09 15:32:27 +00:00
|
|
|
/// Could not give the PCI device ID back.
|
|
|
|
PutPciDeviceId(pci::PciRootError),
|
2020-03-11 10:23:42 +00:00
|
|
|
|
2020-03-23 11:10:26 +00:00
|
|
|
/// No disk path was specified when one was expected
|
2020-03-13 10:25:17 +00:00
|
|
|
NoDiskPath,
|
2020-03-23 11:10:26 +00:00
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to update guest memory for virtio device.
|
2020-07-02 12:25:19 +00:00
|
|
|
UpdateMemoryForVirtioDevice(virtio_devices::Error),
|
2020-03-04 02:16:27 +00:00
|
|
|
|
|
|
|
/// Cannot create virtio-mem device
|
|
|
|
CreateVirtioMem(io::Error),
|
|
|
|
|
|
|
|
/// Cannot find a memory range for virtio-mem memory
|
|
|
|
VirtioMemRangeAllocation,
|
2020-03-26 13:53:43 +00:00
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to update guest memory for VFIO PCI device.
|
2021-09-14 09:02:10 +00:00
|
|
|
UpdateMemoryForVfioPciDevice(vfio_ioctls::VfioError),
|
2020-04-24 15:58:03 +00:00
|
|
|
|
|
|
|
/// Trying to use a directory for pmem but no size specified
|
|
|
|
PmemWithDirectorySizeMissing,
|
|
|
|
|
|
|
|
/// Trying to use a size that is not multiple of 2MiB
|
|
|
|
PmemSizeNotAligned,
|
2020-04-27 17:12:00 +00:00
|
|
|
|
|
|
|
/// Could not find the node in the device tree.
|
|
|
|
MissingNode,
|
2020-04-29 10:02:54 +00:00
|
|
|
|
2020-04-29 13:18:04 +00:00
|
|
|
/// Resource was already found.
|
|
|
|
ResourceAlreadyExists,
|
|
|
|
|
2021-07-26 13:18:11 +00:00
|
|
|
/// Expected resources for virtio-pmem could not be found.
|
|
|
|
MissingVirtioPmemResources,
|
2020-05-11 16:09:00 +00:00
|
|
|
|
|
|
|
/// Missing PCI b/d/f from the DeviceNode.
|
|
|
|
MissingDeviceNodePciBdf,
|
2020-07-17 15:28:12 +00:00
|
|
|
|
|
|
|
/// No support for device passthrough
|
|
|
|
NoDevicePassthroughSupport,
|
2020-10-14 10:04:42 +00:00
|
|
|
|
2023-06-08 19:41:37 +00:00
|
|
|
/// No socket option support for console device
|
|
|
|
NoSocketOptionSupportForConsoleDevice,
|
|
|
|
|
2020-10-14 10:04:42 +00:00
|
|
|
/// Failed to resize virtio-balloon
|
|
|
|
VirtioBalloonResize(virtio_devices::balloon::Error),
|
|
|
|
|
|
|
|
/// Missing virtio-balloon, can't proceed as expected.
|
|
|
|
MissingVirtioBalloon,
|
2021-01-13 10:04:33 +00:00
|
|
|
|
2021-09-14 09:02:10 +00:00
|
|
|
/// Missing virtual IOMMU device
|
|
|
|
MissingVirtualIommu,
|
|
|
|
|
2021-01-13 10:04:33 +00:00
|
|
|
/// Failed to do power button notification
|
|
|
|
PowerButtonNotification(io::Error),
|
2021-01-28 08:11:25 +00:00
|
|
|
|
2021-03-07 13:39:11 +00:00
|
|
|
/// Failed to do AArch64 GPIO power button notification
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
2021-03-25 17:01:21 +00:00
|
|
|
AArch64PowerButtonNotification(devices::legacy::GpioDeviceError),
|
2021-03-07 13:39:11 +00:00
|
|
|
|
2021-01-28 08:11:25 +00:00
|
|
|
/// Failed to set O_DIRECT flag to file descriptor
|
|
|
|
SetDirectIo,
|
|
|
|
|
|
|
|
/// Failed to create FixedVhdDiskAsync
|
|
|
|
CreateFixedVhdDiskAsync(io::Error),
|
2021-01-28 15:23:03 +00:00
|
|
|
|
|
|
|
/// Failed to create FixedVhdDiskSync
|
|
|
|
CreateFixedVhdDiskSync(io::Error),
|
2021-03-03 10:59:53 +00:00
|
|
|
|
2021-08-11 22:12:48 +00:00
|
|
|
/// Failed to create QcowDiskSync
|
|
|
|
CreateQcowDiskSync(qcow::Error),
|
|
|
|
|
2021-07-26 15:51:03 +00:00
|
|
|
/// Failed to create FixedVhdxDiskSync
|
2023-07-12 02:24:28 +00:00
|
|
|
CreateFixedVhdxDiskSync(vhdx::VhdxError),
|
2021-07-26 15:51:03 +00:00
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to add DMA mapping handler to virtio-mem device.
|
2021-03-03 10:59:53 +00:00
|
|
|
AddDmaMappingHandlerVirtioMem(virtio_devices::mem::Error),
|
|
|
|
|
2021-11-19 05:04:40 +00:00
|
|
|
/// Failed to remove DMA mapping handler from virtio-mem device.
|
2021-03-03 10:59:53 +00:00
|
|
|
RemoveDmaMappingHandlerVirtioMem(virtio_devices::mem::Error),
|
2021-06-09 13:43:36 +00:00
|
|
|
|
2021-08-11 10:19:33 +00:00
|
|
|
/// Failed to create vfio-user client
|
|
|
|
VfioUserCreateClient(vfio_user::Error),
|
|
|
|
|
2021-06-09 13:43:36 +00:00
|
|
|
/// Failed to create VFIO user device
|
|
|
|
VfioUserCreate(VfioUserPciDeviceError),
|
|
|
|
|
|
|
|
/// Failed to map region from VFIO user device into guest
|
|
|
|
VfioUserMapRegion(VfioUserPciDeviceError),
|
|
|
|
|
|
|
|
/// Failed to DMA map VFIO user device.
|
|
|
|
VfioUserDmaMap(VfioUserPciDeviceError),
|
|
|
|
|
|
|
|
/// Failed to DMA unmap VFIO user device.
|
|
|
|
VfioUserDmaUnmap(VfioUserPciDeviceError),
|
|
|
|
|
|
|
|
/// Failed to update memory mappings for VFIO user device
|
|
|
|
UpdateMemoryForVfioUserPciDevice(VfioUserPciDeviceError),
|
2021-11-17 17:30:42 +00:00
|
|
|
|
|
|
|
/// Cannot duplicate file descriptor
|
|
|
|
DupFd(vmm_sys_util::errno::Error),
|
2022-03-10 14:26:49 +00:00
|
|
|
|
|
|
|
/// Failed to DMA map virtio device.
|
|
|
|
VirtioDmaMap(std::io::Error),
|
|
|
|
|
|
|
|
/// Failed to DMA unmap virtio device.
|
|
|
|
VirtioDmaUnmap(std::io::Error),
|
2022-03-18 17:41:08 +00:00
|
|
|
|
|
|
|
/// Cannot hotplug device behind vIOMMU
|
|
|
|
InvalidIommuHotplug,
|
2022-04-05 02:52:22 +00:00
|
|
|
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
/// Invalid identifier as it is not unique.
|
|
|
|
IdentifierNotUnique(String),
|
2022-05-04 09:36:26 +00:00
|
|
|
|
|
|
|
/// Invalid identifier
|
|
|
|
InvalidIdentifier(String),
|
2022-05-17 14:33:33 +00:00
|
|
|
|
|
|
|
/// Error activating virtio device
|
|
|
|
VirtioActivate(ActivateError),
|
2022-10-18 15:14:43 +00:00
|
|
|
|
|
|
|
/// Failed retrieving device state from snapshot
|
|
|
|
RestoreGetState(MigratableError),
|
2023-06-15 04:02:45 +00:00
|
|
|
|
|
|
|
/// Cannot create a PvPanic device
|
|
|
|
PvPanicCreate(devices::pvpanic::PvPanicError),
|
2023-12-07 19:45:08 +00:00
|
|
|
|
|
|
|
/// Cannot create a RateLimiterGroup
|
|
|
|
RateLimiterGroupCreate(rate_limiter::group::Error),
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
2023-12-07 19:45:08 +00:00
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
pub type DeviceManagerResult<T> = result::Result<T, DeviceManagerError>;
|
|
|
|
|
2021-01-20 16:12:02 +00:00
|
|
|
const DEVICE_MANAGER_ACPI_SIZE: usize = 0x10;
|
|
|
|
|
2021-01-14 03:03:53 +00:00
|
|
|
const TIOCSPTLCK: libc::c_int = 0x4004_5431;
|
|
|
|
const TIOCGTPEER: libc::c_int = 0x5441;
|
|
|
|
|
2022-08-25 08:50:05 +00:00
|
|
|
pub fn create_pty() -> io::Result<(File, File, PathBuf)> {
|
2021-01-14 03:03:53 +00:00
|
|
|
// Try to use /dev/pts/ptmx first then fall back to /dev/ptmx
|
|
|
|
// This is done to try and use the devpts filesystem that
|
|
|
|
// could be available for use in the process's namespace first.
|
|
|
|
// Ideally these are all the same file though but different
|
|
|
|
// kernels could have things setup differently.
|
|
|
|
// See https://www.kernel.org/doc/Documentation/filesystems/devpts.txt
|
|
|
|
// for further details.
|
2021-08-31 15:07:25 +00:00
|
|
|
|
2022-08-25 08:50:05 +00:00
|
|
|
let custom_flags = libc::O_NONBLOCK;
|
2021-01-14 03:03:53 +00:00
|
|
|
let main = match OpenOptions::new()
|
|
|
|
.read(true)
|
|
|
|
.write(true)
|
2021-08-31 15:07:25 +00:00
|
|
|
.custom_flags(custom_flags)
|
2021-01-14 03:03:53 +00:00
|
|
|
.open("/dev/pts/ptmx")
|
|
|
|
{
|
|
|
|
Ok(f) => f,
|
|
|
|
_ => OpenOptions::new()
|
|
|
|
.read(true)
|
|
|
|
.write(true)
|
2021-08-31 15:07:25 +00:00
|
|
|
.custom_flags(custom_flags)
|
2021-01-14 03:03:53 +00:00
|
|
|
.open("/dev/ptmx")?,
|
|
|
|
};
|
|
|
|
let mut unlock: libc::c_ulong = 0;
|
2021-11-17 17:06:47 +00:00
|
|
|
// SAFETY: FFI call into libc, trivially safe
|
2022-12-07 16:03:08 +00:00
|
|
|
unsafe { libc::ioctl(main.as_raw_fd(), TIOCSPTLCK as _, &mut unlock) };
|
2021-01-14 03:03:53 +00:00
|
|
|
|
2023-08-31 13:00:19 +00:00
|
|
|
// SAFETY: FFI call into libc, trivially safe
|
2021-01-14 03:03:53 +00:00
|
|
|
let sub_fd = unsafe {
|
|
|
|
libc::ioctl(
|
|
|
|
main.as_raw_fd(),
|
2022-12-07 16:03:08 +00:00
|
|
|
TIOCGTPEER as _,
|
2021-01-14 03:03:53 +00:00
|
|
|
libc::O_NOCTTY | libc::O_RDWR,
|
|
|
|
)
|
|
|
|
};
|
|
|
|
if sub_fd == -1 {
|
|
|
|
return vmm_sys_util::errno::errno_result().map_err(|e| e.into());
|
|
|
|
}
|
|
|
|
|
2022-12-14 11:41:15 +00:00
|
|
|
let proc_path = PathBuf::from(format!("/proc/self/fd/{sub_fd}"));
|
2021-01-14 03:03:53 +00:00
|
|
|
let path = read_link(proc_path)?;
|
|
|
|
|
2021-11-17 17:06:47 +00:00
|
|
|
// SAFETY: sub_fd is checked to be valid before being wrapped in File
|
2021-01-14 03:03:53 +00:00
|
|
|
Ok((main, unsafe { File::from_raw_fd(sub_fd) }, path))
|
|
|
|
}
|
|
|
|
|
2020-01-29 15:53:12 +00:00
|
|
|
#[derive(Default)]
|
2019-09-06 15:42:41 +00:00
|
|
|
pub struct Console {
|
2021-09-02 15:56:37 +00:00
|
|
|
console_resizer: Option<Arc<virtio_devices::ConsoleResizer>>,
|
2019-09-06 15:42:41 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
impl Console {
|
2023-02-27 12:07:07 +00:00
|
|
|
pub fn need_resize(&self) -> bool {
|
|
|
|
if let Some(_resizer) = self.console_resizer.as_ref() {
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
false
|
|
|
|
}
|
|
|
|
|
2021-09-10 11:27:08 +00:00
|
|
|
pub fn update_console_size(&self) {
|
2021-09-02 15:56:37 +00:00
|
|
|
if let Some(resizer) = self.console_resizer.as_ref() {
|
2021-09-10 11:27:08 +00:00
|
|
|
resizer.update_console_size()
|
2019-09-06 15:42:41 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-10-04 15:08:43 +00:00
|
|
|
pub(crate) struct AddressManager {
|
|
|
|
pub(crate) allocator: Arc<Mutex<SystemAllocator>>,
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2021-10-04 15:08:43 +00:00
|
|
|
pub(crate) io_bus: Arc<Bus>,
|
|
|
|
pub(crate) mmio_bus: Arc<Bus>,
|
2022-11-06 10:51:12 +00:00
|
|
|
pub(crate) vm: Arc<dyn hypervisor::Vm>,
|
2020-05-12 15:25:30 +00:00
|
|
|
device_tree: Arc<Mutex<DeviceTree>>,
|
2023-11-14 19:31:57 +00:00
|
|
|
pci_mmio32_allocators: Vec<Arc<Mutex<AddressAllocator>>>,
|
|
|
|
pci_mmio64_allocators: Vec<Arc<Mutex<AddressAllocator>>>,
|
2019-10-28 16:53:13 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
impl DeviceRelocation for AddressManager {
|
|
|
|
fn move_bar(
|
|
|
|
&self,
|
|
|
|
old_base: u64,
|
|
|
|
new_base: u64,
|
|
|
|
len: u64,
|
|
|
|
pci_dev: &mut dyn PciDevice,
|
|
|
|
region_type: PciBarRegionType,
|
2019-10-29 01:15:08 +00:00
|
|
|
) -> std::result::Result<(), std::io::Error> {
|
2019-10-28 16:53:13 +00:00
|
|
|
match region_type {
|
2021-03-25 17:01:21 +00:00
|
|
|
PciBarRegionType::IoRegion => {
|
2020-06-03 08:59:35 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
{
|
|
|
|
// Update system allocator
|
|
|
|
self.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.free_io_addresses(GuestAddress(old_base), len as GuestUsize);
|
2019-10-29 01:15:08 +00:00
|
|
|
|
2020-06-03 08:59:35 +00:00
|
|
|
self.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_io_addresses(
|
|
|
|
Some(GuestAddress(new_base)),
|
|
|
|
len as GuestUsize,
|
|
|
|
None,
|
|
|
|
)
|
|
|
|
.ok_or_else(|| {
|
|
|
|
io::Error::new(io::ErrorKind::Other, "failed allocating new IO range")
|
|
|
|
})?;
|
2019-10-28 16:53:13 +00:00
|
|
|
|
2020-06-03 08:59:35 +00:00
|
|
|
// Update PIO bus
|
|
|
|
self.io_bus
|
|
|
|
.update_range(old_base, len, new_base, len)
|
|
|
|
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
|
|
|
|
}
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
error!("I/O region is not supported");
|
2019-10-28 16:53:13 +00:00
|
|
|
}
|
|
|
|
PciBarRegionType::Memory32BitRegion | PciBarRegionType::Memory64BitRegion => {
|
2023-11-14 19:31:57 +00:00
|
|
|
let allocators = if region_type == PciBarRegionType::Memory32BitRegion {
|
|
|
|
&self.pci_mmio32_allocators
|
2019-10-28 16:53:13 +00:00
|
|
|
} else {
|
2023-11-14 19:31:57 +00:00
|
|
|
&self.pci_mmio64_allocators
|
|
|
|
};
|
|
|
|
|
|
|
|
// Find the specific allocator that this BAR was allocated from and use it for new one
|
|
|
|
for allocator in allocators {
|
|
|
|
let allocator_base = allocator.lock().unwrap().base();
|
|
|
|
let allocator_end = allocator.lock().unwrap().end();
|
|
|
|
|
|
|
|
if old_base >= allocator_base.0 && old_base <= allocator_end.0 {
|
|
|
|
allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.free(GuestAddress(old_base), len as GuestUsize);
|
|
|
|
|
|
|
|
allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate(Some(GuestAddress(new_base)), len as GuestUsize, Some(len))
|
|
|
|
.ok_or_else(|| {
|
|
|
|
io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
|
|
|
"failed allocating new MMIO range",
|
2021-10-15 10:38:06 +00:00
|
|
|
)
|
2023-11-14 19:31:57 +00:00
|
|
|
})?;
|
2020-03-11 09:12:37 +00:00
|
|
|
|
2023-11-14 19:31:57 +00:00
|
|
|
break;
|
2021-10-15 10:38:06 +00:00
|
|
|
}
|
2019-10-28 16:53:13 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// Update MMIO bus
|
|
|
|
self.mmio_bus
|
|
|
|
.update_range(old_base, len, new_base, len)
|
2019-10-29 01:15:08 +00:00
|
|
|
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
|
2019-10-28 16:53:13 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-04-07 16:16:19 +00:00
|
|
|
// Update the device_tree resources associated with the device
|
|
|
|
if let Some(id) = pci_dev.id() {
|
|
|
|
if let Some(node) = self.device_tree.lock().unwrap().get_mut(&id) {
|
2020-05-12 15:25:30 +00:00
|
|
|
let mut resource_updated = false;
|
|
|
|
for resource in node.resources.iter_mut() {
|
2022-04-15 09:27:53 +00:00
|
|
|
if let Resource::PciBar { base, type_, .. } = resource {
|
|
|
|
if PciBarRegionType::from(*type_) == region_type && *base == old_base {
|
|
|
|
*base = new_base;
|
|
|
|
resource_updated = true;
|
|
|
|
break;
|
2020-05-12 15:25:30 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if !resource_updated {
|
|
|
|
return Err(io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
|
|
|
format!(
|
2022-12-14 11:41:15 +00:00
|
|
|
"Couldn't find a resource with base 0x{old_base:x} for device {id}"
|
2020-05-12 15:25:30 +00:00
|
|
|
),
|
|
|
|
));
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
return Err(io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
2022-12-14 11:41:15 +00:00
|
|
|
format!("Couldn't find device {id} from device tree"),
|
2020-05-12 15:25:30 +00:00
|
|
|
));
|
|
|
|
}
|
2022-04-07 16:16:19 +00:00
|
|
|
}
|
2020-05-12 15:25:30 +00:00
|
|
|
|
2022-04-07 16:16:19 +00:00
|
|
|
let any_dev = pci_dev.as_any();
|
|
|
|
if let Some(virtio_pci_dev) = any_dev.downcast_ref::<VirtioPciDevice>() {
|
2019-10-30 16:13:29 +00:00
|
|
|
let bar_addr = virtio_pci_dev.config_bar_addr();
|
|
|
|
if bar_addr == new_base {
|
2019-10-30 18:03:02 +00:00
|
|
|
for (event, addr) in virtio_pci_dev.ioeventfds(old_base) {
|
2019-10-30 16:41:08 +00:00
|
|
|
let io_addr = IoEventAddress::Mmio(addr);
|
2020-07-03 09:16:49 +00:00
|
|
|
self.vm.unregister_ioevent(event, &io_addr).map_err(|e| {
|
|
|
|
io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
2022-12-14 11:41:15 +00:00
|
|
|
format!("failed to unregister ioevent: {e:?}"),
|
2020-07-03 09:16:49 +00:00
|
|
|
)
|
|
|
|
})?;
|
2019-10-30 16:41:08 +00:00
|
|
|
}
|
2019-10-30 18:03:02 +00:00
|
|
|
for (event, addr) in virtio_pci_dev.ioeventfds(new_base) {
|
2019-10-30 16:13:29 +00:00
|
|
|
let io_addr = IoEventAddress::Mmio(addr);
|
2020-07-03 09:16:49 +00:00
|
|
|
self.vm
|
2020-06-22 16:31:42 +00:00
|
|
|
.register_ioevent(event, &io_addr, None)
|
|
|
|
.map_err(|e| {
|
|
|
|
io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
2022-12-14 11:41:15 +00:00
|
|
|
format!("failed to register ioevent: {e:?}"),
|
2020-06-22 16:31:42 +00:00
|
|
|
)
|
|
|
|
})?;
|
2019-10-30 16:13:29 +00:00
|
|
|
}
|
2020-04-17 17:30:33 +00:00
|
|
|
} else {
|
|
|
|
let virtio_dev = virtio_pci_dev.virtio_device();
|
|
|
|
let mut virtio_dev = virtio_dev.lock().unwrap();
|
|
|
|
if let Some(mut shm_regions) = virtio_dev.get_shm_regions() {
|
|
|
|
if shm_regions.addr.raw_value() == old_base {
|
2020-07-16 10:47:02 +00:00
|
|
|
let mem_region = self.vm.make_user_memory_region(
|
2020-07-04 11:38:17 +00:00
|
|
|
shm_regions.mem_slot,
|
|
|
|
old_base,
|
2021-07-03 13:58:39 +00:00
|
|
|
shm_regions.len,
|
2020-07-04 11:38:17 +00:00
|
|
|
shm_regions.host_addr,
|
|
|
|
false,
|
2020-11-11 18:16:39 +00:00
|
|
|
false,
|
2020-07-04 11:38:17 +00:00
|
|
|
);
|
2020-04-17 17:30:33 +00:00
|
|
|
|
2021-07-03 13:58:39 +00:00
|
|
|
self.vm.remove_user_memory_region(mem_region).map_err(|e| {
|
2020-06-22 16:31:42 +00:00
|
|
|
io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
2022-12-14 11:41:15 +00:00
|
|
|
format!("failed to remove user memory region: {e:?}"),
|
2020-06-22 16:31:42 +00:00
|
|
|
)
|
|
|
|
})?;
|
2020-04-17 17:30:33 +00:00
|
|
|
|
|
|
|
// Create new mapping by inserting new region to KVM.
|
2020-07-16 10:47:02 +00:00
|
|
|
let mem_region = self.vm.make_user_memory_region(
|
|
|
|
shm_regions.mem_slot,
|
|
|
|
new_base,
|
|
|
|
shm_regions.len,
|
|
|
|
shm_regions.host_addr,
|
|
|
|
false,
|
2020-11-11 18:16:39 +00:00
|
|
|
false,
|
2020-07-16 10:47:02 +00:00
|
|
|
);
|
2020-04-17 17:30:33 +00:00
|
|
|
|
2021-07-03 13:58:39 +00:00
|
|
|
self.vm.create_user_memory_region(mem_region).map_err(|e| {
|
2020-06-02 02:29:54 +00:00
|
|
|
io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
2022-12-14 11:41:15 +00:00
|
|
|
format!("failed to create user memory regions: {e:?}"),
|
2020-06-02 02:29:54 +00:00
|
|
|
)
|
|
|
|
})?;
|
2020-04-17 17:30:33 +00:00
|
|
|
|
|
|
|
// Update shared memory regions to reflect the new mapping.
|
|
|
|
shm_regions.addr = GuestAddress(new_base);
|
|
|
|
virtio_dev.set_shm_regions(shm_regions).map_err(|e| {
|
|
|
|
io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
2022-12-14 11:41:15 +00:00
|
|
|
format!("failed to update shared memory regions: {e:?}"),
|
2020-04-17 17:30:33 +00:00
|
|
|
)
|
|
|
|
})?;
|
|
|
|
}
|
|
|
|
}
|
2019-10-30 15:15:38 +00:00
|
|
|
}
|
2019-10-28 16:53:13 +00:00
|
|
|
}
|
|
|
|
|
2019-10-29 01:15:08 +00:00
|
|
|
pci_dev.move_bar(old_base, new_base)
|
2019-10-28 16:53:13 +00:00
|
|
|
}
|
2019-10-23 22:14:13 +00:00
|
|
|
}
|
|
|
|
|
2020-04-28 10:08:51 +00:00
|
|
|
#[derive(Serialize, Deserialize)]
|
|
|
|
struct DeviceManagerState {
|
2020-04-30 17:10:31 +00:00
|
|
|
device_tree: DeviceTree,
|
2020-04-28 10:08:51 +00:00
|
|
|
device_id_cnt: Wrapping<usize>,
|
|
|
|
}
|
|
|
|
|
2021-03-04 23:34:45 +00:00
|
|
|
#[derive(Debug)]
|
|
|
|
pub struct PtyPair {
|
|
|
|
pub main: File,
|
|
|
|
pub path: PathBuf,
|
|
|
|
}
|
|
|
|
|
2021-09-16 19:36:46 +00:00
|
|
|
impl Clone for PtyPair {
|
2021-03-04 23:34:45 +00:00
|
|
|
fn clone(&self) -> Self {
|
|
|
|
PtyPair {
|
|
|
|
main: self.main.try_clone().unwrap(),
|
|
|
|
path: self.path.clone(),
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
#[derive(Clone)]
|
|
|
|
pub enum PciDeviceHandle {
|
2021-03-17 17:26:01 +00:00
|
|
|
Vfio(Arc<Mutex<VfioPciDevice>>),
|
|
|
|
Virtio(Arc<Mutex<VirtioPciDevice>>),
|
2021-06-09 13:43:36 +00:00
|
|
|
VfioUser(Arc<Mutex<VfioUserPciDevice>>),
|
2021-03-17 17:26:01 +00:00
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
#[derive(Clone)]
|
|
|
|
struct MetaVirtioDevice {
|
|
|
|
virtio_device: Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: bool,
|
|
|
|
id: String,
|
|
|
|
pci_segment: u16,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: Option<Arc<dyn ExternalDmaMapping>>,
|
2022-03-10 13:05:36 +00:00
|
|
|
}
|
|
|
|
|
2022-07-25 12:36:31 +00:00
|
|
|
#[derive(Default)]
|
|
|
|
pub struct AcpiPlatformAddresses {
|
|
|
|
pub pm_timer_address: Option<GenericAddress>,
|
|
|
|
pub reset_reg_address: Option<GenericAddress>,
|
|
|
|
pub sleep_control_reg_address: Option<GenericAddress>,
|
|
|
|
pub sleep_status_reg_address: Option<GenericAddress>,
|
|
|
|
}
|
|
|
|
|
2019-10-23 22:14:13 +00:00
|
|
|
pub struct DeviceManager {
|
2022-07-20 22:51:15 +00:00
|
|
|
// The underlying hypervisor
|
|
|
|
hypervisor_type: HypervisorType,
|
|
|
|
|
2019-10-23 22:14:13 +00:00
|
|
|
// Manage address space related to devices
|
|
|
|
address_manager: Arc<AddressManager>,
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2019-09-06 15:42:41 +00:00
|
|
|
// Console abstraction
|
|
|
|
console: Arc<Console>,
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2021-01-14 03:03:53 +00:00
|
|
|
// console PTY
|
2021-03-04 23:34:45 +00:00
|
|
|
console_pty: Option<Arc<Mutex<PtyPair>>>,
|
2021-01-14 03:03:53 +00:00
|
|
|
|
|
|
|
// serial PTY
|
2021-03-04 23:34:45 +00:00
|
|
|
serial_pty: Option<Arc<Mutex<PtyPair>>>,
|
2021-01-14 03:03:53 +00:00
|
|
|
|
2024-01-17 21:12:43 +00:00
|
|
|
// debug-console PTY
|
|
|
|
debug_console_pty: Option<Arc<Mutex<PtyPair>>>,
|
|
|
|
|
2021-09-16 19:47:36 +00:00
|
|
|
// Serial Manager
|
|
|
|
serial_manager: Option<Arc<SerialManager>>,
|
|
|
|
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
// pty foreground status,
|
|
|
|
console_resize_pipe: Option<Arc<File>>,
|
|
|
|
|
vmm: reset to the original termios
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-04-27 12:57:28 +00:00
|
|
|
// To restore on exit.
|
|
|
|
original_termios_opt: Arc<Mutex<Option<termios>>>,
|
vmm: only touch the tty flags if it's being used
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-03-22 22:22:46 +00:00
|
|
|
|
2020-05-25 08:27:08 +00:00
|
|
|
// Interrupt controller
|
2020-05-25 08:59:09 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2020-05-25 08:27:08 +00:00
|
|
|
interrupt_controller: Option<Arc<Mutex<ioapic::Ioapic>>>,
|
2020-05-25 08:59:09 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
interrupt_controller: Option<Arc<Mutex<gic::Gic>>>,
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2022-04-28 14:21:14 +00:00
|
|
|
// Things to be added to the commandline (e.g. aarch64 early console)
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
2019-09-11 15:22:00 +00:00
|
|
|
cmdline_additions: Vec<String>,
|
2019-10-02 20:57:20 +00:00
|
|
|
|
2019-11-27 15:28:22 +00:00
|
|
|
// ACPI GED notification device
|
2021-03-25 17:01:21 +00:00
|
|
|
ged_notification_device: Option<Arc<Mutex<devices::AcpiGedDevice>>>,
|
2019-12-06 16:14:32 +00:00
|
|
|
|
|
|
|
// VM configuration
|
|
|
|
config: Arc<Mutex<VmConfig>>,
|
2019-11-18 23:24:31 +00:00
|
|
|
|
2019-12-20 15:25:06 +00:00
|
|
|
// Memory Manager
|
2020-01-31 11:55:30 +00:00
|
|
|
memory_manager: Arc<Mutex<MemoryManager>>,
|
2020-02-14 10:08:14 +00:00
|
|
|
|
2022-11-03 15:10:18 +00:00
|
|
|
// CPU Manager
|
|
|
|
cpu_manager: Arc<Mutex<CpuManager>>,
|
|
|
|
|
2020-02-14 10:08:14 +00:00
|
|
|
// The virtio devices on the system
|
2022-03-10 13:05:36 +00:00
|
|
|
virtio_devices: Vec<MetaVirtioDevice>,
|
2020-02-14 09:55:19 +00:00
|
|
|
|
2020-03-04 14:46:40 +00:00
|
|
|
// List of bus devices
|
|
|
|
// Let the DeviceManager keep strong references to the BusDevice devices.
|
|
|
|
// This allows the IO and MMIO buses to be provided with Weak references,
|
|
|
|
// which prevents cyclic dependencies.
|
|
|
|
bus_devices: Vec<Arc<Mutex<dyn BusDevice>>>,
|
|
|
|
|
2020-04-27 08:37:56 +00:00
|
|
|
// Counter to keep track of the consumed device IDs.
|
|
|
|
device_id_cnt: Wrapping<usize>,
|
|
|
|
|
2021-10-05 13:55:15 +00:00
|
|
|
pci_segments: Vec<PciSegment>,
|
2021-09-17 08:59:30 +00:00
|
|
|
|
2020-05-12 09:49:12 +00:00
|
|
|
#[cfg_attr(target_arch = "aarch64", allow(dead_code))]
|
2020-02-27 17:10:30 +00:00
|
|
|
// MSI Interrupt Manager
|
|
|
|
msi_interrupt_manager: Arc<dyn InterruptManager<GroupConfig = MsiIrqGroupConfig>>,
|
2020-02-27 17:24:47 +00:00
|
|
|
|
2021-03-29 08:23:58 +00:00
|
|
|
#[cfg_attr(feature = "mshv", allow(dead_code))]
|
2021-02-09 08:17:29 +00:00
|
|
|
// Legacy Interrupt Manager
|
|
|
|
legacy_interrupt_manager: Option<Arc<dyn InterruptManager<GroupConfig = LegacyIrqGroupConfig>>>,
|
|
|
|
|
2020-07-17 15:20:47 +00:00
|
|
|
// Passthrough device handle
|
2022-07-21 13:15:15 +00:00
|
|
|
passthrough_device: Option<VfioDeviceFd>,
|
2020-02-27 17:43:47 +00:00
|
|
|
|
2021-09-14 09:02:10 +00:00
|
|
|
// VFIO container
|
|
|
|
// Only one container can be created, therefore it is stored as part of the
|
|
|
|
// DeviceManager to be reused.
|
|
|
|
vfio_container: Option<Arc<VfioContainer>>,
|
|
|
|
|
2020-02-27 17:43:47 +00:00
|
|
|
// Paravirtualized IOMMU
|
2020-07-02 12:25:19 +00:00
|
|
|
iommu_device: Option<Arc<Mutex<virtio_devices::Iommu>>>,
|
2022-03-18 16:57:12 +00:00
|
|
|
iommu_mapping: Option<Arc<IommuMapping>>,
|
2020-02-28 11:29:43 +00:00
|
|
|
|
2021-06-14 13:38:24 +00:00
|
|
|
// PCI information about devices attached to the paravirtualized IOMMU
|
|
|
|
// It contains the virtual IOMMU PCI BDF along with the list of PCI BDF
|
|
|
|
// representing the devices attached to the virtual IOMMU. This is useful
|
|
|
|
// information for filling the ACPI VIOT table.
|
2021-10-18 16:29:42 +00:00
|
|
|
iommu_attached_devices: Option<(PciBdf, Vec<PciBdf>)>,
|
2021-06-14 13:38:24 +00:00
|
|
|
|
2020-04-27 17:12:00 +00:00
|
|
|
// Tree of devices, representing the dependencies between devices.
|
|
|
|
// Useful for introspection, snapshot and restore.
|
2020-05-12 13:53:09 +00:00
|
|
|
device_tree: Arc<Mutex<DeviceTree>>,
|
2020-04-28 17:57:28 +00:00
|
|
|
|
|
|
|
// Exit event
|
|
|
|
exit_evt: EventFd,
|
|
|
|
reset_evt: EventFd,
|
2020-06-09 06:17:42 +00:00
|
|
|
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
2021-03-25 17:01:21 +00:00
|
|
|
id_to_dev_info: HashMap<(DeviceType, String), MmioDeviceInfo>,
|
2020-08-04 02:45:53 +00:00
|
|
|
|
|
|
|
// seccomp action
|
|
|
|
seccomp_action: SeccompAction,
|
2020-09-11 14:15:29 +00:00
|
|
|
|
|
|
|
// List of guest NUMA nodes.
|
|
|
|
numa_nodes: NumaNodes,
|
2020-10-14 10:04:42 +00:00
|
|
|
|
|
|
|
// Possible handle to the virtio-balloon device
|
|
|
|
balloon: Option<Arc<Mutex<virtio_devices::Balloon>>>,
|
2020-11-09 13:29:05 +00:00
|
|
|
|
|
|
|
// Virtio Device activation EventFd to allow the VMM thread to trigger device
|
|
|
|
// activation and thus start the threads from the VMM thread
|
|
|
|
activate_evt: EventFd,
|
2021-01-20 16:12:02 +00:00
|
|
|
|
|
|
|
acpi_address: GuestAddress,
|
2022-03-28 10:53:22 +00:00
|
|
|
|
2021-10-08 09:49:26 +00:00
|
|
|
selected_segment: usize,
|
2021-03-03 10:59:53 +00:00
|
|
|
|
2021-11-18 07:03:26 +00:00
|
|
|
// Possible handle to the virtio-mem device
|
2021-03-03 10:59:53 +00:00
|
|
|
virtio_mem_devices: Vec<Arc<Mutex<virtio_devices::Mem>>>,
|
2021-03-07 12:11:07 +00:00
|
|
|
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
// GPIO device for AArch64
|
2021-03-25 17:01:21 +00:00
|
|
|
gpio_device: Option<Arc<Mutex<devices::legacy::Gpio>>>,
|
2021-07-20 07:59:32 +00:00
|
|
|
|
2023-06-15 04:02:45 +00:00
|
|
|
// pvpanic device
|
|
|
|
pvpanic_device: Option<Arc<Mutex<devices::PvPanicDevice>>>,
|
|
|
|
|
2021-07-20 07:59:32 +00:00
|
|
|
// Flag to force setting the iommu on virtio devices
|
|
|
|
force_iommu: bool,
|
2021-08-06 13:39:09 +00:00
|
|
|
|
vmm: Cache whether io_uring is supported in DeviceManager
Probing for whether the io_uring is supported is time consuming so cache
this value if it is known to reduce the cost for secondary block devices
that are added.
Before:
cloud-hypervisor: 3.988896ms: <vmm> INFO:vmm/src/device_manager.rs:1901 -- Creating virtio-block device: DiskConfig { path: Some("/home/rob/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk0"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.129591ms: <vmm> INFO:vmm/src/device_manager.rs:1983 -- Using asynchronous RAW disk file (io_uring)
cloud-hypervisor: 14.159853ms: <vmm> INFO:vmm/src/device_manager.rs:1901 -- Creating virtio-block device: DiskConfig { path: Some("/tmp/disk"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk1"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 22.110281ms: <vmm> INFO:vmm/src/device_manager.rs:1983 -- Using asynchronous RAW disk file (io_uring)
After:
cloud-hypervisor: 4.880411ms: <vmm> INFO:vmm/src/device_manager.rs:1916 -- Creating virtio-block device: DiskConfig { path: Some("/home/rob/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk0"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.105123ms: <vmm> INFO:vmm/src/device_manager.rs:1998 -- Using asynchronous RAW disk file (io_uring)
cloud-hypervisor: 14.134837ms: <vmm> INFO:vmm/src/device_manager.rs:1916 -- Creating virtio-block device: DiskConfig { path: Some("/tmp/disk"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk1"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.221869ms: <vmm> INFO:vmm/src/device_manager.rs:1998 -- Using asynchronous RAW disk file (io_uring)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-11-12 10:38:51 +00:00
|
|
|
// io_uring availability if detected
|
|
|
|
io_uring_supported: Option<bool>,
|
2022-04-29 07:58:22 +00:00
|
|
|
|
2023-10-19 02:59:22 +00:00
|
|
|
// aio availability if detected
|
|
|
|
aio_supported: Option<bool>,
|
|
|
|
|
2022-04-29 07:58:22 +00:00
|
|
|
// List of unique identifiers provided at boot through the configuration.
|
|
|
|
boot_id_list: BTreeSet<String>,
|
2022-05-04 15:36:57 +00:00
|
|
|
|
|
|
|
// Start time of the VM
|
|
|
|
timestamp: Instant,
|
2022-05-17 14:33:33 +00:00
|
|
|
|
|
|
|
// Pending activations
|
|
|
|
pending_activations: Arc<Mutex<Vec<VirtioPciDeviceActivator>>>,
|
2022-07-25 12:36:31 +00:00
|
|
|
|
|
|
|
// Addresses for ACPI platform devices e.g. ACPI PM timer, sleep/reset registers
|
|
|
|
acpi_platform_addresses: AcpiPlatformAddresses,
|
2022-10-18 15:14:43 +00:00
|
|
|
|
|
|
|
snapshot: Option<Snapshot>,
|
2023-12-07 19:45:08 +00:00
|
|
|
|
|
|
|
rate_limit_groups: HashMap<String, Arc<RateLimiterGroup>>,
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
impl DeviceManager {
|
2020-09-11 14:15:29 +00:00
|
|
|
#[allow(clippy::too_many_arguments)]
|
2019-09-04 13:55:14 +00:00
|
|
|
pub fn new(
|
2022-11-03 15:10:18 +00:00
|
|
|
#[cfg(target_arch = "x86_64")] io_bus: Arc<Bus>,
|
|
|
|
mmio_bus: Arc<Bus>,
|
2022-07-20 22:51:15 +00:00
|
|
|
hypervisor_type: HypervisorType,
|
2020-07-03 09:16:49 +00:00
|
|
|
vm: Arc<dyn hypervisor::Vm>,
|
2020-01-31 16:23:49 +00:00
|
|
|
config: Arc<Mutex<VmConfig>>,
|
2020-01-31 11:55:30 +00:00
|
|
|
memory_manager: Arc<Mutex<MemoryManager>>,
|
2022-11-03 15:10:18 +00:00
|
|
|
cpu_manager: Arc<Mutex<CpuManager>>,
|
|
|
|
exit_evt: EventFd,
|
|
|
|
reset_evt: EventFd,
|
2020-08-04 02:45:53 +00:00
|
|
|
seccomp_action: SeccompAction,
|
2022-03-28 10:53:22 +00:00
|
|
|
numa_nodes: NumaNodes,
|
2020-11-09 13:29:05 +00:00
|
|
|
activate_evt: &EventFd,
|
2021-07-20 07:59:32 +00:00
|
|
|
force_iommu: bool,
|
2022-04-29 07:58:22 +00:00
|
|
|
boot_id_list: BTreeSet<String>,
|
2022-05-04 15:36:57 +00:00
|
|
|
timestamp: Instant,
|
2022-10-18 15:14:43 +00:00
|
|
|
snapshot: Option<Snapshot>,
|
2022-11-03 15:10:18 +00:00
|
|
|
dynamic: bool,
|
2020-02-27 09:29:03 +00:00
|
|
|
) -> DeviceManagerResult<Arc<Mutex<Self>>> {
|
2022-09-16 15:56:36 +00:00
|
|
|
trace_scoped!("DeviceManager::new");
|
|
|
|
|
2022-11-28 16:40:17 +00:00
|
|
|
let (device_tree, device_id_cnt) = if let Some(snapshot) = snapshot.as_ref() {
|
2022-12-02 13:37:49 +00:00
|
|
|
let state: DeviceManagerState = snapshot.to_state().unwrap();
|
2022-11-28 16:40:17 +00:00
|
|
|
(
|
|
|
|
Arc::new(Mutex::new(state.device_tree.clone())),
|
|
|
|
state.device_id_cnt,
|
|
|
|
)
|
|
|
|
} else {
|
|
|
|
(Arc::new(Mutex::new(DeviceTree::new())), Wrapping(0))
|
|
|
|
};
|
2020-05-12 15:25:30 +00:00
|
|
|
|
2021-10-15 10:38:06 +00:00
|
|
|
let num_pci_segments =
|
|
|
|
if let Some(platform_config) = config.lock().unwrap().platform.as_ref() {
|
|
|
|
platform_config.num_pci_segments
|
|
|
|
} else {
|
|
|
|
1
|
|
|
|
};
|
|
|
|
|
2023-11-14 19:31:57 +00:00
|
|
|
let create_mmio_allocators = |start, end, num_pci_segments, alignment| {
|
|
|
|
// Start each PCI segment mmio range on an aligned boundary
|
|
|
|
let pci_segment_mmio_size =
|
|
|
|
(end - start + 1) / (alignment * num_pci_segments as u64) * alignment;
|
2021-10-15 10:38:06 +00:00
|
|
|
|
2023-11-14 19:31:57 +00:00
|
|
|
let mut mmio_allocators = vec![];
|
|
|
|
for i in 0..num_pci_segments as u64 {
|
|
|
|
let mmio_start = start + i * pci_segment_mmio_size;
|
|
|
|
let allocator = Arc::new(Mutex::new(
|
|
|
|
AddressAllocator::new(GuestAddress(mmio_start), pci_segment_mmio_size).unwrap(),
|
|
|
|
));
|
|
|
|
mmio_allocators.push(allocator)
|
|
|
|
}
|
2021-10-15 10:38:06 +00:00
|
|
|
|
2023-11-14 19:31:57 +00:00
|
|
|
mmio_allocators
|
|
|
|
};
|
|
|
|
|
|
|
|
let start_of_mmio32_area = layout::MEM_32BIT_DEVICES_START.0;
|
|
|
|
let end_of_mmio32_area = layout::MEM_32BIT_DEVICES_START.0 + layout::MEM_32BIT_DEVICES_SIZE;
|
|
|
|
let pci_mmio32_allocators = create_mmio_allocators(
|
|
|
|
start_of_mmio32_area,
|
|
|
|
end_of_mmio32_area,
|
|
|
|
num_pci_segments,
|
|
|
|
4 << 10,
|
|
|
|
);
|
|
|
|
|
|
|
|
let start_of_mmio64_area = memory_manager.lock().unwrap().start_of_device_area().0;
|
|
|
|
let end_of_mmio64_area = memory_manager.lock().unwrap().end_of_device_area().0;
|
|
|
|
let pci_mmio64_allocators = create_mmio_allocators(
|
|
|
|
start_of_mmio64_area,
|
|
|
|
end_of_mmio64_area,
|
|
|
|
num_pci_segments,
|
|
|
|
4 << 30,
|
|
|
|
);
|
2021-10-15 10:38:06 +00:00
|
|
|
|
2019-10-23 22:14:13 +00:00
|
|
|
let address_manager = Arc::new(AddressManager {
|
2020-03-16 17:58:23 +00:00
|
|
|
allocator: memory_manager.lock().unwrap().allocator(),
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2022-11-03 15:10:18 +00:00
|
|
|
io_bus,
|
|
|
|
mmio_bus,
|
2020-07-03 09:16:49 +00:00
|
|
|
vm: vm.clone(),
|
2020-05-12 15:25:30 +00:00
|
|
|
device_tree: Arc::clone(&device_tree),
|
2023-11-14 19:31:57 +00:00
|
|
|
pci_mmio32_allocators,
|
|
|
|
pci_mmio64_allocators,
|
2019-10-23 22:14:13 +00:00
|
|
|
});
|
|
|
|
|
2020-02-04 09:59:53 +00:00
|
|
|
// First we create the MSI interrupt manager, the legacy one is created
|
|
|
|
// later, after the IOAPIC device creation.
|
|
|
|
// The reason we create the MSI one first is because the IOAPIC needs it,
|
|
|
|
// and then the legacy interrupt manager needs an IOAPIC. So we're
|
|
|
|
// handling a linear dependency chain:
|
|
|
|
// msi_interrupt_manager <- IOAPIC <- legacy_interrupt_manager.
|
2020-02-04 11:04:10 +00:00
|
|
|
let msi_interrupt_manager: Arc<dyn InterruptManager<GroupConfig = MsiIrqGroupConfig>> =
|
2020-07-29 12:09:52 +00:00
|
|
|
Arc::new(MsiInterruptManager::new(
|
2020-01-22 17:39:46 +00:00
|
|
|
Arc::clone(&address_manager.allocator),
|
2020-07-03 09:16:49 +00:00
|
|
|
vm,
|
2020-01-22 17:39:46 +00:00
|
|
|
));
|
|
|
|
|
2021-01-20 16:12:02 +00:00
|
|
|
let acpi_address = address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-10-29 09:58:41 +00:00
|
|
|
.allocate_platform_mmio_addresses(None, DEVICE_MANAGER_ACPI_SIZE as u64, None)
|
2021-03-25 17:01:21 +00:00
|
|
|
.ok_or(DeviceManagerError::AllocateIoPort)?;
|
2021-10-05 10:07:14 +00:00
|
|
|
|
2021-10-11 15:35:52 +00:00
|
|
|
let mut pci_irq_slots = [0; 32];
|
|
|
|
PciSegment::reserve_legacy_interrupts_for_pci_devices(
|
|
|
|
&address_manager,
|
|
|
|
&mut pci_irq_slots,
|
|
|
|
)?;
|
|
|
|
|
2021-10-11 14:55:35 +00:00
|
|
|
let mut pci_segments = vec![PciSegment::new_default_segment(
|
|
|
|
&address_manager,
|
2023-11-14 19:31:57 +00:00
|
|
|
Arc::clone(&address_manager.pci_mmio32_allocators[0]),
|
|
|
|
Arc::clone(&address_manager.pci_mmio64_allocators[0]),
|
2021-10-11 15:35:52 +00:00
|
|
|
&pci_irq_slots,
|
2021-10-11 14:55:35 +00:00
|
|
|
)?];
|
|
|
|
|
2021-10-15 10:38:06 +00:00
|
|
|
for i in 1..num_pci_segments as usize {
|
|
|
|
pci_segments.push(PciSegment::new(
|
|
|
|
i as u16,
|
2023-10-16 02:36:34 +00:00
|
|
|
numa_node_id_from_pci_segment_id(&numa_nodes, i as u16),
|
2021-10-15 10:38:06 +00:00
|
|
|
&address_manager,
|
2023-11-14 19:31:57 +00:00
|
|
|
Arc::clone(&address_manager.pci_mmio32_allocators[i]),
|
|
|
|
Arc::clone(&address_manager.pci_mmio64_allocators[i]),
|
2021-10-15 10:38:06 +00:00
|
|
|
&pci_irq_slots,
|
|
|
|
)?);
|
2021-10-11 14:55:35 +00:00
|
|
|
}
|
|
|
|
|
2022-11-03 15:10:18 +00:00
|
|
|
if dynamic {
|
|
|
|
let acpi_address = address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_platform_mmio_addresses(None, CPU_MANAGER_ACPI_SIZE as u64, None)
|
|
|
|
.ok_or(DeviceManagerError::AllocateMmioAddress)?;
|
|
|
|
|
|
|
|
address_manager
|
|
|
|
.mmio_bus
|
|
|
|
.insert(
|
|
|
|
cpu_manager.clone(),
|
|
|
|
acpi_address.0,
|
|
|
|
CPU_MANAGER_ACPI_SIZE as u64,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
|
|
|
|
cpu_manager.lock().unwrap().set_acpi_address(acpi_address);
|
|
|
|
}
|
|
|
|
|
2023-12-07 19:45:08 +00:00
|
|
|
let mut rate_limit_groups = HashMap::<String, Arc<RateLimiterGroup>>::new();
|
|
|
|
if let Some(rate_limit_groups_cfg) = config.lock().unwrap().rate_limit_groups.as_ref() {
|
|
|
|
for rate_limit_group_cfg in rate_limit_groups_cfg {
|
|
|
|
let rate_limit_cfg = rate_limit_group_cfg.rate_limiter_config;
|
|
|
|
let bw = rate_limit_cfg.bandwidth.unwrap_or_default();
|
|
|
|
let ops = rate_limit_cfg.ops.unwrap_or_default();
|
|
|
|
let mut rate_limit_group = RateLimiterGroup::new(
|
|
|
|
&rate_limit_group_cfg.id,
|
|
|
|
bw.size,
|
|
|
|
bw.one_time_burst.unwrap_or(0),
|
|
|
|
bw.refill_time,
|
|
|
|
ops.size,
|
|
|
|
ops.one_time_burst.unwrap_or(0),
|
|
|
|
ops.refill_time,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::RateLimiterGroupCreate)?;
|
|
|
|
|
|
|
|
let exit_evt = exit_evt.try_clone().map_err(DeviceManagerError::EventFd)?;
|
|
|
|
|
|
|
|
rate_limit_group.start_thread(exit_evt).unwrap();
|
|
|
|
rate_limit_groups
|
|
|
|
.insert(rate_limit_group_cfg.id.clone(), Arc::new(rate_limit_group));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
let device_manager = DeviceManager {
|
2022-07-20 22:51:15 +00:00
|
|
|
hypervisor_type,
|
2020-02-27 10:36:30 +00:00
|
|
|
address_manager: Arc::clone(&address_manager),
|
2020-01-29 15:53:12 +00:00
|
|
|
console: Arc::new(Console::default()),
|
2020-05-25 08:27:08 +00:00
|
|
|
interrupt_controller: None,
|
2022-04-28 14:21:14 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
2020-04-28 17:57:28 +00:00
|
|
|
cmdline_additions: Vec::new(),
|
2020-01-29 15:53:12 +00:00
|
|
|
ged_notification_device: None,
|
2020-01-31 16:23:49 +00:00
|
|
|
config,
|
2020-01-31 11:55:30 +00:00
|
|
|
memory_manager,
|
2022-11-03 15:10:18 +00:00
|
|
|
cpu_manager,
|
2020-02-14 10:08:14 +00:00
|
|
|
virtio_devices: Vec::new(),
|
2020-04-28 17:57:28 +00:00
|
|
|
bus_devices: Vec::new(),
|
2022-11-28 16:40:17 +00:00
|
|
|
device_id_cnt,
|
2020-04-28 17:57:28 +00:00
|
|
|
msi_interrupt_manager,
|
2021-02-09 08:17:29 +00:00
|
|
|
legacy_interrupt_manager: None,
|
2020-07-17 15:20:47 +00:00
|
|
|
passthrough_device: None,
|
2021-09-14 09:02:10 +00:00
|
|
|
vfio_container: None,
|
2020-02-27 17:43:47 +00:00
|
|
|
iommu_device: None,
|
2022-03-18 16:57:12 +00:00
|
|
|
iommu_mapping: None,
|
2021-06-14 13:38:24 +00:00
|
|
|
iommu_attached_devices: None,
|
2021-10-11 14:55:35 +00:00
|
|
|
pci_segments,
|
2020-05-12 15:25:30 +00:00
|
|
|
device_tree,
|
2022-11-03 15:10:18 +00:00
|
|
|
exit_evt,
|
|
|
|
reset_evt,
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
id_to_dev_info: HashMap::new(),
|
2020-08-04 02:45:53 +00:00
|
|
|
seccomp_action,
|
2020-09-11 14:15:29 +00:00
|
|
|
numa_nodes,
|
2020-10-14 10:04:42 +00:00
|
|
|
balloon: None,
|
2020-11-09 13:29:05 +00:00
|
|
|
activate_evt: activate_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2021-01-20 16:12:02 +00:00
|
|
|
acpi_address,
|
2021-10-08 09:49:26 +00:00
|
|
|
selected_segment: 0,
|
2021-01-14 03:03:53 +00:00
|
|
|
serial_pty: None,
|
2021-09-16 19:47:36 +00:00
|
|
|
serial_manager: None,
|
2021-01-14 03:03:53 +00:00
|
|
|
console_pty: None,
|
2024-01-17 21:12:43 +00:00
|
|
|
debug_console_pty: None,
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
console_resize_pipe: None,
|
vmm: reset to the original termios
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-04-27 12:57:28 +00:00
|
|
|
original_termios_opt: Arc::new(Mutex::new(None)),
|
2021-03-03 10:59:53 +00:00
|
|
|
virtio_mem_devices: Vec::new(),
|
2021-03-07 12:11:07 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
gpio_device: None,
|
2023-06-15 04:02:45 +00:00
|
|
|
pvpanic_device: None,
|
2021-07-20 07:59:32 +00:00
|
|
|
force_iommu,
|
vmm: Cache whether io_uring is supported in DeviceManager
Probing for whether the io_uring is supported is time consuming so cache
this value if it is known to reduce the cost for secondary block devices
that are added.
Before:
cloud-hypervisor: 3.988896ms: <vmm> INFO:vmm/src/device_manager.rs:1901 -- Creating virtio-block device: DiskConfig { path: Some("/home/rob/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk0"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.129591ms: <vmm> INFO:vmm/src/device_manager.rs:1983 -- Using asynchronous RAW disk file (io_uring)
cloud-hypervisor: 14.159853ms: <vmm> INFO:vmm/src/device_manager.rs:1901 -- Creating virtio-block device: DiskConfig { path: Some("/tmp/disk"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk1"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 22.110281ms: <vmm> INFO:vmm/src/device_manager.rs:1983 -- Using asynchronous RAW disk file (io_uring)
After:
cloud-hypervisor: 4.880411ms: <vmm> INFO:vmm/src/device_manager.rs:1916 -- Creating virtio-block device: DiskConfig { path: Some("/home/rob/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk0"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.105123ms: <vmm> INFO:vmm/src/device_manager.rs:1998 -- Using asynchronous RAW disk file (io_uring)
cloud-hypervisor: 14.134837ms: <vmm> INFO:vmm/src/device_manager.rs:1916 -- Creating virtio-block device: DiskConfig { path: Some("/tmp/disk"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk1"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.221869ms: <vmm> INFO:vmm/src/device_manager.rs:1998 -- Using asynchronous RAW disk file (io_uring)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-11-12 10:38:51 +00:00
|
|
|
io_uring_supported: None,
|
2023-10-19 02:59:22 +00:00
|
|
|
aio_supported: None,
|
2022-04-29 07:58:22 +00:00
|
|
|
boot_id_list,
|
2022-05-04 15:36:57 +00:00
|
|
|
timestamp,
|
2022-05-17 14:33:33 +00:00
|
|
|
pending_activations: Arc::new(Mutex::new(Vec::default())),
|
2022-07-25 12:36:31 +00:00
|
|
|
acpi_platform_addresses: AcpiPlatformAddresses::default(),
|
2022-10-18 15:14:43 +00:00
|
|
|
snapshot,
|
2023-12-07 19:45:08 +00:00
|
|
|
rate_limit_groups,
|
2020-01-29 15:33:30 +00:00
|
|
|
};
|
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
let device_manager = Arc::new(Mutex::new(device_manager));
|
2019-05-12 17:38:24 +00:00
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
address_manager
|
2021-01-20 16:12:02 +00:00
|
|
|
.mmio_bus
|
2020-04-28 17:57:28 +00:00
|
|
|
.insert(
|
|
|
|
Arc::clone(&device_manager) as Arc<Mutex<dyn BusDevice>>,
|
2021-01-20 16:12:02 +00:00
|
|
|
acpi_address.0,
|
|
|
|
DEVICE_MANAGER_ACPI_SIZE as u64,
|
2020-04-28 17:57:28 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
2020-01-29 15:53:12 +00:00
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
Ok(device_manager)
|
|
|
|
}
|
2020-01-29 15:53:12 +00:00
|
|
|
|
2021-03-04 23:34:45 +00:00
|
|
|
pub fn serial_pty(&self) -> Option<PtyPair> {
|
2021-01-14 03:03:53 +00:00
|
|
|
self.serial_pty
|
|
|
|
.as_ref()
|
2021-03-04 23:34:45 +00:00
|
|
|
.map(|pty| pty.lock().unwrap().clone())
|
2021-01-14 03:03:53 +00:00
|
|
|
}
|
|
|
|
|
2021-03-04 23:34:45 +00:00
|
|
|
pub fn console_pty(&self) -> Option<PtyPair> {
|
2021-01-14 03:03:53 +00:00
|
|
|
self.console_pty
|
|
|
|
.as_ref()
|
2021-03-04 23:34:45 +00:00
|
|
|
.map(|pty| pty.lock().unwrap().clone())
|
2021-01-14 03:03:53 +00:00
|
|
|
}
|
|
|
|
|
2024-01-17 21:12:43 +00:00
|
|
|
pub fn debug_console_pty(&self) -> Option<PtyPair> {
|
|
|
|
self.debug_console_pty
|
|
|
|
.as_ref()
|
|
|
|
.map(|pty| pty.lock().unwrap().clone())
|
|
|
|
}
|
|
|
|
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
pub fn console_resize_pipe(&self) -> Option<Arc<File>> {
|
2024-02-06 21:00:52 +00:00
|
|
|
self.console_resize_pipe.clone()
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
}
|
|
|
|
|
2021-03-04 23:34:45 +00:00
|
|
|
pub fn create_devices(
|
|
|
|
&mut self,
|
|
|
|
serial_pty: Option<PtyPair>,
|
|
|
|
console_pty: Option<PtyPair>,
|
2024-01-17 21:12:43 +00:00
|
|
|
debug_console_pty: Option<PtyPair>,
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
console_resize_pipe: Option<File>,
|
vmm: reset to the original termios
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-04-27 12:57:28 +00:00
|
|
|
original_termios_opt: Arc<Mutex<Option<termios>>>,
|
2021-03-04 23:34:45 +00:00
|
|
|
) -> DeviceManagerResult<()> {
|
2022-09-16 15:56:36 +00:00
|
|
|
trace_scoped!("create_devices");
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
let mut virtio_devices: Vec<MetaVirtioDevice> = Vec::new();
|
2020-01-29 15:33:30 +00:00
|
|
|
|
2020-05-25 08:27:08 +00:00
|
|
|
let interrupt_controller = self.add_interrupt_controller()?;
|
2020-02-14 10:08:14 +00:00
|
|
|
|
2022-11-03 15:10:18 +00:00
|
|
|
self.cpu_manager
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.set_interrupt_controller(interrupt_controller.clone());
|
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
// Now we can create the legacy interrupt manager, which needs the freshly
|
|
|
|
// formed IOAPIC device.
|
|
|
|
let legacy_interrupt_manager: Arc<
|
|
|
|
dyn InterruptManager<GroupConfig = LegacyIrqGroupConfig>,
|
2020-06-25 14:56:49 +00:00
|
|
|
> = Arc::new(LegacyUserspaceInterruptManager::new(Arc::clone(
|
2020-05-25 08:27:08 +00:00
|
|
|
&interrupt_controller,
|
|
|
|
)));
|
2020-02-27 10:36:30 +00:00
|
|
|
|
2021-01-20 16:12:02 +00:00
|
|
|
{
|
2022-03-24 11:03:26 +00:00
|
|
|
if let Some(acpi_address) = self.memory_manager.lock().unwrap().acpi_address() {
|
|
|
|
self.address_manager
|
|
|
|
.mmio_bus
|
|
|
|
.insert(
|
|
|
|
Arc::clone(&self.memory_manager) as Arc<Mutex<dyn BusDevice>>,
|
|
|
|
acpi_address.0,
|
|
|
|
MEMORY_MANAGER_ACPI_SIZE as u64,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
}
|
2021-01-20 16:12:02 +00:00
|
|
|
}
|
2020-02-27 10:36:30 +00:00
|
|
|
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2020-04-28 17:57:28 +00:00
|
|
|
self.add_legacy_devices(
|
|
|
|
self.reset_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
|
|
|
)?;
|
|
|
|
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
self.add_legacy_devices(&legacy_interrupt_manager)?;
|
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
{
|
|
|
|
self.ged_notification_device = self.add_acpi_devices(
|
|
|
|
&legacy_interrupt_manager,
|
|
|
|
self.reset_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
|
|
|
)?;
|
|
|
|
}
|
|
|
|
|
vmm: reset to the original termios
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-04-27 12:57:28 +00:00
|
|
|
self.original_termios_opt = original_termios_opt;
|
|
|
|
|
2024-01-17 21:12:43 +00:00
|
|
|
self.console = self.add_console_devices(
|
2021-03-04 23:34:45 +00:00
|
|
|
&legacy_interrupt_manager,
|
|
|
|
&mut virtio_devices,
|
|
|
|
serial_pty,
|
|
|
|
console_pty,
|
2024-01-17 21:12:43 +00:00
|
|
|
debug_console_pty,
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
console_resize_pipe,
|
2021-03-04 23:34:45 +00:00
|
|
|
)?;
|
2020-04-28 17:57:28 +00:00
|
|
|
|
2022-08-15 15:34:54 +00:00
|
|
|
if let Some(tpm) = self.config.clone().lock().unwrap().tpm.as_ref() {
|
|
|
|
let tpm_dev = self.add_tpm_device(tpm.socket.clone())?;
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&tpm_dev) as Arc<Mutex<dyn BusDevice>>)
|
|
|
|
}
|
2021-02-09 08:17:29 +00:00
|
|
|
self.legacy_interrupt_manager = Some(legacy_interrupt_manager);
|
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
virtio_devices.append(&mut self.make_virtio_devices()?);
|
|
|
|
|
2020-10-15 15:34:35 +00:00
|
|
|
self.add_pci_devices(virtio_devices.clone())?;
|
2020-04-28 17:57:28 +00:00
|
|
|
|
|
|
|
self.virtio_devices = virtio_devices;
|
|
|
|
|
2023-06-15 04:02:45 +00:00
|
|
|
if self.config.clone().lock().unwrap().pvpanic {
|
|
|
|
self.pvpanic_device = self.add_pvpanic_device()?;
|
|
|
|
}
|
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
Ok(())
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2020-04-28 10:08:51 +00:00
|
|
|
fn state(&self) -> DeviceManagerState {
|
|
|
|
DeviceManagerState {
|
2020-05-12 13:53:09 +00:00
|
|
|
device_tree: self.device_tree.lock().unwrap().clone(),
|
2020-04-28 10:08:51 +00:00
|
|
|
device_id_cnt: self.device_id_cnt,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-09-06 12:58:48 +00:00
|
|
|
fn get_msi_iova_space(&mut self) -> (u64, u64) {
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
{
|
|
|
|
let vcpus = self.config.lock().unwrap().cpus.boot_vcpus;
|
2022-08-30 00:49:57 +00:00
|
|
|
let vgic_config = gic::Gic::create_default_config(vcpus.into());
|
|
|
|
(
|
|
|
|
vgic_config.msi_addr,
|
|
|
|
vgic_config.msi_addr + vgic_config.msi_size - 1,
|
|
|
|
)
|
2021-09-06 12:58:48 +00:00
|
|
|
}
|
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
(0xfee0_0000, 0xfeef_ffff)
|
|
|
|
}
|
|
|
|
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
/// Gets the information of the devices registered up to some point in time.
|
2021-03-25 17:01:21 +00:00
|
|
|
pub fn get_device_info(&self) -> &HashMap<(DeviceType, String), MmioDeviceInfo> {
|
2020-06-09 06:17:42 +00:00
|
|
|
&self.id_to_dev_info
|
|
|
|
}
|
|
|
|
|
2019-11-18 11:23:27 +00:00
|
|
|
#[allow(unused_variables)]
|
|
|
|
fn add_pci_devices(
|
2020-01-29 15:33:30 +00:00
|
|
|
&mut self,
|
2022-03-10 13:05:36 +00:00
|
|
|
virtio_devices: Vec<MetaVirtioDevice>,
|
2019-11-18 11:23:27 +00:00
|
|
|
) -> DeviceManagerResult<()> {
|
2020-10-15 15:34:35 +00:00
|
|
|
let iommu_id = String::from(IOMMU_DEVICE_NAME);
|
2020-04-27 12:51:15 +00:00
|
|
|
|
2022-03-18 16:57:12 +00:00
|
|
|
let iommu_device = if self.config.lock().unwrap().iommu {
|
2021-09-07 15:10:48 +00:00
|
|
|
let (device, mapping) = virtio_devices::Iommu::new(
|
|
|
|
iommu_id.clone(),
|
|
|
|
self.seccomp_action.clone(),
|
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2021-09-06 12:58:48 +00:00
|
|
|
self.get_msi_iova_space(),
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), iommu_id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2021-09-07 15:10:48 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioIommu)?;
|
2020-10-15 15:34:35 +00:00
|
|
|
let device = Arc::new(Mutex::new(device));
|
|
|
|
self.iommu_device = Some(Arc::clone(&device));
|
2022-03-18 16:57:12 +00:00
|
|
|
self.iommu_mapping = Some(mapping);
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2020-10-15 15:34:35 +00:00
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(iommu_id.clone(), device_node!(iommu_id, device));
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2022-03-18 16:57:12 +00:00
|
|
|
Some(device)
|
2020-10-15 15:34:35 +00:00
|
|
|
} else {
|
2022-03-18 16:57:12 +00:00
|
|
|
None
|
2020-10-15 15:34:35 +00:00
|
|
|
};
|
2019-11-18 11:23:27 +00:00
|
|
|
|
2020-10-15 15:34:35 +00:00
|
|
|
let mut iommu_attached_devices = Vec::new();
|
2021-10-04 14:33:29 +00:00
|
|
|
{
|
2022-03-10 13:05:36 +00:00
|
|
|
for handle in virtio_devices {
|
2022-03-18 16:57:12 +00:00
|
|
|
let mapping: Option<Arc<IommuMapping>> = if handle.iommu {
|
|
|
|
self.iommu_mapping.clone()
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
};
|
2022-03-10 13:05:36 +00:00
|
|
|
|
|
|
|
let dev_id = self.add_virtio_pci_device(
|
|
|
|
handle.virtio_device,
|
2022-03-18 16:57:12 +00:00
|
|
|
&mapping,
|
2022-03-10 13:05:36 +00:00
|
|
|
handle.id,
|
|
|
|
handle.pci_segment,
|
2022-03-10 13:16:08 +00:00
|
|
|
handle.dma_handler,
|
2022-03-10 13:05:36 +00:00
|
|
|
)?;
|
|
|
|
|
|
|
|
if handle.iommu {
|
2021-10-04 14:33:29 +00:00
|
|
|
iommu_attached_devices.push(dev_id);
|
|
|
|
}
|
2019-11-18 11:23:27 +00:00
|
|
|
}
|
|
|
|
|
2021-10-04 14:33:29 +00:00
|
|
|
let mut vfio_iommu_device_ids = self.add_vfio_devices()?;
|
|
|
|
iommu_attached_devices.append(&mut vfio_iommu_device_ids);
|
2020-02-27 16:44:14 +00:00
|
|
|
|
2021-10-04 14:33:29 +00:00
|
|
|
let mut vfio_user_iommu_device_ids = self.add_user_devices()?;
|
|
|
|
iommu_attached_devices.append(&mut vfio_user_iommu_device_ids);
|
2021-06-09 13:43:36 +00:00
|
|
|
|
2022-02-11 09:33:55 +00:00
|
|
|
// Add all devices from forced iommu segments
|
|
|
|
if let Some(platform_config) = self.config.lock().unwrap().platform.as_ref() {
|
|
|
|
if let Some(iommu_segments) = platform_config.iommu_segments.as_ref() {
|
|
|
|
for segment in iommu_segments {
|
|
|
|
for device in 0..32 {
|
|
|
|
let bdf = PciBdf::new(*segment, 0, device, 0);
|
|
|
|
if !iommu_attached_devices.contains(&bdf) {
|
|
|
|
iommu_attached_devices.push(bdf);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-10-04 14:33:29 +00:00
|
|
|
if let Some(iommu_device) = iommu_device {
|
2022-03-10 13:16:08 +00:00
|
|
|
let dev_id = self.add_virtio_pci_device(iommu_device, &None, iommu_id, 0, None)?;
|
2021-10-04 14:33:29 +00:00
|
|
|
self.iommu_attached_devices = Some((dev_id, iommu_attached_devices));
|
|
|
|
}
|
2019-11-18 11:23:27 +00:00
|
|
|
}
|
|
|
|
|
2021-10-05 14:00:23 +00:00
|
|
|
for segment in &self.pci_segments {
|
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
if let Some(pci_config_io) = segment.pci_config_io.as_ref() {
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(pci_config_io) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
}
|
2019-11-18 11:23:27 +00:00
|
|
|
|
2021-10-05 14:00:23 +00:00
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&segment.pci_config_mmio) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
}
|
2019-11-18 11:35:05 +00:00
|
|
|
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
2020-05-12 09:49:12 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
2020-05-25 08:27:08 +00:00
|
|
|
fn add_interrupt_controller(
|
|
|
|
&mut self,
|
|
|
|
) -> DeviceManagerResult<Arc<Mutex<dyn InterruptController>>> {
|
2020-05-25 08:59:09 +00:00
|
|
|
let interrupt_controller: Arc<Mutex<gic::Gic>> = Arc::new(Mutex::new(
|
|
|
|
gic::Gic::new(
|
|
|
|
self.config.lock().unwrap().cpus.boot_vcpus,
|
|
|
|
Arc::clone(&self.msi_interrupt_manager),
|
2022-11-06 10:51:12 +00:00
|
|
|
self.address_manager.vm.clone(),
|
2020-05-25 08:59:09 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateInterruptController)?,
|
|
|
|
));
|
|
|
|
|
|
|
|
self.interrupt_controller = Some(interrupt_controller.clone());
|
|
|
|
|
2022-11-29 12:35:35 +00:00
|
|
|
// Restore the vGic if this is in the process of restoration
|
|
|
|
let id = String::from(gic::GIC_SNAPSHOT_ID);
|
|
|
|
if let Some(vgic_snapshot) = snapshot_from_id(self.snapshot.as_ref(), &id) {
|
|
|
|
// PMU support is optional. Nothing should be impacted if the PMU initialization failed.
|
|
|
|
if self
|
|
|
|
.cpu_manager
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.init_pmu(arch::aarch64::fdt::AARCH64_PMU_IRQ + 16)
|
|
|
|
.is_err()
|
|
|
|
{
|
|
|
|
info!("Failed to initialize PMU");
|
|
|
|
}
|
|
|
|
|
|
|
|
let vgic_state = vgic_snapshot
|
2022-12-02 13:37:49 +00:00
|
|
|
.to_state()
|
2022-11-29 12:35:35 +00:00
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?;
|
|
|
|
let saved_vcpu_states = self.cpu_manager.lock().unwrap().get_saved_states();
|
|
|
|
interrupt_controller
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.restore_vgic(vgic_state, &saved_vcpu_states)
|
|
|
|
.unwrap();
|
|
|
|
}
|
|
|
|
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, interrupt_controller));
|
2020-05-25 08:59:09 +00:00
|
|
|
|
|
|
|
Ok(interrupt_controller)
|
2020-05-12 09:49:12 +00:00
|
|
|
}
|
|
|
|
|
2021-06-01 12:03:05 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
pub fn get_interrupt_controller(&mut self) -> Option<&Arc<Mutex<gic::Gic>>> {
|
|
|
|
self.interrupt_controller.as_ref()
|
|
|
|
}
|
|
|
|
|
2020-05-12 09:49:12 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2020-05-25 08:27:08 +00:00
|
|
|
fn add_interrupt_controller(
|
|
|
|
&mut self,
|
|
|
|
) -> DeviceManagerResult<Arc<Mutex<dyn InterruptController>>> {
|
2020-04-28 13:26:35 +00:00
|
|
|
let id = String::from(IOAPIC_DEVICE_NAME);
|
2020-04-28 13:22:38 +00:00
|
|
|
|
2019-12-05 16:36:28 +00:00
|
|
|
// Create IOAPIC
|
2020-05-25 08:27:08 +00:00
|
|
|
let interrupt_controller = Arc::new(Mutex::new(
|
2020-04-30 18:08:04 +00:00
|
|
|
ioapic::Ioapic::new(
|
|
|
|
id.clone(),
|
|
|
|
APIC_START,
|
|
|
|
Arc::clone(&self.msi_interrupt_manager),
|
2022-11-25 15:04:37 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-04-30 18:08:04 +00:00
|
|
|
)
|
2020-05-25 08:27:08 +00:00
|
|
|
.map_err(DeviceManagerError::CreateInterruptController)?,
|
2020-01-22 21:55:02 +00:00
|
|
|
));
|
2019-11-18 11:02:37 +00:00
|
|
|
|
2020-05-25 08:27:08 +00:00
|
|
|
self.interrupt_controller = Some(interrupt_controller.clone());
|
2020-04-28 17:57:28 +00:00
|
|
|
|
|
|
|
self.address_manager
|
2019-12-05 16:36:28 +00:00
|
|
|
.mmio_bus
|
2020-05-25 08:27:08 +00:00
|
|
|
.insert(interrupt_controller.clone(), IOAPIC_START.0, IOAPIC_SIZE)
|
2019-12-05 16:36:28 +00:00
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
2019-11-18 11:02:37 +00:00
|
|
|
|
2020-04-28 17:57:28 +00:00
|
|
|
self.bus_devices
|
2020-05-25 08:27:08 +00:00
|
|
|
.push(Arc::clone(&interrupt_controller) as Arc<Mutex<dyn BusDevice>>);
|
2020-04-28 17:57:28 +00:00
|
|
|
|
2020-04-30 18:08:04 +00:00
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
2020-05-05 08:23:32 +00:00
|
|
|
self.device_tree
|
2020-05-12 13:53:09 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2020-05-25 08:27:08 +00:00
|
|
|
.insert(id.clone(), device_node!(id, interrupt_controller));
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2020-05-25 08:27:08 +00:00
|
|
|
Ok(interrupt_controller)
|
2019-11-18 11:02:37 +00:00
|
|
|
}
|
|
|
|
|
2019-11-27 15:28:22 +00:00
|
|
|
fn add_acpi_devices(
|
2020-01-29 15:53:12 +00:00
|
|
|
&mut self,
|
2020-02-04 11:04:10 +00:00
|
|
|
interrupt_manager: &Arc<dyn InterruptManager<GroupConfig = LegacyIrqGroupConfig>>,
|
2019-11-18 10:55:04 +00:00
|
|
|
reset_evt: EventFd,
|
|
|
|
exit_evt: EventFd,
|
2021-03-25 17:01:21 +00:00
|
|
|
) -> DeviceManagerResult<Option<Arc<Mutex<devices::AcpiGedDevice>>>> {
|
2023-08-03 14:13:03 +00:00
|
|
|
let vcpus_kill_signalled = self
|
|
|
|
.cpu_manager
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.vcpus_kill_signalled()
|
|
|
|
.clone();
|
2021-01-20 15:22:51 +00:00
|
|
|
let shutdown_device = Arc::new(Mutex::new(devices::AcpiShutdownDevice::new(
|
2023-08-03 14:13:03 +00:00
|
|
|
exit_evt,
|
|
|
|
reset_evt,
|
|
|
|
vcpus_kill_signalled,
|
2019-11-27 15:28:22 +00:00
|
|
|
)));
|
2019-11-18 10:55:04 +00:00
|
|
|
|
2020-03-04 14:46:40 +00:00
|
|
|
self.bus_devices
|
2021-01-20 15:22:51 +00:00
|
|
|
.push(Arc::clone(&shutdown_device) as Arc<Mutex<dyn BusDevice>>);
|
2020-03-04 14:46:40 +00:00
|
|
|
|
2021-01-25 10:48:45 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
{
|
2022-08-11 09:43:16 +00:00
|
|
|
let shutdown_pio_address: u16 = 0x600;
|
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_io_addresses(Some(GuestAddress(shutdown_pio_address.into())), 0x8, None)
|
|
|
|
.ok_or(DeviceManagerError::AllocateIoPort)?;
|
|
|
|
|
2021-01-25 10:48:45 +00:00
|
|
|
self.address_manager
|
|
|
|
.io_bus
|
2022-08-11 09:43:16 +00:00
|
|
|
.insert(shutdown_device, shutdown_pio_address.into(), 0x4)
|
2021-01-25 10:48:45 +00:00
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
2022-08-11 09:43:16 +00:00
|
|
|
|
2022-07-25 12:36:31 +00:00
|
|
|
self.acpi_platform_addresses.sleep_control_reg_address =
|
2022-08-11 09:43:16 +00:00
|
|
|
Some(GenericAddress::io_port_address::<u8>(shutdown_pio_address));
|
2022-07-25 12:36:31 +00:00
|
|
|
self.acpi_platform_addresses.sleep_status_reg_address =
|
2022-08-11 09:43:16 +00:00
|
|
|
Some(GenericAddress::io_port_address::<u8>(shutdown_pio_address));
|
2022-07-25 12:36:31 +00:00
|
|
|
self.acpi_platform_addresses.reset_reg_address =
|
2022-08-11 09:43:16 +00:00
|
|
|
Some(GenericAddress::io_port_address::<u8>(shutdown_pio_address));
|
2021-01-25 10:48:45 +00:00
|
|
|
}
|
2019-11-18 10:55:04 +00:00
|
|
|
|
2020-01-29 15:53:12 +00:00
|
|
|
let ged_irq = self
|
|
|
|
.address_manager
|
2019-12-09 15:07:31 +00:00
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_irq()
|
|
|
|
.unwrap();
|
2020-01-20 15:00:18 +00:00
|
|
|
let interrupt_group = interrupt_manager
|
2020-02-04 11:04:10 +00:00
|
|
|
.create_group(LegacyIrqGroupConfig {
|
|
|
|
irq: ged_irq as InterruptIndex,
|
|
|
|
})
|
2020-01-20 15:00:18 +00:00
|
|
|
.map_err(DeviceManagerError::CreateInterruptGroup)?;
|
2021-01-20 15:32:10 +00:00
|
|
|
let ged_address = self
|
|
|
|
.address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-10-29 09:58:41 +00:00
|
|
|
.allocate_platform_mmio_addresses(
|
|
|
|
None,
|
|
|
|
devices::acpi::GED_DEVICE_ACPI_SIZE as u64,
|
|
|
|
None,
|
|
|
|
)
|
2021-03-25 17:01:21 +00:00
|
|
|
.ok_or(DeviceManagerError::AllocateMmioAddress)?;
|
2021-03-25 17:01:21 +00:00
|
|
|
let ged_device = Arc::new(Mutex::new(devices::AcpiGedDevice::new(
|
2020-01-20 15:00:18 +00:00
|
|
|
interrupt_group,
|
|
|
|
ged_irq,
|
2021-01-20 15:32:10 +00:00
|
|
|
ged_address,
|
2020-01-20 15:00:18 +00:00
|
|
|
)));
|
2020-01-29 15:53:12 +00:00
|
|
|
self.address_manager
|
2021-01-20 15:32:10 +00:00
|
|
|
.mmio_bus
|
|
|
|
.insert(
|
|
|
|
ged_device.clone(),
|
|
|
|
ged_address.0,
|
|
|
|
devices::acpi::GED_DEVICE_ACPI_SIZE as u64,
|
|
|
|
)
|
2020-01-24 10:36:39 +00:00
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
2021-01-20 15:32:10 +00:00
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&ged_device) as Arc<Mutex<dyn BusDevice>>);
|
2020-07-23 09:17:07 +00:00
|
|
|
|
2021-03-25 17:01:21 +00:00
|
|
|
let pm_timer_device = Arc::new(Mutex::new(devices::AcpiPmTimerDevice::new()));
|
2020-07-23 09:17:07 +00:00
|
|
|
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&pm_timer_device) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
2021-01-25 10:48:45 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
{
|
2022-08-11 09:43:16 +00:00
|
|
|
let pm_timer_pio_address: u16 = 0x608;
|
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_io_addresses(Some(GuestAddress(pm_timer_pio_address.into())), 0x4, None)
|
|
|
|
.ok_or(DeviceManagerError::AllocateIoPort)?;
|
|
|
|
|
2021-01-25 10:48:45 +00:00
|
|
|
self.address_manager
|
|
|
|
.io_bus
|
2022-08-11 09:43:16 +00:00
|
|
|
.insert(pm_timer_device, pm_timer_pio_address.into(), 0x4)
|
2021-01-25 10:48:45 +00:00
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
2022-07-25 12:36:31 +00:00
|
|
|
|
|
|
|
self.acpi_platform_addresses.pm_timer_address =
|
2022-08-11 09:43:16 +00:00
|
|
|
Some(GenericAddress::io_port_address::<u32>(pm_timer_pio_address));
|
2021-01-25 10:48:45 +00:00
|
|
|
}
|
2020-07-23 09:17:07 +00:00
|
|
|
|
2019-11-27 15:28:22 +00:00
|
|
|
Ok(Some(ged_device))
|
2019-11-18 10:55:04 +00:00
|
|
|
}
|
|
|
|
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2020-01-31 11:55:30 +00:00
|
|
|
fn add_legacy_devices(&mut self, reset_evt: EventFd) -> DeviceManagerResult<()> {
|
2023-08-03 14:13:03 +00:00
|
|
|
let vcpus_kill_signalled = self
|
|
|
|
.cpu_manager
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.vcpus_kill_signalled()
|
|
|
|
.clone();
|
2019-11-18 10:44:01 +00:00
|
|
|
// Add a shutdown device (i8042)
|
2022-03-29 11:40:34 +00:00
|
|
|
let i8042 = Arc::new(Mutex::new(devices::legacy::I8042Device::new(
|
|
|
|
reset_evt.try_clone().unwrap(),
|
2023-08-03 14:13:03 +00:00
|
|
|
vcpus_kill_signalled.clone(),
|
2022-03-29 11:40:34 +00:00
|
|
|
)));
|
2019-11-18 10:44:01 +00:00
|
|
|
|
2020-03-04 14:46:40 +00:00
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&i8042) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
2020-01-29 15:53:12 +00:00
|
|
|
self.address_manager
|
2019-11-18 10:44:01 +00:00
|
|
|
.io_bus
|
vmm: device_manager: Remove redundant clones
Address updated clippy errors:
error: redundant clone
--> vmm/src/device_manager.rs:699:32
|
699 | .insert(acpi_device.clone(), 0x3c0, 0x4)
| ^^^^^^^^ help: remove this
|
= note: `-D clippy::redundant-clone` implied by `-D warnings`
note: this value is dropped without further use
--> vmm/src/device_manager.rs:699:21
|
699 | .insert(acpi_device.clone(), 0x3c0, 0x4)
| ^^^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: redundant clone
--> vmm/src/device_manager.rs:737:26
|
737 | .insert(i8042.clone(), 0x61, 0x4)
| ^^^^^^^^ help: remove this
|
note: this value is dropped without further use
--> vmm/src/device_manager.rs:737:21
|
737 | .insert(i8042.clone(), 0x61, 0x4)
| ^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: redundant clone
--> vmm/src/device_manager.rs:754:29
|
754 | .insert(cmos.clone(), 0x70, 0x2)
| ^^^^^^^^ help: remove this
|
note: this value is dropped without further use
--> vmm/src/device_manager.rs:754:25
|
754 | .insert(cmos.clone(), 0x70, 0x2)
| ^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-19 17:02:36 +00:00
|
|
|
.insert(i8042, 0x61, 0x4)
|
2019-11-18 10:44:01 +00:00
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
{
|
|
|
|
// Add a CMOS emulated device
|
2020-01-31 11:55:30 +00:00
|
|
|
let mem_size = self
|
|
|
|
.memory_manager
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.guest_memory()
|
2020-02-11 16:22:40 +00:00
|
|
|
.memory()
|
2020-01-31 11:55:30 +00:00
|
|
|
.last_addr()
|
|
|
|
.0
|
|
|
|
+ 1;
|
2019-11-18 10:44:01 +00:00
|
|
|
let mem_below_4g = std::cmp::min(arch::layout::MEM_32BIT_RESERVED_START.0, mem_size);
|
|
|
|
let mem_above_4g = mem_size.saturating_sub(arch::layout::RAM_64BIT_START.0);
|
|
|
|
|
|
|
|
let cmos = Arc::new(Mutex::new(devices::legacy::Cmos::new(
|
|
|
|
mem_below_4g,
|
|
|
|
mem_above_4g,
|
2022-03-29 11:40:34 +00:00
|
|
|
reset_evt,
|
2023-08-05 08:43:12 +00:00
|
|
|
Some(vcpus_kill_signalled),
|
2019-11-18 10:44:01 +00:00
|
|
|
)));
|
|
|
|
|
2020-03-04 14:46:40 +00:00
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&cmos) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
2020-01-29 15:53:12 +00:00
|
|
|
self.address_manager
|
2019-11-18 10:44:01 +00:00
|
|
|
.io_bus
|
vmm: device_manager: Remove redundant clones
Address updated clippy errors:
error: redundant clone
--> vmm/src/device_manager.rs:699:32
|
699 | .insert(acpi_device.clone(), 0x3c0, 0x4)
| ^^^^^^^^ help: remove this
|
= note: `-D clippy::redundant-clone` implied by `-D warnings`
note: this value is dropped without further use
--> vmm/src/device_manager.rs:699:21
|
699 | .insert(acpi_device.clone(), 0x3c0, 0x4)
| ^^^^^^^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: redundant clone
--> vmm/src/device_manager.rs:737:26
|
737 | .insert(i8042.clone(), 0x61, 0x4)
| ^^^^^^^^ help: remove this
|
note: this value is dropped without further use
--> vmm/src/device_manager.rs:737:21
|
737 | .insert(i8042.clone(), 0x61, 0x4)
| ^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
error: redundant clone
--> vmm/src/device_manager.rs:754:29
|
754 | .insert(cmos.clone(), 0x70, 0x2)
| ^^^^^^^^ help: remove this
|
note: this value is dropped without further use
--> vmm/src/device_manager.rs:754:25
|
754 | .insert(cmos.clone(), 0x70, 0x2)
| ^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_clone
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-19 17:02:36 +00:00
|
|
|
.insert(cmos, 0x70, 0x2)
|
2019-11-18 10:44:01 +00:00
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
2022-09-26 16:02:18 +00:00
|
|
|
|
2020-04-17 09:39:28 +00:00
|
|
|
let fwdebug = Arc::new(Mutex::new(devices::legacy::FwDebugDevice::new()));
|
|
|
|
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&fwdebug) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.io_bus
|
|
|
|
.insert(fwdebug, 0x402, 0x1)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
}
|
2019-11-18 10:44:01 +00:00
|
|
|
|
2022-05-04 15:36:57 +00:00
|
|
|
// 0x80 debug port
|
|
|
|
let debug_port = Arc::new(Mutex::new(devices::legacy::DebugPort::new(self.timestamp)));
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&debug_port) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
self.address_manager
|
|
|
|
.io_bus
|
|
|
|
.insert(debug_port, 0x80, 0x1)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
|
2019-11-18 10:44:01 +00:00
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
fn add_legacy_devices(
|
|
|
|
&mut self,
|
|
|
|
interrupt_manager: &Arc<dyn InterruptManager<GroupConfig = LegacyIrqGroupConfig>>,
|
|
|
|
) -> DeviceManagerResult<()> {
|
|
|
|
// Add a RTC device
|
|
|
|
let rtc_irq = self
|
|
|
|
.address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_irq()
|
|
|
|
.unwrap();
|
|
|
|
|
|
|
|
let interrupt_group = interrupt_manager
|
|
|
|
.create_group(LegacyIrqGroupConfig {
|
|
|
|
irq: rtc_irq as InterruptIndex,
|
|
|
|
})
|
|
|
|
.map_err(DeviceManagerError::CreateInterruptGroup)?;
|
|
|
|
|
2021-03-25 17:01:21 +00:00
|
|
|
let rtc_device = Arc::new(Mutex::new(devices::legacy::Rtc::new(interrupt_group)));
|
2020-06-09 06:17:42 +00:00
|
|
|
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&rtc_device) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
2022-04-05 09:05:24 +00:00
|
|
|
let addr = arch::layout::LEGACY_RTC_MAPPED_IO_START;
|
2020-06-09 06:17:42 +00:00
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.mmio_bus
|
2020-08-23 07:45:44 +00:00
|
|
|
.insert(rtc_device, addr.0, MMIO_LEN)
|
2020-06-09 06:17:42 +00:00
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
|
|
|
|
self.id_to_dev_info.insert(
|
2021-03-25 17:01:21 +00:00
|
|
|
(DeviceType::Rtc, "rtc".to_string()),
|
|
|
|
MmioDeviceInfo {
|
2020-06-09 06:17:42 +00:00
|
|
|
addr: addr.0,
|
2021-11-28 02:52:15 +00:00
|
|
|
len: MMIO_LEN,
|
2020-06-09 06:17:42 +00:00
|
|
|
irq: rtc_irq,
|
|
|
|
},
|
|
|
|
);
|
|
|
|
|
2021-03-07 12:11:07 +00:00
|
|
|
// Add a GPIO device
|
2022-05-04 09:36:26 +00:00
|
|
|
let id = String::from(GPIO_DEVICE_NAME);
|
2021-03-07 12:11:07 +00:00
|
|
|
let gpio_irq = self
|
|
|
|
.address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_irq()
|
|
|
|
.unwrap();
|
|
|
|
|
|
|
|
let interrupt_group = interrupt_manager
|
|
|
|
.create_group(LegacyIrqGroupConfig {
|
|
|
|
irq: gpio_irq as InterruptIndex,
|
|
|
|
})
|
|
|
|
.map_err(DeviceManagerError::CreateInterruptGroup)?;
|
|
|
|
|
2021-03-25 17:01:21 +00:00
|
|
|
let gpio_device = Arc::new(Mutex::new(devices::legacy::Gpio::new(
|
2021-03-07 12:11:07 +00:00
|
|
|
id.clone(),
|
|
|
|
interrupt_group,
|
2022-11-25 16:17:53 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2021-03-07 12:11:07 +00:00
|
|
|
)));
|
|
|
|
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&gpio_device) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
2022-04-05 09:05:24 +00:00
|
|
|
let addr = arch::layout::LEGACY_GPIO_MAPPED_IO_START;
|
2021-03-07 12:11:07 +00:00
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.mmio_bus
|
|
|
|
.insert(gpio_device.clone(), addr.0, MMIO_LEN)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
|
|
|
|
self.gpio_device = Some(gpio_device.clone());
|
|
|
|
|
|
|
|
self.id_to_dev_info.insert(
|
2021-03-25 17:01:21 +00:00
|
|
|
(DeviceType::Gpio, "gpio".to_string()),
|
|
|
|
MmioDeviceInfo {
|
2021-03-07 12:11:07 +00:00
|
|
|
addr: addr.0,
|
2021-11-28 02:52:15 +00:00
|
|
|
len: MMIO_LEN,
|
2021-03-07 12:11:07 +00:00
|
|
|
irq: gpio_irq,
|
|
|
|
},
|
|
|
|
);
|
|
|
|
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, gpio_device));
|
|
|
|
|
2020-06-09 06:17:42 +00:00
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
2024-01-17 21:12:43 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
fn add_debug_console_device(
|
|
|
|
&mut self,
|
|
|
|
debug_console_writer: Box<dyn io::Write + Send>,
|
|
|
|
) -> DeviceManagerResult<Arc<Mutex<DebugConsole>>> {
|
|
|
|
let id = String::from(DEBUGCON_DEVICE_NAME);
|
|
|
|
let debug_console = Arc::new(Mutex::new(DebugConsole::new(
|
|
|
|
id.clone(),
|
|
|
|
debug_console_writer,
|
|
|
|
)));
|
|
|
|
|
|
|
|
let port = self
|
|
|
|
.config
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.debug_console
|
|
|
|
.clone()
|
|
|
|
.iobase
|
|
|
|
.map(|port| port as u64)
|
|
|
|
.unwrap_or(debug_console::DEFAULT_PORT);
|
|
|
|
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&debug_console) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_io_addresses(Some(GuestAddress(port)), 0x1, None)
|
|
|
|
.ok_or(DeviceManagerError::AllocateIoPort)?;
|
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.io_bus
|
|
|
|
.insert(debug_console.clone(), port, 0x1)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
|
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, debug_console));
|
|
|
|
|
|
|
|
Ok(debug_console)
|
|
|
|
}
|
|
|
|
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
fn add_serial_device(
|
|
|
|
&mut self,
|
|
|
|
interrupt_manager: &Arc<dyn InterruptManager<GroupConfig = LegacyIrqGroupConfig>>,
|
|
|
|
serial_writer: Option<Box<dyn io::Write + Send>>,
|
|
|
|
) -> DeviceManagerResult<Arc<Mutex<Serial>>> {
|
|
|
|
// Serial is tied to IRQ #4
|
|
|
|
let serial_irq = 4;
|
|
|
|
|
2022-05-04 09:36:26 +00:00
|
|
|
let id = String::from(SERIAL_DEVICE_NAME);
|
2020-06-09 06:17:42 +00:00
|
|
|
|
|
|
|
let interrupt_group = interrupt_manager
|
|
|
|
.create_group(LegacyIrqGroupConfig {
|
|
|
|
irq: serial_irq as InterruptIndex,
|
|
|
|
})
|
|
|
|
.map_err(DeviceManagerError::CreateInterruptGroup)?;
|
|
|
|
|
|
|
|
let serial = Arc::new(Mutex::new(Serial::new(
|
|
|
|
id.clone(),
|
|
|
|
interrupt_group,
|
|
|
|
serial_writer,
|
2022-11-25 15:44:42 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-06-09 06:17:42 +00:00
|
|
|
)));
|
|
|
|
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&serial) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_io_addresses(Some(GuestAddress(0x3f8)), 0x8, None)
|
2021-03-25 17:01:21 +00:00
|
|
|
.ok_or(DeviceManagerError::AllocateIoPort)?;
|
2020-06-09 06:17:42 +00:00
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.io_bus
|
|
|
|
.insert(serial.clone(), 0x3f8, 0x8)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
|
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, serial));
|
|
|
|
|
|
|
|
Ok(serial)
|
|
|
|
}
|
|
|
|
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
fn add_serial_device(
|
|
|
|
&mut self,
|
|
|
|
interrupt_manager: &Arc<dyn InterruptManager<GroupConfig = LegacyIrqGroupConfig>>,
|
|
|
|
serial_writer: Option<Box<dyn io::Write + Send>>,
|
2021-03-25 17:01:21 +00:00
|
|
|
) -> DeviceManagerResult<Arc<Mutex<Pl011>>> {
|
2022-05-04 09:36:26 +00:00
|
|
|
let id = String::from(SERIAL_DEVICE_NAME);
|
2020-06-09 06:17:42 +00:00
|
|
|
|
|
|
|
let serial_irq = self
|
|
|
|
.address_manager
|
|
|
|
.allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.allocate_irq()
|
|
|
|
.unwrap();
|
|
|
|
|
|
|
|
let interrupt_group = interrupt_manager
|
|
|
|
.create_group(LegacyIrqGroupConfig {
|
|
|
|
irq: serial_irq as InterruptIndex,
|
|
|
|
})
|
|
|
|
.map_err(DeviceManagerError::CreateInterruptGroup)?;
|
|
|
|
|
2021-03-25 17:01:21 +00:00
|
|
|
let serial = Arc::new(Mutex::new(devices::legacy::Pl011::new(
|
2020-06-09 06:17:42 +00:00
|
|
|
id.clone(),
|
|
|
|
interrupt_group,
|
|
|
|
serial_writer,
|
2022-05-04 15:36:57 +00:00
|
|
|
self.timestamp,
|
2022-11-25 16:07:46 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-06-09 06:17:42 +00:00
|
|
|
)));
|
|
|
|
|
|
|
|
self.bus_devices
|
|
|
|
.push(Arc::clone(&serial) as Arc<Mutex<dyn BusDevice>>);
|
|
|
|
|
2022-04-05 09:05:24 +00:00
|
|
|
let addr = arch::layout::LEGACY_SERIAL_MAPPED_IO_START;
|
2020-06-09 06:17:42 +00:00
|
|
|
|
|
|
|
self.address_manager
|
|
|
|
.mmio_bus
|
2021-03-12 05:18:52 +00:00
|
|
|
.insert(serial.clone(), addr.0, MMIO_LEN)
|
2020-06-09 06:17:42 +00:00
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
|
|
|
|
self.id_to_dev_info.insert(
|
|
|
|
(DeviceType::Serial, DeviceType::Serial.to_string()),
|
2021-03-25 17:01:21 +00:00
|
|
|
MmioDeviceInfo {
|
2020-06-09 06:17:42 +00:00
|
|
|
addr: addr.0,
|
2021-11-28 02:52:15 +00:00
|
|
|
len: MMIO_LEN,
|
2020-06-09 06:17:42 +00:00
|
|
|
irq: serial_irq,
|
|
|
|
},
|
|
|
|
);
|
|
|
|
|
|
|
|
self.cmdline_additions
|
2021-03-12 05:18:52 +00:00
|
|
|
.push(format!("earlycon=pl011,mmio,0x{:08x}", addr.0));
|
2020-06-09 06:17:42 +00:00
|
|
|
|
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, serial));
|
|
|
|
|
|
|
|
Ok(serial)
|
|
|
|
}
|
|
|
|
|
2021-01-14 03:03:53 +00:00
|
|
|
fn modify_mode<F: FnOnce(&mut termios)>(
|
vmm: reset to the original termios
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-04-27 12:57:28 +00:00
|
|
|
&mut self,
|
2021-01-14 03:03:53 +00:00
|
|
|
fd: RawFd,
|
|
|
|
f: F,
|
|
|
|
) -> vmm_sys_util::errno::Result<()> {
|
2021-11-17 17:06:47 +00:00
|
|
|
// SAFETY: safe because we check the return value of isatty.
|
2021-01-14 03:03:53 +00:00
|
|
|
if unsafe { isatty(fd) } != 1 {
|
|
|
|
return Ok(());
|
|
|
|
}
|
|
|
|
|
2021-11-17 17:06:47 +00:00
|
|
|
// SAFETY: The following pair are safe because termios gets totally overwritten by tcgetattr
|
|
|
|
// and we check the return result.
|
2021-01-14 03:03:53 +00:00
|
|
|
let mut termios: termios = unsafe { zeroed() };
|
2022-11-16 23:23:22 +00:00
|
|
|
// SAFETY: see above
|
2021-01-14 03:03:53 +00:00
|
|
|
let ret = unsafe { tcgetattr(fd, &mut termios as *mut _) };
|
|
|
|
if ret < 0 {
|
|
|
|
return vmm_sys_util::errno::errno_result();
|
|
|
|
}
|
vmm: reset to the original termios
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-04-27 12:57:28 +00:00
|
|
|
let mut original_termios_opt = self.original_termios_opt.lock().unwrap();
|
|
|
|
if original_termios_opt.is_none() {
|
|
|
|
*original_termios_opt = Some(termios);
|
|
|
|
}
|
2021-01-14 03:03:53 +00:00
|
|
|
f(&mut termios);
|
2021-11-17 17:06:47 +00:00
|
|
|
// SAFETY: Safe because the syscall will only read the extent of termios and we check
|
|
|
|
// the return result.
|
2021-01-14 03:03:53 +00:00
|
|
|
let ret = unsafe { tcsetattr(fd, TCSANOW, &termios as *const _) };
|
|
|
|
if ret < 0 {
|
|
|
|
return vmm_sys_util::errno::errno_result();
|
|
|
|
}
|
|
|
|
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
2023-08-22 10:02:56 +00:00
|
|
|
fn set_raw_mode(&mut self, f: &dyn AsRawFd) -> vmm_sys_util::errno::Result<()> {
|
2021-11-17 17:06:47 +00:00
|
|
|
// SAFETY: FFI call. Variable t is guaranteed to be a valid termios from modify_mode.
|
2021-11-17 14:03:18 +00:00
|
|
|
self.modify_mode(f.as_raw_fd(), |t| unsafe { cfmakeraw(t) })
|
2021-01-14 03:03:53 +00:00
|
|
|
}
|
|
|
|
|
2023-03-23 15:40:56 +00:00
|
|
|
fn listen_for_sigwinch_on_tty(&mut self, pty_sub: File) -> std::io::Result<()> {
|
2022-07-20 22:51:15 +00:00
|
|
|
let seccomp_filter = get_seccomp_filter(
|
|
|
|
&self.seccomp_action,
|
|
|
|
Thread::PtyForeground,
|
|
|
|
self.hypervisor_type,
|
|
|
|
)
|
|
|
|
.unwrap();
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
|
2023-03-23 17:14:35 +00:00
|
|
|
self.console_resize_pipe =
|
|
|
|
Some(Arc::new(start_sigwinch_listener(seccomp_filter, pty_sub)?));
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
2021-09-02 15:56:37 +00:00
|
|
|
fn add_virtio_console_device(
|
|
|
|
&mut self,
|
2022-03-10 13:05:36 +00:00
|
|
|
virtio_devices: &mut Vec<MetaVirtioDevice>,
|
2021-09-02 15:56:37 +00:00
|
|
|
console_pty: Option<PtyPair>,
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
resize_pipe: Option<File>,
|
2021-09-02 15:56:37 +00:00
|
|
|
) -> DeviceManagerResult<Option<Arc<virtio_devices::ConsoleResizer>>> {
|
|
|
|
let console_config = self.config.lock().unwrap().console.clone();
|
|
|
|
let endpoint = match console_config.mode {
|
|
|
|
ConsoleOutputMode::File => {
|
|
|
|
let file = File::create(console_config.file.as_ref().unwrap())
|
|
|
|
.map_err(DeviceManagerError::ConsoleOutputFileOpen)?;
|
|
|
|
Endpoint::File(file)
|
|
|
|
}
|
|
|
|
ConsoleOutputMode::Pty => {
|
|
|
|
if let Some(pty) = console_pty {
|
|
|
|
self.config.lock().unwrap().console.file = Some(pty.path.clone());
|
|
|
|
let file = pty.main.try_clone().unwrap();
|
|
|
|
self.console_pty = Some(Arc::new(Mutex::new(pty)));
|
2022-01-04 10:54:30 +00:00
|
|
|
self.console_resize_pipe = resize_pipe.map(Arc::new);
|
2022-08-25 08:50:05 +00:00
|
|
|
Endpoint::PtyPair(file.try_clone().unwrap(), file)
|
2021-09-02 15:56:37 +00:00
|
|
|
} else {
|
2023-08-22 10:02:56 +00:00
|
|
|
let (main, sub, path) =
|
2022-08-25 08:50:05 +00:00
|
|
|
create_pty().map_err(DeviceManagerError::ConsolePtyOpen)?;
|
2023-08-22 10:02:56 +00:00
|
|
|
self.set_raw_mode(&sub)
|
2021-09-02 15:56:37 +00:00
|
|
|
.map_err(DeviceManagerError::SetPtyRaw)?;
|
|
|
|
self.config.lock().unwrap().console.file = Some(path.clone());
|
|
|
|
let file = main.try_clone().unwrap();
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
assert!(resize_pipe.is_none());
|
2023-03-23 15:40:56 +00:00
|
|
|
self.listen_for_sigwinch_on_tty(sub).unwrap();
|
2022-08-18 16:22:33 +00:00
|
|
|
self.console_pty = Some(Arc::new(Mutex::new(PtyPair { main, path })));
|
2022-08-25 08:50:05 +00:00
|
|
|
Endpoint::PtyPair(file.try_clone().unwrap(), file)
|
2021-09-02 15:56:37 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
ConsoleOutputMode::Tty => {
|
2021-11-17 17:30:42 +00:00
|
|
|
// Duplicating the file descriptors like this is needed as otherwise
|
|
|
|
// they will be closed on a reboot and the numbers reused
|
|
|
|
|
|
|
|
// SAFETY: FFI call to dup. Trivially safe.
|
|
|
|
let stdout = unsafe { libc::dup(libc::STDOUT_FILENO) };
|
|
|
|
if stdout == -1 {
|
|
|
|
return vmm_sys_util::errno::errno_result().map_err(DeviceManagerError::DupFd);
|
|
|
|
}
|
|
|
|
// SAFETY: stdout is valid and owned solely by us.
|
2023-08-22 10:02:56 +00:00
|
|
|
let stdout = unsafe { File::from_raw_fd(stdout) };
|
vmm: only touch the tty flags if it's being used
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-03-22 22:22:46 +00:00
|
|
|
|
|
|
|
// Make sure stdout is in raw mode, if it's a terminal.
|
2023-08-22 10:02:56 +00:00
|
|
|
let _ = self.set_raw_mode(&stdout);
|
2021-11-17 17:30:42 +00:00
|
|
|
|
2023-03-23 17:36:34 +00:00
|
|
|
// SAFETY: FFI call. Trivially safe.
|
|
|
|
if unsafe { libc::isatty(libc::STDOUT_FILENO) } == 1 {
|
|
|
|
self.listen_for_sigwinch_on_tty(stdout.try_clone().unwrap())
|
|
|
|
.unwrap();
|
|
|
|
}
|
|
|
|
|
2021-09-02 15:56:37 +00:00
|
|
|
// If an interactive TTY then we can accept input
|
2021-11-17 17:06:47 +00:00
|
|
|
// SAFETY: FFI call. Trivially safe.
|
2021-09-02 15:56:37 +00:00
|
|
|
if unsafe { libc::isatty(libc::STDIN_FILENO) == 1 } {
|
2021-11-17 17:30:42 +00:00
|
|
|
// SAFETY: FFI call to dup. Trivially safe.
|
|
|
|
let stdin = unsafe { libc::dup(libc::STDIN_FILENO) };
|
|
|
|
if stdin == -1 {
|
|
|
|
return vmm_sys_util::errno::errno_result()
|
|
|
|
.map_err(DeviceManagerError::DupFd);
|
|
|
|
}
|
|
|
|
// SAFETY: stdin is valid and owned solely by us.
|
|
|
|
let stdin = unsafe { File::from_raw_fd(stdin) };
|
|
|
|
|
|
|
|
Endpoint::FilePair(stdout, stdin)
|
2021-09-02 15:56:37 +00:00
|
|
|
} else {
|
2021-11-17 17:30:42 +00:00
|
|
|
Endpoint::File(stdout)
|
2021-09-02 15:56:37 +00:00
|
|
|
}
|
|
|
|
}
|
2023-06-08 19:41:37 +00:00
|
|
|
ConsoleOutputMode::Socket => {
|
|
|
|
return Err(DeviceManagerError::NoSocketOptionSupportForConsoleDevice);
|
|
|
|
}
|
2021-09-02 15:56:37 +00:00
|
|
|
ConsoleOutputMode::Null => Endpoint::Null,
|
|
|
|
ConsoleOutputMode::Off => return Ok(None),
|
|
|
|
};
|
|
|
|
let id = String::from(CONSOLE_DEVICE_NAME);
|
|
|
|
|
|
|
|
let (virtio_console_device, console_resizer) = virtio_devices::Console::new(
|
|
|
|
id.clone(),
|
|
|
|
endpoint,
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
self.console_resize_pipe
|
|
|
|
.as_ref()
|
|
|
|
.map(|p| p.try_clone().unwrap()),
|
2021-09-02 15:56:37 +00:00
|
|
|
self.force_iommu | console_config.iommu,
|
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2021-09-02 15:56:37 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioConsole)?;
|
|
|
|
let virtio_console_device = Arc::new(Mutex::new(virtio_console_device));
|
2022-03-10 13:05:36 +00:00
|
|
|
virtio_devices.push(MetaVirtioDevice {
|
|
|
|
virtio_device: Arc::clone(&virtio_console_device)
|
|
|
|
as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: console_config.iommu,
|
|
|
|
id: id.clone(),
|
|
|
|
pci_segment: 0,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: None,
|
2022-03-10 13:05:36 +00:00
|
|
|
});
|
2021-09-02 15:56:37 +00:00
|
|
|
|
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, virtio_console_device));
|
|
|
|
|
2021-09-08 14:04:49 +00:00
|
|
|
// Only provide a resizer (for SIGWINCH handling) if the console is attached to the TTY
|
|
|
|
Ok(if matches!(console_config.mode, ConsoleOutputMode::Tty) {
|
|
|
|
Some(console_resizer)
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
})
|
2021-09-02 15:56:37 +00:00
|
|
|
}
|
|
|
|
|
2024-01-17 21:12:43 +00:00
|
|
|
/// Adds all devices that behave like a console with respect to the VM
|
|
|
|
/// configuration. This includes:
|
|
|
|
/// - debug-console
|
|
|
|
/// - serial-console
|
|
|
|
/// - virtio-console
|
|
|
|
fn add_console_devices(
|
2020-01-29 15:53:12 +00:00
|
|
|
&mut self,
|
2020-02-04 11:04:10 +00:00
|
|
|
interrupt_manager: &Arc<dyn InterruptManager<GroupConfig = LegacyIrqGroupConfig>>,
|
2022-03-10 13:05:36 +00:00
|
|
|
virtio_devices: &mut Vec<MetaVirtioDevice>,
|
2021-03-04 23:34:45 +00:00
|
|
|
serial_pty: Option<PtyPair>,
|
|
|
|
console_pty: Option<PtyPair>,
|
2024-01-17 21:12:43 +00:00
|
|
|
#[cfg(target_arch = "x86_64")] debug_console_pty: Option<PtyPair>,
|
|
|
|
#[cfg(not(target_arch = "x86_64"))] _: Option<PtyPair>,
|
vmm: notify virtio-console of pty resizes
When a pty is resized (using the TIOCSWINSZ ioctl -- see ioctl_tty(2)),
the kernel will send a SIGWINCH signal to the pty's foreground process
group to notify it of the resize. This is the only way to be notified
by the kernel of a pty resize.
We can't just make the cloud-hypervisor process's process group the
foreground process group though, because a process can only set the
foreground process group of its controlling terminal, and
cloud-hypervisor's controlling terminal will often be the terminal the
user is running it in. To work around this, we fork a subprocess in a
new process group, and set its process group to be the foreground
process group of the pty. The subprocess additionally must be running
in a new session so that it can have a different controlling
terminal. This subprocess writes a byte to a pipe every time the pty
is resized, and the virtio-console device can listen for this in its
epoll loop.
Alternatives I considered were to have the subprocess just send
SIGWINCH to its parent, and to use an eventfd instead of a pipe.
I decided against the signal approach because re-purposing a signal
that has a very specific meaning (even if this use was only slightly
different to its normal meaning) felt unclean, and because it would
have required using pidfds to avoid race conditions if
cloud-hypervisor had terminated, which added complexity. I decided
against using an eventfd because using a pipe instead allows the child
to be notified (via poll(2)) when nothing is reading from the pipe any
more, meaning it can be reliably notified of parent death and
terminate itself immediately.
I used clone3(2) instead of fork(2) because without
CLONE_CLEAR_SIGHAND the subprocess would inherit signal-hook's signal
handlers, and there's no other straightforward way to restore all signal
handlers to their defaults in the child process. The only way to do
it would be to iterate through all possible signals, or maintain a
global list of monitored signals ourselves (vmm:vm::HANDLED_SIGNALS is
insufficient because it doesn't take into account e.g. the SIGSYS
signal handler that catches seccomp violations).
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2021-09-10 11:12:17 +00:00
|
|
|
console_resize_pipe: Option<File>,
|
2019-11-18 10:21:37 +00:00
|
|
|
) -> DeviceManagerResult<Arc<Console>> {
|
2020-01-31 11:42:48 +00:00
|
|
|
let serial_config = self.config.lock().unwrap().serial.clone();
|
2019-12-05 14:50:38 +00:00
|
|
|
let serial_writer: Option<Box<dyn io::Write + Send>> = match serial_config.mode {
|
2019-11-18 10:21:37 +00:00
|
|
|
ConsoleOutputMode::File => Some(Box::new(
|
2019-12-05 14:50:38 +00:00
|
|
|
File::create(serial_config.file.as_ref().unwrap())
|
2019-11-18 10:21:37 +00:00
|
|
|
.map_err(DeviceManagerError::SerialOutputFileOpen)?,
|
|
|
|
)),
|
2021-01-14 03:03:53 +00:00
|
|
|
ConsoleOutputMode::Pty => {
|
2024-01-17 21:12:43 +00:00
|
|
|
if let Some(pty) = serial_pty.clone() {
|
2021-03-04 23:34:45 +00:00
|
|
|
self.config.lock().unwrap().serial.file = Some(pty.path.clone());
|
|
|
|
self.serial_pty = Some(Arc::new(Mutex::new(pty)));
|
|
|
|
} else {
|
2023-08-22 10:02:56 +00:00
|
|
|
let (main, sub, path) =
|
2022-08-25 08:50:05 +00:00
|
|
|
create_pty().map_err(DeviceManagerError::SerialPtyOpen)?;
|
2023-08-22 10:02:56 +00:00
|
|
|
self.set_raw_mode(&sub)
|
2021-03-04 23:34:45 +00:00
|
|
|
.map_err(DeviceManagerError::SetPtyRaw)?;
|
|
|
|
self.config.lock().unwrap().serial.file = Some(path.clone());
|
2022-08-18 16:22:33 +00:00
|
|
|
self.serial_pty = Some(Arc::new(Mutex::new(PtyPair { main, path })));
|
2021-03-04 23:34:45 +00:00
|
|
|
}
|
2021-09-24 05:32:41 +00:00
|
|
|
None
|
2021-01-14 03:03:53 +00:00
|
|
|
}
|
vmm: only touch the tty flags if it's being used
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-03-22 22:22:46 +00:00
|
|
|
ConsoleOutputMode::Tty => {
|
2023-08-22 10:02:56 +00:00
|
|
|
let out = stdout();
|
|
|
|
let _ = self.set_raw_mode(&out);
|
vmm: only touch the tty flags if it's being used
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-03-22 22:22:46 +00:00
|
|
|
Some(Box::new(out))
|
|
|
|
}
|
2023-06-08 19:41:37 +00:00
|
|
|
ConsoleOutputMode::Off | ConsoleOutputMode::Null | ConsoleOutputMode::Socket => None,
|
2019-11-18 10:21:37 +00:00
|
|
|
};
|
2021-09-16 19:47:36 +00:00
|
|
|
if serial_config.mode != ConsoleOutputMode::Off {
|
|
|
|
let serial = self.add_serial_device(interrupt_manager, serial_writer)?;
|
|
|
|
self.serial_manager = match serial_config.mode {
|
2023-06-08 19:41:37 +00:00
|
|
|
ConsoleOutputMode::Pty | ConsoleOutputMode::Tty | ConsoleOutputMode::Socket => {
|
|
|
|
let serial_manager = SerialManager::new(
|
|
|
|
serial,
|
|
|
|
self.serial_pty.clone(),
|
|
|
|
serial_config.mode,
|
|
|
|
serial_config.socket,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateSerialManager)?;
|
2021-09-16 19:47:36 +00:00
|
|
|
if let Some(mut serial_manager) = serial_manager {
|
|
|
|
serial_manager
|
|
|
|
.start_thread(
|
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::SpawnSerialManager)?;
|
|
|
|
Some(Arc::new(serial_manager))
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
}
|
|
|
|
}
|
|
|
|
_ => None,
|
|
|
|
};
|
|
|
|
}
|
2019-11-18 10:21:37 +00:00
|
|
|
|
2024-01-17 21:12:43 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
{
|
|
|
|
let debug_console_config = self.config.lock().unwrap().debug_console.clone();
|
|
|
|
let debug_console_writer: Option<Box<dyn io::Write + Send>> = match debug_console_config
|
|
|
|
.mode
|
|
|
|
{
|
|
|
|
ConsoleOutputMode::File => Some(Box::new(
|
|
|
|
File::create(debug_console_config.file.as_ref().unwrap())
|
|
|
|
.map_err(DeviceManagerError::DebugconOutputFileOpen)?,
|
|
|
|
)),
|
|
|
|
ConsoleOutputMode::Pty => {
|
|
|
|
if let Some(pty) = debug_console_pty {
|
|
|
|
self.config.lock().unwrap().debug_console.file = Some(pty.path.clone());
|
|
|
|
self.debug_console_pty = Some(Arc::new(Mutex::new(pty)));
|
|
|
|
} else {
|
|
|
|
let (main, sub, path) =
|
|
|
|
create_pty().map_err(DeviceManagerError::DebugconPtyOpen)?;
|
|
|
|
self.set_raw_mode(&sub)
|
|
|
|
.map_err(DeviceManagerError::SetPtyRaw)?;
|
|
|
|
self.config.lock().unwrap().debug_console.file = Some(path.clone());
|
|
|
|
self.debug_console_pty = Some(Arc::new(Mutex::new(PtyPair { main, path })));
|
|
|
|
}
|
|
|
|
None
|
|
|
|
}
|
|
|
|
ConsoleOutputMode::Tty => {
|
|
|
|
let out = stdout();
|
|
|
|
let _ = self.set_raw_mode(&out);
|
|
|
|
Some(Box::new(out))
|
|
|
|
}
|
|
|
|
ConsoleOutputMode::Off | ConsoleOutputMode::Null | ConsoleOutputMode::Socket => {
|
|
|
|
None
|
|
|
|
}
|
|
|
|
};
|
|
|
|
if let Some(writer) = debug_console_writer {
|
|
|
|
let _ = self.add_debug_console_device(writer)?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
vmm: reset to the original termios
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-04-27 12:57:28 +00:00
|
|
|
let console_resizer =
|
|
|
|
self.add_virtio_console_device(virtio_devices, console_pty, console_resize_pipe)?;
|
2019-11-18 10:21:37 +00:00
|
|
|
|
2021-09-16 19:47:36 +00:00
|
|
|
Ok(Arc::new(Console { console_resizer }))
|
2019-11-18 10:21:37 +00:00
|
|
|
}
|
|
|
|
|
2022-08-15 15:34:54 +00:00
|
|
|
fn add_tpm_device(
|
|
|
|
&mut self,
|
|
|
|
tpm_path: PathBuf,
|
|
|
|
) -> DeviceManagerResult<Arc<Mutex<devices::tpm::Tpm>>> {
|
|
|
|
// Create TPM Device
|
|
|
|
let tpm = devices::tpm::Tpm::new(tpm_path.to_str().unwrap().to_string()).map_err(|e| {
|
|
|
|
DeviceManagerError::CreateTpmDevice(anyhow!("Failed to create TPM Device : {:?}", e))
|
|
|
|
})?;
|
|
|
|
let tpm = Arc::new(Mutex::new(tpm));
|
|
|
|
|
|
|
|
// Add TPM Device to mmio
|
|
|
|
self.address_manager
|
|
|
|
.mmio_bus
|
|
|
|
.insert(
|
|
|
|
tpm.clone(),
|
|
|
|
arch::layout::TPM_START.0,
|
|
|
|
arch::layout::TPM_SIZE,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::BusError)?;
|
|
|
|
|
|
|
|
Ok(tpm)
|
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
|
|
|
let mut devices: Vec<MetaVirtioDevice> = Vec::new();
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2019-09-10 16:14:16 +00:00
|
|
|
// Create "standard" virtio devices (net/block/rng)
|
2020-01-31 11:42:48 +00:00
|
|
|
devices.append(&mut self.make_virtio_block_devices()?);
|
|
|
|
devices.append(&mut self.make_virtio_net_devices()?);
|
|
|
|
devices.append(&mut self.make_virtio_rng_devices()?);
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
// Add virtio-fs if required
|
2020-01-31 11:42:48 +00:00
|
|
|
devices.append(&mut self.make_virtio_fs_devices()?);
|
2019-09-04 13:55:14 +00:00
|
|
|
|
|
|
|
// Add virtio-pmem if required
|
2020-01-31 11:42:48 +00:00
|
|
|
devices.append(&mut self.make_virtio_pmem_devices()?);
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2019-09-04 18:14:54 +00:00
|
|
|
// Add virtio-vsock if required
|
2020-04-27 08:15:30 +00:00
|
|
|
devices.append(&mut self.make_virtio_vsock_devices()?);
|
2019-09-04 18:14:54 +00:00
|
|
|
|
2020-03-04 02:16:27 +00:00
|
|
|
devices.append(&mut self.make_virtio_mem_devices()?);
|
|
|
|
|
2020-03-20 03:43:37 +00:00
|
|
|
// Add virtio-balloon if required
|
|
|
|
devices.append(&mut self.make_virtio_balloon_devices()?);
|
|
|
|
|
2020-09-25 09:39:45 +00:00
|
|
|
// Add virtio-watchdog device
|
|
|
|
devices.append(&mut self.make_virtio_watchdog_devices()?);
|
|
|
|
|
2022-03-11 11:31:25 +00:00
|
|
|
// Add vDPA devices if required
|
|
|
|
devices.append(&mut self.make_vdpa_devices()?);
|
|
|
|
|
2019-09-10 16:14:16 +00:00
|
|
|
Ok(devices)
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2023-10-19 02:59:22 +00:00
|
|
|
// Cache whether aio is supported to avoid checking for very block device
|
|
|
|
fn aio_is_supported(&mut self) -> bool {
|
|
|
|
if let Some(supported) = self.aio_supported {
|
|
|
|
return supported;
|
|
|
|
}
|
|
|
|
|
|
|
|
let supported = block_aio_is_supported();
|
|
|
|
self.aio_supported = Some(supported);
|
|
|
|
supported
|
|
|
|
}
|
|
|
|
|
vmm: Cache whether io_uring is supported in DeviceManager
Probing for whether the io_uring is supported is time consuming so cache
this value if it is known to reduce the cost for secondary block devices
that are added.
Before:
cloud-hypervisor: 3.988896ms: <vmm> INFO:vmm/src/device_manager.rs:1901 -- Creating virtio-block device: DiskConfig { path: Some("/home/rob/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk0"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.129591ms: <vmm> INFO:vmm/src/device_manager.rs:1983 -- Using asynchronous RAW disk file (io_uring)
cloud-hypervisor: 14.159853ms: <vmm> INFO:vmm/src/device_manager.rs:1901 -- Creating virtio-block device: DiskConfig { path: Some("/tmp/disk"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk1"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 22.110281ms: <vmm> INFO:vmm/src/device_manager.rs:1983 -- Using asynchronous RAW disk file (io_uring)
After:
cloud-hypervisor: 4.880411ms: <vmm> INFO:vmm/src/device_manager.rs:1916 -- Creating virtio-block device: DiskConfig { path: Some("/home/rob/workloads/focal-server-cloudimg-amd64-custom-20210609-0.raw"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk0"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.105123ms: <vmm> INFO:vmm/src/device_manager.rs:1998 -- Using asynchronous RAW disk file (io_uring)
cloud-hypervisor: 14.134837ms: <vmm> INFO:vmm/src/device_manager.rs:1916 -- Creating virtio-block device: DiskConfig { path: Some("/tmp/disk"), readonly: false, direct: false, iommu: false, num_queues: 1, queue_size: 128, vhost_user: false, vhost_socket: None, poll_queue: true, rate_limiter_config: None, id: Some("_disk1"), disable_io_uring: false, pci_segment: 0 }
cloud-hypervisor: 14.221869ms: <vmm> INFO:vmm/src/device_manager.rs:1998 -- Using asynchronous RAW disk file (io_uring)
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2021-11-12 10:38:51 +00:00
|
|
|
// Cache whether io_uring is supported to avoid probing for very block device
|
|
|
|
fn io_uring_is_supported(&mut self) -> bool {
|
|
|
|
if let Some(supported) = self.io_uring_supported {
|
|
|
|
return supported;
|
|
|
|
}
|
|
|
|
|
|
|
|
let supported = block_io_uring_is_supported();
|
|
|
|
self.io_uring_supported = Some(supported);
|
|
|
|
supported
|
|
|
|
}
|
|
|
|
|
2020-03-13 09:38:42 +00:00
|
|
|
fn make_virtio_block_device(
|
|
|
|
&mut self,
|
2020-04-15 16:09:12 +00:00
|
|
|
disk_cfg: &mut DiskConfig,
|
2022-03-10 13:05:36 +00:00
|
|
|
) -> DeviceManagerResult<MetaVirtioDevice> {
|
2020-04-27 09:21:15 +00:00
|
|
|
let id = if let Some(id) = &disk_cfg.id {
|
|
|
|
id.clone()
|
|
|
|
} else {
|
|
|
|
let id = self.next_device_name(DISK_DEVICE_NAME_PREFIX)?;
|
|
|
|
disk_cfg.id = Some(id.clone());
|
|
|
|
id
|
|
|
|
};
|
2020-04-15 16:09:12 +00:00
|
|
|
|
2021-05-18 14:32:20 +00:00
|
|
|
info!("Creating virtio-block device: {:?}", disk_cfg);
|
|
|
|
|
2022-10-18 15:14:43 +00:00
|
|
|
let snapshot = snapshot_from_id(self.snapshot.as_ref(), id.as_str());
|
|
|
|
|
2022-05-05 08:38:17 +00:00
|
|
|
let (virtio_device, migratable_device) = if disk_cfg.vhost_user {
|
2020-11-06 10:14:24 +00:00
|
|
|
let socket = disk_cfg.vhost_socket.as_ref().unwrap().clone();
|
2020-03-13 09:38:42 +00:00
|
|
|
let vu_cfg = VhostUserConfig {
|
2020-11-06 10:14:24 +00:00
|
|
|
socket,
|
2020-03-13 09:38:42 +00:00
|
|
|
num_queues: disk_cfg.num_queues,
|
|
|
|
queue_size: disk_cfg.queue_size,
|
|
|
|
};
|
2022-05-05 08:38:17 +00:00
|
|
|
let vhost_user_block = Arc::new(Mutex::new(
|
2021-09-03 10:43:30 +00:00
|
|
|
match virtio_devices::vhost_user::Blk::new(
|
|
|
|
id.clone(),
|
|
|
|
vu_cfg,
|
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-03-29 19:27:05 +00:00
|
|
|
self.force_iommu,
|
2022-10-18 15:14:43 +00:00
|
|
|
snapshot
|
2022-12-02 13:37:49 +00:00
|
|
|
.map(|s| s.to_versioned_state())
|
2022-10-18 15:14:43 +00:00
|
|
|
.transpose()
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2021-09-03 10:43:30 +00:00
|
|
|
) {
|
2020-09-18 15:22:46 +00:00
|
|
|
Ok(vub_device) => vub_device,
|
|
|
|
Err(e) => {
|
|
|
|
return Err(DeviceManagerError::CreateVhostUserBlk(e));
|
|
|
|
}
|
|
|
|
},
|
2020-03-13 09:38:42 +00:00
|
|
|
));
|
|
|
|
|
2022-05-05 08:38:17 +00:00
|
|
|
(
|
|
|
|
Arc::clone(&vhost_user_block) as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
vhost_user_block as Arc<Mutex<dyn Migratable>>,
|
|
|
|
)
|
2020-03-13 09:38:42 +00:00
|
|
|
} else {
|
|
|
|
let mut options = OpenOptions::new();
|
|
|
|
options.read(true);
|
|
|
|
options.write(!disk_cfg.readonly);
|
|
|
|
if disk_cfg.direct {
|
|
|
|
options.custom_flags(libc::O_DIRECT);
|
|
|
|
}
|
|
|
|
// Open block device path
|
2021-01-28 08:11:25 +00:00
|
|
|
let mut file: File = options
|
2020-03-13 09:38:42 +00:00
|
|
|
.open(
|
|
|
|
disk_cfg
|
|
|
|
.path
|
|
|
|
.as_ref()
|
|
|
|
.ok_or(DeviceManagerError::NoDiskPath)?
|
|
|
|
.clone(),
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::Disk)?;
|
2021-01-28 08:11:25 +00:00
|
|
|
let image_type =
|
|
|
|
detect_image_type(&mut file).map_err(DeviceManagerError::DetectImageType)?;
|
2021-01-22 08:56:38 +00:00
|
|
|
|
|
|
|
let image = match image_type {
|
2021-01-28 08:11:25 +00:00
|
|
|
ImageType::FixedVhd => {
|
|
|
|
// Use asynchronous backend relying on io_uring if the
|
|
|
|
// syscalls are supported.
|
2023-07-10 09:55:55 +00:00
|
|
|
if cfg!(feature = "io_uring")
|
|
|
|
&& !disk_cfg.disable_io_uring
|
|
|
|
&& self.io_uring_is_supported()
|
|
|
|
{
|
2021-01-28 08:11:25 +00:00
|
|
|
info!("Using asynchronous fixed VHD disk file (io_uring)");
|
2023-07-10 09:55:55 +00:00
|
|
|
|
|
|
|
#[cfg(not(feature = "io_uring"))]
|
|
|
|
unreachable!("Checked in if statement above");
|
|
|
|
#[cfg(feature = "io_uring")]
|
|
|
|
{
|
|
|
|
Box::new(
|
|
|
|
FixedVhdDiskAsync::new(file)
|
|
|
|
.map_err(DeviceManagerError::CreateFixedVhdDiskAsync)?,
|
|
|
|
) as Box<dyn DiskFile>
|
|
|
|
}
|
2021-01-28 08:11:25 +00:00
|
|
|
} else {
|
2021-01-28 15:23:03 +00:00
|
|
|
info!("Using synchronous fixed VHD disk file");
|
|
|
|
Box::new(
|
|
|
|
FixedVhdDiskSync::new(file)
|
|
|
|
.map_err(DeviceManagerError::CreateFixedVhdDiskSync)?,
|
|
|
|
) as Box<dyn DiskFile>
|
2021-01-28 08:11:25 +00:00
|
|
|
}
|
|
|
|
}
|
2020-03-13 09:38:42 +00:00
|
|
|
ImageType::Raw => {
|
2020-10-07 09:23:20 +00:00
|
|
|
// Use asynchronous backend relying on io_uring if the
|
|
|
|
// syscalls are supported.
|
2023-07-10 09:55:55 +00:00
|
|
|
if cfg!(feature = "io_uring")
|
|
|
|
&& !disk_cfg.disable_io_uring
|
|
|
|
&& self.io_uring_is_supported()
|
|
|
|
{
|
2021-01-22 13:19:09 +00:00
|
|
|
info!("Using asynchronous RAW disk file (io_uring)");
|
2023-07-10 09:55:55 +00:00
|
|
|
|
|
|
|
#[cfg(not(feature = "io_uring"))]
|
|
|
|
unreachable!("Checked in if statement above");
|
|
|
|
#[cfg(feature = "io_uring")]
|
|
|
|
{
|
|
|
|
Box::new(RawFileDisk::new(file)) as Box<dyn DiskFile>
|
|
|
|
}
|
2023-10-19 02:59:22 +00:00
|
|
|
} else if !disk_cfg.disable_aio && self.aio_is_supported() {
|
|
|
|
info!("Using asynchronous RAW disk file (aio)");
|
|
|
|
Box::new(RawFileDiskAio::new(file)) as Box<dyn DiskFile>
|
2020-10-07 09:23:20 +00:00
|
|
|
} else {
|
2021-01-22 13:19:09 +00:00
|
|
|
info!("Using synchronous RAW disk file");
|
2021-01-28 16:29:33 +00:00
|
|
|
Box::new(RawFileDiskSync::new(file)) as Box<dyn DiskFile>
|
2021-01-22 08:56:38 +00:00
|
|
|
}
|
2020-03-13 09:38:42 +00:00
|
|
|
}
|
|
|
|
ImageType::Qcow2 => {
|
2021-01-22 13:19:09 +00:00
|
|
|
info!("Using synchronous QCOW disk file");
|
2021-08-11 22:12:48 +00:00
|
|
|
Box::new(
|
|
|
|
QcowDiskSync::new(file, disk_cfg.direct)
|
|
|
|
.map_err(DeviceManagerError::CreateQcowDiskSync)?,
|
|
|
|
) as Box<dyn DiskFile>
|
2020-03-13 09:38:42 +00:00
|
|
|
}
|
2021-07-26 15:51:03 +00:00
|
|
|
ImageType::Vhdx => {
|
|
|
|
info!("Using synchronous VHDX disk file");
|
|
|
|
Box::new(
|
|
|
|
VhdxDiskSync::new(file)
|
|
|
|
.map_err(DeviceManagerError::CreateFixedVhdxDiskSync)?,
|
|
|
|
) as Box<dyn DiskFile>
|
|
|
|
}
|
2020-07-30 10:44:05 +00:00
|
|
|
};
|
2021-01-22 08:56:38 +00:00
|
|
|
|
2023-12-07 19:45:08 +00:00
|
|
|
let rate_limit_group =
|
|
|
|
if let Some(rate_limiter_cfg) = disk_cfg.rate_limiter_config.as_ref() {
|
|
|
|
// Create an anonymous RateLimiterGroup that is dropped when the Disk
|
|
|
|
// is dropped.
|
|
|
|
let bw = rate_limiter_cfg.bandwidth.unwrap_or_default();
|
|
|
|
let ops = rate_limiter_cfg.ops.unwrap_or_default();
|
|
|
|
let mut rate_limit_group = RateLimiterGroup::new(
|
|
|
|
disk_cfg.id.as_ref().unwrap(),
|
|
|
|
bw.size,
|
|
|
|
bw.one_time_burst.unwrap_or(0),
|
|
|
|
bw.refill_time,
|
|
|
|
ops.size,
|
|
|
|
ops.one_time_burst.unwrap_or(0),
|
|
|
|
ops.refill_time,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::RateLimiterGroupCreate)?;
|
|
|
|
|
|
|
|
rate_limit_group
|
|
|
|
.start_thread(
|
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
|
|
|
)
|
|
|
|
.unwrap();
|
|
|
|
|
|
|
|
Some(Arc::new(rate_limit_group))
|
|
|
|
} else if let Some(rate_limit_group) = disk_cfg.rate_limit_group.as_ref() {
|
|
|
|
self.rate_limit_groups.get(rate_limit_group).cloned()
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
};
|
|
|
|
|
2024-01-25 00:37:47 +00:00
|
|
|
let queue_affinity = if let Some(queue_affinity) = disk_cfg.queue_affinity.as_ref() {
|
|
|
|
queue_affinity
|
|
|
|
.iter()
|
|
|
|
.map(|a| (a.queue_index, a.host_cpus.clone()))
|
|
|
|
.collect()
|
|
|
|
} else {
|
|
|
|
BTreeMap::new()
|
|
|
|
};
|
|
|
|
|
2022-05-05 08:38:17 +00:00
|
|
|
let virtio_block = Arc::new(Mutex::new(
|
2021-01-22 10:15:13 +00:00
|
|
|
virtio_devices::Block::new(
|
2021-01-22 08:56:38 +00:00
|
|
|
id.clone(),
|
|
|
|
image,
|
|
|
|
disk_cfg
|
|
|
|
.path
|
|
|
|
.as_ref()
|
|
|
|
.ok_or(DeviceManagerError::NoDiskPath)?
|
|
|
|
.clone(),
|
|
|
|
disk_cfg.readonly,
|
2021-07-20 07:59:32 +00:00
|
|
|
self.force_iommu | disk_cfg.iommu,
|
2021-01-22 08:56:38 +00:00
|
|
|
disk_cfg.num_queues,
|
|
|
|
disk_cfg.queue_size,
|
2023-09-08 15:18:51 +00:00
|
|
|
disk_cfg.serial.clone(),
|
2021-01-22 08:56:38 +00:00
|
|
|
self.seccomp_action.clone(),
|
2023-12-07 19:45:08 +00:00
|
|
|
rate_limit_group,
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
snapshot
|
2022-12-02 13:37:49 +00:00
|
|
|
.map(|s| s.to_versioned_state())
|
2022-10-18 15:14:43 +00:00
|
|
|
.transpose()
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2024-01-25 00:37:47 +00:00
|
|
|
queue_affinity,
|
2021-01-22 08:56:38 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioBlock)?,
|
|
|
|
));
|
|
|
|
|
2022-05-05 08:38:17 +00:00
|
|
|
(
|
|
|
|
Arc::clone(&virtio_block) as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
virtio_block as Arc<Mutex<dyn Migratable>>,
|
|
|
|
)
|
|
|
|
};
|
2021-01-22 08:56:38 +00:00
|
|
|
|
2022-05-05 08:38:17 +00:00
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, migratable_device));
|
|
|
|
|
|
|
|
Ok(MetaVirtioDevice {
|
|
|
|
virtio_device,
|
|
|
|
iommu: disk_cfg.iommu,
|
|
|
|
id,
|
|
|
|
pci_segment: disk_cfg.pci_segment,
|
|
|
|
dma_handler: None,
|
|
|
|
})
|
2020-03-13 09:38:42 +00:00
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_block_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2019-09-10 16:14:16 +00:00
|
|
|
let mut devices = Vec::new();
|
|
|
|
|
2020-04-15 16:09:12 +00:00
|
|
|
let mut block_devices = self.config.lock().unwrap().disks.clone();
|
|
|
|
if let Some(disk_list_cfg) = &mut block_devices {
|
|
|
|
for disk_cfg in disk_list_cfg.iter_mut() {
|
2020-03-13 09:38:42 +00:00
|
|
|
devices.push(self.make_virtio_block_device(disk_cfg)?);
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
}
|
2020-04-15 16:09:12 +00:00
|
|
|
self.config.lock().unwrap().disks = block_devices;
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2019-09-10 16:14:16 +00:00
|
|
|
Ok(devices)
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2020-03-13 09:38:42 +00:00
|
|
|
fn make_virtio_net_device(
|
|
|
|
&mut self,
|
2020-04-15 16:09:12 +00:00
|
|
|
net_cfg: &mut NetConfig,
|
2022-03-10 13:05:36 +00:00
|
|
|
) -> DeviceManagerResult<MetaVirtioDevice> {
|
2020-04-27 09:29:16 +00:00
|
|
|
let id = if let Some(id) = &net_cfg.id {
|
|
|
|
id.clone()
|
|
|
|
} else {
|
|
|
|
let id = self.next_device_name(NET_DEVICE_NAME_PREFIX)?;
|
|
|
|
net_cfg.id = Some(id.clone());
|
|
|
|
id
|
|
|
|
};
|
2021-05-18 14:32:20 +00:00
|
|
|
info!("Creating virtio-net device: {:?}", net_cfg);
|
2020-04-15 16:09:12 +00:00
|
|
|
|
2022-10-18 15:14:43 +00:00
|
|
|
let snapshot = snapshot_from_id(self.snapshot.as_ref(), id.as_str());
|
|
|
|
|
2022-05-05 09:08:32 +00:00
|
|
|
let (virtio_device, migratable_device) = if net_cfg.vhost_user {
|
2020-11-06 10:14:24 +00:00
|
|
|
let socket = net_cfg.vhost_socket.as_ref().unwrap().clone();
|
2020-03-13 09:38:42 +00:00
|
|
|
let vu_cfg = VhostUserConfig {
|
2020-11-06 10:14:24 +00:00
|
|
|
socket,
|
2020-03-13 09:38:42 +00:00
|
|
|
num_queues: net_cfg.num_queues,
|
|
|
|
queue_size: net_cfg.queue_size,
|
|
|
|
};
|
2021-05-04 08:33:35 +00:00
|
|
|
let server = match net_cfg.vhost_mode {
|
|
|
|
VhostMode::Client => false,
|
|
|
|
VhostMode::Server => true,
|
|
|
|
};
|
2022-05-05 09:08:32 +00:00
|
|
|
let vhost_user_net = Arc::new(Mutex::new(
|
2021-05-25 15:14:53 +00:00
|
|
|
match virtio_devices::vhost_user::Net::new(
|
|
|
|
id.clone(),
|
|
|
|
net_cfg.mac,
|
2022-09-26 13:58:54 +00:00
|
|
|
net_cfg.mtu,
|
2021-05-25 15:14:53 +00:00
|
|
|
vu_cfg,
|
|
|
|
server,
|
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-03-29 19:27:05 +00:00
|
|
|
self.force_iommu,
|
2022-10-18 15:14:43 +00:00
|
|
|
snapshot
|
2022-12-02 13:37:49 +00:00
|
|
|
.map(|s| s.to_versioned_state())
|
2022-10-18 15:14:43 +00:00
|
|
|
.transpose()
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2022-12-20 12:04:08 +00:00
|
|
|
net_cfg.offload_tso,
|
|
|
|
net_cfg.offload_ufo,
|
|
|
|
net_cfg.offload_csum,
|
2021-05-25 15:14:53 +00:00
|
|
|
) {
|
2020-09-18 15:22:46 +00:00
|
|
|
Ok(vun_device) => vun_device,
|
|
|
|
Err(e) => {
|
|
|
|
return Err(DeviceManagerError::CreateVhostUserNet(e));
|
|
|
|
}
|
|
|
|
},
|
2020-03-13 09:38:42 +00:00
|
|
|
));
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2022-05-05 09:08:32 +00:00
|
|
|
(
|
|
|
|
Arc::clone(&vhost_user_net) as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
vhost_user_net as Arc<Mutex<dyn Migratable>>,
|
|
|
|
)
|
2020-03-13 09:38:42 +00:00
|
|
|
} else {
|
2022-10-18 15:14:43 +00:00
|
|
|
let state = snapshot
|
2022-12-02 13:37:49 +00:00
|
|
|
.map(|s| s.to_versioned_state())
|
2022-10-18 15:14:43 +00:00
|
|
|
.transpose()
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?;
|
|
|
|
|
2022-05-05 09:08:32 +00:00
|
|
|
let virtio_net = if let Some(ref tap_if_name) = net_cfg.tap {
|
2020-03-13 09:38:42 +00:00
|
|
|
Arc::new(Mutex::new(
|
2020-07-02 12:25:19 +00:00
|
|
|
virtio_devices::Net::new(
|
2020-04-27 12:38:24 +00:00
|
|
|
id.clone(),
|
2020-03-13 09:38:42 +00:00
|
|
|
Some(tap_if_name),
|
2023-12-04 10:21:07 +00:00
|
|
|
Some(net_cfg.ip),
|
|
|
|
Some(net_cfg.mask),
|
2020-03-13 09:38:42 +00:00
|
|
|
Some(net_cfg.mac),
|
2020-06-05 11:00:34 +00:00
|
|
|
&mut net_cfg.host_mac,
|
2022-09-26 13:58:54 +00:00
|
|
|
net_cfg.mtu,
|
2021-07-20 07:59:32 +00:00
|
|
|
self.force_iommu | net_cfg.iommu,
|
2020-03-13 09:38:42 +00:00
|
|
|
net_cfg.num_queues,
|
|
|
|
net_cfg.queue_size,
|
2020-08-04 18:27:17 +00:00
|
|
|
self.seccomp_action.clone(),
|
2021-03-17 22:41:52 +00:00
|
|
|
net_cfg.rate_limiter_config,
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
state,
|
2022-12-20 12:04:08 +00:00
|
|
|
net_cfg.offload_tso,
|
|
|
|
net_cfg.offload_ufo,
|
|
|
|
net_cfg.offload_csum,
|
2020-03-13 09:38:42 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioNet)?,
|
|
|
|
))
|
2021-01-27 04:52:50 +00:00
|
|
|
} else if let Some(fds) = &net_cfg.fds {
|
2023-04-15 00:47:58 +00:00
|
|
|
let net = virtio_devices::Net::from_tap_fds(
|
|
|
|
id.clone(),
|
|
|
|
fds,
|
|
|
|
Some(net_cfg.mac),
|
|
|
|
net_cfg.mtu,
|
|
|
|
self.force_iommu | net_cfg.iommu,
|
|
|
|
net_cfg.queue_size,
|
|
|
|
self.seccomp_action.clone(),
|
|
|
|
net_cfg.rate_limiter_config,
|
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
|
|
|
state,
|
|
|
|
net_cfg.offload_tso,
|
|
|
|
net_cfg.offload_ufo,
|
|
|
|
net_cfg.offload_csum,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioNet)?;
|
|
|
|
|
|
|
|
// SAFETY: 'fds' are valid because TAP devices are created successfully
|
|
|
|
unsafe {
|
|
|
|
self.config.lock().unwrap().add_preserved_fds(fds.clone());
|
|
|
|
}
|
|
|
|
|
|
|
|
Arc::new(Mutex::new(net))
|
2020-03-13 09:38:42 +00:00
|
|
|
} else {
|
|
|
|
Arc::new(Mutex::new(
|
2020-07-02 12:25:19 +00:00
|
|
|
virtio_devices::Net::new(
|
2020-04-27 12:38:24 +00:00
|
|
|
id.clone(),
|
2020-03-13 09:38:42 +00:00
|
|
|
None,
|
|
|
|
Some(net_cfg.ip),
|
|
|
|
Some(net_cfg.mask),
|
|
|
|
Some(net_cfg.mac),
|
2020-06-05 11:00:34 +00:00
|
|
|
&mut net_cfg.host_mac,
|
2022-09-26 13:58:54 +00:00
|
|
|
net_cfg.mtu,
|
2021-07-20 07:59:32 +00:00
|
|
|
self.force_iommu | net_cfg.iommu,
|
2020-03-13 09:38:42 +00:00
|
|
|
net_cfg.num_queues,
|
|
|
|
net_cfg.queue_size,
|
2020-08-04 18:27:17 +00:00
|
|
|
self.seccomp_action.clone(),
|
2021-03-17 22:41:52 +00:00
|
|
|
net_cfg.rate_limiter_config,
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
state,
|
2022-12-20 12:04:08 +00:00
|
|
|
net_cfg.offload_tso,
|
|
|
|
net_cfg.offload_ufo,
|
|
|
|
net_cfg.offload_csum,
|
2020-03-13 09:38:42 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioNet)?,
|
|
|
|
))
|
|
|
|
};
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2022-05-05 09:08:32 +00:00
|
|
|
(
|
|
|
|
Arc::clone(&virtio_net) as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
virtio_net as Arc<Mutex<dyn Migratable>>,
|
|
|
|
)
|
|
|
|
};
|
|
|
|
|
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, migratable_device));
|
|
|
|
|
|
|
|
Ok(MetaVirtioDevice {
|
|
|
|
virtio_device,
|
|
|
|
iommu: net_cfg.iommu,
|
|
|
|
id,
|
|
|
|
pci_segment: net_cfg.pci_segment,
|
|
|
|
dma_handler: None,
|
|
|
|
})
|
2020-03-13 09:38:42 +00:00
|
|
|
}
|
|
|
|
|
2020-01-27 15:14:07 +00:00
|
|
|
/// Add virto-net and vhost-user-net devices
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_net_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2019-09-10 16:14:16 +00:00
|
|
|
let mut devices = Vec::new();
|
2020-04-15 16:09:12 +00:00
|
|
|
let mut net_devices = self.config.lock().unwrap().net.clone();
|
|
|
|
if let Some(net_list_cfg) = &mut net_devices {
|
|
|
|
for net_cfg in net_list_cfg.iter_mut() {
|
2020-03-13 09:38:42 +00:00
|
|
|
devices.push(self.make_virtio_net_device(net_cfg)?);
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
}
|
2020-04-15 16:09:12 +00:00
|
|
|
self.config.lock().unwrap().net = net_devices;
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2019-09-10 16:14:16 +00:00
|
|
|
Ok(devices)
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_rng_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2019-09-10 16:14:16 +00:00
|
|
|
let mut devices = Vec::new();
|
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
// Add virtio-rng if required
|
2020-01-31 11:42:48 +00:00
|
|
|
let rng_config = self.config.lock().unwrap().rng.clone();
|
2019-12-05 14:50:38 +00:00
|
|
|
if let Some(rng_path) = rng_config.src.to_str() {
|
2021-05-18 14:32:20 +00:00
|
|
|
info!("Creating virtio-rng device: {:?}", rng_config);
|
2020-04-27 07:47:26 +00:00
|
|
|
let id = String::from(RNG_DEVICE_NAME);
|
|
|
|
|
2019-11-18 23:10:42 +00:00
|
|
|
let virtio_rng_device = Arc::new(Mutex::new(
|
2020-08-04 17:46:49 +00:00
|
|
|
virtio_devices::Rng::new(
|
|
|
|
id.clone(),
|
|
|
|
rng_path,
|
2021-07-20 07:59:32 +00:00
|
|
|
self.force_iommu | rng_config.iommu,
|
2020-08-04 17:46:49 +00:00
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-08-04 17:46:49 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioRng)?,
|
2019-11-18 23:10:42 +00:00
|
|
|
));
|
2022-03-10 13:05:36 +00:00
|
|
|
devices.push(MetaVirtioDevice {
|
|
|
|
virtio_device: Arc::clone(&virtio_rng_device)
|
|
|
|
as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: rng_config.iommu,
|
|
|
|
id: id.clone(),
|
|
|
|
pci_segment: 0,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: None,
|
2022-03-10 13:05:36 +00:00
|
|
|
});
|
2019-11-19 00:53:23 +00:00
|
|
|
|
2020-04-30 18:08:04 +00:00
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
2020-05-05 08:23:32 +00:00
|
|
|
self.device_tree
|
2020-05-12 13:53:09 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2020-05-05 08:23:32 +00:00
|
|
|
.insert(id.clone(), device_node!(id, virtio_rng_device));
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2019-09-10 16:14:16 +00:00
|
|
|
Ok(devices)
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2020-04-14 09:21:24 +00:00
|
|
|
fn make_virtio_fs_device(
|
2020-04-08 13:04:03 +00:00
|
|
|
&mut self,
|
2020-04-27 07:55:25 +00:00
|
|
|
fs_cfg: &mut FsConfig,
|
2022-03-10 13:05:36 +00:00
|
|
|
) -> DeviceManagerResult<MetaVirtioDevice> {
|
2020-04-27 11:53:45 +00:00
|
|
|
let id = if let Some(id) = &fs_cfg.id {
|
|
|
|
id.clone()
|
|
|
|
} else {
|
|
|
|
let id = self.next_device_name(FS_DEVICE_NAME_PREFIX)?;
|
|
|
|
fs_cfg.id = Some(id.clone());
|
|
|
|
id
|
|
|
|
};
|
2020-04-27 07:55:25 +00:00
|
|
|
|
2021-05-18 14:32:20 +00:00
|
|
|
info!("Creating virtio-fs device: {:?}", fs_cfg);
|
|
|
|
|
2020-05-05 08:23:32 +00:00
|
|
|
let mut node = device_node!(id);
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2020-06-04 19:19:24 +00:00
|
|
|
if let Some(fs_socket) = fs_cfg.socket.to_str() {
|
2020-04-14 09:21:24 +00:00
|
|
|
let virtio_fs_device = Arc::new(Mutex::new(
|
2020-07-02 12:25:19 +00:00
|
|
|
virtio_devices::vhost_user::Fs::new(
|
2020-04-27 12:38:24 +00:00
|
|
|
id.clone(),
|
2020-06-04 19:19:24 +00:00
|
|
|
fs_socket,
|
2020-04-14 09:21:24 +00:00
|
|
|
&fs_cfg.tag,
|
|
|
|
fs_cfg.num_queues,
|
|
|
|
fs_cfg.queue_size,
|
2022-05-26 11:12:29 +00:00
|
|
|
None,
|
2020-08-18 04:30:53 +00:00
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-03-29 19:27:05 +00:00
|
|
|
self.force_iommu,
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-04-14 09:21:24 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioFs)?,
|
|
|
|
));
|
|
|
|
|
2020-04-30 18:08:04 +00:00
|
|
|
// Update the device tree with the migratable device.
|
|
|
|
node.migratable = Some(Arc::clone(&virtio_fs_device) as Arc<Mutex<dyn Migratable>>);
|
2020-05-12 13:53:09 +00:00
|
|
|
self.device_tree.lock().unwrap().insert(id.clone(), node);
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
Ok(MetaVirtioDevice {
|
|
|
|
virtio_device: Arc::clone(&virtio_fs_device)
|
|
|
|
as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: false,
|
2021-10-11 16:34:29 +00:00
|
|
|
id,
|
2022-03-10 13:05:36 +00:00
|
|
|
pci_segment: fs_cfg.pci_segment,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: None,
|
2022-03-10 13:05:36 +00:00
|
|
|
})
|
2020-04-14 09:21:24 +00:00
|
|
|
} else {
|
|
|
|
Err(DeviceManagerError::NoVirtioFsSock)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_fs_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2020-04-14 09:21:24 +00:00
|
|
|
let mut devices = Vec::new();
|
|
|
|
|
2020-04-27 07:55:25 +00:00
|
|
|
let mut fs_devices = self.config.lock().unwrap().fs.clone();
|
|
|
|
if let Some(fs_list_cfg) = &mut fs_devices {
|
|
|
|
for fs_cfg in fs_list_cfg.iter_mut() {
|
2020-04-14 09:21:24 +00:00
|
|
|
devices.push(self.make_virtio_fs_device(fs_cfg)?);
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
}
|
2020-04-27 07:55:25 +00:00
|
|
|
self.config.lock().unwrap().fs = fs_devices;
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2019-09-10 16:14:16 +00:00
|
|
|
Ok(devices)
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2020-03-13 09:38:42 +00:00
|
|
|
fn make_virtio_pmem_device(
|
|
|
|
&mut self,
|
2020-04-15 16:09:12 +00:00
|
|
|
pmem_cfg: &mut PmemConfig,
|
2022-03-10 13:05:36 +00:00
|
|
|
) -> DeviceManagerResult<MetaVirtioDevice> {
|
2020-04-27 11:36:41 +00:00
|
|
|
let id = if let Some(id) = &pmem_cfg.id {
|
|
|
|
id.clone()
|
|
|
|
} else {
|
|
|
|
let id = self.next_device_name(PMEM_DEVICE_NAME_PREFIX)?;
|
|
|
|
pmem_cfg.id = Some(id.clone());
|
|
|
|
id
|
|
|
|
};
|
2020-04-15 16:09:12 +00:00
|
|
|
|
2021-05-18 14:32:20 +00:00
|
|
|
info!("Creating virtio-pmem device: {:?}", pmem_cfg);
|
|
|
|
|
2020-05-05 08:23:32 +00:00
|
|
|
let mut node = device_node!(id);
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2020-04-29 14:43:03 +00:00
|
|
|
// Look for the id in the device tree. If it can be found, that means
|
|
|
|
// the device is being restored, otherwise it's created from scratch.
|
2020-05-12 13:53:09 +00:00
|
|
|
let region_range = if let Some(node) = self.device_tree.lock().unwrap().get(&id) {
|
2021-09-03 09:30:06 +00:00
|
|
|
info!("Restoring virtio-pmem {} resources", id);
|
2020-04-29 14:43:03 +00:00
|
|
|
|
|
|
|
let mut region_range: Option<(u64, u64)> = None;
|
|
|
|
for resource in node.resources.iter() {
|
|
|
|
match resource {
|
|
|
|
Resource::MmioAddressRange { base, size } => {
|
|
|
|
if region_range.is_some() {
|
|
|
|
return Err(DeviceManagerError::ResourceAlreadyExists);
|
|
|
|
}
|
|
|
|
|
|
|
|
region_range = Some((*base, *size));
|
|
|
|
}
|
|
|
|
_ => {
|
|
|
|
error!("Unexpected resource {:?} for {}", resource, id);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if region_range.is_none() {
|
2021-07-26 13:18:11 +00:00
|
|
|
return Err(DeviceManagerError::MissingVirtioPmemResources);
|
2020-04-29 14:43:03 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
region_range
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
};
|
2020-04-27 17:12:00 +00:00
|
|
|
|
2020-03-13 09:38:42 +00:00
|
|
|
let (custom_flags, set_len) = if pmem_cfg.file.is_dir() {
|
2020-04-24 15:58:03 +00:00
|
|
|
if pmem_cfg.size.is_none() {
|
|
|
|
return Err(DeviceManagerError::PmemWithDirectorySizeMissing);
|
|
|
|
}
|
2020-03-13 09:38:42 +00:00
|
|
|
(O_TMPFILE, true)
|
|
|
|
} else {
|
|
|
|
(0, false)
|
|
|
|
};
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-04-24 15:58:03 +00:00
|
|
|
let mut file = OpenOptions::new()
|
2020-03-13 09:38:42 +00:00
|
|
|
.read(true)
|
|
|
|
.write(!pmem_cfg.discard_writes)
|
|
|
|
.custom_flags(custom_flags)
|
|
|
|
.open(&pmem_cfg.file)
|
|
|
|
.map_err(DeviceManagerError::PmemFileOpen)?;
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-04-24 15:58:03 +00:00
|
|
|
let size = if let Some(size) = pmem_cfg.size {
|
|
|
|
if set_len {
|
|
|
|
file.set_len(size)
|
|
|
|
.map_err(DeviceManagerError::PmemFileSetLen)?;
|
|
|
|
}
|
|
|
|
size
|
|
|
|
} else {
|
|
|
|
file.seek(SeekFrom::End(0))
|
|
|
|
.map_err(DeviceManagerError::PmemFileSetLen)?
|
|
|
|
};
|
|
|
|
|
|
|
|
if size % 0x20_0000 != 0 {
|
|
|
|
return Err(DeviceManagerError::PmemSizeNotAligned);
|
2020-03-13 09:38:42 +00:00
|
|
|
}
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-04-29 14:43:03 +00:00
|
|
|
let (region_base, region_size) = if let Some((base, size)) = region_range {
|
|
|
|
// The memory needs to be 2MiB aligned in order to support
|
|
|
|
// hugepages.
|
2021-10-15 14:49:49 +00:00
|
|
|
self.pci_segments[pmem_cfg.pci_segment as usize]
|
2023-11-14 19:31:57 +00:00
|
|
|
.mem64_allocator
|
2020-04-29 14:43:03 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-10-15 14:49:49 +00:00
|
|
|
.allocate(
|
2020-04-29 14:43:03 +00:00
|
|
|
Some(GuestAddress(base)),
|
|
|
|
size as GuestUsize,
|
|
|
|
Some(0x0020_0000),
|
|
|
|
)
|
|
|
|
.ok_or(DeviceManagerError::PmemRangeAllocation)?;
|
|
|
|
|
|
|
|
(base, size)
|
|
|
|
} else {
|
|
|
|
// The memory needs to be 2MiB aligned in order to support
|
|
|
|
// hugepages.
|
2021-10-15 14:49:49 +00:00
|
|
|
let base = self.pci_segments[pmem_cfg.pci_segment as usize]
|
2023-11-14 19:31:57 +00:00
|
|
|
.mem64_allocator
|
2020-04-29 14:43:03 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-10-15 14:49:49 +00:00
|
|
|
.allocate(None, size as GuestUsize, Some(0x0020_0000))
|
2020-04-29 14:43:03 +00:00
|
|
|
.ok_or(DeviceManagerError::PmemRangeAllocation)?;
|
|
|
|
|
|
|
|
(base.raw_value(), size)
|
|
|
|
};
|
|
|
|
|
2020-03-13 09:38:42 +00:00
|
|
|
let cloned_file = file.try_clone().map_err(DeviceManagerError::CloneFile)?;
|
|
|
|
let mmap_region = MmapRegion::build(
|
|
|
|
Some(FileOffset::new(cloned_file, 0)),
|
2020-04-29 14:43:03 +00:00
|
|
|
region_size as usize,
|
2020-10-02 08:51:38 +00:00
|
|
|
PROT_READ | PROT_WRITE,
|
2020-03-13 09:38:42 +00:00
|
|
|
MAP_NORESERVE
|
|
|
|
| if pmem_cfg.discard_writes {
|
|
|
|
MAP_PRIVATE
|
|
|
|
} else {
|
|
|
|
MAP_SHARED
|
|
|
|
},
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::NewMmapRegion)?;
|
2020-04-20 14:30:36 +00:00
|
|
|
let host_addr: u64 = mmap_region.as_ptr() as u64;
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-04-20 14:30:36 +00:00
|
|
|
let mem_slot = self
|
|
|
|
.memory_manager
|
2020-03-13 09:38:42 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2022-04-12 09:19:58 +00:00
|
|
|
.create_userspace_mapping(region_base, region_size, host_addr, false, false, false)
|
2020-03-13 09:38:42 +00:00
|
|
|
.map_err(DeviceManagerError::MemoryManager)?;
|
2019-11-19 23:22:20 +00:00
|
|
|
|
2020-07-02 12:25:19 +00:00
|
|
|
let mapping = virtio_devices::UserspaceMapping {
|
2020-04-20 14:30:36 +00:00
|
|
|
host_addr,
|
|
|
|
mem_slot,
|
2020-04-29 14:43:03 +00:00
|
|
|
addr: GuestAddress(region_base),
|
|
|
|
len: region_size,
|
2022-04-12 09:19:58 +00:00
|
|
|
mergeable: false,
|
2020-04-20 14:30:36 +00:00
|
|
|
};
|
|
|
|
|
2020-03-13 09:38:42 +00:00
|
|
|
let virtio_pmem_device = Arc::new(Mutex::new(
|
2020-07-02 12:25:19 +00:00
|
|
|
virtio_devices::Pmem::new(
|
2020-04-27 12:38:24 +00:00
|
|
|
id.clone(),
|
2020-04-27 11:36:41 +00:00
|
|
|
file,
|
2020-04-29 14:43:03 +00:00
|
|
|
GuestAddress(region_base),
|
2020-04-27 11:36:41 +00:00
|
|
|
mapping,
|
|
|
|
mmap_region,
|
2021-07-20 07:59:32 +00:00
|
|
|
self.force_iommu | pmem_cfg.iommu,
|
2020-08-04 19:25:06 +00:00
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-04-27 11:36:41 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioPmem)?,
|
2020-03-13 09:38:42 +00:00
|
|
|
));
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-04-30 18:08:04 +00:00
|
|
|
// Update the device tree with correct resource information and with
|
|
|
|
// the migratable device.
|
|
|
|
node.resources.push(Resource::MmioAddressRange {
|
|
|
|
base: region_base,
|
|
|
|
size: region_size,
|
|
|
|
});
|
|
|
|
node.migratable = Some(Arc::clone(&virtio_pmem_device) as Arc<Mutex<dyn Migratable>>);
|
2020-05-12 13:53:09 +00:00
|
|
|
self.device_tree.lock().unwrap().insert(id.clone(), node);
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
Ok(MetaVirtioDevice {
|
|
|
|
virtio_device: Arc::clone(&virtio_pmem_device)
|
|
|
|
as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: pmem_cfg.iommu,
|
2020-04-27 12:38:24 +00:00
|
|
|
id,
|
2022-03-10 13:05:36 +00:00
|
|
|
pci_segment: pmem_cfg.pci_segment,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: None,
|
2022-03-10 13:05:36 +00:00
|
|
|
})
|
2020-03-13 09:38:42 +00:00
|
|
|
}
|
2019-11-19 00:53:23 +00:00
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_pmem_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2020-03-13 09:38:42 +00:00
|
|
|
let mut devices = Vec::new();
|
|
|
|
// Add virtio-pmem if required
|
2020-04-15 16:09:12 +00:00
|
|
|
let mut pmem_devices = self.config.lock().unwrap().pmem.clone();
|
|
|
|
if let Some(pmem_list_cfg) = &mut pmem_devices {
|
|
|
|
for pmem_cfg in pmem_list_cfg.iter_mut() {
|
2020-03-13 09:38:42 +00:00
|
|
|
devices.push(self.make_virtio_pmem_device(pmem_cfg)?);
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
}
|
2020-04-27 08:57:39 +00:00
|
|
|
self.config.lock().unwrap().pmem = pmem_devices;
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2019-09-10 16:14:16 +00:00
|
|
|
Ok(devices)
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2020-04-28 17:10:32 +00:00
|
|
|
fn make_virtio_vsock_device(
|
2020-04-08 13:04:03 +00:00
|
|
|
&mut self,
|
2020-04-27 08:15:30 +00:00
|
|
|
vsock_cfg: &mut VsockConfig,
|
2022-03-10 13:05:36 +00:00
|
|
|
) -> DeviceManagerResult<MetaVirtioDevice> {
|
2020-04-27 11:49:54 +00:00
|
|
|
let id = if let Some(id) = &vsock_cfg.id {
|
|
|
|
id.clone()
|
|
|
|
} else {
|
|
|
|
let id = self.next_device_name(VSOCK_DEVICE_NAME_PREFIX)?;
|
|
|
|
vsock_cfg.id = Some(id.clone());
|
|
|
|
id
|
|
|
|
};
|
2020-04-27 08:15:30 +00:00
|
|
|
|
2021-05-18 14:32:20 +00:00
|
|
|
info!("Creating virtio-vsock device: {:?}", vsock_cfg);
|
|
|
|
|
2020-04-27 08:15:30 +00:00
|
|
|
let socket_path = vsock_cfg
|
2020-06-04 19:19:24 +00:00
|
|
|
.socket
|
2020-04-27 08:15:30 +00:00
|
|
|
.to_str()
|
|
|
|
.ok_or(DeviceManagerError::CreateVsockConvertPath)?;
|
|
|
|
let backend =
|
2020-07-02 12:25:19 +00:00
|
|
|
virtio_devices::vsock::VsockUnixBackend::new(vsock_cfg.cid, socket_path.to_string())
|
2020-04-27 08:15:30 +00:00
|
|
|
.map_err(DeviceManagerError::CreateVsockBackend)?;
|
|
|
|
|
|
|
|
let vsock_device = Arc::new(Mutex::new(
|
2020-07-02 12:25:19 +00:00
|
|
|
virtio_devices::Vsock::new(
|
2020-05-05 09:54:38 +00:00
|
|
|
id.clone(),
|
|
|
|
vsock_cfg.cid,
|
2020-06-04 19:19:24 +00:00
|
|
|
vsock_cfg.socket.clone(),
|
2020-05-05 09:54:38 +00:00
|
|
|
backend,
|
2021-07-20 07:59:32 +00:00
|
|
|
self.force_iommu | vsock_cfg.iommu,
|
2020-09-09 02:35:03 +00:00
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-05-05 09:54:38 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioVsock)?,
|
2020-04-27 08:15:30 +00:00
|
|
|
));
|
|
|
|
|
2020-04-30 18:08:04 +00:00
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
2020-05-05 08:23:32 +00:00
|
|
|
self.device_tree
|
2020-05-12 13:53:09 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2020-05-05 08:23:32 +00:00
|
|
|
.insert(id.clone(), device_node!(id, vsock_device));
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
Ok(MetaVirtioDevice {
|
|
|
|
virtio_device: Arc::clone(&vsock_device)
|
|
|
|
as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: vsock_cfg.iommu,
|
2020-04-27 12:38:24 +00:00
|
|
|
id,
|
2022-03-10 13:05:36 +00:00
|
|
|
pci_segment: vsock_cfg.pci_segment,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: None,
|
2022-03-10 13:05:36 +00:00
|
|
|
})
|
2020-04-27 08:15:30 +00:00
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_vsock_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2019-09-10 16:14:16 +00:00
|
|
|
let mut devices = Vec::new();
|
2020-04-28 17:10:32 +00:00
|
|
|
|
2020-04-27 08:15:30 +00:00
|
|
|
let mut vsock = self.config.lock().unwrap().vsock.clone();
|
|
|
|
if let Some(ref mut vsock_cfg) = &mut vsock {
|
|
|
|
devices.push(self.make_virtio_vsock_device(vsock_cfg)?);
|
2019-09-04 18:14:54 +00:00
|
|
|
}
|
2020-04-27 08:15:30 +00:00
|
|
|
self.config.lock().unwrap().vsock = vsock;
|
2019-09-04 18:14:54 +00:00
|
|
|
|
2019-09-10 16:14:16 +00:00
|
|
|
Ok(devices)
|
2019-09-04 18:14:54 +00:00
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_mem_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2020-03-04 02:16:27 +00:00
|
|
|
let mut devices = Vec::new();
|
|
|
|
|
2020-04-27 11:44:43 +00:00
|
|
|
let mm = self.memory_manager.clone();
|
2022-09-16 16:18:22 +00:00
|
|
|
let mut mm = mm.lock().unwrap();
|
|
|
|
for (memory_zone_id, memory_zone) in mm.memory_zones_mut().iter_mut() {
|
|
|
|
if let Some(virtio_mem_zone) = memory_zone.virtio_mem_zone_mut() {
|
2021-09-22 15:26:30 +00:00
|
|
|
info!("Creating virtio-mem device: id = {}", memory_zone_id);
|
2021-08-05 10:01:35 +00:00
|
|
|
|
2021-09-22 15:26:30 +00:00
|
|
|
let node_id = numa_node_id_from_memory_zone_id(&self.numa_nodes, memory_zone_id)
|
2020-09-11 14:15:29 +00:00
|
|
|
.map(|i| i as u16);
|
|
|
|
|
2020-09-10 09:28:21 +00:00
|
|
|
let virtio_mem_device = Arc::new(Mutex::new(
|
|
|
|
virtio_devices::Mem::new(
|
2021-09-22 15:26:30 +00:00
|
|
|
memory_zone_id.clone(),
|
2020-09-14 20:24:36 +00:00
|
|
|
virtio_mem_zone.region(),
|
2020-09-10 09:28:21 +00:00
|
|
|
self.seccomp_action.clone(),
|
2020-09-11 14:15:29 +00:00
|
|
|
node_id,
|
2020-09-14 20:36:58 +00:00
|
|
|
virtio_mem_zone.hotplugged_size(),
|
2021-02-03 08:46:14 +00:00
|
|
|
virtio_mem_zone.hugepages(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2021-09-23 08:42:00 +00:00
|
|
|
virtio_mem_zone.blocks_state().clone(),
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), memory_zone_id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-09-10 09:28:21 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioMem)?,
|
|
|
|
));
|
2020-04-27 11:44:43 +00:00
|
|
|
|
2022-09-16 16:18:22 +00:00
|
|
|
// Update the virtio-mem zone so that it has a handle onto the
|
|
|
|
// virtio-mem device, which will be used for triggering a resize
|
|
|
|
// if needed.
|
|
|
|
virtio_mem_zone.set_virtio_device(Arc::clone(&virtio_mem_device));
|
|
|
|
|
2021-03-03 10:59:53 +00:00
|
|
|
self.virtio_mem_devices.push(Arc::clone(&virtio_mem_device));
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
devices.push(MetaVirtioDevice {
|
|
|
|
virtio_device: Arc::clone(&virtio_mem_device)
|
|
|
|
as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: false,
|
|
|
|
id: memory_zone_id.clone(),
|
|
|
|
pci_segment: 0,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: None,
|
2022-03-10 13:05:36 +00:00
|
|
|
});
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2020-09-10 09:28:21 +00:00
|
|
|
// Fill the device tree with a new node. In case of restore, we
|
|
|
|
// know there is nothing to do, so we can simply override the
|
|
|
|
// existing entry.
|
2021-09-22 15:26:30 +00:00
|
|
|
self.device_tree.lock().unwrap().insert(
|
|
|
|
memory_zone_id.clone(),
|
|
|
|
device_node!(memory_zone_id, virtio_mem_device),
|
|
|
|
);
|
2020-09-10 09:28:21 +00:00
|
|
|
}
|
2020-03-04 02:16:27 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
Ok(devices)
|
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_balloon_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2020-03-20 03:43:37 +00:00
|
|
|
let mut devices = Vec::new();
|
|
|
|
|
2020-10-14 10:04:42 +00:00
|
|
|
if let Some(balloon_config) = &self.config.lock().unwrap().balloon {
|
2020-03-20 03:43:37 +00:00
|
|
|
let id = String::from(BALLOON_DEVICE_NAME);
|
2021-05-18 14:32:20 +00:00
|
|
|
info!("Creating virtio-balloon device: id = {}", id);
|
2020-03-20 03:43:37 +00:00
|
|
|
|
|
|
|
let virtio_balloon_device = Arc::new(Mutex::new(
|
2020-06-23 09:52:30 +00:00
|
|
|
virtio_devices::Balloon::new(
|
|
|
|
id.clone(),
|
2020-10-14 10:04:42 +00:00
|
|
|
balloon_config.size,
|
2021-01-21 11:22:29 +00:00
|
|
|
balloon_config.deflate_on_oom,
|
2022-02-10 13:47:33 +00:00
|
|
|
balloon_config.free_page_reporting,
|
2020-08-18 02:58:13 +00:00
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-06-23 09:52:30 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioBalloon)?,
|
2020-03-20 03:43:37 +00:00
|
|
|
));
|
|
|
|
|
2020-10-14 10:04:42 +00:00
|
|
|
self.balloon = Some(virtio_balloon_device.clone());
|
2020-03-20 03:43:37 +00:00
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
devices.push(MetaVirtioDevice {
|
|
|
|
virtio_device: Arc::clone(&virtio_balloon_device)
|
|
|
|
as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: false,
|
|
|
|
id: id.clone(),
|
|
|
|
pci_segment: 0,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: None,
|
2022-03-10 13:05:36 +00:00
|
|
|
});
|
2020-03-20 03:43:37 +00:00
|
|
|
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, virtio_balloon_device));
|
|
|
|
}
|
|
|
|
|
|
|
|
Ok(devices)
|
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
fn make_virtio_watchdog_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
2020-09-25 09:39:45 +00:00
|
|
|
let mut devices = Vec::new();
|
|
|
|
|
|
|
|
if !self.config.lock().unwrap().watchdog {
|
|
|
|
return Ok(devices);
|
|
|
|
}
|
|
|
|
|
|
|
|
let id = String::from(WATCHDOG_DEVICE_NAME);
|
2021-05-18 14:32:20 +00:00
|
|
|
info!("Creating virtio-watchdog device: id = {}", id);
|
2020-09-25 09:39:45 +00:00
|
|
|
|
|
|
|
let virtio_watchdog_device = Arc::new(Mutex::new(
|
|
|
|
virtio_devices::Watchdog::new(
|
|
|
|
id.clone(),
|
|
|
|
self.reset_evt.try_clone().unwrap(),
|
|
|
|
self.seccomp_action.clone(),
|
2021-09-07 15:10:48 +00:00
|
|
|
self.exit_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2020-09-25 09:39:45 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVirtioWatchdog)?,
|
|
|
|
));
|
2022-03-10 13:05:36 +00:00
|
|
|
devices.push(MetaVirtioDevice {
|
|
|
|
virtio_device: Arc::clone(&virtio_watchdog_device)
|
|
|
|
as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
|
|
|
iommu: false,
|
|
|
|
id: id.clone(),
|
|
|
|
pci_segment: 0,
|
2022-03-10 13:16:08 +00:00
|
|
|
dma_handler: None,
|
2022-03-10 13:05:36 +00:00
|
|
|
});
|
2020-09-25 09:39:45 +00:00
|
|
|
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(id.clone(), device_node!(id, virtio_watchdog_device));
|
|
|
|
|
|
|
|
Ok(devices)
|
|
|
|
}
|
|
|
|
|
2022-03-11 11:31:25 +00:00
|
|
|
fn make_vdpa_device(
|
|
|
|
&mut self,
|
|
|
|
vdpa_cfg: &mut VdpaConfig,
|
|
|
|
) -> DeviceManagerResult<MetaVirtioDevice> {
|
|
|
|
let id = if let Some(id) = &vdpa_cfg.id {
|
|
|
|
id.clone()
|
|
|
|
} else {
|
|
|
|
let id = self.next_device_name(VDPA_DEVICE_NAME_PREFIX)?;
|
|
|
|
vdpa_cfg.id = Some(id.clone());
|
|
|
|
id
|
|
|
|
};
|
|
|
|
|
|
|
|
info!("Creating vDPA device: {:?}", vdpa_cfg);
|
|
|
|
|
|
|
|
let device_path = vdpa_cfg
|
|
|
|
.path
|
|
|
|
.to_str()
|
|
|
|
.ok_or(DeviceManagerError::CreateVdpaConvertPath)?;
|
|
|
|
|
|
|
|
let vdpa_device = Arc::new(Mutex::new(
|
|
|
|
virtio_devices::Vdpa::new(
|
|
|
|
id.clone(),
|
|
|
|
device_path,
|
|
|
|
self.memory_manager.lock().unwrap().guest_memory(),
|
|
|
|
vdpa_cfg.num_queues as u16,
|
2022-10-18 15:14:43 +00:00
|
|
|
versioned_state_from_id(self.snapshot.as_ref(), id.as_str())
|
|
|
|
.map_err(DeviceManagerError::RestoreGetState)?,
|
2022-03-11 11:31:25 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::CreateVdpa)?,
|
|
|
|
));
|
|
|
|
|
|
|
|
// Create the DMA handler that is required by the vDPA device
|
|
|
|
let vdpa_mapping = Arc::new(VdpaDmaMapping::new(
|
|
|
|
Arc::clone(&vdpa_device),
|
|
|
|
Arc::new(self.memory_manager.lock().unwrap().guest_memory()),
|
|
|
|
));
|
|
|
|
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2022-10-12 09:23:16 +00:00
|
|
|
.insert(id.clone(), device_node!(id, vdpa_device));
|
2022-03-11 11:31:25 +00:00
|
|
|
|
|
|
|
Ok(MetaVirtioDevice {
|
|
|
|
virtio_device: vdpa_device as Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
2022-04-04 09:29:54 +00:00
|
|
|
iommu: vdpa_cfg.iommu,
|
2022-03-11 11:31:25 +00:00
|
|
|
id,
|
|
|
|
pci_segment: vdpa_cfg.pci_segment,
|
|
|
|
dma_handler: Some(vdpa_mapping),
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
|
|
|
fn make_vdpa_devices(&mut self) -> DeviceManagerResult<Vec<MetaVirtioDevice>> {
|
|
|
|
let mut devices = Vec::new();
|
|
|
|
// Add vdpa if required
|
|
|
|
let mut vdpa_devices = self.config.lock().unwrap().vdpa.clone();
|
|
|
|
if let Some(vdpa_list_cfg) = &mut vdpa_devices {
|
|
|
|
for vdpa_cfg in vdpa_list_cfg.iter_mut() {
|
|
|
|
devices.push(self.make_vdpa_device(vdpa_cfg)?);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
self.config.lock().unwrap().vdpa = vdpa_devices;
|
|
|
|
|
|
|
|
Ok(devices)
|
|
|
|
}
|
|
|
|
|
2020-04-27 08:37:56 +00:00
|
|
|
fn next_device_name(&mut self, prefix: &str) -> DeviceManagerResult<String> {
|
2020-03-06 10:34:24 +00:00
|
|
|
let start_id = self.device_id_cnt;
|
|
|
|
loop {
|
|
|
|
// Generate the temporary name.
|
|
|
|
let name = format!("{}{}", prefix, self.device_id_cnt);
|
|
|
|
// Increment the counter.
|
|
|
|
self.device_id_cnt += Wrapping(1);
|
|
|
|
// Check if the name is already in use.
|
2022-04-29 07:58:22 +00:00
|
|
|
if !self.boot_id_list.contains(&name)
|
|
|
|
&& !self.device_tree.lock().unwrap().contains_key(&name)
|
|
|
|
{
|
2020-04-27 08:37:56 +00:00
|
|
|
return Ok(name);
|
2020-03-06 10:34:24 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
if self.device_id_cnt == start_id {
|
|
|
|
// We went through a full loop and there's nothing else we can
|
|
|
|
// do.
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
2020-04-15 16:58:49 +00:00
|
|
|
Err(DeviceManagerError::NoAvailableDeviceName)
|
2020-03-06 10:34:24 +00:00
|
|
|
}
|
|
|
|
|
2020-07-17 15:28:12 +00:00
|
|
|
fn add_passthrough_device(
|
|
|
|
&mut self,
|
|
|
|
device_cfg: &mut DeviceConfig,
|
2021-10-18 16:29:42 +00:00
|
|
|
) -> DeviceManagerResult<(PciBdf, String)> {
|
2021-03-22 09:10:12 +00:00
|
|
|
// If the passthrough device has not been created yet, it is created
|
|
|
|
// here and stored in the DeviceManager structure for future needs.
|
|
|
|
if self.passthrough_device.is_none() {
|
|
|
|
self.passthrough_device = Some(
|
|
|
|
self.address_manager
|
|
|
|
.vm
|
|
|
|
.create_passthrough_device()
|
|
|
|
.map_err(|e| DeviceManagerError::CreatePassthroughDevice(e.into()))?,
|
|
|
|
);
|
|
|
|
}
|
|
|
|
|
2021-10-04 14:33:29 +00:00
|
|
|
self.add_vfio_device(device_cfg)
|
2020-07-17 15:28:12 +00:00
|
|
|
}
|
|
|
|
|
2021-09-14 09:02:10 +00:00
|
|
|
fn create_vfio_container(&self) -> DeviceManagerResult<Arc<VfioContainer>> {
|
2020-07-21 14:23:39 +00:00
|
|
|
let passthrough_device = self
|
|
|
|
.passthrough_device
|
|
|
|
.as_ref()
|
|
|
|
.ok_or(DeviceManagerError::NoDevicePassthroughSupport)?;
|
|
|
|
|
2022-07-21 13:15:15 +00:00
|
|
|
let dup = passthrough_device
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::VfioCreate)?;
|
2022-07-09 12:13:44 +00:00
|
|
|
|
2021-09-14 09:02:10 +00:00
|
|
|
Ok(Arc::new(
|
2022-07-21 13:15:15 +00:00
|
|
|
VfioContainer::new(Some(Arc::new(dup))).map_err(DeviceManagerError::VfioCreate)?,
|
2021-09-14 09:02:10 +00:00
|
|
|
))
|
|
|
|
}
|
2019-11-12 14:36:07 +00:00
|
|
|
|
2021-09-14 09:02:10 +00:00
|
|
|
fn add_vfio_device(
|
|
|
|
&mut self,
|
|
|
|
device_cfg: &mut DeviceConfig,
|
2021-10-18 16:29:42 +00:00
|
|
|
) -> DeviceManagerResult<(PciBdf, String)> {
|
2022-04-07 14:36:44 +00:00
|
|
|
let vfio_name = if let Some(id) = &device_cfg.id {
|
|
|
|
id.clone()
|
|
|
|
} else {
|
|
|
|
let id = self.next_device_name(VFIO_DEVICE_NAME_PREFIX)?;
|
|
|
|
device_cfg.id = Some(id.clone());
|
|
|
|
id
|
|
|
|
};
|
|
|
|
|
|
|
|
let (pci_segment_id, pci_device_bdf, resources) =
|
|
|
|
self.pci_resources(&vfio_name, device_cfg.pci_segment)?;
|
2021-09-14 09:02:10 +00:00
|
|
|
|
|
|
|
let mut needs_dma_mapping = false;
|
|
|
|
|
|
|
|
// Here we create a new VFIO container for two reasons. Either this is
|
|
|
|
// the first VFIO device, meaning we need a new VFIO container, which
|
|
|
|
// will be shared with other VFIO devices. Or the new VFIO device is
|
|
|
|
// attached to a vIOMMU, meaning we must create a dedicated VFIO
|
|
|
|
// container. In the vIOMMU use case, we can't let all devices under
|
|
|
|
// the same VFIO container since we couldn't map/unmap memory for each
|
|
|
|
// device. That's simply because the map/unmap operations happen at the
|
|
|
|
// VFIO container level.
|
|
|
|
let vfio_container = if device_cfg.iommu {
|
|
|
|
let vfio_container = self.create_vfio_container()?;
|
|
|
|
|
|
|
|
let vfio_mapping = Arc::new(VfioDmaMapping::new(
|
|
|
|
Arc::clone(&vfio_container),
|
|
|
|
Arc::new(self.memory_manager.lock().unwrap().guest_memory()),
|
|
|
|
));
|
2020-02-27 16:15:25 +00:00
|
|
|
|
|
|
|
if let Some(iommu) = &self.iommu_device {
|
|
|
|
iommu
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-10-18 16:29:42 +00:00
|
|
|
.add_external_mapping(pci_device_bdf.into(), vfio_mapping);
|
2021-09-14 09:02:10 +00:00
|
|
|
} else {
|
|
|
|
return Err(DeviceManagerError::MissingVirtualIommu);
|
2020-02-27 16:15:25 +00:00
|
|
|
}
|
2021-09-14 09:02:10 +00:00
|
|
|
|
|
|
|
vfio_container
|
|
|
|
} else if let Some(vfio_container) = &self.vfio_container {
|
|
|
|
Arc::clone(vfio_container)
|
2021-03-03 10:59:53 +00:00
|
|
|
} else {
|
2021-09-14 09:02:10 +00:00
|
|
|
let vfio_container = self.create_vfio_container()?;
|
|
|
|
needs_dma_mapping = true;
|
|
|
|
self.vfio_container = Some(Arc::clone(&vfio_container));
|
|
|
|
|
|
|
|
vfio_container
|
|
|
|
};
|
|
|
|
|
|
|
|
let vfio_device = VfioDevice::new(&device_cfg.path, Arc::clone(&vfio_container))
|
|
|
|
.map_err(DeviceManagerError::VfioCreate)?;
|
|
|
|
|
|
|
|
if needs_dma_mapping {
|
|
|
|
// Register DMA mapping in IOMMU.
|
|
|
|
// Do not register virtio-mem regions, as they are handled directly by
|
|
|
|
// virtio-mem device itself.
|
|
|
|
for (_, zone) in self.memory_manager.lock().unwrap().memory_zones().iter() {
|
|
|
|
for region in zone.regions() {
|
|
|
|
vfio_container
|
|
|
|
.vfio_dma_map(
|
|
|
|
region.start_addr().raw_value(),
|
2022-11-01 21:52:40 +00:00
|
|
|
region.len(),
|
2021-09-14 09:02:10 +00:00
|
|
|
region.as_ptr() as u64,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::VfioDmaMap)?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
let vfio_mapping = Arc::new(VfioDmaMapping::new(
|
|
|
|
Arc::clone(&vfio_container),
|
|
|
|
Arc::new(self.memory_manager.lock().unwrap().guest_memory()),
|
|
|
|
));
|
|
|
|
|
2021-03-03 10:59:53 +00:00
|
|
|
for virtio_mem_device in self.virtio_mem_devices.iter() {
|
|
|
|
virtio_mem_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-09-21 09:00:22 +00:00
|
|
|
.add_dma_mapping_handler(
|
|
|
|
VirtioMemMappingSource::Container,
|
|
|
|
vfio_mapping.clone(),
|
|
|
|
)
|
2021-03-03 10:59:53 +00:00
|
|
|
.map_err(DeviceManagerError::AddDmaMappingHandlerVirtioMem)?;
|
|
|
|
}
|
2020-02-27 16:15:25 +00:00
|
|
|
}
|
|
|
|
|
2021-10-04 14:33:29 +00:00
|
|
|
let legacy_interrupt_group =
|
|
|
|
if let Some(legacy_interrupt_manager) = &self.legacy_interrupt_manager {
|
|
|
|
Some(
|
|
|
|
legacy_interrupt_manager
|
|
|
|
.create_group(LegacyIrqGroupConfig {
|
2021-10-07 17:07:13 +00:00
|
|
|
irq: self.pci_segments[pci_segment_id as usize].pci_irq_slots
|
2021-10-18 16:29:42 +00:00
|
|
|
[pci_device_bdf.device() as usize]
|
2021-10-04 14:33:29 +00:00
|
|
|
as InterruptIndex,
|
|
|
|
})
|
|
|
|
.map_err(DeviceManagerError::CreateInterruptGroup)?,
|
|
|
|
)
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
};
|
2021-02-09 10:51:49 +00:00
|
|
|
|
2022-06-08 14:41:24 +00:00
|
|
|
let memory_manager = self.memory_manager.clone();
|
|
|
|
|
2021-10-14 15:31:03 +00:00
|
|
|
let vfio_pci_device = VfioPciDevice::new(
|
2022-04-07 16:16:19 +00:00
|
|
|
vfio_name.clone(),
|
2020-07-03 09:16:49 +00:00
|
|
|
&self.address_manager.vm,
|
2019-11-12 14:36:07 +00:00
|
|
|
vfio_device,
|
2021-02-23 14:01:31 +00:00
|
|
|
vfio_container,
|
2022-04-22 09:08:29 +00:00
|
|
|
self.msi_interrupt_manager.clone(),
|
2021-02-09 10:51:49 +00:00
|
|
|
legacy_interrupt_group,
|
2021-02-23 14:01:31 +00:00
|
|
|
device_cfg.iommu,
|
2022-01-18 11:52:56 +00:00
|
|
|
pci_device_bdf,
|
2022-06-08 14:41:24 +00:00
|
|
|
Arc::new(move || memory_manager.lock().unwrap().allocate_memory_slot()),
|
2022-11-28 14:12:18 +00:00
|
|
|
vm_migration::snapshot_from_id(self.snapshot.as_ref(), vfio_name.as_str()),
|
2024-02-01 20:41:10 +00:00
|
|
|
device_cfg.x_nv_gpudirect_clique,
|
2019-11-12 14:36:07 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::VfioPciCreate)?;
|
2020-02-27 16:15:25 +00:00
|
|
|
|
2021-10-14 15:31:03 +00:00
|
|
|
let vfio_pci_device = Arc::new(Mutex::new(vfio_pci_device));
|
|
|
|
|
2022-04-07 15:20:25 +00:00
|
|
|
let new_resources = self.add_pci_device(
|
2021-10-14 15:31:03 +00:00
|
|
|
vfio_pci_device.clone(),
|
|
|
|
vfio_pci_device.clone(),
|
2021-10-07 17:07:13 +00:00
|
|
|
pci_segment_id,
|
2021-10-14 15:31:03 +00:00
|
|
|
pci_device_bdf,
|
2022-04-07 14:36:44 +00:00
|
|
|
resources,
|
2021-10-14 15:31:03 +00:00
|
|
|
)?;
|
|
|
|
|
2022-11-28 14:12:18 +00:00
|
|
|
vfio_pci_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.map_mmio_regions()
|
|
|
|
.map_err(DeviceManagerError::VfioMapRegion)?;
|
2020-09-10 08:57:53 +00:00
|
|
|
|
2022-06-07 14:35:53 +00:00
|
|
|
let mut node = device_node!(vfio_name, vfio_pci_device);
|
2020-11-30 10:53:31 +00:00
|
|
|
|
2022-04-07 15:20:25 +00:00
|
|
|
// Update the device tree with correct resource information.
|
|
|
|
node.resources = new_resources;
|
2022-02-16 08:55:26 +00:00
|
|
|
node.pci_bdf = Some(pci_device_bdf);
|
2021-03-18 12:09:52 +00:00
|
|
|
node.pci_device_handle = Some(PciDeviceHandle::Vfio(vfio_pci_device));
|
|
|
|
|
2020-11-30 10:53:31 +00:00
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(vfio_name.clone(), node);
|
2020-03-06 10:34:24 +00:00
|
|
|
|
2020-06-11 14:48:25 +00:00
|
|
|
Ok((pci_device_bdf, vfio_name))
|
2020-02-27 16:15:25 +00:00
|
|
|
}
|
|
|
|
|
2020-10-05 17:13:28 +00:00
|
|
|
fn add_pci_device(
|
2020-09-10 08:57:53 +00:00
|
|
|
&mut self,
|
2020-10-05 17:13:28 +00:00
|
|
|
bus_device: Arc<Mutex<dyn BusDevice>>,
|
|
|
|
pci_device: Arc<Mutex<dyn PciDevice>>,
|
2021-10-07 16:47:44 +00:00
|
|
|
segment_id: u16,
|
2021-10-18 16:29:42 +00:00
|
|
|
bdf: PciBdf,
|
2022-04-07 12:22:07 +00:00
|
|
|
resources: Option<Vec<Resource>>,
|
2022-04-07 15:20:25 +00:00
|
|
|
) -> DeviceManagerResult<Vec<Resource>> {
|
2020-10-05 17:13:28 +00:00
|
|
|
let bars = pci_device
|
2020-09-10 08:57:53 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-10-15 10:38:06 +00:00
|
|
|
.allocate_bars(
|
2022-01-05 16:48:09 +00:00
|
|
|
&self.address_manager.allocator,
|
2021-10-15 10:38:06 +00:00
|
|
|
&mut self.pci_segments[segment_id as usize]
|
2023-11-14 19:31:57 +00:00
|
|
|
.mem32_allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap(),
|
|
|
|
&mut self.pci_segments[segment_id as usize]
|
|
|
|
.mem64_allocator
|
2021-10-15 10:38:06 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap(),
|
2022-04-07 12:22:07 +00:00
|
|
|
resources,
|
2021-10-15 10:38:06 +00:00
|
|
|
)
|
2020-09-10 08:57:53 +00:00
|
|
|
.map_err(DeviceManagerError::AllocateBars)?;
|
|
|
|
|
2021-10-07 16:47:44 +00:00
|
|
|
let mut pci_bus = self.pci_segments[segment_id as usize]
|
|
|
|
.pci_bus
|
|
|
|
.lock()
|
|
|
|
.unwrap();
|
2021-10-04 14:33:29 +00:00
|
|
|
|
2020-09-10 08:57:53 +00:00
|
|
|
pci_bus
|
2021-10-18 16:29:42 +00:00
|
|
|
.add_device(bdf.device() as u32, pci_device)
|
2020-09-10 08:57:53 +00:00
|
|
|
.map_err(DeviceManagerError::AddPciDevice)?;
|
|
|
|
|
2020-10-05 17:13:28 +00:00
|
|
|
self.bus_devices.push(Arc::clone(&bus_device));
|
2020-09-10 08:57:53 +00:00
|
|
|
|
|
|
|
pci_bus
|
|
|
|
.register_mapping(
|
2020-10-05 17:13:28 +00:00
|
|
|
bus_device,
|
2020-09-10 08:57:53 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
self.address_manager.io_bus.as_ref(),
|
|
|
|
self.address_manager.mmio_bus.as_ref(),
|
|
|
|
bars.clone(),
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::AddPciDevice)?;
|
|
|
|
|
2022-04-07 15:20:25 +00:00
|
|
|
let mut new_resources = Vec::new();
|
2022-04-14 16:13:57 +00:00
|
|
|
for bar in bars {
|
2022-04-15 09:27:53 +00:00
|
|
|
new_resources.push(Resource::PciBar {
|
|
|
|
index: bar.idx(),
|
|
|
|
base: bar.addr(),
|
|
|
|
size: bar.size(),
|
|
|
|
type_: bar.region_type().into(),
|
|
|
|
prefetchable: bar.prefetchable().into(),
|
|
|
|
});
|
2022-04-07 15:20:25 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
Ok(new_resources)
|
2020-09-10 08:57:53 +00:00
|
|
|
}
|
|
|
|
|
2021-10-18 16:29:42 +00:00
|
|
|
fn add_vfio_devices(&mut self) -> DeviceManagerResult<Vec<PciBdf>> {
|
2019-10-08 05:05:08 +00:00
|
|
|
let mut iommu_attached_device_ids = Vec::new();
|
2020-03-09 13:08:59 +00:00
|
|
|
let mut devices = self.config.lock().unwrap().devices.clone();
|
2019-11-28 00:45:10 +00:00
|
|
|
|
2020-03-09 13:08:59 +00:00
|
|
|
if let Some(device_list_cfg) = &mut devices {
|
|
|
|
for device_cfg in device_list_cfg.iter_mut() {
|
2021-10-04 14:33:29 +00:00
|
|
|
let (device_id, _) = self.add_passthrough_device(device_cfg)?;
|
2020-03-11 17:23:11 +00:00
|
|
|
if device_cfg.iommu && self.iommu_device.is_some() {
|
2020-02-27 16:15:25 +00:00
|
|
|
iommu_attached_device_ids.push(device_id);
|
2019-10-07 17:48:44 +00:00
|
|
|
}
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
}
|
2020-03-09 13:08:59 +00:00
|
|
|
|
|
|
|
// Update the list of devices
|
|
|
|
self.config.lock().unwrap().devices = devices;
|
|
|
|
|
2019-10-08 05:05:08 +00:00
|
|
|
Ok(iommu_attached_device_ids)
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2021-06-09 13:43:36 +00:00
|
|
|
fn add_vfio_user_device(
|
|
|
|
&mut self,
|
|
|
|
device_cfg: &mut UserDeviceConfig,
|
2021-10-18 16:29:42 +00:00
|
|
|
) -> DeviceManagerResult<(PciBdf, String)> {
|
2022-04-07 14:36:44 +00:00
|
|
|
let vfio_user_name = if let Some(id) = &device_cfg.id {
|
|
|
|
id.clone()
|
|
|
|
} else {
|
|
|
|
let id = self.next_device_name(VFIO_USER_DEVICE_NAME_PREFIX)?;
|
|
|
|
device_cfg.id = Some(id.clone());
|
|
|
|
id
|
|
|
|
};
|
|
|
|
|
|
|
|
let (pci_segment_id, pci_device_bdf, resources) =
|
|
|
|
self.pci_resources(&vfio_user_name, device_cfg.pci_segment)?;
|
2021-10-04 14:33:29 +00:00
|
|
|
|
|
|
|
let legacy_interrupt_group =
|
|
|
|
if let Some(legacy_interrupt_manager) = &self.legacy_interrupt_manager {
|
|
|
|
Some(
|
|
|
|
legacy_interrupt_manager
|
|
|
|
.create_group(LegacyIrqGroupConfig {
|
2021-10-07 17:07:13 +00:00
|
|
|
irq: self.pci_segments[pci_segment_id as usize].pci_irq_slots
|
2021-10-18 16:29:42 +00:00
|
|
|
[pci_device_bdf.device() as usize]
|
2021-10-04 14:33:29 +00:00
|
|
|
as InterruptIndex,
|
|
|
|
})
|
|
|
|
.map_err(DeviceManagerError::CreateInterruptGroup)?,
|
|
|
|
)
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
};
|
2021-06-09 13:43:36 +00:00
|
|
|
|
2021-08-11 10:19:33 +00:00
|
|
|
let client = Arc::new(Mutex::new(
|
|
|
|
vfio_user::Client::new(&device_cfg.socket)
|
|
|
|
.map_err(DeviceManagerError::VfioUserCreateClient)?,
|
|
|
|
));
|
|
|
|
|
2022-06-08 14:41:24 +00:00
|
|
|
let memory_manager = self.memory_manager.clone();
|
|
|
|
|
2021-06-09 13:43:36 +00:00
|
|
|
let mut vfio_user_pci_device = VfioUserPciDevice::new(
|
2022-04-07 16:16:19 +00:00
|
|
|
vfio_user_name.clone(),
|
2021-06-09 13:43:36 +00:00
|
|
|
&self.address_manager.vm,
|
2021-08-11 10:53:21 +00:00
|
|
|
client.clone(),
|
2022-04-22 09:08:29 +00:00
|
|
|
self.msi_interrupt_manager.clone(),
|
2021-06-09 13:43:36 +00:00
|
|
|
legacy_interrupt_group,
|
2022-01-18 11:52:56 +00:00
|
|
|
pci_device_bdf,
|
2022-06-08 14:41:24 +00:00
|
|
|
Arc::new(move || memory_manager.lock().unwrap().allocate_memory_slot()),
|
2022-11-28 14:12:18 +00:00
|
|
|
vm_migration::snapshot_from_id(self.snapshot.as_ref(), vfio_user_name.as_str()),
|
2021-06-09 13:43:36 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::VfioUserCreate)?;
|
|
|
|
|
2021-08-11 10:53:21 +00:00
|
|
|
let memory = self.memory_manager.lock().unwrap().guest_memory();
|
|
|
|
let vfio_user_mapping = Arc::new(VfioUserDmaMapping::new(client, Arc::new(memory)));
|
|
|
|
for virtio_mem_device in self.virtio_mem_devices.iter() {
|
|
|
|
virtio_mem_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.add_dma_mapping_handler(
|
2021-10-18 16:29:42 +00:00
|
|
|
VirtioMemMappingSource::Device(pci_device_bdf.into()),
|
2021-08-11 10:53:21 +00:00
|
|
|
vfio_user_mapping.clone(),
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::AddDmaMappingHandlerVirtioMem)?;
|
|
|
|
}
|
|
|
|
|
2021-06-09 13:43:36 +00:00
|
|
|
for (_, zone) in self.memory_manager.lock().unwrap().memory_zones().iter() {
|
|
|
|
for region in zone.regions() {
|
|
|
|
vfio_user_pci_device
|
|
|
|
.dma_map(region)
|
|
|
|
.map_err(DeviceManagerError::VfioUserDmaMap)?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
let vfio_user_pci_device = Arc::new(Mutex::new(vfio_user_pci_device));
|
|
|
|
|
2022-04-07 15:20:25 +00:00
|
|
|
let new_resources = self.add_pci_device(
|
2021-06-09 13:43:36 +00:00
|
|
|
vfio_user_pci_device.clone(),
|
|
|
|
vfio_user_pci_device.clone(),
|
2021-10-07 17:07:13 +00:00
|
|
|
pci_segment_id,
|
2021-06-09 13:43:36 +00:00
|
|
|
pci_device_bdf,
|
2022-04-07 14:36:44 +00:00
|
|
|
resources,
|
2021-06-09 13:43:36 +00:00
|
|
|
)?;
|
|
|
|
|
2022-11-28 14:12:18 +00:00
|
|
|
// Note it is required to call 'add_pci_device()' in advance to have the list of
|
|
|
|
// mmio regions provisioned correctly
|
|
|
|
vfio_user_pci_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.map_mmio_regions()
|
|
|
|
.map_err(DeviceManagerError::VfioUserMapRegion)?;
|
2022-04-28 22:20:24 +00:00
|
|
|
|
2022-06-07 14:37:21 +00:00
|
|
|
let mut node = device_node!(vfio_user_name, vfio_user_pci_device);
|
2021-06-09 13:43:36 +00:00
|
|
|
|
2022-04-07 15:20:25 +00:00
|
|
|
// Update the device tree with correct resource information.
|
|
|
|
node.resources = new_resources;
|
2022-02-16 08:55:26 +00:00
|
|
|
node.pci_bdf = Some(pci_device_bdf);
|
2021-06-09 13:43:36 +00:00
|
|
|
node.pci_device_handle = Some(PciDeviceHandle::VfioUser(vfio_user_pci_device));
|
|
|
|
|
|
|
|
self.device_tree
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.insert(vfio_user_name.clone(), node);
|
|
|
|
|
|
|
|
Ok((pci_device_bdf, vfio_user_name))
|
|
|
|
}
|
|
|
|
|
2021-10-18 16:29:42 +00:00
|
|
|
fn add_user_devices(&mut self) -> DeviceManagerResult<Vec<PciBdf>> {
|
2021-06-09 13:43:36 +00:00
|
|
|
let mut user_devices = self.config.lock().unwrap().user_devices.clone();
|
|
|
|
|
|
|
|
if let Some(device_list_cfg) = &mut user_devices {
|
|
|
|
for device_cfg in device_list_cfg.iter_mut() {
|
2021-10-04 14:33:29 +00:00
|
|
|
let (_device_id, _id) = self.add_vfio_user_device(device_cfg)?;
|
2021-06-09 13:43:36 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// Update the list of devices
|
|
|
|
self.config.lock().unwrap().user_devices = user_devices;
|
|
|
|
|
|
|
|
Ok(vec![])
|
|
|
|
}
|
|
|
|
|
2019-09-04 13:55:14 +00:00
|
|
|
fn add_virtio_pci_device(
|
2020-01-29 15:53:12 +00:00
|
|
|
&mut self,
|
2022-03-10 11:30:41 +00:00
|
|
|
virtio_device: Arc<Mutex<dyn virtio_devices::VirtioDevice>>,
|
2019-10-02 20:57:20 +00:00
|
|
|
iommu_mapping: &Option<Arc<IommuMapping>>,
|
2020-04-27 14:19:40 +00:00
|
|
|
virtio_device_id: String,
|
2021-10-08 13:47:35 +00:00
|
|
|
pci_segment_id: u16,
|
2022-03-10 13:39:38 +00:00
|
|
|
dma_handler: Option<Arc<dyn ExternalDmaMapping>>,
|
2021-10-18 16:29:42 +00:00
|
|
|
) -> DeviceManagerResult<PciBdf> {
|
2022-12-14 11:41:15 +00:00
|
|
|
let id = format!("{VIRTIO_PCI_DEVICE_NAME_PREFIX}-{virtio_device_id}");
|
2020-04-27 14:19:40 +00:00
|
|
|
|
2020-04-27 17:12:00 +00:00
|
|
|
// Add the new virtio-pci node to the device tree.
|
2020-05-05 08:23:32 +00:00
|
|
|
let mut node = device_node!(id);
|
2020-04-30 14:39:45 +00:00
|
|
|
node.children = vec![virtio_device_id.clone()];
|
2020-04-27 17:12:00 +00:00
|
|
|
|
2022-04-07 14:33:22 +00:00
|
|
|
let (pci_segment_id, pci_device_bdf, resources) =
|
|
|
|
self.pci_resources(&id, pci_segment_id)?;
|
2020-05-11 16:09:00 +00:00
|
|
|
|
2020-04-27 17:12:00 +00:00
|
|
|
// Update the existing virtio node by setting the parent.
|
2020-05-12 13:53:09 +00:00
|
|
|
if let Some(node) = self.device_tree.lock().unwrap().get_mut(&virtio_device_id) {
|
2020-04-27 17:12:00 +00:00
|
|
|
node.parent = Some(id.clone());
|
|
|
|
} else {
|
|
|
|
return Err(DeviceManagerError::MissingNode);
|
|
|
|
}
|
|
|
|
|
2019-12-05 15:42:15 +00:00
|
|
|
// Allows support for one MSI-X vector per queue. It also adds 1
|
|
|
|
// as we need to take into account the dedicated vector to notify
|
|
|
|
// about a virtio config change.
|
2019-11-18 23:10:42 +00:00
|
|
|
let msix_num = (virtio_device.lock().unwrap().queue_max_sizes().len() + 1) as u16;
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2021-10-21 10:41:16 +00:00
|
|
|
// Create the AccessPlatform trait from the implementation IommuMapping.
|
|
|
|
// This will provide address translation for any virtio device sitting
|
|
|
|
// behind a vIOMMU.
|
|
|
|
let access_platform: Option<Arc<dyn AccessPlatform>> = if let Some(mapping) = iommu_mapping
|
|
|
|
{
|
|
|
|
Some(Arc::new(AccessPlatformMapping::new(
|
2021-10-18 16:29:42 +00:00
|
|
|
pci_device_bdf.into(),
|
2021-10-21 10:41:16 +00:00
|
|
|
mapping.clone(),
|
|
|
|
)))
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
};
|
2019-10-02 20:57:20 +00:00
|
|
|
|
2020-01-31 11:55:30 +00:00
|
|
|
let memory = self.memory_manager.lock().unwrap().guest_memory();
|
2022-03-10 14:26:49 +00:00
|
|
|
|
2022-03-10 15:31:06 +00:00
|
|
|
// Map DMA ranges if a DMA handler is available and if the device is
|
|
|
|
// not attached to a virtual IOMMU.
|
2022-03-10 14:26:49 +00:00
|
|
|
if let Some(dma_handler) = &dma_handler {
|
2022-03-10 15:31:06 +00:00
|
|
|
if iommu_mapping.is_some() {
|
|
|
|
if let Some(iommu) = &self.iommu_device {
|
|
|
|
iommu
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.add_external_mapping(pci_device_bdf.into(), dma_handler.clone());
|
|
|
|
} else {
|
|
|
|
return Err(DeviceManagerError::MissingVirtualIommu);
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
// Let every virtio-mem device handle the DMA map/unmap through the
|
|
|
|
// DMA handler provided.
|
|
|
|
for virtio_mem_device in self.virtio_mem_devices.iter() {
|
|
|
|
virtio_mem_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.add_dma_mapping_handler(
|
|
|
|
VirtioMemMappingSource::Device(pci_device_bdf.into()),
|
|
|
|
dma_handler.clone(),
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::AddDmaMappingHandlerVirtioMem)?;
|
|
|
|
}
|
2022-03-10 14:26:49 +00:00
|
|
|
|
2022-03-10 15:31:06 +00:00
|
|
|
// Do not register virtio-mem regions, as they are handled directly by
|
|
|
|
// virtio-mem devices.
|
|
|
|
for (_, zone) in self.memory_manager.lock().unwrap().memory_zones().iter() {
|
|
|
|
for region in zone.regions() {
|
|
|
|
let gpa = region.start_addr().0;
|
|
|
|
let size = region.len();
|
|
|
|
dma_handler
|
|
|
|
.map(gpa, gpa, size)
|
|
|
|
.map_err(DeviceManagerError::VirtioDmaMap)?;
|
|
|
|
}
|
2022-03-10 14:26:49 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-10-11 15:51:15 +00:00
|
|
|
let device_type = virtio_device.lock().unwrap().device_type();
|
2022-04-07 12:22:07 +00:00
|
|
|
let virtio_pci_device = Arc::new(Mutex::new(
|
|
|
|
VirtioPciDevice::new(
|
|
|
|
id.clone(),
|
|
|
|
memory,
|
|
|
|
virtio_device,
|
|
|
|
msix_num,
|
|
|
|
access_platform,
|
|
|
|
&self.msi_interrupt_manager,
|
|
|
|
pci_device_bdf.into(),
|
|
|
|
self.activate_evt
|
|
|
|
.try_clone()
|
|
|
|
.map_err(DeviceManagerError::EventFd)?,
|
|
|
|
// All device types *except* virtio block devices should be allocated a 64-bit bar
|
|
|
|
// The block devices should be given a 32-bit BAR so that they are easily accessible
|
|
|
|
// to firmware without requiring excessive identity mapping.
|
|
|
|
// The exception being if not on the default PCI segment.
|
|
|
|
pci_segment_id > 0 || device_type != VirtioDeviceType::Block as u32,
|
|
|
|
dma_handler,
|
2022-05-17 14:33:33 +00:00
|
|
|
self.pending_activations.clone(),
|
2022-10-21 15:57:20 +00:00
|
|
|
vm_migration::snapshot_from_id(self.snapshot.as_ref(), id.as_str()),
|
2022-04-07 12:22:07 +00:00
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::VirtioDevice)?,
|
|
|
|
));
|
2020-05-11 18:02:19 +00:00
|
|
|
|
2022-04-07 15:20:25 +00:00
|
|
|
let new_resources = self.add_pci_device(
|
2020-09-10 08:57:53 +00:00
|
|
|
virtio_pci_device.clone(),
|
2020-10-05 17:13:28 +00:00
|
|
|
virtio_pci_device.clone(),
|
2021-10-07 17:07:13 +00:00
|
|
|
pci_segment_id,
|
2020-09-10 08:57:53 +00:00
|
|
|
pci_device_bdf,
|
2022-04-07 12:22:07 +00:00
|
|
|
resources,
|
2020-09-10 08:57:53 +00:00
|
|
|
)?;
|
2019-09-04 13:55:14 +00:00
|
|
|
|
2020-09-10 08:57:53 +00:00
|
|
|
let bar_addr = virtio_pci_device.lock().unwrap().config_bar_addr();
|
|
|
|
for (event, addr) in virtio_pci_device.lock().unwrap().ioeventfds(bar_addr) {
|
2019-09-04 13:55:14 +00:00
|
|
|
let io_addr = IoEventAddress::Mmio(addr);
|
2020-01-31 16:07:08 +00:00
|
|
|
self.address_manager
|
2020-07-03 09:16:49 +00:00
|
|
|
.vm
|
2020-06-02 02:29:54 +00:00
|
|
|
.register_ioevent(event, &io_addr, None)
|
|
|
|
.map_err(|e| DeviceManagerError::RegisterIoevent(e.into()))?;
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2020-05-11 18:02:19 +00:00
|
|
|
// Update the device tree with correct resource information.
|
2022-04-07 12:22:07 +00:00
|
|
|
node.resources = new_resources;
|
2020-04-30 18:08:04 +00:00
|
|
|
node.migratable = Some(Arc::clone(&virtio_pci_device) as Arc<Mutex<dyn Migratable>>);
|
2022-02-16 08:55:26 +00:00
|
|
|
node.pci_bdf = Some(pci_device_bdf);
|
2021-03-18 12:09:52 +00:00
|
|
|
node.pci_device_handle = Some(PciDeviceHandle::Virtio(virtio_pci_device));
|
2020-05-12 13:53:09 +00:00
|
|
|
self.device_tree.lock().unwrap().insert(id, node);
|
2020-04-30 18:08:04 +00:00
|
|
|
|
2020-03-18 17:24:35 +00:00
|
|
|
Ok(pci_device_bdf)
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2023-06-15 04:02:45 +00:00
|
|
|
fn add_pvpanic_device(
|
|
|
|
&mut self,
|
|
|
|
) -> DeviceManagerResult<Option<Arc<Mutex<devices::PvPanicDevice>>>> {
|
|
|
|
let id = String::from(PVPANIC_DEVICE_NAME);
|
|
|
|
let pci_segment_id = 0x0_u16;
|
|
|
|
|
|
|
|
info!("Creating pvpanic device {}", id);
|
|
|
|
|
|
|
|
let (pci_segment_id, pci_device_bdf, resources) =
|
|
|
|
self.pci_resources(&id, pci_segment_id)?;
|
|
|
|
|
|
|
|
let snapshot = snapshot_from_id(self.snapshot.as_ref(), id.as_str());
|
|
|
|
|
|
|
|
let pvpanic_device = devices::PvPanicDevice::new(id.clone(), snapshot)
|
|
|
|
.map_err(DeviceManagerError::PvPanicCreate)?;
|
|
|
|
|
|
|
|
let pvpanic_device = Arc::new(Mutex::new(pvpanic_device));
|
|
|
|
|
|
|
|
let new_resources = self.add_pci_device(
|
|
|
|
pvpanic_device.clone(),
|
|
|
|
pvpanic_device.clone(),
|
|
|
|
pci_segment_id,
|
|
|
|
pci_device_bdf,
|
|
|
|
resources,
|
|
|
|
)?;
|
|
|
|
|
|
|
|
let mut node = device_node!(id, pvpanic_device);
|
|
|
|
|
|
|
|
node.resources = new_resources;
|
|
|
|
node.pci_bdf = Some(pci_device_bdf);
|
|
|
|
node.pci_device_handle = None;
|
|
|
|
|
|
|
|
self.device_tree.lock().unwrap().insert(id, node);
|
|
|
|
|
|
|
|
Ok(Some(pvpanic_device))
|
|
|
|
}
|
|
|
|
|
2022-04-07 14:33:22 +00:00
|
|
|
fn pci_resources(
|
|
|
|
&self,
|
|
|
|
id: &str,
|
|
|
|
pci_segment_id: u16,
|
|
|
|
) -> DeviceManagerResult<(u16, PciBdf, Option<Vec<Resource>>)> {
|
|
|
|
// Look for the id in the device tree. If it can be found, that means
|
|
|
|
// the device is being restored, otherwise it's created from scratch.
|
|
|
|
Ok(
|
|
|
|
if let Some(node) = self.device_tree.lock().unwrap().get(id) {
|
|
|
|
info!("Restoring virtio-pci {} resources", id);
|
|
|
|
let pci_device_bdf: PciBdf = node
|
|
|
|
.pci_bdf
|
|
|
|
.ok_or(DeviceManagerError::MissingDeviceNodePciBdf)?;
|
|
|
|
let pci_segment_id = pci_device_bdf.segment();
|
|
|
|
|
|
|
|
self.pci_segments[pci_segment_id as usize]
|
|
|
|
.pci_bus
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.get_device_id(pci_device_bdf.device() as usize)
|
|
|
|
.map_err(DeviceManagerError::GetPciDeviceId)?;
|
|
|
|
|
|
|
|
(pci_segment_id, pci_device_bdf, Some(node.resources.clone()))
|
|
|
|
} else {
|
|
|
|
let pci_device_bdf =
|
|
|
|
self.pci_segments[pci_segment_id as usize].next_device_bdf()?;
|
|
|
|
|
|
|
|
(pci_segment_id, pci_device_bdf, None)
|
|
|
|
},
|
|
|
|
)
|
|
|
|
}
|
|
|
|
|
2020-06-09 06:17:42 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2020-09-09 14:30:31 +00:00
|
|
|
pub fn io_bus(&self) -> &Arc<Bus> {
|
2020-03-04 15:24:15 +00:00
|
|
|
&self.address_manager.io_bus
|
2019-09-04 14:20:09 +00:00
|
|
|
}
|
|
|
|
|
2020-09-09 14:30:31 +00:00
|
|
|
pub fn mmio_bus(&self) -> &Arc<Bus> {
|
2019-10-23 22:14:13 +00:00
|
|
|
&self.address_manager.mmio_bus
|
2019-09-04 14:20:09 +00:00
|
|
|
}
|
|
|
|
|
2019-11-11 14:57:41 +00:00
|
|
|
pub fn allocator(&self) -> &Arc<Mutex<SystemAllocator>> {
|
|
|
|
&self.address_manager.allocator
|
|
|
|
}
|
|
|
|
|
2020-05-25 08:27:08 +00:00
|
|
|
pub fn interrupt_controller(&self) -> Option<Arc<Mutex<dyn InterruptController>>> {
|
2021-03-29 06:16:35 +00:00
|
|
|
self.interrupt_controller
|
|
|
|
.as_ref()
|
|
|
|
.map(|ic| ic.clone() as Arc<Mutex<dyn InterruptController>>)
|
2019-09-04 14:20:09 +00:00
|
|
|
}
|
2019-09-06 15:42:41 +00:00
|
|
|
|
2021-10-05 14:06:55 +00:00
|
|
|
pub(crate) fn pci_segments(&self) -> &Vec<PciSegment> {
|
|
|
|
&self.pci_segments
|
|
|
|
}
|
|
|
|
|
2019-09-06 15:42:41 +00:00
|
|
|
pub fn console(&self) -> &Arc<Console> {
|
|
|
|
&self.console
|
|
|
|
}
|
2019-09-11 15:22:00 +00:00
|
|
|
|
2022-04-28 14:21:14 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
2019-09-11 15:22:00 +00:00
|
|
|
pub fn cmdline_additions(&self) -> &[String] {
|
|
|
|
self.cmdline_additions.as_slice()
|
|
|
|
}
|
2019-10-02 20:57:20 +00:00
|
|
|
|
2021-02-26 10:53:02 +00:00
|
|
|
pub fn update_memory(&self, new_region: &Arc<GuestRegionMmap>) -> DeviceManagerResult<()> {
|
2022-03-10 13:05:36 +00:00
|
|
|
for handle in self.virtio_devices.iter() {
|
|
|
|
handle
|
|
|
|
.virtio_device
|
2020-03-23 11:10:26 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-03-05 13:44:54 +00:00
|
|
|
.add_memory_region(new_region)
|
2020-03-23 11:10:26 +00:00
|
|
|
.map_err(DeviceManagerError::UpdateMemoryForVirtioDevice)?;
|
2022-03-10 14:26:49 +00:00
|
|
|
|
|
|
|
if let Some(dma_handler) = &handle.dma_handler {
|
2022-03-10 15:31:06 +00:00
|
|
|
if !handle.iommu {
|
|
|
|
let gpa = new_region.start_addr().0;
|
|
|
|
let size = new_region.len();
|
|
|
|
dma_handler
|
|
|
|
.map(gpa, gpa, size)
|
|
|
|
.map_err(DeviceManagerError::VirtioDmaMap)?;
|
|
|
|
}
|
2022-03-10 14:26:49 +00:00
|
|
|
}
|
2020-03-23 11:10:26 +00:00
|
|
|
}
|
|
|
|
|
2020-03-26 13:53:43 +00:00
|
|
|
// Take care of updating the memory for VFIO PCI devices.
|
2021-09-14 09:02:10 +00:00
|
|
|
if let Some(vfio_container) = &self.vfio_container {
|
|
|
|
vfio_container
|
|
|
|
.vfio_dma_map(
|
|
|
|
new_region.start_addr().raw_value(),
|
2022-11-01 21:52:40 +00:00
|
|
|
new_region.len(),
|
2021-09-14 09:02:10 +00:00
|
|
|
new_region.as_ptr() as u64,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::UpdateMemoryForVfioPciDevice)?;
|
|
|
|
}
|
|
|
|
|
|
|
|
// Take care of updating the memory for vfio-user devices.
|
2021-03-18 12:09:52 +00:00
|
|
|
{
|
|
|
|
let device_tree = self.device_tree.lock().unwrap();
|
|
|
|
for pci_device_node in device_tree.pci_devices() {
|
2022-01-18 15:46:20 +00:00
|
|
|
if let PciDeviceHandle::VfioUser(vfio_user_pci_device) = pci_device_node
|
2021-03-18 12:09:52 +00:00
|
|
|
.pci_device_handle
|
|
|
|
.as_ref()
|
|
|
|
.ok_or(DeviceManagerError::MissingPciDevice)?
|
|
|
|
{
|
2022-01-18 15:46:20 +00:00
|
|
|
vfio_user_pci_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.dma_map(new_region)
|
|
|
|
.map_err(DeviceManagerError::UpdateMemoryForVfioUserPciDevice)?;
|
2021-03-18 12:09:52 +00:00
|
|
|
}
|
2020-03-26 13:53:43 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2020-03-23 11:10:26 +00:00
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
2020-11-09 13:29:05 +00:00
|
|
|
pub fn activate_virtio_devices(&self) -> DeviceManagerResult<()> {
|
2022-05-17 14:33:33 +00:00
|
|
|
for mut activator in self.pending_activations.lock().unwrap().drain(..) {
|
|
|
|
activator
|
|
|
|
.activate()
|
|
|
|
.map_err(DeviceManagerError::VirtioActivate)?;
|
2020-11-09 13:29:05 +00:00
|
|
|
}
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
2019-11-27 15:28:22 +00:00
|
|
|
pub fn notify_hotplug(
|
|
|
|
&self,
|
2021-01-12 15:10:05 +00:00
|
|
|
_notification_type: AcpiNotificationFlags,
|
2019-11-27 15:28:22 +00:00
|
|
|
) -> DeviceManagerResult<()> {
|
|
|
|
return self
|
|
|
|
.ged_notification_device
|
|
|
|
.as_ref()
|
|
|
|
.unwrap()
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.notify(_notification_type)
|
|
|
|
.map_err(DeviceManagerError::HotPlugNotification);
|
|
|
|
}
|
2020-02-27 13:00:46 +00:00
|
|
|
|
2020-06-11 14:48:25 +00:00
|
|
|
pub fn add_device(
|
|
|
|
&mut self,
|
|
|
|
device_cfg: &mut DeviceConfig,
|
|
|
|
) -> DeviceManagerResult<PciDeviceInfo> {
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
self.validate_identifier(&device_cfg.id)?;
|
|
|
|
|
2022-03-18 17:41:08 +00:00
|
|
|
if device_cfg.iommu && !self.is_iommu_segment(device_cfg.pci_segment) {
|
|
|
|
return Err(DeviceManagerError::InvalidIommuHotplug);
|
|
|
|
}
|
|
|
|
|
2021-10-18 16:29:42 +00:00
|
|
|
let (bdf, device_name) = self.add_passthrough_device(device_cfg)?;
|
2020-02-28 11:29:43 +00:00
|
|
|
|
|
|
|
// Update the PCIU bitmap
|
2021-10-18 16:29:42 +00:00
|
|
|
self.pci_segments[device_cfg.pci_segment as usize].pci_devices_up |= 1 << bdf.device();
|
2020-02-28 11:29:43 +00:00
|
|
|
|
2020-06-11 14:48:25 +00:00
|
|
|
Ok(PciDeviceInfo {
|
|
|
|
id: device_name,
|
2021-10-18 16:29:42 +00:00
|
|
|
bdf,
|
2020-06-11 14:48:25 +00:00
|
|
|
})
|
2020-02-27 13:00:46 +00:00
|
|
|
}
|
2020-03-06 15:53:20 +00:00
|
|
|
|
2021-07-30 14:48:58 +00:00
|
|
|
pub fn add_user_device(
|
|
|
|
&mut self,
|
|
|
|
device_cfg: &mut UserDeviceConfig,
|
|
|
|
) -> DeviceManagerResult<PciDeviceInfo> {
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
self.validate_identifier(&device_cfg.id)?;
|
|
|
|
|
2021-10-18 16:29:42 +00:00
|
|
|
let (bdf, device_name) = self.add_vfio_user_device(device_cfg)?;
|
2021-07-30 14:48:58 +00:00
|
|
|
|
|
|
|
// Update the PCIU bitmap
|
2021-10-18 16:29:42 +00:00
|
|
|
self.pci_segments[device_cfg.pci_segment as usize].pci_devices_up |= 1 << bdf.device();
|
2021-07-30 14:48:58 +00:00
|
|
|
|
|
|
|
Ok(PciDeviceInfo {
|
|
|
|
id: device_name,
|
2021-10-18 16:29:42 +00:00
|
|
|
bdf,
|
2021-07-30 14:48:58 +00:00
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2020-03-09 10:49:15 +00:00
|
|
|
pub fn remove_device(&mut self, id: String) -> DeviceManagerResult<()> {
|
2021-03-18 12:09:52 +00:00
|
|
|
// The node can be directly a PCI node in case the 'id' refers to a
|
|
|
|
// VFIO device or a virtio-pci one.
|
|
|
|
// In case the 'id' refers to a virtio device, we must find the PCI
|
|
|
|
// node by looking at the parent.
|
|
|
|
let device_tree = self.device_tree.lock().unwrap();
|
|
|
|
let node = device_tree
|
|
|
|
.get(&id)
|
|
|
|
.ok_or(DeviceManagerError::UnknownDeviceId(id))?;
|
|
|
|
|
|
|
|
let pci_device_node = if node.pci_bdf.is_some() && node.pci_device_handle.is_some() {
|
|
|
|
node
|
|
|
|
} else {
|
|
|
|
let parent = node
|
|
|
|
.parent
|
|
|
|
.as_ref()
|
|
|
|
.ok_or(DeviceManagerError::MissingNode)?;
|
|
|
|
device_tree
|
|
|
|
.get(parent)
|
|
|
|
.ok_or(DeviceManagerError::MissingNode)?
|
|
|
|
};
|
2020-04-27 17:12:00 +00:00
|
|
|
|
2021-10-18 16:29:42 +00:00
|
|
|
let pci_device_bdf: PciBdf = pci_device_node
|
2021-03-18 12:09:52 +00:00
|
|
|
.pci_bdf
|
2022-02-16 08:55:26 +00:00
|
|
|
.ok_or(DeviceManagerError::MissingDeviceNodePciBdf)?;
|
2021-10-26 10:19:01 +00:00
|
|
|
let pci_segment_id = pci_device_bdf.segment();
|
2021-10-08 08:27:13 +00:00
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
let pci_device_handle = pci_device_node
|
|
|
|
.pci_device_handle
|
|
|
|
.as_ref()
|
|
|
|
.ok_or(DeviceManagerError::MissingPciDevice)?;
|
|
|
|
#[allow(irrefutable_let_patterns)]
|
|
|
|
if let PciDeviceHandle::Virtio(virtio_pci_device) = pci_device_handle {
|
|
|
|
let device_type = VirtioDeviceType::from(
|
|
|
|
virtio_pci_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.virtio_device()
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.device_type(),
|
|
|
|
);
|
|
|
|
match device_type {
|
2021-03-25 16:54:09 +00:00
|
|
|
VirtioDeviceType::Net
|
|
|
|
| VirtioDeviceType::Block
|
|
|
|
| VirtioDeviceType::Pmem
|
|
|
|
| VirtioDeviceType::Fs
|
|
|
|
| VirtioDeviceType::Vsock => {}
|
2021-03-18 12:09:52 +00:00
|
|
|
_ => return Err(DeviceManagerError::RemovalNotAllowed(device_type)),
|
2021-03-17 13:14:41 +00:00
|
|
|
}
|
2020-03-09 10:49:15 +00:00
|
|
|
}
|
2021-03-17 13:14:41 +00:00
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
// Update the PCID bitmap
|
2021-10-18 16:29:42 +00:00
|
|
|
self.pci_segments[pci_segment_id as usize].pci_devices_down |= 1 << pci_device_bdf.device();
|
2021-03-18 12:09:52 +00:00
|
|
|
|
|
|
|
Ok(())
|
2020-03-09 10:49:15 +00:00
|
|
|
}
|
|
|
|
|
2021-10-07 17:07:13 +00:00
|
|
|
pub fn eject_device(&mut self, pci_segment_id: u16, device_id: u8) -> DeviceManagerResult<()> {
|
2021-10-08 09:49:26 +00:00
|
|
|
info!(
|
|
|
|
"Ejecting device_id = {} on segment_id={}",
|
|
|
|
device_id, pci_segment_id
|
|
|
|
);
|
|
|
|
|
2020-03-06 16:52:40 +00:00
|
|
|
// Convert the device ID into the corresponding b/d/f.
|
2021-10-18 16:29:42 +00:00
|
|
|
let pci_device_bdf = PciBdf::new(pci_segment_id, 0, device_id, 0);
|
2020-03-06 16:52:40 +00:00
|
|
|
|
2020-03-09 15:32:27 +00:00
|
|
|
// Give the PCI device ID back to the PCI bus.
|
2021-10-07 17:07:13 +00:00
|
|
|
self.pci_segments[pci_segment_id as usize]
|
2021-10-04 14:33:29 +00:00
|
|
|
.pci_bus
|
|
|
|
.lock()
|
2020-03-09 15:32:27 +00:00
|
|
|
.unwrap()
|
|
|
|
.put_device_id(device_id as usize)
|
|
|
|
.map_err(DeviceManagerError::PutPciDeviceId)?;
|
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
// Remove the device from the device tree along with its children.
|
|
|
|
let mut device_tree = self.device_tree.lock().unwrap();
|
|
|
|
let pci_device_node = device_tree
|
2022-02-16 08:55:26 +00:00
|
|
|
.remove_node_by_pci_bdf(pci_device_bdf)
|
2021-03-18 12:09:52 +00:00
|
|
|
.ok_or(DeviceManagerError::MissingPciDevice)?;
|
2022-05-03 10:01:46 +00:00
|
|
|
|
|
|
|
// For VFIO and vfio-user the PCI device id is the id.
|
|
|
|
// For virtio we overwrite it later as we want the id of the
|
|
|
|
// underlying device.
|
|
|
|
let mut id = pci_device_node.id;
|
|
|
|
let pci_device_handle = pci_device_node
|
|
|
|
.pci_device_handle
|
|
|
|
.ok_or(DeviceManagerError::MissingPciDevice)?;
|
|
|
|
if matches!(pci_device_handle, PciDeviceHandle::Virtio(_)) {
|
|
|
|
// The virtio-pci device has a single child
|
|
|
|
if !pci_device_node.children.is_empty() {
|
|
|
|
assert_eq!(pci_device_node.children.len(), 1);
|
|
|
|
let child_id = &pci_device_node.children[0];
|
|
|
|
id = child_id.clone();
|
|
|
|
}
|
|
|
|
}
|
2021-03-18 12:09:52 +00:00
|
|
|
for child in pci_device_node.children.iter() {
|
|
|
|
device_tree.remove(child);
|
|
|
|
}
|
2021-03-03 10:59:53 +00:00
|
|
|
|
2022-03-10 15:31:06 +00:00
|
|
|
let mut iommu_attached = false;
|
|
|
|
if let Some((_, iommu_attached_devices)) = &self.iommu_attached_devices {
|
|
|
|
if iommu_attached_devices.contains(&pci_device_bdf) {
|
|
|
|
iommu_attached = true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2022-03-10 14:26:49 +00:00
|
|
|
let (pci_device, bus_device, virtio_device, remove_dma_handler) = match pci_device_handle {
|
2021-08-11 10:53:21 +00:00
|
|
|
// No need to remove any virtio-mem mapping here as the container outlives all devices
|
2021-09-14 09:02:10 +00:00
|
|
|
PciDeviceHandle::Vfio(vfio_pci_device) => (
|
|
|
|
Arc::clone(&vfio_pci_device) as Arc<Mutex<dyn PciDevice>>,
|
|
|
|
Arc::clone(&vfio_pci_device) as Arc<Mutex<dyn BusDevice>>,
|
2022-03-10 11:30:41 +00:00
|
|
|
None as Option<Arc<Mutex<dyn virtio_devices::VirtioDevice>>>,
|
2022-03-10 14:26:49 +00:00
|
|
|
false,
|
2021-09-14 09:02:10 +00:00
|
|
|
),
|
2021-03-18 12:09:52 +00:00
|
|
|
PciDeviceHandle::Virtio(virtio_pci_device) => {
|
2022-03-10 14:26:49 +00:00
|
|
|
let dev = virtio_pci_device.lock().unwrap();
|
|
|
|
let bar_addr = dev.config_bar_addr();
|
|
|
|
for (event, addr) in dev.ioeventfds(bar_addr) {
|
2021-03-18 12:09:52 +00:00
|
|
|
let io_addr = IoEventAddress::Mmio(addr);
|
|
|
|
self.address_manager
|
|
|
|
.vm
|
|
|
|
.unregister_ioevent(event, &io_addr)
|
|
|
|
.map_err(|e| DeviceManagerError::UnRegisterIoevent(e.into()))?;
|
|
|
|
}
|
2020-03-11 09:05:37 +00:00
|
|
|
|
2022-03-10 14:26:49 +00:00
|
|
|
if let Some(dma_handler) = dev.dma_handler() {
|
2022-03-10 15:31:06 +00:00
|
|
|
if !iommu_attached {
|
|
|
|
for (_, zone) in self.memory_manager.lock().unwrap().memory_zones().iter() {
|
|
|
|
for region in zone.regions() {
|
|
|
|
let iova = region.start_addr().0;
|
|
|
|
let size = region.len();
|
|
|
|
dma_handler
|
|
|
|
.unmap(iova, size)
|
|
|
|
.map_err(DeviceManagerError::VirtioDmaUnmap)?;
|
|
|
|
}
|
2022-03-10 14:26:49 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
(
|
|
|
|
Arc::clone(&virtio_pci_device) as Arc<Mutex<dyn PciDevice>>,
|
|
|
|
Arc::clone(&virtio_pci_device) as Arc<Mutex<dyn BusDevice>>,
|
2022-03-10 14:26:49 +00:00
|
|
|
Some(dev.virtio_device()),
|
2022-03-10 15:31:06 +00:00
|
|
|
dev.dma_handler().is_some() && !iommu_attached,
|
2021-03-18 12:09:52 +00:00
|
|
|
)
|
|
|
|
}
|
2021-06-09 13:43:36 +00:00
|
|
|
PciDeviceHandle::VfioUser(vfio_user_pci_device) => {
|
|
|
|
let mut dev = vfio_user_pci_device.lock().unwrap();
|
|
|
|
for (_, zone) in self.memory_manager.lock().unwrap().memory_zones().iter() {
|
|
|
|
for region in zone.regions() {
|
|
|
|
dev.dma_unmap(region)
|
|
|
|
.map_err(DeviceManagerError::VfioUserDmaUnmap)?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
(
|
|
|
|
Arc::clone(&vfio_user_pci_device) as Arc<Mutex<dyn PciDevice>>,
|
|
|
|
Arc::clone(&vfio_user_pci_device) as Arc<Mutex<dyn BusDevice>>,
|
2022-03-10 11:30:41 +00:00
|
|
|
None as Option<Arc<Mutex<dyn virtio_devices::VirtioDevice>>>,
|
2022-03-10 14:26:49 +00:00
|
|
|
true,
|
2021-06-09 13:43:36 +00:00
|
|
|
)
|
|
|
|
}
|
2021-03-18 12:09:52 +00:00
|
|
|
};
|
2020-03-06 16:52:40 +00:00
|
|
|
|
2022-03-10 14:26:49 +00:00
|
|
|
if remove_dma_handler {
|
|
|
|
for virtio_mem_device in self.virtio_mem_devices.iter() {
|
|
|
|
virtio_mem_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.remove_dma_mapping_handler(VirtioMemMappingSource::Device(
|
|
|
|
pci_device_bdf.into(),
|
|
|
|
))
|
|
|
|
.map_err(DeviceManagerError::RemoveDmaMappingHandlerVirtioMem)?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
// Free the allocated BARs
|
|
|
|
pci_device
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2021-10-15 10:38:06 +00:00
|
|
|
.free_bars(
|
|
|
|
&mut self.address_manager.allocator.lock().unwrap(),
|
|
|
|
&mut self.pci_segments[pci_segment_id as usize]
|
2023-11-14 19:31:57 +00:00
|
|
|
.mem32_allocator
|
|
|
|
.lock()
|
|
|
|
.unwrap(),
|
|
|
|
&mut self.pci_segments[pci_segment_id as usize]
|
|
|
|
.mem64_allocator
|
2021-10-15 10:38:06 +00:00
|
|
|
.lock()
|
|
|
|
.unwrap(),
|
|
|
|
)
|
2021-03-18 12:09:52 +00:00
|
|
|
.map_err(DeviceManagerError::FreePciBars)?;
|
2020-03-06 16:52:40 +00:00
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
// Remove the device from the PCI bus
|
2021-10-07 17:07:13 +00:00
|
|
|
self.pci_segments[pci_segment_id as usize]
|
2021-10-04 14:33:29 +00:00
|
|
|
.pci_bus
|
|
|
|
.lock()
|
2021-03-18 12:09:52 +00:00
|
|
|
.unwrap()
|
|
|
|
.remove_by_device(&pci_device)
|
|
|
|
.map_err(DeviceManagerError::RemoveDeviceFromPciBus)?;
|
2020-04-20 15:40:11 +00:00
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
// Remove the device from the IO bus
|
|
|
|
self.io_bus()
|
|
|
|
.remove_by_device(&bus_device)
|
|
|
|
.map_err(DeviceManagerError::RemoveDeviceFromIoBus)?;
|
|
|
|
|
|
|
|
// Remove the device from the MMIO bus
|
|
|
|
self.mmio_bus()
|
|
|
|
.remove_by_device(&bus_device)
|
|
|
|
.map_err(DeviceManagerError::RemoveDeviceFromMmioBus)?;
|
|
|
|
|
|
|
|
// Remove the device from the list of BusDevice held by the
|
|
|
|
// DeviceManager.
|
|
|
|
self.bus_devices
|
|
|
|
.retain(|dev| !Arc::ptr_eq(dev, &bus_device));
|
2020-04-20 15:40:11 +00:00
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
// Shutdown and remove the underlying virtio-device if present
|
|
|
|
if let Some(virtio_device) = virtio_device {
|
|
|
|
for mapping in virtio_device.lock().unwrap().userspace_mappings() {
|
|
|
|
self.memory_manager
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.remove_userspace_mapping(
|
|
|
|
mapping.addr.raw_value(),
|
|
|
|
mapping.len,
|
|
|
|
mapping.host_addr,
|
|
|
|
mapping.mergeable,
|
|
|
|
mapping.mem_slot,
|
|
|
|
)
|
|
|
|
.map_err(DeviceManagerError::MemoryManager)?;
|
2020-04-08 14:39:50 +00:00
|
|
|
}
|
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
virtio_device.lock().unwrap().shutdown();
|
2020-03-06 16:52:40 +00:00
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
self.virtio_devices
|
2022-03-10 13:05:36 +00:00
|
|
|
.retain(|handler| !Arc::ptr_eq(&handler.virtio_device, &virtio_device));
|
2020-03-06 16:52:40 +00:00
|
|
|
}
|
2021-03-18 12:09:52 +00:00
|
|
|
|
2022-05-03 10:01:46 +00:00
|
|
|
event!(
|
|
|
|
"vm",
|
|
|
|
"device-removed",
|
|
|
|
"id",
|
|
|
|
&id,
|
|
|
|
"bdf",
|
|
|
|
pci_device_bdf.to_string()
|
|
|
|
);
|
|
|
|
|
2021-03-18 12:09:52 +00:00
|
|
|
// At this point, the device has been removed from all the list and
|
|
|
|
// buses where it was stored. At the end of this function, after
|
|
|
|
// any_device, bus_device and pci_device are released, the actual
|
|
|
|
// device will be dropped.
|
|
|
|
Ok(())
|
2020-03-06 15:53:20 +00:00
|
|
|
}
|
2020-03-23 16:18:10 +00:00
|
|
|
|
|
|
|
fn hotplug_virtio_pci_device(
|
|
|
|
&mut self,
|
2022-03-10 13:05:36 +00:00
|
|
|
handle: MetaVirtioDevice,
|
2020-06-11 14:48:25 +00:00
|
|
|
) -> DeviceManagerResult<PciDeviceInfo> {
|
2020-04-20 17:35:16 +00:00
|
|
|
// Add the virtio device to the device manager list. This is important
|
|
|
|
// as the list is used to notify virtio devices about memory updates
|
|
|
|
// for instance.
|
2022-03-10 13:05:36 +00:00
|
|
|
self.virtio_devices.push(handle.clone());
|
2020-04-20 17:35:16 +00:00
|
|
|
|
2022-03-18 16:57:12 +00:00
|
|
|
let mapping: Option<Arc<IommuMapping>> = if handle.iommu {
|
|
|
|
self.iommu_mapping.clone()
|
|
|
|
} else {
|
|
|
|
None
|
|
|
|
};
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
let bdf = self.add_virtio_pci_device(
|
|
|
|
handle.virtio_device,
|
2022-03-18 16:57:12 +00:00
|
|
|
&mapping,
|
2022-03-10 13:05:36 +00:00
|
|
|
handle.id.clone(),
|
|
|
|
handle.pci_segment,
|
2022-03-10 13:16:08 +00:00
|
|
|
handle.dma_handler,
|
2022-03-10 13:05:36 +00:00
|
|
|
)?;
|
2020-03-23 16:18:10 +00:00
|
|
|
|
|
|
|
// Update the PCIU bitmap
|
2022-03-10 13:05:36 +00:00
|
|
|
self.pci_segments[handle.pci_segment as usize].pci_devices_up |= 1 << bdf.device();
|
2020-03-23 16:18:10 +00:00
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
Ok(PciDeviceInfo { id: handle.id, bdf })
|
2020-03-23 16:18:10 +00:00
|
|
|
}
|
|
|
|
|
2022-03-18 17:41:08 +00:00
|
|
|
fn is_iommu_segment(&self, pci_segment_id: u16) -> bool {
|
|
|
|
self.config
|
|
|
|
.lock()
|
|
|
|
.as_ref()
|
|
|
|
.unwrap()
|
|
|
|
.platform
|
|
|
|
.as_ref()
|
|
|
|
.map(|pc| {
|
|
|
|
pc.iommu_segments
|
|
|
|
.as_ref()
|
|
|
|
.map(|v| v.contains(&pci_segment_id))
|
|
|
|
.unwrap_or_default()
|
|
|
|
})
|
|
|
|
.unwrap_or_default()
|
|
|
|
}
|
|
|
|
|
2020-06-11 14:48:25 +00:00
|
|
|
pub fn add_disk(&mut self, disk_cfg: &mut DiskConfig) -> DeviceManagerResult<PciDeviceInfo> {
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
self.validate_identifier(&disk_cfg.id)?;
|
|
|
|
|
2022-03-18 17:41:08 +00:00
|
|
|
if disk_cfg.iommu && !self.is_iommu_segment(disk_cfg.pci_segment) {
|
|
|
|
return Err(DeviceManagerError::InvalidIommuHotplug);
|
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
let device = self.make_virtio_block_device(disk_cfg)?;
|
|
|
|
self.hotplug_virtio_pci_device(device)
|
2020-03-23 16:18:10 +00:00
|
|
|
}
|
2020-03-23 16:18:10 +00:00
|
|
|
|
2020-06-11 14:48:25 +00:00
|
|
|
pub fn add_fs(&mut self, fs_cfg: &mut FsConfig) -> DeviceManagerResult<PciDeviceInfo> {
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
self.validate_identifier(&fs_cfg.id)?;
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
let device = self.make_virtio_fs_device(fs_cfg)?;
|
|
|
|
self.hotplug_virtio_pci_device(device)
|
2020-04-14 09:21:24 +00:00
|
|
|
}
|
|
|
|
|
2020-06-11 14:48:25 +00:00
|
|
|
pub fn add_pmem(&mut self, pmem_cfg: &mut PmemConfig) -> DeviceManagerResult<PciDeviceInfo> {
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
self.validate_identifier(&pmem_cfg.id)?;
|
|
|
|
|
2022-03-18 17:41:08 +00:00
|
|
|
if pmem_cfg.iommu && !self.is_iommu_segment(pmem_cfg.pci_segment) {
|
|
|
|
return Err(DeviceManagerError::InvalidIommuHotplug);
|
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
let device = self.make_virtio_pmem_device(pmem_cfg)?;
|
|
|
|
self.hotplug_virtio_pci_device(device)
|
2020-03-23 16:18:10 +00:00
|
|
|
}
|
2020-03-23 16:18:10 +00:00
|
|
|
|
2020-06-11 14:48:25 +00:00
|
|
|
pub fn add_net(&mut self, net_cfg: &mut NetConfig) -> DeviceManagerResult<PciDeviceInfo> {
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
self.validate_identifier(&net_cfg.id)?;
|
|
|
|
|
2022-03-18 17:41:08 +00:00
|
|
|
if net_cfg.iommu && !self.is_iommu_segment(net_cfg.pci_segment) {
|
|
|
|
return Err(DeviceManagerError::InvalidIommuHotplug);
|
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
let device = self.make_virtio_net_device(net_cfg)?;
|
|
|
|
self.hotplug_virtio_pci_device(device)
|
2020-03-23 16:18:10 +00:00
|
|
|
}
|
2020-04-28 14:59:08 +00:00
|
|
|
|
2022-03-11 10:32:53 +00:00
|
|
|
pub fn add_vdpa(&mut self, vdpa_cfg: &mut VdpaConfig) -> DeviceManagerResult<PciDeviceInfo> {
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
self.validate_identifier(&vdpa_cfg.id)?;
|
|
|
|
|
2022-04-04 09:29:54 +00:00
|
|
|
if vdpa_cfg.iommu && !self.is_iommu_segment(vdpa_cfg.pci_segment) {
|
|
|
|
return Err(DeviceManagerError::InvalidIommuHotplug);
|
|
|
|
}
|
|
|
|
|
2022-03-11 10:32:53 +00:00
|
|
|
let device = self.make_vdpa_device(vdpa_cfg)?;
|
|
|
|
self.hotplug_virtio_pci_device(device)
|
|
|
|
}
|
|
|
|
|
2020-06-11 14:48:25 +00:00
|
|
|
pub fn add_vsock(&mut self, vsock_cfg: &mut VsockConfig) -> DeviceManagerResult<PciDeviceInfo> {
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
self.validate_identifier(&vsock_cfg.id)?;
|
|
|
|
|
2022-03-18 17:41:08 +00:00
|
|
|
if vsock_cfg.iommu && !self.is_iommu_segment(vsock_cfg.pci_segment) {
|
|
|
|
return Err(DeviceManagerError::InvalidIommuHotplug);
|
|
|
|
}
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
let device = self.make_virtio_vsock_device(vsock_cfg)?;
|
|
|
|
self.hotplug_virtio_pci_device(device)
|
2020-04-28 14:59:08 +00:00
|
|
|
}
|
2020-06-24 11:53:19 +00:00
|
|
|
|
|
|
|
pub fn counters(&self) -> HashMap<String, HashMap<&'static str, Wrapping<u64>>> {
|
|
|
|
let mut counters = HashMap::new();
|
|
|
|
|
2022-03-10 13:05:36 +00:00
|
|
|
for handle in &self.virtio_devices {
|
|
|
|
let virtio_device = handle.virtio_device.lock().unwrap();
|
2020-06-24 11:53:19 +00:00
|
|
|
if let Some(device_counters) = virtio_device.counters() {
|
2022-03-10 13:05:36 +00:00
|
|
|
counters.insert(handle.id.clone(), device_counters.clone());
|
2020-06-24 11:53:19 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
counters
|
|
|
|
}
|
2020-10-14 10:04:42 +00:00
|
|
|
|
|
|
|
pub fn resize_balloon(&mut self, size: u64) -> DeviceManagerResult<()> {
|
|
|
|
if let Some(balloon) = &self.balloon {
|
|
|
|
return balloon
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.resize(size)
|
|
|
|
.map_err(DeviceManagerError::VirtioBalloonResize);
|
|
|
|
}
|
|
|
|
|
|
|
|
warn!("No balloon setup: Can't resize the balloon");
|
|
|
|
Err(DeviceManagerError::MissingVirtioBalloon)
|
|
|
|
}
|
|
|
|
|
|
|
|
pub fn balloon_size(&self) -> u64 {
|
|
|
|
if let Some(balloon) = &self.balloon {
|
|
|
|
return balloon.lock().unwrap().get_actual();
|
|
|
|
}
|
|
|
|
|
|
|
|
0
|
|
|
|
}
|
2020-11-30 09:06:13 +00:00
|
|
|
|
|
|
|
pub fn device_tree(&self) -> Arc<Mutex<DeviceTree>> {
|
|
|
|
self.device_tree.clone()
|
|
|
|
}
|
2021-01-13 10:04:33 +00:00
|
|
|
|
2021-03-07 13:39:11 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2021-01-13 10:04:33 +00:00
|
|
|
pub fn notify_power_button(&self) -> DeviceManagerResult<()> {
|
2021-01-30 01:02:16 +00:00
|
|
|
self.ged_notification_device
|
2021-01-13 10:04:33 +00:00
|
|
|
.as_ref()
|
|
|
|
.unwrap()
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.notify(AcpiNotificationFlags::POWER_BUTTON_CHANGED)
|
2021-01-30 01:02:16 +00:00
|
|
|
.map_err(DeviceManagerError::PowerButtonNotification)
|
2021-01-13 10:04:33 +00:00
|
|
|
}
|
2021-03-07 13:39:11 +00:00
|
|
|
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
pub fn notify_power_button(&self) -> DeviceManagerResult<()> {
|
2022-03-28 10:53:22 +00:00
|
|
|
// There are two use cases:
|
|
|
|
// 1. Users will use direct kernel boot with device tree.
|
|
|
|
// 2. Users will use ACPI+UEFI boot.
|
|
|
|
|
2023-08-31 13:00:19 +00:00
|
|
|
// Trigger a GPIO pin 3 event to satisfy use case 1.
|
2022-03-28 10:53:22 +00:00
|
|
|
self.gpio_device
|
2021-03-07 13:39:11 +00:00
|
|
|
.as_ref()
|
|
|
|
.unwrap()
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.trigger_key(3)
|
2022-03-28 10:53:22 +00:00
|
|
|
.map_err(DeviceManagerError::AArch64PowerButtonNotification)?;
|
2023-08-31 13:00:19 +00:00
|
|
|
// Trigger a GED power button event to satisfy use case 2.
|
2022-03-28 10:53:22 +00:00
|
|
|
return self
|
|
|
|
.ged_notification_device
|
|
|
|
.as_ref()
|
|
|
|
.unwrap()
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.notify(AcpiNotificationFlags::POWER_BUTTON_CHANGED)
|
|
|
|
.map_err(DeviceManagerError::PowerButtonNotification);
|
2021-03-07 13:39:11 +00:00
|
|
|
}
|
2021-06-14 13:38:24 +00:00
|
|
|
|
2021-10-18 16:29:42 +00:00
|
|
|
pub fn iommu_attached_devices(&self) -> &Option<(PciBdf, Vec<PciBdf>)> {
|
2021-06-14 13:38:24 +00:00
|
|
|
&self.iommu_attached_devices
|
|
|
|
}
|
2022-04-05 02:52:22 +00:00
|
|
|
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
fn validate_identifier(&self, id: &Option<String>) -> DeviceManagerResult<()> {
|
|
|
|
if let Some(id) = id {
|
2022-05-04 09:36:26 +00:00
|
|
|
if id.starts_with("__") {
|
|
|
|
return Err(DeviceManagerError::InvalidIdentifier(id.clone()));
|
|
|
|
}
|
|
|
|
|
vmm: Ensure hotplugged device identifier is unique
Whenever a device (virtio, vfio, vfio-user or vdpa) is hotplugged, we
must verify the provided identifier is unique, otherwise we must return
an error.
Particularly, this will prevent issues with identifiers for serial,
console, IOAPIC, balloon, rng, watchdog, iommu and gpio since all of
these are hardcoded by the VMM.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2022-04-29 08:23:07 +00:00
|
|
|
if self.device_tree.lock().unwrap().contains_key(id) {
|
|
|
|
return Err(DeviceManagerError::IdentifierNotUnique(id.clone()));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
Ok(())
|
|
|
|
}
|
2022-07-25 12:36:31 +00:00
|
|
|
|
|
|
|
pub(crate) fn acpi_platform_addresses(&self) -> &AcpiPlatformAddresses {
|
|
|
|
&self.acpi_platform_addresses
|
|
|
|
}
|
2019-09-04 13:55:14 +00:00
|
|
|
}
|
|
|
|
|
2020-09-11 14:15:29 +00:00
|
|
|
fn numa_node_id_from_memory_zone_id(numa_nodes: &NumaNodes, memory_zone_id: &str) -> Option<u32> {
|
|
|
|
for (numa_node_id, numa_node) in numa_nodes.iter() {
|
2021-08-06 23:28:42 +00:00
|
|
|
if numa_node.memory_zones.contains(&memory_zone_id.to_owned()) {
|
2020-09-11 14:15:29 +00:00
|
|
|
return Some(*numa_node_id);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
None
|
|
|
|
}
|
|
|
|
|
2023-10-16 02:36:34 +00:00
|
|
|
fn numa_node_id_from_pci_segment_id(numa_nodes: &NumaNodes, pci_segment_id: u16) -> u32 {
|
|
|
|
for (numa_node_id, numa_node) in numa_nodes.iter() {
|
|
|
|
if numa_node.pci_segments.contains(&pci_segment_id) {
|
|
|
|
return *numa_node_id;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
0
|
|
|
|
}
|
|
|
|
|
2022-08-12 20:28:04 +00:00
|
|
|
struct TpmDevice {}
|
|
|
|
|
|
|
|
impl Aml for TpmDevice {
|
2023-03-02 08:01:33 +00:00
|
|
|
fn to_aml_bytes(&self, sink: &mut dyn acpi_tables::AmlSink) {
|
2022-08-12 20:28:04 +00:00
|
|
|
aml::Device::new(
|
|
|
|
"TPM2".into(),
|
|
|
|
vec![
|
|
|
|
&aml::Name::new("_HID".into(), &"MSFT0101"),
|
|
|
|
&aml::Name::new("_STA".into(), &(0xF_usize)),
|
|
|
|
&aml::Name::new(
|
|
|
|
"_CRS".into(),
|
|
|
|
&aml::ResourceTemplate::new(vec![&aml::Memory32Fixed::new(
|
|
|
|
true,
|
|
|
|
layout::TPM_START.0 as u32,
|
|
|
|
layout::TPM_SIZE as u32,
|
|
|
|
)]),
|
|
|
|
),
|
|
|
|
],
|
|
|
|
)
|
2023-03-02 08:01:33 +00:00
|
|
|
.to_aml_bytes(sink)
|
2022-08-12 20:28:04 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-12-06 16:14:32 +00:00
|
|
|
impl Aml for DeviceManager {
|
2023-03-02 08:01:33 +00:00
|
|
|
fn to_aml_bytes(&self, sink: &mut dyn acpi_tables::AmlSink) {
|
2021-02-02 23:27:56 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
use arch::aarch64::DeviceInfoForFdt;
|
|
|
|
|
2021-10-08 09:49:26 +00:00
|
|
|
let mut pci_scan_methods = Vec::new();
|
|
|
|
for i in 0..self.pci_segments.len() {
|
|
|
|
pci_scan_methods.push(aml::MethodCall::new(
|
2023-04-27 17:07:09 +00:00
|
|
|
format!("\\_SB_.PC{i:02X}.PCNT").as_str().into(),
|
2021-10-08 09:49:26 +00:00
|
|
|
vec![],
|
|
|
|
));
|
|
|
|
}
|
|
|
|
let mut pci_scan_inner: Vec<&dyn Aml> = Vec::new();
|
|
|
|
for method in &pci_scan_methods {
|
|
|
|
pci_scan_inner.push(method)
|
|
|
|
}
|
|
|
|
|
2020-02-26 16:03:15 +00:00
|
|
|
// PCI hotplug controller
|
2021-11-04 17:12:59 +00:00
|
|
|
aml::Device::new(
|
|
|
|
"_SB_.PHPR".into(),
|
|
|
|
vec![
|
2023-03-02 08:01:33 +00:00
|
|
|
&aml::Name::new("_HID".into(), &aml::EISAName::new("PNP0A06")),
|
2021-11-04 17:12:59 +00:00
|
|
|
&aml::Name::new("_STA".into(), &0x0bu8),
|
|
|
|
&aml::Name::new("_UID".into(), &"PCI Hotplug Controller"),
|
|
|
|
&aml::Mutex::new("BLCK".into(), 0),
|
|
|
|
&aml::Name::new(
|
|
|
|
"_CRS".into(),
|
|
|
|
&aml::ResourceTemplate::new(vec![&aml::AddressSpace::new_memory(
|
2023-09-07 12:02:03 +00:00
|
|
|
aml::AddressSpaceCacheable::NotCacheable,
|
2020-02-26 16:03:15 +00:00
|
|
|
true,
|
2022-11-01 21:52:40 +00:00
|
|
|
self.acpi_address.0,
|
2021-11-04 17:12:59 +00:00
|
|
|
self.acpi_address.0 + DEVICE_MANAGER_ACPI_SIZE as u64 - 1,
|
2023-06-08 15:49:52 +00:00
|
|
|
None,
|
2021-11-04 17:12:59 +00:00
|
|
|
)]),
|
|
|
|
),
|
|
|
|
// OpRegion and Fields map MMIO range into individual field values
|
|
|
|
&aml::OpRegion::new(
|
|
|
|
"PCST".into(),
|
|
|
|
aml::OpRegionSpace::SystemMemory,
|
2023-03-02 08:01:33 +00:00
|
|
|
&(self.acpi_address.0 as usize),
|
|
|
|
&DEVICE_MANAGER_ACPI_SIZE,
|
2021-11-04 17:12:59 +00:00
|
|
|
),
|
|
|
|
&aml::Field::new(
|
|
|
|
"PCST".into(),
|
|
|
|
aml::FieldAccessType::DWord,
|
2023-03-02 08:01:33 +00:00
|
|
|
aml::FieldLockRule::NoLock,
|
2021-11-04 17:12:59 +00:00
|
|
|
aml::FieldUpdateRule::WriteAsZeroes,
|
|
|
|
vec![
|
|
|
|
aml::FieldEntry::Named(*b"PCIU", 32),
|
|
|
|
aml::FieldEntry::Named(*b"PCID", 32),
|
|
|
|
aml::FieldEntry::Named(*b"B0EJ", 32),
|
|
|
|
aml::FieldEntry::Named(*b"PSEG", 32),
|
|
|
|
],
|
|
|
|
),
|
|
|
|
&aml::Method::new(
|
|
|
|
"PCEJ".into(),
|
|
|
|
2,
|
|
|
|
true,
|
|
|
|
vec![
|
|
|
|
// Take lock defined above
|
|
|
|
&aml::Acquire::new("BLCK".into(), 0xffff),
|
|
|
|
// Choose the current segment
|
|
|
|
&aml::Store::new(&aml::Path::new("PSEG"), &aml::Arg(1)),
|
|
|
|
// Write PCI bus number (in first argument) to I/O port via field
|
|
|
|
&aml::ShiftLeft::new(&aml::Path::new("B0EJ"), &aml::ONE, &aml::Arg(0)),
|
|
|
|
// Release lock
|
|
|
|
&aml::Release::new("BLCK".into()),
|
|
|
|
// Return 0
|
|
|
|
&aml::Return::new(&aml::ZERO),
|
|
|
|
],
|
|
|
|
),
|
|
|
|
&aml::Method::new("PSCN".into(), 0, true, pci_scan_inner),
|
|
|
|
],
|
|
|
|
)
|
2023-03-02 08:01:33 +00:00
|
|
|
.to_aml_bytes(sink);
|
2021-11-04 17:12:59 +00:00
|
|
|
|
|
|
|
for segment in &self.pci_segments {
|
2023-03-02 08:01:33 +00:00
|
|
|
segment.to_aml_bytes(sink);
|
2021-11-04 17:12:59 +00:00
|
|
|
}
|
2020-02-26 16:03:15 +00:00
|
|
|
|
2021-12-08 13:20:30 +00:00
|
|
|
let mut mbrd_memory = Vec::new();
|
|
|
|
|
|
|
|
for segment in &self.pci_segments {
|
|
|
|
mbrd_memory.push(aml::Memory32Fixed::new(
|
|
|
|
true,
|
|
|
|
segment.mmio_config_address as u32,
|
|
|
|
layout::PCI_MMIO_CONFIG_SIZE_PER_SEGMENT as u32,
|
|
|
|
))
|
|
|
|
}
|
|
|
|
|
|
|
|
let mut mbrd_memory_refs = Vec::new();
|
|
|
|
for mbrd_memory_ref in &mbrd_memory {
|
|
|
|
mbrd_memory_refs.push(mbrd_memory_ref as &dyn Aml);
|
|
|
|
}
|
|
|
|
|
2021-11-04 17:12:59 +00:00
|
|
|
aml::Device::new(
|
2019-12-06 16:14:32 +00:00
|
|
|
"_SB_.MBRD".into(),
|
|
|
|
vec![
|
2023-03-02 08:01:33 +00:00
|
|
|
&aml::Name::new("_HID".into(), &aml::EISAName::new("PNP0C02")),
|
2019-12-06 16:14:32 +00:00
|
|
|
&aml::Name::new("_UID".into(), &aml::ZERO),
|
2021-12-08 13:20:30 +00:00
|
|
|
&aml::Name::new("_CRS".into(), &aml::ResourceTemplate::new(mbrd_memory_refs)),
|
2019-12-06 16:14:32 +00:00
|
|
|
],
|
|
|
|
)
|
2023-03-02 08:01:33 +00:00
|
|
|
.to_aml_bytes(sink);
|
2019-12-06 16:14:32 +00:00
|
|
|
|
2021-02-02 23:27:56 +00:00
|
|
|
// Serial device
|
|
|
|
#[cfg(target_arch = "x86_64")]
|
|
|
|
let serial_irq = 4;
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
let serial_irq =
|
|
|
|
if self.config.lock().unwrap().serial.clone().mode != ConsoleOutputMode::Off {
|
|
|
|
self.get_device_info()
|
|
|
|
.clone()
|
|
|
|
.get(&(DeviceType::Serial, DeviceType::Serial.to_string()))
|
|
|
|
.unwrap()
|
|
|
|
.irq()
|
|
|
|
} else {
|
|
|
|
// If serial is turned off, add a fake device with invalid irq.
|
|
|
|
31
|
|
|
|
};
|
2021-11-04 17:12:59 +00:00
|
|
|
if self.config.lock().unwrap().serial.mode != ConsoleOutputMode::Off {
|
|
|
|
aml::Device::new(
|
|
|
|
"_SB_.COM1".into(),
|
|
|
|
vec![
|
|
|
|
&aml::Name::new(
|
|
|
|
"_HID".into(),
|
2021-02-02 23:27:56 +00:00
|
|
|
#[cfg(target_arch = "x86_64")]
|
2023-03-02 08:01:33 +00:00
|
|
|
&aml::EISAName::new("PNP0501"),
|
2021-02-02 23:27:56 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
2021-11-04 17:12:59 +00:00
|
|
|
&"ARMH0011",
|
|
|
|
),
|
|
|
|
&aml::Name::new("_UID".into(), &aml::ZERO),
|
2022-01-16 17:53:02 +00:00
|
|
|
&aml::Name::new("_DDN".into(), &"COM1"),
|
2021-11-04 17:12:59 +00:00
|
|
|
&aml::Name::new(
|
|
|
|
"_CRS".into(),
|
|
|
|
&aml::ResourceTemplate::new(vec![
|
|
|
|
&aml::Interrupt::new(true, true, false, false, serial_irq),
|
|
|
|
#[cfg(target_arch = "x86_64")]
|
2023-03-02 08:01:33 +00:00
|
|
|
&aml::IO::new(0x3f8, 0x3f8, 0, 0x8),
|
2021-11-04 17:12:59 +00:00
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
&aml::Memory32Fixed::new(
|
|
|
|
true,
|
2022-04-05 09:05:24 +00:00
|
|
|
arch::layout::LEGACY_SERIAL_MAPPED_IO_START.raw_value() as u32,
|
2021-11-04 17:12:59 +00:00
|
|
|
MMIO_LEN as u32,
|
|
|
|
),
|
|
|
|
]),
|
|
|
|
),
|
|
|
|
],
|
|
|
|
)
|
2023-03-02 08:01:33 +00:00
|
|
|
.to_aml_bytes(sink);
|
2021-11-04 17:12:59 +00:00
|
|
|
}
|
2019-12-06 16:14:32 +00:00
|
|
|
|
2023-03-02 08:01:33 +00:00
|
|
|
aml::Name::new("_S5_".into(), &aml::Package::new(vec![&5u8])).to_aml_bytes(sink);
|
2019-12-06 16:14:32 +00:00
|
|
|
|
2021-11-04 17:12:59 +00:00
|
|
|
aml::Device::new(
|
2021-01-13 10:01:35 +00:00
|
|
|
"_SB_.PWRB".into(),
|
|
|
|
vec![
|
2023-03-02 08:01:33 +00:00
|
|
|
&aml::Name::new("_HID".into(), &aml::EISAName::new("PNP0C0C")),
|
2021-01-13 10:01:35 +00:00
|
|
|
&aml::Name::new("_UID".into(), &aml::ZERO),
|
|
|
|
],
|
|
|
|
)
|
2023-03-02 08:01:33 +00:00
|
|
|
.to_aml_bytes(sink);
|
2021-01-13 10:01:35 +00:00
|
|
|
|
2022-08-12 20:28:04 +00:00
|
|
|
if self.config.lock().unwrap().tpm.is_some() {
|
|
|
|
// Add tpm device
|
2023-03-02 08:01:33 +00:00
|
|
|
TpmDevice {}.to_aml_bytes(sink);
|
2022-08-12 20:28:04 +00:00
|
|
|
}
|
|
|
|
|
2021-11-04 17:12:59 +00:00
|
|
|
self.ged_notification_device
|
2020-02-25 07:20:59 +00:00
|
|
|
.as_ref()
|
|
|
|
.unwrap()
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
2023-03-02 08:01:33 +00:00
|
|
|
.to_aml_bytes(sink)
|
2019-12-06 16:14:32 +00:00
|
|
|
}
|
|
|
|
}
|
2019-11-18 23:24:31 +00:00
|
|
|
|
|
|
|
impl Pausable for DeviceManager {
|
|
|
|
fn pause(&mut self) -> result::Result<(), MigratableError> {
|
2020-05-12 13:53:09 +00:00
|
|
|
for (_, device_node) in self.device_tree.lock().unwrap().iter() {
|
2020-04-30 18:27:19 +00:00
|
|
|
if let Some(migratable) = &device_node.migratable {
|
|
|
|
migratable.lock().unwrap().pause()?;
|
|
|
|
}
|
2019-11-18 23:24:31 +00:00
|
|
|
}
|
2021-09-01 02:31:15 +00:00
|
|
|
// On AArch64, the pause of device manager needs to trigger
|
|
|
|
// a "pause" of GIC, which will flush the GIC pending tables
|
|
|
|
// and ITS tables to guest RAM.
|
|
|
|
#[cfg(target_arch = "aarch64")]
|
|
|
|
{
|
2022-05-31 11:53:22 +00:00
|
|
|
self.get_interrupt_controller()
|
|
|
|
.unwrap()
|
|
|
|
.lock()
|
|
|
|
.unwrap()
|
|
|
|
.pause()?;
|
2021-09-02 08:29:29 +00:00
|
|
|
};
|
2019-11-18 23:24:31 +00:00
|
|
|
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
|
|
|
fn resume(&mut self) -> result::Result<(), MigratableError> {
|
2020-05-12 13:53:09 +00:00
|
|
|
for (_, device_node) in self.device_tree.lock().unwrap().iter() {
|
2020-04-30 18:27:19 +00:00
|
|
|
if let Some(migratable) = &device_node.migratable {
|
|
|
|
migratable.lock().unwrap().resume()?;
|
|
|
|
}
|
2019-11-18 23:24:31 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-12-02 21:50:38 +00:00
|
|
|
impl Snapshottable for DeviceManager {
|
|
|
|
fn id(&self) -> String {
|
|
|
|
DEVICE_MANAGER_SNAPSHOT_ID.to_string()
|
|
|
|
}
|
|
|
|
|
2020-08-21 12:31:58 +00:00
|
|
|
fn snapshot(&mut self) -> std::result::Result<Snapshot, MigratableError> {
|
2022-12-02 15:03:50 +00:00
|
|
|
let mut snapshot = Snapshot::from_data(SnapshotData::new_from_state(&self.state())?);
|
2019-12-02 21:50:38 +00:00
|
|
|
|
2020-04-28 10:08:51 +00:00
|
|
|
// We aggregate all devices snapshots.
|
2020-05-12 13:53:09 +00:00
|
|
|
for (_, device_node) in self.device_tree.lock().unwrap().iter() {
|
2020-04-30 18:19:02 +00:00
|
|
|
if let Some(migratable) = &device_node.migratable {
|
2022-12-02 14:31:53 +00:00
|
|
|
let mut migratable = migratable.lock().unwrap();
|
|
|
|
snapshot.add_snapshot(migratable.id(), migratable.snapshot()?);
|
2020-04-30 18:19:02 +00:00
|
|
|
}
|
2019-12-02 21:50:38 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
Ok(snapshot)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-05-01 16:59:51 +00:00
|
|
|
impl Transportable for DeviceManager {}
|
2021-08-04 14:38:01 +00:00
|
|
|
|
|
|
|
impl Migratable for DeviceManager {
|
|
|
|
fn start_dirty_log(&mut self) -> std::result::Result<(), MigratableError> {
|
|
|
|
for (_, device_node) in self.device_tree.lock().unwrap().iter() {
|
|
|
|
if let Some(migratable) = &device_node.migratable {
|
|
|
|
migratable.lock().unwrap().start_dirty_log()?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
|
|
|
fn stop_dirty_log(&mut self) -> std::result::Result<(), MigratableError> {
|
|
|
|
for (_, device_node) in self.device_tree.lock().unwrap().iter() {
|
|
|
|
if let Some(migratable) = &device_node.migratable {
|
|
|
|
migratable.lock().unwrap().stop_dirty_log()?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
|
|
|
fn dirty_log(&mut self) -> std::result::Result<MemoryRangeTable, MigratableError> {
|
|
|
|
let mut tables = Vec::new();
|
|
|
|
for (_, device_node) in self.device_tree.lock().unwrap().iter() {
|
|
|
|
if let Some(migratable) = &device_node.migratable {
|
|
|
|
tables.push(migratable.lock().unwrap().dirty_log()?);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
Ok(MemoryRangeTable::new_from_tables(tables))
|
|
|
|
}
|
2021-08-09 08:46:41 +00:00
|
|
|
|
2022-01-29 02:53:58 +00:00
|
|
|
fn start_migration(&mut self) -> std::result::Result<(), MigratableError> {
|
|
|
|
for (_, device_node) in self.device_tree.lock().unwrap().iter() {
|
|
|
|
if let Some(migratable) = &device_node.migratable {
|
|
|
|
migratable.lock().unwrap().start_migration()?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
Ok(())
|
|
|
|
}
|
|
|
|
|
2021-08-09 08:46:41 +00:00
|
|
|
fn complete_migration(&mut self) -> std::result::Result<(), MigratableError> {
|
|
|
|
for (_, device_node) in self.device_tree.lock().unwrap().iter() {
|
|
|
|
if let Some(migratable) = &device_node.migratable {
|
|
|
|
migratable.lock().unwrap().complete_migration()?;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
Ok(())
|
|
|
|
}
|
2021-08-04 14:38:01 +00:00
|
|
|
}
|
2020-02-14 10:08:14 +00:00
|
|
|
|
2020-02-28 11:29:43 +00:00
|
|
|
const PCIU_FIELD_OFFSET: u64 = 0;
|
|
|
|
const PCID_FIELD_OFFSET: u64 = 4;
|
2020-03-06 15:53:20 +00:00
|
|
|
const B0EJ_FIELD_OFFSET: u64 = 8;
|
2021-10-08 09:49:26 +00:00
|
|
|
const PSEG_FIELD_OFFSET: u64 = 12;
|
2020-02-28 11:29:43 +00:00
|
|
|
const PCIU_FIELD_SIZE: usize = 4;
|
|
|
|
const PCID_FIELD_SIZE: usize = 4;
|
2020-03-06 15:53:20 +00:00
|
|
|
const B0EJ_FIELD_SIZE: usize = 4;
|
2021-10-08 09:49:26 +00:00
|
|
|
const PSEG_FIELD_SIZE: usize = 4;
|
2020-02-28 11:29:43 +00:00
|
|
|
|
|
|
|
impl BusDevice for DeviceManager {
|
|
|
|
fn read(&mut self, base: u64, offset: u64, data: &mut [u8]) {
|
|
|
|
match offset {
|
|
|
|
PCIU_FIELD_OFFSET => {
|
|
|
|
assert!(data.len() == PCIU_FIELD_SIZE);
|
2021-10-08 09:49:26 +00:00
|
|
|
data.copy_from_slice(
|
|
|
|
&self.pci_segments[self.selected_segment]
|
|
|
|
.pci_devices_up
|
|
|
|
.to_le_bytes(),
|
|
|
|
);
|
2020-02-28 11:29:43 +00:00
|
|
|
// Clear the PCIU bitmap
|
2021-10-08 09:49:26 +00:00
|
|
|
self.pci_segments[self.selected_segment].pci_devices_up = 0;
|
2020-02-28 11:29:43 +00:00
|
|
|
}
|
|
|
|
PCID_FIELD_OFFSET => {
|
|
|
|
assert!(data.len() == PCID_FIELD_SIZE);
|
2021-10-08 09:49:26 +00:00
|
|
|
data.copy_from_slice(
|
|
|
|
&self.pci_segments[self.selected_segment]
|
|
|
|
.pci_devices_down
|
|
|
|
.to_le_bytes(),
|
|
|
|
);
|
2020-02-28 11:29:43 +00:00
|
|
|
// Clear the PCID bitmap
|
2021-10-08 09:49:26 +00:00
|
|
|
self.pci_segments[self.selected_segment].pci_devices_down = 0;
|
2020-02-28 11:29:43 +00:00
|
|
|
}
|
2021-04-21 07:45:26 +00:00
|
|
|
B0EJ_FIELD_OFFSET => {
|
|
|
|
assert!(data.len() == B0EJ_FIELD_SIZE);
|
|
|
|
// Always return an empty bitmap since the eject is always
|
|
|
|
// taken care of right away during a write access.
|
2021-09-27 13:52:10 +00:00
|
|
|
data.fill(0);
|
2021-04-21 07:45:26 +00:00
|
|
|
}
|
2021-10-08 09:49:26 +00:00
|
|
|
PSEG_FIELD_OFFSET => {
|
|
|
|
assert_eq!(data.len(), PSEG_FIELD_SIZE);
|
|
|
|
data.copy_from_slice(&(self.selected_segment as u32).to_le_bytes());
|
|
|
|
}
|
2020-03-06 15:53:20 +00:00
|
|
|
_ => error!(
|
|
|
|
"Accessing unknown location at base 0x{:x}, offset 0x{:x}",
|
|
|
|
base, offset
|
|
|
|
),
|
2020-02-28 11:29:43 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
debug!(
|
|
|
|
"PCI_HP_REG_R: base 0x{:x}, offset 0x{:x}, data {:?}",
|
|
|
|
base, offset, data
|
|
|
|
)
|
|
|
|
}
|
|
|
|
|
2021-10-08 09:49:26 +00:00
|
|
|
fn write(&mut self, base: u64, offset: u64, data: &[u8]) -> Option<Arc<std::sync::Barrier>> {
|
2020-03-06 15:53:20 +00:00
|
|
|
match offset {
|
|
|
|
B0EJ_FIELD_OFFSET => {
|
|
|
|
assert!(data.len() == B0EJ_FIELD_SIZE);
|
|
|
|
let mut data_array: [u8; 4] = [0, 0, 0, 0];
|
2021-06-23 20:42:17 +00:00
|
|
|
data_array.copy_from_slice(data);
|
2021-09-27 13:52:10 +00:00
|
|
|
let mut slot_bitmap = u32::from_le_bytes(data_array);
|
2020-03-06 15:53:20 +00:00
|
|
|
|
2021-09-27 13:52:10 +00:00
|
|
|
while slot_bitmap > 0 {
|
|
|
|
let slot_id = slot_bitmap.trailing_zeros();
|
2021-10-08 09:49:26 +00:00
|
|
|
if let Err(e) = self.eject_device(self.selected_segment as u16, slot_id as u8) {
|
2021-09-27 13:52:10 +00:00
|
|
|
error!("Failed ejecting device {}: {:?}", slot_id, e);
|
2020-03-06 15:53:20 +00:00
|
|
|
}
|
2021-09-27 13:52:10 +00:00
|
|
|
slot_bitmap &= !(1 << slot_id);
|
2020-03-06 15:53:20 +00:00
|
|
|
}
|
|
|
|
}
|
2021-10-08 09:49:26 +00:00
|
|
|
PSEG_FIELD_OFFSET => {
|
|
|
|
assert_eq!(data.len(), PSEG_FIELD_SIZE);
|
|
|
|
let mut data_array: [u8; 4] = [0, 0, 0, 0];
|
|
|
|
data_array.copy_from_slice(data);
|
|
|
|
let selected_segment = u32::from_le_bytes(data_array) as usize;
|
|
|
|
if selected_segment >= self.pci_segments.len() {
|
|
|
|
error!(
|
|
|
|
"Segment selection out of range: {} >= {}",
|
|
|
|
selected_segment,
|
|
|
|
self.pci_segments.len()
|
|
|
|
);
|
|
|
|
return None;
|
|
|
|
}
|
|
|
|
self.selected_segment = selected_segment;
|
|
|
|
}
|
2020-03-06 15:53:20 +00:00
|
|
|
_ => error!(
|
|
|
|
"Accessing unknown location at base 0x{:x}, offset 0x{:x}",
|
|
|
|
base, offset
|
|
|
|
),
|
|
|
|
}
|
|
|
|
|
2020-02-28 11:29:43 +00:00
|
|
|
debug!(
|
|
|
|
"PCI_HP_REG_W: base 0x{:x}, offset 0x{:x}, data {:?}",
|
|
|
|
base, offset, data
|
2020-12-04 09:23:47 +00:00
|
|
|
);
|
|
|
|
|
|
|
|
None
|
2020-02-28 11:29:43 +00:00
|
|
|
}
|
|
|
|
}
|
2020-02-27 10:40:05 +00:00
|
|
|
|
2020-02-14 10:08:14 +00:00
|
|
|
impl Drop for DeviceManager {
|
|
|
|
fn drop(&mut self) {
|
2022-03-10 13:05:36 +00:00
|
|
|
for handle in self.virtio_devices.drain(..) {
|
|
|
|
handle.virtio_device.lock().unwrap().shutdown();
|
2020-02-14 10:08:14 +00:00
|
|
|
}
|
vmm: only touch the tty flags if it's being used
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-03-22 22:22:46 +00:00
|
|
|
|
vmm: reset to the original termios
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-04-27 12:57:28 +00:00
|
|
|
if let Some(termios) = *self.original_termios_opt.lock().unwrap() {
|
|
|
|
// SAFETY: FFI call
|
|
|
|
let _ = unsafe { tcsetattr(stdout().lock().as_raw_fd(), TCSANOW, &termios) };
|
vmm: only touch the tty flags if it's being used
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
2023-03-22 22:22:46 +00:00
|
|
|
}
|
2020-02-14 10:08:14 +00:00
|
|
|
}
|
|
|
|
}
|