Update to the latest vm-memory and all the crates that also depend upon
it.
Fix some deprecation warnings.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This feature flag gates the development for SEV-SNP enabled guest.
Also add a helper function to identify if SNP should be enabled for the
guest.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This commit builds on top of the `Monitor::subscribe` function and
makes it possible to broadcast events published from `event-monitor`
over D-Bus.
The broadcasting functionality is enabled if the D-Bus API is enabled
and users who wish to also enable the file based `event-monitor` can do
so with the CLI arg `--event-monitor`.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
warning: this argument is a mutable reference, but not used mutably
--> vmm/src/sigwinch_listener.rs:121:38
|
121 | fn set_foreground_process_group(tty: &mut File) -> io::Result<()> {
| ^^^^^^^^^ help: consider changing to: `&File`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
warning: this argument is a mutable reference, but not used mutably
--> vmm/src/device_manager.rs:1908:35
|
1908 | fn set_raw_mode(&mut self, f: &mut dyn AsRawFd) -> vmm_sys_util::errno::Result<()> {
| ^^^^^^^^^^^^^^^^ help: consider changing to: `&dyn AsRawFd`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_pass_by_ref_mut
= note: `#[warn(clippy::needless_pass_by_ref_mut)]` on by default
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Add pending removed vcpu check according to VcpuState.removing, which
can avoid cloud hypervisor hangup during continual vcpu resize.
Fix#5419
Signed-off-by: Yi Wang <foxywang@tencent.com>
This patch modifies `event_monitor` to ensure that concurrent access to
`event_log` from multiple threads is safe. Previously, the `event_log`
function would acquire a reference to the event log file and write
to it without doing any synchronization, which made it prone to
data races. This issue likely went under the radar because the
relevant `SAFETY` comment on the unsafe block was incomplete.
The new implementation spawns a dedicated thread named `event-monitor`
solely for writing to the file. It uses the MPMC channel exposed by
`flume` to pass messages to the `event-monitor` thread. Since
`flume::Sender<T>` implements `Sync`, it is safe for multiple threads
to share it and send messages to the `event-monitor` thread.
This is not possible with `std::sync::mpsc::Sender<T>` since it's
`!Sync`, meaning it is not safe for it to be shared between different
threads.
The `event_monitor::set_monitor` function now only initializes
the required global state and returns an instance of the
`Monitor` struct. This decouples the actual logging logic from the
`event_monitor` crate. The `event-monitor` thread is then spawned by
the `vmm` crate.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
With the addition of the spinning waiting for the exit event to be
received in the CMOS device a regression was introduced into the CMOS
fuzzer. Since there is nothing to receive the event in the fuzzer and
there is nothing to update the bit the that the device is looping on;
introducing an infinite loop.
Use an Option<> type so that when running the device in the fuzzer no
Arc<AtomicBool> is provided effectively disabling the spinning logic.
Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61165
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The reset system is asynchronous with an I/O event (PIO or MMIO) for
ACPI/i8042/CMOS triggering a write to the reset_evt event handler. The
VMM thread will pick up this event on the VMM main loop and then trigger
a shutdown in the CpuManager. However since there is some delay between
the CPU threads being marked to be killed (through the
CpuManager::cpus_kill_signalled bool) it is possible for the guest vCPU
that triggered the exit to be re-entered when the vCPU KVM_RUN is called
after the I/O exit is completed.
This is undesirable and in particular the Linux kernel will attempt to
jump to real mode after a CMOS based exit - this is unsupported in
nested KVM on AMD on Azure and will trigger an error in KVM_RUN.
Solve this problem by spinning in the device that has triggered the
reset until the vcpus_kill_signalled boolean has been updated
indicating that the VMM thread has received the event and called
CpuManager::shutdown(). In particular if this bool is set then the vCPU
threads will not re-enter the guest.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Split interrupt source group restore into two steps, first restore
the irqfd for each interrupt source entry, and second restore the
GSI routing of the entire interrupt source group.
This patch will reduce restore latency of interrupt source group,
and in a 200-concurrent restore test, the patch reduced the
average IOAPIC restore time from 15ms to 1ms.
Signed-off-by: Yong He <alexyonghe@tencent.com>
If the VMM is not already paused then pause the VM prior to executing
the coredump and then resume it after. If the VM is already paused then
the original state is maintained.
Signed-off-by: Yi Wang <foxywang@tencent.com>
Add MSHV_CREATE_DEVICE, MSHV_SET_DEVICE_ATTR ioctls to filters. These
ioctls are required to passthrough PCI devices on mshv.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
The pause of vcpu is async now, which makes the vm pause is not
synchronised. As virtio device calls paused_sync wait() to make
sure device_manager pause synchronously, if we make vcpu pause
synchronously, the vm pause can be synchronously then. After
vm.pause() returns the vm is really paused now.
This patch adds a AtomicBool variable to mark vcpu paused state,
to make sure the pause of CpuManager is synchronised.
Signed-off-by: Yi Wang <foxywang@tencent.com>
This commit merges crates `qcow`, `vhdx` and `block_util` into the
crate `block`, which can allow `qcow` to use functions from `block_util`
without introducing a circular crate dependency.
This commit is based on crosvm implementation:
f2eecc4152
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Using this over the LocalApic supports APIC IDs (and hence number of
vCPUs) above 256 (size increases from u8 to u32.)
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
If the VM has been configured but not yet booted, all we need to do to
support removing a device is to remove it from the config, so it will
never be created.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This fixes the valid VM config unit tests, which would otherwise fail
to deserialize their expected JSON config due to the missing "gdb" field.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
warning: usage of `Arc<T>` where `T` is not `Send` or `Sync`
--> virtio-devices/src/vsock/device.rs:376:22
|
376 | backend: Arc::new(RwLock::new(backend)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= help: consider using `Rc<T>` instead or wrapping `T` in a std::sync type like `Mutex<T>`
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#arc_with_non_send_sync
= note: `#[warn(clippy::arc_with_non_send_sync)]` on by default
The vsock backend may be shared between threads, so the type `B` in
`Vsock` should be `VsockBackend` and `Sync`.
Considering that `api_receiver` and `gdb_receiver` are only used in vmm
threads, the `Arc` can be replaced by `Rc`.
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
warning: useless use of `vec!`
--> test_infra/src/lib.rs:111:30
|
111 | let mut events = vec![epoll::Event::new(epoll::Events::empty(), 0); 1];
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: you can use an array directly: `[epoll::Event::new(epoll::Events::empty(), 0); 1]`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_vec
= note: `#[warn(clippy::useless_vec)]` on by default
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
This gives users the chance to reduce the number of dependencies
included, which is generally good practice and also reduces code size.
Furthermore, `io_uring` specifically is a strong contender for something
one may wish to disable due to the syscall API's many security issues[1]
[1]: https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html
Signed-off-by: Manish Goregaokar <manishsmail@gmail.com>
This does the same thing as df2a7c17 ("vmm: Ignore and warn TAP FDs
sent via the HTTP request body"), but for the vm.create endpoint,
which also previously would accept file descriptors in the body, and
try to use whatever fd occupied that number as a TAP device.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Port of df2a7c17 ("vmm: Ignore and warn TAP FDs sent via the HTTP
request body"), but for the vm.create endpoint, which would previously
accept file descriptors in the body, and try to use whatever fd
occupied that number as a TAP device.
Since I had to move the wrapping of the net config in an Arc until
after it was modified, I made the same change to all other endpoints,
so the style stays consistent.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This commit renames `ram_region_sub_size` to `ram_region_available_size`
and make its value align down to the default page size or hugepage
size of the current memory zone, which can prevent the memory zone from
being split into misaligned parts. And if the available size of ram
region is zero, this region will be marked as consumed even it has
unused space.
Note that there is two methods to use hugepages.
1. Specify `hugepages` for `memory` or `memory-zone`, if the
`hugepage_size` is not specified, the value can be got by `statfs`
for `/dev/hugepages`.
2. Specify a `file` in hugetlbfs for `memory-zone`, the hugepage size
can also be got by `statfs` for the file.
The value for alignment will be the hugepage size if this memory zone
is using hugepages, otherwise the value will be default page size of
system.
Fixes: #5463
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
The previous `arch_memory_regions` function will provide some memory
regions with the specified memory size and fill all the previous
regions before using the next one, but sometimes there may be no need
to fill up the previous one, e.g., the previous one should be aligned
with hugepage size.
This commit make `arch_memory_regions` function not take any
parameters and return the max available regions, the memory manager
can use them on demand.
Fixes: #5463
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
SerialBuffer uses VecDeque::extend, which calls realloc, which a
maximum buffer size of 1 MiB. Starting at allocation sizes of
128 KiB, musl's mallocng allocator will use mremap for the allocation.
Since this was not permitted by the seccomp rules, heavy write load
could crash cloud-hypervisor with a seccomp failure. (Encountered
using virtio-console, but I don't see any reason it wouldn't happen
for the legacy serial device too.)
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Bump to the latest rust-vmm crates, including vm-memory, vfio,
vfio-bindings, vfio-user, virtio-bindings, virtio-queue, linux-loader,
vhost, and vhost-user-backend,
Signed-off-by: Bo Chen <chen.bo@intel.com>
Introduces three new CLI options, `dbus-service-name`,
`dbus-object-path` and `dbus-system-bus` to configure the DBus API.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit applies the previously created seccomp filter
to the `DbusApi` thread.
Also encloses the main loop of the `DBusApi` thread using
`std::panic::catch_unwind` and `AssertUnwindSafe` in order to mirror
the behavior of the HTTP API.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit adds support for graceful shutdown of the DBusApi thread
using `futures::channel::oneshot` channels. By using oneshot channels,
we ensure that the thread has enough time to send a response to the
`VmmShutdown` method call before it is terminated. Without this step,
the thread may be terminated before it can send a response, resulting
in an error message on the client side stating that the message
recipient disconnected from the message bus without providing a reply.
Also changes the default values for DBus service name, object path
and interface name.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
This commit introduces three new dependencies: `zbus`, `futures`
and `blocking`. `blocking` is used to call the Internal API in zbus'
async context which is driven by `futures::executor`. They are all
behind the `dbus_api` feature flag.
The D-Bus API implementation is behind the same `dbus_api` feature
flag as well.
Signed-off-by: Omer Faruk Bayram <omer.faruk@sartura.hr>
The refactoring on deferring address space allocation (#5169) broke TDX,
as TDX initialization needs to access guest memory for encryption and
measurement of guest pages.
Signed-off-by: Bo Chen <chen.bo@intel.com>
The current implementation of breadth first traversal for device tree
uses a temporary vector, therefore causes unnecessary memory copy.
Remove it and do it within vector nodes.
Signed-off-by: Hao Xu <howeyxu@tencent.com>
Unlike KVM, there's no internal handling for topoolgy under MSHV. Thus,
if no topology has been passed during the CH launch, use the boot CPUs
count to construct the topology struct.
Signed-off-by: Anatol Belski <anbelski@linux.microsoft.com>
Originally the AML only accepted one hex number for PCI segment
numbering. Change it to accept two numbers. That makes it possible to
add up to 256 PCI segments.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
When I refactored this to centralise resetting the tty into
DeviceManager::drop, I tested that the tty was reset if an error
happened on the vmm thread, but not on the main thread. It turns out
that if an error happened on the main thread, the process would just
exit, so drop handlers on other threads wouldn't get run.
To fix this, I've changed start_vmm() to write to the VMM's exit
eventfd and then join the thread if an error happens after the vmm
thread is started.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Previously, we used two different functions for configuring ttys.
vmm_sys_util::terminal::Terminal::set_raw_mode() was used to configure
stdio ttys, and cfmakeraw() was used to configure ptys created by
cloud-hypervisor. When I centralized the stdio tty cleanup, I also
switched to using cfmakeraw() everywhere, to avoid duplication.
cfmakeraw sets the OPOST flag, but when we later reset the ttys, we
used vmm_sys_util::terminal::Terminal::set_canon_mode(), which does
not unset this flag. This meant that the terminal was getting mostly,
but not fully, reset.
To fix this without depending on the implementation of cfmakeraw(),
let's just store the original termios for stdio terminals, and restore
them to exactly the state we found them in when cloud-hypervisor exits.
Fixes: b6feae0a ("vmm: only touch the tty flags if it's being used")
Signed-off-by: Alyssa Ross <hi@alyssa.is>
In particular the Std::write() API requires that the value implements
AsBytes and copies the slice representation into the table data. This
avoids unaligned writes which can cause a panic with the updated
toolchain.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
For structures that are used in SDT ACPI tables it is necessary for them
to implement this trait for the newly safe Std::write() API.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
This is used on older kernels where close_range() is not available.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Fixes: 505f4dfa ("vmm: close all unused fds in sigwinch listener")
On KVM this is provided by an ioctl, on MSHV this is constant. Although
there is a HV_MAXIMUM_PROCESSORS constant the MSHV ioctl API is limited
to u8.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
The custom 'clone' duplicates 'preserved_fds' so that the validation
logic can be safely carried out on the clone of the VmConfig.
The custom 'drop' ensures 'preserved_fds' are safely closed when the
holding VmConfig instance is destroyed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
Preserved FDs are the ones that share the same life-time as its holding
VmConfig instance, such as FDs for creating TAP devices.
Preserved FDs will stay open as long as the holding VmConfig instance is
valid, and will be closed when the holding VmConfig instance is destroyed.
Signed-off-by: Bo Chen <chen.bo@intel.com>
When neither serial nor console are connected to the tty,
cloud-hypervisor shouldn't touch the tty at all. One way in which
this is annoying is that if I am running cloud-hypervisor without it
using my terminal, I expect to be able to suspend it with ^Z like any
other process, but that doesn't work if it's put the terminal into raw
mode.
Instead of putting the tty into raw mode when a VM is created or
restored, do it when a serial or console device is created. Since we
now know it can't be put into raw mode until the Vm object is created,
we can move setting it back to canon mode into the drop handler for
that object, which should always be run in normal operation. We still
also put the tty into canon mode in the SIGTERM / SIGINT handler, but
check whether the tty was actually used, rather than whether stdin is
a tty. This requires passing on_tty around as an atomic boolean.
I explored more of an abstraction over the tty — having an object that
encapsulated stdout and put the tty into raw mode when initialized and
into canon mode when dropped — but it wasn't practical, mostly due to
the special requirements of the signal handler. I also investigated
whether the SIGWINCH listener process could be used here, which I
think would have worked but I'm hesitant to involve it in serial
handling as well as conosle handling.
There's no longer a check for whether the file descriptor is a tty
before setting it into canon mode — it's redundant, because if it's
not a tty it just won't respond to the ioctl.
Tested by shutting down through the API, SIGTERM, and an error
injected after setting raw mode.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
If the VM is shut down, either it's going to be started again, in
which case we still want to be in raw mode, or the process is about to
exit, in which case canon mode will be set at the end of main.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Having PMU in guests isn't critical, and not all hardware supports
it (e.g. Apple Silicon).
CpuManager::init_pmu already has a fallback for if PMU is not
supported by the VCPU, but we weren't getting that far, because we
would always try to initialise the VCPU with KVM_ARM_VCPU_PMU_V3, and
then bail when it returned with EINVAL.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Previously, we were only using it for PTYs, because for PTYs there's
no alternative. But since we have to have it for PTYs anyway, if we
also use it for TTYs, we can eliminate all of the code that handled
SIGWINCH for TTYs.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Now that the SIGWINCH listener has fallbacks for older kernels, we
don't expect it to routinely fail, so if there's an error setting it
up, we want to know about it.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
This will allow the SIGWINCH listener to run on kernels older than
5.5, although on those kernels it will have to make 64 syscalls to
reset all the signal handlers.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
The PTY main file descriptor had to be introduced as a parameter to
start_sigwinch_listener, so that it could be closed in the child.
Really the SIGWINCH listener process should not have any file
descriptors open, except for the ones it needs to function, so let's
make it more robust by having it close all other file descriptors.
For recent kernels, we can do this very conveniently with
close_range(2), but for older kernels, we have to fall back to closing
open file descriptors one at a time.
Signed-off-by: Alyssa Ross <hi@alyssa.is>
Significant API changes have occured, most significantly is the switch
to an approach which does not require vm-memory and can run no_std.
Signed-off-by: Rob Bradford <rbradford@rivosinc.com>
Now cloud hypervisor will start signal thread to catch
SIGWINCH signal, cloud hypervisor then will resize the
guest console via vconsole.
This patch skip starting signal thread when there is no
need to resize guest console, such as console is not
configured.
Signed-off-by: Yong He <alexyonghe@tencent.com>
The PR #2333 added I/O rate limiter on block device, with some options
in `DiskConfig`. And the PR #2401 added rate limiter on virtio-net
device with same options, but it still throws `Error::ParseDisk`.
This commit fixes it with correct values.
Fixes: #2401
Signed-off-by: Yu Li <liyu.yukiteru@bytedance.com>
Once error occur, vcpu thread may exit, this should
be critical event for the whole VM, we should fire
exit event and set vcpu state.
If we don't set vcpu state, the shutdown process
will hang at signal_thread, which is waiting the
vcpu state to change.
Signed-off-by: Yong He <alexyonghe@tencent.com>
We need to provide valid FDs while creating 'NetConfig' instances even
for unit tests. Closing invalid FDs would cause random unit test
failures.
Also, two identical 'NetConfig' instances are not allowed any more,
because it would lead to close the same FD twice. This is consistent
with the fact that a clone of a "NetConfig" instance is no
longer *equal* to the instance itself.
Fixes: #5203
Signed-off-by: Bo Chen <chen.bo@intel.com>
These are owned by the config (and are duplicated before being used to
create the `Tap` for the virtio-net device.)
By implementing Drop on NetConfig we have issues with moving out of
members that don't implement the Copy trait. This requires a small
adjustment to the unit tests that use the Default::default() function.
Fixes: #5197
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The custom version duplicates any FDs that have been provided so that
the validation logic used on hotplug, which takes a clone of the config,
can be safely carried out.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This code is indentical to what is in this repository. When a release
gets made we can then switch to that.
Fixes: #5122
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
If swtpm becomes unresponsive, guest gets blocked at "recvmsg" on tpm's
data FD. This change adds a timeout to the data fd socket. If swtpm
becomes unresponsive guest waits for "timeout" (secs) and continues to
run after returning an I/O error to tpm commands.
Signed-off-by: Praveen K Paladugu <prapal@linux.microsoft.com>
We can ideally defer the address space allocation till we start the
vCPUs for the very first time. Because the VM will not access the memory
until the CPUs start running. Thus there is no need to allocate the
address space eagerly and wait till the time we are going to start the
vCPUs for the first time.
Signed-off-by: Jinank Jain <jinankjain@microsoft.com>
This hypervisor leaf includes details of the TSC frequency if that is
available from KVM. This can be used to efficiently calculate time
passed when there is an invariant TSC.
TEST=Run `cpuid` in the guest and observe the frequency populated.
Fixes: #5178
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Change "thead" to "thread".
Also make sure the two messages are distinguishable by adding "vmm" and
"vm" prefix.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
In order to comply with latest TDX version, we rely onto the branch
kvm-upstream-2022.08.07-v5.19-rc8 from https://github.com/intel/tdx
repository. Updates are based on changes that happened in
arch/x86/include/uapi/asm/kvm.h headers file.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
A few breaking changes:
1. `-vvv` needs to be written as `-v -v -v`.
2. `--disk D1 D2` and others need to be written as `--disk D1 --disk D2`.
3. `--option=value` needs to be written as `--option value`
Change integration tests to adapt to the breaking changes.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Add new configuration for offloading features, including
Checksum/TSO/UFO, and set these offloading features as
enabled by default.
Fixes: #4792.
Signed-off-by: Yong He <alexyonghe@tencent.com>
MSHV does not require to ensure MMIO/PIO exits complete
before pausing. This patch makes sure the above requirement
by checking the hypervisor type run-time.
Fixes#5037
Signed-off-by: Muminul Islam <muislam@microsoft.com>
This functionality has been obsoleted by our native support for
hugepages and shared memory.
See: #5082
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
To align the logging messages with the rest of the code, this
message should be aligned with another similar occurrence in
epoll_helper.rs
Signed-off-by: Philipp Schuster <philipp.schuster@cyberus-technology.de>
The double underscore made it different from how other projects would
name this particular macro.
No functional change.
Signed-off-by: Wei Liu <liuwe@microsoft.com>
Remove from the documentation and API definition but continue support
using the field (with a deprecation warning.)
See: #4837
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
This simplifies the Snapshot creation as we expect a SnapshotData to be
provided most of the time.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The information about the identifier related to a Snapshot is only
relevant from the BTreeMap perspective, which is why we can get rid of
the duplicated identifier in every Snapshot structure.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
