1
0
mirror of https://gitlab.com/libvirt/libvirt.git synced 2025-03-07 17:28:15 +00:00
Jon Kohler 1cc7737f69 qemu: add support for qemu switchover-ack
Add plumbing for QEMU's switchover-ack migration capability, which
helps lower the downtime during VFIO migrations. This capability is
enabled by default as long as both the source and destination support
it.

Note: switchover-ack depends on the return path capability, so this may
not be used when VIR_MIGRATE_TUNNELLED flag is set.

Extensive details about the qemu switchover-ack implementation are
available in the qemu series v6 cover letter [1] where the highlight is
the extreme reduction in guest visible downtime. In addition to the
original test results below, I saw a roughly ~20% reduction in downtime
for VFIO VGPU devices at minimum.

  === Test results ===

  The below table shows the downtime of two identical migrations. In the
  first migration swithcover ack is disabled and in the second it is
  enabled. The migrated VM is assigned with a mlx5 VFIO device which has
  300MB of device data to be migrated.

  +----------------------+-----------------------+----------+
  |    Switchover ack    | VFIO device data size | Downtime |
  +----------------------+-----------------------+----------+
  |       Disabled       |         300MB         |  1900ms  |
  |       Enabled        |         300MB         |  420ms   |
  +----------------------+-----------------------+----------+

  Switchover ack gives a roughly 4.5 times improvement in downtime.
  The 1480ms difference is time that is used for resource allocation for
  the VFIO device in the destination. Without switchover ack, this time is
  spent when the source VM is stopped and thus the downtime is much
  higher. With switchover ack, the time is spent when the source VM is
  still running.

[1] https://patchwork.kernel.org/project/qemu-devel/cover/20230621111201.29729-1-avihaih@nvidia.com/

Signed-off-by: Jon Kohler <jon@nutanix.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Avihai Horon <avihaih@nvidia.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: YangHang Liu <yanghliu@redhat.com>
Reviewed-by: Jiri Denemark <jdenemar@redhat.com>
2024-06-25 09:51:00 +02:00
..
2024-02-26 11:03:51 +01:00
2024-06-19 12:15:26 +02:00
2024-06-21 09:56:57 +02:00
2024-06-19 14:28:38 +02:00

       libvirt library code README
       ===========================

The directory provides the bulk of the libvirt codebase. Everything
except for the libvirtd daemon and client tools. The build uses a
large number of libtool convenience libraries - one for each child
directory, and then links them together for the final libvirt.so,
although some bits get linked directly to libvirtd daemon instead.

The files directly in this directory are supporting the public API
entry points & data structures.

There are two core shared modules to be aware of:

 * util/  - a collection of shared APIs that can be used by any
            code. This directory is always in the include path
            for all things built

 * conf/  - APIs for parsing / manipulating all the official XML
            files used by the public API. This directory is only
            in the include path for driver implementation modules

 * vmx/   - VMware VMX config handling (used by esx/ and vmware/)


Then there are the hypervisor implementations:

 * bhyve         - bhyve - The BSD Hypervisor
 * esx/          - VMware ESX and GSX support using vSphere API over SOAP
 * hyperv/       - Microsoft Hyper-V support using WinRM
 * lxc/          - Linux Native Containers
 * openvz/       - OpenVZ containers using cli tools
 * qemu/         - QEMU / KVM using qemu CLI/monitor
 * remote/       - Generic libvirt native RPC client
 * test/         - A "mock" driver for testing
 * vbox/         - Virtual Box using native API
 * vmware/       - VMware Workstation and Player using the vmrun tool
 * xen/          - Xen using hypercalls, XenD SEXPR & XenStore


Finally some secondary drivers that are shared for several HVs.
Currently these are used by LXC, OpenVZ, QEMU and Xen drivers.
The ESX, Hyper-V, Remote, Test & VirtualBox drivers all
implement the secondary drivers directly

 * cpu/          - CPU feature management
 * interface/    - Host network interface management
 * network/      - Virtual NAT networking
 * nwfilter/     - Network traffic filtering rules
 * node_device/  - Host device enumeration
 * secret/       - Secret management
 * security/     - Mandatory access control drivers
 * storage/      - Storage management drivers


Since both the hypervisor and secondary drivers can be built as
dlopen()able modules, it is *FORBIDDEN* to have build dependencies
between these directories. Drivers are only allowed to depend on
the public API, and the internal APIs in the util/ and conf/
directories