mirror of
https://github.com/cloud-hypervisor/cloud-hypervisor.git
synced 2024-12-23 06:05:21 +00:00
140 lines
4.8 KiB
Markdown
140 lines
4.8 KiB
Markdown
# Live Migration
|
|
|
|
This document gives two examples of how to use the live migration
|
|
support in Cloud Hypervisor:
|
|
|
|
1. local migration - migrating between two VMs running on the same
|
|
machine;
|
|
1. nested-vm migration - migrating between two nested VMs whose host VMs
|
|
are running on the same machine.
|
|
|
|
## Local Migration (Suitable for Live Upgrade of VMM)
|
|
Launch the source VM (on the host machine):
|
|
```bash
|
|
$ target/release/cloud-hypervisor
|
|
--kernel ~/workloads/vmlinux \
|
|
--disk path=~/workloads/focal.raw \
|
|
--cpus boot=1 --memory size=1G,shared=on \
|
|
--cmdline "root=/dev/vda1 console=ttyS0" \
|
|
--serial tty --console off --api-socket /tmp/api1
|
|
```
|
|
|
|
Launch the destination VM from the same directory (on the host machine):
|
|
```bash
|
|
$ target/release/cloud-hypervisor --api-socket /tmp/api2
|
|
```
|
|
|
|
Get ready for receiving migration for the destination VM (on the host machine):
|
|
```bash
|
|
$ target/release/ch-remote --api-socket /tmp/api2 receive-migration unix:/tmp/sock
|
|
```
|
|
|
|
Start to send migration for the source VM (on the host machine):
|
|
```bash
|
|
$ target/release/ch-remote --api-socket /tmp/api1 send-migration --local unix:/tmp/sock
|
|
```
|
|
|
|
When the above commands completed, the source VM should be successfully
|
|
migrated to the destination VM. Now the destination VM is running while
|
|
the source VM is terminated gracefully.
|
|
|
|
## Nested-VM Migration
|
|
|
|
Launch VM 1 (on the host machine) with an extra virtio-blk device for
|
|
exposing a guest image for the nested source VM:
|
|
> Note: the example below also attached an additional virtio-blk device
|
|
> with a dummy image for testing purpose (which is optional).
|
|
```bash
|
|
$ head -c 1M < /dev/urandom > tmp.img # create a dummy image for testing
|
|
$ sudo /target/release/cloud-hypervisor \
|
|
--serial tty --console off \
|
|
--cpus boot=1 --memory size=512M \
|
|
--kernel vmlinux \
|
|
--cmdline "root=/dev/vda1 console=ttyS0" \
|
|
--disk path=focal-1.raw path=focal-nested.raw --disk path=tmp.img\
|
|
--net ip=192.168.101.1
|
|
```
|
|
|
|
Launch VM 2 (on the host machine) with an extra virtio-blk device for
|
|
exposing the same guest image for the nested destination VM:
|
|
```bash
|
|
$ sudo /target/release/cloud-hypervisor \
|
|
--serial tty --console off \
|
|
--cpus boot=1 --memory size=512M \
|
|
--kernel vmlinux \
|
|
--cmdline "root=/dev/vda1 console=ttyS0" \
|
|
--disk path=focal-2.raw path=focal-nested.raw --disk path=tmp.img\
|
|
--net ip=192.168.102.1
|
|
```
|
|
|
|
Launch the nested source VM (inside the guest OS of the VM 1) :
|
|
```bash
|
|
vm-1:~$ sudo ./cloud-hypervisor \
|
|
--serial tty --console off \
|
|
--memory size=128M \
|
|
--kernel vmlinux \
|
|
--cmdline "console=ttyS0 root=/dev/vda1" \
|
|
--disk path=/dev/vdb --disk path=/dev/vdc \
|
|
--api-socket /tmp/api1 \
|
|
--net ip=192.168.100.1
|
|
vm-1:~$ # setup the guest network if needed
|
|
vm-1:~$ sudo ip addr add 192.168.101.2/24 dev ens4
|
|
vm-1:~$ sudo ip link set up dev ens4
|
|
vm-1:~$ sudo ip r add default via 192.168.101.1
|
|
```
|
|
Optional: Run the guest workload below (on the guest OS of the nested source VM),
|
|
which performs intensive virtio-blk operations. Now the console of the nested
|
|
source VM should repeatedly print `"equal"`, and our goal is migrating
|
|
this VM and the running workload without interruption.
|
|
```bash
|
|
#/bin/bash
|
|
|
|
# On the guest OS of the nested source VM
|
|
|
|
input="/dev/vdb"
|
|
result=$(md5sum $input)
|
|
tmp=$(md5sum $input)
|
|
|
|
while [[ "$result" == "$tmp" ]]
|
|
do
|
|
echo "equal"
|
|
tmp=$(md5sum $input)
|
|
done
|
|
|
|
echo "not equal"
|
|
echo "result = $result"
|
|
echo "tmp = $tmp"
|
|
```
|
|
|
|
Launch the nested destination VM (inside the guest OS of the VM 2):
|
|
```bash
|
|
vm-2:~$ sudo ./cloud-hypervisor --api-socket /tmp/api2
|
|
vm-2:~$ # setup the guest network with the following commands if needed
|
|
vm-2:~$ sudo ip addr add 192.168.102.2/24 dev ens4
|
|
vm-2:~$ sudo ip link set up dev ens4
|
|
vm-2:~$ sudo ip r add default via 192.168.102.1
|
|
vm-2:~$ ping 192.168.101.2 # This should succeed
|
|
```
|
|
> Note: If the above ping failed, please check the iptables rule on the
|
|
> host machine, e.g. whether the policy for the `FORWARD` chain is set
|
|
> to `DROP` (which is the default setting configured by Docker).
|
|
|
|
Get ready for receiving migration for the nested destination VM (inside
|
|
the guest OS of the VM 2):
|
|
```bash
|
|
vm-2:~$ sudo ./ch-remote --api-socket /tmp/api2 receive-migration unix:/tmp/sock2
|
|
vm-2:~$ sudo socat TCP-LISTEN:6000,reuseaddr UNIX-CLIENT:/tmp/sock2
|
|
```
|
|
|
|
Start to send migration for the nested source VM (inside the guest OS of
|
|
the VM 1):
|
|
```bash
|
|
vm-1:~$ sudo socat UNIX-LISTEN:/tmp/sock1,reuseaddr TCP:192.168.102.2:6000
|
|
vm-1:~$ sudo ./ch-remote --api-socket /tmp/api1 send-migration unix:/tmp/sock1
|
|
```
|
|
|
|
When the above commands completed, the source VM should be successfully
|
|
migrated to the destination VM without interrupting our testing guest
|
|
workload. Now the destination VM is running the testing guest workload
|
|
while the source VM is terminated gracefully.
|