Commit Graph

6524 Commits

Author SHA1 Message Date
Rob Bradford
3f02ccaa8c qcow: Add support for QCOW v2 header
The QCOW2 format is documented here:
https://git.qemu.org/?p=qemu.git;a=blob;f=docs/interop/qcow2.txt;hb=HEAD

The only difference between v2 and v3 is the addition of some extra
fields into the header in v3 for which there are default values in v2.

This introduces a new unit test for the behaviour but it has been
manually verified by the converting the image from v3 to v2
with a command like:

qemu-img convert -O qcow2 -o compat=0.10 clear-29620-cloud.img clear-29620-cloud.img.v2

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-19 17:21:54 +02:00
Rob Bradford
6f65f3406e build: Ensure caps needed for unit test are set
In some situations it is possible for the setting of the capabilities to
fail due to the variable naming of the build artifacts resulting in the
first parameter to setcap being rejected and thus the whole command
failing.

Use xargs -n 1 to ensure that every potential target independently has
its caps set.

Further it was observed that in some situations the binary produced by
cargo test --all --no-run would not be used and instead a new binary
would be produced when the test was run using the second method. This
again would result in test failures as that binary did not have the
desired capabilities set. Therefore build the test binaries with the
same methodology used to run them.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-19 12:00:10 +02:00
Rob Bradford
998140f1b0 tests: Remove single test limit
Run the tests with default parallelisation.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-18 18:01:18 +02:00
Rob Bradford
492ab7a1a8 build: Use tmpfs for /tmp
On the Jenkins build slaves disk I/O is a bottlneck so make /tmp a tmpfs
which removes I/O issues when running lots of VMs at the same time.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-18 18:01:18 +02:00
Rob Bradford
80f33113cb tests: Use incrementing IP and mac address for VMs
This allows us to test multiple VMs at once.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-18 18:01:18 +02:00
Rob Bradford
93c2099ab6 tests: Abstract guest management under a struct
Create a struct to handle all the details for the guest under test
including details of network and disks.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-18 18:01:18 +02:00
Rob Bradford
eab639efe3 tests: Support customising the cloud-init network details
Allow replacement of the network details used for the VM. By replacing
those from the file checked into the source tree we can continue to use
the file in the tree for manual testing but adjust the network per-VM to
allow parallel testing.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-18 18:01:18 +02:00
Rob Bradford
e9f01740e8 tests: Create cloud-init image from source files in tests
In the future this will provide the basis for the ability to customise
the cloud-init file per VM.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-18 18:01:18 +02:00
Rob Bradford
0776d9d7ae tests: Sleep more in order to speed up tests
By sleeping more earlier this will speed up the tests as the SSH
connection will complete on the first attempt and thus alleviate timeout
and backoff delays.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-18 18:01:18 +02:00
Rob Bradford
7ebfe90985 tests: Use a temporary directory for the temporary test files
Use the tempdir crate to create a temporary directory that is deleted
when the structure goes out of scope.

Use this temporary directory for all temporary test files created by the
tests. The cloud init file is still in /tmp as that is created by the
test wrapper code.

This is the first stage towards being able to run the integration tests
in parallel.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-18 18:01:18 +02:00
Rob Bradford
78fe807284 build: Run unit tests on the Jenkins server
The addition of [workspace] to the top level Cargo.toml is necessary to
have the binaries colocated together.

The Cargo.lock files have also been refreshed by the change to the
Cargo.toml.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
1dfe16c382 vhost_rs: vhost_user: Update unit tests
The Master::connect() function now takes a queue count.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
7499210d0c vm-virtio: net: Remove attributes for test exclusions
Now that the tests are in use this import and function is used.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
af15ce9dc3 vm-virtio: Update test activate() function
The type of interrupt_evt has changed along with the addition of an
msix_config member for the virtio device.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
a50c54671c qcow: Make unit tests pass
Rather than relying on shared memory for a temporary file for QCOW
testing instead use tempfile crate to get a temporary file. The vector
cache tests also need a trivial update after the refactor.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
9a17871630 pci: Make unit tests compile
Another member was added to the configuration struct.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
74d079f7da pci: Mark add_capability test as #[ignore] as it is currently failing
See #105

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
18d52869c5 arch: x86_64: Make unit tests pass
With the adoption for rust-vmm linux-loader crate some small changes
were needed to update the unit tests to reflect this change:

* configure_system now takes an extra parameter
* the e820 entry structure comes from the linux-loader crate

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
224f77500c devices: serial: Make the serial unit tests pass
Some refactoring has taken place since the unit tests were written:

The read/write in BusDevice now take a base address and the interrupt
handling code has changed necessitating the need for a new TestInterrupt
struct.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
9e372a8e48 net_util: Bump pnet dependency
The older version of pnet had a bug which broke some of the behaviour
that the unit tests relied upon.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-16 17:09:05 +02:00
Rob Bradford
cb81f8be5b vmm: Make serial port controllable via command line
Add a "--serial" command line that takes as input either "off", "tty"
(default and current behaviour) and "file=/path/to/file".

When "--serial off" is used the serial device is not added to the VM
configuration at all.

Integration tests added that check for interrupts present (or not) and
that when sending to a file the file contains the expected serial
output.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-11 12:17:58 +01:00
Samuel Ortiz
00df79a530 README: Really fix the memory parameters examples
For real now...

Fixes: #96

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-11 08:49:38 +01:00
Samuel Ortiz
bc742d6a5f main: Add a PCI bridge class integration test
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-10 09:53:02 +02:00
Samuel Ortiz
2b2c31d206 pci: Use device PCI header type for our root bridge
The Bridge header type is really meant to be for PCI-to-PCI bridges.

Fixes: #85

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-10 09:53:02 +02:00
Samuel Ortiz
d6d9b76c7e README: Fix the memory parameter examples
We need to give it a unit.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-10 08:35:10 +02:00
Samuel Ortiz
7ed073805d config: Fix default memory size parameter
We need to give it a suffix.

Fixes: #96

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-10 08:35:10 +02:00
Sebastien Boeuf
d9ce29117e vmm: Flag --disk should be optional
Now that cloud-hypervisor VMM supports virtio-pmem, it can directly
boot a VM from an image exposed as a persistent memory block device.

That's why there is no need to force the --disk option as being
mandatory.

Fixes #90

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-07-09 21:58:02 +02:00
Sebastien Boeuf
f0a76ad424 vmm: Add support for multiple virtio-net devices
Until now, the VMM was only accepting a single instance of virtio-net
device. This commit extends the virtio-net support by allowing several
devices to be created for a single VM.

Fixes #71

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-07-09 18:55:30 +01:00
Sebastien Boeuf
a2947f9a9f cli: Accept K,M,G suffixes for size parameters
For every parameter dealing with a size as option, such as memory or
virtio-pmem, the CLI can now parse sizes with the suffixes K, M or G.

Fixes #70

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-07-09 15:22:26 +01:00
Jing Liu
2bb0b22cc1 pci: Refine pci topology
PciConfigIo is a legacy pci bus dispatcher, which manages all pci
devices including a pci root bridge. However, it is unnecessary to
design a complex hierarchy which redirects every access by PciRoot.

Since pci root bridge is also a pci device instance, and only contains
easy config space read/write, and PciConfigIo actually acts as a pci bus
to dispatch resource based resolving when VMExit, we re-arrange to make
the pci hierarchy clean.

Signed-off-by: Jing Liu <jing2.liu@linux.intel.com>
2019-07-09 10:01:18 +02:00
Samuel Ortiz
4605ecf1a8 pci: Extend the Device trait to carry the device BARs
When reading from or writing to a PCI BAR to handle a VM exit, we need
to have the BAR address itself to be able to support multiple BARs PCI
devices.

Fixes: #87

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-08 07:39:21 +02:00
Samuel Ortiz
8173e1ccd7 devices: Extend the Bus trait to carry the device range base
With the range base for the IO/MMIO vm exit address, a device with
multiple ranges has all the needed information for resolving which of
its range the exit is coming from

Fixes: #87

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-08 07:39:21 +02:00
Samuel Ortiz
42e545806c devices: bus: Return the range base address when resolving
When resolving an IO address to a device, return the range base address,
the offset, and the device itself.
This is needed for devices with multiple IO regions to find out which
region an IO/MMIO exit is coming from.

We also use this change as an opportunity to rename get_device to
resolve as we're doing more than just getting a device.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-08 07:39:21 +02:00
Samuel Ortiz
33796c4b0b devices: bus: Sync with crosvm code
Partial sync as we're not going to use the the full_addr boolean.
This is based on crosvm commit 44863792.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-08 07:39:21 +02:00
Samuel Ortiz
4a15316101 vm-virtio: Fix the network and storage PCI class and sub-class
Use the virtio device type to generate the righ class and subclass.

Fixes: #83

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-02 17:37:12 +02:00
Samuel Ortiz
77684f473d vm-virtio: Implement the u32 to VirtioDeviceType conversion
The From trait allows us to compare and convert an integer
with and into a virtio device type.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-02 17:37:12 +02:00
Samuel Ortiz
0b7fb42a6c pci: Export network and mass storage sub classes
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-07-02 17:37:12 +02:00
Rob Bradford
49d6b495d5 vmm: Remove println! from debugging
Remove println! left over from virtio-fs development.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-07-02 13:50:50 +02:00
Sebastien Boeuf
34e09923a5 vmm: Add support for multiple virtio-pmem devices
Until now, the VMM was only accepting a single instance of virtio-pmem
device. This commit extend the virtio-pmem support by allowing several
devices to be created for a single VM.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-07-01 14:38:55 +01:00
Sebastien Boeuf
785db6295e test: Add virtio-pmem integration tests
Add 2 integration tests to validate virtio-pmem works as expected.
One test takes care of checking the ability to read and write to this
persistent memory from the guest, and validates that the data is
carried over the virtualization boundary.
The other test ensures the VM can be booted directly from an image
that would be passed through virtio-pmem.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-07-01 14:38:55 +01:00
Sebastien Boeuf
294c26bfb7 vmm: Add virtio-pmem support to cloud-hypervisor
This patch plumbs the virtio-pmem device to the VMM. By adding a new
command line option "--pmem", we can now expose some persistent memory
to the guest OS, backed by the provided source.

The point of having such support in cloud-hypervisor is to be able to
share some memory between the host and the guest as DAXable.
One interesting use case is to boot directly from an image passed
through virtio-pmem, instead of going through virtio-blk. This can
allow good performances while avoiding the guest cache, which would
prevent the VM memory footprint from growing too much.

Fixes #68

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-07-01 14:38:55 +01:00
Sebastien Boeuf
8862d61042 vm-virtio: Add virtio-pmem implementation
This commit introduces the implementation of the virtio-pmem device
based on the pending proposal of the virtio specification here:
https://lists.oasis-open.org/archives/virtio-dev/201903/msg00083.html

It is also based on the kernel patches coming along with the virtio
proposal: https://lkml.org/lkml/2019/6/12/624

And it is based off of the current crosvm implementation found in
devices/src/virtio/pmem.rs relying on commit
bb340d9a94d48514cbe310d05e1ce539aae31264

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-07-01 14:38:55 +01:00
Sebastien Boeuf
c0336e83ea docs: Add virtio-fs documentation
Add some documentation specific to virtio-fs and how to perform
filesystem sharing between host and guest with cloud-hypervisor.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00
Sebastien Boeuf
1cb2378499 vmm: Add support for multiple virtio-fs devices
Until now, the VMM was only accepting a single instance of a virtio-fs
device. This commit extend the virtio-fs support by allowing several
devices to be created for a single VM.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00
Sebastien Boeuf
0fcca3ed74 tests: Add virtio-fs testing
This commit introduces the testing of the --fs option based on the
virtio-fs implementation. This does not simply add a test, but also
updates the integration script by generating a new kernel embedding
the virtio-fs patches and by downloading the virtiofsd daemon.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00
Sebastien Boeuf
53085c7ccc memory: Allow memory to be backed by a file
In the context of vhost-user, we need the guest RAM to be backed by
a file in order to be accessed by an external process. This patch
adds the new flag "file=" to the "--memory" option so that we can
specify from the command line if the memory needs to be backed, and
by which specific file.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00
Sebastien Boeuf
2ede30b6d3 vmm: Add virtio-fs support to the VMM
The user can now share some files and directories with the guest by
providing the corresponding vhost-user socket. The virtiofsd daemon
should be started by the user before to start the VM.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00
Sebastien Boeuf
1ddc8f2f0d vm-virtio: Add vhost-user-fs support
The vhost-user-fs or virtio-fs device allows files and directories to
be shared between host and guest. This patch adds the implementation
of this device to the cloud-hypervisor device model.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00
Sebastien Boeuf
8f70771b1d vhost_rs: Copy vhost crate from jiangliu/v1
In order to avoid cloud-hypervisor to rely on a pending PR for the empty
crate "vhost", this commit temporarily copies the content of the crate
based on branch jiangliu/v1 18b5081d9199c76eca49da1971c9d1a65e53e5ff.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00
Sebastien Boeuf
8dc06aa50d vm-virtio: Remove unneeded code
Remove legacy code coming from Firecracker and/or Crosvm.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-06-27 21:46:00 +02:00