Commit Graph

890 Commits

Author SHA1 Message Date
dependabot-preview[bot]
7909f7a8c9 build(deps): bump serde_derive from 1.0.103 to 1.0.104
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.103 to 1.0.104.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.103...v1.0.104)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-16 07:05:03 +00:00
Rob Bradford
23150f8647 tests: Throttle tests based on disk space
On our CI the /tmp filesystem is mounted as tmpfs and this is the
location where the test disk images are located. When the CI worker
nodes have less memory and fewer CPUs the tmpfs fills up as the tests
run in parallel.

Introduce a mechanism to reduce the parallelism of the tests based on
starvation of the tmpfs disk availability.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-13 16:42:59 +01:00
Rob Bradford
d59fe4eee7 ci: Add some time measurements to CI scripts
Add some use of "time" to measure the wallclock time for various items
within our CI scripts.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-13 13:42:52 +00:00
Samuel Ortiz
fa0fdc6500 cargo: Update Cargo.lock for the 0.4.0 release
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-13 12:21:13 +01:00
Samuel Ortiz
cec884e863 release: v0.4.0
Expand release notes and bump Cargo.toml.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-13 11:42:59 +01:00
Rob Bradford
6444e29b04 docs: Add CPU hot plug documentation
Add details of how to add vCPUs to the running VM.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-13 10:34:41 +00:00
dependabot-preview[bot]
a002093a55 build(deps): bump anyhow from 1.0.23 to 1.0.25
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.23 to 1.0.25.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.23...1.0.25)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-13 08:29:30 +01:00
dependabot-preview[bot]
43f0478fa8 build(deps): bump thiserror from 1.0.6 to 1.0.9
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.6 to 1.0.9.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.6...1.0.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-13 06:38:08 +00:00
Samuel Ortiz
664431ff14 vsock: vhost_user: vfio: Fix potential host memory overflow
The vsock packets that we're building are resolving guest addresses to
host ones and use the latter as raw pointers.
If the corresponding guest mapped buffer spans across several regions in
the guest, they will do so in the host as well. Since we have no
guarantees that host regions are contiguous, it may lead the VMM into
trying to access memory outside of its memory space.

For now we fix that by ensuring that the guest buffers do not span
across several regions. If they do, we error out.
Ideally, we should enhance the rust-vmm memory model to support safe
acces across host regions.

Fixes CVE-2019-18960

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 22:15:50 +01:00
Rob Bradford
1e97d1413e README: Update for newer distribution support
Highlight that we support Ubuntu Bionic and Eoan as well as Clear Linux
and update the Clear Linux versions referenced.

Also update the firmware URL to point to the latest version.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-12 13:36:28 +01:00
Samuel Ortiz
e8e21aeb7e README: Update the --cpus command line examples
We recommend to specify the boot cpus, even though the old syntax is still
supported.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 11:55:37 +00:00
Sebastien Boeuf
a40a70ad86 ci: Rely on latest virtiofsd version
Virtio-fs maintainers recently updated the virtiofsd daemon through
their official branch on Gitlab. It includes fixes that were needed for
cloud-hypervisor to work correctly with it.

Jenkinsfile needs to be updated since the virtiofsd build requires both
libseccomp and libcap-ng to be present on the system.

One thing to notice, because the latest branch introduced a change
regarding libfuse behavior, the counterpart patch has been added to the
custom kernel branch "virtio-fs-virtio-iommu".

Fixes #536

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-12 11:09:36 +01:00
Samuel Ortiz
f0b7412495 vmm: device_manager: Add all virtio devices to the migratable list
We want to track all migratable devices through the DeviceManager.

Fixes: #341

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
37557c8b35 vmm: vm: Implement the Pausable trait
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
9756fc2dd0 vmm: cpu_manager: Implement the Pausable trait
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
35dd1523c9 vmm: device_manager: Implement the Pausable trait
Since the Snapshotable placeholder and Migratable traits are provided as
well, the DeviceManager object and all its objects are now Migratable.

All Migratable devices are tracked as Arc<Mutex<dyn Migratable>>
references.

Keeping track of all migratable devices allows for implementing the
Migratable trait for the DeviceManager structure, making the whole
device model potentially migratable.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
a122da4bef vm-virtio: vhost: Implement the Pausable trait for all vhost-user devices
Due to the amount of code currently duplicated across vhost-user devices,
the stats for this commit is on the large side but it's mostly more
duplicated code, unfortunately.

Migratable and Snapshotable placeholder implementations are provided as
well, making all vhost-user devices Migratable.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
dae0b2ef72 vm-virtio: Implement the Pausable trait for all virtio devices
Due to the amount of code currently duplicated across virtio devices,
the stats for this commit is on the large side but it's mostly more
duplicated code, unfortunately.

Migratable and Snapshotable placeholder implementations are provided as
well, making all virtio devices Migratable.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
35d7721683 vmm: Convert virtio devices to Arc<Mutex<T>>
Migratable devices can be virtio or legacy devices.
In any case, they can potentially be tracked through one of the IO bus
as an Arc<Mutex<dyn BusDevice>>. In order for the DeviceManager to also
keep track of such devices as Migratable trait objects, they must be
shared as mutable atomic references, i.e. Arc<Mutex<T>>. That forces all
Migratable objects to be tracked as Arc<Mutex<dyn Migratable>>.

Virtio devices are typically migratable, and thus for them to be
referenced by the DeviceManager, they now should be built as
Arc<Mutex<VirtioDevice>>.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
5450de0f5e cargo: Do not run fmt on anyhow's build code
The anyhow crate generates some incorrectly indented code from its
build.rs code. We don't want to run cargo fmt on this code.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
0361df4ae0 vm-device: Initial Migratable trait
The Migratable trait groups all expected capabilities of devices and
components that can be migrated.

For a component to be migrated, it must be able to pause and resume.
Once paused, it should be able to provide a snapshot of itself. It
should also be able to restore itself from a snaphot.

As a consequence, the Migratable trait will be split between the
Pausable and the Snapshotable traits. This commit only adds the
Pausable one.

All migratable devices will be tracked from the DeviceManager.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Rob Bradford
36daf9c0b0 ci: Skip testing RFC or WIP PRs
To alleviate load on the CI only test PRs that are not marked as RFC or
WIP.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-11 18:22:12 +01:00
Sebastien Boeuf
64c5e3d8cb vmm: api: Adjust FsConfig for OpenAPI
The FsConfig structure has been recently adjusted so that the default
value matches between OpenAPI and CLI. Unfortunately, with the current
description, there is no way from the OpenAPI to describe a cache_size
value "None", so that DAX does not get enabled. Usually, using a Rust
"Option" works because the default value is None. But in this case, the
default value is Some(8G), which means we cannot describe a None.

This commit tackles the problem, introducing an explicit parameter
"dax", and leaving "cache_size" as a simple u64 integer.

This way, the default value is dax=true and cache_size=8G, but it lets
the opportunity to disable DAX entirely with dax=false, which will
simply ignore the cache_size value.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
4bfd51cc42 vmm: api: Match VhostUserBlkConfig defaults between CLI and HTTP API
In order to let the CLI and the HTTP API behave the same regarding the
VhostUserBlkConfig structure, this patch defines some default values
for num_queues, queue_size and wce.

num_queues is 1, queue_size is 128 and wce is true.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
1c2587f8cb vmm: api: Match VhostUserNetConfig defaults between CLI and HTTP API
In order to let the CLI and the HTTP API behave the same regarding the
VhostUserNetConfig structure, this patch defines some default values
for num_queues, queue_size and mac.

num_queues is 2 since that's a pair of TX/RX queues, queue_size is 256
and mac is a randomly generated value.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
5e0bbf9c3b vmm: Don't factorize vhost-user configurations
We want to set different default configurations for vhost-user-net and
vhost-user-blk, which is the reason why the common part corresponding to
the number of queues and the queue size cannot be embedded.

This prepares for the following commit, matching API and CLI behaviors.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
793327cff8 vmm: api: Make ConsoleConfig default match between CLI and HTTP API
A simple patch making sure the field "file" is provisioned with the same
default value through CLI and OpenAPI.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
cc08c44cb9 vmm: api: Make MemoryConfig default match between CLI and HTTP API
Just making sure we have a serde default for the field "file" since it
is not a required field in the OpenAPI definition.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
5a72225856 vmm: api: Update CpuConfig name to match the internal name
All structures match between the OpenAPI definition and the internal
configuration code, that's why CpuConfig is being renamed into
CpusConfig.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
f7c215d92d cli: Fix default CPU argument
By default, and in order to avoid falling into the legacy CLI usage, the
CPU argument should at least include "boot=" to define the number of
CPUs.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Rob Bradford
59ae01ff71 ci: Cancel older builders on Jenkins
When a new build is triggered cancel any older builds to conserve
resources.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-11 15:50:01 +00:00
Rob Bradford
c61104df47 vmm: Port to latest vmm-sys-util
The signal handling for vCPU signals has changed in the latest release
so switch to the new API.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-11 14:11:11 +00:00
Sebastien Boeuf
4c92f89f0f ci: Add OpenAPI validation
We need to validate that OpenAPI YAML definition is not broken by each
and every pull request. The easiest way is to rely on the Docker image
provided by OpenAPITools, as it allows us to validate the definition
with one simple command.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 12:53:31 +00:00
Sebastien Boeuf
93bd88e681 ci: Simplify travis.yml
Now that Cargo tests are being run on the worker node directly, we can
leave the bare minimum in Travis, which is taking care of the release
deployment.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 12:53:31 +00:00
Sebastien Boeuf
d42ef186a1 ci: Offload cargo tests to the worker node VM
Because the resources on the amount of worker nodes we can have access
to through Travis is limited, we offload the burden of running all tests
related to Cargo inside the Azure VM directly.

This will have the positive effect of stopping the build very early in
case something goes wrong during the Cargo testing.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 12:53:31 +00:00
Sebastien Boeuf
ee528ae808 vmm: api: Make FsConfig defaults match between CLI and HTTP API
In order to let the CLI and the HTTP API behave the same regarding the
FsConfig structure, this patch defines some default values for
num_queues, queue_size and the cache_size.

num_queues is set to 1, queue_size is set to 1024, and cache_size is set
to Some(8G) which means that DAX is enabled by default with a shared
region of 8GiB.

Fixes #508

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-09 23:42:23 -08:00
Sebastien Boeuf
befd342da4 vmm: api: Make NetConfig defaults match between CLI and HTTP API
In order to let the CLI and the HTTP API behave the same regarding the
NetConfig structure, this patch defines some default values for tap, ip,
mask, mac and iommu.

tap is None, ip is 192.168.249.1, mask is 255.255.255.0, mac is a
randomly generated value, and iommu is false.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-09 23:19:24 -08:00
Sebastien Boeuf
eff78f746a resources: Prevent kernel config interactive shell from showing up
Following the recent addition of CONFIG_EXPERT=y, the kernel config now
asks for several options through interactive shell. This is not
convenient when locally running the integration tests script as the
shell asks for input.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-10 06:24:11 +01:00
Jose Carlos Venegas Munoz
99e608c240 openapi: Fix schema
Fix openapi schema to be a valid yaml.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-12-09 14:30:15 -08:00
Rob Bradford
f994665610 vmm: Reduce the minimum IRQ constant
Now that the GED device does not use a hardcoded IRQ number the starting
IRQ number can be restored (needed for the hardcoded serial port IRQ.)

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-09 16:58:00 +00:00
Rob Bradford
ba59c62044 vmm, devices: Remove hardcoded IRQ number for GED device
Remove the previously hardcoded IRQ number used for the GED device.
Instead allocate the IRQ using the allocator and use that value in the
definition in the ACPI device.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-09 16:58:00 +00:00
Qiu Wenbo
ce1bd9c7ff resources: turn on CONFIG_ACPI_REDUCED_HARDWARE_ONLY
We need turn on CONFIG_ACPI_REDUCED_HARDWARE_ONLY to support cpu hotplug
feature.

Signed-off-by: Qiu Wenbo <qiuwenbo@phytium.com.cn>
2019-12-09 10:49:17 +01:00
dependabot-preview[bot]
0374c3dc71 build(deps): bump ssh2 from 0.5.0 to 0.6.0
Bumps [ssh2](https://github.com/alexcrichton/ssh2-rs) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/alexcrichton/ssh2-rs/releases)
- [Commits](https://github.com/alexcrichton/ssh2-rs/compare/0.5.0...0.6.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-09 05:53:30 +00:00
Sebastien Boeuf
aa94e9b8f3 Revert "vmm: api: Modify FsConfig to be OpenAPI friendly"
This reverts commit defc5dcd9c.
2019-12-06 18:08:10 +00:00
Rob Bradford
9b1ba14f2d vmm: Delegate device related ACPI DSDT table work to DeviceManager
Move the code for handling the creation of the DSDT entries for devices
into the DeviceManager.

This will make it easier to handle device hotplug and also in the future
remove some hardcoded ACPI constants.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-06 17:44:00 +00:00
Rob Bradford
60e6609011 vmm: Delegate CPU related ACPI tables to CpuManager
Move the code for generating the MADT (APIC) table and the DSDT
generation for CPU related functionality into the CpuManager.

There is no functional change just code rearrangement.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-06 17:44:00 +00:00
Sebastien Boeuf
defc5dcd9c vmm: api: Modify FsConfig to be OpenAPI friendly
When consumer of the HTTP API try to interact with cloud-hypervisor,
they have to provide the equivalent of the config structure related to
each component they need. Problem is, the Rust enum type "Option" cannot
be obtained from the OpenAPI YAML definition.

This patch intends to fix this inconsistency between what is possible
through the CLI and what's possible through the HTTP API by using simple
types bool and int64 instead of Option<u64>.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-06 06:38:48 -08:00
Sebastien Boeuf
66e00ce710 ci: Extend VFIO integration test
In order to validate that multiple devices can be passed through and
they are still fully functional, this patch extends the existing VFIO
test to pass a second virtio-net device, and verifies that both
interfaces are functional by ssh'ing into each network interface.

Fixes #503

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-06 12:53:09 +01:00
Rob Bradford
59d01712ad vmm: Remove kernel based IOAPIC handling from the device manager
Previously the device setup code assumed that if no IOAPIC was passed in
then the device should be added to the kernel irqchip. As an earlier
change meant that there was always a userspace IOAPIC this kernel based
code can be removed.

The accessor still returns an Option type to leave scope for
implementing a situation without an IOAPIC (no serial or GED device).
This change does not add support no-IOAPIC mode as the original code did
not either.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-06 12:34:06 +01:00
Rob Bradford
afea6a10a2 vmm: Stop initialising kernel based IOAPIC/PIC
Now that we require the modern capabilities we can stop creating a
kernel base irqchip.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-06 12:34:06 +01:00