Commit Graph

1205 Commits

Author SHA1 Message Date
Yang Zhong
cee01edb97 vhost-user-blk backend: add readonly support
The current backend only support rw, and we also need
add readonly support.

The new command:
vhost_user_blk \
  --backend "image=/home/test.img, \
            sock=/home/path/vhost.socket, \
            readonly=true"

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
2019-12-18 09:45:11 +01:00
Rob Bradford
91549ddd71 tests: Add integration test for CPU unplug
Enhance the CPU hotplug test to also unplug the CPUs.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-18 08:23:53 +00:00
Rob Bradford
a6878accd5 vmm: cpu: Implement CPU removal
When the running OS has been told that a CPU should be removed it will
shutdown the CPU and then signal to the hypervisor via the "_EJ0" method
on the device that ultimately writes into an I/O port than the vCPU
should be shutdown. Upon notification the hypervisor signals to the
individual thread that it should shutdown and waits for that thread to
end.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-18 08:23:53 +00:00
Rob Bradford
7b3fc72aea vmm: cpu: Notify guest OS that it should offline vCPUs
Allow the resizing of the number of vCPUs to less than the current
active vCPUs. This does not currently remove them from the system but
the kernel will take them offline.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-18 08:23:53 +00:00
Rob Bradford
7e81b0ded7 vmm: cpu: Create vCPU state for all possible vCPUs
This will make it more straightforward when we attempt to remove vCPUs.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-18 08:23:53 +00:00
Rob Bradford
156ea392a2 vmm: cpu: Only do ACPI notify on newly added vCPUs
When we add a vCPU set an "inserting" boolean that is exposed as an ACPI
field that will be checked for and reset when the ACPI GED notification
for CPU devices happens.

This change is a precursor for CPU unplug.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-16 23:57:14 +01:00
Rob Bradford
e8313e3e69 vmm: acpi: Refactor ACPI CPU notification
Continue to notify on all vCPUs but instead separate the notification
functionality into two methods, CSCN that walks through all the CPUs
and CTFY which notifies based on the numerical CPU id. This is an
interim step towards only notifying on changed CPUs and ultimately CPU
removal.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-16 23:57:14 +01:00
Sebastien Boeuf
43d2e09e1f ci: Add unit tests to compare CLI and OpenAPI
The goal here is to ensure that CLI and OpenAPI both behave as closely
as possible, and also that they behave as expected.

Leveraging the reorganization of the code, we can now compare two
VmConfig structures generated from one CLI entry on one side, and from
an OpenAPI entry (JSON payload) on the other side.

Fixes #535

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-16 16:48:59 +01:00
Sebastien Boeuf
d1390906c8 vmm: config: Derive Debug and PartialEq for configuration structures
In anticipation for the writing of unit tests comparing two VmConfig
structures, this commit derives the PartialEq trait for VmConfig and
all embedded structures.

This patch also derives the Debug trait for the same set of structures
so that we can print them to facilitate debugging.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-16 16:48:59 +01:00
Sebastien Boeuf
93f5f6ed45 vmm: config: Provide a default empty command line through OpenAPI
The OpenAPI should not have to provide a command line since the CLI
considers the command line as an empty string if nothing is provided.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-16 16:48:59 +01:00
Sebastien Boeuf
43bd0e53c4 main: Move VmParams creation into a dedicated function
This brings more modularity to the code, which will be helpful when we
will later test the CLI and OpenAPI generate the same VmConfig output.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-16 16:48:59 +01:00
Sebastien Boeuf
17a167dbb6 main: Move default values preparation into a dedicated function
This brings more modularity to the code, which will be helpful when we
will later test the CLI and OpenAPI generate the same VmConfig output.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-16 16:48:59 +01:00
Sebastien Boeuf
9c3a7ddcc9 main: Move clap::App creation into a dedicated function
This brings more modularity to the code, which will be helpful when we
will later test the CLI and OpenAPI generate the same VmConfig output.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-16 16:48:59 +01:00
dependabot-preview[bot]
11750efb78 build(deps): bump log from 0.4.8 to 0.4.10
Bumps [log](https://github.com/rust-lang/log) from 0.4.8 to 0.4.10.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.10)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-16 14:46:05 +00:00
dependabot-preview[bot]
dea5776fb9 build(deps): bump serde from 1.0.103 to 1.0.104
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.103 to 1.0.104.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.103...v1.0.104)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-16 08:15:47 +00:00
dependabot-preview[bot]
7909f7a8c9 build(deps): bump serde_derive from 1.0.103 to 1.0.104
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.103 to 1.0.104.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.103...v1.0.104)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-16 07:05:03 +00:00
Rob Bradford
23150f8647 tests: Throttle tests based on disk space
On our CI the /tmp filesystem is mounted as tmpfs and this is the
location where the test disk images are located. When the CI worker
nodes have less memory and fewer CPUs the tmpfs fills up as the tests
run in parallel.

Introduce a mechanism to reduce the parallelism of the tests based on
starvation of the tmpfs disk availability.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-13 16:42:59 +01:00
Rob Bradford
d59fe4eee7 ci: Add some time measurements to CI scripts
Add some use of "time" to measure the wallclock time for various items
within our CI scripts.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-13 13:42:52 +00:00
Samuel Ortiz
fa0fdc6500 cargo: Update Cargo.lock for the 0.4.0 release
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-13 12:21:13 +01:00
Samuel Ortiz
cec884e863 release: v0.4.0
Expand release notes and bump Cargo.toml.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-13 11:42:59 +01:00
Rob Bradford
6444e29b04 docs: Add CPU hot plug documentation
Add details of how to add vCPUs to the running VM.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-13 10:34:41 +00:00
dependabot-preview[bot]
a002093a55 build(deps): bump anyhow from 1.0.23 to 1.0.25
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.23 to 1.0.25.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.23...1.0.25)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-13 08:29:30 +01:00
dependabot-preview[bot]
43f0478fa8 build(deps): bump thiserror from 1.0.6 to 1.0.9
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.6 to 1.0.9.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.6...1.0.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-12-13 06:38:08 +00:00
Samuel Ortiz
664431ff14 vsock: vhost_user: vfio: Fix potential host memory overflow
The vsock packets that we're building are resolving guest addresses to
host ones and use the latter as raw pointers.
If the corresponding guest mapped buffer spans across several regions in
the guest, they will do so in the host as well. Since we have no
guarantees that host regions are contiguous, it may lead the VMM into
trying to access memory outside of its memory space.

For now we fix that by ensuring that the guest buffers do not span
across several regions. If they do, we error out.
Ideally, we should enhance the rust-vmm memory model to support safe
acces across host regions.

Fixes CVE-2019-18960

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 22:15:50 +01:00
Rob Bradford
1e97d1413e README: Update for newer distribution support
Highlight that we support Ubuntu Bionic and Eoan as well as Clear Linux
and update the Clear Linux versions referenced.

Also update the firmware URL to point to the latest version.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-12 13:36:28 +01:00
Samuel Ortiz
e8e21aeb7e README: Update the --cpus command line examples
We recommend to specify the boot cpus, even though the old syntax is still
supported.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 11:55:37 +00:00
Sebastien Boeuf
a40a70ad86 ci: Rely on latest virtiofsd version
Virtio-fs maintainers recently updated the virtiofsd daemon through
their official branch on Gitlab. It includes fixes that were needed for
cloud-hypervisor to work correctly with it.

Jenkinsfile needs to be updated since the virtiofsd build requires both
libseccomp and libcap-ng to be present on the system.

One thing to notice, because the latest branch introduced a change
regarding libfuse behavior, the counterpart patch has been added to the
custom kernel branch "virtio-fs-virtio-iommu".

Fixes #536

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-12 11:09:36 +01:00
Samuel Ortiz
f0b7412495 vmm: device_manager: Add all virtio devices to the migratable list
We want to track all migratable devices through the DeviceManager.

Fixes: #341

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
37557c8b35 vmm: vm: Implement the Pausable trait
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
9756fc2dd0 vmm: cpu_manager: Implement the Pausable trait
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
35dd1523c9 vmm: device_manager: Implement the Pausable trait
Since the Snapshotable placeholder and Migratable traits are provided as
well, the DeviceManager object and all its objects are now Migratable.

All Migratable devices are tracked as Arc<Mutex<dyn Migratable>>
references.

Keeping track of all migratable devices allows for implementing the
Migratable trait for the DeviceManager structure, making the whole
device model potentially migratable.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
a122da4bef vm-virtio: vhost: Implement the Pausable trait for all vhost-user devices
Due to the amount of code currently duplicated across vhost-user devices,
the stats for this commit is on the large side but it's mostly more
duplicated code, unfortunately.

Migratable and Snapshotable placeholder implementations are provided as
well, making all vhost-user devices Migratable.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
dae0b2ef72 vm-virtio: Implement the Pausable trait for all virtio devices
Due to the amount of code currently duplicated across virtio devices,
the stats for this commit is on the large side but it's mostly more
duplicated code, unfortunately.

Migratable and Snapshotable placeholder implementations are provided as
well, making all virtio devices Migratable.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
35d7721683 vmm: Convert virtio devices to Arc<Mutex<T>>
Migratable devices can be virtio or legacy devices.
In any case, they can potentially be tracked through one of the IO bus
as an Arc<Mutex<dyn BusDevice>>. In order for the DeviceManager to also
keep track of such devices as Migratable trait objects, they must be
shared as mutable atomic references, i.e. Arc<Mutex<T>>. That forces all
Migratable objects to be tracked as Arc<Mutex<dyn Migratable>>.

Virtio devices are typically migratable, and thus for them to be
referenced by the DeviceManager, they now should be built as
Arc<Mutex<VirtioDevice>>.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
5450de0f5e cargo: Do not run fmt on anyhow's build code
The anyhow crate generates some incorrectly indented code from its
build.rs code. We don't want to run cargo fmt on this code.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Samuel Ortiz
0361df4ae0 vm-device: Initial Migratable trait
The Migratable trait groups all expected capabilities of devices and
components that can be migrated.

For a component to be migrated, it must be able to pause and resume.
Once paused, it should be able to provide a snapshot of itself. It
should also be able to restore itself from a snaphot.

As a consequence, the Migratable trait will be split between the
Pausable and the Snapshotable traits. This commit only adds the
Pausable one.

All migratable devices will be tracked from the DeviceManager.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-12-12 08:50:36 +01:00
Rob Bradford
36daf9c0b0 ci: Skip testing RFC or WIP PRs
To alleviate load on the CI only test PRs that are not marked as RFC or
WIP.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-11 18:22:12 +01:00
Sebastien Boeuf
64c5e3d8cb vmm: api: Adjust FsConfig for OpenAPI
The FsConfig structure has been recently adjusted so that the default
value matches between OpenAPI and CLI. Unfortunately, with the current
description, there is no way from the OpenAPI to describe a cache_size
value "None", so that DAX does not get enabled. Usually, using a Rust
"Option" works because the default value is None. But in this case, the
default value is Some(8G), which means we cannot describe a None.

This commit tackles the problem, introducing an explicit parameter
"dax", and leaving "cache_size" as a simple u64 integer.

This way, the default value is dax=true and cache_size=8G, but it lets
the opportunity to disable DAX entirely with dax=false, which will
simply ignore the cache_size value.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
4bfd51cc42 vmm: api: Match VhostUserBlkConfig defaults between CLI and HTTP API
In order to let the CLI and the HTTP API behave the same regarding the
VhostUserBlkConfig structure, this patch defines some default values
for num_queues, queue_size and wce.

num_queues is 1, queue_size is 128 and wce is true.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
1c2587f8cb vmm: api: Match VhostUserNetConfig defaults between CLI and HTTP API
In order to let the CLI and the HTTP API behave the same regarding the
VhostUserNetConfig structure, this patch defines some default values
for num_queues, queue_size and mac.

num_queues is 2 since that's a pair of TX/RX queues, queue_size is 256
and mac is a randomly generated value.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
5e0bbf9c3b vmm: Don't factorize vhost-user configurations
We want to set different default configurations for vhost-user-net and
vhost-user-blk, which is the reason why the common part corresponding to
the number of queues and the queue size cannot be embedded.

This prepares for the following commit, matching API and CLI behaviors.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
793327cff8 vmm: api: Make ConsoleConfig default match between CLI and HTTP API
A simple patch making sure the field "file" is provisioned with the same
default value through CLI and OpenAPI.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
cc08c44cb9 vmm: api: Make MemoryConfig default match between CLI and HTTP API
Just making sure we have a serde default for the field "file" since it
is not a required field in the OpenAPI definition.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
5a72225856 vmm: api: Update CpuConfig name to match the internal name
All structures match between the OpenAPI definition and the internal
configuration code, that's why CpuConfig is being renamed into
CpusConfig.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Sebastien Boeuf
f7c215d92d cli: Fix default CPU argument
By default, and in order to avoid falling into the legacy CLI usage, the
CPU argument should at least include "boot=" to define the number of
CPUs.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 15:50:24 +00:00
Rob Bradford
59ae01ff71 ci: Cancel older builders on Jenkins
When a new build is triggered cancel any older builds to conserve
resources.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-11 15:50:01 +00:00
Rob Bradford
c61104df47 vmm: Port to latest vmm-sys-util
The signal handling for vCPU signals has changed in the latest release
so switch to the new API.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-12-11 14:11:11 +00:00
Sebastien Boeuf
4c92f89f0f ci: Add OpenAPI validation
We need to validate that OpenAPI YAML definition is not broken by each
and every pull request. The easiest way is to rely on the Docker image
provided by OpenAPITools, as it allows us to validate the definition
with one simple command.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 12:53:31 +00:00
Sebastien Boeuf
93bd88e681 ci: Simplify travis.yml
Now that Cargo tests are being run on the worker node directly, we can
leave the bare minimum in Travis, which is taking care of the release
deployment.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 12:53:31 +00:00
Sebastien Boeuf
d42ef186a1 ci: Offload cargo tests to the worker node VM
Because the resources on the amount of worker nodes we can have access
to through Travis is limited, we offload the burden of running all tests
related to Cargo inside the Azure VM directly.

This will have the positive effect of stopping the build very early in
case something goes wrong during the Cargo testing.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-12-11 12:53:31 +00:00