Commit Graph

978 Commits

Author SHA1 Message Date
Sebastien Boeuf
f1c7f0c0b8 ci: Add integration test for vhost_user_fs daemon
In order to validate the new virtio-fs daemon written in Rust is
behaving correctly, a new integration test has been added. Important to
note that for now, only a test with cache=none and dax=off can be added
since the daemon does not support shared memory region yet.

The long term goal being to replace virtiofsd with vhost_user_daemon
once it will reach parity regarding the supported features.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
50b0e58c88 vhost_user_fs: Allow specific shared directory to be specified
Because the vhost_user_backend crate needs some changes to support
moving the process to a different mount namespace and perform a pivot
root, it is not possible to change '/' to the given shared directory.

This commit, as a temporary measure, let the code point at the given
shared directory.

The long term solution is to perform the mount namespace change and the
pivot root as this will provide greater security.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
ba17758ac0 src: Add vhost-user-fs daemon
This patch implements a vhost-user-fs daemon based on Rust. It only
supports communicating through the virtqueues. The support for the
shared memory region associated with DAX will be added later.

It relies on all the code copied over from the crosvm repository, based
on the commit 961461350c0b6824e5f20655031bf6c6bf6b7c30.

It also relies on the vhost_user_backend crate, limiting the amount of
code needed to get this daemon up and running.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
5c128023da vhost_user_fs: Add Server structure to consume FileSystem implementation
Add a Server type that links the FUSE protocol with the virtio
transport. It parses messages sent on the virtio queue and then
calls the appropriate method of the Filesystem trait.

This code has been ported over from crosvm commit
961461350c0b6824e5f20655031bf6c6bf6b7c30.

One small modification has been applied to the original code. Because
cloud-hypervisor didn't have the macro used by crosvm, the match
statement in the function handle_message() has been updated.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
e1fccc3615 vhost_user_fs: Add virtio descriptor helper traits
Introduce helpers to split a virtio descriptor into its readable part on
one side, and into its writable part on the other side. This is useful
to separate the request from the reply.

This code has been ported over from crosvm commit
961461350c0b6824e5f20655031bf6c6bf6b7c30.

Two important modifications have been applied to the original code:
- GuestMemory is replaced by GuestMemoryMmap from the vm-memory crate,
  which comes with different ways of accessing the memory regions.
- VolatileSlice has different methods, which means the code has been
  updated accordingly.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
5f7935f8e0 vhost_user_fs: Add file traits to handle writing volatile memory
The vm_memory implementation for VolatileSlice is able to read and write
to a source or destination which implements a Read or Write trait.

Unfortunately, this is not enough for this specific use case as we need
to be able to write to a file at a specific offset, which is not
provided by the Read or Write trait.

This code has been ported over from crosvm commit
961461350c0b6824e5f20655031bf6c6bf6b7c30.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
e33ccb0c95 vhost_user_fs: Implement FileSystem trait for Passthrough
Add a "passthrough" file system implementation that just forwards its
requests to the appropriate system call.

This code has been ported over from crosvm commit
961461350c0b6824e5f20655031bf6c6bf6b7c30.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
1b76c0a183 vhost_user_fs: Add FileSystem trait
Add the `Filesystem` trait, which is the main interface between the
transport and the actual file system implementation.

This code has been ported over from crosvm commit
961461350c0b6824e5f20655031bf6c6bf6b7c30.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
1e65bda0a7 vhost_user_fs: Add multikey module
The multikey module provides a BTreeMap implementation that can use one
of 2 different kinds of keys to look up a value. This is needed by the
virtio-fs server since it needs to be able to look up keys either by
u64 or by a (ino_t, dev_t) pair.

This code has been ported over from crosvm commit
961461350c0b6824e5f20655031bf6c6bf6b7c30.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
cd1684bd2e vhost_user_fs: Add FUSE definitions
To be able to deal with FUSE requests, this commit introduces FUSE
definitions, copied over from the crosvm commit
961461350c0b6824e5f20655031bf6c6bf6b7c30.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
03361a6c29 vhost_user_fs: Add new crate
This new crate will be dedicated to vhost_user_fs specific code that can
be used as a library from the vhost-user-fs daemon.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Sebastien Boeuf
8845326aa2 vm-virtio: Introduce DescriptorChain iterator
In order to iterate over a chain of descriptor chains, this code has
been ported over from crosvm, based on the commit
961461350c0b6824e5f20655031bf6c6bf6b7c30.

The main modification compared to the original code is the way the
sorting between readable and writable descriptors happens.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 22:17:47 +01:00
Rob Bradford
3d6b5459ef ci: Make the integration test binary run with same caps
By giving the same caps to both cloud-hypervisor and the test binary, we
can access information under /proc related to the VM PID.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-22 15:28:10 +00:00
Rob Bradford
ac118c9924 ci: Parse the smaps file with Rust
Instead of using bash and awk, using Rust allows us to retrieve
information about a VM process with the right permissions as we are not
forced to spawn a new child process.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-22 15:28:10 +00:00
Sebastien Boeuf
bdb7bcdbe3 ci: Add integration test for mergeable memory
The test validates that when the mergeable option is enabled, the
resulting PSS for two instances of cloud-hypervisor is lower than two
instances not using the mergeable flag.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 15:28:10 +00:00
Sebastien Boeuf
f979380620 vmm: Mark guest persistent memory pages as mergeable
In case the VM is started with the flag "--pmem mergeable=on", it means
the user expects the guest persistent memory pages to be marked as
mergeable. This commit relies on the madvise(MADV_MERGEABLE) system call
to inform the host kernel about these pages.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 15:28:10 +00:00
Sebastien Boeuf
0f9afc3017 vmm: Add mergeable=on|off option to --pmem flag
In order to let the user indicate if the persistent memory pages should
be marked as mergeable or not, a new option is being introduced.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 15:28:10 +00:00
Sebastien Boeuf
e4e8062dda vmm: Mark guest RAM pages as mergeable
In case the VM is started with the flag "--memory mergeable=on", it
means the user expects the guest RAM pages to be marked as mergeable.
This commit relies on the madvise(MADV_MERGEABLE) system call to inform
the host kernel about these pages.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 15:28:10 +00:00
Sebastien Boeuf
880f62bab8 vmm: Add mergeable=on|off option to --memory flag
In order to let the user indicate if the guest RAM pages should be
marked as mergeable or not, a new option is being introduced.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-22 15:28:10 +00:00
Rob Bradford
0213177027 Jenkinsfile: Add timeout for build
In order to conserve CI resources limit build execution to one hour.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-22 06:52:21 -08:00
Rob Bradford
d642060378 Jenkinsfile: Switch to pipeline (declarative format)
Switch the Jenkinsfile from the scripted format over to the declarative
format.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-22 06:52:21 -08:00
Jose Carlos Venegas Munoz
1d852e9ce5 vmm: Provide vmm version to start_vmm_thread
When vmm.ping give a response, we expect get the version from
the VMM not the vmm create

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-11-21 15:04:11 -08:00
Jose Carlos Venegas Munoz
a518651402 http: api: implement vmm.ping
vmm.ping will help to check if http API server is up and
running.

This also removes the vmm.info endpoint.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-11-21 15:04:11 -08:00
Rob Bradford
348a1bc30e vmm: cpu: Allocate I/O port for the CPU manager
The CPU manager uses an I/O port and to prevent potential clashes with
assignment for PCI devices ensure that it is allocated by the allocator.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
07cdb37dda vmm: cpu & acpi: Query CPU manager for CPU status
Rather than hardcode the CPU status for all the CPUs instead query from
the CPU manager via the I/O port that is is on via the ACPI tables.

Each CPU device has a _STA method that calls into the CSTA method which
reads and writes the I/O ports via the PRST field which exposes the I/O
port through and OpRegion.

As we only support boot CPUS report that all the CPUs are enabled for
now.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
5faf8b756c vmm: acpi: Add an _MAT for the CPU devices containing a LAPIC
The Linux kernel expects all CPUs, whether they be enabled or disabled
to have an _MAT entry containing the LAPIC details for this CPU with the
enabled bit set to 1 (in the flags.)

In the MADT table the same bit is used to determine if the CPU is
present at boot vs available later.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
e51ebe045f acpi_tables: Add support for Buffer objects
These are necessary to populate "_MAT" entries for CPU devices.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
1da0ff395d vmm: cpu: Add the CpuManager onto the IO bus
This allows the kernel (via ACPI based controls) to query and control
the CPU state.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
39a1b8f4db acpi_tables: aml: Add support for calling methods
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
d5bb0781e4 acpi_tables: aml: Add support for while loops
Which also requires adding support for addition.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
89f0db2173 acpi_tables: aml: Add support for device notification
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
d6696e1bdd acpi_tables: aml: Add support for mutexes
And add support for operations on them.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
3d70ce9ad1 acpi_tables: aml: Add "if" with local variables and arguments
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
d06623fb97 acpi_tables: aml: Add support for OpRegion
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
93ee6f5e62 acpi_tables: aml: Add support for field definitions
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
4b5ce23d97 acpi_tables: Add PkgLength variant that does not include itself
This is necessary as adding support for NamedFields requires a PkgLength
calculation that does not include the length itself.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
50c8335d3d vmm: device_manager: Expose the SystemAllocator
This allows other code to allocate I/O ports for use on the (already)
exposed IO bus.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Rob Bradford
1ac1231292 vmm: Encase CpuManager within an Arc<Mutex<>>
This is necessary to be able to add the CpuManager onto the IO bus.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 09:17:15 -08:00
Wu Zongyong
d7dc1a9226 pci: don't cleanup msi/msix interrupts repeatedly
We disabled msi/msix twice inside Drop trait for VfioPciDevice,
which resulted in error message "Could not disable MSI-X". Eliminating
this error by check whether the msi/msix capability is enabled.

Signed-off-by: Wu Zongyong <wuzongyong@linux.alibaba.com>
CC: Liu Jiang <gerry@linux.alibaba.com>
2019-11-21 06:38:36 -08:00
Wu Zongyong
66fde245b3 vfio: use correct flags to disable interrupts
The comments of vfio kernel module said that individual subindex
interrupts can be disabled using the -1 value for DATA_EVENTFD or
the index can be disabled as a whole with:
    flags = (DATA_NONE|ACTION_TRIGGER), count = 0.

Signed-off-by: Wu Zongyong <wuzongyong@linux.alibaba.com>
CC: Liu Jiang <gerry@linux.alibaba.com>
2019-11-21 06:38:36 -08:00
Rob Bradford
8ec89bc884 misc: Update to new repository locations
Update all references to the new repository locations. Many of these will
redirect however the one used for the hypervisor-fw binary does not so
this is required to allow the builds to pass.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-11-21 11:39:11 +00:00
Sebastien Boeuf
64305dab16 docs: device_model: Fix formatting error
Fix the summary table format.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-19 15:45:52 -08:00
Sebastien Boeuf
b55d75ea62 docs: Add device model
This commit introduces a dedicated document describing the device model
supported by cloud-hypervisor VMM.

It needs to be updated anytime a new device will be added in the future.

Fixes #437

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-11-19 14:10:36 -08:00
Samuel Ortiz
f0e618431d vmm: device_manager: Use consistent naming when adding devices
When adding devices to the guest, and populating the device model, we
should prefix the routines with add_. When we're just creating the
device objects but not yet adding them we use make_.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-11-19 13:36:21 -08:00
Samuel Ortiz
a2ee681665 vmm: device_manager: Add an MMIO devices creation routine
In order to reduce the DeviceManager's new() complexity, we can move the
MMIO devices creation code into its own routine.

Fixes: #441

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-11-19 13:36:21 -08:00
Samuel Ortiz
79b8f8e477 vmm: device_manager: Add a PCI devices creation routine
In order to reduce the DeviceManager's new() complexity, we can move the
PCI devices creation code into its own routine.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-11-19 13:36:21 -08:00
Samuel Ortiz
5087f633f6 vmm: device_manager: Add an IOAPIC creation routine
In order to reduce the DeviceManager's new() complexity, we can move the
ACPI device creation code into its own routine.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-11-19 13:36:21 -08:00
Samuel Ortiz
ce1765c8af vmm: device_manager: Add an ACPI device creation routine
In order to reduce the DeviceManager's new() complexity, we can move the
ACPI device creation code into its own routine.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-11-19 13:36:21 -08:00
Samuel Ortiz
cfca2759fc vmm: device_manager: Add a legacy devices creation routine
In order to reduce the DeviceManager's new() complexity, we can move the
legacy devices creation code into its own routine.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-11-19 13:36:21 -08:00
Samuel Ortiz
4b469b98cf vmm: device_manager: Add a console creation routine
In order to reduce the DeviceManager's new() complexity, we can move the
console creation code into its own routine.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-11-19 13:36:21 -08:00