Commit Graph

1447 Commits

Author SHA1 Message Date
Sergio Lopez
07cc73bddc vhost_user_fs: add a flag to disable extended attributes
Extended attributes (xattr) support has a huge impact on write
performance. The reason for this is that, if enabled, FUSE sends a
setxattr request after each write operation, and due to the inode
locking inside the kernel during said request, the ability to execute
the operations in parallel becomes heavily limited.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-13 15:20:34 +00:00
Sergio Lopez
710520e9a1 vhost_user_fs: Process requests in parallel with a thread pool
This change enables vhost_user_fs to process multiple requests in
parallel by scheduling them into a ThreadPool (from the Futures
crate).

Parallelism on a single file is limited by the nature of the operation
executed on it. A recent commit replaced the Mutex that protects the
File within HandleData with a RwLock, to allow some operations (at
this moment, only "read" and "write") to proceed in parallel by
acquiring a read lock.

A more complex approach was also implemented [1], involving
instrumentation through vhost_user_backend to be able to serialize
completions, reducing the pressure on the vring RwLock. This strategy
improved the performance on some corner cases, while making it worse
on other, more common ones. This fact, in addition to it requiring
wider changes through the source code, prompted me to drop it in favor
of this one.

[1] https://github.com/slp/cloud-hypervisor/tree/vuf_async

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-13 15:20:34 +00:00
Sergio Lopez
90309b5106 vm-virtio: queue: Add methods to switch a descriptor context
"DescriptorChain"s are tied to the lifetime of the referenced
GuestMemoryMmap object (for good reasons), but sometimes (i.e., when
processing descriptors from different contexts) we may need to switch
them to point a different GuestMemoryMmap.

Here we introduce the structure DescriptorHead, which holds the data
needed to rebuild a DescriptorChain, the method "get_head" which
returns the DescriptorHead for a DescriptorChain, and the method
"new_from_head", which allows to create a new DescriptorChain with a
DescriptorHead and a new reference to a GuestMemoryMmap.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-13 15:20:34 +00:00
Sergio Lopez
2294c2d2c9 Add .rustfmt.toml to the project
For rustfmt to accept modern syntax like "async move", a .rustfmt.toml
with "edition = 2018" needs to be created.

https://github.com/rust-lang/rustfmt/issues/3149

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-13 15:20:34 +00:00
Sergio Lopez
48c4885b47 vhost_user_fs: replace HandleData's File Mutex with RwLock
Replace HandleData's File Mutex with a RwLock to have more granularity
on the lock. This allows operations on the same File that are safe to
be run in parallel (at this moment, read and write), to acquire a read
lock to avoid waiting on each other.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-13 15:20:34 +00:00
Samuel Ortiz
134e64c612 arch, qcow: Fix 1.42.0 clippy warnings
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-03-13 12:26:15 +00:00
Samuel Ortiz
6ea85ca86c resources: Dockerfile: Update Rust toolchain
To the latest stable version: 1.42.0

Fixes: #909

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-03-13 12:26:15 +00:00
Rob Bradford
4579afa091 vmm: For --disk error if socket and path is specified
This is an error as the path should be specfied by the unmanaged
backend.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-13 11:41:52 +00:00
Rob Bradford
7e599b4450 vmm: Make disk path optional
When using "--disk" with a vhost socket and not using self spawning then
it is not necessary or helpful to specify the path.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-13 11:41:52 +00:00
Samuel Ortiz
477d924528 github: Build from a rust toolchain matrix
So that we can run our builds against several toolchains (beta, stable,
nightly).

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-03-13 11:41:34 +00:00
Rob Bradford
4f2469e054 main: Remove "--vhost-user-net"
This option was superseded by using "--net" with "vhost_user=true". This
option wasn't being parsed any more but was left over.

Fixes: #806

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-13 11:00:43 +00:00
Sebastien Boeuf
8d785bbd5f pci: Fix the PciBus using HashMap instead of Vec
By using a Vec to hold the list of devices on the PciBus, there's a
problem when we use unplug. Indeed, the vector of devices gets reduced
and if the unplugged device was not the last one from the list, every
other device after this one is shifted on the bus.

To solve this problem, a HashMap is used. This allows to keep track of
the exact place where each device stands on the bus.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-13 10:54:34 +01:00
dependabot-preview[bot]
04f2ccd16d build(deps): bump ryu from 1.0.2 to 1.0.3
Bumps [ryu](https://github.com/dtolnay/ryu) from 1.0.2 to 1.0.3.
- [Release notes](https://github.com/dtolnay/ryu/releases)
- [Commits](https://github.com/dtolnay/ryu/compare/1.0.2...1.0.3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-13 08:44:41 +00:00
dependabot-preview[bot]
02265bb589 build(deps): bump regex-syntax from 0.6.16 to 0.6.17
Bumps [regex-syntax](https://github.com/rust-lang/regex) from 0.6.16 to 0.6.17.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-13 08:14:13 +00:00
Jose Carlos Venegas Munoz
40b38a4222 openapi: Make desired_ram int64 format
The option desired_ram is in byte, make larger the amount of memory to
add.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2020-03-12 23:17:56 +01:00
Rob Bradford
ca3b39c0be bin: Fix wrapping in help strings
Some of the help strings had extra newlines in them or otherwise strange
wrapping. The strings were rewrapped with the nightly version of rustfmt
that supports string formatting.

Fixes: #899

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-12 18:03:18 +00:00
Rob Bradford
ee1ba56cfe build: Use "wrap_help" feature for clap
This prevents the output being wrapped at 120 characters and giving
strange results.

Fixes: #899

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-12 18:03:18 +00:00
Sergio Lopez
3957d1ee27 vhost_user_backend: call get_used_event from needs_notification
This change, combined with the compiler hint to inline get_used_event,
shortens the window between the memory read and the actual check by
calling get_used_event from needs_notification.

Without it, when putting enough pressure on the vring, it's possible
that a notification is wrongly omitted, causing the queue to stall.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-12 14:34:21 +00:00
Sergio Lopez
536323d9fb vm-virtio: queue: hint that get_used_event should be inlined
get_used_event is used from vhost_user_backend:needs_notification to
check whether an interrupt must be sent to the guest to notify there
are new items in the queue. Shorten the update window by asking the
the compiler to inline this method, so a write won't slip between the
read of the memory contents and the actual check.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-12 14:34:21 +00:00
Sergio Lopez
401e1d2489 vm-virtio: queue: fix a barrier comment at update_avail_event
The barrier had a comment coming from other context. Adjust it to be
relevant to its own context.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-12 14:34:21 +00:00
Sergio Lopez
e0bdfe826e vm-virtio: queue: add a missing memory barrier in get_used_event
Add a missing memory barrier in get_used_event to make sure we see the
last value written by the guest.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2020-03-12 14:34:21 +00:00
Sebastien Boeuf
df2570a472 resources: Simplify kernel config filename
Since we only keep one single version of the kernel config file in our
repository, there is no reason to keep the filename complex.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-12 14:07:11 +01:00
Sebastien Boeuf
9ab648b8f7 resources: Enable VIRTIO_MEM support
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-12 14:07:11 +01:00
Sebastien Boeuf
03398532dc ci: Bump to kernel 5.6-rc4
The kernel version is updated from 5.5-rc1 to 5.6-rc4, including the
updated kernel config file.

The kernel branch contains virtio-fs, virtio-iommu and virtio-mem
patches that are not upstream yet. It also contains one fix for
virtio-vsock which will be merged upstream in the next release.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-12 14:07:11 +01:00
Rob Bradford
abccf766ce tests: Use ch-remote to add/remove devices in test_vfio
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-12 12:56:10 +01:00
Rob Bradford
5c3ce9dd00 tests: Extend ch-remote helper to support optional single argument
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-12 12:56:10 +01:00
Rob Bradford
9a7d9c9465 ch-remote: Support removing VFIO devices
Add a "remove-device" command that allows removing VFIO devices from the
VM after boot.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-12 12:56:10 +01:00
Rob Bradford
0d53ba4395 ch-remote: Support adding VFIO devices
Add an "add-device" command that allows adding VFIO devices to the VM
after boot.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-12 12:56:10 +01:00
Rob Bradford
babefbd9bf main: Remove spurious second help line for "--device"
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-12 12:56:10 +01:00
Samuel Ortiz
63c5d09537 github: Trigger the build job on PRs
And not on pushes only.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-03-12 11:27:43 +01:00
Samuel Ortiz
8cbb6d0faf github: Replace Travis CI with github actions
We define 1 workflow with 2 jobs: One for the regular build and another
one for creating a release and uploading the corresponding assets.
They both run conditionally, depending on the gihtub event that triggered
the action.

Fixes: #825

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-03-12 09:18:01 +01:00
Sebastien Boeuf
efba48dddb vmm: Don't put a VFIO device behind the vIOMMU by default
With some of the factorization that happened to be able to support VFIO
hotplug, one mistake was made. In case a vIOMMU is created through a
virtio-iommu device, and no matter the "iommu" option value from the
VFIO device parameter, the VFIO device was always placed behind the
virtual IOMMU.

This commit fixes this wrong behavior by making sure the device
configuration is taken into account to decide if it should be attached
or not to the virtual IOMMU.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 19:50:31 +01:00
Sebastien Boeuf
34412c9b41 vmm: Add id option to VFIO hotplug
Add a new id option to the VFIO hotplug command so that it matches the
VFIO coldplug semantic.

This is done by refactoring the existing code for VFIO hotplug, where
VmAddDeviceData structure is replaced by DeviceConfig. This structure is
the one used whenever a VFIO device is coldplugged, which is why it
makes sense to reuse it for the hotplug codepath.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 19:50:31 +01:00
Samuel Ortiz
18dc916380 vmm: Switch to the micro-http package
It's been extracted from the Firecracker code base.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2020-03-11 17:38:01 +01:00
Sebastien Boeuf
9023444ad3 vmm: Add id field to --device through CLI
Add the ability to specify the "id" associated with a device, by adding
an extra option to the parameter --device.

This new option is not mandatory, and by default, the VMM will take care
of finding a unique identifier.

If the identifier provided by the user through this new option is not
unique, an error will be thrown and the VM won't be started.

Fixes #881

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 13:10:57 +00:00
Sebastien Boeuf
f4a956a60a vmm: Remove 32 bits MMIO range from correct address space
The 32 bits MMIO address space is handled separately from the 64 bits
one. For this reason, we need to invoke the appropriate freeing function
to remove a range from this address space.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 13:10:30 +00:00
Sebastien Boeuf
432eb5b70a vmm: Free PCI BARs when unplugging PCI device
Now that PciDevice trait has a dedicated function to remove the bars,
the DeviceManager can invoke this function whenever a PCI device is
unplugged from the VM.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 13:10:30 +00:00
Sebastien Boeuf
f0dff8b58c vfio: pci: Remove KVM user memory region when cleaning up
Whenever a VfioPciDevice is dropped, the regions previously mapped are
being unmapped. But there's also a need for removing the region from a
KVM perspective.

This commit extends the existing unmap_mmio_regions() function to take
care of removing the KVM region at the same time the regions are being
unmapped.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 13:10:30 +00:00
Sebastien Boeuf
34d1f435f4 vfio: pci: Implement free_bars() from the PciDevice trait
In order to provide the tools for a complete cleanup whenever a VFIO PCI
device is removed from the VM, the VfioPciDevice implements free_bars()
method from PciDevice trait. This will take care of removing the IO and
MMIO ranges previously reserved through the vm-allocator.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 13:10:30 +00:00
Sebastien Boeuf
b8e1cf2d4e vm-allocator: Add new function to free 32 bits MMIO address space
The allocator already had functions to allocate and free both IO and 64
bits MMIO address spaces, but it only had an allocating function for 32
bits MMIO address space.

With this new function, it will be now possible to remove cleanly some
ranges from the 32 bits MMIO address space.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 13:10:30 +00:00
Sebastien Boeuf
f3dc245c4f pci: Extend PciDevice trait with new free_bars() method
The point of this new method is to let the caller decide when the
implementation of the PciDevice should free the BARs previously
allocated through the other method allocate_bars().

This provides a way to perform proper cleanup for any PCI device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-11 13:10:30 +00:00
Rob Bradford
911a2d61d7 tests: Use ch-remote to resize the VM
Remove manually constructed API request in the integration tests.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-11 08:22:09 +01:00
Rob Bradford
21160f7490 ch-remote: Add "resize" command
This command lets you change the number of vCPUs and RAM that the VM
has.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-11 08:22:09 +01:00
Rob Bradford
bb2d04b39d ch-remote: Add support for sending a request body
Support sending a request body this will usually be JSON encoded data
representing the details of the request.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-11 08:22:09 +01:00
Rob Bradford
bde4f735ab ch-remote: Refactor HTTP response handling
Extract HTTP response handling into its own function.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2020-03-11 08:22:09 +01:00
dependabot-preview[bot]
6ed23bb3ab build(deps): bump micro_http from 9bbde4f to 6b3e5f0
Bumps [micro_http](https://github.com/firecracker-microvm/firecracker) from `9bbde4f` to `6b3e5f0`.
- [Release notes](https://github.com/firecracker-microvm/firecracker/releases)
- [Commits](9bbde4fb6a...6b3e5f0a18)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-11 07:15:40 +00:00
dependabot-preview[bot]
5edd812792 build(deps): bump backtrace-sys from 0.1.33 to 0.1.34
Bumps [backtrace-sys](https://github.com/alexcrichton/backtrace-rs) from 0.1.33 to 0.1.34.
- [Release notes](https://github.com/alexcrichton/backtrace-rs/releases)
- [Commits](https://github.com/alexcrichton/backtrace-rs/compare/backtrace-sys-0.1.33...backtrace-sys-0.1.34)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-03-11 07:15:28 +00:00
Sebastien Boeuf
f7277140bb ci: Add integration test for VFIO hot-unplug
Extend the existing test_vfio by unplugging the previously hotplugged
device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-10 17:05:06 +00:00
Sebastien Boeuf
b50cbe5064 pci: Give PCI device ID back when removing a device
Upon removal of a PCI device, make sure we don't hold onto the device ID
as it could be reused for another device later.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-10 17:05:06 +00:00
Sebastien Boeuf
df71aaee3f pci: Make the device ID allocation smarter
In order to handle the case where devices are very often plugged and
unplugged from a VM, we need to handle the PCI device ID allocation
better.

Any PCI device could be removed, which means we cannot simply rely on
the vector size to give the next available PCI device ID.

That's why this patch stores in memory the information about the 32
slots availability. Based on this information, whenever a new slot is
needed, the code can correctly provide an available ID, or simply return
an error because all slots are taken.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2020-03-10 17:05:06 +00:00