Commit Graph

7855 Commits

Author SHA1 Message Date
Rob Bradford
8b78e1221e tests: Use newgrp to run unit tests
Rather than set filesystem permissions on the /dev/kvm device instead
use the kvm group added by installing qemu for running the unit tests.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-16 09:54:07 -07:00
Rob Bradford
f5a6e3c1ca build: Drop vendor directory from rustfmt command
We've not had a vendor directory for some time now.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-16 09:54:07 -07:00
Rob Bradford
d6e3b703ab tests: Rename virtiofsd build path
Adjust to reflect that it's QEMU being built here in preparation for
subsequent PRs that also want to build QEMU.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-16 09:54:07 -07:00
Rob Bradford
71154d8362 tests: Use "-f" on directory rm commands
When running the script from an interactive environment there are always
some files inside the git directory that rm prompts to delete so instead
pass "-f" to avoid that.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-16 09:54:07 -07:00
dependabot-preview[bot]
7245cf7c5e build(deps): bump rust-argon2 from 0.5.0 to 0.5.1
Bumps [rust-argon2](https://github.com/sru-systems/rust-argon2) from 0.5.0 to 0.5.1.
- [Release notes](https://github.com/sru-systems/rust-argon2/releases)
- [Changelog](https://github.com/sru-systems/rust-argon2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sru-systems/rust-argon2/compare/0.5.0...0.5.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-16 07:49:03 +00:00
dependabot-preview[bot]
7adb9d55ec build(deps): bump libc from 0.2.61 to 0.2.62
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.61 to 0.2.62.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.61...0.2.62)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-16 07:22:34 +00:00
dependabot-preview[bot]
760791abbe build(deps): bump openssl-sys from 0.9.48 to 0.9.49
Bumps [openssl-sys](https://github.com/sfackler/rust-openssl) from 0.9.48 to 0.9.49.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-sys-v0.9.48...openssl-sys-v0.9.49)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-16 06:54:47 +00:00
dependabot-preview[bot]
7b718f3029 build(deps): bump vmm-sys-util from 5f8c251 to 7222869
Bumps [vmm-sys-util](https://github.com/rust-vmm/vmm-sys-util) from `5f8c251` to `7222869`.
- [Release notes](https://github.com/rust-vmm/vmm-sys-util/releases)
- [Commits](5f8c251355...7222869ed3)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-16 06:54:23 +00:00
Rob Bradford
08ed88c8d1 tests: Remove potential sources of nested panics
panic()ing after a panic() has already been recovered by the credibility
test system (i.e. after an aver! has failed) results in an abort which
triggers SIGILL.

Adjust the SSH based commands to generate a Result<...,Error> which we
then either propagate through the test block. Or if the function is
directly being evaluated in an aver! macro call .unwrap_with_default()
(or .unwrap_or() in the case where the default would be wrong.)

See #182

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-15 11:37:42 -07:00
Sebastien Boeuf
ab6a8f19f0 tests: Fix virtio-fs with dax=off integration test
When virtio-fs is being tested through the integration tests, there is
one specific test where DAX and cache region are disabled. In this case
the virtiofsd daemon should be used with the correct option cache=none
instead of cache=always.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-15 19:24:46 +01:00
Rob Bradford
567eda45ec tests: Retrieve the bionic image from the Azure storage bucket
Avoid network delays by grabbing the bionic image from the local storage
bucket.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-15 18:57:53 +01:00
Johan Kuijpers
0affdd0df7 docs: Add networking HOWTO
Signed-off-by: Johan Kuijpers <johan.kuijpers@ericsson.com>
2019-08-15 17:52:09 +01:00
Sebastien Boeuf
658c076eb2 linters: Fix clippy issues
Latest clippy version complains about our existing code for the
following reasons:

- trait objects without an explicit `dyn` are deprecated
- `...` range patterns are deprecated
- lint `clippy::const_static_lifetime` has been renamed to
  `clippy::redundant_static_lifetimes`
- unnecessary `unsafe` block
- unneeded return statement

All these issues have been fixed through this patch, and rustfmt has
been run to cleanup potential formatting errors due to those changes.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-15 09:10:04 -07:00
Samuel Ortiz
c8364172a3 docs: Add debug I/O port HOWTO
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-08-15 16:06:54 +02:00
Samuel Ortiz
c52e276a5c vmm: Log debug ioport timestamps
We timestamp the VM creation time, and log the elapsed time between that
instant and the debug ioport events.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-08-15 16:06:54 +02:00
Samuel Ortiz
48a9300667 vmm: Log 0x80 IO port writes
The 0x80 IO port is typically used for BIOS debugging and testing on
bare metal x86 platforms.
We use that port and its dedicated 16 debug codes to time and track the
guest boot process.

Fixes #63

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-08-15 16:06:54 +02:00
dependabot-preview[bot]
10abfd4448 build(deps): bump vmm-sys-util from 54e256b to 5f8c251
Bumps [vmm-sys-util](https://github.com/rust-vmm/vmm-sys-util) from `54e256b` to `5f8c251`.
- [Release notes](https://github.com/rust-vmm/vmm-sys-util/releases)
- [Commits](54e256b2cb...5f8c251355)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-15 09:34:46 +00:00
dependabot-preview[bot]
6678cbfb79 build(deps): bump getrandom from 0.1.8 to 0.1.9
Bumps [getrandom](https://github.com/rust-random/getrandom) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/rust-random/getrandom/releases)
- [Changelog](https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/getrandom/compare/v0.1.8...v0.1.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-15 08:44:23 +00:00
Rob Bradford
513d2fdcf6 arch: x86_64: Update linux-loader crate
The linux-loader crate has been updated with a regnerated bootparams.rs
which has changed the API slightly. Update to the latest linux-loader
and adapt the code to reflect the changes:

* e820_map is renamed to e820_table (and all similar variables updated)
* e820entry is renamed to boot_e820_entry
* The E820 type constants are not no longer included

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-15 10:42:44 +02:00
Samuel Ortiz
76e3a30c31 pci: Simplify PciDevice trait
We do not use the on_device_sandboxed() and
register_device_capabilities() methods.

Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
2019-08-14 18:13:44 +02:00
dependabot-preview[bot]
0d53a5890d build(deps): bump rustc-demangle from 0.1.15 to 0.1.16
Bumps [rustc-demangle](https://github.com/alexcrichton/rustc-demangle) from 0.1.15 to 0.1.16.
- [Release notes](https://github.com/alexcrichton/rustc-demangle/releases)
- [Commits](https://github.com/alexcrichton/rustc-demangle/compare/0.1.15...0.1.16)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-14 08:12:28 +00:00
Sebastien Boeuf
b3c809a78c tests: Fix virtio-pmem
By introducing new kernel configuration related to DAX support, the
tests are not working as they were before. The format of the image
passed through virtio-pmem needs to be in proper raw format, otherwise
the virtio-pmem driver cannot complete its probing.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
af9a72eab6 tests: Add virtio-fs tests with dax=on and dax=off
The existing integration tests are extended to support both use cases
where dax=on and dax=off.

In order to support DAX, the kernel configuration needs to be updated to
include CONFIG_FS_DAX and CONFIG_ZONE_DEVICE.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
2e0508cdc6 vm-virtio: fs: Add DAX shared region support
This patch enables the vhost-user protocol features to let the slave
initiates some request towards the master (VMM). It also takes care of
receiving the requests from the slave and take appropriate actions based
on the request type.

The way the flow works now are as follow:
 - The VMM creates a region of memory that is made available to the
   guest by exposing it through the virtio-fs PCI BAR 2.
 - The virtio-fs device is created by the VMM, exposing some protocol
   features bits to virtiofsd, letting it know that it can send some
   request to the VMM through a dedicated socket.
 - On behalf of the guest driver asking for reading or writing a file,
   virtiofsd sends a request to the VMM, asking for a file descriptor to
   be mapped into the shared memory region at a specific offset.
 - The guest can directly read/write the file at the offset of the
   memory region.

This implementation is more performant than the one using exclusively
the virtqueues. With the virtqueues, the content of the file needs to be
copied to the queues every time the guest is asking to access it.
With the shared memory region, the virtqueues become the control plane
where the libfuse commands are sent to virtiofsd. The data plane is
literally the whole memory region which does not need any extra copy of
the file content. The only penalty is the first time a file is accessed,
it needs to be mapped into the VMM virtual address space.

Another interesting case where this solution will not perform as well as
expected is when a file is larger than the region itself. This means the
file needs to be mapped in several times, but more than that this means
it needs to be remapped every time it's being accessed.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
3c29c47783 vmm: Create shared memory region for virtio-fs
When the cache_size parameter from virtio-fs device is not empty, the
VMM creates a dedicated memory region where the shared files will be
memory mapped by the virtio-fs device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
74225ab5b3 config: Add option dax and cache_size to virtio-fs
In order to support the more performant version of virtio-fs, that is
the one relying on a shared memory region between host and guest, we
introduce two new parameters to the --fs device.

The "dax" parameter allows the user to choose if he wants to use the
shared memory region with virtio-fs. By default, this parameter is "on".

The "cache_size" parameter allows the user to specify the amount of
memory that should be shared between host and guest. By default, the
value of this parameter is 8Gib as advised by virtio-fs maintainers.

Note that dax=off and cache_size are incompatible.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
f30ba069b7 vm-virtio: Allocate shared memory regions on dedicated BAR
In the context of shared memory regions, they could not be present for
most of the virtio devices. For this reason, we prefer dedicate a BAR
for the shared memory regions.

Another reason is that memory regions, if there are several, can be
allocated all at once as a contiguous region, which then can be used as
its own BAR. It would be more complicated to try to allocate the BAR 0
holding the regular information about the virtio-pci device along with
the shared memory regions.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
e0fda0611c vm-virtio: Remove virtio-pci dependency from VirtioDevice
This patch cleans up the VirtioDevice trait. Since some function are PCI
specific and since they are not even used, it makes sense to remove them
from the trait definition.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
e2b38cc050 vm-virtio: Extend VirtioDevice trait to retrieve shared memory regions
Based on the newly added SharedMemoryConfig capability to the virtio
specification, and based on the fact that it is not tied to the type of
transport (pci or mmio), we can create as part of the VirtioDevice trait
a new method that will provide the shared memory regions associated with
the device.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
d97079d793 vm-virtio: Update VirtioPciCap and introduce VirtioPciCap64
Based on the latest version of the virtio specification, the structure
virtio_pci_cap has been updated and a new structure virtio_pci_cap64 has
been introduced.

virtio_pci_cap now includes a field "id" that does not modify the
existing structure size since there was a 3 bytes reserved field
already there. The id is used in the context of shared memory regions
which need to be identified since there could be more than one of this
kind of capability.

virtio_pci_cap64 is a new structure that includes virtio_pci_cap and
extends it to allow 64 bits offsets and 64 bits region length. This is
used in the context of shared memory regions capability, as we might need
to describe regions of 4G or more, that could be placed at a 4G offset
or more in the associated BAR.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
d180deb679 vm-virtio: pci: Fix PCI capability length
The length of the PCI capability as it is being calculated by the guest
was not accurate since it was not including the implicit 2 bytes offset.

The reason for this offset is that the structure itself does not contain
the capability ID (1 byte) and the next capability pointer (1 byte), but
the structure exposed through PCI config space does include those bytes.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
c6feb03dc0 vhost_rs: Allow MasterReqHandler to reply when needed
No matter if the communication is coming from the master or the slave,
it should always reply with an ack if the message header specifies that
it expects a reply.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
ef2e8b6bc2 tests: Update virtio-fs mount command
Because the way to mount virtio-fs filesystem changed with newest
kernel, we need to update the mount command in our integration tests.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
3645bf8d0f tests: Build virtiofsd from specific branch
In order to be able to use the latest version from virtiofsd binary, the
integration tests will now build it directly from a branch located on
sboeuf's QEMU fork. The same way the kernel is hosted on sboeuf's linux
kernel fork, this allows to update the version of the virtiofs daemon
based on latest patches from virtio-fs maintainers.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
Sebastien Boeuf
021e8d9e13 tests: Move to new kernel 5.3-rc3
The last kernel we were using included some manual porting of the
virtio-pmem and virtio-fs patches. By moving to 5.3-rc3, we are now
closer to upstream since the virtio-pmem patches are part of the linux
kernel now. Additionally, this includes the latest patches from
virtio-fs maintainers, which works with the latest version of virtiofsd.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-13 13:57:53 +02:00
dependabot-preview[bot]
4b3fd906f7 build(deps): bump cc from 1.0.38 to 1.0.40
Bumps [cc](https://github.com/alexcrichton/cc-rs) from 1.0.38 to 1.0.40.
- [Release notes](https://github.com/alexcrichton/cc-rs/releases)
- [Commits](https://github.com/alexcrichton/cc-rs/compare/1.0.38...1.0.40)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-13 07:43:10 +00:00
dependabot-preview[bot]
bc5b72ff73 build(deps): bump libc from 0.2.60 to 0.2.61
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.60 to 0.2.61.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.60...0.2.61)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-13 07:21:11 +00:00
Rob Bradford
6c06420a11 vm-virtio: net: Fix out-of-range slice panic when under load
The numbr of bytes read was being incorrectly increased by the potential
length of the end of the sliced data rather than the number of bytes
that was in the range. This caused a panic when the the network was
under load by using iperf.

It's important to note that in the Firecracker code base the function
that read_slice() returns the number of bytes read which is used to
increment this counter. The VM memory version however only returns the
empty unit "()".

Fixes: #166

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-12 15:35:11 +01:00
dependabot-preview[bot]
b608671031 build(deps): bump syn from 0.15.43 to 0.15.44
Bumps [syn](https://github.com/dtolnay/syn) from 0.15.43 to 0.15.44.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/0.15.43...0.15.44)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-12 11:07:32 +00:00
dependabot-preview[bot]
97c964891c build(deps): bump arc-swap from 0.4.1 to 0.4.2
Bumps [arc-swap](https://github.com/vorner/arc-swap) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/vorner/arc-swap/releases)
- [Changelog](https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vorner/arc-swap/compare/v0.4.1...v0.4.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-12 11:07:05 +00:00
fazlamehrab
df5058ec0a vm-virtio: Implement console size config feature
One of the features of the virtio console device is its size can be
configured and updated. Our first iteration of the console device
implementation is lack of this feature. As a result, it had a
default fixed size which could not be changed. This commit implements
the console config feature and lets us change the console size from
the vmm side.

During the activation of the device, vmm reads the current terminal
size, sets the console configuration accordinly, and lets the driver
know about this configuration by sending an interrupt. Later, if
someone changes the terminal size, the vmm detects the corresponding
event, updates the configuration, and sends interrupt as before. As a
result, the console device driver, in the guest, updates the console
size.

Signed-off-by: A K M Fazla Mehrab <fazla.mehrab.akm@intel.com>
2019-08-09 13:55:43 -07:00
Rob Bradford
d9a355f85a vmm: Add new "null" serial/console output mode
Poor performance was observed when booting kernels with "console=ttyS0"
and the serial port disabled.

This change introduces a "null" console output mode and makes it the
default for the serial console. In this case the serial port
is advertised as per other output modes but there is no input and any
output is dropped.

Fixes: #163

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-09 09:04:48 -07:00
Rob Bradford
f910476dd7 vmm: Only send stdin input to serial/console if it can handle it
Do not send the contents of stdin to the serial or console device if
they're not in tty mode.

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-09 09:04:48 -07:00
Sebastien Boeuf
aa44726658 vm-virtio: Don't trigger an MSI-X interrupt if not enabled
Relying on the newly added MSI-X helper, the interrupt callback checks
the interrupts are enabled on the device before to try triggering the
interrupt.

Fixes #156

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-08 17:38:47 +01:00
Sebastien Boeuf
c0e2bbb23f pci: Add MSI-X helper to check if interrupts are enabled
In order to check if device's interrupts are enabled, this patch adds
a helper function to the MsixConfig structure so that at any point in
time we can check if an interrupt should be delivered or not.

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-08 17:38:47 +01:00
dependabot-preview[bot]
4c9759e4fc build(deps): bump syn from 0.15.42 to 0.15.43
Bumps [syn](https://github.com/dtolnay/syn) from 0.15.42 to 0.15.43.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/0.15.42...0.15.43)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-08 07:33:38 +00:00
dependabot-preview[bot]
b7ed71b012 build(deps): bump vmm-sys-util from 9014b7a to 54e256b
Bumps [vmm-sys-util](https://github.com/rust-vmm/vmm-sys-util) from `9014b7a` to `54e256b`.
- [Release notes](https://github.com/rust-vmm/vmm-sys-util/releases)
- [Commits](9014b7ab94...54e256b2cb)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-08 07:13:38 +00:00
dependabot-preview[bot]
8fcaf91d3b build(deps): bump redox_users from 0.3.0 to 0.3.1
Bumps redox_users from 0.3.0 to 0.3.1.

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2019-08-08 07:11:11 +00:00
Sebastien Boeuf
87195c9ccc pci: Fix vector control read/write from/to MSI-X table
The vector control offset is at the 4th byte of each MSI-X table entry.
For that reason, it is located at 0xc, and not 0x10 as implemented.

This commit fixes the current MSI-X code, allowing proper reading and
writing of each vector control register in the MSI-X table.

Fixes #156

Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
2019-08-08 08:10:18 +01:00
Rob Bradford
fca911e5f3 main: Add logging support controlled by command line
This makes the log macros (error!, warn!, info!, etc) in the code work.
It currently defaults to showing only error! messages, but by passing an
increasing number of "-v"s on the command line the verbosity can be
increased.

By default log output goes onto stderr but it can also be sent to a
file.

Fixes: #121

Signed-off-by: Rob Bradford <robert.bradford@intel.com>
2019-08-07 15:48:17 +01:00