The integration test test_memory_mergeable_on has been fairly unstable
for quite some time now. Because it can take some time for the VM to be
spawned and to be able to perform a correct measure of the PSS, this
commit simply increases the time before such measure is done.
This should return more accurate PSS results, which should help
stabilize the test.
Fixes#781
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that vhost_user_fs rust daemon supports virtiofs's dax mode, this adds
the two dax tests accordingly.
Signed-off-by: Liu Bo <bo.liu@linux.alibaba.com>
Add a build-script to propagate the git commit hash to other crates at
compile time through environment variables, and display the hash along
with the '--version' option.
Fixes#729
Signed-off-by: Bo Chen <chen.bo@intel.com>
test_vfio has been failing consistently on the CI so mark it with
a "#[ignore]" and then forceably build it again but ignore the build
result.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Check the rust formatting rather than just reformatting code on the CI
agent.
Also fix a formatting error that slipped in whilst the cargo fmt check
was not working correctly.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
A previous version of this change attempted to avoid panicking by not
using .expect() when handling an error when attempting to write to the
log file. Unfortunately the macro eprintln!() that was used to replace
the .expect() also has the behaviour of panicking if stderr cannot be
used. Instead swallow the error completely as if writing to the log has
failed at logging time it is almost certainly the case that any message
about the log would also not be seen.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Since the vhost-user-blk binary will be removed and the newer
release will integrate this block backend into cloud-hypervisor
binary. The block backend code has been added num_queues cmdline
support, we need update multiple queues help info for this
block-backend in the cloud-hypervisor.
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
We measure the memory overhead that the VMM process adds to the guest VM
and compare it with a maximum acceptable limit. The test is run against
a simple VM, running 1 vCPU and 512MB of RAM. Although this is not by
any mean a comprehensive VMM overhead measurement, it will allow us to
detect when and if any PR makes our code cross an arbitrary memory
overhead threshold.
Fixes: #64
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
As this can happen during the running of the VMM we should be very
careful not to panic() as that can lead to a thread being used by the VM
disappearing.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add a socket and vhost_user parameter to this option so that the same
configuration option can be used for both virtio-block and
vhost-user-block. For now it is necessary to specify both vhost_user
and socket parameters as auto activation is not yet implemented. The wce
parameter for supporting "Write Cache Enabling" is also added to the
disk configuration.
The original command line parameter is still supported for now and will
be removed in a future release.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Add a socket and vhost_user parameter to this option so that the same
configuration option can be used for both virtio-net and vhost-user-net.
For now it is necessary to specify both vhost_user and socket parameters
as auto activation is not yet implemented. The original command line
parameter is still supported for now.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to reduce the amount of times VMs are being started through
integration tests, this commit consolidates very similar tests related
to virtio-blk into a single one.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Just add a new integration test to verify that multiqueue support is
correctly supported and that we can find the right amount of queues in
the guest.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The number of queues and the size of each queue were not configurable.
In anticipation for adding multiqueue support, this commit introduces
some new parameters to let the user decide about the number of queues
and the queue size.
Note that the default values for each of these parameters are identical
to the default values used for vhost-user-blk, that is 1 for the number
of queues and 128 for the queue size.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Add explicit exclusions with --net-backend from --block-backend (and
vice-versa.) And also with "--kernel" as this is the option for "VM boot" that is never optional.
Ideally we would conflcit the backend arguments against the "vm-group"
however this does not work as it includes some arguments that have a
default value set and thus clap thinks those arguments are always
provided. Conflicting with "--kernel" is thus a reasonable compromise.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Generate the VM params until after the logging has been enabled. This
will make it possible to reuse the logging for vhost backends.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
Previous commit added 'direct' support for virtio-blk to be able to
open files backing disk images with O_DIRECT, so add an integration
test for this new feature.
Signed-off-by: Sergio Lopez <slp@redhat.com>
Previous commit added 'readonly' support for virtio-blk to be able to
configure read-only devices, so add an integration test for this new
feature.
Signed-off-by: Sergio Lopez <slp@redhat.com>
This test is a variant of test_boot_vhost_user_blk(), named
test_boot_vhost_user_blk_direct(), that instances the vhost-user-blk
daemon with 'direct=true', to test this recently introduced feature
for opening files with O_DIRECT.
Signed-off-by: Sergio Lopez <slp@redhat.com>
Update queue number with 4 to verify if vhost-user-net device
and backend could work well with multiple queue.
Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
There are two new options num_queues and queue_size defined for
virtio-net, add them in test_valid_vm_config_net which is used
to validate that both the CLI and the OpenAPI will generate the
same configuration.
Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
Add num_queues and queue_size for virtio-net device to make them configurable,
while add the associated options in command line.
Update cloud-hypervisor.yaml with the new options for NetConfig.
Signed-off-by: Cathy Zhang <cathy.zhang@intel.com>
This specifies how much address space should be reserved for hotplugging
of RAM. This space is reserved by adding move the start of the device
area by the desired amount.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
With the latest version of virtiofsd, some changes have been made
to the socket path parameter. This needs to be updated in the cloud
hypervisor codebase, so that our CI keeps running correctly.
Fixes#611
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The cache= option from the virtiofsd backend is meant to be used
depending on how the user wants to use the guest page cache. For most
use cases, we want to bypass the guest page cache to reduce the guest
memory footprint, which is why "cache=none" should be the default.
"cache=always" is the other option when the user wants to do something
specific by using the guest page cache, but should not be the default
anymore.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
With latest kernel, virtio-fs mount command has been simplified, and
this needs to be applied everywhere in our tests and documentation.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Because we don't always reach the expected footprint improvements with
KSM, let's review the numbers. By reducing the expectations and
increasing the amount of pages to scan, this should stabilize the CI.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This patch is to check if block device is readonly
when backend set readonly=true.
The lsblk command can show the RO value in the guest.
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
The goal here is to ensure that CLI and OpenAPI both behave as closely
as possible, and also that they behave as expected.
Leveraging the reorganization of the code, we can now compare two
VmConfig structures generated from one CLI entry on one side, and from
an OpenAPI entry (JSON payload) on the other side.
Fixes#535
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This brings more modularity to the code, which will be helpful when we
will later test the CLI and OpenAPI generate the same VmConfig output.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This brings more modularity to the code, which will be helpful when we
will later test the CLI and OpenAPI generate the same VmConfig output.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
This brings more modularity to the code, which will be helpful when we
will later test the CLI and OpenAPI generate the same VmConfig output.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
On our CI the /tmp filesystem is mounted as tmpfs and this is the
location where the test disk images are located. When the CI worker
nodes have less memory and fewer CPUs the tmpfs fills up as the tests
run in parallel.
Introduce a mechanism to reduce the parallelism of the tests based on
starvation of the tmpfs disk availability.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
By default, and in order to avoid falling into the legacy CLI usage, the
CPU argument should at least include "boot=" to define the number of
CPUs.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to validate that multiple devices can be passed through and
they are still fully functional, this patch extends the existing VFIO
test to pass a second virtio-net device, and verifies that both
interfaces are functional by ssh'ing into each network interface.
Fixes#503
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
The easiest way to detect if the kernel is willing to accept hotplug
vCPUs is to check the dmesg output.
Switch the test to bionic as the Clear Cloud image lacks "dmesg."
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
In order to validate the new virtio-fs daemon written in Rust is
behaving correctly, a new integration test has been added. Important to
note that for now, only a test with cache=none and dax=off can be added
since the daemon does not support shared memory region yet.
The long term goal being to replace virtiofsd with vhost_user_daemon
once it will reach parity regarding the supported features.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Instead of using bash and awk, using Rust allows us to retrieve
information about a VM process with the right permissions as we are not
forced to spawn a new child process.
Signed-off-by: Rob Bradford <robert.bradford@intel.com>
The test validates that when the mergeable option is enabled, the
resulting PSS for two instances of cloud-hypervisor is lower than two
instances not using the mergeable flag.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to let the user indicate if the persistent memory pages should
be marked as mergeable or not, a new option is being introduced.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
In order to let the user indicate if the guest RAM pages should be
marked as mergeable or not, a new option is being introduced.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When vmm.ping give a response, we expect get the version from
the VMM not the vmm create
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
vmm.ping will help to check if http API server is up and
running.
This also removes the vmm.info endpoint.
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
Use the new vhost-user-blk backend for the integration tests,
eliminating the need for building vubd using the implementation in
QEMU.
Signed-off-by: Sergio Lopez <slp@redhat.com>
A new ClearLinux image has been uploaded to the Azure storage account.
It is based off of the ClearLinux cloudguest image 31310 version, with
two extra bundles added to it.
First bundle is sysadmin-basic to include utility like netcat, and the
second bundle is iperf, adding the iperf binary to the image.
The image is 2G in size.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
When VFIO devices are created and if the device is attached to the
virtual IOMMU, the ExternalDmaMapping trait implementation is created
and associated with the device. The idea is to build a hash map of
device IDs with their associated trait implementation.
This hash map is provided to the virtual IOMMU device so that it knows
how to properly trigger external mappings associated with VFIO devices.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Having the virtual IOMMU created with --iommu is one thing, but we also
need a way to decide if a VFIO device should be attached to the virtual
IOMMU or not. That's why we introduce an extra option "iommu" with the
value "on" or "off". By default, the device is not attached, which means
"iommu=off".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We pause a VM from the API, then SSH'ing into it should fail.
After resuming, SSH'ing should work again.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
Because the L2 VM running in the VFIO integration test is actually
running as L3 (since the CI runs in a VM), it can take quite some
time for this VM to boot.
The way to solve this issue is to extend the sleep time before to try
communicating with the L2 VM, but also to speed up the boot time by
using virtio-console instead of serial. We suspect the use of serial,
implying PIO VM exits for each character on the serial port is quite
expensive compared to the paravirtualized console.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Azure virtual machines can have private IPs in the 172.16.x.x range,
causing some issues with the VFIO test. By using 172.17.x.x for this
test, we avoid IP conflicts.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that our custom kernel includes all the patches for the full support
of virtio-iommu, we can go one step further by attaching the virtio-net
device to the virtual IOMMU and use it to SSH some commands validating
both disks and the network card are isolated into their own IOMMU group.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Now that cloud-hypervisor can expose a virtual IOMMU to its guest VM,
the integration test validating the VFIO support with virtio-net can be
updated to use cloud-hypervisor exclusively.
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Having the virtual IOMMU created with --iommu is one thing, but we also
need a way to decide if a virtio-vsock device should be attached to this
virtual IOMMU or not. That's why we introduce an extra option "iommu"
with the value "on" or "off". By default, the device is not attached,
which means "iommu=off".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Having the virtual IOMMU created with --iommu is one thing, but we also
need a way to decide if a virtio-console device should be attached to
this virtual IOMMU or not. That's why we introduce an extra option
"iommu" with the value "on" or "off". By default, the device is not
attached, which means "iommu=off".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Having the virtual IOMMU created with --iommu is one thing, but we also
need a way to decide if a virtio-pmem device should be attached to this
virtual IOMMU or not. That's why we introduce an extra option "iommu"
with the value "on" or "off". By default, the device is not attached,
which means "iommu=off".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Having the virtual IOMMU created with --iommu is one thing, but we also
need a way to decide if a virtio-rng device should be attached to this
virtual IOMMU or not. That's why we introduce an extra option "iommu"
with the value "on" or "off". By default, the device is not attached,
which means "iommu=off".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Having the virtual IOMMU created with --iommu is one thing, but we also
need a way to decide if a virtio-net device should be attached to this
virtual IOMMU or not. That's why we introduce an extra option "iommu"
with the value "on" or "off". By default, the device is not attached,
which means "iommu=off".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
Having the virtual IOMMU created with --iommu is one thing, but we also
need a way to decide if a virtio-blk device should be attached to this
virtual IOMMU or not. That's why we introduce an extra option "iommu"
with the value "on" or "off". By default, the device is not attached,
which means "iommu=off".
One side effect of this new option is that we had to introduce a new
option for the disk path, simply called "path=".
Signed-off-by: Sebastien Boeuf <sebastien.boeuf@intel.com>
We use the serde crate to serialize and deserialize the VmVConfig
structure. This structure will be passed from the HTTP API caller as a
JSON payload and we need to deserialize it into a VmConfig.
For a convenient use of the HTTP API, we also provide Default traits
implementations for some of the VmConfig fields (vCPUs, memory, etc...).
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The kernel path was the only mandatory command line option.
With the addition of the --api-socket option, we can run without a
kernel path and get it later through the API.
Since we can end up with VM configurations that are no longer valid by
default, we need to provide a validation check for it. For now, if the
kernel path is not defined, the VM configuration is invalid.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
The API server will unconditionally run through a UNIX domain socket
which default path is /run/user/<uid>/cloud-hypervisor.<pid>.
The --api-socket command line option allows to override that default
value with some custom socket path.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
With the API server socket option, we will be able to support a model
where the user can start cloud-hypervisor with no options or an
alternative API server socket path. In this case, we don't want to try
to start a new guest VM, and for that we need to know if the user has
set any VM configuration at all. Grouping all VM configuration specific
options together is one way to be able to know about it.
If the user has not set any VM configuration, we only start the API
server. If it has set anything, we will verify that the overall
configuration is valid and will implicitly convert that configuration
into a request to the API server.
Signed-off-by: Samuel Ortiz <sameo@linux.intel.com>
