Commit Graph

26446 Commits

Author SHA1 Message Date
Michal Privoznik
022705b81f virNetClientStream: Wire up VIR_NET_STREAM_HOLE
Whenever server sends a client stream packet (either regular with
actual data or stream skip one) it is queued on @st->rx. So the
list is a mixture of both types of stream packets. So now that we
have all the helpers needed we can wire their processing up. But
since virNetClientStreamRecvPacket doesn't support
VIR_STREAM_RECV_STOP_AT_HOLE flag yet, let's turn all received
skips into zeroes repeating requested times.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
d6e5347ce3 remote: Implement virStreamRecvHole
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
57760ec1e2 Introduce virNetClientStreamRecvHole
This function will fetch previously processed stream holes and
return their sum.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
486656f168 virNetClientStreamRecvPacket: Introduce @flags argument
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
c331f56b93 remote_driver: Implement virStreamSendHole
Now that we have RPC wrappers over VIR_NET_STREAM_HOLE we can
start wiring them up. This commit wires up situation when a
client wants to send a hole to daemon.

To keep stream offsets synchronous, upon successful call on the
daemon skip the same hole in local part of the stream.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
c6e0fcce4c virnetclientstream: Introduce virNetClientStreamHandleHole
This is a function that handles an incoming STREAM_HOLE packet.
Even though it is not wired up yet, it will be soon. At the
beginning do couple of checks whether server plays nicely and
sent us a STREAM_HOLE packed only after we've enabled sparse
streams. Then decodes the message payload to see how big the hole
is and stores it in passed @length argument.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
470aaa20d4 daemon: Implement VIR_NET_STREAM_HOLE handling
Basically, whenever the new type of stream packet arrives to the
daemon call this function that decodes it and calls
virStreamSendHole(). Otherwise a regular data stream packet has
arrived and therefore continue its processing.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
87f2a5c65d virnetclientstream: Introduce virNetClientStreamSendHole
While the previous commit implemented a helper for sending a
STREAM_HOLE packet for daemon, this is a client's counterpart.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
fa19af0a13 daemon: Introduce virNetServerProgramSendStreamHole
This is just a helper function that takes in a length value,
encodes it into XDR and sends to client.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
2e5671adf2 Teach wireshark plugin about VIR_NET_STREAM_HOLE
Ideally, this would be generated, but to achieve that
corresponding XDR definitions needed to go into a different .x
file. But they belong just to the one that they are right now.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
79d16419c4 Introduce VIR_NET_STREAM_HOLE message type
This is a special type of stream packet, that is bidirectional
and contains information regarding how many bytes each side will
be skipping in the stream.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
7ddf7cbffd RPC: Introduce virNetStreamHole
This is going to be RPC representation for virStreamSendHole.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
5f4f9d87a9 Add new flag to daemonCreateClientStream and virNetClientStreamNew
Add a new argument to daemonCreateClientStream in order to allow for
future expansion to mark that a specific stream can be used to skip
data, such as the case with sparsely populated files. The new flag will
be the eventual decision point between client/server to decide whether
both ends can support and want to use sparse streams.

A new bool 'allowSkip' is added to both _virNetClientStream and
daemonClientStream in order to perform the tracking.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
8b6ffe4077 virNetClientStreamNew: Track origin stream
Add a virStreamPtr pointer to the _virNetClientStream
in order to reverse track the parent stream.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
84b5079232 Introduce virStreamInData
This is just an internal API, that calls corresponding function
in stream driver. This function will set @data = 1 if the
underlying file is in data section, or @data = 0 if it is in a
hole. At any rate, @length is set to number of bytes remaining in
the section the file currently is.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
a524951091 Introduce virStreamSparseSendAll
This is just a wrapper over new function that have been just
introduced: virStreamSendHole() . It's very similar to
virStreamSendAll() except it handles sparse streams well.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
3a1e2e920e Introduce virStreamSparseRecvAll
This is just a wrapper over new functions that have been just
introduced: virStreamRecvFlags(), virStreamRecvHole(). It's very
similar to virStreamRecvAll() except it handles sparse streams
well.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
87e4a842b7 Introduce VIR_STREAM_RECV_STOP_AT_HOLE flag
Add a new flag to virStreamRecvFlags in order to handle being able to
stop reading from the stream so that the consumer can generate a "hole"
in stream target. Generation of a hole replaces the need to receive and
handle a sequence of zero bytes for sparse stream targets.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
b6aedd2f05 Introduce virStreamRecvHole
This function is basically a counterpart for virStreamSendHole().
If one side of a stream called virStreamSendHole() the other
should call virStreamRecvHole() to get the size of the hole.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
69a573d010 Introduce virStreamSendHole
This API is used to tell the other side of the stream to skip
some bytes in the stream. This can be used to create a sparse
file on the receiving side of a stream.

It takes @length argument, which says how big the hole is. This
skipping is done from the current point of stream. Since our
streams are not rewindable like regular files, we don't need
@whence argument like seek(2) has.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
65b9cd6797 Implement virStreamRecvFlags to some drivers
There are three virStreamDriver's currently supported:

 * virFDStream
 * remote driver
 * ESX driver

For now, backend virStreamRecvFlags support for only remote driver and
ESX driver is sufficient. Future patches will update virFDStream.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
a35e2836b3 Introduce virStreamRecvFlags
This patch is adding the virStreamRecvFlags as a variant to the
virStreamRecv function in order to allow for future expansion of
functionality for processing sparse streams using a @flags
argument.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
b928d140f4 util: Introduce virFileInData
This function takes a FD and determines whether the current
position is in data section or in a hole. In addition to that,
it also determines how much bytes are there remaining till the
current section ends.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
07c2399c01 virfdstream: Use messages instead of pipe
One big downside of using the pipe to transfer the data is that
we can really transfer just bare data. No metadata can be carried
through unless some formatted messages are introduced. That would
be quite painful to achieve so let's use a message queue. It's
fairly easy to exchange info between threads now that iohelper is
no longer used.

The reason why we cannot use the FD for plain files directly is
that despite us setting noblock flag on the FD, any
read()/write() blocks regardless (which is a show stopper since
those parts of the code are run from the event loop) and poll()
reports such FD as always readable/writable - even though the
subsequent operation might block.

The pipe is still not gone though. It is used to signal the event
loop that an event occurred (e.g. data is available for reading
in the queue, or vice versa).

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Gordon Messmer
ca21d75d25 virsh: add --io when attaching disks to guests
virt-install and virt-manager both default to explicitly setting
"io='native'" in the disk "driver" tag. virsh, however, does not and also
does not provide an option to specify that setting at all.  As a result,
disks use a different IO mechanism (the default, "threads") when attached
post-setup using virsh.  Adding this option allows users to keep disk
performance consistent for disks attached at install, and those attached
afterward.
2017-05-17 13:50:19 -04:00
Peter Krempa
ed61e0b368 qemu: driver: Allow passing disk target as top image with block commit
Since we allow active layer block commit the users are allowed to commit
the top of the chain (e.g. vda) into the backing image. The API would
not accept that parameter, as it tried to look up the image in the
backing chain.

Add the ability to use the top level image target name explicitly as the
top image of the block commit operation.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1451394
2017-05-17 17:16:15 +02:00
Andrea Bolognani
bb09afd5a2 news: Update for GIC version on TCG changes
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-05-16 16:48:30 +02:00
Andrea Bolognani
5645badd1f gic: Remove VIR_GIC_VERSION_DEFAULT
The QEMU default is GICv2, and some of the code in libvirt
relies on the exact value. Stop pretending that's not the
case and use GICv2 explicitly where needed.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-05-16 16:48:30 +02:00
Andrea Bolognani
bc07101a7c qemu: Use GICv2 for aarch64/virt TCG guests
There are currently some limitations in the emulated GICv3
that make it unsuitable as a default. Use GICv2 instead.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1450433

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-05-16 16:48:30 +02:00
Andrea Bolognani
b24eaf6210 tests: Check default GIC version for aarch64/virt TCG guests
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-05-16 16:48:30 +02:00
Andrea Bolognani
5290d4fdaf qemu: Use qemuDomainMachineIsVirt() more
Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2017-05-16 16:48:30 +02:00
Pavel Hrdina
ed99660446 qemu: improve detection of UNIX path generated by libvirt
Currently we consider all UNIX paths with specific prefix as generated
by libvirt, but that's a wrong assumption.  Let's make the detection
better by actually checking whether the whole path matches one of the
paths that we generate or generated in the past.

The UNIX path isn't stored in config XML since libvirt-1.3.1.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1446980

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-05-16 11:33:49 +02:00
Daniel P. Berrange
31020664e9 remove hack for debian etch limits.h
The debian etch distro was end-of-life a long time ago so we no
longer need the ULLONG_MAX hack. In any case gnulib now provides
an equivalent fix by default, and so our definition now triggers
syntax-check rule failure

src/internal.h:#    define ULLONG_MAX   ULONG_LONG_MAX
maint.mk: define the above via some gnulib .h file
maint.mk:843: recipe for target 'sc_prohibit_always-defined_macros' failed

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-05-15 17:00:09 +01:00
Daniel P. Berrange
52de9b28db maint: update to latest gnulib
This pulls in the fixes for poll() on Win32 which finally
makes the remote driver work again.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-05-15 16:13:03 +01:00
Ján Tomko
935d927aa8 conf: add ABI stability checks for IOMMU options
https://bugzilla.redhat.com/show_bug.cgi?id=1427005
2017-05-15 15:44:12 +02:00
Ján Tomko
3a276c6524 conf: split out virDomainIOMMUDefCheckABIStability 2017-05-15 15:44:12 +02:00
Ján Tomko
a56914486c qemu: format caching-mode on iommu command line
Format the caching-mode option for the intel-iommu device,
based on its <driver caching> attribute value.

https://bugzilla.redhat.com/show_bug.cgi?id=1427005
2017-05-15 15:44:11 +02:00
Ján Tomko
d12781b47e conf: add caching_mode attribute to iommu device
Add a new attribute to control the caching mode.

https://bugzilla.redhat.com/show_bug.cgi?id=1427005
2017-05-15 15:44:11 +02:00
Ján Tomko
04028a9db9 qemu: format intel-iommu,intremap on the command line
https://bugzilla.redhat.com/show_bug.cgi?id=1427005
2017-05-15 15:44:11 +02:00
Ján Tomko
2020e2c6f2 conf: add <driver intremap> to <iommu>
Add a new attribute to control interrupt remapping.

https://bugzilla.redhat.com/show_bug.cgi?id=1427005
2017-05-15 15:44:11 +02:00
Ján Tomko
6b5c6314b2 qemu: format kernel_irqchip on the command line
Add kernel_irqchip=split/on to the QEMU command line
and a capability that looks for it in query-command-line-options
output. For the 'split' option, use a version check
since it cannot be reasonably probed.

https://bugzilla.redhat.com/show_bug.cgi?id=1427005
2017-05-15 15:44:11 +02:00
Ján Tomko
8023b21a95 conf: add <ioapic driver> to <features>
Add a new <ioapic> element with a driver attribute.

Possible values are qemu and kvm. With 'qemu', the I/O
APIC can be put in the userspace even for KVM domains.

https://bugzilla.redhat.com/show_bug.cgi?id=1427005
2017-05-15 15:41:17 +02:00
Peter Krempa
14789b7ea8 util: conf: Don't log when adding commented out lines
virConfAddEntry spams debug logs even for fully commented out lines.
Skip such messages to avoid:

2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
2017-05-12 12:35:38.867+0000: 10820: debug : virConfAddEntry:241 : Add entry (null) (nil)
...

This also fixes NULL passed to printf.
2017-05-15 13:51:25 +02:00
Serge Hallyn
756ef0c353 storage: use 0711 as the default perms for dirs
There should be no need to make dir based pools world/group readable.
So use 0711, not 0755, as the default perms for storage dirs.

Updates in v2:
 - adapt commit wording to mention dropping group readable as well

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2017-05-15 12:09:24 +01:00
Daniel P. Berrange
3213e369c3 Detect VMDK version 3 files
The metadata libvirt cares about is identical for version 3
as for previous versions, so we merely need list the new
version number.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-05-15 12:09:04 +01:00
Christian Ehrhardt
aeda1b8c56 qemu: monitor: do not report error on shutdown
If a shutdown is expected because it was triggered via libvirt we can
also expect the monitor to close. In those cases do not report an
internal error like:
  "internal error: End of file from qemu monitor"

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2017-05-15 12:34:19 +02:00
Martin Kletzander
ec337aee9b test-wrap-argv.pl: Accept short parameter -i for --in-place
I like to use it that way and every time I try running it I just
instinctively use '-i' (like with sed, etc.) and it makes sense, IMHO.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2017-05-15 12:34:19 +02:00
Pavel Hrdina
0918b84968 util: introduce virBufferEscapeRegex
Add a helper to escape all possible meta-characters used for
POSIX extended regular expressions.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-05-12 16:54:33 +02:00
Pavel Hrdina
f15f155403 util: introduce virStringMatch
Simply tries to match the provided regex on a string and returns
the result.  Useful if caller don't care about the matched substring
and want to just test if some pattern patches a string.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-05-12 16:51:18 +02:00
Nikolay Shirokovskiy
bb9055b87e docs: install html fonts and related 2017-05-12 16:49:08 +03:00