Commit Graph

631 Commits

Author SHA1 Message Date
Marc Hartmayer
e9538813ec rpc: first allocate the memory and then set the count
Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Reviewed-by: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Reviewed-by: Bjoern Walk <bwalk@linux.vnet.ibm.com>
2017-06-12 19:11:30 -04:00
Michal Privoznik
1a4b21f1c6 virNetClientStreamQueuePacket: Set st->incomingEOF on the end of stream
While reworking client side of streams, I had to postpone payload
decoding so that stream holes and stream data can be
distinguished in virNetClientStreamRecvPacket. That's merely what
18944b7aea does. However, I accidentally removed one important
bit: when server sends us an empty STREAM packet (with no
payload) - meaning end of stream - st->incomingEOF flag needs to
be set. It used to be before I touched the code. After I removed
it, virNetClientStreamRecvPacket will try to fetch more data from
the stream, but it will never come.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Jim Fehlig <jfehlig@suse.com>
2017-06-07 18:00:25 +02:00
Richard W.M. Jones
b088f85d42 rpc: Double buffer size instead of quadrupling buffer size.
When increasing the buffer size up to VIR_NET_MESSAGE_MAX, we
currently quadruple it each time.  This unfortunately means that we
cannot allow certain buffer sizes -- for example the current
VIR_NET_MESSAGE_MAX == 33554432 can never be "hit" since ‘newlen’
jumps from 16MB to 64MB.

Instead of quadrupling, double it each time.

Thanks: Daniel Berrange.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
2017-05-26 13:53:31 +01:00
Peter Krempa
97863780e8 rpc: Bump maximum message size to 32M
While most of the APIs are okay with 16M messages, the bulk stats API
can run into the limit in big configurations. Before we devise a new
plan for this, bump this limit slightly to accomodate some more configs.
2017-05-24 14:02:29 +02:00
Michal Privoznik
0da4a635bc virStream: Forbid negative seeks
Currently, we don't assign any meaning to that. Our current view
on virStream is that it's merely a pipe. And pipes don't support
seeking.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 15:05:18 +02:00
Michal Privoznik
8f08f28f74 gendispatch: Introduce @sparseflag for our calls
Now, not all APIs are going to support sparse streams. To some it
makes no sense at all, e.g. virDomainOpenConsole() or
virDomainOpenChannel(). To others, we will need a special flag to
indicate that client wants to enable sparse streams. Instead of
having to write RPC dispatchers by hand we can just annotate in
our .x files that a certain flag to certain RPC call enables this
feature. For instance:

     /**
      * @generate: both
      * @readstream: 1
      * @sparseflag: VIR_SPARSE_STREAM
      * @acl: storage_vol:data_read
      */
     REMOTE_PROC_DOMAIN_SOME_API = XXX,

Therefore, whenever client calls virDomainSomeAPI(..,
VIR_SPARSE_STREAM); daemon will mark that down and send stream
skips when possible.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
31024b3d05 remote_driver: Implement VIR_STREAM_RECV_STOP_AT_HOLE
This is fairly trivial now that we have everything in place.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
022705b81f virNetClientStream: Wire up VIR_NET_STREAM_HOLE
Whenever server sends a client stream packet (either regular with
actual data or stream skip one) it is queued on @st->rx. So the
list is a mixture of both types of stream packets. So now that we
have all the helpers needed we can wire their processing up. But
since virNetClientStreamRecvPacket doesn't support
VIR_STREAM_RECV_STOP_AT_HOLE flag yet, let's turn all received
skips into zeroes repeating requested times.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
57760ec1e2 Introduce virNetClientStreamRecvHole
This function will fetch previously processed stream holes and
return their sum.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
486656f168 virNetClientStreamRecvPacket: Introduce @flags argument
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
c6e0fcce4c virnetclientstream: Introduce virNetClientStreamHandleHole
This is a function that handles an incoming STREAM_HOLE packet.
Even though it is not wired up yet, it will be soon. At the
beginning do couple of checks whether server plays nicely and
sent us a STREAM_HOLE packed only after we've enabled sparse
streams. Then decodes the message payload to see how big the hole
is and stores it in passed @length argument.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
87f2a5c65d virnetclientstream: Introduce virNetClientStreamSendHole
While the previous commit implemented a helper for sending a
STREAM_HOLE packet for daemon, this is a client's counterpart.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
fa19af0a13 daemon: Introduce virNetServerProgramSendStreamHole
This is just a helper function that takes in a length value,
encodes it into XDR and sends to client.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
79d16419c4 Introduce VIR_NET_STREAM_HOLE message type
This is a special type of stream packet, that is bidirectional
and contains information regarding how many bytes each side will
be skipping in the stream.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
7ddf7cbffd RPC: Introduce virNetStreamHole
This is going to be RPC representation for virStreamSendHole.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
5f4f9d87a9 Add new flag to daemonCreateClientStream and virNetClientStreamNew
Add a new argument to daemonCreateClientStream in order to allow for
future expansion to mark that a specific stream can be used to skip
data, such as the case with sparsely populated files. The new flag will
be the eventual decision point between client/server to decide whether
both ends can support and want to use sparse streams.

A new bool 'allowSkip' is added to both _virNetClientStream and
daemonClientStream in order to perform the tracking.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Michal Privoznik
8b6ffe4077 virNetClientStreamNew: Track origin stream
Add a virStreamPtr pointer to the _virNetClientStream
in order to reverse track the parent stream.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-05-18 07:42:13 +02:00
Jiri Denemark
42faf316ec client: Report proper close reason
When we get a POLLHUP or VIR_EVENT_HANDLE_HANGUP event for a client, we
still want to read from the socket to process any accumulated data. But
doing so inevitably results in an error and a call to
virNetClientMarkClose before we get to processing the hangup event (and
another call to virNetClientMarkClose). However the close reason passed
to the second virNetClientMarkClose call is ignored because another one
was already set. We need to pass the correct close reason when marking
the socket to be closed for the first time.

https://bugzilla.redhat.com/show_bug.cgi?id=1373859

Signed-off-by: Jiri Denemark <jdenemar@redhat.com>
2017-05-02 18:53:24 +02:00
Michal Privoznik
1a4a4ffa3e lib: Fix c99 style comments
We prefer c89 style of comments.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-04-27 14:13:19 +02:00
Yi Wang
ab5bb6f346 rpc: fix keep alive timer segfault
ka maybe have been freeed in virObjectUnref, application using
virKeepAliveTimer will segfault when unlock ka. We should keep
ka's refs positive before using it.

#0  0x00007fd8f79970e8 in virClassIsDerivedFrom (klass=0xdeadbeef, parent=0x7fd8e8001b80) at util/virobject.c:169
#1  0x00007fd8f799742e in virObjectIsClass (anyobj=anyobj entry=0x7fd8e800b9c0, klass=<optimized out>) at util/virobject.c:365
#2  0x00007fd8f79974e4 in virObjectUnlock (anyobj=0x7fd8e800b9c0) at util/virobject.c:338
#3  0x00007fd8f7ac477e in virKeepAliveTimer (timer=<optimized out>, opaque=0x7fd8e800b9c0) at rpc/virkeepalive.c:177
#4  0x00007fd8f7e5c9cf in libvirt_virEventInvokeTimeoutCallback () from /usr/lib64/python2.7/site-packages/libvirtmod.so
#5  0x00007fd8ff64db94 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#6  0x00007fd8ff64f1ad in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#7  0x00007fd8ff64d85f in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#8  0x00007fd8ff64d950 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#9  0x00007fd8ff64d950 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#10 0x00007fd8ff64f1ad in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#11 0x00007fd8ff5dc098 in function_call () from /lib64/libpython2.7.so.1.0
#12 0x00007fd8ff5b7073 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#13 0x00007fd8ff5c6085 in instancemethod_call () from /lib64/libpython2.7.so.1.0
#14 0x00007fd8ff5b7073 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#15 0x00007fd8ff648ff7 in PyEval_CallObjectWithKeywords () from /lib64/libpython2.7.so.1.0
#16 0x00007fd8ff67d7e2 in t_bootstrap () from /lib64/libpython2.7.so.1.0
#17 0x00007fd8ff358df3 in start_thread () from /lib64/libpthread.so.0
#18 0x00007fd8fe97d3ed in clone () from /lib64/libc.so.6

Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2017-04-24 12:19:13 +02:00
Daniel P. Berrange
234ac4e18d Fix error reporting when poll returns POLLHUP/POLLERR
In the RPC client event loop code, if poll() returns only a POLLHUP
or POLLERR status, then we end up reporting a bogus error message:

  error: failed to connect to the hypervisor
  error: An error occurred, but the cause is unknown

We do actually report an error, but we virNetClientMarkClose method
has already captured the error status before we report it, so the
real error gets thrown away. The key fix is to report the error
before calling virNetClientMarkClose(). In changing this, we also
split out reporting of POLLHUP vs POLLERR to make any future bugs
easier to diagnose.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-04-20 10:15:46 +01:00
Daniel P. Berrange
f7d7825d06 Ignore SASL deprecation warnings on OS-X
Apple have annotated all SASL functions as deprecated for
unknown reasons. Since they still work, lets just ignore
the warnings. If Apple finally delete the SASL functions
our configure check should already catch that

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-04-19 10:51:51 +01:00
Daniel P. Berrange
728cacc8ab annotate all mocked functions with noinline
CLang's optimizer is more aggressive at inlining functions than
gcc and so will often inline functions that our tests want to
mock-override. This causes the test to fail in bizarre ways.

We don't want to disable inlining completely, but we must at
least prevent inlining of mocked functions. Fortunately there
is a 'noinline' attribute that lets us control this per function.

A syntax check rule is added that parses tests/*mock.c to extract
the list of functions that are mocked (restricted to names starting
with 'vir' prefix). It then checks that src/*.h header file to
ensure it has a 'ATTRIBUTE_NOINLINE' annotation. This should prevent
use from bit-rotting in future.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-04-19 10:51:51 +01:00
Pavel Hrdina
ab0a461b8c rpc: fix resource leak
Commit 252610f7dd switched to use hash to store servers.
Function virHashGetItems returns allocated array which needs
to be freed also for successful path, not only if there is
an error.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2017-04-11 13:23:00 +02:00
Marc Hartmayer
fd6e3f48ed refactoring: Use the return value of virObjectRef directly
Use the return value of virObjectRef directly. This way, it's easier
for another reader to identify the reason why the additional reference
is required.

Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Reviewed-by: Bjoern Walk <bwalk@linux.vnet.ibm.com>
2017-04-10 14:49:20 +02:00
Peter Krempa
21a2d1b69c rpc: serverclient: Add option to suppress errors on EOF
The protocol may not use an explicit API to close the connection and
just close the socket instead. Add option to suppress errors in such
case.
2017-03-17 17:20:22 +01:00
Peter Krempa
494c302c7f rpc: socket: Add possibility to suppress errors on read hangup
In some cases a read error due to connection hangup is expected. This
patch adds a flag that removes the logging of a virError in such case.
2017-03-17 17:19:00 +01:00
Daniel P. Berrange
83c1ab5838 Report what TLS priority string we use for a session
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-03-16 11:03:33 +00:00
Daniel P. Berrange
7310739d1f Short circuit SASL auth when no mechanisms are available
If the SASL config does not have any mechanisms we currently
just report an empty list to the client which will then
fail to identify a usable mechanism. This is a server config
error, so we should fail immediately on the server side.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-03-16 11:01:39 +00:00
Marc Hartmayer
d6bc7622f0 rpc: Fix potentially segfaults
We have to allocate first and if, and only if, it was successful we
can set the count. A segfault has occurred in
virNetServerServiceNewPostExecRestart() when VIR_ALLOC_N(svc->socks,
n) has failed, but svc->nsocsk = n was already set. Thus
virObejectUnref(svc) was called and therefore it was possible that
virNetServerServiceDispose was called => segmentation fault.  For
safeness NULL pointer check were added in
virNetServerServiceDispose().

Signed-off-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com>
Reviewed-by: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Reviewed-by: Bjoern Walk <bwalk@linux.vnet.ibm.com>
2017-02-12 15:02:42 -05:00
Pino Toscano
1a5de3fe2e remote: do not check for an existing config dir
When composing the path to the default known_hosts file (for the libssh
and libssh2 drivers), do not check whether the configuration directory
(determined by virGetUserConfigDirectory()) exists: both the drivers can
handle non-existing files, and are able to create them (and their
directories) in that case.

This adds a small behaviour change: before, the key for an unknown host,
and manually accepted, was saved only if the configuration directory
existed -- a bit incoherent behaviour though.
2017-01-11 13:38:04 +01:00
Pino Toscano
45c4a70c70 remote: fix logic for known_hosts and keyfile checks
If any of them is specified for the libssh and libssh2 drivers, there is
no need to depend on checks based on other paths: in particular, a
specified path for known_hosts was ignored if the local config directory
could not be determined, and the path for keyfile was ignored if the
home could not be determined.

Instead, lazily determine and use these two paths only in case they are
needed.
2017-01-11 13:37:45 +01:00
Pino Toscano
408a1ce5f8 rpc: libssh: allow a NULL known_hosts file
Make sure that virNetLibsshSessionSetHostKeyVerification accepts a NULL
value for the path to the known_hosts file:
- call ssh_options_set(SSH_OPTIONS_KNOWNHOSTS) anyway, using /dev/null,
  otherwise libssh will use its default path
- do not call ssh_write_knownhost when no known hosts file was set

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1406457
2017-01-11 13:37:24 +01:00
Roman Bogorodskiy
e4cb660160 genprotocol.pl: add darwin to fixup list 2016-11-24 17:17:43 +03:00
Pino Toscano
22eaee8e01 remote: expose a new libssh transport
Implement in virtNetClient and VirNetSocket the needed functions to
expose a new libssh transport, providing all the options that the
libssh2 transport supports.
2016-11-15 15:50:51 +01:00
Pino Toscano
6917467c2b libssh_transport: add new libssh-based transport
Implement a new libssh transport, which uses libssh to communicate with
remote hosts, and add all the build system stuff (search of libssh,
private symbols, etc) to built it.

This new transport supports all the common ssh authentication methods,
making use of libvirt's auth callbacks for interaction with the user.
2016-11-15 15:50:51 +01:00
Pino Toscano
24ee5dc907 virnetsocket: improve search for default SSH key
Add a couple of helper functions to check whether one of the default
names of SSH keys (as documented in ssh-keygen(1)) exists, and use them
to specify a key for the libssh2 transport if none was passed.
2016-11-15 15:50:51 +01:00
Pino Toscano
0e9fec979d virNetSocket: allow to not close FD
Add an internal variable to mark the FD as "not owned" by the
virNetSocket, in case the internal implementation takes the actual
ownership of the descriptor; this avoids a warning when closing the
socket, as the FD would be invalid.
2016-11-15 15:50:51 +01:00
Erik Skultety
238b20b47f rpc: virnetserver: Remove dead code checking the client limits
Prior to commit 2737aaaf, we allowed every client to connect successfully,
however, if accepting a client would eventually lead to an overcommit of the
limits, we would disconnect it immediately with "Too many active clients,
dropping connection from...". Recent changes refactored the code in a way, that
it is not possible for the client-related callback to be dispatched and the
client to be accepted if the limits wouldn't permit to do so, therefore a check
if a connection should be dropped due to limits violation has become a dead
code that could be removed.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-08-02 14:51:13 +02:00
Erik Skultety
e9ce8a7d24 admin: rpc: virnetserver: Fix updating of the client limits
Commit 2737aaaf changed our policy for accepting new clients in a way, that
instead of accepting new clients only to disconnect them immediately, since
that would overcommit the limit, we temporarily disable polling for the
dedicated file descriptor, so any new connection will queue on the socket.
Commit 8b1f0469 then added the possibility to change the limits during runtime
but it didn't re-enable polling for the previously disabled file descriptor,
thus any new connection would still continue to queue on the socket. This patch
forces an update of the services each time the limits were changed in some way.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1357776

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-08-02 14:51:13 +02:00
Erik Skultety
5b9f735f16 rpc: virnetserver: Add code to CheckLimits to handle suspending of services
So far, virNetServerCheckLimits was only used to possibly re-enable accepting
new clients that might have previously been disabled due to client limits
violation (max_clients, max_anonymous_clients). This patch refactors
virNetServerAddClient, which is currently the only place where the services get
disabled, in order to use the virNetServerCheckLimits helper instead of
checking the limits by itself.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-08-02 14:51:13 +02:00
Erik Skultety
17bc333411 rpc: virnetserver: Move virNetServerCheckLimits which is static up in the file
Since virNetServerAddClient checks for the limits in order to temporarily
suspend the services, thus not accepting any more clients, there is no reason
why virNetServerCheckLimits, which is only responsible for re-enabling
previously disabled services according to the limits, could not do both. To be
able to do that however, it needs to be moved up in the file since it's static
(and because it's just a helper and there's only one caller it should remain
 static).

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-08-02 14:51:13 +02:00
Erik Skultety
2e5417bc46 rpc: virnetserver: Rename ClientSetProcessingControls to ClientSetLimits
The original naming was just a leftover that should have been fixed in commit
8b1f0469.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-08-02 14:51:13 +02:00
Ján Tomko
a3f565b339 Fix possible invalid read in adminClientGetInfo
virNetServerClientGetInfo returns the client's remote address
as a string, which is a part of the client object.

Use VIR_STRDUP to make a copy which can be freely accessed
even after the virNetServerClient object is unlocked.

To reproduce, put a sleep between virObjectUnlock in
virNetServerClientGetInfo and virTypedParamsAddString in
adminClientGetInfo, then close the queried connection during
that sleep.
2016-06-29 16:13:12 +02:00
Ján Tomko
541bd183f7 Rename virNetClient*AddrString
Add SASL at the end to make the format obvious.
2016-06-23 22:31:20 +02:00
Ján Tomko
eaf7ff3873 Rename virNetServerClient*AddrString
Add SASL at the end to make the format obvious.
2016-06-23 22:23:21 +02:00
Ján Tomko
ea537e7b36 Add SASL to virNetSocket{Local,Remote}AddrString
Rename them to virNetSocket{Local,Remote}AddrStringSASL
to make their format more obvious.
2016-06-23 22:23:21 +02:00
Ján Tomko
dbce70b243 virNetSocket: rename AddrStr to AddrStrSASL
Make it more obvious that these are in the SASL format.
2016-06-23 22:23:21 +02:00
Ján Tomko
23b6559d65 Introduce virNetServerClientRemoteAddrStringURI
Use it in virNetServerClientGetInfo to switch back to using
the URI-format (separated by ':') instead of the SASL format
(separated by ';').

Also use it in the error message reported by virNetServerAddClient.
2016-06-23 22:23:02 +02:00
Ján Tomko
2da6a13e5f Introduce virNetSocketRemoteAddrStringURI
It will return the socket address and port in a URI-like
format: [::1]:1234
Add a test case to virnetsockettest.
2016-06-23 22:21:36 +02:00
Ján Tomko
0f7eeb20ad Revert "virnetsocket: Provide socket address format in a more standard form"
This partially reverts commit 9b45c9f049.

It changed the default format of socket address from the one SASL
requires, but did not adjust all the callers.

It also removed the test coverage for it.

Revert most of the changes except the virSocketAddrFormatFull support
for URI-formatted strings.

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=1345743 while
reverting the format used by virt-admin's client-info command from
the URI one to the SASL one.

https://bugzilla.redhat.com/show_bug.cgi?id=1345743
2016-06-23 22:15:06 +02:00
Peter Krempa
830f9b624d rpcgen: Add support for generating funcs returning alloc'd typed params
Since it's rather tedious to write the dispatchers for functions that
return an array of typed parameters (which are rather common) let's add
some rpcgen code to generate them.
2016-06-22 08:15:45 +02:00
Daniel P. Berrange
214489f550 rpc: allow priority string to be passed to TLS context
Extend the virNetTLSContextNew* constructors to allow
the TLS priority string to be passed in, overriding the
compile time default.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-06-08 13:48:45 +01:00
Daniel P. Berrange
cbb2e91ecc configure: allow setting default TLS priority string
Currently libvirt calls gnutls_set_default_priority()
which on old systems resolves to "NORMAL" while new
systems it resolves to "@SYSTEM". Either way, this
is a global default that is identical across all apps.

We want to allow distros to flexibility to define a
custom default string for libvirt priority, so add
a --tls-priority=STRING  flag to configure to enable
this to be set.

It is expected that distros would use this when creating
RPM/Deb/etc packages, according to their preferred crypto
handling policies.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-06-08 13:48:45 +01:00
Daniel P. Berrange
20c5ded9d0 rpc: set gnutls log function at global init time
Currently we set the gnutls log function when creating a
TLS context, however, the setting is in fact global, not
per context. So we should be setting it when we first call
gnutls_global_init() instead.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-06-08 13:48:45 +01:00
Daniel P. Berrange
d8a8af3492 tls: remove support for gnutls 1.x.x, require 2.2.0
We need to use the gnutls_priority_set_direct method which
was not introduced until 2.1.7, so bump version to 2.2.0
which is the first stable release with it included. This
release dates from Dec 2007 so it is reasonable to ditch
support for the 1.x.x series for gnutls releases entirely.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-06-08 13:48:45 +01:00
Jovanka Gulicoska
b29e08dbe3 More usage of virGetLastErrorMessage
Convert to virGetLastErrorMessage() in the rest of the code
2016-05-19 15:17:03 -04:00
Erik Skultety
8b1f04693d admin: Introduce virAdmServerSetClientLimits
Opposite operation to virAdmServerGetClientLimits. Understandably though,
setting values for current number of clients connected or still waiting
for authentication does not make sense, since changes to these values are event
dependent, i.e. a client connects - counter is increased. Thus only the limits
to maximum clients connected and waiting for authentication can be set. Should
a request for other controls to be set arrive (provided such a setting will
be first introduced to the config), the set of configuration controls can be
later expanded (thanks to typed params). This patch also introduces a
constraint that the maximum number of clients waiting for authentication has to
be less than the overall maximum number of clients connected and any attempt to
violate this constraint will be denied.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-19 12:31:53 +02:00
Erik Skultety
abf29786f2 virnetserver: Introduce server's client-related limits getters
Add some trivial getters for client related attributes to virnetserver before
any admin method can be introduced.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-19 11:11:13 +02:00
Cole Robinson
b6238738ea rpc: use virNetMessageClearPayload in client
This removes the opencoded payload freeing in the client, to use
the shared virNetMessageClearPayload call. Two changes:

- ClearPayload sets nfds=0, which fixes a potential crash if
  an error path called virNetMessageFree/Clear on the message
  after fds was free'd
- We drop the inner loop VIR_FORCE_CLOSE... this may mean fds are
  kept open a little bit longer if the call is blocking but in
  practice I don't think it will have any effect
2016-05-05 14:28:19 -04:00
Cole Robinson
64bd680d42 rpc: Clear more in virNetMessageClearPayload
Set all counters to 0. This doesn't impact current users, but
future users will want this
2016-05-05 14:28:19 -04:00
Cole Robinson
220c4e85b3 rpc: Add virNetMessageClearPayload
Handles freeing the buffer and fds, but not the message details.
Use it to drop some duplicate code.
2016-05-05 14:28:19 -04:00
Michal Privoznik
b17e610e1f virNetServerClientNewPostExecRestart: Avoid align problems
I've noticed this while trying to compile libvirt on my arm box.

  CC       rpc/libvirt_net_rpc_server_la-virnetserverclient.lo
rpc/virnetserverclient.c: In function 'virNetServerClientNewPostExecRestart':
rpc/virnetserverclient.c:516:45: error: cast increases required alignment of target type [-Werror=cast-align]
                                             (long long *) &timestamp) < 0) {
                                             ^
cc1: all warnings being treated as errors

Problem is, @timestap is defined as time_t which is 32 bits long,
and we are typecasting it to long long which is 64bits long.
Solution is to make @timestamp type of long long. At the same
time, we can make @conn_time in _virNetServerClient struct long
long too. There is no need for it to be type of time_t.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-05-05 13:48:56 +02:00
Michal Privoznik
2a3a2c2f5b virNetServerClientNewPostExecRestart: Drop useless typecasts
In this function, @id is defined as unsigned long long. When
passing this variable to virJSONValueObjectGetNumberUlong(),
well address of this variable, it's typecasted to ull*. There
is no need for that. It's a same story with @nrequests_max.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-05-05 13:48:53 +02:00
Erik Skultety
8420a53edf virnetserverclient: Add an internal method to retrieve client's identity
This method just aggregates various client object attributes, like socket
address, connection type (RO/RW), and some TCP/TLS/UNIX identity in an atomic
manner.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-03 15:52:50 +02:00
Erik Skultety
bde2cb6136 virneserverclient: Introduce virNetServerClientHasSASLSession
We do have a similar method, serving the same purpose, for TLS, but we lack
one for SASL. So introduce one, in order for other modules to be able to find
out, if a SASL session is active, or better said, that a SASL session exists
at all.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-03 15:52:50 +02:00
Erik Skultety
9b45c9f049 virnetsocket: Provide socket address format in a more standard form
Our socket address format is in a rather non-standard format and that is
because sasl library requires the IP address and service to be delimited by a
semicolon. The string form is a completely internal matter, however once the
admin interfaces to retrieve client identity information are merged, we should
return the socket address string in a common format, e.g. format defined by
URI rfc-3986, i.e. the IP address and service are delimited by a colon and
in case of an IPv6 address, square brackets are added:

Examples:
    127.0.0.1:1234
    [::1]:1234

This patch changes our default format to the one described above, while adding
separate methods to request the non-standard SASL format using semicolon as a
delimiter.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-03 15:52:50 +02:00
Erik Skultety
52a2eef948 admin: Introduce virAdmServerLookupClient
Just like with server-related APIs, before any of client-based APIs can be
called, a reference to a client-side client object needs to be obtained. For
this purpose, a lookup method should exist. Apart from the client retrieval
logic, a new error code for non-existent client had to be added as well.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-03 15:52:50 +02:00
Erik Skultety
42b06aa65d rpc: virnetserverclient: Implement client connection transport retrieval
Although we document 6 types of transport that we support, internally we can
only differentiate between TCP, TLS, and UNIX transports only, since both SSH
and libssh2 transports, due to using netcat, behave in the exactly the same
way as a UNIX socket.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-03 10:04:49 +02:00
Erik Skultety
04bab54d05 rpc: virnetserver: Support retrieval of a list of clients
For now, the list copy is done simply by locking the whole server, walking the
original and increasing the refcount on each object. We may want to change
the list to a lockable object (like list of domains) later in the future if
we discover some performance issues related to locking the whole server in
order to walk the whole list of clients, possibly issuing some 'ForEach'
callback.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-02 22:26:23 +02:00
Erik Skultety
4bd430748c rpc: gendispatch: Tune it to support client structure
Now that libvirt-admin supports another client-side object and provided that
we want to generate as many both client-side and server-side RPC dispatchers,
support for this needs to be added to gendispatch.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-02 22:26:23 +02:00
Erik Skultety
a32135b3b1 rpc: virnetserverclient: Introduce new attribute conn_time to client
Besides ID, libvirt should provide several parameters to help the user
distinguish two clients from each other. One of them is the connection
timestamp. This patch also adds a testcase for proper JSON formatting of the
new attribute too (proper formatting of older clients that did not support
this attribute yet is included in the existing tests) - in order to
testGenerateJSON to work, a mock of time_t time(time_t *timer) needed to be
created.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-02 22:25:52 +02:00
Erik Skultety
5841d64d25 rpc: virnetserverclient: Identify clients by an integer ID
Admin API needs a way of addressing specific clients. Unlike servers, which we
are happy to address by names both because its name reflects its purpose (to
some extent) and we only have two of them (so far), naming clients doesn't make
any sense, since a) each client is an anonymous, i.e. not recognized after a
disconnect followed by a reconnect, b) we can't predict what kind of requests
it's going to send to daemon, and c) the are loads of them comming and going,
so the only viable option is to use an ID which is of a reasonably wide data
type.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-05-02 22:25:51 +02:00
Cole Robinson
8958dde506 rpc: protocol: Clarify VIR_NET_ERROR usage with streams
The described protocol semantics really only apply to server initiated
stream messages. Document the semantics for client messages.
2016-05-02 10:13:04 -04:00
Michal Privoznik
18944b7aea virnetclientstream: Process stream messages later
There are two functions on the client that handle incoming stream
data.  The first one virNetClientStreamQueuePacket() is a low
level function that just processes the incoming stream data from
the socket and stores it into an internal structure. This happens
in the client event loop therefore the shorter the callbacks are,
the better. The second function virNetClientStreamRecvPacket()
then handles copying data from internal structure into a client
provided buffer.
Change introduced in this commit makes just that: new queue for
incoming stream packets is introduced. Then instead of copying
data into intermediate internal buffer and then copying them into
user buffer, incoming stream messages are queue into the queue
and data is copied just once - in the upper layer function
virNetClientStreamRecvPacket(). In the end, there's just one
copying of data and therefore shorter event loop callback. This
should boost the performance which has proven to be the case in
my testing.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-05-02 07:56:38 +02:00
Michal Privoznik
435ee578a0 Revert "rpc: Fix slow volume download (virsh vol-download)"
This reverts commit d9c9e138f2.

Unfortunately, things are going to be handled differently so this
commit must go.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-05-02 07:56:38 +02:00
Ben Gray
5ba48584fb rpc: Don't leak fd via CreateXMLWithFiles
FD passing APIs like CreateXMLWithFiles or OpenGraphicsFD will leak
file descriptors. The user passes in an fd, which is dup()'d in
virNetClientProgramCall. The new fd is what is transfered to the
server virNetClientIOWriteMessage.

Once all the fds have been written though, the parent msg->fds list
is immediately free'd, so the individual fds are never closed.

This closes each FD as its send to the server, so all fds have been
closed by the time msg->fds is free'd.

https://bugzilla.redhat.com/show_bug.cgi?id=1159766
2016-04-29 06:30:06 -04:00
Michal Privoznik
30898d80d7 virNet{Client,Server}*Stream: Update comment
After 434de30da5 the status values are prefixed VIR_NET_
rather than REMOTE_.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-04-21 16:29:41 +02:00
Michal Privoznik
8def0a22a3 daemonClientStream: Use unsigned int to store stream @serial
The stream serial number is the serial number of the RPC call
that initiated a data transfer. And as such can never be
negative. Moreover, when looking up internal state for a stream,
the serial numbers are compared. But hey, the serial number in
message header is unsigned too!

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-04-21 16:29:41 +02:00
Erik Skultety
93ab4da5f4 admin: Introduce virAdmServerSetThreadPoolParameters
Since threadpool increments the current number of threads according to current
load, i.e. how many jobs are waiting in the queue. The count however, is
constrained by max and min limits of workers. The logic of this new API works
like this:
    1) setting the minimum
        a) When the limit is increased, depending on the current number of
           threads, new threads are possibly spawned if the current number of
           threads is less than the new minimum limit
        b) Decreasing the minimum limit has no possible effect on the current
           number of threads
    2) setting the maximum
        a) Icreasing the maximum limit has no immediate effect on the current
           number of threads, it only allows the threadpool to spawn more
           threads when new jobs, that would otherwise end up queued, arrive.
        b) Decreasing the maximum limit may affect the current number of
           threads, if the current number of threads is less than the new
           maximum limit. Since there may be some ongoing time-consuming jobs
           that would effectively block this API from killing any threads.
           Therefore, this API is asynchronous with best-effort execution,
           i.e. the necessary number of workers will be terminated once they
           finish their previous job, unless other workers had already
           terminated, decreasing the limit to the requested value.
    3) setting priority workers
        - both increase and decrease in count of these workers have an
          immediate impact on the current number of workers, new ones will be
          spawned or some of them get terminated respectively.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-04-18 17:07:46 +02:00
Erik Skultety
caa16d3168 admin: Introduce virAdmServerGethreadPoolParameters
New API to retrieve current server workerpool specs. Since it uses typed
parameters, more specs to retrieve can be further included in the pool of
supported ones.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
2016-04-18 17:07:36 +02:00
Cole Robinson
9b69f02243 rpc: daemon: Fix virtlog/virtlock daemon reload
Trying to reload/SIGUSR1 virtlogd or virtlockd fails with:

error : virNetDaemonRun:747 : internal error: Not all servers restored, cannot run server

Commit 252610f7 changed the daemon state json to allow tracking
multiple servers. However it missed clearing dmn->srvObject after
the json is empty, like the previous code paths handled.  Later on in
virNewDaemonRun, dmn->srvObject is expected to be empty otherwise we
throw the above error.

https://bugzilla.redhat.com/show_bug.cgi?id=1311013
2016-04-13 11:07:21 -04:00
Pavel Hrdina
d713a6b120 build: add GCC 6.0 -Wlogical-op workaround
fdstream.c: In function 'virFDStreamWrite':
fdstream.c:390:29: error: logical 'or' of equal expressions [-Werror=logical-op]
        if (errno == EAGAIN || errno == EWOULDBLOCK) {
                            ^~

Fedora rawhide now uses gcc 6.0 and there is a bug with -Wlogical-op
producing false warnings.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69602

Use GCC pragma push/pop and ignore -Wlogical-op for GCC that supports
push/pop pragma and also has this bug.

Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
2016-04-11 12:09:14 +02:00
Fabiano Fidêncio
ab74e8763d build: workaround broken SASL header (again)
Compilation for xdg-app failed due to a buggy SASL headers present on
the used runtime (org.gnome.Sdk 3.18).

In file included from rpc/virnetsaslcontext.h:24:0,
                 from rpc/virnetsaslcontext.c:25:
/usr/include/sasl/sasl.h:230:38: error: unknown type name 'size_t'
 typedef void *sasl_realloc_t(void *, size_t);
                                      ^
/usr/include/sasl/sasl.h:235:5: error: unknown type name 'sasl_realloc_t'
     sasl_realloc_t *,

Use the same workaround as commit 1be3dfd did.

Signed-off-by: Ján Tomko <jtomko@redhat.com>
2016-03-31 17:36:49 +02:00
Michal Privoznik
9b48ef9c51 gendispatch: Avoid comparing signed and unsigned vars
The adminDispatchConnectListServers() function is generated by
our great perl script. However, it has a tiny flaw: if
adminConnectListServers() it calls fails, the control jumps onto
cleanup label where we try to free any list of servers built so
far. However, in the loop @i is unsigned (size_t) while @nresults
is signed (int). Currently, it does no harm because of the check
for @result being non-NULL. But if that ever changes in the
future, this bug will be hard to chase.

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-03-24 10:49:39 +01:00
Cole Robinson
ca0c06f400 rpc: wait longer for session daemon to start up
https://bugzilla.redhat.com/show_bug.cgi?id=1271183

We only wait 0.5 seconds for the session daemon to start up and present
its socket, which isn't sufficient for many users. Bump up the sleep
interval and retry amount so we wait for a total of 5.0 seconds.
2016-03-16 19:21:44 -04:00
Martin Kletzander
eeb1e6fe9f gendispatch: Use proper error for limit checking
All other places use VIR_ERR_RPC except this one, let's be consistent,
shall we?

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
5b7d09804c gendispatch: Support modern listing of more types
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
576c54f9b1 gendispatch: Remember the name of snapshot variable name
Until now, the script assumed that snapshot name is 'snap', but that's
going to change.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
1cefc14f69 gendispatch: Accept server as an argument
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
6de2ef5d35 gendispatch: Be able to generate multi-return values
Let's call it modern_ret_as_list as opposed to single_ret_as_list.  The
latter was able to return list of things.  However the new, more modern,
version came and it is used since listAllDomains till nowadays in
ListServers.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
30c4931d28 gendispatch: Cluster, don't capture if not needed
We were using parentheses for grouping admin|remote even though we didn't
need to capture what's in it.  That caused some changes to be greater
than needed and, to be honest, some confusion as well.  Let's use it as
it should be used.  It'll also make future changes more consistent.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
d4218ca64f Change virNetDaemonGetServerNames to virNetDaemonGetServers
For now it does not matter which ones we return as the code is similarly
complex, however it will fit in with other constructs in the future,
mainly when we will be able to generate dispatch helpers.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
6541a2b4ac daemon: Properly check for clients
virHashForEach() returns 0 if everything went nice, so our session
daemon was timing out even when there was a client connected.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1315606

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
cd0a980ee1 daemon: Set error for unknown server name
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
fd4f278e1a daemon: Get server name from the server itself
Since servers know their name, there is no need to supply such
information twice.  Also defeats inconsistencies.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Martin Kletzander
dad3b07814 server: Store server name in server object
At first I did not want to do this, but after trying to implement some
newer feaures in the admin API I realized we need that to make our lives
easier.  On the other hand they are not saved redundantly and the
virNetServer objects are still kept in a hash table.

Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-03-11 12:58:41 +01:00
Erik Skultety
1a07c2efb3 admin: Introduce adminDaemonConnectListServers API
This API is merely a convenience API, i.e. when managing clients connected to
daemon's servers, we should know (convenience) which server the specific client
is connected to. This implies a client-side representation of a server along
with a basic API to let the administrating client know what servers are actually
available on the daemon.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-02-17 12:46:34 +01:00
Erik Skultety
252610f7dd virnetdaemon: Store servers in a hash table
Since the daemon can manage and add (at fresh start) multiple servers,
we also should be able to add them from a JSON state file in case of a
daemon restart, so post exec restart support for multiple servers is also
provided. Patch also updates virnetdaemontest accordingly.

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
2016-02-17 12:46:34 +01:00
Michal Privoznik
862298a2e7 dbus: Don't unref NULL messages
Apparently we are not the only ones with dumb free functions
because dbus_message_unref() does not accept NULL either. But if
I were to vote, this one is even more evil. Instead of returning
an error just like we do it immediately dereference any pointer
passed and thus crash you app. Well done DBus!

  Program received signal SIGSEGV, Segmentation fault.
  [Switching to Thread 0x7f878ebda700 (LWP 31264)]
  0x00007f87be4016e5 in ?? () from /usr/lib64/libdbus-1.so.3
  (gdb) bt
  #0  0x00007f87be4016e5 in ?? () from /usr/lib64/libdbus-1.so.3
  #1  0x00007f87be3f004e in dbus_message_unref () from /usr/lib64/libdbus-1.so.3
  #2  0x00007f87bf6ecf95 in virSystemdGetMachineNameByPID (pid=9849) at util/virsystemd.c:228
  #3  0x00007f879761bd4d in qemuConnectCgroup (driver=0x7f87600a32a0, vm=0x7f87600c7550) at qemu/qemu_cgroup.c:909
  #4  0x00007f87976386b7 in qemuProcessReconnect (opaque=0x7f87600db840) at qemu/qemu_process.c:3386
  #5  0x00007f87bf6edfff in virThreadHelper (data=0x7f87600d5580) at util/virthread.c:206
  #6  0x00007f87bb602334 in start_thread (arg=0x7f878ebda700) at pthread_create.c:333
  #7  0x00007f87bb3481bd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
  (gdb) frame 2
  #2  0x00007f87bf6ecf95 in virSystemdGetMachineNameByPID (pid=9849) at util/virsystemd.c:228
  228         dbus_message_unref(reply);
  (gdb) p reply
  $1 = (DBusMessage *) 0x0

Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
2016-02-11 11:35:02 +01:00