I've noticed this while trying to compile libvirt on my arm box.
CC rpc/libvirt_net_rpc_server_la-virnetserverclient.lo
rpc/virnetserverclient.c: In function 'virNetServerClientNewPostExecRestart':
rpc/virnetserverclient.c:516:45: error: cast increases required alignment of target type [-Werror=cast-align]
(long long *) ×tamp) < 0) {
^
cc1: all warnings being treated as errors
Problem is, @timestap is defined as time_t which is 32 bits long,
and we are typecasting it to long long which is 64bits long.
Solution is to make @timestamp type of long long. At the same
time, we can make @conn_time in _virNetServerClient struct long
long too. There is no need for it to be type of time_t.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
In this function, @id is defined as unsigned long long. When
passing this variable to virJSONValueObjectGetNumberUlong(),
well address of this variable, it's typecasted to ull*. There
is no need for that. It's a same story with @nrequests_max.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This method just aggregates various client object attributes, like socket
address, connection type (RO/RW), and some TCP/TLS/UNIX identity in an atomic
manner.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
We do have a similar method, serving the same purpose, for TLS, but we lack
one for SASL. So introduce one, in order for other modules to be able to find
out, if a SASL session is active, or better said, that a SASL session exists
at all.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Our socket address format is in a rather non-standard format and that is
because sasl library requires the IP address and service to be delimited by a
semicolon. The string form is a completely internal matter, however once the
admin interfaces to retrieve client identity information are merged, we should
return the socket address string in a common format, e.g. format defined by
URI rfc-3986, i.e. the IP address and service are delimited by a colon and
in case of an IPv6 address, square brackets are added:
Examples:
127.0.0.1:1234
[::1]:1234
This patch changes our default format to the one described above, while adding
separate methods to request the non-standard SASL format using semicolon as a
delimiter.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Just like with server-related APIs, before any of client-based APIs can be
called, a reference to a client-side client object needs to be obtained. For
this purpose, a lookup method should exist. Apart from the client retrieval
logic, a new error code for non-existent client had to be added as well.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Although we document 6 types of transport that we support, internally we can
only differentiate between TCP, TLS, and UNIX transports only, since both SSH
and libssh2 transports, due to using netcat, behave in the exactly the same
way as a UNIX socket.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
For now, the list copy is done simply by locking the whole server, walking the
original and increasing the refcount on each object. We may want to change
the list to a lockable object (like list of domains) later in the future if
we discover some performance issues related to locking the whole server in
order to walk the whole list of clients, possibly issuing some 'ForEach'
callback.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Now that libvirt-admin supports another client-side object and provided that
we want to generate as many both client-side and server-side RPC dispatchers,
support for this needs to be added to gendispatch.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Besides ID, libvirt should provide several parameters to help the user
distinguish two clients from each other. One of them is the connection
timestamp. This patch also adds a testcase for proper JSON formatting of the
new attribute too (proper formatting of older clients that did not support
this attribute yet is included in the existing tests) - in order to
testGenerateJSON to work, a mock of time_t time(time_t *timer) needed to be
created.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Admin API needs a way of addressing specific clients. Unlike servers, which we
are happy to address by names both because its name reflects its purpose (to
some extent) and we only have two of them (so far), naming clients doesn't make
any sense, since a) each client is an anonymous, i.e. not recognized after a
disconnect followed by a reconnect, b) we can't predict what kind of requests
it's going to send to daemon, and c) the are loads of them comming and going,
so the only viable option is to use an ID which is of a reasonably wide data
type.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
There are two functions on the client that handle incoming stream
data. The first one virNetClientStreamQueuePacket() is a low
level function that just processes the incoming stream data from
the socket and stores it into an internal structure. This happens
in the client event loop therefore the shorter the callbacks are,
the better. The second function virNetClientStreamRecvPacket()
then handles copying data from internal structure into a client
provided buffer.
Change introduced in this commit makes just that: new queue for
incoming stream packets is introduced. Then instead of copying
data into intermediate internal buffer and then copying them into
user buffer, incoming stream messages are queue into the queue
and data is copied just once - in the upper layer function
virNetClientStreamRecvPacket(). In the end, there's just one
copying of data and therefore shorter event loop callback. This
should boost the performance which has proven to be the case in
my testing.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
This reverts commit d9c9e138f22c48626f719f880920e04c639e0177.
Unfortunately, things are going to be handled differently so this
commit must go.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
FD passing APIs like CreateXMLWithFiles or OpenGraphicsFD will leak
file descriptors. The user passes in an fd, which is dup()'d in
virNetClientProgramCall. The new fd is what is transfered to the
server virNetClientIOWriteMessage.
Once all the fds have been written though, the parent msg->fds list
is immediately free'd, so the individual fds are never closed.
This closes each FD as its send to the server, so all fds have been
closed by the time msg->fds is free'd.
https://bugzilla.redhat.com/show_bug.cgi?id=1159766
The stream serial number is the serial number of the RPC call
that initiated a data transfer. And as such can never be
negative. Moreover, when looking up internal state for a stream,
the serial numbers are compared. But hey, the serial number in
message header is unsigned too!
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Since threadpool increments the current number of threads according to current
load, i.e. how many jobs are waiting in the queue. The count however, is
constrained by max and min limits of workers. The logic of this new API works
like this:
1) setting the minimum
a) When the limit is increased, depending on the current number of
threads, new threads are possibly spawned if the current number of
threads is less than the new minimum limit
b) Decreasing the minimum limit has no possible effect on the current
number of threads
2) setting the maximum
a) Icreasing the maximum limit has no immediate effect on the current
number of threads, it only allows the threadpool to spawn more
threads when new jobs, that would otherwise end up queued, arrive.
b) Decreasing the maximum limit may affect the current number of
threads, if the current number of threads is less than the new
maximum limit. Since there may be some ongoing time-consuming jobs
that would effectively block this API from killing any threads.
Therefore, this API is asynchronous with best-effort execution,
i.e. the necessary number of workers will be terminated once they
finish their previous job, unless other workers had already
terminated, decreasing the limit to the requested value.
3) setting priority workers
- both increase and decrease in count of these workers have an
immediate impact on the current number of workers, new ones will be
spawned or some of them get terminated respectively.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
New API to retrieve current server workerpool specs. Since it uses typed
parameters, more specs to retrieve can be further included in the pool of
supported ones.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Trying to reload/SIGUSR1 virtlogd or virtlockd fails with:
error : virNetDaemonRun:747 : internal error: Not all servers restored, cannot run server
Commit 252610f7 changed the daemon state json to allow tracking
multiple servers. However it missed clearing dmn->srvObject after
the json is empty, like the previous code paths handled. Later on in
virNewDaemonRun, dmn->srvObject is expected to be empty otherwise we
throw the above error.
https://bugzilla.redhat.com/show_bug.cgi?id=1311013
fdstream.c: In function 'virFDStreamWrite':
fdstream.c:390:29: error: logical 'or' of equal expressions [-Werror=logical-op]
if (errno == EAGAIN || errno == EWOULDBLOCK) {
^~
Fedora rawhide now uses gcc 6.0 and there is a bug with -Wlogical-op
producing false warnings.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69602
Use GCC pragma push/pop and ignore -Wlogical-op for GCC that supports
push/pop pragma and also has this bug.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Compilation for xdg-app failed due to a buggy SASL headers present on
the used runtime (org.gnome.Sdk 3.18).
In file included from rpc/virnetsaslcontext.h:24:0,
from rpc/virnetsaslcontext.c:25:
/usr/include/sasl/sasl.h:230:38: error: unknown type name 'size_t'
typedef void *sasl_realloc_t(void *, size_t);
^
/usr/include/sasl/sasl.h:235:5: error: unknown type name 'sasl_realloc_t'
sasl_realloc_t *,
Use the same workaround as commit 1be3dfd did.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
The adminDispatchConnectListServers() function is generated by
our great perl script. However, it has a tiny flaw: if
adminConnectListServers() it calls fails, the control jumps onto
cleanup label where we try to free any list of servers built so
far. However, in the loop @i is unsigned (size_t) while @nresults
is signed (int). Currently, it does no harm because of the check
for @result being non-NULL. But if that ever changes in the
future, this bug will be hard to chase.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
https://bugzilla.redhat.com/show_bug.cgi?id=1271183
We only wait 0.5 seconds for the session daemon to start up and present
its socket, which isn't sufficient for many users. Bump up the sleep
interval and retry amount so we wait for a total of 5.0 seconds.
Let's call it modern_ret_as_list as opposed to single_ret_as_list. The
latter was able to return list of things. However the new, more modern,
version came and it is used since listAllDomains till nowadays in
ListServers.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
We were using parentheses for grouping admin|remote even though we didn't
need to capture what's in it. That caused some changes to be greater
than needed and, to be honest, some confusion as well. Let's use it as
it should be used. It'll also make future changes more consistent.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
For now it does not matter which ones we return as the code is similarly
complex, however it will fit in with other constructs in the future,
mainly when we will be able to generate dispatch helpers.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
virHashForEach() returns 0 if everything went nice, so our session
daemon was timing out even when there was a client connected.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1315606
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Since servers know their name, there is no need to supply such
information twice. Also defeats inconsistencies.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
At first I did not want to do this, but after trying to implement some
newer feaures in the admin API I realized we need that to make our lives
easier. On the other hand they are not saved redundantly and the
virNetServer objects are still kept in a hash table.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
This API is merely a convenience API, i.e. when managing clients connected to
daemon's servers, we should know (convenience) which server the specific client
is connected to. This implies a client-side representation of a server along
with a basic API to let the administrating client know what servers are actually
available on the daemon.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Since the daemon can manage and add (at fresh start) multiple servers,
we also should be able to add them from a JSON state file in case of a
daemon restart, so post exec restart support for multiple servers is also
provided. Patch also updates virnetdaemontest accordingly.
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
Apparently we are not the only ones with dumb free functions
because dbus_message_unref() does not accept NULL either. But if
I were to vote, this one is even more evil. Instead of returning
an error just like we do it immediately dereference any pointer
passed and thus crash you app. Well done DBus!
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f878ebda700 (LWP 31264)]
0x00007f87be4016e5 in ?? () from /usr/lib64/libdbus-1.so.3
(gdb) bt
#0 0x00007f87be4016e5 in ?? () from /usr/lib64/libdbus-1.so.3
#1 0x00007f87be3f004e in dbus_message_unref () from /usr/lib64/libdbus-1.so.3
#2 0x00007f87bf6ecf95 in virSystemdGetMachineNameByPID (pid=9849) at util/virsystemd.c:228
#3 0x00007f879761bd4d in qemuConnectCgroup (driver=0x7f87600a32a0, vm=0x7f87600c7550) at qemu/qemu_cgroup.c:909
#4 0x00007f87976386b7 in qemuProcessReconnect (opaque=0x7f87600db840) at qemu/qemu_process.c:3386
#5 0x00007f87bf6edfff in virThreadHelper (data=0x7f87600d5580) at util/virthread.c:206
#6 0x00007f87bb602334 in start_thread (arg=0x7f878ebda700) at pthread_create.c:333
#7 0x00007f87bb3481bd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) frame 2
#2 0x00007f87bf6ecf95 in virSystemdGetMachineNameByPID (pid=9849) at util/virsystemd.c:228
228 dbus_message_unref(reply);
(gdb) p reply
$1 = (DBusMessage *) 0x0
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Commit 8cd1d54 consolidates both daemon and remote driver typed param
serialization functions. The consolidation now enforces client to use
VIR_TYPED_PARAM_STRING_OKAY flag to properly serialize string parameters, which
server has used for quite some time now. And this caused an issue, since the
commit had not adjusted client remote calls appropriately, thus causing a
failure in blkiotune, numatune and migration APIs (as per Xen CI tests). This
patch adjusts both remote_driver.c and gendispatch.pl to properly address this
issue.
http://lists.xenproject.org/archives/html/xen-devel/2016-02/msg01012.html
Signed-off-by: Joao Martins <joao.m.martins@oracle.com>
Signed-off-by: Erik Skultety <eskultet@redhat.com>
Same as for deserializer, this method might get handy for admin one day.
The major reason for this patch is to stay consistent with idea, i.e.
when deserializer can be shared, why not serializer as well. The only
problem to be solved was that the daemon side serializer uses a code
snippet which handles sparse arrays returned by some APIs as well as
removes any string parameters that can't be returned to older clients.
This patch makes of the new virTypedParameterRemote datatype introduced
by one of the pvious patches.
Since the method is static to remote_driver, it can't even be used by our
daemon. Other than that, it would be useful to be able to use it with admin as
well. This patch uses the new virTypedParameterRemote datatype introduced in
one of previous patches.
Currently, the deserializer is hardcoded into remote_driver which makes
it impossible for admin to use it. One way to achieve a shared implementation
(besides moving the code to another module) would be pass @ret_params_val as a
void pointer as opposed to the remote_typed_param pointer and add a new extra
argument specifying which of those two protocols is being used and typecast
the pointer at the function entry. An example from remote_protocol:
struct remote_typed_param_value {
int type;
union {
int i;
u_int ui;
int64_t l;
uint64_t ul;
double d;
int b;
remote_nonnull_string s;
} remote_typed_param_value_u;
};
typedef struct remote_typed_param_value remote_typed_param_value;
struct remote_typed_param {
remote_nonnull_string field;
remote_typed_param_value value;
};
That would leave us with a bunch of if-then-elses that needed to be used across
the method. This patch takes the other approach using the new datatype
introduced in one of earlier commits.
In our generator for some code we put empty lines in the output
to separate blocks of code. However, in some cases we put couple
of spaces on the empty line too. It's not bug, it just isn't
nice.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
The generated output is dependent on perl hashtable ordering, which
gives different results for i686 and x86_64. Fix this by sorting
the hash keys before iterating over them
https://bugzilla.redhat.com/show_bug.cgi?id=1173641
On every socket connect(2) attempt we were re-launching session
libvirtd, up to 100 times in 5 seconds.
This understandably caused some weird load races and intermittent
qemu:///session startup failures
https://bugzilla.redhat.com/show_bug.cgi?id=1271183
When we autolaunch libvirtd for session URIs, we spin in a retry
loop waiting for the daemon to start and the connect(2) to succeed.
However if we exceed the retry count, we don't explicitly raise an
error, which can yield a slew of different error messages elsewhere
in the code.
Explicitly raise the last connect(2) failure if we run out of retries.
- Add some debugging
- Make the loop dependent only on retries
- Make it explicit that connect(2) success exits the loop
- Invert the error checking logic
When we are receiving data in smaller chunks it might happen that
virNetServerClientDispatchRead() will be called multiple times. And as
that happens, if it is a message that also transfer headers, we decode
the number of them every single time and, unfortunately, also allocate
the memory for them. That causes a leak, in the best scenario.
Best viewed with '-w'.
Signed-off-by: Martin Kletzander <mkletzan@redhat.com>
