src: warn if client hits the max requests limit

Since they are simply normal RPC messages, the keep alive packets are
subject to the "max_client_requests" limit just like any API calls.

Thus, if a client hits the 'max_client_requests' limit and all the
pending API calls take a long time to complete, it may result in
keep-alives firing and dropping the client connection.

This has been seen by a number of users with the default value of
max_client_requests=5, by issuing 5 concurrent live migration
operations.

By printing a warning message when this happens, admins will be alerted
to the fact that their active clients are exceeding the default client
requests limit.

Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This commit is contained in:
Daniel P. Berrangé 2022-08-24 16:15:24 +01:00
parent 83c6d80abc
commit 031878c236

View File

@ -93,6 +93,9 @@ struct _virNetServerClient
* throttling calculations */
size_t nrequests;
size_t nrequests_max;
/* True if we've warned about nrequests hittin
* the server limit already */
bool nrequests_warning;
/* Zero or one messages being received. Zero if
* nrequests >= max_clients and throttling */
virNetMessage *rx;
@ -1261,6 +1264,11 @@ static virNetMessage *virNetServerClientDispatchRead(virNetServerClient *client)
client->rx->buffer = g_new0(char, client->rx->bufferLength);
client->nrequests++;
}
} else if (!client->nrequests_warning) {
client->nrequests_warning = true;
VIR_WARN("Client hit max requests limit %zd. This may result "
"in keep-alive timeouts. Consider tuning the "
"max_client_requests server parameter", client->nrequests);
}
virNetServerClientUpdateEvent(client);