Remote support

NB. Remote support is available only as a series of patches posted on libvir-list against libvirt CVS. It is only for experimental use at the moment. — Richard Jones, 2007-04-18.

Libvirt allows you to access hypervisors running on remote machines through authenticated and encrypted connections.

Basic usage

On the remote machine, libvirtd should be running. See the section on configuring libvirtd for more information.

To tell libvirt that you want to access a remote resource, you should supply a hostname in the normal URI that is passed to virConnectOpen (or virsh -c ...). For example, if you normally use qemu:///system to access the system-wide QEMU daemon, then to access the system-wide QEMU daemon on a remote machine called oirase you would use qemu://oirase/system.

The section on remote URIs describes in more detail these remote URIs.

From an API point of view, apart from the change in URI, the API should behave the same. For example, ordinary calls are routed over the remote connection transparently, and values or errors from the remote side are returned to you as if they happened locally. Some differences you may notice:

Additional errors can be generated, specifically ones relating to failures in the remote transport itself.
Remote calls are handled synchronously, so they will be much slower than, say, direct hypervisor calls.

Transports

Remote libvirt supports a range of transports:

tls: TLS 1.0 (SSL 3.1) authenticated and encrypted TCP/IP socket, usually listening on a public port number. To use this you will need to generate client and server certificates. The standard port is 16514.
unix: Unix domain socket. Since this is only accessible on the local machine, it is not encrypted, and uses Unix permissions or SELinux for authentication. The standard socket names are /var/run/libvirt/libvirt-sock and /var/run/libvirt/libvirt-sock-ro (the latter for read-only connections).
ssh: Transported over an ordinary ssh (secure shell) connection. Requires Netcat (nc) installed on the remote machine, and the remote libvirtd should be listening on the unix transport. You should use some sort of ssh key management (eg. ssh-agent) otherwise programs which use this transport will stop to ask for a password.
ext: Any external program which can make a connection to the remote machine by means outside the scope of libvirt.
tcp: Unencrypted TCP/IP socket. Not recommended for production use, this is normally disabled, but an administrator can enable it for testing or use over a trusted network. The standard port is 16509.

The default transport, if no other is specified, is tls.

Remote URIs

Remote URIs have the general form ("[...]" meaning an optional part):

driver[+transport]://[username@]hostname[:port]/[path][?extraparameters]

Either the transport or the hostname must be given in order to distinguish this from a local URI.

Some examples:

xen+ssh://rjones@towada/
— Connect to a remote Xen hypervisor on host towada using ssh transport and ssh username rjones.
xen://towada/
— Connect to a remote Xen hypervisor on host towada using TLS.
xen://towada/?no_verify=1
— Connect to a remote Xen hypervisor on host towada using TLS. Do not verify the server's certificate.
qemu+unix:///system?socket=/opt/libvirt/run/libvirt/libvirt-sock
— Connect to the local qemu instances over a non-standard Unix socket (the full path to the Unix socket is supplied explicitly in this case).
test+tcp://localhost:5000/default
— Connect to a libvirtd daemon offering unencrypted TCP/IP connections on localhost port 5000 and use the test driver with default settings.

Extra parameters

Extra parameters can be added to remote URIs as part of the query string (the part following ?). Remote URIs understand the extra parameters shown below. Any others are passed unmodified through to the back end. Note that parameter values must be URI-escaped.

Name	Transports	Meaning
`name`	any transport	The name passed to the remote virConnectOpen function. The name is normally formed by removing transport, hostname, port number, username and extra parameters from the remote URI, but in certain very complex cases it may be better to supply the name explicitly.
		Example: `name=qemu:///system`
`command`	ssh, ext	The external command. For ext transport this is required. For ssh the default is `ssh`. The PATH is searched for the command.
		Example: `command=/opt/openssh/bin/ssh`
`socket`	unix, ssh	The path to the Unix domain socket, which overrides the compiled-in default. For ssh transport, this is passed to the remote netcat command (see next).
		Example: `socket=/opt/libvirt/run/libvirt/libvirt-sock`
`netcat`	ssh	The name of the netcat command on the remote machine. The default is `nc`. For ssh transport, libvirt constructs an ssh command which looks like: command -p port [-l username] hostname netcat -U socket where port, username, hostname can be specified as part of the remote URI, and command, netcat and socket come from extra parameters (or sensible defaults).
		Example: `netcat=/opt/netcat/bin/nc`
`no_verify`	tls	If set to a non-zero value, this disables client checks of the server's certificate. Note that to disable server checks of the client's certificate or IP address you must change the libvirtd configuration.
		Example: `no_verify=1`

Generating TLS certificates

This section to follow.

libvirtd configuration

This section to follow.

IPv6 support

IPv6 has received some limited testing and should work. Problems with libvirt and IPv6 should be reported as bugs.

Limitations

Remote storage: To be fully useful, particularly for creating new domains, it should be possible to enumerate and provision storage on the remote machine. This is currently in the design phase.
Migration: We expect libvirt will support migration, and obviously remote support is what makes migration worthwhile. This is also in the design phase. Issues to discuss include which path the migration data should follow (eg. client to client direct, or client to server to client) and security.
Fine-grained authentication: libvirt in general, but in particular the remote case should support more fine-grained authentication for operations, rather than just read-write/read-only as at present.

Please come and discuss these issues and more on the mailing list.

Implementation notes

The current implementation uses XDR-encoded packets with a simple remote procedure call implementation which also supports asynchronous messaging and asynchronous and out-of-order replies, although these latter features are not used at the moment.

The implementation should be considered strictly internal to libvirt and subject to change at any time without notice. If you wish to talk to libvirtd, link to libvirt. If there is a problem that means you think you need to use the protocol directly, please first discuss this on the mailing list.

The messaging protocol is described in qemud/remote_protocol.x.

Authentication and encryption (for TLS) is done using GnuTLS and the RPC protocol is unaware of this layer.

Protocol messages are sent using a simple 32 bit length word (encoded XDR int) followed by the message header (XDR remote_message_header) followed by the message body. The length count includes the length word itself, and is measured in bytes. Maximum message size is REMOTE_MESSAGE_MAX and to avoid denial of services attacks on the XDR decoders strings are individually limited to REMOTE_STRING_MAX bytes. In the TLS case, messages may be split over TLS records, but a TLS record cannot contain parts of more than one message. In the common RPC case a single REMOTE_CALL message is sent from client to server, and the server then replies synchronously with a single REMOTE_REPLY message, but other forms of messaging are also possible.

The protocol contains support for multiple program types and protocol versioning, modelled after SunRPC.