Libvirt the virtualization API

Note: this is the flat content of the web site

libvirt

what is libvirt?

Libvirt is a C toolkit to interract with the virtualization capabilities of recent versions of Linux (and other OSes). It is free software available under the GNU Lesser General Public License. Virtualization of the Linux Operating System means the ability to run multiple instances of Operating Systems concurently on a single hardware system where the basic resources are driven by a Linux instance. The library aim at providing long term stable C API initially for the Xen paravirtualization but should be able to integrate other virtualization mechanisms if needed.

Releases

Here is the list of official releases, however since it is early on in the development of libvirt, it is preferable when possible to just use the CVS version or snapshot, contact the mailing list and check the ChangeLog to gauge progresses.

0.1.0: Apr 10 2006

• building fixes: --with-xen-distdir option (Ronald Aigner), out of tree build and pkginfo cflag fix (Daniel Berrange)
• enhancement and fixes of the XML description format (David Lutterkort and Jim Fehlig)
• new APIs: for Node information and Reboot
• internal code cleanup: refactoring internals into a driver model, more error handling, structure sharing, thread safety and ref counting
• bug fixes: error message (Jim Meyering), error allocation in virsh (Jim Meyering), virDomainLookupByID (Jim Fehlig),
• documentation: updates on architecture, and format, typo fix (Jim Meyering)
• bindings: exception handling in examples (Jim Meyering), perl ones out of tree (Daniel Berrange)
• virsh: more options, create, nodeinfo (Karel Zak), renaming of some options (Karel Zak), use stderr only for errors (Karel Zak), man page (Andrew Puch)

0.0.6: Feb 28 2006

• add UUID lookup and extract API
• add error handling APIs both synchronous and asynchronous
• added minimal hook for error handling at the python level, improved the python bindings
• augment the documentation and tests to cover error handling

0.0.5: Feb 23 2006

• Added XML description parsing, dependance to libxml2, implemented the creation API virDomainCreateLinux()
• new APIs to lookup and name domain by UUID
• fixed the XML dump when using the Xend access
• Fixed a few more problem related to the name change
• Adding regression tests in python and examples in C
• web site improvement, extended the documentation to cover the XML format and Python API
• Added devhelp help for Gnome/Gtk programmers

0.0.4: Feb 10 2006

• Fix various bugs introduced in the name change

0.0.3: Feb 9 2006

• Switch name from from 'libvir' to libvirt
• Starting infrastructure to add code examples
• Update of python bindings for completeness

0.0.2: Jan 29 2006

• Update of the documentation, web site redesign (Diana Fong)
• integration of HTTP xend RPC based on libxend by Anthony Liquori for most operations
• Adding Save and Restore APIs
• extended the virsh command line tool (Karel Zak)
• remove xenstore transactions (Anthony Liguori)
• fix the Python bindings bug when domain and connections where freed

0.0.1: Dec 19 2005

• First release
• Basic management of existing Xen domains
• Minimal autogenerated Python bindings

Introduction

Libvirt is a C toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes), but libvirt won't try to provide all possible interfaces for interacting with the virtualization features.

To avoid ambiguity about the terms used here here are the definitions for some of the specific concepts used in libvirt documentation:

• a node is a single physical machine
• an hypervisor is a layer of software allowing to virtualize a node in a set of virtual machines with possibly different configurations than the node itself
• a domain is an instance of an operating system running on a virtualized machine provided by the hypervisor

Now we can define the goal of libvirt: to provide the lowest possible generic and stable layer to manage domains on a node.

This implies the following:

• the API should not be targetted to a single virtualization environment though Xen is the current default, which also means that some very specific capabilities which are not generic enough may not be provided as libvirt APIs
• the API should allow to do efficiently and cleanly all the operations needed to manage domains on a node
• the API will not try to provide hight level multi-nodes management features like load balancing, though they could be implemented on top of libvirt
• stability of the API is a big concern, libvirt should isolate applications from the frequent changes expected at the lower level of the virtualization framework

So libvirt should be a building block for higher level management tools and for applications focusing on virtualization of a single node (the only exception being domain migration between node capabilities which may need to be added at the libvirt level). Where possible libvirt should be extendable to be able to provide the same API for remote nodes, however this is not the case at the moment, the code currently handle only local node accesses.

libvirt architecture

This is in a large part Xen specific since this is the only hypervisor supported at the moment

When running in a Xen environment, programs using libvirt have to execute in "Domain 0", which is the primary Linux OS loaded on the machine. That OS kernel provides most if not all of the actual drivers used by the set of domains. It also runs the Xen Store, a database of informations shared by the hypervisor, the kernels, the drivers and the xen daemon. Xend. The xen daemon supervise the control and execution of the sets of domains. The hypervisor, drivers, kernels and daemons communicate though a shared system bus implemented in the hypervisor. The figure below tries to provide a view of this environment:

The library can be initialized in 2 ways depending on the level of priviledge of the embedding program. If it runs with root access, virConnectOpen() can be used, it will use three different ways to connect to the Xen infrastructure:

• a connection to the Xen Daemon though an HTTP RPC layer
• a read/write connection to the Xen Store
• use Xen Hypervisor calls

The library will usually interract with the Xen daemon for any operation changing the state of the system, but for performance and accuracy reasons may talk directly to the hypervisor when gathering state informations at least when possible (i.e. when the running program using libvirt has root priviledge access).

If it runs without root access virConnectOpenReadOnly() should be used to connect to initialize the library. It will try to open the read-only socket /var/run/xenstored/socket_ro to connect to the Xen Store and also try to use the RPC to the Xen daemon. In this case use of hypervisor calls and write to the Xen Store will not be possible, restraining the amount of APIs available and slowing down information gathering about domains.

Internal architecture

As the previous section explains, libvirt can communicate using different channels with the current hypervisor, and should also be able to use different kind of hypervisor. To simplify the internal design, code, ease maintainance and simplify the support of other virtualization engine the internals have been structured as one core component, the libvirt.c module acting as a front-end for the library API and a set of hypvisor drivers defining a common set of routines. That way the Xen Daemon accces, the Xen Store one, the Hypervisor hypercall are all isolated in separate C modules implementing at least a subset of the common operations defined by the drivers present in driver.h:

• xend_internal: implements the driver functions though the Xen Daemon
• xs_internal: implements the subset of the driver availble though the Xen Store
• xen_internal: provide the implementation of the functions possible via direct hypervisor access

Note that a given driver may only implement a subset of those functions, for example saving a domain state to disk and restoring it is only possible though the Xen Daemon, on the other hand all interfaces allow to query the runtime state of a given domain.

Downloads

The latest versions of libvirt can be found on the libvirt.org server ( HTTP, FTP). You will find there the released versions as well as snapshot tarballs updated from CVS head every hour

Anonymous CVS is also available, first register onto the server:

cvs -d :pserver:anoncvs@libvirt.org:2401/data/cvs login

it will request a password, enter anoncvs. Then you can checkout the development tree with:

cvs -d :pserver:anoncvs@libvirt.org:2401/data/cvs co libvirt

Use ./autogen.sh to configure the local checkout, then make and make install, as usual. All normal cvs commands are now available except commiting to the base.

XML Format

The library use an XML format to describe domains, as input to virDomainCreateLinux() and as the output of virDomainGetXMLDesc(), the following is an example of the format as returned by the shell command virsh xmldump fc4 , where fc4 was one of the running domains:

<domain type='xen' id='18'>
  <name>fc4</name>
  <os>
    <type>linux</type>
    <kernel>/boot/vmlinuz-2.6.15-1.43_FC5guest</kernel>
    <initrd>/boot/initrd-2.6.15-1.43_FC5guest.img</initrd>
    <root>/dev/sda1</root>
    <cmdline> ro selinux=0 3</cmdline>
  </os>
  <memory>131072</memory>
  <vcpu>1</vcpu>
  <devices>
    <disk type='file'>
      <source file='/u/fc4.img'/>
      <target dev='sda1'/>
    </disk>
    <interface type='bridge'>
      <source bridge='xenbr0'/>
      <mac address='aa:00:00:00:00:11'/>
      <script path='/etc/xen/scripts/vif-bridge'/>
    </interface>
  </devices>
</domain>

The root element must be called domain with no namespace, the type attribute indicates the kind of hypervisor used, 'xen' is the default value. The id attribute gives the domain id at runtime (not however that this may change, for example if the domain is saved to disk and restored). The domain has a few children whose order is not significant:

• name: the domain name, preferably ASCII based
• memory: the maximum memory allocated to the domain in kilobytes
• vcpu: the number of virtual cpu configured for the domain
• os: a block describing the Operating System, its content will be dependant on the OS type
  • type: indicate the OS type, always linux at this point
  • kernel: path to the kernel on the Domain 0 filesystem
  • initrd: an optional path for the init ramdisk on the Domain 0 filesystem
  • cmdline: optional command line to the kernel
  • root: the root filesystem from the guest viewpoint, it may be passed as part of the cmdline content too
• devices: a list of disk and interface descriptions in no special order

The format of the devices and their type may grow over time, but the following should be sufficient for basic use:

A disk device indicates a block device, it can have two values for the type attribute either 'file' or 'block' corresponding to the 2 options availble at the Xen layer. It has two mandatory children, and one optional one in no specific order:

• source with a file attribute containing the path in Domain 0 to the file or a dev attribute if using a block device, containing the device name ('hda5' or '/dev/hda5')
• target indicates in a dev attribute the device where it is mapped in the guest
• readonly an optional empty element indicating the device is read-only

An interface element describes a network device mapped on the guest, it also has a type whose value is currently 'bridge', it also have a number of children in no specific order:

• source: indicating the bridge name
• mac: the optional mac address provided in the address attribute
• ip: the optional IP address provided in the address attribute
• script: the script used to bridge the interfcae in the Domain 0
• target: and optional target indicating the device name.

Life cycle actions for the domain can also be expressed in the XML format, they drive what should be happening if the domain crashes, is rebooted or is poweroff. There is various actions possible when this happen:

• destroy: The domain is cleaned up (that's the default normal processing in Xen)
• restart: A new domain is started in place of the old one with the same configuration parameters
• preserve: The domain will remain in memory until it is destroyed manually, it won't be running but allows for post-mortem debugging
• rename-restart: a variant of the previous one but where the old domain is renamed before being saved to allow a restart

The following could be used for a Xen production system:

<domain>
  ...
  <on_reboot>restart</on_reboot>
  <on_poweroff>destroy</on_poweroff>
  <on_crash>rename-restart</on_crash>
  ...
</domain>

While the format may be extended in various ways as support for more hypervisor types and features are added, it is expected that this core subset will remain functional in spite of the evolution of the library.

Binding for Python

Libvirt comes with direct support for the Python language (just make sure you installed the libvirt-python package if not compiling from sources). Also note that Daniel Berrange provides bindings for Perl too.

The Python binding should be complete and are mostly automatically generated from the formal description of the API in xml. The bindings are articulated around 2 classes virConnect and virDomain mapping to the C types. Functions in the C API taking either type as argument then becomes methods for the classes, their name is just stripped from the virConnect or virDomain(Get) prefix and the first letter gets converted to lower case, for example the C functions:

int virConnectNumOfDomains (virConnectPtr conn);

int virDomainSetMaxMemory (virDomainPtr domain, unsigned long memory);

become

virConn::numOfDomains(self)

virDomain::setMaxMemory(self, memory)

This process is fully automated, you can get a summary of the conversion in the file libvirtclass.txt present in the python dir or in the docs.There is a couple of function who don't map directly to their C counterparts due to specificities in their argument conversions:

• virConnectListDomains is replaced by virDomain::listDomainsID(self) which returns a list of the integer ID for the currently running domains
• virDomainGetInfo is replaced by virDomain::info() which returns a list of
  1. state: one of the state values (virDomainState)
  2. maxMemory: the maximum memory used by the domain
  3. memory: the current amount of memory used by the domain
  4. nbVirtCPU: the number of virtual CPU
  5. cpuTime: the time used by the domain in nanoseconds

So let's look at a simple example inspired from the basic.py test found in python/tests/ in the source tree:

import libvirt
import sys

conn = libvirt.openReadOnly(None)
if conn == None:
    print 'Failed to open connection to the hypervisor'
    sys.exit(1)

try:
    dom0 = conn.lookupByName("Domain-0")
except:
    print 'Failed to find the main domain'
    sys.exit(1)

print "Domain 0: id %d running %s" % (dom0.ID(), dom0.OSType())
print dom0.info()

There is not much to comment about it, it really is a straight mapping from the C API, the only points to notice are:

• the import of the module called libvirt
• getting a connection to the hypervisor, in that case using the openReadOnly function allows the code to execute as a normal user.
• getting an object representing the Domain 0 using lookupByName
• if the domain is not found a libvirtError exception will be raised
• extracting and printing some informations about the domain using various methods associated to the virDomain class.

Handling of errors

The main goals of libvirt when it comes to error handling are:

• provide as much detail as possible
• provide the informations as soon as possible
• dont force the library user into one style of error handling

As result the library provide both synchronous, callback based and asynchronous error reporting. When an error happens in the library code the error is logged, allowing to retrieve it later and if the user registered an error callback it will be called synchronously. Once the call to libvirt ends the error can be detected by the return value and the full information for the last logged error can be retrieved.

To avoid as much as prossible troubles with a global variable in a multithreaded environment, libvirt will associate when possible the errors to the current connection they are related to, that way the error is stored in a dynamic structure which can be made thread specific. Error callback can be set specifically to a connection with

So error handling in the code is the following:

1. if the error can be associated to a connection for example when failing to look up a domain
   1. if there is a callback associated to the connection set with virConnSetErrorFunc, call it with the error informations
   2. otherwise if there is a global callback set with virSetErrorFunc, call it with the error information
   3. otherwise call virDefaultErrorFunc which is the default error function of the library issuing the error on stderr
   4. save the error in the connection for later retrieval with virConnGetLastError
2. otherwise like when failing to create an hypervisor connection:
   1. if there is a global callback set with virSetErrorFunc, call it with the error information
   2. otherwise call virDefaultErrorFunc which is the default error function of the library issuing the error on stderr
   3. save the error in the connection for later retrieval with virGetLastError

In all cases the error informations are provided as a virErrorPtr pointer to read-only structure virError containing the following fields:

• code: an error number from the virErrorNumber enum
• domain: an enum indicating which part of libvirt raised the error see virErrorDomain
• level: the error level, usually VIR_ERR_ERROR, though there is room for warnings like VIR_ERR_WARNING
• message: the full human-readable formatted string of the error
• conn: if available a pointer to the virConnectPtr connection to the hypervisor where this happened
• dom: if available a pointer to the virDomainPtr domain targetted in the operation

and then extra raw informations about the error which may be initialized to 0 or NULL if unused

• str1, str2, str3: string informations, usually str1 is the error message format
• int1, int2: integer informations

So usually, setting up specific error handling with libvirt consist of registering an handler with with virSetErrorFunc or with virConnSetErrorFunc, chech the value of the code value, take appropriate action, if needed let libvirt print the error on stderr by calling virDefaultErrorFunc. For asynchronous error handing, set such a function doing nothing to avoid the error being reported on stderr, and call virConnGetLastError or virGetLastError when an API call returned an error value. It can be a good idea to use virResetError or virConnResetLastError once an error has been processed fully.

At the python level, there only a global reporting callback function at this point, see the error.py example about it:

def handler(ctxt, err):
    global errno

    #print "handler(%s, %s)" % (ctxt, err)
    errno = err

libvirt.registerErrorHandler(handler, 'context') 

the second argument to the registerErrorHandler function is passed as the fist argument of the callback like in the C version. The error is a tuple containing the same field as a virError in C, but cast to Python.

FAQ

Table of Contents:

• License(s)
• Installation
• Compilation
• Developer corner

License(s)

1. Licensing Terms for libvirt

   libvirt is released under the GNU Lesser General Public License, see the file COPYING.LIB in the distribution for the precise wording. The only library that libvirt depends upon is the Xen store access library which is also licenced under the LGPL.

2. Can I embed libvirt in a proprietary application ?

   Yes. The LGPL allows you to embed libvirt into a proprietary application. It would be graceful to send-back bug fixes and improvements as patches for possible incorporation in the main development tree. It will decrease your maintainance costs anyway if you do so.

Installation

1. Where can I get libvirt ?

   The original distribution comes from ftp://libvirt.org/libvirt/.

2. I can't install the libvirt/libvirt-devel RPM packages due to failed dependencies

   The most generic solution is to re-fetch the latest src.rpm , and rebuild it locally with

   rpm --rebuild libvirt-xxx.src.rpm.

   If everything goes well it will generate two binary rpm packages (one providing the shared libs and virsh, and the other one, the -devel package, providing includes, static libraries and scripts needed to build applications with libvirt that you can install locally.

   One can also rebuild the RPMs from a tarball:

   rpmbuild -ta libdir-xxx.tar.gz

   Or from a configured tree with:

   make rpm

3. Failure to use the API for non-root users

   Large parts of the API may only be accessible with root priviledges, however the read only access to the xenstore data doesnot have to be forbidden to user, at least for monitoring purposes. If "virsh dominfo" fails to run as an user, change the mode of the xenstore read-only socket with:

   chmod 666 /var/run/xenstored/socket_ro

   and also make sure that the Xen Daemon is running correctly with local HTTP server enabled, this is defined in /etc/xen/xend-config.sxp which need the following line to be enabled:

   (xend-http-server yes)

   If needed restart the xend daemon after making the change with the following command run as root:

   service xend restart

Compilation

1. What is the process to compile libvirt ?

   As most UNIX libraries libvirt follows the "standard":

   gunzip -c libvirt-xxx.tar.gz | tar xvf -

   cd libvirt-xxxx

   ./configure --help

   to see the options, then the compilation/installation proper

   ./configure [possible options]

   make

   make install

   At that point you may have to rerun ldconfig or a similar utility to update your list of installed shared libs.

2. What other libraries are needed to compile/install libvirt ?

   Libvirt requires libxenstore, which is usually provided by the xen packages as well as the public headers to compile against libxenstore.

3. I use the CVS version and there is no configure script

   The configure script (and other Makefiles) are generated. Use the autogen.sh script to regenerate the configure script and Makefiles, like:

   ./autogen.sh --prefix=/usr --disable-shared

Developer corner

1. Troubles compiling or linking programs using libvirt

   To simplify the process of reusing the library, libvirt comes with pkgconfig support, which can be used directly from autoconf support or via the pkg-config command line tool, like:

   pkg-config libvirt --libs

Reporting bugs and getting help

There is a mailing-list libvir-list@redhat.com for libvirt, with an on-line archive. Please subscribe to this list before posting by visiting the associated Web page and follow the instructions. Patches with explanations and provided as attachments are really appreciated and will be discussed on the mailing list. If possible generate the patches by using cvs diff -u in a CVS checkout.

We expect to use Red Hat Bugzilla to track bugs for libvirt, though there isn't a libvirt software module defined yet, in the meantime use the mailing-list, thanks !.