The libvirt VMware ESX driver can manage VMware ESX/ESXi 3.5/4.x/5.x and VMware GSX 2.0, also called VMware Server 2.0, and possibly later versions. Since 0.8.3 the driver can also connect to a VMware vCenter 2.5/4.x/5.x (VPX).
None. Any out-of-the-box installation of VPX/ESX(i)/GSX should work. No preparations are required on the server side, no libvirtd must be installed on the ESX server. The driver uses version 2.5 of the remote, SOAP based VMware Virtual Infrastructure API (VI API) to communicate with the ESX server, like the VMware Virtual Infrastructure Client (VI client) does. Since version 4.0 this API is called VMware vSphere API.
Some example remote connection URIs for the driver are:
vpx://example-vcenter.com/dc1/srv1 (VPX over HTTPS, select ESX server 'srv1' in datacenter 'dc1') esx://example-esx.com (ESX over HTTPS) gsx://example-gsx.com (GSX over HTTPS) esx://example-esx.com/?transport=http (ESX over HTTP) esx://example-esx.com/?no_verify=1 (ESX over HTTPS, but doesn't verify the server's SSL certificate)
Note: In contrast to other drivers, the ESX driver is
a client-side-only driver. It connects to the ESX server using HTTP(S).
Therefore, the remote transport mechanism
provided by the remote driver and libvirtd will not work, and you
cannot use URIs like esx+ssh://example.com.
URIs have this general form ([...] marks an optional part).
type://[username@]hostname[:port]/[[folder/...]datacenter/[folder/...][cluster/]server][?extraparameters]
The type:// is either esx:// or
gsx:// or vpx:// since 0.8.3.
The driver selects the default port depending on the type://.
For esx:// and vpx:// the default HTTPS port
is 443, for gsx:// it is 8333.
If the port parameter is given, it overrides the default port.
A vpx:// connection is currently restricted to a single
ESX server. This might be relaxed in the future. The path part of the
URI is used to specify the datacenter and the ESX server in it. If the
ESX server is part of a cluster then the cluster has to be specified too.
An example: ESX server example-esx.com is managed by
vCenter example-vcenter.com and part of cluster
cluster1. This cluster is part of datacenter dc1.
vpx://example-vcenter.com/dc1/cluster1/example-esx.com
Datacenters and clusters can be organized in folders, those have to be specified as well. The driver can handle folders since 0.9.7.
vpx://example-vcenter.com/folder1/dc1/folder2/example-esx.com
Extra parameters can be added to a URI as part of the query string
(the part following ?). A single parameter is formed by a
name=value pair. Multiple parameters are separated by
&.
?no_verify=1&auto_answer=1&proxy=socks://example-proxy.com:23456
The driver understands the extra parameters shown below.
| Name | Values | Meaning |
|---|---|---|
transport
|
http or https
|
Overrides the default HTTPS transport. For esx://
and vpx:// the default HTTP port is 80, for
gsx:// it is 8222.
|
vcenter
|
Hostname of a VMware vCenter or *
|
In order to perform a migration the driver needs to know the
VMware vCenter for the ESX server. If set to *,
the driver connects to the vCenter known to the ESX server.
This parameter in useful when connecting to an ESX server only.
|
no_verify
|
0 or 1
|
If set to 1, this disables libcurl client checks of the server's SSL certificate. The default value it 0. See the Certificates for HTTPS section for details. |
auto_answer
|
0 or 1
|
If set to 1, the driver answers all questions with the default answer. If set to 0, questions are reported as errors. The default value it 0. Since 0.7.5. |
proxy
|
[type://]hostname[:port]
|
Allows to specify a proxy for HTTP and HTTPS communication.
Since 0.8.2.
The optional type part may be one of:
http, socks, socks4,
socks4a or socks5. The default is
http and socks is synonymous for
socks5. The optional port allows to
override the default port 1080.
|
In order to perform any useful operation the driver needs to log into
the ESX server. Therefore, only virConnectOpenAuth can be
used to connect to an ESX server, virConnectOpen and
virConnectOpenReadOnly don't work.
To log into an ESX server or vCenter the driver will request
credentials using the callback passed to the
virConnectOpenAuth function. The driver passes the
hostname as challenge parameter to the callback. This enables the
callback to distinguish between requests for ESX server and vCenter.
Note: During the ongoing driver development, testing
is done using an unrestricted root account. Problems may
occur if you use a restricted account. Detailed testing with restricted
accounts has not been done yet.
By default the ESX driver uses HTTPS to communicate with an ESX server. Proper HTTPS communication requires correctly configured SSL certificates. This certificates are different from the ones libvirt uses for secure communication over TLS to a libvirtd one a remote server.
By default the driver tries to verify the server's SSL certificate using the CA certificate pool installed on your client computer. With an out-of-the-box installed ESX server this won't work, because a newly installed ESX server uses auto-generated self-signed certificates. Those are singed by a CA certificate that is typically not known to your client computer and libvirt will report an error like this one:
error: internal error curl_easy_perform() returned an error: Peer certificate cannot be authenticated with known CA certificates (60)
Where are two ways to solve this problem:
no_verify=1 extra parameter
to disable server certificate verification.
There are also other causes for connection problems than the HTTPS certificate related ones.
$ virsh -c esx+tcp://example.com/ error: unable to connect to libvirtd at 'example.com': Connection refused
$ virsh -c esx+tls://example.com/ error: Cannot access CA certificate '/etc/pki/CA/cacert.pem': No such file or directory
$ virsh -c esx+ssh://example.com/ error: cannot recv data: ssh: connect to host example.com port 22: Connection refused
$ virsh -c esx+ssh://example.com/ error: cannot recv data: Resource temporarily unavailable
$ virsh -c esx://example.com/ error: Cannot access CA certificate '/etc/pki/CA/cacert.pem': No such file or directory
Don't let this error message confuse you. Setting up certificates as described on the remote transport mechanism page does not help, as this is not a certificate related problem.
To fix this problem you need to update your libvirt to 0.7.0 or newer. You may also see this error when you use a libvirt version that contains the ESX driver but you or your distro disabled the ESX driver during compilation. Since 0.8.3 the error message has been improved in this case:
$ virsh -c esx://example.com/ error: invalid argument in libvirt was built without the 'esx' driver
Some methods of the VI API start tasks, for example
PowerOnVM_Task(). Such tasks may be blocked by questions
if the ESX server detects an issue with the domain that requires user
interaction. The ESX driver cannot prompt the user to answer a
question, libvirt doesn't have an API for something like this.
The VI API provides the AnswerVM() method to
programmatically answer a questions. So the driver has two options
how to handle such a situation: either answer the questions with the
default answer or report the question as an error and cancel the
blocked task if possible. The
auto_answer query parameter
controls the answering behavior.
There are several specialties in the domain XML config for ESX domains.
There are some restrictions for some values of the domain XML config. The driver will complain if this restrictions are violated.
00:0c:29 and
00:50:56. Since 0.7.6
arbitrary MAC addresses are supported.
Storage is managed in datastores. VMware uses a special path format to reference files in a datastore. Basically, the datastore name is put into squared braces in front of the path.
[datastore] directory/filename
To define a new domain the driver converts the domain XML into a VMware VMX file and uploads it to a datastore known to the ESX server. Because multiple datastores may be known to an ESX server the driver needs to decide to which datastore the VMX file should be uploaded. The driver deduces this information from the path of the source of the first file-based harddisk listed in the domain XML.
VMware has registered two MAC address prefixes for domains:
00:0c:29 and 00:50:56. These prefixes are
split into ranges for different purposes.
| Range | Purpose |
|---|---|
00:0c:29:00:00:00 - 00:0c:29:ff:ff:ff
|
An ESX server autogenerates MAC addresses from this range if the VMX file doesn't contain a MAC address when trying to start a domain. |
00:50:56:00:00:00 - 00:50:56:3f:ff:ff
|
MAC addresses from this range can by manually assigned by the user in the VI client. |
00:50:56:80:00:00 - 00:50:56:bf:ff:ff
|
A VI client autogenerates MAC addresses from this range for newly defined domains. |
The VMX files generated by the ESX driver always contain a MAC address,
because libvirt generates a random one if an interface element in the
domain XML file lacks a MAC address.
Since 0.7.6 the ESX driver sets the prefix
for generated MAC addresses to 00:0c:29. Before 0.7.6
the 00:50:56 prefix was used. Sometimes this resulted in
the generation of out-of-range MAC address that were rejected by the
ESX server.
Also since 0.7.6 every MAC address outside this ranges can be used. For such MAC addresses the ESX server-side check is disabled in the VMX file to stop the ESX server from rejecting out-of-predefined-range MAC addresses.
ethernet0.checkMACAddress = "false"
VMware ESX supports different models of SCSI controllers and network cards.
auto.vmdk file and use it. Autodetection fails when a
SCSI controller has multiple disks attached and the SCSI controller
models referenced in the .vmdk files are inconsistent.
Since 0.8.3
buslogiclsilogiclsisas1068vmpvscsiHere a domain XML snippet:
...
<disk type='file' device='disk'>
<source file='[local-storage] Fedora11/Fedora11.vmdk'/>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' unit='0'/>
</disk>
<controller type='scsi' index='0' model='lsilogic'/>
...
The controller element is supported since 0.8.2.
Prior to this <driver name='lsilogic'/> was abused to
specify the SCSI controller model. This attribute usage is deprecated now.
...
<disk type='file' device='disk'>
<driver name='lsilogic'/>
<source file='[local-storage] Fedora11/Fedora11.vmdk'/>
<target dev='sda' bus='scsi'/>
</disk>
...
vlancevmxnet, vmxnet2, vmxnet3e1000Here a domain XML snippet:
...
<interface type='bridge'>
<mac address='00:50:56:25:48:c7'/>
<source bridge='VM Network'/>
<model type='e1000'/>
</interface>
...
The ESX driver currently supports a native config format known as
vmware-vmx to handle VMware VMX configs.
The virsh domxml-from-native provides a way to convert an
existing VMware VMX config into a domain XML config that can then be
used by libvirt.
$ cat > demo.vmx << EOF
#!/usr/bin/vmware
config.version = "8"
virtualHW.version = "4"
floppy0.present = "false"
nvram = "Fedora11.nvram"
deploymentPlatform = "windows"
virtualHW.productCompatibility = "hosted"
tools.upgrade.policy = "useGlobal"
powerType.powerOff = "default"
powerType.powerOn = "default"
powerType.suspend = "default"
powerType.reset = "default"
displayName = "Fedora11"
extendedConfigFile = "Fedora11.vmxf"
scsi0.present = "true"
scsi0.sharedBus = "none"
scsi0.virtualDev = "lsilogic"
memsize = "1024"
scsi0:0.present = "true"
scsi0:0.fileName = "/vmfs/volumes/498076b2-02796c1a-ef5b-000ae484a6a3/Fedora11/Fedora11.vmdk"
scsi0:0.deviceType = "scsi-hardDisk"
ide0:0.present = "true"
ide0:0.clientDevice = "true"
ide0:0.deviceType = "cdrom-raw"
ide0:0.startConnected = "false"
ethernet0.present = "true"
ethernet0.networkName = "VM Network"
ethernet0.addressType = "vpx"
ethernet0.generatedAddress = "00:50:56:91:48:c7"
chipset.onlineStandby = "false"
guestOSAltName = "Red Hat Enterprise Linux 5 (32-Bit)"
guestOS = "rhel5"
uuid.bios = "50 11 5e 16 9b dc 49 d7-f1 71 53 c4 d7 f9 17 10"
snapshot.action = "keep"
sched.cpu.min = "0"
sched.cpu.units = "mhz"
sched.cpu.shares = "normal"
sched.mem.minsize = "0"
sched.mem.shares = "normal"
toolScripts.afterPowerOn = "true"
toolScripts.afterResume = "true"
toolScripts.beforeSuspend = "true"
toolScripts.beforePowerOff = "true"
scsi0:0.redo = ""
tools.syncTime = "false"
uuid.location = "56 4d b5 06 a2 bd fb eb-ae 86 f7 d8 49 27 d0 c4"
sched.cpu.max = "unlimited"
sched.swap.derivedName = "/vmfs/volumes/498076b2-02796c1a-ef5b-000ae484a6a3/Fedora11/Fedora11-7de040d8.vswp"
tools.remindInstall = "TRUE"
EOF
$ virsh -c esx://example.com domxml-from-native vmware-vmx demo.vmx
Enter username for example.com [root]:
Enter root password for example.com:
<domain type='vmware'>
<name>Fedora11</name>
<uuid>50115e16-9bdc-49d7-f171-53c4d7f91710</uuid>
<memory>1048576</memory>
<currentMemory>1048576</currentMemory>
<vcpu>1</vcpu>
<os>
<type arch='i686'>hvm</type>
</os>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<devices>
<disk type='file' device='disk'>
<source file='[local-storage] Fedora11/Fedora11.vmdk'/>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' unit='0'/>
</disk>
<controller type='scsi' index='0' model='lsilogic'/>
<interface type='bridge'>
<mac address='00:50:56:91:48:c7'/>
<source bridge='VM Network'/>
</interface>
</devices>
</domain>
The virsh domxml-to-native provides a way to convert a
domain XML config into a VMware VMX config.
$ cat > demo.xml << EOF
<domain type='vmware'>
<name>Fedora11</name>
<uuid>50115e16-9bdc-49d7-f171-53c4d7f91710</uuid>
<memory>1048576</memory>
<currentMemory>1048576</currentMemory>
<vcpu>1</vcpu>
<os>
<type arch='x86_64'>hvm</type>
</os>
<devices>
<disk type='file' device='disk'>
<source file='[local-storage] Fedora11/Fedora11.vmdk'/>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' unit='0'/>
</disk>
<controller type='scsi' index='0' model='lsilogic'/>
<interface type='bridge'>
<mac address='00:50:56:25:48:c7'/>
<source bridge='VM Network'/>
</interface>
</devices>
</domain>
EOF
$ virsh -c esx://example.com domxml-to-native vmware-vmx demo.xml
Enter username for example.com [root]:
Enter root password for example.com:
config.version = "8"
virtualHW.version = "4"
guestOS = "other-64"
uuid.bios = "50 11 5e 16 9b dc 49 d7-f1 71 53 c4 d7 f9 17 10"
displayName = "Fedora11"
memsize = "1024"
numvcpus = "1"
scsi0.present = "true"
scsi0.virtualDev = "lsilogic"
scsi0:0.present = "true"
scsi0:0.deviceType = "scsi-hardDisk"
scsi0:0.fileName = "/vmfs/volumes/local-storage/Fedora11/Fedora11.vmdk"
ethernet0.present = "true"
ethernet0.networkName = "VM Network"
ethernet0.connectionType = "bridged"
ethernet0.addressType = "static"
ethernet0.address = "00:50:56:25:48:C7"
<domain type='vmware'>
<name>Fedora11</name>
<uuid>50115e16-9bdc-49d7-f171-53c4d7f91710</uuid>
<memory>1048576</memory>
<currentMemory>1048576</currentMemory>
<vcpu>1</vcpu>
<os>
<type arch='x86_64'>hvm</type>
</os>
<devices>
<disk type='file' device='disk'>
<source file='[local-storage] Fedora11/Fedora11.vmdk'/>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' unit='0'/>
</disk>
<controller type='scsi' index='0'/>
<interface type='bridge'>
<mac address='00:50:56:25:48:c7'/>
<source bridge='VM Network'/>
</interface>
</devices>
</domain>
A migration cannot be initiated on an ESX server directly, a VMware
vCenter is necessary for this. The vcenter query
parameter must be set either to the hostname or IP address of the
vCenter managing the ESX server or to *. Setting it
to * causes the driver to connect to the vCenter known to
the ESX server. If the ESX server is not managed by a vCenter an error
is reported.
esx://example.com/?vcenter=example-vcenter.com
Here's an example how to migrate the domain Fedora11 from
ESX server example-src.com to ESX server
example-dst.com implicitly involving vCenter
example-vcenter.com using virsh.
$ virsh -c esx://example-src.com/?vcenter=* migrate Fedora11 esx://example-dst.com/?vcenter=* Enter username for example-src.com [root]: Enter root password for example-src.com: Enter username for example-vcenter.com [administrator]: Enter administrator password for example-vcenter.com: Enter username for example-dst.com [root]: Enter root password for example-dst.com: Enter username for example-vcenter.com [administrator]: Enter administrator password for example-vcenter.com:
Since 0.8.3 you can directly connect to a vCenter.
This simplifies migration a bit. Here's the same migration as above but
using vpx:// connections and assuming both ESX server are in
datacenter dc1 and aren't part of a cluster.
$ virsh -c vpx://example-vcenter.com/dc1/example-src.com migrate Fedora11 vpx://example-vcenter.com/dc1/example-dst.com Enter username for example-vcenter.com [administrator]: Enter administrator password for example-vcenter.com: Enter username for example-vcenter.com [administrator]: Enter administrator password for example-vcenter.com:
The driver exposes the ESX CPU scheduler. The parameters listed below are available to control the scheduler.
reservationlimitshareslow, normal and
high.
Some actions require installed VMware tools. If the VMware tools are not installed in the guest and one of the actions below is to be performed the ESX server raises an error and the driver reports it.
virDomainReboot
virDomainShutdown