2014-03-01 09:04:56 +00:00
|
|
|
<?xml version="1.0" encoding="UTF-8"?>
|
|
|
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
|
|
|
<body>
|
|
|
|
|
<h1>Bhyve driver</h1>
|
|
|
|
|
|
|
|
|
|
<ul id="toc"></ul>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Bhyve is a FreeBSD hypervisor. It first appeared in FreeBSD 10.0. However, it's
|
|
|
|
|
recommended to keep tracking FreeBSD 10-STABLE to make sure all new features
|
|
|
|
|
of bhyve are supported.
|
|
|
|
|
|
|
|
|
|
In order to enable bhyve on your FreeBSD host, you'll need to load the <code>vmm</code>
|
|
|
|
|
kernel module. Additionally, <code>if_tap</code> and <code>if_bridge</code> modules
|
|
|
|
|
should be loaded for networking support.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Additional information on bhyve could be obtained on <a href="http://bhyve.org/">bhyve.org</a>.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h2><a name="uri">Connections to the Bhyve driver</a></h2>
|
|
|
|
|
<p>
|
|
|
|
|
The libvirt bhyve driver is a single-instance privileged driver. Some sample
|
|
|
|
|
connection URIs are:
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
|
|
|
|
bhyve:///system (local access)
|
|
|
|
|
bhyve+unix:///system (local access)
|
|
|
|
|
bhyve+ssh://root@example.com/system (remote access, SSH tunnelled)
|
|
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<h2><a name="exconfig">Example guest domain XML configurations</a></h2>
|
|
|
|
|
|
|
|
|
|
<h3>Example config</h3>
|
|
|
|
|
<p>
|
|
|
|
|
The bhyve driver in libvirt is in its early stage and under active development. So it supports
|
2014-11-08 16:48:30 +00:00
|
|
|
only limited number of features bhyve provides.
|
2014-03-01 09:04:56 +00:00
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>
|
2014-07-24 15:52:11 +00:00
|
|
|
Note: in older libvirt versions, only a single network device and a single
|
|
|
|
|
disk device were supported per-domain. However,
|
|
|
|
|
<span class="since">since 1.2.6</span> the libvirt bhyve driver supports
|
|
|
|
|
up to 31 PCI devices.
|
2014-03-01 09:04:56 +00:00
|
|
|
</p>
|
|
|
|
|
|
2014-11-08 16:48:30 +00:00
|
|
|
<p>
|
|
|
|
|
Note: the Bhyve driver in libvirt will boot whichever device is first. If you
|
|
|
|
|
want to install from CD, put the CD device first. If not, put the root HDD
|
|
|
|
|
first.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Note: Only the SATA bus is supported. Only <code>cdrom</code>- and
|
|
|
|
|
<code>disk</code>-type disks are supported.
|
|
|
|
|
</p>
|
|
|
|
|
|
2014-03-01 09:04:56 +00:00
|
|
|
<pre>
|
|
|
|
|
<domain type='bhyve'>
|
2014-11-08 16:48:30 +00:00
|
|
|
<name>bhyve</name>
|
|
|
|
|
<uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
|
2014-03-01 09:04:56 +00:00
|
|
|
<memory>219136</memory>
|
|
|
|
|
<currentMemory>219136</currentMemory>
|
|
|
|
|
<vcpu>1</vcpu>
|
|
|
|
|
<os>
|
|
|
|
|
<type>hvm</type>
|
|
|
|
|
</os>
|
|
|
|
|
<features>
|
|
|
|
|
<apic/>
|
|
|
|
|
<acpi/>
|
|
|
|
|
</features>
|
|
|
|
|
<clock offset='utc'/>
|
|
|
|
|
<on_poweroff>destroy</on_poweroff>
|
|
|
|
|
<on_reboot>restart</on_reboot>
|
|
|
|
|
<on_crash>destroy</on_crash>
|
|
|
|
|
<devices>
|
|
|
|
|
<disk type='file'>
|
|
|
|
|
<driver name='file' type='raw'/>
|
|
|
|
|
<source file='/path/to/bhyve_freebsd.img'/>
|
|
|
|
|
<target dev='hda' bus='sata'/>
|
|
|
|
|
</disk>
|
2014-07-24 15:52:11 +00:00
|
|
|
<disk type='file' device='cdrom'>
|
|
|
|
|
<driver name='file' type='raw'/>
|
|
|
|
|
<source file='/path/to/cdrom.iso'/>
|
|
|
|
|
<target dev='hdc' bus='sata'/>
|
2014-11-08 16:48:30 +00:00
|
|
|
<readonly/>
|
2014-07-24 15:52:11 +00:00
|
|
|
</disk>
|
2014-03-01 09:04:56 +00:00
|
|
|
<interface type='bridge'>
|
|
|
|
|
<model type='virtio'/>
|
|
|
|
|
<source bridge="virbr0"/>
|
|
|
|
|
</interface>
|
|
|
|
|
</devices>
|
|
|
|
|
</domain>
|
|
|
|
|
</pre>
|
|
|
|
|
|
2014-11-08 16:48:30 +00:00
|
|
|
<p>(The <disk> sections may be swapped in order to install from
|
|
|
|
|
<em>cdrom.iso</em>.)</p>
|
|
|
|
|
|
|
|
|
|
<h3>Example config (Linux guest)</h3>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Note the addition of <bootloader>.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
|
|
|
|
<domain type='bhyve'>
|
|
|
|
|
<name>linux_guest</name>
|
|
|
|
|
<uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
|
|
|
|
|
<memory>131072</memory>
|
|
|
|
|
<currentMemory>131072</currentMemory>
|
|
|
|
|
<vcpu>1</vcpu>
|
|
|
|
|
<bootloader>/usr/local/sbin/grub-bhyve</bootloader>
|
|
|
|
|
<os>
|
|
|
|
|
<type>hvm</type>
|
|
|
|
|
</os>
|
|
|
|
|
<features>
|
|
|
|
|
<apic/>
|
|
|
|
|
<acpi/>
|
|
|
|
|
</features>
|
|
|
|
|
<clock offset='utc'/>
|
|
|
|
|
<on_poweroff>destroy</on_poweroff>
|
|
|
|
|
<on_reboot>restart</on_reboot>
|
|
|
|
|
<on_crash>destroy</on_crash>
|
|
|
|
|
<devices>
|
|
|
|
|
<disk type='file' device='disk'>
|
|
|
|
|
<driver name='file' type='raw'/>
|
|
|
|
|
<source file='/path/to/guest_hdd.img'/>
|
|
|
|
|
<target dev='hda' bus='sata'/>
|
|
|
|
|
</disk>
|
|
|
|
|
<disk type='file' device='cdrom'>
|
|
|
|
|
<driver name='file' type='raw'/>
|
|
|
|
|
<source file='/path/to/cdrom.iso'/>
|
|
|
|
|
<target dev='hdc' bus='sata'/>
|
|
|
|
|
<readonly/>
|
|
|
|
|
</disk>
|
|
|
|
|
<interface type='bridge'>
|
|
|
|
|
<model type='virtio'/>
|
|
|
|
|
<source bridge="virbr0"/>
|
|
|
|
|
</interface>
|
|
|
|
|
</devices>
|
|
|
|
|
</domain>
|
|
|
|
|
</pre>
|
2014-04-21 13:16:58 +00:00
|
|
|
|
|
|
|
|
<h2><a name="usage">Guest usage / management</a></h2>
|
|
|
|
|
|
|
|
|
|
<h3><a name="console">Connecting to a guest console</a></h3>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
Guest console connection is supported through the <code>nmdm</code> device. It could be enabled by adding
|
|
|
|
|
the following to the domain XML (<span class="since">Since 1.2.4</span>):
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
...
|
|
|
|
|
<devices>
|
|
|
|
|
<serial type="nmdm">
|
|
|
|
|
<source master="/dev/nmdm0A" slave="/dev/nmdm0B"/>
|
|
|
|
|
</serial>
|
|
|
|
|
</devices>
|
|
|
|
|
...</pre>
|
2014-04-21 13:16:58 +00:00
|
|
|
|
2014-05-11 07:25:08 +00:00
|
|
|
|
|
|
|
|
<p>Make sure to load the <code>nmdm</code> kernel module if you plan to use that.</p>
|
|
|
|
|
|
2014-04-21 13:16:58 +00:00
|
|
|
<p>
|
|
|
|
|
Then <code>virsh console</code> command can be used to connect to the text console
|
|
|
|
|
of a guest.</p>
|
|
|
|
|
|
|
|
|
|
<p><b>NB:</b> Some versions of bhyve have a bug that prevents guests from booting
|
|
|
|
|
until the console is opened by a client. This bug was fixed in FreeBSD
|
|
|
|
|
<a href="http://svnweb.freebsd.org/changeset/base/262884">r262884</a>. If
|
|
|
|
|
an older version is used, one either has to open a console manually with <code>virsh console</code>
|
|
|
|
|
to let a guest boot or start a guest using:</p>
|
|
|
|
|
|
|
|
|
|
<pre>start --console domname</pre>
|
|
|
|
|
|
2014-11-08 16:48:30 +00:00
|
|
|
<p><b>NB:</b> An bootloader configured to require user interaction will prevent
|
|
|
|
|
the domain from starting (and thus <code>virsh console</code> or <code>start
|
|
|
|
|
--console</code> from functioning) until the user interacts with it manually on
|
|
|
|
|
the VM host. Because users typically do not have access to the VM host,
|
|
|
|
|
interactive bootloaders are unsupported by libvirt. <em>However,</em> if you happen to
|
|
|
|
|
run into this scenario and also happen to have access to the Bhyve host
|
|
|
|
|
machine, you may select a boot option and allow the domain to finish starting
|
|
|
|
|
by using an alternative terminal client on the VM host to connect to the
|
|
|
|
|
domain-configured null modem device. One example (assuming
|
|
|
|
|
<code>/dev/nmdm0B</code> is configured as the slave end of the domain serial
|
|
|
|
|
device) is:</p>
|
|
|
|
|
|
|
|
|
|
<pre>cu -l /dev/nmdm0B</pre>
|
|
|
|
|
|
2014-05-11 07:25:08 +00:00
|
|
|
<h3><a name="xmltonative">Converting from domain XML to Bhyve args</a></h3>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
The <code>virsh domxml-to-native</code> command can preview the actual
|
|
|
|
|
<code>bhyve</code> commands that will be executed for a given domain.
|
|
|
|
|
It outputs two lines, the first line is a <code>bhyveload</code> command and
|
|
|
|
|
the second is a <code>bhyve</code> command.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>Please note that the <code>virsh domxml-to-native</code> doesn't do any
|
|
|
|
|
real actions other than printing the command, for example, it doesn't try to
|
|
|
|
|
find a proper TAP interface and create it, like what is done when starting
|
|
|
|
|
a domain; and always returns <code>tap0</code> for the network interface. So
|
|
|
|
|
if you're going to run these commands manually, most likely you might want to
|
|
|
|
|
tweak them.</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
|
|
|
|
# virsh -c "bhyve:///system" domxml-to-native --format bhyve-argv --xml /path/to/bhyve.xml
|
|
|
|
|
/usr/sbin/bhyveload -m 214 -d /home/user/vm1.img vm1
|
|
|
|
|
/usr/sbin/bhyve -c 2 -m 214 -A -I -H -P -s 0:0,hostbridge -s 3:0,virtio-net,tap0,mac=52:54:00:5d:74:e3 -s 2:0,virtio-blk,/home/user/vm1.img -s 1,lpc -l com1,/dev/nmdm0A vm1
|
|
|
|
|
</pre>
|
2014-04-21 13:16:58 +00:00
|
|
|
|
2014-09-14 05:17:54 +00:00
|
|
|
<h3><a name="zfsvolume">Using ZFS volumes</a></h3>
|
|
|
|
|
|
|
|
|
|
<p>It's possible to use ZFS volumes as disk devices <span class="since">since 1.2.8</span>.
|
|
|
|
|
An example of domain XML device entry for that will look like:</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
...
|
|
|
|
|
<disk type='volume' device='disk'>
|
|
|
|
|
<source pool='zfspool' volume='vol1'/>
|
|
|
|
|
<target dev='vdb' bus='virtio'/>
|
|
|
|
|
</disk>
|
|
|
|
|
...</pre>
|
2014-09-14 05:17:54 +00:00
|
|
|
|
|
|
|
|
<p>Please refer to the <a href="storage.html">Storage documentation</a> for more details on storage
|
|
|
|
|
management.</p>
|
|
|
|
|
|
2014-11-08 16:48:30 +00:00
|
|
|
<h3><a name="grubbhyve">Using grub2-bhyve or Alternative Bootloaders</a></h3>
|
|
|
|
|
|
|
|
|
|
<p>It's possible to boot non-FreeBSD guests by specifying an explicit
|
|
|
|
|
bootloader, e.g. <code>grub-bhyve(1)</code>. Arguments to the bootloader may be
|
|
|
|
|
specified as well. If the bootloader is <code>grub-bhyve</code> and arguments
|
2014-11-12 22:31:53 +00:00
|
|
|
are omitted, libvirt will try and infer boot ordering from user-supplied
|
|
|
|
|
<boot order='N'> configuration in the domain. Failing that, it will boot
|
|
|
|
|
the first disk in the domain (either <code>cdrom</code>- or
|
|
|
|
|
<code>disk</code>-type devices). If the disk type is <code>disk</code>, it will
|
|
|
|
|
attempt to boot from the first partition in the disk image.</p>
|
2014-11-08 16:48:30 +00:00
|
|
|
|
|
|
|
|
<pre>
|
2016-11-11 22:40:27 +00:00
|
|
|
...
|
|
|
|
|
<bootloader>/usr/local/sbin/grub-bhyve</bootloader>
|
|
|
|
|
<bootloader_args>...</bootloader_args>
|
|
|
|
|
...
|
2014-11-08 16:48:30 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>Caveat: <code>bootloader_args</code> does not support any quoting.
|
|
|
|
|
Filenames, etc, must not have spaces or they will be tokenized incorrectly.</p>
|
|
|
|
|
|
2015-07-19 08:20:36 +00:00
|
|
|
<h3><a name="clockconfig">Clock configuration</a></h3>
|
|
|
|
|
|
|
|
|
|
<p>Originally bhyve supported only localtime for RTC. Support for UTC time was introduced in
|
|
|
|
|
<a href="http://svnweb.freebsd.org/changeset/base/284894">r284894</a> for <i>10-STABLE</i> and
|
|
|
|
|
in <a href="http://svnweb.freebsd.org/changeset/base/279225">r279225</a> for <i>-CURRENT</i>.
|
|
|
|
|
It's possible to use this in libvirt <span class="since">since 1.2.18</span>, just place the
|
|
|
|
|
following to domain XML:</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
|
|
|
|
<domain type="bhyve">
|
|
|
|
|
...
|
|
|
|
|
<clock offset='utc'/>
|
|
|
|
|
...
|
|
|
|
|
</domain>
|
|
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<p>Please note that if you run the older bhyve version that doesn't support UTC time, you'll
|
|
|
|
|
fail to start a domain. As UTC is used as a default when you do not specify clock settings,
|
|
|
|
|
you'll need to explicitly specify 'localtime' in this case:</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
|
|
|
|
<domain type="bhyve">
|
|
|
|
|
...
|
|
|
|
|
<clock offset='localtime'/>
|
|
|
|
|
...
|
|
|
|
|
</domain>
|
2017-02-12 16:42:31 +00:00
|
|
|
</pre>
|
|
|
|
|
|
|
|
|
|
<h3><a name="e1000">e1000 NIC</a></h3>
|
|
|
|
|
|
|
|
|
|
<p>As of <a href="https://svnweb.freebsd.org/changeset/base/302504">r302504</a> bhyve
|
|
|
|
|
supports Intel e1000 network adapter emulation. It's supported in libvirt
|
|
|
|
|
<span class="since">since 3.1.0</span> and could be used as follows:</p>
|
|
|
|
|
|
|
|
|
|
<pre>
|
|
|
|
|
...
|
|
|
|
|
<interface type='bridge'>
|
|
|
|
|
<source bridge='virbr0'/>
|
|
|
|
|
<model type='<b>e1000</b>'/>
|
|
|
|
|
</interface>
|
|
|
|
|
...
|
2015-07-19 08:20:36 +00:00
|
|
|
</pre>
|
|
|
|
|
|
2014-03-01 09:04:56 +00:00
|
|
|
</body>
|
|
|
|
|
</html>
|
