From nobody Mon Feb 9 20:10:20 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1648469462; cv=none; d=zohomail.com; s=zohoarc; b=IWLotfp/ACl8CG7GHNi2twRYWQOK+lngfSPOaXOo+N0Qff2a8UBj4fUrR47ouqzsn5CV4ndfCiC2Lyh481kRBmDIOFp8fWwQHBwP2YpPnT5Y29Ztx9oEQyDbtQ5r8QB1lvwCx7K+B0ZNy8ei8do+M1uCCB03magn4NjSJ7BsMps= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1648469462; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=OeDWW5/Gr/efVMAAZKW2E9rHkV7Eie6C038w12X0syc=; b=L1FIjmcFwVSSXHgTTwGREQEisx2dK5hCmiinsKBhhVis+bMxRz0EBsVyuh0LzJGwKzMM9nybXN/BpINnDGsdX7Ds2wK3dQJWTgzcM6A1KdCrrbRtrfEByjhdSAPhaQ3BfOLnfIVFvkYtLqf3cpESyH/iskPASKOWQUTOd6Hk5x8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 164846946238962.144980795860306; Mon, 28 Mar 2022 05:11:02 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-674-5xfUOOxlMP-kGE62jmR_Rg-1; Mon, 28 Mar 2022 08:10:54 -0400 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 52A8E18A6591; Mon, 28 Mar 2022 12:10:52 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3939A401DBC; Mon, 28 Mar 2022 12:10:52 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 0F874193F6E8; Mon, 28 Mar 2022 12:10:50 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id B88E7193F6DD for ; Mon, 28 Mar 2022 12:10:49 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 9C9A0141DC2C; Mon, 28 Mar 2022 12:10:49 +0000 (UTC) Received: from speedmetal.lan (unknown [10.40.208.35]) by smtp.corp.redhat.com (Postfix) with ESMTP id E71D41402648 for ; Mon, 28 Mar 2022 12:10:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648469461; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=OeDWW5/Gr/efVMAAZKW2E9rHkV7Eie6C038w12X0syc=; b=C9J9yl1TmcMjthqCrcn9e5VO5mbKkMsktpfAfPK6JvCziiT0+fMk1++CcIzI/jt1LeK6Ac eEJhtS34Fc72Cp2h9dGT5VqeuENNPU1QNzHZhf/CTDdLkFaHWlY0wznhfcQRRB+seFB2bZ Sp/rSoLW3pRDgPOO8iZ7XVRTFBTvCcE= X-MC-Unique: 5xfUOOxlMP-kGE62jmR_Rg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Peter Krempa To: libvir-list@redhat.com Subject: [PATCH 03/29] docs: Convert 'drvbhyve' page to rST Date: Mon, 28 Mar 2022 14:10:18 +0200 Message-Id: <65975fa7e553e5908196077a78824de174355452.1648469356.git.pkrempa@redhat.com> In-Reply-To: References: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.85 on 10.11.54.7 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 2.85 on 10.11.54.10 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=libvir-list-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1648469466024100001 Content-Type: text/plain; charset="utf-8"; x-default="true" Signed-off-by: Peter Krempa Reviewed-by: Erik Skultety --- docs/drvbhyve.html.in | 583 ------------------------------------------ docs/drvbhyve.rst | 582 +++++++++++++++++++++++++++++++++++++++++ docs/meson.build | 2 +- 3 files changed, 583 insertions(+), 584 deletions(-) delete mode 100644 docs/drvbhyve.html.in create mode 100644 docs/drvbhyve.rst diff --git a/docs/drvbhyve.html.in b/docs/drvbhyve.html.in deleted file mode 100644 index 228e8b2bd5..0000000000 --- a/docs/drvbhyve.html.in +++ /dev/null @@ -1,583 +0,0 @@ - - - - -

Bhyve driver

- -
    - -

    -Bhyve is a FreeBSD hypervisor. It first appeared in FreeBSD 10.0. However,= it's -recommended to keep tracking FreeBSD 10-STABLE to make sure all new featur= es -of bhyve are supported. - -In order to enable bhyve on your FreeBSD host, you'll need to load the vmm -kernel module. Additionally, if_tap and if_bridge modules -should be loaded for networking support. Also, since= 3.2.0 the -virt-host-validate(1) supports the bhyve host validation and = could be -used like this: -

    - -
    -$ virt-host-validate bhyve
- BHYVE: Checking for vmm module                                           =
   : PASS
- BHYVE: Checking for if_tap module                                        =
   : PASS
- BHYVE: Checking for if_bridge module                                     =
   : PASS
- BHYVE: Checking for nmdm module                                          =
   : PASS
-$
-
    - -

    -Additional information on bhyve could be obtained on bhyve.org. -

    - -

    Connections to the Bhyve driver

    -

    -The libvirt bhyve driver is a single-instance privileged driver. Some samp= le -connection URIs are: -

    - -
    -bhyve:///system                     (local access)
-bhyve+unix:///system                (local access)
-bhyve+ssh://root@example.com/system (remote access, SSH tunnelled)
-
    - -

    Example guest domain XML configurations

    - -

    Example config

    -

    -The bhyve driver in libvirt is in its early stage and under active develop= ment. So it supports -only limited number of features bhyve provides. -

    - -

    -Note: in older libvirt versions, only a single network device and a single -disk device were supported per-domain. However, -since 1.2.6 the libvirt bhyve driver supports -up to 31 PCI devices. -

    - -

    -Note: the Bhyve driver in libvirt will boot whichever device is first. If = you -want to install from CD, put the CD device first. If not, put the root HDD -first. -

    - -

    -Note: Only the SATA bus is supported. Only cdrom- and -disk-type disks are supported. -

    - -
    -<domain type=3D'bhyve'>
-    <name>bhyve</name>
-    <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
-    <memory>219136</memory>
-    <currentMemory>219136</currentMemory>
-    <vcpu>1</vcpu>
-    <os>
-       <type>hvm</type>
-    </os>
-    <features>
-      <apic/>
-      <acpi/>
-    </features>
-    <clock offset=3D'utc'/>
-    <on_poweroff>destroy</on_poweroff>
-    <on_reboot>restart</on_reboot>
-    <on_crash>destroy</on_crash>
-    <devices>
-      <disk type=3D'file'>
-        <driver name=3D'file' type=3D'raw'/>
-        <source file=3D'/path/to/bhyve_freebsd.img'/>
-        <target dev=3D'hda' bus=3D'sata'/>
-      </disk>
-      <disk type=3D'file' device=3D'cdrom'>
-        <driver name=3D'file' type=3D'raw'/>
-        <source file=3D'/path/to/cdrom.iso'/>
-        <target dev=3D'hdc' bus=3D'sata'/>
-        <readonly/>
-      </disk>
-      <interface type=3D'bridge'>
-        <model type=3D'virtio'/>
-        <source bridge=3D"virbr0"/>
-      </interface>
-    </devices>
-</domain>
-
    - -

    (The <disk> sections may be swapped in order to install from -cdrom.iso.)

    - -

    Example config (Linux guest)

    - -

    -Note the addition of <bootloader>. -

    - -
    -<domain type=3D'bhyve'>
-    <name>linux_guest</name>
-    <uuid>df3be7e7-a104-11e3-aeb0-50e5492bd3dc</uuid>
-    <memory>131072</memory>
-    <currentMemory>131072</currentMemory>
-    <vcpu>1</vcpu>
-    <bootloader>/usr/local/sbin/grub-bhyve</bootloader>
-    <os>
-       <type>hvm</type>
-    </os>
-    <features>
-      <apic/>
-      <acpi/>
-    </features>
-    <clock offset=3D'utc'/>
-    <on_poweroff>destroy</on_poweroff>
-    <on_reboot>restart</on_reboot>
-    <on_crash>destroy</on_crash>
-    <devices>
-      <disk type=3D'file' device=3D'disk'>
-        <driver name=3D'file' type=3D'raw'/>
-        <source file=3D'/path/to/guest_hdd.img'/>
-        <target dev=3D'hda' bus=3D'sata'/>
-      </disk>
-      <disk type=3D'file' device=3D'cdrom'>
-        <driver name=3D'file' type=3D'raw'/>
-        <source file=3D'/path/to/cdrom.iso'/>
-        <target dev=3D'hdc' bus=3D'sata'/>
-        <readonly/>
-      </disk>
-      <interface type=3D'bridge'>
-        <model type=3D'virtio'/>
-        <source bridge=3D"virbr0"/>
-      </interface>
-    </devices>
-</domain>
-
    - -

    Example config (Linux UEFI guest, VNC, tablet)

    - -

    This is an example to boot into Fedora 25 installation:

    - -
    -<domain type=3D'bhyve'>
-    <name>fedora_uefi_vnc_tablet</name>
-    <memory unit=3D'G'>4</memory>
-    <vcpu>2</vcpu>
-    <os>
-       <type>hvm</type>
-       <loader readonly=3D"yes" type=3D"pflash"&=
gt;/usr/local/share/uefi-firmware/BHYVE_UEFI.fd</loader>
-    </os>
-    <features>
-      <apic/>
-      <acpi/>
-    </features>
-    <clock offset=3D'utc'/>
-    <on_poweroff>destroy</on_poweroff>
-    <on_reboot>restart</on_reboot>
-    <on_crash>destroy</on_crash>
-    <devices>
-      <disk type=3D'file' device=3D'cdrom'>
-        <driver name=3D'file' type=3D'raw'/>
-          <source file=3D'/path/to/Fedora-Workstation-Live-x86_64-25-1.=
3.iso'/>
-        <target dev=3D'hdc' bus=3D'sata'/>
-        <readonly/>
-      </disk>
-      <disk type=3D'file' device=3D'disk'>
-        <driver name=3D'file' type=3D'raw'/>
-        <source file=3D'/path/to/linux_uefi.img'/>
-        <target dev=3D'hda' bus=3D'sata'/>
-        </disk>
-      <interface type=3D'bridge'>
-        <model type=3D'virtio'/>
-        <source bridge=3D"virbr0"/>
-      </interface>
-      <serial type=3D"nmdm">
-        <source master=3D"/dev/nmdm0A" slave=3D"/dev/nmd=
m0B"/>
-      </serial>
-      <graphics type=3D'vnc' port=3D'5904'>
-        <listen type=3D'address' address=3D'127.0.0.1'/>
-      </graphics>
-      <controller type=3D'usb' model=3D'nec-xhci'/>
-      <input type=3D'tablet' bus=3D'usb'/>
-    </devices>
-</domain>
-
    - -

    Please refer to the UEFI section for a more detai= led explanation.

    - -

    Guest usage / management

    - -

    Connecting to a guest console

    - -

    -Guest console connection is supported through the nmdm device= . It could be enabled by adding -the following to the domain XML (Since 1.2.4): -

    - -
    -...
-<devices>
-  <serial type=3D"nmdm">
-    <source master=3D"/dev/nmdm0A" slave=3D"/dev/nmdm0B"/>
-  </serial>
-</devices>
-...
    - - -

    Make sure to load the nmdm kernel module if you plan to us= e that.

    - -

    -Then virsh console command can be used to connect to the text= console -of a guest.

    - -

    NB: Some versions of bhyve have a bug that prevents guests from = booting -until the console is opened by a client. This bug was fixed in -FreeBSD chang= eset r262884. If -an older version is used, one either has to open a console manually with <= code>virsh console -to let a guest boot or start a guest using:

    - -
    start --console domname
    - -

    NB: A bootloader configured to require user interaction will pre= vent -the domain from starting (and thus virsh console or sta= rt ---console from functioning) until the user interacts with it manual= ly on -the VM host. Because users typically do not have access to the VM host, -interactive bootloaders are unsupported by libvirt. However, if y= ou happen to -run into this scenario and also happen to have access to the Bhyve host -machine, you may select a boot option and allow the domain to finish start= ing -by using an alternative terminal client on the VM host to connect to the -domain-configured null modem device. One example (assuming -/dev/nmdm0B is configured as the slave end of the domain seri= al -device) is:

    - -
    cu -l /dev/nmdm0B
    - -

    Converting from domain XML to Bhyve args

    - -

    -The virsh domxml-to-native command can preview the actual -bhyve commands that will be executed for a given domain. -It outputs two lines, the first line is a bhyveload command a= nd -the second is a bhyve command. -

    - -

    Please note that the virsh domxml-to-native doesn't do any -real actions other than printing the command, for example, it doesn't try = to -find a proper TAP interface and create it, like what is done when starting -a domain; and always returns tap0 for the network interface. = So -if you're going to run these commands manually, most likely you might want= to -tweak them.

    - -
    -# virsh -c "bhyve:///system"  domxml-to-native --format bhyve-argv --xml /=
path/to/bhyve.xml
-/usr/sbin/bhyveload -m 214 -d /home/user/vm1.img vm1
-/usr/sbin/bhyve -c 2 -m 214 -A -I -H -P -s 0:0,hostbridge \
-    -s 3:0,virtio-net,tap0,mac=3D52:54:00:5d:74:e3 -s 2:0,virtio-blk,/home=
/user/vm1.img \
-    -s 1,lpc -l com1,/dev/nmdm0A vm1
-
    - -

    Using ZFS volumes

    - -

    It's possible to use ZFS volumes as disk devices = since 1.2.8. -An example of domain XML device entry for that will look like:

    - -
    -...
-<disk type=3D'volume' device=3D'disk'>
-  <source pool=3D'zfspool' volume=3D'vol1'/>
-  <target dev=3D'vdb' bus=3D'virtio'/>
-</disk>
-...
    - -

    Please refer to the Storage documentation = for more details on storage -management.

    - -

    Using grub2-bhyve or Alternative Bootloaders - -

    It's possible to boot non-FreeBSD guests by specifying an explicit -bootloader, e.g. grub-bhyve(1). Arguments to the bootloader m= ay be -specified as well. If the bootloader is grub-bhyve and argume= nts -are omitted, libvirt will try and infer boot ordering from user-supplied -<boot order=3D'N'> configuration in the domain. Failing that, it wil= l boot -the first disk in the domain (either cdrom- or -disk-type devices). If the disk type is disk, it= will -attempt to boot from the first partition in the disk image.

    - -
    -...
-<bootloader>/usr/local/sbin/grub-bhyve</bootloader>
-<bootloader_args>...</bootloader_args>
-...
-
    - -

    Caveat: bootloader_args does not support any quoting. -Filenames, etc, must not have spaces or they will be tokenized incorrectly= .

    - -

    Using UEFI bootrom, VNC, and USB tablet

    - -

    Since 3.2.0, in addition to grub-bhyve, -non-FreeBSD guests could be also booted using an UEFI boot ROM, provided b= oth guest OS and -installed bhyve(1) version support UEFI. To use that, l= oader -should be specified in the os section:

    - -
    -<domain type=3D'bhyve'>
-    ...
-    <os>
-       <type>hvm</type>
-       <loader readonly=3D"yes" type=3D"pflash">/usr/local/share/uef=
i-firmware/BHYVE_UEFI.fd</loader>
-    </os>
-    ...
-
    - -

    This uses the UEFI firmware provided by -the sysuti= ls/bhyve-firmware -FreeBSD port.

    - -

    VNC and the tablet input device could be configured this way:

    - -
    -<domain type=3D'bhyve'>
-    <devices>
-      ...
-      <graphics type=3D'vnc' port=3D'5904'>
-        <listen type=3D'address' address=3D'127.0.0.1'/>
-      </graphics>
-      <controller type=3D'usb' model=3D'nec-xhci'/>
-      <input type=3D'tablet' bus=3D'usb'/>
-    </devices>
-    ...
-</domain>
-
    - -

    This way, VNC will be accessible on 127.0.0.1:5904.

    - -

    Please note that the tablet device requires to have a USB controller -of the nec-xhci model. Currently, only a single controller of= this -type and a single tablet are supported per domain.

    - -

    Since 3.5.0, it's possible to configure ho= w the video device is exposed -to the guest using the vgaconf attribute:

    - -
    -<domain type=3D'bhyve'>
-    <devices>
-    ...
-      <graphics type=3D'vnc' port=3D'5904'>
-        <listen type=3D'address' address=3D'127.0.0.1'/>
-      </graphics>
-      <video>
-        <driver vgaconf=3D'on'/>
-        <model type=3D'gop' heads=3D'1' primary=3D'yes'/>
-      </video>
-      ...
-    </devices>
-    ...
-</domain>
-
    - -

    If not specified, bhyve's default mode for vgaconf -will be used. Please refer to the -bhyve(8) -manual page and the bhyve wiki<= /a> for more details on using -the vgaconf option.

    - -

    Since 3.7.0, it's possible to use au= toport -to let libvirt allocate VNC port automatically (instead of explicitly spec= ifying -it with the port attribute):

    - -
    -    <graphics type=3D'vnc' autoport=3D'yes'>
-
    - -

    Since 6.8.0, it's possible to set framebuf= fer resolution -using the resolution sub-element:

    - -
    -   <video>
-     <model type=3D'gop' heads=3D'1' primary=3D'yes'>
-       <resolution x=3D'800' y=3D'600'/>
-     </model>
-   </video>
-
    - -

    Since 6.8.0, VNC server can be configured = to use -password based authentication:

    - -
    -  <graphics type=3D'vnc' port=3D'5904' passwd=3D'foobar'>
-    <listen type=3D'address' address=3D'127.0.0.1'/>
-  </graphics>
-
    - -

    Note: VNC password authentication is known to be cryptographically weak. -Additionally, the password is passed as a command line argument in clear t= ext. -Make sure you understand the risks associated with this feature before usi= ng it.

    - -

    Clock configuration

    - -

    Originally bhyve supported only localtime for RTC. Support for UTC time= was introduced in -FreeBSD chang= eset r284894 -for 10-STABLE and -in changeset = r279225 -for -CURRENT. It's possible to use this in libvirt since 1.2.18, -just place the following to domain XML:

    - -
    -<domain type=3D"bhyve">
-    ...
-    <clock offset=3D'utc'/>
-    ...
-</domain>
-
    - -

    Please note that if you run the older bhyve version that doesn't suppor= t UTC time, you'll -fail to start a domain. As UTC is used as a default when you do not specif= y clock settings, -you'll need to explicitly specify 'localtime' in this case:

    - -
    -<domain type=3D"bhyve">
-    ...
-    <clock offset=3D'localtime'/>
-    ...
-</domain>
-
    - -

    e1000 NIC

    - -

    As of Free= BSD changeset r302504 -bhyve supports Intel e1000 network adapter emulation. It's supported in li= bvirt -since 3.1.0 and could be used as follows:

    - -
    -...
-    <interface type=3D'bridge'>
-      <source bridge=3D'virbr0'/>
-      <model type=3D'e1000'/>
-    </interface>
-...
-
    - -

    Sound device

    - -

    As of Free= BSD changeset r349355 -bhyve supports sound device emulation. It's supported in libvirt -since 6.7.0.

    - -
    -...
-  <sound model=3D'ich7'>
-    <audio id=3D'1'/>
-  </sound>
-  <audio id=3D'1' type=3D'oss'>
-    <input dev=3D'/dev/dsp0'/>
-    <output dev=3D'/dev/dsp0'/>
-  </audio>
-...
-
    - -

    Here, the sound element specifies the sound device as it's= exposed -to the guest, with ich7 being the only supported model now, -and the audio element specifies how the guest device is mapped -to the host sound device.

    - -

    Virtio-9p filesystem

    - -

    As of Free= BSD changeset r366413 -bhyve supports sharing arbitrary directory tree between the guest and the = host. -It's supported in libvirt since 6.9.0.

    - -
    -...
-  <filesystem>
-    <source dir=3D'/shared/dir'/>
-    <target dir=3D'shared_dir'/>
-  </filesystem>
-...
-
    - -

    This share could be made read only by adding the <readonly/>= ; sub-element.

    - -

    In the Linux guest, this could be mounted using:

    - -
    mount -t 9p shared_dir /mnt/shared_dir
    - -

    Wiring guest memory

    - -

    Since 4.4.0, it's possible to specify that= guest memory should -be wired and cannot be swapped out as follows:

    -
    -<domain type=3D"bhyve">
-    ...
-    <memoryBacking>
-      <locked/>
-    </memoryBacking>
-    ...
-</domain>
-
    - -

    CPU topology

    - -

    Since 4.5.0, it's possible to specify gues= t CPU topology, if bhyve -supports that. Support for specifying guest CPU topology was added to bhyv= e in -FreeBSD chang= eset r332298 -for -CURRENT. -Example:

    -
    -<domain type=3D"bhyve">
-    ...
-    <cpu>
-      <topology sockets=3D'1' cores=3D'2' threads=3D'1'/>
-    </cpu>
-    ...
-</domain>
-
    - -

    Ignoring unknown MSRs reads and writes

    - -

    Since 5.1.0, it's possible to make bhyve -ignore accesses to unimplemented Model Specific Registers (MSRs). -Example:

    - -
    -<domain type=3D"bhyve">
-    ...
-    <features>
-      ...
-      <msrs unknown=3D'ignore'/>
-      ...
-    </features>
-    ...
-</domain>
-
    - -

    Pass-through of arbitrary bhyve commands - -

    Since 5.1.0, it's possible to pass additio= nal command-line -arguments to the bhyve process when starting the domain using the -<bhyve:commandline> element under domain. -To supply an argument, use the element <bhyve:arg> with -the attribute value set to additional argument to be added. -The arg element may be repeated multiple times. To use this XML addition, = it is necessary -to issue an XML namespace request (the special xmlns:name attribute) -that pulls in http://libvirt.org/schemas/domain/bhyve/1.0; -typically, the namespace is given the name of bhyve. -

    -

    Example:

    -
    -<domain type=3D"bhyve" xmlns:bhyve=3D"http://libvirt.org/schemas/domain=
/bhyve/1.0">
-  ...
-  <bhyve:commandline>
-    <bhyve:arg value=3D'-somebhyvearg'/>
-  </bhyve:commandline>
-</domain>
-
    - -

    Note that these extensions are for testing and development purposes onl= y. -They are unsupported, using them may result in inconsistent state, -and upgrading either bhyve or libvirtd maybe break behavior of a domain th= at -was relying on a specific commands pass-through.

    - - - diff --git a/docs/drvbhyve.rst b/docs/drvbhyve.rst new file mode 100644 index 0000000000..95ef4e9b49 --- /dev/null +++ b/docs/drvbhyve.rst @@ -0,0 +1,582 @@ +.. role:: since + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Bhyve driver +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. contents:: + +Bhyve is a FreeBSD hypervisor. It first appeared in FreeBSD 10.0. However,= it's +recommended to keep tracking FreeBSD 10-STABLE to make sure all new featur= es of +bhyve are supported. In order to enable bhyve on your FreeBSD host, you'll= need +to load the ``vmm`` kernel module. Additionally, ``if_tap`` and ``if_bridg= e`` +modules should be loaded for networking support. Also, :since:`since 3.2.0= ` the +``virt-host-validate(1)`` supports the bhyve host validation and could be = used +like this: + +:: + + $ virt-host-validate bhyve + BHYVE: Checking for vmm module = : PASS + BHYVE: Checking for if_tap module = : PASS + BHYVE: Checking for if_bridge module = : PASS + BHYVE: Checking for nmdm module = : PASS + $ + +Additional information on bhyve could be obtained on +`bhyve.org `__. + +Connections to the Bhyve driver +------------------------------- + +The libvirt bhyve driver is a single-instance privileged driver. Some samp= le +connection URIs are: + +:: + + bhyve:///system (local access) + bhyve+unix:///system (local access) + bhyve+ssh://root@example.com/system (remote access, SSH tunnelled) + +Example guest domain XML configurations +--------------------------------------- + +Example config +~~~~~~~~~~~~~~ + +The bhyve driver in libvirt is in its early stage and under active develop= ment. +So it supports only limited number of features bhyve provides. + +Note: in older libvirt versions, only a single network device and a single= disk +device were supported per-domain. However, :since:`since 1.2.6` the libvirt +bhyve driver supports up to 31 PCI devices. + +Note: the Bhyve driver in libvirt will boot whichever device is first. If = you +want to install from CD, put the CD device first. If not, put the root HDD +first. + +Note: Only the SATA bus is supported. Only ``cdrom``- and ``disk``-type di= sks +are supported. + +:: + + + bhyve + df3be7e7-a104-11e3-aeb0-50e5492bd3dc + 219136 + 219136 + 1 + + hvm + + + + + + + destroy + restart + destroy + + + + + + + + + + + + + + + + + + + +(The sections may be swapped in order to install from *cdrom.iso*.) + +Example config (Linux guest) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Note the addition of . + +:: + + + linux_guest + df3be7e7-a104-11e3-aeb0-50e5492bd3dc + 131072 + 131072 + 1 + /usr/local/sbin/grub-bhyve + + hvm + + + + + + + destroy + restart + destroy + + + + + + + + + + + + + + + + + + + +Example config (Linux UEFI guest, VNC, tablet) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This is an example to boot into Fedora 25 installation: + +:: + + + fedora_uefi_vnc_tablet + 4 + 2 + + hvm + /usr/local/share/uefi-f= irmware/BHYVE_UEFI.fd + + + + + + + destroy + restart + destroy + + + + + + + + + + + + + + + + + + + + + + + + + + + +Please refer to the `UEFI <#uefi>`__ section for a more detailed explanati= on. + +Guest usage / management +------------------------ + +Connecting to a guest console +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Guest console connection is supported through the ``nmdm`` device. It coul= d be +enabled by adding the following to the domain XML ( :since:`Since 1.2.4` ): + +:: + + ... + + + + + + ... + +Make sure to load the ``nmdm`` kernel module if you plan to use that. + +Then ``virsh console`` command can be used to connect to the text console = of a +guest. + +**NB:** Some versions of bhyve have a bug that prevents guests from booting +until the console is opened by a client. This bug was fixed in `FreeBSD +changeset r262884 `__. I= f an +older version is used, one either has to open a console manually with +``virsh console`` to let a guest boot or start a guest using: + +:: + + start --console domname + +**NB:** A bootloader configured to require user interaction will prevent t= he +domain from starting (and thus ``virsh console`` or ``start --console`` fr= om +functioning) until the user interacts with it manually on the VM host. Bec= ause +users typically do not have access to the VM host, interactive bootloaders= are +unsupported by libvirt. *However,* if you happen to run into this scenario= and +also happen to have access to the Bhyve host machine, you may select a boot +option and allow the domain to finish starting by using an alternative ter= minal +client on the VM host to connect to the domain-configured null modem devic= e. One +example (assuming ``/dev/nmdm0B`` is configured as the slave end of the do= main +serial device) is: + +:: + + cu -l /dev/nmdm0B + +Converting from domain XML to Bhyve args +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``virsh domxml-to-native`` command can preview the actual ``bhyve`` co= mmands +that will be executed for a given domain. It outputs two lines, the first = line +is a ``bhyveload`` command and the second is a ``bhyve`` command. + +Please note that the ``virsh domxml-to-native`` doesn't do any real actions +other than printing the command, for example, it doesn't try to find a pro= per +TAP interface and create it, like what is done when starting a domain; and +always returns ``tap0`` for the network interface. So if you're going to r= un +these commands manually, most likely you might want to tweak them. + +:: + + # virsh -c "bhyve:///system" domxml-to-native --format bhyve-argv --xm= l /path/to/bhyve.xml + /usr/sbin/bhyveload -m 214 -d /home/user/vm1.img vm1 + /usr/sbin/bhyve -c 2 -m 214 -A -I -H -P -s 0:0,hostbridge \ + -s 3:0,virtio-net,tap0,mac=3D52:54:00:5d:74:e3 -s 2:0,virtio-blk,/h= ome/user/vm1.img \ + -s 1,lpc -l com1,/dev/nmdm0A vm1 + +Using ZFS volumes +~~~~~~~~~~~~~~~~~ + +It's possible to use ZFS volumes as disk devices :since:`since 1.2.8` . An +example of domain XML device entry for that will look like: + +:: + + ... + + + + + ... + +Please refer to the `Storage documentation `__ for more deta= ils on +storage management. + +Using grub2-bhyve or Alternative Bootloaders +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It's possible to boot non-FreeBSD guests by specifying an explicit bootloa= der, +e.g. ``grub-bhyve(1)``. Arguments to the bootloader may be specified as we= ll. If +the bootloader is ``grub-bhyve`` and arguments are omitted, libvirt will t= ry and +infer boot ordering from user-supplied configuration in= the +domain. Failing that, it will boot the first disk in the domain (either +``cdrom``- or ``disk``-type devices). If the disk type is ``disk``, it will +attempt to boot from the first partition in the disk image. + +:: + + ... + /usr/local/sbin/grub-bhyve + ... + ... + +Caveat: ``bootloader_args`` does not support any quoting. Filenames, etc, = must +not have spaces or they will be tokenized incorrectly. + +Using UEFI bootrom, VNC, and USB tablet +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:since:`Since 3.2.0` , in addition to `grub-bhyve <#grubbhyve>`__, non-Fre= eBSD +guests could be also booted using an UEFI boot ROM, provided both guest OS= and +installed ``bhyve(1)`` version support UEFI. To use that, ``loader`` shoul= d be +specified in the ``os`` section: + +:: + + + ... + + hvm + /usr/local/share/uefi-f= irmware/BHYVE_UEFI.fd + + ... + +This uses the UEFI firmware provided by the +`sysutils/bhyve-firmware `__ +FreeBSD port. + +VNC and the tablet input device could be configured this way: + +:: + + + + ... + + + + + + + ... + + +This way, VNC will be accessible on ``127.0.0.1:5904``. + +Please note that the tablet device requires to have a USB controller of the +``nec-xhci`` model. Currently, only a single controller of this type and a +single tablet are supported per domain. + +:since:`Since 3.5.0` , it's possible to configure how the video device is +exposed to the guest using the ``vgaconf`` attribute: + +:: + + + + ... + + + + + ... + + ... + + +If not specified, bhyve's default mode for ``vgaconf`` will be used. Please +refer to the +`bhyve(8) `__ +manual page and the `bhyve wiki `__ for mo= re +details on using the ``vgaconf`` option. + +:since:`Since 3.7.0` , it's possible to use ``autoport`` to let libvirt al= locate +VNC port automatically (instead of explicitly specifying it with the ``por= t`` +attribute): + +:: + + + +:since:`Since 6.8.0` , it's possible to set framebuffer resolution using t= he +``resolution`` sub-element: + +:: + + + +:since:`Since 6.8.0` , VNC server can be configured to use password based +authentication: + +:: + + + + + +Note: VNC password authentication is known to be cryptographically weak. +Additionally, the password is passed as a command line argument in clear t= ext. +Make sure you understand the risks associated with this feature before usi= ng it. + +Clock configuration +~~~~~~~~~~~~~~~~~~~ + +Originally bhyve supported only localtime for RTC. Support for UTC time was +introduced in `FreeBSD changeset +r284894 `__ for *10-STAB= LE* +and in `changeset r279225 `__ +for *-CURRENT*. It's possible to use this in libvirt :since:`since 1.2.18`= , +just place the following to domain XML: + +:: + + + ... + + ... + + +Please note that if you run the older bhyve version that doesn't support U= TC +time, you'll fail to start a domain. As UTC is used as a default when you = do not +specify clock settings, you'll need to explicitly specify 'localtime' in t= his +case: + +:: + + + ... + + ... + + +e1000 NIC +~~~~~~~~~ + +As of `FreeBSD changeset +r302504 `__ bhyve suppor= ts +Intel e1000 network adapter emulation. It's supported in libvirt :since:`s= ince +3.1.0` and could be used as follows: + +:: + + ... + + + + + ... + +Sound device +~~~~~~~~~~~~ + +As of `FreeBSD changeset +r349355 `__ bhyve suppor= ts +sound device emulation. It's supported in libvirt :since:`since 6.7.0` . + +:: + + ... + + + + ... + +Here, the ``sound`` element specifies the sound device as it's exposed to = the +guest, with ``ich7`` being the only supported model now, and the ``audio`` +element specifies how the guest device is mapped to the host sound device. + +Virtio-9p filesystem +~~~~~~~~~~~~~~~~~~~~ + +As of `FreeBSD changeset +r366413 `__ bhyve suppor= ts +sharing arbitrary directory tree between the guest and the host. It's supp= orted +in libvirt :since:`since 6.9.0` . + +:: + + ... + + + + + ... + +This share could be made read only by adding the ```` sub-eleme= nt. + +In the Linux guest, this could be mounted using: + +:: + + mount -t 9p shared_dir /mnt/shared_dir + +Wiring guest memory +~~~~~~~~~~~~~~~~~~~ + +:since:`Since 4.4.0` , it's possible to specify that guest memory should be +wired and cannot be swapped out as follows: + +:: + + + ... + + + + ... + + +CPU topology +~~~~~~~~~~~~ + +:since:`Since 4.5.0` , it's possible to specify guest CPU topology, if bhy= ve +supports that. Support for specifying guest CPU topology was added to bhyv= e in +`FreeBSD changeset r332298 `__ +for *-CURRENT*. Example: + +:: + + + ... + + + + ... + + +Ignoring unknown MSRs reads and writes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:since:`Since 5.1.0` , it's possible to make bhyve ignore accesses to +unimplemented Model Specific Registers (MSRs). Example: + +:: + + + ... + + ... + + ... + + ... + + +Pass-through of arbitrary bhyve commands +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:since:`Since 5.1.0` , it's possible to pass additional command-line argum= ents +to the bhyve process when starting the domain using the ```` +element under ``domain``. To supply an argument, use the element ```` +with the attribute ``value`` set to additional argument to be added. The a= rg +element may be repeated multiple times. To use this XML addition, it is +necessary to issue an XML namespace request (the special ``xmlns:name`` +attribute) that pulls in ``http://libvirt.org/schemas/domain/bhyve/1.0``; +typically, the namespace is given the name of ``bhyve``. + +Example: + +:: + + + ... + + + + + +Note that these extensions are for testing and development purposes only. = They +are **unsupported**, using them may result in inconsistent state, and upgr= ading +either bhyve or libvirtd maybe break behavior of a domain that was relying= on a +specific commands pass-through. diff --git a/docs/meson.build b/docs/meson.build index bb7e27e031..a6c3077f25 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -22,7 +22,6 @@ docs_html_in_files =3D [ 'csharp', 'dbus', 'docs', - 'drvbhyve', 'drvesx', 'drvhyperv', 'drvlxc', @@ -79,6 +78,7 @@ docs_rst_files =3D [ 'daemons', 'downloads', 'drivers', + 'drvbhyve', 'drvch', 'drvqemu', 'errors', --=20 2.35.1