[libvirt PATCHv2 10/10] docs: add virtiofs kbase

Ján Tomko posted 10 patches 6 years ago
There is a newer version of this series
[libvirt PATCHv2 10/10] docs: add virtiofs kbase
Posted by Ján Tomko 6 years ago
Add a document describing the usage of virtiofs.
---
 docs/kbase.html.in      |   3 +
 docs/kbase/virtiofs.rst | 153 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 156 insertions(+)
 create mode 100644 docs/kbase/virtiofs.rst

diff --git a/docs/kbase.html.in b/docs/kbase.html.in
index c156414c41..7d6caf3cb1 100644
--- a/docs/kbase.html.in
+++ b/docs/kbase.html.in
@@ -29,6 +29,9 @@
         <dt><a href="kbase/backing_chains.html">Backing chain management</a></dt>
         <dd>Explanation of how disk backing chain specification impacts libvirt's
           behaviour and basic troubleshooting steps of disk problems.</dd>
+
+        <dt><a href="kbase/virtiofs.html">Virtio-FS</a></dt>
+        <dd>Share a filesystem between the guest and the host</dd>
       </dl>
     </div>
 
diff --git a/docs/kbase/virtiofs.rst b/docs/kbase/virtiofs.rst
new file mode 100644
index 0000000000..83351e0a28
--- /dev/null
+++ b/docs/kbase/virtiofs.rst
@@ -0,0 +1,153 @@
+============================
+Sharing files with Virtio-FS
+============================
+
+=== 8< delete before merging 8< ===
+NOTE: if you're looking at this note, this is just a proposal.
+See the up-to-date version on: https://libvirt.org/kbase/virtiofs.html
+=== 8< --------------------- 8< ===
+
+.. contents::
+
+=========
+Virtio-FS
+=========
+
+Virtio-FS is a delicious delicacy aiming to provide an easy-to-configure
+way of sharing filesystems between the host and the virtual machine.
+
+See https://virtio-fs.gitlab.io/
+
+==========
+Host setup
+==========
+
+The host-side virtiofsd daemon, like other vhost-user backed devices,
+requres shared memory between the host and the guest. As of QEMU 4.2, this
+requires specifying a NUMA topology for the guest and explicitly specifying
+a memory backend. Multiple options are available:
+
+Either of the following:
+
+1. Use file-backed memory
+
+Configure the directory where the files backing the memory will be stored
+with the ``memory_backing_dir`` option in ``/etc/libvirt/qemu.conf``
+
+::
+
+   # This directory is used for memoryBacking source if configured as file.
+   # NOTE: big files will be stored here
+   memory_backing_dir = "/dev/shm/"
+
+2. Use hugepage-backed memory
+
+Make sure there are enough huge pages allocated for the requested guest memory.
+For exmaple, for one guest with 2 GiB of RAM backed by 2 MiB hugepages:
+
+::
+
+   # virsh allocpages 2M 1024
+
+===========
+Guest setup
+===========
+
+1. Specify the NUMA topology
+
+in the domain XML of the guest.
+For the simplest one-node topology for a guest with 2GiB of RAM and 8 vCPUs:
+
+::
+
+   <domain>
+     ...
+     <cpu ...>
+       <numa>
+         <cell id='0' cpus='0-7' memory='2' unit='GiB' memAccess='shared'/>
+       </numa>
+     </cpu>
+     ...
+   </domain>
+
+Note that the CPU element might already be specified and only one is allowed.
+
+2. Specify the memory backend
+
+Either of the following:
+
+2.1. File-backed memory
+
+::
+
+   <domain>
+     ...
+     <memoryBacking>
+       <access mode='shared'/>
+     </memoryBacking>
+     ...
+   </domain>
+
+This will create a file in the directory specified in ``qemu.conf``
+
+2.2. Hugepage-backed memory
+
+::
+
+   <domain>
+     ...
+     <memoryBacking>
+       <hugepages>
+         <page size='2' unit='M'/>
+       </hugepages>
+       <access mode='shared'/>
+     </memoryBacking>
+     ...
+   </domain>
+
+3. Add the ``vhost-user-fs`` QEMU device via the ``filesystem`` element
+
+::
+
+   <domain>
+     ...
+     <devices>
+       ...
+       <filesystem type='mount' accessmode='passthrough'>
+         <driver type='virtiofs'>
+           <binary>/usr/libexec/virtiofsd</binary>
+         </driver>
+         <source dir='/path'/>
+         <target dir='mount_tag'/>
+       </filesystem>
+       ...
+     </devices>
+   </domain>
+
+Note that despite its name, the ``target dir`` is actually a mount tag and does
+not have to correspond to the desired mount point in the guest.
+
+So far, ``passthrough`` is the only supported access mode and it requires
+running the ``virtiofsd`` daemon as root.
+
+4. Boot the guest and mount the filesystem
+
+::
+
+   guest# mount -t virtiofs mount_tag /mnt/mount/path
+
+Note: this requires virtiofs support in the guest kernel (Linux v5.4 or later)
+
+===================
+Optional parameters
+===================
+
+More optional elements can be specified
+
+::
+
+  <driver type='virtiofs' queue='1024' xattr='on'>
+    <binary>/usr/libexec/virtiofsd</binary>
+    <cache mode='always' size='1024' unit='GiB'/>
+    <lock posix_lock='on' flock='on'/>
+  </driver>
-- 
2.21.0

Re: [libvirt PATCHv2 10/10] docs: add virtiofs kbase
Posted by Peter Krempa 6 years ago
On Thu, Jan 23, 2020 at 18:46:11 +0100, Ján Tomko wrote:
> Add a document describing the usage of virtiofs.
> ---
>  docs/kbase.html.in      |   3 +
>  docs/kbase/virtiofs.rst | 153 ++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 156 insertions(+)
>  create mode 100644 docs/kbase/virtiofs.rst

The XML documentation added to formatdomain.html is really lacking. At
the very least this document should be linked from there.

> 
> diff --git a/docs/kbase.html.in b/docs/kbase.html.in
> index c156414c41..7d6caf3cb1 100644
> --- a/docs/kbase.html.in
> +++ b/docs/kbase.html.in
> @@ -29,6 +29,9 @@
>          <dt><a href="kbase/backing_chains.html">Backing chain management</a></dt>
>          <dd>Explanation of how disk backing chain specification impacts libvirt's
>            behaviour and basic troubleshooting steps of disk problems.</dd>
> +
> +        <dt><a href="kbase/virtiofs.html">Virtio-FS</a></dt>
> +        <dd>Share a filesystem between the guest and the host</dd>
>        </dl>
>      </div>
>  
> diff --git a/docs/kbase/virtiofs.rst b/docs/kbase/virtiofs.rst
> new file mode 100644
> index 0000000000..83351e0a28
> --- /dev/null
> +++ b/docs/kbase/virtiofs.rst
> @@ -0,0 +1,153 @@
> +============================
> +Sharing files with Virtio-FS
> +============================
> +
> +=== 8< delete before merging 8< ===
> +NOTE: if you're looking at this note, this is just a proposal.
> +See the up-to-date version on: https://libvirt.org/kbase/virtiofs.html
> +=== 8< --------------------- 8< ===
> +
> +.. contents::
> +
> +=========
> +Virtio-FS
> +=========
> +
> +Virtio-FS is a delicious delicacy aiming to provide an easy-to-configure
> +way of sharing filesystems between the host and the virtual machine.
> +
> +See https://virtio-fs.gitlab.io/
> +
> +==========
> +Host setup
> +==========
> +
> +The host-side virtiofsd daemon, like other vhost-user backed devices,
> +requres shared memory between the host and the guest. As of QEMU 4.2, this
> +requires specifying a NUMA topology for the guest and explicitly specifying
> +a memory backend. Multiple options are available:
> +
> +Either of the following:
> +
> +1. Use file-backed memory
> +
> +Configure the directory where the files backing the memory will be stored
> +with the ``memory_backing_dir`` option in ``/etc/libvirt/qemu.conf``
> +
> +::
> +
> +   # This directory is used for memoryBacking source if configured as file.
> +   # NOTE: big files will be stored here
> +   memory_backing_dir = "/dev/shm/"
> +
> +2. Use hugepage-backed memory
> +
> +Make sure there are enough huge pages allocated for the requested guest memory.
> +For exmaple, for one guest with 2 GiB of RAM backed by 2 MiB hugepages:
> +
> +::
> +
> +   # virsh allocpages 2M 1024
> +
> +===========
> +Guest setup
> +===========
> +
> +1. Specify the NUMA topology
> +
> +in the domain XML of the guest.
> +For the simplest one-node topology for a guest with 2GiB of RAM and 8 vCPUs:
> +
> +::
> +
> +   <domain>
> +     ...
> +     <cpu ...>
> +       <numa>
> +         <cell id='0' cpus='0-7' memory='2' unit='GiB' memAccess='shared'/>
> +       </numa>
> +     </cpu>
> +     ...
> +   </domain>
> +
> +Note that the CPU element might already be specified and only one is allowed.
> +
> +2. Specify the memory backend
> +
> +Either of the following:
> +
> +2.1. File-backed memory
> +
> +::
> +
> +   <domain>
> +     ...
> +     <memoryBacking>
> +       <access mode='shared'/>
> +     </memoryBacking>
> +     ...
> +   </domain>
> +
> +This will create a file in the directory specified in ``qemu.conf``
> +
> +2.2. Hugepage-backed memory
> +
> +::
> +
> +   <domain>
> +     ...
> +     <memoryBacking>
> +       <hugepages>
> +         <page size='2' unit='M'/>
> +       </hugepages>
> +       <access mode='shared'/>
> +     </memoryBacking>
> +     ...
> +   </domain>
> +
> +3. Add the ``vhost-user-fs`` QEMU device via the ``filesystem`` element
> +
> +::
> +
> +   <domain>
> +     ...
> +     <devices>
> +       ...
> +       <filesystem type='mount' accessmode='passthrough'>
> +         <driver type='virtiofs'>
> +           <binary>/usr/libexec/virtiofsd</binary>
> +         </driver>
> +         <source dir='/path'/>
> +         <target dir='mount_tag'/>
> +       </filesystem>
> +       ...
> +     </devices>
> +   </domain>
> +
> +Note that despite its name, the ``target dir`` is actually a mount tag and does
> +not have to correspond to the desired mount point in the guest.

In that case it should perhaps also have a different attribute.

> +
> +So far, ``passthrough`` is the only supported access mode and it requires
> +running the ``virtiofsd`` daemon as root.
> +
> +4. Boot the guest and mount the filesystem
> +
> +::
> +
> +   guest# mount -t virtiofs mount_tag /mnt/mount/path
> +
> +Note: this requires virtiofs support in the guest kernel (Linux v5.4 or later)
> +
> +===================
> +Optional parameters
> +===================
> +
> +More optional elements can be specified
> +
> +::
> +
> +  <driver type='virtiofs' queue='1024' xattr='on'>
> +    <binary>/usr/libexec/virtiofsd</binary>
> +    <cache mode='always' size='1024' unit='GiB'/>
> +    <lock posix_lock='on' flock='on'/>
> +  </driver>
> -- 
> 2.21.0
> 

Re: [libvirt PATCHv2 10/10] docs: add virtiofs kbase
Posted by Ján Tomko 6 years ago
On Fri, Jan 24, 2020 at 10:41:32AM +0100, Peter Krempa wrote:
>On Thu, Jan 23, 2020 at 18:46:11 +0100, Ján Tomko wrote:
>> Add a document describing the usage of virtiofs.
>> ---
>>  docs/kbase.html.in      |   3 +
>>  docs/kbase/virtiofs.rst | 153 ++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 156 insertions(+)
>>  create mode 100644 docs/kbase/virtiofs.rst

[...]

>> +3. Add the ``vhost-user-fs`` QEMU device via the ``filesystem`` element
>> +
>> +::
>> +
>> +   <domain>
>> +     ...
>> +     <devices>
>> +       ...
>> +       <filesystem type='mount' accessmode='passthrough'>
>> +         <driver type='virtiofs'>
>> +           <binary>/usr/libexec/virtiofsd</binary>
>> +         </driver>
>> +         <source dir='/path'/>
>> +         <target dir='mount_tag'/>
>> +       </filesystem>
>> +       ...
>> +     </devices>
>> +   </domain>
>> +
>> +Note that despite its name, the ``target dir`` is actually a mount tag and does
>> +not have to correspond to the desired mount point in the guest.
>
>In that case it should perhaps also have a different attribute.
>

I was considering that, however we already use it the same way for 9pfs.
https://www.redhat.com/archives/libvir-list/2019-November/msg01150.html

Jano
Re: [libvirt PATCHv2 10/10] docs: add virtiofs kbase
Posted by Eric Blake 6 years ago
On 1/23/20 11:46 AM, Ján Tomko wrote:
> Add a document describing the usage of virtiofs.
> ---
>   docs/kbase.html.in      |   3 +
>   docs/kbase/virtiofs.rst | 153 ++++++++++++++++++++++++++++++++++++++++
>   2 files changed, 156 insertions(+)
>   create mode 100644 docs/kbase/virtiofs.rst
> 

> +=== 8< delete before merging 8< ===
> +NOTE: if you're looking at this note, this is just a proposal.
> +See the up-to-date version on: https://libvirt.org/kbase/virtiofs.html
> +=== 8< --------------------- 8< ===

I'm assuming this will be updated ;)

> +
> +.. contents::
> +
> +=========
> +Virtio-FS
> +=========
> +
> +Virtio-FS is a delicious delicacy aiming to provide an easy-to-configure

Fun language, but diverges from the official description...

> +way of sharing filesystems between the host and the virtual machine.
> +
> +See https://virtio-fs.gitlab.io/

...that you find here: "Virtio-fs is a shared file system that lets 
virtual machines access a directory tree on the host. Unlike existing 
approaches, it is designed to offer local file system semantics and 
performance."

I'm not opposed to a fun turn of a phrase in what is otherwise dry 
technical documentation, but wonder if it helps our case here.


> +
> +2. Use hugepage-backed memory
> +
> +Make sure there are enough huge pages allocated for the requested guest memory.
> +For exmaple, for one guest with 2 GiB of RAM backed by 2 MiB hugepages:

example

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [libvirt PATCHv2 10/10] docs: add virtiofs kbase
Posted by Ján Tomko 6 years ago
On Thu, Jan 23, 2020 at 03:58:43PM -0600, Eric Blake wrote:
>On 1/23/20 11:46 AM, Ján Tomko wrote:
>>Add a document describing the usage of virtiofs.
>>---
>>  docs/kbase.html.in      |   3 +
>>  docs/kbase/virtiofs.rst | 153 ++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 156 insertions(+)
>>  create mode 100644 docs/kbase/virtiofs.rst
>>
>
>>+=== 8< delete before merging 8< ===
>>+NOTE: if you're looking at this note, this is just a proposal.
>>+See the up-to-date version on: https://libvirt.org/kbase/virtiofs.html
>>+=== 8< --------------------- 8< ===
>
>I'm assuming this will be updated ;)
>
>>+
>>+.. contents::
>>+
>>+=========
>>+Virtio-FS
>>+=========
>>+
>>+Virtio-FS is a delicious delicacy aiming to provide an easy-to-configure
>
>Fun language, but diverges from the official description...
>
>>+way of sharing filesystems between the host and the virtual machine.
>>+
>>+See https://virtio-fs.gitlab.io/
>
>...that you find here: "Virtio-fs is a shared file system that lets 
>virtual machines access a directory tree on the host. Unlike existing 
>approaches, it is designed to offer local file system semantics and 
>performance."
>
>I'm not opposed to a fun turn of a phrase in what is otherwise dry 
>technical documentation, but wonder if it helps our case here.
>

Oops, I totally forgot about that placeholder text.

Jano

>
>>+
>>+2. Use hugepage-backed memory
>>+
>>+Make sure there are enough huge pages allocated for the requested guest memory.
>>+For exmaple, for one guest with 2 GiB of RAM backed by 2 MiB hugepages:
>
>example
>
>-- 
>Eric Blake, Principal Software Engineer
>Red Hat, Inc.           +1-919-301-3226
>Virtualization:  qemu.org | libvirt.org