[libvirt PATCH v5 0/6] Add support for vDPA network devices

Jonathon Jongsma posted 6 patches 3 years, 5 months ago
Test syntax-check failed
Failed in applying to current master (apply log)
create mode 100644 tests/qemuxml2argvdata/net-vdpa.x86_64-latest.args
create mode 100644 tests/qemuxml2argvdata/net-vdpa.xml
create mode 100644 tests/qemuxml2xmloutdata/net-vdpa.xml
[libvirt PATCH v5 0/6] Add support for vDPA network devices
Posted by Jonathon Jongsma 3 years, 5 months ago
vDPA network devices allow high-performance networking in a virtual machine by
providing a wire-speed data path. These devices require a vendor-specific host
driver but the data path follows the virtio specification.

The support for vDPA devices was recently added to qemu. This allows
libvirt to support these devices. This patchset requires that the device is
configured on the host with the appropriate vendor-specific driver.  This will
create a chardev on the host at e.g. /dev/vhost-vdpa-0. That chardev path can
then be used to define a new interface with type=3D'vdpa'.

Note that in order for hot-unplug to work properly, you may need to apply a
qemu patch[1] for now. Without the patch, qemu will not close the fd properly
and any subsequent attempts to use the vdpa chardev will fail like this:

    virsh # attach-device guest1 vdpa.xml
    error: Failed to attach device from vdpa.xml
    error: Unable to open '/dev/vhost-vdpa-0' for vdpa device: Device or reso=
urce busy

[1] https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg06374.html

Changes in v5:
 - rebased to latest master
 - fixed a case where qemuDomainObjExitMonitor() was not called on an error p=
ath
 - Improved the nodedev xml. It now includes the path to the chardev in /dev
   - also updated the nodedev xml schema
 - added sample nodedev-dumpxml output to the commit message of patch #6

Jonathon Jongsma (6):
  conf: Add support for vDPA network devices
  qemu: add vhost-vdpa capability
  qemu: add vdpa support
  qemu: add monitor functions for handling file descriptors
  qemu: support hotplug of vdpa devices
  Include vdpa devices in node device list

 docs/formatdomain.rst                         |  24 +++
 docs/formatnode.html.in                       |   9 +
 docs/schemas/domaincommon.rng                 |  15 ++
 docs/schemas/nodedev.rng                      |  10 +
 include/libvirt/libvirt-nodedev.h             |   1 +
 src/conf/domain_conf.c                        |  31 ++++
 src/conf/domain_conf.h                        |   4 +
 src/conf/netdev_bandwidth_conf.c              |   1 +
 src/conf/node_device_conf.c                   |  14 ++
 src/conf/node_device_conf.h                   |  11 +-
 src/conf/virnodedeviceobj.c                   |   4 +-
 src/libxl/libxl_conf.c                        |   1 +
 src/libxl/xen_common.c                        |   1 +
 src/lxc/lxc_controller.c                      |   1 +
 src/lxc/lxc_driver.c                          |   3 +
 src/lxc/lxc_process.c                         |   1 +
 src/node_device/node_device_udev.c            |  53 ++++++
 src/qemu/qemu_capabilities.c                  |   2 +
 src/qemu/qemu_capabilities.h                  |   1 +
 src/qemu/qemu_command.c                       |  36 +++-
 src/qemu/qemu_command.h                       |   3 +-
 src/qemu/qemu_domain.c                        |   6 +-
 src/qemu/qemu_hotplug.c                       |  75 +++++++-
 src/qemu/qemu_interface.c                     |  25 +++
 src/qemu/qemu_interface.h                     |   2 +
 src/qemu/qemu_migration.c                     |  10 +-
 src/qemu/qemu_monitor.c                       |  93 ++++++++++
 src/qemu/qemu_monitor.h                       |  41 +++++
 src/qemu/qemu_monitor_json.c                  | 173 ++++++++++++++++++
 src/qemu/qemu_monitor_json.h                  |  12 ++
 src/qemu/qemu_process.c                       |   2 +
 src/qemu/qemu_validate.c                      |  15 ++
 src/vmx/vmx.c                                 |   1 +
 .../caps_5.1.0.x86_64.xml                     |   1 +
 .../caps_5.2.0.x86_64.xml                     |   1 +
 tests/qemuhotplugmock.c                       |   9 +
 tests/qemuhotplugtest.c                       |  16 ++
 .../qemuhotplug-interface-vdpa.xml            |   4 +
 .../qemuhotplug-base-live+interface-vdpa.xml  |  57 ++++++
 .../net-vdpa.x86_64-latest.args               |  38 ++++
 tests/qemuxml2argvdata/net-vdpa.xml           |  28 +++
 tests/qemuxml2argvmock.c                      |  11 +-
 tests/qemuxml2argvtest.c                      |   1 +
 tests/qemuxml2xmloutdata/net-vdpa.xml         |  34 ++++
 tests/qemuxml2xmltest.c                       |   1 +
 tools/virsh-domain.c                          |   1 +
 tools/virsh-nodedev.c                         |   3 +
 47 files changed, 870 insertions(+), 16 deletions(-)
 create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-interface-vdpa.x=
ml
 create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+interf=
ace-vdpa.xml
 create mode 100644 tests/qemuxml2argvdata/net-vdpa.x86_64-latest.args
 create mode 100644 tests/qemuxml2argvdata/net-vdpa.xml
 create mode 100644 tests/qemuxml2xmloutdata/net-vdpa.xml

--=20
2.26.2


Re: [libvirt PATCH v5 0/6] Add support for vDPA network devices
Posted by Laine Stump 3 years, 5 months ago
On 10/14/20 1:08 PM, Jonathon Jongsma wrote:
> vDPA network devices allow high-performance networking in a virtual machine by
> providing a wire-speed data path. These devices require a vendor-specific host
> driver but the data path follows the virtio specification.
> 
> The support for vDPA devices was recently added to qemu. This allows
> libvirt to support these devices. This patchset requires that the device is
> configured on the host with the appropriate vendor-specific driver.  This will
> create a chardev on the host at e.g. /dev/vhost-vdpa-0. That chardev path can
> then be used to define a new interface with type=3D'vdpa'.
> 
> Note that in order for hot-unplug to work properly, you may need to apply a
> qemu patch[1] for now. Without the patch, qemu will not close the fd properly
> and any subsequent attempts to use the vdpa chardev will fail like this:
> 
>      virsh # attach-device guest1 vdpa.xml
>      error: Failed to attach device from vdpa.xml
>      error: Unable to open '/dev/vhost-vdpa-0' for vdpa device: Device or reso=
> urce busy
> 
> [1] https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg06374.html
> 
> Changes in v5:
>   - rebased to latest master
>   - fixed a case where qemuDomainObjExitMonitor() was not called on an error p=
> ath
>   - Improved the nodedev xml. It now includes the path to the chardev in /dev
>     - also updated the nodedev xml schema
>   - added sample nodedev-dumpxml output to the commit message of patch #6
> 
> Jonathon Jongsma (6):
>    conf: Add support for vDPA network devices
>    qemu: add vhost-vdpa capability
>    qemu: add vdpa support
>    qemu: add monitor functions for handling file descriptors
>    qemu: support hotplug of vdpa devices
>    Include vdpa devices in node device list

Reviewed-by: Laine Stump <laine@redhat.com> for 1-5

For patch 6 (the nodedev XML additions) I'm holding off on that in case 
anyone has an opinion on changes that should be made there (you had 
expressed some concern about it in IRC) (HINT HINT!!! ANY TAKERS?)

I've done CI testing with all the patches, and am pushing 1-5 now, which 
will hopefully encourage wider testing among people with real hardware 
that's capable of VDPA.



> 
>   docs/formatdomain.rst                         |  24 +++
>   docs/formatnode.html.in                       |   9 +
>   docs/schemas/domaincommon.rng                 |  15 ++
>   docs/schemas/nodedev.rng                      |  10 +
>   include/libvirt/libvirt-nodedev.h             |   1 +
>   src/conf/domain_conf.c                        |  31 ++++
>   src/conf/domain_conf.h                        |   4 +
>   src/conf/netdev_bandwidth_conf.c              |   1 +
>   src/conf/node_device_conf.c                   |  14 ++
>   src/conf/node_device_conf.h                   |  11 +-
>   src/conf/virnodedeviceobj.c                   |   4 +-
>   src/libxl/libxl_conf.c                        |   1 +
>   src/libxl/xen_common.c                        |   1 +
>   src/lxc/lxc_controller.c                      |   1 +
>   src/lxc/lxc_driver.c                          |   3 +
>   src/lxc/lxc_process.c                         |   1 +
>   src/node_device/node_device_udev.c            |  53 ++++++
>   src/qemu/qemu_capabilities.c                  |   2 +
>   src/qemu/qemu_capabilities.h                  |   1 +
>   src/qemu/qemu_command.c                       |  36 +++-
>   src/qemu/qemu_command.h                       |   3 +-
>   src/qemu/qemu_domain.c                        |   6 +-
>   src/qemu/qemu_hotplug.c                       |  75 +++++++-
>   src/qemu/qemu_interface.c                     |  25 +++
>   src/qemu/qemu_interface.h                     |   2 +
>   src/qemu/qemu_migration.c                     |  10 +-
>   src/qemu/qemu_monitor.c                       |  93 ++++++++++
>   src/qemu/qemu_monitor.h                       |  41 +++++
>   src/qemu/qemu_monitor_json.c                  | 173 ++++++++++++++++++
>   src/qemu/qemu_monitor_json.h                  |  12 ++
>   src/qemu/qemu_process.c                       |   2 +
>   src/qemu/qemu_validate.c                      |  15 ++
>   src/vmx/vmx.c                                 |   1 +
>   .../caps_5.1.0.x86_64.xml                     |   1 +
>   .../caps_5.2.0.x86_64.xml                     |   1 +
>   tests/qemuhotplugmock.c                       |   9 +
>   tests/qemuhotplugtest.c                       |  16 ++
>   .../qemuhotplug-interface-vdpa.xml            |   4 +
>   .../qemuhotplug-base-live+interface-vdpa.xml  |  57 ++++++
>   .../net-vdpa.x86_64-latest.args               |  38 ++++
>   tests/qemuxml2argvdata/net-vdpa.xml           |  28 +++
>   tests/qemuxml2argvmock.c                      |  11 +-
>   tests/qemuxml2argvtest.c                      |   1 +
>   tests/qemuxml2xmloutdata/net-vdpa.xml         |  34 ++++
>   tests/qemuxml2xmltest.c                       |   1 +
>   tools/virsh-domain.c                          |   1 +
>   tools/virsh-nodedev.c                         |   3 +
>   47 files changed, 870 insertions(+), 16 deletions(-)
>   create mode 100644 tests/qemuhotplugtestdevices/qemuhotplug-interface-vdpa.x=
> ml
>   create mode 100644 tests/qemuhotplugtestdomains/qemuhotplug-base-live+interf=
> ace-vdpa.xml
>   create mode 100644 tests/qemuxml2argvdata/net-vdpa.x86_64-latest.args
>   create mode 100644 tests/qemuxml2argvdata/net-vdpa.xml
>   create mode 100644 tests/qemuxml2xmloutdata/net-vdpa.xml
> 
> --=20
> 2.26.2
> 

Re: [libvirt PATCH v5 0/6] Add support for vDPA network devices
Posted by Jonathon Jongsma 3 years, 5 months ago
On Tue, 20 Oct 2020 15:16:48 -0400
Laine Stump <laine@redhat.com> wrote:

> On 10/14/20 1:08 PM, Jonathon Jongsma wrote:
> > vDPA network devices allow high-performance networking in a virtual
> > machine by providing a wire-speed data path. These devices require
> > a vendor-specific host driver but the data path follows the virtio
> > specification.
> > 
> > The support for vDPA devices was recently added to qemu. This allows
> > libvirt to support these devices. This patchset requires that the
> > device is configured on the host with the appropriate
> > vendor-specific driver.  This will create a chardev on the host at
> > e.g. /dev/vhost-vdpa-0. That chardev path can then be used to
> > define a new interface with type=3D'vdpa'.
> > 
> > Note that in order for hot-unplug to work properly, you may need to
> > apply a qemu patch[1] for now. Without the patch, qemu will not
> > close the fd properly and any subsequent attempts to use the vdpa
> > chardev will fail like this:
> > 
> >      virsh # attach-device guest1 vdpa.xml
> >      error: Failed to attach device from vdpa.xml
> >      error: Unable to open '/dev/vhost-vdpa-0' for vdpa device:
> > Device or reso= urce busy
> > 
> > [1]
> > https://lists.nongnu.org/archive/html/qemu-devel/2020-09/msg06374.html
> > 
> > Changes in v5:
> >   - rebased to latest master
> >   - fixed a case where qemuDomainObjExitMonitor() was not called on
> > an error p= ath
> >   - Improved the nodedev xml. It now includes the path to the
> > chardev in /dev
> >     - also updated the nodedev xml schema
> >   - added sample nodedev-dumpxml output to the commit message of
> > patch #6
> > 
> > Jonathon Jongsma (6):
> >    conf: Add support for vDPA network devices
> >    qemu: add vhost-vdpa capability
> >    qemu: add vdpa support
> >    qemu: add monitor functions for handling file descriptors
> >    qemu: support hotplug of vdpa devices
> >    Include vdpa devices in node device list  
> 
> Reviewed-by: Laine Stump <laine@redhat.com> for 1-5
> 
> For patch 6 (the nodedev XML additions) I'm holding off on that in
> case anyone has an opinion on changes that should be made there (you
> had expressed some concern about it in IRC) (HINT HINT!!! ANY TAKERS?)

I wouldn't say "concern", necessarily. I just wanted to make sure that
there wasn't a different element name that might be more consistent
with other parts of the XML schema. 

> 
> I've done CI testing with all the patches, and am pushing 1-5 now,
> which will hopefully encourage wider testing among people with real
> hardware that's capable of VDPA.
> 
> 
> 
> > 
> >   docs/formatdomain.rst                         |  24 +++
> >   docs/formatnode.html.in                       |   9 +
> >   docs/schemas/domaincommon.rng                 |  15 ++
> >   docs/schemas/nodedev.rng                      |  10 +
> >   include/libvirt/libvirt-nodedev.h             |   1 +
> >   src/conf/domain_conf.c                        |  31 ++++
> >   src/conf/domain_conf.h                        |   4 +
> >   src/conf/netdev_bandwidth_conf.c              |   1 +
> >   src/conf/node_device_conf.c                   |  14 ++
> >   src/conf/node_device_conf.h                   |  11 +-
> >   src/conf/virnodedeviceobj.c                   |   4 +-
> >   src/libxl/libxl_conf.c                        |   1 +
> >   src/libxl/xen_common.c                        |   1 +
> >   src/lxc/lxc_controller.c                      |   1 +
> >   src/lxc/lxc_driver.c                          |   3 +
> >   src/lxc/lxc_process.c                         |   1 +
> >   src/node_device/node_device_udev.c            |  53 ++++++
> >   src/qemu/qemu_capabilities.c                  |   2 +
> >   src/qemu/qemu_capabilities.h                  |   1 +
> >   src/qemu/qemu_command.c                       |  36 +++-
> >   src/qemu/qemu_command.h                       |   3 +-
> >   src/qemu/qemu_domain.c                        |   6 +-
> >   src/qemu/qemu_hotplug.c                       |  75 +++++++-
> >   src/qemu/qemu_interface.c                     |  25 +++
> >   src/qemu/qemu_interface.h                     |   2 +
> >   src/qemu/qemu_migration.c                     |  10 +-
> >   src/qemu/qemu_monitor.c                       |  93 ++++++++++
> >   src/qemu/qemu_monitor.h                       |  41 +++++
> >   src/qemu/qemu_monitor_json.c                  | 173
> > ++++++++++++++++++ src/qemu/qemu_monitor_json.h                  |
> > 12 ++ src/qemu/qemu_process.c                       |   2 +
> >   src/qemu/qemu_validate.c                      |  15 ++
> >   src/vmx/vmx.c                                 |   1 +
> >   .../caps_5.1.0.x86_64.xml                     |   1 +
> >   .../caps_5.2.0.x86_64.xml                     |   1 +
> >   tests/qemuhotplugmock.c                       |   9 +
> >   tests/qemuhotplugtest.c                       |  16 ++
> >   .../qemuhotplug-interface-vdpa.xml            |   4 +
> >   .../qemuhotplug-base-live+interface-vdpa.xml  |  57 ++++++
> >   .../net-vdpa.x86_64-latest.args               |  38 ++++
> >   tests/qemuxml2argvdata/net-vdpa.xml           |  28 +++
> >   tests/qemuxml2argvmock.c                      |  11 +-
> >   tests/qemuxml2argvtest.c                      |   1 +
> >   tests/qemuxml2xmloutdata/net-vdpa.xml         |  34 ++++
> >   tests/qemuxml2xmltest.c                       |   1 +
> >   tools/virsh-domain.c                          |   1 +
> >   tools/virsh-nodedev.c                         |   3 +
> >   47 files changed, 870 insertions(+), 16 deletions(-)
> >   create mode 100644
> > tests/qemuhotplugtestdevices/qemuhotplug-interface-vdpa.x= ml
> >   create mode 100644
> > tests/qemuhotplugtestdomains/qemuhotplug-base-live+interf=
> > ace-vdpa.xml create mode 100644
> > tests/qemuxml2argvdata/net-vdpa.x86_64-latest.args create mode
> > 100644 tests/qemuxml2argvdata/net-vdpa.xml create mode 100644
> > tests/qemuxml2xmloutdata/net-vdpa.xml
> > 
> > --=20
> > 2.26.2
> >   
>