[v3] vhost-vDPA multiqueue

[PATCH V3 00/10] vhost-vDPA multiqueue

Posted by Jason Wang 2 years, 8 months ago

Hi All:

This patch implements the multiqueue support for vhost-vDPA. The most
important requirement si the control virtqueue support. The virtio-net
and vhost-net core are tweak to support control virtqueue as if what
data queue pairs are done: a dedicated vhost_net device which is
coupled with the NetClientState is intrdouced so most of the existing
vhost codes could be reused with minor changes. This means the control
virtqueue will bypass the Qemu. With the control virtqueue, vhost-vDPA
are extend to support creating and destroying multiqueue queue pairs
plus the control virtqueue.

For the future, if we want to support live migration, we can either do
shadow cvq on top or introduce new interfaces for reporting device
states.

Tests are done via the vp_vdpa driver in L1 guest.

Changes since V2:

- rebase to qemu master
- use "queue_pairs" instead of "qps"
- typo fixes

Changes since V1:

- start and stop vhost devices when all queues were setup
- fix the case when driver doesn't support MQ but device support
- correctly set the batching capability to avoid a map/unmap storm
- various other tweaks

Jason Wang (10):
  vhost-vdpa: open device fd in net_init_vhost_vdpa()
  vhost-vdpa: classify one time request
  vhost-vdpa: prepare for the multiqueue support
  vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
  net: introduce control client
  vhost-net: control virtqueue support
  virtio-net: use "queue_pairs" instead of "queues" when possible
  vhost: record the last virtqueue index for the virtio device
  virtio-net: vhost control virtqueue support
  vhost-vdpa: multiqueue support

 hw/net/vhost_net.c             |  55 ++++++++---
 hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
 hw/virtio/vhost-vdpa.c         |  56 +++++++++--
 include/hw/virtio/vhost-vdpa.h |   1 +
 include/hw/virtio/vhost.h      |   2 +
 include/hw/virtio/virtio-net.h |   5 +-
 include/net/net.h              |   5 +
 include/net/vhost_net.h        |   6 +-
 net/net.c                      |  24 ++++-
 net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
 10 files changed, 328 insertions(+), 118 deletions(-)

-- 
2.25.1

Re: [PATCH V3 00/10] vhost-vDPA multiqueue

Posted by Jason Wang 2 years, 7 months ago

On Tue, Sep 7, 2021 at 5:03 PM Jason Wang <jasowang@redhat.com> wrote:
>
> Hi All:
>
> This patch implements the multiqueue support for vhost-vDPA. The most
> important requirement si the control virtqueue support. The virtio-net
> and vhost-net core are tweak to support control virtqueue as if what
> data queue pairs are done: a dedicated vhost_net device which is
> coupled with the NetClientState is intrdouced so most of the existing
> vhost codes could be reused with minor changes. This means the control
> virtqueue will bypass the Qemu. With the control virtqueue, vhost-vDPA
> are extend to support creating and destroying multiqueue queue pairs
> plus the control virtqueue.
>
> For the future, if we want to support live migration, we can either do
> shadow cvq on top or introduce new interfaces for reporting device
> states.
>
> Tests are done via the vp_vdpa driver in L1 guest.

Michael, does this look good to you? If yes, do you plan to merge this
or I can do that?

Thanks

>
> Changes since V2:
>
> - rebase to qemu master
> - use "queue_pairs" instead of "qps"
> - typo fixes
>
> Changes since V1:
>
> - start and stop vhost devices when all queues were setup
> - fix the case when driver doesn't support MQ but device support
> - correctly set the batching capability to avoid a map/unmap storm
> - various other tweaks
>
> Jason Wang (10):
>   vhost-vdpa: open device fd in net_init_vhost_vdpa()
>   vhost-vdpa: classify one time request
>   vhost-vdpa: prepare for the multiqueue support
>   vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
>   net: introduce control client
>   vhost-net: control virtqueue support
>   virtio-net: use "queue_pairs" instead of "queues" when possible
>   vhost: record the last virtqueue index for the virtio device
>   virtio-net: vhost control virtqueue support
>   vhost-vdpa: multiqueue support
>
>  hw/net/vhost_net.c             |  55 ++++++++---
>  hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
>  hw/virtio/vhost-vdpa.c         |  56 +++++++++--
>  include/hw/virtio/vhost-vdpa.h |   1 +
>  include/hw/virtio/vhost.h      |   2 +
>  include/hw/virtio/virtio-net.h |   5 +-
>  include/net/net.h              |   5 +
>  include/net/vhost_net.h        |   6 +-
>  net/net.c                      |  24 ++++-
>  net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
>  10 files changed, 328 insertions(+), 118 deletions(-)
>
> --
> 2.25.1
>

RE: [PATCH V3 00/10] vhost-vDPA multiqueue

Posted by Zhang, Chen 2 years, 8 months ago


> -----Original Message-----
> From: Qemu-devel <qemu-devel-
> bounces+chen.zhang=intel.com@nongnu.org> On Behalf Of Jason Wang
> Sent: Tuesday, September 7, 2021 5:03 PM
> To: mst@redhat.com; jasowang@redhat.com; qemu-devel@nongnu.org
> Cc: eperezma@redhat.com; elic@nvidia.com; gdawar@xilinx.com; Zhu,
> Lingshan <lingshan.zhu@intel.com>; lulu@redhat.com
> Subject: [PATCH V3 00/10] vhost-vDPA multiqueue
> 
> Hi All:
> 
> This patch implements the multiqueue support for vhost-vDPA. The most
> important requirement si the control virtqueue support. The virtio-net and

Typo:
 S/si/is


> vhost-net core are tweak to support control virtqueue as if what data queue
> pairs are done: a dedicated vhost_net device which is coupled with the
> NetClientState is intrdouced so most of the existing vhost codes could be
> reused with minor changes. This means the control virtqueue will bypass the
> Qemu. With the control virtqueue, vhost-vDPA are extend to support
> creating and destroying multiqueue queue pairs plus the control virtqueue.
> 
> For the future, if we want to support live migration, we can either do shadow
> cvq on top or introduce new interfaces for reporting device states.
> 
> Tests are done via the vp_vdpa driver in L1 guest.
> 
> Changes since V2:
> 
> - rebase to qemu master
> - use "queue_pairs" instead of "qps"
> - typo fixes
> 
> Changes since V1:
> 
> - start and stop vhost devices when all queues were setup
> - fix the case when driver doesn't support MQ but device support
> - correctly set the batching capability to avoid a map/unmap storm
> - various other tweaks
> 
> Jason Wang (10):
>   vhost-vdpa: open device fd in net_init_vhost_vdpa()
>   vhost-vdpa: classify one time request
>   vhost-vdpa: prepare for the multiqueue support
>   vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
>   net: introduce control client
>   vhost-net: control virtqueue support
>   virtio-net: use "queue_pairs" instead of "queues" when possible
>   vhost: record the last virtqueue index for the virtio device
>   virtio-net: vhost control virtqueue support
>   vhost-vdpa: multiqueue support
> 
>  hw/net/vhost_net.c             |  55 ++++++++---
>  hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
>  hw/virtio/vhost-vdpa.c         |  56 +++++++++--
>  include/hw/virtio/vhost-vdpa.h |   1 +
>  include/hw/virtio/vhost.h      |   2 +
>  include/hw/virtio/virtio-net.h |   5 +-
>  include/net/net.h              |   5 +
>  include/net/vhost_net.h        |   6 +-
>  net/net.c                      |  24 ++++-
>  net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
>  10 files changed, 328 insertions(+), 118 deletions(-)
> 
> --
> 2.25.1
>

Re: [PATCH V3 00/10] vhost-vDPA multiqueue

Posted by Jason Wang 2 years, 8 months ago

On Thu, Sep 9, 2021 at 11:40 PM Zhang, Chen <chen.zhang@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Qemu-devel <qemu-devel-
> > bounces+chen.zhang=intel.com@nongnu.org> On Behalf Of Jason Wang
> > Sent: Tuesday, September 7, 2021 5:03 PM
> > To: mst@redhat.com; jasowang@redhat.com; qemu-devel@nongnu.org
> > Cc: eperezma@redhat.com; elic@nvidia.com; gdawar@xilinx.com; Zhu,
> > Lingshan <lingshan.zhu@intel.com>; lulu@redhat.com
> > Subject: [PATCH V3 00/10] vhost-vDPA multiqueue
> >
> > Hi All:
> >
> > This patch implements the multiqueue support for vhost-vDPA. The most
> > important requirement si the control virtqueue support. The virtio-net and
>
> Typo:
>  S/si/is

Will fix (it there's a respin).

Thanks

>
>
> > vhost-net core are tweak to support control virtqueue as if what data queue
> > pairs are done: a dedicated vhost_net device which is coupled with the
> > NetClientState is intrdouced so most of the existing vhost codes could be
> > reused with minor changes. This means the control virtqueue will bypass the
> > Qemu. With the control virtqueue, vhost-vDPA are extend to support
> > creating and destroying multiqueue queue pairs plus the control virtqueue.
> >
> > For the future, if we want to support live migration, we can either do shadow
> > cvq on top or introduce new interfaces for reporting device states.
> >
> > Tests are done via the vp_vdpa driver in L1 guest.
> >
> > Changes since V2:
> >
> > - rebase to qemu master
> > - use "queue_pairs" instead of "qps"
> > - typo fixes
> >
> > Changes since V1:
> >
> > - start and stop vhost devices when all queues were setup
> > - fix the case when driver doesn't support MQ but device support
> > - correctly set the batching capability to avoid a map/unmap storm
> > - various other tweaks
> >
> > Jason Wang (10):
> >   vhost-vdpa: open device fd in net_init_vhost_vdpa()
> >   vhost-vdpa: classify one time request
> >   vhost-vdpa: prepare for the multiqueue support
> >   vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
> >   net: introduce control client
> >   vhost-net: control virtqueue support
> >   virtio-net: use "queue_pairs" instead of "queues" when possible
> >   vhost: record the last virtqueue index for the virtio device
> >   virtio-net: vhost control virtqueue support
> >   vhost-vdpa: multiqueue support
> >
> >  hw/net/vhost_net.c             |  55 ++++++++---
> >  hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
> >  hw/virtio/vhost-vdpa.c         |  56 +++++++++--
> >  include/hw/virtio/vhost-vdpa.h |   1 +
> >  include/hw/virtio/vhost.h      |   2 +
> >  include/hw/virtio/virtio-net.h |   5 +-
> >  include/net/net.h              |   5 +
> >  include/net/vhost_net.h        |   6 +-
> >  net/net.c                      |  24 ++++-
> >  net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
> >  10 files changed, 328 insertions(+), 118 deletions(-)
> >
> > --
> > 2.25.1
> >
>