[PATCH V4 00/10] vhost-vDPA multiqueue

Jason Wang posted 10 patches 2 years, 6 months ago
Test checkpatch failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20211011042829.4159-1-jasowang@redhat.com
Maintainers: Jason Wang <jasowang@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>
There is a newer version of this series
hw/net/vhost_net-stub.c        |   4 +-
hw/net/vhost_net.c             |  55 ++++++++---
hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
hw/virtio/vhost-vdpa.c         |  56 +++++++++--
include/hw/virtio/vhost-vdpa.h |   1 +
include/hw/virtio/vhost.h      |   2 +
include/hw/virtio/virtio-net.h |   5 +-
include/net/net.h              |   5 +
include/net/vhost_net.h        |   6 +-
net/net.c                      |  24 ++++-
net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
11 files changed, 330 insertions(+), 120 deletions(-)
[PATCH V4 00/10] vhost-vDPA multiqueue
Posted by Jason Wang 2 years, 6 months ago
Hi All:

This patch implements the multiqueue support for vhost-vDPA. The most
important requirement si the control virtqueue support. The virtio-net
and vhost-net core are tweak to support control virtqueue as if what
data queue pairs are done: a dedicated vhost_net device which is
coupled with the NetClientState is intrdouced so most of the existing
vhost codes could be reused with minor changes. This means the control
virtqueue will bypass the Qemu. With the control virtqueue, vhost-vDPA
are extend to support creating and destroying multiqueue queue pairs
plus the control virtqueue.

For the future, if we want to support live migration, we can either do
shadow cvq on top or introduce new interfaces for reporting device
states.

Tests are done via the vp_vdpa driver in L1 guest.

Changes since V3:

- fix build with vhost disabled

Changes since V2:

- rebase to qemu master
- use "queue_pairs" instead of "qps"
- typo fixes

Changes since V1:

- start and stop vhost devices when all queues were setup
- fix the case when driver doesn't support MQ but device support
- correctly set the batching capability to avoid a map/unmap storm
- various other tweaks

Jason Wang (10):
  vhost-vdpa: open device fd in net_init_vhost_vdpa()
  vhost-vdpa: classify one time request
  vhost-vdpa: prepare for the multiqueue support
  vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
  net: introduce control client
  vhost-net: control virtqueue support
  virtio-net: use "queue_pairs" instead of "queues" when possible
  vhost: record the last virtqueue index for the virtio device
  virtio-net: vhost control virtqueue support
  vhost-vdpa: multiqueue support

 hw/net/vhost_net-stub.c        |   4 +-
 hw/net/vhost_net.c             |  55 ++++++++---
 hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
 hw/virtio/vhost-vdpa.c         |  56 +++++++++--
 include/hw/virtio/vhost-vdpa.h |   1 +
 include/hw/virtio/vhost.h      |   2 +
 include/hw/virtio/virtio-net.h |   5 +-
 include/net/net.h              |   5 +
 include/net/vhost_net.h        |   6 +-
 net/net.c                      |  24 ++++-
 net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
 11 files changed, 330 insertions(+), 120 deletions(-)

-- 
2.25.1


Re: [PATCH V4 00/10] vhost-vDPA multiqueue
Posted by Michael S. Tsirkin 2 years, 6 months ago
On Mon, Oct 11, 2021 at 12:28:19PM +0800, Jason Wang wrote:
> Hi All:
> 
> This patch implements the multiqueue support for vhost-vDPA. The most
> important requirement si the control virtqueue support. The virtio-net
> and vhost-net core are tweak to support control virtqueue as if what
> data queue pairs are done: a dedicated vhost_net device which is
> coupled with the NetClientState is intrdouced so most of the existing
> vhost codes could be reused with minor changes. This means the control
> virtqueue will bypass the Qemu. With the control virtqueue, vhost-vDPA
> are extend to support creating and destroying multiqueue queue pairs
> plus the control virtqueue.
> 
> For the future, if we want to support live migration, we can either do
> shadow cvq on top or introduce new interfaces for reporting device
> states.


I had to defer this due to rework I picked up.
I pushed an experimental tree - it still has issues but
if you rebase on top of that I can apply.

> Tests are done via the vp_vdpa driver in L1 guest.
> 
> Changes since V3:
> 
> - fix build with vhost disabled
> 
> Changes since V2:
> 
> - rebase to qemu master
> - use "queue_pairs" instead of "qps"
> - typo fixes
> 
> Changes since V1:
> 
> - start and stop vhost devices when all queues were setup
> - fix the case when driver doesn't support MQ but device support
> - correctly set the batching capability to avoid a map/unmap storm
> - various other tweaks
> 
> Jason Wang (10):
>   vhost-vdpa: open device fd in net_init_vhost_vdpa()
>   vhost-vdpa: classify one time request
>   vhost-vdpa: prepare for the multiqueue support
>   vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
>   net: introduce control client
>   vhost-net: control virtqueue support
>   virtio-net: use "queue_pairs" instead of "queues" when possible
>   vhost: record the last virtqueue index for the virtio device
>   virtio-net: vhost control virtqueue support
>   vhost-vdpa: multiqueue support
> 
>  hw/net/vhost_net-stub.c        |   4 +-
>  hw/net/vhost_net.c             |  55 ++++++++---
>  hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
>  hw/virtio/vhost-vdpa.c         |  56 +++++++++--
>  include/hw/virtio/vhost-vdpa.h |   1 +
>  include/hw/virtio/vhost.h      |   2 +
>  include/hw/virtio/virtio-net.h |   5 +-
>  include/net/net.h              |   5 +
>  include/net/vhost_net.h        |   6 +-
>  net/net.c                      |  24 ++++-
>  net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
>  11 files changed, 330 insertions(+), 120 deletions(-)
> 
> -- 
> 2.25.1


Re: [PATCH V4 00/10] vhost-vDPA multiqueue
Posted by Jason Wang 2 years, 6 months ago
On Tue, Oct 19, 2021 at 3:21 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Mon, Oct 11, 2021 at 12:28:19PM +0800, Jason Wang wrote:
> > Hi All:
> >
> > This patch implements the multiqueue support for vhost-vDPA. The most
> > important requirement si the control virtqueue support. The virtio-net
> > and vhost-net core are tweak to support control virtqueue as if what
> > data queue pairs are done: a dedicated vhost_net device which is
> > coupled with the NetClientState is intrdouced so most of the existing
> > vhost codes could be reused with minor changes. This means the control
> > virtqueue will bypass the Qemu. With the control virtqueue, vhost-vDPA
> > are extend to support creating and destroying multiqueue queue pairs
> > plus the control virtqueue.
> >
> > For the future, if we want to support live migration, we can either do
> > shadow cvq on top or introduce new interfaces for reporting device
> > states.
>
>
> I had to defer this due to rework I picked up.
> I pushed an experimental tree - it still has issues but
> if you rebase on top of that I can apply.

Will do. For the tree, is it
git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git next?

Thanks


>
> > Tests are done via the vp_vdpa driver in L1 guest.
> >
> > Changes since V3:
> >
> > - fix build with vhost disabled
> >
> > Changes since V2:
> >
> > - rebase to qemu master
> > - use "queue_pairs" instead of "qps"
> > - typo fixes
> >
> > Changes since V1:
> >
> > - start and stop vhost devices when all queues were setup
> > - fix the case when driver doesn't support MQ but device support
> > - correctly set the batching capability to avoid a map/unmap storm
> > - various other tweaks
> >
> > Jason Wang (10):
> >   vhost-vdpa: open device fd in net_init_vhost_vdpa()
> >   vhost-vdpa: classify one time request
> >   vhost-vdpa: prepare for the multiqueue support
> >   vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
> >   net: introduce control client
> >   vhost-net: control virtqueue support
> >   virtio-net: use "queue_pairs" instead of "queues" when possible
> >   vhost: record the last virtqueue index for the virtio device
> >   virtio-net: vhost control virtqueue support
> >   vhost-vdpa: multiqueue support
> >
> >  hw/net/vhost_net-stub.c        |   4 +-
> >  hw/net/vhost_net.c             |  55 ++++++++---
> >  hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
> >  hw/virtio/vhost-vdpa.c         |  56 +++++++++--
> >  include/hw/virtio/vhost-vdpa.h |   1 +
> >  include/hw/virtio/vhost.h      |   2 +
> >  include/hw/virtio/virtio-net.h |   5 +-
> >  include/net/net.h              |   5 +
> >  include/net/vhost_net.h        |   6 +-
> >  net/net.c                      |  24 ++++-
> >  net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
> >  11 files changed, 330 insertions(+), 120 deletions(-)
> >
> > --
> > 2.25.1
>


Re: [PATCH V4 00/10] vhost-vDPA multiqueue
Posted by Michael S. Tsirkin 2 years, 6 months ago
On Tue, Oct 19, 2021 at 03:24:36PM +0800, Jason Wang wrote:
> On Tue, Oct 19, 2021 at 3:21 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Mon, Oct 11, 2021 at 12:28:19PM +0800, Jason Wang wrote:
> > > Hi All:
> > >
> > > This patch implements the multiqueue support for vhost-vDPA. The most
> > > important requirement si the control virtqueue support. The virtio-net
> > > and vhost-net core are tweak to support control virtqueue as if what
> > > data queue pairs are done: a dedicated vhost_net device which is
> > > coupled with the NetClientState is intrdouced so most of the existing
> > > vhost codes could be reused with minor changes. This means the control
> > > virtqueue will bypass the Qemu. With the control virtqueue, vhost-vDPA
> > > are extend to support creating and destroying multiqueue queue pairs
> > > plus the control virtqueue.
> > >
> > > For the future, if we want to support live migration, we can either do
> > > shadow cvq on top or introduce new interfaces for reporting device
> > > states.
> >
> >
> > I had to defer this due to rework I picked up.
> > I pushed an experimental tree - it still has issues but
> > if you rebase on top of that I can apply.
> 
> Will do. For the tree, is it
> git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git next?
> 
> Thanks
> 

pci

> >
> > > Tests are done via the vp_vdpa driver in L1 guest.
> > >
> > > Changes since V3:
> > >
> > > - fix build with vhost disabled
> > >
> > > Changes since V2:
> > >
> > > - rebase to qemu master
> > > - use "queue_pairs" instead of "qps"
> > > - typo fixes
> > >
> > > Changes since V1:
> > >
> > > - start and stop vhost devices when all queues were setup
> > > - fix the case when driver doesn't support MQ but device support
> > > - correctly set the batching capability to avoid a map/unmap storm
> > > - various other tweaks
> > >
> > > Jason Wang (10):
> > >   vhost-vdpa: open device fd in net_init_vhost_vdpa()
> > >   vhost-vdpa: classify one time request
> > >   vhost-vdpa: prepare for the multiqueue support
> > >   vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
> > >   net: introduce control client
> > >   vhost-net: control virtqueue support
> > >   virtio-net: use "queue_pairs" instead of "queues" when possible
> > >   vhost: record the last virtqueue index for the virtio device
> > >   virtio-net: vhost control virtqueue support
> > >   vhost-vdpa: multiqueue support
> > >
> > >  hw/net/vhost_net-stub.c        |   4 +-
> > >  hw/net/vhost_net.c             |  55 ++++++++---
> > >  hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
> > >  hw/virtio/vhost-vdpa.c         |  56 +++++++++--
> > >  include/hw/virtio/vhost-vdpa.h |   1 +
> > >  include/hw/virtio/vhost.h      |   2 +
> > >  include/hw/virtio/virtio-net.h |   5 +-
> > >  include/net/net.h              |   5 +
> > >  include/net/vhost_net.h        |   6 +-
> > >  net/net.c                      |  24 ++++-
> > >  net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
> > >  11 files changed, 330 insertions(+), 120 deletions(-)
> > >
> > > --
> > > 2.25.1
> >


Re: [PATCH V4 00/10] vhost-vDPA multiqueue
Posted by Michael S. Tsirkin 2 years, 6 months ago
On Tue, Oct 19, 2021 at 06:44:38AM -0400, Michael S. Tsirkin wrote:
> On Tue, Oct 19, 2021 at 03:24:36PM +0800, Jason Wang wrote:
> > On Tue, Oct 19, 2021 at 3:21 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Mon, Oct 11, 2021 at 12:28:19PM +0800, Jason Wang wrote:
> > > > Hi All:
> > > >
> > > > This patch implements the multiqueue support for vhost-vDPA. The most
> > > > important requirement si the control virtqueue support. The virtio-net
> > > > and vhost-net core are tweak to support control virtqueue as if what
> > > > data queue pairs are done: a dedicated vhost_net device which is
> > > > coupled with the NetClientState is intrdouced so most of the existing
> > > > vhost codes could be reused with minor changes. This means the control
> > > > virtqueue will bypass the Qemu. With the control virtqueue, vhost-vDPA
> > > > are extend to support creating and destroying multiqueue queue pairs
> > > > plus the control virtqueue.
> > > >
> > > > For the future, if we want to support live migration, we can either do
> > > > shadow cvq on top or introduce new interfaces for reporting device
> > > > states.
> > >
> > >
> > > I had to defer this due to rework I picked up.
> > > I pushed an experimental tree - it still has issues but
> > > if you rebase on top of that I can apply.
> > 
> > Will do. For the tree, is it
> > git://git.kernel.org/pub/scm/virt/kvm/mst/qemu.git next?
> > 
> > Thanks
> > 
> 
> pci

pushed now

> > >
> > > > Tests are done via the vp_vdpa driver in L1 guest.
> > > >
> > > > Changes since V3:
> > > >
> > > > - fix build with vhost disabled
> > > >
> > > > Changes since V2:
> > > >
> > > > - rebase to qemu master
> > > > - use "queue_pairs" instead of "qps"
> > > > - typo fixes
> > > >
> > > > Changes since V1:
> > > >
> > > > - start and stop vhost devices when all queues were setup
> > > > - fix the case when driver doesn't support MQ but device support
> > > > - correctly set the batching capability to avoid a map/unmap storm
> > > > - various other tweaks
> > > >
> > > > Jason Wang (10):
> > > >   vhost-vdpa: open device fd in net_init_vhost_vdpa()
> > > >   vhost-vdpa: classify one time request
> > > >   vhost-vdpa: prepare for the multiqueue support
> > > >   vhost-vdpa: let net_vhost_vdpa_init() returns NetClientState *
> > > >   net: introduce control client
> > > >   vhost-net: control virtqueue support
> > > >   virtio-net: use "queue_pairs" instead of "queues" when possible
> > > >   vhost: record the last virtqueue index for the virtio device
> > > >   virtio-net: vhost control virtqueue support
> > > >   vhost-vdpa: multiqueue support
> > > >
> > > >  hw/net/vhost_net-stub.c        |   4 +-
> > > >  hw/net/vhost_net.c             |  55 ++++++++---
> > > >  hw/net/virtio-net.c            | 165 ++++++++++++++++++---------------
> > > >  hw/virtio/vhost-vdpa.c         |  56 +++++++++--
> > > >  include/hw/virtio/vhost-vdpa.h |   1 +
> > > >  include/hw/virtio/vhost.h      |   2 +
> > > >  include/hw/virtio/virtio-net.h |   5 +-
> > > >  include/net/net.h              |   5 +
> > > >  include/net/vhost_net.h        |   6 +-
> > > >  net/net.c                      |  24 ++++-
> > > >  net/vhost-vdpa.c               | 127 ++++++++++++++++++++++---
> > > >  11 files changed, 330 insertions(+), 120 deletions(-)
> > > >
> > > > --
> > > > 2.25.1
> > >