[PATCH v10 00/10]vhost-vdpa: add support for configure interrupt

Cindy Lu posted 10 patches 2 years, 6 months ago
Test checkpatch passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20211104164827.21911-1-lulu@redhat.com
Maintainers: Stefan Hajnoczi <stefanha@redhat.com>, "Gonglei (Arei)" <arei.gonglei@huawei.com>, Gerd Hoffmann <kraxel@redhat.com>, "Dr. David Alan Gilbert" <dgilbert@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>, Jason Wang <jasowang@redhat.com>, "Marc-André Lureau" <marcandre.lureau@redhat.com>
There is a newer version of this series
hw/display/vhost-user-gpu.c       |   6 +
hw/net/vhost_net.c                |   9 +
hw/net/virtio-net.c               |  10 +-
hw/virtio/trace-events            |   1 +
hw/virtio/vhost-user-fs.c         |   6 +
hw/virtio/vhost-vdpa.c            |   7 +
hw/virtio/vhost-vsock-common.c    |   6 +
hw/virtio/vhost.c                 |  76 +++++++++
hw/virtio/virtio-crypto.c         |   6 +
hw/virtio/virtio-mmio.c           |  27 +++
hw/virtio/virtio-pci.c            | 269 +++++++++++++++++++++---------
hw/virtio/virtio-pci.h            |   4 +-
hw/virtio/virtio.c                |  29 ++++
include/hw/virtio/vhost-backend.h |   3 +
include/hw/virtio/vhost.h         |   4 +
include/hw/virtio/virtio.h        |   7 +
include/net/vhost_net.h           |   2 +
17 files changed, 389 insertions(+), 83 deletions(-)
[PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Cindy Lu 2 years, 6 months ago
these patches add the support for configure interrupt

These codes are all tested in vp-vdpa (support configure interrupt)
vdpa_sim (not support configure interrupt), virtio tap device

test in virtio-pci bus and virtio-mmio bus

Change in v2:
Add support for virtio-mmio bus
active the notifier while the backend support configure interrupt
misc fixes from v1

Change in v3
fix the coding style problems

Change in v4
misc fixes from v3
merge the set_config_notifier to set_guest_notifier
when vdpa start, check the feature by VIRTIO_NET_F_STATUS

Change in v5
misc fixes from v4
split the code to introduce configure interrupt type and the callback function
will init the configure interrupt in all virtio-pci and virtio-mmio bus, but will
only active while using vhost-vdpa driver

Change in v6
misc fixes from v5
decouple vq from interrupt setting and misc process
fix the bug in virtio_net_handle_rx
use -1 as the queue number to identify if the interrupt is configure interrupt

Change in v7
misc fixes from v6
decouple vq from interrupt setting and misc process
decouple vq from vector use/release process
decouple vq from set notifier fd handler process
move config_notifier and masked_config_notifier to VirtIODevice
fix the bug in virtio_net_handle_rx, add more information
add VIRTIO_CONFIG_IRQ_IDX as the queue number to identify if the interrupt is configure interrupt

Change in v8
misc fixes from v7
decouple vq from interrupt setting and misc process
decouple vq from vector use/release process
decouple vq from set notifier fd handler process
move the vhost configure interrupt to vhost_net

Change in v9
misc fixes from v8
address the comments from v8

Change in v10
fix the hang issue in qtest
address the comments from v9

Cindy Lu (10):
  virtio: introduce macro IRTIO_CONFIG_IRQ_IDX
  virtio-pci: decouple notifier from interrupt process
  virtio-pci: decouple the single vector from the interrupt process
  vhost: introduce new VhostOps vhost_set_config_call
  vhost-vdpa: add support for config interrupt
  virtio: add support for configure interrupt
  vhost: add support for configure interrupt
  virtio-net: add support for configure interrupt
  virtio-mmio: add support for configure interrupt
  virtio-pci: add support for configure interrupt

 hw/display/vhost-user-gpu.c       |   6 +
 hw/net/vhost_net.c                |   9 +
 hw/net/virtio-net.c               |  10 +-
 hw/virtio/trace-events            |   1 +
 hw/virtio/vhost-user-fs.c         |   6 +
 hw/virtio/vhost-vdpa.c            |   7 +
 hw/virtio/vhost-vsock-common.c    |   6 +
 hw/virtio/vhost.c                 |  76 +++++++++
 hw/virtio/virtio-crypto.c         |   6 +
 hw/virtio/virtio-mmio.c           |  27 +++
 hw/virtio/virtio-pci.c            | 269 +++++++++++++++++++++---------
 hw/virtio/virtio-pci.h            |   4 +-
 hw/virtio/virtio.c                |  29 ++++
 include/hw/virtio/vhost-backend.h |   3 +
 include/hw/virtio/vhost.h         |   4 +
 include/hw/virtio/virtio.h        |   7 +
 include/net/vhost_net.h           |   2 +
 17 files changed, 389 insertions(+), 83 deletions(-)

-- 
2.21.3


Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Michael S. Tsirkin 2 years, 6 months ago
On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
> these patches add the support for configure interrupt
> 
> These codes are all tested in vp-vdpa (support configure interrupt)
> vdpa_sim (not support configure interrupt), virtio tap device
> 
> test in virtio-pci bus and virtio-mmio bus

Thanks! Given we are in freeze, I can't merge this now.
Will queue it until after release, if possible please
ping me after the release to help make sure it's not lost then.

> Change in v2:
> Add support for virtio-mmio bus
> active the notifier while the backend support configure interrupt
> misc fixes from v1
> 
> Change in v3
> fix the coding style problems
> 
> Change in v4
> misc fixes from v3
> merge the set_config_notifier to set_guest_notifier
> when vdpa start, check the feature by VIRTIO_NET_F_STATUS
> 
> Change in v5
> misc fixes from v4
> split the code to introduce configure interrupt type and the callback function
> will init the configure interrupt in all virtio-pci and virtio-mmio bus, but will
> only active while using vhost-vdpa driver
> 
> Change in v6
> misc fixes from v5
> decouple vq from interrupt setting and misc process
> fix the bug in virtio_net_handle_rx
> use -1 as the queue number to identify if the interrupt is configure interrupt
> 
> Change in v7
> misc fixes from v6
> decouple vq from interrupt setting and misc process
> decouple vq from vector use/release process
> decouple vq from set notifier fd handler process
> move config_notifier and masked_config_notifier to VirtIODevice
> fix the bug in virtio_net_handle_rx, add more information
> add VIRTIO_CONFIG_IRQ_IDX as the queue number to identify if the interrupt is configure interrupt
> 
> Change in v8
> misc fixes from v7
> decouple vq from interrupt setting and misc process
> decouple vq from vector use/release process
> decouple vq from set notifier fd handler process
> move the vhost configure interrupt to vhost_net
> 
> Change in v9
> misc fixes from v8
> address the comments from v8
> 
> Change in v10
> fix the hang issue in qtest
> address the comments from v9
> 
> Cindy Lu (10):
>   virtio: introduce macro IRTIO_CONFIG_IRQ_IDX
>   virtio-pci: decouple notifier from interrupt process
>   virtio-pci: decouple the single vector from the interrupt process
>   vhost: introduce new VhostOps vhost_set_config_call
>   vhost-vdpa: add support for config interrupt
>   virtio: add support for configure interrupt
>   vhost: add support for configure interrupt
>   virtio-net: add support for configure interrupt
>   virtio-mmio: add support for configure interrupt
>   virtio-pci: add support for configure interrupt
> 
>  hw/display/vhost-user-gpu.c       |   6 +
>  hw/net/vhost_net.c                |   9 +
>  hw/net/virtio-net.c               |  10 +-
>  hw/virtio/trace-events            |   1 +
>  hw/virtio/vhost-user-fs.c         |   6 +
>  hw/virtio/vhost-vdpa.c            |   7 +
>  hw/virtio/vhost-vsock-common.c    |   6 +
>  hw/virtio/vhost.c                 |  76 +++++++++
>  hw/virtio/virtio-crypto.c         |   6 +
>  hw/virtio/virtio-mmio.c           |  27 +++
>  hw/virtio/virtio-pci.c            | 269 +++++++++++++++++++++---------
>  hw/virtio/virtio-pci.h            |   4 +-
>  hw/virtio/virtio.c                |  29 ++++
>  include/hw/virtio/vhost-backend.h |   3 +
>  include/hw/virtio/vhost.h         |   4 +
>  include/hw/virtio/virtio.h        |   7 +
>  include/net/vhost_net.h           |   2 +
>  17 files changed, 389 insertions(+), 83 deletions(-)
> 
> -- 
> 2.21.3


Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Michael S. Tsirkin 2 years, 3 months ago
On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
> these patches add the support for configure interrupt
> 
> These codes are all tested in vp-vdpa (support configure interrupt)
> vdpa_sim (not support configure interrupt), virtio tap device
> 
> test in virtio-pci bus and virtio-mmio bus
> 
> Change in v2:
> Add support for virtio-mmio bus
> active the notifier while the backend support configure interrupt
> misc fixes from v1
> 
> Change in v3
> fix the coding style problems
> 
> Change in v4
> misc fixes from v3
> merge the set_config_notifier to set_guest_notifier
> when vdpa start, check the feature by VIRTIO_NET_F_STATUS
> 
> Change in v5
> misc fixes from v4
> split the code to introduce configure interrupt type and the callback function
> will init the configure interrupt in all virtio-pci and virtio-mmio bus, but will
> only active while using vhost-vdpa driver
> 
> Change in v6
> misc fixes from v5
> decouple vq from interrupt setting and misc process
> fix the bug in virtio_net_handle_rx
> use -1 as the queue number to identify if the interrupt is configure interrupt
> 
> Change in v7
> misc fixes from v6
> decouple vq from interrupt setting and misc process
> decouple vq from vector use/release process
> decouple vq from set notifier fd handler process
> move config_notifier and masked_config_notifier to VirtIODevice
> fix the bug in virtio_net_handle_rx, add more information
> add VIRTIO_CONFIG_IRQ_IDX as the queue number to identify if the interrupt is configure interrupt
> 
> Change in v8
> misc fixes from v7
> decouple vq from interrupt setting and misc process
> decouple vq from vector use/release process
> decouple vq from set notifier fd handler process
> move the vhost configure interrupt to vhost_net
> 
> Change in v9
> misc fixes from v8
> address the comments from v8
> 
> Change in v10
> fix the hang issue in qtest
> address the comments from v9
> 
> Cindy Lu (10):
>   virtio: introduce macro IRTIO_CONFIG_IRQ_IDX
>   virtio-pci: decouple notifier from interrupt process
>   virtio-pci: decouple the single vector from the interrupt process
>   vhost: introduce new VhostOps vhost_set_config_call
>   vhost-vdpa: add support for config interrupt
>   virtio: add support for configure interrupt
>   vhost: add support for configure interrupt
>   virtio-net: add support for configure interrupt
>   virtio-mmio: add support for configure interrupt
>   virtio-pci: add support for configure interrupt
> 
>  hw/display/vhost-user-gpu.c       |   6 +
>  hw/net/vhost_net.c                |   9 +
>  hw/net/virtio-net.c               |  10 +-
>  hw/virtio/trace-events            |   1 +
>  hw/virtio/vhost-user-fs.c         |   6 +
>  hw/virtio/vhost-vdpa.c            |   7 +
>  hw/virtio/vhost-vsock-common.c    |   6 +
>  hw/virtio/vhost.c                 |  76 +++++++++
>  hw/virtio/virtio-crypto.c         |   6 +
>  hw/virtio/virtio-mmio.c           |  27 +++
>  hw/virtio/virtio-pci.c            | 269 +++++++++++++++++++++---------
>  hw/virtio/virtio-pci.h            |   4 +-
>  hw/virtio/virtio.c                |  29 ++++
>  include/hw/virtio/vhost-backend.h |   3 +
>  include/hw/virtio/vhost.h         |   4 +
>  include/hw/virtio/virtio.h        |   7 +
>  include/net/vhost_net.h           |   2 +
>  17 files changed, 389 insertions(+), 83 deletions(-)

So I just realized something. The spec says:

The device MUST set the Device Configuration Interrupt bit
in \field{ISR status} before sending a device configuration
change notification to the driver.

and I don't see how these patches achieve this: it requires
that config interrupts go through userspace.

Revert, and think more about it? Or did I miss something?


> -- 
> 2.21.3


Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Cindy Lu 2 years, 3 months ago
On Mon, Jan 10, 2022 at 1:56 AM Michael S. Tsirkin <mst@redhat.com> wrote:

> On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
> > these patches add the support for configure interrupt
> >
> > These codes are all tested in vp-vdpa (support configure interrupt)
> > vdpa_sim (not support configure interrupt), virtio tap device
> >
> > test in virtio-pci bus and virtio-mmio bus
> >
> > Change in v2:
> > Add support for virtio-mmio bus
> > active the notifier while the backend support configure interrupt
> > misc fixes from v1
> >
> > Change in v3
> > fix the coding style problems
> >
> > Change in v4
> > misc fixes from v3
> > merge the set_config_notifier to set_guest_notifier
> > when vdpa start, check the feature by VIRTIO_NET_F_STATUS
> >
> > Change in v5
> > misc fixes from v4
> > split the code to introduce configure interrupt type and the callback
> function
> > will init the configure interrupt in all virtio-pci and virtio-mmio bus,
> but will
> > only active while using vhost-vdpa driver
> >
> > Change in v6
> > misc fixes from v5
> > decouple vq from interrupt setting and misc process
> > fix the bug in virtio_net_handle_rx
> > use -1 as the queue number to identify if the interrupt is configure
> interrupt
> >
> > Change in v7
> > misc fixes from v6
> > decouple vq from interrupt setting and misc process
> > decouple vq from vector use/release process
> > decouple vq from set notifier fd handler process
> > move config_notifier and masked_config_notifier to VirtIODevice
> > fix the bug in virtio_net_handle_rx, add more information
> > add VIRTIO_CONFIG_IRQ_IDX as the queue number to identify if the
> interrupt is configure interrupt
> >
> > Change in v8
> > misc fixes from v7
> > decouple vq from interrupt setting and misc process
> > decouple vq from vector use/release process
> > decouple vq from set notifier fd handler process
> > move the vhost configure interrupt to vhost_net
> >
> > Change in v9
> > misc fixes from v8
> > address the comments from v8
> >
> > Change in v10
> > fix the hang issue in qtest
> > address the comments from v9
> >
> > Cindy Lu (10):
> >   virtio: introduce macro IRTIO_CONFIG_IRQ_IDX
> >   virtio-pci: decouple notifier from interrupt process
> >   virtio-pci: decouple the single vector from the interrupt process
> >   vhost: introduce new VhostOps vhost_set_config_call
> >   vhost-vdpa: add support for config interrupt
> >   virtio: add support for configure interrupt
> >   vhost: add support for configure interrupt
> >   virtio-net: add support for configure interrupt
> >   virtio-mmio: add support for configure interrupt
> >   virtio-pci: add support for configure interrupt
> >
> >  hw/display/vhost-user-gpu.c       |   6 +
> >  hw/net/vhost_net.c                |   9 +
> >  hw/net/virtio-net.c               |  10 +-
> >  hw/virtio/trace-events            |   1 +
> >  hw/virtio/vhost-user-fs.c         |   6 +
> >  hw/virtio/vhost-vdpa.c            |   7 +
> >  hw/virtio/vhost-vsock-common.c    |   6 +
> >  hw/virtio/vhost.c                 |  76 +++++++++
> >  hw/virtio/virtio-crypto.c         |   6 +
> >  hw/virtio/virtio-mmio.c           |  27 +++
> >  hw/virtio/virtio-pci.c            | 269 +++++++++++++++++++++---------
> >  hw/virtio/virtio-pci.h            |   4 +-
> >  hw/virtio/virtio.c                |  29 ++++
> >  include/hw/virtio/vhost-backend.h |   3 +
> >  include/hw/virtio/vhost.h         |   4 +
> >  include/hw/virtio/virtio.h        |   7 +
> >  include/net/vhost_net.h           |   2 +
> >  17 files changed, 389 insertions(+), 83 deletions(-)
>
> So I just realized something. The spec says:
>
> The device MUST set the Device Configuration Interrupt bit
> in \field{ISR status} before sending a device configuration
> change notification to the driver.
>
> and I don't see how these patches achieve this: it requires
> that config interrupts go through userspace.
>
> Revert, and think more about it? Or did I miss something?
>
>  Thanks, Micheal, I'm ok to revert these patchs and there are also several
bugs  I need to fix.
I will post a new version soon

>
> > --
> > 2.21.3
>
>
Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Jason Wang 2 years, 3 months ago
On Mon, Jan 10, 2022 at 9:37 AM Cindy Lu <lulu@redhat.com> wrote:
>
>
>
>
> On Mon, Jan 10, 2022 at 1:56 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>>
>> On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
>> > these patches add the support for configure interrupt
>> >
>> > These codes are all tested in vp-vdpa (support configure interrupt)
>> > vdpa_sim (not support configure interrupt), virtio tap device
>> >
>> > test in virtio-pci bus and virtio-mmio bus
>> >
>> > Change in v2:
>> > Add support for virtio-mmio bus
>> > active the notifier while the backend support configure interrupt
>> > misc fixes from v1
>> >
>> > Change in v3
>> > fix the coding style problems
>> >
>> > Change in v4
>> > misc fixes from v3
>> > merge the set_config_notifier to set_guest_notifier
>> > when vdpa start, check the feature by VIRTIO_NET_F_STATUS
>> >
>> > Change in v5
>> > misc fixes from v4
>> > split the code to introduce configure interrupt type and the callback function
>> > will init the configure interrupt in all virtio-pci and virtio-mmio bus, but will
>> > only active while using vhost-vdpa driver
>> >
>> > Change in v6
>> > misc fixes from v5
>> > decouple vq from interrupt setting and misc process
>> > fix the bug in virtio_net_handle_rx
>> > use -1 as the queue number to identify if the interrupt is configure interrupt
>> >
>> > Change in v7
>> > misc fixes from v6
>> > decouple vq from interrupt setting and misc process
>> > decouple vq from vector use/release process
>> > decouple vq from set notifier fd handler process
>> > move config_notifier and masked_config_notifier to VirtIODevice
>> > fix the bug in virtio_net_handle_rx, add more information
>> > add VIRTIO_CONFIG_IRQ_IDX as the queue number to identify if the interrupt is configure interrupt
>> >
>> > Change in v8
>> > misc fixes from v7
>> > decouple vq from interrupt setting and misc process
>> > decouple vq from vector use/release process
>> > decouple vq from set notifier fd handler process
>> > move the vhost configure interrupt to vhost_net
>> >
>> > Change in v9
>> > misc fixes from v8
>> > address the comments from v8
>> >
>> > Change in v10
>> > fix the hang issue in qtest
>> > address the comments from v9
>> >
>> > Cindy Lu (10):
>> >   virtio: introduce macro IRTIO_CONFIG_IRQ_IDX
>> >   virtio-pci: decouple notifier from interrupt process
>> >   virtio-pci: decouple the single vector from the interrupt process
>> >   vhost: introduce new VhostOps vhost_set_config_call
>> >   vhost-vdpa: add support for config interrupt
>> >   virtio: add support for configure interrupt
>> >   vhost: add support for configure interrupt
>> >   virtio-net: add support for configure interrupt
>> >   virtio-mmio: add support for configure interrupt
>> >   virtio-pci: add support for configure interrupt
>> >
>> >  hw/display/vhost-user-gpu.c       |   6 +
>> >  hw/net/vhost_net.c                |   9 +
>> >  hw/net/virtio-net.c               |  10 +-
>> >  hw/virtio/trace-events            |   1 +
>> >  hw/virtio/vhost-user-fs.c         |   6 +
>> >  hw/virtio/vhost-vdpa.c            |   7 +
>> >  hw/virtio/vhost-vsock-common.c    |   6 +
>> >  hw/virtio/vhost.c                 |  76 +++++++++
>> >  hw/virtio/virtio-crypto.c         |   6 +
>> >  hw/virtio/virtio-mmio.c           |  27 +++
>> >  hw/virtio/virtio-pci.c            | 269 +++++++++++++++++++++---------
>> >  hw/virtio/virtio-pci.h            |   4 +-
>> >  hw/virtio/virtio.c                |  29 ++++
>> >  include/hw/virtio/vhost-backend.h |   3 +
>> >  include/hw/virtio/vhost.h         |   4 +
>> >  include/hw/virtio/virtio.h        |   7 +
>> >  include/net/vhost_net.h           |   2 +
>> >  17 files changed, 389 insertions(+), 83 deletions(-)
>>
>> So I just realized something. The spec says:
>>
>> The device MUST set the Device Configuration Interrupt bit
>> in \field{ISR status} before sending a device configuration
>> change notification to the driver.
>>
>> and I don't see how these patches achieve this: it requires
>> that config interrupts go through userspace.
>>
>> Revert, and think more about it? Or did I miss something?
>>
>  Thanks, Micheal, I'm ok to revert these patchs and there are also several bugs  I need to fix.
> I will post a new version soon

Please test with vectors=0 for the new version to make it work as expected.

Thanks

>>
>>
>> > --
>> > 2.21.3
>>


Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Cindy Lu 2 years, 3 months ago
On Mon, Jan 10, 2022 at 10:37 AM Jason Wang <jasowang@redhat.com> wrote:

> On Mon, Jan 10, 2022 at 9:37 AM Cindy Lu <lulu@redhat.com> wrote:
> >
> >
> >
> >
> > On Mon, Jan 10, 2022 at 1:56 AM Michael S. Tsirkin <mst@redhat.com>
> wrote:
> >>
> >> On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
> >> > these patches add the support for configure interrupt
> >> >
> >> > These codes are all tested in vp-vdpa (support configure interrupt)
> >> > vdpa_sim (not support configure interrupt), virtio tap device
> >> >
> >> > test in virtio-pci bus and virtio-mmio bus
> >> >
> >> > Change in v2:
> >> > Add support for virtio-mmio bus
> >> > active the notifier while the backend support configure interrupt
> >> > misc fixes from v1
> >> >
> >> > Change in v3
> >> > fix the coding style problems
> >> >
> >> > Change in v4
> >> > misc fixes from v3
> >> > merge the set_config_notifier to set_guest_notifier
> >> > when vdpa start, check the feature by VIRTIO_NET_F_STATUS
> >> >
> >> > Change in v5
> >> > misc fixes from v4
> >> > split the code to introduce configure interrupt type and the callback
> function
> >> > will init the configure interrupt in all virtio-pci and virtio-mmio
> bus, but will
> >> > only active while using vhost-vdpa driver
> >> >
> >> > Change in v6
> >> > misc fixes from v5
> >> > decouple vq from interrupt setting and misc process
> >> > fix the bug in virtio_net_handle_rx
> >> > use -1 as the queue number to identify if the interrupt is configure
> interrupt
> >> >
> >> > Change in v7
> >> > misc fixes from v6
> >> > decouple vq from interrupt setting and misc process
> >> > decouple vq from vector use/release process
> >> > decouple vq from set notifier fd handler process
> >> > move config_notifier and masked_config_notifier to VirtIODevice
> >> > fix the bug in virtio_net_handle_rx, add more information
> >> > add VIRTIO_CONFIG_IRQ_IDX as the queue number to identify if the
> interrupt is configure interrupt
> >> >
> >> > Change in v8
> >> > misc fixes from v7
> >> > decouple vq from interrupt setting and misc process
> >> > decouple vq from vector use/release process
> >> > decouple vq from set notifier fd handler process
> >> > move the vhost configure interrupt to vhost_net
> >> >
> >> > Change in v9
> >> > misc fixes from v8
> >> > address the comments from v8
> >> >
> >> > Change in v10
> >> > fix the hang issue in qtest
> >> > address the comments from v9
> >> >
> >> > Cindy Lu (10):
> >> >   virtio: introduce macro IRTIO_CONFIG_IRQ_IDX
> >> >   virtio-pci: decouple notifier from interrupt process
> >> >   virtio-pci: decouple the single vector from the interrupt process
> >> >   vhost: introduce new VhostOps vhost_set_config_call
> >> >   vhost-vdpa: add support for config interrupt
> >> >   virtio: add support for configure interrupt
> >> >   vhost: add support for configure interrupt
> >> >   virtio-net: add support for configure interrupt
> >> >   virtio-mmio: add support for configure interrupt
> >> >   virtio-pci: add support for configure interrupt
> >> >
> >> >  hw/display/vhost-user-gpu.c       |   6 +
> >> >  hw/net/vhost_net.c                |   9 +
> >> >  hw/net/virtio-net.c               |  10 +-
> >> >  hw/virtio/trace-events            |   1 +
> >> >  hw/virtio/vhost-user-fs.c         |   6 +
> >> >  hw/virtio/vhost-vdpa.c            |   7 +
> >> >  hw/virtio/vhost-vsock-common.c    |   6 +
> >> >  hw/virtio/vhost.c                 |  76 +++++++++
> >> >  hw/virtio/virtio-crypto.c         |   6 +
> >> >  hw/virtio/virtio-mmio.c           |  27 +++
> >> >  hw/virtio/virtio-pci.c            | 269
> +++++++++++++++++++++---------
> >> >  hw/virtio/virtio-pci.h            |   4 +-
> >> >  hw/virtio/virtio.c                |  29 ++++
> >> >  include/hw/virtio/vhost-backend.h |   3 +
> >> >  include/hw/virtio/vhost.h         |   4 +
> >> >  include/hw/virtio/virtio.h        |   7 +
> >> >  include/net/vhost_net.h           |   2 +
> >> >  17 files changed, 389 insertions(+), 83 deletions(-)
> >>
> >> So I just realized something. The spec says:
> >>
> >> The device MUST set the Device Configuration Interrupt bit
> >> in \field{ISR status} before sending a device configuration
> >> change notification to the driver.
> >>
> >> and I don't see how these patches achieve this: it requires
> >> that config interrupts go through userspace.
> >>
> >> Revert, and think more about it? Or did I miss something?
> >>
> >  Thanks, Micheal, I'm ok to revert these patchs and there are also
> several bugs  I need to fix.
> > I will post a new version soon
>
> Please test with vectors=0 for the new version to make it work as expected.
>
> Thanks
>
> Sure, I will, Thanka Jason

> >>
> >>
> >> > --
> >> > 2.21.3
> >>
>
>
Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Michael S. Tsirkin 2 years, 3 months ago
On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
> these patches add the support for configure interrupt
> 
> These codes are all tested in vp-vdpa (support configure interrupt)
> vdpa_sim (not support configure interrupt), virtio tap device
> 
> test in virtio-pci bus and virtio-mmio bus

I am thinking maybe we should revert patches 9,10 for now.
E.g. I'm starting to worry this will make us run out of eventfds.
Can we limit the impact allocating these to only vdpa?

> Change in v2:
> Add support for virtio-mmio bus
> active the notifier while the backend support configure interrupt
> misc fixes from v1
> 
> Change in v3
> fix the coding style problems
> 
> Change in v4
> misc fixes from v3
> merge the set_config_notifier to set_guest_notifier
> when vdpa start, check the feature by VIRTIO_NET_F_STATUS
> 
> Change in v5
> misc fixes from v4
> split the code to introduce configure interrupt type and the callback function
> will init the configure interrupt in all virtio-pci and virtio-mmio bus, but will
> only active while using vhost-vdpa driver
> 
> Change in v6
> misc fixes from v5
> decouple vq from interrupt setting and misc process
> fix the bug in virtio_net_handle_rx
> use -1 as the queue number to identify if the interrupt is configure interrupt
> 
> Change in v7
> misc fixes from v6
> decouple vq from interrupt setting and misc process
> decouple vq from vector use/release process
> decouple vq from set notifier fd handler process
> move config_notifier and masked_config_notifier to VirtIODevice
> fix the bug in virtio_net_handle_rx, add more information
> add VIRTIO_CONFIG_IRQ_IDX as the queue number to identify if the interrupt is configure interrupt
> 
> Change in v8
> misc fixes from v7
> decouple vq from interrupt setting and misc process
> decouple vq from vector use/release process
> decouple vq from set notifier fd handler process
> move the vhost configure interrupt to vhost_net
> 
> Change in v9
> misc fixes from v8
> address the comments from v8
> 
> Change in v10
> fix the hang issue in qtest
> address the comments from v9
> 
> Cindy Lu (10):
>   virtio: introduce macro IRTIO_CONFIG_IRQ_IDX
>   virtio-pci: decouple notifier from interrupt process
>   virtio-pci: decouple the single vector from the interrupt process
>   vhost: introduce new VhostOps vhost_set_config_call
>   vhost-vdpa: add support for config interrupt
>   virtio: add support for configure interrupt
>   vhost: add support for configure interrupt
>   virtio-net: add support for configure interrupt
>   virtio-mmio: add support for configure interrupt
>   virtio-pci: add support for configure interrupt
> 
>  hw/display/vhost-user-gpu.c       |   6 +
>  hw/net/vhost_net.c                |   9 +
>  hw/net/virtio-net.c               |  10 +-
>  hw/virtio/trace-events            |   1 +
>  hw/virtio/vhost-user-fs.c         |   6 +
>  hw/virtio/vhost-vdpa.c            |   7 +
>  hw/virtio/vhost-vsock-common.c    |   6 +
>  hw/virtio/vhost.c                 |  76 +++++++++
>  hw/virtio/virtio-crypto.c         |   6 +
>  hw/virtio/virtio-mmio.c           |  27 +++
>  hw/virtio/virtio-pci.c            | 269 +++++++++++++++++++++---------
>  hw/virtio/virtio-pci.h            |   4 +-
>  hw/virtio/virtio.c                |  29 ++++
>  include/hw/virtio/vhost-backend.h |   3 +
>  include/hw/virtio/vhost.h         |   4 +
>  include/hw/virtio/virtio.h        |   7 +
>  include/net/vhost_net.h           |   2 +
>  17 files changed, 389 insertions(+), 83 deletions(-)
> 
> -- 
> 2.21.3


Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Stefan Hajnoczi 2 years, 6 months ago
On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
> these patches add the support for configure interrupt
> 
> These codes are all tested in vp-vdpa (support configure interrupt)
> vdpa_sim (not support configure interrupt), virtio tap device
> 
> test in virtio-pci bus and virtio-mmio bus

Hi,
vhost-user has a configuration space change notification but it uses a
slave channel message (VHOST_USER_SLAVE_CONFIG_CHANGE_MSG) instead of an
eventfd. Ideally the vhost kernel ioctl and vhost-user interfaces would
follow the same design.

I'm concerned "common" vhost code is going to end up with lots of
callbacks that are not available uniformly across vhost kernel, vdpa,
and vhost-user. That makes it hard to understand and debug vhost, plus
differences make it harder to to correctly extend these interfaces in
the future.

Is the decision to a new eventfd-based interface instead of
vhost_chr_read/write() deliberate?

If yes, do you think VHOST_USER_SLAVE_CONFIG_CHANGE_MSG was a design
mistake in vhost-user that needs to be fixed?

Stefan
Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Jason Wang 2 years, 5 months ago
在 2021/11/8 下午6:53, Stefan Hajnoczi 写道:
> On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
>> these patches add the support for configure interrupt
>>
>> These codes are all tested in vp-vdpa (support configure interrupt)
>> vdpa_sim (not support configure interrupt), virtio tap device
>>
>> test in virtio-pci bus and virtio-mmio bus
> Hi,
> vhost-user has a configuration space change notification but it uses a
> slave channel message (VHOST_USER_SLAVE_CONFIG_CHANGE_MSG) instead of an
> eventfd. Ideally the vhost kernel ioctl and vhost-user interfaces would
> follow the same design.
>
> I'm concerned "common" vhost code is going to end up with lots of
> callbacks that are not available uniformly across vhost kernel, vdpa,
> and vhost-user. That makes it hard to understand and debug vhost, plus
> differences make it harder to to correctly extend these interfaces in
> the future.
>
> Is the decision to a new eventfd-based interface instead of
> vhost_chr_read/write() deliberate?


I think this is a good question. Here're some reasons for using eventfd 
from the kernel perspective:

1) the eventfd is used for relaying interrupts for vqs, so we choose to 
use that for the config interrupt
2) make it possible to be used for irq bypassing (posted interrupt)

 From Qemu's perspective, as shown in this series, it looks more easier 
to be integrated with the both vhost and transport layer (e.g vector 
masking etc).

Thanks


>
> If yes, do you think VHOST_USER_SLAVE_CONFIG_CHANGE_MSG was a design
> mistake in vhost-user that needs to be fixed?
>
> Stefan


Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Stefan Hajnoczi 2 years, 5 months ago
On Thu, Nov 11, 2021 at 12:41:02PM +0800, Jason Wang wrote:
> 
> 在 2021/11/8 下午6:53, Stefan Hajnoczi 写道:
> > On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
> > > these patches add the support for configure interrupt
> > > 
> > > These codes are all tested in vp-vdpa (support configure interrupt)
> > > vdpa_sim (not support configure interrupt), virtio tap device
> > > 
> > > test in virtio-pci bus and virtio-mmio bus
> > Hi,
> > vhost-user has a configuration space change notification but it uses a
> > slave channel message (VHOST_USER_SLAVE_CONFIG_CHANGE_MSG) instead of an
> > eventfd. Ideally the vhost kernel ioctl and vhost-user interfaces would
> > follow the same design.
> > 
> > I'm concerned "common" vhost code is going to end up with lots of
> > callbacks that are not available uniformly across vhost kernel, vdpa,
> > and vhost-user. That makes it hard to understand and debug vhost, plus
> > differences make it harder to to correctly extend these interfaces in
> > the future.
> > 
> > Is the decision to a new eventfd-based interface instead of
> > vhost_chr_read/write() deliberate?
> 
> 
> I think this is a good question. Here're some reasons for using eventfd from
> the kernel perspective:
> 
> 1) the eventfd is used for relaying interrupts for vqs, so we choose to use
> that for the config interrupt
> 2) make it possible to be used for irq bypassing (posted interrupt)

Interesting point. Posted interrupts aren't supported by vhost-user's
slave channel message. Since configuration change notifications are rare
it's probably not a performance problem, but still.

This makes me think vhost-user's approach is sub-optimal, it should have
been an eventfd :(. Maybe the idea was that a slave message is less
complex than adding an additional interface to set a configuration
change notification eventfd.

Let's not worry about it too much. I guess in the long run vhost-user
can be rebased on top of the vDPA kernel interface (I wrote about that
here:
https://blog.vmsplice.net/2020/09/on-unifying-vhost-user-and-virtio.html).

Stefan
Re: [PATCH v10 00/10]vhost-vdpa: add support for configure interrupt
Posted by Jason Wang 2 years, 5 months ago
On Thu, Nov 11, 2021 at 5:22 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>
> On Thu, Nov 11, 2021 at 12:41:02PM +0800, Jason Wang wrote:
> >
> > 在 2021/11/8 下午6:53, Stefan Hajnoczi 写道:
> > > On Fri, Nov 05, 2021 at 12:48:17AM +0800, Cindy Lu wrote:
> > > > these patches add the support for configure interrupt
> > > >
> > > > These codes are all tested in vp-vdpa (support configure interrupt)
> > > > vdpa_sim (not support configure interrupt), virtio tap device
> > > >
> > > > test in virtio-pci bus and virtio-mmio bus
> > > Hi,
> > > vhost-user has a configuration space change notification but it uses a
> > > slave channel message (VHOST_USER_SLAVE_CONFIG_CHANGE_MSG) instead of an
> > > eventfd. Ideally the vhost kernel ioctl and vhost-user interfaces would
> > > follow the same design.
> > >
> > > I'm concerned "common" vhost code is going to end up with lots of
> > > callbacks that are not available uniformly across vhost kernel, vdpa,
> > > and vhost-user. That makes it hard to understand and debug vhost, plus
> > > differences make it harder to to correctly extend these interfaces in
> > > the future.
> > >
> > > Is the decision to a new eventfd-based interface instead of
> > > vhost_chr_read/write() deliberate?
> >
> >
> > I think this is a good question. Here're some reasons for using eventfd from
> > the kernel perspective:
> >
> > 1) the eventfd is used for relaying interrupts for vqs, so we choose to use
> > that for the config interrupt
> > 2) make it possible to be used for irq bypassing (posted interrupt)
>
> Interesting point. Posted interrupts aren't supported by vhost-user's
> slave channel message. Since configuration change notifications are rare
> it's probably not a performance problem, but still.

Yes.

>
> This makes me think vhost-user's approach is sub-optimal, it should have
> been an eventfd :(. Maybe the idea was that a slave message is less
> complex than adding an additional interface to set a configuration
> change notification eventfd.

I agree.

>
> Let's not worry about it too much. I guess in the long run vhost-user
> can be rebased on top of the vDPA kernel interface (I wrote about that
> here:
> https://blog.vmsplice.net/2020/09/on-unifying-vhost-user-and-virtio.html).

Interesting blog, I think this is a good direction we might go. A full
"transport" of virtio instead of just the datapath offloading.

Thanks

>
> Stefan