[PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern

Kangjie Xu posted 24 patches 1 year, 7 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/cover.1660611460.git.kangjie.xu@linux.alibaba.com
Maintainers: "Michael S. Tsirkin" <mst@redhat.com>, Eduardo Habkost <eduardo@habkost.net>, Marcel Apfelbaum <marcel.apfelbaum@gmail.com>, "Philippe Mathieu-Daudé" <f4bug@amsat.org>, Yanan Wang <wangyanan55@huawei.com>, Jason Wang <jasowang@redhat.com>
docs/interop/vhost-user.rst                   | 10 +++
hw/core/machine.c                             |  1 +
hw/net/vhost_net.c                            | 79 +++++++++++++++++++
hw/net/virtio-net.c                           | 58 ++++++++++++--
hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
hw/virtio/vhost.c                             | 79 +++++++++++++++++--
hw/virtio/virtio-pci.c                        | 20 +++++
hw/virtio/virtio.c                            | 62 +++++++++++----
include/hw/virtio/vhost-backend.h             |  6 ++
include/hw/virtio/vhost.h                     |  5 ++
include/hw/virtio/virtio-pci.h                |  1 +
include/hw/virtio/virtio.h                    |  8 +-
include/net/vhost_net.h                       |  4 +
.../standard-headers/linux/virtio_config.h    |  5 ++
include/standard-headers/linux/virtio_pci.h   |  2 +
15 files changed, 371 insertions(+), 36 deletions(-)
[PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Kangjie Xu 1 year, 7 months ago
The virtio queue reset function has already been defined in the virtio spec 1.2.
The relevant virtio spec information is here:

    https://github.com/oasis-tcs/virtio-spec/issues/124
    https://github.com/oasis-tcs/virtio-spec/issues/139

This patch set is to support this function in QEMU. It consists of several parts:
1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
3. Patches 13-19 support vq stop and vq restart for vhost-user.
4. Patches 20-22 support vq reset and re-enable for virtio-net.
5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.

The process of virtqueue reset can be concluded as:
1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
2. Then the virtqueue can be optionally restarted(re-enabled).

Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
calling process for different modes below.
virtio-net:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
    -> virtio_queue_reset() [virtio]
        -> virtio_net_queue_reset() [virtio-net]
        -> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
    -> set enabled, reset status of vq.

vhost-kernel:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
    -> virtio_queue_reset() [virtio]
        -> virtio_net_queue_reset() [virtio-net]
            -> vhost_net_virtqueue_stop() [vhost-net]
                -> vhost_net_set_backend() [vhost]
                -> vhost_dev_virtqueue_stop()
                    -> vhost_virtqueue_unmap()
        -> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
    -> virtio_queue_enable() [virtio]
        -> virtio_net_queue_enable() [virtio-net]
            -> vhost_net_virtqueue_restart() [vhost-net]
                -> vhost_dev_virtqueue_restart() [vhost]
                    -> vhost_virtqueue_start()
                -> vhost_net_set_backend()
    -> set enabled, reset status of vq.

vhost-user:
1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
    -> virtio_queue_reset() [virtio]
        -> virtio_net_queue_reset() [virtio-net]
            -> vhost_net_virtqueue_stop() [vhost-net]
                -> vhost_dev_virtqueue_stop() [vhost]
                    -> vhost_user_reset_vring() [vhost-user]
                        -> send VHOST_USER_RESET_VRING to the device
                    -> vhost_virtqueue_unmap()
        -> __virtio_queue_reset()
2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
    -> virtio_queue_enable() [virtio]
        -> virtio_net_queue_enable() [virtio-net]
            -> vhost_net_virtqueue_restart() [vhost-net]
                -> vhost_dev_virtqueue_restart() [vhost]
                    -> vhost_virtqueue_start()
                    -> vhost_user_set_single_vring_enable [vhost-user]
                        -> send VHOST_USER_SET_VRING_ENABLE to the device
    -> set enabled, reset status of vq.


Test environment:
    Host: 5.19.0-rc3 (With vq reset support)
    Qemu: QEMU emulator version 7.0.50
    Guest: 5.19.0-rc3 (With vq reset support)
    DPDK: 22.07-rc1 (With vq reset support)
    Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;

    The drvier can resize the virtio queue, then virtio queue reset function should
    be triggered.

    The default is split mode, modify Qemu virtio-net to add PACKED feature to 
    test packed mode.

Guest Kernel Patch:
    https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/

DPDK Patch:
    https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7

Host Kernel Patch:
    https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3

Looking forward to your review and comments. Thanks.

Kangjie Xu (19):
  virtio: introduce virtio_queue_enable()
  virtio: core: vq reset feature negotation support
  virtio-pci: support queue enable
  vhost: extract the logic of unmapping the vrings and desc
  vhost: introduce vhost_dev_virtqueue_stop()
  vhost: introduce vhost_dev_virtqueue_restart()
  vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
  vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
  docs: vhost-user: add VHOST_USER_RESET_VRING message
  vhost-user: introduce vhost_reset_vring() interface
  vhost-user: add op to enable or disable a single vring
  vhost: vhost-user: update vhost_dev_virtqueue_stop()
  vhost: vhost-user: update vhost_dev_virtqueue_restart()
  vhost-net: vhost-user: update vhost_net_virtqueue_stop()
  vhost-net: vhost-user: update vhost_net_virtqueue_restart()
  virtio-net: introduce flush_or_purge_queued_packets()
  virtio-net: support queue_enable
  vhost: vhost-kernel: enable vq reset feature
  vhost: vhost-user: enable vq reset feature

Xuan Zhuo (5):
  virtio: sync relevant definitions with linux
  virtio: introduce __virtio_queue_reset()
  virtio: introduce virtio_queue_reset()
  virtio-pci: support queue reset
  virtio-net: support queue reset

 docs/interop/vhost-user.rst                   | 10 +++
 hw/core/machine.c                             |  1 +
 hw/net/vhost_net.c                            | 79 +++++++++++++++++++
 hw/net/virtio-net.c                           | 58 ++++++++++++--
 hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
 hw/virtio/vhost.c                             | 79 +++++++++++++++++--
 hw/virtio/virtio-pci.c                        | 20 +++++
 hw/virtio/virtio.c                            | 62 +++++++++++----
 include/hw/virtio/vhost-backend.h             |  6 ++
 include/hw/virtio/vhost.h                     |  5 ++
 include/hw/virtio/virtio-pci.h                |  1 +
 include/hw/virtio/virtio.h                    |  8 +-
 include/net/vhost_net.h                       |  4 +
 .../standard-headers/linux/virtio_config.h    |  5 ++
 include/standard-headers/linux/virtio_pci.h   |  2 +
 15 files changed, 371 insertions(+), 36 deletions(-)

-- 
2.32.0
Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Michael S. Tsirkin 1 year, 7 months ago
On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
> The virtio queue reset function has already been defined in the virtio spec 1.2.
> The relevant virtio spec information is here:
> 
>     https://github.com/oasis-tcs/virtio-spec/issues/124
>     https://github.com/oasis-tcs/virtio-spec/issues/139
> 
> This patch set is to support this function in QEMU. It consists of several parts:
> 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
> 
> The process of virtqueue reset can be concluded as:
> 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> 2. Then the virtqueue can be optionally restarted(re-enabled).
> 
> Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> calling process for different modes below.
> virtio-net:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>     -> virtio_queue_reset() [virtio]
>         -> virtio_net_queue_reset() [virtio-net]
>         -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>     -> set enabled, reset status of vq.
> 
> vhost-kernel:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>     -> virtio_queue_reset() [virtio]
>         -> virtio_net_queue_reset() [virtio-net]
>             -> vhost_net_virtqueue_stop() [vhost-net]
>                 -> vhost_net_set_backend() [vhost]
>                 -> vhost_dev_virtqueue_stop()
>                     -> vhost_virtqueue_unmap()
>         -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>     -> virtio_queue_enable() [virtio]
>         -> virtio_net_queue_enable() [virtio-net]
>             -> vhost_net_virtqueue_restart() [vhost-net]
>                 -> vhost_dev_virtqueue_restart() [vhost]
>                     -> vhost_virtqueue_start()
>                 -> vhost_net_set_backend()
>     -> set enabled, reset status of vq.
> 
> vhost-user:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>     -> virtio_queue_reset() [virtio]
>         -> virtio_net_queue_reset() [virtio-net]
>             -> vhost_net_virtqueue_stop() [vhost-net]
>                 -> vhost_dev_virtqueue_stop() [vhost]
>                     -> vhost_user_reset_vring() [vhost-user]
>                         -> send VHOST_USER_RESET_VRING to the device
>                     -> vhost_virtqueue_unmap()
>         -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>     -> virtio_queue_enable() [virtio]
>         -> virtio_net_queue_enable() [virtio-net]
>             -> vhost_net_virtqueue_restart() [vhost-net]
>                 -> vhost_dev_virtqueue_restart() [vhost]
>                     -> vhost_virtqueue_start()
>                     -> vhost_user_set_single_vring_enable [vhost-user]
>                         -> send VHOST_USER_SET_VRING_ENABLE to the device
>     -> set enabled, reset status of vq.
> 
> 
> Test environment:
>     Host: 5.19.0-rc3 (With vq reset support)
>     Qemu: QEMU emulator version 7.0.50
>     Guest: 5.19.0-rc3 (With vq reset support)
>     DPDK: 22.07-rc1 (With vq reset support)
>     Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
> 
>     The drvier can resize the virtio queue, then virtio queue reset function should
>     be triggered.
> 
>     The default is split mode, modify Qemu virtio-net to add PACKED feature to 
>     test packed mode.

legacy mode testing?

> Guest Kernel Patch:
>     https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
> 
> DPDK Patch:
>     https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
> 
> Host Kernel Patch:
>     https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
> 
> Looking forward to your review and comments. Thanks.
> 
> Kangjie Xu (19):
>   virtio: introduce virtio_queue_enable()
>   virtio: core: vq reset feature negotation support
>   virtio-pci: support queue enable
>   vhost: extract the logic of unmapping the vrings and desc
>   vhost: introduce vhost_dev_virtqueue_stop()
>   vhost: introduce vhost_dev_virtqueue_restart()
>   vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
>   vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
>   docs: vhost-user: add VHOST_USER_RESET_VRING message
>   vhost-user: introduce vhost_reset_vring() interface
>   vhost-user: add op to enable or disable a single vring
>   vhost: vhost-user: update vhost_dev_virtqueue_stop()
>   vhost: vhost-user: update vhost_dev_virtqueue_restart()
>   vhost-net: vhost-user: update vhost_net_virtqueue_stop()
>   vhost-net: vhost-user: update vhost_net_virtqueue_restart()
>   virtio-net: introduce flush_or_purge_queued_packets()
>   virtio-net: support queue_enable
>   vhost: vhost-kernel: enable vq reset feature
>   vhost: vhost-user: enable vq reset feature
> 
> Xuan Zhuo (5):
>   virtio: sync relevant definitions with linux
>   virtio: introduce __virtio_queue_reset()
>   virtio: introduce virtio_queue_reset()
>   virtio-pci: support queue reset
>   virtio-net: support queue reset
> 
>  docs/interop/vhost-user.rst                   | 10 +++
>  hw/core/machine.c                             |  1 +
>  hw/net/vhost_net.c                            | 79 +++++++++++++++++++
>  hw/net/virtio-net.c                           | 58 ++++++++++++--
>  hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
>  hw/virtio/vhost.c                             | 79 +++++++++++++++++--
>  hw/virtio/virtio-pci.c                        | 20 +++++
>  hw/virtio/virtio.c                            | 62 +++++++++++----
>  include/hw/virtio/vhost-backend.h             |  6 ++
>  include/hw/virtio/vhost.h                     |  5 ++
>  include/hw/virtio/virtio-pci.h                |  1 +
>  include/hw/virtio/virtio.h                    |  8 +-
>  include/net/vhost_net.h                       |  4 +
>  .../standard-headers/linux/virtio_config.h    |  5 ++
>  include/standard-headers/linux/virtio_pci.h   |  2 +
>  15 files changed, 371 insertions(+), 36 deletions(-)
> 
> -- 
> 2.32.0
Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Xuan Zhuo 1 year, 7 months ago
On Tue, 16 Aug 2022 02:14:10 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
> > The virtio queue reset function has already been defined in the virtio spec 1.2.
> > The relevant virtio spec information is here:
> >
> >     https://github.com/oasis-tcs/virtio-spec/issues/124
> >     https://github.com/oasis-tcs/virtio-spec/issues/139
> >
> > This patch set is to support this function in QEMU. It consists of several parts:
> > 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> > 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> > 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> > 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> > 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
> >
> > The process of virtqueue reset can be concluded as:
> > 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> > 2. Then the virtqueue can be optionally restarted(re-enabled).
> >
> > Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> > calling process for different modes below.
> > virtio-net:
> > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> >     -> virtio_queue_reset() [virtio]
> >         -> virtio_net_queue_reset() [virtio-net]
> >         -> __virtio_queue_reset()
> > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> >     -> set enabled, reset status of vq.
> >
> > vhost-kernel:
> > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> >     -> virtio_queue_reset() [virtio]
> >         -> virtio_net_queue_reset() [virtio-net]
> >             -> vhost_net_virtqueue_stop() [vhost-net]
> >                 -> vhost_net_set_backend() [vhost]
> >                 -> vhost_dev_virtqueue_stop()
> >                     -> vhost_virtqueue_unmap()
> >         -> __virtio_queue_reset()
> > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> >     -> virtio_queue_enable() [virtio]
> >         -> virtio_net_queue_enable() [virtio-net]
> >             -> vhost_net_virtqueue_restart() [vhost-net]
> >                 -> vhost_dev_virtqueue_restart() [vhost]
> >                     -> vhost_virtqueue_start()
> >                 -> vhost_net_set_backend()
> >     -> set enabled, reset status of vq.
> >
> > vhost-user:
> > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> >     -> virtio_queue_reset() [virtio]
> >         -> virtio_net_queue_reset() [virtio-net]
> >             -> vhost_net_virtqueue_stop() [vhost-net]
> >                 -> vhost_dev_virtqueue_stop() [vhost]
> >                     -> vhost_user_reset_vring() [vhost-user]
> >                         -> send VHOST_USER_RESET_VRING to the device
> >                     -> vhost_virtqueue_unmap()
> >         -> __virtio_queue_reset()
> > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> >     -> virtio_queue_enable() [virtio]
> >         -> virtio_net_queue_enable() [virtio-net]
> >             -> vhost_net_virtqueue_restart() [vhost-net]
> >                 -> vhost_dev_virtqueue_restart() [vhost]
> >                     -> vhost_virtqueue_start()
> >                     -> vhost_user_set_single_vring_enable [vhost-user]
> >                         -> send VHOST_USER_SET_VRING_ENABLE to the device
> >     -> set enabled, reset status of vq.
> >
> >
> > Test environment:
> >     Host: 5.19.0-rc3 (With vq reset support)
> >     Qemu: QEMU emulator version 7.0.50
> >     Guest: 5.19.0-rc3 (With vq reset support)
> >     DPDK: 22.07-rc1 (With vq reset support)
> >     Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
> >
> >     The drvier can resize the virtio queue, then virtio queue reset function should
> >     be triggered.
> >
> >     The default is split mode, modify Qemu virtio-net to add PACKED feature to
> >     test packed mode.
>
> legacy mode testing?


legacy does not support vq reset.

Thanks.

>
> > Guest Kernel Patch:
> >     https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
> >
> > DPDK Patch:
> >     https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
> >
> > Host Kernel Patch:
> >     https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
> >
> > Looking forward to your review and comments. Thanks.
> >
> > Kangjie Xu (19):
> >   virtio: introduce virtio_queue_enable()
> >   virtio: core: vq reset feature negotation support
> >   virtio-pci: support queue enable
> >   vhost: extract the logic of unmapping the vrings and desc
> >   vhost: introduce vhost_dev_virtqueue_stop()
> >   vhost: introduce vhost_dev_virtqueue_restart()
> >   vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> >   vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> >   docs: vhost-user: add VHOST_USER_RESET_VRING message
> >   vhost-user: introduce vhost_reset_vring() interface
> >   vhost-user: add op to enable or disable a single vring
> >   vhost: vhost-user: update vhost_dev_virtqueue_stop()
> >   vhost: vhost-user: update vhost_dev_virtqueue_restart()
> >   vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> >   vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> >   virtio-net: introduce flush_or_purge_queued_packets()
> >   virtio-net: support queue_enable
> >   vhost: vhost-kernel: enable vq reset feature
> >   vhost: vhost-user: enable vq reset feature
> >
> > Xuan Zhuo (5):
> >   virtio: sync relevant definitions with linux
> >   virtio: introduce __virtio_queue_reset()
> >   virtio: introduce virtio_queue_reset()
> >   virtio-pci: support queue reset
> >   virtio-net: support queue reset
> >
> >  docs/interop/vhost-user.rst                   | 10 +++
> >  hw/core/machine.c                             |  1 +
> >  hw/net/vhost_net.c                            | 79 +++++++++++++++++++
> >  hw/net/virtio-net.c                           | 58 ++++++++++++--
> >  hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
> >  hw/virtio/vhost.c                             | 79 +++++++++++++++++--
> >  hw/virtio/virtio-pci.c                        | 20 +++++
> >  hw/virtio/virtio.c                            | 62 +++++++++++----
> >  include/hw/virtio/vhost-backend.h             |  6 ++
> >  include/hw/virtio/vhost.h                     |  5 ++
> >  include/hw/virtio/virtio-pci.h                |  1 +
> >  include/hw/virtio/virtio.h                    |  8 +-
> >  include/net/vhost_net.h                       |  4 +
> >  .../standard-headers/linux/virtio_config.h    |  5 ++
> >  include/standard-headers/linux/virtio_pci.h   |  2 +
> >  15 files changed, 371 insertions(+), 36 deletions(-)
> >
> > --
> > 2.32.0
>
Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Michael S. Tsirkin 1 year, 7 months ago
On Tue, Aug 16, 2022 at 02:15:57PM +0800, Xuan Zhuo wrote:
> On Tue, 16 Aug 2022 02:14:10 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
> > > The virtio queue reset function has already been defined in the virtio spec 1.2.
> > > The relevant virtio spec information is here:
> > >
> > >     https://github.com/oasis-tcs/virtio-spec/issues/124
> > >     https://github.com/oasis-tcs/virtio-spec/issues/139
> > >
> > > This patch set is to support this function in QEMU. It consists of several parts:
> > > 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> > > 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> > > 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> > > 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> > > 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
> > >
> > > The process of virtqueue reset can be concluded as:
> > > 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> > > 2. Then the virtqueue can be optionally restarted(re-enabled).
> > >
> > > Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> > > calling process for different modes below.
> > > virtio-net:
> > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > >     -> virtio_queue_reset() [virtio]
> > >         -> virtio_net_queue_reset() [virtio-net]
> > >         -> __virtio_queue_reset()
> > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > >     -> set enabled, reset status of vq.
> > >
> > > vhost-kernel:
> > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > >     -> virtio_queue_reset() [virtio]
> > >         -> virtio_net_queue_reset() [virtio-net]
> > >             -> vhost_net_virtqueue_stop() [vhost-net]
> > >                 -> vhost_net_set_backend() [vhost]
> > >                 -> vhost_dev_virtqueue_stop()
> > >                     -> vhost_virtqueue_unmap()
> > >         -> __virtio_queue_reset()
> > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > >     -> virtio_queue_enable() [virtio]
> > >         -> virtio_net_queue_enable() [virtio-net]
> > >             -> vhost_net_virtqueue_restart() [vhost-net]
> > >                 -> vhost_dev_virtqueue_restart() [vhost]
> > >                     -> vhost_virtqueue_start()
> > >                 -> vhost_net_set_backend()
> > >     -> set enabled, reset status of vq.
> > >
> > > vhost-user:
> > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > >     -> virtio_queue_reset() [virtio]
> > >         -> virtio_net_queue_reset() [virtio-net]
> > >             -> vhost_net_virtqueue_stop() [vhost-net]
> > >                 -> vhost_dev_virtqueue_stop() [vhost]
> > >                     -> vhost_user_reset_vring() [vhost-user]
> > >                         -> send VHOST_USER_RESET_VRING to the device
> > >                     -> vhost_virtqueue_unmap()
> > >         -> __virtio_queue_reset()
> > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > >     -> virtio_queue_enable() [virtio]
> > >         -> virtio_net_queue_enable() [virtio-net]
> > >             -> vhost_net_virtqueue_restart() [vhost-net]
> > >                 -> vhost_dev_virtqueue_restart() [vhost]
> > >                     -> vhost_virtqueue_start()
> > >                     -> vhost_user_set_single_vring_enable [vhost-user]
> > >                         -> send VHOST_USER_SET_VRING_ENABLE to the device
> > >     -> set enabled, reset status of vq.
> > >
> > >
> > > Test environment:
> > >     Host: 5.19.0-rc3 (With vq reset support)
> > >     Qemu: QEMU emulator version 7.0.50
> > >     Guest: 5.19.0-rc3 (With vq reset support)
> > >     DPDK: 22.07-rc1 (With vq reset support)
> > >     Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
> > >
> > >     The drvier can resize the virtio queue, then virtio queue reset function should
> > >     be triggered.
> > >
> > >     The default is split mode, modify Qemu virtio-net to add PACKED feature to
> > >     test packed mode.
> >
> > legacy mode testing?
> 
> 
> legacy does not support vq reset.
> 
> Thanks.

yes but did it break with all these code changes?

> >
> > > Guest Kernel Patch:
> > >     https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
> > >
> > > DPDK Patch:
> > >     https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
> > >
> > > Host Kernel Patch:
> > >     https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
> > >
> > > Looking forward to your review and comments. Thanks.
> > >
> > > Kangjie Xu (19):
> > >   virtio: introduce virtio_queue_enable()
> > >   virtio: core: vq reset feature negotation support
> > >   virtio-pci: support queue enable
> > >   vhost: extract the logic of unmapping the vrings and desc
> > >   vhost: introduce vhost_dev_virtqueue_stop()
> > >   vhost: introduce vhost_dev_virtqueue_restart()
> > >   vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> > >   vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> > >   docs: vhost-user: add VHOST_USER_RESET_VRING message
> > >   vhost-user: introduce vhost_reset_vring() interface
> > >   vhost-user: add op to enable or disable a single vring
> > >   vhost: vhost-user: update vhost_dev_virtqueue_stop()
> > >   vhost: vhost-user: update vhost_dev_virtqueue_restart()
> > >   vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> > >   vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> > >   virtio-net: introduce flush_or_purge_queued_packets()
> > >   virtio-net: support queue_enable
> > >   vhost: vhost-kernel: enable vq reset feature
> > >   vhost: vhost-user: enable vq reset feature
> > >
> > > Xuan Zhuo (5):
> > >   virtio: sync relevant definitions with linux
> > >   virtio: introduce __virtio_queue_reset()
> > >   virtio: introduce virtio_queue_reset()
> > >   virtio-pci: support queue reset
> > >   virtio-net: support queue reset
> > >
> > >  docs/interop/vhost-user.rst                   | 10 +++
> > >  hw/core/machine.c                             |  1 +
> > >  hw/net/vhost_net.c                            | 79 +++++++++++++++++++
> > >  hw/net/virtio-net.c                           | 58 ++++++++++++--
> > >  hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
> > >  hw/virtio/vhost.c                             | 79 +++++++++++++++++--
> > >  hw/virtio/virtio-pci.c                        | 20 +++++
> > >  hw/virtio/virtio.c                            | 62 +++++++++++----
> > >  include/hw/virtio/vhost-backend.h             |  6 ++
> > >  include/hw/virtio/vhost.h                     |  5 ++
> > >  include/hw/virtio/virtio-pci.h                |  1 +
> > >  include/hw/virtio/virtio.h                    |  8 +-
> > >  include/net/vhost_net.h                       |  4 +
> > >  .../standard-headers/linux/virtio_config.h    |  5 ++
> > >  include/standard-headers/linux/virtio_pci.h   |  2 +
> > >  15 files changed, 371 insertions(+), 36 deletions(-)
> > >
> > > --
> > > 2.32.0
> >
Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Kangjie Xu 1 year, 7 months ago
在 2022/8/16 14:22, Michael S. Tsirkin 写道:
> On Tue, Aug 16, 2022 at 02:15:57PM +0800, Xuan Zhuo wrote:
>> On Tue, 16 Aug 2022 02:14:10 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
>>> On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
>>>> The virtio queue reset function has already been defined in the virtio spec 1.2.
>>>> The relevant virtio spec information is here:
>>>>
>>>>      https://github.com/oasis-tcs/virtio-spec/issues/124
>>>>      https://github.com/oasis-tcs/virtio-spec/issues/139
>>>>
>>>> This patch set is to support this function in QEMU. It consists of several parts:
>>>> 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
>>>> 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
>>>> 3. Patches 13-19 support vq stop and vq restart for vhost-user.
>>>> 4. Patches 20-22 support vq reset and re-enable for virtio-net.
>>>> 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
>>>>
>>>> The process of virtqueue reset can be concluded as:
>>>> 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
>>>> 2. Then the virtqueue can be optionally restarted(re-enabled).
>>>>
>>>> Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
>>>> calling process for different modes below.
>>>> virtio-net:
>>>> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>>>>      -> virtio_queue_reset() [virtio]
>>>>          -> virtio_net_queue_reset() [virtio-net]
>>>>          -> __virtio_queue_reset()
>>>> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>>>>      -> set enabled, reset status of vq.
>>>>
>>>> vhost-kernel:
>>>> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>>>>      -> virtio_queue_reset() [virtio]
>>>>          -> virtio_net_queue_reset() [virtio-net]
>>>>              -> vhost_net_virtqueue_stop() [vhost-net]
>>>>                  -> vhost_net_set_backend() [vhost]
>>>>                  -> vhost_dev_virtqueue_stop()
>>>>                      -> vhost_virtqueue_unmap()
>>>>          -> __virtio_queue_reset()
>>>> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>>>>      -> virtio_queue_enable() [virtio]
>>>>          -> virtio_net_queue_enable() [virtio-net]
>>>>              -> vhost_net_virtqueue_restart() [vhost-net]
>>>>                  -> vhost_dev_virtqueue_restart() [vhost]
>>>>                      -> vhost_virtqueue_start()
>>>>                  -> vhost_net_set_backend()
>>>>      -> set enabled, reset status of vq.
>>>>
>>>> vhost-user:
>>>> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>>>>      -> virtio_queue_reset() [virtio]
>>>>          -> virtio_net_queue_reset() [virtio-net]
>>>>              -> vhost_net_virtqueue_stop() [vhost-net]
>>>>                  -> vhost_dev_virtqueue_stop() [vhost]
>>>>                      -> vhost_user_reset_vring() [vhost-user]
>>>>                          -> send VHOST_USER_RESET_VRING to the device
>>>>                      -> vhost_virtqueue_unmap()
>>>>          -> __virtio_queue_reset()
>>>> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>>>>      -> virtio_queue_enable() [virtio]
>>>>          -> virtio_net_queue_enable() [virtio-net]
>>>>              -> vhost_net_virtqueue_restart() [vhost-net]
>>>>                  -> vhost_dev_virtqueue_restart() [vhost]
>>>>                      -> vhost_virtqueue_start()
>>>>                      -> vhost_user_set_single_vring_enable [vhost-user]
>>>>                          -> send VHOST_USER_SET_VRING_ENABLE to the device
>>>>      -> set enabled, reset status of vq.
>>>>
>>>>
>>>> Test environment:
>>>>      Host: 5.19.0-rc3 (With vq reset support)
>>>>      Qemu: QEMU emulator version 7.0.50
>>>>      Guest: 5.19.0-rc3 (With vq reset support)
>>>>      DPDK: 22.07-rc1 (With vq reset support)
>>>>      Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
>>>>
>>>>      The drvier can resize the virtio queue, then virtio queue reset function should
>>>>      be triggered.
>>>>
>>>>      The default is split mode, modify Qemu virtio-net to add PACKED feature to
>>>>      test packed mode.
>>> legacy mode testing?
>>
>> legacy does not support vq reset.
>>
>> Thanks.
> yes but did it break with all these code changes?

For pci legacy mode, we tested resizing the vq using 'ethtool -G', it 
reported an error code -ENOENT because the transport did not support the 
vq reset feature.

We also tested the connectivity and performance in pci legacy mode using 
iperf 3.9. We placed a client in the host and a server in the guest, or 
swapped the them.

In all the cases, the connectivity did not break down. The performance 
was about the same with all our code changes.

Thanks.

>>>> Guest Kernel Patch:
>>>>      https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
>>>>
>>>> DPDK Patch:
>>>>      https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
>>>>
>>>> Host Kernel Patch:
>>>>      https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
>>>>
>>>> Looking forward to your review and comments. Thanks.
>>>>
>>>> Kangjie Xu (19):
>>>>    virtio: introduce virtio_queue_enable()
>>>>    virtio: core: vq reset feature negotation support
>>>>    virtio-pci: support queue enable
>>>>    vhost: extract the logic of unmapping the vrings and desc
>>>>    vhost: introduce vhost_dev_virtqueue_stop()
>>>>    vhost: introduce vhost_dev_virtqueue_restart()
>>>>    vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
>>>>    vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
>>>>    docs: vhost-user: add VHOST_USER_RESET_VRING message
>>>>    vhost-user: introduce vhost_reset_vring() interface
>>>>    vhost-user: add op to enable or disable a single vring
>>>>    vhost: vhost-user: update vhost_dev_virtqueue_stop()
>>>>    vhost: vhost-user: update vhost_dev_virtqueue_restart()
>>>>    vhost-net: vhost-user: update vhost_net_virtqueue_stop()
>>>>    vhost-net: vhost-user: update vhost_net_virtqueue_restart()
>>>>    virtio-net: introduce flush_or_purge_queued_packets()
>>>>    virtio-net: support queue_enable
>>>>    vhost: vhost-kernel: enable vq reset feature
>>>>    vhost: vhost-user: enable vq reset feature
>>>>
>>>> Xuan Zhuo (5):
>>>>    virtio: sync relevant definitions with linux
>>>>    virtio: introduce __virtio_queue_reset()
>>>>    virtio: introduce virtio_queue_reset()
>>>>    virtio-pci: support queue reset
>>>>    virtio-net: support queue reset
>>>>
>>>>   docs/interop/vhost-user.rst                   | 10 +++
>>>>   hw/core/machine.c                             |  1 +
>>>>   hw/net/vhost_net.c                            | 79 +++++++++++++++++++
>>>>   hw/net/virtio-net.c                           | 58 ++++++++++++--
>>>>   hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
>>>>   hw/virtio/vhost.c                             | 79 +++++++++++++++++--
>>>>   hw/virtio/virtio-pci.c                        | 20 +++++
>>>>   hw/virtio/virtio.c                            | 62 +++++++++++----
>>>>   include/hw/virtio/vhost-backend.h             |  6 ++
>>>>   include/hw/virtio/vhost.h                     |  5 ++
>>>>   include/hw/virtio/virtio-pci.h                |  1 +
>>>>   include/hw/virtio/virtio.h                    |  8 +-
>>>>   include/net/vhost_net.h                       |  4 +
>>>>   .../standard-headers/linux/virtio_config.h    |  5 ++
>>>>   include/standard-headers/linux/virtio_pci.h   |  2 +
>>>>   15 files changed, 371 insertions(+), 36 deletions(-)
>>>>
>>>> --
>>>> 2.32.0

Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Xuan Zhuo 1 year, 7 months ago
On Tue, 16 Aug 2022 02:22:16 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Tue, Aug 16, 2022 at 02:15:57PM +0800, Xuan Zhuo wrote:
> > On Tue, 16 Aug 2022 02:14:10 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Tue, Aug 16, 2022 at 09:06:12AM +0800, Kangjie Xu wrote:
> > > > The virtio queue reset function has already been defined in the virtio spec 1.2.
> > > > The relevant virtio spec information is here:
> > > >
> > > >     https://github.com/oasis-tcs/virtio-spec/issues/124
> > > >     https://github.com/oasis-tcs/virtio-spec/issues/139
> > > >
> > > > This patch set is to support this function in QEMU. It consists of several parts:
> > > > 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> > > > 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> > > > 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> > > > 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> > > > 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
> > > >
> > > > The process of virtqueue reset can be concluded as:
> > > > 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> > > > 2. Then the virtqueue can be optionally restarted(re-enabled).
> > > >
> > > > Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> > > > calling process for different modes below.
> > > > virtio-net:
> > > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > >     -> virtio_queue_reset() [virtio]
> > > >         -> virtio_net_queue_reset() [virtio-net]
> > > >         -> __virtio_queue_reset()
> > > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > >     -> set enabled, reset status of vq.
> > > >
> > > > vhost-kernel:
> > > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > >     -> virtio_queue_reset() [virtio]
> > > >         -> virtio_net_queue_reset() [virtio-net]
> > > >             -> vhost_net_virtqueue_stop() [vhost-net]
> > > >                 -> vhost_net_set_backend() [vhost]
> > > >                 -> vhost_dev_virtqueue_stop()
> > > >                     -> vhost_virtqueue_unmap()
> > > >         -> __virtio_queue_reset()
> > > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > >     -> virtio_queue_enable() [virtio]
> > > >         -> virtio_net_queue_enable() [virtio-net]
> > > >             -> vhost_net_virtqueue_restart() [vhost-net]
> > > >                 -> vhost_dev_virtqueue_restart() [vhost]
> > > >                     -> vhost_virtqueue_start()
> > > >                 -> vhost_net_set_backend()
> > > >     -> set enabled, reset status of vq.
> > > >
> > > > vhost-user:
> > > > 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
> > > >     -> virtio_queue_reset() [virtio]
> > > >         -> virtio_net_queue_reset() [virtio-net]
> > > >             -> vhost_net_virtqueue_stop() [vhost-net]
> > > >                 -> vhost_dev_virtqueue_stop() [vhost]
> > > >                     -> vhost_user_reset_vring() [vhost-user]
> > > >                         -> send VHOST_USER_RESET_VRING to the device
> > > >                     -> vhost_virtqueue_unmap()
> > > >         -> __virtio_queue_reset()
> > > > 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
> > > >     -> virtio_queue_enable() [virtio]
> > > >         -> virtio_net_queue_enable() [virtio-net]
> > > >             -> vhost_net_virtqueue_restart() [vhost-net]
> > > >                 -> vhost_dev_virtqueue_restart() [vhost]
> > > >                     -> vhost_virtqueue_start()
> > > >                     -> vhost_user_set_single_vring_enable [vhost-user]
> > > >                         -> send VHOST_USER_SET_VRING_ENABLE to the device
> > > >     -> set enabled, reset status of vq.
> > > >
> > > >
> > > > Test environment:
> > > >     Host: 5.19.0-rc3 (With vq reset support)
> > > >     Qemu: QEMU emulator version 7.0.50
> > > >     Guest: 5.19.0-rc3 (With vq reset support)
> > > >     DPDK: 22.07-rc1 (With vq reset support)
> > > >     Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
> > > >
> > > >     The drvier can resize the virtio queue, then virtio queue reset function should
> > > >     be triggered.
> > > >
> > > >     The default is split mode, modify Qemu virtio-net to add PACKED feature to
> > > >     test packed mode.
> > >
> > > legacy mode testing?
> >
> >
> > legacy does not support vq reset.
> >
> > Thanks.
>
> yes but did it break with all these code changes?

OK, I see, we'll test this.

Thanks.


>
> > >
> > > > Guest Kernel Patch:
> > > >     https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
> > > >
> > > > DPDK Patch:
> > > >     https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
> > > >
> > > > Host Kernel Patch:
> > > >     https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
> > > >
> > > > Looking forward to your review and comments. Thanks.
> > > >
> > > > Kangjie Xu (19):
> > > >   virtio: introduce virtio_queue_enable()
> > > >   virtio: core: vq reset feature negotation support
> > > >   virtio-pci: support queue enable
> > > >   vhost: extract the logic of unmapping the vrings and desc
> > > >   vhost: introduce vhost_dev_virtqueue_stop()
> > > >   vhost: introduce vhost_dev_virtqueue_restart()
> > > >   vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
> > > >   vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
> > > >   docs: vhost-user: add VHOST_USER_RESET_VRING message
> > > >   vhost-user: introduce vhost_reset_vring() interface
> > > >   vhost-user: add op to enable or disable a single vring
> > > >   vhost: vhost-user: update vhost_dev_virtqueue_stop()
> > > >   vhost: vhost-user: update vhost_dev_virtqueue_restart()
> > > >   vhost-net: vhost-user: update vhost_net_virtqueue_stop()
> > > >   vhost-net: vhost-user: update vhost_net_virtqueue_restart()
> > > >   virtio-net: introduce flush_or_purge_queued_packets()
> > > >   virtio-net: support queue_enable
> > > >   vhost: vhost-kernel: enable vq reset feature
> > > >   vhost: vhost-user: enable vq reset feature
> > > >
> > > > Xuan Zhuo (5):
> > > >   virtio: sync relevant definitions with linux
> > > >   virtio: introduce __virtio_queue_reset()
> > > >   virtio: introduce virtio_queue_reset()
> > > >   virtio-pci: support queue reset
> > > >   virtio-net: support queue reset
> > > >
> > > >  docs/interop/vhost-user.rst                   | 10 +++
> > > >  hw/core/machine.c                             |  1 +
> > > >  hw/net/vhost_net.c                            | 79 +++++++++++++++++++
> > > >  hw/net/virtio-net.c                           | 58 ++++++++++++--
> > > >  hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
> > > >  hw/virtio/vhost.c                             | 79 +++++++++++++++++--
> > > >  hw/virtio/virtio-pci.c                        | 20 +++++
> > > >  hw/virtio/virtio.c                            | 62 +++++++++++----
> > > >  include/hw/virtio/vhost-backend.h             |  6 ++
> > > >  include/hw/virtio/vhost.h                     |  5 ++
> > > >  include/hw/virtio/virtio-pci.h                |  1 +
> > > >  include/hw/virtio/virtio.h                    |  8 +-
> > > >  include/net/vhost_net.h                       |  4 +
> > > >  .../standard-headers/linux/virtio_config.h    |  5 ++
> > > >  include/standard-headers/linux/virtio_pci.h   |  2 +
> > > >  15 files changed, 371 insertions(+), 36 deletions(-)
> > > >
> > > > --
> > > > 2.32.0
> > >
>
Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Jason Wang 1 year, 7 months ago
在 2022/8/16 09:06, Kangjie Xu 写道:
> The virtio queue reset function has already been defined in the virtio spec 1.2.
> The relevant virtio spec information is here:
>
>      https://github.com/oasis-tcs/virtio-spec/issues/124
>      https://github.com/oasis-tcs/virtio-spec/issues/139
>
> This patch set is to support this function in QEMU. It consists of several parts:
> 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
>
> The process of virtqueue reset can be concluded as:
> 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> 2. Then the virtqueue can be optionally restarted(re-enabled).
>
> Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> calling process for different modes below.
> virtio-net:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>      -> virtio_queue_reset() [virtio]
>          -> virtio_net_queue_reset() [virtio-net]
>          -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>      -> set enabled, reset status of vq.
>
> vhost-kernel:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>      -> virtio_queue_reset() [virtio]
>          -> virtio_net_queue_reset() [virtio-net]
>              -> vhost_net_virtqueue_stop() [vhost-net]
>                  -> vhost_net_set_backend() [vhost]
>                  -> vhost_dev_virtqueue_stop()
>                      -> vhost_virtqueue_unmap()
>          -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>      -> virtio_queue_enable() [virtio]
>          -> virtio_net_queue_enable() [virtio-net]
>              -> vhost_net_virtqueue_restart() [vhost-net]
>                  -> vhost_dev_virtqueue_restart() [vhost]
>                      -> vhost_virtqueue_start()
>                  -> vhost_net_set_backend()
>      -> set enabled, reset status of vq.
>
> vhost-user:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>      -> virtio_queue_reset() [virtio]
>          -> virtio_net_queue_reset() [virtio-net]
>              -> vhost_net_virtqueue_stop() [vhost-net]
>                  -> vhost_dev_virtqueue_stop() [vhost]
>                      -> vhost_user_reset_vring() [vhost-user]
>                          -> send VHOST_USER_RESET_VRING to the device
>                      -> vhost_virtqueue_unmap()
>          -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>      -> virtio_queue_enable() [virtio]
>          -> virtio_net_queue_enable() [virtio-net]
>              -> vhost_net_virtqueue_restart() [vhost-net]
>                  -> vhost_dev_virtqueue_restart() [vhost]
>                      -> vhost_virtqueue_start()
>                      -> vhost_user_set_single_vring_enable [vhost-user]
>                          -> send VHOST_USER_SET_VRING_ENABLE to the device
>      -> set enabled, reset status of vq.
>
>
> Test environment:
>      Host: 5.19.0-rc3 (With vq reset support)
>      Qemu: QEMU emulator version 7.0.50
>      Guest: 5.19.0-rc3 (With vq reset support)
>      DPDK: 22.07-rc1 (With vq reset support)
>      Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
>
>      The drvier can resize the virtio queue, then virtio queue reset function should
>      be triggered.
>
>      The default is split mode, modify Qemu virtio-net to add PACKED feature to
>      test packed mode.
>
> Guest Kernel Patch:
>      https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
>
> DPDK Patch:
>      https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
>
> Host Kernel Patch:
>      https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
>
> Looking forward to your review and comments. Thanks.


I suggest to split vhost-user part into another series since:

1) vhost-net kernel is ready for this future

2) vhost-user probably requires some extensions of the current protocol

Then it can speed up the merging and reduce the rebasing of a huge series.

Thanks


>
> Kangjie Xu (19):
>    virtio: introduce virtio_queue_enable()
>    virtio: core: vq reset feature negotation support
>    virtio-pci: support queue enable
>    vhost: extract the logic of unmapping the vrings and desc
>    vhost: introduce vhost_dev_virtqueue_stop()
>    vhost: introduce vhost_dev_virtqueue_restart()
>    vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
>    vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
>    docs: vhost-user: add VHOST_USER_RESET_VRING message
>    vhost-user: introduce vhost_reset_vring() interface
>    vhost-user: add op to enable or disable a single vring
>    vhost: vhost-user: update vhost_dev_virtqueue_stop()
>    vhost: vhost-user: update vhost_dev_virtqueue_restart()
>    vhost-net: vhost-user: update vhost_net_virtqueue_stop()
>    vhost-net: vhost-user: update vhost_net_virtqueue_restart()
>    virtio-net: introduce flush_or_purge_queued_packets()
>    virtio-net: support queue_enable
>    vhost: vhost-kernel: enable vq reset feature
>    vhost: vhost-user: enable vq reset feature
>
> Xuan Zhuo (5):
>    virtio: sync relevant definitions with linux
>    virtio: introduce __virtio_queue_reset()
>    virtio: introduce virtio_queue_reset()
>    virtio-pci: support queue reset
>    virtio-net: support queue reset
>
>   docs/interop/vhost-user.rst                   | 10 +++
>   hw/core/machine.c                             |  1 +
>   hw/net/vhost_net.c                            | 79 +++++++++++++++++++
>   hw/net/virtio-net.c                           | 58 ++++++++++++--
>   hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
>   hw/virtio/vhost.c                             | 79 +++++++++++++++++--
>   hw/virtio/virtio-pci.c                        | 20 +++++
>   hw/virtio/virtio.c                            | 62 +++++++++++----
>   include/hw/virtio/vhost-backend.h             |  6 ++
>   include/hw/virtio/vhost.h                     |  5 ++
>   include/hw/virtio/virtio-pci.h                |  1 +
>   include/hw/virtio/virtio.h                    |  8 +-
>   include/net/vhost_net.h                       |  4 +
>   .../standard-headers/linux/virtio_config.h    |  5 ++
>   include/standard-headers/linux/virtio_pci.h   |  2 +
>   15 files changed, 371 insertions(+), 36 deletions(-)
>


Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Jason Wang 1 year, 7 months ago
在 2022/8/16 09:06, Kangjie Xu 写道:
> Host Kernel Patch:
>      https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3


Btw, this patch could be posted right now. Then it would be more easier 
for people to try you series.

Thanks


Re: [PATCH 00/24] Support VIRTIO_F_RING_RESET for virtio-net, vhost-user, vhost-kernel in virtio pci-modern
Posted by Kangjie Xu 1 year, 7 months ago
Forgot to append changelog for v2 in the cover letter, so I add it in 
this email.


changelog:

1. Add support for vhost-net scenario

2. Add a new vhost-user message VHOST_USER_RESET_VRING

3. Add migration compatibility for virtqueue reset


Looking forward to your review and comments to this patch set, thanks. :)


Best regards,

Kangjie

在 2022/8/16 09:06, Kangjie Xu 写道:
> The virtio queue reset function has already been defined in the virtio spec 1.2.
> The relevant virtio spec information is here:
>
>      https://github.com/oasis-tcs/virtio-spec/issues/124
>      https://github.com/oasis-tcs/virtio-spec/issues/139
>
> This patch set is to support this function in QEMU. It consists of several parts:
> 1. Patches 1-7 are the basic interfaces for vq reset in virtio and virtio-pci.
> 2. Patches 8-12 support vq stop and vq restart for vhost-kernel.
> 3. Patches 13-19 support vq stop and vq restart for vhost-user.
> 4. Patches 20-22 support vq reset and re-enable for virtio-net.
> 5. Patches 23-24 enable the vq reset feature for vhost-kernel and vhost-user.
>
> The process of virtqueue reset can be concluded as:
> 1. The virtqueue is disabled when VIRTIO_PCI_COMMON_Q_RESET is written.
> 2. Then the virtqueue can be optionally restarted(re-enabled).
>
> Since this patch set involves multiple modules and seems a bit messy, we briefly describe the
> calling process for different modes below.
> virtio-net:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>      -> virtio_queue_reset() [virtio]
>          -> virtio_net_queue_reset() [virtio-net]
>          -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>      -> set enabled, reset status of vq.
>
> vhost-kernel:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>      -> virtio_queue_reset() [virtio]
>          -> virtio_net_queue_reset() [virtio-net]
>              -> vhost_net_virtqueue_stop() [vhost-net]
>                  -> vhost_net_set_backend() [vhost]
>                  -> vhost_dev_virtqueue_stop()
>                      -> vhost_virtqueue_unmap()
>          -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>      -> virtio_queue_enable() [virtio]
>          -> virtio_net_queue_enable() [virtio-net]
>              -> vhost_net_virtqueue_restart() [vhost-net]
>                  -> vhost_dev_virtqueue_restart() [vhost]
>                      -> vhost_virtqueue_start()
>                  -> vhost_net_set_backend()
>      -> set enabled, reset status of vq.
>
> vhost-user:
> 1. VIRTIO_PCI_COMMON_Q_RESET is written [virtio-pci]
>      -> virtio_queue_reset() [virtio]
>          -> virtio_net_queue_reset() [virtio-net]
>              -> vhost_net_virtqueue_stop() [vhost-net]
>                  -> vhost_dev_virtqueue_stop() [vhost]
>                      -> vhost_user_reset_vring() [vhost-user]
>                          -> send VHOST_USER_RESET_VRING to the device
>                      -> vhost_virtqueue_unmap()
>          -> __virtio_queue_reset()
> 2. VIRTIO_PCI_COMMON_Q_ENABLE is written [virtio-pci]
>      -> virtio_queue_enable() [virtio]
>          -> virtio_net_queue_enable() [virtio-net]
>              -> vhost_net_virtqueue_restart() [vhost-net]
>                  -> vhost_dev_virtqueue_restart() [vhost]
>                      -> vhost_virtqueue_start()
>                      -> vhost_user_set_single_vring_enable [vhost-user]
>                          -> send VHOST_USER_SET_VRING_ENABLE to the device
>      -> set enabled, reset status of vq.
>
>
> Test environment:
>      Host: 5.19.0-rc3 (With vq reset support)
>      Qemu: QEMU emulator version 7.0.50
>      Guest: 5.19.0-rc3 (With vq reset support)
>      DPDK: 22.07-rc1 (With vq reset support)
>      Test Cmd: ethtool -g eth1; ethtool -G eth1 rx $1 tx $2; ethtool -g eth1;
>
>      The drvier can resize the virtio queue, then virtio queue reset function should
>      be triggered.
>
>      The default is split mode, modify Qemu virtio-net to add PACKED feature to
>      test packed mode.
>
> Guest Kernel Patch:
>      https://lore.kernel.org/bpf/20220801063902.129329-1-xuanzhuo@linux.alibaba.com/
>
> DPDK Patch:
>      https://github.com/middaywords/dpdk/compare/72206323a5dd3182b13f61b25a64abdddfee595c...eabadfac7953da66bc10ffb8284b490d09bb7ec7
>
> Host Kernel Patch:
>      https://github.com/middaywords/linux/commit/19a91e0d7167b2031e46078c6215c213b89cb2c3
>
> Looking forward to your review and comments. Thanks.
>
> Kangjie Xu (19):
>    virtio: introduce virtio_queue_enable()
>    virtio: core: vq reset feature negotation support
>    virtio-pci: support queue enable
>    vhost: extract the logic of unmapping the vrings and desc
>    vhost: introduce vhost_dev_virtqueue_stop()
>    vhost: introduce vhost_dev_virtqueue_restart()
>    vhost-net: vhost-kernel: introduce vhost_net_virtqueue_stop()
>    vhost-net: vhost-kernel: introduce vhost_net_virtqueue_restart()
>    docs: vhost-user: add VHOST_USER_RESET_VRING message
>    vhost-user: introduce vhost_reset_vring() interface
>    vhost-user: add op to enable or disable a single vring
>    vhost: vhost-user: update vhost_dev_virtqueue_stop()
>    vhost: vhost-user: update vhost_dev_virtqueue_restart()
>    vhost-net: vhost-user: update vhost_net_virtqueue_stop()
>    vhost-net: vhost-user: update vhost_net_virtqueue_restart()
>    virtio-net: introduce flush_or_purge_queued_packets()
>    virtio-net: support queue_enable
>    vhost: vhost-kernel: enable vq reset feature
>    vhost: vhost-user: enable vq reset feature
>
> Xuan Zhuo (5):
>    virtio: sync relevant definitions with linux
>    virtio: introduce __virtio_queue_reset()
>    virtio: introduce virtio_queue_reset()
>    virtio-pci: support queue reset
>    virtio-net: support queue reset
>
>   docs/interop/vhost-user.rst                   | 10 +++
>   hw/core/machine.c                             |  1 +
>   hw/net/vhost_net.c                            | 79 +++++++++++++++++++
>   hw/net/virtio-net.c                           | 58 ++++++++++++--
>   hw/virtio/vhost-user.c                        | 67 ++++++++++++++--
>   hw/virtio/vhost.c                             | 79 +++++++++++++++++--
>   hw/virtio/virtio-pci.c                        | 20 +++++
>   hw/virtio/virtio.c                            | 62 +++++++++++----
>   include/hw/virtio/vhost-backend.h             |  6 ++
>   include/hw/virtio/vhost.h                     |  5 ++
>   include/hw/virtio/virtio-pci.h                |  1 +
>   include/hw/virtio/virtio.h                    |  8 +-
>   include/net/vhost_net.h                       |  4 +
>   .../standard-headers/linux/virtio_config.h    |  5 ++
>   include/standard-headers/linux/virtio_pci.h   |  2 +
>   15 files changed, 371 insertions(+), 36 deletions(-)
>