[RFC v3 0/6] virtio,vhost: Add VIRTIO_F_IN_ORDER support

Jonah Palmer posted 6 patches 3 weeks, 1 day ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20240408153408.3527586-1-jonah.palmer@oracle.com
Maintainers: Raphael Norwitz <raphael@enfabrica.net>, "Michael S. Tsirkin" <mst@redhat.com>, Kevin Wolf <kwolf@redhat.com>, Hanna Reitz <hreitz@redhat.com>, Jason Wang <jasowang@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, Fam Zheng <fam@euphon.net>, Stefan Hajnoczi <stefanha@redhat.com>
hw/block/vhost-user-blk.c    |   1 +
hw/net/vhost_net.c           |   2 +
hw/scsi/vhost-scsi.c         |   1 +
hw/scsi/vhost-user-scsi.c    |   1 +
hw/virtio/vhost-user-fs.c    |   1 +
hw/virtio/vhost-user-vsock.c |   1 +
hw/virtio/virtio.c           | 118 ++++++++++++++++++++++++++++++++++-
include/hw/virtio/virtio.h   |   5 +-
net/vhost-vdpa.c             |   1 +
9 files changed, 127 insertions(+), 4 deletions(-)
[RFC v3 0/6] virtio,vhost: Add VIRTIO_F_IN_ORDER support
Posted by Jonah Palmer 3 weeks, 1 day ago
The goal of these patches is to add support to a variety of virtio and
vhost devices for the VIRTIO_F_IN_ORDER transport feature. This feature
indicates that all buffers are used by the device in the same order in
which they were made available by the driver.

These patches attempt to implement a generalized, non-device-specific
solution to support this feature.

The core feature behind this solution is a buffer mechanism in the form
of a VirtQueue's used_elems VirtQueueElement array. This allows devices
who always use buffers in-order by default to have a minimal overhead
impact. Devices that may not always use buffers in-order likely will
experience a performance hit. How large that performance hit is will
depend on how frequent elements are completed out-of-order.

A VirtQueue whose device who uses this feature will use its used_elems
VirtQueueElement array to hold used VirtQueueElements. The index that
used elements are placed in used_elems is the same index on the
used/descriptor ring that would satisfy the in-order requirement. In
other words, used elements are placed in their in-order locations on
used_elems and are only written to the used/descriptor ring once the
elements on used_elems are able to continue their expected order.

To differentiate between a "used" and "unused" element on the used_elems
array (a "used" element being an element that has returned from
processing and an "unused" element being an element that has not yet
been processed), we added a boolean 'filled' member to the
VirtQueueElement struct. This flag is set to true when the element comes
back from processing (virtqueue_ordered_fill) and then set back to false
once it's been written to the used/descriptor ring
(virtqueue_ordered_flush).

---
v3: Add elements to used_elems during virtqueue_split/packed_pop
    Replace current_seq_idx usage with vq->last_avail_idx
    Remove used_seq_idx, leverage used_idx and last_avail_idx for
    searching used_elems
    Remove seq_idx in VirtQueueElement
    Add boolean to VirtQueueElement to signal element status
    Add virtqueue_ordered_fill/flush functions for ordering

v2: Use a VirtQueue's used_elems array as a buffer mechanism

v1: Implement custom GLib GHashTable as a buffer mechanism

Jonah Palmer (6):
  virtio: Add bool to VirtQueueElement
  virtio: virtqueue_pop - VIRTIO_F_IN_ORDER support
  virtio: virtqueue_ordered_fill - VIRTIO_F_IN_ORDER support
  virtio: virtqueue_ordered_flush - VIRTIO_F_IN_ORDER support
  vhost,vhost-user: Add VIRTIO_F_IN_ORDER to vhost feature bits
  virtio: Add VIRTIO_F_IN_ORDER property definition

 hw/block/vhost-user-blk.c    |   1 +
 hw/net/vhost_net.c           |   2 +
 hw/scsi/vhost-scsi.c         |   1 +
 hw/scsi/vhost-user-scsi.c    |   1 +
 hw/virtio/vhost-user-fs.c    |   1 +
 hw/virtio/vhost-user-vsock.c |   1 +
 hw/virtio/virtio.c           | 118 ++++++++++++++++++++++++++++++++++-
 include/hw/virtio/virtio.h   |   5 +-
 net/vhost-vdpa.c             |   1 +
 9 files changed, 127 insertions(+), 4 deletions(-)

-- 
2.39.3
Re: [RFC v3 0/6] virtio,vhost: Add VIRTIO_F_IN_ORDER support
Posted by Lei Yang 2 weeks, 1 day ago
QE tested this series with packed=on/off, in_order=true and vhost=off
under regression tests, everything are works fine.

Tested-by: Lei Yang <leiyang@redhat.com>

On Mon, Apr 8, 2024 at 11:34 PM Jonah Palmer <jonah.palmer@oracle.com> wrote:
>
> The goal of these patches is to add support to a variety of virtio and
> vhost devices for the VIRTIO_F_IN_ORDER transport feature. This feature
> indicates that all buffers are used by the device in the same order in
> which they were made available by the driver.
>
> These patches attempt to implement a generalized, non-device-specific
> solution to support this feature.
>
> The core feature behind this solution is a buffer mechanism in the form
> of a VirtQueue's used_elems VirtQueueElement array. This allows devices
> who always use buffers in-order by default to have a minimal overhead
> impact. Devices that may not always use buffers in-order likely will
> experience a performance hit. How large that performance hit is will
> depend on how frequent elements are completed out-of-order.
>
> A VirtQueue whose device who uses this feature will use its used_elems
> VirtQueueElement array to hold used VirtQueueElements. The index that
> used elements are placed in used_elems is the same index on the
> used/descriptor ring that would satisfy the in-order requirement. In
> other words, used elements are placed in their in-order locations on
> used_elems and are only written to the used/descriptor ring once the
> elements on used_elems are able to continue their expected order.
>
> To differentiate between a "used" and "unused" element on the used_elems
> array (a "used" element being an element that has returned from
> processing and an "unused" element being an element that has not yet
> been processed), we added a boolean 'filled' member to the
> VirtQueueElement struct. This flag is set to true when the element comes
> back from processing (virtqueue_ordered_fill) and then set back to false
> once it's been written to the used/descriptor ring
> (virtqueue_ordered_flush).
>
> ---
> v3: Add elements to used_elems during virtqueue_split/packed_pop
>     Replace current_seq_idx usage with vq->last_avail_idx
>     Remove used_seq_idx, leverage used_idx and last_avail_idx for
>     searching used_elems
>     Remove seq_idx in VirtQueueElement
>     Add boolean to VirtQueueElement to signal element status
>     Add virtqueue_ordered_fill/flush functions for ordering
>
> v2: Use a VirtQueue's used_elems array as a buffer mechanism
>
> v1: Implement custom GLib GHashTable as a buffer mechanism
>
> Jonah Palmer (6):
>   virtio: Add bool to VirtQueueElement
>   virtio: virtqueue_pop - VIRTIO_F_IN_ORDER support
>   virtio: virtqueue_ordered_fill - VIRTIO_F_IN_ORDER support
>   virtio: virtqueue_ordered_flush - VIRTIO_F_IN_ORDER support
>   vhost,vhost-user: Add VIRTIO_F_IN_ORDER to vhost feature bits
>   virtio: Add VIRTIO_F_IN_ORDER property definition
>
>  hw/block/vhost-user-blk.c    |   1 +
>  hw/net/vhost_net.c           |   2 +
>  hw/scsi/vhost-scsi.c         |   1 +
>  hw/scsi/vhost-user-scsi.c    |   1 +
>  hw/virtio/vhost-user-fs.c    |   1 +
>  hw/virtio/vhost-user-vsock.c |   1 +
>  hw/virtio/virtio.c           | 118 ++++++++++++++++++++++++++++++++++-
>  include/hw/virtio/virtio.h   |   5 +-
>  net/vhost-vdpa.c             |   1 +
>  9 files changed, 127 insertions(+), 4 deletions(-)
>
> --
> 2.39.3
>