[PATCH 20/40] vdpa: avoid mapping flush across reset

Si-Wei Liu posted 40 patches 11 months, 3 weeks ago
[PATCH 20/40] vdpa: avoid mapping flush across reset
Posted by Si-Wei Liu 11 months, 3 weeks ago
Leverage the IOTLB_PERSIST and DESC_ASID features to achieve
a slightly light weight reset path, without resorting to
suspend and resume. Not as best but it offers significant
time saving too, which should somehow play its role in live
migration down time reduction by large.

It benefits two cases:
  - normal virtio reset in the VM, e.g. guest reboot, don't
    have to tear down all iotlb mapping and set up again.
  - SVQ switching, in which data vq's descriptor table and
    vrings are moved to a different ASID than where its
    buffers reside. Along with the use of persistent iotlb,
    it would save substantial time from pinning and mapping
    unneccessarily when moving descriptors on to or out of
    shadow mode.

Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>
---
 hw/virtio/vhost-vdpa.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
index 31e0a55..47c764b 100644
--- a/hw/virtio/vhost-vdpa.c
+++ b/hw/virtio/vhost-vdpa.c
@@ -633,6 +633,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp)
                                      0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH |
                                      0x1ULL << VHOST_BACKEND_F_IOTLB_ASID |
                                      0x1ULL << VHOST_BACKEND_F_DESC_ASID |
+                                     0x1ULL << VHOST_BACKEND_F_IOTLB_PERSIST |
                                      0x1ULL << VHOST_BACKEND_F_SUSPEND;
     int ret;
 
@@ -1493,8 +1494,6 @@ static void vhost_vdpa_maybe_flush_map(struct vhost_dev *dev)
 
 static void vhost_vdpa_reset_status(struct vhost_dev *dev)
 {
-    struct vhost_vdpa *v = dev->opaque;
-
     if (!vhost_vdpa_last_dev(dev)) {
         return;
     }
@@ -1502,9 +1501,7 @@ static void vhost_vdpa_reset_status(struct vhost_dev *dev)
     vhost_vdpa_reset_device(dev);
     vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
                                VIRTIO_CONFIG_S_DRIVER);
-    memory_listener_unregister(&v->shared->listener);
-    v->shared->listener_registered = false;
-
+    vhost_vdpa_maybe_flush_map(dev);
 }
 
 static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base,
-- 
1.8.3.1
Re: [PATCH 20/40] vdpa: avoid mapping flush across reset
Posted by Jason Wang 10 months, 2 weeks ago
On Fri, Dec 8, 2023 at 2:52 AM Si-Wei Liu <si-wei.liu@oracle.com> wrote:
>
> Leverage the IOTLB_PERSIST and DESC_ASID features to achieve
> a slightly light weight reset path, without resorting to
> suspend and resume. Not as best but it offers significant
> time saving too, which should somehow play its role in live
> migration down time reduction by large.
>
> It benefits two cases:
>   - normal virtio reset in the VM, e.g. guest reboot, don't
>     have to tear down all iotlb mapping and set up again.
>   - SVQ switching, in which data vq's descriptor table and
>     vrings are moved to a different ASID than where its
>     buffers reside. Along with the use of persistent iotlb,
>     it would save substantial time from pinning and mapping
>     unneccessarily when moving descriptors on to or out of
>     shadow mode.
>
> Signed-off-by: Si-Wei Liu <si-wei.liu@oracle.com>

Looks good to me.

Thanks

> ---
>  hw/virtio/vhost-vdpa.c | 7 ++-----
>  1 file changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c
> index 31e0a55..47c764b 100644
> --- a/hw/virtio/vhost-vdpa.c
> +++ b/hw/virtio/vhost-vdpa.c
> @@ -633,6 +633,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **errp)
>                                       0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH |
>                                       0x1ULL << VHOST_BACKEND_F_IOTLB_ASID |
>                                       0x1ULL << VHOST_BACKEND_F_DESC_ASID |
> +                                     0x1ULL << VHOST_BACKEND_F_IOTLB_PERSIST |
>                                       0x1ULL << VHOST_BACKEND_F_SUSPEND;
>      int ret;
>
> @@ -1493,8 +1494,6 @@ static void vhost_vdpa_maybe_flush_map(struct vhost_dev *dev)
>
>  static void vhost_vdpa_reset_status(struct vhost_dev *dev)
>  {
> -    struct vhost_vdpa *v = dev->opaque;
> -
>      if (!vhost_vdpa_last_dev(dev)) {
>          return;
>      }
> @@ -1502,9 +1501,7 @@ static void vhost_vdpa_reset_status(struct vhost_dev *dev)
>      vhost_vdpa_reset_device(dev);
>      vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE |
>                                 VIRTIO_CONFIG_S_DRIVER);
> -    memory_listener_unregister(&v->shared->listener);
> -    v->shared->listener_registered = false;
> -
> +    vhost_vdpa_maybe_flush_map(dev);
>  }
>
>  static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base,
> --
> 1.8.3.1
>