[v2] vdpa_sim: set last_used_idx as last_avail_idx in vdpasim_queue_ready

[PATCH v2] vdpa_sim: set last_used_idx as last_avail_idx in vdpasim_queue_ready

Posted by Eugenio Pérez 2 years, 7 months ago

Starting from an used_idx different than 0 is needed in use cases like
virtual machine migration.  Not doing so and letting the caller set an
avail idx different than 0 causes destination device to try to use old
buffers that source driver already recover and are not available
anymore.

Since vdpa_sim does not support receive inflight descriptors as a
destination of a migration, let's set both avail_idx and used_idx the
same at vq start.  This is how vhost-user works in a
VHOST_SET_VRING_BASE call.

Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
it would be reset at vdpasim_queue_ready.  The last_avail_idx case is
fixed with commit a09f493c ("vdpa_sim: not reset state in
vdpasim_queue_ready").  Since the only option is to make it equal to
last_avail_idx, adding the only change needed here.

This was discovered and tested live migrating the vdpa_sim_net device.

Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
---
Cherry-picked from patch 2/2 of the series [1]. Differences are:
* Set the value of used_idx at vdpasim_queue_ready instead of fetching
  from the guest vring like vhost-kernel.

v2: Actually update last_used_idx only at vdpasim_queue_ready.

Note that commit id present in the patch text is not in master but in
git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.

[1] https://lkml.org/lkml/2023/1/18/1041
---
 drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 6a0a65814626..79ac585e40b9 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
 			  (uintptr_t)vq->device_addr);
 
 	vq->vring.last_avail_idx = last_avail_idx;
+	vq->vring.last_used_idx = last_avail_idx;
 	vq->vring.notify = vdpasim_vq_notify;
 }
 
-- 
2.31.1

Re: [PATCH v2] vdpa_sim: set last_used_idx as last_avail_idx in vdpasim_queue_ready

Posted by Michael S. Tsirkin 2 years, 6 months ago

On Fri, Feb 03, 2023 at 03:25:01PM +0100, Eugenio Pérez wrote:
> Starting from an used_idx different than 0 is needed in use cases like
> virtual machine migration.  Not doing so and letting the caller set an
> avail idx different than 0 causes destination device to try to use old
> buffers that source driver already recover and are not available
> anymore.
> 
> Since vdpa_sim does not support receive inflight descriptors as a
> destination of a migration, let's set both avail_idx and used_idx the
> same at vq start.  This is how vhost-user works in a
> VHOST_SET_VRING_BASE call.
> 
> Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
> it would be reset at vdpasim_queue_ready.  The last_avail_idx case is
> fixed with commit a09f493c ("vdpa_sim: not reset state in
> vdpasim_queue_ready").  Since the only option is to make it equal to
> last_avail_idx, adding the only change needed here.
> 
> This was discovered and tested live migrating the vdpa_sim_net device.
> 
> Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
> Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> ---
> Cherry-picked from patch 2/2 of the series [1]. Differences are:
> * Set the value of used_idx at vdpasim_queue_ready instead of fetching
>   from the guest vring like vhost-kernel.
> 
> v2: Actually update last_used_idx only at vdpasim_queue_ready.
> 
> Note that commit id present in the patch text is not in master but in
> git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
> 
> [1] https://lkml.org/lkml/2023/1/18/1041


Can you post with a fixed hash please?

> ---
>  drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> index 6a0a65814626..79ac585e40b9 100644
> --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
> +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> @@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
>  			  (uintptr_t)vq->device_addr);
>  
>  	vq->vring.last_avail_idx = last_avail_idx;
> +	vq->vring.last_used_idx = last_avail_idx;
>  	vq->vring.notify = vdpasim_vq_notify;
>  }
>  
> -- 
> 2.31.1

Re: [PATCH v2] vdpa_sim: set last_used_idx as last_avail_idx in vdpasim_queue_ready

Posted by Eugenio Perez Martin 2 years, 6 months ago

On Tue, Feb 28, 2023 at 1:13 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Feb 03, 2023 at 03:25:01PM +0100, Eugenio Pérez wrote:
> > Starting from an used_idx different than 0 is needed in use cases like
> > virtual machine migration.  Not doing so and letting the caller set an
> > avail idx different than 0 causes destination device to try to use old
> > buffers that source driver already recover and are not available
> > anymore.
> >
> > Since vdpa_sim does not support receive inflight descriptors as a
> > destination of a migration, let's set both avail_idx and used_idx the
> > same at vq start.  This is how vhost-user works in a
> > VHOST_SET_VRING_BASE call.
> >
> > Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
> > it would be reset at vdpasim_queue_ready.  The last_avail_idx case is
> > fixed with commit a09f493c ("vdpa_sim: not reset state in
> > vdpasim_queue_ready").  Since the only option is to make it equal to
> > last_avail_idx, adding the only change needed here.
> >
> > This was discovered and tested live migrating the vdpa_sim_net device.
> >
> > Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
> > Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> > ---
> > Cherry-picked from patch 2/2 of the series [1]. Differences are:
> > * Set the value of used_idx at vdpasim_queue_ready instead of fetching
> >   from the guest vring like vhost-kernel.
> >
> > v2: Actually update last_used_idx only at vdpasim_queue_ready.
> >
> > Note that commit id present in the patch text is not in master but in
> > git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
> >
> > [1] https://lkml.org/lkml/2023/1/18/1041
>
>
> Can you post with a fixed hash please?
>

Sent v3, please let me know if I need to do further modifications.

Thanks!

> > ---
> >  drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
> >  1 file changed, 1 insertion(+)
> >
> > diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> > index 6a0a65814626..79ac585e40b9 100644
> > --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
> > +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> > @@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
> >                         (uintptr_t)vq->device_addr);
> >
> >       vq->vring.last_avail_idx = last_avail_idx;
> > +     vq->vring.last_used_idx = last_avail_idx;
> >       vq->vring.notify = vdpasim_vq_notify;
> >  }
> >
> > --
> > 2.31.1
>

Re: [PATCH v2] vdpa_sim: set last_used_idx as last_avail_idx in vdpasim_queue_ready

Posted by Stefano Garzarella 2 years, 6 months ago

On Fri, Feb 03, 2023 at 03:25:01PM +0100, Eugenio Pérez wrote:
>Starting from an used_idx different than 0 is needed in use cases like
>virtual machine migration.  Not doing so and letting the caller set an
>avail idx different than 0 causes destination device to try to use old
>buffers that source driver already recover and are not available
>anymore.
>
>Since vdpa_sim does not support receive inflight descriptors as a
>destination of a migration, let's set both avail_idx and used_idx the
>same at vq start.  This is how vhost-user works in a
>VHOST_SET_VRING_BASE call.
>
>Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
>it would be reset at vdpasim_queue_ready.  The last_avail_idx case is
>fixed with commit a09f493c ("vdpa_sim: not reset state in
>vdpasim_queue_ready").  Since the only option is to make it equal to
>last_avail_idx, adding the only change needed here.
>
>This was discovered and tested live migrating the vdpa_sim_net device.
>
>Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
>Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
>---
>Cherry-picked from patch 2/2 of the series [1]. Differences are:
>* Set the value of used_idx at vdpasim_queue_ready instead of fetching
>  from the guest vring like vhost-kernel.
>
>v2: Actually update last_used_idx only at vdpasim_queue_ready.
>
>Note that commit id present in the patch text is not in master but in
>git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
>
>[1] https://lkml.org/lkml/2023/1/18/1041
>---
> drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
> 1 file changed, 1 insertion(+)
>
>diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
>index 6a0a65814626..79ac585e40b9 100644
>--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
>+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
>@@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
> 			  (uintptr_t)vq->device_addr);
>
> 	vq->vring.last_avail_idx = last_avail_idx;
>+	vq->vring.last_used_idx = last_avail_idx;
> 	vq->vring.notify = vdpasim_vq_notify;
> }
>
>-- 2.31.1
>

If you need to resend, I'd add a comment in the code following the 
commit description.

Anyway, the patch LGTM:

Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>

Thanks,
Stefano

Re: [PATCH v2] vdpa_sim: set last_used_idx as last_avail_idx in vdpasim_queue_ready

Posted by Eugenio Perez Martin 2 years, 6 months ago

On Tue, Feb 28, 2023 at 11:56 AM Stefano Garzarella <sgarzare@redhat.com> wrote:
>
> On Fri, Feb 03, 2023 at 03:25:01PM +0100, Eugenio Pérez wrote:
> >Starting from an used_idx different than 0 is needed in use cases like
> >virtual machine migration.  Not doing so and letting the caller set an
> >avail idx different than 0 causes destination device to try to use old
> >buffers that source driver already recover and are not available
> >anymore.
> >
> >Since vdpa_sim does not support receive inflight descriptors as a
> >destination of a migration, let's set both avail_idx and used_idx the
> >same at vq start.  This is how vhost-user works in a
> >VHOST_SET_VRING_BASE call.
> >
> >Although the simple fix is to set last_used_idx at vdpasim_set_vq_state,
> >it would be reset at vdpasim_queue_ready.  The last_avail_idx case is
> >fixed with commit a09f493c ("vdpa_sim: not reset state in
> >vdpasim_queue_ready").  Since the only option is to make it equal to
> >last_avail_idx, adding the only change needed here.
> >
> >This was discovered and tested live migrating the vdpa_sim_net device.
> >
> >Fixes: 2c53d0f64c06 ("vdpasim: vDPA device simulator")
> >Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
> >---
> >Cherry-picked from patch 2/2 of the series [1]. Differences are:
> >* Set the value of used_idx at vdpasim_queue_ready instead of fetching
> >  from the guest vring like vhost-kernel.
> >
> >v2: Actually update last_used_idx only at vdpasim_queue_ready.
> >
> >Note that commit id present in the patch text is not in master but in
> >git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git.
> >
> >[1] https://lkml.org/lkml/2023/1/18/1041
> >---
> > drivers/vdpa/vdpa_sim/vdpa_sim.c | 1 +
> > 1 file changed, 1 insertion(+)
> >
> >diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> >index 6a0a65814626..79ac585e40b9 100644
> >--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
> >+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> >@@ -68,6 +68,7 @@ static void vdpasim_queue_ready(struct vdpasim *vdpasim, unsigned int idx)
> >                         (uintptr_t)vq->device_addr);
> >
> >       vq->vring.last_avail_idx = last_avail_idx;
> >+      vq->vring.last_used_idx = last_avail_idx;
> >       vq->vring.notify = vdpasim_vq_notify;
> > }
> >
> >-- 2.31.1
> >
>
> If you need to resend, I'd add a comment in the code following the
> commit description.
>
> Anyway, the patch LGTM:
>
> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
>

Added in v3, thanks for the review!

> Thanks,
> Stefano
>