ublk: restore auto buf unregister refcount optimization

[PATCH] ublk: restore auto buf unregister refcount optimization

Posted by Caleb Sander Mateos 1 week, 2 days ago

Commit 1ceeedb59749 ("ublk: optimize UBLK_IO_UNREGISTER_IO_BUF on daemon
task") optimized ublk request buffer unregistration to use a non-atomic
reference count decrement when performed on the ublk_io's daemon task.
The optimization applied to auto buffer unregistration, which happens as
part of handling UBLK_IO_COMMIT_AND_FETCH_REQ on the daemon task.
However, commit b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
reordered the ublk_sub_req_ref() for the completed request before the
io_buffer_unregister_bvec() call. As a result, task_registered_buffers
is already 0 when io_buffer_unregister_bvec() calls ublk_io_release()
and the non-atomic refcount optimization doesn't apply.
Move the io_buffer_unregister_bvec() call back to before
ublk_need_complete_req() to restore the reference counting optimization.

Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
Fixes: b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
---
 drivers/block/ublk_drv.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 7981decd1cee..f864a0f2f572 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -3243,15 +3243,15 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
 		if (ret)
 			goto out;
 		io->res = result;
 		req = ublk_fill_io_cmd(io, cmd);
 		ret = ublk_config_io_buf(ub, io, cmd, addr, &buf_idx);
+		if (buf_idx != UBLK_INVALID_BUF_IDX)
+			io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
 		compl = ublk_need_complete_req(ub, io);
 
 		/* can't touch 'ublk_io' any more */
-		if (buf_idx != UBLK_INVALID_BUF_IDX)
-			io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
 		if (req_op(req) == REQ_OP_ZONE_APPEND)
 			req->__sector = addr;
 		if (compl)
 			__ublk_complete_rq(req, io, ublk_dev_need_map_io(ub), NULL);
 
-- 
2.45.2

Re: [PATCH] ublk: restore auto buf unregister refcount optimization

Posted by Jens Axboe 1 week ago

On Wed, 28 Jan 2026 13:56:34 -0700, Caleb Sander Mateos wrote:
> Commit 1ceeedb59749 ("ublk: optimize UBLK_IO_UNREGISTER_IO_BUF on daemon
> task") optimized ublk request buffer unregistration to use a non-atomic
> reference count decrement when performed on the ublk_io's daemon task.
> The optimization applied to auto buffer unregistration, which happens as
> part of handling UBLK_IO_COMMIT_AND_FETCH_REQ on the daemon task.
> However, commit b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
> reordered the ublk_sub_req_ref() for the completed request before the
> io_buffer_unregister_bvec() call. As a result, task_registered_buffers
> is already 0 when io_buffer_unregister_bvec() calls ublk_io_release()
> and the non-atomic refcount optimization doesn't apply.
> Move the io_buffer_unregister_bvec() call back to before
> ublk_need_complete_req() to restore the reference counting optimization.
> 
> [...]

Applied, thanks!

[1/1] ublk: restore auto buf unregister refcount optimization
      commit: ad5f2e2908c9b79a86529281a48e94d644d43dc7

Best regards,
-- 
Jens Axboe

Re: [PATCH] ublk: restore auto buf unregister refcount optimization

Posted by Ming Lei 1 week, 1 day ago

On Wed, Jan 28, 2026 at 01:56:34PM -0700, Caleb Sander Mateos wrote:
> Commit 1ceeedb59749 ("ublk: optimize UBLK_IO_UNREGISTER_IO_BUF on daemon
> task") optimized ublk request buffer unregistration to use a non-atomic
> reference count decrement when performed on the ublk_io's daemon task.
> The optimization applied to auto buffer unregistration, which happens as
> part of handling UBLK_IO_COMMIT_AND_FETCH_REQ on the daemon task.
> However, commit b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
> reordered the ublk_sub_req_ref() for the completed request before the
> io_buffer_unregister_bvec() call. As a result, task_registered_buffers
> is already 0 when io_buffer_unregister_bvec() calls ublk_io_release()
> and the non-atomic refcount optimization doesn't apply.
> Move the io_buffer_unregister_bvec() call back to before
> ublk_need_complete_req() to restore the reference counting optimization.
> 
> Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> Fixes: b749965edda8 ("ublk: remove ublk_commit_and_fetch()")

Reviewed-by: Ming Lei <ming.lei@redhat.com>

Thanks,
Ming

Re: [PATCH] ublk: restore auto buf unregister refcount optimization

Posted by Caleb Sander Mateos 1 week, 2 days ago

On Wed, Jan 28, 2026 at 12:56 PM Caleb Sander Mateos
<csander@purestorage.com> wrote:
>
> Commit 1ceeedb59749 ("ublk: optimize UBLK_IO_UNREGISTER_IO_BUF on daemon
> task") optimized ublk request buffer unregistration to use a non-atomic
> reference count decrement when performed on the ublk_io's daemon task.
> The optimization applied to auto buffer unregistration, which happens as
> part of handling UBLK_IO_COMMIT_AND_FETCH_REQ on the daemon task.
> However, commit b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
> reordered the ublk_sub_req_ref() for the completed request before the
> io_buffer_unregister_bvec() call. As a result, task_registered_buffers
> is already 0 when io_buffer_unregister_bvec() calls ublk_io_release()
> and the non-atomic refcount optimization doesn't apply.
> Move the io_buffer_unregister_bvec() call back to before
> ublk_need_complete_req() to restore the reference counting optimization.
>
> Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> Fixes: b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
> ---
>  drivers/block/ublk_drv.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
> index 7981decd1cee..f864a0f2f572 100644
> --- a/drivers/block/ublk_drv.c
> +++ b/drivers/block/ublk_drv.c
> @@ -3243,15 +3243,15 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
>                 if (ret)
>                         goto out;
>                 io->res = result;
>                 req = ublk_fill_io_cmd(io, cmd);
>                 ret = ublk_config_io_buf(ub, io, cmd, addr, &buf_idx);
> +               if (buf_idx != UBLK_INVALID_BUF_IDX)
> +                       io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
>                 compl = ublk_need_complete_req(ub, io);
>
>                 /* can't touch 'ublk_io' any more */
> -               if (buf_idx != UBLK_INVALID_BUF_IDX)
> -                       io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
>                 if (req_op(req) == REQ_OP_ZONE_APPEND)
>                         req->__sector = addr;
>                 if (compl)
>                         __ublk_complete_rq(req, io, ublk_dev_need_map_io(ub), NULL);

I also noticed that the "can't touch 'ublk_io' any more" comment
doesn't make much sense, as __ublk_complete_rq() still accesses (and
even mutates) the struct ublk_io. Am I misunderstanding the comment?
It looks like this might be a race condition for
UBLK_U_IO_COMMIT_IO_CMDS, as __ublk_complete_rq() is called without
holding the ublk_io spinlock.

Thanks,
Caleb

Re: [PATCH] ublk: restore auto buf unregister refcount optimization

Posted by Ming Lei 1 week, 2 days ago

On Wed, Jan 28, 2026 at 01:08:17PM -0800, Caleb Sander Mateos wrote:
> On Wed, Jan 28, 2026 at 12:56 PM Caleb Sander Mateos
> <csander@purestorage.com> wrote:
> >
> > Commit 1ceeedb59749 ("ublk: optimize UBLK_IO_UNREGISTER_IO_BUF on daemon
> > task") optimized ublk request buffer unregistration to use a non-atomic
> > reference count decrement when performed on the ublk_io's daemon task.
> > The optimization applied to auto buffer unregistration, which happens as
> > part of handling UBLK_IO_COMMIT_AND_FETCH_REQ on the daemon task.
> > However, commit b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
> > reordered the ublk_sub_req_ref() for the completed request before the
> > io_buffer_unregister_bvec() call. As a result, task_registered_buffers
> > is already 0 when io_buffer_unregister_bvec() calls ublk_io_release()
> > and the non-atomic refcount optimization doesn't apply.
> > Move the io_buffer_unregister_bvec() call back to before
> > ublk_need_complete_req() to restore the reference counting optimization.
> >
> > Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> > Fixes: b749965edda8 ("ublk: remove ublk_commit_and_fetch()")
> > ---
> >  drivers/block/ublk_drv.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
> > index 7981decd1cee..f864a0f2f572 100644
> > --- a/drivers/block/ublk_drv.c
> > +++ b/drivers/block/ublk_drv.c
> > @@ -3243,15 +3243,15 @@ static int ublk_ch_uring_cmd_local(struct io_uring_cmd *cmd,
> >                 if (ret)
> >                         goto out;
> >                 io->res = result;
> >                 req = ublk_fill_io_cmd(io, cmd);
> >                 ret = ublk_config_io_buf(ub, io, cmd, addr, &buf_idx);
> > +               if (buf_idx != UBLK_INVALID_BUF_IDX)
> > +                       io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
> >                 compl = ublk_need_complete_req(ub, io);
> >
> >                 /* can't touch 'ublk_io' any more */
> > -               if (buf_idx != UBLK_INVALID_BUF_IDX)
> > -                       io_buffer_unregister_bvec(cmd, buf_idx, issue_flags);
> >                 if (req_op(req) == REQ_OP_ZONE_APPEND)
> >                         req->__sector = addr;
> >                 if (compl)
> >                         __ublk_complete_rq(req, io, ublk_dev_need_map_io(ub), NULL);
> 
> I also noticed that the "can't touch 'ublk_io' any more" comment
> doesn't make much sense, as __ublk_complete_rq() still accesses (and
> even mutates) the struct ublk_io. Am I misunderstanding the comment?

Yes, it can be removed, originally this code block may be reused for
BATCH_IO, but finally it doesn't work toward this way, so can you remove
it in this patch given it is introduced in b749965edda8 ("ublk: remove ublk_commit_and_fetch()")?

> It looks like this might be a race condition for
> UBLK_U_IO_COMMIT_IO_CMDS, as __ublk_complete_rq() is called without
> holding the ublk_io spinlock.

It is actually fine for __ublk_complete_rq() to manipulate io->res lockless:

1) UBLK_IO_FLAG_OWNED_BY_SRV is cleared & checked with io->lock, so any new
UBLK_U_IO_COMMIT_IO_CMDS will be failed

2) the current IO request isn't completed yet, so new io command handling
won't be started.


Thanks,
Ming