io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's
reference count in case it has been cloned into another io_ring_ctx's
registered buffer table. This is an expensive operation and unnecessary
in the common case that the io_mapped_ubuf is only registered once.
Load the reference count first and check whether it's 1. In that case,
skip the atomic decrement and immediately free the io_mapped_ubuf.

Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
---
 io_uring/rsrc.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
index 94a9db030e0e..9a1f24a43035 100644
--- a/io_uring/rsrc.c
+++ b/io_uring/rsrc.c
@@ -133,12 +133,14 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
 		kvfree(imu);
 }
 
 static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
 {
-	if (!refcount_dec_and_test(&imu->refs))
-		return;
+	if (unlikely(refcount_read(&imu->refs) > 1)) {
+		if (!refcount_dec_and_test(&imu->refs))
+			return;
+	}
 
 	if (imu->acct_pages)
 		io_unaccount_mem(ctx, imu->acct_pages);
 	imu->release(imu->priv);
 	io_free_imu(ctx, imu);
-- 
2.45.2

On Thu, 19 Jun 2025 08:34:34 -0600, Caleb Sander Mateos wrote:
> io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's
> reference count in case it has been cloned into another io_ring_ctx's
> registered buffer table. This is an expensive operation and unnecessary
> in the common case that the io_mapped_ubuf is only registered once.
> Load the reference count first and check whether it's 1. In that case,
> skip the atomic decrement and immediately free the io_mapped_ubuf.
> 
> [...]

Applied, thanks!

[1/1] io_uring/rsrc: skip atomic refcount for uncloned buffers
      commit: daa01d954b13a178c216b6a91f8451a7b83b3bf6

Best regards,
-- 
Jens Axboe

Hi Jens,
Any concerns with this one? I thought it was a fairly straightforward
optimization in the ublk zero-copy I/O path.

Thanks,
Caleb

On Thu, Jun 19, 2025 at 10:34 AM Caleb Sander Mateos
<csander@purestorage.com> wrote:
>
> io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's
> reference count in case it has been cloned into another io_ring_ctx's
> registered buffer table. This is an expensive operation and unnecessary
> in the common case that the io_mapped_ubuf is only registered once.
> Load the reference count first and check whether it's 1. In that case,
> skip the atomic decrement and immediately free the io_mapped_ubuf.
>
> Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> ---
>  io_uring/rsrc.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c
> index 94a9db030e0e..9a1f24a43035 100644
> --- a/io_uring/rsrc.c
> +++ b/io_uring/rsrc.c
> @@ -133,12 +133,14 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
>                 kvfree(imu);
>  }
>
>  static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)
>  {
> -       if (!refcount_dec_and_test(&imu->refs))
> -               return;
> +       if (unlikely(refcount_read(&imu->refs) > 1)) {
> +               if (!refcount_dec_and_test(&imu->refs))
> +                       return;
> +       }
>
>         if (imu->acct_pages)
>                 io_unaccount_mem(ctx, imu->acct_pages);
>         imu->release(imu->priv);
>         io_free_imu(ctx, imu);
> --
> 2.45.2
>

On 7/2/25 3:11 PM, Caleb Sander Mateos wrote:
> Hi Jens,
> Any concerns with this one? I thought it was a fairly straightforward
> optimization in the ublk zero-copy I/O path.

Nope looks fine, I just have a largish backlog from being gone for 10
days. I'll queue it up for 6.17.

-- 
Jens Axboe

On Wed, Jul 2, 2025 at 7:10 PM Jens Axboe <axboe@kernel.dk> wrote:
>
> On 7/2/25 3:11 PM, Caleb Sander Mateos wrote:
> > Hi Jens,
> > Any concerns with this one? I thought it was a fairly straightforward
> > optimization in the ublk zero-copy I/O path.
>
> Nope looks fine, I just have a largish backlog from being gone for 10
> days. I'll queue it up for 6.17.

No worries, I was away for a week too. Just wanted to make sure it
hadn't fallen off the radar.

Thanks,
Caleb