[v2] io_uring: avoid uring_lock for IORING_SETUP_SINGLE_ISSUER

[PATCH v2 3/5] io_uring: clear IORING_SETUP_SINGLE_ISSUER for IORING_SETUP_SQPOLL

Posted by Caleb Sander Mateos 4 weeks ago

IORING_SETUP_SINGLE_ISSUER doesn't currently enable any optimizations,
but it will soon be used to avoid taking io_ring_ctx's uring_lock when
submitting from the single issuer task. If the IORING_SETUP_SQPOLL flag
is set, the SQ thread is the sole task issuing SQEs. However, other
tasks may make io_uring_register() syscalls, which must be synchronized
with SQE submission. So it wouldn't be safe to skip the uring_lock
around the SQ thread's submission even if IORING_SETUP_SINGLE_ISSUER is
set. Therefore, clear IORING_SETUP_SINGLE_ISSUER from the io_ring_ctx
flags if IORING_SETUP_SQPOLL is set.

Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
---
 io_uring/io_uring.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index 42f6bfbb99d3..c7af9dc3d95a 100644
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -3724,10 +3724,19 @@ static int io_uring_sanitise_params(struct io_uring_params *p)
 	 */
 	if ((flags & (IORING_SETUP_CQE32|IORING_SETUP_CQE_MIXED)) ==
 	    (IORING_SETUP_CQE32|IORING_SETUP_CQE_MIXED))
 		return -EINVAL;
 
+	/*
+	 * If IORING_SETUP_SQPOLL is set, only the SQ thread issues SQEs,
+	 * but other threads may call io_uring_register() concurrently.
+	 * We still need uring_lock to synchronize these io_ring_ctx accesses,
+	 * so disable the single issuer optimizations.
+	 */
+	if (flags & IORING_SETUP_SQPOLL)
+		p->flags &= ~IORING_SETUP_SINGLE_ISSUER;
+
 	return 0;
 }
 
 int io_uring_fill_params(unsigned entries, struct io_uring_params *p)
 {
-- 
2.45.2

Re: [PATCH v2 3/5] io_uring: clear IORING_SETUP_SINGLE_ISSUER for IORING_SETUP_SQPOLL

Posted by Jens Axboe 3 weeks, 3 days ago

On 9/4/25 11:09 AM, Caleb Sander Mateos wrote:
> IORING_SETUP_SINGLE_ISSUER doesn't currently enable any optimizations,
> but it will soon be used to avoid taking io_ring_ctx's uring_lock when
> submitting from the single issuer task. If the IORING_SETUP_SQPOLL flag
> is set, the SQ thread is the sole task issuing SQEs. However, other
> tasks may make io_uring_register() syscalls, which must be synchronized
> with SQE submission. So it wouldn't be safe to skip the uring_lock
> around the SQ thread's submission even if IORING_SETUP_SINGLE_ISSUER is
> set. Therefore, clear IORING_SETUP_SINGLE_ISSUER from the io_ring_ctx
> flags if IORING_SETUP_SQPOLL is set.
> 
> Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> ---
>  io_uring/io_uring.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
> index 42f6bfbb99d3..c7af9dc3d95a 100644
> --- a/io_uring/io_uring.c
> +++ b/io_uring/io_uring.c
> @@ -3724,10 +3724,19 @@ static int io_uring_sanitise_params(struct io_uring_params *p)
>  	 */
>  	if ((flags & (IORING_SETUP_CQE32|IORING_SETUP_CQE_MIXED)) ==
>  	    (IORING_SETUP_CQE32|IORING_SETUP_CQE_MIXED))
>  		return -EINVAL;
>  
> +	/*
> +	 * If IORING_SETUP_SQPOLL is set, only the SQ thread issues SQEs,
> +	 * but other threads may call io_uring_register() concurrently.
> +	 * We still need uring_lock to synchronize these io_ring_ctx accesses,
> +	 * so disable the single issuer optimizations.
> +	 */
> +	if (flags & IORING_SETUP_SQPOLL)
> +		p->flags &= ~IORING_SETUP_SINGLE_ISSUER;
> +

As mentioned I think this is fine. Just for posterity, one solution
here would be to require that the task doing eg io_uring_register() on a
setup with SINGLE_ISSUER|SQPOLL would be required to park and unpark the
SQ thread before doing what it needs to do. That should get us most/all
of the way there to enabling it with SQPOLL as well.

-- 
Jens Axboe

Re: [PATCH v2 3/5] io_uring: clear IORING_SETUP_SINGLE_ISSUER for IORING_SETUP_SQPOLL

Posted by Caleb Sander Mateos 3 weeks, 3 days ago

On Mon, Sep 8, 2025 at 7:13 AM Jens Axboe <axboe@kernel.dk> wrote:
>
> On 9/4/25 11:09 AM, Caleb Sander Mateos wrote:
> > IORING_SETUP_SINGLE_ISSUER doesn't currently enable any optimizations,
> > but it will soon be used to avoid taking io_ring_ctx's uring_lock when
> > submitting from the single issuer task. If the IORING_SETUP_SQPOLL flag
> > is set, the SQ thread is the sole task issuing SQEs. However, other
> > tasks may make io_uring_register() syscalls, which must be synchronized
> > with SQE submission. So it wouldn't be safe to skip the uring_lock
> > around the SQ thread's submission even if IORING_SETUP_SINGLE_ISSUER is
> > set. Therefore, clear IORING_SETUP_SINGLE_ISSUER from the io_ring_ctx
> > flags if IORING_SETUP_SQPOLL is set.
> >
> > Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
> > ---
> >  io_uring/io_uring.c | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> >
> > diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
> > index 42f6bfbb99d3..c7af9dc3d95a 100644
> > --- a/io_uring/io_uring.c
> > +++ b/io_uring/io_uring.c
> > @@ -3724,10 +3724,19 @@ static int io_uring_sanitise_params(struct io_uring_params *p)
> >        */
> >       if ((flags & (IORING_SETUP_CQE32|IORING_SETUP_CQE_MIXED)) ==
> >           (IORING_SETUP_CQE32|IORING_SETUP_CQE_MIXED))
> >               return -EINVAL;
> >
> > +     /*
> > +      * If IORING_SETUP_SQPOLL is set, only the SQ thread issues SQEs,
> > +      * but other threads may call io_uring_register() concurrently.
> > +      * We still need uring_lock to synchronize these io_ring_ctx accesses,
> > +      * so disable the single issuer optimizations.
> > +      */
> > +     if (flags & IORING_SETUP_SQPOLL)
> > +             p->flags &= ~IORING_SETUP_SINGLE_ISSUER;
> > +
>
> As mentioned I think this is fine. Just for posterity, one solution
> here would be to require that the task doing eg io_uring_register() on a
> setup with SINGLE_ISSUER|SQPOLL would be required to park and unpark the
> SQ thread before doing what it needs to do. That should get us most/all
> of the way there to enabling it with SQPOLL as well.

Right, though that may make io_uring_register() significantly slower
and disruptive to the I/O path. Another option would be to proxy all
registrations to the SQ thread via task_work. I think leaving the
current behavior as-is makes the most sense to avoid any regressions.
If someone is interested in optimizing the IORING_SETUP_SQPOLL &&
IORING_SETUP_SINGLE_ISSUER use case, they're more than welcome to!

I appreciate your feedback on the series. Do you have any other thoughts on it?

Best,
Caleb

Re: [PATCH v2 3/5] io_uring: clear IORING_SETUP_SINGLE_ISSUER for IORING_SETUP_SQPOLL

Posted by Jens Axboe 3 weeks, 3 days ago

On 9/8/25 12:11 PM, Caleb Sander Mateos wrote:
> On Mon, Sep 8, 2025 at 7:13?AM Jens Axboe <axboe@kernel.dk> wrote:
>>
>> On 9/4/25 11:09 AM, Caleb Sander Mateos wrote:
>>> IORING_SETUP_SINGLE_ISSUER doesn't currently enable any optimizations,
>>> but it will soon be used to avoid taking io_ring_ctx's uring_lock when
>>> submitting from the single issuer task. If the IORING_SETUP_SQPOLL flag
>>> is set, the SQ thread is the sole task issuing SQEs. However, other
>>> tasks may make io_uring_register() syscalls, which must be synchronized
>>> with SQE submission. So it wouldn't be safe to skip the uring_lock
>>> around the SQ thread's submission even if IORING_SETUP_SINGLE_ISSUER is
>>> set. Therefore, clear IORING_SETUP_SINGLE_ISSUER from the io_ring_ctx
>>> flags if IORING_SETUP_SQPOLL is set.
>>>
>>> Signed-off-by: Caleb Sander Mateos <csander@purestorage.com>
>>> ---
>>>  io_uring/io_uring.c | 9 +++++++++
>>>  1 file changed, 9 insertions(+)
>>>
>>> diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
>>> index 42f6bfbb99d3..c7af9dc3d95a 100644
>>> --- a/io_uring/io_uring.c
>>> +++ b/io_uring/io_uring.c
>>> @@ -3724,10 +3724,19 @@ static int io_uring_sanitise_params(struct io_uring_params *p)
>>>        */
>>>       if ((flags & (IORING_SETUP_CQE32|IORING_SETUP_CQE_MIXED)) ==
>>>           (IORING_SETUP_CQE32|IORING_SETUP_CQE_MIXED))
>>>               return -EINVAL;
>>>
>>> +     /*
>>> +      * If IORING_SETUP_SQPOLL is set, only the SQ thread issues SQEs,
>>> +      * but other threads may call io_uring_register() concurrently.
>>> +      * We still need uring_lock to synchronize these io_ring_ctx accesses,
>>> +      * so disable the single issuer optimizations.
>>> +      */
>>> +     if (flags & IORING_SETUP_SQPOLL)
>>> +             p->flags &= ~IORING_SETUP_SINGLE_ISSUER;
>>> +
>>
>> As mentioned I think this is fine. Just for posterity, one solution
>> here would be to require that the task doing eg io_uring_register() on a
>> setup with SINGLE_ISSUER|SQPOLL would be required to park and unpark the
>> SQ thread before doing what it needs to do. That should get us most/all
>> of the way there to enabling it with SQPOLL as well.
> 
> Right, though that may make io_uring_register() significantly slower
> and disruptive to the I/O path. Another option would be to proxy all
> registrations to the SQ thread via task_work. I think leaving the
> current behavior as-is makes the most sense to avoid any regressions.
> If someone is interested in optimizing the IORING_SETUP_SQPOLL &&
> IORING_SETUP_SINGLE_ISSUER use case, they're more than welcome to!

True, though for most cases that won't matter, but for some it certainly
could. I certainly agree that this is a problen that's best deferred
anyway, SQPOLL is a bit of an oddball use case anyway.

> I appreciate your feedback on the series. Do you have any other
> thoughts on it?

Looks pretty clean to me, no big concerns honestly.

-- 
Jens Axboe