io_uring/io-wq.c | 31 +++++++++++++++++++++++++++++++ io_uring/io-wq.h | 1 + io_uring/tctx.c | 11 +++++++++++ 3 files changed, 43 insertions(+)
io_uring uses io-wq to offload regular file I/O. When that happens, the kernel creates per-task iou-wrk-<tgid> workers (PF_IO_WORKER) via create_io_thread(), so the worker is part of the process thread group and shows up under /proc/<pid>/task/. io-wq shrinks the pool on idle, but it intentionally keeps the last worker around indefinitely as a keepalive to avoid churn. Combined with io_uring's per-task context lifetime (tctx stays attached to the task until exit), a process may permanently retain an idle iou-wrk thread even after it has closed its last io_uring instance and has no active rings. The keepalive behavior is a reasonable default(I guess): workloads may have bursty I/O patterns, and always tearing down the last worker would add thread churn and latency. Creating io-wq workers goes through create_io_thread() (copy_process), which is not cheap to do repeatedly. However, CRIU currently doesn't cope well with such workers being part of the checkpointed thread group. The iou-wrk thread is a kernel-managed worker (PF_IO_WORKER) running io_wq_worker() on a kernel stack, rather than a normal userspace thread executing application code. In our setup, if the iou-wrk thread remains present after quiescing and closing the last io_uring instance, criu dump may hang while trying to stop and dump the thread group. Besides the resource overhead and surprising userspace-visible threads, this is a problem for checkpoint/restore. CRIU needs to freeze and dump all threads in the thread group. With a lingering iou-wrk thread, we observed criu dump can hang even after the ring has been quiesced and the io_uring fd closed, e.g.: criu dump -t $PID -D images -o dump.log -v4 --shell-job ps -T -p $PID -o pid,tid,comm | grep iou-wrk This series is a kernel-side enabler for checkpoint/restore in the current reality where userspace needs to quiesce and close io_uring rings before dump. It is not trying to make io_uring rings checkpointable, nor does it change what CRIU can or cannot restore (e.g. in-flight SQEs/CQEs, SQPOLL, SQE128/CQE32, registered resources). Even with userspace gaining limited io_uring support, this series only targets the specific "no active io_uring contexts left, but an idle iou-wrk keepalive thread remains" case. This series adds an explicit exit-on-idle mode to io-wq, and toggles it from io_uring task context when the task has no active io_uring contexts (xa_empty(&tctx->xa)). The mode is cleared on subsequent io_uring usage, so the default behavior for active io_uring users is unchanged. Tested on x86_64 with CRIU 4.2. With this series applied, after closing the ring iou-wrk exited within ~200ms and criu dump completed. Li Chen (2): io-wq: add exit-on-idle mode io_uring: allow io-wq workers to exit when unused io_uring/io-wq.c | 31 +++++++++++++++++++++++++++++++ io_uring/io-wq.h | 1 + io_uring/tctx.c | 11 +++++++++++ 3 files changed, 43 insertions(+) -- 2.52.0
On 2/2/26 7:37 AM, Li Chen wrote: > io_uring uses io-wq to offload regular file I/O. When that happens, the kernel > creates per-task iou-wrk-<tgid> workers (PF_IO_WORKER) via create_io_thread(), > so the worker is part of the process thread group and shows up under > /proc/<pid>/task/. > > io-wq shrinks the pool on idle, but it intentionally keeps the last worker > around indefinitely as a keepalive to avoid churn. Combined with io_uring's > per-task context lifetime (tctx stays attached to the task until exit), a > process may permanently retain an idle iou-wrk thread even after it has closed > its last io_uring instance and has no active rings. > > The keepalive behavior is a reasonable default(I guess): workloads may have > bursty I/O patterns, and always tearing down the last worker would add thread > churn and latency. Creating io-wq workers goes through create_io_thread() > (copy_process), which is not cheap to do repeatedly. > > However, CRIU currently doesn't cope well with such workers being part of the > checkpointed thread group. The iou-wrk thread is a kernel-managed worker > (PF_IO_WORKER) running io_wq_worker() on a kernel stack, rather than a normal > userspace thread executing application code. In our setup, if the iou-wrk > thread remains present after quiescing and closing the last io_uring instance, > criu dump may hang while trying to stop and dump the thread group. > > Besides the resource overhead and surprising userspace-visible threads, this is > a problem for checkpoint/restore. CRIU needs to freeze and dump all threads in > the thread group. With a lingering iou-wrk thread, we observed criu dump can > hang even after the ring has been quiesced and the io_uring fd closed, e.g.: > > criu dump -t $PID -D images -o dump.log -v4 --shell-job > ps -T -p $PID -o pid,tid,comm | grep iou-wrk > > This series is a kernel-side enabler for checkpoint/restore in the current > reality where userspace needs to quiesce and close io_uring rings before dump. > It is not trying to make io_uring rings checkpointable, nor does it change what > CRIU can or cannot restore (e.g. in-flight SQEs/CQEs, SQPOLL, SQE128/CQE32, > registered resources). Even with userspace gaining limited io_uring support, > this series only targets the specific "no active io_uring contexts left, but an > idle iou-wrk keepalive thread remains" case. > > This series adds an explicit exit-on-idle mode to io-wq, and toggles it from > io_uring task context when the task has no active io_uring contexts > (xa_empty(&tctx->xa)). The mode is cleared on subsequent io_uring usage, so the > default behavior for active io_uring users is unchanged. > > Tested on x86_64 with CRIU 4.2. > With this series applied, after closing the ring iou-wrk exited within ~200ms > and criu dump completed. Applied with the mentioned commit message and IO_WQ_BIT_EXIT_ON_IDLE test placement. -- Jens Axboe
Hi Jens, ---- On Mon, 02 Feb 2026 23:21:22 +0800 Jens Axboe <axboe@kernel.dk> wrote --- > On 2/2/26 7:37 AM, Li Chen wrote: > > io_uring uses io-wq to offload regular file I/O. When that happens, the kernel > > creates per-task iou-wrk-<tgid> workers (PF_IO_WORKER) via create_io_thread(), > > so the worker is part of the process thread group and shows up under > > /proc/<pid>/task/. > > > > io-wq shrinks the pool on idle, but it intentionally keeps the last worker > > around indefinitely as a keepalive to avoid churn. Combined with io_uring's > > per-task context lifetime (tctx stays attached to the task until exit), a > > process may permanently retain an idle iou-wrk thread even after it has closed > > its last io_uring instance and has no active rings. > > > > The keepalive behavior is a reasonable default(I guess): workloads may have > > bursty I/O patterns, and always tearing down the last worker would add thread > > churn and latency. Creating io-wq workers goes through create_io_thread() > > (copy_process), which is not cheap to do repeatedly. > > > > However, CRIU currently doesn't cope well with such workers being part of the > > checkpointed thread group. The iou-wrk thread is a kernel-managed worker > > (PF_IO_WORKER) running io_wq_worker() on a kernel stack, rather than a normal > > userspace thread executing application code. In our setup, if the iou-wrk > > thread remains present after quiescing and closing the last io_uring instance, > > criu dump may hang while trying to stop and dump the thread group. > > > > Besides the resource overhead and surprising userspace-visible threads, this is > > a problem for checkpoint/restore. CRIU needs to freeze and dump all threads in > > the thread group. With a lingering iou-wrk thread, we observed criu dump can > > hang even after the ring has been quiesced and the io_uring fd closed, e.g.: > > > > criu dump -t $PID -D images -o dump.log -v4 --shell-job > > ps -T -p $PID -o pid,tid,comm | grep iou-wrk > > > > This series is a kernel-side enabler for checkpoint/restore in the current > > reality where userspace needs to quiesce and close io_uring rings before dump. > > It is not trying to make io_uring rings checkpointable, nor does it change what > > CRIU can or cannot restore (e.g. in-flight SQEs/CQEs, SQPOLL, SQE128/CQE32, > > registered resources). Even with userspace gaining limited io_uring support, > > this series only targets the specific "no active io_uring contexts left, but an > > idle iou-wrk keepalive thread remains" case. > > > > This series adds an explicit exit-on-idle mode to io-wq, and toggles it from > > io_uring task context when the task has no active io_uring contexts > > (xa_empty(&tctx->xa)). The mode is cleared on subsequent io_uring usage, so the > > default behavior for active io_uring users is unchanged. > > > > Tested on x86_64 with CRIU 4.2. > > With this series applied, after closing the ring iou-wrk exited within ~200ms > > and criu dump completed. > > Applied with the mentioned commit message and IO_WQ_BIT_EXIT_ON_IDLE test > placement. Thanks a lot for your review! If you still want a test, I'm happy to write it. Since you've already tweaked/applied the v1 series, I can send the test as a standalone follow-up patch (no v2). If kselftest is preferred, I'll base it on the same CRIU-style workload: spawn iou-wrk-* via io_uring, quiesce/close the last ring, and check the worker exits within a short timeout. Regards, Li
On 2/2/26 5:37 PM, Li Chen wrote: > Hi Jens, > > ---- On Mon, 02 Feb 2026 23:21:22 +0800 Jens Axboe <axboe@kernel.dk> wrote --- > > On 2/2/26 7:37 AM, Li Chen wrote: > > > io_uring uses io-wq to offload regular file I/O. When that happens, the kernel > > > creates per-task iou-wrk-<tgid> workers (PF_IO_WORKER) via create_io_thread(), > > > so the worker is part of the process thread group and shows up under > > > /proc/<pid>/task/. > > > > > > io-wq shrinks the pool on idle, but it intentionally keeps the last worker > > > around indefinitely as a keepalive to avoid churn. Combined with io_uring's > > > per-task context lifetime (tctx stays attached to the task until exit), a > > > process may permanently retain an idle iou-wrk thread even after it has closed > > > its last io_uring instance and has no active rings. > > > > > > The keepalive behavior is a reasonable default(I guess): workloads may have > > > bursty I/O patterns, and always tearing down the last worker would add thread > > > churn and latency. Creating io-wq workers goes through create_io_thread() > > > (copy_process), which is not cheap to do repeatedly. > > > > > > However, CRIU currently doesn't cope well with such workers being part of the > > > checkpointed thread group. The iou-wrk thread is a kernel-managed worker > > > (PF_IO_WORKER) running io_wq_worker() on a kernel stack, rather than a normal > > > userspace thread executing application code. In our setup, if the iou-wrk > > > thread remains present after quiescing and closing the last io_uring instance, > > > criu dump may hang while trying to stop and dump the thread group. > > > > > > Besides the resource overhead and surprising userspace-visible threads, this is > > > a problem for checkpoint/restore. CRIU needs to freeze and dump all threads in > > > the thread group. With a lingering iou-wrk thread, we observed criu dump can > > > hang even after the ring has been quiesced and the io_uring fd closed, e.g.: > > > > > > criu dump -t $PID -D images -o dump.log -v4 --shell-job > > > ps -T -p $PID -o pid,tid,comm | grep iou-wrk > > > > > > This series is a kernel-side enabler for checkpoint/restore in the current > > > reality where userspace needs to quiesce and close io_uring rings before dump. > > > It is not trying to make io_uring rings checkpointable, nor does it change what > > > CRIU can or cannot restore (e.g. in-flight SQEs/CQEs, SQPOLL, SQE128/CQE32, > > > registered resources). Even with userspace gaining limited io_uring support, > > > this series only targets the specific "no active io_uring contexts left, but an > > > idle iou-wrk keepalive thread remains" case. > > > > > > This series adds an explicit exit-on-idle mode to io-wq, and toggles it from > > > io_uring task context when the task has no active io_uring contexts > > > (xa_empty(&tctx->xa)). The mode is cleared on subsequent io_uring usage, so the > > > default behavior for active io_uring users is unchanged. > > > > > > Tested on x86_64 with CRIU 4.2. > > > With this series applied, after closing the ring iou-wrk exited within ~200ms > > > and criu dump completed. > > > > Applied with the mentioned commit message and IO_WQ_BIT_EXIT_ON_IDLE test > > placement. > > Thanks a lot for your review! > > If you still want a test, I'm happy to write it. Since you've already > tweaked/applied the v1 series, I can send the test as a standalone > follow-up patch (no v2). > > If kselftest is preferred, I'll base it on the same CRIU-style workload: > spawn iou-wrk-* via io_uring, quiesce/close the last ring, and check the > worker exits within a short timeout. That sounds like the right way to do the test. Preferably a liburing test/ case would be better, we don't do a lot of in-kernel selftests so far. But liburing has everything. -- Jens Axboe
Hi Jens, ---- On Tue, 03 Feb 2026 10:29:50 +0800 Jens Axboe <axboe@kernel.dk> wrote --- > On 2/2/26 5:37 PM, Li Chen wrote: > > Hi Jens, > > > > ---- On Mon, 02 Feb 2026 23:21:22 +0800 Jens Axboe <axboe@kernel.dk> wrote --- > > > On 2/2/26 7:37 AM, Li Chen wrote: > > > > io_uring uses io-wq to offload regular file I/O. When that happens, the kernel > > > > creates per-task iou-wrk-<tgid> workers (PF_IO_WORKER) via create_io_thread(), > > > > so the worker is part of the process thread group and shows up under > > > > /proc/<pid>/task/. > > > > > > > > io-wq shrinks the pool on idle, but it intentionally keeps the last worker > > > > around indefinitely as a keepalive to avoid churn. Combined with io_uring's > > > > per-task context lifetime (tctx stays attached to the task until exit), a > > > > process may permanently retain an idle iou-wrk thread even after it has closed > > > > its last io_uring instance and has no active rings. > > > > > > > > The keepalive behavior is a reasonable default(I guess): workloads may have > > > > bursty I/O patterns, and always tearing down the last worker would add thread > > > > churn and latency. Creating io-wq workers goes through create_io_thread() > > > > (copy_process), which is not cheap to do repeatedly. > > > > > > > > However, CRIU currently doesn't cope well with such workers being part of the > > > > checkpointed thread group. The iou-wrk thread is a kernel-managed worker > > > > (PF_IO_WORKER) running io_wq_worker() on a kernel stack, rather than a normal > > > > userspace thread executing application code. In our setup, if the iou-wrk > > > > thread remains present after quiescing and closing the last io_uring instance, > > > > criu dump may hang while trying to stop and dump the thread group. > > > > > > > > Besides the resource overhead and surprising userspace-visible threads, this is > > > > a problem for checkpoint/restore. CRIU needs to freeze and dump all threads in > > > > the thread group. With a lingering iou-wrk thread, we observed criu dump can > > > > hang even after the ring has been quiesced and the io_uring fd closed, e.g.: > > > > > > > > criu dump -t $PID -D images -o dump.log -v4 --shell-job > > > > ps -T -p $PID -o pid,tid,comm | grep iou-wrk > > > > > > > > This series is a kernel-side enabler for checkpoint/restore in the current > > > > reality where userspace needs to quiesce and close io_uring rings before dump. > > > > It is not trying to make io_uring rings checkpointable, nor does it change what > > > > CRIU can or cannot restore (e.g. in-flight SQEs/CQEs, SQPOLL, SQE128/CQE32, > > > > registered resources). Even with userspace gaining limited io_uring support, > > > > this series only targets the specific "no active io_uring contexts left, but an > > > > idle iou-wrk keepalive thread remains" case. > > > > > > > > This series adds an explicit exit-on-idle mode to io-wq, and toggles it from > > > > io_uring task context when the task has no active io_uring contexts > > > > (xa_empty(&tctx->xa)). The mode is cleared on subsequent io_uring usage, so the > > > > default behavior for active io_uring users is unchanged. > > > > > > > > Tested on x86_64 with CRIU 4.2. > > > > With this series applied, after closing the ring iou-wrk exited within ~200ms > > > > and criu dump completed. > > > > > > Applied with the mentioned commit message and IO_WQ_BIT_EXIT_ON_IDLE test > > > placement. > > > > Thanks a lot for your review! > > > > If you still want a test, I'm happy to write it. Since you've already > > tweaked/applied the v1 series, I can send the test as a standalone > > follow-up patch (no v2). > > > > If kselftest is preferred, I'll base it on the same CRIU-style workload: > > spawn iou-wrk-* via io_uring, quiesce/close the last ring, and check the > > worker exits within a short timeout. > > That sounds like the right way to do the test. Preferably a liburing > test/ case would be better, we don't do a lot of in-kernel selftests so > far. But liburing has everything. Thanks for your suggestion. I just adapted my local test program to liburing and posted the liburing PR here: https://github.com/axboe/liburing/pull/1529 Regards, Li
On 2/3/26 12:47 AM, Li Chen wrote: > Hi Jens, > > > If you still want a test, I'm happy to write it. Since you've already > > > tweaked/applied the v1 series, I can send the test as a standalone > > > follow-up patch (no v2). > > > > > > If kselftest is preferred, I'll base it on the same CRIU-style workload: > > > spawn iou-wrk-* via io_uring, quiesce/close the last ring, and check the > > > worker exits within a short timeout. > > > > That sounds like the right way to do the test. Preferably a liburing > > test/ case would be better, we don't do a lot of in-kernel selftests so > > far. But liburing has everything. > > Thanks for your suggestion. I just adapted my local test program to > liburing and posted the liburing PR here: > https://github.com/axboe/liburing/pull/1529 Thanks, merged! -- Jens Axboe
© 2016 - 2026 Red Hat, Inc.