From nobody Thu Apr 2 20:26:36 2026 Received: from mail-qv1-f102.google.com (mail-qv1-f102.google.com [209.85.219.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 350EF426D15 for ; Mon, 2 Mar 2026 17:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.102 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472566; cv=none; b=geDSpwx+Jzf7MrNyguYFr5uTjRwWh6SSXA+bL77W5x3opWrMzQ8P2fKbpz2NwYUYv6m8E6sLx88r7sAgh40CnMqlL1OX9OCp/iKYiavlmGhpGkOA3+rPUwUU4jy2ptXBPnjxl5vGxRs9YViwzMUIG00xuIVX9OpkXSeObHARsMI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472566; c=relaxed/simple; bh=uxTyPfsI/g87msoy9eUMmN8qbKp3Wpg2CUlMnqYjrOo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O9M4uD92mjtST7NXc8nuvZ50eKmJpby8fOoU3QYYhduhB/ZydoXwBl9JQ4+pNgBMQJB5ni7hVICvlqfSXzvQveXP61pXo+VmZ/0Eh51+qI+FZdk0DVtVmARjcN0X5Iq/60pucK6c+cFY02XqErpv2l9ITR8gVd7eLMttQTpnnv8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=a3Mt3AH7; arc=none smtp.client-ip=209.85.219.102 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="a3Mt3AH7" Received: by mail-qv1-f102.google.com with SMTP id 6a1803df08f44-899e9a3bcf8so5941696d6.3 for ; Mon, 02 Mar 2026 09:29:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772472562; x=1773077362; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KD/WfRuGbaSn+6cq0PfA2+nK80LzEXM/LyfAsAFa1e0=; b=a3Mt3AH7mpCI8ZCxeh7fP3kVPnUOG3r4oxGgMSl0M1b0r7KrrXZmk90izESZ1e32Lp 9pbitYA2AIah4Po2eGawo7W1+GAizmI1xnnk9p9VE2nVvoShwA2ixsP/DOVcFns4VXYt nGlo3ElYGEyK4/SaNtSXWMoHWPh0Rh4Q+qG8OUYYUhtxUHv5lelxeczilvJRAaJDoXad OwnXVSyloLsSpZoPrA85PwYKkooJBRR9xazW4ZsHJduEAUwtulkEg5ur+HrySM+aHArI iYjljgJ9rThSiKqyYihobbm4E0cpEdMPjt9vQQ3Y+/1qsPWieRBe/vA1RDucYhSGn8pE ysaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772472562; x=1773077362; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=KD/WfRuGbaSn+6cq0PfA2+nK80LzEXM/LyfAsAFa1e0=; b=mXq5dSKFajVK/QV08HHpIRnpPd5PD2tnmeuBXiLuQGwqbg1bcnOvlmoBLB6+uFa8z4 n6SiuQHtm3hnzD84KGEJJ0nqb2MxWJK8rq9oJdN7YQBxCf/bIgc29e4h7KO6IE9ER6NZ 7kAVlXzMl9zntBTXMu6cgk9Aeeg2ca7hecAyfw+hcfZOuC1/b3Rb5YSPe2qVOXDM7Jkb c9jLir3SNqgCmOfplgznisMAkm7QoYUDb8PJRFTAGQe7dfCiewCnMZv7zw4w2kqS9uKi M1bJOokbhI25eEnC89sIMuJn4ocVBSHIyX7wak04qLQAr/DZrjr3KO7Vm9hb8ihfMoWJ KrHg== X-Forwarded-Encrypted: i=1; AJvYcCVPys0JtEoqczaYXsnx5T6ITZ9FpjpItgBka9od5lqd+EMuKmZze6cjNwgkAmwtP/KkMLPdiQkev2wNXzA=@vger.kernel.org X-Gm-Message-State: AOJu0Yw51YdIzRTqsOMFlhSa2TEJ9Sz0EIhoG0YdgO2yC1K6IOhreijq 7PlyzMEBJ5f/CyRvnLXHZ86xZ4zz4aqV1c/p314UOAjo2lbrqphu5O8ex5iVOBkIfecFM+oXlJ0 utWgvD0z1/f4M/+JJeqv2otNsYmELURZx/GF/ X-Gm-Gg: ATEYQzy4QahYwTZF3jUvLDqRvxY/l5ib4GIvQkNustNiN9Gn3H5nE9eOCg12kRMUvvW yNKfyLP5/ScydIwqv86KumGAxaF2Yhrca2Nd5UW4RA0yRRuSsllbyE3/uWqg4K4l9leZD995ti+ VCxbCCS24C5lAw5yN3I9Sn2taEHBe6eWV5VJeFIDzL1j1Xg0fEA2mCmVIesN4a6D6CjfkUUCj/Y qxoTvlasNjA8oQVL54ZBv2jWw2slPiGsak3hxcrUWnZQBwtl78KRlKFnaPF3QUtktJnZW88E6Zn nipvGMOAB86bJ8/uuXpHIRAfzk4dxiIA59x9Q9wO3H5DkloGJjl1jv+tH21Dd1Ndu67rrNmsxjU kvJ7Q0k6nXVZ5HjqpFcYmnmKO9Jfx4zU71u01x7vxk2ggSXIejkgTig== X-Received: by 2002:a0c:e006:0:b0:89a:591:15a9 with SMTP id 6a1803df08f44-89a05911899mr16836046d6.6.1772472562007; Mon, 02 Mar 2026 09:29:22 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.128]) by smtp-relay.gmail.com with ESMTPS id 6a1803df08f44-899eda739e2sm6897186d6.21.2026.03.02.09.29.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Mar 2026 09:29:22 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 6A64F340316; Mon, 2 Mar 2026 10:29:21 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id 63FE1E41FBD; Mon, 2 Mar 2026 10:29:21 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v5 1/5] io_uring: add REQ_F_IOPOLL Date: Mon, 2 Mar 2026 10:29:10 -0700 Message-ID: <20260302172914.2488599-2-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260302172914.2488599-1-csander@purestorage.com> References: <20260302172914.2488599-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A subsequent commit will allow uring_cmds to files that don't implement ->uring_cmd_iopoll() to be issued to IORING_SETUP_IOPOLL io_urings. This means the ctx's IORING_SETUP_IOPOLL flag isn't sufficient to determine whether a given request needs to be iopolled. Introduce a request flag REQ_F_IOPOLL set in ->issue() if a request needs to be iopolled to completion. Set the flag in io_rw_init_file() and io_uring_cmd() for requests issued to IORING_SETUP_IOPOLL ctxs. Use the request flag instead of IORING_SETUP_IOPOLL in places dealing with a specific request. A future possibility would be to add an option to enable/disable iopoll in the io_uring SQE instead of determining it from IORING_SETUP_IOPOLL. Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta --- include/linux/io_uring_types.h | 3 +++ io_uring/io_uring.c | 9 ++++----- io_uring/rw.c | 11 ++++++----- io_uring/uring_cmd.c | 5 +++-- 4 files changed, 16 insertions(+), 12 deletions(-) diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index 3e4a82a6f817..d74b2a8c7305 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -541,10 +541,11 @@ enum { REQ_F_BUFFERS_COMMIT_BIT, REQ_F_BUF_NODE_BIT, REQ_F_HAS_METADATA_BIT, REQ_F_IMPORT_BUFFER_BIT, REQ_F_SQE_COPIED_BIT, + REQ_F_IOPOLL_BIT, =20 /* not a real bit, just to check we're not overflowing the space */ __REQ_F_LAST_BIT, }; =20 @@ -632,10 +633,12 @@ enum { * For SEND_ZC, whether to import buffers (i.e. the first issue). */ REQ_F_IMPORT_BUFFER =3D IO_REQ_FLAG(REQ_F_IMPORT_BUFFER_BIT), /* ->sqe_copy() has been called, if necessary */ REQ_F_SQE_COPIED =3D IO_REQ_FLAG(REQ_F_SQE_COPIED_BIT), + /* request must be iopolled to completion (set in ->issue()) */ + REQ_F_IOPOLL =3D IO_REQ_FLAG(REQ_F_IOPOLL_BIT), }; =20 struct io_tw_req { struct io_kiocb *req; }; diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index aa95703165f1..e7f392e962bd 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -354,11 +354,10 @@ static struct io_kiocb *__io_prep_linked_timeout(stru= ct io_kiocb *req) } =20 static void io_prep_async_work(struct io_kiocb *req) { const struct io_issue_def *def =3D &io_issue_defs[req->opcode]; - struct io_ring_ctx *ctx =3D req->ctx; =20 if (!(req->flags & REQ_F_CREDS)) { req->flags |=3D REQ_F_CREDS; req->creds =3D get_current_cred(); } @@ -376,11 +375,11 @@ static void io_prep_async_work(struct io_kiocb *req) =20 /* don't serialize this request if the fs doesn't need it */ if (should_hash && (req->file->f_flags & O_DIRECT) && (req->file->f_op->fop_flags & FOP_DIO_PARALLEL_WRITE)) should_hash =3D false; - if (should_hash || (ctx->flags & IORING_SETUP_IOPOLL)) + if (should_hash || (req->flags & REQ_F_IOPOLL)) io_wq_hash_work(&req->work, file_inode(req->file)); } else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) { if (def->unbound_nonreg_file) atomic_or(IO_WQ_WORK_UNBOUND, &req->work.flags); } @@ -1417,11 +1416,11 @@ static int io_issue_sqe(struct io_kiocb *req, unsig= ned int issue_flags) =20 if (ret =3D=3D IOU_ISSUE_SKIP_COMPLETE) { ret =3D 0; =20 /* If the op doesn't have a file, we're not polling for it */ - if ((req->ctx->flags & IORING_SETUP_IOPOLL) && def->iopoll_queue) + if ((req->flags & REQ_F_IOPOLL) && def->iopoll_queue) io_iopoll_req_issued(req, issue_flags); } return ret; } =20 @@ -1433,11 +1432,11 @@ int io_poll_issue(struct io_kiocb *req, io_tw_token= _t tw) int ret; =20 io_tw_lock(req->ctx, tw); =20 WARN_ON_ONCE(!req->file); - if (WARN_ON_ONCE(req->ctx->flags & IORING_SETUP_IOPOLL)) + if (WARN_ON_ONCE(req->flags & REQ_F_IOPOLL)) return -EFAULT; =20 ret =3D __io_issue_sqe(req, issue_flags, &io_issue_defs[req->opcode]); =20 WARN_ON_ONCE(ret =3D=3D IOU_ISSUE_SKIP_COMPLETE); @@ -1531,11 +1530,11 @@ void io_wq_submit_work(struct io_wq_work *work) * We can get EAGAIN for iopolled IO even though we're * forcing a sync submission from here, since we can't * wait for request slots on the block side. */ if (!needs_poll) { - if (!(req->ctx->flags & IORING_SETUP_IOPOLL)) + if (!(req->flags & REQ_F_IOPOLL)) break; if (io_wq_worker_stopped()) break; cond_resched(); continue; diff --git a/io_uring/rw.c b/io_uring/rw.c index 1a5f262734e8..3bdb9914e673 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -502,11 +502,11 @@ static bool io_rw_should_reissue(struct io_kiocb *req) struct io_ring_ctx *ctx =3D req->ctx; =20 if (!S_ISBLK(mode) && !S_ISREG(mode)) return false; if ((req->flags & REQ_F_NOWAIT) || (io_wq_current_is_worker() && - !(ctx->flags & IORING_SETUP_IOPOLL))) + !(req->flags & REQ_F_IOPOLL))) return false; /* * If ref is dying, we might be running poll reap from the exit work. * Don't attempt to reissue from that path, just let it fail with * -EAGAIN. @@ -638,11 +638,11 @@ static inline void io_rw_done(struct io_kiocb *req, s= size_t ret) ret =3D -EINTR; break; } } =20 - if (req->ctx->flags & IORING_SETUP_IOPOLL) + if (req->flags & REQ_F_IOPOLL) io_complete_rw_iopoll(&rw->kiocb, ret); else io_complete_rw(&rw->kiocb, ret); } =20 @@ -652,11 +652,11 @@ static int kiocb_done(struct io_kiocb *req, ssize_t r= et, struct io_rw *rw =3D io_kiocb_to_cmd(req, struct io_rw); unsigned final_ret =3D io_fixup_rw_res(req, ret); =20 if (ret >=3D 0 && req->flags & REQ_F_CUR_POS) req->file->f_pos =3D rw->kiocb.ki_pos; - if (ret >=3D 0 && !(req->ctx->flags & IORING_SETUP_IOPOLL)) { + if (ret >=3D 0 && !(req->flags & REQ_F_IOPOLL)) { u32 cflags =3D 0; =20 __io_complete_rw_common(req, ret); /* * Safe to call io_end from here as we're inline @@ -874,10 +874,11 @@ static int io_rw_init_file(struct io_kiocb *req, fmod= e_t mode, int rw_type) req->flags |=3D REQ_F_NOWAIT; =20 if (ctx->flags & IORING_SETUP_IOPOLL) { if (!(kiocb->ki_flags & IOCB_DIRECT) || !file->f_op->iopoll) return -EOPNOTSUPP; + req->flags |=3D REQ_F_IOPOLL; kiocb->private =3D NULL; kiocb->ki_flags |=3D IOCB_HIPRI; req->iopoll_completed =3D 0; if (ctx->flags & IORING_SETUP_HYBRID_IOPOLL) { /* make sure every req only blocks once*/ @@ -961,11 +962,11 @@ static int __io_read(struct io_kiocb *req, struct io_= br_sel *sel, if (ret =3D=3D -EAGAIN) { /* If we can poll, just do that. */ if (io_file_can_poll(req)) return -EAGAIN; /* IOPOLL retry should happen for io-wq threads */ - if (!force_nonblock && !(req->ctx->flags & IORING_SETUP_IOPOLL)) + if (!force_nonblock && !(req->flags & REQ_F_IOPOLL)) goto done; /* no retry on NONBLOCK nor RWF_NOWAIT */ if (req->flags & REQ_F_NOWAIT) goto done; ret =3D 0; @@ -1186,11 +1187,11 @@ int io_write(struct io_kiocb *req, unsigned int iss= ue_flags) /* no retry on NONBLOCK nor RWF_NOWAIT */ if (ret2 =3D=3D -EAGAIN && (req->flags & REQ_F_NOWAIT)) goto done; if (!force_nonblock || ret2 !=3D -EAGAIN) { /* IOPOLL retry should happen for io-wq threads */ - if (ret2 =3D=3D -EAGAIN && (req->ctx->flags & IORING_SETUP_IOPOLL)) + if (ret2 =3D=3D -EAGAIN && (req->flags & REQ_F_IOPOLL)) goto ret_eagain; =20 if (ret2 !=3D req->cqe.res && ret2 >=3D 0 && need_complete_io(req)) { trace_io_uring_short_write(req->ctx, kiocb->ki_pos - ret2, req->cqe.res, ret2); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index ee7b49f47cb5..b651c63f6e20 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -108,11 +108,11 @@ void io_uring_cmd_mark_cancelable(struct io_uring_cmd= *cmd, * Doing cancelations on IOPOLL requests are not supported. Both * because they can't get canceled in the block stack, but also * because iopoll completion data overlaps with the hash_node used * for tracking. */ - if (ctx->flags & IORING_SETUP_IOPOLL) + if (req->flags & REQ_F_IOPOLL) return; =20 if (!(cmd->flags & IORING_URING_CMD_CANCELABLE)) { cmd->flags |=3D IORING_URING_CMD_CANCELABLE; io_ring_submit_lock(ctx, issue_flags); @@ -165,11 +165,11 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd,= s32 ret, u64 res2, if (req->ctx->flags & IORING_SETUP_CQE_MIXED) req->cqe.flags |=3D IORING_CQE_F_32; io_req_set_cqe32_extra(req, res2, 0); } io_req_uring_cleanup(req, issue_flags); - if (req->ctx->flags & IORING_SETUP_IOPOLL) { + if (req->flags & REQ_F_IOPOLL) { /* order with io_iopoll_req_issued() checking ->iopoll_complete */ smp_store_release(&req->iopoll_completed, 1); } else if (issue_flags & IO_URING_F_COMPLETE_DEFER) { if (WARN_ON_ONCE(issue_flags & IO_URING_F_UNLOCKED)) return; @@ -258,10 +258,11 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int i= ssue_flags) if (io_is_compat(ctx)) issue_flags |=3D IO_URING_F_COMPAT; if (ctx->flags & IORING_SETUP_IOPOLL) { if (!file->f_op->uring_cmd_iopoll) return -EOPNOTSUPP; + req->flags |=3D REQ_F_IOPOLL; issue_flags |=3D IO_URING_F_IOPOLL; req->iopoll_completed =3D 0; if (ctx->flags & IORING_SETUP_HYBRID_IOPOLL) { /* make sure every req only blocks once */ req->flags &=3D ~REQ_F_IOPOLL_STATE; --=20 2.45.2 From nobody Thu Apr 2 20:26:36 2026 Received: from mail-pf1-f228.google.com (mail-pf1-f228.google.com [209.85.210.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72946425CD2 for ; Mon, 2 Mar 2026 17:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.228 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472564; cv=none; b=C8xn5cNStr5EuVL//KSro0onVyrBE6GJlCoPdlhkpfncBmFwQKeJuLkZ+04Inm7zwPP5sFp3yU36IiV+6m20dGADsi+fTeiCua6ynG8qgu03cyaRLD6YDzwcXFEMoB8UoZJ/ZOOKBSNUDu8uaqh5wE4M8DmP9iE6z1/p8eRyU1g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472564; c=relaxed/simple; bh=mZRg9aJ/vT9KGxR37N9gh9XV0GFWpQmAWSI7PzYmIhE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Vc+itFpg+hUJRioJT5lPV9cqhbrhUL2jEa9vMB82l0JM0G2577+x50inp2z0z3w0PPcP3732PIKZUPqDDGHoAHJggraw/4kRTIUixMAZUFB+HB1OOKLh5yBg/SASXMYSVtyYHleHPksDdTNgAT9O1GwymTYtxTWjxo8M8iTwkMg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=N/6xq2qk; arc=none smtp.client-ip=209.85.210.228 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="N/6xq2qk" Received: by mail-pf1-f228.google.com with SMTP id d2e1a72fcca58-824acdfb73dso260781b3a.2 for ; Mon, 02 Mar 2026 09:29:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772472563; x=1773077363; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qVH9fXQsVA7juGOEz+1dE42o2vyEutyAkezbvq+KUlE=; b=N/6xq2qkrtExtQSZN0gFAwb2sYl8vKAs/zM+B6ok/seGbGtB2OCmNemofhZztUXpJt sIEAUjmYxscGST04cpu18EAdgsfj5Y2VKfle4ujB4GsvadmTpcBNyKD7BQz9cq1quAoX PmAhO99AnamEypVz3tGSVogl3g6RgDZHz3SdDOkixMBy++Ut/Ls0xXEdq9Ul9GWcV83/ rAQ0+bhi1fIhQIc9CSDAuAPWMKPxFoDMvg6yce3BUM7lNKDJe3EmTr5dl0AlReIeScOv oU1vKpJTtRyG4jmD8lMXDLIfPExVl6sPb0lEe0tRkbNroMA0ixULNi86s8FkpUalHJfH v9ZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772472563; x=1773077363; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=qVH9fXQsVA7juGOEz+1dE42o2vyEutyAkezbvq+KUlE=; b=EMosUDDIoXHZKSDM2fFC0CVU17hsD0gJYbfp+UKPi81CU7f0m8r3pJU9ovdNFNsmmq i2kyhi/E8259ksrGSxuyQruYrES9WEI+Mbvn/mMhf4h45+8bZ6NwCVAhXfi6QGOFvXLw gi72dsa4XBm6URX+mzNmIGCrN6TL1+3zN4Ox9ZITGQlEtjxQMA57ZNxNEZ3KfoxSyYyV v4BhQ1yRznZqMlMxrVqn0Kiv1HFgsKS2ZAvmCevYgncdG7o4eAMNEkeOXTP8TjvBT+uh E4Cq+sXfNvz/9iBSVRGdP6WUWizQtezpcqWI5GN1CRGiaAvKriQuBZsZFwNsZmewnO+A +AfQ== X-Forwarded-Encrypted: i=1; AJvYcCXx/rn2g4XPOC9q9O+KSeQfqDhM8aYBpxMJvB0WyLquphQfC3PaFOCfNnh7rqAlu7jrjXJ30PtYrGP99C0=@vger.kernel.org X-Gm-Message-State: AOJu0YwqQXoHbqIDHJX3XXyEPdYrJ37FYB3KikwbWiDU8ngsFlgGi1JS fQHi/bEmml69YoqaK8dLvCHC130KPmVeAfjuyr8MB59QLrrD1QCMkdFPlmlq5TvUQVtMCW87OAB 7+/X7KnofUiCwAfwhEJ2IPLlW2UTvCNmXaGPE X-Gm-Gg: ATEYQzwuCO9XFdh3Wg3YwxXrNWMt7k0JEZZ0dTkEwPALGkvtayTfC82Kr2LM7rXTzWN v6syjDRI/kU4N0/AQZ9XwHQkSMXZNJSF+B6ANUU/sPPiFaOieCdqw1vIyZeV7AF8OWzn+utJ7+B FKJ6qmJsGpQdAYbI6OtkZJlr35uySy5JgW2ajrhcQ4Ee52yX1CJUmMYRNW/7HRjYx/SF913eT+c xOPLJE0lhsjFSfj8HGPgUEQDFXcdOWQhh0OaBpCykwe9E6tf3H90A3b8rwK26vUaZlX3MiMVM5t bVVmkHpEeCnWXMNl+BXVfcM7RxVQUJhF638XPG0mlf4I73eyXDjPflQfMLwpIcFhhDHPhLyfuEh fR21yMiOx6BdItzIMFK/vRQn1gdWbTbgiOvxfW0cRre2EDEf/7Ja4Og== X-Received: by 2002:a17:902:ec84:b0:2ad:c1fa:ceee with SMTP id d9443c01a7336-2ae2e4fe432mr94384075ad.7.1772472562651; Mon, 02 Mar 2026 09:29:22 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.129]) by smtp-relay.gmail.com with ESMTPS id d9443c01a7336-2adfb16ae40sm16178815ad.13.2026.03.02.09.29.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Mar 2026 09:29:22 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id C2CED3404C4; Mon, 2 Mar 2026 10:29:21 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id BED93E41FBD; Mon, 2 Mar 2026 10:29:21 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v5 2/5] io_uring: remove iopoll_queue from struct io_issue_def Date: Mon, 2 Mar 2026 10:29:11 -0700 Message-ID: <20260302172914.2488599-3-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260302172914.2488599-1-csander@purestorage.com> References: <20260302172914.2488599-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The opcode iopoll_queue flag is now redundant with REQ_F_IOPOLL. Only io_{read,write}{,_fixed}() and io_uring_cmd() set the REQ_F_IOPOLL flag, and the opcodes with these ->issue() implementations are precisely the ones that set iopoll_queue. So don't bother checking the iopoll_queue flag in io_issue_sqe(). Remove the unused flag from struct io_issue_def. Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta --- io_uring/io_uring.c | 3 +-- io_uring/opdef.c | 10 ---------- io_uring/opdef.h | 2 -- 3 files changed, 1 insertion(+), 14 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index e7f392e962bd..46f39831d27c 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1415,12 +1415,11 @@ static int io_issue_sqe(struct io_kiocb *req, unsig= ned int issue_flags) } =20 if (ret =3D=3D IOU_ISSUE_SKIP_COMPLETE) { ret =3D 0; =20 - /* If the op doesn't have a file, we're not polling for it */ - if ((req->flags & REQ_F_IOPOLL) && def->iopoll_queue) + if (req->flags & REQ_F_IOPOLL) io_iopoll_req_issued(req, issue_flags); } return ret; } =20 diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 645980fa4651..c3ef52b70811 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -65,11 +65,10 @@ const struct io_issue_def io_issue_defs[] =3D { .buffer_select =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .vectored =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_readv, .issue =3D io_read, }, @@ -80,11 +79,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollout =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .vectored =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_writev, .issue =3D io_write, }, @@ -100,11 +98,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollin =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_read_fixed, .issue =3D io_read_fixed, }, [IORING_OP_WRITE_FIXED] =3D { @@ -114,11 +111,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollout =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_write_fixed, .issue =3D io_write_fixed, }, [IORING_OP_POLL_ADD] =3D { @@ -248,11 +244,10 @@ const struct io_issue_def io_issue_defs[] =3D { .buffer_select =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_read, .issue =3D io_read, }, [IORING_OP_WRITE] =3D { @@ -262,11 +257,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollout =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_write, .issue =3D io_write, }, [IORING_OP_FADVISE] =3D { @@ -421,11 +415,10 @@ const struct io_issue_def io_issue_defs[] =3D { [IORING_OP_URING_CMD] =3D { .buffer_select =3D 1, .needs_file =3D 1, .plug =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_cmd), .prep =3D io_uring_cmd_prep, .issue =3D io_uring_cmd, }, [IORING_OP_SEND_ZC] =3D { @@ -554,11 +547,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollin =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .vectored =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_readv_fixed, .issue =3D io_read, }, @@ -569,11 +561,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollout =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .vectored =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_writev_fixed, .issue =3D io_write, }, @@ -591,11 +582,10 @@ const struct io_issue_def io_issue_defs[] =3D { [IORING_OP_URING_CMD128] =3D { .buffer_select =3D 1, .needs_file =3D 1, .plug =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .is_128 =3D 1, .async_size =3D sizeof(struct io_async_cmd), .prep =3D io_uring_cmd_prep, .issue =3D io_uring_cmd, }, diff --git a/io_uring/opdef.h b/io_uring/opdef.h index faf3955dce8b..667f981e63b0 100644 --- a/io_uring/opdef.h +++ b/io_uring/opdef.h @@ -23,12 +23,10 @@ struct io_issue_def { unsigned pollin : 1; unsigned pollout : 1; unsigned poll_exclusive : 1; /* skip auditing */ unsigned audit_skip : 1; - /* have to be put into the iopoll list */ - unsigned iopoll_queue : 1; /* vectored opcode, set if 1) vectored, and 2) handler needs to know */ unsigned vectored : 1; /* set to 1 if this opcode uses 128b sqes in a mixed sq */ unsigned is_128 : 1; =20 --=20 2.45.2 From nobody Thu Apr 2 20:26:36 2026 Received: from mail-pg1-f225.google.com (mail-pg1-f225.google.com [209.85.215.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C0743427A03 for ; Mon, 2 Mar 2026 17:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.225 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472565; cv=none; b=lldmJrYzhQiQ9EmmXVmnd8aS5TjF1z67YZ7TzZyqyfmC+5e6C1cXKqywF7e0qTOqZUyFVC4ptByz/nC47mHTWvXSGTn9zMRVRKJ4E2hZAW859fpoYHwLiI/JAHQjhLnAQ7XyqZFhHlFwVWkOmt+29UdcnKSzj0eP6jDXg4DTqUk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472565; c=relaxed/simple; bh=EHWetMLtF/7h2Jzc9WukOzl1UiPE5W/f2SGGcvcIhqM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UbzBvQAySB3JxspVquTMqK+juAF82izzLUeU/wljDYBIvwFLQsENVMR2y7ZNsuAyOnawhKXpd9EimrRXUzMweDpnUeqIQ7K+Ij81j62M9V246ems67Y7inYtGrW2w0v7X9aKNsU78Y4pl/qC2UACw5DBlu0RJlP0cHlP1lmsCXM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=IX543Idp; arc=none smtp.client-ip=209.85.215.225 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="IX543Idp" Received: by mail-pg1-f225.google.com with SMTP id 41be03b00d2f7-c6e85f76efeso313122a12.1 for ; Mon, 02 Mar 2026 09:29:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772472563; x=1773077363; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UJYfmLiCMCsowVS6yyBvopz5p0Wjd6fpqKLBR9qiyiM=; b=IX543Idpvg6IdhRShudCrNaH9AYqWbuEJe4zU+fFBSBzVZ0imoaSD5Iy+f93AJR87w JB9o397S8WwPnUL/2c6uJgCyYe5/mKIpsoe2736oKt+Wh3ekV7EkMD07P8peCVyRcEa1 aUMxj+J6Jz74Xlz2eSX8mWWIH7cWbwtEqpplg9lDr+UNyZ4GUUEvfU7SW6wc/d5/3ya0 sAngLNmvUZnjSMc/QZq8+V7anfYIZ+h7qZiHU7dCmFDTcUJZUXXo6F8cnxjadcGPzf+E WsJShp1KP9rOVQqf4+dhK3F+T0Q2VeCfReqaZOhM8bP3yTQbjWpuvPzCFIWUJQinJbEg WaNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772472563; x=1773077363; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=UJYfmLiCMCsowVS6yyBvopz5p0Wjd6fpqKLBR9qiyiM=; b=ZuI6nDqP8KRQhBW8GV3K1jZzfIJ79VEMPZ2cUmUE5J7SqQQG+smIu6t1QPeXRjQIm+ cyAxNmgiCCIkIS9QQrC4BBqv5veB6/xcXxn4PYnY6dpPaERKue1RvokYjRFIgUVYJfgg DXOh65UPCbuW8tpm2ft+ZQbO0FAU2ZvUEqCL99MeplrBZRaRBCxyF6RN1gCpoMewurBz uxIj5VJN+S5UCe/1hnW0NGR9CmEJCy2gVPa8Gd1fKjemDGOlJELmpm3l+RHeVRq20+6F GDnMuVl+904gGJ+XQJ/4GHEP8MzGzVQJwghHdlcwrxcJRf+uaeg9YSVSxY+eQtlZ4IcC n2tw== X-Forwarded-Encrypted: i=1; AJvYcCV/LS2pctr0Ztxz/tx4fFkHcvmya1fnu19H7TgJaBkLQKJbrFelPNYt5IPDLHH/K01LekWai856VttWqcc=@vger.kernel.org X-Gm-Message-State: AOJu0YzxNSVT/pxFrKkqQ4Ur9ml269L33aHPWWlvPWgPYt71pJ2Vj7At 1h3+TY/LIBiWfyXip9RMt/vHXU0QmwNvKko0Xq36rX9Ns4UvttIwTXFd29ABm9bJizBCtKJjxw2 /lM5C79GaZhxeOOBQM0wLnT14+awk9qClMmCT X-Gm-Gg: ATEYQzziwUhe6KudikzJBIUzVuaUq+W2X19fmxwwhbqEXo/8m4QmM4ZYrM3QjVIaAMN Ugg+lLoxBZQGngqrg1ELaDsV/J0WLmdz2a0SCK8Czdo9TtRxXjPFkD0UpwGB9J2j5Xfib7xKDuB hSD1cTzxBaE/3IhqKGhhkuByj8OMIX6JAafPTG86Ap1qQnBvNT/aTLFmDwO6rXJanH3cpUvx4hb fEbVY+cqT1+Voiz8w4m9tSzdSdFNQaXehPQv1ncSgMmsA7jO8ak+ZCpUMYDhuzyT7rrO2aqNEz1 Ob2ZW1GXbw32LC898d4JllN7B5+ngAvUxWFbDPySPVfMvylx5kcmEwPDZdmZ3nPvSrFi6fFWf7f 8A8ViO9gnZLXER2Gb3KLQvwhkUPf4qwlglCS0CmeM1TSw4O2VHEVunA== X-Received: by 2002:a17:90b:3c09:b0:359:8d95:4a57 with SMTP id 98e67ed59e1d1-3598d954d03mr2682124a91.6.1772472563024; Mon, 02 Mar 2026 09:29:23 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.128]) by smtp-relay.gmail.com with ESMTPS id 98e67ed59e1d1-3597c40a81asm921671a91.3.2026.03.02.09.29.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Mar 2026 09:29:23 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id F0454340506; Mon, 2 Mar 2026 10:29:21 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id EDAC2E41FBD; Mon, 2 Mar 2026 10:29:21 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v5 3/5] io_uring: count CQEs in io_iopoll_check() Date: Mon, 2 Mar 2026 10:29:12 -0700 Message-ID: <20260302172914.2488599-4-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260302172914.2488599-1-csander@purestorage.com> References: <20260302172914.2488599-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A subsequent commit will allow uring_cmds that don't use iopoll on IORING_SETUP_IOPOLL io_urings. As a result, CQEs can be posted without setting the iopoll_completed flag for a request in iopoll_list or going through task work. For example, a UBLK_U_IO_FETCH_IO_CMDS command could call io_uring_mshot_cmd_post_cqe() to directly post a CQE. The io_iopoll_check() loop currently only counts completions posted in io_do_iopoll() when determining whether the min_events threshold has been met. It also exits early if there are any existing CQEs before polling, or if any CQEs are posted while running task work. CQEs posted via io_uring_mshot_cmd_post_cqe() or other mechanisms won't be counted against min_events. Explicitly check the available CQEs in each io_iopoll_check() loop iteration to account for CQEs posted in any fashion. Signed-off-by: Caleb Sander Mateos --- io_uring/io_uring.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 46f39831d27c..b4625695bb3a 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1184,11 +1184,10 @@ __cold void io_iopoll_try_reap_events(struct io_rin= g_ctx *ctx) io_move_task_work_from_local(ctx); } =20 static int io_iopoll_check(struct io_ring_ctx *ctx, unsigned int min_event= s) { - unsigned int nr_events =3D 0; unsigned long check_cq; =20 min_events =3D min(min_events, ctx->cq_entries); =20 lockdep_assert_held(&ctx->uring_lock); @@ -1227,34 +1226,30 @@ static int io_iopoll_check(struct io_ring_ctx *ctx,= unsigned int min_events) * the poll to the issued list. Otherwise we can spin here * forever, while the workqueue is stuck trying to acquire the * very same mutex. */ if (list_empty(&ctx->iopoll_list) || io_task_work_pending(ctx)) { - u32 tail =3D ctx->cached_cq_tail; - (void) io_run_local_work_locked(ctx, min_events); =20 if (task_work_pending(current) || list_empty(&ctx->iopoll_list)) { mutex_unlock(&ctx->uring_lock); io_run_task_work(); mutex_lock(&ctx->uring_lock); } /* some requests don't go through iopoll_list */ - if (tail !=3D ctx->cached_cq_tail || list_empty(&ctx->iopoll_list)) + if (list_empty(&ctx->iopoll_list)) break; } ret =3D io_do_iopoll(ctx, !min_events); if (unlikely(ret < 0)) return ret; =20 if (task_sigpending(current)) return -EINTR; if (need_resched()) break; - - nr_events +=3D ret; - } while (nr_events < min_events); + } while (io_cqring_events(ctx) < min_events); =20 return 0; } =20 void io_req_task_complete(struct io_tw_req tw_req, io_tw_token_t tw) --=20 2.45.2 From nobody Thu Apr 2 20:26:36 2026 Received: from mail-pg1-f228.google.com (mail-pg1-f228.google.com [209.85.215.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6A6B428846 for ; Mon, 2 Mar 2026 17:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.228 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472565; cv=none; b=emKqL/bRIaiL89GZUeQVcW1M9JvgMwZO4fP/XddUTX1L4VbKwwuT4MHaCcNjiYRdbF1nGVgYceJ1JFOMBIV17IFuSEuLckR3NJtR9EsA+9eHjHg8JjgBR6rYH5E+YRHkwc4eikOnd9blgS+Twu45zMcNfIfM0ocr4cotVMRD9SU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472565; c=relaxed/simple; bh=NBvNotESp7kUasR2rqxc9z9U7+RvFpA7PAQQGkHDOZA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=R1QVVrtQ4hpCkTUXD7pxdJ2/JDi1H2De6ibTDQfBv5ACGB7E9EEamUXHZCrCwbw/7xbDtI1OUcBjSAywDPKqidU2uYFfnS8gdBlpXeuA3gcUhmM07FJayymNBuk0RLqZyvhrw4eIFUKbG3Rzl5CHHnQzvyPWRqX/dMKB0k9B4WM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=dPY+CUXn; arc=none smtp.client-ip=209.85.215.228 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="dPY+CUXn" Received: by mail-pg1-f228.google.com with SMTP id 41be03b00d2f7-c70c1da52c4so332173a12.2 for ; Mon, 02 Mar 2026 09:29:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772472563; x=1773077363; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YA4oP39D1EwOno4AkPegcttVQHiaILWd3dC9XNEopg8=; b=dPY+CUXnAnAP93i0m2sogrKJdOcv3PfhHRonzPORJvAXbcJSuuK28VFLW3HA0+Va0v J9cs/lNv9vtrZfHbQNNAvmpcgYHE5+S5IJZiQFOGSsPwejH4TI6Gs8OUdQpF6swm5J2N FQK5ASiNUXjew4TwMHoeHEHHUSJXsb9pOckSo+0nPzA6d0ukoOhS8Z5jnlwqkp/vqKbB maBr7q85RYZXyBnUkMn7/j97WvzntNcNBvAFr51rhFGZLCyJT9MQOhVX8wiLuAYUdDkF CTfm/PJO9/TGX2s29+JjwYef1CYcWWfps5vsVe0owNBj2/7973i0BubNNWl/VdHr5FQn 0Byw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772472563; x=1773077363; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YA4oP39D1EwOno4AkPegcttVQHiaILWd3dC9XNEopg8=; b=awKhfyyV6TE3vCBUaMOdQk93sW8d1ps8RuPYVMK1sJiBKLnAU9dopPEkEqf7W4Hwk8 +iAEXYmMIr1kZbpMuXNHsAQSjNw6PmwJFBLfjubWEwAj79VZNwBVF+7D4/iSAD6tySYz 2GukKxhdIl7GARYR5wjkBXXX/WJxmBcFY9Pn+TnNxWA8BxNegPZ5ID+TfeqEoeK+/cY1 Ds28VgrZ04LQirPUQvhmDjJpa8eUHYEoedATvA17IuSklq22l58tGTlI5Z2QR3nQZD/x XJvu8dp4JnUN7UGhsJja60Va9PxGAYSeFVwEmQFFsILmXrTNmcFzVg4NW/ESrvKw/shb U2uQ== X-Forwarded-Encrypted: i=1; AJvYcCUaRFbRjtjXefTbn855KbPMxyIBjVVmuISeorvEwYiKLHTVLZMp/2w9f8DlwljoUqCHdoAGLEZ8MtwtsR8=@vger.kernel.org X-Gm-Message-State: AOJu0YzN+PCckaKjb1taDxNBNou+WpwXPm1ifZanHdqx+zdVEyXdU72y q5f1cwXw6CXbpTCx75VBEDb0E8NSrUhc/v8/QLRl+P5juznkI3YKZKuPhWwiRP8dIBomYqBv99U OH9Z32tkAHddpczYB6mERlUS0Jb+LXdG8TLkGYeOfEvN5xZoEb+5d X-Gm-Gg: ATEYQzxRiKSkg56Zjyljd0rCVk2CutZ6QhJfn7Anq/I+2r5YfwH+jWYdkZKTav5CRC0 /nj3f9QUV8NDT2LU03LAyUdKV0H+4401WJE2+ISp0iwPdhT3o++TnzOlYLPGymuCk+tFfX+DC+r nVrrHIPg5228Db5W8mZiGXRUxrlLpZ98fb1X5lvR6RZUdVolnSPmnyye56Qyi0tgOr8h+vyBFmA i1qDOv54Rgu6mxUnLj8FPQS1f2J9sbJK8VpvLpxH+y+AFCnb4ndNeLmoLHZcU29/r/LiulRDpr4 KePNJRVdbZ+aTgydQyDiy0UvECszpXFuF3SJcvi0JScMeVcVKrPiRmBKx90W3jDH5K1Acvy6iQg 6itBYvw2PFcWJGhnUXAQJEbte3Kf+BaAhlTQhlJ4= X-Received: by 2002:a17:90b:3949:b0:359:95ec:5a2f with SMTP id 98e67ed59e1d1-35995ec5c72mr1230896a91.6.1772472563254; Mon, 02 Mar 2026 09:29:23 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([2620:125:9017:12:36:3:5:0]) by smtp-relay.gmail.com with ESMTPS id 98e67ed59e1d1-359939787dasm287026a91.1.2026.03.02.09.29.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Mar 2026 09:29:23 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 2FE0A340199; Mon, 2 Mar 2026 10:29:22 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id 2C3B1E41FBD; Mon, 2 Mar 2026 10:29:22 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v5 4/5] io_uring/uring_cmd: allow non-iopoll cmds with IORING_SETUP_IOPOLL Date: Mon, 2 Mar 2026 10:29:13 -0700 Message-ID: <20260302172914.2488599-5-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260302172914.2488599-1-csander@purestorage.com> References: <20260302172914.2488599-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, creating an io_uring with IORING_SETUP_IOPOLL requires all requests issued to it to support iopoll. This prevents, for example, using ublk zero-copy together with IORING_SETUP_IOPOLL, as ublk zero-copy buffer registrations are performed using a uring_cmd. There's no technical reason why these non-iopoll uring_cmds can't be supported. They will either complete synchronously or via an external mechanism that calls io_uring_cmd_done(), io_uring_cmd_post_mshot_cqe32(), or io_uring_mshot_cmd_post_cqe(), so they don't need to be polled. Allow uring_cmd requests to be issued to IORING_SETUP_IOPOLL io_urings even if their files don't implement ->uring_cmd_iopoll(). For these uring_cmd requests, skip initializing struct io_kiocb's iopoll fields, don't set REQ_F_IOPOLL, and don't set IO_URING_F_IOPOLL in issue_flags. Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta --- io_uring/uring_cmd.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index b651c63f6e20..7b25dcd9d05f 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -255,13 +255,11 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int i= ssue_flags) issue_flags |=3D IO_URING_F_SQE128; if (ctx->flags & (IORING_SETUP_CQE32 | IORING_SETUP_CQE_MIXED)) issue_flags |=3D IO_URING_F_CQE32; if (io_is_compat(ctx)) issue_flags |=3D IO_URING_F_COMPAT; - if (ctx->flags & IORING_SETUP_IOPOLL) { - if (!file->f_op->uring_cmd_iopoll) - return -EOPNOTSUPP; + if (ctx->flags & IORING_SETUP_IOPOLL && file->f_op->uring_cmd_iopoll) { req->flags |=3D REQ_F_IOPOLL; issue_flags |=3D IO_URING_F_IOPOLL; req->iopoll_completed =3D 0; if (ctx->flags & IORING_SETUP_HYBRID_IOPOLL) { /* make sure every req only blocks once */ --=20 2.45.2 From nobody Thu Apr 2 20:26:36 2026 Received: from mail-dl1-f99.google.com (mail-dl1-f99.google.com [74.125.82.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C134541B37F for ; Mon, 2 Mar 2026 17:29:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.82.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472565; cv=none; b=nEjCspax8JnuPry7dhAnLsV8pmGlx7DHbhH45qMqT12pwXzy7rhFUWDBPL/s4Ry8P9EXmq402MOdDlrW0xvZ258D6Svp44ewjxN78Ia13KJy9OTQwwm9bY0LxT0IEqnq7DjCF7FHRaGBksGdLmZ8vF3/iovXF53WRD5ilct9p2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772472565; c=relaxed/simple; bh=leoJQLYUVaLoPMfpnaJ83Ki6TjRT6ZQ1hoz2AIfe5rc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pMpHvrX71uZjDWlXRPwDWFODFuGWV/LtDbXG8aIpMYlmrZCRObYD1fWLDr5rLXNgvDcNbRXEEXkhJQoFkmVjSTs5mswyqAQqrzakCnhxylew7u6sSmJ4aGDiZ9HkwZs9BfylCOx2xyT1TT8pa83uMVhRYc/jQ2PtlreOy5V0zrY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=cczm6zxJ; arc=none smtp.client-ip=74.125.82.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="cczm6zxJ" Received: by mail-dl1-f99.google.com with SMTP id a92af1059eb24-1273882d901so276076c88.1 for ; Mon, 02 Mar 2026 09:29:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772472564; x=1773077364; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bC/PSnv7yOwEPTd/AMtRMqWpAAmzjzQLzKujNS/xPPY=; b=cczm6zxJiC0VuqWJSZE9D7jhV/ZxSeYWcVxp3OioQllDMv+9A2SNiLHyGnkQRzreUe /WHuBhV5O5Zv/ezYWvJ6utuwgjOAooyd50JwIGI1aogjn84so63Ji9EBasgiGMp00O4G A/E9Yt7YF4pWHHB6umnNf+UEgvyxf1H9RxCbpKW3TkryCSn1GVETegI37R0yc/O1bWLc Fz0Lq4zh3t/OBkQAxHNBjyTF/Gju2mDwY5iB7zy8ZZuTncwoLPoHmvuX4DnF0bO799QM eiJpMv3wP6o4TlraLvko+nc0YrrNKe3IVChXpPB8B6alnBJ+3rn59VCE8A477nxnWckL 4gtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772472564; x=1773077364; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=bC/PSnv7yOwEPTd/AMtRMqWpAAmzjzQLzKujNS/xPPY=; b=xAJwyq+S1fZYec+7m3zCwy8FLq6cbDHzSVdM4pTxEId8yxMvSd7UsWXQIFB01G/Z4z iixr5Vu1tbHysk40gsIPsGF/FyNJy8Yltac5Py8GnC/Z3TSG5Z9ykZ0G75G6bnK6FwJI lkZYVCaspf+vWJpxLMpqbffLsH+PhEbYolFFkTuBHMoESyJltldo21xsJQgq4E0s9Eug umQptiMLWWOPplzWDTcg/KN15GQ7jwxbUaBmovJSGynTIbWioZyvfd0Ic9/Tvp1jNNKE 6oKcTk2MVgJLkzx/TsRBONcGoQ/V9eVbA/GI/kkZJ5sT+lGia/fJIALlvOP9g9pp3La7 ND2w== X-Forwarded-Encrypted: i=1; AJvYcCVsb9ijuRWEETqngFtv7oHNDDCvOmnORGrvQ7zyxDH4+znx4W9ImE2nM1Gzc5EIiETKVPURzX5G5ikdFsk=@vger.kernel.org X-Gm-Message-State: AOJu0Yzy7DYnjCstIJxQzDiXOFHW8E/5oT65fp0CL5LbjvYeEbTqK8q9 BjTwQxtRahb/twXYpvNzMD2m5J04a+2tFQ/RVkjEsQ8DQg+mvLVrOYw+hZqzOyPi/NXuSgOPt+G FmhhAZxIEnmOwNHlo9cwf8mw/cZ0g6eudcdFk X-Gm-Gg: ATEYQzy+5YYNnK7JKCFjAudsImhrjMEYrlUOiU0N20qdlap+rGEFEZxmJ2kCAxMawyJ JUHrcYpWkXJeM722CDsBnjwhMjVrvLn9XFugR5oo/J3+gnldUS3j5caMvCBiL4dHJZKNfYWEliQ 9wh+RZ8dUPZiwXpT/GXHsgegttak8Q2s0LwP7J9GLFZK7095Z7fMSaZoiBD4pGdV5VqbPzv1wY3 JXQwEYwcIrGT3EcyF0igMZ+LaY1bnwQToIHKiukJLZ3CcJCJR/3UW3ja/g8PvKzltL17INtpTW/ s1j/gCMVvTeROZkKKDcoA+S2tX8/1d4SBimjIE5EB1EU4G+X+J8rmTp/Yqec8//dymk5R4WTW5D DPV/yNacd8D8KI90kuEbacgbdgNAyTsYQJTxv9cvoEOz4PIDPreN43w== X-Received: by 2002:a05:7300:724c:b0:2bd:db75:c28b with SMTP id 5a478bee46e88-2bde1e90dd9mr2349446eec.7.1772472563561; Mon, 02 Mar 2026 09:29:23 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.128]) by smtp-relay.gmail.com with ESMTPS id a92af1059eb24-12789a20cd1sm2213499c88.6.2026.03.02.09.29.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Mar 2026 09:29:23 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 5CDAE3405ED; Mon, 2 Mar 2026 10:29:22 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id 5A1E8E41FBD; Mon, 2 Mar 2026 10:29:22 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v5 5/5] nvme: remove nvme_dev_uring_cmd() IO_URING_F_IOPOLL check Date: Mon, 2 Mar 2026 10:29:14 -0700 Message-ID: <20260302172914.2488599-6-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260302172914.2488599-1-csander@purestorage.com> References: <20260302172914.2488599-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" nvme_dev_uring_cmd() is part of struct file_operations nvme_dev_fops, which doesn't implement ->uring_cmd_iopoll(). So it won't be called with issue_flags that include IO_URING_F_IOPOLL. Drop the unnecessary IO_URING_F_IOPOLL check in nvme_dev_uring_cmd(). Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta --- drivers/nvme/host/ioctl.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 8844bbd39515..9597a87cf05d 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -784,14 +784,10 @@ int nvme_ns_head_chr_uring_cmd(struct io_uring_cmd *i= oucmd, int nvme_dev_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_fla= gs) { struct nvme_ctrl *ctrl =3D ioucmd->file->private_data; int ret; =20 - /* IOPOLL not supported yet */ - if (issue_flags & IO_URING_F_IOPOLL) - return -EOPNOTSUPP; - ret =3D nvme_uring_cmd_checks(issue_flags); if (ret) return ret; =20 switch (ioucmd->cmd_op) { --=20 2.45.2