From nobody Sat Apr 18 11:08:29 2026 Received: from mail-vk1-f229.google.com (mail-vk1-f229.google.com [209.85.221.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 770424779B6 for ; Fri, 27 Feb 2026 22:35:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.229 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231711; cv=none; b=H+3rdufsS0xeXY/bGmTE2CBg6NlVhiOeH/z89rPUWyKydhCf5AxUaINoVEc7F/TPcRHGfuN30JJm1aWXSDXqlGCywdymeVENrAjW4Khe42RObn11qm9vPkRTCQ9Ud+JVZZ5QIr0X7/l4w/tF+WDEZyxL9JHu9H3Le9zejMEBgYI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231711; c=relaxed/simple; bh=jitPk8dZO7gURW0hp4tgB7XQfWHqvV5p0qzZsigRrzI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kSMxoe+17U2wrIQpifnJsbA0GpZ7FtmbO9r0A29kUAB2TmWnhPl430wHgbu1NM8Ah/QDxgmEr5WbkiLTW4glMpU+JIclYJJJ+2mgCRtN3NdD9r0Zkh97zn0JQ7pFRHkLp8SgsNc0wHFFRiFZNL9BUYySpipXgzpyO2rq7WBeBPo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=Wwd7h+dc; arc=none smtp.client-ip=209.85.221.229 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="Wwd7h+dc" Received: by mail-vk1-f229.google.com with SMTP id 71dfb90a1353d-56a96ef0feaso131074e0c.2 for ; Fri, 27 Feb 2026 14:35:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772231708; x=1772836508; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FOsPp0sNuMqBvgksLLC7w2ENUHuNEhH517t3YeRJRmc=; b=Wwd7h+dcR3SndiU9NtECqJXQH7Ii2M2V8rpOQdeLln8zwaKKyUfTK0wy6fDvstLCcg lzjNaUglnhHRcCOpVYmeUqd7hc+DcLt/M88K1C4B6uexIsZ9izKApoyS65Yzz8Uv9d7l Vs/bjQeNqtDOGd5gsCMbJMCZs5AwR1b9OPCdpqbpveOTQ4U20V/tNVF3kGwRUwOCSq6k EQFsPOMNM89Hp8ypUWnQWHCKotm53BEY/qtNLh1Z9abAR3WKgGf85KOU6RLcnbsltLO4 K7y7RLUsVM9tEr/mL+4tvb++UgZ3klYEVUGhewFe5H5gYFKQ0UU74ZonHYUOJUCblZ+s Fawg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772231708; x=1772836508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=FOsPp0sNuMqBvgksLLC7w2ENUHuNEhH517t3YeRJRmc=; b=vblIvD34BnpOMnBEMZOFbdbNvpFQg//h6w7zP411ESIkKGqg8gneG5A8XuTB6i1uU9 NVU9MuV+mcZgoEfs9yvyqVltOnSeX0jdeagqC34br9SjXfEFavhueHja7hhtWx6w0Bkv 5tMjWjvRYm4LMB7uucmfaopC00DAq3fvqsF+mBTDzQV4E1cqQKRADaRlMq0O+BYM/BSv AzMT+brP481h9VML+XqmvTYIdLtguScRqX8t1Xj5N7j1m5Aevt4vvnhcZei3jhsV1NDH EBtznFccT0cMkiP0jMxp66bRRW64yMXrnUbyUElYNaUaVghEr8kH3HSlBQKMJmz4k4uJ uLzQ== X-Forwarded-Encrypted: i=1; AJvYcCWFXrdyiD2ex9cir2SGC32OsXjTHkQCO8gCu+/RrJ05K8DvMD9z7Mf3egg1GWEW3BHjK7e///+S3ptqy2Q=@vger.kernel.org X-Gm-Message-State: AOJu0Yz6f3GJSey//xpNmIUICFAoZQ29/S43MotS3BogZteOCcnylwpw /zUFg2852SUKjOHfRTES5oUVkTvl40l+0r04Gyf/Wces8qVkVBV65x6DQjdGuqdrCF7NBFIYS5O qg4hrtjiycGd9eMwmZCY8nkrFQ54fCmfRx3oa3jbe+Bx9pUFhCdJR X-Gm-Gg: ATEYQzwKtrRedL60TKI/gDdyQWZqrg8OmwtcD+MzkDV1vcvHqICKRcAx0S0TXWCo+N4 ZNUJh6kjyw9464U3TZCrJYINWaXK4sxVhvgLjRujdB50uQgzVBVQD8xQRmxxYqDaTiQFuwIz/4S v4V+sOgQjYFc6pfN/yhbkyUuc0eaAFDlk73FGImQN1rL1mCaTre+bz3GtCQn6PBZf1vqvh7h9Fa FXlptSAwHS97sbdL34i7ukEl4/VPI3bLQb1Sbv1iltiK/FEo9J5s43z+DM9K1MZQ2QjQoJMC5o1 XLs/CHA15kSEkf+P5eFzEjsvnW42FpHQ5zcgSRRkPRCXZQxjOyUdpTGVU1+0Bz9zX6LqAWwXTsT iow3qyVYyXbi2J4U8zhJTd8u6WpcFRvvDUlQXbpY= X-Received: by 2002:a05:6122:6217:b0:566:3608:f8db with SMTP id 71dfb90a1353d-56aa0a96612mr1226798e0c.2.1772231708429; Fri, 27 Feb 2026 14:35:08 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([2620:125:9017:12:36:3:5:0]) by smtp-relay.gmail.com with ESMTPS id 71dfb90a1353d-56a91e287fbsm859123e0c.3.2026.02.27.14.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Feb 2026 14:35:08 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id B6F3C340B96; Fri, 27 Feb 2026 15:35:06 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id B285EE420D8; Fri, 27 Feb 2026 15:35:06 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v4 1/5] io_uring: add REQ_F_IOPOLL Date: Fri, 27 Feb 2026 15:34:59 -0700 Message-ID: <20260227223504.1162421-2-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260227223504.1162421-1-csander@purestorage.com> References: <20260227223504.1162421-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A subsequent commit will allow uring_cmds to commands that don't implement ->uring_cmd_iopoll() to be issued to IORING_SETUP_IOPOLL io_urings. This means the ctx's IORING_SETUP_IOPOLL flag isn't sufficient to determine whether a given request needs to be iopolled. Introduce a request flag REQ_F_IOPOLL set in ->issue() if a request needs to be iopolled to completion. Set the flag in io_rw_init_file() and io_uring_cmd() for requests issued to IORING_SETUP_IOPOLL ctxs. Use the request flag instead of IORING_SETUP_IOPOLL in places dealing with a specific request. A future possibility would be to add an option to enable/disable iopoll in the io_uring SQE instead of determining it from IORING_SETUP_IOPOLL. Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta --- include/linux/io_uring_types.h | 3 +++ io_uring/io_uring.c | 9 ++++----- io_uring/rw.c | 11 ++++++----- io_uring/uring_cmd.c | 5 +++-- 4 files changed, 16 insertions(+), 12 deletions(-) diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index 3e4a82a6f817..d74b2a8c7305 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -541,10 +541,11 @@ enum { REQ_F_BUFFERS_COMMIT_BIT, REQ_F_BUF_NODE_BIT, REQ_F_HAS_METADATA_BIT, REQ_F_IMPORT_BUFFER_BIT, REQ_F_SQE_COPIED_BIT, + REQ_F_IOPOLL_BIT, =20 /* not a real bit, just to check we're not overflowing the space */ __REQ_F_LAST_BIT, }; =20 @@ -632,10 +633,12 @@ enum { * For SEND_ZC, whether to import buffers (i.e. the first issue). */ REQ_F_IMPORT_BUFFER =3D IO_REQ_FLAG(REQ_F_IMPORT_BUFFER_BIT), /* ->sqe_copy() has been called, if necessary */ REQ_F_SQE_COPIED =3D IO_REQ_FLAG(REQ_F_SQE_COPIED_BIT), + /* request must be iopolled to completion (set in ->issue()) */ + REQ_F_IOPOLL =3D IO_REQ_FLAG(REQ_F_IOPOLL_BIT), }; =20 struct io_tw_req { struct io_kiocb *req; }; diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index aa95703165f1..e7f392e962bd 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -354,11 +354,10 @@ static struct io_kiocb *__io_prep_linked_timeout(stru= ct io_kiocb *req) } =20 static void io_prep_async_work(struct io_kiocb *req) { const struct io_issue_def *def =3D &io_issue_defs[req->opcode]; - struct io_ring_ctx *ctx =3D req->ctx; =20 if (!(req->flags & REQ_F_CREDS)) { req->flags |=3D REQ_F_CREDS; req->creds =3D get_current_cred(); } @@ -376,11 +375,11 @@ static void io_prep_async_work(struct io_kiocb *req) =20 /* don't serialize this request if the fs doesn't need it */ if (should_hash && (req->file->f_flags & O_DIRECT) && (req->file->f_op->fop_flags & FOP_DIO_PARALLEL_WRITE)) should_hash =3D false; - if (should_hash || (ctx->flags & IORING_SETUP_IOPOLL)) + if (should_hash || (req->flags & REQ_F_IOPOLL)) io_wq_hash_work(&req->work, file_inode(req->file)); } else if (!req->file || !S_ISBLK(file_inode(req->file)->i_mode)) { if (def->unbound_nonreg_file) atomic_or(IO_WQ_WORK_UNBOUND, &req->work.flags); } @@ -1417,11 +1416,11 @@ static int io_issue_sqe(struct io_kiocb *req, unsig= ned int issue_flags) =20 if (ret =3D=3D IOU_ISSUE_SKIP_COMPLETE) { ret =3D 0; =20 /* If the op doesn't have a file, we're not polling for it */ - if ((req->ctx->flags & IORING_SETUP_IOPOLL) && def->iopoll_queue) + if ((req->flags & REQ_F_IOPOLL) && def->iopoll_queue) io_iopoll_req_issued(req, issue_flags); } return ret; } =20 @@ -1433,11 +1432,11 @@ int io_poll_issue(struct io_kiocb *req, io_tw_token= _t tw) int ret; =20 io_tw_lock(req->ctx, tw); =20 WARN_ON_ONCE(!req->file); - if (WARN_ON_ONCE(req->ctx->flags & IORING_SETUP_IOPOLL)) + if (WARN_ON_ONCE(req->flags & REQ_F_IOPOLL)) return -EFAULT; =20 ret =3D __io_issue_sqe(req, issue_flags, &io_issue_defs[req->opcode]); =20 WARN_ON_ONCE(ret =3D=3D IOU_ISSUE_SKIP_COMPLETE); @@ -1531,11 +1530,11 @@ void io_wq_submit_work(struct io_wq_work *work) * We can get EAGAIN for iopolled IO even though we're * forcing a sync submission from here, since we can't * wait for request slots on the block side. */ if (!needs_poll) { - if (!(req->ctx->flags & IORING_SETUP_IOPOLL)) + if (!(req->flags & REQ_F_IOPOLL)) break; if (io_wq_worker_stopped()) break; cond_resched(); continue; diff --git a/io_uring/rw.c b/io_uring/rw.c index 1a5f262734e8..3bdb9914e673 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -502,11 +502,11 @@ static bool io_rw_should_reissue(struct io_kiocb *req) struct io_ring_ctx *ctx =3D req->ctx; =20 if (!S_ISBLK(mode) && !S_ISREG(mode)) return false; if ((req->flags & REQ_F_NOWAIT) || (io_wq_current_is_worker() && - !(ctx->flags & IORING_SETUP_IOPOLL))) + !(req->flags & REQ_F_IOPOLL))) return false; /* * If ref is dying, we might be running poll reap from the exit work. * Don't attempt to reissue from that path, just let it fail with * -EAGAIN. @@ -638,11 +638,11 @@ static inline void io_rw_done(struct io_kiocb *req, s= size_t ret) ret =3D -EINTR; break; } } =20 - if (req->ctx->flags & IORING_SETUP_IOPOLL) + if (req->flags & REQ_F_IOPOLL) io_complete_rw_iopoll(&rw->kiocb, ret); else io_complete_rw(&rw->kiocb, ret); } =20 @@ -652,11 +652,11 @@ static int kiocb_done(struct io_kiocb *req, ssize_t r= et, struct io_rw *rw =3D io_kiocb_to_cmd(req, struct io_rw); unsigned final_ret =3D io_fixup_rw_res(req, ret); =20 if (ret >=3D 0 && req->flags & REQ_F_CUR_POS) req->file->f_pos =3D rw->kiocb.ki_pos; - if (ret >=3D 0 && !(req->ctx->flags & IORING_SETUP_IOPOLL)) { + if (ret >=3D 0 && !(req->flags & REQ_F_IOPOLL)) { u32 cflags =3D 0; =20 __io_complete_rw_common(req, ret); /* * Safe to call io_end from here as we're inline @@ -874,10 +874,11 @@ static int io_rw_init_file(struct io_kiocb *req, fmod= e_t mode, int rw_type) req->flags |=3D REQ_F_NOWAIT; =20 if (ctx->flags & IORING_SETUP_IOPOLL) { if (!(kiocb->ki_flags & IOCB_DIRECT) || !file->f_op->iopoll) return -EOPNOTSUPP; + req->flags |=3D REQ_F_IOPOLL; kiocb->private =3D NULL; kiocb->ki_flags |=3D IOCB_HIPRI; req->iopoll_completed =3D 0; if (ctx->flags & IORING_SETUP_HYBRID_IOPOLL) { /* make sure every req only blocks once*/ @@ -961,11 +962,11 @@ static int __io_read(struct io_kiocb *req, struct io_= br_sel *sel, if (ret =3D=3D -EAGAIN) { /* If we can poll, just do that. */ if (io_file_can_poll(req)) return -EAGAIN; /* IOPOLL retry should happen for io-wq threads */ - if (!force_nonblock && !(req->ctx->flags & IORING_SETUP_IOPOLL)) + if (!force_nonblock && !(req->flags & REQ_F_IOPOLL)) goto done; /* no retry on NONBLOCK nor RWF_NOWAIT */ if (req->flags & REQ_F_NOWAIT) goto done; ret =3D 0; @@ -1186,11 +1187,11 @@ int io_write(struct io_kiocb *req, unsigned int iss= ue_flags) /* no retry on NONBLOCK nor RWF_NOWAIT */ if (ret2 =3D=3D -EAGAIN && (req->flags & REQ_F_NOWAIT)) goto done; if (!force_nonblock || ret2 !=3D -EAGAIN) { /* IOPOLL retry should happen for io-wq threads */ - if (ret2 =3D=3D -EAGAIN && (req->ctx->flags & IORING_SETUP_IOPOLL)) + if (ret2 =3D=3D -EAGAIN && (req->flags & REQ_F_IOPOLL)) goto ret_eagain; =20 if (ret2 !=3D req->cqe.res && ret2 >=3D 0 && need_complete_io(req)) { trace_io_uring_short_write(req->ctx, kiocb->ki_pos - ret2, req->cqe.res, ret2); diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index ee7b49f47cb5..b651c63f6e20 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -108,11 +108,11 @@ void io_uring_cmd_mark_cancelable(struct io_uring_cmd= *cmd, * Doing cancelations on IOPOLL requests are not supported. Both * because they can't get canceled in the block stack, but also * because iopoll completion data overlaps with the hash_node used * for tracking. */ - if (ctx->flags & IORING_SETUP_IOPOLL) + if (req->flags & REQ_F_IOPOLL) return; =20 if (!(cmd->flags & IORING_URING_CMD_CANCELABLE)) { cmd->flags |=3D IORING_URING_CMD_CANCELABLE; io_ring_submit_lock(ctx, issue_flags); @@ -165,11 +165,11 @@ void __io_uring_cmd_done(struct io_uring_cmd *ioucmd,= s32 ret, u64 res2, if (req->ctx->flags & IORING_SETUP_CQE_MIXED) req->cqe.flags |=3D IORING_CQE_F_32; io_req_set_cqe32_extra(req, res2, 0); } io_req_uring_cleanup(req, issue_flags); - if (req->ctx->flags & IORING_SETUP_IOPOLL) { + if (req->flags & REQ_F_IOPOLL) { /* order with io_iopoll_req_issued() checking ->iopoll_complete */ smp_store_release(&req->iopoll_completed, 1); } else if (issue_flags & IO_URING_F_COMPLETE_DEFER) { if (WARN_ON_ONCE(issue_flags & IO_URING_F_UNLOCKED)) return; @@ -258,10 +258,11 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int i= ssue_flags) if (io_is_compat(ctx)) issue_flags |=3D IO_URING_F_COMPAT; if (ctx->flags & IORING_SETUP_IOPOLL) { if (!file->f_op->uring_cmd_iopoll) return -EOPNOTSUPP; + req->flags |=3D REQ_F_IOPOLL; issue_flags |=3D IO_URING_F_IOPOLL; req->iopoll_completed =3D 0; if (ctx->flags & IORING_SETUP_HYBRID_IOPOLL) { /* make sure every req only blocks once */ req->flags &=3D ~REQ_F_IOPOLL_STATE; --=20 2.45.2 From nobody Sat Apr 18 11:08:29 2026 Received: from mail-pl1-f227.google.com (mail-pl1-f227.google.com [209.85.214.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E27C33859E3 for ; Fri, 27 Feb 2026 22:35:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.227 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231710; cv=none; b=D1Tz0f4fzMWThH+PNPWG3PFPdf7+es9C5/OkgDgILWVib0xIZh4YYTUA88CclewG7T7kGEHNWJ0Oy2fQKvP7SzH+ODfTN4lIkfQOFrJnEi3D+MDn5ccP6IMnM9iEq4BiqiYUql3mSGxlRGG5uc2p890DOfOkhNx5zfKPJ/HQ1ck= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231710; c=relaxed/simple; bh=mZRg9aJ/vT9KGxR37N9gh9XV0GFWpQmAWSI7PzYmIhE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T9/8fsQi5Ia4veCrqhpMxFfmq87MNZwb3dRiFM5c2qxfoQppFkkhaquwe/z+gGRm0rRsyWOu1+AvojGwRErIp/tietjltpN4/OE1v76tumOo69lAbnhk1u16CEPvluzEUs8fXeMQi+XhG5eYxJPDRsnNyss+HGVXJnGIf+eqtNA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=OPb3jkTm; arc=none smtp.client-ip=209.85.214.227 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="OPb3jkTm" Received: by mail-pl1-f227.google.com with SMTP id d9443c01a7336-2ae3a007bd1so163095ad.2 for ; Fri, 27 Feb 2026 14:35:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772231708; x=1772836508; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qVH9fXQsVA7juGOEz+1dE42o2vyEutyAkezbvq+KUlE=; b=OPb3jkTmEXXLtcPxCUrNGX64Mbsybraq0N6cFAJ18tF37G2/MXJV/9V12QETppXROe rHKH7JrhEeM7tVBMqVSdkfLIm7xKEkVd2V+eTIlPPO8SNGr0YXvpGoqljx9vkkLhOSn6 b+Z1/GjTYhrwRr+k+SK1bn05M7R2O+e93YSScgudgXQ/U/F56ScmqE2WfJZS1WCZX+uH TTHH+QpQLnDsz5+FZSujlKXC31LbbwxHxxrf+j7wFnafcESYUG2yHpa+eFa5DP3BCNCj NWN5I+Y/f8lFPojDZuIvm/1PsW23wg4WCvK7PdCl+8e3xDPikbjn17WggLcmuodAK1l0 2tAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772231708; x=1772836508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=qVH9fXQsVA7juGOEz+1dE42o2vyEutyAkezbvq+KUlE=; b=VsGfB89k1ODhjZPFrBmftYgXwUHfq76fsBltGCGYGQqga0NxWTxSEnwZoIPQ4P6elr IY8Xc0DdWob+IdB/aTMLTg9BdDDhF7Aov90P7A59V7w+7uego/fk/lSD/7N6QTc4Y8TT rz1rVqVWeq3AdM/V785mS23wgT/Sxm6dCYWWdsWIauzjMeo891U3vDeKnR9yUZz/VtPp SLc+Tb91vX3iCzOouyt02zVblLmZSlwhk33WCUgl8JVYgvhpSR47yMJtjkq+VWYB0JVQ R2rxvn3HX+iZQNRAVMIp1ITOjFy0z1U/fTChnDzStfdo21tlqU1hZUmM3PaJ7D4osCXG 1u4A== X-Forwarded-Encrypted: i=1; AJvYcCUadFcjuLaKDfNwprLbmh6dzfcHwPvPQmK+O62ihKNnAZRy7gKROHyd1IXM6bEQROss6BU1IiWOnhtn4bw=@vger.kernel.org X-Gm-Message-State: AOJu0YzhsxMFrVdBkqPkRRgGSqn9lJ6NxiAtLcXg0hV3OGwS5lq0bBu7 LUpwIAOlNfKEMigN9KyDAacUnSi/ORt00sTgw1Kczsy0+aJrufu84y4eK2u1pYAbt2jJLGlJwMT GKbNVSPV2B5ETZaY7aUjNOHF0fDoxG+dC9Ku5 X-Gm-Gg: ATEYQzyLGpdCU5vUVVAsR2Y3tH07uPpP539q+a7bP7sgnIjQrxI6Ne6P9W9JxVIrvE3 lgTp5j9/fTHg4YhxmZFn7EvzDzFjU67M2cSNosuEhpTe8+MJgyrUyVxxCLzPA91CD8rGl0Ejonc J800uDcY9kDF+8HfueRApYnmK+07oxDdbWwBW9oH5oYTGfPByKGkkKt/4kJAGCsrVcEL5bx6J7C cRrnfwYaVwXxTlgETqLxvbWvctMxMmROunhviX3qPGzECDoLAQc4dM+Ky90EfO9Pmkli+WKwMgh N5gJIWIJKi4BhIW9fVca9sZP9X23MdAD+RsllbRk6ZQFWFTN0e7CEgNU/42AvGSU1w/A6J0imdk a/1SvPrVWaSDcE1ewk0d6uZwVB/MA4MlvFfJfsQr0BxSM3SUp03BKbA== X-Received: by 2002:a17:902:e847:b0:2a7:cbe3:a6e3 with SMTP id d9443c01a7336-2ae2e3e0d9bmr31622965ad.2.1772231708194; Fri, 27 Feb 2026 14:35:08 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.129]) by smtp-relay.gmail.com with ESMTPS id d9443c01a7336-2adfb5eec86sm7901965ad.41.2026.02.27.14.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Feb 2026 14:35:08 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id E9C1534179D; Fri, 27 Feb 2026 15:35:06 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id E5700E420D8; Fri, 27 Feb 2026 15:35:06 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v4 2/5] io_uring: remove iopoll_queue from struct io_issue_def Date: Fri, 27 Feb 2026 15:35:00 -0700 Message-ID: <20260227223504.1162421-3-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260227223504.1162421-1-csander@purestorage.com> References: <20260227223504.1162421-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The opcode iopoll_queue flag is now redundant with REQ_F_IOPOLL. Only io_{read,write}{,_fixed}() and io_uring_cmd() set the REQ_F_IOPOLL flag, and the opcodes with these ->issue() implementations are precisely the ones that set iopoll_queue. So don't bother checking the iopoll_queue flag in io_issue_sqe(). Remove the unused flag from struct io_issue_def. Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta --- io_uring/io_uring.c | 3 +-- io_uring/opdef.c | 10 ---------- io_uring/opdef.h | 2 -- 3 files changed, 1 insertion(+), 14 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index e7f392e962bd..46f39831d27c 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1415,12 +1415,11 @@ static int io_issue_sqe(struct io_kiocb *req, unsig= ned int issue_flags) } =20 if (ret =3D=3D IOU_ISSUE_SKIP_COMPLETE) { ret =3D 0; =20 - /* If the op doesn't have a file, we're not polling for it */ - if ((req->flags & REQ_F_IOPOLL) && def->iopoll_queue) + if (req->flags & REQ_F_IOPOLL) io_iopoll_req_issued(req, issue_flags); } return ret; } =20 diff --git a/io_uring/opdef.c b/io_uring/opdef.c index 645980fa4651..c3ef52b70811 100644 --- a/io_uring/opdef.c +++ b/io_uring/opdef.c @@ -65,11 +65,10 @@ const struct io_issue_def io_issue_defs[] =3D { .buffer_select =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .vectored =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_readv, .issue =3D io_read, }, @@ -80,11 +79,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollout =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .vectored =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_writev, .issue =3D io_write, }, @@ -100,11 +98,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollin =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_read_fixed, .issue =3D io_read_fixed, }, [IORING_OP_WRITE_FIXED] =3D { @@ -114,11 +111,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollout =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_write_fixed, .issue =3D io_write_fixed, }, [IORING_OP_POLL_ADD] =3D { @@ -248,11 +244,10 @@ const struct io_issue_def io_issue_defs[] =3D { .buffer_select =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_read, .issue =3D io_read, }, [IORING_OP_WRITE] =3D { @@ -262,11 +257,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollout =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_write, .issue =3D io_write, }, [IORING_OP_FADVISE] =3D { @@ -421,11 +415,10 @@ const struct io_issue_def io_issue_defs[] =3D { [IORING_OP_URING_CMD] =3D { .buffer_select =3D 1, .needs_file =3D 1, .plug =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .async_size =3D sizeof(struct io_async_cmd), .prep =3D io_uring_cmd_prep, .issue =3D io_uring_cmd, }, [IORING_OP_SEND_ZC] =3D { @@ -554,11 +547,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollin =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .vectored =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_readv_fixed, .issue =3D io_read, }, @@ -569,11 +561,10 @@ const struct io_issue_def io_issue_defs[] =3D { .pollout =3D 1, .plug =3D 1, .audit_skip =3D 1, .ioprio =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .vectored =3D 1, .async_size =3D sizeof(struct io_async_rw), .prep =3D io_prep_writev_fixed, .issue =3D io_write, }, @@ -591,11 +582,10 @@ const struct io_issue_def io_issue_defs[] =3D { [IORING_OP_URING_CMD128] =3D { .buffer_select =3D 1, .needs_file =3D 1, .plug =3D 1, .iopoll =3D 1, - .iopoll_queue =3D 1, .is_128 =3D 1, .async_size =3D sizeof(struct io_async_cmd), .prep =3D io_uring_cmd_prep, .issue =3D io_uring_cmd, }, diff --git a/io_uring/opdef.h b/io_uring/opdef.h index faf3955dce8b..667f981e63b0 100644 --- a/io_uring/opdef.h +++ b/io_uring/opdef.h @@ -23,12 +23,10 @@ struct io_issue_def { unsigned pollin : 1; unsigned pollout : 1; unsigned poll_exclusive : 1; /* skip auditing */ unsigned audit_skip : 1; - /* have to be put into the iopoll list */ - unsigned iopoll_queue : 1; /* vectored opcode, set if 1) vectored, and 2) handler needs to know */ unsigned vectored : 1; /* set to 1 if this opcode uses 128b sqes in a mixed sq */ unsigned is_128 : 1; =20 --=20 2.45.2 From nobody Sat Apr 18 11:08:29 2026 Received: from mail-yx1-f99.google.com (mail-yx1-f99.google.com [74.125.224.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE8CE3D902B for ; Fri, 27 Feb 2026 22:35:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.224.99 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231711; cv=none; b=pioshhH3y9ECQrz89/qlxSHpQPRybbbWPfYRTujZDOtrlJORFlXVN8x7joN9FrZ3hJUgge7IBdABnfyD1hDouBOOs3o2FLb3Fh6GyNFqeNti7CnDyTk+5gVvRtmcaOJPGv/vQ94lCDNls9uPdVUCyY+JKxr+bHxdGEE+Ai3wHxo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231711; c=relaxed/simple; bh=x/lgR7lfxoNQ8NgcIqgi/nGp9L0HR70DpaM2xD91Dbc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NpyeU2aOt9UyT4wHSLpjI/awzu6FMB3Kr5lc71mKXzewXTQluQnjUbCIX1PxraSC+hN8+J7/GuYuQkXSfAKAFKcS9lYLZvgmQbwwoJmYn333RXFAOM0BLw3v4700YtBfeZTN+6yPm2hrSrzWtz6IY9+mEnmXhBLcll6+yMn/uNg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=WlCXbMQo; arc=none smtp.client-ip=74.125.224.99 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="WlCXbMQo" Received: by mail-yx1-f99.google.com with SMTP id 956f58d0204a3-64c9cbec9b0so252140d50.1 for ; Fri, 27 Feb 2026 14:35:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772231708; x=1772836508; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=doNIdz9wMn9gx8zF7Ky2pC6GD/iifRgNWdZEMqpkv1c=; b=WlCXbMQohgfDevE37gEl1e8EgNC0Z6xLKsdJVlurSnx9q07v1xhQ1vGJIoXwEHDw8u mSyt/WEGenHvRUmx+Y/pgo39YFJIVlBX1NYOVDzbJzREH3ou9RGIk3UW167282iiDg2x zCSf/6bzmz7+G7CJeGNhek4fUNAoVt/0dWwk0Z7798Z5IcZMppJv2C0cVGA20FXVNwMX iU25QNuYl79/PrDr0HV0GDUCdIiyx3kcL+++fGndqxExKebAZSyv9IsiK+E5gB5E5YFM uI3rWl1EJqUafrTJipmQmS2qKLDT6NBV1To+F350xkAlqpfFsASriT04RtR65Ert6TpC svgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772231708; x=1772836508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=doNIdz9wMn9gx8zF7Ky2pC6GD/iifRgNWdZEMqpkv1c=; b=hkdgxwXwjhpyqYI+2Mtc6OuWepwSbgohg50i4UiKAWvHVMJJTB1pcQpLQcTVzrYmlQ iYwgM+RilymjqtQIqqNm5gPK7q4DRNnLFBTXXnka7HrKXzTZ+U5yy9AQfNeUUJkNxUO1 Bcmv2fUV/oXvCNUhPjyd8qxtgIbtYxHQL1ANNCTRJXeU6mBBBqC6DFmBz2rVagBFFjO4 mvi++A/0lGDAcQ6ggAP3OBVakEbtahDpT4/7RZbqZFn6geDyVZNEKFs5AVwktnE2lBvJ 91mefwMH9N9q4OqOWkNzJ4+6h2+cNAx9R0cZRIuUmIJ4XulUDrHECmMB4HG3OdzbM36O pZ3w== X-Forwarded-Encrypted: i=1; AJvYcCUW+vRRoeS4sqspnLgKSZImHdY5kYvT/+lisGmcxDoEZdyQMwIm4th5zUosMqsx3gtawuY5HRMjsZzdrY0=@vger.kernel.org X-Gm-Message-State: AOJu0YxPVxfdZXszA8DrOiMkoGaSCb/ueSkQXeOVxNQpVhTVoOCcuhI8 P+qOoI+J09CiNnC2sop7FOPrQkJE/Nayu0l1zJf23fMVI1NuIGzGeXL/gyLgKoFvrmeekt+s8Nx atOFDzBjPG3CxbPf5vjWm+ELZFqR5O0JE6tXi X-Gm-Gg: ATEYQzzxLT0eMLVoZGhoFSfG3D5sv3thjcvQuNxCYgKe0wWQrVlPECJovnAX7PvsT/5 Q5uf5DOcMdp6yHrK846l2dAs3WBPW/wFp3XeNuo1edNWqaBWd40MdTKmryyvX4zgb7Bk35BPs5P 7tGCmC+DxWRIskalIetZQeFVxlE9uu6VxaxY2d30sAbiEiHfIvNxPEOO14Qw2I27h5P/k8Gu+Rx W3hmETvdO1ArKO/HG+qmKGwPpczkLeDAHU0fgY6+4R07j127zKDUBYjd27rS8dmZQ9AWdfv4tg5 aUeZ/QBLEZ4xdlHRPs1GMj0KhEz+i5dwq+DKu0AzukxbFoMAm0kVNBEVdFwXqbIXr8WskG5md9U 5Un1iCuVNyOgU5z4XAbwgTIt9xl2jhF4Pq3ABULtMGDNPcf4MZYNRCg== X-Received: by 2002:a05:690c:c4fa:b0:797:a57b:d200 with SMTP id 00721157ae682-798850f351bmr35803657b3.0.1772231707838; Fri, 27 Feb 2026 14:35:07 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.128]) by smtp-relay.gmail.com with ESMTPS id 00721157ae682-79876ab7b69sm6884587b3.1.2026.02.27.14.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Feb 2026 14:35:07 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 255C9341DB3; Fri, 27 Feb 2026 15:35:07 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id 213C2E420D8; Fri, 27 Feb 2026 15:35:07 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v4 3/5] io_uring: count CQEs in io_iopoll_check() Date: Fri, 27 Feb 2026 15:35:01 -0700 Message-ID: <20260227223504.1162421-4-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260227223504.1162421-1-csander@purestorage.com> References: <20260227223504.1162421-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A subsequent commit will allow uring_cmds that don't use iopoll on IORING_SETUP_IOPOLL io_urings. As a result, CQEs can be posted without setting the iopoll_completed flag for a request in iopoll_list or going through task work. For example, a UBLK_U_IO_FETCH_IO_CMDS command could call io_uring_mshot_cmd_post_cqe() to directly post a CQE. The io_iopoll_check() loop currently only counts completions posted in io_do_iopoll() when determining whether the min_events threshold has been met. It also exits early if there are any existing CQEs before polling, or if any CQEs are posted while running task work. CQEs posted via io_uring_mshot_cmd_post_cqe() or other mechanisms won't be counted against min_events. Explicitly check the available CQEs in each io_iopoll_check() loop iteration to account for CQEs posted in any fashion. Signed-off-by: Caleb Sander Mateos --- io_uring/io_uring.c | 18 +++--------------- 1 file changed, 3 insertions(+), 15 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 46f39831d27c..5f694052f501 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1184,11 +1184,10 @@ __cold void io_iopoll_try_reap_events(struct io_rin= g_ctx *ctx) io_move_task_work_from_local(ctx); } =20 static int io_iopoll_check(struct io_ring_ctx *ctx, unsigned int min_event= s) { - unsigned int nr_events =3D 0; unsigned long check_cq; =20 min_events =3D min(min_events, ctx->cq_entries); =20 lockdep_assert_held(&ctx->uring_lock); @@ -1205,19 +1204,12 @@ static int io_iopoll_check(struct io_ring_ctx *ctx,= unsigned int min_events) * dropped CQE. */ if (check_cq & BIT(IO_CHECK_CQ_DROPPED_BIT)) return -EBADR; } - /* - * Don't enter poll loop if we already have events pending. - * If we do, we can potentially be spinning for commands that - * already triggered a CQE (eg in error). - */ - if (io_cqring_events(ctx)) - return 0; =20 - do { + while (io_cqring_events(ctx) < min_events) { int ret =3D 0; =20 /* * If a submit got punted to a workqueue, we can have the * application entering polling for a command before it gets @@ -1227,34 +1219,30 @@ static int io_iopoll_check(struct io_ring_ctx *ctx,= unsigned int min_events) * the poll to the issued list. Otherwise we can spin here * forever, while the workqueue is stuck trying to acquire the * very same mutex. */ if (list_empty(&ctx->iopoll_list) || io_task_work_pending(ctx)) { - u32 tail =3D ctx->cached_cq_tail; - (void) io_run_local_work_locked(ctx, min_events); =20 if (task_work_pending(current) || list_empty(&ctx->iopoll_list)) { mutex_unlock(&ctx->uring_lock); io_run_task_work(); mutex_lock(&ctx->uring_lock); } /* some requests don't go through iopoll_list */ - if (tail !=3D ctx->cached_cq_tail || list_empty(&ctx->iopoll_list)) + if (list_empty(&ctx->iopoll_list)) break; } ret =3D io_do_iopoll(ctx, !min_events); if (unlikely(ret < 0)) return ret; =20 if (task_sigpending(current)) return -EINTR; if (need_resched()) break; - - nr_events +=3D ret; - } while (nr_events < min_events); + } =20 return 0; } =20 void io_req_task_complete(struct io_tw_req tw_req, io_tw_token_t tw) --=20 2.45.2 From nobody Sat Apr 18 11:08:29 2026 Received: from mail-qt1-f226.google.com (mail-qt1-f226.google.com [209.85.160.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2936F42669A for ; Fri, 27 Feb 2026 22:35:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.226 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231710; cv=none; b=D7VatRvnbnbNsR+JVpkL2/VVTeEccpZZmXVOINQVzoP37cbwwHzel2kjyHAGY+4KAObl85lMFfZBaf0Dpb03eG8olik/HYqK+u19EFi1/+VrWLQ5yy5hWwwvQmx6b/Lto1Hk6DstTTWPeRk48/KeYcNXcuLwOpubdCK6mo6B5Oo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231710; c=relaxed/simple; bh=NBvNotESp7kUasR2rqxc9z9U7+RvFpA7PAQQGkHDOZA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ce5M98Uy9SnbnFvP2mhdE0ldtvQhNbdDP5iP6MiS9ioR9uLeWR78VrWUlF/YHcIfzyDmaElgVUKN5O/tn0MM3aOD0A4FBLqgQ2oRUFJu3m1cAvdGD+Ze1W9D2wZOmlXs0rtHRZMMlNaLfh6a87WpQFYAO5Uq2k6Zy+bh7zhsjLk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=G3vY3kzy; arc=none smtp.client-ip=209.85.160.226 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="G3vY3kzy" Received: by mail-qt1-f226.google.com with SMTP id d75a77b69052e-5032e59c8d3so2946411cf.2 for ; Fri, 27 Feb 2026 14:35:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772231708; x=1772836508; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YA4oP39D1EwOno4AkPegcttVQHiaILWd3dC9XNEopg8=; b=G3vY3kzyejLtNBv6NJe3Cbqb8SpZ7L52IpzXBZlXoMY8BwfLKsGxyEbJeFlyoG+c8g l7U0S+WGmR3UobhDwpdBoI8ngxlYwLPpDxbq/KnlP4MGO95zXS8v6sDG4hNlmgttOvYm 561bR+xEV+rHfGP6Biy/5laBlP98t5UiUCMw/e6mA1VGmGcqgGiPQbRBio+5yfctdPK7 jGJMBO1tdQoVa6UfxRYci/QHuHBATeMGixRdehNbH3jRFiE9ojfJXvIwkwRPmYfpTuDG 35UpU/keLpt+OuQPebIIhJUXYFDbHxh6bZFfZIE/VdONneVkzJSBBtLk4/ajFXAF0DUt 6zLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772231708; x=1772836508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=YA4oP39D1EwOno4AkPegcttVQHiaILWd3dC9XNEopg8=; b=BkZczUNBzEkZ3Zk77mHiNFnByF3KYTmnDG/1zN00YHmwiSr8Nc+3tc1U9ZNXaeoO/W p1ElROw0/vkzbNYfY2BJU5n+yCVkYWlPt77ayubUJsl0Ye2yV5l243PwCfmpFqeEgJkE U7ybAl/dxZBBxDYWXn1VEO72RTxI/HXegzsveyiyndXT8Bxz1Nd+N4jHZILczC2YEgq+ /fwGgTnfgAlWS18T+dBGXevHS1+RSCpPd8v40eqx/EM1lpY6IOyZkW6URs9exnLrhX7Y FFpsAHoGNixo480xL6tSRVtA0mPHFTBYkWfvKASThHBihOhEPIeG/ace7IzrRN4cGrLt s+7w== X-Forwarded-Encrypted: i=1; AJvYcCVC+MrQyMQfGZWwYY51RtmSAf/KDz4p9VPZ7mYjumehM4LzZjuWvHsuMrN6oOkYRJvmcTjJCeX8Fn33RtQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yzm6KGdyLKrXWE5T50DfgPusSCkc9sEymmSs77mlb99NXnVQPku Rcezh1Lx02M1iNiBD+45Ys76Xv2b9JDF5mqVB35vfrj6LYdmNpYVFT8VjCDDTWD6AjkzQG5/teN tBE2Vdmfijihy+KGqPPhG7t4eBa4hG0Ui5cq8 X-Gm-Gg: ATEYQzyguV0WAAxkiouAXXhN2Q+HzstkrVUgp+bhhsqDl3wOfwEh1yRP4ubnLlZ0pTb JvzG+LtZXKpaa7t9wOyj1ziXM4V0M4yptmRObGHajf+zG9mprgeryW/E+m1jgIBw8HOUFwHpkPg HZ05MK84LlG4stAa5tnA9hicjBrL7TDDSdWGFwGrnFxoC5gf7oHmiwl3tOUtJ67BGLFtYB22a6j 4A6ydNl3WJUmYzvlSHp3Lp7l24bJfI/zqZs6CxhRH+4zmxzPK5ZimOqZ5GfQdCvuxOGVak0kzvx KXPM7nor9roN2SEx/NWS906X9gLc6SiVs40qnwN67wTGFMfn8FtVMthhsYVjKdtr9EOcswzPOH9 CdO5PIzcyYaOm1nSJCjGK5dZLPvuEN+xxOIPpJ1L9m/fw7tFDvI0caw== X-Received: by 2002:a05:622a:178f:b0:506:bfa3:55c8 with SMTP id d75a77b69052e-5075297bae4mr45875231cf.5.1772231708069; Fri, 27 Feb 2026 14:35:08 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.129]) by smtp-relay.gmail.com with ESMTPS id d75a77b69052e-50744a78567sm7946111cf.6.2026.02.27.14.35.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Feb 2026 14:35:08 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 55B0C341F2A; Fri, 27 Feb 2026 15:35:07 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id 51DE3E420D8; Fri, 27 Feb 2026 15:35:07 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v4 4/5] io_uring/uring_cmd: allow non-iopoll cmds with IORING_SETUP_IOPOLL Date: Fri, 27 Feb 2026 15:35:02 -0700 Message-ID: <20260227223504.1162421-5-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260227223504.1162421-1-csander@purestorage.com> References: <20260227223504.1162421-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, creating an io_uring with IORING_SETUP_IOPOLL requires all requests issued to it to support iopoll. This prevents, for example, using ublk zero-copy together with IORING_SETUP_IOPOLL, as ublk zero-copy buffer registrations are performed using a uring_cmd. There's no technical reason why these non-iopoll uring_cmds can't be supported. They will either complete synchronously or via an external mechanism that calls io_uring_cmd_done(), io_uring_cmd_post_mshot_cqe32(), or io_uring_mshot_cmd_post_cqe(), so they don't need to be polled. Allow uring_cmd requests to be issued to IORING_SETUP_IOPOLL io_urings even if their files don't implement ->uring_cmd_iopoll(). For these uring_cmd requests, skip initializing struct io_kiocb's iopoll fields, don't set REQ_F_IOPOLL, and don't set IO_URING_F_IOPOLL in issue_flags. Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta --- io_uring/uring_cmd.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index b651c63f6e20..7b25dcd9d05f 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -255,13 +255,11 @@ int io_uring_cmd(struct io_kiocb *req, unsigned int i= ssue_flags) issue_flags |=3D IO_URING_F_SQE128; if (ctx->flags & (IORING_SETUP_CQE32 | IORING_SETUP_CQE_MIXED)) issue_flags |=3D IO_URING_F_CQE32; if (io_is_compat(ctx)) issue_flags |=3D IO_URING_F_COMPAT; - if (ctx->flags & IORING_SETUP_IOPOLL) { - if (!file->f_op->uring_cmd_iopoll) - return -EOPNOTSUPP; + if (ctx->flags & IORING_SETUP_IOPOLL && file->f_op->uring_cmd_iopoll) { req->flags |=3D REQ_F_IOPOLL; issue_flags |=3D IO_URING_F_IOPOLL; req->iopoll_completed =3D 0; if (ctx->flags & IORING_SETUP_HYBRID_IOPOLL) { /* make sure every req only blocks once */ --=20 2.45.2 From nobody Sat Apr 18 11:08:29 2026 Received: from mail-ot1-f97.google.com (mail-ot1-f97.google.com [209.85.210.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FFEF47A0C6 for ; Fri, 27 Feb 2026 22:35:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.97 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231712; cv=none; b=PAP3FPzWPZZXjdtxnhM8SIwB24FNStUF4NtCmCxpkPDetbx0CS6sd8sMD2BHECzNqZzwi6AuIiQ8hUrvHvqEvjJcQsMyOjnARHZStDUsfn8TgiHN8pyz3BS2FZm7kFM5lh5z5/NCJZ8CnuD2qoWkcIcuD9QVgu3NcjkYtJo2f6A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772231712; c=relaxed/simple; bh=leoJQLYUVaLoPMfpnaJ83Ki6TjRT6ZQ1hoz2AIfe5rc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TRJMKr8YMkaLphhHmeW4fSc5Fqn57sjYq6Y4LP3Xp5s1AxahPSp3PqEXakNtYn5WqM0qRFohj35wPFPWCaGYe1U5xnSYj0+tbmef5koxRk+3SOs2USDQeserl8tfA0qiK7/jAK/BX382bkDUIMIscfLVFoGhuI/uQ68wk0a9jvU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=VQLoDKup; arc=none smtp.client-ip=209.85.210.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="VQLoDKup" Received: by mail-ot1-f97.google.com with SMTP id 46e09a7af769-7d2ebc06f66so391051a34.0 for ; Fri, 27 Feb 2026 14:35:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1772231708; x=1772836508; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bC/PSnv7yOwEPTd/AMtRMqWpAAmzjzQLzKujNS/xPPY=; b=VQLoDKupuJgNIOaqL0VxdWMOv18q3vC5o3ztF0azusyMRSoQch0qaspgKAjsJUtceh lulR0nNdkJPApr5t+zMi87OCFAWGjEc/7EawCV0R1FRSG4ngMI32THlxxLNBM1zfMfaB 8MothmsbvU71hANd5ipcLYmMUVahnWX+Bu0x7CmfPWyiIjsjS6SbT9qNMxPAiL8altMM dDzSD6RiSpvVuj+hMQRiFFa+Cl0uKEJOd818TyoTUJieQGk7+ncwTP2fs/GqV39hceOh 35jBm1DuHHPbT18XiBHVZGzaWuWUbAKOnsrpQQnO8rFJs4LLe9URaDNfQBD/4ky4m+qG nOdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1772231708; x=1772836508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=bC/PSnv7yOwEPTd/AMtRMqWpAAmzjzQLzKujNS/xPPY=; b=DM65pTv5yARNaSU63qBkaGgXzEU1Yu0msW6bSu3IWyPrAg4I23k2h5rKSQLSXRqiDu uioC2bT+OtqWqGunM+PFSP4mZ8VmQhlY73ZEcr84drI8mO2VWo/GLUAz4XvMHQomqFg6 ExuMlbCLHDsrNqbN+OyoNjtXJTkWG/S1U+1Z5E8xvekpIuFkM8+gWgzBY+fagf+9QBao eCA/tuKgVVxWzk95F5YL2OuGV9ithkL3Bg0upwDTVOrL7tLrwyuiGbRzf8M0aLKEYOuR y9ecmj0tEadih8Vtzl2sppZTRK8xzbxXqPPsM1EkrBzWmjJ5QEdQNFo0UfblcZOK31Yd 6XCQ== X-Forwarded-Encrypted: i=1; AJvYcCWTIL0rKNVTpweg+5K7UARhIGopvcUDVpnySA3RCKhujD1378s2DAzwce4w2l56reQtSP8Ns6aVBoFQz/I=@vger.kernel.org X-Gm-Message-State: AOJu0Yy4wMvlJTtdJLXGd3GUg9pICn/bRZghKh5OhkWGy0RACyvvYLQt NyqWXElmTgH/s+TTOSIwsbe9L5E3DnBIknj7kop1hnr5jNmAMkYC4G7rd0JngGxCnlyPwloh62H jEW6H58mJ20jxCM4pLBa4ufcdZv9Qa+nRGWNVl+4KhIlP1m4KOU3A X-Gm-Gg: ATEYQzz3O9Tmf39lMCaEFu8MT9tniER/xa43g5C7gi8Nt/qPIQOs2bggQPvmaCOcWTP U7DMEDhZwmHLiWdEnlhDoEYYirKOp3Ql9SLffn7Mch+IjV0IWM9AqTycmYZ7z+2PmyiYFwCktrc yRwrZJbHUp563f6b5bQmtAfeIdw0iYFS+9a2STcEAcl3e6EaQ0F1BhS1BYwWx4gfkU97O1Fm+WM KK5JH3OM+gzfq3WvE7VQWAYdQOGPEifGV06suTWQ4HJG70SSDKg7FvHqNmGQirJlPRmTlX/65Ui uMMcHZWjqremMcHGoj35po6ACcW7G8FjNoRUNoHnxxzt8EjwjI7zGg/8WAysJrmJSQBd7/tTOwl KnK8z040qV+O5NlBLyxHSjgF4hhQbddNdCzIJeZY= X-Received: by 2002:a05:6830:91e:b0:7d1:4980:2537 with SMTP id 46e09a7af769-7d591bccd6amr2761555a34.4.1772231708517; Fri, 27 Feb 2026 14:35:08 -0800 (PST) Received: from c7-smtp-2023.dev.purestorage.com ([2620:125:9017:12:36:3:5:0]) by smtp-relay.gmail.com with ESMTPS id 46e09a7af769-7d58644a35asm829296a34.1.2026.02.27.14.35.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 Feb 2026 14:35:08 -0800 (PST) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.112.29.101]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 8A93F3420C0; Fri, 27 Feb 2026 15:35:07 -0700 (MST) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id 86A24E420D8; Fri, 27 Feb 2026 15:35:07 -0700 (MST) From: Caleb Sander Mateos To: Jens Axboe , Christoph Hellwig , Keith Busch , Sagi Grimberg Cc: io-uring@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, Anuj Gupta , Kanchan Joshi , Ming Lei , Caleb Sander Mateos Subject: [PATCH v4 5/5] nvme: remove nvme_dev_uring_cmd() IO_URING_F_IOPOLL check Date: Fri, 27 Feb 2026 15:35:03 -0700 Message-ID: <20260227223504.1162421-6-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20260227223504.1162421-1-csander@purestorage.com> References: <20260227223504.1162421-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" nvme_dev_uring_cmd() is part of struct file_operations nvme_dev_fops, which doesn't implement ->uring_cmd_iopoll(). So it won't be called with issue_flags that include IO_URING_F_IOPOLL. Drop the unnecessary IO_URING_F_IOPOLL check in nvme_dev_uring_cmd(). Signed-off-by: Caleb Sander Mateos Reviewed-by: Kanchan Joshi Reviewed-by: Anuj Gupta --- drivers/nvme/host/ioctl.c | 4 ---- 1 file changed, 4 deletions(-) diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index 8844bbd39515..9597a87cf05d 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -784,14 +784,10 @@ int nvme_ns_head_chr_uring_cmd(struct io_uring_cmd *i= oucmd, int nvme_dev_uring_cmd(struct io_uring_cmd *ioucmd, unsigned int issue_fla= gs) { struct nvme_ctrl *ctrl =3D ioucmd->file->private_data; int ret; =20 - /* IOPOLL not supported yet */ - if (issue_flags & IO_URING_F_IOPOLL) - return -EOPNOTSUPP; - ret =3D nvme_uring_cmd_checks(issue_flags); if (ret) return ret; =20 switch (ioucmd->cmd_op) { --=20 2.45.2