From nobody Sat Feb 7 17:56:26 2026 Received: from mail-wr1-f97.google.com (mail-wr1-f97.google.com [209.85.221.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EACC227A10F for ; Thu, 23 Oct 2025 20:18:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.97 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761250721; cv=none; b=USBt3OILIffbHn2pvoNNlGVzTBKXhjm9qcSEdkJ1frRNJWTh8yvAUlXKS5T4goUh4O5s1QCJfFpddAAMw9Z/RdVyS0g6/enzOlbHvH7+2kYLVfVm1i97Ip8DJp5tRkm/Fp6DxnpSo54MfptIio5Y0bXgiAGq2snbVozTY77GW0o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761250721; c=relaxed/simple; bh=5e89GXt6DhKyASePnDrD7cthV1505P8u/yKg4DMfB4Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DCG31xguoUOcJK/xoR5AJtzFm9mX1DeNNdfizCOgTZCAWGg8Ecl78WJYVIT6NqhOIhHdpYTj0LxKPxxkkg2glPbPXTRjfvgRQwGQ0qv5MG1na0FkZAtcF4lTaeIlS0y/OKHvLiFrgM6Y3lUYS+2BcWTHdhfypqWJV4Tk9qodROA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=Ce2+7RrS; arc=none smtp.client-ip=209.85.221.97 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="Ce2+7RrS" Received: by mail-wr1-f97.google.com with SMTP id ffacd0b85a97d-427054641f0so199974f8f.1 for ; Thu, 23 Oct 2025 13:18:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1761250716; x=1761855516; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=96BiLrpc0TjSBBUxSZ5BCVjSp2BMxOcWJzQbIZwxLNw=; b=Ce2+7RrSDt64qEdAiiM7W8oVRbhsCRxQmw6H+uO9r++8Jr5zF+3c14fhcDeUAUICoY 8pm039TwNICcmH3sYwyzHtfckVKK9jwA+KS7HWIhMfpEOi90tFh3SNTKj0uwDp376Fmh uQThz9o+7YgS/FO7HkZ+AIQL91Ce3Uztgwh3JFnHZ1cPq7PwFpCz76sO63MRY2A0Rf9s 3hfu8oOr6voaDXiwXUrsQrXwVRZWsqMBubzlBHs2bzloDsRe1q+kJS7P0MndMMgC+d7q 5LLoD6TlqlHtfsxfzQ5Qnlgt/b1Poh1nlMV0zX/ImVmCMgsN0/UXAx4tuCiX52RkCuQ+ 7sCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761250716; x=1761855516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=96BiLrpc0TjSBBUxSZ5BCVjSp2BMxOcWJzQbIZwxLNw=; b=X6TG74hA/DEfMmoTSiyXwEKI2uteWYBI5fNULXTVOJeY2XklSnyw8zLbv8VwoFRlRG 9vx0REVjM//Fn7FlvYPUXtXJGqQzhLN0843ke3rNfQyL/wCufh53myaFqAp4X2sukJga lnYuN/x4bSC1ctN4zNTaMl6W8JkkAdnNYCo6dqV3d9XjK9bNExZgqZVGBWOaBCnL59Ux t9uY0Cn39YuHtrgJfcVhJidcNxGncnR0FSfrh7M2YVSLgNuGVOzO26N81WygWNOFpJCC r6eVNefc7bO+CHal1HC4T8eRDbIvlBS/Nj/pdFdh20I5DyII2rAL4e9M0GVRt5tbFC6q NcCA== X-Forwarded-Encrypted: i=1; AJvYcCXw/PDxvZgNpvRII/n5O0PlrmLhGMSZtImQWX/MbvkSj7pxiQiPHhGxh8dFU/TzvGmCgVaVst7f8/AfhyI=@vger.kernel.org X-Gm-Message-State: AOJu0Yw0xm5z1NWpRIWBrOgmK2geOl+RaHVUqDU+s7QZUI7Ld7Rb3Y+R Bk0yZbqmD2sdLUESf9iQgTO0HVesT2jTjqhbagrxO4ywYMTNKw+jejkaA/BcFgV4Qgfs4moMDZ0 RkSYT9syBP6c33uK+FmdMMadrRCGPVGU5O7W5ypsBqDuIU9KOVyVN X-Gm-Gg: ASbGncvl0tQsSc3i2zbBe+pNedOgyK3al6aQOiZHuiQP5KsV38Ou5tVCzb+NszRRs/g HW19hxE5XuAmOor/EepTdAOGYL3BsQQBT+0PjOl1j2yCmxWcbmIN2YEWjx81tfPkGIeMdFyeZgu Mpshv/gHfKsHAhYcltAjnzdUe+TsZyu3yzeKRz1ehCMth/pPyP04CnEc5uDTGl1aUe2WJDwUuf4 +wgENvr20XW2+InoBR7+GbRpd9EOAHlVnos0X9nfve3gSF8Kf/qdhWSeoBr3DZ0/1a/mZ9oGROG HljMLx/m1ktl4RtXLCyACQRb6Iyz0HUzjQPgWlFRhqT1TeSGCtzaF8N94h+5Dt2Jr3vss3TL+Ff wA4/XUiw1yGQmpNNa X-Google-Smtp-Source: AGHT+IHWadkx9kp6MhgZHi2KvIKXfztJZsFFBsc7zE5C/VRcCJIv9FwFI0VQ0GJaihcf27tHQSLLkNs29ZhV X-Received: by 2002:a05:6000:290e:b0:426:ff0e:b563 with SMTP id ffacd0b85a97d-4284e52d956mr4357297f8f.3.1761250716288; Thu, 23 Oct 2025 13:18:36 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([2620:125:9017:12:36:3:5:0]) by smtp-relay.gmail.com with ESMTPS id ffacd0b85a97d-429898e70easm248962f8f.38.2025.10.23.13.18.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Oct 2025 13:18:36 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (unknown [IPv6:2620:125:9007:640:ffff::1199]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id EBE47340772; Thu, 23 Oct 2025 14:18:33 -0600 (MDT) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id E9B1BE41B1D; Thu, 23 Oct 2025 14:18:33 -0600 (MDT) From: Caleb Sander Mateos To: Jens Axboe , Miklos Szeredi , Ming Lei , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chris Mason , David Sterba Cc: io-uring@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Caleb Sander Mateos Subject: [PATCH v2 1/3] io_uring: expose io_should_terminate_tw() Date: Thu, 23 Oct 2025 14:18:28 -0600 Message-ID: <20251023201830.3109805-2-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20251023201830.3109805-1-csander@purestorage.com> References: <20251023201830.3109805-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A subsequent commit will call io_should_terminate_tw() from an inline function in include/linux/io_uring/cmd.h, so move it from an io_uring internal header to include/linux/io_uring.h. Callers outside io_uring should not call it directly. Signed-off-by: Caleb Sander Mateos --- include/linux/io_uring.h | 14 ++++++++++++++ io_uring/io_uring.h | 13 ------------- 2 files changed, 14 insertions(+), 13 deletions(-) diff --git a/include/linux/io_uring.h b/include/linux/io_uring.h index 85fe4e6b275c..c2a12287b821 100644 --- a/include/linux/io_uring.h +++ b/include/linux/io_uring.h @@ -1,13 +1,27 @@ /* SPDX-License-Identifier: GPL-2.0-or-later */ #ifndef _LINUX_IO_URING_H #define _LINUX_IO_URING_H =20 +#include #include #include #include =20 +/* + * Terminate the request if either of these conditions are true: + * + * 1) It's being executed by the original task, but that task is marked + * with PF_EXITING as it's exiting. + * 2) PF_KTHREAD is set, in which case the invoker of the task_work is + * our fallback task_work. + */ +static inline bool io_should_terminate_tw(struct io_ring_ctx *ctx) +{ + return (current->flags & (PF_KTHREAD | PF_EXITING)) || percpu_ref_is_dyin= g(&ctx->refs); +} + #if defined(CONFIG_IO_URING) void __io_uring_cancel(bool cancel_all); void __io_uring_free(struct task_struct *tsk); void io_uring_unreg_ringfd(void); const char *io_uring_get_opcode(u8 opcode); diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h index 46d9141d772a..78777bf1ea4b 100644 --- a/io_uring/io_uring.h +++ b/io_uring/io_uring.h @@ -556,23 +556,10 @@ static inline bool io_allowed_run_tw(struct io_ring_c= tx *ctx) { return likely(!(ctx->flags & IORING_SETUP_DEFER_TASKRUN) || ctx->submitter_task =3D=3D current); } =20 -/* - * Terminate the request if either of these conditions are true: - * - * 1) It's being executed by the original task, but that task is marked - * with PF_EXITING as it's exiting. - * 2) PF_KTHREAD is set, in which case the invoker of the task_work is - * our fallback task_work. - */ -static inline bool io_should_terminate_tw(struct io_ring_ctx *ctx) -{ - return (current->flags & (PF_KTHREAD | PF_EXITING)) || percpu_ref_is_dyin= g(&ctx->refs); -} - static inline void io_req_queue_tw_complete(struct io_kiocb *req, s32 res) { io_req_set_res(req, res, 0); req->io_task_work.func =3D io_req_task_complete; io_req_task_work_add(req); --=20 2.45.2 From nobody Sat Feb 7 17:56:26 2026 Received: from mail-wr1-f98.google.com (mail-wr1-f98.google.com [209.85.221.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91154262D14 for ; Thu, 23 Oct 2025 20:18:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.98 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761250719; cv=none; b=Err+o+ZdxgzKUNFOjTueuW26RG3YMu+FPy9d7ydvDtX/OCyZAnwosrLnTqcLnChRQcHCXBKdCpSTL77yQDxQGWun9smAy2zsYZ9ax6Ac0HXpE/pNdKtEgMbotjQmk8e1Y9vT6P7KdvW/g+M/JkqXGw/F8XMaXAmI2oOPk2FChlc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761250719; c=relaxed/simple; bh=1OFERuWCzicdiztTF/q7QW5F9IVu0Slo/TPxet3gSrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=T08uoFbI1hRxdN2MIrIrrwiP+I0jOwXYfBeMMrDBFce9RXC1lX4gCyCW5pBmpyQXo5wusb9/sCy74oN4D6H378hCsUwXWGd/5ao6CxEabL2hNPU0UabHKyo5n2w2cUWDOOddUY9HlwQ3lFiTisHLdaWsRVvb5/FpaXN8YSpr+SQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=NvSnp2B4; arc=none smtp.client-ip=209.85.221.98 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="NvSnp2B4" Received: by mail-wr1-f98.google.com with SMTP id ffacd0b85a97d-42708097bbdso79539f8f.3 for ; Thu, 23 Oct 2025 13:18:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1761250716; x=1761855516; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QYyTEOzXOC/G6qb7jIG39TqesYubyS/P5gHR3qDN/K4=; b=NvSnp2B4XQ3uSah1VnCLVzstzJ+AyVbGSSz6nvwx31bwaNFfuP9e+t5qsdSmj3csdL qdL8c96xJa4t6erKX8FpthpOA66kw/QxsbCuY43ttI5bfdIGljix76VEwiY1aUTQ/w93 BaeBzqfDtJdsBlGt5DYjuADmnTCCoJtKqstfrc22u0lZd+wR2WiKzuD7XHkxSq4Wj0F4 ab7A+3hLbMtE9zV2Wq0zyuXtdI+X2jzuuO+6179o5unNX7HSV5Ix2HXJrggKjxcB6Pq2 UJexbANFpr+ycCckeopRFNVtf4Z2idnawISAIo7MZkpnEpfew/yPY4wTcRfTsac+B9PS 6t/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761250716; x=1761855516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QYyTEOzXOC/G6qb7jIG39TqesYubyS/P5gHR3qDN/K4=; b=R7Llm0cDxsTvQEcJduPMGDEBMvX74MPQRmzAKhGdEqaFeYqSVr5Mst5rBfTxUX7nZz zLxZLDusmEtpzAXrKyapqr6QWjnsUOfn/EzJT4cnFP2xK9pvZDlL7ttGw+wBRnxEeYR3 MgG2oxf82G6T7E7HOWmPfkAwtDEeqI0uYiAPm59a+UsfPzB5PYNNYmR1jjHe9M9OOonL wJTlmRGEYFNkhcI5ENbTJA9dFly5Qzpa6zfbc1tS7op4gwe5dtX8SPRMMLvtRfLD/AkZ T/Dwl5AnQ3wJel5sqC49A0MTk3J6AY09w5ymhWXff0Wpw/Q5R+3d3W2agPiNuko1gdd4 7yZA== X-Forwarded-Encrypted: i=1; AJvYcCW2FL8q5ZUfE6I3GiDEJRbzaLtox9rFbInGwqbMt6rdTLStGXXxZOIMSfJDsME3f4MjgS15FWRkFKTZoSQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yz8VHzTR0ZMJ+K0lq6Iydur0DQbdACXQOnVzw3HGqIGWtDbBjzU dXy/9HlIfCqaL/eSjXygixeezI6JBhlwFOUPglwZpOy5l1KfKONPHyDLWFSsXa6GIfP3rb0O2xi k5JxbszIaQemT/6a94eRCnhHEAKOmxkws6BAJcAiwVU/qkhvq4JTo X-Gm-Gg: ASbGnctJoMFntroAX73RMg5HOkk/1Jqw6KAasUbhuyxE4IChLh/xjIpdys6MliUwLKr aBAXRNFnqgaqN2XCQGbegUL2p2DA7lzG1Aczqa8wTSl5ZqDKEaVLYwwC5GT5rOH3ZsDiflEDFCW HrBQRxQgxbtwN97LwOFQW2B8f9//yXh+CRg5eADiRvOYZtL/TkV3Xy1KyNXL/WI6Lleu/wkxFue fE1OQthX2D1WgbyG5ROoqF5HD/JLQpOUxQrkSs9OfeOlobajvQjBsqNcCDtrbLPdH1WmLZl+OI6 NNZxZtYAsl5mtGyN/Jsi6ZImdcIhadf9PlHd0kditCFeKzxmt0mKL2M1uWEAubLfFbT6KJWNb4Z QeUn1yHBk7nV200Zs X-Google-Smtp-Source: AGHT+IFNnp/5OUSlZLEwl60VMSFyQi/nSnMaue4pn8b3+029XCgF8WPzS6GPvaglBgdRg0DxMPMd5qnGf6Bb X-Received: by 2002:a5d:4e45:0:b0:427:529:5e48 with SMTP id ffacd0b85a97d-42705295e6dmr7178838f8f.5.1761250715805; Thu, 23 Oct 2025 13:18:35 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([2620:125:9017:12:36:3:5:0]) by smtp-relay.gmail.com with ESMTPS id ffacd0b85a97d-429898e96fasm244723f8f.35.2025.10.23.13.18.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Oct 2025 13:18:35 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (dev-csander.dev.purestorage.com [10.7.70.37]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 5D2C8340875; Thu, 23 Oct 2025 14:18:34 -0600 (MDT) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id 59FA6E41B1D; Thu, 23 Oct 2025 14:18:34 -0600 (MDT) From: Caleb Sander Mateos To: Jens Axboe , Miklos Szeredi , Ming Lei , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chris Mason , David Sterba Cc: io-uring@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Caleb Sander Mateos Subject: [PATCH v2 2/3] io_uring/uring_cmd: call io_should_terminate_tw() when needed Date: Thu, 23 Oct 2025 14:18:29 -0600 Message-ID: <20251023201830.3109805-3-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20251023201830.3109805-1-csander@purestorage.com> References: <20251023201830.3109805-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Most uring_cmd task work callbacks don't check IO_URING_F_TASK_DEAD. But it's computed unconditionally in io_uring_cmd_work(). Add a helper io_uring_cmd_should_terminate_tw() and call it instead of checking IO_URING_F_TASK_DEAD in the one callback, fuse_uring_send_in_task(). Remove the now unused IO_URING_F_TASK_DEAD. Signed-off-by: Caleb Sander Mateos --- fs/fuse/dev_uring.c | 2 +- include/linux/io_uring/cmd.h | 7 ++++++- include/linux/io_uring_types.h | 1 - io_uring/uring_cmd.c | 6 +----- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c index f6b12aebb8bb..71b0c9662716 100644 --- a/fs/fuse/dev_uring.c +++ b/fs/fuse/dev_uring.c @@ -1214,11 +1214,11 @@ static void fuse_uring_send_in_task(struct io_uring= _cmd *cmd, { struct fuse_ring_ent *ent =3D uring_cmd_to_ring_ent(cmd); struct fuse_ring_queue *queue =3D ent->queue; int err; =20 - if (!(issue_flags & IO_URING_F_TASK_DEAD)) { + if (!io_uring_cmd_should_terminate_tw(cmd)) { err =3D fuse_uring_prepare_send(ent, ent->fuse_req); if (err) { fuse_uring_next_fuse_req(ent, queue, issue_flags); return; } diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index 7509025b4071..b84b97c21b43 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -1,11 +1,11 @@ /* SPDX-License-Identifier: GPL-2.0-or-later */ #ifndef _LINUX_IO_URING_CMD_H #define _LINUX_IO_URING_CMD_H =20 #include -#include +#include #include =20 /* only top 8 bits of sqe->uring_cmd_flags for kernel internal use */ #define IORING_URING_CMD_CANCELABLE (1U << 30) /* io_uring_cmd is being issued again */ @@ -143,10 +143,15 @@ static inline void io_uring_cmd_complete_in_task(stru= ct io_uring_cmd *ioucmd, io_uring_cmd_tw_t task_work_cb) { __io_uring_cmd_do_in_task(ioucmd, task_work_cb, 0); } =20 +static inline bool io_uring_cmd_should_terminate_tw(struct io_uring_cmd *c= md) +{ + return io_should_terminate_tw(cmd_to_io_kiocb(cmd)->ctx); +} + static inline struct task_struct *io_uring_cmd_get_task(struct io_uring_cm= d *cmd) { return cmd_to_io_kiocb(cmd)->tctx->task; } =20 diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index c2ea6280901d..278c4a25c9e8 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -37,11 +37,10 @@ enum io_uring_cmd_flags { IO_URING_F_IOPOLL =3D (1 << 10), =20 /* set when uring wants to cancel a previously issued command */ IO_URING_F_CANCEL =3D (1 << 11), IO_URING_F_COMPAT =3D (1 << 12), - IO_URING_F_TASK_DEAD =3D (1 << 13), }; =20 struct io_wq_work_node { struct io_wq_work_node *next; }; diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index d1e3ba62ee8e..35bdac35cf4d 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -114,17 +114,13 @@ void io_uring_cmd_mark_cancelable(struct io_uring_cmd= *cmd, EXPORT_SYMBOL_GPL(io_uring_cmd_mark_cancelable); =20 static void io_uring_cmd_work(struct io_kiocb *req, io_tw_token_t tw) { struct io_uring_cmd *ioucmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); - unsigned int flags =3D IO_URING_F_COMPLETE_DEFER; - - if (io_should_terminate_tw(req->ctx)) - flags |=3D IO_URING_F_TASK_DEAD; =20 /* task_work executor checks the deffered list completion */ - ioucmd->task_work_cb(ioucmd, flags); + ioucmd->task_work_cb(ioucmd, IO_URING_F_COMPLETE_DEFER); } =20 void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, io_uring_cmd_tw_t task_work_cb, unsigned flags) --=20 2.45.2 From nobody Sat Feb 7 17:56:26 2026 Received: from mail-wr1-f98.google.com (mail-wr1-f98.google.com [209.85.221.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D03B3270EBC for ; Thu, 23 Oct 2025 20:18:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.98 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761250721; cv=none; b=RhUKaJYoKd+E7N9frstGUBc8FataghdtbK1YN+8Mq7OEfvXrwbFVDNMVgNJXvMqgDfxEu9ZQ6pmJSFdw++Pw4zyg5PJ+C8SzIzOIO2sK6lCqgVoGR9ktOR6YakpGoonpDtSVvMtVtJiVmigIi1UTuPX4NeR9WNZODEIJ1NTvw0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1761250721; c=relaxed/simple; bh=YLz5K1hAMTmuVA/ovyB70FrmtxVA0sRFV6UGU8kE6eo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DJ/eOzi8b6X7Bj7bw/CEjhrR1N8fsDSKWkA90WGmridTpF34WPAfnjzmrrf4VJXotcS2/QMHTHPeXrq72G9Hjoa2QxgLNwgY4Gh1FbOhWTneNabsIQKoFDHPl+Pof07SPrN/5MrJCAVdrnAPfg0tLWAeBQzJnyetPzF1/Zl9pc8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=IkJaAyfv; arc=none smtp.client-ip=209.85.221.98 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="IkJaAyfv" Received: by mail-wr1-f98.google.com with SMTP id ffacd0b85a97d-4270a072a0bso225532f8f.0 for ; Thu, 23 Oct 2025 13:18:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1761250716; x=1761855516; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pWqWl9I6OaTJzpoQyIopXGaUVLHhxAn4/wC5EGBqgEo=; b=IkJaAyfvreH5wZ3sYPrq7Bp6dE6fkciLEaaHCEgbh0bc1sm7mukNilF/ToJNwlo1jO wQfZ00xnTjR+I8d3d+uPlOdy77b+TnUDprAZUbhiBAYcnBU+ePV5o92ih+YHAhuLfnmH Rst1niYy42jXd4KjEDc4tePpIPw2gkgVRXMuxHUTrgZMNKZ+sQVH9PJFcDY7KhNZ4s7w kQek6b8id6t2iUcJ0nYz+CuEu2CqX2SRpAQAKnkvyW61ViwmPwJUKldxsF124RZdVwfw 65w1Pz9x7WAP1k3oWvHQSDSznYp09LH0kNP0iO62skLooI0LkORooSxM8/B/BwRDV62q gCSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761250716; x=1761855516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pWqWl9I6OaTJzpoQyIopXGaUVLHhxAn4/wC5EGBqgEo=; b=djYdaBYVnkmp22pNiKneUgMAxwSeOZWvqzlzniMuS/sBpD4UnuwUW1+zmVLw9U94is 9s9GJLp8HUVB0Bs9NDEZqh26wkW/waFPjFDk3hW+hrw3p2qtxJCkdg60R77b/fqS/VF0 wccYWA5SF1GAMGq1IjGRFEizPxyrUGIyJZDGZFgN6aMh4CQoK51ICR7V+Wah0GulMPfL fTbHeWol55GzLuEUy6h0KmmPiFCC0sdkYhWqmzVM5DqJfKTZfoWI4VqujO9piX34cCfT oxTtN+oZVYuFEYYdpN3wmRFYxhg1yqTSOwZSL1CrXPiBXnWvp5l9jF+jB/KmK5nr+qak lQXg== X-Forwarded-Encrypted: i=1; AJvYcCVCyDNyBXMOmPmiSIyGGzHKH/pO8HT3jVrIU79fKOeFGi1MN5qxQ1ARjUwNxF+v+5YGia+dw/Xc0tjyP6A=@vger.kernel.org X-Gm-Message-State: AOJu0YyANoleoXM737c0etFKRloE3o3ubFBVNc/9x40J3pg0Sour9PQz vW186xMIwgpZS6Jolaw884HYQrJrM9XVJX49aKQIYXECalmmdeoMXYMwGIyheZsTX16m/R9G/jz l90tQ4HSPIP1L3gz19YsSJaAP2B2iF5x4JGUu X-Gm-Gg: ASbGncuhjuI848oYmBG/JushzzEvoC3D3odOYMtMJtVlJHMK9DLXJwRcESXKmA7cURW ZwoO1Mxd7Vz3bF7mmGYq4jy0G2UmEb/2u1zyKYsItQ3dS7xTvJEcxIwOk7cngor3rYFZ9G4sLTp JiR1c+bo5p/GfUmOcYl9WPPpQ+ISCWuRYp5qhRAhCdw71BCCnfnhn+ETu5VYjkri2A7X9BJfc/v oMWEiE+B797Jl9g3hr9DtT+5UKWjz7FON3sGyykDfU2BkDpO4QcqY82ZDqH5kSi9o0fHt/Zqfl2 kpFJpyomNzvWaWNKMBDwmL7Yd5rqiQcvmy2NNu/nP+NIwAd8YtoEq+ncrmVbHFf7BgqV7qv0gYi bz5bh/0wqti1ZycNI3FqPM/UCbK7Ote0= X-Google-Smtp-Source: AGHT+IG+F7LvVb9q/feNRCz+uAzHxkM23HOpSSc2ZgklRM8Pol3wHaimWcAj/kEpeUsdrKVgqlM3uocuvYeU X-Received: by 2002:a05:600c:1e8b:b0:46e:36f9:c57e with SMTP id 5b1f17b1804b1-47494305991mr53576475e9.5.1761250716185; Thu, 23 Oct 2025 13:18:36 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.129]) by smtp-relay.gmail.com with ESMTPS id 5b1f17b1804b1-475c427ca8bsm10719765e9.3.2025.10.23.13.18.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Oct 2025 13:18:36 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-csander.dev.purestorage.com (unknown [IPv6:2620:125:9007:640:ffff::1199]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id ACE5E340884; Thu, 23 Oct 2025 14:18:34 -0600 (MDT) Received: by dev-csander.dev.purestorage.com (Postfix, from userid 1557716354) id ABEF0E41B1D; Thu, 23 Oct 2025 14:18:34 -0600 (MDT) From: Caleb Sander Mateos To: Jens Axboe , Miklos Szeredi , Ming Lei , Keith Busch , Christoph Hellwig , Sagi Grimberg , Chris Mason , David Sterba Cc: io-uring@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org, Caleb Sander Mateos Subject: [PATCH v2 3/3] io_uring/uring_cmd: avoid double indirect call in task work dispatch Date: Thu, 23 Oct 2025 14:18:30 -0600 Message-ID: <20251023201830.3109805-4-csander@purestorage.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20251023201830.3109805-1-csander@purestorage.com> References: <20251023201830.3109805-1-csander@purestorage.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" io_uring task work dispatch makes an indirect call to struct io_kiocb's io_task_work.func field to allow running arbitrary task work functions. In the uring_cmd case, this calls io_uring_cmd_work(), which immediately makes another indirect call to struct io_uring_cmd's task_work_cb field. Define the uring_cmd task work callbacks as functions whose signatures match io_req_tw_func_t. Define a IO_URING_CMD_TASK_WORK_ISSUE_FLAGS constant in io_uring/cmd.h to avoid manufacturing issue_flags in the uring_cmd task work callbacks. Now uring_cmd task work dispatch makes a single indirect call to the uring_cmd implementation's callback. This also allows removing the task_work_cb field from struct io_uring_cmd, freeing up some additional storage space. Signed-off-by: Caleb Sander Mateos --- block/ioctl.c | 4 +++- drivers/block/ublk_drv.c | 15 +++++++++------ drivers/nvme/host/ioctl.c | 5 +++-- fs/btrfs/ioctl.c | 4 +++- fs/fuse/dev_uring.c | 5 +++-- include/linux/io_uring/cmd.h | 16 +++++++--------- io_uring/uring_cmd.c | 13 ++----------- 7 files changed, 30 insertions(+), 32 deletions(-) diff --git a/block/ioctl.c b/block/ioctl.c index d7489a56b33c..5c10d48fab27 100644 --- a/block/ioctl.c +++ b/block/ioctl.c @@ -767,13 +767,15 @@ long compat_blkdev_ioctl(struct file *file, unsigned = cmd, unsigned long arg) struct blk_iou_cmd { int res; bool nowait; }; =20 -static void blk_cmd_complete(struct io_uring_cmd *cmd, unsigned int issue_= flags) +static void blk_cmd_complete(struct io_kiocb *req, io_tw_token_t tw) { + struct io_uring_cmd *cmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); struct blk_iou_cmd *bic =3D io_uring_cmd_to_pdu(cmd, struct blk_iou_cmd); + unsigned int issue_flags =3D IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; =20 if (bic->res =3D=3D -EAGAIN && bic->nowait) io_uring_cmd_issue_blocking(cmd); else io_uring_cmd_done(cmd, bic->res, issue_flags); diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 0c74a41a6753..00439d1879b0 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -1346,13 +1346,14 @@ static void ublk_dispatch_req(struct ublk_queue *ub= q, =20 if (ublk_prep_auto_buf_reg(ubq, req, io, issue_flags)) ublk_complete_io_cmd(io, req, UBLK_IO_RES_OK, issue_flags); } =20 -static void ublk_cmd_tw_cb(struct io_uring_cmd *cmd, - unsigned int issue_flags) +static void ublk_cmd_tw_cb(struct io_kiocb *req, io_tw_token_t tw) { + struct io_uring_cmd *cmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); + unsigned int issue_flags =3D IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; struct ublk_uring_cmd_pdu *pdu =3D ublk_get_uring_cmd_pdu(cmd); struct ublk_queue *ubq =3D pdu->ubq; =20 ublk_dispatch_req(ubq, pdu->req, issue_flags); } @@ -1364,13 +1365,14 @@ static void ublk_queue_cmd(struct ublk_queue *ubq, = struct request *rq) =20 pdu->req =3D rq; io_uring_cmd_complete_in_task(cmd, ublk_cmd_tw_cb); } =20 -static void ublk_cmd_list_tw_cb(struct io_uring_cmd *cmd, - unsigned int issue_flags) +static void ublk_cmd_list_tw_cb(struct io_kiocb *req, io_tw_token_t tw) { + struct io_uring_cmd *cmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); + unsigned int issue_flags =3D IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; struct ublk_uring_cmd_pdu *pdu =3D ublk_get_uring_cmd_pdu(cmd); struct request *rq =3D pdu->req_list; struct request *next; =20 do { @@ -2521,13 +2523,14 @@ static inline struct request *__ublk_check_and_get_= req(struct ublk_device *ub, fail_put: ublk_put_req_ref(io, req); return NULL; } =20 -static void ublk_ch_uring_cmd_cb(struct io_uring_cmd *cmd, - unsigned int issue_flags) +static void ublk_ch_uring_cmd_cb(struct io_kiocb *req, io_tw_token_t tw) { + struct io_uring_cmd *cmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); + unsigned int issue_flags =3D IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; int ret =3D ublk_ch_uring_cmd_local(cmd, issue_flags); =20 if (ret !=3D -EIOCBQUEUED) io_uring_cmd_done(cmd, ret, issue_flags); } diff --git a/drivers/nvme/host/ioctl.c b/drivers/nvme/host/ioctl.c index c212fa952c0f..df39cee94de1 100644 --- a/drivers/nvme/host/ioctl.c +++ b/drivers/nvme/host/ioctl.c @@ -396,13 +396,14 @@ static inline struct nvme_uring_cmd_pdu *nvme_uring_c= md_pdu( struct io_uring_cmd *ioucmd) { return io_uring_cmd_to_pdu(ioucmd, struct nvme_uring_cmd_pdu); } =20 -static void nvme_uring_task_cb(struct io_uring_cmd *ioucmd, - unsigned issue_flags) +static void nvme_uring_task_cb(struct io_kiocb *req, io_tw_token_t tw) { + struct io_uring_cmd *ioucmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); + unsigned int issue_flags =3D IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; struct nvme_uring_cmd_pdu *pdu =3D nvme_uring_cmd_pdu(ioucmd); =20 if (pdu->bio) blk_rq_unmap_user(pdu->bio); io_uring_cmd_done32(ioucmd, pdu->status, pdu->result, issue_flags); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 185bef0df1c2..3b62eb8a50dc 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -4647,13 +4647,15 @@ struct btrfs_uring_priv { struct io_btrfs_cmd { struct btrfs_uring_encoded_data *data; struct btrfs_uring_priv *priv; }; =20 -static void btrfs_uring_read_finished(struct io_uring_cmd *cmd, unsigned i= nt issue_flags) +static void btrfs_uring_read_finished(struct io_kiocb *req, io_tw_token_t = tw) { + struct io_uring_cmd *cmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); struct io_btrfs_cmd *bc =3D io_uring_cmd_to_pdu(cmd, struct io_btrfs_cmd); + unsigned int issue_flags =3D IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; struct btrfs_uring_priv *priv =3D bc->priv; struct btrfs_inode *inode =3D BTRFS_I(file_inode(priv->iocb.ki_filp)); struct extent_io_tree *io_tree =3D &inode->io_tree; pgoff_t index; u64 cur; diff --git a/fs/fuse/dev_uring.c b/fs/fuse/dev_uring.c index 71b0c9662716..051136e94a33 100644 --- a/fs/fuse/dev_uring.c +++ b/fs/fuse/dev_uring.c @@ -1207,13 +1207,14 @@ static void fuse_uring_send(struct fuse_ring_ent *e= nt, struct io_uring_cmd *cmd, /* * This prepares and sends the ring request in fuse-uring task context. * User buffers are not mapped yet - the application does not have permiss= ion * to write to it - this has to be executed in ring task context. */ -static void fuse_uring_send_in_task(struct io_uring_cmd *cmd, - unsigned int issue_flags) +static void fuse_uring_send_in_task(struct io_kiocb *req, io_tw_token_t tw) { + struct io_uring_cmd *cmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); + unsigned int issue_flags =3D IO_URING_CMD_TASK_WORK_ISSUE_FLAGS; struct fuse_ring_ent *ent =3D uring_cmd_to_ring_ent(cmd); struct fuse_ring_queue *queue =3D ent->queue; int err; =20 if (!io_uring_cmd_should_terminate_tw(cmd)) { diff --git a/include/linux/io_uring/cmd.h b/include/linux/io_uring/cmd.h index b84b97c21b43..3efad93404f9 100644 --- a/include/linux/io_uring/cmd.h +++ b/include/linux/io_uring/cmd.h @@ -9,18 +9,13 @@ /* only top 8 bits of sqe->uring_cmd_flags for kernel internal use */ #define IORING_URING_CMD_CANCELABLE (1U << 30) /* io_uring_cmd is being issued again */ #define IORING_URING_CMD_REISSUE (1U << 31) =20 -typedef void (*io_uring_cmd_tw_t)(struct io_uring_cmd *cmd, - unsigned issue_flags); - struct io_uring_cmd { struct file *file; const struct io_uring_sqe *sqe; - /* callback to defer completions to task context */ - io_uring_cmd_tw_t task_work_cb; u32 cmd_op; u32 flags; u8 pdu[32]; /* available inline for free use */ }; =20 @@ -58,11 +53,11 @@ int io_uring_cmd_import_fixed_vec(struct io_uring_cmd *= ioucmd, */ void __io_uring_cmd_done(struct io_uring_cmd *cmd, s32 ret, u64 res2, unsigned issue_flags, bool is_cqe32); =20 void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, - io_uring_cmd_tw_t task_work_cb, + io_req_tw_func_t task_work_cb, unsigned flags); =20 /* * Note: the caller should never hard code @issue_flags and only use the * mask provided by the core io_uring code. @@ -107,11 +102,11 @@ static inline int io_uring_cmd_import_fixed_vec(struc= t io_uring_cmd *ioucmd, static inline void __io_uring_cmd_done(struct io_uring_cmd *cmd, s32 ret, u64 ret2, unsigned issue_flags, bool is_cqe32) { } static inline void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, - io_uring_cmd_tw_t task_work_cb, unsigned flags) + io_req_tw_func_t task_work_cb, unsigned flags) { } static inline void io_uring_cmd_mark_cancelable(struct io_uring_cmd *cmd, unsigned int issue_flags) { @@ -130,19 +125,22 @@ static inline bool io_uring_mshot_cmd_post_cqe(struct= io_uring_cmd *ioucmd, { return true; } #endif =20 +/* task_work executor checks the deferred list completion */ +#define IO_URING_CMD_TASK_WORK_ISSUE_FLAGS IO_URING_F_COMPLETE_DEFER + /* users must follow the IOU_F_TWQ_LAZY_WAKE semantics */ static inline void io_uring_cmd_do_in_task_lazy(struct io_uring_cmd *ioucm= d, - io_uring_cmd_tw_t task_work_cb) + io_req_tw_func_t task_work_cb) { __io_uring_cmd_do_in_task(ioucmd, task_work_cb, IOU_F_TWQ_LAZY_WAKE); } =20 static inline void io_uring_cmd_complete_in_task(struct io_uring_cmd *iouc= md, - io_uring_cmd_tw_t task_work_cb) + io_req_tw_func_t task_work_cb) { __io_uring_cmd_do_in_task(ioucmd, task_work_cb, 0); } =20 static inline bool io_uring_cmd_should_terminate_tw(struct io_uring_cmd *c= md) diff --git a/io_uring/uring_cmd.c b/io_uring/uring_cmd.c index 35bdac35cf4d..5a80d35658dc 100644 --- a/io_uring/uring_cmd.c +++ b/io_uring/uring_cmd.c @@ -111,29 +111,20 @@ void io_uring_cmd_mark_cancelable(struct io_uring_cmd= *cmd, io_ring_submit_unlock(ctx, issue_flags); } } EXPORT_SYMBOL_GPL(io_uring_cmd_mark_cancelable); =20 -static void io_uring_cmd_work(struct io_kiocb *req, io_tw_token_t tw) -{ - struct io_uring_cmd *ioucmd =3D io_kiocb_to_cmd(req, struct io_uring_cmd); - - /* task_work executor checks the deffered list completion */ - ioucmd->task_work_cb(ioucmd, IO_URING_F_COMPLETE_DEFER); -} - void __io_uring_cmd_do_in_task(struct io_uring_cmd *ioucmd, - io_uring_cmd_tw_t task_work_cb, + io_req_tw_func_t task_work_cb, unsigned flags) { struct io_kiocb *req =3D cmd_to_io_kiocb(ioucmd); =20 if (WARN_ON_ONCE(req->flags & REQ_F_APOLL_MULTISHOT)) return; =20 - ioucmd->task_work_cb =3D task_work_cb; - req->io_task_work.func =3D io_uring_cmd_work; + req->io_task_work.func =3D task_work_cb; __io_req_task_work_add(req, flags); } EXPORT_SYMBOL_GPL(__io_uring_cmd_do_in_task); =20 static inline void io_req_set_cqe32_extra(struct io_kiocb *req, --=20 2.45.2