From nobody Thu Dec 18 04:46:33 2025 Received: from mail-yw1-f228.google.com (mail-yw1-f228.google.com [209.85.128.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B914224891 for ; Wed, 16 Apr 2025 19:46:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.228 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744832781; cv=none; b=ANZ/rY0GAKzjoEGr1eR92kvjB2NsaoOHsc35QUm1dClQhWZlwFLkaoufYe3CEo3Rk6jzBdVcyHXnlFavR6OYEd/HZgfLV+nu2eXBpFYjwzl1/s6IthxkkMmH5c3b1qOiJHSv9ugE/t81YGO1jGqbAV+agmZVyFOXO3vfcnwokz8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744832781; c=relaxed/simple; bh=tYWKo49qWwobGHivnVbrXRT7o/0NtMaP2FGGsVhHUk0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BpjdMSB5AbycO2IvXDK91qyxs400BqE7hUrbm1MXaaDnftie6/HHj237jUaHZPf7tF0AqHyiQBSqpnVyGZAdBM9q8KaQiu3jAffAWCVbKuIzyFXqt3i6aHvigNrznRSX7rxs1Lcp3WbWCxsdkYcZ1OjiKmpJQ1FP+LZeX2cwtXw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=P9cxdZWM; arc=none smtp.client-ip=209.85.128.228 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="P9cxdZWM" Received: by mail-yw1-f228.google.com with SMTP id 00721157ae682-702599fa7c5so8314707b3.1 for ; Wed, 16 Apr 2025 12:46:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1744832777; x=1745437577; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=OofhF9O27cVrb8WKj94QBUeimdnvDakh4zV4gN59Auk=; b=P9cxdZWMqrsCEMoQNaOHOiknddPAA1vvdcpLEjhLd4R83CD+6hgWg7dH8XJTTW13mY 2d/fnmrqKEnjH9bwhSKDQFfPhRq3lQ0sIHwNWw5nTDiNpjk9/IPttXKXJoh6CNaH8tDW F/eutFH7mAhqNETC+Wbs867u+qQXTINblJue93iVWnzsYjkAaHcODWwNYS6LzhSB0t1v QVRj0nM1brG95gwRkyLKM8egeDf/5+VPmf+QsQ8mJeO8voGhDL1r3gfEKIUfSVCM2udF 4m7ukMr/e48tUWIXxUelwf/H1jMSxRHOAPbdKT8UVlTdr51y1sYNOz0z0e3//iT2LnhD SlvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744832777; x=1745437577; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OofhF9O27cVrb8WKj94QBUeimdnvDakh4zV4gN59Auk=; b=L9r9jXlyZWUkJ1yMu1zYLyj2Z3tYn1pPuRohbohuPBN+2mKf5/837YdaZbyf9iAE6l 2F/fOX4YqbbKHOlOBdP0pTwobzNxdzI/XJyUewX6omqyxTi3bzh2UW8IqzQI7lx4+CMT lqbsSbsWpd5Une8bmiRPfFpMBysHQquMLsC2fRXiOoCVVE8ckEkVsHlK4YwJJY6Bn1VW 0SdIHBbsnkZJ23I9B+lDiSuv7qDc/bhKcwnPheiPT4a2nm72fBkhvJN6r7dDdRI4V0Q0 iMtAJWLl3uvX4ywQ11j0xJFO/HX+Ku/EUqa0nmUcVMfuhncuCRBOCmPsPCgtxU9NgmZA vrVg== X-Forwarded-Encrypted: i=1; AJvYcCUFZ6TZp3smqBL7bnZRrT91A9cAovf29+Lqki6IlWc2bEMlQJzSvyfc66X2q0UGfqH1RXE3JD+iDfBnXNI=@vger.kernel.org X-Gm-Message-State: AOJu0YxSqiRKeNRdadUmj75kqUzzWdYcRa6x0lBEu0modFWmCi7u9g67 u33kg3yNYeMWoiSrcIi6jLxyIL83ig09NZZxYLSWNf5NbqV8+PfYyOgOZ0gEYmxDe618jNB3aez zuJDaxEOaNT7Od3TbM7dm+JYe/ZnuvryI X-Gm-Gg: ASbGncs+o2T5wq0rq7F7FwWPZt7/H6/kUiXRr70zZKsiqn5w+A8HcUEtKIpzvDBnogg 3s4gx7oX/uJeKZbVta4eDR2gvm2fBsIbs8TpWqIv2/gUsq1d3ep2OI3LAIdaEgy9Ad8+4jntME+ LKSUWBNou4+7qlirQidmXM/soBM+zcyl472PUrCMZ4c3UWQMgTnd+abXOoh5F1fDokSl0Qiub/0 6FU11ImPFVsXZpS4xhzAEc4qygN9Ta7omgxIgxz00WUaeHkl/Q06IVQ5v4ZxiW+2CsmJPeuahD3 T/OcC5c+f3A//Q3UOwxxQlYcnAixpkqufNG8Scnxy0EqXg== X-Google-Smtp-Source: AGHT+IHK6kJ+1KjaTnTQVEhaL8MlXYaxdyW3jy6ShqO7QymT3Ojn4Obrefs2o7lpRfVUMFUZ8u7oSSQml4nx X-Received: by 2002:a05:690c:3393:b0:6f9:4c00:53ae with SMTP id 00721157ae682-706beb2df29mr1335057b3.8.1744832777325; Wed, 16 Apr 2025 12:46:17 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.128]) by smtp-relay.gmail.com with ESMTPS id 00721157ae682-7053e0f43b6sm7203417b3.11.2025.04.16.12.46.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Apr 2025 12:46:17 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-ushankar.dev.purestorage.com (dev-ushankar.dev.purestorage.com [10.7.70.36]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 8D61C340424; Wed, 16 Apr 2025 13:46:16 -0600 (MDT) Received: by dev-ushankar.dev.purestorage.com (Postfix, from userid 1557716368) id 81E3FE402BD; Wed, 16 Apr 2025 13:46:16 -0600 (MDT) From: Uday Shankar Date: Wed, 16 Apr 2025 13:46:05 -0600 Subject: [PATCH v5 1/4] ublk: require unique task per io instead of unique task per hctx Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250416-ublk_task_per_io-v5-1-9261ad7bff20@purestorage.com> References: <20250416-ublk_task_per_io-v5-0-9261ad7bff20@purestorage.com> In-Reply-To: <20250416-ublk_task_per_io-v5-0-9261ad7bff20@purestorage.com> To: Ming Lei , Jens Axboe , Caleb Sander Mateos Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Uday Shankar X-Mailer: b4 0.14.2 Currently, ublk_drv associates to each hardware queue (hctx) a unique task (called the queue's ubq_daemon) which is allowed to issue COMMIT_AND_FETCH commands against the hctx. If any other task attempts to do so, the command fails immediately with EINVAL. When considered together with the block layer architecture, the result is that for each CPU C on the system, there is a unique ublk server thread which is allowed to handle I/O submitted on CPU C. This can lead to suboptimal performance under imbalanced load generation. For an extreme example, suppose all the load is generated on CPUs mapping to a single ublk server thread. Then that thread may be fully utilized and become the bottleneck in the system, while other ublk server threads are totally idle. This issue can also be addressed directly in the ublk server without kernel support by having threads dequeue I/Os and pass them around to ensure even load. But this solution requires inter-thread communication at least twice for each I/O (submission and completion), which is generally a bad pattern for performance. The problem gets even worse with zero copy, as more inter-thread communication would be required to have the buffer register/unregister calls to come from the correct thread. Therefore, address this issue in ublk_drv by requiring a unique task per I/O instead of per queue/hctx. Imbalanced load can then be balanced across all ublk server threads by having threads issue FETCH_REQs in a round-robin manner. As a small toy example, consider a system with a single ublk device having 2 queues, each of queue depth 4. A ublk server having 4 threads could issue its FETCH_REQs against this device as follows (where each entry is the qid,tag pair that the FETCH_REQ targets): poller thread: T0 T1 T2 T3 0,0 0,1 0,2 0,3 1,3 1,0 1,1 1,2 Since tags appear to be allocated in sequential chunks, this setup provides a rough approximation to distributing I/Os round-robin across all ublk server threads, while letting I/Os stay fully thread-local. Signed-off-by: Uday Shankar Reviewed-by: Caleb Sander Mateos --- drivers/block/ublk_drv.c | 76 ++++++++++++++++++++++----------------------= ---- 1 file changed, 35 insertions(+), 41 deletions(-) diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index cdb1543fa4a9817aa2ca2fca66720f589cf222be..4e4863d873e6593ad94c72cbf97= 1f792df5795ae 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -150,6 +150,7 @@ struct ublk_io { int res; =20 struct io_uring_cmd *cmd; + struct task_struct *task; }; =20 struct ublk_queue { @@ -157,11 +158,9 @@ struct ublk_queue { int q_depth; =20 unsigned long flags; - struct task_struct *ubq_daemon; struct ublksrv_io_desc *io_cmd_buf; =20 bool force_abort; - bool timeout; bool canceling; bool fail_io; /* copy of dev->state =3D=3D UBLK_S_DEV_FAIL_IO */ unsigned short nr_io_ready; /* how many ios setup */ @@ -1072,11 +1071,6 @@ static inline struct ublk_uring_cmd_pdu *ublk_get_ur= ing_cmd_pdu( return io_uring_cmd_to_pdu(ioucmd, struct ublk_uring_cmd_pdu); } =20 -static inline bool ubq_daemon_is_dying(struct ublk_queue *ubq) -{ - return ubq->ubq_daemon->flags & PF_EXITING; -} - /* todo: handle partial completion */ static inline void __ublk_complete_rq(struct request *req) { @@ -1224,13 +1218,13 @@ static void ublk_dispatch_req(struct ublk_queue *ub= q, /* * Task is exiting if either: * - * (1) current !=3D ubq_daemon. + * (1) current !=3D io->task. * io_uring_cmd_complete_in_task() tries to run task_work - * in a workqueue if ubq_daemon(cmd's task) is PF_EXITING. + * in a workqueue if cmd's task is PF_EXITING. * * (2) current->flags & PF_EXITING. */ - if (unlikely(current !=3D ubq->ubq_daemon || current->flags & PF_EXITING)= ) { + if (unlikely(current !=3D io->task || current->flags & PF_EXITING)) { __ublk_abort_rq(ubq, req); return; } @@ -1336,23 +1330,20 @@ static void ublk_queue_cmd_list(struct ublk_queue *= ubq, struct rq_list *l) static enum blk_eh_timer_return ublk_timeout(struct request *rq) { struct ublk_queue *ubq =3D rq->mq_hctx->driver_data; + struct ublk_io *io =3D &ubq->ios[rq->tag]; unsigned int nr_inflight =3D 0; int i; =20 if (ubq->flags & UBLK_F_UNPRIVILEGED_DEV) { - if (!ubq->timeout) { - send_sig(SIGKILL, ubq->ubq_daemon, 0); - ubq->timeout =3D true; - } - + send_sig(SIGKILL, io->task, 0); return BLK_EH_DONE; } =20 - if (!ubq_daemon_is_dying(ubq)) + if (!(io->task->flags & PF_EXITING)) return BLK_EH_RESET_TIMER; =20 for (i =3D 0; i < ubq->q_depth; i++) { - struct ublk_io *io =3D &ubq->ios[i]; + io =3D &ubq->ios[i]; =20 if (!(io->flags & UBLK_IO_FLAG_ACTIVE)) nr_inflight++; @@ -1552,8 +1543,8 @@ static void ublk_commit_completion(struct ublk_device= *ub, } =20 /* - * Called from ubq_daemon context via cancel fn, meantime quiesce ublk - * blk-mq queue, so we are called exclusively with blk-mq and ubq_daemon + * Called from io task context via cancel fn, meantime quiesce ublk + * blk-mq queue, so we are called exclusively with blk-mq and io task * context, so everything is serialized. */ static void ublk_abort_queue(struct ublk_device *ub, struct ublk_queue *ub= q) @@ -1669,13 +1660,13 @@ static void ublk_uring_cmd_cancel_fn(struct io_urin= g_cmd *cmd, return; =20 task =3D io_uring_cmd_get_task(cmd); - if (WARN_ON_ONCE(task && task !=3D ubq->ubq_daemon)) + io =3D &ubq->ios[pdu->tag]; + if (WARN_ON_ONCE(task && task !=3D io->task)) return; =20 ub =3D ubq->dev; need_schedule =3D ublk_abort_requests(ub, ubq); =20 - io =3D &ubq->ios[pdu->tag]; WARN_ON_ONCE(io->cmd !=3D cmd); ublk_cancel_cmd(ubq, io, issue_flags); =20 @@ -1836,8 +1827,6 @@ static void ublk_mark_io_ready(struct ublk_device *ub= , struct ublk_queue *ubq) mutex_lock(&ub->mutex); ubq->nr_io_ready++; if (ublk_queue_ready(ubq)) { - ubq->ubq_daemon =3D current; - get_task_struct(ubq->ubq_daemon); ub->nr_queues_ready++; =20 if (capable(CAP_SYS_ADMIN)) @@ -1934,6 +1923,7 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd *c= md, const struct ublksrv_io_cmd *ub_cmd) { struct ublk_device *ub =3D cmd->file->private_data; + struct task_struct *task; struct ublk_queue *ubq; struct ublk_io *io; u32 cmd_op =3D cmd->cmd_op; @@ -1952,13 +1942,13 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd = *cmd, if (!ubq || ub_cmd->q_id !=3D ubq->q_id) goto out; =20 - if (ubq->ubq_daemon && ubq->ubq_daemon !=3D current) - goto out; - if (tag >=3D ubq->q_depth) goto out; =20 io =3D &ubq->ios[tag]; + task =3D READ_ONCE(io->task); + if (task && task !=3D current) + goto out; =20 /* there is pending io cmd, something must be wrong */ if (io->flags & UBLK_IO_FLAG_ACTIVE) { @@ -2011,6 +2001,7 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd *c= md, } =20 ublk_fill_io_cmd(io, cmd, ub_cmd->addr); + WRITE_ONCE(io->task, get_task_struct(current)); ublk_mark_io_ready(ub, ubq); break; case UBLK_IO_COMMIT_AND_FETCH_REQ: @@ -2248,9 +2239,15 @@ static void ublk_deinit_queue(struct ublk_device *ub= , int q_id) { int size =3D ublk_queue_cmd_buf_size(ub, q_id); struct ublk_queue *ubq =3D ublk_get_queue(ub, q_id); + struct ublk_io *io; + int i; + + for (i =3D 0; i < ubq->q_depth; i++) { + io =3D &ubq->ios[i]; + if (io->task) + put_task_struct(io->task); + } =20 - if (ubq->ubq_daemon) - put_task_struct(ubq->ubq_daemon); if (ubq->io_cmd_buf) free_pages((unsigned long)ubq->io_cmd_buf, get_order(size)); } @@ -2936,15 +2933,8 @@ static void ublk_queue_reinit(struct ublk_device *ub= , struct ublk_queue *ubq) { int i; =20 - WARN_ON_ONCE(!(ubq->ubq_daemon && ubq_daemon_is_dying(ubq))); - /* All old ioucmds have to be completed */ ubq->nr_io_ready =3D 0; - /* old daemon is PF_EXITING, put it now */ - put_task_struct(ubq->ubq_daemon); - /* We have to reset it to NULL, otherwise ub won't accept new FETCH_REQ */ - ubq->ubq_daemon =3D NULL; - ubq->timeout =3D false; ubq->canceling =3D false; =20 for (i =3D 0; i < ubq->q_depth; i++) { @@ -2954,6 +2944,10 @@ static void ublk_queue_reinit(struct ublk_device *ub= , struct ublk_queue *ubq) io->flags =3D 0; io->cmd =3D NULL; io->addr =3D 0; + + WARN_ON_ONCE(!(io->task && (io->task->flags & PF_EXITING))); + put_task_struct(io->task); + io->task =3D NULL; } } =20 @@ -2993,7 +2987,7 @@ static int ublk_ctrl_start_recovery(struct ublk_devic= e *ub, pr_devel("%s: start recovery for dev id %d.\n", __func__, header->dev_id); for (i =3D 0; i < ub->dev_info.nr_hw_queues; i++) ublk_queue_reinit(ub, ublk_get_queue(ub, i)); - /* set to NULL, otherwise new ubq_daemon cannot mmap the io_cmd_buf */ + /* set to NULL, otherwise new tasks cannot mmap the io_cmd_buf */ ub->mm =3D NULL; ub->nr_queues_ready =3D 0; ub->nr_privileged_daemon =3D 0; @@ -3011,14 +3005,14 @@ static int ublk_ctrl_end_recovery(struct ublk_devic= e *ub, int ret =3D -EINVAL; int i; =20 - pr_devel("%s: Waiting for new ubq_daemons(nr: %d) are ready, dev id %d...= \n", - __func__, ub->dev_info.nr_hw_queues, header->dev_id); - /* wait until new ubq_daemon sending all FETCH_REQ */ + pr_devel("%s: Waiting for all FETCH_REQs, dev id %d...\n", __func__, + header->dev_id); + if (wait_for_completion_interruptible(&ub->completion)) return -EINTR; =20 - pr_devel("%s: All new ubq_daemons(nr: %d) are ready, dev id %d\n", - __func__, ub->dev_info.nr_hw_queues, header->dev_id); + pr_devel("%s: All FETCH_REQs received, dev id %d\n", __func__, + header->dev_id); =20 mutex_lock(&ub->mutex); if (ublk_nosrv_should_stop_dev(ub)) --=20 2.34.1 From nobody Thu Dec 18 04:46:33 2025 Received: from mail-yw1-f225.google.com (mail-yw1-f225.google.com [209.85.128.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7F2E2248AF for ; Wed, 16 Apr 2025 19:46:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.225 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744832783; cv=none; b=o4d+kNZ2JB3W9cfejWeUv5ZWRR6d2DPYO+nAETfh4O/vjLjl5Huzxg9+4Kh/s3lkM2NjoRgL1nY9Q/Wa6UQlFZntFgIiwSYuFX5w/9+WNWOrrAqOfwTAz0scie5/0FD6KEtvIAMYrZhZzY0FryHpoEaOMQ497OB4BpqcoKCglSc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744832783; c=relaxed/simple; bh=JGLutgZ75KHuHWrR2YnD7fo4yUfX4ANugcfm5HFDpUA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AdvFs15GX4ORc49o2JJsIBTduJzcPTJ32CKwpSktQhEYYVkWpOR8JTtFN+On0R+QnO200VD+PD19eoHIt1LCkqPhNBeYlZhNeoricNrCEC9BwwfVB3YwWCQiY5OV7USwNpuNdUUuLlIbDHGSwtLeapPRLRLmtNJeKueOv7bRBjA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=MxkWFXI+; arc=none smtp.client-ip=209.85.128.225 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="MxkWFXI+" Received: by mail-yw1-f225.google.com with SMTP id 00721157ae682-6fead015247so67369127b3.2 for ; Wed, 16 Apr 2025 12:46:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1744832777; x=1745437577; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=yLmRu9G8uG9ADbZkf+KZnj2bBZ+uuBo+D+GI2ZjdHfs=; b=MxkWFXI+gm6vp+1VuUzcxacQhsvZEGfVd/awsC62+rSPsU1BeYC75nKRcROI6uN1zG 6VhqIk/g1aSMUfj5T5tkprPfcsxWCJXW6ChB3l6JzA2Qq5eHaATyo4oljhwV2quBNAyu BXTvrSIpyEejnpmQZk4lc4Y+UU0POzCfVncn4bDin98XUpKebALNiKAFxexk7moEFf20 cEeDyuXAonhdof0MZFhqrzCO1KwEAa1XlXSxOfj8+ezE68DKuiCfC4aG4gdzfnvV+2Hk aiRTmHPUq1p52rH4TxORjeFKTaoNQUeQBtT+f21x/XZsV2XiOWdz4xezvsFvRH7wNktv R7qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744832777; x=1745437577; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yLmRu9G8uG9ADbZkf+KZnj2bBZ+uuBo+D+GI2ZjdHfs=; b=JR3l22jOVL+JAGahonVvmIi6yq/thuhjNFad3i8u6dQFRXjz9PNh+FPC4OqIBJp0PK pNhxUCr7Hau841GZVju+kdK6sNFoMa/t9OTSbbdORBuqIm3R6T76uV3p+iyo+BOsRWSj NLsgqNFoQgPHAUGv0Mh+tCnhhnSxTg6N8ARaAD4cZTckiJnfc/4TN/0mifoHyvlzz4u/ kPU0+hL4CeebRtKcVNkyCUrBZYf0kNog9l4XGmKaNZiLzHEck6algRaG94gFAcNgTn4W IS+cO4NGtmNb8vlU/mtZdVglDrbhKHw93d+n8VA5ZUMQN8t2GqvgtdulqA+uZ9c0nNDW pC7g== X-Forwarded-Encrypted: i=1; AJvYcCXVULw/03HgiO1hugjpJMkxlEe4sZ6gcG7eH+qJJrEi3tC8gGa+6JwBT6bVhkDSf/dw+JkAIdXHbph8Wnw=@vger.kernel.org X-Gm-Message-State: AOJu0YxFyaXfFPTuFlUlX1/YBEWu3omA5QZS2CgIRnTtCWDFTzEaDW9I q69fXP1Vf9YvFuNtv87gYk7AfLb7P+P3ftXQMOY/6j/TWrI0Bx7B63bBDsTHjYLsN9y0Mwr/0jW xW6RDr9nMwhb4xnRL4TrdfLN06KlnZgCg X-Gm-Gg: ASbGncuk+bg6YK9XMbM5yKp++lvcHXwSjX7IJam6flpSan4uDGjUg+m754dPAa6F+2Q Pa9zoUQTaPaJ2xvjQ8Wxz+yFlgM/PkzeHfVrX8PV9n8tk0k8MviTGyxfFNyaYV0uVijl6DMm+R9 7cjpL8qefJSug0M/xvlTpW8ZJ0wMTzINbAtbWKqeZBBVoNxO+Nf/jvldzdHj60BBdmLBgqZtwk7 76g4A4008LKtcL/I5bEqzvH3S/naW5Gq44h1xfzlOOnw2TFeS3Lqg4jT75m1V0WKMnB8u3idBvr iGNPRMZNQRy8xErLYK30sffDCL+rnIWF0t1Z1MwZStImWA== X-Google-Smtp-Source: AGHT+IGi/8SRV0dhXq13NAA6SwhXdPei0dwUitF9GbZqgDA1TOyEtsaJOWONfQ3mtyQfIdcmABsU3yUJ2EB+ X-Received: by 2002:a05:690c:6e09:b0:702:52fb:4649 with SMTP id 00721157ae682-706b335cfa2mr49583987b3.27.1744832777538; Wed, 16 Apr 2025 12:46:17 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([208.88.159.128]) by smtp-relay.gmail.com with ESMTPS id 00721157ae682-7053e0f393fsm7345227b3.2.2025.04.16.12.46.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Apr 2025 12:46:17 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-ushankar.dev.purestorage.com (dev-ushankar.dev.purestorage.com [IPv6:2620:125:9007:640:7:70:36:0]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 9097934058D; Wed, 16 Apr 2025 13:46:16 -0600 (MDT) Received: by dev-ushankar.dev.purestorage.com (Postfix, from userid 1557716368) id 87502E417E1; Wed, 16 Apr 2025 13:46:16 -0600 (MDT) From: Uday Shankar Date: Wed, 16 Apr 2025 13:46:06 -0600 Subject: [PATCH v5 2/4] ublk: mark ublk_queue as const for ublk_commit_and_fetch Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250416-ublk_task_per_io-v5-2-9261ad7bff20@purestorage.com> References: <20250416-ublk_task_per_io-v5-0-9261ad7bff20@purestorage.com> In-Reply-To: <20250416-ublk_task_per_io-v5-0-9261ad7bff20@purestorage.com> To: Ming Lei , Jens Axboe , Caleb Sander Mateos Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Uday Shankar X-Mailer: b4 0.14.2 We now allow multiple tasks to operate on I/Os belonging to the same queue concurrently. This means that any writes to ublk_queue in the I/O path are potential sources of data races. Try to prevent these by marking ublk_queue pointers as const when handling COMMIT_AND_FETCH. Move the logic for this command into its own function ublk_commit_and_fetch. Also open code ublk_commit_completion in ublk_commit_and_fetch to reduce the number of parameters/avoid a redundant lookup. Suggested-by: Ming Lei Signed-off-by: Uday Shankar Reviewed-by: Caleb Sander Mateos --- drivers/block/ublk_drv.c | 91 +++++++++++++++++++++++---------------------= ---- 1 file changed, 43 insertions(+), 48 deletions(-) diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 4e4863d873e6593ad94c72cbf971f792df5795ae..b44cbcbcc9d1735c398dc9ac7e9= 3f4c8736b9201 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -1518,30 +1518,6 @@ static int ublk_ch_mmap(struct file *filp, struct vm= _area_struct *vma) return remap_pfn_range(vma, vma->vm_start, pfn, sz, vma->vm_page_prot); } =20 -static void ublk_commit_completion(struct ublk_device *ub, - const struct ublksrv_io_cmd *ub_cmd) -{ - u32 qid =3D ub_cmd->q_id, tag =3D ub_cmd->tag; - struct ublk_queue *ubq =3D ublk_get_queue(ub, qid); - struct ublk_io *io =3D &ubq->ios[tag]; - struct request *req; - - /* now this cmd slot is owned by nbd driver */ - io->flags &=3D ~UBLK_IO_FLAG_OWNED_BY_SRV; - io->res =3D ub_cmd->result; - - /* find the io request and complete */ - req =3D blk_mq_tag_to_rq(ub->tag_set.tags[qid], tag); - if (WARN_ON_ONCE(unlikely(!req))) - return; - - if (req_op(req) =3D=3D REQ_OP_ZONE_APPEND) - req->__sector =3D ub_cmd->zone_append_lba; - - if (likely(!blk_should_fake_timeout(req->q))) - ublk_put_req_ref(ubq, req); -} - /* * Called from io task context via cancel fn, meantime quiesce ublk * blk-mq queue, so we are called exclusively with blk-mq and io task @@ -1918,6 +1894,45 @@ static int ublk_unregister_io_buf(struct io_uring_cm= d *cmd, return io_buffer_unregister_bvec(cmd, index, issue_flags); } =20 +static int ublk_commit_and_fetch(const struct ublk_queue *ubq, + struct ublk_io *io, struct io_uring_cmd *cmd, + const struct ublksrv_io_cmd *ub_cmd, + struct request *req) +{ + if (!(io->flags & UBLK_IO_FLAG_OWNED_BY_SRV)) + return -EINVAL; + + if (ublk_need_map_io(ubq)) { + /* + * COMMIT_AND_FETCH_REQ has to provide IO buffer if + * NEED GET DATA is not enabled or it is Read IO. + */ + if (!ub_cmd->addr && (!ublk_need_get_data(ubq) || + req_op(req) =3D=3D REQ_OP_READ)) + return -EINVAL; + } else if (req_op(req) !=3D REQ_OP_ZONE_APPEND && ub_cmd->addr) { + /* + * User copy requires addr to be unset when command is + * not zone append + */ + return -EINVAL; + } + + ublk_fill_io_cmd(io, cmd, ub_cmd->addr); + + /* now this cmd slot is owned by ublk driver */ + io->flags &=3D ~UBLK_IO_FLAG_OWNED_BY_SRV; + io->res =3D ub_cmd->result; + + if (req_op(req) =3D=3D REQ_OP_ZONE_APPEND) + req->__sector =3D ub_cmd->zone_append_lba; + + if (likely(!blk_should_fake_timeout(req->q))) + ublk_put_req_ref(ubq, req); + + return 0; +} + static int __ublk_ch_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags, const struct ublksrv_io_cmd *ub_cmd) @@ -1929,7 +1944,6 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd *c= md, u32 cmd_op =3D cmd->cmd_op; unsigned tag =3D ub_cmd->tag; int ret =3D -EINVAL; - struct request *req; =20 pr_devel("%s: received: cmd op %d queue %d tag %d result %d\n", __func__, cmd->cmd_op, ub_cmd->q_id, tag, @@ -2005,30 +2019,11 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd = *cmd, ublk_mark_io_ready(ub, ubq); break; case UBLK_IO_COMMIT_AND_FETCH_REQ: - req =3D blk_mq_tag_to_rq(ub->tag_set.tags[ub_cmd->q_id], tag); - - if (!(io->flags & UBLK_IO_FLAG_OWNED_BY_SRV)) - goto out; - - if (ublk_need_map_io(ubq)) { - /* - * COMMIT_AND_FETCH_REQ has to provide IO buffer if - * NEED GET DATA is not enabled or it is Read IO. - */ - if (!ub_cmd->addr && (!ublk_need_get_data(ubq) || - req_op(req) =3D=3D REQ_OP_READ)) - goto out; - } else if (req_op(req) !=3D REQ_OP_ZONE_APPEND && ub_cmd->addr) { - /* - * User copy requires addr to be unset when command is - * not zone append - */ - ret =3D -EINVAL; + ret =3D ublk_commit_and_fetch( + ubq, io, cmd, ub_cmd, + blk_mq_tag_to_rq(ub->tag_set.tags[ub_cmd->q_id], tag)); + if (ret) goto out; - } - - ublk_fill_io_cmd(io, cmd, ub_cmd->addr); - ublk_commit_completion(ub, ub_cmd); break; case UBLK_IO_NEED_GET_DATA: if (!(io->flags & UBLK_IO_FLAG_OWNED_BY_SRV)) --=20 2.34.1 From nobody Thu Dec 18 04:46:33 2025 Received: from mail-il1-f227.google.com (mail-il1-f227.google.com [209.85.166.227]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6440E225390 for ; Wed, 16 Apr 2025 19:46:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.166.227 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744832782; cv=none; b=kngaz9h+SD7jJqQGm2aCNTlxKyg5/aP4K//LLhI75eAcXv7/2LFgZ++a2NYU341hX6wJS+FGZdBb33Iq/3z9JkUtIp5/5yxVRPjPgnaXwNcq9qlZULmROyPnxJ8sDWfvp1JTZEcdnYramzOr+RrePGlshzNCMLJ/fcFOBYVmhE4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744832782; c=relaxed/simple; bh=6uOhhTg2M6Xc3JIAa7jixcn3vOphgn9bCMpImyxQ0aM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=BG3P8I6Uyxc5vz6fK+9ogPzS0wEBhy/JyI7so5T4qLEKmegw6IIWscjTBKXDqxFjjstZnE+8I77TPt3BpIuw+yr+ut/RQtHGL+dmFgVNTpTHXm6u/xrvamiLDzufvQDJzM3u+6vVIFEVjH2Z+wO8WRKvrEtgmazl+WisKJXZrxw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=cVCFFYKB; arc=none smtp.client-ip=209.85.166.227 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="cVCFFYKB" Received: by mail-il1-f227.google.com with SMTP id e9e14a558f8ab-3d80bbf3aefso231025ab.1 for ; Wed, 16 Apr 2025 12:46:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1744832778; x=1745437578; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=yq5esKCTo1m4AEup60elUDKAMcHgxmTnJjt4cP7Zh0E=; b=cVCFFYKBZ8ZtVyS+h53nof2wbdQMKdyHyg74Z7r/2scyY+qDqkSBNXmN8VzLOtCtyj CgEyVQ8374B3jqagSCv5Plszp2fmWoM/laYZ6JWTrIbiORWJ0bQwLnKeB1SFUA7hPQs0 XvkNdsAomqxRWob1TP7jvaV6vR0a7k+2DLEW+SkbQyPqvCv7yWjEArGy6DWRkkTUra6N Vhdgb+4Whlet0ThS5P0FRPD91JnG2vgI0bs14ZGsx8hV/65drzbXUlSVYR2EAs4MG/0c URYn8QRcGZhRDhfAUy1PQabuTB1c4d2tquoXFz06EbJOnpy7FgngHZob/3LiK9UyxWmM ub3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744832778; x=1745437578; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yq5esKCTo1m4AEup60elUDKAMcHgxmTnJjt4cP7Zh0E=; b=WJEWG1i1kkUtmpYy6KatVon9yW9U2qNxfE1uuSCx+JgedgqEEwG58Eac4FL3P88E9M GAxgX6/ZTILdE4SSQKzpEL4qAMMRyYCbxMWCac/KjnKwQsdtKCr4TPhXW6bgcyZYEztF Gtwz51+mPyA526x9gL0VyKT6sJJEbdrPwi0Q5R4R5XVPQOZ+xscuEPaMWG8WtVDJLDkk j4Q+bI42SRDSFQnofYaj/sNyrPz+abg4IKOWCZVZJix6wVlP2Wn27FPAlnLzjz1iO6Mi AA6PGBdcLj6Qy3xN7xMpW7jX0Er5tDKJyHwmureTBSUQx6QLyFpgw1Nj0bBquPFPYBM/ LlNw== X-Forwarded-Encrypted: i=1; AJvYcCUi5E22RnAEKXNuwrqAMFluwZS/QWjWMYo/zt5x6AVztOdql124PEAPydC7xgc6dWkpHulrWnN9cOnl16g=@vger.kernel.org X-Gm-Message-State: AOJu0Yx0FXwbCDFpTziGJzVpbvsKKeY6yHuVhFlScZmEq8RHe2E66ePy L18cQ/PoYZ0eeRV8MCjrD99xtZszZJOzkRBfMybGwfqZzmHnKM/YCp2WDcYnv+TsGcYvg8U1r3+ R4Oj3fgQl42Zb4qwLiGgGng7Ys0IhY7IkpCZLW5t5fbbcT3CY X-Gm-Gg: ASbGncswaoOj93/i+MCsNvawQGnnfltvLVOvTBARJpFCMPIkzopXwg3IDlMsNA3XlZQ gbdj6SR3nY2IIQIZUiyDM4qQsJj+mBAMsjjEBuodqp9sowgEg9ys/LEli0XkHUswnYZL5nlJCg6 cnBkmf+SnBN8xLw9v9ffAedxGDTmhDI+6eAYpPvoPxIIBRhzyoGFO9P7VPKwxwQMgKpDZAhftF7 IjOJFG+EqOD74FWDaFSezo//SjPQnBqymt5wS6c9aslqdIIjDzwdQNRPGxmXOuJPLe2NyQWcTlk BFGCVa07LwikJttdmUJ2w5bsDqutIW4= X-Google-Smtp-Source: AGHT+IGFFL/1z+Lvd5ijEsYgZEptvHunfhVfOarSer7ZQyhQrlZqQd0cZQYZgZi9sOPDgPcEnaNHXINcKFi1 X-Received: by 2002:a05:6e02:1fc3:b0:3d3:cdb0:a227 with SMTP id e9e14a558f8ab-3d815b123fbmr33400865ab.9.1744832778357; Wed, 16 Apr 2025 12:46:18 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([2620:125:9017:12:36:3:5:0]) by smtp-relay.gmail.com with ESMTPS id e9e14a558f8ab-3d7dbaa438csm8540685ab.41.2025.04.16.12.46.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Apr 2025 12:46:18 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-ushankar.dev.purestorage.com (dev-ushankar.dev.purestorage.com [10.7.70.36]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 8F90B3404A1; Wed, 16 Apr 2025 13:46:16 -0600 (MDT) Received: by dev-ushankar.dev.purestorage.com (Postfix, from userid 1557716368) id 8E5C0E409A2; Wed, 16 Apr 2025 13:46:16 -0600 (MDT) From: Uday Shankar Date: Wed, 16 Apr 2025 13:46:07 -0600 Subject: [PATCH v5 3/4] ublk: mark ublk_queue as const for ublk_register_io_buf Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250416-ublk_task_per_io-v5-3-9261ad7bff20@purestorage.com> References: <20250416-ublk_task_per_io-v5-0-9261ad7bff20@purestorage.com> In-Reply-To: <20250416-ublk_task_per_io-v5-0-9261ad7bff20@purestorage.com> To: Ming Lei , Jens Axboe , Caleb Sander Mateos Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Uday Shankar X-Mailer: b4 0.14.2 We now allow multiple tasks to operate on I/Os belonging to the same queue concurrently. This means that any writes to ublk_queue in the I/O path are potential sources of data races. Try to prevent these by marking ublk_queue pointers as const in ublk_register_io_buf. Suggested-by: Ming Lei Signed-off-by: Uday Shankar Reviewed-by: Caleb Sander Mateos Reviewed-by: Ming Lei --- drivers/block/ublk_drv.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index b44cbcbcc9d1735c398dc9ac7e93f4c8736b9201..215ab45b00e10150e58d7f5ea5b= 5d13e40a1aa79 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -211,7 +211,7 @@ struct ublk_params_header { static bool ublk_abort_requests(struct ublk_device *ub, struct ublk_queue = *ubq); =20 static inline struct request *__ublk_check_and_get_req(struct ublk_device = *ub, - struct ublk_queue *ubq, int tag, size_t offset); + const struct ublk_queue *ubq, int tag, size_t offset); static inline unsigned int ublk_req_build_flags(struct request *req); static inline struct ublksrv_io_desc *ublk_get_iod(struct ublk_queue *ubq, int tag); @@ -1867,7 +1867,7 @@ static void ublk_io_release(void *priv) } =20 static int ublk_register_io_buf(struct io_uring_cmd *cmd, - struct ublk_queue *ubq, unsigned int tag, + const struct ublk_queue *ubq, unsigned int tag, unsigned int index, unsigned int issue_flags) { struct ublk_device *ub =3D cmd->file->private_data; @@ -2044,7 +2044,7 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd *c= md, } =20 static inline struct request *__ublk_check_and_get_req(struct ublk_device = *ub, - struct ublk_queue *ubq, int tag, size_t offset) + const struct ublk_queue *ubq, int tag, size_t offset) { struct request *req; =20 --=20 2.34.1 From nobody Thu Dec 18 04:46:33 2025 Received: from mail-qv1-f98.google.com (mail-qv1-f98.google.com [209.85.219.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE2561DF260 for ; Wed, 16 Apr 2025 19:46:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.98 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744832780; cv=none; b=qOid0FPgH/5K4HqkDz6qca+Am9DooJHaw9xFF54NolX1kao/GDqKmVFyu9l+DQVJKXybr8gysWn1r74kvptTh7rGmIpwHHWKndtKXxBwEaWLw16j8xc3AoJ+joCH8yj2XzQEEpSfY6YewjhwmCej0JEQQwYq7xi8U1U5wlrIdCU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744832780; c=relaxed/simple; bh=KZyqYa2QO8C5zOaR5Qlffvttg9elvID00aTwi/drGnQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ECdoJds/NNPD6r7vvkxbBINWDGS0fVtq9635qS9U9Rqih8HgTzPSbSnuKFOWXBbj3HnJ7FtE2nTu16e+Kd9L1wwKIvjYH5PcR1GHcXdeDZFg53dhaWV9vM/bnCflwQS5tqTOj2Ze5qPbakStxHvDhT0KldjibOt4zEs+YN0d/Qo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com; spf=fail smtp.mailfrom=purestorage.com; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b=HL1tspZ7; arc=none smtp.client-ip=209.85.219.98 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=purestorage.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=purestorage.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=purestorage.com header.i=@purestorage.com header.b="HL1tspZ7" Received: by mail-qv1-f98.google.com with SMTP id 6a1803df08f44-6f0cfbe2042so650636d6.1 for ; Wed, 16 Apr 2025 12:46:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=purestorage.com; s=google2022; t=1744832778; x=1745437578; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=5FSkTMVQKVF8+T3ftLHZwHo1bf1YTsSlwi8hM6dxAd4=; b=HL1tspZ7i2igxAAb8JPRRjmOoVYQki5/VkK8e37jI5nm734wAlXG2AdJSGuc8sTLw/ 4+g2MFlNxFBth1Ic3y/dHajDdmLRZfXqQnib4xhPSqXdu3RmFcfjfBXJine2WcXYI4FH 7pdQCGtpqCCdRzFUgcQxMxUxig7yF8ncqAs0NGuQVFSELoRpJeUc4q4XzkXPuYyoIYpo HKfZCSWjhrAfoQzjX1gbnzP2mJmqTECdy95h/hmxfLqfiJrW/54VAM/V6cOGw6F66xiv 5ooeSa+3fhXmfHmh3zymeP0RuAShO1bT7SStQnXKevTMsKBSpzbLFFMbMH28dteSWYRH xPLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744832778; x=1745437578; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5FSkTMVQKVF8+T3ftLHZwHo1bf1YTsSlwi8hM6dxAd4=; b=r1WEJPhlkXqGWRq9Ndd3DwRvBkXCYir1iBW217u4nMcwQJHyN+qQQ7qFL1na0At7d2 UfyPNiaJR5RCpZ9Zq+dcvqslGz8kXfA+9NNwc0elsA3GTJD3NzvaGrOOoND5bED/J3aa Uoei+miQ+dqbE7+e3u5j0LxMEEZS11urTUxQ29fGQzzthGjjzYVTedP/BrvMT/yDgJFI oB5PIWfEdkwa4RKP5bIWvHnrkhvFovlZGgxGJWyfQIzh3H87q9dRLZq3l5EUl8EbToJm DJrNQ0JaYZvR2x4LVaNqPZ2oJlBo1hRd8Tnm3Ph80YdEmN6O6EYEMnYOu9GQquC50M2A gepg== X-Forwarded-Encrypted: i=1; AJvYcCXIZB6m4D3e7cxyWeC4vmQfoZ0/tpqSkMoHX8U/RoFpmdwds1Q5rkzLyd0oFxD9XD7slYy6QZlJ18DwBrY=@vger.kernel.org X-Gm-Message-State: AOJu0YxVRgXLHAXzFd/amZC5HwYUHnZ4K0/0i2FeBK60Znf7TC2aNODp y1WusqGgZkSCY2Qjq6P6t3JmBTI6lxBrubvDatMD5brqjZHsdp2Ct8xejGRlA0IKS/01awT8Nru pPIrCJNhr2VmsnnyOmrCWa3j8rjefIVyvkX0d+8p4Fq3kS+m+ X-Gm-Gg: ASbGnct6OHd8dZPYBxH7c3gWc6fXTno+NP8qsWz7Orj3p/WyugGvqdLvmJtq58i/Lzm goT7K/mt4i/+lxX2BX8I2/sUU1S1n4AYFNwc2wE1fFl7cUNyZjAV3QkcQaZaHstbn6uwlOVE28y /NHwPvNaS7ZL8k1jLF/Tzl2me1DQYfrPrv6tTkhszaUXxrIwZXePQVwA6icggJvQGGGDGJ6BRtE zpLuQBLd1bTK8cfOpRK9+lWI7aHqQm6u+xUXr+ZlAribO0tbO1hsDj/XmNEfVl/Atel3XZM9Yvs t6NegJ0W14SQpOzjblBFcO6+jc4PHmM= X-Google-Smtp-Source: AGHT+IEdAOO/a/NWoIVz4hFvCWYt1cOs67Bo6EUpAOIqrH3uPHmMDf4q+6XjtQIc4pDUHkA3q+f3b3HhDEiG X-Received: by 2002:a05:6214:cae:b0:6e2:485d:fddd with SMTP id 6a1803df08f44-6f2ba8dd00cmr2037156d6.1.1744832777729; Wed, 16 Apr 2025 12:46:17 -0700 (PDT) Received: from c7-smtp-2023.dev.purestorage.com ([2620:125:9017:12:36:3:5:0]) by smtp-relay.gmail.com with ESMTPS id 6a1803df08f44-6f0dea12d3dsm8179116d6.59.2025.04.16.12.46.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Apr 2025 12:46:17 -0700 (PDT) X-Relaying-Domain: purestorage.com Received: from dev-ushankar.dev.purestorage.com (dev-ushankar.dev.purestorage.com [IPv6:2620:125:9007:640:7:70:36:0]) by c7-smtp-2023.dev.purestorage.com (Postfix) with ESMTP id 9849F34067F; Wed, 16 Apr 2025 13:46:16 -0600 (MDT) Received: by dev-ushankar.dev.purestorage.com (Postfix, from userid 1557716368) id 95775E407EC; Wed, 16 Apr 2025 13:46:16 -0600 (MDT) From: Uday Shankar Date: Wed, 16 Apr 2025 13:46:08 -0600 Subject: [PATCH v5 4/4] ublk: mark ublk_queue as const for ublk_handle_need_get_data Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250416-ublk_task_per_io-v5-4-9261ad7bff20@purestorage.com> References: <20250416-ublk_task_per_io-v5-0-9261ad7bff20@purestorage.com> In-Reply-To: <20250416-ublk_task_per_io-v5-0-9261ad7bff20@purestorage.com> To: Ming Lei , Jens Axboe , Caleb Sander Mateos Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Uday Shankar X-Mailer: b4 0.14.2 We now allow multiple tasks to operate on I/Os belonging to the same queue concurrently. This means that any writes to ublk_queue in the I/O path are potential sources of data races. Try to prevent these by marking ublk_queue pointers as const in ublk_handle_need_get_data. Also move a bit more of the NEED_GET_DATA-specific logic into ublk_handle_need_get_data, to make the pattern in __ublk_ch_uring_cmd more uniform. Suggested-by: Ming Lei Signed-off-by: Uday Shankar Reviewed-by: Caleb Sander Mateos --- drivers/block/ublk_drv.c | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c index 215ab45b00e10150e58d7f5ea5b5d13e40a1aa79..5f9679c03305576bee586388cab= 82a6ea5472f8b 100644 --- a/drivers/block/ublk_drv.c +++ b/drivers/block/ublk_drv.c @@ -1291,7 +1291,7 @@ static void ublk_cmd_tw_cb(struct io_uring_cmd *cmd, ublk_dispatch_req(ubq, pdu->req, issue_flags); } =20 -static void ublk_queue_cmd(struct ublk_queue *ubq, struct request *rq) +static void ublk_queue_cmd(const struct ublk_queue *ubq, struct request *r= q) { struct io_uring_cmd *cmd =3D ubq->ios[rq->tag].cmd; struct ublk_uring_cmd_pdu *pdu =3D ublk_get_uring_cmd_pdu(cmd); @@ -1813,15 +1813,6 @@ static void ublk_mark_io_ready(struct ublk_device *u= b, struct ublk_queue *ubq) mutex_unlock(&ub->mutex); } =20 -static void ublk_handle_need_get_data(struct ublk_device *ub, int q_id, - int tag) -{ - struct ublk_queue *ubq =3D ublk_get_queue(ub, q_id); - struct request *req =3D blk_mq_tag_to_rq(ub->tag_set.tags[q_id], tag); - - ublk_queue_cmd(ubq, req); -} - static inline int ublk_check_cmd_op(u32 cmd_op) { u32 ioc_type =3D _IOC_TYPE(cmd_op); @@ -1933,6 +1924,20 @@ static int ublk_commit_and_fetch(const struct ublk_q= ueue *ubq, return 0; } =20 +static int ublk_get_data(const struct ublk_queue *ubq, struct ublk_io *io, + struct io_uring_cmd *cmd, + const struct ublksrv_io_cmd *ub_cmd, + struct request *req) +{ + if (!(io->flags & UBLK_IO_FLAG_OWNED_BY_SRV)) + return -EINVAL; + + ublk_fill_io_cmd(io, cmd, ub_cmd->addr); + ublk_queue_cmd(ubq, req); + + return 0; +} + static int __ublk_ch_uring_cmd(struct io_uring_cmd *cmd, unsigned int issue_flags, const struct ublksrv_io_cmd *ub_cmd) @@ -2026,10 +2031,11 @@ static int __ublk_ch_uring_cmd(struct io_uring_cmd = *cmd, goto out; break; case UBLK_IO_NEED_GET_DATA: - if (!(io->flags & UBLK_IO_FLAG_OWNED_BY_SRV)) + ret =3D ublk_get_data( + ubq, io, cmd, ub_cmd, + blk_mq_tag_to_rq(ub->tag_set.tags[ub_cmd->q_id], tag)); + if (ret) goto out; - ublk_fill_io_cmd(io, cmd, ub_cmd->addr); - ublk_handle_need_get_data(ub, ub_cmd->q_id, ub_cmd->tag); break; default: goto out; --=20 2.34.1