From nobody Tue Dec 2 02:04:06 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 966F828468D for ; Fri, 21 Nov 2025 02:00:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763690444; cv=none; b=b3qfg0Irogd3EeGctf6LJBALcMjeDFxWePLSQklZ23CgX1cyi4qiIBZq44qbq/s8gPcLIG+wM8oGJAA52Gtwm+pVuNb0hsJu5MXTNZZzB/aFrK+afiHYQ7ltHyXQWBDLggMMy7nAfbxgT47JsUO/hRtqF9CGLQbNGIC0b4N8QFI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763690444; c=relaxed/simple; bh=xULntA9KaTCMNqtKCYzwq0qsTeENqxmaTtLL2r3pa4U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O1VVzVXr1oN8QUICkD0gyusFEmCqsb6555XjG2liCYQMq+cE3t8FmDQZlaDTy2nL0VNIf6Ox/VU7tt5i7MV9l2RTeGkI9UkI7Jufm+eUxYL085RGe2TxEnIDQ7/y+L/INis6BWM5AJgTl3Ou0cD7HUtY/PRocfdAJ4fJNVu3Dvw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=G1rYgTPz; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="G1rYgTPz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1763690441; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SyjtOYHZ7lFXaCvIMC62T4TLifj3kcvBA2fFustx0E8=; b=G1rYgTPzRM7jTvQb1N4l24Flf0PSx1ZQhgp/DthHHX2TAhiZ5Tl2JxyAbCFb9jpiI+EC++ MLirgFjgJ0Jz5reiZ51gN+hSJOCYxhcGPP3wzRU4eIUaMluMkvQ6rO4xvVlbxQpuHOdB3w PnUguuVy9zH5+KULHyRtMpuHPVz0Io0= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-1-klqe1fvCPmSf9EnZpjvwJQ-1; Thu, 20 Nov 2025 21:00:38 -0500 X-MC-Unique: klqe1fvCPmSf9EnZpjvwJQ-1 X-Mimecast-MFC-AGG-ID: klqe1fvCPmSf9EnZpjvwJQ_1763690436 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C192D1800342; Fri, 21 Nov 2025 02:00:36 +0000 (UTC) Received: from localhost (unknown [10.72.116.211]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AE96218004A3; Fri, 21 Nov 2025 02:00:35 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org Cc: Caleb Sander Mateos , Uday Shankar , Stefani Seibold , Andrew Morton , linux-kernel@vger.kernel.org, Ming Lei Subject: [PATCH V4 23/27] selftests: ublk: handle UBLK_U_IO_PREP_IO_CMDS Date: Fri, 21 Nov 2025 09:58:45 +0800 Message-ID: <20251121015851.3672073-24-ming.lei@redhat.com> In-Reply-To: <20251121015851.3672073-1-ming.lei@redhat.com> References: <20251121015851.3672073-1-ming.lei@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Content-Type: text/plain; charset="utf-8" Implement support for UBLK_U_IO_PREP_IO_CMDS in the batch I/O framework: - Add batch command initialization and setup functions - Implement prep command queueing with proper buffer management - Add command completion handling for prep and commit commands - Integrate batch I/O setup into thread initialization - Update CQE handling to support batch commands The implementation uses the previously established buffer management infrastructure to queue UBLK_U_IO_PREP_IO_CMDS commands. Commands are prepared in the first thread context and use commit buffers for efficient command batching. Key changes: - ublk_batch_queue_prep_io_cmds() prepares I/O command batches - ublk_batch_compl_cmd() handles batch command completions - Modified thread setup to use batch operations when enabled - Enhanced buffer index calculation for batch mode Signed-off-by: Ming Lei --- tools/testing/selftests/ublk/batch.c | 113 +++++++++++++++++++++++++++ tools/testing/selftests/ublk/kublk.c | 46 ++++++++--- tools/testing/selftests/ublk/kublk.h | 22 ++++++ 3 files changed, 171 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/ublk/batch.c b/tools/testing/selftests= /ublk/batch.c index 609e6073c9c0..01f00c21dfdb 100644 --- a/tools/testing/selftests/ublk/batch.c +++ b/tools/testing/selftests/ublk/batch.c @@ -150,3 +150,116 @@ void ublk_batch_free_buf(struct ublk_thread *t) { free_batch_commit_buf(t); } + +static void ublk_init_batch_cmd(struct ublk_thread *t, __u16 q_id, + struct io_uring_sqe *sqe, unsigned op, + unsigned short elem_bytes, + unsigned short nr_elem, + unsigned short buf_idx) +{ + struct ublk_batch_io *cmd; + __u64 user_data; + + cmd =3D (struct ublk_batch_io *)ublk_get_sqe_cmd(sqe); + + ublk_set_sqe_cmd_op(sqe, op); + + sqe->fd =3D 0; /* dev->fds[0] */ + sqe->opcode =3D IORING_OP_URING_CMD; + sqe->flags =3D IOSQE_FIXED_FILE; + + cmd->q_id =3D q_id; + cmd->flags =3D 0; + cmd->reserved =3D 0; + cmd->elem_bytes =3D elem_bytes; + cmd->nr_elem =3D nr_elem; + + user_data =3D build_user_data(buf_idx, _IOC_NR(op), 0, q_id, 0); + io_uring_sqe_set_data64(sqe, user_data); + + t->cmd_inflight +=3D 1; + + ublk_dbg(UBLK_DBG_IO_CMD, "%s: thread %u qid %d cmd_op %x data %lx " + "nr_elem %u elem_bytes %u buf_size %u buf_idx %d " + "cmd_inflight %u\n", + __func__, t->idx, q_id, op, user_data, + cmd->nr_elem, cmd->elem_bytes, + nr_elem * elem_bytes, buf_idx, t->cmd_inflight); +} + +static void ublk_setup_commit_sqe(struct ublk_thread *t, + struct io_uring_sqe *sqe, + unsigned short buf_idx) +{ + struct ublk_batch_io *cmd; + + cmd =3D (struct ublk_batch_io *)ublk_get_sqe_cmd(sqe); + + /* Use plain user buffer instead of fixed buffer */ + cmd->flags |=3D t->cmd_flags; +} + +int ublk_batch_queue_prep_io_cmds(struct ublk_thread *t, struct ublk_queue= *q) +{ + unsigned short nr_elem =3D q->q_depth; + unsigned short buf_idx =3D ublk_alloc_commit_buf(t); + struct io_uring_sqe *sqe; + void *buf; + int i; + + ublk_assert(buf_idx !=3D UBLKS_T_COMMIT_BUF_INV_IDX); + + ublk_io_alloc_sqes(t, &sqe, 1); + + ublk_assert(nr_elem =3D=3D q->q_depth); + buf =3D ublk_get_commit_buf(t, buf_idx); + for (i =3D 0; i < nr_elem; i++) { + struct ublk_batch_elem *elem =3D (struct ublk_batch_elem *)( + buf + i * t->commit_buf_elem_size); + struct ublk_io *io =3D &q->ios[i]; + + elem->tag =3D i; + elem->result =3D 0; + + if (ublk_queue_use_auto_zc(q)) + elem->buf_index =3D ublk_batch_io_buf_idx(t, q, i); + else if (!ublk_queue_no_buf(q)) + elem->buf_addr =3D (__u64)io->buf_addr; + } + + sqe->addr =3D (__u64)buf; + sqe->len =3D t->commit_buf_elem_size * nr_elem; + + ublk_init_batch_cmd(t, q->q_id, sqe, UBLK_U_IO_PREP_IO_CMDS, + t->commit_buf_elem_size, nr_elem, buf_idx); + ublk_setup_commit_sqe(t, sqe, buf_idx); + return 0; +} + +static void ublk_batch_compl_commit_cmd(struct ublk_thread *t, + const struct io_uring_cqe *cqe, + unsigned op) +{ + unsigned short buf_idx =3D user_data_to_tag(cqe->user_data); + + if (op =3D=3D _IOC_NR(UBLK_U_IO_PREP_IO_CMDS)) + ublk_assert(cqe->res =3D=3D 0); + else if (op =3D=3D _IOC_NR(UBLK_U_IO_COMMIT_IO_CMDS)) + ;//assert(cqe->res =3D=3D t->commit_buf_size); + else + ublk_assert(0); + + ublk_free_commit_buf(t, buf_idx); +} + +void ublk_batch_compl_cmd(struct ublk_thread *t, + const struct io_uring_cqe *cqe) +{ + unsigned op =3D user_data_to_op(cqe->user_data); + + if (op =3D=3D _IOC_NR(UBLK_U_IO_PREP_IO_CMDS) || + op =3D=3D _IOC_NR(UBLK_U_IO_COMMIT_IO_CMDS)) { + ublk_batch_compl_commit_cmd(t, cqe, op); + return; + } +} diff --git a/tools/testing/selftests/ublk/kublk.c b/tools/testing/selftests= /ublk/kublk.c index 29594612edc9..e981fcf18475 100644 --- a/tools/testing/selftests/ublk/kublk.c +++ b/tools/testing/selftests/ublk/kublk.c @@ -795,28 +795,32 @@ static void ublk_handle_cqe(struct ublk_thread *t, { struct ublk_dev *dev =3D t->dev; unsigned q_id =3D user_data_to_q_id(cqe->user_data); - struct ublk_queue *q =3D &dev->q[q_id]; unsigned cmd_op =3D user_data_to_op(cqe->user_data); =20 if (cqe->res < 0 && cqe->res !=3D -ENODEV) - ublk_err("%s: res %d userdata %llx queue state %x\n", __func__, - cqe->res, cqe->user_data, q->flags); + ublk_err("%s: res %d userdata %llx thread state %x\n", __func__, + cqe->res, cqe->user_data, t->state); =20 - ublk_dbg(UBLK_DBG_IO_CMD, "%s: res %d (qid %d tag %u cmd_op %u target %d/= %d) stopping %d\n", - __func__, cqe->res, q->q_id, user_data_to_tag(cqe->user_data), - cmd_op, is_target_io(cqe->user_data), + ublk_dbg(UBLK_DBG_IO_CMD, "%s: res %d (thread %d qid %d tag %u cmd_op %x " + "data %lx target %d/%d) stopping %d\n", + __func__, cqe->res, t->idx, q_id, + user_data_to_tag(cqe->user_data), + cmd_op, cqe->user_data, is_target_io(cqe->user_data), user_data_to_tgt_data(cqe->user_data), (t->state & UBLKS_T_STOPPING)); =20 /* Don't retrieve io in case of target io */ if (is_target_io(cqe->user_data)) { - ublksrv_handle_tgt_cqe(t, q, cqe); + ublksrv_handle_tgt_cqe(t, &dev->q[q_id], cqe); return; } =20 t->cmd_inflight--; =20 - ublk_handle_uring_cmd(t, q, cqe); + if (ublk_thread_batch_io(t)) + ublk_batch_compl_cmd(t, cqe); + else + ublk_handle_uring_cmd(t, &dev->q[q_id], cqe); } =20 static int ublk_reap_events_uring(struct ublk_thread *t) @@ -873,6 +877,22 @@ static void ublk_thread_set_sched_affinity(const struc= t ublk_thread_info *info) info->dev->dev_info.dev_id, info->idx); } =20 +static void ublk_batch_setup_queues(struct ublk_thread *t) +{ + int i; + + /* setup all queues in the 1st thread */ + for (i =3D 0; i < t->dev->dev_info.nr_hw_queues; i++) { + struct ublk_queue *q =3D &t->dev->q[i]; + int ret; + + ret =3D ublk_batch_queue_prep_io_cmds(t, q); + ublk_assert(ret =3D=3D 0); + ret =3D ublk_process_io(t); + ublk_assert(ret >=3D 0); + } +} + static __attribute__((noinline)) int __ublk_io_handler_fn(struct ublk_thre= ad_info *info) { struct ublk_thread t =3D { @@ -893,8 +913,14 @@ static __attribute__((noinline)) int __ublk_io_handler= _fn(struct ublk_thread_inf ublk_dbg(UBLK_DBG_THREAD, "tid %d: ublk dev %d thread %u started\n", gettid(), dev_id, t.idx); =20 - /* submit all io commands to ublk driver */ - ublk_submit_fetch_commands(&t); + if (!ublk_thread_batch_io(&t)) { + /* submit all io commands to ublk driver */ + ublk_submit_fetch_commands(&t); + } else if (!t.idx) { + /* prepare all io commands in the 1st thread context */ + ublk_batch_setup_queues(&t); + } + do { if (ublk_process_io(&t) < 0) break; diff --git a/tools/testing/selftests/ublk/kublk.h b/tools/testing/selftests= /ublk/kublk.h index e75c28680783..51fad0f4419b 100644 --- a/tools/testing/selftests/ublk/kublk.h +++ b/tools/testing/selftests/ublk/kublk.h @@ -398,10 +398,16 @@ static inline void ublk_set_sqe_cmd_op(struct io_urin= g_sqe *sqe, __u32 cmd_op) addr[1] =3D 0; } =20 +static inline unsigned short ublk_batch_io_buf_idx( + const struct ublk_thread *t, const struct ublk_queue *q, + unsigned tag); + static inline unsigned short ublk_io_buf_idx(const struct ublk_thread *t, const struct ublk_queue *q, unsigned tag) { + if (ublk_queue_batch_io(q)) + return ublk_batch_io_buf_idx(t, q, tag); return q->ios[tag].buf_index; } =20 @@ -464,6 +470,22 @@ static inline int ublk_queue_no_buf(const struct ublk_= queue *q) return ublk_queue_use_zc(q) || ublk_queue_use_auto_zc(q); } =20 +/* + * Each IO's buffer index has to be calculated by this helper for + * UBLKS_T_BATCH_IO + */ +static inline unsigned short ublk_batch_io_buf_idx( + const struct ublk_thread *t, const struct ublk_queue *q, + unsigned tag) +{ + return tag; +} + +/* Queue UBLK_U_IO_PREP_IO_CMDS for a specific queue with batch elements */ +int ublk_batch_queue_prep_io_cmds(struct ublk_thread *t, struct ublk_queue= *q); +/* Handle completion of batch I/O commands (prep/commit) */ +void ublk_batch_compl_cmd(struct ublk_thread *t, + const struct io_uring_cqe *cqe); /* Initialize batch I/O state and calculate buffer parameters */ void ublk_batch_prepare(struct ublk_thread *t); /* Allocate and register commit buffers for batch operations */ --=20 2.47.0