From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C57CF283C9D for ; Sat, 1 Aug 2026 15:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599242; cv=none; b=OJVSlzWf82ZP4n9VYL7xjJfDGsZs4miG4HfwC2WcvktNNpZP81K613s3qgI48AGBs+9Ll31p+yhKL76R5v6mTPloMTh3q77MJf5DXb1IL0HmDeUClQ3yQVYDZVlr/FIvaagqBviOq2VmOvcpmi/xn7b1bcuLdzqZFJptFtksYYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599242; c=relaxed/simple; bh=7PmUGqqIa6T1WmPU+PO3EGT6SBBf7drQLdetG/fPGS8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JoccJTdacS4pdJtII4WvJsRP38Eyikg20e2AnRr5UGIQAW5BoVn+vj+d9/C3/+FuYAFhX4mehlqIaTCzaTZXdjpdECqWniteQMgcV+Zi7bmbUcOidyNDuqCDCRqIDpIW1u+J4Q/mojlWTP9PSEWbG1Z1UnvNVcCX2Cmq1xll724= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Dk9etQ6h; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Dk9etQ6h" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-2caed617615so25861675ad.3 for ; Sat, 01 Aug 2026 08:47:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599234; x=1786204034; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=zDHMc8aFblNi/hrooe3cgS0zDU46LdNoPkDB9yiNAnQ=; b=Dk9etQ6h0jEe/rlnhYhzM5+opMsrmtiJvownfCVwAW+qEK3qTaBFjaHBKMxp+l+NgP XQQfE6e40e5WhHX5GjIEzQMZn6FmtuF2dAb8MO1rUAkqcEdsLAFGgSyjP2MgUEShBAfb OflBER6x2ci0dcpo5IYgDPmaPuDEvbG2MBRzKP5F4QW8rp5u8CXKPpNVjk61A+tQ6cd6 RFmaKyIhr4VcF1XC4WCAdTs+RD/oZ3xovWJuG/RdkhUs4pUko45uiCL4RW0BkAudnMRG K3YdnGJhjVWIPBJu32NhsJi76AZS734sj53hBGBJzUmSzCYByq5cmtLKA1UYmp2RWIBx 9MuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599234; x=1786204034; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=zDHMc8aFblNi/hrooe3cgS0zDU46LdNoPkDB9yiNAnQ=; b=SelNHiSwFNvV1HNutNTi0rhJgEszMT7mAwWMjEnH/pPv/dSHQJa+WOcpfSSf5sb7I4 mEY8Fhguj39ckmrvPQx2qF8KxkOcasivXyUXrzJI9GlaIqwMDjATiRVYhIQS1dKVIYvO zQzT/uy71KKwtsi1ZKTjmqwc2n0PXY+0eC16MkaR8uzeD9vvAQ8VL1qKAmzTbS5f1S1O LW1usQxlaf4eCEcJdfqtIO5oncz8TksDrbMXW0U6N1w98LuEl/iQfQn2zvA0PXATDVC7 y53FTGin2h8i7c3zP8Tc/1ry1IKpSYYF0b5FpKw+MOMjd43QLTCAo+oNl4Ucn4plZgQG anpg== X-Forwarded-Encrypted: i=1; AHgh+Rq2hLkb4HtrVnaeDpCg19uSrQn9So8Xkjm2Eh08bQSC4GlXLVO/fmZVZvnCv9j7dR37zFFbpF4g6sdt4+o=@vger.kernel.org X-Gm-Message-State: AOJu0YwRhLNhfsNdJN0bSn5q/e8rI15etfAGSmub/zlQZuC9mB68Gg3A pYf3UDgapptSXeuYUMJLHzDXa9USuZNItNNrHeFS7iaXQFO46RGIglDI X-Gm-Gg: AR+sD12WwMzzKjbGLUrHJhENr2ABpY4GRzkQu5hh9+OIE1DY69zRqqiCDTyjVTScjw4 YlpC28CVqILj+fUOpdaIhledRqPIEpc5vDQ7+meGgs8Cdu2IG1qVXRVKm5LUV5fpjGskgmXe2qH 4WMyF60uJE6Yr03JqiZ/OrxgbfGRg47/thhYgDDg9VKdqjX0R7oS0GZWJtNWm4N/UUbZYB3CMoU aae2CT3Cr14V6jAxJn1Q7AGiQntXC6YMoTB8Okka/T4CGVdP4e61XzXgZCYSdbtA2QoKhPT2K5n MLYlL+Bwpo+1iXnogKgLtjlNVu7tlTy/XSv9kEg9ZhdKaWAhCY/GhoahAVyjYyItjaikpBVZrBf VK0/Kmc17hTBSSnaGpefS/QWJLYz9Ky6j5HVO+mDDNtgiznnCMedmNN7Pu0A+2BZmLtFWd5wcou j5YL7YJLhsuVdhYhf2hkj/YuSHRCHn7GV/PRvZaxUz7r7I5AqN/hfH94vEdCNd6tfZ9KX+knMv6 ket4IwTuejsyQg4ai2vQ5O9dcOyLV4HgN5E2EmImgFvKJw= X-Received: by 2002:a17:903:1a67:b0:2cf:7e56:22c4 with SMTP id d9443c01a7336-2d0523e1c85mr42035155ad.30.1785599233638; Sat, 01 Aug 2026 08:47:13 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.46.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:47:12 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 01/16] dma-buf: introduce initial file I/O infrastructure Date: Sat, 1 Aug 2026 16:46:13 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The goal is to be able to natively use dma-buf in the read-write / IO path. This patch adds basic building blocks serving as a glue and API between drivers and upper layer subsystems providing the uAPI. Later patches implement it for NVMe raw block devices and expose it to the user space via io_uring. There are two main objects. struct dma_buf_io_ctx and struct dma_buf_io_map. The ctx is used during initial registration and serves as an interface between the upper layer user like io_uring and to the importer subsystem / driver. The map represents the actual dma map established for the target device[s] with dma_buf_map_attachment() and stored in a device specific format. The context is created via a new file operation ->init_dma_buf_io_ctx. The ctx-map separation exists to support map invalidation (see dma_buf_io_invalidate_mappings()). A ctx can create multiple maps during its lifetime, but there can only be no more than one (active) map attached to it. Invalidation drops the active map if present, and the next map will only be attempted to be created once there is a new request that wants to use the dma-buf IO ctx. The primary task of the dma_buf_io_map object is to count requests using it and to wait for their completion when we want to destroy the DMA map. Invalidation is delayed by inserting a dma fence, which is signaled when all flight requests are completed. [un]mapping and any work with dma addresses is delegated to the importer driver via an ops table stored in the ctx, see struct dma_buf_io_ops. Only the target driver / subsystem knows about devices it wants to use the dma-buf with, especially in case of multi-device filesystems or stacking in the future. Signed-off-by: Pavel Begunkov --- drivers/dma-buf/Makefile | 2 +- drivers/dma-buf/dma-buf-io.c | 278 +++++++++++++++++++++++++++++++++++ include/linux/dma-buf-io.h | 92 ++++++++++++ include/linux/fs.h | 2 + 4 files changed, 373 insertions(+), 1 deletion(-) create mode 100644 drivers/dma-buf/dma-buf-io.c create mode 100644 include/linux/dma-buf-io.h diff --git a/drivers/dma-buf/Makefile b/drivers/dma-buf/Makefile index b25d7550bacf..523731b0f83e 100644 --- a/drivers/dma-buf/Makefile +++ b/drivers/dma-buf/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y :=3D dma-buf.o dma-fence.o dma-fence-array.o dma-fence-chain.o \ - dma-fence-unwrap.o dma-resv.o dma-buf-mapping.o + dma-fence-unwrap.o dma-resv.o dma-buf-mapping.o dma-buf-io.o obj-$(CONFIG_DMABUF_HEAPS) +=3D dma-heap.o obj-$(CONFIG_DMABUF_HEAPS) +=3D heaps/ obj-$(CONFIG_SYNC_FILE) +=3D sync_file.o diff --git a/drivers/dma-buf/dma-buf-io.c b/drivers/dma-buf/dma-buf-io.c new file mode 100644 index 000000000000..24f58f80c45c --- /dev/null +++ b/drivers/dma-buf/dma-buf-io.c @@ -0,0 +1,278 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Common infrastructure for supporing dma-buf in the I/O path. + * + * Copyright (C) 2026 Pavel Begunkov + */ +#include +#include + +struct dma_buf_io_fence { + struct dma_fence base; + spinlock_t lock; +}; + +static const char *dma_buf_io_fence_drv_name(struct dma_fence *fence) +{ + /* default fence release kfree's the base pointer */ + BUILD_BUG_ON(offsetof(struct dma_buf_io_fence, base)); + + return "dma-buf-io-ctx"; +} + +static const char *dma_buf_io_fence_timeline_name(struct dma_fence *fence) +{ + return "dma-buf-io-ctx"; +} + +const struct dma_fence_ops dma_buf_io_fence_ops =3D { + .get_driver_name =3D dma_buf_io_fence_drv_name, + .get_timeline_name =3D dma_buf_io_fence_timeline_name, +}; + +static void dma_buf_io_ctx_destroy_work(struct work_struct *work) +{ + struct dma_buf_io_ctx *ctx =3D container_of(work, struct dma_buf_io_ctx, + release_work); + + if (WARN_ON_ONCE(refcount_read(&ctx->refs))) + return; + + ctx->dev_ops->release(ctx); + dma_buf_put(ctx->dmabuf); + kfree(ctx); +} + +static void dma_buf_io_map_release_work(struct work_struct *work) +{ + struct dma_buf_io_map *map =3D container_of(work, struct dma_buf_io_map, + release_work); + struct dma_buf_io_fence *fence =3D map->fence; + struct dma_buf_io_ctx *ctx =3D map->ctx; + struct dma_buf *dmabuf =3D ctx->dmabuf; + + /* the release path must wait for fences */ + if (WARN_ON_ONCE(refcount_read(&ctx->refs) =3D=3D 0)) + return; + + /* Prevent from destoying the ctx while unmapping */ + refcount_inc(&ctx->refs); + + /* + * There are no more requests using the map, we can signal the fence. + * It should be done before taking the resv lock as someone could be + * waiting for the fence while holding the lock. + */ + dma_fence_signal(&fence->base); + + dma_resv_lock(dmabuf->resv, NULL); + ctx->dev_ops->unmap(ctx, map); + dma_resv_unlock(dmabuf->resv); + + dma_fence_put(&fence->base); + percpu_ref_exit(&map->refs); + kfree(map); + + if (refcount_dec_and_test(&ctx->refs)) { + /* + * Destruction needs to wait for I/O and dma fences. Defer it to + * simplify locking. + */ + INIT_WORK(&ctx->release_work, dma_buf_io_ctx_destroy_work); + queue_work(system_wq, &ctx->release_work); + } +} + +static void dma_buf_io_map_refs_release(struct percpu_ref *ref) +{ + struct dma_buf_io_map *map =3D container_of(ref, struct dma_buf_io_map, r= efs); + + /* might sleep, use a worker */ + INIT_WORK(&map->release_work, dma_buf_io_map_release_work); + queue_work(system_wq, &map->release_work); +} + +int dma_buf_io_init_map(struct dma_buf_io_ctx *ctx, struct dma_buf_io_map = *map) +{ + struct dma_buf_io_fence *fence =3D NULL; + int ret; + + fence =3D kzalloc(sizeof(*fence), GFP_KERNEL); + if (!fence) + return -ENOMEM; + + ret =3D percpu_ref_init(&map->refs, dma_buf_io_map_refs_release, 0, GFP_K= ERNEL); + if (ret) { + kfree(fence); + return ret; + } + + spin_lock_init(&fence->lock); + dma_fence_init(&fence->base, &dma_buf_io_fence_ops, &fence->lock, + ctx->fence_ctx, atomic_inc_return(&ctx->fence_seq)); + map->fence =3D fence; + map->ctx =3D ctx; + return 0; +} +EXPORT_SYMBOL_NS_GPL(dma_buf_io_init_map, "DMA_BUF"); + +struct dma_buf_io_map *dma_buf_io_create_map(struct dma_buf_io_ctx *ctx) +{ + struct dma_buf *dmabuf =3D ctx->dmabuf; + struct dma_buf_io_map *map; + long ret; + +retry: + /* + * ->dmabuf_map() will be calling dma_buf_map_attachment(), for which + * we'll need to wait for fences. Do a bit nicer and try to wait + * without the resv lock first. + */ + ret =3D dma_resv_wait_timeout(dmabuf->resv, DMA_RESV_USAGE_KERNEL, + true, MAX_SCHEDULE_TIMEOUT); + if (!ret) + ret =3D -EAGAIN; + if (ret < 0) + return ERR_PTR(ret); + + ret =3D dma_resv_lock_interruptible(dmabuf->resv, NULL); + if (ret) + return ERR_PTR(ret); + + map =3D dma_buf_io_get_map(ctx); + if (map) { + ret =3D 0; + goto out; + } + + if (dma_resv_wait_timeout(dmabuf->resv, DMA_RESV_USAGE_KERNEL, + true, 0) < 0) { + dma_resv_unlock(dmabuf->resv); + goto retry; + } + + map =3D ctx->dev_ops->map(ctx); + if (IS_ERR(map)) { + ret =3D PTR_ERR(map); + goto out; + } + + if (WARN_ON_ONCE(!map->seg_shift)) + return ERR_PTR(-EFAULT); + + percpu_ref_get(&map->refs); + rcu_assign_pointer(ctx->map, map); +out: + dma_resv_unlock(dmabuf->resv); + if (ret < 0) + return ERR_PTR(ret); + return map; +} + +static void dma_buf_io_drop_map(struct dma_buf_io_ctx *ctx) +{ + struct dma_buf *dmabuf =3D ctx->dmabuf; + struct dma_buf_io_map *map; + int ret; + + dma_resv_assert_held(dmabuf->resv); + + map =3D rcu_dereference_protected(ctx->map, + dma_resv_held(dmabuf->resv)); + if (!map) + return; + rcu_assign_pointer(ctx->map, NULL); + + ret =3D dma_resv_reserve_fences(dmabuf->resv, 1); + if (WARN_ON_ONCE(ret)) { + struct dma_fence *fence =3D &map->fence->base; + + dma_fence_get(fence); + percpu_ref_kill(&map->refs); + dma_fence_wait(fence, false); + dma_fence_put(fence); + return; + } + + dma_resv_add_fence(dmabuf->resv, &map->fence->base, + DMA_RESV_USAGE_KERNEL); + /* + * Delay destruction until all inflight requests using the map are + * gone. It'll also signal the fence then. + */ + percpu_ref_kill(&map->refs); +} + +void dma_buf_io_invalidate_mappings(struct dma_buf_io_ctx *ctx) +{ + dma_buf_io_drop_map(ctx); +} +EXPORT_SYMBOL_NS_GPL(dma_buf_io_invalidate_mappings, "DMA_BUF"); + +static void dma_buf_io_ctx_release_work(struct work_struct *work) +{ + struct dma_buf_io_ctx *ctx =3D container_of(work, struct dma_buf_io_ctx, + release_work); + struct dma_buf *dmabuf =3D ctx->dmabuf; + long ret; + + dma_resv_lock(dmabuf->resv, NULL); + /* Remove the last map, there should be no new ones going forward. */ + dma_buf_io_drop_map(ctx); + dma_resv_unlock(dmabuf->resv); + + /* Wait until all maps are destroyed. */ + ret =3D dma_resv_wait_timeout(dmabuf->resv, DMA_RESV_USAGE_KERNEL, + false, MAX_SCHEDULE_TIMEOUT); + + if (WARN_ON_ONCE(ret <=3D 0)) + return; + if (WARN_ON_ONCE(rcu_dereference_protected(ctx->map, true))) + return; + + if (refcount_dec_and_test(&ctx->refs)) + dma_buf_io_ctx_destroy_work(&ctx->release_work); +} + +void dma_buf_io_ctx_release(struct dma_buf_io_ctx *ctx) +{ + /* + * Destruction needs to wait for I/O and dma fences. Defer it to + * simplify locking. + */ + INIT_WORK(&ctx->release_work, dma_buf_io_ctx_release_work); + queue_work(system_wq, &ctx->release_work); +} + +int dma_buf_io_ctx_create(struct file *file, + struct dma_buf_io_ctx *ctx, + struct dma_buf *dmabuf, + enum dma_data_direction dir) +{ + int ret; + + if (!file->f_op->init_dma_buf_io_ctx) + return -EOPNOTSUPP; + + memset(ctx, 0, sizeof(*ctx)); + ctx->fence_ctx =3D dma_fence_context_alloc(1); + ctx->dir =3D dir; + ctx->dmabuf =3D dmabuf; + refcount_set(&ctx->refs, 1); + get_dma_buf(dmabuf); + + ret =3D file->f_op->init_dma_buf_io_ctx(file, ctx); + if (ret) { + memset(ctx, 0, sizeof(*ctx)); + dma_buf_put(dmabuf); + return ret; + } + + if (WARN_ON_ONCE(!ctx->dev_ops || + !ctx->dev_ops->map || + !ctx->dev_ops->unmap || + !ctx->dev_ops->release)) + return -EINVAL; + + return ret; +} diff --git a/include/linux/dma-buf-io.h b/include/linux/dma-buf-io.h new file mode 100644 index 000000000000..f31c7375ae91 --- /dev/null +++ b/include/linux/dma-buf-io.h @@ -0,0 +1,92 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __DMA_BUF_IO_H__ +#define __DMA_BUF_IO_H__ + +#include + +struct dma_buf_io_fence; +struct dma_buf_io_ctx; +struct dma_buf_io_map; + +struct dma_buf_io_ops { + /* + * Create a new map for the given ctx. Called with the reservation + * lock held. + */ + struct dma_buf_io_map *(*map)(struct dma_buf_io_ctx *ctx); + + /* + * Clean up device specific parts of the @map. Called with the + * reservation lock held. + */ + void (*unmap)(struct dma_buf_io_ctx *ctx, struct dma_buf_io_map *map); + + /* + * The user tries to destroy the ctx. Release all device specific + * parts of the token. + */ + void (*release)(struct dma_buf_io_ctx *); +}; + +struct dma_buf_io_map { + /* + * Counts attached requests and other users. Device specific unmapping + * is deferred until all refs are dropped. + */ + struct percpu_ref refs; + unsigned seg_shift; + + struct work_struct release_work; + struct dma_buf_io_fence *fence; + struct dma_buf_io_ctx *ctx; +}; + +struct dma_buf_io_ctx { + struct dma_buf_io_map __rcu *map; + struct dma_buf *dmabuf; + enum dma_data_direction dir; + + atomic_t fence_seq; + u64 fence_ctx; + struct work_struct release_work; + refcount_t refs; + + void *dev_priv; + const struct dma_buf_io_ops *dev_ops; +}; + +int dma_buf_io_ctx_create(struct file *file, + struct dma_buf_io_ctx *ctx, + struct dma_buf *dmabuf, + enum dma_data_direction dir); +void dma_buf_io_ctx_release(struct dma_buf_io_ctx *ctx); + +struct dma_buf_io_map *dma_buf_io_create_map(struct dma_buf_io_ctx *ctx); + +static inline struct dma_buf_io_map * +dma_buf_io_get_map(struct dma_buf_io_ctx *ctx) +{ + struct dma_buf_io_map *map; + + guard(rcu)(); + + map =3D rcu_dereference(ctx->map); + if (unlikely(!map || !percpu_ref_tryget_live_rcu(&map->refs))) + return NULL; + + return map; +} + +static inline void dma_buf_io_map_drop(struct dma_buf_io_map *map) +{ + percpu_ref_put(&map->refs); +} + +/* + * Device API + */ + +void dma_buf_io_invalidate_mappings(struct dma_buf_io_ctx *ctx); +int dma_buf_io_init_map(struct dma_buf_io_ctx *ctx, struct dma_buf_io_map = *map); + +#endif diff --git a/include/linux/fs.h b/include/linux/fs.h index 50ce731a2b78..e62fa653e148 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1914,6 +1914,7 @@ struct dir_context { #define COPY_FILE_SPLICE (1 << 0) =20 struct io_uring_cmd; +struct dma_buf_io_ctx; struct offset_ctx; =20 typedef unsigned int __bitwise fop_flags_t; @@ -1962,6 +1963,7 @@ struct file_operations { int (*uring_cmd_iopoll)(struct io_uring_cmd *, struct io_comp_batch *, unsigned int poll_flags); int (*mmap_prepare)(struct vm_area_desc *); + int (*init_dma_buf_io_ctx)(struct file *, struct dma_buf_io_ctx *); } __randomize_layout; =20 /* Supports async buffered reads */ --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1CDCB35B63F for ; Sat, 1 Aug 2026 15:47:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599263; cv=none; b=iPq5tdw1Jb8ScTc7gUFzLZL8TQJrrfkvlpZOeOZSGJ8/V4jkz3sc4Nx64fNo2QN1Qfw3Ns4uqc6Bw3tsXDGD72xdf92ATQzWkBMB+4kyMjv+Dazde2TORjIPP9wLLcRX811hMt8aYCDrVtJaVP/BS63pySXRv7m1osF6C8FBRkc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599263; c=relaxed/simple; bh=Sk885w4deQUdDIZkngs+gEwd3P7jw50PmIJ0N6VNkjo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CJfr/Pc4gQhRDEASQXq+VwsC/qCARxU5RSmHJR9U1JTjSYP9hxdlASyl28pK1tFhM442kvkXhQtIcmVDOf3xjZXMwC5Nu4SzO8hn4e6iMW8bkW00ndYAdBqeT5qb08n8p9rT0v9xyFF6f65UplRI8oQDYS2yqe5pH+MzILg/btA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AnZIGVMa; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AnZIGVMa" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2d0407aedd6so21216195ad.0 for ; Sat, 01 Aug 2026 08:47:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599255; x=1786204055; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=JlAMfxnCI+4aukUkFCGEvLd7cna91lGu4ajCL8U6GrU=; b=AnZIGVMaeM7VNzhjV69IBYj69IGlN8AkwrX08snF88V+gWeBxQdHr722rrQs5E7nCQ 9BEfnfmP0tNpVRZCdLsJ+K4T6eTuutyxtmLWlD7gSp9QwXpG0uzPc6rRk00bVAfdga1P c92UpK293K23Gk7Wvn6q8b+ioEd/E7Ov5zJlC3rx7Ec5smQyqRvP3ZJOxxmaYEjSU+VS lP4i2FVHQ21Z5ordwM5KxsD3vDeT2yOZnUYvcjE/kVrJLtVzJGItCrsyqfJRXc9blS5l 2B8jZ3gZeQUnH3OS5Cva9kOFKpsJg9KRVLrfWFcAr7G31zOGaiH5ed1ChYPl50HLqYxe yQeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599255; x=1786204055; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=JlAMfxnCI+4aukUkFCGEvLd7cna91lGu4ajCL8U6GrU=; b=kTwFmF+lej5ILccKkJMrJLzSohztJLZI8GRXlRv4AS1EkR1wlEtRB0OmIKffT1/Ibs /u52WlMvVS7EaJvwFS5xXdIIYZYXJFlokH0SvM4sXPbzltOVF4QTdMZD1fHWSKMsZP4I 76zrHxnKsKo609C3vDJP9mhXQbyLyfV7Zapw9o8Ey+aKNmBiAn9rUK7CBBVSYSfH4s2D 0LULeYaoEnAGghjYF5Y1TfX/1nzscVo1WvDPYQFpEYoePEg3DO61wJGTPpg0OvK23T36 y75ldGYkjD4DUP/NOYTjzIBPzVIJxWEhQNMQ99rlA+o6wVmIQLmYjylTxS9HWddyAjT4 6yqQ== X-Forwarded-Encrypted: i=1; AHgh+RoCwBJEzKUrXluxgasoSzc5tkn+oiqOCximNT2X3P5Z6frsG18XT8qNvv0AVlOtv/jFrRdRvtUAFGkpEZM=@vger.kernel.org X-Gm-Message-State: AOJu0Yx0x0Tur7SFW2Qp1h4eX7rJdlm4qqNBWG20HiVKvpfsUOKysd3I jQYOrIBtvqtJCO9TAU6fDAsddzLy6V9Zn8OrTx35vX1UtyPAGOqWsOGq X-Gm-Gg: AR+sD12i1nc70d/wS+KNoZVCbtdyMjg6XUerT/EyKggFHBYHv1RVbWZge2bx6jFOU4f eKsNMIAGaOe0oRMMfiC8WI0V5p4lr6ULx6tQcS23oSFGaK/3b29MZ/t55GosPgqU9/OTYXanl+M ktdI68+wl0AcY/36tfNQZQ3rGIwYGdQxCsnx/GegMjZwVoPT29GKYWfx8br4famQ3k+iwlROsYi QAUbiQWS4sTPTsVx2j0Xrg3G6GdmAcq/V41n3ajqaOeTahN9qHDUhBhxvrN2vpBlb3g/CdaQE2a WuHC4Eh4wAsd7eX37VStIOyBoSS+5WmdqgtXP4gHdcsRVRZcnj6WhJnxZXC3/cNFjoaBhCygpVa JR/NFHegN36MKLKAyQpJTb3kFduU+Qy+ZgbKcObBwLIRjJPu28Vv6oDSLcyJVeTZprcXX1ZNcfn RhrroHA5dcHpC4UeWnclr++m30jdGmx21Tsn7W5oVqc5SqKDyhovT7goWMjbY8SWr4YpfRTqSjv ms8+mKEyCaiUW2mLORBNfZXbH1vv81nnnSZWDusdrcbGpn5vxR0U7rGg70= X-Received: by 2002:a17:902:d2c4:b0:2ca:e5c:7fba with SMTP id d9443c01a7336-2d0524da9d3mr35313785ad.39.1785599254851; Sat, 01 Aug 2026 08:47:34 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.47.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:47:34 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 02/16] iov_iter: add iterator type for dmabuf maps Date: Sat, 1 Aug 2026 16:46:14 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce a new iterator type for dmabuf maps. The map in an opaque object with internals and format specific to the subsystem / driver, and only it can use that subsystem / driver for issuing IO. The task of the middle layers is to pass the map / iterator further down, maybe doing basic splitting and length checking. The iterator can only be used by operations of the file the associated map was created for. Suggested-by: Keith Busch Reviewed-by: Christoph Hellwig Signed-off-by: Pavel Begunkov --- include/linux/uio.h | 11 +++++++++++ lib/iov_iter.c | 29 +++++++++++++++++++++++------ 2 files changed, 34 insertions(+), 6 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index fe2e985d74d2..638c1116e912 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -12,6 +12,7 @@ =20 struct page; struct folio_queue; +struct dma_buf_io_map; =20 typedef unsigned int __bitwise iov_iter_extraction_t; =20 @@ -29,6 +30,7 @@ enum iter_type { ITER_FOLIOQ, ITER_XARRAY, ITER_DISCARD, + ITER_DMABUF_MAP, }; =20 #define ITER_SOURCE 1 // =3D=3D WRITE @@ -71,6 +73,7 @@ struct iov_iter { const struct folio_queue *folioq; struct xarray *xarray; void __user *ubuf; + struct dma_buf_io_map *dmabuf_map; }; size_t count; }; @@ -155,6 +158,11 @@ static inline bool iov_iter_is_xarray(const struct iov= _iter *i) return iov_iter_type(i) =3D=3D ITER_XARRAY; } =20 +static inline bool iov_iter_is_dmabuf_map(const struct iov_iter *i) +{ + return iov_iter_type(i) =3D=3D ITER_DMABUF_MAP; +} + static inline unsigned char iov_iter_rw(const struct iov_iter *i) { return i->data_source ? WRITE : READ; @@ -300,6 +308,9 @@ void iov_iter_folio_queue(struct iov_iter *i, unsigned = int direction, unsigned int first_slot, unsigned int offset, size_t count); void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xa= rray *xarray, loff_t start, size_t count); +void iov_iter_dmabuf_map(struct iov_iter *i, unsigned int direction, + struct dma_buf_io_map *map, + loff_t off, size_t count); ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages, size_t maxsize, unsigned maxpages, size_t *start); ssize_t iov_iter_get_pages_alloc2(struct iov_iter *i, struct page ***pages, diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 34a52e9ba9e1..de5c1040a4a0 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -575,7 +575,8 @@ void iov_iter_advance(struct iov_iter *i, size_t size) { if (unlikely(i->count < size)) size =3D i->count; - if (likely(iter_is_ubuf(i)) || unlikely(iov_iter_is_xarray(i))) { + if (likely(iter_is_ubuf(i)) || unlikely(iov_iter_is_xarray(i)) || + unlikely(iov_iter_is_dmabuf_map(i))) { i->iov_offset +=3D size; i->count -=3D size; } else if (likely(iter_is_iovec(i) || iov_iter_is_kvec(i))) { @@ -631,7 +632,8 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll) return; } unroll -=3D i->iov_offset; - if (iov_iter_is_xarray(i) || iter_is_ubuf(i)) { + if (iov_iter_is_xarray(i) || iter_is_ubuf(i) || + iov_iter_is_dmabuf_map(i)) { BUG(); /* We should never go beyond the start of the specified * range since we might then be straying into pages that * aren't pinned. @@ -775,6 +777,20 @@ void iov_iter_xarray(struct iov_iter *i, unsigned int = direction, } EXPORT_SYMBOL(iov_iter_xarray); =20 +void iov_iter_dmabuf_map(struct iov_iter *i, unsigned int direction, + struct dma_buf_io_map *map, + loff_t off, size_t count) +{ + WARN_ON(direction & ~(READ | WRITE)); + *i =3D (struct iov_iter){ + .iter_type =3D ITER_DMABUF_MAP, + .data_source =3D direction, + .dmabuf_map =3D map, + .count =3D count, + .iov_offset =3D off, + }; +} + /** * iov_iter_discard - Initialise an I/O iterator that discards data * @i: The iterator to initialise. @@ -841,7 +857,7 @@ static unsigned long iov_iter_alignment_bvec(const stru= ct iov_iter *i) =20 unsigned long iov_iter_alignment(const struct iov_iter *i) { - if (likely(iter_is_ubuf(i))) { + if (likely(iter_is_ubuf(i)) || iov_iter_is_dmabuf_map(i)) { size_t size =3D i->count; if (size) return ((unsigned long)i->ubuf + i->iov_offset) | size; @@ -872,7 +888,7 @@ unsigned long iov_iter_gap_alignment(const struct iov_i= ter *i) size_t size =3D i->count; unsigned k; =20 - if (iter_is_ubuf(i)) + if (iter_is_ubuf(i) || iov_iter_is_dmabuf_map(i)) return 0; =20 if (WARN_ON(!iter_is_iovec(i))) @@ -1469,11 +1485,12 @@ EXPORT_SYMBOL_GPL(import_ubuf); void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state) { if (WARN_ON_ONCE(!iov_iter_is_bvec(i) && !iter_is_iovec(i) && - !iter_is_ubuf(i)) && !iov_iter_is_kvec(i)) + !iter_is_ubuf(i) && !iov_iter_is_kvec(i) && + !iov_iter_is_dmabuf_map(i))) return; i->iov_offset =3D state->iov_offset; i->count =3D state->count; - if (iter_is_ubuf(i)) + if (iter_is_ubuf(i) || iov_iter_is_dmabuf_map(i)) return; /* * For the *vec iters, nr_segs + iov is constant - if we increment --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF581329C7B for ; Sat, 1 Aug 2026 15:47:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599281; cv=none; b=XD3ljWse5+1X3JMD7h/Y6HlXFT5EQPdZF2fHHcLBvFs8Jf2SdR4si0i3YCeio4Ojk06tHpBOKWCfKF/ILxGPVVLcIeiqSCnNZfGIEJv+Dlm8S4LM43ytBgJupMUVqIXAT+PVK0iCxzM1PFcL1TnFt9WwvkeNheK5q4I/1u0LMqY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599281; c=relaxed/simple; bh=gmet9nt2AErxi+ued1v1g6ydKzgnyVvRxT1AXL/5MK4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=e7Dsk892nN07OGFD+F5B0xRpvmGMPptch9ccBShF5dt0Q2YRztvb/bbTg8nBZzH/Pd1Wsny9bVF3wtEumYHFCaABsuQtWacCSY0IeICSbMNTHzO8NdgwIMz3Uiq8SJ4km2pO4BlCewe5G57FC8kQNYYENkjL5w2xDvKLqL1e9hc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TweWA7Za; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TweWA7Za" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-2cacb8416a1so20094925ad.1 for ; Sat, 01 Aug 2026 08:47:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599276; x=1786204076; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fNwG3AVDWWb9LJ452AKZwLvgJ2n9C62K2MctNIPjTBk=; b=TweWA7ZaekXZQ5FERlNzCYd2UUYWt5gVycoYbF56VzlhrzeMa7GizQjca8EqY3NK7y 2TFA9AGERjS3tb35Je+MdZJ3Wj0B6oJhS1/S7KgPCXpg3cPzMTBlYJ2BKfgq8Az3QDXx 8fATfR3zGfvNk6HqSMR+GxKGcIq+jjGYROUVnu80Ep+KjvRXq/TDn4T829XbWXhwwuNJ A4nEO/KD5mUzblDLG0eVf725aMTCPPACfMkHmTWsFaJ8qqRMjdE3YZvg51G94hRLL0IV 5vVWPfx9G6YyQgnz4GIG3REIiSVQQLtpJEg030oo7HnqjJFR9jbjV4j4kl6xx0aZ8+oM IVng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599276; x=1786204076; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fNwG3AVDWWb9LJ452AKZwLvgJ2n9C62K2MctNIPjTBk=; b=g3AVDb7ultcjrblQVoxIStFI+gNr0/68Re7OxWj+addcvXfkZaBaDfjU/uPa7VDmZG JtKliCj3RIazfDcN9JmRUP2jq+HNgNbsDNY/Q8RT0gA/VoFWxkryZc2FshwCdpi2a6cM Wvg0txsmcRG/rFDbA2T02AXKStFBKE7jLkCt+wDoGJ5s898c3hqQZM2xYcaYAL8mMgE+ ZKAE1XXORtoAysymsdeVwYWN7kGpxO/Dc0I7ox6Ao8MzBBOYdlv9hmHqWUSN3/lIr/eA H1zM85MtYVloZ4dml3HxVyedPi9TUAAowMU0+F1HSe6WAcVCEWfAlknpqFiVr5GNBdnI BcTw== X-Forwarded-Encrypted: i=1; AHgh+RqBS49ACPw/qXRFoMMCBbgdymnrtFFnDOMvxuC1kf6+7ADZTEJrICfhTVQhLyn0MCLwG+VsI4DeFMmYa00=@vger.kernel.org X-Gm-Message-State: AOJu0YyqEpUMBh3yW6rViESKo1owq3Jwisl6rdczR0aw3HvlSffH3BlT fD9cT52jD0rFAW/JyFKLiqmmHjEhRmKGIrBKVBq47aMitd2Q736/cgpD X-Gm-Gg: AR+sD13L6aEt+LszI/lgh/yRq/UVbV+u54WXGVmAJ/03uRpweSe6s6e4vzlMfGCkqdQ lB8afzDwB9TclUSugdaR7X51117yR2UeqOVT6ubcUpQWtarHxcqYQzY9izCwGnxgXrVR1HCyzws rRrqZHFm6kJi4kedHfankLtC7nLHsHTvt6juXnKFuGoZpcgHV04kFvsWNjYA+LaTkH+8aoTEhxN dI1XgpP+QcRPDuWu39z3sy5sHBBpGS09WhPZLUGYvuOYvcCtzrdK8f4NK21oO+zjA7nRHRFDwfC pl4SAjAg+vVsi+j0bOjvF5OWPoLWAMtHgIVleoO2VOJbvSWOr9Uzcf0mBdSHb4ZinOw19EOaYEd xXlo6RCcjtMvwMOS8xmo1cjsTGDm/jeqHvYSHYHBo2z9MBeNy5kI1p6OBFIs1GoP+mOFung+isy j9UTMVF+0J2w/jdS4CRiv2GKqnMM+G6WHksa3cgISoy/UP4e5+EuljWrHUMGlwVf/dheX4/XOjh 9xDqI+2rvy84az4O3689tEcgkF/YXMbhXqUZwerdZF4HUg= X-Received: by 2002:a17:903:24e:b0:2c0:e2ea:6b0c with SMTP id d9443c01a7336-2d0521c66abmr36508465ad.21.1785599276070; Sat, 01 Aug 2026 08:47:56 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.47.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:47:55 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 03/16] block: rename bi_bvec_done Date: Sat, 1 Aug 2026 16:46:15 +0100 Message-ID: <4c5836c6b69ba764419eb5eff7b6d69a3cc348d8.1785596451.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" struct bvec_iter::bi_bvec_done is used an offset in the current bvec, let's rename it accordingly for better clarity. We're also going to use it for non-bvec based iteration, so drop the "bvec" part. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Pavel Begunkov --- Documentation/block/biovecs.rst | 8 ++++---- block/bio.c | 4 ++-- block/blk-merge.c | 8 ++++---- block/blk-mq-dma.c | 2 +- block/blk.h | 2 +- drivers/block/loop.c | 2 +- drivers/block/zloop.c | 2 +- drivers/md/dm-io-rewind.c | 10 +++++----- drivers/md/dm-pcache/segment.c | 4 ++-- drivers/nvdimm/btt.c | 2 +- drivers/nvme/host/tcp.c | 2 +- fs/btrfs/misc.h | 2 +- include/linux/bvec.h | 14 +++++++------- io_uring/net.c | 4 ++-- lib/iov_iter.c | 2 +- net/ceph/messenger.c | 4 ++-- 16 files changed, 36 insertions(+), 36 deletions(-) diff --git a/Documentation/block/biovecs.rst b/Documentation/block/biovecs.= rst index 11126ed6f40f..49da2147b7d8 100644 --- a/Documentation/block/biovecs.rst +++ b/Documentation/block/biovecs.rst @@ -16,16 +16,16 @@ bv_len by the number of bytes completed in that biovec. In the new scheme of things, everything that must be mutated in order to partially complete a bio is segregated into struct bvec_iter: bi_sector, bi_size and bi_idx have been moved there; and instead of modifying bv_offs= et -and bv_len, struct bvec_iter has bi_bvec_done, which represents the number= of +and bv_len, struct bvec_iter has bi_offset, which represents the number of bytes completed in the current bvec. =20 There are a bunch of new helper macros for hiding the gory details - in particular, presenting the illusion of partially completed biovecs so that -normal code doesn't have to deal with bi_bvec_done. +normal code doesn't have to deal with bi_offset. =20 * Driver code should no longer refer to biovecs directly; we now have bio_iovec() and bio_iter_iovec() macros that return literal struct biov= ecs, - constructed from the raw biovecs but taking into account bi_bvec_done a= nd + constructed from the raw biovecs but taking into account bi_offset and bi_size. =20 bio_for_each_segment() has been updated to take a bvec_iter argument @@ -101,7 +101,7 @@ Other implications: I.e. instead of using bio_iovec_idx() (or bio->bi_iovec[bio->bi_idx]), = you now use bio_iter_iovec(), which takes a bvec_iter and returns a literal struct bio_vec - constructed on the fly from the raw biovec but - taking into account bi_bvec_done (and bi_size). + taking into account bi_offset (and bi_size). =20 * bi_vcnt can't be trusted or relied upon by driver code - i.e. anything = that doesn't actually own the bio. The reason is twofold: firstly, it's not diff --git a/block/bio.c b/block/bio.c index 500389f332d9..4074a0496b93 100644 --- a/block/bio.c +++ b/block/bio.c @@ -229,7 +229,7 @@ void bio_init(struct bio *bio, struct block_device *bde= v, struct bio_vec *table, bio->bi_iter.bi_sector =3D 0; bio->bi_iter.bi_size =3D 0; bio->bi_iter.bi_idx =3D 0; - bio->bi_iter.bi_bvec_done =3D 0; + bio->bi_iter.bi_offset =3D 0; bio->bi_end_io =3D NULL; bio->bi_private =3D NULL; #ifdef CONFIG_BLK_CGROUP @@ -1188,7 +1188,7 @@ void bio_iov_bvec_set(struct bio *bio, const struct i= ov_iter *iter) =20 bio->bi_io_vec =3D (struct bio_vec *)iter->bvec; bio->bi_iter.bi_idx =3D 0; - bio->bi_iter.bi_bvec_done =3D iter->iov_offset; + bio->bi_iter.bi_offset =3D iter->iov_offset; bio->bi_iter.bi_size =3D iov_iter_count(iter); bio_set_flag(bio, BIO_CLONED); } diff --git a/block/blk-merge.c b/block/blk-merge.c index ab1161ca69f1..258a726071d1 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -33,7 +33,7 @@ static inline void bio_get_last_bvec(struct bio *bio, str= uct bio_vec *bv) =20 bio_advance_iter(bio, &iter, iter.bi_size); =20 - if (!iter.bi_bvec_done) + if (!iter.bi_offset) idx =3D iter.bi_idx - 1; else /* in the middle of bvec */ idx =3D iter.bi_idx; @@ -41,11 +41,11 @@ static inline void bio_get_last_bvec(struct bio *bio, s= truct bio_vec *bv) *bv =3D bio->bi_io_vec[idx]; =20 /* - * iter.bi_bvec_done records actual length of the last bvec + * iter.bi_offset records actual length of the last bvec * if this bio ends in the middle of one io vector */ - if (iter.bi_bvec_done) - bv->bv_len =3D iter.bi_bvec_done; + if (iter.bi_offset) + bv->bv_len =3D iter.bi_offset; } =20 static inline bool bio_will_gap(struct request_queue *q, diff --git a/block/blk-mq-dma.c b/block/blk-mq-dma.c index bfdb9ed70741..88fd9cbc951f 100644 --- a/block/blk-mq-dma.c +++ b/block/blk-mq-dma.c @@ -44,7 +44,7 @@ static bool blk_map_iter_next(struct request *req, struct= blk_map_iter *iter, * one could be merged into it. This typically happens when moving to * the next bio, but some callers also don't pack bvecs tight. */ - while (!iter->iter.bi_size || !iter->iter.bi_bvec_done) { + while (!iter->iter.bi_size || !iter->iter.bi_offset) { struct bio_vec next; =20 if (!__blk_map_iter_next(iter)) diff --git a/block/blk.h b/block/blk.h index eaac05815cb0..50abfd932886 100644 --- a/block/blk.h +++ b/block/blk.h @@ -406,7 +406,7 @@ static inline bool bio_may_need_split(struct bio *bio, return true; =20 bv =3D __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter); - if (bio->bi_iter.bi_size > bv->bv_len - bio->bi_iter.bi_bvec_done) + if (bio->bi_iter.bi_size > bv->bv_len - bio->bi_iter.bi_offset) return true; if ((bv->bv_offset | bv->bv_len) & lim->dma_alignment) return true; diff --git a/drivers/block/loop.c b/drivers/block/loop.c index 26d7130c3f55..8639fa34b847 100644 --- a/drivers/block/loop.c +++ b/drivers/block/loop.c @@ -379,7 +379,7 @@ static int lo_rw_aio(struct loop_device *lo, struct loo= p_cmd *cmd, iov_iter_bvec(&iter, rw, __bvec_iter_bvec(rq->bio->bi_io_vec, rq->bio->bi_iter), nr_bvec, blk_rq_bytes(rq)); - iter.iov_offset =3D rq->bio->bi_iter.bi_bvec_done; + iter.iov_offset =3D rq->bio->bi_iter.bi_offset; } atomic_set(&cmd->ref, 2); =20 diff --git a/drivers/block/zloop.c b/drivers/block/zloop.c index f97a20cfdb7c..0fef5db8e5b8 100644 --- a/drivers/block/zloop.c +++ b/drivers/block/zloop.c @@ -554,7 +554,7 @@ static int zloop_do_rw(struct zloop_cmd *cmd) iov_iter_bvec(&iter, rw, __bvec_iter_bvec(rq->bio->bi_io_vec, rq->bio->bi_iter), nr_bvec, blk_rq_bytes(rq)); - iter.iov_offset =3D rq->bio->bi_iter.bi_bvec_done; + iter.iov_offset =3D rq->bio->bi_iter.bi_offset; } =20 cmd->iocb.ki_pos =3D (cmd->sector - zone->start) << SECTOR_SHIFT; diff --git a/drivers/md/dm-io-rewind.c b/drivers/md/dm-io-rewind.c index 6155b0117c9d..04f3fc8aeb6f 100644 --- a/drivers/md/dm-io-rewind.c +++ b/drivers/md/dm-io-rewind.c @@ -16,12 +16,12 @@ static inline bool dm_bvec_iter_rewind(const struct bio= _vec *bv, int idx; =20 iter->bi_size +=3D bytes; - if (bytes <=3D iter->bi_bvec_done) { - iter->bi_bvec_done -=3D bytes; + if (bytes <=3D iter->bi_offset) { + iter->bi_offset -=3D bytes; return true; } =20 - bytes -=3D iter->bi_bvec_done; + bytes -=3D iter->bi_offset; idx =3D iter->bi_idx - 1; =20 while (idx >=3D 0 && bytes && bytes > bv[idx].bv_len) { @@ -32,13 +32,13 @@ static inline bool dm_bvec_iter_rewind(const struct bio= _vec *bv, if (WARN_ONCE(idx < 0 && bytes, "Attempted to rewind iter beyond bvec's boundaries\n")) { iter->bi_size -=3D bytes; - iter->bi_bvec_done =3D 0; + iter->bi_offset =3D 0; iter->bi_idx =3D 0; return false; } =20 iter->bi_idx =3D idx; - iter->bi_bvec_done =3D bv[idx].bv_len - bytes; + iter->bi_offset =3D bv[idx].bv_len - bytes; return true; } =20 diff --git a/drivers/md/dm-pcache/segment.c b/drivers/md/dm-pcache/segment.c index 7e9818701445..8f8816e1c539 100644 --- a/drivers/md/dm-pcache/segment.c +++ b/drivers/md/dm-pcache/segment.c @@ -14,7 +14,7 @@ int segment_copy_to_bio(struct pcache_segment *segment, =20 iov_iter_bvec(&iter, ITER_DEST, &bio->bi_io_vec[bio->bi_iter.bi_idx], bio_segments(bio), bio->bi_iter.bi_size); - iter.iov_offset =3D bio->bi_iter.bi_bvec_done; + iter.iov_offset =3D bio->bi_iter.bi_offset; if (bio_off) iov_iter_advance(&iter, bio_off); =20 @@ -35,7 +35,7 @@ int segment_copy_from_bio(struct pcache_segment *segment, =20 iov_iter_bvec(&iter, ITER_SOURCE, &bio->bi_io_vec[bio->bi_iter.bi_idx], bio_segments(bio), bio->bi_iter.bi_size); - iter.iov_offset =3D bio->bi_iter.bi_bvec_done; + iter.iov_offset =3D bio->bi_iter.bi_offset; if (bio_off) iov_iter_advance(&iter, bio_off); =20 diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c index 7e1112960d7f..5d910a64503d 100644 --- a/drivers/nvdimm/btt.c +++ b/drivers/nvdimm/btt.c @@ -1155,7 +1155,7 @@ static int btt_rw_integrity(struct btt *btt, struct b= io_integrity_payload *bip, bv =3D bvec_iter_bvec(bip->bip_vec, bip->bip_iter); /* * The 'bv' obtained from bvec_iter_bvec has its .bv_len and - * .bv_offset already adjusted for iter->bi_bvec_done, and we + * .bv_offset already adjusted for iter->bi_offset, and we * can use those directly */ =20 diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c index ba5c7b3e2a7c..ce03a0ea4ded 100644 --- a/drivers/nvme/host/tcp.c +++ b/drivers/nvme/host/tcp.c @@ -357,7 +357,7 @@ static void nvme_tcp_init_iter(struct nvme_tcp_request = *req, iov_iter_bvec(&req->iter, dir, __bvec_iter_bvec(bio->bi_io_vec, bio->bi_iter), nr_bvec, bio->bi_iter.bi_size); - req->iter.iov_offset =3D bio->bi_iter.bi_bvec_done; + req->iter.iov_offset =3D bio->bi_iter.bi_offset; } } =20 diff --git a/fs/btrfs/misc.h b/fs/btrfs/misc.h index 694be6d0562a..802060943180 100644 --- a/fs/btrfs/misc.h +++ b/fs/btrfs/misc.h @@ -74,7 +74,7 @@ static inline struct bvec_iter init_bvec_iter_for_bio(str= uct bio *bio) .bi_sector =3D 0, .bi_size =3D bio_size, .bi_idx =3D 0, - .bi_bvec_done =3D 0, + .bi_offset =3D 0, }; } =20 diff --git a/include/linux/bvec.h b/include/linux/bvec.h index 92837e2743f1..fc566ee1c1ff 100644 --- a/include/linux/bvec.h +++ b/include/linux/bvec.h @@ -110,7 +110,7 @@ struct bvec_iter { /* * Current offset in the bvec entry pointed to by `bi_idx`. */ - unsigned int bi_bvec_done; + unsigned int bi_offset; } __packed __aligned(4); =20 struct bvec_iter_all { @@ -135,14 +135,14 @@ mp_bvec_iter_page(const struct bio_vec *bvecs, const = struct bvec_iter iter) static __always_inline unsigned int mp_bvec_iter_len(const struct bio_vec *bvecs, const struct bvec_iter iter) { - return min(__bvec_iter_bvec(bvecs, iter)->bv_len - iter.bi_bvec_done, + return min(__bvec_iter_bvec(bvecs, iter)->bv_len - iter.bi_offset, iter.bi_size); } =20 static __always_inline unsigned int mp_bvec_iter_offset(const struct bio_vec *bvecs, const struct bvec_iter it= er) { - return __bvec_iter_bvec(bvecs, iter)->bv_offset + iter.bi_bvec_done; + return __bvec_iter_bvec(bvecs, iter)->bv_offset + iter.bi_offset; } =20 static __always_inline unsigned int @@ -204,7 +204,7 @@ static inline bool bvec_iter_advance(const struct bio_v= ec *bv, } =20 iter->bi_size -=3D bytes; - bytes +=3D iter->bi_bvec_done; + bytes +=3D iter->bi_offset; =20 while (bytes && bytes >=3D bv[idx].bv_len) { bytes -=3D bv[idx].bv_len; @@ -212,7 +212,7 @@ static inline bool bvec_iter_advance(const struct bio_v= ec *bv, } =20 iter->bi_idx =3D idx; - iter->bi_bvec_done =3D bytes; + iter->bi_offset =3D bytes; return true; } =20 @@ -223,13 +223,13 @@ static inline bool bvec_iter_advance(const struct bio= _vec *bv, static inline void bvec_iter_advance_single(const struct bio_vec *bv, struct bvec_iter *iter, unsigned int bytes) { - unsigned int done =3D iter->bi_bvec_done + bytes; + unsigned int done =3D iter->bi_offset + bytes; =20 if (done =3D=3D bv[iter->bi_idx].bv_len) { done =3D 0; iter->bi_idx++; } - iter->bi_bvec_done =3D done; + iter->bi_offset =3D done; iter->bi_size -=3D bytes; } =20 diff --git a/io_uring/net.c b/io_uring/net.c index a74d15f7b7d2..439c99ad1884 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -1471,7 +1471,7 @@ static int io_sg_from_iter(struct sk_buff *skb, return zerocopy_fill_skb_from_iter(skb, from, length); =20 bi.bi_size =3D min(from->count, length); - bi.bi_bvec_done =3D from->iov_offset; + bi.bi_offset =3D from->iov_offset; bi.bi_idx =3D 0; =20 while (bi.bi_size && frag < MAX_SKB_FRAGS) { @@ -1490,7 +1490,7 @@ static int io_sg_from_iter(struct sk_buff *skb, from->bvec +=3D bi.bi_idx; from->nr_segs -=3D bi.bi_idx; from->count -=3D copied; - from->iov_offset =3D bi.bi_bvec_done; + from->iov_offset =3D bi.bi_offset; =20 skb->data_len +=3D copied; skb->len +=3D copied; diff --git a/lib/iov_iter.c b/lib/iov_iter.c index de5c1040a4a0..d6f5d7e180b5 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -1651,7 +1651,7 @@ static ssize_t iov_iter_extract_bvec_pages(struct iov= _iter *i, } bi.bi_idx =3D 0; bi.bi_size =3D maxsize; - bi.bi_bvec_done =3D skip; + bi.bi_offset =3D skip; =20 maxpages =3D want_pages_array(pages, maxsize, skip, maxpages); if (!maxpages) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 34b3097b4c7b..9c1b6cf8c36f 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -762,7 +762,7 @@ static bool ceph_msg_data_bio_advance(struct ceph_msg_d= ata_cursor *cursor, if (!cursor->resid) return false; /* no more data */ =20 - if (!bytes || (it->iter.bi_size && it->iter.bi_bvec_done && + if (!bytes || (it->iter.bi_size && it->iter.bi_offset && page =3D=3D bio_iter_page(it->bio, it->iter))) return false; /* more bytes to process in this segment */ =20 @@ -817,7 +817,7 @@ static bool ceph_msg_data_bvecs_advance(struct ceph_msg= _data_cursor *cursor, if (!cursor->resid) return false; /* no more data */ =20 - if (!bytes || (cursor->bvec_iter.bi_bvec_done && + if (!bytes || (cursor->bvec_iter.bi_offset && page =3D=3D bvec_iter_page(bvecs, cursor->bvec_iter))) return false; /* more bytes to process in this segment */ =20 --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90EC721FF2A for ; Sat, 1 Aug 2026 15:48:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599299; cv=none; b=f4bwHM69EbnPWGcsfHavxi+YFaFyKB5XivB4fNwnaerLAe5hJ6wCVVAnB8Nwbkgqt1dpc2AyRVICCTf87foxhnizh9LtvZ0hfyYXtZDhA7i49UoNuou1YHbtv+jZ7vENtqwZu2Drb7GEyxnllh99aK1tasVUwD+1OILBH2gGXoM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599299; c=relaxed/simple; bh=X3zRjuSdVBD5i7gQmFjvEEAtiBuCzXBqrfu9vE0MnhU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F2fr4lma21IsvK4ZXnvIsmPpZwyxLNLxb0Py17Y4w6wV4ZGOyDMp6MoYTa52dJS5FgpQkUqMrZLpNQZtwmCms1H9o1LvLyw4yjFc48QuJYbI6jqADz5RF/GiFLhPuoiRR8jdBhjV8/uYWeEh9VapqZ+KGsMsl6iEQ8Ygm+th4Ug= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=s5Yo/707; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="s5Yo/707" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2cc891373e0so21196875ad.2 for ; Sat, 01 Aug 2026 08:48:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599297; x=1786204097; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=R0GcIDTsY2CDASlyPqk465WOkbyFB/dgtSdtycnJJZM=; b=s5Yo/707MVtufZXlHpHeR9eOrEXTbwgJPhF30vx7pJGrFzN0hryZ/O84JB91C4DJFA TbHYUGYAXTW6nMuGTNQtbiGKzpYEL73jW/itojS9/HilH4tYfy2jMvI15KYkZhiECBud CMwX7/yc3W6yyRGL1mBGzkmfAnYZ3XwdGBpwe024v1IsKk4ZBqgr7shg2iNzUU12r/pG VTCGh7jP54IJsoHCaPyqt86UdTYcW9xsmoaMvH+qVR033WqEJ6V+aeAoXboHyY87aam0 lWSJ+5dsyJmPYmlHC8trWpIu+mHIlM0FNORMvLLyxc/Zlayu05yoAXz/zvgDhti7Tz+C M7Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599297; x=1786204097; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=R0GcIDTsY2CDASlyPqk465WOkbyFB/dgtSdtycnJJZM=; b=ajkZDZzdSCQLlAU0GZS9TR1jycKXtb7GiL58fk+JXMoR33mjFkq0OL/atY6C+E9Lli 32XYeiMKw6D934YVONSgBuz2SlSldsuMBELoV78yQ2VOaPxvqxXYXi73Nl2bTYZDPHg4 p7HlX3eo3mr9bpQiSHKm8c1FYfB2w3FVeURjCf1IRMkWCs5ssvceGILgZYEqrGA17IAX SDGB6NFO4OPjdehfS85pSKmV3epN9MYwenChMELrLatlS9b5G3X9Xbw+Im6Xw2PwDHk3 aofruQVJ4UkIrCtBIHXdCbu7vpCiMo8f1qkiDn8rNS7qq8nVMC3B1U/wUmAXZxNh+ytj AxlQ== X-Forwarded-Encrypted: i=1; AHgh+RpQQzRtc7w8FpKWL4OgQ7E7HhVMRXeMjSDsuGTqKpJfQGpPOYRszQ3gxiTIQ6swH5d7fyzLZLVKhf8I/+Q=@vger.kernel.org X-Gm-Message-State: AOJu0YzpXECpBXKYjttRtzzRCurD5DSMVKA0UGMGo2bBhwnHNiEu1KFG 1njulblHRRVxblXUKe3lbXVvMleucHjJBCb7BxYUO6f6hExi24C8i7a0 X-Gm-Gg: AR+sD127fJijh9zo9F+JT+qXvxgme0nzVcEpGy8c1rvnfC4lh8b7/LaFcLc6aKxU/G3 wwIPbSXUxqLSAXeiJcZt2PduvM6uLdziFmkiyqeTRmUnzc2ZVrFE2OYgyIb00WX8QKvs7mntFoK sN6TG7dgCDLLl6bPNRvSXcBO9d1UUjrBpSzyszkOLu0iP3nMGg6NiNABfzGejNXns5k4L0THGxv CWWz7a9vK4jGoqsin/PwzT0aXLnbqKgeNggFPcMQK16+rrwRt/VegZC8RyDByMX7QGUO9Rysfpe GvJg45kvkstTorc8V/dUb3ObX8MQbEiQJUwKwd/I/1F/gUUz6b0apNZ94Ez2JNzkKIzWgml2bSb hfEIZaSMTzqb5WeL3oVSFaSNjXz4nK2IhMEnJMrPiME5lIPvK/h8wOX1MEt1FMSpcSQ9uHEK9ZH FNWyyFpzqBfaCj62Yuhv551Pj6kmCV26SKh4SF40Z5g2yiiuBUbHfoZDySHtvbYkHGvPVpFfyVR Bn7XLBqhT3QAjMfhhYqNEPGfTVh4bcBQRPw9bysLtLUVTM= X-Received: by 2002:a17:903:22c3:b0:2c0:a555:80d6 with SMTP id d9443c01a7336-2d0521bcb54mr36749555ad.2.1785599297067; Sat, 01 Aug 2026 08:48:17 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.47.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:48:16 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 04/16] block: always adjust bi_offset on bio_advance_iter Date: Sat, 1 Aug 2026 16:46:16 +0100 Message-ID: <64f559a1d1ac43880149de5950187beee6bc2aaf.1785596451.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Extend bio_no_advance_iter() iteration to also increment the offset. It's not currently needed because the currently listed request types don't have a buffer, but will be useful once we add bios backed by dma-buf, which don't have a bvec but work with a single range. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Pavel Begunkov --- drivers/md/dm-io-rewind.c | 6 ++++-- include/linux/bio.h | 12 ++++++++---- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/drivers/md/dm-io-rewind.c b/drivers/md/dm-io-rewind.c index 04f3fc8aeb6f..b22719c6411c 100644 --- a/drivers/md/dm-io-rewind.c +++ b/drivers/md/dm-io-rewind.c @@ -113,10 +113,12 @@ static inline void dm_bio_rewind_iter(const struct bi= o *bio, iter->bi_sector -=3D bytes >> 9; =20 /* No advance means no rewind */ - if (bio_no_advance_iter(bio)) + if (bio_no_advance_iter(bio)) { iter->bi_size +=3D bytes; - else + iter->bi_offset -=3D bytes; + } else { dm_bvec_iter_rewind(bio->bi_io_vec, iter, bytes); + } } =20 /** diff --git a/include/linux/bio.h b/include/linux/bio.h index 0445ecba3b24..f9b8903c6a87 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -113,11 +113,13 @@ static inline void bio_advance_iter(const struct bio = *bio, { iter->bi_sector +=3D bytes >> 9; =20 - if (bio_no_advance_iter(bio)) + if (bio_no_advance_iter(bio)) { iter->bi_size -=3D bytes; - else + iter->bi_offset +=3D bytes; + } else { bvec_iter_advance(bio->bi_io_vec, iter, bytes); /* TODO: It is reasonable to complete bio with error here. */ + } } =20 /* @bytes should be less or equal to bvec[i->bi_idx].bv_len */ @@ -127,10 +129,12 @@ static inline void bio_advance_iter_single(const stru= ct bio *bio, { iter->bi_sector +=3D bytes >> 9; =20 - if (bio_no_advance_iter(bio)) + if (bio_no_advance_iter(bio)) { iter->bi_size -=3D bytes; - else + iter->bi_offset +=3D bytes; + } else { bvec_iter_advance_single(bio->bi_io_vec, iter, bytes); + } } =20 void __bio_advance(struct bio *, unsigned bytes); --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E9B743148A7 for ; Sat, 1 Aug 2026 15:48:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599320; cv=none; b=jRnyxnk6EfnSN51NmMjl3T8XhsuzYR/KKLn0K7v0C8KJA1VZmv/FqPQORvR/lvVtclpYMNTQPuOhEbMTUi2C9iwNL+FjeLW6kkwA8wWGVt/gQYwc2JgOyM2V6rxxeQyk/+Rb3xArAyWg1MjgTOCFZlpJkR9Fqjo8AcfHPEa4xsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599320; c=relaxed/simple; bh=1jBAtnFGRJ/F8gXs/co1VF+epb8VpkeomtOw2fyD9ZA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bdoNMvHQcmNqu+3LqMrHwr1WpdbyB9+lpqXob033Mcj12zuQz7i4/gFm1gJujyKpaw0I8FiSIilWqL3aHp6cGaoDkdoGF0+uWC9GZwiayG54/ijxISuZ4YYhkfv0HkNu2LxIiLrbsmD5a1pikYKr1QTUWX9gA2t1KucAHK33na4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Mfc1+mx3; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Mfc1+mx3" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-2ce98cb8165so20866875ad.1 for ; Sat, 01 Aug 2026 08:48:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599318; x=1786204118; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=fyuZF+rseQvIVevAALiITcY143h/8lAr4kofVhDXVW8=; b=Mfc1+mx3iaWZQWiA5QbqTZ5Zw+/2gz7kxceU2jPXHLFc7Q551+c6i9znSKBY40Mkzq nNqleKOAjLfLlqvNbSdbx8mSOGzT+mcVb9LdOvSnZDurqec52Ik42xRqu3/XDqdmM6iQ phiNHZDOOoTJNIGi/wJSHcNqv9HCobynIk3XdFlR+UzJK4EA2YqSHNu3UBwuHnb9dGe8 pUZa4XywpGm7sKEwummT7fdZUxHFNHWyz49tJRuLiaKVi0UuvTQWzyX/TvZo0grubca6 AiqWBvrzLV02UC8WX+M5waI5pApeOP3t7+NQnPkP3HY62bGqbKua4X02C9O8YktU+rZy Ft9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599318; x=1786204118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=fyuZF+rseQvIVevAALiITcY143h/8lAr4kofVhDXVW8=; b=Ae7Gu20jYbMUwL6N3hpA8bAAHoGitNU0cuezRVkeX7vsXd1M//IAUrQSzQb1iEMBUP tz5sHlGqTO5n12GY1vIXAxfm2aiocy3y4nQw2Ev6dwsmg2E+7Kr0lXxuhvxRfYp2H+7C xOoWWlT4Gv5JbNdrUWCLScSYB9GEeKsRZgQDnLefQC011AaHn5nVxjnQ4ZImv3rxPsix ++N9AdtYZMEFBLrO8nSwr9ua5BU5MSD/r2iCHUzJo/+MZPOwVzBvXgifh33cextvrowr zt0VT7HL1cXCQDEjX7gQUm8qAT/Ej88HCSO4QZvcT6mfIiIp0ipS8QWvKGwLQxUf6F8V 5FXA== X-Forwarded-Encrypted: i=1; AHgh+RqnyNlqx6YG+PrR9gan/X8e9s4CjXuLH6Dql2BXbSqnRCIq4g7UZt7uFC5GYx1sXd+HghFLTQctFlsHJYA=@vger.kernel.org X-Gm-Message-State: AOJu0YzOSK2VE6zPPLN1iuZ89UQ6oo57lB0gj0gMlnSpL8VHfA7JZKLt RVCDRKM9IgRCMWUosWOwcyiCO3skjeSX6buvgS6ffDR54J/m3cyJ4b4L X-Gm-Gg: AR+sD12rsLf8bDPRATkOaHQqKkwIxoVy/ca0DcGW1TUurZkWNltZXWtyH6bKpCj+CuK 0NlGxrNgR+J/sywzGzyagFeVs4kXri4kP/2KoS0AZiTs6yD5ALnQpvCXDmEEYNNlw8tg+OyuOoy l+3Yjetn93v47zvxFaOo35gPVGoeIHXt3CM6Hf9Qi9zaFdZ7F5lELuY86dai+2Zfs/rOSrNg9JF wFzB16eJRqsBO/G4AJ7UHQw6ECbNuv3C58oefZayq5S9wBHriHNmXjVnxFuRv1H+4gggSqMaeX7 cVENydk1+L6xyEk+tXGM3rgwQmHk4fptLwvMXoEDaJmwDqER+IH7C5mU7s+yrbKkPX0PDUhUPSA LE/t1xhR9VyWiv3i5BAIK6YdJcFqrpIflMxxS5LwCuQw9pt5KrU8G5k2aZ5WIqUyMnsVfo3lROV 0iBQfyNLVlPbcWwp66ptH3VllTJq9jkjOdorIzQX9I18wo/dLTDNBYTFMtuuN5ch/TnCwfTsGiN bKbQhbOfgidN3tbXFy0Txrrdw44kzAuzcgeBc0xmsgAEi0= X-Received: by 2002:a17:903:2c0d:b0:2c7:f5c6:c4a6 with SMTP id d9443c01a7336-2d0535b3b1amr33657505ad.7.1785599318082; Sat, 01 Aug 2026 08:48:38 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.48.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:48:37 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 05/16] block: move bvec init into __bio_clone Date: Sat, 1 Aug 2026 16:46:17 +0100 Message-ID: <6ecfe8f9b1c6bfb8665fba7daf55d9ad7a8a3243.1785596451.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Consolidate bi_io_vec assignment for cloning in __bio_clone to keep any further changes in one place. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Pavel Begunkov --- block/bio.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/block/bio.c b/block/bio.c index 4074a0496b93..00f99d03ac91 100644 --- a/block/bio.c +++ b/block/bio.c @@ -860,6 +860,7 @@ static int __bio_clone(struct bio *bio, struct bio *bio= _src, gfp_t gfp) bio->bi_write_hint =3D bio_src->bi_write_hint; bio->bi_write_stream =3D bio_src->bi_write_stream; bio->bi_iter =3D bio_src->bi_iter; + bio->bi_io_vec =3D bio_src->bi_io_vec; =20 if (bio->bi_bdev) { if (bio->bi_bdev =3D=3D bio_src->bi_bdev && @@ -902,8 +903,6 @@ struct bio *bio_alloc_clone(struct block_device *bdev, = struct bio *bio_src, bio_put(bio); return NULL; } - bio->bi_io_vec =3D bio_src->bi_io_vec; - return bio; } EXPORT_SYMBOL(bio_alloc_clone); @@ -923,7 +922,7 @@ int bio_init_clone(struct block_device *bdev, struct bi= o *bio, { int ret; =20 - bio_init(bio, bdev, bio_src->bi_io_vec, 0, bio_src->bi_opf); + bio_init(bio, bdev, NULL, 0, bio_src->bi_opf); ret =3D __bio_clone(bio, bio_src, gfp); if (ret) bio_uninit(bio); --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5ADF8340A52 for ; Sat, 1 Aug 2026 15:49:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599343; cv=none; b=OxGUciMpt/E7aPGE//4oaE/PAvVmgzisTbVxIiIWcnno1VpE/0NFZNGcYFOqwXteQ3vvOW3rPO+y8hl5frYM5lszCk5T+y6En/nnlRwJKMDw7lOTxaAgU1khZPbvfRXZoO6SCbcvbn85puTMm6/Aops3fuRdU5Swlkd1twK3/Yw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599343; c=relaxed/simple; bh=l/ge8TugHemWMqrNxxHjTUj8j/KX2qzLG7MG78we1Ss=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oKYOes2HBRKtJqI2WFG6yEFL2R0YZsrx4YupQE/T5yMJlcXMt4T91xWtwUu55TPuyRCqP32Tdjtls5LmfCr92uS4jpnyof1mpibZSoJEKcaQIKuKofzmXfcx/xZfX/Kpge6hAdvsH2BGSHxxnjF5DrI/eAuFd36kR9pjwssbldk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XM9ktqES; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XM9ktqES" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2cab973140bso24785115ad.3 for ; Sat, 01 Aug 2026 08:48:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599339; x=1786204139; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7ojEbSoOEAjcpRPJZN1aYCrlxFzJ3/XljVJRrzaFsbs=; b=XM9ktqESqb3mFRQ8X89TnKjm0U7zRTnOGXTc9kenfXUtbDBAOuxVkoNQ6LpqJV+JQ2 UXuqk0Et+vWNhoXqeM6M/Ghna75QUy9aAW4TRs9LY6h1Mm0PC7lCqdpGUC/NmU0pNPZZ XDLnZo0nkIRsQ5Ymu6DGVDuRswfY3hXN3V1PkxQPPRJ4/o2cYRR43v1ARr7wJCKrh4PB e167VzWtGagxJ2ANl37Ea4YSb0TF1jbtLX/qFICqFKHShJru9NNateuhe2WkrBfQUSwV kuH0mYfGVs0GkyZoRTeFn/iPvD3y93ETWhFzy8C4wTS5q8vfdm49Jh+j9caT7z/+A8+5 a0kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599339; x=1786204139; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=7ojEbSoOEAjcpRPJZN1aYCrlxFzJ3/XljVJRrzaFsbs=; b=QeQGHrVyGi+DtFDzKjg7sBHjbrEd4VniA0vmTgjK1b/bcjlVGGAbtIzIBt5/KvKbee JblQJt/7AFw3Gt0vK/dbkE15XK+hg/c1L63JHLCFUB/TpicR/caGMsC5Xn3iqnbb4q8O w7UPVyo6U6dH7CAYCnl/Nw4d9Iqny0gMIwQZxovFvd1r4hvhdnNqT581DhmSHf6em376 CHps+Y9C4QsMNoJo/1gqaiTrCr86F+OiliCS5gfLHIEbNEReQQ1mjC9EyMkyvCqby7Md 1rk251dRaRP9yjT+Dg6MCev4RjbcjHrB8+IYwH4irJXTsQAcjLMgMAeNmysZPdIztGTY 35nQ== X-Forwarded-Encrypted: i=1; AHgh+RpVcmMeoGiXsVAF1Osme2uveqxOFpG4BnKaWBOmk+qoedZ5dM3+hiA8uGkM0V1CnWXenyI8P9lPcd/kXXE=@vger.kernel.org X-Gm-Message-State: AOJu0YyvSHd9LscDpIc6fxQBPvtJgH+RrpjcT/Z91Qnng66WYmGqDngD +2SAh6IcwTS/saEVy5a85SYvn55qOAWH+dKRV8nDwekJKOW2VHddtWJS X-Gm-Gg: AR+sD12dulkPVq0muQgV0zroq+oVRiIYl3STozQPAqApm2cX6FxNISghVpLjfPtkCqR cEH3CJqKsP7OAL+WAdNeAMw0AQ2Cto6/sSBpQv5nzZcWA+eYcjlfayvRdTky5rKsIc5r2T0MPar cc9KOU17F4r+7ksFDOMUqmg5c5dOTXiK9bb+Fjow0v6mXwEffh7uhbXchKn9YXQzaieSGaC09mQ ua0hZ3LFJ7hy+EoXnu2YNJjwn9eZcMgjkNin7a8Qi2kCYGg9mhi1hVS7RWJabaU6j7yR+P2/pQJ iepN+Jwx9532TCFzHJJkErlOgGD2s5T+Z9r/sYCUWzlFUV+BumeU+8oNhifRlCGyv1kEJizWxjs fvpXvYvN/+u/rtejvcfc4hR+p8UtyOO5r58EPUFMNOWU0ByUAPKGzQWCiibZTVMQX1EvAOZimdG m38KehS3hcukHOI4YS8icpZ9sBJ/T1NsKLqseNw7EZrk3lbpTS0sQyQnLcDtRP05H5ZEfbiJYvK tV3xgJt5h/tmtNZNZeXzOtQ0kjU9KtfLM7fbrTAYjAi7NU= X-Received: by 2002:a17:903:19c7:b0:2d0:4021:bb6b with SMTP id d9443c01a7336-2d0520362cdmr40118035ad.0.1785599339056; Sat, 01 Aug 2026 08:48:59 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.48.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:48:58 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 06/16] block: introduce bio_iov_iter_set() Date: Sat, 1 Aug 2026 16:46:18 +0100 Message-ID: <4686a0e47fc14f3f888967a80d45a6f66044f1e0.1785596451.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In preparation to supporting dma-buf backed iterators and bios, introduce bio_iov_iter_set() which attempts to set up the bio directly from the given iterator. For now, it only supports bvec and expects users to check the result and fall back to other means if fails, but later we'll add more types. Suggested-by: Christoph Hellwig Signed-off-by: Pavel Begunkov Reviewed-by: Christoph Hellwig --- block/bio.c | 13 ++++++++----- block/blk-map.c | 2 +- block/fops.c | 16 +++++++--------- include/linux/bio.h | 2 +- 4 files changed, 17 insertions(+), 16 deletions(-) diff --git a/block/bio.c b/block/bio.c index 00f99d03ac91..898b2f5ef8c8 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1181,8 +1181,11 @@ void __bio_release_pages(struct bio *bio, bool mark_= dirty) } EXPORT_SYMBOL_GPL(__bio_release_pages); =20 -void bio_iov_bvec_set(struct bio *bio, const struct iov_iter *iter) +bool bio_iov_iter_set(struct bio *bio, const struct iov_iter *iter) { + if (!iov_iter_is_bvec(iter)) + return false; + WARN_ON_ONCE(bio->bi_max_vecs); =20 bio->bi_io_vec =3D (struct bio_vec *)iter->bvec; @@ -1190,6 +1193,7 @@ void bio_iov_bvec_set(struct bio *bio, const struct i= ov_iter *iter) bio->bi_iter.bi_offset =3D iter->iov_offset; bio->bi_iter.bi_size =3D iov_iter_count(iter); bio_set_flag(bio, BIO_CLONED); + return true; } =20 /* @@ -1284,10 +1288,9 @@ int bio_iov_iter_get_pages(struct bio *bio, struct i= ov_iter *iter, if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED))) return -EIO; =20 - if (iov_iter_is_bvec(iter)) { - bio_iov_bvec_set(bio, iter); - - if (!bio_iov_bvec_aligned(bio, mem_align_mask)) + if (bio_iov_iter_set(bio, iter)) { + if (iov_iter_is_bvec(iter) && + !bio_iov_bvec_aligned(bio, mem_align_mask)) return -EINVAL; =20 iov_iter_advance(iter, bio->bi_iter.bi_size); diff --git a/block/blk-map.c b/block/blk-map.c index 615d29bb840e..9cb9605d1f62 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -473,7 +473,7 @@ static int blk_rq_map_user_bvec(struct request *rq, con= st struct iov_iter *iter) bio =3D blk_rq_map_bio_alloc(rq, 0, GFP_KERNEL); if (!bio) return -ENOMEM; - bio_iov_bvec_set(bio, iter); + bio_iov_iter_set(bio, iter); =20 ret =3D blk_rq_append_bio(rq, bio); if (ret) diff --git a/block/fops.c b/block/fops.c index 3c2099dfef1d..d11923053afe 100644 --- a/block/fops.c +++ b/block/fops.c @@ -342,15 +342,13 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb = *iocb, bio->bi_end_io =3D blkdev_bio_end_io_async; bio->bi_ioprio =3D iocb->ki_ioprio; =20 - if (iov_iter_is_bvec(iter)) { - /* - * Users don't rely on the iterator being in any particular - * state for async I/O returning -EIOCBQUEUED, hence we can - * avoid expensive iov_iter_advance(). Bypass - * bio_iov_iter_get_pages() and set the bvec directly. - */ - bio_iov_bvec_set(bio, iter); - } else { + /* + * Users don't rely on the iterator being in any particular + * state for async I/O returning -EIOCBQUEUED, hence we can + * avoid expensive iov_iter_advance(). Bypass + * bio_iov_iter_get_pages() and set the bvec directly. + */ + if (!bio_iov_iter_set(bio, iter)) { ret =3D blkdev_iov_iter_get_pages(bio, iter, bdev); if (unlikely(ret)) goto out_bio_put; diff --git a/include/linux/bio.h b/include/linux/bio.h index f9b8903c6a87..0d27e0c72905 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -522,7 +522,7 @@ int bdev_rw_virt(struct block_device *bdev, sector_t se= ctor, void *data, int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter, unsigned mem_align_mask, unsigned len_align_mask); =20 -void bio_iov_bvec_set(struct bio *bio, const struct iov_iter *iter); +bool bio_iov_iter_set(struct bio *bio, const struct iov_iter *iter); void __bio_release_pages(struct bio *bio, bool mark_dirty); extern void bio_set_pages_dirty(struct bio *bio); extern void bio_check_pages_dirty(struct bio *bio); --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BBF0D30E0DC for ; Sat, 1 Aug 2026 15:49:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599362; cv=none; b=BcnlS+GZEQ6PYdFxy4NbEtOXA1RlgegenteWtgQDtmGm5Q+t7gi/8hqWYE6lLJAYfZrS+mUL1R/5uyDv3C7EjxYJll7+osjbLR+Z1tOSvjxMr/I9YG/+90+QPX5Cl39v/UU176EUTpJLF6sHZ3eOTSiMyQi9LIPWuu/0xjltESE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599362; c=relaxed/simple; bh=mP13sIaz/uSaF6Esn0yYPf+U2z6tVI/3VUVHDykoSls=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pVeDO6SCVFwgXGzq4hcHBvW/y1A7nkYM4kxnRCo1EGJ4xGw5EmTkO6Gdku1dhy3uYlxDvyPm9QiCkYlwieTNZNbTEIfv3RHfs+nCGhXiqn85FajVyO6hCbjpjMrQ/Rr5VESpNhpLF/rDxnPns0yKSnFtr4Rvm0UAtqGQxYFPTv8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=XBPsq6sN; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XBPsq6sN" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2ced3386430so20017985ad.1 for ; Sat, 01 Aug 2026 08:49:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599360; x=1786204160; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=t+AvYci/1M+OHdtKziq8qWWexl7ZKxnQwPLL2mZLyuw=; b=XBPsq6sNGXGZBFT8foKN8feA9HtMTUedsR6cktaPmy/bwKihbuJ5V3Eqt5oOlmEFlQ t09+vDn2bP8EjrYgYGWTWeuvkYxlSzi82SrdqTnmNljl8Sfoh483KxIjnb39JlciS0BC wwj4L4gIbCE+dYWnFcmGo8J4sSKlVKCeG8WAImAUTL3WNVz6QBHjla+LFfh/W/9+Li4f xwgC7aUHqJGjeRiyjswKCGD6Wg287DXNvDfGjR40CBvw9dypITNHOS4yZjnvLpCtf8du nPMktRxs5jnkvHY5XLYHbPmIYAtCkIDA3BEGb0+5WbwQsqEO6iMOZYMtKaskzqfdNRt3 ZL/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599360; x=1786204160; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=t+AvYci/1M+OHdtKziq8qWWexl7ZKxnQwPLL2mZLyuw=; b=iyYlMtmjyVkdU7R62ngo9TW0YzFyurWeL/YM/t5zSM+2pyHk0sFGdBgDrYtEtg6Xbl jbZVl0upqksRpq6H4ZWlSgIwDG1u//QbnA2hSzW8XCunMA74lfy5G9IN5TOefu1mcRQu wt+y9Oo0XPDdtkgC/aRUgASs1H49KjSAt/VkcITjKAuNluYcJ8CVG7duov2VXn+BFYkC ASf8S0RpNCba85iGQxPGUr4M1/txowpzD2nYyddcdcfhxl1f2ynmvVwac2gDjfiVTzOs F+RfdWAf0eWgogoXjvP+RdRlkdj6RAVD04Z11vuU5sSXTydrS2JaahcmuJ3uE+UXHmgQ iLXA== X-Forwarded-Encrypted: i=1; AHgh+RpjFBkyNmixwiNDzl//fofFuxbfOodB7SxwmY4btzEg86pCMPuGrOVzrmwZndhBEno6zEmkwgg5M1BwzPQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yz622k/XdYv7V5xLtLXZeIUpkY5dfqIRJyxUOsVcaPaqk/5eJYX zBEr1kVFaZ4qfz0ke6s5EZF95bqtTxgURM9vL9oytwqCFbTSBwXcjLTB X-Gm-Gg: AR+sD10nKsfA7edsh6p01yF4d4l8sUgzk4ZQ12ECF9DZcCDs3VVcfNxOohpEqFbaSsT +Jjc9+N+QZjobBiYBCXyELfmkM7e2vtGOO0iOmra4blURMX6IsQfIpleWWEwnmrxlJ9zB57rzLN jdhSmZDmh9yWzwvmKUHGwM2Wo0CnQpmem4RZW80i4+cPCqSaJeTSRcvJ1rASDjbysR0JTXZQtJe BUAFljnJl7HL9Tgkd0lLVoHNjUXPe+qYnK2VbLWhqpT+K5T+amF+oyH9hggt6NEdBhRTTlbmYDx qLTXmxAZkaphNUBTkky/GJL+Oa/ZFM5mWhKStfjASpzRwJmFwZV1o90jj2671iGUB2BU4CZatDh WssY9kqf7UBFKopEoHk2Hsqte4DzmZjCNWdXVBORixw/amqZD2dSPIldJNLr1y0xFnNXFXsRCFz HFel0ivEwmOZY62EFkCsvoTo11o3UVRlPrFUbisoBHe39Ce2LgBCgi97zXRLAKdMcGV3xeJ5DHE VSH6C5GYUesftsgFyT24vgfhLmtx04ICayDLcCUTjOOZ3w= X-Received: by 2002:a17:903:1450:b0:2ca:ca48:c380 with SMTP id d9443c01a7336-2d05243a2e9mr42273125ad.47.1785599360127; Sat, 01 Aug 2026 08:49:20 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.48.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:49:19 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 07/16] block: introduce dma map backed bio type Date: Sat, 1 Aug 2026 16:46:19 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Premapped buffers don't require a generic bio_vec since these have already been dma mapped. Repurpose the bi_io_vec space to strore dmabuf maps as they are mutually exclusive. Suggested-by: Keith Busch Signed-off-by: Pavel Begunkov --- block/bio.c | 15 +++++++++++++-- block/blk-merge.c | 37 +++++++++++++++++++++++++++++++++++++ block/fops.c | 2 +- include/linux/bio.h | 9 +++++---- include/linux/blk-mq.h | 7 +++++++ include/linux/blk_types.h | 14 +++++++++++++- include/linux/bvec.h | 3 ++- 7 files changed, 78 insertions(+), 9 deletions(-) diff --git a/block/bio.c b/block/bio.c index 898b2f5ef8c8..1602eab05761 100644 --- a/block/bio.c +++ b/block/bio.c @@ -860,7 +860,11 @@ static int __bio_clone(struct bio *bio, struct bio *bi= o_src, gfp_t gfp) bio->bi_write_hint =3D bio_src->bi_write_hint; bio->bi_write_stream =3D bio_src->bi_write_stream; bio->bi_iter =3D bio_src->bi_iter; - bio->bi_io_vec =3D bio_src->bi_io_vec; + + if (op_is_dmabuf(bio->bi_opf)) + bio->bi_dmabuf_map =3D bio_src->bi_dmabuf_map; + else + bio->bi_io_vec =3D bio_src->bi_io_vec; =20 if (bio->bi_bdev) { if (bio->bi_bdev =3D=3D bio_src->bi_bdev && @@ -1183,16 +1187,23 @@ EXPORT_SYMBOL_GPL(__bio_release_pages); =20 bool bio_iov_iter_set(struct bio *bio, const struct iov_iter *iter) { - if (!iov_iter_is_bvec(iter)) + if (!iov_iter_is_bvec(iter) && !iov_iter_is_dmabuf_map(iter)) return false; =20 WARN_ON_ONCE(bio->bi_max_vecs); =20 + static_assert(offsetof(struct bio, bi_io_vec) =3D=3D + offsetof(struct bio, bi_dmabuf_map)); + static_assert(offsetof(struct iov_iter, bvec) =3D=3D + offsetof(struct iov_iter, dmabuf_map)); + bio->bi_io_vec =3D (struct bio_vec *)iter->bvec; bio->bi_iter.bi_idx =3D 0; bio->bi_iter.bi_offset =3D iter->iov_offset; bio->bi_iter.bi_size =3D iov_iter_count(iter); bio_set_flag(bio, BIO_CLONED); + if (iov_iter_is_dmabuf_map(iter)) + bio->bi_opf |=3D REQ_NOMERGE | REQ_DMABUF; return true; } =20 diff --git a/block/blk-merge.c b/block/blk-merge.c index 258a726071d1..1beedc42a85d 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -9,6 +9,7 @@ #include #include #include +#include =20 #include =20 @@ -319,6 +320,28 @@ static inline unsigned int bvec_seg_gap(struct bio_vec= *bvprv, return bv->bv_offset | (bvprv->bv_offset + bvprv->bv_len); } =20 +static inline int bio_split_io_at_dmabuf(struct bio *bio, + const struct queue_limits *lim, unsigned *segs, + unsigned max_bytes, unsigned len_align_mask, + unsigned start_align_mask) +{ + unsigned bytes =3D min(bio->bi_iter.bi_size, max_bytes); + unsigned seg_shift =3D bio->bi_dmabuf_map->seg_shift; + unsigned offset =3D bio->bi_iter.bi_offset & ((1U << seg_shift) - 1); + + if ((bio->bi_iter.bi_offset & start_align_mask) || + (bio->bi_iter.bi_size & len_align_mask)) + return -EINVAL; + + /* single contiguous range into the dma-buf */ + *segs =3D 1; + + bytes =3D min(bytes, ((unsigned)lim->max_segments << seg_shift) - offset); + if (bytes !=3D bio->bi_iter.bi_size) + return bytes; + return 0; +} + /** * bio_split_io_at - check if and where to split a bio * @bio: [in] bio to be split @@ -346,6 +369,19 @@ int bio_split_io_at(struct bio *bio, const struct queu= e_limits *lim, len_align_mask |=3D (bc->bc_key->crypto_cfg.data_unit_size - 1); } =20 + if (op_is_dmabuf(bio->bi_opf)) { + int ret; + + ret =3D bio_split_io_at_dmabuf(bio, lim, &nsegs, max_bytes, + len_align_mask, start_align_mask); + if (ret < 0) + return ret; + if (!ret) + goto out; + bytes =3D ret; + goto split; + } + bio_for_each_bvec(bv, bio, iter) { if (bv.bv_offset & start_align_mask || bv.bv_len & len_align_mask) @@ -376,6 +412,7 @@ int bio_split_io_at(struct bio *bio, const struct queue= _limits *lim, bvprvp =3D &bvprv; } =20 +out: *segs =3D nsegs; bio->bi_bvec_gap_bit =3D ffs(gaps); return 0; diff --git a/block/fops.c b/block/fops.c index d11923053afe..56bbacf6e317 100644 --- a/block/fops.c +++ b/block/fops.c @@ -346,7 +346,7 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb *i= ocb, * Users don't rely on the iterator being in any particular * state for async I/O returning -EIOCBQUEUED, hence we can * avoid expensive iov_iter_advance(). Bypass - * bio_iov_iter_get_pages() and set the bvec directly. + * bio_iov_iter_get_pages() and set the bvec/dmabuf directly. */ if (!bio_iov_iter_set(bio, iter)) { ret =3D blkdev_iov_iter_get_pages(bio, iter, bdev); diff --git a/include/linux/bio.h b/include/linux/bio.h index 0d27e0c72905..22ce9deb2feb 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -80,7 +80,8 @@ static inline bool bio_no_advance_iter(const struct bio *= bio) { return bio_op(bio) =3D=3D REQ_OP_DISCARD || bio_op(bio) =3D=3D REQ_OP_SECURE_ERASE || - bio_op(bio) =3D=3D REQ_OP_WRITE_ZEROES; + bio_op(bio) =3D=3D REQ_OP_WRITE_ZEROES || + op_is_dmabuf(bio->bi_opf); } =20 static inline void *bio_data(struct bio *bio) @@ -438,12 +439,12 @@ static inline void bio_wouldblock_error(struct bio *b= io) =20 /* * Calculate number of bvec segments that should be allocated to fit data - * pointed by @iter. If @iter is backed by bvec it's going to be reused - * instead of allocating a new one. + * pointed by @iter. If @iter is backed by a bvec or a dmabuf, the bvec ar= ray / + * the dma map are going to be reused, and so no extra allocation is requi= red. */ static inline int bio_iov_vecs_to_alloc(struct iov_iter *iter, int max_seg= s) { - if (iov_iter_is_bvec(iter)) + if (iov_iter_is_bvec(iter) || iov_iter_is_dmabuf_map(iter)) return 0; return iov_iter_npages(iter, max_segs); } diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index af878597afb8..7c7504c84e09 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -1017,6 +1017,13 @@ static inline void *blk_mq_rq_to_pdu(struct request = *rq) return rq + 1; } =20 +static inline bool blk_mq_rq_is_dmabuf(struct request *rq) +{ + if (!IS_ENABLED(CONFIG_DMA_SHARED_BUFFER)) + return false; + return rq->bio && op_is_dmabuf(rq->bio->bi_opf); +} + static inline struct blk_mq_hw_ctx *queue_hctx(struct request_queue *q, in= t id) { struct blk_mq_hw_ctx *hctx; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index d49d97a050d0..a305951f0312 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -233,7 +233,12 @@ struct bio { atomic_t __bi_remaining; =20 /* The actual vec list, preserved by bio_reset() */ - struct bio_vec *bi_io_vec; + union { + struct bio_vec *bi_io_vec; + /* Driver specific dma map, valid IFF REQ_DMABUF is set */ + struct dma_buf_io_map *bi_dmabuf_map; + }; + struct bvec_iter bi_iter; =20 union { @@ -402,6 +407,7 @@ enum req_flag_bits { __REQ_DRV, /* for driver use */ __REQ_FS_PRIVATE, /* for file system (submitter) use */ __REQ_ATOMIC, /* for atomic write operations */ + __REQ_DMABUF, /* Using premmaped dma buffers */ /* * Command specific flags, keep last: */ @@ -434,6 +440,7 @@ enum req_flag_bits { #define REQ_DRV (__force blk_opf_t)(1ULL << __REQ_DRV) #define REQ_FS_PRIVATE (__force blk_opf_t)(1ULL << __REQ_FS_PRIVATE) #define REQ_ATOMIC (__force blk_opf_t)(1ULL << __REQ_ATOMIC) +#define REQ_DMABUF (__force blk_opf_t)(1ULL << __REQ_DMABUF) =20 #define REQ_NOUNMAP (__force blk_opf_t)(1ULL << __REQ_NOUNMAP) =20 @@ -487,6 +494,11 @@ static inline bool op_is_discard(blk_opf_t op) return (op & REQ_OP_MASK) =3D=3D REQ_OP_DISCARD; } =20 +static inline bool op_is_dmabuf(blk_opf_t op) +{ + return op & REQ_DMABUF; +} + /* * Check if a bio or request operation is a zone management operation. */ diff --git a/include/linux/bvec.h b/include/linux/bvec.h index fc566ee1c1ff..b63914ff56e3 100644 --- a/include/linux/bvec.h +++ b/include/linux/bvec.h @@ -108,7 +108,8 @@ struct bvec_iter { unsigned int bi_idx; =20 /* - * Current offset in the bvec entry pointed to by `bi_idx`. + * Current offset in the bvec entry pointed to by `bi_idx` or into + * a dma-buf map. */ unsigned int bi_offset; } __packed __aligned(4); --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD03224DCF6 for ; Sat, 1 Aug 2026 15:49:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599383; cv=none; b=WAEUTyqcT/jmU9YVym/c2u5Sk3/9aPpg+69jTX8VqBBk5Kr2dGGFDkI8I8pXIXWPZzwIqD4BJtBPHsb+0FU0y6TRFHOvRwxT2KbVTOwAmsv0Q+jFBV71XgvWHAgsOCU35x6qXQuoP1QfGoGbZ4+6Y/n0rSwcQH7AbV4UW+2FFDg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599383; c=relaxed/simple; bh=g0r+cM2oJU8p0aW6ejF1reDF155X4adWxCLLoLHu2uw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CDeiCOIfXbuxJi9Uz5Pa1kuKX7cXLfME4VjaRPYnu32mQNGXnH/Ko3e5VWBz9eGq+LDTzQUtKE6C/vp7Rmbm3l08EC6jnuiqzGVTLJouI5yqPqE8z1gOpNrS6CLjJi4sVuhmObdELQsID6QtTJbvlEM1n91pItWs7hYSr++5e+c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Phykw574; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Phykw574" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2cc7e86e7aeso19736945ad.2 for ; Sat, 01 Aug 2026 08:49:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599381; x=1786204181; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=XTGwQGVc0bqlcGV09Qbia8ah/Xj1ofLPkaifBOkmHiQ=; b=Phykw574YQXrFOsxNwwUAuKFs97gDWG/x7biAdTYNgmBmxuHcAoz8mYAokHEBqiEQ9 CDk8ydDMzaSdwlmktajb3Js/dgO6pOjiVb6rCrJLLWnPvQn530GyjGo6bWqXfns7sy4K SIy/ncATqOp1kmsB2cpn5bZkIeJSu4/+fMz8JBCr14qSRz6S3a+i/XXknL3WLz2ogeuq Jn/br9Op2rMlMvzLg1TZrj+S0mngigDj2OLcKfYHeC1vXPy1i71TBGDL79FWylg/yU18 BJbtBRhV0oU2RC94desVnsFnNHzKwbyH9k8khD3hojHwuF+4pN1bcRiM4NVhrw8s9piA 6Jpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599381; x=1786204181; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=XTGwQGVc0bqlcGV09Qbia8ah/Xj1ofLPkaifBOkmHiQ=; b=LdMtDfsDx2n9Od0vuw0bugjusvTy8M0hjyY6b0WuJ6mEv1rtYS3eOw5vrA8PUvz7sA OdqYvVSdAPsetibkhzAWQ9R0MbnCcaaU+N7M6KqUJ3IRS9pLZ7db5vK+cUF7Uq402seJ P+k1NISePe1ExnTk8SZLQSX7q5JSdcNj9R6FREo/laoJ2ZAIjXaOuuPepAIq80+PZ2u+ IaatiOchIZEtNe/o8ZRmZE+qrau8iV86oc8o3udfzyp+I/QSsIeJWga8NtDDV8ZnrIKd KGLMwjnD/sC54i6uui/1cmTKpp59UrhqI/1nyJmNOjjQaEUiODlE7yKLPB27GKV14vXm H/SQ== X-Forwarded-Encrypted: i=1; AHgh+RoIGKpaNvUGZZ8F++/YFV8YWvOKAnMVtIs5r1h+ns8uTHgMuM1cwR6dcYMvgx3JQ7x6NiUQZ0DrE9vdsq0=@vger.kernel.org X-Gm-Message-State: AOJu0Yzbdq7HfsbptD3DWNGEdlBNhEEw2MhrfBDjhrflAuU7Zu+4CzLx 9aY53n3UpzKgpUgckxPALdnUYwLyoeh5qmYLBdP64nhf+xWOCal2xUrg X-Gm-Gg: AR+sD12EzGtsaAF3/mdYiBYyCgVu0p4tAi7Aov6Q42va+FNIfCU/7ZOA630/F78g3+B RDqaj1aAPWgjWpSejOJYmp1K2DznuiFTppnhk3qVhD+I/ts1gq0pPKPn3RiP1lKiG8lN9ZZR2kf rxYmqgUb72JQd1ex3gcM8apr/dcN6dcH2hFxPhM7J/yLhO5c0urB10qJ9ozPm4rp7p7HQ92peBC r3UhRQ4CxlDbfPHz67HDYW63/KPrt+ifekSAVhaVbw0M+Hp8I5nJ7RNH0TC3kBq4JeVCkdhUtXV yX9vewclPDruSO2K1nW4BdyCDyoEXHMt14qCeA7WunLtLTgqeW7SOu2IrglwTjpCBycifp8Z1sQ OlMdWP4aGN2btMjJ909JRYJqrAr5qit4ZxVwLg0YZdt+J8ejeI75jcHkG0IsDpZHL/baLxU+5TJ 4YLlc6JixxEKNvwjIOlsFgjbfCKsQl5/OzWgs0qq5pqAOiUQUyqnJKQKwovAqTUCSUcTuDQGTch btskUlr4PciPDowEmaE/GxToAG7ECYgrtnJDDPiim50dTE= X-Received: by 2002:a17:903:2450:b0:2cf:9347:f445 with SMTP id d9443c01a7336-2d05219f5c9mr36286255ad.10.1785599381195; Sat, 01 Aug 2026 08:49:41 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.49.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:49:40 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 08/16] block: add dma-buf support for raw bdev Date: Sat, 1 Aug 2026 16:46:20 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a simple proxy implementation of init_dma_buf_io_ctx() forwarding the call to a new struct block_device_operations operation. Also reject dma-buf backed iterators for buffered IO. Reviewed-by: Christoph Hellwig [pavel: reject dma-buf without O_DIRECT] Signed-off-by: Pavel Begunkov --- block/fops.c | 19 +++++++++++++++++++ include/linux/blkdev.h | 2 ++ 2 files changed, 21 insertions(+) diff --git a/block/fops.c b/block/fops.c index 56bbacf6e317..3c403afb0f64 100644 --- a/block/fops.c +++ b/block/fops.c @@ -767,6 +767,9 @@ static ssize_t blkdev_write_iter(struct kiocb *iocb, st= ruct iov_iter *from) ret =3D direct_write_fallback(iocb, from, ret, blkdev_buffered_write(iocb, from)); } else { + if (unlikely(iov_iter_is_dmabuf_map(from))) + return -EOPNOTSUPP; + /* * Take i_rwsem and invalidate_lock to avoid racing with * set_blocksize changing i_blkbits/folio order and punching @@ -821,6 +824,8 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, str= uct iov_iter *to) if (ret < 0 || !count) goto reexpand; } + if (unlikely(iov_iter_is_dmabuf_map(to))) + return -EOPNOTSUPP; =20 /* * Take i_rwsem and invalidate_lock to avoid racing with set_blocksize @@ -924,6 +929,19 @@ static int blkdev_mmap_prepare(struct vm_area_desc *de= sc) return generic_file_mmap_prepare(desc); } =20 +static int blkdev_init_dma_buf_io_ctx(struct file *file, + struct dma_buf_io_ctx *ctx) +{ + struct block_device *bdev =3D file_bdev(file); + struct gendisk *disk =3D bdev->bd_disk; + + if (!(file->f_flags & O_DIRECT)) + return -EINVAL; + if (!disk->fops->init_dma_buf_io_ctx) + return -EINVAL; + return disk->fops->init_dma_buf_io_ctx(bdev, ctx); +} + const struct file_operations def_blk_fops =3D { .open =3D blkdev_open, .release =3D blkdev_release, @@ -942,6 +960,7 @@ const struct file_operations def_blk_fops =3D { .fallocate =3D blkdev_fallocate, .uring_cmd =3D blkdev_uring_cmd, .fop_flags =3D FOP_BUFFER_RASYNC | FOP_DONTCACHE, + .init_dma_buf_io_ctx =3D blkdev_init_dma_buf_io_ctx, }; =20 static __init int blkdev_init(void) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 9213a5716f95..9f4c92e51dc9 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1682,6 +1682,8 @@ struct block_device_operations { /* returns the length of the identifier or a negative errno: */ int (*get_unique_id)(struct gendisk *disk, u8 id[16], enum blk_unique_id id_type); + int (*init_dma_buf_io_ctx)(struct block_device *, + struct dma_buf_io_ctx *); struct module *owner; const struct pr_ops *pr_ops; =20 --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B8DB21A42D for ; Sat, 1 Aug 2026 15:50:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599405; cv=none; b=Z6uqnH5U7uB7qkP+BszASgVGD15EXyp++tp63JVDtyWLLe27ygITNUQY+7PxDK8Nb77tVwCS2YgMwh94LsjqiIwzajb9j/KaWcLSJZOdhgYRqAmmgE7W1hLQKPJ0rx0VvxUOhQecUXoqbImSyn/X6+j+sGER4Zuwixs5Mx39/es= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599405; c=relaxed/simple; bh=QiYI1PT4002/zHbFYDjhl9RAIZDQ6iU9y7QUyjTJBIA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tXZae9TOCsSasbhST/yBbwOzQyGhRZeJtiP15f73vgVRvctOMn8C6RBxVUAeVo4T7Jdhc2luSyLbebOnjmYIxsMyl92NstFQ2Nz5yeglXjAkTeQ/VjjdRV/up5sJoA/AWnvRxlF/sCnM3QPu1ycGWlgYmtO4U7TaqhnCjUSTSew= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=p/RFl8er; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="p/RFl8er" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-2d049069377so14210905ad.0 for ; Sat, 01 Aug 2026 08:50:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599403; x=1786204203; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=3jtuIHi3+Rs23Pr4RqXoeVIdwK8MwXNBLfA4ntKbnrQ=; b=p/RFl8ereMEv0eGvjeLLWbOG3WErNbjFmUD2P/iePZjbKclNaK5OcDKNeMhOSniYH8 6KIg6F6AI9fH9IRK45viDXPiLE+2Hp42YGRY5TWt7s/g3J8PwNG0NPyqGj6WHTfMDUFy GAMDsvH50jpNeMs6Xuw4C6uunMHB89tJnIrJkx942u0GKl3e3lD5eFUcnDIuxqyep8CP oci1DpcaFYarpbM+eshEx+CymIXC2R9pSmLlPfss26h9XdoF+I4BeD7ltJIdBCzp3euP qJ0VR5Zs2/fLswluc/F4UwksjCooPX5rPX1YgGnkE9lUQsDtAsTmemSYNkoqE60hNo/R MrRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599403; x=1786204203; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=3jtuIHi3+Rs23Pr4RqXoeVIdwK8MwXNBLfA4ntKbnrQ=; b=CazZ0PUYo8sRNs77M8KBNsgWozQ6kXri0ldLXWOpmo1pKAENjv3e/oEEQbgkI9yWug ulcT7kfgbn8xmfywwFGfftEHBYTgqmqL8eY/mHmhgYRSJhpRkdqvU8loNEKs47khYL/x 696kBh0WWroY6B62PhhzUA68GFEqXEOBFrwUMx+IQi1RJTFPKz4Y5BNVEd6m1VTdtt9D GtevFgugegBCCSO+IRcxk5XN40DoSpKcMIJBbr0uu0gZ0XG7C7JZA4XB+rKH/EvHHlK0 hjM56C3Ea5zd/DU05/fMYhYreoDyfS6GC57U2blJ9Mq3t3zkd3/c55bk4S25lv2Kk1QP TVPg== X-Forwarded-Encrypted: i=1; AHgh+Rq9MPkIT1l6dciNsbV52yoX3VjLLurhJopKotPa8gP3KqESIi065rhpfacCuY++tbbxhpY4laZvPHF+HUw=@vger.kernel.org X-Gm-Message-State: AOJu0YziTkhKSjD76ASX1CzSXEQ/ktX/NHcUxijPLN/vYiMGfMPoezTL O4pMUnL/ZOI6H48hsZG2xOUMK5gb6V5Fm/xnMCBotfIQtViguU22UXun X-Gm-Gg: AR+sD10zv6J1itCpUuPfjnXVHXzI+UF/wtRm2jM8LODfPWtTy3vD68spP83YFg3dCgX 6/adY6szaY1WRtmxBDgmJv6/nz2FBzq67jzAN8mO3VwNHIAmm3TXIl67Vvll0oZqv+vEOXqig5P Vy8lrcBOPX8WgoIhKXkwYsMV2b5msUuZT3EUHsbYkzbK/K2eLZgbVSj5rMX402i9ivxLtw/xDO0 g8HZmZqtSkOQxWqcGcOH3RQFrR0dTCxXQTtP6q4wDAe/bq/75sItrQPj2yIE12iK3LSASCdMgMJ VMsVal8LIO1xEh4gyova+pDU+18fyIhnUwlqWA9+yb+uHNeDy/hUPrOSIHjFwFRpI9rlmN3Zt7t QQsG9NAo4XfSh0G9cqXZ6WzST5ftBhMXJ20ezwywZG0vmoWlCXUGwxaHxjvPtWiK4zh2HRRsqsi QiqWn3ItSLSDmZ6XjkYCxRisPPDPiiMXgvCREbSqImwK+Cop5vx0hUQiWSDVCso52bbx/rWIXsU TnNLVtK9aDvPSYtk518ajR47TcQWjLGpxi2R9cYXAW6Yro= X-Received: by 2002:a17:903:32c5:b0:2c6:a981:b591 with SMTP id d9443c01a7336-2d0523bd6b9mr37885665ad.30.1785599402798; Sat, 01 Aug 2026 08:50:02 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.49.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:50:02 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 09/16] nvme-pci: implement dma-buf backed requests Date: Sat, 1 Aug 2026 16:46:21 +0100 Message-ID: <9e3dd1b2d71dc0f9bd556b6a6d6e48fb6a8d727b.1785596451.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enable BIO_DMABUF_MAP backed requests. On registration we map the dma-buf and store it as a prp list, which is then used to initialise requests. Suggested-by: Keith Busch Signed-off-by: Pavel Begunkov --- drivers/nvme/host/core.c | 12 ++ drivers/nvme/host/nvme.h | 2 + drivers/nvme/host/pci.c | 265 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 279 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 453c1f0b2dd0..8bd407603d92 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -2676,6 +2676,17 @@ static int nvme_report_zones(struct gendisk *disk, s= ector_t sector, #define nvme_report_zones NULL #endif /* CONFIG_BLK_DEV_ZONED */ =20 +static int nvme_init_dma_buf_io_ctx(struct block_device *bdev, + struct dma_buf_io_ctx *ctx) +{ + struct nvme_ns *ns =3D bdev->bd_disk->private_data; + struct nvme_ctrl *ctrl =3D ns->ctrl; + + if (!ctrl->ops->init_dma_buf_io_ctx) + return -EINVAL; + return ctrl->ops->init_dma_buf_io_ctx(ctrl, ctx); +} + const struct block_device_operations nvme_bdev_ops =3D { .owner =3D THIS_MODULE, .ioctl =3D nvme_ioctl, @@ -2686,6 +2697,7 @@ const struct block_device_operations nvme_bdev_ops = =3D { .get_unique_id =3D nvme_get_unique_id, .report_zones =3D nvme_report_zones, .pr_ops =3D &nvme_pr_ops, + .init_dma_buf_io_ctx =3D nvme_init_dma_buf_io_ctx, }; =20 static int nvme_wait_ready(struct nvme_ctrl *ctrl, u32 mask, u32 val, diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 824651cc898d..034c31af8fec 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -652,6 +652,8 @@ struct nvme_ctrl_ops { int (*get_address)(struct nvme_ctrl *ctrl, char *buf, int size); void (*print_device_info)(struct nvme_ctrl *ctrl); bool (*supports_pci_p2pdma)(struct nvme_ctrl *ctrl); + int (*init_dma_buf_io_ctx)(struct nvme_ctrl *ctrl, + struct dma_buf_io_ctx *ctx); unsigned long (*get_virt_boundary)(struct nvme_ctrl *ctrl, bool is_admin); }; =20 diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 69932d640b53..5608ede6c479 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -27,6 +27,8 @@ #include #include #include +#include +#include =20 #include "trace.h" #include "nvme.h" @@ -393,6 +395,13 @@ struct nvme_queue { struct completion delete_done; }; =20 +struct nvme_dmabuf_map { + struct dma_buf_io_map base; + struct sg_table *sgt; + unsigned nr_entries; + dma_addr_t dma_list[]; +}; + /* bits for iod->flags */ enum nvme_iod_flags { /* this command has been aborted by the timeout handler */ @@ -859,6 +868,138 @@ static void nvme_free_descriptors(struct request *req) } } =20 +static inline struct nvme_dmabuf_map * +to_nvme_dmabuf_map(struct dma_buf_io_map *map) +{ + return container_of(map, struct nvme_dmabuf_map, base); +} + +static void nvme_dmabuf_map_sync_for_cpu(struct nvme_dev *nvme_dev, + struct request *req) +{ + struct device *dev =3D nvme_dev->dev; + enum dma_data_direction dma_dir; + struct bio *bio =3D req->bio; + struct nvme_dmabuf_map *map =3D to_nvme_dmabuf_map(bio->bi_dmabuf_map); + dma_addr_t *dma_list =3D map->dma_list; + unsigned offset =3D bio->bi_iter.bi_offset; + unsigned map_idx =3D offset / NVME_CTRL_PAGE_SIZE; + int length =3D blk_rq_payload_bytes(req) + + (offset & (NVME_CTRL_PAGE_SIZE - 1)); + + dma_dir =3D rq_data_dir(req) =3D=3D READ ? DMA_FROM_DEVICE : DMA_TO_DEVIC= E; + + while (length > 0) { + dma_sync_single_for_cpu(dev, dma_list[map_idx++], + NVME_CTRL_PAGE_SIZE, dma_dir); + length -=3D NVME_CTRL_PAGE_SIZE; + } +} + +static void nvme_dmabuf_map_sync_for_device(struct nvme_dev *nvme_dev, + struct request *req) +{ + struct device *dev =3D nvme_dev->dev; + enum dma_data_direction dma_dir; + struct bio *bio =3D req->bio; + struct nvme_dmabuf_map *map =3D to_nvme_dmabuf_map(bio->bi_dmabuf_map); + dma_addr_t *dma_list =3D map->dma_list; + unsigned offset =3D bio->bi_iter.bi_offset; + unsigned map_idx =3D offset / NVME_CTRL_PAGE_SIZE; + int length =3D blk_rq_payload_bytes(req) + + (offset & (NVME_CTRL_PAGE_SIZE - 1)); + + dma_dir =3D rq_data_dir(req) =3D=3D READ ? DMA_FROM_DEVICE : DMA_TO_DEVIC= E; + + while (length > 0) { + dma_sync_single_for_device(dev, dma_list[map_idx++], + NVME_CTRL_PAGE_SIZE, dma_dir); + length -=3D NVME_CTRL_PAGE_SIZE; + } +} + +static void nvme_rq_clean_dmabuf_map(struct nvme_dev *dev, + struct request *req) +{ + struct nvme_iod *iod =3D blk_mq_rq_to_pdu(req); + + nvme_dmabuf_map_sync_for_cpu(dev, req); + + if (!(iod->flags & IOD_SINGLE_SEGMENT)) + nvme_free_descriptors(req); +} + +static blk_status_t nvme_rq_setup_dmabuf_map(struct request *req, + struct nvme_queue *nvmeq) +{ + struct nvme_iod *iod =3D blk_mq_rq_to_pdu(req); + struct bio *bio =3D req->bio; + struct nvme_dmabuf_map *map =3D to_nvme_dmabuf_map(bio->bi_dmabuf_map); + unsigned bvec_done =3D bio->bi_iter.bi_offset; + unsigned map_idx =3D bvec_done / NVME_CTRL_PAGE_SIZE; + unsigned offset =3D bvec_done & (NVME_CTRL_PAGE_SIZE - 1); + int length =3D blk_rq_payload_bytes(req) - (NVME_CTRL_PAGE_SIZE - offset); + dma_addr_t *dma_list =3D map->dma_list; + u64 prp1_dma =3D dma_list[map_idx++] + offset; + u64 dma_addr, prp2_dma; + dma_addr_t prp_dma; + __le64 *prp_list; + unsigned i; + + nvme_dmabuf_map_sync_for_device(nvmeq->dev, req); + + if (length <=3D 0) { + prp2_dma =3D 0; + goto done; + } + + if (length <=3D NVME_CTRL_PAGE_SIZE) { + prp2_dma =3D dma_list[map_idx]; + goto done; + } + + if (DIV_ROUND_UP(length, NVME_CTRL_PAGE_SIZE) <=3D + NVME_SMALL_POOL_SIZE / sizeof(__le64)) + iod->flags |=3D IOD_SMALL_DESCRIPTOR; + + prp_list =3D dma_pool_alloc(nvme_dma_pool(nvmeq, iod), GFP_ATOMIC, + &prp_dma); + if (!prp_list) + return BLK_STS_RESOURCE; + + iod->descriptors[iod->nr_descriptors++] =3D prp_list; + prp2_dma =3D prp_dma; + i =3D 0; + for (;;) { + if (i =3D=3D NVME_CTRL_PAGE_SIZE >> 3) { + __le64 *old_prp_list =3D prp_list; + + prp_list =3D dma_pool_alloc(nvmeq->descriptor_pools.large, + GFP_ATOMIC, &prp_dma); + if (!prp_list) + goto free_prps; + iod->descriptors[iod->nr_descriptors++] =3D prp_list; + prp_list[0] =3D old_prp_list[i - 1]; + old_prp_list[i - 1] =3D cpu_to_le64(prp_dma); + i =3D 1; + } + + dma_addr =3D dma_list[map_idx++]; + prp_list[i++] =3D cpu_to_le64(dma_addr); + + length -=3D NVME_CTRL_PAGE_SIZE; + if (length <=3D 0) + break; + } +done: + iod->cmd.common.dptr.prp1 =3D cpu_to_le64(prp1_dma); + iod->cmd.common.dptr.prp2 =3D cpu_to_le64(prp2_dma); + return BLK_STS_OK; +free_prps: + nvme_free_descriptors(req); + return BLK_STS_RESOURCE; +} + static void nvme_free_prps(struct request *req, unsigned int attrs) { struct nvme_iod *iod =3D blk_mq_rq_to_pdu(req); @@ -937,6 +1078,11 @@ static void nvme_unmap_data(struct request *req) struct device *dma_dev =3D nvmeq->dev->dev; unsigned int attrs =3D 0; =20 + if (blk_mq_rq_is_dmabuf(req)) { + nvme_rq_clean_dmabuf_map(nvmeq->dev, req); + return; + } + if (iod->flags & IOD_SINGLE_SEGMENT) { static_assert(offsetof(union nvme_data_ptr, prp1) =3D=3D offsetof(union nvme_data_ptr, sgl.addr)); @@ -1251,6 +1397,9 @@ static blk_status_t nvme_map_data(struct request *req) struct blk_dma_iter iter; blk_status_t ret; =20 + if (blk_mq_rq_is_dmabuf(req)) + return nvme_rq_setup_dmabuf_map(req, nvmeq); + /* * Try to skip the DMA iterator for single segment requests, as that * significantly improves performances for small I/O sizes. @@ -2269,6 +2418,118 @@ static int nvme_create_queue(struct nvme_queue *nvm= eq, int qid, bool polled) return result; } =20 +#ifdef CONFIG_DMA_SHARED_BUFFER +static void nvme_dmabuf_invalidate_mappings(struct dma_buf_attachment *att= ach) +{ + struct dma_buf_io_ctx *ctx =3D attach->importer_priv; + + dma_buf_io_invalidate_mappings(ctx); +} + +const struct dma_buf_attach_ops nvme_dmabuf_importer_ops =3D { + .invalidate_mappings =3D nvme_dmabuf_invalidate_mappings, + .allow_peer2peer =3D true, +}; + +static struct dma_buf_io_map *nvme_dma_buf_io_map(struct dma_buf_io_ctx *c= tx) +{ + unsigned nr_entries =3D ctx->dmabuf->size / NVME_CTRL_PAGE_SIZE; + struct dma_buf_attachment *attach =3D ctx->dev_priv; + unsigned long tmp, i =3D 0; + unsigned seg_shift =3D ~0U; + struct nvme_dmabuf_map *map; + struct scatterlist *sg; + struct sg_table *sgt; + int ret; + + dma_resv_assert_held(ctx->dmabuf->resv); + + map =3D kmalloc_flex(*map, dma_list, nr_entries); + if (!map) + return ERR_PTR(-ENOMEM); + + sgt =3D dma_buf_map_attachment(attach, ctx->dir); + if (IS_ERR(sgt)) { + ret =3D PTR_ERR(sgt); + sgt =3D NULL; + goto err; + } + + for_each_sgtable_dma_sg(sgt, sg, tmp) { + dma_addr_t dma_addr =3D sg_dma_address(sg); + unsigned long sg_len =3D sg_dma_len(sg); + + if (sg_len % NVME_CTRL_PAGE_SIZE) { + ret =3D -EINVAL; + goto err; + } + seg_shift =3D min(seg_shift, __ffs(sg_len)); + + while (sg_len) { + map->dma_list[i++] =3D dma_addr; + dma_addr +=3D NVME_CTRL_PAGE_SIZE; + sg_len -=3D NVME_CTRL_PAGE_SIZE; + } + } + + if (WARN_ON_ONCE(seg_shift < NVME_CTRL_PAGE_SHIFT)) + return ERR_PTR(-EFAULT); + + ret =3D dma_buf_io_init_map(ctx, &map->base); + if (ret) + goto err; + map->base.seg_shift =3D seg_shift; + map->nr_entries =3D nr_entries; + map->sgt =3D sgt; + return &map->base; +err: + if (sgt) + dma_buf_unmap_attachment(attach, sgt, ctx->dir); + kfree(map); + return ERR_PTR(ret); +} + +static void nvme_dma_buf_io_unmap(struct dma_buf_io_ctx *ctx, + struct dma_buf_io_map *map_base) +{ + struct dma_buf_attachment *attach =3D ctx->dev_priv; + struct nvme_dmabuf_map *map =3D to_nvme_dmabuf_map(map_base); + + dma_resv_assert_held(ctx->dmabuf->resv); + + dma_buf_unmap_attachment(attach, map->sgt, ctx->dir); +} + +static void nvme_dma_buf_io_release(struct dma_buf_io_ctx *ctx) +{ + struct dma_buf_attachment *attach =3D ctx->dev_priv; + + dma_buf_detach(ctx->dmabuf, attach); +} + +const struct dma_buf_io_ops nvme_dma_buf_io_ops =3D { + .map =3D nvme_dma_buf_io_map, + .unmap =3D nvme_dma_buf_io_unmap, + .release =3D nvme_dma_buf_io_release, +}; + +static int nvme_pci_init_dma_buf_io_ctx(struct nvme_ctrl *ctrl, + struct dma_buf_io_ctx *ctx) +{ + struct dma_buf_attachment *attach; + struct nvme_dev *dev =3D to_nvme_dev(ctrl); + + attach =3D dma_buf_dynamic_attach(ctx->dmabuf, dev->dev, + &nvme_dmabuf_importer_ops, ctx); + if (IS_ERR(attach)) + return PTR_ERR(attach); + + ctx->dev_priv =3D attach; + ctx->dev_ops =3D &nvme_dma_buf_io_ops; + return 0; +} +#endif + static const struct blk_mq_ops nvme_mq_admin_ops =3D { .queue_rq =3D nvme_queue_rq, .complete =3D nvme_pci_complete_rq, @@ -3563,6 +3824,9 @@ static const struct nvme_ctrl_ops nvme_pci_ctrl_ops = =3D { .print_device_info =3D nvme_pci_print_device_info, .supports_pci_p2pdma =3D nvme_pci_supports_pci_p2pdma, .get_virt_boundary =3D nvme_pci_get_virt_boundary, +#ifdef CONFIG_DMA_SHARED_BUFFER + .init_dma_buf_io_ctx =3D nvme_pci_init_dma_buf_io_ctx, +#endif }; =20 static int nvme_dev_map(struct nvme_dev *dev) @@ -4327,5 +4591,6 @@ MODULE_AUTHOR("Matthew Wilcox = "); MODULE_LICENSE("GPL"); MODULE_VERSION("1.0"); MODULE_DESCRIPTION("NVMe host PCIe transport driver"); +MODULE_IMPORT_NS("DMA_BUF"); module_init(nvme_init); module_exit(nvme_exit); --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB10E2D7DE9 for ; Sat, 1 Aug 2026 15:50:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599426; cv=none; b=BaLxBIua+1fnRVY0tfo3mgTE5sHA//+n5xRqVv9V9aLbVLE/xpcel4VgOzs9b3KzTYpMrjjdr5NgxPROjxkuV1fjN0WdLMQFDM6pWav13uEI+qYHcs8Bv70ojhQlv5Ju1d2Wr9Kagxl+myhOUyWW9mQk25iChecOuaiYtSTt/kU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599426; c=relaxed/simple; bh=5jZAhWEaWbKaEJ0h/lpCMDFZsTZrSyaTXCzcWYUXgLU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qrWuqPus9QXA+M/paOdVLYpvZbxbE7zAbIY65DGC3aGadcYwTVPXlhhjc+IZVzz75o3bQC4fk9X1S3hSg7+ro3S+/K1foVgsQyoh0GbhYemSojCrjMtt+bOscUcpLpQ/XvUw8aJpIZtBaCOd2Bvu5fxQJv10gHxcJIxLxundHlA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=S8M7ui1M; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="S8M7ui1M" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2cf6d65d8a7so26950295ad.0 for ; Sat, 01 Aug 2026 08:50:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599424; x=1786204224; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=UQw568s99USSdIytlPm2Vpoz5MduETOEjKRltqiPXcQ=; b=S8M7ui1M6pVxwnF0ZWZcibVPh5VuBiUtN70UqmuTG0Pxj4auY5LB2yJ662kz75HeQh x6aa3YQ2w2cUCj1H/zgsw6w1614lui4k+ZtkM9Vrm8osQ2givsBkxawLreVDmJBZDtOI oEhuDISM7a1bYmDAhhRvlsjqAZE0ncxnAljIYA6w7ZRh0nT13QZ7VX1YHBpovz8LBd2V Q6mTrtObaxBNmR5w3/6QDirbDoaQQCOn+Li3nMoJKEVUTILZpjSv0nYOkobL49RNX2fV vXloOWCNX0Rs8+vWRbN9OulBzWIBEUN0+tcLVfjbwC4TSMDjmfvjQ3w+fXzj7ez5uUzi Qgpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599424; x=1786204224; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=UQw568s99USSdIytlPm2Vpoz5MduETOEjKRltqiPXcQ=; b=gpGVED+Q4AotV7wsdX9ANslwS0IPxNCTIWkl3fxt9Xdbke8YVMUMwt8zyvems7vcxD CJrrryW8ayqfQALfTCT/nTDNJkGX4CcfduTPwwjw5CX4zW5YxQ0iYhtf5Ym5igC6n2Mq 5NTyYvKgBcq0jBll+E3ny0bGcD5TrnucB1G0HhO2AGhlvq3nnEI+hIe9PDiDILE/3WwU 7m3lfyufUqCCeP3sf0gh+lW2SYOePdkcEY/tZ1OhIwWfao0qu3A3T9FnspwSMCwCZQc9 /oDkNV1phhPgq1TS3CUjiCcf9rEI1EJE8H4xeNkr8kKgip5Pl8Ah68DUGelfRi+4JaXH L4DA== X-Forwarded-Encrypted: i=1; AHgh+RrR7c8TWu7g+O/CqUcCjZ/9q33SsQ8qiCkfkuSS2eyArYj3CFSBK/zheDNTfdnhEuDJ2Fi6dBr2vMJkVqI=@vger.kernel.org X-Gm-Message-State: AOJu0YxeV9V0JWd4QoS1J7SM13iax5mqxwsoLvaEANs3vy6Hn09NHYQG zeF+TWV/2JWltrR7plJK951cC8keoQVpZ2ZYGGEi8wxzSNrI+F7A6kh8 X-Gm-Gg: AR+sD11Wd46tGLkdy2icZKhszZ2+mywuLnpT8aHxf5qpgYNxgDe+chZwBQjxjKW0m7Q om6yIaXin8UsBpPgoyf744QuOvBzgoyvn+8zbZwQJEQvYCfXkJGB1PR/v8c36/1FlzhSAc8xtGK c/k7tVJzIpQp8FWAH/OBaRkFNlrqRXdu3dJakp9vTdvvgZBpax9bXByPLUsCSQz/9o2p1fwT2i0 5qRaEHttfC1dg0WSb9qNN6Omr83wH6Ws2AnXqIWJnhUyeRyTFdupwGTaFBi8MYFcbZNJnS9YF4Z bc2Be5Bl6Is9ZJppPUqK8CSDpJtSqbGPHEnOUArkV9+Ce6BYq0rKQMenIzDDNeRLn6yWv3SKv41 l/5TRJ5L0EsepGPxMfWJYWB5Uo6bd68yopo+7MuSZ1qwLG47cuM3iLGSyVFNT7gzbL8qHAxrnQS B7mjUw+8BIk9vmi7hB2K/zUDcxI6StzhhLPzZRkmTDdZ6CPHEyk/rng3Saht2cRXw9As5YB92YA 40p9ERowUOAdFoj5p025ZvxbBEjaizXMmzvCW9xqrqD8Ym4+QsthNmaRQ== X-Received: by 2002:a17:902:d2cf:b0:2ca:a03a:29b2 with SMTP id d9443c01a7336-2d0521a5657mr42155635ad.8.1785599424386; Sat, 01 Aug 2026 08:50:24 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.50.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:50:23 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 10/16] nvme-pci: rename nvme_pci_sgl_set_data to nvme_pci_dma_iter_set_sgl Date: Sat, 1 Aug 2026 16:46:22 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Anuj Gupta Rename the blk_dma_iter based nvme_pci_sgl_set_data() to nvme_pci_dma_iter_set_sgl(). Suggested-by: Christoph Hellwig Signed-off-by: Anuj Gupta Signed-off-by: Pavel Begunkov Reviewed-by: Christoph Hellwig --- drivers/nvme/host/pci.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 5608ede6c479..7b8ed101a13b 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -1287,7 +1287,7 @@ static blk_status_t nvme_pci_setup_data_prp(struct re= quest *req, return BLK_STS_IOERR; } =20 -static void nvme_pci_sgl_set_data(struct nvme_sgl_desc *sge, +static void nvme_pci_dma_iter_set_sgl(struct nvme_sgl_desc *sge, struct blk_dma_iter *iter) { sge->addr =3D cpu_to_le64(iter->addr); @@ -1317,7 +1317,7 @@ static blk_status_t nvme_pci_setup_data_sgl(struct re= quest *req, iod->cmd.common.flags =3D NVME_CMD_SGL_METABUF; =20 if (entries =3D=3D 1 || blk_rq_dma_map_coalesce(&iod->dma_state)) { - nvme_pci_sgl_set_data(&iod->cmd.common.dptr.sgl, iter); + nvme_pci_dma_iter_set_sgl(&iod->cmd.common.dptr.sgl, iter); iod->total_len +=3D iter->len; return BLK_STS_OK; } @@ -1339,7 +1339,7 @@ static blk_status_t nvme_pci_setup_data_sgl(struct re= quest *req, iter->status =3D BLK_STS_IOERR; break; } - nvme_pci_sgl_set_data(&sg_list[mapped++], iter); + nvme_pci_dma_iter_set_sgl(&sg_list[mapped++], iter); iod->total_len +=3D iter->len; } while (blk_rq_dma_map_iter_next(req, nvmeq->dev->dev, iter)); =20 @@ -1502,13 +1502,13 @@ static blk_status_t nvme_pci_setup_meta_iter(struct= request *req) iod->cmd.common.metadata =3D cpu_to_le64(sgl_dma); if (entries =3D=3D 1) { iod->meta_total_len =3D iter.len; - nvme_pci_sgl_set_data(sg_list, &iter); + nvme_pci_dma_iter_set_sgl(sg_list, &iter); return BLK_STS_OK; } =20 sgl_dma +=3D sizeof(*sg_list); do { - nvme_pci_sgl_set_data(&sg_list[++i], &iter); + nvme_pci_dma_iter_set_sgl(&sg_list[++i], &iter); iod->meta_total_len +=3D iter.len; } while (blk_rq_integrity_dma_map_iter_next(req, dev->dev, &iter)); =20 --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CDDD326928 for ; Sat, 1 Aug 2026 15:50:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599448; cv=none; b=cP4I7twLDOyw+KJwYfckKpLOJuLNyn/pviMPQvDd5tAQkbMdJwN+E7UUOkdbK712ajxuecvbGefw+SDd/VfDz5xq4WPOQKMmsq4Hv4/OQ5v47dXE9LYfpqTu1h1gii0RANqEGd1Omx7NjXkKv+hD/DebTdy94ohebNcT3usyzg0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599448; c=relaxed/simple; bh=x+OUwPvED0A9WpApYgDpbJAjUumA7PzzdtZ3Hd4LWNk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WHNim8gY0cd3hFRCrevlMBQ4L9WybHmrrV5vKDIdR9b1nhBDRvcEfTUW1E4CbLG+OFIgW2aUhCsaqHrg8/BDB8zikYrfB+w5Lj/0EgDQskseSSsAnoV8snS9ckQljYwoEdlHbte3Cvl+SOlu0StW8p6IPpN1N5INIWrfdpc3r6k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jAQnFFZE; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jAQnFFZE" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2cedda2ce6fso16513805ad.1 for ; Sat, 01 Aug 2026 08:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599445; x=1786204245; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=rSOyaa4+UsoEyQf3LWEsLm65BesmXLGHIq97BZD3zcI=; b=jAQnFFZEe92YzJjuyl5ExhN/f4TgMsnYItGtGyaqfnzkv0mxjQh5jKP8KtGKTZy4zi +S4cIe/HlKY4pic324hCiieASW84ic48OH6M8Uk8WqyuQ/bjQN5bfgN6Vm6lgWxPceSE lXHzxCs6ZCvP6wms2CcRIHT/z2hJIP+KbfSJnKcteJ8QiNdrEYekaCV3IOyltgalLwmV 6DYYxvWUdEUB8SLv+i81otpdSdoSV5I41yy2yjDjMpmufl1Hyu7HT1ydE1QV0C/qf6Bt T8ZEVkB+HBw0BpWcbjouOErJx0mQSy8HHsqW2Iysv0hEJzml1m2RsDPT4cGQmECa0wGg RDFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599445; x=1786204245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=rSOyaa4+UsoEyQf3LWEsLm65BesmXLGHIq97BZD3zcI=; b=HcXvSnO8ZrfIGbp6fxhtIt/p+MW7kVB1YkGPTjQT+0IhHFXWFHawDgulGMFqBeSzhB J5vHFavjgLOpm2BDMYE1HO8m6mDUuKfY+eODl1cQxWZUsX9m3sF9Tn4I4Gx3X+PIN8Di Icrmqrz2tCjsg0zjc5cD78GYBUYoMjuVDedj558F4q5eriI0r4Z1Eq/6MmixwTts7GhS 3nCPTEa5hwrE3ccyooelRruS2aLtccVntYhEXWbdlHxiQIv9VBJmg/YsY+WPaVJ3mtEb EQIRSTDrAN8q3WSE+tH5grO3Ij2U4Qv4djZbBn2Wle1pKW5ygSo2FhImVcV0IZOFgdbm Hm0Q== X-Forwarded-Encrypted: i=1; AHgh+Ro4vpk4rcpI7YSCc2JcfJCfZ52ElWW9eTrB4Xk3OkQxn2vbTqI9wKHcn7rgXfS5bl3DikHrhz8gWMcHZMQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzfeXe1VWljZa2mvYVC/4xe4vPZKivPAZ6KbI2x5SX7/bqKdTsw xoOIZKFf4KKXZUBEuWNo2hoC3DNZsBrc553hS3PaSMONvh2bsh5XefNx6WjQD5+a X-Gm-Gg: AR+sD124nfBvYImRu6Baf8trTkJS4bz0U3z1rVKlN49zbseU8beEUzNSXl4aGWzwIQ6 llyP8Z8eYe4diuNeBNmM4TmgC/vl9kfqpeR5VAPxLX4pFpLVvRTiiocL98QspONnUqsy+RTYzR1 afaRZEsSk87rI6JfJjKYTNh6etWs8GqQG9CDiduI7WeBaAL3z1+KBGIAeNdo9TEW4iBl67+2MBG OQY27FF4/bBNa3H6cphRodU998pt+TF/u9I8el6HYtnjDYfvLVN/o+LUYEqGY5FnFqIe1T3Hnd6 hNXFoUusQUr3LC+fRIBOaW46pnTLT8LP6Y4CS/hNeBeHTXci+7Nu/QAe2Dz/JVdKtF4hgEPR5Fl d7yTfjln0/nrz+TZmjJjmW9nsBZXzHM3J3jujiEJLyPGO8gycoTneeRZbpFZQVCkILxaqRu1P1L EGpt1PFIS0BXicMkhAl8itCsICrnCFHfUKAyAnRzpmBjDuf7yE8KenIMXcTU7LaLwCBcFmZyCcg rA6GZZlzba1JNdIOg4I7oQSf4IvowhdR7COsefLioX/sa4= X-Received: by 2002:a17:903:1b43:b0:2ca:d820:b9a6 with SMTP id d9443c01a7336-2d05242b8a0mr39311295ad.23.1785599445373; Sat, 01 Aug 2026 08:50:45 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.50.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:50:44 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 11/16] nvme-pci: add SGL support for the dmabuf path Date: Sat, 1 Aug 2026 16:46:23 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Anuj Gupta Add SGL support in addition to PRP for dmabuf-backed requests, building the descriptor list from the mapping's sg_table. Signed-off-by: Anuj Gupta Signed-off-by: Pavel Begunkov Reviewed-by: Christoph Hellwig --- drivers/nvme/host/pci.c | 153 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 149 insertions(+), 4 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 7b8ed101a13b..67e26d6e8f25 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -1287,12 +1287,18 @@ static blk_status_t nvme_pci_setup_data_prp(struct = request *req, return BLK_STS_IOERR; } =20 +static void nvme_pci_sgl_set_data(struct nvme_sgl_desc *sge, + dma_addr_t addr, u32 len) +{ + sge->addr =3D cpu_to_le64(addr); + sge->length =3D cpu_to_le32(len); + sge->type =3D NVME_SGL_FMT_DATA_DESC << 4; +} + static void nvme_pci_dma_iter_set_sgl(struct nvme_sgl_desc *sge, struct blk_dma_iter *iter) { - sge->addr =3D cpu_to_le64(iter->addr); - sge->length =3D cpu_to_le32(iter->len); - sge->type =3D NVME_SGL_FMT_DATA_DESC << 4; + nvme_pci_sgl_set_data(sge, iter->addr, iter->len); } =20 static void nvme_pci_sgl_set_seg(struct nvme_sgl_desc *sge, @@ -1303,6 +1309,145 @@ static void nvme_pci_sgl_set_seg(struct nvme_sgl_de= sc *sge, sge->type =3D NVME_SGL_FMT_LAST_SEG_DESC << 4; } =20 +static unsigned int nvme_pci_dmabuf_sgl_nents(struct request *req) +{ + struct bio *bio =3D req->bio; + struct nvme_dmabuf_map *map =3D to_nvme_dmabuf_map(bio->bi_dmabuf_map); + struct scatterlist *sg; + unsigned long tmp; + size_t offset =3D bio->bi_iter.bi_offset; + size_t remaining =3D blk_rq_payload_bytes(req); + unsigned int nents =3D 0; + + for_each_sgtable_dma_sg(map->sgt, sg, tmp) { + size_t sg_len =3D sg_dma_len(sg); + + if (!remaining) + break; + if (offset >=3D sg_len) { + offset -=3D sg_len; + continue; + } + + sg_len -=3D offset; + offset =3D 0; + + do { + size_t chunk =3D min(remaining, sg_len); + + nents++; + sg_len -=3D chunk; + remaining -=3D chunk; + } while (sg_len && remaining); + } + + if (unlikely(remaining)) + return 0; + + return nents; +} + +static blk_status_t nvme_rq_setup_dmabuf_sgl(struct request *req, + struct nvme_queue *nvmeq, unsigned int entries) +{ + struct nvme_iod *iod =3D blk_mq_rq_to_pdu(req); + struct bio *bio =3D req->bio; + struct nvme_dmabuf_map *map =3D to_nvme_dmabuf_map(bio->bi_dmabuf_map); + size_t length =3D blk_rq_payload_bytes(req); + struct nvme_sgl_desc *sg_list =3D &iod->cmd.common.dptr.sgl; + bool pooled =3D entries > 1; + dma_addr_t sgl_dma =3D 0; + unsigned int mapped =3D 0; + unsigned long tmp; + struct scatterlist *sg; + size_t offset =3D bio->bi_iter.bi_offset; + size_t remaining =3D length; + + if (!entries) + return BLK_STS_IOERR; + + iod->cmd.common.flags =3D NVME_CMD_SGL_METABUF; + iod->total_len =3D length; + + nvme_dmabuf_map_sync_for_device(nvmeq->dev, req); + + /* entries =3D=3D 1 fits in the inline descriptor; more needs a pool. */ + if (pooled) { + if (entries <=3D NVME_SMALL_POOL_SIZE / sizeof(*sg_list)) + iod->flags |=3D IOD_SMALL_DESCRIPTOR; + + sg_list =3D dma_pool_alloc(nvme_dma_pool(nvmeq, iod), GFP_ATOMIC, + &sgl_dma); + if (!sg_list) + return BLK_STS_RESOURCE; + iod->descriptors[iod->nr_descriptors++] =3D sg_list; + } + + for_each_sgtable_dma_sg(map->sgt, sg, tmp) { + size_t sg_len =3D sg_dma_len(sg); + dma_addr_t addr =3D sg_dma_address(sg); + + if (!remaining) + break; + if (offset >=3D sg_len) { + offset -=3D sg_len; + continue; + } + + addr +=3D offset; + sg_len -=3D offset; + offset =3D 0; + + do { + u32 chunk =3D min_t(size_t, remaining, sg_len); + + if (WARN_ON_ONCE(mapped =3D=3D entries)) + goto err_free; + nvme_pci_sgl_set_data(&sg_list[mapped++], addr, chunk); + + addr +=3D chunk; + sg_len -=3D chunk; + remaining -=3D chunk; + } while (sg_len && remaining); + } + + if (unlikely(remaining)) + goto err_free; + + if (pooled) + nvme_pci_sgl_set_seg(&iod->cmd.common.dptr.sgl, sgl_dma, + mapped); + return BLK_STS_OK; +err_free: + if (pooled) { + iod->nr_descriptors--; + dma_pool_free(nvme_dma_pool(nvmeq, iod), sg_list, sgl_dma); + } + return BLK_STS_IOERR; +} + +static blk_status_t nvme_rq_setup_dmabuf(struct request *req, + struct nvme_queue *nvmeq, enum nvme_use_sgl use_sgl) +{ + unsigned int entries; + size_t avg_seg; + + if (use_sgl =3D=3D SGL_UNSUPPORTED) + return nvme_rq_setup_dmabuf_map(req, nvmeq); + + entries =3D nvme_pci_dmabuf_sgl_nents(req); + + if (use_sgl =3D=3D SGL_FORCED) + return nvme_rq_setup_dmabuf_sgl(req, nvmeq, entries); + + avg_seg =3D entries ? + DIV_ROUND_UP(blk_rq_payload_bytes(req), entries) : 0; + if (sgl_threshold && avg_seg >=3D sgl_threshold) + return nvme_rq_setup_dmabuf_sgl(req, nvmeq, entries); + + return nvme_rq_setup_dmabuf_map(req, nvmeq); +} + static blk_status_t nvme_pci_setup_data_sgl(struct request *req, struct blk_dma_iter *iter) { @@ -1398,7 +1543,7 @@ static blk_status_t nvme_map_data(struct request *req) blk_status_t ret; =20 if (blk_mq_rq_is_dmabuf(req)) - return nvme_rq_setup_dmabuf_map(req, nvmeq); + return nvme_rq_setup_dmabuf(req, nvmeq, use_sgl); =20 /* * Try to skip the DMA iterator for single segment requests, as that --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CCEF3148A7 for ; Sat, 1 Aug 2026 15:51:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599468; cv=none; b=gqdLDwnUfE1N9V2SFDZ2cPScTfwJWIKctwe/G/Ubf89vwjEGlyk7zhPOeFg54b3p5dJrrIoFg2XqotLjUrsShDicfdXnXM4fKFIK/iS4gUx9cdkG194AIuXSFi0Nmdv98ggZ49mVn3tSD44mnCqnHs3DzgQNKGg2POUNpUEv8Uw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599468; c=relaxed/simple; bh=oZ3BFAy9cLxeqXthNExBNBD2GMUwMIJfwvoNUFcW7pw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ptyyrrFTcntJwIs7PFfcCvW4In49gYHYAQW+sNu0U6JTXTLZVz4YXEbEWO+pDpHJHAXAm/pAc9WlQtm9atgPfMeepKLjq5MdiQ9KVlcOoaebnPyQLtJMZbVVZoBLVlXYos+0eqMPXBe0Ss43kYqUdXOk5WqlGN7sK9/HlUUdwTg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hTbetJY9; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hTbetJY9" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2cace91f112so21812915ad.0 for ; Sat, 01 Aug 2026 08:51:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599466; x=1786204266; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/6LlJ8gBFfbSI101GpfmueMUxhwNSOZCGEsyX/W5uwU=; b=hTbetJY9iLmkAok3xbBOyDoHCromjW74NdSsghiynUF1G9VDZcJQlX+D6XPTP9xrfE g8Zqvku7qkq5mDv7Zdmm85ifBGnCgAkIix9iSN84hxDThi5JYoaTQ/2RtFM4G9+mIXxB g0mnjEH0C//eQBda+9B8EkGEs48Vhef06stN3G1aEW9CDmeUQDwRp84f2yEx8/y+eMbP QdeWQUqWPridDHMU6UKjqTq7EHmxKgvL4KSuJyIGYtE9MVD8XcXvCj3O93DC2wYMcUfx 1MxFpI4xuHsgvNTtTE0fcAhuEyIjR/2ZKukPK3lFskBs5y2qxf4rm8UW6kv0mN9NvpFm /J0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599466; x=1786204266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=/6LlJ8gBFfbSI101GpfmueMUxhwNSOZCGEsyX/W5uwU=; b=WT5V840wUuLR8ojN1XnR/AXJsFXM4gz5VFRcGaRBj7TCEyX84SZmbB6IgxcV7R4Etr KtOK/Kd6PPs6xRPvU34N3nrs02iVxsFG8jhCjnyWoFqKu8lCAPwGEIh6K8luAXX9y6Se 7OSFf4GDU1NTKxmHhTGzvOsz+JwEMrsGelJf6Gq54g7vlMsuXEboR/BIZsh2Y1pV4eb1 oXfj4djXgX2+7a24x0o4uBUkWq2WSEvr+GXSujP8POCR+zrlgVB1zlq3vRCHUywoWzlc 0PDp1rG2cFe+MyF8NuXpaf/YA+YWPFRynQPAUKOEvCuAPQh9vJlOuFc56w7OCimrt090 uqZw== X-Forwarded-Encrypted: i=1; AHgh+Rpqn88aw/HfAdidSxA4z8lX4B5a1QflFFWPtRighUCvRnWAcxpi9v6hvoEJUCSDBzI4OOoY1S9fImd9+po=@vger.kernel.org X-Gm-Message-State: AOJu0Yz881q+ipL+k8lVYm3E/L1cMox/+eQT3H+34er/H6eN49qWLPGN MmIPk1VyOHADE4AUH4Hr6flbd7ZAFrouSjfW4u4dTwnmcvcvJNadZN0FhAZEojws X-Gm-Gg: AR+sD11b/WVw/v2Pn7B4PBi9Ba1pctGp0Ttlj8ytIPF7BBd0PSGBcv1dabyQiZuWiBD uVYiG/7AhYYxmCJ/M7B9ngwzeYHB7h235y99mozdhVHsqrwhnVU65u7Bdob8hOjSxIoONuifcVx hd2jlYMo01cgn7ZK4CQQi2YhkILcMBTkzv6FOsWG7itDTiTmXBJWzAJx4aZ40ID7dEkjgTi/wot RcQnUaC4R4GGqgztcmLs7pgUdDnbsEDTIvxxSuniXGc6nn5gNBq08NgPaHUkpVyRB9E2k2t+MnW XiV8foHUzfgcCEWsSfKPw4i5WEzOjnLvctuEXkXqs3dp/jEAlVM4ZISyqUa7FIm20Mga14+GB4D Hkyrc1ZPGkcnlzEmIWiDSzBe9rZYRBcbERHxJ/IdEZkRnBu6miA2h1yeLMKxdEbHp0Hs5HMNQfY HTbEGftAOxFuTf7zVULDPdKsCCn67TNB4L8JYfHM5n0DIuTWyxY5HoOlmE93Ky6xL29f7vWeB8z dFneMTEvwwnYsylSNuX1WT9/zSPwlC6oN1C5KFcv6T+GCk= X-Received: by 2002:a17:902:d544:b0:2cf:b330:e0e8 with SMTP id d9443c01a7336-2d0521bc383mr40814585ad.10.1785599466354; Sat, 01 Aug 2026 08:51:06 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.50.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:51:05 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 12/16] io_uring/rsrc: introduce buf registration structure Date: Sat, 1 Aug 2026 16:46:24 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In preparation to following changes, instead of passing an iovec for buffer registration introduce a new structure. It'll be moved to uapi later, but for now it's initialised early from a user provided iovec. Signed-off-by: Pavel Begunkov --- io_uring/rsrc.c | 47 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 15 deletions(-) diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 8d0f2ee24e0c..8af371ba6c06 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -27,8 +27,13 @@ struct io_rsrc_update { u32 offset; }; =20 +struct io_uring_regbuf_desc { + __u64 uaddr; + __u64 size; +}; + static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx, - struct iovec *iov); + struct io_uring_regbuf_desc *desc); =20 static int hpage_acct_ref(struct io_ring_ctx *ctx, struct page *hpage, bool *acct_new) @@ -81,6 +86,15 @@ static bool hpage_acct_unref(struct io_ring_ctx *ctx, st= ruct page *hpage) =20 #define IO_CACHED_BVECS_SEGS 32 =20 +static void io_iov_to_regbuf_desc(const struct iovec *iov, + struct io_uring_regbuf_desc *desc) +{ + *desc =3D (struct io_uring_regbuf_desc) { + .uaddr =3D (u64)(uintptr_t)iov->iov_base, + .size =3D iov->iov_len, + }; +} + int __io_account_mem(struct user_struct *user, unsigned long nr_pages) { unsigned long page_limit, cur_pages, new_pages; @@ -381,6 +395,7 @@ static int __io_sqe_buffers_update(struct io_ring_ctx *= ctx, return -EINVAL; =20 for (done =3D 0; done < nr_args; done++) { + struct io_uring_regbuf_desc desc; struct io_rsrc_node *node; u64 tag =3D 0; =20 @@ -394,7 +409,9 @@ static int __io_sqe_buffers_update(struct io_ring_ctx *= ctx, err =3D -EFAULT; break; } - node =3D io_sqe_buffer_register(ctx, iov); + + io_iov_to_regbuf_desc(iov, &desc); + node =3D io_sqe_buffer_register(ctx, &desc); if (IS_ERR(node)) { err =3D PTR_ERR(node); break; @@ -853,26 +870,26 @@ bool io_check_coalesce_buffer(struct page **page_arra= y, int nr_pages, } =20 static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx, - struct iovec *iov) + struct io_uring_regbuf_desc *desc) { + unsigned long uaddr =3D (unsigned long)desc->uaddr; + size_t size =3D desc->size; struct io_mapped_ubuf *imu =3D NULL; struct page **pages =3D NULL; struct io_rsrc_node *node; unsigned long off; - size_t size; int ret, nr_pages, i; struct io_imu_folio_data data; bool coalesced =3D false; =20 - if (!iov->iov_base) { - if (iov->iov_len) + if (!uaddr) { + if (size) return ERR_PTR(-EFAULT); /* remove the buffer without installing a new one */ return NULL; } =20 - ret =3D io_validate_user_buf_range((unsigned long)iov->iov_base, - iov->iov_len); + ret =3D io_validate_user_buf_range(uaddr, size); if (ret) return ERR_PTR(ret); =20 @@ -881,8 +898,7 @@ static struct io_rsrc_node *io_sqe_buffer_register(stru= ct io_ring_ctx *ctx, return ERR_PTR(-ENOMEM); =20 ret =3D -ENOMEM; - pages =3D io_pin_pages((unsigned long) iov->iov_base, iov->iov_len, - &nr_pages); + pages =3D io_pin_pages(uaddr, size, &nr_pages); if (IS_ERR(pages)) { ret =3D PTR_ERR(pages); pages =3D NULL; @@ -904,10 +920,9 @@ static struct io_rsrc_node *io_sqe_buffer_register(str= uct io_ring_ctx *ctx, if (ret) goto done; =20 - size =3D iov->iov_len; /* store original address for later verification */ - imu->ubuf =3D (unsigned long) iov->iov_base; - imu->len =3D iov->iov_len; + imu->ubuf =3D uaddr; + imu->len =3D size; imu->folio_shift =3D PAGE_SHIFT; imu->release =3D io_release_ubuf; imu->priv =3D imu; @@ -917,7 +932,7 @@ static struct io_rsrc_node *io_sqe_buffer_register(stru= ct io_ring_ctx *ctx, imu->folio_shift =3D data.folio_shift; refcount_set(&imu->refs, 1); =20 - off =3D (unsigned long)iov->iov_base & ~PAGE_MASK; + off =3D uaddr & ~PAGE_MASK; if (coalesced) off +=3D data.first_folio_page_idx << PAGE_SHIFT; =20 @@ -969,6 +984,7 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, vo= id __user *arg, memset(iov, 0, sizeof(*iov)); =20 for (i =3D 0; i < nr_args; i++) { + struct io_uring_regbuf_desc desc; struct io_rsrc_node *node; u64 tag =3D 0; =20 @@ -992,7 +1008,8 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, v= oid __user *arg, } } =20 - node =3D io_sqe_buffer_register(ctx, iov); + io_iov_to_regbuf_desc(iov, &desc); + node =3D io_sqe_buffer_register(ctx, &desc); if (IS_ERR(node)) { ret =3D PTR_ERR(node); break; --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13443275B1A for ; Sat, 1 Aug 2026 15:51:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599489; cv=none; b=XseGCWDle9lgzJMxblpznTV5vncK7M3fZxmA1/DxbhnWJiJ1hsehzidEzYsAXl882jyTbhlaBnN6MNkRWD+QsudPpXB/l3vHRXg0KYOEbvpksv9eMtRu9tBJEj3WmaDxXBw1YpUHBav5EMn7kK4vbFwlLzpYEe6nJIDtFioVYkM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599489; c=relaxed/simple; bh=zMJagFkkbrY21o8z8rVc1idAHLHFZUQVRbfwdxyhOfE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Sl6nHTSk+7HZe1PNMKwW3cGUMqXs1pJMx1eISg0e2M6+2drxhP9sCqBC1GLlIVd1w4oHo5F/jUIO0J8sbXThvXmvDscEDSlfJkd5+Ge1Z08054+UPT0x+wCEziM3zLWXVAirVPUVz6XfI/GawTWMK3dEwygACybl7OtC7EOuHU0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=J8p8OOE2; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="J8p8OOE2" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2cc891373e0so21211355ad.2 for ; Sat, 01 Aug 2026 08:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599487; x=1786204287; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=GRFULSZF3QJGSk4pMI6BKC5bl2viqB4gAVktXW+uhWs=; b=J8p8OOE2aDvOPhnhuHW6/3kcC6bC4kw1limrdida6/HILpX3te6yKkDsngItz3jnwO ml4ncMm7K8KAWbS9j56F7qLWyDD8ag4UWatpL4KSapAY8XwpD/C9rlap3SyN/CueRJEx KvUjrlQo1tKCayMMjS2A/PIOzD9DoH329KKUkvK+miMLQmctHTgx8hGGSDcSn/irI346 y5NYGTTiBbtg6X/4PRNeLURCm6Ff9fKzBkiLtSJmI+XvNSsy6Je0CDDScqNfPtqA9NCC t4VUO4+aOOr7Szeu91k8pWmR8CAIex3BrXOa419C6NievRxPItdNgo0CR3EWUtIG48Cp hcCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599487; x=1786204287; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=GRFULSZF3QJGSk4pMI6BKC5bl2viqB4gAVktXW+uhWs=; b=lSRwLHzkdoMpW1as3XR4yAWXZbyMX0ICD2Eq3ux3Gh+z1nrJe5X2O4hNf+APuD9gHB AC+YEq7HyrKDwdNyJa8sv0aNCk7i26M+qptMqk5FQly2l/LulM+lddYnnTWi8bJ9rIQj fcZ9xZZJ+87Cznc3RqcpAqiHWG1jOMBQBYFaNji7pljGEPOOaV8KywdgZhYi7Uv9sTFT +w0wZPGGlzZl1+5ao7mqX23hlbbDEtOeJ2kgZ/PSUF1FdTE6QZO6ruGPgNiN5NJH46GA HMBsZo7p5r1jzI0MF1FC30jESiEfZVnTkZoxjPP+b652C9xUvhxumsnQW6INmQYGuwPt 9vOQ== X-Forwarded-Encrypted: i=1; AHgh+RrpVwUh6SeOLSAWYEXIgs8tkeDZ30v0F9Hf/gl2p52Iy8yWppDUOmTWNjft5cLgF/Kp1hkQvUMVGUU41wk=@vger.kernel.org X-Gm-Message-State: AOJu0Ywr951dsB+g/O48xFngty5fs7F9Hv2qXdYFinx3uF3v6ZvGwznU hewe9x9B4lLPI6zX/hkjrr3t/Oqjy2T/n+EFDSMMOdINWMZsPBjh3EVB X-Gm-Gg: AR+sD109NHXvOQoWXzvdCK7OPIK4KT0ELsNRc0EsKNsQ4Z1cOlRM+aD+5ej7zfQD3DG aNfxAjH+wrIE9Yge59z9UUiSC0fL2gvR0ZxCj6iFk5HR5gSpXfQEmbiL92/5gorHo4P1dsyNfCz ZU9OH9dayaet15zj9WE+7QXf58Le/mQf7yIbZWvlTAMJU6lNfQdhh9c/CsRfKKw5wFakYPiuxlb fuIemjM9Ko5a4KGMjX9DuHPMDZZSYvu77p/F8yGyZJ8OKde2VaDjvW2vDLXSshPtLoUMe7MLaVa TDekTd+kaezQ5k7HLVVWe44Xd4U0NyGXLiUrW+YHsJ7d5Asl5yrlIr424TbbZv9Smys30atVqr8 hJejUNMXYSnc8f50JWsFDTA4z5yhP6PmRCdApkWQDJE72w82NK6tlzMo97QMpuw4tZRdmh8Wh/P p9gRy0N9M5iR9FMhnBtnan5yA8RyFC7NZlquyK2+cMBYZ7ObnYfk6+CYa4gHiZMvY1p0C5yuSeF wnk+eOOWdcVHIYTByckpHHjLfht062CQGy8ZAgC2afPFPo= X-Received: by 2002:a17:903:298b:b0:2ca:ecf6:9104 with SMTP id d9443c01a7336-2d0521bcbb7mr39843045ad.4.1785599487521; Sat, 01 Aug 2026 08:51:27 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:51:26 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 13/16] io_uring/rsrc: extend buffer update Date: Sat, 1 Aug 2026 16:46:25 +0100 Message-ID: <9bca69c80c76a49eb353faaaf3ea00a82e9cecbc.1785596451.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We need to pass more information to buffer registration than we can fit into a single struct iovec. This patch allows users to optionally pass struct io_uring_regbuf_desc. Apart from having more space for future use cases, it also introduces registration types. Currently, the type can be either of IO_REGBUF_TYPE_UADDR, which mirrors the iovec path, or IO_REGBUF_TYPE_EMPTY for leaving a buffer table slot empty. The next patch introduces a dmabuf backed type, and can be useful for other extensions like splicing a list of user addresses (i.e. iovec[]), interoperability with zcrx, kernel allocated memory like was brough up by Cristoph. Note, the type only represents a registration option, which is distinct from how io_uring internally stores it. The flags field is not used yet but always useful to have, e.g. we can encode read-only / write-only restrictions using it. Signed-off-by: Pavel Begunkov --- include/uapi/linux/io_uring.h | 27 +++++++++++++- io_uring/rsrc.c | 69 ++++++++++++++++++++++------------- 2 files changed, 69 insertions(+), 27 deletions(-) diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 909fb7aea638..98b259901185 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -790,13 +790,38 @@ struct io_uring_rsrc_update { =20 struct io_uring_rsrc_update2 { __u32 offset; - __u32 resv; + __u32 flags; __aligned_u64 data; __aligned_u64 tags; __u32 nr; __u32 resv2; }; =20 +/* struct io_uring_rsrc_update2::flags */ +enum io_uring_rsrc_reg_flags { + /* + * Use the extended descriptor format for buffer updates, + * see struct io_uring_regbuf_desc + */ + IORING_RSRC_UPDATE_EXTENDED =3D 1U << 1, +}; + +/* Buffer registration type, passed in struct io_uring_regbuf_desc::type */ +enum io_uring_regbuf_type { + IO_REGBUF_TYPE_EMPTY, + IO_REGBUF_TYPE_UADDR, + + __IO_REGBUF_TYPE_MAX, +}; + +struct io_uring_regbuf_desc { + __u32 type; /* enum io_uring_regbuf_type */ + __u32 flags; + __u64 size; + __u64 uaddr; + __u64 __resv[7]; +}; + /* Skip updating fd indexes set to this value in the fd table */ #define IORING_REGISTER_FILES_SKIP (-2) =20 diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 8af371ba6c06..406619158fbb 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -27,11 +27,6 @@ struct io_rsrc_update { u32 offset; }; =20 -struct io_uring_regbuf_desc { - __u64 uaddr; - __u64 size; -}; - static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx, struct io_uring_regbuf_desc *desc); =20 @@ -90,9 +85,12 @@ static void io_iov_to_regbuf_desc(const struct iovec *io= v, struct io_uring_regbuf_desc *desc) { *desc =3D (struct io_uring_regbuf_desc) { + .type =3D IO_REGBUF_TYPE_UADDR, .uaddr =3D (u64)(uintptr_t)iov->iov_base, .size =3D iov->iov_len, }; + if (!desc->uaddr) + desc->type =3D IO_REGBUF_TYPE_EMPTY; } =20 int __io_account_mem(struct user_struct *user, unsigned long nr_pages) @@ -323,6 +321,8 @@ static int __io_sqe_files_update(struct io_ring_ctx *ct= x, return -ENXIO; if (up->offset + nr_args > ctx->file_table.data.nr) return -EINVAL; + if (up->flags) + return -EINVAL; =20 for (done =3D 0; done < nr_args; done++) { u64 tag =3D 0; @@ -382,9 +382,8 @@ static int __io_sqe_buffers_update(struct io_ring_ctx *= ctx, struct io_uring_rsrc_update2 *up, unsigned int nr_args) { + bool extended =3D up->flags & IORING_RSRC_UPDATE_EXTENDED; u64 __user *tags =3D u64_to_user_ptr(up->tags); - struct iovec fast_iov, *iov; - struct iovec __user *uvec; u64 user_data =3D up->data; __u32 done; int i, err; @@ -393,29 +392,49 @@ static int __io_sqe_buffers_update(struct io_ring_ctx= *ctx, return -ENXIO; if (up->offset + nr_args > ctx->buf_table.nr) return -EINVAL; + if (up->flags & ~IORING_RSRC_UPDATE_EXTENDED) + return -EINVAL; =20 for (done =3D 0; done < nr_args; done++) { struct io_uring_regbuf_desc desc; struct io_rsrc_node *node; u64 tag =3D 0; =20 - uvec =3D u64_to_user_ptr(user_data); - iov =3D iovec_from_user(uvec, 1, 1, &fast_iov, io_is_compat(ctx)); - if (IS_ERR(iov)) { - err =3D PTR_ERR(iov); - break; - } if (tags && copy_from_user(&tag, &tags[done], sizeof(tag))) { err =3D -EFAULT; break; } =20 - io_iov_to_regbuf_desc(iov, &desc); + if (extended) { + if (copy_from_user(&desc, u64_to_user_ptr(user_data), + sizeof(desc))) { + err =3D -EFAULT; + break; + } + user_data +=3D sizeof(desc); + } else { + struct iovec __user *uvec =3D u64_to_user_ptr(user_data); + struct iovec fast_iov, *iov; + + if (io_is_compat(ctx)) + user_data +=3D sizeof(struct compat_iovec); + else + user_data +=3D sizeof(struct iovec); + + iov =3D iovec_from_user(uvec, 1, 1, &fast_iov, io_is_compat(ctx)); + if (IS_ERR(iov)) { + err =3D PTR_ERR(iov); + break; + } + io_iov_to_regbuf_desc(iov, &desc); + } + node =3D io_sqe_buffer_register(ctx, &desc); if (IS_ERR(node)) { err =3D PTR_ERR(node); break; } + if (tag) { if (!node) { err =3D -EINVAL; @@ -426,10 +445,6 @@ static int __io_sqe_buffers_update(struct io_ring_ctx = *ctx, i =3D array_index_nospec(up->offset + done, ctx->buf_table.nr); io_reset_rsrc_node(ctx, &ctx->buf_table, i); ctx->buf_table.nodes[i] =3D node; - if (io_is_compat(ctx)) - user_data +=3D sizeof(struct compat_iovec); - else - user_data +=3D sizeof(struct iovec); } return done ? done : err; } @@ -464,7 +479,7 @@ int io_register_files_update(struct io_ring_ctx *ctx, v= oid __user *arg, memset(&up, 0, sizeof(up)); if (copy_from_user(&up, arg, sizeof(struct io_uring_rsrc_update))) return -EFAULT; - if (up.resv || up.resv2) + if (up.resv2) return -EINVAL; return __io_register_rsrc_update(ctx, IORING_RSRC_FILE, &up, nr_args); } @@ -478,7 +493,7 @@ int io_register_rsrc_update(struct io_ring_ctx *ctx, vo= id __user *arg, return -EINVAL; if (copy_from_user(&up, arg, sizeof(up))) return -EFAULT; - if (!up.nr || up.resv || up.resv2) + if (!up.nr || up.resv2) return -EINVAL; return __io_register_rsrc_update(ctx, type, &up, up.nr); } @@ -578,12 +593,9 @@ int io_files_update(struct io_kiocb *req, unsigned int= issue_flags) struct io_uring_rsrc_update2 up2; int ret; =20 + memset(&up2, 0, sizeof(up2)); up2.offset =3D up->offset; up2.data =3D up->arg; - up2.nr =3D 0; - up2.tags =3D 0; - up2.resv =3D 0; - up2.resv2 =3D 0; =20 if (up->offset =3D=3D IORING_FILE_INDEX_ALLOC) { ret =3D io_files_update_with_index_alloc(req, issue_flags); @@ -882,8 +894,13 @@ static struct io_rsrc_node *io_sqe_buffer_register(str= uct io_ring_ctx *ctx, struct io_imu_folio_data data; bool coalesced =3D false; =20 - if (!uaddr) { - if (size) + if (desc->type >=3D __IO_REGBUF_TYPE_MAX) + return ERR_PTR(-EINVAL); + if (!mem_is_zero(&desc->__resv, sizeof(desc->__resv)) || desc->flags) + return ERR_PTR(-EINVAL); + + if (desc->type =3D=3D IO_REGBUF_TYPE_EMPTY) { + if (uaddr || size) return ERR_PTR(-EFAULT); /* remove the buffer without installing a new one */ return NULL; --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11872331ED0 for ; Sat, 1 Aug 2026 15:51:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599510; cv=none; b=KYTUc433+WwKGZOiyberxXixTUaBUV7Qhczn5aPR1dGSchMNM2h8zBFnph7la2AJMnSXg1Tm+02CDOK+KmWIfS5xgO1/BEKOE6uwglGUAYigUBWNv9zzdplyUR48bmsUW1AL6onmjrzgRm69lmaHcnxfp8Q6Rfa1HV8MATN7Ivw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599510; c=relaxed/simple; bh=PHUqDIjZ/fDz2a0/0v9uQuWQT4kuiEnCn87yNoeA1yY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ceyifdA5bUClvJbDPSQaKkrcy4C3enoNl+neSOXUnCzBv/X2TqduG7VJ1UHPj5Gr1L9NGmie0qWAAbux0UxlM/mU2N0x1HFP0dt9s0AgTExi2DiuXa6R697wu3R9Oq2lZZpK3OCkDaOnTg62CXynyf7ZTNcQk9x92yvvSzo8XTY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NgBn7OYN; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NgBn7OYN" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-2cea3004256so22570165ad.0 for ; Sat, 01 Aug 2026 08:51:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599508; x=1786204308; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ijUzk6DmoV2KfP6NttCifTbOeJnZ78uDbymc6z2HdV8=; b=NgBn7OYNrfRieIBwlg4KoAGaQqQTlisou0OcugD6Ej6l/TaCFca1OiqCFyMpkC//QX r3UzS7SaCQ0sXrHi0EcoqlS4Ji6HKYLrWuwEJBOJt4z4LQTy7a1Sd86sPqoLTYbS5LWz lNyKClBDTeDgtecTN+13oelghUTMJ+Rk9K+65P+ktmgYx3fzP29/3slmIrJuXwEcqjK5 HZihhFREitU7d+ghO6rCdjhDYR9CoWDEjMYxL8E0TOjUIkmdI4wd6zO+NZR73cE7hvah 7vUJQry9aFAW3SldEaNxvG9FSt2XLDzrfqX5y3najTxpF3VuU2eOA+ZOvct8Y0iX7o0C uKzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599508; x=1786204308; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ijUzk6DmoV2KfP6NttCifTbOeJnZ78uDbymc6z2HdV8=; b=ZNG8x0iockD30wTMeMnpExnJGG+0oTS9IHXas9ydDwPAJZ9l7v4i3W9asSte/LEuua AhYNH3SzS1QHOg0K8VmLplQgh70L9bBYOZmDZgQaJRqymRiKB5UaU7lLMs9rnqvKQKXL Sbfb3/Vlf/WOQqOEANtjd7Cd5tEn/0tlcB7qa1JxK3zYR9Ix3ySw/3MmdcUhh3zy5pSd y6BFGDygvjTJ+IqVFH8SITPFHFJJMoE+uUIaBgWq6udhoEhSVh3XgKtqCh4ZuNRI9bZP cUrfZcyxHMfeFJOoCbf1OTkdgG610qHAgMAK6w1iN7lZlV/mfqbNtq38yPdJg1UY40hp KniA== X-Forwarded-Encrypted: i=1; AHgh+Rr+Ko1dBLHOKufndje+38Q+P5JWODBljN6p8QIwUb4zwN4mz8lZybVh1iiLXnvdju2mrxVPHx0s7AByMb0=@vger.kernel.org X-Gm-Message-State: AOJu0YzoyfGvjq6g+CWxizjHD3misg3G/Z495GHVVinvu9cTxZ/EK/h5 PCwDBC8k3jPFp5UJyTySScUKbu4g11ug9QlgqMdV4o3rI3qO+xx9MOqx X-Gm-Gg: AR+sD11ZgcpEoXlPO3+qLkW3dyV/A26SyhYaNXk6bqFCXHIagX2vvdR/ds0nacgZPG8 2K/Fk/sde6rdmheWF0T4BeX39XDA1pU93dr24fe27//dVOT32oPq3Ydn+r8w6YWa9Fv3QZ0t+JX HRjhOS9my2CqrrBDfDL3MhTSaeseK4mdCicX0nfvEE/XJf/+BaHdg8eN6stG6SDbOuEzhtYZGha VNSDV6dIJDrRWp6AId9huqUmjN2QDtuR7qjW8qEQLd86TJ90p46cTI2tbJiO9wlyVYod+6vo/93 KsfnB/5SCm4gzdnrA6dW1wcF0zaQa9d7DAItPAubwocPRh/75poRaz0qufAL/C3vUYrbEYU9Rea EvLXe0mDhH+sJgK0tZmBWzFSJO15iyk1W8DsSnyJFrlGMmm/zQTldREjStKYz6p8t3RckSXOU0Z MgNzwTgT+EajK3O9KI4XF5riB7qG/jF5UVaGPN8bmFULkyMHZUuG81kHAvzPv2XdnRTcK0wEf5Z t3tAkUDEVD11ULx49QPka7SFGmRUR9hMy0xcVcdbZBwCNC7qhfhppAq1rg= X-Received: by 2002:a17:902:e945:b0:2cc:9179:335 with SMTP id d9443c01a7336-2d0521e87a4mr39523185ad.8.1785599508438; Sat, 01 Aug 2026 08:51:48 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.51.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:51:47 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 14/16] io_uring/rsrc: add uncloneable regbuf flag Date: Sat, 1 Aug 2026 16:46:26 +0100 Message-ID: <65be6058af60c33328f9ccd8133d24e2bbbd0f84.1785596451.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" It's hard to implement cloning if the internal structure needs to be mutable and/or relies on other ring resources. In preparation to such buffer types, add a flag indicating that the buffer can't be cloned. It might be possible to add cloning in the future for them, but that would likely need reallocating the structure and reacquiring resources in case by case manner. Signed-off-by: Pavel Begunkov --- io_uring/rsrc.c | 5 +++++ io_uring/rsrc.h | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 406619158fbb..f041b0db77af 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -1381,6 +1381,11 @@ static int io_clone_buffers(struct io_ring_ctx *ctx,= struct io_ring_ctx *src_ctx if (!src_node) { dst_node =3D NULL; } else { + if (src_node->buf->flags & IO_REGBUF_F_UNCLONEABLE) { + io_rsrc_data_free(ctx, &data); + return -ENOMEM; + } + dst_node =3D io_rsrc_node_alloc(ctx, IORING_RSRC_BUFFER); if (!dst_node) { io_rsrc_data_free(ctx, &data); diff --git a/io_uring/rsrc.h b/io_uring/rsrc.h index 98ae8ef51009..83aa86e6f320 100644 --- a/io_uring/rsrc.h +++ b/io_uring/rsrc.h @@ -29,7 +29,8 @@ enum { }; =20 enum { - IO_REGBUF_F_KBUF =3D 1, + IO_REGBUF_F_KBUF =3D 1 << 0, + IO_REGBUF_F_UNCLONEABLE =3D 1 << 1, }; =20 struct io_mapped_ubuf { --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3ED93546DE for ; Sat, 1 Aug 2026 15:52:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599532; cv=none; b=ltkZzpgv8zuRkmlvEgVVejSTZvgGFlNq1HI29fzNhQQc787/WRmejLtiukRryE0rNyraS6h/SyiHdtfR2HiIXfdhfUv0e6E+oUGAQLAQiCc/RThJmDS09v7at0ofbEhCvt0tBZqlqXKj807nfXJUZmzk7Rh6/0kVxIs58KLqUu4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599532; c=relaxed/simple; bh=7qlj+n0xMTYMM8wbxh5YbBlihhCnKaVVIBe7eTfypEw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sDuBiL4H2+KK5xlgdEMlBWMFu/AuJLbun3PDlF9WFEwSrwTCsKwm8npeGTQQpBB1AH9ZKDc57EyZDSWWmFom6oBgLMw9mzartWjTZ4rBr9a+j0G3iMHEVh09otym5l+nxXlAQ49S4zQ3sg9J9SxJqJqwjB/RVefAIySCJfQO6Jk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Y3I/+/QG; arc=none smtp.client-ip=209.85.214.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Y3I/+/QG" Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2cf27856f9cso20479375ad.2 for ; Sat, 01 Aug 2026 08:52:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599530; x=1786204330; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=4DE9iwPHyfteVjhRD5r56Chvjj2EVxXiQr3A0KjARNc=; b=Y3I/+/QGz8AWLgiulI6sUP9MyxA5O8wTFT0Aqgz4KW0m6qhl6x+ZLnO71suTdlTGdL 30EuWjpibEb2nl9zRLU4yq3gzjepyvdUEaTbQvlPp802+I+wvljk97/Vrw/PmKgUe4A/ /V9jOTpHfI1LzbzXk46hMvnO2z9Lktfnp14lpg+52yW/qg5HkRQcfCJpcdnmbQXDjgyd pSeaK26w50qa5Py2luzzQvkhjCwe1d6oXA2nIW2SSAuk8qu9Lc9BJup38LOQzfvZX5SP 4dD/sJGrHPjCyq53HOohqHToRnY9hYKQd5Wruu+3oPcPw/wPJOa698VNHJ7zTCjE2Qiz nlcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599530; x=1786204330; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=4DE9iwPHyfteVjhRD5r56Chvjj2EVxXiQr3A0KjARNc=; b=a+j1okF6csJzb6FXD/bGIg8jspOuL74f9I4Rq4m0pBXsD+7iHNmZtmN7fdTzkZngii oFH7HwqwHOxvIXwrxOB5AVlwZE7ZZYhYeth+nQToOOjfxtLEEo3jvyoLtlSrnIQ5RC2s 1dHHJ3oOQGaCooPwimHOA051cmdTdjUOjN/G6r+gScTrkY0khcbS9rztWZFJxc1ST6Dt tPMA1G1N81a+/uOWA7a8aKtreMXq4BoFAhnm0hTEzp92oGuc7wJJuu1lQaQNf7StL+oh 8+1Hlf446EvL9YumknrhzJnF+HFxIX6nxSnrxKlq6XM0UKTP1ZT4nuXv8SkeKw2IkdL9 IwKg== X-Forwarded-Encrypted: i=1; AHgh+RpPcn8fCMDplobE3ZO3XpUFNZozX1hhMVczLQVVTh0guultgr8zd148Nph4Dm7UGnQY9OvAvnTF8cRqPSc=@vger.kernel.org X-Gm-Message-State: AOJu0Yyer8M8RjaMe/ZnaD4Ns4w692+UANxcQdvi6k89s2OABM0Z2IMN DsgOlvzsG8Q1XbnzzAfNLBKBJhtcOhFz2tASpbIimjZNbbqxSwAWo1Uc X-Gm-Gg: AR+sD13sjfY8/DRDbjvKUw8XBClnGTroknDWqq2LUTwk9THhuNxtgjE0d7PqvrmsQUE 6tOLfdCP/4t5WqFeXceuksD70X3t2EbDtPhzMBbHHJMdmM4upbdexfCurByA1NvALoKiMMJIK96 uv1kj8Rudqojs+G7gi2xe3SO4Suz3JsSpobSY2ylbZpUQKBhvfh6NSk2atGNNkPVrZwPxIa3A/B HvZBZx3vMGrFsg0QcqyUvCD6yMSC601IkkwNKSOBmTYkI+qbG3/yisffDwAxDf2gsyp8u/TOHIZ fOqd4F/zQ87yuD/5vBkd2zHmowACiZ9VqtQRxnfKKL4iZ7P3oMDyfPe0B1QOZRjGLdXJ52fS8I7 ybOn2cHVN+RCPeN/5aSwF6zj8G/rR0R8SsPDyeFgPTCMYC5+EoiF68j47KeMntuOAA8OXyvL+sw PncvWrw7Q/3FcpjEyhbRY8dq3ZZ1HeGna0GRNrjWQ33aOMnzo8gIbhg71cs03DYrfAMT+sCj5wH JbZNGgejzNd2LFlSf02aJIeRlk+VT7fx6L105YDYKqwFNA= X-Received: by 2002:a17:903:1aa6:b0:2cf:32bc:9722 with SMTP id d9443c01a7336-2d0524671d5mr36849685ad.34.1785599529680; Sat, 01 Aug 2026 08:52:09 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.51.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:52:09 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 15/16] io_uring/rsrc: add regbuf import flags Date: Sat, 1 Aug 2026 16:46:27 +0100 Message-ID: X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We'll have special registered buffer types that can't be used with all opcodes and need special handling. Add separate flags to control registered buffer import, which will be used to specify what kind of buffers the caller can handle. Signed-off-by: Pavel Begunkov --- io_uring/rsrc.c | 8 ++++---- io_uring/rsrc.h | 24 ++++++++++++++++++++---- 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index f041b0db77af..fcdc4edd2858 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -1245,9 +1245,9 @@ inline struct io_rsrc_node *io_find_buf_node(struct i= o_kiocb *req, return NULL; } =20 -int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter, +int __io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter, u64 buf_addr, size_t len, int ddir, - unsigned issue_flags) + unsigned issue_flags, unsigned import_flags) { struct io_rsrc_node *node; =20 @@ -1656,9 +1656,9 @@ static int io_kern_bvec_size(struct iovec *iov, unsig= ned nr_iovs, return 0; } =20 -int io_import_reg_vec(int ddir, struct iov_iter *iter, +int __io_import_reg_vec(int ddir, struct iov_iter *iter, struct io_kiocb *req, struct iou_vec *vec, - unsigned nr_iovs, unsigned issue_flags) + unsigned nr_iovs, unsigned issue_flags, unsigned import_flags) { struct io_rsrc_node *node; struct io_mapped_ubuf *imu; diff --git a/io_uring/rsrc.h b/io_uring/rsrc.h index 83aa86e6f320..f12abaf63270 100644 --- a/io_uring/rsrc.h +++ b/io_uring/rsrc.h @@ -65,12 +65,28 @@ int io_rsrc_data_alloc(struct io_rsrc_data *data, unsig= ned nr); =20 struct io_rsrc_node *io_find_buf_node(struct io_kiocb *req, unsigned issue_flags); -int io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter, +int __io_import_reg_buf(struct io_kiocb *req, struct iov_iter *iter, u64 buf_addr, size_t len, int ddir, - unsigned issue_flags); -int io_import_reg_vec(int ddir, struct iov_iter *iter, + unsigned issue_flags, unsigned import_flags); +int __io_import_reg_vec(int ddir, struct iov_iter *iter, struct io_kiocb *req, struct iou_vec *vec, - unsigned nr_iovs, unsigned issue_flags); + unsigned nr_iovs, unsigned issue_flags, + unsigned import_flags); + +static inline int io_import_reg_buf(struct io_kiocb *req, struct iov_iter = *iter, + u64 buf_addr, size_t len, int ddir, + unsigned issue_flags) +{ + return __io_import_reg_buf(req, iter, buf_addr, len, ddir, issue_flags, 0= ); +} + +static inline int io_import_reg_vec(int ddir, struct iov_iter *iter, + struct io_kiocb *req, struct iou_vec *vec, + unsigned nr_iovs, unsigned issue_flags) +{ + return __io_import_reg_vec(ddir, iter, req, vec, nr_iovs, issue_flags, 0); +} + int io_prep_reg_iovec(struct io_kiocb *req, struct iou_vec *iv, const struct iovec __user *uvec, size_t uvec_segs); =20 --=20 2.54.0 From nobody Sat Oct 3 12:02:28 2026 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0439337105 for ; Sat, 1 Aug 2026 15:52:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599554; cv=none; b=P6US2drVQnAw0C1Rd/1hgp2S+G+CROtT4XhO07kDAs1+jlAliyJLYZ1ZP1OG5KoM7BDBrmAoKdjnkM0uMgxS/nVmIy9d3kC6j5sw0YwiTgPWkZtggqsY6EyZvKBM9BtS+T0byUaNGnlYGyJ5NIIxxKHbnO1xPMQLZRy6x9P9kPM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785599554; c=relaxed/simple; bh=xQSghJzaUYqDPB1ggORLjLa8iy+vN2lmRYgeYFBaKdE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=c2UJIeXs5gomJqHQw3ZpEoVmusl/a+mCYCxxRx0phNrch7b3+z5nwkxTXCMk3G9WFRF4ynq9T7IL6RALr/EbIPScaI0ZkwUsc8OlXccvpVr16XI1Xmwf6yplHcz+nTXWKdIqig8ppQqU3nrrTx39Y/w93hh9s4Dm/3BdWNdCMBo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=G8iTKkgT; arc=none smtp.client-ip=209.85.214.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="G8iTKkgT" Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-2cf52d15d88so16334235ad.2 for ; Sat, 01 Aug 2026 08:52:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1785599551; x=1786204351; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=xsD2wsQNoOd5TnS/nE5N1rbXimmf2sYMA/5eIflw47E=; b=G8iTKkgT8TIOThO29caVbsRYEAJmzY1HjfZgbrNb9vDFlq5XV/jo6toSmh6e3CDasa juKI/CTqz3Zlirc4eje9DflMGLV4JrnDRIyQbEWqa/rFLOznV87WWYHQEl4sz3j2YMBP NvawH46IZWOM0d1+kxmAFnM9fnhMVJcmoSD6jWQtnGYqd8SMvWC3UG6CCilaQZ30smIG /Fn2jdv3rGMb/I0b25pe9I2XWMkO8yV8p0InyHJXFXlX8L1ZGlDQudjdEwmkMA928ePM GAteNWoSlKiyC7qbc8OY0Wof4Vz+CGE5pfQ1YMAjW4xzQnHNHdwD+7erROpvsKmnK7sx KV6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785599551; x=1786204351; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=xsD2wsQNoOd5TnS/nE5N1rbXimmf2sYMA/5eIflw47E=; b=dbpK2FXlgIaJmynKNGT2L2g8EAWJ4ziIlw15/UWuqsrI26rtcnuU2t5Ebi7aG9Z7sm J9IAUzlccNVFfJtFW45vPqWQGwAOiRpsTW1QrgFFUM5QHbEBL624fs9vHDV1JXzcFDqr gpwKVQfWAMdGLPPp6fQ+r5Oy5VhyKJZVgRiGBYyZWNUfEntnU8K0kM2tpcyibOhZNOx/ Q5Mrm68gGE/FsqP/nV5GvbvFIYSIJEySYRXrYedzCAMD+xecmq+JXoHF1m1CdmjvxntF GcpwOj5682/36b/ADDct8mUQhw8wgO1sq7lLgPQDE3xRWWchTZc+DDf3JhDz57PWG8Xk x4HQ== X-Forwarded-Encrypted: i=1; AHgh+RrTOfx2nBmNqUnGn/dvrOyL+nccikWM+loqPW3Nw14cNQ1SHcILNVfbnTrGhPgt3YiE8Nc5HQPUYSeoRNA=@vger.kernel.org X-Gm-Message-State: AOJu0Yw/A40E+u6QBIqGCsm0HbAR9vy8SLNGoCqz1ish9vhM1wGdAaIN 3Au6NvG04ydgLTsXMDHgukXOqT3IijPWtzrgW42nqOVOy9gZwf2Rk3DQ X-Gm-Gg: AR+sD12DelQeTa5/LgUuQNdkNaFiXbGxz8+gAnIzbXKSxvWGL5jbA5ca4JC6Q3hG381 8K2DazT82UNJhv115UpVrubZ4HIgVcZwpel3sscW1TCuBp9vO4wsCjuMhJz6E2+a8dx3gSS9q+g 5zg0/dD59vGA4u+vHB3F0JsiNGo7v43uJYPwVl2Xa2t9TSrxiFCxkbMRiSZLWXVVA41lfH0OVdF iAOWQFRMtvG8xpcMBaGHibyhoxTflXCD7WKCVS7cOiARzC9lAT5LV1wqgsNHAk5Y6QiLurEb4S8 R1e9c3IPH3We94cDKISwktj4BhkdRi8swy3cwdgElHzZke82GrgTmK85yKUUfHsukDhGKqdIKze H9mF37tfkPZ8fY1AYSDlI3Ksk7PHAUHSQ6NfPRy8vcipzuAQOmouze/ecLyDAYZPKt6crdHwIVN DofwnRdJGFtRhlWrK4KS4MxBmTL/+w7PzgBFKBtT2X6bTARzqM9ZeDxF7xT9fELCR3UfKuRwxsc lVQw1ZYckXWEY/W9Z7Q62j1BHTbH7LEIH/GAxcZQLW9USM= X-Received: by 2002:a17:902:cecf:b0:2c9:97a8:aff4 with SMTP id d9443c01a7336-2d0523f7772mr36487975ad.45.1785599550923; Sat, 01 Aug 2026 08:52:30 -0700 (PDT) Received: from 127.net ([2620:10d:c092:600::1:61d0]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2d04b12202bsm18287605ad.66.2026.08.01.08.52.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 01 Aug 2026 08:52:30 -0700 (PDT) From: Pavel Begunkov To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org, io-uring@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org Cc: asml.silence@gmail.com, Alexander Viro , Christian Brauner , Andrew Morton , Sumit Semwal , =?UTF-8?q?Christian=20K=C3=B6nig?= , Nitesh Shetty , Kanchan Joshi , Anuj Gupta , Tushar Gohad , William Power , Phil Cayton , Jason Gunthorpe , Damien Le Moal , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , Benjamin Marzinski , Vishal Verma , David Sterba , Ilya Dryomov , dm-devel@lists.linux.dev, nvdimm@lists.linux.dev, linux-btrfs@vger.kernel.org, ceph-devel@vger.kernel.org Subject: [PATCH v5 16/16] io_uring/rsrc: add dmabuf backed registered buffers Date: Sat, 1 Aug 2026 16:46:28 +0100 Message-ID: <830cd62ae8958351a9fcd6d663dea3430921f95b.1785596451.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Implement dmabuf backed registered buffers. To register them, the user should specify IO_REGBUF_TYPE_DMABUF for the regitration and pass the desired dmabuf fd and a file for which it should be registered. From there, it can be used with io_uring read/write requests IORING_OP_{READ,WRITE}_FIXED) as normal. The requests should be issued against the file specified during registration, and otherwise they'll be failed. The user should also be prepared to handle spurious -EAGAIN by reissuing the request. Internally, dmabuf registered buffers is an optin feature for io_uring request opcodes and they should pass a special flag on import to use it. Suggested-by: David Wei Suggested-by: Vishal Verma Suggested-by: Tushar Gohad Signed-off-by: Pavel Begunkov --- include/linux/io_uring_types.h | 5 + include/uapi/linux/io_uring.h | 6 +- io_uring/io_uring.c | 3 +- io_uring/rsrc.c | 167 ++++++++++++++++++++++++++++++++- io_uring/rsrc.h | 18 ++++ io_uring/rw.c | 6 +- 6 files changed, 197 insertions(+), 8 deletions(-) diff --git a/include/linux/io_uring_types.h b/include/linux/io_uring_types.h index a2c623a67a25..f90586d353c1 100644 --- a/include/linux/io_uring_types.h +++ b/include/linux/io_uring_types.h @@ -10,6 +10,7 @@ =20 struct iou_loop_params; struct io_uring_bpf_ops; +struct dma_buf_io_map; =20 enum { /* @@ -594,6 +595,7 @@ enum { REQ_F_IMPORT_BUFFER_BIT, REQ_F_SQE_COPIED_BIT, REQ_F_IOPOLL_BIT, + REQ_F_DROP_DMABUF_BIT, =20 /* not a real bit, just to check we're not overflowing the space */ __REQ_F_LAST_BIT, @@ -689,6 +691,8 @@ enum { REQ_F_SQE_COPIED =3D IO_REQ_FLAG(REQ_F_SQE_COPIED_BIT), /* request must be iopolled to completion (set in ->issue()) */ REQ_F_IOPOLL =3D IO_REQ_FLAG(REQ_F_IOPOLL_BIT), + /* there is a dma map attached to request that needs to be dropped */ + REQ_F_DROP_DMABUF =3D IO_REQ_FLAG(REQ_F_DROP_DMABUF_BIT), }; =20 struct io_tw_req { @@ -811,6 +815,7 @@ struct io_kiocb { /* custom credentials, valid IFF REQ_F_CREDS is set */ const struct cred *creds; struct io_wq_work work; + struct dma_buf_io_map *dmabuf_map; =20 struct io_big_cqe { u64 extra1; diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 98b259901185..c39297e8beb6 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -810,6 +810,7 @@ enum io_uring_rsrc_reg_flags { enum io_uring_regbuf_type { IO_REGBUF_TYPE_EMPTY, IO_REGBUF_TYPE_UADDR, + IO_REGBUF_TYPE_DMABUF, =20 __IO_REGBUF_TYPE_MAX, }; @@ -819,7 +820,10 @@ struct io_uring_regbuf_desc { __u32 flags; __u64 size; __u64 uaddr; - __u64 __resv[7]; + + __s32 dmabuf_fd; + __s32 target_fd; + __u64 __resv[6]; }; =20 /* Skip updating fd indexes set to this value in the fd table */ diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 4c83a94b4bdc..d1d748b5c5dc 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -109,7 +109,7 @@ =20 #define IO_REQ_CLEAN_SLOW_FLAGS (REQ_F_REFCOUNT | IO_REQ_LINK_FLAGS | \ REQ_F_REISSUE | REQ_F_POLLED | \ - IO_REQ_CLEAN_FLAGS) + IO_REQ_CLEAN_FLAGS | REQ_F_DROP_DMABUF) =20 #define IO_TCTX_REFS_CACHE_NR (1U << 10) =20 @@ -1114,6 +1114,7 @@ static void io_free_batch_list(struct io_ring_ctx *ct= x, io_queue_next(req); if (unlikely(req->flags & IO_REQ_CLEAN_FLAGS)) io_clean_op(req); + io_req_drop_dmabuf(req); } io_put_file(req); io_req_put_rsrc_nodes(req); diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index fcdc4edd2858..5074b1e4b050 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -10,6 +10,7 @@ #include #include #include +#include =20 #include =20 @@ -881,6 +882,94 @@ bool io_check_coalesce_buffer(struct page **page_array= , int nr_pages, return true; } =20 +struct io_regbuf_dma { + struct dma_buf_io_ctx ctx; + struct file *target_file; +}; + +static void io_release_reg_dmabuf(void *priv) +{ + struct io_regbuf_dma *db =3D priv; + + fput(db->target_file); + dma_buf_io_ctx_release(&db->ctx); +} + +static struct io_rsrc_node *io_register_dmabuf(struct io_ring_ctx *ctx, + struct io_uring_regbuf_desc *desc) +{ + struct io_rsrc_node *node =3D NULL; + struct io_mapped_ubuf *imu =3D NULL; + struct io_regbuf_dma *regbuf =3D NULL; + struct file *target_file =3D NULL; + struct dma_buf *dmabuf =3D NULL; + int ret; + + if (!IS_ENABLED(CONFIG_DMA_SHARED_BUFFER)) + return ERR_PTR(-EOPNOTSUPP); + if (ctx->flags & IORING_SETUP_IOPOLL) + return ERR_PTR(-EOPNOTSUPP); + if (desc->uaddr || desc->size) + return ERR_PTR(-EINVAL); + + ret =3D -ENOMEM; + node =3D io_rsrc_node_alloc(ctx, IORING_RSRC_BUFFER); + if (!node) + return ERR_PTR(-ENOMEM); + imu =3D io_alloc_imu(ctx, 0); + if (!imu) + goto err; + regbuf =3D kzalloc(sizeof(*regbuf), GFP_KERNEL); + if (!regbuf) + goto err; + + ret =3D -EBADF; + target_file =3D fget(desc->target_fd); + if (!target_file) + goto err; + + dmabuf =3D dma_buf_get(desc->dmabuf_fd); + if (IS_ERR(dmabuf)) { + ret =3D PTR_ERR(dmabuf); + dmabuf =3D NULL; + goto err; + } + if (dmabuf->size > SZ_1G) { + ret =3D -EINVAL; + goto err; + } + + ret =3D dma_buf_io_ctx_create(target_file, ®buf->ctx, dmabuf, + DMA_BIDIRECTIONAL); + if (ret) + goto err; + + regbuf->target_file =3D target_file; + imu->nr_bvecs =3D 0; + imu->ubuf =3D 0; + imu->len =3D dmabuf->size; + imu->folio_shift =3D 0; + imu->release =3D io_release_reg_dmabuf; + imu->priv =3D regbuf; + imu->flags =3D IO_REGBUF_F_DMABUF; + imu->dir =3D IO_IMU_DEST | IO_IMU_SOURCE; + refcount_set(&imu->refs, 1); + node->buf =3D imu; + dma_buf_put(dmabuf); + return node; +err: + kfree(regbuf); + if (imu) + io_free_imu(ctx, imu); + if (node) + io_cache_free(&ctx->node_cache, node); + if (target_file) + fput(target_file); + if (dmabuf) + dma_buf_put(dmabuf); + return ERR_PTR(ret); +} + static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx, struct io_uring_regbuf_desc *desc) { @@ -899,6 +988,12 @@ static struct io_rsrc_node *io_sqe_buffer_register(str= uct io_ring_ctx *ctx, if (!mem_is_zero(&desc->__resv, sizeof(desc->__resv)) || desc->flags) return ERR_PTR(-EINVAL); =20 + if (desc->type =3D=3D IO_REGBUF_TYPE_DMABUF) + return io_register_dmabuf(ctx, desc); + + if (desc->dmabuf_fd || desc->target_fd) + return ERR_PTR(-EINVAL); + if (desc->type =3D=3D IO_REGBUF_TYPE_EMPTY) { if (uaddr || size) return ERR_PTR(-EFAULT); @@ -1170,9 +1265,64 @@ static int io_import_kbuf(int ddir, struct iov_iter = *iter, return 0; } =20 -static int io_import_fixed(int ddir, struct iov_iter *iter, +void io_drop_dmabuf_node(struct io_kiocb *req) +{ + struct io_mapped_ubuf *imu; + + if (!IS_ENABLED(CONFIG_DMA_SHARED_BUFFER)) + return; + if (WARN_ON_ONCE(req->buf_node->type !=3D IORING_RSRC_BUFFER)) + return; + imu =3D req->buf_node->buf; + if (WARN_ON_ONCE(!(imu->flags & IO_REGBUF_F_DMABUF))) + return; + dma_buf_io_map_drop(req->dmabuf_map); + req->flags &=3D ~REQ_F_DROP_DMABUF; +} + +static int io_import_dmabuf(struct io_kiocb *req, + int ddir, struct iov_iter *iter, + struct io_mapped_ubuf *imu, + size_t len, size_t offset, + unsigned issue_flags) +{ + struct io_regbuf_dma *db =3D imu->priv; + struct dma_buf_io_map *map; + + if (!IS_ENABLED(CONFIG_DMA_SHARED_BUFFER)) + return -EOPNOTSUPP; + if (!len) + return -EFAULT; + if (req->file !=3D db->target_file) + return -EBADF; + + if (req->flags & REQ_F_DROP_DMABUF) { + map =3D req->dmabuf_map; + goto init_iter; + } + + map =3D dma_buf_io_get_map(&db->ctx); + if (unlikely(!map)) { + if (!(issue_flags & IO_URING_F_IOWQ)) + return -EAGAIN; + map =3D dma_buf_io_create_map(&db->ctx); + if (IS_ERR(map)) + return PTR_ERR(map); + } + + req->dmabuf_map =3D map; + req->flags |=3D REQ_F_DROP_DMABUF; +init_iter: + iov_iter_dmabuf_map(iter, ddir, map, offset, len); + return 0; +} + +static int io_import_fixed(struct io_kiocb *req, + int ddir, struct iov_iter *iter, struct io_mapped_ubuf *imu, - u64 buf_addr, size_t len) + u64 buf_addr, size_t len, + unsigned issue_flags, + unsigned import_flags) { const struct bio_vec *bvec; size_t folio_mask; @@ -1192,6 +1342,12 @@ static int io_import_fixed(int ddir, struct iov_iter= *iter, =20 offset =3D buf_addr - imu->ubuf; =20 + if (imu->flags & IO_REGBUF_F_DMABUF) { + if (!(import_flags & IO_REGBUF_IMPORT_ALLOW_DMABUF)) + return -EFAULT; + return io_import_dmabuf(req, ddir, iter, imu, len, offset, + issue_flags); + } if (imu->flags & IO_REGBUF_F_KBUF) return io_import_kbuf(ddir, iter, imu, len, offset); =20 @@ -1254,7 +1410,8 @@ int __io_import_reg_buf(struct io_kiocb *req, struct = iov_iter *iter, node =3D io_find_buf_node(req, issue_flags); if (!node) return -EFAULT; - return io_import_fixed(ddir, iter, node->buf, buf_addr, len); + return io_import_fixed(req, ddir, iter, node->buf, buf_addr, len, + issue_flags, import_flags); } =20 static int io_buffer_acct_cloned_hpages(struct io_ring_ctx *ctx, @@ -1676,7 +1833,9 @@ int __io_import_reg_vec(int ddir, struct iov_iter *it= er, iovec_off =3D vec->nr - nr_iovs; iov =3D vec->iovec + iovec_off; =20 - if (imu->flags & IO_REGBUF_F_KBUF) { + if (imu->flags & IO_REGBUF_F_DMABUF) { + return -EOPNOTSUPP; + } else if (imu->flags & IO_REGBUF_F_KBUF) { int ret =3D io_kern_bvec_size(iov, nr_iovs, imu, &nr_segs); =20 if (unlikely(ret)) diff --git a/io_uring/rsrc.h b/io_uring/rsrc.h index f12abaf63270..3cb9b57f159b 100644 --- a/io_uring/rsrc.h +++ b/io_uring/rsrc.h @@ -31,6 +31,11 @@ enum { enum { IO_REGBUF_F_KBUF =3D 1 << 0, IO_REGBUF_F_UNCLONEABLE =3D 1 << 1, + IO_REGBUF_F_DMABUF =3D 1 << 3, +}; + +enum { + IO_REGBUF_IMPORT_ALLOW_DMABUF =3D 1 << 1, }; =20 struct io_mapped_ubuf { @@ -173,4 +178,17 @@ static inline void io_alloc_cache_vec_kasan(struct iou= _vec *iv) io_vec_free(iv); } =20 +void io_drop_dmabuf_node(struct io_kiocb *req); + +static inline void io_req_drop_dmabuf(struct io_kiocb *req) +{ + if (!IS_ENABLED(CONFIG_DMA_SHARED_BUFFER)) + return; + if (!(req->flags & REQ_F_DROP_DMABUF)) + return; + if (WARN_ON_ONCE(!(req->flags & REQ_F_BUF_NODE))) + return; + io_drop_dmabuf_node(req); +} + #endif diff --git a/io_uring/rw.c b/io_uring/rw.c index 95038cfda615..11ff7b5a1164 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -380,8 +380,8 @@ static int io_init_rw_fixed(struct io_kiocb *req, unsig= ned int issue_flags, if (io->bytes_done) return 0; =20 - ret =3D io_import_reg_buf(req, &io->iter, rw->addr, rw->len, ddir, - issue_flags); + ret =3D __io_import_reg_buf(req, &io->iter, rw->addr, rw->len, ddir, + issue_flags, IO_REGBUF_IMPORT_ALLOW_DMABUF); iov_iter_save_state(&io->iter, &io->iter_state); return ret; } @@ -591,6 +591,8 @@ static void io_complete_rw(struct kiocb *kiocb, long re= s) struct io_rw *rw =3D container_of(kiocb, struct io_rw, kiocb); struct io_kiocb *req =3D cmd_to_io_kiocb(rw); =20 + io_req_drop_dmabuf(req); + __io_complete_rw_common(req, res); io_req_set_res(req, io_fixup_rw_res(req, res), 0); req->io_task_work.func =3D io_req_rw_complete; --=20 2.54.0