From nobody Tue Apr 7 19:54:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 053C2327C08; Wed, 11 Mar 2026 19:09:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256142; cv=none; b=YtiuxbPz8oATUqVsW0+Ybqrb9SsmCL4VqQgvpTDzUGjDVMNRTdUgTk9eyns0f70gsLDSVatzkEY7n+ZmBMEi8I4tf/lpj0cyjYongJmOYBLdcNY5HzCzdlxR3IQTQHhsqijA21ib+sN0+mSXjcZlgjTHRtSvZXnjVs9ukK8X5Ok= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256142; c=relaxed/simple; bh=Rb7pGPHPDn2aZQQvadJE9PAK4yDKeRrpgUBQyVOU34k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=RytpqBMUr3NQPItzouUWbHC4yTmKuB0uXdNYXgnPYtFkfSPQ+riez2+8kuuCg2YbeQfcdVhwQHWMdQKog0vLiZcnUd5a6NGMAcVZda6KV+3nxarQ6i/0yDKDHSxEJO6Fu8Wjf0ofHnngBD6qz+MyZG21QaYTK+nNfwfTY9eEOZ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=t6Q+l0cn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="t6Q+l0cn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 059A3C4AF09; Wed, 11 Mar 2026 19:09:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773256141; bh=Rb7pGPHPDn2aZQQvadJE9PAK4yDKeRrpgUBQyVOU34k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t6Q+l0cnDFbPljYyv2EXPuve6Y9BMQ4YF+mnAza1NJccaQKUWdO1Q2BghWaQ1FoHT MOnlUXqUDTnm/7+vptTBowOzAK/hXaqbxr7zYjsO1yoogF0svatd4Rfj6R1eqkaNcn o0/x1FObC/g8ppv2cCeDUuKn+Xo1qBUs794lJKw/VOlgkly8kD9wdYWJcVRHvbA46V C1v5ApXRuA7ycrshsRO668Fgwa3JF+jf2VOObCUBQE083YMwm2VpniTUH5o7hnAEis wcPpuY/QPeAv+bv/IPMygcJ/hGb4RaENbyJCMELlY44REV2vD1zmN/Un5Kh0XjLbz4 cdoYGJDHFbIyA== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Joerg Roedel , Will Deacon , Andrew Morton Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 1/8] dma-debug: Allow multiple invocations of overlapping entries Date: Wed, 11 Mar 2026 21:08:44 +0200 Message-ID: <20260311-dma-debug-overlap-v2-1-e00bc2ca346d@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> References: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky Repeated DMA mappings with DMA_ATTR_CPU_CACHE_CLEAN trigger the following splat. This prevents using the attribute in cases where a DMA region is shared and reused more than seven times. ------------[ cut here ]------------ DMA-API: exceeded 7 overlapping mappings of cacheline 0x000000000438c440 WARNING: kernel/dma/debug.c:467 at add_dma_entry+0x219/0x280, CPU#4: ibv_r= c_pingpong/1644 Modules linked in: xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetli= nk iptable_nat nf_nat xt_addrtype br_netfilter rpcsec_gss_krb5 auth_rpcgss = oid_registry overlay mlx5_fwctl zram zsmalloc mlx5_ib fuse rpcrdma rdma_ucm= ib_uverbs ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ipoib i= w_cm ib_cm mlx5_core ib_core CPU: 4 UID: 2733 PID: 1644 Comm: ibv_rc_pingpong Not tainted 6.19.0+ #129 = PREEMPT Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21= b5a4aeb02-prebuilt.qemu.org 04/01/2014 RIP: 0010:add_dma_entry+0x221/0x280 Code: c0 0f 84 f2 fe ff ff 83 e8 01 89 05 6d 99 11 01 e9 e4 fe ff ff 0f 8e= 1f ff ff ff 48 8d 3d 07 ef 2d 01 be 07 00 00 00 48 89 e2 <67> 48 0f b9 3a = e9 06 ff ff ff 48 c7 c7 98 05 2b 82 c6 05 72 92 28 RSP: 0018:ff1100010e657970 EFLAGS: 00010002 RAX: 0000000000000007 RBX: ff1100010234eb00 RCX: 0000000000000000 RDX: ff1100010e657970 RSI: 0000000000000007 RDI: ffffffff82678660 RBP: 000000000438c440 R08: 0000000000000228 R09: 0000000000000000 R10: 00000000000001be R11: 000000000000089d R12: 0000000000000800 R13: 00000000ffffffef R14: 0000000000000202 R15: ff1100010234eb00 FS: 00007fb15f3f6740(0000) GS:ff110008dcc19000(0000) knlGS:00000000000000= 00 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fb15f32d3a0 CR3: 0000000116f59001 CR4: 0000000000373eb0 Call Trace: debug_dma_map_sg+0x1b4/0x390 __dma_map_sg_attrs+0x6d/0x1a0 dma_map_sgtable+0x19/0x30 ib_umem_get+0x284/0x3b0 [ib_uverbs] mlx5_ib_reg_user_mr+0x68/0x2a0 [mlx5_ib] ib_uverbs_reg_mr+0x17f/0x2a0 [ib_uverbs] ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xc2/0x130 [ib_uverbs] ib_uverbs_cmd_verbs+0xa0b/0xae0 [ib_uverbs] ? ib_uverbs_handler_UVERBS_METHOD_QUERY_PORT_SPEED+0xe0/0xe0 [ib_uverbs] ? mmap_region+0x7a/0xb0 ? do_mmap+0x3b8/0x5c0 ib_uverbs_ioctl+0xa7/0x110 [ib_uverbs] __x64_sys_ioctl+0x14f/0x8b0 ? ksys_mmap_pgoff+0xc5/0x190 do_syscall_64+0x8c/0xbf0 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x7fb15f5e4eed Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00= 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 = ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 RSP: 002b:00007ffe09a5c540 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 00007ffe09a5c5d0 RCX: 00007fb15f5e4eed RDX: 00007ffe09a5c5f0 RSI: 00000000c0181b01 RDI: 0000000000000003 RBP: 00007ffe09a5c590 R08: 0000000000000028 R09: 00007ffe09a5c794 R10: 0000000000000001 R11: 0000000000000246 R12: 00007ffe09a5c794 R13: 000000000000000c R14: 0000000025a49170 R15: 000000000000000c ---[ end trace 0000000000000000 ]--- Fixes: 61868dc55a11 ("dma-mapping: add DMA_ATTR_CPU_CACHE_CLEAN") Signed-off-by: Leon Romanovsky --- kernel/dma/debug.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 86f87e43438c3..be207be749968 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -453,7 +453,7 @@ static int active_cacheline_set_overlap(phys_addr_t cln= , int overlap) return overlap; } =20 -static void active_cacheline_inc_overlap(phys_addr_t cln) +static void active_cacheline_inc_overlap(phys_addr_t cln, bool is_cache_cl= ean) { int overlap =3D active_cacheline_read_overlap(cln); =20 @@ -462,7 +462,7 @@ static void active_cacheline_inc_overlap(phys_addr_t cl= n) /* If we overflowed the overlap counter then we're potentially * leaking dma-mappings. */ - WARN_ONCE(overlap > ACTIVE_CACHELINE_MAX_OVERLAP, + WARN_ONCE(!is_cache_clean && overlap > ACTIVE_CACHELINE_MAX_OVERLAP, pr_fmt("exceeded %d overlapping mappings of cacheline %pa\n"), ACTIVE_CACHELINE_MAX_OVERLAP, &cln); } @@ -495,7 +495,7 @@ static int active_cacheline_insert(struct dma_debug_ent= ry *entry, if (rc =3D=3D -EEXIST) { struct dma_debug_entry *existing; =20 - active_cacheline_inc_overlap(cln); + active_cacheline_inc_overlap(cln, entry->is_cache_clean); existing =3D radix_tree_lookup(&dma_active_cacheline, cln); /* A lookup failure here after we got -EEXIST is unexpected. */ WARN_ON(!existing); --=20 2.53.0 From nobody Tue Apr 7 19:54:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EE67367F4A; Wed, 11 Mar 2026 19:09:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256152; cv=none; b=nHuJT53rvHVs59M6uKodj9JeKYvvf8Nndy6Clh7jPM7yr//DelLeBVBdQOblCeF+OF0n739x7UPrnHQALClH1u8CI5BTFuscUtx9ieMOURLRIcH9JQtKqSV51E1PKDYeC1yXqG93IJIM8dKrbim6Ef1lFDPU2HLSogMJl5o9aFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256152; c=relaxed/simple; bh=Y7IxYgFPwpNVvKXeK9nR1uoDv1sEB/976av/0RuDIXA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=qj4q6m8AqqxCoH2+uuPhxKlo9+xeV1skU0SboWNRogxtdaoYpzQv1TLfD+YONCUlL/CjjUe4NEsX6L1gH5wvMRwq5dbq73uSkNbdvDLDKIESZ7ZmhIr+1CTgvNZN94Bcl3jdMFl9SPQknl1y0fO7B6RwUApZM2PNOlPNLHkWf5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PZjly+FD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PZjly+FD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3AAC1C116C6; Wed, 11 Mar 2026 19:09:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773256151; bh=Y7IxYgFPwpNVvKXeK9nR1uoDv1sEB/976av/0RuDIXA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PZjly+FD7RhrJxb1JD1VgY4spyEehNCANCCnx9mVnO9J4cWhenNclePe/JK1Iy/Zy +lffzBEJ96sc2SLnsy9exQHyisEszVRb6oOq7EVzSXq8W0Yh6WxIr+2XUnyKA/cLf1 m/pz6fl6AezPbzmXIBszDFIE6tSwhBG826fO3jBsXgikveu0HxOOIIzjIiVCwod0no h8cuXF1gV1UO9KAr6ZbxDb+aFehyKFej3C2cqyHB/TvpuI/c+GbC7iBQIckvCaERVV LBj5kuI7vjP92AtWxD4vuJJ7/CW8N+ksu9l1OeuVKXABVX+04P7FuPyblNDEpBBNcp 5RSfo2p/vb5sg== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Joerg Roedel , Will Deacon , Andrew Morton Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 2/8] dma-mapping: handle DMA_ATTR_CPU_CACHE_CLEAN in trace output Date: Wed, 11 Mar 2026 21:08:45 +0200 Message-ID: <20260311-dma-debug-overlap-v2-2-e00bc2ca346d@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> References: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky Tracing prints decoded DMA attribute flags, but it does not yet include the recently added DMA_ATTR_CPU_CACHE_CLEAN. Add support for decoding and displaying this attribute in the trace output. Fixes: 61868dc55a11 ("dma-mapping: add DMA_ATTR_CPU_CACHE_CLEAN") Signed-off-by: Leon Romanovsky --- include/trace/events/dma.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/trace/events/dma.h b/include/trace/events/dma.h index 33e99e792f1aa..69cb3805ee81c 100644 --- a/include/trace/events/dma.h +++ b/include/trace/events/dma.h @@ -32,7 +32,8 @@ TRACE_DEFINE_ENUM(DMA_NONE); { DMA_ATTR_ALLOC_SINGLE_PAGES, "ALLOC_SINGLE_PAGES" }, \ { DMA_ATTR_NO_WARN, "NO_WARN" }, \ { DMA_ATTR_PRIVILEGED, "PRIVILEGED" }, \ - { DMA_ATTR_MMIO, "MMIO" }) + { DMA_ATTR_MMIO, "MMIO" }, \ + { DMA_ATTR_CPU_CACHE_CLEAN, "CACHE_CLEAN" }) =20 DECLARE_EVENT_CLASS(dma_map, TP_PROTO(struct device *dev, phys_addr_t phys_addr, dma_addr_t dma_addr, --=20 2.53.0 From nobody Tue Apr 7 19:54:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70C45359A8B; Wed, 11 Mar 2026 19:09:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256145; cv=none; b=MdJVt/goFjHOykpenmAMOnyqLmkqHtla5EdKzQE/TtETJ1Wjrpyg5oq5BU/IjSzmrFt0PcAe+it8nZN6wDrnijW9zWu8TMrA5oycSdTgG0zlLncwYBaoDPn4QM12/Hb7d9Nn08/OmK02py+6+EZNupCvlwoF1uyPew7u9ghrnaI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256145; c=relaxed/simple; bh=M+wL0ybsphtwzmN54IoSxHDbpmj04d4gwnhHVj/1g/U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=IRnpR3EHlxHK+qYaTG8n+pKpg3foQJqmvdy1+HydtsjOcRWbzdcym7y1KHyLziY2+MeYJh/W/5HIX7LrNpdysdpdMJiKzD6IN4fOtvDBJKIH2kWFMbTtd+SoiIZ9yCxjgdwmVEcTPI5qs33r7XCuhLsXt/CxDjO/b9G9mQ14wpc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bq6HPes1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bq6HPes1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 748B8C4CEF7; Wed, 11 Mar 2026 19:09:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773256145; bh=M+wL0ybsphtwzmN54IoSxHDbpmj04d4gwnhHVj/1g/U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bq6HPes1GCOpxb+476WWxenSgikmhptaTzm7G7VrOIhF19PrU6EP/jCrt0DRzVs25 rzff1g5gPQv2CmJYWz12JIQrto6jkTuJlviAhCCwhitUjZYB4viFnTElRDZzAmOcyn 0x2F9IE6jw9qWIKCAzjAwWTG/5DW+gZce23SzSWxrccQTA41bPjU8b6SGjhHEOpT+4 wqk0yrxRLUEmbplSHSSBsA5u5t2M0q/t09z1NzzETRa2LzzIIA5VjkY/RStLd2q+zj MWlm4GrYD4/BATmaVtrTawWypMziYPLi3u5dQeNTQ0RYxSIoXIv+zHr2XlqFKQZRDU hw49rGkodBjdQ== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Joerg Roedel , Will Deacon , Andrew Morton Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 3/8] dma-mapping: Clarify valid conditions for CPU cache line overlap Date: Wed, 11 Mar 2026 21:08:46 +0200 Message-ID: <20260311-dma-debug-overlap-v2-3-e00bc2ca346d@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> References: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky Rename the DMA_ATTR_CPU_CACHE_CLEAN attribute to better reflect that it is debugging aid to inform DMA core code that CPU cache line overlaps are allowed, and refine the documentation describing its use. Signed-off-by: Leon Romanovsky --- Documentation/core-api/dma-attributes.rst | 22 ++++++++++++++-------- drivers/virtio/virtio_ring.c | 10 +++++----- include/linux/dma-mapping.h | 8 ++++---- include/trace/events/dma.h | 2 +- kernel/dma/debug.c | 2 +- 5 files changed, 25 insertions(+), 19 deletions(-) diff --git a/Documentation/core-api/dma-attributes.rst b/Documentation/core= -api/dma-attributes.rst index 1d7bfad73b1c7..48cfe86cc06d7 100644 --- a/Documentation/core-api/dma-attributes.rst +++ b/Documentation/core-api/dma-attributes.rst @@ -149,11 +149,17 @@ For architectures that require cache flushing for DMA= coherence DMA_ATTR_MMIO will not perform any cache flushing. The address provided must never be mapped cacheable into the CPU. =20 -DMA_ATTR_CPU_CACHE_CLEAN ------------------------- - -This attribute indicates the CPU will not dirty any cacheline overlapping = this -DMA_FROM_DEVICE/DMA_BIDIRECTIONAL buffer while it is mapped. This allows -multiple small buffers to safely share a cacheline without risk of data -corruption, suppressing DMA debug warnings about overlapping mappings. -All mappings sharing a cacheline should have this attribute. +DMA_ATTR_DEBUGGING_IGNORE_CACHELINES +------------------------------------ + +This attribute indicates that CPU cache lines may overlap for buffers mapp= ed +with DMA_FROM_DEVICE or DMA_BIDIRECTIONAL. + +Such overlap may occur when callers map multiple small buffers that reside +within the same cache line. In this case, callers must guarantee that the = CPU +will not dirty these cache lines after the mappings are established. When = this +condition is met, multiple buffers can safely share a cache line without r= isking +data corruption. + +All mappings that share a cache line must set this attribute to suppress D= MA +debug warnings about overlapping mappings. diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 335692d41617a..fbca7ce1c6bf0 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2912,10 +2912,10 @@ EXPORT_SYMBOL_GPL(virtqueue_add_inbuf); * @data: the token identifying the buffer. * @gfp: how to do memory allocations (if necessary). * - * Same as virtqueue_add_inbuf but passes DMA_ATTR_CPU_CACHE_CLEAN to indi= cate - * that the CPU will not dirty any cacheline overlapping this buffer while= it - * is available, and to suppress overlapping cacheline warnings in DMA deb= ug - * builds. + * Same as virtqueue_add_inbuf but passes DMA_ATTR_DEBUGGING_IGNORE_CACHEL= INES + * to indicate that the CPU will not dirty any cacheline overlapping this = buffer + * while it is available, and to suppress overlapping cacheline warnings i= n DMA + * debug builds. * * Caller must ensure we don't call this with other virtqueue operations * at the same time (except where noted). @@ -2928,7 +2928,7 @@ int virtqueue_add_inbuf_cache_clean(struct virtqueue = *vq, gfp_t gfp) { return virtqueue_add(vq, &sg, num, 0, 1, data, NULL, false, gfp, - DMA_ATTR_CPU_CACHE_CLEAN); + DMA_ATTR_DEBUGGING_IGNORE_CACHELINES); } EXPORT_SYMBOL_GPL(virtqueue_add_inbuf_cache_clean); =20 diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 29973baa05816..da44394b3a1a7 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -80,11 +80,11 @@ #define DMA_ATTR_MMIO (1UL << 10) =20 /* - * DMA_ATTR_CPU_CACHE_CLEAN: Indicates the CPU will not dirty any cacheline - * overlapping this buffer while it is mapped for DMA. All mappings sharing - * a cacheline must have this attribute for this to be considered safe. + * DMA_ATTR_DEBUGGING_IGNORE_CACHELINES: Indicates the CPU cache line can = be + * overlapped. All mappings sharing a cacheline must have this attribute f= or + * this to be considered safe. */ -#define DMA_ATTR_CPU_CACHE_CLEAN (1UL << 11) +#define DMA_ATTR_DEBUGGING_IGNORE_CACHELINES (1UL << 11) =20 /* * A dma_addr_t can hold any valid DMA or bus address for the platform. I= t can diff --git a/include/trace/events/dma.h b/include/trace/events/dma.h index 69cb3805ee81c..8c64bc0721fe4 100644 --- a/include/trace/events/dma.h +++ b/include/trace/events/dma.h @@ -33,7 +33,7 @@ TRACE_DEFINE_ENUM(DMA_NONE); { DMA_ATTR_NO_WARN, "NO_WARN" }, \ { DMA_ATTR_PRIVILEGED, "PRIVILEGED" }, \ { DMA_ATTR_MMIO, "MMIO" }, \ - { DMA_ATTR_CPU_CACHE_CLEAN, "CACHE_CLEAN" }) + { DMA_ATTR_DEBUGGING_IGNORE_CACHELINES, "CACHELINES_OVERLAP" }) =20 DECLARE_EVENT_CLASS(dma_map, TP_PROTO(struct device *dev, phys_addr_t phys_addr, dma_addr_t dma_addr, diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index be207be749968..83e1cfe05f08d 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -601,7 +601,7 @@ static void add_dma_entry(struct dma_debug_entry *entry= , unsigned long attrs) unsigned long flags; int rc; =20 - entry->is_cache_clean =3D !!(attrs & DMA_ATTR_CPU_CACHE_CLEAN); + entry->is_cache_clean =3D attrs & DMA_ATTR_DEBUGGING_IGNORE_CACHELINES; =20 bucket =3D get_hash_bucket(entry, &flags); hash_bucket_add(bucket, entry); --=20 2.53.0 From nobody Tue Apr 7 19:54:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E105D366DCC; Wed, 11 Mar 2026 19:09:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256149; cv=none; b=MQ9GCDAi8enyT52wRJJdoUdAjtTpckkIls6QxxJVRruRWPAnktL4FKgQCAp/bxPHaeMN6KELfFkDqytRFZtTZp4DEEru37vaZftCSGjw/9C1uf+ueiycI5BvbYwsj28ZQ+1ztmQFkKYhENL44AYr+k7JHdmJ0jWyFnof2Yn7+5M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256149; c=relaxed/simple; bh=Caib8MH0BZPS5GoBVp5443hRjoVeLP/iROeH38ZKBek=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Z8n93Kpz1PIW6+GO7inilj6LIOYWwEO/wUhv+ppnBFWNv6dk5q5IQapmqlTAHluvQs4rYUmY0Kxr0buygGQ5ridaR1pG+HWZDoZk2yZXMaeAXk2b5EOT/Vfan9e03zORhuiiT336WKjWVdO8hukIlhf/Mcn/WwJKmMhFyVw3kSs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=uX7fuLiv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="uX7fuLiv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id ECEB6C4CEF7; Wed, 11 Mar 2026 19:09:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773256148; bh=Caib8MH0BZPS5GoBVp5443hRjoVeLP/iROeH38ZKBek=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uX7fuLiv2dqrXYExHFjcHx7N9C8rKXe9c03Xu0WIBHJmwCB9rHlFRYKVS82RDSyi4 SbhA5HmIgdVVR4KsVb4/1mFJ6KqvncGYuwNwZSda5Zjon4pY9/tgioJs2Xm+pql3oh LWyIiOyY0mGkP9F1jPPS2RsqY9jiS9KoqgyrB3rH9emmpAG8tV5IR6xKxuWhUftFkI fZ3veNg46sNPb7csrLsgu0bdyxFY4IjnGZY9iHwkH2f2DipheEWOq90pkMcFloOXvW Q8awvutvZGHd2RgO5mAfioeREd771cZIuR58KwK+upvv/yITUKz0VacsDBuOe3Sr4T IsZWX4Mz5m9Mw== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Joerg Roedel , Will Deacon , Andrew Morton Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 4/8] dma-mapping: Introduce DMA require coherency attribute Date: Wed, 11 Mar 2026 21:08:47 +0200 Message-ID: <20260311-dma-debug-overlap-v2-4-e00bc2ca346d@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> References: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky The mapping buffers which carry this attribute require DMA coherent system. This means that they can't take SWIOTLB path, can perform CPU cache overlap and doesn't perform cache flushing. Signed-off-by: Leon Romanovsky --- Documentation/core-api/dma-attributes.rst | 12 ++++++++++++ include/linux/dma-mapping.h | 7 +++++++ include/trace/events/dma.h | 3 ++- kernel/dma/debug.c | 3 ++- kernel/dma/mapping.c | 6 ++++++ 5 files changed, 29 insertions(+), 2 deletions(-) diff --git a/Documentation/core-api/dma-attributes.rst b/Documentation/core= -api/dma-attributes.rst index 48cfe86cc06d7..69d094f144c70 100644 --- a/Documentation/core-api/dma-attributes.rst +++ b/Documentation/core-api/dma-attributes.rst @@ -163,3 +163,15 @@ data corruption. =20 All mappings that share a cache line must set this attribute to suppress D= MA debug warnings about overlapping mappings. + +DMA_ATTR_REQUIRE_COHERENT +------------------------- + +The mapping buffers which carry this attribute require DMA coherent system= . This means +that they can't take SWIOTLB path, can perform CPU cache overlap and doesn= 't perform +cache flushing. + +If the mapping has this attribute then it is prevented from running on sys= tems +where these cache artifacts can cause corruption, and as such doesn't need +cache overlapping debugging code (same behavior as for +DMA_ATTR_DEBUGGING_IGNORE_CACHELINES). diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index da44394b3a1a7..482b919f040f7 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -86,6 +86,13 @@ */ #define DMA_ATTR_DEBUGGING_IGNORE_CACHELINES (1UL << 11) =20 +/* + * DMA_ATTR_REQUIRE_COHERENT: Indicates that DMA coherency is required. + * All mappings that carry this attribute can't work with SWIOTLB and cache + * flushing. + */ +#define DMA_ATTR_REQUIRE_COHERENT (1UL << 12) + /* * A dma_addr_t can hold any valid DMA or bus address for the platform. I= t can * be given to a device to use as a DMA source or target. It is specific = to a diff --git a/include/trace/events/dma.h b/include/trace/events/dma.h index 8c64bc0721fe4..63597b0044247 100644 --- a/include/trace/events/dma.h +++ b/include/trace/events/dma.h @@ -33,7 +33,8 @@ TRACE_DEFINE_ENUM(DMA_NONE); { DMA_ATTR_NO_WARN, "NO_WARN" }, \ { DMA_ATTR_PRIVILEGED, "PRIVILEGED" }, \ { DMA_ATTR_MMIO, "MMIO" }, \ - { DMA_ATTR_DEBUGGING_IGNORE_CACHELINES, "CACHELINES_OVERLAP" }) + { DMA_ATTR_DEBUGGING_IGNORE_CACHELINES, "CACHELINES_OVERLAP" }, \ + { DMA_ATTR_REQUIRE_COHERENT, "REQUIRE_COHERENT" }) =20 DECLARE_EVENT_CLASS(dma_map, TP_PROTO(struct device *dev, phys_addr_t phys_addr, dma_addr_t dma_addr, diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 83e1cfe05f08d..0677918f06a80 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -601,7 +601,8 @@ static void add_dma_entry(struct dma_debug_entry *entry= , unsigned long attrs) unsigned long flags; int rc; =20 - entry->is_cache_clean =3D attrs & DMA_ATTR_DEBUGGING_IGNORE_CACHELINES; + entry->is_cache_clean =3D attrs & (DMA_ATTR_DEBUGGING_IGNORE_CACHELINES | + DMA_ATTR_REQUIRE_COHERENT); =20 bucket =3D get_hash_bucket(entry, &flags); hash_bucket_add(bucket, entry); diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c index 3928a509c44c2..6d3dd0bd3a886 100644 --- a/kernel/dma/mapping.c +++ b/kernel/dma/mapping.c @@ -164,6 +164,9 @@ dma_addr_t dma_map_phys(struct device *dev, phys_addr_t= phys, size_t size, if (WARN_ON_ONCE(!dev->dma_mask)) return DMA_MAPPING_ERROR; =20 + if (!dev_is_dma_coherent(dev) && (attrs & DMA_ATTR_REQUIRE_COHERENT)) + return DMA_MAPPING_ERROR; + if (dma_map_direct(dev, ops) || (!is_mmio && arch_dma_map_phys_direct(dev, phys + size))) addr =3D dma_direct_map_phys(dev, phys, size, dir, attrs); @@ -235,6 +238,9 @@ static int __dma_map_sg_attrs(struct device *dev, struc= t scatterlist *sg, =20 BUG_ON(!valid_dma_direction(dir)); =20 + if (!dev_is_dma_coherent(dev) && (attrs & DMA_ATTR_REQUIRE_COHERENT)) + return -EOPNOTSUPP; + if (WARN_ON_ONCE(!dev->dma_mask)) return 0; =20 --=20 2.53.0 From nobody Tue Apr 7 19:54:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55C643CF032; Wed, 11 Mar 2026 19:09:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256167; cv=none; b=HQj+w29171Ow5h7l8P4L6Vinb8c7wFkc7olWZYYidW8OfUAGSonlFHL3cY59g37sf3FwGJaO3LiO8L6cOTad3VmwAMSlWTO5R8yPcGwLy2DzCVnJcEj6AF3/eOJOAqFtRm9q39n1H+Jd0QNroe4av/ShUudwqhOAweNYH8LTzhQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256167; c=relaxed/simple; bh=8pwuaCiDUSCBcuJ0n9V3xnzMZmgOorpLbuob290/9q4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=puo1njSIq/6zKedsvaB/xzth4NRfb/WxHRy5OGZND7oJfgyR78VAgSsKoSqH/QB9IrWtYDrJK6xUMi4bPaa9xfdj9Q+fMCW3+56E2Y3EtaSJgzzUJjz/2HAXjHj7+jQ0z9ufHHGjY42r1mwbXYdNiLXol32NAMBc+rvjZ7Xu+Wk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fHKnRO6y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fHKnRO6y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8658CC4CEF7; Wed, 11 Mar 2026 19:09:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773256167; bh=8pwuaCiDUSCBcuJ0n9V3xnzMZmgOorpLbuob290/9q4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fHKnRO6yqtY5rofM6SDr4DHDapAJhsI0XJQi4O9p65/khxIvwwDdKmxcfP8QYi6A2 m104LZo+8JeIN3EoFgQM5Elab2cHsl5k8qdT1jPMH3o3Dzz6r5kBM+nkshU7bgmuWj iEcMBWWDHna5UV8c+CTUfuJ6W6gkfU1mWntrsCIn5VobxHkCLxQcD7/p0cM11R/5kr F2/iLjoVMLIfU9H6nkJoCL4rhAdIR+aXP4WOX2Dvv4Kv8MnhhPCcvvORJXpnDAduD5 uVQcX6poI5WbBIl7IXu3Irg6tUKQGpfuh/AJs2iPba4QxwYO9dwxL3VAGMO2PcW34p Tzux8OKSdH0Mw== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Joerg Roedel , Will Deacon , Andrew Morton Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 5/8] dma-direct: prevent SWIOTLB path when DMA_ATTR_REQUIRE_COHERENT is set Date: Wed, 11 Mar 2026 21:08:48 +0200 Message-ID: <20260311-dma-debug-overlap-v2-5-e00bc2ca346d@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> References: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky DMA_ATTR_REQUIRE_COHERENT indicates that SWIOTLB must not be used. Ensure the SWIOTLB path is declined whenever the DMA direct path is selected. Signed-off-by: Leon Romanovsky --- kernel/dma/direct.h | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/kernel/dma/direct.h b/kernel/dma/direct.h index e89f175e9c2d0..6184ff303f080 100644 --- a/kernel/dma/direct.h +++ b/kernel/dma/direct.h @@ -84,7 +84,7 @@ static inline dma_addr_t dma_direct_map_phys(struct devic= e *dev, dma_addr_t dma_addr; =20 if (is_swiotlb_force_bounce(dev)) { - if (attrs & DMA_ATTR_MMIO) + if (attrs & (DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT)) return DMA_MAPPING_ERROR; =20 return swiotlb_map(dev, phys, size, dir, attrs); @@ -98,7 +98,8 @@ static inline dma_addr_t dma_direct_map_phys(struct devic= e *dev, dma_addr =3D phys_to_dma(dev, phys); if (unlikely(!dma_capable(dev, dma_addr, size, true)) || dma_kmalloc_needs_bounce(dev, size, dir)) { - if (is_swiotlb_active(dev)) + if (is_swiotlb_active(dev) && + !(attrs & DMA_ATTR_REQUIRE_COHERENT)) return swiotlb_map(dev, phys, size, dir, attrs); =20 goto err_overflow; @@ -123,7 +124,7 @@ static inline void dma_direct_unmap_phys(struct device = *dev, dma_addr_t addr, { phys_addr_t phys; =20 - if (attrs & DMA_ATTR_MMIO) + if (attrs & (DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT)) /* nothing to do: uncached and no swiotlb */ return; =20 --=20 2.53.0 From nobody Tue Apr 7 19:54:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7781636E469; Wed, 11 Mar 2026 19:09:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256156; cv=none; b=dPULr7ZoYZnBFV3jUFalN8gWjFprzOXi2RgNx2yXYsRxq381plYsh4NyHAbU8kTtfdAmCbwGzXZ2Vrz25EGzK5juwynptNHDrPvBjsrNgVtKV4SWI2BwLd4LwNtB4trYe7C9Bd7Z6epvCabGYR0zJGnpxRKnQcvyrrWXumG8GC4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256156; c=relaxed/simple; bh=aXwCdr26DGL4+NaZEE3PT+YDkt3E2yVN7za/ReeSxow=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=AsSemurQszG8PmNZprEbEhs7OoZ8UlIqM4aetnUyv41ElcEzsNenwZlzf2afH9qn9BAoLrlgQY+9nS7TvG3ZChOh+ph9njw0p1uSnSE7epU2mrjgXivQQAmhN2wl4s34w0vRV8G9xF7843DxTZVgEaSaAUPhBTNzm5UbRzXdB0M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hfT6VO/H; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hfT6VO/H" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AFB91C4CEF7; Wed, 11 Mar 2026 19:09:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773256156; bh=aXwCdr26DGL4+NaZEE3PT+YDkt3E2yVN7za/ReeSxow=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hfT6VO/H2ei6g2r3v5zyGv7rxwT6gZVYEAZ3q/JxZD65/o54Se7wSvuPVHXwVG3KV lGDu5gO5rDUufjp8q+LowrU3Ke5y5jS1Y0bexFfXEB3tzQzQyOOftdHqt+R8iv65vI r7wvN46gsOkly1ZVrVr/Sl/g1XDPv9kDCCPnM7WliJ2076XljXXXFEhaxHnyVTNvjr gfEXB4mq7aNOf4+zV1wf2x9Jz+Ri4AjvTPzWFPes3dJEnWw/e595Owg7+LvCwGscfZ iT0Mis1SQnjow6HB3HyeJr9hbn/JtUUGlGfFied+XD1ozj+5qSP1XAToPA53V0fZrm dZB0p85e4RYfg== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Joerg Roedel , Will Deacon , Andrew Morton Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 6/8] iommu/dma: add support for DMA_ATTR_REQUIRE_COHERENT attribute Date: Wed, 11 Mar 2026 21:08:49 +0200 Message-ID: <20260311-dma-debug-overlap-v2-6-e00bc2ca346d@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> References: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky Add support for the DMA_ATTR_REQUIRE_COHERENT attribute to the exported functions. This attribute indicates that the SWIOTLB path must not be used and that no sync operations should be performed. Signed-off-by: Leon Romanovsky --- drivers/iommu/dma-iommu.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 5dac64be61bb2..94d5141696424 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1211,7 +1211,7 @@ dma_addr_t iommu_dma_map_phys(struct device *dev, phy= s_addr_t phys, size_t size, */ if (dev_use_swiotlb(dev, size, dir) && iova_unaligned(iovad, phys, size)) { - if (attrs & DMA_ATTR_MMIO) + if (attrs & (DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT)) return DMA_MAPPING_ERROR; =20 phys =3D iommu_dma_map_swiotlb(dev, phys, size, dir, attrs); @@ -1223,7 +1223,8 @@ dma_addr_t iommu_dma_map_phys(struct device *dev, phy= s_addr_t phys, size_t size, arch_sync_dma_for_device(phys, size, dir); =20 iova =3D __iommu_dma_map(dev, phys, size, prot, dma_mask); - if (iova =3D=3D DMA_MAPPING_ERROR && !(attrs & DMA_ATTR_MMIO)) + if (iova =3D=3D DMA_MAPPING_ERROR && + !(attrs & (DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT))) swiotlb_tbl_unmap_single(dev, phys, size, dir, attrs); return iova; } @@ -1233,7 +1234,7 @@ void iommu_dma_unmap_phys(struct device *dev, dma_add= r_t dma_handle, { phys_addr_t phys; =20 - if (attrs & DMA_ATTR_MMIO) { + if (attrs & (DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT)) { __iommu_dma_unmap(dev, dma_handle, size); return; } @@ -1945,9 +1946,21 @@ int dma_iova_link(struct device *dev, struct dma_iov= a_state *state, if (WARN_ON_ONCE(iova_start_pad && offset > 0)) return -EIO; =20 + /* + * DMA_IOVA_USE_SWIOTLB is set on state after some entry + * took SWIOTLB path, which we were supposed to prevent + * for DMA_ATTR_REQUIRE_COHERENT attribute. + */ + if (WARN_ON_ONCE((state->__size & DMA_IOVA_USE_SWIOTLB) && + (attrs & DMA_ATTR_REQUIRE_COHERENT))) + return -EOPNOTSUPP; + + if (!dev_is_dma_coherent(dev) && (attrs & DMA_ATTR_REQUIRE_COHERENT)) + return -EOPNOTSUPP; + if (dev_use_swiotlb(dev, size, dir) && iova_unaligned(iovad, phys, size)) { - if (attrs & DMA_ATTR_MMIO) + if (attrs & (DMA_ATTR_MMIO | DMA_ATTR_REQUIRE_COHERENT)) return -EPERM; =20 return iommu_dma_iova_link_swiotlb(dev, state, phys, offset, --=20 2.53.0 From nobody Tue Apr 7 19:54:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AD173AE6ED; Wed, 11 Mar 2026 19:09:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256160; cv=none; b=r+sCnCddOlElRc2IKjrIJI3saa3ITp+zQFKp5mn++fEYryDrkImukbrKFVLnaZZs93jzWtVja+ENuWIg/Lhhs4qNLcVvmm0PuWfIpwFlxSQMhf9NdE3zAyrd99NEZTJNi3WW1sucJg7APVn6L6oanY03mL1WVc61EKg9onaHJjY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256160; c=relaxed/simple; bh=gbK+v7FulbHCf4xeffX6pcY7t9uYKpGBULCh9FUaZDQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LVebcod5xX/cxzbfvZQyIMZ8XEXPJRXq6wMW2hX6uYLk7ncXxAPNh5w+hsN/oDk3POjwCLiC6cie3JogbnW9KtkdEQ/K8H1h6DlZL+1wOxv8H9FQV1Rk0M1lQGiYYwiUoSZu0DTlygRxjpMpOCNo8aso5wvQfjxzTmVawpveiGw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WH+8WPiK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WH+8WPiK" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 753B9C4CEF7; Wed, 11 Mar 2026 19:09:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773256160; bh=gbK+v7FulbHCf4xeffX6pcY7t9uYKpGBULCh9FUaZDQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WH+8WPiKHF8Dt8v7aKQoWV/m4UNwNYjgwSonq778FSXuQHPHb/lEGPIG7EyWj2uMv fZ1RkdjHx10MB5/q80znPzHQfuxWxreymIWQhCn1Ned3fMaBibrotQZVr03CCgdCSR DdKDV3J3Rd5ObPVLvOnsb1o7er6N1LQGE7sqKbJYQKGE0WmPuGqR43BGPXkVok/6+L TDvUNLoGp6DrhtujPksC89No8EXlSkkIXhG15iNEt/GKTR3AtvNBbpSrxc8LBHuuiw mPCxIQbK2A7bytiGW/D33YzwAuEg957pOE10uIkuiQtOzbtTknj18zbKxWsqx4WKS+ O+RAatznjIzVA== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Joerg Roedel , Will Deacon , Andrew Morton Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 7/8] RDMA/umem: Tell DMA mapping that UMEM requires coherency Date: Wed, 11 Mar 2026 21:08:50 +0200 Message-ID: <20260311-dma-debug-overlap-v2-7-e00bc2ca346d@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> References: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky The RDMA subsystem exposes DMA regions through the verbs interface, which assumes a coherent system. Use the DMA_ATTR_REQUIRE_COHERENCE attribute to ensure coherency and avoid taking the SWIOTLB path. In addition, a given region may be exported multiple times, which will trig= ger warnings about cacheline overlaps. These warnings are suppressed when using the new attribute. infiniband rocep8s0f0: mlx5_ib_reg_user_mr:1592:(pid 5812): start 0x2b28c00= 0, iova 0x2b28c000, length 0x1000, access_flags 0x1 infiniband rocep8s0f0: mlx5_ib_reg_user_mr:1592:(pid 5812): start 0x2b28c00= 1, iova 0x2b28c001, length 0xfff, access_flags 0x1 ------------[ cut here ]------------ DMA-API: mlx5_core 0000:08:00.0: cacheline tracking EEXIST, overlapping ma= ppings aren't supported WARNING: kernel/dma/debug.c:620 at add_dma_entry+0x1bb/0x280, CPU#6: ibv_r= c_pingpong/5812 Modules linked in: veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink nf= netlink iptable_nat nf_nat xt_addrtype br_netfilter rpcsec_gss_krb5 auth_rp= cgss oid_registry overlay mlx5_fwctl zram zsmalloc mlx5_ib fuse rpcrdma rdm= a_ucm ib_uverbs ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ip= oib iw_cm ib_cm mlx5_core ib_core CPU: 6 UID: 2733 PID: 5812 Comm: ibv_rc_pingpong Tainted: G W = 6.19.0+ #129 PREEMPT Tainted: [W]=3DWARN Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21= b5a4aeb02-prebuilt.qemu.org 04/01/2014 RIP: 0010:add_dma_entry+0x1be/0x280 Code: 8b 7b 10 48 85 ff 0f 84 c3 00 00 00 48 8b 6f 50 48 85 ed 75 03 48 8b= 2f e8 ff 8e 6a 00 48 89 c6 48 8d 3d 55 ef 2d 01 48 89 ea <67> 48 0f b9 3a = 48 85 db 74 1a 48 c7 c7 b0 00 2b 82 e8 9c 25 fd ff RSP: 0018:ff11000138717978 EFLAGS: 00010286 RAX: ffffffffa02d7831 RBX: ff1100010246de00 RCX: 0000000000000000 RDX: ff110001036fac30 RSI: ffffffffa02d7831 RDI: ffffffff82678650 RBP: ff110001036fac30 R08: ff11000110dcb4a0 R09: ff11000110dcb478 R10: 0000000000000000 R11: ffffffff824b30a8 R12: 0000000000000000 R13: 00000000ffffffef R14: 0000000000000202 R15: ff1100010246de00 FS: 00007f59b411c740(0000) GS:ff110008dcc99000(0000) knlGS:00000000000000= 00 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ffe538f7000 CR3: 000000010e066005 CR4: 0000000000373eb0 Call Trace: debug_dma_map_sg+0x1b4/0x390 __dma_map_sg_attrs+0x6d/0x1a0 dma_map_sgtable+0x19/0x30 ib_umem_get+0x254/0x380 [ib_uverbs] mlx5_ib_reg_user_mr+0x68/0x2a0 [mlx5_ib] ib_uverbs_reg_mr+0x17f/0x2a0 [ib_uverbs] ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xc2/0x130 [ib_uverbs] ib_uverbs_cmd_verbs+0xa0b/0xae0 [ib_uverbs] ? ib_uverbs_handler_UVERBS_METHOD_QUERY_PORT_SPEED+0xe0/0xe0 [ib_uverbs] ? mmap_region+0x7a/0xb0 ? do_mmap+0x3b8/0x5c0 ib_uverbs_ioctl+0xa7/0x110 [ib_uverbs] __x64_sys_ioctl+0x14f/0x8b0 ? ksys_mmap_pgoff+0xc5/0x190 do_syscall_64+0x8c/0xbf0 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x7f59b430aeed Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00= 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 = ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 RSP: 002b:00007ffe538f9430 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 00007ffe538f94c0 RCX: 00007f59b430aeed RDX: 00007ffe538f94e0 RSI: 00000000c0181b01 RDI: 0000000000000003 RBP: 00007ffe538f9480 R08: 0000000000000028 R09: 00007ffe538f9684 R10: 0000000000000001 R11: 0000000000000246 R12: 00007ffe538f9684 R13: 000000000000000c R14: 000000002b28d170 R15: 000000000000000c ---[ end trace 0000000000000000 ]--- Signed-off-by: Leon Romanovsky Reviewed-by: Jason Gunthorpe --- drivers/infiniband/core/umem.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index cff4fcca2c345..edc34c69f0f23 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -55,7 +55,8 @@ static void __ib_umem_release(struct ib_device *dev, stru= ct ib_umem *umem, int d =20 if (dirty) ib_dma_unmap_sgtable_attrs(dev, &umem->sgt_append.sgt, - DMA_BIDIRECTIONAL, 0); + DMA_BIDIRECTIONAL, + DMA_ATTR_REQUIRE_COHERENT); =20 for_each_sgtable_sg(&umem->sgt_append.sgt, sg, i) { unpin_user_page_range_dirty_lock(sg_page(sg), @@ -169,7 +170,7 @@ struct ib_umem *ib_umem_get(struct ib_device *device, u= nsigned long addr, unsigned long lock_limit; unsigned long new_pinned; unsigned long cur_base; - unsigned long dma_attr =3D 0; + unsigned long dma_attr =3D DMA_ATTR_REQUIRE_COHERENT; struct mm_struct *mm; unsigned long npages; int pinned, ret; --=20 2.53.0 From nobody Tue Apr 7 19:54:48 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35DCA2E62C4; Wed, 11 Mar 2026 19:09:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256164; cv=none; b=slSUUojRqrlLaT5GkvtfCTeBJJeuRatTS6JeDqRQ0p8BavW4DyBV56RfvUkVh/hFG/CrWmBoQoB3lwUnBOXHFuMIIZWri4j2ltFcn9+vsF13aZPFEHbxPvKpr1w+fHalDiIgNRd0EdLLmhFV+Qzr7V0Dm5JpgTKDWKu/K24R3Fo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773256164; c=relaxed/simple; bh=0UoS5bXBagSWSjZg7zeW+3A+vnJkjeQqWbOv9i9THFE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UCDdqz9wHcL8f4P9ZC+8kAHacF/zwC1SfsS6Dju6Nk+YkMtflGtbzM4TsrjPbjVA/p8+Kr7Cez67ALGKCulnyzINsjaIfftK7DmLYo0rRFswz5+jrwcVi/79FBGq9Zd8VdeIwF69wteaMErMM53VVen+HB0aIsZNWGOS+6I12cE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LbG1uNUg; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LbG1uNUg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 11621C4CEF7; Wed, 11 Mar 2026 19:09:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1773256163; bh=0UoS5bXBagSWSjZg7zeW+3A+vnJkjeQqWbOv9i9THFE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LbG1uNUgq/sPRE98pW+ZT3GTFULJ3gWkdpAIhlzPdLfdX5pxbYb4ITlF1lhbKP6iq MQghWEVT49xo+z6Jd1VwWinJTQF25MNokpn8ET2N3EwR8syaKhSSHoWafZm52xyvQM 0KkhwoP8HpVYEKOfw6TefQuqKlcbeGxQQdMP7j/xhwru4WpEu1Rq/0kN0tmpmcvwCU K2zLd56Qbc+IV0Qp0sp9JYawQz8u83W38bdwA8Pab+Bwk2cIXDM+qCJB+/qWhFf4sT 3aEVAVELkWJ8vkKSvt+0Kot1TRelXPsq7crc7VKcS9VDPgF8WwzsW+jpN/wXdaYTuh MNcha2q9yYS6w== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Joerg Roedel , Will Deacon , Andrew Morton Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v2 8/8] mm/hmm: Indicate that HMM requires DMA coherency Date: Wed, 11 Mar 2026 21:08:51 +0200 Message-ID: <20260311-dma-debug-overlap-v2-8-e00bc2ca346d@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> References: <20260311-dma-debug-overlap-v2-0-e00bc2ca346d@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky HMM mirroring can work on coherent systems without SWIOTLB path only. Until introduction of DMA_ATTR_REQUIRE_COHERENT, there was no reliable way to indicate that and various approximation was done: int hmm_dma_map_alloc(struct device *dev, struct hmm_dma_map *map, size_t nr_entries, size_t dma_entry_size) { <...> /* * The HMM API violates our normal DMA buffer ownership rules and c= an't * transfer buffer ownership. The dma_addressing_limited() check i= s a * best approximation to ensure no swiotlb buffering happens. */ dma_need_sync =3D !dev->dma_skip_sync; if (dma_need_sync || dma_addressing_limited(dev)) return -EOPNOTSUPP; So let's mark mapped buffers with DMA_ATTR_REQUIRE_COHERENT attribute to prevent DMA debugging warnings for cache overlapped entries. Signed-off-by: Leon Romanovsky Reviewed-by: Jason Gunthorpe --- mm/hmm.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hmm.c b/mm/hmm.c index f6c4ddff4bd61..5955f2f0c83db 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -778,7 +778,7 @@ dma_addr_t hmm_dma_map_pfn(struct device *dev, struct h= mm_dma_map *map, struct page *page =3D hmm_pfn_to_page(pfns[idx]); phys_addr_t paddr =3D hmm_pfn_to_phys(pfns[idx]); size_t offset =3D idx * map->dma_entry_size; - unsigned long attrs =3D 0; + unsigned long attrs =3D DMA_ATTR_REQUIRE_COHERENT; dma_addr_t dma_addr; int ret; =20 @@ -871,7 +871,7 @@ bool hmm_dma_unmap_pfn(struct device *dev, struct hmm_d= ma_map *map, size_t idx) struct dma_iova_state *state =3D &map->state; dma_addr_t *dma_addrs =3D map->dma_list; unsigned long *pfns =3D map->pfn_list; - unsigned long attrs =3D 0; + unsigned long attrs =3D DMA_ATTR_REQUIRE_COHERENT; =20 if ((pfns[idx] & valid_dma) !=3D valid_dma) return false; --=20 2.53.0