From nobody Thu Apr 9 14:57:52 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA7DA3290BB; Sat, 7 Mar 2026 16:50:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772902210; cv=none; b=HGSxrZgwb+NZu1pQg43fnCQjWTknJXc6PiAAOWodIYGtqgx3/kIO8mzvZV2/+OlWSPXBFmsQ5mH7QzT8g4o7g+vocQk+GVsR6gV86YQIeCAI9t+vSvdWhddMPky54FIlJ2qZgwhk+3/9uE0Thgirt8qbSrdQfoNQZQl0ZVk04/A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772902210; c=relaxed/simple; bh=Rb7pGPHPDn2aZQQvadJE9PAK4yDKeRrpgUBQyVOU34k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=EyCJNE4MwWkyimIFT0GQcFyIp+p+VMM77sn8JuyY+rauJ0wRVXOEsjT3BPLJfslYlC6XcJrIJr6s1ChzQO/0dlip76MH248bigZfMEDsd5FKpi5UsSr94NnnWJwmHBeJH68rF9UsT4uDPqIonZ3XgPAUHrwsgvdsGB74TYsgNa8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rLeiBAEc; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rLeiBAEc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1CD0FC19422; Sat, 7 Mar 2026 16:50:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772902209; bh=Rb7pGPHPDn2aZQQvadJE9PAK4yDKeRrpgUBQyVOU34k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rLeiBAEckuUcYuKrCPAL+4ZPzvpyDeeYYFacMAT/MPozrdgUgAABSacrh+WcuO7ea MsdYa8sytC84GBXvu3B63teNEdwPp87OozFpC751Ev6QmWRaMYiNC7D0JieMzHIWXs YdC/WquaTy4ZLl9cV30g7Mw9AYnl0tZugTfY+smnYCCBjwNQcILlt1Oz0k4a2OSHeK Ctj2FyxZFZOpZ/MpjegNYOAQ32FWhIqZujpPNQm3ATdLro0SqyLoaMZ84/duiinLYG 6OUSljus2zjhMFM0XuHJtYzlqe3+WoaemijsYic9JFkFHg0DSsBVGYgEqSCWlBNk68 4idPw40FJITTg== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org Subject: [PATCH 1/3] dma-debug: Allow multiple invocations of overlapping entries Date: Sat, 7 Mar 2026 18:49:55 +0200 Message-ID: <20260307-dma-debug-overlap-v1-1-c034c38872af@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260307-dma-debug-overlap-v1-0-c034c38872af@nvidia.com> References: <20260307-dma-debug-overlap-v1-0-c034c38872af@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky Repeated DMA mappings with DMA_ATTR_CPU_CACHE_CLEAN trigger the following splat. This prevents using the attribute in cases where a DMA region is shared and reused more than seven times. ------------[ cut here ]------------ DMA-API: exceeded 7 overlapping mappings of cacheline 0x000000000438c440 WARNING: kernel/dma/debug.c:467 at add_dma_entry+0x219/0x280, CPU#4: ibv_r= c_pingpong/1644 Modules linked in: xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetli= nk iptable_nat nf_nat xt_addrtype br_netfilter rpcsec_gss_krb5 auth_rpcgss = oid_registry overlay mlx5_fwctl zram zsmalloc mlx5_ib fuse rpcrdma rdma_ucm= ib_uverbs ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ipoib i= w_cm ib_cm mlx5_core ib_core CPU: 4 UID: 2733 PID: 1644 Comm: ibv_rc_pingpong Not tainted 6.19.0+ #129 = PREEMPT Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21= b5a4aeb02-prebuilt.qemu.org 04/01/2014 RIP: 0010:add_dma_entry+0x221/0x280 Code: c0 0f 84 f2 fe ff ff 83 e8 01 89 05 6d 99 11 01 e9 e4 fe ff ff 0f 8e= 1f ff ff ff 48 8d 3d 07 ef 2d 01 be 07 00 00 00 48 89 e2 <67> 48 0f b9 3a = e9 06 ff ff ff 48 c7 c7 98 05 2b 82 c6 05 72 92 28 RSP: 0018:ff1100010e657970 EFLAGS: 00010002 RAX: 0000000000000007 RBX: ff1100010234eb00 RCX: 0000000000000000 RDX: ff1100010e657970 RSI: 0000000000000007 RDI: ffffffff82678660 RBP: 000000000438c440 R08: 0000000000000228 R09: 0000000000000000 R10: 00000000000001be R11: 000000000000089d R12: 0000000000000800 R13: 00000000ffffffef R14: 0000000000000202 R15: ff1100010234eb00 FS: 00007fb15f3f6740(0000) GS:ff110008dcc19000(0000) knlGS:00000000000000= 00 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fb15f32d3a0 CR3: 0000000116f59001 CR4: 0000000000373eb0 Call Trace: debug_dma_map_sg+0x1b4/0x390 __dma_map_sg_attrs+0x6d/0x1a0 dma_map_sgtable+0x19/0x30 ib_umem_get+0x284/0x3b0 [ib_uverbs] mlx5_ib_reg_user_mr+0x68/0x2a0 [mlx5_ib] ib_uverbs_reg_mr+0x17f/0x2a0 [ib_uverbs] ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xc2/0x130 [ib_uverbs] ib_uverbs_cmd_verbs+0xa0b/0xae0 [ib_uverbs] ? ib_uverbs_handler_UVERBS_METHOD_QUERY_PORT_SPEED+0xe0/0xe0 [ib_uverbs] ? mmap_region+0x7a/0xb0 ? do_mmap+0x3b8/0x5c0 ib_uverbs_ioctl+0xa7/0x110 [ib_uverbs] __x64_sys_ioctl+0x14f/0x8b0 ? ksys_mmap_pgoff+0xc5/0x190 do_syscall_64+0x8c/0xbf0 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x7fb15f5e4eed Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00= 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 = ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 RSP: 002b:00007ffe09a5c540 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 00007ffe09a5c5d0 RCX: 00007fb15f5e4eed RDX: 00007ffe09a5c5f0 RSI: 00000000c0181b01 RDI: 0000000000000003 RBP: 00007ffe09a5c590 R08: 0000000000000028 R09: 00007ffe09a5c794 R10: 0000000000000001 R11: 0000000000000246 R12: 00007ffe09a5c794 R13: 000000000000000c R14: 0000000025a49170 R15: 000000000000000c ---[ end trace 0000000000000000 ]--- Fixes: 61868dc55a11 ("dma-mapping: add DMA_ATTR_CPU_CACHE_CLEAN") Signed-off-by: Leon Romanovsky --- kernel/dma/debug.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index 86f87e43438c3..be207be749968 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -453,7 +453,7 @@ static int active_cacheline_set_overlap(phys_addr_t cln= , int overlap) return overlap; } =20 -static void active_cacheline_inc_overlap(phys_addr_t cln) +static void active_cacheline_inc_overlap(phys_addr_t cln, bool is_cache_cl= ean) { int overlap =3D active_cacheline_read_overlap(cln); =20 @@ -462,7 +462,7 @@ static void active_cacheline_inc_overlap(phys_addr_t cl= n) /* If we overflowed the overlap counter then we're potentially * leaking dma-mappings. */ - WARN_ONCE(overlap > ACTIVE_CACHELINE_MAX_OVERLAP, + WARN_ONCE(!is_cache_clean && overlap > ACTIVE_CACHELINE_MAX_OVERLAP, pr_fmt("exceeded %d overlapping mappings of cacheline %pa\n"), ACTIVE_CACHELINE_MAX_OVERLAP, &cln); } @@ -495,7 +495,7 @@ static int active_cacheline_insert(struct dma_debug_ent= ry *entry, if (rc =3D=3D -EEXIST) { struct dma_debug_entry *existing; =20 - active_cacheline_inc_overlap(cln); + active_cacheline_inc_overlap(cln, entry->is_cache_clean); existing =3D radix_tree_lookup(&dma_active_cacheline, cln); /* A lookup failure here after we got -EEXIST is unexpected. */ WARN_ON(!existing); --=20 2.53.0 From nobody Thu Apr 9 14:57:52 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EED3A3126B9; Sat, 7 Mar 2026 16:50:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772902217; cv=none; b=fgpBCxCIFyPw1UY4CcSY9qMz308RgtYrjNh/wTcvBqUFOu0EJbXsQHkFuZHDrkoRuTDz9JF9IYNXlBRozHbqZYSBaaLvnLTg3JUjbwkGOggbilm8jy5dQjdSlde6e/JpeF8ly4v3QL/ThCzdesgZnf/HwLmcmcPxwZDwxl25P2Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772902217; c=relaxed/simple; bh=+np5iOOqktMdnsPjtJMq7wRZ0OVJ8pTuP3tHO2f6YKk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Gf8tbWqCupdgV8fbRSkzQNUJmbfsPVT2NbO2ZVKKd6XEQSZglC1xCfwoneVZVPEynSjiS+3uPJM+q6BHrcsTA79QPxZWmOi0Sg3IQ2pjPa2F7tyytWPSVUQKO5ISaXNiP+gkLamEXLLhT/h8W2h6pEv2mzIHhLWGza49vRxgRZM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VHoxlU8I; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VHoxlU8I" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 36265C2BC87; Sat, 7 Mar 2026 16:50:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772902216; bh=+np5iOOqktMdnsPjtJMq7wRZ0OVJ8pTuP3tHO2f6YKk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VHoxlU8IHwNFA65SC7u7n6bKu5lC9mOMaZXCHE+abZN2Niba6YHRs9isYcsV45tB3 ZyQ6VS0AkW20MqNLYjLmG/7+hMSWUVdezHFRW0SuUwTueH942Zcn6j/TUZ8XmIUYS3 7kkt0PGgeeFNNFxJs7L9CLTkVjNbSf3CF52h1AhmS0ZB3afa2BXJM4D5qU2hn8Mplj AKhNHbDUMbYHu4w/Md11wytq1qksjJoNW6rwAMpVXffWKJAbwA5gLbauKxrz7KVCiM qgj0IN6PKz+EXd06qzgQvUVoda2gJmZ8I10cQ95CYutdM7tZQJq4+qiEJETLbAPzZP Zp9PUsrj4Vfjg== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org Subject: [PATCH 2/3] dma-mapping: Clarify valid conditions for CPU cache line overlap Date: Sat, 7 Mar 2026 18:49:56 +0200 Message-ID: <20260307-dma-debug-overlap-v1-2-c034c38872af@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260307-dma-debug-overlap-v1-0-c034c38872af@nvidia.com> References: <20260307-dma-debug-overlap-v1-0-c034c38872af@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky Rename the DMA_ATTR_CPU_CACHE_CLEAN attribute to reflect that it allows CPU cache overlaps to exist, and document a slightly different but still valid use case involving overlapping CPU cache lines. Signed-off-by: Leon Romanovsky --- Documentation/core-api/dma-attributes.rst | 26 ++++++++++++++++++-------- drivers/virtio/virtio_ring.c | 4 ++-- include/linux/dma-mapping.h | 8 ++++---- kernel/dma/debug.c | 2 +- 4 files changed, 25 insertions(+), 15 deletions(-) diff --git a/Documentation/core-api/dma-attributes.rst b/Documentation/core= -api/dma-attributes.rst index 1d7bfad73b1c7..6b73d92c62721 100644 --- a/Documentation/core-api/dma-attributes.rst +++ b/Documentation/core-api/dma-attributes.rst @@ -149,11 +149,21 @@ For architectures that require cache flushing for DMA= coherence DMA_ATTR_MMIO will not perform any cache flushing. The address provided must never be mapped cacheable into the CPU. =20 -DMA_ATTR_CPU_CACHE_CLEAN ------------------------- - -This attribute indicates the CPU will not dirty any cacheline overlapping = this -DMA_FROM_DEVICE/DMA_BIDIRECTIONAL buffer while it is mapped. This allows -multiple small buffers to safely share a cacheline without risk of data -corruption, suppressing DMA debug warnings about overlapping mappings. -All mappings sharing a cacheline should have this attribute. +DMA_ATTR_CPU_CACHE_OVERLAP +-------------------------- + +This attribute indicates that CPU cache lines may overlap for buffers mapp= ed +with DMA_FROM_DEVICE or DMA_BIDIRECTIONAL. + +Such overlap may occur when callers map multiple small buffers that reside +within the same cache line. In this case, callers must guarantee that the = CPU +will not dirty these cache lines after the mappings are established. When = this +condition is met, multiple buffers can safely share a cache line without r= isking +data corruption. + +Another valid use case is on systems that are CPU-coherent and do not use +SWIOTLB, where the caller can guarantee that no cache maintenance operatio= ns +(such as flushes) will be performed that could overwrite shared cache line= s. + +All mappings that share a cache line must set this attribute to suppress D= MA +debug warnings about overlapping mappings. diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c index 335692d41617a..bf51ae9a39169 100644 --- a/drivers/virtio/virtio_ring.c +++ b/drivers/virtio/virtio_ring.c @@ -2912,7 +2912,7 @@ EXPORT_SYMBOL_GPL(virtqueue_add_inbuf); * @data: the token identifying the buffer. * @gfp: how to do memory allocations (if necessary). * - * Same as virtqueue_add_inbuf but passes DMA_ATTR_CPU_CACHE_CLEAN to indi= cate + * Same as virtqueue_add_inbuf but passes DMA_ATTR_CPU_CACHE_OVERLAP to in= dicate * that the CPU will not dirty any cacheline overlapping this buffer while= it * is available, and to suppress overlapping cacheline warnings in DMA deb= ug * builds. @@ -2928,7 +2928,7 @@ int virtqueue_add_inbuf_cache_clean(struct virtqueue = *vq, gfp_t gfp) { return virtqueue_add(vq, &sg, num, 0, 1, data, NULL, false, gfp, - DMA_ATTR_CPU_CACHE_CLEAN); + DMA_ATTR_CPU_CACHE_OVERLAP); } EXPORT_SYMBOL_GPL(virtqueue_add_inbuf_cache_clean); =20 diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h index 29973baa05816..45efede1a6cce 100644 --- a/include/linux/dma-mapping.h +++ b/include/linux/dma-mapping.h @@ -80,11 +80,11 @@ #define DMA_ATTR_MMIO (1UL << 10) =20 /* - * DMA_ATTR_CPU_CACHE_CLEAN: Indicates the CPU will not dirty any cacheline - * overlapping this buffer while it is mapped for DMA. All mappings sharing - * a cacheline must have this attribute for this to be considered safe. + * DMA_ATTR_CPU_CACHE_OVERLAP: Indicates the CPU cache line can be overlap= ped. + * All mappings sharing a cacheline must have this attribute for this + * to be considered safe. */ -#define DMA_ATTR_CPU_CACHE_CLEAN (1UL << 11) +#define DMA_ATTR_CPU_CACHE_OVERLAP (1UL << 11) =20 /* * A dma_addr_t can hold any valid DMA or bus address for the platform. I= t can diff --git a/kernel/dma/debug.c b/kernel/dma/debug.c index be207be749968..603be342063f1 100644 --- a/kernel/dma/debug.c +++ b/kernel/dma/debug.c @@ -601,7 +601,7 @@ static void add_dma_entry(struct dma_debug_entry *entry= , unsigned long attrs) unsigned long flags; int rc; =20 - entry->is_cache_clean =3D !!(attrs & DMA_ATTR_CPU_CACHE_CLEAN); + entry->is_cache_clean =3D attrs & DMA_ATTR_CPU_CACHE_OVERLAP; =20 bucket =3D get_hash_bucket(entry, &flags); hash_bucket_add(bucket, entry); --=20 2.53.0 From nobody Thu Apr 9 14:57:52 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C7EC329C7F; Sat, 7 Mar 2026 16:50:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772902213; cv=none; b=iWIlADQaHvHiaXQcxPFpL12SQ0M0auNAiQt5qTbZr9HOd5qTnIPHXJcjKliUYtxXFw0fvn83kyaVz6DZv9fRm7VlDROoaQmnhYogK7nsdpa7nUOJf9c1o6vsnFey2KGA3ldO4LsOPugFUTTfFbyj/Je6SoyLFHOJNIfQL1jY4es= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772902213; c=relaxed/simple; bh=1DSnUB7QM/N59SEckdGEQ/tMYUzqKaN/J5qEg7lviFo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=S487Gx+f0CocqXhesLI9GcnOxsjEZtFu387QqHC7wtygR20hR7PulryaMPePdja3scCbg9E9qXa6Tlib16IqSuFHVLC8bfjJx1DgDCFgbWQXpbnqeElhD38H6UcMibhnjVdbxIgt31E3TlIr/rWiGYLXim5Z7pFiuWo5oTPVMwI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YTY/oyEi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YTY/oyEi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id AAB0AC2BC86; Sat, 7 Mar 2026 16:50:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772902213; bh=1DSnUB7QM/N59SEckdGEQ/tMYUzqKaN/J5qEg7lviFo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YTY/oyEiaUE08OFzSSzmJLXht3DQTWMUsa3ziwMfj948nciKAHuk7Ob5U+0CyXDaR nA45hiIY9pebdypvlxk5n1soIZHc3qbnE6JjQqjhKsP34Dux489gy09UAmUB4RYaeE URDVf6Y1vZt175tc1Zo6yBhUNhIu8oK76PthO4vKvU7uLAnBuXCJYB8rMsDllLqMKe 80Q/QMlc359sOjfp/pIuPYFTUhoOT8i3bZ2ckMtIBjuR7TeODfoHqrmI40XqsLanVL 7uM2JRzm3P6OVBVCKDwRVoOORb5bHJtCKFGRAvk4o+DRwMkpAYO5fgvH1jz4FssOak c4/pREKQs/j2w== From: Leon Romanovsky To: Marek Szyprowski , Robin Murphy , "Michael S. Tsirkin" , Petr Tesarik , Jonathan Corbet , Shuah Khan , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Jason Gunthorpe , Leon Romanovsky Cc: iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, virtualization@lists.linux.dev, linux-rdma@vger.kernel.org Subject: [PATCH 3/3] RDMA/umem: Tell DMA debug that cacheline overlap is expected Date: Sat, 7 Mar 2026 18:49:57 +0200 Message-ID: <20260307-dma-debug-overlap-v1-3-c034c38872af@nvidia.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260307-dma-debug-overlap-v1-0-c034c38872af@nvidia.com> References: <20260307-dma-debug-overlap-v1-0-c034c38872af@nvidia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Mailer: b4 0.15-dev-18f8f Content-Transfer-Encoding: quoted-printable From: Leon Romanovsky The RDMA subsystem exposes DMA regions through the verbs interface. A given region can be exported multiple times, which can trigger warnings about cacheline overlaps. In this case the warnings are false positives, because RDMA does not use SWIOTLB and uverbs operate only on CPU=E2=80=91co= herent architectures. infiniband rocep8s0f0: mlx5_ib_reg_user_mr:1592:(pid 5812): start 0x2b28c00= 0, iova 0x2b28c000, length 0x1000, access_flags 0x1 infiniband rocep8s0f0: mlx5_ib_reg_user_mr:1592:(pid 5812): start 0x2b28c00= 1, iova 0x2b28c001, length 0xfff, access_flags 0x1 ------------[ cut here ]------------ DMA-API: mlx5_core 0000:08:00.0: cacheline tracking EEXIST, overlapping ma= ppings aren't supported WARNING: kernel/dma/debug.c:620 at add_dma_entry+0x1bb/0x280, CPU#6: ibv_r= c_pingpong/5812 Modules linked in: veth xt_conntrack xt_MASQUERADE nf_conntrack_netlink nf= netlink iptable_nat nf_nat xt_addrtype br_netfilter rpcsec_gss_krb5 auth_rp= cgss oid_registry overlay mlx5_fwctl zram zsmalloc mlx5_ib fuse rpcrdma rdm= a_ucm ib_uverbs ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ip= oib iw_cm ib_cm mlx5_core ib_core CPU: 6 UID: 2733 PID: 5812 Comm: ibv_rc_pingpong Tainted: G W = 6.19.0+ #129 PREEMPT Tainted: [W]=3DWARN Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21= b5a4aeb02-prebuilt.qemu.org 04/01/2014 RIP: 0010:add_dma_entry+0x1be/0x280 Code: 8b 7b 10 48 85 ff 0f 84 c3 00 00 00 48 8b 6f 50 48 85 ed 75 03 48 8b= 2f e8 ff 8e 6a 00 48 89 c6 48 8d 3d 55 ef 2d 01 48 89 ea <67> 48 0f b9 3a = 48 85 db 74 1a 48 c7 c7 b0 00 2b 82 e8 9c 25 fd ff RSP: 0018:ff11000138717978 EFLAGS: 00010286 RAX: ffffffffa02d7831 RBX: ff1100010246de00 RCX: 0000000000000000 RDX: ff110001036fac30 RSI: ffffffffa02d7831 RDI: ffffffff82678650 RBP: ff110001036fac30 R08: ff11000110dcb4a0 R09: ff11000110dcb478 R10: 0000000000000000 R11: ffffffff824b30a8 R12: 0000000000000000 R13: 00000000ffffffef R14: 0000000000000202 R15: ff1100010246de00 FS: 00007f59b411c740(0000) GS:ff110008dcc99000(0000) knlGS:00000000000000= 00 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007ffe538f7000 CR3: 000000010e066005 CR4: 0000000000373eb0 Call Trace: debug_dma_map_sg+0x1b4/0x390 __dma_map_sg_attrs+0x6d/0x1a0 dma_map_sgtable+0x19/0x30 ib_umem_get+0x254/0x380 [ib_uverbs] mlx5_ib_reg_user_mr+0x68/0x2a0 [mlx5_ib] ib_uverbs_reg_mr+0x17f/0x2a0 [ib_uverbs] ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xc2/0x130 [ib_uverbs] ib_uverbs_cmd_verbs+0xa0b/0xae0 [ib_uverbs] ? ib_uverbs_handler_UVERBS_METHOD_QUERY_PORT_SPEED+0xe0/0xe0 [ib_uverbs] ? mmap_region+0x7a/0xb0 ? do_mmap+0x3b8/0x5c0 ib_uverbs_ioctl+0xa7/0x110 [ib_uverbs] __x64_sys_ioctl+0x14f/0x8b0 ? ksys_mmap_pgoff+0xc5/0x190 do_syscall_64+0x8c/0xbf0 entry_SYSCALL_64_after_hwframe+0x4b/0x53 RIP: 0033:0x7f59b430aeed Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00= 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 = ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 RSP: 002b:00007ffe538f9430 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 00007ffe538f94c0 RCX: 00007f59b430aeed RDX: 00007ffe538f94e0 RSI: 00000000c0181b01 RDI: 0000000000000003 RBP: 00007ffe538f9480 R08: 0000000000000028 R09: 00007ffe538f9684 R10: 0000000000000001 R11: 0000000000000246 R12: 00007ffe538f9684 R13: 000000000000000c R14: 000000002b28d170 R15: 000000000000000c ---[ end trace 0000000000000000 ]--- Signed-off-by: Leon Romanovsky --- drivers/infiniband/core/umem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index cff4fcca2c345..4ae04b6e6927c 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -169,7 +169,7 @@ struct ib_umem *ib_umem_get(struct ib_device *device, u= nsigned long addr, unsigned long lock_limit; unsigned long new_pinned; unsigned long cur_base; - unsigned long dma_attr =3D 0; + unsigned long dma_attr =3D DMA_ATTR_CPU_CACHE_OVERLAP; struct mm_struct *mm; unsigned long npages; int pinned, ret; --=20 2.53.0