From nobody Sat Feb 7 17:55:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B76F40BE0; Wed, 21 Feb 2024 11:35:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515315; cv=none; b=gJQCfjF4pKiHnmfdEz2/PNCFOrFzlEpfN3gnVar5oVrGfQfZYvTVCrR8df5DYXfjEEuNP1dNWou68UfzVKb3AOCDUrahKsqVLkpMc13EpBi68Vuof+STWHRDVlARkQKIASjKydeXL+u5Mg9KgY+g3D53seejN3650iMfqq7G4WE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515315; c=relaxed/simple; bh=DZkEV6s5aP5quEZxpzFqOQnvutbKA7WWR4M+rQsKFsY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=V/nSX8uw+Ot/EsB4zC1lwX4bY1Cx+rWLhHxf7kEaqJeMzMV75MBHbP7sI9Da1lnhFfYGF7fZLi0kOGv2x0uh4k0pjUNUjgh2vsgug3q1mIY3wNqV5ivJMMEFG1Ll6kyqHOL0KW+BmjZs9z+4m/KVL7WyMz2AbKX+MMl7n5E9QEU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Hmlv1oAy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Hmlv1oAy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 79FD2C43390; Wed, 21 Feb 2024 11:35:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708515314; bh=DZkEV6s5aP5quEZxpzFqOQnvutbKA7WWR4M+rQsKFsY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Hmlv1oAyagKNMDZ7lQk1uRkrQuLUDueXty1RTJlXzC5X4FVDcTBCLw2r3XqB6z144 3MfYBcfu/VqxGA8Gatg71oS8BkXr+V1ZMHns7BL5n+hw8iMSqAxMT7NRdS38XEEIV5 TcKeEA4Wb/k3HQt6p2Sm5kzJ9FKJXi59S4rCBAY5TqELNRVlA5QXKR14Qy4tABzMSR XWyo1HmZI3t096eQkKbZ4AHTQfdyKCu8CpDgSiBPvYp6jaY3kT+7X9Me1yirdPPg0z fbER28UFGtG/4Hvb4M/EaoZxmhtBe5NbHvdf1J4+MwduXALUqYyk+OLWR3EAOj4n9C C4/MiqYEkpc3Q== From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , iommu@lists.linux.dev, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Petr Tesarik , Dexuan Cui , Nicolin Chen , Michael Kelley Subject: [PATCH v4 1/5] swiotlb: Fix double-allocation of slots due to broken alignment handling Date: Wed, 21 Feb 2024 11:35:00 +0000 Message-Id: <20240221113504.7161-2-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240221113504.7161-1-will@kernel.org> References: <20240221113504.7161-1-will@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Commit bbb73a103fbb ("swiotlb: fix a braino in the alignment check fix"), which was a fix for commit 0eee5ae10256 ("swiotlb: fix slot alignment checks"), causes a functional regression with vsock in a virtual machine using bouncing via a restricted DMA SWIOTLB pool. When virtio allocates the virtqueues for the vsock device using dma_alloc_coherent(), the SWIOTLB search can return page-unaligned allocations if 'area->index' was left unaligned by a previous allocation from the buffer: # Final address in brackets is the SWIOTLB address returned to the caller | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_ma= sk 0x800 stride 0x2: got slot 1645-1649/7168 (0x98326800) | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_ma= sk 0x800 stride 0x2: got slot 1649-1653/7168 (0x98328800) | virtio-pci 0000:00:07.0: orig_addr 0x0 alloc_size 0x2000, iotlb_align_ma= sk 0x800 stride 0x2: got slot 1653-1657/7168 (0x9832a800) This ends badly (typically buffer corruption and/or a hang) because swiotlb_alloc() is expecting a page-aligned allocation and so blindly returns a pointer to the 'struct page' corresponding to the allocation, therefore double-allocating the first half (2KiB slot) of the 4KiB page. Fix the problem by treating the allocation alignment separately to any additional alignment requirements from the device, using the maximum of the two as the stride to search the buffer slots and taking care to ensure a minimum of page-alignment for buffers larger than a page. Fixes: bbb73a103fbb ("swiotlb: fix a braino in the alignment check fix") Fixes: 0eee5ae10256 ("swiotlb: fix slot alignment checks") Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: Dexuan Cui Reviewed-by: Petr Tesarik Signed-off-by: Will Deacon Tested-by: Nicolin Chen --- kernel/dma/swiotlb.c | 28 +++++++++++++++------------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index b079a9a8e087..2ec2cc81f1a2 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -982,7 +982,7 @@ static int swiotlb_search_pool_area(struct device *dev,= struct io_tlb_pool *pool phys_to_dma_unencrypted(dev, pool->start) & boundary_mask; unsigned long max_slots =3D get_max_slots(boundary_mask); unsigned int iotlb_align_mask =3D - dma_get_min_align_mask(dev) | alloc_align_mask; + dma_get_min_align_mask(dev) & ~(IO_TLB_SIZE - 1); unsigned int nslots =3D nr_slots(alloc_size), stride; unsigned int offset =3D swiotlb_align_offset(dev, orig_addr); unsigned int index, slots_checked, count =3D 0, i; @@ -993,19 +993,18 @@ static int swiotlb_search_pool_area(struct device *de= v, struct io_tlb_pool *pool BUG_ON(!nslots); BUG_ON(area_index >=3D pool->nareas); =20 + /* + * For mappings with an alignment requirement don't bother looping to + * unaligned slots once we found an aligned one. + */ + stride =3D get_max_slots(max(alloc_align_mask, iotlb_align_mask)); + /* * For allocations of PAGE_SIZE or larger only look for page aligned * allocations. */ if (alloc_size >=3D PAGE_SIZE) - iotlb_align_mask |=3D ~PAGE_MASK; - iotlb_align_mask &=3D ~(IO_TLB_SIZE - 1); - - /* - * For mappings with an alignment requirement don't bother looping to - * unaligned slots once we found an aligned one. - */ - stride =3D (iotlb_align_mask >> IO_TLB_SHIFT) + 1; + stride =3D umax(stride, PAGE_SHIFT - IO_TLB_SHIFT + 1); =20 spin_lock_irqsave(&area->lock, flags); if (unlikely(nslots > pool->area_nslabs - area->used)) @@ -1015,11 +1014,14 @@ static int swiotlb_search_pool_area(struct device *= dev, struct io_tlb_pool *pool index =3D area->index; =20 for (slots_checked =3D 0; slots_checked < pool->area_nslabs; ) { - slot_index =3D slot_base + index; + phys_addr_t tlb_addr; =20 - if (orig_addr && - (slot_addr(tbl_dma_addr, slot_index) & - iotlb_align_mask) !=3D (orig_addr & iotlb_align_mask)) { + slot_index =3D slot_base + index; + tlb_addr =3D slot_addr(tbl_dma_addr, slot_index); + + if ((tlb_addr & alloc_align_mask) || + (orig_addr && (tlb_addr & iotlb_align_mask) !=3D + (orig_addr & iotlb_align_mask))) { index =3D wrap_area_index(pool, index + 1); slots_checked++; continue; --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 17:55:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B35545BE3; Wed, 21 Feb 2024 11:35:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515317; cv=none; b=OeJAV+jVStidBVpIlb55HhQz0fzUQWwwQu34fJfQMbv5xWYi/wbknlMh/HnVs5Dg57OHb6YrNvPV9c94Ga/GPmafP4wiUm4Z+5SAHjsrphgLUp4dqZG/YRuDpXoMjssSGt0hQ+7qdVyTORuZUHo44+hfRfiwo0ZpA5rDSrk2Xk8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515317; c=relaxed/simple; bh=E1k4A1Ppx138Pm5Tuk5+a08g4wuk3s7KWHOxJ3SgNxE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ps/3JgfCbFOqxtXgcRB/rNH/Q4hZ5GTrAuadPAUIw8Vl+0o04dNlPOFYZnj4Pa6eiV2VXt70wwUQjaD7PwKaKjwJi47UO+TQY3RwL9nJ5K/Dy0Fi+FwVHZhUhZ7HfJryBh/FmG0x6ljhiG2MDpXUk3w2CVncfawp87EEp/AvQlk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=j7lVKM9m; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="j7lVKM9m" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 15FF9C433F1; Wed, 21 Feb 2024 11:35:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708515317; bh=E1k4A1Ppx138Pm5Tuk5+a08g4wuk3s7KWHOxJ3SgNxE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j7lVKM9mW0eOPd7Z7L8Sci+7dRRS5SYzVi7djmh1te6RU4EO7uXYko2hb05EiHxRl WMHYWNCE5ctRJpXh1aHPoxbOufpcrWemNyMEH+zcvUhacqARTYtSB+4F0bwKDONsW8 xaQlRge7TKqehUCMJ6d1qFxh3kFYRB3oHsFPJLO5ytfyiUvuQqTrnAJI7U/rS8whqR UtWzPkur31+3Xu2I6RTYkbNW7K6V1TMC2nbPEPJSxJxfi3j2bvqUzZf0A069pvLSbv 07Q/cCavfmcNgMzEh8e1AtA2fISLjKqkXqMp5uQ4iQ+a0P1ksbGFTRFRRNOUXvfi32 EpxqduiXZCBdQ== From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , iommu@lists.linux.dev, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Petr Tesarik , Dexuan Cui , Nicolin Chen , Michael Kelley Subject: [PATCH v4 2/5] swiotlb: Enforce page alignment in swiotlb_alloc() Date: Wed, 21 Feb 2024 11:35:01 +0000 Message-Id: <20240221113504.7161-3-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240221113504.7161-1-will@kernel.org> References: <20240221113504.7161-1-will@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When allocating pages from a restricted DMA pool in swiotlb_alloc(), the buffer address is blindly converted to a 'struct page *' that is returned to the caller. In the unlikely event of an allocation bug, page-unaligned addresses are not detected and slots can silently be double-allocated. Add a simple check of the buffer alignment in swiotlb_alloc() to make debugging a little easier if something has gone wonky. Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: Dexuan Cui Reviewed-by: Petr Tesarik Signed-off-by: Will Deacon Tested-by: Nicolin Chen --- kernel/dma/swiotlb.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 2ec2cc81f1a2..ab7fbb40bc55 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -1643,6 +1643,12 @@ struct page *swiotlb_alloc(struct device *dev, size_= t size) return NULL; =20 tlb_addr =3D slot_addr(pool->start, index); + if (unlikely(!PAGE_ALIGNED(tlb_addr))) { + dev_WARN_ONCE(dev, 1, "Cannot allocate pages from non page-aligned swiot= lb addr 0x%pa.\n", + &tlb_addr); + swiotlb_release_slots(dev, tlb_addr); + return NULL; + } =20 return pfn_to_page(PFN_DOWN(tlb_addr)); } --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 17:55:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25E5546436; Wed, 21 Feb 2024 11:35:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515320; cv=none; b=gzpaFFJsbA0iJQnL2sK8AnVaFDDl7SpoXwjEB3j8eZprJJB2Lv2g0rS1A0Y3JahHerdeDZAKn3WKCPQugCXNFkvkTYUsPpFbDnPiXd8qiss+onToZA17xSjZGUJNvNzdt862W0Xmmk9PvQqO4IOsUGMzcH8g1iusoYA9G99D9Yk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515320; c=relaxed/simple; bh=VIfdB40mPKeGpRjLdseuZPhrPH2I9FgxnF4NhRToiQg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gOrboZjzz0gg7EWqZhKzb/6LGZyuxzhNS4xyuJM31bMC89Wg1LglmzN6hQvU96Ki+Mj+1HWbHXbWttqI6g0JzME8drTsRrh4ZvljrC2c0Hr3it4zkeFx9Y4I85/DVorOWfFydfcw8UNfx9HRKF+0y1OCJIRcx0IIBz+udw+vqFI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=t0sBwdZ4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="t0sBwdZ4" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A5F70C43390; Wed, 21 Feb 2024 11:35:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708515319; bh=VIfdB40mPKeGpRjLdseuZPhrPH2I9FgxnF4NhRToiQg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t0sBwdZ43tBOm8bQ6pbQMpxjNYBZGxcdwUAVZR9mCB1h2pGI/Zarvg+PUGCEocdC8 5Me8Id80ygsdrJQqtOloQh9dbvaMLdvaWzsNgs9InPJQGSB+QJKslKsm2eVvRcvWIn TdAERFKSxNAWbd7MnHnhdHNXNIlW3uFh08nXz6peJ2UyJRYOs9WGYNM88NcuCcETbD KPOzgm4u8M0SdtFm9PiD/BHFlTv5ZjyxDV6FCnNs4YKub1cTJjj4RYD/GEbzuUnvB0 UnRpBQBtVgi4cve1lBQFy6AjtTQU1V5nHYu3IdrWhru/iIoUCA202dCBQjeA0cTUKA B9L5v0vySUmEw== From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , iommu@lists.linux.dev, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Petr Tesarik , Dexuan Cui , Nicolin Chen , Michael Kelley Subject: [PATCH v4 3/5] swiotlb: Honour dma_alloc_coherent() alignment in swiotlb_alloc() Date: Wed, 21 Feb 2024 11:35:02 +0000 Message-Id: <20240221113504.7161-4-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240221113504.7161-1-will@kernel.org> References: <20240221113504.7161-1-will@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" core-api/dma-api-howto.rst states the following properties of dma_alloc_coherent(): | The CPU virtual address and the DMA address are both guaranteed to | be aligned to the smallest PAGE_SIZE order which is greater than or | equal to the requested size. However, swiotlb_alloc() passes zero for the 'alloc_align_mask' parameter of swiotlb_find_slots() and so this property is not upheld. Instead, allocations larger than a page are aligned to PAGE_SIZE, Calculate the mask corresponding to the page order suitable for holding the allocation and pass that to swiotlb_find_slots(). Cc: Christoph Hellwig Cc: Marek Szyprowski Cc: Robin Murphy Cc: Dexuan Cui Fixes: e81e99bacc9f ("swiotlb: Support aligned swiotlb buffers") Reviewed-by: Petr Tesarik Signed-off-by: Will Deacon Tested-by: Nicolin Chen --- kernel/dma/swiotlb.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index ab7fbb40bc55..c20324fba814 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -1633,12 +1633,14 @@ struct page *swiotlb_alloc(struct device *dev, size= _t size) struct io_tlb_mem *mem =3D dev->dma_io_tlb_mem; struct io_tlb_pool *pool; phys_addr_t tlb_addr; + unsigned int align; int index; =20 if (!mem) return NULL; =20 - index =3D swiotlb_find_slots(dev, 0, size, 0, &pool); + align =3D (1 << (get_order(size) + PAGE_SHIFT)) - 1; + index =3D swiotlb_find_slots(dev, 0, size, align, &pool); if (index =3D=3D -1) return NULL; =20 --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 17:55:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B546B46557; Wed, 21 Feb 2024 11:35:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515322; cv=none; b=W9dw+hFt3/Jl3f7xW4sd3e1fliuwmR/rj8CDoDIbfDzyz0IWtBDl9SgS1kQmIDU9aDaaq9gtoWbGxqpbyYwg6xb+O4GPqH6GBRWfCR6CSPfCvfmB3Y2yIcFWzbON7XTmIHgtusVEcDH/HuR2RChhM8DT6y7c059SOhMuUaCfS90= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515322; c=relaxed/simple; bh=Ay0zHMle2EoSZH9+Ztkdc4xcUkKiiiqwiAyt88r4Wt0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GzFJy/62yG+mlm2YGSvFmrAYY5xnWnJ9Pw/pOELSvHtzaJZ/4nQinMmdhYpnX9Vz737X5rDsBzfGo7s1uhTnlV52tF/k4sU5y5CUjVnZfu2W08w40Iq4tTlVZ7PCelyvukh7lKNkZSZMonCNtukU0ea/6h0SAUz2i5dRMDNi1TA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=vFrIla50; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="vFrIla50" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3E16AC433F1; Wed, 21 Feb 2024 11:35:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708515322; bh=Ay0zHMle2EoSZH9+Ztkdc4xcUkKiiiqwiAyt88r4Wt0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vFrIla50aCwBLWXr37fkHe/ZdJAIDpLFrGPnrHHnqAbixhRkmJEhmnm9ru0HJP3Iz /iuQ2EX+2nvdc9EOIepmqns7xzBxJUCpX/YFRbJJz+CriBCFWvJXbFpUW/KrYOvViA qBUIDkfMTBlLg0uJlJDRV7b6S/HUN+//UlHf3/dTffDjeBSJAJ8IjUdVhbSxoZEk/4 eroLsFE3q7kdqrLECimCq28uGCnX+kFWO7JGO40hKUiE4KDr2aVpQdfrJB8cgTQFW2 U/6GzeYaAmt4vTEfVeb9+weSR+EUxJ2+0YJftJNoDw3cz22u/q42Ppq0DjdimBqFa5 PQrS3/qLvLDCg== From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , iommu@lists.linux.dev, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Petr Tesarik , Dexuan Cui , Nicolin Chen , Michael Kelley Subject: [PATCH v4 4/5] swiotlb: Fix alignment checks when both allocation and DMA masks are present Date: Wed, 21 Feb 2024 11:35:03 +0000 Message-Id: <20240221113504.7161-5-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240221113504.7161-1-will@kernel.org> References: <20240221113504.7161-1-will@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Nicolin reports that swiotlb buffer allocations fail for an NVME device behind an IOMMU using 64KiB pages. This is because we end up with a minimum allocation alignment of 64KiB (for the IOMMU to map the buffer safely) but a minimum DMA alignment mask corresponding to a 4KiB NVME page (i.e. preserving the 4KiB page offset from the original allocation). If the original address is not 4KiB-aligned, the allocation will fail because swiotlb_search_pool_area() erroneously compares these unmasked bits with the 64KiB-aligned candidate allocation. Tweak swiotlb_search_pool_area() so that the DMA alignment mask is reduced based on the required alignment of the allocation. Fixes: 82612d66d51d ("iommu: Allow the dma-iommu api to use bounce buffers") Reported-by: Nicolin Chen Link: https://lore.kernel.org/r/cover.1707851466.git.nicolinc@nvidia.com Signed-off-by: Will Deacon Tested-by: Nicolin Chen --- kernel/dma/swiotlb.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index c20324fba814..c381a7ed718f 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -981,8 +981,7 @@ static int swiotlb_search_pool_area(struct device *dev,= struct io_tlb_pool *pool dma_addr_t tbl_dma_addr =3D phys_to_dma_unencrypted(dev, pool->start) & boundary_mask; unsigned long max_slots =3D get_max_slots(boundary_mask); - unsigned int iotlb_align_mask =3D - dma_get_min_align_mask(dev) & ~(IO_TLB_SIZE - 1); + unsigned int iotlb_align_mask =3D dma_get_min_align_mask(dev); unsigned int nslots =3D nr_slots(alloc_size), stride; unsigned int offset =3D swiotlb_align_offset(dev, orig_addr); unsigned int index, slots_checked, count =3D 0, i; @@ -993,6 +992,14 @@ static int swiotlb_search_pool_area(struct device *dev= , struct io_tlb_pool *pool BUG_ON(!nslots); BUG_ON(area_index >=3D pool->nareas); =20 + /* + * Ensure that the allocation is at least slot-aligned and update + * 'iotlb_align_mask' to ignore bits that will be preserved when + * offsetting into the allocation. + */ + alloc_align_mask |=3D (IO_TLB_SIZE - 1); + iotlb_align_mask &=3D ~alloc_align_mask; + /* * For mappings with an alignment requirement don't bother looping to * unaligned slots once we found an aligned one. --=20 2.44.0.rc0.258.g7320e95886-goog From nobody Sat Feb 7 17:55:31 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14D9847A48; Wed, 21 Feb 2024 11:35:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515325; cv=none; b=BhpWoZPVctLFOp2RkPk6689vDWHdAUv85eNmTLmV8jjXnnjDsBMfZmnqYIF8QLCoJxvUU0OqiJeFCJpI/UKIe3D5lFS+FFOvSiVZ+BqxHeaWtzgyqqY8AcIlhHRI669xQSw/8Fbma7jPW6uYSWJFsGCJWa0VuajKMb1i1r9XQ+o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515325; c=relaxed/simple; bh=tme5Bcu1kmkgKcTRsUL+dtCitanw38/NkbYErn3Nj2U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=N3ChHse7lMGsi8dJw5sxQg5q8UbV+qg6neDCi045WNpLp9dpjaatINnNHZ6t97nAkItTsoDF5kQBJIVAgfWAOClpdf3FwoEgGVpphZbzELff5jrDSjHlez00IjslTkrMHxkmlkapmRv6FeBmXkIbYkaysKPWeSIkZi7fS6EFeU0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=qOo6yhNv; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="qOo6yhNv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C9EFFC433C7; Wed, 21 Feb 2024 11:35:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1708515324; bh=tme5Bcu1kmkgKcTRsUL+dtCitanw38/NkbYErn3Nj2U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qOo6yhNvqRON3GEiSyeQg497wNAQR9efRB1LG8MEA41J47ZbVcgKLSp6JFyA6ffM1 vOOsgkvNTUvaXPDO7FwKdJx94Gr+BVB4zl2MgBJ7o9qPJc+L59hNzgd3so+zWO1MCZ xa0ukx54zHqqH/+vD9Ls00XlK86l3el5kbcrP/whwe59Ijr62yn60LMblUoVM7NXT8 25Ds0o/c6tg3oZX1q5taHI+zubZwTNZgblLennFqm3KCsgMUum/nSNhHHkbvM5TKRH ps2wHDQTHkOa/Y5TjOUVST781XSfIjk3Z2Eos6k8Y/IT+Dbwww9wuxSn/5ChSM/rna 0Xw3eEAvDEyPg== From: Will Deacon To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, Will Deacon , iommu@lists.linux.dev, Christoph Hellwig , Marek Szyprowski , Robin Murphy , Petr Tesarik , Dexuan Cui , Nicolin Chen , Michael Kelley Subject: [PATCH v4 5/5] iommu/dma: Force swiotlb_max_mapping_size on an untrusted device Date: Wed, 21 Feb 2024 11:35:04 +0000 Message-Id: <20240221113504.7161-6-will@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20240221113504.7161-1-will@kernel.org> References: <20240221113504.7161-1-will@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Nicolin Chen The swiotlb does not support a mapping size > swiotlb_max_mapping_size(). On the other hand, with a 64KB PAGE_SIZE configuration, it's observed that an NVME device can map a size between 300KB~512KB, which certainly failed the swiotlb mappings, though the default pool of swiotlb has many slots: systemd[1]: Started Journal Service. =3D> nvme 0000:00:01.0: swiotlb buffer is full (sz: 327680 bytes), total 3= 2768 (slots), used 32 (slots) note: journal-offline[392] exited with irqs disabled note: journal-offline[392] exited with preempt_count 1 Call trace: [ 3.099918] swiotlb_tbl_map_single+0x214/0x240 [ 3.099921] iommu_dma_map_page+0x218/0x328 [ 3.099928] dma_map_page_attrs+0x2e8/0x3a0 [ 3.101985] nvme_prep_rq.part.0+0x408/0x878 [nvme] [ 3.102308] nvme_queue_rqs+0xc0/0x300 [nvme] [ 3.102313] blk_mq_flush_plug_list.part.0+0x57c/0x600 [ 3.102321] blk_add_rq_to_plug+0x180/0x2a0 [ 3.102323] blk_mq_submit_bio+0x4c8/0x6b8 [ 3.103463] __submit_bio+0x44/0x220 [ 3.103468] submit_bio_noacct_nocheck+0x2b8/0x360 [ 3.103470] submit_bio_noacct+0x180/0x6c8 [ 3.103471] submit_bio+0x34/0x130 [ 3.103473] ext4_bio_write_folio+0x5a4/0x8c8 [ 3.104766] mpage_submit_folio+0xa0/0x100 [ 3.104769] mpage_map_and_submit_buffers+0x1a4/0x400 [ 3.104771] ext4_do_writepages+0x6a0/0xd78 [ 3.105615] ext4_writepages+0x80/0x118 [ 3.105616] do_writepages+0x90/0x1e8 [ 3.105619] filemap_fdatawrite_wbc+0x94/0xe0 [ 3.105622] __filemap_fdatawrite_range+0x68/0xb8 [ 3.106656] file_write_and_wait_range+0x84/0x120 [ 3.106658] ext4_sync_file+0x7c/0x4c0 [ 3.106660] vfs_fsync_range+0x3c/0xa8 [ 3.106663] do_fsync+0x44/0xc0 Since untrusted devices might go down the swiotlb pathway with dma-iommu, these devices should not map a size larger than swiotlb_max_mapping_size. To fix this bug, add iommu_dma_max_mapping_size() for untrusted devices to take into account swiotlb_max_mapping_size() v.s. iova_rcache_range() from the iommu_dma_opt_mapping_size(). Fixes: 82612d66d51d ("iommu: Allow the dma-iommu api to use bounce buffers") Signed-off-by: Nicolin Chen Link: https://lore.kernel.org/r/ee51a3a5c32cf885b18f6416171802669f4a718a.17= 07851466.git.nicolinc@nvidia.com Signed-off-by: Will Deacon Acked-by: Robin Murphy Tested-by: Nicolin Chen --- drivers/iommu/dma-iommu.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 50ccc4f1ef81..7d1a20da6d94 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1706,6 +1706,13 @@ static size_t iommu_dma_opt_mapping_size(void) return iova_rcache_range(); } =20 +static size_t iommu_dma_max_mapping_size(struct device *dev) +{ + if (is_swiotlb_active(dev) && dev_is_untrusted(dev)) + return swiotlb_max_mapping_size(dev); + return SIZE_MAX; +} + static const struct dma_map_ops iommu_dma_ops =3D { .flags =3D DMA_F_PCI_P2PDMA_SUPPORTED, .alloc =3D iommu_dma_alloc, @@ -1728,6 +1735,7 @@ static const struct dma_map_ops iommu_dma_ops =3D { .unmap_resource =3D iommu_dma_unmap_resource, .get_merge_boundary =3D iommu_dma_get_merge_boundary, .opt_mapping_size =3D iommu_dma_opt_mapping_size, + .max_mapping_size =3D iommu_dma_max_mapping_size, }; =20 /* --=20 2.44.0.rc0.258.g7320e95886-goog