From nobody Thu Dec 18 19:27:06 2025 Received: from mail.tkos.co.il (mail.tkos.co.il [84.110.109.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7E99144D1C; Mon, 29 Jul 2024 10:51:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=84.110.109.230 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722250319; cv=none; b=eeE4/cN4E985qUnUUqSuxCaTmG4fJoF7o+dut02V4UmyYQMDI3nwBDjXUHdc/wYvk6Ufj3ibHfzKa40mDUYlz/qhEkdB6cUr7mo9+3ezMaasrdWjPXsfYbgPmjqiUVyQlsBo0Rw2PGGSMB8z8c7aUcorwBEJCLb2SL0GQov6Q2U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722250319; c=relaxed/simple; bh=Za3ndOc802AihmgHa1yG/1ah7Y+qALAilhokvJR5l6o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=getJERRzb4l41ITIhPVWw6AOeiWx+sjDvquCg792hDVIiM/PV+qlOhfJaFD4cwks+Jq/QMhrJE60Qmb+14bj0OMusDvVEdZJwn8Uhlml8E3ZKjsPeQpzns28R7diMNnnsxwN2mgllv5YJz9l2vem6ouMpL2TuZjdq24n+WDuhqQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=tkos.co.il; spf=pass smtp.mailfrom=tkos.co.il; dkim=pass (2048-bit key) header.d=tkos.co.il header.i=@tkos.co.il header.b=sqYrURUN; arc=none smtp.client-ip=84.110.109.230 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=tkos.co.il Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=tkos.co.il Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=tkos.co.il header.i=@tkos.co.il header.b="sqYrURUN" Received: from tarshish.tkos.co.il (unknown [10.0.8.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.tkos.co.il (Postfix) with ESMTPS id 9C11D440822; Mon, 29 Jul 2024 13:50:32 +0300 (IDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tkos.co.il; s=default; t=1722250232; bh=Za3ndOc802AihmgHa1yG/1ah7Y+qALAilhokvJR5l6o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sqYrURUNeVzskJP5y542ZUwGS+Yuc8/B6oLImG2wmK1bofHj9wdMUUCE+ZNo3QwtV C9couwt2AgMZhgvSGDZhKyrVYGTfNYoxhTIjl+YEdDsYUq7vT/52VuqzjF2Frp1Uvr wLubP/hRfj5O5ezNtoG6/ej+TSbNf5hpEAl4UsAbSCd2ZvvqgyOzobLzNjS35zVPMH vs1JX+wzlRu7zbGJqLEaaF0nfp1tefez3QpSt1mBhVPrbyUZk19dmMWSpLg7wtuD9W JSYE9nfzIc6YtLFlQCsttUXpM3a7cjfkVN80+tYDYZVVzYdqWMzKnVHrr79LwyZeDz 6UTsuJtIJu/QA== From: Baruch Siach To: Christoph Hellwig , Marek Szyprowski , Catalin Marinas , Will Deacon Cc: Baruch Siach , Robin Murphy , iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, =?UTF-8?q?Petr=20Tesa=C5=99=C3=ADk?= , Ramon Fried , Elad Nachman Subject: [PATCH v3 1/3] dma-mapping: improve DMA zone selection Date: Mon, 29 Jul 2024 13:51:24 +0300 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When device DMA limit does not fit in DMA32 zone it should use DMA zone, even when DMA zone is stricter than needed. Same goes for devices that can't allocate from the entire normal zone. Limit to DMA32 in that case. Reported-by: Catalin Marinas Signed-off-by: Baruch Siach --- kernel/dma/direct.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 4480a3cd92e0..3b4be4ca3b08 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -4,7 +4,7 @@ * * DMA operations that map physical memory directly without using an IOMMU. */ -#include /* for max_pfn */ +#include #include #include #include @@ -59,9 +59,9 @@ static gfp_t dma_direct_optimal_gfp_mask(struct device *d= ev, u64 *phys_limit) * zones. */ *phys_limit =3D dma_to_phys(dev, dma_limit); - if (*phys_limit <=3D DMA_BIT_MASK(zone_dma_bits)) + if (*phys_limit < DMA_BIT_MASK(32)) return GFP_DMA; - if (*phys_limit <=3D DMA_BIT_MASK(32)) + if (*phys_limit < memblock_end_of_DRAM()) return GFP_DMA32; return 0; } --=20 2.43.0 From nobody Thu Dec 18 19:27:06 2025 Received: from mail.tkos.co.il (mail.tkos.co.il [84.110.109.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D702144D34; Mon, 29 Jul 2024 10:52:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=84.110.109.230 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722250325; cv=none; b=vEIQFWC4VL56/qAYHeul7KCfJ8pQ1VEmStZ4mDphxHxbBVs3kw1ZaQmu8o5hZYjmPDCvfR9g2qDXwHnevc+Wmr9iH870TqTXn/DhelWAmTcgDNEU5wgS60D/52NG30QqTAbZJIK1RumCSw0/4eRmtLvVdhQWSzcrjlmoYeI1vqg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722250325; c=relaxed/simple; bh=RsVuCrJqd/4oflOyCnW5tYkyh6HR+mMHI8z3aK7dj3A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RAkmkhuBMlfL/I7qK6plqbp8atfJQ5eCDf4NLMf6jjWkHB0KiEJXwKFK5jG/hPFnonar0lHGMqJxJSHxeX4Q/ViqXPiUglmIrQNUWMjfmuH88yCm1SqZMi674NmK0dWjaFWNo7BxXc/I1uJainOx07djHHZXkV3DWztUl8ziw6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=tkos.co.il; spf=pass smtp.mailfrom=tkos.co.il; dkim=pass (2048-bit key) header.d=tkos.co.il header.i=@tkos.co.il header.b=RB5B1Wfd; arc=none smtp.client-ip=84.110.109.230 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=tkos.co.il Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=tkos.co.il Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=tkos.co.il header.i=@tkos.co.il header.b="RB5B1Wfd" Received: from tarshish.tkos.co.il (unknown [10.0.8.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.tkos.co.il (Postfix) with ESMTPS id DBCDA4408CD; Mon, 29 Jul 2024 13:50:32 +0300 (IDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tkos.co.il; s=default; t=1722250233; bh=RsVuCrJqd/4oflOyCnW5tYkyh6HR+mMHI8z3aK7dj3A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RB5B1WfdkmWSwn/AKbhsEsooumaaKNPHCyplUVRa5nNrb9Eqky4nFoeWystTLcmGG z8asTxPicr9dpGq7EHhS9XB1CsLLBYSK7CPLvj+e0CWhl4ayXpnsB/+bMIDCkgnC0J KXCWi2e9OmnulwAridQazeH9+ovQJ1SctWqCcVDWMX4tSBNOC0MzAjSAMpqHkvlvrr LwlIyvJBRoJQje2PkY3jZ00hH/ABZjJf4APxnwTY3EevUBEtBNsZnpDUL82XAAJ0sY tXPUJTpsTQvn7fdQG5S8j+W54rkebWymkCYsTTjUnH1XRIjaixBZdNaAneKHaH6/yp eFjIv8hxjNjqQ== From: Baruch Siach To: Christoph Hellwig , Marek Szyprowski , Catalin Marinas , Will Deacon Cc: Baruch Siach , Robin Murphy , iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, =?UTF-8?q?Petr=20Tesa=C5=99=C3=ADk?= , Ramon Fried , Elad Nachman Subject: [PATCH v3 2/3] dma-mapping: replace zone_dma_bits by zone_dma_limit Date: Mon, 29 Jul 2024 13:51:25 +0300 Message-ID: <053fa4806a2c63efcde80caca473a8b670a2701c.1722249878.git.baruch@tkos.co.il> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Catalin Marinas Hardware DMA limit might not be power of 2. When RAM range starts above 0, say 4GB, DMA limit of 30 bits should end at 5GB. A single high bit can not encode this limit. Use direct phys_addr_t limit address for DMA zone limit. Signed-off-by: Catalin Marinas Signed-off-by: Baruch Siach --- arch/arm64/mm/init.c | 34 +++++++++++++--------------------- arch/powerpc/mm/mem.c | 9 ++++----- arch/s390/mm/init.c | 2 +- include/linux/dma-direct.h | 2 +- kernel/dma/direct.c | 4 ++-- kernel/dma/pool.c | 2 +- kernel/dma/swiotlb.c | 4 ++-- 7 files changed, 24 insertions(+), 33 deletions(-) diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 9b5ab6818f7f..870fd967c610 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -114,36 +114,28 @@ static void __init arch_reserve_crashkernel(void) low_size, high); } =20 -/* - * Return the maximum physical address for a zone accessible by the given = bits - * limit. If DRAM starts above 32-bit, expand the zone to the maximum - * available memory, otherwise cap it at 32-bit. - */ -static phys_addr_t __init max_zone_phys(unsigned int zone_bits) +static phys_addr_t __init max_zone_phys(phys_addr_t zone_limit) { - phys_addr_t zone_mask =3D DMA_BIT_MASK(zone_bits); - phys_addr_t phys_start =3D memblock_start_of_DRAM(); - - if (phys_start > U32_MAX) - zone_mask =3D PHYS_ADDR_MAX; - else if (phys_start > zone_mask) - zone_mask =3D U32_MAX; + /* We have RAM in low 32-bit area, keep DMA zone there */ + if (memblock_start_of_DRAM() < U32_MAX) + zone_limit =3D min(U32_MAX, zone_limit); =20 - return min(zone_mask, memblock_end_of_DRAM() - 1) + 1; + return min(zone_limit, memblock_end_of_DRAM() - 1) + 1; } =20 static void __init zone_sizes_init(void) { unsigned long max_zone_pfns[MAX_NR_ZONES] =3D {0}; - unsigned int __maybe_unused acpi_zone_dma_bits; - unsigned int __maybe_unused dt_zone_dma_bits; - phys_addr_t __maybe_unused dma32_phys_limit =3D max_zone_phys(32); + phys_addr_t __maybe_unused acpi_zone_dma_limit; + phys_addr_t __maybe_unused dt_zone_dma_limit; + phys_addr_t __maybe_unused dma32_phys_limit =3D + max_zone_phys(DMA_BIT_MASK(32)); =20 #ifdef CONFIG_ZONE_DMA - acpi_zone_dma_bits =3D fls64(acpi_iort_dma_get_max_cpu_address()); - dt_zone_dma_bits =3D fls64(of_dma_get_max_cpu_address(NULL)); - zone_dma_bits =3D min3(32U, dt_zone_dma_bits, acpi_zone_dma_bits); - arm64_dma_phys_limit =3D max_zone_phys(zone_dma_bits); + acpi_zone_dma_limit =3D acpi_iort_dma_get_max_cpu_address(); + dt_zone_dma_limit =3D of_dma_get_max_cpu_address(NULL); + zone_dma_limit =3D min(dt_zone_dma_limit, acpi_zone_dma_limit); + arm64_dma_phys_limit =3D max_zone_phys(zone_dma_limit); max_zone_pfns[ZONE_DMA] =3D PFN_DOWN(arm64_dma_phys_limit); #endif #ifdef CONFIG_ZONE_DMA32 diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index d325217ab201..342c006cc1b8 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -216,7 +216,7 @@ static int __init mark_nonram_nosave(void) * everything else. GFP_DMA32 page allocations automatically fall back to * ZONE_DMA. * - * By using 31-bit unconditionally, we can exploit zone_dma_bits to inform= the + * By using 31-bit unconditionally, we can exploit zone_dma_limit to infor= m the * generic DMA mapping code. 32-bit only devices (if not handled by an IO= MMU * anyway) will take a first dip into ZONE_NORMAL and get otherwise served= by * ZONE_DMA. @@ -252,13 +252,12 @@ void __init paging_init(void) * powerbooks. */ if (IS_ENABLED(CONFIG_PPC32)) - zone_dma_bits =3D 30; + zone_dma_limit =3D DMA_BIT_MASK(30); else - zone_dma_bits =3D 31; + zone_dma_limit =3D DMA_BIT_MASK(31); =20 #ifdef CONFIG_ZONE_DMA - max_zone_pfns[ZONE_DMA] =3D min(max_low_pfn, - 1UL << (zone_dma_bits - PAGE_SHIFT)); + max_zone_pfns[ZONE_DMA] =3D min(max_low_pfn, zone_dma_limit >> PAGE_SHIFT= ); #endif max_zone_pfns[ZONE_NORMAL] =3D max_low_pfn; #ifdef CONFIG_HIGHMEM diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c index ddcd39ef4346..91fc2b91adfc 100644 --- a/arch/s390/mm/init.c +++ b/arch/s390/mm/init.c @@ -97,7 +97,7 @@ void __init paging_init(void) =20 vmem_map_init(); sparse_init(); - zone_dma_bits =3D 31; + zone_dma_limit =3D DMA_BIT_MASK(31); memset(max_zone_pfns, 0, sizeof(max_zone_pfns)); max_zone_pfns[ZONE_DMA] =3D virt_to_pfn(MAX_DMA_ADDRESS); max_zone_pfns[ZONE_NORMAL] =3D max_low_pfn; diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h index edbe13d00776..98b7d8015043 100644 --- a/include/linux/dma-direct.h +++ b/include/linux/dma-direct.h @@ -12,7 +12,7 @@ #include #include =20 -extern unsigned int zone_dma_bits; +extern phys_addr_t zone_dma_limit; =20 /* * Record the mapping of CPU physical to DMA addresses for a given region. diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 3b4be4ca3b08..3dbc0b89d6fb 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -20,7 +20,7 @@ * it for entirely different regions. In that case the arch code needs to * override the variable below for dma-direct to work properly. */ -unsigned int zone_dma_bits __ro_after_init =3D 24; +phys_addr_t zone_dma_limit __ro_after_init =3D DMA_BIT_MASK(24); =20 static inline dma_addr_t phys_to_dma_direct(struct device *dev, phys_addr_t phys) @@ -580,7 +580,7 @@ int dma_direct_supported(struct device *dev, u64 mask) * part of the check. */ if (IS_ENABLED(CONFIG_ZONE_DMA)) - min_mask =3D min_t(u64, min_mask, DMA_BIT_MASK(zone_dma_bits)); + min_mask =3D min_t(u64, min_mask, zone_dma_limit); return mask >=3D phys_to_dma_unencrypted(dev, min_mask); } =20 diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c index d10613eb0f63..410a7b40e496 100644 --- a/kernel/dma/pool.c +++ b/kernel/dma/pool.c @@ -70,7 +70,7 @@ static bool cma_in_zone(gfp_t gfp) /* CMA can't cross zone boundaries, see cma_activate_area() */ end =3D cma_get_base(cma) + size - 1; if (IS_ENABLED(CONFIG_ZONE_DMA) && (gfp & GFP_DMA)) - return end <=3D DMA_BIT_MASK(zone_dma_bits); + return end <=3D zone_dma_limit; if (IS_ENABLED(CONFIG_ZONE_DMA32) && (gfp & GFP_DMA32)) return end <=3D DMA_BIT_MASK(32); return true; diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index df68d29740a0..dfd83e5ee0b3 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -450,7 +450,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, if (!remap) io_tlb_default_mem.can_grow =3D true; if (IS_ENABLED(CONFIG_ZONE_DMA) && (gfp_mask & __GFP_DMA)) - io_tlb_default_mem.phys_limit =3D DMA_BIT_MASK(zone_dma_bits); + io_tlb_default_mem.phys_limit =3D zone_dma_limit; else if (IS_ENABLED(CONFIG_ZONE_DMA32) && (gfp_mask & __GFP_DMA32)) io_tlb_default_mem.phys_limit =3D DMA_BIT_MASK(32); else @@ -629,7 +629,7 @@ static struct page *swiotlb_alloc_tlb(struct device *de= v, size_t bytes, } =20 gfp &=3D ~GFP_ZONEMASK; - if (phys_limit <=3D DMA_BIT_MASK(zone_dma_bits)) + if (phys_limit <=3D zone_dma_limit) gfp |=3D __GFP_DMA; else if (phys_limit <=3D DMA_BIT_MASK(32)) gfp |=3D __GFP_DMA32; --=20 2.43.0 From nobody Thu Dec 18 19:27:06 2025 Received: from mail.tkos.co.il (mail.tkos.co.il [84.110.109.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA7EB1448F1; Mon, 29 Jul 2024 10:51:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=84.110.109.230 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722250320; cv=none; b=bp/B04RDxjQeMlU6obJXuo7Q5bc8ggaK+UiTYWnOcZfVfogtBHjn3PbjwQUcJr0Rskd1TZqYCEWGk+C6LnKHi4FLEBilJcbfD6ZAD/t1xabUnXCw/OO++kIATOYUILH0zGfmyACfhjBPlf9UKR+iDbaodPEJjcDeOOYMRZo8gvQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722250320; c=relaxed/simple; bh=jwVrNkJU6TTXZr03Aq1yS4EYBFAKJQ5ZGtsXERqr5z8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qyJp5CIA932iDjU3XqM4JFxoyBF0wQaBZgneA33ntXRhzjfOI0Htbgu+E1elRSumfxBE7AVJ8y8DElD3kgmq35SRQ7u3CxCMAcYYAqQf2spc6kMLXKKd3+qnnaT6/bJ1UiZHympxc1tXqi2oiuR90H/hAzyD4JB3bMufqGeP4gY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=tkos.co.il; spf=pass smtp.mailfrom=tkos.co.il; dkim=pass (2048-bit key) header.d=tkos.co.il header.i=@tkos.co.il header.b=vBf0lRuB; arc=none smtp.client-ip=84.110.109.230 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=tkos.co.il Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=tkos.co.il Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=tkos.co.il header.i=@tkos.co.il header.b="vBf0lRuB" Received: from tarshish.tkos.co.il (unknown [10.0.8.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.tkos.co.il (Postfix) with ESMTPS id 29561440F5F; Mon, 29 Jul 2024 13:50:33 +0300 (IDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tkos.co.il; s=default; t=1722250233; bh=jwVrNkJU6TTXZr03Aq1yS4EYBFAKJQ5ZGtsXERqr5z8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vBf0lRuBpzD9o62Xp531a74Gq1NOBiZZUSIxG/oPDdoC8/1xjDtYtq/akdkDcE0kj D1/LaqCflZE12HPEnrCkolPF8KvnZtr0KrWCTKoSR/xXowyJXF3UOmXUIhLeKQV+o1 DIHAXmMYS3wurN++0F2Ju5m35D+DKYCAgyT5xBmceFFbv2mGTM4Ju+wr8gnSO79YxY /GvtYpbvpKizJHES0JaD2aqYkam5SjvYosOTgWeHXdlvH1I8p3GU3NQ2+uxTWff2bQ uF95ALYYrb+0bwo2KdyQGoaHC7xT0IU01q0mAmBI5/zLflThrleJzDthkYSfLQq70b sxZH+zJqWlzmA== From: Baruch Siach To: Christoph Hellwig , Marek Szyprowski , Catalin Marinas , Will Deacon Cc: Baruch Siach , Robin Murphy , iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-s390@vger.kernel.org, =?UTF-8?q?Petr=20Tesa=C5=99=C3=ADk?= , Ramon Fried , Elad Nachman Subject: [PATCH v3 3/3] dma-direct: use RAM start to offset zone_dma_limit Date: Mon, 29 Jul 2024 13:51:26 +0300 Message-ID: <629b184354fa22cb32a90bd1fa0e1dc258251f81.1722249878.git.baruch@tkos.co.il> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Current code using zone_dma_limit assume that all address range below limit is suitable for DMA. For some existing platforms this assumption is not correct. DMA range might have non zero lower limit. Commit 791ab8b2e3db ("arm64: Ignore any DMA offsets in the max_zone_phys() calculation") made DMA/DMA32 zones span the entire RAM when RAM starts above 32-bits. This breaks hardware with DMA area that start above 32-bits. But the commit log says that "we haven't noticed any such hardware". It turns out that such hardware does exist. One such platform has RAM starting at 32GB with an internal bus that has the following DMA limits: #address-cells =3D <2>; #size-cells =3D <2>; dma-ranges =3D <0x00 0xc0000000 0x08 0x00000000 0x00 0x40000000>; Devices under this bus can see 1GB of DMA range between 3GB-4GB in each device address space. This range is mapped to CPU memory at 32GB-33GB. With current code DMA allocations for devices under this bus are not limited to DMA area, leading to run-time allocation failure. Add start of RAM address to zone_dma_limit to make DMA allocation for constrained devices possible. The result is DMA zone that properly reflects the hardware constraints as follows: [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000800000000-0x000000083fffffff] [ 0.000000] DMA32 empty [ 0.000000] Normal [mem 0x0000000840000000-0x0000000bffffffff] Rename the dma_direct_supported() local 'min_mask' variable to better describe its use as limit. Suggested-by: Catalin Marinas Signed-off-by: Baruch Siach --- kernel/dma/direct.c | 7 ++++--- kernel/dma/pool.c | 3 ++- kernel/dma/swiotlb.c | 4 ++-- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 3dbc0b89d6fb..bd7972d3b101 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -563,7 +563,7 @@ int dma_direct_mmap(struct device *dev, struct vm_area_= struct *vma, =20 int dma_direct_supported(struct device *dev, u64 mask) { - u64 min_mask =3D (max_pfn - 1) << PAGE_SHIFT; + u64 min_limit =3D (max_pfn - 1) << PAGE_SHIFT; =20 /* * Because 32-bit DMA masks are so common we expect every architecture @@ -580,8 +580,9 @@ int dma_direct_supported(struct device *dev, u64 mask) * part of the check. */ if (IS_ENABLED(CONFIG_ZONE_DMA)) - min_mask =3D min_t(u64, min_mask, zone_dma_limit); - return mask >=3D phys_to_dma_unencrypted(dev, min_mask); + min_limit =3D min_t(u64, min_limit, + memblock_start_of_DRAM() + zone_dma_limit); + return mask >=3D phys_to_dma_unencrypted(dev, min_limit); } =20 /* diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c index 410a7b40e496..ded3d841c88c 100644 --- a/kernel/dma/pool.c +++ b/kernel/dma/pool.c @@ -12,6 +12,7 @@ #include #include #include +#include =20 static struct gen_pool *atomic_pool_dma __ro_after_init; static unsigned long pool_size_dma; @@ -70,7 +71,7 @@ static bool cma_in_zone(gfp_t gfp) /* CMA can't cross zone boundaries, see cma_activate_area() */ end =3D cma_get_base(cma) + size - 1; if (IS_ENABLED(CONFIG_ZONE_DMA) && (gfp & GFP_DMA)) - return end <=3D zone_dma_limit; + return end <=3D memblock_start_of_DRAM() + zone_dma_limit; if (IS_ENABLED(CONFIG_ZONE_DMA32) && (gfp & GFP_DMA32)) return end <=3D DMA_BIT_MASK(32); return true; diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index dfd83e5ee0b3..2813eeb8b375 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -450,7 +450,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, if (!remap) io_tlb_default_mem.can_grow =3D true; if (IS_ENABLED(CONFIG_ZONE_DMA) && (gfp_mask & __GFP_DMA)) - io_tlb_default_mem.phys_limit =3D zone_dma_limit; + io_tlb_default_mem.phys_limit =3D memblock_start_of_DRAM() + zone_dma_li= mit; else if (IS_ENABLED(CONFIG_ZONE_DMA32) && (gfp_mask & __GFP_DMA32)) io_tlb_default_mem.phys_limit =3D DMA_BIT_MASK(32); else @@ -629,7 +629,7 @@ static struct page *swiotlb_alloc_tlb(struct device *de= v, size_t bytes, } =20 gfp &=3D ~GFP_ZONEMASK; - if (phys_limit <=3D zone_dma_limit) + if (phys_limit <=3D memblock_start_of_DRAM() + zone_dma_limit) gfp |=3D __GFP_DMA; else if (phys_limit <=3D DMA_BIT_MASK(32)) gfp |=3D __GFP_DMA32; --=20 2.43.0