From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A61043DA5C; Thu, 24 Sep 2026 10:06:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244377; cv=none; b=T5r7M3l1p0djG0KB2FIPZzmmsEPNLN/CafbP29qAsu+EtkKlsmkILuXrJV8DFGFLdBcQvTovdhtpRnDmI/h+ZqQ3Y7YVgQj9A+xMmxAa5rz2CYEOymZxXIMHtODeNSe6OXR1+orBCnBRTrMkfqri3GWhMN2Ei2l88Eb25SRDaPE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244377; c=relaxed/simple; bh=B/ilBvQaukranZL4YbLeq5rqFwWJqKrxYlAeRc3LaKE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tNO7sQ8/X/FzfeVP/JdFChXDT1Q/Z5IolbgYL1HzKoqiehf1OEeple0bzksFjbanM8UuFr6rN8TjW8Ty2WyDDaxshMOXd/g68PDZ1O4devlbUIDSHqUlVAkV4pYWBDS7OJaXTSzAv2tm09ZIJ44ytdgM2Ac8j9q2V6g+Y9FojmA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ch5tx7zD; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ch5tx7zD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A26BD1F00893; Thu, 24 Sep 2026 10:05:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244367; bh=6i5Bn/p9hoYkPkAS4fE0FC3u08lww1IyLVqKIgULjmw=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Ch5tx7zDK2am4maKTos8NryZBOb9Q9yKA2U1aj8MpTofoC0f4brL3bAeytSaOvGYy fDsWrt8SVwJVn9N1tSC9N7RcwWe3yGqb9p6XSr09zdW6qYzPndPqljKfmUQl7WxIAO ulbER1KfvK1A+1r06J85qrf9/l7nA8SJoiENU7EMpivAr8GTvf0biNlSXhA5R8ZS1o PqzwZEI1d+rqcIXxLuUvmNB7qS+eDjNsrYtizjnVC25Jm9KuFhBltE4xoGZbO84fYQ zMCmmYyUuQTLuB4dwHz/aOgbihC2Fsm2XyqIyVf9nCSvR/8f40Pbdy6G6PuW0zk3WW sY7AF4kTeKdMw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 01/14] mm: Add an allocator for CoCo shared memory Date: Thu, 24 Sep 2026 15:35:16 +0530 Message-ID: <20260924100529.1398790-2-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Confidential-computing guests may require memory shared with the host to be aligned and transitioned in units larger than PAGE_SIZE. Several DMA users need struct page-backed allocations satisfying these requirements. Provide a common allocator instead of requiring each user to open-code this sequence. Add alloc_cc_shared_pages() and its node-aware variant. The allocator rounds the requested size to the architecture's shared granule, allocates suitably aligned contiguous pages and transitions the complete range to shared state. It also preserves the caller's GFP policy. A private-to-shared transition may alter memory contents. Mask __GFP_ZERO from the underlying allocation and when requested, clear the complete transitioned range after cc_make_shared() succeeds. Return the page and transitioned size so free_cc_shared_pages() can restore the complete range to private state before freeing it. If private state cannot be established, retain the allocation instead of returning a possibly shared page to the buddy allocator. Also provide the shared-granule geometry and byte-oriented transition helpers used by the allocator and by callers managing their own backing memory. Cc: Andrew Morton Signed-off-by: Aneesh Kumar K.V (Arm) --- include/linux/cc_shared.h | 39 +++++++ mm/Makefile | 1 + mm/cc_shared.c | 232 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 272 insertions(+) create mode 100644 include/linux/cc_shared.h create mode 100644 mm/cc_shared.c diff --git a/include/linux/cc_shared.h b/include/linux/cc_shared.h new file mode 100644 index 000000000000..5f8db7c468c5 --- /dev/null +++ b/include/linux/cc_shared.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_CC_SHARED_H +#define _LINUX_CC_SHARED_H + +#include +#include + +struct page; + +struct cc_shared_pages { + struct page *page; + size_t shared_size; +}; + +struct cc_shared_layout { + size_t requested_size; + size_t shared_size; + size_t alignment; +}; + +/* + * Architectures may override this to return the granule used for transiti= ons + * between private and shared memory. The value must be a power of two and= no + * smaller than PAGE_SIZE. + */ +size_t arch_cc_shared_granule_size(void); + +size_t cc_shared_granule_size(void); +int cc_shared_calc_layout(size_t requested, struct cc_shared_layout *layou= t); +bool cc_shared_range_valid(phys_addr_t base, size_t size); +int cc_make_shared(void *addr, size_t size); +int cc_make_private(void *addr, size_t size); +int alloc_cc_shared_pages_node(int nid, gfp_t gfp, + size_t requested, struct cc_shared_pages *mem); +int alloc_cc_shared_pages(gfp_t gfp, + size_t requested, struct cc_shared_pages *mem); +void free_cc_shared_pages(struct cc_shared_pages *mem); + +#endif /* _LINUX_CC_SHARED_H */ diff --git a/mm/Makefile b/mm/Makefile index e7245cb88c66..6e6544428422 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -56,6 +56,7 @@ obj-y :=3D filemap.o mempool.o oom_kill.o fadvise.o \ compaction.o show_mem.o \ interval_tree.o list_lru.o workingset.o \ debug.o gup.o mmap_lock.o vma_init.o $(mmu-y) +obj-y +=3D cc_shared.o =20 # Give 'page_alloc' its own module-parameter namespace page-alloc-y :=3D page_alloc.o diff --git a/mm/cc_shared.c b/mm/cc_shared.c new file mode 100644 index 000000000000..85e16f4504b8 --- /dev/null +++ b/mm/cc_shared.c @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2026 ARM Ltd. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +size_t __weak arch_cc_shared_granule_size(void) +{ + return PAGE_SIZE; +} + +size_t cc_shared_granule_size(void) +{ + size_t granule =3D arch_cc_shared_granule_size(); + + if (WARN_ON_ONCE(granule < PAGE_SIZE || !is_power_of_2(granule))) + return PAGE_SIZE; + + return granule; +} +EXPORT_SYMBOL_GPL(cc_shared_granule_size); + +int cc_shared_calc_layout(size_t requested, struct cc_shared_layout *layou= t) +{ + size_t granule, rounded; + + if (!requested || !layout) + return -EINVAL; + + granule =3D cc_shared_granule_size(); + if (check_add_overflow(requested, granule - 1, &rounded)) + return -EOVERFLOW; + + rounded =3D ALIGN_DOWN(rounded, granule); + layout->requested_size =3D requested; + layout->shared_size =3D rounded; + layout->alignment =3D granule; + + return 0; +} +EXPORT_SYMBOL_GPL(cc_shared_calc_layout); + +bool cc_shared_range_valid(phys_addr_t base, size_t size) +{ + size_t granule =3D cc_shared_granule_size(); + + if (!size) + return false; + + return IS_ALIGNED(base, granule) && IS_ALIGNED(size, granule); +} +EXPORT_SYMBOL_GPL(cc_shared_range_valid); + +static int cc_validate_transition(void *addr, size_t size) +{ + phys_addr_t phys; + + if (!addr || !size || !PAGE_ALIGNED(addr) || + !virt_addr_valid(addr)) + return -EINVAL; + + phys =3D page_to_phys(virt_to_page(addr)); + if (!cc_shared_range_valid(phys, size)) + return -EINVAL; + + return 0; +} + +int cc_make_shared(void *addr, size_t size) +{ + int ret =3D cc_validate_transition(addr, size); + + if (ret) + return ret; + + return set_memory_decrypted((unsigned long)addr, size >> PAGE_SHIFT); +} + +int cc_make_private(void *addr, size_t size) +{ + int ret =3D cc_validate_transition(addr, size); + + if (ret) + return ret; + + return set_memory_encrypted((unsigned long)addr, size >> PAGE_SHIFT); +} + +static int __alloc_cc_shared_pages_node(int nid, gfp_t gfp, + size_t requested, + struct cc_shared_pages *mem) +{ + struct cc_shared_layout layout; + struct page *page; + unsigned int order; + bool zero =3D gfp & __GFP_ZERO; + int ret; + + ret =3D cc_shared_calc_layout(requested, &layout); + if (ret) + return ret; + + order =3D get_order(layout.shared_size); + if (order > MAX_PAGE_ORDER) + return -EINVAL; + + /* + * State transitions require a linear-map address and may modify memory. + * Allocate from low memory and defer requested zeroing until afterwards. + */ + gfp &=3D ~(__GFP_HIGHMEM | __GFP_ZERO); + if (nid =3D=3D NUMA_NO_NODE) + page =3D alloc_pages(gfp, order); + else + page =3D alloc_pages_node(nid, gfp, order); + if (!page) + return -ENOMEM; + + ret =3D cc_make_shared(page_address(page), layout.shared_size); + if (ret) { + if (!cc_make_private(page_address(page), layout.shared_size)) + __free_pages(page, order); + else + pr_warn_ratelimited("leaking %zu bytes with uncertain shared state\n", + layout.shared_size); + return ret; + } + + if (zero) + memset(page_address(page), 0, layout.shared_size); + + mem->page =3D page; + mem->shared_size =3D layout.shared_size; + return 0; +} + +/** + * alloc_cc_shared_pages_node - allocate memory that can be shared + * @nid: NUMA node from which to allocate, or %NUMA_NO_NODE + * @gfp: allocation flags + * @requested: number of bytes requested; must be nonzero + * @mem: storage for the allocated page and the size of the shared range + * + * Allocate at least @requested bytes and make the allocation shared when + * memory encryption is active. A memory-state transition requires a valid + * linear-map address, so such allocations never come from high memory + * + * The shared range may be rounded up to the architecture's transition + * granule. On success, @mem->shared_size records the actual size that was + * made shared and must be retained unchanged for free_cc_shared_pages(). + * @mem is not modified on failure. + * + * Return: 0 on success, or a negative error code on failure. + */ +int alloc_cc_shared_pages_node(int nid, gfp_t gfp, + size_t requested, + struct cc_shared_pages *mem) +{ + struct page *page; + unsigned int order; + + if (!mem || !requested) + return -EINVAL; + + if (cc_platform_has(CC_ATTR_MEM_ENCRYPT)) + return __alloc_cc_shared_pages_node(nid, gfp, requested, mem); + + order =3D get_order(requested); + if (order > MAX_PAGE_ORDER) + return -EINVAL; + + if (nid =3D=3D NUMA_NO_NODE) + page =3D alloc_pages(gfp, order); + else + page =3D alloc_pages_node(nid, gfp, order); + if (!page) + return -ENOMEM; + + mem->page =3D page; + mem->shared_size =3D requested; + return 0; +} +EXPORT_SYMBOL_GPL(alloc_cc_shared_pages_node); + +/** + * alloc_cc_shared_pages - allocate memory that can be shared + * @gfp: allocation flags + * @requested: number of bytes requested; must be nonzero + * @mem: storage for the allocated page and the size of the shared range + * + * Equivalent to alloc_cc_shared_pages_node() with %NUMA_NO_NODE. + * + * Return: 0 on success, or a negative error code on failure. + */ +int alloc_cc_shared_pages(gfp_t gfp, + size_t requested, struct cc_shared_pages *mem) +{ + return alloc_cc_shared_pages_node(NUMA_NO_NODE, gfp, requested, mem); +} +EXPORT_SYMBOL_GPL(alloc_cc_shared_pages); + +void free_cc_shared_pages(struct cc_shared_pages *mem) +{ + if (!mem || !mem->page) + return; + + if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT)) + goto free_pages; + + if (cc_make_private(page_address(mem->page), mem->shared_size)) { + pr_warn_ratelimited("leaking %zu bytes that cannot be made private\n", + mem->shared_size); + return; + } + +free_pages: + __free_pages(mem->page, get_order(mem->shared_size)); + mem->page =3D NULL; + mem->shared_size =3D 0; +} +EXPORT_SYMBOL_GPL(free_cc_shared_pages); --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E25E043CE56; Thu, 24 Sep 2026 10:06:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244395; cv=none; b=pe7VqWcv3kvHrVvvvrZc2EPsknLy6a5RACLd3OcLtIko7TQSO7rvxm4H7oW+uRVMIkseI8gnfmnrAOj3S7k/BeZO5Me0EVNC2qK9Ryed7aTmbT6GEFhgAw9MhSTjban1CJ4Ju7mYB9ZADLF+c5LzMpLUfCcqVPQRankqe6hz0nI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244395; c=relaxed/simple; bh=2QYvsokSFmesMS1hra/fv9j0z3vvm9MNiLpq/KcP4pc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uP1WA4+wLhy1ienuame0vWubcu4JnJNjFr15NptY54RcWSmeSx/t6a4Lf/9nPEbgUu6aEIwerleTflGYy3sNUbiJiedG2gXGprUnJSnC6wLUl3O14nzy8TFtRnKCvGr3H+S7xDU4TmsAK8LVDD/VyBeyxiOhrTOCRzOWj4thC3Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AM+mpu9q; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AM+mpu9q" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BD31D1F000FF; Thu, 24 Sep 2026 10:06:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244384; bh=Ied2AelGOAUoBLypHiwb2BL1zLogQy2LqP9NEecl9Ec=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=AM+mpu9qGXxDz7bD9e86xWdXibQkZA/5cUFq8K+AJhu+OLOZOG3vMF9mBRTc+zGqI yRj9S4yaKhbuaIGypSGuFG49BwMWg7fZoz4YBrRNXGjOa8KkLOa6Kqdg9Mmjwt0q9Q ehXg0pJlZ1ALhgUqcXmE6XX5MLHMwgqnU3WirpecNc1Es05E37mMF/ufHlErk7/MSf iYgfs7myFqVwvkuOx0gREfI/UFZ+04YCqr2grhPtJdc/U8KkMMPJgqf3yT/2Rfmqui JTCFvApGvxYleMqambDrmxZ3rg9Pz/jTjTmV4Ve6a523KZYTJFPlknUQuc9r3kvPDT oi++Sb59RmMoQ== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 02/14] mm: Zero memory during shared memory transitions Date: Thu, 24 Sep 2026 15:35:17 +0530 Message-ID: <20260924100529.1398790-3-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Architectures need to zero memory at different points in a private-to-shared transition. For example, pKVM must clear memory before sharing it, while Arm CCA can only clear it after the RSI transition has completed. Make zeroing an implicit property of set_memory_decrypted(). Each architect= ure or platform performs the clear at the safe point in its transition. On x86, perform it in the prepare callback while the private mapping remains accessible. This avoids adding a flag to set_memory_decrypted() and threadi= ng it through the architecture-specific callbacks. Keep allocations on platforms without memory encryption on the ordinary page allocator path so the original GFP constraints, including __GFP_ZERO, remain intact. Remove post-transition memset() calls that are now redundant and explicitly clear allocations on paths where no transition occurs. This intentionally makes every private-to-shared transition destructive; callers can no longer use set_memory_decrypted() to preserve existing conte= nts. Assisted-by: Codex:gpt-5 Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: Madhavan Srinivasan Cc: Michael Ellerman Cc: Nicholas Piggin Cc: Christophe Leroy (CS GROUP) Cc: Ritesh Harjani (IBM) Cc: Shrikanth Hegde Cc: Alexander Gordeev Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Vasily Gorbik Cc: Christian Borntraeger Cc: Sven Schnelle Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Dave Hansen Cc: x86@kernel.org Cc: H. Peter Anvin Cc: Kiryl Shutsemau Cc: Rick Edgecombe Cc: K. Y. Srinivasan Cc: Haiyang Zhang Cc: Wei Liu Cc: Dexuan Cui Cc: Long Li Cc: Paolo Bonzini Cc: Vitaly Kuznetsov Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Marek Szyprowski Cc: Robin Murphy Cc: Andrew Morton Signed-off-by: Aneesh Kumar K.V (Arm) --- arch/arm64/mm/pageattr.c | 3 ++ arch/powerpc/platforms/pseries/svm.c | 2 + arch/s390/mm/init.c | 3 ++ arch/x86/coco/tdx/tdx.c | 3 ++ arch/x86/hyperv/hv_init.c | 6 ++- arch/x86/hyperv/ivm.c | 4 ++ arch/x86/kernel/kvmclock.c | 6 +-- arch/x86/mm/mem_encrypt_amd.c | 4 ++ drivers/hv/connection.c | 41 +++++++++---------- drivers/hv/hv.c | 11 ++--- drivers/hv/hv_common.c | 2 - drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c | 3 ++ kernel/dma/direct.c | 22 ++++++---- mm/cc_shared.c | 7 +--- 14 files changed, 72 insertions(+), 45 deletions(-) diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index bbe98ac9ad8c..ae1f5de66fad 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -339,6 +339,9 @@ static int realm_set_memory_decrypted(unsigned long add= r, int numpages) { int ret =3D __set_memory_enc_dec(addr, numpages, false); =20 + if (!ret) + memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT); + WARN(ret, "Failed to decrypt memory, %d pages will be leaked", numpages); =20 diff --git a/arch/powerpc/platforms/pseries/svm.c b/arch/powerpc/platforms/= pseries/svm.c index 7a403dbd35ee..a344e094fe1c 100644 --- a/arch/powerpc/platforms/pseries/svm.c +++ b/arch/powerpc/platforms/pseries/svm.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -59,6 +60,7 @@ int set_memory_decrypted(unsigned long addr, int numpages) if (!PAGE_ALIGNED(addr)) return -EINVAL; =20 + memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT); uv_share_page(PHYS_PFN(__pa(addr)), numpages); =20 return 0; diff --git a/arch/s390/mm/init.c b/arch/s390/mm/init.c index be7e009e7b59..f2e28a2710e0 100644 --- a/arch/s390/mm/init.c +++ b/arch/s390/mm/init.c @@ -129,6 +129,9 @@ int set_memory_encrypted(unsigned long vaddr, int numpa= ges) int set_memory_decrypted(unsigned long vaddr, int numpages) { int i; + + memset((void *)vaddr, 0, (size_t)numpages << PAGE_SHIFT); + /* make specified pages shared (swiotlb, dma_alloca) */ for (i =3D 0; i < numpages; ++i) { uv_set_shared(virt_to_phys((void *)vaddr)); diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c index f904a636d449..1f1f39082391 100644 --- a/arch/x86/coco/tdx/tdx.c +++ b/arch/x86/coco/tdx/tdx.c @@ -978,6 +978,9 @@ static bool tdx_enc_status_changed(unsigned long vaddr,= int numpages, bool enc) static int tdx_enc_status_change_prepare(unsigned long vaddr, int numpages, bool enc) { + if (!enc) + memset((void *)vaddr, 0, (size_t)numpages << PAGE_SHIFT); + /* * Only handle shared->private conversion here. * See the comment in tdx_early_init(). diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 0b4a1c0b0b16..a42468e81c34 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -156,8 +156,10 @@ static int hv_cpu_init(unsigned int cpu) * page in non-root partition here. */ if (*hvp && !ms_hyperv.paravisor_present && hv_isolation_type_snp()) { - WARN_ON_ONCE(set_memory_decrypted((unsigned long)(*hvp), 1)); - memset(*hvp, 0, PAGE_SIZE); + int ret; + + ret =3D set_memory_decrypted((unsigned long)*hvp, 1); + WARN_ON_ONCE(ret); } } =20 diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 2ce4dfe53472..e9c7799a0094 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -755,6 +755,10 @@ static int hv_mark_gpa_visibility(u16 count, const u64= pfn[], */ static int hv_vtom_clear_present(unsigned long kbuffer, int pagecount, boo= l enc) { + if (!enc) + memset((void *)kbuffer, 0, + (size_t)pagecount << PAGE_SHIFT); + return set_memory_np(kbuffer, pagecount); } =20 diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index cb3d0ca1fa22..3e9b7ad3dea2 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -248,7 +248,7 @@ static void __init kvmclock_init_mem(void) * be mapped decrypted. */ if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { - r =3D set_memory_decrypted((unsigned long) hvclock_mem, + r =3D set_memory_decrypted((unsigned long)hvclock_mem, 1UL << order); if (r) { __free_pages(p, order); @@ -256,9 +256,9 @@ static void __init kvmclock_init_mem(void) pr_warn("kvmclock: set_memory_decrypted() failed. Disabling\n"); return; } + } else { + memset(hvclock_mem, 0, PAGE_SIZE << order); } - - memset(hvclock_mem, 0, PAGE_SIZE << order); } =20 static int __init kvm_setup_vsyscall_timeinfo(void) diff --git a/arch/x86/mm/mem_encrypt_amd.c b/arch/x86/mm/mem_encrypt_amd.c index 2f8c32173972..47cae102acd2 100644 --- a/arch/x86/mm/mem_encrypt_amd.c +++ b/arch/x86/mm/mem_encrypt_amd.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -285,6 +286,9 @@ static void enc_dec_hypercall(unsigned long vaddr, unsi= gned long size, bool enc) =20 static int amd_enc_status_change_prepare(unsigned long vaddr, int npages, = bool enc) { + if (!enc) + memset((void *)vaddr, 0, (size_t)npages << PAGE_SHIFT); + /* * To maintain the security guarantees of SEV-SNP guests, make sure * to invalidate the memory before encryption attribute is cleared. diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c index 1ab3581b096a..3be779093f3c 100644 --- a/drivers/hv/connection.c +++ b/drivers/hv/connection.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -263,29 +264,27 @@ int vmbus_connect(void) goto cleanup; } =20 - ret =3D set_memory_decrypted((unsigned long) - vmbus_connection.monitor_pages[0], 1); - ret |=3D set_memory_decrypted((unsigned long) - vmbus_connection.monitor_pages[1], 1); - if (ret) { - /* - * If set_memory_decrypted() fails, the encryption state - * of the memory is unknown. So leak the memory instead - * of risking returning decrypted memory to the free list. - * For simplicity, always handle both pages the same. - */ - vmbus_connection.monitor_pages[0] =3D NULL; - vmbus_connection.monitor_pages[1] =3D NULL; - goto cleanup; + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { + ret =3D set_memory_decrypted((unsigned long)vmbus_connection.monitor_pag= es[0], + 1); + ret |=3D set_memory_decrypted((unsigned long)vmbus_connection.monitor_pa= ges[1], + 1); + if (ret) { + /* + * If set_memory_decrypted() fails, the encryption state + * of the memory is unknown. So leak the memory instead + * of risking returning decrypted memory to the free list. + * For simplicity, always handle both pages the same. + */ + vmbus_connection.monitor_pages[0] =3D NULL; + vmbus_connection.monitor_pages[1] =3D NULL; + goto cleanup; + } + } else { + memset(vmbus_connection.monitor_pages[0], 0, HV_HYP_PAGE_SIZE); + memset(vmbus_connection.monitor_pages[1], 0, HV_HYP_PAGE_SIZE); } =20 - /* - * Set_memory_decrypted() will change the memory contents if - * decryption occurs, so zero monitor pages here. - */ - memset(vmbus_connection.monitor_pages[0], 0x00, HV_HYP_PAGE_SIZE); - memset(vmbus_connection.monitor_pages[1], 0x00, HV_HYP_PAGE_SIZE); - msginfo =3D kzalloc(sizeof(*msginfo) + sizeof(struct vmbus_channel_initiate_contact), GFP_KERNEL); diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index fe50090dcc01..6e836a02910a 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -123,12 +123,13 @@ static int hv_alloc_page(void **page, bool decrypt, c= onst char *note) if (!*page) return -ENOMEM; =20 - if (decrypt) + if (decrypt) { ret =3D set_memory_decrypted((unsigned long)*page, 1); - if (ret) - goto failed; - - memset(*page, 0, PAGE_SIZE); + if (ret) + goto failed; + } else { + memset(*page, 0, PAGE_SIZE); + } return 0; =20 failed: diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index 31256cb22b39..af6b1531d600 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -505,8 +505,6 @@ int hv_common_cpu_init(unsigned int cpu) /* It may be unsafe to free 'mem' */ return ret; } - - memset(mem, 0x00, pgcount * HV_HYP_PAGE_SIZE); } =20 /* diff --git a/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c b/drivers/virt/c= oco/pkvm-guest/arm-pkvm-guest.c index 26fe9c3f22e3..7d922e0a28f9 100644 --- a/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c +++ b/drivers/virt/coco/pkvm-guest/arm-pkvm-guest.c @@ -13,6 +13,7 @@ #include #include #include +#include =20 #include =20 @@ -61,6 +62,8 @@ static int pkvm_set_memory_encrypted(unsigned long addr, = int numpages) =20 static int pkvm_set_memory_decrypted(unsigned long addr, int numpages) { + memset((void *)addr, 0, (size_t)numpages << PAGE_SHIFT); + return __set_memory_range(ARM_SMCCC_VENDOR_HYP_KVM_MEM_SHARE_FUNC_ID, addr, numpages); } diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index da665ca22d5c..356d4e09e1c8 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -211,7 +211,8 @@ void *dma_direct_alloc(struct device *dev, size_t size, if (force_dma_unencrypted(dev)) attrs |=3D __DMA_ATTR_ALLOC_CC_SHARED; =20 - if (attrs & __DMA_ATTR_ALLOC_CC_SHARED) { + mark_mem_decrypt =3D attrs & __DMA_ATTR_ALLOC_CC_SHARED; + if (mark_mem_decrypt) { /* * Unencrypted/shared DMA requires a linear-mapped buffer * address to look up the PFN and set architecture-required PFN @@ -219,7 +220,6 @@ void *dma_direct_alloc(struct device *dev, size_t size, * allocation. */ allow_highmem =3D false; - mark_mem_decrypt =3D true; } =20 size =3D PAGE_ALIGN(size); @@ -324,7 +324,9 @@ void *dma_direct_alloc(struct device *dev, size_t size, cpu_addr =3D page_address(page); } =20 - memset(cpu_addr, 0, size); + /* Zero after remapping because the page may be in HighMem. */ + if (!mark_mem_decrypt) + memset(cpu_addr, 0, size); =20 if (set_uncached) { void *uncached_cpu_addr; @@ -435,10 +437,13 @@ struct page *dma_direct_alloc_pages(struct device *de= v, size_t size, unsigned long attrs =3D 0; struct page *page; void *cpu_addr; + bool mark_mem_decrypt; =20 if (force_dma_unencrypted(dev)) attrs |=3D __DMA_ATTR_ALLOC_CC_SHARED; =20 + mark_mem_decrypt =3D attrs & __DMA_ATTR_ALLOC_CC_SHARED; + if ((attrs & __DMA_ATTR_ALLOC_CC_SHARED) && dma_direct_use_pool(dev, gfp)) return dma_direct_alloc_from_pool(dev, size, dma_handle, &cpu_addr, gfp, attrs); @@ -449,6 +454,7 @@ struct page *dma_direct_alloc_pages(struct device *dev,= size_t size, return NULL; =20 cpu_addr =3D page_address(page); + mark_mem_decrypt =3D false; goto setup_page; } =20 @@ -457,11 +463,13 @@ struct page *dma_direct_alloc_pages(struct device *de= v, size_t size, return NULL; =20 cpu_addr =3D page_address(page); - if ((attrs & __DMA_ATTR_ALLOC_CC_SHARED) && - dma_set_decrypted(dev, cpu_addr, size)) - goto out_leak_pages; setup_page: - memset(cpu_addr, 0, size); + if (mark_mem_decrypt) { + if (dma_set_decrypted(dev, cpu_addr, size)) + goto out_leak_pages; + } else { + memset(cpu_addr, 0, size); + } *dma_handle =3D phys_to_dma_direct(dev, page_to_phys(page), attrs & __DMA_ATTR_ALLOC_CC_SHARED); return page; diff --git a/mm/cc_shared.c b/mm/cc_shared.c index 85e16f4504b8..9eb79832a702 100644 --- a/mm/cc_shared.c +++ b/mm/cc_shared.c @@ -104,7 +104,6 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t = gfp, struct cc_shared_layout layout; struct page *page; unsigned int order; - bool zero =3D gfp & __GFP_ZERO; int ret; =20 ret =3D cc_shared_calc_layout(requested, &layout); @@ -117,7 +116,8 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t = gfp, =20 /* * State transitions require a linear-map address and may modify memory. - * Allocate from low memory and defer requested zeroing until afterwards. + * Allocate from low memory and let the architecture place zeroing at the + * appropriate point in the transition. */ gfp &=3D ~(__GFP_HIGHMEM | __GFP_ZERO); if (nid =3D=3D NUMA_NO_NODE) @@ -137,9 +137,6 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t = gfp, return ret; } =20 - if (zero) - memset(page_address(page), 0, layout.shared_size); - mem->page =3D page; mem->shared_size =3D layout.shared_size; return 0; --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6A4E3F327F; Thu, 24 Sep 2026 10:06:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244413; cv=none; b=sL5HTogkD7qz0a7Ztoe4Hhq50Yaklu6YydM5Sw+XTKaHAd/87ev14RmUVKH2WflQPxblvjOUiWaGli+y9G/rp4JnQ2OIX7ZyYqLks6YcqrWRWBVA4GDOhwiLa8EfOX07PRzSNnIrF/uxwBFZgmM+VQnhNqqg2SbLEWD5HEg9Ypc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244413; c=relaxed/simple; bh=iewJ/UzRcwvim8mGh0VCVWZcnN1eKM0tebXGdS7Tqsw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bSzzLI0X8ahSTcQ6ahx3jatlrT4nXf6Dbxbrt5GuKmyZ4eK95OFgFnUe2xqe0iDn1RlDDuIOKw7HGy4Unk026agLLVkf+7xRKfsbUE0Ar/uyiq/yxMY2AblpdLkWm9WBEmbm/sulz/Ym4TDi7tqwksscmi57lWYxl61t3u0umPk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=OA0Wwjkr; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="OA0Wwjkr" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 982691F00898; Thu, 24 Sep 2026 10:06:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244400; bh=+jezVnsczHca2+N3DdTe1/ZWux4ulhsHRjwwcueBcvY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=OA0Wwjkrad2J01S7AVOXgv9vB7OxITzitM9cqnBHo/R+SMmR/Wb6JnEi/VOvV2Mr5 2GQ0T5KPCERTOQQyaKhjIMcAEoJCmXuPc/1IlVl4GqUieGyYG9M7bq9V9hdPzuGu75 zm5bZ6Vq9QKL5c2wtyNV07tR16E9ZCOrsDt/JCGwoBvTAiOwutEZ44qNObS1JXTIQm thqkWUr0JA9TdGwc9tUQRVAHzwZRsT4bDyZTeHuozNp3wPg4eqZQgmF/val0gVwhXN oIIrRZN/VaX3X3Ibu2efYFQqrrjq3udn0ykeO2gGCkLVxqnNNfkpFsX/NM1LKugnmA wsfiHOZHqec5A== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 03/14] irqchip/gic-v3-its: Resolve the default NUMA node explicitly Date: Thu, 24 Sep 2026 15:35:18 +0530 Message-ID: <20260924100529.1398790-4-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" alloc_pages_node() resolves NUMA_NO_NODE to numa_mem_id() internally. In preparation for switching ITS allocations to the CoCo shared memory allocator, resolve the default node explicitly in its_alloc_pages(). The CoCo shared memory allocator interprets NUMA_NO_NODE as a request to apply the current task's memory policy. Passing numa_mem_id() instead preserves the existing nearest-memory-node placement across the allocator switch. This patch does not change the current allocation behavior. Cc: Marc Zyngier Cc: Thomas Gleixner Cc: Radu Rendec Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/irqchip/irq-gic-v3-its.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index e9807af23537..ac5507acb1a0 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -32,6 +32,7 @@ #include #include #include +#include =20 #include #include @@ -236,7 +237,7 @@ static struct page *its_alloc_pages_node(int node, gfp_= t gfp, =20 static struct page *its_alloc_pages(gfp_t gfp, unsigned int order) { - return its_alloc_pages_node(NUMA_NO_NODE, gfp, order); + return its_alloc_pages_node(numa_mem_id(), gfp, order); } =20 static void its_free_pages(void *addr, unsigned int order) --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 52B073E44F3; Thu, 24 Sep 2026 10:06:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244425; cv=none; b=fVL0JJZGtElu8LyKD7aGYrGvIMUOTxDV3pahZQJnJrpqezqQSv3OFt+23+TwjXcmpqvs11ZRwIaA4fFcHnrQ1opNzkfbfXbZ2CRSBx2NcFFqznj+d7oFJ+dhYGmiZBqqhe4jAEQeYljOrhXmQ+6JnrmGkcHs2M3K9MUibRcZAcM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244425; c=relaxed/simple; bh=Z0tay+mZQxzzjyDmj//51K3kjDG5Z6SEp4+DBkaYiS8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YvImHReubClSm/68cZ1W04ZBm7DDVdxK13zZrOWrtc/kBmhTKjB+tfziiTjOJhcQ4kOs0U6S8943weBD423bNjVMKXV6eZdtQtM3TcF4FQ2Yixk0atrHDrXyyefoecxFk+YLviOQNi64Qy3OgkaKmYgo6QbB2lgOqH7YRIcKcP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MBbTPQiM; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MBbTPQiM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6FDE41F000FF; Thu, 24 Sep 2026 10:06:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244417; bh=waix5vjh+pKFV3Wslxy857438fq/HyMRB7bQ/yb+Wag=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=MBbTPQiMrIlhFnAQgodW4mh3L6x9pAnVoy9CpXDiD5kwwKVpSU3cW85m6YO3TllUe +8u1KpU1qq8ya8T0dvH9jh9swr5NkeWtJ0Jd25sXVXkLI4kJPPUC12YjRJqZRefRui Dr8TzQTF2X7fdYDA28ugxZlsWVSwiyDXVbEOHF18q78gXVUmWvnjBgJTLq7m5529Ex 8vXjEnYwjiugyhtg4AKJmlx9fPxgE3qm1L6Pf7xfudwJrSN6SlBAc1JbgB/SJzlNwz dtHD71eWILpn/kglR9wjXyVr+pY2jUtdefsLu9FHGEk86hCjUr/+mEpozltE+1KBYE +LiKQH7VzRRKA== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 04/14] irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory allocator Date: Thu, 24 Sep 2026 15:35:19 +0530 Message-ID: <20260924100529.1398790-5-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" ITS command queues and tables are shared with the host in a confidential-computing guest. The ITS allocator currently assumes that the guest PAGE_SIZE is sufficient for both allocation alignment and private/shared state changes. Route ITS page allocations through alloc_cc_shared_pages_node(). Keep the tracked table size based on the ITS-requested order. On release, reconstruct the common allocation metadata from the original ITS order and the architecture granule size so the same transitioned range and buddy order are used. Cc: Marc Zyngier Cc: Thomas Gleixner Cc: Radu Rendec Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/irqchip/irq-gic-v3-its.c | 40 ++++++++++++-------------------- 1 file changed, 15 insertions(+), 25 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index ac5507acb1a0..4814161df02e 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -19,7 +20,6 @@ #include #include #include -#include #include #include #include @@ -29,7 +29,6 @@ #include #include #include -#include #include #include #include @@ -214,25 +213,13 @@ static gfp_t gfp_flags_quirk; static struct page *its_alloc_pages_node(int node, gfp_t gfp, unsigned int order) { - struct page *page; - int ret =3D 0; - - page =3D alloc_pages_node(node, gfp | gfp_flags_quirk, order); - - if (!page) - return NULL; + struct cc_shared_pages mem; =20 - ret =3D set_memory_decrypted((unsigned long)page_address(page), - 1 << order); - /* - * If set_memory_decrypted() fails then we don't know what state the - * page is in, so we can't free it. Instead we leak it. - * set_memory_decrypted() will already have WARNed. - */ - if (ret) + if (alloc_cc_shared_pages_node(node, gfp | gfp_flags_quirk, + PAGE_SIZE << order, &mem)) return NULL; =20 - return page; + return mem.page; } =20 static struct page *its_alloc_pages(gfp_t gfp, unsigned int order) @@ -242,13 +229,15 @@ static struct page *its_alloc_pages(gfp_t gfp, unsign= ed int order) =20 static void its_free_pages(void *addr, unsigned int order) { - /* - * If the memory cannot be encrypted again then we must leak the pages. - * set_memory_encrypted() will already have WARNed. - */ - if (set_memory_encrypted((unsigned long)addr, 1 << order)) + struct cc_shared_layout layout; + struct cc_shared_pages mem; + + if (WARN_ON(cc_shared_calc_layout(PAGE_SIZE << order, &layout))) return; - free_pages((unsigned long)addr, order); + + mem.page =3D virt_to_page(addr); + mem.shared_size =3D layout.shared_size; + free_cc_shared_pages(&mem); } =20 static struct gen_pool *itt_pool; @@ -273,7 +262,8 @@ static void *itt_alloc_pool(int node, int size) if (!page) break; =20 - gen_pool_add(itt_pool, (unsigned long)page_address(page), PAGE_SIZE, nod= e); + gen_pool_add(itt_pool, (unsigned long)page_address(page), + cc_shared_granule_size(), node); } while (!addr); =20 return (void *)addr; --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2AB54718C3; Thu, 24 Sep 2026 10:07:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244446; cv=none; b=N8kxde95roQBpYEr/ugiWROA6NfGx++WDTqUGZUGv+IX0Nxe4J5gP8GJTC1+icOm10xZ/7YlZeT9Ebm9IyVPT0x8SV+4YRCTEye0APsQkOu0UBXAsVuw9SlQTwUZosoqH3cGA2uxb8O5WJ0yF7rwdbJC6HaRJ4ZF46rtS9P6y1Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244446; c=relaxed/simple; bh=5FjY2hxzRnVVGENuXbN0QPkVDmjXfmmSiENd3va9wqI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GNNP2xlljfwKoPIn2DYs/GNe2Ndc5vDtmH5ckgRoxmoU6AxYr74uMnTbdZimxyXYDzE3I74ymFvN2xS7DEll/TZigVBKAJkeiApaRKb4PYHUkhYPEmCKLYJnGeyE0r8rznpYrsuPOhF12E+Lxc1fEQ4ksl3vnWDEMGYcME6d8rM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=P4weohzL; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="P4weohzL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 471DA1F00898; Thu, 24 Sep 2026 10:06:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244434; bh=3VEiNyRElR21dM0ldw2tBNyPvjJVs1EiiOGwEtuW83w=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=P4weohzLy5CJIC6NuQMtaN228WRSyTv1qq2kfyS1SqVpAoivQBh8rbXTWcgdPM8Vm VkkcKKNe815WzpckDV2vo5sLg6oY90jZ+O7Gsbe2IyHqMt3ths8C3qowAtvY9pyuMb BLNm9sLUo2KCev6yRDGHrv2jlxYgCizHAZmwA0FgSr73VZ4nhceMq7E5IbgrJ9i855 NVMxhiRGI4wBX+25qxXp+4gTVOp/T+V6wF5ql7Pqx1cI/HG4Elq5MQ9PyHKdZUov1x Q6OiNyChps0dsTSqSw6oo9mcoRb5XaZdDEVCFf8DpX8VwhVWCRhURI9TR4YLA+7jTL bqdnABkbNAlrw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 05/14] dma-contiguous: Derive shared alignment from DMA attributes Date: Thu, 24 Sep 2026 15:35:20 +0530 Message-ID: <20260924100529.1398790-6-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Confidential-computing shared DMA allocations require CMA backing memory to be aligned to the architecture shared-memory granule. Passing explicit alignment orders through the DMA allocation stack exposes this derived constraint to callers and requires separate preferred and mandatory values. The DMA core already records whether an allocation needs shared backing in __DMA_ATTR_ALLOC_CC_SHARED. Pass allocation attributes to the DMA-contiguous entry points and derive the mandatory alignment there before calling CMA. Preserve the existing alignment clamping for ordinary allocations, while rejecting shared allocations whose granule exceeds CONFIG_CMA_ALIGNMENT so callers can fall back to the page allocator. Cc: Russell King Cc: Robin Murphy Cc: "Joerg Roedel (AMD)" Cc: Will Deacon Cc: Marek Szyprowski Cc: Andrew Morton Cc: Baoquan He Cc: Mike Rapoport Cc: Pasha Tatashin Cc: Pratyush Yadav Signed-off-by: Aneesh Kumar K.V (Arm) --- arch/arm/mm/dma-mapping.c | 5 +++-- drivers/iommu/dma-iommu.c | 2 +- include/linux/dma-map-ops.h | 9 ++++---- kernel/dma/contiguous.c | 41 +++++++++++++++++++++++++++---------- kernel/dma/direct.c | 2 +- kernel/dma/ops_helpers.c | 2 +- kernel/dma/pool.c | 2 +- kernel/kexec_file.c | 3 ++- 8 files changed, 44 insertions(+), 22 deletions(-) diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c index 7761099dde9e..9714fcd51941 100644 --- a/arch/arm/mm/dma-mapping.c +++ b/arch/arm/mm/dma-mapping.c @@ -398,7 +398,8 @@ static void *__alloc_from_contiguous(struct device *dev= , size_t size, struct page *page; void *ptr =3D NULL; =20 - page =3D dma_alloc_from_contiguous(dev, count, order, gfp & __GFP_NOWARN); + page =3D dma_alloc_from_contiguous(dev, count, order, 0, + gfp & __GFP_NOWARN); if (!page) return NULL; =20 @@ -866,7 +867,7 @@ static struct page **__iommu_alloc_buffer(struct device= *dev, size_t size, unsigned long order =3D get_order(size); struct page *page; =20 - page =3D dma_alloc_from_contiguous(dev, count, order, + page =3D dma_alloc_from_contiguous(dev, count, order, 0, gfp & __GFP_NOWARN); if (!page) goto error; diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 58c624513cd4..59baf2687612 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1627,7 +1627,7 @@ static void *iommu_dma_alloc_pages(struct device *dev= , size_t size, struct page *page =3D NULL; void *cpu_addr; =20 - page =3D dma_alloc_contiguous(dev, alloc_size, gfp); + page =3D dma_alloc_contiguous(dev, alloc_size, gfp, 0); if (!page) page =3D alloc_pages_node(node, gfp, get_order(alloc_size)); if (!page) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 8fae2b7deb20..f3490dd3f8ac 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -99,10 +99,11 @@ int __init dma_contiguous_reserve_area(phys_addr_t size= , phys_addr_t base, phys_addr_t limit, struct cma **res_cma, bool fixed); =20 struct page *dma_alloc_from_contiguous(struct device *dev, size_t count, - unsigned int order, bool no_warn); + unsigned int order, unsigned long attrs, bool no_warn); bool dma_release_from_contiguous(struct device *dev, struct page *pages, int count); -struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t g= fp); +struct page *dma_alloc_contiguous(struct device *dev, size_t size, + gfp_t gfp, unsigned long attrs); void dma_free_contiguous(struct device *dev, struct page *page, size_t siz= e); =20 void dma_contiguous_early_fixup(phys_addr_t base, unsigned long size); @@ -125,7 +126,7 @@ static inline int dma_contiguous_reserve_area(phys_addr= _t size, return -ENOSYS; } static inline struct page *dma_alloc_from_contiguous(struct device *dev, - size_t count, unsigned int order, bool no_warn) + size_t count, unsigned int order, unsigned long attrs, bool no_warn) { return NULL; } @@ -136,7 +137,7 @@ static inline bool dma_release_from_contiguous(struct d= evice *dev, } /* Use fallback alloc() and free() when CONFIG_DMA_CMA=3Dn */ static inline struct page *dma_alloc_contiguous(struct device *dev, size_t= size, - gfp_t gfp) + gfp_t gfp, unsigned long attrs) { return NULL; } diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 66093460584e..1a29565c9185 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -39,6 +39,7 @@ =20 #include =20 +#include #include #include #include @@ -357,19 +358,29 @@ int __init dma_contiguous_reserve_area(phys_addr_t si= ze, phys_addr_t base, * dma_alloc_from_contiguous() - allocate pages from contiguous area * @dev: Pointer to device for which the allocation is performed. * @count: Requested number of pages. - * @align: Requested alignment of pages (in PAGE_SIZE order). + * @align: Preferred alignment of pages (in PAGE_SIZE order). + * @attrs: DMA allocation attributes. * @no_warn: Avoid printing message about failed allocation. * * This function allocates memory buffer for specified device. It uses * device specific contiguous memory area if available or the default * global one. Requires architecture specific dev_get_cma_area() helper * function. + * + * The preferred alignment is capped at CONFIG_CMA_ALIGNMENT. The internal + * shared-allocation attribute requires at least the architecture shared + * granule alignment and fails if that exceeds the CMA alignment limit. */ struct page *dma_alloc_from_contiguous(struct device *dev, size_t count, - unsigned int align, bool no_warn) + unsigned int align, unsigned long attrs, bool no_warn) { - if (align > CONFIG_CMA_ALIGNMENT) - align =3D CONFIG_CMA_ALIGNMENT; + unsigned int required_align =3D 0; + + if (attrs & __DMA_ATTR_ALLOC_CC_SHARED) + required_align =3D get_order(cc_shared_granule_size()); + if (required_align > CONFIG_CMA_ALIGNMENT) + return NULL; + align =3D min(max(align, required_align), CONFIG_CMA_ALIGNMENT); =20 return cma_alloc(dev_get_cma_area(dev), count, align, no_warn); } @@ -390,10 +401,9 @@ bool dma_release_from_contiguous(struct device *dev, s= truct page *pages, return cma_release(dev_get_cma_area(dev), pages, count); } =20 -static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t = gfp) +static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t = gfp, + unsigned int align) { - unsigned int align =3D min(get_order(size), CONFIG_CMA_ALIGNMENT); - return cma_alloc(cma, size >> PAGE_SHIFT, align, gfp & __GFP_NOWARN); } =20 @@ -402,6 +412,7 @@ static struct page *cma_alloc_aligned(struct cma *cma, = size_t size, gfp_t gfp) * @dev: Pointer to device for which the allocation is performed. * @size: Requested allocation size. * @gfp: Allocation flags. + * @attrs: DMA allocation attributes. * * tries to use device specific contiguous memory area if available, or it * tries to use per-numa cma, if the allocation fails, it will fallback to @@ -412,8 +423,11 @@ static struct page *cma_alloc_aligned(struct cma *cma,= size_t size, gfp_t gfp) * there is no need to waste CMA pages for that kind; it also helps reduce * fragmentations. */ -struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t g= fp) +struct page *dma_alloc_contiguous(struct device *dev, size_t size, + gfp_t gfp, unsigned long attrs) { + unsigned int required_align =3D 0; + unsigned int align =3D get_order(size); #ifdef CONFIG_DMA_NUMA_CMA int nid =3D dev_to_node(dev); #endif @@ -421,8 +435,13 @@ struct page *dma_alloc_contiguous(struct device *dev, = size_t size, gfp_t gfp) /* CMA can be used only in the context which permits sleeping */ if (!gfpflags_allow_blocking(gfp)) return NULL; + if (attrs & __DMA_ATTR_ALLOC_CC_SHARED) + required_align =3D get_order(cc_shared_granule_size()); + if (required_align > CONFIG_CMA_ALIGNMENT) + return NULL; + align =3D min(max(align, required_align), CONFIG_CMA_ALIGNMENT); if (dev->cma_area) - return cma_alloc_aligned(dev->cma_area, size, gfp); + return cma_alloc_aligned(dev->cma_area, size, gfp, align); if (size <=3D PAGE_SIZE) return NULL; =20 @@ -431,7 +450,7 @@ struct page *dma_alloc_contiguous(struct device *dev, s= ize_t size, gfp_t gfp) struct cma *cma =3D dma_contiguous_numa_area[nid]; struct page *page; if (cma) { - page =3D cma_alloc_aligned(cma, size, gfp); + page =3D cma_alloc_aligned(cma, size, gfp, align); if (page) return page; } @@ -440,7 +459,7 @@ struct page *dma_alloc_contiguous(struct device *dev, s= ize_t size, gfp_t gfp) if (!dma_contiguous_default_area) return NULL; =20 - return cma_alloc_aligned(dma_contiguous_default_area, size, gfp); + return cma_alloc_aligned(dma_contiguous_default_area, size, gfp, align); } =20 /** diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 356d4e09e1c8..cb14419f8a09 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -124,7 +124,7 @@ static struct page *__dma_direct_alloc_pages(struct dev= ice *dev, size_t size, WARN_ON_ONCE(!PAGE_ALIGNED(size)); =20 gfp |=3D dma_direct_optimal_gfp_mask(dev, &phys_limit); - page =3D dma_alloc_contiguous(dev, size, gfp); + page =3D dma_alloc_contiguous(dev, size, gfp, 0); if (page) { if (dma_coherent_ok(dev, page_to_phys(page), size) && (allow_highmem || !PageHighMem(page))) diff --git a/kernel/dma/ops_helpers.c b/kernel/dma/ops_helpers.c index 6b5f9208d31c..8320cc0fada5 100644 --- a/kernel/dma/ops_helpers.c +++ b/kernel/dma/ops_helpers.c @@ -66,7 +66,7 @@ struct page *dma_common_alloc_pages(struct device *dev, s= ize_t size, struct page *page; phys_addr_t phys; =20 - page =3D dma_alloc_contiguous(dev, size, gfp); + page =3D dma_alloc_contiguous(dev, size, gfp, 0); if (!page) page =3D alloc_pages_node(dev_to_node(dev), gfp, get_order(size)); if (!page) diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c index 00f422a1e896..70b7f64b17ab 100644 --- a/kernel/dma/pool.c +++ b/kernel/dma/pool.c @@ -99,7 +99,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_po= ol, size_t pool_size, pool_size =3D 1 << (PAGE_SHIFT + order); if (cma_in_zone(gfp)) page =3D dma_alloc_from_contiguous(NULL, 1 << order, - order, false); + order, 0, false); if (!page) page =3D alloc_pages(gfp | __GFP_NOWARN, order); } while (!page && order-- > 0); diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index 59fb9d71e9d8..2a337ce7264e 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -682,7 +682,8 @@ static int kexec_alloc_contig(struct kexec_buf *kbuf) if (kbuf->image->type =3D=3D KEXEC_TYPE_CRASH) return -EPERM; =20 - p =3D dma_alloc_from_contiguous(NULL, nr_pages, get_order(kbuf->buf_align= ), true); + p =3D dma_alloc_from_contiguous(NULL, nr_pages, + get_order(kbuf->buf_align), 0, true); if (!p) return -ENOMEM; =20 --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD10C46D572; Thu, 24 Sep 2026 10:07:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244462; cv=none; b=QMOClEN6lOY3QksNd9d2d3P9x7wKo6UM+8B7TxudCTJQMZbeRpnVhToF2JJAjcJZqPOaY7Li2y/TKD/9RkBwbx6dMnvlb2k+23vnEeK56L+yGlcF7YNmUuQWsrNw45OBdJwaFaGdfsB/G7zSw8Q2OeO5FaXrAMtaKRRauvc/XOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244462; c=relaxed/simple; bh=iuU84skys+c4t0D22DM1dXxzWja6PMh9eOBC2aAUPFQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NpvfoPz7tNENcjoA95k70q10/AectT4hYEQJX6Q5lBhKt0Wg234NOueO8rN/e/XQFZej5aQ0C6jfvwHxqJkmlkxpuPUnwKXAgl8KZB3ozDlEnHPFH2jGp/letJV7eOv6cP+OU/rtrkZXDdkZCzYSSBAsZdxrlMMszFMzDIt7kcE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LuTvXw7z; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LuTvXw7z" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 22E981F000FF; Thu, 24 Sep 2026 10:07:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244451; bh=RaYIuvu7EWcx0kiHd4ySablfum8pI6JmFP40gOiNsBk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=LuTvXw7zqVNKQ4skqgavCDiVE8G/hbg2xCHYgmDtdbNSGuedcAsJXDvTuoJvyFscS EHVopMYcPM3szQABVqcGXam6pvpupTAuxZJ5OMb/ysSAaeYV5x2sPEAmgrokBcz319 dYSINIICwuHmGPzZo3FdmCMFjyVzDSjWZ7/OQPsSlFJtRW9vIYgiTi/RiOlSe+YkOQ a6oWvGFwJmhfYMS47NNaAbzFqfggPNWxhdZfbqWkcuua5X93I6ziIP4/QopgOCw9K3 JFpgHCo+ancimgLdB8OhxnkKJE1n9P4JeqUiS7vwniTX/Xfbe/7jZotMYGaxHKbxjB CGmucjMSnw+VQ== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 06/14] dma-pool: Allocate CoCo atomic pools using CoCo shared memory allocator Date: Thu, 24 Sep 2026 15:35:21 +0530 Message-ID: <20260924100529.1398790-7-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Atomic DMA allocations cannot perform a potentially sleeping private/shared state transition at allocation time. The atomic DMA pools avoid this by transitioning their backing allocations in a sleepable context and suballocating memory that is already shared. Architectures may require these transitions to use a shared granule size larger than PAGE_SIZE. The existing fallback loop can reduce the backing allocation below the order required by that size, producing a range that cannot be safely transitioned. For pools marked cc_shared, round the requested pool size up to a multiple of the shared granule size and prevent allocation fallback below the order required by that size. Pass __DMA_ATTR_ALLOC_CC_SHARED to dma_alloc_from_contiguous() so it enforces the mandatory alignment. If CMA cannot satisfy it, fall back to the buddy allocator. Non-shared pools pass zero attributes and retain the existing CMA alignment policy. Use the cc_make_shared() and cc_make_private() helpers to transition the backing allocation and validate its address and size. Individual atomic allocations may remain smaller than the shared granule size because the backing allocation remains owned by the pool and in the shared state. Cc: Marek Szyprowski Cc: Robin Murphy Assisted-by: Codex:gpt-5 Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/pool.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c index 70b7f64b17ab..f059312c15aa 100644 --- a/kernel/dma/pool.c +++ b/kernel/dma/pool.c @@ -4,12 +4,12 @@ * Copyright (C) 2020 Google LLC */ #include +#include #include #include #include #include #include -#include #include #include #include @@ -85,6 +85,9 @@ static bool cma_in_zone(gfp_t gfp) static int atomic_pool_expand(struct dma_gen_pool *dma_pool, size_t pool_s= ize, gfp_t gfp) { + struct cc_shared_layout layout; + unsigned long attrs =3D 0; + unsigned int min_order =3D 0; unsigned int order; struct page *page =3D NULL; bool leak_pages =3D false; @@ -92,6 +95,17 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_p= ool, size_t pool_size, int ret =3D -ENOMEM; pgprot_t prot __maybe_unused; =20 + if (dma_pool->cc_shared) { + ret =3D cc_shared_calc_layout(pool_size, &layout); + if (ret) + goto out; + pool_size =3D layout.shared_size; + min_order =3D get_order(layout.alignment); + if (min_order > MAX_PAGE_ORDER) + return -E2BIG; + attrs =3D __DMA_ATTR_ALLOC_CC_SHARED; + } + /* Cannot allocate larger than MAX_PAGE_ORDER */ order =3D min(get_order(pool_size), MAX_PAGE_ORDER); =20 @@ -99,10 +113,10 @@ static int atomic_pool_expand(struct dma_gen_pool *dma= _pool, size_t pool_size, pool_size =3D 1 << (PAGE_SHIFT + order); if (cma_in_zone(gfp)) page =3D dma_alloc_from_contiguous(NULL, 1 << order, - order, 0, false); + order, attrs, false); if (!page) page =3D alloc_pages(gfp | __GFP_NOWARN, order); - } while (!page && order-- > 0); + } while (!page && order-- > min_order); if (!page) goto out; =20 @@ -126,8 +140,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_= pool, size_t pool_size, * shrink so no re-encryption occurs in dma_direct_free(). */ if (dma_pool->cc_shared) { - ret =3D set_memory_decrypted((unsigned long)page_to_virt(page), - 1 << order); + ret =3D cc_make_shared(page_to_virt(page), pool_size); if (ret) { leak_pages =3D true; goto remove_mapping; @@ -144,7 +157,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_= pool, size_t pool_size, =20 encrypt_mapping: if (dma_pool->cc_shared && - set_memory_encrypted((unsigned long)page_to_virt(page), 1 << order)) + cc_make_private(page_to_virt(page), pool_size)) leak_pages =3D true; =20 remove_mapping: --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42CCF47140F; Thu, 24 Sep 2026 10:07:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244483; cv=none; b=X5XfKKfu8jSi0X8aZydPhTYxMHSo1pSRwRZmgn1ZOARKwkAcgofVqCfyZTZ19f9yy64BJs3Yaaxz/GeKI0xqyRkU3T3eBmPcAAXAWSpC+RoAkXTtQT+SBaaMuOHrOxoSEIp3Z5Ww4eKGB8cOL/H14tyTW4Pt58iKHogIsmYMSXc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244483; c=relaxed/simple; bh=wmWWe2U/E72QDG+2xlM7yuHMbAAZEIXrUVkI/yKGFq8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pVoMhBh1blRazxXbKvIwOLqt/GzC8p8M75eREa5Er+EMJNnkA1AmoWQlwjn/TL92EFvKzmG9JgD1NZPhF0LV7hDMiCMT5I96YoZF2gysfteIwda3JNEi26Hyaqe+V/gxivp/dO0XrLQ3FuSE8sr9yNvMr+n3uoFOm5JyCYNkLp0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hNxRqbRX; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hNxRqbRX" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F19E81F00893; Thu, 24 Sep 2026 10:07:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244468; bh=GJ0WVrMy94SvEspEtHLkpWdnYwmIG1CQN164cHPWINA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=hNxRqbRXCBgOSlhg7QIsbDMl/whpfkl0ji4BaBHpuxUxIl14Vx/ArJ7tNq0y1ndJF y9K5awNXnNh1KYw2+eWvCuc/PssjIkDyqaZWu4L877wC/jKUjqJ1vtwayk98/dUizt 3iMzqyg/Nr0Yi1NyIP5FNWknNNPLaEoaD4qY7DkTirbj7N+UPwQVnWdkefhMMilbyX qzg/oa0RKZLl0sPbOB816fgurHhifb89rdCCSvVAa8jEUMDiGPzlFZ1/Pr/Yn3kxKf +J7YuR28nAYg+9+WmWPWuGTOd4kD+JTXcvMadE+gkobA8U2AGqOoDVzjqNn7LAMkRu O52L6EXUAbO6Q== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 07/14] dma-direct: Align CoCo shared DMA allocations to the shared granule size Date: Thu, 24 Sep 2026 15:35:22 +0530 Message-ID: <20260924100529.1398790-8-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the common CoCo shared-memory geometry helpers for allocations backed directly by CMA or the page allocator. Round the backing allocation to a whole shared granule and pass the resolved shared-allocation attribute through the DMA contiguous allocator so it can enforce the mandatory alignment. Transition the complete range through the common shared/private helpers. Recompute the layout before freeing ordinary direct allocations so the transition back to private memory and dma_free_contiguous() cover exactly the range acquired by the allocation path. If restoring private state fails, retain the existing fail-safe behavior and leak the pages rather than returning potentially shared memory to the allocator. This also applies the same rules to dma_direct_alloc_pages(), covering callers which require a struct page result rather than a CPU virtual address. Cc: Marek Szyprowski Cc: Robin Murphy Assisted-by: Codex:gpt-5 Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/direct.c | 51 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 39 insertions(+), 12 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index cb14419f8a09..09a8a842fc6f 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -11,10 +11,10 @@ #include #include #include -#include #include #include #include +#include =20 #include "direct.h" =20 @@ -85,7 +85,7 @@ static int dma_set_decrypted(struct device *dev, void *va= ddr, size_t size) { int ret; =20 - ret =3D set_memory_decrypted((unsigned long)vaddr, PFN_UP(size)); + ret =3D cc_make_shared(vaddr, size); if (ret) pr_warn_ratelimited("leaking DMA memory that can't be decrypted\n"); return ret; @@ -95,7 +95,7 @@ static int dma_set_encrypted(struct device *dev, void *va= ddr, size_t size) { int ret; =20 - ret =3D set_memory_encrypted((unsigned long)vaddr, PFN_UP(size)); + ret =3D cc_make_private(vaddr, size); if (ret) pr_warn_ratelimited("leaking DMA memory that can't be re-encrypted\n"); return ret; @@ -115,7 +115,7 @@ static struct page *dma_direct_alloc_swiotlb(struct dev= ice *dev, size_t size, } =20 static struct page *__dma_direct_alloc_pages(struct device *dev, size_t si= ze, - gfp_t gfp, bool allow_highmem) + gfp_t gfp, bool allow_highmem, unsigned long attrs) { int node =3D dev_to_node(dev); struct page *page; @@ -124,7 +124,7 @@ static struct page *__dma_direct_alloc_pages(struct dev= ice *dev, size_t size, WARN_ON_ONCE(!PAGE_ALIGNED(size)); =20 gfp |=3D dma_direct_optimal_gfp_mask(dev, &phys_limit); - page =3D dma_alloc_contiguous(dev, size, gfp, 0); + page =3D dma_alloc_contiguous(dev, size, gfp, attrs); if (page) { if (dma_coherent_ok(dev, page_to_phys(page), size) && (allow_highmem || !PageHighMem(page))) @@ -184,7 +184,7 @@ static void *dma_direct_alloc_no_mapping(struct device = *dev, size_t size, { struct page *page; =20 - page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true); + page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true, 0); if (!page) return NULL; =20 @@ -205,6 +205,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, bool remap =3D false, set_uncached =3D false; bool mark_mem_decrypt =3D false; bool allow_highmem =3D true; + struct cc_shared_layout layout; struct page *page; void *cpu_addr; =20 @@ -285,8 +286,15 @@ void *dma_direct_alloc(struct device *dev, size_t size, return NULL; } =20 + if (mark_mem_decrypt) { + if (cc_shared_calc_layout(size, &layout)) + return NULL; + size =3D layout.shared_size; + } + /* we always manually zero the memory once we are done */ - page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, allow_hig= hmem); + page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, + allow_highmem, attrs); if (!page) return NULL; =20 @@ -305,7 +313,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, void *lm_addr; =20 lm_addr =3D page_address(page); - if (set_memory_decrypted((unsigned long)lm_addr, PFN_UP(size))) + if (dma_set_decrypted(dev, lm_addr, size)) goto out_leak_pages; } =20 @@ -364,6 +372,7 @@ void dma_direct_free(struct device *dev, size_t size, phys_addr_t phys; bool mark_mem_encrypted =3D false; struct io_tlb_pool *swiotlb_pool; + struct cc_shared_layout layout; unsigned int page_order =3D get_order(size); =20 /* @@ -408,6 +417,12 @@ void dma_direct_free(struct device *dev, size_t size, /* Swiotlb doesn't need a page attribute update on free */ mark_mem_encrypted =3D false; =20 + if (mark_mem_encrypted) { + if (WARN_ON_ONCE(cc_shared_calc_layout(size, &layout))) + return; + size =3D layout.shared_size; + } + if (is_vmalloc_addr(cpu_addr)) { vunmap(cpu_addr); } else { @@ -419,10 +434,8 @@ void dma_direct_free(struct device *dev, size_t size, void *lm_addr; =20 lm_addr =3D phys_to_virt(phys); - if (set_memory_encrypted((unsigned long)lm_addr, PFN_UP(size))) { - pr_warn_ratelimited("leaking DMA memory that can't be re-encrypted\n"); + if (dma_set_encrypted(dev, lm_addr, size)) return; - } } =20 if (swiotlb_pool) @@ -435,6 +448,7 @@ struct page *dma_direct_alloc_pages(struct device *dev,= size_t size, dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp) { unsigned long attrs =3D 0; + struct cc_shared_layout layout; struct page *page; void *cpu_addr; bool mark_mem_decrypt; @@ -458,7 +472,13 @@ struct page *dma_direct_alloc_pages(struct device *dev= , size_t size, goto setup_page; } =20 - page =3D __dma_direct_alloc_pages(dev, size, gfp, false); + if (mark_mem_decrypt) { + if (cc_shared_calc_layout(size, &layout)) + return NULL; + size =3D layout.shared_size; + } + + page =3D __dma_direct_alloc_pages(dev, size, gfp, false, attrs); if (!page) return NULL; =20 @@ -484,6 +504,7 @@ void dma_direct_free_pages(struct device *dev, size_t s= ize, phys_addr_t phys; void *vaddr =3D page_address(page); struct io_tlb_pool *swiotlb_pool; + struct cc_shared_layout layout; /* * if the device had requested for an unencrypted buffer, * convert it to encrypted on free @@ -500,6 +521,12 @@ void dma_direct_free_pages(struct device *dev, size_t = size, if (swiotlb_pool) mark_mem_encrypted =3D false; =20 + if (mark_mem_encrypted) { + if (WARN_ON_ONCE(cc_shared_calc_layout(size, &layout))) + return; + size =3D layout.shared_size; + } + if (mark_mem_encrypted && dma_set_encrypted(dev, vaddr, size)) return; =20 --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDDD63F7AB7; Thu, 24 Sep 2026 10:08:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244500; cv=none; b=qvp1JjJEpGj2V4ILKwnHTME/khsynGYM2tNf7VjJQ/37Pp+GynU6Xgc2IOteuVjuND7grLI4J6d/ulekJXa6kg5+6y9wIrYpCsnprzW4ikP4VwtsbYvSCs3Vop+kvePeeMiECxfB8h+RX/PZGjtcstI0wmSG94HPqeG2z3sLEWk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244500; c=relaxed/simple; bh=hZQBA8kosHWf25YA9gNguhFO63/KXQNjdgD9qVPciCs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O7AXusQotWPCsC+0eRRk6QtlClmkpiiwt6d/L4vUC714KGUo1AE+P8FuzqvGcXQRag0Er5AoA6fvQ+F4SRjf6VjTIRwW9SF1DUbOmJBMvtmZ0XOGt0dsOXr4A2+aHRCkErQzIACavOKRcqLwq9dJYh3xIyHab/QDpljMIIsd1XE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jOTRGxxD; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jOTRGxxD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C9C051F000FF; Thu, 24 Sep 2026 10:07:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244485; bh=BGXN0p+tMDUsztkVAtcaWVHfdiW84XCTWOSW1jg9k7A=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jOTRGxxDsAY98FX6mcu4pcKINkDtOF2FH4sRUbg6eYP/v2p6BML6vq3Z0bgTXXi5e oFJbrkUd1n9E6zqSQY5I4XSUWKrWJHeuetk4eiU9j1199VSorr+fLu76TU8bQyQsav rkF7W6bHvONtqYJWJX7RczBTKuXWd28xr8zw//G6Rouh/7u9LOQSqYQ+FFlsDcMc0U h3yQj4EHndV6oq6ey6hp5PXC6ozRqUSq7xqGus4NtabWlmfCvEOG2ULnPGGylMB+GA 7pNIaQFtyClmABKsj/ZUywPt/uW3D/DBGipMhTu6HWw4fQcQgXS/RSNIvBilY1mhKt ctP8w0bePzduw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 08/14] swiotlb: Align shared IO TLB pools to the shared granule size Date: Thu, 24 Sep 2026 15:35:23 +0530 Message-ID: <20260924100529.1398790-9-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Align and size the early memblock pool to the common CoCo shared granule. Use the same rounded extent when the pool is transitioned and when it is released. This deliberately leaves the slot count unchanged: any rounded tail belongs to the pool allocation but is not advertised as allocatable SWIOTLB space. Replace the default and dynamic pool set_memory calls with the helpers so alignment validation and architecture dispatch are kept in one place. As before, pages are intentionally leaked if their private state cannot be restored safely. Cc: Marek Szyprowski Cc: Robin Murphy Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/swiotlb.c | 40 +++++++++++++++++++++++----------------- 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index ded7016a46a7..394db93b7eb9 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -22,6 +22,7 @@ =20 #include #include +#include #include #include #include @@ -38,7 +39,6 @@ #include #include #include -#include #include #include #include @@ -369,13 +369,13 @@ void __init swiotlb_update_mem_attributes(void) =20 if (!mem->nslabs || mem->late_alloc) return; - bytes =3D PAGE_ALIGN(mem->nslabs << IO_TLB_SHIFT); + + bytes =3D ALIGN(mem->nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); =20 if (io_tlb_default_mem.cc_shared) { int ret; =20 - ret =3D set_memory_decrypted((unsigned long)mem->vaddr, - bytes >> PAGE_SHIFT); + ret =3D cc_make_shared(mem->vaddr, bytes); if (ret) { pr_warn("Failed to decrypt default memory pool, disabling it\n"); swiotlb_mark_pool_used(mem); @@ -436,8 +436,8 @@ static void __init *swiotlb_memblock_alloc(unsigned lon= g nslabs, unsigned int flags, int (*remap)(void *tlb, unsigned long nslabs)) { - size_t bytes =3D PAGE_ALIGN(nslabs << IO_TLB_SHIFT); void *tlb; + size_t bytes =3D ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); =20 /* * By default allocate the bounce buffer memory from low memory, but @@ -445,9 +445,9 @@ static void __init *swiotlb_memblock_alloc(unsigned lon= g nslabs, * memory encryption. */ if (flags & SWIOTLB_ANY) - tlb =3D memblock_alloc(bytes, PAGE_SIZE); + tlb =3D memblock_alloc(bytes, cc_shared_granule_size()); else - tlb =3D memblock_alloc_low(bytes, PAGE_SIZE); + tlb =3D memblock_alloc_low(bytes, cc_shared_granule_size()); =20 if (!tlb) { pr_warn("%s: Failed to allocate %zu bytes tlb structure\n", @@ -456,7 +456,7 @@ static void __init *swiotlb_memblock_alloc(unsigned lon= g nslabs, } =20 if (remap && remap(tlb, nslabs) < 0) { - memblock_free(tlb, PAGE_ALIGN(bytes)); + memblock_free(tlb, bytes); pr_warn("%s: Failed to remap %zu bytes\n", __func__, bytes); return NULL; } @@ -578,7 +578,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, swiotlb_adjust_nareas(num_possible_cpus()); =20 retry: - order =3D get_order(nslabs << IO_TLB_SHIFT); + order =3D get_order(ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size(= ))); nslabs =3D SLABS_PER_PAGE << order; =20 while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) { @@ -587,6 +587,8 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, if (vstart) break; order--; + if (order < get_order(cc_shared_granule_size())) + break; nslabs =3D SLABS_PER_PAGE << order; retried =3D true; } @@ -626,8 +628,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, goto error_slots; =20 if (io_tlb_default_mem.cc_shared) { - rc =3D set_memory_decrypted((unsigned long)vstart, - (nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT); + rc =3D cc_make_shared(vstart, nslabs << IO_TLB_SHIFT); if (rc) { leak_pages =3D true; goto error_decrypt; @@ -667,11 +668,11 @@ void __init swiotlb_exit(void) =20 pr_info("tearing down default memory pool\n"); tbl_vaddr =3D (unsigned long)phys_to_virt(mem->start); - tbl_size =3D PAGE_ALIGN(mem->end - mem->start); + tbl_size =3D ALIGN(mem->end - mem->start, cc_shared_granule_size()); slots_size =3D PAGE_ALIGN(array_size(sizeof(*mem->slots), mem->nslabs)); =20 if (io_tlb_default_mem.cc_shared) { - if (set_memory_encrypted(tbl_vaddr, tbl_size >> PAGE_SHIFT)) + if (cc_make_private((void *)tbl_vaddr, tbl_size)) leak_pages =3D true; } =20 @@ -711,12 +712,15 @@ void __init swiotlb_exit(void) static struct page *alloc_dma_pages(gfp_t gfp, size_t bytes, u64 phys_limit, unsigned long attrs) { - unsigned int order =3D get_order(bytes); bool cc_shared =3D attrs & __DMA_ATTR_ALLOC_CC_SHARED; + unsigned int order; struct page *page; phys_addr_t paddr; void *vaddr; =20 + if (cc_shared) + bytes =3D ALIGN(bytes, cc_shared_granule_size()); + order =3D get_order(bytes); page =3D alloc_pages(gfp, order); if (!page) return NULL; @@ -728,13 +732,13 @@ static struct page *alloc_dma_pages(gfp_t gfp, size_t= bytes, } =20 vaddr =3D phys_to_virt(paddr); - if (cc_shared && set_memory_decrypted((unsigned long)vaddr, PFN_UP(bytes)= )) + if (cc_shared && cc_make_shared(vaddr, bytes)) goto error; return page; =20 error: /* Intentional leak if pages cannot be encrypted again. */ - if (cc_shared && !set_memory_encrypted((unsigned long)vaddr, PFN_UP(bytes= ))) + if (cc_shared && !cc_make_private(vaddr, bytes)) __free_pages(page, order); return NULL; } @@ -807,9 +811,11 @@ static void swiotlb_free_tlb(void *vaddr, size_t bytes= , bool cc_shared) dma_free_from_pool(NULL, vaddr, bytes)) return; =20 + if (cc_shared) + bytes =3D ALIGN(bytes, cc_shared_granule_size()); /* Intentional leak if pages cannot be encrypted again. */ if (!cc_shared || - !set_memory_encrypted((unsigned long)vaddr, PFN_UP(bytes))) + !cc_make_private(vaddr, bytes)) __free_pages(virt_to_page(vaddr), get_order(bytes)); } =20 --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB95D471CFC; Thu, 24 Sep 2026 10:08:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244510; cv=none; b=Bq6Hzp6xGcwarGwjpwq1BQ+rmlk90c1cS/vskv66Tup7jF9LkQubidZJyC01//evum6SynEGFnNO1Q8uvtebbeiaesHRewfbu5w6KcX9JDnUca9FwsOL/zlYelLtUwlELneszulLEAnPqRVwtDjquxu/RwKzWdpMMxAdvySQCRk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244510; c=relaxed/simple; bh=EykRrBrl5pMvUZvna1SJssJT49OHh5BFqflNGuwHeb0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IiOhtBiVfx6l9IFjbUgIwK+Ot4s3XH9TWMLcFn1mczyYHf6iZSzmHI5hrKmGFjbrTtyKVUtbdpFNHc7EVPkfiS9t2KThpgQ8DgJizef6iAvO/z3QkGrd8Z8tbMG+HidQ8TqaE1c2G8jeLtHrAJtY4zFLXrgsA+hUcTj6Ta+ecRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=P5n6MGK2; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="P5n6MGK2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BAFC51F00893; Thu, 24 Sep 2026 10:08:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244502; bh=eJJYWMaIvtyGybZXxXutyXnVsQUWKbvJkU+PAxP2pXo=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=P5n6MGK29mMINRCHXIC8ydY+dR/BkV3ROd/aUdApCPS+tbFHhPY9xh+IR68I0PFZX qdeg4Edf9fGFKzHej5CvdNa2YRkY0pKwBFvYUy0sQDmi8swhQP3oWtFnfxTcXfbUHP 1ZlVi6klX6hVnRO31ygdNMLxpMEYjdZIeH894C1JnC9BS/PlnPfqVEt2z+3MFrEC2R PicLxLThzW/dl8ZVGDCk5J0DJiWSl7rspSCPEHVq1HnG2MDl3siNIpqesN4z5Q7Zbh ZHj+EnHjZ0NNkfeLftPM+pEIwR7vHerabIurntNfIjJLl15srVpl7Bxc2nciD4DSb7 9ifohJ9q8AY+w== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 09/14] swiotlb: Reject misaligned restricted DMA pools for CoCo guests Date: Thu, 24 Sep 2026 15:35:24 +0530 Message-ID: <20260924100529.1398790-10-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A restricted DMA pool is a firmware-described ownership boundary. Unlike a pool allocated by SWIOTLB itself, its base cannot be moved and its size cannot be rounded without claiming memory outside the reserved region. Confidential-computing guests may require private/shared state changes to cover units larger than PAGE_SIZE. Passing a misaligned restricted region to set_memory_decrypted() can therefore fail. Validate both the physical base and the complete reserved size with the common CoCo shared-range helper before allocating SWIOTLB metadata or performing a state transition. Perform the transition through cc_make_shared(). Cc: Marek Szyprowski Cc: Robin Murphy Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/swiotlb.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 394db93b7eb9..5f65b11260e0 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -2008,6 +2008,14 @@ static int rmem_swiotlb_device_init(struct reserved_= mem *rmem, return -EINVAL; } =20 + if (cc_platform_has(CC_ATTR_MEM_ENCRYPT) && + !cc_shared_range_valid(rmem->base, rmem->size)) { + dev_err(dev, + "Restricted DMA pool must be aligned to %#zx bytes for memory encryptio= n\n", + cc_shared_granule_size()); + return -EINVAL; + } + /* * Since multiple devices can share the same pool, the private data, * io_tlb_mem struct, will be initialized by the first device attached @@ -2041,8 +2049,8 @@ static int rmem_swiotlb_device_init(struct reserved_m= em *rmem, int ret; =20 mem->cc_shared =3D true; - ret =3D set_memory_decrypted((unsigned long)phys_to_virt(rmem->base), - rmem->size >> PAGE_SHIFT); + ret =3D cc_make_shared(phys_to_virt(rmem->base), + rmem->size); if (ret) { dev_err(dev, "Failed to decrypt restricted DMA pool\n"); kfree(pool->areas); --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38B00471CE7; Thu, 24 Sep 2026 10:08:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244538; cv=none; b=tdt5aBfGwRukxZ1XdnfM74DoB/iyN7AQBwqHjmGM0/pYd7yB28nJF7tr3Oce1P3GOs5mgX5rz0uZ5RPo+mvDN1go5FwVCRgZYcs8ImSnJTCvVDQadfJC/7wKg9OlYqZ4drucNAIq78qme+H3PzZUdSAsykgDcYJVEm3YOU5RPFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244538; c=relaxed/simple; bh=fez1QIY1xDqFbTon4QQ4lOambSvvvpt0uieqDGRC7kA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=M0DD7avtnKmBl4BWKUS8AO5wjYU7wvKAerBkBxCdp+BaNkDNC4ledLjYcWO7UWD3TaRksbQNmZhNrTjYY9qa+tKlXd1WW+w4fY6Le0o3VE5OvEvu0zwMhm0xUNKuf4bJRQXAOguVWKEI7LeR4SFC7oPwbEBbc+l5+j9SAPK7g/0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=h7vVYIE0; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="h7vVYIE0" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9AACF1F000FF; Thu, 24 Sep 2026 10:08:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244518; bh=wOq7lOU2FcK7bFNXejlGbOi1XU8BMJcPZcRxtw9e4Sw=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=h7vVYIE0aMAfRYhJEF3IrnQkTPB+l8URFB8a2ES5HlMFPDWYqoftWe2Ff8rVJSgHz //Op1zX/iwMgv9mXBkckDouFTPX0yIsuZxqyC0hSZbgJNOHGHh/SORbTcQbJNWXXo2 nUcxq5smZ+bkK8UdPl+zQcHG5xIFmbrqMpdGAiQ26FV2D4kvBW8Su/Mij/C3EDF8gK l4YYQMnmrMqIuikyKNc99VEKE2PCSbB7BLEPIu5WwybWevD7t2nNjyS/K4mLgpcVP/ /sv9QW2fUVc4LunoMvqqgAhjOq6/Ihho9u3wzSDnaqja06rEay4I3ZS+EXulKYaZK7 HA1XNHuabSvRw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 10/14] dma-buf: system_heap: Limit scatterlist entries to the buffer size Date: Thu, 24 Sep 2026 15:35:25 +0530 Message-ID: <20260924100529.1398790-11-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The system heap currently allocates each backing page no larger than the remaining dma-buf length. It can therefore use the complete compound-page size for every scatterlist entry while keeping the total length equal to the buffer size. Shared backing allocations may need to be rounded up to an architecture shared granule size. A backing allocation can then be larger than the remaining buffer length. Describing the complete allocation in the scatterlist would incorrectly expose the rounded tail to scatterlist consumers as part of the dma-buf. Track the remaining buffer length while constructing the scatterlist and limit each entry to the smaller of the compound-page size and the remaining length. The complete backing allocation remains owned by the heap and is still released normally. This does not change behavior with the current allocation policy, but prepares the heap for shared-granule-sized backing allocations. Cc: Sumit Semwal Cc: Benjamin Gaignard Cc: Brian Starkey Cc: John Stultz Cc: "T.J. Mercier" Cc: "Christian K=C3=B6nig" Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/dma-buf/heaps/system_heap.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/sy= stem_heap.c index c8959eadc71d..b5b8cdf65f23 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -406,6 +406,7 @@ static struct dma_buf *system_heap_allocate(struct dma_= heap *heap, struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); unsigned long size_remaining =3D len; + unsigned long sg_remaining =3D len; unsigned int max_order =3D orders[0]; struct system_heap_priv *priv =3D dma_heap_get_drvdata(heap); bool cc_shared =3D priv->cc_shared; @@ -454,7 +455,11 @@ static struct dma_buf *system_heap_allocate(struct dma= _heap *heap, =20 sg =3D table->sgl; list_for_each_entry_safe(page, tmp_page, &pages, lru) { - sg_set_page(sg, page, page_size(page), 0); + unsigned long sg_len; + + sg_len =3D min_t(unsigned long, page_size(page), sg_remaining); + sg_set_page(sg, page, sg_len, 0); + sg_remaining -=3D sg_len; sg =3D sg_next(sg); list_del(&page->lru); } --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C46D04734D0; Thu, 24 Sep 2026 10:08:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244556; cv=none; b=V7ef3nuWPCbyc5CIm+Bu1HA6Fo1IKLjJHFxxG+AIwr7UUcWNOKOegS53+00q8rlvcMNUPcKXd9KcKxpRnShEBEU8R/04I0Pg4Eunln+9jrn0tmH0n3uZyxMhCffWFs5D/EXd1UAjHyonHOHvet6P7qPd1QyypA/1AGS4Hqi55Vk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244556; c=relaxed/simple; bh=9T+jVTSN5hZQjYfq9qtnzx226IldZZujAZIyb4HEuQQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MhgryeU5PIX/wav1acnauvjJx3xuyfHwWUdZqaWTta3sGbH655dgQHNA+CHdGf8tdstv5JCXdjorgFtNEsqK83stvDPiVaLlEnQKseMFzOhf1APZ+D3596S10OPSzETpn2PVM4tHMFTxWa52iHb0WSzO5uaGio5EgpKvqC6MUGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UuUMyUgz; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UuUMyUgz" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70A8A1F00893; Thu, 24 Sep 2026 10:08:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244535; bh=/F4y/7yxyUDiC6xbvLu9ph4vBP4DM40ibzlyQTAOh0c=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=UuUMyUgzUZWtthyh61vaCwP7ZLcWxHojwv+uEizpKSKXauzTbqg6EVPm0mJDppj3X ztyjgCs08HO+QzCrg8WCWibDXFvPYpFUKjIYI4O6sAoOJr+8RltFEDWZuHR05Mmh3j drE4nAushbHuhZXqr7khJTBd9CfA7GwiknoMQ8yVPjKzfEYauZlDgzQk77n+jj0y3T q8sHWy4IAz2LpKhmDLPNDY5p9N9napTpJ2vyOkZCuS505Ogd89yBRjUeGWBX4ZwPsU cgg41AK1Qx1QJI002BGvsJUwlMCFUqXRoyHGKNpPql89lBdfad+a7TqNmPIQYO3YtP eNHWgKjV1jtWQ== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 11/14] dma-buf: system_heap: Allocate shared buffers using CoCo shared memory allocator Date: Thu, 24 Sep 2026 15:35:26 +0530 Message-ID: <20260924100529.1398790-12-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The system_cc_shared heap currently allocates its backing pages using the normal system-heap order policy and changes each resulting compound page to shared state. That is unsafe when an architecture requires state changes in units larger than PAGE_SIZE: an order-0 tail is neither sufficiently aligned nor large enough to transition independently. Use the common CoCo shared-page allocator for every backing allocation of t= he shared heap. Preserve the existing preferred-order search and its GFP polic= y: each candidate order is passed to the common allocator as a byte request, a= nd that allocator rounds it up when the architecture shared granule is larger. Add __GFP_COMP for shared allocations because an order-0 candidate can be rounded into a high-order allocation. The system heap uses compound_order() and page_size() for accounting and release, so the returned allocation must retain compound-page semantics. Calculate a rounded internal backing length but retain the original length in dma_buf::size. The preceding scatterlist-length change ensures that the rounded tail is not included in DMA mappings or other operations. This permits a 4 KiB request on a 64 KiB shared-granule system without exposing the extra 60 KiB to an importer. Cc: Sumit Semwal Cc: Benjamin Gaignard Cc: Brian Starkey Cc: John Stultz Cc: "T.J. Mercier" Cc: "Christian K=C3=B6nig" Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/dma-buf/heaps/system_heap.c | 121 ++++++++++++---------------- 1 file changed, 53 insertions(+), 68 deletions(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/sy= stem_heap.c index b5b8cdf65f23..5970b76416ae 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -11,14 +11,13 @@ */ =20 #include +#include #include #include #include #include #include -#include #include -#include #include #include #include @@ -65,34 +64,6 @@ static gfp_t order_flags[] =3D {HIGH_ORDER_GFP, HIGH_ORD= ER_GFP, LOW_ORDER_GFP}; static const unsigned int orders[] =3D {8, 4, 0}; #define NUM_ORDERS ARRAY_SIZE(orders) =20 -static int system_heap_set_page_decrypted(struct page *page) -{ - unsigned long addr =3D (unsigned long)page_address(page); - unsigned int nr_pages =3D 1 << compound_order(page); - int ret; - - ret =3D set_memory_decrypted(addr, nr_pages); - if (ret) - pr_warn_ratelimited("dma-buf system heap: failed to decrypt page at %p\n= ", - page_address(page)); - - return ret; -} - -static int system_heap_set_page_encrypted(struct page *page) -{ - unsigned long addr =3D (unsigned long)page_address(page); - unsigned int nr_pages =3D 1 << compound_order(page); - int ret; - - ret =3D set_memory_encrypted(addr, nr_pages); - if (ret) - pr_warn_ratelimited("dma-buf system heap: failed to re-encrypt page at %= p, leaking memory\n", - page_address(page)); - - return ret; -} - static int dup_sg_table(struct sg_table *from, struct sg_table *to) { struct scatterlist *sg, *new_sg; @@ -337,6 +308,20 @@ static void system_heap_vunmap(struct dma_buf *dmabuf,= struct iosys_map *map) iosys_map_clear(map); } =20 +static void system_heap_free_page(struct page *page, bool cc_shared) +{ + struct cc_shared_pages mem; + + if (!cc_shared) { + __free_pages(page, compound_order(page)); + return; + } + + mem.page =3D page; + mem.shared_size =3D page_size(page); + free_cc_shared_pages(&mem); +} + static void system_heap_dma_buf_release(struct dma_buf *dmabuf) { struct system_heap_buffer *buffer =3D dmabuf->priv; @@ -345,19 +330,8 @@ static void system_heap_dma_buf_release(struct dma_buf= *dmabuf) int i; =20 table =3D &buffer->sg_table; - for_each_sgtable_sg(table, sg, i) { - struct page *page =3D sg_page(sg); - - /* - * Intentionally leak pages that cannot be re-encrypted - * to prevent shared memory from being reused. - */ - if (cc_shared_buffer(buffer) && - system_heap_set_page_encrypted(page)) - continue; - - __free_pages(page, compound_order(page)); - } + for_each_sgtable_sg(table, sg, i) + system_heap_free_page(sg_page(sg), cc_shared_buffer(buffer)); sg_free_table(table); kfree(buffer); } @@ -375,22 +349,39 @@ static const struct dma_buf_ops system_heap_buf_ops = =3D { .release =3D system_heap_dma_buf_release, }; =20 +static struct page *system_heap_alloc_order(unsigned int order, + gfp_t flags, bool cc_shared) +{ + struct cc_shared_pages mem; + + if (!cc_shared) + return alloc_pages(flags, order); + + /* The shared granule can raise the actual allocation order. */ + flags |=3D __GFP_COMP; + if (alloc_cc_shared_pages(flags, PAGE_SIZE << order, &mem)) + return NULL; + + return mem.page; +} + static struct page *alloc_largest_available(unsigned long size, - unsigned int max_order) + unsigned int max_order, bool cc_shared) { struct page *page; - int i; gfp_t flags; + int i; =20 for (i =3D 0; i < NUM_ORDERS; i++) { if (size < (PAGE_SIZE << orders[i])) continue; if (max_order < orders[i]) continue; + flags =3D order_flags[i]; if (mem_accounting) flags |=3D __GFP_ACCOUNT; - page =3D alloc_pages(flags, orders[i]); + page =3D system_heap_alloc_order(orders[i], flags, cc_shared); if (!page) continue; return page; @@ -405,6 +396,7 @@ static struct dma_buf *system_heap_allocate(struct dma_= heap *heap, { struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); + struct cc_shared_layout layout; unsigned long size_remaining =3D len; unsigned long sg_remaining =3D len; unsigned int max_order =3D orders[0]; @@ -417,6 +409,16 @@ static struct dma_buf *system_heap_allocate(struct dma= _heap *heap, struct page *page, *tmp_page; int i, ret =3D -ENOMEM; =20 + if (cc_shared) { + int err; + + err =3D cc_shared_calc_layout(len, &layout); + if (err) + return ERR_PTR(err); + + size_remaining =3D layout.shared_size; + } + buffer =3D kzalloc_obj(*buffer); if (!buffer) return ERR_PTR(-ENOMEM); @@ -439,7 +441,8 @@ static struct dma_buf *system_heap_allocate(struct dma_= heap *heap, goto free_buffer; } =20 - page =3D alloc_largest_available(size_remaining, max_order); + page =3D alloc_largest_available(size_remaining, max_order, + cc_shared); if (!page) goto free_buffer; =20 @@ -464,14 +467,6 @@ static struct dma_buf *system_heap_allocate(struct dma= _heap *heap, list_del(&page->lru); } =20 - if (cc_shared_buffer(buffer)) { - for_each_sgtable_sg(table, sg, i) { - ret =3D system_heap_set_page_decrypted(sg_page(sg)); - if (ret) - goto free_pages; - } - } - /* create the dmabuf */ exp_info.exp_name =3D dma_heap_get_name(heap); exp_info.ops =3D &system_heap_buf_ops; @@ -486,22 +481,12 @@ static struct dma_buf *system_heap_allocate(struct dm= a_heap *heap, return dmabuf; =20 free_pages: - for_each_sgtable_sg(table, sg, i) { - struct page *p =3D sg_page(sg); - - /* - * Intentionally leak pages that cannot be re-encrypted - * to prevent shared memory from being reused. - */ - if (cc_shared_buffer(buffer) && - system_heap_set_page_encrypted(p)) - continue; - __free_pages(p, compound_order(p)); - } + for_each_sgtable_sg(table, sg, i) + system_heap_free_page(sg_page(sg), cc_shared); sg_free_table(table); free_buffer: list_for_each_entry_safe(page, tmp_page, &pages, lru) - __free_pages(page, compound_order(page)); + system_heap_free_page(page, cc_shared); kfree(buffer); =20 return ERR_PTR(ret); --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41D3C472F9D; Thu, 24 Sep 2026 10:09:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244569; cv=none; b=bJk/EY6i28z8puFMQMxY4XFoTU3ysTszplK9sS6YahQuHIVHASgyL488UlEUKCGxaDXCuZvbr0X6/vvqAf3oGSHKUVIRkcNlzR8KHzWiZ6H2uvb2vdJMlWlkc0b+SgQP1gq2Ny3vV/HBmDi/zNkIGxOUkMZRxRTTuMD3+3LUHIM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244569; c=relaxed/simple; bh=lDW6U1FnScxJsSNN6Pa4/AWXZj02eRphfqM01/LbbKM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jwiEswtdEa7ce70TcpVxFiRcjyjMytbhoq4bZsCUS207OAxSIQV2VPw4w2snn0BGbibUz2td4XQuZ9McEA5PqN0z5HWu96wAlI7zLeqzSY1pEce3XpSxIVNTu1Ih8sWEb13O/kHqt6vPaQfyeK9HpenMcuLuSNWiW7m0Wl3yiSM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AO84OOU6; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AO84OOU6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4E3321F00898; Thu, 24 Sep 2026 10:08:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244552; bh=FjLv3i6JJ122LJ7wUfvpGm0WJbMJkppUMfbaqHtvr8U=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=AO84OOU6N+eaEkLdLQBjyQ/t1HP1RG2S2enaWx2C+z4bHKjzh2wO9UHy+bjDug10m qqpDXV03jBDwD1QvXv6r7BGWk1YxbBM77iHGeE1PuNEz7eZq011ieT3+ptwXy1/01f mRcAhqBbBLGPtfoyZji6bRl6UCa4FPjUFvBuyBNLXNf0FljNqb86DW9nAs2mhaVKwt 71ieV5+ZFfHUU3s+hcb8awOchCzEpLPuRD44lMHFYnXqkdQOkNUfAYTSob85+18+sT uf5uYl+M706JqXG4oAF1584cr54KQZrL4+V5Q3r2oj5mY/ie1Afu34xkkrghnL8SU6 jX3DHt2ADqFTQ== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 12/14] swiotlb: Make rounded shared pool capacity allocatable Date: Thu, 24 Sep 2026 15:35:27 +0530 Message-ID: <20260924100529.1398790-13-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" CoCo shared memory may need to be allocated and transitioned in units larger than the requested object. Before this change, users handled the resulting capacity as follows: User Rounded capacity reused dma-buf system heap no DMA-direct no regular GIC tables no small GIC ITTs yes, through a gen_pool early SWIOTLB pool no late SWIOTLB pool yes persistent dynamic SWIOTLB no transient dynamic SWIOTLB no, one mapping only atomic DMA pools yes, through a gen_pool restricted SWIOTLB pool no additional padding Improve the early and persistent dynamic SWIOTLB pools. They already own and transition backing rounded to the shared granule size, and SWIOTLB is itself a suballocator. Advertise the rounded extent as slots, size the slot metadata to match. This makes the extra capacity available without reserving more backing memory. Keep transient dynamic pools unchanged. A transient pool belongs to one DMA mapping and is destroyed when that mapping is unmapped, so its spare backing cannot satisfy a later request without changing the lifetime model. Do not attempt the same optimization for dma-buf, DMA-direct or regular GIC objects. Those allocations have independent caller-visible sizes and lifetimes. Reusing their padding requires a shared-granule suballocator with reference counting, per-object mappings and accounting. Note: For the current 64 KiB CCA shared granule size, SWIOTLB pool sizes are already multiples of the 256 KiB IO_TLB segment size. Consequently, the rounding does not change any runtime values on current CCA systems. It instead makes the code express the intended invariant that pool metadata describes the complete shared-granule-aligned backing allocation. Cc: Marek Szyprowski Cc: Robin Murphy Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/swiotlb.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 5f65b11260e0..8d71f2e0fd85 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -330,6 +330,14 @@ static inline unsigned long nr_slots(u64 val) return DIV_ROUND_UP(val, IO_TLB_SIZE); } =20 +static unsigned long swiotlb_align_nslabs(unsigned long nslabs) +{ + unsigned long granule_nslabs; + + granule_nslabs =3D cc_shared_granule_size() >> IO_TLB_SHIFT; + return ALIGN(nslabs, granule_nslabs); +} + static void swiotlb_mark_pool_used(struct io_tlb_pool *pool) { unsigned long i; @@ -433,11 +441,12 @@ static void add_mem_pool(struct io_tlb_mem *mem, stru= ct io_tlb_pool *pool) } =20 static void __init *swiotlb_memblock_alloc(unsigned long nslabs, - unsigned int flags, + unsigned long *alloc_nslabs, unsigned int flags, int (*remap)(void *tlb, unsigned long nslabs)) { + unsigned long aligned_nslabs =3D swiotlb_align_nslabs(nslabs); + size_t bytes =3D aligned_nslabs << IO_TLB_SHIFT; void *tlb; - size_t bytes =3D ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); =20 /* * By default allocate the bounce buffer memory from low memory, but @@ -455,12 +464,13 @@ static void __init *swiotlb_memblock_alloc(unsigned l= ong nslabs, return NULL; } =20 - if (remap && remap(tlb, nslabs) < 0) { + if (remap && remap(tlb, aligned_nslabs) < 0) { memblock_free(tlb, bytes); pr_warn("%s: Failed to remap %zu bytes\n", __func__, bytes); return NULL; } =20 + *alloc_nslabs =3D aligned_nslabs; return tlb; } =20 @@ -473,6 +483,7 @@ void __init swiotlb_init_remap(bool addressing_limit, u= nsigned int flags, { struct io_tlb_pool *mem =3D &io_tlb_default_mem.defpool; unsigned long nslabs; + unsigned long alloc_nslabs; unsigned int nareas; size_t alloc_size; void *tlb; @@ -497,13 +508,14 @@ void __init swiotlb_init_remap(bool addressing_limit,= unsigned int flags, swiotlb_adjust_nareas(num_possible_cpus()); =20 nslabs =3D default_nslabs; - nareas =3D limit_nareas(default_nareas, nslabs); - while ((tlb =3D swiotlb_memblock_alloc(nslabs, flags, remap)) =3D=3D NULL= ) { + while ((tlb =3D swiotlb_memblock_alloc(nslabs, &alloc_nslabs, flags, + remap)) =3D=3D NULL) { if (nslabs <=3D IO_TLB_MIN_SLABS) return; nslabs =3D ALIGN(nslabs >> 1, IO_TLB_SEGSIZE); - nareas =3D limit_nareas(nareas, nslabs); } + nslabs =3D alloc_nslabs; + nareas =3D limit_nareas(default_nareas, nslabs); =20 if (default_nslabs !=3D nslabs) { pr_info("SWIOTLB bounce buffer size adjusted %lu -> %lu slabs", @@ -869,6 +881,12 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct d= evice *dev, tlb_size =3D nslabs << IO_TLB_SHIFT; } =20 + /* Transient pools are tied to one mapping and cannot reuse padding. */ + if (mem->cc_shared && !dev) { + nslabs =3D swiotlb_align_nslabs(nslabs); + tlb_size =3D nslabs << IO_TLB_SHIFT; + } + slot_order =3D get_order(array_size(sizeof(*pool->slots), nslabs)); pool->slots =3D (struct io_tlb_slot *) __get_free_pages(gfp, slot_order); --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6ADF945A2B4; Thu, 24 Sep 2026 10:09:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244581; cv=none; b=HOMAiEdW/W5o5C0r2eL5hYzaEu0DoQGeaHeELnuUF885aireV0+wa0qe2FH3VeYhMdAFYSvnw5hD0CFRt9/kP1JFimxuFgzcKk2yVW3jHWuMpomYlisTCH243FlPDl8ewEBAdWBLNTi0ECgXTmNjLavGrHlz5awiw9MP9J+4vTI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244581; c=relaxed/simple; bh=avIOrMrcCb3ADh7Va+Vlx0jLbixEvoTxHwR3rEyIMcs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rSVjuoNDvEAk0kg52VcDDUQs77g3mzX2WmG6nxJsLuCGxYJhANlIMEd55as8zQi914Oyx8bdn8Nm1BJ9khEFozyZe5hrA1oK8y78tubbd8mSr7RgOc7J5s0w7Jgfoq5pGKcDGi2Hw7xFIipNFfpfiEmJ6JoLQdwgS4aoToSjP0U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dnSLogll; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dnSLogll" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 507301F000FF; Thu, 24 Sep 2026 10:09:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244569; bh=T/L8GH36WphB4FWEap/OQWo2XNq0RD33mRMiQ8zqE3c=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=dnSLogllfOwGPjiiYgYeYNLUbV922KBGPQBr1iWfsc0tB53dUpUKxf7Bdtcpcr3bI N/yMdNt2Ov0UkVr3MhgwO2+hKGw0VJvasN8PtaGsg7DTl5Ql1ufzTr95NdJDCRfBOD wRMGS47TdnCWOSqXI+wAq/TqxOS5zw8/ZLyamBfgIKhWLJU98AI0JXzOWvht07ccMo 2JB/877rqVQmvUlpiMOQYNo009UWntXGbqTXDczrJzMPMIo32vZaRkqz6B5foRNisc tn0L4hnJrri/j4++dOckyGH9o5Ual1j5zyEjqvpx9L8hqj3USKaRp7UJIrXQt3YkOi q/ADwv496KpUg== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 13/14] mm: Assert CoCo shared allocations may sleep Date: Thu, 24 Sep 2026 15:35:28 +0530 Message-ID: <20260924100529.1398790-14-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" On platforms with memory encryption, alloc_cc_shared_pages_node() transitions the allocated memory to the shared state. set_memory_decrypted() may sleep, so the transition must run from a sleepable context. Cc: Andrew Morton Signed-off-by: Aneesh Kumar K.V (Arm) --- mm/cc_shared.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/cc_shared.c b/mm/cc_shared.c index 9eb79832a702..6f68a3f44ef8 100644 --- a/mm/cc_shared.c +++ b/mm/cc_shared.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -106,6 +107,8 @@ static int __alloc_cc_shared_pages_node(int nid, gfp_t = gfp, unsigned int order; int ret; =20 + might_sleep(); + ret =3D cc_shared_calc_layout(requested, &layout); if (ret) return ret; --=20 2.43.0 From nobody Thu Sep 24 12:02:34 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E95A47427E; Thu, 24 Sep 2026 10:09:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244602; cv=none; b=TZ27mzRHrm5Sw4jtmbn3FH/3gimD80/a4OfPZMKL6+XPSnO/6sG8T1absytrT0W0knmDZl3K7pVx5hiR4nhIeUK7p363j57e31+Wn74kKnvalyQSd53Hec4aoVTN9e2upPeS0XnrWnm2tyOCLx2Iu439jsTVTA46A3bRxwbkRuI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790244602; c=relaxed/simple; bh=a5wZVDxozWcGbA+/SuJ6Ip5ryygvsJtEYlgQ+RR1o0A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZfbWfPeHGvJC/4nSOP2eHVteIG7Rc8+LvHf0kwpvGRlmR07tLEroHnCtJe2jFQQ50Wl/EpVhiaSBMCddHZeyjlGR5qtIHzKTZb3OWd2zssdP8SbHnFFRxj2ilmx9hV/6SGGJGFXLJxxlTODEh02SO8xYuxJxsRhDpSbqRrRZr8A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Hn9hv8eW; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Hn9hv8eW" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 26EA21F00893; Thu, 24 Sep 2026 10:09:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790244586; bh=dwj3iE+AENM1wdyeAW0ucsrV1xoTqhkmjLwoH3q3QAw=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Hn9hv8eWpaJ0p+isry01mjUR+t12CaIbyEhnYYrvyrYgoLrjQYYfgPDJQdaSjfaPh 50aE3RW/t7UTODmesgPWy0ykAx61CbS4D8dbfsypnco6ojqRMUwF5OtdYDnTstc9Iz 8k6nZlY00DBlv3G/7lbhrvb9n8+bCjYSXakBrGqK4DtvHWl2lG8/1uIwzhqKepwQqd DHbK/opblE1OlSJEblhsQ8zPACxiJ3fuzRt6fdYGbBz++jf6Za8pckAUtcXcqECLqB AVrLitP9+Im6vOn8+550xQCIMvJHFC3d7TSv1eybV24j9U20ScblwOD7FdGj1Kf6sc NRGOq4IPnJXrQ== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Baoquan He , Mike Rapoport , Pasha Tatashin , Pratyush Yadav , Catalin Marinas , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , Russell King , Benjamin Gaignard , Brian Starkey , John Stultz , Mark Rutland , Radu Rendec , "T.J. Mercier" , Madhavan Srinivasan , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Ritesh Harjani , Shrikanth Hegde , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Kiryl Shutsemau , Rick Edgecombe , "K . Y . Srinivasan" , Haiyang Zhang , Wei Liu , Dexuan Cui , Long Li , Paolo Bonzini , Vitaly Kuznetsov , Andy Lutomirski , Peter Zijlstra , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v8 14/14] irqchip/gic-v3-its: Preallocate VPE L1 tables Date: Thu, 24 Sep 2026 15:35:29 +0530 Message-ID: <20260924100529.1398790-15-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260924100529.1398790-1-aneesh.kumar@kernel.org> References: <20260924100529.1398790-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" VPE L1 tables are allocated from the CPU-starting callback, where sleeping = is not allowed. Remove the page allocation from this atomic path. Use a blocking allocation for the boot CPU and record the resulting table order. Allocate a page of that order from the sleepable CPU hotplug prepare stage for each secondary CPU. Consume the preallocated page only when the = CPU cannot inherit an existing redistributor or ITS table. Reclaim unused preallocations after the CPU reaches the online state, and f= rom the prepare-state teardown when CPU bring-up aborts. Assisted-by: Codex:gpt-5 Cc: Marc Zyngier Cc: Thomas Gleixner Cc: Radu Rendec Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/irqchip/irq-gic-v3-its.c | 110 +++++++++++++++++++++++++++-- drivers/irqchip/irq-gic-v3.c | 4 +- include/linux/irqchip/arm-gic-v3.h | 3 +- 3 files changed, 108 insertions(+), 9 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index 4814161df02e..a8f37fb97845 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -195,6 +195,7 @@ static DEFINE_PER_CPU(struct cpu_lpi_count, cpu_lpi_cou= nt); static LIST_HEAD(its_nodes); static DEFINE_RAW_SPINLOCK(its_lock); static struct rdists *gic_rdists; +static unsigned int vpe_l1_prealloc_order =3D UINT_MAX; static struct irq_domain *its_parent; =20 static unsigned long its_list_map; @@ -2884,12 +2885,35 @@ static bool allocate_vpe_l2_table(int cpu, u32 id) return true; } =20 -static int allocate_vpe_l1_table(void) +static unsigned int vpe_l1_table_order(u64 val) +{ + unsigned int psz; + u64 npg; + + switch (FIELD_GET(GICR_VPROPBASER_4_1_PAGE_SIZE, val)) { + default: + case GIC_PAGE_SIZE_4K: + psz =3D SZ_4K; + break; + case GIC_PAGE_SIZE_16K: + psz =3D SZ_16K; + break; + case GIC_PAGE_SIZE_64K: + psz =3D SZ_64K; + break; + } + + npg =3D FIELD_GET(GICR_VPROPBASER_4_1_SIZE, val) + 1; + return get_order(npg * psz); +} + +static int allocate_vpe_l1_table(bool use_prealloc) { void __iomem *vlpi_base =3D gic_data_rdist_vlpi_base(); u64 val, gpsz, npg, pa; unsigned int psz =3D SZ_64K; unsigned int np, epp, esz; + unsigned int order; struct page *page; =20 if (!gic_rdists->has_rvpeid) @@ -2982,7 +3006,16 @@ static int allocate_vpe_l1_table(void) =20 pr_debug("np =3D %d, npg =3D %lld, psz =3D %d, epp =3D %d, esz =3D %d\n", np, npg, psz, epp, esz); - page =3D its_alloc_pages(GFP_ATOMIC | __GFP_ZERO, get_order(np * PAGE_SIZ= E)); + order =3D get_order(np * PAGE_SIZE); + if (use_prealloc) { + if (WARN_ON_ONCE(order !=3D vpe_l1_prealloc_order)) + return -EINVAL; + + page =3D gic_data_rdist()->vpe_l1_prealloc; + gic_data_rdist()->vpe_l1_prealloc =3D NULL; + } else { + page =3D its_alloc_pages(GFP_KERNEL | __GFP_ZERO, order); + } if (!page) return -ENOMEM; =20 @@ -2999,6 +3032,9 @@ static int allocate_vpe_l1_table(void) val |=3D GICR_VPROPBASER_4_1_VALID; =20 out: + if (!use_prealloc) + vpe_l1_prealloc_order =3D vpe_l1_table_order(val); + gicr_write_vpropbaser(val, vlpi_base + GICR_VPROPBASER); cpumask_set_cpu(smp_processor_id(), gic_data_rdist()->vpe_table_mask); =20 @@ -3138,7 +3174,7 @@ static u64 its_clear_vpend_valid(void __iomem *vlpi_b= ase, u64 clr, u64 set) return val; } =20 -static void its_cpu_init_lpis(void) +static void its_cpu_init_lpis(bool use_prealloc) { void __iomem *rbase =3D gic_data_rdist_rd_base(); struct page *pend_page; @@ -3251,7 +3287,7 @@ static void its_cpu_init_lpis(void) val =3D its_clear_vpend_valid(vlpi_base, 0, 0); } =20 - if (allocate_vpe_l1_table()) { + if (allocate_vpe_l1_table(use_prealloc)) { /* * If the allocation has failed, we're in massive trouble. * Disable direct injection, and pray that no VM was @@ -5422,7 +5458,7 @@ static int redist_disable_lpis(void) return 0; } =20 -int its_cpu_init(void) +int its_cpu_init(bool use_prealloc) { if (!list_empty(&its_nodes)) { int ret; @@ -5431,13 +5467,66 @@ int its_cpu_init(void) if (ret) return ret; =20 - its_cpu_init_lpis(); + its_cpu_init_lpis(use_prealloc); its_cpu_init_collections(); } =20 return 0; } =20 +static int its_vpe_l1_prepare(unsigned int cpu) +{ + struct page **prealloc; + + if (!gic_rdists->has_rvpeid || + vpe_l1_prealloc_order =3D=3D UINT_MAX || + (gic_data_rdist_cpu(cpu)->flags & RD_LOCAL_LPI_ENABLED)) + return 0; + + prealloc =3D &gic_data_rdist_cpu(cpu)->vpe_l1_prealloc; + if (*prealloc) + return 0; + + *prealloc =3D its_alloc_pages_node(cpu_to_node(cpu), + GFP_KERNEL | __GFP_ZERO, + vpe_l1_prealloc_order); + return *prealloc ? 0 : -ENOMEM; +} + +static int its_vpe_l1_cleanup(unsigned int cpu) +{ + struct page *page; + + page =3D xchg(&gic_data_rdist_cpu(cpu)->vpe_l1_prealloc, NULL); + if (page) + its_free_pages(page_address(page), vpe_l1_prealloc_order); + + return 0; +} + +static int __init its_vpe_l1_cpuhp_init(void) +{ + int prepare_state, state; + + state =3D cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN, + "irqchip/arm/gicv3-vpe:prepare", + its_vpe_l1_prepare, + its_vpe_l1_cleanup); + if (state < 0) + return state; + prepare_state =3D state; + + state =3D cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, + "irqchip/arm/gicv3-vpe:online", + its_vpe_l1_cleanup, NULL); + if (state < 0) { + cpuhp_remove_state_nocalls(prepare_state); + return state; + } + + return 0; +} + static void rdist_memreserve_cpuhp_cleanup_workfn(struct work_struct *work) { cpuhp_remove_state_nocalls(gic_rdists->cpuhp_memreserve_state); @@ -5862,6 +5951,15 @@ int __init its_init(struct fwnode_handle *handle, st= ruct rdists *rdists, if (WARN_ON(!has_v4_1 && rdists->has_rvpeid)) rdists->has_rvpeid =3D false; =20 + if (rdists->has_rvpeid) { + err =3D its_vpe_l1_cpuhp_init(); + if (err) { + rdists->has_rvpeid =3D false; + rdists->has_vlpis =3D false; + pr_err("ITS: Failed to prepare VPE tables, disabling GICv4 support\n"); + } + } + if (has_v4 & rdists->has_vlpis) { const struct irq_domain_ops *sgi_ops; =20 diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 6e1fa5b247fc..e2e00372dcfb 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -1320,7 +1320,7 @@ static int gic_starting_cpu(unsigned int cpu) gic_cpu_init(); =20 if (gic_dist_supports_lpis()) - its_cpu_init(); + its_cpu_init(true); =20 return 0; } @@ -2053,7 +2053,7 @@ static int __init gic_init_bases(phys_addr_t dist_phy= s_base, =20 if (gic_dist_supports_lpis()) { its_init(handle, &gic_data.rdists, gic_data.domain, dist_prio_irq); - its_cpu_init(); + its_cpu_init(false); its_lpi_memreserve_init(); } else { if (IS_ENABLED(CONFIG_ARM_GIC_V2M)) diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm= -gic-v3.h index ea5fd2374ebe..a5565a745100 100644 --- a/include/linux/irqchip/arm-gic-v3.h +++ b/include/linux/irqchip/arm-gic-v3.h @@ -621,6 +621,7 @@ struct rdists { u64 flags; cpumask_t *vpe_table_mask; void *vpe_l1_base; + struct page *vpe_l1_prealloc; } __percpu *rdist; phys_addr_t prop_table_pa; void *prop_table_va; @@ -637,7 +638,7 @@ struct rdists { struct irq_domain; struct fwnode_handle; int __init its_lpi_memreserve_init(void); -int its_cpu_init(void); +int its_cpu_init(bool use_prealloc); int its_init(struct fwnode_handle *handle, struct rdists *rdists, struct irq_domain *domain, u8 irq_prio); int mbi_init(struct fwnode_handle *fwnode, struct irq_domain *parent); --=20 2.43.0