From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2169C2DC783; Mon, 21 Sep 2026 14:49:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002154; cv=none; b=OHwpOBva7IziTB419E8ITjF9BRvmhEHdCTfLWCFs0lbNCueakEWVNSGwRYsHBgL/DihBVyw+Yj0PhmVcED4jsR1EW3vinfLTIQoHTz+MEeXfAdsSdEYSMTqAxGvzv9bsQ1kgsX4TNFsQFiAJHo28gI4M1/1H3TkKb31A2NRbotM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002154; c=relaxed/simple; bh=ifD/kJ4/+PxNKv2UC8zL5w7iehZmNr46IJQAfLbWwGs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j5WaxtlVAFBUXBPutdrUuycWg9XFKzaqF/drtV6eHVG7vxZKDf8CRmZ1CZRdBhkWDwuPTgplY/ziBc3/wFUbOCKqf3d9pLkE6T220Ha7CO0IBrDjTwZ92NxbZ2S0c8HNjfIR3L9AqDs7aedV/0ZRw/R8aBdEh8k5ef7HRTj4PjE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Y+kLs6SQ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y+kLs6SQ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8793A1F00893; Mon, 21 Sep 2026 14:49:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002152; bh=JdzsdQpKyf1oi4MN56R5Qf9hEJiO6vl06z2ni7VOm34=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Y+kLs6SQLu85HXfzZNOLdVmzLtG8mzYHcUSGiSn1iF44uWoScyqBbhGJ5r0nsK/ma FIK15tCOZoLUHS+PwSHzIoVNZTS+z+H9j+yHajmGDZLJeT5W++foKiWJFE27qu/Kvi TbLBk+PmZ6h/KYLkO/NGiXwmBZc2fy/RXSarlfUmbGTkZFi5PWpPHwLO4U66CAKAK1 jspex32vrRlNo8fuaOWIMQeW23gztZ9EXJCKytESjUn74vf1svigMLszelQIvmvxrU qZOf+y8QRisgQbX0dfD0v46buL7LDd9OESNuOzlcfExGOhpDBIF4A9TEQW4rUpzpaZ lS+nUdoKGfTdw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 01/13] arm64: realm: Add RHI helper to query IPA state change alignment Date: Mon, 21 Sep 2026 20:18:35 +0530 Message-ID: <20260921144847.501151-2-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Arm CCA guests can run with a PAGE_SIZE that is smaller than the granule size used by the host to track Realm IPA state. In that configuration, changing a guest PAGE_SIZE range between protected and shared state can cover only part of the host-owned state-change unit. The Realm Host Interface specification (Arm DEN0148) [1] defines the Host Configuration interface for querying host capabilities, including the IPA state change alignment. Add the RHI HostConf definitions and a get_ipa_state_change_alignment() helper. The helper uses RSI_HOST_CALL to query the supported HostConf version and features, and reads the IPA state change alignment when available. It falls back to PAGE_SIZE if the interface is unavailable or returns an invalid value. [1] https://support.arm.com/documentation/den0148/latest/ Signed-off-by: Aneesh Kumar K.V (Arm) --- MAINTAINERS | 1 + drivers/firmware/arm_rmm/rsi.c | 47 ++++++++++++++++++++++++++++++++++ include/linux/arm-rsi-cmds.h | 10 ++++++++ include/linux/arm-smccc-rhi.h | 25 ++++++++++++++++++ include/linux/arm-smccc-rsi.h | 7 +++++ 5 files changed, 90 insertions(+) create mode 100644 include/linux/arm-smccc-rhi.h diff --git a/MAINTAINERS b/MAINTAINERS index 1e543163a8cc..87ec876868eb 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3956,6 +3956,7 @@ F: drivers/firmware/arm_rmm/ F: drivers/virt/coco/arm-cca-guest/ F: drivers/virt/coco/pkvm-guest/ F: include/linux/arm-rsi-cmds.h +F: include/linux/arm-smccc-rhi.h F: include/linux/arm-smccc-rsi.h F: tools/testing/selftests/arm64/ X: arch/arm64/boot/dts/ diff --git a/drivers/firmware/arm_rmm/rsi.c b/drivers/firmware/arm_rmm/rsi.c index 0fcc4e33bd92..883f6091ff95 100644 --- a/drivers/firmware/arm_rmm/rsi.c +++ b/drivers/firmware/arm_rmm/rsi.c @@ -8,7 +8,9 @@ #include #include #include +#include #include +#include #include =20 #include @@ -162,6 +164,51 @@ static int realm_register_memory_enc_ops(void) return arm64_mem_crypt_ops_register(&realm_crypt_ops); } =20 +/* we need an aligned struct for rsi_host_call. slab is not yet ready */ +static struct rsi_host_call hostconf_call __initdata; +static unsigned long __maybe_unused __init get_ipa_state_change_alignment(= void) +{ + long ret; + unsigned long shared_granule_size; + + memset(&hostconf_call, 0, sizeof(hostconf_call)); + hostconf_call.gprs[0] =3D RHI_HOSTCONF_VERSION; + ret =3D rsi_host_call(lm_alias(&hostconf_call)); + if (ret !=3D RSI_SUCCESS) + goto err_out; + + if (hostconf_call.gprs[0] !=3D RHI_HOSTCONF_VER_1_0) + goto err_out; + + memset(&hostconf_call, 0, sizeof(hostconf_call)); + hostconf_call.gprs[0] =3D RHI_HOSTCONF_FEATURES; + ret =3D rsi_host_call(lm_alias(&hostconf_call)); + if (ret !=3D RSI_SUCCESS) + goto err_out; + + if (!(hostconf_call.gprs[0] & RHI_HOSTCONF_FEATURE_GET_IPA_CHANGE_ALIGNME= NT)) + goto err_out; + + memset(&hostconf_call, 0, sizeof(hostconf_call)); + hostconf_call.gprs[0] =3D RHI_HOSTCONF_GET_IPA_CHANGE_ALIGNMENT; + ret =3D rsi_host_call(lm_alias(&hostconf_call)); + if (ret !=3D RSI_SUCCESS) + goto err_out; + + shared_granule_size =3D hostconf_call.gprs[0]; + if (shared_granule_size & (SZ_4K - 1) || + !is_power_of_2(shared_granule_size)) + goto err_out; + + return max(PAGE_SIZE, shared_granule_size); +err_out: + /* + * For failure condition assume host is built with 4K page size + * and hence IPA state change alignment can be guest PAGE_SIZE. + */ + return PAGE_SIZE; +} + void __init arm64_rsi_init(void) { if (arm_smccc_1_1_get_conduit() !=3D SMCCC_CONDUIT_SMC) diff --git a/include/linux/arm-rsi-cmds.h b/include/linux/arm-rsi-cmds.h index 3f7a6a833993..996f1621b996 100644 --- a/include/linux/arm-rsi-cmds.h +++ b/include/linux/arm-rsi-cmds.h @@ -236,4 +236,14 @@ static inline unsigned long rsi_attestation_token_cont= inue(phys_addr_t granule, return res.a0; } =20 +static inline unsigned long rsi_host_call(struct rsi_host_call *rhi_call) +{ + phys_addr_t addr =3D virt_to_phys(rhi_call); + struct arm_smccc_res res; + + arm_smccc_1_1_invoke(SMC_RSI_HOST_CALL, addr, &res); + + return res.a0; +} + #endif /* __LINUX_ARM_RSI_CMDS_H_ */ diff --git a/include/linux/arm-smccc-rhi.h b/include/linux/arm-smccc-rhi.h new file mode 100644 index 000000000000..be89185cddae --- /dev/null +++ b/include/linux/arm-smccc-rhi.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2026 ARM Ltd. + */ + +#ifndef __LINUX_ARM_SMCCC_RHI_H_ +#define __LINUX_ARM_SMCCC_RHI_H_ + +#include + +#define SMC_RHI_CALL(func) \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_64, \ + ARM_SMCCC_OWNER_STANDARD_HYP,\ + (func)) + +#define RHI_HOSTCONF_VER_1_0 0x10000 +#define RHI_HOSTCONF_VERSION SMC_RHI_CALL(0x004E) + +#define RHI_HOSTCONF_FEATURE_GET_IPA_CHANGE_ALIGNMENT BIT(0) +#define RHI_HOSTCONF_FEATURES SMC_RHI_CALL(0x004F) + +#define RHI_HOSTCONF_GET_IPA_CHANGE_ALIGNMENT SMC_RHI_CALL(0x0050) + +#endif /* __LINUX_ARM_SMCCC_RHI_H_ */ diff --git a/include/linux/arm-smccc-rsi.h b/include/linux/arm-smccc-rsi.h index fddb77986f70..3532a3f08f4e 100644 --- a/include/linux/arm-smccc-rsi.h +++ b/include/linux/arm-smccc-rsi.h @@ -182,6 +182,13 @@ struct realm_config { */ #define SMC_RSI_IPA_STATE_GET SMC_RSI_FID(0x198) =20 +struct rsi_host_call { + union { + u16 imm; + u64 padding0; + }; + u64 gprs[31]; +} __aligned(0x100); /* * Make a Host call. * --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA0672DC783; Mon, 21 Sep 2026 14:49:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002162; cv=none; b=tjbu3gZj7uOZVFRttRsj0WAK49Hu8Nft7+jEt+TRh2sEAh3E/uulfTKo4ZwDQFxYaSN9+mhqjEPW6YOqGSq3ICKkiPuDOrO2mQ1euVMueqiuMGGRi4Fz13kAojFqlAcEX1d5JnS5k5PjhhBrlwz9W8P6XyIl/OgQVXPMs3DEQ/4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002162; c=relaxed/simple; bh=zB2zeKx2YeUdzvzpowAomppcMNlet1jlGVEh7GvRdTo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X/0sRVuAb6qmdDIsPYiE/1BKFppCC1dPTckqUziueUj+VkeeUV7RnRmE7yh32dpqhNOOy2jj+obIuHFWm9dJmclEhziQ5mVJrJkmQ/7h0SfSHfJNze1pzymyFXTTLxFHijCLarm49rEZ66x1PDn6UevdSIs1c+XalQvVQfLWOy0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HdZns7tR; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HdZns7tR" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E21C1F000FF; Mon, 21 Sep 2026 14:49:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002160; bh=t7sVvZ0TpR9jG7Z6PhTPLhhY4EWJmZkPJ/dFrOpNRfI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=HdZns7tReCovmHsXVFD0CwBYCulv7+HG4ka6oZYWJ8kj6CqlQcNkjR0nDk82rlhve f/a0B57WTKwdXlOAriV0QCKNZ/FFLLLyhvt3Kuu4C23Q1XEadXBHFCeSsQFVtrQjsS HKWwYcRjC5Vg1EuEAGD0vvxNwOuubN8KeP9oyU+2BNE5O+/LKxKJF2orKg7Z0tB3A8 iPx16UnvXUpcO31eD4eMzo9preD6/uaQqef3DwhNu3y++KqMYHK/nv93CUjcfhJ1KW Ot1Z0BTcMqt1vWP+CAtR06TdGl0x9DMMryLn4zt+HEJvReHDbPDJfGwOlEtlvvxpht oapMlkvCwpPYg== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 02/13] mm: Add an allocator for CoCo shared memory Date: Mon, 21 Sep 2026 20:18:36 +0530 Message-ID: <20260921144847.501151-3-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Confidential-computing guests may require memory shared with the host to be aligned and transitioned in units larger than PAGE_SIZE. Several DMA users need struct page-backed allocations satisfying these requirements. Provide a common allocator instead of requiring each user to open-code this sequence. Add alloc_cc_shared_pages() and its node-aware variant. The allocator rounds the requested size to the architecture's shared granule, allocates suitably aligned contiguous pages and transitions the complete range to shared state. It also preserves the caller's GFP policy. A private-to-shared transition may alter memory contents. Mask __GFP_ZERO from the underlying allocation and when requested, clear the complete transitioned range after cc_make_shared() succeeds. Return the page and transitioned size so free_cc_shared_pages() can restore the complete range to private state before freeing it. If private state cannot be established, retain the allocation instead of returning a possibly shared page to the buddy allocator. Also provide the shared-granule geometry and byte-oriented transition helpers used by the allocator and by callers managing their own backing memory. Signed-off-by: Aneesh Kumar K.V (Arm) --- include/linux/cc_shared.h | 39 +++++++++ mm/Makefile | 1 + mm/cc_shared.c | 172 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 212 insertions(+) create mode 100644 include/linux/cc_shared.h create mode 100644 mm/cc_shared.c diff --git a/include/linux/cc_shared.h b/include/linux/cc_shared.h new file mode 100644 index 000000000000..5f8db7c468c5 --- /dev/null +++ b/include/linux/cc_shared.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_CC_SHARED_H +#define _LINUX_CC_SHARED_H + +#include +#include + +struct page; + +struct cc_shared_pages { + struct page *page; + size_t shared_size; +}; + +struct cc_shared_layout { + size_t requested_size; + size_t shared_size; + size_t alignment; +}; + +/* + * Architectures may override this to return the granule used for transiti= ons + * between private and shared memory. The value must be a power of two and= no + * smaller than PAGE_SIZE. + */ +size_t arch_cc_shared_granule_size(void); + +size_t cc_shared_granule_size(void); +int cc_shared_calc_layout(size_t requested, struct cc_shared_layout *layou= t); +bool cc_shared_range_valid(phys_addr_t base, size_t size); +int cc_make_shared(void *addr, size_t size); +int cc_make_private(void *addr, size_t size); +int alloc_cc_shared_pages_node(int nid, gfp_t gfp, + size_t requested, struct cc_shared_pages *mem); +int alloc_cc_shared_pages(gfp_t gfp, + size_t requested, struct cc_shared_pages *mem); +void free_cc_shared_pages(struct cc_shared_pages *mem); + +#endif /* _LINUX_CC_SHARED_H */ diff --git a/mm/Makefile b/mm/Makefile index e7245cb88c66..6e6544428422 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -56,6 +56,7 @@ obj-y :=3D filemap.o mempool.o oom_kill.o fadvise.o \ compaction.o show_mem.o \ interval_tree.o list_lru.o workingset.o \ debug.o gup.o mmap_lock.o vma_init.o $(mmu-y) +obj-y +=3D cc_shared.o =20 # Give 'page_alloc' its own module-parameter namespace page-alloc-y :=3D page_alloc.o diff --git a/mm/cc_shared.c b/mm/cc_shared.c new file mode 100644 index 000000000000..3e33681218f1 --- /dev/null +++ b/mm/cc_shared.c @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2026 ARM Ltd. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +size_t __weak arch_cc_shared_granule_size(void) +{ + return PAGE_SIZE; +} + +size_t cc_shared_granule_size(void) +{ + size_t granule =3D arch_cc_shared_granule_size(); + + if (WARN_ON_ONCE(granule < PAGE_SIZE || !is_power_of_2(granule))) + return PAGE_SIZE; + + return granule; +} +EXPORT_SYMBOL_GPL(cc_shared_granule_size); + +int cc_shared_calc_layout(size_t requested, struct cc_shared_layout *layou= t) +{ + size_t granule, rounded; + + if (!requested || !layout) + return -EINVAL; + + granule =3D cc_shared_granule_size(); + if (check_add_overflow(requested, granule - 1, &rounded)) + return -EOVERFLOW; + + rounded =3D ALIGN_DOWN(rounded, granule); + layout->requested_size =3D requested; + layout->shared_size =3D rounded; + layout->alignment =3D granule; + + return 0; +} +EXPORT_SYMBOL_GPL(cc_shared_calc_layout); + +bool cc_shared_range_valid(phys_addr_t base, size_t size) +{ + size_t granule =3D cc_shared_granule_size(); + + if (!size) + return false; + + return IS_ALIGNED(base, granule) && IS_ALIGNED(size, granule); +} +EXPORT_SYMBOL_GPL(cc_shared_range_valid); + +static int cc_validate_transition(void *addr, size_t size) +{ + phys_addr_t phys; + + if (!addr || !size || !PAGE_ALIGNED(addr) || + !virt_addr_valid(addr)) + return -EINVAL; + + phys =3D page_to_phys(virt_to_page(addr)); + if (!cc_shared_range_valid(phys, size)) + return -EINVAL; + + return 0; +} + +int cc_make_shared(void *addr, size_t size) +{ + int ret =3D cc_validate_transition(addr, size); + + if (ret) + return ret; + + return set_memory_decrypted((unsigned long)addr, size >> PAGE_SHIFT); +} + +int cc_make_private(void *addr, size_t size) +{ + int ret =3D cc_validate_transition(addr, size); + + if (ret) + return ret; + + return set_memory_encrypted((unsigned long)addr, size >> PAGE_SHIFT); +} + +int alloc_cc_shared_pages_node(int nid, gfp_t gfp, + size_t requested, struct cc_shared_pages *mem) +{ + struct cc_shared_layout layout; + struct page *page; + unsigned int order; + bool zero =3D gfp & __GFP_ZERO; + int ret; + + if (!mem) + return -EINVAL; + + ret =3D cc_shared_calc_layout(requested, &layout); + if (ret) + return ret; + + order =3D get_order(layout.shared_size); + if (order > MAX_PAGE_ORDER) + return -EINVAL; + + /* + * State transitions require a linear-map address and may modify memory. + * Allocate from low memory and defer requested zeroing until afterwards. + */ + gfp &=3D ~(__GFP_HIGHMEM | __GFP_ZERO); + if (nid =3D=3D NUMA_NO_NODE) + page =3D alloc_pages(gfp, order); + else + page =3D alloc_pages_node(nid, gfp, order); + if (!page) + return -ENOMEM; + + ret =3D cc_make_shared(page_address(page), layout.shared_size); + if (ret) { + if (!cc_make_private(page_address(page), layout.shared_size)) + __free_pages(page, order); + else + pr_warn_ratelimited("leaking %zu bytes with uncertain shared state\n", + layout.shared_size); + return ret; + } + + if (zero) + memset(page_address(page), 0, layout.shared_size); + + mem->page =3D page; + mem->shared_size =3D layout.shared_size; + return 0; +} +EXPORT_SYMBOL_GPL(alloc_cc_shared_pages_node); + +int alloc_cc_shared_pages(gfp_t gfp, + size_t requested, struct cc_shared_pages *mem) +{ + return alloc_cc_shared_pages_node(NUMA_NO_NODE, gfp, requested, mem); +} +EXPORT_SYMBOL_GPL(alloc_cc_shared_pages); + +void free_cc_shared_pages(struct cc_shared_pages *mem) +{ + if (!mem || !mem->page) + return; + + if (cc_make_private(page_address(mem->page), mem->shared_size)) { + pr_warn_ratelimited("leaking %zu bytes that cannot be made private\n", + mem->shared_size); + return; + } + + __free_pages(mem->page, get_order(mem->shared_size)); + mem->page =3D NULL; + mem->shared_size =3D 0; +} +EXPORT_SYMBOL_GPL(free_cc_shared_pages); --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21A5A3233E8; Mon, 21 Sep 2026 14:49:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002170; cv=none; b=tk8uWAsDXrr6zd/8hk+1YNXiXhQywtMyOW9ihPIKZAzB3Wx4XTDZnKOgvtVcLemLX0BT6qPM60HdZ3aujv3MU7UcdiEgwpKgMocPlG5TAYheeSWc5w0RHi8NX6saymzSBxCmfYdnWAzZmwFSRDVWZzfaZbvh4yg0jo71h/uwPWQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002170; c=relaxed/simple; bh=m2MH69T4bqEk5LFJZ7xmCp4truzSoJU2K5PjcpbIjYE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aJiPp015VaQ084selJaydGDyACISiOb990cMNT5vCPc2IWO+Fm72RlicMLwnoOLTu5MXjk4oshKUnVTm+1205kptHa0UL6tiIl+FtEB3GNTP5cHo1DIpGEaLTDwYrvZw5aF9aZOCOhkGgpzYXZJq/Ql/zUc0SXG6fQigCJVdGyo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=avX+lq+F; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="avX+lq+F" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3AFFD1F0089A; Mon, 21 Sep 2026 14:49:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002168; bh=AgD4D7h4EfSsMTxEOq8ADKWRNeWiq2+KsY/R5C+YQNM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=avX+lq+FEOvHDH2Ka+90xabX+/3XfcNlbMJOgo8hQbVT7KtytJirI2WAK3ZkdU+A2 dhOvEYZsoG4EK7R7gE/4hye/lvQQedFCROIZN1l6Ld90l61kqkUYnN03fFNFiaYeNw xYhBUCcKknnZvHXKI7GHaDux6pUgVHLc0uY4qpQE1aVbJbF6TcenTqDGPVxMhfNzEU A5n9MGaMcMz9ytuyWSYbK8Q+RhhbKuySFp6CYcloHxaLPeHMEykf8QuR+nfpaHndMC eMeu0po3UwytgA8nKwZWgmhH5E9118kfO7P0hK8BFGlTQZgTXiEdpTbbrdUfSBmXiV vdTOxKBQ5YG6g== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 03/13] arm64: realm: Expose the CCA shared granule size through mem_encrypt ops Date: Mon, 21 Sep 2026 20:18:37 +0530 Message-ID: <20260921144847.501151-4-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" CCA guests must align shared/private memory transitions to the IPA state change alignment reported by the host. This alignment can be larger than PAGE_SIZE, so transitioning only a PAGE_SIZE-sized subrange may leave part of the host-managed granule in the wrong state. Cache the RHI-reported alignment during Realm initialization and expose it through the arm64 memory encryption operations. Implement arch_cc_shared_granule_size() so the common CoCo shared-memory allocator uses the same constraint. Use PAGE_SIZE when no suitable backend is registered. The common cc_make_shared() and cc_make_private() wrappers validate both the base and size against the architecture's shared granule size. Remove the PAGE_SIZE-only address checks from the lower-level arm64 transition hooks so that range validation remains in the common wrappers. Signed-off-by: Aneesh Kumar K.V (Arm) --- arch/arm64/include/asm/mem_encrypt.h | 1 + arch/arm64/mm/mem_encrypt.c | 13 +++++++++++-- drivers/firmware/arm_rmm/rsi.c | 13 ++++++++++++- 3 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/mem_encrypt.h b/arch/arm64/include/asm/= mem_encrypt.h index ef8b8463e52b..32064338c9b0 100644 --- a/arch/arm64/include/asm/mem_encrypt.h +++ b/arch/arm64/include/asm/mem_encrypt.h @@ -10,6 +10,7 @@ struct device; struct arm64_mem_crypt_ops { int (*encrypt)(unsigned long addr, int numpages); int (*decrypt)(unsigned long addr, int numpages); + size_t (*shared_granule_size)(void); }; =20 int arm64_mem_crypt_ops_register(const struct arm64_mem_crypt_ops *ops); diff --git a/arch/arm64/mm/mem_encrypt.c b/arch/arm64/mm/mem_encrypt.c index ee3c0ab04384..e6c059d20595 100644 --- a/arch/arm64/mm/mem_encrypt.c +++ b/arch/arm64/mm/mem_encrypt.c @@ -16,6 +16,7 @@ #include #include #include +#include #include =20 #include @@ -33,7 +34,7 @@ int arm64_mem_crypt_ops_register(const struct arm64_mem_c= rypt_ops *ops) =20 int set_memory_encrypted(unsigned long addr, int numpages) { - if (likely(!crypt_ops) || WARN_ON(!PAGE_ALIGNED(addr))) + if (likely(!crypt_ops)) return 0; =20 return crypt_ops->encrypt(addr, numpages); @@ -42,9 +43,17 @@ EXPORT_SYMBOL_GPL(set_memory_encrypted); =20 int set_memory_decrypted(unsigned long addr, int numpages) { - if (likely(!crypt_ops) || WARN_ON(!PAGE_ALIGNED(addr))) + if (likely(!crypt_ops)) return 0; =20 return crypt_ops->decrypt(addr, numpages); } EXPORT_SYMBOL_GPL(set_memory_decrypted); + +size_t arch_cc_shared_granule_size(void) +{ + if (likely(!crypt_ops) || !crypt_ops->shared_granule_size) + return PAGE_SIZE; + + return crypt_ops->shared_granule_size(); +} diff --git a/drivers/firmware/arm_rmm/rsi.c b/drivers/firmware/arm_rmm/rsi.c index 883f6091ff95..b59099d1f318 100644 --- a/drivers/firmware/arm_rmm/rsi.c +++ b/drivers/firmware/arm_rmm/rsi.c @@ -18,6 +18,7 @@ #include =20 static struct realm_config config; +static size_t ipa_state_change_alignment =3D PAGE_SIZE; =20 unsigned long prot_ns_shared; EXPORT_SYMBOL(prot_ns_shared); @@ -154,9 +155,17 @@ static int realm_set_memory_decrypted(unsigned long ad= dr, int numpages) return ret; } =20 +static size_t realm_shared_granule_size(void) +{ + if (is_realm_world()) + return ipa_state_change_alignment; + return PAGE_SIZE; +} + static const struct arm64_mem_crypt_ops realm_crypt_ops =3D { .encrypt =3D realm_set_memory_encrypted, .decrypt =3D realm_set_memory_decrypted, + .shared_granule_size =3D realm_shared_granule_size, }; =20 static int realm_register_memory_enc_ops(void) @@ -166,7 +175,7 @@ static int realm_register_memory_enc_ops(void) =20 /* we need an aligned struct for rsi_host_call. slab is not yet ready */ static struct rsi_host_call hostconf_call __initdata; -static unsigned long __maybe_unused __init get_ipa_state_change_alignment(= void) +static unsigned long __init get_ipa_state_change_alignment(void) { long ret; unsigned long shared_granule_size; @@ -217,6 +226,8 @@ void __init arm64_rsi_init(void) return; if (WARN_ON(rsi_get_realm_config(lm_alias(&config)))) return; + + ipa_state_change_alignment =3D get_ipa_state_change_alignment(); prot_ns_shared =3D __phys_to_pte_val(BIT(config.ipa_bits - 1)); =20 if (arm64_ioremap_prot_hook_register(realm_ioremap_hook)) --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A130246F4BB; Mon, 21 Sep 2026 14:49:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002177; cv=none; b=j8I+8hWa6Xruw55caZdGN7gNhZskiHsohH+7ivaia2U90WapNrGHANmBihrKeqBH0lw8ImZbU+w2mT1OjfpAFXpcYLVw3s0u8iD5LGS2WCmvCnQJaDwHAh+fQnsoMRCjAcoSuJO/aR2JwDaSr18xI/w6CUXETpV6QlnlBoK2RK8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002177; c=relaxed/simple; bh=53V69p39PUi4l6s3POXdnLWsBJZpvCWbAlqdcc7UTzk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RSq/DjgjqucGCt3gr/5H6bf2VQfU3yVz6M4Mdm4uqJy+EGp1f3Jc+MxxlR2kdpxSW65rM9Cry5PUOc+FZlccjPQINLt9oFydGkgpE157EgZ30DmiD+CanpQXMTbd3mhRIuWl8iclLxvFF+JefUKYGvFVTcl1k8XwdjwdSTsoe70= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=A1qRWRjl; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="A1qRWRjl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 69DCA1F000FF; Mon, 21 Sep 2026 14:49:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002176; bh=NUhVS/70oe+xVDq7uGuyK6On8HV25TTU2SaXV4Mxt0M=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=A1qRWRjl1RV2uS6/gMaLuJ2pewyPR4lUHdxCKfj6tWJ7rGg/GojHBKVpQxQjMXmS7 lY/S//7gau/68XfSGzMXBxaEMlB4ozHdEDhscfQu8EZpTtDlYz4eNXG2JKnXNuWtAu BesJL1+xMU8cIT7oCt0WaP8iyHhp83v3hmLLmamhE4RaEwCPOItgvTeld9nqcMfBzd fg42y/fS6NwDQZ9GpZazOkEimF6PsA6coMmQZM1qB6Wvq/kXB7yjuyaV4YKO3pKgR5 P/F300kBVcQYxP0VTJLVWehPnUpnJzr/xmVPpj5xMqxWRog4njX/mK/wons75YlO+s PhP5XTRddPZlw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 04/13] irqchip/gic-v3-its: Resolve the default NUMA node explicitly Date: Mon, 21 Sep 2026 20:18:38 +0530 Message-ID: <20260921144847.501151-5-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" alloc_pages_node() resolves NUMA_NO_NODE to numa_mem_id() internally. In preparation for switching ITS allocations to the CoCo shared memory allocator, resolve the default node explicitly in its_alloc_pages(). The CoCo shared memory allocator interprets NUMA_NO_NODE as a request to apply the current task's memory policy. Passing numa_mem_id() instead preserves the existing nearest-memory-node placement across the allocator switch. This patch does not change the current allocation behavior. Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/irqchip/irq-gic-v3-its.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index e9807af23537..ac5507acb1a0 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -32,6 +32,7 @@ #include #include #include +#include =20 #include #include @@ -236,7 +237,7 @@ static struct page *its_alloc_pages_node(int node, gfp_= t gfp, =20 static struct page *its_alloc_pages(gfp_t gfp, unsigned int order) { - return its_alloc_pages_node(NUMA_NO_NODE, gfp, order); + return its_alloc_pages_node(numa_mem_id(), gfp, order); } =20 static void its_free_pages(void *addr, unsigned int order) --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F330246F4BB; Mon, 21 Sep 2026 14:49:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002186; cv=none; b=RXAh+HyKZ2mbozCwjqEmAfEazA4AjcFNOcu0vMxKI6pGzu8xa96ItRleetNH+xPQU6tc/rfQaKoiz/H08pX5rKnby2cSbsWlzg+QSQWWUgvcFv7aGbtUb4b3q7YhieXKTWpXxgzAChsm1fj8udUTd299Bko0Ub98/dDOLSgvOd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002186; c=relaxed/simple; bh=UzoVCxnriQMKM9/Fv4JmiHZNIyxD4P6v8oevkMOlQ0g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fGCh/iVKjCXjjArbrdNiv4XQ5KFi/dB8DXUGd30YtNc4qDkEvoPt+1Ak++yYmG/xSf6MC1d0uPP47nJnPnrLCQf8yhwrNasJha2NqSeTUMlQXijgh+qTAZJrUc2mVxEiFbOAJ5433SpZTsIYKEdTkcU7uUQd6bmv/JWA6RedNrE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Kc8P2140; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Kc8P2140" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 25CA71F00893; Mon, 21 Sep 2026 14:49:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002184; bh=HSe1gOHGix6Y2SkfM68PVLIwg5C/r0qNlcWeNP5Fwxs=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Kc8P2140ujn+VDpWuuA0syxxlPiR5Dl/sChZvXp0DKgAkTCax/cLTbAp8LB0IWXAB R4hVFA8/L+PQBfe/eKPRAHKK3vjVTVdKREnLY52b3114xvDk3GOmnJLuphnn5R+2sm uOTWUiy5I/4uo1Ehng06km71pPSO/H3d3vc7lO+Gx1bVyxhyybF3v6MrEd5Hy3uB/W f1R2+b/CTIVMQSZLAmFcf1ixeWZgNZEF7GsQ3UxDkz+aw0go90MnrLv0EykJ1GFlpI enwffS8MUCShd5hX7jN0XvuJEjTM/ibsrm8B2tGuAGjBx+1oFq7lO33iYzkp/Ypz4I NeRaMoJfvCMAw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 05/13] irqchip/gic-v3-its: Allocate shared tables using CoCo shared memory allocator Date: Mon, 21 Sep 2026 20:18:39 +0530 Message-ID: <20260921144847.501151-6-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" ITS command queues and tables are shared with the host in a confidential-computing guest. The ITS allocator currently assumes that the guest PAGE_SIZE is sufficient for both allocation alignment and private/shared state changes. Route ITS page allocations through alloc_cc_shared_pages_node(). Keep the tracked table size based on the ITS-requested order. On release, reconstruct the common allocation metadata from the original ITS order and the architecture granule size so the same transitioned range and buddy order are used. Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/irqchip/irq-gic-v3-its.c | 40 ++++++++++++-------------------- 1 file changed, 15 insertions(+), 25 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index ac5507acb1a0..4814161df02e 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -19,7 +20,6 @@ #include #include #include -#include #include #include #include @@ -29,7 +29,6 @@ #include #include #include -#include #include #include #include @@ -214,25 +213,13 @@ static gfp_t gfp_flags_quirk; static struct page *its_alloc_pages_node(int node, gfp_t gfp, unsigned int order) { - struct page *page; - int ret =3D 0; - - page =3D alloc_pages_node(node, gfp | gfp_flags_quirk, order); - - if (!page) - return NULL; + struct cc_shared_pages mem; =20 - ret =3D set_memory_decrypted((unsigned long)page_address(page), - 1 << order); - /* - * If set_memory_decrypted() fails then we don't know what state the - * page is in, so we can't free it. Instead we leak it. - * set_memory_decrypted() will already have WARNed. - */ - if (ret) + if (alloc_cc_shared_pages_node(node, gfp | gfp_flags_quirk, + PAGE_SIZE << order, &mem)) return NULL; =20 - return page; + return mem.page; } =20 static struct page *its_alloc_pages(gfp_t gfp, unsigned int order) @@ -242,13 +229,15 @@ static struct page *its_alloc_pages(gfp_t gfp, unsign= ed int order) =20 static void its_free_pages(void *addr, unsigned int order) { - /* - * If the memory cannot be encrypted again then we must leak the pages. - * set_memory_encrypted() will already have WARNed. - */ - if (set_memory_encrypted((unsigned long)addr, 1 << order)) + struct cc_shared_layout layout; + struct cc_shared_pages mem; + + if (WARN_ON(cc_shared_calc_layout(PAGE_SIZE << order, &layout))) return; - free_pages((unsigned long)addr, order); + + mem.page =3D virt_to_page(addr); + mem.shared_size =3D layout.shared_size; + free_cc_shared_pages(&mem); } =20 static struct gen_pool *itt_pool; @@ -273,7 +262,8 @@ static void *itt_alloc_pool(int node, int size) if (!page) break; =20 - gen_pool_add(itt_pool, (unsigned long)page_address(page), PAGE_SIZE, nod= e); + gen_pool_add(itt_pool, (unsigned long)page_address(page), + cc_shared_granule_size(), node); } while (!addr); =20 return (void *)addr; --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DB243233E8; Mon, 21 Sep 2026 14:49:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002195; cv=none; b=LS5lhADNrJUTF6ykFo6watPlWfjOsWfx07w+JWRfLNRMOSsmTlalPDawKKeQIiu5qtcJ8q/y1muSvMw/GFv44gF4y872Wg6XTOh8yYI9P04eVb/x/cMg3+WptH7t9dnctVFDPaWxxkKkMkJ0I6AynLjNdv1Kn5eVM1FzqnvgssY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002195; c=relaxed/simple; bh=YGAvUXdHwA2bj2Y65/x62DxpbXFeL/Eml2oKLw6RTN0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F6aWRrVS220HOsIR+t4GSo3JBHvGFSh0gCc5S5KFoaYzFhJGtKZjPtEer2ha7QWbrB8JozR1NseRx2pEUkm2g9wJXcIebbxCU6bEbIxpuWDEuq78hA0gZk4LbKPPvQ46cP3rut7/wU/FIezxeHMd4Dgpd5qmy++vd+mwS8Q9AtY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YJdFcyfj; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YJdFcyfj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 63D501F00899; Mon, 21 Sep 2026 14:49:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002194; bh=sX7dUzzV4Hpq5I4TQA062/lgzQvMj3dJFADJlWWRou0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=YJdFcyfj9wySKBLUv88So2kWSKHUboALGjctLGYfvYsp8z/a/Kps7TgjPZbRm9goL sgTjrs5YQHKrzXJ3k8841BLqoq9r7xbExhIpUjjgZoUXiX7fRtzSKUqnWNUwLTuGQ6 ChP1vDUy1+53pQyNL+ZUNR3aU7Vg4iIuLPUQp/QBlZHsiX0bT+slgQZkeVXT58WH8X p7g1gM0r8ntvcDzvFPmBumXENBCN9mrW51ESwQ3rQRKXNrnGkFp8hbLrJJbxk2EP0m c/TBrFVHAQtHmymlb7JxIR7fEjesCenC7H/EYsav5tet8A2khH5jjopMUVaiqO40kt dPk0Kgwc25zVw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 06/13] dma-contiguous: Accept an explicit minimum alignment Date: Mon, 21 Sep 2026 20:18:40 +0530 Message-ID: <20260921144847.501151-7-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Confidential-computing shared allocations need CMA backing allocations to be aligned to the architecture private/shared state-change granule size. Passing a DMA attribute into CMA would make a generic physical allocator interpret DMA and confidential-computing policy that has already been resolved by its caller. Add an explicit minimum alignment order to dma_alloc_contiguous(). Preserve the existing size-derived alignment for ordinary callers by passing zero, and reject a request that exceeds CONFIG_CMA_ALIGNMENT. For supported requests, use the larger of the size-derived and requested orders, capped by the configured CMA alignment as before. Also distinguish preferred and required alignment for dma_alloc_from_contiguous(). Existing callers pass zero as the required alignment and retain the current clamping behavior. Callers that require a minimum alignment can request it explicitly and receive NULL when CMA cannot satisfy it. Signed-off-by: Aneesh Kumar K.V (Arm) --- arch/arm/mm/dma-mapping.c | 5 +++-- drivers/iommu/dma-iommu.c | 2 +- include/linux/dma-map-ops.h | 10 ++++++---- kernel/dma/contiguous.c | 33 +++++++++++++++++++++++---------- kernel/dma/direct.c | 2 +- kernel/dma/ops_helpers.c | 2 +- kernel/dma/pool.c | 2 +- kernel/kexec_file.c | 3 ++- 8 files changed, 38 insertions(+), 21 deletions(-) diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c index 7761099dde9e..9714fcd51941 100644 --- a/arch/arm/mm/dma-mapping.c +++ b/arch/arm/mm/dma-mapping.c @@ -398,7 +398,8 @@ static void *__alloc_from_contiguous(struct device *dev= , size_t size, struct page *page; void *ptr =3D NULL; =20 - page =3D dma_alloc_from_contiguous(dev, count, order, gfp & __GFP_NOWARN); + page =3D dma_alloc_from_contiguous(dev, count, order, 0, + gfp & __GFP_NOWARN); if (!page) return NULL; =20 @@ -866,7 +867,7 @@ static struct page **__iommu_alloc_buffer(struct device= *dev, size_t size, unsigned long order =3D get_order(size); struct page *page; =20 - page =3D dma_alloc_from_contiguous(dev, count, order, + page =3D dma_alloc_from_contiguous(dev, count, order, 0, gfp & __GFP_NOWARN); if (!page) goto error; diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 58c624513cd4..59baf2687612 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1627,7 +1627,7 @@ static void *iommu_dma_alloc_pages(struct device *dev= , size_t size, struct page *page =3D NULL; void *cpu_addr; =20 - page =3D dma_alloc_contiguous(dev, alloc_size, gfp); + page =3D dma_alloc_contiguous(dev, alloc_size, gfp, 0); if (!page) page =3D alloc_pages_node(node, gfp, get_order(alloc_size)); if (!page) diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h index 8fae2b7deb20..5fccda7e5c69 100644 --- a/include/linux/dma-map-ops.h +++ b/include/linux/dma-map-ops.h @@ -99,10 +99,11 @@ int __init dma_contiguous_reserve_area(phys_addr_t size= , phys_addr_t base, phys_addr_t limit, struct cma **res_cma, bool fixed); =20 struct page *dma_alloc_from_contiguous(struct device *dev, size_t count, - unsigned int order, bool no_warn); + unsigned int order, unsigned int required_order, bool no_warn); bool dma_release_from_contiguous(struct device *dev, struct page *pages, int count); -struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t g= fp); +struct page *dma_alloc_contiguous(struct device *dev, size_t size, + gfp_t gfp, unsigned int align_order); void dma_free_contiguous(struct device *dev, struct page *page, size_t siz= e); =20 void dma_contiguous_early_fixup(phys_addr_t base, unsigned long size); @@ -125,7 +126,8 @@ static inline int dma_contiguous_reserve_area(phys_addr= _t size, return -ENOSYS; } static inline struct page *dma_alloc_from_contiguous(struct device *dev, - size_t count, unsigned int order, bool no_warn) + size_t count, unsigned int order, unsigned int required_order, + bool no_warn) { return NULL; } @@ -136,7 +138,7 @@ static inline bool dma_release_from_contiguous(struct d= evice *dev, } /* Use fallback alloc() and free() when CONFIG_DMA_CMA=3Dn */ static inline struct page *dma_alloc_contiguous(struct device *dev, size_t= size, - gfp_t gfp) + gfp_t gfp, unsigned int align_order) { return NULL; } diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c index 66093460584e..a3eb3299b817 100644 --- a/kernel/dma/contiguous.c +++ b/kernel/dma/contiguous.c @@ -357,19 +357,25 @@ int __init dma_contiguous_reserve_area(phys_addr_t si= ze, phys_addr_t base, * dma_alloc_from_contiguous() - allocate pages from contiguous area * @dev: Pointer to device for which the allocation is performed. * @count: Requested number of pages. - * @align: Requested alignment of pages (in PAGE_SIZE order). + * @align: Preferred alignment of pages (in PAGE_SIZE order). + * @required_align: Minimum required alignment (in PAGE_SIZE order). * @no_warn: Avoid printing message about failed allocation. * * This function allocates memory buffer for specified device. It uses * device specific contiguous memory area if available or the default * global one. Requires architecture specific dev_get_cma_area() helper * function. + * + * The preferred alignment is capped at CONFIG_CMA_ALIGNMENT. Return NULL + * if the required alignment exceeds this limit. A required alignment of + * zero preserves the preferred-alignment clamping behavior. */ struct page *dma_alloc_from_contiguous(struct device *dev, size_t count, - unsigned int align, bool no_warn) + unsigned int align, unsigned int required_align, bool no_warn) { - if (align > CONFIG_CMA_ALIGNMENT) - align =3D CONFIG_CMA_ALIGNMENT; + if (required_align > CONFIG_CMA_ALIGNMENT) + return NULL; + align =3D min(max(align, required_align), CONFIG_CMA_ALIGNMENT); =20 return cma_alloc(dev_get_cma_area(dev), count, align, no_warn); } @@ -390,9 +396,14 @@ bool dma_release_from_contiguous(struct device *dev, s= truct page *pages, return cma_release(dev_get_cma_area(dev), pages, count); } =20 -static struct page *cma_alloc_aligned(struct cma *cma, size_t size, gfp_t = gfp) +static struct page *cma_alloc_aligned(struct cma *cma, size_t size, + gfp_t gfp, unsigned int align_order) { - unsigned int align =3D min(get_order(size), CONFIG_CMA_ALIGNMENT); + unsigned int align; + + if (align_order > CONFIG_CMA_ALIGNMENT) + return NULL; + align =3D min(max(get_order(size), align_order), CONFIG_CMA_ALIGNMENT); =20 return cma_alloc(cma, size >> PAGE_SHIFT, align, gfp & __GFP_NOWARN); } @@ -402,6 +413,7 @@ static struct page *cma_alloc_aligned(struct cma *cma, = size_t size, gfp_t gfp) * @dev: Pointer to device for which the allocation is performed. * @size: Requested allocation size. * @gfp: Allocation flags. + * @align_order: Minimum alignment as a power-of-two page order. * * tries to use device specific contiguous memory area if available, or it * tries to use per-numa cma, if the allocation fails, it will fallback to @@ -412,7 +424,8 @@ static struct page *cma_alloc_aligned(struct cma *cma, = size_t size, gfp_t gfp) * there is no need to waste CMA pages for that kind; it also helps reduce * fragmentations. */ -struct page *dma_alloc_contiguous(struct device *dev, size_t size, gfp_t g= fp) +struct page *dma_alloc_contiguous(struct device *dev, size_t size, + gfp_t gfp, unsigned int align_order) { #ifdef CONFIG_DMA_NUMA_CMA int nid =3D dev_to_node(dev); @@ -422,7 +435,7 @@ struct page *dma_alloc_contiguous(struct device *dev, s= ize_t size, gfp_t gfp) if (!gfpflags_allow_blocking(gfp)) return NULL; if (dev->cma_area) - return cma_alloc_aligned(dev->cma_area, size, gfp); + return cma_alloc_aligned(dev->cma_area, size, gfp, align_order); if (size <=3D PAGE_SIZE) return NULL; =20 @@ -431,7 +444,7 @@ struct page *dma_alloc_contiguous(struct device *dev, s= ize_t size, gfp_t gfp) struct cma *cma =3D dma_contiguous_numa_area[nid]; struct page *page; if (cma) { - page =3D cma_alloc_aligned(cma, size, gfp); + page =3D cma_alloc_aligned(cma, size, gfp, align_order); if (page) return page; } @@ -440,7 +453,7 @@ struct page *dma_alloc_contiguous(struct device *dev, s= ize_t size, gfp_t gfp) if (!dma_contiguous_default_area) return NULL; =20 - return cma_alloc_aligned(dma_contiguous_default_area, size, gfp); + return cma_alloc_aligned(dma_contiguous_default_area, size, gfp, align_or= der); } =20 /** diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index da665ca22d5c..d968a0c81e73 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -124,7 +124,7 @@ static struct page *__dma_direct_alloc_pages(struct dev= ice *dev, size_t size, WARN_ON_ONCE(!PAGE_ALIGNED(size)); =20 gfp |=3D dma_direct_optimal_gfp_mask(dev, &phys_limit); - page =3D dma_alloc_contiguous(dev, size, gfp); + page =3D dma_alloc_contiguous(dev, size, gfp, 0); if (page) { if (dma_coherent_ok(dev, page_to_phys(page), size) && (allow_highmem || !PageHighMem(page))) diff --git a/kernel/dma/ops_helpers.c b/kernel/dma/ops_helpers.c index 6b5f9208d31c..8320cc0fada5 100644 --- a/kernel/dma/ops_helpers.c +++ b/kernel/dma/ops_helpers.c @@ -66,7 +66,7 @@ struct page *dma_common_alloc_pages(struct device *dev, s= ize_t size, struct page *page; phys_addr_t phys; =20 - page =3D dma_alloc_contiguous(dev, size, gfp); + page =3D dma_alloc_contiguous(dev, size, gfp, 0); if (!page) page =3D alloc_pages_node(dev_to_node(dev), gfp, get_order(size)); if (!page) diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c index 00f422a1e896..70b7f64b17ab 100644 --- a/kernel/dma/pool.c +++ b/kernel/dma/pool.c @@ -99,7 +99,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_po= ol, size_t pool_size, pool_size =3D 1 << (PAGE_SHIFT + order); if (cma_in_zone(gfp)) page =3D dma_alloc_from_contiguous(NULL, 1 << order, - order, false); + order, 0, false); if (!page) page =3D alloc_pages(gfp | __GFP_NOWARN, order); } while (!page && order-- > 0); diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index 59fb9d71e9d8..2a337ce7264e 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -682,7 +682,8 @@ static int kexec_alloc_contig(struct kexec_buf *kbuf) if (kbuf->image->type =3D=3D KEXEC_TYPE_CRASH) return -EPERM; =20 - p =3D dma_alloc_from_contiguous(NULL, nr_pages, get_order(kbuf->buf_align= ), true); + p =3D dma_alloc_from_contiguous(NULL, nr_pages, + get_order(kbuf->buf_align), 0, true); if (!p) return -ENOMEM; =20 --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 343894A64D3; Mon, 21 Sep 2026 14:50:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002206; cv=none; b=p7NcfRTTyRWEm7aiVgdXut/kg55SkwEeDNsTVMy4Xv+032tOCo9/i3l9/7W9G5AraoeD6ATzJ/lX86a2i8DuFUSaGEc1UTBAGRPiR6DiVO0KGwkJIEmVOeSJTo0Oi4PYs/BZykETxF83ADWsNWCgWlXxsBsV60wfz6BRZ7W61UU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002206; c=relaxed/simple; bh=xfg4l7aAPcFuNNw0eq2vq9ZJl7eqy9aAtKlDZYYEOu0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nMyrV+tgnd9CgPF4QZSE9TG++rGFbaMy95El4FPkkOGYXllRI0whOTB5f0IXnQfaZsscfBzuBtBjdX/G7zWU58h8SnXjEfXby2gnuHGt6mA0CxxyEJKPfBpX8rcWLFD/SNyYvlUc+Kt/Qj2hkgdboOURRaWDVHARyTP/INirnsQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GbU3J4YR; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GbU3J4YR" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 500401F000FF; Mon, 21 Sep 2026 14:49:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002204; bh=J0RXC9wgCT9tCmz+jINf5SBQKcG/vPYAJMGd4hap8Yw=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=GbU3J4YRMftmRU3jVFK0P2ygG00J8C4jBu4Ys1tpUN98glJT1tX1ZPswO6N6z6lAA WnjkUlnCLjFKCszyx2pEUTo3+5QIPTCs2doBtoVsBQS5mu/mREAiBQLWiXe4+wSLZm mSsfl5BK3UnuU6Gr3t60HLkoElNHWWH6WjIi93pYYLmUZ9v6tZ2mMB55GX6AZ/V3oV D18fKW95kPaoC7bvZrUyXgSyBqZrS8ScRtMvemwr2xSlV/oeZwfDMlmc1+pQMh3odX R0cR/d8No7rGW8ggtnjhCOriMqycOZC8UOGIZfaYntGncIiFOc+6D2M7lKuHb5IuR3 WekYxl0d0gA9g== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 07/13] dma-pool: Allocate CoCo atomic pools using CoCo shared memory allocator Date: Mon, 21 Sep 2026 20:18:41 +0530 Message-ID: <20260921144847.501151-8-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Atomic DMA allocations cannot perform a potentially sleeping private/shared state transition at allocation time. The atomic DMA pools avoid this by transitioning their backing allocations in a sleepable context and suballocating memory that is already shared. Architectures may require these transitions to use a shared granule size larger than PAGE_SIZE. The existing fallback loop can reduce the backing allocation below the order required by that size, producing a range that cannot be safely transitioned. For pools marked cc_shared, round the requested pool size up to a multiple of the shared granule size and prevent allocation fallback below the order required by that size. Pass this minimum order to dma_alloc_from_contiguous() as a required alignment so that CMA cannot silently clamp it. If CMA cannot satisfy the alignment, fall back to the buddy allocator. Non-shared pools pass zero and retain the existing CMA alignment policy. Use the cc_make_shared() and cc_make_private() helpers to transition the backing allocation and validate its address and size. Individual atomic allocations may remain smaller than the shared granule size because the backing allocation remains owned by the pool and in the shared state. Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/pool.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/kernel/dma/pool.c b/kernel/dma/pool.c index 70b7f64b17ab..651d3a99c574 100644 --- a/kernel/dma/pool.c +++ b/kernel/dma/pool.c @@ -4,12 +4,12 @@ * Copyright (C) 2020 Google LLC */ #include +#include #include #include #include #include #include -#include #include #include #include @@ -85,6 +85,8 @@ static bool cma_in_zone(gfp_t gfp) static int atomic_pool_expand(struct dma_gen_pool *dma_pool, size_t pool_s= ize, gfp_t gfp) { + struct cc_shared_layout layout; + unsigned int min_order =3D 0; unsigned int order; struct page *page =3D NULL; bool leak_pages =3D false; @@ -92,6 +94,16 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_p= ool, size_t pool_size, int ret =3D -ENOMEM; pgprot_t prot __maybe_unused; =20 + if (dma_pool->cc_shared) { + ret =3D cc_shared_calc_layout(pool_size, &layout); + if (ret) + goto out; + pool_size =3D layout.shared_size; + min_order =3D get_order(layout.alignment); + if (min_order > MAX_PAGE_ORDER) + return -E2BIG; + } + /* Cannot allocate larger than MAX_PAGE_ORDER */ order =3D min(get_order(pool_size), MAX_PAGE_ORDER); =20 @@ -99,10 +111,10 @@ static int atomic_pool_expand(struct dma_gen_pool *dma= _pool, size_t pool_size, pool_size =3D 1 << (PAGE_SHIFT + order); if (cma_in_zone(gfp)) page =3D dma_alloc_from_contiguous(NULL, 1 << order, - order, 0, false); + order, min_order, false); if (!page) page =3D alloc_pages(gfp | __GFP_NOWARN, order); - } while (!page && order-- > 0); + } while (!page && order-- > min_order); if (!page) goto out; =20 @@ -126,8 +138,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_= pool, size_t pool_size, * shrink so no re-encryption occurs in dma_direct_free(). */ if (dma_pool->cc_shared) { - ret =3D set_memory_decrypted((unsigned long)page_to_virt(page), - 1 << order); + ret =3D cc_make_shared(page_to_virt(page), pool_size); if (ret) { leak_pages =3D true; goto remove_mapping; @@ -144,7 +155,7 @@ static int atomic_pool_expand(struct dma_gen_pool *dma_= pool, size_t pool_size, =20 encrypt_mapping: if (dma_pool->cc_shared && - set_memory_encrypted((unsigned long)page_to_virt(page), 1 << order)) + cc_make_private(page_to_virt(page), pool_size)) leak_pages =3D true; =20 remove_mapping: --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E2C24AB1AF; Mon, 21 Sep 2026 14:50:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002216; cv=none; b=e/rVepVSXR5hPHom5cK7MDP9BcfjLCZBtT20qGC2wQY/PikAD1r7I5Y+TpBHntk2t5F0+g8ehlCXbllHCsUQoju3uSAfJ0g4b7JpQQ93KAL6IlWchZpvG+6vpgDHkvb3rcitrYd03sls0RHWn7OPEjf7WClUWNnCfmSyPdLCnzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002216; c=relaxed/simple; bh=DSsfArB10PIcNzF5fIFeyybye5zWSuucaXodfFVDKmo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Jm+x56HHnFNOYkMLXAzmFg1Q9/gO2W8T0d/Ggsg+OUIsdNHyIOHtuKuFXCfJKNqM0mDiBiMM9qQ3CwUpMNIYyG14lKAm2A8bgw0MGc1rQmxTGciM+SYlFRjLbLWlVD2m5Ddk/omeNeDaAQeXysa5z2lEq+L1x5pmH9R/qBwkKbg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ik8PkMSE; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ik8PkMSE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 817AE1F008A1; Mon, 21 Sep 2026 14:50:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002214; bh=aWYdZ91y2ZIXgzfNWppX/K87BNGkPgAm/JMTyJDzEh4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ik8PkMSEyJLfZFUGTT09e/znzmIj1IQB1TBp51D5k+Yy0ho+aw3/dRdBdIpRmKYnr ZX0EP5g0NObBCy2Gjw2BuyvDeKhrqS+fY/IdnQoAEEECDsWQ3gk00OnyJEj94hJEsS jcs/OA949tuP2PX2vfy+us7YkGRG7OZKcMEHoajnenBSSE1rRr5PMTaippawTE2vpC 678DJsoB1qY8zfYxIX4yfz7lpI2H6IGYT22cB5GPSXDOBmyMgKu3O0QYqM4ivx4SfI Cld32H3UEmro8rXX+UU0djHeoYDylyNw0S8cJfuOsD6TdJzT/bK/IAFSMizS0sB+UB 4AZHXeqwquuiw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 08/13] dma-direct: Align CoCo shared DMA allocations to the shared granule size Date: Mon, 21 Sep 2026 20:18:42 +0530 Message-ID: <20260921144847.501151-9-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the common CoCo shared-memory geometry helpers for allocations backed directly by CMA or the page allocator. Round the backing allocation to a whole shared granule, pass the required alignment order through the DMA contiguous allocator, and transition the complete range through the common shared/private helpers. Recompute the layout before freeing ordinary direct allocations so the transition back to private memory and dma_free_contiguous() cover exactly the range acquired by the allocation path. If restoring private state fails, retain the existing fail-safe behavior and leak the pages rather than returning potentially shared memory to the allocator. This also applies the same rules to dma_direct_alloc_pages(), covering callers which require a struct page result rather than a CPU virtual address. Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/direct.c | 55 +++++++++++++++++++++++++++++++++++---------- 1 file changed, 43 insertions(+), 12 deletions(-) diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index d968a0c81e73..d293198384c3 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -11,10 +11,10 @@ #include #include #include -#include #include #include #include +#include =20 #include "direct.h" =20 @@ -85,7 +85,7 @@ static int dma_set_decrypted(struct device *dev, void *va= ddr, size_t size) { int ret; =20 - ret =3D set_memory_decrypted((unsigned long)vaddr, PFN_UP(size)); + ret =3D cc_make_shared(vaddr, size); if (ret) pr_warn_ratelimited("leaking DMA memory that can't be decrypted\n"); return ret; @@ -95,7 +95,7 @@ static int dma_set_encrypted(struct device *dev, void *va= ddr, size_t size) { int ret; =20 - ret =3D set_memory_encrypted((unsigned long)vaddr, PFN_UP(size)); + ret =3D cc_make_private(vaddr, size); if (ret) pr_warn_ratelimited("leaking DMA memory that can't be re-encrypted\n"); return ret; @@ -115,7 +115,7 @@ static struct page *dma_direct_alloc_swiotlb(struct dev= ice *dev, size_t size, } =20 static struct page *__dma_direct_alloc_pages(struct device *dev, size_t si= ze, - gfp_t gfp, bool allow_highmem) + gfp_t gfp, bool allow_highmem, unsigned int align_order) { int node =3D dev_to_node(dev); struct page *page; @@ -124,7 +124,7 @@ static struct page *__dma_direct_alloc_pages(struct dev= ice *dev, size_t size, WARN_ON_ONCE(!PAGE_ALIGNED(size)); =20 gfp |=3D dma_direct_optimal_gfp_mask(dev, &phys_limit); - page =3D dma_alloc_contiguous(dev, size, gfp, 0); + page =3D dma_alloc_contiguous(dev, size, gfp, align_order); if (page) { if (dma_coherent_ok(dev, page_to_phys(page), size) && (allow_highmem || !PageHighMem(page))) @@ -184,7 +184,7 @@ static void *dma_direct_alloc_no_mapping(struct device = *dev, size_t size, { struct page *page; =20 - page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true); + page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, true, 0); if (!page) return NULL; =20 @@ -205,6 +205,8 @@ void *dma_direct_alloc(struct device *dev, size_t size, bool remap =3D false, set_uncached =3D false; bool mark_mem_decrypt =3D false; bool allow_highmem =3D true; + struct cc_shared_layout layout; + unsigned int align_order =3D 0; struct page *page; void *cpu_addr; =20 @@ -285,8 +287,16 @@ void *dma_direct_alloc(struct device *dev, size_t size, return NULL; } =20 + if (mark_mem_decrypt) { + if (cc_shared_calc_layout(size, &layout)) + return NULL; + size =3D layout.shared_size; + align_order =3D get_order(layout.alignment); + } + /* we always manually zero the memory once we are done */ - page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, allow_hig= hmem); + page =3D __dma_direct_alloc_pages(dev, size, gfp & ~__GFP_ZERO, + allow_highmem, align_order); if (!page) return NULL; =20 @@ -305,7 +315,7 @@ void *dma_direct_alloc(struct device *dev, size_t size, void *lm_addr; =20 lm_addr =3D page_address(page); - if (set_memory_decrypted((unsigned long)lm_addr, PFN_UP(size))) + if (dma_set_decrypted(dev, lm_addr, size)) goto out_leak_pages; } =20 @@ -362,6 +372,7 @@ void dma_direct_free(struct device *dev, size_t size, phys_addr_t phys; bool mark_mem_encrypted =3D false; struct io_tlb_pool *swiotlb_pool; + struct cc_shared_layout layout; unsigned int page_order =3D get_order(size); =20 /* @@ -406,6 +417,12 @@ void dma_direct_free(struct device *dev, size_t size, /* Swiotlb doesn't need a page attribute update on free */ mark_mem_encrypted =3D false; =20 + if (mark_mem_encrypted) { + if (WARN_ON_ONCE(cc_shared_calc_layout(size, &layout))) + return; + size =3D layout.shared_size; + } + if (is_vmalloc_addr(cpu_addr)) { vunmap(cpu_addr); } else { @@ -417,10 +434,8 @@ void dma_direct_free(struct device *dev, size_t size, void *lm_addr; =20 lm_addr =3D phys_to_virt(phys); - if (set_memory_encrypted((unsigned long)lm_addr, PFN_UP(size))) { - pr_warn_ratelimited("leaking DMA memory that can't be re-encrypted\n"); + if (dma_set_encrypted(dev, lm_addr, size)) return; - } } =20 if (swiotlb_pool) @@ -433,6 +448,8 @@ struct page *dma_direct_alloc_pages(struct device *dev,= size_t size, dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp) { unsigned long attrs =3D 0; + struct cc_shared_layout layout; + unsigned int align_order =3D 0; struct page *page; void *cpu_addr; =20 @@ -452,7 +469,14 @@ struct page *dma_direct_alloc_pages(struct device *dev= , size_t size, goto setup_page; } =20 - page =3D __dma_direct_alloc_pages(dev, size, gfp, false); + if (attrs & __DMA_ATTR_ALLOC_CC_SHARED) { + if (cc_shared_calc_layout(size, &layout)) + return NULL; + size =3D layout.shared_size; + align_order =3D get_order(layout.alignment); + } + + page =3D __dma_direct_alloc_pages(dev, size, gfp, false, align_order); if (!page) return NULL; =20 @@ -476,6 +500,7 @@ void dma_direct_free_pages(struct device *dev, size_t s= ize, phys_addr_t phys; void *vaddr =3D page_address(page); struct io_tlb_pool *swiotlb_pool; + struct cc_shared_layout layout; /* * if the device had requested for an unencrypted buffer, * convert it to encrypted on free @@ -492,6 +517,12 @@ void dma_direct_free_pages(struct device *dev, size_t = size, if (swiotlb_pool) mark_mem_encrypted =3D false; =20 + if (mark_mem_encrypted) { + if (WARN_ON_ONCE(cc_shared_calc_layout(size, &layout))) + return; + size =3D layout.shared_size; + } + if (mark_mem_encrypted && dma_set_encrypted(dev, vaddr, size)) return; =20 --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 695E94AA01B; Mon, 21 Sep 2026 14:50:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002225; cv=none; b=objR3rzv8Hvo8XL6z8TlvwEUCQobx6Cj12LNsk+ggaTqMRwzJmxjRbkGvg71D53mbHrpo2MLA4VJjOfMGbPx4US47bXyM1Cf1xCY1Z9IVtPgIfJCdlSu7oTQSYdLb//palzXpfK2etzo8djLvMspTlPG64z7rRySLtinUosqQDo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002225; c=relaxed/simple; bh=n60ywsDXdJuMaFjQxbfN22Ty1YGsfuy5RFFHaRZGs2s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZZWSTUSr3D+C7dGucXg5T8rrosZ7yXvnKp2J5yRkfwduKnyt946YUeTH/ddVntnpRX7aaIHjrCp4Sn9WVsyO4hucEBtHlYbORdQM0OJH2X5l9YKxSRw45TUqfQ/4AOxQi28gKdl/EbXxTYa2kRMh8dp2RfI9wbLcJyEyVHCJ9TY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HhhVBHd7; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HhhVBHd7" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5EAD21F00893; Mon, 21 Sep 2026 14:50:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002224; bh=2XD9KbmPFTDvG/++FINRoeRjfo1Owk1SK7Ef5Ny4pkE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=HhhVBHd70NwE6tFDSR7EWwSk5kUn7dx1kha7j/ncW7XELIYaoWokPSNoeR4Gb7csH manH/KE1JbmJ0i/7twyR8DQ5XrSNGVGXxOuKAD+RPrR+xXRboXZTQPlJ3QmEqu859i HZU0YFWzKtkC9U5gYk+hQL+Nk1Q/jBfPYKwtVIz5itpVENE+9110tsSKuHgFvohU87 NP77rfhWL7u4mvtugGaq0ws9V+9f5uQ3MDPuE4Cb8F3CfiH0z1Pbw4bXnIUOU/GE2O JQkgUxAsfSs1Z5A0oGnQxHSA6wqJm37P5biOyVae7LUBXaPY5G/IlRwsFO2EbC8dFE osNmzMILFDf9Q== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 09/13] swiotlb: Align shared IO TLB pools to the shared granule size Date: Mon, 21 Sep 2026 20:18:43 +0530 Message-ID: <20260921144847.501151-10-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Align and size the early memblock pool to the common CoCo shared granule. Use the same rounded extent when the pool is transitioned and when it is released. Re-zero the extent after a successful private-to-shared transition because the architecture operation may change memory contents. This deliberately leaves the slot count unchanged: any rounded tail belongs to the pool allocation but is not advertised as allocatable SWIOTLB space. Replace the default and dynamic pool set_memory calls with the helpers so alignment validation and architecture dispatch are kept in one place. As before, pages are intentionally leaked if their private state cannot be restored safely. Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/swiotlb.c | 42 +++++++++++++++++++++++++----------------- 1 file changed, 25 insertions(+), 17 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index ded7016a46a7..9928d75efc2d 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -22,6 +22,7 @@ =20 #include #include +#include #include #include #include @@ -38,7 +39,6 @@ #include #include #include -#include #include #include #include @@ -369,16 +369,18 @@ void __init swiotlb_update_mem_attributes(void) =20 if (!mem->nslabs || mem->late_alloc) return; - bytes =3D PAGE_ALIGN(mem->nslabs << IO_TLB_SHIFT); + + bytes =3D ALIGN(mem->nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); =20 if (io_tlb_default_mem.cc_shared) { int ret; =20 - ret =3D set_memory_decrypted((unsigned long)mem->vaddr, - bytes >> PAGE_SHIFT); + ret =3D cc_make_shared(mem->vaddr, bytes); if (ret) { pr_warn("Failed to decrypt default memory pool, disabling it\n"); swiotlb_mark_pool_used(mem); + } else { + memset(mem->vaddr, 0, bytes); } } } @@ -436,8 +438,8 @@ static void __init *swiotlb_memblock_alloc(unsigned lon= g nslabs, unsigned int flags, int (*remap)(void *tlb, unsigned long nslabs)) { - size_t bytes =3D PAGE_ALIGN(nslabs << IO_TLB_SHIFT); void *tlb; + size_t bytes =3D ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); =20 /* * By default allocate the bounce buffer memory from low memory, but @@ -445,9 +447,9 @@ static void __init *swiotlb_memblock_alloc(unsigned lon= g nslabs, * memory encryption. */ if (flags & SWIOTLB_ANY) - tlb =3D memblock_alloc(bytes, PAGE_SIZE); + tlb =3D memblock_alloc(bytes, cc_shared_granule_size()); else - tlb =3D memblock_alloc_low(bytes, PAGE_SIZE); + tlb =3D memblock_alloc_low(bytes, cc_shared_granule_size()); =20 if (!tlb) { pr_warn("%s: Failed to allocate %zu bytes tlb structure\n", @@ -456,7 +458,7 @@ static void __init *swiotlb_memblock_alloc(unsigned lon= g nslabs, } =20 if (remap && remap(tlb, nslabs) < 0) { - memblock_free(tlb, PAGE_ALIGN(bytes)); + memblock_free(tlb, bytes); pr_warn("%s: Failed to remap %zu bytes\n", __func__, bytes); return NULL; } @@ -578,7 +580,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, swiotlb_adjust_nareas(num_possible_cpus()); =20 retry: - order =3D get_order(nslabs << IO_TLB_SHIFT); + order =3D get_order(ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size(= ))); nslabs =3D SLABS_PER_PAGE << order; =20 while ((SLABS_PER_PAGE << order) > IO_TLB_MIN_SLABS) { @@ -587,6 +589,8 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, if (vstart) break; order--; + if (order < get_order(cc_shared_granule_size())) + break; nslabs =3D SLABS_PER_PAGE << order; retried =3D true; } @@ -626,8 +630,7 @@ int swiotlb_init_late(size_t size, gfp_t gfp_mask, goto error_slots; =20 if (io_tlb_default_mem.cc_shared) { - rc =3D set_memory_decrypted((unsigned long)vstart, - (nslabs << IO_TLB_SHIFT) >> PAGE_SHIFT); + rc =3D cc_make_shared(vstart, nslabs << IO_TLB_SHIFT); if (rc) { leak_pages =3D true; goto error_decrypt; @@ -667,11 +670,11 @@ void __init swiotlb_exit(void) =20 pr_info("tearing down default memory pool\n"); tbl_vaddr =3D (unsigned long)phys_to_virt(mem->start); - tbl_size =3D PAGE_ALIGN(mem->end - mem->start); + tbl_size =3D ALIGN(mem->end - mem->start, cc_shared_granule_size()); slots_size =3D PAGE_ALIGN(array_size(sizeof(*mem->slots), mem->nslabs)); =20 if (io_tlb_default_mem.cc_shared) { - if (set_memory_encrypted(tbl_vaddr, tbl_size >> PAGE_SHIFT)) + if (cc_make_private((void *)tbl_vaddr, tbl_size)) leak_pages =3D true; } =20 @@ -711,12 +714,15 @@ void __init swiotlb_exit(void) static struct page *alloc_dma_pages(gfp_t gfp, size_t bytes, u64 phys_limit, unsigned long attrs) { - unsigned int order =3D get_order(bytes); bool cc_shared =3D attrs & __DMA_ATTR_ALLOC_CC_SHARED; + unsigned int order; struct page *page; phys_addr_t paddr; void *vaddr; =20 + if (cc_shared) + bytes =3D ALIGN(bytes, cc_shared_granule_size()); + order =3D get_order(bytes); page =3D alloc_pages(gfp, order); if (!page) return NULL; @@ -728,13 +734,13 @@ static struct page *alloc_dma_pages(gfp_t gfp, size_t= bytes, } =20 vaddr =3D phys_to_virt(paddr); - if (cc_shared && set_memory_decrypted((unsigned long)vaddr, PFN_UP(bytes)= )) + if (cc_shared && cc_make_shared(vaddr, bytes)) goto error; return page; =20 error: /* Intentional leak if pages cannot be encrypted again. */ - if (cc_shared && !set_memory_encrypted((unsigned long)vaddr, PFN_UP(bytes= ))) + if (cc_shared && !cc_make_private(vaddr, bytes)) __free_pages(page, order); return NULL; } @@ -807,9 +813,11 @@ static void swiotlb_free_tlb(void *vaddr, size_t bytes= , bool cc_shared) dma_free_from_pool(NULL, vaddr, bytes)) return; =20 + if (cc_shared) + bytes =3D ALIGN(bytes, cc_shared_granule_size()); /* Intentional leak if pages cannot be encrypted again. */ if (!cc_shared || - !set_memory_encrypted((unsigned long)vaddr, PFN_UP(bytes))) + !cc_make_private(vaddr, bytes)) __free_pages(virt_to_page(vaddr), get_order(bytes)); } =20 --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C21BA4A9D49; Mon, 21 Sep 2026 14:50:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002235; cv=none; b=hWVo3p7F7QnTjBEZyRUDy6pWdZWF8iV2BDzDv/hR39eiSxevWHBjzbvxG6jNp/f0kQf9nBbjePUGsYsvMY2SSqU9XaWJXHx5NEaHrHVMDWVnpBp4S6XAjwKsbQSX5Ko4HjRokEvxhKdYhKQcTbqd94LNqGB3yNW8c3dz0LkMY4o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002235; c=relaxed/simple; bh=vGksCyHyAkFbUYd9dsUIKEBHw/XWRfLaAVMwPBqAwMk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=un5E4Bb7RzsNzbwOC0obPc8xLGmsMrj27+K06oQXieb2rRCoIMjc4Pq8JU+20tJiVW0HUiYsuCOOaIF5YoQR8qA9XyuNT90zMhfdCgs4VTPEUV4lrCAQJ7kg1QaW8+zvte9zgbaAhZLp36ibtpt468BjWKB8I759JDqDTEZIWbA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Psocjzb6; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Psocjzb6" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 441611F00898; Mon, 21 Sep 2026 14:50:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002234; bh=HsGwfxBcGa2X8eifj86iWN6Y5mB+a2kx/vCpPCjEWWg=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Psocjzb6bowHSDioUIF9/6tvJDU4QGBRjJjxTFtyWDQk/oft1o4Ri+wYY8Q5qVirE kqBOewXMKNfLKWyfWfmRQDs0BAbqll4f1bcCNdAzPNx7tU7gy+7rGsnC5ILEr6UEo5 v1g7rqi1Aht05l0fIpjnwsguZWG6xUS2U2gJWyL4ZxAKLzo/36bVRfUZ9Qfj9OpRRJ MF3cO+ivHMYiYkd+66D0ig61BpdP0tNcsvHZSeuG7LX5q9xooHfyg4toVn00kxehkp 8qZq9va6WawQxeeOKd61ivw0oVChTUbrGK15k3D2Ndt7EX//ZubzVxK/gi6rxepOn1 6SredcbkRgL/w== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 10/13] swiotlb: Reject misaligned restricted DMA pools for CoCo guests Date: Mon, 21 Sep 2026 20:18:44 +0530 Message-ID: <20260921144847.501151-11-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A restricted DMA pool is a firmware-described ownership boundary. Unlike a pool allocated by SWIOTLB itself, its base cannot be moved and its size cannot be rounded without claiming memory outside the reserved region. Confidential-computing guests may require private/shared state changes to cover units larger than PAGE_SIZE. Passing a misaligned restricted region to set_memory_decrypted() can therefore fail. Validate both the physical base and the complete reserved size with the common CoCo shared-range helper before allocating SWIOTLB metadata or performing a state transition. Perform the transition through cc_make_shared(). Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/swiotlb.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index 9928d75efc2d..cb67105b8812 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -2010,6 +2010,14 @@ static int rmem_swiotlb_device_init(struct reserved_= mem *rmem, return -EINVAL; } =20 + if (cc_platform_has(CC_ATTR_MEM_ENCRYPT) && + !cc_shared_range_valid(rmem->base, rmem->size)) { + dev_err(dev, + "Restricted DMA pool must be aligned to %#zx bytes for memory encryptio= n\n", + cc_shared_granule_size()); + return -EINVAL; + } + /* * Since multiple devices can share the same pool, the private data, * io_tlb_mem struct, will be initialized by the first device attached @@ -2043,8 +2051,7 @@ static int rmem_swiotlb_device_init(struct reserved_m= em *rmem, int ret; =20 mem->cc_shared =3D true; - ret =3D set_memory_decrypted((unsigned long)phys_to_virt(rmem->base), - rmem->size >> PAGE_SHIFT); + ret =3D cc_make_shared(phys_to_virt(rmem->base), rmem->size); if (ret) { dev_err(dev, "Failed to decrypt restricted DMA pool\n"); kfree(pool->areas); --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5165E4A99D6; Mon, 21 Sep 2026 14:50:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002245; cv=none; b=MDyyMeuLEmx6b4HOV1cssVyNdn1cLRnNrSGCb99wAc+QcnF3LPgkBzpkkv4Uw0+HNUV/bC5eIMuPd+wY5LcGA+3tWjNiW4R1kiJKHuxaEUZTRY8Oi7wLpSt0dsKOStdobV+HkNrn1x7rzCerFw2APGs0qdnK3aYdu1WGCfZgD7I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002245; c=relaxed/simple; bh=drypGKbE/uJUN8VQPTwrS3YIr5N/QFKxqxtDqB4F6so=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=se5/dckfIoJryU1CUoJq/7OP221c6JpFM306E7JPvxvAqN0M//ihD8adhVMi0QAbES+p+ymGUZxA3euLRVRJUwAocBka3D98xxp6dSuAfOg2ehEbzI4wVVr4z6nrGEJbW5dTjYiOLO51MT2rIi8rRLkK/RKnTwIB44t81Kkc3P4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PjUxmdmq; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PjUxmdmq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 612241F00893; Mon, 21 Sep 2026 14:50:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002244; bh=siS6q1O96r8tBJzuPFNLC7opJrdAHxEvYaTkWIfgUUU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=PjUxmdmq75evpjC4UkLaIYo0hP1XgCpj8SoLSF8Mj/tcBn6ZXbtjFhVAaKSTA07Pz lcTzehyS2LIa8dVZJu0DH7jWgBEqhuXNdci/JIOQjepi2K6XsYv9W8nt+AQOrN2R5A GwhI+wHs4GccO3WeI8bWmNcjtBIETFgcFnkfcJJEuwLLSycNBG1sqPgI3051Zr8kZ6 U97zOFM3SrsTI1AnQl5GJtgARF76druhEyK+73UdsBpZrzCA9fClz5xVqiCrtuMxzV w1xa4hEs4oNB/M+V03LS2ikLG8NnJCu3G2/YsWbURiG3Y/RuHL5MIzUGNI/ccucgQc T9CCrMe1mJY2w== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 11/13] dma-buf: system_heap: Limit scatterlist entries to the buffer size Date: Mon, 21 Sep 2026 20:18:45 +0530 Message-ID: <20260921144847.501151-12-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The system heap currently allocates each backing page no larger than the remaining dma-buf length. It can therefore use the complete compound-page size for every scatterlist entry while keeping the total length equal to the buffer size. Shared backing allocations may need to be rounded up to an architecture shared granule size. A backing allocation can then be larger than the remaining buffer length. Describing the complete allocation in the scatterlist would incorrectly expose the rounded tail to scatterlist consumers as part of the dma-buf. Track the remaining buffer length while constructing the scatterlist and limit each entry to the smaller of the compound-page size and the remaining length. The complete backing allocation remains owned by the heap and is still released normally. This does not change behavior with the current allocation policy, but prepares the heap for shared-granule-sized backing allocations. Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/dma-buf/heaps/system_heap.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/sy= stem_heap.c index c8959eadc71d..b5b8cdf65f23 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -406,6 +406,7 @@ static struct dma_buf *system_heap_allocate(struct dma_= heap *heap, struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); unsigned long size_remaining =3D len; + unsigned long sg_remaining =3D len; unsigned int max_order =3D orders[0]; struct system_heap_priv *priv =3D dma_heap_get_drvdata(heap); bool cc_shared =3D priv->cc_shared; @@ -454,7 +455,11 @@ static struct dma_buf *system_heap_allocate(struct dma= _heap *heap, =20 sg =3D table->sgl; list_for_each_entry_safe(page, tmp_page, &pages, lru) { - sg_set_page(sg, page, page_size(page), 0); + unsigned long sg_len; + + sg_len =3D min_t(unsigned long, page_size(page), sg_remaining); + sg_set_page(sg, page, sg_len, 0); + sg_remaining -=3D sg_len; sg =3D sg_next(sg); list_del(&page->lru); } --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC8B24A99D6; Mon, 21 Sep 2026 14:50:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002255; cv=none; b=WcV0LJkcGtyy1BQNqfJm51jLvDaeJCElrcUjrZ9YVZaG/KKlchxFlm/HzQ546gdBCiRGjdYZF5FO4PrMcWDKYFY+HQOOZEXs0H5+lteV/sV04jlvjd72idJ/jQ4I/JauzfIfcVNFtJgd3PqxlcPWxWPDq7JXYbSy5SEVUB48WjI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002255; c=relaxed/simple; bh=0nxO8GAQTby2dzIN+fHvmidM+eXHnBKAqX9CYKt9hSY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p/qm9Kh+XUxVQTFCQCsgBudBabK0vaZEGJ4uU+/HNUW3cwnf6xPcB8roowsAniF72zwo7M5jnWuZVhORzP9RzbYOkJZYHkWssi6CcSpc1rmoKM/1L4quPvqkODiBaad8Gscf7Hy0iQVkIJaVA8bMedZc/lf4i3jyuuTbQsLSsf4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ck3o087i; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ck3o087i" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B14E11F000FF; Mon, 21 Sep 2026 14:50:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002253; bh=8DvGDN8B0fy3MEMDtp1UmGcUak3xg58IFiLeBCZo6/s=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ck3o087iBTLP7bnheoTSKrQKTTXclUbW2E1CiIgXY+3mdFm01bdkgut73uwzE6lIL 2/XR+olitu0IQTYInC0J5i0aMJD6w4/EDRujKFVYQPApzNKARiAGFeW5mrMvVmRmb7 2PteYvQDRNHihMaerqOzcRzfADlFMW5jlIKe+5ZoTKEpBjQNdr6SPOM+9u8BtTKpv1 ka1eYnY1zSi/jv3PQugkhsJddm8+icaUWp6VGnHNLpfq6qsGSbqxp77oReEVHjdAAv Ou64udbe8N3tWZhe/8Fv/R2AnEjkcfneRo5XIFwQhu0mj1hVCHmeHn22+hNCJxuzVY h9lKR+t9FlBLw== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 12/13] dma-buf: system_heap: Allocate shared buffers using CoCo shared memory allocator Date: Mon, 21 Sep 2026 20:18:46 +0530 Message-ID: <20260921144847.501151-13-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The system_cc_shared heap currently allocates its backing pages using the normal system-heap order policy and changes each resulting compound page to shared state. That is unsafe when an architecture requires state changes in units larger than PAGE_SIZE: an order-0 tail is neither sufficiently aligned nor large enough to transition independently. Use the common CoCo shared-page allocator for every backing allocation of t= he shared heap. Preserve the existing preferred-order search and its GFP polic= y: each candidate order is passed to the common allocator as a byte request, a= nd that allocator rounds it up when the architecture shared granule is larger. Add __GFP_COMP for shared allocations because an order-0 candidate can be rounded into a high-order allocation. The system heap uses compound_order() and page_size() for accounting and release, so the returned allocation must retain compound-page semantics. Calculate a rounded internal backing length but retain the original length in dma_buf::size. The preceding scatterlist-length change ensures that the rounded tail is not included in DMA mappings or other operations. This permits a 4 KiB request on a 64 KiB shared-granule system without exposing the extra 60 KiB to an importer. Signed-off-by: Aneesh Kumar K.V (Arm) --- drivers/dma-buf/heaps/system_heap.c | 119 ++++++++++++---------------- 1 file changed, 51 insertions(+), 68 deletions(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/sy= stem_heap.c index b5b8cdf65f23..35ac029dc41b 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -11,14 +11,13 @@ */ =20 #include +#include #include #include #include #include #include -#include #include -#include #include #include #include @@ -65,34 +64,6 @@ static gfp_t order_flags[] =3D {HIGH_ORDER_GFP, HIGH_ORD= ER_GFP, LOW_ORDER_GFP}; static const unsigned int orders[] =3D {8, 4, 0}; #define NUM_ORDERS ARRAY_SIZE(orders) =20 -static int system_heap_set_page_decrypted(struct page *page) -{ - unsigned long addr =3D (unsigned long)page_address(page); - unsigned int nr_pages =3D 1 << compound_order(page); - int ret; - - ret =3D set_memory_decrypted(addr, nr_pages); - if (ret) - pr_warn_ratelimited("dma-buf system heap: failed to decrypt page at %p\n= ", - page_address(page)); - - return ret; -} - -static int system_heap_set_page_encrypted(struct page *page) -{ - unsigned long addr =3D (unsigned long)page_address(page); - unsigned int nr_pages =3D 1 << compound_order(page); - int ret; - - ret =3D set_memory_encrypted(addr, nr_pages); - if (ret) - pr_warn_ratelimited("dma-buf system heap: failed to re-encrypt page at %= p, leaking memory\n", - page_address(page)); - - return ret; -} - static int dup_sg_table(struct sg_table *from, struct sg_table *to) { struct scatterlist *sg, *new_sg; @@ -337,6 +308,20 @@ static void system_heap_vunmap(struct dma_buf *dmabuf,= struct iosys_map *map) iosys_map_clear(map); } =20 +static void system_heap_free_page(struct page *page, bool cc_shared) +{ + struct cc_shared_pages mem; + + if (!cc_shared) { + __free_pages(page, compound_order(page)); + return; + } + + mem.page =3D page; + mem.shared_size =3D page_size(page); + free_cc_shared_pages(&mem); +} + static void system_heap_dma_buf_release(struct dma_buf *dmabuf) { struct system_heap_buffer *buffer =3D dmabuf->priv; @@ -345,19 +330,8 @@ static void system_heap_dma_buf_release(struct dma_buf= *dmabuf) int i; =20 table =3D &buffer->sg_table; - for_each_sgtable_sg(table, sg, i) { - struct page *page =3D sg_page(sg); - - /* - * Intentionally leak pages that cannot be re-encrypted - * to prevent shared memory from being reused. - */ - if (cc_shared_buffer(buffer) && - system_heap_set_page_encrypted(page)) - continue; - - __free_pages(page, compound_order(page)); - } + for_each_sgtable_sg(table, sg, i) + system_heap_free_page(sg_page(sg), cc_shared_buffer(buffer)); sg_free_table(table); kfree(buffer); } @@ -375,22 +349,39 @@ static const struct dma_buf_ops system_heap_buf_ops = =3D { .release =3D system_heap_dma_buf_release, }; =20 +static struct page *system_heap_alloc_order(unsigned int order, + gfp_t flags, bool cc_shared) +{ + struct cc_shared_pages mem; + + if (!cc_shared) + return alloc_pages(flags, order); + + /* The shared granule can raise the actual allocation order. */ + flags |=3D __GFP_COMP; + if (alloc_cc_shared_pages(flags, PAGE_SIZE << order, &mem)) + return NULL; + + return mem.page; +} + static struct page *alloc_largest_available(unsigned long size, - unsigned int max_order) + unsigned int max_order, bool cc_shared) { struct page *page; - int i; gfp_t flags; + int i; =20 for (i =3D 0; i < NUM_ORDERS; i++) { if (size < (PAGE_SIZE << orders[i])) continue; if (max_order < orders[i]) continue; + flags =3D order_flags[i]; if (mem_accounting) flags |=3D __GFP_ACCOUNT; - page =3D alloc_pages(flags, orders[i]); + page =3D system_heap_alloc_order(orders[i], flags, cc_shared); if (!page) continue; return page; @@ -405,6 +396,7 @@ static struct dma_buf *system_heap_allocate(struct dma_= heap *heap, { struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); + struct cc_shared_layout layout; unsigned long size_remaining =3D len; unsigned long sg_remaining =3D len; unsigned int max_order =3D orders[0]; @@ -417,6 +409,14 @@ static struct dma_buf *system_heap_allocate(struct dma= _heap *heap, struct page *page, *tmp_page; int i, ret =3D -ENOMEM; =20 + if (cc_shared) { + ret =3D cc_shared_calc_layout(len, &layout); + if (ret) + return ERR_PTR(ret); + + size_remaining =3D layout.shared_size; + } + buffer =3D kzalloc_obj(*buffer); if (!buffer) return ERR_PTR(-ENOMEM); @@ -439,7 +439,8 @@ static struct dma_buf *system_heap_allocate(struct dma_= heap *heap, goto free_buffer; } =20 - page =3D alloc_largest_available(size_remaining, max_order); + page =3D alloc_largest_available(size_remaining, max_order, + cc_shared); if (!page) goto free_buffer; =20 @@ -464,14 +465,6 @@ static struct dma_buf *system_heap_allocate(struct dma= _heap *heap, list_del(&page->lru); } =20 - if (cc_shared_buffer(buffer)) { - for_each_sgtable_sg(table, sg, i) { - ret =3D system_heap_set_page_decrypted(sg_page(sg)); - if (ret) - goto free_pages; - } - } - /* create the dmabuf */ exp_info.exp_name =3D dma_heap_get_name(heap); exp_info.ops =3D &system_heap_buf_ops; @@ -486,22 +479,12 @@ static struct dma_buf *system_heap_allocate(struct dm= a_heap *heap, return dmabuf; =20 free_pages: - for_each_sgtable_sg(table, sg, i) { - struct page *p =3D sg_page(sg); - - /* - * Intentionally leak pages that cannot be re-encrypted - * to prevent shared memory from being reused. - */ - if (cc_shared_buffer(buffer) && - system_heap_set_page_encrypted(p)) - continue; - __free_pages(p, compound_order(p)); - } + for_each_sgtable_sg(table, sg, i) + system_heap_free_page(sg_page(sg), cc_shared); sg_free_table(table); free_buffer: list_for_each_entry_safe(page, tmp_page, &pages, lru) - __free_pages(page, compound_order(page)); + system_heap_free_page(page, cc_shared); kfree(buffer); =20 return ERR_PTR(ret); --=20 2.43.0 From nobody Thu Sep 24 18:37:41 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9F00490C04; Mon, 21 Sep 2026 14:51:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002266; cv=none; b=lbOsz2S9C6N+Rge1mhNsjoGGFDayb35sTaWE3mkUE9uSxsmGHt0Rs6RGQ37IP7uvc56ySALgJ33OrWgWRLpyQFt9GQqHP91V9pWw7Gd8Fjx1+MQST0K+ufAFp3MPEsII6jIUWGQkEGz9BzdobEvc4Lv4Jr5oE6fSgIvVeniNAxk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790002266; c=relaxed/simple; bh=9aqYdx3/aCoXRLrN6ORA+acXRHkWhUu8fsADCxSSXJY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=t/gAhRUodKh3IRtfKQQdXQFVRVj4mTpcREhSPypKCMh0GoKYBlrS/P/oo2KwC6zrR2GDfE+I/1Sz9rjahZZuT5bIi4cLwvxXOqDOFhLdsLUW7Ob/l/R/4DMcQBcQYJ3RkHY+yoVmvVb/gdGsGC6NGqt51nGRoWTRdDaIRXCPDE8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=U2t85MqE; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="U2t85MqE" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 77EE21F00893; Mon, 21 Sep 2026 14:50:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790002264; bh=RIpxqITDfA7RmolIrlPIbBKnqZiSPFBAJdQmg5tAnIk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=U2t85MqEar9/QE6fbQs7GNiO7mB2uSgtDrQex0/73Ec54N3fnUaqTEvimBm4eBh5X Mx7reovji1y0h9moEFn5hOC1cwh4mQ1xNkSDZv1aKe5jZbyNPpY6qd8s0qXiiaepDg vpI3sL1MsD2yNWaA2h7v2fj/Z0cfBMWQRwFa487uO6sTn+Z1INBuR7Ifd5ndf98yls 1incq5rJC0DTmnsm2F8SmMB0/9YCzOL3YnishM1Zuy9vjpkRuAbnWCnqfuUK0wHZGb L725hd/X0O6zpYej/UqSZ4PFCbDhcjLR9kbVZ/qZxUXm6yYazpLKIenCh2gvu2BCwV IlJ65pPpNT/eg== From: "Aneesh Kumar K.V (Arm)" To: linux-coco@lists.linux.dev, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev Cc: "Aneesh Kumar K.V (Arm)" , Andrew Morton , Catalin Marinas , christian.koenig@amd.com, Jason Gunthorpe , Joerg Roedel , Marc Zyngier , Marek Szyprowski , Robin Murphy , Steven Price , Sumit Semwal , Suzuki K Poulose , Thomas Gleixner , Will Deacon , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v7 13/13] swiotlb: Make rounded shared pool capacity allocatable Date: Mon, 21 Sep 2026 20:18:47 +0530 Message-ID: <20260921144847.501151-14-aneesh.kumar@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260921144847.501151-1-aneesh.kumar@kernel.org> References: <20260921144847.501151-1-aneesh.kumar@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" CoCo shared memory may need to be allocated and transitioned in units larger than the requested object. Before this change, users handled the resulting capacity as follows: User Rounded capacity reused dma-buf system heap no DMA-direct no regular GIC tables no small GIC ITTs yes, through a gen_pool early SWIOTLB pool no late SWIOTLB pool yes persistent dynamic SWIOTLB no transient dynamic SWIOTLB no, one mapping only atomic DMA pools yes, through a gen_pool restricted SWIOTLB pool no additional padding Improve the early and persistent dynamic SWIOTLB pools. They already own and transition backing rounded to the shared granule size, and SWIOTLB is itself a suballocator. Advertise the rounded extent as slots, size the slot metadata to match. This makes the extra capacity available without reserving more backing memory. Keep transient dynamic pools unchanged. A transient pool belongs to one DMA mapping and is destroyed when that mapping is unmapped, so its spare backing cannot satisfy a later request without changing the lifetime model. Do not attempt the same optimization for dma-buf, DMA-direct or regular GIC objects. Those allocations have independent caller-visible sizes and lifetimes. Reusing their padding requires a shared-granule suballocator with reference counting, per-object mappings and accounting. Note: For the current 64 KiB CCA shared granule size, SWIOTLB pool sizes are already multiples of the 256 KiB IO_TLB segment size. Consequently, the rounding does not change any runtime values on current CCA systems. It instead makes the code express the intended invariant that pool metadata describes the complete shared-granule-aligned backing allocation. Signed-off-by: Aneesh Kumar K.V (Arm) --- kernel/dma/swiotlb.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index cb67105b8812..9577a8807b07 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -330,6 +330,14 @@ static inline unsigned long nr_slots(u64 val) return DIV_ROUND_UP(val, IO_TLB_SIZE); } =20 +static unsigned long swiotlb_align_nslabs(unsigned long nslabs) +{ + unsigned long granule_nslabs; + + granule_nslabs =3D cc_shared_granule_size() >> IO_TLB_SHIFT; + return ALIGN(nslabs, granule_nslabs); +} + static void swiotlb_mark_pool_used(struct io_tlb_pool *pool) { unsigned long i; @@ -435,11 +443,12 @@ static void add_mem_pool(struct io_tlb_mem *mem, stru= ct io_tlb_pool *pool) } =20 static void __init *swiotlb_memblock_alloc(unsigned long nslabs, - unsigned int flags, + unsigned long *alloc_nslabs, unsigned int flags, int (*remap)(void *tlb, unsigned long nslabs)) { + unsigned long aligned_nslabs =3D swiotlb_align_nslabs(nslabs); + size_t bytes =3D aligned_nslabs << IO_TLB_SHIFT; void *tlb; - size_t bytes =3D ALIGN(nslabs << IO_TLB_SHIFT, cc_shared_granule_size()); =20 /* * By default allocate the bounce buffer memory from low memory, but @@ -457,12 +466,13 @@ static void __init *swiotlb_memblock_alloc(unsigned l= ong nslabs, return NULL; } =20 - if (remap && remap(tlb, nslabs) < 0) { + if (remap && remap(tlb, aligned_nslabs) < 0) { memblock_free(tlb, bytes); pr_warn("%s: Failed to remap %zu bytes\n", __func__, bytes); return NULL; } =20 + *alloc_nslabs =3D aligned_nslabs; return tlb; } =20 @@ -475,6 +485,7 @@ void __init swiotlb_init_remap(bool addressing_limit, u= nsigned int flags, { struct io_tlb_pool *mem =3D &io_tlb_default_mem.defpool; unsigned long nslabs; + unsigned long alloc_nslabs; unsigned int nareas; size_t alloc_size; void *tlb; @@ -499,13 +510,14 @@ void __init swiotlb_init_remap(bool addressing_limit,= unsigned int flags, swiotlb_adjust_nareas(num_possible_cpus()); =20 nslabs =3D default_nslabs; - nareas =3D limit_nareas(default_nareas, nslabs); - while ((tlb =3D swiotlb_memblock_alloc(nslabs, flags, remap)) =3D=3D NULL= ) { + while ((tlb =3D swiotlb_memblock_alloc(nslabs, &alloc_nslabs, flags, + remap)) =3D=3D NULL) { if (nslabs <=3D IO_TLB_MIN_SLABS) return; nslabs =3D ALIGN(nslabs >> 1, IO_TLB_SEGSIZE); - nareas =3D limit_nareas(nareas, nslabs); } + nslabs =3D alloc_nslabs; + nareas =3D limit_nareas(default_nareas, nslabs); =20 if (default_nslabs !=3D nslabs) { pr_info("SWIOTLB bounce buffer size adjusted %lu -> %lu slabs", @@ -871,6 +883,12 @@ static struct io_tlb_pool *swiotlb_alloc_pool(struct d= evice *dev, tlb_size =3D nslabs << IO_TLB_SHIFT; } =20 + /* Transient pools are tied to one mapping and cannot reuse padding. */ + if (mem->cc_shared && !dev) { + nslabs =3D swiotlb_align_nslabs(nslabs); + tlb_size =3D nslabs << IO_TLB_SHIFT; + } + slot_order =3D get_order(array_size(sizeof(*pool->slots), nslabs)); pool->slots =3D (struct io_tlb_slot *) __get_free_pages(gfp, slot_order); --=20 2.43.0