From nobody Sun May 24 20:33:27 2026 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DEEE43A7822 for ; Thu, 21 May 2026 20:59:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779397152; cv=none; b=j5GQ/sXJekQo/CRQ9+EwRTcmoP/H+FIizUKkSfhKKibAhgehbs6Nb75zsS/Y+iloLpLXm1HjR7duItNkMp2mM5VdXJFhQbHaVl4OHyGNdMQdr6BSPMvWWgk4Dtqpof+x6lH6ri7SVA+Zws5LC+MQyevxgGylkDjD74tfzCT/yn8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779397152; c=relaxed/simple; bh=TDoKRg10lElIvDp7LbnUDQEt9G9Gb00Gv0MOMuiiRno=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=ZuIqD9jBiyetZ5tvWpmz+RBienyx35Oo5DkUe6kvx77q6jWpwQlQDtqj3HFOil0vVWe7TZx926TphSYZ3vbuBF7xVhmyUdzeLMIdq6u9CZEfFs3XcAVpnBb5TyCva3ULkSaJewvnIqexzTUlOlUZjTlQbcqftels6rtcbGQ1kfs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b=gyZvdHKT; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.microsoft.com header.i=@linux.microsoft.com header.b="gyZvdHKT" Received: from linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net (linux.microsoft.com [13.77.154.182]) by linux.microsoft.com (Postfix) with ESMTPSA id E17FA20B7167; Thu, 21 May 2026 13:59:02 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com E17FA20B7167 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1779397142; bh=zZjL9udFwn6W4wb/ID8EFAgii+j45nXdSF7vNb7mlSY=; h=From:To:Cc:Subject:Date:From; b=gyZvdHKTjs+bUew1d1t06iSTvj3AkZsIvPu3cLKcknyeHxF0Bcg0Kerz9Lp4dG5T+ 08vIqN/seEQRXM8wbrbiEK7BriT6jAIYR7mWLoDYYKGeRYZ5sFEXwK41b1fgtnflZZ uHbdwDIAXZVQwosV0oEcLRoKj0PsZQ2ueZi04+ow= From: Kameron Carr To: akpm@linux-foundation.org, urezki@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, rppt@kernel.org, catalin.marinas@arm.com, mhklinux@outlook.com Subject: [RFC PATCH] mm/vmalloc: add vmalloc_decrypted() and vzalloc_decrypted() Date: Thu, 21 May 2026 13:58:34 -0700 Message-ID: <20260521205834.1012925-1-kameroncarr@linux.microsoft.com> X-Mailer: git-send-email 2.43.7 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In confidential computing environments (arm64 CCA, x86 SEV/TDX), guest memory is encrypted by default and must be explicitly transitioned to a decrypted/shared state for host-visible access. Calling set_memory_decrypted() on a vmalloc address is not supported, and not recommended as it would be inefficient to decrypt the pages after they have been mapped. Add vmalloc_decrypted() and vzalloc_decrypted() which decrypt pages on the linear map before creating the vmalloc mapping via vmap(), so physical pages are never mapped with conflicting encryption attributes across aliases. A new VM_DECRYPTED flag marks these allocations so that vfree() automatically re-encrypts pages before returning them to the page allocator. Suggested-by: Catalin Marinas Link: https://lore.kernel.org/linux-arm-kernel/ZmNJdSxSz-sYpVgI@arm.com/ Signed-off-by: Kameron Carr --- include/linux/vmalloc.h | 7 ++ mm/vmalloc.c | 163 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 170 insertions(+) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 3b02c0c6b371..d87e1953da55 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -38,6 +38,7 @@ struct iov_iter; /* in uio.h */ #define VM_DEFER_KMEMLEAK 0 #endif #define VM_SPARSE 0x00001000 /* sparse vm_area. not all pages are present= . */ +#define VM_DECRYPTED 0x00002000 /* pages decrypted for host-shared access= , re-encrypt on vfree */ =20 /* bits [20..32] reserved for arch specific ioremap internals */ =20 @@ -153,6 +154,12 @@ extern void *vmalloc_noprof(unsigned long size) __allo= c_size(1); extern void *vzalloc_noprof(unsigned long size) __alloc_size(1); #define vzalloc(...) alloc_hooks(vzalloc_noprof(__VA_ARGS__)) =20 +extern void *vmalloc_decrypted_noprof(unsigned long size) __alloc_size(1); +#define vmalloc_decrypted(...) alloc_hooks(vmalloc_decrypted_noprof(__VA_A= RGS__)) + +extern void *vzalloc_decrypted_noprof(unsigned long size) __alloc_size(1); +#define vzalloc_decrypted(...) alloc_hooks(vzalloc_decrypted_noprof(__VA_A= RGS__)) + extern void *vmalloc_user_noprof(unsigned long size) __alloc_size(1); #define vmalloc_user(...) alloc_hooks(vmalloc_user_noprof(__VA_ARGS__)) =20 diff --git a/mm/vmalloc.c b/mm/vmalloc.c index eabb86b13b7e..0e7f0033aa84 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3416,6 +3416,103 @@ void vfree_atomic(const void *addr) schedule_work(&p->wq); } =20 +/* + * Transition a single contiguous block of @nr pages at index @idx in + * @area->pages to encrypted or decrypted state. On failure, the block's + * page-pointer slots are cleared so the standard free path will not return + * the pages to the allocator (they are leaked). + */ +static int __vm_pages_enc_dec(struct vm_struct *area, unsigned int idx, + unsigned int nr, bool encrypt) +{ + unsigned long addr =3D + (unsigned long)kasan_reset_tag(page_address(area->pages[idx])); + int err =3D encrypt ? set_memory_encrypted(addr, nr) : + set_memory_decrypted(addr, nr); + + if (err) + memset(&area->pages[idx], 0, nr * sizeof(*area->pages)); + return err; +} + +/* + * Compact @area->pages, removing slots previously zeroed by + * __vm_pages_enc_dec(). Returns the number of leaked pages + * (old nr_pages - new nr_pages). + */ +static unsigned int vm_compact_leaked_pages(struct vm_struct *area) +{ + unsigned int i, dst; + unsigned int old_nr =3D area->nr_pages; + + for (i =3D 0, dst =3D 0; i < area->nr_pages; i++) { + if (area->pages[i]) + area->pages[dst++] =3D area->pages[i]; + } + area->nr_pages =3D dst; + return old_nr - dst; +} + +/* + * Re-encrypt the linear-map alias of all pages backing a VM_DECRYPTED are= a. + * Best-effort: on per-block failure the loop continues so as many pages as + * possible are returned to the encrypted state. Pages that fail to + * transition are left out of area->pages and leaked. + */ +static int vm_pages_encrypt(struct vm_struct *area) +{ + unsigned int nr =3D 1U << vm_area_page_order(area); + unsigned int i; + unsigned int leaked; + int ret =3D 0; + + for (i =3D 0; i < area->nr_pages; i +=3D nr) { + int err =3D __vm_pages_enc_dec(area, i, nr, true); + + if (err && !ret) + ret =3D err; + } + + leaked =3D vm_compact_leaked_pages(area); + if (leaked) + pr_warn("vmalloc: re-encryption failed, leaked %u pages\n", + leaked); + return ret; +} + +/* + * Decrypt the linear-map alias of all pages backing a VM_DECRYPTED area. + * On failure, the already-decrypted prefix is rolled back to encrypted. + * Pages that fail either the initial decrypt or the rollback re-encrypt a= re + * left out of area->pages and leaked. + */ +static int vm_pages_decrypt(struct vm_struct *area) +{ + unsigned int nr =3D 1U << vm_area_page_order(area); + unsigned int i; + unsigned int leaked; + int ret =3D 0; + + for (i =3D 0; i < area->nr_pages; i +=3D nr) { + ret =3D __vm_pages_enc_dec(area, i, nr, false); + if (ret) + goto rollback; + } + return 0; + +rollback: + while (i) { + i -=3D nr; + __vm_pages_enc_dec(area, i, nr, true); + } + + leaked =3D vm_compact_leaked_pages(area); + if (leaked) + pr_warn("vmalloc: decryption failed, leaked %u pages\n", + leaked); + return ret; +} + /** * vfree - Release memory allocated by vmalloc() * @addr: Memory base address @@ -3457,6 +3554,9 @@ void vfree(const void *addr) return; } =20 + if (unlikely(vm->flags & VM_DECRYPTED)) + vm_pages_encrypt(vm); + if (unlikely(vm->flags & VM_FLUSH_RESET_PERMS)) vm_reset_perms(vm); =20 @@ -3895,6 +3995,22 @@ static void *__vmalloc_area_node(struct vm_struct *a= rea, gfp_t gfp_mask, goto fail; } =20 + /* + * For VM_DECRYPTED areas, decrypt each + * page on the linear map before creating the vmalloc alias. + */ + if (area->flags & VM_DECRYPTED) { + if (vm_pages_decrypt(area)) { + /* + * vm_pages_decrypt() re-encrypted what it could; + * clear VM_DECRYPTED so the deferred cleanup path + * doesn't try to re-encrypt again. + */ + area->flags &=3D ~VM_DECRYPTED; + goto fail; + } + } + /* * page tables allocations ignore external gfp mask, enforce it * by the scope API @@ -4203,6 +4319,50 @@ void *vzalloc_noprof(unsigned long size) } EXPORT_SYMBOL(vzalloc_noprof); =20 +/** + * vmalloc_decrypted - allocate virtually contiguous decrypted memory + * @size: allocation size + * + * Allocate pages in decrypted/shared state for host-visible access in + * confidential computing environments. Pages are automatically + * re-encrypted on vfree(). + * + * Return: pointer to the allocated memory or %NULL on error + */ +void *vmalloc_decrypted_noprof(unsigned long size) +{ + return __vmalloc_node_range_noprof(size, 1, VMALLOC_START, VMALLOC_END, + GFP_KERNEL, + pgprot_decrypted(PAGE_KERNEL), + VM_DECRYPTED, NUMA_NO_NODE, + __builtin_return_address(0)); +} +EXPORT_SYMBOL(vmalloc_decrypted_noprof); + +/** + * vzalloc_decrypted - allocate zeroed virtually contiguous decrypted memo= ry + * @size: allocation size + * + * Like vmalloc_decrypted(), but the memory is set to zero. + * + * Return: pointer to the allocated memory or %NULL on error + */ +void *vzalloc_decrypted_noprof(unsigned long size) +{ + void *addr; + + addr =3D __vmalloc_node_range_noprof(size, 1, VMALLOC_START, VMALLOC_END, + GFP_KERNEL, + pgprot_decrypted(PAGE_KERNEL), + VM_DECRYPTED, NUMA_NO_NODE, + __builtin_return_address(0)); + if (addr) + memset(addr, 0, size); + + return addr; +} +EXPORT_SYMBOL(vzalloc_decrypted_noprof); + /** * vmalloc_user - allocate zeroed virtually contiguous memory for userspace * @size: allocation size @@ -5271,6 +5431,9 @@ static int vmalloc_info_show(struct seq_file *m, void= *p) if (v->flags & VM_DMA_COHERENT) seq_puts(m, " dma-coherent"); =20 + if (v->flags & VM_DECRYPTED) + seq_puts(m, " decrypted"); + if (is_vmalloc_addr(v->pages)) seq_puts(m, " vpages"); =20 base-commit: e9add7501ad3297dad9b90ce201266830a68ab47 --=20 2.45.4