From nobody Tue Jan 27 00:13:48 2026 Received: from fra-out-002.esa.eu-central-1.outbound.mail-perimeter.amazon.com (fra-out-002.esa.eu-central-1.outbound.mail-perimeter.amazon.com [3.65.3.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 36965349B0F; Mon, 26 Jan 2026 16:47:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=3.65.3.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769446057; cv=none; b=SFBg2jmnFb5HSI7rPmwAexzD8lQiaJhJvfc0t8gRg7MoebvwFhwDscSdNsBhB4VUrcs9WC/gTxDZZFyhlv19u4dipvT1D932RzunIFBkdRKOGu609b/MDU2fMSnu9s/cCP33UXNYoTR3IuWYI7eaPl6TkYatGaOIDR8V/XMCQ10= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769446057; c=relaxed/simple; bh=ZXlEDo/g6nxMcchNZQF+SNFo2CnMNwUgdNVKQQ5GxEk=; h=From:To:CC:Subject:Date:Message-ID:References:In-Reply-To: Content-Type:MIME-Version; b=EBczFyNnRmRcPT299cqZbGIN35qY0pif8pPssJJ3/bVyiFmBCPZIurPK2mWQQuI0Yc1b0a2srhBdGW/DJaCWtlV/o1KW2FKQrFFH7hbTYFCV6tbAibQt8dUi52Riv7XfjmXp250657Huv4rbTpkExTsHll6fMqgBa8Z5BnQM+XY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.co.uk; spf=pass smtp.mailfrom=amazon.co.uk; dkim=pass (2048-bit key) header.d=amazon.co.uk header.i=@amazon.co.uk header.b=mI321WdA; arc=none smtp.client-ip=3.65.3.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.co.uk Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.co.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=amazon.co.uk header.i=@amazon.co.uk header.b="mI321WdA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.co.uk; i=@amazon.co.uk; q=dns/txt; s=amazoncorp2; t=1769446053; x=1800982053; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=itJUHovmx8ThQnuXFjcLy1G60dRyIFRRQr44NmgQ9VQ=; b=mI321WdAFo+KMRXPJtUnPVSbqTxfMiU9hexlA+6c22cLS8I0EZ5xZkj+ TpKC9gTvy1FpfZYfwx6OXYknverV2A4U+dq7ctwG3re+9v7/MQzPfSjeB IpqRxmsYLEoEsDh1hx7cQ2pSwfMN2QrtqwLLl3oPg8WgNrGhNlP7VAuGY nD5sq2rjCgAcLJefOSAp/zGzSZBuqU1yxIuWQSZBm9MVsZ2pxVfbz2QHB KHwLygYwmUDbFUUjKO8WUk2OsVewy4oElZuFg49ITlg3IQCq4S2L0HWXE ATRHU4GEC8P/5Mx/OS3i6GFj8rfXEXb+2d1jRRWPH6g9ys7uksDFCBj6D g==; X-CSE-ConnectionGUID: mAl7W5q2Q9qKqPQn17E9NQ== X-CSE-MsgGUID: vLAaNxp6S+iNE/SUHLTamA== X-IronPort-AV: E=Sophos;i="6.21,255,1763424000"; d="scan'208";a="8460917" Received: from ip-10-6-11-83.eu-central-1.compute.internal (HELO smtpout.naws.eu-central-1.prod.farcaster.email.amazon.dev) ([10.6.11.83]) by internal-fra-out-002.esa.eu-central-1.outbound.mail-perimeter.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jan 2026 16:47:12 +0000 Received: from EX19MTAEUC001.ant.amazon.com [54.240.197.233:20657] by smtpin.naws.eu-central-1.prod.farcaster.email.amazon.dev [10.0.13.191:2525] with esmtp (Farcaster) id 7aedb640-8db8-4ae8-804d-436f487cb697; Mon, 26 Jan 2026 16:47:12 +0000 (UTC) X-Farcaster-Flow-ID: 7aedb640-8db8-4ae8-804d-436f487cb697 Received: from EX19D005EUB002.ant.amazon.com (10.252.51.103) by EX19MTAEUC001.ant.amazon.com (10.252.51.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.35; Mon, 26 Jan 2026 16:47:12 +0000 Received: from EX19D005EUB003.ant.amazon.com (10.252.51.31) by EX19D005EUB002.ant.amazon.com (10.252.51.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.2562.35; Mon, 26 Jan 2026 16:47:11 +0000 Received: from EX19D005EUB003.ant.amazon.com ([fe80::b825:becb:4b38:da0c]) by EX19D005EUB003.ant.amazon.com ([fe80::b825:becb:4b38:da0c%3]) with mapi id 15.02.2562.035; Mon, 26 Jan 2026 16:47:11 +0000 From: "Kalyazin, Nikita" To: "kvm@vger.kernel.org" , "linux-doc@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" , "kvmarm@lists.linux.dev" , "linux-fsdevel@vger.kernel.org" , "linux-mm@kvack.org" , "bpf@vger.kernel.org" , "linux-kselftest@vger.kernel.org" , "kernel@xen0n.name" , "linux-riscv@lists.infradead.org" , "linux-s390@vger.kernel.org" , "loongarch@lists.linux.dev" CC: "pbonzini@redhat.com" , "corbet@lwn.net" , "maz@kernel.org" , "oupton@kernel.org" , "joey.gouly@arm.com" , "suzuki.poulose@arm.com" , "yuzenghui@huawei.com" , "catalin.marinas@arm.com" , "will@kernel.org" , "seanjc@google.com" , "tglx@kernel.org" , "mingo@redhat.com" , "bp@alien8.de" , "dave.hansen@linux.intel.com" , "x86@kernel.org" , "hpa@zytor.com" , "luto@kernel.org" , "peterz@infradead.org" , "willy@infradead.org" , "akpm@linux-foundation.org" , "david@kernel.org" , "lorenzo.stoakes@oracle.com" , "vbabka@suse.cz" , "rppt@kernel.org" , "surenb@google.com" , "mhocko@suse.com" , "ast@kernel.org" , "daniel@iogearbox.net" , "andrii@kernel.org" , "martin.lau@linux.dev" , "eddyz87@gmail.com" , "song@kernel.org" , "yonghong.song@linux.dev" , "john.fastabend@gmail.com" , "kpsingh@kernel.org" , "sdf@fomichev.me" , "haoluo@google.com" , "jolsa@kernel.org" , "jgg@ziepe.ca" , "jhubbard@nvidia.com" , "peterx@redhat.com" , "jannh@google.com" , "pfalcato@suse.de" , "shuah@kernel.org" , "riel@surriel.com" , "ryan.roberts@arm.com" , "jgross@suse.com" , "yu-cheng.yu@intel.com" , "kas@kernel.org" , "coxu@redhat.com" , "kevin.brodsky@arm.com" , "ackerleytng@google.com" , "maobibo@loongson.cn" , "prsampat@amd.com" , "mlevitsk@redhat.com" , "jmattson@google.com" , "jthoughton@google.com" , "agordeev@linux.ibm.com" , "alex@ghiti.fr" , "aou@eecs.berkeley.edu" , "borntraeger@linux.ibm.com" , "chenhuacai@kernel.org" , "dev.jain@arm.com" , "gor@linux.ibm.com" , "hca@linux.ibm.com" , "palmer@dabbelt.com" , "pjw@kernel.org" , "shijie@os.amperecomputing.com" , "svens@linux.ibm.com" , "thuth@redhat.com" , "wyihan@google.com" , "yang@os.amperecomputing.com" , "Jonathan.Cameron@huawei.com" , "Liam.Howlett@oracle.com" , "urezki@gmail.com" , "zhengqi.arch@bytedance.com" , "gerald.schaefer@linux.ibm.com" , "jiayuan.chen@shopee.com" , "lenb@kernel.org" , "osalvador@suse.de" , "pavel@kernel.org" , "rafael@kernel.org" , "vannapurve@google.com" , "jackmanb@google.com" , "aneesh.kumar@kernel.org" , "patrick.roy@linux.dev" , "Thomson, Jack" , "Itazuri, Takahiro" , "Manwaring, Derek" , "Cali, Marco" , "Kalyazin, Nikita" Subject: [PATCH v10 02/15] set_memory: add folio_{zap,restore}_direct_map helpers Thread-Topic: [PATCH v10 02/15] set_memory: add folio_{zap,restore}_direct_map helpers Thread-Index: AQHcjuNqThDEVtMYx0OshrAVDnsQEg== Date: Mon, 26 Jan 2026 16:47:11 +0000 Message-ID: <20260126164445.11867-3-kalyazin@amazon.com> References: <20260126164445.11867-1-kalyazin@amazon.com> In-Reply-To: <20260126164445.11867-1-kalyazin@amazon.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: Content-Transfer-Encoding: quoted-printable Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" From: Nikita Kalyazin These allow guest_memfd to remove its memory from the direct map. Only implement them for architectures that have direct map. In folio_zap_direct_map(), flush TLB on architectures where set_direct_map_valid_noflush() does not flush it internally. The new helpers need to be accessible to KVM on architectures that support guest_memfd (x86 and arm64). Since arm64 does not support building KVM as a module, only export them on x86. Direct map removal gives guest_memfd the same protection that memfd_secret does, such as hardening against Spectre-like attacks through in-kernel gadgets. Reviewed-by: Ackerley Tng Signed-off-by: Nikita Kalyazin --- arch/arm64/include/asm/set_memory.h | 2 ++ arch/arm64/mm/pageattr.c | 12 ++++++++++++ arch/loongarch/include/asm/set_memory.h | 2 ++ arch/loongarch/mm/pageattr.c | 12 ++++++++++++ arch/riscv/include/asm/set_memory.h | 2 ++ arch/riscv/mm/pageattr.c | 12 ++++++++++++ arch/s390/include/asm/set_memory.h | 2 ++ arch/s390/mm/pageattr.c | 12 ++++++++++++ arch/x86/include/asm/set_memory.h | 2 ++ arch/x86/mm/pat/set_memory.c | 20 ++++++++++++++++++++ include/linux/set_memory.h | 10 ++++++++++ 11 files changed, 88 insertions(+) diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/s= et_memory.h index c71a2a6812c4..49fd54f3c265 100644 --- a/arch/arm64/include/asm/set_memory.h +++ b/arch/arm64/include/asm/set_memory.h @@ -15,6 +15,8 @@ int set_direct_map_invalid_noflush(const void *addr); int set_direct_map_default_noflush(const void *addr); int set_direct_map_valid_noflush(const void *addr, unsigned long numpages, bool valid); +int folio_zap_direct_map(struct folio *folio); +int folio_restore_direct_map(struct folio *folio); bool kernel_page_present(struct page *page); =20 int set_memory_encrypted(unsigned long addr, int numpages); diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c index e2bdc3c1f992..0b88b0344499 100644 --- a/arch/arm64/mm/pageattr.c +++ b/arch/arm64/mm/pageattr.c @@ -356,6 +356,18 @@ int set_direct_map_valid_noflush(const void *addr, uns= igned long numpages, return set_memory_valid((unsigned long)addr, numpages, valid); } =20 +int folio_zap_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), false); +} + +int folio_restore_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), true); +} + #ifdef CONFIG_DEBUG_PAGEALLOC /* * This is - apart from the return value - doing the same diff --git a/arch/loongarch/include/asm/set_memory.h b/arch/loongarch/inclu= de/asm/set_memory.h index 5e9b67b2fea1..1cdec6afe209 100644 --- a/arch/loongarch/include/asm/set_memory.h +++ b/arch/loongarch/include/asm/set_memory.h @@ -19,5 +19,7 @@ int set_direct_map_invalid_noflush(const void *addr); int set_direct_map_default_noflush(const void *addr); int set_direct_map_valid_noflush(const void *addr, unsigned long numpages, bool valid); +int folio_zap_direct_map(struct folio *folio); +int folio_restore_direct_map(struct folio *folio); =20 #endif /* _ASM_LOONGARCH_SET_MEMORY_H */ diff --git a/arch/loongarch/mm/pageattr.c b/arch/loongarch/mm/pageattr.c index c1b2be915038..be397fddc991 100644 --- a/arch/loongarch/mm/pageattr.c +++ b/arch/loongarch/mm/pageattr.c @@ -235,3 +235,15 @@ int set_direct_map_valid_noflush(const void *addr, uns= igned long numpages, =20 return __set_memory((unsigned long)addr, 1, set, clear); } + +int folio_zap_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), false); +} + +int folio_restore_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), true); +} diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/s= et_memory.h index a87eabd7fc78..208755d9d45e 100644 --- a/arch/riscv/include/asm/set_memory.h +++ b/arch/riscv/include/asm/set_memory.h @@ -44,6 +44,8 @@ int set_direct_map_invalid_noflush(const void *addr); int set_direct_map_default_noflush(const void *addr); int set_direct_map_valid_noflush(const void *addr, unsigned long numpages, bool valid); +int folio_zap_direct_map(struct folio *folio); +int folio_restore_direct_map(struct folio *folio); bool kernel_page_present(struct page *page); =20 #endif /* __ASSEMBLER__ */ diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c index 0a457177a88c..9a8237658c48 100644 --- a/arch/riscv/mm/pageattr.c +++ b/arch/riscv/mm/pageattr.c @@ -402,6 +402,18 @@ int set_direct_map_valid_noflush(const void *addr, uns= igned long numpages, return __set_memory((unsigned long)addr, numpages, set, clear); } =20 +int folio_zap_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), false); +} + +int folio_restore_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), true); +} + #ifdef CONFIG_DEBUG_PAGEALLOC static int debug_pagealloc_set_page(pte_t *pte, unsigned long addr, void *= data) { diff --git a/arch/s390/include/asm/set_memory.h b/arch/s390/include/asm/set= _memory.h index 3e43c3c96e67..a51ff50df3ca 100644 --- a/arch/s390/include/asm/set_memory.h +++ b/arch/s390/include/asm/set_memory.h @@ -64,6 +64,8 @@ int set_direct_map_invalid_noflush(const void *addr); int set_direct_map_default_noflush(const void *addr); int set_direct_map_valid_noflush(const void *addr, unsigned long numpages, bool valid); +int folio_zap_direct_map(struct folio *folio); +int folio_restore_direct_map(struct folio *folio); bool kernel_page_present(struct page *page); =20 #endif diff --git a/arch/s390/mm/pageattr.c b/arch/s390/mm/pageattr.c index e231757bb0e0..f739fee0e110 100644 --- a/arch/s390/mm/pageattr.c +++ b/arch/s390/mm/pageattr.c @@ -413,6 +413,18 @@ int set_direct_map_valid_noflush(const void *addr, uns= igned long numpages, return __set_memory((unsigned long)addr, numpages, flags); } =20 +int folio_zap_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), false); +} + +int folio_restore_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), true); +} + bool kernel_page_present(struct page *page) { unsigned long addr; diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_m= emory.h index f912191f0853..febbfbdc39df 100644 --- a/arch/x86/include/asm/set_memory.h +++ b/arch/x86/include/asm/set_memory.h @@ -91,6 +91,8 @@ int set_direct_map_invalid_noflush(const void *addr); int set_direct_map_default_noflush(const void *addr); int set_direct_map_valid_noflush(const void *addr, unsigned long numpages, bool valid); +int folio_zap_direct_map(struct folio *folio); +int folio_restore_direct_map(struct folio *folio); bool kernel_page_present(struct page *page); =20 extern int kernel_set_to_readonly; diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index bc8e1c23175b..4a5a3124a92d 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -2657,6 +2657,26 @@ int set_direct_map_valid_noflush(const void *addr, u= nsigned long numpages, return __set_pages_np(addr, numpages); } =20 +int folio_zap_direct_map(struct folio *folio) +{ + const void *addr =3D folio_address(folio); + int ret; + + ret =3D set_direct_map_valid_noflush(addr, folio_nr_pages(folio), false); + flush_tlb_kernel_range((unsigned long)addr, + (unsigned long)addr + folio_size(folio)); + + return ret; +} +EXPORT_SYMBOL_FOR_MODULES(folio_zap_direct_map, "kvm"); + +int folio_restore_direct_map(struct folio *folio) +{ + return set_direct_map_valid_noflush(folio_address(folio), + folio_nr_pages(folio), true); +} +EXPORT_SYMBOL_FOR_MODULES(folio_restore_direct_map, "kvm"); + #ifdef CONFIG_DEBUG_PAGEALLOC void __kernel_map_pages(struct page *page, int numpages, int enable) { diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 1a2563f525fc..e2e6485f88db 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -41,6 +41,16 @@ static inline int set_direct_map_valid_noflush(const voi= d *addr, return 0; } =20 +static inline int folio_zap_direct_map(struct folio *folio) +{ + return 0; +} + +static inline int folio_restore_direct_map(struct folio *folio) +{ + return 0; +} + static inline bool kernel_page_present(struct page *page) { return true; --=20 2.50.1