From nobody Sat Oct 4 14:36:16 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A296125CC75; Fri, 15 Aug 2025 09:00:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248413; cv=none; b=oaZl3XVaIsGEYhzc9azSqBUIkCq88VLGcmqFqICfUjROi2u7btW9vKXfI4mozPmLTGCvDiA0jdHgod88i0wk5MI2EsOYHBxfxCc5tXIm3rLOYghyq9q3iWXcNdnboGp7X4jwDBEFDnVGogtI4q/gMsDOhK+hPNw+w9ikYH8dbt0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248413; c=relaxed/simple; bh=ko3+9khrfOwQzVozVPDS3ipvQJBFL4FHR8kAzAUR5ek=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jtD+sI+YE//2ioDAoOzctZ14NU0I2EkabPM7AM+iDLxOZcc7UrCG2I0QJFxbPrKg73GIt7dh/CoWEXpybkf7xYqaspYBC79UzQGgIP9S1yGKyBzltVfKNeDIr07AcQ4QzvjDv4QodTHpEUaj+q3CMCzwE0099wazp4fHH1Lel6U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 18FDD1E32; Fri, 15 Aug 2025 02:00:04 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D38593F63F; Fri, 15 Aug 2025 02:00:07 -0700 (PDT) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Lorenzo Stoakes , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , Quentin Perret , Ryan Roberts , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v2 1/8] arm64: kpkeys: Avoid unnecessary writes to POR_EL1 Date: Fri, 15 Aug 2025 09:59:53 +0100 Message-ID: <20250815090000.2182450-2-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250815090000.2182450-1-kevin.brodsky@arm.com> References: <20250815090000.2182450-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Nested uses of kpkeys guards are about to be introduced, which means that kpkeys_set_level() may not actually need to change the value of POR_EL1. Since updating POR_EL1 requires an expensive ISB, let's skip the write if the value is unchanged, by returning KPKEYS_PKEY_REG_INVAL. This will cause the matching kpkeys_restore_pkey_reg() call to bail out without calling arch_kpkeys_restore_pkey_reg(). Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkey= s.h index 64d6e22740ec..70e21df84252 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -43,6 +43,9 @@ static __always_inline int arch_kpkeys_set_level(int leve= l) u64 prev_por =3D read_sysreg_s(SYS_POR_EL1); u64 new_por =3D por_set_kpkeys_level(prev_por, level); =20 + if (new_por =3D=3D prev_por) + return KPKEYS_PKEY_REG_INVAL; + __kpkeys_set_pkey_reg_nosync(new_por); isb(); =20 --=20 2.47.0 From nobody Sat Oct 4 14:36:16 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9884026056D; Fri, 15 Aug 2025 09:00:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248422; cv=none; b=T2E1CkIWW4NkHy/AczmOBksvTAmU4XJHMiQE9atgTW2KIMIFflDrfwRMsYShZ30oAsjnaTaeif7L/pl4UAbet61qPaw9S9sgCGxVMvf7p5ISBMy7pOKm8aIgYdx/Q2rWxvXisjN4NbeB5OVzqz1ISYasXU/7ahH0x3Dn3uNfxWw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248422; c=relaxed/simple; bh=oRYJGYT7GhTfO6Vb9BBb5JQJjgYK/sMCj3mh0C+ZpFo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pwfHrg+pv3HbSgKyShAR/Wbi0hG0frAFsan5HqbgVEIgNtsjzVd0xRR4fMF3HCsM1+r4BdAQYl3FTr9Vvv4xChhESgK5QNBvbTYS2YzOCaqNH6BYUF+Jx5CPsXIplWssKE/3T1Ed+yFHJLlI78VbMNx0FOqUCwzya+KdR+FiTrA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C51341E5E; Fri, 15 Aug 2025 02:00:08 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8EE173F63F; Fri, 15 Aug 2025 02:00:12 -0700 (PDT) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Lorenzo Stoakes , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , Quentin Perret , Ryan Roberts , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v2 2/8] mm: kpkeys: Introduce unrestricted level Date: Fri, 15 Aug 2025 09:59:54 +0100 Message-ID: <20250815090000.2182450-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250815090000.2182450-1-kevin.brodsky@arm.com> References: <20250815090000.2182450-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Highly privileged components, such as allocators, may require write access to arbitrary data. To that end, introduce a kpkeys level that grants write access to all kpkeys. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 4 +++- include/linux/kpkeys.h | 3 ++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkey= s.h index 70e21df84252..ded5d6e988dc 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -28,7 +28,9 @@ static inline u64 por_set_kpkeys_level(u64 por, int level) { por =3D por_elx_set_pkey_perms(por, KPKEYS_PKEY_DEFAULT, POE_RWX); por =3D por_elx_set_pkey_perms(por, KPKEYS_PKEY_PGTABLES, - level =3D=3D KPKEYS_LVL_PGTABLES ? POE_RW : POE_R); + level =3D=3D KPKEYS_LVL_PGTABLES || + level =3D=3D KPKEYS_LVL_UNRESTRICTED + ? POE_RW : POE_R); =20 return por; } diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index 5f4b096374ba..48f240bea8e1 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -10,9 +10,10 @@ struct folio; =20 #define KPKEYS_LVL_DEFAULT 0 #define KPKEYS_LVL_PGTABLES 1 +#define KPKEYS_LVL_UNRESTRICTED 2 =20 #define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT -#define KPKEYS_LVL_MAX KPKEYS_LVL_PGTABLES +#define KPKEYS_LVL_MAX KPKEYS_LVL_UNRESTRICTED =20 #define __KPKEYS_GUARD(name, set_level, restore_pkey_reg, set_arg, ...) \ __DEFINE_CLASS_IS_CONDITIONAL(name, false); \ --=20 2.47.0 From nobody Sat Oct 4 14:36:16 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6F80726B973; Fri, 15 Aug 2025 09:00:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248424; cv=none; b=PN72eqOm8zgzZolyYoypdzL/pt4rU7NQpVTHalzOGhgQ6oQ0NcFnC125jHFhcRwd18kl/+cGQqtRyXdAgj55TOWYf52AGHOQc9EBDs3YjBDsb2NINKy0Al9SnGUO4kXBfksqVHGcF077DDx4C+fJBlf2sOFiYRpLThKri7+c4NA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248424; c=relaxed/simple; bh=ZPfcOoWQ/HLgw4TeqfWkBpXQXIhFNxVm6jpE2KxNZE8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JyCnURGZ5B4ajZSnHsV55cwW0ikOvhKQjvkppiCEkIHQhq/yD7hGnk+Cf21tEUn17djpNV54CwvY8aYX2JLzdE7Z3OpgZDmVZ1lhJjZImBN0MFZUBF6YadytRUWYuYIbCUy8KNxs8ubRkD7CaqfHlQoz7fXFQYh7wj1ES6rbNo8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A6A322C40; Fri, 15 Aug 2025 02:00:13 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4A9B43F63F; Fri, 15 Aug 2025 02:00:17 -0700 (PDT) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Lorenzo Stoakes , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , Quentin Perret , Ryan Roberts , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v2 3/8] slab: Introduce SLAB_SET_PKEY Date: Fri, 15 Aug 2025 09:59:55 +0100 Message-ID: <20250815090000.2182450-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250815090000.2182450-1-kevin.brodsky@arm.com> References: <20250815090000.2182450-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce the SLAB_SET_PKEY flag to request a kmem_cache whose slabs are mapped with a non-default pkey, if kernel pkeys (kpkeys) are supported. The pkey to be used is specified via a new pkey field in struct kmem_cache_args. The setting/resetting of the pkey is done directly at the slab level (allocate_slab/__free_slab) to avoid having to propagate the pkey value down to the page level. Memory mapped with a non-default pkey cannot be written to at the default kpkeys level. This is handled by switching to the unrestricted kpkeys level (granting write access to all pkeys) when writing to a slab with SLAB_SET_PKEY. The merging of slabs with SLAB_SET_PKEY is conservatively prevented, though it should be possible to merge slabs with the same configured pkey. Signed-off-by: Kevin Brodsky --- include/linux/slab.h | 21 ++++++++++++++++ mm/slab.h | 7 +++++- mm/slab_common.c | 2 +- mm/slub.c | 58 +++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 85 insertions(+), 3 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index d5a8ab98035c..8cf8f655e794 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -59,6 +59,9 @@ enum _slab_flag_bits { _SLAB_CMPXCHG_DOUBLE, #ifdef CONFIG_SLAB_OBJ_EXT _SLAB_NO_OBJ_EXT, +#endif +#ifdef CONFIG_ARCH_HAS_KPKEYS + _SLAB_SET_PKEY, #endif _SLAB_FLAGS_LAST_BIT }; @@ -244,6 +247,12 @@ enum _slab_flag_bits { #define SLAB_NO_OBJ_EXT __SLAB_FLAG_UNUSED #endif =20 +#ifdef CONFIG_ARCH_HAS_KPKEYS +#define SLAB_SET_PKEY __SLAB_FLAG_BIT(_SLAB_SET_PKEY) +#else +#define SLAB_SET_PKEY __SLAB_FLAG_UNUSED +#endif + /* * ZERO_SIZE_PTR will be returned for zero sized kmalloc requests. * @@ -335,6 +344,18 @@ struct kmem_cache_args { * %NULL means no constructor. */ void (*ctor)(void *); + /** + * @pkey: The pkey to map the allocated pages with. + * + * If the SLAB flags include SLAB_SET_PKEY, and if kernel pkeys are + * supported, objects are allocated in pages mapped with the protection + * key specified by @pkey. Otherwise, this field is ignored. + * + * Note that if @pkey is a non-default pkey, some overhead is incurred + * when internal slab functions switch the pkey register to write to the + * slab (e.g. setting a free pointer). + */ + int pkey; }; =20 struct kmem_cache *__kmem_cache_create_args(const char *name, diff --git a/mm/slab.h b/mm/slab.h index 248b34c839b7..01c404f2f1db 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -287,6 +287,10 @@ struct kmem_cache { unsigned int usersize; /* Usercopy region size */ #endif =20 +#ifdef CONFIG_ARCH_HAS_KPKEYS + int pkey; +#endif + struct kmem_cache_node *node[MAX_NUMNODES]; }; =20 @@ -438,7 +442,8 @@ static inline bool is_kmalloc_normal(struct kmem_cache = *s) SLAB_TYPESAFE_BY_RCU | SLAB_DEBUG_OBJECTS | \ SLAB_NOLEAKTRACE | SLAB_RECLAIM_ACCOUNT | \ SLAB_TEMPORARY | SLAB_ACCOUNT | \ - SLAB_NO_USER_FLAGS | SLAB_KMALLOC | SLAB_NO_MERGE) + SLAB_NO_USER_FLAGS | SLAB_KMALLOC | SLAB_NO_MERGE | \ + SLAB_SET_PKEY) =20 #define SLAB_DEBUG_FLAGS (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_CONSISTENCY_CHECKS) diff --git a/mm/slab_common.c b/mm/slab_common.c index bfe7c40eeee1..7b26062629a1 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -47,7 +47,7 @@ struct kmem_cache *kmem_cache; */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_NO_MERGE) + SLAB_FAILSLAB | SLAB_NO_MERGE | SLAB_SET_PKEY) =20 #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) diff --git a/mm/slub.c b/mm/slub.c index 30003763d224..f6aec6ed7135 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -44,6 +44,7 @@ #include #include #include +#include =20 #include #include @@ -477,6 +478,15 @@ static nodemask_t slab_nodes; static struct workqueue_struct *flushwq; #endif =20 +#ifdef CONFIG_ARCH_HAS_KPKEYS +KPKEYS_GUARD_COND(kpkeys_slab_write, + KPKEYS_LVL_UNRESTRICTED, + unlikely(s->flags & SLAB_SET_PKEY), + struct kmem_cache *s) +#else +KPKEYS_GUARD_NOOP(kpkeys_slab_write, struct kmem_cache *s) +#endif + /******************************************************************** * Core slab cache functions *******************************************************************/ @@ -563,6 +573,8 @@ static inline void set_freepointer(struct kmem_cache *s= , void *object, void *fp) BUG_ON(object =3D=3D fp); /* naive detection of double free or corruption= */ #endif =20 + guard(kpkeys_slab_write)(s); + freeptr_addr =3D (unsigned long)kasan_reset_tag((void *)freeptr_addr); *(freeptr_t *)freeptr_addr =3D freelist_ptr_encode(s, fp, freeptr_addr); } @@ -802,6 +814,8 @@ static inline void set_orig_size(struct kmem_cache *s, p +=3D get_info_end(s); p +=3D sizeof(struct track) * 2; =20 + guard(kpkeys_slab_write)(s); + *(unsigned int *)p =3D orig_size; } =20 @@ -986,6 +1000,8 @@ static void set_track_update(struct kmem_cache *s, voi= d *object, { struct track *p =3D get_track(s, object, alloc); =20 + guard(kpkeys_slab_write)(s); + #ifdef CONFIG_STACKDEPOT p->handle =3D handle; #endif @@ -1010,6 +1026,8 @@ static void init_tracking(struct kmem_cache *s, void = *object) if (!(s->flags & SLAB_STORE_USER)) return; =20 + guard(kpkeys_slab_write)(s); + p =3D get_track(s, object, TRACK_ALLOC); memset(p, 0, 2*sizeof(struct track)); } @@ -1191,6 +1209,8 @@ static void init_object(struct kmem_cache *s, void *o= bject, u8 val) u8 *p =3D kasan_reset_tag(object); unsigned int poison_size =3D s->object_size; =20 + guard(kpkeys_slab_write)(s); + if (s->flags & SLAB_RED_ZONE) { /* * Here and below, avoid overwriting the KMSAN shadow. Keeping @@ -2399,6 +2419,8 @@ bool slab_free_hook(struct kmem_cache *s, void *x, bo= ol init, int rsize; unsigned int inuse, orig_size; =20 + guard(kpkeys_slab_write)(s); + inuse =3D get_info_end(s); orig_size =3D get_orig_size(s, x); if (!kasan_has_integrated_init()) @@ -2631,6 +2653,8 @@ static __always_inline void unaccount_slab(struct sla= b *slab, int order, -(PAGE_SIZE << order)); } =20 +static void __free_slab(struct kmem_cache *s, struct slab *slab); + static struct slab *allocate_slab(struct kmem_cache *s, gfp_t flags, int n= ode) { struct slab *slab; @@ -2681,6 +2705,18 @@ static struct slab *allocate_slab(struct kmem_cache = *s, gfp_t flags, int node) =20 setup_slab_debug(s, slab, start); =20 +#ifdef CONFIG_ARCH_HAS_KPKEYS + if (unlikely(s->flags & SLAB_SET_PKEY)) { + int ret =3D set_memory_pkey((unsigned long)start, + 1 << oo_order(oo), s->pkey); + + if (WARN_ON(ret)) { + __free_slab(s, slab); + return NULL; + } + } +#endif + shuffle =3D shuffle_freelist(s, slab); =20 if (!shuffle) { @@ -2721,6 +2757,11 @@ static void __free_slab(struct kmem_cache *s, struct= slab *slab) __folio_clear_slab(folio); mm_account_reclaimed_pages(pages); unaccount_slab(slab, order, s); +#ifdef CONFIG_ARCH_HAS_KPKEYS + if (unlikely(s->flags & SLAB_SET_PKEY)) + WARN_ON(set_memory_pkey((unsigned long)folio_address(folio), + pages, 0)); +#endif free_frozen_pages(&folio->page, order); } =20 @@ -4118,9 +4159,11 @@ static __always_inline void maybe_wipe_obj_freeptr(s= truct kmem_cache *s, void *obj) { if (unlikely(slab_want_init_on_free(s)) && obj && - !freeptr_outside_object(s)) + !freeptr_outside_object(s)) { + guard(kpkeys_slab_write)(s); memset((void *)((char *)kasan_reset_tag(obj) + s->offset), 0, sizeof(void *)); + } } =20 static __fastpath_inline @@ -4920,6 +4963,7 @@ __do_krealloc(const void *p, size_t new_size, gfp_t f= lags) /* Zero out spare memory. */ if (want_init_on_alloc(flags)) { kasan_disable_current(); + guard(kpkeys_slab_write)(s); if (orig_size && orig_size < new_size) memset(kasan_reset_tag(p) + orig_size, 0, new_size - orig_size); else @@ -4929,6 +4973,7 @@ __do_krealloc(const void *p, size_t new_size, gfp_t f= lags) =20 /* Setup kmalloc redzone when needed */ if (s && slub_debug_orig_size(s)) { + guard(kpkeys_slab_write)(s); set_orig_size(s, (void *)p, new_size); if (s->flags & SLAB_RED_ZONE && new_size < ks) memset_no_sanitize_memory(kasan_reset_tag(p) + new_size, @@ -6410,6 +6455,17 @@ int do_kmem_cache_create(struct kmem_cache *s, const= char *name, s->useroffset =3D args->useroffset; s->usersize =3D args->usersize; #endif +#ifdef CONFIG_ARCH_HAS_KPKEYS + s->pkey =3D args->pkey; + + if (s->flags & SLAB_SET_PKEY) { + if (s->pkey >=3D arch_max_pkey()) + goto out; + + if (!arch_kpkeys_enabled() || s->pkey =3D=3D KPKEYS_PKEY_DEFAULT) + s->flags &=3D ~SLAB_SET_PKEY; + } +#endif =20 if (!calculate_sizes(args, s)) goto out; --=20 2.47.0 From nobody Sat Oct 4 14:36:16 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EC0A72701DA; Fri, 15 Aug 2025 09:00:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248428; cv=none; b=TwU5RDEdxBBiheD3mXUDiFquO7tNcL4OMLO/bBAknlMD1Pfci2K8BW2hPiuB1a8CaqV7M0Yn8qa5RPUZ72IbItDR1Uq1SlKOJD54XrAPiNmCo9zFWo5IXMHhoodxsb2WA23gNKhQvUrtvh+oqa73noFWzoxteIq6hQmBKgIGPMw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248428; c=relaxed/simple; bh=3DqzR4RysG4uEXoDRVuAQmNC/sAC367RgTRLeXOmfCk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X6F73Z+yV4j7ryYIlBWRdfiUk7D8TeaN2pPDZDvPnRcAVDemyBmC+vY/iNuBwMzcoralqu412Js/MS1KiMl8c+HyfhjjOQ3sZcQZNHeLlRz34ggzpGAnGjmKTK+pXree6EtvSqnKmXSz89Val4Zn3dJOoUOm4dPiUDgpuXvpnMk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 614402C41; Fri, 15 Aug 2025 02:00:18 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 283233F63F; Fri, 15 Aug 2025 02:00:22 -0700 (PDT) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Lorenzo Stoakes , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , Quentin Perret , Ryan Roberts , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v2 4/8] rcu: Allow processing kpkeys-protected data Date: Fri, 15 Aug 2025 09:59:56 +0100 Message-ID: <20250815090000.2182450-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250815090000.2182450-1-kevin.brodsky@arm.com> References: <20250815090000.2182450-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Data assigned a non-default pkey is not writable at the default kpkeys level. If such data is managed via RCU, some mechanism is required to temporarily grant write access to the data's struct rcu_head, for instance when zeroing the callback pointer. There is unfortunately no straightforward way for RCU to know whether the managed data is mapped with a non-default pkey. This patch takes the easy route and switches to the unrestricted kpkeys level whenever struct rcu_head is written; this should work reliably but it is clearly suboptimal. That behaviour is enabled by selecting CONFIG_KPKEYS_UNRESTRICTED_RCU. This patch isn't comprehensive, in particular it does not take care of Tiny RCU. Signed-off-by: Kevin Brodsky --- kernel/rcu/rcu.h | 7 +++++++ kernel/rcu/rcu_segcblist.c | 13 +++++++++---- kernel/rcu/tree.c | 3 ++- mm/Kconfig | 2 ++ 4 files changed, 20 insertions(+), 5 deletions(-) diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h index 9cf01832a6c3..71e9a695f4eb 100644 --- a/kernel/rcu/rcu.h +++ b/kernel/rcu/rcu.h @@ -10,6 +10,7 @@ #ifndef __LINUX_RCU_H #define __LINUX_RCU_H =20 +#include #include #include =20 @@ -691,4 +692,10 @@ int rcu_stall_notifier_call_chain(unsigned long val, v= oid *v); static inline int rcu_stall_notifier_call_chain(unsigned long val, void *v= ) { return NOTIFY_DONE; } #endif // #else // #if defined(CONFIG_RCU_STALL_COMMON) && defined(CONFIG_= RCU_CPU_STALL_NOTIFIER) =20 +#ifdef CONFIG_KPKEYS_UNRESTRICTED_RCU +KPKEYS_GUARD(kpkeys_rcu, KPKEYS_LVL_UNRESTRICTED) +#else +KPKEYS_GUARD_NOOP(kpkeys_rcu) +#endif + #endif /* __LINUX_RCU_H */ diff --git a/kernel/rcu/rcu_segcblist.c b/kernel/rcu/rcu_segcblist.c index 298a2c573f02..e7d6c8370b70 100644 --- a/kernel/rcu/rcu_segcblist.c +++ b/kernel/rcu/rcu_segcblist.c @@ -12,6 +12,7 @@ #include #include =20 +#include "rcu.h" #include "rcu_segcblist.h" =20 /* Initialize simple callback list. */ @@ -332,7 +333,8 @@ void rcu_segcblist_enqueue(struct rcu_segcblist *rsclp, rcu_segcblist_inc_len(rsclp); rcu_segcblist_inc_seglen(rsclp, RCU_NEXT_TAIL); rhp->next =3D NULL; - WRITE_ONCE(*rsclp->tails[RCU_NEXT_TAIL], rhp); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(*rsclp->tails[RCU_NEXT_TAIL], rhp); WRITE_ONCE(rsclp->tails[RCU_NEXT_TAIL], &rhp->next); } =20 @@ -360,7 +362,8 @@ bool rcu_segcblist_entrain(struct rcu_segcblist *rsclp, if (!rcu_segcblist_segempty(rsclp, i)) break; rcu_segcblist_inc_seglen(rsclp, i); - WRITE_ONCE(*rsclp->tails[i], rhp); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(*rsclp->tails[i], rhp); for (; i <=3D RCU_NEXT_TAIL; i++) WRITE_ONCE(rsclp->tails[i], &rhp->next); return true; @@ -381,7 +384,8 @@ void rcu_segcblist_extract_done_cbs(struct rcu_segcblis= t *rsclp, rclp->len =3D rcu_segcblist_get_seglen(rsclp, RCU_DONE_TAIL); *rclp->tail =3D rsclp->head; WRITE_ONCE(rsclp->head, *rsclp->tails[RCU_DONE_TAIL]); - WRITE_ONCE(*rsclp->tails[RCU_DONE_TAIL], NULL); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(*rsclp->tails[RCU_DONE_TAIL], NULL); rclp->tail =3D rsclp->tails[RCU_DONE_TAIL]; for (i =3D RCU_CBLIST_NSEGS - 1; i >=3D RCU_DONE_TAIL; i--) if (rsclp->tails[i] =3D=3D rsclp->tails[RCU_DONE_TAIL]) @@ -436,7 +440,8 @@ void rcu_segcblist_insert_done_cbs(struct rcu_segcblist= *rsclp, if (!rclp->head) return; /* No callbacks to move. */ rcu_segcblist_add_seglen(rsclp, RCU_DONE_TAIL, rclp->len); - *rclp->tail =3D rsclp->head; + scoped_guard(kpkeys_rcu) + *rclp->tail =3D rsclp->head; WRITE_ONCE(rsclp->head, rclp->head); for (i =3D RCU_DONE_TAIL; i < RCU_CBLIST_NSEGS; i++) if (&rsclp->head =3D=3D rsclp->tails[i]) diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 174ee243b349..2eada18c04d5 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -2601,7 +2601,8 @@ static void rcu_do_batch(struct rcu_data *rdp) =20 f =3D rhp->func; debug_rcu_head_callback(rhp); - WRITE_ONCE(rhp->func, (rcu_callback_t)0L); + scoped_guard(kpkeys_rcu) + WRITE_ONCE(rhp->func, (rcu_callback_t)0L); f(rhp); =20 rcu_lock_release(&rcu_callback_map); diff --git a/mm/Kconfig b/mm/Kconfig index e34edf5c41e7..c023f74a2201 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1178,6 +1178,8 @@ config ARCH_HAS_KPKEYS # ARCH_HAS_KPKEYS must be selected when selecting this option config ARCH_HAS_KPKEYS_HARDENED_PGTABLES bool +config KPKEYS_UNRESTRICTED_RCU + bool =20 config ARCH_USES_PG_ARCH_2 bool --=20 2.47.0 From nobody Sat Oct 4 14:36:16 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BF686265CB2; Fri, 15 Aug 2025 09:00:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248433; cv=none; b=EaKITHAAtI1DOhDS7cxhFH+sploK15ig301QFVR58pCEZ8EWD7/MXxt5u87i5qPaotNHtpX2hghBJjv+pgxEoj0hjS8KqpJiSAnFNRGmP5ZjUtlVZw0eIlP4MqbUTccN0VU362MhS4mN90VvEE8T+iHqFhciv0xq43KAABkhMlg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248433; c=relaxed/simple; bh=m/5CGff6IdcBeFe4D1gj4vmKsG9NpC3ew0deQKHJII0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jyssWOl6k+ZMhB80oVBa7uycS3OiRGbL1Jmy3p41aAq7sst2F7EMbVd3mNzV5ZsAB4s1vrxRJw1Sfvu8JkBH3naybCCwA/yL6hD1k5lpFPRPmK38iSgXFVLDjAsgKYxaTF48RrDLP0DQIJvhkffySZ+Bw1lLD0YoO0IqIzlgYns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1A53D2C42; Fri, 15 Aug 2025 02:00:23 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D78FF3F63F; Fri, 15 Aug 2025 02:00:26 -0700 (PDT) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Lorenzo Stoakes , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , Quentin Perret , Ryan Roberts , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v2 5/8] mm: kpkeys: Introduce cred pkey/level Date: Fri, 15 Aug 2025 09:59:57 +0100 Message-ID: <20250815090000.2182450-6-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250815090000.2182450-1-kevin.brodsky@arm.com> References: <20250815090000.2182450-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We will need a separate pkey to protect struct cred. Allocate one as well as a new kpkeys level that grants write access to that pkey. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 7 ++++++- include/asm-generic/kpkeys.h | 4 ++++ include/linux/kpkeys.h | 3 ++- 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkey= s.h index ded5d6e988dc..ffb7622391c3 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -13,7 +13,8 @@ * used in assembly. */ #define POR_EL1_INIT (POR_ELx_PERM_PREP(KPKEYS_PKEY_DEFAULT, POE_RWX) | \ - POR_ELx_PERM_PREP(KPKEYS_PKEY_PGTABLES, POE_R)) + POR_ELx_PERM_PREP(KPKEYS_PKEY_PGTABLES, POE_R) | \ + POR_ELx_PERM_PREP(KPKEYS_PKEY_CRED, POE_R)) =20 #ifndef __ASSEMBLY__ =20 @@ -31,6 +32,10 @@ static inline u64 por_set_kpkeys_level(u64 por, int leve= l) level =3D=3D KPKEYS_LVL_PGTABLES || level =3D=3D KPKEYS_LVL_UNRESTRICTED ? POE_RW : POE_R); + por =3D por_elx_set_pkey_perms(por, KPKEYS_PKEY_CRED, + level =3D=3D KPKEYS_LVL_CRED || + level =3D=3D KPKEYS_LVL_UNRESTRICTED + ? POE_RW : POE_R); =20 return por; } diff --git a/include/asm-generic/kpkeys.h b/include/asm-generic/kpkeys.h index cec92334a9f3..56a2fc9fe4a6 100644 --- a/include/asm-generic/kpkeys.h +++ b/include/asm-generic/kpkeys.h @@ -2,6 +2,10 @@ #ifndef __ASM_GENERIC_KPKEYS_H #define __ASM_GENERIC_KPKEYS_H =20 +#ifndef KPKEYS_PKEY_CRED +#define KPKEYS_PKEY_CRED 2 +#endif + #ifndef KPKEYS_PKEY_PGTABLES #define KPKEYS_PKEY_PGTABLES 1 #endif diff --git a/include/linux/kpkeys.h b/include/linux/kpkeys.h index 48f240bea8e1..0e555b505b33 100644 --- a/include/linux/kpkeys.h +++ b/include/linux/kpkeys.h @@ -10,7 +10,8 @@ struct folio; =20 #define KPKEYS_LVL_DEFAULT 0 #define KPKEYS_LVL_PGTABLES 1 -#define KPKEYS_LVL_UNRESTRICTED 2 +#define KPKEYS_LVL_CRED 2 +#define KPKEYS_LVL_UNRESTRICTED 3 =20 #define KPKEYS_LVL_MIN KPKEYS_LVL_DEFAULT #define KPKEYS_LVL_MAX KPKEYS_LVL_UNRESTRICTED --=20 2.47.0 From nobody Sat Oct 4 14:36:16 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6A3FF26056D; Fri, 15 Aug 2025 09:00:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248438; cv=none; b=sz45C1XbPl8RjLapAYbymj10h5VqAY3S+j3SS6BUyrqigdhRdFBqzoUDaJv+HH2UgB4e4/rZwlWuAXiEQhJk4n0ROWsfHv3nu+hlA3Uz9vEbI2dCCRc2bdppJnasK1GJ3ZB7nvz0+LGx1LQ2moW3/NpaOocJmF3/pUfFDkmH/pg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248438; c=relaxed/simple; bh=oaUjVWDqxZhoAsqTVxDAFltZwS97f995rLnVivY6MVA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tkhG9XStKK9RHYqeZkvj8T6xU7+4yPU9G1IU8Vo0VqgxJZnKF9vlHWoQoLdzcablm6NFJIJdNmTkjsTJNdLiCwR6FS3g3efFA03Js3Tk4gZGLaiCrNX5SvMoR/Xp42gI1A2OoqspO7QgEX9IS/OYxAusb95WSczCKigwwgIh0Rc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CA1E62C43; Fri, 15 Aug 2025 02:00:27 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9416A3F63F; Fri, 15 Aug 2025 02:00:31 -0700 (PDT) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Lorenzo Stoakes , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , Quentin Perret , Ryan Roberts , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v2 6/8] cred: Protect live struct cred with kpkeys Date: Fri, 15 Aug 2025 09:59:58 +0100 Message-ID: <20250815090000.2182450-7-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250815090000.2182450-1-kevin.brodsky@arm.com> References: <20250815090000.2182450-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This patch introduces a feature to prevent unintended modifications of live credentials, by moving them to protected memory when they are installed via commit_creds(). The protection mechanism is kernel pkeys (kpkeys): protected memory is mapped with a non-default pkey and write access is disabled by default. As a result, task->{cred,real_cred} can only be written to by switching to a higher kpkeys level. The kpkeys_hardened_cred feature is enabled by choosing CONFIG_KPKEYS_HARDENED_CRED=3Dy and running on a system supporting kpkeys. Credentials are not directly allocated in protected memory, as that would force all code preparing new credentials to switch kpkeys level. To avoid such disruption, prepare_creds() and variants still allocate standard memory. When commit_creds() is called, the credentials are copied to protected memory, and the temporary object (in a standard kmalloc slab) is freed. This approach does not work so transparently when it comes to override_creds(), because it does not consume the reference: the object it gets passed cannot be moved. Callers of override_creds() will need to explicitly call a new protect_creds() helper to move the credentials to protected memory once they are done preparing them. Some of these callers use the unmodified output of prepare_creds(); prepare_protected_creds() is introduced to avoid an unnecessary copy in such cases. This patch does not handle these situations, but it does not break them either (credentials installed by override_creds() will simply be unprotected). Various helpers need to modify live credentials. To that end, guard(kpkeys_hardened_cred) is introduced to switch to the kpkeys level that enables write access to KPKEYS_PKEY_CRED. Signed-off-by: Kevin Brodsky --- include/linux/cred.h | 12 +++ kernel/cred.c | 179 +++++++++++++++++++++++++++++++------ security/Kconfig.hardening | 13 +++ 3 files changed, 177 insertions(+), 27 deletions(-) diff --git a/include/linux/cred.h b/include/linux/cred.h index a102a10f833f..8eacc4f3de60 100644 --- a/include/linux/cred.h +++ b/include/linux/cred.h @@ -16,10 +16,17 @@ #include #include #include +#include =20 struct cred; struct inode; =20 +#ifdef CONFIG_KPKEYS_HARDENED_CRED +KPKEYS_GUARD(kpkeys_hardened_cred, KPKEYS_LVL_CRED) +#else +KPKEYS_GUARD_NOOP(kpkeys_hardened_cred) +#endif + /* * COW Supplementary groups list */ @@ -162,6 +169,8 @@ extern int set_create_files_as(struct cred *, struct in= ode *); extern int cred_fscmp(const struct cred *, const struct cred *); extern void __init cred_init(void); extern int set_cred_ucounts(struct cred *); +extern struct cred *prepare_protected_creds(void); +extern struct cred *protect_creds(struct cred *); =20 static inline bool cap_ambient_invariant_ok(const struct cred *cred) { @@ -199,6 +208,7 @@ static inline const struct cred *get_cred_many(const st= ruct cred *cred, int nr) struct cred *nonconst_cred =3D (struct cred *) cred; if (!cred) return cred; + guard(kpkeys_hardened_cred)(); nonconst_cred->non_rcu =3D 0; atomic_long_add(nr, &nonconst_cred->usage); return cred; @@ -223,6 +233,7 @@ static inline const struct cred *get_cred_rcu(const str= uct cred *cred) struct cred *nonconst_cred =3D (struct cred *) cred; if (!cred) return NULL; + guard(kpkeys_hardened_cred)(); if (!atomic_long_inc_not_zero(&nonconst_cred->usage)) return NULL; nonconst_cred->non_rcu =3D 0; @@ -246,6 +257,7 @@ static inline void put_cred_many(const struct cred *_cr= ed, int nr) struct cred *cred =3D (struct cred *) _cred; =20 if (cred) { + guard(kpkeys_hardened_cred)(); if (atomic_long_sub_and_test(nr, &cred->usage)) __put_cred(cred); } diff --git a/kernel/cred.c b/kernel/cred.c index 9676965c0981..95d316f73786 100644 --- a/kernel/cred.c +++ b/kernel/cred.c @@ -20,6 +20,8 @@ #include #include =20 +#include "../mm/slab.h" + #if 0 #define kdebug(FMT, ...) \ printk("[%-5.5s%5u] " FMT "\n", \ @@ -62,6 +64,48 @@ struct cred init_cred =3D { .ucounts =3D &init_ucounts, }; =20 +static bool hardened_cred_enabled(void) +{ + return IS_ENABLED(CONFIG_KPKEYS_HARDENED_CRED) && arch_kpkeys_enabled(); +} + +static bool cred_is_protected(const struct cred *cred) +{ + struct slab *slab; + + slab =3D virt_to_slab(cred); + if (!slab) + return false; + + return slab->slab_cache->flags & SLAB_SET_PKEY; +} + +static struct cred *alloc_unprotected_creds(gfp_t flags) +{ + if (hardened_cred_enabled()) + return kmalloc(sizeof(struct cred), flags); + else + return kmem_cache_alloc(cred_jar, flags); +} + +static struct cred *alloc_protected_creds(gfp_t flags) +{ + return kmem_cache_alloc(cred_jar, flags); +} + +static void free_creds(struct cred *cred) +{ + bool cred_in_jar =3D true; + + if (hardened_cred_enabled()) + cred_in_jar =3D cred_is_protected(cred); + + if (cred_in_jar) + kmem_cache_free(cred_jar, cred); + else + kfree(cred); +} + /* * The RCU callback to actually dispose of a set of credentials */ @@ -75,7 +119,8 @@ static void put_cred_rcu(struct rcu_head *rcu) panic("CRED: put_cred_rcu() sees %p with usage %ld\n", cred, atomic_long_read(&cred->usage)); =20 - security_cred_free(cred); + scoped_guard(kpkeys_hardened_cred) + security_cred_free(cred); key_put(cred->session_keyring); key_put(cred->process_keyring); key_put(cred->thread_keyring); @@ -86,7 +131,7 @@ static void put_cred_rcu(struct rcu_head *rcu) if (cred->ucounts) put_ucounts(cred->ucounts); put_user_ns(cred->user_ns); - kmem_cache_free(cred_jar, cred); + free_creds(cred); } =20 /** @@ -174,7 +219,7 @@ struct cred *cred_alloc_blank(void) { struct cred *new; =20 - new =3D kmem_cache_zalloc(cred_jar, GFP_KERNEL); + new =3D alloc_unprotected_creds(GFP_KERNEL | __GFP_ZERO); if (!new) return NULL; =20 @@ -189,29 +234,10 @@ struct cred *cred_alloc_blank(void) return NULL; } =20 -/** - * prepare_creds - Prepare a new set of credentials for modification - * - * Prepare a new set of task credentials for modification. A task's creds - * shouldn't generally be modified directly, therefore this function is us= ed to - * prepare a new copy, which the caller then modifies and then commits by - * calling commit_creds(). - * - * Preparation involves making a copy of the objective creds for modificat= ion. - * - * Returns a pointer to the new creds-to-be if successful, NULL otherwise. - * - * Call commit_creds() or abort_creds() to clean up. - */ -struct cred *prepare_creds(void) +static struct cred *__prepare_creds(struct cred *new) { struct task_struct *task =3D current; const struct cred *old; - struct cred *new; - - new =3D kmem_cache_alloc(cred_jar, GFP_KERNEL); - if (!new) - return NULL; =20 kdebug("prepare_creds() alloc %p", new); =20 @@ -248,8 +274,57 @@ struct cred *prepare_creds(void) abort_creds(new); return NULL; } + +/** + * prepare_creds - Prepare a new set of credentials for modification + * + * Prepare a new set of task credentials for modification. A task's creds + * shouldn't generally be modified directly, therefore this function is us= ed to + * prepare a new copy, which the caller then modifies and then commits by + * calling commit_creds(). + * + * Preparation involves making a copy of the objective creds for modificat= ion. + * + * Returns a pointer to the new creds-to-be if successful, NULL otherwise. + * + * Call commit_creds() or abort_creds() to clean up. + */ +struct cred *prepare_creds(void) +{ + struct cred *new; + + new =3D alloc_unprotected_creds(GFP_KERNEL); + if (!new) + return NULL; + + return __prepare_creds(new); +} EXPORT_SYMBOL(prepare_creds); =20 + +/** + * prepare_protected_creds - Prepare a new set of credentials in protected + * memory + * + * This function is equivalent to protect_creds(prepare_creds()), but avoi= ds + * the copy in prepare_creds() by directly allocating the credentials in + * protected memory. The returned object may only be modified by switching= to + * a higher kpkeys level, if kpkeys_hardened_cred is enabled. + */ +struct cred *prepare_protected_creds(void) +{ + struct cred *new; + + new =3D alloc_protected_creds(GFP_KERNEL); + if (!new) + return NULL; + + guard(kpkeys_hardened_cred)(); + + return __prepare_creds(new); +} +EXPORT_SYMBOL(prepare_protected_creds); + /* * Prepare credentials for current to perform an execve() * - The caller must hold ->cred_guard_mutex @@ -309,7 +384,9 @@ int copy_creds(struct task_struct *p, unsigned long clo= ne_flags) return 0; } =20 - new =3D prepare_creds(); + guard(kpkeys_hardened_cred)(); + + new =3D prepare_protected_creds(); if (!new) return -ENOMEM; =20 @@ -400,6 +477,10 @@ int commit_creds(struct cred *new) BUG_ON(task->cred !=3D old); BUG_ON(atomic_long_read(&new->usage) < 1); =20 + guard(kpkeys_hardened_cred)(); + + new =3D protect_creds(new); + get_cred(new); /* we will require a ref for the subj creds too */ =20 /* dumpability changes */ @@ -555,9 +636,16 @@ int set_cred_ucounts(struct cred *new) */ void __init cred_init(void) { + slab_flags_t flags =3D SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT; + struct kmem_cache_args args =3D {}; + + if (hardened_cred_enabled()) { + flags |=3D SLAB_SET_PKEY; + args.pkey =3D KPKEYS_PKEY_CRED; + } + /* allocate a slab in which we can store credentials */ - cred_jar =3D KMEM_CACHE(cred, - SLAB_HWCACHE_ALIGN | SLAB_PANIC | SLAB_ACCOUNT); + cred_jar =3D kmem_cache_create("cred", sizeof(struct cred), &args, flags); } =20 /** @@ -584,7 +672,7 @@ struct cred *prepare_kernel_cred(struct task_struct *da= emon) if (WARN_ON_ONCE(!daemon)) return NULL; =20 - new =3D kmem_cache_alloc(cred_jar, GFP_KERNEL); + new =3D alloc_unprotected_creds(GFP_KERNEL); if (!new) return NULL; =20 @@ -627,6 +715,43 @@ struct cred *prepare_kernel_cred(struct task_struct *d= aemon) } EXPORT_SYMBOL(prepare_kernel_cred); =20 +/** + * protect_creds - Move a set of credentials to protected memory + * @cred: The credentials to protect + * + * If kpkeys_hardened_cred is enabled, this function transfers @cred to + * protected memory. The returned object may only be modified by switching= to a + * higher kpkeys level, for instance by using guard(kpkeys_hardened_cred). + * + * Because the credentials are copied to a new location and the old locati= on is + * freed, any exising reference to @cred becomes invalid after this functi= on is + * called. For this reason only the caller should have a reference to @cre= d. + * + * If any failure occurs, or if kpkeys_hardened_cred is disabled, @cred is + * returned unmodified. + */ +struct cred *protect_creds(struct cred *cred) +{ + struct cred *protected_cred; + + if (!hardened_cred_enabled()) + return cred; + + if (WARN_ON(atomic_long_read(&cred->usage) !=3D 1)) + return cred; + + protected_cred =3D alloc_protected_creds(GFP_KERNEL); + if (WARN_ON(!protected_cred)) + return cred; + + guard(kpkeys_hardened_cred)(); + + *protected_cred =3D *cred; + kfree(cred); + return protected_cred; +} +EXPORT_SYMBOL(protect_creds); + /** * set_security_override - Set the security ID in a set of credentials * @new: The credentials to alter diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index 653663008096..cb494448c7ae 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -289,6 +289,19 @@ config KPKEYS_HARDENED_PGTABLES_KUNIT_TEST =20 If unsure, say N. =20 +config KPKEYS_HARDENED_CRED + bool "Harden task credentials using kernel pkeys" + depends on ARCH_HAS_KPKEYS + select KPKEYS_UNRESTRICTED_RCU + help + This option enforces the immutability of tasks credentials + (struct cred) by allocating them with a non-default protection (pkey) + and only enabling write access to that pkey in a limited set of cred + helpers. + + This option has no effect if the system does not support + kernel pkeys. + endmenu =20 config CC_HAS_RANDSTRUCT --=20 2.47.0 From nobody Sat Oct 4 14:36:16 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 77CBF26056D; Fri, 15 Aug 2025 09:00:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248444; cv=none; b=XA0zITMzI02P2aIbt0s4I3fssXnO72z3bBTsue66zaaNKn+W/4lWYRnpxB4JiBMjBF21bURPRY3CZ18r99aW1EBMgJdkr2wT9708bV76lfdTK3DWRR0ArD1wCZVXY7kkF2PokP50nDneOpT68S6oRUxMwfbsjbfeqC6nUgNlT88= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248444; c=relaxed/simple; bh=zwNPHx3VPaaB0fr5c8AD+jkzh7NHAyTQ/A7q4gQ1SG4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QRb76l7uGiSqXqy9OM9fjLbvfUngVrYZ+pIx28ZVnCFu3tF6x1Mv01/PNHT3Z9ukEzE7lVX217oBHch8Fn/RcwIweJz2BsisVbrpO/fw435dqAjUPZBZeZQUWRKf7Hpxyw2vrOXJei8Kt/ycZHyRkDFaWUv2URUCBsbZRHZdgSQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8898D2C46; Fri, 15 Aug 2025 02:00:32 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4FCD33F63F; Fri, 15 Aug 2025 02:00:36 -0700 (PDT) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Lorenzo Stoakes , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , Quentin Perret , Ryan Roberts , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v2 7/8] fs: Protect creds installed by override_creds() Date: Fri, 15 Aug 2025 09:59:59 +0100 Message-ID: <20250815090000.2182450-8-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250815090000.2182450-1-kevin.brodsky@arm.com> References: <20250815090000.2182450-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The kpkeys_hardened_cred feature, when enabled, automatically protects credentials installed by commit_creds(). However, because override_creds() does not consume its argument, it is up to its callers to protect the credentials before calling override_creds(). This is done by calling protect_creds(), moving the credentials to a protected memory location. In some cases, the credentials returned by prepare_creds() are passed to override_creds() as-is. In such situation where write access to the credentials is not needed, prepare_protected_creds() is used to avoid the copy incurred by a separate call to protect_creds(). This patch covers the main users of override_creds(), but it is not comprehensive. This patch is a no-op if kpkeys_hardened_cred isn't enabled. Signed-off-by: Kevin Brodsky --- fs/aio.c | 2 +- fs/fuse/passthrough.c | 2 +- fs/nfs/nfs4idmap.c | 2 +- fs/nfsd/auth.c | 2 +- fs/nfsd/nfs4recover.c | 2 +- fs/nfsd/nfsfh.c | 2 +- fs/open.c | 2 +- fs/overlayfs/dir.c | 1 + fs/overlayfs/super.c | 2 +- 9 files changed, 9 insertions(+), 8 deletions(-) diff --git a/fs/aio.c b/fs/aio.c index 7fc7b6221312..7529399bb71d 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -1658,7 +1658,7 @@ static int aio_fsync(struct fsync_iocb *req, const st= ruct iocb *iocb, if (unlikely(!req->file->f_op->fsync)) return -EINVAL; =20 - req->creds =3D prepare_creds(); + req->creds =3D prepare_protected_creds(); if (!req->creds) return -ENOMEM; =20 diff --git a/fs/fuse/passthrough.c b/fs/fuse/passthrough.c index 607ef735ad4a..4451651b1e51 100644 --- a/fs/fuse/passthrough.c +++ b/fs/fuse/passthrough.c @@ -248,7 +248,7 @@ int fuse_backing_open(struct fuse_conn *fc, struct fuse= _backing_map *map) goto out_fput; =20 fb->file =3D file; - fb->cred =3D prepare_creds(); + fb->cred =3D prepare_protected_creds(); refcount_set(&fb->count, 1); =20 res =3D fuse_backing_id_alloc(fc, fb); diff --git a/fs/nfs/nfs4idmap.c b/fs/nfs/nfs4idmap.c index 00932500fce4..6eef34b02513 100644 --- a/fs/nfs/nfs4idmap.c +++ b/fs/nfs/nfs4idmap.c @@ -228,7 +228,7 @@ int nfs_idmap_init(void) set_bit(KEY_FLAG_ROOT_CAN_CLEAR, &keyring->flags); cred->thread_keyring =3D keyring; cred->jit_keyring =3D KEY_REQKEY_DEFL_THREAD_KEYRING; - id_resolver_cache =3D cred; + id_resolver_cache =3D protect_creds(cred); return 0; =20 failed_reg_legacy: diff --git a/fs/nfsd/auth.c b/fs/nfsd/auth.c index 4dc327e02456..09b377a97147 100644 --- a/fs/nfsd/auth.c +++ b/fs/nfsd/auth.c @@ -79,7 +79,7 @@ int nfsd_setuser(struct svc_cred *cred, struct svc_export= *exp) else new->cap_effective =3D cap_raise_nfsd_set(new->cap_effective, new->cap_permitted); - put_cred(override_creds(new)); + put_cred(override_creds(protect_creds(new))); return 0; =20 oom: diff --git a/fs/nfsd/nfs4recover.c b/fs/nfsd/nfs4recover.c index 2231192ec33f..63ffa7936246 100644 --- a/fs/nfsd/nfs4recover.c +++ b/fs/nfsd/nfs4recover.c @@ -82,7 +82,7 @@ nfs4_save_creds(const struct cred **original_creds) =20 new->fsuid =3D GLOBAL_ROOT_UID; new->fsgid =3D GLOBAL_ROOT_GID; - *original_creds =3D override_creds(new); + *original_creds =3D override_creds(protect_creds(new)); return 0; } =20 diff --git a/fs/nfsd/nfsfh.c b/fs/nfsd/nfsfh.c index 74cf1f4de174..887ee5adb2dc 100644 --- a/fs/nfsd/nfsfh.c +++ b/fs/nfsd/nfsfh.c @@ -223,7 +223,7 @@ static __be32 nfsd_set_fh_dentry(struct svc_rqst *rqstp= , struct net *net, new->cap_effective =3D cap_raise_nfsd_set(new->cap_effective, new->cap_permitted); - put_cred(override_creds(new)); + put_cred(override_creds(protect_creds(new))); } else { error =3D nfsd_setuser_and_check_port(rqstp, cred, exp); if (error) diff --git a/fs/open.c b/fs/open.c index 9655158c3885..351ac9e86a15 100644 --- a/fs/open.c +++ b/fs/open.c @@ -461,7 +461,7 @@ static const struct cred *access_override_creds(void) * freeing. */ override_cred->non_rcu =3D 1; - return override_creds(override_cred); + return override_creds(protect_creds(override_cred)); } =20 static int do_faccessat(int dfd, const char __user *filename, int mode, in= t flags) diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c index 70b8687dc45e..7e7d4f26198d 100644 --- a/fs/overlayfs/dir.c +++ b/fs/overlayfs/dir.c @@ -575,6 +575,7 @@ static const struct cred *ovl_setup_cred_for_create(str= uct dentry *dentry, * We must be called with creator creds already, otherwise we risk * leaking creds. */ + override_cred =3D protect_creds(override_cred); old_cred =3D override_creds(override_cred); WARN_ON_ONCE(old_cred !=3D ovl_creds(dentry->d_sb)); =20 diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c index df85a76597e9..0a45760ff7ae 100644 --- a/fs/overlayfs/super.c +++ b/fs/overlayfs/super.c @@ -1326,7 +1326,7 @@ int ovl_fill_super(struct super_block *sb, struct fs_= context *fc) =20 err =3D -ENOMEM; if (!ofs->creator_cred) - ofs->creator_cred =3D cred =3D prepare_creds(); + ofs->creator_cred =3D cred =3D prepare_protected_creds(); else cred =3D (struct cred *)ofs->creator_cred; if (!cred) --=20 2.47.0 From nobody Sat Oct 4 14:36:16 2025 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 107292D77FC; Fri, 15 Aug 2025 09:00:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248447; cv=none; b=ovpdpEPtbcKW6X7IuxVjFGEY3mVkkMawYWUS2hF/2Qx5ht2B3XIiw/3JkMDVTj/ta7ObpKFm1PirZb+8SHNfc8X7cbIZR9UR3MsHAkxO/3P8fEyYWNoLBgwkkWKFjO6BrNSzw2gtCrmDJ1lqcYU6B8cIRjg02eSxaeIlxgbH1DE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755248447; c=relaxed/simple; bh=zGULMEeUfZazzhoJ7TSekE/SSLTUH4C5lJRFqnn8B/M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=K9D/PMqnb1+uaJL+G2m02tcWu1s1LZ+LeNSKb0WIAKKHamcPZr1xXSECcULPKVJed7OPKJBIHR+AtGcMiHGVVv7yNrexBpH79QeK8PT9vC2mB6c01dZiFzAodHHbt4c1GTznlvrN60ZswvE4pHhE4Q5yuEeRy2xhWefSxxuYOWg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 43FF02D8E; Fri, 15 Aug 2025 02:00:37 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0AC283F63F; Fri, 15 Aug 2025 02:00:40 -0700 (PDT) From: Kevin Brodsky To: linux-hardening@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Andy Lutomirski , Catalin Marinas , Dave Hansen , David Howells , "Eric W. Biederman" , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Lorenzo Stoakes , Marc Zyngier , Mark Brown , Matthew Wilcox , Maxwell Bland , "Mike Rapoport (IBM)" , Peter Zijlstra , Pierre Langlois , Quentin Perret , Ryan Roberts , Thomas Gleixner , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, x86@kernel.org Subject: [RFC PATCH v2 8/8] mm: Add basic tests for kpkeys_hardened_cred Date: Fri, 15 Aug 2025 10:00:00 +0100 Message-ID: <20250815090000.2182450-9-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250815090000.2182450-1-kevin.brodsky@arm.com> References: <20250815090000.2182450-1-kevin.brodsky@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add basic tests for the kpkeys_hardened_pgtables feature: try to perform a direct write to current->{cred,real_cred} and ensure it fails. Also check that prepare_creds, protect_creds, prepare_protected_creds behave as expected. Signed-off-by: Kevin Brodsky --- mm/Makefile | 1 + mm/tests/kpkeys_hardened_cred_kunit.c | 79 +++++++++++++++++++++++++++ security/Kconfig.hardening | 11 ++++ 3 files changed, 91 insertions(+) create mode 100644 mm/tests/kpkeys_hardened_cred_kunit.c diff --git a/mm/Makefile b/mm/Makefile index b1e6cf7f753c..c79af57c0aa5 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -149,3 +149,4 @@ obj-$(CONFIG_TMPFS_QUOTA) +=3D shmem_quota.o obj-$(CONFIG_PT_RECLAIM) +=3D pt_reclaim.o obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES) +=3D kpkeys_hardened_pgtables.o obj-$(CONFIG_KPKEYS_HARDENED_PGTABLES_KUNIT_TEST) +=3D tests/kpkeys_harden= ed_pgtables_kunit.o +obj-$(CONFIG_KPKEYS_HARDENED_CRED_KUNIT_TEST) +=3D tests/kpkeys_hardened_c= red_kunit.o diff --git a/mm/tests/kpkeys_hardened_cred_kunit.c b/mm/tests/kpkeys_harden= ed_cred_kunit.c new file mode 100644 index 000000000000..ed07469b504c --- /dev/null +++ b/mm/tests/kpkeys_hardened_cred_kunit.c @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include +#include +#include + +static int increment_cred_uid_nofault(struct cred *cred) +{ + uid_t val =3D __kuid_val(cred->uid) + 1; + + return copy_to_kernel_nofault(&cred->uid, &val, sizeof(cred->uid)); +} + +static void write_current_creds(struct kunit *test) +{ + int ret; + + if (!arch_kpkeys_enabled()) + kunit_skip(test, "kpkeys are not supported"); + + ret =3D increment_cred_uid_nofault((struct cred *)current->cred); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Write to current->cred wasn't prevented"); + + ret =3D increment_cred_uid_nofault((struct cred *)current->real_cred); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Write to current->real_cred wasn't prevented"); +} + +static void write_new_creds(struct kunit *test) +{ + struct cred *cred, *protected_cred; + int ret; + + if (!arch_kpkeys_enabled()) + kunit_skip(test, "kpkeys are not supported"); + + /* prepare_creds() + protect_creds() */ + cred =3D prepare_creds(); + KUNIT_ASSERT_NOT_NULL(test, cred); + + ret =3D increment_cred_uid_nofault(cred); + KUNIT_EXPECT_EQ_MSG(test, ret, 0, + "Failed to write to unprotected creds"); + + protected_cred =3D protect_creds(cred); + KUNIT_EXPECT_PTR_NE_MSG(test, cred, protected_cred, + "protect_creds() failed to move creds to protected memory"); + + ret =3D increment_cred_uid_nofault(protected_cred); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Write to protected_cred wasn't prevented"); + + put_cred(protected_cred); + + /* prepare_protected_creds() */ + protected_cred =3D prepare_protected_creds(); + + ret =3D increment_cred_uid_nofault(protected_cred); + KUNIT_EXPECT_EQ_MSG(test, ret, -EFAULT, + "Write to protected_cred wasn't prevented"); + + put_cred(protected_cred); + +} + +static struct kunit_case kpkeys_hardened_cred_test_cases[] =3D { + KUNIT_CASE(write_current_creds), + KUNIT_CASE(write_new_creds), + {} +}; + +static struct kunit_suite kpkeys_hardened_cred_test_suite =3D { + .name =3D "Hardened credentials using kpkeys", + .test_cases =3D kpkeys_hardened_cred_test_cases, +}; +kunit_test_suite(kpkeys_hardened_cred_test_suite); + +MODULE_DESCRIPTION("Tests for the kpkeys_hardened_cred feature"); +MODULE_LICENSE("GPL"); diff --git a/security/Kconfig.hardening b/security/Kconfig.hardening index cb494448c7ae..7ceb1e6846f2 100644 --- a/security/Kconfig.hardening +++ b/security/Kconfig.hardening @@ -302,6 +302,17 @@ config KPKEYS_HARDENED_CRED This option has no effect if the system does not support kernel pkeys. =20 +config KPKEYS_HARDENED_CRED_KUNIT_TEST + tristate "KUnit tests for kpkeys_hardened_cred" if !KUNIT_ALL_TESTS + depends on KPKEYS_HARDENED_CRED + depends on KUNIT + default KUNIT_ALL_TESTS + help + Enable this option to check that the kpkeys_hardened_cred feature + functions as intended, i.e. prevents arbitrary writes to live credentia= ls. + + If unsure, say N. + endmenu =20 config CC_HAS_RANDSTRUCT --=20 2.47.0