From nobody Sat Jul 25 21:19:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3DF1437465 for ; Mon, 13 Jul 2026 14:29:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952948; cv=none; b=p43C1NETwWrTlB3fx5LUuMeA+el+e+JRjMubr0I07nZZqxuM/gdHD18GKSyLoz9arHy9VtmWbG79MUuYZYItjmCevahLUgFREPcoV2NeMM4poWfPNetCYuImLLiFQeZy+jR5gFwXHNsRJvC7+f5O3Flfj1D8FOTJtDchXsEvPUI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952948; c=relaxed/simple; bh=Z8zSIMD1uY7VwC3850XNWXHtV/DpyqyVEr6QdDaCjro=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=PcJZFoVRfoJVUyw55f9hpHH+uOFWoW02n7JMWGb8DEaqvb8nhlwCNdmhurAjk0UsS/cywq8nsIpcbBr6IsYxJjChskJ0WP0SgzcdWrDH6N9LIOz8Ix6XX5mstziCExgbHEKDSoqpHUnQzssLW8iu1JN7d9ls4jobNb96OevWLH4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DndLLiB9; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DndLLiB9" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6A7571F00A3A; Mon, 13 Jul 2026 14:29:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1783952947; bh=wtRyuqLxHec+AIEMCGWYLc5A5j22LJ1kSJp888lWRJ4=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=DndLLiB9yUJTEojiBiWraCz1r9jKwR36ryUJncr/4zWiv3MHsqiRNTmQNtKlVUEHV UykmOD2ijPBOjopIPvnikZR3gNLAPuc7Md7uSw3p1bwi1/cV+3iUZX1czCDY99aapU oZu6Eu+doMHnf/hi5CRIqnuZkDCHi1skwRclj9V75PxfK/MVMM5uHBmUxUoQJgsIW9 mEckIMBn9W8hQKX59fS/AGPrTF2FfqlhP2TyTjawPWf9pcqn/m1fvfZw+Wa64AXdON 1pAtAyfXoNMF1BhYwVrlJK4NM2vQvM0/Igszfjd8HmBp5Ms39oOx29keR4s+fP1HL7 J4p2nGb4R8Q6A== From: "Harry Yoo (Oracle)" Date: Mon, 13 Jul 2026 23:28:49 +0900 Subject: [PATCH slab/for-next-fixes v3 1/4] mm/slab: fix a memory leak due to bootstrapping sheaves twice Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260713-kmalloc-no-objext-v3-1-47c7bd138de7@kernel.org> References: <20260713-kmalloc-no-objext-v3-0-47c7bd138de7@kernel.org> In-Reply-To: <20260713-kmalloc-no-objext-v3-0-47c7bd138de7@kernel.org> To: Vlastimil Babka , Harry Yoo , Andrew Morton , Hao Li , Christoph Lameter , David Rientjes , Roman Gushchin , "Liam R. Howlett" , Suren Baghdasaryan , Hao Ge , Kees Cook , Pedro Falcato , Shakeel Butt , Danielle Constantino Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org X-Mailer: b4 0.14.3 When kmalloc caches are aliased, multiple cache pointers reference the same kmem_cache. As a result, iterating over kmalloc indices and bootstrapping sheaves can bootstrap the same cache more than once and leak memory. Currently, this could happen when the architecture specifies minimum alignment for slab caches that is larger than ARCH_KMALLOC_MINALIGN. Bootstrap sheaves only when the cache does not have them already. Add a warning when bootstrap_cache_sheaves() is called for a cache that already has sheaves enabled. Cc: stable@vger.kernel.org Fixes: 913ffd3a1bf5 ("slab: handle kmalloc sheaves bootstrap") Signed-off-by: Harry Yoo (Oracle) Reviewed-by: Suren Baghdasaryan Tested-by: Shakeel Butt --- mm/slub.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 65febe957886..f9461a0c47d3 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -8497,6 +8497,8 @@ static void __init bootstrap_cache_sheaves(struct kme= m_cache *s) bool failed =3D false; int node, cpu; =20 + VM_WARN_ON_ONCE(cache_has_sheaves(s)); + capacity =3D calculate_sheaf_capacity(s, &empty_args); =20 /* capacity can be 0 due to debugging or SLUB_TINY */ @@ -8548,8 +8550,11 @@ static void __init bootstrap_kmalloc_sheaves(void) =20 for (type =3D KMALLOC_NORMAL; type <=3D KMALLOC_PARTITION_END; type++) { for (int idx =3D 0; idx < KMALLOC_SHIFT_HIGH + 1; idx++) { - if (kmalloc_caches[type][idx]) - bootstrap_cache_sheaves(kmalloc_caches[type][idx]); + struct kmem_cache *s =3D kmalloc_caches[type][idx]; + + /* Do not bootstrap twice when caches are aliased */ + if (s && !cache_has_sheaves(s)) + bootstrap_cache_sheaves(s); } } } --=20 2.53.0 From nobody Sat Jul 25 21:19:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79E272D060D for ; Mon, 13 Jul 2026 14:29:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952952; cv=none; b=UJXiEXygTotohJA2ohtX6vvealT+o0HddOAGiBBKu8oOQVSayvglHgsarthPbdpbYdRKuRRxDZPbVdlzgKqEDqkNOBEz5yr1QzKWQzcUHGSXfe7+iQU2lZPwMsj1465RUwdwKVidne+K5xsKHaGQO3ETQp1/Bb81Nvjy23JaMv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952952; c=relaxed/simple; bh=K1a3EtzTeF+HcDRELSmK01zHs39BG2hyHFPstEtuRBI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=B7J1thyAE4nUOIJoxa/MqF5i56l0d0e7LdV1OfLrU1+TcfXagikmK+ArmcoMDdTx6BwDaBbWN6iOXgXxnXayn5CHnUlh1yGm3zNQMFATVW1w9cwWQIzKq7JRxBJF8GEjHOrUQdXVMD0kcqM0GP5ebtbfrFw7GpReRcC6P/e6oBI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZQJ58xVv; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZQJ58xVv" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B86C21F000E9; Mon, 13 Jul 2026 14:29:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1783952951; bh=vDVLveXJA8tyZM5x+2AUKuT7u3jiA9Hoaj06+pMMJeE=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=ZQJ58xVvCyDtYjqCq2vQ9fn6eA8hROHL62vQG/NKYOJtBYpofOJt8rhG1UdEODJrg VCgH+2ZXab3dlhStSwalPlGfgkt6JQxjYXh0UFNVHFZIkPKNwVXvXzXDjXk1q2VuTV YjsYPHrA/RZx7+Mcf0sCZda3bytbJ/Hv8MEd0Y17YF5bTTs4Y8HcJTB4lpvY65t4lu uU+w8DsylDL7x2YjVkF/BDdrfEec8Nq4f5agwpvyK4/hqITgAMQQhRof42QzyPmTvk f2I/M+6BHWZtIFtutnxcFK7pYzy+Ca7guzfr9YUmk0/WqCw4JOi/0O0PO9ci7tCtZr Du4K/95AV7g+w== From: "Harry Yoo (Oracle)" Date: Mon, 13 Jul 2026 23:28:50 +0900 Subject: [PATCH slab/for-next-fixes v3 2/4] mm/slab: decouple SLAB_NO_SHEAVES from SLAB_NO_OBJ_EXT Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260713-kmalloc-no-objext-v3-2-47c7bd138de7@kernel.org> References: <20260713-kmalloc-no-objext-v3-0-47c7bd138de7@kernel.org> In-Reply-To: <20260713-kmalloc-no-objext-v3-0-47c7bd138de7@kernel.org> To: Vlastimil Babka , Harry Yoo , Andrew Morton , Hao Li , Christoph Lameter , David Rientjes , Roman Gushchin , "Liam R. Howlett" , Suren Baghdasaryan , Hao Ge , Kees Cook , Pedro Falcato , Shakeel Butt , Danielle Constantino Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org X-Mailer: b4 0.14.3 Bootstrap caches are created with SLAB_NO_OBJ_EXT to disallow sheaves and obj_exts. To allow disabling obj_exts while allowing sheaves, decouple SLAB_NO_SHEAVES from SLAB_NO_OBJ_EXT. Bootstrap caches now have both SLAB_NO_SHEAVES and SLAB_NO_OBJ_EXT. No functional change intended. Cc: stable@vger.kernel.org Fixes: e47c897a2949 ("slab: add sheaves to most caches") Reviewed-by: Vlastimil Babka (SUSE) Signed-off-by: Harry Yoo (Oracle) Reviewed-by: Suren Baghdasaryan Tested-by: Shakeel Butt --- include/linux/slab.h | 13 +++++++++++-- mm/slub.c | 10 ++++++---- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 51f03f18c9a7..08d7b6c9c4d6 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -58,10 +58,13 @@ enum _slab_flag_bits { #endif _SLAB_OBJECT_POISON, _SLAB_CMPXCHG_DOUBLE, +#ifdef CONFIG_SLAB_OBJ_EXT _SLAB_NO_OBJ_EXT, -#if defined(CONFIG_SLAB_OBJ_EXT) && defined(CONFIG_64BIT) +#ifdef CONFIG_64BIT _SLAB_OBJ_EXT_IN_OBJ, #endif +#endif + _SLAB_NO_SHEAVES, _SLAB_FLAGS_LAST_BIT }; =20 @@ -239,8 +242,14 @@ enum _slab_flag_bits { #endif #define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */ =20 -/* Slab created using create_boot_cache */ +/* Slab caches without obj_exts array */ +#ifdef CONFIG_SLAB_OBJ_EXT #define SLAB_NO_OBJ_EXT __SLAB_FLAG_BIT(_SLAB_NO_OBJ_EXT) +#else +#define SLAB_NO_OBJ_EXT __SLAB_FLAG_UNUSED +#endif + +#define SLAB_NO_SHEAVES __SLAB_FLAG_BIT(_SLAB_NO_SHEAVES) =20 #if defined(CONFIG_SLAB_OBJ_EXT) && defined(CONFIG_64BIT) #define SLAB_OBJ_EXT_IN_OBJ __SLAB_FLAG_BIT(_SLAB_OBJ_EXT_IN_OBJ) diff --git a/mm/slub.c b/mm/slub.c index f9461a0c47d3..abe748b7dddb 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -7799,12 +7799,12 @@ static unsigned int calculate_sheaf_capacity(struct= kmem_cache *s, return 0; =20 /* - * Bootstrap caches can't have sheaves for now (SLAB_NO_OBJ_EXT). + * Bootstrap caches can't have sheaves for now (SLAB_NO_SHEAVES). * SLAB_NOLEAKTRACE caches (e.g., kmemleak's object_cache) must not * have sheaves to avoid recursion when sheaf allocation triggers * kmemleak tracking. */ - if (s->flags & (SLAB_NO_OBJ_EXT | SLAB_NOLEAKTRACE)) + if (s->flags & (SLAB_NO_SHEAVES | SLAB_NOLEAKTRACE)) return 0; =20 /* @@ -8586,7 +8586,8 @@ void __init kmem_cache_init(void) =20 create_boot_cache(kmem_cache_node, "kmem_cache_node", sizeof(struct kmem_cache_node), - SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0); + SLAB_HWCACHE_ALIGN | SLAB_NO_SHEAVES | SLAB_NO_OBJ_EXT, + 0, 0); =20 hotplug_node_notifier(slab_memory_callback, SLAB_CALLBACK_PRI); =20 @@ -8596,7 +8597,8 @@ void __init kmem_cache_init(void) create_boot_cache(kmem_cache, "kmem_cache", offsetof(struct kmem_cache, per_node) + nr_node_ids * sizeof(struct kmem_cache_per_node_ptrs), - SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0); + SLAB_HWCACHE_ALIGN | SLAB_NO_SHEAVES | SLAB_NO_OBJ_EXT, + 0, 0); =20 kmem_cache =3D bootstrap(&boot_kmem_cache); kmem_cache_node =3D bootstrap(&boot_kmem_cache_node); --=20 2.53.0 From nobody Sat Jul 25 21:19:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12D1F43747E for ; Mon, 13 Jul 2026 14:29:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952956; cv=none; b=RtRyLwhEn649ZAiMDe57U11s5ztao59FIwlusgKfz4wUbIBR7QbEhiWtrnBcayQjSnSLkmUFo2yLagusmA4d07U3JO/MB77JCEwPW+ECxPzX7pLZ3bAzQY1t21sxaiIFw6n7ftH4oidc7rR9k7aB1Ms5F7V8dPlev7UnB0RuIPw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952956; c=relaxed/simple; bh=SVjaGVBQ6fVJ7TeOg7nz4OYxhsCIprN42ZdttG5AMUU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=h3aP41NdbAYE0dfEEkRzPPwn7VA3zx53v5FkeirxG5CN4xM/7GodUaIDP2rFsaec+TmXSpnlP04uxVbv7jeqn7pSJx7B3MCQDQVLKJYkl4OJw4a1m9B+7zSJCoO8MNrrvTkgHRBlfBL21pgt29fG7eJunEJKuakhapMAjQ6Foco= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LEQmqES1; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LEQmqES1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 768411F00A3A; Mon, 13 Jul 2026 14:29:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1783952955; bh=1wi/OOk6iyYw3fd8OIurB+6r9k/Y8S3vhngrBwnEdq4=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=LEQmqES12IM7BZYGacYv5CkjZGspMkqnLIom6M+NjntyYlJdl+QmrQSBCnIMDwraJ HIkc/SfwOyoxNJY3jNzXF2rIJPxUqe5gzty/+ey68UhtmgvthyIRA8PhjTWWvIMk+5 DHaVAiTwOGrDO57KH3f62eALuJeGz6lIICfiuxD6gKWFHIsBt0QYs0Rh9bYPVrY3Xj XWBcsfYxfAGSm2a03xVW9WGQg5c+rp7XJUVbdfUETtHhg8DKhF3q+IYLYTOj1QMJhh /3/gUfgw/r76Vh+cVLREKvrG+Dx6C7FYdaeBz4deKREOInQgxxcV4QOnfrBOcE6I6r DGfbRxALVxyDw== From: "Harry Yoo (Oracle)" Date: Mon, 13 Jul 2026 23:28:51 +0900 Subject: [PATCH slab/for-next-fixes v3 3/4] lib/alloc_tag: introduce mem_alloc_profiling_permanently_disabled() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260713-kmalloc-no-objext-v3-3-47c7bd138de7@kernel.org> References: <20260713-kmalloc-no-objext-v3-0-47c7bd138de7@kernel.org> In-Reply-To: <20260713-kmalloc-no-objext-v3-0-47c7bd138de7@kernel.org> To: Vlastimil Babka , Harry Yoo , Andrew Morton , Hao Li , Christoph Lameter , David Rientjes , Roman Gushchin , "Liam R. Howlett" , Suren Baghdasaryan , Hao Ge , Kees Cook , Pedro Falcato , Shakeel Butt , Danielle Constantino Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org X-Mailer: b4 0.14.3 mem_alloc_profiling_enabled() tells whether memalloc profiling is currently enabled. However, even when this function returns false, it can be enabled later. However, this is not enough. Some optimizations can be applied only when memalloc profiling is permanently disabled. For example, to skip the creation of KMALLOC_NO_OBJ_EXT caches at boot time, mem_profiling must be set to "never", "0" w/ debugging on, or have been shutdown so that it can no longer be enabled. Introduce mem_alloc_profiling_permanently_disabled() for this purpose. Fixes: 4b8736964640 ("mm/slab: add allocation accounting into slab allocati= on and free paths") Cc: stable@vger.kernel.org Signed-off-by: Harry Yoo (Oracle) Acked-by: Suren Baghdasaryan Tested-by: Shakeel Butt --- include/linux/alloc_tag.h | 3 +++ lib/alloc_tag.c | 9 +++++++++ 2 files changed, 12 insertions(+) diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index 068ba2e77c5d..6ed9f82e639f 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -134,6 +134,8 @@ static inline bool mem_alloc_profiling_enabled(void) &mem_alloc_profiling_key); } =20 +bool mem_alloc_profiling_permanently_disabled(void); + static inline struct alloc_tag_counters alloc_tag_read(struct alloc_tag *t= ag) { struct alloc_tag_counters v =3D { 0, 0 }; @@ -239,6 +241,7 @@ static inline bool alloc_tag_is_inaccurate(struct alloc= _tag *tag) =20 #define DEFINE_ALLOC_TAG(_alloc_tag) static inline bool mem_alloc_profiling_enabled(void) { return false; } +static inline bool mem_alloc_profiling_permanently_disabled(void) { return= true; } static inline void alloc_tag_add(union codetag_ref *ref, struct alloc_tag = *tag, size_t bytes) {} static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {} diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c index d9be1cf5187d..e5b218176c5a 100644 --- a/lib/alloc_tag.c +++ b/lib/alloc_tag.c @@ -26,6 +26,15 @@ static bool mem_profiling_support =3D true; static bool mem_profiling_support; #endif =20 +/* + * Memory allocation profiling is permanently disabled and cannot be enabl= ed. + * Must be called after setup_early_mem_profiling(). + */ +bool mem_alloc_profiling_permanently_disabled(void) +{ + return !mem_profiling_support; +} + static struct codetag_type *alloc_tag_cttype; =20 #ifdef CONFIG_ARCH_MODULE_NEEDS_WEAK_PER_CPU --=20 2.53.0 From nobody Sat Jul 25 21:19:03 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A570D438FFD for ; Mon, 13 Jul 2026 14:29:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952960; cv=none; b=D6W3aIt6LXa8XQ2ML2GMLNqdo8l9CKAuIZyArsONJK/TmzPCIw8BGUbp5aUqTif+3AJssExNW+AoKmVMcEIqHs9uQDWDSRqs73EFvCbPj7sIoG0eVKE7roQRyJwhZh8jXCxbGuGxjbnq7Mp78N1vqqXQZ7/oz3OdKZ+kTXWrKo4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952960; c=relaxed/simple; bh=Q9Dcv+mKv2Rh35FW9oyvoJ0ysMbLyuI1iRNb1AaqS1A=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rmoIecdnUIRLSEBrQhV4RlPR6yI7khk1rAAOHrGaJ28mLsHuaLIyK9dREo0wHYSNxef0Vgn6RgQwW3C2on02PnNh3CSuOi+5mrbutzp4kidrxPc2u0fK3rT226f7YCqJiElbGkAkOXMiVBEYR4cPhZC3ao/l3s9xng2N7uE78ik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VNg/PuJN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VNg/PuJN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 645821F000E9; Mon, 13 Jul 2026 14:29:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1783952958; bh=M+dMLsE0vS/qsVUxRHHDfljHJQyJQ2zcN6UK8b+H6iM=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=VNg/PuJNIXLPlO5c9U5yV6vhP/qp5kN9xrWyxyDFjnUxJhSDa8t9LterpHt1xgFT+ ADL+CKBq4DTQn0tFyEXtXPp0FmKTuCSPcHJKn/vjD+7KIA8Glsjm7yNBTfoxOmnjyw lubOJx1HKrBquguQt0y33Sc6lm8Xo4bPTPwYesN3cRhlpVHpBKc0RAygui1xGeH585 7ISUESL8jKJAW0QhWt6hBBaDZPG4ItF7Y000A1zjGYCgXm6GR5nT2ZpBiVri4y5qBY 6Khk5vT4TNq1vJFlM83LN6WPk50zlpVbOSEU3O8vZfKbE7QBZSfJyBTmCedEj+1wlX ZTOZIj3abCQnw== From: "Harry Yoo (Oracle)" Date: Mon, 13 Jul 2026 23:28:52 +0900 Subject: [PATCH slab/for-next-fixes v3 4/4] mm/slab: prevent unbounded recursion in free path with new kmalloc type Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260713-kmalloc-no-objext-v3-4-47c7bd138de7@kernel.org> References: <20260713-kmalloc-no-objext-v3-0-47c7bd138de7@kernel.org> In-Reply-To: <20260713-kmalloc-no-objext-v3-0-47c7bd138de7@kernel.org> To: Vlastimil Babka , Harry Yoo , Andrew Morton , Hao Li , Christoph Lameter , David Rientjes , Roman Gushchin , "Liam R. Howlett" , Suren Baghdasaryan , Hao Ge , Kees Cook , Pedro Falcato , Shakeel Butt , Danielle Constantino Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org X-Mailer: b4 0.14.3 Commit 280ea9c3154b ("mm/slab: avoid allocating slabobj_ext array from its own slab") avoided recursive allocation of obj_exts from kmalloc caches of the same size, by bumping the obj_exts array's allocation size whenever the array size equals the size of the object being allocated. However, as reported by Danielle Costantino and Shakeel Butt, even slabs from kmalloc caches of different sizes can form a cycle by allocating obj_exts arrays from each other [1]: What happened: a KMALLOC_NORMAL slab's obj_exts array (used by allocation profiling / memcg accounting) is itself kmalloc()'d from a KMALLOC_NORMAL cache, so the "slab holds another slab's obj_exts array" relation can form cycles. With sizeof(struct slabobj_ext) =3D=3D 16 and the host's geometry: - kmalloc-512 has 64 objects/slab -> array is 64*16 =3D=3D 1024 bytes, served from kmalloc-1k; - kmalloc-1k has 32 objects/slab -> array is 32*16 =3D=3D 512 bytes, served from kmalloc-512. A kmalloc-512 slab and a kmalloc-1k slab therefore hold each other's obj_exts array. Discarding one frees the other's array, which empties and discards that slab, which frees the first's array, and so on: __free_slab() -> free_slab_obj_exts() -> kfree() -> discard_slab() -> __free_slab() recurses along the cycle until the stack is exhausted. With memory allocation profiling, this allows unbounded recursion in the free path and led to a stack overflow on a production host in the Meta fleet [1]: BUG: TASK stack guard page was hit Oops: stack guard page RIP: 0010:kfree+0x8/0x5d0 Call Trace: __free_slab+0x66/0xc0 kfree+0x3f0/0x5d0 ... ( ~125x __free_slab <-> kfree ) ... do_syscall_64 It is proposed [1] to resolve this issue by always serving the obj_exts array allocation from kmalloc caches (or large kmalloc) of sizes larger than the object size. However, as pointed out by Vlastimil Babka [2], this can waste an excessive amount of memory as slabs from large kmalloc sizes (e.g. kmalloc-8k) generally need obj_exts arrays much smaller than the object size. Therefore, rather than bumping the size, let us take a different approach; disallow formation of cycles between kmalloc types when allocating obj_exts arrays. Currently, all obj_exts arrays are served from normal kmalloc caches. Cycles cannot be created if obj_exts arrays of normal kmalloc caches are served from a special kmalloc type that can never have obj_exts arrays. To achieve this, create a new kmalloc type called KMALLOC_NO_OBJ_EXT. KMALLOC_NO_OBJ_EXT caches are created with SLAB_NO_OBJ_EXT flag when either 1) memory allocation profiling is not permanently disabled, or 2) kmalloc types with a priority higher than KMALLOC_CGROUP are aliased with KMALLOC_NORMAL. Sheaf bootstrapping for KMALLOC_NO_OBJ_EXT caches now must be deferred because allocation of a barn can trigger obj_exts array allocation of normal kmalloc caches when the KMALLOC_NO_OBJ_EXT cache for that size is not ready yet. For simplicity, perform bootstrapping of sheaves for all kmalloc caches later. Introduce a new slab alloc flag, SLAB_ALLOC_NO_OBJ_EXT, to prevent allocation of obj_exts arrays, and let kmalloc_slab() override the type to KMALLOC_NO_OBJ_EXT when specified. Note that kmalloc_type() remains unchanged because kmalloc_flags() bypasses the kmalloc fastpath. Do not pass SLAB_ALLOC_NO_RECURSE to kmalloc_flags() in alloc_slab_obj_exts() and instead use SLAB_ALLOC_NO_OBJ_EXT only when the objects are allocated from normal kmalloc caches. While this prevents unbounded recursive allocation of obj_exts, it allows KMALLOC_NO_OBJ_EXT caches to have sheaves. Since sheaf allocations specify SLAB_ALLOC_NO_RECURSE that prevents allocation of both sheaves and obj_exts arrays, the recursion depth is bounded. obj_exts arrays for non-kmalloc-normal caches can now have a valid tag. Do not call mark_obj_codetag_empty() when freeing an obj_exts array to avoid false warnings. KMALLOC_NO_OBJ_EXT don't need this as they never allocate those arrays. Reported-by: Danielle Costantino Reported-by: Shakeel Butt Closes: https://lore.kernel.org/linux-mm/20260625230029.703750-1-shakeel.bu= tt@linux.dev [1] Fixes: 4b8736964640 ("mm/slab: add allocation accounting into slab allocati= on and free paths") Cc: stable@vger.kernel.org Link: https://lore.kernel.org/linux-mm/c5c4208d-a6f0-413e-bad9-49be12f12d55= @kernel.org [2] Signed-off-by: Harry Yoo (Oracle) Reviewed-by: Suren Baghdasaryan Tested-by: Shakeel Butt --- include/linux/slab.h | 6 ++++ mm/slab.h | 32 ++++++++++++++++-- mm/slab_common.c | 19 ++++++++++- mm/slub.c | 92 +++++++++++++++++++-----------------------------= ---- 4 files changed, 87 insertions(+), 62 deletions(-) diff --git a/include/linux/slab.h b/include/linux/slab.h index 08d7b6c9c4d6..3a14df50766a 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -708,6 +708,9 @@ enum kmalloc_cache_type { #endif #ifndef CONFIG_MEMCG KMALLOC_CGROUP =3D KMALLOC_NORMAL, +#endif +#ifndef CONFIG_SLAB_OBJ_EXT + KMALLOC_NO_OBJ_EXT =3D KMALLOC_NORMAL, #endif KMALLOC_PARTITION_START =3D KMALLOC_NORMAL, KMALLOC_PARTITION_END =3D KMALLOC_PARTITION_START + KMALLOC_PARTITION_CAC= HES_NR, @@ -721,6 +724,9 @@ enum kmalloc_cache_type { #endif #ifdef CONFIG_MEMCG KMALLOC_CGROUP, +#endif +#ifdef CONFIG_SLAB_OBJ_EXT + KMALLOC_NO_OBJ_EXT, #endif NR_KMALLOC_TYPES }; diff --git a/mm/slab.h b/mm/slab.h index 281a65233795..f5e336b6b6b0 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -22,6 +22,7 @@ #define SLAB_ALLOC_NOLOCK 0x01 /* a kmalloc_nolock() allocation */ #define SLAB_ALLOC_NEW_SLAB 0x02 /* a flag for alloc_slab_obj_exts() */ #define SLAB_ALLOC_NO_RECURSE 0x04 /* prevent kmalloc() recursion */ +#define SLAB_ALLOC_NO_OBJ_EXT 0x08 /* prevent obj_exts array allocation */ =20 static inline bool alloc_flags_allow_spinning(const unsigned int alloc_fla= gs) { @@ -386,12 +387,17 @@ static inline unsigned int size_index_elem(unsigned i= nt bytes) * KMALLOC_MAX_CACHE_SIZE and the caller must check that. */ static inline struct kmem_cache * -kmalloc_slab(size_t size, kmem_buckets *b, gfp_t flags, kmalloc_token_t to= ken) +kmalloc_slab(size_t size, kmem_buckets *b, gfp_t flags, kmalloc_token_t to= ken, + unsigned int alloc_flags) { unsigned int index; + enum kmalloc_cache_type type =3D kmalloc_type(flags, token); + + if (alloc_flags & SLAB_ALLOC_NO_OBJ_EXT) + type =3D KMALLOC_NO_OBJ_EXT; =20 if (!b) - b =3D &kmalloc_caches[kmalloc_type(flags, token)]; + b =3D &kmalloc_caches[type]; if (size <=3D 192) index =3D kmalloc_size_index[size_index_elem(size)]; else @@ -426,7 +432,8 @@ static inline bool is_kmalloc_normal(struct kmem_cache = *s) { if (!is_kmalloc_cache(s)) return false; - return !(s->flags & (SLAB_CACHE_DMA|SLAB_ACCOUNT|SLAB_RECLAIM_ACCOUNT)); + + return !(s->flags & (SLAB_CACHE_DMA|SLAB_ACCOUNT|SLAB_RECLAIM_ACCOUNT|SLA= B_NO_OBJ_EXT)); } =20 bool __kfree_rcu_sheaf(struct kmem_cache *s, void *obj); @@ -529,6 +536,25 @@ static inline void metadata_access_disable(void) kasan_enable_current(); } =20 +/* + * Return true if KMALLOC_NORMAL caches may need obj_exts arrays. + * + * Memory allocation profiling requires obj_exts for all caches. + * Memcg usually doesn't need them for normal kmalloc caches, but kmalloc = types + * with a priority higher than KMALLOC_CGROUP can be aliased with KMALLOC_= NORMAL. + */ +static inline bool need_kmalloc_no_objext(void) +{ + if (!mem_alloc_profiling_permanently_disabled()) + return true; + + if (!mem_cgroup_kmem_disabled() && + (KMALLOC_NORMAL =3D=3D KMALLOC_RECLAIM)) + return true; + + return false; +} + #ifdef CONFIG_SLAB_OBJ_EXT =20 /* diff --git a/mm/slab_common.c b/mm/slab_common.c index b6426d7ceec9..03ecac12cd86 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -783,11 +783,15 @@ u8 kmalloc_size_index[24] __ro_after_init =3D { size_t kmalloc_size_roundup(size_t size) { if (size && size <=3D KMALLOC_MAX_CACHE_SIZE) { + struct kmem_cache *s; + /* * The flags don't matter since size_index is common to all. * Neither does the caller for just getting ->object_size. */ - return kmalloc_slab(size, NULL, GFP_KERNEL, __kmalloc_token(0))->object_= size; + s =3D kmalloc_slab(size, NULL, GFP_KERNEL, __kmalloc_token(0), + SLAB_ALLOC_DEFAULT); + return s->object_size; } =20 /* Above the smaller buckets, size is a multiple of page size. */ @@ -843,6 +847,12 @@ EXPORT_SYMBOL(kmalloc_size_roundup); #define KMALLOC_PARTITION_NAME(N, sz) #endif =20 +#ifdef CONFIG_SLAB_OBJ_EXT +#define KMALLOC_NO_OBJ_EXT_NAME(sz) .name[KMALLOC_NO_OBJ_EXT] =3D "kmalloc= -no-objext-" #sz, +#else +#define KMALLOC_NO_OBJ_EXT_NAME(sz) +#endif + #define INIT_KMALLOC_INFO(__size, __short_size) \ { \ .name[KMALLOC_NORMAL] =3D "kmalloc-" #__short_size, \ @@ -850,6 +860,7 @@ EXPORT_SYMBOL(kmalloc_size_roundup); KMALLOC_CGROUP_NAME(__short_size) \ KMALLOC_DMA_NAME(__short_size) \ KMALLOC_PARTITION_NAME(KMALLOC_PARTITION_CACHES_NR, __short_size) \ + KMALLOC_NO_OBJ_EXT_NAME(__short_size) \ .size =3D __size, \ } =20 @@ -957,6 +968,12 @@ new_kmalloc_cache(int idx, enum kmalloc_cache_type typ= e) return; } flags |=3D SLAB_ACCOUNT; + } else if (IS_ENABLED(CONFIG_SLAB_OBJ_EXT) && type =3D=3D KMALLOC_NO_OBJ_= EXT) { + if (!need_kmalloc_no_objext()) { + kmalloc_caches[type][idx] =3D kmalloc_caches[KMALLOC_NORMAL][idx]; + return; + } + flags |=3D SLAB_NO_OBJ_EXT | SLAB_NO_MERGE; } else if (IS_ENABLED(CONFIG_ZONE_DMA) && (type =3D=3D KMALLOC_DMA)) { flags |=3D SLAB_CACHE_DMA; } diff --git a/mm/slub.c b/mm/slub.c index abe748b7dddb..a34f9b8770dc 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2123,42 +2123,6 @@ static inline void init_slab_obj_exts(struct slab *s= lab) slab->obj_exts =3D 0; } =20 -/* - * Calculate the allocation size for slabobj_ext array. - * - * When memory allocation profiling is enabled, the obj_exts array - * could be allocated from the same slab cache it's being allocated for. - * This would prevent the slab from ever being freed because it would - * always contain at least one allocated object (its own obj_exts array). - * - * To avoid this, increase the allocation size when we detect the array - * may come from the same cache, forcing it to use a different cache. - */ -static inline size_t obj_exts_alloc_size(struct kmem_cache *s, - struct slab *slab, gfp_t gfp) -{ - size_t sz =3D sizeof(struct slabobj_ext) * slab->objects; - struct kmem_cache *obj_exts_cache; - - if (sz > KMALLOC_MAX_CACHE_SIZE) - return sz; - - if (!is_kmalloc_normal(s)) - return sz; - - obj_exts_cache =3D kmalloc_slab(sz, NULL, gfp, __kmalloc_token(0)); - /* - * We can't simply compare s with obj_exts_cache, because partitioned kma= lloc - * caches have multiple caches per size, selected by caller address or ty= pe. - * Since caller address or type may differ between kmalloc_slab() and act= ual - * allocation, bump size when sizes are equal. - */ - if (s->object_size =3D=3D obj_exts_cache->object_size) - return obj_exts_cache->object_size + 1; - - return sz; -} - int alloc_slab_obj_exts(struct slab *slab, struct kmem_cache *s, gfp_t gfp, unsigned int alloc_flags) { @@ -2168,14 +2132,20 @@ int alloc_slab_obj_exts(struct slab *slab, struct k= mem_cache *s, unsigned long new_exts; unsigned long old_exts; struct slabobj_ext *vec; - size_t sz; + size_t sz =3D sizeof(struct slabobj_ext) * slab->objects; =20 gfp &=3D ~OBJCGS_CLEAR_MASK; - /* Prevent recursive extension vector allocation */ - alloc_flags |=3D SLAB_ALLOC_NO_RECURSE; - alloc_flags &=3D ~SLAB_ALLOC_NEW_SLAB; + /* + * In most cases, obj_exts arrays are allocated from normal kmalloc. + * However, normal kmalloc caches must allocate them from + * KMALLOC_NO_OBJ_EXT caches to prevent recursion. + */ + if (is_kmalloc_normal(s)) { + VM_WARN_ON_ONCE(!need_kmalloc_no_objext()); + alloc_flags |=3D SLAB_ALLOC_NO_OBJ_EXT; + } =20 - sz =3D obj_exts_alloc_size(s, slab, gfp); + alloc_flags &=3D ~SLAB_ALLOC_NEW_SLAB; =20 /* This will use kmalloc_nolock() if alloc_flags say so */ vec =3D kmalloc_flags(sz, gfp | __GFP_ZERO, alloc_flags, slab_nid(slab)); @@ -2193,8 +2163,21 @@ int alloc_slab_obj_exts(struct slab *slab, struct km= em_cache *s, return -ENOMEM; } =20 - VM_WARN_ON_ONCE(virt_to_slab(vec) !=3D NULL && - virt_to_slab(vec)->slab_cache =3D=3D s); + if (IS_ENABLED(CONFIG_DEBUG_VM)) { + struct kmem_cache *exts_cache; + struct slab *exts_slab; + + exts_slab =3D virt_to_slab(vec); + if (exts_slab) { + /* + * The vector must be allocated from either normal or + * KMALLOC_NO_OBJ_EXT kmalloc caches to avoid cycles. + */ + exts_cache =3D exts_slab->slab_cache; + WARN_ON_ONCE(!is_kmalloc_normal(exts_cache) && + !(exts_cache->flags & SLAB_NO_OBJ_EXT)); + } + } =20 new_exts =3D (unsigned long)vec; #ifdef CONFIG_MEMCG @@ -2217,7 +2200,6 @@ int alloc_slab_obj_exts(struct slab *slab, struct kme= m_cache *s, * assign slabobj_exts in parallel. In this case the existing * objcg vector should be reused. */ - mark_obj_codetag_empty(vec); if (unlikely(!allow_spin)) kfree_nolock(vec); else @@ -2253,14 +2235,6 @@ static inline void free_slab_obj_exts(struct slab *s= lab, bool allow_spin) return; } =20 - /* - * obj_exts was created with SLAB_ALLOC_NO_RECURSE flag, therefore its - * corresponding extension will be NULL. alloc_tag_sub() will throw a - * warning if slab has extensions but the extension of an object is - * NULL, therefore replace NULL with CODETAG_EMPTY to indicate that - * the extension for obj_exts is expected to be NULL. - */ - mark_obj_codetag_empty(obj_exts); if (allow_spin) kfree(obj_exts); else @@ -5356,7 +5330,7 @@ void *__do_kmalloc_node(kmem_buckets *b, gfp_t flags,= int node, if (unlikely(!size)) return ZERO_SIZE_PTR; =20 - s =3D kmalloc_slab(size, b, flags, token); + s =3D kmalloc_slab(size, b, flags, token, ac->alloc_flags); =20 ret =3D slab_alloc_node(s, flags, node, ac); ret =3D kasan_kmalloc(s, ret, size, flags); @@ -5419,7 +5393,9 @@ static void *__kmalloc_nolock_noprof(DECL_TOKEN_PARAM= S(size, token), gfp_t gfp_f retry: if (unlikely(size > KMALLOC_MAX_CACHE_SIZE)) return NULL; - s =3D kmalloc_slab(size, NULL, gfp_flags, PASS_TOKEN_PARAM(token)); + + s =3D kmalloc_slab(size, NULL, gfp_flags, PASS_TOKEN_PARAM(token), + ac->alloc_flags); =20 if (!(s->flags & __CMPXCHG_DOUBLE) && !kmem_cache_debug(s)) /* @@ -7979,10 +7955,10 @@ static int calculate_sizes(struct kmem_cache_args *= args, struct kmem_cache *s) s->allocflags |=3D __GFP_RECLAIMABLE; =20 /* - * For KMALLOC_NORMAL caches we enable sheaves later by - * bootstrap_kmalloc_sheaves() to avoid recursion + * For kmalloc caches we enable sheaves later by + * bootstrap_kmalloc_sheaves() to avoid recursion. */ - if (!is_kmalloc_normal(s)) + if (!is_kmalloc_cache(s)) s->sheaf_capacity =3D calculate_sheaf_capacity(s, args); =20 /* @@ -8548,7 +8524,7 @@ static void __init bootstrap_kmalloc_sheaves(void) { enum kmalloc_cache_type type; =20 - for (type =3D KMALLOC_NORMAL; type <=3D KMALLOC_PARTITION_END; type++) { + for (type =3D KMALLOC_NORMAL; type < NR_KMALLOC_TYPES; type++) { for (int idx =3D 0; idx < KMALLOC_SHIFT_HIGH + 1; idx++) { struct kmem_cache *s =3D kmalloc_caches[type][idx]; =20 --=20 2.53.0