In the first step to replace cpu (partial) slabs with sheaves, enable
sheaves for almost all caches. Treat args->sheaf_capacity as a minimum,
and calculate sheaf capacity with a formula that roughly follows the
formula for number of objects in cpu partial slabs in set_cpu_partial().
This should achieve roughly similar contention on the barn spin lock as
there's currently for node list_lock without sheaves, to make
benchmarking results comparable. It can be further tuned later.
Don't enable sheaves for bootstrap caches as that wouldn't work. In
order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists
even for !CONFIG_SLAB_OBJ_EXT.
This limitation will be lifted for kmalloc caches after the necessary
bootstrapping changes.
Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid
recursion with kmemleak tracking (thanks to Breno Leitao).
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
include/linux/slab.h | 6 ------
mm/slub.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++----
2 files changed, 52 insertions(+), 10 deletions(-)
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 2482992248dc..2682ee57ec90 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -57,9 +57,7 @@ enum _slab_flag_bits {
#endif
_SLAB_OBJECT_POISON,
_SLAB_CMPXCHG_DOUBLE,
-#ifdef CONFIG_SLAB_OBJ_EXT
_SLAB_NO_OBJ_EXT,
-#endif
_SLAB_FLAGS_LAST_BIT
};
@@ -238,11 +236,7 @@ enum _slab_flag_bits {
#define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */
/* Slab created using create_boot_cache */
-#ifdef CONFIG_SLAB_OBJ_EXT
#define SLAB_NO_OBJ_EXT __SLAB_FLAG_BIT(_SLAB_NO_OBJ_EXT)
-#else
-#define SLAB_NO_OBJ_EXT __SLAB_FLAG_UNUSED
-#endif
/*
* ZERO_SIZE_PTR will be returned for zero sized kmalloc requests.
diff --git a/mm/slub.c b/mm/slub.c
index 9d86c0505dcd..594f5fac39b3 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -7880,6 +7880,53 @@ static void set_cpu_partial(struct kmem_cache *s)
#endif
}
+static unsigned int calculate_sheaf_capacity(struct kmem_cache *s,
+ struct kmem_cache_args *args)
+
+{
+ unsigned int capacity;
+ size_t size;
+
+
+ if (IS_ENABLED(CONFIG_SLUB_TINY) || s->flags & SLAB_DEBUG_FLAGS)
+ return 0;
+
+ /*
+ * Bootstrap caches can't have sheaves for now (SLAB_NO_OBJ_EXT).
+ * SLAB_NOLEAKTRACE caches (e.g., kmemleak's object_cache) must not
+ * have sheaves to avoid recursion when sheaf allocation triggers
+ * kmemleak tracking.
+ */
+ if (s->flags & (SLAB_NO_OBJ_EXT | SLAB_NOLEAKTRACE))
+ return 0;
+
+ /*
+ * For now we use roughly similar formula (divided by two as there are
+ * two percpu sheaves) as what was used for percpu partial slabs, which
+ * should result in similar lock contention (barn or list_lock)
+ */
+ if (s->size >= PAGE_SIZE)
+ capacity = 4;
+ else if (s->size >= 1024)
+ capacity = 12;
+ else if (s->size >= 256)
+ capacity = 26;
+ else
+ capacity = 60;
+
+ /* Increment capacity to make sheaf exactly a kmalloc size bucket */
+ size = struct_size_t(struct slab_sheaf, objects, capacity);
+ size = kmalloc_size_roundup(size);
+ capacity = (size - struct_size_t(struct slab_sheaf, objects, 0)) / sizeof(void *);
+
+ /*
+ * Respect an explicit request for capacity that's typically motivated by
+ * expected maximum size of kmem_cache_prefill_sheaf() to not end up
+ * using low-performance oversize sheaves
+ */
+ return max(capacity, args->sheaf_capacity);
+}
+
/*
* calculate_sizes() determines the order and the distribution of data within
* a slab object.
@@ -8014,6 +8061,10 @@ static int calculate_sizes(struct kmem_cache_args *args, struct kmem_cache *s)
if (s->flags & SLAB_RECLAIM_ACCOUNT)
s->allocflags |= __GFP_RECLAIMABLE;
+ /* kmalloc caches need extra care to support sheaves */
+ if (!is_kmalloc_cache(s))
+ s->sheaf_capacity = calculate_sheaf_capacity(s, args);
+
/*
* Determine the number of objects per slab
*/
@@ -8618,15 +8669,12 @@ int do_kmem_cache_create(struct kmem_cache *s, const char *name,
set_cpu_partial(s);
- if (args->sheaf_capacity && !IS_ENABLED(CONFIG_SLUB_TINY)
- && !(s->flags & SLAB_DEBUG_FLAGS)) {
+ if (s->sheaf_capacity) {
s->cpu_sheaves = alloc_percpu(struct slub_percpu_sheaves);
if (!s->cpu_sheaves) {
err = -ENOMEM;
goto out;
}
- // TODO: increase capacity to grow slab_sheaf up to next kmalloc size?
- s->sheaf_capacity = args->sheaf_capacity;
}
#ifdef CONFIG_NUMA
--
2.52.0On Fri, Jan 23, 2026 at 07:52:44AM +0100, Vlastimil Babka wrote:
> Date: Fri, 23 Jan 2026 07:52:44 +0100
> From: Vlastimil Babka <vbabka@suse.cz>
> Subject: [PATCH v4 06/22] slab: add sheaves to most caches
> X-Mailer: b4 0.14.3
>
> In the first step to replace cpu (partial) slabs with sheaves, enable
> sheaves for almost all caches. Treat args->sheaf_capacity as a minimum,
> and calculate sheaf capacity with a formula that roughly follows the
> formula for number of objects in cpu partial slabs in set_cpu_partial().
>
> This should achieve roughly similar contention on the barn spin lock as
> there's currently for node list_lock without sheaves, to make
> benchmarking results comparable. It can be further tuned later.
>
> Don't enable sheaves for bootstrap caches as that wouldn't work. In
> order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists
> even for !CONFIG_SLAB_OBJ_EXT.
>
> This limitation will be lifted for kmalloc caches after the necessary
> bootstrapping changes.
>
> Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid
> recursion with kmemleak tracking (thanks to Breno Leitao).
>
> Reviewed-by: Suren Baghdasaryan <surenb@google.com>
> Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
> ---
> include/linux/slab.h | 6 ------
> mm/slub.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++----
> 2 files changed, 52 insertions(+), 10 deletions(-)
vm_area_cachep's capacity seems to be adjusted to 60 and
maple_node_cache keeps 32 as the args setting.
I still use will-it-scale to evaluate the impact of this patch, and
performance results appear to be on par with previous ones (*) - doesn't
have regression on my cases.
Based on the results of previous capacity adjustments testing, I think
it shows that the capacity of the maple_node_cache appears to have the
significant impact.
There may still be room for optimization in maple_node_cache. As a
general-purpose algorithm at present, I think it has achieved its
intended purpose based on my test results. So,
Tested-by: Zhao Liu <zhao1.liu@intel.com>
(*): The previous ones include 2 cases:
1) w/o this series, and directly based on the previous commit ("slub:
keep empty main sheaf as spare in __pcs_replace_empty_main()").
2) w/o this single patch, and based on the previous patch 5.
Regards,
Zhao
On 1/29/26 08:24, Zhao Liu wrote:
> On Fri, Jan 23, 2026 at 07:52:44AM +0100, Vlastimil Babka wrote:
>> Date: Fri, 23 Jan 2026 07:52:44 +0100
>> From: Vlastimil Babka <vbabka@suse.cz>
>> Subject: [PATCH v4 06/22] slab: add sheaves to most caches
>> X-Mailer: b4 0.14.3
>>
>> In the first step to replace cpu (partial) slabs with sheaves, enable
>> sheaves for almost all caches. Treat args->sheaf_capacity as a minimum,
>> and calculate sheaf capacity with a formula that roughly follows the
>> formula for number of objects in cpu partial slabs in set_cpu_partial().
>>
>> This should achieve roughly similar contention on the barn spin lock as
>> there's currently for node list_lock without sheaves, to make
>> benchmarking results comparable. It can be further tuned later.
>>
>> Don't enable sheaves for bootstrap caches as that wouldn't work. In
>> order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists
>> even for !CONFIG_SLAB_OBJ_EXT.
>>
>> This limitation will be lifted for kmalloc caches after the necessary
>> bootstrapping changes.
>>
>> Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid
>> recursion with kmemleak tracking (thanks to Breno Leitao).
>>
>> Reviewed-by: Suren Baghdasaryan <surenb@google.com>
>> Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
>> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
>> ---
>> include/linux/slab.h | 6 ------
>> mm/slub.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++----
>> 2 files changed, 52 insertions(+), 10 deletions(-)
>
> vm_area_cachep's capacity seems to be adjusted to 60 and
> maple_node_cache keeps 32 as the args setting.
Good to know. It is a bit larger.
Hm I could have probably applied the args capacity before doing the roundup
to make sheaf fill whole kmalloc size. Would add a few object for maple node
I guess.
> I still use will-it-scale to evaluate the impact of this patch, and
> performance results appear to be on par with previous ones (*) - doesn't
> have regression on my cases.
>
> Based on the results of previous capacity adjustments testing, I think
> it shows that the capacity of the maple_node_cache appears to have the
> significant impact.
>
> There may still be room for optimization in maple_node_cache. As a
> general-purpose algorithm at present, I think it has achieved its
> intended purpose based on my test results. So,
>
> Tested-by: Zhao Liu <zhao1.liu@intel.com>
Thanks!
>
>
> (*): The previous ones include 2 cases:
> 1) w/o this series, and directly based on the previous commit ("slub:
> keep empty main sheaf as spare in __pcs_replace_empty_main()").
> 2) w/o this single patch, and based on the previous patch 5.
>
> Regards,
> Zhao
>
>
Hi Vlastimil, > > vm_area_cachep's capacity seems to be adjusted to 60 and > > maple_node_cache keeps 32 as the args setting. > > Good to know. It is a bit larger. > Hm I could have probably applied the args capacity before doing the roundup > to make sheaf fill whole kmalloc size. Would add a few object for maple node > I guess. Re-considerring this formula: the nr_objects in set_cpu_partial() in fact represents the half-full case since it was used to calculate nr_slabs in slub_set_cpu_partial(). Therefore, the maximum capacity of this partial approach should be nr_objects * 2 (and should actually be even larger, since it doesn't account for the object on CPU's freelist). But here, for sheaf, the implicit assumption is that it is completely full, so that for the maximum capacity of objects per CPU, the sheaf approach is "half" that of the partial approach. Is this expected? I'm considering whether we should remove the “divide by two” and instead calculate the sheaf capacity based on half-full assumption (e.h., full main & empty spare). Thanks, Zhao
On 1/30/26 08:15, Zhao Liu wrote: > Hi Vlastimil, > >> > vm_area_cachep's capacity seems to be adjusted to 60 and >> > maple_node_cache keeps 32 as the args setting. >> >> Good to know. It is a bit larger. >> Hm I could have probably applied the args capacity before doing the roundup >> to make sheaf fill whole kmalloc size. Would add a few object for maple node >> I guess. > > Re-considerring this formula: > > the nr_objects in set_cpu_partial() in fact represents the half-full > case since it was used to calculate nr_slabs in slub_set_cpu_partial(). > > Therefore, the maximum capacity of this partial approach should be > nr_objects * 2 (and should actually be even larger, since it doesn't > account for the object on CPU's freelist). > > But here, for sheaf, the implicit assumption is that it is completely > full, so that for the maximum capacity of objects per CPU, the sheaf > approach is "half" that of the partial approach. > > Is this expected? I'm considering whether we should remove the > “divide by two” and instead calculate the sheaf capacity based on > half-full assumption (e.h., full main & empty spare). Yeah the non-doubling was intentional. Sheaves can be always fully filled up by freeing and fully emptied by allocating. Cpu partial slabs don't make freeing as cheap (except never needing the list_lock perhaps) because it's the locked double cmpxchg patch. For allocation, we might be obtaining them from the partial list either on allocation where they might have any number of free objects between 1 and slab size, or we put them to partial list on a first free to a full slab - and then there will be just single free object, but we hope there will be more frees before we turn that slab into cpu slab. So the half-full case is an estimate that might be even actually less as the freeing side is biased to almost-full slabs. With sheaves no such estimate is necessary. Not accounting for the cpu slab size was maybe a mistake. In any case we can tune the sizes later, but this should not be based on synthetic benchmark only. Thanks, Vlastimil > Thanks, > Zhao > >
* Vlastimil Babka <vbabka@suse.cz> [260123 01:53]:
> In the first step to replace cpu (partial) slabs with sheaves, enable
> sheaves for almost all caches. Treat args->sheaf_capacity as a minimum,
> and calculate sheaf capacity with a formula that roughly follows the
> formula for number of objects in cpu partial slabs in set_cpu_partial().
>
> This should achieve roughly similar contention on the barn spin lock as
> there's currently for node list_lock without sheaves, to make
> benchmarking results comparable. It can be further tuned later.
>
> Don't enable sheaves for bootstrap caches as that wouldn't work. In
> order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists
> even for !CONFIG_SLAB_OBJ_EXT.
>
> This limitation will be lifted for kmalloc caches after the necessary
> bootstrapping changes.
>
> Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid
> recursion with kmemleak tracking (thanks to Breno Leitao).
>
> Reviewed-by: Suren Baghdasaryan <surenb@google.com>
> Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
> Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Is there a way to force a specific limit to the sheaf capacity if you
want a lower number than what is calculated in
calculate_sheaf_capacity()? That is, it seems your code always decides
if the specified sheaf number is smaller right now. I'm not sure it's
practical to want a smaller number though.
Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com>
> ---
> include/linux/slab.h | 6 ------
> mm/slub.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++----
> 2 files changed, 52 insertions(+), 10 deletions(-)
>
> diff --git a/include/linux/slab.h b/include/linux/slab.h
> index 2482992248dc..2682ee57ec90 100644
> --- a/include/linux/slab.h
> +++ b/include/linux/slab.h
> @@ -57,9 +57,7 @@ enum _slab_flag_bits {
> #endif
> _SLAB_OBJECT_POISON,
> _SLAB_CMPXCHG_DOUBLE,
> -#ifdef CONFIG_SLAB_OBJ_EXT
> _SLAB_NO_OBJ_EXT,
> -#endif
> _SLAB_FLAGS_LAST_BIT
> };
>
> @@ -238,11 +236,7 @@ enum _slab_flag_bits {
> #define SLAB_TEMPORARY SLAB_RECLAIM_ACCOUNT /* Objects are short-lived */
>
> /* Slab created using create_boot_cache */
> -#ifdef CONFIG_SLAB_OBJ_EXT
> #define SLAB_NO_OBJ_EXT __SLAB_FLAG_BIT(_SLAB_NO_OBJ_EXT)
> -#else
> -#define SLAB_NO_OBJ_EXT __SLAB_FLAG_UNUSED
> -#endif
>
> /*
> * ZERO_SIZE_PTR will be returned for zero sized kmalloc requests.
> diff --git a/mm/slub.c b/mm/slub.c
> index 9d86c0505dcd..594f5fac39b3 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -7880,6 +7880,53 @@ static void set_cpu_partial(struct kmem_cache *s)
> #endif
> }
>
> +static unsigned int calculate_sheaf_capacity(struct kmem_cache *s,
> + struct kmem_cache_args *args)
> +
> +{
> + unsigned int capacity;
> + size_t size;
> +
> +
> + if (IS_ENABLED(CONFIG_SLUB_TINY) || s->flags & SLAB_DEBUG_FLAGS)
> + return 0;
> +
> + /*
> + * Bootstrap caches can't have sheaves for now (SLAB_NO_OBJ_EXT).
> + * SLAB_NOLEAKTRACE caches (e.g., kmemleak's object_cache) must not
> + * have sheaves to avoid recursion when sheaf allocation triggers
> + * kmemleak tracking.
> + */
> + if (s->flags & (SLAB_NO_OBJ_EXT | SLAB_NOLEAKTRACE))
> + return 0;
> +
> + /*
> + * For now we use roughly similar formula (divided by two as there are
> + * two percpu sheaves) as what was used for percpu partial slabs, which
> + * should result in similar lock contention (barn or list_lock)
> + */
> + if (s->size >= PAGE_SIZE)
> + capacity = 4;
> + else if (s->size >= 1024)
> + capacity = 12;
> + else if (s->size >= 256)
> + capacity = 26;
> + else
> + capacity = 60;
> +
> + /* Increment capacity to make sheaf exactly a kmalloc size bucket */
> + size = struct_size_t(struct slab_sheaf, objects, capacity);
> + size = kmalloc_size_roundup(size);
> + capacity = (size - struct_size_t(struct slab_sheaf, objects, 0)) / sizeof(void *);
> +
> + /*
> + * Respect an explicit request for capacity that's typically motivated by
> + * expected maximum size of kmem_cache_prefill_sheaf() to not end up
> + * using low-performance oversize sheaves
> + */
> + return max(capacity, args->sheaf_capacity);
> +}
> +
> /*
> * calculate_sizes() determines the order and the distribution of data within
> * a slab object.
> @@ -8014,6 +8061,10 @@ static int calculate_sizes(struct kmem_cache_args *args, struct kmem_cache *s)
> if (s->flags & SLAB_RECLAIM_ACCOUNT)
> s->allocflags |= __GFP_RECLAIMABLE;
>
> + /* kmalloc caches need extra care to support sheaves */
> + if (!is_kmalloc_cache(s))
> + s->sheaf_capacity = calculate_sheaf_capacity(s, args);
> +
> /*
> * Determine the number of objects per slab
> */
> @@ -8618,15 +8669,12 @@ int do_kmem_cache_create(struct kmem_cache *s, const char *name,
>
> set_cpu_partial(s);
>
> - if (args->sheaf_capacity && !IS_ENABLED(CONFIG_SLUB_TINY)
> - && !(s->flags & SLAB_DEBUG_FLAGS)) {
> + if (s->sheaf_capacity) {
> s->cpu_sheaves = alloc_percpu(struct slub_percpu_sheaves);
> if (!s->cpu_sheaves) {
> err = -ENOMEM;
> goto out;
> }
> - // TODO: increase capacity to grow slab_sheaf up to next kmalloc size?
> - s->sheaf_capacity = args->sheaf_capacity;
> }
>
> #ifdef CONFIG_NUMA
>
> --
> 2.52.0
>
>
On 1/27/26 17:34, Liam R. Howlett wrote: > * Vlastimil Babka <vbabka@suse.cz> [260123 01:53]: >> In the first step to replace cpu (partial) slabs with sheaves, enable >> sheaves for almost all caches. Treat args->sheaf_capacity as a minimum, >> and calculate sheaf capacity with a formula that roughly follows the >> formula for number of objects in cpu partial slabs in set_cpu_partial(). >> >> This should achieve roughly similar contention on the barn spin lock as >> there's currently for node list_lock without sheaves, to make >> benchmarking results comparable. It can be further tuned later. >> >> Don't enable sheaves for bootstrap caches as that wouldn't work. In >> order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists >> even for !CONFIG_SLAB_OBJ_EXT. >> >> This limitation will be lifted for kmalloc caches after the necessary >> bootstrapping changes. >> >> Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid >> recursion with kmemleak tracking (thanks to Breno Leitao). >> >> Reviewed-by: Suren Baghdasaryan <surenb@google.com> >> Reviewed-by: Harry Yoo <harry.yoo@oracle.com> >> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> > > Is there a way to force a specific limit to the sheaf capacity if you > want a lower number than what is calculated in > calculate_sheaf_capacity()? No. > That is, it seems your code always decides > if the specified sheaf number is smaller right now. That's right. It shouldn't break anything, AFAICS. > I'm not sure it's > practical to want a smaller number though. Yeah I found no such reason, so until we stumble upon one, we don't need such support I think? > Reviewed-by: Liam R. Howlett <Liam.Howlett@oracle.com> Thanks!
On Fri, Jan 23, 2026 at 07:52:44AM +0100, Vlastimil Babka wrote: > In the first step to replace cpu (partial) slabs with sheaves, enable > sheaves for almost all caches. Treat args->sheaf_capacity as a minimum, > and calculate sheaf capacity with a formula that roughly follows the > formula for number of objects in cpu partial slabs in set_cpu_partial(). > > This should achieve roughly similar contention on the barn spin lock as > there's currently for node list_lock without sheaves, to make > benchmarking results comparable. It can be further tuned later. > > Don't enable sheaves for bootstrap caches as that wouldn't work. In > order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists > even for !CONFIG_SLAB_OBJ_EXT. > > This limitation will be lifted for kmalloc caches after the necessary > bootstrapping changes. > > Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid > recursion with kmemleak tracking (thanks to Breno Leitao). > > Reviewed-by: Suren Baghdasaryan <surenb@google.com> > Reviewed-by: Harry Yoo <harry.yoo@oracle.com> > Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Tested-by: Breno Leitao <leitao@debian.org>
On Fri, Jan 23, 2026 at 07:52:44AM +0100, Vlastimil Babka wrote: > In the first step to replace cpu (partial) slabs with sheaves, enable > sheaves for almost all caches. Treat args->sheaf_capacity as a minimum, > and calculate sheaf capacity with a formula that roughly follows the > formula for number of objects in cpu partial slabs in set_cpu_partial(). > > This should achieve roughly similar contention on the barn spin lock as > there's currently for node list_lock without sheaves, to make > benchmarking results comparable. It can be further tuned later. > > Don't enable sheaves for bootstrap caches as that wouldn't work. In > order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists > even for !CONFIG_SLAB_OBJ_EXT. > > This limitation will be lifted for kmalloc caches after the necessary > bootstrapping changes. > > Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid > recursion with kmemleak tracking (thanks to Breno Leitao). > > Reviewed-by: Suren Baghdasaryan <surenb@google.com> > Reviewed-by: Harry Yoo <harry.yoo@oracle.com> > Signed-off-by: Vlastimil Babka <vbabka@suse.cz> Also, looks good to me. As a side node, while looking into the test results reported by Zhao Liu [1], I ran a quick test of the current patchset with the will-it-scale mmap2 workload. In my runs, tuning capacity up or down did indeed have a noticeable impact on performance. Hopefully we can make this tuning even smarter in follow-up work. [1] https://lore.kernel.org/linux-mm/aWi9nAbIkTfYFoMM@intel.com/ Reviewed-by: Hao Li <hao.li@linux.dev> -- Thanks, Hao
On 1/26/26 07:36, Hao Li wrote: > On Fri, Jan 23, 2026 at 07:52:44AM +0100, Vlastimil Babka wrote: >> In the first step to replace cpu (partial) slabs with sheaves, enable >> sheaves for almost all caches. Treat args->sheaf_capacity as a minimum, >> and calculate sheaf capacity with a formula that roughly follows the >> formula for number of objects in cpu partial slabs in set_cpu_partial(). >> >> This should achieve roughly similar contention on the barn spin lock as >> there's currently for node list_lock without sheaves, to make >> benchmarking results comparable. It can be further tuned later. >> >> Don't enable sheaves for bootstrap caches as that wouldn't work. In >> order to recognize them by SLAB_NO_OBJ_EXT, make sure the flag exists >> even for !CONFIG_SLAB_OBJ_EXT. >> >> This limitation will be lifted for kmalloc caches after the necessary >> bootstrapping changes. >> >> Also do not enable sheaves for SLAB_NOLEAKTRACE caches to avoid >> recursion with kmemleak tracking (thanks to Breno Leitao). >> >> Reviewed-by: Suren Baghdasaryan <surenb@google.com> >> Reviewed-by: Harry Yoo <harry.yoo@oracle.com> >> Signed-off-by: Vlastimil Babka <vbabka@suse.cz> > > Also, looks good to me. > > As a side node, while looking into the test results reported by Zhao Liu [1], I > ran a quick test of the current patchset with the will-it-scale mmap2 workload. > In my runs, tuning capacity up or down did indeed have a noticeable impact on > performance. Hopefully we can make this tuning even smarter in follow-up work. Right, thanks for checking that. > [1] https://lore.kernel.org/linux-mm/aWi9nAbIkTfYFoMM@intel.com/ > > Reviewed-by: Hao Li <hao.li@linux.dev> Thanks!
© 2016 - 2026 Red Hat, Inc.