When a slab cache has a constructor, the free pointer is placed after the
object because certain fields must not be overwritten even after the
object is freed.

However, some fields that the constructor does not care can safely be
overwritten. Allow specifying the free pointer offset within the object,
reducing the overall object size when some fields can be reused for the
free pointer.

Signed-off-by: Harry Yoo <harry.yoo@oracle.com>
---
 mm/slab_common.c | 2 +-
 mm/slub.c        | 6 ++++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/mm/slab_common.c b/mm/slab_common.c
index 932d13ada36c..2c2ed2452271 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -231,7 +231,7 @@ static struct kmem_cache *create_cache(const char *name,
 	err = -EINVAL;
 	if (args->use_freeptr_offset &&
 	    (args->freeptr_offset >= object_size ||
-	     !(flags & SLAB_TYPESAFE_BY_RCU) ||
+	     (!(flags & SLAB_TYPESAFE_BY_RCU) && !args->ctor) ||
 	     !IS_ALIGNED(args->freeptr_offset, __alignof__(freeptr_t))))
 		goto out;
 
diff --git a/mm/slub.c b/mm/slub.c
index 462a39d57b3a..64705cb3734f 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -7781,7 +7781,8 @@ static int calculate_sizes(struct kmem_cache_args *args, struct kmem_cache *s)
 	s->inuse = size;
 
 	if (((flags & SLAB_TYPESAFE_BY_RCU) && !args->use_freeptr_offset) ||
-	    (flags & SLAB_POISON) || s->ctor ||
+	    (flags & SLAB_POISON) ||
+	    (s->ctor && !args->use_freeptr_offset) ||
 	    ((flags & SLAB_RED_ZONE) &&
 	     (s->object_size < sizeof(void *) || slub_debug_orig_size(s)))) {
 		/*
@@ -7802,7 +7803,8 @@ static int calculate_sizes(struct kmem_cache_args *args, struct kmem_cache *s)
 		 */
 		s->offset = size;
 		size += sizeof(void *);
-	} else if ((flags & SLAB_TYPESAFE_BY_RCU) && args->use_freeptr_offset) {
+	} else if (((flags & SLAB_TYPESAFE_BY_RCU) || s->ctor) &&
+			args->use_freeptr_offset) {
 		s->offset = args->freeptr_offset;
 	} else {
 		/*
-- 
2.43.0

On Mon, Oct 27, 2025 at 5:29 AM Harry Yoo <harry.yoo@oracle.com> wrote:
>
> When a slab cache has a constructor, the free pointer is placed after the
> object because certain fields must not be overwritten even after the
> object is freed.
>
> However, some fields that the constructor does not care can safely be
> overwritten. Allow specifying the free pointer offset within the object,
> reducing the overall object size when some fields can be reused for the
> free pointer.

Documentation explicitly says that ctor currently isn't supported with
custom free pointers:
https://elixir.bootlin.com/linux/v6.18-rc3/source/include/linux/slab.h#L318
It obviously needs to be updated but I suspect there was a reason for
this limitation. Have you investigated why it's not supported? I
remember looking into it when I was converting vm_area_struct cache to
use SLAB_TYPESAFE_BY_RCU but I can't recall the details now...

>
> Signed-off-by: Harry Yoo <harry.yoo@oracle.com>
> ---
>  mm/slab_common.c | 2 +-
>  mm/slub.c        | 6 ++++--
>  2 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/mm/slab_common.c b/mm/slab_common.c
> index 932d13ada36c..2c2ed2452271 100644
> --- a/mm/slab_common.c
> +++ b/mm/slab_common.c
> @@ -231,7 +231,7 @@ static struct kmem_cache *create_cache(const char *name,
>         err = -EINVAL;
>         if (args->use_freeptr_offset &&
>             (args->freeptr_offset >= object_size ||
> -            !(flags & SLAB_TYPESAFE_BY_RCU) ||
> +            (!(flags & SLAB_TYPESAFE_BY_RCU) && !args->ctor) ||
>              !IS_ALIGNED(args->freeptr_offset, __alignof__(freeptr_t))))
>                 goto out;
>
> diff --git a/mm/slub.c b/mm/slub.c
> index 462a39d57b3a..64705cb3734f 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -7781,7 +7781,8 @@ static int calculate_sizes(struct kmem_cache_args *args, struct kmem_cache *s)
>         s->inuse = size;
>
>         if (((flags & SLAB_TYPESAFE_BY_RCU) && !args->use_freeptr_offset) ||
> -           (flags & SLAB_POISON) || s->ctor ||
> +           (flags & SLAB_POISON) ||
> +           (s->ctor && !args->use_freeptr_offset) ||
>             ((flags & SLAB_RED_ZONE) &&
>              (s->object_size < sizeof(void *) || slub_debug_orig_size(s)))) {
>                 /*
> @@ -7802,7 +7803,8 @@ static int calculate_sizes(struct kmem_cache_args *args, struct kmem_cache *s)
>                  */
>                 s->offset = size;
>                 size += sizeof(void *);
> -       } else if ((flags & SLAB_TYPESAFE_BY_RCU) && args->use_freeptr_offset) {
> +       } else if (((flags & SLAB_TYPESAFE_BY_RCU) || s->ctor) &&
> +                       args->use_freeptr_offset) {
>                 s->offset = args->freeptr_offset;
>         } else {
>                 /*
> --
> 2.43.0
>

On Tue, Oct 28, 2025 at 10:43:16AM -0700, Suren Baghdasaryan wrote:
> On Mon, Oct 27, 2025 at 5:29 AM Harry Yoo <harry.yoo@oracle.com> wrote:
> >
> > When a slab cache has a constructor, the free pointer is placed after the
> > object because certain fields must not be overwritten even after the
> > object is freed.
> >
> > However, some fields that the constructor does not care can safely be
> > overwritten. Allow specifying the free pointer offset within the object,
> > reducing the overall object size when some fields can be reused for the
> > free pointer.

Hi Suren, really appreciate you looking into it!

> Documentation explicitly says that ctor currently isn't supported with
> custom free pointers:
> https://elixir.bootlin.com/linux/v6.18-rc3/source/include/linux/slab.h*L318
> It obviously needs to be updated but I suspect there was a reason for
> this limitation. Have you investigated why it's not supported?

commit 879fb3c274c12 ("mm: add kmem_cache_create_rcu()") says:
> When a kmem cache is created with SLAB_TYPESAFE_BY_RCU the free pointer
> must be located outside of the object because we don't know what part of
> the memory can safely be overwritten as it may be needed to prevent
> object recycling.

The reason the slab allocator requires the free pointer to be
outside the object is the same: we don't know which fields
should not be overwritten, since users may assume a certain state
for specific fields in newly allocated objects.

If users don't initialize certain fields in the constructor, they
should not assume any particular state for those fields, and they may
therefore be overwritten.

> That has the consequence that SLAB_TYPESAFE_BY_RCU may end up adding a
> new cacheline. This is the case for e.g., struct file. After having it
> shrunk down by 40 bytes and having it fit in three cachelines we still
> have SLAB_TYPESAFE_BY_RCU adding a fourth cacheline because it needs to
> accommodate the free pointer.
> 
> Add a new kmem_cache_create_rcu() function that allows the caller to
> specify an offset where the free pointer is supposed to be placed.

I'm not sure why Christian added support only for SLAB_TYPESAFE_BY_RCU
and not for constructors, but I don't see anything that would prevent
extending it to support constructors as well.

> I remember looking into it when I was converting vm_area_struct cache to
> use SLAB_TYPESAFE_BY_RCU but I can't recall the details now...

-- 
Cheers,
Harry / Hyeonggon

On 10/29/25 08:10, Harry Yoo wrote:
> On Tue, Oct 28, 2025 at 10:43:16AM -0700, Suren Baghdasaryan wrote:
>> On Mon, Oct 27, 2025 at 5:29 AM Harry Yoo <harry.yoo@oracle.com> wrote:
>> >
>> > When a slab cache has a constructor, the free pointer is placed after the
>> > object because certain fields must not be overwritten even after the
>> > object is freed.
>> >
>> > However, some fields that the constructor does not care can safely be
>> > overwritten. Allow specifying the free pointer offset within the object,
>> > reducing the overall object size when some fields can be reused for the
>> > free pointer.
> 
> Hi Suren, really appreciate you looking into it!
> 
>> Documentation explicitly says that ctor currently isn't supported with
>> custom free pointers:
>> https://elixir.bootlin.com/linux/v6.18-rc3/source/include/linux/slab.h*L318
>> It obviously needs to be updated but I suspect there was a reason for
>> this limitation. Have you investigated why it's not supported?
> 
> commit 879fb3c274c12 ("mm: add kmem_cache_create_rcu()") says:
>> When a kmem cache is created with SLAB_TYPESAFE_BY_RCU the free pointer
>> must be located outside of the object because we don't know what part of
>> the memory can safely be overwritten as it may be needed to prevent
>> object recycling.
> 
> The reason the slab allocator requires the free pointer to be
> outside the object is the same: we don't know which fields
> should not be overwritten, since users may assume a certain state
> for specific fields in newly allocated objects.
> 
> If users don't initialize certain fields in the constructor, they
> should not assume any particular state for those fields, and they may
> therefore be overwritten.
> 
>> That has the consequence that SLAB_TYPESAFE_BY_RCU may end up adding a
>> new cacheline. This is the case for e.g., struct file. After having it
>> shrunk down by 40 bytes and having it fit in three cachelines we still
>> have SLAB_TYPESAFE_BY_RCU adding a fourth cacheline because it needs to
>> accommodate the free pointer.
>> 
>> Add a new kmem_cache_create_rcu() function that allows the caller to
>> specify an offset where the free pointer is supposed to be placed.
> 
> I'm not sure why Christian added support only for SLAB_TYPESAFE_BY_RCU
> and not for constructors, but I don't see anything that would prevent
> extending it to support constructors as well.

IIRC we considered it and only left it for later because there was no user
yet, so we wouldn't have a proof that it works and we're not missing
something. If you have a user now, it's legit to do it, and there are no
known theoretical obstacles to it. Obviously docs should be updated at the
same time then.
>> I remember looking into it when I was converting vm_area_struct cache to
>> use SLAB_TYPESAFE_BY_RCU but I can't recall the details now...
>