[v1] slub: Fixes freepointer encoding for single free

[PATCH] slub: Fixes freepointer encoding for single free

Posted by Nicolas Bouchinet 1 year, 9 months ago

From: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>

Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing
separately") splits single and bulk object freeing in two functions
slab_free() and slab_free_bulk() which leads slab_free() to call
slab_free_hook() directly instead of slab_free_freelist_hook().

If `init_on_free` is set, slab_free_hook() zeroes the object.
Afterward, if `slub_debug=F` and `CONFIG_SLAB_FREELIST_HARDENED` are
set, the do_slab_free() slowpath executes freelist consistency
checks and try to decode a zeroed freepointer which leads to a
"Freepointer corrupt" detection in check_object().

Object's freepointer thus needs to be properly set using
set_freepointer() after init_on_free.

To reproduce, set `slub_debug=FU init_on_free=1 log_level=7` on the
command line of a kernel build with `CONFIG_SLAB_FREELIST_HARDENED=y`.

dmesg sample log:
[   10.708715] =============================================================================
[   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
[   10.712695] -----------------------------------------------------------------------------
[   10.712695]
[   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
[   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
[   10.716698]
[   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
[   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed

Signed-off-by: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
---
 mm/slub.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/mm/slub.c b/mm/slub.c
index 3aa12b9b323d9..71dbff9ad8f17 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -4342,10 +4342,16 @@ static __fastpath_inline
 void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
 	       unsigned long addr)
 {
+	bool init = false;
+
 	memcg_slab_free_hook(s, slab, &object, 1);
+	init = slab_want_init_on_free(s);
 
-	if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
+	if (likely(slab_free_hook(s, object, init))) {
+		if (init)
+			set_freepointer(s, object, NULL);
 		do_slab_free(s, slab, object, object, 1, addr);
+	}
 }
 
 static __fastpath_inline
-- 
2.44.0

Re: [PATCH] slub: Fixes freepointer encoding for single free

Posted by Xiongwei Song 1 year, 9 months ago

On Wed, Apr 24, 2024 at 8:48 PM Nicolas Bouchinet
<nicolas.bouchinet@clip-os.org> wrote:
>
> From: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
>
> Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing
> separately") splits single and bulk object freeing in two functions
> slab_free() and slab_free_bulk() which leads slab_free() to call
> slab_free_hook() directly instead of slab_free_freelist_hook().
>
> If `init_on_free` is set, slab_free_hook() zeroes the object.
> Afterward, if `slub_debug=F` and `CONFIG_SLAB_FREELIST_HARDENED` are
> set, the do_slab_free() slowpath executes freelist consistency
> checks and try to decode a zeroed freepointer which leads to a
> "Freepointer corrupt" detection in check_object().
>
> Object's freepointer thus needs to be properly set using
> set_freepointer() after init_on_free.
>
> To reproduce, set `slub_debug=FU init_on_free=1 log_level=7` on the
> command line of a kernel build with `CONFIG_SLAB_FREELIST_HARDENED=y`.
>
> dmesg sample log:
> [   10.708715] =============================================================================
> [   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
> [   10.712695] -----------------------------------------------------------------------------
> [   10.712695]
> [   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
> [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c

If init_on_free is set,  slab_free_hook() zeros the object first, then
do_slab_free() calls
set_freepointer() to set the fp value, so there are 8 bytes non-zero
at the moment?
Hence, the issue is not related to init_on_free?

The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from
CONFIG_SLAB_FREELIST_HARDENED enabled?

Thanks,
Xiongwei

> [   10.716698]
> [   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> [   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> [   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> [   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> [   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
> [   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
>
> Signed-off-by: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
> ---
>  mm/slub.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/mm/slub.c b/mm/slub.c
> index 3aa12b9b323d9..71dbff9ad8f17 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -4342,10 +4342,16 @@ static __fastpath_inline
>  void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
>                unsigned long addr)
>  {
> +       bool init = false;
> +
>         memcg_slab_free_hook(s, slab, &object, 1);
> +       init = slab_want_init_on_free(s);
>
> -       if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
> +       if (likely(slab_free_hook(s, object, init))) {
> +               if (init)
> +                       set_freepointer(s, object, NULL);
>                 do_slab_free(s, slab, object, object, 1, addr);
> +       }
>  }
>
>  static __fastpath_inline
> --
> 2.44.0
>
>

Re: [PATCH] slub: Fixes freepointer encoding for single free

Posted by Nicolas Bouchinet 1 year, 9 months ago

On 4/26/24 11:20, Xiongwei Song wrote:
> On Wed, Apr 24, 2024 at 8:48 PM Nicolas Bouchinet
> <nicolas.bouchinet@clip-os.org> wrote:
>> From: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
>>
>> Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing
>> separately") splits single and bulk object freeing in two functions
>> slab_free() and slab_free_bulk() which leads slab_free() to call
>> slab_free_hook() directly instead of slab_free_freelist_hook().
>>
>> If `init_on_free` is set, slab_free_hook() zeroes the object.
>> Afterward, if `slub_debug=F` and `CONFIG_SLAB_FREELIST_HARDENED` are
>> set, the do_slab_free() slowpath executes freelist consistency
>> checks and try to decode a zeroed freepointer which leads to a
>> "Freepointer corrupt" detection in check_object().
>>
>> Object's freepointer thus needs to be properly set using
>> set_freepointer() after init_on_free.
>>
>> To reproduce, set `slub_debug=FU init_on_free=1 log_level=7` on the
>> command line of a kernel build with `CONFIG_SLAB_FREELIST_HARDENED=y`.
>>
>> dmesg sample log:
>> [   10.708715] =============================================================================
>> [   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
>> [   10.712695] -----------------------------------------------------------------------------
>> [   10.712695]
>> [   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
>> [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
> If init_on_free is set,  slab_free_hook() zeros the object first, then
> do_slab_free() calls
> set_freepointer() to set the fp value, so there are 8 bytes non-zero
> at the moment?
> Hence, the issue is not related to init_on_free?
>
> The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from
> CONFIG_SLAB_FREELIST_HARDENED enabled?

My understanding of the bug is that slab_free_hook() indeed zeroes the 
object and its metadata first, then calls do_slab_free() and directly 
calls __slab_free(), head an tail parameters being set to the object.

If `slub_debug=F` (SLAB_CONSISTENCY_CHECKS) is set, the following call 
path can be executed :

free_to_partial_list() ->

free_debug_processing() ->

free_consistency_checks() ->

check_object() ->

check_valid_pointer(get_freepointer())

When check_valid_pointer() is called, its object parameter is first 
decoded by get_freepointer(), if CONFIG_SLAB_FREELIST_HARDENED is 
enabled, zeroed data is then decoded by freelist_ptr_decode() using the 
(ptr.v ^ s->random ^ swab(ptr_addr)) operation.


Does that makes sense to you or do you think I'm missing something ?


Best regards,

Nicolas

> Xiongwei
>
>> [   10.716698]
>> [   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>> [   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>> [   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>> [   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>> [   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
>> [   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
>>
>> Signed-off-by: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
>> ---
>>   mm/slub.c | 8 +++++++-
>>   1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/slub.c b/mm/slub.c
>> index 3aa12b9b323d9..71dbff9ad8f17 100644
>> --- a/mm/slub.c
>> +++ b/mm/slub.c
>> @@ -4342,10 +4342,16 @@ static __fastpath_inline
>>   void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
>>                 unsigned long addr)
>>   {
>> +       bool init = false;
>> +
>>          memcg_slab_free_hook(s, slab, &object, 1);
>> +       init = slab_want_init_on_free(s);
>>
>> -       if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
>> +       if (likely(slab_free_hook(s, object, init))) {
>> +               if (init)
>> +                       set_freepointer(s, object, NULL);
>>                  do_slab_free(s, slab, object, object, 1, addr);
>> +       }
>>   }
>>
>>   static __fastpath_inline
>> --
>> 2.44.0
>>
>>

Re: [PATCH] slub: Fixes freepointer encoding for single free

Posted by Xiongwei Song 1 year, 9 months ago

On Fri, Apr 26, 2024 at 8:18 PM Nicolas Bouchinet
<nicolas.bouchinet@clip-os.org> wrote:
>
> On 4/26/24 11:20, Xiongwei Song wrote:
> > On Wed, Apr 24, 2024 at 8:48 PM Nicolas Bouchinet
> > <nicolas.bouchinet@clip-os.org> wrote:
> >> From: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
> >>
> >> Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing
> >> separately") splits single and bulk object freeing in two functions
> >> slab_free() and slab_free_bulk() which leads slab_free() to call
> >> slab_free_hook() directly instead of slab_free_freelist_hook().
> >>
> >> If `init_on_free` is set, slab_free_hook() zeroes the object.
> >> Afterward, if `slub_debug=F` and `CONFIG_SLAB_FREELIST_HARDENED` are
> >> set, the do_slab_free() slowpath executes freelist consistency
> >> checks and try to decode a zeroed freepointer which leads to a
> >> "Freepointer corrupt" detection in check_object().
> >>
> >> Object's freepointer thus needs to be properly set using
> >> set_freepointer() after init_on_free.
> >>
> >> To reproduce, set `slub_debug=FU init_on_free=1 log_level=7` on the
> >> command line of a kernel build with `CONFIG_SLAB_FREELIST_HARDENED=y`.
> >>
> >> dmesg sample log:
> >> [   10.708715] =============================================================================
> >> [   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
> >> [   10.712695] -----------------------------------------------------------------------------
> >> [   10.712695]
> >> [   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
> >> [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
> > If init_on_free is set,  slab_free_hook() zeros the object first, then
> > do_slab_free() calls
> > set_freepointer() to set the fp value, so there are 8 bytes non-zero
> > at the moment?
> > Hence, the issue is not related to init_on_free?
> >
> > The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from
> > CONFIG_SLAB_FREELIST_HARDENED enabled?
>
> My understanding of the bug is that slab_free_hook() indeed zeroes the
> object and its metadata first, then calls do_slab_free() and directly
> calls __slab_free(), head an tail parameters being set to the object.
>
> If `slub_debug=F` (SLAB_CONSISTENCY_CHECKS) is set, the following call
> path can be executed :
>
> free_to_partial_list() ->
>
> free_debug_processing() ->
>
> free_consistency_checks() ->
>
> check_object() ->
>
> check_valid_pointer(get_freepointer())

I understand the call path. I meant here the freepointer is not ZERO
but an illegal
value( fp=0x7ee4f480ce0ecd7c), then check_valid_pointer return 1 with
it's last line
and then check_object() printed out the error message. I'm not sure if I
misunderstood you.

Thank,
Xiongwei

Re: [PATCH] slub: Fixes freepointer encoding for single free

Posted by Nicolas Bouchinet 1 year, 9 months ago

On 4/26/24 15:14, Xiongwei Song wrote:
> On Fri, Apr 26, 2024 at 8:18 PM Nicolas Bouchinet
> <nicolas.bouchinet@clip-os.org> wrote:
>> On 4/26/24 11:20, Xiongwei Song wrote:
>>> On Wed, Apr 24, 2024 at 8:48 PM Nicolas Bouchinet
>>> <nicolas.bouchinet@clip-os.org> wrote:
>>>> From: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
>>>>
>>>> Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing
>>>> separately") splits single and bulk object freeing in two functions
>>>> slab_free() and slab_free_bulk() which leads slab_free() to call
>>>> slab_free_hook() directly instead of slab_free_freelist_hook().
>>>>
>>>> If `init_on_free` is set, slab_free_hook() zeroes the object.
>>>> Afterward, if `slub_debug=F` and `CONFIG_SLAB_FREELIST_HARDENED` are
>>>> set, the do_slab_free() slowpath executes freelist consistency
>>>> checks and try to decode a zeroed freepointer which leads to a
>>>> "Freepointer corrupt" detection in check_object().
>>>>
>>>> Object's freepointer thus needs to be properly set using
>>>> set_freepointer() after init_on_free.
>>>>
>>>> To reproduce, set `slub_debug=FU init_on_free=1 log_level=7` on the
>>>> command line of a kernel build with `CONFIG_SLAB_FREELIST_HARDENED=y`.
>>>>
>>>> dmesg sample log:
>>>> [   10.708715] =============================================================================
>>>> [   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
>>>> [   10.712695] -----------------------------------------------------------------------------
>>>> [   10.712695]
>>>> [   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
>>>> [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
>>> If init_on_free is set,  slab_free_hook() zeros the object first, then
>>> do_slab_free() calls
>>> set_freepointer() to set the fp value, so there are 8 bytes non-zero
>>> at the moment?
>>> Hence, the issue is not related to init_on_free?
>>>
>>> The fp=0x7ee4f480ce0ecd7c here is beyond kernel memory space, is the issue from
>>> CONFIG_SLAB_FREELIST_HARDENED enabled?
>> My understanding of the bug is that slab_free_hook() indeed zeroes the
>> object and its metadata first, then calls do_slab_free() and directly
>> calls __slab_free(), head an tail parameters being set to the object.
>>
>> If `slub_debug=F` (SLAB_CONSISTENCY_CHECKS) is set, the following call
>> path can be executed :
>>
>> free_to_partial_list() ->
>>
>> free_debug_processing() ->
>>
>> free_consistency_checks() ->
>>
>> check_object() ->
>>
>> check_valid_pointer(get_freepointer())
> I understand the call path. I meant here the freepointer is not ZERO
> but an illegal
> value( fp=0x7ee4f480ce0ecd7c),


Yes this is the reason of this patch. The freepointer is an illegal 
value because the memory range where it sits has been overridden by 
zeroes, set_freepointer() is never called and thus the freepointer is 
never properly set.


  The illegal value is obtained after zeroes has been decoded by 
get_freepointer()->freelist_ptr_decode().


>   then check_valid_pointer return 1 with
> it's last line
> and then check_object() printed out the error message. I'm not sure if I
> misunderstood you.


check_valid_pointer() returns 0 since object < base, the object being 
the decoded fp (0x7ee4f480ce0ecd7c < 0xffff.* base addr), hence 
check_object() returns 0, not 1.

This is why the "Object at 0x%p not freed" slab_fix() is called in 
free_debug_processing().


>
> Thank,
> Xiongwei
>

Re: [PATCH] slub: Fixes freepointer encoding for single free

Posted by Chengming Zhou 1 year, 9 months ago

On 2024/4/24 20:47, Nicolas Bouchinet wrote:
> From: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
> 
> Commit 284f17ac13fe ("mm/slub: handle bulk and single object freeing
> separately") splits single and bulk object freeing in two functions
> slab_free() and slab_free_bulk() which leads slab_free() to call
> slab_free_hook() directly instead of slab_free_freelist_hook().

Right.

> 
> If `init_on_free` is set, slab_free_hook() zeroes the object.
> Afterward, if `slub_debug=F` and `CONFIG_SLAB_FREELIST_HARDENED` are
> set, the do_slab_free() slowpath executes freelist consistency
> checks and try to decode a zeroed freepointer which leads to a
> "Freepointer corrupt" detection in check_object().

IIUC, the "freepointer" can be checked on the free path only when
it's outside the object memory. Here slab_free_hook() zeroed the
freepointer and caused the problem.

But why we should zero the memory outside the object_size? It seems
more reasonable to only zero the object_size when init_on_free is set?

Thanks.

> 
> Object's freepointer thus needs to be properly set using
> set_freepointer() after init_on_free.
> 
> To reproduce, set `slub_debug=FU init_on_free=1 log_level=7` on the
> command line of a kernel build with `CONFIG_SLAB_FREELIST_HARDENED=y`.
> 
> dmesg sample log:
> [   10.708715] =============================================================================
> [   10.710323] BUG kmalloc-rnd-05-32 (Tainted: G    B           T ): Freepointer corrupt
> [   10.712695] -----------------------------------------------------------------------------
> [   10.712695]
> [   10.712695] Slab 0xffffd8bdc400d580 objects=32 used=4 fp=0xffff9d9a80356f80 flags=0x200000000000a00(workingset|slab|node=0|zone=2)
> [   10.716698] Object 0xffff9d9a80356600 @offset=1536 fp=0x7ee4f480ce0ecd7c
> [   10.716698]
> [   10.716698] Bytes b4 ffff9d9a803565f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> [   10.720703] Object   ffff9d9a80356600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> [   10.720703] Object   ffff9d9a80356610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> [   10.724696] Padding  ffff9d9a8035666c: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> [   10.724696] Padding  ffff9d9a8035667c: 00 00 00 00                                      ....
> [   10.724696] FIX kmalloc-rnd-05-32: Object at 0xffff9d9a80356600 not freed
> 
> Signed-off-by: Nicolas Bouchinet <nicolas.bouchinet@ssi.gouv.fr>
> ---
>  mm/slub.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 3aa12b9b323d9..71dbff9ad8f17 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -4342,10 +4342,16 @@ static __fastpath_inline
>  void slab_free(struct kmem_cache *s, struct slab *slab, void *object,
>  	       unsigned long addr)
>  {
> +	bool init = false;
> +
>  	memcg_slab_free_hook(s, slab, &object, 1);
> +	init = slab_want_init_on_free(s);
>  
> -	if (likely(slab_free_hook(s, object, slab_want_init_on_free(s))))
> +	if (likely(slab_free_hook(s, object, init))) {
> +		if (init)
> +			set_freepointer(s, object, NULL);
>  		do_slab_free(s, slab, object, object, 1, addr);
> +	}
>  }
>  
>  static __fastpath_inline