liveupdate: Rework KHO for in-kernel users & Fix memory corruption

[PATCH v6 10/10] liveupdate: kho: allocate metadata directly from the buddy allocator

Posted by Pasha Tatashin 3 months, 3 weeks ago

KHO allocates metadata for its preserved memory map using the slab
allocator via kzalloc(). This metadata is temporary and is used by the
next kernel during early boot to find preserved memory.

A problem arises when KFENCE is enabled. kzalloc() calls can be
randomly intercepted by kfence_alloc(), which services the allocation
from a dedicated KFENCE memory pool. This pool is allocated early in
boot via memblock.

When booting via KHO, the memblock allocator is restricted to a "scratch
area", forcing the KFENCE pool to be allocated within it. This creates a
conflict, as the scratch area is expected to be ephemeral and
overwriteable by a subsequent kexec. If KHO metadata is placed in this
KFENCE pool, it leads to memory corruption when the next kernel is
loaded.

To fix this, modify KHO to allocate its metadata directly from the buddy
allocator instead of slab.

Fixes: fc33e4b44b27 ("kexec: enable KHO support for memory preservation")
Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Reviewed-by: Pratyush Yadav <pratyush@kernel.org>
---
 kernel/liveupdate/kexec_handover.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
index 7c8e89a6b953..92662739a3a2 100644
--- a/kernel/liveupdate/kexec_handover.c
+++ b/kernel/liveupdate/kexec_handover.c
@@ -132,6 +132,8 @@ static struct kho_out kho_out = {
 	.finalized = false,
 };
 
+DEFINE_FREE(kho_free_page, void *, free_page((unsigned long)_T))
+
 static void *xa_load_or_alloc(struct xarray *xa, unsigned long index)
 {
 	void *res = xa_load(xa, index);
@@ -139,7 +141,7 @@ static void *xa_load_or_alloc(struct xarray *xa, unsigned long index)
 	if (res)
 		return res;
 
-	void *elm __free(kfree) = kzalloc(PAGE_SIZE, GFP_KERNEL);
+	void *elm __free(kho_free_page) = (void *)get_zeroed_page(GFP_KERNEL);
 
 	if (!elm)
 		return ERR_PTR(-ENOMEM);
@@ -352,9 +354,9 @@ static_assert(sizeof(struct khoser_mem_chunk) == PAGE_SIZE);
 static struct khoser_mem_chunk *new_chunk(struct khoser_mem_chunk *cur_chunk,
 					  unsigned long order)
 {
-	struct khoser_mem_chunk *chunk __free(kfree) = NULL;
+	struct khoser_mem_chunk *chunk __free(kho_free_page) = NULL;
 
-	chunk = kzalloc(PAGE_SIZE, GFP_KERNEL);
+	chunk = (void *)get_zeroed_page(GFP_KERNEL);
 	if (!chunk)
 		return ERR_PTR(-ENOMEM);
 
-- 
2.51.0.915.g61a8936c21-goog

Re: [PATCH v6 10/10] liveupdate: kho: allocate metadata directly from the buddy allocator

Posted by Mike Rapoport 3 months, 2 weeks ago

On Sat, Oct 18, 2025 at 01:17:56PM -0400, Pasha Tatashin wrote:
> KHO allocates metadata for its preserved memory map using the slab
> allocator via kzalloc(). This metadata is temporary and is used by the
> next kernel during early boot to find preserved memory.
> 
> A problem arises when KFENCE is enabled. kzalloc() calls can be
> randomly intercepted by kfence_alloc(), which services the allocation
> from a dedicated KFENCE memory pool. This pool is allocated early in
> boot via memblock.
> 
> When booting via KHO, the memblock allocator is restricted to a "scratch
> area", forcing the KFENCE pool to be allocated within it. This creates a
> conflict, as the scratch area is expected to be ephemeral and
> overwriteable by a subsequent kexec. If KHO metadata is placed in this
> KFENCE pool, it leads to memory corruption when the next kernel is
> loaded.
> 
> To fix this, modify KHO to allocate its metadata directly from the buddy
> allocator instead of slab.
> 
> Fixes: fc33e4b44b27 ("kexec: enable KHO support for memory preservation")
> Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com>
> Reviewed-by: Pratyush Yadav <pratyush@kernel.org>
> ---
>  kernel/liveupdate/kexec_handover.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> index 7c8e89a6b953..92662739a3a2 100644
> --- a/kernel/liveupdate/kexec_handover.c
> +++ b/kernel/liveupdate/kexec_handover.c
> @@ -132,6 +132,8 @@ static struct kho_out kho_out = {
>  	.finalized = false,
>  };
>  
> +DEFINE_FREE(kho_free_page, void *, free_page((unsigned long)_T))
> +

Just drop kho_ prefix and stick it into include/linux/gfp.h:

DEFINE_FREE(free_page, void *, if (_T) free_page((unsigned long)_T))

>  static void *xa_load_or_alloc(struct xarray *xa, unsigned long index)
>  {
>  	void *res = xa_load(xa, index);
> @@ -139,7 +141,7 @@ static void *xa_load_or_alloc(struct xarray *xa, unsigned long index)
>  	if (res)
>  		return res;
>  
> -	void *elm __free(kfree) = kzalloc(PAGE_SIZE, GFP_KERNEL);
> +	void *elm __free(kho_free_page) = (void *)get_zeroed_page(GFP_KERNEL);
>  
>  	if (!elm)
>  		return ERR_PTR(-ENOMEM);
> @@ -352,9 +354,9 @@ static_assert(sizeof(struct khoser_mem_chunk) == PAGE_SIZE);
>  static struct khoser_mem_chunk *new_chunk(struct khoser_mem_chunk *cur_chunk,
>  					  unsigned long order)
>  {
> -	struct khoser_mem_chunk *chunk __free(kfree) = NULL;
> +	struct khoser_mem_chunk *chunk __free(kho_free_page) = NULL;
>  
> -	chunk = kzalloc(PAGE_SIZE, GFP_KERNEL);
> +	chunk = (void *)get_zeroed_page(GFP_KERNEL);
>  	if (!chunk)
>  		return ERR_PTR(-ENOMEM);
>  
> -- 
> 2.51.0.915.g61a8936c21-goog
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH v6 10/10] liveupdate: kho: allocate metadata directly from the buddy allocator

Posted by Pasha Tatashin 3 months, 2 weeks ago

On Mon, Oct 20, 2025 at 4:05 AM Mike Rapoport <rppt@kernel.org> wrote:
>
> On Sat, Oct 18, 2025 at 01:17:56PM -0400, Pasha Tatashin wrote:
> > KHO allocates metadata for its preserved memory map using the slab
> > allocator via kzalloc(). This metadata is temporary and is used by the
> > next kernel during early boot to find preserved memory.
> >
> > A problem arises when KFENCE is enabled. kzalloc() calls can be
> > randomly intercepted by kfence_alloc(), which services the allocation
> > from a dedicated KFENCE memory pool. This pool is allocated early in
> > boot via memblock.
> >
> > When booting via KHO, the memblock allocator is restricted to a "scratch
> > area", forcing the KFENCE pool to be allocated within it. This creates a
> > conflict, as the scratch area is expected to be ephemeral and
> > overwriteable by a subsequent kexec. If KHO metadata is placed in this
> > KFENCE pool, it leads to memory corruption when the next kernel is
> > loaded.
> >
> > To fix this, modify KHO to allocate its metadata directly from the buddy
> > allocator instead of slab.
> >
> > Fixes: fc33e4b44b27 ("kexec: enable KHO support for memory preservation")
> > Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com>
> > Reviewed-by: Pratyush Yadav <pratyush@kernel.org>
> > ---
> >  kernel/liveupdate/kexec_handover.c | 8 +++++---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c
> > index 7c8e89a6b953..92662739a3a2 100644
> > --- a/kernel/liveupdate/kexec_handover.c
> > +++ b/kernel/liveupdate/kexec_handover.c
> > @@ -132,6 +132,8 @@ static struct kho_out kho_out = {
> >       .finalized = false,
> >  };
> >
> > +DEFINE_FREE(kho_free_page, void *, free_page((unsigned long)_T))
> > +
>
> Just drop kho_ prefix and stick it into include/linux/gfp.h

done

>
> >  static void *xa_load_or_alloc(struct xarray *xa, unsigned long index)
> >  {
> >       void *res = xa_load(xa, index);
> > @@ -139,7 +141,7 @@ static void *xa_load_or_alloc(struct xarray *xa, unsigned long index)
> >       if (res)
> >               return res;
> >
> > -     void *elm __free(kfree) = kzalloc(PAGE_SIZE, GFP_KERNEL);
> > +     void *elm __free(kho_free_page) = (void *)get_zeroed_page(GFP_KERNEL);
> >
> >       if (!elm)
> >               return ERR_PTR(-ENOMEM);
> > @@ -352,9 +354,9 @@ static_assert(sizeof(struct khoser_mem_chunk) == PAGE_SIZE);
> >  static struct khoser_mem_chunk *new_chunk(struct khoser_mem_chunk *cur_chunk,
> >                                         unsigned long order)
> >  {
> > -     struct khoser_mem_chunk *chunk __free(kfree) = NULL;
> > +     struct khoser_mem_chunk *chunk __free(kho_free_page) = NULL;
> >
> > -     chunk = kzalloc(PAGE_SIZE, GFP_KERNEL);
> > +     chunk = (void *)get_zeroed_page(GFP_KERNEL);
> >       if (!chunk)
> >               return ERR_PTR(-ENOMEM);
> >
> > --
> > 2.51.0.915.g61a8936c21-goog
> >
>
> --
> Sincerely yours,
> Mike.

[PATCH v6 01/10] kho: allow to drive kho from within kernel
[PATCH v6 02/10] kho: make debugfs interface optional
[PATCH v6 03/10] kho: drop notifiers
[PATCH v6 04/10] kho: add interfaces to unpreserve folios and page ranes
[PATCH v6 05/10] kho: don't unpreserve memory during abort
[PATCH v6 06/10] liveupdate: kho: move to kernel/liveupdate
[PATCH v6 07/10] kho: move kho debugfs directory to liveupdate
[PATCH v6 08/10] liveupdate: kho: warn and fail on metadata or preserved memory in scratch area
[PATCH v6 09/10] liveupdate: kho: Increase metadata bitmap size to PAGE_SIZE
[PATCH v6 10/10] liveupdate: kho: allocate metadata directly from the buddy allocator