[PATCH v9 1/8] kexec_file: allow to place kexec_buf randomly

Coiby Xu posted 8 patches 6 months, 2 weeks ago
[PATCH v9 1/8] kexec_file: allow to place kexec_buf randomly
Posted by Coiby Xu 6 months, 2 weeks ago
Currently, kexec_buf is placed in order which means for the same
machine, the info in the kexec_buf is always located at the same
position each time the machine is booted. This may cause a risk for
sensitive information like LUKS volume key. Now struct kexec_buf has a
new field random which indicates it's supposed to be placed in a random
position.

Note this feature is enabled only when CONFIG_CRASH_DUMP is enabled. So
it only takes effect for kdump and won't impact kexec reboot.

Suggested-by: Jan Pazdziora <jpazdziora@redhat.com>
Signed-off-by: Coiby Xu <coxu@redhat.com>
Acked-by: Baoquan He <bhe@redhat.com>
---
 include/linux/kexec.h | 30 ++++++++++++++++++++++++++++++
 kernel/kexec_file.c   |  3 +++
 2 files changed, 33 insertions(+)

diff --git a/include/linux/kexec.h b/include/linux/kexec.h
index c8971861521a..1871eaa95432 100644
--- a/include/linux/kexec.h
+++ b/include/linux/kexec.h
@@ -25,6 +25,10 @@
 
 extern note_buf_t __percpu *crash_notes;
 
+#ifdef CONFIG_CRASH_DUMP
+#include <linux/prandom.h>
+#endif
+
 #ifdef CONFIG_KEXEC_CORE
 #include <linux/list.h>
 #include <linux/compat.h>
@@ -169,6 +173,7 @@ int kexec_image_post_load_cleanup_default(struct kimage *image);
  * @buf_min:	The buffer can't be placed below this address.
  * @buf_max:	The buffer can't be placed above this address.
  * @top_down:	Allocate from top of memory.
+ * @random:	Place the buffer at a random position.
  */
 struct kexec_buf {
 	struct kimage *image;
@@ -180,8 +185,33 @@ struct kexec_buf {
 	unsigned long buf_min;
 	unsigned long buf_max;
 	bool top_down;
+#ifdef CONFIG_CRASH_DUMP
+	bool random;
+#endif
 };
 
+
+#ifdef CONFIG_CRASH_DUMP
+static inline void kexec_random_range_start(unsigned long start,
+					    unsigned long end,
+					    struct kexec_buf *kbuf,
+					    unsigned long *temp_start)
+{
+	unsigned short i;
+
+	if (kbuf->random) {
+		get_random_bytes(&i, sizeof(unsigned short));
+		*temp_start = start + (end - start) / USHRT_MAX * i;
+	}
+}
+#else
+static inline void kexec_random_range_start(unsigned long start,
+					    unsigned long end,
+					    struct kexec_buf *kbuf,
+					    unsigned long *temp_start)
+{}
+#endif
+
 int kexec_load_purgatory(struct kimage *image, struct kexec_buf *kbuf);
 int kexec_purgatory_get_set_symbol(struct kimage *image, const char *name,
 				   void *buf, unsigned int size,
diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c
index fba686487e3b..1180c0aa73f6 100644
--- a/kernel/kexec_file.c
+++ b/kernel/kexec_file.c
@@ -445,6 +445,7 @@ static int locate_mem_hole_top_down(unsigned long start, unsigned long end,
 
 	temp_end = min(end, kbuf->buf_max);
 	temp_start = temp_end - kbuf->memsz + 1;
+	kexec_random_range_start(temp_start, temp_end, kbuf, &temp_start);
 
 	do {
 		/* align down start */
@@ -489,6 +490,8 @@ static int locate_mem_hole_bottom_up(unsigned long start, unsigned long end,
 
 	temp_start = max(start, kbuf->buf_min);
 
+	kexec_random_range_start(temp_start, end, kbuf, &temp_start);
+
 	do {
 		temp_start = ALIGN(temp_start, kbuf->buf_align);
 		temp_end = temp_start + kbuf->memsz - 1;
-- 
2.49.0
Re: [PATCH v9 1/8] kexec_file: allow to place kexec_buf randomly
Posted by Breno Leitao 2 months, 3 weeks ago
Hello Coiby,

On Fri, May 02, 2025 at 09:12:35AM +0800, Coiby Xu wrote:
> +static inline void kexec_random_range_start(unsigned long start,
> +					    unsigned long end,
> +					    struct kexec_buf *kbuf,
> +					    unsigned long *temp_start)
> +{
> +	unsigned short i;
> +
> +	if (kbuf->random) {
> +		get_random_bytes(&i, sizeof(unsigned short));
> +		*temp_start = start + (end - start) / USHRT_MAX * i;
> +	}
> +}

On arm64, I am getting the following UBSAN warning when accessing
kbuf->random:

[   32.362428] ------------[ cut here ]------------
[   32.362488] UBSAN: invalid-load in ./include/linux/kexec.h:210:10
[   32.362649] load of value 252 is not a valid value for type '_Bool'

and line 210 is your `if (kbuf->random)`.

Basically kbuf was not initialized in arm hosts, and probably has
garbage.

I am wondering if we should have something like , while the support for arm64 is
not done:

commit 2608bd8c26b62a9a7cc50106e93d3a1ffb1e1188
Author: Breno Leitao <leitao@debian.org>
Date:   Thu Aug 21 04:11:21 2025 -0700

    Initialize the random field of kbuf to zero in the ARM64 kexec image loader

    Ads an explicit initialization for the random member of the kbuf
    structure within the image_load function in
    arch/arm64/kernel/kexec_image.c. Setting kbuf.random to zero ensures
    a deterministic and clean starting state for the buffer used during
    kernel image loading, avoiding this UBSAN issue later, when kbuf.random
    is read.

            [   32.362488] UBSAN: invalid-load in ./include/linux/kexec.h:210:10
            [   32.362649] load of value 252 is not a valid value for type '_Bool'

    Signed-off-by: Breno Leitao <leitao@debian.org>

diff --git a/arch/arm64/kernel/kexec_image.c b/arch/arm64/kernel/kexec_image.c
index 532d72ea42ee8..287b25e674d76 100644
--- a/arch/arm64/kernel/kexec_image.c
+++ b/arch/arm64/kernel/kexec_image.c
@@ -76,6 +76,7 @@ static void *image_load(struct kimage *image,
 	kbuf.buf_min = 0;
 	kbuf.buf_max = ULONG_MAX;
 	kbuf.top_down = false;
+	kbuf.random = 0;

 	kbuf.buffer = kernel;
 	kbuf.bufsz = kernel_len;
Re: [PATCH v9 1/8] kexec_file: allow to place kexec_buf randomly
Posted by Coiby Xu 2 months, 3 weeks ago
On Thu, Aug 21, 2025 at 04:15:53AM -0700, Breno Leitao wrote:
>Hello Coiby,

Hi Breno,

>
>On Fri, May 02, 2025 at 09:12:35AM +0800, Coiby Xu wrote:
>> +static inline void kexec_random_range_start(unsigned long start,
>> +					    unsigned long end,
>> +					    struct kexec_buf *kbuf,
>> +					    unsigned long *temp_start)
>> +{
>> +	unsigned short i;
>> +
>> +	if (kbuf->random) {
>> +		get_random_bytes(&i, sizeof(unsigned short));
>> +		*temp_start = start + (end - start) / USHRT_MAX * i;
>> +	}
>> +}
>
>On arm64, I am getting the following UBSAN warning when accessing
>kbuf->random:
>
>[   32.362428] ------------[ cut here ]------------
>[   32.362488] UBSAN: invalid-load in ./include/linux/kexec.h:210:10
>[   32.362649] load of value 252 is not a valid value for type '_Bool'
>
>and line 210 is your `if (kbuf->random)`.
>
>Basically kbuf was not initialized in arm hosts, and probably has
>garbage.

Thank for explaining the problem to me!

>
>I am wondering if we should have something like , while the support for arm64 is
>not done:
>
>commit 2608bd8c26b62a9a7cc50106e93d3a1ffb1e1188
>Author: Breno Leitao <leitao@debian.org>
>Date:   Thu Aug 21 04:11:21 2025 -0700
>
>    Initialize the random field of kbuf to zero in the ARM64 kexec image loader
>
>    Ads an explicit initialization for the random member of the kbuf
>    structure within the image_load function in
>    arch/arm64/kernel/kexec_image.c. Setting kbuf.random to zero ensures
>    a deterministic and clean starting state for the buffer used during
>    kernel image loading, avoiding this UBSAN issue later, when kbuf.random
>    is read.
>
>            [   32.362488] UBSAN: invalid-load in ./include/linux/kexec.h:210:10
>            [   32.362649] load of value 252 is not a valid value for type '_Bool'
>
>    Signed-off-by: Breno Leitao <leitao@debian.org>
>
>diff --git a/arch/arm64/kernel/kexec_image.c b/arch/arm64/kernel/kexec_image.c
>index 532d72ea42ee8..287b25e674d76 100644
>--- a/arch/arm64/kernel/kexec_image.c
>+++ b/arch/arm64/kernel/kexec_image.c
>@@ -76,6 +76,7 @@ static void *image_load(struct kimage *image,
> 	kbuf.buf_min = 0;
> 	kbuf.buf_max = ULONG_MAX;
> 	kbuf.top_down = false;
>+	kbuf.random = 0;
>
> 	kbuf.buffer = kernel;
> 	kbuf.bufsz = kernel_len;
>

And also thanks for posing a fix! The patch LGTM. Can you add a Fixes
tag 'Fixes: bf454ec31add ("kexec_file: allow to place kexec_buf
randomly")' and then send it to kexec@lists.infradead.org? Thanks!

-- 
Best regards,
Coiby
Re: [PATCH v9 1/8] kexec_file: allow to place kexec_buf randomly
Posted by Andrew Morton 2 months, 2 weeks ago
On Mon, 25 Aug 2025 09:18:53 +0800 Coiby Xu <coxu@redhat.com> wrote:

> >diff --git a/arch/arm64/kernel/kexec_image.c b/arch/arm64/kernel/kexec_image.c
> >index 532d72ea42ee8..287b25e674d76 100644
> >--- a/arch/arm64/kernel/kexec_image.c
> >+++ b/arch/arm64/kernel/kexec_image.c
> >@@ -76,6 +76,7 @@ static void *image_load(struct kimage *image,
> > 	kbuf.buf_min = 0;
> > 	kbuf.buf_max = ULONG_MAX;
> > 	kbuf.top_down = false;
> >+	kbuf.random = 0;
> >
> > 	kbuf.buffer = kernel;
> > 	kbuf.bufsz = kernel_len;
> >
> 
> And also thanks for posing a fix! The patch LGTM. Can you add a Fixes
> tag 'Fixes: bf454ec31add ("kexec_file: allow to place kexec_buf
> randomly")' and then send it to kexec@lists.infradead.org? Thanks!

I turned all this into a regular patch and queued it (see below),
thanks.  No additional actions are needed.

I'm really not liking that code.  I laboriously verified that all
fields of kexec_buf are now initialized, except for `cma'.  Is that a
bug?

This function has a call frequency of about 3x per week.  Can we please
just memset the whole thing so people don't have to worry about this
any more?


From: Breno Leitao <leitao@debian.org>
Subject: kexec/arm64: initialize the random field of kbuf to zero in the image loader
Date: Thu Aug 21 04:11:21 2025 -0700

Add an explicit initialization for the random member of the kbuf structure
within the image_load function in arch/arm64/kernel/kexec_image.c. 
Setting kbuf.random to zero ensures a deterministic and clean starting
state for the buffer used during kernel image loading, avoiding this UBSAN
issue later, when kbuf.random is read.

  [   32.362488] UBSAN: invalid-load in ./include/linux/kexec.h:210:10
  [   32.362649] load of value 252 is not a valid value for type '_Bool'

Link: https://lkml.kernel.org/r/oninomspajhxp4omtdapxnckxydbk2nzmrix7rggmpukpnzadw@c67o7njgdgm3
Fixes: bf454ec31add ("kexec_file: allow to place kexec_buf randomly
Signed-off-by: Breno Leitao <leitao@debian.org>
Cc: Baoquan He <bhe@redhat.com>
Cc: Coiby Xu <coxu@redhat.com>
Cc: "Daniel P. Berrange" <berrange@redhat.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Dave Young <dyoung@redhat.com>
Cc: Kairui Song <ryncsn@gmail.com>
Cc: Liu Pingfan <kernelfans@gmail.com>
Cc: Milan Broz <gmazyland@gmail.com>
Cc: Ondrej Kozina <okozina@redhat.com>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 arch/arm64/kernel/kexec_image.c |    1 +
 1 file changed, 1 insertion(+)

--- a/arch/arm64/kernel/kexec_image.c~kexec-arm64-initialize-the-random-field-of-kbuf-to-zero-in-the-image-loader
+++ a/arch/arm64/kernel/kexec_image.c
@@ -76,6 +76,7 @@ static void *image_load(struct kimage *i
 	kbuf.buf_min = 0;
 	kbuf.buf_max = ULONG_MAX;
 	kbuf.top_down = false;
+	kbuf.random = 0;
 
 	kbuf.buffer = kernel;
 	kbuf.bufsz = kernel_len;
_
Re: [PATCH v9 1/8] kexec_file: allow to place kexec_buf randomly
Posted by Baoquan He 2 months, 2 weeks ago
On 08/25/25 at 06:05pm, Andrew Morton wrote:
> On Mon, 25 Aug 2025 09:18:53 +0800 Coiby Xu <coxu@redhat.com> wrote:
> 
> > >diff --git a/arch/arm64/kernel/kexec_image.c b/arch/arm64/kernel/kexec_image.c
> > >index 532d72ea42ee8..287b25e674d76 100644
> > >--- a/arch/arm64/kernel/kexec_image.c
> > >+++ b/arch/arm64/kernel/kexec_image.c
> > >@@ -76,6 +76,7 @@ static void *image_load(struct kimage *image,
> > > 	kbuf.buf_min = 0;
> > > 	kbuf.buf_max = ULONG_MAX;
> > > 	kbuf.top_down = false;
> > >+	kbuf.random = 0;
> > >
> > > 	kbuf.buffer = kernel;
> > > 	kbuf.bufsz = kernel_len;
> > >
> > 
> > And also thanks for posing a fix! The patch LGTM. Can you add a Fixes
> > tag 'Fixes: bf454ec31add ("kexec_file: allow to place kexec_buf
> > randomly")' and then send it to kexec@lists.infradead.org? Thanks!
> 
> I turned all this into a regular patch and queued it (see below),
> thanks.  No additional actions are needed.
> 
> I'm really not liking that code.  I laboriously verified that all
> fields of kexec_buf are now initialized, except for `cma'.  Is that a
> bug?
> 
> This function has a call frequency of about 3x per week.  Can we please
> just memset the whole thing so people don't have to worry about this
> any more?

Yeah, adding these trivial patches to mute XXSAN warning is annoying.
Maybe arm64 can initialize the local variable kbuf like we do in x86_64
as below, to explicitly set the necessary fields when defining.

static void *bzImage64_load(struct kimage *image, char *kernel,
                            unsigned long kernel_len, char *initrd,
                            unsigned long initrd_len, char *cmdline,
                            unsigned long cmdline_len)
{
......
        struct kexec_buf kbuf = { .image = image, .buf_max = ULONG_MAX,
                                  .top_down = true };
        struct kexec_buf pbuf = { .image = image, .buf_min = MIN_PURGATORY_ADDR,
                                  .buf_max = ULONG_MAX, .top_down = true };
.....
}

> 
> 
> From: Breno Leitao <leitao@debian.org>
> Subject: kexec/arm64: initialize the random field of kbuf to zero in the image loader
> Date: Thu Aug 21 04:11:21 2025 -0700
> 
> Add an explicit initialization for the random member of the kbuf structure
> within the image_load function in arch/arm64/kernel/kexec_image.c. 
> Setting kbuf.random to zero ensures a deterministic and clean starting
> state for the buffer used during kernel image loading, avoiding this UBSAN
> issue later, when kbuf.random is read.
> 
>   [   32.362488] UBSAN: invalid-load in ./include/linux/kexec.h:210:10
>   [   32.362649] load of value 252 is not a valid value for type '_Bool'
> 
> Link: https://lkml.kernel.org/r/oninomspajhxp4omtdapxnckxydbk2nzmrix7rggmpukpnzadw@c67o7njgdgm3
> Fixes: bf454ec31add ("kexec_file: allow to place kexec_buf randomly
> Signed-off-by: Breno Leitao <leitao@debian.org>
> Cc: Baoquan He <bhe@redhat.com>
> Cc: Coiby Xu <coxu@redhat.com>
> Cc: "Daniel P. Berrange" <berrange@redhat.com>
> Cc: Dave Hansen <dave.hansen@intel.com>
> Cc: Dave Young <dyoung@redhat.com>
> Cc: Kairui Song <ryncsn@gmail.com>
> Cc: Liu Pingfan <kernelfans@gmail.com>
> Cc: Milan Broz <gmazyland@gmail.com>
> Cc: Ondrej Kozina <okozina@redhat.com>
> Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
> Cc: <stable@vger.kernel.org>
> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
> ---
> 
>  arch/arm64/kernel/kexec_image.c |    1 +
>  1 file changed, 1 insertion(+)
> 
> --- a/arch/arm64/kernel/kexec_image.c~kexec-arm64-initialize-the-random-field-of-kbuf-to-zero-in-the-image-loader
> +++ a/arch/arm64/kernel/kexec_image.c
> @@ -76,6 +76,7 @@ static void *image_load(struct kimage *i
>  	kbuf.buf_min = 0;
>  	kbuf.buf_max = ULONG_MAX;
>  	kbuf.top_down = false;
> +	kbuf.random = 0;
>  
>  	kbuf.buffer = kernel;
>  	kbuf.bufsz = kernel_len;
> _
>
Re: [PATCH v9 1/8] kexec_file: allow to place kexec_buf randomly
Posted by Breno Leitao 2 months, 2 weeks ago
On Wed, Aug 27, 2025 at 07:38:29PM +0800, Baoquan He wrote:
> On 08/25/25 at 06:05pm, Andrew Morton wrote:
> > On Mon, 25 Aug 2025 09:18:53 +0800 Coiby Xu <coxu@redhat.com> wrote:
> > 
> > > >diff --git a/arch/arm64/kernel/kexec_image.c b/arch/arm64/kernel/kexec_image.c
> > > >index 532d72ea42ee8..287b25e674d76 100644
> > > >--- a/arch/arm64/kernel/kexec_image.c
> > > >+++ b/arch/arm64/kernel/kexec_image.c
> > > >@@ -76,6 +76,7 @@ static void *image_load(struct kimage *image,
> > > > 	kbuf.buf_min = 0;
> > > > 	kbuf.buf_max = ULONG_MAX;
> > > > 	kbuf.top_down = false;
> > > >+	kbuf.random = 0;
> > > >
> > > > 	kbuf.buffer = kernel;
> > > > 	kbuf.bufsz = kernel_len;
> > > >
> > > 
> > > And also thanks for posing a fix! The patch LGTM. Can you add a Fixes
> > > tag 'Fixes: bf454ec31add ("kexec_file: allow to place kexec_buf
> > > randomly")' and then send it to kexec@lists.infradead.org? Thanks!
> > 
> > I turned all this into a regular patch and queued it (see below),
> > thanks.  No additional actions are needed.
> > 
> > I'm really not liking that code.  I laboriously verified that all
> > fields of kexec_buf are now initialized, except for `cma'.  Is that a
> > bug?
> > 
> > This function has a call frequency of about 3x per week.  Can we please
> > just memset the whole thing so people don't have to worry about this
> > any more?
> 
> Yeah, adding these trivial patches to mute XXSAN warning is annoying.

The patchset is quite simple, tho:

https://lore.kernel.org/all/20250827-kbuf_all-v1-0-1df9882bb01a@debian.org/