[v1] mm/kfence: add reboot notifier to disable KFENCE on shutdown

[PATCH] mm/kfence: add reboot notifier to disable KFENCE on shutdown

Posted by Breno Leitao 5 days, 4 hours ago

During system shutdown, KFENCE can cause IPI synchronization issues if
it remains active through the reboot process. To prevent this, register
a reboot notifier that disables KFENCE and cancels any pending timer
work early in the shutdown sequence.

This is only necessary when CONFIG_KFENCE_STATIC_KEYS is enabled, as
this configuration sends IPIs that can interfere with shutdown. Without
static keys, no IPIs are generated and KFENCE can safely remain active.

The notifier uses maximum priority (INT_MAX) to ensure KFENCE shuts
down before other subsystems that might still depend on stable memory
allocation behavior.

This fixes a late kexec CSD lockup[1] when kfence is trying to IPI a CPU
that is busy in a IRQ-disabled context printing characters to the
console.

Link: https://lore.kernel.org/all/sqwajvt7utnt463tzxgwu2yctyn5m6bjwrslsnupfexeml6hkd@v6sqmpbu3vvu/ [1]

Signed-off-by: Breno Leitao <leitao@debian.org>
---
 mm/kfence/core.c | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/mm/kfence/core.c b/mm/kfence/core.c
index 727c20c94ac5..162a026871ab 100644
--- a/mm/kfence/core.c
+++ b/mm/kfence/core.c
@@ -26,6 +26,7 @@
 #include <linux/panic_notifier.h>
 #include <linux/random.h>
 #include <linux/rcupdate.h>
+#include <linux/reboot.h>
 #include <linux/sched/clock.h>
 #include <linux/seq_file.h>
 #include <linux/slab.h>
@@ -820,6 +821,25 @@ static struct notifier_block kfence_check_canary_notifier = {
 static struct delayed_work kfence_timer;
 
 #ifdef CONFIG_KFENCE_STATIC_KEYS
+static int kfence_reboot_callback(struct notifier_block *nb,
+				  unsigned long action, void *data)
+{
+	/*
+	 * Disable kfence to avoid static keys IPI synchronization during
+	 * late shutdown/kexec
+	 */
+	WRITE_ONCE(kfence_enabled, false);
+	/* Cancel any pending timer work */
+	cancel_delayed_work_sync(&kfence_timer);
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block kfence_reboot_notifier = {
+	.notifier_call = kfence_reboot_callback,
+	.priority = INT_MAX, /* Run early to stop timers ASAP */
+};
+
 /* Wait queue to wake up allocation-gate timer task. */
 static DECLARE_WAIT_QUEUE_HEAD(allocation_wait);
 
@@ -901,6 +921,10 @@ static void kfence_init_enable(void)
 	if (kfence_check_on_panic)
 		atomic_notifier_chain_register(&panic_notifier_list, &kfence_check_canary_notifier);
 
+#ifdef CONFIG_KFENCE_STATIC_KEYS
+	register_reboot_notifier(&kfence_reboot_notifier);
+#endif
+
 	WRITE_ONCE(kfence_enabled, true);
 	queue_delayed_work(system_unbound_wq, &kfence_timer, 0);
 

---
base-commit: ab084f0b8d6d2ee4b1c6a28f39a2a7430bdfa7f0
change-id: 20251126-kfence-42c93f9b3979

Best regards,
--  
Breno Leitao <leitao@debian.org>

Re: [PATCH] mm/kfence: add reboot notifier to disable KFENCE on shutdown

Posted by Andrew Morton 5 days, 3 hours ago

On Wed, 26 Nov 2025 09:46:18 -0800 Breno Leitao <leitao@debian.org> wrote:

> During system shutdown, KFENCE can cause IPI synchronization issues if
> it remains active through the reboot process. To prevent this, register
> a reboot notifier that disables KFENCE and cancels any pending timer
> work early in the shutdown sequence.
> 
> This is only necessary when CONFIG_KFENCE_STATIC_KEYS is enabled, as
> this configuration sends IPIs that can interfere with shutdown. Without
> static keys, no IPIs are generated and KFENCE can safely remain active.
> 
> The notifier uses maximum priority (INT_MAX) to ensure KFENCE shuts
> down before other subsystems that might still depend on stable memory
> allocation behavior.
> 
> This fixes a late kexec CSD lockup[1] when kfence is trying to IPI a CPU
> that is busy in a IRQ-disabled context printing characters to the
> console.
> 
> Link: https://lore.kernel.org/all/sqwajvt7utnt463tzxgwu2yctyn5m6bjwrslsnupfexeml6hkd@v6sqmpbu3vvu/ [1]

6.13 kernels and earlier, so I assume we'll want a cc:stable on this. 
And I assume there's really no identifiable Fixes: target.

Re: [PATCH] mm/kfence: add reboot notifier to disable KFENCE on shutdown

Posted by Breno Leitao 4 days, 10 hours ago

On Wed, Nov 26, 2025 at 10:14:53AM -0800, Andrew Morton wrote:
> On Wed, 26 Nov 2025 09:46:18 -0800 Breno Leitao <leitao@debian.org> wrote:
> 
> > During system shutdown, KFENCE can cause IPI synchronization issues if
> > it remains active through the reboot process. To prevent this, register
> > a reboot notifier that disables KFENCE and cancels any pending timer
> > work early in the shutdown sequence.
> > 
> > This is only necessary when CONFIG_KFENCE_STATIC_KEYS is enabled, as
> > this configuration sends IPIs that can interfere with shutdown. Without
> > static keys, no IPIs are generated and KFENCE can safely remain active.
> > 
> > The notifier uses maximum priority (INT_MAX) to ensure KFENCE shuts
> > down before other subsystems that might still depend on stable memory
> > allocation behavior.
> > 
> > This fixes a late kexec CSD lockup[1] when kfence is trying to IPI a CPU
> > that is busy in a IRQ-disabled context printing characters to the
> > console.
> > 
> > Link: https://lore.kernel.org/all/sqwajvt7utnt463tzxgwu2yctyn5m6bjwrslsnupfexeml6hkd@v6sqmpbu3vvu/ [1]
> 
> 6.13 kernels and earlier, so I assume we'll want a cc:stable on this. 
> And I assume there's really no identifiable Fixes: target.

This infrastructure showed up when kfence was created, so, a possible
Fixes: target would point to commit 0ce20dd84089  ("mm: add Kernel
Electric-Fence infrastructure")

Re: [PATCH] mm/kfence: add reboot notifier to disable KFENCE on shutdown

Posted by Andrew Morton 4 days, 2 hours ago

On Thu, 27 Nov 2025 03:12:10 -0800 Breno Leitao <leitao@debian.org> wrote:

> > > This fixes a late kexec CSD lockup[1] when kfence is trying to IPI a CPU
> > > that is busy in a IRQ-disabled context printing characters to the
> > > console.
> > > 
> > > Link: https://lore.kernel.org/all/sqwajvt7utnt463tzxgwu2yctyn5m6bjwrslsnupfexeml6hkd@v6sqmpbu3vvu/ [1]
> > 
> > 6.13 kernels and earlier, so I assume we'll want a cc:stable on this. 
> > And I assume there's really no identifiable Fixes: target.
> 
> This infrastructure showed up when kfence was created, so, a possible
> Fixes: target would point to commit 0ce20dd84089  ("mm: add Kernel
> Electric-Fence infrastructure")

Great, thanks, I added that.

Re: [PATCH] mm/kfence: add reboot notifier to disable KFENCE on shutdown

Posted by Marco Elver 5 days, 4 hours ago

On Wed, 26 Nov 2025 at 18:46, Breno Leitao <leitao@debian.org> wrote:
>
> During system shutdown, KFENCE can cause IPI synchronization issues if
> it remains active through the reboot process. To prevent this, register
> a reboot notifier that disables KFENCE and cancels any pending timer
> work early in the shutdown sequence.
>
> This is only necessary when CONFIG_KFENCE_STATIC_KEYS is enabled, as
> this configuration sends IPIs that can interfere with shutdown. Without
> static keys, no IPIs are generated and KFENCE can safely remain active.
>
> The notifier uses maximum priority (INT_MAX) to ensure KFENCE shuts
> down before other subsystems that might still depend on stable memory
> allocation behavior.
>
> This fixes a late kexec CSD lockup[1] when kfence is trying to IPI a CPU
> that is busy in a IRQ-disabled context printing characters to the
> console.
>
> Link: https://lore.kernel.org/all/sqwajvt7utnt463tzxgwu2yctyn5m6bjwrslsnupfexeml6hkd@v6sqmpbu3vvu/ [1]
>
> Signed-off-by: Breno Leitao <leitao@debian.org>

Looks good as discussed in [1]:

Reviewed-by: Marco Elver <elver@google.com>

> ---
>  mm/kfence/core.c | 24 ++++++++++++++++++++++++
>  1 file changed, 24 insertions(+)
>
> diff --git a/mm/kfence/core.c b/mm/kfence/core.c
> index 727c20c94ac5..162a026871ab 100644
> --- a/mm/kfence/core.c
> +++ b/mm/kfence/core.c
> @@ -26,6 +26,7 @@
>  #include <linux/panic_notifier.h>
>  #include <linux/random.h>
>  #include <linux/rcupdate.h>
> +#include <linux/reboot.h>
>  #include <linux/sched/clock.h>
>  #include <linux/seq_file.h>
>  #include <linux/slab.h>
> @@ -820,6 +821,25 @@ static struct notifier_block kfence_check_canary_notifier = {
>  static struct delayed_work kfence_timer;
>
>  #ifdef CONFIG_KFENCE_STATIC_KEYS
> +static int kfence_reboot_callback(struct notifier_block *nb,
> +                                 unsigned long action, void *data)
> +{
> +       /*
> +        * Disable kfence to avoid static keys IPI synchronization during
> +        * late shutdown/kexec
> +        */
> +       WRITE_ONCE(kfence_enabled, false);
> +       /* Cancel any pending timer work */
> +       cancel_delayed_work_sync(&kfence_timer);
> +
> +       return NOTIFY_OK;
> +}
> +
> +static struct notifier_block kfence_reboot_notifier = {
> +       .notifier_call = kfence_reboot_callback,
> +       .priority = INT_MAX, /* Run early to stop timers ASAP */
> +};
> +
>  /* Wait queue to wake up allocation-gate timer task. */
>  static DECLARE_WAIT_QUEUE_HEAD(allocation_wait);
>
> @@ -901,6 +921,10 @@ static void kfence_init_enable(void)
>         if (kfence_check_on_panic)
>                 atomic_notifier_chain_register(&panic_notifier_list, &kfence_check_canary_notifier);
>
> +#ifdef CONFIG_KFENCE_STATIC_KEYS
> +       register_reboot_notifier(&kfence_reboot_notifier);
> +#endif
> +
>         WRITE_ONCE(kfence_enabled, true);
>         queue_delayed_work(system_unbound_wq, &kfence_timer, 0);
>
>
> ---
> base-commit: ab084f0b8d6d2ee4b1c6a28f39a2a7430bdfa7f0
> change-id: 20251126-kfence-42c93f9b3979
>
> Best regards,
> --
> Breno Leitao <leitao@debian.org>
>