Currently only hw_tags mode of kasan can be enabled or disabled with
kernel parameter kasan=on|off for built kernel. For kasan generic and
sw_tags mode, there's no way to disable them once kernel is built.
This is not convenient sometime, e.g in system kdump is configured.
When the 1st kernel has KASAN enabled and crash triggered to switch to
kdump kernel, the generic or sw_tags mode will cost much extra memory
for kasan shadow while in fact it's meaningless to have kasan in kdump
kernel.

So this patchset moves the kasan=on|off out of hw_tags scope and into
common code to make it visible in generic and sw_tags mode too. Then we
can add kasan=off in kdump kernel to reduce the unneeded meomry cost for
kasan.

Changelog:
====
v1->v2:
- Add __ro_after_init for __ro_after_init, and remove redundant blank
  lines in mm/kasan/common.c. Thanks to Marco.
- Fix a code bug in <linux/kasan-enabled.h> when CONFIG_KASAN is unset,
  this is found out by SeongJae and Lorenzo, and also reported by LKP
  report, thanks to them.
- Add a missing kasan_enabled() checking in kasan_report(). This will
  cause below KASAN report info even though kasan=off is set:
     ==================================================================
     BUG: KASAN: stack-out-of-bounds in tick_program_event+0x130/0x150
     Read of size 4 at addr ffff00005f747778 by task swapper/0/1
     
     CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.16.0+ #8 PREEMPT(voluntary) 
     Hardware name: GIGABYTE R272-P30-JG/MP32-AR0-JG, BIOS F31n (SCP: 2.10.20220810) 09/30/2022
     Call trace:
      show_stack+0x30/0x90 (C)
      dump_stack_lvl+0x7c/0xa0
      print_address_description.constprop.0+0x90/0x310
      print_report+0x104/0x1f0
      kasan_report+0xc8/0x110
      __asan_report_load4_noabort+0x20/0x30
      tick_program_event+0x130/0x150
      ......snip...
     ==================================================================

- Add jump_label_init() calling before kasan_init() in setup_arch() in these
  architectures: xtensa, arm. Because they currenly rely on
  jump_label_init() in main() which is a little late. Then the early static
  key kasan_flag_enabled in kasan_init() won't work.

- In UML architecture, change to enable kasan_flag_enabled in arch_mm_preinit()
  because kasan_init() is enabled before main(), there's no chance to operate
  on static key in kasan_init().

Test:
=====
In v1, I took test on x86_64 for generic mode, and on arm64 for
generic, sw_tags and hw_tags mode. All of them works well.

In v2, I only tested on arm64 for generic, sw_tags and hw_tags mode, it
works. For powerpc, I got a BOOK3S/64 machine, while it says
'KASAN not enabled as it requires radix' and KASAN is disabled. Will
look for other POWER machine to test this.
====

Baoquan He (12):
  mm/kasan: add conditional checks in functions to return directly if
    kasan is disabled
  mm/kasan: move kasan= code to common place
  mm/kasan/sw_tags: don't initialize kasan if it's disabled
  arch/arm: don't initialize kasan if it's disabled
  arch/arm64: don't initialize kasan if it's disabled
  arch/loongarch: don't initialize kasan if it's disabled
  arch/powerpc: don't initialize kasan if it's disabled
  arch/riscv: don't initialize kasan if it's disabled
  arch/x86: don't initialize kasan if it's disabled
  arch/xtensa: don't initialize kasan if it's disabled
  arch/um: don't initialize kasan if it's disabled
  mm/kasan: make kasan=on|off take effect for all three modes

 arch/arm/kernel/setup.c                |  6 +++++
 arch/arm/mm/kasan_init.c               |  6 +++++
 arch/arm64/mm/kasan_init.c             |  7 ++++++
 arch/loongarch/mm/kasan_init.c         |  5 ++++
 arch/powerpc/mm/kasan/init_32.c        |  8 +++++-
 arch/powerpc/mm/kasan/init_book3e_64.c |  6 +++++
 arch/powerpc/mm/kasan/init_book3s_64.c |  6 +++++
 arch/riscv/mm/kasan_init.c             |  6 +++++
 arch/um/kernel/mem.c                   |  6 +++++
 arch/x86/mm/kasan_init_64.c            |  6 +++++
 arch/xtensa/kernel/setup.c             |  1 +
 arch/xtensa/mm/kasan_init.c            |  6 +++++
 include/linux/kasan-enabled.h          | 18 ++++++-------
 mm/kasan/common.c                      | 25 ++++++++++++++++++
 mm/kasan/generic.c                     | 20 +++++++++++++--
 mm/kasan/hw_tags.c                     | 35 ++------------------------
 mm/kasan/init.c                        |  6 +++++
 mm/kasan/quarantine.c                  |  3 +++
 mm/kasan/report.c                      |  4 ++-
 mm/kasan/shadow.c                      | 23 ++++++++++++++++-
 mm/kasan/sw_tags.c                     |  9 +++++++
 21 files changed, 165 insertions(+), 47 deletions(-)

-- 
2.41.0

On 08/12/25 at 08:49pm, Baoquan He wrote:
> Currently only hw_tags mode of kasan can be enabled or disabled with
> kernel parameter kasan=on|off for built kernel. For kasan generic and
> sw_tags mode, there's no way to disable them once kernel is built.
> This is not convenient sometime, e.g in system kdump is configured.
> When the 1st kernel has KASAN enabled and crash triggered to switch to
> kdump kernel, the generic or sw_tags mode will cost much extra memory
> for kasan shadow while in fact it's meaningless to have kasan in kdump
> kernel.
> 
> So this patchset moves the kasan=on|off out of hw_tags scope and into
> common code to make it visible in generic and sw_tags mode too. Then we
> can add kasan=off in kdump kernel to reduce the unneeded meomry cost for
> kasan.
> 
> Changelog:
> ====
> v1->v2:
> - Add __ro_after_init for __ro_after_init, and remove redundant blank
                            ~~~~~~~~~~~~~ s/__ro_after_init/kasan_arg_disabled/
                            Sorry for typo here.
>   lines in mm/kasan/common.c. Thanks to Marco.

Forgot adding related ARCH mailing list or people to CC, add them.

On 08/12/25 at 08:49pm, Baoquan He wrote:
> Currently only hw_tags mode of kasan can be enabled or disabled with
> kernel parameter kasan=on|off for built kernel. For kasan generic and
> sw_tags mode, there's no way to disable them once kernel is built.
> This is not convenient sometime, e.g in system kdump is configured.
> When the 1st kernel has KASAN enabled and crash triggered to switch to
> kdump kernel, the generic or sw_tags mode will cost much extra memory
> for kasan shadow while in fact it's meaningless to have kasan in kdump
> kernel.
> 
> So this patchset moves the kasan=on|off out of hw_tags scope and into
> common code to make it visible in generic and sw_tags mode too. Then we
> can add kasan=off in kdump kernel to reduce the unneeded meomry cost for
> kasan.
> 
> Changelog:
> ====
> v1->v2:
> - Add __ro_after_init for __ro_after_init, and remove redundant blank
>   lines in mm/kasan/common.c. Thanks to Marco.
> - Fix a code bug in <linux/kasan-enabled.h> when CONFIG_KASAN is unset,
>   this is found out by SeongJae and Lorenzo, and also reported by LKP
>   report, thanks to them.
> - Add a missing kasan_enabled() checking in kasan_report(). This will
>   cause below KASAN report info even though kasan=off is set:
>      ==================================================================
>      BUG: KASAN: stack-out-of-bounds in tick_program_event+0x130/0x150
>      Read of size 4 at addr ffff00005f747778 by task swapper/0/1
>      
>      CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.16.0+ #8 PREEMPT(voluntary) 
>      Hardware name: GIGABYTE R272-P30-JG/MP32-AR0-JG, BIOS F31n (SCP: 2.10.20220810) 09/30/2022
>      Call trace:
>       show_stack+0x30/0x90 (C)
>       dump_stack_lvl+0x7c/0xa0
>       print_address_description.constprop.0+0x90/0x310
>       print_report+0x104/0x1f0
>       kasan_report+0xc8/0x110
>       __asan_report_load4_noabort+0x20/0x30
>       tick_program_event+0x130/0x150
>       ......snip...
>      ==================================================================
> 
> - Add jump_label_init() calling before kasan_init() in setup_arch() in these
>   architectures: xtensa, arm. Because they currenly rely on
>   jump_label_init() in main() which is a little late. Then the early static
>   key kasan_flag_enabled in kasan_init() won't work.
> 
> - In UML architecture, change to enable kasan_flag_enabled in arch_mm_preinit()
>   because kasan_init() is enabled before main(), there's no chance to operate
>   on static key in kasan_init().
> 
> Test:
> =====
> In v1, I took test on x86_64 for generic mode, and on arm64 for
> generic, sw_tags and hw_tags mode. All of them works well.
> 
> In v2, I only tested on arm64 for generic, sw_tags and hw_tags mode, it
> works. For powerpc, I got a BOOK3S/64 machine, while it says
> 'KASAN not enabled as it requires radix' and KASAN is disabled. Will
> look for other POWER machine to test this.
> ====
> 
> Baoquan He (12):
>   mm/kasan: add conditional checks in functions to return directly if
>     kasan is disabled
>   mm/kasan: move kasan= code to common place
>   mm/kasan/sw_tags: don't initialize kasan if it's disabled
>   arch/arm: don't initialize kasan if it's disabled
>   arch/arm64: don't initialize kasan if it's disabled
>   arch/loongarch: don't initialize kasan if it's disabled
>   arch/powerpc: don't initialize kasan if it's disabled
>   arch/riscv: don't initialize kasan if it's disabled
>   arch/x86: don't initialize kasan if it's disabled
>   arch/xtensa: don't initialize kasan if it's disabled
>   arch/um: don't initialize kasan if it's disabled
>   mm/kasan: make kasan=on|off take effect for all three modes
> 
>  arch/arm/kernel/setup.c                |  6 +++++
>  arch/arm/mm/kasan_init.c               |  6 +++++
>  arch/arm64/mm/kasan_init.c             |  7 ++++++
>  arch/loongarch/mm/kasan_init.c         |  5 ++++
>  arch/powerpc/mm/kasan/init_32.c        |  8 +++++-
>  arch/powerpc/mm/kasan/init_book3e_64.c |  6 +++++
>  arch/powerpc/mm/kasan/init_book3s_64.c |  6 +++++
>  arch/riscv/mm/kasan_init.c             |  6 +++++
>  arch/um/kernel/mem.c                   |  6 +++++
>  arch/x86/mm/kasan_init_64.c            |  6 +++++
>  arch/xtensa/kernel/setup.c             |  1 +
>  arch/xtensa/mm/kasan_init.c            |  6 +++++
>  include/linux/kasan-enabled.h          | 18 ++++++-------
>  mm/kasan/common.c                      | 25 ++++++++++++++++++
>  mm/kasan/generic.c                     | 20 +++++++++++++--
>  mm/kasan/hw_tags.c                     | 35 ++------------------------
>  mm/kasan/init.c                        |  6 +++++
>  mm/kasan/quarantine.c                  |  3 +++
>  mm/kasan/report.c                      |  4 ++-
>  mm/kasan/shadow.c                      | 23 ++++++++++++++++-
>  mm/kasan/sw_tags.c                     |  9 +++++++
>  21 files changed, 165 insertions(+), 47 deletions(-)
> 
> -- 
> 2.41.0
>

On Tue, Aug 12, 2025 at 2:49 PM Baoquan He <bhe@redhat.com> wrote:
>
> Currently only hw_tags mode of kasan can be enabled or disabled with
> kernel parameter kasan=on|off for built kernel. For kasan generic and
> sw_tags mode, there's no way to disable them once kernel is built.
> This is not convenient sometime, e.g in system kdump is configured.
> When the 1st kernel has KASAN enabled and crash triggered to switch to
> kdump kernel, the generic or sw_tags mode will cost much extra memory
> for kasan shadow while in fact it's meaningless to have kasan in kdump
> kernel.
>
> So this patchset moves the kasan=on|off out of hw_tags scope and into
> common code to make it visible in generic and sw_tags mode too. Then we
> can add kasan=off in kdump kernel to reduce the unneeded meomry cost for
> kasan.

Hi Baoquan,

Could you clarify what are you trying to achieve by disabling
Generic/SW_TAGS KASAN via command-line? Do you want not to see any
KASAN reports produced? Or gain back the performance?

Because for the no reports goal, it would be much easier to add a
command-line parameter to silent the reports.

And the performance goal can only be partially achieved, as you cannot
remove the compiler instrumentation without rebuilding the kernel.
(What are the boot times for KASAN_GENERIC=n vs KASAN_GENERIC=y +
kasan=off vs KASAN_GENERIC=y btw?)

Thank you!

On Tue, Aug 12, 2025 at 6:57 PM Andrey Konovalov <andreyknvl@gmail.com> wrote:
>
> On Tue, Aug 12, 2025 at 2:49 PM Baoquan He <bhe@redhat.com> wrote:
> >
> > Currently only hw_tags mode of kasan can be enabled or disabled with
> > kernel parameter kasan=on|off for built kernel. For kasan generic and
> > sw_tags mode, there's no way to disable them once kernel is built.
> > This is not convenient sometime, e.g in system kdump is configured.
> > When the 1st kernel has KASAN enabled and crash triggered to switch to
> > kdump kernel, the generic or sw_tags mode will cost much extra memory
> > for kasan shadow while in fact it's meaningless to have kasan in kdump
> > kernel.
> >
> > So this patchset moves the kasan=on|off out of hw_tags scope and into
> > common code to make it visible in generic and sw_tags mode too. Then we
> > can add kasan=off in kdump kernel to reduce the unneeded meomry cost for
> > kasan.
>
> Hi Baoquan,
>
> Could you clarify what are you trying to achieve by disabling
> Generic/SW_TAGS KASAN via command-line? Do you want not to see any
> KASAN reports produced? Or gain back the performance?
>
> Because for the no reports goal, it would be much easier to add a
> command-line parameter to silent the reports.
>
> And the performance goal can only be partially achieved, as you cannot
> remove the compiler instrumentation without rebuilding the kernel.
> (What are the boot times for KASAN_GENERIC=n vs KASAN_GENERIC=y +
> kasan=off vs KASAN_GENERIC=y btw?)
>
> Thank you!

Ah, you don't want the shadow memory for kdump, sorry, I somehow missed that.

I'm not familiar with the internals of kdump, but would it be
possible/reasonable to teach kdump to ignore the KASAN shadow region?

On 08/12/25 at 07:14pm, Andrey Konovalov wrote:
> On Tue, Aug 12, 2025 at 6:57 PM Andrey Konovalov <andreyknvl@gmail.com> wrote:
> >
> > On Tue, Aug 12, 2025 at 2:49 PM Baoquan He <bhe@redhat.com> wrote:
> > >
> > > Currently only hw_tags mode of kasan can be enabled or disabled with
> > > kernel parameter kasan=on|off for built kernel. For kasan generic and
> > > sw_tags mode, there's no way to disable them once kernel is built.
> > > This is not convenient sometime, e.g in system kdump is configured.
> > > When the 1st kernel has KASAN enabled and crash triggered to switch to
> > > kdump kernel, the generic or sw_tags mode will cost much extra memory
> > > for kasan shadow while in fact it's meaningless to have kasan in kdump
> > > kernel.
> > >
> > > So this patchset moves the kasan=on|off out of hw_tags scope and into
> > > common code to make it visible in generic and sw_tags mode too. Then we
> > > can add kasan=off in kdump kernel to reduce the unneeded meomry cost for
> > > kasan.
> >
> > Hi Baoquan,
> >
> > Could you clarify what are you trying to achieve by disabling
> > Generic/SW_TAGS KASAN via command-line? Do you want not to see any
> > KASAN reports produced? Or gain back the performance?
> >
> > Because for the no reports goal, it would be much easier to add a
> > command-line parameter to silent the reports.
> >
> > And the performance goal can only be partially achieved, as you cannot
> > remove the compiler instrumentation without rebuilding the kernel.
> > (What are the boot times for KASAN_GENERIC=n vs KASAN_GENERIC=y +
> > kasan=off vs KASAN_GENERIC=y btw?)
> >

Thanks a lot for checking this.

> 
> Ah, you don't want the shadow memory for kdump, sorry, I somehow missed that.

Yeah, for kdump kernel, the shadow is a heavy burden, and most
importantly kasan is useless for kdump. We don't want to capture a
kernel memory bug through kdump kernel running becuase kdump is a
debugging mechanism.

> 
> I'm not familiar with the internals of kdump, but would it be
> possible/reasonable to teach kdump to ignore the KASAN shadow region?

Yes, we can teach kdump to do that. Then people may hate those conditional
check "if (is_kdump_kernel())" being added in kasan code. E.g even
though we skip kasan_init(), we still need to check is_kdump_kernel()
in kasan_populate_vmalloc(), right? 

Combined with the existing kasan_arch_is_ready(), it will make kasan code
ugly. I planned to add kasan_enabled() via static key
kasan_flag_enabled, then it can also easily remove kasan_arch_is_ready()
cleanly.

On Wed, Aug 13, 2025 at 1:14 PM 'Baoquan He' via kasan-dev
<kasan-dev@googlegroups.com> wrote:
>
> > I'm not familiar with the internals of kdump, but would it be
> > possible/reasonable to teach kdump to ignore the KASAN shadow region?
>
> Yes, we can teach kdump to do that. Then people may hate those conditional
> check "if (is_kdump_kernel())" being added in kasan code. E.g even
> though we skip kasan_init(), we still need to check is_kdump_kernel()
> in kasan_populate_vmalloc(), right?
>
> Combined with the existing kasan_arch_is_ready(), it will make kasan code
> ugly. I planned to add kasan_enabled() via static key
> kasan_flag_enabled, then it can also easily remove kasan_arch_is_ready()
> cleanly.

What I had in mind was something different: into the kdump code, we
add a check whether the region of memory it's trying to dump is the
KASAN shadow, and make kdump not to dump this region.

Would this work? Would this help with the issue you have?

(I assume the problem is with the virtual region that is the shadow
memory, as kdump would dump all RAM either way? If not, please clarify
what how does the "heavy burden" that the shadow memory causes
manifests.)

Thank you!

On 08/14/25 at 07:23am, Andrey Konovalov wrote:
> On Wed, Aug 13, 2025 at 1:14 PM 'Baoquan He' via kasan-dev
> <kasan-dev@googlegroups.com> wrote:
> >
> > > I'm not familiar with the internals of kdump, but would it be
> > > possible/reasonable to teach kdump to ignore the KASAN shadow region?
> >
> > Yes, we can teach kdump to do that. Then people may hate those conditional
> > check "if (is_kdump_kernel())" being added in kasan code. E.g even
> > though we skip kasan_init(), we still need to check is_kdump_kernel()
> > in kasan_populate_vmalloc(), right?
> >
> > Combined with the existing kasan_arch_is_ready(), it will make kasan code
> > ugly. I planned to add kasan_enabled() via static key
> > kasan_flag_enabled, then it can also easily remove kasan_arch_is_ready()
> > cleanly.
> 
> What I had in mind was something different: into the kdump code, we
> add a check whether the region of memory it's trying to dump is the
> KASAN shadow, and make kdump not to dump this region.

Ah, I got what you mean. We probably are saying different things.

In order to record memory content of a corrupted kernel, we need reserve
a memory region during bootup of a normal kernel (usually called 1st
kernel) via kernel parameter crashkernel=nMB in advance. Then load
kernel into the crashkernel memory region, that means the region is not
usable for 1st kernel. When 1st kernel collapsed, we stop the 1st kernel
cpu/irq and warmly switch to the loaded kernel in the crashkernel memory
region (usually called kdump kernel). In kdump kernel, it boots up and
enable necessary features to read out the 1st kernel's memory content,
we usually use user space tool like makeudmpfile to filter out unwanted
memory content.

So this patchset intends to disable KASAN to decrease the crashkernel
meomry value because crashkernel is not usable for 1st kernel. As for
shadow memory of 1st kernel, we need recognize it and filter it away
in makedumpfile. 

> 
> Would this work? Would this help with the issue you have?
> 
> (I assume the problem is with the virtual region that is the shadow
> memory, as kdump would dump all RAM either way? If not, please clarify
> what how does the "heavy burden" that the shadow memory causes
> manifests.)
> 
> Thank you!
> 

On Thu, Aug 14, 2025 at 10:56 AM Baoquan He <bhe@redhat.com> wrote:
>
> Ah, I got what you mean. We probably are saying different things.
>
> In order to record memory content of a corrupted kernel, we need reserve
> a memory region during bootup of a normal kernel (usually called 1st
> kernel) via kernel parameter crashkernel=nMB in advance. Then load
> kernel into the crashkernel memory region, that means the region is not
> usable for 1st kernel. When 1st kernel collapsed, we stop the 1st kernel
> cpu/irq and warmly switch to the loaded kernel in the crashkernel memory
> region (usually called kdump kernel). In kdump kernel, it boots up and
> enable necessary features to read out the 1st kernel's memory content,
> we usually use user space tool like makeudmpfile to filter out unwanted
> memory content.
>
> So this patchset intends to disable KASAN to decrease the crashkernel
> meomry value because crashkernel is not usable for 1st kernel. As for
> shadow memory of 1st kernel, we need recognize it and filter it away
> in makedumpfile.

Ah, I see, thank you for the explanation!

So kdump kernel runs with the amount of RAM specified by crashkernel=.
And KASAN's shadow memory increases RAM usage, which means
crashkernel= needs to be set to a higher value for KASAN kernels. Is
my understanding of the problem correct?

On 08/16/25 at 06:50am, Andrey Konovalov wrote:
> On Thu, Aug 14, 2025 at 10:56 AM Baoquan He <bhe@redhat.com> wrote:
> >
> > Ah, I got what you mean. We probably are saying different things.
> >
> > In order to record memory content of a corrupted kernel, we need reserve
> > a memory region during bootup of a normal kernel (usually called 1st
> > kernel) via kernel parameter crashkernel=nMB in advance. Then load
> > kernel into the crashkernel memory region, that means the region is not
> > usable for 1st kernel. When 1st kernel collapsed, we stop the 1st kernel
> > cpu/irq and warmly switch to the loaded kernel in the crashkernel memory
> > region (usually called kdump kernel). In kdump kernel, it boots up and
> > enable necessary features to read out the 1st kernel's memory content,
> > we usually use user space tool like makeudmpfile to filter out unwanted
> > memory content.
> >
> > So this patchset intends to disable KASAN to decrease the crashkernel
> > meomry value because crashkernel is not usable for 1st kernel. As for
> > shadow memory of 1st kernel, we need recognize it and filter it away
> > in makedumpfile.
> 
> Ah, I see, thank you for the explanation!
> 
> So kdump kernel runs with the amount of RAM specified by crashkernel=.
> And KASAN's shadow memory increases RAM usage, which means
> crashkernel= needs to be set to a higher value for KASAN kernels. Is
> my understanding of the problem correct?

Yeah, you are quite right.

When I tested it, on x86_64 and arm64, usually I set crashkernel=256M
and it's sufficient. However, when KASAN is enabled and generic mode is
taken, I need set crashkernel=768M to make vmcore dumping succeed. In
kdump kernel, read_vmcore() uses ioremap to map the old memory of
collapsed kernel for reading out, those vmalloc-ed areas are lazily
freed and cause more shadow memory than what we usually think shadow
memory only costs 1/8 of physical RAM.