arch/x86/Kconfig | 1 + arch/x86/net/bpf_jit_comp.c | 106 +++++++++++++ include/linux/bpf.h | 2 + include/linux/bpf_verifier.h | 2 + include/linux/kasan.h | 13 ++ kernel/bpf/Kconfig | 9 ++ kernel/bpf/core.c | 10 ++ kernel/bpf/verifier.c | 7 + mm/kasan/kasan.h | 10 -- tools/testing/selftests/bpf/prog_tests/kasan.c | 165 +++++++++++++++++++++ tools/testing/selftests/bpf/progs/kasan.c | 146 ++++++++++++++++++ .../testing/selftests/bpf/test_kmods/bpf_testmod.c | 79 ++++++++++ tools/testing/selftests/bpf/test_loader.c | 5 + tools/testing/selftests/bpf/unpriv_helpers.c | 5 + tools/testing/selftests/bpf/unpriv_helpers.h | 1 + 15 files changed, 551 insertions(+), 10 deletions(-)
Hello,
this series aims to bring basic support for KASAN checks to BPF JITed
programs. This follows the first RFC posted in [1].
KASAN allows to spot memory management mistakes by reserving a fraction
of memory as "shadow memory" that will map to the rest of the memory and
allow its monitoring. Each memory-accessing instruction is then
instrumented at build time to call some ASAN check function, that will
analyze the corresponding bits in shadow memory, and if it detects the
access as invalid, trigger a detailed report. The goal of this series is
to replicate this mechanism for BPF programs when they are being JITed
into native instructions: that's then the (runtime) JIT compiler who is
in charge of inserting calls to the corresponding kasan checks, when a
program is being loaded into the kernel. This task involves:
- identifying at program load time the instructions performing memory
accesses
- identifying those accesses properties (size ? read or write ?) to
define the relevant kasan check function to call
- just before the identified instructions:
- perform the basic context saving (ie: saving registers)
- inserting a call to the relevant kasan check function
- restore context
- whenever the instrumented program executes, if it performs an invalid
access, it triggers a kasan report identical to those instrumented on
kernel side at build time.
As discussed in [1], this series is based on some choices and
assumptions:
- it focuses on x86_64 for now, and so only on KASAN_GENERIC
- not all memory accessing BPF instructions are being instrumented:
- it focuses on STX/LDX instructions
- it discards instructions accessing BPF program stack (already
monitored by page guards)
- it discards possibly faulting instructions, like BPF_PROBE_MEM or
BPF_PROBE_ATOMIC insns
The series is marked and sent as RFC:
- to allow collecting feedback early and make sure that it goes into the
right direction
- because it depends on Xu's work to pass data between the verifier and
JIT compilers. This work is not merged yet, see [2]. I have been
tracking the various revisions he sent on the ML and based my local
branch on his work
- because tests brought by this series currently can't run on BPF CI:
they expect kasan multishot to be enabled, otherwise the first test
will make all other kasan-related tests fail.
- because some cases like atomic loads/stores are not instrumented yet
(and are still making me scratch my head)
- because it will hopefully provide a good basis to discuss the topic at
LSFMMBPF (see [3])
Despite this series not being ready for integration yet, anyone
interested in running it locally can perform the following steps to run
the JITed KASAN instrumentation selftests:
- rebasing locally this series on [2]
- building and running the corresponding kernel with kasan_multi_shot
enabled
- running `test_progs -a kasan`
And should get a variety of KASAN tests executed for BPF programs:
#162/1 kasan/bpf_kasan_uaf_read_1:OK
#162/2 kasan/bpf_kasan_uaf_read_2:OK
#162/3 kasan/bpf_kasan_uaf_read_4:OK
#162/4 kasan/bpf_kasan_uaf_read_8:OK
#162/5 kasan/bpf_kasan_uaf_write_1:OK
#162/6 kasan/bpf_kasan_uaf_write_2:OK
#162/7 kasan/bpf_kasan_uaf_write_4:OK
#162/8 kasan/bpf_kasan_uaf_write_8:OK
#162/9 kasan/bpf_kasan_oob_read_1:OK
#162/10 kasan/bpf_kasan_oob_read_2:OK
#162/11 kasan/bpf_kasan_oob_read_4:OK
#162/12 kasan/bpf_kasan_oob_read_8:OK
#162/13 kasan/bpf_kasan_oob_write_1:OK
#162/14 kasan/bpf_kasan_oob_write_2:OK
#162/15 kasan/bpf_kasan_oob_write_4:OK
#162/16 kasan/bpf_kasan_oob_write_8:OK
#162 kasan:OK
Summary: 1/16 PASSED, 0 SKIPPED, 0 FAILED
[1] https://lore.kernel.org/bpf/DG7UG112AVBC.JKYISDTAM30T@bootlin.com/
[2] https://lore.kernel.org/bpf/cover.1776062885.git.xukuohai@hotmail.com/
[3] https://lore.kernel.org/bpf/DGGNCXX79H8O.2P6K8L1QW1M8K@bootlin.com/
Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
---
Alexis Lothoré (eBPF Foundation) (8):
kasan: expose generic kasan helpers
bpf: mark instructions accessing program stack
bpf: add BPF_JIT_KASAN for KASAN instrumentation of JITed programs
bpf, x86: add helper to emit kasan checks in x86 JITed programs
bpf, x86: emit KASAN checks into x86 JITed programs
selftests/bpf: do not run verifier JIT tests when BPF_JIT_KASAN is enabled
bpf, x86: enable KASAN for JITed programs on x86
selftests/bpf: add tests to validate KASAN on JIT programs
arch/x86/Kconfig | 1 +
arch/x86/net/bpf_jit_comp.c | 106 +++++++++++++
include/linux/bpf.h | 2 +
include/linux/bpf_verifier.h | 2 +
include/linux/kasan.h | 13 ++
kernel/bpf/Kconfig | 9 ++
kernel/bpf/core.c | 10 ++
kernel/bpf/verifier.c | 7 +
mm/kasan/kasan.h | 10 --
tools/testing/selftests/bpf/prog_tests/kasan.c | 165 +++++++++++++++++++++
tools/testing/selftests/bpf/progs/kasan.c | 146 ++++++++++++++++++
.../testing/selftests/bpf/test_kmods/bpf_testmod.c | 79 ++++++++++
tools/testing/selftests/bpf/test_loader.c | 5 +
tools/testing/selftests/bpf/unpriv_helpers.c | 5 +
tools/testing/selftests/bpf/unpriv_helpers.h | 1 +
15 files changed, 551 insertions(+), 10 deletions(-)
---
base-commit: 7990a071b32887a1a883952e8cf60134b6d6fea0
change-id: 20260126-kasan-fcd68f64cd7b
Best regards,
--
Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
On 4/13/26 11:28 AM, Alexis Lothoré (eBPF Foundation) wrote:
> Hello,
> this series aims to bring basic support for KASAN checks to BPF JITed
> programs. This follows the first RFC posted in [1].
Hi Alexis,
Thank you for working on this, it's a real testing gap.
I have a few comments, see below.
The series doesn't apply cleanly on bpf-next right now, but I was able
to apply to a little older revision (eb5249b12507).
>
> KASAN allows to spot memory management mistakes by reserving a fraction
> of memory as "shadow memory" that will map to the rest of the memory and
> allow its monitoring. Each memory-accessing instruction is then
> instrumented at build time to call some ASAN check function, that will
> analyze the corresponding bits in shadow memory, and if it detects the
> access as invalid, trigger a detailed report. The goal of this series is
> to replicate this mechanism for BPF programs when they are being JITed
> into native instructions: that's then the (runtime) JIT compiler who is
> in charge of inserting calls to the corresponding kasan checks, when a
> program is being loaded into the kernel. This task involves:
> - identifying at program load time the instructions performing memory
> accesses
> - identifying those accesses properties (size ? read or write ?) to
> define the relevant kasan check function to call
> - just before the identified instructions:
> - perform the basic context saving (ie: saving registers)
> - inserting a call to the relevant kasan check function
> - restore context
> - whenever the instrumented program executes, if it performs an invalid
> access, it triggers a kasan report identical to those instrumented on
> kernel side at build time.
>
> As discussed in [1], this series is based on some choices and
> assumptions:
> - it focuses on x86_64 for now, and so only on KASAN_GENERIC
I wonder if it's feasible to implement KASAN support on the verifier
side in post-verification fixups. AI slop for illustration:
;; Original (1 BPF insn):
dst = *(u64 *)(src + off) ; BPF_LDX | BPF_MEM | BPF_DW
;; Rewrite (~7 BPF insns):
r_tmp1 = src ; BPF_MOV64_REG
r_tmp1 += off ; BPF_ALU64 | BPF_ADD | K (full address)
r_tmp2 = r_tmp1 ; copy
r_tmp2 >>= 3 ; KASAN_SHADOW_SCALE_SHIFT
r_tmp2 += KASAN_SHADOW_OFFSET ; shadow address
r_tmp3 = *(u8 *)(r_tmp2 + 0) ; BPF_LDX | BPF_B (load shadow byte)
if r_tmp3 != 0 goto +2 ; BPF_JNE | PC+2
dst = *(u64 *)(src + off) ; original access (fast path)
goto +1 ; skip slowpath
call __asan_report_load8 ; BPF kfunc
dst = *(u64 *)(src + off) ; retry the access after report (non-fatal)
A sort of inline kasan directly in BPF.
There are plenty of issues with it: instruction limit, exposing asan
API as kfuncs, etc. On the flip side we get cross-arch support out of
the box with no or mininal JIT changes.
Honestly I'm not excited about this approach, but curious if anyone
thought about this, or maybe it was already discussed?
> - not all memory accessing BPF instructions are being instrumented:
> - it focuses on STX/LDX instructions
> - it discards instructions accessing BPF program stack (already
> monitored by page guards)
> - it discards possibly faulting instructions, like BPF_PROBE_MEM or
> BPF_PROBE_ATOMIC insns
>
> The series is marked and sent as RFC:
> - to allow collecting feedback early and make sure that it goes into the
> right direction
> - because it depends on Xu's work to pass data between the verifier and
> JIT compilers. This work is not merged yet, see [2]. I have been
> tracking the various revisions he sent on the ML and based my local
> branch on his work
> - because tests brought by this series currently can't run on BPF CI:
> they expect kasan multishot to be enabled, otherwise the first test
> will make all other kasan-related tests fail.
AFAICT this can be trivially fixed on BPF CI side, we just need to set
kasan_multi_shot for the VMs running the tests. I will do that, your
next revision doesn't have to be and RFC.
> - because some cases like atomic loads/stores are not instrumented yet
> (and are still making me scratch my head)
> - because it will hopefully provide a good basis to discuss the topic at
> LSFMMBPF (see [3])
Apparently, KASAN reporting routine takes a lock [1]:
__asan_load()
-> check_region_inline()
-> kasan_report()
-> start_report()
-> raw_spin_lock_irqsave(&report_lock, *flags);
BPF programs can run in NMI context, and so it appears to be possible
to get an unflagged (because of lockdep_off() in start_report)
deadlock, if an NMI fires on a CPU already holding report_lock.
Although I guess you'd need two KASAN bugs to happen
simultaneously for that to occur?... A rare event, I would hope.
It could be addressed with either in_nmi() check at runtime, or
forbidding kasan for NMI-runnable BPF program types.
That said, this may be a case of being overly defensive to appease the
ai overlords.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/kasan/report.c?h=v7.0#n204
>
> Despite this series not being ready for integration yet, anyone
> interested in running it locally can perform the following steps to run
> the JITed KASAN instrumentation selftests:
> - rebasing locally this series on [2]
> - building and running the corresponding kernel with kasan_multi_shot
> enabled
> - running `test_progs -a kasan`
>
> And should get a variety of KASAN tests executed for BPF programs:
>
> #162/1 kasan/bpf_kasan_uaf_read_1:OK
> #162/2 kasan/bpf_kasan_uaf_read_2:OK
> #162/3 kasan/bpf_kasan_uaf_read_4:OK
> #162/4 kasan/bpf_kasan_uaf_read_8:OK
> #162/5 kasan/bpf_kasan_uaf_write_1:OK
> #162/6 kasan/bpf_kasan_uaf_write_2:OK
> #162/7 kasan/bpf_kasan_uaf_write_4:OK
> #162/8 kasan/bpf_kasan_uaf_write_8:OK
> #162/9 kasan/bpf_kasan_oob_read_1:OK
> #162/10 kasan/bpf_kasan_oob_read_2:OK
> #162/11 kasan/bpf_kasan_oob_read_4:OK
> #162/12 kasan/bpf_kasan_oob_read_8:OK
> #162/13 kasan/bpf_kasan_oob_write_1:OK
> #162/14 kasan/bpf_kasan_oob_write_2:OK
> #162/15 kasan/bpf_kasan_oob_write_4:OK
> #162/16 kasan/bpf_kasan_oob_write_8:OK
> #162 kasan:OK
> Summary: 1/16 PASSED, 0 SKIPPED, 0 FAILED
>
> [1] https://lore.kernel.org/bpf/DG7UG112AVBC.JKYISDTAM30T@bootlin.com/
> [2] https://lore.kernel.org/bpf/cover.1776062885.git.xukuohai@hotmail.com/
> [3] https://lore.kernel.org/bpf/DGGNCXX79H8O.2P6K8L1QW1M8K@bootlin.com/
>
> Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
> ---
> Alexis Lothoré (eBPF Foundation) (8):
> kasan: expose generic kasan helpers
> bpf: mark instructions accessing program stack
> bpf: add BPF_JIT_KASAN for KASAN instrumentation of JITed programs
> bpf, x86: add helper to emit kasan checks in x86 JITed programs
> bpf, x86: emit KASAN checks into x86 JITed programs
> selftests/bpf: do not run verifier JIT tests when BPF_JIT_KASAN is enabled
> bpf, x86: enable KASAN for JITed programs on x86
> selftests/bpf: add tests to validate KASAN on JIT programs
>
> arch/x86/Kconfig | 1 +
> arch/x86/net/bpf_jit_comp.c | 106 +++++++++++++
> include/linux/bpf.h | 2 +
> include/linux/bpf_verifier.h | 2 +
> include/linux/kasan.h | 13 ++
> kernel/bpf/Kconfig | 9 ++
> kernel/bpf/core.c | 10 ++
> kernel/bpf/verifier.c | 7 +
> mm/kasan/kasan.h | 10 --
> tools/testing/selftests/bpf/prog_tests/kasan.c | 165 +++++++++++++++++++++
> tools/testing/selftests/bpf/progs/kasan.c | 146 ++++++++++++++++++
> .../testing/selftests/bpf/test_kmods/bpf_testmod.c | 79 ++++++++++
> tools/testing/selftests/bpf/test_loader.c | 5 +
> tools/testing/selftests/bpf/unpriv_helpers.c | 5 +
> tools/testing/selftests/bpf/unpriv_helpers.h | 1 +
> 15 files changed, 551 insertions(+), 10 deletions(-)
> ---
> base-commit: 7990a071b32887a1a883952e8cf60134b6d6fea0
> change-id: 20260126-kasan-fcd68f64cd7b
>
> Best regards,
> --
> Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
>
Hi Ihor, thanks a lot for the review and help ! On Sat Apr 25, 2026 at 1:10 AM CEST, Ihor Solodrai wrote: > On 4/13/26 11:28 AM, Alexis Lothoré (eBPF Foundation) wrote: >> Hello, >> this series aims to bring basic support for KASAN checks to BPF JITed >> programs. This follows the first RFC posted in [1]. [...] >> The series is marked and sent as RFC: >> - to allow collecting feedback early and make sure that it goes into the >> right direction >> - because it depends on Xu's work to pass data between the verifier and >> JIT compilers. This work is not merged yet, see [2]. I have been >> tracking the various revisions he sent on the ML and based my local >> branch on his work >> - because tests brought by this series currently can't run on BPF CI: >> they expect kasan multishot to be enabled, otherwise the first test >> will make all other kasan-related tests fail. > > AFAICT this can be trivially fixed on BPF CI side, we just need to set > kasan_multi_shot for the VMs running the tests. I will do that, your > next revision doesn't have to be and RFC. Sweet, much appreciated :) I've started receiving some Sashiko reviews on the series (likely thanks to some of your actions), I'll take a look into those and prepare a (non-RFC) v2. Thanks, Alexis -- Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
On Fri, Apr 24, 2026 at 4:10 PM Ihor Solodrai <ihor.solodrai@linux.dev> wrote: > > I wonder if it's feasible to implement KASAN support on the verifier > side in post-verification fixups. AI slop for illustration: > > ;; Original (1 BPF insn): > dst = *(u64 *)(src + off) ; BPF_LDX | BPF_MEM | BPF_DW > > ;; Rewrite (~7 BPF insns): > r_tmp1 = src ; BPF_MOV64_REG > r_tmp1 += off ; BPF_ALU64 | BPF_ADD | K (full address) > r_tmp2 = r_tmp1 ; copy > r_tmp2 >>= 3 ; KASAN_SHADOW_SCALE_SHIFT > r_tmp2 += KASAN_SHADOW_OFFSET ; shadow address > r_tmp3 = *(u8 *)(r_tmp2 + 0) ; BPF_LDX | BPF_B (load shadow byte) > if r_tmp3 != 0 goto +2 ; BPF_JNE | PC+2 > dst = *(u64 *)(src + off) ; original access (fast path) > goto +1 ; skip slowpath > call __asan_report_load8 ; BPF kfunc > dst = *(u64 *)(src + off) ; retry the access after report (non-fatal) > > A sort of inline kasan directly in BPF. > > There are plenty of issues with it: instruction limit, exposing asan > API as kfuncs, etc. On the flip side we get cross-arch support out of > the box with no or mininal JIT changes. > > Honestly I'm not excited about this approach, but curious if anyone > thought about this, or maybe it was already discussed? We discussed this. It won't work because we don't have that many temp registers for once and second it has to preserve all (both callee and caller saved regs). This is arch specific. Second, we do not want other archs. This feature is x86-64 only. It's being added to find _verifier_ bugs. To do that one arch is enough. > > > - not all memory accessing BPF instructions are being instrumented: > > - it focuses on STX/LDX instructions > > - it discards instructions accessing BPF program stack (already > > monitored by page guards) > > - it discards possibly faulting instructions, like BPF_PROBE_MEM or > > BPF_PROBE_ATOMIC insns > > > > The series is marked and sent as RFC: > > - to allow collecting feedback early and make sure that it goes into the > > right direction > > - because it depends on Xu's work to pass data between the verifier and > > JIT compilers. This work is not merged yet, see [2]. I have been > > tracking the various revisions he sent on the ML and based my local > > branch on his work > > - because tests brought by this series currently can't run on BPF CI: > > they expect kasan multishot to be enabled, otherwise the first test > > will make all other kasan-related tests fail. > > AFAICT this can be trivially fixed on BPF CI side, we just need to set > kasan_multi_shot for the VMs running the tests. I will do that, your > next revision doesn't have to be and RFC. +1 > > - because some cases like atomic loads/stores are not instrumented yet > > (and are still making me scratch my head) > > - because it will hopefully provide a good basis to discuss the topic at > > LSFMMBPF (see [3]) > > Apparently, KASAN reporting routine takes a lock [1]: > > __asan_load() > -> check_region_inline() > -> kasan_report() > -> start_report() > -> raw_spin_lock_irqsave(&report_lock, *flags); > > BPF programs can run in NMI context, and so it appears to be possible > to get an unflagged (because of lockdep_off() in start_report) > deadlock, if an NMI fires on a CPU already holding report_lock. > Although I guess you'd need two KASAN bugs to happen > simultaneously for that to occur?... A rare event, I would hope. > > It could be addressed with either in_nmi() check at runtime, or > forbidding kasan for NMI-runnable BPF program types. We don't need that. If this bpf KASAN finds a bug, it means that it found a verifier bug. All things are out of the window. kasan_report() splat can just as well be the last thing that users will see.
On Sat Apr 25, 2026 at 1:28 AM CEST, Alexei Starovoitov wrote: > On Fri, Apr 24, 2026 at 4:10 PM Ihor Solodrai <ihor.solodrai@linux.dev> wrote: >> >> I wonder if it's feasible to implement KASAN support on the verifier >> side in post-verification fixups. AI slop for illustration: >> >> ;; Original (1 BPF insn): >> dst = *(u64 *)(src + off) ; BPF_LDX | BPF_MEM | BPF_DW >> >> ;; Rewrite (~7 BPF insns): >> r_tmp1 = src ; BPF_MOV64_REG >> r_tmp1 += off ; BPF_ALU64 | BPF_ADD | K (full address) >> r_tmp2 = r_tmp1 ; copy >> r_tmp2 >>= 3 ; KASAN_SHADOW_SCALE_SHIFT >> r_tmp2 += KASAN_SHADOW_OFFSET ; shadow address >> r_tmp3 = *(u8 *)(r_tmp2 + 0) ; BPF_LDX | BPF_B (load shadow byte) >> if r_tmp3 != 0 goto +2 ; BPF_JNE | PC+2 >> dst = *(u64 *)(src + off) ; original access (fast path) >> goto +1 ; skip slowpath >> call __asan_report_load8 ; BPF kfunc >> dst = *(u64 *)(src + off) ; retry the access after report (non-fatal) >> >> A sort of inline kasan directly in BPF. >> >> There are plenty of issues with it: instruction limit, exposing asan >> API as kfuncs, etc. On the flip side we get cross-arch support out of >> the box with no or mininal JIT changes. >> >> Honestly I'm not excited about this approach, but curious if anyone >> thought about this, or maybe it was already discussed? > > We discussed this. > It won't work because we don't have that many temp registers for once > and second it has to preserve all (both callee and caller saved regs). > This is arch specific. This "fast path" was indeed part of my initial proposal. For the record: https://lore.kernel.org/bpf/DG7UG112AVBC.JKYISDTAM30T@bootlin.com/ in the "Basic instrumentation" block. -- Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
© 2016 - 2026 Red Hat, Inc.