[PATCH 0/7] Usermode Indirect Branch Tracking

Richard Patel posted 7 patches 1 week ago
arch/x86/Kconfig                            |  17 ++
arch/x86/entry/vdso/common/Makefile.include |   3 +-
arch/x86/include/asm/cpufeatures.h          |   1 +
arch/x86/include/asm/ibt.h                  |  16 ++
arch/x86/include/asm/processor.h            |   5 +
arch/x86/include/uapi/asm/ucontext.h        |   5 +
arch/x86/kernel/Makefile                    |   1 +
arch/x86/kernel/cet.c                       |   3 +-
arch/x86/kernel/cpu/common.c                |  14 +-
arch/x86/kernel/ibt.c                       | 175 ++++++++++++++++++++
arch/x86/kernel/process_64.c                |   2 +
arch/x86/kernel/shstk.c                     |  12 +-
arch/x86/kernel/signal_32.c                 |   5 +
arch/x86/kernel/signal_64.c                 |   6 +
tools/arch/x86/include/asm/cpufeatures.h    |   1 +
tools/testing/selftests/x86/Makefile        |   5 +-
tools/testing/selftests/x86/user_ibt.c      | 157 ++++++++++++++++++
17 files changed, 420 insertions(+), 8 deletions(-)
create mode 100644 arch/x86/kernel/ibt.c
create mode 100644 tools/testing/selftests/x86/user_ibt.c
[PATCH 0/7] Usermode Indirect Branch Tracking
Posted by Richard Patel 1 week ago
I was quite surprised that the Linux kernel still does not allow
userspace to enable x86 IBT (indirect jmp/call integrity).

Compilers and linkers have been emitting 'endbr64' IBT markers and ELF
support notes for a while now.

The hard work was done years ago by Intel:
https://lore.kernel.org/all/20210830182221.3535-1-yu-cheng.yu@intel.com/

In summary, usermode IBT requires 3 things:
1. Set the CET_ENDBR_EN bit in MSR_IA32_U_CET for each IBT-enabled thread
   (PATCH 2,5)
2. Back up the WAIT_FOR_ENDBR bit across signal handling (PATCH 3,4)
3. Provide a way for usermode to enable it (PATCH 5)

This builds on top of Yu Cheng's work, with some adaptations:
- FRED support
- Implemented the existing prctl(PR_CFI_*) API
- Removed ELF parsing (can be added later)

Unresolved questions:
- Is there a cleaner way to do the WAIT_FOR_ENDBR XSAVE fallback?
- What to do about 'notrack jmp *rax'?
  I leave CET_NO_TRACK_EN enabled, which weakens IBT, by enabling a jump
  prefix that skips the ENDBR check. GCC emits it for jump tables
  (-mcet-switch). We could introduce a PR_CFI_IBT_STRICT bit.
- There's some obvious overlap with arch_prctl(ARCH_SHSTK_*).
  Happy to use that API instead.

Richard Patel (7):
  x86: add userspace IBT config option
  x86: shstk: don't clobber IBT bits in U_CET MSR
  x86: signal handler support for IBT
  x86: ban 32-bit sigreturn when user IBT enabled
  x86: expose user IBT via PR_CFI_BRANCH_LANDING_PADS
  x86/entry/vdso: build with IBT support
  selftests/x86: test usermode IBT

 arch/x86/Kconfig                            |  17 ++
 arch/x86/entry/vdso/common/Makefile.include |   3 +-
 arch/x86/include/asm/cpufeatures.h          |   1 +
 arch/x86/include/asm/ibt.h                  |  16 ++
 arch/x86/include/asm/processor.h            |   5 +
 arch/x86/include/uapi/asm/ucontext.h        |   5 +
 arch/x86/kernel/Makefile                    |   1 +
 arch/x86/kernel/cet.c                       |   3 +-
 arch/x86/kernel/cpu/common.c                |  14 +-
 arch/x86/kernel/ibt.c                       | 175 ++++++++++++++++++++
 arch/x86/kernel/process_64.c                |   2 +
 arch/x86/kernel/shstk.c                     |  12 +-
 arch/x86/kernel/signal_32.c                 |   5 +
 arch/x86/kernel/signal_64.c                 |   6 +
 tools/arch/x86/include/asm/cpufeatures.h    |   1 +
 tools/testing/selftests/x86/Makefile        |   5 +-
 tools/testing/selftests/x86/user_ibt.c      | 157 ++++++++++++++++++
 17 files changed, 420 insertions(+), 8 deletions(-)
 create mode 100644 arch/x86/kernel/ibt.c
 create mode 100644 tools/testing/selftests/x86/user_ibt.c

--
2.47.3
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by Peter Zijlstra 6 days, 22 hours ago
On Sun, May 17, 2026 at 01:30:17PM -0500, Richard Patel wrote:
> I was quite surprised that the Linux kernel still does not allow
> userspace to enable x86 IBT (indirect jmp/call integrity).
> 
> Compilers and linkers have been emitting 'endbr64' IBT markers and ELF
> support notes for a while now.
> 
> The hard work was done years ago by Intel:
> https://lore.kernel.org/all/20210830182221.3535-1-yu-cheng.yu@intel.com/
> 
> In summary, usermode IBT requires 3 things:
> 1. Set the CET_ENDBR_EN bit in MSR_IA32_U_CET for each IBT-enabled thread
>    (PATCH 2,5)
> 2. Back up the WAIT_FOR_ENDBR bit across signal handling (PATCH 3,4)
> 3. Provide a way for usermode to enable it (PATCH 5)
> 
> This builds on top of Yu Cheng's work, with some adaptations:
> - FRED support
> - Implemented the existing prctl(PR_CFI_*) API
> - Removed ELF parsing (can be added later)
> 
> Unresolved questions:
> - Is there a cleaner way to do the WAIT_FOR_ENDBR XSAVE fallback?
> - What to do about 'notrack jmp *rax'?
>   I leave CET_NO_TRACK_EN enabled, which weakens IBT, by enabling a jump
>   prefix that skips the ENDBR check. GCC emits it for jump tables
>   (-mcet-switch). We could introduce a PR_CFI_IBT_STRICT bit.
> - There's some obvious overlap with arch_prctl(ARCH_SHSTK_*).
>   Happy to use that API instead.

Right, I think the problem was mostly one of ABI. Various distributions
shipped with 'early' patches using an ABI that hadn't seen upstream
review much, and when it hit we went uhhh, no!

And then glibc and distros were like, but uhmm, it has to be.

So here we are :-(

Anyway, the most contentious part was the whole backwards compat bitmap
crap. When the dynamic linker composes a process of parts that support
IBT and parts that do not, you get to deal with fallout.

The IBT spec has this horrid bitmap thing to try and deal with this, and
those early patches exposed that piece of shit to userspace. Then later
patches (suggested by me) used the ARM64/BTI approach of using PROT_BTI.
We'd use a (software) page-table bit, and upon #CP consult that to see
if we should eat the trap or produce a warn/signal whatever.

I think we were near something workable there when Rick got pulled from
this and put onto something more 'important' and things just haven't
moved ever since.

Anyway, glad to see someone has time to poke at this.
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by Richard Patel 6 days, 13 hours ago
On Mon, May 18, 2026 at 09:36:16AM +0200, Peter Zijlstra wrote:
> Anyway, the most contentious part was the whole backwards compat bitmap
> crap. When the dynamic linker composes a process of parts that support
> IBT and parts that do not, you get to deal with fallout.

Is it acceptable to do all-or-nothing IBT first? And then do a second
round of patches with legacy support?

Until then, ld.so could:
- start with IBT, disable it upon loading incompatible DSO
- allow users to manually lock IBT

I thought this weak form of IBT is better than nothing at all, if
there's a risk that legacy support derails things.

Btw, apparently OpenBSD enforces kernel+user IBT. I think the end goal
is a user_ibt=force command-line param that locks IBT for all processes
on startup.

> The IBT spec has this horrid bitmap thing to try and deal with this, and
> those early patches exposed that piece of shit to userspace. Then later
> patches (suggested by me) used the ARM64/BTI approach of using PROT_BTI.
> We'd use a (software) page-table bit, and upon #CP consult that to see
> if we should eat the trap or produce a warn/signal whatever.

Nice, I'm happy to revive/rebase/test any of this if there's interest.

> I think we were near something workable there when Rick got pulled from
> this and put onto something more 'important' and things just haven't
> moved ever since.
> 
> Anyway, glad to see someone has time to poke at this.

Happy to spend whatever time is needed to land IBT. I'm very glad the
first reaction wasn't "absolutely no way" :-)

Thank you,
-Richard
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by Peter Zijlstra 6 days, 10 hours ago
+ Florian

On Mon, May 18, 2026 at 04:25:40PM +0000, Richard Patel wrote:
> On Mon, May 18, 2026 at 09:36:16AM +0200, Peter Zijlstra wrote:
> > Anyway, the most contentious part was the whole backwards compat bitmap
> > crap. When the dynamic linker composes a process of parts that support
> > IBT and parts that do not, you get to deal with fallout.
> 
> Is it acceptable to do all-or-nothing IBT first? And then do a second
> round of patches with legacy support?
> 
> Until then, ld.so could:
> - start with IBT, disable it upon loading incompatible DSO
> - allow users to manually lock IBT
> 
> I thought this weak form of IBT is better than nothing at all, if
> there's a risk that legacy support derails things.

That wholly depends on how much churn glibc people are willing to put up
with. Added Florian to answer some of that.

> Btw, apparently OpenBSD enforces kernel+user IBT. I think the end goal
> is a user_ibt=force command-line param that locks IBT for all processes
> on startup.
> 
> > The IBT spec has this horrid bitmap thing to try and deal with this, and
> > those early patches exposed that piece of shit to userspace. Then later
> > patches (suggested by me) used the ARM64/BTI approach of using PROT_BTI.
> > We'd use a (software) page-table bit, and upon #CP consult that to see
> > if we should eat the trap or produce a warn/signal whatever.
> 
> Nice, I'm happy to revive/rebase/test any of this if there's interest.
> 
> > I think we were near something workable there when Rick got pulled from
> > this and put onto something more 'important' and things just haven't
> > moved ever since.
> > 
> > Anyway, glad to see someone has time to poke at this.
> 
> Happy to spend whatever time is needed to land IBT. I'm very glad the
> first reaction wasn't "absolutely no way" :-)

Yeah, its just something that fell between the cracks :/
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by David Laight 5 days, 20 hours ago
On Sun, 17 May 2026 13:30:17 -0500
Richard Patel <ripatel@wii.dev> wrote:

> I was quite surprised that the Linux kernel still does not allow
> userspace to enable x86 IBT (indirect jmp/call integrity).
> 
> Compilers and linkers have been emitting 'endbr64' IBT markers and ELF
> support notes for a while now.
> 
> The hard work was done years ago by Intel:
> https://lore.kernel.org/all/20210830182221.3535-1-yu-cheng.yu@intel.com/
> 
> In summary, usermode IBT requires 3 things:
> 1. Set the CET_ENDBR_EN bit in MSR_IA32_U_CET for each IBT-enabled thread
>    (PATCH 2,5)
> 2. Back up the WAIT_FOR_ENDBR bit across signal handling (PATCH 3,4)
> 3. Provide a way for usermode to enable it (PATCH 5)
> 
> This builds on top of Yu Cheng's work, with some adaptations:
> - FRED support
> - Implemented the existing prctl(PR_CFI_*) API
> - Removed ELF parsing (can be added later)
> 
> Unresolved questions:
> - Is there a cleaner way to do the WAIT_FOR_ENDBR XSAVE fallback?
> - What to do about 'notrack jmp *rax'?
>   I leave CET_NO_TRACK_EN enabled, which weakens IBT, by enabling a jump
>   prefix that skips the ENDBR check. GCC emits it for jump tables
>   (-mcet-switch). We could introduce a PR_CFI_IBT_STRICT bit.

Isn't using 'notrack jmp *reg' for jump tables actually more secure?
If an attacker can write code it doesn't matter.
The jump table in is RO memory so can't be written.
But if there are ENDBR on all the jump table targets they become
possibly useful code addresses to arrange to write into some RW
function pointer table - which might be useful.

-- David

> - There's some obvious overlap with arch_prctl(ARCH_SHSTK_*).
>   Happy to use that API instead.
...
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by Richard Patel 5 days, 16 hours ago
On Tue, May 19, 2026 at 10:33:45AM +0100, David Laight wrote:
> Isn't using 'notrack jmp *reg' for jump tables actually more secure?
> If an attacker can write code it doesn't matter.
> The jump table in is RO memory so can't be written.
> But if there are ENDBR on all the jump table targets they become
> possibly useful code addresses to arrange to write into some RW
> function pointer table - which might be useful.

You're right. I was worried about an invalid jump table index at first.
Clang 22 happily optimizes away jump table index bounds checks. GCC 16
seems to be more careful. We should probably patch LLVM to never
optimize it away, e.g.:

	// funny.c
	// clang -c -fcf-protection=branch -O2 -o funny.o funny.c
	// objdump -d funny.o -M intel
	int t0(void), t1(void), t2(void), t3(void);
	int funny(unsigned long target) {
		__builtin_assume(target < 4);
		switch (target) {
		case 0: return t0();
		case 1: return t1();
		case 2: return t2();
		case 3: return t3();
		}
	}

	// Clang 22
	0000000000000000 <funny>:
	       0: f3 0f 1e fa                  endbr64
	       4: 55                           push rbp
	       5: 48 89 e5                     mov rbp, rsp
	       8: 3e ff 24 fd 00 00 00 00      notrack jmp qword ptr [rdi*8+0x0] // vulnerable
	      10: 5d                           pop rbp
	      11: e9 00 00 00 00               jmp 0x16 <funny+0x16>
	      16: 5d                           pop rbp
	      17: e9 00 00 00 00               jmp 0x1c <funny+0x1c>
	      1c: 5d                           pop rbp
	      1d: e9 00 00 00 00               jmp 0x22 <funny+0x22>
	      22: 5d                           pop rbp
	      23: e9 00 00 00 00               jmp 0x28 <funny+0x28>

-Richard
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by David Laight 5 days, 16 hours ago
On Tue, 19 May 2026 13:14:33 +0000
Richard Patel <ripatel@wii.dev> wrote:

> On Tue, May 19, 2026 at 10:33:45AM +0100, David Laight wrote:
> > Isn't using 'notrack jmp *reg' for jump tables actually more secure?
> > If an attacker can write code it doesn't matter.
> > The jump table in is RO memory so can't be written.
> > But if there are ENDBR on all the jump table targets they become
> > possibly useful code addresses to arrange to write into some RW
> > function pointer table - which might be useful.  
> 
> You're right. I was worried about an invalid jump table index at first.
> Clang 22 happily optimizes away jump table index bounds checks. GCC 16
> seems to be more careful. We should probably patch LLVM to never
> optimize it away, e.g.:
> 
> 	// funny.c
> 	// clang -c -fcf-protection=branch -O2 -o funny.o funny.c
> 	// objdump -d funny.o -M intel
> 	int t0(void), t1(void), t2(void), t3(void);
> 	int funny(unsigned long target) {
> 		__builtin_assume(target < 4);

If you use __builtin_assume() you get to clear up the mess.

I don't know if userspace ever cares about speculative array access.
If it does you need one of the mitigration - eg using cmp+cmov
to generate a jump table index that references the 'default'.

-- David

> 		switch (target) {
> 		case 0: return t0();
> 		case 1: return t1();
> 		case 2: return t2();
> 		case 3: return t3();
> 		}
> 	}
> 
> 	// Clang 22
> 	0000000000000000 <funny>:
> 	       0: f3 0f 1e fa                  endbr64
> 	       4: 55                           push rbp
> 	       5: 48 89 e5                     mov rbp, rsp
> 	       8: 3e ff 24 fd 00 00 00 00      notrack jmp qword ptr [rdi*8+0x0] // vulnerable
> 	      10: 5d                           pop rbp
> 	      11: e9 00 00 00 00               jmp 0x16 <funny+0x16>
> 	      16: 5d                           pop rbp
> 	      17: e9 00 00 00 00               jmp 0x1c <funny+0x1c>
> 	      1c: 5d                           pop rbp
> 	      1d: e9 00 00 00 00               jmp 0x22 <funny+0x22>
> 	      22: 5d                           pop rbp
> 	      23: e9 00 00 00 00               jmp 0x28 <funny+0x28>
> 
> -Richard
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by Richard Patel 5 days, 15 hours ago
On Tue, May 19, 2026 at 02:28:08PM +0100, David Laight wrote:
> On Tue, 19 May 2026 13:14:33 +0000
> Richard Patel <ripatel@wii.dev> wrote:
> 
> > On Tue, May 19, 2026 at 10:33:45AM +0100, David Laight wrote:
> > > Isn't using 'notrack jmp *reg' for jump tables actually more secure?
> > > If an attacker can write code it doesn't matter.
> > > The jump table in is RO memory so can't be written.
> > > But if there are ENDBR on all the jump table targets they become
> > > possibly useful code addresses to arrange to write into some RW
> > > function pointer table - which might be useful.  
> > 
> > You're right. I was worried about an invalid jump table index at first.
> > Clang 22 happily optimizes away jump table index bounds checks. GCC 16
> > seems to be more careful. We should probably patch LLVM to never
> > optimize it away, e.g.:
> > 
> > 	// funny.c
> > 	// clang -c -fcf-protection=branch -O2 -o funny.o funny.c
> > 	// objdump -d funny.o -M intel
> > 	int t0(void), t1(void), t2(void), t3(void);
> > 	int funny(unsigned long target) {
> > 		__builtin_assume(target < 4);
> 
> If you use __builtin_assume() you get to clear up the mess.

I'm pretty sure you'd get the same result with cross-function
optimization across a bunch of static functions or LTO. Compiler goes
"oh, this internal function is only reachable from these 3 callers in
the same unit, which all already bound their input params. Guess I will
skip the bounds check".

It is a compiler bug that Clang is at all able to generate unbounded
'notrack jmp' with -fcf-protection=branch, it blows a gap in IBT.

Anyways, I don't think we need kernel support for banning notrack in
userland? There is no ABI (GNU note) standard for 'notrack-free'
binaries AFAIK, and as you point out notrack is a secure way to do
jump tables (if done properly).

> I don't know if userspace ever cares about speculative array access.
> If it does you need one of the mitigration - eg using cmp+cmov
> to generate a jump table index that references the 'default'.

Intel docs say that "CET-IBT limits speculative execution at indirect
branch targets that do not start with ENDBRANCH", with heavy emphasis
on "limits" not "prevents" ... Is it too unreliable in practice?

https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/branch-history-injection.html#inpage-nav-4-3

-- Richard
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by Peter Zijlstra 5 days, 15 hours ago
On Tue, May 19, 2026 at 02:18:04PM +0000, Richard Patel wrote:

> > I don't know if userspace ever cares about speculative array access.
> > If it does you need one of the mitigration - eg using cmp+cmov
> > to generate a jump table index that references the 'default'.
> 
> Intel docs say that "CET-IBT limits speculative execution at indirect
> branch targets that do not start with ENDBRANCH", with heavy emphasis
> on "limits" not "prevents" ... Is it too unreliable in practice?
> 
> https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/technical-documentation/branch-history-injection.html#inpage-nav-4-3

You might find the commit log of: d8122c428076 ("x86/ibt: Implement
FineIBT-BHI mitigation") instructive.
Re: [PATCH 0/7] Usermode Indirect Branch Tracking
Posted by Peter Zijlstra 5 days, 20 hours ago
On Tue, May 19, 2026 at 10:33:45AM +0100, David Laight wrote:
> On Sun, 17 May 2026 13:30:17 -0500
> Richard Patel <ripatel@wii.dev> wrote:
> 
> > I was quite surprised that the Linux kernel still does not allow
> > userspace to enable x86 IBT (indirect jmp/call integrity).
> > 
> > Compilers and linkers have been emitting 'endbr64' IBT markers and ELF
> > support notes for a while now.
> > 
> > The hard work was done years ago by Intel:
> > https://lore.kernel.org/all/20210830182221.3535-1-yu-cheng.yu@intel.com/
> > 
> > In summary, usermode IBT requires 3 things:
> > 1. Set the CET_ENDBR_EN bit in MSR_IA32_U_CET for each IBT-enabled thread
> >    (PATCH 2,5)
> > 2. Back up the WAIT_FOR_ENDBR bit across signal handling (PATCH 3,4)
> > 3. Provide a way for usermode to enable it (PATCH 5)
> > 
> > This builds on top of Yu Cheng's work, with some adaptations:
> > - FRED support
> > - Implemented the existing prctl(PR_CFI_*) API
> > - Removed ELF parsing (can be added later)
> > 
> > Unresolved questions:
> > - Is there a cleaner way to do the WAIT_FOR_ENDBR XSAVE fallback?
> > - What to do about 'notrack jmp *rax'?
> >   I leave CET_NO_TRACK_EN enabled, which weakens IBT, by enabling a jump
> >   prefix that skips the ENDBR check. GCC emits it for jump tables
> >   (-mcet-switch). We could introduce a PR_CFI_IBT_STRICT bit.
> 
> Isn't using 'notrack jmp *reg' for jump tables actually more secure?
> If an attacker can write code it doesn't matter.
> The jump table in is RO memory so can't be written.
> But if there are ENDBR on all the jump table targets they become
> possibly useful code addresses to arrange to write into some RW
> function pointer table - which might be useful.

One of the many reasons the kernel is built without jump-tables.