[RFC PATCH 0/5] rseq: add support for RSEQ operations

odion@efficios.com posted 5 patches 4 weeks, 1 day ago
include/linux/rseq.h                    |  10 +-
include/linux/rseq_entry.h              | 157 ++++++++++++++++-
include/linux/rseq_types.h              |   9 +
include/uapi/linux/prctl.h              |  12 ++
include/uapi/linux/rseq.h               | 104 ++++++++++-
kernel/rseq.c                           | 220 ++++++++++++++++++++++--
kernel/sys.c                            |   5 +
tools/testing/selftests/rseq/.gitignore |   4 +-
tools/testing/selftests/rseq/Makefile   |   4 +-
tools/testing/selftests/rseq/rseq-abi.h | 146 +++++++++++++---
tools/testing/selftests/rseq/rseq.c     |  23 ++-
tools/testing/selftests/rseq/rseq.h     |  75 ++++++++
12 files changed, 713 insertions(+), 56 deletions(-)
[RFC PATCH 0/5] rseq: add support for RSEQ operations
Posted by odion@efficios.com 4 weeks, 1 day ago
From: Olivier Dion <odion@efficios.com>

Hi,

This series introduces RSEQ operations: a per-thread list of operations the
kernel applies, on behalf of a ask, when returning to user space.

The main motivation is to help TCMalloc migrate from RSEQ v1 to RSEQ v2 and use
the RSEQ area registered by glibc. TCMalloc currently relies on RSEQ v1
resetting the cpu_id field to invalidate a per-CPU pointer cached in TLS. This
requires applications to disable glibc's implicit RSEQ registration and prevents
sharing the glibc RSEQ area.

RSEQ operations provide an opt-in replacement for that invalidation
mechanism. User space registers operation nodes with prctl; the kernel keeps the
nodes in a per-thread circular list and applies them on return to user
space. Only tasks with registered operations incur the additional RSEQ exit-path
work.

The first patch adds the UAPI. The next three patches add the task state,
operation execution, and prctl registration support. The final patch adds
selftests covering normal operation and malformed-list and invalid-memory cases.

See also <https://lore.kernel.org/lkml/20260428221058.149538293@kernel.org>.

Thanks,
old

Olivier Dion (5):
  rseq: uapi: add rseq operation definitions
  rseq: add per-task rseq operation state
  rseq: apply operations on exit to user space
  rseq: register and unregister operations via prctl
  selftests/rseq: add coverage for rseq operations

 include/linux/rseq.h                    |  10 +-
 include/linux/rseq_entry.h              | 157 ++++++++++++++++-
 include/linux/rseq_types.h              |   9 +
 include/uapi/linux/prctl.h              |  12 ++
 include/uapi/linux/rseq.h               | 104 ++++++++++-
 kernel/rseq.c                           | 220 ++++++++++++++++++++++--
 kernel/sys.c                            |   5 +
 tools/testing/selftests/rseq/.gitignore |   4 +-
 tools/testing/selftests/rseq/Makefile   |   4 +-
 tools/testing/selftests/rseq/rseq-abi.h | 146 +++++++++++++---
 tools/testing/selftests/rseq/rseq.c     |  23 ++-
 tools/testing/selftests/rseq/rseq.h     |  75 ++++++++
 12 files changed, 713 insertions(+), 56 deletions(-)

-- 
2.54.0
Re: [RFC PATCH 0/5] rseq: add support for RSEQ operations
Posted by Thomas Gleixner 2 weeks, 4 days ago
Olivier!

On Fri, Aug 28 2026 at 11:33, odion@efficios.com wrote:
> This series introduces RSEQ operations: a per-thread list of operations the
> kernel applies, on behalf of a ask, when returning to user space.
>
> The main motivation is to help TCMalloc migrate from RSEQ v1 to RSEQ v2 and use
> the RSEQ area registered by glibc. TCMalloc currently relies on RSEQ v1
> resetting the cpu_id field to invalidate a per-CPU pointer cached in TLS. This
> requires applications to disable glibc's implicit RSEQ registration and prevents
> sharing the glibc RSEQ area.
>
> RSEQ operations provide an opt-in replacement for that invalidation
> mechanism. User space registers operation nodes with prctl; the kernel keeps the
> nodes in a per-thread circular list and applies them on return to user
> space. Only tasks with registered operations incur the additional RSEQ exit-path
> work.

TBH, to me this sounds like a horrible idea. It's yet another
"interpreter" for a very limited use case and a lot of complexity.

I think this can be done in user space with some help of the kernel of
course. If user space has registered an entry point for this
functionality then the kernel can emulate a call to that on return to
user space:

      if (tsk->rseq.needs_fixup) {
                // modifies regs->sp
		create_callframe_on_user stack(regs);
                regs->ip = tsk->rseq.fixup_ip;
      }
      ret_to_user()

The user space fixup does:

fixup_ip()
      ....
      user_rseq.fixup_in_progress = false;
      restore_and_return() // Returns to the original IP

A reasonable limitation for the fixup function should be a strict "no
syscalls and no floating point within the fixup" rule. No floating point
avoids the whole sigframe disaster.

Of course the above is way too simple to be true :) It works except when
it nests. But that's solvable too:

      if (tsk->rseq.needs_fixup) {
                // modifies regs->sp
		create_callframe_on_user stack(regs);
                regs->ip = tsk->rseq.fixup_ip;
                // Save the callframe SP
                tsk->rseq.fixup_sp = regs->sp;
      }

When return to user observes user_rseq.fixup_in_progress then it can
mangle regs before doing anything else:

        regs->ip = tsk->rseq.fixup_abort_ip;
        regs->sp = tsk->rseq.fixup_sp;
        user_rseq.fixup_in_progress = false;
        tsk->rseq.needs_fixup = true;
    
Which rewinds the stack to the callframe and makes the interrupted fixup
continue at the fixup_abort_ip which just restores registers from the
callframe and returns to the original IP.

Then the flow becomes in case of a trivial interrupt which just returns:

    fixup_ip()
      -> interrupt
                                ...
                                if (user_rseq.fixup_in_progress)
                                   rewind_fixup(regs);
                                 ...
                                if (needs_fixup)
                                   setup_fixup(regs);
                                return to user;
       fixup_ip()
          ...
          user_rseq.fixup_in_progress = false
          restore_and_return()
   fixup_abort_ip:
      restore_and_return()      // Returns to orig_ip

In case of a signal delivery:

    fixup_ip()
      -> interrupt
                                ...
                                if (user_rseq.fixup_in_progress)
                                   rewind_fixup(regs);

                                   deliver_signal()
                                     setup_sigframe()
                                  ...
                                if (needs_fixup)
                                   setup_fixup(regs);
                                return to user;
       fixup_ip()
          ...
          user_rseq.fixup_in_progress = false
          restore_and_return()
       signal_handler()
          sys_sigreturn()
                                restore(regs)
                                return to user;

   fixup_abort_ip:
      restore_and_return()      // Returns to orig_ip

There are obviously a ton of details to take care of (/me mumbles shadow
stacks and RSEQ CS interaction), but the general principle should just
work. Emphasis on should and I'm so NOT going to hack that up. :)

I've played around with something similar pre RSEQ, but all I could
still find are the notes I took back then, which I duly transcribed into
todays RSEQ world. The patches themself would be utterly useless anyway
as all parts which need to be touched have been through the mincer at
least once.

The fixup address and the fixup abort address should probably not be
stored in user rseq memory. For an initial POC they just can be
registered through sys_rseq() which also enables the fixup mechanics,
i.e. the unconditional set of tsk->rseq.needs_fixup in schedule(), which
should be completely independent of the rseq.sched_switch and
rseq.ids_changed bits and not change the existing functionality and
semantics like the current POC does. Keep it separate.

Once the POC dust has settled the registration interface should change
for the final implementation because otherwise this would create the
very same problem of who owns it again.

The final solution should provide a trivial dispatch mechanism in the
VDSO and the RSEQ user area should be expanded to provide storage for
registration. The syscall registration would change to registering a lib
specific callback/data pair and the kernel would reorganize the
storage. Whether that's a linked list or a size limited array in the
RSEQ user area does not matter. As this is thread local there is no
concurrency and the syscall can rearange that storage completely
undisturbed. Libraries can manage their own thread local marker which is
checked in the lib specific callback to decide whether they want to run
or not.

Using a VDSO dispatcher and expanding the RSEQ storage for that should
just work out of the box with any libc which supports RSEQ_V2 and would
therefore not create any additional libc dependencies for other
library developers.

The dispatcher entry needs to be trivial ASM for the restore/return

fixup_ip:
        // setup_fixup() in the kernel stored the TLS user RSEQ address in
        // RDI or whatever an architecture uses for the first argument
	CALL	fixup_c
fixup_ip_abort:
	ASM_RESTORE_REGS
        RET

The C part should be trivial too:

fixup_c(struct rseq *rseq) 
{
        for_each_fixup(fn, data, rseq)
        	fn(data);
}

or something along these lines.

Thoughts?

Thanks,

        tglx
Re: [RFC PATCH 0/5] rseq: add support for RSEQ operations
Posted by Olivier Dion 2 weeks, 4 days ago
On Tue, 08 Sep 2026, Thomas Gleixner <tglx@kernel.org> wrote:
> Olivier!
>
> On Fri, Aug 28 2026 at 11:33, odion@efficios.com wrote:
>> This series introduces RSEQ operations: a per-thread list of operations the
>> kernel applies, on behalf of a ask, when returning to user space.
>>
>> The main motivation is to help TCMalloc migrate from RSEQ v1 to RSEQ v2 and use
>> the RSEQ area registered by glibc. TCMalloc currently relies on RSEQ v1
>> resetting the cpu_id field to invalidate a per-CPU pointer cached in TLS. This
>> requires applications to disable glibc's implicit RSEQ registration and prevents
>> sharing the glibc RSEQ area.
>>
>> RSEQ operations provide an opt-in replacement for that invalidation
>> mechanism. User space registers operation nodes with prctl; the kernel keeps the
>> nodes in a per-thread circular list and applies them on return to user
>> space. Only tasks with registered operations incur the additional RSEQ exit-path
>> work.
>
> TBH, to me this sounds like a horrible idea. It's yet another
> "interpreter" for a very limited use case and a lot of complexity.

I post this RFC before going on vacation.  I tend to very much agree
with you reply overall, after revisiting the problem and potential
issues this solution could introduce.

> I think this can be done in user space with some help of the kernel of
> course. If user space has registered an entry point for this
> functionality then the kernel can emulate a call to that on return to
> user space:
>
>       if (tsk->rseq.needs_fixup) {
>                 // modifies regs->sp
> 		create_callframe_on_user stack(regs);
>                 regs->ip = tsk->rseq.fixup_ip;
>       }
>       ret_to_user()
>
> The user space fixup does:
>
> fixup_ip()
>       ....
>       user_rseq.fixup_in_progress = false;
>       restore_and_return() // Returns to the original IP

Yes.  This is what I had in mind (see reply to Florian).

> A reasonable limitation for the fixup function should be a strict "no
> syscalls and no floating point within the fixup" rule. No floating point
> avoids the whole sigframe disaster.

I suppose that no floating point also mean no xsave performance
trashing, which is obviously something we want to avoid.  Is that what
you mean by sigframe disaster?  Also, I am not sure how we can enforce
this no syscall/floating point policies other than asking users to be
good citizen.

> Of course the above is way too simple to be true :) It works except when
> it nests. But that's solvable too:
>
>       if (tsk->rseq.needs_fixup) {
>                 // modifies regs->sp
> 		create_callframe_on_user stack(regs);
>                 regs->ip = tsk->rseq.fixup_ip;
>                 // Save the callframe SP
>                 tsk->rseq.fixup_sp = regs->sp;
>       }
>
> When return to user observes user_rseq.fixup_in_progress then it can
> mangle regs before doing anything else:
>
>         regs->ip = tsk->rseq.fixup_abort_ip;
>         regs->sp = tsk->rseq.fixup_sp;
>         user_rseq.fixup_in_progress = false;
>         tsk->rseq.needs_fixup = true;
>
> Which rewinds the stack to the callframe and makes the interrupted fixup
> continue at the fixup_abort_ip which just restores registers from the
> callframe and returns to the original IP.

So the fixup handler would have an abort label, akin to RSEQ region.
But then there will be no guarantee that the fixup operations succeed?
Or did I completely misunderstood?

[...]

> There are obviously a ton of details to take care of (/me mumbles shadow
> stacks and RSEQ CS interaction), but the general principle should just
> work. Emphasis on should and I'm so NOT going to hack that up. :)

I could certainly make a POC out of this for x86.  It could handle
red-zone and shadow-stack to start with.  I don't know if there are
other architecture-specific quirks that need to be aware of, given I am
not familiar enough with architectures outside of x86.

[...]

> Once the POC dust has settled the registration interface should change
> for the final implementation because otherwise this would create the
> very same problem of who owns it again.

Right we don't want to play in that movie again :-)

> The final solution should provide a trivial dispatch mechanism in the
> VDSO and the RSEQ user area should be expanded to provide storage for
> registration. The syscall registration would change to registering a lib
> specific callback/data pair and the kernel would reorganize the
> storage. Whether that's a linked list or a size limited array in the
> RSEQ user area does not matter. As this is thread local there is no
> concurrency and the syscall can rearange that storage completely
> undisturbed. Libraries can manage their own thread local marker which is
> checked in the lib specific callback to decide whether they want to run
> or not.
>
> Using a VDSO dispatcher and expanding the RSEQ storage for that should
> just work out of the box with any libc which supports RSEQ_V2 and would
> therefore not create any additional libc dependencies for other
> library developers.
>
> The dispatcher entry needs to be trivial ASM for the restore/return
>
> fixup_ip:
>         // setup_fixup() in the kernel stored the TLS user RSEQ address in
>         // RDI or whatever an architecture uses for the first argument
> 	CALL	fixup_c
> fixup_ip_abort:
> 	ASM_RESTORE_REGS
>         RET
>
> The C part should be trivial too:
>
> fixup_c(struct rseq *rseq) 
> {
>         for_each_fixup(fn, data, rseq)
>         	fn(data);
> }
>
> or something along these lines.

Thanks for the insight.  I think this will certainly help me in making
the POC.

One aspect that I have not think of for now is allowing fast
registration/unregistration.  This RFC uses a syscall for registration.
My original intent for this was to do a edge trigger detection of the
first/last registration/unregistration in the syscall and set/unset a
flag into the RSEQ state of the thread task.  That way, we don't need to
touch another cache line to know if the thread registered something.
This is obviously incompatible with short-live registrations.  However,
I did not come with a case for it yet, so perhaps I am over-thinking
this.

Thanks,
Olivier
-- 
Olivier Dion
EfficiOS Inc.
https://www.efficios.com
Re: [RFC PATCH 0/5] rseq: add support for RSEQ operations
Posted by Thomas Gleixner 2 weeks, 4 days ago
On Tue, Sep 08 2026 at 13:38, Olivier Dion wrote:
> On Tue, 08 Sep 2026, Thomas Gleixner <tglx@kernel.org> wrote:
>> fixup_ip()
>>       ....
>>       user_rseq.fixup_in_progress = false;
>>       restore_and_return() // Returns to the original IP
>
> Yes.  This is what I had in mind (see reply to Florian).
>
>> A reasonable limitation for the fixup function should be a strict "no
>> syscalls and no floating point within the fixup" rule. No floating point
>> avoids the whole sigframe disaster.
>
> I suppose that no floating point also mean no xsave performance
> trashing, which is obviously something we want to avoid.  Is that what
> you mean by sigframe disaster?  Also, I am not sure how we can enforce
> this no syscall/floating point policies other than asking users to be
> good citizen.

The sigframe disaster is that sigaltstack has been insufficient for the
ever growing XSTATE to save (its even uncompressed XSTATE). Also these
fixups really should do a few selective stores and not huge computations
or wipe out a GB of memory. So avoiding the XSAVE/XRSTOR overhead
completely is certainly a benefit.

>> When return to user observes user_rseq.fixup_in_progress then it can
>> mangle regs before doing anything else:
>>
>>         regs->ip = tsk->rseq.fixup_abort_ip;
>>         regs->sp = tsk->rseq.fixup_sp;
>>         user_rseq.fixup_in_progress = false;
>>         tsk->rseq.needs_fixup = true;
>>
>> Which rewinds the stack to the callframe and makes the interrupted fixup
>> continue at the fixup_abort_ip which just restores registers from the
>> callframe and returns to the original IP.
>
> So the fixup handler would have an abort label, akin to RSEQ region.
> But then there will be no guarantee that the fixup operations succeed?
> Or did I completely misunderstood?

They are only aborted when there is nesting, so the nesting context
starts over and redoes them. Once the nesting context returns to the
first fixup (abort IP) the state is correct and then the first fixup
returns to the original return IP.

That abort on nesting avoids the following issue:

user_function()
  ...
  interrupt(#1)
     schedule()
       MMCID changes
       fixup requested
   ...
   setup_callframe(....)
   return to user

 fixup_ip:
    fixup_c(data)
    ...
     interrupt(#2)
       schedule()
         MMCID changes
         fixup requested

     fixup_ip:
        fixup_c(data)
         data::mmcid = user_rseq::mmcid
        restore_and_return()

    data::mmcid = user_rseq::mmcid
    restore_and_return()

If the interrupt #2 hits between the load of user_rseq::mmcid and the
store to data::mmcid of the first fixup, then the return to the first
fixup context would obviously write the wrong ID back.

And you can't skip the fixup on return from interrupt #2 and just return
to the already running one in that case either.

That would be possible if the fixups are truly idempotent, e.g. can only
zero out memory.

The trivial nest case would be:

user_function()
  ...
  interrupt(#1)
     schedule()
       MMCID changes
       fixup requested
   ...
   setup_callframe(....)
   return to user

 fixup_ip:
    fixup_c(data)
    start zeroing

    interrupt(#2)
      schedule()
         MMCID changes
         fixup requested

     observes fixup running
     return to user

    continue zeroing
    restore_and_return()

But that does not work with signals because the signal delivery does not
return to the interrupted IP. It creates the sigframe and returns to the
signal handler, which then goes back with sys_rt_sigreturn(). Then the
kernel restores the interrupted context. So in that case you'd need:

user_function()
  ...
  interrupt(#1)
     schedule()
       MMCID changes
       fixup requested
   ...
   setup_callframe(....)
   // Must be a counter
   user_rseq->fixup_running++;
   return to user

 fixup_ip:
    fixup_c(data)
    start zeroing

    interrupt(#2)
      signal_delivery()
      setup_sigframe()
      observes fixup running
      setup_callframe(....)
      user_rseq->fixup_running++;
      return to user

 fixup_ip:
    fixup_c(data)
    zero everything
    user_rseq->fixup_running--;
      restore_and_return()

   signal_handler()
      sys_rt_sigreturn()
      restore_regs()
      observes fixup running
      return to user
    
 continue zeroing
 user_rseq->fixup_running--;
 restore_and_return()

Idempotent fixup functions restrict obviously what can be done
there. But if that restriction is fine, then this approach works too.

I have no strong opinion either way.

>> There are obviously a ton of details to take care of (/me mumbles shadow
>> stacks and RSEQ CS interaction), but the general principle should just
>> work. Emphasis on should and I'm so NOT going to hack that up. :)
>
> I could certainly make a POC out of this for x86.  It could handle
> red-zone and shadow-stack to start with.  I don't know if there are

I wouldn't even bother with shadow-stacks for a POC. The red-zone skip
when setting up the callframe, i.e. SP - 128 is unavoidable, but that's
it.

> other architecture-specific quirks that need to be aware of, given I am
> not familiar enough with architectures outside of x86.

Well every architecture has some quirks but I'm not aware of one which
would fundamentally stand in the way. The main difference is going to be
how the callframe is set up and how the fixup function needs to
look. That's always architecture specific and there are wizards for each
architecture to help with that :)

> One aspect that I have not think of for now is allowing fast
> registration/unregistration.  This RFC uses a syscall for registration.
> My original intent for this was to do a edge trigger detection of the
> first/last registration/unregistration in the syscall and set/unset a
> flag into the RSEQ state of the thread task.  That way, we don't need to
> touch another cache line to know if the thread registered something.
> This is obviously incompatible with short-live registrations.  However,
> I did not come with a case for it yet, so perhaps I am over-thinking
> this.

TCMalloc wont have a short-lived registration, neither wont a tracer or
something like that. Once a library is initialized it won't go away just
because. So if the facility needs to pause the callback intermittently
then this can simply be:

cb(data)
        if (!data->run)
        	return;

That won't be the end of the world if those pauses are not taking
forever. If they do then the syscall is justified.

Let's get the basic principles working first on a KISS basis and then
think about how to keep it as simple as possible.

So for the POC you neither need multi-lib support nor the VDSO
bits. Just hack up the ASM fixup and a trivial C demonstrator in user
space and register fixup and abort address with a hacked up sys_rseq().

Once that works, the extra bells and whistels are not hard to add. They
are hard to get right, but without the prove of concept wasting time
on them is pretty pointless. :)

Thanks,

        tglx

---
Everything should be made as simple as possible, but not simpler.

                                  - attributed to Albert Einstein
Re: [RFC PATCH 0/5] rseq: add support for RSEQ operations
Posted by Dmitry Vyukov 2 weeks, 3 days ago
On Tue, 8 Sept 2026 at 23:29, Thomas Gleixner <tglx@kernel.org> wrote:
>
> On Tue, Sep 08 2026 at 13:38, Olivier Dion wrote:
> > On Tue, 08 Sep 2026, Thomas Gleixner <tglx@kernel.org> wrote:
> >> fixup_ip()
> >>       ....
> >>       user_rseq.fixup_in_progress = false;
> >>       restore_and_return() // Returns to the original IP
> >
> > Yes.  This is what I had in mind (see reply to Florian).
> >
> >> A reasonable limitation for the fixup function should be a strict "no
> >> syscalls and no floating point within the fixup" rule. No floating point
> >> avoids the whole sigframe disaster.
> >
> > I suppose that no floating point also mean no xsave performance
> > trashing, which is obviously something we want to avoid.  Is that what
> > you mean by sigframe disaster?  Also, I am not sure how we can enforce
> > this no syscall/floating point policies other than asking users to be
> > good citizen.
>
> The sigframe disaster is that sigaltstack has been insufficient for the
> ever growing XSTATE to save (its even uncompressed XSTATE). Also these
> fixups really should do a few selective stores and not huge computations
> or wipe out a GB of memory. So avoiding the XSAVE/XRSTOR overhead
> completely is certainly a benefit.
>
> >> When return to user observes user_rseq.fixup_in_progress then it can
> >> mangle regs before doing anything else:
> >>
> >>         regs->ip = tsk->rseq.fixup_abort_ip;
> >>         regs->sp = tsk->rseq.fixup_sp;
> >>         user_rseq.fixup_in_progress = false;
> >>         tsk->rseq.needs_fixup = true;
> >>
> >> Which rewinds the stack to the callframe and makes the interrupted fixup
> >> continue at the fixup_abort_ip which just restores registers from the
> >> callframe and returns to the original IP.
> >
> > So the fixup handler would have an abort label, akin to RSEQ region.
> > But then there will be no guarantee that the fixup operations succeed?
> > Or did I completely misunderstood?
>
> They are only aborted when there is nesting, so the nesting context
> starts over and redoes them. Once the nesting context returns to the
> first fixup (abort IP) the state is correct and then the first fixup
> returns to the original return IP.
>
> That abort on nesting avoids the following issue:
>
> user_function()
>   ...
>   interrupt(#1)
>      schedule()
>        MMCID changes
>        fixup requested
>    ...
>    setup_callframe(....)
>    return to user
>
>  fixup_ip:
>     fixup_c(data)
>     ...
>      interrupt(#2)
>        schedule()
>          MMCID changes
>          fixup requested
>
>      fixup_ip:
>         fixup_c(data)
>          data::mmcid = user_rseq::mmcid
>         restore_and_return()
>
>     data::mmcid = user_rseq::mmcid
>     restore_and_return()
>
> If the interrupt #2 hits between the load of user_rseq::mmcid and the
> store to data::mmcid of the first fixup, then the return to the first
> fixup context would obviously write the wrong ID back.
>
> And you can't skip the fixup on return from interrupt #2 and just return
> to the already running one in that case either.
>
> That would be possible if the fixups are truly idempotent, e.g. can only
> zero out memory.
>
> The trivial nest case would be:
>
> user_function()
>   ...
>   interrupt(#1)
>      schedule()
>        MMCID changes
>        fixup requested
>    ...
>    setup_callframe(....)
>    return to user
>
>  fixup_ip:
>     fixup_c(data)
>     start zeroing
>
>     interrupt(#2)
>       schedule()
>          MMCID changes
>          fixup requested
>
>      observes fixup running
>      return to user
>
>     continue zeroing
>     restore_and_return()
>
> But that does not work with signals because the signal delivery does not
> return to the interrupted IP. It creates the sigframe and returns to the
> signal handler, which then goes back with sys_rt_sigreturn(). Then the
> kernel restores the interrupted context. So in that case you'd need:
>
> user_function()
>   ...
>   interrupt(#1)
>      schedule()
>        MMCID changes
>        fixup requested
>    ...
>    setup_callframe(....)
>    // Must be a counter
>    user_rseq->fixup_running++;
>    return to user
>
>  fixup_ip:
>     fixup_c(data)
>     start zeroing
>
>     interrupt(#2)
>       signal_delivery()
>       setup_sigframe()
>       observes fixup running
>       setup_callframe(....)
>       user_rseq->fixup_running++;
>       return to user
>
>  fixup_ip:
>     fixup_c(data)
>     zero everything
>     user_rseq->fixup_running--;
>       restore_and_return()
>
>    signal_handler()
>       sys_rt_sigreturn()
>       restore_regs()
>       observes fixup running
>       return to user
>
>  continue zeroing
>  user_rseq->fixup_running--;
>  restore_and_return()
>
> Idempotent fixup functions restrict obviously what can be done
> there. But if that restriction is fine, then this approach works too.
>
> I have no strong opinion either way.
>
> >> There are obviously a ton of details to take care of (/me mumbles shadow
> >> stacks and RSEQ CS interaction), but the general principle should just
> >> work. Emphasis on should and I'm so NOT going to hack that up. :)
> >
> > I could certainly make a POC out of this for x86.  It could handle
> > red-zone and shadow-stack to start with.  I don't know if there are
>
> I wouldn't even bother with shadow-stacks for a POC. The red-zone skip
> when setting up the callframe, i.e. SP - 128 is unavoidable, but that's
> it.
>
> > other architecture-specific quirks that need to be aware of, given I am
> > not familiar enough with architectures outside of x86.
>
> Well every architecture has some quirks but I'm not aware of one which
> would fundamentally stand in the way. The main difference is going to be
> how the callframe is set up and how the fixup function needs to
> look. That's always architecture specific and there are wizards for each
> architecture to help with that :)
>
> > One aspect that I have not think of for now is allowing fast
> > registration/unregistration.  This RFC uses a syscall for registration.
> > My original intent for this was to do a edge trigger detection of the
> > first/last registration/unregistration in the syscall and set/unset a
> > flag into the RSEQ state of the thread task.  That way, we don't need to
> > touch another cache line to know if the thread registered something.
> > This is obviously incompatible with short-live registrations.  However,
> > I did not come with a case for it yet, so perhaps I am over-thinking
> > this.
>
> TCMalloc wont have a short-lived registration, neither wont a tracer or
> something like that. Once a library is initialized it won't go away just
> because. So if the facility needs to pause the callback intermittently
> then this can simply be:
>
> cb(data)
>         if (!data->run)
>                 return;
>
> That won't be the end of the world if those pauses are not taking
> forever. If they do then the syscall is justified.
>
> Let's get the basic principles working first on a KISS basis and then
> think about how to keep it as simple as possible.
>
> So for the POC you neither need multi-lib support nor the VDSO
> bits. Just hack up the ASM fixup and a trivial C demonstrator in user
> space and register fixup and abort address with a hacked up sys_rseq().
>
> Once that works, the extra bells and whistels are not hard to add. They
> are hard to get right, but without the prove of concept wasting time
> on them is pretty pointless. :)

I totally agree with the "yet another general-purpose interpreter" argument.
I was thinking of some schemes where we would need to clear lots of
words, and doing this with an in-kernel interpreter may be too slow;
or clearing just 1 or 2 bytes; or even caching a pointer in the GS
register, and resetting it with wrgsbase.

I think we even brainstormed with Mathieu something similar to
resetting the PC to a custom user-space handler at Plumbers few years
ago, but we got stuck on signal handlers and re-entrancy problem.