[PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events

Clément Léger posted 5 patches 5 months ago
There is a newer version of this series
MAINTAINERS                                   |  15 +
arch/riscv/include/asm/asm.h                  |  14 +-
arch/riscv/include/asm/sbi.h                  |  61 ++
arch/riscv/include/asm/scs.h                  |   7 +
arch/riscv/include/asm/sse.h                  |  47 ++
arch/riscv/include/asm/switch_to.h            |  14 +
arch/riscv/include/asm/thread_info.h          |   1 +
arch/riscv/kernel/Makefile                    |   1 +
arch/riscv/kernel/asm-offsets.c               |  14 +
arch/riscv/kernel/sbi_sse.c                   | 174 +++++
arch/riscv/kernel/sbi_sse_entry.S             | 178 +++++
drivers/firmware/Kconfig                      |   1 +
drivers/firmware/Makefile                     |   1 +
drivers/firmware/riscv/Kconfig                |  15 +
drivers/firmware/riscv/Makefile               |   3 +
drivers/firmware/riscv/riscv_sbi_sse.c        | 701 ++++++++++++++++++
drivers/perf/Kconfig                          |  10 +
drivers/perf/riscv_pmu.c                      |  23 +
drivers/perf/riscv_pmu_sbi.c                  |  71 +-
include/linux/perf/riscv_pmu.h                |   5 +
include/linux/riscv_sbi_sse.h                 |  57 ++
tools/testing/selftests/riscv/Makefile        |   2 +-
tools/testing/selftests/riscv/sse/Makefile    |   5 +
.../selftests/riscv/sse/module/Makefile       |  16 +
.../riscv/sse/module/riscv_sse_test.c         | 513 +++++++++++++
.../selftests/riscv/sse/run_sse_test.sh       |  44 ++
26 files changed, 1979 insertions(+), 14 deletions(-)
create mode 100644 arch/riscv/include/asm/sse.h
create mode 100644 arch/riscv/kernel/sbi_sse.c
create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
create mode 100644 drivers/firmware/riscv/Kconfig
create mode 100644 drivers/firmware/riscv/Makefile
create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
create mode 100644 include/linux/riscv_sbi_sse.h
create mode 100644 tools/testing/selftests/riscv/sse/Makefile
create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh
[PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events
Posted by Clément Léger 5 months ago
The SBI Supervisor Software Events (SSE) extensions provides a mechanism
to inject software events from an SBI implementation to supervisor
software such that it preempts all other supervisor level traps and
interrupts. This extension is introduced by the SBI v3.0 specification[1].

Various events are defined and can be send asynchronously to supervisor
software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
as platform specific events. Events can be either local (per-hart) or
global. Events can be nested on top of each other based on priority and
can interrupt the kernel at any time.

First patch adds the SSE definitions. Second one adds support for SSE
at arch level (entry code and stack allocations) and third one at driver
level. Finally, the last patch add support for SSE events in the SBI PMU
driver. Additional testing for that part is highly welcomed since there
are a lot of possible path that needs to be exercised.

Amongst the specific points that needs to be handle is the interruption
at any point of the kernel execution and more specifically at the
beginning of exception handling. Due to the fact that the exception entry
implementation uses the SCRATCH CSR as both the current task struct and
as the temporary register to switch the stack and save register, it is
difficult to reliably get the current task struct if we get interrupted
at this specific moment (ie, it might contain 0, the task pointer or tp).
A fixup-like mechanism is not possible due to the nested nature of SSE
which makes it really hard to obtain the original interruption site. In
order to retrieve the task in a reliable manner, add an additional
__sse_entry_task per_cpu array which stores the current task. Ideally,
we would need to modify the way we retrieve/store the current task in
exception handling so that it does not depend on the place where it's
interrupted.

Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
handled and does not adds any overhead to existing code. Moreover, it
provides "true" NMI-like interrupts which can interrupt the kernel at
any time (even in exception handling). This is particularly crucial for
RAS errors which needs to be handled as fast as possible to avoid any
fault propagation.

A test suite is available as a separate kselftest module. In order to
build it, you can use the following command:

$ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests

Then load the module using:

$ sh run_sse_test.sh

A KVM SBI SSE extension implementation is available at [2].

Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1]
Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2]

---

Changes in v7:
 - Check return values of sse_on_each_cpu()
 - Fix typos in commit
 - Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT
 - Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK
 - Add tlb flush for vmap stack to avoid taking exception during sse
   handler upon stack access. (Alex)
 - Move some assembly instruction to slow path
 - Renamed sse.c to sbi_sse.c, ditto for other files
 - Renamed RISCV_SSE to RISCV_SBI_SSE
 - Renamed sse_event_handler to sse_event_handler_fn
 - Put ifdef around sse_evt in PMU SBI driver

Changes in v6:
 - Fix comment in assembly argument
 - Check hart id to be the expected one in order to skip CPU id
   matching in sse assembly.

Changes in v5:
 - Added a SSE test module in kselftests
 - Removed an unused variable
 - Applied checkpatch.pl --strict and fix all errors
 - Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock()
 - Fix wrong variable returned in sse_get_event
 - Remove useless init of events list
 - Remove useless empty for loop on cpus
 - Set sse_available as  __ro_after_init
 - Changed a few pr_debug to pr_warn
 - Fix event enabled stated updated in case of failure
 - Change no_lock to nolock
 - Rename attr_buf to attr
 - renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed
   the second argument
 - Simplify return value in sse_event_attr_set_nolock()
 - Remove while loop(-EINVAL) for event cpu set call
 - Renamed interrupted_state_phys to interrupted_phys
 - Use scoped_guards/guard for sse_mutex
 - Remove useless struct forward declaration in sse.h
 - Add more explanations as to why we set SIE bit in IP
 - Unconditionnally set SIE in SIP
 - Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free()
 - Replace move instructions with mv
 - Rename NR_CPUS asm symbol to ASM_NR_CPUS
 - Restore SSTATUS first in sse_entry return path so that it works for
   double trap without any modification later.
 - Implement proper per cpu revert if enable/register fails

Changes in v4:
 - Fix a bug when using per_cpu ptr for local event (Andrew)
 - Add sse_event_disable/enable_local()
 - Add pmu_disable/pmu_enable() to disable/enable SSE event
 - Update event ID description according to the latest spec
 - Fix comment about arguments in handle_sse()
 - Added Himanchu as a SSE reviewer
 - Used SYM_DATA_*() macros instead of hardcoded labels
 - Invoke softirqs only if not returning to kernel with irqs disabled
 - Remove invalid state check for write attribute function.
 - Remove useless bneq statement in sse_entry.S

Changes in v3:
 - Split arch/driver support
 - Fix potential register failure reporting
 - Set a few pr_err as pr_debug
 - Allow CONFIG_RISCV_SSE to be disabled
 - Fix build without CONFIG_RISCV_SSE
 - Remove fixup-like mechanism and use a per-cpu array
 - Fixed SSCRATCH being corrupted when interrupting the kernel in early
   exception path.
 - Split SSE assembly from entry.S
 - Add Himanchu SSE mask/unmask and runtime PM support.
 - Disable user memory access/floating point/vector in SSE handler
 - Rebased on master

v2: https://lore.kernel.org/linux-riscv/20240112111720.2975069-1-cleger@rivosinc.com/

Changes in v2:
 - Implemented specification v2
 - Fix various error handling cases
 - Added shadow stack support

v1: https://lore.kernel.org/linux-riscv/20231026143122.279437-1-cleger@rivosinc.com/

Clément Léger (5):
  riscv: add SBI SSE extension definitions
  riscv: add support for SBI Supervisor Software Events extension
  drivers: firmware: add riscv SSE support
  perf: RISC-V: add support for SSE event
  selftests/riscv: add SSE test module

 MAINTAINERS                                   |  15 +
 arch/riscv/include/asm/asm.h                  |  14 +-
 arch/riscv/include/asm/sbi.h                  |  61 ++
 arch/riscv/include/asm/scs.h                  |   7 +
 arch/riscv/include/asm/sse.h                  |  47 ++
 arch/riscv/include/asm/switch_to.h            |  14 +
 arch/riscv/include/asm/thread_info.h          |   1 +
 arch/riscv/kernel/Makefile                    |   1 +
 arch/riscv/kernel/asm-offsets.c               |  14 +
 arch/riscv/kernel/sbi_sse.c                   | 174 +++++
 arch/riscv/kernel/sbi_sse_entry.S             | 178 +++++
 drivers/firmware/Kconfig                      |   1 +
 drivers/firmware/Makefile                     |   1 +
 drivers/firmware/riscv/Kconfig                |  15 +
 drivers/firmware/riscv/Makefile               |   3 +
 drivers/firmware/riscv/riscv_sbi_sse.c        | 701 ++++++++++++++++++
 drivers/perf/Kconfig                          |  10 +
 drivers/perf/riscv_pmu.c                      |  23 +
 drivers/perf/riscv_pmu_sbi.c                  |  71 +-
 include/linux/perf/riscv_pmu.h                |   5 +
 include/linux/riscv_sbi_sse.h                 |  57 ++
 tools/testing/selftests/riscv/Makefile        |   2 +-
 tools/testing/selftests/riscv/sse/Makefile    |   5 +
 .../selftests/riscv/sse/module/Makefile       |  16 +
 .../riscv/sse/module/riscv_sse_test.c         | 513 +++++++++++++
 .../selftests/riscv/sse/run_sse_test.sh       |  44 ++
 26 files changed, 1979 insertions(+), 14 deletions(-)
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
 create mode 100644 tools/testing/selftests/riscv/sse/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
 create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh

-- 
2.43.0

Re: [External] [PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events
Posted by yunhui cui 3 months, 1 week ago
Hi Clément,

On Tue, Sep 9, 2025 at 2:19 AM Clément Léger <cleger@rivosinc.com> wrote:
>
> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
> to inject software events from an SBI implementation to supervisor
> software such that it preempts all other supervisor level traps and
> interrupts. This extension is introduced by the SBI v3.0 specification[1].
>
> Various events are defined and can be send asynchronously to supervisor
> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
> as platform specific events. Events can be either local (per-hart) or
> global. Events can be nested on top of each other based on priority and
> can interrupt the kernel at any time.
>
> First patch adds the SSE definitions. Second one adds support for SSE
> at arch level (entry code and stack allocations) and third one at driver
> level. Finally, the last patch add support for SSE events in the SBI PMU
> driver. Additional testing for that part is highly welcomed since there
> are a lot of possible path that needs to be exercised.
>
> Amongst the specific points that needs to be handle is the interruption
> at any point of the kernel execution and more specifically at the
> beginning of exception handling. Due to the fact that the exception entry
> implementation uses the SCRATCH CSR as both the current task struct and
> as the temporary register to switch the stack and save register, it is
> difficult to reliably get the current task struct if we get interrupted
> at this specific moment (ie, it might contain 0, the task pointer or tp).
> A fixup-like mechanism is not possible due to the nested nature of SSE
> which makes it really hard to obtain the original interruption site. In
> order to retrieve the task in a reliable manner, add an additional
> __sse_entry_task per_cpu array which stores the current task. Ideally,
> we would need to modify the way we retrieve/store the current task in
> exception handling so that it does not depend on the place where it's
> interrupted.
>
> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
> handled and does not adds any overhead to existing code. Moreover, it
> provides "true" NMI-like interrupts which can interrupt the kernel at
> any time (even in exception handling). This is particularly crucial for
> RAS errors which needs to be handled as fast as possible to avoid any
> fault propagation.
>
> A test suite is available as a separate kselftest module. In order to
> build it, you can use the following command:
>
> $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests
>
> Then load the module using:
>
> $ sh run_sse_test.sh
>
> A KVM SBI SSE extension implementation is available at [2].
>
> Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1]
> Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2]
>
> ---
>
> Changes in v7:
>  - Check return values of sse_on_each_cpu()
>  - Fix typos in commit
>  - Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT
>  - Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK
>  - Add tlb flush for vmap stack to avoid taking exception during sse
>    handler upon stack access. (Alex)
>  - Move some assembly instruction to slow path
>  - Renamed sse.c to sbi_sse.c, ditto for other files
>  - Renamed RISCV_SSE to RISCV_SBI_SSE
>  - Renamed sse_event_handler to sse_event_handler_fn
>  - Put ifdef around sse_evt in PMU SBI driver
>
> Changes in v6:
>  - Fix comment in assembly argument
>  - Check hart id to be the expected one in order to skip CPU id
>    matching in sse assembly.
>
> Changes in v5:
>  - Added a SSE test module in kselftests
>  - Removed an unused variable
>  - Applied checkpatch.pl --strict and fix all errors
>  - Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock()
>  - Fix wrong variable returned in sse_get_event
>  - Remove useless init of events list
>  - Remove useless empty for loop on cpus
>  - Set sse_available as  __ro_after_init
>  - Changed a few pr_debug to pr_warn
>  - Fix event enabled stated updated in case of failure
>  - Change no_lock to nolock
>  - Rename attr_buf to attr
>  - renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed
>    the second argument
>  - Simplify return value in sse_event_attr_set_nolock()
>  - Remove while loop(-EINVAL) for event cpu set call
>  - Renamed interrupted_state_phys to interrupted_phys
>  - Use scoped_guards/guard for sse_mutex
>  - Remove useless struct forward declaration in sse.h
>  - Add more explanations as to why we set SIE bit in IP
>  - Unconditionnally set SIE in SIP
>  - Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free()
>  - Replace move instructions with mv
>  - Rename NR_CPUS asm symbol to ASM_NR_CPUS
>  - Restore SSTATUS first in sse_entry return path so that it works for
>    double trap without any modification later.
>  - Implement proper per cpu revert if enable/register fails
>
> Changes in v4:
>  - Fix a bug when using per_cpu ptr for local event (Andrew)
>  - Add sse_event_disable/enable_local()
>  - Add pmu_disable/pmu_enable() to disable/enable SSE event
>  - Update event ID description according to the latest spec
>  - Fix comment about arguments in handle_sse()
>  - Added Himanchu as a SSE reviewer
>  - Used SYM_DATA_*() macros instead of hardcoded labels
>  - Invoke softirqs only if not returning to kernel with irqs disabled
>  - Remove invalid state check for write attribute function.
>  - Remove useless bneq statement in sse_entry.S
>
> Changes in v3:
>  - Split arch/driver support
>  - Fix potential register failure reporting
>  - Set a few pr_err as pr_debug
>  - Allow CONFIG_RISCV_SSE to be disabled
>  - Fix build without CONFIG_RISCV_SSE
>  - Remove fixup-like mechanism and use a per-cpu array
>  - Fixed SSCRATCH being corrupted when interrupting the kernel in early
>    exception path.
>  - Split SSE assembly from entry.S
>  - Add Himanchu SSE mask/unmask and runtime PM support.
>  - Disable user memory access/floating point/vector in SSE handler
>  - Rebased on master
>
> v2: https://lore.kernel.org/linux-riscv/20240112111720.2975069-1-cleger@rivosinc.com/
>
> Changes in v2:
>  - Implemented specification v2
>  - Fix various error handling cases
>  - Added shadow stack support
>
> v1: https://lore.kernel.org/linux-riscv/20231026143122.279437-1-cleger@rivosinc.com/
>
> Clément Léger (5):
>   riscv: add SBI SSE extension definitions
>   riscv: add support for SBI Supervisor Software Events extension
>   drivers: firmware: add riscv SSE support
>   perf: RISC-V: add support for SSE event
>   selftests/riscv: add SSE test module
>
>  MAINTAINERS                                   |  15 +
>  arch/riscv/include/asm/asm.h                  |  14 +-
>  arch/riscv/include/asm/sbi.h                  |  61 ++
>  arch/riscv/include/asm/scs.h                  |   7 +
>  arch/riscv/include/asm/sse.h                  |  47 ++
>  arch/riscv/include/asm/switch_to.h            |  14 +
>  arch/riscv/include/asm/thread_info.h          |   1 +
>  arch/riscv/kernel/Makefile                    |   1 +
>  arch/riscv/kernel/asm-offsets.c               |  14 +
>  arch/riscv/kernel/sbi_sse.c                   | 174 +++++
>  arch/riscv/kernel/sbi_sse_entry.S             | 178 +++++
>  drivers/firmware/Kconfig                      |   1 +
>  drivers/firmware/Makefile                     |   1 +
>  drivers/firmware/riscv/Kconfig                |  15 +
>  drivers/firmware/riscv/Makefile               |   3 +
>  drivers/firmware/riscv/riscv_sbi_sse.c        | 701 ++++++++++++++++++
>  drivers/perf/Kconfig                          |  10 +
>  drivers/perf/riscv_pmu.c                      |  23 +
>  drivers/perf/riscv_pmu_sbi.c                  |  71 +-
>  include/linux/perf/riscv_pmu.h                |   5 +
>  include/linux/riscv_sbi_sse.h                 |  57 ++
>  tools/testing/selftests/riscv/Makefile        |   2 +-
>  tools/testing/selftests/riscv/sse/Makefile    |   5 +
>  .../selftests/riscv/sse/module/Makefile       |  16 +
>  .../riscv/sse/module/riscv_sse_test.c         | 513 +++++++++++++
>  .../selftests/riscv/sse/run_sse_test.sh       |  44 ++
>  26 files changed, 1979 insertions(+), 14 deletions(-)
>  create mode 100644 arch/riscv/include/asm/sse.h
>  create mode 100644 arch/riscv/kernel/sbi_sse.c
>  create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
>  create mode 100644 drivers/firmware/riscv/Kconfig
>  create mode 100644 drivers/firmware/riscv/Makefile
>  create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c

These newly added files use different GPL-xxx licenses respectively:
arch/riscv/kernel/sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-or-later
drivers/firmware/riscv/riscv_sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-only
arch/riscv/include/asm/sse.h: /* SPDX-License-Identifier: GPL-2.0-only */
Should they be consistent, or is this by design?


>  create mode 100644 include/linux/riscv_sbi_sse.h
>  create mode 100644 tools/testing/selftests/riscv/sse/Makefile
>  create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
>  create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
>  create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh
>
> --
> 2.43.0
>

Thanks,
Yunhui
Re: [External] [PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events
Posted by Clément Léger 3 months, 1 week ago

On 10/27/25 02:47, yunhui cui wrote:
> Hi Clément,
> 
> On Tue, Sep 9, 2025 at 2:19 AM Clément Léger <cleger@rivosinc.com> wrote:
>>
>> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
>> to inject software events from an SBI implementation to supervisor
>> software such that it preempts all other supervisor level traps and
>> interrupts. This extension is introduced by the SBI v3.0 specification[1].
>>
>> Various events are defined and can be send asynchronously to supervisor
>> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
>> as platform specific events. Events can be either local (per-hart) or
>> global. Events can be nested on top of each other based on priority and
>> can interrupt the kernel at any time.
>>
>> First patch adds the SSE definitions. Second one adds support for SSE
>> at arch level (entry code and stack allocations) and third one at driver
>> level. Finally, the last patch add support for SSE events in the SBI PMU
>> driver. Additional testing for that part is highly welcomed since there
>> are a lot of possible path that needs to be exercised.
>>
>> Amongst the specific points that needs to be handle is the interruption
>> at any point of the kernel execution and more specifically at the
>> beginning of exception handling. Due to the fact that the exception entry
>> implementation uses the SCRATCH CSR as both the current task struct and
>> as the temporary register to switch the stack and save register, it is
>> difficult to reliably get the current task struct if we get interrupted
>> at this specific moment (ie, it might contain 0, the task pointer or tp).
>> A fixup-like mechanism is not possible due to the nested nature of SSE
>> which makes it really hard to obtain the original interruption site. In
>> order to retrieve the task in a reliable manner, add an additional
>> __sse_entry_task per_cpu array which stores the current task. Ideally,
>> we would need to modify the way we retrieve/store the current task in
>> exception handling so that it does not depend on the place where it's
>> interrupted.
>>
>> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
>> handled and does not adds any overhead to existing code. Moreover, it
>> provides "true" NMI-like interrupts which can interrupt the kernel at
>> any time (even in exception handling). This is particularly crucial for
>> RAS errors which needs to be handled as fast as possible to avoid any
>> fault propagation.
>>
>> A test suite is available as a separate kselftest module. In order to
>> build it, you can use the following command:
>>
>> $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests
>>
>> Then load the module using:
>>
>> $ sh run_sse_test.sh
>>
>> A KVM SBI SSE extension implementation is available at [2].
>>
>> Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1]
>> Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2]
>>
>> ---
>>
>> Changes in v7:
>>  - Check return values of sse_on_each_cpu()
>>  - Fix typos in commit
>>  - Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT
>>  - Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK
>>  - Add tlb flush for vmap stack to avoid taking exception during sse
>>    handler upon stack access. (Alex)
>>  - Move some assembly instruction to slow path
>>  - Renamed sse.c to sbi_sse.c, ditto for other files
>>  - Renamed RISCV_SSE to RISCV_SBI_SSE
>>  - Renamed sse_event_handler to sse_event_handler_fn
>>  - Put ifdef around sse_evt in PMU SBI driver
>>
>> Changes in v6:
>>  - Fix comment in assembly argument
>>  - Check hart id to be the expected one in order to skip CPU id
>>    matching in sse assembly.
>>
>> Changes in v5:
>>  - Added a SSE test module in kselftests
>>  - Removed an unused variable
>>  - Applied checkpatch.pl --strict and fix all errors
>>  - Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock()
>>  - Fix wrong variable returned in sse_get_event
>>  - Remove useless init of events list
>>  - Remove useless empty for loop on cpus
>>  - Set sse_available as  __ro_after_init
>>  - Changed a few pr_debug to pr_warn
>>  - Fix event enabled stated updated in case of failure
>>  - Change no_lock to nolock
>>  - Rename attr_buf to attr
>>  - renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed
>>    the second argument
>>  - Simplify return value in sse_event_attr_set_nolock()
>>  - Remove while loop(-EINVAL) for event cpu set call
>>  - Renamed interrupted_state_phys to interrupted_phys
>>  - Use scoped_guards/guard for sse_mutex
>>  - Remove useless struct forward declaration in sse.h
>>  - Add more explanations as to why we set SIE bit in IP
>>  - Unconditionnally set SIE in SIP
>>  - Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free()
>>  - Replace move instructions with mv
>>  - Rename NR_CPUS asm symbol to ASM_NR_CPUS
>>  - Restore SSTATUS first in sse_entry return path so that it works for
>>    double trap without any modification later.
>>  - Implement proper per cpu revert if enable/register fails
>>
>> Changes in v4:
>>  - Fix a bug when using per_cpu ptr for local event (Andrew)
>>  - Add sse_event_disable/enable_local()
>>  - Add pmu_disable/pmu_enable() to disable/enable SSE event
>>  - Update event ID description according to the latest spec
>>  - Fix comment about arguments in handle_sse()
>>  - Added Himanchu as a SSE reviewer
>>  - Used SYM_DATA_*() macros instead of hardcoded labels
>>  - Invoke softirqs only if not returning to kernel with irqs disabled
>>  - Remove invalid state check for write attribute function.
>>  - Remove useless bneq statement in sse_entry.S
>>
>> Changes in v3:
>>  - Split arch/driver support
>>  - Fix potential register failure reporting
>>  - Set a few pr_err as pr_debug
>>  - Allow CONFIG_RISCV_SSE to be disabled
>>  - Fix build without CONFIG_RISCV_SSE
>>  - Remove fixup-like mechanism and use a per-cpu array
>>  - Fixed SSCRATCH being corrupted when interrupting the kernel in early
>>    exception path.
>>  - Split SSE assembly from entry.S
>>  - Add Himanchu SSE mask/unmask and runtime PM support.
>>  - Disable user memory access/floating point/vector in SSE handler
>>  - Rebased on master
>>
>> v2: https://lore.kernel.org/linux-riscv/20240112111720.2975069-1-cleger@rivosinc.com/
>>
>> Changes in v2:
>>  - Implemented specification v2
>>  - Fix various error handling cases
>>  - Added shadow stack support
>>
>> v1: https://lore.kernel.org/linux-riscv/20231026143122.279437-1-cleger@rivosinc.com/
>>
>> Clément Léger (5):
>>   riscv: add SBI SSE extension definitions
>>   riscv: add support for SBI Supervisor Software Events extension
>>   drivers: firmware: add riscv SSE support
>>   perf: RISC-V: add support for SSE event
>>   selftests/riscv: add SSE test module
>>
>>  MAINTAINERS                                   |  15 +
>>  arch/riscv/include/asm/asm.h                  |  14 +-
>>  arch/riscv/include/asm/sbi.h                  |  61 ++
>>  arch/riscv/include/asm/scs.h                  |   7 +
>>  arch/riscv/include/asm/sse.h                  |  47 ++
>>  arch/riscv/include/asm/switch_to.h            |  14 +
>>  arch/riscv/include/asm/thread_info.h          |   1 +
>>  arch/riscv/kernel/Makefile                    |   1 +
>>  arch/riscv/kernel/asm-offsets.c               |  14 +
>>  arch/riscv/kernel/sbi_sse.c                   | 174 +++++
>>  arch/riscv/kernel/sbi_sse_entry.S             | 178 +++++
>>  drivers/firmware/Kconfig                      |   1 +
>>  drivers/firmware/Makefile                     |   1 +
>>  drivers/firmware/riscv/Kconfig                |  15 +
>>  drivers/firmware/riscv/Makefile               |   3 +
>>  drivers/firmware/riscv/riscv_sbi_sse.c        | 701 ++++++++++++++++++
>>  drivers/perf/Kconfig                          |  10 +
>>  drivers/perf/riscv_pmu.c                      |  23 +
>>  drivers/perf/riscv_pmu_sbi.c                  |  71 +-
>>  include/linux/perf/riscv_pmu.h                |   5 +
>>  include/linux/riscv_sbi_sse.h                 |  57 ++
>>  tools/testing/selftests/riscv/Makefile        |   2 +-
>>  tools/testing/selftests/riscv/sse/Makefile    |   5 +
>>  .../selftests/riscv/sse/module/Makefile       |  16 +
>>  .../riscv/sse/module/riscv_sse_test.c         | 513 +++++++++++++
>>  .../selftests/riscv/sse/run_sse_test.sh       |  44 ++
>>  26 files changed, 1979 insertions(+), 14 deletions(-)
>>  create mode 100644 arch/riscv/include/asm/sse.h
>>  create mode 100644 arch/riscv/kernel/sbi_sse.c
>>  create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
>>  create mode 100644 drivers/firmware/riscv/Kconfig
>>  create mode 100644 drivers/firmware/riscv/Makefile
>>  create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
> 
> These newly added files use different GPL-xxx licenses respectively:
> arch/riscv/kernel/sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-or-later
> drivers/firmware/riscv/riscv_sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-only
> arch/riscv/include/asm/sse.h: /* SPDX-License-Identifier: GPL-2.0-only */
> Should they be consistent, or is this by design?

Hi Yunhui,

That is not intended, I'll fix it.

Thanks,

Clément

> 
> 
>>  create mode 100644 include/linux/riscv_sbi_sse.h
>>  create mode 100644 tools/testing/selftests/riscv/sse/Makefile
>>  create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
>>  create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
>>  create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh
>>
>> --
>> 2.43.0
>>
> 
> Thanks,
> Yunhui

Re: [PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events
Posted by Paul Walmsley 4 months ago
Hi Clément, 

On Mon, 8 Sep 2025, Clément Léger wrote:

> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
> to inject software events from an SBI implementation to supervisor
> software such that it preempts all other supervisor level traps and
> interrupts. This extension is introduced by the SBI v3.0 specification[1].
> 
> Various events are defined and can be send asynchronously to supervisor
> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
> as platform specific events. Events can be either local (per-hart) or
> global. Events can be nested on top of each other based on priority and
> can interrupt the kernel at any time.
> 
> First patch adds the SSE definitions. Second one adds support for SSE
> at arch level (entry code and stack allocations) and third one at driver
> level. Finally, the last patch add support for SSE events in the SBI PMU
> driver. Additional testing for that part is highly welcomed since there
> are a lot of possible path that needs to be exercised.
> 
> Amongst the specific points that needs to be handle is the interruption
> at any point of the kernel execution and more specifically at the
> beginning of exception handling. Due to the fact that the exception entry
> implementation uses the SCRATCH CSR as both the current task struct and
> as the temporary register to switch the stack and save register, it is
> difficult to reliably get the current task struct if we get interrupted
> at this specific moment (ie, it might contain 0, the task pointer or tp).
> A fixup-like mechanism is not possible due to the nested nature of SSE
> which makes it really hard to obtain the original interruption site. In
> order to retrieve the task in a reliable manner, add an additional
> __sse_entry_task per_cpu array which stores the current task. Ideally,
> we would need to modify the way we retrieve/store the current task in
> exception handling so that it does not depend on the place where it's
> interrupted.
> 
> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
> handled and does not adds any overhead to existing code. Moreover, it
> provides "true" NMI-like interrupts which can interrupt the kernel at
> any time (even in exception handling). This is particularly crucial for
> RAS errors which needs to be handled as fast as possible to avoid any
> fault propagation.
> 
> A test suite is available as a separate kselftest module. In order to
> build it, you can use the following command:
> 
> $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests
> 
> Then load the module using:
> 
> $ sh run_sse_test.sh
> 
> A KVM SBI SSE extension implementation is available at [2].
> 
> Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1]
> Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2]

I updated these to apply on Linus' current master, commit 5472d60c129f, 
cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() 
suggestion, and pushed them up here to make it convenient for folks to 
integrate and test:

  https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18

Am assuming you didn't have other changes that you wanted to make; let me 
know if that's not the case.

I noticed that you asked for folks to do additional testing, particularly 
of the SBI PMU driver integration, but didn't notice any additional 
Tested-by:s.  It would be great if other folks on the list could do some 
focused testing now, particularly since we're on v7 of this series, and 
I'm sure others care about this.


thanks,

- Paul

Re: [PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events
Posted by Clément Léger 3 months, 3 weeks ago
Hi Paul,

Thanks for caring about this series ;)

On 10/10/2025 03:32, Paul Walmsley wrote:
> Hi Clément, 
> 
> On Mon, 8 Sep 2025, Clément Léger wrote:
> 
>> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
>> to inject software events from an SBI implementation to supervisor
>> software such that it preempts all other supervisor level traps and
>> interrupts. This extension is introduced by the SBI v3.0 specification[1].
>>
>> Various events are defined and can be send asynchronously to supervisor
>> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
>> as platform specific events. Events can be either local (per-hart) or
>> global. Events can be nested on top of each other based on priority and
>> can interrupt the kernel at any time.
>>
>> First patch adds the SSE definitions. Second one adds support for SSE
>> at arch level (entry code and stack allocations) and third one at driver
>> level. Finally, the last patch add support for SSE events in the SBI PMU
>> driver. Additional testing for that part is highly welcomed since there
>> are a lot of possible path that needs to be exercised.
>>
>> Amongst the specific points that needs to be handle is the interruption
>> at any point of the kernel execution and more specifically at the
>> beginning of exception handling. Due to the fact that the exception entry
>> implementation uses the SCRATCH CSR as both the current task struct and
>> as the temporary register to switch the stack and save register, it is
>> difficult to reliably get the current task struct if we get interrupted
>> at this specific moment (ie, it might contain 0, the task pointer or tp).
>> A fixup-like mechanism is not possible due to the nested nature of SSE
>> which makes it really hard to obtain the original interruption site. In
>> order to retrieve the task in a reliable manner, add an additional
>> __sse_entry_task per_cpu array which stores the current task. Ideally,
>> we would need to modify the way we retrieve/store the current task in
>> exception handling so that it does not depend on the place where it's
>> interrupted.
>>
>> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
>> handled and does not adds any overhead to existing code. Moreover, it
>> provides "true" NMI-like interrupts which can interrupt the kernel at
>> any time (even in exception handling). This is particularly crucial for
>> RAS errors which needs to be handled as fast as possible to avoid any
>> fault propagation.
>>
>> A test suite is available as a separate kselftest module. In order to
>> build it, you can use the following command:
>>
>> $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests
>>
>> Then load the module using:
>>
>> $ sh run_sse_test.sh
>>
>> A KVM SBI SSE extension implementation is available at [2].
>>
>> Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1]
>> Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2]
> 
> I updated these to apply on Linus' current master, commit 5472d60c129f, 
> cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() 
> suggestion, and pushed them up here to make it convenient for folks to 
> integrate and test:
> 
>   https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18
> 
> Am assuming you didn't have other changes that you wanted to make; let me 
> know if that's not the case.

Indeed, I do not have pending modification for this series. The KVM SBI
SSE support will be submitted later that year.

> 
> I noticed that you asked for folks to do additional testing, particularly 
> of the SBI PMU driver integration, but didn't notice any additional 
> Tested-by:s.  It would be great if other folks on the list could do some 
> focused testing now, particularly since we're on v7 of this series, and 
> I'm sure others care about this.

 I would have prefer a bit more reviews and testing before going through
since SSE can be quite intrusive at execution time (even though the
classic IRQ path should not be impacted, I expect mostly feedback/bugs
from the SSE handling path itself). We'll see in a few days if people
find such problems.

Thanks,

Clément


> 
> 
> thanks,
> 
> - Paul
> 

Re: [PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events
Posted by Andrew Jones 3 months, 3 weeks ago
On Mon, Oct 13, 2025 at 08:53:01AM +0200, Clément Léger wrote:
...
>  I would have prefer a bit more reviews and testing before going through
> since SSE can be quite intrusive at execution time (even though the
> classic IRQ path should not be impacted, I expect mostly feedback/bugs
> from the SSE handling path itself). We'll see in a few days if people
> find such problems.

I've been planning to compare this series with the code we merged into
kvm-unit-tests since the kvm-unit-tests series is where I put the effort
into reviewing wrt the spec, etc. On a skim of this series I see several
similarities with the kvm-unit-tests code so I should try to actually do
that comparison exercise, but I'm not sure when I'll get the time...

Thanks,
drew
Re: [PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events
Posted by Himanshu Chauhan 3 months, 3 weeks ago
On Mon, Oct 13, 2025 at 12:23 PM Clément Léger <cleger@rivosinc.com> wrote:
>
> Hi Paul,
>
> Thanks for caring about this series ;)
>
> On 10/10/2025 03:32, Paul Walmsley wrote:
> > Hi Clément,
> >
> > On Mon, 8 Sep 2025, Clément Léger wrote:
> >
> >> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
> >> to inject software events from an SBI implementation to supervisor
> >> software such that it preempts all other supervisor level traps and
> >> interrupts. This extension is introduced by the SBI v3.0 specification[1].
> >>
> >> Various events are defined and can be send asynchronously to supervisor
> >> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
> >> as platform specific events. Events can be either local (per-hart) or
> >> global. Events can be nested on top of each other based on priority and
> >> can interrupt the kernel at any time.
> >>
> >> First patch adds the SSE definitions. Second one adds support for SSE
> >> at arch level (entry code and stack allocations) and third one at driver
> >> level. Finally, the last patch add support for SSE events in the SBI PMU
> >> driver. Additional testing for that part is highly welcomed since there
> >> are a lot of possible path that needs to be exercised.
> >>
> >> Amongst the specific points that needs to be handle is the interruption
> >> at any point of the kernel execution and more specifically at the
> >> beginning of exception handling. Due to the fact that the exception entry
> >> implementation uses the SCRATCH CSR as both the current task struct and
> >> as the temporary register to switch the stack and save register, it is
> >> difficult to reliably get the current task struct if we get interrupted
> >> at this specific moment (ie, it might contain 0, the task pointer or tp).
> >> A fixup-like mechanism is not possible due to the nested nature of SSE
> >> which makes it really hard to obtain the original interruption site. In
> >> order to retrieve the task in a reliable manner, add an additional
> >> __sse_entry_task per_cpu array which stores the current task. Ideally,
> >> we would need to modify the way we retrieve/store the current task in
> >> exception handling so that it does not depend on the place where it's
> >> interrupted.
> >>
> >> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
> >> handled and does not adds any overhead to existing code. Moreover, it
> >> provides "true" NMI-like interrupts which can interrupt the kernel at
> >> any time (even in exception handling). This is particularly crucial for
> >> RAS errors which needs to be handled as fast as possible to avoid any
> >> fault propagation.
> >>
> >> A test suite is available as a separate kselftest module. In order to
> >> build it, you can use the following command:
> >>
> >> $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests
> >>
> >> Then load the module using:
> >>
> >> $ sh run_sse_test.sh
> >>
> >> A KVM SBI SSE extension implementation is available at [2].
> >>
> >> Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1]
> >> Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2]
> >
> > I updated these to apply on Linus' current master, commit 5472d60c129f,
> > cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info()
> > suggestion, and pushed them up here to make it convenient for folks to
> > integrate and test:
> >
> >   https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18
> >
> > Am assuming you didn't have other changes that you wanted to make; let me
> > know if that's not the case.
>
> Indeed, I do not have pending modification for this series. The KVM SBI
> SSE support will be submitted later that year.

I am testing this series with my RAS patch set. Will update on this.

Thanks
Regards
Himanshu

>
> >
> > I noticed that you asked for folks to do additional testing, particularly
> > of the SBI PMU driver integration, but didn't notice any additional
> > Tested-by:s.  It would be great if other folks on the list could do some
> > focused testing now, particularly since we're on v7 of this series, and
> > I'm sure others care about this.
>
>  I would have prefer a bit more reviews and testing before going through
> since SSE can be quite intrusive at execution time (even though the
> classic IRQ path should not be impacted, I expect mostly feedback/bugs
> from the SSE handling path itself). We'll see in a few days if people
> find such problems.
>
> Thanks,
>
> Clément
>
>
> >
> >
> > thanks,
> >
> > - Paul
> >
>
Re: [PATCH v7 0/5] riscv: add support for SBI Supervisor Software Events
Posted by Paul Walmsley 4 months ago
On Thu, 9 Oct 2025, Paul Walmsley wrote:

> I updated these to apply on Linus' current master, commit 5472d60c129f, 
> cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() 
> suggestion, and pushed them up here to make it convenient for folks to 
> integrate and test:
> 
>   https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18

Sorry, wrong link - here's the correct one:

   https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/log/?h=experimental/sse-v7


- Paul