MAINTAINERS | 15 + arch/riscv/include/asm/asm.h | 14 +- arch/riscv/include/asm/sbi.h | 61 ++ arch/riscv/include/asm/scs.h | 7 + arch/riscv/include/asm/sse.h | 47 ++ arch/riscv/include/asm/switch_to.h | 14 + arch/riscv/include/asm/thread_info.h | 1 + arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/asm-offsets.c | 14 + arch/riscv/kernel/sbi_sse.c | 174 +++++ arch/riscv/kernel/sbi_sse_entry.S | 178 +++++ drivers/firmware/Kconfig | 1 + drivers/firmware/Makefile | 1 + drivers/firmware/riscv/Kconfig | 15 + drivers/firmware/riscv/Makefile | 3 + drivers/firmware/riscv/riscv_sbi_sse.c | 701 ++++++++++++++++++ drivers/perf/Kconfig | 10 + drivers/perf/riscv_pmu.c | 23 + drivers/perf/riscv_pmu_sbi.c | 71 +- include/linux/perf/riscv_pmu.h | 5 + include/linux/riscv_sbi_sse.h | 57 ++ tools/testing/selftests/riscv/Makefile | 2 +- tools/testing/selftests/riscv/sse/Makefile | 5 + .../selftests/riscv/sse/module/Makefile | 16 + .../riscv/sse/module/riscv_sse_test.c | 513 +++++++++++++ .../selftests/riscv/sse/run_sse_test.sh | 44 ++ 26 files changed, 1979 insertions(+), 14 deletions(-) create mode 100644 arch/riscv/include/asm/sse.h create mode 100644 arch/riscv/kernel/sbi_sse.c create mode 100644 arch/riscv/kernel/sbi_sse_entry.S create mode 100644 drivers/firmware/riscv/Kconfig create mode 100644 drivers/firmware/riscv/Makefile create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c create mode 100644 include/linux/riscv_sbi_sse.h create mode 100644 tools/testing/selftests/riscv/sse/Makefile create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh
The SBI Supervisor Software Events (SSE) extensions provides a mechanism to inject software events from an SBI implementation to supervisor software such that it preempts all other supervisor level traps and interrupts. This extension is introduced by the SBI v3.0 specification[1]. Various events are defined and can be send asynchronously to supervisor software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well as platform specific events. Events can be either local (per-hart) or global. Events can be nested on top of each other based on priority and can interrupt the kernel at any time. First patch adds the SSE definitions. Second one adds support for SSE at arch level (entry code and stack allocations) and third one at driver level. Finally, the last patch add support for SSE events in the SBI PMU driver. Additional testing for that part is highly welcomed since there are a lot of possible path that needs to be exercised. Amongst the specific points that needs to be handle is the interruption at any point of the kernel execution and more specifically at the beginning of exception handling. Due to the fact that the exception entry implementation uses the SCRATCH CSR as both the current task struct and as the temporary register to switch the stack and save register, it is difficult to reliably get the current task struct if we get interrupted at this specific moment (ie, it might contain 0, the task pointer or tp). A fixup-like mechanism is not possible due to the nested nature of SSE which makes it really hard to obtain the original interruption site. In order to retrieve the task in a reliable manner, add an additional __sse_entry_task per_cpu array which stores the current task. Ideally, we would need to modify the way we retrieve/store the current task in exception handling so that it does not depend on the place where it's interrupted. Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are handled and does not adds any overhead to existing code. Moreover, it provides "true" NMI-like interrupts which can interrupt the kernel at any time (even in exception handling). This is particularly crucial for RAS errors which needs to be handled as fast as possible to avoid any fault propagation. A test suite is available as a separate kselftest module. In order to build it, you can use the following command: $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests Then load the module using: $ sh run_sse_test.sh A KVM SBI SSE extension implementation is available at [2]. Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1] Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2] --- Changes in v7: - Check return values of sse_on_each_cpu() - Fix typos in commit - Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT - Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK - Add tlb flush for vmap stack to avoid taking exception during sse handler upon stack access. (Alex) - Move some assembly instruction to slow path - Renamed sse.c to sbi_sse.c, ditto for other files - Renamed RISCV_SSE to RISCV_SBI_SSE - Renamed sse_event_handler to sse_event_handler_fn - Put ifdef around sse_evt in PMU SBI driver Changes in v6: - Fix comment in assembly argument - Check hart id to be the expected one in order to skip CPU id matching in sse assembly. Changes in v5: - Added a SSE test module in kselftests - Removed an unused variable - Applied checkpatch.pl --strict and fix all errors - Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock() - Fix wrong variable returned in sse_get_event - Remove useless init of events list - Remove useless empty for loop on cpus - Set sse_available as __ro_after_init - Changed a few pr_debug to pr_warn - Fix event enabled stated updated in case of failure - Change no_lock to nolock - Rename attr_buf to attr - renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed the second argument - Simplify return value in sse_event_attr_set_nolock() - Remove while loop(-EINVAL) for event cpu set call - Renamed interrupted_state_phys to interrupted_phys - Use scoped_guards/guard for sse_mutex - Remove useless struct forward declaration in sse.h - Add more explanations as to why we set SIE bit in IP - Unconditionnally set SIE in SIP - Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free() - Replace move instructions with mv - Rename NR_CPUS asm symbol to ASM_NR_CPUS - Restore SSTATUS first in sse_entry return path so that it works for double trap without any modification later. - Implement proper per cpu revert if enable/register fails Changes in v4: - Fix a bug when using per_cpu ptr for local event (Andrew) - Add sse_event_disable/enable_local() - Add pmu_disable/pmu_enable() to disable/enable SSE event - Update event ID description according to the latest spec - Fix comment about arguments in handle_sse() - Added Himanchu as a SSE reviewer - Used SYM_DATA_*() macros instead of hardcoded labels - Invoke softirqs only if not returning to kernel with irqs disabled - Remove invalid state check for write attribute function. - Remove useless bneq statement in sse_entry.S Changes in v3: - Split arch/driver support - Fix potential register failure reporting - Set a few pr_err as pr_debug - Allow CONFIG_RISCV_SSE to be disabled - Fix build without CONFIG_RISCV_SSE - Remove fixup-like mechanism and use a per-cpu array - Fixed SSCRATCH being corrupted when interrupting the kernel in early exception path. - Split SSE assembly from entry.S - Add Himanchu SSE mask/unmask and runtime PM support. - Disable user memory access/floating point/vector in SSE handler - Rebased on master v2: https://lore.kernel.org/linux-riscv/20240112111720.2975069-1-cleger@rivosinc.com/ Changes in v2: - Implemented specification v2 - Fix various error handling cases - Added shadow stack support v1: https://lore.kernel.org/linux-riscv/20231026143122.279437-1-cleger@rivosinc.com/ Clément Léger (5): riscv: add SBI SSE extension definitions riscv: add support for SBI Supervisor Software Events extension drivers: firmware: add riscv SSE support perf: RISC-V: add support for SSE event selftests/riscv: add SSE test module MAINTAINERS | 15 + arch/riscv/include/asm/asm.h | 14 +- arch/riscv/include/asm/sbi.h | 61 ++ arch/riscv/include/asm/scs.h | 7 + arch/riscv/include/asm/sse.h | 47 ++ arch/riscv/include/asm/switch_to.h | 14 + arch/riscv/include/asm/thread_info.h | 1 + arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/asm-offsets.c | 14 + arch/riscv/kernel/sbi_sse.c | 174 +++++ arch/riscv/kernel/sbi_sse_entry.S | 178 +++++ drivers/firmware/Kconfig | 1 + drivers/firmware/Makefile | 1 + drivers/firmware/riscv/Kconfig | 15 + drivers/firmware/riscv/Makefile | 3 + drivers/firmware/riscv/riscv_sbi_sse.c | 701 ++++++++++++++++++ drivers/perf/Kconfig | 10 + drivers/perf/riscv_pmu.c | 23 + drivers/perf/riscv_pmu_sbi.c | 71 +- include/linux/perf/riscv_pmu.h | 5 + include/linux/riscv_sbi_sse.h | 57 ++ tools/testing/selftests/riscv/Makefile | 2 +- tools/testing/selftests/riscv/sse/Makefile | 5 + .../selftests/riscv/sse/module/Makefile | 16 + .../riscv/sse/module/riscv_sse_test.c | 513 +++++++++++++ .../selftests/riscv/sse/run_sse_test.sh | 44 ++ 26 files changed, 1979 insertions(+), 14 deletions(-) create mode 100644 arch/riscv/include/asm/sse.h create mode 100644 arch/riscv/kernel/sbi_sse.c create mode 100644 arch/riscv/kernel/sbi_sse_entry.S create mode 100644 drivers/firmware/riscv/Kconfig create mode 100644 drivers/firmware/riscv/Makefile create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c create mode 100644 include/linux/riscv_sbi_sse.h create mode 100644 tools/testing/selftests/riscv/sse/Makefile create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh -- 2.43.0
Hi Clément, On Tue, Sep 9, 2025 at 2:19 AM Clément Léger <cleger@rivosinc.com> wrote: > > The SBI Supervisor Software Events (SSE) extensions provides a mechanism > to inject software events from an SBI implementation to supervisor > software such that it preempts all other supervisor level traps and > interrupts. This extension is introduced by the SBI v3.0 specification[1]. > > Various events are defined and can be send asynchronously to supervisor > software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well > as platform specific events. Events can be either local (per-hart) or > global. Events can be nested on top of each other based on priority and > can interrupt the kernel at any time. > > First patch adds the SSE definitions. Second one adds support for SSE > at arch level (entry code and stack allocations) and third one at driver > level. Finally, the last patch add support for SSE events in the SBI PMU > driver. Additional testing for that part is highly welcomed since there > are a lot of possible path that needs to be exercised. > > Amongst the specific points that needs to be handle is the interruption > at any point of the kernel execution and more specifically at the > beginning of exception handling. Due to the fact that the exception entry > implementation uses the SCRATCH CSR as both the current task struct and > as the temporary register to switch the stack and save register, it is > difficult to reliably get the current task struct if we get interrupted > at this specific moment (ie, it might contain 0, the task pointer or tp). > A fixup-like mechanism is not possible due to the nested nature of SSE > which makes it really hard to obtain the original interruption site. In > order to retrieve the task in a reliable manner, add an additional > __sse_entry_task per_cpu array which stores the current task. Ideally, > we would need to modify the way we retrieve/store the current task in > exception handling so that it does not depend on the place where it's > interrupted. > > Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are > handled and does not adds any overhead to existing code. Moreover, it > provides "true" NMI-like interrupts which can interrupt the kernel at > any time (even in exception handling). This is particularly crucial for > RAS errors which needs to be handled as fast as possible to avoid any > fault propagation. > > A test suite is available as a separate kselftest module. In order to > build it, you can use the following command: > > $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests > > Then load the module using: > > $ sh run_sse_test.sh > > A KVM SBI SSE extension implementation is available at [2]. > > Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1] > Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2] > > --- > > Changes in v7: > - Check return values of sse_on_each_cpu() > - Fix typos in commit > - Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT > - Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK > - Add tlb flush for vmap stack to avoid taking exception during sse > handler upon stack access. (Alex) > - Move some assembly instruction to slow path > - Renamed sse.c to sbi_sse.c, ditto for other files > - Renamed RISCV_SSE to RISCV_SBI_SSE > - Renamed sse_event_handler to sse_event_handler_fn > - Put ifdef around sse_evt in PMU SBI driver > > Changes in v6: > - Fix comment in assembly argument > - Check hart id to be the expected one in order to skip CPU id > matching in sse assembly. > > Changes in v5: > - Added a SSE test module in kselftests > - Removed an unused variable > - Applied checkpatch.pl --strict and fix all errors > - Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock() > - Fix wrong variable returned in sse_get_event > - Remove useless init of events list > - Remove useless empty for loop on cpus > - Set sse_available as __ro_after_init > - Changed a few pr_debug to pr_warn > - Fix event enabled stated updated in case of failure > - Change no_lock to nolock > - Rename attr_buf to attr > - renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed > the second argument > - Simplify return value in sse_event_attr_set_nolock() > - Remove while loop(-EINVAL) for event cpu set call > - Renamed interrupted_state_phys to interrupted_phys > - Use scoped_guards/guard for sse_mutex > - Remove useless struct forward declaration in sse.h > - Add more explanations as to why we set SIE bit in IP > - Unconditionnally set SIE in SIP > - Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free() > - Replace move instructions with mv > - Rename NR_CPUS asm symbol to ASM_NR_CPUS > - Restore SSTATUS first in sse_entry return path so that it works for > double trap without any modification later. > - Implement proper per cpu revert if enable/register fails > > Changes in v4: > - Fix a bug when using per_cpu ptr for local event (Andrew) > - Add sse_event_disable/enable_local() > - Add pmu_disable/pmu_enable() to disable/enable SSE event > - Update event ID description according to the latest spec > - Fix comment about arguments in handle_sse() > - Added Himanchu as a SSE reviewer > - Used SYM_DATA_*() macros instead of hardcoded labels > - Invoke softirqs only if not returning to kernel with irqs disabled > - Remove invalid state check for write attribute function. > - Remove useless bneq statement in sse_entry.S > > Changes in v3: > - Split arch/driver support > - Fix potential register failure reporting > - Set a few pr_err as pr_debug > - Allow CONFIG_RISCV_SSE to be disabled > - Fix build without CONFIG_RISCV_SSE > - Remove fixup-like mechanism and use a per-cpu array > - Fixed SSCRATCH being corrupted when interrupting the kernel in early > exception path. > - Split SSE assembly from entry.S > - Add Himanchu SSE mask/unmask and runtime PM support. > - Disable user memory access/floating point/vector in SSE handler > - Rebased on master > > v2: https://lore.kernel.org/linux-riscv/20240112111720.2975069-1-cleger@rivosinc.com/ > > Changes in v2: > - Implemented specification v2 > - Fix various error handling cases > - Added shadow stack support > > v1: https://lore.kernel.org/linux-riscv/20231026143122.279437-1-cleger@rivosinc.com/ > > Clément Léger (5): > riscv: add SBI SSE extension definitions > riscv: add support for SBI Supervisor Software Events extension > drivers: firmware: add riscv SSE support > perf: RISC-V: add support for SSE event > selftests/riscv: add SSE test module > > MAINTAINERS | 15 + > arch/riscv/include/asm/asm.h | 14 +- > arch/riscv/include/asm/sbi.h | 61 ++ > arch/riscv/include/asm/scs.h | 7 + > arch/riscv/include/asm/sse.h | 47 ++ > arch/riscv/include/asm/switch_to.h | 14 + > arch/riscv/include/asm/thread_info.h | 1 + > arch/riscv/kernel/Makefile | 1 + > arch/riscv/kernel/asm-offsets.c | 14 + > arch/riscv/kernel/sbi_sse.c | 174 +++++ > arch/riscv/kernel/sbi_sse_entry.S | 178 +++++ > drivers/firmware/Kconfig | 1 + > drivers/firmware/Makefile | 1 + > drivers/firmware/riscv/Kconfig | 15 + > drivers/firmware/riscv/Makefile | 3 + > drivers/firmware/riscv/riscv_sbi_sse.c | 701 ++++++++++++++++++ > drivers/perf/Kconfig | 10 + > drivers/perf/riscv_pmu.c | 23 + > drivers/perf/riscv_pmu_sbi.c | 71 +- > include/linux/perf/riscv_pmu.h | 5 + > include/linux/riscv_sbi_sse.h | 57 ++ > tools/testing/selftests/riscv/Makefile | 2 +- > tools/testing/selftests/riscv/sse/Makefile | 5 + > .../selftests/riscv/sse/module/Makefile | 16 + > .../riscv/sse/module/riscv_sse_test.c | 513 +++++++++++++ > .../selftests/riscv/sse/run_sse_test.sh | 44 ++ > 26 files changed, 1979 insertions(+), 14 deletions(-) > create mode 100644 arch/riscv/include/asm/sse.h > create mode 100644 arch/riscv/kernel/sbi_sse.c > create mode 100644 arch/riscv/kernel/sbi_sse_entry.S > create mode 100644 drivers/firmware/riscv/Kconfig > create mode 100644 drivers/firmware/riscv/Makefile > create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c These newly added files use different GPL-xxx licenses respectively: arch/riscv/kernel/sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-or-later drivers/firmware/riscv/riscv_sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-only arch/riscv/include/asm/sse.h: /* SPDX-License-Identifier: GPL-2.0-only */ Should they be consistent, or is this by design? > create mode 100644 include/linux/riscv_sbi_sse.h > create mode 100644 tools/testing/selftests/riscv/sse/Makefile > create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile > create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c > create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh > > -- > 2.43.0 > Thanks, Yunhui
On 10/27/25 02:47, yunhui cui wrote: > Hi Clément, > > On Tue, Sep 9, 2025 at 2:19 AM Clément Léger <cleger@rivosinc.com> wrote: >> >> The SBI Supervisor Software Events (SSE) extensions provides a mechanism >> to inject software events from an SBI implementation to supervisor >> software such that it preempts all other supervisor level traps and >> interrupts. This extension is introduced by the SBI v3.0 specification[1]. >> >> Various events are defined and can be send asynchronously to supervisor >> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well >> as platform specific events. Events can be either local (per-hart) or >> global. Events can be nested on top of each other based on priority and >> can interrupt the kernel at any time. >> >> First patch adds the SSE definitions. Second one adds support for SSE >> at arch level (entry code and stack allocations) and third one at driver >> level. Finally, the last patch add support for SSE events in the SBI PMU >> driver. Additional testing for that part is highly welcomed since there >> are a lot of possible path that needs to be exercised. >> >> Amongst the specific points that needs to be handle is the interruption >> at any point of the kernel execution and more specifically at the >> beginning of exception handling. Due to the fact that the exception entry >> implementation uses the SCRATCH CSR as both the current task struct and >> as the temporary register to switch the stack and save register, it is >> difficult to reliably get the current task struct if we get interrupted >> at this specific moment (ie, it might contain 0, the task pointer or tp). >> A fixup-like mechanism is not possible due to the nested nature of SSE >> which makes it really hard to obtain the original interruption site. In >> order to retrieve the task in a reliable manner, add an additional >> __sse_entry_task per_cpu array which stores the current task. Ideally, >> we would need to modify the way we retrieve/store the current task in >> exception handling so that it does not depend on the place where it's >> interrupted. >> >> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are >> handled and does not adds any overhead to existing code. Moreover, it >> provides "true" NMI-like interrupts which can interrupt the kernel at >> any time (even in exception handling). This is particularly crucial for >> RAS errors which needs to be handled as fast as possible to avoid any >> fault propagation. >> >> A test suite is available as a separate kselftest module. In order to >> build it, you can use the following command: >> >> $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests >> >> Then load the module using: >> >> $ sh run_sse_test.sh >> >> A KVM SBI SSE extension implementation is available at [2]. >> >> Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1] >> Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2] >> >> --- >> >> Changes in v7: >> - Check return values of sse_on_each_cpu() >> - Fix typos in commit >> - Rename SBI_SSE_EVENT_SIGNAL to SBI_SSE_EVENT_INJECT >> - Rename SBI_SSE_EVENT_HART_UNMASK/MASK to SBI_SSE_HART_UNMASK/MASK >> - Add tlb flush for vmap stack to avoid taking exception during sse >> handler upon stack access. (Alex) >> - Move some assembly instruction to slow path >> - Renamed sse.c to sbi_sse.c, ditto for other files >> - Renamed RISCV_SSE to RISCV_SBI_SSE >> - Renamed sse_event_handler to sse_event_handler_fn >> - Put ifdef around sse_evt in PMU SBI driver >> >> Changes in v6: >> - Fix comment in assembly argument >> - Check hart id to be the expected one in order to skip CPU id >> matching in sse assembly. >> >> Changes in v5: >> - Added a SSE test module in kselftests >> - Removed an unused variable >> - Applied checkpatch.pl --strict and fix all errors >> - Use scope_guard(cpus_read_lock) instead of manual cpus_read_lock() >> - Fix wrong variable returned in sse_get_event >> - Remove useless init of events list >> - Remove useless empty for loop on cpus >> - Set sse_available as __ro_after_init >> - Changed a few pr_debug to pr_warn >> - Fix event enabled stated updated in case of failure >> - Change no_lock to nolock >> - Rename attr_buf to attr >> - renamed sse_get_event_phys() to sse_event_get_attr_phys() and removed >> the second argument >> - Simplify return value in sse_event_attr_set_nolock() >> - Remove while loop(-EINVAL) for event cpu set call >> - Renamed interrupted_state_phys to interrupted_phys >> - Use scoped_guards/guard for sse_mutex >> - Remove useless struct forward declaration in sse.h >> - Add more explanations as to why we set SIE bit in IP >> - Unconditionnally set SIE in SIP >> - Move SSE_STACK_SIZE adjustement in sse_stack_alloc/free() >> - Replace move instructions with mv >> - Rename NR_CPUS asm symbol to ASM_NR_CPUS >> - Restore SSTATUS first in sse_entry return path so that it works for >> double trap without any modification later. >> - Implement proper per cpu revert if enable/register fails >> >> Changes in v4: >> - Fix a bug when using per_cpu ptr for local event (Andrew) >> - Add sse_event_disable/enable_local() >> - Add pmu_disable/pmu_enable() to disable/enable SSE event >> - Update event ID description according to the latest spec >> - Fix comment about arguments in handle_sse() >> - Added Himanchu as a SSE reviewer >> - Used SYM_DATA_*() macros instead of hardcoded labels >> - Invoke softirqs only if not returning to kernel with irqs disabled >> - Remove invalid state check for write attribute function. >> - Remove useless bneq statement in sse_entry.S >> >> Changes in v3: >> - Split arch/driver support >> - Fix potential register failure reporting >> - Set a few pr_err as pr_debug >> - Allow CONFIG_RISCV_SSE to be disabled >> - Fix build without CONFIG_RISCV_SSE >> - Remove fixup-like mechanism and use a per-cpu array >> - Fixed SSCRATCH being corrupted when interrupting the kernel in early >> exception path. >> - Split SSE assembly from entry.S >> - Add Himanchu SSE mask/unmask and runtime PM support. >> - Disable user memory access/floating point/vector in SSE handler >> - Rebased on master >> >> v2: https://lore.kernel.org/linux-riscv/20240112111720.2975069-1-cleger@rivosinc.com/ >> >> Changes in v2: >> - Implemented specification v2 >> - Fix various error handling cases >> - Added shadow stack support >> >> v1: https://lore.kernel.org/linux-riscv/20231026143122.279437-1-cleger@rivosinc.com/ >> >> Clément Léger (5): >> riscv: add SBI SSE extension definitions >> riscv: add support for SBI Supervisor Software Events extension >> drivers: firmware: add riscv SSE support >> perf: RISC-V: add support for SSE event >> selftests/riscv: add SSE test module >> >> MAINTAINERS | 15 + >> arch/riscv/include/asm/asm.h | 14 +- >> arch/riscv/include/asm/sbi.h | 61 ++ >> arch/riscv/include/asm/scs.h | 7 + >> arch/riscv/include/asm/sse.h | 47 ++ >> arch/riscv/include/asm/switch_to.h | 14 + >> arch/riscv/include/asm/thread_info.h | 1 + >> arch/riscv/kernel/Makefile | 1 + >> arch/riscv/kernel/asm-offsets.c | 14 + >> arch/riscv/kernel/sbi_sse.c | 174 +++++ >> arch/riscv/kernel/sbi_sse_entry.S | 178 +++++ >> drivers/firmware/Kconfig | 1 + >> drivers/firmware/Makefile | 1 + >> drivers/firmware/riscv/Kconfig | 15 + >> drivers/firmware/riscv/Makefile | 3 + >> drivers/firmware/riscv/riscv_sbi_sse.c | 701 ++++++++++++++++++ >> drivers/perf/Kconfig | 10 + >> drivers/perf/riscv_pmu.c | 23 + >> drivers/perf/riscv_pmu_sbi.c | 71 +- >> include/linux/perf/riscv_pmu.h | 5 + >> include/linux/riscv_sbi_sse.h | 57 ++ >> tools/testing/selftests/riscv/Makefile | 2 +- >> tools/testing/selftests/riscv/sse/Makefile | 5 + >> .../selftests/riscv/sse/module/Makefile | 16 + >> .../riscv/sse/module/riscv_sse_test.c | 513 +++++++++++++ >> .../selftests/riscv/sse/run_sse_test.sh | 44 ++ >> 26 files changed, 1979 insertions(+), 14 deletions(-) >> create mode 100644 arch/riscv/include/asm/sse.h >> create mode 100644 arch/riscv/kernel/sbi_sse.c >> create mode 100644 arch/riscv/kernel/sbi_sse_entry.S >> create mode 100644 drivers/firmware/riscv/Kconfig >> create mode 100644 drivers/firmware/riscv/Makefile >> create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c > > These newly added files use different GPL-xxx licenses respectively: > arch/riscv/kernel/sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-or-later > drivers/firmware/riscv/riscv_sbi_sse.c: // SPDX-License-Identifier: GPL-2.0-only > arch/riscv/include/asm/sse.h: /* SPDX-License-Identifier: GPL-2.0-only */ > Should they be consistent, or is this by design? Hi Yunhui, That is not intended, I'll fix it. Thanks, Clément > > >> create mode 100644 include/linux/riscv_sbi_sse.h >> create mode 100644 tools/testing/selftests/riscv/sse/Makefile >> create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile >> create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c >> create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh >> >> -- >> 2.43.0 >> > > Thanks, > Yunhui
Hi Clément, On Mon, 8 Sep 2025, Clément Léger wrote: > The SBI Supervisor Software Events (SSE) extensions provides a mechanism > to inject software events from an SBI implementation to supervisor > software such that it preempts all other supervisor level traps and > interrupts. This extension is introduced by the SBI v3.0 specification[1]. > > Various events are defined and can be send asynchronously to supervisor > software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well > as platform specific events. Events can be either local (per-hart) or > global. Events can be nested on top of each other based on priority and > can interrupt the kernel at any time. > > First patch adds the SSE definitions. Second one adds support for SSE > at arch level (entry code and stack allocations) and third one at driver > level. Finally, the last patch add support for SSE events in the SBI PMU > driver. Additional testing for that part is highly welcomed since there > are a lot of possible path that needs to be exercised. > > Amongst the specific points that needs to be handle is the interruption > at any point of the kernel execution and more specifically at the > beginning of exception handling. Due to the fact that the exception entry > implementation uses the SCRATCH CSR as both the current task struct and > as the temporary register to switch the stack and save register, it is > difficult to reliably get the current task struct if we get interrupted > at this specific moment (ie, it might contain 0, the task pointer or tp). > A fixup-like mechanism is not possible due to the nested nature of SSE > which makes it really hard to obtain the original interruption site. In > order to retrieve the task in a reliable manner, add an additional > __sse_entry_task per_cpu array which stores the current task. Ideally, > we would need to modify the way we retrieve/store the current task in > exception handling so that it does not depend on the place where it's > interrupted. > > Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are > handled and does not adds any overhead to existing code. Moreover, it > provides "true" NMI-like interrupts which can interrupt the kernel at > any time (even in exception handling). This is particularly crucial for > RAS errors which needs to be handled as fast as possible to avoid any > fault propagation. > > A test suite is available as a separate kselftest module. In order to > build it, you can use the following command: > > $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests > > Then load the module using: > > $ sh run_sse_test.sh > > A KVM SBI SSE extension implementation is available at [2]. > > Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1] > Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2] I updated these to apply on Linus' current master, commit 5472d60c129f, cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() suggestion, and pushed them up here to make it convenient for folks to integrate and test: https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18 Am assuming you didn't have other changes that you wanted to make; let me know if that's not the case. I noticed that you asked for folks to do additional testing, particularly of the SBI PMU driver integration, but didn't notice any additional Tested-by:s. It would be great if other folks on the list could do some focused testing now, particularly since we're on v7 of this series, and I'm sure others care about this. thanks, - Paul
Hi Paul, Thanks for caring about this series ;) On 10/10/2025 03:32, Paul Walmsley wrote: > Hi Clément, > > On Mon, 8 Sep 2025, Clément Léger wrote: > >> The SBI Supervisor Software Events (SSE) extensions provides a mechanism >> to inject software events from an SBI implementation to supervisor >> software such that it preempts all other supervisor level traps and >> interrupts. This extension is introduced by the SBI v3.0 specification[1]. >> >> Various events are defined and can be send asynchronously to supervisor >> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well >> as platform specific events. Events can be either local (per-hart) or >> global. Events can be nested on top of each other based on priority and >> can interrupt the kernel at any time. >> >> First patch adds the SSE definitions. Second one adds support for SSE >> at arch level (entry code and stack allocations) and third one at driver >> level. Finally, the last patch add support for SSE events in the SBI PMU >> driver. Additional testing for that part is highly welcomed since there >> are a lot of possible path that needs to be exercised. >> >> Amongst the specific points that needs to be handle is the interruption >> at any point of the kernel execution and more specifically at the >> beginning of exception handling. Due to the fact that the exception entry >> implementation uses the SCRATCH CSR as both the current task struct and >> as the temporary register to switch the stack and save register, it is >> difficult to reliably get the current task struct if we get interrupted >> at this specific moment (ie, it might contain 0, the task pointer or tp). >> A fixup-like mechanism is not possible due to the nested nature of SSE >> which makes it really hard to obtain the original interruption site. In >> order to retrieve the task in a reliable manner, add an additional >> __sse_entry_task per_cpu array which stores the current task. Ideally, >> we would need to modify the way we retrieve/store the current task in >> exception handling so that it does not depend on the place where it's >> interrupted. >> >> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are >> handled and does not adds any overhead to existing code. Moreover, it >> provides "true" NMI-like interrupts which can interrupt the kernel at >> any time (even in exception handling). This is particularly crucial for >> RAS errors which needs to be handled as fast as possible to avoid any >> fault propagation. >> >> A test suite is available as a separate kselftest module. In order to >> build it, you can use the following command: >> >> $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests >> >> Then load the module using: >> >> $ sh run_sse_test.sh >> >> A KVM SBI SSE extension implementation is available at [2]. >> >> Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1] >> Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2] > > I updated these to apply on Linus' current master, commit 5472d60c129f, > cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() > suggestion, and pushed them up here to make it convenient for folks to > integrate and test: > > https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18 > > Am assuming you didn't have other changes that you wanted to make; let me > know if that's not the case. Indeed, I do not have pending modification for this series. The KVM SBI SSE support will be submitted later that year. > > I noticed that you asked for folks to do additional testing, particularly > of the SBI PMU driver integration, but didn't notice any additional > Tested-by:s. It would be great if other folks on the list could do some > focused testing now, particularly since we're on v7 of this series, and > I'm sure others care about this. I would have prefer a bit more reviews and testing before going through since SSE can be quite intrusive at execution time (even though the classic IRQ path should not be impacted, I expect mostly feedback/bugs from the SSE handling path itself). We'll see in a few days if people find such problems. Thanks, Clément > > > thanks, > > - Paul >
On Mon, Oct 13, 2025 at 08:53:01AM +0200, Clément Léger wrote: ... > I would have prefer a bit more reviews and testing before going through > since SSE can be quite intrusive at execution time (even though the > classic IRQ path should not be impacted, I expect mostly feedback/bugs > from the SSE handling path itself). We'll see in a few days if people > find such problems. I've been planning to compare this series with the code we merged into kvm-unit-tests since the kvm-unit-tests series is where I put the effort into reviewing wrt the spec, etc. On a skim of this series I see several similarities with the kvm-unit-tests code so I should try to actually do that comparison exercise, but I'm not sure when I'll get the time... Thanks, drew
On Mon, Oct 13, 2025 at 12:23 PM Clément Léger <cleger@rivosinc.com> wrote: > > Hi Paul, > > Thanks for caring about this series ;) > > On 10/10/2025 03:32, Paul Walmsley wrote: > > Hi Clément, > > > > On Mon, 8 Sep 2025, Clément Léger wrote: > > > >> The SBI Supervisor Software Events (SSE) extensions provides a mechanism > >> to inject software events from an SBI implementation to supervisor > >> software such that it preempts all other supervisor level traps and > >> interrupts. This extension is introduced by the SBI v3.0 specification[1]. > >> > >> Various events are defined and can be send asynchronously to supervisor > >> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well > >> as platform specific events. Events can be either local (per-hart) or > >> global. Events can be nested on top of each other based on priority and > >> can interrupt the kernel at any time. > >> > >> First patch adds the SSE definitions. Second one adds support for SSE > >> at arch level (entry code and stack allocations) and third one at driver > >> level. Finally, the last patch add support for SSE events in the SBI PMU > >> driver. Additional testing for that part is highly welcomed since there > >> are a lot of possible path that needs to be exercised. > >> > >> Amongst the specific points that needs to be handle is the interruption > >> at any point of the kernel execution and more specifically at the > >> beginning of exception handling. Due to the fact that the exception entry > >> implementation uses the SCRATCH CSR as both the current task struct and > >> as the temporary register to switch the stack and save register, it is > >> difficult to reliably get the current task struct if we get interrupted > >> at this specific moment (ie, it might contain 0, the task pointer or tp). > >> A fixup-like mechanism is not possible due to the nested nature of SSE > >> which makes it really hard to obtain the original interruption site. In > >> order to retrieve the task in a reliable manner, add an additional > >> __sse_entry_task per_cpu array which stores the current task. Ideally, > >> we would need to modify the way we retrieve/store the current task in > >> exception handling so that it does not depend on the place where it's > >> interrupted. > >> > >> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are > >> handled and does not adds any overhead to existing code. Moreover, it > >> provides "true" NMI-like interrupts which can interrupt the kernel at > >> any time (even in exception handling). This is particularly crucial for > >> RAS errors which needs to be handled as fast as possible to avoid any > >> fault propagation. > >> > >> A test suite is available as a separate kselftest module. In order to > >> build it, you can use the following command: > >> > >> $ KDIR=<build_dir> make O=build TARGETS="riscv/sse"-j $(($(nproc)-1)) -C tools/testing/selftests > >> > >> Then load the module using: > >> > >> $ sh run_sse_test.sh > >> > >> A KVM SBI SSE extension implementation is available at [2]. > >> > >> Link: https://github.com/riscv-non-isa/riscv-sbi-doc/releases/download/v3.0-rc7/riscv-sbi.pdf [1] > >> Link: https://github.com/rivosinc/linux/tree/dev/cleger/sse_kvm [2] > > > > I updated these to apply on Linus' current master, commit 5472d60c129f, > > cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() > > suggestion, and pushed them up here to make it convenient for folks to > > integrate and test: > > > > https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18 > > > > Am assuming you didn't have other changes that you wanted to make; let me > > know if that's not the case. > > Indeed, I do not have pending modification for this series. The KVM SBI > SSE support will be submitted later that year. I am testing this series with my RAS patch set. Will update on this. Thanks Regards Himanshu > > > > > I noticed that you asked for folks to do additional testing, particularly > > of the SBI PMU driver integration, but didn't notice any additional > > Tested-by:s. It would be great if other folks on the list could do some > > focused testing now, particularly since we're on v7 of this series, and > > I'm sure others care about this. > > I would have prefer a bit more reviews and testing before going through > since SSE can be quite intrusive at execution time (even though the > classic IRQ path should not be impacted, I expect mostly feedback/bugs > from the SSE handling path itself). We'll see in a few days if people > find such problems. > > Thanks, > > Clément > > > > > > > > thanks, > > > > - Paul > > >
On Thu, 9 Oct 2025, Paul Walmsley wrote: > I updated these to apply on Linus' current master, commit 5472d60c129f, > cleaned up the checkpatch.pl --strict issues, applied Anup's pr_info() > suggestion, and pushed them up here to make it convenient for folks to > integrate and test: > > https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/?h=riscv-experimental-for-v6.18 Sorry, wrong link - here's the correct one: https://git.kernel.org/pub/scm/linux/kernel/git/pjw/riscv.git/log/?h=experimental/sse-v7 - Paul
© 2016 - 2026 Red Hat, Inc.