From: WANG Xuerui <git@xen0n.name>

Hi,

The LoongArch-64 base architecture is capable of performing
bounds-checking either before memory accesses or alone, with specialized
instructions generating BCEs (bounds-checking error) in case of failed
assertions (ISA manual Volume 1, Sections 2.2.6.1 [1] and 2.2.10.3 [2]).
This could be useful for managed runtimes, but the exception is not
being handled so far, resulting in SIGSYSes in these cases, which is
incorrect and warrants a fix in itself.

During experimentation, it was discovered that there is already UAPI for
expressing such semantics: SIGSEGV with si_code=SEGV_BNDERR. This was
originally added for Intel MPX, and there is currently no user (!) after
the removal of MPX support a few years ago. Although the semantics is
not a 1:1 match to that of LoongArch, still it is better than
alternatives such as SIGTRAP or SIGBUS of BUS_OBJERR kind, due to being
able to convey both the value that failed assertion and the bound value.

This patch series implements just this approach: translating BCEs into
SIGSEGVs with si_code=SEGV_BNDERR, si_value set to the offending value,
and si_lower and si_upper set to resemble a range with both lower and
upper bound while in fact there is only one.

The instructions are not currently used anywhere yet in the fledgling
LoongArch ecosystem, so it's not very urgent and we could take the time
to figure out the best way forward (should SEGV_BNDERR turn out not
suitable).

[1]: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#bound-check-memory-access-instructions
[2]: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html#_asrtlegt_d

WANG Xuerui (2):
  LoongArch: Add opcodes of bounds-checking instructions
  LoongArch: Relay BCE exceptions to userland as SIGSEGVs with
    si_code=SEGV_BNDERR

 arch/loongarch/include/asm/inst.h   |  26 +++++++
 arch/loongarch/include/asm/kdebug.h |   1 +
 arch/loongarch/kernel/genex.S       |   1 +
 arch/loongarch/kernel/traps.c       | 107 ++++++++++++++++++++++++++++
 4 files changed, 135 insertions(+)

-- 
2.40.0

On Mon, 2023-04-17 at 01:33 +0800, WANG Xuerui wrote:
> From: WANG Xuerui <git@xen0n.name>
> 
> Hi,
> 
> The LoongArch-64 base architecture is capable of performing
> bounds-checking either before memory accesses or alone, with specialized
> instructions generating BCEs (bounds-checking error) in case of failed
> assertions (ISA manual Volume 1, Sections 2.2.6.1 [1] and 2.2.10.3 [2]).
> This could be useful for managed runtimes, but the exception is not
> being handled so far, resulting in SIGSYSes in these cases, which is
> incorrect and warrants a fix in itself.
> 
> During experimentation, it was discovered that there is already UAPI for
> expressing such semantics: SIGSEGV with si_code=SEGV_BNDERR. This was
> originally added for Intel MPX, and there is currently no user (!) after
> the removal of MPX support a few years ago. Although the semantics is
> not a 1:1 match to that of LoongArch, still it is better than
> alternatives such as SIGTRAP or SIGBUS of BUS_OBJERR kind, due to being
> able to convey both the value that failed assertion and the bound value.
> 
> This patch series implements just this approach: translating BCEs into
> SIGSEGVs with si_code=SEGV_BNDERR, si_value set to the offending value,
> and si_lower and si_upper set to resemble a range with both lower and
> upper bound while in fact there is only one.
> 
> The instructions are not currently used anywhere yet in the fledgling
> LoongArch ecosystem, so it's not very urgent and we could take the time
> to figure out the best way forward (should SEGV_BNDERR turn out not
> suitable).

I don't think these instructions can be used in any systematic way
within a Linux userspace in 2023.  IMO they should not exist in
LoongArch at all because they have all the same disadvantages of Intel
MPX; MPX has been removed by Intel in 2019, and LoongArch is designed
after 2019.

If we need some hardware assisted memory safety facility, an extension
similar to ARM TBI or Intel LAM would be much more useful.


Back in the old MIPS-based Loongson CPUs, similar instructions (GSLE,
GSGT, etc.) were included in LoongISA extension and the manual says they
raises "address error" when assert fails.  So SIGSEGV seems the
"backward compatible" (quoted because we absolutely don't need to
maintain any backward compatibility with old MIPS-based implementations)
thing to do.

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

On 2023/4/17 14:47, Xi Ruoyao wrote:
> On Mon, 2023-04-17 at 01:33 +0800, WANG Xuerui wrote:
>> From: WANG Xuerui <git@xen0n.name>
>>
>> Hi,
>>
>> The LoongArch-64 base architecture is capable of performing
>> bounds-checking either before memory accesses or alone, with specialized
>> instructions generating BCEs (bounds-checking error) in case of failed
>> assertions (ISA manual Volume 1, Sections 2.2.6.1 [1] and 2.2.10.3 [2]).
>> This could be useful for managed runtimes, but the exception is not
>> being handled so far, resulting in SIGSYSes in these cases, which is
>> incorrect and warrants a fix in itself.
>>
>> During experimentation, it was discovered that there is already UAPI for
>> expressing such semantics: SIGSEGV with si_code=SEGV_BNDERR. This was
>> originally added for Intel MPX, and there is currently no user (!) after
>> the removal of MPX support a few years ago. Although the semantics is
>> not a 1:1 match to that of LoongArch, still it is better than
>> alternatives such as SIGTRAP or SIGBUS of BUS_OBJERR kind, due to being
>> able to convey both the value that failed assertion and the bound value.
>>
>> This patch series implements just this approach: translating BCEs into
>> SIGSEGVs with si_code=SEGV_BNDERR, si_value set to the offending value,
>> and si_lower and si_upper set to resemble a range with both lower and
>> upper bound while in fact there is only one.
>>
>> The instructions are not currently used anywhere yet in the fledgling
>> LoongArch ecosystem, so it's not very urgent and we could take the time
>> to figure out the best way forward (should SEGV_BNDERR turn out not
>> suitable).
> 
> I don't think these instructions can be used in any systematic way
> within a Linux userspace in 2023.  IMO they should not exist in
> LoongArch at all because they have all the same disadvantages of Intel
> MPX; MPX has been removed by Intel in 2019, and LoongArch is designed
> after 2019.

Well, the difference is IMO significant enough to make LoongArch 
bounds-checking more useful, at least for certain use cases. For 
example, the bounds were a separate register bank in Intel MPX, but in 
LoongArch they are just values in GPRs. This fits naturally into 
JIT-ting or other managed runtimes (e.g. Go) whose slice indexing ops 
already bounds-check with a temporary register per bound anyway, so it's 
just a matter of this snippet (or something like it)

- calculate element address
- if address < base: goto fail
- load/calculate upper bound
- if address >= upper bound: goto fail
- access memory

becoming

- calculate element address
- asrtgt address, base - 1
- load/calculate upper bound
- {ld,st}le address, upper bound

then in SIGSEGV handler, check PC to associate the signal back with the 
exact access op; in this case, the only big problem is that LoongArch 
doesn't provide idiomatic "lower <= val" and "val < upper" semantics for 
the checked loads/stores. I've not benchmarked such memory accesses 
against plain unchecked variants, though, but I guess latency should not 
get too bad (just an extra comparison and an unlikely trap per op).

I've also looked at the other limitations described in the Wikipedia 
page for Intel MPX, and it seems majority of them are due to its use of 
a new register bank (i.e. ISA state). So I'd say the LoongArch feature 
is probably better in that regard. Other than that, I agree they are 
less useful for general memory safety that doesn't require 
application-level cooperation.

> 
> If we need some hardware assisted memory safety facility, an extension
> similar to ARM TBI or Intel LAM would be much more useful.
> 
> 
> Back in the old MIPS-based Loongson CPUs, similar instructions (GSLE,
> GSGT, etc.) were included in LoongISA extension and the manual says they
> raises "address error" when assert fails.  So SIGSEGV seems the
> "backward compatible" (quoted because we absolutely don't need to
> maintain any backward compatibility with old MIPS-based implementations)
> thing to do.

IMO we don't need to even try to keep consistency between Loongson/MIPS 
and Loongson/LoongArch UAPIs. Loongson/MIPS is effectively "on life 
support" due to non-technical reasons we have zero influence, so there's 
no reason ISVs would put out new software for it. And SIGSEGV is IMO 
appropriate no matter what the arch is (the HW exception indicates a 
real/supposed *access to the wrong location* after all), and it's 
important that we do the right thing for a new architecture.

-- 
WANG "xen0n" Xuerui

Linux/LoongArch mailing list: https://lore.kernel.org/loongarch/

On Mon, 2023-04-17 at 15:54 +0800, WANG Xuerui wrote:
> On 2023/4/17 14:47, Xi Ruoyao wrote:
> > On Mon, 2023-04-17 at 01:33 +0800, WANG Xuerui wrote:
> > > From: WANG Xuerui <git@xen0n.name>
> > > 
> > > Hi,
> > > 
> > > The LoongArch-64 base architecture is capable of performing
> > > bounds-checking either before memory accesses or alone, with specialized
> > > instructions generating BCEs (bounds-checking error) in case of failed
> > > assertions (ISA manual Volume 1, Sections 2.2.6.1 [1] and 2.2.10.3 [2]).
> > > This could be useful for managed runtimes, but the exception is not
> > > being handled so far, resulting in SIGSYSes in these cases, which is
> > > incorrect and warrants a fix in itself.
> > > 
> > > During experimentation, it was discovered that there is already UAPI for
> > > expressing such semantics: SIGSEGV with si_code=SEGV_BNDERR. This was
> > > originally added for Intel MPX, and there is currently no user (!) after
> > > the removal of MPX support a few years ago. Although the semantics is
> > > not a 1:1 match to that of LoongArch, still it is better than
> > > alternatives such as SIGTRAP or SIGBUS of BUS_OBJERR kind, due to being
> > > able to convey both the value that failed assertion and the bound value.
> > > 
> > > This patch series implements just this approach: translating BCEs into
> > > SIGSEGVs with si_code=SEGV_BNDERR, si_value set to the offending value,
> > > and si_lower and si_upper set to resemble a range with both lower and
> > > upper bound while in fact there is only one.
> > > 
> > > The instructions are not currently used anywhere yet in the fledgling
> > > LoongArch ecosystem, so it's not very urgent and we could take the time
> > > to figure out the best way forward (should SEGV_BNDERR turn out not
> > > suitable).
> > 
> > I don't think these instructions can be used in any systematic way
> > within a Linux userspace in 2023.  IMO they should not exist in
> > LoongArch at all because they have all the same disadvantages of Intel
> > MPX; MPX has been removed by Intel in 2019, and LoongArch is designed
> > after 2019.
> 
> Well, the difference is IMO significant enough to make LoongArch 
> bounds-checking more useful, at least for certain use cases. For 
> example, the bounds were a separate register bank in Intel MPX, but in
> LoongArch they are just values in GPRs. This fits naturally into 
> JIT-ting or other managed runtimes (e.g. Go) whose slice indexing ops 
> already bounds-check with a temporary register per bound anyway, so it's 
> just a matter of this snippet (or something like it)
> 
> - calculate element address
> - if address < base: goto fail
> - load/calculate upper bound
> - if address >= upper bound: goto fail
> - access memory
> 
> becoming
> 
> - calculate element address
> - asrtgt address, base - 1
> - load/calculate upper bound
> - {ld,st}le address, upper bound
> 
> then in SIGSEGV handler, check PC to associate the signal back with the 
> exact access op;

I remember using the signal handler for "usual" error handling can be a
very bad idea but I can't remember where I've read about it.  Is there
any managed environments doing so in practice?

If we redefine new_ldle/new_stle as "if [[likely]] the address is in-
bound, do the load/store and skip the next instruction; otherwise do
nothing", we can say:

blt        address, base, 1f
new_ldle.d rd, address, upperbound
1:b        panic_oob_access
xor        rd, rd, 42 // use rd to do something

This is more versatile, and useful for building a loop as well:

or            a0, r0, r0
0:new_ldle.d  t1, t0, t2
b             1f
add.d         a0, t1, a0
add.d         t0, t0, 8
b             0b
1:bl          do_something_with_the_sum

Yes it's "non-RISC", but at least more RISC than the current ldle: if
you want a trap anyway you can say

blt        address, base, 1f
new_ldle.d rd, address, upperbound
1:break    {a code defined for OOB}
xor        rd, rd, 42 // use rd

-- 
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University

Hi, Xuerui,

I hope V2 can be applied cleanly without the patch series "LoongArch:
Better backtraces", thanks.

Huacai

On Mon, Apr 17, 2023 at 5:50 PM Xi Ruoyao <xry111@xry111.site> wrote:
>
> On Mon, 2023-04-17 at 15:54 +0800, WANG Xuerui wrote:
> > On 2023/4/17 14:47, Xi Ruoyao wrote:
> > > On Mon, 2023-04-17 at 01:33 +0800, WANG Xuerui wrote:
> > > > From: WANG Xuerui <git@xen0n.name>
> > > >
> > > > Hi,
> > > >
> > > > The LoongArch-64 base architecture is capable of performing
> > > > bounds-checking either before memory accesses or alone, with specialized
> > > > instructions generating BCEs (bounds-checking error) in case of failed
> > > > assertions (ISA manual Volume 1, Sections 2.2.6.1 [1] and 2.2.10.3 [2]).
> > > > This could be useful for managed runtimes, but the exception is not
> > > > being handled so far, resulting in SIGSYSes in these cases, which is
> > > > incorrect and warrants a fix in itself.
> > > >
> > > > During experimentation, it was discovered that there is already UAPI for
> > > > expressing such semantics: SIGSEGV with si_code=SEGV_BNDERR. This was
> > > > originally added for Intel MPX, and there is currently no user (!) after
> > > > the removal of MPX support a few years ago. Although the semantics is
> > > > not a 1:1 match to that of LoongArch, still it is better than
> > > > alternatives such as SIGTRAP or SIGBUS of BUS_OBJERR kind, due to being
> > > > able to convey both the value that failed assertion and the bound value.
> > > >
> > > > This patch series implements just this approach: translating BCEs into
> > > > SIGSEGVs with si_code=SEGV_BNDERR, si_value set to the offending value,
> > > > and si_lower and si_upper set to resemble a range with both lower and
> > > > upper bound while in fact there is only one.
> > > >
> > > > The instructions are not currently used anywhere yet in the fledgling
> > > > LoongArch ecosystem, so it's not very urgent and we could take the time
> > > > to figure out the best way forward (should SEGV_BNDERR turn out not
> > > > suitable).
> > >
> > > I don't think these instructions can be used in any systematic way
> > > within a Linux userspace in 2023.  IMO they should not exist in
> > > LoongArch at all because they have all the same disadvantages of Intel
> > > MPX; MPX has been removed by Intel in 2019, and LoongArch is designed
> > > after 2019.
> >
> > Well, the difference is IMO significant enough to make LoongArch
> > bounds-checking more useful, at least for certain use cases. For
> > example, the bounds were a separate register bank in Intel MPX, but in
> > LoongArch they are just values in GPRs. This fits naturally into
> > JIT-ting or other managed runtimes (e.g. Go) whose slice indexing ops
> > already bounds-check with a temporary register per bound anyway, so it's
> > just a matter of this snippet (or something like it)
> >
> > - calculate element address
> > - if address < base: goto fail
> > - load/calculate upper bound
> > - if address >= upper bound: goto fail
> > - access memory
> >
> > becoming
> >
> > - calculate element address
> > - asrtgt address, base - 1
> > - load/calculate upper bound
> > - {ld,st}le address, upper bound
> >
> > then in SIGSEGV handler, check PC to associate the signal back with the
> > exact access op;
>
> I remember using the signal handler for "usual" error handling can be a
> very bad idea but I can't remember where I've read about it.  Is there
> any managed environments doing so in practice?
>
> If we redefine new_ldle/new_stle as "if [[likely]] the address is in-
> bound, do the load/store and skip the next instruction; otherwise do
> nothing", we can say:
>
> blt        address, base, 1f
> new_ldle.d rd, address, upperbound
> 1:b        panic_oob_access
> xor        rd, rd, 42 // use rd to do something
>
> This is more versatile, and useful for building a loop as well:
>
> or            a0, r0, r0
> 0:new_ldle.d  t1, t0, t2
> b             1f
> add.d         a0, t1, a0
> add.d         t0, t0, 8
> b             0b
> 1:bl          do_something_with_the_sum
>
> Yes it's "non-RISC", but at least more RISC than the current ldle: if
> you want a trap anyway you can say
>
> blt        address, base, 1f
> new_ldle.d rd, address, upperbound
> 1:break    {a code defined for OOB}
> xor        rd, rd, 42 // use rd
>
> --
> Xi Ruoyao <xry111@xry111.site>
> School of Aerospace Science and Technology, Xidian University

On 2023/4/20 16:36, Huacai Chen wrote:
> Hi, Xuerui,
> 
> I hope V2 can be applied cleanly without the patch series "LoongArch:
> Better backtraces", thanks.

I believe it's already the case (just try; I've moved the BADV printing 
for BCE into the better backtraces series before sending this).

I'm only waiting for comments from the other UAPI maintainers on the CC 
list.

-- 
WANG "xen0n" Xuerui

Linux/LoongArch mailing list: https://lore.kernel.org/loongarch/

On Thu, Apr 20, 2023 at 5:38 PM WANG Xuerui <kernel@xen0n.name> wrote:
>
> On 2023/4/20 16:36, Huacai Chen wrote:
> > Hi, Xuerui,
> >
> > I hope V2 can be applied cleanly without the patch series "LoongArch:
> > Better backtraces", thanks.
>
> I believe it's already the case (just try; I've moved the BADV printing
> for BCE into the better backtraces series before sending this).
>
> I'm only waiting for comments from the other UAPI maintainers on the CC
> list.
You changed arch/loongarch/include/asm/kdebug.h, does it have
something to do with UAPI?
On the other hand, kprobe has dropped the notifier mechanism, so this
piece should be changed, I think.

+ /*
+ * notify the kprobe handlers, if instruction is likely to
+ * pertain to them.
+ */
+ if (notify_die(DIE_BOUNDS_CHECK, "Bounds check error", regs, 0,
+       current->thread.trap_nr, SIGSEGV) == NOTIFY_STOP)
+ goto out;

Huacai

>
> --
> WANG "xen0n" Xuerui
>
> Linux/LoongArch mailing list: https://lore.kernel.org/loongarch/
>