bsd-user/x86_64/target_arch_thread.h: Align stack

[PATCH] bsd-user/x86_64/target_arch_thread.h: Align stack

Posted by Ilya Leoshkevich 3 months, 3 weeks ago

bsd-user qemu-x86_64 almost immediately dies with:

    qemu: 0x4002201a68: unhandled CPU exception 0xd - aborting

on FreeBSD 14.1-RELEASE. This is an instruction that requires
alignment:

    (gdb) x/i 0x4002201a68
       0x4002201a68:        movaps %xmm0,-0x40(%rbp)

and the argument is not aligned:

    (gdb) p/x env->regs[5]
    $1 = 0x822443b58

A quick experiment shows that the userspace entry point expects
misaligned rsp:

    (gdb) starti
    (gdb) p/x $rsp
    $1 = 0x7fffffffeaa8

Emulate this behavior in bsd-user.

Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
---
 bsd-user/x86_64/target_arch_thread.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bsd-user/x86_64/target_arch_thread.h b/bsd-user/x86_64/target_arch_thread.h
index 52c28906d6d..25233443c14 100644
--- a/bsd-user/x86_64/target_arch_thread.h
+++ b/bsd-user/x86_64/target_arch_thread.h
@@ -31,7 +31,7 @@ static inline void target_thread_init(struct target_pt_regs *regs,
     struct image_info *infop)
 {
     regs->rax = 0;
-    regs->rsp = infop->start_stack;
+    regs->rsp = (infop->start_stack & ~0xfUL) - 8;
     regs->rip = infop->entry;
     regs->rdi = infop->start_stack;
 }
-- 
2.45.2

Re: [PATCH] bsd-user/x86_64/target_arch_thread.h: Align stack

Posted by Richard Henderson 3 months, 3 weeks ago

On 8/1/24 00:44, Ilya Leoshkevich wrote:
> bsd-user qemu-x86_64 almost immediately dies with:
> 
>      qemu: 0x4002201a68: unhandled CPU exception 0xd - aborting
> 
> on FreeBSD 14.1-RELEASE. This is an instruction that requires
> alignment:
> 
>      (gdb) x/i 0x4002201a68
>         0x4002201a68:        movaps %xmm0,-0x40(%rbp)
> 
> and the argument is not aligned:
> 
>      (gdb) p/x env->regs[5]
>      $1 = 0x822443b58
> 
> A quick experiment shows that the userspace entry point expects
> misaligned rsp:
> 
>      (gdb) starti
>      (gdb) p/x $rsp
>      $1 = 0x7fffffffeaa8
> 
> Emulate this behavior in bsd-user.
> 
> Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> ---
>   bsd-user/x86_64/target_arch_thread.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/bsd-user/x86_64/target_arch_thread.h b/bsd-user/x86_64/target_arch_thread.h
> index 52c28906d6d..25233443c14 100644
> --- a/bsd-user/x86_64/target_arch_thread.h
> +++ b/bsd-user/x86_64/target_arch_thread.h
> @@ -31,7 +31,7 @@ static inline void target_thread_init(struct target_pt_regs *regs,
>       struct image_info *infop)
>   {
>       regs->rax = 0;
> -    regs->rsp = infop->start_stack;
> +    regs->rsp = (infop->start_stack & ~0xfUL) - 8;

The formula in sys/amd64/amd64/exec_machdep.c, exec_setregs is

   ((stack - 8) & ~0xful) + 8

With that,
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH] bsd-user/x86_64/target_arch_thread.h: Align stack

Posted by Warner Losh 3 months, 3 weeks ago

Hmmm... All platforms likely need this....
Oh, they all have it except x86, both 32 and 64-bit.... but i386 is already
properly aligned (at least in the
FreeBSD kernel), so maybe that's not needed.


On Wed, Jul 31, 2024 at 4:43 PM Richard Henderson <
richard.henderson@linaro.org> wrote:

> On 8/1/24 00:44, Ilya Leoshkevich wrote:
> > bsd-user qemu-x86_64 almost immediately dies with:
> >
> >      qemu: 0x4002201a68: unhandled CPU exception 0xd - aborting
> >
> > on FreeBSD 14.1-RELEASE. This is an instruction that requires
> > alignment:
> >
> >      (gdb) x/i 0x4002201a68
> >         0x4002201a68:        movaps %xmm0,-0x40(%rbp)
> >
> > and the argument is not aligned:
> >
> >      (gdb) p/x env->regs[5]
> >      $1 = 0x822443b58
> >
> > A quick experiment shows that the userspace entry point expects
> > misaligned rsp:
> >
> >      (gdb) starti
> >      (gdb) p/x $rsp
> >      $1 = 0x7fffffffeaa8
> >
> > Emulate this behavior in bsd-user.
> >
> > Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
> > ---
> >   bsd-user/x86_64/target_arch_thread.h | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/bsd-user/x86_64/target_arch_thread.h
> b/bsd-user/x86_64/target_arch_thread.h
> > index 52c28906d6d..25233443c14 100644
> > --- a/bsd-user/x86_64/target_arch_thread.h
> > +++ b/bsd-user/x86_64/target_arch_thread.h
> > @@ -31,7 +31,7 @@ static inline void target_thread_init(struct
> target_pt_regs *regs,
> >       struct image_info *infop)
> >   {
> >       regs->rax = 0;
> > -    regs->rsp = infop->start_stack;
> > +    regs->rsp = (infop->start_stack & ~0xfUL) - 8;
>
> The formula in sys/amd64/amd64/exec_machdep.c, exec_setregs is
>
>    ((stack - 8) & ~0xful) + 8
>
> With that,
> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
>

Agreed.

Reviewed-by: Warner Losh <imp@bsdimp.com>

There's a *lot* that's missing on for amd64 emulation relative to the other
platforms.

Warner

Re: [PATCH] bsd-user/x86_64/target_arch_thread.h: Align stack

Posted by Warner Losh 3 months, 3 weeks ago

On Wed, Jul 31, 2024 at 5:03 PM Warner Losh <imp@bsdimp.com> wrote:

> Hmmm... All platforms likely need this....
> Oh, they all have it except x86, both 32 and 64-bit.... but i386 is
> already properly aligned (at least in the
> FreeBSD kernel), so maybe that's not needed.
>
>
> On Wed, Jul 31, 2024 at 4:43 PM Richard Henderson <
> richard.henderson@linaro.org> wrote:
>
>> On 8/1/24 00:44, Ilya Leoshkevich wrote:
>> > bsd-user qemu-x86_64 almost immediately dies with:
>> >
>> >      qemu: 0x4002201a68: unhandled CPU exception 0xd - aborting
>> >
>> > on FreeBSD 14.1-RELEASE. This is an instruction that requires
>> > alignment:
>> >
>> >      (gdb) x/i 0x4002201a68
>> >         0x4002201a68:        movaps %xmm0,-0x40(%rbp)
>> >
>> > and the argument is not aligned:
>> >
>> >      (gdb) p/x env->regs[5]
>> >      $1 = 0x822443b58
>> >
>> > A quick experiment shows that the userspace entry point expects
>> > misaligned rsp:
>> >
>> >      (gdb) starti
>> >      (gdb) p/x $rsp
>> >      $1 = 0x7fffffffeaa8
>> >
>> > Emulate this behavior in bsd-user.
>> >
>> > Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
>> > ---
>> >   bsd-user/x86_64/target_arch_thread.h | 2 +-
>> >   1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/bsd-user/x86_64/target_arch_thread.h
>> b/bsd-user/x86_64/target_arch_thread.h
>> > index 52c28906d6d..25233443c14 100644
>> > --- a/bsd-user/x86_64/target_arch_thread.h
>> > +++ b/bsd-user/x86_64/target_arch_thread.h
>> > @@ -31,7 +31,7 @@ static inline void target_thread_init(struct
>> target_pt_regs *regs,
>> >       struct image_info *infop)
>> >   {
>> >       regs->rax = 0;
>> > -    regs->rsp = infop->start_stack;
>> > +    regs->rsp = (infop->start_stack & ~0xfUL) - 8;
>>
>> The formula in sys/amd64/amd64/exec_machdep.c, exec_setregs is
>>
>>    ((stack - 8) & ~0xful) + 8
>>
>> With that,
>> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
>>
>
> Agreed.
>
> Reviewed-by: Warner Losh <imp@bsdimp.com>
>
> There's a *lot* that's missing on for amd64 emulation relative to the
> other platforms.
>

Forgot to mention that I've queued this change with Richard's suggested
modification.

Warner