To resolve the issue to debug switchable targets, this serias introduces
basic infrastructure for gdbstub and enable support for ARM and RISC-V
targets.

For example, now there is no problem to debug an big-enadian aarch64 target
on x86 host.

  $ qemu-system-aarch64 -gdb tcp::1234,endianness=big ...

Changbin Du (3):
  gdbstub: add basic infrastructure to support switchable endianness
  arm: gdbstub: add support for switchable endianness
  riscv: gdbstub: add support for switchable endianness

 configs/targets/aarch64-softmmu.mak |  1 +
 configs/targets/arm-softmmu.mak     |  1 +
 configs/targets/riscv32-softmmu.mak |  1 +
 configs/targets/riscv64-softmmu.mak |  1 +
 gdbstub.c                           | 11 +++++
 include/exec/gdbstub.h              | 72 +++++++++++++++++++++--------
 qemu-options.hx                     |  7 ++-
 softmmu/vl.c                        | 50 +++++++++++++++++++-
 target/arm/gdbstub.c                |  2 +-
 target/arm/gdbstub64.c              |  2 +-
 target/riscv/gdbstub.c              | 12 ++---
 11 files changed, 131 insertions(+), 29 deletions(-)

-- 
2.32.0

On 8/23/21 4:20 PM, Changbin Du wrote:
> To resolve the issue to debug switchable targets, this serias introduces
> basic infrastructure for gdbstub and enable support for ARM and RISC-V
> targets.
> 
> For example, now there is no problem to debug an big-enadian aarch64 target
> on x86 host.
> 
>   $ qemu-system-aarch64 -gdb tcp::1234,endianness=big ...

I don't understand why you need all that.
Maybe you aren't using gdb-multiarch?

You can install it or start it via QEMU Debian Docker image:

$ docker run -it --rm -v /tmp:/tmp -u $UID --network=host \
  registry.gitlab.com/qemu-project/qemu/qemu/debian10 \
  gdb-multiarch -q \
    --ex 'set architecture aarch64' \
    --ex 'set endian big'
The target architecture is assumed to be aarch64
The target is assumed to be big endian
(gdb) target remote 172.17.0.1:1234
(gdb)

On Mon, 23 Aug 2021 at 16:21, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
> On 8/23/21 4:20 PM, Changbin Du wrote:
> > To resolve the issue to debug switchable targets, this serias introduces
> > basic infrastructure for gdbstub and enable support for ARM and RISC-V
> > targets.
> >
> > For example, now there is no problem to debug an big-enadian aarch64 target
> > on x86 host.
> >
> >   $ qemu-system-aarch64 -gdb tcp::1234,endianness=big ...
>
> I don't understand why you need all that.
> Maybe you aren't using gdb-multiarch?
>
> You can install it or start it via QEMU Debian Docker image:
>
> $ docker run -it --rm -v /tmp:/tmp -u $UID --network=host \
>   registry.gitlab.com/qemu-project/qemu/qemu/debian10 \
>   gdb-multiarch -q \
>     --ex 'set architecture aarch64' \
>     --ex 'set endian big'
> The target architecture is assumed to be aarch64
> The target is assumed to be big endian
> (gdb) target remote 172.17.0.1:1234

I don't think that will help, because an AArch64 CPU (at least
in the boards we model) will always start up in little-endian,
and our gdbstub will always transfer register data etc in
little-endian order, because gdb cannot cope with a target that
isn't always the same endianness. Fixing this requires gdb
changes to be more capable of handling dynamic target changes
(this would also help with eg debugging across 32<->64 bit switches);
as I understand it that gdb work would be pretty significant,
and at least for aarch64 pretty much nobody cares about
big-endian, so nobody's got round to doing it yet.

Our target/ppc/gdbstub.c code takes a different tack: it
always sends register data in the same order the CPU is
currently in, which has a different set of cases when it
goes wrong.

thanks
-- PMM

On 8/23/21 5:30 PM, Peter Maydell wrote:
> On Mon, 23 Aug 2021 at 16:21, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>>
>> On 8/23/21 4:20 PM, Changbin Du wrote:
>>> To resolve the issue to debug switchable targets, this serias introduces
>>> basic infrastructure for gdbstub and enable support for ARM and RISC-V
>>> targets.
>>>
>>> For example, now there is no problem to debug an big-enadian aarch64 target
>>> on x86 host.
>>>
>>>   $ qemu-system-aarch64 -gdb tcp::1234,endianness=big ...
>>
>> I don't understand why you need all that.
>> Maybe you aren't using gdb-multiarch?
>>
>> You can install it or start it via QEMU Debian Docker image:
>>
>> $ docker run -it --rm -v /tmp:/tmp -u $UID --network=host \
>>   registry.gitlab.com/qemu-project/qemu/qemu/debian10 \
>>   gdb-multiarch -q \
>>     --ex 'set architecture aarch64' \
>>     --ex 'set endian big'
>> The target architecture is assumed to be aarch64
>> The target is assumed to be big endian
>> (gdb) target remote 172.17.0.1:1234
> 
> I don't think that will help, because an AArch64 CPU (at least
> in the boards we model) will always start up in little-endian,
> and our gdbstub will always transfer register data etc in
> little-endian order, because gdb cannot cope with a target that
> isn't always the same endianness. Fixing this requires gdb
> changes to be more capable of handling dynamic target changes
> (this would also help with eg debugging across 32<->64 bit switches);
> as I understand it that gdb work would be pretty significant,
> and at least for aarch64 pretty much nobody cares about
> big-endian, so nobody's got round to doing it yet.
> 
> Our target/ppc/gdbstub.c code takes a different tack: it
> always sends register data in the same order the CPU is
> currently in, which has a different set of cases when it
> goes wrong.

I remember having tested the 'setend be' instruction (from
https://github.com/pcrost/arm-be-test) 3 years ago using
it. Connected as little endian, set breakpoint, on BP hit
disconnect, set big-endian, reconnect, keep single-stepping
(in the same gdb session).

I doubt anything changed since.

On Mon, Aug 23, 2021 at 04:30:05PM +0100, Peter Maydell wrote:
> On Mon, 23 Aug 2021 at 16:21, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
> >
> > On 8/23/21 4:20 PM, Changbin Du wrote:
> > > To resolve the issue to debug switchable targets, this serias introduces
> > > basic infrastructure for gdbstub and enable support for ARM and RISC-V
> > > targets.
> > >
> > > For example, now there is no problem to debug an big-enadian aarch64 target
> > > on x86 host.
> > >
> > >   $ qemu-system-aarch64 -gdb tcp::1234,endianness=big ...
> >
> > I don't understand why you need all that.
> > Maybe you aren't using gdb-multiarch?
> >
> > You can install it or start it via QEMU Debian Docker image:
> >
> > $ docker run -it --rm -v /tmp:/tmp -u $UID --network=host \
> >   registry.gitlab.com/qemu-project/qemu/qemu/debian10 \
> >   gdb-multiarch -q \
> >     --ex 'set architecture aarch64' \
> >     --ex 'set endian big'
> > The target architecture is assumed to be aarch64
> > The target is assumed to be big endian
> > (gdb) target remote 172.17.0.1:1234
> 
> I don't think that will help, because an AArch64 CPU (at least
> in the boards we model) will always start up in little-endian,
> and our gdbstub will always transfer register data etc in
> little-endian order, because gdb cannot cope with a target that
> isn't always the same endianness. Fixing this requires gdb
Yes, that's the problem.

> changes to be more capable of handling dynamic target changes
> (this would also help with eg debugging across 32<->64 bit switches);
> as I understand it that gdb work would be pretty significant,
> and at least for aarch64 pretty much nobody cares about
> big-endian, so nobody's got round to doing it yet.
> 
Mostly we do not care dynamic target changes because nearly all OS will setup
endianness mode by its first instruction. And dynamic changes for gdb is hard
since the byte order of debugging info in elf is fixed. And currently the GDB
remote protocol does not support querying endianness info from remote.

So usually we needn't change byte order during a debug session, but we just want
the qemu gdbstub can send data in and handle data it received in right byte order.
This patch does this work with the help of users via the option 'endianness='.

> Our target/ppc/gdbstub.c code takes a different tack: it
> always sends register data in the same order the CPU is
> currently in, which has a different set of cases when it
> goes wrong.
>
Yes, I tried to do this before. But as I said above GDB unable to handle dynamic
target changing. Maybe we can take this way as 'endianness=default'? Anyway,
this requires each target provides a interface to determine the current byte
order.

> thanks
> -- PMM

-- 
Cheers,
Changbin Du

On Tue, 24 Aug 2021 at 00:05, Changbin Du <changbin.du@gmail.com> wrote:
>
> On Mon, Aug 23, 2021 at 04:30:05PM +0100, Peter Maydell wrote:
> > changes to be more capable of handling dynamic target changes
> > (this would also help with eg debugging across 32<->64 bit switches);
> > as I understand it that gdb work would be pretty significant,
> > and at least for aarch64 pretty much nobody cares about
> > big-endian, so nobody's got round to doing it yet.
> >
> Mostly we do not care dynamic target changes because nearly all OS will setup
> endianness mode by its first instruction. And dynamic changes for gdb is hard
> since the byte order of debugging info in elf is fixed. And currently the GDB
> remote protocol does not support querying endianness info from remote.
>
> So usually we needn't change byte order during a debug session, but we just want
> the qemu gdbstub can send data in and handle data it received in right byte order.
> This patch does this work with the help of users via the option 'endianness='.

I'm not a huge fan of putting in workarounds that deal with the
problem for specific cases and require users to tweak options settings,
rather than solving the problem in a more general way that would
let it all Just Work for all cases.

-- PMM

On Tue, Aug 24, 2021 at 10:11:14AM +0100, Peter Maydell wrote:
> On Tue, 24 Aug 2021 at 00:05, Changbin Du <changbin.du@gmail.com> wrote:
> >
> > On Mon, Aug 23, 2021 at 04:30:05PM +0100, Peter Maydell wrote:
> > > changes to be more capable of handling dynamic target changes
> > > (this would also help with eg debugging across 32<->64 bit switches);
> > > as I understand it that gdb work would be pretty significant,
> > > and at least for aarch64 pretty much nobody cares about
> > > big-endian, so nobody's got round to doing it yet.
> > >
> > Mostly we do not care dynamic target changes because nearly all OS will setup
> > endianness mode by its first instruction. And dynamic changes for gdb is hard
> > since the byte order of debugging info in elf is fixed. And currently the GDB
> > remote protocol does not support querying endianness info from remote.
> >
> > So usually we needn't change byte order during a debug session, but we just want
> > the qemu gdbstub can send data in and handle data it received in right byte order.
> > This patch does this work with the help of users via the option 'endianness='.
> 
> I'm not a huge fan of putting in workarounds that deal with the
> problem for specific cases and require users to tweak options settings,
> rather than solving the problem in a more general way that would
> let it all Just Work for all cases.
>
Probably we can add a new callback 'gdb_get_endianness' for CPUClass, and use
this callback to determine if bswap is needed every time we read/write cpu
registers. What's your thought?

> -- PMM

-- 
Cheers,
Changbin Du

On Fri, 27 Aug 2021 at 15:49, Changbin Du <changbin.du@gmail.com> wrote:
>
> On Tue, Aug 24, 2021 at 10:11:14AM +0100, Peter Maydell wrote:
> > On Tue, 24 Aug 2021 at 00:05, Changbin Du <changbin.du@gmail.com> wrote:
> > >
> > > On Mon, Aug 23, 2021 at 04:30:05PM +0100, Peter Maydell wrote:
> > > > changes to be more capable of handling dynamic target changes
> > > > (this would also help with eg debugging across 32<->64 bit switches);
> > > > as I understand it that gdb work would be pretty significant,
> > > > and at least for aarch64 pretty much nobody cares about
> > > > big-endian, so nobody's got round to doing it yet.
> > > >
> > > Mostly we do not care dynamic target changes because nearly all OS will setup
> > > endianness mode by its first instruction. And dynamic changes for gdb is hard
> > > since the byte order of debugging info in elf is fixed. And currently the GDB
> > > remote protocol does not support querying endianness info from remote.
> > >
> > > So usually we needn't change byte order during a debug session, but we just want
> > > the qemu gdbstub can send data in and handle data it received in right byte order.
> > > This patch does this work with the help of users via the option 'endianness='.
> >
> > I'm not a huge fan of putting in workarounds that deal with the
> > problem for specific cases and require users to tweak options settings,
> > rather than solving the problem in a more general way that would
> > let it all Just Work for all cases.
> >
> Probably we can add a new callback 'gdb_get_endianness' for CPUClass, and use
> this callback to determine if bswap is needed every time we read/write cpu
> registers. What's your thought?

I think that you need to start by talking to the gdb folks about
how debugging a dynamic endianness target should work. Fixing
this probably goes something like:
 * agree on design for how dynamic endianness, 32-64 mode changes,
   etc, should be handled by gdb
 * make gdb changes
 * document required gdbstub protocol enhancements (ie how the stub
   tells gdb about endianness changes, whether this changes how we
   send register information, memory data, etc)
 * implement those changes in QEMU

You seem to be trying to start with the final step, not the first one :-)

-- PMM

On Mon, Aug 23, 2021 at 05:21:07PM +0200, Philippe Mathieu-Daudé wrote:
> On 8/23/21 4:20 PM, Changbin Du wrote:
> > To resolve the issue to debug switchable targets, this serias introduces
> > basic infrastructure for gdbstub and enable support for ARM and RISC-V
> > targets.
> > 
> > For example, now there is no problem to debug an big-enadian aarch64 target
> > on x86 host.
> > 
> >   $ qemu-system-aarch64 -gdb tcp::1234,endianness=big ...
> 
> I don't understand why you need all that.
> Maybe you aren't using gdb-multiarch?
>
Nope, my gdb support all architectures.

> You can install it or start it via QEMU Debian Docker image:
> 
> $ docker run -it --rm -v /tmp:/tmp -u $UID --network=host \
>   registry.gitlab.com/qemu-project/qemu/qemu/debian10 \
>   gdb-multiarch -q \
>     --ex 'set architecture aarch64' \
>     --ex 'set endian big'
> The target architecture is assumed to be aarch64
> The target is assumed to be big endian
> (gdb) target remote 172.17.0.1:1234
> (gdb)
>
The gdb has no problem to read endianness and arch info from elf. The problem
is how qemu gdbstub handles the byte order it received.

Now let's try to debug a big-enadian aarch64 linux kernel.

1) start qemu with '-gdb tcp::1234'

$ gdb-multiarch vmlinux
(gdb) target remote :1234
Remote debugging using :1234
0x0000004000000000 in ?? ()
=> 0x0000004000000000:	Cannot access memory at address 0x4000000000
(gdb) ni
Cannot access memory at address 0x4000000000
(gdb) show architecture 
The target architecture is set to "auto" (currently "aarch64").
(gdb) show endian 
The target endianness is set automatically (currently big endian).

You see it an't work, not to mention adding breakpoints.

2) start qemu with '-gdb tcp::1234,endianness=big'

$ gdb-multiarch vmlinux
(gdb) target remote :1234
Remote debugging using :1234
0x0000000040000000 in ?? ()
=> 0x0000000040000000:	c0 00 00 58	ldr	x0, 0x40000018
(gdb) ni
0x0000000040000004 in ?? ()
=> 0x0000000040000004:	e1 03 1f aa	mov	x1, xzr
(gdb) b start_kernel
Breakpoint 1 at 0xffff800011130ee8 (2 locations)
(gdb) c
Continuing.

Thread 1 hit Breakpoint 1, 0xffff800011130ee8 in start_kernel ()
=> 0xffff800011130ee8 <start_kernel+0>:	5f 24 03 d5	bti	c
(gdb) bt
#0  0xffff800011130ee8 in start_kernel ()
#1  0xffff8000111303c8 in __primary_switched () at arch/arm64/kernel/head.S:467
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

okay, now it works fine.

-- 
Cheers,
Changbin Du

On Mon, 23 Aug 2021 at 15:20, Changbin Du <changbin.du@gmail.com> wrote:
>
> To resolve the issue to debug switchable targets, this serias introduces
> basic infrastructure for gdbstub and enable support for ARM and RISC-V
> targets.

As I understand it, fixing this problem requires support
from gdb, not merely changes to QEMU. You would need to be
able to have QEMU tell gdb "the guest's endianness is now
little/big" and have gdb cope with that change.

> For example, now there is no problem to debug an big-enadian aarch64 target
> on x86 host.
>
>   $ qemu-system-aarch64 -gdb tcp::1234,endianness=big ...

I don't feel like this is the right approach. QEMU already
knows the endianness of the guest at any point.

thanks
-- PMM