RE: [for-5.0 PATCH 00/11] Support for reverse debugging with GDB

Pavel Dovgalyuk posted 11 patches 4 years, 2 months ago
Only 0 patches received!
There is a newer version of this series
RE: [for-5.0 PATCH 00/11] Support for reverse debugging with GDB
Posted by Pavel Dovgalyuk 4 years, 2 months ago
> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> Pavel Dovgalyuk <pavel.dovgaluk@gmail.com> writes:
> 
> > GDB remote protocol supports reverse debugging of the targets.
> > It includes 'reverse step' and 'reverse continue' operations.
> > The first one finds the previous step of the execution,
> > and the second one is intended to stop at the last breakpoint that
> > would happen when the program is executed normally.
> >
> > Reverse debugging is possible in the replay mode, when at least
> > one snapshot was created at the record or replay phase.
> > QEMU can use these snapshots for travelling back in time with GDB.
> >
> > Running the execution in replay mode allows using GDB reverse debugging
> > commands:
> >  - reverse-stepi (or rsi): Steps one instruction to the past.
> >    QEMU loads on of the prior snapshots and proceeds to the desired
> >    instruction forward. When that step is reaches, execution stops.
> >  - reverse-continue (or rc): Runs execution "backwards".
> >    QEMU tries to find breakpoint or watchpoint by loaded prior snapshot
> >    and replaying the execution. Then QEMU loads snapshots again and
> >    replays to the latest breakpoint. When there are no breakpoints in
> >    the examined section of the execution, QEMU finds one more snapshot
> >    and tries again. After the first snapshot is processed, execution
> >    stops at this snapshot.
> >
> > The set of patches include the following modifications:
> >  - gdbstub update for reverse debugging support
> >  - functions that automatically perform reverse step and reverse
> >    continue operations
> >  - hmp/qmp commands for manipulating the replay process
> >  - improvement of the snapshotting for saving the execution step
> >    in the snapshot parameters
> >
> > The patches are available in the repository:
> > https://github.com/ispras/qemu/tree/rr-191223
> 
> So I tried with your additional patch. Launching QEMU as:
> 
>   ./aarch64-softmmu//qemu-system-aarch64 -monitor none \
>      -display none -M virt -cpu max -display none \
>      -semihosting-config enable=on \
>      -kernel ./tests/tcg/aarch64-softmmu/memory \
>      -icount shift=5,rr=replay,rrfile=record.bin \
>      -s -S -d trace:gdbstub\*
> 
> And gdb:
> 
>   gdb tests/tcg/aarch64-softmmu/memory \
>     -ex "target remote localhost:1234"
> 
> I get the following log:
> 
>   (gdb) x/3i $pc
>   => 0x400037b0 <__start>:        adr     x0, 0x40003000 <vector_table>
>      0x400037b4 <__start+4>:      msr     vbar_el1, x0
>      0x400037b8 <__start+8>:      adrp    x0, 0x40200000
>   (gdb) p/x $x0
>   $1 = 0x0
>   (gdb) si
>   92              msr     vbar_el1, x0
>   (gdb) p/x $x0
>   $2 = 0x40003000
>   (gdb) rsi
>   warning: Remote failure reply: E14
> 
>   Program stopped.
>   __start () at /home/alex.bennee/lsrc/qemu.git/tests/tcg/aarch64/system/boot.S:92
>   92              msr     vbar_el1, x0
>   (gdb) p/x $x0
>   $3 = 0x40003000
> 
> So it doesn't seem to be working.

That's ok, you'll need at least one VM snapshot available to recover the initial VM state.
Try changing the command lines in the following way:

First, create empty.qcow2 which will be used for saving the snapshots.
Then record with initial snapshot and attached empty.qcow2:

   ./aarch64-softmmu//qemu-system-aarch64 -monitor none \
      -display none -M virt -cpu max \
      -kernel ./tests/tcg/aarch64-softmmu/memory \
      -icount shift=5,rr=record,rrfile=record.bin,rrsnapshot=init \
      -drive file=empty.qcow2

After that you can replay the execution with the support of reverse debugging:

   ./aarch64-softmmu//qemu-system-aarch64 -monitor none \
      -display none -M virt -cpu max \
      -kernel ./tests/tcg/aarch64-softmmu/memory \
      -icount shift=5,rr=replay,rrfile=record.bin,rrsnapshot=init \
      -drive file=empty.qcow2

I removed "-semihosting-config enable=on", because I'm not sure what is it for.
It may break the replay, because there is no implementation of
semihosting input record/replay.

Pavel Dovgalyuk


Re: [for-5.0 PATCH 00/11] Support for reverse debugging with GDB
Posted by Alex Bennée 4 years, 2 months ago
Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:

>> From: Alex Bennée [mailto:alex.bennee@linaro.org]
>> Pavel Dovgalyuk <pavel.dovgaluk@gmail.com> writes:
>> 
>> > GDB remote protocol supports reverse debugging of the targets.
>> > It includes 'reverse step' and 'reverse continue' operations.
>> > The first one finds the previous step of the execution,
>> > and the second one is intended to stop at the last breakpoint that
>> > would happen when the program is executed normally.
>> >
>> > Reverse debugging is possible in the replay mode, when at least
>> > one snapshot was created at the record or replay phase.
>> > QEMU can use these snapshots for travelling back in time with GDB.
>> >
>> > Running the execution in replay mode allows using GDB reverse debugging
>> > commands:
>> >  - reverse-stepi (or rsi): Steps one instruction to the past.
>> >    QEMU loads on of the prior snapshots and proceeds to the desired
>> >    instruction forward. When that step is reaches, execution stops.
>> >  - reverse-continue (or rc): Runs execution "backwards".
>> >    QEMU tries to find breakpoint or watchpoint by loaded prior snapshot
>> >    and replaying the execution. Then QEMU loads snapshots again and
>> >    replays to the latest breakpoint. When there are no breakpoints in
>> >    the examined section of the execution, QEMU finds one more snapshot
>> >    and tries again. After the first snapshot is processed, execution
>> >    stops at this snapshot.
>> >
>> > The set of patches include the following modifications:
>> >  - gdbstub update for reverse debugging support
>> >  - functions that automatically perform reverse step and reverse
>> >    continue operations
>> >  - hmp/qmp commands for manipulating the replay process
>> >  - improvement of the snapshotting for saving the execution step
>> >    in the snapshot parameters
>> >
>> > The patches are available in the repository:
>> > https://github.com/ispras/qemu/tree/rr-191223
>> 
>> So I tried with your additional patch. Launching QEMU as:
>> 
>>   ./aarch64-softmmu//qemu-system-aarch64 -monitor none \
>>      -display none -M virt -cpu max -display none \
>>      -semihosting-config enable=on \
>>      -kernel ./tests/tcg/aarch64-softmmu/memory \
>>      -icount shift=5,rr=replay,rrfile=record.bin \
>>      -s -S -d trace:gdbstub\*
>> 
>> And gdb:
>> 
>>   gdb tests/tcg/aarch64-softmmu/memory \
>>     -ex "target remote localhost:1234"
>> 
>> I get the following log:
>> 
>>   (gdb) x/3i $pc
>>   => 0x400037b0 <__start>:        adr     x0, 0x40003000 <vector_table>
>>      0x400037b4 <__start+4>:      msr     vbar_el1, x0
>>      0x400037b8 <__start+8>:      adrp    x0, 0x40200000
>>   (gdb) p/x $x0
>>   $1 = 0x0
>>   (gdb) si
>>   92              msr     vbar_el1, x0
>>   (gdb) p/x $x0
>>   $2 = 0x40003000
>>   (gdb) rsi
>>   warning: Remote failure reply: E14
>> 
>>   Program stopped.
>>   __start () at /home/alex.bennee/lsrc/qemu.git/tests/tcg/aarch64/system/boot.S:92
>>   92              msr     vbar_el1, x0
>>   (gdb) p/x $x0
>>   $3 = 0x40003000
>> 
>> So it doesn't seem to be working.
>
> That's ok, you'll need at least one VM snapshot available to recover the initial VM state.
> Try changing the command lines in the following way:
>
> First, create empty.qcow2 which will be used for saving the snapshots.
> Then record with initial snapshot and attached empty.qcow2:
>
>    ./aarch64-softmmu//qemu-system-aarch64 -monitor none \
>       -display none -M virt -cpu max \
>       -kernel ./tests/tcg/aarch64-softmmu/memory \
>       -icount shift=5,rr=record,rrfile=record.bin,rrsnapshot=init \
>       -drive file=empty.qcow2

./aarch64-softmmu//qemu-system-aarch64 -monitor none -display none -M virt -cpu max -display none -semihosting-config enable=on -kernel ./tests/tcg/aarch64-softmmu/memory -icount shift=5,rr=record,rrfile=record.bin,rrsnapshot=init -drive file=empty.qcow2
qemu-system-aarch64: invalid accelerator kvm
qemu-system-aarch64: falling back to tcg
qemu-system-aarch64: The qcow format used by node '#block163' does not support live migration
qemu-system-aarch64: Could not create snapshot for icount record

I don't know if this is down to the current state of the tree but I
haven't been able to get it working.

>
> After that you can replay the execution with the support of reverse debugging:
>
>    ./aarch64-softmmu//qemu-system-aarch64 -monitor none \
>       -display none -M virt -cpu max \
>       -kernel ./tests/tcg/aarch64-softmmu/memory \
>       -icount shift=5,rr=replay,rrfile=record.bin,rrsnapshot=init \
>       -drive file=empty.qcow2
>
> I removed "-semihosting-config enable=on", because I'm not sure what is it for.
> It may break the replay, because there is no implementation of
> semihosting input record/replay.

For this testcase semihosting in just a convenient output device (in
lieu of a serial device). We probably need to come up with a strategy on
how we handle all these devices otherwise we will end up with a random
selection of hardware combinations which work.

>
> Pavel Dovgalyuk


-- 
Alex Bennée

RE: [for-5.0 PATCH 00/11] Support for reverse debugging with GDB
Posted by Pavel Dovgalyuk 4 years, 2 months ago
> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
> 
> >> From: Alex Bennée [mailto:alex.bennee@linaro.org]
> >> Pavel Dovgalyuk <pavel.dovgaluk@gmail.com> writes:
> >>
> >> > GDB remote protocol supports reverse debugging of the targets.
> >> > It includes 'reverse step' and 'reverse continue' operations.
> >> > The first one finds the previous step of the execution,
> >> > and the second one is intended to stop at the last breakpoint that
> >> > would happen when the program is executed normally.
> >> >
> >> > Reverse debugging is possible in the replay mode, when at least
> >> > one snapshot was created at the record or replay phase.
> >> > QEMU can use these snapshots for travelling back in time with GDB.
> >> >
> >> > Running the execution in replay mode allows using GDB reverse debugging
> >> > commands:
> >> >  - reverse-stepi (or rsi): Steps one instruction to the past.
> >> >    QEMU loads on of the prior snapshots and proceeds to the desired
> >> >    instruction forward. When that step is reaches, execution stops.
> >> >  - reverse-continue (or rc): Runs execution "backwards".
> >> >    QEMU tries to find breakpoint or watchpoint by loaded prior snapshot
> >> >    and replaying the execution. Then QEMU loads snapshots again and
> >> >    replays to the latest breakpoint. When there are no breakpoints in
> >> >    the examined section of the execution, QEMU finds one more snapshot
> >> >    and tries again. After the first snapshot is processed, execution
> >> >    stops at this snapshot.
> >> >
> >> > The set of patches include the following modifications:
> >> >  - gdbstub update for reverse debugging support
> >> >  - functions that automatically perform reverse step and reverse
> >> >    continue operations
> >> >  - hmp/qmp commands for manipulating the replay process
> >> >  - improvement of the snapshotting for saving the execution step
> >> >    in the snapshot parameters
> >> >
> >> > The patches are available in the repository:
> >> > https://github.com/ispras/qemu/tree/rr-191223
> >>
> >> So I tried with your additional patch. Launching QEMU as:
> >>
> >>   ./aarch64-softmmu//qemu-system-aarch64 -monitor none \
> >>      -display none -M virt -cpu max -display none \
> >>      -semihosting-config enable=on \
> >>      -kernel ./tests/tcg/aarch64-softmmu/memory \
> >>      -icount shift=5,rr=replay,rrfile=record.bin \
> >>      -s -S -d trace:gdbstub\*
> >>
> >> And gdb:
> >>
> >>   gdb tests/tcg/aarch64-softmmu/memory \
> >>     -ex "target remote localhost:1234"
> >>
> >> I get the following log:
> >>
> >>   (gdb) x/3i $pc
> >>   => 0x400037b0 <__start>:        adr     x0, 0x40003000 <vector_table>
> >>      0x400037b4 <__start+4>:      msr     vbar_el1, x0
> >>      0x400037b8 <__start+8>:      adrp    x0, 0x40200000
> >>   (gdb) p/x $x0
> >>   $1 = 0x0
> >>   (gdb) si
> >>   92              msr     vbar_el1, x0
> >>   (gdb) p/x $x0
> >>   $2 = 0x40003000
> >>   (gdb) rsi
> >>   warning: Remote failure reply: E14
> >>
> >>   Program stopped.
> >>   __start () at /home/alex.bennee/lsrc/qemu.git/tests/tcg/aarch64/system/boot.S:92
> >>   92              msr     vbar_el1, x0
> >>   (gdb) p/x $x0
> >>   $3 = 0x40003000
> >>
> >> So it doesn't seem to be working.
> >
> > That's ok, you'll need at least one VM snapshot available to recover the initial VM state.
> > Try changing the command lines in the following way:
> >
> > First, create empty.qcow2 which will be used for saving the snapshots.
> > Then record with initial snapshot and attached empty.qcow2:
> >
> >    ./aarch64-softmmu//qemu-system-aarch64 -monitor none \
> >       -display none -M virt -cpu max \
> >       -kernel ./tests/tcg/aarch64-softmmu/memory \
> >       -icount shift=5,rr=record,rrfile=record.bin,rrsnapshot=init \
> >       -drive file=empty.qcow2
> 
> ./aarch64-softmmu//qemu-system-aarch64 -monitor none -display none -M virt -cpu max -display
> none -semihosting-config enable=on -kernel ./tests/tcg/aarch64-softmmu/memory -icount
> shift=5,rr=record,rrfile=record.bin,rrsnapshot=init -drive file=empty.qcow2


> qemu-system-aarch64: invalid accelerator kvm
> qemu-system-aarch64: falling back to tcg
> qemu-system-aarch64: The qcow format used by node '#block163' does not support live migration
> qemu-system-aarch64: Could not create snapshot for icount record

It seems that you have some problems with your disk image. Is it qcow2 or just qcow?

> For this testcase semihosting in just a convenient output device (in
> lieu of a serial device). 

I tried this test kernel with your options and everything was ok.

> We probably need to come up with a strategy on
> how we handle all these devices otherwise we will end up with a random
> selection of hardware combinations which work.

All correctly implemented virtual hardware should support record/replay.
But real semihosting (like file IO) should not, because it provides
untracked virtual machine inputs.

Pavel Dovgalyuk