tests/functional: Adapt reverse_debugging to run w/o Avocado
The goal of this series is to remove Avocado as a dependency for running
the reverse_debugging functional test.
After several rounds of discussions about v1 and v2, and experiments
done by Daniel and Thomas (thanks for all the experiments and comments
so far), I've taken a new approach and moved away from using a runner
for GDB. The changes, I believe, are much simpler now.
This new series uses GDB's machine interface (MI) via the pygdbmi module
(thanks Manos and Peter for the inputs). pygdbmi provides a controller
to start GDB and communicate with it through MI, so there is no longer a
risk of version clashes between libpython in GDB and Python modules in
the pyvenv, as it could, in theory, happen when GDB executes the test
script via -x option.
Also, as Daniel pointed out, the overall test output is pretty bad and
currently does not allow one to easily follow the sequence of GDB
commands used in the test. I took this opportunity to improve the output
and it now prints the sequence in a format that can be copied and pasted
directly into GDB. For instance, the test output for STEPS = 4 is:
TAP version 13
# $ set debug remote 1
# $ target remote localhost:62464
# Remote debugging using localhost:62464
# 0x0000000040000000 in ?? ()
# $ set debug remote 0
# $ print $pc
# $1 = (void (*)()) 0x40000000
# $ stepi
# 0x0000000040000004 in ?? ()
# $ print $pc
# $2 = (void (*)()) 0x40000004
# $ stepi
# 0x0000000040000008 in ?? ()
# $ print $pc
# $3 = (void (*)()) 0x40000008
# $ stepi
# 0x000000004000000c in ?? ()
# $ print $pc
# $4 = (void (*)()) 0x4000000c
# $ stepi
# 0x0000000040000010 in ?? ()
# $ reverse-stepi
# 0x000000004000000c in ?? ()
# $ print $pc
# $5 = (void (*)()) 0x4000000c
# $ reverse-stepi
# 0x0000000040000008 in ?? ()
# $ print $pc
# $6 = (void (*)()) 0x40000008
# $ reverse-stepi
# 0x0000000040000004 in ?? ()
# $ print $pc
# $7 = (void (*)()) 0x40000004
# $ reverse-stepi
# 0x0000000040000000 in ?? ()
# $ print $pc
# $8 = (void (*)()) 0x40000000
# $ print $pc
# $9 = (void (*)()) 0x40000000
# $ stepi
# 0x0000000040000004 in ?? ()
# $ print $pc
# $10 = (void (*)()) 0x40000004
# $ stepi
# 0x0000000040000008 in ?? ()
# $ print $pc
# $11 = (void (*)()) 0x40000008
# $ stepi
# 0x000000004000000c in ?? ()
# $ print $pc
# $12 = (void (*)()) 0x4000000c
# $ stepi
# 0x0000000040000010 in ?? ()
# $ break *0x40000000
# Breakpoint 1 at 0x40000000
# $ break *0x40000004
# Breakpoint 2 at 0x40000004
# $ break *0x40000008
# Breakpoint 3 at 0x40000008
# $ break *0x4000000c
# Breakpoint 4 at 0x4000000c
# $ continue
# Continuing.
# Program received signal SIGINT, Interrupt.
# 0xffff45a2feaa2050 in ?? ()
# $ print $pc
# $13 = (void (*)()) 0xffff45a2feaa2050
# **** Hit replay-break at icount=3691561, pc=0xffff45a2feaa2050 ****
# $ reverse-continue
# Continuing.
# Breakpoint 4, 0x000000004000000c in ?? ()
# $ print $pc
# $14 = (void (*)()) 0x4000000c
# **** Hit breakpoint at the first PC in reverse order (0x4000000c) ****
ok 1 test_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt
1..1
As can be observed above, the TAP protocol is respected, and Meson
correctly displays GDB's test output in testlog-thorough.txt.
Because the pygdbmi "shim" is so thin, I had to write a trivial GDB
class around it to easily capture and print the payloads returned by its
write() method. The GDB class allows clean, single-line commands to be
used in the tests through method chaining, making them easier to follow,
for example:
pc = gdb.cli("print $pc").get_add()
The test is kept “skipped” for aarch64, ppc64, and x86_64, so it is
necessary to set QEMU_TEST_FLAKY_TESTS=1 in the test environment to
effectively run the test on these archs.
On aarch64, the test is flaky, but there is a fix that I’ve tested while
writing this series [0] that resolves it. On ppc64 and x86_64, the test
always fails: on ppc64, GDB gets a bogus PC, and on x86_64, the last
part of the test (reverse-continue) does not hit the last executed PC
(as it should) but instead jumps to the beginning of the code (first PC
in forward order).
Thus, to effectively run the reverse_debugging test on aarch64:
$ export QEMU_TEST_FLAKY_TESTS=1
$ make check-functional
or even, to run only the reverse_debug test:
$ ./pyvenv/bin/meson test --verbose --no-rebuild -t 1 --setup thorough --suite func-thorough func-aarch64-reverse_debug
Cheers,
Gustavo
v1:
https://patchew.org/QEMU/20250819143916.4138035-1-gustavo.romero@linaro.org/
v2:
https://patchew.org/QEMU/20250904154640.52687-1-gustavo.romero@linaro.org/
v3:
- Use pygdbmi instead of run-test.py or any other GDB runner
- No changes in meson.build, except to set QEMU_TEST_GDB in the test env.
- Improved test output to show all GDB commands used in the test
Gustavo Romero (4):
python: Install pygdbmi in venv
tests/functional: Provide GDB to the functional tests
tests/functional: Adapt reverse_debugging to run w/o Avocado
tests/functional: Adapt arches to reverse_debugging w/o Avocado
configure | 2 +
meson_options.txt | 2 +
.../wheels/pygdbmi-0.11.0.0-py3-none-any.whl | Bin 0 -> 21258 bytes
pythondeps.toml | 1 +
scripts/meson-buildoptions.sh | 2 +
.../functional/aarch64/test_reverse_debug.py | 13 +-
tests/functional/meson.build | 6 +
tests/functional/ppc64/test_reverse_debug.py | 15 +-
tests/functional/reverse_debugging.py | 308 +++++++++++-------
tests/functional/x86_64/test_reverse_debug.py | 19 +-
10 files changed, 217 insertions(+), 151 deletions(-)
create mode 100644 python/wheels/pygdbmi-0.11.0.0-py3-none-any.whl
--
2.34.1