meson.build | 4 + tests/functional/meson.build | 77 ++++++++- tests/functional/qemu_test/testcase.py | 1 + tests/functional/reverse_debugging.py | 158 +++++++++--------- .../functional/test_aarch64_reverse_debug.py | 19 +-- tests/functional/test_ppc64_reverse_debug.py | 17 +- tests/functional/test_x86_64_reverse_debug.py | 21 +-- tests/guest-debug/run-test.py | 81 +++++---- 8 files changed, 223 insertions(+), 155 deletions(-)
The goal of this series is to remove Avocado as a dependency for running the reverse_debugging functional test. This test, the last one I’m aware of that relies on Avocado, requires it because of the need for GDB to test reverse stepping and continue. In this series, we leveraged the run-test.py script used in the check-tcg tests, making it a GDB runner capable of calling a test script without spawning any VMs. In this configuration, the test script can manage the VMs and also import gdb from the test script, making the Python GDB API available inside the functional test. The test is kept “skipped” for aarch64, ppc64, and x86_64, so it is necessary to set QEMU_TEST_FLAKY_TESTS=1 in the test environment to effectively run the test on these archs. On aarch64, the test is flaky, but there is a fix that I’ve tested while writing this series [0] that resolves it. On ppc64 and x86_64, the test always fails: on ppc64, GDB gets a bogus PC, and on x86_64, the last part of the test (reverse-continue) does not hit the last executed PC (as it should) but instead jumps to the beginning of the code (fist PC in forward order). Thus, to run the reverse_debugging test effectively on aarch64: $ export QEMU_TEST_FLAKY_TESTS=1 $ make check-functional Cheers, Gustavo [0] https://gitlab.com/qemu-project/qemu/-/issues/2921 Gustavo Romero (4): tests/guest-debug: Make QEMU optional in run-test.py tests/functional: Support tests that require a runner tests/functional: Mark main in QemuBaseTest class as a static method tests/functional: Adapt reverse_debugging to run w/o Avocado meson.build | 4 + tests/functional/meson.build | 77 ++++++++- tests/functional/qemu_test/testcase.py | 1 + tests/functional/reverse_debugging.py | 158 +++++++++--------- .../functional/test_aarch64_reverse_debug.py | 19 +-- tests/functional/test_ppc64_reverse_debug.py | 17 +- tests/functional/test_x86_64_reverse_debug.py | 21 +-- tests/guest-debug/run-test.py | 81 +++++---- 8 files changed, 223 insertions(+), 155 deletions(-) -- 2.34.1
On 19/08/2025 16.39, Gustavo Romero wrote: > The goal of this series is to remove Avocado as a dependency for running > the reverse_debugging functional test. > > This test, the last one I’m aware of that relies on Avocado, requires it > because of the need for GDB to test reverse stepping and continue. Hi! Please note that there are currently also some efforts going on to extract the GDB part from avocado into a more self-contained python module called aautils, which might help here, too: https://github.com/avocado-framework/aautils/issues/82 > In this series, we leveraged the run-test.py script used in the > check-tcg tests, making it a GDB runner capable of calling a test script > without spawning any VMs. In this configuration, the test script can > manage the VMs and also import gdb from the test script, making the > Python GDB API available inside the functional test. > > The test is kept “skipped” for aarch64, ppc64, and x86_64, so it is > necessary to set QEMU_TEST_FLAKY_TESTS=1 in the test environment to > effectively run the test on these archs. > > On aarch64, the test is flaky, but there is a fix that I’ve tested while > writing this series [0] that resolves it. On ppc64 and x86_64, the test > always fails: on ppc64, GDB gets a bogus PC, and on x86_64, the last > part of the test (reverse-continue) does not hit the last executed PC > (as it should) but instead jumps to the beginning of the code (fist PC > in forward order). At least the x86 test used to work for me most of the time, it just fails occasionally. Same for the ppc64 test if I disable the powernv subtest there. > Thus, to run the reverse_debugging test effectively on aarch64: > > $ export QEMU_TEST_FLAKY_TESTS=1 > $ make check-functional I gave it a try, but this did not work for me, the test was not run at all anymore. Are there any patches needed on top? Anyway, shifting to a different test harness here makes me wonder whether the whole reverse_debug test should maybe be rather moved to tests/tcg instead, where we already have the basic support for the stuff from tests/guest-debug/ ? The aarch64 would require a different guest payload, of course, in that case, so not sure whether it's feasible? Thomas
On Mon, Aug 25, 2025 at 12:29:41PM +0200, Thomas Huth wrote: > On 19/08/2025 16.39, Gustavo Romero wrote: > > The goal of this series is to remove Avocado as a dependency for running > > the reverse_debugging functional test. > > > > This test, the last one I’m aware of that relies on Avocado, requires it > > because of the need for GDB to test reverse stepping and continue. > > Hi! > > Please note that there are currently also some efforts going on to extract > the GDB part from avocado into a more self-contained python module called > aautils, which might help here, too: > > https://github.com/avocado-framework/aautils/issues/82 If we did go the route of wanting the pygdb code, I'd prefer we just included it in tests/functional/qemu_test, rather than adding a dep on aautils which doesn't seem broadly relevant. Or pygdb should just be spun out as a standalone python project, as it doesn't need much of a dep on anything else. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Hello, Thomas!
On 8/25/25 07:29, Thomas Huth wrote:
> On 19/08/2025 16.39, Gustavo Romero wrote:
>> The goal of this series is to remove Avocado as a dependency for running
>> the reverse_debugging functional test.
>>
>> This test, the last one I’m aware of that relies on Avocado, requires it
>> because of the need for GDB to test reverse stepping and continue.
>
> Hi!
>
> Please note that there are currently also some efforts going on to extract the GDB part from avocado into a more self-contained python module called aautils, which might help here, too:
>
> https://github.com/avocado-framework/aautils/issues/82
Thanks, I was not aware of it. But note that the Avocado GDB API requires one
to deal with GDB packets directly, which is unnecessary for tests like reverse
debug. I think that in general, in QEMU, we should try to avoid using it as
much as possible because the tests became annoying to read without any strong
need for it.
In the commit message for 4/4 I mention that benefit of using the GDB Python
API instead.
I think we should aim to make the tests in QEMU _extremely_ easy to list, run,
and read. The original reverse_debugging test, using Avocado's GDB API, for
instance, is one example we should try to avoid. Say some developer is changing
something not related to RR but ends up breaking reverse_debugging. The developer
should at least be able to promptly run the QEMU, connect GDB, and reproduce the
issue manually following the exact commands in the test, without having to
consult the GDB protocol manual and to find out which GDB command is mapped to
which GDB packet.
In that sense I confess I'm not 100% happy with the output of reverse_debugging
yet, but one problem at a time :)
>> In this series, we leveraged the run-test.py script used in the
>> check-tcg tests, making it a GDB runner capable of calling a test script
>> without spawning any VMs. In this configuration, the test script can
>> manage the VMs and also import gdb from the test script, making the
>> Python GDB API available inside the functional test.
>>
>> The test is kept “skipped” for aarch64, ppc64, and x86_64, so it is
>> necessary to set QEMU_TEST_FLAKY_TESTS=1 in the test environment to
>> effectively run the test on these archs.
>>
>> On aarch64, the test is flaky, but there is a fix that I’ve tested while
>> writing this series [0] that resolves it. On ppc64 and x86_64, the test
>> always fails: on ppc64, GDB gets a bogus PC, and on x86_64, the last
>> part of the test (reverse-continue) does not hit the last executed PC
>> (as it should) but instead jumps to the beginning of the code (fist PC
>> in forward order).
>
> At least the x86 test used to work for me most of the time, it just fails occasionally. Same for the ppc64 test if I disable the powernv subtest there.
ok, I'll double check the state on x86 in master. I understand we're not
testing reverse_debugging in our CI currently?
>> Thus, to run the reverse_debugging test effectively on aarch64:
>>
>> $ export QEMU_TEST_FLAKY_TESTS=1
>> $ make check-functional
>
> I gave it a try, but this did not work for me, the test was not run at all anymore. Are there any patches needed on top?
hmm that's odd. I'm able to run it with 'make check-functional' and with 'meson test'...
This is how I'm running it (let me know if I'm missing something):
gromero@gromero0:/mnt/git/qemu_$ b4 shazam 20250819143916.4138035-1-gustavo.romero@linaro.org
Grabbing thread from lore.kernel.org/all/20250819143916.4138035-1-gustavo.romero@linaro.org/t.mbox.gz
Checking for newer revisions
Grabbing search results from lore.kernel.org
Analyzing 9 messages in the thread
Looking for additional code-review trailers on lore.kernel.org
Analyzing 0 code-review messages
Checking attestation on all messages, may take a moment...
---
✓ [PATCH 1/4] tests/guest-debug: Make QEMU optional in run-test.py
✓ [PATCH 2/4] tests/functional: Support tests that require a runner
✓ [PATCH 3/4] tests/functional: Mark main in QemuBaseTest class as a static method
+ Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (✓ DKIM/redhat.com)
✓ [PATCH 4/4] tests/functional: Adapt reverse_debugging to run w/o Avocado
---
✓ Signed: DKIM/linaro.org
---
Total patches: 4
---
Applying: tests/guest-debug: Make QEMU optional in run-test.py
Applying: tests/functional: Support tests that require a runner
Applying: tests/functional: Mark main in QemuBaseTest class as a static method
Applying: tests/functional: Adapt reverse_debugging to run w/o Avocado
gromero@gromero0:/mnt/git/qemu_$
gromero@gromero0:/mnt/git/qemu_/build$ ../configure --target-list=aarch64-softmmu --disable-docs
gromero@gromero0:/mnt/git/qemu_/build$ make -j 32
gromero@gromero0:/mnt/git/qemu_/build$ time make -j 15 check-functional
[1/2] Generating tests/functional/func-precache-aarch64-aarch64_reverse_debug with a custom command (wrapped by meson to set env)
2025-08-25 12:50:04,215 - qemu-test - INFO - Attempting to cache '/home/gromero/.cache/qemu/download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7'
2025-08-25 12:50:04,225 - qemu-test - DEBUG - Using cached asset /home/gromero/.cache/qemu/download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7 for https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/Everything/aarch64/os/images/pxeboot/vmlinuz
GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination off' -ex 'set confirm off' -ex "py sys.argv=['/mnt/git/qemu_/tests/functional/test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/test_aarch64_reverse_debug.py
[0/1] Running external command precache-functional (wrapped by meson to set env)
make[1]: Entering directory '/mnt/git/qemu_/build'
[1/6] Generating qemu-version.h with a custom command (wrapped by meson to capture output)
/mnt/git/qemu_/build/pyvenv/bin/meson test --no-rebuild -t 1 --setup thorough --num-processes 10 --print-errorlogs --suite func --suite func-quick --suite func-thorough
1/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_virt_gpu SKIP 1.95s 0 subtests passed
2/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_smmu SKIP 4.17s 0 subtests passed
3/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_raspi4 OK 28.11s 2 subtests passed
4/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_device_passthrough OK 31.34s 1 subtests passed
5/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_reverse_debug OK 11.73s <------ 'exitcode', not TAP, so no subtests-like output
6/27 qemu:func-quick+func-aarch64 / func-aarch64-migration OK 0.97s 2 subtests passed
7/27 qemu:func-quick+func-aarch64 / func-aarch64-empty_cpu_model OK 0.17s 1 subtests passed
[...]
gromero@gromero0:/mnt/git/qemu_/build$ grep -A 58 "= 5/27 =" /mnt/git/qemu_/build/meson-logs/testlog-thorough.txt
==================================== 5/27 ====================================
test: qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_reverse_debug
start time: 13:44:21
duration: 11.73s
result: exit status 0
command: LD_LIBRARY_PATH=/mnt/git/qemu_/build/contrib/plugins:/mnt/git/qemu_/build/tests/tcg/plugins QEMU_BUILD_ROOT=/mnt/git/qemu_/build MALLOC_PERTURB_=138 G_TEST_SLOW=1 ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 RUST_BACKTRACE=1 QEMU_TEST_QEMU_IMG=/mnt/git/qemu_/build/qemu-img MESON_TEST_ITERATION=1 PYTHONPATH=/mnt/git/qemu_/python:/mnt/git/qemu_/tests/functional:/mnt/git/qemu_/build/pyvenv/lib/python3.10/site-packages QEMU_TEST_QEMU_BINARY=/mnt/git/qemu_/build/qemu-system-aarch64 UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 /mnt/git/qemu_/build/pyvenv/bin/python3 /mnt/git/qemu_/tests/guest-debug/run-test.py --gdb /usr/bin/gdb-multiarch --test /mnt/git/qemu_/tests/functional/test_aarch64_reverse_debug.py
----------------------------------- stdout -----------------------------------
Formatting '/mnt/git/qemu_/build/tests/functional/aarch64/test_aarch64_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt/scratch/disk.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
TAP version 13
0x0000000040000000 in ?? ()
0x0000000040000004 in ?? ()
0x0000000040000008 in ?? ()
0x000000004000000c in ?? ()
0x0000000040000010 in ?? ()
0x0000000040000014 in ?? ()
0x0000000040080000 in ?? ()
0x0000000040080004 in ?? ()
0x00000000410d0000 in ?? ()
0x00000000410d0020 in ?? ()
0x00000000410d0024 in ?? ()
0x00000000410d0020 in ?? ()
0x00000000410d0000 in ?? ()
0x0000000040080004 in ?? ()
0x0000000040080000 in ?? ()
0x0000000040000014 in ?? ()
0x0000000040000010 in ?? ()
0x000000004000000c in ?? ()
0x0000000040000008 in ?? ()
0x0000000040000004 in ?? ()
0x0000000040000000 in ?? ()
0x0000000040000004 in ?? ()
0x0000000040000008 in ?? ()
0x000000004000000c in ?? ()
0x0000000040000010 in ?? ()
0x0000000040000014 in ?? ()
0x0000000040080000 in ?? ()
0x0000000040080004 in ?? ()
0x00000000410d0000 in ?? ()
0x00000000410d0020 in ?? ()
0x00000000410d0024 in ?? ()
Breakpoint 1 at 0x40000000
Breakpoint 2 at 0x40000004
Breakpoint 3 at 0x40000008
Breakpoint 4 at 0x4000000c
Breakpoint 5 at 0x40000010
Breakpoint 6 at 0x40000014
Breakpoint 7 at 0x40080000
Breakpoint 8 at 0x40080004
Breakpoint 9 at 0x410d0000
Breakpoint 10 at 0x410d0020
Program received signal SIGINT, Interrupt.
0xffff3e25348a2050 in ?? ()
Breakpoint 10, 0x00000000410d0020 in ?? ()
ok 1 test_aarch64_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt
1..1
GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination off' -ex 'set confirm off' -ex "py sys.argv=['/mnt/git/qemu_/tests/functional/test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/test_aarch64_reverse_debug.py
==============================================================================
gromero@gromero0:/mnt/git/qemu_/build$
Also, using meson test:
gromero@gromero0:/mnt/git/qemu_/build$ cat /mnt/git/qemu_/build/meson-logs/testlog-thorough.txt
Log of Meson test suite run on 2025-08-25T13:54:13.544507
Inherited environment: SHELL=/bin/bash PYENV_SHELL=bash LC_ADDRESS=en_US.UTF-8 LC_NAME=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 PWD=/mnt/git/qemu_/build LOGNAME=gromero XDG_SESSION_TYPE=tty TVM_HOME=/home/gromero/git/tvm MOTD_SHOWN=pam HOME=/home/gromero LANG=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:' SSH_CONNECTION='192.168.122.1 57300 192.168.122.158 22' LESSCLOSE='/usr/bin/lesspipe %s %s' XDG_SESSION_CLASS=user PYTHONPATH=/home/gromero/git/tvm/python: TERM=xterm-256color LC_IDENTIFICATION=en_US.UTF-8 LESSOPEN='| /usr/bin/lesspipe %s' QEMU_TEST_FLAKY_TESTS=1 USER=gromero SHLVL=1 LC_TELEPHONE=en_US.UTF-8 LC_MEASUREMENT=en_US.UTF-8 XDG_SESSION_ID=2738 XDG_RUNTIME_DIR=/run/user/1000 SSH_CLIENT='192.168.122.1 57300 22' PYENV_ROOT=/home/gromero/.pyenv LC_TIME=en_US.UTF-8 XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop PATH=/home/gromero/.pyenv/shims:/home/gromero/.pyenv/bin:/home/gromero/.local/bin:/home/gromero/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus SSH_TTY=/dev/pts/0 LC_NUMERIC=en_US.UTF-8 OLDPWD=/mnt/git/qemu_ _=/mnt/git/qemu_rr_fix/build/pyvenv/bin/meson
==================================== 1/1 =====================================
test: qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_reverse_debug
start time: 13:54:13
duration: 2.65s
result: exit status 0
command: PYTHONPATH=/mnt/git/qemu_/python:/mnt/git/qemu_/tests/functional:/mnt/git/qemu_/build/pyvenv/lib/python3.10/site-packages MSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 LD_LIBRARY_PATH=/mnt/git/qemu_/build/contrib/plugins:/mnt/git/qemu_/build/tests/tcg/plugins RUST_BACKTRACE=1 MALLOC_PERTURB_=243 ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 QEMU_TEST_QEMU_BINARY=/mnt/git/qemu_/build/qemu-system-aarch64 QEMU_BUILD_ROOT=/mnt/git/qemu_/build UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 G_TEST_SLOW=1 MESON_TEST_ITERATION=1 QEMU_TEST_QEMU_IMG=/mnt/git/qemu_/build/qemu-img SPEED=thorough /mnt/git/qemu_/build/pyvenv/bin/python3 /mnt/git/qemu_/tests/guest-debug/run-test.py --gdb /usr/bin/gdb-multiarch --test /mnt/git/qemu_/tests/functional/test_aarch64_reverse_debug.py
----------------------------------- stdout -----------------------------------
Formatting '/mnt/git/qemu_/build/tests/functional/aarch64/test_aarch64_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt/scratch/disk.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=134217728 lazy_refcounts=off refcount_bits=16
TAP version 13
0x0000000040000000 in ?? ()
0x0000000040000004 in ?? ()
0x0000000040000008 in ?? ()
0x000000004000000c in ?? ()
0x0000000040000010 in ?? ()
0x0000000040000014 in ?? ()
0x0000000040080000 in ?? ()
0x0000000040080004 in ?? ()
0x00000000410d0000 in ?? ()
0x00000000410d0020 in ?? ()
0x00000000410d0024 in ?? ()
0x00000000410d0020 in ?? ()
0x00000000410d0000 in ?? ()
0x0000000040080004 in ?? ()
0x0000000040080000 in ?? ()
0x0000000040000014 in ?? ()
0x0000000040000010 in ?? ()
0x000000004000000c in ?? ()
0x0000000040000008 in ?? ()
0x0000000040000004 in ?? ()
0x0000000040000000 in ?? ()
0x0000000040000004 in ?? ()
0x0000000040000008 in ?? ()
0x000000004000000c in ?? ()
0x0000000040000010 in ?? ()
0x0000000040000014 in ?? ()
0x0000000040080000 in ?? ()
0x0000000040080004 in ?? ()
0x00000000410d0000 in ?? ()
0x00000000410d0020 in ?? ()
0x00000000410d0024 in ?? ()
Breakpoint 1 at 0x40000000
Breakpoint 2 at 0x40000004
Breakpoint 3 at 0x40000008
Breakpoint 4 at 0x4000000c
Breakpoint 5 at 0x40000010
Breakpoint 6 at 0x40000014
Breakpoint 7 at 0x40080000
Breakpoint 8 at 0x40080004
Breakpoint 9 at 0x410d0000
Breakpoint 10 at 0x410d0020
Program received signal SIGINT, Interrupt.
0xffff47bcc84a2050 in ?? ()
Breakpoint 10, 0x00000000410d0020 in ?? ()
ok 1 test_aarch64_reverse_debug.ReverseDebugging_AArch64.test_aarch64_virt
1..1
GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination off' -ex 'set confirm off' -ex "py sys.argv=['/mnt/git/qemu_/tests/functional/test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/test_aarch64_reverse_debug.py
==============================================================================
Ok: 1
Fail: 0
gromero@gromero0:/mnt/git/qemu_/build$
Any chance you got confused by no subtests-like output caused by 'exitcode'
used instead of ´tap'?
> Anyway, shifting to a different test harness here makes me wonder whether the whole reverse_debug test should maybe be rather moved to tests/tcg instead, where we already have the basic support for the stuff from tests/guest-debug/ ?
> The aarch64 would require a different guest payload, of course, in that case, so not sure whether it's feasible?
I think reverse_debugging is really a functional test. It requires GDB, yes,
but also QMP and booting a whole kernel and the feature itself makes me think
it's a functional test. I wouldn't move it to tcg-check just for the sake of
no adding a new way to run test in meson.build in functional tests.
Cheers,
Gustavo
On Mon, Aug 25, 2025 at 11:04:58AM -0300, Gustavo Romero wrote: > Hello, Thomas! > > On 8/25/25 07:29, Thomas Huth wrote: > > On 19/08/2025 16.39, Gustavo Romero wrote: > > > The goal of this series is to remove Avocado as a dependency for running > > > the reverse_debugging functional test. > > > > > > This test, the last one I’m aware of that relies on Avocado, requires it > > > because of the need for GDB to test reverse stepping and continue. > > > > Hi! > > > > Please note that there are currently also some efforts going on to extract the GDB part from avocado into a more self-contained python module called aautils, which might help here, too: > > > > https://github.com/avocado-framework/aautils/issues/82 > > Thanks, I was not aware of it. But note that the Avocado GDB API requires one > to deal with GDB packets directly, which is unnecessary for tests like reverse > debug. I think that in general, in QEMU, we should try to avoid using it as > much as possible because the tests became annoying to read without any strong > need for it. > > In the commit message for 4/4 I mention that benefit of using the GDB Python > API instead. > > I think we should aim to make the tests in QEMU _extremely_ easy to list, run, > and read. Yes, I really like how your patch improves readability of the test by using GDB commands rather than raw packets. > > Anyway, shifting to a different test harness here makes me wonder whether the whole reverse_debug test should maybe be rather moved to tests/tcg instead, where we already have the basic support for the stuff from tests/guest-debug/ ? > > The aarch64 would require a different guest payload, of course, in that case, so not sure whether it's feasible? > > I think reverse_debugging is really a functional test. It requires GDB, yes, > but also QMP and booting a whole kernel and the feature itself makes me think > it's a functional test. I wouldn't move it to tcg-check just for the sake of > no adding a new way to run test in meson.build in functional tests. Agreed, this conceptually feels like it belongs as a functional test. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 25/08/2025 16.04, Gustavo Romero wrote: ... >> Anyway, shifting to a different test harness here makes me wonder whether >> the whole reverse_debug test should maybe be rather moved to tests/tcg >> instead, where we already have the basic support for the stuff from tests/ >> guest-debug/ ? >> The aarch64 would require a different guest payload, of course, in that >> case, so not sure whether it's feasible? > > I think reverse_debugging is really a functional test. It requires GDB, yes, > but also QMP and booting a whole kernel and the feature itself makes me think > it's a functional test. I wouldn't move it to tcg-check just for the sake of > no adding a new way to run test in meson.build in functional tests. Ok, fair point. But thinking about this twice, wouldn't it be nicer to keep the current test harness and simply launch the script from the test_* methods instead? You could redirect stdout of the script into a log file, so that it cannot mess with the TAP output of the test. Or is there something that I missed and this cannot work? Thomas
Hi Thomas, On 8/26/25 05:06, Thomas Huth wrote: > On 25/08/2025 16.04, Gustavo Romero wrote: > ... >>> Anyway, shifting to a different test harness here makes me wonder whether the whole reverse_debug test should maybe be rather moved to tests/tcg instead, where we already have the basic support for the stuff from tests/ guest-debug/ ? >>> The aarch64 would require a different guest payload, of course, in that case, so not sure whether it's feasible? >> >> I think reverse_debugging is really a functional test. It requires GDB, yes, >> but also QMP and booting a whole kernel and the feature itself makes me think >> it's a functional test. I wouldn't move it to tcg-check just for the sake of >> no adding a new way to run test in meson.build in functional tests. > > Ok, fair point. But thinking about this twice, wouldn't it be nicer to keep the current test harness and simply launch the script from the test_* methods instead? If no additional module is used, then GDB needs to execute the test_* scripts via '-x' option because only this way the script can do "import gdb", hence we need a runner that will call gdb and pass the script to it. Naturally, we can instead use a module like pygdbmi (as Manos suggested) or aautils (with the caveats I mentioned about having to deal with GDB packets directly) that will simply wrap up calling gdb and passing it the script with subprocess etc., so no need to augment the test harness. The trade off I can see is: re-use the runner we use in check-tcg and augment the test harness to accommodate it in the functional tests _vs_ use the "runner" from another module, bring another python dependency and deal with another GDB API (I like using GDB native Python API because it's simple, sufficient and self-contained, but I'm open to experiment with pygdbmi), use another runner (other than the one in tests/guest-debug/test_gdbstub.py) and don't modify the test harness. > You could redirect stdout of the script into a log file, so that it cannot mess with the TAP output of the test. Or is there something that I missed and this cannot work? Yeah, I really tried it but redirecting to a file using GDB Python API didn't work :-( I've tried to redirect GDB's output from the test script using "set logging redirect on" but something is off. I even tried on GDB master, but besides direct not taking effect (output is still printed to the stdout) I got a GDB internal error when switching it off. So, yes, it would work (even adding a # in the output messages so the TAP protocol would simply ignore the lines at the same time we would be able to see the output from GDB), but I was not able to make it work with GDB native Python API, so turned to use the "exitcode". Cheers, Gustavo
On 25/08/2025 16.04, Gustavo Romero wrote: > Hello, Thomas! > > On 8/25/25 07:29, Thomas Huth wrote: >> On 19/08/2025 16.39, Gustavo Romero wrote: >>> The goal of this series is to remove Avocado as a dependency for running >>> the reverse_debugging functional test. >>> >>> This test, the last one I’m aware of that relies on Avocado, requires it >>> because of the need for GDB to test reverse stepping and continue. ... >> I gave it a try, but this did not work for me, the test was not run at all >> anymore. Are there any patches needed on top? > > hmm that's odd. I'm able to run it with 'make check-functional' and with > 'meson test'... > > This is how I'm running it (let me know if I'm missing something): ... > gromero@gromero0:/mnt/git/qemu_$ > gromero@gromero0:/mnt/git/qemu_/build$ ../configure --target-list=aarch64- > softmmu --disable-docs > gromero@gromero0:/mnt/git/qemu_/build$ make -j 32 > gromero@gromero0:/mnt/git/qemu_/build$ time make -j 15 check-functional > [1/2] Generating tests/functional/func-precache-aarch64- > aarch64_reverse_debug with a custom command (wrapped by meson to set env) > 2025-08-25 12:50:04,215 - qemu-test - INFO - Attempting to cache '/home/ > gromero/.cache/qemu/ > download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7' > 2025-08-25 12:50:04,225 - qemu-test - DEBUG - Using cached asset /home/ > gromero/.cache/qemu/ > download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7 > for https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/ > Everything/aarch64/os/images/pxeboot/vmlinuz > GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination off' -ex > 'set confirm off' -ex "py sys.argv=['/mnt/git/qemu_/tests/functional/ > test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/ > test_aarch64_reverse_debug.py > [0/1] Running external command precache-functional (wrapped by meson to set > env) > make[1]: Entering directory '/mnt/git/qemu_/build' > [1/6] Generating qemu-version.h with a custom command (wrapped by meson to > capture output) > /mnt/git/qemu_/build/pyvenv/bin/meson test --no-rebuild -t 1 --setup > thorough --num-processes 10 --print-errorlogs --suite func --suite func- > quick --suite func-thorough > 1/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64- > aarch64_virt_gpu SKIP 1.95s 0 subtests passed I tried a couple of times now, and finally realized that it's the "gdb-multiarch" binary that is missing on Fedora. And as far as I can see, there is also no package that provides this on Fedora? So if we go ahead with your patches, this test will only run on certain distros that provide this binary. Thomas
On Tue, Aug 26, 2025 at 09:51:27AM +0200, Thomas Huth wrote: > On 25/08/2025 16.04, Gustavo Romero wrote: > > Hello, Thomas! > > > > On 8/25/25 07:29, Thomas Huth wrote: > > > On 19/08/2025 16.39, Gustavo Romero wrote: > > > > The goal of this series is to remove Avocado as a dependency for running > > > > the reverse_debugging functional test. > > > > > > > > This test, the last one I’m aware of that relies on Avocado, requires it > > > > because of the need for GDB to test reverse stepping and continue. > ... > > > I gave it a try, but this did not work for me, the test was not run > > > at all anymore. Are there any patches needed on top? > > > > hmm that's odd. I'm able to run it with 'make check-functional' and with > > 'meson test'... > > > > This is how I'm running it (let me know if I'm missing something): > ... > > gromero@gromero0:/mnt/git/qemu_$ > > gromero@gromero0:/mnt/git/qemu_/build$ ../configure > > --target-list=aarch64- softmmu --disable-docs > > gromero@gromero0:/mnt/git/qemu_/build$ make -j 32 > > gromero@gromero0:/mnt/git/qemu_/build$ time make -j 15 check-functional > > [1/2] Generating tests/functional/func-precache-aarch64- > > aarch64_reverse_debug with a custom command (wrapped by meson to set > > env) > > 2025-08-25 12:50:04,215 - qemu-test - INFO - Attempting to cache '/home/ > > gromero/.cache/qemu/ > > download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7' > > 2025-08-25 12:50:04,225 - qemu-test - DEBUG - Using cached asset /home/ > > gromero/.cache/qemu/ > > download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7 > > for > > https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/ > > Everything/aarch64/os/images/pxeboot/vmlinuz > > GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination off' > > -ex 'set confirm off' -ex "py > > sys.argv=['/mnt/git/qemu_/tests/functional/ > > test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/ > > test_aarch64_reverse_debug.py > > [0/1] Running external command precache-functional (wrapped by meson to > > set env) > > make[1]: Entering directory '/mnt/git/qemu_/build' > > [1/6] Generating qemu-version.h with a custom command (wrapped by meson > > to capture output) > > /mnt/git/qemu_/build/pyvenv/bin/meson test --no-rebuild -t 1 --setup > > thorough --num-processes 10 --print-errorlogs --suite func --suite > > func- quick --suite func-thorough > > 1/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64- > > aarch64_virt_gpu SKIP 1.95s 0 subtests > > passed > > I tried a couple of times now, and finally realized that it's the > "gdb-multiarch" binary that is missing on Fedora. And as far as I can see, > there is also no package that provides this on Fedora? So if we go ahead > with your patches, this test will only run on certain distros that provide > this binary. 'gdb-multiarch' is a command name invented by Debian. On Fedora, the regular 'gdb' binary is built with support for multiple architectures for the purpose of remote debugging. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On Tue, Aug 26, 2025 at 03:10:42PM +0100, Daniel P. Berrangé wrote: > On Tue, Aug 26, 2025 at 09:51:27AM +0200, Thomas Huth wrote: > > On 25/08/2025 16.04, Gustavo Romero wrote: > > > Hello, Thomas! > > > > > > On 8/25/25 07:29, Thomas Huth wrote: > > > > On 19/08/2025 16.39, Gustavo Romero wrote: > > > > > The goal of this series is to remove Avocado as a dependency for running > > > > > the reverse_debugging functional test. > > > > > > > > > > This test, the last one I’m aware of that relies on Avocado, requires it > > > > > because of the need for GDB to test reverse stepping and continue. > > ... > > > > I gave it a try, but this did not work for me, the test was not run > > > > at all anymore. Are there any patches needed on top? > > > > > > hmm that's odd. I'm able to run it with 'make check-functional' and with > > > 'meson test'... > > > > > > This is how I'm running it (let me know if I'm missing something): > > ... > > > gromero@gromero0:/mnt/git/qemu_$ > > > gromero@gromero0:/mnt/git/qemu_/build$ ../configure > > > --target-list=aarch64- softmmu --disable-docs > > > gromero@gromero0:/mnt/git/qemu_/build$ make -j 32 > > > gromero@gromero0:/mnt/git/qemu_/build$ time make -j 15 check-functional > > > [1/2] Generating tests/functional/func-precache-aarch64- > > > aarch64_reverse_debug with a custom command (wrapped by meson to set > > > env) > > > 2025-08-25 12:50:04,215 - qemu-test - INFO - Attempting to cache '/home/ > > > gromero/.cache/qemu/ > > > download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7' > > > 2025-08-25 12:50:04,225 - qemu-test - DEBUG - Using cached asset /home/ > > > gromero/.cache/qemu/ > > > download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7 > > > for > > > https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/ > > > Everything/aarch64/os/images/pxeboot/vmlinuz > > > GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination off' > > > -ex 'set confirm off' -ex "py > > > sys.argv=['/mnt/git/qemu_/tests/functional/ > > > test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/ > > > test_aarch64_reverse_debug.py > > > [0/1] Running external command precache-functional (wrapped by meson to > > > set env) > > > make[1]: Entering directory '/mnt/git/qemu_/build' > > > [1/6] Generating qemu-version.h with a custom command (wrapped by meson > > > to capture output) > > > /mnt/git/qemu_/build/pyvenv/bin/meson test --no-rebuild -t 1 --setup > > > thorough --num-processes 10 --print-errorlogs --suite func --suite > > > func- quick --suite func-thorough > > > 1/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64- > > > aarch64_virt_gpu SKIP 1.95s 0 subtests > > > passed > > > > I tried a couple of times now, and finally realized that it's the > > "gdb-multiarch" binary that is missing on Fedora. And as far as I can see, > > there is also no package that provides this on Fedora? So if we go ahead > > with your patches, this test will only run on certain distros that provide > > this binary. > > 'gdb-multiarch' is a command name invented by Debian. > > On Fedora, the regular 'gdb' binary is built with support for > multiple architectures for the purpose of remote debugging. The set of archs supported by GDB in Fedora is s390-linux-gnu powerpc-linux-gnu arm-linux-gnu aarch64-linux-gnu riscv64-linux-gnu x86_64-redhat-linux-gnu you can query this with 'gdb --configuration' and parsing the '--enable-targets' args - I've not found a better way :-( With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Hi Daniel! On 8/26/25 12:22, Daniel P. Berrangé wrote: > On Tue, Aug 26, 2025 at 03:10:42PM +0100, Daniel P. Berrangé wrote: >> On Tue, Aug 26, 2025 at 09:51:27AM +0200, Thomas Huth wrote: >>> On 25/08/2025 16.04, Gustavo Romero wrote: >>>> Hello, Thomas! >>>> >>>> On 8/25/25 07:29, Thomas Huth wrote: >>>>> On 19/08/2025 16.39, Gustavo Romero wrote: >>>>>> The goal of this series is to remove Avocado as a dependency for running >>>>>> the reverse_debugging functional test. >>>>>> >>>>>> This test, the last one I’m aware of that relies on Avocado, requires it >>>>>> because of the need for GDB to test reverse stepping and continue. >>> ... >>>>> I gave it a try, but this did not work for me, the test was not run >>>>> at all anymore. Are there any patches needed on top? >>>> >>>> hmm that's odd. I'm able to run it with 'make check-functional' and with >>>> 'meson test'... >>>> >>>> This is how I'm running it (let me know if I'm missing something): >>> ... >>>> gromero@gromero0:/mnt/git/qemu_$ >>>> gromero@gromero0:/mnt/git/qemu_/build$ ../configure >>>> --target-list=aarch64- softmmu --disable-docs >>>> gromero@gromero0:/mnt/git/qemu_/build$ make -j 32 >>>> gromero@gromero0:/mnt/git/qemu_/build$ time make -j 15 check-functional >>>> [1/2] Generating tests/functional/func-precache-aarch64- >>>> aarch64_reverse_debug with a custom command (wrapped by meson to set >>>> env) >>>> 2025-08-25 12:50:04,215 - qemu-test - INFO - Attempting to cache '/home/ >>>> gromero/.cache/qemu/ >>>> download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7' >>>> 2025-08-25 12:50:04,225 - qemu-test - DEBUG - Using cached asset /home/ >>>> gromero/.cache/qemu/ >>>> download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7 >>>> for >>>> https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/ >>>> Everything/aarch64/os/images/pxeboot/vmlinuz >>>> GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination off' >>>> -ex 'set confirm off' -ex "py >>>> sys.argv=['/mnt/git/qemu_/tests/functional/ >>>> test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/ >>>> test_aarch64_reverse_debug.py >>>> [0/1] Running external command precache-functional (wrapped by meson to >>>> set env) >>>> make[1]: Entering directory '/mnt/git/qemu_/build' >>>> [1/6] Generating qemu-version.h with a custom command (wrapped by meson >>>> to capture output) >>>> /mnt/git/qemu_/build/pyvenv/bin/meson test --no-rebuild -t 1 --setup >>>> thorough --num-processes 10 --print-errorlogs --suite func --suite >>>> func- quick --suite func-thorough >>>> 1/27 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64- >>>> aarch64_virt_gpu SKIP 1.95s 0 subtests >>>> passed >>> >>> I tried a couple of times now, and finally realized that it's the >>> "gdb-multiarch" binary that is missing on Fedora. And as far as I can see, >>> there is also no package that provides this on Fedora? So if we go ahead >>> with your patches, this test will only run on certain distros that provide >>> this binary. >> >> 'gdb-multiarch' is a command name invented by Debian. >> >> On Fedora, the regular 'gdb' binary is built with support for >> multiple architectures for the purpose of remote debugging. > > The set of archs supported by GDB in Fedora is > > s390-linux-gnu > powerpc-linux-gnu > arm-linux-gnu > aarch64-linux-gnu > riscv64-linux-gnu > x86_64-redhat-linux-gnu > > you can query this with 'gdb --configuration' and parsing the > '--enable-targets' args - I've not found a better way :-( Thanks a lot for checking it closely. We already detect it via: source_path/scripts/probe-gdb-support.py in configure to detect the arches. That's what Alex is suggesting, to use the gdb from configure in the test harness or scripts that rely on GDB. Cheers, Gustavo
Thomas Huth <thuth@redhat.com> writes: > On 25/08/2025 16.04, Gustavo Romero wrote: >> Hello, Thomas! >> On 8/25/25 07:29, Thomas Huth wrote: >>> On 19/08/2025 16.39, Gustavo Romero wrote: >>>> The goal of this series is to remove Avocado as a dependency for running >>>> the reverse_debugging functional test. >>>> >>>> This test, the last one I’m aware of that relies on Avocado, requires it >>>> because of the need for GDB to test reverse stepping and continue. > ... >>> I gave it a try, but this did not work for me, the test was not run >>> at all anymore. Are there any patches needed on top? >> hmm that's odd. I'm able to run it with 'make check-functional' and >> with 'meson test'... >> This is how I'm running it (let me know if I'm missing something): > ... >> gromero@gromero0:/mnt/git/qemu_$ >> gromero@gromero0:/mnt/git/qemu_/build$ ../configure >> --target-list=aarch64- softmmu --disable-docs >> gromero@gromero0:/mnt/git/qemu_/build$ make -j 32 >> gromero@gromero0:/mnt/git/qemu_/build$ time make -j 15 check-functional >> [1/2] Generating tests/functional/func-precache-aarch64- >> aarch64_reverse_debug with a custom command (wrapped by meson to set >> env) >> 2025-08-25 12:50:04,215 - qemu-test - INFO - Attempting to cache >> '/home/ gromero/.cache/qemu/ >> download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7' >> 2025-08-25 12:50:04,225 - qemu-test - DEBUG - Using cached asset >> /home/ gromero/.cache/qemu/ >> download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7 >> for >> https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/ >> Everything/aarch64/os/images/pxeboot/vmlinuz >> GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination >> off' -ex 'set confirm off' -ex "py >> sys.argv=['/mnt/git/qemu_/tests/functional/ >> test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/ >> test_aarch64_reverse_debug.py >> [0/1] Running external command precache-functional (wrapped by meson >> to set env) >> make[1]: Entering directory '/mnt/git/qemu_/build' >> [1/6] Generating qemu-version.h with a custom command (wrapped by >> meson to capture output) >> /mnt/git/qemu_/build/pyvenv/bin/meson test --no-rebuild -t 1 >> --setup thorough --num-processes 10 --print-errorlogs --suite >> func --suite func- quick --suite func-thorough >> 1/27 qemu:func-thorough+func-aarch64-thorough+thorough / >> func-aarch64- aarch64_virt_gpu SKIP >> 1.95s 0 subtests passed > > I tried a couple of times now, and finally realized that it's the > "gdb-multiarch" binary that is missing on Fedora. And as far as I can > see, there is also no package that provides this on Fedora? So if we > go ahead with your patches, this test will only run on certain distros > that provide this binary. I've mentioned in another review comment we should use the same gdb as set/detected from configure as we do for the check-tcg tests. > > Thomas -- Alex Bennée Virtualisation Tech Lead @ Linaro
On Tue, Aug 26, 2025 at 11:28 AM Alex Bennée <alex.bennee@linaro.org> wrote: > > Thomas Huth <thuth@redhat.com> writes: > > > On 25/08/2025 16.04, Gustavo Romero wrote: > >> Hello, Thomas! > >> On 8/25/25 07:29, Thomas Huth wrote: > >>> On 19/08/2025 16.39, Gustavo Romero wrote: > >>>> The goal of this series is to remove Avocado as a dependency for running > >>>> the reverse_debugging functional test. > >>>> > >>>> This test, the last one I’m aware of that relies on Avocado, requires it > >>>> because of the need for GDB to test reverse stepping and continue. > > ... > >>> I gave it a try, but this did not work for me, the test was not run > >>> at all anymore. Are there any patches needed on top? > >> hmm that's odd. I'm able to run it with 'make check-functional' and > >> with 'meson test'... > >> This is how I'm running it (let me know if I'm missing something): > > ... > >> gromero@gromero0:/mnt/git/qemu_$ > >> gromero@gromero0:/mnt/git/qemu_/build$ ../configure > >> --target-list=aarch64- softmmu --disable-docs > >> gromero@gromero0:/mnt/git/qemu_/build$ make -j 32 > >> gromero@gromero0:/mnt/git/qemu_/build$ time make -j 15 check-functional > >> [1/2] Generating tests/functional/func-precache-aarch64- > >> aarch64_reverse_debug with a custom command (wrapped by meson to set > >> env) > >> 2025-08-25 12:50:04,215 - qemu-test - INFO - Attempting to cache > >> '/home/ gromero/.cache/qemu/ > >> download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7' > >> 2025-08-25 12:50:04,225 - qemu-test - DEBUG - Using cached asset > >> /home/ gromero/.cache/qemu/ > >> download/7e1430b81c26bdd0da025eeb8fbd77b5dc961da4364af26e771bd39f379cbbf7 > >> for > >> https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/29/ > >> Everything/aarch64/os/images/pxeboot/vmlinuz > >> GDB CMD: /usr/bin/gdb-multiarch -q -n -batch -ex 'set pagination > >> off' -ex 'set confirm off' -ex "py > >> sys.argv=['/mnt/git/qemu_/tests/functional/ > >> test_aarch64_reverse_debug.py']" -x /mnt/git/qemu_/tests/functional/ > >> test_aarch64_reverse_debug.py > >> [0/1] Running external command precache-functional (wrapped by meson > >> to set env) > >> make[1]: Entering directory '/mnt/git/qemu_/build' > >> [1/6] Generating qemu-version.h with a custom command (wrapped by > >> meson to capture output) > >> /mnt/git/qemu_/build/pyvenv/bin/meson test --no-rebuild -t 1 > >> --setup thorough --num-processes 10 --print-errorlogs --suite > >> func --suite func- quick --suite func-thorough > >> 1/27 qemu:func-thorough+func-aarch64-thorough+thorough / > >> func-aarch64- aarch64_virt_gpu SKIP > >> 1.95s 0 subtests passed > > > > I tried a couple of times now, and finally realized that it's the > > "gdb-multiarch" binary that is missing on Fedora. And as far as I can > > see, there is also no package that provides this on Fedora? So if we > > go ahead with your patches, this test will only run on certain distros > > that provide this binary. > > I've mentioned in another review comment we should use the same gdb as > set/detected from configure as we do for the check-tcg tests. FWIW you can pass your own binary and command flags to https://github.com/cs01/pygdbmi It uses subprocess and wraps all stdout/stderr of gdb, like avocado. So you can call it from the functional test directly.
On Mon, Aug 25, 2025 at 1:31 PM Thomas Huth <thuth@redhat.com> wrote: > > On 19/08/2025 16.39, Gustavo Romero wrote: > > The goal of this series is to remove Avocado as a dependency for running > > the reverse_debugging functional test. > > > > This test, the last one I’m aware of that relies on Avocado, requires it > > because of the need for GDB to test reverse stepping and continue. > > Hi! > > Please note that there are currently also some efforts going on to extract > the GDB part from avocado into a more self-contained python module called > aautils, which might help here, too: > > https://github.com/avocado-framework/aautils/issues/82 > There's also https://github.com/cs01/pygdbmi which I have used before, it seems like it does the same thing as avocado's gdb module but it could be missing some functionality > > In this series, we leveraged the run-test.py script used in the > > check-tcg tests, making it a GDB runner capable of calling a test script > > without spawning any VMs. In this configuration, the test script can > > manage the VMs and also import gdb from the test script, making the > > Python GDB API available inside the functional test. > > > > The test is kept “skipped” for aarch64, ppc64, and x86_64, so it is > > necessary to set QEMU_TEST_FLAKY_TESTS=1 in the test environment to > > effectively run the test on these archs. > > > > On aarch64, the test is flaky, but there is a fix that I’ve tested while > > writing this series [0] that resolves it. On ppc64 and x86_64, the test > > always fails: on ppc64, GDB gets a bogus PC, and on x86_64, the last > > part of the test (reverse-continue) does not hit the last executed PC > > (as it should) but instead jumps to the beginning of the code (fist PC > > in forward order). > > At least the x86 test used to work for me most of the time, it just fails > occasionally. Same for the ppc64 test if I disable the powernv subtest there. > > > Thus, to run the reverse_debugging test effectively on aarch64: > > > > $ export QEMU_TEST_FLAKY_TESTS=1 > > $ make check-functional > > I gave it a try, but this did not work for me, the test was not run at all > anymore. Are there any patches needed on top? > > Anyway, shifting to a different test harness here makes me wonder whether > the whole reverse_debug test should maybe be rather moved to tests/tcg > instead, where we already have the basic support for the stuff from > tests/guest-debug/ ? > The aarch64 would require a different guest payload, of course, in that > case, so not sure whether it's feasible? > > Thomas > >
Hi Manos! On 8/25/25 08:00, Manos Pitsidianakis wrote: > On Mon, Aug 25, 2025 at 1:31 PM Thomas Huth <thuth@redhat.com> wrote: >> >> On 19/08/2025 16.39, Gustavo Romero wrote: >>> The goal of this series is to remove Avocado as a dependency for running >>> the reverse_debugging functional test. >>> >>> This test, the last one I’m aware of that relies on Avocado, requires it >>> because of the need for GDB to test reverse stepping and continue. >> >> Hi! >> >> Please note that there are currently also some efforts going on to extract >> the GDB part from avocado into a more self-contained python module called >> aautils, which might help here, too: >> >> https://github.com/avocado-framework/aautils/issues/82 >> > > There's also https://github.com/cs01/pygdbmi which I have used before, > it seems like it does the same thing as avocado's gdb module but it > could be missing some functionality Thanks for the suggestion. I recall now that I've hastily looked at pygdbmi when writing the mte.py test. I don't think it misses any functionality for our current purposes here, but it has its own runner so we'll be using two different GDB runners for the tests. I´d like to avoid it and stick with the simplest runner which doesn't add another module dependency. The same for the test interface. In general, pygdbmi seems more than it would be necessary for the GDB tests we have currently. I would like to keep the tests as easy as possible to be executed manually and I think that adding another API rather than GDB's own Python API, currently, will not help. Cheers, Gustavo >>> In this series, we leveraged the run-test.py script used in the >>> check-tcg tests, making it a GDB runner capable of calling a test script >>> without spawning any VMs. In this configuration, the test script can >>> manage the VMs and also import gdb from the test script, making the >>> Python GDB API available inside the functional test. >>> >>> The test is kept “skipped” for aarch64, ppc64, and x86_64, so it is >>> necessary to set QEMU_TEST_FLAKY_TESTS=1 in the test environment to >>> effectively run the test on these archs. >>> >>> On aarch64, the test is flaky, but there is a fix that I’ve tested while >>> writing this series [0] that resolves it. On ppc64 and x86_64, the test >>> always fails: on ppc64, GDB gets a bogus PC, and on x86_64, the last >>> part of the test (reverse-continue) does not hit the last executed PC >>> (as it should) but instead jumps to the beginning of the code (fist PC >>> in forward order). >> >> At least the x86 test used to work for me most of the time, it just fails >> occasionally. Same for the ppc64 test if I disable the powernv subtest there. >> >>> Thus, to run the reverse_debugging test effectively on aarch64: >>> >>> $ export QEMU_TEST_FLAKY_TESTS=1 >>> $ make check-functional >> >> I gave it a try, but this did not work for me, the test was not run at all >> anymore. Are there any patches needed on top? >> >> Anyway, shifting to a different test harness here makes me wonder whether >> the whole reverse_debug test should maybe be rather moved to tests/tcg >> instead, where we already have the basic support for the stuff from >> tests/guest-debug/ ? >> The aarch64 would require a different guest payload, of course, in that >> case, so not sure whether it's feasible? >> >> Thomas >> >>
© 2016 - 2025 Red Hat, Inc.