[PATCH 00/15] iotests: add enhanced debugging info to qemu-io failures

John Snow posted 15 patches 2 years, 1 month ago
Failed in applying to current master (apply log)
There is a newer version of this series
tests/qemu-iotests/030                        | 85 +++++++++++--------
tests/qemu-iotests/040                        | 47 +++++-----
tests/qemu-iotests/056                        |  2 +-
tests/qemu-iotests/149                        |  6 +-
tests/qemu-iotests/163                        |  5 +-
tests/qemu-iotests/205                        |  4 +-
tests/qemu-iotests/216                        | 12 +--
tests/qemu-iotests/218                        |  5 +-
tests/qemu-iotests/224                        |  4 +-
tests/qemu-iotests/242                        |  4 +-
tests/qemu-iotests/245                        |  2 +-
tests/qemu-iotests/255                        |  4 +-
tests/qemu-iotests/258                        | 12 +--
tests/qemu-iotests/298                        | 16 ++--
tests/qemu-iotests/303                        |  4 +-
tests/qemu-iotests/310                        | 22 ++---
tests/qemu-iotests/iotests.py                 | 74 ++++++++--------
tests/qemu-iotests/tests/image-fleecing       | 14 +--
.../qemu-iotests/tests/migration-permissions  | 28 +++---
.../tests/mirror-ready-cancel-error           |  2 +-
.../qemu-iotests/tests/nbd-reconnect-on-open  |  2 +-
.../qemu-iotests/tests/stream-error-on-reset  |  4 +-
22 files changed, 184 insertions(+), 174 deletions(-)
[PATCH 00/15] iotests: add enhanced debugging info to qemu-io failures
Posted by John Snow 2 years, 1 month ago
Howdy,

This series does for qemu_io() what we've done for qemu_img() and makes
this a function that checks the return code by default and raises an
Exception when things do not go according to plan.

This series removes qemu_io_pipe_and_status(), qemu_io_silent(), and
qemu_io_silent_check() in favor of just qemu_io().

RFC:

- There are a few remaining uses of qemu-io that don't go through qemu_io;
QemuIoInteractive is a user that is used in 205, 298, 299, and 307. It
... did not appear worth it to morph qemu_tool_popen into something that
could be used by both QemuIoInteractive *and* qemu_io(), so I left it
alone. It's probably fine for now. (But it does bother me, a little.)

- qemu_io_popen itself is used by the nbd-reconnect-on-open test, and it
seems like a legitimate use -- it wants concurrency. Like the above
problem, I couldn't find a way to bring it into the fold, so I
didn't. (Meh.) I eventually plan to add asyncio subprocess management to
machine.py, and I could tackle stuff like this then. It's not worth it
now.

- Several patches in this patchset ("fixup" in the title) are designed to
be merged-on-commit. I know that's not usually how we do things, but I
thought it was actually nicer than pre-squashing it because it gives me
more flexibility on re-spin.

- Uh, actually, test 040 fails with this patchset and I don't understand
  if it's intentional, harmless, a test design problem, or worse:

======================================================================
ERROR: test_filterless_commit (__main__.TestCommitWithFilters)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jsnow/src/qemu/tests/qemu-iotests/040", line 822, in tearDown
    self.do_test_io('read')
  File "/home/jsnow/src/qemu/tests/qemu-iotests/040", line 751, in do_test_io
    qemu_io('-f', iotests.imgfmt,
  File "/home/jsnow/src/qemu/tests/qemu-iotests/iotests.py", line 365, in qemu_io
    return qemu_tool(*qemu_io_wrap_args(args),
  File "/home/jsnow/src/qemu/tests/qemu-iotests/iotests.py", line 242, in qemu_tool
    raise VerboseProcessError(

qemu.utils.VerboseProcessError: Command
  '('/home/jsnow/src/qemu/bin/git/tests/qemu-iotests/../../qemu-io',
  '--cache', 'writeback', '--aio', 'threads', '-f', 'qcow2', '-c',
  'read -P 4 3M 1M',
  '/home/jsnow/src/qemu/bin/git/tests/qemu-iotests/scratch/3.img')'
  returned non-zero exit status 1.
  ┏━ output ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  ┃ qemu-io: can't open device
  ┃ /home/jsnow/src/qemu/bin/git/tests/qemu-iotests/scratch/3.img:
  ┃ Could not open backing file: Could not open backing file: Throttle
  ┃ group 'tg' does not exist
  ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

It looks like we start with the img chain 3->2->1->0, then we commit 2
down into 1, but checking '3' fails somewhere in the backing
chain. Maybe a real bug?

John Snow (15):
  iotests: replace calls to log(qemu_io(...)) with qemu_io_log()
  iotests/163: Fix broken qemu-io invocation
  iotests: Don't check qemu_io() output for specific error strings
  iotests/040: Don't check image pattern on zero-length image
  iotests: create generic qemu_tool() function
  iotests: rebase qemu_io() on top of qemu_tool()
  iotests/030: fixup
  iotests/149: fixup
  iotests/205: fixup
  iotests/245: fixup
  iotests/migration-permissions: fixup
  iotests/migration-permissions: use assertRaises() for qemu_io()
    negative test
  iotests: remove qemu_io_pipe_and_status()
  iotests: remove qemu_io_silent() and qemu_io_silent_check().
  iotests: make qemu_io_log() check return codes by default

 tests/qemu-iotests/030                        | 85 +++++++++++--------
 tests/qemu-iotests/040                        | 47 +++++-----
 tests/qemu-iotests/056                        |  2 +-
 tests/qemu-iotests/149                        |  6 +-
 tests/qemu-iotests/163                        |  5 +-
 tests/qemu-iotests/205                        |  4 +-
 tests/qemu-iotests/216                        | 12 +--
 tests/qemu-iotests/218                        |  5 +-
 tests/qemu-iotests/224                        |  4 +-
 tests/qemu-iotests/242                        |  4 +-
 tests/qemu-iotests/245                        |  2 +-
 tests/qemu-iotests/255                        |  4 +-
 tests/qemu-iotests/258                        | 12 +--
 tests/qemu-iotests/298                        | 16 ++--
 tests/qemu-iotests/303                        |  4 +-
 tests/qemu-iotests/310                        | 22 ++---
 tests/qemu-iotests/iotests.py                 | 74 ++++++++--------
 tests/qemu-iotests/tests/image-fleecing       | 14 +--
 .../qemu-iotests/tests/migration-permissions  | 28 +++---
 .../tests/mirror-ready-cancel-error           |  2 +-
 .../qemu-iotests/tests/nbd-reconnect-on-open  |  2 +-
 .../qemu-iotests/tests/stream-error-on-reset  |  4 +-
 22 files changed, 184 insertions(+), 174 deletions(-)

-- 
2.34.1

Re: [PATCH 00/15] iotests: add enhanced debugging info to qemu-io failures
Posted by Hanna Reitz 2 years, 1 month ago
On 18.03.22 21:36, John Snow wrote:
> Howdy,

Heya,

[...]

> - Uh, actually, test 040 fails with this patchset and I don't understand
>    if it's intentional, harmless, a test design problem, or worse:
>
> ======================================================================
> ERROR: test_filterless_commit (__main__.TestCommitWithFilters)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File "/home/jsnow/src/qemu/tests/qemu-iotests/040", line 822, in tearDown
>      self.do_test_io('read')
>    File "/home/jsnow/src/qemu/tests/qemu-iotests/040", line 751, in do_test_io
>      qemu_io('-f', iotests.imgfmt,
>    File "/home/jsnow/src/qemu/tests/qemu-iotests/iotests.py", line 365, in qemu_io
>      return qemu_tool(*qemu_io_wrap_args(args),
>    File "/home/jsnow/src/qemu/tests/qemu-iotests/iotests.py", line 242, in qemu_tool
>      raise VerboseProcessError(
>
> qemu.utils.VerboseProcessError: Command
>    '('/home/jsnow/src/qemu/bin/git/tests/qemu-iotests/../../qemu-io',
>    '--cache', 'writeback', '--aio', 'threads', '-f', 'qcow2', '-c',
>    'read -P 4 3M 1M',
>    '/home/jsnow/src/qemu/bin/git/tests/qemu-iotests/scratch/3.img')'
>    returned non-zero exit status 1.
>    ┏━ output ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>    ┃ qemu-io: can't open device
>    ┃ /home/jsnow/src/qemu/bin/git/tests/qemu-iotests/scratch/3.img:
>    ┃ Could not open backing file: Could not open backing file: Throttle
>    ┃ group 'tg' does not exist
>    ┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>
> It looks like we start with the img chain 3->2->1->0, then we commit 2
> down into 1, but checking '3' fails somewhere in the backing
> chain. Maybe a real bug?

Looks like my hunch was right: The problem is that it’s hard to figure 
out a good backing file string when there are filters involved, and so 
in one test here we generate one that contains a JSON description of the 
backing subgraph including a throttle node. Outside of qemu, that 
doesn’t make much sense, hence the error.

(And because we checked only for “pattern verification failed” 
specifically, that error here never surfaced.)

I think (hope?) we can expect management tools to manually specify 
backing file strings in such cases, like the attached diff does. That 
seems to fix the problem.

Hanna