[PATCH v4 0/7] tracetool: add test suite to improve reviewability

Daniel P. Berrangé posted 7 patches 2 months, 3 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20250819161053.464641-1-berrange@redhat.com
Maintainers: "Alex Bennée" <alex.bennee@linaro.org>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Thomas Huth <thuth@redhat.com>, Stefan Hajnoczi <stefanha@redhat.com>, Mads Ynddal <mads@ynddal.dk>, Paolo Bonzini <pbonzini@redhat.com>, Markus Armbruster <armbru@redhat.com>, Michael Roth <michael.roth@amd.com>
There is a newer version of this series
MAINTAINERS                                  |   1 +
docs/devel/testing/main.rst                  |  40 +++++++
scripts/checkpatch.pl                        |   3 +
scripts/tracetool/__init__.py                |   9 +-
scripts/tracetool/format/c.py                |   3 +-
scripts/tracetool/format/d.py                |   3 +-
scripts/tracetool/format/h.py                |  17 +--
scripts/tracetool/format/log_stap.py         |   1 +
scripts/tracetool/format/simpletrace_stap.py |   1 +
scripts/tracetool/format/stap.py             |   1 +
scripts/tracetool/format/ust_events_c.py     |   1 +
scripts/tracetool/format/ust_events_h.py     |   1 +
tests/Makefile.include                       |   1 +
tests/meson.build                            |   1 +
tests/qapi-schema/test-qapi.py               |   7 +-
tests/tracetool/dtrace.c                     |  32 ++++++
tests/tracetool/dtrace.d                     |  10 ++
tests/tracetool/dtrace.h                     |  45 ++++++++
tests/tracetool/dtrace.log-stap              |  15 +++
tests/tracetool/dtrace.simpletrace-stap      |  16 +++
tests/tracetool/dtrace.stap                  |  14 +++
tests/tracetool/ftrace.c                     |  32 ++++++
tests/tracetool/ftrace.h                     |  59 +++++++++++
tests/tracetool/log.c                        |  32 ++++++
tests/tracetool/log.h                        |  43 ++++++++
tests/tracetool/meson.build                  |  25 +++++
tests/tracetool/simple.c                     |  61 +++++++++++
tests/tracetool/simple.h                     |  40 +++++++
tests/tracetool/syslog.c                     |  32 ++++++
tests/tracetool/syslog.h                     |  43 ++++++++
tests/tracetool/trace-events                 |   5 +
tests/tracetool/tracetool-test.py            | 105 +++++++++++++++++++
tests/tracetool/ust.c                        |  32 ++++++
tests/tracetool/ust.h                        |  41 ++++++++
tests/tracetool/ust.ust-events-c             |  14 +++
tests/tracetool/ust.ust-events-h             |  56 ++++++++++
36 files changed, 821 insertions(+), 21 deletions(-)
create mode 100644 tests/tracetool/dtrace.c
create mode 100644 tests/tracetool/dtrace.d
create mode 100644 tests/tracetool/dtrace.h
create mode 100644 tests/tracetool/dtrace.log-stap
create mode 100644 tests/tracetool/dtrace.simpletrace-stap
create mode 100644 tests/tracetool/dtrace.stap
create mode 100644 tests/tracetool/ftrace.c
create mode 100644 tests/tracetool/ftrace.h
create mode 100644 tests/tracetool/log.c
create mode 100644 tests/tracetool/log.h
create mode 100644 tests/tracetool/meson.build
create mode 100644 tests/tracetool/simple.c
create mode 100644 tests/tracetool/simple.h
create mode 100644 tests/tracetool/syslog.c
create mode 100644 tests/tracetool/syslog.h
create mode 100644 tests/tracetool/trace-events
create mode 100755 tests/tracetool/tracetool-test.py
create mode 100644 tests/tracetool/ust.c
create mode 100644 tests/tracetool/ust.h
create mode 100644 tests/tracetool/ust.ust-events-c
create mode 100644 tests/tracetool/ust.ust-events-h
[PATCH v4 0/7] tracetool: add test suite to improve reviewability
Posted by Daniel P. Berrangé 2 months, 3 weeks ago
To repeat the start of the commit message in patch 5....

When reviewing tracetool patches it is often very unclear what the
expected output will be for the generated backends. Compounding
this is that a default build will only enable the 'log' trace
backend, so developers won't see generated code for other backends
without making a special effort. Some backends are also platform
specific, so can't be enabled in QEMU builds, even though tracetool
could generate the code.

To address this, introduce a test suite for tracetool which is
conceptually similar to the qapi-schema test. It is a simple
python program that runs tracetool and compares the actual output
to historical reference output kept in git.

This was inspired by noticing the now pointless "__nocheck__"
method wrapping in the inline probe functions.

Changed in v4:

 - Drop patch adding '-' as an output filename for tracetool
 - Rewrite test to always use relative filenames for both
   inputs and outputs
 - Pass python code through 'black'
 - Minor docs punctuation tweak
 - Drop unused python method

Changed in v3:

 - Also modify the QAPI schema test to use QEMU_TEST_REGENERATE=1
   env & document / hint this
 - Make tracetool accept '-' as shorthand for stdout to
   avoid relative paths in the reference output

Changed in v2:

 - Add tracetool info to docs/devel/testing.rst

Daniel P. Berrangé (7):
  checkpatch: cull trailing '*/' in SPDX check
  tracetool: eliminate trailing whitespace in C format
  tracetool: avoid space after "*" in arg types
  tracetool: include SPDX-License-Identifier in generated files
  tracetool: add test suite for tracetool with reference output
  tracetool: drop the probe "__nocheck__" wrapping
  qapi: switch to use QEMU_TEST_REGENERATE env var

 MAINTAINERS                                  |   1 +
 docs/devel/testing/main.rst                  |  40 +++++++
 scripts/checkpatch.pl                        |   3 +
 scripts/tracetool/__init__.py                |   9 +-
 scripts/tracetool/format/c.py                |   3 +-
 scripts/tracetool/format/d.py                |   3 +-
 scripts/tracetool/format/h.py                |  17 +--
 scripts/tracetool/format/log_stap.py         |   1 +
 scripts/tracetool/format/simpletrace_stap.py |   1 +
 scripts/tracetool/format/stap.py             |   1 +
 scripts/tracetool/format/ust_events_c.py     |   1 +
 scripts/tracetool/format/ust_events_h.py     |   1 +
 tests/Makefile.include                       |   1 +
 tests/meson.build                            |   1 +
 tests/qapi-schema/test-qapi.py               |   7 +-
 tests/tracetool/dtrace.c                     |  32 ++++++
 tests/tracetool/dtrace.d                     |  10 ++
 tests/tracetool/dtrace.h                     |  45 ++++++++
 tests/tracetool/dtrace.log-stap              |  15 +++
 tests/tracetool/dtrace.simpletrace-stap      |  16 +++
 tests/tracetool/dtrace.stap                  |  14 +++
 tests/tracetool/ftrace.c                     |  32 ++++++
 tests/tracetool/ftrace.h                     |  59 +++++++++++
 tests/tracetool/log.c                        |  32 ++++++
 tests/tracetool/log.h                        |  43 ++++++++
 tests/tracetool/meson.build                  |  25 +++++
 tests/tracetool/simple.c                     |  61 +++++++++++
 tests/tracetool/simple.h                     |  40 +++++++
 tests/tracetool/syslog.c                     |  32 ++++++
 tests/tracetool/syslog.h                     |  43 ++++++++
 tests/tracetool/trace-events                 |   5 +
 tests/tracetool/tracetool-test.py            | 105 +++++++++++++++++++
 tests/tracetool/ust.c                        |  32 ++++++
 tests/tracetool/ust.h                        |  41 ++++++++
 tests/tracetool/ust.ust-events-c             |  14 +++
 tests/tracetool/ust.ust-events-h             |  56 ++++++++++
 36 files changed, 821 insertions(+), 21 deletions(-)
 create mode 100644 tests/tracetool/dtrace.c
 create mode 100644 tests/tracetool/dtrace.d
 create mode 100644 tests/tracetool/dtrace.h
 create mode 100644 tests/tracetool/dtrace.log-stap
 create mode 100644 tests/tracetool/dtrace.simpletrace-stap
 create mode 100644 tests/tracetool/dtrace.stap
 create mode 100644 tests/tracetool/ftrace.c
 create mode 100644 tests/tracetool/ftrace.h
 create mode 100644 tests/tracetool/log.c
 create mode 100644 tests/tracetool/log.h
 create mode 100644 tests/tracetool/meson.build
 create mode 100644 tests/tracetool/simple.c
 create mode 100644 tests/tracetool/simple.h
 create mode 100644 tests/tracetool/syslog.c
 create mode 100644 tests/tracetool/syslog.h
 create mode 100644 tests/tracetool/trace-events
 create mode 100755 tests/tracetool/tracetool-test.py
 create mode 100644 tests/tracetool/ust.c
 create mode 100644 tests/tracetool/ust.h
 create mode 100644 tests/tracetool/ust.ust-events-c
 create mode 100644 tests/tracetool/ust.ust-events-h

-- 
2.50.1


Re: [PATCH v4 0/7] tracetool: add test suite to improve reviewability
Posted by Stefan Hajnoczi 2 months, 3 weeks ago
On Tue, Aug 19, 2025 at 05:10:46PM +0100, Daniel P. Berrangé wrote:
> To repeat the start of the commit message in patch 5....
> 
> When reviewing tracetool patches it is often very unclear what the
> expected output will be for the generated backends. Compounding
> this is that a default build will only enable the 'log' trace
> backend, so developers won't see generated code for other backends
> without making a special effort. Some backends are also platform
> specific, so can't be enabled in QEMU builds, even though tracetool
> could generate the code.
> 
> To address this, introduce a test suite for tracetool which is
> conceptually similar to the qapi-schema test. It is a simple
> python program that runs tracetool and compares the actual output
> to historical reference output kept in git.
> 
> This was inspired by noticing the now pointless "__nocheck__"
> method wrapping in the inline probe functions.
> 
> Changed in v4:
> 
>  - Drop patch adding '-' as an output filename for tracetool
>  - Rewrite test to always use relative filenames for both
>    inputs and outputs
>  - Pass python code through 'black'
>  - Minor docs punctuation tweak
>  - Drop unused python method
> 
> Changed in v3:
> 
>  - Also modify the QAPI schema test to use QEMU_TEST_REGENERATE=1
>    env & document / hint this
>  - Make tracetool accept '-' as shorthand for stdout to
>    avoid relative paths in the reference output
> 
> Changed in v2:
> 
>  - Add tracetool info to docs/devel/testing.rst
> 
> Daniel P. Berrangé (7):
>   checkpatch: cull trailing '*/' in SPDX check
>   tracetool: eliminate trailing whitespace in C format
>   tracetool: avoid space after "*" in arg types
>   tracetool: include SPDX-License-Identifier in generated files
>   tracetool: add test suite for tracetool with reference output
>   tracetool: drop the probe "__nocheck__" wrapping
>   qapi: switch to use QEMU_TEST_REGENERATE env var
> 
>  MAINTAINERS                                  |   1 +
>  docs/devel/testing/main.rst                  |  40 +++++++
>  scripts/checkpatch.pl                        |   3 +
>  scripts/tracetool/__init__.py                |   9 +-
>  scripts/tracetool/format/c.py                |   3 +-
>  scripts/tracetool/format/d.py                |   3 +-
>  scripts/tracetool/format/h.py                |  17 +--
>  scripts/tracetool/format/log_stap.py         |   1 +
>  scripts/tracetool/format/simpletrace_stap.py |   1 +
>  scripts/tracetool/format/stap.py             |   1 +
>  scripts/tracetool/format/ust_events_c.py     |   1 +
>  scripts/tracetool/format/ust_events_h.py     |   1 +
>  tests/Makefile.include                       |   1 +
>  tests/meson.build                            |   1 +
>  tests/qapi-schema/test-qapi.py               |   7 +-
>  tests/tracetool/dtrace.c                     |  32 ++++++
>  tests/tracetool/dtrace.d                     |  10 ++
>  tests/tracetool/dtrace.h                     |  45 ++++++++
>  tests/tracetool/dtrace.log-stap              |  15 +++
>  tests/tracetool/dtrace.simpletrace-stap      |  16 +++
>  tests/tracetool/dtrace.stap                  |  14 +++
>  tests/tracetool/ftrace.c                     |  32 ++++++
>  tests/tracetool/ftrace.h                     |  59 +++++++++++
>  tests/tracetool/log.c                        |  32 ++++++
>  tests/tracetool/log.h                        |  43 ++++++++
>  tests/tracetool/meson.build                  |  25 +++++
>  tests/tracetool/simple.c                     |  61 +++++++++++
>  tests/tracetool/simple.h                     |  40 +++++++
>  tests/tracetool/syslog.c                     |  32 ++++++
>  tests/tracetool/syslog.h                     |  43 ++++++++
>  tests/tracetool/trace-events                 |   5 +
>  tests/tracetool/tracetool-test.py            | 105 +++++++++++++++++++
>  tests/tracetool/ust.c                        |  32 ++++++
>  tests/tracetool/ust.h                        |  41 ++++++++
>  tests/tracetool/ust.ust-events-c             |  14 +++
>  tests/tracetool/ust.ust-events-h             |  56 ++++++++++
>  36 files changed, 821 insertions(+), 21 deletions(-)
>  create mode 100644 tests/tracetool/dtrace.c
>  create mode 100644 tests/tracetool/dtrace.d
>  create mode 100644 tests/tracetool/dtrace.h
>  create mode 100644 tests/tracetool/dtrace.log-stap
>  create mode 100644 tests/tracetool/dtrace.simpletrace-stap
>  create mode 100644 tests/tracetool/dtrace.stap
>  create mode 100644 tests/tracetool/ftrace.c
>  create mode 100644 tests/tracetool/ftrace.h
>  create mode 100644 tests/tracetool/log.c
>  create mode 100644 tests/tracetool/log.h
>  create mode 100644 tests/tracetool/meson.build
>  create mode 100644 tests/tracetool/simple.c
>  create mode 100644 tests/tracetool/simple.h
>  create mode 100644 tests/tracetool/syslog.c
>  create mode 100644 tests/tracetool/syslog.h
>  create mode 100644 tests/tracetool/trace-events
>  create mode 100755 tests/tracetool/tracetool-test.py
>  create mode 100644 tests/tracetool/ust.c
>  create mode 100644 tests/tracetool/ust.h
>  create mode 100644 tests/tracetool/ust.ust-events-c
>  create mode 100644 tests/tracetool/ust.ust-events-h
> 
> -- 
> 2.50.1
> 

Thanks, applied to my tracing-next tree:
https://gitlab.com/stefanha/qemu/commits/tracing-next

Stefan