[PATCH 00/20] trace: bugfixes, cleanups and optimizations

Daniel P. Berrangé posted 20 patches 3 weeks, 1 day ago
There is a newer version of this series
audio/audio-mixeng-be.c                      |   2 +-
hw/display/trace-events                      |   2 +-
hw/ppc/vof.c                                 |  13 +--
hw/xen/trace-events                          |   4 +-
include/qemu/osdep.h                         |   8 ++
monitor/qmp.c                                |   4 +-
qapi/trace.json                              |   5 +-
scripts/tracetool/__init__.py                |  32 ++----
scripts/tracetool/backend/__init__.py        |   3 +-
scripts/tracetool/backend/dtrace.py          |   2 -
scripts/tracetool/backend/log.py             |   3 +-
scripts/tracetool/backend/simple.py          |   4 +-
scripts/tracetool/format/c.py                |  74 +++++++-------
scripts/tracetool/format/d.py                |   3 -
scripts/tracetool/format/h.py                |  42 ++++----
scripts/tracetool/format/log_stap.py         |   3 -
scripts/tracetool/format/rs.py               |  30 +++---
scripts/tracetool/format/simpletrace_stap.py |   3 -
scripts/tracetool/format/stap.py             |   3 -
scripts/tracetool/format/ust_events_c.py     |   3 -
scripts/tracetool/format/ust_events_h.py     |   7 +-
stubs/trace-control.c                        |   6 --
target/hppa/trace-events                     |  28 ++---
tests/tracetool/all.c                        |  61 +++++++++++
tests/tracetool/all.d                        |  10 ++
tests/tracetool/all.h                        | 102 +++++++++++++++++++
tests/tracetool/all.log-stap                 |  15 +++
tests/tracetool/all.rs                       |  90 ++++++++++++++++
tests/tracetool/all.simpletrace-stap         |  16 +++
tests/tracetool/all.stap                     |  14 +++
tests/tracetool/all.ust-events-c             |  14 +++
tests/tracetool/all.ust-events-h             |  56 ++++++++++
tests/tracetool/dtrace.c                     |  10 +-
tests/tracetool/dtrace.h                     |   6 +-
tests/tracetool/dtrace.rs                    |   6 +-
tests/tracetool/ftrace.c                     |  10 +-
tests/tracetool/ftrace.h                     |  14 ++-
tests/tracetool/ftrace.rs                    |   6 +-
tests/tracetool/log.c                        |  10 +-
tests/tracetool/log.h                        |  20 ++--
tests/tracetool/log.rs                       |   6 +-
tests/tracetool/meson.build                  |   4 +-
tests/tracetool/nop.c                        |   7 ++
tests/tracetool/nop.h                        |  25 +++++
tests/tracetool/nop.rs                       |  51 ++++++++++
tests/tracetool/simple.c                     |  12 +--
tests/tracetool/simple.h                     |  10 +-
tests/tracetool/simple.rs                    |   6 +-
tests/tracetool/syslog.c                     |  10 +-
tests/tracetool/syslog.h                     |  14 ++-
tests/tracetool/syslog.rs                    |   6 +-
tests/tracetool/tracetool-test.py            |  12 ++-
tests/tracetool/ust.c                        |  10 +-
tests/tracetool/ust.h                        |   6 +-
trace/control-internal.h                     |   8 +-
trace/control-target.c                       |  29 ------
trace/control.c                              |  19 +---
trace/control.h                              |  43 +-------
trace/event-internal.h                       |  19 +---
trace/ftrace.c                               |   3 +-
trace/qmp.c                                  |  47 +++------
trace/simple.c                               |   9 +-
62 files changed, 704 insertions(+), 396 deletions(-)
create mode 100644 tests/tracetool/all.c
create mode 100644 tests/tracetool/all.d
create mode 100644 tests/tracetool/all.h
create mode 100644 tests/tracetool/all.log-stap
create mode 100644 tests/tracetool/all.rs
create mode 100644 tests/tracetool/all.simpletrace-stap
create mode 100644 tests/tracetool/all.stap
create mode 100644 tests/tracetool/all.ust-events-c
create mode 100644 tests/tracetool/all.ust-events-h
create mode 100644 tests/tracetool/nop.c
create mode 100644 tests/tracetool/nop.h
create mode 100644 tests/tracetool/nop.rs
[PATCH 00/20] trace: bugfixes, cleanups and optimizations
Posted by Daniel P. Berrangé 3 weeks, 1 day ago
This series started off with fixing some compile failures seen
with dtrace and ust backends, and got somewhat out of hand as
I looked more deeply at the trace code. So it ended up doing
various bits of code cleanup and memory usage optimizations.

The optimizations aren't huge against each individual binaries.
There are too many scenarios to show the full set, but here
are some binary size examples

The current git master baseline for different enabled
backends  on the x86_64 emulator:

  nop 88546360
  dtrace 90604496
  ftrace 93265944
  log 93799528
  syslog 93962704
  simple 97064528
  ust 162999944
  dtrace,ftrace,log,syslog,simple,ust 174438088

'ust' has insanely high overhead in terms of binary sizes,
inexplicably insane.

'ftrace/log/syslog' are all similar because their overhead
is dominated by the printf() format strings which are
identical in all.

'simple' is a bit worse because of the helper functions we
must generated for each probe

After this series is applied the results for x86_64 are

  nop 88510248
  dtrace 90576160
  ftrace 93156320
  log 93735064
  syslog 93853480
  simple 97050856
  ust 162957256
  dtrace,ftrace,log,syslog,simple,ust 174431128

IOW, for 'dtrace', we save 28k, for 'log' we save 62k

For the aarch64 binary, we see bigger saving in some
cases:

  baseline dtrace 127610144
  baseline log 131050320
     trace dtrace 127459264
     trace log  130998872

IOW, for 'dtrace' we save 150k, for 'log' we only save
50k. No clue why the saving is not so significant for
the 'log' backend but not a big deal.

Daniel P. Berrangé (20):
  osdep: add 'ignore_value(x)' macro
  trace: use 'ignore_value()' instead of a dummy variable
  trace: fix handling of char** in UST backend
  hw/xen: replace "long unsigned int" with "unsigned long " for dtrace
  trace: block use of "long int" for dtrace
  trace: drop unused trace_event_set_state_dynamic_init()
  trace: remove obsolete references to vcpu property
  trace: drop 'id' from TraceEvent struct when not required
  trace: change "DSTATE" variables from uint16_t to uint8_t
  trace: add testing of 'nop' trace backend
  trace: don't emit TraceEvent structs when 'nop' backend is used
  trace: add test of generation with all backends enabled
  trace: remove redundant headers from simple trace backend code
  audio: remove mistaken use of trace_event_get_state()
  hw/ppc: remove mistaken use of trace_event_get_state()
  trace: include qemu_loglevel_mask(LOG_TRACE) in guard
  trace: remove "disable" property from trace files
  trace: stop tracking static disabled state
  trace: remove parsing of "disable" event property
  trace: remove 'trace_event_get_state' wrapper

 audio/audio-mixeng-be.c                      |   2 +-
 hw/display/trace-events                      |   2 +-
 hw/ppc/vof.c                                 |  13 +--
 hw/xen/trace-events                          |   4 +-
 include/qemu/osdep.h                         |   8 ++
 monitor/qmp.c                                |   4 +-
 qapi/trace.json                              |   5 +-
 scripts/tracetool/__init__.py                |  32 ++----
 scripts/tracetool/backend/__init__.py        |   3 +-
 scripts/tracetool/backend/dtrace.py          |   2 -
 scripts/tracetool/backend/log.py             |   3 +-
 scripts/tracetool/backend/simple.py          |   4 +-
 scripts/tracetool/format/c.py                |  74 +++++++-------
 scripts/tracetool/format/d.py                |   3 -
 scripts/tracetool/format/h.py                |  42 ++++----
 scripts/tracetool/format/log_stap.py         |   3 -
 scripts/tracetool/format/rs.py               |  30 +++---
 scripts/tracetool/format/simpletrace_stap.py |   3 -
 scripts/tracetool/format/stap.py             |   3 -
 scripts/tracetool/format/ust_events_c.py     |   3 -
 scripts/tracetool/format/ust_events_h.py     |   7 +-
 stubs/trace-control.c                        |   6 --
 target/hppa/trace-events                     |  28 ++---
 tests/tracetool/all.c                        |  61 +++++++++++
 tests/tracetool/all.d                        |  10 ++
 tests/tracetool/all.h                        | 102 +++++++++++++++++++
 tests/tracetool/all.log-stap                 |  15 +++
 tests/tracetool/all.rs                       |  90 ++++++++++++++++
 tests/tracetool/all.simpletrace-stap         |  16 +++
 tests/tracetool/all.stap                     |  14 +++
 tests/tracetool/all.ust-events-c             |  14 +++
 tests/tracetool/all.ust-events-h             |  56 ++++++++++
 tests/tracetool/dtrace.c                     |  10 +-
 tests/tracetool/dtrace.h                     |   6 +-
 tests/tracetool/dtrace.rs                    |   6 +-
 tests/tracetool/ftrace.c                     |  10 +-
 tests/tracetool/ftrace.h                     |  14 ++-
 tests/tracetool/ftrace.rs                    |   6 +-
 tests/tracetool/log.c                        |  10 +-
 tests/tracetool/log.h                        |  20 ++--
 tests/tracetool/log.rs                       |   6 +-
 tests/tracetool/meson.build                  |   4 +-
 tests/tracetool/nop.c                        |   7 ++
 tests/tracetool/nop.h                        |  25 +++++
 tests/tracetool/nop.rs                       |  51 ++++++++++
 tests/tracetool/simple.c                     |  12 +--
 tests/tracetool/simple.h                     |  10 +-
 tests/tracetool/simple.rs                    |   6 +-
 tests/tracetool/syslog.c                     |  10 +-
 tests/tracetool/syslog.h                     |  14 ++-
 tests/tracetool/syslog.rs                    |   6 +-
 tests/tracetool/tracetool-test.py            |  12 ++-
 tests/tracetool/ust.c                        |  10 +-
 tests/tracetool/ust.h                        |   6 +-
 trace/control-internal.h                     |   8 +-
 trace/control-target.c                       |  29 ------
 trace/control.c                              |  19 +---
 trace/control.h                              |  43 +-------
 trace/event-internal.h                       |  19 +---
 trace/ftrace.c                               |   3 +-
 trace/qmp.c                                  |  47 +++------
 trace/simple.c                               |   9 +-
 62 files changed, 704 insertions(+), 396 deletions(-)
 create mode 100644 tests/tracetool/all.c
 create mode 100644 tests/tracetool/all.d
 create mode 100644 tests/tracetool/all.h
 create mode 100644 tests/tracetool/all.log-stap
 create mode 100644 tests/tracetool/all.rs
 create mode 100644 tests/tracetool/all.simpletrace-stap
 create mode 100644 tests/tracetool/all.stap
 create mode 100644 tests/tracetool/all.ust-events-c
 create mode 100644 tests/tracetool/all.ust-events-h
 create mode 100644 tests/tracetool/nop.c
 create mode 100644 tests/tracetool/nop.h
 create mode 100644 tests/tracetool/nop.rs

-- 
2.55.0


Re: [PATCH 00/20] trace: bugfixes, cleanups and optimizations
Posted by marcandre.lureau@redhat.com 3 weeks, 1 day ago
Hi,

> This series started off with fixing some compile failures seen
> with dtrace and ust backends, and got somewhat out of hand as
> I looked more deeply at the trace code. So it ended up doing
> various bits of code cleanup and memory usage optimizations.
> 
> The optimizations aren't huge against each individual binaries.
> There are too many scenarios to show the full set, but here
> are some binary size examples
> 
> The current git master baseline for different enabled
> backends  on the x86_64 emulator:
> 
>   nop 88546360
>   dtrace 90604496
>   ftrace 93265944
>   log 93799528
>   syslog 93962704
>   simple 97064528
>   ust 162999944
>   dtrace,ftrace,log,syslog,simple,ust 174438088
> 
> 'ust' has insanely high overhead in terms of binary sizes,
> inexplicably insane.
> 
> 'ftrace/log/syslog' are all similar because their overhead
> is dominated by the printf() format strings which are
> identical in all.
> 
> 'simple' is a bit worse because of the helper functions we
> must generated for each probe
> 
> After this series is applied the results for x86_64 are
> 
>   nop 88510248
>   dtrace 90576160
>   ftrace 93156320
>   log 93735064
>   syslog 93853480
>   simple 97050856
>   ust 162957256
>   dtrace,ftrace,log,syslog,simple,ust 174431128
> 
> IOW, for 'dtrace', we save 28k, for 'log' we save 62k
> 
> For the aarch64 binary, we see bigger saving in some
> cases:
> 
>   baseline dtrace 127610144
>   baseline log 131050320
>      trace dtrace 127459264
>      trace log  130998872
> 
> IOW, for 'dtrace' we save 150k, for 'log' we only save
> 50k. No clue why the saving is not so significant for
> the 'log' backend but not a big deal.

Need tracing.rst update, and maybe better commit message, but lgtm otherwise,
series:

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>

-- 
Marc-André Lureau <marcandre.lureau@redhat.com>