[PATCH v3 00/33] Printbufs

Kent Overstreet posted 33 patches 3 years, 10 months ago
There is a newer version of this series
Documentation/core-api/printk-formats.rst |   22 +
arch/powerpc/kernel/process.c             |   16 +-
arch/powerpc/kernel/security.c            |   75 +-
arch/powerpc/platforms/pseries/papr_scm.c |   34 +-
arch/x86/kernel/cpu/resctrl/rdtgroup.c    |   16 +-
drivers/clk/tegra/clk-bpmp.c              |   21 +-
drivers/input/joystick/analog.c           |   23 +-
drivers/pci/p2pdma.c                      |   17 +-
fs/d_path.c                               |   34 +
include/linux/dcache.h                    |    1 +
include/linux/kernel.h                    |   12 +
include/linux/pretty-printers.h           |   10 +
include/linux/printbuf.h                  |  245 +++
include/linux/seq_buf.h                   |  162 --
include/linux/string.h                    |    5 +
include/linux/string_helpers.h            |    8 +-
include/linux/trace_events.h              |    2 +-
include/linux/trace_seq.h                 |   14 +-
kernel/trace/trace.c                      |   45 +-
kernel/trace/trace_dynevent.c             |   34 +-
kernel/trace/trace_events_filter.c        |    2 +-
kernel/trace/trace_events_synth.c         |   32 +-
kernel/trace/trace_functions_graph.c      |    6 +-
kernel/trace/trace_kprobe.c               |    2 +-
kernel/trace/trace_seq.c                  |  111 +-
lib/Makefile                              |    4 +-
lib/hexdump.c                             |  246 +--
lib/pretty-printers.c                     |   59 +
lib/printbuf.c                            |  252 +++
lib/seq_buf.c                             |  397 -----
lib/string_helpers.c                      |  141 +-
lib/test_hexdump.c                        |   30 +-
lib/test_printf.c                         |   26 +-
lib/vsprintf.c                            | 1716 ++++++++++-----------
mm/memcontrol.c                           |   68 +-
tools/testing/nvdimm/test/ndtest.c        |   22 +-
36 files changed, 1943 insertions(+), 1967 deletions(-)
create mode 100644 include/linux/pretty-printers.h
create mode 100644 include/linux/printbuf.h
delete mode 100644 include/linux/seq_buf.h
create mode 100644 lib/pretty-printers.c
create mode 100644 lib/printbuf.c
delete mode 100644 lib/seq_buf.c
[PATCH v3 00/33] Printbufs
Posted by Kent Overstreet 3 years, 10 months ago
Printbufs, your new data structure for all your string-building and outputting
needs!

git repo: https://evilpiepirate.org/git/bcachefs.git/log/?h=printbuf_v3

Benefits:
 - Replaces passing & returning raw char * pointers and lengths in a ton of
   places, including especially vsprintf.c, with a much saner calling convention
 - New helpers which greatly simplify and cleanup aforementioned vsprintf.c
 - New standard calling convention/naming for pretty printers!
 - New printf format string - %pf(%p) - for calling pretty printers by passing
   them to sprintf instead of sticking them in vsprintf.c behind weird dispatch
   code
 - printbufs can auto-heap allocate! No need for statically sized buffers,
   unless you want to do that
 - Tabstops and indenting, for greatly improved formatting of multi-line output

...and probably more that I've forgotten to mention.

Changes since last patche:

New namespace prefixes:
-----------------------

We're not overloading pr_* anymore: any standard library code that outputs to a
printbuf should use the prt_ prefix. This is not for printbuf control code -
that uses the printbuf_ prefix; prt_ is just for things that print.

string_escape_mem():
--------------------

string_escape_mem() has now been properly converted to printbufs instead of just
adding a printbuf-style wrapper; the new printbuf helpers simplify that code
quite a bit.

hexdump:
--------

The hexdump code has been converted to printbuf and also reorganized and cleaned
up quite a bit, with better naming too. We now have
 - prt_hex_bytes(), for printing a few hex bytes, with optional grouping
 - prt_hex_line(), for printing a whole line of hex output with ascii characters
   at the end
 - prt_hex_dump(), for printing a whole multiline hex dump

Important behaviour change:

Previously, the hex dump code would _byte swap the output on little endian_.
Since this is not exactly standard behaviour for a hex dumper (binutils doesn't
do this), and is confusing as hell if you're trying to map byte offsets in
structs to your hex output, I've dropped it. Since we only use the hex dumper in
debug output, nothing should break, but to avoid confusion I've put this front
and center in the commit message for that patch.

tracing:
--------

This iteration of the patch series finally converts tracing to printbufs, which
is the last seq_buf user and that code is now also deleted. The tracing
conversion was pretty uneventful, not much to say here (except that it had its
own unique implementation of hex dumping with byte swabbing on little endian;
this is now replaced with just a call to prt_hex_bytes()).

Kent Overstreet (33):
  lib/printbuf: New data structure for printing strings
  lib/string_helpers: Convert string_escape_mem() to printbuf
  vsprintf: Convert to printbuf
  lib/hexdump: Convert to printbuf
  vsprintf: %pf(%p)
  lib/string_helpers: string_get_size() now returns characters wrote
  lib/printbuf: Heap allocation
  lib/printbuf: Tabstops, indenting
  lib/printbuf: Unit specifiers
  lib/pretty-printers: prt_string_option(), prt_bitflags()
  vsprintf: Improve number()
  vsprintf: prt_u64_minwidth(), prt_u64()
  test_printf: Drop requirement that sprintf not write past nul
  vsprintf: Start consolidating printf_spec handling
  vsprintf: Refactor resource_string()
  vsprintf: Refactor fourcc_string()
  vsprintf: Refactor ip_addr_string()
  vsprintf: Refactor mac_address_string()
  vsprintf: time_and_date() no longer takes printf_spec
  vsprintf: flags_string() no longer takes printf_spec
  vsprintf: Refactor device_node_string, fwnode_string
  vsprintf: Refactor hex_string, bitmap_string_list, bitmap_string
  Input/joystick/analog: Convert from seq_buf -> printbuf
  mm/memcontrol.c: Convert to printbuf
  clk: tegra: bpmp: Convert to printbuf
  tools/testing/nvdimm: Convert to printbuf
  powerpc: Convert to printbuf
  x86/resctrl: Convert to printbuf
  PCI/P2PDMA: Convert to printbuf
  tracing: trace_events_synth: Convert to printbuf
  d_path: prt_path()
  tracing: Convert to printbuf
  Delete seq_buf

 Documentation/core-api/printk-formats.rst |   22 +
 arch/powerpc/kernel/process.c             |   16 +-
 arch/powerpc/kernel/security.c            |   75 +-
 arch/powerpc/platforms/pseries/papr_scm.c |   34 +-
 arch/x86/kernel/cpu/resctrl/rdtgroup.c    |   16 +-
 drivers/clk/tegra/clk-bpmp.c              |   21 +-
 drivers/input/joystick/analog.c           |   23 +-
 drivers/pci/p2pdma.c                      |   17 +-
 fs/d_path.c                               |   34 +
 include/linux/dcache.h                    |    1 +
 include/linux/kernel.h                    |   12 +
 include/linux/pretty-printers.h           |   10 +
 include/linux/printbuf.h                  |  245 +++
 include/linux/seq_buf.h                   |  162 --
 include/linux/string.h                    |    5 +
 include/linux/string_helpers.h            |    8 +-
 include/linux/trace_events.h              |    2 +-
 include/linux/trace_seq.h                 |   14 +-
 kernel/trace/trace.c                      |   45 +-
 kernel/trace/trace_dynevent.c             |   34 +-
 kernel/trace/trace_events_filter.c        |    2 +-
 kernel/trace/trace_events_synth.c         |   32 +-
 kernel/trace/trace_functions_graph.c      |    6 +-
 kernel/trace/trace_kprobe.c               |    2 +-
 kernel/trace/trace_seq.c                  |  111 +-
 lib/Makefile                              |    4 +-
 lib/hexdump.c                             |  246 +--
 lib/pretty-printers.c                     |   59 +
 lib/printbuf.c                            |  252 +++
 lib/seq_buf.c                             |  397 -----
 lib/string_helpers.c                      |  141 +-
 lib/test_hexdump.c                        |   30 +-
 lib/test_printf.c                         |   26 +-
 lib/vsprintf.c                            | 1716 ++++++++++-----------
 mm/memcontrol.c                           |   68 +-
 tools/testing/nvdimm/test/ndtest.c        |   22 +-
 36 files changed, 1943 insertions(+), 1967 deletions(-)
 create mode 100644 include/linux/pretty-printers.h
 create mode 100644 include/linux/printbuf.h
 delete mode 100644 include/linux/seq_buf.h
 create mode 100644 lib/pretty-printers.c
 create mode 100644 lib/printbuf.c
 delete mode 100644 lib/seq_buf.c

-- 
2.36.0
Re: [PATCH v3 00/33] Printbufs
Posted by Steven Rostedt 3 years, 10 months ago
On Sat,  4 Jun 2022 15:30:09 -0400
Kent Overstreet <kent.overstreet@gmail.com> wrote:

> Printbufs, your new data structure for all your string-building and outputting
> needs!
> 
> git repo: https://evilpiepirate.org/git/bcachefs.git/log/?h=printbuf_v3

Hi Kent,

Just wanted to let you know that I want to review this and I'm not
ignoring it. But there's a still quite a bit in my queue that I must
review before I get to this, so it may still be a while. :-/

-- Steve
Re: [PATCH v3 00/33] Printbufs
Posted by Kent Overstreet 3 years, 10 months ago
On Sun, Jun 05, 2022 at 12:21:10PM -0400, Steven Rostedt wrote:
> On Sat,  4 Jun 2022 15:30:09 -0400
> Kent Overstreet <kent.overstreet@gmail.com> wrote:
> 
> > Printbufs, your new data structure for all your string-building and outputting
> > needs!
> > 
> > git repo: https://evilpiepirate.org/git/bcachefs.git/log/?h=printbuf_v3
> 
> Hi Kent,
> 
> Just wanted to let you know that I want to review this and I'm not
> ignoring it. But there's a still quite a bit in my queue that I must
> review before I get to this, so it may still be a while. :-/

Could you just review the parts that touch your code then? The rest has already
seen some review and Petr's going to do more, and I'd like to get this patchset
done and squared away so I can go back to building my OOM debugging improvements
on top of this stuff :)