[PATCH v4 next 00/23] Enhance printf()

david.laight.linux@gmail.com posted 23 patches 1 month, 1 week ago
There is a newer version of this series
tools/include/nolibc/compiler.h              |   3 +
tools/include/nolibc/stdio.h                 | 501 ++++++++++++++-----
tools/testing/selftests/nolibc/nolibc-test.c | 157 +++---
3 files changed, 469 insertions(+), 192 deletions(-)
[PATCH v4 next 00/23] Enhance printf()
Posted by david.laight.linux@gmail.com 1 month, 1 week ago
From: David Laight <david.laight.linux@gmail.com>

Update printf() so that it handles almost all the non-fp formats.
In particular:
- Left alignment.
- Zero padding.
- Alternate form "%#x" and "%#o".
- Field precision.
- Variable field width and precision.
- Width modifiers q, L, t and z.
- Conversion specifiers i, o and X (X generates lower case).
About the only things that are missing are wide chanacters and floating point.

The tests are updated to match.

Bloat/savings (in nolibc-test, but excluding the program) to patch 14:
(Measured for v3)
    Function                                     old     new   delta
    _nolibc_u64toa_base.isra                       -     143    +143
    strerror                                       -      78     +78
    __nolibc_sprintf_cb                           58      91     +33
    itoa_r.isra                                   60      75     +15
    utoa_r.isra                                  144       -    -144
    __nolibc_printf                             1081     729    -352
(All these functions include ~40 bytes for the stack protector code.)
utoa_r.isra and _nolibc_u64toa_base.isra pretty much cancel each other out.
itoa_r.isra grows slightly since it calls _nolibc_u64toa_base().
strerror() used to be inlined, but over half of it is the stack check.
While some of the code added to __nolibc_sprintf_cb() has come out of
__nolibc_printf() 16-20 bytes is removed from the caller.
So there is a net saving of about 280 bytes (including losing a copy of
the number to ascii code).

The later patches add code back in:
    patch 13 - conversion flags " +#"            +80 bytes
    patch 14 - left aligning fields              +38 bytes
    patch 15 - zero padding and field precision +260 bytes
    patch 16 - octal output                      +34 bytes
So probably about +130 bytes, but it will depend on what the application
actually calls and inlining decisions made by the compiler.
(All x86-64, other architectures will vary.)

The biggest size change is probably removing the .data from strerror().
This reduced the program binary file by 4k if it is the only initialised
data in a small program.

Changes for v4:
- Old patches 2, 3 and 6 have been applied to nolibc-next and removed.
- Patch 7 (fix padding) has been moved before the selftest changes
  so that the tests don't fail.
- NOLIBC_IGNORE_ERRNO is left only applying to %m and not strerror().
- Avoid calling memcpy(tgt, NULL, 0) in the vsnprintf callback function.
- Some of the patches have been split.
The final version of stdio.h only differs from v3 in the strerror()
and vsnprintf callback code.

Significant changes for v3:
The patches have been re-ordered, split and joined but the final code
is pretty much the same as v2.
- Include the patch to stdlib.h that optimises the 'number to ascii'
  functions. This is needed for the final patch that adds octal support.
  This includes a fix to the previous version that could generate negative
  digits.
- Add octal support in the final patch.
- Update the selftests as new features are added.
- The patch to buffer fprintf() output has been removed.

Changes for v2:
Mostly changes to improve the readability of the code.
- New patch #1 inserted to rename the variable 'c' to 'ch'.
- Use #define 'magic' for the bit-masks that check multiple characters.
  The check for the conversion flag characters is then based on:
        ch_flag = _NOLIBC_PF_CHAR_IS_ONE_OF(ch, ' ', '#', '+', '-', '0');
- Re-order the changes so that the old patch 10 (Use bit-pattern for
  integral formats) is done at the same time as bit-masks are used for
  the flags characters and length modifiers.
  This means the restructuring changes are done before new features are
  added.
- Put all the changes to the selftest together at the end.
  There is one extra test for ("%#01x", 0x1234) (should be "0x1234")
  which is problematic because once you've removed the length of the "0x"
  from the field width there are -1 character postions for the digits.

David Laight (23):
  tools/nolibc: Add _NOLIBC_OPTIMIZER_HIDE_VAR() to compiler.h
  tools/nolibc/printf: Move snprintf length check to callback
  selftests/nolibc: Return correct value when printf test fails
  selftests/nolibc: check vsnprintf() output buffer before the length
  selftests/nolibc: Use length of 'expected' string to check snprintf()
    output
  selftests/nolibc: Check that snprintf() doesn't write beyond the
    buffer end
  selftests/nolibc: Let EXPECT_VFPRINTF() tests be skipped
  selftests/nolibc: Rename w to written in expect_vfprintf()
  tools/nolibc: Implement strerror() in terms of strerror_r()
  tools/nolibc: Rename the 'errnum' parameter to strerror()
  tools/nolibc/printf: Output pad characters in 16 byte chunks
  tools/nolibc/printf: Simplify __nolibc_printf()
  tools/nolibc/printf: Use goto and reduce indentation
  tools/nolibc/printf: Use bit-masks to hold requested  flag, length and
    conversion chars
  tools/nolibc/printf: Add support for length modifiers tzqL and formats
    iX
  tools/nolibc/printf: Handle "%s" with the numeric formats
  tools/nolibc/printf: Prepend sign to converted number
  tools/nolibc/printf: Add support for conversion flags space and plus
  tools/nolibc/printf: Special case 0 and add support for %#x
  tools/nolibc/printf: Add support for left aligning fields
  tools/nolibc/printf: Add support for zero padding and field precision
  tools/nolibc/printf: Add support for octal output
  selftests/nolibc: Use printf variable field widths and precisions

 tools/include/nolibc/compiler.h              |   3 +
 tools/include/nolibc/stdio.h                 | 501 ++++++++++++++-----
 tools/testing/selftests/nolibc/nolibc-test.c | 157 +++---
 3 files changed, 469 insertions(+), 192 deletions(-)

-- 
2.39.5
Re: [PATCH v4 next 00/23] Enhance printf()
Posted by Thomas Weißschuh 1 month ago
Hi David,

On 2026-03-02 10:17:52+0000, david.laight.linux@gmail.com wrote:
> From: David Laight <david.laight.linux@gmail.com>

(...)

I am happy with the patches of this series.

> David Laight (23):
>   tools/nolibc: Add _NOLIBC_OPTIMIZER_HIDE_VAR() to compiler.h

>   tools/nolibc/printf: Move snprintf length check to callback
>   selftests/nolibc: Return correct value when printf test fails
>   selftests/nolibc: check vsnprintf() output buffer before the length
>   selftests/nolibc: Use length of 'expected' string to check snprintf() output
>   selftests/nolibc: Check that snprintf() doesn't write beyond the buffer end
>   selftests/nolibc: Let EXPECT_VFPRINTF() tests be skipped

>   selftests/nolibc: Rename w to written in expect_vfprintf()

Unfortunately b4 chokes on these patches because this patch is missing
the 'v4' tag in the subject prefix. Given that the one below needs some
changes anyways, I was lazy and applied the series only up until here.
(Patch 1 is also not applied, as there was no user yet for
_NOLIBC_OPTIMIZER_HIDE_VAR() ).

Could you rebase the series on nolibc-next, add the error handling
to strerror_r(), fix the wording nitpicks from Willy and resend the
patches? I can also try to fix this up locally, but that would be more
work on my side than it would be for you I reckon.
Let me know if this is an issue and I'll try to make it work.

Willy:

I interpreted your mail [0] as Acked-by for the whole series.

[0] https://lore.kernel.org/lkml/aawDlqLdpgsfGI4r@1wt.eu/

>   tools/nolibc: Implement strerror() in terms of strerror_r()
>   tools/nolibc: Rename the 'errnum' parameter to strerror()
>   tools/nolibc/printf: Output pad characters in 16 byte chunks
>   tools/nolibc/printf: Simplify __nolibc_printf()
>   tools/nolibc/printf: Use goto and reduce indentation
>   tools/nolibc/printf: Use bit-masks to hold requested  flag, length and conversion chars
>   tools/nolibc/printf: Add support for length modifiers tzqL and formats iX
>   tools/nolibc/printf: Handle "%s" with the numeric formats
>   tools/nolibc/printf: Prepend sign to converted number
>   tools/nolibc/printf: Add support for conversion flags space and plus
>   tools/nolibc/printf: Special case 0 and add support for %#x
>   tools/nolibc/printf: Add support for left aligning fields
>   tools/nolibc/printf: Add support for zero padding and field precision
>   tools/nolibc/printf: Add support for octal output
>   selftests/nolibc: Use printf variable field widths and precisions
> 
>  tools/include/nolibc/compiler.h              |   3 +
>  tools/include/nolibc/stdio.h                 | 501 ++++++++++++++-----
>  tools/testing/selftests/nolibc/nolibc-test.c | 157 +++---
>  3 files changed, 469 insertions(+), 192 deletions(-)
Re: [PATCH v4 next 00/23] Enhance printf()
Posted by Willy Tarreau 1 month ago
On Sat, Mar 07, 2026 at 07:02:30PM +0100, Thomas Weißschuh wrote:
> Hi David,
> 
> On 2026-03-02 10:17:52+0000, david.laight.linux@gmail.com wrote:
> > From: David Laight <david.laight.linux@gmail.com>
> 
> (...)
> 
> I am happy with the patches of this series.
> 
> > David Laight (23):
> >   tools/nolibc: Add _NOLIBC_OPTIMIZER_HIDE_VAR() to compiler.h
> 
> >   tools/nolibc/printf: Move snprintf length check to callback
> >   selftests/nolibc: Return correct value when printf test fails
> >   selftests/nolibc: check vsnprintf() output buffer before the length
> >   selftests/nolibc: Use length of 'expected' string to check snprintf() output
> >   selftests/nolibc: Check that snprintf() doesn't write beyond the buffer end
> >   selftests/nolibc: Let EXPECT_VFPRINTF() tests be skipped
> 
> >   selftests/nolibc: Rename w to written in expect_vfprintf()
> 
> Unfortunately b4 chokes on these patches because this patch is missing
> the 'v4' tag in the subject prefix. Given that the one below needs some
> changes anyways, I was lazy and applied the series only up until here.
> (Patch 1 is also not applied, as there was no user yet for
> _NOLIBC_OPTIMIZER_HIDE_VAR() ).
> 
> Could you rebase the series on nolibc-next, add the error handling
> to strerror_r(), fix the wording nitpicks from Willy and resend the
> patches? I can also try to fix this up locally, but that would be more
> work on my side than it would be for you I reckon.
> Let me know if this is an issue and I'll try to make it work.
> 
> Willy:
> 
> I interpreted your mail [0] as Acked-by for the whole series.
> [0] https://lore.kernel.org/lkml/aawDlqLdpgsfGI4r@1wt.eu/

Yep definitely!

Willy
Re: [PATCH v4 next 00/23] Enhance printf()
Posted by David Laight 1 month ago
On Sat, 7 Mar 2026 19:02:30 +0100
Thomas Weißschuh <linux@weissschuh.net> wrote:

> Hi David,
> 
> On 2026-03-02 10:17:52+0000, david.laight.linux@gmail.com wrote:
> > From: David Laight <david.laight.linux@gmail.com>  
> 
> (...)
> 
> I am happy with the patches of this series.
> 
> > David Laight (23):
> >   tools/nolibc: Add _NOLIBC_OPTIMIZER_HIDE_VAR() to compiler.h  
> 
> >   tools/nolibc/printf: Move snprintf length check to callback
> >   selftests/nolibc: Return correct value when printf test fails
> >   selftests/nolibc: check vsnprintf() output buffer before the length
> >   selftests/nolibc: Use length of 'expected' string to check snprintf() output
> >   selftests/nolibc: Check that snprintf() doesn't write beyond the buffer end
> >   selftests/nolibc: Let EXPECT_VFPRINTF() tests be skipped  
> 
> >   selftests/nolibc: Rename w to written in expect_vfprintf()  
> 
> Unfortunately b4 chokes on these patches because this patch is missing
> the 'v4' tag in the subject prefix.

I noticed that after sending them.
I add it by hand (there might be an easier way) but it is easy to
miss it when doing a final check/update of the patches.
I then sent them without a final-final check.

> Given that the one below needs some
> changes anyways, I was lazy and applied the series only up until here.
> (Patch 1 is also not applied, as there was no user yet for
> _NOLIBC_OPTIMIZER_HIDE_VAR() ).
> 
> Could you rebase the series on nolibc-next, add the error handling
> to strerror_r(), fix the wording nitpicks from Willy and resend the
> patches? I can also try to fix this up locally, but that would be more
> work on my side than it would be for you I reckon.
> Let me know if this is an issue and I'll try to make it work.

That shouldn't be too hard.
Was a right PITA moving them from linus's tree because --3way doesn't
work when the patches come from different git trees.
I'll just create a new branch, use 'git am' to apply each patch
then edit and amend.
Then recover all the info after the --- line.
I'm getting used to that sequence :-)

	David

> 
> Willy:
> 
> I interpreted your mail [0] as Acked-by for the whole series.
> 
> [0] https://lore.kernel.org/lkml/aawDlqLdpgsfGI4r@1wt.eu/
> 
> >   tools/nolibc: Implement strerror() in terms of strerror_r()
> >   tools/nolibc: Rename the 'errnum' parameter to strerror()
> >   tools/nolibc/printf: Output pad characters in 16 byte chunks
> >   tools/nolibc/printf: Simplify __nolibc_printf()
> >   tools/nolibc/printf: Use goto and reduce indentation
> >   tools/nolibc/printf: Use bit-masks to hold requested  flag, length and conversion chars
> >   tools/nolibc/printf: Add support for length modifiers tzqL and formats iX
> >   tools/nolibc/printf: Handle "%s" with the numeric formats
> >   tools/nolibc/printf: Prepend sign to converted number
> >   tools/nolibc/printf: Add support for conversion flags space and plus
> >   tools/nolibc/printf: Special case 0 and add support for %#x
> >   tools/nolibc/printf: Add support for left aligning fields
> >   tools/nolibc/printf: Add support for zero padding and field precision
> >   tools/nolibc/printf: Add support for octal output
> >   selftests/nolibc: Use printf variable field widths and precisions
> > 
> >  tools/include/nolibc/compiler.h              |   3 +
> >  tools/include/nolibc/stdio.h                 | 501 ++++++++++++++-----
> >  tools/testing/selftests/nolibc/nolibc-test.c | 157 +++---
> >  3 files changed, 469 insertions(+), 192 deletions(-)  
Re: [PATCH v4 next 00/23] Enhance printf()
Posted by Thomas Weißschuh 1 month ago
On 2026-03-07 22:03:48+0000, David Laight wrote:
> On Sat, 7 Mar 2026 19:02:30 +0100
> Thomas Weißschuh <linux@weissschuh.net> wrote:
> 
> > Hi David,
> > 
> > On 2026-03-02 10:17:52+0000, david.laight.linux@gmail.com wrote:
> > > From: David Laight <david.laight.linux@gmail.com>  
> > 
> > (...)
> > 
> > I am happy with the patches of this series.
> > 
> > > David Laight (23):
> > >   tools/nolibc: Add _NOLIBC_OPTIMIZER_HIDE_VAR() to compiler.h  
> > 
> > >   tools/nolibc/printf: Move snprintf length check to callback
> > >   selftests/nolibc: Return correct value when printf test fails
> > >   selftests/nolibc: check vsnprintf() output buffer before the length
> > >   selftests/nolibc: Use length of 'expected' string to check snprintf() output
> > >   selftests/nolibc: Check that snprintf() doesn't write beyond the buffer end
> > >   selftests/nolibc: Let EXPECT_VFPRINTF() tests be skipped  
> > 
> > >   selftests/nolibc: Rename w to written in expect_vfprintf()  
> > 
> > Unfortunately b4 chokes on these patches because this patch is missing
> > the 'v4' tag in the subject prefix.
> 
> I noticed that after sending them.
> I add it by hand (there might be an easier way) but it is easy to
> miss it when doing a final check/update of the patches.
> I then sent them without a final-final check.

git send-email/format-patch have --reroll-count for that.
b4 does manages the reroll count automatically.

> > Given that the one below needs some
> > changes anyways, I was lazy and applied the series only up until here.
> > (Patch 1 is also not applied, as there was no user yet for
> > _NOLIBC_OPTIMIZER_HIDE_VAR() ).
> > 
> > Could you rebase the series on nolibc-next, add the error handling
> > to strerror_r(), fix the wording nitpicks from Willy and resend the
> > patches? I can also try to fix this up locally, but that would be more
> > work on my side than it would be for you I reckon.
> > Let me know if this is an issue and I'll try to make it work.
> 
> That shouldn't be too hard.
> Was a right PITA moving them from linus's tree because --3way doesn't
> work when the patches come from different git trees.
> I'll just create a new branch, use 'git am' to apply each patch
> then edit and amend.
> Then recover all the info after the --- line.
> I'm getting used to that sequence :-)

This looks like a usecase for 'git rebase (--onto)'.

In any case: Thanks!


Thomas
Re: [PATCH v4 next 00/23] Enhance printf()
Posted by Willy Tarreau 1 month ago
Hi David,

On Mon, Mar 02, 2026 at 10:17:52AM +0000, david.laight.linux@gmail.com wrote:
> From: David Laight <david.laight.linux@gmail.com>
> 
> Update printf() so that it handles almost all the non-fp formats.
> In particular:
> - Left alignment.
> - Zero padding.
> - Alternate form "%#x" and "%#o".
> - Field precision.
> - Variable field width and precision.
> - Width modifiers q, L, t and z.
> - Conversion specifiers i, o and X (X generates lower case).
> About the only things that are missing are wide chanacters and floating point.
> 
> The tests are updated to match.
> 
> Bloat/savings (in nolibc-test, but excluding the program) to patch 14:
> (Measured for v3)
>     Function                                     old     new   delta
>     _nolibc_u64toa_base.isra                       -     143    +143
>     strerror                                       -      78     +78
>     __nolibc_sprintf_cb                           58      91     +33
>     itoa_r.isra                                   60      75     +15
>     utoa_r.isra                                  144       -    -144
>     __nolibc_printf                             1081     729    -352
> (All these functions include ~40 bytes for the stack protector code.)
> utoa_r.isra and _nolibc_u64toa_base.isra pretty much cancel each other out.
> itoa_r.isra grows slightly since it calls _nolibc_u64toa_base().
> strerror() used to be inlined, but over half of it is the stack check.
> While some of the code added to __nolibc_sprintf_cb() has come out of
> __nolibc_printf() 16-20 bytes is removed from the caller.
> So there is a net saving of about 280 bytes (including losing a copy of
> the number to ascii code).
> 
> The later patches add code back in:
>     patch 13 - conversion flags " +#"            +80 bytes
>     patch 14 - left aligning fields              +38 bytes
>     patch 15 - zero padding and field precision +260 bytes
>     patch 16 - octal output                      +34 bytes
> So probably about +130 bytes, but it will depend on what the application
> actually calls and inlining decisions made by the compiler.
> (All x86-64, other architectures will vary.)
> 
> The biggest size change is probably removing the .data from strerror().
> This reduced the program binary file by 4k if it is the only initialised
> data in a small program.

Thanks for this work. I'm personally fine with the series overall, even
though there are tiny comments here and there but nothing blocking. Let's
wait for Thomas' finaly review however.

I agree that having a more featureful printf() will definitely help in
various areas since it's used a lot in low-level userland code and testing.

Thanks,
Willy