Hello Paul,

Here comes an update to the nolibc subsystem to keep it in sync with
the out-of-tree project and to bring a number of usability improvements.

The approach here consists in better splitting the layers so that
arch-specific syscall functions are now in their own files (one per
arch), then syscalls are implemented on top of this in a generic file,
then the generic libc stuff is exposed. This way it becomes easier to
add support for new architectures if needed.

I exported 3 functions as weak symbols: memset(), memmove() and abort().
For the first two ones, gcc/clang may issue calls to them and will not
always fall back to the builtin equivalent. For abort(), I've seen it
referenced by libgcc from some divide functions.

A few stdio-level functions were added (unbuffered), including a quite
limited but usable printf() which significantly eases debugging of test
programs by allowing to print return codes or pointers for example.

A few functions and syscalls like abort(), raise(), time() and usleep()
were added. More str* functions were added as well. open() now takes a
vararg on the 3rd argument so as to ease porting of existing programs.

Finally, the functions and definitions were moved to the appropriate
files as documented in their man pages when relevant, so that simple
programs which only use a few include files among stdio.h, stdlib.h,
string.h, unistd.h, errno.h, ctype.h, signal.h, time.h or types.h can
build without modification. Obviously this remains fairly limited in
scope, but for test programs that's usually fine.

I could verify that a few of my programs and that your rcutorture test
continue to work (not much surprising since most of the work only
consists in moving functions between files).

There's obviously no rush, but I preferred to resync the version here
given that Mark expressed some interest recently, and I'd hate it if
he had to add new syscalls that already existed out of tree! Comments,
suggestions and feature requests welcome as usual. If you prefer to
pull, it's also available here, on top of 5.17-rc3:

  git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20220206-nolibc-split-2

Thank you!
Willy

---
Willy Tarreau (42):
  tools/nolibc: use pselect6 on RISCV
  tools/nolibc: guard the main file against multiple inclusion
  tools/nolibc/std: move the standard type definitions to std.h
  tools/nolibc/types: split syscall-specific definitions into their own files
  tools/nolibc/arch: split arch-specific code into individual files
  tools/nolibc/sys: split the syscall definitions into their own file
  tools/nolibc/stdlib: extract the stdlib-specific functions to their own file
  tools/nolibc/string: split the string functions into string.h
  tools/nolibc/ctype: split the is* functions to ctype.h
  tools/nolibc/ctype: add the missing is* functions
  tools/nolibc/types: move the FD_* functions to macros in types.h
  tools/nolibc/types: make FD_SETSIZE configurable
  tools/nolibc/types: move makedev to types.h and make it a macro
  tools/nolibc/stdlib: move ltoa() to stdlib.h
  tools/nolibc/stdlib: replace the ltoa() function with more efficient ones
  tools/nolibc/stdlib: add i64toa() and u64toa()
  tools/nolibc/stdlib: add utoh() and u64toh()
  tools/nolibc/stdio: add a minimal set of stdio functions
  tools/nolibc/stdio: add stdin/stdout/stderr and fget*/fput* functions
  tools/nolibc/stdio: add fwrite() to stdio
  tools/nolibc/stdio: add a minimal [vf]printf() implementation
  tools/nolibc/types: define EXIT_SUCCESS and EXIT_FAILURE
  tools/nolibc/stdio: add perror() to report the errno value
  tools/nolibc/sys: make open() take a vararg on the 3rd argument
  tools/nolibc/stdlib: avoid a 64-bit shift in u64toh_r()
  tools/nolibc/stdlib: make raise() use the lower level syscalls only
  tools/nolibc/sys: make getpgrp(), getpid(), gettid() not set errno
  tools/nolibc/string: use unidirectional variants for memcpy()
  tools/nolibc/string: slightly simplify memmove()
  tools/nolibc/string: add strncpy() and strlcpy()
  tools/nolibc/string: add tiny versions of strncat() and strlcat()
  tools/nolibc: move exported functions to their own section
  tools/nolibc/arch: mark the _start symbol as weak
  tools/nolibc/types: define PATH_MAX and MAXPATHLEN
  tools/nolibc/string: export memset() and memmove()
  tools/nolibc/errno: extract errno.h from sys.h
  tools/nolibc/unistd: extract msleep(), sleep(), tcsetpgrp() to unistd.h
  tools/nolibc/unistd: add usleep()
  tools/nolibc/signal: move raise() to signal.h
  tools/nolibc/time: create time.h with time()
  tools/nolibc: also mention how to build by just setting the include path
  tools/nolibc/stdlib: implement abort()

 tools/include/nolibc/arch-aarch64.h |  200 +++
 tools/include/nolibc/arch-arm.h     |  205 +++
 tools/include/nolibc/arch-i386.h    |  197 +++
 tools/include/nolibc/arch-mips.h    |  216 +++
 tools/include/nolibc/arch-riscv.h   |  205 +++
 tools/include/nolibc/arch-x86_64.h  |  216 +++
 tools/include/nolibc/arch.h         |   32 +
 tools/include/nolibc/ctype.h        |   99 ++
 tools/include/nolibc/errno.h        |   27 +
 tools/include/nolibc/nolibc.h       | 2540 +--------------------------
 tools/include/nolibc/signal.h       |   22 +
 tools/include/nolibc/std.h          |   49 +
 tools/include/nolibc/stdio.h        |  296 ++++
 tools/include/nolibc/stdlib.h       |  311 ++++
 tools/include/nolibc/string.h       |  212 +++
 tools/include/nolibc/sys.h          | 1168 ++++++++++++
 tools/include/nolibc/time.h         |   28 +
 tools/include/nolibc/types.h        |  184 ++
 tools/include/nolibc/unistd.h       |   54 +
 19 files changed, 3756 insertions(+), 2505 deletions(-)
 create mode 100644 tools/include/nolibc/arch-aarch64.h
 create mode 100644 tools/include/nolibc/arch-arm.h
 create mode 100644 tools/include/nolibc/arch-i386.h
 create mode 100644 tools/include/nolibc/arch-mips.h
 create mode 100644 tools/include/nolibc/arch-riscv.h
 create mode 100644 tools/include/nolibc/arch-x86_64.h
 create mode 100644 tools/include/nolibc/arch.h
 create mode 100644 tools/include/nolibc/ctype.h
 create mode 100644 tools/include/nolibc/errno.h
 create mode 100644 tools/include/nolibc/signal.h
 create mode 100644 tools/include/nolibc/std.h
 create mode 100644 tools/include/nolibc/stdio.h
 create mode 100644 tools/include/nolibc/stdlib.h
 create mode 100644 tools/include/nolibc/string.h
 create mode 100644 tools/include/nolibc/sys.h
 create mode 100644 tools/include/nolibc/time.h
 create mode 100644 tools/include/nolibc/types.h
 create mode 100644 tools/include/nolibc/unistd.h

-- 
2.35.1

On Mon, Feb 07, 2022 at 05:23:12PM +0100, Willy Tarreau wrote:
> Hello Paul,
> 
> Here comes an update to the nolibc subsystem to keep it in sync with
> the out-of-tree project and to bring a number of usability improvements.
> 
> The approach here consists in better splitting the layers so that
> arch-specific syscall functions are now in their own files (one per
> arch), then syscalls are implemented on top of this in a generic file,
> then the generic libc stuff is exposed. This way it becomes easier to
> add support for new architectures if needed.
> 
> I exported 3 functions as weak symbols: memset(), memmove() and abort().
> For the first two ones, gcc/clang may issue calls to them and will not
> always fall back to the builtin equivalent. For abort(), I've seen it
> referenced by libgcc from some divide functions.
> 
> A few stdio-level functions were added (unbuffered), including a quite
> limited but usable printf() which significantly eases debugging of test
> programs by allowing to print return codes or pointers for example.
> 
> A few functions and syscalls like abort(), raise(), time() and usleep()
> were added. More str* functions were added as well. open() now takes a
> vararg on the 3rd argument so as to ease porting of existing programs.
> 
> Finally, the functions and definitions were moved to the appropriate
> files as documented in their man pages when relevant, so that simple
> programs which only use a few include files among stdio.h, stdlib.h,
> string.h, unistd.h, errno.h, ctype.h, signal.h, time.h or types.h can
> build without modification. Obviously this remains fairly limited in
> scope, but for test programs that's usually fine.
> 
> I could verify that a few of my programs and that your rcutorture test
> continue to work (not much surprising since most of the work only
> consists in moving functions between files).
> 
> There's obviously no rush, but I preferred to resync the version here
> given that Mark expressed some interest recently, and I'd hate it if
> he had to add new syscalls that already existed out of tree! Comments,
> suggestions and feature requests welcome as usual. If you prefer to
> pull, it's also available here, on top of 5.17-rc3:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20220206-nolibc-split-2

I queued these from email to get some time on them.  If an update is
needed (for example, based on discussion with David Laight), please feel
free to send me an update and I can drop the current set in favor of a
new version.

Good stuff, by the way, thank you!!!

Hmmm...  I should make rcutorture able to smoke-test this.  My thought
would be to add some code to rcutorture's init.c that tried out the
system calls.  But I bet that you already have some test code.  ;-)

One approach would be to place the test code in tools/nolibc, and
have rcutorture's init.c either #include that or link to it.

Thoughts?  Especially thoughts about better testing approaches?

							Thanx, Paul

> Thank you!
> Willy
> 
> ---
> Willy Tarreau (42):
>   tools/nolibc: use pselect6 on RISCV
>   tools/nolibc: guard the main file against multiple inclusion
>   tools/nolibc/std: move the standard type definitions to std.h
>   tools/nolibc/types: split syscall-specific definitions into their own files
>   tools/nolibc/arch: split arch-specific code into individual files
>   tools/nolibc/sys: split the syscall definitions into their own file
>   tools/nolibc/stdlib: extract the stdlib-specific functions to their own file
>   tools/nolibc/string: split the string functions into string.h
>   tools/nolibc/ctype: split the is* functions to ctype.h
>   tools/nolibc/ctype: add the missing is* functions
>   tools/nolibc/types: move the FD_* functions to macros in types.h
>   tools/nolibc/types: make FD_SETSIZE configurable
>   tools/nolibc/types: move makedev to types.h and make it a macro
>   tools/nolibc/stdlib: move ltoa() to stdlib.h
>   tools/nolibc/stdlib: replace the ltoa() function with more efficient ones
>   tools/nolibc/stdlib: add i64toa() and u64toa()
>   tools/nolibc/stdlib: add utoh() and u64toh()
>   tools/nolibc/stdio: add a minimal set of stdio functions
>   tools/nolibc/stdio: add stdin/stdout/stderr and fget*/fput* functions
>   tools/nolibc/stdio: add fwrite() to stdio
>   tools/nolibc/stdio: add a minimal [vf]printf() implementation
>   tools/nolibc/types: define EXIT_SUCCESS and EXIT_FAILURE
>   tools/nolibc/stdio: add perror() to report the errno value
>   tools/nolibc/sys: make open() take a vararg on the 3rd argument
>   tools/nolibc/stdlib: avoid a 64-bit shift in u64toh_r()
>   tools/nolibc/stdlib: make raise() use the lower level syscalls only
>   tools/nolibc/sys: make getpgrp(), getpid(), gettid() not set errno
>   tools/nolibc/string: use unidirectional variants for memcpy()
>   tools/nolibc/string: slightly simplify memmove()
>   tools/nolibc/string: add strncpy() and strlcpy()
>   tools/nolibc/string: add tiny versions of strncat() and strlcat()
>   tools/nolibc: move exported functions to their own section
>   tools/nolibc/arch: mark the _start symbol as weak
>   tools/nolibc/types: define PATH_MAX and MAXPATHLEN
>   tools/nolibc/string: export memset() and memmove()
>   tools/nolibc/errno: extract errno.h from sys.h
>   tools/nolibc/unistd: extract msleep(), sleep(), tcsetpgrp() to unistd.h
>   tools/nolibc/unistd: add usleep()
>   tools/nolibc/signal: move raise() to signal.h
>   tools/nolibc/time: create time.h with time()
>   tools/nolibc: also mention how to build by just setting the include path
>   tools/nolibc/stdlib: implement abort()
> 
>  tools/include/nolibc/arch-aarch64.h |  200 +++
>  tools/include/nolibc/arch-arm.h     |  205 +++
>  tools/include/nolibc/arch-i386.h    |  197 +++
>  tools/include/nolibc/arch-mips.h    |  216 +++
>  tools/include/nolibc/arch-riscv.h   |  205 +++
>  tools/include/nolibc/arch-x86_64.h  |  216 +++
>  tools/include/nolibc/arch.h         |   32 +
>  tools/include/nolibc/ctype.h        |   99 ++
>  tools/include/nolibc/errno.h        |   27 +
>  tools/include/nolibc/nolibc.h       | 2540 +--------------------------
>  tools/include/nolibc/signal.h       |   22 +
>  tools/include/nolibc/std.h          |   49 +
>  tools/include/nolibc/stdio.h        |  296 ++++
>  tools/include/nolibc/stdlib.h       |  311 ++++
>  tools/include/nolibc/string.h       |  212 +++
>  tools/include/nolibc/sys.h          | 1168 ++++++++++++
>  tools/include/nolibc/time.h         |   28 +
>  tools/include/nolibc/types.h        |  184 ++
>  tools/include/nolibc/unistd.h       |   54 +
>  19 files changed, 3756 insertions(+), 2505 deletions(-)
>  create mode 100644 tools/include/nolibc/arch-aarch64.h
>  create mode 100644 tools/include/nolibc/arch-arm.h
>  create mode 100644 tools/include/nolibc/arch-i386.h
>  create mode 100644 tools/include/nolibc/arch-mips.h
>  create mode 100644 tools/include/nolibc/arch-riscv.h
>  create mode 100644 tools/include/nolibc/arch-x86_64.h
>  create mode 100644 tools/include/nolibc/arch.h
>  create mode 100644 tools/include/nolibc/ctype.h
>  create mode 100644 tools/include/nolibc/errno.h
>  create mode 100644 tools/include/nolibc/signal.h
>  create mode 100644 tools/include/nolibc/std.h
>  create mode 100644 tools/include/nolibc/stdio.h
>  create mode 100644 tools/include/nolibc/stdlib.h
>  create mode 100644 tools/include/nolibc/string.h
>  create mode 100644 tools/include/nolibc/sys.h
>  create mode 100644 tools/include/nolibc/time.h
>  create mode 100644 tools/include/nolibc/types.h
>  create mode 100644 tools/include/nolibc/unistd.h
> 
> -- 
> 2.35.1
> 

On Mon, Feb 07, 2022 at 04:00:28PM -0800, Paul E. McKenney wrote:
> I queued these from email to get some time on them.  If an update is
> needed (for example, based on discussion with David Laight), please feel
> free to send me an update and I can drop the current set in favor of a
> new version.

OK thanks!

> Good stuff, by the way, thank you!!!

thanks ;-)

> Hmmm...  I should make rcutorture able to smoke-test this.  My thought
> would be to add some code to rcutorture's init.c that tried out the
> system calls.

I don't think we should abuse rcutorture to test nolibc, however you
could decide that rcutorture could benefit from some improvements
(take arguments, parse env or /proc/cmdline, report some info such
as syscall errors etc).

> But I bet that you already have some test code.  ;-)

Actually my testing remains limited in terms of reproducibility. For
most syscalls I used to just modify a simple test file (hello.c) to
update the syscall and generally run it under strace; in addition,
my preinit code builds with it and all my kernels rely on it, which
gives me extra confidence I didn't break common stuff.

I've thought about starting to create a real test suite for all these
calls, either just one syscall or function per file (for easier testing),
or one test file per test unit (e.g. one for stdlib, one for sys, etc),
or maybe something in between.

> One approach would be to place the test code in tools/nolibc, and
> have rcutorture's init.c either #include that or link to it.

There could be an interesting idea to develop here. While I don't find
it logical to abuse rcutorture for this, we could instead consider a
more general init-based test suite that could cover various aspects
including syscalls and rcutorture. We could let the user choose at
build time what to implement (so that we don't waste testing time on
the libc aspect when rcutorture is needed for example), and/or force
tests on the cmdline (convenient under qemu for example). It could
also be used to verify if some syscalls were dropped due to some
config options being turned off.

I was thinking about having just a numbered suite that iterates over
numerous tests and provides a test number, a function/syscall name,
a variant and a result. Something that could easily be copy-pasted
to ease reporting of breakage.

This could be especially helpful to port to other architectures. For
example we're still missing PPC and I don't have much experience there
(the only one I have access to runs and old version of AIX). And a
test suite could easily tell if something is broken such as a
non-implemented syscall.

I was hesitant about where to place such a test suite, because I didn't
want to pollute the include files with tests. In the out-of-tree code
there's already a "hello.c" file and I started to think about moving the
rest to include/. But your proposal of tools/nolibc makes sense in that
it would keep the suite separate from the include files. Actually I'd
rather put that under tools/testing/selftests/nolibc. With this done,
I think it would remove some maintenance burden and I could probably
just kill the out-of-tree project and consider that the up-to-date one
is in the kernel.

> Thoughts?  Especially thoughts about better testing approaches?

Hehe, each time I think I'm done for a while on this project, you
manage to fuel me with many interesting ideas for improvements ;-)

Cheers,
Willy

On Tue, Feb 08, 2022 at 05:41:50AM +0100, Willy Tarreau wrote:
> On Mon, Feb 07, 2022 at 04:00:28PM -0800, Paul E. McKenney wrote:
> > I queued these from email to get some time on them.  If an update is
> > needed (for example, based on discussion with David Laight), please feel
> > free to send me an update and I can drop the current set in favor of a
> > new version.
> 
> OK thanks!
> 
> > Good stuff, by the way, thank you!!!
> 
> thanks ;-)
> 
> > Hmmm...  I should make rcutorture able to smoke-test this.  My thought
> > would be to add some code to rcutorture's init.c that tried out the
> > system calls.
> 
> I don't think we should abuse rcutorture to test nolibc, however you
> could decide that rcutorture could benefit from some improvements
> (take arguments, parse env or /proc/cmdline, report some info such
> as syscall errors etc).
> 
> > But I bet that you already have some test code.  ;-)
> 
> Actually my testing remains limited in terms of reproducibility. For
> most syscalls I used to just modify a simple test file (hello.c) to
> update the syscall and generally run it under strace; in addition,
> my preinit code builds with it and all my kernels rely on it, which
> gives me extra confidence I didn't break common stuff.
> 
> I've thought about starting to create a real test suite for all these
> calls, either just one syscall or function per file (for easier testing),
> or one test file per test unit (e.g. one for stdlib, one for sys, etc),
> or maybe something in between.

I am mainly looking for a regression test I can run, with or without
rcutorture.  ;-)

> > One approach would be to place the test code in tools/nolibc, and
> > have rcutorture's init.c either #include that or link to it.
> 
> There could be an interesting idea to develop here. While I don't find
> it logical to abuse rcutorture for this, we could instead consider a
> more general init-based test suite that could cover various aspects
> including syscalls and rcutorture. We could let the user choose at
> build time what to implement (so that we don't waste testing time on
> the libc aspect when rcutorture is needed for example), and/or force
> tests on the cmdline (convenient under qemu for example). It could
> also be used to verify if some syscalls were dropped due to some
> config options being turned off.
> 
> I was thinking about having just a numbered suite that iterates over
> numerous tests and provides a test number, a function/syscall name,
> a variant and a result. Something that could easily be copy-pasted
> to ease reporting of breakage.
> 
> This could be especially helpful to port to other architectures. For
> example we're still missing PPC and I don't have much experience there
> (the only one I have access to runs and old version of AIX). And a
> test suite could easily tell if something is broken such as a
> non-implemented syscall.

All excellent points!

> I was hesitant about where to place such a test suite, because I didn't
> want to pollute the include files with tests. In the out-of-tree code
> there's already a "hello.c" file and I started to think about moving the
> rest to include/. But your proposal of tools/nolibc makes sense in that
> it would keep the suite separate from the include files. Actually I'd
> rather put that under tools/testing/selftests/nolibc. With this done,
> I think it would remove some maintenance burden and I could probably
> just kill the out-of-tree project and consider that the up-to-date one
> is in the kernel.

Agreed, tools/testing/selftests/nolibc makes a lot of sense.  As does
having things in-tree, especially given that it seems to have several
contributors now, which presumably means at least that many users.

> > Thoughts?  Especially thoughts about better testing approaches?
> 
> Hehe, each time I think I'm done for a while on this project, you
> manage to fuel me with many interesting ideas for improvements ;-)

"It is a service I provide."  ;-)

							Thanx, Paul