[v3] kselftest harness and nolibc compatibility

[PATCH v3 00/32] kselftest harness and nolibc compatibility

Posted by Thomas Weißschuh 10 months ago

Nolibc is useful for selftests as the test programs can be very small,
and compiled with just a kernel crosscompiler, without userspace support.
Currently nolibc is only usable with kselftest.h, not the more
convenient to use kselftest_harness.h
This series provides this compatibility by adding new features to nolibc
and removing the usage of problematic features from the harness.

The first half of the series are changes to the harness, the second one
are for nolibc. Both parts are very independent and should go through
different trees.
The last patch is not meant to be applied and serves as test that
everything works together correctly.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
---
Changes in v3:
- Send patches to correct kselftest harness maintainers
- Move harness selftest to dedicated directory
- Add harness selftest to MAINTAINERS
- Integrate harness selftest cleanup with the selftest framework
- Consistently use "kselftest harness" in commit messages
- Properly propagate kselftest harness failure
- Link to v2: https://lore.kernel.org/r/20250407-nolibc-kselftest-harness-v2-0-f8812f76e930@linutronix.de

Changes in v2:
- Rebase unto v6.15-rc1
- Rename internal nolibc symbols
- Handle edge case of waitpid(INT_MIN) == ESRCH
- Fix arm configurations for final testing patch
- Clean up global getopt.h variable declarations
- Add Acks from Willy
- Link to v1: https://lore.kernel.org/r/20250304-nolibc-kselftest-harness-v1-0-adca7cd231e2@linutronix.de

---
Thomas Weißschuh (32):
      selftests: harness: Add kselftest harness selftest
      selftests: harness: Use C89 comment style
      selftests: harness: Ignore unused variant argument warning
      selftests: harness: Mark functions without prototypes static
      selftests: harness: Remove inline qualifier for wrappers
      selftests: harness: Remove dependency on libatomic
      selftests: harness: Implement test timeouts through pidfd
      selftests: harness: Don't set setup_completed for fixtureless tests
      selftests: harness: Always provide "self" and "variant"
      selftests: harness: Move teardown conditional into test metadata
      selftests: harness: Add teardown callback to test metadata
      selftests: harness: Stop using setjmp()/longjmp()
      selftests: harness: Guard includes on nolibc
      tools/nolibc: handle intmax_t/uintmax_t in printf
      tools/nolibc: use intmax definitions from compiler
      tools/nolibc: use pselect6_time64 if available
      tools/nolibc: use ppoll_time64 if available
      tools/nolibc: add tolower() and toupper()
      tools/nolibc: add _exit()
      tools/nolibc: add setpgrp()
      tools/nolibc: implement waitpid() in terms of waitid()
      Revert "selftests/nolibc: use waitid() over waitpid()"
      tools/nolibc: add dprintf() and vdprintf()
      tools/nolibc: add getopt()
      tools/nolibc: allow different write callbacks in printf
      tools/nolibc: allow limiting of printf destination size
      tools/nolibc: add snprintf() and friends
      selftests/nolibc: use snprintf() for printf tests
      selftests/nolibc: rename vfprintf test suite
      selftests/nolibc: add test for snprintf() truncation
      tools/nolibc: implement width padding in printf()
      HACK: selftests/nolibc: demonstrate usage of the kselftest harness

 MAINTAINERS                                        |    1 +
 tools/include/nolibc/Makefile                      |    1 +
 tools/include/nolibc/getopt.h                      |  101 ++
 tools/include/nolibc/nolibc.h                      |    1 +
 tools/include/nolibc/stdint.h                      |    4 +-
 tools/include/nolibc/stdio.h                       |  127 +-
 tools/include/nolibc/string.h                      |   17 +
 tools/include/nolibc/sys.h                         |  105 +-
 tools/testing/selftests/Makefile                   |    1 +
 tools/testing/selftests/kselftest_harness.h        |  181 +-
 .../testing/selftests/kselftest_harness/.gitignore |    2 +
 tools/testing/selftests/kselftest_harness/Makefile |    7 +
 .../selftests/kselftest_harness/harness-selftest.c |  129 ++
 .../kselftest_harness/harness-selftest.expected    |   62 +
 .../kselftest_harness/harness-selftest.sh          |   13 +
 tools/testing/selftests/nolibc/Makefile            |   13 +-
 tools/testing/selftests/nolibc/harness-selftest.c  |    1 +
 tools/testing/selftests/nolibc/nolibc-test.c       | 1729 +-------------------
 tools/testing/selftests/nolibc/run-tests.sh        |    2 +-
 19 files changed, 637 insertions(+), 1860 deletions(-)
---
base-commit: 0af2f6be1b4281385b618cb86ad946eded089ac8
change-id: 20250130-nolibc-kselftest-harness-8b2c8cac43bf

Best regards,
-- 
Thomas Weißschuh <thomas.weissschuh@linutronix.de>

Re: [PATCH v3 00/32] kselftest harness and nolibc compatibility

Posted by Thomas Weißschuh 9 months, 3 weeks ago

Hi,

On Fri, Apr 11, 2025 at 11:00:24AM +0200, Thomas Weißschuh wrote:
> Nolibc is useful for selftests as the test programs can be very small,
> and compiled with just a kernel crosscompiler, without userspace support.
> Currently nolibc is only usable with kselftest.h, not the more
> convenient to use kselftest_harness.h
> This series provides this compatibility by adding new features to nolibc
> and removing the usage of problematic features from the harness.
> 
> The first half of the series are changes to the harness, the second one
> are for nolibc. Both parts are very independent and should go through
> different trees.

I need a few nolibc bits of this series (snprintf() and prep patches) to base
further patches on. For that I'd like to pick up all the nolibc patches from
this series through the nolibc tree. They got Acks from Willy.

Any objections?


Thomas

> The last patch is not meant to be applied and serves as test that
> everything works together correctly.
> 
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> ---
> Changes in v3:
> - Send patches to correct kselftest harness maintainers
> - Move harness selftest to dedicated directory
> - Add harness selftest to MAINTAINERS
> - Integrate harness selftest cleanup with the selftest framework
> - Consistently use "kselftest harness" in commit messages
> - Properly propagate kselftest harness failure
> - Link to v2: https://lore.kernel.org/r/20250407-nolibc-kselftest-harness-v2-0-f8812f76e930@linutronix.de
> 
> Changes in v2:
> - Rebase unto v6.15-rc1
> - Rename internal nolibc symbols
> - Handle edge case of waitpid(INT_MIN) == ESRCH
> - Fix arm configurations for final testing patch
> - Clean up global getopt.h variable declarations
> - Add Acks from Willy
> - Link to v1: https://lore.kernel.org/r/20250304-nolibc-kselftest-harness-v1-0-adca7cd231e2@linutronix.de
> 
> ---
> Thomas Weißschuh (32):
>       selftests: harness: Add kselftest harness selftest
>       selftests: harness: Use C89 comment style
>       selftests: harness: Ignore unused variant argument warning
>       selftests: harness: Mark functions without prototypes static
>       selftests: harness: Remove inline qualifier for wrappers
>       selftests: harness: Remove dependency on libatomic
>       selftests: harness: Implement test timeouts through pidfd
>       selftests: harness: Don't set setup_completed for fixtureless tests
>       selftests: harness: Always provide "self" and "variant"
>       selftests: harness: Move teardown conditional into test metadata
>       selftests: harness: Add teardown callback to test metadata
>       selftests: harness: Stop using setjmp()/longjmp()
>       selftests: harness: Guard includes on nolibc
>       tools/nolibc: handle intmax_t/uintmax_t in printf
>       tools/nolibc: use intmax definitions from compiler
>       tools/nolibc: use pselect6_time64 if available
>       tools/nolibc: use ppoll_time64 if available
>       tools/nolibc: add tolower() and toupper()
>       tools/nolibc: add _exit()
>       tools/nolibc: add setpgrp()
>       tools/nolibc: implement waitpid() in terms of waitid()
>       Revert "selftests/nolibc: use waitid() over waitpid()"
>       tools/nolibc: add dprintf() and vdprintf()
>       tools/nolibc: add getopt()
>       tools/nolibc: allow different write callbacks in printf
>       tools/nolibc: allow limiting of printf destination size
>       tools/nolibc: add snprintf() and friends
>       selftests/nolibc: use snprintf() for printf tests
>       selftests/nolibc: rename vfprintf test suite
>       selftests/nolibc: add test for snprintf() truncation
>       tools/nolibc: implement width padding in printf()
>       HACK: selftests/nolibc: demonstrate usage of the kselftest harness
> 
>  MAINTAINERS                                        |    1 +
>  tools/include/nolibc/Makefile                      |    1 +
>  tools/include/nolibc/getopt.h                      |  101 ++
>  tools/include/nolibc/nolibc.h                      |    1 +
>  tools/include/nolibc/stdint.h                      |    4 +-
>  tools/include/nolibc/stdio.h                       |  127 +-
>  tools/include/nolibc/string.h                      |   17 +
>  tools/include/nolibc/sys.h                         |  105 +-
>  tools/testing/selftests/Makefile                   |    1 +
>  tools/testing/selftests/kselftest_harness.h        |  181 +-
>  .../testing/selftests/kselftest_harness/.gitignore |    2 +
>  tools/testing/selftests/kselftest_harness/Makefile |    7 +
>  .../selftests/kselftest_harness/harness-selftest.c |  129 ++
>  .../kselftest_harness/harness-selftest.expected    |   62 +
>  .../kselftest_harness/harness-selftest.sh          |   13 +
>  tools/testing/selftests/nolibc/Makefile            |   13 +-
>  tools/testing/selftests/nolibc/harness-selftest.c  |    1 +
>  tools/testing/selftests/nolibc/nolibc-test.c       | 1729 +-------------------
>  tools/testing/selftests/nolibc/run-tests.sh        |    2 +-
>  19 files changed, 637 insertions(+), 1860 deletions(-)
> ---
> base-commit: 0af2f6be1b4281385b618cb86ad946eded089ac8
> change-id: 20250130-nolibc-kselftest-harness-8b2c8cac43bf
> 
> Best regards,
> -- 
> Thomas Weißschuh <thomas.weissschuh@linutronix.de>
>

Re: [PATCH v3 00/32] kselftest harness and nolibc compatibility

Posted by Willy Tarreau 9 months, 3 weeks ago

Hi Thomas,

On Tue, Apr 22, 2025 at 10:48:28AM +0200, Thomas Weißschuh wrote:
> Hi,
> 
> On Fri, Apr 11, 2025 at 11:00:24AM +0200, Thomas Weißschuh wrote:
> > Nolibc is useful for selftests as the test programs can be very small,
> > and compiled with just a kernel crosscompiler, without userspace support.
> > Currently nolibc is only usable with kselftest.h, not the more
> > convenient to use kselftest_harness.h
> > This series provides this compatibility by adding new features to nolibc
> > and removing the usage of problematic features from the harness.
> > 
> > The first half of the series are changes to the harness, the second one
> > are for nolibc. Both parts are very independent and should go through
> > different trees.
> 
> I need a few nolibc bits of this series (snprintf() and prep patches) to base
> further patches on. For that I'd like to pick up all the nolibc patches from
> this series through the nolibc tree. They got Acks from Willy.
> 
> Any objections?

No objection on my side!

Willy

Re: [PATCH v3 00/32] kselftest harness and nolibc compatibility

Posted by Shuah Khan 9 months, 3 weeks ago

On 4/22/25 02:51, Willy Tarreau wrote:
> Hi Thomas,
> 
> On Tue, Apr 22, 2025 at 10:48:28AM +0200, Thomas Weißschuh wrote:
>> Hi,
>>
>> On Fri, Apr 11, 2025 at 11:00:24AM +0200, Thomas Weißschuh wrote:
>>> Nolibc is useful for selftests as the test programs can be very small,
>>> and compiled with just a kernel crosscompiler, without userspace support.
>>> Currently nolibc is only usable with kselftest.h, not the more
>>> convenient to use kselftest_harness.h
>>> This series provides this compatibility by adding new features to nolibc
>>> and removing the usage of problematic features from the harness.
>>>
>>> The first half of the series are changes to the harness, the second one
>>> are for nolibc. Both parts are very independent and should go through
>>> different trees.
>>
>> I need a few nolibc bits of this series (snprintf() and prep patches) to base
>> further patches on. For that I'd like to pick up all the nolibc patches from
>> this series through the nolibc tree. They got Acks from Willy.
>>
>> Any objections?
> 
> No objection on my side!
> 

Thanks.

Kees, do you have any comments on this series? If you are okay
with it, I would like to apply this for next.

thanks,
-- Shuah

Re: [PATCH v3 00/32] kselftest harness and nolibc compatibility

Posted by Kees Cook 9 months, 2 weeks ago

On Thu, Apr 24, 2025 at 02:06:34PM -0600, Shuah Khan wrote:
> On 4/22/25 02:51, Willy Tarreau wrote:
> > Hi Thomas,
> > 
> > On Tue, Apr 22, 2025 at 10:48:28AM +0200, Thomas Weißschuh wrote:
> > > Hi,
> > > 
> > > On Fri, Apr 11, 2025 at 11:00:24AM +0200, Thomas Weißschuh wrote:
> > > > Nolibc is useful for selftests as the test programs can be very small,
> > > > and compiled with just a kernel crosscompiler, without userspace support.
> > > > Currently nolibc is only usable with kselftest.h, not the more
> > > > convenient to use kselftest_harness.h
> > > > This series provides this compatibility by adding new features to nolibc
> > > > and removing the usage of problematic features from the harness.
> > > > 
> > > > The first half of the series are changes to the harness, the second one
> > > > are for nolibc. Both parts are very independent and should go through
> > > > different trees.
> > > 
> > > I need a few nolibc bits of this series (snprintf() and prep patches) to base
> > > further patches on. For that I'd like to pick up all the nolibc patches from
> > > this series through the nolibc tree. They got Acks from Willy.
> > > 
> > > Any objections?
> > 
> > No objection on my side!
> > 
> 
> Thanks.
> 
> Kees, do you have any comments on this series? If you are okay
> with it, I would like to apply this for next.

Fine by me! :)

-- 
Kees Cook

Re: [PATCH v3 00/32] kselftest harness and nolibc compatibility

Posted by Shuah Khan 9 months, 2 weeks ago

On 4/24/25 14:06, Shuah Khan wrote:
> On 4/22/25 02:51, Willy Tarreau wrote:
>> Hi Thomas,
>>
>> On Tue, Apr 22, 2025 at 10:48:28AM +0200, Thomas Weißschuh wrote:
>>> Hi,
>>>
>>> On Fri, Apr 11, 2025 at 11:00:24AM +0200, Thomas Weißschuh wrote:
>>>> Nolibc is useful for selftests as the test programs can be very small,
>>>> and compiled with just a kernel crosscompiler, without userspace support.
>>>> Currently nolibc is only usable with kselftest.h, not the more
>>>> convenient to use kselftest_harness.h
>>>> This series provides this compatibility by adding new features to nolibc
>>>> and removing the usage of problematic features from the harness.
>>>>
>>>> The first half of the series are changes to the harness, the second one
>>>> are for nolibc. Both parts are very independent and should go through
>>>> different trees.
>>>
>>> I need a few nolibc bits of this series (snprintf() and prep patches) to base
>>> further patches on. For that I'd like to pick up all the nolibc patches from
>>> this series through the nolibc tree. They got Acks from Willy.
>>>
>>> Any objections?
>>
>> No objection on my side!
>>
> 
> Thanks.
> 
> Kees, do you have any comments on this series? If you are okay
> with it, I would like to apply this for next.
> 

Thomas,

Can this be part of no libc pull request for 6.16-rc1 which I will
be fielding this time around?

If so here is my Ack
Acked-by: Shuah Khan <skhan@linuxfoundation.org>

thanks,
-- Shuah

Re: [PATCH v3 00/32] kselftest harness and nolibc compatibility

Posted by Thomas Weißschuh 9 months, 2 weeks ago

On 2025-04-29 12:52:56-0600, Shuah Khan wrote:
> On 4/24/25 14:06, Shuah Khan wrote:
> > On 4/22/25 02:51, Willy Tarreau wrote:
> > > Hi Thomas,
> > > 
> > > On Tue, Apr 22, 2025 at 10:48:28AM +0200, Thomas Weißschuh wrote:
> > > > Hi,
> > > > 
> > > > On Fri, Apr 11, 2025 at 11:00:24AM +0200, Thomas Weißschuh wrote:
> > > > > Nolibc is useful for selftests as the test programs can be very small,
> > > > > and compiled with just a kernel crosscompiler, without userspace support.
> > > > > Currently nolibc is only usable with kselftest.h, not the more
> > > > > convenient to use kselftest_harness.h
> > > > > This series provides this compatibility by adding new features to nolibc
> > > > > and removing the usage of problematic features from the harness.
> > > > > 
> > > > > The first half of the series are changes to the harness, the second one
> > > > > are for nolibc. Both parts are very independent and should go through
> > > > > different trees.
> > > > 
> > > > I need a few nolibc bits of this series (snprintf() and prep patches) to base
> > > > further patches on. For that I'd like to pick up all the nolibc patches from
> > > > this series through the nolibc tree. They got Acks from Willy.
> > > > 
> > > > Any objections?
> > > 
> > > No objection on my side!
> > > 
> > 
> > Thanks.
> > 
> > Kees, do you have any comments on this series? If you are okay
> > with it, I would like to apply this for next.
> > 
> 
> Thomas,
> 
> Can this be part of no libc pull request for 6.16-rc1 which I will
> be fielding this time around?

Sure, I'll pick it up.

> If so here is my Ack
> Acked-by: Shuah Khan <skhan@linuxfoundation.org>

Thanks,
Thomas