The nolibc tests are not real kselftests, they work differently and
provide a different interface. Users trying to use them like real
selftests may be confused and the tests are not executed by CI systems.

To make space for an integration with the kselftest framework, move the
custom tests out of the way.
The custom tests are still useful to keep as they provide functionality
not provided by kselftests.

Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 tools/testing/selftests/nolibc/{Makefile => Makefile.nolibc} | 0
 tools/testing/selftests/nolibc/run-tests.sh                  | 2 +-
 2 files changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/nolibc/Makefile b/tools/testing/selftests/nolibc/Makefile.nolibc
similarity index 100%
rename from tools/testing/selftests/nolibc/Makefile
rename to tools/testing/selftests/nolibc/Makefile.nolibc
diff --git a/tools/testing/selftests/nolibc/run-tests.sh b/tools/testing/selftests/nolibc/run-tests.sh
index 8277599e6441a933d9c1ec5003acf49b06df226f..279fbd93ef70497868689b7f1e14ddc6c5c1a15f 100755
--- a/tools/testing/selftests/nolibc/run-tests.sh
+++ b/tools/testing/selftests/nolibc/run-tests.sh
@@ -169,7 +169,7 @@ test_arch() {
 	if [ "$werror" -ne 0 ]; then
 		CFLAGS_EXTRA="$CFLAGS_EXTRA -Werror"
 	fi
-	MAKE=(make -j"${nproc}" XARCH="${arch}" CROSS_COMPILE="${cross_compile}" LLVM="${llvm}" O="${build_dir}")
+	MAKE=(make -f Makefile.nolibc -j"${nproc}" XARCH="${arch}" CROSS_COMPILE="${cross_compile}" LLVM="${llvm}" O="${build_dir}")
 
 	case "$test_mode" in
 		'system')

-- 
2.50.0

Hi Thomas,

On Fri, Jun 20, 2025 at 11:39:32PM +0200, Thomas Weißschuh wrote:
> The nolibc tests are not real kselftests, they work differently and
> provide a different interface. Users trying to use them like real
> selftests may be confused and the tests are not executed by CI systems.
> 
> To make space for an integration with the kselftest framework, move the
> custom tests out of the way.
> The custom tests are still useful to keep as they provide functionality
> not provided by kselftests.

I'm wondering, what prevents us from merging the new rules into the
current makefile instead of renaming it, especially considering the
fact that we initially took care of not confiscating the "all" target ?
I'm asking because: 

  $ make -f Makefile.nolibc help

is clearly less convenient and intuitive than:

  $ make help

Regards,
Willy

On 2025-06-21 06:14:21+0200, Willy Tarreau wrote:
> Hi Thomas,
> 
> On Fri, Jun 20, 2025 at 11:39:32PM +0200, Thomas Weißschuh wrote:
> > The nolibc tests are not real kselftests, they work differently and
> > provide a different interface. Users trying to use them like real
> > selftests may be confused and the tests are not executed by CI systems.
> > 
> > To make space for an integration with the kselftest framework, move the
> > custom tests out of the way.
> > The custom tests are still useful to keep as they provide functionality
> > not provided by kselftests.
> 
> I'm wondering, what prevents us from merging the new rules into the
> current makefile instead of renaming it, especially considering the
> fact that we initially took care of not confiscating the "all" target ?

We'll have conflicts around CFLAGS, the nolibc-test target and probably
other things. It will also make everything harder to understand and may
break unexpectedly in the future.

> I'm asking because: 
> 
>   $ make -f Makefile.nolibc help
> 
> is clearly less convenient and intuitive than:
> 
>   $ make help

Is your issue specifically with the help target?
We should be able to show the help message from the main Makefile with a
hint to the Makefile.nolibc.

Another, more general, possibility would be to move the special Makefile
to tools/testing/nolibc/ and keep only the selftest parts in
tools/testing/selftests/nolibc/.

On Sat, Jun 21, 2025 at 10:34:38AM +0200, Thomas Weißschuh wrote:
> On 2025-06-21 06:14:21+0200, Willy Tarreau wrote:
> > Hi Thomas,
> > 
> > On Fri, Jun 20, 2025 at 11:39:32PM +0200, Thomas Weißschuh wrote:
> > > The nolibc tests are not real kselftests, they work differently and
> > > provide a different interface. Users trying to use them like real
> > > selftests may be confused and the tests are not executed by CI systems.
> > > 
> > > To make space for an integration with the kselftest framework, move the
> > > custom tests out of the way.
> > > The custom tests are still useful to keep as they provide functionality
> > > not provided by kselftests.
> > 
> > I'm wondering, what prevents us from merging the new rules into the
> > current makefile instead of renaming it, especially considering the
> > fact that we initially took care of not confiscating the "all" target ?
> 
> We'll have conflicts around CFLAGS, the nolibc-test target and probably
> other things.

OK I understand.

> It will also make everything harder to understand and may
> break unexpectedly in the future.
> 
> > I'm asking because: 
> > 
> >   $ make -f Makefile.nolibc help
> > 
> > is clearly less convenient and intuitive than:
> > 
> >   $ make help
> 
> Is your issue specifically with the help target?

Not just but that's an entry point. Admittedly it's not a big problem,
I was merely asking if there was a real reason for splitting them apart
or if it was just to keep the stuff clean.

> We should be able to show the help message from the main Makefile with a
> hint to the Makefile.nolibc.

I thought about it as well, we could have a help target in the main
makefile that just emits "Please run make -f Makefile.nolibc with the
following targets:", and then runs "make -f Makefile.nolibc help".

> Another, more general, possibility would be to move the special Makefile
> to tools/testing/nolibc/ and keep only the selftest parts in
> tools/testing/selftests/nolibc/.

I hadn't thought about this, but that could indeed make sense. Let's see
later how it goes and let's not add burden about this for now. Please just
keep your patch as-is.

Thanks,
Willy

On 2025-06-21 10:47:39+0200, Willy Tarreau wrote:
> On Sat, Jun 21, 2025 at 10:34:38AM +0200, Thomas Weißschuh wrote:
> > On 2025-06-21 06:14:21+0200, Willy Tarreau wrote:
> > > Hi Thomas,
> > > 
> > > On Fri, Jun 20, 2025 at 11:39:32PM +0200, Thomas Weißschuh wrote:
> > > > The nolibc tests are not real kselftests, they work differently and
> > > > provide a different interface. Users trying to use them like real
> > > > selftests may be confused and the tests are not executed by CI systems.
> > > > 
> > > > To make space for an integration with the kselftest framework, move the
> > > > custom tests out of the way.
> > > > The custom tests are still useful to keep as they provide functionality
> > > > not provided by kselftests.
> > > 
> > > I'm wondering, what prevents us from merging the new rules into the
> > > current makefile instead of renaming it, especially considering the
> > > fact that we initially took care of not confiscating the "all" target ?
> > 
> > We'll have conflicts around CFLAGS, the nolibc-test target and probably
> > other things.
> 
> OK I understand.
> 
> > It will also make everything harder to understand and may
> > break unexpectedly in the future.
> > 
> > > I'm asking because: 
> > > 
> > >   $ make -f Makefile.nolibc help
> > > 
> > > is clearly less convenient and intuitive than:
> > > 
> > >   $ make help
> > 
> > Is your issue specifically with the help target?
> 
> Not just but that's an entry point. Admittedly it's not a big problem,
> I was merely asking if there was a real reason for splitting them apart
> or if it was just to keep the stuff clean.
> 
> > We should be able to show the help message from the main Makefile with a
> > hint to the Makefile.nolibc.
> 
> I thought about it as well, we could have a help target in the main
> makefile that just emits "Please run make -f Makefile.nolibc with the
> following targets:", and then runs "make -f Makefile.nolibc help".

I'll do that.

> > Another, more general, possibility would be to move the special Makefile
> > to tools/testing/nolibc/ and keep only the selftest parts in
> > tools/testing/selftests/nolibc/.
> 
> I hadn't thought about this, but that could indeed make sense. Let's see
> later how it goes and let's not add burden about this for now. Please just
> keep your patch as-is.

Sounds good. Could you give a formal Ack for the patch/series?


Thomas

Hi Thomas,

On Mon, Jun 23, 2025 at 10:18:19PM +0200, Thomas Weißschuh wrote:
(...)
> > I hadn't thought about this, but that could indeed make sense. Let's see
> > later how it goes and let's not add burden about this for now. Please just
> > keep your patch as-is.
> 
> Sounds good. Could you give a formal Ack for the patch/series?

Sure, as usual, for the relevant parts of the series:

Acked-by: Willy Tarreau <w@1wt.eu>

Cheers,
Willy