[PATCH v3 04/16] kbuild: userprogs: add nolibc support

Thomas Weißschuh posted 16 patches 4 months ago
There is a newer version of this series
[PATCH v3 04/16] kbuild: userprogs: add nolibc support
Posted by Thomas Weißschuh 4 months ago
Userprogs are built with the regular kernel compiler $CC.
A kernel compiler does not necessarily contain a libc which is required
for a normal userspace application.
However the kernel tree does contain a minimal libc implementation
"nolibc" which can be used to build userspace applications.

Introduce support to build userprogs against nolibc instead of the
default libc of the compiler, which may not exist.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>

---
This could probably be moved out of the generic kbuild makefiles.
I think the ergonimics would suffer and this functionality could be
used by other users of userprogs.

Also this does currently not support out-of-tree builds.
For that tools/include/nolibc/*.h and usr/include/*.h would need to be
installed into the build directory.
---
 Documentation/kbuild/makefiles.rst | 13 +++++++++++++
 scripts/Makefile.userprogs         | 13 ++++++++++---
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst
index 8aef3650c1f32b6b197e0dc777e26775d371a081..4cc7a1b89f1803857a4723284613111e9ad71d92 100644
--- a/Documentation/kbuild/makefiles.rst
+++ b/Documentation/kbuild/makefiles.rst
@@ -974,6 +974,19 @@ When linking bpfilter_umh, it will be passed the extra option -static.
 
 From command line, :ref:`USERCFLAGS and USERLDFLAGS <userkbuildflags>` will also be used.
 
+Building userprogs against nolibc
+---------------------------------
+
+Not all kernel toolchains provide a libc.
+Simple userprogs can be built against a very simple libc call "nolibc" provided
+by the kernel source tree.
+This requires ``CONFIG_HEADERS_INSTALL=y``.
+
+Example::
+
+  # lib/kunit/Makefile
+  uapi-preinit-nolibc := $(CONFIG_ARCH_HAS_NOLIBC)
+
 When userspace programs are actually built
 ------------------------------------------
 
diff --git a/scripts/Makefile.userprogs b/scripts/Makefile.userprogs
index f3a7e1ef3753b54303718fae97f4b3c9d4eac07c..b1633a9de6c86a023c70a717bac0b80b89d01431 100644
--- a/scripts/Makefile.userprogs
+++ b/scripts/Makefile.userprogs
@@ -16,10 +16,17 @@ user-csingle	:= $(addprefix $(obj)/, $(user-csingle))
 user-cmulti	:= $(addprefix $(obj)/, $(user-cmulti))
 user-cobjs	:= $(addprefix $(obj)/, $(user-cobjs))
 
+user_nolibc_ccflags := -nostdlib -nostdinc -static -fno-ident -fno-asynchronous-unwind-tables \
+		      -ffreestanding -fno-stack-protector \
+		      -isystem $(objtree)/usr/include -include $(srctree)/tools/include/nolibc/nolibc.h -isystem $(srctree)/tools/include/nolibc/
+user_nolibc_ldflags := -nostdlib -nostdinc -static
+
 user_ccflags	= -Wp,-MMD,$(depfile) $(KBUILD_USERCFLAGS) $(userccflags) \
-			$($(target-stem)-userccflags)
-user_ldflags	= $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags)
-user_ldlibs	= $(userldlibs) $($(target-stem)-userldlibs)
+			$($(target-stem)-userccflags) $(if $($(target-stem)-nolibc),$(user_nolibc_ccflags))
+user_ldflags	= $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags) \
+			$(if $($(target-stem)-nolibc),$(user_nolibc_ldflags))
+user_ldlibs	= $(userldlibs) $($(target-stem)-userldlibs) \
+			$(if $($(target-stem)-nolibc),$(user_nolibc_ldlibs))
 
 # Create an executable from a single .c file
 quiet_cmd_user_cc_c = CC [U]  $@

-- 
2.49.0

Re: [PATCH v3 04/16] kbuild: userprogs: add nolibc support
Posted by Masahiro Yamada 3 months, 3 weeks ago
On Wed, Jun 11, 2025 at 4:38 PM Thomas Weißschuh
<thomas.weissschuh@linutronix.de> wrote:
>
> Userprogs are built with the regular kernel compiler $CC.
> A kernel compiler does not necessarily contain a libc which is required
> for a normal userspace application.
> However the kernel tree does contain a minimal libc implementation
> "nolibc" which can be used to build userspace applications.
>
> Introduce support to build userprogs against nolibc instead of the
> default libc of the compiler, which may not exist.
>
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
>
> ---
> This could probably be moved out of the generic kbuild makefiles.
> I think the ergonimics would suffer and this functionality could be
> used by other users of userprogs.
>
> Also this does currently not support out-of-tree builds.
> For that tools/include/nolibc/*.h and usr/include/*.h would need to be
> installed into the build directory.
> ---
>  Documentation/kbuild/makefiles.rst | 13 +++++++++++++
>  scripts/Makefile.userprogs         | 13 ++++++++++---
>  2 files changed, 23 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst
> index 8aef3650c1f32b6b197e0dc777e26775d371a081..4cc7a1b89f1803857a4723284613111e9ad71d92 100644
> --- a/Documentation/kbuild/makefiles.rst
> +++ b/Documentation/kbuild/makefiles.rst
> @@ -974,6 +974,19 @@ When linking bpfilter_umh, it will be passed the extra option -static.
>
>  From command line, :ref:`USERCFLAGS and USERLDFLAGS <userkbuildflags>` will also be used.
>
> +Building userprogs against nolibc
> +---------------------------------
> +
> +Not all kernel toolchains provide a libc.
> +Simple userprogs can be built against a very simple libc call "nolibc" provided
> +by the kernel source tree.
> +This requires ``CONFIG_HEADERS_INSTALL=y``.
> +
> +Example::
> +
> +  # lib/kunit/Makefile
> +  uapi-preinit-nolibc := $(CONFIG_ARCH_HAS_NOLIBC)
> +
>  When userspace programs are actually built
>  ------------------------------------------
>
> diff --git a/scripts/Makefile.userprogs b/scripts/Makefile.userprogs
> index f3a7e1ef3753b54303718fae97f4b3c9d4eac07c..b1633a9de6c86a023c70a717bac0b80b89d01431 100644
> --- a/scripts/Makefile.userprogs
> +++ b/scripts/Makefile.userprogs
> @@ -16,10 +16,17 @@ user-csingle        := $(addprefix $(obj)/, $(user-csingle))
>  user-cmulti    := $(addprefix $(obj)/, $(user-cmulti))
>  user-cobjs     := $(addprefix $(obj)/, $(user-cobjs))
>
> +user_nolibc_ccflags := -nostdlib -nostdinc -static -fno-ident -fno-asynchronous-unwind-tables \
> +                     -ffreestanding -fno-stack-protector \
> +                     -isystem $(objtree)/usr/include -include $(srctree)/tools/include/nolibc/nolibc.h -isystem $(srctree)/tools/include/nolibc/

The tools/ directory is a different world, and Kbuild scripts do not know
anything about it.

And, you do not need to implement this in scripts/Makefile.userprogs
because you can move this to lib/kunit/Makefile.kunit-uapi or somewhere.









> +user_nolibc_ldflags := -nostdlib -nostdinc -static
> +
>  user_ccflags   = -Wp,-MMD,$(depfile) $(KBUILD_USERCFLAGS) $(userccflags) \
> -                       $($(target-stem)-userccflags)
> -user_ldflags   = $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags)
> -user_ldlibs    = $(userldlibs) $($(target-stem)-userldlibs)
> +                       $($(target-stem)-userccflags) $(if $($(target-stem)-nolibc),$(user_nolibc_ccflags))
> +user_ldflags   = $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags) \
> +                       $(if $($(target-stem)-nolibc),$(user_nolibc_ldflags))
> +user_ldlibs    = $(userldlibs) $($(target-stem)-userldlibs) \
> +                       $(if $($(target-stem)-nolibc),$(user_nolibc_ldlibs))
>
>  # Create an executable from a single .c file
>  quiet_cmd_user_cc_c = CC [U]  $@
>
> --
> 2.49.0
>
>


--
Best Regards
Masahiro Yamada
Re: [PATCH v3 04/16] kbuild: userprogs: add nolibc support
Posted by Thomas Weißschuh 3 months, 3 weeks ago
On Tue, Jun 17, 2025 at 12:35:07AM +0900, Masahiro Yamada wrote:
> On Wed, Jun 11, 2025 at 4:38 PM Thomas Weißschuh
> <thomas.weissschuh@linutronix.de> wrote:
> >
> > Userprogs are built with the regular kernel compiler $CC.
> > A kernel compiler does not necessarily contain a libc which is required
> > for a normal userspace application.
> > However the kernel tree does contain a minimal libc implementation
> > "nolibc" which can be used to build userspace applications.
> >
> > Introduce support to build userprogs against nolibc instead of the
> > default libc of the compiler, which may not exist.
> >
> > Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> >
> > ---
> > This could probably be moved out of the generic kbuild makefiles.
> > I think the ergonimics would suffer and this functionality could be
> > used by other users of userprogs.
> >
> > Also this does currently not support out-of-tree builds.
> > For that tools/include/nolibc/*.h and usr/include/*.h would need to be
> > installed into the build directory.

<snip>

> > --- a/scripts/Makefile.userprogs
> > +++ b/scripts/Makefile.userprogs
> > @@ -16,10 +16,17 @@ user-csingle        := $(addprefix $(obj)/, $(user-csingle))
> >  user-cmulti    := $(addprefix $(obj)/, $(user-cmulti))
> >  user-cobjs     := $(addprefix $(obj)/, $(user-cobjs))
> >
> > +user_nolibc_ccflags := -nostdlib -nostdinc -static -fno-ident -fno-asynchronous-unwind-tables \
> > +                     -ffreestanding -fno-stack-protector \
> > +                     -isystem $(objtree)/usr/include -include $(srctree)/tools/include/nolibc/nolibc.h -isystem $(srctree)/tools/include/nolibc/
> 
> The tools/ directory is a different world, and Kbuild scripts do not know
> anything about it.

Ack.

How does this statement affect the next patch which creates
tools/include/nolibc/Kconfig.nolibc ?
Is it fine to create the Kconfig file in tools/ or should I move it?
I do want to maintain this file as part of nolibc and not KUnit.
The possibilities I see are init/Kconfig.nolibc or lib/Kconfig.nolibc.

> And, you do not need to implement this in scripts/Makefile.userprogs
> because you can move this to lib/kunit/Makefile.kunit-uapi or somewhere.

Understood. This is not unexpected, as hinted in the commit message.

> > +user_nolibc_ldflags := -nostdlib -nostdinc -static
> > +
> >  user_ccflags   = -Wp,-MMD,$(depfile) $(KBUILD_USERCFLAGS) $(userccflags) \
> > -                       $($(target-stem)-userccflags)
> > -user_ldflags   = $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags)
> > -user_ldlibs    = $(userldlibs) $($(target-stem)-userldlibs)
> > +                       $($(target-stem)-userccflags) $(if $($(target-stem)-nolibc),$(user_nolibc_ccflags))
> > +user_ldflags   = $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags) \
> > +                       $(if $($(target-stem)-nolibc),$(user_nolibc_ldflags))
> > +user_ldlibs    = $(userldlibs) $($(target-stem)-userldlibs) \
> > +                       $(if $($(target-stem)-nolibc),$(user_nolibc_ldlibs))
> >
> >  # Create an executable from a single .c file
> >  quiet_cmd_user_cc_c = CC [U]  $@


Thomas
Re: [PATCH v3 04/16] kbuild: userprogs: add nolibc support
Posted by Masahiro Yamada 3 months, 3 weeks ago
On Tue, Jun 17, 2025 at 4:59 PM Thomas Weißschuh
<thomas.weissschuh@linutronix.de> wrote:
>
> On Tue, Jun 17, 2025 at 12:35:07AM +0900, Masahiro Yamada wrote:
> > On Wed, Jun 11, 2025 at 4:38 PM Thomas Weißschuh
> > <thomas.weissschuh@linutronix.de> wrote:
> > >
> > > Userprogs are built with the regular kernel compiler $CC.
> > > A kernel compiler does not necessarily contain a libc which is required
> > > for a normal userspace application.
> > > However the kernel tree does contain a minimal libc implementation
> > > "nolibc" which can be used to build userspace applications.
> > >
> > > Introduce support to build userprogs against nolibc instead of the
> > > default libc of the compiler, which may not exist.
> > >
> > > Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> > >
> > > ---
> > > This could probably be moved out of the generic kbuild makefiles.
> > > I think the ergonimics would suffer and this functionality could be
> > > used by other users of userprogs.
> > >
> > > Also this does currently not support out-of-tree builds.
> > > For that tools/include/nolibc/*.h and usr/include/*.h would need to be
> > > installed into the build directory.
>
> <snip>
>
> > > --- a/scripts/Makefile.userprogs
> > > +++ b/scripts/Makefile.userprogs
> > > @@ -16,10 +16,17 @@ user-csingle        := $(addprefix $(obj)/, $(user-csingle))
> > >  user-cmulti    := $(addprefix $(obj)/, $(user-cmulti))
> > >  user-cobjs     := $(addprefix $(obj)/, $(user-cobjs))
> > >
> > > +user_nolibc_ccflags := -nostdlib -nostdinc -static -fno-ident -fno-asynchronous-unwind-tables \
> > > +                     -ffreestanding -fno-stack-protector \
> > > +                     -isystem $(objtree)/usr/include -include $(srctree)/tools/include/nolibc/nolibc.h -isystem $(srctree)/tools/include/nolibc/
> >
> > The tools/ directory is a different world, and Kbuild scripts do not know
> > anything about it.
>
> Ack.
>
> How does this statement affect the next patch which creates
> tools/include/nolibc/Kconfig.nolibc ?
> Is it fine to create the Kconfig file in tools/ or should I move it?
> I do want to maintain this file as part of nolibc and not KUnit.
> The possibilities I see are init/Kconfig.nolibc or lib/Kconfig.nolibc.

Personally I like init/Kconfig.nolibc.
(I am even fine with adding this new entry to init/Kconfig)
But, this file is not what I maintain. It is up to you after all.



> > And, you do not need to implement this in scripts/Makefile.userprogs
> > because you can move this to lib/kunit/Makefile.kunit-uapi or somewhere.
>
> Understood. This is not unexpected, as hinted in the commit message.

One existing example is lib/vdso/Makefile.include, which is included
from multiple Makefiles.





> > > +user_nolibc_ldflags := -nostdlib -nostdinc -static
> > > +
> > >  user_ccflags   = -Wp,-MMD,$(depfile) $(KBUILD_USERCFLAGS) $(userccflags) \
> > > -                       $($(target-stem)-userccflags)
> > > -user_ldflags   = $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags)
> > > -user_ldlibs    = $(userldlibs) $($(target-stem)-userldlibs)
> > > +                       $($(target-stem)-userccflags) $(if $($(target-stem)-nolibc),$(user_nolibc_ccflags))
> > > +user_ldflags   = $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags) \
> > > +                       $(if $($(target-stem)-nolibc),$(user_nolibc_ldflags))
> > > +user_ldlibs    = $(userldlibs) $($(target-stem)-userldlibs) \
> > > +                       $(if $($(target-stem)-nolibc),$(user_nolibc_ldlibs))
> > >
> > >  # Create an executable from a single .c file
> > >  quiet_cmd_user_cc_c = CC [U]  $@
>
>
> Thomas


--
Best Regards
Masahiro Yamada
Re: [PATCH v3 04/16] kbuild: userprogs: add nolibc support
Posted by Nicolas Schier 4 months ago
On Wed, Jun 11, 2025 at 09:38:10AM +0200, Thomas Weißschuh wrote:
> Userprogs are built with the regular kernel compiler $CC.
> A kernel compiler does not necessarily contain a libc which is required
> for a normal userspace application.
> However the kernel tree does contain a minimal libc implementation
> "nolibc" which can be used to build userspace applications.
> 
> Introduce support to build userprogs against nolibc instead of the
> default libc of the compiler, which may not exist.
> 
> Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> 
> ---
> This could probably be moved out of the generic kbuild makefiles.
> I think the ergonimics would suffer and this functionality could be
> used by other users of userprogs.
> 
> Also this does currently not support out-of-tree builds.
> For that tools/include/nolibc/*.h and usr/include/*.h would need to be
> installed into the build directory.

Thanks!

Reviewed-by: Nicolas Schier <n.schier@avm.de>

Probably overkill, but might it make sense to abort *-nolibc compilation 
requests for out-of-tree builds?

Kind regards,
Nicolas