1. Patchew
  2. linux
  3. View series

[PATCH v3] kheaders: make it possible to override TAR

Sam James posted 1 patch 2 months, 2 weeks ago
There is a newer version of this series
Makefile               | 3 ++-
kernel/gen_kheaders.sh | 6 +++---
2 files changed, 5 insertions(+), 4 deletions(-)
[PATCH v3] kheaders: make it possible to override TAR
Posted by Sam James 2 months, 2 weeks ago 
From: Michał Górny <mgorny@gentoo.org>

Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
archive reproducible") introduced a number of options specific to GNU
tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
the script to fail to work on systems where `tar` is not GNU tar.  This
can occur e.g. on recent Gentoo Linux installations that support using
bsdtar from libarchive instead.

Add a `TAR` make variable to make it possible to override the tar
executable used, e.g. by specifying:

  make TAR=gtar

Link: https://bugs.gentoo.org/884061
Reported-by: Sam James <sam@gentoo.org>
Tested-by: Sam James <sam@gentoo.org>
Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
Signed-off-by: Sam James <sam@gentoo.org>
---
v3: Rebase, cover more tar instances.

 Makefile               | 3 ++-
 kernel/gen_kheaders.sh | 6 +++---
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index c09766beb7eff..22d6037d738fe 100644
--- a/Makefile
+++ b/Makefile
@@ -543,6 +543,7 @@ LZMA		= lzma
 LZ4		= lz4
 XZ		= xz
 ZSTD		= zstd
+TAR		= tar
 
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF)
@@ -622,7 +623,7 @@ export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
 export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
 export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
 export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
-export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
+export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD TAR
 export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS KBUILD_PROCMACROLDFLAGS LDFLAGS_MODULE
 export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS
 
diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
index c9e5dc068e854..bb609a9ed72b4 100755
--- a/kernel/gen_kheaders.sh
+++ b/kernel/gen_kheaders.sh
@@ -66,13 +66,13 @@ if [ "$building_out_of_srctree" ]; then
 		cd $srctree
 		for f in $dir_list
 			do find "$f" -name "*.h";
-		done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
+		done | ${TAR:-tar} -c -f - -T - | ${TAR:-tar} -xf - -C "${tmpdir}"
 	)
 fi
 
 for f in $dir_list;
 	do find "$f" -name "*.h";
-done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
+done | ${TAR:-tar} -c -f - -T - | ${TAR:-tar} -xf - -C "${tmpdir}"
 
 # Always exclude include/generated/utsversion.h
 # Otherwise, the contents of the tarball may vary depending on the build steps.
@@ -88,7 +88,7 @@ xargs -0 -P8 -n1 \
 rm -f "${tmpdir}.contents.txt"
 
 # Create archive and try to normalize metadata for reproducibility.
-tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
+${TAR:-tar} "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
     --owner=0 --group=0 --sort=name --numeric-owner --mode=u=rw,go=r,a+X \
     -I $XZ -cf $tarfile -C "${tmpdir}/" . > /dev/null
 
-- 
2.50.1
Re: [PATCH v3] kheaders: make it possible to override TAR
Posted by Nathan Chancellor 2 months, 2 weeks ago 
On Sat, Jul 19, 2025 at 04:24:05PM +0100, Sam James wrote:
> From: Michał Górny <mgorny@gentoo.org>
> 
> Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
> archive reproducible") introduced a number of options specific to GNU
> tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
> the script to fail to work on systems where `tar` is not GNU tar.  This
> can occur e.g. on recent Gentoo Linux installations that support using
> bsdtar from libarchive instead.
> 
> Add a `TAR` make variable to make it possible to override the tar
> executable used, e.g. by specifying:
> 
>   make TAR=gtar
> 
> Link: https://bugs.gentoo.org/884061
> Reported-by: Sam James <sam@gentoo.org>
> Tested-by: Sam James <sam@gentoo.org>
> Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Michał Górny <mgorny@gentoo.org>
> Signed-off-by: Sam James <sam@gentoo.org>
> ---

Reviewed-by: Nathan Chancellor <nathan@kernel.org>

I assume that other places that call tar within the build process are
not problematic because they do not use GNU specific options, such as
scripts/Makefile.package and scripts/package/install-extmod-build, or
maybe that people just have not tried building those packages with
bsdtar?

> v3: Rebase, cover more tar instances.
> 
>  Makefile               | 3 ++-
>  kernel/gen_kheaders.sh | 6 +++---
>  2 files changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index c09766beb7eff..22d6037d738fe 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -543,6 +543,7 @@ LZMA		= lzma
>  LZ4		= lz4
>  XZ		= xz
>  ZSTD		= zstd
> +TAR		= tar
>  
>  CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
>  		  -Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF)
> @@ -622,7 +623,7 @@ export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
>  export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
>  export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
>  export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> -export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
> +export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD TAR
>  export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS KBUILD_PROCMACROLDFLAGS LDFLAGS_MODULE
>  export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS
>  
> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> index c9e5dc068e854..bb609a9ed72b4 100755
> --- a/kernel/gen_kheaders.sh
> +++ b/kernel/gen_kheaders.sh
> @@ -66,13 +66,13 @@ if [ "$building_out_of_srctree" ]; then
>  		cd $srctree
>  		for f in $dir_list
>  			do find "$f" -name "*.h";
> -		done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
> +		done | ${TAR:-tar} -c -f - -T - | ${TAR:-tar} -xf - -C "${tmpdir}"
>  	)
>  fi
>  
>  for f in $dir_list;
>  	do find "$f" -name "*.h";
> -done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
> +done | ${TAR:-tar} -c -f - -T - | ${TAR:-tar} -xf - -C "${tmpdir}"
>  
>  # Always exclude include/generated/utsversion.h
>  # Otherwise, the contents of the tarball may vary depending on the build steps.
> @@ -88,7 +88,7 @@ xargs -0 -P8 -n1 \
>  rm -f "${tmpdir}.contents.txt"
>  
>  # Create archive and try to normalize metadata for reproducibility.
> -tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
> +${TAR:-tar} "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
>      --owner=0 --group=0 --sort=name --numeric-owner --mode=u=rw,go=r,a+X \
>      -I $XZ -cf $tarfile -C "${tmpdir}/" . > /dev/null
>  
> -- 
> 2.50.1
>
Re: [PATCH v3] kheaders: make it possible to override TAR
Posted by Michał Górny 2 months, 2 weeks ago 
On Sat, 2025-07-19 at 16:10 -0400, Nathan Chancellor wrote:
> On Sat, Jul 19, 2025 at 04:24:05PM +0100, Sam James wrote:
> > From: Michał Górny <mgorny@gentoo.org>
> > 
> > Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
> > archive reproducible") introduced a number of options specific to GNU
> > tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
> > the script to fail to work on systems where `tar` is not GNU tar.  This
> > can occur e.g. on recent Gentoo Linux installations that support using
> > bsdtar from libarchive instead.
> > 
> > Add a `TAR` make variable to make it possible to override the tar
> > executable used, e.g. by specifying:
> > 
> >   make TAR=gtar
> > 
> > Link: https://bugs.gentoo.org/884061
> > Reported-by: Sam James <sam@gentoo.org>
> > Tested-by: Sam James <sam@gentoo.org>
> > Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
> > Signed-off-by: Michał Górny <mgorny@gentoo.org>
> > Signed-off-by: Sam James <sam@gentoo.org>
> > ---
> 
> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> 
> I assume that other places that call tar within the build process are
> not problematic because they do not use GNU specific options, such as
> scripts/Makefile.package and scripts/package/install-extmod-build, or
> maybe that people just have not tried building those packages with
> bsdtar?

Precisely.  We focused on the one place which actually breaks our build,
to avoid touching too many subsystems simultaneously.  If this is
desirable, I can look into replacing the other instances.


-- 
Best regards,
Michał Górny
Re: [PATCH v3] kheaders: make it possible to override TAR
Posted by Masahiro Yamada 2 months, 1 week ago 
On Mon, Jul 21, 2025 at 4:08 AM Michał Górny <mgorny@gentoo.org> wrote:
>
> On Sat, 2025-07-19 at 16:10 -0400, Nathan Chancellor wrote:
> > On Sat, Jul 19, 2025 at 04:24:05PM +0100, Sam James wrote:
> > > From: Michał Górny <mgorny@gentoo.org>
> > >
> > > Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
> > > archive reproducible") introduced a number of options specific to GNU
> > > tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
> > > the script to fail to work on systems where `tar` is not GNU tar.  This
> > > can occur e.g. on recent Gentoo Linux installations that support using
> > > bsdtar from libarchive instead.
> > >
> > > Add a `TAR` make variable to make it possible to override the tar
> > > executable used, e.g. by specifying:
> > >
> > >   make TAR=gtar
> > >
> > > Link: https://bugs.gentoo.org/884061
> > > Reported-by: Sam James <sam@gentoo.org>
> > > Tested-by: Sam James <sam@gentoo.org>
> > > Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
> > > Signed-off-by: Michał Górny <mgorny@gentoo.org>
> > > Signed-off-by: Sam James <sam@gentoo.org>
> > > ---
> >
> > Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> >
> > I assume that other places that call tar within the build process are
> > not problematic because they do not use GNU specific options, such as
> > scripts/Makefile.package and scripts/package/install-extmod-build, or
> > maybe that people just have not tried building those packages with
> > bsdtar?
>
> Precisely.  We focused on the one place which actually breaks our build,
> to avoid touching too many subsystems simultaneously.  If this is
> desirable, I can look into replacing the other instances.
>

This patch is not applicable to my tree.

Please rebase to the kbuild tree (or linux-next).




-- 
Best Regards
Masahiro Yamada
Re: [PATCH v3] kheaders: make it possible to override TAR
Posted by Nicolas Schier 2 months, 2 weeks ago 
On Sun, Jul 20, 2025 at 09:08:20PM +0200 Michał Górny wrote:
> On Sat, 2025-07-19 at 16:10 -0400, Nathan Chancellor wrote:
[...]
> > I assume that other places that call tar within the build process are
> > not problematic because they do not use GNU specific options, such as
> > scripts/Makefile.package and scripts/package/install-extmod-build, or
> > maybe that people just have not tried building those packages with
> > bsdtar?
> 
> Precisely.  We focused on the one place which actually breaks our build,
> to avoid touching too many subsystems simultaneously.  If this is
> desirable, I can look into replacing the other instances.

Without further investigation, I think it makes sense to use the same
tar-instance for all tar calls.  Thus, I'd appreciate if you could replace
those as well.

Kind regards,
Nicolas

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.