kheaders: prefer gtar over tar for better compatibility

[PATCH] kheaders: prefer gtar over tar for better compatibility

Posted by Michał Górny 2 years, 9 months ago

Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
archive reproducible") introduced a number of options specific to GNU
tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
the script to fail to work on systems where `tar` is not GNU tar.  This
can occur e.g. on recent Gentoo Linux installations that support using
bsdtar from libarchive instead.

To achieve better portability, try using `gtar` over `tar` if the former
is available.  This is the name frequently used on systems featuring
support for installing GNU tar alongside another tar implementation.
If `gtar` is not present, `tar` is used for compatibility with regular
systems.

Link: https://bugs.gentoo.org/884061
Reported-by: Sam James <sam@gentoo.org>
Tested-by: Sam James <sam@gentoo.org>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 kernel/gen_kheaders.sh | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
index 473036b43..d2445af7f 100755
--- a/kernel/gen_kheaders.sh
+++ b/kernel/gen_kheaders.sh
@@ -80,11 +80,20 @@ done | cpio --quiet -pdu $cpio_dir >/dev/null 2>&1
 find $cpio_dir -type f -print0 |
 	xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;'
 
+# The following tar invocations use options specific to GNU tar. On some
+# systems (e.g. Gentoo), `tar` can be a different tool (e.g. bsdtar), and GNU
+# tar can be found as `gtar`.
+if [ -x "$(command -v gtar)" ]; then
+	tar=gtar
+else
+	tar=tar
+fi
+
 # Create archive and try to normalize metadata for reproducibility.
 # For compatibility with older versions of tar, files are fed to tar
 # pre-sorted, as --sort=name might not be available.
 find $cpio_dir -printf "./%P\n" | LC_ALL=C sort | \
-    tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
+    $tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
     --owner=0 --group=0 --numeric-owner --no-recursion \
     -I $XZ -cf $tarfile -C $cpio_dir/ -T - > /dev/null
 
-- 
2.39.0

Re: [PATCH] kheaders: prefer gtar over tar for better compatibility

Posted by Masahiro Yamada 2 years, 8 months ago

On Sat, Dec 17, 2022 at 5:42 PM Michał Górny <mgorny@gentoo.org> wrote:
>
> Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
> archive reproducible") introduced a number of options specific to GNU
> tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
> the script to fail to work on systems where `tar` is not GNU tar.  This
> can occur e.g. on recent Gentoo Linux installations that support using
> bsdtar from libarchive instead.
>
> To achieve better portability, try using `gtar` over `tar` if the former
> is available.  This is the name frequently used on systems featuring
> support for installing GNU tar alongside another tar implementation.
> If `gtar` is not present, `tar` is used for compatibility with regular
> systems.
>
> Link: https://bugs.gentoo.org/884061
> Reported-by: Sam James <sam@gentoo.org>
> Tested-by: Sam James <sam@gentoo.org>
> Signed-off-by: Michał Górny <mgorny@gentoo.org>
> ---
>  kernel/gen_kheaders.sh | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> index 473036b43..d2445af7f 100755
> --- a/kernel/gen_kheaders.sh
> +++ b/kernel/gen_kheaders.sh
> @@ -80,11 +80,20 @@ done | cpio --quiet -pdu $cpio_dir >/dev/null 2>&1
>  find $cpio_dir -type f -print0 |
>         xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;'
>
> +# The following tar invocations use options specific to GNU tar. On some
> +# systems (e.g. Gentoo), `tar` can be a different tool (e.g. bsdtar), and GNU
> +# tar can be found as `gtar`.
> +if [ -x "$(command -v gtar)" ]; then
> +       tar=gtar
> +else
> +       tar=tar
> +fi
> +
>  # Create archive and try to normalize metadata for reproducibility.
>  # For compatibility with older versions of tar, files are fed to tar
>  # pre-sorted, as --sort=name might not be available.
>  find $cpio_dir -printf "./%P\n" | LC_ALL=C sort | \
> -    tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
> +    $tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
>      --owner=0 --group=0 --numeric-owner --no-recursion \
>      -I $XZ -cf $tarfile -C $cpio_dir/ -T - > /dev/null



I checked the options in bsd tar [1].

Presumably, there is no way to make it work for both
due to incompatible options (--owner vs --uid, --group vs --gid).


Instead of inserting a workaround like this,
another way is to allow users to override a variable
from the command line.


See the top Makefile, for example,


AWK             = awk


Then, users can do "make AWK=gawk"





[1] https://www.freebsd.org/cgi/man.cgi?tar(1)






> --
> 2.39.0
>

Re: [PATCH] kheaders: prefer gtar over tar for better compatibility

Posted by Michał Górny 2 years, 8 months ago

On Sun, 2022-12-25 at 01:45 +0900, Masahiro Yamada wrote:
> Instead of inserting a workaround like this,
> another way is to allow users to override a variable
> from the command line.
> 
> 
> See the top Makefile, for example,
> 
> 
> AWK             = awk
> 
> 
> Then, users can do "make AWK=gawk"

I'm sorry but are you requesting that I remove the check and use $TAR
instead, or allow overriding with TAR, and fall back to gtar or tar
respectively?  If the former, should the script unconditionally assume
that TAR will be always set in the environment, or include fallback to
tar for when the script is run directly?

> 
-- 
Best regards,
Michał Górny

Re: [PATCH] kheaders: prefer gtar over tar for better compatibility

Posted by Sam James 2 years, 8 months ago


> On 25 Dec 2022, at 16:33, Michał Górny <mgorny@gentoo.org> wrote:
> 
> On Sun, 2022-12-25 at 01:45 +0900, Masahiro Yamada wrote:
>> Instead of inserting a workaround like this,
>> another way is to allow users to override a variable
>> from the command line.
>> 
>> 
>> See the top Makefile, for example,
>> 
>> 
>> AWK             = awk
>> 
>> 
>> Then, users can do "make AWK=gawk"
> 
> I'm sorry but are you requesting that I remove the check and use $TAR
> instead, or allow overriding with TAR, and fall back to gtar or tar
> respectively?  If the former, should the script unconditionally assume
> that TAR will be always set in the environment, or include fallback to
> tar for when the script is run directly?


Masahiro, what do you reckon? Thanks.

Best,
sam

Re: [PATCH] kheaders: prefer gtar over tar for better compatibility

Posted by Sam James 2 years, 7 months ago


> On 17 Jan 2023, at 19:01, Sam James <sam@gentoo.org> wrote:
> 
> 
> 
>> On 25 Dec 2022, at 16:33, Michał Górny <mgorny@gentoo.org> wrote:
>> 
>> On Sun, 2022-12-25 at 01:45 +0900, Masahiro Yamada wrote:
>>> Instead of inserting a workaround like this,
>>> another way is to allow users to override a variable
>>> from the command line.
>>> 
>>> 
>>> See the top Makefile, for example,
>>> 
>>> 
>>> AWK             = awk
>>> 
>>> 
>>> Then, users can do "make AWK=gawk"
>> 
>> I'm sorry but are you requesting that I remove the check and use $TAR
>> instead, or allow overriding with TAR, and fall back to gtar or tar
>> respectively?  If the former, should the script unconditionally assume
>> that TAR will be always set in the environment, or include fallback to
>> tar for when the script is run directly?
> 
> 
> Masahiro, what do you reckon? Thanks.
> 

Ping.

> Best,
> sam

Re: [PATCH] kheaders: prefer gtar over tar for better compatibility

Posted by Sam James 2 years, 6 months ago

Sam James <sam@gentoo.org> writes:

> [[PGP Signed Part:Undecided]]
>
>
>> On 17 Jan 2023, at 19:01, Sam James <sam@gentoo.org> wrote:
>> 
>> 
>> 
>>> On 25 Dec 2022, at 16:33, Michał Górny <mgorny@gentoo.org> wrote:
>>> 
>>> On Sun, 2022-12-25 at 01:45 +0900, Masahiro Yamada wrote:
>>>> Instead of inserting a workaround like this,
>>>> another way is to allow users to override a variable
>>>> from the command line.
>>>> 
>>>> 
>>>> See the top Makefile, for example,
>>>> 
>>>> 
>>>> AWK             = awk
>>>> 
>>>> 
>>>> Then, users can do "make AWK=gawk"
>>> 
>>> I'm sorry but are you requesting that I remove the check and use $TAR
>>> instead, or allow overriding with TAR, and fall back to gtar or tar
>>> respectively?  If the former, should the script unconditionally assume
>>> that TAR will be always set in the environment, or include fallback to
>>> tar for when the script is run directly?
>> 
>> 
>> Masahiro, what do you reckon? Thanks.
>> 
>
> Ping.

ping^2. We'd really love to get this in to fix a bug for our users in
Gentoo.

>
>> Best,
>> sam
>
>
> [[End of PGP Signed Part]]

Re: [PATCH] kheaders: prefer gtar over tar for better compatibility

Posted by Masahiro Yamada 2 years, 6 months ago

I thought I had relied this before, but
something awkward was happening because
the original email was addressed to
yamada.masahiro@socionext.com, which is
no longer valid.

Resending from masahiroy@kernel.org...






In the top Makefile, please add

TAR  = tar

and export TAR.



In kernel/gen_kheaders.sh,
please replace 'tar' with ${TAR:-tar}





On Gentoo systems, you can do

 $ make TAR=gtar


or if you need to run gen_kheaders.sh directly

 $ TAR=gtar scripts/gen_kheaders.sh


Thanks.








On Fri, Mar 17, 2023 at 9:05 AM Sam James <sam@gentoo.org> wrote:
>
>
> Sam James <sam@gentoo.org> writes:
>
> > [[PGP Signed Part:Undecided]]
> >
> >
> >> On 17 Jan 2023, at 19:01, Sam James <sam@gentoo.org> wrote:
> >>
> >>
> >>
> >>> On 25 Dec 2022, at 16:33, Michał Górny <mgorny@gentoo.org> wrote:
> >>>
> >>> On Sun, 2022-12-25 at 01:45 +0900, Masahiro Yamada wrote:
> >>>> Instead of inserting a workaround like this,
> >>>> another way is to allow users to override a variable
> >>>> from the command line.
> >>>>
> >>>>
> >>>> See the top Makefile, for example,
> >>>>
> >>>>
> >>>> AWK             = awk
> >>>>
> >>>>
> >>>> Then, users can do "make AWK=gawk"
> >>>
> >>> I'm sorry but are you requesting that I remove the check and use $TAR
> >>> instead, or allow overriding with TAR, and fall back to gtar or tar
> >>> respectively?  If the former, should the script unconditionally assume
> >>> that TAR will be always set in the environment, or include fallback to
> >>> tar for when the script is run directly?
> >>
> >>
> >> Masahiro, what do you reckon? Thanks.
> >>
> >
> > Ping.
>
> ping^2. We'd really love to get this in to fix a bug for our users in
> Gentoo.
>
> >
> >> Best,
> >> sam
> >
> >
> > [[End of PGP Signed Part]]
>

--
Best Regards
Masahiro Yamada

[PATCH v2] kheaders: make it possible to override TAR

Posted by Michał Górny 2 years, 5 months ago

Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
archive reproducible") introduced a number of options specific to GNU
tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
the script to fail to work on systems where `tar` is not GNU tar.  This
can occur e.g. on recent Gentoo Linux installations that support using
bsdtar from libarchive instead.

Add a `TAR` make variable to make it possible to override the tar
executable used, e.g. by specifying:

  make TAR=gtar

Link: https://bugs.gentoo.org/884061
Reported-by: Sam James <sam@gentoo.org>
Tested-by: Sam James <sam@gentoo.org>
Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
 Makefile               | 3 ++-
 kernel/gen_kheaders.sh | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/Makefile b/Makefile
index 5aeea3d98..50045059c 100644
--- a/Makefile
+++ b/Makefile
@@ -520,6 +520,7 @@ LZMA		= lzma
 LZ4		= lz4c
 XZ		= xz
 ZSTD		= zstd
+TAR		= tar
 
 PAHOLE_FLAGS	= $(shell PAHOLE=$(PAHOLE) $(srctree)/scripts/pahole-flags.sh)
 
@@ -599,7 +600,7 @@ export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN CARGO
 export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
 export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
 export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
-export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
+export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD TAR
 export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
 export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS
 
diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
index 1ef9a8751..82d539648 100755
--- a/kernel/gen_kheaders.sh
+++ b/kernel/gen_kheaders.sh
@@ -86,7 +86,7 @@ find $cpio_dir -type f -print0 |
 # For compatibility with older versions of tar, files are fed to tar
 # pre-sorted, as --sort=name might not be available.
 find $cpio_dir -printf "./%P\n" | LC_ALL=C sort | \
-    tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
+    ${TAR:-tar} "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
     --owner=0 --group=0 --numeric-owner --no-recursion \
     -I $XZ -cf $tarfile -C $cpio_dir/ -T - > /dev/null
 
-- 
2.40.0

[PATCH v3] kheaders: make it possible to override TAR

Posted by Sam James 1 month, 4 weeks ago

From: Michał Górny <mgorny@gentoo.org>

Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
archive reproducible") introduced a number of options specific to GNU
tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
the script to fail to work on systems where `tar` is not GNU tar.  This
can occur e.g. on recent Gentoo Linux installations that support using
bsdtar from libarchive instead.

Add a `TAR` make variable to make it possible to override the tar
executable used, e.g. by specifying:

  make TAR=gtar

Link: https://bugs.gentoo.org/884061
Reported-by: Sam James <sam@gentoo.org>
Tested-by: Sam James <sam@gentoo.org>
Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Michał Górny <mgorny@gentoo.org>
Signed-off-by: Sam James <sam@gentoo.org>
---
v3: Rebase, cover more tar instances.

 Makefile               | 3 ++-
 kernel/gen_kheaders.sh | 6 +++---
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index c09766beb7eff..22d6037d738fe 100644
--- a/Makefile
+++ b/Makefile
@@ -543,6 +543,7 @@ LZMA		= lzma
 LZ4		= lz4
 XZ		= xz
 ZSTD		= zstd
+TAR		= tar
 
 CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
 		  -Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF)
@@ -622,7 +623,7 @@ export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
 export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
 export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
 export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
-export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
+export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD TAR
 export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS KBUILD_PROCMACROLDFLAGS LDFLAGS_MODULE
 export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS
 
diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
index c9e5dc068e854..bb609a9ed72b4 100755
--- a/kernel/gen_kheaders.sh
+++ b/kernel/gen_kheaders.sh
@@ -66,13 +66,13 @@ if [ "$building_out_of_srctree" ]; then
 		cd $srctree
 		for f in $dir_list
 			do find "$f" -name "*.h";
-		done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
+		done | ${TAR:-tar} -c -f - -T - | ${TAR:-tar} -xf - -C "${tmpdir}"
 	)
 fi
 
 for f in $dir_list;
 	do find "$f" -name "*.h";
-done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
+done | ${TAR:-tar} -c -f - -T - | ${TAR:-tar} -xf - -C "${tmpdir}"
 
 # Always exclude include/generated/utsversion.h
 # Otherwise, the contents of the tarball may vary depending on the build steps.
@@ -88,7 +88,7 @@ xargs -0 -P8 -n1 \
 rm -f "${tmpdir}.contents.txt"
 
 # Create archive and try to normalize metadata for reproducibility.
-tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
+${TAR:-tar} "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
     --owner=0 --group=0 --sort=name --numeric-owner --mode=u=rw,go=r,a+X \
     -I $XZ -cf $tarfile -C "${tmpdir}/" . > /dev/null
 
-- 
2.50.1

Re: [PATCH v3] kheaders: make it possible to override TAR

Posted by Nathan Chancellor 1 month, 4 weeks ago

On Sat, Jul 19, 2025 at 04:24:05PM +0100, Sam James wrote:
> From: Michał Górny <mgorny@gentoo.org>
> 
> Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
> archive reproducible") introduced a number of options specific to GNU
> tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
> the script to fail to work on systems where `tar` is not GNU tar.  This
> can occur e.g. on recent Gentoo Linux installations that support using
> bsdtar from libarchive instead.
> 
> Add a `TAR` make variable to make it possible to override the tar
> executable used, e.g. by specifying:
> 
>   make TAR=gtar
> 
> Link: https://bugs.gentoo.org/884061
> Reported-by: Sam James <sam@gentoo.org>
> Tested-by: Sam James <sam@gentoo.org>
> Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Michał Górny <mgorny@gentoo.org>
> Signed-off-by: Sam James <sam@gentoo.org>
> ---

Reviewed-by: Nathan Chancellor <nathan@kernel.org>

I assume that other places that call tar within the build process are
not problematic because they do not use GNU specific options, such as
scripts/Makefile.package and scripts/package/install-extmod-build, or
maybe that people just have not tried building those packages with
bsdtar?

> v3: Rebase, cover more tar instances.
> 
>  Makefile               | 3 ++-
>  kernel/gen_kheaders.sh | 6 +++---
>  2 files changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index c09766beb7eff..22d6037d738fe 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -543,6 +543,7 @@ LZMA		= lzma
>  LZ4		= lz4
>  XZ		= xz
>  ZSTD		= zstd
> +TAR		= tar
>  
>  CHECKFLAGS     := -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \
>  		  -Wbitwise -Wno-return-void -Wno-unknown-attribute $(CF)
> @@ -622,7 +623,7 @@ export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
>  export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
>  export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AWK INSTALLKERNEL
>  export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> -export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD
> +export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD TAR
>  export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS KBUILD_PROCMACROLDFLAGS LDFLAGS_MODULE
>  export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS
>  
> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> index c9e5dc068e854..bb609a9ed72b4 100755
> --- a/kernel/gen_kheaders.sh
> +++ b/kernel/gen_kheaders.sh
> @@ -66,13 +66,13 @@ if [ "$building_out_of_srctree" ]; then
>  		cd $srctree
>  		for f in $dir_list
>  			do find "$f" -name "*.h";
> -		done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
> +		done | ${TAR:-tar} -c -f - -T - | ${TAR:-tar} -xf - -C "${tmpdir}"
>  	)
>  fi
>  
>  for f in $dir_list;
>  	do find "$f" -name "*.h";
> -done | tar -c -f - -T - | tar -xf - -C "${tmpdir}"
> +done | ${TAR:-tar} -c -f - -T - | ${TAR:-tar} -xf - -C "${tmpdir}"
>  
>  # Always exclude include/generated/utsversion.h
>  # Otherwise, the contents of the tarball may vary depending on the build steps.
> @@ -88,7 +88,7 @@ xargs -0 -P8 -n1 \
>  rm -f "${tmpdir}.contents.txt"
>  
>  # Create archive and try to normalize metadata for reproducibility.
> -tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
> +${TAR:-tar} "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
>      --owner=0 --group=0 --sort=name --numeric-owner --mode=u=rw,go=r,a+X \
>      -I $XZ -cf $tarfile -C "${tmpdir}/" . > /dev/null
>  
> -- 
> 2.50.1
>

Re: [PATCH v3] kheaders: make it possible to override TAR

Posted by Michał Górny 1 month, 4 weeks ago

On Sat, 2025-07-19 at 16:10 -0400, Nathan Chancellor wrote:
> On Sat, Jul 19, 2025 at 04:24:05PM +0100, Sam James wrote:
> > From: Michał Górny <mgorny@gentoo.org>
> > 
> > Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
> > archive reproducible") introduced a number of options specific to GNU
> > tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
> > the script to fail to work on systems where `tar` is not GNU tar.  This
> > can occur e.g. on recent Gentoo Linux installations that support using
> > bsdtar from libarchive instead.
> > 
> > Add a `TAR` make variable to make it possible to override the tar
> > executable used, e.g. by specifying:
> > 
> >   make TAR=gtar
> > 
> > Link: https://bugs.gentoo.org/884061
> > Reported-by: Sam James <sam@gentoo.org>
> > Tested-by: Sam James <sam@gentoo.org>
> > Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
> > Signed-off-by: Michał Górny <mgorny@gentoo.org>
> > Signed-off-by: Sam James <sam@gentoo.org>
> > ---
> 
> Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> 
> I assume that other places that call tar within the build process are
> not problematic because they do not use GNU specific options, such as
> scripts/Makefile.package and scripts/package/install-extmod-build, or
> maybe that people just have not tried building those packages with
> bsdtar?

Precisely.  We focused on the one place which actually breaks our build,
to avoid touching too many subsystems simultaneously.  If this is
desirable, I can look into replacing the other instances.


-- 
Best regards,
Michał Górny

Re: [PATCH v3] kheaders: make it possible to override TAR

Posted by Masahiro Yamada 1 month, 3 weeks ago

On Mon, Jul 21, 2025 at 4:08 AM Michał Górny <mgorny@gentoo.org> wrote:
>
> On Sat, 2025-07-19 at 16:10 -0400, Nathan Chancellor wrote:
> > On Sat, Jul 19, 2025 at 04:24:05PM +0100, Sam James wrote:
> > > From: Michał Górny <mgorny@gentoo.org>
> > >
> > > Commit 86cdd2fdc4e39c388d39c7ba2396d1a9dfd66226 ("kheaders: make headers
> > > archive reproducible") introduced a number of options specific to GNU
> > > tar to the `tar` invocation in `gen_kheaders.sh` script.  This causes
> > > the script to fail to work on systems where `tar` is not GNU tar.  This
> > > can occur e.g. on recent Gentoo Linux installations that support using
> > > bsdtar from libarchive instead.
> > >
> > > Add a `TAR` make variable to make it possible to override the tar
> > > executable used, e.g. by specifying:
> > >
> > >   make TAR=gtar
> > >
> > > Link: https://bugs.gentoo.org/884061
> > > Reported-by: Sam James <sam@gentoo.org>
> > > Tested-by: Sam James <sam@gentoo.org>
> > > Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
> > > Signed-off-by: Michał Górny <mgorny@gentoo.org>
> > > Signed-off-by: Sam James <sam@gentoo.org>
> > > ---
> >
> > Reviewed-by: Nathan Chancellor <nathan@kernel.org>
> >
> > I assume that other places that call tar within the build process are
> > not problematic because they do not use GNU specific options, such as
> > scripts/Makefile.package and scripts/package/install-extmod-build, or
> > maybe that people just have not tried building those packages with
> > bsdtar?
>
> Precisely.  We focused on the one place which actually breaks our build,
> to avoid touching too many subsystems simultaneously.  If this is
> desirable, I can look into replacing the other instances.
>

This patch is not applicable to my tree.

Please rebase to the kbuild tree (or linux-next).




-- 
Best Regards
Masahiro Yamada

Re: [PATCH v3] kheaders: make it possible to override TAR

Posted by Nicolas Schier 1 month, 4 weeks ago

On Sun, Jul 20, 2025 at 09:08:20PM +0200 Michał Górny wrote:
> On Sat, 2025-07-19 at 16:10 -0400, Nathan Chancellor wrote:
[...]
> > I assume that other places that call tar within the build process are
> > not problematic because they do not use GNU specific options, such as
> > scripts/Makefile.package and scripts/package/install-extmod-build, or
> > maybe that people just have not tried building those packages with
> > bsdtar?
> 
> Precisely.  We focused on the one place which actually breaks our build,
> to avoid touching too many subsystems simultaneously.  If this is
> desirable, I can look into replacing the other instances.

Without further investigation, I think it makes sense to use the same
tar-instance for all tar calls.  Thus, I'd appreciate if you could replace
those as well.

Kind regards,
Nicolas