Remap source path prefixes in all output, including compiler
diagnostics, debug information, macro expansions, etc.
This removes a few absolute paths from the binary and also makes it
possible to use core::panic::Location properly.

Equivalent to the same configuration done for C sources in
commit 1d3730f0012f ("kbuild: support -fmacro-prefix-map for external modules")
and commit a73619a845d5 ("kbuild: use -fmacro-prefix-map to make __FILE__ a relative path").

Link: https://doc.rust-lang.org/rustc/command-line-arguments.html#--remap-path-prefix-remap-source-names-in-output
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
---
 Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Makefile b/Makefile
index 9e0d63d9d94b90672f91929e5e148e5a0c346cb6..ac35083180c825b72f13149ec3acfd7d6d74ef98 100644
--- a/Makefile
+++ b/Makefile
@@ -1068,6 +1068,7 @@ endif
 # change __FILE__ to the relative path to the source directory
 ifdef building_out_of_srctree
 KBUILD_CPPFLAGS += $(call cc-option,-fmacro-prefix-map=$(srcroot)/=)
+KBUILD_RUSTFLAGS += $(call rustc-option,--remap-path-prefix=$(srcroot)/=)
 endif
 
 # include additional Makefiles when needed

---
base-commit: beeb78d46249cab8b2b8359a2ce8fa5376b5ad2d
change-id: 20250210-rust-path-remap-e97cec71e61a

Best regards,
-- 
Thomas Weißschuh <linux@weissschuh.net>

On Mon, Feb 10, 2025 at 6:11 PM Thomas Weißschuh <linux@weissschuh.net> wrote:
>
> Remap source path prefixes in all output, including compiler
> diagnostics, debug information, macro expansions, etc.

Hmm... We don't do all the cases in the C side -- the docs ask to use
`KCFLAGS` when one wants to remove them in the debug info:

    https://docs.kernel.org/kbuild/reproducible-builds.html#absolute-filenames

I am not sure if there is a reason not to cover all cases in C (Cc'ing Ben).

If there is a reason to not do it for the debug info by default (or if
we want to make it consistent with C even if there is no reason), then
I think we would need `-Zremap-path-scope=...` too, which is still
unstable, sadly (Cc'ing Urgau who implemented it for Rust 1.75.0):

    https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/remap-path-scope.html
    https://github.com/rust-lang/rust/issues/111540

In such case, we would also need probably to mention in the
`reproducible-builds` docs how to achieve the same as in C (e.g.
passing an extra `-Zremap-path-scope=debug`, since it aggregates with
the previous ones, according to compiler flag docs).

>  # change __FILE__ to the relative path to the source directory

Perhaps we could add a note to this comment with what we do in the Rust side.

> +KBUILD_RUSTFLAGS += $(call rustc-option,--remap-path-prefix=$(srcroot)/=)

I don't think we need `rustc-option`, since the flag is available
since a long time ago (Rust 1.26.0).

So we should be able to just do:

    KBUILD_RUSTFLAGS += --remap-path-prefix=$(srcroot)/=

Cheers,
Miguel

On Mon, Feb 17, 2025 at 10:23 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Mon, Feb 10, 2025 at 6:11 PM Thomas Weißschuh <linux@weissschuh.net> wrote:
> >
> > Remap source path prefixes in all output, including compiler
> > diagnostics, debug information, macro expansions, etc.
>
> Hmm... We don't do all the cases in the C side -- the docs ask to use
> `KCFLAGS` when one wants to remove them in the debug info:
>
>     https://docs.kernel.org/kbuild/reproducible-builds.html#absolute-filenames
>
> I am not sure if there is a reason not to cover all cases in C (Cc'ing Ben).


GCC manual mentions the below about the -fdebug-prefix-map=old=new


"It can also be used to change an absolute path to
a relative path by using . for new.
This can give more reproducible builds, which are location
independent, but may require an extra command to tell GDB
where to find the source files."


I guess "the extra command" might be a bit annoying.






-- 
Best Regards
Masahiro Yamada

On Tue, 2025-03-11 at 20:03 +0900, Masahiro Yamada wrote:
> On Mon, Feb 17, 2025 at 10:23 PM Miguel Ojeda
> <miguel.ojeda.sandonis@gmail.com> wrote:
> > 
> > On Mon, Feb 10, 2025 at 6:11 PM Thomas Weißschuh <linux@weissschuh.net> wrote:
> > > 
> > > Remap source path prefixes in all output, including compiler
> > > diagnostics, debug information, macro expansions, etc.
> > 
> > Hmm... We don't do all the cases in the C side -- the docs ask to use
> > `KCFLAGS` when one wants to remove them in the debug info:
> > 
> >     https://docs.kernel.org/kbuild/reproducible-builds.html#absolute-filenames
> > 
> > I am not sure if there is a reason not to cover all cases in C (Cc'ing Ben).

I think we should use the prefix-map options by default, for both C and
Rust code.

The default of using absolute filenames only works when the build and
debug systems have their sources in the same absolute directory.  For
some developers this will always be true because they're the same
system.  In the general case, and particularly in production, it's
unlikely to be true.  See below.

> GCC manual mentions the below about the -fdebug-prefix-map=old=new
> 
> 
> "It can also be used to change an absolute path to
> a relative path by using . for new.
> This can give more reproducible builds, which are location
> independent, but may require an extra command to tell GDB
> where to find the source files."
> 
> 
> I guess "the extra command" might be a bit annoying.

The command in question is "dir <source-root-directory>".  It's not
particulary annoying.  It also isn't needed when invoking gdb with the
kernel source root as its current directory.

Running that command adds the directory to gdb's source search path,
causing it to look for sources referenced in debug info by:

1. Appending the full filename to that directory
2. Appending the base filename to that directory

(This is slightly simplified from the docuemntation at:
<https://sourceware.org/gdb/current/onlinedocs/gdb.html/Source-Path.html#Source-Path>.)

When the debug info has relative filenames, case 1 should always work.

When the debug info has absolute filenames, case 2 can work if all
sources are in the same directory.  But the kernel has source files
spread across many directories, so there's no general way to make gdb
find them all.

Ben.

-- 
Ben Hutchings
A free society is one where it is safe to be unpopular.
                                                      - Adlai Stevenson

On Wed, Mar 12, 2025 at 2:36 AM Ben Hutchings <ben@decadent.org.uk> wrote:
>
> On Tue, 2025-03-11 at 20:03 +0900, Masahiro Yamada wrote:
> > On Mon, Feb 17, 2025 at 10:23 PM Miguel Ojeda
> > <miguel.ojeda.sandonis@gmail.com> wrote:
> > >
> > > On Mon, Feb 10, 2025 at 6:11 PM Thomas Weißschuh <linux@weissschuh.net> wrote:
> > > >
> > > > Remap source path prefixes in all output, including compiler
> > > > diagnostics, debug information, macro expansions, etc.
> > >
> > > Hmm... We don't do all the cases in the C side -- the docs ask to use
> > > `KCFLAGS` when one wants to remove them in the debug info:
> > >
> > >     https://docs.kernel.org/kbuild/reproducible-builds.html#absolute-filenames
> > >
> > > I am not sure if there is a reason not to cover all cases in C (Cc'ing Ben).
>
> I think we should use the prefix-map options by default, for both C and
> Rust code.

A patch is appreciated.
(top-level Makefile change with updates
in Documentation/kbuild/reproducible-builds.rst)

>
> The default of using absolute filenames only works when the build and
> debug systems have their sources in the same absolute directory.  For
> some developers this will always be true because they're the same
> system.  In the general case, and particularly in production, it's
> unlikely to be true.  See below.
>
> > GCC manual mentions the below about the -fdebug-prefix-map=old=new
> >
> >
> > "It can also be used to change an absolute path to
> > a relative path by using . for new.
> > This can give more reproducible builds, which are location
> > independent, but may require an extra command to tell GDB
> > where to find the source files."
> >
> >
> > I guess "the extra command" might be a bit annoying.
>
> The command in question is "dir <source-root-directory>".  It's not
> particulary annoying.  It also isn't needed when invoking gdb with the
> kernel source root as its current directory.
>
> Running that command adds the directory to gdb's source search path,
> causing it to look for sources referenced in debug info by:
>
> 1. Appending the full filename to that directory
> 2. Appending the base filename to that directory
>
> (This is slightly simplified from the docuemntation at:
> <https://sourceware.org/gdb/current/onlinedocs/gdb.html/Source-Path.html#Source-Path>.)
>
> When the debug info has relative filenames, case 1 should always work.
>
> When the debug info has absolute filenames, case 2 can work if all
> sources are in the same directory.  But the kernel has source files
> spread across many directories, so there's no general way to make gdb
> find them all.
>
> Ben.
>
> --
> Ben Hutchings
> A free society is one where it is safe to be unpopular.
>                                                       - Adlai Stevenson



-- 
Best Regards
Masahiro Yamada

On 2025-03-12 04:07:51+0900, Masahiro Yamada wrote:
> On Wed, Mar 12, 2025 at 2:36 AM Ben Hutchings <ben@decadent.org.uk> wrote:
> >
> > On Tue, 2025-03-11 at 20:03 +0900, Masahiro Yamada wrote:
> > > On Mon, Feb 17, 2025 at 10:23 PM Miguel Ojeda
> > > <miguel.ojeda.sandonis@gmail.com> wrote:
> > > >
> > > > On Mon, Feb 10, 2025 at 6:11 PM Thomas Weißschuh <linux@weissschuh.net> wrote:
> > > > >
> > > > > Remap source path prefixes in all output, including compiler
> > > > > diagnostics, debug information, macro expansions, etc.
> > > >
> > > > Hmm... We don't do all the cases in the C side -- the docs ask to use
> > > > `KCFLAGS` when one wants to remove them in the debug info:
> > > >
> > > >     https://docs.kernel.org/kbuild/reproducible-builds.html#absolute-filenames
> > > >
> > > > I am not sure if there is a reason not to cover all cases in C (Cc'ing Ben).
> >
> > I think we should use the prefix-map options by default, for both C and
> > Rust code.
> 
> A patch is appreciated.
> (top-level Makefile change with updates
> in Documentation/kbuild/reproducible-builds.rst)

I'm happy to send a patch.
The new patch will conflict with the rust patch.
Could you push the current kbuild tree to git.kernel.org so the new
patch can be based on top of it?

> > The default of using absolute filenames only works when the build and
> > debug systems have their sources in the same absolute directory.  For
> > some developers this will always be true because they're the same
> > system.  In the general case, and particularly in production, it's
> > unlikely to be true.  See below.
> >
> > > GCC manual mentions the below about the -fdebug-prefix-map=old=new
> > >
> > >
> > > "It can also be used to change an absolute path to
> > > a relative path by using . for new.
> > > This can give more reproducible builds, which are location
> > > independent, but may require an extra command to tell GDB
> > > where to find the source files."
> > >
> > >
> > > I guess "the extra command" might be a bit annoying.
> >
> > The command in question is "dir <source-root-directory>".  It's not
> > particulary annoying.  It also isn't needed when invoking gdb with the
> > kernel source root as its current directory.
> >
> > Running that command adds the directory to gdb's source search path,
> > causing it to look for sources referenced in debug info by:
> >
> > 1. Appending the full filename to that directory
> > 2. Appending the base filename to that directory
> >
> > (This is slightly simplified from the docuemntation at:
> > <https://sourceware.org/gdb/current/onlinedocs/gdb.html/Source-Path.html#Source-Path>.)
> >
> > When the debug info has relative filenames, case 1 should always work.
> >
> > When the debug info has absolute filenames, case 2 can work if all
> > sources are in the same directory.  But the kernel has source files
> > spread across many directories, so there's no general way to make gdb
> > find them all.
> >
> > Ben.
> >
> > --
> > Ben Hutchings
> > A free society is one where it is safe to be unpopular.
> >                                                       - Adlai Stevenson

On Wed, Mar 12, 2025 at 6:05 AM Thomas Weißschuh <linux@weissschuh.net> wrote:
>
> On 2025-03-12 04:07:51+0900, Masahiro Yamada wrote:
> > On Wed, Mar 12, 2025 at 2:36 AM Ben Hutchings <ben@decadent.org.uk> wrote:
> > >
> > > On Tue, 2025-03-11 at 20:03 +0900, Masahiro Yamada wrote:
> > > > On Mon, Feb 17, 2025 at 10:23 PM Miguel Ojeda
> > > > <miguel.ojeda.sandonis@gmail.com> wrote:
> > > > >
> > > > > On Mon, Feb 10, 2025 at 6:11 PM Thomas Weißschuh <linux@weissschuh.net> wrote:
> > > > > >
> > > > > > Remap source path prefixes in all output, including compiler
> > > > > > diagnostics, debug information, macro expansions, etc.
> > > > >
> > > > > Hmm... We don't do all the cases in the C side -- the docs ask to use
> > > > > `KCFLAGS` when one wants to remove them in the debug info:
> > > > >
> > > > >     https://docs.kernel.org/kbuild/reproducible-builds.html#absolute-filenames
> > > > >
> > > > > I am not sure if there is a reason not to cover all cases in C (Cc'ing Ben).
> > >
> > > I think we should use the prefix-map options by default, for both C and
> > > Rust code.
> >
> > A patch is appreciated.
> > (top-level Makefile change with updates
> > in Documentation/kbuild/reproducible-builds.rst)
>
> I'm happy to send a patch.
> The new patch will conflict with the rust patch.
> Could you push the current kbuild tree to git.kernel.org so the new
> patch can be based on top of it?

Yes, I pushed.




--
Best Regards
Masahiro Yamada

On Thu, Mar 13, 2025 at 1:35 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Yes, I pushed.

I have noticed in linux-next that the flag triggers an ICE when
building the docs with `O=` -- reduced and reported to upstream here:

    https://github.com/rust-lang/rust/issues/138520

Moreover, the flag was only added (to `rustdoc`, not `rustc`) starting
with 1.81.0, so we couldn't pass it anyway unconditionally to
`rustdoc` (it is OK for `rustc`).

In the meantime, the easiest to avoid both issues is to avoid passing
the flag when building docs -- I have sent a patch:

    https://lore.kernel.org/rust-for-linux/20250314213333.2230670-1-ojeda@kernel.org/

Cheers,
Miguel

On Tue, Mar 11, 2025 at 12:04 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> GCC manual mentions the below about the -fdebug-prefix-map=old=new
>
> "It can also be used to change an absolute path to
> a relative path by using . for new.
> This can give more reproducible builds, which are location
> independent, but may require an extra command to tell GDB
> where to find the source files."
>
> I guess "the extra command" might be a bit annoying.

Ah, so you mean that could be reason for not doing it for the debug
case -- I see, thanks!

Cheers,
Miguel

On Tue, Feb 11, 2025 at 2:11 AM Thomas Weißschuh <linux@weissschuh.net> wrote:
>
> Remap source path prefixes in all output, including compiler
> diagnostics, debug information, macro expansions, etc.
> This removes a few absolute paths from the binary and also makes it
> possible to use core::panic::Location properly.
>
> Equivalent to the same configuration done for C sources in
> commit 1d3730f0012f ("kbuild: support -fmacro-prefix-map for external modules")
> and commit a73619a845d5 ("kbuild: use -fmacro-prefix-map to make __FILE__ a relative path").
>
> Link: https://doc.rust-lang.org/rustc/command-line-arguments.html#--remap-path-prefix-remap-source-names-in-output
> Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>


I will apply this if Miguel gives Ack.




> ---
>  Makefile | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/Makefile b/Makefile
> index 9e0d63d9d94b90672f91929e5e148e5a0c346cb6..ac35083180c825b72f13149ec3acfd7d6d74ef98 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1068,6 +1068,7 @@ endif
>  # change __FILE__ to the relative path to the source directory
>  ifdef building_out_of_srctree
>  KBUILD_CPPFLAGS += $(call cc-option,-fmacro-prefix-map=$(srcroot)/=)
> +KBUILD_RUSTFLAGS += $(call rustc-option,--remap-path-prefix=$(srcroot)/=)
>  endif
>
>  # include additional Makefiles when needed
>
> ---
> base-commit: beeb78d46249cab8b2b8359a2ce8fa5376b5ad2d
> change-id: 20250210-rust-path-remap-e97cec71e61a
>
> Best regards,
> --
> Thomas Weißschuh <linux@weissschuh.net>
>


-- 
Best Regards
Masahiro Yamada

On Sat, Feb 15, 2025 at 1:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> I will apply this if Miguel gives Ack.

Thanks Masahiro -- I left some comments above.

This does remove some absolute paths for me, though not all (e.g.
`core.o` that comes from the sysroot). Since it is an improvement
already, if you want to apply it:

Acked-by: Miguel Ojeda <ojeda@kernel.org>

I leave below a diff that works for `core.o` for me -- it would do it
in all cases, not just `building_out_of_srctree`, since it is outside
the repository in all cases, which I think makes sense, e.g. in an
in-tree build I get:

    $ strings rust/core.o | grep validations.rs
    lib/rustlib/src/rust/library/core/src/str/validations.rs

Cheers,
Miguel

diff --git a/rust/Makefile b/rust/Makefile
index ea3849eb78f6..42f242472031 100644
--- a/rust/Makefile
+++ b/rust/Makefile
@@ -438,7 +438,7 @@ $(obj)/exports.o: private skip_gendwarfksyms = 1
 $(obj)/core.o: private skip_clippy = 1
 $(obj)/core.o: private skip_flags = -Wunreachable_pub
 $(obj)/core.o: private rustc_objcopy = $(foreach
sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))
-$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
+$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
--remap-path-prefix=$(rustc_sysroot)/=
 $(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs \
     $(wildcard $(objtree)/include/config/RUSTC_VERSION_TEXT) FORCE
        +$(call if_changed_rule,rustc_library)

On Mon, Feb 17, 2025 at 10:26 PM Miguel Ojeda
<miguel.ojeda.sandonis@gmail.com> wrote:
>
> On Sat, Feb 15, 2025 at 1:57 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
> >
> > I will apply this if Miguel gives Ack.
>
> Thanks Masahiro -- I left some comments above.
>
> This does remove some absolute paths for me, though not all (e.g.
> `core.o` that comes from the sysroot). Since it is an improvement
> already, if you want to apply it:
>
> Acked-by: Miguel Ojeda <ojeda@kernel.org>
>
> I leave below a diff that works for `core.o` for me -- it would do it
> in all cases, not just `building_out_of_srctree`, since it is outside
> the repository in all cases, which I think makes sense, e.g. in an
> in-tree build I get:
>
>     $ strings rust/core.o | grep validations.rs
>     lib/rustlib/src/rust/library/core/src/str/validations.rs
>
> Cheers,
> Miguel
>
> diff --git a/rust/Makefile b/rust/Makefile
> index ea3849eb78f6..42f242472031 100644
> --- a/rust/Makefile
> +++ b/rust/Makefile
> @@ -438,7 +438,7 @@ $(obj)/exports.o: private skip_gendwarfksyms = 1
>  $(obj)/core.o: private skip_clippy = 1
>  $(obj)/core.o: private skip_flags = -Wunreachable_pub
>  $(obj)/core.o: private rustc_objcopy = $(foreach
> sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))
> -$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
> +$(obj)/core.o: private rustc_target_flags = $(core-cfgs)
> --remap-path-prefix=$(rustc_sysroot)/=
>  $(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs \
>      $(wildcard $(objtree)/include/config/RUSTC_VERSION_TEXT) FORCE
>         +$(call if_changed_rule,rustc_library)

LGTM. Thanks.


-- 
Best Regards
Masahiro Yamada