Makefile | 1 + arch/um/Makefile | 2 ++ include/linux/compiler-version.h | 10 ++++++++++ include/linux/vermagic.h | 1 - scripts/Makefile.gcc-plugins | 2 +- scripts/Makefile.ubsan | 1 + scripts/basic/Makefile | 20 +++++++++++++++----- scripts/gcc-plugins/Makefile | 8 ++++++++ 8 files changed, 38 insertions(+), 7 deletions(-)
v2: - switch from -include to -I with a -D gated include compiler-version.h v1: https://lore.kernel.org/lkml/20250501193839.work.525-kees@kernel.org/ Hi, This is my attempt to introduce dependencies that track the various compiler behaviors that may globally change the build that aren't represented by either compiler flags nor the compiler version (CC_VERSION_TEXT). Namely, this is to detect when the contents of a file the compiler uses changes. We have 3 such situations currently in the tree: - If any of the GCC plugins change, we need to rebuild everything that was built with them, as they may have changed their behavior and those behaviors may need to be synchronized across all translation units. (The most obvious of these is the randstruct GCC plugin, but is true for most of them.) - If the randstruct seed itself changes (whether for GCC plugins or Clang), the entire tree needs to be rebuilt since the randomization of structures may change between compilation units if not. - If the integer-wrap-ignore.scl file for Clang's integer wrapping sanitizer changes, a full rebuild is needed as the coverage for wrapping types may have changed, once again cause behavior differences between compilation units. The best way I found to deal with this is to: - Generate a .h file that is updated when the specific dependencies change. e.g.: randstruct_hash.h depends on randstruct.seed - Add a -I argument globally to be able to locate the .h file. e.g.: -I$(objtree)/scripts/basic - Add a conditional -D argument for each separate case e.g.: RANDSTRUCT_CFLAGS += -DRANDSTRUCT - Include the .h file from compiler-version.h through an #ifdef for the define e.g.: #ifdef RANDSTUCT #include "randstruct_hash.h" #endif This means that all targets gain the dependency (via fixdep), but only when the defines are active, which means they are trivially controlled by the existing CFLAGS removal mechanisms that are already being used to turn off each of the above features. -Kees Kees Cook (3): gcc-plugins: Force full rebuild when plugins change randstruct: Force full rebuild when seed changes integer-wrap: Force full rebuild when .scl file changes Makefile | 1 + arch/um/Makefile | 2 ++ include/linux/compiler-version.h | 10 ++++++++++ include/linux/vermagic.h | 1 - scripts/Makefile.gcc-plugins | 2 +- scripts/Makefile.ubsan | 1 + scripts/basic/Makefile | 20 +++++++++++++++----- scripts/gcc-plugins/Makefile | 8 ++++++++ 8 files changed, 38 insertions(+), 7 deletions(-) -- 2.34.1
On Sat, May 3, 2025 at 7:54 AM Kees Cook <kees@kernel.org> wrote: > > v2: > - switch from -include to -I with a -D gated include compiler-version.h > v1: https://lore.kernel.org/lkml/20250501193839.work.525-kees@kernel.org/ What do you think of my patch as a prerequisite? https://lore.kernel.org/linux-kbuild/20250503084145.1994176-1-masahiroy@kernel.org/T/#u Perhaps, can you implement this series more simply? My idea is to touch a single include/generated/global-rebuild.h rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc. When the file is touched, the entire kernel source tree will be rebuilt. This may rebuild more than needed (e.g. vdso) but I do not think it is a big deal. -- Best Regards Masahiro Yamada
On Sat, May 03, 2025 at 06:39:28PM +0900, Masahiro Yamada wrote: > On Sat, May 3, 2025 at 7:54 AM Kees Cook <kees@kernel.org> wrote: > > > > v2: > > - switch from -include to -I with a -D gated include compiler-version.h > > v1: https://lore.kernel.org/lkml/20250501193839.work.525-kees@kernel.org/ > > > What do you think of my patch as a prerequisite? > https://lore.kernel.org/linux-kbuild/20250503084145.1994176-1-masahiroy@kernel.org/T/#u > Perhaps, can you implement this series more simply? > > My idea is to touch a single include/generated/global-rebuild.h > rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc. > > When the file is touched, the entire kernel source tree will be rebuilt. > This may rebuild more than needed (e.g. vdso) but I do not think > it is a big deal. This is roughly where I started when trying to implement this, but I didn't like the ergonomics of needing to scatter "touch" calls all over, which was especially difficult for targets that shared a build rule but may not all need to trigger a global rebuild. But what ultimately pushed me away from it was when I needed to notice if a non-built source file changed (the Clang .scl file), and I saw that I need to be dependency driven rather than target driven. (Though perhaps there is a way to address this with your global-rebuild.h?) As far as doing a full rebuild, if it had been available last week, I probably would have used it, but now given the work that Nicolas, you, and I have put into this, we have a viable way (I think) to make this more specific. It does end up being a waste of time/resources to rebuild stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that does add up when I'm iterating on something that keeps triggering a full rebuild. We already have to do the argument filtering for targets that don't want randstruct, etc, so why not capitalize on that and make the rebuild avoid those files too? So, I think the global-rebuild.h idea is a good one (though I think it should maybe be included in compiler-version.h just to avoid yet more compiler command line arguments), I'd really like to try to have the specific dependency-based way to get it done. I'll send a v3, and see what you think? -Kees -- Kees Cook
On Sun, May 4, 2025 at 2:37 AM Kees Cook <kees@kernel.org> wrote:
>
> On Sat, May 03, 2025 at 06:39:28PM +0900, Masahiro Yamada wrote:
> > On Sat, May 3, 2025 at 7:54 AM Kees Cook <kees@kernel.org> wrote:
> > >
> > > v2:
> > > - switch from -include to -I with a -D gated include compiler-version.h
> > > v1: https://lore.kernel.org/lkml/20250501193839.work.525-kees@kernel.org/
> >
> >
> > What do you think of my patch as a prerequisite?
> > https://lore.kernel.org/linux-kbuild/20250503084145.1994176-1-masahiroy@kernel.org/T/#u
> > Perhaps, can you implement this series more simply?
> >
> > My idea is to touch a single include/generated/global-rebuild.h
> > rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc.
> >
> > When the file is touched, the entire kernel source tree will be rebuilt.
> > This may rebuild more than needed (e.g. vdso) but I do not think
> > it is a big deal.
>
> This is roughly where I started when trying to implement this, but I
> didn't like the ergonomics of needing to scatter "touch" calls all over,
> which was especially difficult for targets that shared a build rule but
> may not all need to trigger a global rebuild. But what ultimately pushed
> me away from it was when I needed to notice if a non-built source file
> changed (the Clang .scl file), and I saw that I need to be dependency
> driven rather than target driven. (Though perhaps there is a way to
> address this with your global-rebuild.h?)
>
> As far as doing a full rebuild, if it had been available last week, I
> probably would have used it, but now given the work that Nicolas, you,
> and I have put into this, we have a viable way (I think) to make this
> more specific. It does end up being a waste of time/resources to rebuild
> stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that
> does add up when I'm iterating on something that keeps triggering a full
> rebuild. We already have to do the argument filtering for targets that
> don't want randstruct, etc, so why not capitalize on that and make the
> rebuild avoid those files too?
efi-stub, vdso are very small.
Unless this turns out to be painful, I prefer
a simpler implementation.
You will see how .scl file is handled.
See the below code:
diff --git a/Kbuild b/Kbuild
index f327ca86990c..85747239314c 100644
--- a/Kbuild
+++ b/Kbuild
@@ -67,10 +67,20 @@ targets += $(atomic-checks)
$(atomic-checks): $(obj)/.checked-%: include/linux/atomic/% FORCE
$(call if_changed,check_sha1)
+rebuild-$(CONFIG_GCC_PLUGINS) += $(addprefix
scripts/gcc-plugins/, $(GCC_PLUGIN))
+rebuild-$(CONFIG_RANDSTRUCT) += include/generated/randstruct_hash.h
+rebuild-$(CONFIG_UBSAN_INTEGER_WRAP) += scripts/integer-wrap-ignore.scl
+
+quiet_cmd_touch = TOUCH $@
+ cmd_touch = touch $@
+
+include/generated/global-rebuild.h: $(rebuild-y)
+ $(call cmd,touch)
+
# A phony target that depends on all the preparation targets
PHONY += prepare
-prepare: $(offsets-file) missing-syscalls $(atomic-checks)
+prepare: $(offsets-file) missing-syscalls $(atomic-checks)
include/generated/global-rebuild.h
@:
# Ordinary directory descending
diff --git a/Makefile b/Makefile
index b29cc321ffd9..f963a72b0761 100644
--- a/Makefile
+++ b/Makefile
@@ -558,7 +558,8 @@ USERINCLUDE := \
-I$(srctree)/include/uapi \
-I$(objtree)/include/generated/uapi \
-include $(srctree)/include/linux/compiler-version.h \
- -include $(srctree)/include/linux/kconfig.h
+ -include $(srctree)/include/linux/kconfig.h \
+ -include $(objtree)/include/generated/global-rebuild.h
# Use LINUXINCLUDE when you must reference the include/ directory.
# Needed to be compatible with the O= option
@@ -1250,6 +1251,12 @@ endif
include/config/kernel.release: FORCE
$(call filechk,kernel.release)
+quiet_cmd_touch = TOUCH $@
+ cmd_touch = touch $@
+
+include/generated/global-rebuild.h:
+ $(call cmd,touch)
+
# Additional helpers built in scripts/
# Carefully list dependencies so we do not try to build scripts twice
# in parallel
@@ -1266,6 +1273,7 @@ scripts: scripts_basic scripts_dtc
PHONY += prepare archprepare
archprepare: outputmakefile archheaders archscripts scripts
include/config/kernel.release \
+ include/generated/global-rebuild.h \
asm-generic $(version_h) include/generated/utsrelease.h \
include/generated/compile.h include/generated/autoconf.h \
include/generated/rustc_cfg remove-stale-files
diff --git a/arch/um/Makefile b/arch/um/Makefile
index 1d36a613aad8..f564a26c1364 100644
--- a/arch/um/Makefile
+++ b/arch/um/Makefile
@@ -73,7 +73,8 @@ USER_CFLAGS = $(patsubst
$(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include \
-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__ \
-include $(srctree)/include/linux/compiler-version.h \
- -include $(srctree)/include/linux/kconfig.h
+ -include $(srctree)/include/linux/kconfig.h \
+ -include $(objtree)/include/generated/global-rebuild.h
#This will adjust *FLAGS accordingly to the platform.
include $(srctree)/$(ARCH_DIR)/Makefile-os-Linux
--
Best Regards
Masahiro Yamada
On Fri, May 09, 2025 at 01:44:09AM +0900, Masahiro Yamada wrote: > On Sun, May 4, 2025 at 2:37 AM Kees Cook <kees@kernel.org> wrote: > > > > On Sat, May 03, 2025 at 06:39:28PM +0900, Masahiro Yamada wrote: > > > On Sat, May 3, 2025 at 7:54 AM Kees Cook <kees@kernel.org> wrote: > > > > > > > > v2: > > > > - switch from -include to -I with a -D gated include compiler-version.h > > > > v1: https://lore.kernel.org/lkml/20250501193839.work.525-kees@kernel.org/ > > > > > > > > > What do you think of my patch as a prerequisite? > > > https://lore.kernel.org/linux-kbuild/20250503084145.1994176-1-masahiroy@kernel.org/T/#u > > > Perhaps, can you implement this series more simply? > > > > > > My idea is to touch a single include/generated/global-rebuild.h > > > rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc. > > > > > > When the file is touched, the entire kernel source tree will be rebuilt. > > > This may rebuild more than needed (e.g. vdso) but I do not think > > > it is a big deal. > > > > This is roughly where I started when trying to implement this, but I > > didn't like the ergonomics of needing to scatter "touch" calls all over, > > which was especially difficult for targets that shared a build rule but > > may not all need to trigger a global rebuild. But what ultimately pushed > > me away from it was when I needed to notice if a non-built source file > > changed (the Clang .scl file), and I saw that I need to be dependency > > driven rather than target driven. (Though perhaps there is a way to > > address this with your global-rebuild.h?) > > > > As far as doing a full rebuild, if it had been available last week, I > > probably would have used it, but now given the work that Nicolas, you, > > and I have put into this, we have a viable way (I think) to make this > > more specific. It does end up being a waste of time/resources to rebuild > > stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that > > does add up when I'm iterating on something that keeps triggering a full > > rebuild. We already have to do the argument filtering for targets that > > don't want randstruct, etc, so why not capitalize on that and make the > > rebuild avoid those files too? > > > efi-stub, vdso are very small. > > Unless this turns out to be painful, I prefer > a simpler implementation. > > You will see how .scl file is handled. > > See the below code: > > > diff --git a/Kbuild b/Kbuild > index f327ca86990c..85747239314c 100644 > --- a/Kbuild > +++ b/Kbuild > @@ -67,10 +67,20 @@ targets += $(atomic-checks) > $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/% FORCE > $(call if_changed,check_sha1) > > +rebuild-$(CONFIG_GCC_PLUGINS) += $(addprefix > scripts/gcc-plugins/, $(GCC_PLUGIN)) > +rebuild-$(CONFIG_RANDSTRUCT) += include/generated/randstruct_hash.h These are in $(objtree) > +rebuild-$(CONFIG_UBSAN_INTEGER_WRAP) += scripts/integer-wrap-ignore.scl This is in $(srctree) > + > +quiet_cmd_touch = TOUCH $@ > + cmd_touch = touch $@ > + > +include/generated/global-rebuild.h: $(rebuild-y) > + $(call cmd,touch) Is this rule going to find the right versions of the dependencies? > --- a/Makefile > +++ b/Makefile > @@ -558,7 +558,8 @@ USERINCLUDE := \ > -I$(srctree)/include/uapi \ > -I$(objtree)/include/generated/uapi \ > -include $(srctree)/include/linux/compiler-version.h \ > - -include $(srctree)/include/linux/kconfig.h > + -include $(srctree)/include/linux/kconfig.h \ > + -include $(objtree)/include/generated/global-rebuild.h Instead of adding a new file, why not just touch compiler-version.h? But whatever the case, sure, I can live with this. :) -Kees -- Kees Cook
On Fri, May 9, 2025 at 1:56 AM Kees Cook <kees@kernel.org> wrote: > > On Fri, May 09, 2025 at 01:44:09AM +0900, Masahiro Yamada wrote: > > On Sun, May 4, 2025 at 2:37 AM Kees Cook <kees@kernel.org> wrote: > > > > > > On Sat, May 03, 2025 at 06:39:28PM +0900, Masahiro Yamada wrote: > > > > On Sat, May 3, 2025 at 7:54 AM Kees Cook <kees@kernel.org> wrote: > > > > > > > > > > v2: > > > > > - switch from -include to -I with a -D gated include compiler-version.h > > > > > v1: https://lore.kernel.org/lkml/20250501193839.work.525-kees@kernel.org/ > > > > > > > > > > > > What do you think of my patch as a prerequisite? > > > > https://lore.kernel.org/linux-kbuild/20250503084145.1994176-1-masahiroy@kernel.org/T/#u > > > > Perhaps, can you implement this series more simply? > > > > > > > > My idea is to touch a single include/generated/global-rebuild.h > > > > rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc. > > > > > > > > When the file is touched, the entire kernel source tree will be rebuilt. > > > > This may rebuild more than needed (e.g. vdso) but I do not think > > > > it is a big deal. > > > > > > This is roughly where I started when trying to implement this, but I > > > didn't like the ergonomics of needing to scatter "touch" calls all over, > > > which was especially difficult for targets that shared a build rule but > > > may not all need to trigger a global rebuild. But what ultimately pushed > > > me away from it was when I needed to notice if a non-built source file > > > changed (the Clang .scl file), and I saw that I need to be dependency > > > driven rather than target driven. (Though perhaps there is a way to > > > address this with your global-rebuild.h?) > > > > > > As far as doing a full rebuild, if it had been available last week, I > > > probably would have used it, but now given the work that Nicolas, you, > > > and I have put into this, we have a viable way (I think) to make this > > > more specific. It does end up being a waste of time/resources to rebuild > > > stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that > > > does add up when I'm iterating on something that keeps triggering a full > > > rebuild. We already have to do the argument filtering for targets that > > > don't want randstruct, etc, so why not capitalize on that and make the > > > rebuild avoid those files too? > > > > > > efi-stub, vdso are very small. > > > > Unless this turns out to be painful, I prefer > > a simpler implementation. > > > > You will see how .scl file is handled. > > > > See the below code: > > > > > > diff --git a/Kbuild b/Kbuild > > index f327ca86990c..85747239314c 100644 > > --- a/Kbuild > > +++ b/Kbuild > > @@ -67,10 +67,20 @@ targets += $(atomic-checks) > > $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/% FORCE > > $(call if_changed,check_sha1) > > > > +rebuild-$(CONFIG_GCC_PLUGINS) += $(addprefix > > scripts/gcc-plugins/, $(GCC_PLUGIN)) > > +rebuild-$(CONFIG_RANDSTRUCT) += include/generated/randstruct_hash.h > > These are in $(objtree) Yes. > > +rebuild-$(CONFIG_UBSAN_INTEGER_WRAP) += scripts/integer-wrap-ignore.scl > > This is in $(srctree) Yes. > > + > > +quiet_cmd_touch = TOUCH $@ > > + cmd_touch = touch $@ > > + > > +include/generated/global-rebuild.h: $(rebuild-y) > > + $(call cmd,touch) > > Is this rule going to find the right versions of the dependencies? I think so, but please test it. > > --- a/Makefile > > +++ b/Makefile > > @@ -558,7 +558,8 @@ USERINCLUDE := \ > > -I$(srctree)/include/uapi \ > > -I$(objtree)/include/generated/uapi \ > > -include $(srctree)/include/linux/compiler-version.h \ > > - -include $(srctree)/include/linux/kconfig.h > > + -include $(srctree)/include/linux/kconfig.h \ > > + -include $(objtree)/include/generated/global-rebuild.h > > Instead of adding a new file, why not just touch compiler-version.h? Because the compiler-version.h is in $(srctree), which might be in the read-only file system. > But whatever the case, sure, I can live with this. :) > > -Kees > > -- > Kees Cook -- Best Regards Masahiro Yamada
On Fri, May 09, 2025 at 08:13:18AM +0900, Masahiro Yamada wrote: > On Fri, May 9, 2025 at 1:56 AM Kees Cook <kees@kernel.org> wrote: > > > > On Fri, May 09, 2025 at 01:44:09AM +0900, Masahiro Yamada wrote: > > > On Sun, May 4, 2025 at 2:37 AM Kees Cook <kees@kernel.org> wrote: > > > > > > > > On Sat, May 03, 2025 at 06:39:28PM +0900, Masahiro Yamada wrote: > > > > > On Sat, May 3, 2025 at 7:54 AM Kees Cook <kees@kernel.org> wrote: > > > > > > > > > > > > v2: > > > > > > - switch from -include to -I with a -D gated include compiler-version.h > > > > > > v1: https://lore.kernel.org/lkml/20250501193839.work.525-kees@kernel.org/ > > > > > > > > > > > > > > > What do you think of my patch as a prerequisite? > > > > > https://lore.kernel.org/linux-kbuild/20250503084145.1994176-1-masahiroy@kernel.org/T/#u > > > > > Perhaps, can you implement this series more simply? > > > > > > > > > > My idea is to touch a single include/generated/global-rebuild.h > > > > > rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc. > > > > > > > > > > When the file is touched, the entire kernel source tree will be rebuilt. > > > > > This may rebuild more than needed (e.g. vdso) but I do not think > > > > > it is a big deal. > > > > > > > > This is roughly where I started when trying to implement this, but I > > > > didn't like the ergonomics of needing to scatter "touch" calls all over, > > > > which was especially difficult for targets that shared a build rule but > > > > may not all need to trigger a global rebuild. But what ultimately pushed > > > > me away from it was when I needed to notice if a non-built source file > > > > changed (the Clang .scl file), and I saw that I need to be dependency > > > > driven rather than target driven. (Though perhaps there is a way to > > > > address this with your global-rebuild.h?) > > > > > > > > As far as doing a full rebuild, if it had been available last week, I > > > > probably would have used it, but now given the work that Nicolas, you, > > > > and I have put into this, we have a viable way (I think) to make this > > > > more specific. It does end up being a waste of time/resources to rebuild > > > > stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that > > > > does add up when I'm iterating on something that keeps triggering a full > > > > rebuild. We already have to do the argument filtering for targets that > > > > don't want randstruct, etc, so why not capitalize on that and make the > > > > rebuild avoid those files too? > > > > > > > > > efi-stub, vdso are very small. > > > > > > Unless this turns out to be painful, I prefer > > > a simpler implementation. > > > > > > You will see how .scl file is handled. > > > > > > See the below code: > > > > > > > > > diff --git a/Kbuild b/Kbuild > > > index f327ca86990c..85747239314c 100644 > > > --- a/Kbuild > > > +++ b/Kbuild > > > @@ -67,10 +67,20 @@ targets += $(atomic-checks) > > > $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/% FORCE > > > $(call if_changed,check_sha1) > > > > > > +rebuild-$(CONFIG_GCC_PLUGINS) += $(addprefix > > > scripts/gcc-plugins/, $(GCC_PLUGIN)) > > > +rebuild-$(CONFIG_RANDSTRUCT) += include/generated/randstruct_hash.h > > > > These are in $(objtree) > > Yes. > > > > +rebuild-$(CONFIG_UBSAN_INTEGER_WRAP) += scripts/integer-wrap-ignore.scl > > > > This is in $(srctree) > > Yes. > > > > + > > > +quiet_cmd_touch = TOUCH $@ > > > + cmd_touch = touch $@ > > > + > > > +include/generated/global-rebuild.h: $(rebuild-y) > > > + $(call cmd,touch) > > > > Is this rule going to find the right versions of the dependencies? > > I think so, but please test it. The patch was white-space damaged and wrapped, but I rebuilt it manually and it mostly works. There still seems to be some ordering issues, as some stuff gets rebuilt on a record build: # Clean the tree and pick an "everything" build $ make O=gcc-test clean allmodconfig -s # Make a target normally $ make O=gcc-test kernel/seccomp.o -s # Touch a gcc plugin that was in .config $ touch scripts/gcc-plugins/stackleak_plugin.c # Build and a full rebuild is triggered (good) $ make O=gcc-test kernel/seccomp.o make[1]: Entering directory '/srv/code/gcc-test' GEN Makefile DESCEND objtool HOSTCXX scripts/gcc-plugins/stackleak_plugin.so INSTALL libsubcmd_headers TOUCH include/generated/global-rebuild.h CC kernel/bounds.s CC arch/x86/kernel/asm-offsets.s CALL ../scripts/checksyscalls.sh CC kernel/seccomp.o make[1]: Leaving directory '/srv/code/gcc-test' # Build again, but more stuff gets built $ make O=gcc-test kernel/seccomp.o make[1]: Entering directory '/srv/code/gcc-test' GEN Makefile DESCEND objtool CC scripts/mod/empty.o CC scripts/mod/devicetable-offsets.s INSTALL libsubcmd_headers MKELF scripts/mod/elfconfig.h HOSTCC scripts/mod/modpost.o HOSTCC scripts/mod/sumversion.o HOSTCC scripts/mod/symsearch.o HOSTCC scripts/mod/file2alias.o HOSTLD scripts/mod/modpost CALL ../scripts/checksyscalls.sh make[1]: Leaving directory '/srv/code/gcc-test' # Third time finally everything is stable $ hmake O=gcc-test kernel/seccomp.o make[1]: Entering directory '/srv/code/gcc-test' GEN Makefile DESCEND objtool CALL ../scripts/checksyscalls.sh INSTALL libsubcmd_headers make[1]: Leaving directory '/srv/code/gcc-test' Note that scripts/mod/* gets rebuilt on the second rebuild. -- Kees Cook
On Fri, May 9, 2025 at 8:59 AM Kees Cook <kees@kernel.org> wrote: > > On Fri, May 09, 2025 at 08:13:18AM +0900, Masahiro Yamada wrote: > > On Fri, May 9, 2025 at 1:56 AM Kees Cook <kees@kernel.org> wrote: > > > > > > On Fri, May 09, 2025 at 01:44:09AM +0900, Masahiro Yamada wrote: > > > > On Sun, May 4, 2025 at 2:37 AM Kees Cook <kees@kernel.org> wrote: > > > > > > > > > > On Sat, May 03, 2025 at 06:39:28PM +0900, Masahiro Yamada wrote: > > > > > > On Sat, May 3, 2025 at 7:54 AM Kees Cook <kees@kernel.org> wrote: > > > > > > > > > > > > > > v2: > > > > > > > - switch from -include to -I with a -D gated include compiler-version.h > > > > > > > v1: https://lore.kernel.org/lkml/20250501193839.work.525-kees@kernel.org/ > > > > > > > > > > > > > > > > > > What do you think of my patch as a prerequisite? > > > > > > https://lore.kernel.org/linux-kbuild/20250503084145.1994176-1-masahiroy@kernel.org/T/#u > > > > > > Perhaps, can you implement this series more simply? > > > > > > > > > > > > My idea is to touch a single include/generated/global-rebuild.h > > > > > > rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc. > > > > > > > > > > > > When the file is touched, the entire kernel source tree will be rebuilt. > > > > > > This may rebuild more than needed (e.g. vdso) but I do not think > > > > > > it is a big deal. > > > > > > > > > > This is roughly where I started when trying to implement this, but I > > > > > didn't like the ergonomics of needing to scatter "touch" calls all over, > > > > > which was especially difficult for targets that shared a build rule but > > > > > may not all need to trigger a global rebuild. But what ultimately pushed > > > > > me away from it was when I needed to notice if a non-built source file > > > > > changed (the Clang .scl file), and I saw that I need to be dependency > > > > > driven rather than target driven. (Though perhaps there is a way to > > > > > address this with your global-rebuild.h?) > > > > > > > > > > As far as doing a full rebuild, if it had been available last week, I > > > > > probably would have used it, but now given the work that Nicolas, you, > > > > > and I have put into this, we have a viable way (I think) to make this > > > > > more specific. It does end up being a waste of time/resources to rebuild > > > > > stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that > > > > > does add up when I'm iterating on something that keeps triggering a full > > > > > rebuild. We already have to do the argument filtering for targets that > > > > > don't want randstruct, etc, so why not capitalize on that and make the > > > > > rebuild avoid those files too? > > > > > > > > > > > > efi-stub, vdso are very small. > > > > > > > > Unless this turns out to be painful, I prefer > > > > a simpler implementation. > > > > > > > > You will see how .scl file is handled. > > > > > > > > See the below code: > > > > > > > > > > > > diff --git a/Kbuild b/Kbuild > > > > index f327ca86990c..85747239314c 100644 > > > > --- a/Kbuild > > > > +++ b/Kbuild > > > > @@ -67,10 +67,20 @@ targets += $(atomic-checks) > > > > $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/% FORCE > > > > $(call if_changed,check_sha1) > > > > > > > > +rebuild-$(CONFIG_GCC_PLUGINS) += $(addprefix > > > > scripts/gcc-plugins/, $(GCC_PLUGIN)) > > > > +rebuild-$(CONFIG_RANDSTRUCT) += include/generated/randstruct_hash.h > > > > > > These are in $(objtree) > > > > Yes. > > > > > > +rebuild-$(CONFIG_UBSAN_INTEGER_WRAP) += scripts/integer-wrap-ignore.scl > > > > > > This is in $(srctree) > > > > Yes. > > > > > > + > > > > +quiet_cmd_touch = TOUCH $@ > > > > + cmd_touch = touch $@ > > > > + > > > > +include/generated/global-rebuild.h: $(rebuild-y) > > > > + $(call cmd,touch) > > > > > > Is this rule going to find the right versions of the dependencies? > > > > I think so, but please test it. > > The patch was white-space damaged and wrapped, but I rebuilt it manually > and it mostly works. There still seems to be some ordering issues, as > some stuff gets rebuilt on a record build: > > # Clean the tree and pick an "everything" build > $ make O=gcc-test clean allmodconfig -s > > # Make a target normally > $ make O=gcc-test kernel/seccomp.o -s > > # Touch a gcc plugin that was in .config > $ touch scripts/gcc-plugins/stackleak_plugin.c > > # Build and a full rebuild is triggered (good) > $ make O=gcc-test kernel/seccomp.o > make[1]: Entering directory '/srv/code/gcc-test' > GEN Makefile > DESCEND objtool > HOSTCXX scripts/gcc-plugins/stackleak_plugin.so > INSTALL libsubcmd_headers > TOUCH include/generated/global-rebuild.h > CC kernel/bounds.s > CC arch/x86/kernel/asm-offsets.s > CALL ../scripts/checksyscalls.sh > CC kernel/seccomp.o > make[1]: Leaving directory '/srv/code/gcc-test' > > # Build again, but more stuff gets built > $ make O=gcc-test kernel/seccomp.o > make[1]: Entering directory '/srv/code/gcc-test' > GEN Makefile > DESCEND objtool > CC scripts/mod/empty.o > CC scripts/mod/devicetable-offsets.s > INSTALL libsubcmd_headers > MKELF scripts/mod/elfconfig.h > HOSTCC scripts/mod/modpost.o > HOSTCC scripts/mod/sumversion.o > HOSTCC scripts/mod/symsearch.o > HOSTCC scripts/mod/file2alias.o > HOSTLD scripts/mod/modpost > CALL ../scripts/checksyscalls.sh > make[1]: Leaving directory '/srv/code/gcc-test' > > # Third time finally everything is stable > $ hmake O=gcc-test kernel/seccomp.o > make[1]: Entering directory '/srv/code/gcc-test' > GEN Makefile > DESCEND objtool > CALL ../scripts/checksyscalls.sh > INSTALL libsubcmd_headers > make[1]: Leaving directory '/srv/code/gcc-test' > > > Note that scripts/mod/* gets rebuilt on the second rebuild. Hmm. OK, my code did not work. I accept your patch set (although I am not a big fan of the added complexity...) Could you move the normalize_path macro to scripts/Kbuild? -- Best Regards Masahiro Yamada
© 2016 - 2026 Red Hat, Inc.