1. Patchew
  2. linux
  3. View series

[PATCH] kbuild: Support separating source and output of the external module

Jing Leng posted 1 patch 2 months, 2 weeks ago 
Documentation/kbuild/modules.rst | 45 +++++++++++++++++++++++---------
scripts/Makefile.build           | 28 ++++++++++----------
scripts/Makefile.clean           |  2 +-
scripts/Makefile.host            |  6 ++---
scripts/Makefile.lib             |  2 +-
scripts/Makefile.modpost         |  2 +-
6 files changed, 52 insertions(+), 33 deletions(-)
[PATCH] kbuild: Support separating source and output of the external module
Posted by Jing Leng 2 months, 2 weeks ago 
Revert commit "9a0ebe5011f4", originally when compiling external modules,
the compiled output can be separated from the source code with the
following command:
    make -C <kernel_src> O=<kernel_out> src=<module_src> M=<module_out>
It is used to https://github.com/lengjingzju/cbuild-ng (inc.mod.mk).

But "src" is an internal variable, so this patch adds a variable "S"
which is specifically used to change the default value of "src".

Fixes: 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")
Signed-off-by: Jing Leng <3090101217@zju.edu.cn>
---
 Documentation/kbuild/modules.rst | 45 +++++++++++++++++++++++---------
 scripts/Makefile.build           | 28 ++++++++++----------
 scripts/Makefile.clean           |  2 +-
 scripts/Makefile.host            |  6 ++---
 scripts/Makefile.lib             |  2 +-
 scripts/Makefile.modpost         |  2 +-
 6 files changed, 52 insertions(+), 33 deletions(-)

diff --git a/Documentation/kbuild/modules.rst b/Documentation/kbuild/modules.rst
index 131863142cbb..1c4d3810b1dd 100644
--- a/Documentation/kbuild/modules.rst
+++ b/Documentation/kbuild/modules.rst
@@ -72,14 +72,23 @@ executed to make module versioning work.
 2.1 Command Syntax
 ==================
 
-	The command to build an external module is::
+	The full command to build an external module is::
 
-		$ make -C <path_to_kernel_src> M=$PWD
+		$ make -C <kernel_src> O=<kernel_out> S=<module_src> M=<module_out>
 
-	The kbuild system knows that an external module is being built
-	due to the "M=<dir>" option given in the command.
+	If <kernel_out> is the same as <kernel_src>, "O=" can be omitted::
 
-	To build against the running kernel use::
+		$ make -C <kernel_src> S=<module_src> M=<module_out>
+
+	If <module_out> is the same as <module_src>, "S=" can be omitted::
+
+		$ make -C <kernel_src> O=<kernel_out> M=<module_src>
+
+	If both of the above are satisfied, the command is::
+
+		$ make -C <kernel_src> M=<module_src>
+
+	So to build against the running kernel can use::
 
 		$ make -C /lib/modules/`uname -r`/build M=$PWD
 
@@ -91,20 +100,29 @@ executed to make module versioning work.
 2.2 Options
 ===========
 
-	($KDIR refers to the path of the kernel source directory.)
-
-	make -C $KDIR M=$PWD
+	make -C <kernel_src> O=<kernel_out> S=<module_src> M=<module_out>
 
-	-C $KDIR
+	-C <kernel_src>
 		The directory where the kernel source is located.
 		"make" will actually change to the specified directory
 		when executing and will change back when finished.
 
-	M=$PWD
+	O=<kernel_out>
+		The value given to "O" is the absolute path of the
+		directory where the kernel output (.config file) is located.
+		If <kernel_out> is the same as <kernel_src>,
+		"O=<kernel_out>" can be omitted.
+
+	S=<module_src>
+		The value given to "S" is the absolute path of the
+		directory where the external module source (kbuild file)
+		is located. If <module_out> is the same as <module_src>,
+		"S=<module_src>" can be omitted.
+
+	M=<module_out>
 		Informs kbuild that an external module is being built.
 		The value given to "M" is the absolute path of the
-		directory where the external module (kbuild file) is
-		located.
+		directory where the external module output is located.
 
 2.3 Targets
 ===========
@@ -112,7 +130,7 @@ executed to make module versioning work.
 	When building an external module, only a subset of the "make"
 	targets are available.
 
-	make -C $KDIR M=$PWD [target]
+	make -C <kernel_src> O=<kernel_out> S=<module_src> M=<module_out> [target]
 
 	The default will build the module(s) located in the current
 	directory, so a target does not need to be specified. All
@@ -151,6 +169,7 @@ executed to make module versioning work.
 		make -C $KDIR M=$PWD foo.ko
 		make -C $KDIR M=$PWD ./
 
+	($KDIR refers to the path of the kernel source directory.)
 
 3. Creating a Kbuild File for an External Module
 ================================================
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index a5ac8ed1936f..44b38cdb76df 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -3,7 +3,7 @@
 # Building
 # ==========================================================================
 
-src := $(if $(VPATH),$(VPATH)/)$(obj)
+src := $(if $(S),$(S),$(if $(VPATH),$(VPATH)/)$(obj))
 
 PHONY := $(obj)/
 $(obj)/:
@@ -113,13 +113,13 @@ endif
 quiet_cmd_cc_s_c = CC $(quiet_modtag)  $@
       cmd_cc_s_c = $(CC) $(filter-out $(DEBUG_CFLAGS) $(CC_FLAGS_LTO), $(c_flags)) -fverbose-asm -S -o $@ $<
 
-$(obj)/%.s: $(obj)/%.c FORCE
+$(obj)/%.s: $(src)/%.c FORCE
 	$(call if_changed_dep,cc_s_c)
 
 quiet_cmd_cpp_i_c = CPP $(quiet_modtag) $@
 cmd_cpp_i_c       = $(CPP) $(c_flags) -o $@ $<
 
-$(obj)/%.i: $(obj)/%.c FORCE
+$(obj)/%.i: $(src)/%.c FORCE
 	$(call if_changed_dep,cpp_i_c)
 
 genksyms = scripts/genksyms/genksyms		\
@@ -133,7 +133,7 @@ cmd_gensymtypes_c = $(CPP) -D__GENKSYMS__ $(c_flags) $< | $(genksyms)
 quiet_cmd_cc_symtypes_c = SYM $(quiet_modtag) $@
       cmd_cc_symtypes_c = $(call cmd_gensymtypes_c,true,$@) >/dev/null
 
-$(obj)/%.symtypes : $(obj)/%.c FORCE
+$(obj)/%.symtypes : $(src)/%.c FORCE
 	$(call cmd,cc_symtypes_c)
 
 # LLVM assembly
@@ -141,7 +141,7 @@ $(obj)/%.symtypes : $(obj)/%.c FORCE
 quiet_cmd_cc_ll_c = CC $(quiet_modtag)  $@
       cmd_cc_ll_c = $(CC) $(c_flags) -emit-llvm -S -fno-discard-value-names -o $@ $<
 
-$(obj)/%.ll: $(obj)/%.c FORCE
+$(obj)/%.ll: $(src)/%.c FORCE
 	$(call if_changed_dep,cc_ll_c)
 
 # C (.c) files
@@ -240,7 +240,7 @@ define rule_as_o_S
 endef
 
 # Built-in and composite module parts
-$(obj)/%.o: $(obj)/%.c $(recordmcount_source) FORCE
+$(obj)/%.o: $(src)/%.c $(recordmcount_source) FORCE
 	$(call if_changed_rule,cc_o_c)
 	$(call cmd,force_checksrc)
 
@@ -257,7 +257,7 @@ quiet_cmd_cc_lst_c = MKLST   $@
 		     $(CONFIG_SHELL) $(srctree)/scripts/makelst $*.o \
 				     System.map $(OBJDUMP) > $@
 
-$(obj)/%.lst: $(obj)/%.c FORCE
+$(obj)/%.lst: $(src)/%.c FORCE
 	$(call if_changed_dep,cc_lst_c)
 
 # Compile Rust sources (.rs)
@@ -290,7 +290,7 @@ rust_common_cmd = \
 quiet_cmd_rustc_o_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
       cmd_rustc_o_rs = $(rust_common_cmd) --emit=obj=$@ $<
 
-$(obj)/%.o: $(obj)/%.rs FORCE
+$(obj)/%.o: $(src)/%.rs FORCE
 	+$(call if_changed_dep,rustc_o_rs)
 
 quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
@@ -298,19 +298,19 @@ quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
 	$(rust_common_cmd) -Zunpretty=expanded $< >$@; \
 	command -v $(RUSTFMT) >/dev/null && $(RUSTFMT) $@
 
-$(obj)/%.rsi: $(obj)/%.rs FORCE
+$(obj)/%.rsi: $(src)/%.rs FORCE
 	+$(call if_changed_dep,rustc_rsi_rs)
 
 quiet_cmd_rustc_s_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
       cmd_rustc_s_rs = $(rust_common_cmd) --emit=asm=$@ $<
 
-$(obj)/%.s: $(obj)/%.rs FORCE
+$(obj)/%.s: $(src)/%.rs FORCE
 	+$(call if_changed_dep,rustc_s_rs)
 
 quiet_cmd_rustc_ll_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@
       cmd_rustc_ll_rs = $(rust_common_cmd) --emit=llvm-ir=$@ $<
 
-$(obj)/%.ll: $(obj)/%.rs FORCE
+$(obj)/%.ll: $(src)/%.rs FORCE
 	+$(call if_changed_dep,rustc_ll_rs)
 
 # Compile assembler sources (.S)
@@ -336,14 +336,14 @@ cmd_gensymtypes_S =                                                         \
 quiet_cmd_cc_symtypes_S = SYM $(quiet_modtag) $@
       cmd_cc_symtypes_S = $(call cmd_gensymtypes_S,true,$@) >/dev/null
 
-$(obj)/%.symtypes : $(obj)/%.S FORCE
+$(obj)/%.symtypes : $(src)/%.S FORCE
 	$(call cmd,cc_symtypes_S)
 
 
 quiet_cmd_cpp_s_S = CPP $(quiet_modtag) $@
 cmd_cpp_s_S       = $(CPP) $(a_flags) -o $@ $<
 
-$(obj)/%.s: $(obj)/%.S FORCE
+$(obj)/%.s: $(src)/%.S FORCE
 	$(call if_changed_dep,cpp_s_S)
 
 quiet_cmd_as_o_S = AS $(quiet_modtag)  $@
@@ -358,7 +358,7 @@ cmd_gen_symversions_S = $(call gen_symversions,S)
 
 endif
 
-$(obj)/%.o: $(obj)/%.S FORCE
+$(obj)/%.o: $(src)/%.S FORCE
 	$(call if_changed_rule,as_o_S)
 
 targets += $(filter-out $(subdir-builtin), $(real-obj-y))
diff --git a/scripts/Makefile.clean b/scripts/Makefile.clean
index 4fcfab40ed61..6ce7514aefc8 100644
--- a/scripts/Makefile.clean
+++ b/scripts/Makefile.clean
@@ -3,7 +3,7 @@
 # Cleaning up
 # ==========================================================================
 
-src := $(if $(VPATH),$(VPATH)/)$(obj)
+src := $(if $(S),$(S),$(if $(VPATH),$(VPATH)/)$(obj))
 
 PHONY := __clean
 __clean:
diff --git a/scripts/Makefile.host b/scripts/Makefile.host
index e85be7721a48..3c17e6ba421c 100644
--- a/scripts/Makefile.host
+++ b/scripts/Makefile.host
@@ -112,7 +112,7 @@ endif
 quiet_cmd_host-csingle 	= HOSTCC  $@
       cmd_host-csingle	= $(HOSTCC) $(hostc_flags) $(KBUILD_HOSTLDFLAGS) -o $@ $< \
 		$(KBUILD_HOSTLDLIBS) $(HOSTLDLIBS_$(target-stem))
-$(host-csingle): $(obj)/%: $(obj)/%.c FORCE
+$(host-csingle): $(obj)/%: $(src)/%.c FORCE
 	$(call if_changed_dep,host-csingle)
 
 # Link an executable based on list of .o files, all plain c
@@ -129,7 +129,7 @@ $(call multi_depend, $(host-cmulti), , -objs)
 # host-cobjs -> .o
 quiet_cmd_host-cobjs	= HOSTCC  $@
       cmd_host-cobjs	= $(HOSTCC) $(hostc_flags) -c -o $@ $<
-$(host-cobjs): $(obj)/%.o: $(obj)/%.c FORCE
+$(host-cobjs): $(obj)/%.o: $(src)/%.c FORCE
 	$(call if_changed_dep,host-cobjs)
 
 # Link an executable based on list of .o files, a mixture of .c and .cc
@@ -146,7 +146,7 @@ $(call multi_depend, $(host-cxxmulti), , -objs -cxxobjs)
 # Create .o file from a single .cc (C++) file
 quiet_cmd_host-cxxobjs	= HOSTCXX $@
       cmd_host-cxxobjs	= $(HOSTCXX) $(hostcxx_flags) -c -o $@ $<
-$(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE
+$(host-cxxobjs): $(obj)/%.o: $(src)/%.cc FORCE
 	$(call if_changed_dep,host-cxxobjs)
 
 # Create executable from a single Rust crate (which may consist of
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 207325eaf1d1..6f961aa5790d 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -430,7 +430,7 @@ DT_TMP_SCHEMA := $(objtree)/$(DT_BINDING_DIR)/processed-schema.json
 dtb-check-enabled = $(if $(filter %.dtb, $@),y)
 endif
 
-$(obj)/%.dtb: $(obj)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
+$(obj)/%.dtb: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
 	$(call if_changed_dep,dtc)
 
 $(obj)/%.dtbo: $(src)/%.dtso $(DTC) FORCE
diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost
index 44936ebad161..12afe2acbe05 100644
--- a/scripts/Makefile.modpost
+++ b/scripts/Makefile.modpost
@@ -112,7 +112,7 @@ else
 
 # set src + obj - they may be used in the modules's Makefile
 obj := $(KBUILD_EXTMOD)
-src := $(if $(VPATH),$(VPATH)/)$(obj)
+src := $(if $(S),$(S),$(if $(VPATH),$(VPATH)/)$(obj))
 
 # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS
 include $(kbuild-file)
-- 
2.25.1
Re: [PATCH] kbuild: Support separating source and output of the external module
Posted by kernel test robot 2 months, 2 weeks ago 
Hi Jing,

kernel test robot noticed the following build errors:

[auto build test ERROR on masahiroy-kbuild/fixes]
[also build test ERROR on linus/master v6.11-rc7]
[cannot apply to masahiroy-kbuild/for-next next-20240912]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Jing-Leng/kbuild-Support-separating-source-and-output-of-the-external-module/20240911-192242
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git fixes
patch link:    https://lore.kernel.org/r/20240911112017.14427-1-3090101217%40zju.edu.cn
patch subject: [PATCH] kbuild: Support separating source and output of the external module
config: x86_64-kexec (attached as .config)
compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240912/202409121947.7NkNsyuC-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202409121947.7NkNsyuC-lkp@intel.com/

All errors (new ones prefixed by >>):

>> make[3]: *** No rule to make target 'scripts/kconfig/lexer.lex.c', needed by 'scripts/kconfig/lexer.lex.o'.
>> make[3]: *** No rule to make target 'scripts/kconfig/parser.tab.c', needed by 'scripts/kconfig/parser.tab.o'.
   make[3]: Target 'oldconfig' not remade because of errors.
   make[2]: *** [Makefile:680: oldconfig] Error 2
   make[1]: *** [Makefile:224: __sub-make] Error 2
   make[1]: Target 'oldconfig' not remade because of errors.
   make: *** [Makefile:224: __sub-make] Error 2
   make: Target 'oldconfig' not remade because of errors.
--
>> make[3]: *** No rule to make target 'scripts/kconfig/lexer.lex.c', needed by 'scripts/kconfig/lexer.lex.o'.
>> make[3]: *** No rule to make target 'scripts/kconfig/parser.tab.c', needed by 'scripts/kconfig/parser.tab.o'.
   make[3]: Target 'olddefconfig' not remade because of errors.
   make[2]: *** [Makefile:680: olddefconfig] Error 2
   make[1]: *** [Makefile:224: __sub-make] Error 2
   make[1]: Target 'olddefconfig' not remade because of errors.
   make: *** [Makefile:224: __sub-make] Error 2
   make: Target 'olddefconfig' not remade because of errors.

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Re: [PATCH] kbuild: Support separating source and output of the external module
Posted by Masahiro Yamada 2 months, 2 weeks ago 
On Wed, Sep 11, 2024 at 8:20 PM Jing Leng <3090101217@zju.edu.cn> wrote:
>
> Revert commit "9a0ebe5011f4", originally when compiling external modules,
> the compiled output can be separated from the source code with the
> following command:
>     make -C <kernel_src> O=<kernel_out> src=<module_src> M=<module_out>
> It is used to https://github.com/lengjingzju/cbuild-ng (inc.mod.mk).
>
> But "src" is an internal variable, so this patch adds a variable "S"
> which is specifically used to change the default value of "src".
>
> Fixes: 9a0ebe5011f4 ("kbuild: use $(obj)/ instead of $(src)/ for common pattern rules")
> Signed-off-by: Jing Leng <3090101217@zju.edu.cn>


NACK.

Always sign, every time I receive a hack like this over and  over again.


-- 
Best Regards
Masahiro Yamada

Patchew 2.1-devel-b67e4c2

© 2016 - 2024 Red Hat, Inc.