kbuild: get lib-y objects back to static library

[PATCH] kbuild: get lib-y objects back to static library

Posted by Masahiro Yamada 2 years, 3 months ago

Revert the following two commits:

 - 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y")
 - 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably")

Now, lib-y is back to static library again, and the link order is
consistent w/wo CONFIG_MODULES.

Since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by
modpost"), .vmlinux.export.c contains references to exported symbols.
If a symbol in a lib-y object is exported, that object is always linked
even without any explicit user in vmlinux.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
---

 Makefile | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/Makefile b/Makefile
index 87a9eef3fb4b..71e9c65d9eae 100644
--- a/Makefile
+++ b/Makefile
@@ -1113,12 +1113,7 @@ export ARCH_DRIVERS	:= $(drivers-y) $(drivers-m)
 # Externally visible symbols (used by link-vmlinux.sh)
 
 KBUILD_VMLINUX_OBJS := ./built-in.a
-ifdef CONFIG_MODULES
-KBUILD_VMLINUX_OBJS += $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
-KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
-else
 KBUILD_VMLINUX_LIBS := $(patsubst %/,%/lib.a, $(libs-y))
-endif
 
 export KBUILD_VMLINUX_LIBS
 export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
-- 
2.39.2

Re: [PATCH] kbuild: get lib-y objects back to static library

Posted by kernel test robot 2 years, 3 months ago

Hi Masahiro,

kernel test robot noticed the following build errors:

[auto build test ERROR on masahiroy-kbuild/for-next]
[also build test ERROR on masahiroy-kbuild/fixes linus/master v6.5-rc7 next-20230825]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Masahiro-Yamada/kbuild-get-lib-y-objects-back-to-static-library/20230823-201119
base:   https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git for-next
patch link:    https://lore.kernel.org/r/20230823120816.824352-1-masahiroy%40kernel.org
patch subject: [PATCH] kbuild: get lib-y objects back to static library
config: m68k-randconfig-r011-20230823 (https://download.01.org/0day-ci/archive/20230826/202308261038.mYLTBV2P-lkp@intel.com/config)
compiler: m68k-linux-gcc (GCC) 13.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230826/202308261038.mYLTBV2P-lkp@intel.com/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202308261038.mYLTBV2P-lkp@intel.com/

All errors (new ones prefixed by >>, old ones prefixed by <<):

WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-npgtech.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-odroid.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pctv-sedna.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pine64.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pinnacle-color.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pinnacle-grey.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pinnacle-pctv-hd.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pixelview-002t.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pixelview-mk12.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pixelview-new.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pixelview.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-powercolor-real-angel.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-proteus-2309.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-purpletv.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-pv951.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-rc6-mce.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-real-audio-220-32-keys.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-reddo.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-snapstream-firefly.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-streamzap.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-su3000.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-tanix-tx3mini.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-tanix-tx5max.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-tbs-nec.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-technisat-ts35.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-technisat-usb2.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-terratec-cinergy-c-pci.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-terratec-cinergy-s2-hd.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-terratec-cinergy-xs.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-terratec-slim-2.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-terratec-slim.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-tevii-nec.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-tivo.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-total-media-in-hand-02.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-total-media-in-hand.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-trekstor.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-tt-1500.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-twinhan1027.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-twinhan-dtv-cab-ci.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-vega-s9x.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-videomate-m1f.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-videomate-s350.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-videomate-tv-pvr.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-videostrong-kii-pro.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-wetek-hub.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-wetek-play2.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-winfast.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-winfast-usbii-deluxe.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-x96max.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-xbox-360.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-xbox-dvd.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/rc/keymaps/rc-zx-irdec.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/dvb-frontends/au8522_decoder.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/dvb-frontends/mb86a16.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/v4l2-core/v4l2-async.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/media/v4l2-core/v4l2-fwnode.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/watchdog/menz69_wdt.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/mmc/core/mmc_core.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/mmc/core/sdio_uart.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-belkin.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-vivaldi-common.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-keytouch.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-lenovo.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-mf.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-microsoft.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-monterey.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-ortek.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-pl.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-primax.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-sjoy.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-speedlink.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-topseed.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-xinmo.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hid/hid-zydacron.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/staging/greybus/gb-bootrom.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/staging/greybus/gb-light.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/staging/greybus/gb-power-supply.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/staging/greybus/gb-raw.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/staging/greybus/gb-audio-manager.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/staging/greybus/gb-gbphy.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/staging/greybus/gb-sdio.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/staging/greybus/gb-uart.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/hwtracing/intel_th/intel_th_msu_sink.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/mtd/chips/cfi_util.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/mtd/chips/cfi_cmdset_0020.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/mtd/maps/map_funcs.o
WARNING: modpost: missing MODULE_DESCRIPTION() in drivers/greybus/greybus.o
WARNING: modpost: missing MODULE_DESCRIPTION() in sound/core/snd-pcm-dmaengine.o
WARNING: modpost: missing MODULE_DESCRIPTION() in sound/soc/codecs/snd-soc-sigmadsp.o
WARNING: modpost: missing MODULE_DESCRIPTION() in sound/soc/codecs/snd-soc-wm-adsp.o
WARNING: modpost: missing MODULE_DESCRIPTION() in sound/soc/xilinx/snd-soc-xlnx-i2s.o
WARNING: modpost: missing MODULE_DESCRIPTION() in sound/ac97_bus.o
WARNING: modpost: missing MODULE_DESCRIPTION() in net/8021q/8021q.o
WARNING: modpost: missing MODULE_DESCRIPTION() in net/xdp/xsk_diag.o
WARNING: modpost: missing MODULE_DESCRIPTION() in net/atm/atm.o
WARNING: modpost: missing MODULE_DESCRIPTION() in net/caif/caif.o
WARNING: modpost: missing MODULE_DESCRIPTION() in net/caif/caif_socket.o
WARNING: modpost: missing MODULE_DESCRIPTION() in net/caif/caif_usb.o
WARNING: modpost: missing MODULE_DESCRIPTION() in net/nfc/nfc_digital.o
WARNING: modpost: missing MODULE_DESCRIPTION() in net/hsr/hsr.o
>> ERROR: modpost: "__lshrdi3" [drivers/mtd/nand/raw/nand.ko] undefined!
>> ERROR: modpost: "__lshrdi3" [drivers/mtd/nand/raw/diskonchip.ko] undefined!
>> ERROR: modpost: "__lshrdi3" [drivers/mtd/nand/raw/nandsim.ko] undefined!
>> ERROR: modpost: "__memcat_p" [drivers/hwtracing/stm/stm_core.ko] undefined!

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

Re: [PATCH] kbuild: get lib-y objects back to static library

Posted by Masahiro Yamada 2 years, 3 months ago

On Thu, Aug 24, 2023 at 10:05 AM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> Revert the following two commits:
>
>  - 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y")
>  - 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably")
>
> Now, lib-y is back to static library again, and the link order is
> consistent w/wo CONFIG_MODULES.
>
> Since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by
> modpost"), .vmlinux.export.c contains references to exported symbols.
> If a symbol in a lib-y object is exported, that object is always linked
> even without any explicit user in vmlinux.
>
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---



I just recalled that presumably ARM code must be fixed beforehand.

https://lore.kernel.org/lkml/20230619143725.57967-1-masahiroy@kernel.org/

$(mmu-y) is added to lib-y, but they cannot go into a static library.







-- 
Best Regards
Masahiro Yamada

Re: [PATCH] kbuild: get lib-y objects back to static library

Posted by Nathan Chancellor 2 years, 3 months ago

Hi Masahiro,

On Wed, Aug 23, 2023 at 09:08:16PM +0900, Masahiro Yamada wrote:
> Revert the following two commits:
> 
>  - 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y")
>  - 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably")
> 
> Now, lib-y is back to static library again, and the link order is
> consistent w/wo CONFIG_MODULES.
> 
> Since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by
> modpost"), .vmlinux.export.c contains references to exported symbols.
> If a symbol in a lib-y object is exported, that object is always linked
> even without any explicit user in vmlinux.
> 
> Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> ---
> 
>  Makefile | 5 -----
>  1 file changed, 5 deletions(-)
> 
> diff --git a/Makefile b/Makefile
> index 87a9eef3fb4b..71e9c65d9eae 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1113,12 +1113,7 @@ export ARCH_DRIVERS	:= $(drivers-y) $(drivers-m)
>  # Externally visible symbols (used by link-vmlinux.sh)
>  
>  KBUILD_VMLINUX_OBJS := ./built-in.a
> -ifdef CONFIG_MODULES
> -KBUILD_VMLINUX_OBJS += $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
> -KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
> -else
>  KBUILD_VMLINUX_LIBS := $(patsubst %/,%/lib.a, $(libs-y))
> -endif
>  
>  export KBUILD_VMLINUX_LIBS
>  export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
> -- 
> 2.39.2
> 

The build error that prompted 7273ad2b08f8 from [1] appears to return
with this patch, for example:

$ make -skj"$(nproc)" ARCH=arm64 LLVM=1 O=build mrproper virtconfig all
...
ERROR: modpost: "__memcat_p" [drivers/hwtracing/stm/stm_core.ko] undefined!
...

[1]: https://github.com/ClangBuiltLinux/linux/issues/515

Cheers,
Nathan

Re: [PATCH] kbuild: get lib-y objects back to static library

Posted by Nathan Chancellor 2 years, 3 months ago

On Wed, Aug 23, 2023 at 01:20:42PM -0700, Nathan Chancellor wrote:
> Hi Masahiro,
> 
> On Wed, Aug 23, 2023 at 09:08:16PM +0900, Masahiro Yamada wrote:
> > Revert the following two commits:
> > 
> >  - 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y")
> >  - 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably")
> > 
> > Now, lib-y is back to static library again, and the link order is
> > consistent w/wo CONFIG_MODULES.
> > 
> > Since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by
> > modpost"), .vmlinux.export.c contains references to exported symbols.
> > If a symbol in a lib-y object is exported, that object is always linked
> > even without any explicit user in vmlinux.
> > 
> > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > ---
> > 
> >  Makefile | 5 -----
> >  1 file changed, 5 deletions(-)
> > 
> > diff --git a/Makefile b/Makefile
> > index 87a9eef3fb4b..71e9c65d9eae 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -1113,12 +1113,7 @@ export ARCH_DRIVERS	:= $(drivers-y) $(drivers-m)
> >  # Externally visible symbols (used by link-vmlinux.sh)
> >  
> >  KBUILD_VMLINUX_OBJS := ./built-in.a
> > -ifdef CONFIG_MODULES
> > -KBUILD_VMLINUX_OBJS += $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
> > -KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
> > -else
> >  KBUILD_VMLINUX_LIBS := $(patsubst %/,%/lib.a, $(libs-y))
> > -endif
> >  
> >  export KBUILD_VMLINUX_LIBS
> >  export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
> > -- 
> > 2.39.2
> > 
> 
> The build error that prompted 7273ad2b08f8 from [1] appears to return
> with this patch, for example:
> 
> $ make -skj"$(nproc)" ARCH=arm64 LLVM=1 O=build mrproper virtconfig all
> ...
> ERROR: modpost: "__memcat_p" [drivers/hwtracing/stm/stm_core.ko] undefined!
> ...
> 
> [1]: https://github.com/ClangBuiltLinux/linux/issues/515

One thing I forgot to note with this is this occurs when targeting arm,
arm64, hexagon, i386, powerpc, s390, and x86_64 with various in-tree
configurations (I can provide a list if necessary).

Cheers,
Nathan

Re: [PATCH] kbuild: get lib-y objects back to static library

Posted by Nick Desaulniers 2 years, 3 months ago

On Thu, Aug 24, 2023 at 11:59 AM Nathan Chancellor <nathan@kernel.org> wrote:
>
> On Wed, Aug 23, 2023 at 01:20:42PM -0700, Nathan Chancellor wrote:
> > Hi Masahiro,
> >
> > On Wed, Aug 23, 2023 at 09:08:16PM +0900, Masahiro Yamada wrote:
> > > Revert the following two commits:
> > >
> > >  - 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y")
> > >  - 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably")
> > >
> > > Now, lib-y is back to static library again, and the link order is
> > > consistent w/wo CONFIG_MODULES.
> > >
> > > Since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by
> > > modpost"), .vmlinux.export.c contains references to exported symbols.
> > > If a symbol in a lib-y object is exported, that object is always linked
> > > even without any explicit user in vmlinux.
> > >
> > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > > ---
> > >
> > >  Makefile | 5 -----
> > >  1 file changed, 5 deletions(-)
> > >
> > > diff --git a/Makefile b/Makefile
> > > index 87a9eef3fb4b..71e9c65d9eae 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -1113,12 +1113,7 @@ export ARCH_DRIVERS  := $(drivers-y) $(drivers-m)
> > >  # Externally visible symbols (used by link-vmlinux.sh)
> > >
> > >  KBUILD_VMLINUX_OBJS := ./built-in.a
> > > -ifdef CONFIG_MODULES
> > > -KBUILD_VMLINUX_OBJS += $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
> > > -KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
> > > -else
> > >  KBUILD_VMLINUX_LIBS := $(patsubst %/,%/lib.a, $(libs-y))
> > > -endif
> > >
> > >  export KBUILD_VMLINUX_LIBS
> > >  export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
> > > --
> > > 2.39.2
> > >
> >
> > The build error that prompted 7273ad2b08f8 from [1] appears to return
> > with this patch, for example:
> >
> > $ make -skj"$(nproc)" ARCH=arm64 LLVM=1 O=build mrproper virtconfig all
> > ...
> > ERROR: modpost: "__memcat_p" [drivers/hwtracing/stm/stm_core.ko] undefined!
> > ...
> >
> > [1]: https://github.com/ClangBuiltLinux/linux/issues/515
>
> One thing I forgot to note with this is this occurs when targeting arm,
> arm64, hexagon, i386, powerpc, s390, and x86_64 with various in-tree
> configurations (I can provide a list if necessary).

shot in the dark here, but does this help? (Nathan indicates it does,
at least for `ARCH=arm64 virtconfig`)
```
diff --git a/lib/Makefile b/lib/Makefile
index 1ffae65bb7ee..289ab15f8659 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -33,9 +33,11 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
         flex_proportions.o ratelimit.o \
         is_single_threaded.o plist.o decompress.o kobject_uevent.o \
         earlycpio.o seq_buf.o siphash.o dec_and_lock.o \
-        nmi_backtrace.o win_minmax.o memcat_p.o \
+        nmi_backtrace.o win_minmax.o \
         buildid.o

+obj-y  += memcat_p.o
+
 lib-$(CONFIG_PRINTK) += dump_stack.o
 lib-$(CONFIG_SMP) += cpumask.o

```
Masahiro,
I don't fully understand "when to use obj-y vs lib-y" in a kernel
Makefile.  I've read Documentation/kbuild/makefiles.rst and I still
don't get it.  Is this answered elsewhere in the docs? Can you explain
it to me?  Do we need to update the docs?


-- 
Thanks,
~Nick Desaulniers

Re: [PATCH] kbuild: get lib-y objects back to static library

Posted by Masahiro Yamada 2 years, 3 months ago

On Sat, Aug 26, 2023 at 6:45 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Thu, Aug 24, 2023 at 11:59 AM Nathan Chancellor <nathan@kernel.org> wrote:
> >
> > On Wed, Aug 23, 2023 at 01:20:42PM -0700, Nathan Chancellor wrote:
> > > Hi Masahiro,
> > >
> > > On Wed, Aug 23, 2023 at 09:08:16PM +0900, Masahiro Yamada wrote:
> > > > Revert the following two commits:
> > > >
> > > >  - 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y")
> > > >  - 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably")
> > > >
> > > > Now, lib-y is back to static library again, and the link order is
> > > > consistent w/wo CONFIG_MODULES.
> > > >
> > > > Since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by
> > > > modpost"), .vmlinux.export.c contains references to exported symbols.
> > > > If a symbol in a lib-y object is exported, that object is always linked
> > > > even without any explicit user in vmlinux.
> > > >
> > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > > > ---
> > > >
> > > >  Makefile | 5 -----
> > > >  1 file changed, 5 deletions(-)
> > > >
> > > > diff --git a/Makefile b/Makefile
> > > > index 87a9eef3fb4b..71e9c65d9eae 100644
> > > > --- a/Makefile
> > > > +++ b/Makefile
> > > > @@ -1113,12 +1113,7 @@ export ARCH_DRIVERS  := $(drivers-y) $(drivers-m)
> > > >  # Externally visible symbols (used by link-vmlinux.sh)
> > > >
> > > >  KBUILD_VMLINUX_OBJS := ./built-in.a
> > > > -ifdef CONFIG_MODULES
> > > > -KBUILD_VMLINUX_OBJS += $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
> > > > -KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
> > > > -else
> > > >  KBUILD_VMLINUX_LIBS := $(patsubst %/,%/lib.a, $(libs-y))
> > > > -endif
> > > >
> > > >  export KBUILD_VMLINUX_LIBS
> > > >  export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
> > > > --
> > > > 2.39.2
> > > >
> > >
> > > The build error that prompted 7273ad2b08f8 from [1] appears to return
> > > with this patch, for example:
> > >
> > > $ make -skj"$(nproc)" ARCH=arm64 LLVM=1 O=build mrproper virtconfig all
> > > ...
> > > ERROR: modpost: "__memcat_p" [drivers/hwtracing/stm/stm_core.ko] undefined!
> > > ...
> > >
> > > [1]: https://github.com/ClangBuiltLinux/linux/issues/515
> >
> > One thing I forgot to note with this is this occurs when targeting arm,
> > arm64, hexagon, i386, powerpc, s390, and x86_64 with various in-tree
> > configurations (I can provide a list if necessary).
>
> shot in the dark here, but does this help? (Nathan indicates it does,
> at least for `ARCH=arm64 virtconfig`)
> ```
> diff --git a/lib/Makefile b/lib/Makefile
> index 1ffae65bb7ee..289ab15f8659 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -33,9 +33,11 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
>          flex_proportions.o ratelimit.o \
>          is_single_threaded.o plist.o decompress.o kobject_uevent.o \
>          earlycpio.o seq_buf.o siphash.o dec_and_lock.o \
> -        nmi_backtrace.o win_minmax.o memcat_p.o \
> +        nmi_backtrace.o win_minmax.o \
>          buildid.o
>
> +obj-y  += memcat_p.o
> +


No. it is not a solution.

We will end up with a whack-a-mole game
because this kind of issue (EXPORT_SYMBOL from lib-y)
happens in general.






>  lib-$(CONFIG_PRINTK) += dump_stack.o
>  lib-$(CONFIG_SMP) += cpumask.o
>
> ```
> Masahiro,
> I don't fully understand "when to use obj-y vs lib-y" in a kernel
> Makefile.  I've read Documentation/kbuild/makefiles.rst and I still
> don't get it.  Is this answered elsewhere in the docs? Can you explain
> it to me?  Do we need to update the docs?


The original intention (that existed before I became the Kbuild maintainer) was:

 obj-y     linked to vmlinux unconditionally

 lib-y  :  the compiled object goes into a static library. Hence,
conditionally linked.
           (If no symbol in lib-y object is referenced, it is not linked at all)





If ARCH supports CONFIG_LD_DEAD_CODE_DATA_ELIMINATION,
it is more optimal.


With lib-y, objects are conditionally linked.

With CONFIG_LD_DEAD_CODE_DATA_ELIMINATION,
symbols are conditionally linked.





One solution could be to delete lib-y entirely.


-- 
Best Regards
Masahiro Yamada

Re: [PATCH] kbuild: get lib-y objects back to static library

Posted by Nick Desaulniers 2 years, 3 months ago

On Fri, Aug 25, 2023 at 5:06 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Sat, Aug 26, 2023 at 6:45 AM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> >
> > On Thu, Aug 24, 2023 at 11:59 AM Nathan Chancellor <nathan@kernel.org> wrote:
> > >
> > > On Wed, Aug 23, 2023 at 01:20:42PM -0700, Nathan Chancellor wrote:
> > > > Hi Masahiro,
> > > >
> > > > On Wed, Aug 23, 2023 at 09:08:16PM +0900, Masahiro Yamada wrote:
> > > > > Revert the following two commits:
> > > > >
> > > > >  - 7273ad2b08f8 ("kbuild: link lib-y objects to vmlinux forcibly when CONFIG_MODULES=y")
> > > > >  - 7f2084fa55e6 ("[kbuild] handle exports in lib-y objects reliably")
> > > > >
> > > > > Now, lib-y is back to static library again, and the link order is
> > > > > consistent w/wo CONFIG_MODULES.
> > > > >
> > > > > Since commit ddb5cdbafaaa ("kbuild: generate KSYMTAB entries by
> > > > > modpost"), .vmlinux.export.c contains references to exported symbols.
> > > > > If a symbol in a lib-y object is exported, that object is always linked
> > > > > even without any explicit user in vmlinux.
> > > > >
> > > > > Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
> > > > > ---
> > > > >
> > > > >  Makefile | 5 -----
> > > > >  1 file changed, 5 deletions(-)
> > > > >
> > > > > diff --git a/Makefile b/Makefile
> > > > > index 87a9eef3fb4b..71e9c65d9eae 100644
> > > > > --- a/Makefile
> > > > > +++ b/Makefile
> > > > > @@ -1113,12 +1113,7 @@ export ARCH_DRIVERS  := $(drivers-y) $(drivers-m)
> > > > >  # Externally visible symbols (used by link-vmlinux.sh)
> > > > >
> > > > >  KBUILD_VMLINUX_OBJS := ./built-in.a
> > > > > -ifdef CONFIG_MODULES
> > > > > -KBUILD_VMLINUX_OBJS += $(patsubst %/, %/lib.a, $(filter %/, $(libs-y)))
> > > > > -KBUILD_VMLINUX_LIBS := $(filter-out %/, $(libs-y))
> > > > > -else
> > > > >  KBUILD_VMLINUX_LIBS := $(patsubst %/,%/lib.a, $(libs-y))
> > > > > -endif
> > > > >
> > > > >  export KBUILD_VMLINUX_LIBS
> > > > >  export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
> > > > > --
> > > > > 2.39.2
> > > > >
> > > >
> > > > The build error that prompted 7273ad2b08f8 from [1] appears to return
> > > > with this patch, for example:
> > > >
> > > > $ make -skj"$(nproc)" ARCH=arm64 LLVM=1 O=build mrproper virtconfig all
> > > > ...
> > > > ERROR: modpost: "__memcat_p" [drivers/hwtracing/stm/stm_core.ko] undefined!
> > > > ...
> > > >
> > > > [1]: https://github.com/ClangBuiltLinux/linux/issues/515
> > >
> > > One thing I forgot to note with this is this occurs when targeting arm,
> > > arm64, hexagon, i386, powerpc, s390, and x86_64 with various in-tree
> > > configurations (I can provide a list if necessary).
> >
> > shot in the dark here, but does this help? (Nathan indicates it does,
> > at least for `ARCH=arm64 virtconfig`)
> > ```
> > diff --git a/lib/Makefile b/lib/Makefile
> > index 1ffae65bb7ee..289ab15f8659 100644
> > --- a/lib/Makefile
> > +++ b/lib/Makefile
> > @@ -33,9 +33,11 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> >          flex_proportions.o ratelimit.o \
> >          is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> >          earlycpio.o seq_buf.o siphash.o dec_and_lock.o \
> > -        nmi_backtrace.o win_minmax.o memcat_p.o \
> > +        nmi_backtrace.o win_minmax.o \
> >          buildid.o
> >
> > +obj-y  += memcat_p.o
> > +
>
>
> No. it is not a solution.
>
> We will end up with a whack-a-mole game
> because this kind of issue (EXPORT_SYMBOL from lib-y)
> happens in general.
>
>
>
>
>
>
> >  lib-$(CONFIG_PRINTK) += dump_stack.o
> >  lib-$(CONFIG_SMP) += cpumask.o
> >
> > ```
> > Masahiro,
> > I don't fully understand "when to use obj-y vs lib-y" in a kernel
> > Makefile.  I've read Documentation/kbuild/makefiles.rst and I still
> > don't get it.  Is this answered elsewhere in the docs? Can you explain
> > it to me?  Do we need to update the docs?
>
>
> The original intention (that existed before I became the Kbuild maintainer) was:
>
>  obj-y     linked to vmlinux unconditionally
>
>  lib-y  :  the compiled object goes into a static library. Hence,
> conditionally linked.
>            (If no symbol in lib-y object is referenced, it is not linked at all)
>
>
>
>
>
> If ARCH supports CONFIG_LD_DEAD_CODE_DATA_ELIMINATION,
> it is more optimal.
>
>
> With lib-y, objects are conditionally linked.

Then wrt. memcat_p, it sounds like there should be a Kconfig for that
that drivers such as drivers/hwtracing/stm/stm_core.ko can express a
dependency on. Yeah?

>
> With CONFIG_LD_DEAD_CODE_DATA_ELIMINATION,
> symbols are conditionally linked.
>
>
>
>
>
> One solution could be to delete lib-y entirely.
>
>
> --
> Best Regards
> Masahiro Yamada



-- 
Thanks,
~Nick Desaulniers