scripts/mksysmap: skip objtool __pfx_ symbols

[PATCH] scripts/mksysmap: skip objtool __pfx_ symbols

Posted by Arnd Bergmann 8 months, 3 weeks ago

From: Arnd Bergmann <arnd@arndb.de>

The recently added testcase for overly long symbols triggers when
CONFIG_FUNCTION_PADDING_CFI is set:

Symbol __pfx_snnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7n too long for kallsyms (517 >= 512).
Please increase KSYM_NAME_LEN both in kernel and kallsyms.c

Change the mksymtab table so the prefixed symbols are not included
in kallsyms.

Fixes: c104c16073b7 ("Kunit to check the longest symbol length")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
 scripts/mksysmap | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/mksysmap b/scripts/mksysmap
index 3accbdb269ac..33df38ede1e0 100755
--- a/scripts/mksysmap
+++ b/scripts/mksysmap
@@ -59,6 +59,9 @@
 # EXPORT_SYMBOL (namespace)
 / __kstrtabns_/d
 
+# prefixed symbols from objtool
+/ __pfx_/d
+
 # ---------------------------------------------------------------------------
 # Ignored suffixes
 #  (do not forget '$' after each pattern)
-- 
2.39.5

Re: [PATCH] scripts/mksysmap: skip objtool __pfx_ symbols

Posted by Josh Poimboeuf 8 months, 1 week ago

On Fri, Mar 28, 2025 at 11:48:19AM +0100, Arnd Bergmann wrote:
> From: Arnd Bergmann <arnd@arndb.de>
> 
> The recently added testcase for overly long symbols triggers when
> CONFIG_FUNCTION_PADDING_CFI is set:
> 
> Symbol __pfx_snnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7n too long for kallsyms (517 >= 512).
> Please increase KSYM_NAME_LEN both in kernel and kallsyms.c
> 
> Change the mksymtab table so the prefixed symbols are not included
> in kallsyms.
> 
> Fixes: c104c16073b7 ("Kunit to check the longest symbol length")
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>

I'm not sure we want to remove the __pfx_ symbols from kallsyms. There
can be actual code there.

For example, FineIBT writes code in the __pfx area which can trigger an
#UD.  And we'd want a sane backtrace for that.

Maybe objtool should error out when trying to prefix a function with a
name longer than (KSYM_NAME_LEN - 6).  And the kunit test could be
adjusted accordingly for CONFIG_PREFIX_SYMBOLS.

-- 
Josh

Re: [PATCH] scripts/mksysmap: skip objtool __pfx_ symbols

Posted by Peter Zijlstra 8 months, 1 week ago

On Tue, Apr 08, 2025 at 06:58:49PM -0700, Josh Poimboeuf wrote:
> On Fri, Mar 28, 2025 at 11:48:19AM +0100, Arnd Bergmann wrote:
> > From: Arnd Bergmann <arnd@arndb.de>
> > 
> > The recently added testcase for overly long symbols triggers when
> > CONFIG_FUNCTION_PADDING_CFI is set:
> > 
> > Symbol __pfx_snnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7n too long for kallsyms (517 >= 512).
> > Please increase KSYM_NAME_LEN both in kernel and kallsyms.c
> > 
> > Change the mksymtab table so the prefixed symbols are not included
> > in kallsyms.
> > 
> > Fixes: c104c16073b7 ("Kunit to check the longest symbol length")
> > Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> 
> I'm not sure we want to remove the __pfx_ symbols from kallsyms. There
> can be actual code there.
> 
> For example, FineIBT writes code in the __pfx area which can trigger an
> #UD.  And we'd want a sane backtrace for that.

On top of that, clang kcfi builds do a similar thing, they will generate
__cfi_ prefixed symbols.

And yes, those symbols exist for a reason, there is code there under
various circumstances and backtraces look really weird without these
symbols on -- notably the code in the prefix will be attributed to
whatever symbol comes before, most confusing.

So yeah, don't remove these symbols, and fix the kunit test.

Re: [PATCH] scripts/mksysmap: skip objtool __pfx_ symbols

Posted by Arnd Bergmann 8 months, 1 week ago

On Fri, Apr 11, 2025, at 08:50, Peter Zijlstra wrote:
> On Tue, Apr 08, 2025 at 06:58:49PM -0700, Josh Poimboeuf wrote:
>> On Fri, Mar 28, 2025 at 11:48:19AM +0100, Arnd Bergmann wrote:
>> 
>> For example, FineIBT writes code in the __pfx area which can trigger an
>> #UD.  And we'd want a sane backtrace for that.
>
> On top of that, clang kcfi builds do a similar thing, they will generate
> __cfi_ prefixed symbols.
>
> And yes, those symbols exist for a reason, there is code there under
> various circumstances and backtraces look really weird without these
> symbols on -- notably the code in the prefix will be attributed to
> whatever symbol comes before, most confusing.
>
> So yeah, don't remove these symbols, and fix the kunit test.

kallsyms already removes some CFI symbol names based on regular
expressions:

# CFI type identifiers
/ __kcfi_typeid_/d
/ __kvm_nvhe___kcfi_typeid_/d
/ __pi___kcfi_typeid_/d

Do you think it should not remove some of them?

I ran into another problem with generated symbols that I don't
understand yet, and added this line to avoid the build failures:

/ _GLOBAL__sub_/d

This one is 534 characters long:
_GLOBAL__sub_I_65535_1_snnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7n

      Arnd

Re: [PATCH] scripts/mksysmap: skip objtool __pfx_ symbols

Posted by Peter Zijlstra 8 months, 1 week ago

On Fri, Apr 11, 2025 at 09:47:51AM +0200, Arnd Bergmann wrote:
> On Fri, Apr 11, 2025, at 08:50, Peter Zijlstra wrote:
> > On Tue, Apr 08, 2025 at 06:58:49PM -0700, Josh Poimboeuf wrote:
> >> On Fri, Mar 28, 2025 at 11:48:19AM +0100, Arnd Bergmann wrote:
> >> 
> >> For example, FineIBT writes code in the __pfx area which can trigger an
> >> #UD.  And we'd want a sane backtrace for that.
> >
> > On top of that, clang kcfi builds do a similar thing, they will generate
> > __cfi_ prefixed symbols.
> >
> > And yes, those symbols exist for a reason, there is code there under
> > various circumstances and backtraces look really weird without these
> > symbols on -- notably the code in the prefix will be attributed to
> > whatever symbol comes before, most confusing.
> >
> > So yeah, don't remove these symbols, and fix the kunit test.
> 
> kallsyms already removes some CFI symbol names based on regular
> expressions:
> 
> # CFI type identifiers
> / __kcfi_typeid_/d
> / __kvm_nvhe___kcfi_typeid_/d
> / __pi___kcfi_typeid_/d
> 
> Do you think it should not remove some of them?

Those are typeid symbols, those are fine to remove.

> I ran into another problem with generated symbols that I don't
> understand yet, and added this line to avoid the build failures:
> 
> / _GLOBAL__sub_/d
> 
> This one is 534 characters long:
> _GLOBAL__sub_I_65535_1_snnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7n

I've not seen those before; google seems to suggest this is part of
static initializers.

Re: [PATCH] scripts/mksysmap: skip objtool __pfx_ symbols

Posted by Steven Rostedt 8 months, 1 week ago

On Fri, 11 Apr 2025 12:58:49 +0200
Peter Zijlstra <peterz@infradead.org> wrote:

> > This one is 534 characters long:
> > _GLOBAL__sub_I_65535_1_snnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7n  
> 
> I've not seen those before; google seems to suggest this is part of
> static initializers.

I just hit this on my allyesconfig build:

  NM      .tmp_vmlinux1.syms
  KSYMS   .tmp_vmlinux1.kallsyms.S
Symbol __cfi_snnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7n too long for kallsyms (517 >= 512).
Please increase KSYM_NAME_LEN both in kernel and kallsyms.c
  AS      .tmp_vmlinux1.kallsyms.o
  LD      .tmp_vmlinux2
ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE
ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE
ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE
make[3]: *** [/work/build/trace/nobackup/linux-test.git/scripts/Makefile.vmlinux:91: vmlinux.unstripped] Error 1
make[2]: *** [/work/build/trace/nobackup/linux-test.git/Makefile:1239: vmlinux] Error 2
make[1]: *** [/work/build/trace/nobackup/linux-test.git/Makefile:248: __sub-make] Error 2
make[1]: Leaving directory '/work/build/nobackup/tracetest'
make: *** [Makefile:248: __sub-make] Error 2

I grepped for that symbol and it lives in: lib/tests/longest_symbol_kunit.o

Even after removing that test, my allyesconfig still fails to build with:

ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE
ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE
ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE

I guess I'll just have to remove allyesconfig from my test suite. I
still do allmodconfig which appears to still work. At least that will
shorten my test suite time as allyesconfig takes around a half hour to
complete.

-- Steve

Re: [PATCH] scripts/mksysmap: skip objtool __pfx_ symbols

Posted by David Laight 8 months ago

On Sat, 12 Apr 2025 10:22:18 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

...
> I just hit this on my allyesconfig build:
> 
>   NM      .tmp_vmlinux1.syms
>   KSYMS   .tmp_vmlinux1.kallsyms.S
> Symbol __cfi_snnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nnng1h2i3j4k5l6m7ng1h2i3j4k5l6m7nng1h2i3j4k5l6m7ng1h2i3j4k5l6m7n too long for kallsyms (517 >= 512).
> Please increase KSYM_NAME_LEN both in kernel and kallsyms.c
...
> 
> I grepped for that symbol and it lives in: lib/tests/longest_symbol_kunit.o

Looks like it is carefully counted to be 511 characters.
And then the compiler adds __cfi_ making 517.
I guess the test could be changed?

Is it possible to remove the __cfi_ symbols (to save kernel memory) and then
use a single bit to indicate that the previous few bytes (the same number for
all such symbols) belong to the following symbol?
Then stack backtraces would say "foo-n" instead "__cfi_foo+n".

	David

Re: [PATCH] scripts/mksysmap: skip objtool __pfx_ symbols

Posted by Nathan Chancellor 8 months, 1 week ago

On Sat, Apr 12, 2025 at 10:22:18AM -0400, Steven Rostedt wrote:
> Even after removing that test, my allyesconfig still fails to build with:
> 
> ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE
> ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE
> ld.lld: error: kernel image bigger than KERNEL_IMAGE_SIZE
> 
> I guess I'll just have to remove allyesconfig from my test suite. I
> still do allmodconfig which appears to still work. At least that will
> shorten my test suite time as allyesconfig takes around a half hour to
> complete.

I noticed this from our CI too (I don't see the actual error there as
the build just times out before it can get there but I can see it
locally with all the same tools) and it seems like it is caused by
Linus's commit 6f110a5e4f99 ("Disable SLUB_TINY for build testing"),
which causes CONFIG_KASAN to be enabled for all{mod,yes}config again,
which contributes to blowing out the kernel size. I see this with both
GCC and Clang so I guess we will pursue the same solution and just stop
testing allyesconfig (unless we wanted to stop turning on KASAN for
all*config).

Cheers,
Nathan