From nobody Thu Apr 9 19:17:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 901B83C6A52; Tue, 3 Mar 2026 18:22:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772562125; cv=none; b=OKmteYpaGEAqf6K2fSG1ktlt6J5MU/VwBtALzNFKhsLQbprB1nbRie4bF79XG/AU9FZTa9cfoE65apRdhDkrEiXA3K/WrCdREDUoNjYObLCWAC5J7zfzM/hNa5raxenoKZTkgJLSHzzqGE169HeKnCZUMXDMWt3bMAccud2AQoM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772562125; c=relaxed/simple; bh=prcTe600dhzBqY+0AFbNt8YZeT0cfW0sRSn9pmt4m7c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Kk61ciNRZkoeNPz2aJ1GkkNIgLMvAkpgCtqEQSroYcY1LrTe5r4ZlzU4SCIH7izLysT0lOSQUcB8EqzsS3QqD6hSYACRrQvfAIJbms8RlISOaU41xBvvXgk4kpRomYJEZ5IeH30P51cOJWy42h4x9J7/XsQ+dShocN3pgNpN+qg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fkvZQL2R; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fkvZQL2R" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 19BA1C2BC87; Tue, 3 Mar 2026 18:22:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772562125; bh=prcTe600dhzBqY+0AFbNt8YZeT0cfW0sRSn9pmt4m7c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fkvZQL2R+Bd4u7gTUuHN5LLyvFyxsosOK6fZcJZUykMSCMIiWPEpGV6rxRZREzeXJ 7DJSrGYcHt7Icsn91hkhioqLTV2EW4B3E8AYoOIUBCXmxhTHImlDb0M+AVheMIaSyZ umjzmx2TKoZHpZacqCHq2nq7eK/sTEpJP6ESQFMYl4hPhzSM99pWq0leTBmOHS8C98 tFpxD4jIjiJbWnoVboGvsqpucvjh4dkP1J6lbr+2zlvQyC/HjavYxWljXC28ZAlnIJ qBkdwsHWBvMSqQtAZVhCRTmY+KXl1NNHQ+96q+XvdCPHKq1N493jDGWqYxEoZrHhIA V102CeZs8w3Vw== From: Sasha Levin To: Andrew Morton , Masahiro Yamada , Luis Chamberlain , Linus Torvalds , Richard Weinberger , Juergen Gross , Geert Uytterhoeven , James Bottomley Cc: Jonathan Corbet , Nathan Chancellor , Nicolas Schier , Petr Pavlu , Daniel Gomez , Greg KH , Petr Mladek , Steven Rostedt , Kees Cook , Peter Zijlstra , Thorsten Leemhuis , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-doc@vger.kernel.org, Sasha Levin Subject: [PATCH 1/3] kallsyms: embed source file:line info in kernel stack traces Date: Tue, 3 Mar 2026 13:21:01 -0500 Message-ID: <20260303182103.3523438-2-sashal@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260303182103.3523438-1-sashal@kernel.org> References: <20260303182103.3523438-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add CONFIG_KALLSYMS_LINEINFO, which embeds a compact address-to-line lookup table in the kernel image so stack traces directly print source file and line number information: root@localhost:~# echo c > /proc/sysrq-trigger [ 11.201987] sysrq: Trigger a crash [ 11.202831] Kernel panic - not syncing: sysrq triggered crash [ 11.206218] Call Trace: [ 11.206501] [ 11.206749] dump_stack_lvl+0x5d/0x80 (lib/dump_stack.c:94) [ 11.207403] vpanic+0x36e/0x620 (kernel/panic.c:650) [ 11.208565] ? __lock_acquire+0x465/0x2240 (kernel/locking/lockdep.c:4= 674) [ 11.209324] panic+0xc9/0xd0 (kernel/panic.c:787) [ 11.211873] ? find_held_lock+0x2b/0x80 (kernel/locking/lockdep.c:5350) [ 11.212597] ? lock_release+0xd3/0x300 (kernel/locking/lockdep.c:5535) [ 11.213312] sysrq_handle_crash+0x1a/0x20 (drivers/tty/sysrq.c:154) [ 11.214005] __handle_sysrq.cold+0x66/0x256 (drivers/tty/sysrq.c:611) [ 11.214712] write_sysrq_trigger+0x65/0x80 (drivers/tty/sysrq.c:1221) [ 11.215424] proc_reg_write+0x1bd/0x3c0 (fs/proc/inode.c:330) [ 11.216061] vfs_write+0x1c6/0xff0 (fs/read_write.c:686) [ 11.218848] ksys_write+0xfa/0x200 (fs/read_write.c:740) [ 11.222394] do_syscall_64+0xf3/0x690 (arch/x86/entry/syscall_64.c:63) [ 11.223942] entry_SYSCALL_64_after_hwframe+0x77/0x7f (arch/x86/entry/= entry_64.S:121) At build time, a new host tool (scripts/gen_lineinfo) reads DWARF .debug_line from vmlinux using libdw (elfutils), extracts all address-to-file:line mappings, and generates an assembly file with sorted parallel arrays (offsets from _text, file IDs, and line numbers). These are linked into vmlinux as .rodata. At runtime, kallsyms_lookup_lineinfo() does a binary search on the table and __sprint_symbol() appends "(file:line)" to each stack frame. The lookup uses offsets from _text so it works with KASLR, requires no locks or allocations, and is safe in any context including panic. The feature requires CONFIG_DEBUG_INFO (for DWARF data) and elfutils (libdw-dev) on the build host. Memory footprint measured with a 1852-option x86_64 config: Table: 4,597,583 entries from 4,841 source files lineinfo_addrs[] 4,597,583 x u32 =3D 17.5 MiB lineinfo_file_ids[] 4,597,583 x u16 =3D 8.8 MiB lineinfo_lines[] 4,597,583 x u32 =3D 17.5 MiB file_offsets + filenames ~ 0.1 MiB Total .rodata increase: ~ 44.0 MiB vmlinux (stripped): 529 MiB -> 573 MiB (+44 MiB / +8.3%) Note: this probably won't be something we roll into "production", but it might be useful for the average user given the relatively low memory footprint, in canary deployments for hyperscalers, or by default for folks who run tests/fuzzing/etc. Disclaimer: this was vibe coded over an afternoon with an AI coding assistant. The .config used for testing is a simple KVM guest configuration for local development and testing. Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Sasha Levin --- Documentation/admin-guide/index.rst | 1 + .../admin-guide/kallsyms-lineinfo.rst | 72 +++ MAINTAINERS | 6 + include/linux/kallsyms.h | 32 +- init/Kconfig | 20 + kernel/kallsyms.c | 69 +++ kernel/kallsyms_internal.h | 10 + scripts/.gitignore | 1 + scripts/Makefile | 3 + scripts/gen_lineinfo.c | 444 ++++++++++++++++++ scripts/kallsyms.c | 16 + scripts/link-vmlinux.sh | 66 ++- 12 files changed, 736 insertions(+), 4 deletions(-) create mode 100644 Documentation/admin-guide/kallsyms-lineinfo.rst create mode 100644 scripts/gen_lineinfo.c diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guid= e/index.rst index b734f8a2a2c48..1801b9880aeb7 100644 --- a/Documentation/admin-guide/index.rst +++ b/Documentation/admin-guide/index.rst @@ -73,6 +73,7 @@ problems and bugs in particular. ramoops dynamic-debug-howto init + kallsyms-lineinfo kdump/index perf/index pstore-blk diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentatio= n/admin-guide/kallsyms-lineinfo.rst new file mode 100644 index 0000000000000..4dffc18dbcf5a --- /dev/null +++ b/Documentation/admin-guide/kallsyms-lineinfo.rst @@ -0,0 +1,72 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Kallsyms Source Line Info (LINEINFO) +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Overview +=3D=3D=3D=3D=3D=3D=3D=3D + +``CONFIG_KALLSYMS_LINEINFO`` embeds DWARF-derived source file and line num= ber +mappings into the kernel image so that stack traces include +``(file.c:123)`` annotations next to each symbol. This makes it significa= ntly +easier to pinpoint the exact source location during debugging, without nee= ding +to manually cross-reference addresses with ``addr2line``. + +Enabling the Feature +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Enable the following kernel configuration options:: + + CONFIG_KALLSYMS=3Dy + CONFIG_DEBUG_INFO=3Dy + CONFIG_KALLSYMS_LINEINFO=3Dy + +Build dependency: the host tool ``scripts/gen_lineinfo`` requires ``libdw`` +from elfutils. Install the development package: + +- Debian/Ubuntu: ``apt install libdw-dev`` +- Fedora/RHEL: ``dnf install elfutils-devel`` +- Arch Linux: ``pacman -S elfutils`` + +Example Output +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Without ``CONFIG_KALLSYMS_LINEINFO``:: + + Call Trace: + + dump_stack_lvl+0x5d/0x80 + do_syscall_64+0x82/0x190 + entry_SYSCALL_64_after_hwframe+0x76/0x7e + +With ``CONFIG_KALLSYMS_LINEINFO``:: + + Call Trace: + + dump_stack_lvl+0x5d/0x80 (lib/dump_stack.c:123) + do_syscall_64+0x82/0x190 (arch/x86/entry/common.c:52) + entry_SYSCALL_64_after_hwframe+0x76/0x7e + +Note that assembly routines (such as ``entry_SYSCALL_64_after_hwframe``) a= re +not annotated because they lack DWARF debug information. + +Memory Overhead +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The lineinfo tables are stored in ``.rodata`` and typically add approximat= ely +44 MiB to the kernel image for a standard configuration (~4.6 million DWARF +line entries, ~10 bytes per entry after deduplication). + +Known Limitations +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +- **vmlinux only**: Only symbols in the core kernel image are annotated. + Module symbols are not covered. +- **4 GiB offset limit**: Address offsets from ``_text`` are stored as 32-= bit + values. Entries beyond 4 GiB from ``_text`` are skipped at build time w= ith + a warning. +- **65535 file limit**: Source file IDs are stored as 16-bit values. Buil= ds + with more than 65535 unique source files will fail with an error. +- **No assembly annotations**: Functions implemented in assembly that lack + DWARF ``.debug_line`` data are not annotated. diff --git a/MAINTAINERS b/MAINTAINERS index 61bf550fd37c2..ab987e74bb0f5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14278,6 +14278,12 @@ F: lib/Kconfig.kmsan F: mm/kmsan/ F: scripts/Makefile.kmsan =20 +KALLSYMS LINEINFO +M: Sasha Levin +S: Maintained +F: Documentation/admin-guide/kallsyms-lineinfo.rst +F: scripts/gen_lineinfo.c + KPROBES M: Naveen N Rao M: "David S. Miller" diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h index d5dd54c53ace6..e1d00e1373779 100644 --- a/include/linux/kallsyms.h +++ b/include/linux/kallsyms.h @@ -16,10 +16,19 @@ #include =20 #define KSYM_NAME_LEN 512 + +#ifdef CONFIG_KALLSYMS_LINEINFO +/* Extra space for " (path/to/file.c:12345)" suffix */ +#define KSYM_LINEINFO_LEN 128 +#else +#define KSYM_LINEINFO_LEN 0 +#endif + #define KSYM_SYMBOL_LEN (sizeof("%s+%#lx/%#lx [%s %s]") + \ (KSYM_NAME_LEN - 1) + \ 2*(BITS_PER_LONG*3/10) + (MODULE_NAME_LEN - 1) + \ - (BUILD_ID_SIZE_MAX * 2) + 1) + (BUILD_ID_SIZE_MAX * 2) + 1 + \ + KSYM_LINEINFO_LEN) =20 struct cred; struct module; @@ -96,6 +105,19 @@ extern int sprint_backtrace_build_id(char *buffer, unsi= gned long address); =20 int lookup_symbol_name(unsigned long addr, char *symname); =20 +#ifdef CONFIG_KALLSYMS_LINEINFO +bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start, + const char **file, unsigned int *line); +#else +static inline bool kallsyms_lookup_lineinfo(unsigned long addr, + unsigned long sym_start, + const char **file, + unsigned int *line) +{ + return false; +} +#endif + #else /* !CONFIG_KALLSYMS */ =20 static inline unsigned long kallsyms_lookup_name(const char *name) @@ -164,6 +186,14 @@ static inline int kallsyms_on_each_match_symbol(int (*= fn)(void *, unsigned long) { return -EOPNOTSUPP; } + +static inline bool kallsyms_lookup_lineinfo(unsigned long addr, + unsigned long sym_start, + const char **file, + unsigned int *line) +{ + return false; +} #endif /*CONFIG_KALLSYMS*/ =20 static inline void print_ip_sym(const char *loglvl, unsigned long ip) diff --git a/init/Kconfig b/init/Kconfig index b55deae9256c7..c39f27e6393a8 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2050,6 +2050,26 @@ config KALLSYMS_ALL =20 Say N unless you really need all symbols, or kernel live patching. =20 +config KALLSYMS_LINEINFO + bool "Embed source file:line information in stack traces" + depends on KALLSYMS && DEBUG_INFO + help + Embeds an address-to-source-line mapping table in the kernel + image so that stack traces directly include file:line information, + similar to what scripts/decode_stacktrace.sh provides but without + needing external tools or a vmlinux with debug info at runtime. + + When enabled, stack traces will look like: + + kmem_cache_alloc_noprof+0x60/0x630 (mm/slub.c:3456) + anon_vma_clone+0x2ed/0xcf0 (mm/rmap.c:412) + + This requires elfutils (libdw-dev/elfutils-devel) on the build host. + Adds approximately 44MB to a typical kernel image (10 bytes per + DWARF line-table entry, ~4.6M entries for a typical config). + + If unsure, say N. + # end of the "standard kernel features (expert users)" menu =20 config ARCH_HAS_MEMBARRIER_CALLBACKS diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index aec2f06858afd..2b9c9d6322a3e 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -467,6 +467,62 @@ static int append_buildid(char *buffer, const char *= modname, =20 #endif /* CONFIG_STACKTRACE_BUILD_ID */ =20 +#ifdef CONFIG_KALLSYMS_LINEINFO +bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start, + const char **file, unsigned int *line) +{ + unsigned long long raw_offset; + unsigned int offset, low, high, mid, file_id; + unsigned long line_addr; + + if (!lineinfo_num_entries) + return false; + + /* Compute offset from _text */ + if (addr < (unsigned long)_text) + return false; + + raw_offset =3D addr - (unsigned long)_text; + if (raw_offset > UINT_MAX) + return false; + offset =3D (unsigned int)raw_offset; + + /* Binary search for largest entry <=3D offset */ + low =3D 0; + high =3D lineinfo_num_entries; + while (low < high) { + mid =3D low + (high - low) / 2; + if (lineinfo_addrs[mid] <=3D offset) + low =3D mid + 1; + else + high =3D mid; + } + + if (low =3D=3D 0) + return false; + low--; + + /* + * Validate that the matched lineinfo entry belongs to the same + * symbol. Without this check, assembly routines or other + * functions lacking DWARF data would inherit the file:line of + * a preceding C function. + */ + line_addr =3D (unsigned long)_text + lineinfo_addrs[low]; + if (line_addr < sym_start) + return false; + + file_id =3D lineinfo_file_ids[low]; + *line =3D lineinfo_lines[low]; + + if (file_id >=3D lineinfo_num_files) + return false; + + *file =3D &lineinfo_filenames[lineinfo_file_offsets[file_id]]; + return true; +} +#endif /* CONFIG_KALLSYMS_LINEINFO */ + /* Look up a kernel symbol and return it in a text buffer. */ static int __sprint_symbol(char *buffer, unsigned long address, int symbol_offset, int add_offset, int add_buildid) @@ -497,6 +553,19 @@ static int __sprint_symbol(char *buffer, unsigned long= address, len +=3D sprintf(buffer + len, "]"); } =20 +#ifdef CONFIG_KALLSYMS_LINEINFO + if (!modname) { + const char *li_file; + unsigned int li_line; + unsigned long sym_start =3D address - offset; + + if (kallsyms_lookup_lineinfo(address, sym_start, + &li_file, &li_line)) + len +=3D snprintf(buffer + len, KSYM_SYMBOL_LEN - len, + " (%s:%u)", li_file, li_line); + } +#endif + return len; } =20 diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h index 81a867dbe57d4..868a1d5035212 100644 --- a/kernel/kallsyms_internal.h +++ b/kernel/kallsyms_internal.h @@ -15,4 +15,14 @@ extern const u16 kallsyms_token_index[]; extern const unsigned int kallsyms_markers[]; extern const u8 kallsyms_seqs_of_names[]; =20 +#ifdef CONFIG_KALLSYMS_LINEINFO +extern const u32 lineinfo_num_entries; +extern const u32 lineinfo_addrs[]; +extern const u16 lineinfo_file_ids[]; +extern const u32 lineinfo_lines[]; +extern const u32 lineinfo_num_files; +extern const u32 lineinfo_file_offsets[]; +extern const char lineinfo_filenames[]; +#endif + #endif // LINUX_KALLSYMS_INTERNAL_H_ diff --git a/scripts/.gitignore b/scripts/.gitignore index 4215c2208f7e4..e175714c18b61 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only /asn1_compiler +/gen_lineinfo /gen_packed_field_checks /generate_rust_target /insert-sys-cert diff --git a/scripts/Makefile b/scripts/Makefile index 0941e5ce7b575..ffe89875b3295 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -4,6 +4,7 @@ # the kernel for the build process. =20 hostprogs-always-$(CONFIG_KALLSYMS) +=3D kallsyms +hostprogs-always-$(CONFIG_KALLSYMS_LINEINFO) +=3D gen_lineinfo hostprogs-always-$(BUILD_C_RECORDMCOUNT) +=3D recordmcount hostprogs-always-$(CONFIG_BUILDTIME_TABLE_SORT) +=3D sorttable hostprogs-always-$(CONFIG_ASN1) +=3D asn1_compiler @@ -36,6 +37,8 @@ HOSTLDLIBS_sorttable =3D -lpthread HOSTCFLAGS_asn1_compiler.o =3D -I$(srctree)/include HOSTCFLAGS_sign-file.o =3D $(shell $(HOSTPKG_CONFIG) --cflags libcrypto 2>= /dev/null) HOSTLDLIBS_sign-file =3D $(shell $(HOSTPKG_CONFIG) --libs libcrypto 2> /de= v/null || echo -lcrypto) +HOSTCFLAGS_gen_lineinfo.o =3D $(shell $(HOSTPKG_CONFIG) --cflags libdw 2> = /dev/null) +HOSTLDLIBS_gen_lineinfo =3D $(shell $(HOSTPKG_CONFIG) --libs libdw 2> /dev= /null || echo -ldw -lelf -lz) =20 ifdef CONFIG_UNWINDER_ORC ifeq ($(ARCH),x86_64) diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c new file mode 100644 index 0000000000000..9eebfaca5857c --- /dev/null +++ b/scripts/gen_lineinfo.c @@ -0,0 +1,444 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * gen_lineinfo.c - Generate address-to-source-line lookup tables from DWA= RF + * + * Copyright (C) 2026 Sasha Levin + * + * Reads DWARF .debug_line from a vmlinux ELF file and outputs an assembly + * file containing sorted lookup tables that the kernel uses to annotate + * stack traces with source file:line information. + * + * Requires libdw from elfutils. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static unsigned int skipped_overflow; + +struct line_entry { + unsigned int offset; /* offset from _text */ + unsigned int file_id; + unsigned int line; +}; + +struct file_entry { + char *name; + unsigned int id; + unsigned int str_offset; +}; + +static struct line_entry *entries; +static unsigned int num_entries; +static unsigned int entries_capacity; + +static struct file_entry *files; +static unsigned int num_files; +static unsigned int files_capacity; + +#define FILE_HASH_BITS 13 +#define FILE_HASH_SIZE (1 << FILE_HASH_BITS) + +struct file_hash_entry { + const char *name; + unsigned int id; +}; + +static struct file_hash_entry file_hash[FILE_HASH_SIZE]; + +static unsigned int hash_str(const char *s) +{ + unsigned int h =3D 5381; + + for (; *s; s++) + h =3D h * 33 + (unsigned char)*s; + return h & (FILE_HASH_SIZE - 1); +} + +static void add_entry(unsigned int offset, unsigned int file_id, + unsigned int line) +{ + if (num_entries >=3D entries_capacity) { + entries_capacity =3D entries_capacity ? entries_capacity * 2 : 65536; + entries =3D realloc(entries, entries_capacity * sizeof(*entries)); + if (!entries) { + fprintf(stderr, "out of memory\n"); + exit(1); + } + } + entries[num_entries].offset =3D offset; + entries[num_entries].file_id =3D file_id; + entries[num_entries].line =3D line; + num_entries++; +} + +static unsigned int find_or_add_file(const char *name) +{ + unsigned int h =3D hash_str(name); + + /* Open-addressing lookup with linear probing */ + while (file_hash[h].name) { + if (!strcmp(file_hash[h].name, name)) + return file_hash[h].id; + h =3D (h + 1) & (FILE_HASH_SIZE - 1); + } + + if (num_files >=3D 65535) { + fprintf(stderr, + "gen_lineinfo: too many source files (%u > 65535)\n", + num_files); + exit(1); + } + + if (num_files >=3D files_capacity) { + files_capacity =3D files_capacity ? files_capacity * 2 : 4096; + files =3D realloc(files, files_capacity * sizeof(*files)); + if (!files) { + fprintf(stderr, "out of memory\n"); + exit(1); + } + } + files[num_files].name =3D strdup(name); + files[num_files].id =3D num_files; + + /* Insert into hash table (points to files[] entry) */ + file_hash[h].name =3D files[num_files].name; + file_hash[h].id =3D num_files; + + num_files++; + return num_files - 1; +} + +/* + * Strip a filename to a kernel-relative path. + * + * For absolute paths, strip the comp_dir prefix (from DWARF) to get + * a kernel-tree-relative path, or fall back to the basename. + */ +static const char *make_relative(const char *path, const char *comp_dir) +{ + const char *p; + + /* If already relative, use as-is */ + if (path[0] !=3D '/') + return path; + + /* comp_dir from DWARF is the most reliable method */ + if (comp_dir) { + size_t len =3D strlen(comp_dir); + + if (!strncmp(path, comp_dir, len) && path[len] =3D=3D '/') + return path + len + 1; + } + + /* Fall back to basename */ + p =3D strrchr(path, '/'); + return p ? p + 1 : path; +} + +static int compare_entries(const void *a, const void *b) +{ + const struct line_entry *ea =3D a; + const struct line_entry *eb =3D b; + + if (ea->offset !=3D eb->offset) + return ea->offset < eb->offset ? -1 : 1; + if (ea->file_id !=3D eb->file_id) + return ea->file_id < eb->file_id ? -1 : 1; + if (ea->line !=3D eb->line) + return ea->line < eb->line ? -1 : 1; + return 0; +} + +static unsigned long long find_text_addr(Elf *elf) +{ + size_t nsyms, i; + Elf_Scn *scn =3D NULL; + GElf_Shdr shdr; + + while ((scn =3D elf_nextscn(elf, scn)) !=3D NULL) { + Elf_Data *data; + + if (!gelf_getshdr(scn, &shdr)) + continue; + if (shdr.sh_type !=3D SHT_SYMTAB) + continue; + + data =3D elf_getdata(scn, NULL); + if (!data) + continue; + + nsyms =3D shdr.sh_size / shdr.sh_entsize; + for (i =3D 0; i < nsyms; i++) { + GElf_Sym sym; + const char *name; + + if (!gelf_getsym(data, i, &sym)) + continue; + name =3D elf_strptr(elf, shdr.sh_link, sym.st_name); + if (name && !strcmp(name, "_text")) + return sym.st_value; + } + } + + fprintf(stderr, "Cannot find _text symbol\n"); + exit(1); +} + +static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr) +{ + Dwarf_Off off =3D 0, next_off; + size_t hdr_size; + + while (dwarf_nextcu(dwarf, off, &next_off, &hdr_size, + NULL, NULL, NULL) =3D=3D 0) { + Dwarf_Die cudie; + Dwarf_Lines *lines; + size_t nlines; + Dwarf_Attribute attr; + const char *comp_dir =3D NULL; + + if (!dwarf_offdie(dwarf, off + hdr_size, &cudie)) + goto next; + + if (dwarf_attr(&cudie, DW_AT_comp_dir, &attr)) + comp_dir =3D dwarf_formstring(&attr); + + if (dwarf_getsrclines(&cudie, &lines, &nlines) !=3D 0) + goto next; + + for (size_t i =3D 0; i < nlines; i++) { + Dwarf_Line *line =3D dwarf_onesrcline(lines, i); + Dwarf_Addr addr; + const char *src; + const char *rel; + unsigned int file_id, loffset; + int lineno; + + if (!line) + continue; + + if (dwarf_lineaddr(line, &addr) !=3D 0) + continue; + if (dwarf_lineno(line, &lineno) !=3D 0) + continue; + if (lineno =3D=3D 0) + continue; + + src =3D dwarf_linesrc(line, NULL, NULL); + if (!src) + continue; + + if (addr < text_addr) + continue; + + { + unsigned long long raw_offset =3D addr - text_addr; + + if (raw_offset > UINT_MAX) { + skipped_overflow++; + continue; + } + loffset =3D (unsigned int)raw_offset; + } + + rel =3D make_relative(src, comp_dir); + file_id =3D find_or_add_file(rel); + + add_entry(loffset, file_id, (unsigned int)lineno); + } +next: + off =3D next_off; + } +} + +static void deduplicate(void) +{ + unsigned int i, j; + + if (num_entries < 2) + return; + + /* Sort by offset, then file_id, then line for stability */ + qsort(entries, num_entries, sizeof(*entries), compare_entries); + + /* + * Remove duplicate entries: + * - Same offset: keep first (deterministic from stable sort keys) + * - Same file:line as previous kept entry: redundant for binary + * search -- any address between them resolves to the earlier one + */ + j =3D 0; + for (i =3D 1; i < num_entries; i++) { + if (entries[i].offset =3D=3D entries[j].offset) + continue; + if (entries[i].file_id =3D=3D entries[j].file_id && + entries[i].line =3D=3D entries[j].line) + continue; + j++; + if (j !=3D i) + entries[j] =3D entries[i]; + } + num_entries =3D j + 1; +} + +static void compute_file_offsets(void) +{ + unsigned int offset =3D 0; + + for (unsigned int i =3D 0; i < num_files; i++) { + files[i].str_offset =3D offset; + offset +=3D strlen(files[i].name) + 1; + } +} + +static void print_escaped_asciz(const char *s) +{ + printf("\t.asciz \""); + for (; *s; s++) { + if (*s =3D=3D '"' || *s =3D=3D '\\') + putchar('\\'); + putchar(*s); + } + printf("\"\n"); +} + +static void output_assembly(void) +{ + printf("/* SPDX-License-Identifier: GPL-2.0 */\n"); + printf("/*\n"); + printf(" * Automatically generated by scripts/gen_lineinfo\n"); + printf(" * Do not edit.\n"); + printf(" */\n\n"); + + printf("\t.section .rodata, \"a\"\n\n"); + + /* Number of entries */ + printf("\t.globl lineinfo_num_entries\n"); + printf("\t.balign 4\n"); + printf("lineinfo_num_entries:\n"); + printf("\t.long %u\n\n", num_entries); + + /* Number of files */ + printf("\t.globl lineinfo_num_files\n"); + printf("\t.balign 4\n"); + printf("lineinfo_num_files:\n"); + printf("\t.long %u\n\n", num_files); + + /* Sorted address offsets from _text */ + printf("\t.globl lineinfo_addrs\n"); + printf("\t.balign 4\n"); + printf("lineinfo_addrs:\n"); + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.long 0x%x\n", entries[i].offset); + printf("\n"); + + /* File IDs, parallel to addrs (u16 -- supports up to 65535 files) */ + printf("\t.globl lineinfo_file_ids\n"); + printf("\t.balign 2\n"); + printf("lineinfo_file_ids:\n"); + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.short %u\n", entries[i].file_id); + printf("\n"); + + /* Line numbers, parallel to addrs */ + printf("\t.globl lineinfo_lines\n"); + printf("\t.balign 4\n"); + printf("lineinfo_lines:\n"); + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.long %u\n", entries[i].line); + printf("\n"); + + /* File string offset table */ + printf("\t.globl lineinfo_file_offsets\n"); + printf("\t.balign 4\n"); + printf("lineinfo_file_offsets:\n"); + for (unsigned int i =3D 0; i < num_files; i++) + printf("\t.long %u\n", files[i].str_offset); + printf("\n"); + + /* Concatenated NUL-terminated filenames */ + printf("\t.globl lineinfo_filenames\n"); + printf("lineinfo_filenames:\n"); + for (unsigned int i =3D 0; i < num_files; i++) + print_escaped_asciz(files[i].name); + printf("\n"); +} + +int main(int argc, char *argv[]) +{ + int fd; + Elf *elf; + Dwarf *dwarf; + unsigned long long text_addr; + + if (argc !=3D 2) { + fprintf(stderr, "Usage: %s \n", argv[0]); + return 1; + } + + fd =3D open(argv[1], O_RDONLY); + if (fd < 0) { + fprintf(stderr, "Cannot open %s: %s\n", argv[1], + strerror(errno)); + return 1; + } + + elf_version(EV_CURRENT); + elf =3D elf_begin(fd, ELF_C_READ, NULL); + if (!elf) { + fprintf(stderr, "elf_begin failed: %s\n", + elf_errmsg(elf_errno())); + close(fd); + return 1; + } + + text_addr =3D find_text_addr(elf); + + dwarf =3D dwarf_begin_elf(elf, DWARF_C_READ, NULL); + if (!dwarf) { + fprintf(stderr, "dwarf_begin_elf failed: %s\n", + dwarf_errmsg(dwarf_errno())); + fprintf(stderr, "Is %s built with CONFIG_DEBUG_INFO?\n", + argv[1]); + elf_end(elf); + close(fd); + return 1; + } + + process_dwarf(dwarf, text_addr); + + if (skipped_overflow) + fprintf(stderr, + "lineinfo: warning: %u entries skipped (offset > 4 GiB from _text)\n", + skipped_overflow); + + deduplicate(); + compute_file_offsets(); + + fprintf(stderr, "lineinfo: %u entries, %u files\n", + num_entries, num_files); + + output_assembly(); + + dwarf_end(dwarf); + elf_end(elf); + close(fd); + + /* Cleanup */ + free(entries); + for (unsigned int i =3D 0; i < num_files; i++) + free(files[i].name); + free(files); + + return 0; +} diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 37d5c095ad22a..42662c4fbc6c9 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -78,6 +78,17 @@ static char *sym_name(const struct sym_entry *s) =20 static bool is_ignored_symbol(const char *name, char type) { + /* Ignore lineinfo symbols for kallsyms pass stability */ + static const char * const lineinfo_syms[] =3D { + "lineinfo_addrs", + "lineinfo_file_ids", + "lineinfo_file_offsets", + "lineinfo_filenames", + "lineinfo_lines", + "lineinfo_num_entries", + "lineinfo_num_files", + }; + if (type =3D=3D 'u' || type =3D=3D 'n') return true; =20 @@ -90,6 +101,11 @@ static bool is_ignored_symbol(const char *name, char ty= pe) return true; } =20 + for (size_t i =3D 0; i < ARRAY_SIZE(lineinfo_syms); i++) { + if (!strcmp(name, lineinfo_syms[i])) + return true; + } + return false; } =20 diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index f99e196abeea4..640209f2e9eb9 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -103,7 +103,7 @@ vmlinux_link() ${ld} ${ldflags} -o ${output} \ ${wl}--whole-archive ${objs} ${wl}--no-whole-archive \ ${wl}--start-group ${libs} ${wl}--end-group \ - ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} + ${kallsymso} ${lineinfo_o} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldli= bs} } =20 # Create ${2}.o file with all symbols from the ${1} object file @@ -129,6 +129,26 @@ kallsyms() kallsymso=3D${2}.o } =20 +# Generate lineinfo tables from DWARF debug info in a temporary vmlinux. +# ${1} - temporary vmlinux with debug info +# Output: sets lineinfo_o to the generated .o file +gen_lineinfo() +{ + info LINEINFO .tmp_lineinfo.S + if ! scripts/gen_lineinfo "${1}" > .tmp_lineinfo.S; then + echo >&2 "Failed to generate lineinfo from ${1}" + echo >&2 "Try to disable CONFIG_KALLSYMS_LINEINFO" + exit 1 + fi + + info AS .tmp_lineinfo.o + ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \ + ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \ + -c -o .tmp_lineinfo.o .tmp_lineinfo.S + + lineinfo_o=3D.tmp_lineinfo.o +} + # Perform kallsyms for the given temporary vmlinux. sysmap_and_kallsyms() { @@ -155,6 +175,7 @@ sorttable() cleanup() { rm -f .btf.* + rm -f .tmp_lineinfo.* rm -f .tmp_vmlinux.nm-sort rm -f System.map rm -f vmlinux @@ -183,6 +204,7 @@ fi btf_vmlinux_bin_o=3D btfids_vmlinux=3D kallsymso=3D +lineinfo_o=3D strip_debug=3D generate_map=3D =20 @@ -198,10 +220,44 @@ if is_enabled CONFIG_KALLSYMS; then kallsyms .tmp_vmlinux0.syms .tmp_vmlinux0.kallsyms fi =20 +if is_enabled CONFIG_KALLSYMS_LINEINFO; then + # Generate a dummy empty lineinfo object for the initial link, + # same pattern as the dummy kallsyms above. The real lineinfo + # is generated from .tmp_vmlinux1 after it has been linked with + # debug info. + cat > .tmp_lineinfo.S <<'EOAS' + .section .rodata, "a" + .globl lineinfo_num_entries + .balign 4 +lineinfo_num_entries: + .long 0 + .globl lineinfo_num_files + .balign 4 +lineinfo_num_files: + .long 0 + .globl lineinfo_addrs +lineinfo_addrs: + .globl lineinfo_file_ids +lineinfo_file_ids: + .globl lineinfo_lines +lineinfo_lines: + .globl lineinfo_file_offsets +lineinfo_file_offsets: + .globl lineinfo_filenames +lineinfo_filenames: +EOAS + ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \ + ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \ + -c -o .tmp_lineinfo.o .tmp_lineinfo.S + lineinfo_o=3D.tmp_lineinfo.o +fi + if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then =20 - # The kallsyms linking does not need debug symbols, but the BTF does. - if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + # The kallsyms linking does not need debug symbols, but BTF and + # lineinfo generation do. + if ! is_enabled CONFIG_DEBUG_INFO_BTF && + ! is_enabled CONFIG_KALLSYMS_LINEINFO; then strip_debug=3D1 fi =20 @@ -219,6 +275,10 @@ if is_enabled CONFIG_DEBUG_INFO_BTF; then btfids_vmlinux=3D.tmp_vmlinux1.BTF_ids fi =20 +if is_enabled CONFIG_KALLSYMS_LINEINFO; then + gen_lineinfo .tmp_vmlinux1 +fi + if is_enabled CONFIG_KALLSYMS; then =20 # kallsyms support --=20 2.51.0 From nobody Thu Apr 9 19:17:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5817B1891A9; Tue, 3 Mar 2026 18:22:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772562128; cv=none; b=emV1Y0ELE7mcMu36B2IOW4IS48ruUpHPSyND3ubh7R0nOagmGb5XcF4r9W0zzHLGJKkA7Coy4vAHqJPNU9joo7drxUoaNyLE+2fCoMBQCMcBfGC/3zDoAvbHD25LFc3a/5nGRHxoK7ZaXgR2uFOQ2kSpbsAu0uliEeBdYOvAb2E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772562128; c=relaxed/simple; bh=KEoCO8m+FUxF4B3WkKHS/khs8xkMwjfLc3f/e0By5og=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=AgFfJmBA6WFpteB7/izwK3dqLOZt9/MPSR2JO6lExIanrrQbuLaw7M4iE9ysUg0Wt030Xlv430/6BdQ6xtoPh2uawarfNEMruW1y7h0UV+z0hjPTfJcxxvi09egnRTr7jiSu1y0xkQRq3ZWyIy0WIrhsFmVEU+9nVh44Rkq9e2s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FTL1hzeH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FTL1hzeH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A272BC116C6; Tue, 3 Mar 2026 18:22:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772562128; bh=KEoCO8m+FUxF4B3WkKHS/khs8xkMwjfLc3f/e0By5og=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FTL1hzeHzMzVB70BgVxZ1Ux55xCIDqReNWn+6nAj7DyLHG2QwQM2VrV32MVg4THhK bqoRV08jDq681jUBahhdzvGRjIam/m71333QDK12W/r+A3xclu49Fpz1h8dYk41Wyv /D92nq1GmU2ji4HF2jWIZWwUtBdQ/EhtHdQFzahSvnjJOUBhnXX9Ne2yip8rw2t/y2 nypwxu4u9FHCwVkAFT0VQRrc0vLp9MX5+g38Lz8fIt+pvnSaShT84MEvrR2yvy7Ax3 DPnLyseV4z3u+deafPlwe3rqHESU94HktKoKMgIeWOWD2hbaak8+MeYjncrfG0XDrF eS/HxK1+g8K/A== From: Sasha Levin To: Andrew Morton , Masahiro Yamada , Luis Chamberlain , Linus Torvalds , Richard Weinberger , Juergen Gross , Geert Uytterhoeven , James Bottomley Cc: Jonathan Corbet , Nathan Chancellor , Nicolas Schier , Petr Pavlu , Daniel Gomez , Greg KH , Petr Mladek , Steven Rostedt , Kees Cook , Peter Zijlstra , Thorsten Leemhuis , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-doc@vger.kernel.org, Sasha Levin Subject: [PATCH 2/3] kallsyms: extend lineinfo to loadable modules Date: Tue, 3 Mar 2026 13:21:02 -0500 Message-ID: <20260303182103.3523438-3-sashal@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260303182103.3523438-1-sashal@kernel.org> References: <20260303182103.3523438-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Add CONFIG_KALLSYMS_LINEINFO_MODULES, which extends the CONFIG_KALLSYMS_LINEINFO feature to loadable kernel modules. At build time, each .ko is post-processed by scripts/gen-mod-lineinfo.sh (modeled on gen-btf.sh) which runs scripts/gen_lineinfo --module on the .ko, generates a .mod_lineinfo section containing a compact binary table of .text-relative offsets, file IDs, line numbers, and filenames, and embeds it back into the .ko via objcopy. At runtime, module_lookup_lineinfo() performs a binary search on the module's .mod_lineinfo section, and __sprint_symbol() calls it for addresses that fall within a module. The lookup is NMI/panic-safe (no locks, no allocations) - the data lives in read-only module memory and is freed automatically when the module is unloaded. Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Sasha Levin --- .../admin-guide/kallsyms-lineinfo.rst | 40 ++++- MAINTAINERS | 2 + include/linux/mod_lineinfo.h | 68 +++++++ include/linux/module.h | 19 ++ init/Kconfig | 13 ++ kernel/kallsyms.c | 22 ++- kernel/module/kallsyms.c | 95 ++++++++++ kernel/module/main.c | 4 + scripts/Makefile | 1 + scripts/Makefile.modfinal | 6 + scripts/gen-mod-lineinfo.sh | 48 +++++ scripts/gen_lineinfo.c | 166 ++++++++++++++++-- 12 files changed, 458 insertions(+), 26 deletions(-) create mode 100644 include/linux/mod_lineinfo.h create mode 100755 scripts/gen-mod-lineinfo.sh diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentatio= n/admin-guide/kallsyms-lineinfo.rst index 4dffc18dbcf5a..21450569d5324 100644 --- a/Documentation/admin-guide/kallsyms-lineinfo.rst +++ b/Documentation/admin-guide/kallsyms-lineinfo.rst @@ -51,22 +51,46 @@ With ``CONFIG_KALLSYMS_LINEINFO``:: Note that assembly routines (such as ``entry_SYSCALL_64_after_hwframe``) a= re not annotated because they lack DWARF debug information. =20 +Module Support +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +``CONFIG_KALLSYMS_LINEINFO_MODULES`` extends the feature to loadable kernel +modules. When enabled, each ``.ko`` is post-processed at build time to em= bed +a ``.mod_lineinfo`` section containing the same kind of address-to-source +mapping. + +Enable in addition to the base options:: + + CONFIG_MODULES=3Dy + CONFIG_KALLSYMS_LINEINFO_MODULES=3Dy + +Stack traces from module code will then include annotations:: + + my_driver_func+0x30/0x100 [my_driver] (drivers/foo/bar.c:123) + +The ``.mod_lineinfo`` section is loaded into read-only module memory along= side +the module text. No additional runtime memory allocation is required; the= data +is freed when the module is unloaded. + Memory Overhead =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -The lineinfo tables are stored in ``.rodata`` and typically add approximat= ely -44 MiB to the kernel image for a standard configuration (~4.6 million DWARF -line entries, ~10 bytes per entry after deduplication). +The vmlinux lineinfo tables are stored in ``.rodata`` and typically add +approximately 44 MiB to the kernel image for a standard configuration +(~4.6 million DWARF line entries, ~10 bytes per entry after deduplication). + +Per-module lineinfo adds approximately 10 bytes per DWARF line entry to ea= ch +``.ko`` file. =20 Known Limitations =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -- **vmlinux only**: Only symbols in the core kernel image are annotated. - Module symbols are not covered. -- **4 GiB offset limit**: Address offsets from ``_text`` are stored as 32-= bit - values. Entries beyond 4 GiB from ``_text`` are skipped at build time w= ith - a warning. +- **4 GiB offset limit**: Address offsets from ``_text`` (vmlinux) or + ``.text`` base (modules) are stored as 32-bit values. Entries beyond + 4 GiB are skipped at build time with a warning. - **65535 file limit**: Source file IDs are stored as 16-bit values. Buil= ds with more than 65535 unique source files will fail with an error. - **No assembly annotations**: Functions implemented in assembly that lack DWARF ``.debug_line`` data are not annotated. +- **No init text**: For modules, functions in ``.init.text`` are not annot= ated + because that memory is freed after module initialization. diff --git a/MAINTAINERS b/MAINTAINERS index ab987e74bb0f5..d04abafd9eb77 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14282,6 +14282,8 @@ KALLSYMS LINEINFO M: Sasha Levin S: Maintained F: Documentation/admin-guide/kallsyms-lineinfo.rst +F: include/linux/mod_lineinfo.h +F: scripts/gen-mod-lineinfo.sh F: scripts/gen_lineinfo.c =20 KPROBES diff --git a/include/linux/mod_lineinfo.h b/include/linux/mod_lineinfo.h new file mode 100644 index 0000000000000..d62e9608f0f82 --- /dev/null +++ b/include/linux/mod_lineinfo.h @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * mod_lineinfo.h - Binary format for per-module source line information + * + * This header defines the layout of the .mod_lineinfo section embedded + * in loadable kernel modules. It is dual-use: included from both the + * kernel and the userspace gen_lineinfo tool. + * + * Section layout (all values in target-native endianness): + * + * struct mod_lineinfo_header (16 bytes) + * u32 addrs[num_entries] -- offsets from .text base, sorted + * u16 file_ids[num_entries] -- parallel to addrs + * <2-byte pad if num_entries is odd> + * u32 lines[num_entries] -- parallel to addrs + * u32 file_offsets[num_files] -- byte offset into filenames[] + * char filenames[filenames_size] -- concatenated NUL-terminated strings + */ +#ifndef _LINUX_MOD_LINEINFO_H +#define _LINUX_MOD_LINEINFO_H + +#ifdef __KERNEL__ +#include +#else +#include +typedef uint32_t u32; +typedef uint16_t u16; +#endif + +struct mod_lineinfo_header { + u32 num_entries; + u32 num_files; + u32 filenames_size; /* total bytes of concatenated filenames */ + u32 reserved; /* padding, must be 0 */ +}; + +/* Offset helpers: compute byte offset from start of section to each array= */ + +static inline u32 mod_lineinfo_addrs_off(void) +{ + return sizeof(struct mod_lineinfo_header); +} + +static inline u32 mod_lineinfo_file_ids_off(u32 num_entries) +{ + return mod_lineinfo_addrs_off() + num_entries * sizeof(u32); +} + +static inline u32 mod_lineinfo_lines_off(u32 num_entries) +{ + /* u16 file_ids[] may need 2-byte padding to align lines[] to 4 bytes */ + u32 off =3D mod_lineinfo_file_ids_off(num_entries) + + num_entries * sizeof(u16); + return (off + 3) & ~3u; +} + +static inline u32 mod_lineinfo_file_offsets_off(u32 num_entries) +{ + return mod_lineinfo_lines_off(num_entries) + num_entries * sizeof(u32); +} + +static inline u32 mod_lineinfo_filenames_off(u32 num_entries, u32 num_file= s) +{ + return mod_lineinfo_file_offsets_off(num_entries) + + num_files * sizeof(u32); +} + +#endif /* _LINUX_MOD_LINEINFO_H */ diff --git a/include/linux/module.h b/include/linux/module.h index 14f391b186c6d..1c5840e736ec7 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -508,6 +508,10 @@ struct module { void *btf_data; void *btf_base_data; #endif +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + void *lineinfo_data; /* .mod_lineinfo section in MOD_RODATA */ + unsigned int lineinfo_data_size; +#endif #ifdef CONFIG_JUMP_LABEL struct jump_entry *jump_entries; unsigned int num_jump_entries; @@ -1021,6 +1025,21 @@ static inline unsigned long find_kallsyms_symbol_val= ue(struct module *mod, =20 #endif /* CONFIG_MODULES && CONFIG_KALLSYMS */ =20 +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES +bool module_lookup_lineinfo(struct module *mod, unsigned long addr, + unsigned long sym_start, + const char **file, unsigned int *line); +#else +static inline bool module_lookup_lineinfo(struct module *mod, + unsigned long addr, + unsigned long sym_start, + const char **file, + unsigned int *line) +{ + return false; +} +#endif + /* Define __free(module_put) macro for struct module *. */ DEFINE_FREE(module_put, struct module *, if (_T) module_put(_T)) =20 diff --git a/init/Kconfig b/init/Kconfig index c39f27e6393a8..bf53275bc405a 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2070,6 +2070,19 @@ config KALLSYMS_LINEINFO =20 If unsure, say N. =20 +config KALLSYMS_LINEINFO_MODULES + bool "Embed source file:line information in module stack traces" + depends on KALLSYMS_LINEINFO && MODULES + help + Extends KALLSYMS_LINEINFO to loadable kernel modules. Each .ko + gets a lineinfo table generated from its DWARF data at build time, + so stack traces from module code include (file.c:123) annotations. + + Requires elfutils (libdw-dev/elfutils-devel) on the build host. + Increases .ko sizes by approximately 10 bytes per DWARF line entry. + + If unsure, say N. + # end of the "standard kernel features (expert users)" menu =20 config ARCH_HAS_MEMBARRIER_CALLBACKS diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 2b9c9d6322a3e..cea74992e5427 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -554,13 +554,27 @@ static int __sprint_symbol(char *buffer, unsigned lon= g address, } =20 #ifdef CONFIG_KALLSYMS_LINEINFO - if (!modname) { + { const char *li_file; unsigned int li_line; unsigned long sym_start =3D address - offset; - - if (kallsyms_lookup_lineinfo(address, sym_start, - &li_file, &li_line)) + bool found =3D false; + + if (!modname) + found =3D kallsyms_lookup_lineinfo(address, sym_start, + &li_file, &li_line); +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + else { + struct module *mod =3D __module_address(address); + + if (mod) + found =3D module_lookup_lineinfo(mod, address, + sym_start, + &li_file, + &li_line); + } +#endif + if (found) len +=3D snprintf(buffer + len, KSYM_SYMBOL_LEN - len, " (%s:%u)", li_file, li_line); } diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c index 0fc11e45df9b9..7af414bd65e79 100644 --- a/kernel/module/kallsyms.c +++ b/kernel/module/kallsyms.c @@ -494,3 +494,98 @@ int module_kallsyms_on_each_symbol(const char *modname, mutex_unlock(&module_mutex); return ret; } + +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES +#include + +/* + * Look up source file:line for an address within a loaded module. + * Uses the .mod_lineinfo section embedded in the .ko at build time. + * + * Safe in NMI/panic context: no locks, no allocations. + * Caller must hold RCU read lock (or be in a context where the module + * cannot be unloaded). + */ +bool module_lookup_lineinfo(struct module *mod, unsigned long addr, + unsigned long sym_start, + const char **file, unsigned int *line) +{ + const struct mod_lineinfo_header *hdr; + const void *base; + const u32 *addrs, *lines, *file_offsets; + const u16 *file_ids; + const char *filenames; + u32 num_entries, num_files, filenames_size; + unsigned long text_base; + unsigned int offset; + unsigned long long raw_offset; + unsigned int low, high, mid; + u16 file_id; + + base =3D mod->lineinfo_data; + if (!base) + return false; + + if (mod->lineinfo_data_size < sizeof(*hdr)) + return false; + + hdr =3D base; + num_entries =3D hdr->num_entries; + num_files =3D hdr->num_files; + filenames_size =3D hdr->filenames_size; + + if (num_entries =3D=3D 0) + return false; + + /* Validate section is large enough for all arrays */ + if (mod->lineinfo_data_size < + mod_lineinfo_filenames_off(num_entries, num_files) + filenames_size) + return false; + + addrs =3D base + mod_lineinfo_addrs_off(); + file_ids =3D base + mod_lineinfo_file_ids_off(num_entries); + lines =3D base + mod_lineinfo_lines_off(num_entries); + file_offsets =3D base + mod_lineinfo_file_offsets_off(num_entries); + filenames =3D base + mod_lineinfo_filenames_off(num_entries, num_files); + + /* Compute offset from module .text base */ + text_base =3D (unsigned long)mod->mem[MOD_TEXT].base; + if (addr < text_base) + return false; + + raw_offset =3D addr - text_base; + if (raw_offset > UINT_MAX) + return false; + offset =3D (unsigned int)raw_offset; + + /* Binary search for largest entry <=3D offset */ + low =3D 0; + high =3D num_entries; + while (low < high) { + mid =3D low + (high - low) / 2; + if (addrs[mid] <=3D offset) + low =3D mid + 1; + else + high =3D mid; + } + + if (low =3D=3D 0) + return false; + low--; + + /* Ensure the matched entry belongs to the same symbol */ + if (text_base + addrs[low] < sym_start) + return false; + + file_id =3D file_ids[low]; + if (file_id >=3D num_files) + return false; + + if (file_offsets[file_id] >=3D filenames_size) + return false; + + *file =3D &filenames[file_offsets[file_id]]; + *line =3D lines[low]; + return true; +} +#endif /* CONFIG_KALLSYMS_LINEINFO_MODULES */ diff --git a/kernel/module/main.c b/kernel/module/main.c index 2bac4c7cd019a..7b6ff9f7411b0 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -2648,6 +2648,10 @@ static int find_module_sections(struct module *mod, = struct load_info *info) mod->btf_base_data =3D any_section_objs(info, ".BTF.base", 1, &mod->btf_base_data_size); #endif +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + mod->lineinfo_data =3D any_section_objs(info, ".mod_lineinfo", 1, + &mod->lineinfo_data_size); +#endif #ifdef CONFIG_JUMP_LABEL mod->jump_entries =3D section_objs(info, "__jump_table", sizeof(*mod->jump_entries), diff --git a/scripts/Makefile b/scripts/Makefile index ffe89875b3295..651df2a867ffb 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -5,6 +5,7 @@ =20 hostprogs-always-$(CONFIG_KALLSYMS) +=3D kallsyms hostprogs-always-$(CONFIG_KALLSYMS_LINEINFO) +=3D gen_lineinfo +hostprogs-always-$(CONFIG_KALLSYMS_LINEINFO_MODULES) +=3D gen_lineinfo hostprogs-always-$(BUILD_C_RECORDMCOUNT) +=3D recordmcount hostprogs-always-$(CONFIG_BUILDTIME_TABLE_SORT) +=3D sorttable hostprogs-always-$(CONFIG_ASN1) +=3D asn1_compiler diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index adcbcde16a071..3941cf624526b 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -46,6 +46,9 @@ quiet_cmd_btf_ko =3D BTF [M] $@ $(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmli= nux $@; \ fi; =20 +quiet_cmd_lineinfo_ko =3D LINEINFO [M] $@ + cmd_lineinfo_ko =3D $(CONFIG_SHELL) $(srctree)/scripts/gen-mod-linei= nfo.sh $@ + # Same as newer-prereqs, but allows to exclude specified extra dependencies newer_prereqs_except =3D $(filter-out $(PHONY) $(1),$?) =20 @@ -59,6 +62,9 @@ if_changed_except =3D $(if $(call newer_prereqs_except,$(= 2))$(cmd-check), \ +$(call if_changed_except,ld_ko_o,$(objtree)/vmlinux) ifdef CONFIG_DEBUG_INFO_BTF_MODULES +$(if $(newer-prereqs),$(call cmd,btf_ko)) +endif +ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + +$(if $(newer-prereqs),$(call cmd,lineinfo_ko)) endif +$(call cmd,check_tracepoint) =20 diff --git a/scripts/gen-mod-lineinfo.sh b/scripts/gen-mod-lineinfo.sh new file mode 100755 index 0000000000000..fa8a914b8363b --- /dev/null +++ b/scripts/gen-mod-lineinfo.sh @@ -0,0 +1,48 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# gen-mod-lineinfo.sh - Embed source line info into a kernel module (.ko) +# +# Reads DWARF from the .ko, generates a .mod_lineinfo section, and +# embeds it back into the .ko. Modeled on scripts/gen-btf.sh. + +set -e + +if [ $# -ne 1 ]; then + echo "Usage: $0 " >&2 + exit 1 +fi + +KO=3D"$1" + +cleanup() { + rm -f "${KO}.lineinfo.S" "${KO}.lineinfo.o" "${KO}.lineinfo.bin" +} +trap cleanup EXIT + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + +# Generate assembly from DWARF -- if it fails (no DWARF), silently skip +if ! ${objtree}/scripts/gen_lineinfo --module "${KO}" > "${KO}.lineinfo.S"= 2>/dev/null; then + exit 0 +fi + +# Compile assembly to object file +${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \ + ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_MODULE} \ + -c -o "${KO}.lineinfo.o" "${KO}.lineinfo.S" + +# Extract raw section content +${OBJCOPY} -O binary --only-section=3D.mod_lineinfo \ + "${KO}.lineinfo.o" "${KO}.lineinfo.bin" + +# Embed into the .ko with alloc,readonly flags +${OBJCOPY} --add-section ".mod_lineinfo=3D${KO}.lineinfo.bin" \ + --set-section-flags .mod_lineinfo=3Dalloc,readonly \ + "${KO}" + +exit 0 diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c index 9eebfaca5857c..609de59f47ffd 100644 --- a/scripts/gen_lineinfo.c +++ b/scripts/gen_lineinfo.c @@ -23,8 +23,16 @@ #include #include =20 +#include "../include/linux/mod_lineinfo.h" + +static int module_mode; + static unsigned int skipped_overflow; =20 +/* .text range for module mode (keep only runtime code) */ +static unsigned long long text_section_start; +static unsigned long long text_section_end; + struct line_entry { unsigned int offset; /* offset from _text */ unsigned int file_id; @@ -123,26 +131,46 @@ static unsigned int find_or_add_file(const char *name) * * For absolute paths, strip the comp_dir prefix (from DWARF) to get * a kernel-tree-relative path, or fall back to the basename. + * + * For relative paths (common in modules), libdw may produce a bogus + * doubled path like "net/foo/bar.c/net/foo/bar.c" due to ET_REL DWARF + * quirks. Detect and strip such duplicates. */ static const char *make_relative(const char *path, const char *comp_dir) { const char *p; =20 - /* If already relative, use as-is */ - if (path[0] !=3D '/') - return path; + if (path[0] =3D=3D '/') { + /* Try comp_dir prefix from DWARF */ + if (comp_dir) { + size_t len =3D strlen(comp_dir); =20 - /* comp_dir from DWARF is the most reliable method */ - if (comp_dir) { - size_t len =3D strlen(comp_dir); + if (!strncmp(path, comp_dir, len) && path[len] =3D=3D '/') + return path + len + 1; + } =20 - if (!strncmp(path, comp_dir, len) && path[len] =3D=3D '/') - return path + len + 1; + /* Fall back to basename */ + p =3D strrchr(path, '/'); + return p ? p + 1 : path; } =20 - /* Fall back to basename */ - p =3D strrchr(path, '/'); - return p ? p + 1 : path; + /* + * Relative path =E2=80=94 check for duplicated-path quirk from libdw + * on ET_REL files (e.g., "a/b.c/a/b.c" =E2=86=92 "a/b.c"). + */ + { + size_t len =3D strlen(path); + + for (p =3D path; (p =3D strchr(p, '/')) !=3D NULL; p++) { + size_t prefix =3D p - path; + size_t rest =3D len - prefix - 1; + + if (rest =3D=3D prefix && !memcmp(path, p + 1, prefix)) + return p + 1; + } + } + + return path; } =20 static int compare_entries(const void *a, const void *b) @@ -194,6 +222,29 @@ static unsigned long long find_text_addr(Elf *elf) exit(1); } =20 +static void find_text_section_range(Elf *elf) +{ + Elf_Scn *scn =3D NULL; + GElf_Shdr shdr; + size_t shstrndx; + + if (elf_getshdrstrndx(elf, &shstrndx) !=3D 0) + return; + + while ((scn =3D elf_nextscn(elf, scn)) !=3D NULL) { + const char *name; + + if (!gelf_getshdr(scn, &shdr)) + continue; + name =3D elf_strptr(elf, shstrndx, shdr.sh_name); + if (name && !strcmp(name, ".text")) { + text_section_start =3D shdr.sh_addr; + text_section_end =3D shdr.sh_addr + shdr.sh_size; + return; + } + } +} + static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr) { Dwarf_Off off =3D 0, next_off; @@ -241,6 +292,16 @@ static void process_dwarf(Dwarf *dwarf, unsigned long = long text_addr) if (addr < text_addr) continue; =20 + /* + * In module mode, keep only .text addresses. + * In ET_REL .ko files, .init.text/.exit.text may + * overlap with .text address ranges, so we must + * explicitly check against the .text bounds. + */ + if (module_mode && text_section_end > text_section_start && + (addr < text_section_start || addr >=3D text_section_end)) + continue; + { unsigned long long raw_offset =3D addr - text_addr; =20 @@ -374,6 +435,63 @@ static void output_assembly(void) printf("\n"); } =20 +static void output_module_assembly(void) +{ + unsigned int filenames_size =3D 0; + + for (unsigned int i =3D 0; i < num_files; i++) + filenames_size +=3D strlen(files[i].name) + 1; + + printf("/* SPDX-License-Identifier: GPL-2.0 */\n"); + printf("/*\n"); + printf(" * Automatically generated by scripts/gen_lineinfo --module\n"); + printf(" * Do not edit.\n"); + printf(" */\n\n"); + + printf("\t.section .mod_lineinfo, \"a\"\n\n"); + + /* Header: num_entries, num_files, filenames_size, reserved */ + printf("\t.balign 4\n"); + printf("\t.long %u\n", num_entries); + printf("\t.long %u\n", num_files); + printf("\t.long %u\n", filenames_size); + printf("\t.long 0\n\n"); + + /* addrs[] */ + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.long 0x%x\n", entries[i].offset); + if (num_entries) + printf("\n"); + + /* file_ids[] */ + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.short %u\n", entries[i].file_id); + + /* Padding to align lines[] to 4 bytes */ + if (num_entries & 1) + printf("\t.short 0\n"); + if (num_entries) + printf("\n"); + + /* lines[] */ + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.long %u\n", entries[i].line); + if (num_entries) + printf("\n"); + + /* file_offsets[] */ + for (unsigned int i =3D 0; i < num_files; i++) + printf("\t.long %u\n", files[i].str_offset); + if (num_files) + printf("\n"); + + /* filenames[] */ + for (unsigned int i =3D 0; i < num_files; i++) + print_escaped_asciz(files[i].name); + if (num_files) + printf("\n"); +} + int main(int argc, char *argv[]) { int fd; @@ -381,8 +499,14 @@ int main(int argc, char *argv[]) Dwarf *dwarf; unsigned long long text_addr; =20 + if (argc >=3D 2 && !strcmp(argv[1], "--module")) { + module_mode =3D 1; + argv++; + argc--; + } + if (argc !=3D 2) { - fprintf(stderr, "Usage: %s \n", argv[0]); + fprintf(stderr, "Usage: %s [--module] \n", argv[0]); return 1; } =20 @@ -402,7 +526,18 @@ int main(int argc, char *argv[]) return 1; } =20 - text_addr =3D find_text_addr(elf); + if (module_mode) { + /* + * .ko files are ET_REL after ld -r. libdw applies + * relocations using section addresses, so DWARF addresses + * include the .text sh_addr. Use .text sh_addr as the + * base so offsets are .text-relative. + */ + find_text_section_range(elf); + text_addr =3D text_section_start; + } else { + text_addr =3D find_text_addr(elf); + } =20 dwarf =3D dwarf_begin_elf(elf, DWARF_C_READ, NULL); if (!dwarf) { @@ -428,7 +563,10 @@ int main(int argc, char *argv[]) fprintf(stderr, "lineinfo: %u entries, %u files\n", num_entries, num_files); =20 - output_assembly(); + if (module_mode) + output_module_assembly(); + else + output_assembly(); =20 dwarf_end(dwarf); elf_end(elf); --=20 2.51.0 From nobody Thu Apr 9 19:17:04 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BE393D5649; Tue, 3 Mar 2026 18:22:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772562131; cv=none; b=eGcgsHEZn+MycF9IJe7OMm4+BMRnfduSqY0+8u3Yk+Sfl1YpuIvbpNd3DFtbO2Cnn81od1TVRHf82UbVmIQdTGSk0vjqR58Baj11Yb8M0JCaCNrBl5LOAfQgQTIYKn8tF3LVHI095ovA1f50xUpimkq5BaJMW+O4iyX9wi22fnk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772562131; c=relaxed/simple; bh=nDWvOEm8xdTNZOjDXW7jKTX5VdLCJzXdd/PYX/jWghs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jvTkQ3913U3UDxlB2QnmDqcnpDBBNyFR0GvGxOKAZYJLicNgjts7mIJM6tKNeluLc2t7IJc7nnu9PU6dBgWh66Z0IR2fmBldPQiudZVonpEPu9zrdk2sE8GKUi6lQ/JvbOXdc7SlFFU9t7yTnOiZ32kZHM7UGNOaTyfQJso+UNk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LEGAoSCw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LEGAoSCw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 380BCC19425; Tue, 3 Mar 2026 18:22:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772562130; bh=nDWvOEm8xdTNZOjDXW7jKTX5VdLCJzXdd/PYX/jWghs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LEGAoSCwwHr4B0nSqPeQJhui3JhtxP/g/inp8A28TUBnAkqopJ93+A9GZVO+4vckQ 7q5q1upK9YgWwdOpsJGM/uCooVGlDCJlWe1VT1Wfw5gK36LHeFoF0dafEVBQSbfpn4 ibkt1kYKbafSz0/lUw97pRo1n5TAuRVXOwRefg7lYjxs+jix0gcpCgX+NWK3tGTi4z CWLX1lff8yKSFN51T5BOG+31pqW/sAtYVbfRGhd6M6BDAjWVfW96X+SyBxTdTxwFh0 NV8o85TtSeKbxVSScTM8I31wdl97eYEf2W4ARlcvj1KZ5rPA4da+R6U+rinSSzXPGa EMENMjWEjnnsg== From: Sasha Levin To: Andrew Morton , Masahiro Yamada , Luis Chamberlain , Linus Torvalds , Richard Weinberger , Juergen Gross , Geert Uytterhoeven , James Bottomley Cc: Jonathan Corbet , Nathan Chancellor , Nicolas Schier , Petr Pavlu , Daniel Gomez , Greg KH , Petr Mladek , Steven Rostedt , Kees Cook , Peter Zijlstra , Thorsten Leemhuis , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-doc@vger.kernel.org, Sasha Levin Subject: [PATCH 3/3] kallsyms: delta-compress lineinfo tables for ~2.7x size reduction Date: Tue, 3 Mar 2026 13:21:03 -0500 Message-ID: <20260303182103.3523438-4-sashal@kernel.org> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20260303182103.3523438-1-sashal@kernel.org> References: <20260303182103.3523438-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace the flat uncompressed parallel arrays (lineinfo_addrs[], lineinfo_file_ids[], lineinfo_lines[]) with a block-indexed, delta-encoded, ULEB128 varint compressed format. The sorted address array has small deltas between consecutive entries (typically 1-50 bytes), file IDs have high locality (delta often 0, same file), and line numbers change slowly. Delta-encoding followed by ULEB128 varint compression shrinks most values from 4 bytes to 1. Entries are grouped into blocks of 64. A small uncompressed block index (first addr + byte offset per block) enables O(log(N/64)) binary search, followed by sequential decode of at most 64 varints within the matching block. All decode state lives on the stack -- zero allocations, still safe for NMI/panic context. Measured on a defconfig+debug x86_64 build (3,017,154 entries, 4,822 source files, 47,144 blocks): Before (flat arrays): lineinfo_addrs[] 12,068,616 bytes (u32 x 3.0M) lineinfo_file_ids[] 6,034,308 bytes (u16 x 3.0M) lineinfo_lines[] 12,068,616 bytes (u32 x 3.0M) Total: 30,171,540 bytes (28.8 MiB, 10.0 bytes/entry) After (block-indexed delta + ULEB128): lineinfo_block_addrs[] 188,576 bytes (184 KiB) lineinfo_block_offsets[] 188,576 bytes (184 KiB) lineinfo_data[] 10,926,128 bytes (10.4 MiB) Total: 11,303,280 bytes (10.8 MiB, 3.7 bytes/entry) Savings: 18.0 MiB (2.7x reduction) Booted in QEMU and verified with SysRq-l that annotations still work: default_idle+0x9/0x10 (arch/x86/kernel/process.c:767) default_idle_call+0x6c/0xb0 (kernel/sched/idle.c:122) do_idle+0x335/0x490 (kernel/sched/idle.c:191) cpu_startup_entry+0x4e/0x60 (kernel/sched/idle.c:429) rest_init+0x1aa/0x1b0 (init/main.c:760) Suggested-by: Juergen Gross Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Sasha Levin --- .../admin-guide/kallsyms-lineinfo.rst | 7 +- include/linux/mod_lineinfo.h | 103 ++++++++-- init/Kconfig | 8 +- kernel/kallsyms.c | 91 +++++++-- kernel/kallsyms_internal.h | 7 +- kernel/module/kallsyms.c | 107 +++++++--- scripts/gen_lineinfo.c | 192 ++++++++++++++---- scripts/kallsyms.c | 7 +- scripts/link-vmlinux.sh | 16 +- 9 files changed, 423 insertions(+), 115 deletions(-) diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentatio= n/admin-guide/kallsyms-lineinfo.rst index 21450569d5324..fe92c5dde16b3 100644 --- a/Documentation/admin-guide/kallsyms-lineinfo.rst +++ b/Documentation/admin-guide/kallsyms-lineinfo.rst @@ -76,10 +76,11 @@ Memory Overhead =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 The vmlinux lineinfo tables are stored in ``.rodata`` and typically add -approximately 44 MiB to the kernel image for a standard configuration -(~4.6 million DWARF line entries, ~10 bytes per entry after deduplication). +approximately 10-15 MiB to the kernel image for a standard configuration +(~4.6 million DWARF line entries, ~2-3 bytes per entry after delta +compression). =20 -Per-module lineinfo adds approximately 10 bytes per DWARF line entry to ea= ch +Per-module lineinfo adds approximately 2-3 bytes per DWARF line entry to e= ach ``.ko`` file. =20 Known Limitations diff --git a/include/linux/mod_lineinfo.h b/include/linux/mod_lineinfo.h index d62e9608f0f82..ab758acfadceb 100644 --- a/include/linux/mod_lineinfo.h +++ b/include/linux/mod_lineinfo.h @@ -8,13 +8,19 @@ * * Section layout (all values in target-native endianness): * - * struct mod_lineinfo_header (16 bytes) - * u32 addrs[num_entries] -- offsets from .text base, sorted - * u16 file_ids[num_entries] -- parallel to addrs - * <2-byte pad if num_entries is odd> - * u32 lines[num_entries] -- parallel to addrs + * struct mod_lineinfo_header (24 bytes) + * u32 block_addrs[num_blocks] -- first addr per block, for binary se= arch + * u32 block_offsets[num_blocks] -- byte offset into compressed data st= ream + * u8 data[data_size] -- ULEB128 delta-compressed entries * u32 file_offsets[num_files] -- byte offset into filenames[] * char filenames[filenames_size] -- concatenated NUL-terminated strings + * + * Compressed stream format (per block of LINEINFO_BLOCK_ENTRIES entries): + * Entry 0: file_id (ULEB128), line (ULEB128) + * addr is in block_addrs[] + * Entry 1..N: addr_delta (ULEB128), + * file_id_delta (zigzag-encoded ULEB128), + * line_delta (zigzag-encoded ULEB128) */ #ifndef _LINUX_MOD_LINEINFO_H #define _LINUX_MOD_LINEINFO_H @@ -25,44 +31,107 @@ #include typedef uint32_t u32; typedef uint16_t u16; +typedef uint8_t u8; #endif =20 +#define LINEINFO_BLOCK_ENTRIES 64 + struct mod_lineinfo_header { u32 num_entries; u32 num_files; u32 filenames_size; /* total bytes of concatenated filenames */ + u32 num_blocks; + u32 data_size; /* total bytes of compressed data stream */ u32 reserved; /* padding, must be 0 */ }; =20 /* Offset helpers: compute byte offset from start of section to each array= */ =20 -static inline u32 mod_lineinfo_addrs_off(void) +static inline u32 mod_lineinfo_block_addrs_off(void) { return sizeof(struct mod_lineinfo_header); } =20 -static inline u32 mod_lineinfo_file_ids_off(u32 num_entries) +static inline u32 mod_lineinfo_block_offsets_off(u32 num_blocks) { - return mod_lineinfo_addrs_off() + num_entries * sizeof(u32); + return mod_lineinfo_block_addrs_off() + num_blocks * sizeof(u32); } =20 -static inline u32 mod_lineinfo_lines_off(u32 num_entries) +static inline u32 mod_lineinfo_data_off(u32 num_blocks) { - /* u16 file_ids[] may need 2-byte padding to align lines[] to 4 bytes */ - u32 off =3D mod_lineinfo_file_ids_off(num_entries) + - num_entries * sizeof(u16); - return (off + 3) & ~3u; + return mod_lineinfo_block_offsets_off(num_blocks) + + num_blocks * sizeof(u32); } =20 -static inline u32 mod_lineinfo_file_offsets_off(u32 num_entries) +static inline u32 mod_lineinfo_file_offsets_off(u32 num_blocks, u32 data_s= ize) { - return mod_lineinfo_lines_off(num_entries) + num_entries * sizeof(u32); + return mod_lineinfo_data_off(num_blocks) + data_size; } =20 -static inline u32 mod_lineinfo_filenames_off(u32 num_entries, u32 num_file= s) +static inline u32 mod_lineinfo_filenames_off(u32 num_blocks, u32 data_size, + u32 num_files) { - return mod_lineinfo_file_offsets_off(num_entries) + + return mod_lineinfo_file_offsets_off(num_blocks, data_size) + num_files * sizeof(u32); } =20 +/* Zigzag encoding: map signed to unsigned so small magnitudes are small */ +static inline u32 zigzag_encode(int32_t v) +{ + return ((u32)v << 1) ^ (u32)(v >> 31); +} + +static inline int32_t zigzag_decode(u32 v) +{ + return (int32_t)((v >> 1) ^ -(v & 1)); +} + +/* + * Read a ULEB128 varint from a byte stream. + * Returns the decoded value and advances *pos past the encoded bytes. + * If *pos would exceed 'end', returns 0 and sets *pos =3D end (safe for + * NMI/panic context -- no crash, just a missed annotation). + */ +static inline u32 lineinfo_read_uleb128(const u8 *data, u32 *pos, u32 end) +{ + u32 result =3D 0; + unsigned int shift =3D 0; + + while (*pos < end) { + u8 byte =3D data[*pos]; + (*pos)++; + result |=3D (u32)(byte & 0x7f) << shift; + if (!(byte & 0x80)) + return result; + shift +=3D 7; + if (shift >=3D 32) { + /* Malformed -- skip remaining continuation bytes */ + while (*pos < end && (data[*pos] & 0x80)) + (*pos)++; + if (*pos < end) + (*pos)++; + return result; + } + } + return result; +} + +/* Write a ULEB128 varint -- build tool only */ +#ifndef __KERNEL__ +static inline unsigned int lineinfo_write_uleb128(u8 *buf, u32 value) +{ + unsigned int len =3D 0; + + do { + u8 byte =3D value & 0x7f; + + value >>=3D 7; + if (value) + byte |=3D 0x80; + buf[len++] =3D byte; + } while (value); + return len; +} +#endif /* !__KERNEL__ */ + #endif /* _LINUX_MOD_LINEINFO_H */ diff --git a/init/Kconfig b/init/Kconfig index bf53275bc405a..6e3795b3dbd62 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2065,8 +2065,9 @@ config KALLSYMS_LINEINFO anon_vma_clone+0x2ed/0xcf0 (mm/rmap.c:412) =20 This requires elfutils (libdw-dev/elfutils-devel) on the build host. - Adds approximately 44MB to a typical kernel image (10 bytes per - DWARF line-table entry, ~4.6M entries for a typical config). + Adds approximately 10-15MB to a typical kernel image (~2-3 bytes + per entry after delta compression, ~4.6M entries for a typical + config). =20 If unsure, say N. =20 @@ -2079,7 +2080,8 @@ config KALLSYMS_LINEINFO_MODULES so stack traces from module code include (file.c:123) annotations. =20 Requires elfutils (libdw-dev/elfutils-devel) on the build host. - Increases .ko sizes by approximately 10 bytes per DWARF line entry. + Increases .ko sizes by approximately 2-3 bytes per DWARF line + entry after delta compression. =20 If unsure, say N. =20 diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index cea74992e5427..de4aa8fcfd69d 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -468,14 +468,20 @@ static int append_buildid(char *buffer, const char = *modname, #endif /* CONFIG_STACKTRACE_BUILD_ID */ =20 #ifdef CONFIG_KALLSYMS_LINEINFO +#include + bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start, const char **file, unsigned int *line) { unsigned long long raw_offset; - unsigned int offset, low, high, mid, file_id; - unsigned long line_addr; - - if (!lineinfo_num_entries) + unsigned int offset, low, high, mid, block; + unsigned int cur_addr, cur_file_id, cur_line; + unsigned int best_file_id =3D 0, best_line =3D 0; + unsigned int block_entries, data_end; + bool found =3D false; + u32 pos; + + if (!lineinfo_num_entries || !lineinfo_num_blocks) return false; =20 /* Compute offset from _text */ @@ -487,12 +493,12 @@ bool kallsyms_lookup_lineinfo(unsigned long addr, uns= igned long sym_start, return false; offset =3D (unsigned int)raw_offset; =20 - /* Binary search for largest entry <=3D offset */ + /* Binary search on block_addrs[] to find the right block */ low =3D 0; - high =3D lineinfo_num_entries; + high =3D lineinfo_num_blocks; while (low < high) { mid =3D low + (high - low) / 2; - if (lineinfo_addrs[mid] <=3D offset) + if (lineinfo_block_addrs[mid] <=3D offset) low =3D mid + 1; else high =3D mid; @@ -500,25 +506,68 @@ bool kallsyms_lookup_lineinfo(unsigned long addr, uns= igned long sym_start, =20 if (low =3D=3D 0) return false; - low--; + block =3D low - 1; =20 - /* - * Validate that the matched lineinfo entry belongs to the same - * symbol. Without this check, assembly routines or other - * functions lacking DWARF data would inherit the file:line of - * a preceding C function. - */ - line_addr =3D (unsigned long)_text + lineinfo_addrs[low]; - if (line_addr < sym_start) - return false; + /* How many entries in this block? */ + block_entries =3D LINEINFO_BLOCK_ENTRIES; + if (block =3D=3D lineinfo_num_blocks - 1) { + unsigned int remaining =3D lineinfo_num_entries - block * LINEINFO_BLOCK= _ENTRIES; + + if (remaining < block_entries) + block_entries =3D remaining; + } + + /* Determine end of this block's data in the compressed stream */ + if (block + 1 < lineinfo_num_blocks) + data_end =3D lineinfo_block_offsets[block + 1]; + else + data_end =3D UINT_MAX; /* last block: read to end */ + + /* Decode entry 0: addr from block_addrs, file_id and line from stream */ + pos =3D lineinfo_block_offsets[block]; + cur_addr =3D lineinfo_block_addrs[block]; + cur_file_id =3D lineinfo_read_uleb128(lineinfo_data, &pos, data_end); + cur_line =3D lineinfo_read_uleb128(lineinfo_data, &pos, data_end); + + /* Check entry 0 */ + if (cur_addr <=3D offset && + (unsigned long)_text + cur_addr >=3D sym_start) { + best_file_id =3D cur_file_id; + best_line =3D cur_line; + found =3D true; + } =20 - file_id =3D lineinfo_file_ids[low]; - *line =3D lineinfo_lines[low]; + /* Decode entries 1..N */ + for (unsigned int i =3D 1; i < block_entries; i++) { + unsigned int addr_delta; + int32_t file_delta, line_delta; + + addr_delta =3D lineinfo_read_uleb128(lineinfo_data, &pos, data_end); + file_delta =3D zigzag_decode(lineinfo_read_uleb128(lineinfo_data, &pos, = data_end)); + line_delta =3D zigzag_decode(lineinfo_read_uleb128(lineinfo_data, &pos, = data_end)); + + cur_addr +=3D addr_delta; + cur_file_id =3D (unsigned int)((int32_t)cur_file_id + file_delta); + cur_line =3D (unsigned int)((int32_t)cur_line + line_delta); + + if (cur_addr > offset) + break; + + if ((unsigned long)_text + cur_addr >=3D sym_start) { + best_file_id =3D cur_file_id; + best_line =3D cur_line; + found =3D true; + } + } + + if (!found) + return false; =20 - if (file_id >=3D lineinfo_num_files) + if (best_file_id >=3D lineinfo_num_files) return false; =20 - *file =3D &lineinfo_filenames[lineinfo_file_offsets[file_id]]; + *file =3D &lineinfo_filenames[lineinfo_file_offsets[best_file_id]]; + *line =3D best_line; return true; } #endif /* CONFIG_KALLSYMS_LINEINFO */ diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h index 868a1d5035212..691be44440395 100644 --- a/kernel/kallsyms_internal.h +++ b/kernel/kallsyms_internal.h @@ -17,10 +17,11 @@ extern const u8 kallsyms_seqs_of_names[]; =20 #ifdef CONFIG_KALLSYMS_LINEINFO extern const u32 lineinfo_num_entries; -extern const u32 lineinfo_addrs[]; -extern const u16 lineinfo_file_ids[]; -extern const u32 lineinfo_lines[]; extern const u32 lineinfo_num_files; +extern const u32 lineinfo_num_blocks; +extern const u32 lineinfo_block_addrs[]; +extern const u32 lineinfo_block_offsets[]; +extern const u8 lineinfo_data[]; extern const u32 lineinfo_file_offsets[]; extern const char lineinfo_filenames[]; #endif diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c index 7af414bd65e79..0ead1bb69de4e 100644 --- a/kernel/module/kallsyms.c +++ b/kernel/module/kallsyms.c @@ -512,15 +512,19 @@ bool module_lookup_lineinfo(struct module *mod, unsig= ned long addr, { const struct mod_lineinfo_header *hdr; const void *base; - const u32 *addrs, *lines, *file_offsets; - const u16 *file_ids; + const u32 *blk_addrs, *blk_offsets, *file_offsets; + const u8 *data; const char *filenames; - u32 num_entries, num_files, filenames_size; + u32 num_entries, num_files, filenames_size, num_blocks, data_size; unsigned long text_base; unsigned int offset; unsigned long long raw_offset; - unsigned int low, high, mid; - u16 file_id; + unsigned int low, high, mid, block; + unsigned int cur_addr, cur_file_id, cur_line; + unsigned int best_file_id =3D 0, best_line =3D 0; + unsigned int block_entries, data_end; + bool found =3D false; + u32 pos; =20 base =3D mod->lineinfo_data; if (!base) @@ -533,20 +537,24 @@ bool module_lookup_lineinfo(struct module *mod, unsig= ned long addr, num_entries =3D hdr->num_entries; num_files =3D hdr->num_files; filenames_size =3D hdr->filenames_size; + num_blocks =3D hdr->num_blocks; + data_size =3D hdr->data_size; =20 - if (num_entries =3D=3D 0) + if (num_entries =3D=3D 0 || num_blocks =3D=3D 0) return false; =20 /* Validate section is large enough for all arrays */ if (mod->lineinfo_data_size < - mod_lineinfo_filenames_off(num_entries, num_files) + filenames_size) + mod_lineinfo_filenames_off(num_blocks, data_size, num_files) + + filenames_size) return false; =20 - addrs =3D base + mod_lineinfo_addrs_off(); - file_ids =3D base + mod_lineinfo_file_ids_off(num_entries); - lines =3D base + mod_lineinfo_lines_off(num_entries); - file_offsets =3D base + mod_lineinfo_file_offsets_off(num_entries); - filenames =3D base + mod_lineinfo_filenames_off(num_entries, num_files); + blk_addrs =3D base + mod_lineinfo_block_addrs_off(); + blk_offsets =3D base + mod_lineinfo_block_offsets_off(num_blocks); + data =3D base + mod_lineinfo_data_off(num_blocks); + file_offsets =3D base + mod_lineinfo_file_offsets_off(num_blocks, data_si= ze); + filenames =3D base + mod_lineinfo_filenames_off(num_blocks, data_size, + num_files); =20 /* Compute offset from module .text base */ text_base =3D (unsigned long)mod->mem[MOD_TEXT].base; @@ -558,12 +566,12 @@ bool module_lookup_lineinfo(struct module *mod, unsig= ned long addr, return false; offset =3D (unsigned int)raw_offset; =20 - /* Binary search for largest entry <=3D offset */ + /* Binary search on block_addrs[] to find the right block */ low =3D 0; - high =3D num_entries; + high =3D num_blocks; while (low < high) { mid =3D low + (high - low) / 2; - if (addrs[mid] <=3D offset) + if (blk_addrs[mid] <=3D offset) low =3D mid + 1; else high =3D mid; @@ -571,21 +579,74 @@ bool module_lookup_lineinfo(struct module *mod, unsig= ned long addr, =20 if (low =3D=3D 0) return false; - low--; + block =3D low - 1; =20 - /* Ensure the matched entry belongs to the same symbol */ - if (text_base + addrs[low] < sym_start) + /* How many entries in this block? */ + block_entries =3D LINEINFO_BLOCK_ENTRIES; + if (block =3D=3D num_blocks - 1) { + unsigned int remaining =3D num_entries - block * LINEINFO_BLOCK_ENTRIES; + + if (remaining < block_entries) + block_entries =3D remaining; + } + + /* Determine end of this block's data in the compressed stream */ + if (block + 1 < num_blocks) + data_end =3D blk_offsets[block + 1]; + else + data_end =3D data_size; + + /* Decode entry 0: addr from block_addrs, file_id and line from stream */ + pos =3D blk_offsets[block]; + if (pos >=3D data_end) + return false; + + cur_addr =3D blk_addrs[block]; + cur_file_id =3D lineinfo_read_uleb128(data, &pos, data_end); + cur_line =3D lineinfo_read_uleb128(data, &pos, data_end); + + /* Check entry 0 */ + if (cur_addr <=3D offset && + text_base + cur_addr >=3D sym_start) { + best_file_id =3D cur_file_id; + best_line =3D cur_line; + found =3D true; + } + + /* Decode entries 1..N */ + for (unsigned int i =3D 1; i < block_entries; i++) { + unsigned int addr_delta; + int32_t file_delta, line_delta; + + addr_delta =3D lineinfo_read_uleb128(data, &pos, data_end); + file_delta =3D zigzag_decode(lineinfo_read_uleb128(data, &pos, data_end)= ); + line_delta =3D zigzag_decode(lineinfo_read_uleb128(data, &pos, data_end)= ); + + cur_addr +=3D addr_delta; + cur_file_id =3D (unsigned int)((int32_t)cur_file_id + file_delta); + cur_line =3D (unsigned int)((int32_t)cur_line + line_delta); + + if (cur_addr > offset) + break; + + if (text_base + cur_addr >=3D sym_start) { + best_file_id =3D cur_file_id; + best_line =3D cur_line; + found =3D true; + } + } + + if (!found) return false; =20 - file_id =3D file_ids[low]; - if (file_id >=3D num_files) + if (best_file_id >=3D num_files) return false; =20 - if (file_offsets[file_id] >=3D filenames_size) + if (file_offsets[best_file_id] >=3D filenames_size) return false; =20 - *file =3D &filenames[file_offsets[file_id]]; - *line =3D lines[low]; + *file =3D &filenames[file_offsets[best_file_id]]; + *line =3D best_line; return true; } #endif /* CONFIG_KALLSYMS_LINEINFO_MODULES */ diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c index 609de59f47ffd..9507ed9bcbe55 100644 --- a/scripts/gen_lineinfo.c +++ b/scripts/gen_lineinfo.c @@ -8,6 +8,9 @@ * file containing sorted lookup tables that the kernel uses to annotate * stack traces with source file:line information. * + * The output uses a block-indexed, delta-encoded, ULEB128-compressed form= at + * for ~3-4x size reduction compared to flat arrays. + * * Requires libdw from elfutils. */ =20 @@ -53,6 +56,15 @@ static struct file_entry *files; static unsigned int num_files; static unsigned int files_capacity; =20 +/* Compressed output */ +static unsigned char *compressed_data; +static unsigned int compressed_size; +static unsigned int compressed_capacity; + +static unsigned int *block_addrs; +static unsigned int *block_offsets; +static unsigned int num_blocks; + #define FILE_HASH_BITS 13 #define FILE_HASH_SIZE (1 << FILE_HASH_BITS) =20 @@ -352,6 +364,93 @@ static void deduplicate(void) num_entries =3D j + 1; } =20 +static void compressed_ensure(unsigned int need) +{ + if (compressed_size + need <=3D compressed_capacity) + return; + compressed_capacity =3D compressed_capacity ? compressed_capacity * 2 : 1= 024 * 1024; + while (compressed_capacity < compressed_size + need) + compressed_capacity *=3D 2; + compressed_data =3D realloc(compressed_data, compressed_capacity); + if (!compressed_data) { + fprintf(stderr, "out of memory\n"); + exit(1); + } +} + +static void compress_entries(void) +{ + unsigned int i, block; + + if (num_entries =3D=3D 0) { + num_blocks =3D 0; + return; + } + + num_blocks =3D (num_entries + LINEINFO_BLOCK_ENTRIES - 1) / LINEINFO_BLOC= K_ENTRIES; + block_addrs =3D calloc(num_blocks, sizeof(*block_addrs)); + block_offsets =3D calloc(num_blocks, sizeof(*block_offsets)); + if (!block_addrs || !block_offsets) { + fprintf(stderr, "out of memory\n"); + exit(1); + } + + for (block =3D 0; block < num_blocks; block++) { + unsigned int base =3D block * LINEINFO_BLOCK_ENTRIES; + unsigned int count =3D num_entries - base; + unsigned int prev_addr, prev_file_id, prev_line; + unsigned char buf[10]; /* max 5 bytes per ULEB128 */ + + if (count > LINEINFO_BLOCK_ENTRIES) + count =3D LINEINFO_BLOCK_ENTRIES; + + block_addrs[block] =3D entries[base].offset; + block_offsets[block] =3D compressed_size; + + /* Entry 0: file_id (ULEB128), line (ULEB128) */ + compressed_ensure(20); + compressed_size +=3D lineinfo_write_uleb128( + compressed_data + compressed_size, + entries[base].file_id); + compressed_size +=3D lineinfo_write_uleb128( + compressed_data + compressed_size, + entries[base].line); + + prev_addr =3D entries[base].offset; + prev_file_id =3D entries[base].file_id; + prev_line =3D entries[base].line; + + /* Entries 1..N: deltas */ + for (i =3D 1; i < count; i++) { + unsigned int idx =3D base + i; + unsigned int addr_delta; + int32_t file_delta, line_delta; + unsigned int n; + + addr_delta =3D entries[idx].offset - prev_addr; + file_delta =3D (int32_t)entries[idx].file_id - (int32_t)prev_file_id; + line_delta =3D (int32_t)entries[idx].line - (int32_t)prev_line; + + compressed_ensure(15); + n =3D lineinfo_write_uleb128(buf, addr_delta); + memcpy(compressed_data + compressed_size, buf, n); + compressed_size +=3D n; + + n =3D lineinfo_write_uleb128(buf, zigzag_encode(file_delta)); + memcpy(compressed_data + compressed_size, buf, n); + compressed_size +=3D n; + + n =3D lineinfo_write_uleb128(buf, zigzag_encode(line_delta)); + memcpy(compressed_data + compressed_size, buf, n); + compressed_size +=3D n; + + prev_addr =3D entries[idx].offset; + prev_file_id =3D entries[idx].file_id; + prev_line =3D entries[idx].line; + } + } +} + static void compute_file_offsets(void) { unsigned int offset =3D 0; @@ -395,28 +494,40 @@ static void output_assembly(void) printf("lineinfo_num_files:\n"); printf("\t.long %u\n\n", num_files); =20 - /* Sorted address offsets from _text */ - printf("\t.globl lineinfo_addrs\n"); + /* Number of blocks */ + printf("\t.globl lineinfo_num_blocks\n"); printf("\t.balign 4\n"); - printf("lineinfo_addrs:\n"); - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.long 0x%x\n", entries[i].offset); - printf("\n"); + printf("lineinfo_num_blocks:\n"); + printf("\t.long %u\n\n", num_blocks); =20 - /* File IDs, parallel to addrs (u16 -- supports up to 65535 files) */ - printf("\t.globl lineinfo_file_ids\n"); - printf("\t.balign 2\n"); - printf("lineinfo_file_ids:\n"); - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.short %u\n", entries[i].file_id); + /* Block first-addresses for binary search */ + printf("\t.globl lineinfo_block_addrs\n"); + printf("\t.balign 4\n"); + printf("lineinfo_block_addrs:\n"); + for (unsigned int i =3D 0; i < num_blocks; i++) + printf("\t.long 0x%x\n", block_addrs[i]); printf("\n"); =20 - /* Line numbers, parallel to addrs */ - printf("\t.globl lineinfo_lines\n"); + /* Block byte offsets into compressed stream */ + printf("\t.globl lineinfo_block_offsets\n"); printf("\t.balign 4\n"); - printf("lineinfo_lines:\n"); - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.long %u\n", entries[i].line); + printf("lineinfo_block_offsets:\n"); + for (unsigned int i =3D 0; i < num_blocks; i++) + printf("\t.long %u\n", block_offsets[i]); + printf("\n"); + + /* Compressed data stream */ + printf("\t.globl lineinfo_data\n"); + printf("lineinfo_data:\n"); + for (unsigned int i =3D 0; i < compressed_size; i++) { + if ((i % 16) =3D=3D 0) + printf("\t.byte "); + else + printf(","); + printf("0x%02x", compressed_data[i]); + if ((i % 16) =3D=3D 15 || i =3D=3D compressed_size - 1) + printf("\n"); + } printf("\n"); =20 /* File string offset table */ @@ -450,33 +561,38 @@ static void output_module_assembly(void) =20 printf("\t.section .mod_lineinfo, \"a\"\n\n"); =20 - /* Header: num_entries, num_files, filenames_size, reserved */ + /* Header: num_entries, num_files, filenames_size, num_blocks, data_size,= reserved */ printf("\t.balign 4\n"); printf("\t.long %u\n", num_entries); printf("\t.long %u\n", num_files); printf("\t.long %u\n", filenames_size); + printf("\t.long %u\n", num_blocks); + printf("\t.long %u\n", compressed_size); printf("\t.long 0\n\n"); =20 - /* addrs[] */ - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.long 0x%x\n", entries[i].offset); - if (num_entries) + /* block_addrs[] */ + for (unsigned int i =3D 0; i < num_blocks; i++) + printf("\t.long 0x%x\n", block_addrs[i]); + if (num_blocks) printf("\n"); =20 - /* file_ids[] */ - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.short %u\n", entries[i].file_id); - - /* Padding to align lines[] to 4 bytes */ - if (num_entries & 1) - printf("\t.short 0\n"); - if (num_entries) + /* block_offsets[] */ + for (unsigned int i =3D 0; i < num_blocks; i++) + printf("\t.long %u\n", block_offsets[i]); + if (num_blocks) printf("\n"); =20 - /* lines[] */ - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.long %u\n", entries[i].line); - if (num_entries) + /* compressed data[] */ + for (unsigned int i =3D 0; i < compressed_size; i++) { + if ((i % 16) =3D=3D 0) + printf("\t.byte "); + else + printf(","); + printf("0x%02x", compressed_data[i]); + if ((i % 16) =3D=3D 15 || i =3D=3D compressed_size - 1) + printf("\n"); + } + if (compressed_size) printf("\n"); =20 /* file_offsets[] */ @@ -558,10 +674,11 @@ int main(int argc, char *argv[]) skipped_overflow); =20 deduplicate(); + compress_entries(); compute_file_offsets(); =20 - fprintf(stderr, "lineinfo: %u entries, %u files\n", - num_entries, num_files); + fprintf(stderr, "lineinfo: %u entries, %u files, %u blocks, %u compressed= bytes\n", + num_entries, num_files, num_blocks, compressed_size); =20 if (module_mode) output_module_assembly(); @@ -577,6 +694,9 @@ int main(int argc, char *argv[]) for (unsigned int i =3D 0; i < num_files; i++) free(files[i].name); free(files); + free(compressed_data); + free(block_addrs); + free(block_offsets); =20 return 0; } diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 42662c4fbc6c9..94fbdad3df7c6 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -80,11 +80,12 @@ static bool is_ignored_symbol(const char *name, char ty= pe) { /* Ignore lineinfo symbols for kallsyms pass stability */ static const char * const lineinfo_syms[] =3D { - "lineinfo_addrs", - "lineinfo_file_ids", + "lineinfo_block_addrs", + "lineinfo_block_offsets", + "lineinfo_data", "lineinfo_file_offsets", "lineinfo_filenames", - "lineinfo_lines", + "lineinfo_num_blocks", "lineinfo_num_entries", "lineinfo_num_files", }; diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 640209f2e9eb9..3c122cf9b95c5 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -235,12 +235,16 @@ lineinfo_num_entries: .balign 4 lineinfo_num_files: .long 0 - .globl lineinfo_addrs -lineinfo_addrs: - .globl lineinfo_file_ids -lineinfo_file_ids: - .globl lineinfo_lines -lineinfo_lines: + .globl lineinfo_num_blocks + .balign 4 +lineinfo_num_blocks: + .long 0 + .globl lineinfo_block_addrs +lineinfo_block_addrs: + .globl lineinfo_block_offsets +lineinfo_block_offsets: + .globl lineinfo_data +lineinfo_data: .globl lineinfo_file_offsets lineinfo_file_offsets: .globl lineinfo_filenames --=20 2.51.0