From nobody Fri Oct 2 12:19:52 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6CDE638237D; Sat, 1 Aug 2026 14:32:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785594751; cv=none; b=UZYJ5zmlSY33eVZkYQdlXUae4CXZ9EuX4+vRItjrdAFjSarEtsBIqFtUw9Wonp5DMTStvMWnnjKQfIDKwZmFS+V8+nyqUhGexbi4G9P3mxJcJVvFawBr34mKYFZykdbaVYraboXnG1eFZhP12s23mRM0gshzfCHJx8mc2suzjgM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785594751; c=relaxed/simple; bh=FYAjL3I1sQm/CHklEwBfvh8EkoLfJDbmSyaumlIiihE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OcrSxurQlzet6odbYGUrNr++ae+JgcQ3/gzw/zp6H1sUvXriXcEXJAPzG+6FVpM9Yp+nlguMiygclt+ylWlDxlEnWxk10BHOgf17u5M3yg/Gmn5QGLmDpd/m3xm7pVhA7kvCJtJm3Ppr743ls2tHl3ShZ6b6RqV2fsA5aSgTu1Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fV0d72rT; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fV0d72rT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 058981F00ACA; Sat, 1 Aug 2026 14:32:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785594748; bh=gq4nKXXB8KALDCEhMnvkbKjRwzEnx9ZFNew+p6wlK/Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=fV0d72rTNP4WUeBMMD6hFFLO/ZzvyV0+FWp9kQflFtCmHs43VNNbB0dc57KLCrzK0 GjWcXrwcbkiDUJXH6KCVKB367sFsOODc4iT4wyt3T8xkUXhXFB+aYwQrRHD+9+/V9j m9QbfhW9Qn6puNHVCWmMKax/Fn1O5n6EWrfK7aWRaJifDuiilLvBOPR/eduP6Vi6wn jV9qsBI9pVN5IiYCgiOgogyCjpqidRjugJvm/m8rW6xCLYFWtD0mxO2DepkA/IPgTM tZwH4sppELgb+UBh7fWyO38TkNPshKPCcJnc+JBhTYWBFFgvvyunoQpYWGjymdEvbe SHwYr5gRT2xlA== From: Sasha Levin To: Andrew Morton , Masahiro Yamada , Luis Chamberlain , Linus Torvalds , Richard Weinberger , Juergen Gross , Geert Uytterhoeven , James Bottomley Cc: Sasha Levin , Jonathan Corbet , Nathan Chancellor , Nicolas Schier , Petr Pavlu , Daniel Gomez , Greg KH , Petr Mladek , Steven Rostedt , Kees Cook , Peter Zijlstra , Thorsten Leemhuis , Vlastimil Babka , Helge Deller , Randy Dunlap , Laurent Pinchart , Vivian Wang , Zhen Lei , Sami Tolvanen , linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-doc@vger.kernel.org Subject: [PATCH v8 1/4] kallsyms: embed source file:line info in kernel stack traces Date: Sat, 1 Aug 2026 10:32:15 -0400 Message-ID: <20260801143219.3975824-2-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260801143219.3975824-1-sashal@kernel.org> References: <20260801143219.3975824-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Add CONFIG_KALLSYMS_LINEINFO, which embeds a compact address-to-line lookup table in the kernel image so stack traces directly print source file and line number information: root@localhost:~# echo c > /proc/sysrq-trigger [ 11.201987] sysrq: Trigger a crash [ 11.202831] Kernel panic - not syncing: sysrq triggered crash [ 11.206218] Call Trace: [ 11.206501] [ 11.206749] dump_stack_lvl+0x5d/0x80 (lib/dump_stack.c:94) [ 11.207403] vpanic+0x36e/0x620 (kernel/panic.c:650) [ 11.208565] ? __lock_acquire+0x465/0x2240 (kernel/locking/lockdep.c:4= 674) [ 11.209324] panic+0xc9/0xd0 (kernel/panic.c:787) [ 11.211873] ? find_held_lock+0x2b/0x80 (kernel/locking/lockdep.c:5350) [ 11.212597] ? lock_release+0xd3/0x300 (kernel/locking/lockdep.c:5535) [ 11.213312] sysrq_handle_crash+0x1a/0x20 (drivers/tty/sysrq.c:154) [ 11.214005] __handle_sysrq.cold+0x66/0x256 (drivers/tty/sysrq.c:611) [ 11.214712] write_sysrq_trigger+0x65/0x80 (drivers/tty/sysrq.c:1221) [ 11.215424] proc_reg_write+0x1bd/0x3c0 (fs/proc/inode.c:330) [ 11.216061] vfs_write+0x1c6/0xff0 (fs/read_write.c:686) [ 11.218848] ksys_write+0xfa/0x200 (fs/read_write.c:740) [ 11.222394] do_syscall_64+0xf3/0x690 (arch/x86/entry/syscall_64.c:63) [ 11.223942] entry_SYSCALL_64_after_hwframe+0x77/0x7f (arch/x86/entry/= entry_64.S:121) At build time, a new host tool (scripts/gen_lineinfo) reads DWARF .debug_line from vmlinux using libdw (elfutils), extracts all address-to-file:line mappings, and generates an assembly file with sorted parallel arrays (offsets from _text, file IDs, and line numbers). These are linked into vmlinux as .rodata. At runtime, kallsyms_lookup_lineinfo() does a binary search on the table and __sprint_symbol() appends "(file:line)" to each stack frame. The lookup uses offsets from _text so it works with KASLR, requires no locks or allocations, and is safe in any context including panic. The feature requires CONFIG_DEBUG_INFO (for DWARF data) and libelf and libdw (from elfutils) on the build host. Memory footprint, measured with: make ARCH=3Dx86_64 O=3D$B defconfig ./scripts/config --file $B/.config -e DEBUG_INFO \ -e DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT make ARCH=3Dx86_64 O=3D$B olddefconfig && make ARCH=3Dx86_64 O=3D$B -j$(n= proc) strip -g $B/vmlinux -o vmlinux.nodbg && stat -c %s vmlinux.nodbg Table: 1,794,950 entries from 4,232 source files lineinfo_addrs[] 1,794,950 x u32 =3D 6.8 MiB lineinfo_file_ids[] 1,794,950 x u16 =3D 3.4 MiB lineinfo_lines[] 1,794,950 x u32 =3D 6.8 MiB file_offsets + filenames =3D 0.1 MiB Total .rodata increase: =3D 17.2 MiB vmlinux (stripped): 51.1 MiB -> 69.1 MiB (+18.0 MiB / +35.3%) That is the cost of the uncompressed format introduced here; the delta compression added later in this series brings it down to +8.0 MiB (+15.7%) on the same config. Suggested-by: Petr Pavlu Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Sasha Levin --- Documentation/admin-guide/index.rst | 1 + .../admin-guide/kallsyms-lineinfo.rst | 72 ++ MAINTAINERS | 6 + include/linux/kallsyms.h | 18 +- init/Kconfig | 20 + kernel/kallsyms.c | 101 +- kernel/kallsyms_internal.h | 9 + scripts/.gitignore | 1 + scripts/Makefile | 3 + scripts/empty_lineinfo.S | 30 + scripts/gen_lineinfo.c | 969 ++++++++++++++++++ scripts/kallsyms.c | 11 + scripts/link-vmlinux.sh | 43 +- 13 files changed, 1273 insertions(+), 11 deletions(-) create mode 100644 Documentation/admin-guide/kallsyms-lineinfo.rst create mode 100644 scripts/empty_lineinfo.S create mode 100644 scripts/gen_lineinfo.c diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guid= e/index.rst index cd28dfe91b06..37456e08fe43 100644 --- a/Documentation/admin-guide/index.rst +++ b/Documentation/admin-guide/index.rst @@ -73,6 +73,7 @@ problems and bugs in particular. ramoops dynamic-debug-howto init + kallsyms-lineinfo kdump/index perf/index pstore-blk diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentatio= n/admin-guide/kallsyms-lineinfo.rst new file mode 100644 index 000000000000..4c5de44e68fc --- /dev/null +++ b/Documentation/admin-guide/kallsyms-lineinfo.rst @@ -0,0 +1,72 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Kallsyms Source Line Info (LINEINFO) +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Overview +=3D=3D=3D=3D=3D=3D=3D=3D + +``CONFIG_KALLSYMS_LINEINFO`` embeds DWARF-derived source file and line num= ber +mappings into the kernel image so that stack traces include +``(file.c:123)`` annotations next to each symbol. This makes it significa= ntly +easier to pinpoint the exact source location during debugging, without nee= ding +to manually cross-reference addresses with ``addr2line``. + +Enabling the Feature +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Enable the following kernel configuration options:: + + CONFIG_KALLSYMS=3Dy + CONFIG_DEBUG_INFO=3Dy + CONFIG_KALLSYMS_LINEINFO=3Dy + +Build dependency: the host tool ``scripts/gen_lineinfo`` requires ``libelf= `` +and ``libdw`` from elfutils. Install the development packages: + +- Debian/Ubuntu: ``apt install libdw-dev libelf-dev`` +- Fedora/RHEL: ``dnf install elfutils-devel elfutils-libelf-devel`` +- Arch Linux: ``pacman -S libelf`` + +Example Output +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Without ``CONFIG_KALLSYMS_LINEINFO``:: + + Call Trace: + + dump_stack_lvl+0x5d/0x80 + do_syscall_64+0x82/0x190 + entry_SYSCALL_64_after_hwframe+0x76/0x7e + +With ``CONFIG_KALLSYMS_LINEINFO``:: + + Call Trace: + + dump_stack_lvl+0x5d/0x80 (lib/dump_stack.c:123) + do_syscall_64+0x82/0x190 (arch/x86/entry/common.c:52) + entry_SYSCALL_64_after_hwframe+0x76/0x7e + +Note that assembly routines (such as ``entry_SYSCALL_64_after_hwframe``) a= re +not annotated because they lack DWARF debug information. + +Memory Overhead +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The lineinfo tables are stored in ``.rodata`` and typically add approximat= ely +44 MiB to the kernel image for a standard configuration (~4.6 million DWARF +line entries, ~10 bytes per entry after deduplication). + +Known Limitations +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +- **vmlinux only**: Only symbols in the core kernel image are annotated. + Module symbols are not covered. +- **4 GiB offset limit**: Address offsets from ``_text`` are stored as 32-= bit + values. Entries beyond 4 GiB from ``_text`` are skipped at build time w= ith + a warning. +- **65535 file limit**: Source file IDs are stored as 16-bit values. Buil= ds + with more than 65535 unique source files will fail with an error. +- **No assembly annotations**: Functions implemented in assembly that lack + DWARF ``.debug_line`` data are not annotated. diff --git a/MAINTAINERS b/MAINTAINERS index 4a8b0fd665ce..b98d57b1ee1d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13931,6 +13931,12 @@ S: Maintained F: Documentation/hwmon/k8temp.rst F: drivers/hwmon/k8temp.c =20 +KALLSYMS LINEINFO +M: Sasha Levin +S: Maintained +F: Documentation/admin-guide/kallsyms-lineinfo.rst +F: scripts/gen_lineinfo.c + KASAN M: Andrey Ryabinin R: Alexander Potapenko diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h index d5dd54c53ace..53cc25a6e85d 100644 --- a/include/linux/kallsyms.h +++ b/include/linux/kallsyms.h @@ -16,10 +16,15 @@ #include =20 #define KSYM_NAME_LEN 512 + +/* Extra space for " (path/to/file.c:12345)" suffix when lineinfo is enabl= ed */ +#define KSYM_LINEINFO_LEN (IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) ? 128 : 0) + #define KSYM_SYMBOL_LEN (sizeof("%s+%#lx/%#lx [%s %s]") + \ (KSYM_NAME_LEN - 1) + \ 2*(BITS_PER_LONG*3/10) + (MODULE_NAME_LEN - 1) + \ - (BUILD_ID_SIZE_MAX * 2) + 1) + (BUILD_ID_SIZE_MAX * 2) + 1 + \ + KSYM_LINEINFO_LEN) =20 struct cred; struct module; @@ -96,6 +101,9 @@ extern int sprint_backtrace_build_id(char *buffer, unsig= ned long address); =20 int lookup_symbol_name(unsigned long addr, char *symname); =20 +bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start, + const char **file, unsigned int *line); + #else /* !CONFIG_KALLSYMS */ =20 static inline unsigned long kallsyms_lookup_name(const char *name) @@ -164,6 +172,14 @@ static inline int kallsyms_on_each_match_symbol(int (*= fn)(void *, unsigned long) { return -EOPNOTSUPP; } + +static inline bool kallsyms_lookup_lineinfo(unsigned long addr, + unsigned long sym_start, + const char **file, + unsigned int *line) +{ + return false; +} #endif /*CONFIG_KALLSYMS*/ =20 static inline void print_ip_sym(const char *loglvl, unsigned long ip) diff --git a/init/Kconfig b/init/Kconfig index 5230d4879b1c..635305ec0594 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2087,6 +2087,26 @@ config KALLSYMS_ALL =20 Say N unless you really need all symbols, or kernel live patching. =20 +config KALLSYMS_LINEINFO + bool "Embed source file:line information in stack traces" + depends on KALLSYMS && DEBUG_INFO + help + Embeds an address-to-source-line mapping table in the kernel + image so that stack traces directly include file:line information, + similar to what scripts/decode_stacktrace.sh provides but without + needing external tools or a vmlinux with debug info at runtime. + + When enabled, stack traces will look like: + + kmem_cache_alloc_noprof+0x60/0x630 (mm/slub.c:3456) + anon_vma_clone+0x2ed/0xcf0 (mm/rmap.c:412) + + This requires libelf and libdw (from elfutils) on the build host. + Costs 10 bytes per DWARF line-table entry; for x86_64_defconfig + with CONFIG_DEBUG_INFO that is about 18MB. + + If unsure, say N. + # end of the "standard kernel features (expert users)" menu =20 config ARCH_HAS_MEMBARRIER_CALLBACKS diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index aec2f06858af..baf52e81118b 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -467,13 +467,82 @@ static int append_buildid(char *buffer, const char = *modname, =20 #endif /* CONFIG_STACKTRACE_BUILD_ID */ =20 +bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start, + const char **file, unsigned int *line) +{ + unsigned long raw_offset, raw_min; + unsigned int offset, min_offset =3D 0, low, high, mid, file_id; + + if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) || !lineinfo_num_entries) + return false; + + /* Compute offset from _text */ + if (addr < (unsigned long)_text) + return false; + + /* + * Round-trip through unsigned int rather than comparing against + * UINT_MAX: unsigned long is already 32 bits on 32-bit targets, so + * that comparison would be dead code there. + */ + raw_offset =3D addr - (unsigned long)_text; + offset =3D raw_offset; + if (offset !=3D raw_offset) + return false; + + /* + * The search below returns the closest entry at or below @offset, so + * a symbol without line entries of its own (assembly without debug + * info, or anything past the _etext cap like .init.text) would + * inherit the last entry of whatever precedes it. Bound the result + * to entries at or above the resolved symbol's start. + */ + if (sym_start > (unsigned long)_text) { + raw_min =3D sym_start - (unsigned long)_text; + + if (raw_min <=3D raw_offset) + min_offset =3D raw_min; + } + + /* Binary search for largest entry <=3D offset */ + low =3D 0; + high =3D lineinfo_num_entries; + while (low < high) { + mid =3D low + (high - low) / 2; + if (lineinfo_addrs[mid] <=3D offset) + low =3D mid + 1; + else + high =3D mid; + } + + if (low =3D=3D 0) + return false; + low--; + + if (lineinfo_addrs[low] < min_offset) + return false; + + file_id =3D lineinfo_file_ids[low]; + *line =3D lineinfo_lines[low]; + + if (file_id >=3D lineinfo_num_files) + return false; + + if (lineinfo_file_offsets[file_id] >=3D lineinfo_filenames_size) + return false; + + *file =3D &lineinfo_filenames[lineinfo_file_offsets[file_id]]; + return true; +} + /* Look up a kernel symbol and return it in a text buffer. */ static int __sprint_symbol(char *buffer, unsigned long address, - int symbol_offset, int add_offset, int add_buildid) + int symbol_offset, int add_offset, int add_buildid, + int add_lineinfo) { char *modname; const unsigned char *buildid; - unsigned long offset, size; + unsigned long offset, size, sym_start; int len; =20 /* Prevent module removal until modname and modbuildid are printed */ @@ -485,6 +554,7 @@ static int __sprint_symbol(char *buffer, unsigned long = address, if (!len) return sprintf(buffer, "0x%lx", address - symbol_offset); =20 + sym_start =3D address - offset; offset -=3D symbol_offset; =20 if (add_offset) @@ -497,6 +567,23 @@ static int __sprint_symbol(char *buffer, unsigned long= address, len +=3D sprintf(buffer + len, "]"); } =20 + /* + * Append "(file:line)" only for stack-backtrace consumers. Plain + * sprint_symbol() backs %ps, and many existing format strings tack + * literal "()" after %ps to indicate a function call ("foo() + * replaced with bar()"); appending lineinfo there would produce a + * confusing "foo (file:line)()". + */ + if (add_lineinfo && IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) && !modname) { + const char *li_file; + unsigned int li_line; + + if (kallsyms_lookup_lineinfo(address, sym_start, + &li_file, &li_line)) + len +=3D snprintf(buffer + len, KSYM_SYMBOL_LEN - len, + " (%s:%u)", li_file, li_line); + } + return len; } =20 @@ -513,7 +600,7 @@ static int __sprint_symbol(char *buffer, unsigned long = address, */ int sprint_symbol(char *buffer, unsigned long address) { - return __sprint_symbol(buffer, address, 0, 1, 0); + return __sprint_symbol(buffer, address, 0, 1, 0, 0); } EXPORT_SYMBOL_GPL(sprint_symbol); =20 @@ -530,7 +617,7 @@ EXPORT_SYMBOL_GPL(sprint_symbol); */ int sprint_symbol_build_id(char *buffer, unsigned long address) { - return __sprint_symbol(buffer, address, 0, 1, 1); + return __sprint_symbol(buffer, address, 0, 1, 1, 0); } EXPORT_SYMBOL_GPL(sprint_symbol_build_id); =20 @@ -547,7 +634,7 @@ EXPORT_SYMBOL_GPL(sprint_symbol_build_id); */ int sprint_symbol_no_offset(char *buffer, unsigned long address) { - return __sprint_symbol(buffer, address, 0, 0, 0); + return __sprint_symbol(buffer, address, 0, 0, 0, 0); } EXPORT_SYMBOL_GPL(sprint_symbol_no_offset); =20 @@ -567,7 +654,7 @@ EXPORT_SYMBOL_GPL(sprint_symbol_no_offset); */ int sprint_backtrace(char *buffer, unsigned long address) { - return __sprint_symbol(buffer, address, -1, 1, 0); + return __sprint_symbol(buffer, address, -1, 1, 0, 1); } =20 /** @@ -587,7 +674,7 @@ int sprint_backtrace(char *buffer, unsigned long addres= s) */ int sprint_backtrace_build_id(char *buffer, unsigned long address) { - return __sprint_symbol(buffer, address, -1, 1, 1); + return __sprint_symbol(buffer, address, -1, 1, 1, 1); } =20 /* To avoid using get_symbol_offset for every symbol, we carry prefix alon= g. */ diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h index 81a867dbe57d..d7374ce444d8 100644 --- a/kernel/kallsyms_internal.h +++ b/kernel/kallsyms_internal.h @@ -15,4 +15,13 @@ extern const u16 kallsyms_token_index[]; extern const unsigned int kallsyms_markers[]; extern const u8 kallsyms_seqs_of_names[]; =20 +extern const u32 lineinfo_num_entries; +extern const u32 lineinfo_addrs[]; +extern const u16 lineinfo_file_ids[]; +extern const u32 lineinfo_lines[]; +extern const u32 lineinfo_num_files; +extern const u32 lineinfo_file_offsets[]; +extern const u32 lineinfo_filenames_size; +extern const char lineinfo_filenames[]; + #endif // LINUX_KALLSYMS_INTERNAL_H_ diff --git a/scripts/.gitignore b/scripts/.gitignore index 4215c2208f7e..e175714c18b6 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only /asn1_compiler +/gen_lineinfo /gen_packed_field_checks /generate_rust_target /insert-sys-cert diff --git a/scripts/Makefile b/scripts/Makefile index 3434a82a119f..976c607c8d96 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -4,6 +4,7 @@ # the kernel for the build process. =20 hostprogs-always-$(CONFIG_KALLSYMS) +=3D kallsyms +hostprogs-always-$(CONFIG_KALLSYMS_LINEINFO) +=3D gen_lineinfo hostprogs-always-$(BUILD_C_RECORDMCOUNT) +=3D recordmcount hostprogs-always-$(CONFIG_BUILDTIME_TABLE_SORT) +=3D sorttable hostprogs-always-$(CONFIG_ASN1) +=3D asn1_compiler @@ -37,6 +38,8 @@ HOSTCFLAGS_asn1_compiler.o =3D -I$(srctree)/include HOSTCFLAGS_sign-file.o =3D $(shell $(HOSTPKG_CONFIG) --cflags libcrypto 2>= /dev/null) HOSTCFLAGS_sign-file.o +=3D -I$(srctree)/tools/include/uapi/ HOSTLDLIBS_sign-file =3D $(shell $(HOSTPKG_CONFIG) --libs libcrypto 2> /de= v/null || echo -lcrypto) +HOSTCFLAGS_gen_lineinfo.o =3D $(shell $(HOSTPKG_CONFIG) --cflags libdw 2> = /dev/null) +HOSTLDLIBS_gen_lineinfo =3D $(shell $(HOSTPKG_CONFIG) --libs libdw 2> /dev= /null || echo -ldw -lelf) =20 ifdef CONFIG_UNWINDER_ORC ifeq ($(ARCH),x86_64) diff --git a/scripts/empty_lineinfo.S b/scripts/empty_lineinfo.S new file mode 100644 index 000000000000..e058c4113712 --- /dev/null +++ b/scripts/empty_lineinfo.S @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2026 Sasha Levin + * + * Empty lineinfo stub for the initial vmlinux link. + * The real lineinfo is generated from .tmp_vmlinux1 by gen_lineinfo. + */ + .section .rodata, "a" + .globl lineinfo_num_entries + .balign 4 +lineinfo_num_entries: + .long 0 + .globl lineinfo_num_files + .balign 4 +lineinfo_num_files: + .long 0 + .globl lineinfo_addrs +lineinfo_addrs: + .globl lineinfo_file_ids +lineinfo_file_ids: + .globl lineinfo_lines +lineinfo_lines: + .globl lineinfo_file_offsets +lineinfo_file_offsets: + .globl lineinfo_filenames_size + .balign 4 +lineinfo_filenames_size: + .long 0 + .globl lineinfo_filenames +lineinfo_filenames: diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c new file mode 100644 index 000000000000..a3ee54eaad05 --- /dev/null +++ b/scripts/gen_lineinfo.c @@ -0,0 +1,969 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * gen_lineinfo.c - Generate address-to-source-line lookup tables from DWA= RF + * + * Copyright (C) 2026 Sasha Levin + * + * Reads DWARF .debug_line from a vmlinux ELF file and outputs an assembly + * file containing sorted lookup tables that the kernel uses to annotate + * stack traces with source file:line information. + * + * Requires libelf and libdw from elfutils. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define LINEINFO_PREFIX "gen_lineinfo: " + +static bool verbose; + +#define verbose_msg(fmt, ...) \ + do { \ + if (verbose) \ + fprintf(stderr, LINEINFO_PREFIX fmt "\n", \ + ##__VA_ARGS__); \ + } while (0) + +#define warn(fmt, ...) \ + fprintf(stderr, LINEINFO_PREFIX "warning: " fmt "\n", ##__VA_ARGS__) + +#define error(fmt, ...) \ + do { \ + fprintf(stderr, LINEINFO_PREFIX "error: " fmt "\n", \ + ##__VA_ARGS__); \ + exit(1); \ + } while (0) + +static unsigned int skipped_overflow; + +/* + * vmlinux mode: end of the invariant .text region. Zero means "no cap" + * (graceful fallback when _etext is absent on some build). + */ +static unsigned long long text_end_addr; + +struct line_entry { + unsigned int offset; /* offset from _text */ + unsigned int file_id; + unsigned int line; +}; + +/* + * Individually allocated so files[] can grow without invalidating the + * hlist_node linkage. + */ +struct file_entry { + struct hlist_node hnode; + unsigned int id; + unsigned int str_offset; + char name[]; +}; + +static struct line_entry *entries; +static unsigned int num_entries; +static unsigned int entries_capacity; + +static struct file_entry **files; +static unsigned int num_files; +static unsigned int files_capacity; + +static HASHTABLE_DEFINE(file_hashtable, 1U << 13); + +static void add_entry(unsigned int offset, unsigned int file_id, + unsigned int line) +{ + if (num_entries >=3D entries_capacity) { + entries_capacity =3D entries_capacity ? entries_capacity * 2 : 65536; + entries =3D xrealloc(entries, entries_capacity * sizeof(*entries)); + } + entries[num_entries].offset =3D offset; + entries[num_entries].file_id =3D file_id; + entries[num_entries].line =3D line; + num_entries++; +} + +static unsigned int find_or_add_file(const char *name) +{ + unsigned int key =3D hash_str(name); + struct file_entry *f; + size_t len; + + hash_for_each_possible(file_hashtable, f, hnode, key) + if (!strcmp(f->name, name)) + return f->id; + + if (num_files >=3D 65535) + error("too many source files (%u > 65535)", num_files); + + if (num_files >=3D files_capacity) { + files_capacity =3D files_capacity ? files_capacity * 2 : 4096; + files =3D xrealloc(files, files_capacity * sizeof(*files)); + } + + len =3D strlen(name); + f =3D xmalloc(sizeof(*f) + len + 1); + memset(f, 0, sizeof(*f)); + memcpy(f->name, name, len + 1); + f->id =3D num_files; + + files[num_files] =3D f; + hash_add(file_hashtable, &f->hnode, key); + + return num_files++; +} + +/* + * Well-known top-level directories in the kernel source tree. Only used + * as a last resort, when a path matches none of the build roots below -- + * e.g. an object compiled outside any of them. + */ +static const char * const kernel_dirs[] =3D { + "arch/", "block/", "certs/", "crypto/", "drivers/", "fs/", + "include/", "init/", "io_uring/", "ipc/", "kernel/", "lib/", + "mm/", "net/", "rust/", "samples/", "scripts/", "security/", + "sound/", "tools/", "usr/", "virt/", +}; + +/* Absolute build and source roots, longest first. */ +struct path_root { + char *path; + size_t len; +}; + +static struct path_root path_roots[8]; +static unsigned int num_path_roots; + +/* + * Lexically canonicalize @path in place: collapse repeated slashes, drop + * "." components and resolve ".." against the preceding component. Purely + * textual -- nothing is stat()ed, because DWARF can name generated files + * that do not exist yet when gen_lineinfo runs. + */ +static void normalize_path(char *path) +{ + bool absolute =3D path[0] =3D=3D '/'; + char *base, *out =3D path; + const char *in =3D path; + + if (absolute) + *out++ =3D *in++; + base =3D out; + + while (*in) { + const char *seg =3D in; + size_t seglen; + + while (*in && *in !=3D '/') + in++; + seglen =3D in - seg; + while (*in =3D=3D '/') + in++; + + if (!seglen || (seglen =3D=3D 1 && seg[0] =3D=3D '.')) + continue; + + if (seglen =3D=3D 2 && seg[0] =3D=3D '.' && seg[1] =3D=3D '.') { + if (out > base) { + /* Pop the previously emitted component. */ + while (out > base && out[-1] !=3D '/') + out--; + if (out > base) + out--; /* and its separator */ + continue; + } + /* "/.." is "/"; a leading ".." in a relative path stays. */ + if (absolute) + continue; + } + + /* + * Separator first: writing it after the component would land + * on the byte @in still points at whenever nothing has been + * compacted yet, clobbering the terminator and running the + * loop off the end of the string. + */ + if (out > base) + *out++ =3D '/'; + memmove(out, seg, seglen); + out +=3D seglen; + } + + if (out =3D=3D base && !absolute) + *out++ =3D '.'; + *out =3D '\0'; +} + +static int compare_path_roots(const void *a, const void *b) +{ + const struct path_root *ra =3D a, *rb =3D b; + + if (ra->len !=3D rb->len) + return ra->len > rb->len ? -1 : 1; + return 0; +} + +static void add_path_root(const char *path) +{ + char buf[PATH_MAX]; + + if (!path || !*path) + return; + + if (path[0] =3D=3D '/') { + if (snprintf(buf, sizeof(buf), "%s", path) >=3D (int)sizeof(buf)) + return; + } else { + char cwd[PATH_MAX]; + + /* kbuild runs host tools with cwd =3D=3D $objtree. */ + if (!getcwd(cwd, sizeof(cwd))) + return; + if (snprintf(buf, sizeof(buf), "%s/%s", cwd, path) >=3D (int)sizeof(buf)) + return; + } + + normalize_path(buf); + + /* "/" would match every absolute path. */ + if (!strcmp(buf, "/")) + return; + + for (unsigned int i =3D 0; i < num_path_roots; i++) + if (!strcmp(path_roots[i].path, buf)) + return; + + if (num_path_roots =3D=3D ARRAY_SIZE(path_roots)) + return; + + path_roots[num_path_roots].path =3D xstrdup(buf); + path_roots[num_path_roots].len =3D strlen(buf); + num_path_roots++; +} + +/* + * Collect the roots that DWARF paths get made relative to. kbuild exports + * all three, so no Makefile plumbing is needed: $objtree and $srctree cov= er + * in-tree and O=3D builds, and $srcroot covers M=3D external modules, who= se + * sources live under neither. + */ +static void init_path_roots(void) +{ + static const char * const vars[] =3D { "objtree", "srctree", "srcroot" }; + + for (unsigned int i =3D 0; i < ARRAY_SIZE(vars); i++) { + const char *val =3D getenv(vars[i]); + char *real; + + if (!val || !*val) + continue; + + add_path_root(val); + + /* + * Register the resolved form as well, so a symlinked tree + * matches whichever spelling the compiler recorded. Only + * the roots are resolved this way -- never a DWARF path. + */ + real =3D realpath(val, NULL); + if (real) { + add_path_root(real); + free(real); + } + } + + qsort(path_roots, num_path_roots, sizeof(*path_roots), + compare_path_roots); +} + +/* + * Strip a DWARF filename down to a kernel-tree-relative path. + * + * Per DWARF, a relative DW_AT_name is relative to the CU's DW_AT_comp_dir, + * so the two are joined and canonicalized first. The result is then made + * relative to the longest matching build root. Everything after that is a + * fallback for objects built outside the tree. + */ +static const char *make_relative(const char *path, const char *comp_dir) +{ + static char buf[PATH_MAX]; + const char *p; + + if (path[0] =3D=3D '/') { + if (snprintf(buf, sizeof(buf), "%s", path) >=3D (int)sizeof(buf)) + return path; + } else if (comp_dir && comp_dir[0] =3D=3D '/') { + if (snprintf(buf, sizeof(buf), "%s/%s", comp_dir, path) >=3D + (int)sizeof(buf)) + return path; + } else { + /* Nothing absolute to anchor against. */ + return path; + } + + normalize_path(buf); + + for (unsigned int i =3D 0; i < num_path_roots; i++) { + size_t len =3D path_roots[i].len; + + if (!strncmp(buf, path_roots[i].path, len) && buf[len] =3D=3D '/') + return buf + len + 1; + } + + /* + * comp_dir may still be a usable prefix even when it is not one of + * the roots -- but only if stripping it leaves a directory + * component, otherwise the kernel_dirs scan recovers more. + */ + if (comp_dir) { + size_t len =3D strlen(comp_dir); + + if (!strncmp(buf, comp_dir, len) && buf[len] =3D=3D '/' && + strchr(buf + len + 1, '/')) + return buf + len + 1; + } + + for (p =3D strchr(buf, '/'); p; p =3D strchr(p + 1, '/')) + for (unsigned int i =3D 0; i < ARRAY_SIZE(kernel_dirs); i++) + if (!strncmp(p + 1, kernel_dirs[i], + strlen(kernel_dirs[i]))) + return p + 1; + + p =3D strrchr(buf, '/'); + return p ? p + 1 : buf; +} + +static int compare_entries(const void *a, const void *b) +{ + const struct line_entry *ea =3D a; + const struct line_entry *eb =3D b; + + if (ea->offset !=3D eb->offset) + return ea->offset < eb->offset ? -1 : 1; + if (ea->file_id !=3D eb->file_id) + return ea->file_id < eb->file_id ? -1 : 1; + if (ea->line !=3D eb->line) + return ea->line < eb->line ? -1 : 1; + return 0; +} + +/* + * Look up a vmlinux symbol by exact name and return its st_value, or + * @fallback if absent. Aborts when @required and the symbol is missing. + */ +static unsigned long long find_vmlinux_sym(Elf *elf, const char *name, + unsigned long long fallback, + bool required) +{ + size_t nsyms, i; + Elf_Scn *scn =3D NULL; + GElf_Shdr shdr; + + while ((scn =3D elf_nextscn(elf, scn)) !=3D NULL) { + Elf_Data *data; + + if (!gelf_getshdr(scn, &shdr)) + continue; + if (shdr.sh_type !=3D SHT_SYMTAB) + continue; + + data =3D elf_getdata(scn, NULL); + if (!data) + continue; + + nsyms =3D shdr.sh_size / shdr.sh_entsize; + for (i =3D 0; i < nsyms; i++) { + GElf_Sym sym; + const char *sname; + + if (!gelf_getsym(data, i, &sym)) + continue; + sname =3D elf_strptr(elf, shdr.sh_link, sym.st_name); + if (sname && !strcmp(sname, name)) + return sym.st_value; + } + } + + if (required) + error("cannot find %s symbol", name); + return fallback; +} + +static unsigned long long find_text_addr(Elf *elf) +{ + return find_vmlinux_sym(elf, "_text", 0, true); +} + +/* + * vmlinux is linked in multiple passes: gen_lineinfo runs against + * .tmp_vmlinux1 (which carries an empty lineinfo stub), then real tables + * are linked in for the final image. Sections placed AFTER .rodata + * (.init.text, .exit.text, ...) shift forward as .rodata grows to hold + * the real lineinfo blob, so DWARF addresses we'd capture for them in + * pass 1 would be stale in the final kernel. Cap captured addresses at + * _etext, the symbol that marks the end of .text =E2=80=94 placed before = .rodata + * in every architecture's vmlinux.lds.S, so its addresses are invariant + * across the relink. Returns 0 if _etext is absent (no cap; v3 behavior). + */ +static unsigned long long find_text_end_addr(Elf *elf) +{ + return find_vmlinux_sym(elf, "_etext", 0, false); +} + +static int compare_uints(const void *a, const void *b) +{ + unsigned int ua =3D *(const unsigned int *)a; + unsigned int ub =3D *(const unsigned int *)b; + + if (ua !=3D ub) + return ua < ub ? -1 : 1; + return 0; +} + +/* Sorted, duplicate-free extents of every function symbol. */ +struct sym_start { + unsigned int offset; + unsigned int size; +}; + +static struct sym_start *sym_starts; +static unsigned int num_sym_starts; +static unsigned int sym_starts_capacity; + +/* Sorted offsets one past the end of each DWARF line-program sequence. */ +static unsigned int *seq_ends; +static unsigned int num_seq_ends; +static unsigned int seq_ends_capacity; + +static void append_offset(unsigned int **arr, unsigned int *count, + unsigned int *capacity, unsigned int value) +{ + if (*count >=3D *capacity) { + *capacity =3D *capacity ? *capacity * 2 : 16384; + *arr =3D xrealloc(*arr, *capacity * sizeof(**arr)); + } + (*arr)[(*count)++] =3D value; +} + +static void sort_unique(unsigned int *arr, unsigned int *count) +{ + unsigned int j =3D 0; + + if (*count < 2) + return; + + qsort(arr, *count, sizeof(*arr), compare_uints); + for (unsigned int i =3D 1; i < *count; i++) { + if (arr[i] =3D=3D arr[j]) + continue; + if (++j !=3D i) + arr[j] =3D arr[i]; + } + *count =3D j + 1; +} + +/* + * Record the end of a line-program sequence. @addr is one past the last + * covered byte, so the sequence's own coverage is tested using addr - 1. + */ +static void record_seq_end(unsigned long long addr, + unsigned long long text_addr) +{ + unsigned long long raw; + + if (addr <=3D text_addr) + return; + if (text_end_addr && addr - 1 >=3D text_end_addr) + return; + + raw =3D addr - text_addr; + if (raw > UINT_MAX) + return; + + append_offset(&seq_ends, &num_seq_ends, &seq_ends_capacity, + (unsigned int)raw); +} + +static int compare_sym_starts(const void *a, const void *b) +{ + const struct sym_start *sa =3D a, *sb =3D b; + + if (sa->offset !=3D sb->offset) + return sa->offset < sb->offset ? -1 : 1; + /* Larger extent first, so the dedup below keeps it. */ + if (sa->size !=3D sb->size) + return sa->size > sb->size ? -1 : 1; + return 0; +} + +/* + * Collect the extent of every function symbol. deduplicate() uses these = to + * make sure each function keeps an entry at its own first byte; without t= hat + * the kernel's symbol-boundary check rejects the preceding function's ent= ry + * and the frame goes unannotated. + */ +static void collect_symbol_starts(Elf *elf, unsigned long long text_addr) +{ + Elf_Scn *scn =3D NULL; + GElf_Shdr shdr; + + while ((scn =3D elf_nextscn(elf, scn)) !=3D NULL) { + Elf_Data *data; + size_t nsyms; + + if (!gelf_getshdr(scn, &shdr)) + continue; + if (shdr.sh_type !=3D SHT_SYMTAB || !shdr.sh_entsize) + continue; + + data =3D elf_getdata(scn, NULL); + if (!data) + continue; + + nsyms =3D shdr.sh_size / shdr.sh_entsize; + for (size_t i =3D 0; i < nsyms; i++) { + GElf_Sym sym; + unsigned long long raw; + + if (!gelf_getsym(data, i, &sym)) + continue; + if (GELF_ST_TYPE(sym.st_info) !=3D STT_FUNC) + continue; + if (sym.st_value < text_addr) + continue; + if (text_end_addr && sym.st_value >=3D text_end_addr) + continue; + + raw =3D sym.st_value - text_addr; + if (raw > UINT_MAX) + continue; + + if (num_sym_starts >=3D sym_starts_capacity) { + sym_starts_capacity =3D sym_starts_capacity ? + sym_starts_capacity * 2 : 16384; + sym_starts =3D xrealloc(sym_starts, + sym_starts_capacity * + sizeof(*sym_starts)); + } + sym_starts[num_sym_starts].offset =3D (unsigned int)raw; + sym_starts[num_sym_starts].size =3D + sym.st_size > UINT_MAX ? UINT_MAX : + (unsigned int)sym.st_size; + num_sym_starts++; + } + } + + if (num_sym_starts > 1) { + unsigned int j =3D 0; + + qsort(sym_starts, num_sym_starts, sizeof(*sym_starts), + compare_sym_starts); + for (unsigned int i =3D 1; i < num_sym_starts; i++) { + if (sym_starts[i].offset =3D=3D sym_starts[j].offset) + continue; + if (++j !=3D i) + sym_starts[j] =3D sym_starts[i]; + } + num_sym_starts =3D j + 1; + } +} + +static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr) +{ + Dwarf_Off off =3D 0, next_off; + size_t hdr_size; + + while (dwarf_nextcu(dwarf, off, &next_off, &hdr_size, + NULL, NULL, NULL) =3D=3D 0) { + Dwarf_Die cudie; + Dwarf_Lines *lines; + size_t nlines; + Dwarf_Attribute attr; + const char *comp_dir =3D NULL; + + if (!dwarf_offdie(dwarf, off + hdr_size, &cudie)) + goto next; + + if (dwarf_attr(&cudie, DW_AT_comp_dir, &attr)) + comp_dir =3D dwarf_formstring(&attr); + + if (dwarf_getsrclines(&cudie, &lines, &nlines) !=3D 0) + goto next; + + for (size_t i =3D 0; i < nlines; i++) { + Dwarf_Line *line =3D dwarf_onesrcline(lines, i); + Dwarf_Addr addr; + const char *src; + const char *rel; + unsigned int file_id, loffset; + bool endseq =3D false; + int lineno; + + if (!line) + continue; + + if (dwarf_lineaddr(line, &addr) !=3D 0) + continue; + + /* + * An end_sequence row marks the first address NOT + * covered by this sequence; libdw repeats the previous + * line number on it, so keeping it as an entry would + * extend a function's annotation past its own end. + * Record the boundary instead -- deduplicate() needs + * it to tell "this row still covers the next symbol" + * from "coverage stopped here". + */ + if (dwarf_lineendsequence(line, &endseq) =3D=3D 0 && endseq) { + record_seq_end(addr, text_addr); + continue; + } + + if (dwarf_lineno(line, &lineno) !=3D 0) + continue; + if (lineno =3D=3D 0) + continue; + + src =3D dwarf_linesrc(line, NULL, NULL); + if (!src) + continue; + + if (addr < text_addr) + continue; + /* + * Skip addresses past _etext. Sections after .rodata + * shift when the real lineinfo replaces the empty stub + * during the multi-pass vmlinux link, so any address + * we'd capture there would be stale by the time the + * final kernel runs. + */ + if (text_end_addr && addr >=3D text_end_addr) + continue; + + { + unsigned long long raw_offset =3D addr - text_addr; + + if (raw_offset > UINT_MAX) { + skipped_overflow++; + continue; + } + loffset =3D (unsigned int)raw_offset; + } + + rel =3D make_relative(src, comp_dir); + file_id =3D find_or_add_file(rel); + + add_entry(loffset, file_id, (unsigned int)lineno); + } +next: + off =3D next_off; + } +} + +/* True if some line-program sequence ends in (@lo, @hi]. */ +static bool seq_end_between(unsigned int lo, unsigned int hi) +{ + unsigned int low =3D 0, high =3D num_seq_ends; + + /* First index whose value exceeds @lo. */ + while (low < high) { + unsigned int mid =3D low + (high - low) / 2; + + if (seq_ends[mid] <=3D lo) + low =3D mid + 1; + else + high =3D mid; + } + + return low < num_seq_ends && seq_ends[low] <=3D hi; +} + +/* + * Give every function an entry at its own first byte. + * + * Compilers routinely emit no line row at a symbol's start: .cold + * fragments in particular are covered by a row belonging to the function + * they were split out of. That used to resolve fine, but the kernel now + * refuses any entry below the resolved symbol's start, so those frames + * would print unannotated. Copy the covering row down to the symbol + * start instead. + * + * Two things have to hold before that is honest: + * + * - the covering row's sequence must not have ended in between, or it + * describes code that stopped before this symbol. This is what keeps + * the __SCT__* static-call trampolines unannotated. + * + * - the line program must place at least one row inside the symbol, so + * we know it describes this symbol's code at all. This is what keeps + * the __pfx_* padding stubs unannotated: they are pure alignment + * padding, and no compiler ever emits a row inside one. + * + * Symbols failing either test keep no annotation, which is the correct + * answer for hand-written assembly. + */ +static void synthesize_symbol_starts(void) +{ + unsigned int base_entries =3D num_entries; + unsigned int cursor =3D 0; + + if (!base_entries || !num_sym_starts) + return; + + sort_unique(seq_ends, &num_seq_ends); + + for (unsigned int i =3D 0; i < num_sym_starts; i++) { + unsigned int start =3D sym_starts[i].offset; + unsigned int end =3D start + sym_starts[i].size; + + while (cursor + 1 < base_entries && + entries[cursor + 1].offset <=3D start) + cursor++; + + if (entries[cursor].offset > start) + continue; /* nothing covers this symbol */ + if (entries[cursor].offset =3D=3D start) + continue; /* already has its own entry */ + if (seq_end_between(entries[cursor].offset, start)) + continue; /* coverage stopped before here */ + + /* Overflow, or no row inside [start, end): not our code. */ + if (end <=3D start) + continue; + if (cursor + 1 >=3D base_entries || + entries[cursor + 1].offset >=3D end) + continue; + + add_entry(start, entries[cursor].file_id, entries[cursor].line); + } +} + +static void deduplicate(void) +{ + unsigned int sym_cursor =3D 0; + unsigned int i, j; + + if (num_entries < 2) + return; + + /* Sort by offset, then file_id, then line for stability */ + qsort(entries, num_entries, sizeof(*entries), compare_entries); + + synthesize_symbol_starts(); + qsort(entries, num_entries, sizeof(*entries), compare_entries); + + /* + * Remove duplicate entries: + * - Same offset: keep first (deterministic from stable sort keys) + * - Same file:line as previous kept entry: redundant for binary + * search -- any address between them resolves to the earlier one + * + * Entries sitting on a symbol start are exempt from the second rule: + * they are the only thing standing between that symbol and the + * kernel's boundary check. + */ + j =3D 0; + for (i =3D 1; i < num_entries; i++) { + bool at_symbol_start; + + if (entries[i].offset =3D=3D entries[j].offset) + continue; + + while (sym_cursor < num_sym_starts && + sym_starts[sym_cursor].offset < entries[i].offset) + sym_cursor++; + at_symbol_start =3D sym_cursor < num_sym_starts && + sym_starts[sym_cursor].offset =3D=3D entries[i].offset; + + if (!at_symbol_start && + entries[i].file_id =3D=3D entries[j].file_id && + entries[i].line =3D=3D entries[j].line) + continue; + + j++; + if (j !=3D i) + entries[j] =3D entries[i]; + } + num_entries =3D j + 1; +} + +static void compute_file_offsets(void) +{ + unsigned int offset =3D 0; + + for (unsigned int i =3D 0; i < num_files; i++) { + files[i]->str_offset =3D offset; + offset +=3D strlen(files[i]->name) + 1; + } +} + +static void print_escaped_asciz(const char *s) +{ + printf("\t.asciz \""); + for (; *s; s++) { + if (*s =3D=3D '"' || *s =3D=3D '\\') + putchar('\\'); + putchar(*s); + } + printf("\"\n"); +} + +static void output_assembly(void) +{ + printf("/* SPDX-License-Identifier: GPL-2.0 */\n"); + printf("/*\n"); + printf(" * Automatically generated by scripts/gen_lineinfo\n"); + printf(" * Do not edit.\n"); + printf(" */\n\n"); + + printf("\t.section .rodata, \"a\"\n\n"); + + /* Number of entries */ + printf("\t.globl lineinfo_num_entries\n"); + printf("\t.balign 4\n"); + printf("lineinfo_num_entries:\n"); + printf("\t.long %u\n\n", num_entries); + + /* Number of files */ + printf("\t.globl lineinfo_num_files\n"); + printf("\t.balign 4\n"); + printf("lineinfo_num_files:\n"); + printf("\t.long %u\n\n", num_files); + + /* Sorted address offsets from _text */ + printf("\t.globl lineinfo_addrs\n"); + printf("\t.balign 4\n"); + printf("lineinfo_addrs:\n"); + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.long 0x%x\n", entries[i].offset); + printf("\n"); + + /* File IDs, parallel to addrs (u16 -- supports up to 65535 files) */ + printf("\t.globl lineinfo_file_ids\n"); + printf("\t.balign 2\n"); + printf("lineinfo_file_ids:\n"); + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.short %u\n", entries[i].file_id); + printf("\n"); + + /* Line numbers, parallel to addrs */ + printf("\t.globl lineinfo_lines\n"); + printf("\t.balign 4\n"); + printf("lineinfo_lines:\n"); + for (unsigned int i =3D 0; i < num_entries; i++) + printf("\t.long %u\n", entries[i].line); + printf("\n"); + + /* File string offset table */ + printf("\t.globl lineinfo_file_offsets\n"); + printf("\t.balign 4\n"); + printf("lineinfo_file_offsets:\n"); + for (unsigned int i =3D 0; i < num_files; i++) + printf("\t.long %u\n", files[i]->str_offset); + printf("\n"); + + /* Filenames size */ + { + unsigned int fsize =3D 0; + + for (unsigned int i =3D 0; i < num_files; i++) + fsize +=3D strlen(files[i]->name) + 1; + printf("\t.globl lineinfo_filenames_size\n"); + printf("\t.balign 4\n"); + printf("lineinfo_filenames_size:\n"); + printf("\t.long %u\n\n", fsize); + } + + /* Concatenated NUL-terminated filenames */ + printf("\t.globl lineinfo_filenames\n"); + printf("lineinfo_filenames:\n"); + for (unsigned int i =3D 0; i < num_files; i++) + print_escaped_asciz(files[i]->name); + printf("\n"); +} + +int main(int argc, char *argv[]) +{ + const char *kbuild_verbose =3D getenv("KBUILD_VERBOSE"); + unsigned long long text_addr; + Dwarf *dwarf; + Elf *elf; + int fd; + + if (kbuild_verbose && strchr(kbuild_verbose, '1')) + verbose =3D true; + + while (argc > 2 && (!strcmp(argv[1], "-v") || + !strcmp(argv[1], "--verbose"))) { + verbose =3D true; + memmove(&argv[1], &argv[2], (argc - 2) * sizeof(char *)); + argc--; + } + + if (argc !=3D 2) { + fprintf(stderr, "Usage: %s [-v] \n", argv[0]); + return 1; + } + + init_path_roots(); + + fd =3D open(argv[1], O_RDONLY); + if (fd < 0) + error("cannot open %s: %s", argv[1], strerror(errno)); + + elf_version(EV_CURRENT); + elf =3D elf_begin(fd, ELF_C_READ_MMAP, NULL); + if (!elf) + error("elf_begin failed: %s", elf_errmsg(elf_errno())); + + text_addr =3D find_text_addr(elf); + text_end_addr =3D find_text_end_addr(elf); + collect_symbol_starts(elf, text_addr); + + dwarf =3D dwarf_begin_elf(elf, DWARF_C_READ, NULL); + if (!dwarf) + error("dwarf_begin_elf failed: %s\n" + LINEINFO_PREFIX "error: is %s built with CONFIG_DEBUG_INFO?", + dwarf_errmsg(dwarf_errno()), argv[1]); + + process_dwarf(dwarf, text_addr); + + if (skipped_overflow) + warn("%u entries skipped (offset > 4 GiB from _text)", + skipped_overflow); + + deduplicate(); + compute_file_offsets(); + + verbose_msg("%u entries, %u files", num_entries, num_files); + + output_assembly(); + + dwarf_end(dwarf); + elf_end(elf); + close(fd); + + /* Cleanup */ + free(entries); + free(sym_starts); + free(seq_ends); + for (unsigned int i =3D 0; i < num_files; i++) + free(files[i]); + free(files); + + return 0; +} diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 37d5c095ad22..e11f101b743b 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -90,6 +90,17 @@ static bool is_ignored_symbol(const char *name, char typ= e) return true; } =20 + /* + * The generated lineinfo tables (scripts/gen_lineinfo, stubbed by + * scripts/empty_lineinfo.S) are read-only data whose size and + * addresses change between kallsyms passes. Match them by prefix so + * the set cannot drift as the table layout evolves. Text symbols are + * exempt, so lib/tests/lineinfo_kunit.c's lineinfo_target_*() stay + * resolvable -- the test looks them up by name. + */ + if (toupper(type) !=3D 'T' && !strncmp(name, "lineinfo_", 9)) + return true; + return false; } =20 diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index f99e196abeea..39ca44fbb259 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -103,7 +103,7 @@ vmlinux_link() ${ld} ${ldflags} -o ${output} \ ${wl}--whole-archive ${objs} ${wl}--no-whole-archive \ ${wl}--start-group ${libs} ${wl}--end-group \ - ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} + ${kallsymso} ${lineinfo_o} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldli= bs} } =20 # Create ${2}.o file with all symbols from the ${1} object file @@ -129,6 +129,26 @@ kallsyms() kallsymso=3D${2}.o } =20 +# Generate lineinfo tables from DWARF debug info in a temporary vmlinux. +# ${1} - temporary vmlinux with debug info +# Output: sets lineinfo_o to the generated .o file +gen_lineinfo() +{ + info LINEINFO .tmp_lineinfo.S + if ! scripts/gen_lineinfo "${1}" > .tmp_lineinfo.S; then + echo >&2 "Failed to generate lineinfo from ${1}" + echo >&2 "Try to disable CONFIG_KALLSYMS_LINEINFO" + exit 1 + fi + + info AS .tmp_lineinfo.o + ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \ + ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \ + -c -o .tmp_lineinfo.o .tmp_lineinfo.S + + lineinfo_o=3D.tmp_lineinfo.o +} + # Perform kallsyms for the given temporary vmlinux. sysmap_and_kallsyms() { @@ -155,6 +175,7 @@ sorttable() cleanup() { rm -f .btf.* + rm -f .tmp_lineinfo.* rm -f .tmp_vmlinux.nm-sort rm -f System.map rm -f vmlinux @@ -183,6 +204,7 @@ fi btf_vmlinux_bin_o=3D btfids_vmlinux=3D kallsymso=3D +lineinfo_o=3D strip_debug=3D generate_map=3D =20 @@ -198,10 +220,21 @@ if is_enabled CONFIG_KALLSYMS; then kallsyms .tmp_vmlinux0.syms .tmp_vmlinux0.kallsyms fi =20 +if is_enabled CONFIG_KALLSYMS_LINEINFO; then + # Assemble an empty lineinfo stub for the initial link. + # The real lineinfo is generated from .tmp_vmlinux1 by gen_lineinfo. + ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \ + ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \ + -c -o .tmp_lineinfo.o "${srctree}/scripts/empty_lineinfo.S" + lineinfo_o=3D.tmp_lineinfo.o +fi + if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then =20 - # The kallsyms linking does not need debug symbols, but the BTF does. - if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + # The kallsyms linking does not need debug symbols, but BTF and + # lineinfo generation do. + if ! is_enabled CONFIG_DEBUG_INFO_BTF && + ! is_enabled CONFIG_KALLSYMS_LINEINFO; then strip_debug=3D1 fi =20 @@ -219,6 +252,10 @@ if is_enabled CONFIG_DEBUG_INFO_BTF; then btfids_vmlinux=3D.tmp_vmlinux1.BTF_ids fi =20 +if is_enabled CONFIG_KALLSYMS_LINEINFO; then + gen_lineinfo .tmp_vmlinux1 +fi + if is_enabled CONFIG_KALLSYMS; then =20 # kallsyms support --=20 2.53.0 From nobody Fri Oct 2 12:19:52 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CA6F2571DA; Sat, 1 Aug 2026 14:32:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785594757; cv=none; b=D/gkP0/tAh3QJfG/DdNm/oH8NHFlSkXMzSarqQubhot8wlaTu1JHY2gWDKmuLArTpriYBC/JNeygqB8YEJACEsBC5Jhxs4UVu5Upf8fmY873cFujaL6CLUbnwwiO2/O3zgdG/dxAqEoPD6IQYqgJbwTcIBGb+JsKGkOFiJL7ykM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785594757; c=relaxed/simple; bh=/Xvo5SbWQ0WE8XeR8U6aAFoTbxAZZ+mC/MDuLs7yjeE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=aqg5lZqmkUdrrP+pEOIzqaWdURFYsd67XDC8chIDH+mrtJH4qRt6mAoMD2dBua05akqmmfuCmOguA4laJtpz6b4/IX2fJKpJ6P+5eqPWCKhrhOXifzgS5pP6HKiYevz5B7vxPoxsNrckAPao8/wSYCJWP7pBiu0uuKNxh6lnsdI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jGp+da2z; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jGp+da2z" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 427291F00ACF; Sat, 1 Aug 2026 14:32:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785594751; bh=dQdtJPXcTB7+ef98yEStVRil0uTlju5ZZs3lOMulp6s=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jGp+da2zDxJR5NBtmsZpX6sY5FL+4bMvcNKaWZm6t9xcA8iiENJa2yqbOmm5e3xt9 3BJhNfeD2qtcOTiOlITY92BPyLMBEFbZ/Ztief5oDlnxTof3TZsCVpx6UDygn4vzCA 4MkJ4FeRRACgfM6jNGkB7bUFFOwNoRNyvDhXW+PrcS1rPjQD4s+HzTKsCi4MaHl1pW Ascr1DBem9she4hgPjCEGJna4Su4YHDm7l803X6IqAXfFpldh5r9QH+Q3J/ILhf9VN c/6c5HJGB7nGCjJThNKrvKn0l7hrQPZsYXiigcsEXDRxTgKWIwlxA0eAL3GlWQQEin zOxt/G/HsHIyQ== From: Sasha Levin To: Andrew Morton , Masahiro Yamada , Luis Chamberlain , Linus Torvalds , Richard Weinberger , Juergen Gross , Geert Uytterhoeven , James Bottomley Cc: Sasha Levin , Jonathan Corbet , Nathan Chancellor , Nicolas Schier , Petr Pavlu , Daniel Gomez , Greg KH , Petr Mladek , Steven Rostedt , Kees Cook , Peter Zijlstra , Thorsten Leemhuis , Vlastimil Babka , Helge Deller , Randy Dunlap , Laurent Pinchart , Vivian Wang , Zhen Lei , Sami Tolvanen , linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-doc@vger.kernel.org Subject: [PATCH v8 2/4] kallsyms: extend lineinfo to loadable modules Date: Sat, 1 Aug 2026 10:32:16 -0400 Message-ID: <20260801143219.3975824-3-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260801143219.3975824-1-sashal@kernel.org> References: <20260801143219.3975824-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Add CONFIG_KALLSYMS_LINEINFO_MODULES, which extends the CONFIG_KALLSYMS_LINEINFO feature to loadable kernel modules. At build time, each .ko is post-processed by scripts/gen-mod-lineinfo.sh (modeled on gen-btf.sh) which runs scripts/gen_lineinfo --module on the .ko, generates per-section .mod_lineinfo and .init.mod_lineinfo sections containing compact binary tables of section-relative offsets, file IDs, line numbers, and filenames, and embeds them back into the .ko via a partial link (ld -r). At runtime, module_lookup_lineinfo() walks the section descriptors in each blob, finds the one whose runtime range contains the queried address, and binary-searches that section's table. The lookup is NMI/panic-safe (no locks, no allocations) =E2=80=94 the data lives in read-only module memory and is freed automatically when the module (or its init memory) is unloaded. The gen_lineinfo tool gains --module mode which: - Walks an allowlist of text-like sections (.text, .exit.text, .init.text), gating each on its presence in the .ko. - Uses an ELF relocation against each covered section's symbol as the runtime "anchor", resolved by the module loader's standard apply_relocations() pass =E2=80=94 no implicit base derivation from mod->mem[].base, no special-cased loader logic. - Disambiguates DWARF addresses across sections that all share sh_addr =3D=3D 0 in ET_REL files via per-section synthetic biases applied inside apply_debug_line_relocations() (handles both abs32 and abs64 width relocs). - Handles libdw's ET_REL path-doubling quirk in make_relative(). - Declares empty section stanzas in its output assembly so the resulting lineinfo.o has LOCAL SECTION symbols rather than GLOBAL UND ones; otherwise ld -r would not bind the relocation to the .ko's existing section symbol of the same name and depmod would warn. The build pipeline runs gen-mod-lineinfo.sh after the existing modfinal step: gen_lineinfo --module ${KO} > ${KO}.lineinfo.S ${CC} -c -o ${KO}.lineinfo.o ${KO}.lineinfo.S ${LD} -r ${KO}.lineinfo.o ${KO} -o ${KO}.tmp && mv ${KO}.tmp ${KO} Order matters: lineinfo.o must come first so its zero-byte text contributions stay at offset 0 of the merged sections. The init blob lives in MOD_INIT_RODATA and is revoked via WRITE_ONCE in do_init_module() before do_free_init() releases the memory; the module_init_lineinfo_data() reader uses READ_ONCE so concurrent lookups either see the old pointer (still valid until do_free_init's synchronize_rcu) or NULL. The struct module fields are guarded by #ifdef CONFIG_KALLSYMS_LINEINFO_MODULES and accessed through inline reader accessors so callers don't duplicate the guard. Per-module overhead is approximately 14 bytes per DWARF line entry plus a small fixed cost per covered section descriptor. The next patch in this series delta-compresses the per-section streams to ~3-4 bytes per entry. Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Sasha Levin --- .../admin-guide/kallsyms-lineinfo.rst | 41 +- MAINTAINERS | 3 + include/linux/mod_lineinfo.h | 122 ++ include/linux/module.h | 40 + init/Kconfig | 13 + kernel/kallsyms.c | 19 +- kernel/module/kallsyms.c | 209 +++ kernel/module/main.c | 26 + scripts/Makefile.modfinal | 6 + scripts/gen-mod-lineinfo.sh | 50 + scripts/gen_lineinfo.c | 1134 +++++++++++++++-- 11 files changed, 1531 insertions(+), 132 deletions(-) create mode 100644 include/linux/mod_lineinfo.h create mode 100644 scripts/gen-mod-lineinfo.sh diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentatio= n/admin-guide/kallsyms-lineinfo.rst index 4c5de44e68fc..152037e62216 100644 --- a/Documentation/admin-guide/kallsyms-lineinfo.rst +++ b/Documentation/admin-guide/kallsyms-lineinfo.rst @@ -51,22 +51,47 @@ With ``CONFIG_KALLSYMS_LINEINFO``:: Note that assembly routines (such as ``entry_SYSCALL_64_after_hwframe``) a= re not annotated because they lack DWARF debug information. =20 +Module Support +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +``CONFIG_KALLSYMS_LINEINFO_MODULES`` extends the feature to loadable kernel +modules. When enabled, each ``.ko`` is post-processed at build time to em= bed +a ``.mod_lineinfo`` section containing the same kind of address-to-source +mapping. + +Enable in addition to the base options:: + + CONFIG_MODULES=3Dy + CONFIG_KALLSYMS_LINEINFO_MODULES=3Dy + +Stack traces from module code will then include annotations:: + + my_driver_func+0x30/0x100 [my_driver] (drivers/foo/bar.c:123) + +The ``.mod_lineinfo`` section is loaded into read-only module memory along= side +the module text. No additional runtime memory allocation is required; the= data +is freed when the module is unloaded. + Memory Overhead =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -The lineinfo tables are stored in ``.rodata`` and typically add approximat= ely -44 MiB to the kernel image for a standard configuration (~4.6 million DWARF -line entries, ~10 bytes per entry after deduplication). +The vmlinux lineinfo tables are stored in ``.rodata`` and typically add +approximately 10-15 MiB to the kernel image for a standard configuration +(~4.6 million DWARF line entries, ~2-3 bytes per entry after delta +compression). + +Per-module lineinfo adds approximately 2-3 bytes per DWARF line entry to e= ach +``.ko`` file. =20 Known Limitations =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -- **vmlinux only**: Only symbols in the core kernel image are annotated. - Module symbols are not covered. -- **4 GiB offset limit**: Address offsets from ``_text`` are stored as 32-= bit - values. Entries beyond 4 GiB from ``_text`` are skipped at build time w= ith - a warning. +- **4 GiB offset limit**: Address offsets from ``_text`` (vmlinux) or + ``.text`` base (modules) are stored as 32-bit values. Entries beyond + 4 GiB are skipped at build time with a warning. - **65535 file limit**: Source file IDs are stored as 16-bit values. Buil= ds with more than 65535 unique source files will fail with an error. - **No assembly annotations**: Functions implemented in assembly that lack DWARF ``.debug_line`` data are not annotated. +- **No init text**: For modules, functions in ``.init.text`` are not annot= ated + because that memory is freed after module initialization. diff --git a/MAINTAINERS b/MAINTAINERS index b98d57b1ee1d..d7e1c1de7998 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13935,6 +13935,9 @@ KALLSYMS LINEINFO M: Sasha Levin S: Maintained F: Documentation/admin-guide/kallsyms-lineinfo.rst +F: include/linux/mod_lineinfo.h +F: lib/tests/lineinfo_kunit.c +F: scripts/gen-mod-lineinfo.sh F: scripts/gen_lineinfo.c =20 KASAN diff --git a/include/linux/mod_lineinfo.h b/include/linux/mod_lineinfo.h new file mode 100644 index 000000000000..75d0bfba6ce5 --- /dev/null +++ b/include/linux/mod_lineinfo.h @@ -0,0 +1,122 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * mod_lineinfo.h - Binary format for per-module source line information + * + * This header defines the layout of the .mod_lineinfo and + * .init.mod_lineinfo sections embedded in loadable kernel modules. It + * is dual-use: included from both the kernel and the userspace + * gen_lineinfo tool. + * + * Top-level layout (all values in target-native endianness): + * + * struct mod_lineinfo_root + * struct mod_lineinfo_section sections[hdr.num_sections] + * ... per-section sub-tables, each pointed at by sections[i].table_offs= et + * + * Each mod_lineinfo_section descriptor identifies one ELF text section + * covered by the lineinfo blob. Its .anchor field is an ELF relocation + * resolved at module-load time to the runtime base of the named section, + * eliminating the need to derive the base from mod->mem[].base segments. + * If the relocation fails to resolve (e.g. unknown reloc type), .anchor + * stays zero and lookups silently degrade to "no annotation". + * + * Each per-section sub-table is laid out as a stand-alone + * mod_lineinfo_header followed by parallel arrays: + * + * struct mod_lineinfo_header (16 bytes) + * u32 addrs[num_entries] -- offsets from this section's base, s= orted + * u16 file_ids[num_entries] -- parallel to addrs + * <2-byte pad if num_entries is odd> + * u32 lines[num_entries] -- parallel to addrs + * u32 file_offsets[num_files] -- byte offset into filenames[] + * char filenames[filenames_size] -- concatenated NUL-terminated strings + */ +#ifndef _LINUX_MOD_LINEINFO_H +#define _LINUX_MOD_LINEINFO_H + +#ifdef __KERNEL__ +#include +#include +#include +#else +#include +#include +#include +typedef uint32_t u32; +typedef uint16_t u16; +typedef uint64_t u64; +#ifndef __aligned +#define __aligned(x) __attribute__((__aligned__(x))) +#endif +#ifndef static_assert +#define static_assert(e, ...) _Static_assert(e, #e) +#endif +#endif + +/* + * Per-section descriptor. One entry per ELF text section covered by the + * blob (.text, .exit.text, .init.text, ...). + */ +struct mod_lineinfo_section { + u64 anchor; /* RELOC: runtime base of covered section, or 0 */ + u32 size; /* covered section size in bytes */ + u32 table_offset; /* byte offset from blob start to this section's + * mod_lineinfo_header */ +} __aligned(8); + +/* + * Top-level header. Sits at offset 0 of every .mod_lineinfo / + * .init.mod_lineinfo section. The generated assembly pads to an 8-byte + * boundary after num_sections, so sections[0] must start at offset 8. + * The __aligned(8) on struct mod_lineinfo_section guarantees that even on + * 32-bit targets where the natural alignment of u64 is smaller (4 on i386, + * 2 on m68k) and the compiler would otherwise place sections[] at offset = 4. + */ +struct mod_lineinfo_root { + u32 num_sections; + struct mod_lineinfo_section sections[]; +}; + +static_assert(offsetof(struct mod_lineinfo_root, sections) =3D=3D 8, + "blob layout: sections[] must sit at offset 8 to match the generate= d assembly"); +static_assert(sizeof(struct mod_lineinfo_section) =3D=3D 16, + "blob layout: section descriptors are 16 bytes in the generated ass= embly"); + +struct mod_lineinfo_header { + u32 num_entries; + u32 num_files; + u32 filenames_size; /* total bytes of concatenated filenames */ +}; + +/* Offset helpers: compute byte offset from the per-section header to each= array. */ + +static inline u32 mod_lineinfo_addrs_off(void) +{ + return sizeof(struct mod_lineinfo_header); +} + +static inline u32 mod_lineinfo_file_ids_off(u32 num_entries) +{ + return mod_lineinfo_addrs_off() + num_entries * sizeof(u32); +} + +static inline u32 mod_lineinfo_lines_off(u32 num_entries) +{ + /* u16 file_ids[] may need 2-byte padding to align lines[] to 4 bytes */ + u32 off =3D mod_lineinfo_file_ids_off(num_entries) + + num_entries * sizeof(u16); + return (off + 3) & ~3u; +} + +static inline u32 mod_lineinfo_file_offsets_off(u32 num_entries) +{ + return mod_lineinfo_lines_off(num_entries) + num_entries * sizeof(u32); +} + +static inline u32 mod_lineinfo_filenames_off(u32 num_entries, u32 num_file= s) +{ + return mod_lineinfo_file_offsets_off(num_entries) + + num_files * sizeof(u32); +} + +#endif /* _LINUX_MOD_LINEINFO_H */ diff --git a/include/linux/module.h b/include/linux/module.h index 7566815fabbe..8ae14bb56abf 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -507,6 +507,12 @@ struct module { void *btf_data; void *btf_base_data; #endif +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + void *lineinfo_data; /* .mod_lineinfo section in MOD_RODATA */ + unsigned int lineinfo_data_size; + void *init_lineinfo_data; /* .init.mod_lineinfo, NULL after init runs */ + unsigned int init_lineinfo_data_size; +#endif #ifdef CONFIG_JUMP_LABEL struct jump_entry *jump_entries; unsigned int num_jump_entries; @@ -1020,6 +1026,40 @@ static inline unsigned long find_kallsyms_symbol_val= ue(struct module *mod, =20 #endif /* CONFIG_MODULES && CONFIG_KALLSYMS */ =20 +bool module_lookup_lineinfo(struct module *mod, unsigned long addr, + unsigned long sym_start, + const char **file, unsigned int *line); + +/* + * Reader accessors so callers don't need to duplicate the + * CONFIG_KALLSYMS_LINEINFO_MODULES guard around mod->lineinfo_data / + * mod->init_lineinfo_data field access. Setters/clearers in the loader + * use the field directly under a matching #ifdef. + */ +static inline void *module_lineinfo_data(const struct module *mod, + unsigned int *size) +{ +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + *size =3D mod->lineinfo_data_size; + return mod->lineinfo_data; +#else + *size =3D 0; + return NULL; +#endif +} + +static inline void *module_init_lineinfo_data(const struct module *mod, + unsigned int *size) +{ +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + *size =3D READ_ONCE(mod->init_lineinfo_data_size); + return READ_ONCE(mod->init_lineinfo_data); +#else + *size =3D 0; + return NULL; +#endif +} + /* Define __free(module_put) macro for struct module *. */ DEFINE_FREE(module_put, struct module *, if (_T) module_put(_T)) =20 diff --git a/init/Kconfig b/init/Kconfig index 635305ec0594..8c5cf00cb3ae 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2107,6 +2107,19 @@ config KALLSYMS_LINEINFO =20 If unsure, say N. =20 +config KALLSYMS_LINEINFO_MODULES + bool "Embed source file:line information in module stack traces" + depends on KALLSYMS_LINEINFO && MODULES + help + Extends KALLSYMS_LINEINFO to loadable kernel modules. Each .ko + gets a lineinfo table generated from its DWARF data at build time, + so stack traces from module code include (file.c:123) annotations. + + This requires libelf and libdw (from elfutils) on the build host. + Costs 10 bytes per DWARF line entry in each .ko. + + If unsure, say N. + # end of the "standard kernel features (expert users)" menu =20 config ARCH_HAS_MEMBARRIER_CALLBACKS diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index baf52e81118b..07cfc30c7ad6 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -574,12 +574,25 @@ static int __sprint_symbol(char *buffer, unsigned lon= g address, * replaced with bar()"); appending lineinfo there would produce a * confusing "foo (file:line)()". */ - if (add_lineinfo && IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) && !modname) { + if (add_lineinfo && IS_ENABLED(CONFIG_KALLSYMS_LINEINFO)) { const char *li_file; unsigned int li_line; + bool found =3D false; + + if (!modname) { + found =3D kallsyms_lookup_lineinfo(address, sym_start, + &li_file, &li_line); + } else if (IS_ENABLED(CONFIG_KALLSYMS_LINEINFO_MODULES)) { + struct module *mod =3D __module_address(address); + + if (mod) + found =3D module_lookup_lineinfo(mod, address, + sym_start, + &li_file, + &li_line); + } =20 - if (kallsyms_lookup_lineinfo(address, sym_start, - &li_file, &li_line)) + if (found) len +=3D snprintf(buffer + len, KSYM_SYMBOL_LEN - len, " (%s:%u)", li_file, li_line); } diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c index 0fc11e45df9b..6615a14efba2 100644 --- a/kernel/module/kallsyms.c +++ b/kernel/module/kallsyms.c @@ -494,3 +494,212 @@ int module_kallsyms_on_each_symbol(const char *modnam= e, mutex_unlock(&module_mutex); return ret; } + +#include + +/* + * Search one per-section sub-table for @section_offset using flat parallel + * arrays. @hdr is the per-section header at byte offset @hdr_offset with= in + * @blob. Returns true on hit and populates @file / @line. + */ +static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size, + u32 hdr_offset, + unsigned int section_offset, + unsigned int min_offset, + const char **file, + unsigned int *line) +{ + const struct mod_lineinfo_header *hdr; + const u8 *base; + const u32 *addrs, *lines, *file_offsets; + const u16 *file_ids; + const char *filenames; + u32 num_entries, num_files, filenames_size; + unsigned int low, high, mid; + u16 file_id; + + if (hdr_offset > blob_size || + blob_size - hdr_offset < sizeof(*hdr)) + return false; + + /* + * The header and every array it points at are u32-aligned by + * construction. Refuse anything else rather than take an alignment + * fault here -- this runs from NMI and panic context, where a + * recursive fault would lose the backtrace entirely. + */ + if (!IS_ALIGNED(hdr_offset, sizeof(u32))) + return false; + + base =3D (const u8 *)blob + hdr_offset; + hdr =3D (const struct mod_lineinfo_header *)base; + num_entries =3D hdr->num_entries; + num_files =3D hdr->num_files; + filenames_size =3D hdr->filenames_size; + + if (num_entries =3D=3D 0) + return false; + + /* + * Validate counts before multiplying =E2=80=94 sizing arithmetic could + * otherwise overflow on 32-bit with a malformed blob. Each entry + * contributes one u32 (addrs), one u16 (file_ids), and one u32 + * (lines); each file contributes one u32 (file_offsets). + */ + { + u32 avail =3D blob_size - hdr_offset; + u32 needed =3D mod_lineinfo_filenames_off(num_entries, num_files); + + if (num_entries > U32_MAX / sizeof(u32)) + return false; + if (num_files > U32_MAX / sizeof(u32)) + return false; + if (needed > avail || filenames_size > avail - needed) + return false; + } + + /* + * Filenames are read as NUL-terminated C strings. Require the blob + * to end in NUL so a malformed file_offsets entry can never lead the + * later "%s" consumer past the end of the section. + */ + if (filenames_size =3D=3D 0 || + base[mod_lineinfo_filenames_off(num_entries, num_files) + + filenames_size - 1] !=3D 0) + return false; + + addrs =3D (const u32 *)(base + mod_lineinfo_addrs_off()); + file_ids =3D (const u16 *)(base + mod_lineinfo_file_ids_off(num_entries)); + lines =3D (const u32 *)(base + mod_lineinfo_lines_off(num_entries)); + file_offsets =3D (const u32 *)(base + mod_lineinfo_file_offsets_off(num_e= ntries)); + filenames =3D (const char *)(base + mod_lineinfo_filenames_off(num_entrie= s, num_files)); + + /* Binary search for largest entry <=3D section_offset. */ + low =3D 0; + high =3D num_entries; + while (low < high) { + mid =3D low + (high - low) / 2; + if (addrs[mid] <=3D section_offset) + low =3D mid + 1; + else + high =3D mid; + } + + if (low =3D=3D 0) + return false; + low--; + + /* + * Reject entries below the resolved symbol's start so a symbol + * without line entries of its own does not inherit the preceding + * symbol's annotation. + */ + if (addrs[low] < min_offset) + return false; + + file_id =3D file_ids[low]; + if (file_id >=3D num_files) + return false; + if (file_offsets[file_id] >=3D filenames_size) + return false; + + *file =3D &filenames[file_offsets[file_id]]; + *line =3D lines[low]; + return true; +} + +/* + * Walk a single .mod_lineinfo / .init.mod_lineinfo blob, find the section + * descriptor whose [anchor, anchor+size) range contains @addr, then search + * that section's sub-table. + */ +static bool module_lookup_lineinfo_blob(const void *blob, u32 blob_size, + unsigned long addr, + unsigned long sym_start, + const char **file, unsigned int *line) +{ + const struct mod_lineinfo_root *root; + u32 i, sections_end; + + if (!blob || blob_size < sizeof(*root)) + return false; + + /* The section is emitted with .balign 8; see the note above. */ + if (!IS_ALIGNED((unsigned long)blob, __alignof__(struct mod_lineinfo_sect= ion))) + return false; + + root =3D blob; + if (root->num_sections =3D=3D 0) + return false; + + if (root->num_sections > U32_MAX / sizeof(struct mod_lineinfo_section)) + return false; + sections_end =3D sizeof(*root) + + root->num_sections * sizeof(struct mod_lineinfo_section); + if (sections_end > blob_size) + return false; + + for (i =3D 0; i < root->num_sections; i++) { + const struct mod_lineinfo_section *s =3D &root->sections[i]; + unsigned long base =3D (unsigned long)s->anchor; + unsigned long offset, min_offset =3D 0; + + if (!base) + continue; /* relocation didn't resolve */ + if (addr < base) + continue; + offset =3D addr - base; + /* s->size is u32, so this also bounds offset to u32. */ + if (offset >=3D s->size) + continue; + + if (sym_start > base && sym_start - base <=3D offset) + min_offset =3D sym_start - base; + + return module_lookup_lineinfo_section(blob, blob_size, + s->table_offset, + offset, min_offset, + file, line); + } + + return false; +} + +/* + * Look up source file:line for an address within a loaded module. + * + * Safe in NMI/panic context: no locks, no allocations. + * Caller must hold RCU read lock (or be in a context where the module + * cannot be unloaded). + */ +bool module_lookup_lineinfo(struct module *mod, unsigned long addr, + unsigned long sym_start, + const char **file, unsigned int *line) +{ + const void *blob; + unsigned int size; + + if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO_MODULES)) + return false; + + blob =3D module_lineinfo_data(mod, &size); + if (blob && module_lookup_lineinfo_blob(blob, size, addr, sym_start, + file, line)) + return true; + + /* + * The init blob lives in MOD_INIT_RODATA and is revoked by + * do_init_module() before do_free_init() releases the memory. The + * READ_ONCE inside module_init_lineinfo_data() pairs with the + * WRITE_ONCE in do_init_module so we never see a partial + * pointer/size pair, and an RCU grace period in do_free_init() + * guarantees the memory still exists for the duration of any lookup + * that captured the pointer before the revocation. + */ + blob =3D module_init_lineinfo_data(mod, &size); + if (blob && module_lookup_lineinfo_blob(blob, size, addr, sym_start, + file, line)) + return true; + + return false; +} diff --git a/kernel/module/main.c b/kernel/module/main.c index 46dd8d25a605..46bb2bf799d1 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -2712,6 +2712,19 @@ static int find_module_sections(struct module *mod, = struct load_info *info) mod->btf_base_data =3D any_section_objs(info, ".BTF.base", 1, &mod->btf_base_data_size); #endif +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + /* + * Use section_objs() (not any_section_objs) =E2=80=94 both blobs carry an + * ELF anchor relocation that the module loader resolves via its + * standard apply_relocations() pass, which only walks SHF_ALLOC + * sections. Picking up a non-ALLOC section here would also leave + * the pointer dangling into the temporary load image once freed. + */ + mod->lineinfo_data =3D section_objs(info, ".mod_lineinfo", 1, + &mod->lineinfo_data_size); + mod->init_lineinfo_data =3D section_objs(info, ".init.mod_lineinfo", 1, + &mod->init_lineinfo_data_size); +#endif #ifdef CONFIG_JUMP_LABEL mod->jump_entries =3D section_objs(info, "__jump_table", sizeof(*mod->jump_entries), @@ -3165,6 +3178,19 @@ static noinline int do_init_module(struct module *mo= d) /* .BTF is not SHF_ALLOC and will get removed, so sanitize pointers */ mod->btf_data =3D NULL; mod->btf_base_data =3D NULL; +#endif +#ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + /* + * .init.mod_lineinfo lives in MOD_INIT_RODATA which do_free_init() is + * about to release. Clear the pointer so concurrent stack-trace + * lookups stop dereferencing it; do_free_init()'s synchronize_rcu() + * then waits out any reader that already captured the old pointer. + * WRITE_ONCE pairs with the READ_ONCE inside module_init_lineinfo_data() + * so the compiler can't tear or reorder the revocation across the + * llist_add() that follows. + */ + WRITE_ONCE(mod->init_lineinfo_data, NULL); + WRITE_ONCE(mod->init_lineinfo_data_size, 0); #endif /* * We want to free module_init, but be aware that kallsyms may be diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 01a37ec872b9..2cdd76becb4d 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -46,11 +46,17 @@ quiet_cmd_btf_ko =3D BTF [M] $@ $(CONFIG_SHELL) $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmli= nux $@; \ fi; =20 +quiet_cmd_lineinfo_ko =3D LINEINFO [M] $@ + cmd_lineinfo_ko =3D $(CONFIG_SHELL) $(srctree)/scripts/gen-mod-linei= nfo.sh $@ + # Re-generate module BTFs if either module's .ko or vmlinux changed %.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(C= ONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE +$(call if_changed,ld_ko_o) ifdef CONFIG_DEBUG_INFO_BTF_MODULES +$(if $(newer-prereqs),$(call cmd,btf_ko)) +endif +ifdef CONFIG_KALLSYMS_LINEINFO_MODULES + +$(if $(newer-prereqs),$(call cmd,lineinfo_ko)) endif +$(call cmd,check_tracepoint) =20 diff --git a/scripts/gen-mod-lineinfo.sh b/scripts/gen-mod-lineinfo.sh new file mode 100644 index 000000000000..832d290f3bf4 --- /dev/null +++ b/scripts/gen-mod-lineinfo.sh @@ -0,0 +1,50 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# gen-mod-lineinfo.sh - Embed source line info into a kernel module (.ko) +# +# Reads DWARF from the .ko, generates a .mod_lineinfo section that contains +# an ELF relocation against the module's .text section symbol, and partial- +# links the result back into the .ko via "ld -r" so the relocation rides +# along to the module loader. Modeled on scripts/gen-btf.sh. + +set -e + +if [ $# -ne 1 ]; then + echo "Usage: $0 " >&2 + exit 1 +fi + +KO=3D"$1" + +cleanup() { + rm -f "${KO}.lineinfo.S" "${KO}.lineinfo.o" "${KO}.lineinfo.tmp" +} +trap cleanup EXIT + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + +# Generate assembly from DWARF -- if it fails (no DWARF), silently skip +if ! ${objtree}/scripts/gen_lineinfo --module "${KO}" > "${KO}.lineinfo.S"= ; then + exit 0 +fi + +# Compile assembly to object file +${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \ + ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_MODULE} \ + -c -o "${KO}.lineinfo.o" "${KO}.lineinfo.S" + +# Partial-link lineinfo.o INTO the .ko. Order matters: lineinfo.o must co= me +# FIRST so its empty .text contributes 0 bytes at offset 0 of the merged +# .text, which keeps the .quad .text relocation (against lineinfo.o's local +# .text symbol, which after merge points at offset 0 of merged .text) +# resolving to the start of the module's .text. Reversing inputs here +# silently breaks lookup correctness. +${LD} -r "${KO}.lineinfo.o" "${KO}" -o "${KO}.lineinfo.tmp" +mv "${KO}.lineinfo.tmp" "${KO}" + +exit 0 diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c index a3ee54eaad05..725a8dfbf011 100644 --- a/scripts/gen_lineinfo.c +++ b/scripts/gen_lineinfo.c @@ -49,16 +49,79 @@ static bool verbose; exit(1); \ } while (0) =20 +#include "../include/linux/mod_lineinfo.h" + +static int module_mode; + static unsigned int skipped_overflow; =20 +/* Target ELF traits, captured once in main() and reused at emit time. */ +static bool target_64bit; +static bool target_le; + /* - * vmlinux mode: end of the invariant .text region. Zero means "no cap" - * (graceful fallback when _etext is absent on some build). + * Vmlinux mode only: address range of the *invariant* .text region. + * See find_text_end_addr() for why we cap on _etext. text_end_addr =3D= =3D 0 + * means "no cap available; capture everything above text_addr" (v3 + * behavior, used as graceful fallback if _etext is absent). */ static unsigned long long text_end_addr; =20 +/* + * In module mode we cover several text-like sections, split across two + * output blobs by lifecycle: + * + * .mod_lineinfo -- persistent code (.text, .exit.text); MOD_RODATA + * .init.mod_lineinfo -- init code (.init.text); freed with init memory + * + * In ET_REL .ko files .text/.init.text/.exit.text all have sh_addr =3D=3D= 0, + * so DWARF line addresses (which become sh_addr + addend after relocation) + * collide across sections. We disambiguate by giving each *present* + * covered section a unique synthetic "bias" =E2=80=94 a u32 base address = =E2=80=94 and + * adding that bias to relocated values inside apply_debug_line_relocation= s. + * libdw then yields biased addresses that classify_address() can map back + * to a single section unambiguously. The bias is internal to gen_lineinfo + * and never leaks into the emitted blob. + */ +enum mod_lineinfo_blob { + BLOB_PERSISTENT, + BLOB_INIT, + NUM_BLOBS, +}; + +struct covered_section { + const char *name; /* ELF section name (e.g. ".text") */ + enum mod_lineinfo_blob blob; + unsigned long long bias;/* synthetic base address (set in resolve_*) */ + unsigned long long size; + bool present; /* found in this .ko */ + unsigned int sec_index; /* ELF section header index, for reloc matching */ + unsigned int n_entries; /* DWARF line entries collected for this section = */ +}; + +static struct covered_section all_sections[] =3D { + { .name =3D ".text", .blob =3D BLOB_PERSISTENT }, + { .name =3D ".exit.text", .blob =3D BLOB_PERSISTENT }, + { .name =3D ".init.text", .blob =3D BLOB_INIT }, + { .name =3D ".noinstr.text", .blob =3D BLOB_PERSISTENT }, +}; +#define ALL_SECTIONS ARRAY_SIZE(all_sections) + +/* + * Executable sections present in the .ko but not covered by the blob + * (e.g. .static_call.text, .altinstr_replacement, or the per-function + * .text.* subsections parisc32 modules keep). They get synthetic biases + * from the same cursor as the covered sections so their DWARF sequences + * classify unambiguously and can be dropped, instead of aliasing into + * .text's [0, size) range. + */ +static struct covered_section *extra_sections; +static unsigned int num_extra_sections; +static unsigned long long skipped_uncovered; + struct line_entry { - unsigned int offset; /* offset from _text */ + unsigned int offset; /* offset from covered section's start */ + unsigned int section_id;/* index into covered_sections[] (module mode onl= y) */ unsigned int file_id; unsigned int line; }; @@ -84,14 +147,15 @@ static unsigned int files_capacity; =20 static HASHTABLE_DEFINE(file_hashtable, 1U << 13); =20 -static void add_entry(unsigned int offset, unsigned int file_id, - unsigned int line) +static void add_entry(unsigned int offset, unsigned int section_id, + unsigned int file_id, unsigned int line) { if (num_entries >=3D entries_capacity) { entries_capacity =3D entries_capacity ? entries_capacity * 2 : 65536; entries =3D xrealloc(entries, entries_capacity * sizeof(*entries)); } entries[num_entries].offset =3D offset; + entries[num_entries].section_id =3D section_id; entries[num_entries].file_id =3D file_id; entries[num_entries].line =3D line; num_entries++; @@ -303,6 +367,20 @@ static const char *make_relative(const char *path, con= st char *comp_dir) static char buf[PATH_MAX]; const char *p; =20 + if (path[0] !=3D '/') { + /* + * libdw doubles relative paths on ET_REL input + * (e.g. "a/b.c/a/b.c" -> "a/b.c"). Undo that before the + * path is joined to comp_dir, or the halves stop matching. + */ + size_t len =3D strlen(path); + size_t mid =3D len / 2; + + if (len > 1 && path[mid] =3D=3D '/' && + !memcmp(path, path + mid + 1, mid)) + path +=3D mid + 1; + } + if (path[0] =3D=3D '/') { if (snprintf(buf, sizeof(buf), "%s", path) >=3D (int)sizeof(buf)) return path; @@ -352,6 +430,9 @@ static int compare_entries(const void *a, const void *b) const struct line_entry *ea =3D a; const struct line_entry *eb =3D b; =20 + /* Group by section first so each per-section table is contiguous. */ + if (ea->section_id !=3D eb->section_id) + return ea->section_id < eb->section_id ? -1 : 1; if (ea->offset !=3D eb->offset) return ea->offset < eb->offset ? -1 : 1; if (ea->file_id !=3D eb->file_id) @@ -363,7 +444,8 @@ static int compare_entries(const void *a, const void *b) =20 /* * Look up a vmlinux symbol by exact name and return its st_value, or - * @fallback if absent. Aborts when @required and the symbol is missing. + * @fallback if the symbol is absent (lets callers gracefully skip + * optional bounds like _etext). */ static unsigned long long find_vmlinux_sym(Elf *elf, const char *name, unsigned long long fallback, @@ -409,33 +491,43 @@ static unsigned long long find_text_addr(Elf *elf) } =20 /* - * vmlinux is linked in multiple passes: gen_lineinfo runs against - * .tmp_vmlinux1 (which carries an empty lineinfo stub), then real tables - * are linked in for the final image. Sections placed AFTER .rodata - * (.init.text, .exit.text, ...) shift forward as .rodata grows to hold - * the real lineinfo blob, so DWARF addresses we'd capture for them in - * pass 1 would be stale in the final kernel. Cap captured addresses at - * _etext, the symbol that marks the end of .text =E2=80=94 placed before = .rodata - * in every architecture's vmlinux.lds.S, so its addresses are invariant - * across the relink. Returns 0 if _etext is absent (no cap; v3 behavior). + * Vmlinux is linked in multiple passes: gen_lineinfo runs against + * .tmp_vmlinux1 (which carries the empty lineinfo stub), and the resulting + * tables are then linked into the final vmlinux. Sections placed AFTER + * .rodata (.init.text, .exit.text, ...) shift forward as the real lineinfo + * tables replace the empty stub, so DWARF addresses we'd capture for them + * here are stale by the time the kernel runs. + * + * Cap the captured range at _etext, the symbol that marks the end of the + * .text section. .text is placed BEFORE .rodata in every architecture's + * vmlinux.lds.S, so its addresses are invariant across the relink. + * Returns 0 on architectures or builds that don't expose _etext, in which + * case the cap is disabled (preserving the v3 behavior =E2=80=94 addresse= s past + * .text remain captured but may be off in stack traces). */ static unsigned long long find_text_end_addr(Elf *elf) { return find_vmlinux_sym(elf, "_etext", 0, false); } =20 -static int compare_uints(const void *a, const void *b) +/* + * Ordering shared by entries[], sym_starts[] and seq_ends[]: section firs= t, + * then offset. In module mode every offset is section-relative, so the t= wo + * together are what identifies a location. + */ +static int compare_sec_off(unsigned int sa, unsigned int oa, + unsigned int sb, unsigned int ob) { - unsigned int ua =3D *(const unsigned int *)a; - unsigned int ub =3D *(const unsigned int *)b; - - if (ua !=3D ub) - return ua < ub ? -1 : 1; + if (sa !=3D sb) + return sa < sb ? -1 : 1; + if (oa !=3D ob) + return oa < ob ? -1 : 1; return 0; } =20 /* Sorted, duplicate-free extents of every function symbol. */ struct sym_start { + unsigned int section_id; unsigned int offset; unsigned int size; }; @@ -444,66 +536,45 @@ static struct sym_start *sym_starts; static unsigned int num_sym_starts; static unsigned int sym_starts_capacity; =20 -/* Sorted offsets one past the end of each DWARF line-program sequence. */ -static unsigned int *seq_ends; +/* Sorted locations one past the end of each DWARF line-program sequence. = */ +struct seq_end { + unsigned int section_id; + unsigned int offset; +}; + +static struct seq_end *seq_ends; static unsigned int num_seq_ends; static unsigned int seq_ends_capacity; =20 -static void append_offset(unsigned int **arr, unsigned int *count, - unsigned int *capacity, unsigned int value) +static int compare_seq_ends(const void *a, const void *b) { - if (*count >=3D *capacity) { - *capacity =3D *capacity ? *capacity * 2 : 16384; - *arr =3D xrealloc(*arr, *capacity * sizeof(**arr)); - } - (*arr)[(*count)++] =3D value; -} + const struct seq_end *ea =3D a, *eb =3D b; =20 -static void sort_unique(unsigned int *arr, unsigned int *count) -{ - unsigned int j =3D 0; - - if (*count < 2) - return; - - qsort(arr, *count, sizeof(*arr), compare_uints); - for (unsigned int i =3D 1; i < *count; i++) { - if (arr[i] =3D=3D arr[j]) - continue; - if (++j !=3D i) - arr[j] =3D arr[i]; - } - *count =3D j + 1; + return compare_sec_off(ea->section_id, ea->offset, + eb->section_id, eb->offset); } =20 -/* - * Record the end of a line-program sequence. @addr is one past the last - * covered byte, so the sequence's own coverage is tested using addr - 1. - */ -static void record_seq_end(unsigned long long addr, - unsigned long long text_addr) +static void record_seq_end(unsigned int section_id, unsigned int offset) { - unsigned long long raw; - - if (addr <=3D text_addr) - return; - if (text_end_addr && addr - 1 >=3D text_end_addr) - return; - - raw =3D addr - text_addr; - if (raw > UINT_MAX) - return; - - append_offset(&seq_ends, &num_seq_ends, &seq_ends_capacity, - (unsigned int)raw); + if (num_seq_ends >=3D seq_ends_capacity) { + seq_ends_capacity =3D seq_ends_capacity ? + seq_ends_capacity * 2 : 16384; + seq_ends =3D xrealloc(seq_ends, + seq_ends_capacity * sizeof(*seq_ends)); + } + seq_ends[num_seq_ends].section_id =3D section_id; + seq_ends[num_seq_ends].offset =3D offset; + num_seq_ends++; } =20 static int compare_sym_starts(const void *a, const void *b) { const struct sym_start *sa =3D a, *sb =3D b; + int ret =3D compare_sec_off(sa->section_id, sa->offset, + sb->section_id, sb->offset); =20 - if (sa->offset !=3D sb->offset) - return sa->offset < sb->offset ? -1 : 1; + if (ret) + return ret; /* Larger extent first, so the dedup below keeps it. */ if (sa->size !=3D sb->size) return sa->size > sb->size ? -1 : 1; @@ -516,7 +587,9 @@ static int compare_sym_starts(const void *a, const void= *b) * the kernel's symbol-boundary check rejects the preceding function's ent= ry * and the frame goes unannotated. */ -static void collect_symbol_starts(Elf *elf, unsigned long long text_addr) +static void collect_symbol_starts(Elf *elf, unsigned long long text_addr, + struct covered_section *sections, + unsigned int num_sections) { Elf_Scn *scn =3D NULL; GElf_Shdr shdr; @@ -536,19 +609,40 @@ static void collect_symbol_starts(Elf *elf, unsigned = long long text_addr) =20 nsyms =3D shdr.sh_size / shdr.sh_entsize; for (size_t i =3D 0; i < nsyms; i++) { - GElf_Sym sym; + unsigned int sec_id =3D 0; unsigned long long raw; + GElf_Sym sym; =20 if (!gelf_getsym(data, i, &sym)) continue; if (GELF_ST_TYPE(sym.st_info) !=3D STT_FUNC) continue; - if (sym.st_value < text_addr) - continue; - if (text_end_addr && sym.st_value >=3D text_end_addr) - continue; =20 - raw =3D sym.st_value - text_addr; + if (module_mode) { + /* + * ET_REL: st_value is already relative to the + * symbol's own section, so only sections the + * blob covers are of interest. + */ + for (sec_id =3D 0; sec_id < num_sections; sec_id++) + if (sections[sec_id].present && + sections[sec_id].sec_index =3D=3D + sym.st_shndx) + break; + if (sec_id =3D=3D num_sections) + continue; + if (sym.st_value >=3D sections[sec_id].size) + continue; + raw =3D sym.st_value; + } else { + if (sym.st_value < text_addr) + continue; + if (text_end_addr && + sym.st_value >=3D text_end_addr) + continue; + raw =3D sym.st_value - text_addr; + } + if (raw > UINT_MAX) continue; =20 @@ -559,6 +653,7 @@ static void collect_symbol_starts(Elf *elf, unsigned lo= ng long text_addr) sym_starts_capacity * sizeof(*sym_starts)); } + sym_starts[num_sym_starts].section_id =3D sec_id; sym_starts[num_sym_starts].offset =3D (unsigned int)raw; sym_starts[num_sym_starts].size =3D sym.st_size > UINT_MAX ? UINT_MAX : @@ -573,7 +668,10 @@ static void collect_symbol_starts(Elf *elf, unsigned l= ong long text_addr) qsort(sym_starts, num_sym_starts, sizeof(*sym_starts), compare_sym_starts); for (unsigned int i =3D 1; i < num_sym_starts; i++) { - if (sym_starts[i].offset =3D=3D sym_starts[j].offset) + if (!compare_sec_off(sym_starts[i].section_id, + sym_starts[i].offset, + sym_starts[j].section_id, + sym_starts[j].offset)) continue; if (++j !=3D i) sym_starts[j] =3D sym_starts[i]; @@ -582,7 +680,454 @@ static void collect_symbol_starts(Elf *elf, unsigned = long long text_addr) } } =20 -static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr) +/* + * Populate @sections[].present/sec_index/size/bias. Sections that don't + * exist stay marked absent. Biases are assigned in array order: each + * present section gets a base equal to the running total of preceding + * present sections' sizes, rounded up to 16 to keep ranges sparse. This + * guarantees [bias, bias+size) ranges are pairwise disjoint and fit in + * u32 as long as the sum of all covered text sizes is below 4 GiB. + */ +static void resolve_covered_sections(Elf *elf, + struct covered_section *sections, + unsigned int num_sections) +{ + Elf_Scn *scn =3D NULL; + GElf_Shdr shdr; + size_t shstrndx; + unsigned long long cursor =3D 0; + + if (elf_getshdrstrndx(elf, &shstrndx) !=3D 0) + return; + + while ((scn =3D elf_nextscn(elf, scn)) !=3D NULL) { + const char *name; + + if (!gelf_getshdr(scn, &shdr)) + continue; + name =3D elf_strptr(elf, shstrndx, shdr.sh_name); + if (!name) + continue; + bool covered =3D false; + + for (unsigned int i =3D 0; i < num_sections; i++) { + if (sections[i].present) + continue; + if (strcmp(name, sections[i].name)) + continue; + if (shdr.sh_size > UINT_MAX) { + warn("section %s exceeds 4 GiB (size=3D%llu); skipping", + name, (unsigned long long)shdr.sh_size); + break; + } + sections[i].sec_index =3D elf_ndxscn(scn); + sections[i].size =3D shdr.sh_size; + sections[i].present =3D true; + covered =3D true; + break; + } + + /* + * Track every other executable section too, so its DWARF + * sequences can be biased into their own range and dropped + * instead of polluting a covered section's table. + */ + if (!covered && + (shdr.sh_flags & SHF_EXECINSTR) && (shdr.sh_flags & SHF_ALLOC) && + shdr.sh_size && shdr.sh_size <=3D UINT_MAX) { + struct covered_section *es; + + extra_sections =3D xrealloc(extra_sections, + (num_extra_sections + 1) * + sizeof(*extra_sections)); + es =3D &extra_sections[num_extra_sections++]; + memset(es, 0, sizeof(*es)); + es->name =3D name; + es->sec_index =3D elf_ndxscn(scn); + es->size =3D shdr.sh_size; + es->present =3D true; + } + } + + /* Pack present sections into non-overlapping bias ranges. */ + for (unsigned int i =3D 0; i < num_sections; i++) { + if (!sections[i].present) + continue; + sections[i].bias =3D cursor; + cursor +=3D sections[i].size; + cursor =3D (cursor + 15) & ~15ULL; /* pad for separation */ + } + for (unsigned int i =3D 0; i < num_extra_sections; i++) { + extra_sections[i].bias =3D cursor; + cursor +=3D extra_sections[i].size; + cursor =3D (cursor + 15) & ~15ULL; + } +} + +/* Look up a covered_section by ELF section header index. */ +static struct covered_section *section_by_index(struct covered_section *se= ctions, + unsigned int num_sections, + unsigned int sec_index) +{ + for (unsigned int i =3D 0; i < num_sections; i++) { + if (sections[i].present && sections[i].sec_index =3D=3D sec_index) + return §ions[i]; + } + return NULL; +} + +/* + * Apply .rela.debug_line relocations to a mutable copy of .debug_line dat= a. + * + * elfutils libdw (through at least 0.194) does NOT apply relocations for + * ET_REL files when using dwarf_begin_elf(). The internal libdwfl layer + * does this via __libdwfl_relocate(), but that API is not public. + * + * For DWARF5, the .debug_line file name table uses DW_FORM_line_strp + * references into .debug_line_str. Without relocation, all these offsets + * resolve to 0 (or garbage), causing dwarf_linesrc()/dwarf_filesrc() to + * return wrong filenames (typically the comp_dir for every file). + * + * This function applies the relocations manually so that the patched + * .debug_line data can be fed to dwarf_begin_elf() and produce correct + * results. + * + * See elfutils bug https://sourceware.org/bugzilla/show_bug.cgi?id=3D31447 + * A fix (dwelf_elf_apply_relocs) was proposed but not yet merged as of + * elfutils 0.194: https://sourceware.org/pipermail/elfutils-devel/2024q3/= 007388.html + */ +/* + * Determine the relocation type for a 32-bit absolute reference + * on the given architecture. Returns 0 if unknown. + */ +static unsigned int r_type_abs32(unsigned int e_machine) +{ + switch (e_machine) { + case EM_X86_64: return R_X86_64_32; + case EM_386: return R_386_32; + case EM_AARCH64: return R_AARCH64_ABS32; + case EM_ARM: return R_ARM_ABS32; + case EM_RISCV: return R_RISCV_32; + case EM_S390: return R_390_32; + case EM_MIPS: return R_MIPS_32; + case EM_PPC64: return R_PPC64_ADDR32; + case EM_PPC: return R_PPC_ADDR32; + case EM_LOONGARCH: return R_LARCH_32; + case EM_PARISC: return R_PARISC_DIR32; + default: return 0; + } +} + +/* + * Determine the relocation type for a 64-bit absolute reference + * on the given architecture. Returns 0 on 32-bit-only architectures + * (where DW_LNE_set_address fits in 32 bits and r_type_abs32 covers it). + */ +static unsigned int r_type_abs64(unsigned int e_machine) +{ + switch (e_machine) { + case EM_X86_64: return R_X86_64_64; + case EM_AARCH64: return R_AARCH64_ABS64; + case EM_RISCV: return R_RISCV_64; + case EM_S390: return R_390_64; + case EM_MIPS: return R_MIPS_64; + case EM_PPC64: return R_PPC64_ADDR64; + case EM_LOONGARCH: return R_LARCH_64; + case EM_PARISC: return R_PARISC_DIR64; + default: return 0; + } +} + +/* + * Write a 4- or 8-byte unsigned integer in target byte order. + * Cross-builds (e.g. x86_64 host -> s390 module) need the patched + * .debug_line bytes laid out per the .ko's e_ident[EI_DATA], not the host= 's. + */ +static void elf_write_uint(unsigned char *dst, uint64_t value, size_t size, + bool little_endian) +{ + if (little_endian) { + for (size_t i =3D 0; i < size; i++) + dst[i] =3D (value >> (i * 8)) & 0xff; + } else { + for (size_t i =3D 0; i < size; i++) + dst[i] =3D (value >> ((size - 1 - i) * 8)) & 0xff; + } +} + +/* Counterpart to elf_write_uint: read the implicit addend of an SHT_REL + * relocation, stored in the relocated field itself in target byte order. + */ +static uint64_t elf_read_uint(const unsigned char *src, size_t size, + bool little_endian) +{ + uint64_t value =3D 0; + + if (little_endian) { + for (size_t i =3D 0; i < size; i++) + value |=3D (uint64_t)src[i] << (i * 8); + } else { + for (size_t i =3D 0; i < size; i++) + value |=3D (uint64_t)src[i] << ((size - 1 - i) * 8); + } + return value; +} + +/* + * Apply one .debug_line relocation. Two reloc widths matter: + * abs32 - DW_FORM_line_strp file-table refs into .debug_line_str + * abs64 - DW_LNE_set_address arguments (sequence start PCs) + * Without both, libdw sees zeros and reports wrong filenames or collapses + * every sequence to address 0 (collision after dedup). + * + * @has_addend distinguishes RELA records (explicit @addend) from REL + * records, whose addend is read from the relocated field itself. + */ +static void apply_one_dl_reloc(Elf_Data *dl_data, Elf_Data *sym_data, + bool target_le, + unsigned int abs32_type, unsigned int abs64_type, + GElf_Xword r_info, GElf_Addr r_offset, + GElf_Sxword addend, bool has_addend) +{ + GElf_Sym sym; + unsigned int r_type =3D GELF_R_TYPE(r_info); + size_t r_sym =3D GELF_R_SYM(r_info); + bool is_abs64; + size_t width; + uint64_t value; + + if (abs32_type && r_type =3D=3D abs32_type) + is_abs64 =3D false; + else if (abs64_type && r_type =3D=3D abs64_type) + is_abs64 =3D true; + else + return; + + if (!gelf_getsym(sym_data, r_sym, &sym)) + return; + + width =3D is_abs64 ? 8 : 4; + + if (r_offset + width > dl_data->d_size) + return; + + if (!has_addend) + addend =3D (GElf_Sxword)elf_read_uint( + (unsigned char *)dl_data->d_buf + r_offset, + width, target_le); + + value =3D (uint64_t)(sym.st_value + addend); + + /* + * If the relocation targets one of the tracked text sections, fold + * in that section's synthetic bias so the patched DWARF address + * lands in a unique numeric range. String-ref relocs + * (DW_FORM_line_strp into .debug_line_str) target a different + * section, so the symbol-based check correctly excludes them from + * biasing =E2=80=94 for both abs64 (64-bit ELF) and abs32 (32-bit ELF, + * where DW_LNE_set_address is also 4 bytes wide). + */ + if (module_mode) { + struct covered_section *cs; + + cs =3D section_by_index(all_sections, ALL_SECTIONS, + sym.st_shndx); + if (!cs) + cs =3D section_by_index(extra_sections, + num_extra_sections, + sym.st_shndx); + if (cs) + value +=3D cs->bias; + } + + if (!is_abs64) + value &=3D 0xffffffffULL; + + elf_write_uint((unsigned char *)dl_data->d_buf + r_offset, + value, width, target_le); +} + +/* Walk one .rela.debug_line / .rel.debug_line table, if present. */ +static void apply_dl_reloc_table(Elf_Scn *scn, bool is_rela, + Elf_Data *dl_data, Elf_Data *sym_data, + bool target_le, + unsigned int abs32_type, + unsigned int abs64_type) +{ + GElf_Shdr shdr; + Elf_Data *data; + size_t nrels, i; + + if (!scn) + return; + + data =3D elf_getdata(scn, NULL); + if (!data || !gelf_getshdr(scn, &shdr) || !shdr.sh_entsize) + return; + + nrels =3D shdr.sh_size / shdr.sh_entsize; + + for (i =3D 0; i < nrels; i++) { + if (is_rela) { + GElf_Rela rela; + + if (!gelf_getrela(data, i, &rela)) + continue; + apply_one_dl_reloc(dl_data, sym_data, target_le, + abs32_type, abs64_type, + rela.r_info, rela.r_offset, + rela.r_addend, true); + } else { + GElf_Rel rel; + + if (!gelf_getrel(data, i, &rel)) + continue; + apply_one_dl_reloc(dl_data, sym_data, target_le, + abs32_type, abs64_type, + rel.r_info, rel.r_offset, + 0, false); + } + } +} + +static void apply_debug_line_relocations(Elf *elf) +{ + Elf_Scn *scn =3D NULL; + Elf_Scn *debug_line_scn =3D NULL; + Elf_Scn *rela_debug_line_scn =3D NULL; + Elf_Scn *rel_debug_line_scn =3D NULL; + Elf_Scn *symtab_scn =3D NULL; + GElf_Shdr shdr; + GElf_Ehdr ehdr; + unsigned int abs32_type, abs64_type; + bool target_le; + size_t shstrndx; + Elf_Data *dl_data, *sym_data; + GElf_Shdr sym_shdr; + + if (gelf_getehdr(elf, &ehdr) =3D=3D NULL) + return; + + abs32_type =3D r_type_abs32(ehdr.e_machine); + abs64_type =3D r_type_abs64(ehdr.e_machine); + if (!abs32_type && !abs64_type) + return; + target_le =3D (ehdr.e_ident[EI_DATA] =3D=3D ELFDATA2LSB); + + if (elf_getshdrstrndx(elf, &shstrndx) !=3D 0) + return; + + /* Find the relevant sections */ + while ((scn =3D elf_nextscn(elf, scn)) !=3D NULL) { + const char *name; + + if (!gelf_getshdr(scn, &shdr)) + continue; + name =3D elf_strptr(elf, shstrndx, shdr.sh_name); + if (!name) + continue; + + if (!strcmp(name, ".debug_line")) + debug_line_scn =3D scn; + else if (!strcmp(name, ".rela.debug_line")) + rela_debug_line_scn =3D scn; + else if (!strcmp(name, ".rel.debug_line")) + rel_debug_line_scn =3D scn; + else if (shdr.sh_type =3D=3D SHT_SYMTAB) + symtab_scn =3D scn; + } + + if (!debug_line_scn || !symtab_scn) + return; + if (!rela_debug_line_scn && !rel_debug_line_scn) + return; + + dl_data =3D elf_getdata(debug_line_scn, NULL); + sym_data =3D elf_getdata(symtab_scn, NULL); + if (!dl_data || !sym_data) + return; + + if (!gelf_getshdr(symtab_scn, &sym_shdr)) + return; + + /* + * RELA (64-bit ELF and most 32-bit targets) carries explicit + * addends; REL (i386, arm32, ...) stores the addend in the + * relocated field itself. + */ + apply_dl_reloc_table(rela_debug_line_scn, true, dl_data, sym_data, + target_le, abs32_type, abs64_type); + apply_dl_reloc_table(rel_debug_line_scn, false, dl_data, sym_data, + target_le, abs32_type, abs64_type); +} + +/* + * Decide which covered_section a (biased) DWARF address belongs to. + * apply_debug_line_relocations() has already added the section's bias to + * each line-program PC, so [bias, bias+size) ranges are pairwise disjoint + * and a simple linear scan picks the right bucket. Returns the index + * within @sections, or @num_sections if @addr falls outside every + * present range (caller skips the entry). + */ +static unsigned int classify_address(struct covered_section *sections, + unsigned int num_sections, + unsigned long long addr, + unsigned long long *out_offset) +{ + for (unsigned int i =3D 0; i < num_sections; i++) { + if (!sections[i].present) + continue; + if (addr < sections[i].bias) + continue; + if (addr >=3D sections[i].bias + sections[i].size) + continue; + *out_offset =3D addr - sections[i].bias; + return i; + } + return num_sections; +} + +/* + * Classify the end of a line-program sequence. @addr is one past the last + * covered byte, so the section it belongs to is resolved from addr - 1. + */ +static void classify_seq_end(unsigned long long addr, + unsigned long long text_addr, + struct covered_section *sections, + unsigned int num_sections) +{ + unsigned long long raw; + + if (!addr) + return; + + if (module_mode) { + unsigned long long sec_off; + unsigned int sec_id; + + sec_id =3D classify_address(sections, num_sections, addr - 1, + &sec_off); + if (sec_id =3D=3D num_sections || sec_off + 1 > UINT_MAX) + return; + record_seq_end(sec_id, (unsigned int)sec_off + 1); + return; + } + + if (addr <=3D text_addr) + return; + if (text_end_addr && addr - 1 >=3D text_end_addr) + return; + + raw =3D addr - text_addr; + if (raw > UINT_MAX) + return; + + record_seq_end(0, (unsigned int)raw); +} + +static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr, + struct covered_section *sections, + unsigned int num_sections) { Dwarf_Off off =3D 0, next_off; size_t hdr_size; @@ -609,7 +1154,8 @@ static void process_dwarf(Dwarf *dwarf, unsigned long = long text_addr) Dwarf_Addr addr; const char *src; const char *rel; - unsigned int file_id, loffset; + unsigned int file_id, loffset, sec_id; + unsigned long long sec_off; bool endseq =3D false; int lineno; =20 @@ -629,7 +1175,8 @@ static void process_dwarf(Dwarf *dwarf, unsigned long = long text_addr) * from "coverage stopped here". */ if (dwarf_lineendsequence(line, &endseq) =3D=3D 0 && endseq) { - record_seq_end(addr, text_addr); + classify_seq_end(addr, text_addr, sections, + num_sections); continue; } =20 @@ -642,54 +1189,77 @@ static void process_dwarf(Dwarf *dwarf, unsigned lon= g long text_addr) if (!src) continue; =20 - if (addr < text_addr) - continue; - /* - * Skip addresses past _etext. Sections after .rodata - * shift when the real lineinfo replaces the empty stub - * during the multi-pass vmlinux link, so any address - * we'd capture there would be stale by the time the - * final kernel runs. - */ - if (text_end_addr && addr >=3D text_end_addr) - continue; - - { - unsigned long long raw_offset =3D addr - text_addr; + if (module_mode) { + /* + * In ET_REL .ko files .text/.init.text/.exit.text + * all share sh_addr =3D=3D 0; classify_address picks + * the right bucket from the explicit ranges we + * captured. + */ + sec_id =3D classify_address(sections, num_sections, + addr, &sec_off); + if (sec_id =3D=3D num_sections) { + skipped_uncovered++; + continue; + } + if (sec_off > UINT_MAX) { + skipped_overflow++; + continue; + } + loffset =3D (unsigned int)sec_off; + sections[sec_id].n_entries++; + } else { + unsigned long long raw_offset; =20 + if (addr < text_addr) + continue; + /* + * Skip addresses past _etext. Sections after + * .rodata shift when the real lineinfo replaces + * the empty stub during the multi-pass vmlinux + * link, so any address we'd capture there would + * be stale by the time the final kernel runs. + */ + if (text_end_addr && addr >=3D text_end_addr) + continue; + raw_offset =3D addr - text_addr; if (raw_offset > UINT_MAX) { skipped_overflow++; continue; } loffset =3D (unsigned int)raw_offset; + sec_id =3D 0; } =20 rel =3D make_relative(src, comp_dir); file_id =3D find_or_add_file(rel); =20 - add_entry(loffset, file_id, (unsigned int)lineno); + add_entry(loffset, sec_id, file_id, (unsigned int)lineno); } next: off =3D next_off; } } =20 -/* True if some line-program sequence ends in (@lo, @hi]. */ -static bool seq_end_between(unsigned int lo, unsigned int hi) +/* True if some sequence in @section ends in (@lo, @hi]. */ +static bool seq_end_between(unsigned int section, unsigned int lo, + unsigned int hi) { unsigned int low =3D 0, high =3D num_seq_ends; =20 - /* First index whose value exceeds @lo. */ + /* First index past (@section, @lo). */ while (low < high) { unsigned int mid =3D low + (high - low) / 2; =20 - if (seq_ends[mid] <=3D lo) + if (compare_sec_off(seq_ends[mid].section_id, + seq_ends[mid].offset, section, lo) <=3D 0) low =3D mid + 1; else high =3D mid; } =20 - return low < num_seq_ends && seq_ends[low] <=3D hi; + return low < num_seq_ends && seq_ends[low].section_id =3D=3D section && + seq_ends[low].offset <=3D hi; } =20 /* @@ -724,35 +1294,47 @@ static void synthesize_symbol_starts(void) if (!base_entries || !num_sym_starts) return; =20 - sort_unique(seq_ends, &num_seq_ends); + qsort(seq_ends, num_seq_ends, sizeof(*seq_ends), compare_seq_ends); =20 for (unsigned int i =3D 0; i < num_sym_starts; i++) { + unsigned int sec =3D sym_starts[i].section_id; unsigned int start =3D sym_starts[i].offset; unsigned int end =3D start + sym_starts[i].size; =20 + /* + * Both arrays are sorted by (section_id, offset), so one + * forward cursor tracks the last entry at or below the symbol. + */ while (cursor + 1 < base_entries && - entries[cursor + 1].offset <=3D start) + compare_sec_off(entries[cursor + 1].section_id, + entries[cursor + 1].offset, + sec, start) <=3D 0) cursor++; =20 + if (entries[cursor].section_id !=3D sec) + continue; /* no entries in this section yet */ if (entries[cursor].offset > start) continue; /* nothing covers this symbol */ if (entries[cursor].offset =3D=3D start) continue; /* already has its own entry */ - if (seq_end_between(entries[cursor].offset, start)) + if (seq_end_between(sec, entries[cursor].offset, start)) continue; /* coverage stopped before here */ =20 /* Overflow, or no row inside [start, end): not our code. */ if (end <=3D start) continue; if (cursor + 1 >=3D base_entries || + entries[cursor + 1].section_id !=3D sec || entries[cursor + 1].offset >=3D end) continue; =20 - add_entry(start, entries[cursor].file_id, entries[cursor].line); + add_entry(start, sec, entries[cursor].file_id, + entries[cursor].line); } } =20 -static void deduplicate(void) +static void deduplicate(struct covered_section *sections, + unsigned int num_sections) { unsigned int sym_cursor =3D 0; unsigned int i, j; @@ -760,14 +1342,25 @@ static void deduplicate(void) if (num_entries < 2) return; =20 - /* Sort by offset, then file_id, then line for stability */ + /* + * Sort by section_id, then offset, then file_id, line. This groups + * each section's entries contiguously so the per-section emit can + * iterate a simple range, and ensures the binary search invariant + * (offsets ascending) holds within each section. + */ qsort(entries, num_entries, sizeof(*entries), compare_entries); =20 synthesize_symbol_starts(); qsort(entries, num_entries, sizeof(*entries), compare_entries); =20 /* - * Remove duplicate entries: + * Remove duplicates. Reset on a section_id boundary: the same offset + * can legitimately appear in two different sections (they all start + * at sh_addr 0 in ET_REL), and the "same as previous kept entry" + * collapse is only meaningful inside one section's binary-search + * domain. + * + * Within a section: * - Same offset: keep first (deterministic from stable sort keys) * - Same file:line as previous kept entry: redundant for binary * search -- any address between them resolves to the earlier one @@ -780,14 +1373,26 @@ static void deduplicate(void) for (i =3D 1; i < num_entries; i++) { bool at_symbol_start; =20 + if (entries[i].section_id !=3D entries[j].section_id) { + j++; + if (j !=3D i) + entries[j] =3D entries[i]; + continue; + } if (entries[i].offset =3D=3D entries[j].offset) continue; =20 while (sym_cursor < num_sym_starts && - sym_starts[sym_cursor].offset < entries[i].offset) + compare_sec_off(sym_starts[sym_cursor].section_id, + sym_starts[sym_cursor].offset, + entries[i].section_id, + entries[i].offset) < 0) sym_cursor++; at_symbol_start =3D sym_cursor < num_sym_starts && - sym_starts[sym_cursor].offset =3D=3D entries[i].offset; + !compare_sec_off(sym_starts[sym_cursor].section_id, + sym_starts[sym_cursor].offset, + entries[i].section_id, + entries[i].offset); =20 if (!at_symbol_start && entries[i].file_id =3D=3D entries[j].file_id && @@ -799,6 +1404,14 @@ static void deduplicate(void) entries[j] =3D entries[i]; } num_entries =3D j + 1; + + /* Recompute per-section n_entries from the deduped array. */ + if (sections) { + for (unsigned int k =3D 0; k < num_sections; k++) + sections[k].n_entries =3D 0; + for (i =3D 0; i < num_entries; i++) + sections[entries[i].section_id].n_entries++; + } } =20 static void compute_file_offsets(void) @@ -896,6 +1509,206 @@ static void output_assembly(void) printf("\n"); } =20 +/* + * Emit one per-section table in the simple flat-array layout: + * + * mod_lineinfo_header + * addrs[count] (u32, sorted) + * file_ids[count] (u16) + 2-byte pad if count is odd + * lines[count] (u32) + * file_offsets[] (u32) + * filenames[] + * + * @suffix uniquifies labels so multiple tables can coexist in one blob. + * Caller has sorted entries[] so this section's entries occupy [first, + * first + count). + */ +static void emit_section_table(unsigned int first, unsigned int count, + const char *suffix) +{ + /* + * Align before defining the label, not after: the descriptor stores + * table_offset as .Lhdr - .Lroot, and every field offset inside the + * header is measured from .Lhdr too. Emitting the label first binds + * it to the pre-padding address, so the kernel would read the header + * out of the padding bytes. + */ + printf("\t.balign 4\n"); + printf(".Lhdr%s:\n", suffix); + printf("\t.long %u\t\t/* num_entries */\n", count); + printf("\t.long %u\t\t/* num_files */\n", num_files); + printf("\t.long .Lfilenames_end%s - .Lfilenames%s\n\n", suffix, suffix); + + /* addrs[] */ + for (unsigned int i =3D 0; i < count; i++) + printf("\t.long 0x%x\n", entries[first + i].offset); + + /* file_ids[] */ + for (unsigned int i =3D 0; i < count; i++) + printf("\t.short %u\n", entries[first + i].file_id); + if (count & 1) + printf("\t.short 0\t\t/* pad to align lines[] */\n"); + + /* lines[] */ + for (unsigned int i =3D 0; i < count; i++) + printf("\t.long %u\n", entries[first + i].line); + + /* file_offsets[] */ + printf("\t.balign 4\n"); + for (unsigned int i =3D 0; i < num_files; i++) + printf("\t.long %u\n", files[i]->str_offset); + + /* filenames[] */ + printf(".Lfilenames%s:\n", suffix); + for (unsigned int i =3D 0; i < num_files; i++) + print_escaped_asciz(files[i]->name); + printf(".Lfilenames_end%s:\n", suffix); +} + +/* + * Emit one mod_lineinfo_section descriptor. The "anchor" field is a + * relocation against the named ELF section symbol; the module loader + * resolves it on load to the runtime base of that section. + * + * On 64-bit ELF: 8-byte slot via .quad (R_*_64 reloc). + * On 32-bit ELF: 4-byte reloc via .long , plus 4 bytes of zero + * padding. The two halves are ordered to match target endianness so a + * naive u64 read on the kernel side recovers the relocated value. + */ +static void emit_section_descriptor(const char *section_name, + unsigned long long size, + const char *table_label, + const char *root_label) +{ + if (target_64bit) { + printf("\t.quad %s\t/* sections[].anchor (RELOC) */\n", + section_name); + } else if (target_le) { + printf("\t.long %s\t/* sections[].anchor low (RELOC) */\n", + section_name); + printf("\t.long 0\t\t/* sections[].anchor high pad */\n"); + } else { + printf("\t.long 0\t\t/* sections[].anchor high pad */\n"); + printf("\t.long %s\t/* sections[].anchor low (RELOC) */\n", + section_name); + } + printf("\t.long %llu\t/* sections[].size */\n", size); + printf("\t.long %s - %s\t/* sections[].table_offset */\n", + table_label, root_label); +} + +/* + * Emit one .mod_lineinfo / .init.mod_lineinfo blob. Walks all_sections[] + * picking only entries that (a) belong to the requested blob and (b) + * actually produced at least one DWARF line entry =E2=80=94 sections pres= ent in + * the .ko but without DWARF (e.g. compiler-generated stub thunks) are + * silently skipped. The caller-supplied entries[] is already sorted by + * section_id, so each section's entries are contiguous; we walk the + * master array in order to compute per-section starting indices. + */ +static void emit_blob(const char *output_section, + const char *blob_tag, + enum mod_lineinfo_blob blob) +{ + unsigned int active =3D 0; + unsigned int section_starts[ALL_SECTIONS]; + unsigned int cursor =3D 0; + + for (unsigned int i =3D 0; i < ALL_SECTIONS; i++) { + section_starts[i] =3D cursor; + cursor +=3D all_sections[i].n_entries; + if (all_sections[i].blob =3D=3D blob && all_sections[i].n_entries) + active++; + } + + if (!active) + return; + + printf("\t.section %s, \"a\"\n\n", output_section); + + printf("\t.balign 8\n"); + printf(".Lroot_%s:\n", blob_tag); + printf("\t.long %u\t\t/* num_sections */\n", active); + /* Pad to align the u64 anchor in sections[0] to 8 bytes. */ + printf("\t.balign 8\n"); + + { + unsigned int slot =3D 0; + for (unsigned int i =3D 0; i < ALL_SECTIONS; i++) { + char table_label[64]; + char root_label[64]; + + if (all_sections[i].blob !=3D blob) + continue; + if (!all_sections[i].n_entries) + continue; + snprintf(table_label, sizeof(table_label), + ".Lhdr_%s_%u", blob_tag, slot); + snprintf(root_label, sizeof(root_label), + ".Lroot_%s", blob_tag); + emit_section_descriptor(all_sections[i].name, + all_sections[i].size, + table_label, root_label); + slot++; + } + } + printf("\n"); + + { + unsigned int slot =3D 0; + + for (unsigned int i =3D 0; i < ALL_SECTIONS; i++) { + char suffix[64]; + + if (all_sections[i].blob !=3D blob) + continue; + if (!all_sections[i].n_entries) + continue; + snprintf(suffix, sizeof(suffix), "_%s_%u", + blob_tag, slot); + emit_section_table(section_starts[i], + all_sections[i].n_entries, + suffix); + slot++; + } + } + printf("\n"); +} + +/* + * Declare each text-like section we plan to reference as an empty + * SHF_EXECINSTR section in this object. Without these stanzas the + * assembler treats `.quad .exit.text` as an undefined external symbol; + * after ld -r the resulting GLOBAL UND `.exit.text` doesn't bind to the + * .ko's LOCAL SECTION symbol of the same name, leaving depmod with an + * unresolved-symbol warning and the loader unable to relocate the anchor. + * + * Declaring the section here gives lineinfo.o its own local SECTION + * symbol; ld -r merges sections by name so the local symbol simply + * relocates to offset 0 of the merged section (lineinfo.o is linked + * FIRST so its zero-byte contribution stays at the start). + */ +static void declare_empty_text_sections(void) +{ + for (unsigned int i =3D 0; i < ALL_SECTIONS; i++) { + if (!all_sections[i].present) + continue; + printf("\t.section %s, \"ax\"\n", all_sections[i].name); + } + printf("\n"); +} + +static void output_module_assembly(void) +{ + printf("/* SPDX-License-Identifier: GPL-2.0 */\n"); + printf("/*\n"); + printf(" * Automatically generated by scripts/gen_lineinfo --module\n"); + printf(" * Do not edit.\n"); + printf(" */\n\n"); + + declare_empty_text_sections(); +} + int main(int argc, char *argv[]) { const char *kbuild_verbose =3D getenv("KBUILD_VERBOSE"); @@ -914,8 +1727,15 @@ int main(int argc, char *argv[]) argc--; } =20 + if (argc >=3D 2 && !strcmp(argv[1], "--module")) { + module_mode =3D 1; + argv++; + argc--; + } + if (argc !=3D 2) { - fprintf(stderr, "Usage: %s [-v] \n", argv[0]); + fprintf(stderr, "Usage: %s [-v] [--module] \n", + argv[0]); return 1; } =20 @@ -926,13 +1746,42 @@ int main(int argc, char *argv[]) error("cannot open %s: %s", argv[1], strerror(errno)); =20 elf_version(EV_CURRENT); - elf =3D elf_begin(fd, ELF_C_READ_MMAP, NULL); + /* + * Module mode patches line-program addresses in the in-memory ELF + * data and must never write them back. A private copy-on-write + * mapping gives exactly that, so the .ko can stay open read-only. + */ + elf =3D elf_begin(fd, module_mode ? ELF_C_READ_MMAP_PRIVATE : + ELF_C_READ_MMAP, NULL); if (!elf) error("elf_begin failed: %s", elf_errmsg(elf_errno())); =20 - text_addr =3D find_text_addr(elf); - text_end_addr =3D find_text_end_addr(elf); - collect_symbol_starts(elf, text_addr); + { + GElf_Ehdr ehdr; + + if (gelf_getehdr(elf, &ehdr) =3D=3D NULL) + error("gelf_getehdr failed"); + target_64bit =3D (ehdr.e_ident[EI_CLASS] =3D=3D ELFCLASS64); + target_le =3D (ehdr.e_ident[EI_DATA] =3D=3D ELFDATA2LSB); + } + + if (module_mode) { + /* + * .ko files are ET_REL after ld -r. Resolve covered text + * sections FIRST so apply_debug_line_relocations() can use + * the assigned biases when patching line-program addresses; + * libdw does NOT apply relocations for ET_REL files, so we + * also handle DW_FORM_line_strp refs into .debug_line_str. + */ + resolve_covered_sections(elf, all_sections, ALL_SECTIONS); + apply_debug_line_relocations(elf); + text_addr =3D 0; /* unused in module mode */ + } else { + text_addr =3D find_text_addr(elf); + text_end_addr =3D find_text_end_addr(elf); + } + + collect_symbol_starts(elf, text_addr, all_sections, ALL_SECTIONS); =20 dwarf =3D dwarf_begin_elf(elf, DWARF_C_READ, NULL); if (!dwarf) @@ -940,18 +1789,59 @@ int main(int argc, char *argv[]) LINEINFO_PREFIX "error: is %s built with CONFIG_DEBUG_INFO?", dwarf_errmsg(dwarf_errno()), argv[1]); =20 - process_dwarf(dwarf, text_addr); + if (module_mode) { + unsigned int persistent_total, init_total; =20 - if (skipped_overflow) - warn("%u entries skipped (offset > 4 GiB from _text)", - skipped_overflow); + output_module_assembly(); /* file header only */ =20 - deduplicate(); - compute_file_offsets(); + /* + * Single DWARF pass classifies every line entry into its + * covering section (or skips it). Each entry is tagged with + * the master-array section_id so per-blob emit can filter. + */ + process_dwarf(dwarf, 0, all_sections, ALL_SECTIONS); + deduplicate(all_sections, ALL_SECTIONS); + compute_file_offsets(); + + emit_blob(".mod_lineinfo", "p", BLOB_PERSISTENT); + emit_blob(".init.mod_lineinfo", "i", BLOB_INIT); + + persistent_total =3D 0; + init_total =3D 0; + for (unsigned int i =3D 0; i < ALL_SECTIONS; i++) { + if (all_sections[i].blob =3D=3D BLOB_PERSISTENT) + persistent_total +=3D all_sections[i].n_entries; + else if (all_sections[i].blob =3D=3D BLOB_INIT) + init_total +=3D all_sections[i].n_entries; + } + verbose_msg("persistent %u entries, init %u entries, %u files", + persistent_total, init_total, num_files); =20 - verbose_msg("%u entries, %u files", num_entries, num_files); + /* + * Expected for every module: .static_call.text and friends + * are deliberately not covered. + */ + if (skipped_uncovered) + verbose_msg("%llu entries dropped (outside covered text sections)", + skipped_uncovered); + + if (skipped_overflow) + warn("%u entries skipped (offset > 4 GiB)", + skipped_overflow); + } else { + process_dwarf(dwarf, text_addr, NULL, 0); + + if (skipped_overflow) + warn("%u entries skipped (offset > 4 GiB from _text)", + skipped_overflow); =20 - output_assembly(); + deduplicate(NULL, 0); + compute_file_offsets(); + + verbose_msg("%u entries, %u files", num_entries, num_files); + + output_assembly(); + } =20 dwarf_end(dwarf); elf_end(elf); @@ -961,9 +1851,11 @@ int main(int argc, char *argv[]) free(entries); free(sym_starts); free(seq_ends); + free(extra_sections); for (unsigned int i =3D 0; i < num_files; i++) free(files[i]); free(files); - + for (unsigned int i =3D 0; i < num_path_roots; i++) + free(path_roots[i].path); return 0; } --=20 2.53.0 From nobody Fri Oct 2 12:19:52 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AECF13806CA; Sat, 1 Aug 2026 14:32:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785594757; cv=none; b=CD74JyJKBck/F7VoMk0u8aQ1SkP6djzQmVL1zDTqx4T3q1LnbkI86QpO/gxzXNMmEJq5t3KM/YbSoWikFg5IduUQDTVCDWsZuFLYPqS7ZP+npgynDOIaSC3NbjeeFmOtcmYsg1wnZCi0tdkCTOgn1dzIjHCxAXwpxRtDApRwU4w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785594757; c=relaxed/simple; bh=47f9PxYAsh+G6B7pW7RP1sKV9J2A4IvOBLXRK3jDdNw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=aatHk5nA1Z78Y7Dr/Hjzpm+dIuPd1zejW8D3wa1EQ4rIhXhilQU9MjODd23ry2OnXtoKWPFltz6ctEM3TklsPS18Dt+f47elY2LzVyfqtcQB6rxzXtAzImcrlUxJ9hddadajkyUOLcgfTNfmp8T0tFlTIdcjBc6Bh8rirvl38hQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=M9O18mjg; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="M9O18mjg" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7A6171F00AC4; Sat, 1 Aug 2026 14:32:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785594754; bh=5Dwwy2nkEY+t4ics1PDuq30bEmrNt7XU4z54R7eAyP4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=M9O18mjgBrd1LnIjGmSydSAEXS1XkHdwD/pSexcw8tGkB7zULaX9mJ4DHi9Fu4pKl HZpF8ykCYYzoQ+4wArugFsesveRLrHBA1roBGKl6O11WFdIRPf/lCrteYYmp6laqBA V3dLtClXMviIYoPj928Dh7aykTcstli5Jy5dNxS1ut2KVqaxrHsZ1Bi42L0yZliMAK qe+pnrCD8dOPq3zAPHTPC2HCEraRA1TOquuwBoAAAzIw9iZuqwlQ7qwM3POp/BddhL dT8zkud8aTF7/lgIDqgbwL5D+mC3doayoucEJs45iPTvt/9JcwuUwYrH9YYZH7C4co lx20uZ/WWu7DA== From: Sasha Levin To: Andrew Morton , Masahiro Yamada , Luis Chamberlain , Linus Torvalds , Richard Weinberger , Juergen Gross , Geert Uytterhoeven , James Bottomley Cc: Sasha Levin , Jonathan Corbet , Nathan Chancellor , Nicolas Schier , Petr Pavlu , Daniel Gomez , Greg KH , Petr Mladek , Steven Rostedt , Kees Cook , Peter Zijlstra , Thorsten Leemhuis , Vlastimil Babka , Helge Deller , Randy Dunlap , Laurent Pinchart , Vivian Wang , Zhen Lei , Sami Tolvanen , linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-doc@vger.kernel.org Subject: [PATCH v8 3/4] kallsyms: delta-compress lineinfo tables for ~2.7x size reduction Date: Sat, 1 Aug 2026 10:32:17 -0400 Message-ID: <20260801143219.3975824-4-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260801143219.3975824-1-sashal@kernel.org> References: <20260801143219.3975824-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Replace the flat uncompressed parallel arrays (lineinfo_addrs[], lineinfo_file_ids[], lineinfo_lines[]) with a block-indexed, delta-encoded, ULEB128 varint compressed format. The sorted address array has small deltas between consecutive entries (typically 1-50 bytes), file IDs have high locality (delta often 0, same file), and line numbers change slowly. Delta-encoding followed by ULEB128 varint compression shrinks most values from 4 bytes to 1. Entries are grouped into blocks of 64. A small uncompressed block index (first addr + byte offset per block) enables O(log(N/64)) binary search, followed by sequential decode of at most 64 varints within the matching block. All decode state lives on the stack -- zero allocations, still safe for NMI/panic context. Measured on x86_64_defconfig + CONFIG_DEBUG_INFO (same recipe as patch 1/4; 1,795,278 entries, 4,233 source files, 28,052 blocks): Before (flat arrays): lineinfo_addrs[] 7,179,800 bytes (u32 x 1.79M) lineinfo_file_ids[] 3,589,900 bytes (u16 x 1.79M) lineinfo_lines[] 7,179,800 bytes (u32 x 1.79M) Total: 17,949,500 bytes (17.1 MiB, 10.00 bytes/entry) After (block-indexed delta + ULEB128): lineinfo_block_addrs[] 112,208 bytes (110 KiB) lineinfo_block_offsets[] 112,208 bytes (110 KiB) lineinfo_data[] 6,529,411 bytes (6.2 MiB) Total: 6,753,827 bytes (6.4 MiB, 3.76 bytes/entry) Savings: 10.7 MiB (2.66x reduction) file_offsets[] and filenames[] are unchanged by this patch and are excluded from both totals above. Whole-image effect, measured the same way as patch 1/4: vmlinux (stripped), no lineinfo: 51.1 MiB vmlinux (stripped), before this fix: 69.1 MiB (+18.0 MiB / +35.3%) vmlinux (stripped), after this fix: 59.1 MiB ( +8.0 MiB / +15.7%) Booted in QEMU and verified with SysRq-l that annotations still work: default_idle+0x9/0x10 (arch/x86/kernel/process.c:767) default_idle_call+0x6c/0xb0 (kernel/sched/idle.c:122) do_idle+0x335/0x490 (kernel/sched/idle.c:191) cpu_startup_entry+0x4e/0x60 (kernel/sched/idle.c:429) rest_init+0x1aa/0x1b0 (init/main.c:760) Suggested-by: Juergen Gross Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Sasha Levin --- include/linux/mod_lineinfo.h | 250 ++++++++++++++++++++++++++++++----- init/Kconfig | 8 +- kernel/kallsyms.c | 53 +++----- kernel/kallsyms_internal.h | 8 +- kernel/module/kallsyms.c | 110 +++++++-------- scripts/empty_lineinfo.S | 20 ++- scripts/gen_lineinfo.c | 185 ++++++++++++++++++-------- 7 files changed, 437 insertions(+), 197 deletions(-) diff --git a/include/linux/mod_lineinfo.h b/include/linux/mod_lineinfo.h index 75d0bfba6ce5..47277f9d7e4f 100644 --- a/include/linux/mod_lineinfo.h +++ b/include/linux/mod_lineinfo.h @@ -3,9 +3,9 @@ * mod_lineinfo.h - Binary format for per-module source line information * * This header defines the layout of the .mod_lineinfo and - * .init.mod_lineinfo sections embedded in loadable kernel modules. It - * is dual-use: included from both the kernel and the userspace - * gen_lineinfo tool. + * .init.mod_lineinfo sections embedded in loadable kernel modules. It is + * dual-use: included from both the kernel and the userspace gen_lineinfo + * tool. * * Top-level layout (all values in target-native endianness): * @@ -20,16 +20,27 @@ * If the relocation fails to resolve (e.g. unknown reloc type), .anchor * stays zero and lookups silently degrade to "no annotation". * - * Each per-section sub-table is laid out as a stand-alone - * mod_lineinfo_header followed by parallel arrays: + * Each per-section sub-table is laid out exactly as a stand-alone + * mod_lineinfo_header followed by its arrays: * - * struct mod_lineinfo_header (16 bytes) - * u32 addrs[num_entries] -- offsets from this section's base, s= orted - * u16 file_ids[num_entries] -- parallel to addrs - * <2-byte pad if num_entries is odd> - * u32 lines[num_entries] -- parallel to addrs + * struct mod_lineinfo_header + * u32 block_addrs[num_blocks] -- first addr per block, for binary se= arch + * u32 block_offsets[num_blocks] -- byte offset into compressed data st= ream + * u8 data[data_size] -- LEB128 delta-compressed entries * u32 file_offsets[num_files] -- byte offset into filenames[] * char filenames[filenames_size] -- concatenated NUL-terminated strings + * + * Each sub-array is located by an explicit (offset, size) pair in the + * header, similar to a flattened devicetree. All offsets in the per-sect= ion + * header are relative to that header itself, so a sub-table is fully + * self-describing. + * + * Compressed stream format (per block of LINEINFO_BLOCK_ENTRIES entries): + * Entry 0: file_id (ULEB128), line (ULEB128) + * addr is in block_addrs[] + * Entry 1..N: addr_delta (ULEB128), + * file_id_delta (SLEB128), + * line_delta (SLEB128) */ #ifndef _LINUX_MOD_LINEINFO_H #define _LINUX_MOD_LINEINFO_H @@ -44,6 +55,7 @@ #include typedef uint32_t u32; typedef uint16_t u16; +typedef uint8_t u8; typedef uint64_t u64; #ifndef __aligned #define __aligned(x) __attribute__((__aligned__(x))) @@ -53,6 +65,8 @@ typedef uint64_t u64; #endif #endif =20 +#define LINEINFO_BLOCK_ENTRIES 64 + /* * Per-section descriptor. One entry per ELF text section covered by the * blob (.text, .exit.text, .init.text, ...). @@ -84,39 +98,215 @@ static_assert(sizeof(struct mod_lineinfo_section) =3D= =3D 16, =20 struct mod_lineinfo_header { u32 num_entries; + u32 num_blocks; u32 num_files; - u32 filenames_size; /* total bytes of concatenated filenames */ + u32 blocks_offset; /* offset to block_addrs[] from this header */ + u32 blocks_size; /* bytes: num_blocks * 2 * sizeof(u32) */ + u32 data_offset; /* offset to compressed stream */ + u32 data_size; /* bytes of compressed data */ + u32 files_offset; /* offset to file_offsets[] */ + u32 files_size; /* bytes: num_files * sizeof(u32) */ + u32 filenames_offset; + u32 filenames_size; }; =20 -/* Offset helpers: compute byte offset from the per-section header to each= array. */ +/* + * Descriptor for a lineinfo table, used by the shared lookup function. + * Callers populate this from either linker globals (vmlinux) or a + * validated mod_lineinfo_header (modules). + */ +struct lineinfo_table { + const u32 *blk_addrs; + const u32 *blk_offsets; + const u8 *data; + u32 data_size; + const u32 *file_offsets; + const char *filenames; + u32 num_entries; + u32 num_blocks; + u32 num_files; + u32 filenames_size; +}; =20 -static inline u32 mod_lineinfo_addrs_off(void) +/* + * Read a ULEB128 varint from a byte stream. + * Returns the decoded value and advances *pos past the encoded bytes. + * If *pos would exceed 'end', returns 0 and sets *pos =3D end (safe for + * NMI/panic context: no crash, just a missed annotation). + */ +static inline u32 lineinfo_read_uleb128(const u8 *data, u32 *pos, u32 end) { - return sizeof(struct mod_lineinfo_header); -} + u32 result =3D 0; + unsigned int shift =3D 0; =20 -static inline u32 mod_lineinfo_file_ids_off(u32 num_entries) -{ - return mod_lineinfo_addrs_off() + num_entries * sizeof(u32); + while (*pos < end) { + u8 byte =3D data[*pos]; + (*pos)++; + result |=3D (u32)(byte & 0x7f) << shift; + if (!(byte & 0x80)) + return result; + shift +=3D 7; + if (shift >=3D 32) { + /* Malformed: skip remaining continuation bytes */ + while (*pos < end && (data[*pos] & 0x80)) + (*pos)++; + if (*pos < end) + (*pos)++; + return result; + } + } + return result; } =20 -static inline u32 mod_lineinfo_lines_off(u32 num_entries) +/* Read an SLEB128 varint. Same safety guarantees as above. */ +static inline int32_t lineinfo_read_sleb128(const u8 *data, u32 *pos, u32 = end) { - /* u16 file_ids[] may need 2-byte padding to align lines[] to 4 bytes */ - u32 off =3D mod_lineinfo_file_ids_off(num_entries) + - num_entries * sizeof(u16); - return (off + 3) & ~3u; -} + int32_t result =3D 0; + unsigned int shift =3D 0; + u8 byte =3D 0; =20 -static inline u32 mod_lineinfo_file_offsets_off(u32 num_entries) -{ - return mod_lineinfo_lines_off(num_entries) + num_entries * sizeof(u32); + while (*pos < end) { + byte =3D data[*pos]; + (*pos)++; + result |=3D (int32_t)((u32)(byte & 0x7f) << shift); + shift +=3D 7; + if (!(byte & 0x80)) + break; + if (shift >=3D 32) { + while (*pos < end && (data[*pos] & 0x80)) + (*pos)++; + if (*pos < end) + (*pos)++; + return result; + } + } + + /* Sign-extend if the high bit of the last byte was set */ + if (shift < 32 && (byte & 0x40)) + result |=3D -(1 << shift); + + return result; } =20 -static inline u32 mod_lineinfo_filenames_off(u32 num_entries, u32 num_file= s) +/* + * Search a lineinfo table for the source file and line corresponding to a + * given offset (from _text for vmlinux, from the covered section's base f= or + * modules). @min_offset is the containing symbol's start in the same off= set + * space: entries below it belong to a preceding symbol and are rejected. + * + * Safe for NMI and panic context: no locks, no allocations, all state on = stack. + * Returns true and sets @file and @line on success; false on any failure. + */ +static inline bool lineinfo_search(const struct lineinfo_table *tbl, + unsigned int offset, + unsigned int min_offset, + const char **file, unsigned int *line) { - return mod_lineinfo_file_offsets_off(num_entries) + - num_files * sizeof(u32); + unsigned int low, high, mid, block; + unsigned int cur_addr, cur_file_id, cur_line; + unsigned int best_addr =3D 0, best_file_id =3D 0, best_line =3D 0; + unsigned int block_entries, data_end; + bool found =3D false; + u32 pos; + + if (!tbl->num_entries || !tbl->num_blocks) + return false; + + /* Binary search on blk_addrs[] to find the right block */ + low =3D 0; + high =3D tbl->num_blocks; + while (low < high) { + mid =3D low + (high - low) / 2; + if (tbl->blk_addrs[mid] <=3D offset) + low =3D mid + 1; + else + high =3D mid; + } + + if (low =3D=3D 0) + return false; + block =3D low - 1; + + /* How many entries in this block? */ + block_entries =3D LINEINFO_BLOCK_ENTRIES; + if (block =3D=3D tbl->num_blocks - 1) { + unsigned int remaining =3D tbl->num_entries - + block * LINEINFO_BLOCK_ENTRIES; + + if (remaining < block_entries) + block_entries =3D remaining; + } + + /* Determine end of this block's data in the compressed stream */ + if (block + 1 < tbl->num_blocks) + data_end =3D tbl->blk_offsets[block + 1]; + else + data_end =3D tbl->data_size; + + /* Clamp data_end to actual data size */ + if (data_end > tbl->data_size) + data_end =3D tbl->data_size; + + /* Decode entry 0: addr from blk_addrs, file_id and line from stream */ + pos =3D tbl->blk_offsets[block]; + if (pos >=3D data_end) + return false; + + cur_addr =3D tbl->blk_addrs[block]; + cur_file_id =3D lineinfo_read_uleb128(tbl->data, &pos, data_end); + cur_line =3D lineinfo_read_uleb128(tbl->data, &pos, data_end); + + /* Check entry 0 */ + if (cur_addr <=3D offset) { + best_addr =3D cur_addr; + best_file_id =3D cur_file_id; + best_line =3D cur_line; + found =3D true; + } + + /* Decode entries 1..N */ + for (unsigned int i =3D 1; i < block_entries; i++) { + unsigned int addr_delta; + int32_t file_delta, line_delta; + + addr_delta =3D lineinfo_read_uleb128(tbl->data, &pos, data_end); + file_delta =3D lineinfo_read_sleb128(tbl->data, &pos, data_end); + line_delta =3D lineinfo_read_sleb128(tbl->data, &pos, data_end); + + cur_addr +=3D addr_delta; + cur_file_id =3D (unsigned int)((int32_t)cur_file_id + file_delta); + cur_line =3D (unsigned int)((int32_t)cur_line + line_delta); + + if (cur_addr > offset) + break; + + best_addr =3D cur_addr; + best_file_id =3D cur_file_id; + best_line =3D cur_line; + found =3D true; + } + + if (!found) + return false; + + /* + * The best entry is the closest one at or below @offset; reject it + * if it lies below the resolved symbol's start, so a symbol without + * line entries of its own does not inherit the preceding symbol's + * annotation. + */ + if (best_addr < min_offset) + return false; + + if (best_file_id >=3D tbl->num_files) + return false; + + if (tbl->file_offsets[best_file_id] >=3D tbl->filenames_size) + return false; + + *file =3D &tbl->filenames[tbl->file_offsets[best_file_id]]; + *line =3D best_line; + return true; } =20 #endif /* _LINUX_MOD_LINEINFO_H */ diff --git a/init/Kconfig b/init/Kconfig index 8c5cf00cb3ae..041084efabf7 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2102,8 +2102,9 @@ config KALLSYMS_LINEINFO anon_vma_clone+0x2ed/0xcf0 (mm/rmap.c:412) =20 This requires libelf and libdw (from elfutils) on the build host. - Costs 10 bytes per DWARF line-table entry; for x86_64_defconfig - with CONFIG_DEBUG_INFO that is about 18MB. + Costs about 3.8 bytes per DWARF line-table entry after delta + compression; for x86_64_defconfig with CONFIG_DEBUG_INFO that is + about 8MB. =20 If unsure, say N. =20 @@ -2116,7 +2117,8 @@ config KALLSYMS_LINEINFO_MODULES so stack traces from module code include (file.c:123) annotations. =20 This requires libelf and libdw (from elfutils) on the build host. - Costs 10 bytes per DWARF line entry in each .ko. + Costs about 3.8 bytes per DWARF line entry in each .ko after + delta compression. =20 If unsure, say N. =20 diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 07cfc30c7ad6..b5eff2ba8e5d 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -467,13 +467,17 @@ static int append_buildid(char *buffer, const char = *modname, =20 #endif /* CONFIG_STACKTRACE_BUILD_ID */ =20 +#include + bool kallsyms_lookup_lineinfo(unsigned long addr, unsigned long sym_start, const char **file, unsigned int *line) { unsigned long raw_offset, raw_min; - unsigned int offset, min_offset =3D 0, low, high, mid, file_id; + unsigned int offset, min_offset =3D 0; + struct lineinfo_table tbl; =20 - if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) || !lineinfo_num_entries) + if (!IS_ENABLED(CONFIG_KALLSYMS_LINEINFO) || + !lineinfo_num_entries || !lineinfo_num_blocks) return false; =20 /* Compute offset from _text */ @@ -491,8 +495,8 @@ bool kallsyms_lookup_lineinfo(unsigned long addr, unsig= ned long sym_start, return false; =20 /* - * The search below returns the closest entry at or below @offset, so - * a symbol without line entries of its own (assembly without debug + * The search returns the closest entry at or below the offset, so a + * symbol without line entries of its own (assembly without debug * info, or anything past the _etext cap like .init.text) would * inherit the last entry of whatever precedes it. Bound the result * to entries at or above the resolved symbol's start. @@ -504,35 +508,18 @@ bool kallsyms_lookup_lineinfo(unsigned long addr, uns= igned long sym_start, min_offset =3D raw_min; } =20 - /* Binary search for largest entry <=3D offset */ - low =3D 0; - high =3D lineinfo_num_entries; - while (low < high) { - mid =3D low + (high - low) / 2; - if (lineinfo_addrs[mid] <=3D offset) - low =3D mid + 1; - else - high =3D mid; - } - - if (low =3D=3D 0) - return false; - low--; - - if (lineinfo_addrs[low] < min_offset) - return false; - - file_id =3D lineinfo_file_ids[low]; - *line =3D lineinfo_lines[low]; - - if (file_id >=3D lineinfo_num_files) - return false; - - if (lineinfo_file_offsets[file_id] >=3D lineinfo_filenames_size) - return false; - - *file =3D &lineinfo_filenames[lineinfo_file_offsets[file_id]]; - return true; + tbl.blk_addrs =3D lineinfo_block_addrs; + tbl.blk_offsets =3D lineinfo_block_offsets; + tbl.data =3D lineinfo_data; + tbl.data_size =3D lineinfo_data_size; + tbl.file_offsets =3D lineinfo_file_offsets; + tbl.filenames =3D lineinfo_filenames; + tbl.num_entries =3D lineinfo_num_entries; + tbl.num_blocks =3D lineinfo_num_blocks; + tbl.num_files =3D lineinfo_num_files; + tbl.filenames_size =3D lineinfo_filenames_size; + + return lineinfo_search(&tbl, offset, min_offset, file, line); } =20 /* Look up a kernel symbol and return it in a text buffer. */ diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h index d7374ce444d8..ffe4c658067e 100644 --- a/kernel/kallsyms_internal.h +++ b/kernel/kallsyms_internal.h @@ -16,10 +16,12 @@ extern const unsigned int kallsyms_markers[]; extern const u8 kallsyms_seqs_of_names[]; =20 extern const u32 lineinfo_num_entries; -extern const u32 lineinfo_addrs[]; -extern const u16 lineinfo_file_ids[]; -extern const u32 lineinfo_lines[]; extern const u32 lineinfo_num_files; +extern const u32 lineinfo_num_blocks; +extern const u32 lineinfo_block_addrs[]; +extern const u32 lineinfo_block_offsets[]; +extern const u32 lineinfo_data_size; +extern const u8 lineinfo_data[]; extern const u32 lineinfo_file_offsets[]; extern const u32 lineinfo_filenames_size; extern const char lineinfo_filenames[]; diff --git a/kernel/module/kallsyms.c b/kernel/module/kallsyms.c index 6615a14efba2..d0c0b6f39945 100644 --- a/kernel/module/kallsyms.c +++ b/kernel/module/kallsyms.c @@ -498,9 +498,9 @@ int module_kallsyms_on_each_symbol(const char *modname, #include =20 /* - * Search one per-section sub-table for @section_offset using flat parallel - * arrays. @hdr is the per-section header at byte offset @hdr_offset with= in - * @blob. Returns true on hit and populates @file / @line. + * Search one per-section sub-table for @section_offset. + * @hdr is the per-section header at byte offset @hdr_offset within @blob. + * Returns true on hit and populates @file / @line. */ static bool module_lookup_lineinfo_section(const void *blob, u32 blob_size, u32 hdr_offset, @@ -510,13 +510,8 @@ static bool module_lookup_lineinfo_section(const void = *blob, u32 blob_size, unsigned int *line) { const struct mod_lineinfo_header *hdr; - const u8 *base; - const u32 *addrs, *lines, *file_offsets; - const u16 *file_ids; - const char *filenames; - u32 num_entries, num_files, filenames_size; - unsigned int low, high, mid; - u16 file_id; + struct lineinfo_table tbl; + const void *base; =20 if (hdr_offset > blob_size || blob_size - hdr_offset < sizeof(*hdr)) @@ -532,80 +527,63 @@ static bool module_lookup_lineinfo_section(const void= *blob, u32 blob_size, return false; =20 base =3D (const u8 *)blob + hdr_offset; - hdr =3D (const struct mod_lineinfo_header *)base; - num_entries =3D hdr->num_entries; - num_files =3D hdr->num_files; - filenames_size =3D hdr->filenames_size; + hdr =3D base; =20 - if (num_entries =3D=3D 0) + if (hdr->num_entries =3D=3D 0 || hdr->num_blocks =3D=3D 0) return false; =20 - /* - * Validate counts before multiplying =E2=80=94 sizing arithmetic could - * otherwise overflow on 32-bit with a malformed blob. Each entry - * contributes one u32 (addrs), one u16 (file_ids), and one u32 - * (lines); each file contributes one u32 (file_offsets). - */ + /* Validate each sub-array fits within the remaining blob bytes */ { u32 avail =3D blob_size - hdr_offset; - u32 needed =3D mod_lineinfo_filenames_off(num_entries, num_files); =20 - if (num_entries > U32_MAX / sizeof(u32)) + if (hdr->blocks_offset > avail || + hdr->blocks_size > avail - hdr->blocks_offset) + return false; + if (hdr->data_offset > avail || + hdr->data_size > avail - hdr->data_offset) return false; - if (num_files > U32_MAX / sizeof(u32)) + if (hdr->files_offset > avail || + hdr->files_size > avail - hdr->files_offset) return false; - if (needed > avail || filenames_size > avail - needed) + if (hdr->filenames_offset > avail || + hdr->filenames_size > avail - hdr->filenames_offset) return false; } =20 /* - * Filenames are read as NUL-terminated C strings. Require the blob - * to end in NUL so a malformed file_offsets entry can never lead the - * later "%s" consumer past the end of the section. + * Validate counts before multiplying by element size =E2=80=94 multiplic= ation + * could otherwise overflow on 32-bit builds with a malformed blob. + * num_blocks contributes (addr,offset) u32 pairs; num_files contributes + * one u32 each. */ - if (filenames_size =3D=3D 0 || - base[mod_lineinfo_filenames_off(num_entries, num_files) + - filenames_size - 1] !=3D 0) + if (hdr->num_blocks > hdr->blocks_size / (2 * sizeof(u32))) return false; - - addrs =3D (const u32 *)(base + mod_lineinfo_addrs_off()); - file_ids =3D (const u16 *)(base + mod_lineinfo_file_ids_off(num_entries)); - lines =3D (const u32 *)(base + mod_lineinfo_lines_off(num_entries)); - file_offsets =3D (const u32 *)(base + mod_lineinfo_file_offsets_off(num_e= ntries)); - filenames =3D (const char *)(base + mod_lineinfo_filenames_off(num_entrie= s, num_files)); - - /* Binary search for largest entry <=3D section_offset. */ - low =3D 0; - high =3D num_entries; - while (low < high) { - mid =3D low + (high - low) / 2; - if (addrs[mid] <=3D section_offset) - low =3D mid + 1; - else - high =3D mid; - } - - if (low =3D=3D 0) + if (hdr->num_files > hdr->files_size / sizeof(u32)) return false; - low--; =20 /* - * Reject entries below the resolved symbol's start so a symbol - * without line entries of its own does not inherit the preceding - * symbol's annotation. + * Filenames are read as NUL-terminated C strings. Require the blob + * to end in NUL so a malformed file_offsets entry can never lead the + * later "%s" consumer past the end of the section. */ - if (addrs[low] < min_offset) + if (hdr->filenames_size =3D=3D 0 || + ((const u8 *)base)[hdr->filenames_offset + + hdr->filenames_size - 1] !=3D 0) return false; =20 - file_id =3D file_ids[low]; - if (file_id >=3D num_files) - return false; - if (file_offsets[file_id] >=3D filenames_size) - return false; - - *file =3D &filenames[file_offsets[file_id]]; - *line =3D lines[low]; - return true; + tbl.blk_addrs =3D base + hdr->blocks_offset; + tbl.blk_offsets =3D base + hdr->blocks_offset + + hdr->num_blocks * sizeof(u32); + tbl.data =3D base + hdr->data_offset; + tbl.data_size =3D hdr->data_size; + tbl.file_offsets =3D base + hdr->files_offset; + tbl.filenames =3D base + hdr->filenames_offset; + tbl.num_entries =3D hdr->num_entries; + tbl.num_blocks =3D hdr->num_blocks; + tbl.num_files =3D hdr->num_files; + tbl.filenames_size =3D hdr->filenames_size; + + return lineinfo_search(&tbl, section_offset, min_offset, file, line); } =20 /* @@ -632,6 +610,7 @@ static bool module_lookup_lineinfo_blob(const void *blo= b, u32 blob_size, if (root->num_sections =3D=3D 0) return false; =20 + /* Validate sections[] array fits within the blob */ if (root->num_sections > U32_MAX / sizeof(struct mod_lineinfo_section)) return false; sections_end =3D sizeof(*root) + @@ -667,6 +646,9 @@ static bool module_lookup_lineinfo_blob(const void *blo= b, u32 blob_size, =20 /* * Look up source file:line for an address within a loaded module. + * Uses the .mod_lineinfo / .init.mod_lineinfo sections embedded in the .ko + * at build time. Each section contains one or more per-section sub-tables + * keyed by an ELF-relocation-resolved anchor. * * Safe in NMI/panic context: no locks, no allocations. * Caller must hold RCU read lock (or be in a context where the module diff --git a/scripts/empty_lineinfo.S b/scripts/empty_lineinfo.S index e058c4113712..edd5b1092f05 100644 --- a/scripts/empty_lineinfo.S +++ b/scripts/empty_lineinfo.S @@ -14,12 +14,20 @@ lineinfo_num_entries: .balign 4 lineinfo_num_files: .long 0 - .globl lineinfo_addrs -lineinfo_addrs: - .globl lineinfo_file_ids -lineinfo_file_ids: - .globl lineinfo_lines -lineinfo_lines: + .globl lineinfo_num_blocks + .balign 4 +lineinfo_num_blocks: + .long 0 + .globl lineinfo_block_addrs +lineinfo_block_addrs: + .globl lineinfo_block_offsets +lineinfo_block_offsets: + .globl lineinfo_data_size + .balign 4 +lineinfo_data_size: + .long 0 + .globl lineinfo_data +lineinfo_data: .globl lineinfo_file_offsets lineinfo_file_offsets: .globl lineinfo_filenames_size diff --git a/scripts/gen_lineinfo.c b/scripts/gen_lineinfo.c index 725a8dfbf011..d6300d947c06 100644 --- a/scripts/gen_lineinfo.c +++ b/scripts/gen_lineinfo.c @@ -1414,6 +1414,45 @@ static void deduplicate(struct covered_section *sect= ions, } } =20 +/* + * Emit the LEB128 delta-compressed data stream for one block. + * @base is the absolute index of the first entry, @count is the number of + * entries in this block (<=3D LINEINFO_BLOCK_ENTRIES). Used by both vmli= nux + * mode (one section, full entries[]) and module mode (per-section ranges). + */ +static void emit_block_data_range(unsigned int base, unsigned int count) +{ + if (!count) + return; + + /* Entry 0: file_id, line (both unsigned) */ + printf("\t.uleb128 %u\n", entries[base].file_id); + printf("\t.uleb128 %u\n", entries[base].line); + + /* Entries 1..N: addr_delta (unsigned), file/line deltas (signed) */ + for (unsigned int i =3D 1; i < count; i++) { + unsigned int idx =3D base + i; + + printf("\t.uleb128 %u\n", + entries[idx].offset - entries[idx - 1].offset); + printf("\t.sleb128 %d\n", + (int)entries[idx].file_id - (int)entries[idx - 1].file_id); + printf("\t.sleb128 %d\n", + (int)entries[idx].line - (int)entries[idx - 1].line); + } +} + +/* Vmlinux-mode wrapper: pick block index out of the global entries[]. */ +static void emit_block_data(unsigned int block) +{ + unsigned int base =3D block * LINEINFO_BLOCK_ENTRIES; + unsigned int count =3D num_entries - base; + + if (count > LINEINFO_BLOCK_ENTRIES) + count =3D LINEINFO_BLOCK_ENTRIES; + emit_block_data_range(base, count); +} + static void compute_file_offsets(void) { unsigned int offset =3D 0; @@ -1437,6 +1476,11 @@ static void print_escaped_asciz(const char *s) =20 static void output_assembly(void) { + unsigned int num_blocks; + + num_blocks =3D num_entries ? + (num_entries + LINEINFO_BLOCK_ENTRIES - 1) / LINEINFO_BLOCK_ENTRIES : 0; + printf("/* SPDX-License-Identifier: GPL-2.0 */\n"); printf("/*\n"); printf(" * Automatically generated by scripts/gen_lineinfo\n"); @@ -1457,29 +1501,40 @@ static void output_assembly(void) printf("lineinfo_num_files:\n"); printf("\t.long %u\n\n", num_files); =20 - /* Sorted address offsets from _text */ - printf("\t.globl lineinfo_addrs\n"); + /* Number of blocks */ + printf("\t.globl lineinfo_num_blocks\n"); printf("\t.balign 4\n"); - printf("lineinfo_addrs:\n"); - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.long 0x%x\n", entries[i].offset); - printf("\n"); + printf("lineinfo_num_blocks:\n"); + printf("\t.long %u\n\n", num_blocks); =20 - /* File IDs, parallel to addrs (u16 -- supports up to 65535 files) */ - printf("\t.globl lineinfo_file_ids\n"); - printf("\t.balign 2\n"); - printf("lineinfo_file_ids:\n"); - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.short %u\n", entries[i].file_id); - printf("\n"); + /* Block first-addresses for binary search */ + printf("\t.globl lineinfo_block_addrs\n"); + printf("\t.balign 4\n"); + printf("lineinfo_block_addrs:\n"); + for (unsigned int i =3D 0; i < num_blocks; i++) + printf("\t.long 0x%x\n", entries[i * LINEINFO_BLOCK_ENTRIES].offset); =20 - /* Line numbers, parallel to addrs */ - printf("\t.globl lineinfo_lines\n"); + /* Block byte offsets into compressed stream */ + printf("\t.globl lineinfo_block_offsets\n"); printf("\t.balign 4\n"); - printf("lineinfo_lines:\n"); - for (unsigned int i =3D 0; i < num_entries; i++) - printf("\t.long %u\n", entries[i].line); - printf("\n"); + printf("lineinfo_block_offsets:\n"); + for (unsigned int i =3D 0; i < num_blocks; i++) + printf("\t.long .Lblock_%u - lineinfo_data\n", i); + + /* Compressed data size */ + printf("\t.globl lineinfo_data_size\n"); + printf("\t.balign 4\n"); + printf("lineinfo_data_size:\n"); + printf("\t.long .Ldata_end - lineinfo_data\n\n"); + + /* Compressed data stream */ + printf("\t.globl lineinfo_data\n"); + printf("lineinfo_data:\n"); + for (unsigned int i =3D 0; i < num_blocks; i++) { + printf(".Lblock_%u:\n", i); + emit_block_data(i); + } + printf(".Ldata_end:\n\n"); =20 /* File string offset table */ printf("\t.globl lineinfo_file_offsets\n"); @@ -1487,45 +1542,39 @@ static void output_assembly(void) printf("lineinfo_file_offsets:\n"); for (unsigned int i =3D 0; i < num_files; i++) printf("\t.long %u\n", files[i]->str_offset); - printf("\n"); =20 /* Filenames size */ - { - unsigned int fsize =3D 0; - - for (unsigned int i =3D 0; i < num_files; i++) - fsize +=3D strlen(files[i]->name) + 1; - printf("\t.globl lineinfo_filenames_size\n"); - printf("\t.balign 4\n"); - printf("lineinfo_filenames_size:\n"); - printf("\t.long %u\n\n", fsize); - } + printf("\t.globl lineinfo_filenames_size\n"); + printf("\t.balign 4\n"); + printf("lineinfo_filenames_size:\n"); + printf("\t.long .Lfilenames_end - lineinfo_filenames\n\n"); =20 /* Concatenated NUL-terminated filenames */ printf("\t.globl lineinfo_filenames\n"); printf("lineinfo_filenames:\n"); for (unsigned int i =3D 0; i < num_files; i++) print_escaped_asciz(files[i]->name); - printf("\n"); + printf(".Lfilenames_end:\n"); } =20 /* - * Emit one per-section table in the simple flat-array layout: + * Emit one per-section table. @suffix uniquifies the local labels so + * multiple tables can coexist in a single output blob; @blob_root_label + * is the symbol for the start of the enclosing blob (used for + * table_offset =3D .Lhdr - .Lroot). * - * mod_lineinfo_header - * addrs[count] (u32, sorted) - * file_ids[count] (u16) + 2-byte pad if count is odd - * lines[count] (u32) - * file_offsets[] (u32) - * filenames[] - * - * @suffix uniquifies labels so multiple tables can coexist in one blob. - * Caller has sorted entries[] so this section's entries occupy [first, - * first + count). + * Caller has already sorted entries[] so this section's entries occupy + * the contiguous range [first, first + count). This function emits + * block-relative addresses computed from entries[first + N].offset. */ static void emit_section_table(unsigned int first, unsigned int count, const char *suffix) { + unsigned int num_blocks; + + num_blocks =3D count ? + (count + LINEINFO_BLOCK_ENTRIES - 1) / LINEINFO_BLOCK_ENTRIES : 0; + /* * Align before defining the label, not after: the descriptor stores * table_offset as .Lhdr - .Lroot, and every field offset inside the @@ -1536,29 +1585,45 @@ static void emit_section_table(unsigned int first, = unsigned int count, printf("\t.balign 4\n"); printf(".Lhdr%s:\n", suffix); printf("\t.long %u\t\t/* num_entries */\n", count); + printf("\t.long %u\t\t/* num_blocks */\n", num_blocks); printf("\t.long %u\t\t/* num_files */\n", num_files); + printf("\t.long .Lblk_addrs%s - .Lhdr%s\n", suffix, suffix); + printf("\t.long .Lblk_offsets_end%s - .Lblk_addrs%s\n", suffix, suffix); + printf("\t.long .Ldata%s - .Lhdr%s\n", suffix, suffix); + printf("\t.long .Ldata_end%s - .Ldata%s\n", suffix, suffix); + printf("\t.long .Lfile_offsets%s - .Lhdr%s\n", suffix, suffix); + printf("\t.long .Lfile_offsets_end%s - .Lfile_offsets%s\n", suffix, suffi= x); + printf("\t.long .Lfilenames%s - .Lhdr%s\n", suffix, suffix); printf("\t.long .Lfilenames_end%s - .Lfilenames%s\n\n", suffix, suffix); =20 - /* addrs[] */ - for (unsigned int i =3D 0; i < count; i++) - printf("\t.long 0x%x\n", entries[first + i].offset); - - /* file_ids[] */ - for (unsigned int i =3D 0; i < count; i++) - printf("\t.short %u\n", entries[first + i].file_id); - if (count & 1) - printf("\t.short 0\t\t/* pad to align lines[] */\n"); - - /* lines[] */ - for (unsigned int i =3D 0; i < count; i++) - printf("\t.long %u\n", entries[first + i].line); + printf(".Lblk_addrs%s:\n", suffix); + for (unsigned int i =3D 0; i < num_blocks; i++) + printf("\t.long 0x%x\n", + entries[first + i * LINEINFO_BLOCK_ENTRIES].offset); + + printf(".Lblk_offsets%s:\n", suffix); + for (unsigned int i =3D 0; i < num_blocks; i++) + printf("\t.long .Lblock%s_%u - .Ldata%s\n", suffix, i, suffix); + printf(".Lblk_offsets_end%s:\n\n", suffix); + + printf(".Ldata%s:\n", suffix); + for (unsigned int i =3D 0; i < num_blocks; i++) { + unsigned int base =3D first + i * LINEINFO_BLOCK_ENTRIES; + unsigned int n =3D count - i * LINEINFO_BLOCK_ENTRIES; + + if (n > LINEINFO_BLOCK_ENTRIES) + n =3D LINEINFO_BLOCK_ENTRIES; + printf(".Lblock%s_%u:\n", suffix, i); + emit_block_data_range(base, n); + } + printf(".Ldata_end%s:\n", suffix); =20 - /* file_offsets[] */ printf("\t.balign 4\n"); + printf(".Lfile_offsets%s:\n", suffix); for (unsigned int i =3D 0; i < num_files; i++) printf("\t.long %u\n", files[i]->str_offset); + printf(".Lfile_offsets_end%s:\n\n", suffix); =20 - /* filenames[] */ printf(".Lfilenames%s:\n", suffix); for (unsigned int i =3D 0; i < num_files; i++) print_escaped_asciz(files[i]->name); @@ -1838,7 +1903,11 @@ int main(int argc, char *argv[]) deduplicate(NULL, 0); compute_file_offsets(); =20 - verbose_msg("%u entries, %u files", num_entries, num_files); + verbose_msg("%u entries, %u files, %u blocks", + num_entries, num_files, + num_entries ? + (num_entries + LINEINFO_BLOCK_ENTRIES - 1) / + LINEINFO_BLOCK_ENTRIES : 0); =20 output_assembly(); } --=20 2.53.0 From nobody Fri Oct 2 12:19:52 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF39338331C; Sat, 1 Aug 2026 14:32:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785594760; cv=none; b=Yg6GfTFH3pU8n4N8guiUYYas1j/oc3+bkQJQVoPD3HHraZVd9mnWvdvpLaE5Y5XiTGQLcMLMCiIbRvI65ct52ah//K7MxMZJbZHuR+dEWlw4Hu2UfKBGSmOhYFQl7E7cwi7Jj+9pNPn1BTT8ZC/ChRl2PAuh4+ivd42A0tpMp5s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785594760; c=relaxed/simple; bh=rg66dxUHXg4hTQXvqJ+tqsQ4XUJ7pO9XQsM8J/cXANk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=l/G8h30q6pM9XSYTimBsj9OWtromRBdhcCU3Jo0Ap0WJmmgDbnJHiUqxxBTSMX+j7XL36+l6+yqgJP68J5+BSTXgCFoxsYs32J/ENIOup6Cj7UcKFN7cRYN7kj9oC1O8hZqg5GEBrKf6XM3Ki3hbyii1J4GtO7A2maSVoxkaJSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=nUz+okSd; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="nUz+okSd" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A00551F00ACA; Sat, 1 Aug 2026 14:32:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785594757; bh=v45iZQuI0PTMROlB6Wp4d1iIvXNMMd2K2h6MDtpwnTA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=nUz+okSd9m0lKWOtbrTvohBdzoOLmDbk70W7TIxxsYmscbTq3wymAmNGMSEOMKtYV 3+ZuxtLpja2a7JE6PbmqJm//l7Zd1DMB2EwI3xVKAsWgc68BPJyGX3bNgeBVkRYFsW 0aqxHHX07Ec5MNjUOJCWwDXIsgXSdH/jEZIR64r89yoPvPInRijBVukW5VRcd3nTUI EiipgGUKu8qrkTWx9USfSZuop29JG0EjKkBayEr49Rl5rh9VYCuo5yW29o2DEmfmD8 RHhJdqJhFt1g18YMb5nY+H1GYCrgdsZCG/rZFcbF17V3E7ZRyHtkjcci4jlMUtvPVj I1wOstgAJoFMg== From: Sasha Levin To: Andrew Morton , Masahiro Yamada , Luis Chamberlain , Linus Torvalds , Richard Weinberger , Juergen Gross , Geert Uytterhoeven , James Bottomley Cc: Sasha Levin , Jonathan Corbet , Nathan Chancellor , Nicolas Schier , Petr Pavlu , Daniel Gomez , Greg KH , Petr Mladek , Steven Rostedt , Kees Cook , Peter Zijlstra , Thorsten Leemhuis , Vlastimil Babka , Helge Deller , Randy Dunlap , Laurent Pinchart , Vivian Wang , Zhen Lei , Sami Tolvanen , linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-doc@vger.kernel.org Subject: [PATCH v8 4/4] kallsyms: add KUnit tests for lineinfo feature Date: Sat, 1 Aug 2026 10:32:18 -0400 Message-ID: <20260801143219.3975824-5-sashal@kernel.org> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260801143219.3975824-1-sashal@kernel.org> References: <20260801143219.3975824-1-sashal@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Add a KUnit test module (CONFIG_LINEINFO_KUNIT_TEST) that verifies the kallsyms lineinfo feature produces correct source file:line annotations in stack traces. Export sprint_backtrace() and sprint_backtrace_build_id() as GPL symbols so the test module can exercise the backtrace APIs. Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Sasha Levin --- kernel/kallsyms.c | 2 + lib/Kconfig.debug | 10 + lib/tests/Makefile | 3 + lib/tests/lineinfo_kunit.c | 1040 ++++++++++++++++++++++++++++++++++++ 4 files changed, 1055 insertions(+) create mode 100644 lib/tests/lineinfo_kunit.c diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index b5eff2ba8e5d..b0475f1e7f79 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -656,6 +656,7 @@ int sprint_backtrace(char *buffer, unsigned long addres= s) { return __sprint_symbol(buffer, address, -1, 1, 0, 1); } +EXPORT_SYMBOL_GPL(sprint_backtrace); =20 /** * sprint_backtrace_build_id - Look up a backtrace symbol and return it in= a text buffer @@ -676,6 +677,7 @@ int sprint_backtrace_build_id(char *buffer, unsigned lo= ng address) { return __sprint_symbol(buffer, address, -1, 1, 1, 1); } +EXPORT_SYMBOL_GPL(sprint_backtrace_build_id); =20 /* To avoid using get_symbol_offset for every symbol, we carry prefix alon= g. */ struct kallsym_iter { diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 1244dcac2294..257b5ca442f3 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -3092,6 +3092,16 @@ config LONGEST_SYM_KUNIT_TEST =20 If unsure, say N. =20 +config LINEINFO_KUNIT_TEST + tristate "KUnit tests for kallsyms lineinfo" if !KUNIT_ALL_TESTS + depends on KUNIT && KALLSYMS_LINEINFO + default KUNIT_ALL_TESTS + help + KUnit tests for the kallsyms source line info feature. + Verifies that stack traces include correct (file.c:line) annotations. + + If unsure, say N. + config HW_BREAKPOINT_KUNIT_TEST bool "Test hw_breakpoint constraints accounting" if !KUNIT_ALL_TESTS depends on HAVE_HW_BREAKPOINT diff --git a/lib/tests/Makefile b/lib/tests/Makefile index 4ead57602eac..877416f5f70a 100644 --- a/lib/tests/Makefile +++ b/lib/tests/Makefile @@ -37,6 +37,9 @@ obj-$(CONFIG_LIVEUPDATE_TEST) +=3D liveupdate.o CFLAGS_longest_symbol_kunit.o +=3D $(call cc-disable-warning, missing-prot= otypes) obj-$(CONFIG_LONGEST_SYM_KUNIT_TEST) +=3D longest_symbol_kunit.o =20 +CFLAGS_lineinfo_kunit.o +=3D $(call cc-option,-fno-inline-functions-called= -once) +obj-$(CONFIG_LINEINFO_KUNIT_TEST) +=3D lineinfo_kunit.o + obj-$(CONFIG_MEMCPY_KUNIT_TEST) +=3D memcpy_kunit.o obj-$(CONFIG_MIN_HEAP_KUNIT_TEST) +=3D min_heap_kunit.o CFLAGS_overflow_kunit.o =3D $(call cc-disable-warning, tautological-consta= nt-out-of-range-compare) diff --git a/lib/tests/lineinfo_kunit.c b/lib/tests/lineinfo_kunit.c new file mode 100644 index 000000000000..2e66e8ebb312 --- /dev/null +++ b/lib/tests/lineinfo_kunit.c @@ -0,0 +1,1040 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit tests for kallsyms lineinfo (CONFIG_KALLSYMS_LINEINFO). + * + * Copyright (c) 2026 Sasha Levin + * + * Verifies that sprint_symbol() and related APIs append correct + * " (file.c:NNN)" annotations to kernel symbol lookups. + * + * Build with: CONFIG_LINEINFO_KUNIT_TEST=3Dm (or =3Dy) + * Run with: ./tools/testing/kunit/kunit.py run lineinfo + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +/* --------------- helpers --------------- */ + +static char *alloc_sym_buf(struct kunit *test) +{ + char *buf =3D kunit_kzalloc(test, KSYM_SYMBOL_LEN, GFP_KERNEL); + + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buf); + return buf; +} + +/* + * Function-pointer casts yield the descriptor address on architectures + * with function descriptors (e.g. ppc64 ELFv1, parisc64); resolve to the + * code address the lineinfo tables actually cover. + */ +static unsigned long target_addr(const void *fn) +{ + return (unsigned long)dereference_function_descriptor((void *)fn); +} + +/* + * When the test is built as a module but CONFIG_KALLSYMS_LINEINFO_MODULES + * is disabled, the test module's own functions have no lineinfo tables, so + * every test that expects annotations on local targets must skip rather + * than fail. + */ +static void skip_without_module_lineinfo(struct kunit *test) +{ + if (IS_MODULE(CONFIG_LINEINFO_KUNIT_TEST) && + !IS_ENABLED(CONFIG_KALLSYMS_LINEINFO_MODULES)) + kunit_skip(test, + "KALLSYMS_LINEINFO_MODULES disabled; test module has no lineinfo"); +} + +/* + * Format a symbol with lineinfo annotation. Lineinfo is appended only + * via the sprint_backtrace*() entry points (kernel/kallsyms.c only adds + * the "(file:line)" suffix in stack-trace context =E2=80=94 sprint_symbol= () is + * used by %ps and many existing format strings tack literal "()" after + * %ps, where the annotation would render as "foo (file:line)()"). + * + * sprint_backtrace() subtracts 1 from the address to handle tail-call + * return-address corrections; pass @addr + 1 to recover the original. + */ +static int sprint_with_lineinfo(char *buf, unsigned long addr) +{ + return sprint_backtrace(buf, addr + 1); +} + +/* + * Return true if @buf contains a lineinfo annotation matching + * the pattern " (:)". + * + * The path may be a full path like "lib/tests/lineinfo_kunit.c" or + * a shortened form from module lineinfo (e.g., just a directory name). + */ +static bool has_lineinfo(const char *buf) +{ + const char *p, *colon, *end; + + p =3D strstr(buf, " ("); + if (!p) + return false; + p +=3D 2; /* skip " (" */ + + colon =3D strchr(p, ':'); + if (!colon || colon =3D=3D p) + return false; + + /* After colon: one or more digits then ')' */ + end =3D colon + 1; + if (*end < '0' || *end > '9') + return false; + while (*end >=3D '0' && *end <=3D '9') + end++; + return *end =3D=3D ')'; +} + +/* + * Extract line number from a lineinfo annotation. + * Returns 0 if not found. + */ +static unsigned int extract_line(const char *buf) +{ + const char *p, *colon; + unsigned int line =3D 0; + + p =3D strstr(buf, " ("); + if (!p) + return 0; + + colon =3D strchr(p + 2, ':'); + if (!colon) + return 0; + + colon++; + while (*colon >=3D '0' && *colon <=3D '9') { + line =3D line * 10 + (*colon - '0'); + colon++; + } + return line; +} + +/* + * Check if the lineinfo annotation contains the given filename substring. + */ +static bool lineinfo_contains_file(const char *buf, const char *name) +{ + const char *p, *colon; + + p =3D strstr(buf, " ("); + if (!p) + return false; + + colon =3D strchr(p + 2, ':'); + if (!colon) + return false; + + /* Search for @name between '(' and ':' */ + return strnstr(p + 1, name, colon - p - 1) !=3D NULL; +} + +/* --------------- target functions --------------- */ + +static noinline int lineinfo_target_normal(void) +{ + barrier(); + return 42; +} + +static noinline int lineinfo_target_short(void) +{ + barrier(); + return 1; +} + +static noinline int lineinfo_target_with_arg(int x) +{ + barrier(); + return x + 1; +} + +static noinline int lineinfo_target_many_lines(void) +{ + /* + * volatile forces a load/store per statement so the function keeps + * one line-table row per source line (and a body large enough for + * mid-function lookups) at any optimization level; without it the + * compiler folds the arithmetic to a single constant. + */ + volatile int a =3D 0; + + barrier(); + a +=3D 1; + a +=3D 2; + a +=3D 3; + a +=3D 4; + a +=3D 5; + a +=3D 6; + a +=3D 7; + a +=3D 8; + a +=3D 9; + a +=3D 10; + barrier(); + return a; +} + +static __always_inline int lineinfo_inline_helper(void) +{ + return 99; +} + +static noinline int lineinfo_inline_caller(void) +{ + barrier(); + return lineinfo_inline_helper(); +} + +/* 10-deep call chain */ +static noinline int lineinfo_chain_10(void) { barrier(); return 10; } +static noinline int lineinfo_chain_9(void) { barrier(); return lineinfo_c= hain_10(); } +static noinline int lineinfo_chain_8(void) { barrier(); return lineinfo_c= hain_9(); } +static noinline int lineinfo_chain_7(void) { barrier(); return lineinfo_c= hain_8(); } +static noinline int lineinfo_chain_6(void) { barrier(); return lineinfo_c= hain_7(); } +static noinline int lineinfo_chain_5(void) { barrier(); return lineinfo_c= hain_6(); } +static noinline int lineinfo_chain_4(void) { barrier(); return lineinfo_c= hain_5(); } +static noinline int lineinfo_chain_3(void) { barrier(); return lineinfo_c= hain_4(); } +static noinline int lineinfo_chain_2(void) { barrier(); return lineinfo_c= hain_3(); } +static noinline int lineinfo_chain_1(void) { barrier(); return lineinfo_c= hain_2(); } + +/* --------------- Group A: Basic lineinfo presence --------------- */ + +static void test_normal_function(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + skip_without_module_lineinfo(test); + + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo in: %s", buf); + KUNIT_EXPECT_TRUE_MSG(test, + lineinfo_contains_file(buf, "lineinfo_kunit.c"), + "Wrong file in: %s", buf); +} + +static void test_static_function(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_short); + + skip_without_module_lineinfo(test); + + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo in: %s", buf); +} + +static void test_noinline_function(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_with_arg); + + skip_without_module_lineinfo(test); + + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo in: %s", buf); +} + +static void test_inline_function(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_inline_caller); + + skip_without_module_lineinfo(test); + + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo for inline caller in: %s", buf); + KUNIT_EXPECT_TRUE_MSG(test, + lineinfo_contains_file(buf, "lineinfo_kunit.c"), + "Wrong file in: %s", buf); +} + +static void test_short_function(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_short); + + skip_without_module_lineinfo(test); + + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo for short function in: %s", buf); +} + +static void test_many_lines_function(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_many_lines); + unsigned int line; + + skip_without_module_lineinfo(test); + + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo in: %s", buf); + line =3D extract_line(buf); + KUNIT_EXPECT_GT_MSG(test, line, (unsigned int)0, + "Line number should be > 0 in: %s", buf); +} + +/* --------------- Group B: Deep call chain --------------- */ + +typedef int (*chain_fn_t)(void); + +static void test_deep_call_chain(struct kunit *test) +{ + static const chain_fn_t chain_fns[] =3D { + lineinfo_chain_1, lineinfo_chain_2, + lineinfo_chain_3, lineinfo_chain_4, + lineinfo_chain_5, lineinfo_chain_6, + lineinfo_chain_7, lineinfo_chain_8, + lineinfo_chain_9, lineinfo_chain_10, + }; + char *buf =3D alloc_sym_buf(test); + int i, found =3D 0; + + skip_without_module_lineinfo(test); + + /* Call chain to prevent dead-code elimination */ + KUNIT_ASSERT_EQ(test, lineinfo_chain_1(), 10); + + for (i =3D 0; i < ARRAY_SIZE(chain_fns); i++) { + unsigned long addr =3D target_addr(chain_fns[i]); + + sprint_with_lineinfo(buf, addr); + if (has_lineinfo(buf)) + found++; + } + + /* + * Not every tiny function gets DWARF line info (compiler may + * omit it for very small stubs), but at least some should. + */ + KUNIT_EXPECT_GT_MSG(test, found, 0, + "None of the 10 chain functions had lineinfo"); +} + +/* --------------- Group C: sprint_symbol API variants --------------- */ + +static void test_sprint_symbol_format(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + sprint_symbol(buf, addr); + + /* Should contain +0x and /0x for offset/size */ + KUNIT_EXPECT_NOT_NULL_MSG(test, strstr(buf, "+0x"), + "Missing offset in: %s", buf); + KUNIT_EXPECT_NOT_NULL_MSG(test, strstr(buf, "/0x"), + "Missing size in: %s", buf); + /* + * sprint_symbol() backs %ps, which existing format strings combine + * with literal "()" to indicate function calls; the lineinfo suffix + * is intentionally omitted there to avoid "foo (file:line)()". + */ + KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf), + "Unexpected lineinfo in sprint_symbol output: %s", + buf); +} + +static void test_sprint_backtrace(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + skip_without_module_lineinfo(test); + + /* sprint_backtrace subtracts 1 internally to handle tail calls */ + sprint_backtrace(buf, addr + 1); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo in backtrace: %s", buf); + KUNIT_EXPECT_TRUE_MSG(test, + lineinfo_contains_file(buf, "lineinfo_kunit.c"), + "Wrong file in backtrace: %s", buf); +} + +static void test_sprint_backtrace_build_id(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + skip_without_module_lineinfo(test); + + sprint_backtrace_build_id(buf, addr + 1); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo in backtrace_build_id: %s", buf); +} + +static void test_sprint_symbol_no_offset(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + sprint_symbol_no_offset(buf, addr); + /* No "+0x" in output */ + KUNIT_EXPECT_NULL_MSG(test, strstr(buf, "+0x"), + "Unexpected offset in no_offset: %s", buf); + /* sprint_symbol_no_offset is a sprint_symbol() variant; lineinfo is + * intentionally only appended in sprint_backtrace*() context. + */ + KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf), + "Unexpected lineinfo in no_offset: %s", buf); +} + +/* --------------- Group D: printk format specifiers --------------- */ + +static void test_pS_format(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + void *addr =3D lineinfo_target_normal; + + snprintf(buf, KSYM_SYMBOL_LEN, "%pS", addr); + /* + * %pS uses sprint_symbol(), which intentionally omits the lineinfo + * suffix (see kernel/kallsyms.c::__sprint_symbol). Lineinfo is only + * added via the sprint_backtrace*() entry points, which back %pBb. + */ + KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf), + "Unexpected lineinfo in %%pS: %s", buf); +} + +static void test_pBb_format(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + /* + * %pBb uses sprint_backtrace_build_id which subtracts 1 from the + * address, so pass addr+1 to resolve back to the function. + */ + void *addr =3D (void *)(target_addr(lineinfo_target_normal) + 1); + + skip_without_module_lineinfo(test); + + snprintf(buf, KSYM_SYMBOL_LEN, "%pBb", addr); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo in %%pBb: %s", buf); +} + +static void test_pSR_format(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + void *addr =3D lineinfo_target_normal; + + snprintf(buf, KSYM_SYMBOL_LEN, "%pSR", addr); + /* %pSR is a sprint_symbol() variant; same rationale as %pS. */ + KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf), + "Unexpected lineinfo in %%pSR: %s", buf); +} + +/* --------------- Group E: Address edge cases --------------- */ + +static void test_symbol_start_addr(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + skip_without_module_lineinfo(test); + + /* + * sprint_backtrace() subtracts 1 from the input and reports offset + * relative to the (decremented) address, so an exact "+0x0/" can't + * be expected here. Verify the symbol resolves and carries lineinfo. + */ + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_TRUE_MSG(test, + strnstr(buf, "lineinfo_target_normal", + KSYM_SYMBOL_LEN) !=3D NULL, + "Didn't resolve to expected function: %s", buf); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo at function start: %s", buf); +} + +static void test_symbol_nonzero_offset(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + skip_without_module_lineinfo(test); + + /* + * sprint_backtrace subtracts 1 internally. + * Passing addr+2 resolves to addr+1 which is inside the function + * at a non-zero offset. + */ + sprint_backtrace(buf, addr + 2); + KUNIT_EXPECT_TRUE_MSG(test, + strnstr(buf, "lineinfo_target_normal", + KSYM_SYMBOL_LEN) !=3D NULL, + "Didn't resolve to expected function: %s", buf); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo at non-zero offset: %s", buf); +} + +static void test_unknown_address(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + + sprint_symbol(buf, 1UL); + /* Should be "0x1" with no lineinfo */ + KUNIT_EXPECT_NOT_NULL_MSG(test, strstr(buf, "0x1"), + "Expected hex address for bogus addr: %s", buf); + KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf), + "Unexpected lineinfo for bogus addr: %s", buf); +} + +static void test_kernel_function_lineinfo(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(sprint_symbol); + + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo for sprint_symbol: %s", buf); + KUNIT_EXPECT_TRUE_MSG(test, + lineinfo_contains_file(buf, "kallsyms.c"), + "Expected kallsyms.c in: %s", buf); +} + +static void test_assembly_no_lineinfo(struct kunit *test) +{ +#if IS_BUILTIN(CONFIG_LINEINFO_KUNIT_TEST) + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D (unsigned long)_text; + + sprint_with_lineinfo(buf, addr); + /* + * _text is typically an asm entry point with no DWARF line info. + * If it has lineinfo, it's a C-based entry =E2=80=94 skip in that case. + */ + if (has_lineinfo(buf)) + kunit_skip(test, "_text has lineinfo (C entry?): %s", buf); + + KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf), + "Unexpected lineinfo for asm symbol: %s", buf); +#else + kunit_skip(test, "_text not accessible from modules"); +#endif +} + +static void test_init_text_no_stale_lineinfo(struct kunit *test) +{ +#if IS_BUILTIN(CONFIG_LINEINFO_KUNIT_TEST) + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D (unsigned long)_sinittext + 16; + + /* + * .init.text lies past the _etext cap of the vmlinux lineinfo + * table, so a lookup there must not inherit the annotation of + * the last covered .text entry. + */ + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf), + "Stale lineinfo for init-text address: %s", buf); +#else + kunit_skip(test, "_sinittext not accessible from modules"); +#endif +} + +/* --------------- Group F: Module path --------------- */ + +static void test_module_function_lineinfo(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + skip_without_module_lineinfo(test); + + if (!IS_MODULE(CONFIG_LINEINFO_KUNIT_TEST)) { + kunit_skip(test, "Test only meaningful when built as module"); + return; + } + + sprint_with_lineinfo(buf, addr); + KUNIT_EXPECT_NOT_NULL_MSG(test, + strstr(buf, "[lineinfo_kunit"), + "Missing module name in: %s", buf); + KUNIT_EXPECT_TRUE_MSG(test, has_lineinfo(buf), + "No lineinfo for module function: %s", buf); + KUNIT_EXPECT_TRUE_MSG(test, + lineinfo_contains_file(buf, "lineinfo_kunit.c"), + "Wrong file for module function: %s", buf); +} + +/* --------------- Group G: Stress --------------- */ + +struct lineinfo_stress_data { + unsigned long addr; + atomic_t failures; +}; + +static void lineinfo_stress_fn(void *info) +{ + struct lineinfo_stress_data *data =3D info; + char buf[KSYM_SYMBOL_LEN]; + int i; + + for (i =3D 0; i < 100; i++) { + sprint_with_lineinfo(buf, data->addr); + if (!has_lineinfo(buf)) + atomic_inc(&data->failures); + } +} + +static void test_concurrent_sprint_symbol(struct kunit *test) +{ + struct lineinfo_stress_data data; + + skip_without_module_lineinfo(test); + + data.addr =3D target_addr(lineinfo_target_normal); + atomic_set(&data.failures, 0); + + on_each_cpu(lineinfo_stress_fn, &data, 1); + + KUNIT_EXPECT_EQ_MSG(test, atomic_read(&data.failures), 0, + "Concurrent lineinfo failures detected"); +} + +static void test_rapid_sprint_symbol(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + int i, failures =3D 0; + + skip_without_module_lineinfo(test); + + for (i =3D 0; i < 1000; i++) { + sprint_with_lineinfo(buf, addr); + if (!has_lineinfo(buf)) + failures++; + } + + KUNIT_EXPECT_EQ_MSG(test, failures, 0, + "Rapid sprint_symbol failures: %d/1000", failures); +} + +/* --------------- Group H: Safety and plausibility --------------- */ + +static void test_line_number_plausible(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + unsigned int line; + + skip_without_module_lineinfo(test); + + sprint_with_lineinfo(buf, addr); + KUNIT_ASSERT_TRUE(test, has_lineinfo(buf)); + + line =3D extract_line(buf); + KUNIT_EXPECT_GT_MSG(test, line, (unsigned int)0, + "Line number should be > 0"); + KUNIT_EXPECT_LT_MSG(test, line, (unsigned int)10000, + "Line number %u implausibly large for this file", + line); +} + +static void test_buffer_no_overflow(struct kunit *test) +{ + const size_t canary_size =3D 16; + char *buf; + int i; + + buf =3D kunit_kzalloc(test, KSYM_SYMBOL_LEN + canary_size, GFP_KERNEL); + KUNIT_ASSERT_NOT_NULL(test, buf); + + /* Fill canary area past KSYM_SYMBOL_LEN with 0xAA */ + memset(buf + KSYM_SYMBOL_LEN, 0xAA, canary_size); + + sprint_with_lineinfo(buf, target_addr(lineinfo_target_normal)); + + /* Verify canary bytes are untouched */ + for (i =3D 0; i < canary_size; i++) { + KUNIT_EXPECT_EQ_MSG(test, + (unsigned char)buf[KSYM_SYMBOL_LEN + i], + (unsigned char)0xAA, + "Buffer overflow at offset %d past KSYM_SYMBOL_LEN", + i); + } +} + +static void test_dump_stack_no_crash(struct kunit *test) +{ + /* Just verify dump_stack() completes without panic */ + dump_stack(); + KUNIT_SUCCEED(test); +} + +static void test_sprint_symbol_build_id(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + + sprint_symbol_build_id(buf, addr); + /* Lineinfo is appended only via sprint_backtrace*(); the symbol + * variants intentionally omit it to avoid clashing with format + * strings that already wrap %ps in literal "()". + */ + KUNIT_EXPECT_FALSE_MSG(test, has_lineinfo(buf), + "Unexpected lineinfo in sprint_symbol_build_id: %s", + buf); +} + +static void test_sleb128_edge_cases(struct kunit *test) +{ + u32 pos; + int32_t result; + + /* Value 0: single byte 0x00 */ + { + static const u8 data[] =3D { 0x00 }; + + pos =3D 0; + result =3D lineinfo_read_sleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (int32_t)0); + KUNIT_EXPECT_EQ(test, pos, (u32)1); + } + + /* Value -1: single byte 0x7F */ + { + static const u8 data[] =3D { 0x7f }; + + pos =3D 0; + result =3D lineinfo_read_sleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (int32_t)-1); + KUNIT_EXPECT_EQ(test, pos, (u32)1); + } + + /* Value 1: single byte 0x01 */ + { + static const u8 data[] =3D { 0x01 }; + + pos =3D 0; + result =3D lineinfo_read_sleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (int32_t)1); + KUNIT_EXPECT_EQ(test, pos, (u32)1); + } + + /* Value -64: single byte 0x40 */ + { + static const u8 data[] =3D { 0x40 }; + + pos =3D 0; + result =3D lineinfo_read_sleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (int32_t)-64); + KUNIT_EXPECT_EQ(test, pos, (u32)1); + } + + /* Value 63: single byte 0x3F */ + { + static const u8 data[] =3D { 0x3f }; + + pos =3D 0; + result =3D lineinfo_read_sleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (int32_t)63); + KUNIT_EXPECT_EQ(test, pos, (u32)1); + } + + /* Value -128: two bytes 0x80 0x7F */ + { + static const u8 data[] =3D { 0x80, 0x7f }; + + pos =3D 0; + result =3D lineinfo_read_sleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (int32_t)-128); + KUNIT_EXPECT_EQ(test, pos, (u32)2); + } + + /* Value INT32_MIN: five bytes, high bits land at shift 28 */ + { + static const u8 data[] =3D { 0x80, 0x80, 0x80, 0x80, 0x78 }; + const u8 *p =3D data; + + /* Defeat constant folding so the runtime decoder runs. */ + OPTIMIZER_HIDE_VAR(p); + pos =3D 0; + result =3D lineinfo_read_sleb128(p, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (int32_t)INT_MIN); + KUNIT_EXPECT_EQ(test, pos, (u32)5); + } + + /* Value INT32_MAX: five bytes */ + { + static const u8 data[] =3D { 0xff, 0xff, 0xff, 0xff, 0x07 }; + const u8 *p =3D data; + + OPTIMIZER_HIDE_VAR(p); + pos =3D 0; + result =3D lineinfo_read_sleb128(p, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (int32_t)INT_MAX); + KUNIT_EXPECT_EQ(test, pos, (u32)5); + } +} + +static void test_uleb128_edge_cases(struct kunit *test) +{ + u32 pos, result; + + /* Value 0: single byte 0x00 */ + { + static const u8 data[] =3D { 0x00 }; + + pos =3D 0; + result =3D lineinfo_read_uleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (u32)0); + KUNIT_EXPECT_EQ(test, pos, (u32)1); + } + + /* Value 127: single byte 0x7F */ + { + static const u8 data[] =3D { 0x7F }; + + pos =3D 0; + result =3D lineinfo_read_uleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (u32)127); + KUNIT_EXPECT_EQ(test, pos, (u32)1); + } + + /* Value 128: two bytes 0x80 0x01 */ + { + static const u8 data[] =3D { 0x80, 0x01 }; + + pos =3D 0; + result =3D lineinfo_read_uleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (u32)128); + KUNIT_EXPECT_EQ(test, pos, (u32)2); + } + + /* Max u32 0xFFFFFFFF: 5 bytes */ + { + static const u8 data[] =3D { 0xFF, 0xFF, 0xFF, 0xFF, 0x0F }; + + pos =3D 0; + result =3D lineinfo_read_uleb128(data, &pos, sizeof(data)); + KUNIT_EXPECT_EQ(test, result, (u32)0xFFFFFFFF); + KUNIT_EXPECT_EQ(test, pos, (u32)5); + } + + /* Truncated input: pos >=3D end returns 0 */ + { + static const u8 data[] =3D { 0x80 }; + + pos =3D 0; + result =3D lineinfo_read_uleb128(data, &pos, 0); + KUNIT_EXPECT_EQ_MSG(test, result, (u32)0, + "Expected 0 for empty input"); + } + + /* Truncated mid-varint: continuation byte but end reached */ + { + static const u8 data[] =3D { 0x80 }; + + pos =3D 0; + result =3D lineinfo_read_uleb128(data, &pos, 1); + KUNIT_EXPECT_EQ_MSG(test, result, (u32)0, + "Expected 0 for truncated varint"); + KUNIT_EXPECT_EQ(test, pos, (u32)1); + } +} + +/* + * The symbol-boundary guard must reject an entry belonging to a preceding + * symbol, but must still accept one that sits exactly on the symbol start. + * + * gen_lineinfo used to collapse the second of two entries sharing a + * file:line, which left functions whose first entry was collapsed with + * nothing at or above their own start -- so this rejected them and the + * frame printed unannotated. The generator now keeps (or synthesizes) an + * entry at every symbol start; this pins down the contract it relies on. + */ +static void test_lineinfo_search_min_offset(struct kunit *test) +{ + /* + * One block, two entries with identical file:line at offsets 0 and + * 0x40 -- exactly the pair deduplicate() used to collapse. + * Entry 0 is uleb(file_id), uleb(line); the rest are + * uleb(addr_delta), sleb(file_delta), sleb(line_delta). + */ + static const u8 data[] =3D { 0x00, 0x0a, 0x40, 0x00, 0x00 }; + static const u32 blk_addrs[] =3D { 0 }; + static const u32 blk_offsets[] =3D { 0 }; + static const u32 file_offsets[] =3D { 0 }; + static const char filenames[] =3D "test.c"; + const struct lineinfo_table tbl =3D { + .blk_addrs =3D blk_addrs, + .blk_offsets =3D blk_offsets, + .data =3D data, + .data_size =3D sizeof(data), + .file_offsets =3D file_offsets, + .filenames =3D filenames, + .num_entries =3D 2, + .num_blocks =3D 1, + .num_files =3D 1, + .filenames_size =3D sizeof(filenames), + }; + const char *file; + unsigned int line; + + /* Symbol starts at 0x40 and owns the entry there: must resolve. */ + file =3D NULL; + line =3D 0; + KUNIT_EXPECT_TRUE_MSG(test, + lineinfo_search(&tbl, 0x50, 0x40, &file, &line), + "entry exactly at the symbol start was rejected"); + KUNIT_EXPECT_STREQ(test, file, "test.c"); + KUNIT_EXPECT_EQ(test, line, 10u); + + /* No min_offset bound: same answer. */ + file =3D NULL; + line =3D 0; + KUNIT_EXPECT_TRUE(test, + lineinfo_search(&tbl, 0x50, 0, &file, &line)); + KUNIT_EXPECT_EQ(test, line, 10u); + + /* + * Symbol starts at 0x60, above every entry: the 0x40 entry belongs + * to a preceding symbol and must not leak into this one. + */ + KUNIT_EXPECT_FALSE_MSG(test, + lineinfo_search(&tbl, 0x70, 0x60, &file, &line), + "entry below the symbol start was accepted"); +} + +static void test_line_number_accuracy(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_normal); + unsigned int line; + + skip_without_module_lineinfo(test); + + sprint_with_lineinfo(buf, addr); + KUNIT_ASSERT_TRUE(test, has_lineinfo(buf)); + + line =3D extract_line(buf); + + /* + * lineinfo_target_normal is defined around line 103-107. + * Allow wide range: KASAN instrumentation and module lineinfo + * address mapping can shift the reported line significantly. + */ + KUNIT_EXPECT_GE_MSG(test, line, (unsigned int)50, + "Line %u too low for lineinfo_target_normal", line); + KUNIT_EXPECT_LE_MSG(test, line, (unsigned int)300, + "Line %u too high for lineinfo_target_normal", line); +} + +static void test_many_lines_mid_function(struct kunit *test) +{ + char *buf =3D alloc_sym_buf(test); + unsigned long addr =3D target_addr(lineinfo_target_many_lines); + unsigned int line; + unsigned long mid_addr; + + skip_without_module_lineinfo(test); + + /* Get function size from sprint_with_lineinfo output */ + sprint_with_lineinfo(buf, addr); + KUNIT_ASSERT_TRUE(test, has_lineinfo(buf)); + + /* + * Try an address 8 bytes into the function (past prologue). Hide + * the base from the compiler so this does not become a static + * mid-function relocation (sym+8), which objtool flags as a + * relocation to !ENDBR with CONFIG_X86_KERNEL_IBT. + */ + OPTIMIZER_HIDE_VAR(addr); + mid_addr =3D addr + 8; + sprint_with_lineinfo(buf, mid_addr); + + /* + * Should still resolve to lineinfo_target_many_lines. + * Lineinfo should be present with a plausible line number. + */ + KUNIT_EXPECT_TRUE_MSG(test, + strnstr(buf, "lineinfo_target_many_lines", + KSYM_SYMBOL_LEN) !=3D NULL, + "Mid-function addr resolved to wrong symbol: %s", + buf); + if (has_lineinfo(buf)) { + line =3D extract_line(buf); + KUNIT_EXPECT_GE_MSG(test, line, (unsigned int)50, + "Line %u too low for mid-function", line); + KUNIT_EXPECT_LE_MSG(test, line, (unsigned int)700, + "Line %u too high for mid-function", line); + } +} + +/* --------------- Suite registration --------------- */ + +static struct kunit_case lineinfo_test_cases[] =3D { + /* Group A: Basic lineinfo presence */ + KUNIT_CASE(test_normal_function), + KUNIT_CASE(test_static_function), + KUNIT_CASE(test_noinline_function), + KUNIT_CASE(test_inline_function), + KUNIT_CASE(test_short_function), + KUNIT_CASE(test_many_lines_function), + /* Group B: Deep call chain */ + KUNIT_CASE(test_deep_call_chain), + /* Group C: sprint_symbol API variants */ + KUNIT_CASE(test_sprint_symbol_format), + KUNIT_CASE(test_sprint_backtrace), + KUNIT_CASE(test_sprint_backtrace_build_id), + KUNIT_CASE(test_sprint_symbol_no_offset), + /* Group D: printk format specifiers */ + KUNIT_CASE(test_pS_format), + KUNIT_CASE(test_pBb_format), + KUNIT_CASE(test_pSR_format), + /* Group E: Address edge cases */ + KUNIT_CASE(test_symbol_start_addr), + KUNIT_CASE(test_symbol_nonzero_offset), + KUNIT_CASE(test_unknown_address), + KUNIT_CASE(test_kernel_function_lineinfo), + KUNIT_CASE(test_assembly_no_lineinfo), + KUNIT_CASE(test_init_text_no_stale_lineinfo), + /* Group F: Module path */ + KUNIT_CASE(test_module_function_lineinfo), + /* Group G: Stress */ + KUNIT_CASE_SLOW(test_concurrent_sprint_symbol), + KUNIT_CASE_SLOW(test_rapid_sprint_symbol), + /* Group H: Safety and plausibility */ + KUNIT_CASE(test_line_number_plausible), + KUNIT_CASE(test_buffer_no_overflow), + KUNIT_CASE(test_dump_stack_no_crash), + KUNIT_CASE(test_sprint_symbol_build_id), + /* Group I: Encoding/decoding and accuracy */ + KUNIT_CASE(test_sleb128_edge_cases), + KUNIT_CASE(test_uleb128_edge_cases), + KUNIT_CASE(test_lineinfo_search_min_offset), + KUNIT_CASE(test_line_number_accuracy), + KUNIT_CASE(test_many_lines_mid_function), + {} +}; + +static struct kunit_suite lineinfo_test_suite =3D { + .name =3D "lineinfo", + .test_cases =3D lineinfo_test_cases, +}; +kunit_test_suites(&lineinfo_test_suite); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("KUnit tests for kallsyms lineinfo"); +MODULE_AUTHOR("Sasha Levin"); --=20 2.53.0