Currently resolve_btfids updates .BTF_ids section of an ELF file
in-place, based on the contents of provided BTF, usually within the
same input file, and optionally a BTF base.
This patch changes resolve_btfids behavior to enable BTF
transformations as part of its main operation. To achieve this
in-place ELF write in resolve_btfids is replaced with generation of
the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for
out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
The execution of resolve_btfids and consumption of its output is
orchestrated by scripts/gen-btf.sh introduced in this patch.
The rationale for this approach is that updating ELF in-place with
libelf API is complicated and bug-prone, especially in the context of
the kernel build. On the other hand applying objcopy to manipulate ELF
sections is simpler and more reliable.
There are two distinct paths for BTF generation and resolve_btfids
application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to
ensure correct linking (details below). The patch doesn't change this
approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with
llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1
into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3],
updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a *detached* file with
raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
(potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For the kernel modules creating special .bpf.o file is not necessary,
and so embedding of sections data produced by resolve_btfids is
straightforward with the objcopy.
With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows to delete elf_update() call and satelite
code (like compressed_section_fix [4]).
Endianness handling of .BTF_ids data is also changed. Previously the
"flags" part of the section was bswapped in sets_patch() [5], and then
Elf_Type was modified before elf_update() to signal to libelf that
bswap may be necessary. With this patch we explicitly bswap entire
data buffer on load and on dump.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/20200819092342.259004-1-jolsa@kernel.org/
[5] https://lore.kernel.org/bpf/cover.1707223196.git.vmalik@redhat.com/
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
---
MAINTAINERS | 1 +
scripts/Makefile.modfinal | 5 +-
scripts/gen-btf.sh | 167 ++++++++++++++++++++
scripts/link-vmlinux.sh | 42 +-----
tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++----------
tools/testing/selftests/bpf/Makefile | 5 +
6 files changed, 317 insertions(+), 121 deletions(-)
create mode 100755 scripts/gen-btf.sh
diff --git a/MAINTAINERS b/MAINTAINERS
index 48aabeeed029..5cd34419d952 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c
F: net/sched/cls_bpf.c
F: samples/bpf/
F: scripts/bpf_doc.py
+F: scripts/gen-btf.sh
F: scripts/Makefile.btf
F: scripts/pahole-version.sh
F: tools/bpf/
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 542ba462ed3e..3862fdfa1267 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
cmd_btf_ko = \
if [ ! -f $(objtree)/vmlinux ]; then \
printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
- else \
- LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
- $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \
+ else \
+ $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
fi;
# Same as newer-prereqs, but allows to exclude specified extra dependencies
diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
new file mode 100755
index 000000000000..2dfb7ab289ca
--- /dev/null
+++ b/scripts/gen-btf.sh
@@ -0,0 +1,167 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
+#
+# This script generates BTF data for the provided ELF file.
+#
+# Kernel BTF generation involves these conceptual steps:
+# 1. pahole generates BTF from DWARF data
+# 2. resolve_btfids applies kernel-specific btf2btf
+# transformations and computes data for .BTF_ids section
+# 3. the result gets linked/objcopied into the target binary
+#
+# How step (3) should be done differs between vmlinux, and
+# kernel modules, which is the primary reason for the existence
+# of this script.
+#
+# For modules the script expects vmlinux passed in as --btf_base.
+# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
+# into the input ELF file with objcopy.
+#
+# For vmlinux the input file remains unchanged and two files are produced:
+# - ${1}.btf.o ready for linking into vmlinux
+# - ${1}.btf_ids with .BTF_ids data blob
+# This output is consumed by scripts/link-vmlinux.sh
+
+set -e
+
+usage()
+{
+ echo "Usage: $0 [--btf_base <file>] <target ELF file>"
+ exit 1
+}
+
+BTF_BASE=""
+
+while [ $# -gt 0 ]; do
+ case "$1" in
+ --btf_base)
+ BTF_BASE="$2"
+ shift 2
+ ;;
+ -*)
+ echo "Unknown option: $1" >&2
+ usage
+ ;;
+ *)
+ break
+ ;;
+ esac
+done
+
+if [ $# -ne 1 ]; then
+ usage
+fi
+
+ELF_FILE="$1"
+shift
+
+is_enabled() {
+ grep -q "^$1=y" ${objtree}/include/config/auto.conf
+}
+
+info()
+{
+ printf " %-7s %s\n" "${1}" "${2}"
+}
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+ set -x
+ ;;
+esac
+
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+ exit 0
+fi
+
+gen_btf_data()
+{
+ info BTF "${ELF_FILE}"
+ btf1="${ELF_FILE}.btf.1"
+ ${PAHOLE} -J ${PAHOLE_FLAGS} \
+ ${BTF_BASE:+--btf_base ${BTF_BASE}} \
+ --btf_encode_detached=${btf1} \
+ "${ELF_FILE}"
+
+ info BTFIDS "${ELF_FILE}"
+ RESOLVE_BTFIDS_OPTS=""
+ if is_enabled CONFIG_WERROR; then
+ RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
+ fi
+ if [ -n "${KBUILD_VERBOSE}" ]; then
+ RESOLVE_BTFIDS_OPTS+=" -v "
+ fi
+ ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \
+ ${BTF_BASE:+--btf_base ${BTF_BASE}} \
+ --btf ${btf1} "${ELF_FILE}"
+}
+
+gen_btf_o()
+{
+ local btf_data=${ELF_FILE}.btf.o
+
+ # Create ${btf_data} which contains just .BTF section but no symbols. Add
+ # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
+ # deletes all symbols including __start_BTF and __stop_BTF, which will
+ # be redefined in the linker script.
+ info OBJCOPY "${btf_data}"
+ echo "" | ${CC} -c -x c -o ${btf_data} -
+ ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
+ --set-section-flags .BTF=alloc,readonly ${btf_data}
+ ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
+
+ # Change e_type to ET_REL so that it can be used to link final vmlinux.
+ # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
+ if is_enabled CONFIG_CPU_BIG_ENDIAN; then
+ et_rel='\0\1'
+ else
+ et_rel='\1\0'
+ fi
+ printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
+}
+
+embed_btf_data()
+{
+ info OBJCOPY "${ELF_FILE}"
+ ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}
+
+ # a module might not have a .BTF_ids or .BTF.base section
+ local btf_base="${ELF_FILE}.distilled_base.btf"
+ if [ -f "${btf_base}" ]; then
+ ${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE}
+ fi
+ local btf_ids="${ELF_FILE}.btf_ids"
+ if [ -f "${btf_ids}" ]; then
+ ${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE}
+ fi
+}
+
+cleanup()
+{
+ rm -f "${ELF_FILE}.btf.1"
+ rm -f "${ELF_FILE}.btf"
+ if [ "${BTFGEN_MODE}" = "module" ]; then
+ rm -f "${ELF_FILE}.distilled_base.btf"
+ rm -f "${ELF_FILE}.btf_ids"
+ fi
+}
+trap cleanup EXIT
+
+BTFGEN_MODE="vmlinux"
+if [ -n "${BTF_BASE}" ]; then
+ BTFGEN_MODE="module"
+fi
+
+gen_btf_data
+
+case "${BTFGEN_MODE}" in
+vmlinux)
+ gen_btf_o
+ ;;
+module)
+ embed_btf_data
+ ;;
+esac
+
+exit 0
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 433849ff7529..5bea8795f96d 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -105,34 +105,6 @@ vmlinux_link()
${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
}
-# generate .BTF typeinfo from DWARF debuginfo
-# ${1} - vmlinux image
-gen_btf()
-{
- local btf_data=${1}.btf.o
-
- info BTF "${btf_data}"
- LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
-
- # Create ${btf_data} which contains just .BTF section but no symbols. Add
- # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
- # deletes all symbols including __start_BTF and __stop_BTF, which will
- # be redefined in the linker script. Add 2>/dev/null to suppress GNU
- # objcopy warnings: "empty loadable segment detected at ..."
- ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
- --strip-all ${1} "${btf_data}" 2>/dev/null
- # Change e_type to ET_REL so that it can be used to link final vmlinux.
- # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
- if is_enabled CONFIG_CPU_BIG_ENDIAN; then
- et_rel='\0\1'
- else
- et_rel='\1\0'
- fi
- printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
-
- btf_vmlinux_bin_o=${btf_data}
-}
-
# Create ${2}.o file with all symbols from the ${1} object file
kallsyms()
{
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
fi
btf_vmlinux_bin_o=
+btfids_vmlinux=
kallsymso=
strip_debug=
generate_map=
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
fi
if is_enabled CONFIG_DEBUG_INFO_BTF; then
- if ! gen_btf .tmp_vmlinux1; then
+ if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then
echo >&2 "Failed to generate BTF for vmlinux"
echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
exit 1
fi
+ btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
+ btfids_vmlinux=.tmp_vmlinux1.btf_ids
fi
if is_enabled CONFIG_KALLSYMS; then
@@ -281,14 +256,9 @@ fi
vmlinux_link "${VMLINUX}"
-# fill in BTF IDs
if is_enabled CONFIG_DEBUG_INFO_BTF; then
- info BTFIDS "${VMLINUX}"
- RESOLVE_BTFIDS_ARGS=""
- if is_enabled CONFIG_WERROR; then
- RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
- fi
- ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
+ info OBJCOPY ${btfids_vmlinux}
+ ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
fi
mksysmap "${VMLINUX}" System.map
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index c60d303ca6ed..b8df6256e29e 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -71,9 +71,11 @@
#include <fcntl.h>
#include <errno.h>
#include <linux/btf_ids.h>
+#include <linux/kallsyms.h>
#include <linux/rbtree.h>
#include <linux/zalloc.h>
#include <linux/err.h>
+#include <linux/limits.h>
#include <bpf/btf.h>
#include <bpf/libbpf.h>
#include <subcmd/parse-options.h>
@@ -124,6 +126,7 @@ struct object {
struct btf *btf;
struct btf *base_btf;
+ bool distilled_base;
struct {
int fd;
@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
return btf_id;
}
-/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
-#ifndef SHF_COMPRESSED
-#define SHF_COMPRESSED (1 << 11) /* Section with compressed data. */
-#endif
-
-/*
- * The data of compressed section should be aligned to 4
- * (for 32bit) or 8 (for 64 bit) bytes. The binutils ld
- * sets sh_addralign to 1, which makes libelf fail with
- * misaligned section error during the update:
- * FAILED elf_update(WRITE): invalid section alignment
- *
- * While waiting for ld fix, we fix the compressed sections
- * sh_addralign value manualy.
- */
-static int compressed_section_fix(Elf *elf, Elf_Scn *scn, GElf_Shdr *sh)
+static void bswap_32_data(void *data, u32 nr_bytes)
{
- int expected = gelf_getclass(elf) == ELFCLASS32 ? 4 : 8;
+ u32 cnt, i;
+ u32 *ptr;
- if (!(sh->sh_flags & SHF_COMPRESSED))
- return 0;
+ cnt = nr_bytes / sizeof(u32);
+ ptr = data;
- if (sh->sh_addralign == expected)
- return 0;
-
- pr_debug2(" - fixing wrong alignment sh_addralign %u, expected %u\n",
- sh->sh_addralign, expected);
-
- sh->sh_addralign = expected;
-
- if (gelf_update_shdr(scn, sh) == 0) {
- pr_err("FAILED cannot update section header: %s\n",
- elf_errmsg(-1));
- return -1;
- }
- return 0;
+ for (i = 0; i < cnt; i++)
+ ptr[i] = bswap_32(ptr[i]);
}
static int elf_collect(struct object *obj)
@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)
elf_version(EV_CURRENT);
- elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
+ elf = elf_begin(fd, ELF_C_READ_MMAP_PRIVATE, NULL);
if (!elf) {
close(fd);
pr_err("FAILED cannot create ELF descriptor: %s\n",
@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj)
obj->efile.symbols_shndx = idx;
obj->efile.strtabidx = sh.sh_link;
} else if (!strcmp(name, BTF_IDS_SECTION)) {
+ /*
+ * If target endianness differs from host, we need to bswap32
+ * the .BTF_ids section data on load, because .BTF_ids has
+ * Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in
+ * the target endiannes. We repeat this on dump.
+ */
+ if (obj->efile.encoding != ELFDATANATIVE) {
+ pr_debug("bswap_32 .BTF_ids data from target to host endianness\n");
+ bswap_32_data(data->d_buf, data->d_size);
+ }
obj->efile.idlist = data;
obj->efile.idlist_shndx = idx;
obj->efile.idlist_addr = sh.sh_addr;
- } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
- /* If a .BTF.base section is found, do not resolve
- * BTF ids relative to vmlinux; resolve relative
- * to the .BTF.base section instead. btf__parse_split()
- * will take care of this once the base BTF it is
- * passed is NULL.
- */
- obj->base_btf_path = NULL;
}
-
- if (compressed_section_fix(elf, scn, &sh))
- return -1;
}
return 0;
@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)
return 0;
}
+static inline bool is_envvar_set(const char *var_name)
+{
+ const char *value = getenv(var_name);
+
+ return value && value[0] != '\0';
+}
+
static int load_btf(struct object *obj)
{
struct btf *base_btf = NULL, *btf = NULL;
@@ -571,6 +554,20 @@ static int load_btf(struct object *obj)
obj->base_btf = base_btf;
obj->btf = btf;
+ if (obj->base_btf && is_envvar_set("KBUILD_EXTMOD")) {
+ err = btf__distill_base(obj->btf, &base_btf, &btf);
+ if (err) {
+ pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
+ goto out_err;
+ }
+
+ btf__free(obj->btf);
+ btf__free(obj->base_btf);
+ obj->btf = btf;
+ obj->base_btf = base_btf;
+ obj->distilled_base = true;
+ }
+
return 0;
out_err:
@@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)
*/
BUILD_BUG_ON((u32 *)set8->pairs != &set8->pairs[0].id);
qsort(set8->pairs, set8->cnt, sizeof(set8->pairs[0]), cmp_id);
-
- /*
- * When ELF endianness does not match endianness of the
- * host, libelf will do the translation when updating
- * the ELF. This, however, corrupts SET8 flags which are
- * already in the target endianness. So, let's bswap
- * them to the host endianness and libelf will then
- * correctly translate everything.
- */
- if (obj->efile.encoding != ELFDATANATIVE) {
- int i;
-
- set8->flags = bswap_32(set8->flags);
- for (i = 0; i < set8->cnt; i++) {
- set8->pairs[i].flags =
- bswap_32(set8->pairs[i].flags);
- }
- }
break;
case BTF_ID_KIND_SYM:
default:
@@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)
static int symbols_patch(struct object *obj)
{
- off_t err;
-
if (__symbols_patch(obj, &obj->structs) ||
__symbols_patch(obj, &obj->unions) ||
__symbols_patch(obj, &obj->typedefs) ||
@@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)
if (sets_patch(obj))
return -1;
- /* Set type to ensure endian translation occurs. */
- obj->efile.idlist->d_type = ELF_T_WORD;
+ return 0;
+}
- elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
+static int dump_raw_data(const char *out_path, const void *data, u32 size)
+{
+ int fd, ret;
- err = elf_update(obj->efile.elf, ELF_C_WRITE);
- if (err < 0) {
- pr_err("FAILED elf_update(WRITE): %s\n",
- elf_errmsg(-1));
+ fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
+ if (fd < 0) {
+ pr_err("Couldn't open %s for writing\n", out_path);
+ return fd;
+ }
+
+ ret = write(fd, data, size);
+ if (ret < 0 || ret != size) {
+ pr_err("Failed to write data to %s\n", out_path);
+ close(fd);
+ unlink(out_path);
+ return -1;
+ }
+
+ close(fd);
+ pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
+
+ return 0;
+}
+
+static int dump_raw_btf_ids(struct object *obj, const char *out_path)
+{
+ Elf_Data *data = obj->efile.idlist;
+ int fd, err;
+
+ if (!data || !data->d_buf) {
+ pr_debug("%s has no BTF_ids data to dump\n", obj->path);
+ return 0;
+ }
+
+ /*
+ * If target endianness differs from host, we need to bswap32 the
+ * .BTF_ids section data before dumping so that the output is in
+ * target endianness.
+ */
+ if (obj->efile.encoding != ELFDATANATIVE) {
+ pr_debug("bswap_32 .BTF_ids data from host to target endianness\n");
+ bswap_32_data(data->d_buf, data->d_size);
+ }
+
+ err = dump_raw_data(out_path, data->d_buf, data->d_size);
+ if (err)
+ return -1;
+
+ return 0;
+}
+
+static int dump_raw_btf(struct btf *btf, const char *out_path)
+{
+ const void *raw_btf_data;
+ u32 raw_btf_size;
+ int fd, err;
+
+ raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+ if (raw_btf_data == NULL) {
+ pr_err("btf__raw_data() failed\n");
+ return -1;
}
- pr_debug("update %s for %s\n",
- err >= 0 ? "ok" : "failed", obj->path);
- return err < 0 ? -1 : 0;
+ err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
+ if (err)
+ return -1;
+
+ return 0;
}
static const char * const resolve_btfids_usage[] = {
@@ -824,6 +858,7 @@ int main(int argc, const char **argv)
.funcs = RB_ROOT,
.sets = RB_ROOT,
};
+ char out_path[PATH_MAX];
bool fatal_warnings = false;
struct option btfid_options[] = {
OPT_INCR('v', "verbose", &verbose,
@@ -836,7 +871,7 @@ int main(int argc, const char **argv)
"turn warnings into errors"),
OPT_END()
};
- int err = -1;
+ int err = -1, path_len;
argc = parse_options(argc, argv, btfid_options, resolve_btfids_usage,
PARSE_OPT_STOP_AT_NON_OPTION);
@@ -844,6 +879,11 @@ int main(int argc, const char **argv)
usage_with_options(resolve_btfids_usage, btfid_options);
obj.path = argv[0];
+ strcpy(out_path, obj.path);
+ path_len = strlen(out_path);
+
+ if (load_btf(&obj))
+ goto out;
if (elf_collect(&obj))
goto out;
@@ -854,23 +894,37 @@ int main(int argc, const char **argv)
*/
if (obj.efile.idlist_shndx == -1 ||
obj.efile.symbols_shndx == -1) {
- pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
- err = 0;
- goto out;
+ pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
+ goto dump_btf;
}
if (symbols_collect(&obj))
goto out;
- if (load_btf(&obj))
- goto out;
-
if (symbols_resolve(&obj))
goto out;
if (symbols_patch(&obj))
goto out;
+ out_path[path_len] = '\0';
+ strcat(out_path, ".btf_ids");
+ if (dump_raw_btf_ids(&obj, out_path))
+ goto out;
+
+dump_btf:
+ out_path[path_len] = '\0';
+ strcat(out_path, ".btf");
+ if (dump_raw_btf(obj.btf, out_path))
+ goto out;
+
+ if (obj.base_btf && obj.distilled_base) {
+ out_path[path_len] = '\0';
+ strcat(out_path, ".distilled_base.btf");
+ if (dump_raw_btf(obj.base_btf, out_path))
+ goto out;
+ }
+
if (!(fatal_warnings && warnings))
err = 0;
out:
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index bac22265e7ff..ec7e2a7721c7 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
include ../../../scripts/Makefile.include
CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
CURDIR := $(abspath .)
TOOLSDIR := $(abspath ../../..)
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \
$$(call msg,BINARY,,$$@)
$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+ $(Q)if [ -f $$@.btf_ids ]; then \
+ $(OBJCOPY) --update-section .BTF_ids=$$@.btf_ids $$@; \
+ fi
+ $(Q)rm -f $$@.btf_ids $$@.btf
$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
$(OUTPUT)/$(if $2,$2/)bpftool
--
2.52.0On Thu, 2025-11-27 at 10:52 -0800, Ihor Solodrai wrote:
> Currently resolve_btfids updates .BTF_ids section of an ELF file
> in-place, based on the contents of provided BTF, usually within the
> same input file, and optionally a BTF base.
>
> This patch changes resolve_btfids behavior to enable BTF
> transformations as part of its main operation. To achieve this
> in-place ELF write in resolve_btfids is replaced with generation of
> the following binaries:
> * ${1}.btf with .BTF section data
> * ${1}.distilled_base.btf with .BTF.base section data (for
> out-of-tree modules)
> * ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?
>
> The execution of resolve_btfids and consumption of its output is
> orchestrated by scripts/gen-btf.sh introduced in this patch.
>
> The rationale for this approach is that updating ELF in-place with
> libelf API is complicated and bug-prone, especially in the context of
> the kernel build. On the other hand applying objcopy to manipulate ELF
> sections is simpler and more reliable.
Nit: more context needed, as is the statement raises questions but not
answers them.
>
> There are two distinct paths for BTF generation and resolve_btfids
> application in the kernel build: for vmlinux and for kernel modules.
>
> For the vmlinux binary a .BTF section is added in a roundabout way to
> ensure correct linking (details below). The patch doesn't change this
> approach, only the implementation is a little different.
>
> Before this patch it worked like follows:
>
> * pahole consumed .tmp_vmlinux1 [1] and added .BTF section with
> llvm-objcopy [2] to it
> * then everything except the .BTF section was stripped from .tmp_vmlinux1
> into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
> * resolve_btfids was executed later on vmlinux.unstripped [3],
> updating it in-place
>
> After this patch gen-btf.sh implements the following:
>
> * pahole consumes .tmp_vmlinux1 and produces a *detached* file with
> raw BTF data
> * resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
> (potentially modified) .BTF, and .BTF_ids sections data
> * a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
> BTF output of resolve_btfids
> * .BTF_ids data gets embedded into vmlinux.unstripped in
> link-vmlinux.sh by objcopy --update-section
>
> For the kernel modules creating special .bpf.o file is not necessary,
> and so embedding of sections data produced by resolve_btfids is
> straightforward with the objcopy.
>
> With this patch an ELF file becomes effectively read-only within
> resolve_btfids, which allows to delete elf_update() call and satelite
> code (like compressed_section_fix [4]).
>
> Endianness handling of .BTF_ids data is also changed. Previously the
> "flags" part of the section was bswapped in sets_patch() [5], and then
> Elf_Type was modified before elf_update() to signal to libelf that
> bswap may be necessary. With this patch we explicitly bswap entire
> data buffer on load and on dump.
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
> [2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
Nit: these links are moving target, should refer to a commit or a tag.
> [4] https://lore.kernel.org/bpf/20200819092342.259004-1-jolsa@kernel.org/
> [5] https://lore.kernel.org/bpf/cover.1707223196.git.vmalik@redhat.com/
>
> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
> ---
> MAINTAINERS | 1 +
> scripts/Makefile.modfinal | 5 +-
> scripts/gen-btf.sh | 167 ++++++++++++++++++++
> scripts/link-vmlinux.sh | 42 +-----
> tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++----------
> tools/testing/selftests/bpf/Makefile | 5 +
> 6 files changed, 317 insertions(+), 121 deletions(-)
> create mode 100755 scripts/gen-btf.sh
Since resolve_btfids is now responsible for distilled base generation,
does Makefile.btf need modification to remove "--btf_features=distilled_base"?
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 48aabeeed029..5cd34419d952 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c
> F: net/sched/cls_bpf.c
> F: samples/bpf/
> F: scripts/bpf_doc.py
> +F: scripts/gen-btf.sh
> F: scripts/Makefile.btf
> F: scripts/pahole-version.sh
> F: tools/bpf/
> diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> index 542ba462ed3e..3862fdfa1267 100644
> --- a/scripts/Makefile.modfinal
> +++ b/scripts/Makefile.modfinal
> @@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
> cmd_btf_ko = \
> if [ ! -f $(objtree)/vmlinux ]; then \
> printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
> - else \
> - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
> - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \
> + else \
> + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
> fi;
>
> # Same as newer-prereqs, but allows to exclude specified extra dependencies
> diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
> new file mode 100755
> index 000000000000..2dfb7ab289ca
> --- /dev/null
> +++ b/scripts/gen-btf.sh
> @@ -0,0 +1,167 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
> +#
> +# This script generates BTF data for the provided ELF file.
> +#
> +# Kernel BTF generation involves these conceptual steps:
> +# 1. pahole generates BTF from DWARF data
> +# 2. resolve_btfids applies kernel-specific btf2btf
> +# transformations and computes data for .BTF_ids section
> +# 3. the result gets linked/objcopied into the target binary
> +#
> +# How step (3) should be done differs between vmlinux, and
> +# kernel modules, which is the primary reason for the existence
> +# of this script.
> +#
> +# For modules the script expects vmlinux passed in as --btf_base.
> +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
> +# into the input ELF file with objcopy.
> +#
> +# For vmlinux the input file remains unchanged and two files are produced:
> +# - ${1}.btf.o ready for linking into vmlinux
> +# - ${1}.btf_ids with .BTF_ids data blob
> +# This output is consumed by scripts/link-vmlinux.sh
> +
> +set -e
> +
> +usage()
> +{
> + echo "Usage: $0 [--btf_base <file>] <target ELF file>"
> + exit 1
> +}
> +
> +BTF_BASE=""
> +
> +while [ $# -gt 0 ]; do
> + case "$1" in
> + --btf_base)
> + BTF_BASE="$2"
> + shift 2
> + ;;
> + -*)
> + echo "Unknown option: $1" >&2
> + usage
> + ;;
> + *)
> + break
> + ;;
> + esac
> +done
> +
> +if [ $# -ne 1 ]; then
> + usage
> +fi
> +
> +ELF_FILE="$1"
> +shift
> +
> +is_enabled() {
> + grep -q "^$1=y" ${objtree}/include/config/auto.conf
> +}
> +
> +info()
> +{
> + printf " %-7s %s\n" "${1}" "${2}"
> +}
> +
> +case "${KBUILD_VERBOSE}" in
> +*1*)
> + set -x
> + ;;
> +esac
> +
> +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
> + exit 0
> +fi
> +
> +gen_btf_data()
> +{
> + info BTF "${ELF_FILE}"
> + btf1="${ELF_FILE}.btf.1"
> + ${PAHOLE} -J ${PAHOLE_FLAGS} \
> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
> + --btf_encode_detached=${btf1} \
> + "${ELF_FILE}"
> +
> + info BTFIDS "${ELF_FILE}"
> + RESOLVE_BTFIDS_OPTS=""
> + if is_enabled CONFIG_WERROR; then
> + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
> + fi
> + if [ -n "${KBUILD_VERBOSE}" ]; then
> + RESOLVE_BTFIDS_OPTS+=" -v "
> + fi
> + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \
> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
> + --btf ${btf1} "${ELF_FILE}"
> +}
> +
> +gen_btf_o()
> +{
> + local btf_data=${ELF_FILE}.btf.o
> +
> + # Create ${btf_data} which contains just .BTF section but no symbols. Add
> + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
> + # deletes all symbols including __start_BTF and __stop_BTF, which will
> + # be redefined in the linker script.
> + info OBJCOPY "${btf_data}"
> + echo "" | ${CC} -c -x c -o ${btf_data} -
> + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
> + --set-section-flags .BTF=alloc,readonly ${btf_data}
> + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
> +
> + # Change e_type to ET_REL so that it can be used to link final vmlinux.
> + # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
> + if is_enabled CONFIG_CPU_BIG_ENDIAN; then
> + et_rel='\0\1'
> + else
> + et_rel='\1\0'
> + fi
> + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
> +}
> +
> +embed_btf_data()
> +{
> + info OBJCOPY "${ELF_FILE}"
> + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}
> +
> + # a module might not have a .BTF_ids or .BTF.base section
> + local btf_base="${ELF_FILE}.distilled_base.btf"
> + if [ -f "${btf_base}" ]; then
> + ${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE}
> + fi
> + local btf_ids="${ELF_FILE}.btf_ids"
> + if [ -f "${btf_ids}" ]; then
> + ${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE}
> + fi
> +}
> +
> +cleanup()
> +{
> + rm -f "${ELF_FILE}.btf.1"
> + rm -f "${ELF_FILE}.btf"
> + if [ "${BTFGEN_MODE}" = "module" ]; then
> + rm -f "${ELF_FILE}.distilled_base.btf"
> + rm -f "${ELF_FILE}.btf_ids"
> + fi
> +}
> +trap cleanup EXIT
> +
> +BTFGEN_MODE="vmlinux"
> +if [ -n "${BTF_BASE}" ]; then
> + BTFGEN_MODE="module"
> +fi
> +
> +gen_btf_data
> +
> +case "${BTFGEN_MODE}" in
> +vmlinux)
> + gen_btf_o
> + ;;
> +module)
> + embed_btf_data
> + ;;
> +esac
> +
> +exit 0
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index 433849ff7529..5bea8795f96d 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -105,34 +105,6 @@ vmlinux_link()
> ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
> }
>
> -# generate .BTF typeinfo from DWARF debuginfo
> -# ${1} - vmlinux image
> -gen_btf()
> -{
> - local btf_data=${1}.btf.o
> -
> - info BTF "${btf_data}"
> - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
> -
> - # Create ${btf_data} which contains just .BTF section but no symbols. Add
> - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
> - # deletes all symbols including __start_BTF and __stop_BTF, which will
> - # be redefined in the linker script. Add 2>/dev/null to suppress GNU
> - # objcopy warnings: "empty loadable segment detected at ..."
> - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
> - --strip-all ${1} "${btf_data}" 2>/dev/null
> - # Change e_type to ET_REL so that it can be used to link final vmlinux.
> - # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
> - if is_enabled CONFIG_CPU_BIG_ENDIAN; then
> - et_rel='\0\1'
> - else
> - et_rel='\1\0'
> - fi
> - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
> -
> - btf_vmlinux_bin_o=${btf_data}
> -}
> -
> # Create ${2}.o file with all symbols from the ${1} object file
> kallsyms()
> {
> @@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
> fi
>
> btf_vmlinux_bin_o=
> +btfids_vmlinux=
> kallsymso=
> strip_debug=
> generate_map=
> @@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
> fi
>
> if is_enabled CONFIG_DEBUG_INFO_BTF; then
> - if ! gen_btf .tmp_vmlinux1; then
> + if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then
Nit: maybe pass output file names as parameters for get-btf.sh?
> echo >&2 "Failed to generate BTF for vmlinux"
> echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
> exit 1
> fi
> + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
> + btfids_vmlinux=.tmp_vmlinux1.btf_ids
> fi
>
> if is_enabled CONFIG_KALLSYMS; then
> @@ -281,14 +256,9 @@ fi
>
> vmlinux_link "${VMLINUX}"
>
> -# fill in BTF IDs
> if is_enabled CONFIG_DEBUG_INFO_BTF; then
> - info BTFIDS "${VMLINUX}"
> - RESOLVE_BTFIDS_ARGS=""
> - if is_enabled CONFIG_WERROR; then
> - RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
> - fi
> - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
> + info OBJCOPY ${btfids_vmlinux}
> + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
> fi
Nit: Maybe do this in get-btf.sh as for modules?
To avoid checking for CONFIG_DEBUG_INFO_BTF in two places.
>
> mksysmap "${VMLINUX}" System.map
> diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> index c60d303ca6ed..b8df6256e29e 100644
> --- a/tools/bpf/resolve_btfids/main.c
> +++ b/tools/bpf/resolve_btfids/main.c
> @@ -71,9 +71,11 @@
> #include <fcntl.h>
> #include <errno.h>
> #include <linux/btf_ids.h>
> +#include <linux/kallsyms.h>
> #include <linux/rbtree.h>
> #include <linux/zalloc.h>
> #include <linux/err.h>
> +#include <linux/limits.h>
> #include <bpf/btf.h>
> #include <bpf/libbpf.h>
> #include <subcmd/parse-options.h>
> @@ -124,6 +126,7 @@ struct object {
>
> struct btf *btf;
> struct btf *base_btf;
> + bool distilled_base;
>
> struct {
> int fd;
> @@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
> return btf_id;
> }
>
> -/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
> -#ifndef SHF_COMPRESSED
> -#define SHF_COMPRESSED (1 << 11) /* Section with compressed data. */
> -#endif
> -
> -/*
> - * The data of compressed section should be aligned to 4
> - * (for 32bit) or 8 (for 64 bit) bytes. The binutils ld
> - * sets sh_addralign to 1, which makes libelf fail with
> - * misaligned section error during the update:
> - * FAILED elf_update(WRITE): invalid section alignment
> - *
> - * While waiting for ld fix, we fix the compressed sections
> - * sh_addralign value manualy.
> - */
> -static int compressed_section_fix(Elf *elf, Elf_Scn *scn, GElf_Shdr *sh)
> +static void bswap_32_data(void *data, u32 nr_bytes)
> {
> - int expected = gelf_getclass(elf) == ELFCLASS32 ? 4 : 8;
> + u32 cnt, i;
> + u32 *ptr;
>
> - if (!(sh->sh_flags & SHF_COMPRESSED))
> - return 0;
> + cnt = nr_bytes / sizeof(u32);
> + ptr = data;
>
> - if (sh->sh_addralign == expected)
> - return 0;
> -
> - pr_debug2(" - fixing wrong alignment sh_addralign %u, expected %u\n",
> - sh->sh_addralign, expected);
> -
> - sh->sh_addralign = expected;
> -
> - if (gelf_update_shdr(scn, sh) == 0) {
> - pr_err("FAILED cannot update section header: %s\n",
> - elf_errmsg(-1));
> - return -1;
> - }
> - return 0;
> + for (i = 0; i < cnt; i++)
> + ptr[i] = bswap_32(ptr[i]);
> }
>
> static int elf_collect(struct object *obj)
> @@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)
>
> elf_version(EV_CURRENT);
>
> - elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
> + elf = elf_begin(fd, ELF_C_READ_MMAP_PRIVATE, NULL);
> if (!elf) {
> close(fd);
> pr_err("FAILED cannot create ELF descriptor: %s\n",
> @@ -427,21 +404,20 @@ static int elf_collect(struct object *obj)
> obj->efile.symbols_shndx = idx;
> obj->efile.strtabidx = sh.sh_link;
> } else if (!strcmp(name, BTF_IDS_SECTION)) {
> + /*
> + * If target endianness differs from host, we need to bswap32
> + * the .BTF_ids section data on load, because .BTF_ids has
> + * Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in
> + * the target endiannes. We repeat this on dump.
> + */
> + if (obj->efile.encoding != ELFDATANATIVE) {
> + pr_debug("bswap_32 .BTF_ids data from target to host endianness\n");
> + bswap_32_data(data->d_buf, data->d_size);
> + }
> obj->efile.idlist = data;
> obj->efile.idlist_shndx = idx;
> obj->efile.idlist_addr = sh.sh_addr;
> - } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
> - /* If a .BTF.base section is found, do not resolve
> - * BTF ids relative to vmlinux; resolve relative
> - * to the .BTF.base section instead. btf__parse_split()
> - * will take care of this once the base BTF it is
> - * passed is NULL.
> - */
> - obj->base_btf_path = NULL;
> }
> -
> - if (compressed_section_fix(elf, scn, &sh))
> - return -1;
> }
>
> return 0;
> @@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)
> return 0;
> }
>
> +static inline bool is_envvar_set(const char *var_name)
> +{
> + const char *value = getenv(var_name);
> +
> + return value && value[0] != '\0';
> +}
> +
> static int load_btf(struct object *obj)
> {
> struct btf *base_btf = NULL, *btf = NULL;
> @@ -571,6 +554,20 @@ static int load_btf(struct object *obj)
> obj->base_btf = base_btf;
> obj->btf = btf;
>
> + if (obj->base_btf && is_envvar_set("KBUILD_EXTMOD")) {
This is a bit ugly, maybe use a dedicated parameter instead of
checking environment variable?
> + err = btf__distill_base(obj->btf, &base_btf, &btf);
> + if (err) {
> + pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
> + goto out_err;
> + }
> +
> + btf__free(obj->btf);
> + btf__free(obj->base_btf);
> + obj->btf = btf;
> + obj->base_btf = base_btf;
> + obj->distilled_base = true;
> + }
> +
> return 0;
>
> out_err:
> @@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)
> */
> BUILD_BUG_ON((u32 *)set8->pairs != &set8->pairs[0].id);
> qsort(set8->pairs, set8->cnt, sizeof(set8->pairs[0]), cmp_id);
> -
> - /*
> - * When ELF endianness does not match endianness of the
> - * host, libelf will do the translation when updating
> - * the ELF. This, however, corrupts SET8 flags which are
> - * already in the target endianness. So, let's bswap
> - * them to the host endianness and libelf will then
> - * correctly translate everything.
> - */
> - if (obj->efile.encoding != ELFDATANATIVE) {
> - int i;
> -
> - set8->flags = bswap_32(set8->flags);
> - for (i = 0; i < set8->cnt; i++) {
> - set8->pairs[i].flags =
> - bswap_32(set8->pairs[i].flags);
> - }
> - }
> break;
> case BTF_ID_KIND_SYM:
> default:
> @@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)
>
> static int symbols_patch(struct object *obj)
> {
> - off_t err;
> -
> if (__symbols_patch(obj, &obj->structs) ||
> __symbols_patch(obj, &obj->unions) ||
> __symbols_patch(obj, &obj->typedefs) ||
> @@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)
> if (sets_patch(obj))
> return -1;
>
> - /* Set type to ensure endian translation occurs. */
> - obj->efile.idlist->d_type = ELF_T_WORD;
> + return 0;
> +}
>
> - elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
> +static int dump_raw_data(const char *out_path, const void *data, u32 size)
> +{
> + int fd, ret;
>
> - err = elf_update(obj->efile.elf, ELF_C_WRITE);
> - if (err < 0) {
> - pr_err("FAILED elf_update(WRITE): %s\n",
> - elf_errmsg(-1));
> + fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
> + if (fd < 0) {
> + pr_err("Couldn't open %s for writing\n", out_path);
> + return fd;
> + }
> +
> + ret = write(fd, data, size);
> + if (ret < 0 || ret != size) {
> + pr_err("Failed to write data to %s\n", out_path);
> + close(fd);
> + unlink(out_path);
> + return -1;
> + }
> +
> + close(fd);
> + pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
> +
> + return 0;
> +}
> +
> +static int dump_raw_btf_ids(struct object *obj, const char *out_path)
> +{
> + Elf_Data *data = obj->efile.idlist;
> + int fd, err;
> +
> + if (!data || !data->d_buf) {
> + pr_debug("%s has no BTF_ids data to dump\n", obj->path);
> + return 0;
> + }
> +
> + /*
> + * If target endianness differs from host, we need to bswap32 the
> + * .BTF_ids section data before dumping so that the output is in
> + * target endianness.
> + */
> + if (obj->efile.encoding != ELFDATANATIVE) {
> + pr_debug("bswap_32 .BTF_ids data from host to target endianness\n");
> + bswap_32_data(data->d_buf, data->d_size);
> + }
> +
> + err = dump_raw_data(out_path, data->d_buf, data->d_size);
> + if (err)
> + return -1;
> +
> + return 0;
> +}
> +
> +static int dump_raw_btf(struct btf *btf, const char *out_path)
> +{
> + const void *raw_btf_data;
> + u32 raw_btf_size;
> + int fd, err;
> +
> + raw_btf_data = btf__raw_data(btf, &raw_btf_size);
> + if (raw_btf_data == NULL) {
> + pr_err("btf__raw_data() failed\n");
> + return -1;
> }
>
> - pr_debug("update %s for %s\n",
> - err >= 0 ? "ok" : "failed", obj->path);
> - return err < 0 ? -1 : 0;
> + err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
> + if (err)
> + return -1;
> +
> + return 0;
> }
>
> static const char * const resolve_btfids_usage[] = {
> @@ -824,6 +858,7 @@ int main(int argc, const char **argv)
> .funcs = RB_ROOT,
> .sets = RB_ROOT,
> };
> + char out_path[PATH_MAX];
> bool fatal_warnings = false;
> struct option btfid_options[] = {
> OPT_INCR('v', "verbose", &verbose,
> @@ -836,7 +871,7 @@ int main(int argc, const char **argv)
> "turn warnings into errors"),
> OPT_END()
> };
> - int err = -1;
> + int err = -1, path_len;
>
> argc = parse_options(argc, argv, btfid_options, resolve_btfids_usage,
> PARSE_OPT_STOP_AT_NON_OPTION);
> @@ -844,6 +879,11 @@ int main(int argc, const char **argv)
> usage_with_options(resolve_btfids_usage, btfid_options);
>
> obj.path = argv[0];
> + strcpy(out_path, obj.path);
> + path_len = strlen(out_path);
> +
> + if (load_btf(&obj))
> + goto out;
>
> if (elf_collect(&obj))
> goto out;
> @@ -854,23 +894,37 @@ int main(int argc, const char **argv)
> */
> if (obj.efile.idlist_shndx == -1 ||
> obj.efile.symbols_shndx == -1) {
> - pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
> - err = 0;
> - goto out;
> + pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
> + goto dump_btf;
> }
>
> if (symbols_collect(&obj))
> goto out;
>
> - if (load_btf(&obj))
> - goto out;
> -
> if (symbols_resolve(&obj))
> goto out;
>
> if (symbols_patch(&obj))
> goto out;
>
> + out_path[path_len] = '\0';
> + strcat(out_path, ".btf_ids");
Nit: here and below use snprintf() to check for 'out_path' overflow.
> + if (dump_raw_btf_ids(&obj, out_path))
> + goto out;
> +
> +dump_btf:
> + out_path[path_len] = '\0';
> + strcat(out_path, ".btf");
> + if (dump_raw_btf(obj.btf, out_path))
> + goto out;
> +
> + if (obj.base_btf && obj.distilled_base) {
> + out_path[path_len] = '\0';
> + strcat(out_path, ".distilled_base.btf");
> + if (dump_raw_btf(obj.base_btf, out_path))
> + goto out;
> + }
> +
> if (!(fatal_warnings && warnings))
> err = 0;
> out:
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index bac22265e7ff..ec7e2a7721c7 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
> include ../../../scripts/Makefile.include
>
> CXX ?= $(CROSS_COMPILE)g++
> +OBJCOPY ?= $(CROSS_COMPILE)objcopy
>
> CURDIR := $(abspath .)
> TOOLSDIR := $(abspath ../../..)
> @@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \
> $$(call msg,BINARY,,$$@)
> $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
> $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
> + $(Q)if [ -f $$@.btf_ids ]; then \
> + $(OBJCOPY) --update-section .BTF_ids=$$@.btf_ids $$@; \
> + fi
> + $(Q)rm -f $$@.btf_ids $$@.btf
> $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
> $(OUTPUT)/$(if $2,$2/)bpftool
>
On Fri, Nov 28, 2025 at 2:53 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote:
>
> Currently resolve_btfids updates .BTF_ids section of an ELF file
> in-place, based on the contents of provided BTF, usually within the
> same input file, and optionally a BTF base.
>
> This patch changes resolve_btfids behavior to enable BTF
> transformations as part of its main operation. To achieve this
> in-place ELF write in resolve_btfids is replaced with generation of
> the following binaries:
> * ${1}.btf with .BTF section data
> * ${1}.distilled_base.btf with .BTF.base section data (for
> out-of-tree modules)
> * ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
>
> The execution of resolve_btfids and consumption of its output is
> orchestrated by scripts/gen-btf.sh introduced in this patch.
>
> The rationale for this approach is that updating ELF in-place with
> libelf API is complicated and bug-prone, especially in the context of
> the kernel build. On the other hand applying objcopy to manipulate ELF
> sections is simpler and more reliable.
>
> There are two distinct paths for BTF generation and resolve_btfids
> application in the kernel build: for vmlinux and for kernel modules.
>
> For the vmlinux binary a .BTF section is added in a roundabout way to
> ensure correct linking (details below). The patch doesn't change this
> approach, only the implementation is a little different.
>
> Before this patch it worked like follows:
>
> * pahole consumed .tmp_vmlinux1 [1] and added .BTF section with
> llvm-objcopy [2] to it
> * then everything except the .BTF section was stripped from .tmp_vmlinux1
> into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
> * resolve_btfids was executed later on vmlinux.unstripped [3],
> updating it in-place
>
> After this patch gen-btf.sh implements the following:
>
> * pahole consumes .tmp_vmlinux1 and produces a *detached* file with
> raw BTF data
> * resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
> (potentially modified) .BTF, and .BTF_ids sections data
> * a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
> BTF output of resolve_btfids
> * .BTF_ids data gets embedded into vmlinux.unstripped in
> link-vmlinux.sh by objcopy --update-section
>
> For the kernel modules creating special .bpf.o file is not necessary,
> and so embedding of sections data produced by resolve_btfids is
> straightforward with the objcopy.
>
> With this patch an ELF file becomes effectively read-only within
> resolve_btfids, which allows to delete elf_update() call and satelite
> code (like compressed_section_fix [4]).
>
> Endianness handling of .BTF_ids data is also changed. Previously the
> "flags" part of the section was bswapped in sets_patch() [5], and then
> Elf_Type was modified before elf_update() to signal to libelf that
> bswap may be necessary. With this patch we explicitly bswap entire
> data buffer on load and on dump.
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
> [2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
> [4] https://lore.kernel.org/bpf/20200819092342.259004-1-jolsa@kernel.org/
> [5] https://lore.kernel.org/bpf/cover.1707223196.git.vmalik@redhat.com/
>
> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
> ---
> MAINTAINERS | 1 +
> scripts/Makefile.modfinal | 5 +-
> scripts/gen-btf.sh | 167 ++++++++++++++++++++
> scripts/link-vmlinux.sh | 42 +-----
> tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++----------
> tools/testing/selftests/bpf/Makefile | 5 +
> 6 files changed, 317 insertions(+), 121 deletions(-)
> create mode 100755 scripts/gen-btf.sh
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 48aabeeed029..5cd34419d952 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c
> F: net/sched/cls_bpf.c
> F: samples/bpf/
> F: scripts/bpf_doc.py
> +F: scripts/gen-btf.sh
> F: scripts/Makefile.btf
> F: scripts/pahole-version.sh
> F: tools/bpf/
> diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> index 542ba462ed3e..3862fdfa1267 100644
> --- a/scripts/Makefile.modfinal
> +++ b/scripts/Makefile.modfinal
> @@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
> cmd_btf_ko = \
> if [ ! -f $(objtree)/vmlinux ]; then \
> printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
> - else \
> - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
> - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \
> + else \
> + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
> fi;
>
> # Same as newer-prereqs, but allows to exclude specified extra dependencies
> diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
> new file mode 100755
> index 000000000000..2dfb7ab289ca
> --- /dev/null
> +++ b/scripts/gen-btf.sh
> @@ -0,0 +1,167 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
> +#
> +# This script generates BTF data for the provided ELF file.
> +#
> +# Kernel BTF generation involves these conceptual steps:
> +# 1. pahole generates BTF from DWARF data
> +# 2. resolve_btfids applies kernel-specific btf2btf
> +# transformations and computes data for .BTF_ids section
> +# 3. the result gets linked/objcopied into the target binary
> +#
> +# How step (3) should be done differs between vmlinux, and
> +# kernel modules, which is the primary reason for the existence
> +# of this script.
> +#
> +# For modules the script expects vmlinux passed in as --btf_base.
> +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
> +# into the input ELF file with objcopy.
> +#
> +# For vmlinux the input file remains unchanged and two files are produced:
> +# - ${1}.btf.o ready for linking into vmlinux
> +# - ${1}.btf_ids with .BTF_ids data blob
> +# This output is consumed by scripts/link-vmlinux.sh
> +
> +set -e
> +
> +usage()
> +{
> + echo "Usage: $0 [--btf_base <file>] <target ELF file>"
> + exit 1
> +}
> +
> +BTF_BASE=""
> +
> +while [ $# -gt 0 ]; do
> + case "$1" in
> + --btf_base)
> + BTF_BASE="$2"
> + shift 2
> + ;;
> + -*)
> + echo "Unknown option: $1" >&2
> + usage
> + ;;
> + *)
> + break
> + ;;
> + esac
> +done
> +
> +if [ $# -ne 1 ]; then
> + usage
> +fi
> +
> +ELF_FILE="$1"
> +shift
> +
> +is_enabled() {
> + grep -q "^$1=y" ${objtree}/include/config/auto.conf
> +}
> +
> +info()
> +{
> + printf " %-7s %s\n" "${1}" "${2}"
> +}
> +
> +case "${KBUILD_VERBOSE}" in
> +*1*)
> + set -x
> + ;;
> +esac
> +
> +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
> + exit 0
> +fi
> +
> +gen_btf_data()
> +{
> + info BTF "${ELF_FILE}"
> + btf1="${ELF_FILE}.btf.1"
> + ${PAHOLE} -J ${PAHOLE_FLAGS} \
> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
> + --btf_encode_detached=${btf1} \
> + "${ELF_FILE}"
> +
> + info BTFIDS "${ELF_FILE}"
> + RESOLVE_BTFIDS_OPTS=""
> + if is_enabled CONFIG_WERROR; then
> + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
> + fi
> + if [ -n "${KBUILD_VERBOSE}" ]; then
> + RESOLVE_BTFIDS_OPTS+=" -v "
> + fi
> + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \
> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
> + --btf ${btf1} "${ELF_FILE}"
> +}
> +
> +gen_btf_o()
> +{
> + local btf_data=${ELF_FILE}.btf.o
> +
> + # Create ${btf_data} which contains just .BTF section but no symbols. Add
> + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
> + # deletes all symbols including __start_BTF and __stop_BTF, which will
> + # be redefined in the linker script.
> + info OBJCOPY "${btf_data}"
> + echo "" | ${CC} -c -x c -o ${btf_data} -
> + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
> + --set-section-flags .BTF=alloc,readonly ${btf_data}
> + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
> +
> + # Change e_type to ET_REL so that it can be used to link final vmlinux.
> + # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
> + if is_enabled CONFIG_CPU_BIG_ENDIAN; then
> + et_rel='\0\1'
> + else
> + et_rel='\1\0'
> + fi
> + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
> +}
> +
> +embed_btf_data()
> +{
> + info OBJCOPY "${ELF_FILE}"
> + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}
> +
> + # a module might not have a .BTF_ids or .BTF.base section
> + local btf_base="${ELF_FILE}.distilled_base.btf"
> + if [ -f "${btf_base}" ]; then
> + ${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE}
> + fi
> + local btf_ids="${ELF_FILE}.btf_ids"
> + if [ -f "${btf_ids}" ]; then
> + ${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE}
> + fi
> +}
> +
> +cleanup()
> +{
> + rm -f "${ELF_FILE}.btf.1"
> + rm -f "${ELF_FILE}.btf"
> + if [ "${BTFGEN_MODE}" = "module" ]; then
> + rm -f "${ELF_FILE}.distilled_base.btf"
> + rm -f "${ELF_FILE}.btf_ids"
> + fi
> +}
> +trap cleanup EXIT
> +
> +BTFGEN_MODE="vmlinux"
> +if [ -n "${BTF_BASE}" ]; then
> + BTFGEN_MODE="module"
> +fi
> +
> +gen_btf_data
> +
> +case "${BTFGEN_MODE}" in
> +vmlinux)
> + gen_btf_o
> + ;;
> +module)
> + embed_btf_data
> + ;;
> +esac
> +
> +exit 0
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index 433849ff7529..5bea8795f96d 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -105,34 +105,6 @@ vmlinux_link()
> ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
> }
>
> -# generate .BTF typeinfo from DWARF debuginfo
> -# ${1} - vmlinux image
> -gen_btf()
> -{
> - local btf_data=${1}.btf.o
> -
> - info BTF "${btf_data}"
> - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
> -
> - # Create ${btf_data} which contains just .BTF section but no symbols. Add
> - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
> - # deletes all symbols including __start_BTF and __stop_BTF, which will
> - # be redefined in the linker script. Add 2>/dev/null to suppress GNU
> - # objcopy warnings: "empty loadable segment detected at ..."
> - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
> - --strip-all ${1} "${btf_data}" 2>/dev/null
> - # Change e_type to ET_REL so that it can be used to link final vmlinux.
> - # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
> - if is_enabled CONFIG_CPU_BIG_ENDIAN; then
> - et_rel='\0\1'
> - else
> - et_rel='\1\0'
> - fi
> - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
> -
> - btf_vmlinux_bin_o=${btf_data}
> -}
> -
> # Create ${2}.o file with all symbols from the ${1} object file
> kallsyms()
> {
> @@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
> fi
>
> btf_vmlinux_bin_o=
> +btfids_vmlinux=
> kallsymso=
> strip_debug=
> generate_map=
> @@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
> fi
>
> if is_enabled CONFIG_DEBUG_INFO_BTF; then
> - if ! gen_btf .tmp_vmlinux1; then
> + if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then
> echo >&2 "Failed to generate BTF for vmlinux"
> echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
> exit 1
> fi
> + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
> + btfids_vmlinux=.tmp_vmlinux1.btf_ids
> fi
>
> if is_enabled CONFIG_KALLSYMS; then
> @@ -281,14 +256,9 @@ fi
>
> vmlinux_link "${VMLINUX}"
>
> -# fill in BTF IDs
> if is_enabled CONFIG_DEBUG_INFO_BTF; then
> - info BTFIDS "${VMLINUX}"
> - RESOLVE_BTFIDS_ARGS=""
> - if is_enabled CONFIG_WERROR; then
> - RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
> - fi
> - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
> + info OBJCOPY ${btfids_vmlinux}
> + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
> fi
>
> mksysmap "${VMLINUX}" System.map
> diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> index c60d303ca6ed..b8df6256e29e 100644
> --- a/tools/bpf/resolve_btfids/main.c
> +++ b/tools/bpf/resolve_btfids/main.c
> @@ -71,9 +71,11 @@
> #include <fcntl.h>
> #include <errno.h>
> #include <linux/btf_ids.h>
> +#include <linux/kallsyms.h>
> #include <linux/rbtree.h>
> #include <linux/zalloc.h>
> #include <linux/err.h>
> +#include <linux/limits.h>
> #include <bpf/btf.h>
> #include <bpf/libbpf.h>
> #include <subcmd/parse-options.h>
> @@ -124,6 +126,7 @@ struct object {
>
> struct btf *btf;
> struct btf *base_btf;
> + bool distilled_base;
>
> struct {
> int fd;
> @@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
> return btf_id;
> }
>
> -/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
> -#ifndef SHF_COMPRESSED
> -#define SHF_COMPRESSED (1 << 11) /* Section with compressed data. */
> -#endif
> -
> -/*
> - * The data of compressed section should be aligned to 4
> - * (for 32bit) or 8 (for 64 bit) bytes. The binutils ld
> - * sets sh_addralign to 1, which makes libelf fail with
> - * misaligned section error during the update:
> - * FAILED elf_update(WRITE): invalid section alignment
> - *
> - * While waiting for ld fix, we fix the compressed sections
> - * sh_addralign value manualy.
> - */
> -static int compressed_section_fix(Elf *elf, Elf_Scn *scn, GElf_Shdr *sh)
> +static void bswap_32_data(void *data, u32 nr_bytes)
> {
> - int expected = gelf_getclass(elf) == ELFCLASS32 ? 4 : 8;
> + u32 cnt, i;
> + u32 *ptr;
>
> - if (!(sh->sh_flags & SHF_COMPRESSED))
> - return 0;
> + cnt = nr_bytes / sizeof(u32);
> + ptr = data;
>
> - if (sh->sh_addralign == expected)
> - return 0;
> -
> - pr_debug2(" - fixing wrong alignment sh_addralign %u, expected %u\n",
> - sh->sh_addralign, expected);
> -
> - sh->sh_addralign = expected;
> -
> - if (gelf_update_shdr(scn, sh) == 0) {
> - pr_err("FAILED cannot update section header: %s\n",
> - elf_errmsg(-1));
> - return -1;
> - }
> - return 0;
> + for (i = 0; i < cnt; i++)
> + ptr[i] = bswap_32(ptr[i]);
> }
>
> static int elf_collect(struct object *obj)
> @@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)
>
> elf_version(EV_CURRENT);
>
> - elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
> + elf = elf_begin(fd, ELF_C_READ_MMAP_PRIVATE, NULL);
> if (!elf) {
> close(fd);
> pr_err("FAILED cannot create ELF descriptor: %s\n",
> @@ -427,21 +404,20 @@ static int elf_collect(struct object *obj)
> obj->efile.symbols_shndx = idx;
> obj->efile.strtabidx = sh.sh_link;
> } else if (!strcmp(name, BTF_IDS_SECTION)) {
> + /*
> + * If target endianness differs from host, we need to bswap32
> + * the .BTF_ids section data on load, because .BTF_ids has
> + * Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in
> + * the target endiannes. We repeat this on dump.
> + */
> + if (obj->efile.encoding != ELFDATANATIVE) {
> + pr_debug("bswap_32 .BTF_ids data from target to host endianness\n");
> + bswap_32_data(data->d_buf, data->d_size);
> + }
> obj->efile.idlist = data;
> obj->efile.idlist_shndx = idx;
> obj->efile.idlist_addr = sh.sh_addr;
> - } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
> - /* If a .BTF.base section is found, do not resolve
> - * BTF ids relative to vmlinux; resolve relative
> - * to the .BTF.base section instead. btf__parse_split()
> - * will take care of this once the base BTF it is
> - * passed is NULL.
> - */
> - obj->base_btf_path = NULL;
> }
> -
> - if (compressed_section_fix(elf, scn, &sh))
> - return -1;
> }
>
> return 0;
> @@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)
> return 0;
> }
>
> +static inline bool is_envvar_set(const char *var_name)
> +{
> + const char *value = getenv(var_name);
> +
> + return value && value[0] != '\0';
> +}
> +
> static int load_btf(struct object *obj)
> {
> struct btf *base_btf = NULL, *btf = NULL;
> @@ -571,6 +554,20 @@ static int load_btf(struct object *obj)
> obj->base_btf = base_btf;
> obj->btf = btf;
>
> + if (obj->base_btf && is_envvar_set("KBUILD_EXTMOD")) {
> + err = btf__distill_base(obj->btf, &base_btf, &btf);
> + if (err) {
> + pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
> + goto out_err;
> + }
> +
> + btf__free(obj->btf);
> + btf__free(obj->base_btf);
> + obj->btf = btf;
> + obj->base_btf = base_btf;
> + obj->distilled_base = true;
> + }
> +
> return 0;
>
> out_err:
> @@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)
> */
> BUILD_BUG_ON((u32 *)set8->pairs != &set8->pairs[0].id);
> qsort(set8->pairs, set8->cnt, sizeof(set8->pairs[0]), cmp_id);
> -
> - /*
> - * When ELF endianness does not match endianness of the
> - * host, libelf will do the translation when updating
> - * the ELF. This, however, corrupts SET8 flags which are
> - * already in the target endianness. So, let's bswap
> - * them to the host endianness and libelf will then
> - * correctly translate everything.
> - */
> - if (obj->efile.encoding != ELFDATANATIVE) {
> - int i;
> -
> - set8->flags = bswap_32(set8->flags);
> - for (i = 0; i < set8->cnt; i++) {
> - set8->pairs[i].flags =
> - bswap_32(set8->pairs[i].flags);
> - }
> - }
> break;
> case BTF_ID_KIND_SYM:
> default:
> @@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)
>
> static int symbols_patch(struct object *obj)
> {
> - off_t err;
> -
> if (__symbols_patch(obj, &obj->structs) ||
> __symbols_patch(obj, &obj->unions) ||
> __symbols_patch(obj, &obj->typedefs) ||
> @@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)
> if (sets_patch(obj))
> return -1;
>
> - /* Set type to ensure endian translation occurs. */
> - obj->efile.idlist->d_type = ELF_T_WORD;
> + return 0;
> +}
>
> - elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
> +static int dump_raw_data(const char *out_path, const void *data, u32 size)
> +{
> + int fd, ret;
>
> - err = elf_update(obj->efile.elf, ELF_C_WRITE);
> - if (err < 0) {
> - pr_err("FAILED elf_update(WRITE): %s\n",
> - elf_errmsg(-1));
> + fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
> + if (fd < 0) {
> + pr_err("Couldn't open %s for writing\n", out_path);
> + return fd;
> + }
> +
> + ret = write(fd, data, size);
> + if (ret < 0 || ret != size) {
> + pr_err("Failed to write data to %s\n", out_path);
> + close(fd);
> + unlink(out_path);
> + return -1;
> + }
> +
> + close(fd);
> + pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
> +
> + return 0;
> +}
> +
> +static int dump_raw_btf_ids(struct object *obj, const char *out_path)
> +{
> + Elf_Data *data = obj->efile.idlist;
> + int fd, err;
> +
> + if (!data || !data->d_buf) {
> + pr_debug("%s has no BTF_ids data to dump\n", obj->path);
> + return 0;
> + }
> +
> + /*
> + * If target endianness differs from host, we need to bswap32 the
> + * .BTF_ids section data before dumping so that the output is in
> + * target endianness.
> + */
> + if (obj->efile.encoding != ELFDATANATIVE) {
> + pr_debug("bswap_32 .BTF_ids data from host to target endianness\n");
> + bswap_32_data(data->d_buf, data->d_size);
> + }
> +
> + err = dump_raw_data(out_path, data->d_buf, data->d_size);
> + if (err)
> + return -1;
> +
> + return 0;
> +}
> +
> +static int dump_raw_btf(struct btf *btf, const char *out_path)
> +{
> + const void *raw_btf_data;
> + u32 raw_btf_size;
> + int fd, err;
> +
> + raw_btf_data = btf__raw_data(btf, &raw_btf_size);
> + if (raw_btf_data == NULL) {
> + pr_err("btf__raw_data() failed\n");
> + return -1;
> }
>
> - pr_debug("update %s for %s\n",
> - err >= 0 ? "ok" : "failed", obj->path);
> - return err < 0 ? -1 : 0;
> + err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
> + if (err)
> + return -1;
> +
> + return 0;
> }
>
> static const char * const resolve_btfids_usage[] = {
> @@ -824,6 +858,7 @@ int main(int argc, const char **argv)
> .funcs = RB_ROOT,
> .sets = RB_ROOT,
> };
> + char out_path[PATH_MAX];
> bool fatal_warnings = false;
> struct option btfid_options[] = {
> OPT_INCR('v', "verbose", &verbose,
> @@ -836,7 +871,7 @@ int main(int argc, const char **argv)
> "turn warnings into errors"),
> OPT_END()
> };
> - int err = -1;
> + int err = -1, path_len;
>
> argc = parse_options(argc, argv, btfid_options, resolve_btfids_usage,
> PARSE_OPT_STOP_AT_NON_OPTION);
> @@ -844,6 +879,11 @@ int main(int argc, const char **argv)
> usage_with_options(resolve_btfids_usage, btfid_options);
>
> obj.path = argv[0];
> + strcpy(out_path, obj.path);
> + path_len = strlen(out_path);
> +
> + if (load_btf(&obj))
> + goto out;
>
> if (elf_collect(&obj))
> goto out;
> @@ -854,23 +894,37 @@ int main(int argc, const char **argv)
> */
> if (obj.efile.idlist_shndx == -1 ||
> obj.efile.symbols_shndx == -1) {
> - pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
> - err = 0;
> - goto out;
> + pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
> + goto dump_btf;
> }
>
> if (symbols_collect(&obj))
> goto out;
>
> - if (load_btf(&obj))
> - goto out;
> -
> if (symbols_resolve(&obj))
> goto out;
>
> if (symbols_patch(&obj))
> goto out;
>
> + out_path[path_len] = '\0';
> + strcat(out_path, ".btf_ids");
> + if (dump_raw_btf_ids(&obj, out_path))
> + goto out;
> +
> +dump_btf:
> + out_path[path_len] = '\0';
> + strcat(out_path, ".btf");
> + if (dump_raw_btf(obj.btf, out_path))
> + goto out;
> +
> + if (obj.base_btf && obj.distilled_base) {
> + out_path[path_len] = '\0';
> + strcat(out_path, ".distilled_base.btf");
> + if (dump_raw_btf(obj.base_btf, out_path))
> + goto out;
> + }
> +
> if (!(fatal_warnings && warnings))
> err = 0;
> out:
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index bac22265e7ff..ec7e2a7721c7 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
> include ../../../scripts/Makefile.include
>
> CXX ?= $(CROSS_COMPILE)g++
> +OBJCOPY ?= $(CROSS_COMPILE)objcopy
>
> CURDIR := $(abspath .)
> TOOLSDIR := $(abspath ../../..)
> @@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \
> $$(call msg,BINARY,,$$@)
> $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
> $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
> + $(Q)if [ -f $$@.btf_ids ]; then \
> + $(OBJCOPY) --update-section .BTF_ids=$$@.btf_ids $$@; \
I encountered a resolve_btfids self-test failure when enabling the
BTF sorting feature, with the following error output:
All error logs:
resolve_symbols:PASS:resolve 0 nsec
test_resolve_btfids:PASS:id_check 0 nsec
test_resolve_btfids:PASS:id_check 0 nsec
test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL
The root cause is that prog_tests/resolve_btfids.c retrieves type IDs
from btf_data.bpf.o and compares them against the IDs in test_progs.
However, while the IDs in test_progs are sorted, those in btf_data.bpf.o
remain in their original unsorted state, causing the validation to fail.
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including
the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf
instead. However, I discovered that test_progs.btf is deleted in the
subsequent code section.
What do you think of it?
Thanks,
Donglin
> + fi
> + $(Q)rm -f $$@.btf_ids $$@.btf
> $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
> $(OUTPUT)/$(if $2,$2/)bpftool
>
> --
> 2.52.0
>
On 11/27/25 7:20 PM, Donglin Peng wrote: > On Fri, Nov 28, 2025 at 2:53 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote: >> >> [...] >> >> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile >> index bac22265e7ff..ec7e2a7721c7 100644 >> --- a/tools/testing/selftests/bpf/Makefile >> +++ b/tools/testing/selftests/bpf/Makefile >> @@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch >> include ../../../scripts/Makefile.include >> >> CXX ?= $(CROSS_COMPILE)g++ >> +OBJCOPY ?= $(CROSS_COMPILE)objcopy >> >> CURDIR := $(abspath .) >> TOOLSDIR := $(abspath ../../..) >> @@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ >> $$(call msg,BINARY,,$$@) >> $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ >> $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ >> + $(Q)if [ -f $$@.btf_ids ]; then \ >> + $(OBJCOPY) --update-section .BTF_ids=$$@.btf_ids $$@; \ > > I encountered a resolve_btfids self-test failure when enabling the > BTF sorting feature, with the following error output: > > All error logs: > resolve_symbols:PASS:resolve 0 nsec > test_resolve_btfids:PASS:id_check 0 nsec > test_resolve_btfids:PASS:id_check 0 nsec > test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5) > #369 resolve_btfids:FAIL > > The root cause is that prog_tests/resolve_btfids.c retrieves type IDs > from btf_data.bpf.o and compares them against the IDs in test_progs. > However, while the IDs in test_progs are sorted, those in btf_data.bpf.o > remain in their original unsorted state, causing the validation to fail. > > This presents two potential solutions: > 1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including > the .BTF and .BTF.ext sections > 2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf > instead. However, I discovered that test_progs.btf is deleted in the > subsequent code section. > > What do you think of it? Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF. As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it. resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am). And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros. By the way, the test was written more than 5y ago [1], so it might be outdated too. I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail. But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO. I'll try to think of something. [1] https://lore.kernel.org/bpf/20200703095111.3268961-10-jolsa@kernel.org/ > > Thanks, > Donglin > >> + fi >> + $(Q)rm -f $$@.btf_ids $$@.btf >> $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ >> $(OUTPUT)/$(if $2,$2/)bpftool >> >> -- >> 2.52.0 >>
On 11/27/25 9:52 PM, Ihor Solodrai wrote:
> On 11/27/25 7:20 PM, Donglin Peng wrote:
>> On Fri, Nov 28, 2025 at 2:53 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote:
>>>
>>> [...]
>>>
>>> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
>>> index bac22265e7ff..ec7e2a7721c7 100644
>>> --- a/tools/testing/selftests/bpf/Makefile
>>> +++ b/tools/testing/selftests/bpf/Makefile
>>> @@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
>>> include ../../../scripts/Makefile.include
>>>
>>> CXX ?= $(CROSS_COMPILE)g++
>>> +OBJCOPY ?= $(CROSS_COMPILE)objcopy
>>>
>>> CURDIR := $(abspath .)
>>> TOOLSDIR := $(abspath ../../..)
>>> @@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \
>>> $$(call msg,BINARY,,$$@)
>>> $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
>>> $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
>>> + $(Q)if [ -f $$@.btf_ids ]; then \
>>> + $(OBJCOPY) --update-section .BTF_ids=$$@.btf_ids $$@; \
>>
>> I encountered a resolve_btfids self-test failure when enabling the
>> BTF sorting feature, with the following error output:
>>
>> All error logs:
>> resolve_symbols:PASS:resolve 0 nsec
>> test_resolve_btfids:PASS:id_check 0 nsec
>> test_resolve_btfids:PASS:id_check 0 nsec
>> test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
>> #369 resolve_btfids:FAIL
>>
>> The root cause is that prog_tests/resolve_btfids.c retrieves type IDs
>> from btf_data.bpf.o and compares them against the IDs in test_progs.
>> However, while the IDs in test_progs are sorted, those in btf_data.bpf.o
>> remain in their original unsorted state, causing the validation to fail.
>>
>> This presents two potential solutions:
>> 1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including
>> the .BTF and .BTF.ext sections
>> 2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf
>> instead. However, I discovered that test_progs.btf is deleted in the
>> subsequent code section.
>>
>> What do you think of it?
>
> Within resolve_btfids it's clear that we have to update (sort in this
> case) BTF first, and then resolve the ids based on the changed BTF.
>
> As for the test, we should probably change it to become closer to an
> actual resolve_btfids use-case. Maybe even replace or remove it.
>
> resolve_btfids operates on BTF generated by pahole for
> kernel/module. And the .BTF_ids section makes sense only in kernel
> space AFAIU (might be wrong, let me know if I am).
>
> And in this test we are using BTF produced by LLVM for a BPF program,
> and then create a .BTF_ids section in a user-space app (test_progs /
> resolve_btfids.test.o), although using proper kernel macros.
>
> By the way, the test was written more than 5y ago [1], so it might be
> outdated too.
>
> I think the behavior that we care about is already indirectly tested
> by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_*
> declarations etc. If resolve_btfids is broken, those tests will fail.
>
> But it's also reasonable to have some tests targeting resolve_btfids
> app itself, of course. This one doesn't fit though IMO.
>
> I'll try to think of something.
Hi Donglin,
I discussed this off-list with Andrii, and we agreed that the selftest
itself is reasonable with respect to testing resolve_btfids output.
In this series, I only have to change the test_progs build recipe.
The problem that you've encountered I think can be fixed in the test,
which is basically what you suggested as option 2:
static int resolve_symbols(void)
{
struct btf *btf;
int type_id;
__u32 nr;
btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */
[...]
Instead of reading in the source BTF, we have to load .btf produced by
resolve_btfids. A complication is that it's going to be a different
file for every TRUNNER_BINARY, which has to be accounted for, although
the BTF itself would be identical between relevant runners.
If go this route, I think we should add .btf cleanup to the Makefile
and update local .gitignore
This change is not strictly necessary in this series, but it is for
the BTF sorting series. Let me know if you would like to take this on,
so we don't do the same work twice.
>
> [1] https://lore.kernel.org/bpf/20200703095111.3268961-10-jolsa@kernel.org/
>
>
>>
>> Thanks,
>> Donglin
>>
>>> + fi
>>> + $(Q)rm -f $$@.btf_ids $$@.btf
>>> $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
>>> $(OUTPUT)/$(if $2,$2/)bpftool
>>>
>>> --
>>> 2.52.0
>>>
>
© 2016 - 2025 Red Hat, Inc.