From nobody Fri Dec 19 14:59:05 2025 Received: from out-186.mta0.migadu.com (out-186.mta0.migadu.com [91.218.175.186]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB0122BE032; Fri, 19 Dec 2025 02:01:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.186 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766109712; cv=none; b=X+MghoVvMVBi5DzbOPjghNbxFuhjzZYTQnB4UJlW78tByUMIgZZJ66m2PHptRaVSsRPUfLo6o7m2k1QFbWwM70LdXCdVOEuJ6zlo1rOiDLIDuLMKlwHQMC6fQ6oud2WUxNbRxMN3C3isoahHb6HgjGDA5c89P9FauVXqZZ1s8yE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1766109712; c=relaxed/simple; bh=soA8tJD7fTYFPN0wWHKIwy/eKVhZMRV8ZiBG4stFW7w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CqjNQvXN4YggqJttxCMK7JGqlToqTVkS7brPVja5X82GZMzVLadxVHIKa23lD0K4H/QjAYsWf1jBN7Loae2uQdFVQPoMSwl5nLJ700p/W27SOavekYDQFIi69KRuHVNFnZv7GkdkxuVsm6Av8Ekjw109XxXWz43okTakzilWs/E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Q9i7ZvI+; arc=none smtp.client-ip=91.218.175.186 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Q9i7ZvI+" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1766109705; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yTfTkUDQ1fOGD1fxOWANeNHg9Pe45MJWrztBiRP8mT0=; b=Q9i7ZvI+hv3UYDmyoEgDA8MYK0sqghoAR8D/K37pdIpSNPH+/rJg2wkVJWr3QFGYrJQH1z 1+x3mlIUpgr1fGnAfOsNTGEDr/p1ONS/HCeTJuJAVyiCuIj04WQ20R9ki+oVs7gi1GqQ+E JJX91rGhigWUjAAPHeLW9XVkPnTZBLo= From: Ihor Solodrai To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Andrew Morton , Nathan Chancellor , Nicolas Schier , Jonathan Corbet , Tejun Heo , David Vernet , Andrea Righi , Changwoo Min , Shuah Khan , Nick Desaulniers , Bill Wendling , Justin Stitt , Alan Maguire , Donglin Peng Cc: bpf@vger.kernel.org, dwarves@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, sched-ext@lists.linux.dev Subject: [PATCH bpf-next v6 8/8] resolve_btfids: Change in-place update with raw binary output Date: Thu, 18 Dec 2025 18:00:06 -0800 Message-ID: <20251219020006.785065-9-ihor.solodrai@linux.dev> In-Reply-To: <20251219020006.785065-1-ihor.solodrai@linux.dev> References: <20251219020006.785065-1-ihor.solodrai@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base. Change resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this, in-place ELF write in resolve_btfids is replaced with generation of the following binaries: * ${1}.BTF with .BTF section data * ${1}.BTF_ids with .BTF_ids section data if it existed in ${1} * ${1}.BTF.base with .BTF.base section data for out-of-tree modules The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch. The motivation for emitting binary data is that it allows simplifying resolve_btfids implementation by delegating ELF update to the $OBJCOPY tool [1], which is already widely used across the codebase. There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules. For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking. The patch doesn't change this approach, only the implementation is a little different. Before this patch it worked as follows: * pahole consumed .tmp_vmlinux1 [2] and added .BTF section with llvm-objcopy [3] to it * then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [2], later linked into vmlinux * resolve_btfids was executed later on vmlinux.unstripped [4], updating it in-place After this patch gen-btf.sh implements the following: * pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data * resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data * a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids * .BTF_ids data gets embedded into vmlinux.unstripped in link-vmlinux.sh by objcopy --update-section For kernel modules, creating a special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with objcopy. With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows deleting elf_update() call and satellite code (like compressed_section_fix [5]). Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [6], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump. [1] https://lore.kernel.org/bpf/131b4190-9c49-4f79-a99d-c00fac97fa44@linux.= dev/ [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree= /scripts/link-vmlinux.sh?h=3Dv6.18#n110 [3] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder= .c?h=3Dv1.31#n1803 [4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree= /scripts/link-vmlinux.sh?h=3Dv6.18#n284 [5] https://lore.kernel.org/bpf/20200819092342.259004-1-jolsa@kernel.org/ [6] https://lore.kernel.org/bpf/cover.1707223196.git.vmalik@redhat.com/ Acked-by: Eduard Zingerman Signed-off-by: Ihor Solodrai --- MAINTAINERS | 1 + scripts/Makefile.btf | 12 +- scripts/Makefile.modfinal | 5 +- scripts/Makefile.vmlinux | 2 +- scripts/gen-btf.sh | 157 ++++++++++++ scripts/link-vmlinux.sh | 42 +--- tools/bpf/resolve_btfids/main.c | 224 +++++++++++------- tools/testing/selftests/bpf/.gitignore | 3 + tools/testing/selftests/bpf/Makefile | 9 +- .../selftests/bpf/prog_tests/resolve_btfids.c | 4 +- 10 files changed, 328 insertions(+), 131 deletions(-) create mode 100755 scripts/gen-btf.sh diff --git a/MAINTAINERS b/MAINTAINERS index 5b11839cba9d..cb1898a85b05 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -4766,6 +4766,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/ diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf index 840a55de42da..562a04b40e06 100644 --- a/scripts/Makefile.btf +++ b/scripts/Makefile.btf @@ -18,13 +18,15 @@ pahole-flags-$(call test-ge, $(pahole-ver), 126) =3D -= j$(JOBS) --btf_features=3Denc =20 pahole-flags-$(call test-ge, $(pahole-ver), 130) +=3D --btf_features=3Datt= ributes =20 -ifneq ($(KBUILD_EXTMOD),) -module-pahole-flags-$(call test-ge, $(pahole-ver), 128) +=3D --btf_feature= s=3Ddistilled_base -endif - endif =20 pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) +=3D --lang_exclude=3Drust =20 export PAHOLE_FLAGS :=3D $(pahole-flags-y) -export MODULE_PAHOLE_FLAGS :=3D $(module-pahole-flags-y) + +resolve-btfids-flags-y :=3D +resolve-btfids-flags-$(CONFIG_WERROR) +=3D --fatal_warnings +resolve-btfids-flags-$(if $(KBUILD_EXTMOD),y) +=3D --distill_base +resolve-btfids-flags-$(if $(KBUILD_VERBOSE),y) +=3D --verbose + +export RESOLVE_BTFIDS_FLAGS :=3D $(resolve-btfids-flags-y) diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 149e12ff5700..422c56dc878e 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -42,9 +42,8 @@ quiet_cmd_btf_ko =3D BTF [M] $@ cmd_btf_ko =3D \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\= n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY=3D"$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE= _FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; =20 # Same as newer-prereqs, but allows to exclude specified extra dependencies diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux index cd788cac9d91..20a988f4fe0c 100644 --- a/scripts/Makefile.vmlinux +++ b/scripts/Makefile.vmlinux @@ -71,7 +71,7 @@ targets +=3D vmlinux.unstripped .vmlinux.export.o vmlinux.unstripped: scripts/link-vmlinux.sh vmlinux.o .vmlinux.export.o $(= KBUILD_LDS) FORCE +$(call if_changed_dep,link_vmlinux) ifdef CONFIG_DEBUG_INFO_BTF -vmlinux.unstripped: $(RESOLVE_BTFIDS) +vmlinux.unstripped: $(RESOLVE_BTFIDS) $(srctree)/scripts/gen-btf.sh endif =20 ifdef CONFIG_BUILDTIME_TABLE_SORT diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh new file mode 100755 index 000000000000..06c6d8becaa2 --- /dev/null +++ b/scripts/gen-btf.sh @@ -0,0 +1,157 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates. +# +# This script generates BTF data for the provided ELF file. +# +# Kernel BTF generation involves these conceptual steps: +# 1. pahole generates BTF from DWARF data +# 2. resolve_btfids applies kernel-specific btf2btf +# transformations and computes data for .BTF_ids section +# 3. the result gets linked/objcopied into the target binary +# +# How step (3) should be done differs between vmlinux, and +# kernel modules, which is the primary reason for the existence +# of this script. +# +# For modules the script expects vmlinux passed in as --btf_base. +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded +# into the input ELF file with objcopy. +# +# For vmlinux the input file remains unchanged and two files are produced: +# - ${1}.btf.o ready for linking into vmlinux +# - ${1}.BTF_ids with .BTF_ids data blob +# This output is consumed by scripts/link-vmlinux.sh + +set -e + +usage() +{ + echo "Usage: $0 [--btf_base ] " + exit 1 +} + +BTF_BASE=3D"" + +while [ $# -gt 0 ]; do + case "$1" in + --btf_base) + BTF_BASE=3D"$2" + shift 2 + ;; + -*) + echo "Unknown option: $1" >&2 + usage + ;; + *) + break + ;; + esac +done + +if [ $# -ne 1 ]; then + usage +fi + +ELF_FILE=3D"$1" +shift + +is_enabled() { + grep -q "^$1=3Dy" ${objtree}/include/config/auto.conf +} + +info() +{ + printf " %-7s %s\n" "${1}" "${2}" +} + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1=3D"${ELF_FILE}.BTF.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=3D${btf1} \ + "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf ${btf1} "${ELF_FILE}" +} + +gen_btf_o() +{ + local btf_data=3D${ELF_FILE}.btf.o + + # Create ${btf_data} which contains just .BTF section but no symbols. Add + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all + # deletes all symbols including __start_BTF and __stop_BTF, which will + # be redefined in the linker script. + info OBJCOPY "${btf_data}" + echo "" | ${CC} ${CLANG_FLAGS} -c -x c -o ${btf_data} - + ${OBJCOPY} --add-section .BTF=3D${ELF_FILE}.BTF \ + --set-section-flags .BTF=3Dalloc,readonly ${btf_data} + ${OBJCOPY} --only-section=3D.BTF --strip-all ${btf_data} + + # Change e_type to ET_REL so that it can be used to link final vmlinux. + # GNU ld 2.35+ and lld do not allow an ET_EXEC input. + if is_enabled CONFIG_CPU_BIG_ENDIAN; then + et_rel=3D'\0\1' + else + et_rel=3D'\1\0' + fi + printf "${et_rel}" | dd of=3D"${btf_data}" conv=3Dnotrunc bs=3D1 seek=3D1= 6 status=3Dnone +} + +embed_btf_data() +{ + info OBJCOPY "${ELF_FILE}.BTF" + ${OBJCOPY} --add-section .BTF=3D${ELF_FILE}.BTF ${ELF_FILE} + + # a module might not have a .BTF_ids or .BTF.base section + local btf_base=3D"${ELF_FILE}.BTF.base" + if [ -f "${btf_base}" ]; then + ${OBJCOPY} --add-section .BTF.base=3D${btf_base} ${ELF_FILE} + fi + local btf_ids=3D"${ELF_FILE}.BTF_ids" + if [ -f "${btf_ids}" ]; then + ${OBJCOPY} --update-section .BTF_ids=3D${btf_ids} ${ELF_FILE} + fi +} + +cleanup() +{ + rm -f "${ELF_FILE}.BTF.1" + rm -f "${ELF_FILE}.BTF" + if [ "${BTFGEN_MODE}" =3D "module" ]; then + rm -f "${ELF_FILE}.BTF.base" + rm -f "${ELF_FILE}.BTF_ids" + fi +} +trap cleanup EXIT + +BTFGEN_MODE=3D"vmlinux" +if [ -n "${BTF_BASE}" ]; then + BTFGEN_MODE=3D"module" +fi + +gen_btf_data + +case "${BTFGEN_MODE}" in +vmlinux) + gen_btf_o + ;; +module) + embed_btf_data + ;; +esac + +exit 0 diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 4ab44c73da4d..e2207e612ac3 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -106,34 +106,6 @@ vmlinux_link() ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} } =20 -# generate .BTF typeinfo from DWARF debuginfo -# ${1} - vmlinux image -gen_btf() -{ - local btf_data=3D${1}.btf.o - - info BTF "${btf_data}" - LLVM_OBJCOPY=3D"${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1} - - # Create ${btf_data} which contains just .BTF section but no symbols. Add - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all - # deletes all symbols including __start_BTF and __stop_BTF, which will - # be redefined in the linker script. Add 2>/dev/null to suppress GNU - # objcopy warnings: "empty loadable segment detected at ..." - ${OBJCOPY} --only-section=3D.BTF --set-section-flags .BTF=3Dalloc,readonl= y \ - --strip-all ${1} "${btf_data}" 2>/dev/null - # Change e_type to ET_REL so that it can be used to link final vmlinux. - # GNU ld 2.35+ and lld do not allow an ET_EXEC input. - if is_enabled CONFIG_CPU_BIG_ENDIAN; then - et_rel=3D'\0\1' - else - et_rel=3D'\1\0' - fi - printf "${et_rel}" | dd of=3D"${btf_data}" conv=3Dnotrunc bs=3D1 seek=3D1= 6 status=3Dnone - - btf_vmlinux_bin_o=3D${btf_data} -} - # Create ${2}.o file with all symbols from the ${1} object file kallsyms() { @@ -205,6 +177,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then fi =20 btf_vmlinux_bin_o=3D +btfids_vmlinux=3D kallsymso=3D strip_debug=3D generate_map=3D @@ -232,11 +205,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DE= BUG_INFO_BTF; then fi =20 if is_enabled CONFIG_DEBUG_INFO_BTF; then - if ! gen_btf .tmp_vmlinux1; then + if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then echo >&2 "Failed to generate BTF for vmlinux" echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF" exit 1 fi + btf_vmlinux_bin_o=3D.tmp_vmlinux1.btf.o + btfids_vmlinux=3D.tmp_vmlinux1.BTF_ids fi =20 if is_enabled CONFIG_KALLSYMS; then @@ -289,14 +264,9 @@ fi =20 vmlinux_link "${VMLINUX}" =20 -# fill in BTF IDs if is_enabled CONFIG_DEBUG_INFO_BTF; then - info BTFIDS "${VMLINUX}" - RESOLVE_BTFIDS_ARGS=3D"" - if is_enabled CONFIG_WERROR; then - RESOLVE_BTFIDS_ARGS=3D" --fatal_warnings " - fi - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}" + info OBJCOPY ${btfids_vmlinux} + ${OBJCOPY} --update-section .BTF_ids=3D${btfids_vmlinux} ${VMLINUX} fi =20 mksysmap "${VMLINUX}" System.map diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/mai= n.c index e721e20a2bbd..2cbc252259be 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -71,9 +71,11 @@ #include #include #include +#include #include #include #include +#include #include #include #include @@ -124,6 +126,7 @@ struct object { =20 struct btf *btf; struct btf *base_btf; + bool distill_base; =20 struct { int fd; @@ -324,42 +327,16 @@ static struct btf_id *add_symbol(struct rb_root *root= , char *name, size_t size) return btf_id__add(root, id, BTF_ID_KIND_SYM); } =20 -/* Older libelf.h and glibc elf.h might not yet define the ELF compression= types. */ -#ifndef SHF_COMPRESSED -#define SHF_COMPRESSED (1 << 11) /* Section with compressed data. */ -#endif - -/* - * The data of compressed section should be aligned to 4 - * (for 32bit) or 8 (for 64 bit) bytes. The binutils ld - * sets sh_addralign to 1, which makes libelf fail with - * misaligned section error during the update: - * FAILED elf_update(WRITE): invalid section alignment - * - * While waiting for ld fix, we fix the compressed sections - * sh_addralign value manualy. - */ -static int compressed_section_fix(Elf *elf, Elf_Scn *scn, GElf_Shdr *sh) +static void bswap_32_data(void *data, u32 nr_bytes) { - int expected =3D gelf_getclass(elf) =3D=3D ELFCLASS32 ? 4 : 8; - - if (!(sh->sh_flags & SHF_COMPRESSED)) - return 0; - - if (sh->sh_addralign =3D=3D expected) - return 0; + u32 cnt, i; + u32 *ptr; =20 - pr_debug2(" - fixing wrong alignment sh_addralign %u, expected %u\n", - sh->sh_addralign, expected); + cnt =3D nr_bytes / sizeof(u32); + ptr =3D data; =20 - sh->sh_addralign =3D expected; - - if (gelf_update_shdr(scn, sh) =3D=3D 0) { - pr_err("FAILED cannot update section header: %s\n", - elf_errmsg(-1)); - return -1; - } - return 0; + for (i =3D 0; i < cnt; i++) + ptr[i] =3D bswap_32(ptr[i]); } =20 static int elf_collect(struct object *obj) @@ -380,7 +357,7 @@ static int elf_collect(struct object *obj) =20 elf_version(EV_CURRENT); =20 - elf =3D elf_begin(fd, ELF_C_RDWR_MMAP, NULL); + elf =3D elf_begin(fd, ELF_C_READ_MMAP_PRIVATE, NULL); if (!elf) { close(fd); pr_err("FAILED cannot create ELF descriptor: %s\n", @@ -443,21 +420,20 @@ static int elf_collect(struct object *obj) obj->efile.symbols_shndx =3D idx; obj->efile.strtabidx =3D sh.sh_link; } else if (!strcmp(name, BTF_IDS_SECTION)) { + /* + * If target endianness differs from host, we need to bswap32 + * the .BTF_ids section data on load, because .BTF_ids has + * Elf_Type =3D ELF_T_BYTE, and so libelf returns data buffer in + * the target endianness. We repeat this on dump. + */ + if (obj->efile.encoding !=3D ELFDATANATIVE) { + pr_debug("bswap_32 .BTF_ids data from target to host endianness\n"); + bswap_32_data(data->d_buf, data->d_size); + } obj->efile.idlist =3D data; obj->efile.idlist_shndx =3D idx; obj->efile.idlist_addr =3D sh.sh_addr; - } else if (!strcmp(name, BTF_BASE_ELF_SEC)) { - /* If a .BTF.base section is found, do not resolve - * BTF ids relative to vmlinux; resolve relative - * to the .BTF.base section instead. btf__parse_split() - * will take care of this once the base BTF it is - * passed is NULL. - */ - obj->base_btf_path =3D NULL; } - - if (compressed_section_fix(elf, scn, &sh)) - return -1; } =20 return 0; @@ -587,11 +563,26 @@ static int load_btf(struct object *obj) obj->base_btf =3D base_btf; obj->btf =3D btf; =20 + if (obj->base_btf && obj->distill_base) { + err =3D btf__distill_base(obj->btf, &base_btf, &btf); + if (err) { + pr_err("FAILED to distill base BTF: %s\n", strerror(errno)); + goto out_err; + } + + btf__free(obj->base_btf); + btf__free(obj->btf); + obj->base_btf =3D base_btf; + obj->btf =3D btf; + } + return 0; =20 out_err: btf__free(base_btf); btf__free(btf); + obj->base_btf =3D NULL; + obj->btf =3D NULL; return err; } =20 @@ -760,24 +751,6 @@ static int sets_patch(struct object *obj) */ BUILD_BUG_ON((u32 *)set8->pairs !=3D &set8->pairs[0].id); qsort(set8->pairs, set8->cnt, sizeof(set8->pairs[0]), cmp_id); - - /* - * When ELF endianness does not match endianness of the - * host, libelf will do the translation when updating - * the ELF. This, however, corrupts SET8 flags which are - * already in the target endianness. So, let's bswap - * them to the host endianness and libelf will then - * correctly translate everything. - */ - if (obj->efile.encoding !=3D ELFDATANATIVE) { - int i; - - set8->flags =3D bswap_32(set8->flags); - for (i =3D 0; i < set8->cnt; i++) { - set8->pairs[i].flags =3D - bswap_32(set8->pairs[i].flags); - } - } break; default: pr_err("Unexpected btf_id_kind %d for set '%s'\n", id->kind, id->name); @@ -793,8 +766,6 @@ static int sets_patch(struct object *obj) =20 static int symbols_patch(struct object *obj) { - off_t err; - if (__symbols_patch(obj, &obj->structs) || __symbols_patch(obj, &obj->unions) || __symbols_patch(obj, &obj->typedefs) || @@ -805,20 +776,90 @@ static int symbols_patch(struct object *obj) if (sets_patch(obj)) return -1; =20 - /* Set type to ensure endian translation occurs. */ - obj->efile.idlist->d_type =3D ELF_T_WORD; + return 0; +} =20 - elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY); +static int dump_raw_data(const char *out_path, const void *data, u32 size) +{ + size_t written; + FILE *file; =20 - err =3D elf_update(obj->efile.elf, ELF_C_WRITE); - if (err < 0) { - pr_err("FAILED elf_update(WRITE): %s\n", - elf_errmsg(-1)); + file =3D fopen(out_path, "wb"); + if (!file) { + pr_err("Couldn't open %s for writing\n", out_path); + return -1; + } + + written =3D fwrite(data, 1, size, file); + if (written !=3D size) { + pr_err("Failed to write data to %s\n", out_path); + fclose(file); + unlink(out_path); + return -1; + } + + fclose(file); + pr_debug("Dumped %lu bytes of data to %s\n", size, out_path); + + return 0; +} + +static int dump_raw_btf_ids(struct object *obj, const char *out_path) +{ + Elf_Data *data =3D obj->efile.idlist; + int err; + + if (!data || !data->d_buf) { + pr_debug("%s has no BTF_ids data to dump\n", obj->path); + return 0; + } + + /* + * If target endianness differs from host, we need to bswap32 the + * .BTF_ids section data before dumping so that the output is in + * target endianness. + */ + if (obj->efile.encoding !=3D ELFDATANATIVE) { + pr_debug("bswap_32 .BTF_ids data from host to target endianness\n"); + bswap_32_data(data->d_buf, data->d_size); + } + + err =3D dump_raw_data(out_path, data->d_buf, data->d_size); + if (err) + return -1; + + return 0; +} + +static int dump_raw_btf(struct btf *btf, const char *out_path) +{ + const void *raw_btf_data; + u32 raw_btf_size; + int err; + + raw_btf_data =3D btf__raw_data(btf, &raw_btf_size); + if (!raw_btf_data) { + pr_err("btf__raw_data() failed\n"); + return -1; } =20 - pr_debug("update %s for %s\n", - err >=3D 0 ? "ok" : "failed", obj->path); - return err < 0 ? -1 : 0; + err =3D dump_raw_data(out_path, raw_btf_data, raw_btf_size); + if (err) + return -1; + + return 0; +} + +static inline int make_out_path(char *buf, u32 buf_sz, const char *in_path= , const char *suffix) +{ + int len =3D snprintf(buf, buf_sz, "%s%s", in_path, suffix); + + if (len < 0 || len >=3D buf_sz) { + pr_err("Output path is too long: %s%s\n", in_path, suffix); + return -E2BIG; + } + + return 0; } =20 static const char * const resolve_btfids_usage[] =3D { @@ -840,6 +881,8 @@ int main(int argc, const char **argv) .sets =3D RB_ROOT, }; bool fatal_warnings =3D false; + char out_path[PATH_MAX]; + struct option btfid_options[] =3D { OPT_INCR('v', "verbose", &verbose, "be more verbose (show errors, etc)"), @@ -849,6 +892,8 @@ int main(int argc, const char **argv) "path of file providing base BTF"), OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings, "turn warnings into errors"), + OPT_BOOLEAN(0, "distill_base", &obj.distill_base, + "distill --btf_base and emit .BTF.base section data"), OPT_END() }; int err =3D -1; @@ -860,6 +905,9 @@ int main(int argc, const char **argv) =20 obj.path =3D argv[0]; =20 + if (load_btf(&obj)) + goto out; + if (elf_collect(&obj)) goto out; =20 @@ -869,23 +917,37 @@ int main(int argc, const char **argv) */ if (obj.efile.idlist_shndx =3D=3D -1 || obj.efile.symbols_shndx =3D=3D -1) { - pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n"); - err =3D 0; - goto out; + pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolut= ion\n"); + goto dump_btf; } =20 if (symbols_collect(&obj)) goto out; =20 - if (load_btf(&obj)) - goto out; - if (symbols_resolve(&obj)) goto out; =20 if (symbols_patch(&obj)) goto out; =20 + err =3D make_out_path(out_path, sizeof(out_path), obj.path, BTF_IDS_SECTI= ON); + err =3D err ?: dump_raw_btf_ids(&obj, out_path); + if (err) + goto out; + +dump_btf: + err =3D make_out_path(out_path, sizeof(out_path), obj.path, BTF_ELF_SEC); + err =3D err ?: dump_raw_btf(obj.btf, out_path); + if (err) + goto out; + + if (obj.base_btf && obj.distill_base) { + err =3D make_out_path(out_path, sizeof(out_path), obj.path, BTF_BASE_ELF= _SEC); + err =3D err ?: dump_raw_btf(obj.base_btf, out_path); + if (err) + goto out; + } + if (!(fatal_warnings && warnings)) err =3D 0; out: diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftes= ts/bpf/.gitignore index 19c1638e312a..b8bf51b7a0b0 100644 --- a/tools/testing/selftests/bpf/.gitignore +++ b/tools/testing/selftests/bpf/.gitignore @@ -45,3 +45,6 @@ xdp_synproxy xdp_hw_metadata xdp_features verification_cert.h +*.BTF +*.BTF_ids +*.BTF.base diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests= /bpf/Makefile index ffd0a4c354c7..f28a32b16ff0 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include =20 CXX ?=3D $(CROSS_COMPILE)g++ +OBJCOPY ?=3D $(CROSS_COMPILE)objcopy =20 CURDIR :=3D $(abspath .) TOOLSDIR :=3D $(abspath ../../..) @@ -653,9 +654,10 @@ $(TRUNNER_TEST_OBJS): $(TRUNNER_OUTPUT)/%.test.o: \ | $(TRUNNER_OUTPUT)/%.test.d $$(call msg,TEST-OBJ,$(TRUNNER_BINARY),$$@) $(Q)cd $$(@D) && $$(CC) -I. $$(CFLAGS) -MMD -MT $$@ -c $(CURDIR)/$$< $$(L= DLIBS) -o $$(@F) - $$(if $$(TEST_NEEDS_BTFIDS), \ - $$(call msg,BTFIDS,$(TRUNNER_BINARY),$$@) \ - $(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@) + $$(if $$(TEST_NEEDS_BTFIDS), \ + $$(call msg,BTFIDS,$(TRUNNER_BINARY),$$@) \ + $(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@; \ + $(OBJCOPY) --update-section .BTF_ids=3D$$@.BTF_ids $$@) =20 $(TRUNNER_TEST_OBJS:.o=3D.d): $(TRUNNER_OUTPUT)/%.test.d: \ $(TRUNNER_TESTS_DIR)/%.c \ @@ -894,6 +896,7 @@ EXTRA_CLEAN :=3D $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool $(TEST_KMOD_TARGETS) \ $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \ + *.BTF *.BTF_ids *.BTF.base \ no_alu32 cpuv4 bpf_gcc \ liburandom_read.so) \ $(OUTPUT)/FEATURE-DUMP.selftests \ diff --git a/tools/testing/selftests/bpf/prog_tests/resolve_btfids.c b/tool= s/testing/selftests/bpf/prog_tests/resolve_btfids.c index 51544372f52e..41dfaaabb73f 100644 --- a/tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +++ b/tools/testing/selftests/bpf/prog_tests/resolve_btfids.c @@ -101,9 +101,9 @@ static int resolve_symbols(void) int type_id; __u32 nr; =20 - btf =3D btf__parse_elf("btf_data.bpf.o", NULL); + btf =3D btf__parse_raw("resolve_btfids.test.o.BTF"); if (CHECK(libbpf_get_error(btf), "resolve", - "Failed to load BTF from btf_data.bpf.o\n")) + "Failed to load BTF from resolve_btfids.test.o.BTF\n")) return -1; =20 nr =3D btf__type_cnt(btf); --=20 2.52.0