From nobody Wed Apr 8 20:03:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 606ECECAAA1 for ; Fri, 28 Oct 2022 19:48:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229934AbiJ1TsZ (ORCPT ); Fri, 28 Oct 2022 15:48:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230142AbiJ1TsS (ORCPT ); Fri, 28 Oct 2022 15:48:18 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11C6A24419D for ; Fri, 28 Oct 2022 12:48:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=w2TcGA3omIeq3yxp81jgH+z11BiFQX/+KG2uxbRcnKg=; b=ks+sTiRY6cSm9Wr/5Ajk0dRWgD 8FOEc3MhC69/nxeFJfF4MsM5p2kWKC0wMsBYpMgeuLjF4ynIO07RIqoFH10m/rbeHWi7aAjPcH4XL BL93ujKDf988/1YnHhVgktI3/a9FCh3gG44Sp2dT6uAXcGTfGB0X7JnbFnwtbMPXUnVHzi5hmXtjd iqLtkIK/EC2vzuPd3jlzDWNKIXLHwAqwxyGEx4o2T/I6hFbLiWgvf7q6KMMYp/eW0aF5Bvx+PJjO3 +eUjCJW1JFrIw8bbpdV88/LkK9ictwhcmKs/wwt4bIH9/j4C3Pkv0L8PFQlX6SMutqrEXhwsr7+3+ j/QVtvRQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1ooVKR-007ARB-1D; Fri, 28 Oct 2022 19:47:43 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B57CC300322; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 980AB2C6B7F85; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Message-ID: <20221028194453.330970755@infradead.org> User-Agent: quilt/0.66 Date: Fri, 28 Oct 2022 21:40:23 +0200 From: Peter Zijlstra To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, djwong@kernel.org, yujie.liu@intel.com, tglx@linutronix.de, jpoimboe@kernel.org, joao.moreira@intel.com, samitolvanen@google.com Subject: [PATCH 1/5] kallsyms: Revert "Take callthunks into account" References: <20221028194022.388521751@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This is a full revert of commit: f1389181622a ("kallsyms: Take callthunks into account") The commit assumes a number of things that are not quite right. Notably it assumes every symbol has PADDING_BYTES in front of it that are not claimed by another symbol. This is not true; even when compiled with: -fpatchable-function-entry=3D${PADDING_BYTES},${PADDING_BYTES} Notably things like .cold subfunctions do not need to adhere to this change in ABI. It it also not true when build with CFI_CLANG, which claims these PADDING_BYTES in the __cfi_##name symbol. Once the prefix bytes are not consistent and or otherwise claimed the approach this patch takes goes out the window and kallsym resolution will report invalid symbol names. Therefore revert this to make room for another approach. Reported-by: Reported-by: kernel test robot Signed-off-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/r/202210241614.2ae4c1f5-yujie.liu@intel.com --- kernel/kallsyms.c | 45 +++++---------------------------------------- 1 file changed, 5 insertions(+), 40 deletions(-) --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -293,12 +293,6 @@ static unsigned long get_symbol_pos(unsi return low; } =20 -#ifdef CONFIG_FUNCTION_PADDING_BYTES -#define PADDING_BYTES CONFIG_FUNCTION_PADDING_BYTES -#else -#define PADDING_BYTES 0 -#endif - /* * Lookup an address but don't bother to find any names. */ @@ -306,25 +300,13 @@ int kallsyms_lookup_size_offset(unsigned unsigned long *offset) { char namebuf[KSYM_NAME_LEN]; - int ret; - - addr +=3D PADDING_BYTES; =20 if (is_ksym_addr(addr)) { get_symbol_pos(addr, symbolsize, offset); - ret =3D 1; - goto found; + return 1; } - - ret =3D !!module_address_lookup(addr, symbolsize, offset, NULL, NULL, nam= ebuf); - if (!ret) { - ret =3D !!__bpf_address_lookup(addr, symbolsize, - offset, namebuf); - } -found: - if (ret && offset) - *offset -=3D PADDING_BYTES; - return ret; + return !!module_address_lookup(addr, symbolsize, offset, NULL, NULL, name= buf) || + !!__bpf_address_lookup(addr, symbolsize, offset, namebuf); } =20 static const char *kallsyms_lookup_buildid(unsigned long addr, @@ -337,8 +319,6 @@ static const char *kallsyms_lookup_build namebuf[KSYM_NAME_LEN - 1] =3D 0; namebuf[0] =3D 0; =20 - addr +=3D PADDING_BYTES; - if (is_ksym_addr(addr)) { unsigned long pos; =20 @@ -368,8 +348,6 @@ static const char *kallsyms_lookup_build =20 found: cleanup_symbol_name(namebuf); - if (ret && offset) - *offset -=3D PADDING_BYTES; return ret; } =20 @@ -396,8 +374,6 @@ int lookup_symbol_name(unsigned long add symname[0] =3D '\0'; symname[KSYM_NAME_LEN - 1] =3D '\0'; =20 - addr +=3D PADDING_BYTES; - if (is_ksym_addr(addr)) { unsigned long pos; =20 @@ -425,8 +401,6 @@ int lookup_symbol_attrs(unsigned long ad name[0] =3D '\0'; name[KSYM_NAME_LEN - 1] =3D '\0'; =20 - addr +=3D PADDING_BYTES; - if (is_ksym_addr(addr)) { unsigned long pos; =20 @@ -443,8 +417,6 @@ int lookup_symbol_attrs(unsigned long ad return res; =20 found: - if (offset) - *offset -=3D PADDING_BYTES; cleanup_symbol_name(name); return 0; } @@ -470,15 +442,8 @@ static int __sprint_symbol(char *buffer, len =3D strlen(buffer); offset -=3D symbol_offset; =20 - if (add_offset) { - char s =3D '+'; - - if ((long)offset < 0) { - s =3D '-'; - offset =3D 0UL - offset; - } - len +=3D sprintf(buffer + len, "%c%#lx/%#lx", s, offset, size); - } + if (add_offset) + len +=3D sprintf(buffer + len, "+%#lx/%#lx", offset, size); =20 if (modname) { len +=3D sprintf(buffer + len, " [%s", modname); From nobody Wed Apr 8 20:03:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D99AC38A02 for ; Fri, 28 Oct 2022 19:48:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230144AbiJ1TsH (ORCPT ); Fri, 28 Oct 2022 15:48:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbiJ1Trz (ORCPT ); Fri, 28 Oct 2022 15:47:55 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AED6A241B07 for ; Fri, 28 Oct 2022 12:47:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=IORdw0i358+WCHTJMo8FHxkOZ204c2jlWb6nUgvX6XI=; b=mHVGpEbm8sOcQ7hwd9786o/oxg p0whUxIGOUwNbBtGadYe5jv9R6b0FVQTQZ31Y0chVZi2yOPZeuFHU2k6PMkVd2WCCfp/DpXjArrDN n21ePo41DzyAu18b+ipSYfhhDicimP6xlfVzqsGDBmAdy6N9YcDq1c3svOG8zNwOWqyvVEe/RI6FZ 2UHw4VGe31/g6jdAx3jSJY6jEPN5pCwU+6suxlXpuykF1znt84zgQGaiXOQcLbAmOm989AO4Mdy7k d61HL9Y6l9tFWVqkCMmyPW4BeqtYIWIhru1d9GO6siabvRPt4lTHoPIHBIkGy1DKbEg/cZrskSKUG XNuSXtdQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1ooVKV-001V5n-VM; Fri, 28 Oct 2022 19:47:48 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B74FF30035C; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 99F592C6DAB20; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Message-ID: <20221028194453.396634875@infradead.org> User-Agent: quilt/0.66 Date: Fri, 28 Oct 2022 21:40:24 +0200 From: Peter Zijlstra To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, djwong@kernel.org, yujie.liu@intel.com, tglx@linutronix.de, jpoimboe@kernel.org, joao.moreira@intel.com, samitolvanen@google.com Subject: [PATCH 2/5] objtool: Slice up elf_create_section_symbol() References: <20221028194022.388521751@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In order to facilitate creation of more symbol types, slice up elf_create_section_symbol() to extract a generic helper that deals with adding ELF symbols. Signed-off-by: Peter Zijlstra (Intel) --- tools/objtool/elf.c | 56 ++++++++++++++++++++++++++++++++---------------= ----- 1 file changed, 35 insertions(+), 21 deletions(-) --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -717,11 +717,11 @@ static int elf_update_symbol(struct elf } =20 static struct symbol * -elf_create_section_symbol(struct elf *elf, struct section *sec) +__elf_create_symbol(struct elf *elf, struct symbol *sym) { struct section *symtab, *symtab_shndx; Elf32_Word first_non_local, new_idx; - struct symbol *sym, *old; + struct symbol *old; =20 symtab =3D find_section_by_name(elf, ".symtab"); if (symtab) { @@ -731,27 +731,16 @@ elf_create_section_symbol(struct elf *el return NULL; } =20 - sym =3D calloc(1, sizeof(*sym)); - if (!sym) { - perror("malloc"); - return NULL; - } + new_idx =3D symtab->sh.sh_size / symtab->sh.sh_entsize; =20 - sym->name =3D sec->name; - sym->sec =3D sec; - - // st_name 0 - sym->sym.st_info =3D GELF_ST_INFO(STB_LOCAL, STT_SECTION); - // st_other 0 - // st_value 0 - // st_size 0 + if (GELF_ST_BIND(sym->sym.st_info) !=3D STB_LOCAL) + goto non_local; =20 /* * Move the first global symbol, as per sh_info, into a new, higher * symbol index. This fees up a spot for a new local symbol. */ first_non_local =3D symtab->sh.sh_info; - new_idx =3D symtab->sh.sh_size / symtab->sh.sh_entsize; old =3D find_symbol_by_index(elf, first_non_local); if (old) { old->idx =3D new_idx; @@ -769,18 +758,43 @@ elf_create_section_symbol(struct elf *el new_idx =3D first_non_local; } =20 + /* + * Either way, we will add a LOCAL symbol. + */ + symtab->sh.sh_info +=3D 1; + +non_local: sym->idx =3D new_idx; if (elf_update_symbol(elf, symtab, symtab_shndx, sym)) { WARN("elf_update_symbol"); return NULL; } =20 - /* - * Either way, we added a LOCAL symbol. - */ - symtab->sh.sh_info +=3D 1; + return sym; +} + +static struct symbol * +elf_create_section_symbol(struct elf *elf, struct section *sec) +{ + struct symbol *sym =3D calloc(1, sizeof(*sym)); + + if (!sym) { + perror("malloc"); + return NULL; + } + + sym->name =3D sec->name; + sym->sec =3D sec; + + // st_name 0 + sym->sym.st_info =3D GELF_ST_INFO(STB_LOCAL, STT_SECTION); + // st_other 0 + // st_value 0 + // st_size 0 =20 - elf_add_symbol(elf, sym); + sym =3D __elf_create_symbol(elf, sym); + if (sym) + elf_add_symbol(elf, sym); =20 return sym; } From nobody Wed Apr 8 20:03:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5845AECAAA1 for ; Fri, 28 Oct 2022 19:48:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230099AbiJ1Tr6 (ORCPT ); Fri, 28 Oct 2022 15:47:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43486 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229572AbiJ1Trz (ORCPT ); Fri, 28 Oct 2022 15:47:55 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78069241B37 for ; Fri, 28 Oct 2022 12:47:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=3QcO6i1OADBJrO89uFWVOZLcBxalIqOKw3AE7W+Oknw=; b=unaldPb3HwjbCtO0iBmuGZTk6j juei5a08VijPiQrm129/7iCtv3PUdfkDT4r4emLHSA8PKltscoNN2VYUewY2eG/k0gbbTIvxyWXTK +pWLutlNYuznSLqIlz8sv6PdRv+++S9LYb2GUhA69JqylEY1OXW55EgSsX6ANz7hbW1u74qOGd4j3 GmtIrPhP1EQhaV4HP64GxnKfZfU77aXsRB/qE1sS26zfagxhP87AHpKXfY/EqIaI5gJtHTT/0SaSi PCiL1A1q5QOfVoH73H6y1ihD8g1PZb/RqRIf3nxDvRQCYC5Hkr0WgG0pdoZizfWjdILDZiEJaCcvP EAs2VNLQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1ooVKV-001V5p-VU; Fri, 28 Oct 2022 19:47:48 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id B90B7300872; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 9F8672B9B03D1; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Message-ID: <20221028194453.461658986@infradead.org> User-Agent: quilt/0.66 Date: Fri, 28 Oct 2022 21:40:25 +0200 From: Peter Zijlstra To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, djwong@kernel.org, yujie.liu@intel.com, tglx@linutronix.de, jpoimboe@kernel.org, joao.moreira@intel.com, samitolvanen@google.com Subject: [PATCH 3/5] objtool: Avoid O(bloody terrible) behaviour -- an ode to libelf References: <20221028194022.388521751@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Due to how gelf_update_sym*() requires an Elf_Data pointer, and how libelf keeps Elf_Data in a linked list per section, elf_update_symbol() ends up having to iterate this list on each update to find the correct Elf_Data for the index'ed symbol. By allocating one Elf_Data per new symbol, the list grows per new symbol, giving an effective O(n^2) insertion time. This is obviously bloody terrible. Therefore over-allocate the Elf_Data when an extention is needed. Except it turns out libelf disregards Elf_Scn::sh_size in favour of the sum of Elf_Data::d_size. IOW it will happily write out all the unused space and fill it with: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND entries (aka zeros). Which obviously violates the STB_LOCAL placement rule, and is a general pain in the backside for not being the desired behaviour. Manually fix-up the Elf_Data size to avoid this problem before calling elf_update(). This significantly improves performance when adding a significant number of symbols. Signed-off-by: Peter Zijlstra (Intel) --- tools/objtool/elf.c | 89 +++++++++++++++++++++++++++++++= ++--- tools/objtool/include/objtool/elf.h | 2=20 2 files changed, 84 insertions(+), 7 deletions(-) --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -634,6 +634,12 @@ static int elf_update_symbol(struct elf =20 /* end-of-list */ if (!symtab_data) { + /* + * Over-allocate to avoid O(n^2) symbol creation + * behaviour. The down side is that libelf doesn't + * like this; see elf_truncate_section() for the fixup. + */ + int num =3D max(1U, sym->idx/3); void *buf; =20 if (idx) { @@ -647,28 +653,34 @@ static int elf_update_symbol(struct elf if (t) shndx_data =3D elf_newdata(t); =20 - buf =3D calloc(1, entsize); + buf =3D calloc(num, entsize); if (!buf) { WARN("malloc"); return -1; } =20 symtab_data->d_buf =3D buf; - symtab_data->d_size =3D entsize; + symtab_data->d_size =3D num * entsize; symtab_data->d_align =3D 1; symtab_data->d_type =3D ELF_T_SYM; =20 - symtab->sh.sh_size +=3D entsize; symtab->changed =3D true; + symtab->truncate =3D true; =20 if (t) { - shndx_data->d_buf =3D &sym->sec->idx; - shndx_data->d_size =3D sizeof(Elf32_Word); + buf =3D calloc(num, sizeof(Elf32_Word)); + if (!buf) { + WARN("malloc"); + return -1; + } + + shndx_data->d_buf =3D buf; + shndx_data->d_size =3D num * sizeof(Elf32_Word); shndx_data->d_align =3D sizeof(Elf32_Word); shndx_data->d_type =3D ELF_T_WORD; =20 - symtab_shndx->sh.sh_size +=3D sizeof(Elf32_Word); symtab_shndx->changed =3D true; + symtab_shndx->truncate =3D true; } =20 break; @@ -770,6 +782,14 @@ __elf_create_symbol(struct elf *elf, str return NULL; } =20 + symtab->sh.sh_size +=3D symtab->sh.sh_entsize; + symtab->changed =3D true; + + if (symtab_shndx) { + symtab_shndx->sh.sh_size +=3D sizeof(Elf32_Word); + symtab_shndx->changed =3D true; + } + return sym; } =20 @@ -1286,6 +1306,60 @@ int elf_write_reloc(struct elf *elf, str return 0; } =20 +/* + * When Elf_Scn::sh_size is smaller than the combined Elf_Data::d_size + * do you: + * + * A) adhere to the section header and truncate the data, or + * B) ignore the section header and write out all the data you've got? + * + * Yes, libelf sucks and we need to manually truncate if we over-allocate = data. + */ +static int elf_truncate_section(struct elf *elf, struct section *sec) +{ + u64 size =3D sec->sh.sh_size; + bool truncated =3D false; + Elf_Data *data =3D NULL; + Elf_Scn *s; + + s =3D elf_getscn(elf->elf, sec->idx); + if (!s) { + WARN_ELF("elf_getscn"); + return -1; + } + + for (;;) { + /* get next data descriptor for the relevant section */ + data =3D elf_getdata(s, data); + + if (!data) { + if (size) { + WARN("end of section data but non-zero size left\n"); + return -1; + } + return 0; + } + + if (truncated) { + /* when we remove symbols */ + WARN("truncated; but more data\n"); + return -1; + } + + if (!data->d_size) { + WARN("zero size data"); + return -1; + } + + if (data->d_size > size) { + truncated =3D true; + data->d_size =3D size; + } + + size -=3D data->d_size; + } +} + int elf_write(struct elf *elf) { struct section *sec; @@ -1296,6 +1370,9 @@ int elf_write(struct elf *elf) =20 /* Update changed relocation sections and section headers: */ list_for_each_entry(sec, &elf->sections, list) { + if (sec->truncate) + elf_truncate_section(elf, sec); + if (sec->changed) { s =3D elf_getscn(elf->elf, sec->idx); if (!s) { --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -38,7 +38,7 @@ struct section { Elf_Data *data; char *name; int idx; - bool changed, text, rodata, noinstr, init; + bool changed, text, rodata, noinstr, init, truncate; }; =20 struct symbol { From nobody Wed Apr 8 20:03:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBCFCFA3740 for ; Fri, 28 Oct 2022 19:48:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229946AbiJ1Ts1 (ORCPT ); Fri, 28 Oct 2022 15:48:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43702 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230094AbiJ1TsS (ORCPT ); Fri, 28 Oct 2022 15:48:18 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 11EC52441A3 for ; Fri, 28 Oct 2022 12:48:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=z2Eui6jBMsF3Ela2LYf1dY1EBM7vCZblnAbR38nKIkI=; b=BVqYFG1epHvfEzIIJb0n58krM9 xnJTzqfsYLp/JBl9vrfbgqtBjvzlBYQybmClTjleQ0/gC+pgAFkRaFphTPExqCrYN+dXAjIe9isRM K09mxSJKJPhNe9iTfcsZMTiWLVQlGsQW7B8WC2g0WeBuOb8FqPCSsXfLB3l3rxdXyegK2w/bwmz6k mx4YSscqMTPvHDPfg2Gt9jBpfm0+SKncvir8OnW7z8nI88YiycDbTocKrevtF4ZV2eBUXbRaO7EZO 4Bchr3inxodksr2ZlK62noDmzNgrPPNWy6YoJst3TMhAaTAs07W7JJ5b4Yc+CnfdPRigs2QoWn3qJ Hm2cvT/A==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1ooVKR-007ARC-1D; Fri, 28 Oct 2022 19:47:43 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id BD18B300BAE; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id A37982C6DAB2D; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Message-ID: <20221028194453.526899822@infradead.org> User-Agent: quilt/0.66 Date: Fri, 28 Oct 2022 21:40:26 +0200 From: Peter Zijlstra To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, djwong@kernel.org, yujie.liu@intel.com, tglx@linutronix.de, jpoimboe@kernel.org, joao.moreira@intel.com, samitolvanen@google.com Subject: [PATCH 4/5] objtool: Add option to generate prefix symbols References: <20221028194022.388521751@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When code is compiled with: -fpatchable-function-entry=3D${PADDING_BYTES},${PADDING_BYTES} functions will have PADDING_BYTES of NOP in front of them. Unwinders and other things that symbolize code locations will typically attribute these bytes to the preceding function. Given that these bytes nominally belong to the following symbol this mis-attribution is confusing. Inspired by the fact that CFI_CLANG emits __cfi_##name symbols to claim these bytes, allow objtool to emit __pfx_##name symbols to do the same. Therefore add the objtool --prefix=3DN argument, to conditionally place a __pfx_##name symbol at N bytes ahead of symbol 'name' when: all these preceding bytes are NOP and name-N is an instruction boundary. Signed-off-by: Peter Zijlstra (Intel) --- tools/objtool/builtin-check.c | 1 + tools/objtool/check.c | 27 +++++++++++++++++++++++++++ tools/objtool/elf.c | 30 +++++++++++++++++++++++++++= +++ tools/objtool/include/objtool/builtin.h | 1 + tools/objtool/include/objtool/elf.h | 2 ++ 5 files changed, 61 insertions(+) --- a/tools/objtool/builtin-check.c +++ b/tools/objtool/builtin-check.c @@ -75,6 +75,7 @@ const struct option check_options[] =3D { OPT_BOOLEAN('r', "retpoline", &opts.retpoline, "validate and annotate ret= poline usage"), OPT_BOOLEAN(0, "rethunk", &opts.rethunk, "validate and annotate rethunk= usage"), OPT_BOOLEAN(0, "unret", &opts.unret, "validate entry unret placement"), + OPT_INTEGER(0, "prefix", &opts.prefix, "generate prefix symbols"), OPT_BOOLEAN('l', "sls", &opts.sls, "validate straight-line-speculation mi= tigations"), OPT_BOOLEAN('s', "stackval", &opts.stackval, "validate frame pointer rule= s"), OPT_BOOLEAN('t', "static-call", &opts.static_call, "annotate static calls= "), --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -3972,6 +3972,31 @@ static bool ignore_unreachable_insn(stru return false; } =20 +static int add_prefix_symbol(struct objtool_file *file, struct symbol *fun= c, + struct instruction *insn) +{ + if (!opts.prefix) + return 0; + + for (;;) { + struct instruction *prev =3D list_prev_entry(insn, list); + + if (&prev->list =3D=3D &file->insn_list) + break; + + if (prev->type !=3D INSN_NOP) + break; + + insn =3D prev; + if (func->offset - prev->offset =3D=3D opts.prefix) { + elf_create_prefix_symbol(file->elf, func, -opts.prefix); + break; + } + } + + return 0; +} + static int validate_symbol(struct objtool_file *file, struct section *sec, struct symbol *sym, struct insn_state *state) { @@ -3990,6 +4015,8 @@ static int validate_symbol(struct objtoo if (!insn || insn->ignore || insn->visited) return 0; =20 + add_prefix_symbol(file, sym, insn); + state->uaccess =3D sym->uaccess_safe; =20 ret =3D validate_branch(file, insn_func(insn), insn, *state); --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -819,6 +819,36 @@ elf_create_section_symbol(struct elf *el return sym; } =20 +static int elf_add_string(struct elf *elf, struct section *strtab, char *s= tr); + +struct symbol * +elf_create_prefix_symbol(struct elf *elf, struct symbol *orig, long addend) +{ + struct symbol *sym =3D calloc(1, sizeof(*sym)); + size_t namelen =3D strlen(orig->name) + sizeof("__pfx_"); + char *name =3D malloc(namelen); + + if (!sym || !name) { + perror("malloc"); + return NULL; + } + + snprintf(name, namelen, "__pfx_%s", orig->name); + + sym->name =3D name; + sym->sec =3D orig->sec; + + sym->sym.st_name =3D elf_add_string(elf, NULL, name); + sym->sym.st_info =3D orig->sym.st_info; + sym->sym.st_value =3D orig->sym.st_value + addend; + + sym =3D __elf_create_symbol(elf, sym); + if (sym) + elf_add_symbol(elf, sym); + + return sym; +} + int elf_add_reloc_to_insn(struct elf *elf, struct section *sec, unsigned long offset, unsigned int type, struct section *insn_sec, unsigned long insn_off) --- a/tools/objtool/include/objtool/builtin.h +++ b/tools/objtool/include/objtool/builtin.h @@ -26,6 +26,7 @@ struct opts { bool stackval; bool static_call; bool uaccess; + int prefix; =20 /* options: */ bool backtrace; --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -146,6 +146,8 @@ static inline bool has_multiple_files(st struct elf *elf_open_read(const char *name, int flags); struct section *elf_create_section(struct elf *elf, const char *name, unsi= gned int sh_flags, size_t entsize, int nr); =20 +struct symbol *elf_create_prefix_symbol(struct elf *elf, struct symbol *or= ig, long addend); + int elf_add_reloc(struct elf *elf, struct section *sec, unsigned long offs= et, unsigned int type, struct symbol *sym, s64 addend); int elf_add_reloc_to_insn(struct elf *elf, struct section *sec, From nobody Wed Apr 8 20:03:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DC2CECAAA1 for ; Fri, 28 Oct 2022 19:48:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229592AbiJ1TsL (ORCPT ); Fri, 28 Oct 2022 15:48:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229946AbiJ1Trz (ORCPT ); Fri, 28 Oct 2022 15:47:55 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 76BD6241B35 for ; Fri, 28 Oct 2022 12:47:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=dw2+KyAKk/Vgq7gh3A5Vj3COoPkFBdj0Z6avPUlckh4=; b=qeQgNOm135IQHespPW8tvobp0V xikJ7xB2qqPBPAkaBL+pXNXryXDzY40SdFMIATyVZY1JFlOA6t9rDpCCgLGBBIArPoXgT/POmZ7DT AewooojZW7SNctC8ZLtyopjhGxrOmc4I9PMiiwO3O2sNpz5oRqZTyHgS2GjvkZlPbUqCUAaqBM8DV 4zd6qANrbAaxe06rPPKQ2QJ+jwH62H44tVnTZU7+wXh/N4doeluoBp69JNo6Tlf/PgV5d/wj0Lnyj uH++5Q5Qy4/2Wj5P4dt2SM5aBTdgrcfVOQ2fgd4yqYGW8neDvfi//ilCMsWPX5oTpni4S3yt0SQrr JaTR6KGw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1ooVKW-001V60-V0; Fri, 28 Oct 2022 19:47:49 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 20B90300C19; Fri, 28 Oct 2022 21:47:42 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id A7BBD2C6DAB2E; Fri, 28 Oct 2022 21:47:41 +0200 (CEST) Message-ID: <20221028194453.592512209@infradead.org> User-Agent: quilt/0.66 Date: Fri, 28 Oct 2022 21:40:27 +0200 From: Peter Zijlstra To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, djwong@kernel.org, yujie.liu@intel.com, tglx@linutronix.de, jpoimboe@kernel.org, joao.moreira@intel.com, samitolvanen@google.com Subject: [PATCH 5/5] x86: Add prefix symbols for function padding References: <20221028194022.388521751@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When code is compiled with: -fpatchable-function-entry=3D${PADDING_BYTES},${PADDING_BYTES} functions will have PADDING_BYTES of NOP in front of them. Unwinders and other things that symbolize code locations will typically attribute these bytes to the preceding function. Given that these bytes nominally belong to the following symbol this mis-attribution is confusing. Inspired by the fact that CFI_CLANG emits __cfi_##name symbols to claim these bytes, use objtool to emit __pfx_##name symbols to do the same when CFI_CLANG is not used. This then shows the callthunk for symbol 'name' as: __pfx_##name+0x6/0x10 Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/Kconfig | 4 ++++ scripts/Makefile.lib | 1 + 2 files changed, 5 insertions(+) --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2472,6 +2472,10 @@ config CALL_THUNKS def_bool n select FUNCTION_ALIGNMENT_16B =20 +config PREFIX_SYMBOLS + def_bool y + depends on CALL_THUNKS && !CFI_CLANG + menuconfig SPECULATION_MITIGATIONS bool "Mitigations for speculative execution vulnerabilities" default y --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -265,6 +265,7 @@ objtool-args-$(CONFIG_STACK_VALIDATION) objtool-args-$(CONFIG_HAVE_STATIC_CALL_INLINE) +=3D --static-call objtool-args-$(CONFIG_HAVE_UACCESS_VALIDATION) +=3D --uaccess objtool-args-$(CONFIG_GCOV_KERNEL) +=3D --no-unreachable +objtool-args-$(CONFIG_PREFIX_SYMBOLS) +=3D --prefix=3D$(CONFIG_FUNCTION_= PADDING_BYTES) =20 objtool-args =3D $(objtool-args-y) \ $(if $(delay-objtool), --link) \ From nobody Wed Apr 8 20:03:34 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA0D4C4332F for ; Wed, 2 Nov 2022 21:47:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231311AbiKBVrJ (ORCPT ); Wed, 2 Nov 2022 17:47:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45510 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230315AbiKBVrG (ORCPT ); Wed, 2 Nov 2022 17:47:06 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8478AE0C0 for ; Wed, 2 Nov 2022 14:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=PAqLk4vbz/jMyf0acd8AaUtbY70n/pCagLl/I8+aAy4=; b=Mlqf86ujjgIDW8LACKKsN0DHlm UE/XS328m2Mynx6qIbbIMNH7OMcdrWxt1NG/yi2z8JTyqZAmBPPs6AOxF/FD/MwVvj35nvy4gdk9n vat5p8hyus8Q3AGatMGa6evGstcpSAOYACyj2J4oVg5iUD+gqQD0HYdN85JfhudFs3E64QY7cmn3Q C3GIL3A5aBislzBCL5ULSUHc7IjQqiJw1CsUCoRfGathVddINOSktcpEFfJ4IpzuUxxvkeWBMZPSE T85+0KXqR9+sNisVISO+i+m/lTBBOABPaGL0HvU83unLPNmTcjekGgMN1J0CJbe8CO9FrIJDDFDlk MN3S3f3A==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1oqLZO-008W7N-Cu; Wed, 02 Nov 2022 21:46:47 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id D1282300130; Wed, 2 Nov 2022 22:46:44 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id ADC7C20B23194; Wed, 2 Nov 2022 22:46:44 +0100 (CET) Date: Wed, 2 Nov 2022 22:46:44 +0100 From: Peter Zijlstra To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, djwong@kernel.org, yujie.liu@intel.com, tglx@linutronix.de, jpoimboe@kernel.org, joao.moreira@intel.com, samitolvanen@google.com Subject: [PATCH 6/5] objtool: Optimize elf_dirty_reloc_sym() Message-ID: References: <20221028194022.388521751@infradead.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20221028194022.388521751@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Subject: objtool: Optimize elf_dirty_reloc_sym() From: Peter Zijlstra Date: Wed Nov 2 22:31:19 CET 2022 When moving a symbol in the symtab its index changes and any reloc referring that symtol-table-index will need to be rewritten too. In order to facilitate this, objtool simply marks the whole reloc section 'changed' which will cause the whole section to be re-generated. However, finding the relocs that use any given symbol is implemented rather crudely -- a fully iteration of all sections and their relocs. Given that some builds have over 20k sections (kallsyms etc..) iterating all that for *each* symbol moved takes a bit of time. Instead have each symbol keep a list of relocs that reference it. This *vastly* improves build times for certain configs. Reported-by: Borislav Petkov Signed-off-by: Peter Zijlstra (Intel) --- tools/objtool/elf.c | 27 ++++++++++----------------- tools/objtool/include/objtool/elf.h | 2 ++ 2 files changed, 12 insertions(+), 17 deletions(-) --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -356,6 +356,7 @@ static void elf_add_symbol(struct elf *e struct rb_node *pnode; struct symbol *iter; =20 + INIT_LIST_HEAD(&sym->reloc_list); INIT_LIST_HEAD(&sym->pv_target); sym->alias =3D sym; =20 @@ -557,6 +558,7 @@ int elf_add_reloc(struct elf *elf, struc reloc->sym =3D sym; reloc->addend =3D addend; =20 + list_add_tail(&reloc->sym_reloc_entry, &sym->reloc_list); list_add_tail(&reloc->list, &sec->reloc->reloc_list); elf_hash_add(reloc, &reloc->hash, reloc_hash(reloc)); =20 @@ -573,21 +575,10 @@ int elf_add_reloc(struct elf *elf, struc */ static void elf_dirty_reloc_sym(struct elf *elf, struct symbol *sym) { - struct section *sec; - - list_for_each_entry(sec, &elf->sections, list) { - struct reloc *reloc; - - if (sec->changed) - continue; + struct reloc *reloc; =20 - list_for_each_entry(reloc, &sec->reloc_list, list) { - if (reloc->sym =3D=3D sym) { - sec->changed =3D true; - break; - } - } - } + list_for_each_entry(reloc, &sym->reloc_list, sym_reloc_entry) + reloc->sec->changed =3D true; } =20 /* @@ -902,11 +893,12 @@ static int read_rela_reloc(struct sectio =20 static int read_relocs(struct elf *elf) { + unsigned long nr_reloc, max_reloc =3D 0, tot_reloc =3D 0; struct section *sec; struct reloc *reloc; - int i; unsigned int symndx; - unsigned long nr_reloc, max_reloc =3D 0, tot_reloc =3D 0; + struct symbol *sym; + int i; =20 if (!elf_alloc_hash(reloc, elf->text_size / 16)) return -1; @@ -947,13 +939,14 @@ static int read_relocs(struct elf *elf) =20 reloc->sec =3D sec; reloc->idx =3D i; - reloc->sym =3D find_symbol_by_index(elf, symndx); + reloc->sym =3D sym =3D find_symbol_by_index(elf, symndx); if (!reloc->sym) { WARN("can't find reloc entry symbol %d for %s", symndx, sec->name); return -1; } =20 + list_add_tail(&reloc->sym_reloc_entry, &sym->reloc_list); list_add_tail(&reloc->list, &sec->reloc_list); elf_hash_add(reloc, &reloc->hash, reloc_hash(reloc)); =20 --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -62,6 +62,7 @@ struct symbol { u8 fentry : 1; u8 profiling_func : 1; struct list_head pv_target; + struct list_head reloc_list; }; =20 struct reloc { @@ -73,6 +74,7 @@ struct reloc { }; struct section *sec; struct symbol *sym; + struct list_head sym_reloc_entry; unsigned long offset; unsigned int type; s64 addend;