From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6999C3CC7D8; Wed, 23 Sep 2026 17:18:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183920; cv=none; b=pJOsMCwaK2pfWv5dtknnqEmRUNsxmyPSat5a/yi0w5wPH4cTNHp19/vFG8bCvlgWSC/F9vAVmOR2j3moah+YW3sldRHEZ3NqSnks5/Lwr2DXljbezbgUNao46H+sqS6gX8ekA+lmd/QYJ6CW+PJ6OXEa7CqKg2Ucsa+ZIbu0+QY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183920; c=relaxed/simple; bh=9qDVsxGAQgsnUcoMvP2AEPbuWlsxsglnmo0Z/jq5JDg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=c0vwAVRfaxKw3+SaEdwgH/OO74mN6ewlqlIKq0QO+CSb+tn5vSkTTg37/1LR68NPr/+CH4LjNr0aHPJgLjXWFhHFUt6bG0pMxPOhv/r0OS2k5MY0GfEAFNPuJeqALaK+KabiKe6tDvmTqGE+/fWnGka0MSIEPOetXqY9Mi17m40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bdyjpWmn; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bdyjpWmn" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2CAE21F00898; Wed, 23 Sep 2026 17:18:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790183919; bh=0HYR0xfc9+08B1bJ2OGJ3noSP/JxvewXJ1Rk/+81ZNA=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=bdyjpWmnQvnfs38AnUbgjGpq66148L14yskjCbI3o9t02aln67Hbe61SXQo6ZNBc2 /O4UXt9FBoT/pqbOa36FFTUnCWGOvTCsBMGmu9i+wwXq+CjaZeFiLkOzEPrMXEP6Jf dRFXPtHeMkfmWqImDRmVFJdcGqpLYsuMpyTzek/ejoYWqXzLbD+Qjju/f6fLgCJTxC 26vdo40S1eZ95wpEo76kVV5ooDfTnfbiWYYRP69Quq0yhIYgKLnJr7cGM+qdeRv0Ha Hj15cZE+wJS0kBHrSUm1HycC7zcc+JxwbM1KdmzreZAviVWnkqOYN+P2ZV0PGZaDx4 tzM0KLUod8hNA== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:50 +0100 Subject: [PATCH v4 01/22] kbuild: do not allocate .modinfo in vmlinux Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-1-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=3990; i=ljs@kernel.org; h=from:subject:message-id; bh=9qDVsxGAQgsnUcoMvP2AEPbuWlsxsglnmo0Z/jq5JDg=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF60nhF3mDVl84GMj0bca/SOLtcy+7Hyi+57m/kuj CIebhVfOkpZGMS4GGTFFFmefxHfHyQSNq/zgr8bzBxWJpAhDFycAjAR1/cMfzjFf1seuv9oUcDh dVUf5spm/7SY7fefx2mv+ZV3Vx2zG/wZ/ils9N0wySHp94zm+5cvTo6/aMg055f4S4vI3S/Y54j sfsIDAA== X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Commit 3e86e4d74c04 ("kbuild: keep .modinfo section in vmlinux.unstripped") placed MODULE_INFO() strings in the .modinfo section in vmlinux.unstripped, however it left the section allocatable, so it gets assigned an address and on arm64, arm and riscv, the section is tagged as PT_LOAD. This is useless as the data is ultimately stripped anyway. Doing this results in a lot of unnecessary work - each pass copies the whole file, 450 MiB with relocations for an x86 allmodconfig build and 250 MiB for an arm64 allmodconfig build. This adds ~0.7s on the serial tail of every build for arm64 and ~0.2s for x86 (the tail is single-threaded work done after parallel work has finished). Nothing requires .modinfo to exist at an address, so mark the output section (INFO) so that it's not allocated and use --dump-section to extract it as -O only emits allocated sections. The dump doesn't pick up the arch's OBJCOPYFLAGS any longer as they should not impact the output of this section. Image, bzImage, System.map and modules.builtin.modinfo are unchanged and the stripped vmlinux differs only in its program headers. The objcopy passes are on the serial tail of every build that links vmlinux, no-op builds are unchanged. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 11.2s 11.1s -0.15s (-1%) x86 defconfig, touch mm/vma.c, clang 11.3s 11.1s -0.19s (-2%) x86 defconfig, clean, gcc 32.5s 32.3s -0.16s (-1%) x86 defconfig, clean, clang 34.5s 34.2s -0.26s (-1%) x86 allmodconfig, touch mm/vma.c, gcc 40.1s 39.7s -0.44s (-1%) Assisted-by: LLM Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- include/asm-generic/vmlinux.lds.h | 2 +- scripts/Makefile.vmlinux | 10 ++-------- 2 files changed, 3 insertions(+), 9 deletions(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinu= x.lds.h index 26201821ea7f..a6730d34e8c6 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -855,7 +855,7 @@ KLP_SYMID =20 #define MODINFO \ - .modinfo : { *(.modinfo) } + .modinfo (INFO) : { *(.modinfo) } =20 #ifdef CONFIG_GENERIC_BUG #define BUG_TABLE \ diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux index 6833b517cacb..95b523fe4f9e 100644 --- a/scripts/Makefile.vmlinux +++ b/scripts/Makefile.vmlinux @@ -89,11 +89,8 @@ remove-section-$(CONFIG_ARCH_VMLINUX_NEEDS_RELOCS) +=3D = '.rel.*' =20 remove-symbols :=3D -w --strip-unneeded-symbol=3D'__mod_device_table__*' =20 -# To avoid warnings: "empty loadable segment detected at ..." from GNU obj= copy, -# it is necessary to remove the PT_LOAD flag from the segment. quiet_cmd_strip_relocs =3D OBJCOPY $@ - cmd_strip_relocs =3D $(OBJCOPY) $(patsubst %,--set-section-flags %= =3Dnoload,$(remove-section-y)) $< $@; \ - $(OBJCOPY) $(addprefix --remove-section=3D,$(remo= ve-section-y)) $(remove-symbols) $@ + cmd_strip_relocs =3D $(OBJCOPY) $(addprefix --remove-section=3D,$(re= move-section-y)) $(remove-symbols) $< $@ =20 targets +=3D vmlinux vmlinux: vmlinux.unstripped FORCE @@ -103,10 +100,7 @@ vmlinux: vmlinux.unstripped FORCE # ------------------------------------------------------------------------= --- =20 quiet_cmd_modules_builtin_modinfo =3D GEN $@ - cmd_modules_builtin_modinfo =3D $(cmd_objcopy); \ - chmod -x $@ - -OBJCOPYFLAGS_modules.builtin.modinfo :=3D -j .modinfo -O binary + cmd_modules_builtin_modinfo =3D $(OBJCOPY) -O binary -j .modinfo --d= ump-section .modinfo=3D$@ $< /dev/null =20 targets +=3D modules.builtin.modinfo modules.builtin.modinfo: vmlinux.unstripped FORCE --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D228D3DA7F1; Wed, 23 Sep 2026 17:18:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183933; cv=none; b=cA+FVbc+KKYSZSlpfiaAiR40VerLkmSUCBpEyuPEcviVo7JPvIF+39jxRpLB7Bb60Xl1lEgAU9B1XunvKC78fjk5NDXeEXNi8wkKdhvFc9dd6ec2vq0VivaOg7GMbW6lcTxv49JHHAr+fNQrd6pibW8foA1MRtjTYK5YGD1ZwYg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183933; c=relaxed/simple; bh=KFTL5Jl5Cc2tJgqQyEX9rMU2YFOh6AIwD9HraURgmNE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=u7VIfcgS0QNFqoxNOcA1TyQntq/zcIuIC0Fh4ZuqIOi6G7259sB2wI6TDciQx/i9kjjwLz1ZYkuvv7NFGZcmDYgUP9mQ3F40/CfCeODQ8R1fuhmkupuHa9605l5QqSgY/8paej8JpLNYqBYHWwGBjEPcq6rjHrHDuFUiNi1QO5o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PrNfxidi; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PrNfxidi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87F9D1F000FF; Wed, 23 Sep 2026 17:18:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790183931; bh=VstT85xW5t9bYLaC6vZl3M0+44tjpODycADfnONnakM=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=PrNfxidiJVanZmaN6hUFCyeAx4ca+DImuBqsv+FZysPE1efA+R97T8nJhoJiRJtjJ wHsLiVLAlqsQsOn/UuZ4GJ7xuF0oacxv6/bozxWTTOCct0N1+811Fvr2NAHDpQin4x GeDHj22wuUSuKk/Sk8hC8UQNRPf2f6NoGQfWfAOoAqfDdkNuvMFwgRiMgJwKFfVISQ MXbfWO/x0MdYusfQxyXNUcSegwNZtf3aaKpBVeS2bkxsk5LkuCmy6T7R/MRiy/uKFk hhK5iUrAWut9peM5hYok8s3L0hPbvc/vE6mhv7EDGsdfJfmgM5HJXlfNI3kyeWIYQ/ S73DOcnF84JoA== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:51 +0100 Subject: [PATCH v4 02/22] kallsyms: index symbols by token to speed up table compression Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-2-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=10545; i=ljs@kernel.org; h=from:subject:message-id; bh=KFTL5Jl5Cc2tJgqQyEX9rMU2YFOh6AIwD9HraURgmNE=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF6KmGvnz75426SoTWxvPs969OPSa+Urm9SvbIme9 O9b/9sO945SFgYxLgZZMUWW51/E9weJhM3rvODvBjOHlQlkCAMXpwBM5MBBRoa5asoxEVP512Rs 8y7m8+Eo8rX4fPnHvqRX32RfLzyqvlSS4Q//qt93r/eybOL4lCGkfXTuzQMVG59vchd+vvbteeH 1EqcZAQ== X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 The kallsyms program compresses symbols by figuring out the most commonly used substrings in all of the input symbols then uses special character codes to represent them. For instance, 0xf7 might end up representing "write_", then every single symbol that contains "write_" can use 0xf7 as a shorthand and save 5 bytes each time. 'Special' character codes are any byte value that is not used in any symbol, either due to being an invalid character, or not being present in any symbol (e.g. if no symbol contains 'z', then 'z' can be used as special character). It does this by first figuring out which special characters are available in insert_real_symbols_in_table(), then iterating through every available special character, counting how many times each pair of adjacent characters appear in symbols in build_initial_token_table(). These adjacent pairs are known as 'tokens'. Token counts are initially obtained by build_initial_token_table(), then optimize_result() calls find_best_token() to determine the token that appeared the most number of times and assigns it the next special character. Finally, optimize_result() calls compress_symbols() to replace every token in every symbol with its special character, which updates token_profit[] as it does so. This process is repeated for each remaining available special character, with tokens now perhaps containing previously assigned special characters (e.g. if 'wr' was assigned 0x80, then the token representing 'wri' would be '\x80i'). This compresses that token by 50% in each symbol it appears in (two bytes are now represented by one) and thus by repeatedly doing this kallsyms obtains good symbol compression. However, compress_symbols() is seriously inefficient - it iterates through EVERY symbol for EVERY special character assignment, i.e. ~256 * nr_symbols. Modern x86-64 kernels, for instance, have ~158,000 symbols, so millions of iterations are performed, most of which are entirely unnecessary (tokens don't appear in most symbols). In practice kallsyms spends half its runtime doing this, two or three times per vmlinux link step. Fix this by tracking which symbols each token appears in token_syms[], and only compress symbols which actually need to be updated. Each time a token is compressed that token can no longer appear in any symbol, so that token_syms[] entry can be freed. However new token_syms[] entries must be created for each new token containing the assigned special character, but this is bounded by the number of replacements in the symbol which is very small. In testing on an x86-64 platform using clang, each kallsyms invocation dropped from 0.59s to 0.33s with CONFIG_KALLSYMS_ALL set and from 0.38s to 0.22s without it set. The data was carefully checked and verified to be byte-for-byte identical for six symbol sets (two vmlinux passes, vmlinux.o, three userspace binaries) with all option combinations. As part of this change, additionally refactor the code to be a little easier to follow. kallsyms runs two or three times on the serial tail of every build that links vmlinux, no-op builds do not link and are unchanged. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 11.1s 10.6s -0.53s (-5%) x86 defconfig, touch mm/vma.c, clang 11.1s 10.5s -0.64s (-6%) x86 defconfig, clean, gcc 32.3s 31.3s -1.0s (-3%) x86 defconfig, clean, clang 34.2s 33.3s -0.89s (-3%) x86 allmodconfig, touch mm/vma.c, gcc 39.7s 38.7s -1.0s (-3%) x86 allmodconfig, touch mm/vma.c, clang 40.3s 39.1s -1.3s (-3%) Assisted-by: LLM Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- scripts/kallsyms.c | 143 +++++++++++++++++++++++++++++++++++++++++++++++--= ---- 1 file changed, 129 insertions(+), 14 deletions(-) diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 494852ade6d8..bb05de2aa580 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -58,12 +58,50 @@ static unsigned int table_size, table_cnt; static int all_symbols; static int pc_relative; =20 -static int token_profit[0x10000]; +/* A dynamic array of symbols, encoded by symbol index. */ +struct sym_arr { + unsigned int *sym_indexes; + unsigned int cnt, cap; +}; + +/* Every pair of bytes is a token. */ +#define NR_TOKENS 0x10000 + +static int token_profit[NR_TOKENS]; +static struct sym_arr token_syms[NR_TOKENS]; =20 /* the table that holds the result of the compression */ static unsigned char best_table[256][2]; static unsigned char best_table_len[256]; =20 +static unsigned int sym_arr_last(const struct sym_arr *arr) +{ + return arr->cnt ? arr->sym_indexes[arr->cnt - 1] : UINT_MAX; +} + +static void sym_arr_maybe_expand(struct sym_arr *arr) +{ + if (arr->cap > arr->cnt) + return; + + arr->cap =3D arr->cap ? arr->cap * 2 : 16; + arr->sym_indexes =3D xrealloc(arr->sym_indexes, + arr->cap * sizeof(*arr->sym_indexes)); +} + +static void sym_arr_add(struct sym_arr *arr, unsigned int sym_idx) +{ + sym_arr_maybe_expand(arr); + arr->sym_indexes[arr->cnt++] =3D sym_idx; +} + +static void sym_arr_free(struct sym_arr *arr) +{ + free(arr->sym_indexes); + arr->sym_indexes =3D NULL; + arr->cnt =3D 0; + arr->cap =3D 0; +} =20 static void usage(void) { @@ -458,6 +496,15 @@ static void write_src(void) printf("\n"); } =20 +static unsigned int token_index(unsigned char first, unsigned char second) +{ + return first + (second << 8); +} + +static unsigned int sym_token_index(const unsigned char *symbol, int first= _idx) +{ + return token_index(symbol[first_idx], symbol[first_idx + 1]); +} =20 /* table lookup compression functions */ =20 @@ -467,7 +514,7 @@ static void learn_symbol(const unsigned char *symbol, i= nt len) int i; =20 for (i =3D 0; i < len - 1; i++) - token_profit[ symbol[i] + (symbol[i + 1] << 8) ]++; + token_profit[sym_token_index(symbol, i)]++; } =20 /* decrease the count for all the possible tokens in a symbol */ @@ -476,16 +523,76 @@ static void forget_symbol(const unsigned char *symbol= , int len) int i; =20 for (i =3D 0; i < len - 1; i++) - token_profit[ symbol[i] + (symbol[i + 1] << 8) ]--; + token_profit[sym_token_index(symbol, i)]--; +} + +static void token_add_symbol(unsigned int token_idx, unsigned int sym_idx) +{ + struct sym_arr *arr =3D &token_syms[token_idx]; + + /* Symbol indexes kept in sorted order, check for duplicate. */ + if (sym_arr_last(arr) =3D=3D sym_idx) + return; + + sym_arr_add(arr, sym_idx); +} + +static void symbol_index_all_tokens(const unsigned char *symbol, int len, + unsigned int sym_idx) +{ + int i; + + for (i =3D 0; i < len - 1; i++) { + const unsigned int token_idx =3D sym_token_index(symbol, i); + + token_add_symbol(token_idx, sym_idx); + } +} + +/* + * The symbol just got compressed. The only parts of the symbol that chang= ed + * meaningfully are those containing the newly assigned compressed char, so + * index those. + */ +static void symbol_index_new_tokens(const unsigned char *symbol, int len, + unsigned int sym_idx, int compressed_chr) +{ + int i; + + for (i =3D 0; i < len - 1; i++) { + const unsigned int token_idx =3D sym_token_index(symbol, i); + + if (symbol[i] =3D=3D compressed_chr || + symbol[i + 1] =3D=3D compressed_chr) + token_add_symbol(token_idx, sym_idx); + } } =20 -/* do the initial token count */ static void build_initial_token_table(void) { unsigned int i; =20 for (i =3D 0; i < table_cnt; i++) learn_symbol(table[i]->sym, table[i]->len); + + /* + * The initial occurrence counts tell us exactly how much memory should + * be reserved for each token's symbol array. + */ + for (i =3D 0; i < ARRAY_SIZE(token_syms); i++) { + const int nr_syms =3D token_profit[i]; + + if (!nr_syms) + continue; + + token_syms[i].cap =3D nr_syms; + token_syms[i].sym_indexes =3D + xmalloc(nr_syms * sizeof(unsigned int)); + } + + /* For every symbol, index every token -> symbol it is present in. */ + for (i =3D 0; i < table_cnt; i++) + symbol_index_all_tokens(table[i]->sym, table[i]->len, i); } =20 static unsigned char *find_token(unsigned char *str, int len, @@ -502,27 +609,31 @@ static unsigned char *find_token(unsigned char *str, = int len, =20 /* replace a given token in all the valid symbols. Use the sampled symbols * to update the counts */ -static void compress_symbols(const unsigned char *str, int idx) +static void compress_symbols(const unsigned char *str, int compressed_chr) { - unsigned int i, len, size; + const unsigned int token_idx =3D sym_token_index(str, 0); + struct sym_arr *arr =3D &token_syms[token_idx]; + unsigned int sym_idx, j, len, size; unsigned char *p1, *p2; =20 - for (i =3D 0; i < table_cnt; i++) { + /* Iterate through all symbols this token is found in and compress. */ + for (j =3D 0; j < arr->cnt; j++) { + sym_idx =3D arr->sym_indexes[j]; =20 - len =3D table[i]->len; - p1 =3D table[i]->sym; + len =3D table[sym_idx]->len; + p1 =3D table[sym_idx]->sym; =20 /* find the token on the symbol */ p2 =3D find_token(p1, len, str); if (!p2) continue; =20 /* decrease the counts for this symbol's tokens */ - forget_symbol(table[i]->sym, len); + forget_symbol(table[sym_idx]->sym, len); =20 size =3D len; =20 do { - *p2 =3D idx; + *p2 =3D compressed_chr; p2++; size -=3D (p2 - p1); memmove(p2, p2 + 1, size); @@ -536,11 +647,15 @@ static void compress_symbols(const unsigned char *str= , int idx) =20 } while (p2); =20 - table[i]->len =3D len; + table[sym_idx]->len =3D len; =20 /* increase the counts for this symbol's new tokens */ - learn_symbol(table[i]->sym, len); + learn_symbol(table[sym_idx]->sym, len); + symbol_index_new_tokens(table[sym_idx]->sym, len, sym_idx, + compressed_chr); } + + sym_arr_free(arr); /* No symbol contains this token any more. */ } =20 /* search the token with the maximum profit */ @@ -551,7 +666,7 @@ static int find_best_token(void) bestprofit=3D-10000; best =3D 0; =20 - for (i =3D 0; i < 0x10000; i++) { + for (i =3D 0; i < ARRAY_SIZE(token_profit); i++) { if (token_profit[i] > bestprofit) { best =3D i; bestprofit =3D token_profit[i]; --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E3693E120B; Wed, 23 Sep 2026 17:19:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183946; cv=none; b=HyEBpzl7YPC8PzImeVmkKVBtrfBVSn8OeXILbWBip7thyIbtcxxwiJfvf5ORUPozU85q73RCxcE2q0SFy1pXyFG3+pP85/Oe5I+L+cLHBMfKnL7KKR0WRkMztJKObEkZi5UT5mjn1c4d4nfOOcwAgNBo7BCDlX/u1NBAIB2Fg+g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183946; c=relaxed/simple; bh=gZEYK17Idcl8NVa0SW7AyOgW50rm3aqvYdTeONXjR9Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Vlq77ifxu+/qRghH3c8O4fIv/qT4GXJln/Nc1kChRaS2Kpc1IgB2UNOk2WMqf1myYqi8TVnv3fPRCMy6vLmiQtyKqWlyAhswjni6JycI4kKZLtNG7vWC5xQ1ssjo2JNE4HoNSHzlQBSMeTYNHcokGC+/92QAb/NBES9shg2d0bA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=F8jHkdZx; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="F8jHkdZx" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3E231F00893; Wed, 23 Sep 2026 17:18:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790183943; bh=5V79d3XZb3TxVh4yBOF6BN3Hn1MWKhx5EdJyaUB1dKk=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=F8jHkdZx0rOJXVdma5kxOZWwUDujyYRJQ5TIxP/gCc3LuIBvqWcf1yTK2k6luUERr WiZCiStJGxC8xctVSbhJe7b9mLKR7DIho1OWq1FqvWQwTDNOTvNJFnwclBePJhbGIs V9sgZyM3uD9ctbxzGGMl6R7JSxNUnMg3PV/TIHtEo//DduW8G/jLg/e9H+oofUgHDX TSokwB6otMzGDSs7snuJctuGtA1g3nQBxDhJDn3CPMbwM8f1n+tlXaghl4ia+IxpmP 0/apcGsHbtjTLeLITkEvdhcRjXG7WMVksrUQhqc5mXBIGzHZBo3wPlftMWEzC+fJHx rt37DdFpDUYKw== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:52 +0100 Subject: [PATCH v4 03/22] kallsyms: output binary data to speed output and kallsyms assembly Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-3-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=8735; i=ljs@kernel.org; h=from:subject:message-id; bh=gZEYK17Idcl8NVa0SW7AyOgW50rm3aqvYdTeONXjR9Q=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF568altFsNBu52iaZOWhypIPXySv5fzbGfdyWMPw +5nai/82VHKwiDGxSArpsjy/Iv4/iCRsHmdF/zdYOawMoEMYeDiFICJLJdnZHixcO0LeZvfpb1K F40+PpVMXbH409OsO1crXe9lBatYNgcy/FNjLt8y75rKEpbkj59MpuQecrz/xUjDTUbffKVP8r+ IJ7wA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 kallsyms generates an assembly file that consists mostly of .byte entries containing compressed names, token strings and name-sorted sequence numbers. For an x86-64 build with 158k symbols that is a 37 MiB .S file which takes 0.57s to assemble each of the two to three times it is built over a kernel build. Each time it is generated it also takes kallsyms a similar amount of time to output it. Avoid this overhead by instead outputting this data as binary and importing it into the assembly using the .incbin directive. Tables that are wider than a byte remain part of the assembly to ensure endianness and relative relocations are performed correctly. With this change, the output assembly file shrinks from 37 MiB to 9.8 MiB, with a 2.6 MiB binary data file alongside it, and the object remains identical. The generated binary file is deleted correctly on build clean along with all other ephemeral data. On an x86-64 system with CONFIG_KALLSYMS_ALL set: before after delta scripts/kallsyms 0.24s 0.18s 0.06s assemble 0.57s 0.16s 0.41s Per kallsyms invocation/assembly, for a total of 0.47s time saving upon invocation. An incremental build on the same system was reduced from 11.15s to 9.65s, indicating a total of 1.5 seconds saved over the build. The kallsyms runs and their assembly are on the serial tail of every build that links vmlinux, no-op builds are unchanged. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 10.6s 9.7s -0.89s (-8%) x86 defconfig, touch mm/vma.c, clang 10.5s 9.3s -1.2s (-11%) x86 defconfig, clean, gcc 31.3s 30.3s -1.0s (-3%) x86 defconfig, clean, clang 33.3s 31.7s -1.6s (-5%) x86 allmodconfig, touch mm/vma.c, gcc 38.7s 37.2s -1.4s (-4%) x86 allmodconfig, touch mm/vma.c, clang 39.1s 36.4s -2.6s (-7%) Assisted-by: LLM Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- scripts/kallsyms.c | 84 ++++++++++++++++++++++++++++++++++++---------= ---- scripts/link-vmlinux.sh | 2 +- 2 files changed, 64 insertions(+), 22 deletions(-) diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index bb05de2aa580..b615db767b1d 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -5,7 +5,10 @@ * This software may be used and distributed according to the terms * of the GNU General Public License, incorporated herein by reference. * - * Usage: kallsyms [--all-symbols] in.map > out.S + * Usage: kallsyms [--all-symbols] [--pc-relative] in.map out.bin > out.S + * + * The byte tables go to out.bin and are pulled into out.S with .incb= in; + * wider tables stay assembler source for endianness and relocations. * * Table compression uses all the unused char codes on the symbols and * maps these to the most used substrings (tokens). For instance, it might @@ -105,7 +108,7 @@ static void sym_arr_free(struct sym_arr *arr) =20 static void usage(void) { - fprintf(stderr, "Usage: kallsyms [--all-symbols] in.map > out.S\n"); + fprintf(stderr, "Usage: kallsyms [--all-symbols] [--pc-relative] in.map o= ut.bin > out.S\n"); exit(1); } =20 @@ -322,6 +325,26 @@ static void output_label(const char *label) printf("%s:\n", label); } =20 +static long bin_pos(FILE *file) +{ + const long pos =3D ftell(file); + + if (pos < 0) { + perror("kallsyms: ftell"); + exit(EXIT_FAILURE); + } + + return pos; +} + +static void write_incbin(const char *filename, long start, long end) +{ + if (start >=3D end) + return; + + printf("\t.incbin \"%s\", %ld, %ld\n", filename, start, end - start); +} + /* uncompress a compressed symbol. When this function is called, the best = table * might still be compressed itself, so the function needs to be recursive= */ static int expand_symbol(const unsigned char *data, int len, char *result) @@ -374,11 +397,12 @@ static void sort_symbols_by_name(void) qsort(table, table_cnt, sizeof(table[0]), compare_names); } =20 -static void write_src(void) +static void write_src(FILE *out_bin_file, const char *out_bin_name) { - unsigned int i, k, off; + unsigned int i, off; unsigned int best_idx[256]; unsigned int *markers, markers_cnt; + long bin_start; char buf[KSYM_NAME_LEN]; =20 printf("\t.section .rodata, \"a\"\n"); @@ -393,6 +417,7 @@ static void write_src(void) markers =3D xmalloc(sizeof(*markers) * markers_cnt); =20 output_label("kallsyms_names"); + bin_start =3D bin_pos(out_bin_file); off =3D 0; for (i =3D 0; i < table_cnt; i++) { if ((i & 0xFF) =3D=3D 0) @@ -416,26 +441,24 @@ static void write_src(void) /* Encode length with ULEB128. */ if (table[i]->len <=3D 0x7F) { /* Most symbols use a single byte for the length. */ - printf("\t.byte 0x%02x", table[i]->len); + fputc(table[i]->len, out_bin_file); off +=3D table[i]->len + 1; } else { /* "Big" symbols use two bytes. */ - printf("\t.byte 0x%02x, 0x%02x", - (table[i]->len & 0x7F) | 0x80, - (table[i]->len >> 7) & 0x7F); + fputc((table[i]->len & 0x7F) | 0x80, out_bin_file); + fputc((table[i]->len >> 7) & 0x7F, out_bin_file); off +=3D table[i]->len + 2; } - for (k =3D 0; k < table[i]->len; k++) - printf(", 0x%02x", table[i]->sym[k]); + fwrite(table[i]->sym, 1, table[i]->len, out_bin_file); =20 /* * Now that we wrote out the compressed symbol name, restore the - * original name and print it in the comment. + * original name for the comments below. */ expand_symbol(table[i]->sym, table[i]->len, buf); strcpy((char *)table[i]->sym, buf); - printf("\t/* %s */\n", table[i]->sym); } + write_incbin(out_bin_name, bin_start, bin_pos(out_bin_file)); printf(".size kallsyms_names, . - kallsyms_names\n"); printf("\n"); =20 @@ -448,13 +471,15 @@ static void write_src(void) free(markers); =20 output_label("kallsyms_token_table"); + bin_start =3D bin_pos(out_bin_file); off =3D 0; for (i =3D 0; i < 256; i++) { best_idx[i] =3D off; expand_symbol(best_table[i], best_table_len[i], buf); - printf("\t.asciz\t\"%s\"\n", buf); + fwrite(buf, 1, strlen(buf) + 1, out_bin_file); off +=3D strlen(buf) + 1; } + write_incbin(out_bin_name, bin_start, bin_pos(out_bin_file)); printf(".size kallsyms_token_table, . - kallsyms_token_table\n"); printf("\n"); =20 @@ -487,12 +512,13 @@ static void write_src(void) =20 sort_symbols_by_name(); output_label("kallsyms_seqs_of_names"); - for (i =3D 0; i < table_cnt; i++) - printf("\t.byte 0x%02x, 0x%02x, 0x%02x\t/* %s */\n", - (unsigned char)(table[i]->seq >> 16), - (unsigned char)(table[i]->seq >> 8), - (unsigned char)(table[i]->seq >> 0), - table[i]->sym); + bin_start =3D bin_pos(out_bin_file); + for (i =3D 0; i < table_cnt; i++) { + fputc(table[i]->seq >> 16, out_bin_file); + fputc(table[i]->seq >> 8, out_bin_file); + fputc(table[i]->seq >> 0, out_bin_file); + } + write_incbin(out_bin_name, bin_start, bin_pos(out_bin_file)); printf("\n"); } =20 @@ -803,6 +829,9 @@ static void sort_symbols(void) =20 int main(int argc, char **argv) { + const char *out_bin_name; + FILE *out_bin_file; + while (1) { static const struct option long_options[] =3D { {"all-symbols", no_argument, &all_symbols, 1}, @@ -818,14 +847,27 @@ int main(int argc, char **argv) usage(); } =20 - if (optind >=3D argc) + if (optind + 2 !=3D argc) usage(); =20 + out_bin_name =3D argv[optind + 1]; + out_bin_file =3D fopen(out_bin_name, "w"); + if (!out_bin_file) { + perror(out_bin_name); + exit(EXIT_FAILURE); + } + read_map(argv[optind]); shrink_table(); sort_symbols(); optimize_token_table(); - write_src(); + write_src(out_bin_file, out_bin_name); + + /* Write errors are latched in the stream, so one check at the end covers= them all. */ + if (ferror(out_bin_file) || fclose(out_bin_file)) { + perror(out_bin_name); + exit(EXIT_FAILURE); + } =20 return 0; } diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index ab0b8125c8cb..e88604150d2c 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -132,7 +132,7 @@ kallsyms() fi =20 info KSYMS "${2}.S" - scripts/kallsyms ${kallsymopt} "${1}" > "${2}.S" + scripts/kallsyms ${kallsymopt} "${1}" "${2}.bin" > "${2}.S" =20 info AS "${2}.o" ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \ --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACD793E1CFD; Wed, 23 Sep 2026 17:19:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183957; cv=none; b=LW+dYAd+DbX4TiTr7Do3lI61QIKY26j7+INp3Qo891CdeFSlxxJw5LnHPMmjaLxd5f4wzkkQ9GsPfyeBpiVOL+uyMeT3swKluT0Rn+mw1B2s56ZP2bMHnHrkyCrmy1e4HqzcYXQn7IvTsBjfM6CIiETGKNWxl7YfQ6jf5E5Bmg8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183957; c=relaxed/simple; bh=wNUVaWGrUfq1J8fZ3VtZkSo4KZCUQiuLpouzmc9mPms=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=NtYPu0shXWlKb5BRc/mbLw/oekTakoG6uhQDJsJkOuPFVXdl1g20LV7wE+87maX1pfdsMukG30BE7h8YquoVU3j1oaLP/2w1uSqu1g2j0U1Y2iktJjPpY8g5ax25gAIp5XeQqu+wn9TBx/mMMhcNuKwemobDTM/glvmQemMPZgo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ajlS9Rgc; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ajlS9Rgc" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 533291F000FF; Wed, 23 Sep 2026 17:19:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790183956; bh=EwUTC03iMe3mqDQCwXS0ElhwJ195RFfSMSvYZMg9K5c=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=ajlS9RgcgJmPHoodht9q1Tjuf5/xG6VV5yZ0cMtOxjKRN40NqZx25yf2S1S46TbTx oxL+FAeAVUQPgrWhhpeL06lXnXe1S/qond5RIz1HEwTOZuD5t9JqOgfg7JjcHEBVaT i9JEHZPLF466KVBqI6pEXknFBlyCthgTa1D6vtH8CjUGlNJYO5+ocBLw+G2PJstRaB v2GnA5/E6Mwqt/ORFNEIVMJzqBncb9oR+8w5WNLjj+Qy/uoYYXVFFuSngalJiiZ39Y 26VOG4Fme44YuFKGl0jBz7vhNF4xbEu+vi+CFEZafSkhxd7KffQAzwp1GJZfn05/hq OZqSvZaZeGShQ== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:53 +0100 Subject: [PATCH v4 04/22] sorttable: parse nm output correctly Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-4-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2694; i=ljs@kernel.org; h=from:subject:message-id; bh=wNUVaWGrUfq1J8fZ3VtZkSo4KZCUQiuLpouzmc9mPms=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF7yMz8as2ltfPXBEIaTk5edNbUrLlkulXbQrz9Mn 6O992lyRykLgxgXg6yYIsvzL+L7g0TC5nVe8HeDmcPKBDKEgYtTACZyew4jw6bmxf1ON1g32D6L KxSzr41NTJnFaaTQn2nM7rPySN3BTwz/s180n1zKz7WxfIvY/3ueffm+MzeY/j/64e4/i7m3ck/ z8gMA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 sorttable reads a list of functions from vmlinux, provided by the output of nm, with the -S flag specified providing function sizes. It does so using fscanf(fp, "%16s %16s %c %*s\n", ...) reading address, size, type (a single character) and name. However, nm outputs no size for a symbol which has none, meaning that it misinterprets these entries - interpreting the type character as the size, and, if the name is a single character, ignores the newline and swallows the address of the next entry. Size-less single character names exist (e.g. the assemblers' loop counters left over as local absolute symbols): 0000000000000001 a i 000000000000000f a i 0000000000000050 a j 0000000000000052 a t 000000000000009f a i 0000000000000200 a i sorttable is looking for functions and these are not that, and with nm's output sorted, it just so happens to be that what's swallowed is never a function. However the next commit in this series stops sorting nm's output, so this bug can results in function names being lost. Fix the issue by using '%*[^\n]' in fscanf() rather than '%*s' such that the scan cannot move past the newline. Then, compare the address and size fields (which are always padded to the same width if output by nm) - if they differ, then this is a misread, so simply discard the entry. It's fine to discard these, as sorttable's raison d'=C3=AAtre is to discover whether a call site is contained within a function and an empty function can't contain anything. Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- scripts/sorttable.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/scripts/sorttable.c b/scripts/sorttable.c index d7b50581c732..88e49a5c4251 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -322,10 +322,23 @@ static int parse_symbols(const char *fname) return -1; } =20 - while (fscanf(fp, "%16s %16s %c %*s\n", addr_str, size_str, &type) =3D=3D= 3) { + while (fscanf(fp, "%16s %16s %c%*[^\n]", addr_str, size_str, &type) =3D= =3D 3) { uint64_t addr; uint64_t size; =20 + /* + * nm outputs size-less entries with a missing 2nd value, which + * means fscanf() just misread its fields. + * + * These don't matter as a call site cannot be in an empty + * function, so just skip them. + * + * nm pads address and size to the same width, so if their + * widths differ, this is a misread. + */ + if (strlen(size_str) !=3D strlen(addr_str)) + continue; + /* Only care about functions */ if (type !=3D 't' && type !=3D 'T' && type !=3D 'W') continue; --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 748CC3E1738; Wed, 23 Sep 2026 17:19:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183970; cv=none; b=mlRfrvFZhmPFDSM9x5GQbSR8R+YCg28JXNRkrUGcCeAgQeT+hLlszmtRWHlPfZpRwXb+nTnmUoTq7azgW9Fl+BkWALTHnUcYMFdn7jspUAqE9EQDnKpmTQGfcYSKz40YtyR0n5e18cnTaoSROVPwaLiatm5OvSrNbcntVzvb+NY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183970; c=relaxed/simple; bh=MLUfGBmyVIicidewYxyd0wSyemJP9IVOSHNNzSMSLro=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=g4RbC0TOOr/iCGzM5JOHEe56oP9g/y6p1mS91SkRgBLTuT0fymEHp0++SE01Lmhjji/j1U1F/+3WkgFQ3Tj6MDhh97rrmz69BYq5UIxSW8yoGGI2S2rEaSKNl+Ktcqg0Vd//Htc6Emh4w/HlsLu1LIaAGHqqHgTu4LxbkJqfEBI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IdX2Jlv2; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IdX2Jlv2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE6B51F00893; Wed, 23 Sep 2026 17:19:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790183968; bh=Rr99y5k7+go2x3zYNVayaOE+WWHPPjrM2nwS9MChYr0=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=IdX2Jlv2q9z4c18/1je4x2BTqsOkbTh+Qltdmxsno7HZAIpDyNFFU0CFcnlFf95va 0k3k35Ze7aMEr7uQIIbNrS9W2tPkFiBb3GohT9Z7bTZs9YMoQFGnjbTjAuQABHMAPN nVq9sTjBGCox1kmTyBgfpNUHD0IjWyHKy8js1n08rN/dWVKvCdTNUnsPzmT63Ry1Pa uYM00Op9MLN/4mWp8vDLNJALjCIq3bmDEA29TE2UOTjd3Gbd5UVMATWLZMRlfmA5YX eDT12AYno1FVKahrIU3ZPiWwvb1wyXilIbovcfnNBbMXd0C4Wpa/X9ocugwn7VY7BF 2bSTibPzT/ALw== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:54 +0100 Subject: [PATCH v4 05/22] kbuild: do not sort nm output where the order is irrelevant Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-5-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" , Nicolas Schier X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=5086; i=ljs@kernel.org; h=from:subject:message-id; bh=MLUfGBmyVIicidewYxyd0wSyemJP9IVOSHNNzSMSLro=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF5ac2pzhUtH/aYM2S3FiwqvzVXaJ/2sVGr9XqPy4 qR/Nxb86ChlYRDjYpAVU2R5/kV8f5BI2LzOC/5uMHNYmUCGMHBxCsBE4lcx/NPbOHNVy0KPdcrM FhdEbFuTnSYx2lwPe/burntDcOmqr7IMf3j8l79wD1qtMuF//kzbBdqmV7+1ladn8B5m2f+lYCL /XAYA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Unless instructed otherwise, nm sorts by name. There are places where this is unnecessary - an invocation from scripts/sorttable every vmlinux link, scripts/check-function-names.sh run after every vmlinux.o link, and the x86 VOFFSET and ZOFFSET listings between vmlinux and bzImage. Both GNU nm and llvm-nm accept the same '-p' parameter to disable sorting in these instances, so use that to prevent this unnecessary work. Each nm run on its own, x86-64, median of 5: GNU nm 2.47 llvm-nm 22 before after delta before after delta defconfig sorttable, nm -S vmlinux 0.080s 0.039s -0.041s 0.265s 0.113s -0.152s check-function-names.sh 0.072s 0.033s -0.039s 0.244s 0.102s -0.142s VOFFSET 0.077s 0.034s -0.043s 0.261s 0.113s -0.148s ZOFFSET 0.007s 0.005s -0.002s 0.005s 0.005s 0.000s TOTAL -0.125s -0.442s allmodconfig sorttable, nm -S vmlinux 0.156s 0.063s -0.093s 0.539s 0.223s -0.316s check-function-names.sh 0.149s 0.054s -0.095s 0.537s 0.217s -0.320s VOFFSET 0.146s 0.052s -0.094s 0.524s 0.221s -0.303s ZOFFSET 0.007s 0.005s -0.002s 0.006s 0.004s -0.002s TOTAL -0.284s -0.941s llvm-nm appears to be a lot slower than GNU nm, so these builds naturally improve the most. The four run one after another in the serial tail of every build that links vmlinux so impact kernel builds directly. In an allmodconfig the decompressor is built in parallel while the modules, so only the two before the vmlinux link contribute to build time. The build outputs remain unchanged. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta --------------------------------- x86 defconfig, touch mm/vma.c, gcc 9.7s 9.5s -0.18s (-2%) x86 defconfig, touch mm/vma.c, clang 9.3s 9.0s -0.27s (-3%) x86 defconfig, clean, gcc 30.3s 30.1s -0.16s (-1%) x86 defconfig, clean, clang 31.7s 31.2s -0.55s (-2%) x86 allmodconfig, touch mm/vma.c, clang 36.4s 35.8s -0.67s (-2%) Assisted-by: LLM Reviewed-by: Nicolas Schier Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- arch/x86/boot/Makefile | 2 +- arch/x86/boot/compressed/Makefile | 2 +- scripts/check-function-names.sh | 3 ++- scripts/link-vmlinux.sh | 2 +- 4 files changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/boot/Makefile b/arch/x86/boot/Makefile index 3f9fb3698d66..4a9bce32586d 100644 --- a/arch/x86/boot/Makefile +++ b/arch/x86/boot/Makefile @@ -74,7 +74,7 @@ SETUP_OBJS =3D $(addprefix $(obj)/,$(setup-y)) sed-zoffset :=3D -e 's/^\([0-9a-fA-F]*\) [a-zA-Z] \(startup_32\|efi.._stub= _entry\|efi\(32\)\?_pe_entry\|input_data\|kernel_info\|_end\|_ehead\|_text\= |_e\?data\|_e\?sbat\|z_.*\)$$/\#define ZO_\2 0x\1/p' =20 quiet_cmd_zoffset =3D ZOFFSET $@ - cmd_zoffset =3D $(NM) $< | sed -n $(sed-zoffset) > $@ + cmd_zoffset =3D $(NM) -p $< | sed -n $(sed-zoffset) > $@ =20 targets +=3D zoffset.h $(obj)/zoffset.h: $(obj)/compressed/vmlinux FORCE diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/M= akefile index 06934f9691d6..6ec5e031db1d 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -76,7 +76,7 @@ HOST_EXTRACFLAGS +=3D -I$(srctree)/tools/include sed-voffset :=3D -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_r= odata\|_sinittext\|__inittext_end\|__bss_start\|_end\)$$/\#define VO_\2 _AC= (0x\1,UL)/p' =20 quiet_cmd_voffset =3D VOFFSET $@ - cmd_voffset =3D $(NM) $< | sed -n $(sed-voffset) > $@ + cmd_voffset =3D $(NM) -p $< | sed -n $(sed-voffset) > $@ =20 targets +=3D ../voffset.h =20 diff --git a/scripts/check-function-names.sh b/scripts/check-function-names= .sh index 08071133e5a5..94883e690627 100755 --- a/scripts/check-function-names.sh +++ b/scripts/check-function-names.sh @@ -13,7 +13,8 @@ if [ ! -f "$objfile" ]; then exit 1 fi =20 -bad_symbols=3D$(${NM:-nm} "$objfile" | awk '$2 ~ /^[TtWw]$/ {print $3}' | = grep -E '^(startup|exit|split|unlikely|hot|unknown)(\.|$)') +bad_symbols=3D$(${NM:-nm} -p "$objfile" | awk '$2 ~ /^[TtWw]$/ {print $3}'= | + grep -E '^(startup|exit|split|unlikely|hot|unknown)(\.|$)') =20 if [ -n "$bad_symbols" ]; then echo "$bad_symbols" | while read -r sym; do diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index e88604150d2c..970ca10f8fa9 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -161,7 +161,7 @@ mksysmap() =20 sorttable() { - ${NM} -S ${1} > .tmp_vmlinux.nm-sort + ${NM} -p -S ${1} > .tmp_vmlinux.nm-sort ${objtree}/scripts/sorttable -s .tmp_vmlinux.nm-sort ${1} } =20 --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAF8F3E1CFD; Wed, 23 Sep 2026 17:19:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183983; cv=none; b=TLB/saRPZ1bigKVVRVBOuNtjaysqzok5sGf/pQNXMKNh6GPG7JDMpgI6L11zaep6TKBFsBSQDReiKXu1S1dKi9Iqovpb86s4urfr36x9ty3e/APoP1x2FzBI9GGDi9aM3yVTfbkQu8t+rxc4uu+m/w0EuZo1l4Jw2n+1tcQ6xxY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183983; c=relaxed/simple; bh=6BzAunaFfBZ7gO2YMPYZ/5W5YNHZJ9BTFNzQOzFfBSA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=EnZ+mYrHRJdNgRyke9iMwXfoBHODsmowJa6A1dOsEoaZ9LWFCx5txOt42winobWXQlA730AUnvyhLTgUeJCGfE+n3XSR+JS+B4ZmUXttL/ovvKtYRBFy3SK3KOWcpIlwttPVm3XnBlsoOU6fFd/St61y7EGi/GK2mhUrYG3ncHA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lW5fEQ4O; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lW5fEQ4O" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 617011F000FF; Wed, 23 Sep 2026 17:19:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790183981; bh=SRKYqnyNBmtchL3rolqFIvqHHPLl/9DGnfYhpvyiVi8=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=lW5fEQ4On7/kq8ci+4WOZiJbxQ4c1uyXVR/IK/Q8bH8V0tQbq0gTdTb7tvAqGXL4E DrmGlJWxlLST4ATLMs/N5W+0QSwMrz4sEV+D7bNRXxySn/8uFHPbH6xKy9iq1oUL8C O+s2nWqVl2j7FL3hfnok5PimgWGrLqcVBJI8PfSfzXTegoE33b7ZfBY1quDxzGCc6x lSoK/PE4dm199Zm1Leb1h2VLBA9dX/s4mV+ginjELoUjBwtPOzGT092OKnaU0shGH6 z3XGphgdwnBeqUFchtDQ86+kLMR3BTjxSJ01cONmgIzaBAlLRpEnazqYbOYaCceHI9 rZ+CTMpkXmDHQ== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:55 +0100 Subject: [PATCH v4 06/22] kbuild: only emit vmlinux relocations when required Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-6-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" , Nicolas Schier X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2549; i=ljs@kernel.org; h=from:subject:message-id; bh=6BzAunaFfBZ7gO2YMPYZ/5W5YNHZJ9BTFNzQOzFfBSA=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF5SkRSoW5Cvc6Wp9+Gp1xsmO6p13Pho9YVZ5cXuJ NVawfXzOkpZGMS4GGTFFFmefxHfHyQSNq/zgr8bzBxWJpAhDFycAjCRWmdGhpUsYjMKrP8FSl5Y KLNLOW3hZrHWvb0quTFiH2N5JprLSTH8j5Xh1l24aymv1Jq21M/rkt433n4XOVnx1K3zv2tObdR +zQkA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 A kernel build consists of more than one linking pass on vmlinux.o and vmlinux, at minimum two, and with CONFIG_KALLSYMS and BTF enabled on x86-64 for example there are 5 such stages. For architectures that build their own relocation tables (x86, riscv, mips, s390), vmlinux is linked with the --emit-relocs parameter specified. However, this is only required on the final vmlinux link. Symbol tables of trial links preceding it don't need it because they already check that System.map matches kallsyms symbols on each build. GNU ld is slow at emitting relocation tables, so this results in a reduction in build time. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 9.5s 9.1s -0.42s (-4%) x86 defconfig, clean, gcc 30.1s 29.5s -0.59s (-2%) x86 allmodconfig, touch mm/vma.c, gcc 37.2s 35.8s -1.4s (-4%) Note that this has little impact on LLVM ld which performs this operation more efficiently. Assisted-by: LLM Reviewed-by: Nicolas Schier Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- Makefile | 3 ++- scripts/link-vmlinux.sh | 5 +++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/Makefile b/Makefile index f6745d723f6d..ab3ba78376f7 100644 --- a/Makefile +++ b/Makefile @@ -1265,8 +1265,9 @@ ifdef CONFIG_LD_ORPHAN_WARN LDFLAGS_vmlinux +=3D --orphan-handling=3D$(CONFIG_LD_ORPHAN_WARN_LEVEL) endif =20 +# --emit-relocs is added by scripts/link-vmlinux.sh, for the final link on= ly. ifneq ($(CONFIG_ARCH_VMLINUX_NEEDS_RELOCS),) -LDFLAGS_vmlinux +=3D --emit-relocs --discard-none +LDFLAGS_vmlinux +=3D --discard-none endif =20 # Align the architecture of userspace programs with the kernel diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 970ca10f8fa9..ed5ff97c0667 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -96,6 +96,11 @@ vmlinux_link() ldflags=3D"${ldflags} ${wl}--strip-debug" fi =20 + # Only the final link actually requires the relocations. + if [ "${output}" =3D "${VMLINUX}" ] && is_enabled CONFIG_ARCH_VMLINUX_NEE= DS_RELOCS; then + ldflags=3D"${ldflags} ${wl}--emit-relocs" + fi + if [ -n "${generate_map}" ]; then ldflags=3D"${ldflags} ${wl}-Map=3Dvmlinux.map" fi --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60CDA3E3D86; Wed, 23 Sep 2026 17:19:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183995; cv=none; b=lwy6OVBnDRTP7L5ruC8yBmIJxZZEMRXyDyaEZPxftSN7LjXPevF6Hlzw92cJsLdUVFSC4Czh26zkAe1Sw9/RLyFJfFEeUQrfqtNIAt1ooGFefyVBuvQx3nxYJDccC7nh5XVF/Sxevs/aYNo1awtD9b5sKegCt54sdICV8zZH+Dg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790183995; c=relaxed/simple; bh=9hDjEg13EL4FExzQs0DxiR/XhX7dDNwNUMkWv7ScVTs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TAifhQEgmGE5g+hhGgeRtfPYqDXlLBGMc1Qn+fkz9crjOV6BR0ZolpNkoEccYt5XCiUhUeh3k1XzFzEV64kYi7lD4kUrduBKyU390jT1gWt9L0vS3ARVwFmekefSVJG8TNMhmt2fezGHePX/ol7+VS+067OEeSzFeCxW+5/7/28= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MLMsKyzq; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MLMsKyzq" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0AE51F00893; Wed, 23 Sep 2026 17:19:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790183994; bh=4krp5buGr3rR5dSLar2s7FglGNHp9YTI3G1tX/l0RKk=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=MLMsKyzqMIeBr0SN/dLWueI2/s9AiF+y3GfTKqerkq2RIP6oxDn1b532GYLP+Bd47 IfXrIwVbPNbbSoDMEYuXIKVQBDPtRVxScqfiAShicf2ky4HSmDa7YFTlKYd4qi9vlJ NI3lFEdhVdxHBUt9+XxQ5Xy8861CASog0Zd9VJrSKQrBqZS7M4MMddznDZI8TNetsH jy4UeUfBxI61tVj89+g3yJITd7Ks2sxjB/HqcMTqrU47cXttV1lg3AOOX7qEp8vZxd /RIYjh9bSz0fJFozrjabp9OU9LLUioPR0pBYU3zhWB6WJAS/83BvVR3LgIiOTzRw7u 9kMHfhhpZ9QaQ== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:56 +0100 Subject: [PATCH v4 07/22] elf-parse: add section flags, symbol binding and mapping helpers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-7-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9261; i=ljs@kernel.org; h=from:subject:message-id; bh=9hDjEg13EL4FExzQs0DxiR/XhX7dDNwNUMkWv7ScVTs=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF6uDMr7dJm5T8ggJXPXGnEFsSkvBS81fz43x7Fx3 ycvPdmWjlIWBjEuBlkxRZbnX8T3B4mEzeu84O8GM4eVCWQIAxenAEykI4jhv0vld5afe0KNglaU SqbuW3S/fKvp1YrpMtypmfw6jqcbfBgZJu6Q+X6aa8bJHI5nNhtVj4o/fMbgFj69ru5J4aa50RY HuAA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Extend elf-parse to be able to read the symbol table of vmlinux in kallsyms. This requires the ability to open ELF files in read-only mode, so provide elf_map_ro() to do so. It also requires accessors for section flags and symbol bindings, so provide these via shdr_flags() and sym_bind(). It's useful to be able to open an ELF file, map it, and then have easy access to its section headers, names and symbol table, so introduce struct elf_file to store this. In order to obtain this data also provide elf_open_ro() and elf_close(). Also, check for the file being an ELF file first in elf_parse(). This is the logical thing to check for first, but additionally prevents kallsyms from having to check this it self. Assisted-by: LLM Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- scripts/elf-parse.c | 103 +++++++++++++++++++++++++++++++++++++++++++++---= ---- scripts/elf-parse.h | 54 +++++++++++++++++++++++++++ 2 files changed, 144 insertions(+), 13 deletions(-) diff --git a/scripts/elf-parse.c b/scripts/elf-parse.c index 99869ff91a8c..ce1c2a670244 100644 --- a/scripts/elf-parse.c +++ b/scripts/elf-parse.c @@ -17,15 +17,21 @@ struct elf_funcs elf_parser; * Get the whole file as a programming convenience in order to avoid * malloc+lseek+read+free of many pieces. If successful, then mmap * avoids copying unused pieces; else just read the whole file. - * Open for both read and write. + * Open for both read and write if writable is true, otherwise open + * read-only. */ -static void *map_file(char const *fname, size_t *size) +static void *map_file(char const *fname, size_t *size, bool writable) { - int fd; + int fd, prot =3D PROT_READ, flags =3D MAP_PRIVATE; struct stat sb; void *addr =3D NULL; =20 - fd =3D open(fname, O_RDWR); + if (writable) { + prot |=3D PROT_WRITE; + flags =3D MAP_SHARED; + } + + fd =3D open(fname, writable ? O_RDWR : O_RDONLY); if (fd < 0) { perror(fname); return NULL; @@ -39,7 +45,7 @@ static void *map_file(char const *fname, size_t *size) goto out; } =20 - addr =3D mmap(0, sb.st_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0); + addr =3D mmap(0, sb.st_size, prot, flags, fd, 0); if (addr =3D=3D MAP_FAILED) { fprintf(stderr, "Could not mmap file: %s\n", fname); goto out; @@ -57,6 +63,12 @@ static int elf_parse(const char *fname, void *addr, uint= 32_t types) Elf_Ehdr *ehdr =3D addr; uint16_t type; =20 + if (memcmp(ELFMAG, ehdr->e32.e_ident, SELFMAG) !=3D 0 || + ehdr->e32.e_ident[EI_VERSION] !=3D EV_CURRENT) { + fprintf(stderr, "unrecognized ELF file %s\n", fname); + return -1; + } + switch (ehdr->e32.e_ident[EI_DATA]) { case ELFDATA2LSB: elf_parser.r =3D rle; @@ -78,12 +90,6 @@ static int elf_parse(const char *fname, void *addr, uint= 32_t types) return -1; } =20 - if (memcmp(ELFMAG, ehdr->e32.e_ident, SELFMAG) !=3D 0 || - ehdr->e32.e_ident[EI_VERSION] !=3D EV_CURRENT) { - fprintf(stderr, "unrecognized ELF file %s\n", fname); - return -1; - } - type =3D elf_parser.r2(&ehdr->e32.e_type); if (!((1 << type) & types)) { fprintf(stderr, "Invalid ELF type file %s\n", fname); @@ -103,7 +109,9 @@ static int elf_parse(const char *fname, void *addr, uin= t32_t types) elf_parser.shdr_name =3D shdr32_name; elf_parser.shdr_type =3D shdr32_type; elf_parser.shdr_entsize =3D shdr32_entsize; + elf_parser.shdr_flags =3D shdr32_flags; elf_parser.sym_type =3D sym32_type; + elf_parser.sym_bind =3D sym32_bind; elf_parser.sym_name =3D sym32_name; elf_parser.sym_value =3D sym32_value; elf_parser.sym_shndx =3D sym32_shndx; @@ -133,7 +141,9 @@ static int elf_parse(const char *fname, void *addr, uin= t32_t types) elf_parser.shdr_name =3D shdr64_name; elf_parser.shdr_type =3D shdr64_type; elf_parser.shdr_entsize =3D shdr64_entsize; + elf_parser.shdr_flags =3D shdr64_flags; elf_parser.sym_type =3D sym64_type; + elf_parser.sym_bind =3D sym64_bind; elf_parser.sym_name =3D sym64_name; elf_parser.sym_value =3D sym64_value; elf_parser.sym_shndx =3D sym64_shndx; @@ -174,12 +184,13 @@ int elf_map_long_size(void *addr) return ehdr->e32.e_ident[EI_CLASS] =3D=3D ELFCLASS32 ? 4 : 8; } =20 -void *elf_map(char const *fname, size_t *size, uint32_t types) +static void *__elf_map(char const *fname, size_t *size, uint32_t types, + bool writable) { void *addr; int ret; =20 - addr =3D map_file(fname, size); + addr =3D map_file(fname, size, writable); if (!addr) return NULL; =20 @@ -192,7 +203,73 @@ void *elf_map(char const *fname, size_t *size, uint32_= t types) return addr; } =20 +void *elf_map(char const *fname, size_t *size, uint32_t types) +{ + return __elf_map(fname, size, types, true); +} + +/* For tools that only read the file. */ +void *elf_map_ro(char const *fname, size_t *size, uint32_t types) +{ + return __elf_map(fname, size, types, false); +} + void elf_unmap(void *addr, size_t size) { munmap(addr, size); } + +/* + * Open an ELF file and map it read-only, locating its section headers, se= ction + * names and symbol table which are populated in the elf out parameter. + * + * Returns 0 on success, otherwise -1. + */ +int elf_open_ro(char const *fname, uint32_t types, struct elf_file *elf) +{ + Elf_Ehdr *ehdr; + Elf_Shdr *first; + unsigned int shstrndx, i; + + memset(elf, 0, sizeof(*elf)); + elf->base =3D elf_map_ro(fname, &elf->size, types); + if (!elf->base) + return -1; + + ehdr =3D elf->base; + elf->shdrs =3D (const char *)elf->base + ehdr_shoff(ehdr); + elf->shentsize =3D ehdr_shentsize(ehdr); + first =3D elf_section(elf, 0); + + /* A count or index too large for the header lives in section 0. */ + elf->shnum =3D ehdr_shnum(ehdr); + if (elf->shnum =3D=3D SHN_UNDEF) + elf->shnum =3D shdr_size(first); + shstrndx =3D ehdr_shstrndx(ehdr); + if (shstrndx =3D=3D SHN_XINDEX) + shstrndx =3D shdr_link(first); + elf->shstrtab =3D (const char *)elf->base + + shdr_offset(elf_section(elf, shstrndx)); + + for (i =3D 0; i < elf->shnum && !elf->symtab; i++) + if (shdr_type(elf_section(elf, i)) =3D=3D SHT_SYMTAB) + elf->symtab =3D elf_section(elf, i); + + if (!elf->symtab) { + fprintf(stderr, "%s: no symbol table\n", fname); + elf_close(elf); + return -1; + } + + elf->strtab =3D (const char *)elf->base + + shdr_offset(elf_section(elf, shdr_link(elf->symtab))); + elf->nr_syms =3D shdr_size(elf->symtab) / shdr_entsize(elf->symtab); + + return 0; +} + +void elf_close(struct elf_file *elf) +{ + elf_unmap(elf->base, elf->size); + elf->base =3D NULL; +} diff --git a/scripts/elf-parse.h b/scripts/elf-parse.h index f4411e03069d..c5ad754da254 100644 --- a/scripts/elf-parse.h +++ b/scripts/elf-parse.h @@ -37,10 +37,12 @@ struct elf_funcs { uint64_t (*shdr_offset)(Elf_Shdr *shdr); uint64_t (*shdr_size)(Elf_Shdr *shdr); uint64_t (*shdr_entsize)(Elf_Shdr *shdr); + uint64_t (*shdr_flags)(Elf_Shdr *shdr); uint32_t (*shdr_link)(Elf_Shdr *shdr); uint32_t (*shdr_name)(Elf_Shdr *shdr); uint32_t (*shdr_type)(Elf_Shdr *shdr); uint8_t (*sym_type)(Elf_Sym *sym); + uint8_t (*sym_bind)(Elf_Sym *sym); uint32_t (*sym_name)(Elf_Sym *sym); uint64_t (*sym_value)(Elf_Sym *sym); uint16_t (*sym_shndx)(Elf_Sym *sym); @@ -143,6 +145,7 @@ SHDR_ADDR(addr) SHDR_ADDR(offset) SHDR_ADDR(size) SHDR_ADDR(entsize) +SHDR_ADDR(flags) =20 SHDR_WORD(link) SHDR_WORD(name) @@ -211,6 +214,21 @@ static inline uint8_t sym_type(Elf_Sym *sym) return elf_parser.sym_type(sym); } =20 +static inline uint8_t sym64_bind(Elf_Sym *sym) +{ + return ELF64_ST_BIND(sym->e64.st_info); +} + +static inline uint8_t sym32_bind(Elf_Sym *sym) +{ + return ELF32_ST_BIND(sym->e32.st_info); +} + +static inline uint8_t sym_bind(Elf_Sym *sym) +{ + return elf_parser.sym_bind(sym); +} + SYM_ADDR(value) SYM_WORD(name) SYM_HALF(shndx) @@ -298,8 +316,44 @@ static inline void w8le(uint64_t val, uint64_t *x) } =20 void *elf_map(char const *fname, size_t *size, uint32_t types); +void *elf_map_ro(char const *fname, size_t *size, uint32_t types); void elf_unmap(void *addr, size_t size); int elf_map_machine(void *addr); int elf_map_long_size(void *addr); =20 +/* A mapped file with its section headers, section names and symbol table.= */ +struct elf_file { + void *base; + size_t size; + const char *shdrs; + unsigned int shnum, shentsize; + const char *shstrtab; + Elf_Shdr *symtab; + const char *strtab; + size_t nr_syms; +}; + +int elf_open_ro(char const *fname, uint32_t types, struct elf_file *elf); +void elf_close(struct elf_file *elf); + +static inline Elf_Shdr *elf_section(const struct elf_file *elf, + unsigned int index) +{ + return (Elf_Shdr *)(elf->shdrs + (size_t)index * elf->shentsize); +} + +static inline const char *elf_section_name(const struct elf_file *elf, + Elf_Shdr *shdr) +{ + return elf->shstrtab + shdr_name(shdr); +} + +static inline Elf_Sym *elf_symbol(const struct elf_file *elf, size_t index) +{ + const char *base =3D elf->base; + + return (Elf_Sym *)(base + shdr_offset(elf->symtab) + + index * shdr_entsize(elf->symtab)); +} + #endif /* _SCRIPTS_ELF_PARSE_H */ --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30FA13E49F3; Wed, 23 Sep 2026 17:20:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184009; cv=none; b=H7q2mmdYiYFITjZsRKJlNaEpCjsLcFxT3uwmynU1Yf1yJGfA0XbLH3tMfJVOTNx950bwqrauY7xOC7UrraoaGZ67qJQvPyE3cNVRtjElos0OwQhrjGIYswbWKo1iyR0QNYCPgATG10QsbbBGBEIVZXBJwXoSDzeNmKjLzuY0zvg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184009; c=relaxed/simple; bh=+F5lZUSTLMQrVCl2UCybmkeNyplm9abX0HKB/dJhDIE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WiBraT3rLs9qoDd2tecPHX8lW3VhaG2YWWpybunmCf/V83ujlTgGQVZhZutRO6KnVAApYcN1/Oe5Q1HVSxWpy3l+3zavVpq/9JEKKkfm9GMxZqzBG/T+N+K8p8xK6GSpAtbn7XHyUI9hTCUlWrdFdY//LgQoaCNu4dpoo+owHJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hYot5vWV; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hYot5vWV" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C107C1F000FF; Wed, 23 Sep 2026 17:19:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184006; bh=gs/rcqthB/UiwIG1j/gcnmGn30zmYIO5jhxg0jbTloA=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=hYot5vWVjxRUWJtXmG+eqVGNfqJfJ0B5OT+EGe32rSp0tqr3P5k+HPbJ4G2gf3xz2 gepQFdl4A4h7jC03ys+Cf+hiuIpMW4YupY0aj/wKrv54BF0Q5EXpEcJL+6TwZ4uCYJ y2HNPLoCbngIiPYltOheNwD5oVIWzWIgLL9aL9jTDIqFd0ieAUPrThMfLgmQY3HESU ESh5C269MTc7U64wcXWJPsDQAwcbxHjSJ4CU1/Gz18dOrox7x8soC6FM9fW1ditjrX fXTTSRPlnz7eJXUexU/96rSpnAZFWssR8ayVRfjJxyGF3BaeQgbwnSIFTWncMGPg7V mCBfHXdU2txTQ== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:57 +0100 Subject: [PATCH v4 08/22] kallsyms: reimplement mksysmap in C Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-8-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=23635; i=ljs@kernel.org; h=from:subject:message-id; bh=+F5lZUSTLMQrVCl2UCybmkeNyplm9abX0HKB/dJhDIE=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF7+uuCNTk7ggxOvU6/wi2RPVTzJKarZFd91+PBvt fXBb66pd5SwMIhxMciKKbI8/yK+P0gkbF7nBX83mDmsTCBDGLg4BWAixxMYvgo0MLddbopzSuqU tXvNwPcujCPJkMXoh1zNlch10Sv3MPwV4DRVEvq+bK3q0ajNk40ZQi9d2OC59Z9csLrcgfwJV41 YAQ== X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 mksysmap is a sed script consisting of 30 patterns which link-vmlinux.sh uses to generate *.syms files, and which kallsyms is then called against to generate *.kallsyms files, with the final vmlinux build ultimately generating System.map. For an x86-64 allmodconfig build, this involves three nm runs over a 250 MiB file and half a million lines written and read each time - 0.5s per pass for llvm-nm, and 0.2s for GNU nm, with parsing on top of that. This is unnecessary, instead have kallsyms simply read the ELF file directly making use of the existing elf-parse library in scripts/. This changes kallsyms such that its input is no longer the output from nm, but rather an ELF file. However, if the input file is empty, it outputs an empty table, which retains the same behaviour on first pass that the build system expects. System.map is byte-identical to nm | mksysmap for GNU nm and llvm-nm on two x86 configurations each, and for llvm-nm on arm64, arm, s390 and loongarch defconfigs, so are the kallsyms tables of every pass. Relinking vmlinux, link steps included: before after allmodconfig clang 9.1s 7.9s allmodconfig gcc 7.8s 7.6s defconfig clang 3.7s 3.0s defconfig gcc 3.5s 3.4s Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 9.1s 8.6s -0.43s (-5%) x86 defconfig, touch mm/vma.c, clang 9.0s 8.0s -1.1s (-12%) x86 defconfig, clean, gcc 29.5s 28.9s -0.64s (-2%) x86 defconfig, clean, clang 31.2s 29.8s -1.4s (-5%) x86 allmodconfig, touch mm/vma.c, gcc 35.8s 35.0s -0.77s (-2%) x86 allmodconfig, touch mm/vma.c, clang 35.8s 33.7s -2.0s (-6%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- scripts/Makefile | 5 +- scripts/kallsyms-sysmap.c | 248 ++++++++++++++++++++++++++++++++++++++++++= ++++ scripts/kallsyms.c | 162 +++++++++++++++++------------- scripts/kallsyms.h | 44 ++++++++ scripts/link-vmlinux.sh | 15 +-- scripts/mksysmap | 94 ------------------ 6 files changed, 395 insertions(+), 173 deletions(-) diff --git a/scripts/Makefile b/scripts/Makefile index 3434a82a119f..38d58b0a37e6 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -3,7 +3,7 @@ # scripts contains sources for various helper programs used throughout # the kernel for the build process. =20 -hostprogs-always-$(CONFIG_KALLSYMS) +=3D kallsyms +hostprogs-always-y +=3D kallsyms hostprogs-always-$(BUILD_C_RECORDMCOUNT) +=3D recordmcount hostprogs-always-$(CONFIG_BUILDTIME_TABLE_SORT) +=3D sorttable hostprogs-always-$(CONFIG_ASN1) +=3D asn1_compiler @@ -13,6 +13,7 @@ hostprogs-always-$(CONFIG_RUST_KERNEL_DOCTESTS) +=3D rus= tdoc_test_builder hostprogs-always-$(CONFIG_RUST_KERNEL_DOCTESTS) +=3D rustdoc_test_gen hostprogs-always-$(CONFIG_TRACEPOINTS) +=3D tracepoint-update =20 +kallsyms-objs :=3D kallsyms.o kallsyms-sysmap.o elf-parse.o sorttable-objs :=3D sorttable.o elf-parse.o tracepoint-update-objs :=3D tracepoint-update.o elf-parse.o =20 @@ -30,6 +31,8 @@ rustdoc_test_builder-rust :=3D y rustdoc_test_gen-rust :=3D y =20 HOSTCFLAGS_tracepoint-update.o =3D -I$(srctree)/tools/include +HOSTCFLAGS_kallsyms.o =3D -I$(srctree)/tools/include +HOSTCFLAGS_kallsyms-sysmap.o =3D -I$(srctree)/tools/include HOSTCFLAGS_elf-parse.o =3D -I$(srctree)/tools/include HOSTCFLAGS_sorttable.o =3D -I$(srctree)/tools/include HOSTLDLIBS_sorttable =3D -lpthread diff --git a/scripts/kallsyms-sysmap.c b/scripts/kallsyms-sysmap.c new file mode 100644 index 000000000000..a482968eba51 --- /dev/null +++ b/scripts/kallsyms-sysmap.c @@ -0,0 +1,248 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Obtain symbols from vmlinux for usage by kallsyms. Replaces mksysmap. + * + * To retain compatibility, it provides the same output as nm, only faster. + */ + +#include +#include +#include +#include +#include + +#include "elf-parse.h" +#include "kallsyms.h" + +/* + * What mksysmap dropped from System.map, by name: symbols that are not ne= eded + * there, or not suitable for kallsyms. + */ +static const char *const sysmap_omit_prefixes[] =3D { + /* local symbols for ARM, MIPS, etc. */ + "$", + /* local labels, .LBB, .Ltmpxxx, .L__unnamed_xx, .LASANPC, etc. */ + ".L", + /* arm64 EFI stub namespace */ + "__efistub_", + /* arm64 local symbols in PIE namespace */ + "__pi_$", + "__pi_.L", + /* arm64 local symbols in non-VHE KVM namespace */ + "__kvm_nvhe_$", + "__kvm_nvhe_.L", + /* CFI type identifiers */ + "__kcfi_typeid_", + "__kvm_nvhe___kcfi_typeid_", + "__pi___kcfi_typeid_", + /* CRC from modversions */ + "__crc_", + /* EXPORT_SYMBOL (symbol name) */ + "__kstrtab_", + /* EXPORT_SYMBOL (namespace) */ + "__kstrtabns_", + /* MODULE_DEVICE_TABLE (symbol name) */ + "__mod_device_table__", +}; + +static const char *const sysmap_omit_suffixes[] =3D { + /* arm */ + "_from_arm", + "_from_thumb", + "_veneer", +}; + +static const char *const sysmap_omit_names[] =3D { + /* for LoongArch? */ + "L0", + /* ppc */ + "_SDA_BASE_", + "_SDA2_BASE_", +}; + +/* Symbols that contain the pattern anywhere are dropped. */ +static const char *const sysmap_omit_patterns[] =3D { + /* ppc stub */ + ".long_branch.", + ".plt_branch.", +}; + +/* __*Thunk_: the linker's range extension thunks on arm. */ +static bool is_range_thunk(const char *name) +{ + const char *p; + + if (!string_starts_with(name, "__")) + return false; + for (p =3D name + 2; isalnum((unsigned char)*p); p++) + ; + return p - name >=3D 7 && *p =3D=3D '_' && strncmp(p - 5, "Thunk", 5) =3D= =3D 0; +} + +/* __UNIQUE_ID_modinfo_: the MODULE_INFO() strings of built-in code. */ +static bool is_modinfo_id(const char *name) +{ + static const char prefix[] =3D "__UNIQUE_ID_modinfo_"; + const char *p; + + if (!string_starts_with(name, prefix)) + return false; + for (p =3D name + strlen(prefix); isdigit((unsigned char)*p); p++) + ; + return !*p; +} + +static bool sysmap_omits(const char *name, char type) +{ + size_t i; + + /* Absolute, undefined and debugging symbols. */ + if (type =3D=3D 'a' || type =3D=3D 'N' || type =3D=3D 'U' || type =3D=3D = 'w') + return true; + + for (i =3D 0; i < ARRAY_SIZE(sysmap_omit_prefixes); i++) + if (string_starts_with(name, sysmap_omit_prefixes[i])) + return true; + for (i =3D 0; i < ARRAY_SIZE(sysmap_omit_suffixes); i++) + if (string_ends_with(name, sysmap_omit_suffixes[i])) + return true; + for (i =3D 0; i < ARRAY_SIZE(sysmap_omit_names); i++) + if (strcmp(name, sysmap_omit_names[i]) =3D=3D 0) + return true; + for (i =3D 0; i < ARRAY_SIZE(sysmap_omit_patterns); i++) + if (strstr(name, sysmap_omit_patterns[i])) + return true; + + return is_range_thunk(name) || is_modinfo_id(name); +} + +/* nm's letter for a symbol defined in a section, as BFD classifies it. */ +static char section_symbol_type(Elf_Shdr *shdr, const char *secname) +{ + static const char *const debug_prefixes[] =3D { + ".debug", ".zdebug", ".gnu.debuglto_.debug_", + ".gnu.linkonce.wi.", ".line", ".stab", + }; + const uint64_t flags =3D shdr_flags(shdr); + size_t i; + + if (flags & SHF_EXECINSTR) + return 't'; + if (flags & SHF_ALLOC) { + if (shdr_type(shdr) =3D=3D SHT_NOBITS) + return 'b'; + return flags & SHF_WRITE ? 'd' : 'r'; + } + for (i =3D 0; i < ARRAY_SIZE(debug_prefixes); i++) + if (string_starts_with(secname, debug_prefixes[i])) + return 'N'; + if (shdr_type(shdr) !=3D SHT_NOBITS && !(flags & SHF_WRITE)) + return 'n'; + return '?'; +} + +/* The letter nm prints for a symbol, or 0 for one it leaves out. */ +static char elf_symbol_type(const struct elf_file *elf, Elf_Sym *sym) +{ + unsigned int bind =3D sym_bind(sym), type =3D sym_type(sym); + unsigned int shndx =3D sym_shndx(sym); + Elf_Shdr *shdr; + char c; + + if (type =3D=3D STT_SECTION || type =3D=3D STT_FILE) + return 0; + if (shndx =3D=3D SHN_COMMON) + return 'C'; + if (shndx =3D=3D SHN_UNDEF) { + if (bind =3D=3D STB_WEAK) + return type =3D=3D STT_OBJECT ? 'v' : 'w'; + return 'U'; + } + if (type =3D=3D STT_GNU_IFUNC) + return 'i'; + if (bind =3D=3D STB_WEAK) + return type =3D=3D STT_OBJECT ? 'V' : 'W'; + if (bind =3D=3D STB_GNU_UNIQUE) + return 'u'; + if (bind !=3D STB_GLOBAL && bind !=3D STB_LOCAL) + return '?'; + + if (shndx =3D=3D SHN_ABS) { + c =3D 'a'; + } else if (shndx < elf->shnum) { + shdr =3D elf_section(elf, shndx); + c =3D section_symbol_type(shdr, elf_section_name(elf, shdr)); + } else { + return '?'; + } + + return bind =3D=3D STB_GLOBAL ? toupper(c) : c; +} + +/* nm -n order: by address, then by name. */ +static int compare_symbols(const void *a, const void *b) +{ + const struct sysmap_symbol *sa =3D a, *sb =3D b; + + if (sa->addr !=3D sb->addr) + return sa->addr < sb->addr ? -1 : 1; + return strcmp(sa->name, sb->name); +} + +/* The symbols "nm -n | mksysmap" would list, in that order. */ +static struct sysmap_symbol *elf_read_symbols(const struct elf_file *elf, + size_t *nr_kept) +{ + struct sysmap_symbol *syms =3D xmalloc(elf->nr_syms * sizeof(*syms)); + size_t i, n =3D 0; + + for (i =3D 1; i < elf->nr_syms; i++) { + Elf_Sym *sym =3D elf_symbol(elf, i); + const char *name =3D elf->strtab + sym_name(sym); + char type =3D elf_symbol_type(elf, sym); + + if (!type || sysmap_omits(name, type)) + continue; + + syms[n].addr =3D sym_value(sym); + syms[n].name =3D name; + syms[n].type =3D type; + n++; + } + + qsort(syms, n, sizeof(*syms), compare_symbols); + *nr_kept =3D n; + return syms; +} + +struct sysmap *sysmap_read(const char *path) +{ + struct sysmap *map =3D xcalloc(1, sizeof(*map)); + + /* The names point into the mapping; keep it until the map is freed. */ + if (elf_open_ro(path, (1 << ET_EXEC) | (1 << ET_DYN), &map->elf)) + exit(EXIT_FAILURE); + map->syms =3D elf_read_symbols(&map->elf, &map->nr_syms); + map->addr_width =3D elf_map_long_size(map->elf.base) * 2; + + return map; +} + +void sysmap_write(const struct sysmap *map, FILE *out) +{ + size_t i; + + for (i =3D 0; i < map->nr_syms; i++) { + const struct sysmap_symbol *s =3D &map->syms[i]; + + fprintf(out, "%0*llx %c %s\n", map->addr_width, s->addr, s->type, + s->name); + } +} + +void sysmap_free(struct sysmap *map) +{ + free(map->syms); + elf_close(&map->elf); + free(map); +} diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index b615db767b1d..ecb444078c1c 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -5,7 +5,12 @@ * This software may be used and distributed according to the terms * of the GNU General Public License, incorporated herein by reference. * - * Usage: kallsyms [--all-symbols] [--pc-relative] in.map out.bin > out.S + * Usage: kallsyms [--all-symbols] [--pc-relative] [--sysmap=3Dout.map] in= out.bin > out.S + * kallsyms --sysmap=3Dout.map in + * + * in is vmlinux; an empty file stands for the first link, which has = no + * symbols yet, and gives an empty table. --sysmap also writes the symbols + * in System.map format. * * The byte tables go to out.bin and are pulled into out.S with .incb= in; * wider tables stay assembler source for endianness and relocations. @@ -21,7 +26,6 @@ * */ =20 -#include #include #include #include @@ -29,10 +33,10 @@ #include #include #include - +#include #include =20 -#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof(arr[0])) +#include "kallsyms.h" =20 #define KSYM_NAME_LEN 512 =20 @@ -108,11 +112,13 @@ static void sym_arr_free(struct sym_arr *arr) =20 static void usage(void) { - fprintf(stderr, "Usage: kallsyms [--all-symbols] [--pc-relative] in.map o= ut.bin > out.S\n"); + fprintf(stderr, "Usage: kallsyms [--all-symbols] [--pc-relative] [--sysma= p=3Dout.map]\n" + " in out.bin > out.S\n" + " kallsyms --sysmap=3Dout.map vmlinux\n"); exit(1); } =20 -static char *sym_name(const struct sym_entry *s) +static char *sym_entry_name(const struct sym_entry *s) { return (char *)s->sym + 1; } @@ -153,37 +159,12 @@ static void check_symbol_range(const char *sym, unsig= ned long long addr, } } =20 -static struct sym_entry *read_symbol(FILE *in, char **buf, size_t *buf_len) +static struct sym_entry *add_symbol(unsigned long long addr, char type, + const char *name) { - char *name, type, *p; - unsigned long long addr; - size_t len; - ssize_t readlen; + size_t len =3D strlen(name); struct sym_entry *sym; =20 - errno =3D 0; - readlen =3D getline(buf, buf_len, in); - if (readlen < 0) { - if (errno) { - perror("read_symbol"); - exit(EXIT_FAILURE); - } - return NULL; - } - - if ((*buf)[readlen - 1] =3D=3D '\n') - (*buf)[readlen - 1] =3D 0; - - addr =3D strtoull(*buf, &p, 16); - - if (*buf =3D=3D p || *p++ !=3D ' ' || !isascii((type =3D *p++)) || *p++ != =3D ' ') { - fprintf(stderr, "line format error\n"); - exit(EXIT_FAILURE); - } - - name =3D p; - len =3D strlen(name); - if (len >=3D KSYM_NAME_LEN) { fprintf(stderr, "Symbol %s too long for kallsyms (%zu >=3D %d).\n" "Please increase KSYM_NAME_LEN both in kernel and kallsyms.c\n", @@ -208,7 +189,7 @@ static struct sym_entry *read_symbol(FILE *in, char **b= uf, size_t *buf_len) sym->addr =3D addr; sym->len =3D len; sym->sym[0] =3D type; - strcpy(sym_name(sym), name); + strcpy(sym_entry_name(sym), name); =20 return sym; } @@ -229,14 +210,9 @@ static int symbol_in_range(const struct sym_entry *s, return 0; } =20 -static bool string_starts_with(const char *s, const char *prefix) -{ - return strncmp(s, prefix, strlen(prefix)) =3D=3D 0; -} - static int symbol_valid(const struct sym_entry *s) { - const char *name =3D sym_name(s); + const char *name =3D sym_entry_name(s); =20 /* if --all-symbols is not specified, then symbols outside the text * and inittext sections are discarded */ @@ -286,36 +262,55 @@ static void shrink_table(void) table_cnt =3D pos; } =20 -static void read_map(const char *in) +static void add_table_entry(struct sym_entry *sym) { - FILE *fp; - struct sym_entry *sym; - char *buf =3D NULL; - size_t buflen =3D 0; + sym->seq =3D table_cnt; =20 - fp =3D fopen(in, "r"); - if (!fp) { - perror(in); - exit(1); + if (table_cnt >=3D table_size) { + table_size =3D table_size ? table_size * 2 : 10000; + table =3D xrealloc(table, sizeof(*table) * table_size); } =20 - while (!feof(fp)) { - sym =3D read_symbol(fp, &buf, &buflen); - if (!sym) - continue; + table[table_cnt++] =3D sym; +} =20 - sym->seq =3D table_cnt; +static bool file_is_empty(const char *path) +{ + struct stat st; =20 - if (table_cnt >=3D table_size) { - table_size +=3D 10000; - table =3D xrealloc(table, sizeof(*table) * table_size); - } + if (stat(path, &st)) { + perror(path); + exit(EXIT_FAILURE); + } + + return st.st_size =3D=3D 0; +} + +/* + * Read the symbols from vmlinux, writing System.map if asked to. The first + * link has no symbols yet: an empty file gives an empty table. + */ +static void read_elf(const char *path, FILE *sysmap_out) +{ + struct sysmap *map; + size_t i; + + if (file_is_empty(path)) + return; + + map =3D sysmap_read(path); + if (sysmap_out) + sysmap_write(map, sysmap_out); + + for (i =3D 0; i < map->nr_syms; i++) { + const struct sysmap_symbol *s =3D &map->syms[i]; + struct sym_entry *sym =3D add_symbol(s->addr, s->type, s->name); =20 - table[table_cnt++] =3D sym; + if (sym) + add_table_entry(sym); } =20 - free(buf); - fclose(fp); + sysmap_free(map); } =20 static void output_label(const char *label) @@ -378,7 +373,7 @@ static int compare_names(const void *a, const void *b) const struct sym_entry *sa =3D *(const struct sym_entry **)a; const struct sym_entry *sb =3D *(const struct sym_entry **)b; =20 - ret =3D strcmp(sym_name(sa), sym_name(sb)); + ret =3D strcmp(sym_entry_name(sa), sym_entry_name(sb)); if (!ret) { if (sa->addr > sb->addr) return 1; @@ -756,7 +751,7 @@ static void optimize_token_table(void) /* guess for "linker script provide" symbol */ static int may_be_linker_script_provide_symbol(const struct sym_entry *se) { - const char *symbol =3D sym_name(se); + const char *symbol =3D sym_entry_name(se); int len =3D se->len - 1; =20 if (len < 8) @@ -813,8 +808,8 @@ static int compare_symbols(const void *a, const void *b) return wa - wb; =20 /* sort by the number of prefix underscores */ - wa =3D strspn(sym_name(sa), "_"); - wb =3D strspn(sym_name(sb), "_"); + wa =3D strspn(sym_entry_name(sa), "_"); + wb =3D strspn(sym_entry_name(sb), "_"); if (wa !=3D wb) return wa - wb; =20 @@ -829,13 +824,14 @@ static void sort_symbols(void) =20 int main(int argc, char **argv) { - const char *out_bin_name; - FILE *out_bin_file; + const char *in, *sysmap =3D NULL, *out_bin_name; + FILE *sysmap_out =3D NULL, *out_bin_file; =20 while (1) { static const struct option long_options[] =3D { {"all-symbols", no_argument, &all_symbols, 1}, {"pc-relative", no_argument, &pc_relative, 1}, + {"sysmap", required_argument, NULL, 's'}, {}, }; =20 @@ -843,13 +839,33 @@ int main(int argc, char **argv) =20 if (c =3D=3D -1) break; - if (c !=3D 0) + if (c =3D=3D 's') + sysmap =3D optarg; + else if (c !=3D 0) usage(); } =20 - if (optind + 2 !=3D argc) + if (optind + 2 !=3D argc && !(sysmap && optind + 1 =3D=3D argc)) usage(); =20 + in =3D argv[optind]; + if (sysmap) { + sysmap_out =3D fopen(sysmap, "w"); + if (!sysmap_out) { + perror(sysmap); + exit(EXIT_FAILURE); + } + } + + if (optind + 1 =3D=3D argc) { + read_elf(in, sysmap_out); + if (ferror(sysmap_out) || fclose(sysmap_out)) { + perror(sysmap); + exit(EXIT_FAILURE); + } + return 0; + } + out_bin_name =3D argv[optind + 1]; out_bin_file =3D fopen(out_bin_name, "w"); if (!out_bin_file) { @@ -857,7 +873,11 @@ int main(int argc, char **argv) exit(EXIT_FAILURE); } =20 - read_map(argv[optind]); + read_elf(in, sysmap_out); + if (sysmap_out && (ferror(sysmap_out) || fclose(sysmap_out))) { + perror(sysmap); + exit(EXIT_FAILURE); + } shrink_table(); sort_symbols(); optimize_token_table(); diff --git a/scripts/kallsyms.h b/scripts/kallsyms.h new file mode 100644 index 000000000000..4b2801ba6285 --- /dev/null +++ b/scripts/kallsyms.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef KALLSYMS_H +#define KALLSYMS_H + +#include +#include +#include +#include + +#include +#include "elf-parse.h" + +static inline bool string_starts_with(const char *s, const char *prefix) +{ + return strncmp(s, prefix, strlen(prefix)) =3D=3D 0; +} + +static inline bool string_ends_with(const char *s, const char *suffix) +{ + size_t len =3D strlen(s), suffix_len =3D strlen(suffix); + + return len >=3D suffix_len && strcmp(s + len - suffix_len, suffix) =3D=3D= 0; +} + +/* A symbol as nm lists it. */ +struct sysmap_symbol { + unsigned long long addr; + const char *name; + char type; +}; + +/* The symbols of an ELF file that System.map lists, in its order. */ +struct sysmap { + struct sysmap_symbol *syms; + size_t nr_syms; + int addr_width; /* hex digits of an address */ + struct elf_file elf; /* the mapping the names point into */ +}; + +struct sysmap *sysmap_read(const char *path); +void sysmap_write(const struct sysmap *map, FILE *out); +void sysmap_free(struct sysmap *map); + +#endif /* KALLSYMS_H */ diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index ed5ff97c0667..c520741f85c2 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -123,11 +123,15 @@ kallsymso_changed() ! cmp -s "${kallsymso_prev}.sym" "${kallsymso}.sym" } =20 -# Create ${2}.o file with all symbols from the ${1} object file +# Create ${2}.o with the kallsyms tables for ${1} (the vmlinux, or an empty +# listing for the first pass); list the symbols used in ${3} if given. kallsyms() { local kallsymopt; =20 + if [ -n "${3:-}" ]; then + kallsymopt=3D"--sysmap=3D${3}" + fi if is_enabled CONFIG_KALLSYMS_ALL; then kallsymopt=3D"${kallsymopt} --all-symbols" fi @@ -150,18 +154,15 @@ kallsyms() # Perform kallsyms for the given temporary vmlinux. sysmap_and_kallsyms() { - mksysmap "${1}" "${1}.syms" - kallsyms "${1}.syms" "${1}.kallsyms" - + kallsyms "${1}" "${1}.kallsyms" "${1}.syms" kallsyms_sysmap=3D${1}.syms } =20 # Create map file with all symbols from ${1} -# See mksymap for additional details mksysmap() { - info NM ${2} - ${NM} -n "${1}" | sed -f "${srctree}/scripts/mksysmap" > "${2}" + info SYSMAP ${2} + scripts/kallsyms --sysmap=3D"${2}" "${1}" } =20 sorttable() diff --git a/scripts/mksysmap b/scripts/mksysmap deleted file mode 100755 index 856b26ba2ac0..000000000000 --- a/scripts/mksysmap +++ /dev/null @@ -1,94 +0,0 @@ -#!/bin/sed -f -# SPDX-License-Identifier: GPL-2.0-only -# -# sed script to filter out symbols that are not needed for System.map, -# or not suitable for kallsyms. The input should be 'nm -n '. -# -# System.map is used by module-init tools and some debugging -# tools to retrieve the actual addresses of symbols in the kernel. -# -# readprofile starts reading symbols when _stext is found, and -# continue until it finds a symbol which is not either of 'T', 't', -# 'W' or 'w'. -# -# ------------------------------------------------------------------------= --- -# Ignored symbol types -# - -# a: local absolute symbols -# N: debugging symbols -# U: undefined global symbols -# w: local weak symbols -/ [aNUw] /d - -# ------------------------------------------------------------------------= --- -# Ignored prefixes -# (do not forget a space before each pattern) - -# local symbols for ARM, MIPS, etc. -/ \$/d - -# local labels, .LBB, .Ltmpxxx, .L__unnamed_xx, .LASANPC, etc. -/ \.L/d - -# arm64 EFI stub namespace -/ __efistub_/d - -# arm64 local symbols in PIE namespace -/ __pi_\$/d -/ __pi_\.L/d - -# arm64 local symbols in non-VHE KVM namespace -/ __kvm_nvhe_\$/d -/ __kvm_nvhe_\.L/d - -# lld arm/aarch64/mips thunks -/ __[[:alnum:]]*Thunk_/d - -# CFI type identifiers -/ __kcfi_typeid_/d -/ __kvm_nvhe___kcfi_typeid_/d -/ __pi___kcfi_typeid_/d - -# CRC from modversions -/ __crc_/d - -# EXPORT_SYMBOL (symbol name) -/ __kstrtab_/d - -# EXPORT_SYMBOL (namespace) -/ __kstrtabns_/d - -# MODULE_DEVICE_TABLE (symbol name) -/ __mod_device_table__/d - -# ------------------------------------------------------------------------= --- -# Ignored suffixes -# (do not forget '$' after each pattern) - -# arm -/_from_arm$/d -/_from_thumb$/d -/_veneer$/d - -# ------------------------------------------------------------------------= --- -# Ignored symbols (exact match) -# (do not forget a space before and '$' after each pattern) - -# for LoongArch? -/ L0$/d - -# ppc -/ _SDA_BASE_$/d -/ _SDA2_BASE_$/d - -# MODULE_INFO() -/ __UNIQUE_ID_modinfo_[0-9]*$/d - -# ------------------------------------------------------------------------= --- -# Ignored patterns -# (symbols that contain the pattern are ignored) - -# ppc stub -/\.long_branch\./d -/\.plt_branch\./d --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C043B3E4C61; Wed, 23 Sep 2026 17:20:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184021; cv=none; b=T04VdbXLEgp+7xC+19Wc1nwE3xilLvH+TJHrdRAaqKmFVhtyjAdgPrRx8L+76XsBuMhc0MpkS//wWK0I9wFHo9h13heU9buEWk7NyfnSmuzvXyKrn97IVDMTv990yZTJVmsltQ2mrk+o9JZ5Nn6Y68IyjYdyKSWjxcrqx3Xs+zU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184021; c=relaxed/simple; bh=1tlfspzVyupWDtDIq9Xi6b2r8IeD5B+p03B8UkqK1dA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YuYOkJxzmJUOkwnwOTHEIwvM/s5Hs+eMzPNOHLsVT0NmooptaW1VE2FqjlxAaIeOP8DCRn3UbF9ob30DwZ3YJjYwrHbJBgdUhWCVd35mjJR4+b1P12T00xECZuW/BHvAPRPSjVWJbemNdXRtz4ch8eVyVbWLPuea2Z3XbdW8y6A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mZ46Am8b; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mZ46Am8b" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 50FCE1F00893; Wed, 23 Sep 2026 17:20:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184019; bh=+rwl1Zn9bU3CF5Lv6uJecPAyTurD5CJjPcxpUYcu/S0=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=mZ46Am8b6Wn9IMnH8S2ELNq04sKgdTBqXyocEc8OiLB0sIiwNs/r3xJS7H+Jps8UB Nr6oNJarDDpQaWtichSPBo6WiOxV2I/l6AUnAGHuHANUHxpuu2aHkDkf4NJ3FdIYPs LCw0TVk09iax4Vzz4tQdzXYTXcEPXUz34TFC+3lcSzfEhpFGFPbjAhQeHBqIQ/Dif8 ZFW0n3RzfA4QblsDN/VKZRAVIggc1iQPR5fT9pZ1lWsexJBduO/kql9jVY68tmvSvK U9JRqHbVg4kFTgLsFXUxkNb+L4jPIRFRUxVfE9wd2ZGBl0Y1yNn7XtyaJSKrxrB/bh 7+S3dv4tqFp+g== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:58 +0100 Subject: [PATCH v4 09/22] kbuild: calculate list, composite object state per object Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-9-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" , Nicolas Schier X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=3573; i=ljs@kernel.org; h=from:subject:message-id; bh=1tlfspzVyupWDtDIq9Xi6b2r8IeD5B+p03B8UkqK1dA=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF6e9yE+QOQp734n36qPH5XC1tztqeaztH+w+1ud6 NnbN0xCO0pZGMS4GGTFFFmefxHfHyQSNq/zgr8bzBxWJpAhDFycAjARpuWMDD9uLjv/Tf5qzXuz +M+ue27qrjt6u2Xji/mi5iLfY+4//crH8E9zWuT8XfqLT783vWcXtMFMm4/zb/0vo393ec3/ZTG 5cLMCAA== X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 When each object's command line is expanded, kbuild has to figure out on multiple occasions whether it's built-in or part of a module and which composite object it belongs to. This causes the time spent on each directory in the kernel tree to grow O(n^2) with its object count, which is especially problematic for instance in drivers/gpu/drm/amd/amdgpu with 310 objects. No-op builds (i.e. make -j $(nproc) when nothing has changed) are particularly impacted by this. Fix the issue by calculating this data once and looking it up thereafter instead of recomputing it over and over again. This has a particularly large impact on allmodconfig builds. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, no-op make, gcc 1.1s 1.1s -0.02s (-2%) x86 defconfig, no-op make, clang 1.3s 1.3s -0.01s (-1%) x86 allmodconfig, no-op make, gcc 15.3s 14.5s -0.83s (-5%) x86 allmodconfig, no-op make, clang 16.2s 15.5s -0.65s (-4%) x86 allmodconfig, touch mm/vma.c, gcc 35.0s 34.3s -0.77s (-2%) x86 allmodconfig, touch mm/vma.c, clang 33.7s 33.2s -0.51s (-2%) Assisted-by: LLM Reviewed-by: Nicolas Schier Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- scripts/Makefile.build | 8 ++++++++ scripts/Makefile.lib | 7 +++---- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 4349108e75e1..2cabfe85b798 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -122,6 +122,14 @@ multi-obj-m :=3D $(addprefix $(obj)/, $(multi-obj-m)) subdir-ym :=3D $(addprefix $(obj)/, $(subdir-ym)) endif =20 +# Cache which list each object is in and which composite objects it belong= s to, +# once per object for $(part-of-builtin), $(part-of-module) and $(modname-= multi). +$(foreach o, $(real-obj-y) $(lib-y), $(eval part-of-builtin_$o :=3D y)) +$(foreach o, $(real-obj-m), $(eval part-of-module_$o :=3D y)) +$(foreach m, $(multi-obj-ym), \ + $(foreach o, $(call suffix-search, $m, .o, -objs -y -m), \ + $(eval modname-multi_$o +=3D $(m:.o=3D)))) + ifndef obj $(warning kbuild: Makefile.build is included improperly) endif diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 0a4fdd8bd975..2f447bc25e7b 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -2,8 +2,7 @@ =20 # Finds the multi-part object the current object will be linked into. # If the object belongs to two or more multi-part objects, list them all. -modname-multi =3D $(sort $(foreach m,$(multi-obj-ym),\ - $(if $(filter $*.o, $(call suffix-search, $m, .o, -objs -y -m)),$(m:.o= =3D)))) +modname-multi =3D $(sort $(modname-multi_$*.o)) =20 __modname =3D $(or $(modname-multi),$(basetarget)) =20 @@ -149,8 +148,8 @@ endif # If $(is-kernel-object) is 'y', this object will be linked to vmlinux or = modules is-kernel-object =3D $(or $(part-of-builtin),$(part-of-module)) =20 -part-of-builtin =3D $(if $(filter $(basename $@).o, $(real-obj-y) $(lib-y)= ),y) -part-of-module =3D $(if $(filter $(basename $@).o, $(real-obj-m)),y) +part-of-builtin =3D $(part-of-builtin_$(basename $@).o) +part-of-module =3D $(part-of-module_$(basename $@).o) quiet_modtag =3D $(if $(part-of-module),[M], ) =20 modkern_cflags =3D \ --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 373B635E1AF; Wed, 23 Sep 2026 17:20:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184033; cv=none; b=hjZJzMB1XJkCqAaGEmRFqo0nk/k1RhTO/q2zZ9HFuFOpu5MgEjC1RbcGWGhR/+5twgmkDJMbOtaocyIGiSh2Q/kFuGTBu8TM8U8OVDIty56I7efs2pWPIfSb/vhpDGGQtq2Pdm/yGK4l2YxEOqScBe7FMV8d1DcxuxQsy+LVAGk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184033; c=relaxed/simple; bh=vu58Bnv++YVBx3hdKWip/gCDhzfsNmxCWv+mVy6NYn8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=g/0VXnkXMdUYIpIFOuf8uuZYqZepfQXnixLFP/gokkVp/sCPOqhNDloLI6VMYjk3KPDiL92aovX5piMWBVu/9LiWmywPlJ/T9QqB4+9/RQ+xktsECWCl7U9mD9YtskFS9H+pew5kb8U6VrZ6ku3jA5nXqdEnJQgql455bifvcN4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NYM+0xGj; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NYM+0xGj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DF9CF1F00898; Wed, 23 Sep 2026 17:20:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184031; bh=dE0Cp7cVTXt7sFMVwoXw+v3pYGTpA10+0QkTPfn3kfU=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=NYM+0xGjuWTB7BmzeoSH81K+tWskYgZY7DLSJpUsBBJRDj24N16j46sry8j6gBK+B Qzm037TBmGqorPDC7okGGNtV96YC5T0dEsFT5jqIsL2j0qBFMTlzoMcmC9904Bnycb BOBkvDajMkPtdezLPu3rZzkKwZtDOYVDfMSfBw88evVqcFqHWM58DSXvk4WQ5O3ITF DQ3s6+nPR5wgoSc5Bg/KfcY64+rMPoJhfWBowcCBkCxiIyJw0UTGUI3RWLLmujegD/ cxdWBwoPgBS485iq271d2woPnyHj6RXUrkxmN50TpAtI67GXOElYWqpjFRC0DDCBbk eu4Hq+FlQT0Qg== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:17:59 +0100 Subject: [PATCH v4 10/22] kbuild: implement and use depcheck to check dependency timestamps Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-10-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=17217; i=ljs@kernel.org; h=from:subject:message-id; bh=vu58Bnv++YVBx3hdKWip/gCDhzfsNmxCWv+mVy6NYn8=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF5u2dp/dt2+ySf+tj9hWb1OgulQ6LcXZ/Nsd3JaZ omm7tO93FHKwiDGxSArpsjy/Iv4/iCRsHmdF/zdYOawMoEMYeDiFICJqNxi+GfePOPRZZl7i822 32/akrx1cnXEAb2/bPuPtGpVH9NiPczAyLDGUEOL/+unqysTizLu/ekxCF7IxyHyrXJ53Rllt7M ZPmwA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Each object's .cmd file lists its header dependencies, which often consists of over a thousand dependencies. When little has changed in the tree, this is what make spends most of its time doing, spending over a second, single-threaded when parsing larger directory trees. It performs considerably more work that is actually necessary to get the job done. This can be done faster in C, so implement scripts/basic/depcheck to do so. It does as little work as possible, reading the .cmd files from a directory's targets and running stat on each dependency only a single time. It generates a fragment holding only the saved command line for a target whose dependencies all exist and are older than it, and the entire .cmd file for any other. A dependency that is also a target of the same directory (e.g. header generated by an if_changed rule) may get remade later in the same make run, so do not treat a target as fresh. That way, Makefile.build simply includes the fragment and the amount of work make has to do is significantly reduced when there is not much work to do. If the operation fails, kbuild falls back to using .cmd files. The result is exactly the same as before. For drivers/gpu/drm/amd/amdgpu in an allmodconfig tree the fragment is a fifth of the size of the .cmd files, make reads it in 10ms instead of 380ms, and the instance goes from 2.2s to 0.4s. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, no-op make, gcc 1.1s 0.78s -0.30s (-28%) x86 defconfig, no-op make, clang 1.3s 0.94s -0.34s (-27%) x86 allmodconfig, no-op make, gcc 14.5s 12.9s -1.6s (-11%) x86 allmodconfig, no-op make, clang 15.5s 14.2s -1.3s (-9%) x86 allmodconfig, touch mm/vma.c, gcc 34.3s 33.9s -0.33s (-1%) x86 allmodconfig, touch mm/vma.c, clang 33.2s 32.2s -0.98s (-3%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- Makefile | 1 + scripts/Makefile.build | 15 +- scripts/basic/.gitignore | 1 + scripts/basic/Makefile | 2 +- scripts/basic/depcheck.c | 481 +++++++++++++++++++++++++++++++++++++++++++= ++++ 5 files changed, 498 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index ab3ba78376f7..554c9003a4d9 100644 --- a/Makefile +++ b/Makefile @@ -2241,6 +2241,7 @@ clean: $(clean-dirs) $(call cmd,rmfiles) @find . $(RCS_FIND_IGNORE) \ \( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \ + -o -name '.depcheck.mk' -o -name '.depcheck.mk.tmp' \ -o -name '*.ko.*' -o -name '*.o.thinlto.bc' \ -o -name '*.dtb' -o -name '*.dtbo' \ -o -name '*.dtb.S' -o -name '*.dtbo.S' \ diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 2cabfe85b798..819cc1f750e8 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -592,7 +592,20 @@ $(obj)/: $(if $(KBUILD_BUILTIN), $(targets-for-builtin= )) \ =20 existing-targets :=3D $(wildcard $(sort $(targets))) =20 --include $(foreach f,$(existing-targets),$(dir $(f)).$(notdir $(f)).cmd) +cmd-files :=3D $(wildcard $(foreach f,$(existing-targets),$(dir $(f)).$(no= tdir $(f)).cmd)) + +# depcheck checks timestamps first and outputs only saved command lines of +# up-to-date targets, falling back to .cmd files if it fails. A directory +# whose targets all lie elsewhere may not exist in the output tree. +depcheck :=3D $(wildcard $(objtree)/scripts/basic/depcheck) + +ifneq ($(and $(depcheck),$(cmd-files),$(wildcard $(obj))),) +ifeq ($(shell $(depcheck) $(obj)/.depcheck.mk $(cmd-files) && echo ok),ok) +cmd-files :=3D $(obj)/.depcheck.mk +endif +endif + +-include $(cmd-files) =20 # Create directories for object files if they do not exist obj-dirs :=3D $(sort $(patsubst %/,%, $(dir $(targets)))) diff --git a/scripts/basic/.gitignore b/scripts/basic/.gitignore index 07c195f605a1..761ee14f9477 100644 --- a/scripts/basic/.gitignore +++ b/scripts/basic/.gitignore @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only +/depcheck /fixdep /randstruct.seed diff --git a/scripts/basic/Makefile b/scripts/basic/Makefile index fb8e2c38fbc7..ff98780e474d 100644 --- a/scripts/basic/Makefile +++ b/scripts/basic/Makefile @@ -2,7 +2,7 @@ # # fixdep: used to generate dependency information during build process =20 -hostprogs-always-y +=3D fixdep +hostprogs-always-y +=3D fixdep depcheck =20 # randstruct: the seed is needed before building the gcc-plugin or # before running a Clang kernel build. diff --git a/scripts/basic/depcheck.c b/scripts/basic/depcheck.c new file mode 100644 index 000000000000..119075353f3b --- /dev/null +++ b/scripts/basic/depcheck.c @@ -0,0 +1,481 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * depcheck - check the dependency timestamps of a directory's targets so = that + * make reads only what it needs from their .cmd files. + * + * fixdep writes a .cmd file as: + * + * savedcmd_dir/foo.o :=3D + * + * source_dir/foo.o :=3D dir/foo.c + * + * deps_dir/foo.o :=3D \ + * include/linux/bar.h \ + * $(wildcard include/config/BAZ) \ + * + * dir/foo.o: $(deps_dir/foo.o) + * + * $(deps_dir/foo.o): + * + * and kbuild may append rules of its own after that, such as one making t= he + * target depend on objtool. + * + * For a target that exists and is newer than every dependency listed, mak= e can + * have nothing to do with the list, so it is left out and only what prece= des + * and follows it is passed on; for anything else the .cmd file is passed = on in + * full. + * + * Usage: depcheck <.cmd files...> + * + * The output is a makefile fragment to include in place of the .cmd files= ; a + * non-zero exit status means the caller should include those instead. + */ +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +/* What fixdep writes, see above. */ +#define DEPS_PREFIX "deps_" +#define DEPS_RULE_PREFIX "$(" DEPS_PREFIX +#define RULE_SUFFIX ":" +#define LINE_CONTINUATION " \\" +#define WILDCARD_OPEN "$(wildcard " +#define WILDCARD_CLOSE ")" +#define CMD_SUFFIX ".cmd" + +/* A line of a .cmd file, without its newline. */ +struct line { + const char *text; + size_t len; +}; + +static bool is_blank(char chr) +{ + return chr =3D=3D ' ' || chr =3D=3D '\t'; +} + +static bool str_ends_with(const char *str, const char *suffix) +{ + const size_t len =3D strlen(str), suffix_len =3D strlen(suffix); + + return len >=3D suffix_len && !strcmp(str + len - suffix_len, suffix); +} + +static bool line_starts_with(const struct line *line, const char *prefix) +{ + const size_t len =3D strlen(prefix); + + return line->len >=3D len && !memcmp(line->text, prefix, len); +} + +static bool line_ends_with(const struct line *line, const char *suffix) +{ + const size_t len =3D strlen(suffix); + + return line->len >=3D len && + !memcmp(line->text + line->len - len, suffix, len); +} + +static bool line_is_blank(const struct line *line) +{ + size_t i; + + for (i =3D 0; i < line->len; i++) + if (!is_blank(line->text[i])) + return false; + + return true; +} + +/* Whether the line ends in " \", continuing the list on the next line. */ +static bool line_is_continued(const struct line *line) +{ + return line_ends_with(line, LINE_CONTINUATION); +} + +static void line_strip_continuation(struct line *line) +{ + if (line_is_continued(line)) + line->len -=3D strlen(LINE_CONTINUATION); +} + +static void line_trim(struct line *line) +{ + while (line->len && is_blank(line->text[0])) { + line->text++; + line->len--; + } + while (line->len && is_blank(line->text[line->len - 1])) + line->len--; +} + +/* Take the next line out of [*pos, end); false once there are none left. = */ +static bool next_line(const char **pos, const char *end, struct line *line) +{ + const char *newline; + + if (*pos >=3D end) + return false; + + newline =3D memchr(*pos, '\n', end - *pos); + line->text =3D *pos; + line->len =3D (newline ? newline : end) - *pos; + *pos =3D newline ? newline + 1 : end; + + return true; +} + +/* Describes a dependency file. */ +struct dep { + struct hlist_node hnode; + struct timespec mtime; + bool exists; + char path[]; +}; + +static HASHTABLE_DEFINE(dep_table, 1U << 16); + +/* The targets of this run's .cmd files: anything make may remake this tim= e. */ +struct target { + struct hlist_node hnode; + char path[]; +}; + +static HASHTABLE_DEFINE(target_table, 1U << 12); + +static void add_target(const char *path) +{ + struct target *target =3D xmalloc(sizeof(*target) + strlen(path) + 1); + + strcpy(target->path, path); + hash_add(target_table, &target->hnode, hash_str(path)); +} + +static bool is_target(const char *path) +{ + const unsigned int key =3D hash_str(path); + struct target *target; + + hash_for_each_possible(target_table, target, hnode, key) { + if (!strcmp(target->path, path)) + return true; + } + + return false; +} + +static const struct dep *lookup_dep(const char *path) +{ + const unsigned int key =3D hash_str(path); + struct dep *dep; + struct stat st; + + hash_for_each_possible(dep_table, dep, hnode, key) { + if (!strcmp(dep->path, path)) + return dep; + } + + dep =3D xcalloc(1, sizeof(*dep) + strlen(path) + 1); + strcpy(dep->path, path); + dep->exists =3D !stat(path, &st); + if (dep->exists) + dep->mtime =3D st.st_mtim; + hash_add(dep_table, &dep->hnode, key); + + return dep; +} + +/* Strictly newer, as make compares timestamps. */ +static bool newer(const struct timespec *time_a, const struct timespec *ti= me_b) +{ + if (time_a->tv_sec !=3D time_b->tv_sec) + return time_a->tv_sec > time_b->tv_sec; + + return time_a->tv_nsec > time_b->tv_nsec; +} + +/* + * $(wildcard include/config/FOO) is a prerequisite only while FOO is set: + * unwrap it and say that it is optional. + */ +static bool line_unwrap_wildcard(struct line *line) +{ + if (!line_starts_with(line, WILDCARD_OPEN) || + !line_ends_with(line, WILDCARD_CLOSE)) + return false; + + line->text +=3D strlen(WILDCARD_OPEN); + line->len -=3D strlen(WILDCARD_OPEN) + strlen(WILDCARD_CLOSE); + + return true; +} + +/* fixdep doubles '$' and escapes '#' in a path: undo that into path[]. */ +static bool unescape_path(const struct line *line, char *path, size_t size) +{ + size_t i, out_len =3D 0; + + if (line->len >=3D size) + return false; + + for (i =3D 0; i < line->len; i++) { + const char chr =3D line->text[i]; + const char next_chr =3D i + 1 < line->len ? line->text[i + 1] : '\0'; + + if ((chr =3D=3D '$' && next_chr =3D=3D '$') || (chr =3D=3D '\\' && next_= chr =3D=3D '#')) + i++; + path[out_len++] =3D line->text[i]; + } + path[out_len] =3D '\0'; + + return true; +} + +/* Is the path contained in line older than the target? */ +static bool dep_is_fresh(struct line line, const struct timespec *target) +{ + char path[PATH_MAX]; + const struct dep *dep; + bool optional; + + line_strip_continuation(&line); + line_trim(&line); + if (!line.len) + return false; + + optional =3D line_unwrap_wildcard(&line); + if (!line.len || !unescape_path(&line, path, sizeof(path))) + return false; + + /* A target can get 'remade' so its timestamp is meaningless. */ + if (is_target(path)) + return false; + + dep =3D lookup_dep(path); + if (!dep->exists) + return optional; + + return !newer(&dep->mtime, target); +} + +/* + * Parse a dependency list which consists of one file per line and determi= ne if + * all dependencies are 'fresh', i.e. older than the target. + */ +static bool deps_are_fresh(const char *pos, const char *end, + const struct timespec *target) +{ + struct line line; + + while (next_line(&pos, end, &line)) { + if (line_is_blank(&line)) + return true; + if (!dep_is_fresh(line, target)) + return false; + if (!line_is_continued(&line)) + return true; + } + + return true; +} + +/* + * Find the "deps_" line in the input. + * + * On success returns true and *deps_off is set to its offset and *list is= set + * to the list's own lines start or NULL if the line is not continued. + * + * Otherwise returns false if the prefix cannot be found. + */ +static bool find_deps(const char *buf, size_t len, size_t *deps_off, + const char **list) +{ + const char *pos =3D buf, *end =3D buf + len; + struct line line; + + while (next_line(&pos, end, &line)) { + if (!line_starts_with(&line, DEPS_PREFIX)) + continue; + + *deps_off =3D line.text - buf; + *list =3D line_is_continued(&line) ? pos : NULL; + return true; + } + + return false; +} + +/* + * Is this the target of a .cmd file, i.e. 'dir/.name.cmd names dir/name.'? + */ +static bool target_of(const char *cmd_path, char *target, size_t size) +{ + const char *slash =3D strrchr(cmd_path, '/'); + const char *base =3D slash ? slash + 1 : cmd_path; + const size_t dir_len =3D base - cmd_path; + const size_t affix_len =3D strlen(".") + strlen(CMD_SUFFIX); + const size_t base_len =3D strlen(base); + + if (base[0] !=3D '.' || !str_ends_with(base, CMD_SUFFIX) || + base_len <=3D affix_len) + return false; + + return snprintf(target, size, "%.*s%.*s", (int)dir_len, cmd_path, + (int)(base_len - affix_len), base + 1) < (int)size; +} + +/* + * Does the .cmd file's target exists and is it newer than everything in i= ts + * dependency list? + * + * Returns true if so and sets *deps_off to the start of the list, otherwi= se + * returns false. + */ +static bool target_is_fresh(const char *cmd_path, const char *buf, size_t = len, + size_t *deps_off) +{ + char target[PATH_MAX]; + struct stat st; + const char *list; + + if (!target_of(cmd_path, target, sizeof(target)) || stat(target, &st)) + return false; + if (!find_deps(buf, len, deps_off, &list)) + return false; + if (!list) + return true; + + return deps_are_fresh(list, buf + len, &st.st_mtim); +} + +static char *read_file(const char *path, size_t *len) +{ + FILE *file =3D fopen(path, "r"); + struct stat st; + char *buf; + size_t nr_read =3D 0; + + if (!file) + return NULL; + if (fstat(fileno(file), &st)) { + fclose(file); + return NULL; + } + + buf =3D xmalloc(st.st_size + 1); + while (nr_read < (size_t)st.st_size) { + const size_t chunk =3D fread(buf + nr_read, 1, st.st_size - nr_read, fil= e); + + if (!chunk) + break; + nr_read +=3D chunk; + } + fclose(file); + + buf[nr_read] =3D '\0'; + *len =3D nr_read; + return buf; +} + +/* + * Find the end of the dependency block: the offset just past its closing + * "$(deps_x):" line. Anything after that was appended by kbuild and is no= t for + * us to judge. Returns len if the line cannot be found. + */ +static size_t deps_block_end(const char *buf, size_t len, size_t deps_off) +{ + const char *pos =3D buf + deps_off, *end =3D buf + len; + struct line line; + + while (next_line(&pos, end, &line)) { + if (line_starts_with(&line, DEPS_RULE_PREFIX) && + line_ends_with(&line, RULE_SUFFIX)) + return pos - buf; + } + + return len; +} + +/* The target is fresh: pass on everything but the dependency block. */ +static void write_without_deps(FILE *out, const char *buf, size_t len, + size_t deps_off) +{ + const size_t tail_off =3D deps_block_end(buf, len, deps_off); + + fwrite(buf, 1, deps_off, out); + fputc('\n', out); + fwrite(buf + tail_off, 1, len - tail_off, out); +} + +/* + * Generate a fragment for make from one .cmd file - without the dependency + * block if the target is fresh, otherwise all of it. + */ +static void process(FILE *out, const char *cmd_path) +{ + size_t len, deps_off; + char *buf; + + buf =3D read_file(cmd_path, &len); + if (!buf) + return; + + if (target_is_fresh(cmd_path, buf, len, &deps_off)) + write_without_deps(out, buf, len, deps_off); + else + fwrite(buf, 1, len, out); + + free(buf); +} + +int main(int argc, char **argv) +{ + char tmp_path[PATH_MAX]; + FILE *out; + int i; + + if (argc < 2) { + fprintf(stderr, "usage: %s <.cmd files...>\n", argv[0]); + return 1; + } + + if (snprintf(tmp_path, sizeof(tmp_path), "%s.tmp", argv[1]) >=3D + (int)sizeof(tmp_path)) { + fprintf(stderr, "%s: path too long\n", argv[1]); + return 1; + } + + out =3D fopen(tmp_path, "w"); + if (!out) { + perror(tmp_path); + return 1; + } + + for (i =3D 2; i < argc; i++) { + char target[PATH_MAX]; + + if (target_of(argv[i], target, sizeof(target))) + add_target(target); + } + + for (i =3D 2; i < argc; i++) + process(out, argv[i]); + + /* Write errors are latched in the stream, so one check at the end covers= them all. */ + if (ferror(out) || fclose(out) || rename(tmp_path, argv[1])) { + perror(argv[1]); + unlink(tmp_path); + return 1; + } + + return 0; +} --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85EE33B8950; Wed, 23 Sep 2026 17:20:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184046; cv=none; b=m051nB2rxDvMiqTeG+N1BWRhvb4cG7MKwDI+AL+eq9FKR9zjQHb8R10/Bx8LOpcXOyZbqEDy96fCOZrbkfm9WydIRTz9YxfBFBWAlltC32I5qHqitw+VKuGDTbO8N0XK97xJE7dVuB/dM+weNXdftRONqH+qZv7UySq3HH8S1Xs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184046; c=relaxed/simple; bh=2ycKJa3Z+VPxjsH2ZNqIj7aez7jMdJGovrqulxA0Kxk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=X41/RpsrC9nD70x8QOBevtciojlvdzavcGieb/Nasx5E+ManABAh9qlVyzlEYmJMRSIzcM7aAhtiTR8fbNykQHyYDaZnifGkCB8AvE0JfEFNLeCRCRM7Jn+BRmyxeL3anPRSZOcMfedDVNqH0oMHCysfHAl0YOn55bHNl79WlkA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Qr9bHBGN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Qr9bHBGN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 50E441F000FF; Wed, 23 Sep 2026 17:20:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184044; bh=nAtazO6mnI0TAu3XSRj1hcNvKpCr7+vo326YW4gA3HQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=Qr9bHBGN0mF2HQTspSD3iLMXyTcdzORmlolCOHaxAer4eQhNxjF0zCmtK3Clxr7iP 6RwQ49+v55ui0QH+aAZ0ihxlyxcPPhTXmXuVElYcKELkGfuc9M1g6urv3+w03OH8fG frgkaKADBOCeq512byiCkewp2+fi3lLBup4wVOUsEqGXMkcpaaLZCTuxI/fGnbFgL8 8KZJwLdF1dw2EakkP8ScBh+o4Rte/vxWr1GBR7XhoRJG9xLVFquIcFEk29wtD8mSJ3 +LO51la90FiWZm4NL6xGSquO+U45Omw5tZ+rq4WepLpysS5Xr9sC77yE5TMvOs0lOE zqqBzQxw4iupA== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:00 +0100 Subject: [PATCH v4 11/22] kbuild: move the toolchain checks into scripts/Kconfig.toolchain Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-11-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=14806; i=ljs@kernel.org; h=from:subject:message-id; bh=2ycKJa3Z+VPxjsH2ZNqIj7aez7jMdJGovrqulxA0Kxk=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF7m6dYWXchlHRSooTdh/6oljn5Vh33T7aLvMZzMj Gcrz5raUcrCIMbFICumyPL8i/j+IJGweZ0X/N1g5rAygQxh4OIUgInsaGdk6Ks6ynb6616R/UHc boVT89O5n2qFPzg8gT3xLvOO7mnh+Qz/8xMET0c0bWW4pBTJqVS8JW/2xu/KWxUNlKpu57rO+Nv IBAA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 init/Kconfig opens with two hundred lines identifying the compiler, assembler, linker and rustc and probing which options they support, none of it to do with init. Move them into their own file, sourced from the top of init/Kconfig, so that the next commit can add to them without growing init/Kconfig further. Also update MAINTAINERS to put the new file in the KCONFIG entry. No functional change intended. Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- MAINTAINERS | 1 + init/Kconfig | 197 +-----------------------------------------= --- scripts/Kconfig.toolchain | 198 ++++++++++++++++++++++++++++++++++++++++++= ++++ 3 files changed, 200 insertions(+), 196 deletions(-) diff --git a/MAINTAINERS b/MAINTAINERS index 6215fcb07770..46c4f7e3061c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14057,6 +14057,7 @@ Q: https://patchwork.kernel.org/project/linux-kbuil= d/list/ T: git git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux.git F: Documentation/kbuild/kconfig* F: scripts/Kconfig.include +F: scripts/Kconfig.toolchain F: scripts/kconfig/ =20 KCORE diff --git a/init/Kconfig b/init/Kconfig index 8583d9f06c52..352601b44b5c 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1,201 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only -config CC_VERSION_TEXT - string - default "$(CC_VERSION_TEXT)" - help - This is used in unclear ways: - - - Re-run Kconfig when the compiler is updated - The 'default' property references the environment variable, - CC_VERSION_TEXT so it is recorded in include/config/auto.conf.cmd. - When the compiler is updated, Kconfig will be invoked. - - - Ensure full rebuild when the compiler is updated - include/linux/compiler-version.h contains this option in the comment - line so fixdep adds include/config/CC_VERSION_TEXT into the - auto-generated dependency. When the compiler is updated, syncconfig - will touch it and then every file will be rebuilt. - -config CC_IS_GCC - def_bool $(success,test "$(cc-name)" =3D GCC) - -config GCC_VERSION - int - default $(cc-version) if CC_IS_GCC - default 0 - -config CC_IS_CLANG - def_bool $(success,test "$(cc-name)" =3D Clang) - -config CLANG_VERSION - int - default $(cc-version) if CC_IS_CLANG - default 0 - -config AS_IS_GNU - def_bool $(success,test "$(as-name)" =3D GNU) - -config AS_IS_LLVM - def_bool $(success,test "$(as-name)" =3D LLVM) - -config AS_VERSION - int - # Use clang version if this is the integrated assembler - default CLANG_VERSION if AS_IS_LLVM - default $(as-version) - -config LD_IS_BFD - def_bool $(success,test "$(ld-name)" =3D BFD) - -config LD_VERSION - int - default $(ld-version) if LD_IS_BFD - default 0 - -config LD_IS_LLD - def_bool $(success,test "$(ld-name)" =3D LLD) - -config LLD_VERSION - int - default $(ld-version) if LD_IS_LLD - default 0 - -config RUSTC_VERSION - int - default $(rustc-version) - help - It does not depend on `RUST` since that one may need to use the version - in a `depends on`. - -config RUST_IS_AVAILABLE - def_bool $(success,$(srctree)/scripts/rust_is_available.sh) - help - This shows whether a suitable Rust toolchain is available (found). - - Please see Documentation/rust/quick-start.rst for instructions on how - to satisfy the build requirements of Rust support. - - In particular, the Makefile target 'rustavailable' is useful to check - why the Rust toolchain is not being detected. - -config RUSTC_LLVM_VERSION - int - default $(rustc-llvm-version) - -config RUSTC_LLVM_MAJOR_VERSION - int - default $(shell,expr $(rustc-llvm-version) / 10000) =20 -config RUSTC_CLANG_LLVM_COMPATIBLE - bool - default y if CC_IS_CLANG && RUSTC_LLVM_MAJOR_VERSION =3D $(shell,expr $(c= c-version) / 10000) - help - This indicates whether Rust and Clang use LLVM of the same major - version. - - Operations involving handling LLVM IR or bitcode (e.g. cross-language - LTO) require the same LLVM major version to work properly. For best - compatibility it is recommended that the exact same LLVM is used. - -config ARCH_HAS_CC_CAN_LINK - bool - -config CC_CAN_LINK - bool - default ARCH_CC_CAN_LINK if ARCH_HAS_CC_CAN_LINK - default $(cc_can_link_user,$(m64-flag)) if 64BIT - default $(cc_can_link_user,$(m32-flag)) - -# Fixed in GCC 14, 13.3, 12.4 and 11.5 -# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D113921 -config GCC_ASM_GOTO_OUTPUT_BROKEN - bool - depends on CC_IS_GCC - default y if GCC_VERSION < 110500 - default y if GCC_VERSION >=3D 120000 && GCC_VERSION < 120400 - default y if GCC_VERSION >=3D 130000 && GCC_VERSION < 130300 - -config CC_HAS_ASM_GOTO_OUTPUT - def_bool y - depends on !GCC_ASM_GOTO_OUTPUT_BROKEN - depends on $(success,echo 'int foo(int x) { asm goto ("": "=3Dr"(x) ::: b= ar); return x; bar: return 0; }' | $(CC) -x c - -c -o /dev/null) - -config CC_HAS_ASM_GOTO_TIED_OUTPUT - depends on CC_HAS_ASM_GOTO_OUTPUT - # Detect buggy gcc and clang, fixed in gcc-11 clang-14. - def_bool $(success,echo 'int foo(int *x) { asm goto (".long (%l[bar]) - .= ": "+m"(*x) ::: bar); return *x; bar: return 0; }' | $CC -x c - -c -o /dev/= null) - -config TOOLS_SUPPORT_RELR - def_bool $(success,env "CC=3D$(CC)" "LD=3D$(LD)" "NM=3D$(NM)" "OBJCOPY=3D= $(OBJCOPY)" $(srctree)/scripts/tools-support-relr.sh) - -config CC_HAS_ASM_INLINE - def_bool $(success,echo 'void foo(void) { asm inline (""); }' | $(CC) -x = c - -c -o /dev/null) - -config CC_HAS_ASSUME - bool - # clang needs to be at least 19.1.0 since the meaning of the assume - # attribute changed: - # https://github.com/llvm/llvm-project/commit/c44fa3e8a9a44c2e9a575768a3c= 185354b9f6c17 - default y if CC_IS_CLANG && CLANG_VERSION >=3D 190100 - # supported since gcc 13.1.0 - # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D106654 - default y if CC_IS_GCC && GCC_VERSION >=3D 130100 - -config CC_HAS_NO_PROFILE_FN_ATTR - def_bool $(success,echo '__attribute__((no_profile_instrument_function)) = int x();' | $(CC) -x c - -c -o /dev/null -Werror) - -config CC_HAS_COUNTED_BY - bool - # clang needs to be at least 20.1.0 to avoid potential crashes - # when building structures that contain __counted_by - # https://github.com/ClangBuiltLinux/linux/issues/2114 - # https://github.com/llvm/llvm-project/commit/160fb1121cdf703c3ef5e61fb26= c5659eb581489 - default y if CC_IS_CLANG && CLANG_VERSION >=3D 200100 - # supported since gcc 15.1.0 - # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D108896 - default y if CC_IS_GCC && GCC_VERSION >=3D 150100 - -config CC_HAS_COUNTED_BY_PTR - bool - # supported since clang 22 - default y if CC_IS_CLANG && CLANG_VERSION >=3D 220100 - # supported since gcc 16.0.0 - default y if CC_IS_GCC && GCC_VERSION >=3D 160000 - -config CC_HAS_BROKEN_COUNTED_BY_REF - bool - # https://github.com/llvm/llvm-project/issues/182575 - default y if CC_IS_CLANG && CLANG_VERSION < 220100 - -config CC_HAS_ALLOC_TOKEN - def_bool $(cc-option,-falloc-token-max=3D123) - -config CC_HAS_MULTIDIMENSIONAL_NONSTRING - def_bool $(success,echo 'char tag[][4] __attribute__((__nonstring__)) =3D= { };' | $(CC) $(CLANG_FLAGS) -x c - -c -o /dev/null -Werror) - -config LD_CAN_USE_KEEP_IN_OVERLAY - # ld.lld prior to 21.0.0 did not support KEEP within an overlay descripti= on - # https://github.com/llvm/llvm-project/pull/130661 - def_bool LD_IS_BFD || LLD_VERSION >=3D 210000 - -config RUSTC_HAS_SPAN_FILE - def_bool RUSTC_VERSION >=3D 108800 - -config RUSTC_HAS_UNNECESSARY_TRANSMUTES - def_bool RUSTC_VERSION >=3D 108800 - -config RUSTC_HAS_FILE_WITH_NUL - def_bool RUSTC_VERSION >=3D 108900 - -config RUSTC_HAS_FILE_AS_C_STR - def_bool RUSTC_VERSION >=3D 109100 - -config RUSTC_HAS_SUSPICIOUS_RUNTIME_SYMBOL_DEFINITIONS - def_bool RUSTC_VERSION >=3D 109800 - -config PAHOLE_VERSION - int - default "$(PAHOLE_VERSION)" +source "scripts/Kconfig.toolchain" =20 config CONSTRUCTORS bool diff --git a/scripts/Kconfig.toolchain b/scripts/Kconfig.toolchain new file mode 100644 index 000000000000..fac36246451f --- /dev/null +++ b/scripts/Kconfig.toolchain @@ -0,0 +1,198 @@ +# SPDX-License-Identifier: GPL-2.0-only +config CC_VERSION_TEXT + string + default "$(CC_VERSION_TEXT)" + help + This is used in unclear ways: + + - Re-run Kconfig when the compiler is updated + The 'default' property references the environment variable, + CC_VERSION_TEXT so it is recorded in include/config/auto.conf.cmd. + When the compiler is updated, Kconfig will be invoked. + + - Ensure full rebuild when the compiler is updated + include/linux/compiler-version.h contains this option in the comment + line so fixdep adds include/config/CC_VERSION_TEXT into the + auto-generated dependency. When the compiler is updated, syncconfig + will touch it and then every file will be rebuilt. + +config CC_IS_GCC + def_bool $(success,test "$(cc-name)" =3D GCC) + +config GCC_VERSION + int + default $(cc-version) if CC_IS_GCC + default 0 + +config CC_IS_CLANG + def_bool $(success,test "$(cc-name)" =3D Clang) + +config CLANG_VERSION + int + default $(cc-version) if CC_IS_CLANG + default 0 + +config AS_IS_GNU + def_bool $(success,test "$(as-name)" =3D GNU) + +config AS_IS_LLVM + def_bool $(success,test "$(as-name)" =3D LLVM) + +config AS_VERSION + int + # Use clang version if this is the integrated assembler + default CLANG_VERSION if AS_IS_LLVM + default $(as-version) + +config LD_IS_BFD + def_bool $(success,test "$(ld-name)" =3D BFD) + +config LD_VERSION + int + default $(ld-version) if LD_IS_BFD + default 0 + +config LD_IS_LLD + def_bool $(success,test "$(ld-name)" =3D LLD) + +config LLD_VERSION + int + default $(ld-version) if LD_IS_LLD + default 0 + +config RUSTC_VERSION + int + default $(rustc-version) + help + It does not depend on `RUST` since that one may need to use the version + in a `depends on`. + +config RUST_IS_AVAILABLE + def_bool $(success,$(srctree)/scripts/rust_is_available.sh) + help + This shows whether a suitable Rust toolchain is available (found). + + Please see Documentation/rust/quick-start.rst for instructions on how + to satisfy the build requirements of Rust support. + + In particular, the Makefile target 'rustavailable' is useful to check + why the Rust toolchain is not being detected. + +config RUSTC_LLVM_VERSION + int + default $(rustc-llvm-version) + +config RUSTC_LLVM_MAJOR_VERSION + int + default $(shell,expr $(rustc-llvm-version) / 10000) + +config RUSTC_CLANG_LLVM_COMPATIBLE + bool + default y if CC_IS_CLANG && RUSTC_LLVM_MAJOR_VERSION =3D $(shell,expr $(c= c-version) / 10000) + help + This indicates whether Rust and Clang use LLVM of the same major + version. + + Operations involving handling LLVM IR or bitcode (e.g. cross-language + LTO) require the same LLVM major version to work properly. For best + compatibility it is recommended that the exact same LLVM is used. + +config ARCH_HAS_CC_CAN_LINK + bool + +config CC_CAN_LINK + bool + default ARCH_CC_CAN_LINK if ARCH_HAS_CC_CAN_LINK + default $(cc_can_link_user,$(m64-flag)) if 64BIT + default $(cc_can_link_user,$(m32-flag)) + +# Fixed in GCC 14, 13.3, 12.4 and 11.5 +# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D113921 +config GCC_ASM_GOTO_OUTPUT_BROKEN + bool + depends on CC_IS_GCC + default y if GCC_VERSION < 110500 + default y if GCC_VERSION >=3D 120000 && GCC_VERSION < 120400 + default y if GCC_VERSION >=3D 130000 && GCC_VERSION < 130300 + +config CC_HAS_ASM_GOTO_OUTPUT + def_bool y + depends on !GCC_ASM_GOTO_OUTPUT_BROKEN + depends on $(success,echo 'int foo(int x) { asm goto ("": "=3Dr"(x) ::: b= ar); return x; bar: return 0; }' | $(CC) -x c - -c -o /dev/null) + +config CC_HAS_ASM_GOTO_TIED_OUTPUT + depends on CC_HAS_ASM_GOTO_OUTPUT + # Detect buggy gcc and clang, fixed in gcc-11 clang-14. + def_bool $(success,echo 'int foo(int *x) { asm goto (".long (%l[bar]) - .= ": "+m"(*x) ::: bar); return *x; bar: return 0; }' | $CC -x c - -c -o /dev/= null) + +config TOOLS_SUPPORT_RELR + def_bool $(success,env "CC=3D$(CC)" "LD=3D$(LD)" "NM=3D$(NM)" "OBJCOPY=3D= $(OBJCOPY)" $(srctree)/scripts/tools-support-relr.sh) + +config CC_HAS_ASM_INLINE + def_bool $(success,echo 'void foo(void) { asm inline (""); }' | $(CC) -x = c - -c -o /dev/null) + +config CC_HAS_ASSUME + bool + # clang needs to be at least 19.1.0 since the meaning of the assume + # attribute changed: + # https://github.com/llvm/llvm-project/commit/c44fa3e8a9a44c2e9a575768a3c= 185354b9f6c17 + default y if CC_IS_CLANG && CLANG_VERSION >=3D 190100 + # supported since gcc 13.1.0 + # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D106654 + default y if CC_IS_GCC && GCC_VERSION >=3D 130100 + +config CC_HAS_NO_PROFILE_FN_ATTR + def_bool $(success,echo '__attribute__((no_profile_instrument_function)) = int x();' | $(CC) -x c - -c -o /dev/null -Werror) + +config CC_HAS_COUNTED_BY + bool + # clang needs to be at least 20.1.0 to avoid potential crashes + # when building structures that contain __counted_by + # https://github.com/ClangBuiltLinux/linux/issues/2114 + # https://github.com/llvm/llvm-project/commit/160fb1121cdf703c3ef5e61fb26= c5659eb581489 + default y if CC_IS_CLANG && CLANG_VERSION >=3D 200100 + # supported since gcc 15.1.0 + # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D108896 + default y if CC_IS_GCC && GCC_VERSION >=3D 150100 + +config CC_HAS_COUNTED_BY_PTR + bool + # supported since clang 22 + default y if CC_IS_CLANG && CLANG_VERSION >=3D 220100 + # supported since gcc 16.0.0 + default y if CC_IS_GCC && GCC_VERSION >=3D 160000 + +config CC_HAS_BROKEN_COUNTED_BY_REF + bool + # https://github.com/llvm/llvm-project/issues/182575 + default y if CC_IS_CLANG && CLANG_VERSION < 220100 + +config CC_HAS_ALLOC_TOKEN + def_bool $(cc-option,-falloc-token-max=3D123) + +config CC_HAS_MULTIDIMENSIONAL_NONSTRING + def_bool $(success,echo 'char tag[][4] __attribute__((__nonstring__)) =3D= { };' | $(CC) $(CLANG_FLAGS) -x c - -c -o /dev/null -Werror) + +config LD_CAN_USE_KEEP_IN_OVERLAY + # ld.lld prior to 21.0.0 did not support KEEP within an overlay descripti= on + # https://github.com/llvm/llvm-project/pull/130661 + def_bool LD_IS_BFD || LLD_VERSION >=3D 210000 + +config RUSTC_HAS_SPAN_FILE + def_bool RUSTC_VERSION >=3D 108800 + +config RUSTC_HAS_UNNECESSARY_TRANSMUTES + def_bool RUSTC_VERSION >=3D 108800 + +config RUSTC_HAS_FILE_WITH_NUL + def_bool RUSTC_VERSION >=3D 108900 + +config RUSTC_HAS_FILE_AS_C_STR + def_bool RUSTC_VERSION >=3D 109100 + +config RUSTC_HAS_SUSPICIOUS_RUNTIME_SYMBOL_DEFINITIONS + def_bool RUSTC_VERSION >=3D 109800 + +config PAHOLE_VERSION + int + default "$(PAHOLE_VERSION)" --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6FA83E6DFF; Wed, 23 Sep 2026 17:20:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184059; cv=none; b=EvqQwt3wWyBLuW+clYJRdp0m998FBJdqK/29ACyzjU1AT3+KxzX/f1IvZsdK5YqDPq1DusvRvv6u+QoQrNCI9Ymex/MIVE/5XuVBWwLqpY8whCO5qdJU5q/0rTw90OmpOzUiGYI3/yyv4pgWDfbDNYrcEiUxGSLvGt/z62qb2Wg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184059; c=relaxed/simple; bh=PDJB8j9sKc9Q/yg7cUV7TTvciusJhucipt52hAU2ZhY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AeFFN2URadWO+hqWqvnlwp3Ze6tmafAMR5sEy1pMAN/80XVnGLiQqNB/da06Oc9NGVXJXo5/oiL7pCzcDokVFJ46v9Y3yxMh3b+B3+JhdAyCALza7y66BOZEFB3Z0T4RuqefigiFTiSdMLFsNmsRpt5yfXvVZm3ioECDLEI8ANk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=n4ivzPGH; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="n4ivzPGH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B380F1F00893; Wed, 23 Sep 2026 17:20:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184056; bh=B1ULND6qUkIjbcoTYBYFJ2YhtgqM4e1Z1reiowbMJ8k=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=n4ivzPGHFOaVKa0QM4qp2cl2pNZbc5XzOdstT6g/HfEo5a5jBkxSx30mSH+AdQFxy qh/4kP+atkWGJ12sRHfF5BC4Hr40Yq971TO+GkE3/7UuXYYYUHrw9CmjdZz34wiivg E1qlleJKnrfo8QwtUxJHyuKWZbPtfF51cCvHLxgEsoHy8Vp8T65suXiIZck7r6LD1h nC0g+CpMaK5vaKpiwnO0J5JTrm8C8zfj3v+u4Bz8Ol2C2vBLHCef/fj/MYg9WKN2H6 HT/aItgJrz80pPqDJlwkiKjHdRx0pQd/NDS2rvmAZxi0dCVok5YDDuO/0HKCr38Akj vRAPLlEhkZvuw== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:01 +0100 Subject: [PATCH v4 12/22] kbuild: avoid re-running compiler and linker probes Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-12-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=18854; i=ljs@kernel.org; h=from:subject:message-id; bh=PDJB8j9sKc9Q/yg7cUV7TTvciusJhucipt52hAU2ZhY=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF5+cW7Oi8ofPw/7TXxRM1M57JnkyW/PVSU9Xv9yz zQQ2Cv4s6OUhUGMi0FWTJHl+Rfx/UEiYfM6L/i7wcxhZQIZwsDFKQATmfGd4X+a+5E7fKf/5cZe db9Y3GV04dtJfpmHWRymkRWX3Dy3bLvEyPBtR15Bo/riG8w3D/1PLmVO/bwz6f865sSpRtueTnt jUcoGAA== X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Each kernel make invocation begins with ~30 compiler and linker runs each of which performs duplicate probe for a number of compiler and linker options. This is useless work - the compiler and its version is known, so use these to determine which options are available, once. Using the newly introduced cc-option-str, track the result of each probe in CC_OPT_*, LD_OPT_* variables and have Makefile's simply append the symbol. Some care has to be taken around negative options, i.e. -Wno-xxx, as gcc accepts an unknown -Wno- without complaint and only complains later, so instead probe the positive form for these cases. Use CC_HAS_* For those probes that gate a Makefile conditional rather than a simple option, as per the established pattern for this. This change avoids a subtle issue with ppc32 (big endian) - clang doesn't accept -fno-stack-clash-protection on this sub-target, so keep this option in the Makefile. With gcc and clang, defconfig and allmodconfig, the recorded command lines are unchanged, a build with nothing to do rebuilds nothing and W=3D1 continues to function correctly. Doing this improves all builds, but has a particularly positive impact on no-op builds (builds where nothing has changed). Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, no-op make, gcc 0.78s 0.60s -0.18s (-23%) x86 defconfig, no-op make, clang 0.94s 0.66s -0.28s (-30%) x86 defconfig, touch mm/vma.c, gcc 8.6s 8.5s -0.16s (-2%) x86 defconfig, touch mm/vma.c, clang 8.0s 7.7s -0.24s (-3%) x86 defconfig, clean, clang 29.8s 29.4s -0.37s (-1%) x86 allmodconfig, no-op make, gcc 12.9s 12.7s -0.23s (-2%) x86 allmodconfig, no-op make, clang 14.2s 13.9s -0.29s (-2%) x86 allmodconfig, touch mm/vma.c, clang 32.2s 31.9s -0.30s (-1%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- Makefile | 23 +++++------ arch/arm64/kernel/pi/Makefile | 2 +- arch/riscv/kernel/pi/Makefile | 2 +- arch/x86/Kconfig | 17 ++++++++ arch/x86/Makefile | 12 +++--- drivers/firmware/efi/libstub/Makefile | 2 +- scripts/Kconfig.toolchain | 74 +++++++++++++++++++++++++++++++= ++++ scripts/Makefile.warn | 28 ++++++------- 8 files changed, 126 insertions(+), 34 deletions(-) diff --git a/Makefile b/Makefile index 554c9003a4d9..cbfee8d325a5 100644 --- a/Makefile +++ b/Makefile @@ -946,8 +946,8 @@ KBUILD_RUSTFLAGS +=3D -Coverflow-checks=3D$(if $(CONFIG= _RUST_OVERFLOW_CHECKS),y,n) ifdef CONFIG_CC_IS_GCC # gcc-10 renamed --param=3Dallow-store-data-races=3D0 to # -fno-allow-store-data-races. -KBUILD_CFLAGS +=3D $(call cc-option,--param=3Dallow-store-data-races=3D0) -KBUILD_CFLAGS +=3D $(call cc-option,-fno-allow-store-data-races) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_ALLOW_STORE_DATA_RACES_PARAM) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_NO_ALLOW_STORE_DATA_RACES) endif =20 ifdef CONFIG_READABLE_ASM @@ -1011,18 +1011,20 @@ endif endif =20 # Explicitly clear padding bits during variable initialization -KBUILD_CFLAGS +=3D $(call cc-option,-fzero-init-padding-bits=3Dall) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_ZERO_INIT_PADDING_BITS) =20 # While VLAs have been removed, GCC produces unreachable stack probes # for the randomize_kstack_offset feature. Disable it for all compilers. +# Probed here rather than in Kconfig as clang only accepts it for some +# sub-targets, so the architecture's flags matter. KBUILD_CFLAGS +=3D $(call cc-option, -fno-stack-clash-protection) =20 # Get details on warnings generated due to GCC value tracking. -KBUILD_CFLAGS +=3D $(call cc-option, -fdiagnostics-show-context=3D2) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_DIAGNOSTICS_SHOW_CONTEXT) =20 # Show inlining notes for __attribute__((warning/error)) call chains. # GCC supports this unconditionally while Clang 23+ provides a flag. -KBUILD_CFLAGS +=3D $(call cc-option, -fdiagnostics-show-inlining-chain) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_DIAGNOSTICS_SHOW_INLINING_CHAIN) =20 # Clear used registers at func exit (to reduce data lifetime and ROP gadge= ts). ifdef CONFIG_ZERO_CALL_USED_REGS @@ -1033,7 +1035,7 @@ ifdef CONFIG_FUNCTION_TRACER ifdef CONFIG_FTRACE_MCOUNT_USE_CC CC_FLAGS_FTRACE +=3D -mrecord-mcount ifdef CONFIG_HAVE_NOP_MCOUNT - ifeq ($(call cc-option-yn, -mnop-mcount),y) + ifdef CONFIG_CC_HAS_MNOP_MCOUNT CC_FLAGS_FTRACE +=3D -mnop-mcount CC_FLAGS_USING +=3D -DCC_USING_NOP_MCOUNT endif @@ -1051,8 +1053,7 @@ ifdef CONFIG_FTRACE_MCOUNT_USE_RECORDMCOUNT endif endif ifdef CONFIG_HAVE_FENTRY - # s390-linux-gnu-gcc did not support -mfentry until gcc-9. - ifeq ($(call cc-option-yn, -mfentry),y) + ifdef CONFIG_CC_HAS_MFENTRY CC_FLAGS_FTRACE +=3D -mfentry CC_FLAGS_USING +=3D -DCC_USING_FENTRY endif @@ -1160,7 +1161,7 @@ NOSTDINC_FLAGS +=3D -nostdinc # the kernel uses only C99 flexible arrays for dynamically sized trailing # arrays. Enforce this for everything that may examine structure sizes and # perform bounds checking. -KBUILD_CFLAGS +=3D $(call cc-option, -fstrict-flex-arrays=3D3) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_STRICT_FLEX_ARRAYS) =20 # disable invalid "can't wrap" optimizations for signed / pointers KBUILD_CFLAGS +=3D -fno-strict-overflow @@ -1243,11 +1244,11 @@ LDFLAGS_vmlinux +=3D --build-id=3Dsha1 # COMDAT-deduplicated sections. Use --force-group-allocation to resolve th= ese # groups when linking modules. The option is available from ld.bfd 2.29 and # ld.lld 19.1.0. -KBUILD_LDFLAGS_MODULE +=3D $(call ld-option,--force-group-allocation) +KBUILD_LDFLAGS_MODULE +=3D $(CONFIG_LD_OPT_FORCE_GROUP_ALLOCATION) =20 KBUILD_LDFLAGS +=3D -z noexecstack ifeq ($(CONFIG_LD_IS_BFD),y) -KBUILD_LDFLAGS +=3D $(call ld-option,--no-warn-rwx-segments) +KBUILD_LDFLAGS +=3D $(CONFIG_LD_OPT_NO_WARN_RWX_SEGMENTS) endif =20 ifeq ($(CONFIG_STRIP_ASM_SYMS),y) diff --git a/arch/arm64/kernel/pi/Makefile b/arch/arm64/kernel/pi/Makefile index be92d73c25b2..4a0cbedfdcd6 100644 --- a/arch/arm64/kernel/pi/Makefile +++ b/arch/arm64/kernel/pi/Makefile @@ -9,7 +9,7 @@ KBUILD_CFLAGS :=3D $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFL= AGS)) -fpie \ -include $(srctree)/include/linux/hidden.h \ -D__DISABLE_EXPORTS -ffreestanding -D__NO_FORTIFY \ -fno-asynchronous-unwind-tables -fno-unwind-tables \ - $(call cc-option,-fno-addrsig) + $(CONFIG_CC_OPT_NO_ADDRSIG) =20 # this code may run with the MMU off so disable unaligned accesses CFLAGS_map_range.o +=3D -mstrict-align diff --git a/arch/riscv/kernel/pi/Makefile b/arch/riscv/kernel/pi/Makefile index bc098edac898..4c8e52b3f176 100644 --- a/arch/riscv/kernel/pi/Makefile +++ b/arch/riscv/kernel/pi/Makefile @@ -8,7 +8,7 @@ KBUILD_CFLAGS :=3D $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFL= AGS)) -fpie \ -include $(srctree)/include/linux/hidden.h \ -D__DISABLE_EXPORTS -ffreestanding \ -fno-asynchronous-unwind-tables -fno-unwind-tables \ - $(call cc-option,-fno-addrsig) + $(CONFIG_CC_OPT_NO_ADDRSIG) =20 # Disable LTO KBUILD_CFLAGS :=3D $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS)) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 15fd9ec5ecac..0fea2fc4ba1d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2364,6 +2364,23 @@ config CC_HAS_KCFI_ARITY def_bool $(cc-option,-fsanitize=3Dkcfi -fsanitize-kcfi-arity) depends on CC_IS_CLANG && !RUST =20 +config CC_OPT_INDIRECT_BRANCH_CS_PREFIX + def_string "$(cc-option-str,-mindirect-branch-cs-prefix)" + +config CC_OPT_CF_PROTECTION_NONE + def_string "$(cc-option-str,-fcf-protection=3Dnone)" + +# with jump tables off: the compilers use NOTRACK for them, which kernel I= BT +# does not allow +config CC_OPT_CF_PROTECTION_BRANCH + def_string "$(cc-option-str,-fcf-protection=3Dbranch -fno-jump-tables)" + +config CC_OPT_FALIGN_JUMPS + def_string "$(cc-option-str,-falign-jumps=3D1)" + +config CC_OPT_FALIGN_LOOPS + def_string "$(cc-option-str,-falign-loops=3D1)" + config FUNCTION_PADDING_CFI int default 59 if FUNCTION_ALIGNMENT_64B diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 598f178102ee..aac30ba51b2f 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -20,7 +20,7 @@ ifdef CONFIG_CC_IS_CLANG RETPOLINE_CFLAGS :=3D -mretpoline-external-thunk RETPOLINE_VDSO_CFLAGS :=3D -mretpoline endif -RETPOLINE_CFLAGS +=3D $(call cc-option,-mindirect-branch-cs-prefix) +RETPOLINE_CFLAGS +=3D $(CONFIG_CC_OPT_INDIRECT_BRANCH_CS_PREFIX) =20 ifdef CONFIG_MITIGATION_RETHUNK RETHUNK_CFLAGS :=3D -mfunction-return=3Dthunk-extern @@ -52,7 +52,7 @@ REALMODE_CFLAGS :=3D $(CC_FLAGS_DIALECT) -m16 -g -Os \ -DDISABLE_BRANCH_PROFILING -D__DISABLE_EXPORTS \ -Wall -Wstrict-prototypes -march=3Di386 -mregparm=3D3 \ -fno-strict-aliasing -fomit-frame-pointer -fno-pic \ - -mno-mmx -mno-sse $(call cc-option,-fcf-protection=3Dnone) + -mno-mmx -mno-sse $(CONFIG_CC_OPT_CF_PROTECTION_NONE) =20 REALMODE_CFLAGS +=3D -ffreestanding REALMODE_CFLAGS +=3D -fno-stack-protector @@ -99,10 +99,10 @@ ifeq ($(CONFIG_X86_KERNEL_IBT),y) # # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D104816 # -KBUILD_CFLAGS +=3D $(call cc-option,-fcf-protection=3Dbranch -fno-jump-tab= les) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_CF_PROTECTION_BRANCH) KBUILD_RUSTFLAGS +=3D -Zcf-protection=3Dbranch $(if $(call rustc-min-versi= on,109300),-Cjump-tables=3Dn,-Zno-jump-tables) else -KBUILD_CFLAGS +=3D $(call cc-option,-fcf-protection=3Dnone) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_CF_PROTECTION_NONE) endif =20 ifeq ($(CONFIG_X86_32),y) @@ -138,10 +138,10 @@ else KBUILD_CFLAGS +=3D -m64 =20 # Align jump targets to 1 byte, not the default 16 bytes: - KBUILD_CFLAGS +=3D $(call cc-option,-falign-jumps=3D1) + KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_FALIGN_JUMPS) =20 # Pack loops tightly as well: - KBUILD_CFLAGS +=3D $(call cc-option,-falign-loops=3D1) + KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_FALIGN_LOOPS) =20 # Don't autogenerate traditional x87 instructions KBUILD_CFLAGS +=3D -mno-80387 diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/l= ibstub/Makefile index 77a2b2d74f3f..415f63105b48 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -40,7 +40,7 @@ KBUILD_CFLAGS :=3D $(subst $(CC_FLAGS_FTRACE),,$(cflags= -y)) \ -D__NO_FORTIFY \ -ffreestanding \ -fno-stack-protector \ - $(call cc-option,-fno-addrsig) \ + $(CONFIG_CC_OPT_NO_ADDRSIG) \ -D__DISABLE_EXPORTS =20 # diff --git a/scripts/Kconfig.toolchain b/scripts/Kconfig.toolchain index fac36246451f..d708a175fe48 100644 --- a/scripts/Kconfig.toolchain +++ b/scripts/Kconfig.toolchain @@ -144,6 +144,80 @@ config CC_HAS_ASSUME config CC_HAS_NO_PROFILE_FN_ATTR def_bool $(success,echo '__attribute__((no_profile_instrument_function)) = int x();' | $(CC) -x c - -c -o /dev/null -Werror) =20 +config CC_OPT_ZERO_INIT_PADDING_BITS + def_string "$(cc-option-str,-fzero-init-padding-bits=3Dall)" + +config CC_OPT_NO_ADDRSIG + def_string "$(cc-option-str,-fno-addrsig)" + +config CC_OPT_DIAGNOSTICS_SHOW_CONTEXT + def_string "$(cc-option-str,-fdiagnostics-show-context=3D2)" + +config CC_OPT_DIAGNOSTICS_SHOW_INLINING_CHAIN + def_string "$(cc-option-str,-fdiagnostics-show-inlining-chain)" + +config CC_OPT_STRICT_FLEX_ARRAYS + def_string "$(cc-option-str,-fstrict-flex-arrays=3D3)" + +# gcc-10 renamed --param=3Dallow-store-data-races=3D0 to -fno-allow-store-= data-races +config CC_OPT_ALLOW_STORE_DATA_RACES_PARAM + def_string "$(cc-option-str,--param=3Dallow-store-data-races=3D0)" if CC_= IS_GCC + +config CC_OPT_NO_ALLOW_STORE_DATA_RACES + def_string "$(cc-option-str,-fno-allow-store-data-races)" if CC_IS_GCC + +config CC_HAS_MNOP_MCOUNT + def_bool $(cc-option,-mnop-mcount) + +# s390-linux-gnu-gcc did not support -mfentry until gcc-9. +config CC_HAS_MFENTRY + def_bool $(cc-option,-mfentry) + +config LD_OPT_FORCE_GROUP_ALLOCATION + def_string "$(ld-option-str,--force-group-allocation)" + +config LD_OPT_NO_WARN_RWX_SEGMENTS + def_string "$(ld-option-str,--no-warn-rwx-segments)" + +config CC_OPT_WNO_ADDRESS_OF_PACKED_MEMBER + def_string "-Wno-address-of-packed-member" if $(cc-option,-Waddress-of-pa= cked-member) + +config CC_OPT_WNO_FORMAT_OVERFLOW_NON_KPRINTF + def_string "-Wno-format-overflow-non-kprintf" if CC_IS_CLANG && $(cc-opti= on,-Wformat-overflow-non-kprintf) + +config CC_OPT_WNO_FORMAT_TRUNCATION_NON_KPRINTF + def_string "-Wno-format-truncation-non-kprintf" if CC_IS_CLANG && $(cc-op= tion,-Wformat-truncation-non-kprintf) + +config CC_OPT_WNO_DEFAULT_CONST_INIT_UNSAFE + def_string "-Wno-default-const-init-unsafe" if CC_IS_CLANG && $(cc-option= ,-Wdefault-const-init-unsafe) + +config CC_OPT_WNO_DANGLING_POINTER + def_string "-Wno-dangling-pointer" if $(cc-option,-Wdangling-pointer) + +config CC_OPT_WVLA_LARGER_THAN + def_string "$(cc-option-str,-Wvla-larger-than=3D1)" + +config CC_HAS_WSTRINGOP_OVERFLOW + def_bool $(cc-option,-Wstringop-overflow) + +config CC_OPT_WNO_UNTERMINATED_STRING_INITIALIZATION + def_string "-Wno-unterminated-string-initialization" if $(cc-option,-Wunt= erminated-string-initialization) + +config CC_OPT_WERROR_DESIGNATED_INIT + def_string "$(cc-option-str,-Werror=3Ddesignated-init)" + +config CC_OPT_WENUM_CONVERSION + def_string "$(cc-option-str,-Wenum-conversion)" + +config CC_OPT_WNO_PACKED_NOT_ALIGNED + def_string "-Wno-packed-not-aligned" if $(cc-option,-Wpacked-not-aligned) + +config CC_OPT_WNO_FORMAT_OVERFLOW + def_string "-Wno-format-overflow" if $(cc-option,-Wformat-overflow) + +config CC_OPT_WNO_STRINGOP_TRUNCATION + def_string "-Wno-stringop-truncation" if $(cc-option,-Wstringop-truncatio= n) + config CC_HAS_COUNTED_BY bool # clang needs to be at least 20.1.0 to avoid potential crashes diff --git a/scripts/Makefile.warn b/scripts/Makefile.warn index 35af7d6c6d18..556fa249ca07 100644 --- a/scripts/Makefile.warn +++ b/scripts/Makefile.warn @@ -17,7 +17,7 @@ KBUILD_CFLAGS +=3D -Werror=3Dstrict-prototypes KBUILD_CFLAGS +=3D -Wno-format-security KBUILD_CFLAGS +=3D -Wno-trigraphs KBUILD_CFLAGS +=3D -Wno-frame-address -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-address-of-packed-member) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_ADDRESS_OF_PACKED_MEMBER) KBUILD_CFLAGS +=3D -Wmissing-declarations KBUILD_CFLAGS +=3D -Wmissing-prototypes =20 @@ -30,8 +30,8 @@ KBUILD_CFLAGS-$(CONFIG_CC_NO_ARRAY_BOUNDS) +=3D -Wno-arra= y-bounds ifdef CONFIG_CC_IS_CLANG # Clang checks for overflow/truncation with '%p', while GCC does not: # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D111219 -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-format-overflow-non-kprintf) -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-format-truncation-non-kprintf) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_FORMAT_OVERFLOW_NON_KPRINTF) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_FORMAT_TRUNCATION_NON_KPRINTF) =20 # Clang may emit a warning when a const variable, such as the dummy variab= les # in typecheck(), or const member of an aggregate type are not initialized, @@ -43,7 +43,7 @@ KBUILD_CFLAGS +=3D $(call cc-option, -Wno-format-truncati= on-non-kprintf) # disabled with this same switch, there should not be too much coverage lo= st # because -Wuninitialized will still flag when an uninitialized const vari= able # is used. -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-default-const-init-unsafe) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_DEFAULT_CONST_INIT_UNSAFE) else =20 # gcc inanely warns about local variables called 'main' @@ -54,7 +54,7 @@ endif KBUILD_CFLAGS +=3D -Wno-type-limits =20 # These result in bogus false positives -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-dangling-pointer) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_DANGLING_POINTER) =20 # Stack Variable Length Arrays (VLAs) must not be used in the kernel. # Function array parameters should, however, be usable, but -Wvla will @@ -62,7 +62,7 @@ KBUILD_CFLAGS +=3D $(call cc-option, -Wno-dangling-pointe= r) # types, so depend on GCC for now to keep stack VLAs out of the tree. # https://github.com/llvm/llvm-project/issues/57098 # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D98217 -KBUILD_CFLAGS +=3D $(call cc-option,-Wvla-larger-than=3D1) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WVLA_LARGER_THAN) =20 # disable pointer signed / unsigned warnings in gcc 4.0 KBUILD_CFLAGS +=3D -Wno-pointer-sign @@ -73,11 +73,11 @@ KBUILD_CFLAGS +=3D -Wno-pointer-sign KBUILD_CFLAGS +=3D -Wcast-function-type =20 # Currently, disable -Wstringop-overflow for GCC 11, globally. -KBUILD_CFLAGS-$(CONFIG_CC_NO_STRINGOP_OVERFLOW) +=3D $(call cc-option, -Wn= o-stringop-overflow) -KBUILD_CFLAGS-$(CONFIG_CC_STRINGOP_OVERFLOW) +=3D $(call cc-option, -Wstri= ngop-overflow) +KBUILD_CFLAGS-$(CONFIG_CC_NO_STRINGOP_OVERFLOW) +=3D $(if $(CONFIG_CC_HAS_= WSTRINGOP_OVERFLOW),-Wno-stringop-overflow) +KBUILD_CFLAGS-$(CONFIG_CC_STRINGOP_OVERFLOW) +=3D $(if $(CONFIG_CC_HAS_WST= RINGOP_OVERFLOW),-Wstringop-overflow) =20 # Currently, disable -Wunterminated-string-initialization as broken -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-unterminated-string-initializati= on) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_UNTERMINATED_STRING_INITIALIZATION) =20 # The allocators already balk at large sizes, so silence the compiler # warnings for bounds checks involving those possible values. While @@ -100,10 +100,10 @@ KBUILD_CFLAGS +=3D -Werror=3Ddate-time KBUILD_CFLAGS +=3D -Werror=3Dincompatible-pointer-types =20 # Require designated initializers for all marked structures -KBUILD_CFLAGS +=3D $(call cc-option,-Werror=3Ddesignated-init) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WERROR_DESIGNATED_INIT) =20 # Warn if there is an enum types mismatch -KBUILD_CFLAGS +=3D $(call cc-option,-Wenum-conversion) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WENUM_CONVERSION) =20 KBUILD_CFLAGS +=3D -Wunused =20 @@ -125,12 +125,12 @@ else # Suppress them by using -Wno... except for W=3D1. KBUILD_CFLAGS +=3D -Wno-unused-but-set-variable KBUILD_CFLAGS +=3D -Wno-unused-const-variable -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-packed-not-aligned) -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-format-overflow) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_PACKED_NOT_ALIGNED) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_FORMAT_OVERFLOW) ifdef CONFIG_CC_IS_GCC KBUILD_CFLAGS +=3D -Wno-format-truncation endif -KBUILD_CFLAGS +=3D $(call cc-option, -Wno-stringop-truncation) +KBUILD_CFLAGS +=3D $(CONFIG_CC_OPT_WNO_STRINGOP_TRUNCATION) =20 KBUILD_CFLAGS +=3D -Wno-override-init # alias for -Wno-initializer-overrid= es in clang =20 --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60CB73EA94C; Wed, 23 Sep 2026 17:21:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184078; cv=none; b=MKS79S9Op/zbz+vr416oZLmOOgVR6dd3TH+v8he8VrtqT51YIVLmfhG9a1kZy1Kj9N8/t9p04FD3e+T96+uu98yK0GWKHRn2nn+ljTDH5JNcDqWZW/0RqZYG7I2RHoW4lUIsqB0RpqZ7JPVTh1B5D27QcTz51uOsIaDrTLHPecU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184078; c=relaxed/simple; bh=Ec1/kqBneOXNUE0gtcHjI1qdlGW1hFvqVFIOwSgkaek=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WeW9g7QU0WSzxz3qZjPphr8SX7QdtYPHI7pHzAbSKrkkpl1haMVabZ//u1EFsKVtALlaIT8ghDdh+HKVR4lF9Cg1C2ceiBgQYiJ7MAvVxGc0mxuuaq0ctLhVX3PtaQT+SczeSfYtehdlmZk+2TXPQy0+MG1nEBjMLJ/2ykTkFy4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dOIjIFNy; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dOIjIFNy" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1B4D81F000FF; Wed, 23 Sep 2026 17:20:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184069; bh=B/2HtZkmT+u7/JWW+lpRsHLnC+c6qMifLdIPC//XmMc=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=dOIjIFNyYMZVkaNYw7B1juz2dOCBElSaPHpI+UtrvhExEprOU9xI8O6IE32T/HaCN g7rwjr8MHf6ce/9Mgng/9QdYv11wNTKSjSWnN7p7G/7O6w9SCefLJ7YH8Z5irQUvRW 2vDh4ZT7dWQwsO2KAnrOf5iYyVF+UWUWRhtQ7f+pFgzTHiQnNWuI0EZYNCNI0cZCGr iyjP244uAUmb0n0y5x6WzgyxUV35IZdG5qijOet1qIeXF85pcsHPZm1x8DUxsxlxy7 TpDjNm5pj45DrpTaPQKyrAlqRJ2YNzDiWerPj2VHqSnb81LmSXqjnPDJbCbdJlmN17 yIqm0e5qFgJUQ== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:02 +0100 Subject: [PATCH v4 13/22] modpost: cache section relocation mismatch state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-13-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4714; i=ljs@kernel.org; h=from:subject:message-id; bh=Ec1/kqBneOXNUE0gtcHjI1qdlGW1hFvqVFIOwSgkaek=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF6Z8Vrf3M3J/OM99Rk7W8yf/jUSt9hryKe+fLLWh wlP+AxtO0pZGMS4GGTFFFmefxHfHyQSNq/zgr8bzBxWJpAhDFycAjARkTsM/6NLN7vfW1DwMIDR eGXj8qbctQ766nnPp20tiKr348qKLWb4nyJ0b4Js+o/oEOfvnwNUkgsCU/uCRA5UV0Rccn75vSy EFwA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 For every relocation modpost invokes check_section_mismatch() to determine whether there is any kind of mismatch between the source and destination, and if so which classification applies. Each time it does this it invokes section_mismatch() which iterates through the sectioncheck[] array every time it's called. When walked a relocation section the source is fixed and there aren't many targets, so the same names are looked up over and over again. Therefore cache not only mismatch categorisation but also whether a mismatch even exists for a given section and look up the sections in the cache. Special indices (undefined, absolute, common) take the uncached path as before. This results in very significant speed ups for allmodconfig incremental builds. modpost is on the serial tail of every build that links vmlinux. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 8.5s 8.3s -0.14s (-2%) x86 defconfig, touch mm/vma.c, clang 7.7s 7.6s -0.13s (-2%) x86 allmodconfig, touch mm/vma.c, gcc 33.9s 29.4s -4.5s (-13%) x86 allmodconfig, touch mm/vma.c, clang 31.9s 27.9s -4.0s (-13%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- scripts/mod/modpost.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++= ++-- 1 file changed, 64 insertions(+), 2 deletions(-) diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 75374c64b8cc..0fd43c8a89ea 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -1155,12 +1155,67 @@ static void check_export_symbol(struct module *mod,= struct elf_info *elf, name); } =20 +/* + * mismatch_cache[section index] -> + * 0 - uncached. + * -1 - no mismatch. + * >0 - mismatch index + 1. + */ +static int *mismatch_cache; + +static void init_mismatch_cache(unsigned int num_sections) +{ + mismatch_cache =3D xcalloc(num_sections, sizeof(*mismatch_cache)); +} + +static void reset_mismatch_cache(unsigned int num_sections) +{ + memset(mismatch_cache, 0, num_sections * sizeof(*mismatch_cache)); +} + +static void free_mismatch_cache(void) +{ + free(mismatch_cache); + mismatch_cache =3D NULL; +} + +static const struct sectioncheck +*cache_mismatch(unsigned int secndx, const struct sectioncheck *mismatch) +{ + if (!mismatch) { + mismatch_cache[secndx] =3D -1; + return NULL; + } + + mismatch_cache[secndx] =3D (mismatch - sectioncheck) + 1; + return mismatch; +} + +static const struct sectioncheck *get_section_mismatch(const char *fromsec, + const struct elf_info *elf, unsigned int secndx) +{ + int cached; + + if (secndx >=3D elf->num_sections) + return section_mismatch(fromsec, sec_name(elf, secndx)); + + cached =3D mismatch_cache[secndx]; + if (cached < 0) + return NULL; + if (cached > 0) + return §ioncheck[cached - 1]; + + return cache_mismatch(secndx, + section_mismatch(fromsec, sec_name(elf, secndx))); +} + static void check_section_mismatch(struct module *mod, struct elf_info *el= f, Elf_Sym *sym, unsigned int fsecndx, const char *fromsec, Elf_Addr faddr, Elf_Addr taddr) { - const char *tosec =3D sec_name(elf, get_secindex(elf, sym)); + const unsigned int to_secndx =3D get_secindex(elf, sym); + const char *tosec =3D sec_name(elf, to_secndx); const struct sectioncheck *mismatch; =20 if (module_enabled && elf->export_symbol_secndx =3D=3D fsecndx) { @@ -1168,7 +1223,7 @@ static void check_section_mismatch(struct module *mod= , struct elf_info *elf, return; } =20 - mismatch =3D section_mismatch(fromsec, tosec); + mismatch =3D get_section_mismatch(fromsec, elf, to_secndx); if (!mismatch) return; =20 @@ -1445,6 +1500,8 @@ static void check_sec_ref(struct module *mod, struct = elf_info *elf) { int i; =20 + init_mismatch_cache(elf->num_sections); + /* Walk through all sections */ for (i =3D 0; i < elf->num_sections; i++) { Elf_Shdr *sechdr =3D &elf->sechdrs[i]; @@ -1461,6 +1518,9 @@ static void check_sec_ref(struct module *mod, struct = elf_info *elf) if (match(secname, section_white_list)) continue; =20 + /* Reset cache per-section. */ + reset_mismatch_cache(elf->num_sections); + start =3D sym_get_data_by_offset(elf, i, 0); stop =3D start + sechdr->sh_size; =20 @@ -1472,6 +1532,8 @@ static void check_sec_ref(struct module *mod, struct = elf_info *elf) start, stop); } } + + free_mismatch_cache(); } =20 static char *remove_dot(char *s) --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 813063E6DD2; Wed, 23 Sep 2026 17:21:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184091; cv=none; b=On1H746ekubnMY1l3a+aJs+VJjFeVZQGZClwU4NCYfNI/BqdRLMTHmaQZpC81nxZTe1m5WNHabXU3XqdGoYc6zBwRT61f+HCrL6JFQec0EobKxTdnCKAqm58LFvlWl5pwk5mwIY9zpG4Kei/+1amfzEYa8EkaGZqbyhQsWFuztg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184091; c=relaxed/simple; bh=C7ggmJFDj4rQBqCHgur2cGTYGiZhCXldoiWa5ncWBaU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=EJSh8XMGGNkeW7v7oYKyIZt7Kb4Xr9kxJfkcJ3Lb+NxbWVVh2PhVsVKPq/2s9W/9D6Tjpfc9cCZnNur7liWU4h5uG269VSLVbYi2xB8E2FQTNYS9Cje4aOCLbvZAMOAxWJMyTm1ttI6tefmb1H2PU5ljb/Bn0af4BE7YOI+GJdk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IE8V6HZ1; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IE8V6HZ1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 913CB1F00893; Wed, 23 Sep 2026 17:21:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184081; bh=8CLHTuJS7ELes2g+pw+lNlc50IVNOCNPJpBf62awJw4=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=IE8V6HZ1Vlnlpz2G4J0biaxdgJoLdg4NsnhLdbXj5tHZh+IJeDqxcjW8wXGTX8X80 jBkUOR6ICRpOVSMNWM5SnMTvfX2AvNRqbStzC4+CQzPk7SJwfzNF2l/AhmfToXRqQY NdEoL/9OTxCpLSykaITw/NcR65Lahi4cYGzN9+Gu/DNKqjAyCgH7kbkTRZO4cQzWip mr1Hb6BAKtqFMigpCRUo3nPp3DOHr0dzUKy3MO0JM3a+LwNiYDDN9lo14IBEiEJitG awbsoiuf54/jK1rWLpP/ZTrPuxvPQu4HaAuTfacunUAAYa/d0U2mor7yx5QehewkKi gHIPZMXbY78Rw== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:03 +0100 Subject: [PATCH v4 14/22] modpost: emit module descriptors as assembly Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-14-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=31410; i=ljs@kernel.org; h=from:subject:message-id; bh=C7ggmJFDj4rQBqCHgur2cGTYGiZhCXldoiWa5ncWBaU=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF75NDuIJWX+bo/If2c37Cw32ipjI7w8pr7VZAl/z Nol5Q8/dpSyMIhxMciKKbI8/yK+P0gkbF7nBX83mDmsTCBDGLg4BWAigesZ/oczOxjYWsSrb735 6/u/UFHZim0iH76oSW9tk1//eZ2cXgjDP7PyLkP2xeq9y3WD7mxonq7zQ9CcMzwz4Pi2h6GqG+T PcgIA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 modpost generates a descriptor for every module in the form of a .mod.c file with .modinfo strings, the __this_module descriptor, exported symbol tables and (with CONFIG_MODVERSIONS set), the CRC of imported symbols. These files are compiled like any other kernel C file with all of the -include preamble, as well as including linux/module.h, header dependencies generated by fixdep of a few hundred headers, an objtool run and if LTO is being performed, a link is performed to generate native code. On an x86-64 allmodconfig build 11,189 *.mod.c files are built, each taking ~0.24s of CPU time to compile, and module finalisation as a whole 6,300 CPU seconds, or 64 seconds of wall time when run over 128 threads. It also generates ~1.3 GiB of *.mod.o.cmd files that every subsequent build has to read back. Avoid all this by emitting the descriptors as assembly instead. The layout required (size and alignment of struct module, struct modversion_info, the module's name offsets, init, and exit fields and whether the architecture uses PREL32 ksymtab references) can all be derived from scripts/mod/module-offsets.h. The fields of __this_module are emitted in offset order rather than declaration order, as CONFIG_RANDSTRUCT shuffles struct module so the name, init and exit fields can land anywhere. An assembly file avoids all of the issues previously mentioned so this conversion results in a very significant performance win on kernel build. As a consequence of this change, since module-offsets.c includes linux/module.h, scripts/mod is now built after the generated headers in prepare0, rather than before. Also update .gitignore and make clean to handle .mod.S files, but keep .mod.c files there to ensure that users do not end up with untracked changes/dirty trees after the change takes effect. The sections were confirmed to be byte-for-byte identical to the C version produced - each of .modinfo, .gnu.linkonce.this_module, __ksymtab*, __ksymtab_strings, __kcrctab*, __kflagstab*, __versions, __version_ext_crcs, __version_ext_names and their relocations - for all 8,135 modules of a clang allmodconfig build with CONFIG_COMPILE_TEST off and CONFIG_MODVERSIONS, CONFIG_EXTENDED_MODVERSIONS and CONFIG_MODULE_SRCVERSION_ALL on, and for a sample built with gcc. What differs is what the compiler added around them: the __UNIQUE_ID_* locals, the KASAN constructor for the .mod.c globals, and on x86 a .note.gnu.property that the linker already drops from any module containing an assembly file and the loader never reads. None of these have any impact on the build, however. The assembly carries no debug information, so in order to keep describing __this_module in each module's debug information, define it weakly in module-common.c too, in a section which the module link discards. x86_64 kernels built with gcc and clang, with CONFIG_MODVERSIONS, CONFIG_EXTENDED_MODVERSIONS and CONFIG_MODULE_SRCVERSION_ALL, were booted, every module and an external one loaded and unloaded, and the srcversions checked against modinfo, i386 and arm64 defconfigs (gcc and clang) and a ThinLTO build were also built as part of testing. An x86_64 CONFIG_RANDSTRUCT_FULL build with clang, which places the name field after init and exit, was checked the same way and produces the same .gnu.linkonce.this_module sections and relocations as the C version. On the x86_64 allmodconfig with clang 22, "make modules" with every *.mod.o and *.ko deleted goes from 64.5s (6,306 CPU-s) to 28.5s (518 CPU-s). A consequence of this change is that the make jobs are now so small that make cannot dispatch them quick enough, however the next commit in the series addresses this issue. This impacts clean and no-op builds with a large number of modules most noticeably. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 allmodconfig, no-op make, gcc 12.7s 2.3s -10.4s (-82%) x86 allmodconfig, no-op make, clang 13.9s 2.8s -11.2s (-80%) x86 allmodconfig, clean, gcc 342.6s 306.6s -36.0s (-11%) x86 allmodconfig, clean, clang 340.1s 290.2s -49.8s (-15%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- .gitignore | 1 + Makefile | 4 +- include/linux/vermagic.h | 2 +- scripts/Makefile.modfinal | 10 +- scripts/Makefile.modpost | 2 +- scripts/mod/.gitignore | 1 + scripts/mod/Makefile | 8 + scripts/mod/modpost.c | 606 +++++++++++++++++++++++++++++++--------= ---- scripts/mod/module-offsets.c | 35 +++ scripts/module-common.c | 3 + scripts/tags.sh | 5 +- 11 files changed, 502 insertions(+), 175 deletions(-) diff --git a/.gitignore b/.gitignore index 9875120ea7bd..00fc262b894b 100644 --- a/.gitignore +++ b/.gitignore @@ -38,6 +38,7 @@ *.lzo *.mod *.mod.c +*.mod.S *.o *.o.* *.patch diff --git a/Makefile b/Makefile index cbfee8d325a5..7745bb36e622 100644 --- a/Makefile +++ b/Makefile @@ -1426,8 +1426,8 @@ archprepare: outputmakefile archheaders archscripts s= cripts include/config/kerne include/generated/rustc_cfg remove-stale-files =20 prepare0: archprepare - $(Q)$(MAKE) $(build)=3Dscripts/mod $(Q)$(MAKE) $(build)=3D. prepare + $(Q)$(MAKE) $(build)=3Dscripts/mod =20 # All the preparing.. prepare: prepare0 @@ -2249,7 +2249,7 @@ clean: $(clean-dirs) -o -name '*.dt.yaml' -o -name 'dtbs-list' \ -o -name '*.dwo' -o -name '*.lst' \ -o -name '*.su' -o -name '*.mod' \ - -o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \ + -o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' -o -name '*.mod.S' \ -o -name '*.lex.c' -o -name '*.tab.[ch]' \ -o -name '*.asn1.[ch]' \ -o -name '*.symtypes' -o -name 'modules.order' \ diff --git a/include/linux/vermagic.h b/include/linux/vermagic.h index 335c360d4f9b..09f05d02664c 100644 --- a/include/linux/vermagic.h +++ b/include/linux/vermagic.h @@ -3,7 +3,7 @@ #define _LINUX_VERMAGIC_H =20 #ifndef INCLUDE_VERMAGIC -#error "This header can be included from kernel/module.c or *.mod.c only" +#error "This header can be included from kernel/module.c or scripts/module= -common.c only" #endif =20 #include diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 01a37ec872b9..75e9effdf02c 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -20,10 +20,14 @@ __modfinal: $(modules:%.o=3D%.ko) modname =3D $(notdir $(@:.mod.o=3D)) part-of-module =3D y GCOV_PROFILE :=3D n -ccflags-remove-y :=3D $(CC_FLAGS_CFI) =20 -%.mod.o: %.mod.c FORCE - $(call if_changed_rule,cc_o_c) +# modpost lays the .mod.S out completely (write_mod_S_file()), so = it +# needs only the assembler and no dependency tracking. +quiet_cmd_as_mod_o =3D AS [M] $@ + cmd_as_mod_o =3D $(CC) $(_a_flags) $(modkern_aflags) -c -o $@ $< + +%.mod.o: %.mod.S FORCE + $(call if_changed,as_mod_o) =20 .module-common.o: $(srctree)/scripts/module-common.c FORCE $(call if_changed_rule,cc_o_c) diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost index d7d45067d08b..eecf5f5c99b4 100644 --- a/scripts/Makefile.modpost +++ b/scripts/Makefile.modpost @@ -13,7 +13,7 @@ # Stage 2 is handled by this file and does the following # 1) Find all modules listed in modules.order # 2) modpost is then used to -# 3) create one .mod.c file per module +# 3) create one .mod.S file per module # 4) create one Module.symvers file with CRC for all exported symbols =20 # Step 3 is used to place certain information in the module's ELF diff --git a/scripts/mod/.gitignore b/scripts/mod/.gitignore index 0465ec33c9bf..620ab4362094 100644 --- a/scripts/mod/.gitignore +++ b/scripts/mod/.gitignore @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only /devicetable-offsets.h +/module-offsets.h /elfconfig.h /mk_elfconfig /modpost diff --git a/scripts/mod/Makefile b/scripts/mod/Makefile index c729bc936bae..fbd5099e0441 100644 --- a/scripts/mod/Makefile +++ b/scripts/mod/Makefile @@ -13,10 +13,18 @@ $(obj)/$(devicetable-offsets-file): $(obj)/devicetable-= offsets.s FORCE =20 targets +=3D $(devicetable-offsets-file) devicetable-offsets.s =20 +module-offsets-file :=3D module-offsets.h + +$(obj)/$(module-offsets-file): $(obj)/module-offsets.s FORCE + $(call filechk,offsets,__MODULE_OFFSETS_H__) + +targets +=3D $(module-offsets-file) module-offsets.s + # dependencies on generated files need to be listed explicitly =20 $(obj)/modpost.o $(obj)/file2alias.o $(obj)/sumversion.o $(obj)/symsearch.= o: $(obj)/elfconfig.h $(obj)/file2alias.o: $(obj)/$(devicetable-offsets-file) +$(obj)/modpost.o: $(obj)/$(module-offsets-file) =20 quiet_cmd_elfconfig =3D MKELF $@ cmd_elfconfig =3D $(obj)/mk_elfconfig < $< > $@ diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c index 0fd43c8a89ea..9d96459e492c 100644 --- a/scripts/mod/modpost.c +++ b/scripts/mod/modpost.c @@ -26,6 +26,7 @@ #include #include #include "modpost.h" +#include "module-offsets.h" #include "../../include/linux/license.h" =20 #define MODULE_NS_PREFIX "module:" @@ -1920,39 +1921,6 @@ static void check_modname_len(struct module *mod) mod_error(mod, "module name is too long\n"); } =20 -/** - * Header for the generated file - **/ -static void add_header(struct buffer *b, struct module *mod) -{ - buf_printf(b, "#include \n"); - buf_printf(b, "#include \n"); - buf_printf(b, "#include \n"); - buf_printf(b, "\n"); - buf_printf(b, "MODULE_INFO(name, KBUILD_MODNAME);\n"); - buf_printf(b, "\n"); - buf_printf(b, "__visible struct module __this_module\n"); - buf_printf(b, "__section(\".gnu.linkonce.this_module\") =3D {\n"); - buf_printf(b, "\t.name =3D KBUILD_MODNAME,\n"); - if (mod->has_init) - buf_printf(b, "\t.init =3D init_module,\n"); - if (mod->has_cleanup) - buf_printf(b, "#ifdef CONFIG_MODULE_UNLOAD\n" - "\t.exit =3D cleanup_module,\n" - "#endif\n"); - buf_printf(b, "\t.arch =3D MODULE_ARCH_INIT,\n"); - buf_printf(b, "};\n"); - - if (!external_module) - buf_printf(b, "\nMODULE_INFO(intree, \"Y\");\n"); - - if (strstarts(mod->name, "drivers/staging")) - buf_printf(b, "\nMODULE_INFO(staging, \"Y\");\n"); - - if (strstarts(mod->name, "tools/testing")) - buf_printf(b, "\nMODULE_INFO(test, \"Y\");\n"); -} - static void add_exported_symbols(struct buffer *buf, struct module *mod) { struct symbol *sym; @@ -1990,123 +1958,6 @@ static void add_exported_symbols(struct buffer *buf= , struct module *mod) } } =20 -/** - * Record CRCs for unresolved symbols, supporting long names - */ -static void add_extended_versions(struct buffer *b, struct module *mod) -{ - struct symbol *s; - - if (!extended_modversions) - return; - - buf_printf(b, "\n"); - buf_printf(b, "static const u32 ____version_ext_crcs[]\n"); - buf_printf(b, "__used __section(\"__version_ext_crcs\") =3D {\n"); - list_for_each_entry(s, &mod->unresolved_symbols, list) { - if (!s->module) - continue; - if (!s->crc_valid) { - mod_warn(mod, "symbol '%s' has no CRC!\n", s->name); - continue; - } - buf_printf(b, "\t0x%08x,\n", s->crc); - } - buf_printf(b, "};\n"); - - buf_printf(b, "static const char ____version_ext_names[]\n"); - buf_printf(b, "__used __section(\"__version_ext_names\") =3D\n"); - list_for_each_entry(s, &mod->unresolved_symbols, list) { - if (!s->module) - continue; - if (!s->crc_valid) - /* - * We already warned on this when producing the crc - * table. - * We need to skip its name too, as the indexes in - * both tables need to align. - */ - continue; - buf_printf(b, "\t\"%s\\0\"\n", s->name); - } - buf_printf(b, ";\n"); -} - -/** - * Record CRCs for unresolved symbols - **/ -static void add_versions(struct buffer *b, struct module *mod) -{ - struct symbol *s; - - if (!basic_modversions) - return; - - buf_printf(b, "\n"); - buf_printf(b, "static const struct modversion_info ____versions[]\n"); - buf_printf(b, "__used __section(\"__versions\") =3D {\n"); - - list_for_each_entry(s, &mod->unresolved_symbols, list) { - if (!s->module) - continue; - if (!s->crc_valid) { - mod_warn(mod, "symbol '%s' has no CRC!\n", s->name); - continue; - } - if (strlen(s->name) >=3D MODULE_NAME_LEN) { - if (extended_modversions) { - /* this symbol will only be in the extended info */ - continue; - } else { - mod_error(mod, "too long symbol '%s'\n", s->name); - break; - } - } - buf_printf(b, "\t{ 0x%08x, \"%s\" },\n", - s->crc, s->name); - } - - buf_printf(b, "};\n"); -} - -static void add_depends(struct buffer *b, struct module *mod) -{ - struct symbol *s; - int first =3D 1; - - /* Clear ->seen flag of modules that own symbols needed by this. */ - list_for_each_entry(s, &mod->unresolved_symbols, list) { - if (s->module) - s->module->seen =3D s->module->is_vmlinux; - } - - buf_printf(b, "\n"); - buf_printf(b, "MODULE_INFO(depends, \""); - list_for_each_entry(s, &mod->unresolved_symbols, list) { - const char *p; - if (!s->module) - continue; - - if (s->module->seen) - continue; - - s->module->seen =3D true; - p =3D get_basename(s->module->name); - buf_printf(b, "%s%s", first ? "" : ",", p); - first =3D 0; - } - buf_printf(b, "\");\n"); -} - -static void add_srcversion(struct buffer *b, struct module *mod) -{ - if (mod->srcversion[0]) { - buf_printf(b, "\n"); - buf_printf(b, "MODULE_INFO(srcversion, \"%s\");\n", - mod->srcversion); - } -} - static void write_buf(struct buffer *b, const char *fname) { FILE *file; @@ -2191,30 +2042,452 @@ static void write_vmlinux_export_c_file(struct mod= ule *mod) free(buf.p); } =20 -/* do sanity checks, and generate *.mod.c file */ -static void write_mod_c_file(struct module *mod) +#if MOD_SIZEOF_LONG =3D=3D 8 +#define MOD_PTR_DIRECTIVE ".quad" +#else +#define MOD_PTR_DIRECTIVE ".long" +#endif + +/* See KSYM_FUNC() in include/linux/export-internal.h. */ +#if MOD_FUNC_PLABEL +#define MOD_FUNC_PREFIX "P%" +#else +#define MOD_FUNC_PREFIX "" +#endif + +/* See __KSYM_ALIGN in include/linux/export-internal.h. */ +#if MOD_PREL32_RELOCATIONS || MOD_SIZEOF_LONG =3D=3D 4 +#define KSYM_ALIGN 4 +#else +#define KSYM_ALIGN 8 +#endif + +/* Append the body of an assembler string literal, escaped as needed. */ +static void buf_escaped(struct buffer *buf, const char *str) { - struct buffer buf =3D { }; - struct module_alias *alias, *next; - char fname[PATH_MAX]; - int ret; + unsigned char chr; =20 - add_header(&buf, mod); - add_exported_symbols(&buf, mod); - add_versions(&buf, mod); - add_extended_versions(&buf, mod); - add_depends(&buf, mod); + while ((chr =3D *str++)) { + if (chr =3D=3D '"' || chr =3D=3D '\\') + buf_printf(buf, "\\%c", chr); + else if (isprint(chr)) + buf_printf(buf, "%c", chr); + else + buf_printf(buf, "\\%03o", chr); + } +} =20 - buf_printf(&buf, "\n"); +static void buf_asciz(struct buffer *buf, const char *str) +{ + buf_printf(buf, "\t.asciz \""); + buf_escaped(buf, str); + buf_printf(buf, "\"\n"); +} + +/* The equivalent of MODULE_INFO(tag, info). */ +static void add_asm_modinfo(struct buffer *buf, const char *tag, + const char *info) +{ + buf_printf(buf, "\t.section .modinfo,\"a\",%%progbits\n"); + buf_printf(buf, "\t.asciz \"%s=3D", tag); + buf_escaped(buf, info); + buf_printf(buf, "\"\n"); +} + +/* See __KSYM_REF() in include/linux/export-internal.h. */ +static void add_asm_ksym_ref(struct buffer *buf, const char *prefix, + const char *sym) +{ +#if MOD_PREL32_RELOCATIONS + buf_printf(buf, "\t.long %s%s - .\n", prefix, sym); +#else + buf_printf(buf, "\t" MOD_PTR_DIRECTIVE " %s%s\n", prefix, sym); +#endif +} + +/* The name and namespace strings a ksymtab entry refers to. */ +static void add_asm_kstrtab(struct buffer *buf, const struct symbol *sym) +{ + buf_printf(buf, "\t.section \"__ksymtab_strings\",\"aMS\",%%progbits,1\n"= ); + buf_printf(buf, "__kstrtab_%s:\n", sym->name); + buf_asciz(buf, sym->name); + buf_printf(buf, "__kstrtabns_%s:\n", sym->name); + buf_asciz(buf, sym->namespace); + buf_printf(buf, "\t.previous\n"); +} + +/* The equivalent of SYMBOL_FLAGS(). */ +static void add_asm_kflagstab(struct buffer *buf, const struct symbol *sym) +{ + buf_printf(buf, "\t.section \"___kflagstab+%s\", \"a\"\n", sym->name); + buf_printf(buf, "__flags_%s:\n", sym->name); + buf_printf(buf, "\t.byte 0x%02x\n", get_symbol_flags(sym)); + buf_printf(buf, "\t.previous\n"); +} + +/* The equivalent of KSYMTAB_FUNC()/KSYMTAB_DATA(). */ +static void add_asm_ksymtab(struct buffer *buf, const struct symbol *sym) +{ + const char *name =3D sym->name; + + add_asm_kstrtab(buf, sym); + + buf_printf(buf, "\t.section \"___ksymtab+%s\", \"a\"\n", name); + buf_printf(buf, "\t.balign %d\n", KSYM_ALIGN); + buf_printf(buf, "__ksymtab_%s:\n", name); + add_asm_ksym_ref(buf, sym->is_func ? MOD_FUNC_PREFIX : "", name); + add_asm_ksym_ref(buf, "__kstrtab_", name); + add_asm_ksym_ref(buf, "__kstrtabns_", name); + buf_printf(buf, "\t.previous\n"); + + add_asm_kflagstab(buf, sym); +} + +/* The equivalent of SYMBOL_CRC(). */ +static void add_asm_crc(struct buffer *buf, const struct symbol *sym) +{ + buf_printf(buf, "\t.section \"___kcrctab+%s\",\"a\"\n", sym->name); + buf_printf(buf, "\t.balign 4\n"); + buf_printf(buf, "__crc_%s:\n", sym->name); + buf_printf(buf, "\t.long 0x%08x\n", sym->crc); + buf_printf(buf, "\t.previous\n"); +} + +static bool export_is_kept(const struct symbol *sym) +{ + return !trim_unused_exports || sym->used; +} + +/* The exported symbols and, with modversions, their CRCs. */ +static void add_asm_exported_symbols(struct buffer *buf, struct module *mo= d) +{ + struct symbol *sym; + + list_for_each_entry(sym, &mod->exported_symbols, list) { + if (!export_is_kept(sym)) + continue; + + add_asm_ksymtab(buf, sym); + + if (!modversions) + continue; + + if (!sym->crc_valid) + mod_warn(mod, "EXPORT symbol '%s' version generation failed, symbol wil= l not be versioned.\n" + "Is '%s' prototyped in ?\n", + sym->name, sym->name); + add_asm_crc(buf, sym); + } +} + +/* Zero fill up to the offset. */ +static void asm_skip_to(struct buffer *buf, unsigned int *pos, + unsigned int offset) +{ + if (offset < *pos) + fatal("__this_module field at offset %u overlaps the previous one\n", + offset); + if (offset > *pos) + buf_printf(buf, "\t.skip %u\n", offset - *pos); + + *pos =3D offset; +} + +/* A non-zero field of __this_module: its offset and the name or symbol in= it. */ +struct this_module_field { + unsigned int offset; + bool is_name; + const char *value; +}; + +#define THIS_MODULE_MAX_FIELDS 5 + +static int compare_field_offsets(const void *ptr_a, const void *ptr_b) +{ + const struct this_module_field *field_a =3D ptr_a, *field_b =3D ptr_b; + + if (field_a->offset !=3D field_b->offset) + return field_a->offset < field_b->offset ? -1 : 1; + + return 0; +} + +/* + * The fields of __this_module which are not zero, in offset order. The or= der + * is only known from module-offsets.h, CONFIG_RANDSTRUCT shuffles struct = module. + */ +static unsigned int get_this_module_fields(const struct module *mod, + const char *modname, + struct this_module_field *fields) +{ + unsigned int nr_fields =3D 0; + + fields[nr_fields++] =3D (struct this_module_field) + { MOD_OFF_module_name, true, modname }; + if (mod->has_init) + fields[nr_fields++] =3D (struct this_module_field) + { MOD_OFF_module_init, false, MOD_FUNC_PREFIX "init_module" }; +#ifdef MOD_OFF_module_exit + if (mod->has_cleanup) + fields[nr_fields++] =3D (struct this_module_field) + { MOD_OFF_module_exit, false, MOD_FUNC_PREFIX "cleanup_module" }; +#endif +#ifdef MOD_OFF_module_arch_fixup_start + fields[nr_fields++] =3D (struct this_module_field) + { MOD_OFF_module_arch_fixup_start, false, "__start_fixup" }; + fields[nr_fields++] =3D (struct this_module_field) + { MOD_OFF_module_arch_fixup_end, false, "__stop_fixup" }; +#endif + qsort(fields, nr_fields, sizeof(*fields), compare_field_offsets); + + return nr_fields; +} + +/* Emit one field of __this_module, returning its size. */ +static unsigned int add_asm_this_module_field(struct buffer *buf, + const struct this_module_field *field) +{ + if (field->is_name) { + buf_printf(buf, "\t.ascii \"%s\"\n", field->value); + return strlen(field->value); + } + + buf_printf(buf, "\t" MOD_PTR_DIRECTIVE " %s\n", field->value); + return MOD_SIZEOF_LONG; +} + +/* + * The equivalent of: + * + * __visible struct module __this_module __section(".gnu.linkonce.this_mod= ule") + * =3D { .name =3D KBUILD_MODNAME, .init =3D init_module, .exit =3D clean= up_module, + * .arch =3D MODULE_ARCH_INIT }; + * + * Everything not listed is zero, MODULE_ARCH_INIT included, except on m68= k. + */ +static void add_asm_this_module(struct buffer *buf, const struct module *m= od, + const char *modname) +{ + struct this_module_field fields[THIS_MODULE_MAX_FIELDS]; + const unsigned int nr_fields =3D get_this_module_fields(mod, modname, fie= lds); + unsigned int pos =3D 0, i; + + buf_printf(buf, "\n\t.section .gnu.linkonce.this_module,\"aw\",%%progbits= \n"); + buf_printf(buf, "\t.balign %d\n", MOD_ALIGNOF_struct_module); + buf_printf(buf, "\t.globl __this_module\n"); + buf_printf(buf, "\t.type __this_module, %%object\n"); + buf_printf(buf, "\t.size __this_module, %d\n", MOD_SIZEOF_struct_module); + buf_printf(buf, "__this_module:\n"); + + for (i =3D 0; i < nr_fields; i++) { + asm_skip_to(buf, &pos, fields[i].offset); + pos +=3D add_asm_this_module_field(buf, &fields[i]); + } + + asm_skip_to(buf, &pos, MOD_SIZEOF_struct_module); +} + +/* + * An unresolved symbol without a module is not versioned; one without a C= RC + * cannot be, so warn about it. + */ +static bool skip_unversioned(struct module *mod, const struct symbol *sym) +{ + if (!sym->module) + return true; + if (sym->crc_valid) + return false; + + mod_warn(mod, "symbol '%s' has no CRC!\n", sym->name); + return true; +} + +/* One struct modversion_info: the CRC, then the name padded to the end. */ +static void add_asm_version(struct buffer *buf, const struct symbol *sym) +{ + buf_printf(buf, "\t" MOD_PTR_DIRECTIVE " 0x%08x\n", sym->crc); + buf_printf(buf, "\t.ascii \"%s\"\n", sym->name); + buf_printf(buf, "\t.skip %zu\n", MOD_SIZEOF_struct_modversion_info - + MOD_OFF_modversion_info_name - strlen(sym->name)); +} + +/* + * The equivalent of: + * + * static const struct modversion_info ____versions[] + * __used __section("__versions") =3D { { crc, "name" }, ... }; + * + * for unresolved symbols. + */ +static void add_asm_versions(struct buffer *buf, struct module *mod) +{ + struct symbol *sym; + + if (!basic_modversions) + return; + + buf_printf(buf, "\n\t.section __versions,\"a\",%%progbits\n"); + buf_printf(buf, "\t.balign %d\n", MOD_ALIGNOF_struct_modversion_info); + list_for_each_entry(sym, &mod->unresolved_symbols, list) { + if (skip_unversioned(mod, sym)) + continue; + + if (strlen(sym->name) >=3D MOD_NAME_LEN) { + /* Only the extended table can hold it. */ + if (extended_modversions) + continue; + + mod_error(mod, "too long symbol '%s'\n", sym->name); + break; + } + + add_asm_version(buf, sym); + } +} + +/* + * The equivalent of: + * static const u32 ____version_ext_crcs[] __section("__version_ext_crcs")= =3D { crc, ... }; + * static const char ____version_ext_names[] __section("__version_ext_name= s") =3D "name\0" ...; + * + * for unresolved symbols. + */ +static void add_asm_extended_versions(struct buffer *buf, struct module *m= od) +{ + struct symbol *sym; + + if (!extended_modversions) + return; + + buf_printf(buf, "\n\t.section __version_ext_crcs,\"a\",%%progbits\n"); + buf_printf(buf, "\t.balign 4\n"); + list_for_each_entry(sym, &mod->unresolved_symbols, list) { + if (skip_unversioned(mod, sym)) + continue; + + buf_printf(buf, "\t.long 0x%08x\n", sym->crc); + } + + /* + * A symbol without a CRC was warned about above, and is skipped here + * too so that the names line up with the CRCs. + */ + buf_printf(buf, "\t.section __version_ext_names,\"a\",%%progbits\n"); + list_for_each_entry(sym, &mod->unresolved_symbols, list) { + if (!sym->module || !sym->crc_valid) + continue; + + buf_asciz(buf, sym->name); + } + /* The terminator of the string literal this used to be. */ + buf_printf(buf, "\t.byte 0\n"); +} + +/* MODULE_INFO(depends, ...): the modules this one needs, each once. */ +static void add_asm_depends(struct buffer *buf, struct module *mod) +{ + struct buffer deps =3D { }; + struct symbol *sym; + bool first =3D true; + + /* Clear ->seen of the modules that own symbols this one needs. */ + list_for_each_entry(sym, &mod->unresolved_symbols, list) { + if (sym->module) + sym->module->seen =3D sym->module->is_vmlinux; + } + + list_for_each_entry(sym, &mod->unresolved_symbols, list) { + struct module *owner =3D sym->module; + + if (!owner || owner->seen) + continue; + + owner->seen =3D true; + buf_printf(&deps, "%s%s", first ? "" : ",", + get_basename(owner->name)); + first =3D false; + } + buf_write(&deps, "", 1); + + buf_printf(buf, "\n"); + add_asm_modinfo(buf, "depends", deps.p); + free(deps.p); +} + +/* + * KBUILD_MODNAME: the basename of the module with '-' and ',' replaced by + * '_' (see name-fix in scripts/Makefile.lib). + */ +static char *get_kbuild_modname(const struct module *mod) +{ + char *name =3D xstrdup(get_basename(mod->name)); + char *curr; + + for (curr =3D name; *curr; curr++) { + if (*curr =3D=3D '-' || *curr =3D=3D ',') + *curr =3D '_'; + } + + return name; +} + +/* The module's name, its descriptor, and where it comes from. */ +static void add_asm_header(struct buffer *buf, const struct module *mod, + const char *modname) +{ + buf_printf(buf, "/* Generated by modpost, see scripts/Makefile.modfinal *= /\n\n"); + + add_asm_modinfo(buf, "name", modname); + add_asm_this_module(buf, mod, modname); + buf_printf(buf, "\n"); + + if (!external_module) + add_asm_modinfo(buf, "intree", "Y"); + if (strstarts(mod->name, "drivers/staging")) + add_asm_modinfo(buf, "staging", "Y"); + if (strstarts(mod->name, "tools/testing")) + add_asm_modinfo(buf, "test", "Y"); +} + +static void add_asm_aliases(struct buffer *buf, struct module *mod) +{ + struct module_alias *alias, *next; + + buf_printf(buf, "\n"); list_for_each_entry_safe(alias, next, &mod->aliases, node) { - buf_printf(&buf, "MODULE_ALIAS(\"%s\");\n", alias->str); + add_asm_modinfo(buf, "alias", alias->str); list_del(&alias->node); free(alias); } +} + +static void add_asm_srcversion(struct buffer *buf, const struct module *mo= d) +{ + if (!mod->srcversion[0]) + return; =20 - add_srcversion(&buf, mod); + buf_printf(buf, "\n"); + add_asm_modinfo(buf, "srcversion", mod->srcversion); +} + +static void write_mod_S_file(struct module *mod) +{ + struct buffer buf =3D { }; + char fname[PATH_MAX]; + char *modname =3D get_kbuild_modname(mod); + int ret; =20 - ret =3D snprintf(fname, sizeof(fname), "%s.mod.c", mod->name); + add_asm_header(&buf, mod, modname); + buf_printf(&buf, "\n"); + add_asm_exported_symbols(&buf, mod); + add_asm_versions(&buf, mod); + add_asm_extended_versions(&buf, mod); + add_asm_depends(&buf, mod); + add_asm_aliases(&buf, mod); + add_asm_srcversion(&buf, mod); + buf_printf(&buf, "\n\t.section .note.GNU-stack,\"\",%%progbits\n"); + + ret =3D snprintf(fname, sizeof(fname), "%s.mod.S", mod->name); if (ret >=3D sizeof(fname)) { error("%s: too long path was truncated\n", fname); goto free; @@ -2223,6 +2496,7 @@ static void write_mod_c_file(struct module *mod) write_if_changed(&buf, fname); =20 free: + free(modname); free(buf.p); } =20 @@ -2462,7 +2736,7 @@ int main(int argc, char **argv) if (mod->is_vmlinux) write_vmlinux_export_c_file(mod); else - write_mod_c_file(mod); + write_mod_S_file(mod); } =20 if (missing_namespace_deps) diff --git a/scripts/mod/module-offsets.c b/scripts/mod/module-offsets.c new file mode 100644 index 000000000000..a336dd47aa33 --- /dev/null +++ b/scripts/mod/module-offsets.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Layout of the structures modpost emits into *.mod.S, extracted from the + * target headers as devicetable-offsets.c does for the device tables. + */ +#define COMPILE_OFFSETS +#include +#include + +int main(void) +{ + DEFINE(MOD_SIZEOF_LONG, sizeof(long)); + DEFINE(MOD_PREL32_RELOCATIONS, IS_ENABLED(CONFIG_HAVE_ARCH_PREL32_RELOCAT= IONS)); + DEFINE(MOD_FUNC_PLABEL, IS_ENABLED(CONFIG_PARISC) && IS_ENABLED(CONFIG_64= BIT)); + + DEFINE(MOD_SIZEOF_struct_module, sizeof(struct module)); + DEFINE(MOD_ALIGNOF_struct_module, __alignof__(struct module)); + OFFSET(MOD_OFF_module_name, module, name); + OFFSET(MOD_OFF_module_init, module, init); +#ifdef CONFIG_MODULE_UNLOAD + OFFSET(MOD_OFF_module_exit, module, exit); +#endif +#if defined(CONFIG_M68K) && defined(CONFIG_MMU) + /* MODULE_ARCH_INIT: the only architecture where it is not all zeroes. */ + OFFSET(MOD_OFF_module_arch_fixup_start, module, arch.fixup_start); + OFFSET(MOD_OFF_module_arch_fixup_end, module, arch.fixup_end); +#endif + DEFINE(MOD_NAME_LEN, MODULE_NAME_LEN); + + DEFINE(MOD_SIZEOF_struct_modversion_info, sizeof(struct modversion_info)); + DEFINE(MOD_ALIGNOF_struct_modversion_info, __alignof__(struct modversion_= info)); + OFFSET(MOD_OFF_modversion_info_name, modversion_info, name); + + return 0; +} diff --git a/scripts/module-common.c b/scripts/module-common.c index 12fbc6d3aae8..a75533e709fb 100644 --- a/scripts/module-common.c +++ b/scripts/module-common.c @@ -20,6 +20,9 @@ BUILD_LTO_INFO; =20 MODULE_INFO(vermagic, VERMAGIC_STRING); =20 +/* modpost generates assembly so provide for debug info. */ +__weak __section(".discard.this_module") struct module __this_module; + #ifdef CONFIG_MITIGATION_RETPOLINE MODULE_INFO(retpoline, "Y"); #endif diff --git a/scripts/tags.sh b/scripts/tags.sh index 41e38df96984..c33d0f58a9d4 100755 --- a/scripts/tags.sh +++ b/scripts/tags.sh @@ -14,8 +14,9 @@ fi =20 # RCS_FIND_IGNORE has escaped ()s -- remove them. ignore=3D"$(echo "$RCS_FIND_IGNORE" | sed 's|\\||g' )" -# tags and cscope files should also ignore MODVERSION *.mod.c files -ignore=3D"$ignore ( -name *.mod.c ) -prune -o" +# tags and cscope files should also ignore the modpost-generated *.mod.S f= iles +# and any *.mod.c left behind from before they were assembly +ignore=3D"$ignore ( -name *.mod.c -o -name *.mod.S ) -prune -o" =20 # ignore arbitrary directories if [ -n "${IGNORE_DIRS}" ]; then --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7125E3E6DFA; Wed, 23 Sep 2026 17:21:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184098; cv=none; b=eDF+7GRYR5Zl4h326nlYQ7l+sG0AVg7du+qBo2TKtQnSdIW5KEzwx08bi3IBVqjys/3Xj+MZQaBEisLS/9Rgc81xIwUX7SeW0ehQYVIcn9StapU0WrRQcRbBWBnrGfTwz7ffi4oTI60TYP4UldF6czhQ2EknZL0s9INKmMJan/Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184098; c=relaxed/simple; bh=kGsKHAXdvm6iK3LVpcjsDvzjOzyjcJz2gnviueErSc4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rLLb0ZeS0oUBFNWyYKryOLEi6qY8ljnIYiHj2t2bh6E8c34TJBwVo8WZw6+YfJR9UixF+mtA/B0unpiTQ2w1ZaLayI/tJe4aT7oF0VBM8/3aK9SKP1zTzIQ1EtPzAuQ8+RuqHCalyPhkUWBRez2y3lOOfefVpTB3M381sC3lNIc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mQCfJEkZ; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mQCfJEkZ" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EF0E41F000FF; Wed, 23 Sep 2026 17:21:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184093; bh=4w9neOqf7b6EaJ3ezfUP/vruXEKR7drJS7P+n8rmU2E=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=mQCfJEkZyOMKxwB5YJvvYnLkPIJXowTIPKoC5XthNNSOptZQJ04fZPMc3gZPelYep Esnty9LF8i4P/P6sQraIteZSWSv00l6ZsVeKBAOyWs3vWf0hkiCWLUi4RfjGwx/cfu +rBHCyc2RHhE2AlsnEFitxLlF3cKNDjvDz/A8LFq3M/NMVF62a6ftgFrF0BIP43oO3 yteFzghZGukgliCVTczRm2i1DVvZGhC5pONqL8bjSMPcdjTW1Qb2znLdhD7hcEbo7w WI3DQ7k5yt9PZpzdiHIho1Ovf+pTuzMcpXEdTHkV9+hooDXBVfGkC/J0mSNEZX3jp1 sW5yvHiHli8vg== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:04 +0100 Subject: [PATCH v4 15/22] kbuild: batch module finalisation Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-15-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=5003; i=ljs@kernel.org; h=from:subject:message-id; bh=kGsKHAXdvm6iK3LVpcjsDvzjOzyjcJz2gnviueErSc4=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF6xKs2f3zorRfbGZxmJ9qJ/rN+sm87x8wR/Ttr2d eUC46MyHaUsDGJcDLJiiizPv4jvDxIJm9d5wd8NZg4rE8gQBi5OAZjIVyGGf9rz/vlkdzJ+3vVY r9eAcR6LnOgPtVW2vavmXVLNYf3zV4aRYevKBT+27tmscXqXwKmLOadXcFzT/xI9TfqUio5taet kc2YA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 With the .mod.S change in place, module finalisation on allmodconfig builds consists of a large number of very short-lived jobs. For allmodconfig x86-64 this can be on the order of ~22,000 jobs of a few milliseconds in duration each. Each job entails processing ~22k .cmd files, so the combination of heavy overhead and small individual job results in a lot of unnecessary and repeated work even with all cores being utilised. The solution is to batch by a number of jobs. Determining which value makes sense was done empirically. On a 128-thread threadripper box doing an allmodconfig build, best of 2, *.ko, *.mod.o deleted each time: modules per chunk instances wall ----------------- --------- ------ 1 11171 10.22s 2 5586 6.72s 4 2793 4.97s 8 1397 4.26s 16 699 4.05s 32 350 4.05s 64 175 4.07s 128 88 4.05s 256 44 4.03s 512 22 4.18s Wall time flattens for 16-256 module batches. A slower/lower core machine will do better with fewer modules-per-batch, a faster/higher core machine will do better with more modules-per-batch. Therefore, take the midpoint which works in the most margin in either direction - 128 modules per batch. This naturally scales with module count too as the optimum gains are obtained with higher module count, so fewer batches in this case costs less overhead. Each instance holds only its own modules' variables and the top-level one reads no per-module .cmd files at all, the same rules serve both levels, and an instance is told its chunk with modfinal-first=3D. Only the top-level instance builds .module-common.o, to the chunks it is a plain prerequisite, so no two instances ever write it. "make modules" with every *.mod.o and *.ko deleted goes from 28.9s to 15.9s with clang 22. No-op "make modules" goes from 5.6s to 4.8s, as checking the 22,000 targets is spread over the chunks too. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 allmodconfig, no-op make, gcc 2.3s 1.4s -0.85s (-38%) x86 allmodconfig, no-op make, clang 2.8s 1.8s -0.94s (-34%) x86 allmodconfig, clean, gcc 306.6s 294.1s -12.6s (-4%) x86 allmodconfig, clean, clang 290.2s 283.9s -6.4s (-2%) Assisted-by: LLM Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- scripts/Makefile.modfinal | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 75e9effdf02c..4d5e6e1a0ff9 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -13,9 +13,30 @@ include $(srctree)/scripts/Makefile.lib # find all modules listed in modules.order modules :=3D $(call read-file, modules.order) =20 +modfinal-chunk-size :=3D 128 + +ifdef modfinal-first + +# this instance handles the chunk of modules.order starting at $(modfinal-= first) +modules :=3D $(wordlist $(modfinal-first), $(words $(modules)), $(modules)) +modules :=3D $(wordlist 1, $(modfinal-chunk-size), $(modules)) + __modfinal: $(modules:%.o=3D%.ko) @: =20 +else + +modfinal-chunks :=3D $(addprefix chunk-, $(shell seq 1 $(modfinal-chunk-si= ze) $(words $(modules)))) + +PHONY +=3D $(modfinal-chunks) +$(modfinal-chunks): .module-common.o + $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modfinal modfinal-first=3D$(@:= chunk-%=3D%) + +__modfinal: $(modfinal-chunks) + @: + +endif + # modname and part-of-module are set to make c_flags define proper module = flags modname =3D $(notdir $(@:.mod.o=3D)) part-of-module =3D y @@ -29,8 +50,11 @@ quiet_cmd_as_mod_o =3D AS [M] $@ %.mod.o: %.mod.S FORCE $(call if_changed,as_mod_o) =20 +# Built by the top-level instance alone, the chunks take it as a plain fil= e. +ifndef modfinal-first .module-common.o: $(srctree)/scripts/module-common.c FORCE $(call if_changed_rule,cc_o_c) +endif =20 ifneq ($(WARN_ON_UNUSED_TRACEPOINTS),) cmd_check_tracepoint =3D $(objtree)/scripts/tracepoint-update --module $<; @@ -58,7 +82,11 @@ ifdef CONFIG_DEBUG_INFO_BTF_MODULES endif +$(call cmd,check_tracepoint) =20 -targets +=3D $(modules:%.o=3D%.ko) $(modules:%.o=3D%.mod.o) .module-common= .o +ifdef modfinal-first +targets +=3D $(modules:%.o=3D%.ko) $(modules:%.o=3D%.mod.o) +else +targets +=3D .module-common.o +endif =20 # Add FORCE to the prerequisites of a target to force it to be always rebu= ilt. # ------------------------------------------------------------------------= --- --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBCB93E6DD2; Wed, 23 Sep 2026 17:21:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184109; cv=none; b=sDThrKxYEehXNoGUXIZ+q8FS9t4nnjw9lDX3wMsV6m7/GGRW56oJFAJnZNUgGzFAbfh/jaqaTER0WbGgyU0mYi98z6G4/TXcykWuf602xFxuOjUdKni9XnxVMA6v3XPZlt9WPwsKYdk+xcmBOz5OLjUqsenaWWEY3KO4PfNyeHc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184109; c=relaxed/simple; bh=JsrcNLsDFz81n4CrtQtvecqRzIoQyaHRODeEltcBuYs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ophcdC2fwTTk1vxZyOLduqna+56biJkhAC1o8KOM/V7Q3e2wjcj4R9SMITzR5d/W/ss1zry0yfb06j6KgXyPdqzUIMqVWi8/BrXrDwnRocxrlsygkty9u8MFweBFUeYSNFf+sJK2Njj2su2Gm5Sxb8h3TelTWaUqVJ13nGJk1WY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lTCdrsfC; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lTCdrsfC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5D40C1F00893; Wed, 23 Sep 2026 17:21:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184106; bh=w76QC9QJAdMuOMrYbXs7K9mQ4Sc+nh+ldKukdJl+Sv4=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=lTCdrsfC1wELmvIfvOqOjtXJ/ZUc7ct63u/MijOd3D1W9jPMazKza02LyGd3/36U/ X0aIFutHfwyntHtrtwyZZXP++T9Ryi1xqfAgZAB1LnrsXzJ6bQw+mc1JkwmwbCssU0 v2jEXoDP3RBzljy6UC+3cHbABd0Kz2fiYtH4cythWv0cs9kAvYHwXrYOhnDRTCOawu 6TF/lsJ0PRDPynBTgQHNxS/Q/5Ol30HeX5QvM+PPtPY95WzxqE1qcC5GL+mYkoIfv8 APOYkEdh359JMQZN3kPnkMRlkgEyw5H4pAkEINE8mB8b2FXpv03azaww3kvIMK/gDl 66eejFPP/NfVQ== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:05 +0100 Subject: [PATCH v4 16/22] objtool: do not hash DWARF relocations Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-16-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=3393; i=ljs@kernel.org; h=from:subject:message-id; bh=JsrcNLsDFz81n4CrtQtvecqRzIoQyaHRODeEltcBuYs=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF7x49TRiVZYf+VH+uekbukL9nNiDPdE7YldoL6rf mJ0kF1fRykLgxgXg6yYIsvzL+L7g0TC5nVe8HeDmcPKBDKEgYtTACbi68vIsO3Rmd7ZtgmWOu+a 4+ce/XW/8eJC3Ue/XG1U3nFozd7ifJiRYYrD7XsXWrL5b9sb+V8x9YqZctbZYVJoV4VT37yzGw1 teAE= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 objtool hashes the relocation of each section so it can find them by destination, but DWARF sections are never looked up this way, so there is no reason to hash them. With CONFIG_DEBUG_INFO_DWARF5 set a defconfig vmlinux.o has ~22 million relocations, 20.5 million of which are .debug_* sections. Therefore, don't hash these. With the same configuration the objtool vmlinux.o link-time run takes ~.5s less to complete (2.63s -> 2.14s) and ~130 MiB memory to do it. The output remains unchanged. Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- tools/objtool/elf.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c index a791f4ea6ec1..98a110e3e8f5 100644 --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -1071,7 +1071,8 @@ struct reloc *elf_init_reloc(struct elf *elf, struct = section *rsec, set_reloc_type(elf, reloc, type); set_reloc_addend(elf, reloc, addend); =20 - elf_hash_add(reloc, &reloc->hash, reloc_hash(reloc)); + if (!is_dwarf_section(rsec->base)) + elf_hash_add(reloc, &reloc->hash, reloc_hash(reloc)); set_sym_next_reloc(reloc, sym->relocs); sym->relocs =3D reloc; =20 @@ -1125,16 +1126,14 @@ struct reloc *elf_init_reloc_data_sym(struct elf *e= lf, struct section *sec, =20 static int read_relocs(struct elf *elf) { - unsigned long nr_reloc, max_reloc =3D 0; + unsigned long nr_reloc, max_reloc =3D 0, nr_hashed =3D 0; struct section *rsec; struct reloc *reloc; unsigned int symndx; struct symbol *sym; + bool hashed; int i; =20 - if (!elf_alloc_hash(reloc, elf->num_relocs)) - return -1; - list_for_each_entry(rsec, &elf->sections, list) { if (!is_reloc_sec(rsec)) continue; @@ -1147,6 +1146,23 @@ static int read_relocs(struct elf *elf) =20 rsec->base->rsec =3D rsec; =20 + /* + * DWARF relocations are never looked up by destination, they + * only need to be on their symbol's list. + */ + if (!is_dwarf_section(rsec->base)) + nr_hashed +=3D sec_num_entries(rsec); + } + + if (!elf_alloc_hash(reloc, nr_hashed)) + return -1; + + list_for_each_entry(rsec, &elf->sections, list) { + if (!is_reloc_sec(rsec)) + continue; + + hashed =3D !is_dwarf_section(rsec->base); + /* nr_alloc_relocs=3D0: libelf owns d_buf */ rsec->nr_alloc_relocs =3D 0; =20 @@ -1168,7 +1184,8 @@ static int read_relocs(struct elf *elf) return -1; } =20 - elf_hash_add(reloc, &reloc->hash, reloc_hash(reloc)); + if (hashed) + elf_hash_add(reloc, &reloc->hash, reloc_hash(reloc)); set_sym_next_reloc(reloc, sym->relocs); sym->relocs =3D reloc; =20 @@ -1180,6 +1197,7 @@ static int read_relocs(struct elf *elf) if (opts.stats) { printf("max_reloc: %lu\n", max_reloc); printf("num_relocs: %lu\n", elf->num_relocs); + printf("num_relocs_hashed: %lu\n", nr_hashed); printf("reloc_bits: %d\n", elf->reloc_bits); } =20 @@ -1634,7 +1652,7 @@ static int elf_alloc_reloc(struct elf *elf, struct se= ction *rsec) =20 memcpy(new_relocs, old_relocs, nr_relocs_old * sizeof(struct reloc)); =20 - for (int i =3D 0; i < nr_relocs_old; i++) { + for (int i =3D 0; i < nr_relocs_old && !is_dwarf_section(rsec->base); i++= ) { struct reloc *old =3D &old_relocs[i]; struct reloc *new =3D &new_relocs[i]; u32 key =3D reloc_hash(old); --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C19D73E638D; Wed, 23 Sep 2026 17:21:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184120; cv=none; b=VtshV8zkdYMIphAj6qOvZhW+isgztfMsxaiZBmbMXk/Ihr7Wh9LHxL9j+CQx8bNwZqmgd4dHFt5emZhErHTR163bwCQOhyGh5yxe7e0S4i5fNcDhO6v6Ndxkeq2Ha6jKPNA5CqCcf50dMOpqKBGSrKwTxrVwXi/u9PnUz8YyTsw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184120; c=relaxed/simple; bh=aJWLamm/eoktlyXTzsOgfNH7XQ176gtamMWPuQOOBas=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=tun/nK9RZgwAQX/IrZbpRg+zX4haf0hHnI3PTaXVxXR9Rwjh0vDc7XHeHsXoIq2KwgZ8R9I9ZXqMzW6Y/It/Bal0g8FwHzTzfkdj2id2Nu+D4q7EUualbMFJfDlIz2B0Mr1Pa77gFBrmGw9hn4hZT7AY1GcEFjb5YtHFnotaIdo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lYUbZGgl; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lYUbZGgl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE7041F00898; Wed, 23 Sep 2026 17:21:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184118; bh=F36zB3TpAmg2T+DbkpmIrcomfVdouXy+Zww1GXGCAOQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=lYUbZGgl9IRhManv28ZjkgoGdTU6BwVX748h6h8D5i6ybhwkDF1pnVhuBu7ilVyoW eSqOIj7e83feUnGPwIzchnAIIGI3uFGlNMM0Vb5+6vly0Lm/98CCnL3FIGegmRIqbC 4RDLmK1QsDTrr43/um9pYA2gBMNV9VZ/QW4DL7dZbVw780hFkDt3gxbViKqcqs8AT2 iebgWegbB+rEbFGxEiSOgEpZsrOfTW81BhZsl8tRVDG7v0yUTyECL1FoSVvy59MibH CNwaF79JIsCfPmVmAJPq0Ndw8jOuay3ryigu8ZqqBTBi9B4WIF/UkcgeNjh+VSEL6W Ofu+uqvxUGbfQ== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:06 +0100 Subject: [PATCH v4 17/22] objtool: cache relocations, do less work, eliminate relocation hash Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-17-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=14487; i=ljs@kernel.org; h=from:subject:message-id; bh=aJWLamm/eoktlyXTzsOgfNH7XQ176gtamMWPuQOOBas=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF4JnbbAzLtNMMRzz2f5ssNTOPaq1sjVGQtLWyaYW 37q+RXYUcrCIMbFICumyPL8i/j+IJGweZ0X/N1g5rAygQxh4OIUgIlMr2Zk6OE78t9x71a7hf3h Ee++ZwZKXxRYKNaua8ixaNavvP4N6xkZ1pk11RmE9d0trxaQP+awQkTrS1H3idUHV7DxXEl49M+ fCQA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Instruction relocations are looked up by destination in objtool via a hash which is keyed on a 16-byte (OFFSET_STRIDE) window within the section being walked. It iterates through each 16-byte window, looking up relocations over several passes, before moving on to the next 16-byte window, caching only when a relocation is not found saving further lookups in this case. Improve upon this by introducing a per-section relocation cache storing the first relocation at or after each 64-byte window of data (an empirically determined index range), indexed by chunk. The lookup is implemented an array lookup and touches no shared state, so can be used from multiple threads. The index runs over the relocations in offset order. Usually this is how they are input, however if not, each section is sorted once, storing their offsets in reloc_order[]. This is done lazily on lookup, so nothing pays the cost that doesn't have to, and if everything is sorted nothing does. The output of objtool before and after this change was confirmed to be byte-for-byte identical both for x86_64 defconfig and allmodconfig with gcc and clang. objtool on the gcc allmodconfig vmlinux.o goes from 10.5s to 9.0s, clang from 9.8s to 8.3s and gcc defconfig from 2.08s to 1.79s, with peak memory down ~90 MiB on allmodconfig. Similar performance was also noted on a worst case scenario for unsorted input (gcc but lld for linking), which confirms that the sorting mechanism works correctly with little overhead. objtool on vmlinux.o is on the serial tail of every build that links vmlinux, no-op builds are unchanged. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 8.3s 8.2s -0.14s (-2%) x86 defconfig, touch mm/vma.c, clang 7.6s 7.3s -0.25s (-3%) x86 defconfig, clean, gcc 28.9s 28.6s -0.22s (-1%) x86 defconfig, clean, clang 29.4s 29.1s -0.28s (-1%) x86 allmodconfig, touch mm/vma.c, gcc 29.4s 28.0s -1.4s (-5%) x86 allmodconfig, touch mm/vma.c, clang 27.9s 26.2s -1.7s (-6%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- tools/objtool/elf.c | 280 ++++++++++++++++++++++++++++----= ---- tools/objtool/include/objtool/elf.h | 10 +- 2 files changed, 221 insertions(+), 69 deletions(-) diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c index 98a110e3e8f5..bd463b5df38c 100644 --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -316,35 +316,220 @@ struct symbol *find_global_symbol_by_name(const stru= ct elf *elf, const char *nam return NULL; } =20 -/* If there are multiple matches, return the first one in the range */ +static bool is_dwarf_section(struct section *sec) +{ + return !strncmp(sec->name, ".debug_", 7); +} + +/* Index the first relocation at or after each 64 byte window of the base.= */ +#define RELOC_CACHE_INDEX_SHIFT 6 + +static unsigned long reloc_cache_index(unsigned long offset) +{ + return offset >> RELOC_CACHE_INDEX_SHIFT; +} + +static void reloc_cache_free(struct section *rsec) +{ + free(rsec->reloc_cache); + free(rsec->reloc_order); + rsec->reloc_cache =3D NULL; + rsec->reloc_order =3D NULL; + rsec->nr_cache_windows =3D 0; + rsec->nr_indexed =3D 0; + rsec->sorted =3D false; +} + +/* The relocation at a position of the section's offset order. */ +static struct reloc *reloc_at(struct section *rsec, unsigned int pos) +{ + unsigned int reloc_idx =3D pos; + + if (rsec->reloc_order) + reloc_idx =3D rsec->reloc_order[pos]; + + return &rsec->relocs[reloc_idx]; +} + +static bool relocs_in_order(struct section *rsec) +{ + const unsigned int nr_relocs =3D sec_num_entries(rsec); + unsigned int i; + + for (i =3D 1; i < nr_relocs; i++) { + struct reloc *prev =3D &rsec->relocs[i - 1]; + struct reloc *reloc =3D &rsec->relocs[i]; + + if (reloc_offset(reloc) < reloc_offset(prev)) + return false; + } + + return true; +} + +/* qsort() passes no context, so the relocations being sorted live here. */ +static struct reloc *sort_relocs; + +static int compare_reloc_order(const void *a, const void *b) +{ + const unsigned int *idx_a =3D a; + const unsigned int *idx_b =3D b; + const unsigned long offset_a =3D reloc_offset(&sort_relocs[*idx_a]); + const unsigned long offset_b =3D reloc_offset(&sort_relocs[*idx_b]); + + if (offset_a < offset_b) + return -1; + if (offset_a > offset_b) + return 1; + + /* Equal offsets stay in file order. */ + if (*idx_a < *idx_b) + return -1; + return 1; +} + +/* + * gcc emits the relocations of its jumps in a second pass, so its objects, + * and a vmlinux.o lld links from them, are out of order: sort them once. + */ +static int reloc_order_build(struct section *rsec) +{ + const unsigned int nr_relocs =3D sec_num_entries(rsec); + unsigned int i; + + rsec->reloc_order =3D malloc(nr_relocs * sizeof(*rsec->reloc_order)); + if (!rsec->reloc_order) { + ERROR_GLIBC("malloc"); + return -1; + } + + for (i =3D 0; i < nr_relocs; i++) + rsec->reloc_order[i] =3D i; + + sort_relocs =3D rsec->relocs; + qsort(rsec->reloc_order, nr_relocs, sizeof(*rsec->reloc_order), + compare_reloc_order); + + return 0; +} + +/* Grow the index to cover the base section, new windows start past the en= d. */ +static int reloc_cache_resize(struct section *rsec) +{ + const unsigned int nr_windows =3D reloc_cache_index(sec_size(rsec->base))= + 1; + unsigned int *cache; + + if (nr_windows <=3D rsec->nr_cache_windows) + return 0; + + cache =3D realloc(rsec->reloc_cache, nr_windows * sizeof(*cache)); + if (!cache) { + ERROR_GLIBC("realloc"); + return -1; + } + + while (rsec->nr_cache_windows < nr_windows) + cache[rsec->nr_cache_windows++] =3D rsec->nr_indexed; + rsec->reloc_cache =3D cache; + + return 0; +} + +/* + * Index the next relocation. It must follow the previous one in position = and + * offset, otherwise the index is dropped and the next lookup rebuilds it. + */ +static int reloc_cache_add(struct section *rsec, unsigned int pos) +{ + struct reloc *reloc =3D reloc_at(rsec, pos); + const unsigned long offset =3D reloc_offset(reloc); + const unsigned long window =3D reloc_cache_index(offset); + unsigned long first_window =3D 0; + + if (pos !=3D rsec->nr_indexed) + goto unsorted; + + if (pos) { + struct reloc *prev =3D reloc_at(rsec, pos - 1); + const unsigned long prev_offset =3D reloc_offset(prev); + + if (offset < prev_offset) + goto unsorted; + first_window =3D reloc_cache_index(prev_offset) + 1; + } + + if (reloc_cache_resize(rsec)) + return -1; + if (window >=3D rsec->nr_cache_windows) + goto unsorted; + + while (first_window <=3D window) + rsec->reloc_cache[first_window++] =3D pos; + rsec->nr_indexed =3D pos + 1; + + return 0; + +unsorted: + reloc_cache_free(rsec); + return 0; +} + +static unsigned long num_relocs_sorted; + +static int init_reloc_cache(struct section *rsec) +{ + const unsigned int nr_relocs =3D sec_num_entries(rsec); + unsigned int pos; + + reloc_cache_free(rsec); + if (!relocs_in_order(rsec)) { + if (reloc_order_build(rsec)) + return -1; + num_relocs_sorted +=3D nr_relocs; + } + + rsec->sorted =3D true; + for (pos =3D 0; pos < nr_relocs; pos++) { + if (reloc_cache_add(rsec, pos)) + return -1; + } + + return 0; +} + +static bool reloc_in_range(struct reloc *reloc, unsigned long offset, + unsigned int len) +{ + return reloc_offset(reloc) >=3D offset && reloc_offset(reloc) < offset + = len; +} + +/* If there are multiple matches, return the first one in the range. */ struct reloc *find_reloc_by_dest_range(const struct elf *elf, struct secti= on *sec, unsigned long offset, unsigned int len) { - struct reloc *reloc, *r =3D NULL; - struct section *rsec; - unsigned long o; + struct section *rsec =3D sec->rsec; + unsigned long cache_idx; + unsigned int pos; =20 - rsec =3D sec->rsec; if (!rsec) return NULL; =20 - for_offset_range(o, offset, offset + len) { - elf_hash_for_each_possible(elf, reloc, reloc, hash, - sec_offset_hash(rsec, o)) { - if (reloc->sec !=3D rsec) - continue; + if (!rsec->sorted && init_reloc_cache(rsec)) + exit(1); =20 - if (reloc_offset(reloc) >=3D offset && - reloc_offset(reloc) < offset + len) { - if (!r || reloc_offset(reloc) < reloc_offset(r)) - r =3D reloc; - } - } - if (r && (reloc_offset(r) & OFFSET_STRIDE_MASK) =3D=3D o) - return r; + cache_idx =3D reloc_cache_index(offset); + if (cache_idx >=3D rsec->nr_cache_windows) + return NULL; + + /* The index gives a lower bound, scan on from there. */ + for (pos =3D rsec->reloc_cache[cache_idx]; pos < rsec->nr_indexed; pos++)= { + struct reloc *reloc =3D reloc_at(rsec, pos); + + if (reloc_offset(reloc) >=3D offset) + return reloc_in_range(reloc, offset, len) ? reloc : NULL; } =20 - return r; + return NULL; } =20 struct reloc *find_reloc_by_dest(const struct elf *elf, struct section *se= c, unsigned long offset) @@ -352,11 +537,6 @@ struct reloc *find_reloc_by_dest(const struct elf *elf= , struct section *sec, uns return find_reloc_by_dest_range(elf, sec, offset, 1); } =20 -static bool is_dwarf_section(struct section *sec) -{ - return !strncmp(sec->name, ".debug_", 7); -} - static int read_sections(struct elf *elf) { Elf_Scn *s =3D NULL; @@ -1071,8 +1251,10 @@ struct reloc *elf_init_reloc(struct elf *elf, struct= section *rsec, set_reloc_type(elf, reloc, type); set_reloc_addend(elf, reloc, addend); =20 - if (!is_dwarf_section(rsec->base)) - elf_hash_add(reloc, &reloc->hash, reloc_hash(reloc)); + if (rsec->reloc_order) + reloc_cache_free(rsec); + else if (rsec->sorted && reloc_cache_add(rsec, reloc_idx)) + return NULL; set_sym_next_reloc(reloc, sym->relocs); sym->relocs =3D reloc; =20 @@ -1126,12 +1308,11 @@ struct reloc *elf_init_reloc_data_sym(struct elf *e= lf, struct section *sec, =20 static int read_relocs(struct elf *elf) { - unsigned long nr_reloc, max_reloc =3D 0, nr_hashed =3D 0; + unsigned long nr_reloc, max_reloc =3D 0; struct section *rsec; struct reloc *reloc; unsigned int symndx; struct symbol *sym; - bool hashed; int i; =20 list_for_each_entry(rsec, &elf->sections, list) { @@ -1146,23 +1327,6 @@ static int read_relocs(struct elf *elf) =20 rsec->base->rsec =3D rsec; =20 - /* - * DWARF relocations are never looked up by destination, they - * only need to be on their symbol's list. - */ - if (!is_dwarf_section(rsec->base)) - nr_hashed +=3D sec_num_entries(rsec); - } - - if (!elf_alloc_hash(reloc, nr_hashed)) - return -1; - - list_for_each_entry(rsec, &elf->sections, list) { - if (!is_reloc_sec(rsec)) - continue; - - hashed =3D !is_dwarf_section(rsec->base); - /* nr_alloc_relocs=3D0: libelf owns d_buf */ rsec->nr_alloc_relocs =3D 0; =20 @@ -1184,21 +1348,24 @@ static int read_relocs(struct elf *elf) return -1; } =20 - if (hashed) - elf_hash_add(reloc, &reloc->hash, reloc_hash(reloc)); set_sym_next_reloc(reloc, sym->relocs); sym->relocs =3D reloc; =20 nr_reloc++; } max_reloc =3D max(max_reloc, nr_reloc); + + /* DWARF relocs are never looked up, so are not worth indexing. */ + if (is_dwarf_section(rsec->base)) + continue; + if (init_reloc_cache(rsec)) + return -1; } =20 if (opts.stats) { printf("max_reloc: %lu\n", max_reloc); printf("num_relocs: %lu\n", elf->num_relocs); - printf("num_relocs_hashed: %lu\n", nr_hashed); - printf("reloc_bits: %d\n", elf->reloc_bits); + printf("num_relocs_sorted: %lu\n", num_relocs_sorted); } =20 return 0; @@ -1345,8 +1512,7 @@ struct elf *elf_create_file(GElf_Ehdr *ehdr, const ch= ar *name) if (!elf_alloc_hash(section, 1000) || !elf_alloc_hash(section_name, 1000) || !elf_alloc_hash(symbol, 10000) || - !elf_alloc_hash(symbol_name, 10000) || - !elf_alloc_hash(reloc, 100000)) + !elf_alloc_hash(symbol_name, 10000)) return NULL; =20 null =3D elf_create_section(elf, NULL, 0, 0, SHT_NULL, 0, 0); @@ -1526,6 +1692,8 @@ struct section *elf_create_section(struct elf *elf, c= onst char *name, sec->sh.sh_type =3D type; sec->sh.sh_addralign =3D align; sec->sh.sh_flags =3D flags; + /* Relocations objtool adds are indexed as they come. */ + sec->sorted =3D type =3D=3D SHT_RELA; =20 if (name) { sec->name =3D strdup(name); @@ -1651,16 +1819,6 @@ static int elf_alloc_reloc(struct elf *elf, struct s= ection *rsec) } =20 memcpy(new_relocs, old_relocs, nr_relocs_old * sizeof(struct reloc)); - - for (int i =3D 0; i < nr_relocs_old && !is_dwarf_section(rsec->base); i++= ) { - struct reloc *old =3D &old_relocs[i]; - struct reloc *new =3D &new_relocs[i]; - u32 key =3D reloc_hash(old); - - elf_hash_del(reloc, &old->hash, key); - elf_hash_add(reloc, &new->hash, key); - } - free(old_relocs); done: rsec->relocs =3D new_relocs; diff --git a/tools/objtool/include/objtool/elf.h b/tools/objtool/include/ob= jtool/elf.h index a82517a76a0f..baa4f76b1800 100644 --- a/tools/objtool/include/objtool/elf.h +++ b/tools/objtool/include/objtool/elf.h @@ -59,6 +59,8 @@ struct section { const char *name; int idx; bool _changed, text, rodata, noinstr, init, truncate; + bool sorted; + unsigned int *reloc_cache, *reloc_order, nr_cache_windows, nr_indexed; struct reloc *relocs; unsigned long nr_alloc_relocs; struct section *twin; @@ -106,7 +108,6 @@ struct symbol { }; =20 struct reloc { - struct elf_hash_node hash; struct section *sec; struct symbol *sym; unsigned long _sym_next_reloc; @@ -127,13 +128,11 @@ struct elf { int symbol_name_bits; int section_bits; int section_name_bits; - int reloc_bits; =20 struct elf_hash_node **symbol_hash; struct elf_hash_node **symbol_name_hash; struct elf_hash_node **section_hash; struct elf_hash_node **section_name_hash; - struct elf_hash_node **reloc_hash; =20 struct section *section_data; struct symbol *symbol_data; @@ -575,9 +574,4 @@ static inline u32 sec_offset_hash(struct section *sec, = unsigned long offset) return ol; } =20 -static inline u32 reloc_hash(struct reloc *reloc) -{ - return sec_offset_hash(reloc->sec, reloc_offset(reloc)); -} - #endif /* _OBJTOOL_ELF_H */ --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 256863C7691; Wed, 23 Sep 2026 17:22:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184132; cv=none; b=B61bxOfJ8dhZ4aLDEPjdik9pQSF3MBd2trpL+Bi2nWu3q05HuRlj4klgG2jPxzF548oQnEz22cZ6fCsIt7O+hatowfvMvHblUwI2YmOx3GV+96f0rxHN6E4a6DyKDC7EuTJMclfkCd/2sNzJdJjPS0d+kAjeRLuLMRcZgLY9Sqs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184132; c=relaxed/simple; bh=1MNx2aSSjSgi0WUnp9kWu6YKYuPNVv4LhqSB+6OA3nE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ppGluNJLci+WlegD+PdbPM7N6xkMn3WEHf5W6k+CtNLITbjWCtgFVgK1/7tuZVz3AlFqxkGH78RqzWtN04jdszuZ4NOGhcxc+Dg+XReYe0MQDt+dvGi+mTPVp1CBI1TtsfGcN8akIAMHVNrUObJ2+fKKCr2h2EgW9T/DNCgtRjI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UPsDCIGD; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UPsDCIGD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 280DB1F000FF; Wed, 23 Sep 2026 17:21:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184131; bh=flxRB0RNwZz3AOlLZeEynIEfspQVvaUpjYF/Tjf2+64=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=UPsDCIGDyg1u84TMdOj16FUNyBreuz/tc7euMiNjyhkmq6UrVq3SdvmLqHg4e0OWD XTTb1La8nHitRQtByr9yWWn47ViPBBI3QelGrraZ6k/o0WRGHb9VDDjO1s/Kc7Pf+Y 5j8bZMXhe+IsQMpY8tg0UtFIgkfpsY4Ds0QSWk198HRZ6UbFGJFzFnFCWpH/iQcRlT 7D7j5U4nv8bKw6C+CZO7ZpDcMsT4lNYc+8+QGUjSmVaseyrcagM+C/npSW0DJGRoa3 czc6bfrZdQMbOL4XtnxMVSzRINdwMhBfnWXvy4HLCAfrLNZJA0SF7mORqg46BvplSl voF8XL6yvIH7g== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:07 +0100 Subject: [PATCH v4 18/22] objtool: size the instruction hash to the text Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-18-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=5734; i=ljs@kernel.org; h=from:subject:message-id; bh=1MNx2aSSjSgi0WUnp9kWu6YKYuPNVv4LhqSB+6OA3nE=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF7lXihiJcLXFaF869vqaPZeM+2GPX1SCxUn615el e/24tLXjlIWBjEuBlkxRZbnX8T3B4mEzeu84O8GM4eVCWQIAxenAEzk+21Ghh+778em2s+Izjub fcrG7NNd7elhhzk5gm4I1zq3BPrvt2H47xDW7p90dtYWqTOnH+2ZY+KsqrR0duqr21PM63i9jnp dZwYA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 The instruction hash is a fixed 2^20 buckets, which for an x86-64 allmodconfig vmlinux.o of ~16 million instructions leaves find_insn() walking chains of four or more entries. Size it by the amount of text to be decoded instead, at one bucket per OFFSET_STRIDE window, as sec_offset_hash() keys on those and anything finer would sit empty. This gives 2^20 buckets (8 MiB) for a defconfig vmlinux.o and 2^22 (32 MiB) for allmodconfig, so it is not an egregious use of memory. The output of objtool before and after this change was confirmed to be byte-for-byte identical for x86_64 defconfig and allmodconfig. On a 128-thread machine, objtool on the gcc allmodconfig vmlinux.o goes from 9.0s to 6.8s. A defconfig vmlinux.o already had 2^20 buckets and is unchanged. objtool on vmlinux.o is on the serial tail of every build that links vmlinux, no-op builds are unchanged. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 allmodconfig, touch mm/vma.c, gcc 28.0s 25.7s -2.4s (-8%) x86 allmodconfig, touch mm/vma.c, clang 26.2s 24.1s -2.0s (-8%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- tools/objtool/check.c | 58 +++++++++++++++++++++++++++++= ++-- tools/objtool/include/objtool/objtool.h | 3 +- tools/objtool/objtool.c | 1 - 3 files changed, 58 insertions(+), 4 deletions(-) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 464f6c9d9ff0..62a3e1d4e9e5 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -38,12 +38,22 @@ struct disas_context *objtool_disas_ctx; =20 size_t sym_name_max_len; =20 +static struct hlist_head *insn_hash_head(struct objtool_file *file, + struct section *sec, unsigned long offset) +{ + /* Determine instruction hash based on section index and offset. */ + const u32 sec_hash =3D sec_offset_hash(sec, offset); + const u32 hash =3D hash_min(sec_hash, file->insn_hash_bits); + + return &file->insn_hash[hash]; +} + struct instruction *find_insn(struct objtool_file *file, struct section *sec, unsigned long offset) { struct instruction *insn; =20 - hash_for_each_possible(file->insn_hash, insn, hash, sec_offset_hash(sec, = offset)) { + hlist_for_each_entry(insn, insn_hash_head(file, sec, offset), hash) { if (insn->sec =3D=3D sec && insn->offset =3D=3D offset) return insn; } @@ -403,6 +413,44 @@ static void *cfi_hash_alloc(unsigned long size) static unsigned long nr_insns; static unsigned long nr_insns_visited; =20 +/* + * sec_offset_hash() keys on OFFSET_STRIDE windows, so the instructions of= a + * window share a chain and buckets beyond one per window would sit empty. + */ +#define INSN_HASH_BYTES_PER_BUCKET OFFSET_STRIDE +#define INSN_HASH_MIN_BITS 10 + +static unsigned long total_text_size(struct objtool_file *file) +{ + unsigned long size =3D 0; + struct section *sec; + + for_each_sec(file->elf, sec) + if (is_text_sec(sec)) + size +=3D sec_size(sec); + + return size; +} + +static int alloc_insn_hash(struct objtool_file *file, unsigned long text_s= ize) +{ + const unsigned long nr_buckets =3D text_size / INSN_HASH_BYTES_PER_BUCKET; + const int bits =3D ilog2(nr_buckets); + + file->insn_hash_bits =3D max(INSN_HASH_MIN_BITS, bits); + file->insn_hash =3D calloc(1UL << file->insn_hash_bits, + sizeof(*file->insn_hash)); + if (!file->insn_hash) { + ERROR_GLIBC("calloc"); + return -1; + } + + if (opts.stats) + printf("insn_hash_bits: %d\n", file->insn_hash_bits); + + return 0; +} + /* * Call the arch-specific instruction decoder for all the instructions and= add * them to the global instruction list. @@ -414,6 +462,9 @@ static int decode_instructions(struct objtool_file *fil= e) unsigned long offset; struct instruction *insn; =20 + if (alloc_insn_hash(file, total_text_size(file))) + return -1; + for_each_sec(file->elf, sec) { struct instruction *insns =3D NULL; u8 prev_len =3D 0; @@ -474,7 +525,7 @@ static int decode_instructions(struct objtool_file *fil= e) if (insn->type =3D=3D INSN_BUG) insn->dead_end =3D true; =20 - hash_add(file->insn_hash, &insn->hash, sec_offset_hash(sec, insn->offse= t)); + hlist_add_head(&insn->hash, insn_hash_head(file, sec, insn->offset)); nr_insns++; } =20 @@ -4802,6 +4853,9 @@ void free_insns(struct objtool_file *file) =20 for (chunk =3D chunks; chunk; chunk =3D chunk->next) free(chunk->addr); + + free(file->insn_hash); + file->insn_hash =3D NULL; } =20 const char *objtool_disas_insn(struct instruction *insn) diff --git a/tools/objtool/include/objtool/objtool.h b/tools/objtool/includ= e/objtool/objtool.h index 6dc12a59ad00..79fe82b7397e 100644 --- a/tools/objtool/include/objtool/objtool.h +++ b/tools/objtool/include/objtool/objtool.h @@ -21,7 +21,8 @@ struct pv_state { =20 struct objtool_file { struct elf *elf; - DECLARE_HASHTABLE(insn_hash, 20); + struct hlist_head *insn_hash; + int insn_hash_bits; struct list_head retpoline_call_list; struct list_head return_thunk_list; struct list_head static_call_list; diff --git a/tools/objtool/objtool.c b/tools/objtool/objtool.c index a4e139dee7e9..71e048f8582a 100644 --- a/tools/objtool/objtool.c +++ b/tools/objtool/objtool.c @@ -29,7 +29,6 @@ struct objtool_file *objtool_open_read(const char *filena= me) if (!file.elf) return NULL; =20 - hash_init(file.insn_hash); INIT_LIST_HEAD(&file.retpoline_call_list); INIT_LIST_HEAD(&file.return_thunk_list); INIT_LIST_HEAD(&file.static_call_list); --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 873CE37C0EC; Wed, 23 Sep 2026 17:22:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184145; cv=none; b=SZG3b0359Sb4R6r1GRfR3I7k1BuKFQE5PuBf+DdBnTCy93BRSAA42uSxLiBQs11myUp6Z3PWXtPVz6tR6NgA7aqtQJBxCr6pXTIpFqwwbdRh37lNpXWQmRr6vi5+0pDjL3n8QxWxsBVEvBbg0QBWj5+eAxhNRYJ/GqhEjVzEs2Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184145; c=relaxed/simple; bh=AFU7WGLmwsor86cekJ0DNBRJvoR8xSsmpIEsRcHaphA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=k3TGttpiCeKqh2Eb0q50D8IaaIA5Jirwz/UJY+qXuep8ko+iIi5UJ2QiFuRDBpFykfToTt8A3CbstRMW/p9t6UeysXYE0MprdbHokxVHMtIpMvFAhG5B5FfB4FvnAx9/UIQdsxDZ4yXWUGZdkjOItBRhNSX7t2SZk4dTfGJoPI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=h53O9Yuu; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="h53O9Yuu" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 84EA11F00898; Wed, 23 Sep 2026 17:22:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184143; bh=Kgt7MRzfwT7+cKjpuYBu2VYbVW9bjg716zQeCtE2Fq4=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=h53O9Yuu3CReC8vpcSAvBjGIYuKwAGqXVmCHSD9sOSMMN8VdK84Zb70Felc3XcF/u tLZRcBktLKCtIncNYhBuCA+nUXvRO67gyalWCFE4yuRxMn2GXTYh/gxOT0nSsPwQKb lB+/N8NXqX9pLExt8FrhcVi5kp0qPjr9dXfnupgp3tur3ul7KMv7msoudtzGcZ2O2S Y/4JjKvIpVYCANAdAskW/Ll3sbDv5zrlTMutGFSZwgAuJ0SCD0stPmNdMFapawBEH2 l18gatZL+a6WWXUiZHB3CXuZlx4lnB+RMmRfgnIC8GmV9/slG3pIA3/1zYNCacUGNE oIJeoKdxEflXw== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:08 +0100 Subject: [PATCH v4 19/22] objtool: decode instructions and resolve branch targets in parallel Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-19-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=29100; i=ljs@kernel.org; h=from:subject:message-id; bh=AFU7WGLmwsor86cekJ0DNBRJvoR8xSsmpIEsRcHaphA=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF6tsF7DsOLcapMPiwVr2PyLcm7+WJJs2XSibqbLx HXZJ66t7ChlYRDjYpAVU2R5/kV8f5BI2LzOC/5uMHNYmUCGMHBxCsBEjqky/I9VrzKLS9ecJcMg nt2/XmlBg8cPk7cKr/KCtm65tPzjVhlGhq9MSkZz9wUptxSFZ+26/caBO+RvxGzdcvk3y7S/HTq xggkA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 During a kernel build objtool is used to decode vmlinux.o's instructions and resolve every jump and call destination. This forms a large part of the work objtool does during the build process, and it is all done in serial. Decode these in parallel at a function granularity to speed things up, but limit this to invocations that pass --link, and only where the there is 8 MiB or more text to justify it. In practice this limits this to processing vmlinux.o in the kernel build and modules are processed as they were before. Only the instruction hash is shared between the threads and nothing is ever removed from it, so an insertion is a compare-and-swap on the bucket head. Threading is limited to decoding and the jump pass, so the gain flattens out at 16 threads and any further threads were found to only add overhead. When performing an allmodconfig build, the clang invocation of objtool when processing vmlinux.o took 5.93s on 1 thread, 4.56s on 8, 4.51s on 16 and 4.63s on 128. Therefore cap the thread count at 16 or the number of CPUs, whichever is fewer. Also add the reloc_cache_lock mutex so find_reloc_by_dest_range() sorts relocations without a risk of a race. In practice no contention should be possible, but keep the lock as a guarantee of this. The output of objtool before and after this change was confirmed to be byte-for-byte identical for x86_64 defconfig and allmodconfig with gcc and clang, and for a loongarch defconfig, where objtool runs on every object. On a 128-thread machine, objtool on the clang allmodconfig vmlinux.o goes from 5.2s to 4.4s (6.5s to 4.4s together with the previous two patches), and on defconfig from 1.99s to 1.38s. objtool on vmlinux.o is on the serial tail of every build that links vmlinux, no-op builds are unchanged. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 8.2s 7.7s -0.55s (-7%) x86 defconfig, touch mm/vma.c, clang 7.3s 6.9s -0.44s (-6%) x86 defconfig, clean, gcc 28.6s 28.2s -0.47s (-2%) x86 defconfig, clean, clang 29.1s 28.7s -0.45s (-2%) x86 allmodconfig, touch mm/vma.c, gcc 25.7s 23.7s -2.0s (-8%) x86 allmodconfig, touch mm/vma.c, clang 24.1s 22.5s -1.6s (-7%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- tools/objtool/Makefile | 2 +- tools/objtool/check.c | 717 ++++++++++++++++++++++++++++++++++++--------= ---- tools/objtool/elf.c | 15 +- tools/objtool/objtool.c | 14 +- 4 files changed, 559 insertions(+), 189 deletions(-) diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile index a4484fd22a96..2de50c3917ba 100644 --- a/tools/objtool/Makefile +++ b/tools/objtool/Makefile @@ -63,7 +63,7 @@ INCLUDES :=3D -I$(srctree)/tools/include \ OBJTOOL_CFLAGS :=3D -std=3Dgnu11 -fomit-frame-pointer -O2 -g $(WARNINGS) \ $(INCLUDES) $(LIBELF_FLAGS) $(LIBXXHASH_CFLAGS) $(HOSTCFLAGS) =20 -OBJTOOL_LDFLAGS :=3D $(LIBSUBCMD) $(LIBELF_LIBS) $(LIBXXHASH_LIBS) $(HOSTL= DFLAGS) +OBJTOOL_LDFLAGS :=3D $(LIBSUBCMD) $(LIBELF_LIBS) $(LIBXXHASH_LIBS) -lpthre= ad $(HOSTLDFLAGS) =20 # Allow old libelf to be used: elfshdr :=3D $(shell echo '$(pound)include ' | $(HOSTCC) $(OBJTO= OL_CFLAGS) -x c -E - 2>/dev/null | grep elf_getshdr) diff --git a/tools/objtool/check.c b/tools/objtool/check.c index 62a3e1d4e9e5..52af91e048a2 100644 --- a/tools/objtool/check.c +++ b/tools/objtool/check.c @@ -7,6 +7,9 @@ #include #include #include +#include +#include +#include #include #include =20 @@ -24,6 +27,7 @@ #include #include #include +#include #include #include =20 @@ -64,14 +68,13 @@ struct instruction *find_insn(struct objtool_file *file, struct instruction *next_insn_same_sec(struct objtool_file *file, struct instruction *insn) { - if (insn->idx =3D=3D INSN_CHUNK_MAX) - return find_insn(file, insn->sec, insn->offset + insn->len); + const unsigned long next_offset =3D insn->offset + insn->len; =20 - insn++; - if (!insn->len) - return NULL; + /* A chunk ends at its last slot or an empty one, so look the next up. */ + if (insn->idx =3D=3D INSN_CHUNK_MAX || !insn[1].len) + return find_insn(file, insn->sec, next_offset); =20 - return insn; + return insn + 1; } =20 struct instruction *next_insn_same_func(struct objtool_file *file, @@ -413,6 +416,12 @@ static void *cfi_hash_alloc(unsigned long size) static unsigned long nr_insns; static unsigned long nr_insns_visited; =20 +/* Only the vmlinux.o link, and only if it is this large, runs on several = threads. */ +#define DECODE_THREADED_MIN_TEXT SZ_8M +/* Only decoding and the branch passes are threaded, so more gains nothing= . */ +#define DECODE_MAX_THREADS 16 +#define DECODE_RANGES_PER_THREAD 4 + /* * sec_offset_hash() keys on OFFSET_STRIDE windows, so the instructions of= a * window share a chain and buckets beyond one per window would sit empty. @@ -451,24 +460,350 @@ static int alloc_insn_hash(struct objtool_file *file= , unsigned long text_size) return 0; } =20 +/* Per-thread state, only instruction hash is shared. */ +struct insn_range { + struct section *sec; + unsigned long start, end; + struct instruction *first, *last; + unsigned long nr_insns; + int ret; + + /* + * Each thread writes to its own copy of an objtool file, which are + * combined upon completion. + */ + struct objtool_file shadow; +}; + +#define range_for_each_insn(file, range, insn) \ + for (insn =3D (range)->first; \ + insn && insn->offset < (range)->end; \ + insn =3D next_insn_same_sec(file, insn)) + +typedef int (*range_fn_t)(struct objtool_file *file, struct insn_range *ra= nge); + +struct range_work { + range_fn_t fn; +}; + +static struct insn_range *decode_ranges; +static unsigned int nr_decode_ranges, next_decode_range, nr_decode_threads; + +/* The kernel's try_cmpxchg(); the tools' cmpxchg() is host-arch only. */ +static bool hlist_try_cmpxchg(struct hlist_node **ptr, struct hlist_node *= *old, + struct hlist_node *new) +{ + struct hlist_node *seen =3D __sync_val_compare_and_swap(ptr, *old, new); + + if (seen =3D=3D *old) + return true; + + *old =3D seen; + return false; +} + +/* Nothing is ever removed, so push onto the bucket as llist_add() does. */ +static void insn_hash_add(struct objtool_file *file, struct instruction *i= nsn) +{ + struct hlist_head *head =3D insn_hash_head(file, insn->sec, insn->offset); + struct hlist_node *first =3D READ_ONCE(head->first); + + insn->hash.pprev =3D &head->first; + do { + insn->hash.next =3D first; + } while (!hlist_try_cmpxchg(&head->first, &first, &insn->hash)); +} + +/* The slot after prev in its chunk, or the first of a new chunk. */ +static struct instruction *next_insn_slot(struct instruction *prev) +{ + struct instruction *insn; + + if (prev && prev->idx < INSN_CHUNK_MAX) { + insn =3D prev + 1; + insn->idx =3D prev->idx + 1; + return insn; + } + + insn =3D calloc(INSN_CHUNK_SIZE, sizeof(*insn)); + if (!insn) + ERROR_GLIBC("calloc"); + + return insn; +} + +static int decode_range(struct objtool_file *file, struct insn_range *rang= e) +{ + struct instruction *insn =3D NULL; + struct section *sec =3D range->sec; + unsigned long offset; + u8 prev_len =3D 0; + + for (offset =3D range->start; offset < range->end; offset +=3D insn->len)= { + const unsigned long remaining =3D sec_size(sec) - offset; + + insn =3D next_insn_slot(insn); + if (!insn) + return -1; + + INIT_LIST_HEAD(&insn->call_node); + insn->sec =3D sec; + insn->offset =3D offset; + insn->prev_len =3D prev_len; + + if (arch_decode_instruction(file, sec, offset, remaining, insn)) + return -1; + + prev_len =3D insn->len; + + if (insn->type =3D=3D INSN_BUG) + insn->dead_end =3D true; + + insn_hash_add(file, insn); + if (!range->first) + range->first =3D insn; + range->nr_insns++; + } + range->last =3D insn; + + /* The range ends at a function symbol, so decoding must land on it. */ + if (offset !=3D range->end) { + ERROR("%s: no instruction boundary at %s", sec->name, + offstr(sec, range->end)); + return -1; + } + + return 0; +} + +static int run_threads(void *(*fn)(void *), void *arg, unsigned int nr_thr= eads) +{ + unsigned int nr_started, i; + pthread_t *threads; + int ret =3D 0; + + if (nr_threads <=3D 1) { + fn(arg); + return 0; + } + + threads =3D calloc(nr_threads, sizeof(*threads)); + if (!threads) { + ERROR_GLIBC("calloc"); + return -1; + } + + for (nr_started =3D 0; nr_started < nr_threads; nr_started++) { + if (pthread_create(&threads[nr_started], NULL, fn, arg)) { + ERROR_GLIBC("pthread_create"); + ret =3D -1; + break; + } + } + + for (i =3D 0; i < nr_started; i++) + pthread_join(threads[i], NULL); + + free(threads); + return ret; +} + +/* Hand out the ranges one at a time, or NULL once they are all taken. */ +static struct insn_range *claim_decode_range(void) +{ + const unsigned int idx =3D __sync_fetch_and_add(&next_decode_range, 1); + + return idx < nr_decode_ranges ? &decode_ranges[idx] : NULL; +} + +static void *range_worker(void *arg) +{ + const struct range_work *work =3D arg; + struct insn_range *range; + + while ((range =3D claim_decode_range())) + range->ret =3D work->fn(&range->shadow, range); + + return NULL; +} + +/* The lists in the objtool_file that the passes add instructions to. */ +static const size_t shadow_list_offsets[] =3D { + offsetof(struct objtool_file, retpoline_call_list), + offsetof(struct objtool_file, return_thunk_list), + offsetof(struct objtool_file, static_call_list), + offsetof(struct objtool_file, mcount_loc_list), + offsetof(struct objtool_file, endbr_list), + offsetof(struct objtool_file, call_list), +}; + +static struct list_head *shadow_list(struct objtool_file *file, + unsigned int idx) +{ + return (void *)file + shadow_list_offsets[idx]; +} + +static void init_range_shadow(struct objtool_file *file, + struct insn_range *range) +{ + unsigned int i; + + range->shadow =3D *file; + range->ret =3D 0; + for (i =3D 0; i < ARRAY_SIZE(shadow_list_offsets); i++) + INIT_LIST_HEAD(shadow_list(&range->shadow, i)); +} + +/* Joined in range order, which is the order a single walk would produce. = */ +static int join_range_shadow(struct objtool_file *file, + struct insn_range *range) +{ + unsigned int i; + + for (i =3D 0; i < ARRAY_SIZE(shadow_list_offsets); i++) + list_splice_tail(shadow_list(&range->shadow, i), + shadow_list(file, i)); + + return range->ret; +} + +/* Run a pass over the instructions, one range per thread at a time. */ +static int run_insn_ranges(struct objtool_file *file, range_fn_t fn) +{ + struct range_work work =3D { .fn =3D fn }; + unsigned int i; + int ret =3D 0; + + for (i =3D 0; i < nr_decode_ranges; i++) + init_range_shadow(file, &decode_ranges[i]); + next_decode_range =3D 0; + + if (run_threads(range_worker, &work, nr_decode_threads)) + return -1; + + for (i =3D 0; i < nr_decode_ranges; i++) { + if (join_range_shadow(file, &decode_ranges[i])) + ret =3D -1; + } + + return ret; +} + +static int add_decode_range(struct section *sec, unsigned long start, + unsigned long end) +{ + const size_t size =3D (nr_decode_ranges + 1) * sizeof(*decode_ranges); + struct insn_range *range; + + decode_ranges =3D realloc(decode_ranges, size); + if (!decode_ranges) { + ERROR_GLIBC("realloc"); + return -1; + } + + range =3D &decode_ranges[nr_decode_ranges++]; + memset(range, 0, sizeof(*range)); + range->sec =3D sec; + range->start =3D start; + range->end =3D end; + + return 0; +} + +/* Split a section into ranges of roughly range_size, at function starts. = */ +static int add_decode_ranges(struct section *sec, unsigned long range_size) +{ + const unsigned long size =3D sec_size(sec); + unsigned long start =3D 0; + struct symbol *sym; + + if (!range_size) + return add_decode_range(sec, 0, size); + + sec_for_each_sym(sec, sym) { + if (!is_func_sym(sym) || sym->offset <=3D start || + sym->offset >=3D size) + continue; + if (sym->offset - start < range_size) + continue; + + if (add_decode_range(sec, start, sym->offset)) + return -1; + start =3D sym->offset; + } + + return add_decode_range(sec, start, size); +} + +static void free_decode_ranges(void) +{ + free(decode_ranges); + decode_ranges =3D NULL; + nr_decode_ranges =3D 0; + next_decode_range =3D 0; +} + +/* A range's first instruction follows the last of the range before it. */ +static void link_decode_ranges(void) +{ + unsigned int i; + + for (i =3D 1; i < nr_decode_ranges; i++) { + const struct insn_range *prev =3D &decode_ranges[i - 1]; + struct insn_range *range =3D &decode_ranges[i]; + + if (prev->sec !=3D range->sec || !prev->last || !range->first) + continue; + + range->first->prev_len =3D prev->last->len; + } +} + +static unsigned int decode_threads(unsigned long text_size) +{ + const long nr_cpus =3D sysconf(_SC_NPROCESSORS_ONLN); + + /* Modules are processed inside the parallel build, on one thread. */ + if (!opts.link || opts.module) + return 1; + if (text_size < DECODE_THREADED_MIN_TEXT || nr_cpus < 2) + return 1; + + return min_t(unsigned int, nr_cpus, DECODE_MAX_THREADS); +} + +/* Several ranges per thread so uneven ones balance out; 0 means per secti= on. */ +static unsigned long decode_range_size(unsigned long text_size, + unsigned int nr_threads) +{ + const unsigned int nr_ranges =3D nr_threads * DECODE_RANGES_PER_THREAD; + + if (nr_threads <=3D 1) + return 0; + + return text_size / nr_ranges; +} + /* * Call the arch-specific instruction decoder for all the instructions and= add * them to the global instruction list. */ static int decode_instructions(struct objtool_file *file) { + const unsigned long text_size =3D total_text_size(file); + unsigned long range_size; + struct instruction *insn; struct section *sec; struct symbol *func; - unsigned long offset; - struct instruction *insn; + unsigned int i; =20 - if (alloc_insn_hash(file, total_text_size(file))) + if (alloc_insn_hash(file, text_size)) return -1; =20 + nr_decode_threads =3D decode_threads(text_size); + range_size =3D decode_range_size(text_size, nr_decode_threads); + for_each_sec(file->elf, sec) { - struct instruction *insns =3D NULL; - u8 prev_len =3D 0; - u8 idx =3D 0; =20 if (!is_text_sec(sec)) continue; @@ -493,41 +828,20 @@ static int decode_instructions(struct objtool_file *f= ile) if (!strcmp(sec->name, ".init.text") && !opts.module) sec->init =3D true; =20 - for (offset =3D 0; offset < sec_size(sec); offset +=3D insn->len) { - if (!insns || idx =3D=3D INSN_CHUNK_MAX) { - insns =3D calloc(INSN_CHUNK_SIZE, sizeof(*insn)); - if (!insns) { - ERROR_GLIBC("calloc"); - return -1; - } - idx =3D 0; - } else { - idx++; - } - insn =3D &insns[idx]; - insn->idx =3D idx; - - INIT_LIST_HEAD(&insn->call_node); - insn->sec =3D sec; - insn->offset =3D offset; - insn->prev_len =3D prev_len; + if (add_decode_ranges(sec, range_size)) + return -1; + } =20 - if (arch_decode_instruction(file, sec, offset, sec_size(sec) - offset, = insn)) - return -1; + if (run_insn_ranges(file, decode_range)) + return -1; =20 - prev_len =3D insn->len; + for (i =3D 0; i < nr_decode_ranges; i++) + nr_insns +=3D decode_ranges[i].nr_insns; + link_decode_ranges(); =20 - /* - * By default, "ud2" is a dead end unless otherwise - * annotated, because GCC 7 inserts it for certain - * divide-by-zero cases. - */ - if (insn->type =3D=3D INSN_BUG) - insn->dead_end =3D true; - - hlist_add_head(&insn->hash, insn_hash_head(file, sec, insn->offset)); - nr_insns++; - } + for_each_sec(file->elf, sec) { + if (!is_text_sec(sec)) + continue; =20 sec_for_each_sym(sec, func) { if (!is_notype_sym(func) && !is_func_sym(func)) @@ -1570,133 +1884,147 @@ static bool is_first_func_insn(struct objtool_fil= e *file, /* * Find the destination instructions for all jumps. */ -static int add_jump_destinations(struct objtool_file *file) +static int add_jump_destination(struct objtool_file *file, struct instruct= ion *insn) { - struct instruction *insn; struct reloc *reloc; + struct symbol *func =3D insn_func(insn); + struct instruction *dest_insn; + struct section *dest_sec; + struct symbol *dest_sym; + unsigned long dest_off; =20 - for_each_insn(file, insn) { - struct symbol *func =3D insn_func(insn); - struct instruction *dest_insn; - struct section *dest_sec; - struct symbol *dest_sym; - unsigned long dest_off; + if (!is_static_jump(insn)) + return 0; =20 - if (!is_static_jump(insn)) - continue; + if (insn->jump_dest) { + /* + * handle_group_alt() may have previously set + * 'jump_dest' for some alternatives. + */ + return 0; + } =20 - if (insn->jump_dest) { - /* - * handle_group_alt() may have previously set - * 'jump_dest' for some alternatives. - */ - continue; - } + reloc =3D insn_reloc(file, insn); + if (!reloc) { + dest_sec =3D insn->sec; + dest_off =3D arch_jump_destination(insn); + dest_sym =3D dest_sec->sym; + } else { + dest_sym =3D reloc->sym; + if (is_undef_sym(dest_sym)) { + if (dest_sym->retpoline_thunk) { + if (add_retpoline_call(file, insn)) + return -1; + return 0; + } =20 - reloc =3D insn_reloc(file, insn); - if (!reloc) { - dest_sec =3D insn->sec; - dest_off =3D arch_jump_destination(insn); - dest_sym =3D dest_sec->sym; - } else { - dest_sym =3D reloc->sym; - if (is_undef_sym(dest_sym)) { - if (dest_sym->retpoline_thunk) { - if (add_retpoline_call(file, insn)) - return -1; - continue; - } + if (dest_sym->return_thunk) { + add_return_call(file, insn, true); + return 0; + } =20 - if (dest_sym->return_thunk) { - add_return_call(file, insn, true); - continue; - } + /* External symbol */ + if (func) { + /* External sibling call */ + if (add_call_dest(file, insn, dest_sym, true)) + return -1; + return 0; + } =20 - /* External symbol */ - if (func) { - /* External sibling call */ - if (add_call_dest(file, insn, dest_sym, true)) - return -1; - continue; - } + /* Non-func asm code jumping to external symbol */ + return 0; + } =20 - /* Non-func asm code jumping to external symbol */ - continue; - } + dest_sec =3D dest_sym->sec; + dest_off =3D dest_sym->offset + arch_insn_adjusted_addend(insn, reloc); + } =20 - dest_sec =3D dest_sym->sec; - dest_off =3D dest_sym->offset + arch_insn_adjusted_addend(insn, reloc); + dest_insn =3D find_insn(file, dest_sec, dest_off); + if (!dest_insn) { + struct symbol *sym =3D find_symbol_by_offset(dest_sec, dest_off); + + /* + * retbleed_untrain_ret() jumps to + * __x86_return_thunk(), but objtool can't find + * the thunk's starting RET instruction, + * because the RET is also in the middle of + * another instruction. Objtool only knows + * about the outer instruction. + */ + if (sym && sym->embedded_insn) { + add_return_call(file, insn, false); + return 0; } =20 - dest_insn =3D find_insn(file, dest_sec, dest_off); - if (!dest_insn) { - struct symbol *sym =3D find_symbol_by_offset(dest_sec, dest_off); + /* + * GCOV/KCOV dead code can jump to the end of + * the function/section. + */ + if (file->ignore_unreachables && func && + dest_sec =3D=3D insn->sec && + dest_off =3D=3D func->offset + func->len) + return 0; =20 - /* - * retbleed_untrain_ret() jumps to - * __x86_return_thunk(), but objtool can't find - * the thunk's starting RET instruction, - * because the RET is also in the middle of - * another instruction. Objtool only knows - * about the outer instruction. - */ - if (sym && sym->embedded_insn) { - add_return_call(file, insn, false); - continue; - } + ERROR_INSN(insn, "can't find jump dest instruction at %s", + offstr(dest_sec, dest_off)); + return -1; + } =20 - /* - * GCOV/KCOV dead code can jump to the end of - * the function/section. - */ - if (file->ignore_unreachables && func && - dest_sec =3D=3D insn->sec && - dest_off =3D=3D func->offset + func->len) - continue; + if (!dest_sym || is_sec_sym(dest_sym)) { + dest_sym =3D insn_sym(dest_insn); + if (!dest_sym) + goto set_jump_dest; + } =20 - ERROR_INSN(insn, "can't find jump dest instruction at %s", - offstr(dest_sec, dest_off)); + if (dest_sym->retpoline_thunk && dest_insn->offset =3D=3D dest_sym->offse= t) { + if (add_retpoline_call(file, insn)) return -1; - } + return 0; + } =20 - if (!dest_sym || is_sec_sym(dest_sym)) { - dest_sym =3D insn_sym(dest_insn); - if (!dest_sym) - goto set_jump_dest; - } + if (dest_sym->return_thunk && dest_insn->offset =3D=3D dest_sym->offset) { + add_return_call(file, insn, true); + return 0; + } =20 - if (dest_sym->retpoline_thunk && dest_insn->offset =3D=3D dest_sym->offs= et) { - if (add_retpoline_call(file, insn)) - return -1; - continue; - } + if (!insn_sym(insn) || insn_sym(insn)->pfunc =3D=3D dest_sym->pfunc) + goto set_jump_dest; =20 - if (dest_sym->return_thunk && dest_insn->offset =3D=3D dest_sym->offset)= { - add_return_call(file, insn, true); - continue; - } + /* + * Internal cross-function jump. + */ =20 - if (!insn_sym(insn) || insn_sym(insn)->pfunc =3D=3D dest_sym->pfunc) - goto set_jump_dest; + if (is_first_func_insn(file, dest_insn)) { + /* Internal sibling call */ + if (add_call_dest(file, insn, dest_sym, true)) + return -1; + return 0; + } =20 - /* - * Internal cross-function jump. - */ +set_jump_dest: + insn->jump_dest =3D dest_insn; =20 - if (is_first_func_insn(file, dest_insn)) { - /* Internal sibling call */ - if (add_call_dest(file, insn, dest_sym, true)) - return -1; - continue; - } + return 0; +} =20 -set_jump_dest: - insn->jump_dest =3D dest_insn; +static int add_jump_destinations_range(struct objtool_file *file, + struct insn_range *range) +{ + struct instruction *insn; + + range_for_each_insn(file, range, insn) { + if (add_jump_destination(file, insn)) + return -1; } =20 return 0; } =20 +static int add_jump_destinations(struct objtool_file *file) +{ + return run_insn_ranges(file, add_jump_destinations_range); +} + static struct symbol *find_call_destination(struct section *sec, unsigned = long offset) { struct symbol *call_dest; @@ -1711,59 +2039,82 @@ static struct symbol *find_call_destination(struct = section *sec, unsigned long o /* * Find the destination instructions for all calls. */ -static int add_call_destinations(struct objtool_file *file) +static int add_call_destination(struct objtool_file *file, struct instruct= ion *insn) { - struct instruction *insn; unsigned long dest_off; struct symbol *dest; struct reloc *reloc; + struct symbol *func =3D insn_func(insn); =20 - for_each_insn(file, insn) { - struct symbol *func =3D insn_func(insn); - if (insn->type !=3D INSN_CALL) - continue; + if (insn->type !=3D INSN_CALL) + return 0; =20 - reloc =3D insn_reloc(file, insn); - if (!reloc) { - dest_off =3D arch_jump_destination(insn); - dest =3D find_call_destination(insn->sec, dest_off); + reloc =3D insn_reloc(file, insn); + if (!reloc) { + dest_off =3D arch_jump_destination(insn); + dest =3D find_call_destination(insn->sec, dest_off); =20 - if (add_call_dest(file, insn, dest, false)) - return -1; + if (add_call_dest(file, insn, dest, false)) + return -1; =20 - if (func && func->ignore) - continue; + if (func && func->ignore) + return 0; =20 - if (!insn_call_dest(insn)) { - ERROR_INSN(insn, "unannotated intra-function call"); - return -1; - } + if (!insn_call_dest(insn)) { + ERROR_INSN(insn, "unannotated intra-function call"); + return -1; + } =20 - if (func && !is_func_sym(insn_call_dest(insn))) { - ERROR_INSN(insn, "unsupported call to non-function"); - return -1; - } + if (func && !is_func_sym(insn_call_dest(insn))) { + ERROR_INSN(insn, "unsupported call to non-function"); + return -1; + } =20 - } else if (is_sec_sym(reloc->sym)) { - dest_off =3D arch_insn_adjusted_addend(insn, reloc); - dest =3D find_call_destination(reloc->sym->sec, dest_off); - if (!dest) { - ERROR_INSN(insn, "can't find call dest symbol at %s+0x%lx", - reloc->sym->sec->name, dest_off); - return -1; - } + } else if (is_sec_sym(reloc->sym)) { + dest_off =3D arch_insn_adjusted_addend(insn, reloc); + dest =3D find_call_destination(reloc->sym->sec, dest_off); + if (!dest) { + ERROR_INSN(insn, "can't find call dest symbol at %s+0x%lx", + reloc->sym->sec->name, dest_off); + return -1; + } =20 - if (add_call_dest(file, insn, dest, false)) - return -1; + if (add_call_dest(file, insn, dest, false)) + return -1; =20 - } else if (reloc->sym->retpoline_thunk) { - if (add_retpoline_call(file, insn)) - return -1; + } else if (reloc->sym->retpoline_thunk) { + if (add_retpoline_call(file, insn)) + return -1; =20 - } else { - if (add_call_dest(file, insn, reloc->sym, false)) - return -1; - } + } else { + if (add_call_dest(file, insn, reloc->sym, false)) + return -1; + } + + return 0; +} + +static int add_call_destinations_range(struct objtool_file *file, + struct insn_range *range) +{ + struct instruction *insn; + + range_for_each_insn(file, range, insn) { + if (add_call_destination(file, insn)) + return -1; + } + + return 0; +} + +/* Serial: annotating a call site rewrites instructions the dead end walks= read. */ +static int add_call_destinations(struct objtool_file *file) +{ + unsigned int i; + + for (i =3D 0; i < nr_decode_ranges; i++) { + if (add_call_destinations_range(file, &decode_ranges[i])) + return -1; } =20 return 0; @@ -2732,6 +3083,8 @@ int decode_file(struct objtool_file *file) if (read_annotate(file, __annotate_late)) return -1; =20 + free_decode_ranges(); + return 0; } =20 diff --git a/tools/objtool/elf.c b/tools/objtool/elf.c index bd463b5df38c..308877df6ad6 100644 --- a/tools/objtool/elf.c +++ b/tools/objtool/elf.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include @@ -475,6 +476,7 @@ static int reloc_cache_add(struct section *rsec, unsign= ed int pos) } =20 static unsigned long num_relocs_sorted; +static pthread_mutex_t reloc_cache_lock =3D PTHREAD_MUTEX_INITIALIZER; =20 static int init_reloc_cache(struct section *rsec) { @@ -514,8 +516,17 @@ struct reloc *find_reloc_by_dest_range(const struct el= f *elf, struct section *se if (!rsec) return NULL; =20 - if (!rsec->sorted && init_reloc_cache(rsec)) - exit(1); + /* Decoding looks up from several threads, rebuild an index once. */ + if (!rsec->sorted) { + int ret =3D 0; + + pthread_mutex_lock(&reloc_cache_lock); + if (!rsec->sorted) + ret =3D init_reloc_cache(rsec); + pthread_mutex_unlock(&reloc_cache_lock); + if (ret) + exit(1); + } =20 cache_idx =3D reloc_cache_index(offset); if (cache_idx >=3D rsec->nr_cache_windows) diff --git a/tools/objtool/objtool.c b/tools/objtool/objtool.c index 71e048f8582a..f472dfe7a773 100644 --- a/tools/objtool/objtool.c +++ b/tools/objtool/objtool.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -41,6 +42,8 @@ struct objtool_file *objtool_open_read(const char *filena= me) return &file; } =20 +static pthread_mutex_t pv_ops_lock =3D PTHREAD_MUTEX_INITIALIZER; + int objtool_pv_add(struct objtool_file *f, int idx, struct symbol *func) { if (!opts.noinstr) @@ -59,12 +62,15 @@ int objtool_pv_add(struct objtool_file *f, int idx, str= uct symbol *func) !strcmp(func->name, "_paravirt_ident_64")) return 0; =20 + pthread_mutex_lock(&pv_ops_lock); + /* already added this function */ - if (!list_empty(&func->pv_target)) - return 0; + if (list_empty(&func->pv_target)) { + list_add(&func->pv_target, &f->pv_ops[idx].targets); + f->pv_ops[idx].clean =3D false; + } =20 - list_add(&func->pv_target, &f->pv_ops[idx].targets); - f->pv_ops[idx].clean =3D false; + pthread_mutex_unlock(&pv_ops_lock); return 0; } =20 --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6BE83E51CB; Wed, 23 Sep 2026 17:22:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184157; cv=none; b=pPs3HGiKeZgnEh5UqZm2cnyVypwOhrJ+5ADcmte0H4O7KvintFWyzCJZ4r65L/VgnBRZgZEOUlQBZMTeS6AU/pZBbrhe8mx3WV64A5NIJe74gHq35x6YDKjMLvNSEMlVoQY0Wl3B06vbKfWu8/QD/0U4xkLGr4WcCogKT75YA2s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184157; c=relaxed/simple; bh=kx145CbsbOZNlrJo3YNnQ6X8zzclemP0tqvdTYAGrNw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=t2wj613WVcZ27NgEEt/Vrf9sgEzxLW2H4dHnQVgf/4Wn9hZR0m28uFJcXFrDu8VqID435N+xmDVSH25sXjgmjzSC4mgsp9j7f8czTg+axODPzuIc2qsHWFKZ8ij5MuLDTIugd9eVHOtPmXth3kkBjGVD/7wcO39d61dSKNHFRo8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CW3sRhYG; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CW3sRhYG" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E59621F000FF; Wed, 23 Sep 2026 17:22:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184155; bh=RpyglNkLvgtO+g1zDJNpkY3JopuChXGi3ttcKAuST+8=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=CW3sRhYGfPGHFUr2248WZC9Hf230RyEjEzZbWKjU10wEURx4jBU49P/EFHuafJBm+ OhE4O8riLvUVZKfj4H/BTTHlZzJXwTyytVTgwGIT0k1AJydF/bYU9STRDiFSRDNFBB eRgu/eySu3QFdveEJjwRKEwWO4hyeDY1mumbYmXPEJvmSbke1OrkvTcQnvBAiPLc3y YZgouDBBvQYnRYKk1KGsaCzYvtSWubA0PVabCoAPotC2a41sy+iXsOMwez5xJf7Uu+ avQKUwuhymuDeaGIWkBiUBI62/GgIEWU2vBBJuAZWr1gPu5RfuCY4aNh0ACOFZxBOi mHsJbxX3+BTAg== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:09 +0100 Subject: [PATCH v4 20/22] rust: make exports.o depend on the headers generated for it Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-20-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1286; i=ljs@kernel.org; h=from:subject:message-id; bh=kx145CbsbOZNlrJo3YNnQ6X8zzclemP0tqvdTYAGrNw=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF7dpf38yyJDZYecy4KWgWrRi7NmPFe7ENX/WW0hx 8TavfoyHaUsDGJcDLJiiizPv4jvDxIJm9d5wd8NZg4rE8gQBi5OAZiI0RtGhhuHD+28qdz11fP9 s0evqt3jWD5W/4r/U/FSRNopgLHaNZWRYVrvjjMvdq+d25Z0MPWC1yelO+89WkNXcjr8vFpf37l 8DRMA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 rust/exports.c includes the exports_*_generated.h listing the symbols of the core, bindings, kernel and helpers objects, but this is not expressed in its Makefile. This worked previously because the headers are always-y targets, and make prepare built all of the rust crates first, which is an implicit dependency. Fix this as it is a dependency of the subsequent commit. Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- rust/Makefile | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/rust/Makefile b/rust/Makefile index da1a7409d984..74d21ae6f383 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -728,6 +728,11 @@ endef $(obj)/helpers/helpers.o: $(src)/helpers/helpers.c $(recordmcount_source) = FORCE +$(call if_changed_rule,rust_cc_library) =20 +# The exported symbol lists are generated from the crates' objects. +$(obj)/exports.o: $(obj)/exports_core_generated.h \ + $(obj)/exports_bindings_generated.h $(obj)/exports_kernel_generated.h \ + $(if $(CONFIG_RUST_INLINE_HELPERS),,$(obj)/exports_helpers_generated.h) + # Disable symbol versioning for exports.o to avoid conflicts with the actu= al # symbol versions generated from Rust objects. $(obj)/exports.o: private skip_gendwarfksyms =3D 1 --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AFD043EFFD2; Wed, 23 Sep 2026 17:22:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184170; cv=none; b=JUe8u7cI9Yx99Hb78rCdvyFB1rXf+M6q+l69WxIy2/9MopM/x1uuSo46pTHtq2X0Y0irVaygsuHAIVAAei4g66qsmslBsFT8xSdGrvV4kMj6Sn6Uv9QzX4y5slnQIRCuG57sNp3n3JswFGTY09G6xEgX9Zld8TSobUjEFp7Fr2s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184170; c=relaxed/simple; bh=eSJUSbRq8SVTq7pZLzFoTNHAy5RNavHjGYCUuHNLs38=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=e9hUClRtde4VGT874lf7McnY3OPCHRpsOWmoCsA12E14ghGg737/v6zzVDQnco+CGapIwD2rEIS5uJWfU7MAZxtz9hmlZwuCDeinmb0kQOF7sGZn3JFdhZs7Vr5J1ukVgBrpISVLOmsUAbtL+FrXuD4Cgt+rDTWpKJb7n1IHP/I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=E5NRdg8r; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="E5NRdg8r" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 516C21F00893; Wed, 23 Sep 2026 17:22:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184168; bh=lOhWdRW6MD6ARrbi+/HwwF29XbYzAt8l7ZnfPtNU22Y=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=E5NRdg8rH9T4KS/4qlu8FH6E6Iin9emjykQ5Uwoc5rma/TV18TTEOgSPWn9mOu6ej hvagMt2mhLJep2A2dSXpkX6pElMBAsvBOMeGG2cFdSwWe6OfoB74yPswhniNiGAQTG O7NH8OSbhBbcdn9C+F9LptMQLq/hNDxUBOAi/z4c4lUFEzgCURH8ZVt667a7AJSbWz LJ23rJm922GqDN8a+ruzM8CIW8XzJSHPDs6Zvm1llBKtM1jHdnEnhIBGGytcyMAWGn ClrIxUC7hvvBBq9CEUkfoDAHHQv8p+8aGgg5uNaguRI+sj/q4xJPZum4VrT+DC6yI5 82Q/LyAEqOFvQ== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:10 +0100 Subject: [PATCH v4 21/22] kbuild: build rust crates in parallel with the rest of the build Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-21-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4626; i=ljs@kernel.org; h=from:subject:message-id; bh=eSJUSbRq8SVTq7pZLzFoTNHAy5RNavHjGYCUuHNLs38=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF59+z6O55XgXUERwXCzwri25fGvQwJf2d/X5qjc4 Juy+cayjlIWBjEuBlkxRZbnX8T3B4mEzeu84O8GM4eVCWQIAxenAEzkXzjDf7ey48GGYevfC4XM XapcvmBnlLz3bv09LMwv9syuehOyL4OR4fDaNaJlG5lWSt97Pk1W2VlESPeJfNP5Yva4Q4utpG1 fMQIA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 When CONFIG_RUST is specified make prepare builds all of the rust components before descending into the tree. This means nothing can be done until all of these are compiled, resulting in a stall at the start of every clean CONFIG_RUST build. This also implicitly slows down every rust-capable LLVM allmodconfig build as this enables the CONFIG_RUST option. Fix this by allowing rust crates to be built alongside the C code. Remove the rust build from the make prepare step, then establish a dependency between rust code located elsewhere in the tree upon rust components contained in rust/. Do this by establishing a top-level list of directories containing rust code, KBUILD_RUST_DIRS, upon which the dependency is expressed. Finally, ensure that no rust code is located elsewhere and fail the build if any is found to ensure that nothing is missed in future. Care is taken to ensure the rust/ dependency is also established for any module being built to account for external rust modules which rely upon it, and that only one rust/ build will be performed at any one time. External rust modules builds are unaffected - make modules_prepare ... (where ... are other goals) will build rust/ first. The rustdoc and rusttest goals invoke rust/ directly rather than via Kbuild, so build rust/ first there too. Every Rust and C object was confirmed to be byte-for-byte identical after this change. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig+RUST, clean 36.5s 32.6s -3.9s (-11%) x86 allmodconfig, clean 297.0s 278.3s -18.7s (-6%) Assisted-by: LLM Signed-off-by: Lorenzo Stoakes (ARM) --- Kbuild | 5 +++++ Makefile | 17 ++++++++++++++++- scripts/Makefile.build | 5 ++++- 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/Kbuild b/Kbuild index a6a0192dea08..9670556059dd 100644 --- a/Kbuild +++ b/Kbuild @@ -115,3 +115,8 @@ obj-$(CONFIG_NET) +=3D net/ obj-y +=3D virt/ obj-y +=3D $(ARCH_DRIVERS) obj-$(CONFIG_DRM_HEADER_TEST) +=3D include/ + +# Rust code elsewhere in the tree depends upon rust/. +ifdef CONFIG_RUST +$(KBUILD_RUST_DIRS): | rust +endif diff --git a/Makefile b/Makefile index 7745bb36e622..f0502b02408f 100644 --- a/Makefile +++ b/Makefile @@ -1429,11 +1429,14 @@ prepare0: archprepare $(Q)$(MAKE) $(build)=3D. prepare $(Q)$(MAKE) $(build)=3Dscripts/mod =20 +ifdef CONFIG_RUST +export KBUILD_RUST_DIRS :=3D drivers lib mm samples +endif + # All the preparing.. prepare: prepare0 ifdef CONFIG_RUST +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh - $(Q)$(MAKE) $(build)=3Drust endif =20 PHONY +=3D remove-stale-files @@ -1760,6 +1763,14 @@ modules: modules_prepare # Target to prepare building external modules modules_prepare: prepare $(Q)$(MAKE) $(build)=3Dscripts scripts/module.lds +ifdef CONFIG_RUST +# Avoid two instances of rust/ being built at any one time. +ifneq ($(filter modules_prepare,$(MAKECMDGOALS)),) +ifeq ($(filter all vmlinux modules %Image% dtbs rustdoc rusttest,$(MAKECMD= GOALS)),) + $(Q)$(MAKE) $(build)=3Drust +endif +endif +endif =20 endif # CONFIG_MODULES =20 @@ -2025,13 +2036,17 @@ rustavailable: # Documentation target # # Using the singular to avoid running afoul of `no-dot-config-targets`. +# +# rust/ is built alongside the rest of the tree, so must be built first he= re. PHONY +=3D rustdoc rustdoc: prepare + $(Q)$(MAKE) $(build)=3Drust $(Q)$(MAKE) $(build)=3Drust $@ =20 # Testing target PHONY +=3D rusttest rusttest: prepare + $(Q)$(MAKE) $(build)=3Drust $(Q)$(MAKE) $(build)=3Drust $@ =20 # Formatting targets diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 819cc1f750e8..a809aad060e7 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -360,8 +360,11 @@ quiet_cmd_rustc_o_rs =3D $(RUSTC_OR_CLIPPY_QUIET) $(qu= iet_modtag) $@ $(cmd_ld_single)) \ $(cmd_objtool) =20 +rust-dir-ok =3D $(or $(KBUILD_EXTMOD),$(filter rust $(KBUILD_RUST_DIRS),$(= firstword $(subst /, ,$@)))) + +# Ensure that any rust code located elsewhere from rust/ is listed in KBUI= LD_RUST_DIRS. define rule_rustc_o_rs - $(call cmd_and_fixdep,rustc_o_rs) + $(if $(rust-dir-ok),,$(error $@: Rust code in a directory not listed in K= BUILD_RUST_DIRS, see the top Makefile))$(call cmd_and_fixdep,rustc_o_rs) $(call cmd,gen_objtooldep) endef =20 --=20 2.55.0 From nobody Thu Sep 24 13:39:02 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 511A33E765A; Wed, 23 Sep 2026 17:23:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184182; cv=none; b=pRdZ0QyEsJ55ZnVy0gfHVqEq3E6LQgEmkYtjE0hlCxJ6a1Li+kBzZQ+ZzrBvE7MMDA++Ki6gEVKrX38CSVdFphXrzdyDW0dmA+yG7LbLvbPXDKR4rC3AUMAjUWPLEsYFVAAY//SFblftV1sfelhvYqP8+pb1UKtYuURvCY2ZFaU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790184182; c=relaxed/simple; bh=1zmsc9ZHF8+CoyT22YKQnfv42K05Raj82AafGLUVoSU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=EWkh55CVFzRpWePuAGod207nqTMkTL84zB8yvmcEBDP64zArc4YD58xLx0n13lfIhLSG8OGlL4tbBUpsjwzRr3LGm0HEOmUBo+CkMrJEXyIxRtS0xlVuUnPOTfdVm5QUZbCFBXEVUZLsUpwMxQUBS6FQvQkwCAO2gfET8WykR38= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=HjH3hQsw; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="HjH3hQsw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B6C4D1F000FF; Wed, 23 Sep 2026 17:22:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1790184181; bh=7TYGk1JizHGrlBrKc7yi6wYeMXYeC/fB2mICtyHAjKk=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=HjH3hQswPOEhNyh+0EJsK+8lvQLgcr3wY90IiS4ycWSZkKP4z3Hz+F195NNzsn+5H DOCkRcOHx0XQmfWCxL0x2y7voJVSjw3iitplwzqupe10mATAJ94nar2ZrZLegTO4JS ETtXjajIE2KLsZcE7sBFgxZBdNXyA1Ui0aXWQOwgkE1w/tZITBr7PoBhDuwFP74eUc WozytBVTBGoOUfLjCdhu0GJ4gkB/LxwycdUHuH4s++EAWyVpU4ahOzUm5ZmQo5G+RK 1kvSqlb7dOOveB0AuROR0K/Kr/FbInyTO+eq9TdAUubQQtYWoivr4mLpirXiuhGEUN uOG9C8cUl0w/A== From: "Lorenzo Stoakes (ARM)" Date: Wed, 23 Sep 2026 18:18:11 +0100 Subject: [PATCH v4 22/22] kbuild: compress the kernel with pigz if available Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-build-speedup-v4-22-73128809a4a4@kernel.org> References: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> In-Reply-To: <20260923-build-speedup-v4-0-73128809a4a4@kernel.org> To: Linus Torvalds , Nathan Chancellor , Nicolas Schier , Nick Desaulniers , Bill Wendling , Justin Stitt , Masahiro Yamada , Alexey Gladkov , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , Ard Biesheuvel , Ilias Apalodimas , Josh Poimboeuf , Peter Zijlstra , Miguel Ojeda , Boqun Feng , Gary Guo , =?utf-8?q?Bj=C3=B6rn_Roy_Baron?= , Benno Lossin , Andreas Hindborg , Alice Ryhl , Trevor Gross , Danilo Krummrich , Daniel Almeida , Tamir Duberstein , Alexandre Courbot , =?utf-8?q?Onur_=C3=96zkan?= , Jonathan Corbet , Randy Dunlap , Kees Cook , "Gustavo A. R. Silva" Cc: linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, llvm@lists.linux.dev, linux-riscv@lists.infradead.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, rust-for-linux@vger.kernel.org, linux-doc@vger.kernel.org, Jens Axboe , linux-hardening@vger.kernel.org, Petr Pavlu , "Lorenzo Stoakes (ARM)" , Manuel Ebner X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=10892; i=ljs@kernel.org; h=from:subject:message-id; bh=1zmsc9ZHF8+CoyT22YKQnfv42K05Raj82AafGLUVoSU=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLK2cF5V+qM6cZ+XtAW/Br/evglRvqvu6e0zclxt91O6Y amPfU9PRykLgxgXg6yYIsvzL+L7g0TC5nVe8HeDmcPKBDKEgYtTACbyi5vhv+d/B+Uu/+nK57e4 nd2/yvm3zewtAVnVk7TF4gzeHOM+JcnwP/6o3UdeXaZzG7hDLTxuVGcyNE/nLWEp03YunaVyTfg kPwA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 The gzip step of a kernel build is very lengthy, especially for larger builds such as allmodconfig. gzip itself cannot be run in parallel, however an alternative tool exists that can, providing the same feature set as gzip itself - pigz - which works as a drop-in replacement. On a 128-core Threadripper, gzip -9 of a 36 MiB x86-64 vmlinux.bin takes 1.6s, and with pigz it takes 0.09s, so the performance increase is significant. Therefore introduce the KPGZIP build variable which specifies the compressor to use for the kernel image and default it to pigz if available, or KGZIP otherwise (which itself defaults to gzip). Modules, packaging and the embedded kernel configuration (CONFIG_IKCONFIG) continue to use KGZIP which defaults to a single-threaded gzip instance. Module compression is run in parallel by make, so pigz offers no benefit there, packaging sits outside the build and the embedded configuration is too small to matter. pigz is executed using scripts/jobserver-exec, which reserves the job slots make has free - necessary, as otherwise pigz would just grab unbounded CPUs. The kernel image compression step is always performed at the end of the build so should typically have all job slots available. pigz always provides byte-for-byte identical output across runs regardless of thread count, but pigz and gzip are not guaranteed to generate the same binary output, so reproducible builds must use the same tools consistently. Also update Documentation/kbuild/kbuild.rst to describe KPGZIP and Documentation/process/changes.rst to list the optional pigz dependency. While there, add entries for the other compressor programs. Finally, update scripts/ver_linux to include the pigz version there. Every x86 build ends with the compression of vmlinux.bin, 36MB for defconfig and over 200MB for allmodconfig, no-op builds are unchanged. Whole build, 128-thread Threadripper 9980X, best of N runs: before after delta ------------------------------- x86 defconfig, touch mm/vma.c, gcc 7.7s 6.0s -1.7s (-22%) x86 defconfig, touch mm/vma.c, clang 6.9s 5.4s -1.5s (-21%) x86 defconfig, clean, gcc 28.2s 26.3s -1.8s (-6%) x86 defconfig, clean, clang 28.7s 27.2s -1.5s (-5%) x86 allmodconfig, touch mm/vma.c, gcc 23.7s 15.4s -8.3s (-35%) x86 allmodconfig, touch mm/vma.c, clang 22.5s 15.6s -7.0s (-31%) x86 allmodconfig, clean, gcc 294.1s 278.6s -15.5s (-5%) x86 allmodconfig, clean, clang 283.9s 266.8s -17.1s (-6%) Suggested-by: Kees Cook Link: https://zlib.net/pigz/ Assisted-by: LLM Reviewed-by: Manuel Ebner Reviewed-by: Kees Cook Signed-off-by: Lorenzo Stoakes (ARM) --- Documentation/kbuild/kbuild.rst | 17 +++++++++++++++++ Documentation/process/changes.rst | 8 ++++++++ Makefile | 5 ++++- kernel/Makefile | 6 +++++- scripts/Makefile.lib | 4 ++-- scripts/jobserver-exec | 3 ++- scripts/ver_linux | 1 + tools/lib/python/jobserver.py | 13 +++++++++++++ 8 files changed, 52 insertions(+), 5 deletions(-) diff --git a/Documentation/kbuild/kbuild.rst b/Documentation/kbuild/kbuild.= rst index 5a9013bacfb7..e71a692a559a 100644 --- a/Documentation/kbuild/kbuild.rst +++ b/Documentation/kbuild/kbuild.rst @@ -110,6 +110,23 @@ HOSTLDLIBS ---------- Additional libraries to link against when building host programs. =20 +KGZIP +----- +The gzip program used for compressed modules, packaging and the embedded +kernel configuration. Defaults to gzip. + +KPGZIP +------ +The gzip program used for the compressed kernel image. Defaults to pigz (a +parallel implementation of gzip), run under scripts/jobserver-exec so that= it +uses the job slots make has free, if pigz is installed, otherwise to +``KGZIP``. Set ``KPGZIP=3D`` on the make command line to use another progr= am. + +KBZIP2, KLZOP, LZMA, LZ4, XZ, ZSTD +---------------------------------- +The compressor programs for the other formats. Each defaults to the program +of the same name. + .. _userkbuildflags: =20 USERCFLAGS diff --git a/Documentation/process/changes.rst b/Documentation/process/chan= ges.rst index 0aa232b117b5..28f2ca63318e 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -55,6 +55,7 @@ nfs-utils 1.0.5 showmount --versi= on openssl & libcrypto 1.0.0 openssl version pahole 1.26 pahole --version pcmciautils 004 pccardctl -V +pigz (optional) 2.4 pigz --version PPP 2.4.0 pppd --version procps 3.2.0 ps --version Python 3.9.x python3 --version @@ -206,6 +207,13 @@ GNU AWK GNU AWK is needed if you want kernel builds to generate address range data= for builtin modules (CONFIG_BUILTIN_MODULE_RANGES). =20 +pigz (optional) +--------------- + +pigz is a parallel implementation of gzip. If it is installed the compress= ed +kernel image is produced with it rather than with gzip, see ``KPGZIP`` in +Documentation/kbuild/kbuild.rst. + System utilities **************** =20 diff --git a/Makefile b/Makefile index f0502b02408f..ba71483f7f14 100644 --- a/Makefile +++ b/Makefile @@ -568,6 +568,9 @@ LZMA =3D lzma LZ4 =3D lz4 XZ =3D xz ZSTD =3D zstd +# The kernel image is compressed with pigz, on the job slots make has free, +# if it is installed. Everything else uses KGZIP. +KPGZIP :=3D $(if $(shell command -v pigz 2>/dev/null),$(PYTHON3) $(abs_sr= ctree)/scripts/jobserver-exec pigz -p %PARALLELISM%,$(KGZIP)) TAR =3D tar =20 CHECKFLAGS :=3D -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ \ @@ -648,7 +651,7 @@ export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUST= C_OR_CLIPPY BINDGEN LLVM_ export HOSTRUSTC KBUILD_HOSTRUSTFLAGS export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX Y= ACC AWK INSTALLKERNEL export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX -export KGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD TAR +export KGZIP KPGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD TAR export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS KBUILD_PRO= CMACROLDFLAGS LDFLAGS_MODULE export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS =20 diff --git a/kernel/Makefile b/kernel/Makefile index 1e1a31673577..c64c82c96b40 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -153,9 +153,13 @@ obj-$(CONFIG_SCF_TORTURE_TEST) +=3D scftorture.o =20 $(obj)/configs.o: $(obj)/config_data.gz =20 +# .config is small, so it is compressed with gzip rather than the image co= mpressor. +quiet_cmd_gzip_config =3D GZIP $@ + cmd_gzip_config =3D cat $(real-prereqs) | $(KGZIP) -n -f -9 > $@ + targets +=3D config_data config_data.gz $(obj)/config_data.gz: $(obj)/config_data FORCE - $(call if_changed,gzip) + $(call if_changed,gzip_config) =20 filechk_cat =3D cat $< =20 diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib index 2f447bc25e7b..8936925d4d50 100644 --- a/scripts/Makefile.lib +++ b/scripts/Makefile.lib @@ -332,11 +332,11 @@ quiet_cmd_ar =3D AR $@ quiet_cmd_objcopy =3D OBJCOPY $@ cmd_objcopy =3D $(OBJCOPY) $(OBJCOPYFLAGS) $(OBJCOPYFLAGS_$(@F)) $< $@ =20 -# Gzip +# Gzip, for the kernel image # ------------------------------------------------------------------------= --- =20 quiet_cmd_gzip =3D GZIP $@ - cmd_gzip =3D cat $(real-prereqs) | $(KGZIP) -n -f -9 > $@ + cmd_gzip =3D cat $(real-prereqs) | $(KPGZIP) -n -f -9 > $@ =20 # Bzip2 # ------------------------------------------------------------------------= --- diff --git a/scripts/jobserver-exec b/scripts/jobserver-exec index 21b319e6c9a5..89119acb1b9e 100755 --- a/scripts/jobserver-exec +++ b/scripts/jobserver-exec @@ -4,7 +4,8 @@ """ Determines how many parallel tasks "make" is expecting, as it is not exposed via any special variables, reserves them all, runs a subprocess -with PARALLELISM environment variable set, and releases the jobs back agai= n. +with PARALLELISM environment variable set and any %PARALLELISM% in its +arguments replaced by the count, and releases the jobs back again. =20 See: https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.htm= l#POSIX-Jobserver diff --git a/scripts/ver_linux b/scripts/ver_linux index 00bdaf30d590..e06d7c650710 100755 --- a/scripts/ver_linux +++ b/scripts/ver_linux @@ -57,6 +57,7 @@ BEGIN { printversion("pahole", version("pahole --version")) printversion("Pcmcia-cs", version("cardmgr -V")) printversion("pcmciautils", version("pccardctl -V")) + printversion("pigz", version("pigz --version")) printversion("PPP", version("pppd --version")) printversion("procps", version("ps --version")) printversion("Python", version("python3 -V")) diff --git a/tools/lib/python/jobserver.py b/tools/lib/python/jobserver.py index 0b1ffdf9f7a3..7e94635bf7d9 100755 --- a/tools/lib/python/jobserver.py +++ b/tools/lib/python/jobserver.py @@ -29,6 +29,10 @@ $claim child to do the actual work. The end goal here is to keep the total number of build tasks under the limit established by the initial ``make -j$n_proc`` call. =20 +A command that takes its thread count on the command line rather than from +the environment can use the ``%PARALLELISM%`` token in its arguments, which +is replaced by the count, or by 1 when there is no jobserver. + See: https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.htm= l#POSIX-Jobserver """ @@ -38,6 +42,8 @@ import os import subprocess import sys =20 +PARALLELISM_TOKEN =3D "%PARALLELISM%" + def warn(text, *args): print(f'WARNING: {text}', *args, file =3D sys.stderr) =20 @@ -182,6 +188,9 @@ class JobserverExec: Run a command setting PARALLELISM env variable to the number of available job slots (claim) + 1, e.g. it will reserve claim slots to do the actual build work, plus one to monitor its children. + + Any %PARALLELISM% in the command's arguments is replaced by the sa= me + number, or by 1 when there is no jobserver. """ self.open() # Ensure that self.claim is set =20 @@ -192,4 +201,8 @@ class JobserverExec: if self.claim: os.environ["PARALLELISM"] =3D str(self.claim) =20 + if not isinstance(cmd, str): + parallelism =3D str(self.claim or 1) + cmd =3D [arg.replace(PARALLELISM_TOKEN, parallelism) for arg i= n cmd] + return subprocess.call(cmd, *args, **pwargs) --=20 2.55.0