From nobody Thu Feb 12 04:51:37 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF1BAC77B60 for ; Fri, 28 Apr 2023 09:58:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345947AbjD1J6E (ORCPT ); Fri, 28 Apr 2023 05:58:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36402 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345863AbjD1J5o (ORCPT ); Fri, 28 Apr 2023 05:57:44 -0400 Received: from out187-17.us.a.mail.aliyun.com (out187-17.us.a.mail.aliyun.com [47.90.187.17]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4077176BC for ; Fri, 28 Apr 2023 02:56:51 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R841e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018047207;MF=houwenlong.hwl@antgroup.com;NM=1;PH=DS;RN=16;SR=0;TI=SMTPD_---.STCEPcc_1682675575; Received: from localhost(mailfrom:houwenlong.hwl@antgroup.com fp:SMTPD_---.STCEPcc_1682675575) by smtp.aliyun-inc.com; Fri, 28 Apr 2023 17:52:56 +0800 From: "Hou Wenlong" To: linux-kernel@vger.kernel.org Cc: "Thomas Garnier" , "Lai Jiangshan" , "Kees Cook" , "Hou Wenlong" , "Thomas Gleixner" , "Ingo Molnar" , "Borislav Petkov" , "Dave Hansen" , , "H. Peter Anvin" , "David Woodhouse" , "Peter Zijlstra" , "Brian Gerst" , "Josh Poimboeuf" , "Sami Tolvanen" Subject: [PATCH RFC 20/43] x86/percpu: Adapt percpu references relocation for PIE support Date: Fri, 28 Apr 2023 17:51:00 +0800 Message-Id: <62d7e9e73467b711351a84ebce99372d3dccaa73.1682673543.git.houwenlong.hwl@antgroup.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The original design of percpu references relocation only handles relative references and ignores absolute references. Because percpu variable had already been relative based on segment. And .percpu ELF section has a virtual address of zero and absolute references can be kept if kaslr is enabled. As for a little relative references, they needs to be relocated by negative offset. However, it is not compatible with PIE, because almost all percpu references would be RIP-relative. But RIP-relative addressing could only support -2G ~ +2G. In order to move kernel address below top 2G, percpu relative references wouldn't be relocated, instead, percpu base could be adjusted. As for absolute references, they would be relocated like normal variable. After that, percpu references in .altinstr_replacement section couldn't work right, because no fixups are applied for percpu references in apply_alternatives(). However, it could be caught by objtool. Currently, only call depth tracking uses it, so disable it if X86_PIE is enabled. Suggested-by: Lai Jiangshan Signed-off-by: Hou Wenlong Cc: Thomas Garnier Cc: Kees Cook --- arch/x86/Kconfig | 2 +- arch/x86/kernel/head_64.S | 10 ++++++++++ arch/x86/tools/relocs.c | 17 ++++++++++++++--- 3 files changed, 25 insertions(+), 4 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index b26941ef50ee..715f0734d065 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2635,7 +2635,7 @@ config CPU_UNRET_ENTRY =20 config CALL_DEPTH_TRACKING bool "Mitigate RSB underflow with call depth tracking" - depends on CPU_SUP_INTEL && HAVE_CALL_THUNKS + depends on CPU_SUP_INTEL && HAVE_CALL_THUNKS && !X86_PIE select HAVE_DYNAMIC_FTRACE_NO_PATCHABLE select CALL_THUNKS default y diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 1eed50b7d1ac..94c5defec8cc 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -72,6 +72,11 @@ SYM_CODE_START_NOALIGN(startup_64) leaq INIT_PER_CPU_VAR(fixed_percpu_data)(%rip), %rdx #elif defined(CONFIG_SMP) movabs $__per_cpu_load, %rdx +#ifdef CONFIG_X86_PIE + movabs $__per_cpu_start, %rax + subq %rax, %rdx + movq %rdx, __per_cpu_offset(%rip) +#endif #else xorl %edx, %edx #endif @@ -79,6 +84,11 @@ SYM_CODE_START_NOALIGN(startup_64) shrq $32, %rdx wrmsr =20 +#if defined(CONFIG_X86_PIE) && defined(CONFIG_SMP) + movq __per_cpu_offset(%rip), %rdx + movq %rdx, PER_CPU_VAR(this_cpu_off) +#endif + pushq %rsi call startup_64_setup_env popq %rsi diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index 2925074b9a58..038e9c12fad3 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -848,6 +848,7 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel= , ElfW(Sym) *sym, =20 case R_X86_64_PC32: case R_X86_64_PLT32: +#ifndef CONFIG_X86_PIE /* * PC relative relocations don't need to be adjusted unless * referencing a percpu symbol. @@ -856,6 +857,7 @@ static int do_reloc64(struct section *sec, Elf_Rel *rel= , ElfW(Sym) *sym, */ if (is_percpu_sym(sym, symname)) add_reloc(&relocs32neg, offset); +#endif break; =20 case R_X86_64_PC64: @@ -871,10 +873,18 @@ static int do_reloc64(struct section *sec, Elf_Rel *r= el, ElfW(Sym) *sym, case R_X86_64_32S: case R_X86_64_64: /* - * References to the percpu area don't need to be adjusted. + * References to the percpu area don't need to be adjusted when + * CONFIG_X86_PIE is not enabled. */ - if (is_percpu_sym(sym, symname)) + if (is_percpu_sym(sym, symname)) { +#if CONFIG_X86_PIE + if (r_type !=3D R_X86_64_64) + die("Invalid absolute reference against per-CPU symbol %s\n", + symname); + add_reloc(&relocs64, offset); +#endif break; + } =20 if (shn_abs) { /* @@ -1044,7 +1054,8 @@ static int cmp_relocs(const void *va, const void *vb) =20 static void sort_relocs(struct relocs *r) { - qsort(r->offset, r->count, sizeof(r->offset[0]), cmp_relocs); + if (r->count) + qsort(r->offset, r->count, sizeof(r->offset[0]), cmp_relocs); } =20 static int write32(uint32_t v, FILE *f) --=20 2.31.1