From nobody Sat Feb 7 10:08:17 2026 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE7C21B6D0C for ; Mon, 25 Nov 2024 17:04:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554274; cv=none; b=q6ifjaG7S8bs4TzuPxouaMBmPaedd6tbhdo5oS4sy+nfmymuRBmZ0wPMzHnZIXFKkAnQw1HsDOiKG2rjUhbi8JcsWi497V36lXuuPDBDbiiVbE5gVqFnqAFaH3FnLsZz2Zpg6egIp0QKca2BexL7j3MdrcrxFFngjnrEvGqpO1E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554274; c=relaxed/simple; bh=7S/oLN/DEII9oL7EN3aZBI6qYSXJO9+f1T4AscFQ1gk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YeTZL1HS/YD5FzOq2yipN05Y3uDToud+yWrfKD7F4YPZ7nmAXl3PXJHGv0Mx2LA6vdTL2yQRLCnRU+ZyqUxaAFZWRnxRlWJez2Jl01KqlvuJKof3YQnkR4+d6UqdHkGJUeu+knCon0uYZaFHtU2Q8huqwbG9H/lndVP1E00qf3Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=RPy7pLFY; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RPy7pLFY" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-38242c5b4ffso2584638f8f.1 for ; Mon, 25 Nov 2024 09:04:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732554270; x=1733159070; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=W4AEya7qP9gbIMfOd2WdxGj+0SNfsKysCWPPlG8qhpw=; b=RPy7pLFY1Wpkz8s8yyosKTGoDsf7DPbZnA8BYtm6yqrNVhls6c09UPCSi4DrCg+APA 6rKllEyQPIlITGxKmswSGMIr4siNQkOJ3LGIp3S6lByZRZUercD/fEzwP1PCUkba2Js4 3ZylSCideIj2RHN0ybYPq9fmAivQ6sipGUCINDHyELXgvFXqMAIIkHGNTkZfzVXUCHNf 3Jy4ImpXvs96jRgAD/BCRwo3MdIFTFnK1/WxRxBMBxr1ZMLH37aLqy+n2fFmigOFZwpl 9/TLUjkj/r/7UcCDaSB/IWoM2cILHzmZTVL4aCrn3pvqLZwOHsdOzKtjDSEJcjx9WTaO uQwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732554270; x=1733159070; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=W4AEya7qP9gbIMfOd2WdxGj+0SNfsKysCWPPlG8qhpw=; b=u3Eu2KDZF+gyxfquIDKhLdvbCtZr6tL/J32614escyJK8DCzpc6DTgAVpygx484Czq DLhOjy4xqj2I85ENasBpWM6QnwH016/UMf0GdLq47EPYizUtbSnVDKQUYHfReUw/X4oy +bDSIkB2SFkGRw7RzHW1H4v9ConYcQ0PoAyrzBtzMPnpVf7L20cVqc0zIO9ZgEHTTCGW w8ZAiHs6Dhl0R/sEBe8JRUNh/pqkGW4+rPXtlmhcKBNoGwS6mCzeQMj48adTSbllB5+L jiPmyYyvIhnVnAye4FWdZhKwBqpXfBAx0YzyaCybP+JA9mRp2cjHAAMAlA9SRvJ2SIgJ 3gyw== X-Gm-Message-State: AOJu0YxOLC86SO2a3OPGQ2K2LKCr4zoEOt6vXz5hn2c7fLYdMwYNAfPi 8Em9D2HcAJJWVMdVkZ3Q64/2ev92yvoEEA6WTlnafy+kN8T78jo0ViNp7vvZ/aBziKLhCZvhxGO joXxwK99TDBguhrTghMoT8HqGbP4D2LtoghCejTl3MJKKjIVa0xX+bzAOep1TvyeOEl/mUHk62k 02nFht1i+FJ4loTCVg24SfyuM7Up2qEQ== X-Google-Smtp-Source: AGHT+IH+qO5ae1Hho1Lm/Fon9RV09yeJwfdIfXubNTP0EAwEzYAgMSd0otVbm0JAEWssA/T7qg2RLt5y X-Received: from wmsm10.prod.google.com ([2002:a05:600c:3b0a:b0:432:d7fd:fd75]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:64ac:0:b0:382:51bf:b0c6 with SMTP id ffacd0b85a97d-38260b863b3mr12016305f8f.28.1732554270160; Mon, 25 Nov 2024 09:04:30 -0800 (PST) Date: Mon, 25 Nov 2024 18:04:12 +0100 In-Reply-To: <20241125170411.1898410-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241125170411.1898410-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3819; i=ardb@kernel.org; h=from:subject; bh=mM9vuf5kkZl20az/QVO84EIMF/oOjchevyUP+y7pdwE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JId1lHa+T7seidz6sd8qyOlwc3/VdLmfdtmdzplzr58Un3 nw9f2NGRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjI3RMM/zT8WbOtvmVtXsQl 6rXh02vl9BrdhmazhNaPt49FSE9+MZGRoSuD5YHcn89Cob89XvPFd2UmvIhaNUvmvebrLYceF70 /wgEA X-Mailer: git-send-email 2.47.0.371.ga323438b13-goog Message-ID: <20241125170411.1898410-9-ardb+git@google.com> Subject: [PATCH v3 1/6] x86/sev: Avoid WARN()s and panic()s in early boot code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Kees Cook , Brian Gerst , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Using WARN() or panic() while executing from the early 1:1 mapping is unlikely to do anything useful: the string literals are passed using their kernel virtual addresses which are not even mapped yet. But even if they were, calling into the printk() machinery from the early 1:1 mapped code is not going to get very far. So drop the WARN()s entirely, and replace panic() with a deadloop. Link: https://lore.kernel.org/all/6904c198-9047-14bb-858e-38b531589379@amd.= com/T/#u Signed-off-by: Ard Biesheuvel --- arch/x86/coco/sev/core.c | 15 +++++---------- arch/x86/coco/sev/shared.c | 9 +++++---- 2 files changed, 10 insertions(+), 14 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index c5b0148b8c0a..499b41953e3c 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -777,15 +777,10 @@ early_set_pages_state(unsigned long vaddr, unsigned l= ong paddr, =20 val =3D sev_es_rd_ghcb_msr(); =20 - if (WARN(GHCB_RESP_CODE(val) !=3D GHCB_MSR_PSC_RESP, - "Wrong PSC response code: 0x%x\n", - (unsigned int)GHCB_RESP_CODE(val))) + if (GHCB_RESP_CODE(val) !=3D GHCB_MSR_PSC_RESP) goto e_term; =20 - if (WARN(GHCB_MSR_PSC_RESP_VAL(val), - "Failed to change page state to '%s' paddr 0x%lx error 0x%llx\n", - op =3D=3D SNP_PAGE_STATE_PRIVATE ? "private" : "shared", - paddr, GHCB_MSR_PSC_RESP_VAL(val))) + if (GHCB_MSR_PSC_RESP_VAL(val)) goto e_term; =20 /* Page validation must be performed after changing to private */ @@ -821,7 +816,7 @@ void __head early_snp_set_memory_private(unsigned long = vaddr, unsigned long padd early_set_pages_state(vaddr, paddr, npages, SNP_PAGE_STATE_PRIVATE); } =20 -void __init early_snp_set_memory_shared(unsigned long vaddr, unsigned long= paddr, +void __head early_snp_set_memory_shared(unsigned long vaddr, unsigned long= paddr, unsigned long npages) { /* @@ -2361,8 +2356,8 @@ static __head void svsm_setup(struct cc_blob_sev_info= *cc_info) call.rax =3D SVSM_CORE_CALL(SVSM_CORE_REMAP_CA); call.rcx =3D pa; ret =3D svsm_perform_call_protocol(&call); - if (ret) - panic("Can't remap the SVSM CA, ret=3D%d, rax_out=3D0x%llx\n", ret, call= .rax_out); + while (ret) + cpu_relax(); /* too early to panic */ =20 RIP_REL_REF(boot_svsm_caa) =3D (struct svsm_ca *)pa; RIP_REL_REF(boot_svsm_caa_pa) =3D pa; diff --git a/arch/x86/coco/sev/shared.c b/arch/x86/coco/sev/shared.c index 71de53194089..afb7ffc355fe 100644 --- a/arch/x86/coco/sev/shared.c +++ b/arch/x86/coco/sev/shared.c @@ -1243,7 +1243,7 @@ static void svsm_pval_terminate(struct svsm_pvalidate= _call *pc, int ret, u64 svs __pval_terminate(pfn, action, page_size, ret, svsm_ret); } =20 -static void svsm_pval_4k_page(unsigned long paddr, bool validate) +static void __head svsm_pval_4k_page(unsigned long paddr, bool validate) { struct svsm_pvalidate_call *pc; struct svsm_call call =3D {}; @@ -1275,12 +1275,13 @@ static void svsm_pval_4k_page(unsigned long paddr, = bool validate) =20 ret =3D svsm_perform_call_protocol(&call); if (ret) - svsm_pval_terminate(pc, ret, call.rax_out); + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); =20 native_local_irq_restore(flags); } =20 -static void pvalidate_4k_page(unsigned long vaddr, unsigned long paddr, bo= ol validate) +static void __head pvalidate_4k_page(unsigned long vaddr, unsigned long pa= ddr, + bool validate) { int ret; =20 @@ -1293,7 +1294,7 @@ static void pvalidate_4k_page(unsigned long vaddr, un= signed long paddr, bool val } else { ret =3D pvalidate(vaddr, RMP_PG_SIZE_4K, validate); if (ret) - __pval_terminate(PHYS_PFN(paddr), validate, RMP_PG_SIZE_4K, ret, 0); + sev_es_terminate(SEV_TERM_SET_LINUX, GHCB_TERM_PVALIDATE); } } =20 --=20 2.47.0.371.ga323438b13-goog From nobody Sat Feb 7 10:08:17 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD2541B6D1A for ; Mon, 25 Nov 2024 17:04:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554275; cv=none; b=QhTJSHTaJvp8TeW/eJn8Qazue/HAdkD9QW9xYH//2l2+S4oImk10vGB5NRTSMze0boi8amHMMevXl8QLiFfAt2YDo6FnitR5IA5SxgTnmoFBdiHUfTVkgrQWY1mGJ0bq+29y7BnqlrU76+RGf9hUNPrOJR+60ezHP5VMoQXIl6E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554275; c=relaxed/simple; bh=+3dHa0hesP3zVtBbYkDZG9KKwxOD9MHtJYJmD3szKeg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NEPUyfNagLOz0Zt3hWocxjE85qm4LAgCR9niAM+tqfQv90JcOb6gj9DVsT9lRp6XJh9ztgSrv56faxgXz1i+UTZAvuyqMIgrxPeVFniZGf4Dlpd5I1PPCh61kueiLA8vr8HF/OfMBl+fnPASOPftQgPhBaW12AFNWKUywVklrwE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=W1vqJR2F; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="W1vqJR2F" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4349d895ef8so15109035e9.0 for ; Mon, 25 Nov 2024 09:04:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732554272; x=1733159072; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=y6PR1dk2h7+IF5koNJ9Hiduy7O2xkuw2TgrVl4Qj2oc=; b=W1vqJR2FL0oqxepNwERVDkkLNhyYD31Gvi8WL6UmakuSvKtnP6nPcbyePwu/KVEBRT scW/Ba87QC1p17mEWrpFyuciqa0ae9nQXB3pyFNO2mgzlaCks71UIREKk+jYv2t2o7OK e1v72Nl9/3+web8uAEMeoKo43+dSxA+QLpHyTNLy2ul2lUcpnvW0S51FrlTLRF8Z3J9d rJOPlqdMwwjXwjJToJdm9wNcaW1fOBoHvguFKEj5QxL9KA5T0KRomMhOw185HxS+Fbpk sFATmItR61WrERnhhhBrtiaPn0ihFYLryb14luzUOyjeUpQGlUf/4g4gsN2I6H/JPc4g xdBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732554272; x=1733159072; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=y6PR1dk2h7+IF5koNJ9Hiduy7O2xkuw2TgrVl4Qj2oc=; b=J+0wKuLwfwVeJRb30+pGbqEtrq4a8BCySTKueADtJn6vZkRJVAWH+Th3b0SHcg7RWK EtvbPVhAuMJGQ0KoQdv6otFo//Ozgl9nS67Ir8LjhfCpSkyQ225qyWkx3jfnc8QsOAb6 0jIyNBShQKKzkVKNqqszuqGrtmx32Py+DEtOOL4mYizE2HYNSxKVK7D8SGnH3av1qiom Q/QT8BEEzDWKkBHdt+cEZDfV/ETIWVv4ALxc3Uk7EYJTknuFs4Y43gmMBquk3DgpcLtK G+xwglISS4zPU3Pc2+1a4hOqfsFdfRBy51CPi6ambDcKbE0hDdFPefJ3x3Q4+4tiC8Md pWxw== X-Gm-Message-State: AOJu0YxUFfINYuGVlTgDQqGYiYgRm/DwwxJ/SKI3fBJbHptg1QYNfynJ uBIg9ZdjpoP+8novYY7kdN2InWBqZ64yIwKUaHytj1wj+jQtDaRevekMHo4DKoGAoIOb0zczz9r 3Xg7dnqlpdSYLCGOfDR7jZB1coYqkeOh9qA0d09PjeNbAlglwkFQ7R448eAE+YeX+q+xbAGumie hI+0Obm+qcAZ2ZiRFPp7WGb+1e3+m83g== X-Google-Smtp-Source: AGHT+IHP/NfhOhOrMDpf+TJ/kJJDjJ4W/zFPBBtT1lbimX0fLXAfhJvcsmfNT0bKOvkLGBIyW/57OTxL X-Received: from wmjl17.prod.google.com ([2002:a7b:c351:0:b0:42c:b336:8685]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:c8f:b0:434:a30b:5445 with SMTP id 5b1f17b1804b1-434a30b56b9mr14528815e9.19.1732554272190; Mon, 25 Nov 2024 09:04:32 -0800 (PST) Date: Mon, 25 Nov 2024 18:04:13 +0100 In-Reply-To: <20241125170411.1898410-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241125170411.1898410-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4688; i=ardb@kernel.org; h=from:subject; bh=oV7ao4he/61YWKCmBuI5DWPftgeMewLh5oG5147CmG8=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JId1lHd8Z7aXhB1w49mSIWwvXlTIzdaWu/7wgPPN8gYfA8 fPnXm3oKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABM5eoyR4azSy1S97ycVn8Qb ykQ9SF/ZHee99Yv5l7zsWiMeBzfmfQz/i++pxgkXHn676wXPAUFB2Qomc8WGn6Z8j487bbn29HI VBwA= X-Mailer: git-send-email 2.47.0.371.ga323438b13-goog Message-ID: <20241125170411.1898410-10-ardb+git@google.com> Subject: [PATCH v3 2/6] x86/boot/64: Determine VA/PA offset before entering C code From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Kees Cook , Brian Gerst , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Implicit absolute symbol references (e.g., taking the address of a global variable) must be avoided in the C code that runs from the early 1:1 mapping of the kernel, given that this is a practice that violates assumptions on the part of the toolchain. I.e., RIP-relative and absolute references are expected to produce the same values, and so the compiler is free to choose either. However, the code currently assumes that RIP-relative references are never emitted here. So an explicit virtual-to-physical offset needs to be used instead to derive the kernel virtual addresses of _text and _end, instead of simply taking the addresses and assuming that the compiler will not choose to use a RIP-relative references in this particular case. Currently, phys_base is already used to perform such calculations, but it is derived from the kernel virtual address of _text, which is taken using an implicit absolute symbol reference. So instead, derive this VA-to-PA offset in asm code, and pass it to the C startup code. Signed-off-by: Ard Biesheuvel --- arch/x86/include/asm/setup.h | 2 +- arch/x86/kernel/head64.c | 8 +++++--- arch/x86/kernel/head_64.S | 12 +++++++++--- 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/setup.h b/arch/x86/include/asm/setup.h index 0667b2a88614..85f4fde3515c 100644 --- a/arch/x86/include/asm/setup.h +++ b/arch/x86/include/asm/setup.h @@ -49,7 +49,7 @@ extern unsigned long saved_video_mode; =20 extern void reserve_standard_io_resources(void); extern void i386_reserve_resources(void); -extern unsigned long __startup_64(unsigned long physaddr, struct boot_para= ms *bp); +extern unsigned long __startup_64(unsigned long p2v_offset, struct boot_pa= rams *bp); extern void startup_64_setup_gdt_idt(void); extern void early_setup_idt(void); extern void __init do_early_exception(struct pt_regs *regs, int trapnr); diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 4b9d4557fc94..a7cd4053eeb3 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -138,12 +138,14 @@ static unsigned long __head sme_postprocess_startup(s= truct boot_params *bp, pmdv * doesn't have to generate PC-relative relocations when accessing globals= from * that function. Clang actually does not generate them, which leads to * boot-time crashes. To work around this problem, every global pointer mu= st - * be accessed using RIP_REL_REF(). + * be accessed using RIP_REL_REF(). Kernel virtual addresses can be determ= ined + * by subtracting p2v_offset from the RIP-relative address. */ -unsigned long __head __startup_64(unsigned long physaddr, +unsigned long __head __startup_64(unsigned long p2v_offset, struct boot_params *bp) { pmd_t (*early_pgts)[PTRS_PER_PMD] =3D RIP_REL_REF(early_dynamic_pgts); + unsigned long physaddr =3D (unsigned long)&RIP_REL_REF(_text); unsigned long pgtable_flags; unsigned long load_delta; pgdval_t *pgd; @@ -163,7 +165,7 @@ unsigned long __head __startup_64(unsigned long physadd= r, * Compute the delta between the address I am compiled to run at * and the address I am actually running at. */ - load_delta =3D physaddr - (unsigned long)(_text - __START_KERNEL_map); + load_delta =3D __START_KERNEL_map + p2v_offset; RIP_REL_REF(phys_base) =3D load_delta; =20 /* Is the address not 2M aligned? */ diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 56163e2124cf..31345e0ba006 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -94,13 +94,19 @@ SYM_CODE_START_NOALIGN(startup_64) /* Sanitize CPU configuration */ call verify_cpu =20 + /* + * Derive the kernel's physical-to-virtual offset from the physical and + * virtual addresses of common_startup_64(). + */ + leaq common_startup_64(%rip), %rdi + subq .Lcommon_startup_64(%rip), %rdi + /* * Perform pagetable fixups. Additionally, if SME is active, encrypt * the kernel and retrieve the modifier (SME encryption mask if SME * is active) to be added to the initial pgdir entry that will be * programmed into CR3. */ - leaq _text(%rip), %rdi movq %r15, %rsi call __startup_64 =20 @@ -128,11 +134,11 @@ SYM_CODE_START_NOALIGN(startup_64) =20 /* Branch to the common startup code at its kernel virtual address */ ANNOTATE_RETPOLINE_SAFE - jmp *0f(%rip) + jmp *.Lcommon_startup_64(%rip) SYM_CODE_END(startup_64) =20 __INITRODATA -0: .quad common_startup_64 +SYM_DATA_LOCAL(.Lcommon_startup_64, .quad common_startup_64) =20 .text SYM_CODE_START(secondary_startup_64) --=20 2.47.0.371.ga323438b13-goog From nobody Sat Feb 7 10:08:17 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF3A61B85DB for ; Mon, 25 Nov 2024 17:04:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554277; cv=none; b=uDbY4kc6FghX+3b62Pb+poEVNI/LyIuocd7vmNoegpbu2HpcaX05tKN83m0VNc+pKhPzA1DCGY4wkWWMcQ85s8b9X1eKQNmBRWvo82erHdjN7maAMGFjr1I/j5CcplL6YoMq3uHqdlhp4MbncgSBUQ9FY2uBs58+4AWk4lKoF6w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554277; c=relaxed/simple; bh=Q9nhFnvX5po7qMTBQm2qkwxUv6qGFxXNiDtnJ95wJa8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=B5akMA28oYK4iWdKYu+IIIWDDqRdYnLkQ10tSGKBMD34Aopi/rVfdAyvH5zF4wlLP4OmKsmL8kW4dngshWZHZ6QT1BY4eofahFSRowOtPI7QrP2ilGmM9Bihw6txpYjoRusNvnS+zq9yiMaiZUmcVZhp4E5nflL+njSyL8JR5s4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Oi6YuHZK; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Oi6YuHZK" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4316655b2f1so33984845e9.0 for ; Mon, 25 Nov 2024 09:04:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732554274; x=1733159074; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=BM7VXYp5ri3IbFK6Ch5atB9YeFiiuDxY+E4jisoxWjc=; b=Oi6YuHZKA2xON//byY/JgDhHiMAwxKlP2AgvOaAI1UlQIo74fB2ebMX4SyRZlAZDHJ HfHBE6bcoNSVAUMuS5JdIN/RXQnQeqtHOshUgTcef0TgxfYwyD2cESYTMwbj/KMA1Yic PkWlM3RO655jz+jF+le03/0LmVshngX7B9A97sWB9v2PexdUbTijcEh6b4Iwrz/yJFV9 b8w1sXYJDiG16ZJUVO0vYHwW6avRCfk/g7sfHeSKqZ9bhF2WmY4riOa2bJGfIqOZOSaM NJfoHDe5n3CyLvG25PP6WjqghUwxpktDeD+NdA1TlTuhBBc9n8egkLr9gH6o2izQ85EJ 2YmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732554274; x=1733159074; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BM7VXYp5ri3IbFK6Ch5atB9YeFiiuDxY+E4jisoxWjc=; b=YRx44nAI4uWA5YzIe+nRxRtUhFdSmZcaQvjb12XnFL8DU2VbtuC8pKU9iNO1G3w/vE Q0n5c2+foQsguMUO1El81tUHczDLSrw2goOv8DelceoKNvP/vVlBdNoF+22qS3qKZiGK k8IkEY0x9rh2Wh7jPRCJ0sAGECgq0ZJ+ZfxZqDh2qn7XGMnB9oOOK0Sd8qPADc/blrJf VNb0lzr+gKEznUIRS/aSACRfOfH6zqYpd3W0siM45EpQx4Q0QesMDtQ390h1QTMR/GdY ZhTU54P5bbsQ52o+KRzt3ZlBoNVv64XjXYL3qw1Ix0RaUgRCR+8vpxx6JcX3iLiA6yqG ysoQ== X-Gm-Message-State: AOJu0Yw+o+xqPooyOcVrt3eept03OCshGv+anmmy508p6rkGz2tmMj3n 1Hh/IEy+7cC15m0+raeF+RVxXuhvQ2g3692Y9vBFBJ980QrSPrVNAC8IWJp3bxXS2KcO1uDJ2zq dMP/Kr8juabuVUCkiIUsiL5A+IszdL1k6rhUVyX28EMhVrI9IyP/cwKbVMzLyJx4AofRWenj9hF 1dVRJGExtQJWbWjhLnyaens4cni0uCuA== X-Google-Smtp-Source: AGHT+IGrpQhxDrRtZw7ZKsTRm7aLmXBM7kh5WVYDf/sM62+dDhPXpKmIMG2BcnHQ7D8g+9vRFJWzK+Y6 X-Received: from wmsn38.prod.google.com ([2002:a05:600c:3ba6:b0:431:5cb1:96bf]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b19:b0:42c:bae0:f05b with SMTP id 5b1f17b1804b1-433ce413c8emr103504305e9.1.1732554274240; Mon, 25 Nov 2024 09:04:34 -0800 (PST) Date: Mon, 25 Nov 2024 18:04:14 +0100 In-Reply-To: <20241125170411.1898410-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241125170411.1898410-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4681; i=ardb@kernel.org; h=from:subject; bh=h3YMNGTShKWsXTWvJuEzKY1ErpCIlE1Hv0v55I/Djgw=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JId1lncCnO49Su7uqLFSL6mYtVvwacmYlx/y5H/d/au4PZ rz/J7y/o5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAExE/B7DP7vL7NP2xWtnRptY PK7N6mR0a1R+u1d6vlqqwO9UTk/jIkaGEzfKfp8/7bT/+gWJcmPJQIsjDM8tJ7RnC05LWOsu6fW LCwA= X-Mailer: git-send-email 2.47.0.371.ga323438b13-goog Message-ID: <20241125170411.1898410-11-ardb+git@google.com> Subject: [PATCH v3 3/6] x86/boot/64: Avoid intentional absolute symbol references in .head.text From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Kees Cook , Brian Gerst , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The code in .head.text executes from a 1:1 mapping and cannot generally refer to global variables using their kernel virtual addresses. However, there are some occurrences of such references that are valid: the kernel virtual addresses of _text and _end are needed to populate the page tables correctly, and some other section markers are used in a similar way. To avoid the need for making exceptions to the rule that .head.text must not contain any absolute symbol references, derive these addresses from the RIP-relative 1:1 mapped physical addresses, which can be safely determined using RIP_REL_REF(). Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head64.c | 30 ++++++++++++-------- 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index a7cd4053eeb3..54f9a8faf212 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -91,9 +91,11 @@ static inline bool check_la57_support(void) return true; } =20 -static unsigned long __head sme_postprocess_startup(struct boot_params *bp= , pmdval_t *pmd) +static unsigned long __head sme_postprocess_startup(struct boot_params *bp, + pmdval_t *pmd, + unsigned long p2v_offset) { - unsigned long vaddr, vaddr_end; + unsigned long paddr, paddr_end; int i; =20 /* Encrypt the kernel and related (if SME is active) */ @@ -106,10 +108,10 @@ static unsigned long __head sme_postprocess_startup(s= truct boot_params *bp, pmdv * attribute. */ if (sme_get_me_mask()) { - vaddr =3D (unsigned long)__start_bss_decrypted; - vaddr_end =3D (unsigned long)__end_bss_decrypted; + paddr =3D (unsigned long)&RIP_REL_REF(__start_bss_decrypted); + paddr_end =3D (unsigned long)&RIP_REL_REF(__end_bss_decrypted); =20 - for (; vaddr < vaddr_end; vaddr +=3D PMD_SIZE) { + for (; paddr < paddr_end; paddr +=3D PMD_SIZE) { /* * On SNP, transition the page to shared in the RMP table so that * it is consistent with the page table attribute change. @@ -118,11 +120,11 @@ static unsigned long __head sme_postprocess_startup(s= truct boot_params *bp, pmdv * mapping (kernel .text). PVALIDATE, by way of * early_snp_set_memory_shared(), requires a valid virtual * address but the kernel is currently running off of the identity - * mapping so use __pa() to get a *currently* valid virtual address. + * mapping so use the PA to get a *currently* valid virtual address. */ - early_snp_set_memory_shared(__pa(vaddr), __pa(vaddr), PTRS_PER_PMD); + early_snp_set_memory_shared(paddr, paddr, PTRS_PER_PMD); =20 - i =3D pmd_index(vaddr); + i =3D pmd_index(paddr - p2v_offset); pmd[i] -=3D sme_get_me_mask(); } } @@ -146,6 +148,7 @@ unsigned long __head __startup_64(unsigned long p2v_off= set, { pmd_t (*early_pgts)[PTRS_PER_PMD] =3D RIP_REL_REF(early_dynamic_pgts); unsigned long physaddr =3D (unsigned long)&RIP_REL_REF(_text); + unsigned long va_text, va_end; unsigned long pgtable_flags; unsigned long load_delta; pgdval_t *pgd; @@ -172,6 +175,9 @@ unsigned long __head __startup_64(unsigned long p2v_off= set, if (load_delta & ~PMD_MASK) for (;;); =20 + va_text =3D physaddr - p2v_offset; + va_end =3D (unsigned long)&RIP_REL_REF(_end) - p2v_offset; + /* Include the SME encryption mask in the fixup value */ load_delta +=3D sme_get_me_mask(); =20 @@ -232,7 +238,7 @@ unsigned long __head __startup_64(unsigned long p2v_off= set, pmd_entry +=3D sme_get_me_mask(); pmd_entry +=3D physaddr; =20 - for (i =3D 0; i < DIV_ROUND_UP(_end - _text, PMD_SIZE); i++) { + for (i =3D 0; i < DIV_ROUND_UP(va_end - va_text, PMD_SIZE); i++) { int idx =3D i + (physaddr >> PMD_SHIFT); =20 pmd[idx % PTRS_PER_PMD] =3D pmd_entry + i * PMD_SIZE; @@ -257,11 +263,11 @@ unsigned long __head __startup_64(unsigned long p2v_o= ffset, pmd =3D &RIP_REL_REF(level2_kernel_pgt)->pmd; =20 /* invalidate pages before the kernel image */ - for (i =3D 0; i < pmd_index((unsigned long)_text); i++) + for (i =3D 0; i < pmd_index(va_text); i++) pmd[i] &=3D ~_PAGE_PRESENT; =20 /* fixup pages that are part of the kernel image */ - for (; i <=3D pmd_index((unsigned long)_end); i++) + for (; i <=3D pmd_index(va_end); i++) if (pmd[i] & _PAGE_PRESENT) pmd[i] +=3D load_delta; =20 @@ -269,7 +275,7 @@ unsigned long __head __startup_64(unsigned long p2v_off= set, for (; i < PTRS_PER_PMD; i++) pmd[i] &=3D ~_PAGE_PRESENT; =20 - return sme_postprocess_startup(bp, pmd); + return sme_postprocess_startup(bp, pmd, p2v_offset); } =20 /* Wipe all early page tables except for the kernel symbol map */ --=20 2.47.0.371.ga323438b13-goog From nobody Sat Feb 7 10:08:17 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE19B1B87CC for ; Mon, 25 Nov 2024 17:04:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554279; cv=none; b=hrTJto4PtjAUqMqxKyR1ypx/yevvZNLrmyHSR4PEl7L7p4y6F3VOsH2HgsnJ9hWY2X7wMvcng/VqtteJk+UGqzAAfhaeEHa9j61g2tJzhZaypUylTNXbKFZuZWESGGS77iz9ULYXb98dEu+F7ltUPNTdvWUVOk7Heoi0631c8BI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554279; c=relaxed/simple; bh=CI4anSzhc+btJM996ZvdCCGUYSZiwSHK+Z0r0gQTl7M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=j8Viv94JAI12DRo9Qq8g+grJ0PdG3CupxqmnEZmN2SGNdlPNP6eNOd9n7tlgDO8GK5kaIwD5/5DfEiqfQe6gxLEq6P67/nJhmUO5tXVpB28mMMlCe630TXXWGqusCpTcixoR1LLAmANWZgZaG6wNugUMef1j8sMLIbJw68WgQeY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=yb029CR0; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="yb029CR0" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-434a4ad78a1so1043345e9.0 for ; Mon, 25 Nov 2024 09:04:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732554276; x=1733159076; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=X0X5ksPuZEo9fE4vkgZvo0Iqh/N6D2VPGhMHc4zrWIk=; b=yb029CR0kV5O69wHXaF2Z56hwCq8cB8E2tBHQuiotlbuwmahh2WzuVzXnpTR9vZ0ru axqfy+dczn3vQO4LwgQflEQXE3anQswBuzUeXWKs5PbUEACVaSwaWpN9fFSaWAe6sbhv bRSIgLtifdzmoflAtLdLzjG1rVWFSqaQ0H5aVJxLs9QeMnLCCYGftpAyEYLqLkJebSJh 0cNpttqk9+ouZQiG51XFq4MtyfEmZGKlapXsas32lhhj1lxDvgbQ83sRlg0m3f0nMrJ7 yO/20dyVLJrm/EKeMdP+euDI/NQJ8MY+y7Fu+REaS25hZ2y5iLf41y+hV7xTLqFCz7Wn JHJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732554276; x=1733159076; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=X0X5ksPuZEo9fE4vkgZvo0Iqh/N6D2VPGhMHc4zrWIk=; b=rxCR3WB42Y+dV3zaoQdfV525gD8t3Rx2LWmSg+2np4lotLps8JKZywTPr0VjT58Fpw amMOTKZIiXWyBT8x7I7DXNnenEOEkBUMVMvbm3Z/F+cc5bU6I3nHI80nsLbA+dbpWRye oNjglk+wDczat3WG0MtZlqoelBSH6KbqCZaRFmVGe6HPytMK5X6wDZ46rvPrQsbaVhgH +q8Z0MDlVegBOgW04iqLv/AyLqBVsY2B5rqvb0q/aZo0awfYtla1cS+P49a2iuswWSM1 6wXB8viXgqSnenTs1ZHR4HNYjY4GIPTAeWJ3iSY+YGxoVDsP0IdjOiGKaQQBhJMITJq1 WmbA== X-Gm-Message-State: AOJu0Ywm+JBrkptw2xa5bXS3zJm2d1kXR4yaGYXqnliAWLtsV4lRUz4O hfKhaD4lcy9Fde0GiECtpve+PfnVym0xMkIu3mPKmmM8RW8XFAf5Qazl1xo+m27KeDNxB23ZUTg LNNjQ9ads2ORpqMODrj1BbVwTuftC+YsDS0XID0kmJsrx45zgcfRz0PODOQtp00kQg/1clTdEAK sgOA8iGJZ3OpsYs7eo8nkXZDGePaAnNQ== X-Google-Smtp-Source: AGHT+IHrmSGMgE6f03x/Cmg7ximfWeS/1p7HLiyXtOdJ80HhKgb7hv9SeHXlrujjx+JDF4obC/NMOnTb X-Received: from wmqu8.prod.google.com ([2002:a05:600c:19c8:b0:431:5e49:aa41]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1d28:b0:431:40ca:ce44 with SMTP id 5b1f17b1804b1-433ce4b2ec7mr127410755e9.30.1732554276235; Mon, 25 Nov 2024 09:04:36 -0800 (PST) Date: Mon, 25 Nov 2024 18:04:15 +0100 In-Reply-To: <20241125170411.1898410-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241125170411.1898410-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1850; i=ardb@kernel.org; h=from:subject; bh=gQApAEz6gfUSRYP49oQW0MpQbQMjxKRKRaOdqIRoi3c=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JId1lnZCczOGOolmrK5NzDv/KDzOcwXhn5pWrnVNTrf6Xr H32pUCuo5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEzksBYjw5Z4/1rzKYWGakur 1K+URv9aYVI0PedkYeN99g3MrSLVexkZWjjXRl4M69u2gUfx7qXN0teDN2dknvwhwNnwZhtDVsN +VgA= X-Mailer: git-send-email 2.47.0.371.ga323438b13-goog Message-ID: <20241125170411.1898410-12-ardb+git@google.com> Subject: [PATCH v3 4/6] x86/kernel: Move ENTRY_TEXT to the start of the image From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Kees Cook , Brian Gerst , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Since commit 7734a0f31e99 ("x86/boot: Robustify calling startup_{32,64}() from the dec= ompressor code") it is no longer necessary for .head.text to appear at the start of the image. Considering that ENTRY_TEXT needs to appear PMD-aligned, move it to the start of the image, where the alignment requirement is trivially met. Doing so removes the need to place ENTRY_TEXT at the end of .text and right before .rodata, which appears PMD aligned as well. This will allow .head.text to be moved into a separate output section in a subsequent patch, without incurring more padding overhead. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/vmlinux.lds.S | 26 ++++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 68efd8cd8bf1..c98bc91bafef 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -121,19 +121,6 @@ SECTIONS .text : AT(ADDR(.text) - LOAD_OFFSET) { _text =3D .; _stext =3D .; - /* bootstrapping code */ - HEAD_TEXT - TEXT_TEXT - SCHED_TEXT - LOCK_TEXT - KPROBES_TEXT - SOFTIRQENTRY_TEXT -#ifdef CONFIG_MITIGATION_RETPOLINE - *(.text..__x86.indirect_thunk) - *(.text..__x86.return_thunk) -#endif - STATIC_CALL_TEXT - ALIGN_ENTRY_TEXT_BEGIN *(.text..__x86.rethunk_untrain) ENTRY_TEXT @@ -147,6 +134,19 @@ SECTIONS *(.text..__x86.rethunk_safe) #endif ALIGN_ENTRY_TEXT_END + + /* bootstrapping code */ + HEAD_TEXT + TEXT_TEXT + SCHED_TEXT + LOCK_TEXT + KPROBES_TEXT + SOFTIRQENTRY_TEXT +#ifdef CONFIG_MITIGATION_RETPOLINE + *(.text..__x86.indirect_thunk) + *(.text..__x86.return_thunk) +#endif + STATIC_CALL_TEXT *(.gnu.warning) =20 } :text =3D 0xcccccccc --=20 2.47.0.371.ga323438b13-goog From nobody Sat Feb 7 10:08:17 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECA1C1B87F0 for ; Mon, 25 Nov 2024 17:04:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554281; cv=none; b=hi6T5A7ENJAmABUVdTFMOyAgtq0e2aYCv3gTA5AP3+UazyNsVtE80B05DNpk1MwaRNA6OH6v4LbGiCaLxLVxvR7GTx3zEtdfKMZzWNt8deKzYdWZ90DuavQ58aNxru+sm1buiRJ0nR3NBN0vWMVyc87rAjdrV9zyvf6kQi8i8Fw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554281; c=relaxed/simple; bh=j2tM4QD1SkAtREWke8zQejYp6BaqA5p8HjugUKNYyjs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=j5u9cb0eZ6lnP1EaUQDlnNNyDgXuizMDdUbPqMR9lIb6VAgkzSQaS0wptkHGnle6LXG6P2z+YS6pGCodM46bUY/yzQMwruu6RNd3veP4ZgK9tC0k7/J8rlyyHDPqnPEDbHXtYNoylWbl1QfcYrEb1lKehq/DjBi47l4Lde5dvZg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JzgJ1Lw2; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JzgJ1Lw2" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4349f6b337dso10598675e9.3 for ; Mon, 25 Nov 2024 09:04:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732554278; x=1733159078; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=j3c90HGYf1jO28WhyRh+s+PaLSbRY1NkwjCczIX8X8Y=; b=JzgJ1Lw26MpQwoVtaD+tdHIHWui+TDYupLYB9piyp0oqpEcGub3hviyR44AihSzF6Q fT4mt9i/vYS82PwtrwaePTAXiKTpzmKdSYcBy/qfJNXdCxTq1aOJmHrPNlayl7W+ufdu LXEP34/0Hpd59gDDkTsBfkzrfEu9NriRjkyflXitr2Tm26gJbpy3k/r6rXlVlPRZz4He XkQ4ZFgTZvwB0y1Nd9J4CDBfg9QA8+JyldNcoqIONv/kbkYb9IU1fxO2vUhGo/zBd0sE fLebHTRPxtBxXw3dYYo0azHfOTkVrzOI7TY6OI7W9TYD77QOSeyvY82UoBxVVdxsKRza IpxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732554278; x=1733159078; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=j3c90HGYf1jO28WhyRh+s+PaLSbRY1NkwjCczIX8X8Y=; b=Fxqc4z/nYOsBMmTfd8v5C4mICNzkm8cAejkqoD83Bi1t1V5I8kvGZfQ4aRcf7njrv+ OeGk+cYOkLPtD6X2h11aRHLaqJlPUOdQQiMv6z+oz/ynfsfCE+cwKHcRWRwWB+KKq6/H iqJERLIGYzjRc03j67vh+vNMVvTCGAeOqQzZ8r+NQpbXo4pQ+A1Gk8UG5Jj7mk/mIUvk xiq+oCpOMxtRH51NND3fWRCcTC/Nm6TTN5Dx/JgGBvwQK44z+dZ+f0sT2TqD6+xPKznk KEJ3acTHlW9n5HBrOM34sYUfxgSpe+rHjkR5mDPsbBXPp2pNArtSElWTQiGsB4LVoey1 YqQw== X-Gm-Message-State: AOJu0YwQNCxaVK0egbxuEKBEjreIefBnRCWAI9Pa/a/seQBLr/v1s37H hDu8HfCNGMM9Vftnjz/Pao6px+vUYTr6ru9SugeiS+B3r/MVDnW5dX19WscMg2oSSfiV28r8yv5 FIT2NHd8kVqxaPgBVOZcipCN3X1Y7eXeYIV/TiAw4NJjAVB+uIXB57BdRJe0PW7M5lTZSvRpF1b e6uwee+NYlYKOB2OTVEyGTUlfZXvqAow== X-Google-Smtp-Source: AGHT+IGqAU9eSoeW3uYNkvD7b9yiALLO6GGEjSyjsNZJSVnLsO3KBCkqBuahwUwwZfumVTGo1Ge00/nC X-Received: from wmov21.prod.google.com ([2002:a05:600c:4715:b0:42c:bfc2:aa72]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:2803:b0:382:4a4d:9a10 with SMTP id ffacd0b85a97d-38260b8134dmr7134508f8f.29.1732554278549; Mon, 25 Nov 2024 09:04:38 -0800 (PST) Date: Mon, 25 Nov 2024 18:04:16 +0100 In-Reply-To: <20241125170411.1898410-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241125170411.1898410-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1061; i=ardb@kernel.org; h=from:subject; bh=fDaV6XJBfFZfNWU7YK2EDKa7oVLcbnDeTar+kji37cg=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JId1lnfC5q80cB30Y7c5kHPDLS/Zumv/N5sSqp38mtvo0J za4LirpKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABPxX8TwP/T5XJuZcT1llyct VtZs9Z/fXmtWuOiWgP1e32mmPExL/zL8j7oiX9yyZMHE37ePXrigUDLH+nfSS0tJ/m6P1spHfEL arAA= X-Mailer: git-send-email 2.47.0.371.ga323438b13-goog Message-ID: <20241125170411.1898410-13-ardb+git@google.com> Subject: [PATCH v3 5/6] x86/boot: Move .head.text into its own output section From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Kees Cook , Brian Gerst , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel In order to be able to double check that vmlinux is emitted without absolute symbol references in .head.text, it needs to be distinguishable from the rest of .text in the ELF metadata. So move .head.text into its own ELF section. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/vmlinux.lds.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index c98bc91bafef..9c194df2c8e4 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -135,8 +135,6 @@ SECTIONS #endif ALIGN_ENTRY_TEXT_END =20 - /* bootstrapping code */ - HEAD_TEXT TEXT_TEXT SCHED_TEXT LOCK_TEXT @@ -151,6 +149,11 @@ SECTIONS =20 } :text =3D 0xcccccccc =20 + /* bootstrapping code */ + .head.text : AT(ADDR(.head.text) - LOAD_OFFSET) { + HEAD_TEXT + } :text =3D 0xcccccccc + /* End of text section, which should occupy whole number of pages */ _etext =3D .; . =3D ALIGN(PAGE_SIZE); --=20 2.47.0.371.ga323438b13-goog From nobody Sat Feb 7 10:08:17 2026 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F9891BBBF1 for ; Mon, 25 Nov 2024 17:04:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554284; cv=none; b=rruSG+jeRTZW6d3FE0YcOar1vtFhuaLaXQR5muCerHAHCC6D05V9UT0fhBKTRmTmMcfKmt4Y8P0g3q1SN/V1/TOqAhHvBdQM5VXAkxkv29Cgmd2xwxCrKekPtmfF746hN9NIHe7UNve0ExR7oR6fj1jXXoYVtrJk0qvFE6DNnsI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732554284; c=relaxed/simple; bh=OgZLfOdkYJAHguPWsMKw2rOe00b7KmkS1kMAWr3l+oY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=fADT/pORbT95eK/i4sto0R5c+sgRAKPx+U0f6UiAonrCJ4dR7Hwkyl8Z2nnCheh/7hPKjHYk+p/hTo3ZHDn1bYMOoUu/RsSNGyb1/wVY7WYwmL6FcnVxg5Ujg7UrskYaTZ9gWKTor0yWNyodG66dNEICF2AqripeVyZkbYatqRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=CobyWpQ5; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="CobyWpQ5" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-4349d895ef8so15111075e9.0 for ; Mon, 25 Nov 2024 09:04:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1732554281; x=1733159081; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7e7rZWkrWCKIltWBYJFB5Sv4+3gijylBrzyi+uA36II=; b=CobyWpQ5tAYOLgEFhNd6ZJQyw/Qe6QUgUt8Shy71wLDniHTqeOrz+CtpObutX9irwx yGRq2h9UF5Cm97PIhwMcoEWhdo2UYFHMvGDqkKn1bBK1phwg8yXVHz963P1AU5Ef/6mV smH0vKmUBkWW2RkfWwTk1UunYJ+9T++6h8h5e+IPo52RWflyFqnbqm93lOPb7KK4rs5S LXOkzXt53ZaDnoOyIEBffgxCWP4Y6bv1tORKyJ3vXMo7aUOfD2L3kEza/mxdQMbMWk3I +jQ4N0hIgKIr/IAiusXYOvymRNwkqTExVljEf/ItAQVR5WAu3wF2oRZsGBJmz7fgh2cm yRxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732554281; x=1733159081; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7e7rZWkrWCKIltWBYJFB5Sv4+3gijylBrzyi+uA36II=; b=ThXbz6sRiBSWUR2edpMJIhbLB6+fbfnf1MJjP2IdZj45qsLdwk3guTjxKdB/iZFCQa cv51Od8gMK5U7rQyarYTIB7se/WdPYqLUl3/nD2iUJcrxiOnc79XjFnCl9iTDHtNsPWp FRy3TI2DZ5FOVVQlmO8InYXwICy+f2VwlwjxtQs4XMdZqMsq1MSzOkqeRWPJ6N0Upw/E lGSXscev/HbnuVqS10VAFgtIaDSpTztD1pOU1iT+1t2PUxJT5Z2FCsHiAl2Ucgwp/r7a xn9D6wewBE8gf4h64R75v5gTOJYxSkedqhsGMngB5qWURedDTlFbbWRcdeWsFAsbezC1 qcgQ== X-Gm-Message-State: AOJu0YzVBBts+qkw6zH38d9pQ0ec/O+50eQMXinStEBk8ybpnKmYt6BR hO6sSqJxacKy/HZgIqlj/a3zYYQqH3k9jUsl0vhKj9oPlmorINcUbzIX4E/AZwM5X0DkuRvfVop svd0XjDkvnS7AAZbcO2WC9/IfM6bUdw2BXmKQetdexARJiFb7U62U0dpYNICivOZg09hHOpwVA2 HIS8tPK5zDK0AgDHiIy8OuuQf/ZRbFIw== X-Google-Smtp-Source: AGHT+IHqK8dhfD7pFigahXKEXirWL6H4Hdtrdnqwuz3TB4wp2JcizpNvQU0Mof6GuuxFs+sxkdlaGcwg X-Received: from wman10.prod.google.com ([2002:a05:600c:6c4a:b0:434:a08d:7adb]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4fcc:b0:42c:ba83:3f00 with SMTP id 5b1f17b1804b1-433ce410337mr123783725e9.1.1732554280854; Mon, 25 Nov 2024 09:04:40 -0800 (PST) Date: Mon, 25 Nov 2024 18:04:17 +0100 In-Reply-To: <20241125170411.1898410-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241125170411.1898410-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2472; i=ardb@kernel.org; h=from:subject; bh=S4miXky9ROA9mMVR9FfU2kYSlO0PVsaeXD5POFetRIM=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JId1lncgrrk2HqtzSMu947/z4eOffpdvvzW+PmvJGLaaO5 Wojq83CjlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRMjdGhrlHnn0Q9Kp5sijk ZmXuIyefr+oTrWQUw/h7ji+vrJ53u53hf7rAf/88poPXtn4IbG3Kd/UvqDkkWeq4X6TI5vxy7jv LWAE= X-Mailer: git-send-email 2.47.0.371.ga323438b13-goog Message-ID: <20241125170411.1898410-14-ardb+git@google.com> Subject: [PATCH v3 6/6] x86/boot: Reject absolute references in .head.text From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Kees Cook , Brian Gerst , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The .head.text section used to contain asm code that bootstrapped the page tables and switched to the kernel virtual address space before executing C code. The asm code carefully avoided dereferencing absolute symbol references, as those will fault before the page tables are installed. Today, the .head.text section contains lots of C code too, and getting the compiler to reason about absolute addresses taken from, e.g., section markers such as _text[] or _end[] but never use such absolute references to access global variables [*] is intractible. So instead, forbid the use of absolute references in .head.text entirely, and rely on explicit arithmetic involving VA-to-PA offsets generated by the asm startup code to construct virtual addresses where needed (e.g., to construct the page tables). Note that the 'relocs' tool is only used on the core kernel image when building a relocatable image, but this is the default, and so adding the check there is sufficient to catch new occurrences of code that use absolute references before the kernel mapping is up. [*] it is feasible when using PIC codegen but there is strong pushback to using this for all of the core kernel, and using it only for .head.text is not straight-forward. Signed-off-by: Ard Biesheuvel --- arch/x86/tools/relocs.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/x86/tools/relocs.c b/arch/x86/tools/relocs.c index 27441e5863b2..e937be979ec8 100644 --- a/arch/x86/tools/relocs.c +++ b/arch/x86/tools/relocs.c @@ -841,10 +841,10 @@ static int is_percpu_sym(ElfW(Sym) *sym, const char *= symname) static int do_reloc64(struct section *sec, Elf_Rel *rel, ElfW(Sym) *sym, const char *symname) { + int headtext =3D !strcmp(sec_name(sec->shdr.sh_info), ".head.text"); unsigned r_type =3D ELF64_R_TYPE(rel->r_info); ElfW(Addr) offset =3D rel->r_offset; int shn_abs =3D (sym->st_shndx =3D=3D SHN_ABS) && !is_reloc(S_REL, symnam= e); - if (sym->st_shndx =3D=3D SHN_UNDEF) return 0; =20 @@ -900,6 +900,12 @@ static int do_reloc64(struct section *sec, Elf_Rel *re= l, ElfW(Sym) *sym, break; } =20 + if (headtext) { + die("Absolute reference to symbol '%s' not permitted in .head.text\n", + symname); + break; + } + /* * Relocation offsets for 64 bit kernels are output * as 32 bits and sign extended back to 64 bits when --=20 2.47.0.371.ga323438b13-goog