From nobody Thu Dec 18 13:15:31 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9523C10E5 for ; Tue, 8 Apr 2025 08:53:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102400; cv=none; b=i++oVn8oex/oaiGAuANEFGQoASlXnvnvJl9AHKd2Sw/E6GGQzEaXiu0gcPjfuHx+sTQ2x9w7809Q3thgJdwa1fZMIy82I9vDqrSV+ppapltdFD3HkLqIC9MurtxYwah3OIriIHvSMQd1e/OAX81I5+1RyWvBm/lMB/FIVZz3jrU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102400; c=relaxed/simple; bh=JlE2OQ2AduMmCHQo5eYCN6KCZRNPYziCXg6rcjgqdHM=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jTYNeaoUFno1sI09LJeIgpusbVthsjn3HGD7haQRtBhv02e64yfg9BLKanDkWhw0BQBEDYxuuOzTLyAd+QvhtmKXoFB6l0w4q4RgLrAtGF2NgnxyQ7r6CycruvQk26ObmfxNyunU1G+f8K7NVZozDak7OXRpNbr1fqdLMWlbVt0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bYZaigLG; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bYZaigLG" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43ce8f82e66so34393075e9.3 for ; Tue, 08 Apr 2025 01:53:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744102397; x=1744707197; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=sSE3KrXfCF2z467xGjBqx6qUo0ojkLUVWjm2QnsSXBI=; b=bYZaigLGqqIPCQXYOO+lXl/mJfuhSyXoK65Ec9H7XuEIpBdCPEvq+KEZNMsX/wT6r7 r4tkxpFtiekq73yFnrqct8SU/ry6hyc4cHhfEh2tvh+T49T9CVoP69mw3kWZBO01gpmY Bgt4rBuFz2Tc/vfgV6fY+923GDj4sa0IKIrwS/BaXUBuWRJ8PRom5j5ook5mLMdLZHPN ejUPWQywNSPHDGl4tmP4+TTJGBNIFjpqIFJeOfXq1xcQLcfwkt37l3z62EHuBT5Kd9Rz 0N+XvDa6tEezExLChW3xlYyx+RDORy6BSHWQ10oVfCjin4rgRgltk2wgEZRssetNQQCh bd0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744102397; x=1744707197; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sSE3KrXfCF2z467xGjBqx6qUo0ojkLUVWjm2QnsSXBI=; b=tPR9zxBuBcin39tVtdG2wUu6a5ECFGDrEx4Mf+AoM/Hvg1aoSwvqYhe6MPcrmltdxE UZoTj25VoMMawympZAuftimVQPR+8r1lGQnDrOglP14b8KVnG1+HKGkr/19dtVk5niEU Xgcc5I3cb9o0cMcQ6pub76oT75WchLx625rK94HySZWYbxGQpoy5tLr5g2K7B4JFFJ8D yc7OBie/WVnpxVC+xxr74tKQC5Zle6idjeGae4YXgHczG79pGR9FSxRGL52KgItKNepL 0PO96Ef2lJ56Ui4tyFx4x3BrUY8GX7W7vhKDdP/5Mhy/g8uN2bnMoGueENPb8fzfsvr2 fdlw== X-Forwarded-Encrypted: i=1; AJvYcCUxKtprfgEMzli4Lk2GPRA9rB9V8CoIocnakVALFYORVrdpx2b6+jYqyU6NoMW6IRUHoRTqBJyF7p9wGb0=@vger.kernel.org X-Gm-Message-State: AOJu0YzFM7Odgwjf8bJXfw8GO/SY+cgeCrm4migvoPxyze/D7uHHF3nF +o2D9+pvV75+6CIA6GlGD5ZlbqWudM4zZqaq1b5c48rA+y/LCghqd4drHTKoBM/spmSgUw== X-Google-Smtp-Source: AGHT+IFa0DJqpCQSDbjxFWbigyPtud7VbYW7GW80hRibKF26sQXQzzZjdKrTlP/OKHOu0W764n0BW5kp X-Received: from wmbay15.prod.google.com ([2002:a05:600c:1e0f:b0:43d:48c5:64a2]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1e0a:b0:43c:f70a:2af0 with SMTP id 5b1f17b1804b1-43ed0c50ab7mr163226865e9.16.1744102397047; Tue, 08 Apr 2025 01:53:17 -0700 (PDT) Date: Tue, 8 Apr 2025 10:52:56 +0200 In-Reply-To: <20250408085254.836788-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250408085254.836788-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1053; i=ardb@kernel.org; h=from:subject; bh=+vHTcCsLmbKcBGy048Aa/bn6FL1dz2EfzCduKTSuptU=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3L45dbaltZkrf4KPLdcdq/SXJ//HmtaKOfEWv2terzr 5L7/s6ro5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEwk0Y7hf+kfxSrJkJ0/eN4k t3gUyzAKr9p95/lphpKzHWd6ZF8udmD477T+kNKa7wIHa6v1hVI7FtZXxokoGc95dPNCdbpE2zM bZgA= X-Mailer: git-send-email 2.49.0.504.g3bcea36a83-goog Message-ID: <20250408085254.836788-10-ardb+git@google.com> Subject: [PATCH v3 1/7] x86/boot/startup: Disable objtool validation for library code From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: x86@kernel.org, mingo@kernel.org, linux-kernel@vger.kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The library code built under arch/x86/boot/startup is not intended to be linked into vmlinux but only into the decompressor and/or the EFI stub. This means objtool validation is not needed here, and may result in false positive errors for things like missing retpolines. So disable it for all objects added to lib-y Signed-off-by: Ard Biesheuvel Tested-by: Chaitanya Kumar Borah --- arch/x86/boot/startup/Makefile | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 73946a3f6b3b..8919a1cbcb5a 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -4,3 +4,9 @@ KBUILD_AFLAGS +=3D -D__DISABLE_EXPORTS =20 lib-$(CONFIG_X86_64) +=3D la57toggle.o lib-$(CONFIG_EFI_MIXED) +=3D efi-mixed.o + +# +# Disable objtool validation for all library code, which is intended +# to be linked into the decompressor or the EFI stub but not vmlinux +# +$(patsubst %.o,$(obj)/%.o,$(lib-y)): OBJECT_FILES_NON_STANDARD :=3D y --=20 2.49.0.504.g3bcea36a83-goog From nobody Thu Dec 18 13:15:31 2025 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF171264F94 for ; Tue, 8 Apr 2025 08:53:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102403; cv=none; b=a7V/xlU8TamQAlen9OgkniyaxYef+iw+z6EStDatQRy5mzzNDlycUSpFMxYeOaYh0hnKkwGVndb4GHDlh2kHRQ5XP4k3Xjr6gG3IB6NXpvWMl9qx03d7D2PKlKvXRs/dNYel2fdcBmrBXsWsoyn/bk4dpYPz2pwXA7W0hfyKtZU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102403; c=relaxed/simple; bh=MfWGiN6WSi/Zk+tmGbO8szMKG53sTuUflJC9o3lgens=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=bDsXBbIjVLHDhWOTONKTTXrRyi81ibEnYpSiJg00ze2a2MtokQmp2OGYdK1dw12rqs+9jIA6MoDzwIfrftNDE1jI0XSCQvGz4CQyhTRsPXdnLOGeTl20q8h5yUMkbyW58tvaQhpiJDldN8vt0zBk+fRm/uUevpmVGnhpdjhYzWI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=18ZzQDCo; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="18ZzQDCo" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-391459c0395so2204616f8f.2 for ; Tue, 08 Apr 2025 01:53:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744102399; x=1744707199; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=HO2l3tWFFummWZwJE2HnD5inCchClwn1ne9w9FdWdWQ=; b=18ZzQDCoC0wGUHH5Rfe0wOHs8bQwfdgUmereJK3zVUigtB+SMxbmZbqJUQNPdDSlJN DmrLmst85ewdYFSB4i8bhqvg1puQO69rwKjyXgY4oPXUvXzXddgtdVeZHqRNiwWu68ky LH6hQGdcgn3GZcGV0WMQVL6/i+LhwC0dXhO/tJfEVDPECQSibmin7uTy0SFBWJnsKQxv ZQkGU17ph7i/68eACsmIHNZSLsj7amYGcCD/qWUOD1P3dD5xt6R2NnO+SY8eWIj4a23y TjOZDT3m5WwyX0EWWLaD5myeuhWWco8UniY36/EBYT1WUV/EN3LrEtP9nmJ0uQIkX2TU 3XOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744102399; x=1744707199; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=HO2l3tWFFummWZwJE2HnD5inCchClwn1ne9w9FdWdWQ=; b=Y9YlxeHI6oybGmhGZ5qD//NrYerUFiLSW61zAvZiGtPTYDKM+2q8rtT0thl9pxkZrL eb6VBHG7kxf9Yctw7s92Y8YbwSMWy/tBLTvWI6VRUPrbtLEkP+jtL7DMXI+aaI06Hffi XbysGIziQeIlf83Fk6zSUCryu8cM2yEMJwENmoxunB1gN0HNZGWRk9tWhzsefaltiz4K XyjckfKf4eliHjbrt2hkp6w8TrGcm1cL2HLBY1n07L+GmOTH9UDz4DENmrOjPl11qnPI gqOjVwHuzmtgEP13PqQt30hdZIALDfIX2UiqHoVyHb4AE9RrtIfBS3CypuQqpsmoc8cj dUJQ== X-Forwarded-Encrypted: i=1; AJvYcCW+HTzoRcgFwBAu/ZWzn/PD2S/LpXzZAmlJdmU2GR9JVRjwgqTO93dbmxPv0V+D6zJqRdQTVEDXT8MxcgM=@vger.kernel.org X-Gm-Message-State: AOJu0Yzlyupfs537NzXxWS0wesG29IkKG6ByXBwY3Cgdt7pIKW5CRkuy 2Wjx8tfXURYi5kcf4Gya53zE04ZRWxSC0Y1bJ2g+fGVEEZrMdNCitUjU4jxlfWn0gB3DEw== X-Google-Smtp-Source: AGHT+IE5P19L2OxaCweIeN6QclTC3ENhDs8OX+Js27sfELdEnOiFqRnym6vFy4VvA/lxRLhOmmoibUWT X-Received: from wmgg3.prod.google.com ([2002:a05:600d:3:b0:43c:ef1f:48d3]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:5c84:0:b0:391:4559:8761 with SMTP id ffacd0b85a97d-39cba9333a6mr13125653f8f.36.1744102399199; Tue, 08 Apr 2025 01:53:19 -0700 (PDT) Date: Tue, 8 Apr 2025 10:52:57 +0200 In-Reply-To: <20250408085254.836788-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250408085254.836788-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=8317; i=ardb@kernel.org; h=from:subject; bh=WeI9HuNMCdrTRyg+4/YKtxdJNSRFVjQXqhbM5pjqyAw=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3L41dTXcr+r9zKFiulL+537CGfb86uxw81V2r1pWw4n vOuPlm2o5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEzk9yOG/+VrH89bd+b83VeT D7bLr/v4W3a32Sezuz+1QzW9mTlervVl+B9n8bxEt19tueD0k+d+XW+aEbfr5IXuJxsW/jn5T/B yyBxmAA== X-Mailer: git-send-email 2.49.0.504.g3bcea36a83-goog Message-ID: <20250408085254.836788-11-ardb+git@google.com> Subject: [PATCH v3 2/7] x86/asm: Make rip_rel_ptr() usable from fPIC code From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: x86@kernel.org, mingo@kernel.org, linux-kernel@vger.kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel RIP_REL_REF() is used in non-PIC C code that is called very early, before the kernel virtual mapping is up, which is the mapping that the linker expects. It is currently used in two different ways: - to refer to the value of a global variable, including as an lvalue in assignments; - to take the address of a global variable via the mapping that the code currently executes at. The former case is only needed in non-PIC code, as PIC code will never use absolute symbol references when the address of the symbol is not being used. But taking the address of a variable in PIC code may still require extra care, as a stack allocated struct assignment may be emitted as a memcpy() from a statically allocated copy in .rodata. For instance, this void startup_64_setup_gdt_idt(void) { struct desc_ptr startup_gdt_descr =3D { .address =3D (__force unsigned long)gdt_page.gdt, .size =3D GDT_SIZE - 1, }; may result in an absolute symbol reference in PIC code, even though the struct is allocated on the stack and populated at runtime. To address this case, make rip_rel_ptr() accessible in PIC code, and update any existing uses where the address of a global variable is taken using RIP_REL_REF. Once all code of this nature has been moved into arch/x86/boot/startup and built with -fPIC, RIP_REL_REF() can be retired, and only rip_rel_ptr() will remain. Signed-off-by: Ard Biesheuvel --- arch/x86/coco/sev/core.c | 2 +- arch/x86/coco/sev/shared.c | 4 ++-- arch/x86/include/asm/asm.h | 2 +- arch/x86/kernel/head64.c | 23 ++++++++++---------- arch/x86/mm/mem_encrypt_identity.c | 6 ++--- 5 files changed, 18 insertions(+), 19 deletions(-) diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c index b0c1a7a57497..832f7a7b10b2 100644 --- a/arch/x86/coco/sev/core.c +++ b/arch/x86/coco/sev/core.c @@ -2400,7 +2400,7 @@ static __head void svsm_setup(struct cc_blob_sev_info= *cc_info) * kernel was loaded (physbase), so the get the CA address using * RIP-relative addressing. */ - pa =3D (u64)&RIP_REL_REF(boot_svsm_ca_page); + pa =3D (u64)rip_rel_ptr(&boot_svsm_ca_page); =20 /* * Switch over to the boot SVSM CA while the current CA is still diff --git a/arch/x86/coco/sev/shared.c b/arch/x86/coco/sev/shared.c index 2e4122f8aa6b..04982d356803 100644 --- a/arch/x86/coco/sev/shared.c +++ b/arch/x86/coco/sev/shared.c @@ -475,7 +475,7 @@ static int sev_cpuid_hv(struct ghcb *ghcb, struct es_em= _ctxt *ctxt, struct cpuid */ static const struct snp_cpuid_table *snp_cpuid_get_table(void) { - return &RIP_REL_REF(cpuid_table_copy); + return rip_rel_ptr(&cpuid_table_copy); } =20 /* @@ -1681,7 +1681,7 @@ static bool __head svsm_setup_ca(const struct cc_blob= _sev_info *cc_info) * routine is running identity mapped when called, both by the decompress= or * code and the early kernel code. */ - if (!rmpadjust((unsigned long)&RIP_REL_REF(boot_ghcb_page), RMP_PG_SIZE_4= K, 1)) + if (!rmpadjust((unsigned long)rip_rel_ptr(&boot_ghcb_page), RMP_PG_SIZE_4= K, 1)) return false; =20 /* diff --git a/arch/x86/include/asm/asm.h b/arch/x86/include/asm/asm.h index cc2881576c2c..a9f07799e337 100644 --- a/arch/x86/include/asm/asm.h +++ b/arch/x86/include/asm/asm.h @@ -114,13 +114,13 @@ #endif =20 #ifndef __ASSEMBLER__ -#ifndef __pic__ static __always_inline __pure void *rip_rel_ptr(void *p) { asm("leaq %c1(%%rip), %0" : "=3Dr"(p) : "i"(p)); =20 return p; } +#ifndef __pic__ #define RIP_REL_REF(var) (*(typeof(&(var)))rip_rel_ptr(&(var))) #else #define RIP_REL_REF(var) (var) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index fa9b6339975f..3fb23d805cef 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -106,8 +106,8 @@ static unsigned long __head sme_postprocess_startup(str= uct boot_params *bp, * attribute. */ if (sme_get_me_mask()) { - paddr =3D (unsigned long)&RIP_REL_REF(__start_bss_decrypted); - paddr_end =3D (unsigned long)&RIP_REL_REF(__end_bss_decrypted); + paddr =3D (unsigned long)rip_rel_ptr(__start_bss_decrypted); + paddr_end =3D (unsigned long)rip_rel_ptr(__end_bss_decrypted); =20 for (; paddr < paddr_end; paddr +=3D PMD_SIZE) { /* @@ -144,8 +144,8 @@ static unsigned long __head sme_postprocess_startup(str= uct boot_params *bp, unsigned long __head __startup_64(unsigned long p2v_offset, struct boot_params *bp) { - pmd_t (*early_pgts)[PTRS_PER_PMD] =3D RIP_REL_REF(early_dynamic_pgts); - unsigned long physaddr =3D (unsigned long)&RIP_REL_REF(_text); + pmd_t (*early_pgts)[PTRS_PER_PMD] =3D rip_rel_ptr(early_dynamic_pgts); + unsigned long physaddr =3D (unsigned long)rip_rel_ptr(_text); unsigned long va_text, va_end; unsigned long pgtable_flags; unsigned long load_delta; @@ -174,18 +174,18 @@ unsigned long __head __startup_64(unsigned long p2v_o= ffset, for (;;); =20 va_text =3D physaddr - p2v_offset; - va_end =3D (unsigned long)&RIP_REL_REF(_end) - p2v_offset; + va_end =3D (unsigned long)rip_rel_ptr(_end) - p2v_offset; =20 /* Include the SME encryption mask in the fixup value */ load_delta +=3D sme_get_me_mask(); =20 /* Fixup the physical addresses in the page table */ =20 - pgd =3D &RIP_REL_REF(early_top_pgt)->pgd; + pgd =3D rip_rel_ptr(early_top_pgt); pgd[pgd_index(__START_KERNEL_map)] +=3D load_delta; =20 if (IS_ENABLED(CONFIG_X86_5LEVEL) && la57) { - p4d =3D (p4dval_t *)&RIP_REL_REF(level4_kernel_pgt); + p4d =3D (p4dval_t *)rip_rel_ptr(level4_kernel_pgt); p4d[MAX_PTRS_PER_P4D - 1] +=3D load_delta; =20 pgd[pgd_index(__START_KERNEL_map)] =3D (pgdval_t)p4d | _PAGE_TABLE; @@ -258,7 +258,7 @@ unsigned long __head __startup_64(unsigned long p2v_off= set, * error, causing the BIOS to halt the system. */ =20 - pmd =3D &RIP_REL_REF(level2_kernel_pgt)->pmd; + pmd =3D rip_rel_ptr(level2_kernel_pgt); =20 /* invalidate pages before the kernel image */ for (i =3D 0; i < pmd_index(va_text); i++) @@ -531,7 +531,7 @@ static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTOR= S] __page_aligned_data; static void __head startup_64_load_idt(void *vc_handler) { struct desc_ptr desc =3D { - .address =3D (unsigned long)&RIP_REL_REF(bringup_idt_table), + .address =3D (unsigned long)rip_rel_ptr(bringup_idt_table), .size =3D sizeof(bringup_idt_table) - 1, }; struct idt_data data; @@ -565,11 +565,10 @@ void early_setup_idt(void) */ void __head startup_64_setup_gdt_idt(void) { - struct desc_struct *gdt =3D (void *)(__force unsigned long)gdt_page.gdt; void *handler =3D NULL; =20 struct desc_ptr startup_gdt_descr =3D { - .address =3D (unsigned long)&RIP_REL_REF(*gdt), + .address =3D (unsigned long)rip_rel_ptr((__force void *)&gdt_page), .size =3D GDT_SIZE - 1, }; =20 @@ -582,7 +581,7 @@ void __head startup_64_setup_gdt_idt(void) "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory"); =20 if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) - handler =3D &RIP_REL_REF(vc_no_ghcb); + handler =3D rip_rel_ptr(vc_no_ghcb); =20 startup_64_load_idt(handler); } diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/mm/mem_encrypt_i= dentity.c index 5eecdd92da10..e7fb3779b35f 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/mm/mem_encrypt_identity.c @@ -318,8 +318,8 @@ void __head sme_encrypt_kernel(struct boot_params *bp) * memory from being cached. */ =20 - kernel_start =3D (unsigned long)RIP_REL_REF(_text); - kernel_end =3D ALIGN((unsigned long)RIP_REL_REF(_end), PMD_SIZE); + kernel_start =3D (unsigned long)rip_rel_ptr(_text); + kernel_end =3D ALIGN((unsigned long)rip_rel_ptr(_end), PMD_SIZE); kernel_len =3D kernel_end - kernel_start; =20 initrd_start =3D 0; @@ -345,7 +345,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp) * pagetable structures for the encryption of the kernel * pagetable structures for workarea (in case not currently mapped) */ - execute_start =3D workarea_start =3D (unsigned long)RIP_REL_REF(sme_worka= rea); + execute_start =3D workarea_start =3D (unsigned long)rip_rel_ptr(sme_worka= rea); execute_end =3D execute_start + (PAGE_SIZE * 2) + PMD_SIZE; execute_len =3D execute_end - execute_start; =20 --=20 2.49.0.504.g3bcea36a83-goog From nobody Thu Dec 18 13:15:31 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 07360266572 for ; Tue, 8 Apr 2025 08:53:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102405; cv=none; b=p5tXfPbVv9VMrv+eXOBKZdUWEESBk+BjTd6yZkNS4dqYNMPo1UhhlzLTxi8pWnaRYhz7ZogvinWjZZyrJJH2gGHsCWr8DazSAlBDgjBu9ikDd0w1ySA7SZLQ3FLlD4WQCGL2yUSCvAVeb8RwSLNzpl5d4vRlQhxTmuh13gjNcyU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102405; c=relaxed/simple; bh=UiIXbFws/N6MPsHfKXBZbL6qTKO+a2tli2uXvgO59Zw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=lE64oiLSguHTPg9Upm2G9xUPLxPxVcpPbfJ9OtHhyTiLfgejaMLjWPHRluzpUd1SKxB7Caj+lJ6d5BrUUqF0d4HQPgWR7kDPJR4nda8EOwjefL877Nx9FK9/aCbKHXsjFOy3XFLWGT4UgGB99F6RCkop+sxMK93xJzIGcJ4qP6o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=v2cyxuKy; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="v2cyxuKy" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43efa869b0aso13207615e9.3 for ; Tue, 08 Apr 2025 01:53:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744102401; x=1744707201; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/8h58VkhTYicfCFIDEXpeboAkqejErVKkEny/lhByfs=; b=v2cyxuKyhZEO0ltZiQPdrWRiONt7u28QlImF42wO/858KQoMDu6w18eE7NRbQZQGeK i/gJGd97zerH1R6MIqCOwQ7RUaYfDNwITRZ3q5rspYPWaDl3/mfAWR79tYI/Zivsjgwl +KOeXe8OPZgIPZStJAESucVIrrURJd1mW7PW2YCUfydU5Lt37ron4+f926LHZNPm/+hh P7O5MWKZKN4QByK+aKmzjoKLSj+xA08gipkREJ6h08bz4DX2x0/RII6qkfmP+vlrT/X9 B2+zm2KosHN0CnJKd/dxzaVsmizE3qi0I6s+zzq3LVGx1Vgp9CTZ1I4HkPkPEJSOYocI H1+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744102401; x=1744707201; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/8h58VkhTYicfCFIDEXpeboAkqejErVKkEny/lhByfs=; b=YP6wfDUj4vatb3ud4gj6xMjhLlD2gHf0LHhNnf5f8gpXQV5t1lmRM7XIHb/LnQyVbm 3vG0FE2sr/UnQzYbn2fQICQH0JMz4cX/YwQYdr8fyrA6ShUQP2EXyGUCBpjnhj7P6yVa 2ru8dWMsVe0Y2is2ziEBgxRfFNcOtEZiopHq5flzvbs+W/QyzYW4oEsNJi6cdHtf6dOl Oyd1OirKlW9KTD80LzqsYZbxfDTKll7lvchhKWVKm/6B9HPNtA9h9aNvW/SVdcurHaAU 8ktq7IqWKShqrcSppmmS9+zGqYunEALHiAbyyJfprUUY6gVHNZBtfjS452AkB5Ocq/3H GQfw== X-Forwarded-Encrypted: i=1; AJvYcCXJiOvWQQ2St6G59pvu0IgX+koKcJTc9xiSeSiXgv7+4XACEeOh/dvnB6vpekX9f2N210M6DA8QW08IimQ=@vger.kernel.org X-Gm-Message-State: AOJu0YzgFaEb57MtB5hriBCO7rmaZRQssAkSnGboO5zYeMYCkIyBJKkP huelpavnO8XIRQ8ZRmp1zJB+y3ihKYeZZIfhWIc/hbj1oTzX7su1ne7er+AixUmO2a6yVw== X-Google-Smtp-Source: AGHT+IG3GQ7ePdBsYmKoAHohsV/UHUomyI8w9MTYX++5d0ywPwjqNOuD9XKzu57+zaj/o4O6b/FxTDHq X-Received: from wmby22.prod.google.com ([2002:a05:600c:c056:b0:43c:f517:ac4e]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1387:b0:43c:f513:9585 with SMTP id 5b1f17b1804b1-43ed0bde88amr162598975e9.13.1744102401474; Tue, 08 Apr 2025 01:53:21 -0700 (PDT) Date: Tue, 8 Apr 2025 10:52:58 +0200 In-Reply-To: <20250408085254.836788-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250408085254.836788-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=7715; i=ardb@kernel.org; h=from:subject; bh=pEPidM8yfOLc4MnQp//TasJ2kFqyO78dPBz+ZrxAL10=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3L4zdi4W69nYZ3FBoqnrSw5KqXZCdOLFbrmHn1+sMLv a6CFgs6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwEQc/jIy/Lw7Ver/5mcv1x0U FLn3aXUY87mZX8ueccipFu/9IW8f95qRYVvwunNF2hst+y/O9srqMb/GdC/xj0xQKuOBvgYGvkP zuAA= X-Mailer: git-send-email 2.49.0.504.g3bcea36a83-goog Message-ID: <20250408085254.836788-12-ardb+git@google.com> Subject: [PATCH v3 3/7] x86/boot: Move the early GDT/IDT setup code into startup/ From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: x86@kernel.org, mingo@kernel.org, linux-kernel@vger.kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Move the early GDT/IDT setup code that runs long before the kernel virtual mapping is up into arch/x86/boot/startup/, and build it in a way that ensures that the code tolerates being called from the 1:1 mapping of memory. The code itself is left unchanged by this patch. Also tweak the sed symbol matching pattern in the decompressor to match on lower case 't' or 'b', as these will be emitted by Clang for symbols with hidden linkage. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/Makefile | 2 +- arch/x86/boot/startup/Makefile | 15 ++++ arch/x86/boot/startup/gdt_idt.c | 83 ++++++++++++++++++++ arch/x86/kernel/head64.c | 73 ----------------- 4 files changed, 99 insertions(+), 74 deletions(-) diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/M= akefile index 37b85ce9b2a3..0fcad7b7e007 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -73,7 +73,7 @@ LDFLAGS_vmlinux +=3D -T hostprogs :=3D mkpiggy HOST_EXTRACFLAGS +=3D -I$(srctree)/tools/include =20 -sed-voffset :=3D -e 's/^\([0-9a-fA-F]*\) [ABCDGRSTVW] \(_text\|__start_rod= ata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' +sed-voffset :=3D -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_r= odata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' =20 quiet_cmd_voffset =3D VOFFSET $@ cmd_voffset =3D $(NM) $< | sed -n $(sed-voffset) > $@ diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 8919a1cbcb5a..1beb5de30735 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -1,6 +1,21 @@ # SPDX-License-Identifier: GPL-2.0 =20 KBUILD_AFLAGS +=3D -D__DISABLE_EXPORTS +KBUILD_CFLAGS +=3D -D__DISABLE_EXPORTS -mcmodel=3Dsmall -fPIC \ + -Os -DDISABLE_BRANCH_PROFILING \ + $(DISABLE_STACKLEAK_PLUGIN) \ + -fno-stack-protector -D__NO_FORTIFY \ + -include $(srctree)/include/linux/hidden.h + +# disable ftrace hooks +KBUILD_CFLAGS :=3D $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) +KASAN_SANITIZE :=3D n +KCSAN_SANITIZE :=3D n +KMSAN_SANITIZE :=3D n +UBSAN_SANITIZE :=3D n +KCOV_INSTRUMENT :=3D n + +obj-$(CONFIG_X86_64) +=3D gdt_idt.o =20 lib-$(CONFIG_X86_64) +=3D la57toggle.o lib-$(CONFIG_EFI_MIXED) +=3D efi-mixed.o diff --git a/arch/x86/boot/startup/gdt_idt.c b/arch/x86/boot/startup/gdt_id= t.c new file mode 100644 index 000000000000..1ba6bd5786fe --- /dev/null +++ b/arch/x86/boot/startup/gdt_idt.c @@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +#include +#include +#include +#include +#include + +/* + * Data structures and code used for IDT setup in head_64.S. The bringup-I= DT is + * used until the idt_table takes over. On the boot CPU this happens in + * x86_64_start_kernel(), on secondary CPUs in start_secondary(). In both = cases + * this happens in the functions called from head_64.S. + * + * The idt_table can't be used that early because all the code modifying i= t is + * in idt.c and can be instrumented by tracing or KASAN, which both don't = work + * during early CPU bringup. Also the idt_table has the runtime vectors + * configured which require certain CPU state to be setup already (like TS= S), + * which also hasn't happened yet in early CPU bringup. + */ +static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_d= ata; + +/* This may run while still in the direct mapping */ +static void __head startup_64_load_idt(void *vc_handler) +{ + struct desc_ptr desc =3D { + .address =3D (unsigned long)rip_rel_ptr(bringup_idt_table), + .size =3D sizeof(bringup_idt_table) - 1, + }; + struct idt_data data; + gate_desc idt_desc; + + /* @vc_handler is set only for a VMM Communication Exception */ + if (vc_handler) { + init_idt_data(&data, X86_TRAP_VC, vc_handler); + idt_init_desc(&idt_desc, &data); + native_write_idt_entry((gate_desc *)desc.address, X86_TRAP_VC, &idt_desc= ); + } + + native_load_idt(&desc); +} + +/* This is used when running on kernel addresses */ +void early_setup_idt(void) +{ + void *handler =3D NULL; + + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { + setup_ghcb(); + handler =3D vc_boot_ghcb; + } + + startup_64_load_idt(handler); +} + +/* + * Setup boot CPU state needed before kernel switches to virtual addresses. + */ +void __head startup_64_setup_gdt_idt(void) +{ + void *handler =3D NULL; + + struct desc_ptr startup_gdt_descr =3D { + .address =3D (unsigned long)rip_rel_ptr((__force void *)&gdt_page), + .size =3D GDT_SIZE - 1, + }; + + /* Load GDT */ + native_load_gdt(&startup_gdt_descr); + + /* New GDT is live - reload data segment registers */ + asm volatile("movl %%eax, %%ds\n" + "movl %%eax, %%ss\n" + "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory"); + + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) + handler =3D rip_rel_ptr(vc_no_ghcb); + + startup_64_load_idt(handler); +} diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 3fb23d805cef..9b2ffec4bbad 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -512,76 +512,3 @@ void __init __noreturn x86_64_start_reservations(char = *real_mode_data) =20 start_kernel(); } - -/* - * Data structures and code used for IDT setup in head_64.S. The bringup-I= DT is - * used until the idt_table takes over. On the boot CPU this happens in - * x86_64_start_kernel(), on secondary CPUs in start_secondary(). In both = cases - * this happens in the functions called from head_64.S. - * - * The idt_table can't be used that early because all the code modifying i= t is - * in idt.c and can be instrumented by tracing or KASAN, which both don't = work - * during early CPU bringup. Also the idt_table has the runtime vectors - * configured which require certain CPU state to be setup already (like TS= S), - * which also hasn't happened yet in early CPU bringup. - */ -static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_d= ata; - -/* This may run while still in the direct mapping */ -static void __head startup_64_load_idt(void *vc_handler) -{ - struct desc_ptr desc =3D { - .address =3D (unsigned long)rip_rel_ptr(bringup_idt_table), - .size =3D sizeof(bringup_idt_table) - 1, - }; - struct idt_data data; - gate_desc idt_desc; - - /* @vc_handler is set only for a VMM Communication Exception */ - if (vc_handler) { - init_idt_data(&data, X86_TRAP_VC, vc_handler); - idt_init_desc(&idt_desc, &data); - native_write_idt_entry((gate_desc *)desc.address, X86_TRAP_VC, &idt_desc= ); - } - - native_load_idt(&desc); -} - -/* This is used when running on kernel addresses */ -void early_setup_idt(void) -{ - void *handler =3D NULL; - - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { - setup_ghcb(); - handler =3D vc_boot_ghcb; - } - - startup_64_load_idt(handler); -} - -/* - * Setup boot CPU state needed before kernel switches to virtual addresses. - */ -void __head startup_64_setup_gdt_idt(void) -{ - void *handler =3D NULL; - - struct desc_ptr startup_gdt_descr =3D { - .address =3D (unsigned long)rip_rel_ptr((__force void *)&gdt_page), - .size =3D GDT_SIZE - 1, - }; - - /* Load GDT */ - native_load_gdt(&startup_gdt_descr); - - /* New GDT is live - reload data segment registers */ - asm volatile("movl %%eax, %%ds\n" - "movl %%eax, %%ss\n" - "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory"); - - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) - handler =3D rip_rel_ptr(vc_no_ghcb); - - startup_64_load_idt(handler); -} --=20 2.49.0.504.g3bcea36a83-goog From nobody Thu Dec 18 13:15:31 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A39126657F for ; Tue, 8 Apr 2025 08:53:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102407; cv=none; b=tuARJmlSZ0nuUVXX6Ba36ormUTkwVv9q8dApI+3LlOvgE6sTxk/DMla5313+YdshiMiMQTJiW+Vf5xDcwu3e4hjzUqQNOAVgEJzXxS99AsK5erBTisnKYLOGgze3P1vIfpu2GwzJ1UefFVNvg7JUf1HhGnPSTjgtsTkLqrB0iR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102407; c=relaxed/simple; bh=afAM/r8LLBJ3ce6878gWia6R3fO9omu9O2KzXRnZMwI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Sg5ww+JP5QLgwUNn73PayrqHASez0QsJwr58rupsa9a9w7bGWWGUyUtmrKWI1kTvpLwodhwnl/qIcvHes5oIHL3AtrhLX5lmOKJo/O1vr51jCVidV/TmXt7ElNKFlJPXOjNGOIWXHZZxrB1Up+uFRhXPu53EtRUS6dGTwD+ujWU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=FK4Q2N5L; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="FK4Q2N5L" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43d209dc2d3so34255715e9.3 for ; Tue, 08 Apr 2025 01:53:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744102403; x=1744707203; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=cD0R3YWzzmvI7uAGndwbKruAiXCyd47nNPAPxqQeLtI=; b=FK4Q2N5LhN1V6oAEVATYpNjqy90DE25brdl7RakvzM+Si/QXk8bbtv4V3x4jSrCdin dSVzIKbGXpALZYyDYz1aNx4pLrXK5uOaAMr3Zrtan1REHqqbD0bFQ2NGm3tT5C0UCIYP Mp2v5jKWbp3u0r6j/E53sleiEGfa2cqfxUYaeC9dbKjKmXWf23ZgBv8B4MCsAXs9HZSt G4tcu23Wei+9kZV7SM7NCBpCbWafhZYqrv3DADZeQ+3rsE3iVOGJRh/RFnJo9Ibrqpsi Cl6nVbNyLUEexHUZVlEBfiHwdHHV4RxR/51y3yApqudRhNghlh8KRz5Nk/j329qj0oOM ODJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744102403; x=1744707203; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cD0R3YWzzmvI7uAGndwbKruAiXCyd47nNPAPxqQeLtI=; b=T+NBJKJkQI/q0cJZw1C+WJLvsFiNgxFxIZWaWW+qOaUI+EbsASzpnmqk+Ty8HTw10D Q6d8RZL8dJfBR0Rsg2DALTLKZXrxEj0qbzJiB+zup5XTB13NUvg9xxeBeG+UUlQOyIBn M0Qd1Y5aYtHxJjiC7dpDA+jL2EnT7kLMB/1F1QAX9LFai+SpC2ThV41hFdb83QeHqouX 2qDdLeZuEnSSLS5ysAzRF1JwZ2AVJ4xw7egs6I7lzpHk6exmQKgVqSGt+lheSIHD1KaI SLVwGeR5v3W2dtzUq1Sm2MVWmR1CqS79EgGOO9X8d1yvK60vTYvhO1xl1vWvcbKHzTBv AL9Q== X-Forwarded-Encrypted: i=1; AJvYcCWr7plY0Ysb/9YEje0ZT4Ok+CYWsJ7Q2QH3zBOZIXQTTZx8sJnVFDaiqvUOtaiQU29Ni9HTj/Y8KpPhOkU=@vger.kernel.org X-Gm-Message-State: AOJu0YyBo5bi0ZRzC4qgJ3K1WscyoLKmBP6lyHY/fQsdF8aTfO2BaSwX oyGNsdq8MpttAvR/XGZYqLaaeZDBk8ITIOSyCbbXS+NMJxfYgMCjke0/4Ur6/kCOoKrIaA== X-Google-Smtp-Source: AGHT+IH3hgzl7XMIA+2byfCqeuv9ky4s6Y91FTFpPpgmkni5CTzO8UF27NgEzCp4MuWw4yVPNr1+RNNF X-Received: from wmsp32.prod.google.com ([2002:a05:600c:1da0:b0:43c:f6b3:fa10]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3b1b:b0:43c:ec4c:25b1 with SMTP id 5b1f17b1804b1-43ee076d701mr88955555e9.23.1744102403597; Tue, 08 Apr 2025 01:53:23 -0700 (PDT) Date: Tue, 8 Apr 2025 10:52:59 +0200 In-Reply-To: <20250408085254.836788-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250408085254.836788-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=16737; i=ardb@kernel.org; h=from:subject; bh=tGbwhpUC8P3TmNYLN+90vBme6ochJTirJO/Rz83PktE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3L47ccBU+KfxlvjjBIF910c0q95aNrTz1NFOfsF931W d/R5UtmRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZhI+gGGf2oTg2Wfr/jEuWxf 5d6l/955nm3QELYvNnrH9mq7pPJZxcMM/wwFF/38n2aw86gTU+mkbw95Nkence1e0nszf9+a3bI KoVwA X-Mailer: git-send-email 2.49.0.504.g3bcea36a83-goog Message-ID: <20250408085254.836788-13-ardb+git@google.com> Subject: [PATCH v3 4/7] x86/boot: Move early kernel mapping code into startup/ From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: x86@kernel.org, mingo@kernel.org, linux-kernel@vger.kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The startup code that constructs the kernel virtual mapping runs from the 1:1 mapping of memory itself, and therefore, cannot use absolute symbol references. Before making changes in subsequent patches, move this code into a separate source file under arch/x86/boot/startup/ where all such code will be kept from now on. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/startup/Makefile | 2 +- arch/x86/boot/startup/map_kernel.c | 224 ++++++++++++++++++++ arch/x86/kernel/head64.c | 211 +----------------- 3 files changed, 226 insertions(+), 211 deletions(-) diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 1beb5de30735..10319aee666b 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -15,7 +15,7 @@ KMSAN_SANITIZE :=3D n UBSAN_SANITIZE :=3D n KCOV_INSTRUMENT :=3D n =20 -obj-$(CONFIG_X86_64) +=3D gdt_idt.o +obj-$(CONFIG_X86_64) +=3D gdt_idt.o map_kernel.o =20 lib-$(CONFIG_X86_64) +=3D la57toggle.o lib-$(CONFIG_EFI_MIXED) +=3D efi-mixed.o diff --git a/arch/x86/boot/startup/map_kernel.c b/arch/x86/boot/startup/map= _kernel.c new file mode 100644 index 000000000000..5f1b7e0ba26e --- /dev/null +++ b/arch/x86/boot/startup/map_kernel.c @@ -0,0 +1,224 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD]; +extern unsigned int next_early_pgt; + +static inline bool check_la57_support(void) +{ + if (!IS_ENABLED(CONFIG_X86_5LEVEL)) + return false; + + /* + * 5-level paging is detected and enabled at kernel decompression + * stage. Only check if it has been enabled there. + */ + if (!(native_read_cr4() & X86_CR4_LA57)) + return false; + + RIP_REL_REF(__pgtable_l5_enabled) =3D 1; + RIP_REL_REF(pgdir_shift) =3D 48; + RIP_REL_REF(ptrs_per_p4d) =3D 512; + RIP_REL_REF(page_offset_base) =3D __PAGE_OFFSET_BASE_L5; + RIP_REL_REF(vmalloc_base) =3D __VMALLOC_BASE_L5; + RIP_REL_REF(vmemmap_base) =3D __VMEMMAP_BASE_L5; + + return true; +} + +static unsigned long __head sme_postprocess_startup(struct boot_params *bp, + pmdval_t *pmd, + unsigned long p2v_offset) +{ + unsigned long paddr, paddr_end; + int i; + + /* Encrypt the kernel and related (if SME is active) */ + sme_encrypt_kernel(bp); + + /* + * Clear the memory encryption mask from the .bss..decrypted section. + * The bss section will be memset to zero later in the initialization so + * there is no need to zero it after changing the memory encryption + * attribute. + */ + if (sme_get_me_mask()) { + paddr =3D (unsigned long)rip_rel_ptr(__start_bss_decrypted); + paddr_end =3D (unsigned long)rip_rel_ptr(__end_bss_decrypted); + + for (; paddr < paddr_end; paddr +=3D PMD_SIZE) { + /* + * On SNP, transition the page to shared in the RMP table so that + * it is consistent with the page table attribute change. + * + * __start_bss_decrypted has a virtual address in the high range + * mapping (kernel .text). PVALIDATE, by way of + * early_snp_set_memory_shared(), requires a valid virtual + * address but the kernel is currently running off of the identity + * mapping so use the PA to get a *currently* valid virtual address. + */ + early_snp_set_memory_shared(paddr, paddr, PTRS_PER_PMD); + + i =3D pmd_index(paddr - p2v_offset); + pmd[i] -=3D sme_get_me_mask(); + } + } + + /* + * Return the SME encryption mask (if SME is active) to be used as a + * modifier for the initial pgdir entry programmed into CR3. + */ + return sme_get_me_mask(); +} + +/* Code in __startup_64() can be relocated during execution, but the compi= ler + * doesn't have to generate PC-relative relocations when accessing globals= from + * that function. Clang actually does not generate them, which leads to + * boot-time crashes. To work around this problem, every global pointer mu= st + * be accessed using RIP_REL_REF(). Kernel virtual addresses can be determ= ined + * by subtracting p2v_offset from the RIP-relative address. + */ +unsigned long __head __startup_64(unsigned long p2v_offset, + struct boot_params *bp) +{ + pmd_t (*early_pgts)[PTRS_PER_PMD] =3D rip_rel_ptr(early_dynamic_pgts); + unsigned long physaddr =3D (unsigned long)rip_rel_ptr(_text); + unsigned long va_text, va_end; + unsigned long pgtable_flags; + unsigned long load_delta; + pgdval_t *pgd; + p4dval_t *p4d; + pudval_t *pud; + pmdval_t *pmd, pmd_entry; + bool la57; + int i; + + la57 =3D check_la57_support(); + + /* Is the address too large? */ + if (physaddr >> MAX_PHYSMEM_BITS) + for (;;); + + /* + * Compute the delta between the address I am compiled to run at + * and the address I am actually running at. + */ + load_delta =3D __START_KERNEL_map + p2v_offset; + RIP_REL_REF(phys_base) =3D load_delta; + + /* Is the address not 2M aligned? */ + if (load_delta & ~PMD_MASK) + for (;;); + + va_text =3D physaddr - p2v_offset; + va_end =3D (unsigned long)rip_rel_ptr(_end) - p2v_offset; + + /* Include the SME encryption mask in the fixup value */ + load_delta +=3D sme_get_me_mask(); + + /* Fixup the physical addresses in the page table */ + + pgd =3D rip_rel_ptr(early_top_pgt); + pgd[pgd_index(__START_KERNEL_map)] +=3D load_delta; + + if (IS_ENABLED(CONFIG_X86_5LEVEL) && la57) { + p4d =3D (p4dval_t *)rip_rel_ptr(level4_kernel_pgt); + p4d[MAX_PTRS_PER_P4D - 1] +=3D load_delta; + + pgd[pgd_index(__START_KERNEL_map)] =3D (pgdval_t)p4d | _PAGE_TABLE; + } + + RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 2].pud +=3D load_delta; + RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 1].pud +=3D load_delta; + + for (i =3D FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--) + RIP_REL_REF(level2_fixmap_pgt)[i].pmd +=3D load_delta; + + /* + * Set up the identity mapping for the switchover. These + * entries should *NOT* have the global bit set! This also + * creates a bunch of nonsense entries but that is fine -- + * it avoids problems around wraparound. + */ + + pud =3D &early_pgts[0]->pmd; + pmd =3D &early_pgts[1]->pmd; + RIP_REL_REF(next_early_pgt) =3D 2; + + pgtable_flags =3D _KERNPG_TABLE_NOENC + sme_get_me_mask(); + + if (la57) { + p4d =3D &early_pgts[RIP_REL_REF(next_early_pgt)++]->pmd; + + i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; + pgd[i + 0] =3D (pgdval_t)p4d + pgtable_flags; + pgd[i + 1] =3D (pgdval_t)p4d + pgtable_flags; + + i =3D physaddr >> P4D_SHIFT; + p4d[(i + 0) % PTRS_PER_P4D] =3D (pgdval_t)pud + pgtable_flags; + p4d[(i + 1) % PTRS_PER_P4D] =3D (pgdval_t)pud + pgtable_flags; + } else { + i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; + pgd[i + 0] =3D (pgdval_t)pud + pgtable_flags; + pgd[i + 1] =3D (pgdval_t)pud + pgtable_flags; + } + + i =3D physaddr >> PUD_SHIFT; + pud[(i + 0) % PTRS_PER_PUD] =3D (pudval_t)pmd + pgtable_flags; + pud[(i + 1) % PTRS_PER_PUD] =3D (pudval_t)pmd + pgtable_flags; + + pmd_entry =3D __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL; + /* Filter out unsupported __PAGE_KERNEL_* bits: */ + pmd_entry &=3D RIP_REL_REF(__supported_pte_mask); + pmd_entry +=3D sme_get_me_mask(); + pmd_entry +=3D physaddr; + + for (i =3D 0; i < DIV_ROUND_UP(va_end - va_text, PMD_SIZE); i++) { + int idx =3D i + (physaddr >> PMD_SHIFT); + + pmd[idx % PTRS_PER_PMD] =3D pmd_entry + i * PMD_SIZE; + } + + /* + * Fixup the kernel text+data virtual addresses. Note that + * we might write invalid pmds, when the kernel is relocated + * cleanup_highmap() fixes this up along with the mappings + * beyond _end. + * + * Only the region occupied by the kernel image has so far + * been checked against the table of usable memory regions + * provided by the firmware, so invalidate pages outside that + * region. A page table entry that maps to a reserved area of + * memory would allow processor speculation into that area, + * and on some hardware (particularly the UV platform) even + * speculative access to some reserved areas is caught as an + * error, causing the BIOS to halt the system. + */ + + pmd =3D rip_rel_ptr(level2_kernel_pgt); + + /* invalidate pages before the kernel image */ + for (i =3D 0; i < pmd_index(va_text); i++) + pmd[i] &=3D ~_PAGE_PRESENT; + + /* fixup pages that are part of the kernel image */ + for (; i <=3D pmd_index(va_end); i++) + if (pmd[i] & _PAGE_PRESENT) + pmd[i] +=3D load_delta; + + /* invalidate pages after the kernel image */ + for (; i < PTRS_PER_PMD; i++) + pmd[i] &=3D ~_PAGE_PRESENT; + + return sme_postprocess_startup(bp, pmd, p2v_offset); +} diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 9b2ffec4bbad..6b68a206fa7f 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -47,7 +47,7 @@ * Manage page tables very early on. */ extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD]; -static unsigned int __initdata next_early_pgt; +unsigned int __initdata next_early_pgt; pmdval_t early_pmd_flags =3D __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_= NX); =20 #ifdef CONFIG_X86_5LEVEL @@ -67,215 +67,6 @@ unsigned long vmemmap_base __ro_after_init =3D __VMEMMA= P_BASE_L4; EXPORT_SYMBOL(vmemmap_base); #endif =20 -static inline bool check_la57_support(void) -{ - if (!IS_ENABLED(CONFIG_X86_5LEVEL)) - return false; - - /* - * 5-level paging is detected and enabled at kernel decompression - * stage. Only check if it has been enabled there. - */ - if (!(native_read_cr4() & X86_CR4_LA57)) - return false; - - RIP_REL_REF(__pgtable_l5_enabled) =3D 1; - RIP_REL_REF(pgdir_shift) =3D 48; - RIP_REL_REF(ptrs_per_p4d) =3D 512; - RIP_REL_REF(page_offset_base) =3D __PAGE_OFFSET_BASE_L5; - RIP_REL_REF(vmalloc_base) =3D __VMALLOC_BASE_L5; - RIP_REL_REF(vmemmap_base) =3D __VMEMMAP_BASE_L5; - - return true; -} - -static unsigned long __head sme_postprocess_startup(struct boot_params *bp, - pmdval_t *pmd, - unsigned long p2v_offset) -{ - unsigned long paddr, paddr_end; - int i; - - /* Encrypt the kernel and related (if SME is active) */ - sme_encrypt_kernel(bp); - - /* - * Clear the memory encryption mask from the .bss..decrypted section. - * The bss section will be memset to zero later in the initialization so - * there is no need to zero it after changing the memory encryption - * attribute. - */ - if (sme_get_me_mask()) { - paddr =3D (unsigned long)rip_rel_ptr(__start_bss_decrypted); - paddr_end =3D (unsigned long)rip_rel_ptr(__end_bss_decrypted); - - for (; paddr < paddr_end; paddr +=3D PMD_SIZE) { - /* - * On SNP, transition the page to shared in the RMP table so that - * it is consistent with the page table attribute change. - * - * __start_bss_decrypted has a virtual address in the high range - * mapping (kernel .text). PVALIDATE, by way of - * early_snp_set_memory_shared(), requires a valid virtual - * address but the kernel is currently running off of the identity - * mapping so use the PA to get a *currently* valid virtual address. - */ - early_snp_set_memory_shared(paddr, paddr, PTRS_PER_PMD); - - i =3D pmd_index(paddr - p2v_offset); - pmd[i] -=3D sme_get_me_mask(); - } - } - - /* - * Return the SME encryption mask (if SME is active) to be used as a - * modifier for the initial pgdir entry programmed into CR3. - */ - return sme_get_me_mask(); -} - -/* Code in __startup_64() can be relocated during execution, but the compi= ler - * doesn't have to generate PC-relative relocations when accessing globals= from - * that function. Clang actually does not generate them, which leads to - * boot-time crashes. To work around this problem, every global pointer mu= st - * be accessed using RIP_REL_REF(). Kernel virtual addresses can be determ= ined - * by subtracting p2v_offset from the RIP-relative address. - */ -unsigned long __head __startup_64(unsigned long p2v_offset, - struct boot_params *bp) -{ - pmd_t (*early_pgts)[PTRS_PER_PMD] =3D rip_rel_ptr(early_dynamic_pgts); - unsigned long physaddr =3D (unsigned long)rip_rel_ptr(_text); - unsigned long va_text, va_end; - unsigned long pgtable_flags; - unsigned long load_delta; - pgdval_t *pgd; - p4dval_t *p4d; - pudval_t *pud; - pmdval_t *pmd, pmd_entry; - bool la57; - int i; - - la57 =3D check_la57_support(); - - /* Is the address too large? */ - if (physaddr >> MAX_PHYSMEM_BITS) - for (;;); - - /* - * Compute the delta between the address I am compiled to run at - * and the address I am actually running at. - */ - load_delta =3D __START_KERNEL_map + p2v_offset; - RIP_REL_REF(phys_base) =3D load_delta; - - /* Is the address not 2M aligned? */ - if (load_delta & ~PMD_MASK) - for (;;); - - va_text =3D physaddr - p2v_offset; - va_end =3D (unsigned long)rip_rel_ptr(_end) - p2v_offset; - - /* Include the SME encryption mask in the fixup value */ - load_delta +=3D sme_get_me_mask(); - - /* Fixup the physical addresses in the page table */ - - pgd =3D rip_rel_ptr(early_top_pgt); - pgd[pgd_index(__START_KERNEL_map)] +=3D load_delta; - - if (IS_ENABLED(CONFIG_X86_5LEVEL) && la57) { - p4d =3D (p4dval_t *)rip_rel_ptr(level4_kernel_pgt); - p4d[MAX_PTRS_PER_P4D - 1] +=3D load_delta; - - pgd[pgd_index(__START_KERNEL_map)] =3D (pgdval_t)p4d | _PAGE_TABLE; - } - - RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 2].pud +=3D load_delta; - RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 1].pud +=3D load_delta; - - for (i =3D FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--) - RIP_REL_REF(level2_fixmap_pgt)[i].pmd +=3D load_delta; - - /* - * Set up the identity mapping for the switchover. These - * entries should *NOT* have the global bit set! This also - * creates a bunch of nonsense entries but that is fine -- - * it avoids problems around wraparound. - */ - - pud =3D &early_pgts[0]->pmd; - pmd =3D &early_pgts[1]->pmd; - RIP_REL_REF(next_early_pgt) =3D 2; - - pgtable_flags =3D _KERNPG_TABLE_NOENC + sme_get_me_mask(); - - if (la57) { - p4d =3D &early_pgts[RIP_REL_REF(next_early_pgt)++]->pmd; - - i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; - pgd[i + 0] =3D (pgdval_t)p4d + pgtable_flags; - pgd[i + 1] =3D (pgdval_t)p4d + pgtable_flags; - - i =3D physaddr >> P4D_SHIFT; - p4d[(i + 0) % PTRS_PER_P4D] =3D (pgdval_t)pud + pgtable_flags; - p4d[(i + 1) % PTRS_PER_P4D] =3D (pgdval_t)pud + pgtable_flags; - } else { - i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; - pgd[i + 0] =3D (pgdval_t)pud + pgtable_flags; - pgd[i + 1] =3D (pgdval_t)pud + pgtable_flags; - } - - i =3D physaddr >> PUD_SHIFT; - pud[(i + 0) % PTRS_PER_PUD] =3D (pudval_t)pmd + pgtable_flags; - pud[(i + 1) % PTRS_PER_PUD] =3D (pudval_t)pmd + pgtable_flags; - - pmd_entry =3D __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL; - /* Filter out unsupported __PAGE_KERNEL_* bits: */ - pmd_entry &=3D RIP_REL_REF(__supported_pte_mask); - pmd_entry +=3D sme_get_me_mask(); - pmd_entry +=3D physaddr; - - for (i =3D 0; i < DIV_ROUND_UP(va_end - va_text, PMD_SIZE); i++) { - int idx =3D i + (physaddr >> PMD_SHIFT); - - pmd[idx % PTRS_PER_PMD] =3D pmd_entry + i * PMD_SIZE; - } - - /* - * Fixup the kernel text+data virtual addresses. Note that - * we might write invalid pmds, when the kernel is relocated - * cleanup_highmap() fixes this up along with the mappings - * beyond _end. - * - * Only the region occupied by the kernel image has so far - * been checked against the table of usable memory regions - * provided by the firmware, so invalidate pages outside that - * region. A page table entry that maps to a reserved area of - * memory would allow processor speculation into that area, - * and on some hardware (particularly the UV platform) even - * speculative access to some reserved areas is caught as an - * error, causing the BIOS to halt the system. - */ - - pmd =3D rip_rel_ptr(level2_kernel_pgt); - - /* invalidate pages before the kernel image */ - for (i =3D 0; i < pmd_index(va_text); i++) - pmd[i] &=3D ~_PAGE_PRESENT; - - /* fixup pages that are part of the kernel image */ - for (; i <=3D pmd_index(va_end); i++) - if (pmd[i] & _PAGE_PRESENT) - pmd[i] +=3D load_delta; - - /* invalidate pages after the kernel image */ - for (; i < PTRS_PER_PMD; i++) - pmd[i] &=3D ~_PAGE_PRESENT; - - return sme_postprocess_startup(bp, pmd, p2v_offset); -} - /* Wipe all early page tables except for the kernel symbol map */ static void __init reset_early_page_tables(void) { --=20 2.49.0.504.g3bcea36a83-goog From nobody Thu Dec 18 13:15:31 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10556266596 for ; Tue, 8 Apr 2025 08:53:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102409; cv=none; b=P3MTeiDhUVn6xrVljRBphZelTRdVVRwad11Jq0HgNPBtMaJR+t0ti/sEvgAYOj+b/wm/6lcWFwU+GF/aW5i7Fog/teDanOmFM06ykg+dcsL4SLX1lB/NL3TRPneKm0P2EKsWOC2dXCataqSxePSiMNTrCf8f0vsIfDHjeZtbCJ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102409; c=relaxed/simple; bh=N4MtpQRBVB+U9+XwdsBV8OoaO5bJmSMiovkcXVy1Y8M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ULX7F2eg2ofx6ha1UFXqyxAfQuhVsiPELGvG9DV+ohpt56kexCXtbpA0ZnDZP44SK/926VWrjsq8AyR2HKvqsKpWCF8Fqv0njSZyqHHMaW+d7GlXoWEXT6FT4Z34N1LReV/dn5IZQi76sQf5pOydyXIGdi/Odvd3pgWC4ydll7g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=JC41q92o; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="JC41q92o" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43d51bd9b41so45982225e9.3 for ; Tue, 08 Apr 2025 01:53:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744102405; x=1744707205; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=pfhvtPh90PyQrqwseeYAemYLSGq1dbH+1FaY8eo5bSo=; b=JC41q92oFk6Vtj3L6BZ63eAuC392Llq0DAw4AI/k25YAgx57la3TjEFkNn6050akDz 2wCBr9YOrx8ajmr6yxzTMMOEMJWFowUuFXNqO+47nV0ZqYKnqc38DAYwOiaeiY4+qpUt P0CaoUqCgC5FaseaTV7xt7ZgTcW7/L7Pr9WkdPVESo4YQAm2XyUf++QZ5e3PBGQ138d2 IVSPlDMk6JoPoRYqWsNV6gEhQOD7y/w1YRhrw8U6WueODlQB5v+voE0qoTBYiaJSIbdX rZQD5X8HnMGKc8U+fqpiu+JIk240ZavOY25N12MusvMWoijcB/bZ65G+Wc/upO8/hefp VoUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744102405; x=1744707205; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pfhvtPh90PyQrqwseeYAemYLSGq1dbH+1FaY8eo5bSo=; b=tyKh7NYUF3Wfm2pNoNa/lCD++1zy/NoJhmoqblYUOK3St3WfngXb9UiBjD1jqtPTa6 5RD1fczvN3K5ObujXAycpugCJIVZdX+C+43wJvcwBSg1UXk/kxwxOo9VHPP4hFMy/SRM 1+HYe8F1tFSFltdxF60MH8E35q+X2AkkdKo+3UO+A3edfiVG7QDCVXt9nDluv6ozv5er Df5iLtkx3D/zQc6TkixbQdksGzEh3p+srTJMp/l83PnEof8EekfLf/UeNwQvxjFiQ9iG hrseP97Zg+wPoKfKP3/LpXOuEJoTiOFsJNyrie5iD+ORPkY0z9MVSIFFPWu+L2BjUHlE xXFA== X-Forwarded-Encrypted: i=1; AJvYcCWHdA2AmLcgNx6QoiZrtdlr2DkHMiKO8Du7TG3ZTp6C97qyKlg4sk/xoelt9U//fp4gRJLmgK07CURB26Y=@vger.kernel.org X-Gm-Message-State: AOJu0YxURdqD4gINoMgumM/wUOWokqqcGyBlHZ11j6YDxyKaJaWcKXJg 9f4YtsnQspElL2BhulTqh80nwc2sFz7RpHNJlY22o6jSd6NJqqAhILSqum8VAPFVL3tpoA== X-Google-Smtp-Source: AGHT+IEhPrPgdJzj906H7sww6L5Y+LDSIWu5QMVJjoFQGwSmapUF2HDvtaGfxN7pP2fW7nAn0X+DTnzO X-Received: from wmcq28.prod.google.com ([2002:a05:600c:c11c:b0:43b:c336:7b29]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1819:b0:43c:f895:cb4e with SMTP id 5b1f17b1804b1-43ed0c6b9famr109080695e9.17.1744102405529; Tue, 08 Apr 2025 01:53:25 -0700 (PDT) Date: Tue, 8 Apr 2025 10:53:00 +0200 In-Reply-To: <20250408085254.836788-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250408085254.836788-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=4518; i=ardb@kernel.org; h=from:subject; bh=vBaHbEMhT1iu9JGCBB53txIntdweeE+e86SJD43lhWg=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3L4/dvZgu86r2/v6vpVuiJjY+q5M2m5qqX3i/fWClvz Z3ke/dYRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZjIlHiG/5nvZa1PPyiWC7xp F2G7OSNG4aT2xO0qd60Es20/fzstVsTwT6tH4fTHmaqpkxz07Zf7r60JXlvBOvVvaUmnud6M5Uo XGAE= X-Mailer: git-send-email 2.49.0.504.g3bcea36a83-goog Message-ID: <20250408085254.836788-14-ardb+git@google.com> Subject: [PATCH v3 5/7] x86/boot: Drop RIP_REL_REF() uses from early mapping code From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: x86@kernel.org, mingo@kernel.org, linux-kernel@vger.kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Now that __startup_64() is built using -fPIC, RIP_REL_REF() has become a NOP and can be removed. Only some occurrences of rip_rel_ptr() will remain, to explicitly take the address of certain global structures in the 1:1 mapping of memory. While at it, update the code comment to describe why this is needed. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/startup/map_kernel.c | 41 ++++++++++---------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/arch/x86/boot/startup/map_kernel.c b/arch/x86/boot/startup/map= _kernel.c index 5f1b7e0ba26e..0eac3f17dbd3 100644 --- a/arch/x86/boot/startup/map_kernel.c +++ b/arch/x86/boot/startup/map_kernel.c @@ -26,12 +26,12 @@ static inline bool check_la57_support(void) if (!(native_read_cr4() & X86_CR4_LA57)) return false; =20 - RIP_REL_REF(__pgtable_l5_enabled) =3D 1; - RIP_REL_REF(pgdir_shift) =3D 48; - RIP_REL_REF(ptrs_per_p4d) =3D 512; - RIP_REL_REF(page_offset_base) =3D __PAGE_OFFSET_BASE_L5; - RIP_REL_REF(vmalloc_base) =3D __VMALLOC_BASE_L5; - RIP_REL_REF(vmemmap_base) =3D __VMEMMAP_BASE_L5; + __pgtable_l5_enabled =3D 1; + pgdir_shift =3D 48; + ptrs_per_p4d =3D 512; + page_offset_base =3D __PAGE_OFFSET_BASE_L5; + vmalloc_base =3D __VMALLOC_BASE_L5; + vmemmap_base =3D __VMEMMAP_BASE_L5; =20 return true; } @@ -81,12 +81,14 @@ static unsigned long __head sme_postprocess_startup(str= uct boot_params *bp, return sme_get_me_mask(); } =20 -/* Code in __startup_64() can be relocated during execution, but the compi= ler - * doesn't have to generate PC-relative relocations when accessing globals= from - * that function. Clang actually does not generate them, which leads to - * boot-time crashes. To work around this problem, every global pointer mu= st - * be accessed using RIP_REL_REF(). Kernel virtual addresses can be determ= ined - * by subtracting p2v_offset from the RIP-relative address. +/* + * This code is compiled using PIC codegen because it will execute from the + * early 1:1 mapping of memory, which deviates from the mapping expected b= y the + * linker. Due to this deviation, taking the address of a global variable = will + * produce an ambiguous result when using the plain & operator. Instead, + * rip_rel_ptr() must be used, which will return the RIP-relative address = in + * the 1:1 mapping of memory. Kernel virtual addresses can be determined by + * subtracting p2v_offset from the RIP-relative address. */ unsigned long __head __startup_64(unsigned long p2v_offset, struct boot_params *bp) @@ -113,8 +115,7 @@ unsigned long __head __startup_64(unsigned long p2v_off= set, * Compute the delta between the address I am compiled to run at * and the address I am actually running at. */ - load_delta =3D __START_KERNEL_map + p2v_offset; - RIP_REL_REF(phys_base) =3D load_delta; + phys_base =3D load_delta =3D __START_KERNEL_map + p2v_offset; =20 /* Is the address not 2M aligned? */ if (load_delta & ~PMD_MASK) @@ -138,11 +139,11 @@ unsigned long __head __startup_64(unsigned long p2v_o= ffset, pgd[pgd_index(__START_KERNEL_map)] =3D (pgdval_t)p4d | _PAGE_TABLE; } =20 - RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 2].pud +=3D load_delta; - RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 1].pud +=3D load_delta; + level3_kernel_pgt[PTRS_PER_PUD - 2].pud +=3D load_delta; + level3_kernel_pgt[PTRS_PER_PUD - 1].pud +=3D load_delta; =20 for (i =3D FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--) - RIP_REL_REF(level2_fixmap_pgt)[i].pmd +=3D load_delta; + level2_fixmap_pgt[i].pmd +=3D load_delta; =20 /* * Set up the identity mapping for the switchover. These @@ -153,12 +154,12 @@ unsigned long __head __startup_64(unsigned long p2v_o= ffset, =20 pud =3D &early_pgts[0]->pmd; pmd =3D &early_pgts[1]->pmd; - RIP_REL_REF(next_early_pgt) =3D 2; + next_early_pgt =3D 2; =20 pgtable_flags =3D _KERNPG_TABLE_NOENC + sme_get_me_mask(); =20 if (la57) { - p4d =3D &early_pgts[RIP_REL_REF(next_early_pgt)++]->pmd; + p4d =3D &early_pgts[next_early_pgt++]->pmd; =20 i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; pgd[i + 0] =3D (pgdval_t)p4d + pgtable_flags; @@ -179,7 +180,7 @@ unsigned long __head __startup_64(unsigned long p2v_off= set, =20 pmd_entry =3D __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL; /* Filter out unsupported __PAGE_KERNEL_* bits: */ - pmd_entry &=3D RIP_REL_REF(__supported_pte_mask); + pmd_entry &=3D __supported_pte_mask; pmd_entry +=3D sme_get_me_mask(); pmd_entry +=3D physaddr; =20 --=20 2.49.0.504.g3bcea36a83-goog From nobody Thu Dec 18 13:15:31 2025 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89DB8266B61 for ; Tue, 8 Apr 2025 08:53:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102411; cv=none; b=pCPcCav/mkIwFRjrMIkSEBqXTWx7njhn2ZNEnfcp/CoezTEi5J2GSsHq59H5uPG2UkqEIT2VrdmIuPQ/NH7v9USEruZK1hOnRx8PIfiKs/tvniNtV6HEsBXCUYs8e/1U8y6EziVf2uuv6co0TFhnjydIFCBsVUjA1vUMD6q+1dU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102411; c=relaxed/simple; bh=rGZMX0G8lRLt0dY4UTfyPAeFNjieLVMC1P9MjIC1z4k=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NLLi9zV2Ay4Dmajobx/Reoeua91j9y/L7FHfuRkJ+FAbrGqEmgMtVps09MztvHvZ1n67Gv85vS5mIoO7qBoikyfV7VnuGdHS7hVtEZm1Iurstw9drcBlN6JDwFLY/9eWcBwDMxjtIpiB1Sf/Q4dB97Rff+nCy3AdqXtZaIYtjmU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lBYR0D04; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lBYR0D04" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-3912fc9861cso2160765f8f.1 for ; Tue, 08 Apr 2025 01:53:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744102408; x=1744707208; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Ut3DF7cdGnYGcsB+1y8GBrdozOKwjcN3MDn2MzZHkLM=; b=lBYR0D04NgiJ0QS+0Ffa5Ok/udk+5iT1U844uIBGP1Co7Pon44j9jqtBX33p7XrQTr /9KD0GnorE8R0dXfmbq1tp6LDgmyikgs2bEEYj357Th1H78mwdWrHOPVVM29MAejWLto CVflXJhInonSyz75MgTzMOaOOMEmVeysx39waMI/oeDFlZaCFrUhZ+4xx4QrCWzsf8yO Bsma0Dbpt8+AaeSvzMW7OvBkKTn2deo7Qz6qaMV/CbHFTcI8K1NlxivrMekGpMY3Lmui sEIRTdXkZajI6xslHRUq3WgGOzPpchpxWBvBWG5w/wQPi+rGL4358fhHoZO74AicJC11 7KVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744102408; x=1744707208; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ut3DF7cdGnYGcsB+1y8GBrdozOKwjcN3MDn2MzZHkLM=; b=Xc5Mc8SqU9h2eETBSM3hRONvKFzFWSldfiO916zzk3tIQoc22ZusjSChS/ccUxNibU gGgDoYuDJyi3oHGtGqwMhrVQyMWQDpM/t0zX8ar03T7O2lMpTllNwCzc50O2g38LqNr1 1iPgfrcNSUaF71EWfc03stOpHBhMBmVPESs5pM4NNaQBksS9ZG26xfMCwETpNPHagFAj tJFopu3V4PBp2MGL+XVNPHr44gw3Bw2xmUv/WoCW1ya0bfsPPYsRbFg+9yR1J/dd93aY SsDDokYbyEgWaA3igOWle2gIbx+Ertkv3yv1SE9ibtajNLPRp8M4OdDK73UW97xaMPKa mWUw== X-Forwarded-Encrypted: i=1; AJvYcCVdCnOxbwIpwpZfFMRPN5RFyinOUrorNMqQcSub3qSb2nRrjWIYDgtyNUhGyDHpsYSzY5jKoBDMMhoBjHI=@vger.kernel.org X-Gm-Message-State: AOJu0YzIusQRDXWEYyFUVggDPaHYJbRwrJvcbPADkhN9huvHW4oD/kP3 0g8QBejkhZM0ul10rr6X2MUBYONhoiY71Ah0IjCBB3+gaY9nMHePcu7pUHFyeuMu4s92FA== X-Google-Smtp-Source: AGHT+IGW/r/Kx/mn/IWLI6XXqQKCxtQJ+RSUOZT/+LuyEv9kOyZw0WzMpRZxWL1WPU2nH+qiHEBf4YWJ X-Received: from wmgg3.prod.google.com ([2002:a05:600d:3:b0:43c:ef1f:48d3]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:1acf:b0:39c:1f10:c736 with SMTP id ffacd0b85a97d-39d0de5dfbcmr12437841f8f.43.1744102407776; Tue, 08 Apr 2025 01:53:27 -0700 (PDT) Date: Tue, 8 Apr 2025 10:53:01 +0200 In-Reply-To: <20250408085254.836788-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250408085254.836788-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3139; i=ardb@kernel.org; h=from:subject; bh=1OMnBnM3B5uy1EYv4XyX2QqLee71wb5xFGUqQq+IsUU=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3L4w/OGbPcrtRWK6vreq9/v/qVwNrPZTvlmqZlHNugf eatcw5DRykLgxgHg6yYIovA7L/vdp6eKFXrPEsWZg4rE8gQBi5OAZiI4yyG/2Xrmu7sn3La/LKs Y/z/Ce7yXurMa49OqQ9/mRB9pcGfaQsjw/ZOYV/mS0xfLqzfEaJy8F17rmLA9YkMmjn2mcLbBL5 acAEA X-Mailer: git-send-email 2.49.0.504.g3bcea36a83-goog Message-ID: <20250408085254.836788-15-ardb+git@google.com> Subject: [PATCH v3 6/7] x86/boot: Move early SME init code into startup/ From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: x86@kernel.org, mingo@kernel.org, linux-kernel@vger.kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Move the SME initialization code, which runs from the 1:1 mapping of memory as it operates on the kernel virtual mapping, into the new sub-directory arch/x86/boot/startup/ where all startup code will reside that needs to tolerate executing from the 1:1 mapping. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/startup/Makefile | 1 + arch/x86/{mm/mem_encrypt_identity.c =3D> boot/startup/sme.c} | 2 -- arch/x86/mm/Makefile | 6 ------ 3 files changed, 1 insertion(+), 8 deletions(-) diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 10319aee666b..ccdfc42a4d59 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -16,6 +16,7 @@ UBSAN_SANITIZE :=3D n KCOV_INSTRUMENT :=3D n =20 obj-$(CONFIG_X86_64) +=3D gdt_idt.o map_kernel.o +obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D sme.o =20 lib-$(CONFIG_X86_64) +=3D la57toggle.o lib-$(CONFIG_EFI_MIXED) +=3D efi-mixed.o diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/boot/startup/sme= .c similarity index 99% rename from arch/x86/mm/mem_encrypt_identity.c rename to arch/x86/boot/startup/sme.c index e7fb3779b35f..23d10cda5b58 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/boot/startup/sme.c @@ -45,8 +45,6 @@ #include #include =20 -#include "mm_internal.h" - #define PGD_FLAGS _KERNPG_TABLE_NOENC #define P4D_FLAGS _KERNPG_TABLE_NOENC #define PUD_FLAGS _KERNPG_TABLE_NOENC diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index 32035d5be5a0..3faa60f13a61 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -3,12 +3,10 @@ KCOV_INSTRUMENT_tlb.o :=3D n KCOV_INSTRUMENT_mem_encrypt.o :=3D n KCOV_INSTRUMENT_mem_encrypt_amd.o :=3D n -KCOV_INSTRUMENT_mem_encrypt_identity.o :=3D n KCOV_INSTRUMENT_pgprot.o :=3D n =20 KASAN_SANITIZE_mem_encrypt.o :=3D n KASAN_SANITIZE_mem_encrypt_amd.o :=3D n -KASAN_SANITIZE_mem_encrypt_identity.o :=3D n KASAN_SANITIZE_pgprot.o :=3D n =20 # Disable KCSAN entirely, because otherwise we get warnings that some func= tions @@ -16,12 +14,10 @@ KASAN_SANITIZE_pgprot.o :=3D n KCSAN_SANITIZE :=3D n # Avoid recursion by not calling KMSAN hooks for CEA code. KMSAN_SANITIZE_cpu_entry_area.o :=3D n -KMSAN_SANITIZE_mem_encrypt_identity.o :=3D n =20 ifdef CONFIG_FUNCTION_TRACER CFLAGS_REMOVE_mem_encrypt.o =3D -pg CFLAGS_REMOVE_mem_encrypt_amd.o =3D -pg -CFLAGS_REMOVE_mem_encrypt_identity.o =3D -pg CFLAGS_REMOVE_pgprot.o =3D -pg endif =20 @@ -32,7 +28,6 @@ obj-y +=3D pat/ =20 # Make sure __phys_addr has no stackprotector CFLAGS_physaddr.o :=3D -fno-stack-protector -CFLAGS_mem_encrypt_identity.o :=3D -fno-stack-protector =20 CFLAGS_fault.o :=3D -I $(src)/../include/asm/trace =20 @@ -63,5 +58,4 @@ obj-$(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION) +=3D pti.o obj-$(CONFIG_X86_MEM_ENCRYPT) +=3D mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_amd.o =20 -obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_boot.o --=20 2.49.0.504.g3bcea36a83-goog From nobody Thu Dec 18 13:15:31 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D3CC266583 for ; Tue, 8 Apr 2025 08:53:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102413; cv=none; b=skQkm7tobZHZpGo+znzlNkjc4p8+qN1GHCbLVW39omrrcbZEehnloVqwrtiWc80CZbBkDTd8dQLtshiQ/G203WlA7GRSXYOc85Vl1ZArMI4ChUOqWTkrnsRgvya52OfeKVQasTQJsL4zmDf3cPwAK2FfUHeqeZxp9iYCLJX2/LE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744102413; c=relaxed/simple; bh=nMTMqzq37ZDdZSy/ASJb7GM4lqZTMOHxEOCfZ5XR4c8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=AQSrSIIaH7Ooi0cYCF1JAhh8ueIsE/qeph256OHgAw/Wr4MYi+SkjDjxcQbu5Vmi11OuNUf8OxGvqAEHU1Kkv5Rl15R4boDMOp4SpL5YhTZXZ3JU2aCukF/PANahpisZJFPrRdIgtAho3m0v8YzuIwA2EVNWUNAOysN9M9owTPg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Ou2OvDhq; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Ou2OvDhq" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43d007b2c79so40068135e9.2 for ; Tue, 08 Apr 2025 01:53:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1744102409; x=1744707209; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Bu1M3/PvdkhGWlHIRkQzAzcvhk8sTE/2aa4WDt4cxm0=; b=Ou2OvDhqh0F1r2X1jAwfX2upSEP+PcLqN/rT1Us36IXDGNF3qxlIwKBAgzk6iZrsQ5 xVEElmYCNo63KgwSGlPqSsGpDBtMP8cjOUH95MlsQHDDDZkv+zIGRLoVqNdX4UhqZGS/ j+5xEXk/UaerEG6IXxz54pP1r+DuHs9bxMiB7NDG+GcGdQwE5v7YjRYTV3D3oZgHCEWh +AH4CxIGF8T5z3yEf1shxdqalq6MrYs8pYAeCczvV+7jmk39i44XVwXstd5mkdEUQjiS SXRvDpN5+NcclY6Ws0yn18fu1+mUxj9VYGOhglCASYRK/doejtdrA/UB6zT3LL8m4uW3 5gVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1744102409; x=1744707209; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Bu1M3/PvdkhGWlHIRkQzAzcvhk8sTE/2aa4WDt4cxm0=; b=YVWp02F9HeBEqUWrU9qRCpJDn9YkMI8yfMVt5/nhuDNn8i4GuzppnvDnOkmgPde3sc wXSpwSEWT/xw4qYmvel0GSazmm2/OUH3Biv7wLzY2c1fxjqPNxCN4JmdEUI/DtPSNcGH wsGPj+D1oB9eExzbDwL+W7RzqUCp5rgKBzDSyhqgqUULb46sC9AtkDRYYnHvgqsLILFM VQTLIl7o1zEQgaAn9/OpGtkmeje+FPAEOAir6mFq2CreTyqcj8Y4S5SMuDpPQdOUNgFS lO19VhHdKwpSe6IUa0urqhHTbvrSPRJznQ11o40imsN2ESUX4ujZOczR16Y28ftVyMkE 7IXg== X-Forwarded-Encrypted: i=1; AJvYcCXArS2cqe0Eb+oGJoLB37dpcqv/mbFmygDS+D30JJJsrNX66NWWGyib+pALjqNamwZ+vBnv+iCY7ct3SYQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yww8dI3gqL8v2Aib2uPCShYmmhju7jriGC1UmYaL0p/JR6C/63S pV3d1PftSY7EpWVsAzZ64yA5Pz8tvDaxSjyGZ25sOG+VMBIrmBeRgZEfA7lDrLpyhMAB/w== X-Google-Smtp-Source: AGHT+IE/uHSaVtNKee/dmkihc8dBSaKxeFeUFpLQgCV3qUZJUtmlBi/YqSX0r50x/1aUdAmTZhKi+Myg X-Received: from wmcq3.prod.google.com ([2002:a05:600c:c103:b0:43d:41a2:b768]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1e89:b0:43c:f184:2e16 with SMTP id 5b1f17b1804b1-43ee06134f3mr112581425e9.5.1744102409707; Tue, 08 Apr 2025 01:53:29 -0700 (PDT) Date: Tue, 8 Apr 2025 10:53:02 +0200 In-Reply-To: <20250408085254.836788-9-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250408085254.836788-9-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2567; i=ardb@kernel.org; h=from:subject; bh=HHqxACxmdTzUBkBumD23I1ULhjlW30p2Zg7pYMcwt7Y=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3L448arJuLf4nqf91whyWSa+EJ42CPa6UX9+39nhZjm eN+8vzMjlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRoksM/71OLbmk/1CnadfT by8Y94YeZPL2Ypwfkbj5irzG2pB4Xg+G/wk87GJ1Fzlfrdxhb3yM3cgoi8dvMtMDlxvVC56+Xaa zhgsA X-Mailer: git-send-email 2.49.0.504.g3bcea36a83-goog Message-ID: <20250408085254.836788-16-ardb+git@google.com> Subject: [PATCH v3 7/7] x86/boot: Drop RIP_REL_REF() uses from SME startup code From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: x86@kernel.org, mingo@kernel.org, linux-kernel@vger.kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel RIP_REL_REF() has no effect on code residing in arch/x86/boot/startup, as it is built with -fPIC. So remove any occurrences from the SME startup code. Note the SME is the only caller of cc_set_mask() that requires this, so drop it from there as well. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/startup/sme.c | 11 +++++------ arch/x86/include/asm/coco.h | 2 +- arch/x86/include/asm/mem_encrypt.h | 2 +- 3 files changed, 7 insertions(+), 8 deletions(-) diff --git a/arch/x86/boot/startup/sme.c b/arch/x86/boot/startup/sme.c index 23d10cda5b58..5738b31c8e60 100644 --- a/arch/x86/boot/startup/sme.c +++ b/arch/x86/boot/startup/sme.c @@ -297,8 +297,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp) * instrumentation or checking boot_cpu_data in the cc_platform_has() * function. */ - if (!sme_get_me_mask() || - RIP_REL_REF(sev_status) & MSR_AMD64_SEV_ENABLED) + if (!sme_get_me_mask() || sev_status & MSR_AMD64_SEV_ENABLED) return; =20 /* @@ -524,7 +523,7 @@ void __head sme_enable(struct boot_params *bp) me_mask =3D 1UL << (ebx & 0x3f); =20 /* Check the SEV MSR whether SEV or SME is enabled */ - RIP_REL_REF(sev_status) =3D msr =3D __rdmsr(MSR_AMD64_SEV); + sev_status =3D msr =3D __rdmsr(MSR_AMD64_SEV); feature_mask =3D (msr & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BI= T; =20 /* @@ -560,8 +559,8 @@ void __head sme_enable(struct boot_params *bp) return; } =20 - RIP_REL_REF(sme_me_mask) =3D me_mask; - RIP_REL_REF(physical_mask) &=3D ~me_mask; - RIP_REL_REF(cc_vendor) =3D CC_VENDOR_AMD; + sme_me_mask =3D me_mask; + physical_mask &=3D ~me_mask; + cc_vendor =3D CC_VENDOR_AMD; cc_set_mask(me_mask); } diff --git a/arch/x86/include/asm/coco.h b/arch/x86/include/asm/coco.h index e7225452963f..e1dbf8df1b69 100644 --- a/arch/x86/include/asm/coco.h +++ b/arch/x86/include/asm/coco.h @@ -22,7 +22,7 @@ static inline u64 cc_get_mask(void) =20 static inline void cc_set_mask(u64 mask) { - RIP_REL_REF(cc_mask) =3D mask; + cc_mask =3D mask; } =20 u64 cc_mkenc(u64 val); diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_= encrypt.h index 1530ee301dfe..ea6494628cb0 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -61,7 +61,7 @@ void __init sev_es_init_vc_handling(void); =20 static inline u64 sme_get_me_mask(void) { - return RIP_REL_REF(sme_me_mask); + return sme_me_mask; } =20 #define __bss_decrypted __section(".bss..decrypted") --=20 2.49.0.504.g3bcea36a83-goog