From nobody Thu Apr 10 17:54:14 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0ED1202C26 for ; Tue, 1 Apr 2025 13:34:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514476; cv=none; b=EZy+qolSCJbPZeNCmTuC9wLj8ChXmCRccE0t4Fu5NF9X8ObumysWjYEN3DoLOeKRocZI8sn20Tk9YBQNd6dtyHs4UQ7ZwI8Q7IVO4LkhoRoldxSVJIh7cGPhG5XvldzaKxYG+kJ436WlZDi1YwHMLCUux+/H5FMTxMDhkZisbiE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514476; c=relaxed/simple; bh=+jsVXUY+JNWjKDWkdCkg/3wzBjLZu5mOtImMQ3fql1Q=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=jVs5uKYcdXIBLi/pWKJgBk8zDXcbe/AiYCoLBO/C7oxpA0PWnpm6jFi97Rn/Vti6ioekz6gEZzAbq3APa7GwWJN2oMQwCEl9STq/A5djZasG4iY1KZ9ZdpJRHEKzffRPmnQMxGxqnygAQHQa9jK/Pq0tE14BX0DKzek1q8iLH70= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=P3Ys0OKV; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="P3Ys0OKV" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43d209dc2d3so32753995e9.3 for ; Tue, 01 Apr 2025 06:34:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743514473; x=1744119273; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hdHYlnm7oCIQrR8yQXWXDs/JGWUxXVNjmYJAO6rvmBw=; b=P3Ys0OKVDFQ1j+Mak/eyXxJZN3pgOrYgukznUbGWafaTgdn0fQ0WiSSnJ+kYic7Stv gEK/XKWP5LWS9dndmHkNbGIEwHwEJThGRQd3qk13HPJO2atjhQPOb5/FN3/PxT0G68nr HqWW9Fwzs1Q4HCHgf/ucKvkJF6t0Lh0a3HK1pT5nbj9o9+B5Gkg6gedm1xb/7yOIrEvg YQLB+7NIpc+Sk0NCCUo1abvk4TC8LZomiCNh9yb52nreuCAFQ1YnhsN89uFjnnMVp2iY bnp1DgoyZ9VqpNdY9HKXBgmuC9onQ0wE77ILUX0EP8BGTUqOTXalPPfwpmbs0e2bTKcC lBlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743514473; x=1744119273; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hdHYlnm7oCIQrR8yQXWXDs/JGWUxXVNjmYJAO6rvmBw=; b=VuOkn16SENcsY6QwgtEpMMx8uxE3/OnvktOHNXEkerX62PkkV5lYbRtruGiq88/rvV nv/jclsZRfRqUyTx/qo+Tax1TK8G5KY4R7crV491TovAZlLUcpb8NT9u4jPnRyr1HuRy fQruo6sYJQQekzmxji7vJDMqO6TneD9LvdZOCd5ki46q7k4xeCjd+PfRAKrRvsvCSzdz 1uCamxD5a+7li1hldrQRVzWS2Jc1xSpt/4zKJQy8VJ1Pnna/0ePZLsXdkUt4fmDDXM/w eJdej/s6myhqF9dE/4CsqFg7z9E1enavPCved7adtBZCntKYJidN4L3hIIBjxfEa3EP6 8Qjw== X-Gm-Message-State: AOJu0Yzy2VHGT+8EcMoU+XoPpiJMSowu+hsgc3P8UIHDWJ1WLL6mkDfp PxRWXZ6U25jJAHk+VytVnS0YPRaucdCWRDmNjZ829DX6lfjNj2wPfvS+1azmEthEEKDowQ== X-Google-Smtp-Source: AGHT+IG4x3Uac6Y88MEiNOljIjoS9AYRY3yNgjLa2c+ZBfDE53M+PxszbnOClR7HCFEi1qXaEnB2o3tn X-Received: from wmbjg21.prod.google.com ([2002:a05:600c:a015:b0:43c:f60a:4c59]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1e8f:b0:43c:ec4c:25b1 with SMTP id 5b1f17b1804b1-43db62bbe2amr107359285e9.23.1743514473105; Tue, 01 Apr 2025 06:34:33 -0700 (PDT) Date: Tue, 1 Apr 2025 15:34:18 +0200 In-Reply-To: <20250401133416.1436741-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401133416.1436741-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=3921; i=ardb@kernel.org; h=from:subject; bh=GJdt6JvaRkOjRUstkEclqIh4LycHXHj4RdQlgPesOr0=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3166iwvV57Pr1dd6HFzzxazmNikkIP+57Qks7WsI5Xq mmCzpc6SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwESMnjP8s/7fKHxHZMuuOMeV ya5siWtPzNrNuvd68kbD9dMKF6013Mfw30vUsddv1keFe16/5XnU/vcukHvNEM3SN8G/Nm2ZkUQ JCwA= X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401133416.1436741-9-ardb+git@google.com> Subject: [RFC PATCH 1/6] x86/boot/compressed: Merge local pgtable.h include into asm/boot.h From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Merge the local include "pgtable.h" -which declares the API of the 5-level paging trampoline- into so that its implementation in la57toggle.S as well as the calling code can be decoupled from the traditional decompressor. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/head_64.S | 1 - arch/x86/boot/compressed/la57toggle.S | 1 - arch/x86/boot/compressed/misc.c | 1 - arch/x86/boot/compressed/pgtable.h | 18 ------------------ arch/x86/boot/compressed/pgtable_64.c | 1 - arch/x86/include/asm/boot.h | 10 ++++++++++ 6 files changed, 10 insertions(+), 22 deletions(-) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/= head_64.S index eafd4f185e77..d9dab940ff62 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -35,7 +35,6 @@ #include #include #include -#include "pgtable.h" =20 /* * Fix alignment at 16 bytes. Following CONFIG_FUNCTION_ALIGNMENT will res= ult diff --git a/arch/x86/boot/compressed/la57toggle.S b/arch/x86/boot/compress= ed/la57toggle.S index 9ee002387eb1..370075b4d95b 100644 --- a/arch/x86/boot/compressed/la57toggle.S +++ b/arch/x86/boot/compressed/la57toggle.S @@ -5,7 +5,6 @@ #include #include #include -#include "pgtable.h" =20 /* * This is the 32-bit trampoline that will be copied over to low memory. It diff --git a/arch/x86/boot/compressed/misc.c b/arch/x86/boot/compressed/mis= c.c index 1cdcd4aaf395..94b5991da001 100644 --- a/arch/x86/boot/compressed/misc.c +++ b/arch/x86/boot/compressed/misc.c @@ -14,7 +14,6 @@ =20 #include "misc.h" #include "error.h" -#include "pgtable.h" #include "../string.h" #include "../voffset.h" #include diff --git a/arch/x86/boot/compressed/pgtable.h b/arch/x86/boot/compressed/= pgtable.h deleted file mode 100644 index 6d595abe06b3..000000000000 --- a/arch/x86/boot/compressed/pgtable.h +++ /dev/null @@ -1,18 +0,0 @@ -#ifndef BOOT_COMPRESSED_PAGETABLE_H -#define BOOT_COMPRESSED_PAGETABLE_H - -#define TRAMPOLINE_32BIT_SIZE (2 * PAGE_SIZE) - -#define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE -#define TRAMPOLINE_32BIT_CODE_SIZE 0xA0 - -#ifndef __ASSEMBLER__ - -extern unsigned long *trampoline_32bit; - -extern void trampoline_32bit_src(void *trampoline, bool enable_5lvl); - -extern const u16 trampoline_ljmp_imm_offset; - -#endif /* __ASSEMBLER__ */ -#endif /* BOOT_COMPRESSED_PAGETABLE_H */ diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compress= ed/pgtable_64.c index d8c5de40669d..5a6c7a190e5b 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -4,7 +4,6 @@ #include #include #include -#include "pgtable.h" #include "../string.h" #include "efi.h" =20 diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h index 3f02ff6d333d..02b23aa78955 100644 --- a/arch/x86/include/asm/boot.h +++ b/arch/x86/include/asm/boot.h @@ -74,6 +74,11 @@ # define BOOT_STACK_SIZE 0x1000 #endif =20 +#define TRAMPOLINE_32BIT_SIZE (2 * PAGE_SIZE) + +#define TRAMPOLINE_32BIT_CODE_OFFSET PAGE_SIZE +#define TRAMPOLINE_32BIT_CODE_SIZE 0xA0 + #ifndef __ASSEMBLER__ extern unsigned int output_len; extern const unsigned long kernel_text_size; @@ -83,6 +88,11 @@ unsigned long decompress_kernel(unsigned char *outbuf, u= nsigned long virt_addr, void (*error)(char *x)); =20 extern struct boot_params *boot_params_ptr; +extern unsigned long *trampoline_32bit; +extern const u16 trampoline_ljmp_imm_offset; + +void trampoline_32bit_src(void *trampoline, bool enable_5lvl); + #endif =20 #endif /* _ASM_X86_BOOT_H */ --=20 2.49.0.472.ge94155a9ec-goog From nobody Thu Apr 10 17:54:14 2025 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9AA42040AD for ; Tue, 1 Apr 2025 13:34:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514478; cv=none; b=BOe+R0M7n4HJb4ap5327h5TvO0NA14dy3ghd+jytIi6N1izhaPSC1ajstAtCrzRmOrGR76bELwAiAhyovkikmtR7g1idzNhoGJqk0In/Is33nFhCbXInFkCpeiTbN8L94UZSjtxnonEkwOUvE4RjvwRcxRJr9ppEIbacnRt7/+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514478; c=relaxed/simple; bh=jVv82q0VAwBPvYZ5/rIuosPXebvdrL2vHJ8IsDzi2y8=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Ch2HvKQNXAKLVwzmcLx/vwxVc9X4JVJJ7TYha96ySjtr8BC15wHaHN7SKTO6Q3WYOywgNIHLVmo0aKrAdQ9UgYPtMx0bh3A4Ebneon3CXo5zVCJPrXNAWSo0ADUDJAzhwZ1TGgGHdE56G461N+xgYy0ZoqjupHGiV+gmkfhYjFA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=G3NJwlJO; arc=none smtp.client-ip=209.85.221.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="G3NJwlJO" Received: by mail-wr1-f74.google.com with SMTP id ffacd0b85a97d-3912b54611dso2979889f8f.1 for ; Tue, 01 Apr 2025 06:34:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743514475; x=1744119275; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KkkgdoQS7XsAFt9UmFe68Xk/ys+reCGu9v6By+aRdFc=; b=G3NJwlJOSAQKohRI0zN9krFZa3FmDcluaJKAgvN1g8FyI0ApEtnH0LBhSZrGffrpea Tu0/n3UMX2doH+oIWFJ4tWXAfznTW8lms1NaXFQ6ld/yraNm+B/+EkvPMgytkUIjDvyn 6SDKgV5Ru82mVumNNfVj6BXeXBZUXbFeRjxoYQYkdAS400Aut2hE6OzB7b3gs5xfw1Dd oe0M+HpNP1rj1vVZsHVwvDgO12DjJSfUMt4nBc+6ZbwYFiNi1yaif3imZhC/8I/XIT1P kdfh6MR1o25QdbPMKg/abyXJyoZOgqtcHZKWsPzc4HI3mgVxGka+J+McDzV9hcwWK3aw AQwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743514475; x=1744119275; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KkkgdoQS7XsAFt9UmFe68Xk/ys+reCGu9v6By+aRdFc=; b=bZRORK7N1RK1nNkIa5IlYmn32VeLGZD1i5WNPc5i5gJQZmrY3oxsvh4vHXeaRLyWLQ DhacbW3/7PWtRUSGb5mtCDM9pBhO5P00HJyCbhnN6CZ6HWZooe6mKGDYbRzhBFFym89d HDiPWXyDp5F5++Ts3eTB9OpieiWCzEMHpcdo2eHiCtNRrI++ZN1cevZQRYULMwmHz3Dg CXE+n2J8xBN7XmZtsv7UG2p3u7OAZioV2otghZ3Tt9M0Doj/nf88MvCww5TqTi7CkLnQ a1psbnsbT8aqyInNinNojIvM/02cRCIiSljkyIbtiUV3T6INRoD9Vv7ShUGxXtEgpFcy GHIA== X-Gm-Message-State: AOJu0YzPXjytmo4hZqr/g7CFN1JavfwyLWY4KdQsSWCVrjlcAGtGoyey BnGkb2v2tgaTvL1mq5PO3J+IzL0efDgpXZeBxwIi3DU6tn6iyXEGT/3scoewmqou88gcgg== X-Google-Smtp-Source: AGHT+IH1Rwf0SPfvTDMLTXKr8r2VF8laxDRTirFJIzh3OocUBKb+W0dll+mMOctsSfKR+qI/IAtXpqrI X-Received: from wmco22.prod.google.com ([2002:a05:600c:a316:b0:43d:40ea:764d]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:250f:b0:39c:1f10:d294 with SMTP id ffacd0b85a97d-39c1f10d4e4mr4825842f8f.26.1743514475045; Tue, 01 Apr 2025 06:34:35 -0700 (PDT) Date: Tue, 1 Apr 2025 15:34:19 +0200 In-Reply-To: <20250401133416.1436741-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401133416.1436741-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2642; i=ardb@kernel.org; h=from:subject; bh=Rg+9UesfDX7KMCl1WWKPr+VDxB1ApETGwnITWSbmCXE=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3165iHHY4uD/afSnKPszOfHrj0WLV0h5/xr5jp/9fL/ hfe86qio5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEzEOJmR4WbK2rN+i8pqm1+u 5Ty+lHNTx/7ty24JfU27l7xzopd2+3OGfxq2ZvoFgR7zv5R8WvRAlHmLamzcifaAktn3uVdt0Hn iwgkA X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401133416.1436741-10-ardb+git@google.com> Subject: [RFC PATCH 2/6] x86/boot: Move 5-level paging trampoline into startup code From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The 5-level paging trampoline is used by both the EFI stub and the traditional decompressor. Move it out of the decompressor sources into the newly minted arch/x86/boot/startup/ sub-directory which will hold startup code that may be shared between the decompressor, the EFI stub and the kernel proper, and needs to tolerate being called during early boot, before the kernel virtual mapping has been created. This will allow the 5-level paging trampoline to be used by EFI boot images such as zboot that omit the traditional decompressor entirely. Signed-off-by: Ard Biesheuvel --- arch/x86/Makefile | 1 + arch/x86/boot/compressed/Makefile | 2 +- arch/x86/boot/startup/Makefile | 3 +++ arch/x86/boot/{compressed =3D> startup}/la57toggle.S | 0 4 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index 27efe2dc2aa8..c8703276e3e7 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -287,6 +287,7 @@ archprepare: $(cpufeaturemasks.hdr) ### # Kernel objects =20 +core-y +=3D arch/x86/boot/startup/ libs-y +=3D arch/x86/lib/ =20 # drivers-y are linked after core-y diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/M= akefile index 2eb63536c5d0..468e135de88e 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -98,7 +98,6 @@ ifdef CONFIG_X86_64 vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) +=3D $(obj)/mem_encrypt.o vmlinux-objs-y +=3D $(obj)/pgtable_64.o vmlinux-objs-$(CONFIG_AMD_MEM_ENCRYPT) +=3D $(obj)/sev.o - vmlinux-objs-y +=3D $(obj)/la57toggle.o endif =20 vmlinux-objs-$(CONFIG_ACPI) +=3D $(obj)/acpi.o @@ -107,6 +106,7 @@ vmlinux-objs-$(CONFIG_UNACCEPTED_MEMORY) +=3D $(obj)/me= m.o =20 vmlinux-objs-$(CONFIG_EFI) +=3D $(obj)/efi.o vmlinux-libs-$(CONFIG_EFI_STUB) +=3D $(objtree)/drivers/firmware/efi/libst= ub/lib.a +vmlinux-libs-$(CONFIG_X86_64) +=3D $(objtree)/arch/x86/boot/startup/lib.a =20 $(obj)/vmlinux: $(vmlinux-objs-y) $(vmlinux-libs-y) FORCE $(call if_changed,ld) diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile new file mode 100644 index 000000000000..03519ef4869d --- /dev/null +++ b/arch/x86/boot/startup/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +lib-$(CONFIG_X86_64) +=3D la57toggle.o diff --git a/arch/x86/boot/compressed/la57toggle.S b/arch/x86/boot/startup/= la57toggle.S similarity index 100% rename from arch/x86/boot/compressed/la57toggle.S rename to arch/x86/boot/startup/la57toggle.S --=20 2.49.0.472.ge94155a9ec-goog From nobody Thu Apr 10 17:54:14 2025 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABDE62046AF for ; Tue, 1 Apr 2025 13:34:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514480; cv=none; b=bxXKypHOjii72HMhh8oyCwqyZCdoiA+WjcTq3y5uSZN+uRNShJmIGS5wZlubqN6yyE0xcz5zlSCqW6IKF+OGJD6tNxjo4ufXXNrWo7tZvSbyEsVGwGYZvGlLW4ZxAwQ5lZED3fGuX+PN6sW0cBYhOviRqZkenEgFqVJyD8Rgeyg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514480; c=relaxed/simple; bh=Dja8KUvxraXjphff+I7V+sxjjRAswa3snLgGDyWKopQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pkIMKJ3MaBI+W/dwtwDDLw/OSrOQbiF/aBy5c2L2i30djCedtIzgNjaEU2x6jIJNu9VJYjTfE7skhg3zB0uC//gWIdWpxnwaxrK9IdAng1utkoIQzI1aFixEMap/q+Yyd0pfMBIKI3XqdOfRuaHQG/Y7BLkqnwWHl+nPo7MQ6qA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=142by7MQ; arc=none smtp.client-ip=209.85.221.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="142by7MQ" Received: by mail-wr1-f73.google.com with SMTP id ffacd0b85a97d-3913b2d355fso2136978f8f.1 for ; Tue, 01 Apr 2025 06:34:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743514477; x=1744119277; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SSnzu6xO+PITgBehw1ovPMVaSeDKcwED7ou+XOmk1EU=; b=142by7MQLtDaAxSMOJKuro4MmGKCqEpUgAT0Vi6f9HgCzIFEySs57V3/s3MbafQUQN yAlIrSW/4FUtRRldkxq2/XY7WQDuZNUwwYyFuFRscyJN/kFaAvV1cwVqdIeYprF8Agdd cHXrYCmB9FNgXkiJr6sHaMjfVN5COWEZ6fSVdnu9PJKU/EA8mm0lOLtDz5Lc65e9bYOQ VgBDm3s7LfHZ3tJznHXPTPOw5dQoCDcDlaVMdyNiM7mCweJPbv2t/xJsB5rHOPbFnqXR ltKOgcb0fLit5sS9XCV0W4b044YmhkTEUVr0jEji9lv0IyEdf6KpVYkA8vWHWIWeZL+x N/8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743514477; x=1744119277; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SSnzu6xO+PITgBehw1ovPMVaSeDKcwED7ou+XOmk1EU=; b=Xrv0mmLE66zIkBG5TBaceW8WWUs5is3IeLbkkO6aZVcuHDyhMnCamYzrlLq9pkuv5A EGE88HgmQXK/gjKhH3/no8Z6Xhtya+FFr6BYaU4ptCiWgNxLL9uv2NDLCzpKGf7aNpax H4t4Yju5vvsJRLFsWW46EZHS1f4rK+eMGkm+jl2GEyPoZlDwTbTgtoefvhZjWxuzeRHo NNrJQOvpUszYWZKyOUvNMpTMdOcW2BNgA0EHHOMGX+uNxMeFppgUrfyDK5OjZcOjo1Y/ TQX5G5wc+fHXOLB2t+gf97onGNa4zlD6aEaq0NGCuu9h72LwmjmVwvtQOxkNYq3esv3o qZsw== X-Gm-Message-State: AOJu0Yw7xpGQ2F+5cfgNG7i6n79CMYrIy2RqRD8LzwAVcI8sZfgFGrDM lFj9LrzG13hhb1108LeUisY/xQUphy7YCb38eAWRZaITFbp6Xt9KlRytpZFnQO9Y6kGnrw== X-Google-Smtp-Source: AGHT+IFjiaIDe4SdAmblJ6QP/93wu83vTH7+Yt+H5LymjrIx+mS8SStIVJUrIHaoPRfFDn3l5SseO91N X-Received: from wmbbi15.prod.google.com ([2002:a05:600c:3d8f:b0:43d:1873:dbaf]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a5d:64ec:0:b0:39a:ca05:5232 with SMTP id ffacd0b85a97d-39c120c7ccemr10348788f8f.5.1743514477097; Tue, 01 Apr 2025 06:34:37 -0700 (PDT) Date: Tue, 1 Apr 2025 15:34:20 +0200 In-Reply-To: <20250401133416.1436741-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401133416.1436741-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=1936; i=ardb@kernel.org; h=from:subject; bh=2ckatkKp0na78sWZkNAH6LE50dNZokOehx4dKsaghFc=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3169gPrqefP6/76BO5ufD00Rt+FpkR3G3ip9W15u1Ze /H78YrfHaUsDGIcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAirucY/oe8Ujjq1M47N3fz tsDpXMYyc2b02j+8ve6NlkKx2uYi836G/6Vi1m3ZXPfZPR+dKrgwecJjptchdd+Ffh/TmX2qP7m GlRsA X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401133416.1436741-11-ardb+git@google.com> Subject: [RFC PATCH 3/6] x86/boot: Move EFI mixed mode startup code back under arch/x86 From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Linus expressed a strong preference for arch-specific asm code (i.e., virtually all of it) to reside under arch/ rather than anywhere else. So move the EFI mixed mode startup code back, and put it under arch/x86/boot/startup/ where all shared x86 startup code is going to live. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/startup/Makefile = | 3 +++ drivers/firmware/efi/libstub/x86-mixed.S =3D> arch/x86/boot/startup/efi-mi= xed.S | 0 drivers/firmware/efi/libstub/Makefile = | 1 - 3 files changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 03519ef4869d..73946a3f6b3b 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -1,3 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 =20 +KBUILD_AFLAGS +=3D -D__DISABLE_EXPORTS + lib-$(CONFIG_X86_64) +=3D la57toggle.o +lib-$(CONFIG_EFI_MIXED) +=3D efi-mixed.o diff --git a/drivers/firmware/efi/libstub/x86-mixed.S b/arch/x86/boot/start= up/efi-mixed.S similarity index 100% rename from drivers/firmware/efi/libstub/x86-mixed.S rename to arch/x86/boot/startup/efi-mixed.S diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/l= ibstub/Makefile index d23a1b9fed75..2f173391b63d 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -85,7 +85,6 @@ lib-$(CONFIG_EFI_GENERIC_STUB) +=3D efi-stub.o string.o i= ntrinsics.o systable.o \ lib-$(CONFIG_ARM) +=3D arm32-stub.o lib-$(CONFIG_ARM64) +=3D kaslr.o arm64.o arm64-stub.o smbios.o lib-$(CONFIG_X86) +=3D x86-stub.o smbios.o -lib-$(CONFIG_EFI_MIXED) +=3D x86-mixed.o lib-$(CONFIG_X86_64) +=3D x86-5lvl.o lib-$(CONFIG_RISCV) +=3D kaslr.o riscv.o riscv-stub.o lib-$(CONFIG_LOONGARCH) +=3D loongarch.o loongarch-stub.o --=20 2.49.0.472.ge94155a9ec-goog From nobody Thu Apr 10 17:54:14 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F19C9204C06 for ; Tue, 1 Apr 2025 13:34:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514483; cv=none; b=tPD4dvzFLecGZ2mOr/Ywavm6nxwCANpGGllqev3Nm62jhfCFhimE1pe0xrOHhgfdY/91sMAui44czOHFQf68tqwV7tQ/RjZU5Pn6xZKgrUCOjkSO+gPRVbgParqvW4/sEWv3RHw/WTrDPwJKVfF3krtrGEHZFl20cVrFdyxulp4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514483; c=relaxed/simple; bh=13dN6LyMDIRJPZtbLc4Qnlwfujaf8mnCxnGVXar+9i0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=pazcEQkQJYUhlPAOz3fwPg49FrPTTAfpzltGZE62FI2LGEEx0HdTG8RcejraeT/WKz1xw/EYqA+cjIX1GnjGyIRcI9kAzl0mlwCPIqO8PGNHjssQokUDMyYzvUd6fukDRA4gXHorogudobW7YKPwkLQGJDES6n0HL2K3qMAAQag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=QwueR8CX; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="QwueR8CX" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-43ceb011ea5so39710685e9.2 for ; Tue, 01 Apr 2025 06:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743514479; x=1744119279; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=iwpNwL234IC9yWIvSwlZhIgKkylOGfUY/NpW29/ZOa0=; b=QwueR8CXE1SHDXlMcssYhXj9RaibADn4mGnqJICY/KwI20FhSua7niARSlPKhVvhZp zAUMeZC49YZ/KUhutUZDXFdMnk0Ekl3uKt1wsMctwtl/LUYQwD//quAS4M8JhFBU9Oqu /yY+yoEu0T41n0gqQs/ZA+OFvUV7PS2REsLmpAmB6MO4AUSiljQ6raRsH2fhbakaY+Vz 45uehrYRKMmAWFBQUjtFviYl2AXZwsVwQVoe0bfCjiZnhhY6riwnd0BV6iamCN1meiPC vAT98+qM6SkvENKDLrepy23uT40hksxvVdsj60/HCGNIG+FKLGXkNWcVduZnIOJ90LUJ f2sA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743514479; x=1744119279; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iwpNwL234IC9yWIvSwlZhIgKkylOGfUY/NpW29/ZOa0=; b=u28vQaBIYy0Ta4IzxY9nLqHZiOfhZiAdLqgqBqgXET2/z7LoaWgnJdJS1gKW3aadev 783/9ChXQKpbovn7IK3GnahNqZqGtVY4szcAx6D+Z5kuPICITYZTuBv+khT6h81w4KOA 3eV+nNkc4Jt1fwYJ0tzOQZu6lpqatiahlOeotTX1O7Lv6Xmh4vZHsm6t+ikmHsd6WhAn cwUt4oN1tbhVSslIfWKCbt1H+ggnVVZZDlp9gzfR9yY551nH5YuV3zMABr3xyX/h3jCA ARCoutZgpBJtkZiZ+8nm1mnItWh3Nmee/TIBIX+kb6ks0JiDKhx1BJPVH/vl6iMJxVDc mLNQ== X-Gm-Message-State: AOJu0YwdssF5c3oJjkEM5u3NXnYYg6lGCx9DjSOC75shXykwTNjbDyQn 1WP6vEGN7G00cx4M3VXrD/Mkm4LlQU/K68eDtcKMfsF00sP/VSJdNAVMKYvIULMw7JwLcA== X-Google-Smtp-Source: AGHT+IG4QK2ZXVU8OaDJZrpd4jbGOh2SrGi1jhRE/dSVG0kTZPTHVEXeqFIKnFD0vg8+XQ7RAPuxGyya X-Received: from wmbhc8.prod.google.com ([2002:a05:600c:8708:b0:43d:8244:7f6d]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:3c8a:b0:43d:45a:8fca with SMTP id 5b1f17b1804b1-43db62bf4e4mr119197335e9.30.1743514479363; Tue, 01 Apr 2025 06:34:39 -0700 (PDT) Date: Tue, 1 Apr 2025 15:34:21 +0200 In-Reply-To: <20250401133416.1436741-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401133416.1436741-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=7788; i=ardb@kernel.org; h=from:subject; bh=7CqWOqUn0owfJx93vqlZ0MaFHl2+UzXGIVwFFUz3Lwk=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3167hzdUZKs0sL7l6c+eF38RwBCUHBG/u/zop/KD77r c38xTwMHaUsDGIcDLJiiiwCs/++23l6olSt8yxZmDmsTCBDGLg4BWAiJxgZ/qnz/Ft1teKvVpBB YjCn2DvGlkkRuyR2btO1Nw99bLdoyQ6Gf1oK/bUCaofNbidkMIkt6b2Wtmy5vc02t19T3QVfyxy +wgEA X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401133416.1436741-12-ardb+git@google.com> Subject: [RFC PATCH 4/6] x86/boot: Move early GDT/IDT setup code into startup/ From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Move the early GDT/IDT setup code that runs long before the kernel virtual mapping is up into arch/x86/boot/startup/, and build it in a way that ensures that the code tolerates being called from the 1:1 mapping of memory. This allows the RIP_REL_REF() macro uses to be dropped, and removes the need for emitting the code into the special .head.text section. Also tweak the sed symbol matching pattern in the decompressor to match on lower case 't' or 'b', as these will be emitted by Clang for symbols with hidden linkage. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/Makefile | 2 +- arch/x86/boot/startup/Makefile | 15 ++++ arch/x86/boot/startup/gdt_idt.c | 82 ++++++++++++++++++++ arch/x86/kernel/head64.c | 74 ------------------ 4 files changed, 98 insertions(+), 75 deletions(-) diff --git a/arch/x86/boot/compressed/Makefile b/arch/x86/boot/compressed/M= akefile index 468e135de88e..48541cf54790 100644 --- a/arch/x86/boot/compressed/Makefile +++ b/arch/x86/boot/compressed/Makefile @@ -74,7 +74,7 @@ LDFLAGS_vmlinux +=3D -T hostprogs :=3D mkpiggy HOST_EXTRACFLAGS +=3D -I$(srctree)/tools/include =20 -sed-voffset :=3D -e 's/^\([0-9a-fA-F]*\) [ABCDGRSTVW] \(_text\|__start_rod= ata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' +sed-voffset :=3D -e 's/^\([0-9a-fA-F]*\) [ABbCDGRSTtVW] \(_text\|__start_r= odata\|__bss_start\|_end\)$$/\#define VO_\2 _AC(0x\1,UL)/p' =20 quiet_cmd_voffset =3D VOFFSET $@ cmd_voffset =3D $(NM) $< | sed -n $(sed-voffset) > $@ diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 73946a3f6b3b..34b324cbd5a4 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -1,6 +1,21 @@ # SPDX-License-Identifier: GPL-2.0 =20 KBUILD_AFLAGS +=3D -D__DISABLE_EXPORTS +KBUILD_CFLAGS +=3D -D__DISABLE_EXPORTS -mcmodel=3Dsmall -fPIC \ + -Os -DDISABLE_BRANCH_PROFILING \ + $(DISABLE_STACKLEAK_PLUGIN) \ + -fno-stack-protector -D__NO_FORTIFY \ + -include $(srctree)/include/linux/hidden.h + +# disable ftrace hooks +KBUILD_CFLAGS :=3D $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) +KASAN_SANITIZE :=3D n +KCSAN_SANITIZE :=3D n +KMSAN_SANITIZE :=3D n +UBSAN_SANITIZE :=3D n +KCOV_INSTRUMENT :=3D n + +obj-$(CONFIG_X86_64) +=3D gdt_idt.o =20 lib-$(CONFIG_X86_64) +=3D la57toggle.o lib-$(CONFIG_EFI_MIXED) +=3D efi-mixed.o diff --git a/arch/x86/boot/startup/gdt_idt.c b/arch/x86/boot/startup/gdt_id= t.c new file mode 100644 index 000000000000..b382d5db2586 --- /dev/null +++ b/arch/x86/boot/startup/gdt_idt.c @@ -0,0 +1,82 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +#include +#include +#include +#include + +/* + * Data structures and code used for IDT setup in head_64.S. The bringup-I= DT is + * used until the idt_table takes over. On the boot CPU this happens in + * x86_64_start_kernel(), on secondary CPUs in start_secondary(). In both = cases + * this happens in the functions called from head_64.S. + * + * The idt_table can't be used that early because all the code modifying i= t is + * in idt.c and can be instrumented by tracing or KASAN, which both don't = work + * during early CPU bringup. Also the idt_table has the runtime vectors + * configured which require certain CPU state to be setup already (like TS= S), + * which also hasn't happened yet in early CPU bringup. + */ +static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_d= ata; + +/* This may run while still in the direct mapping */ +static void startup_64_load_idt(void *vc_handler) +{ + struct desc_ptr desc =3D { + .address =3D (unsigned long)bringup_idt_table, + .size =3D sizeof(bringup_idt_table) - 1, + }; + struct idt_data data; + gate_desc idt_desc; + + /* @vc_handler is set only for a VMM Communication Exception */ + if (vc_handler) { + init_idt_data(&data, X86_TRAP_VC, vc_handler); + idt_init_desc(&idt_desc, &data); + native_write_idt_entry((gate_desc *)desc.address, X86_TRAP_VC, &idt_desc= ); + } + + native_load_idt(&desc); +} + +/* This is used when running on kernel addresses */ +void early_setup_idt(void) +{ + void *handler =3D NULL; + + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { + setup_ghcb(); + handler =3D vc_boot_ghcb; + } + + startup_64_load_idt(handler); +} + +/* + * Setup boot CPU state needed before kernel switches to virtual addresses. + */ +void __init startup_64_setup_gdt_idt(void) +{ + void *handler =3D NULL; + + struct desc_ptr startup_gdt_descr =3D { + .address =3D (__force unsigned long)gdt_page.gdt, + .size =3D GDT_SIZE - 1, + }; + + /* Load GDT */ + native_load_gdt(&startup_gdt_descr); + + /* New GDT is live - reload data segment registers */ + asm volatile("movl %%eax, %%ds\n" + "movl %%eax, %%ss\n" + "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory"); + + if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) + handler =3D vc_no_ghcb; + + startup_64_load_idt(handler); +} diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index fa9b6339975f..5b993b545c7e 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -512,77 +512,3 @@ void __init __noreturn x86_64_start_reservations(char = *real_mode_data) =20 start_kernel(); } - -/* - * Data structures and code used for IDT setup in head_64.S. The bringup-I= DT is - * used until the idt_table takes over. On the boot CPU this happens in - * x86_64_start_kernel(), on secondary CPUs in start_secondary(). In both = cases - * this happens in the functions called from head_64.S. - * - * The idt_table can't be used that early because all the code modifying i= t is - * in idt.c and can be instrumented by tracing or KASAN, which both don't = work - * during early CPU bringup. Also the idt_table has the runtime vectors - * configured which require certain CPU state to be setup already (like TS= S), - * which also hasn't happened yet in early CPU bringup. - */ -static gate_desc bringup_idt_table[NUM_EXCEPTION_VECTORS] __page_aligned_d= ata; - -/* This may run while still in the direct mapping */ -static void __head startup_64_load_idt(void *vc_handler) -{ - struct desc_ptr desc =3D { - .address =3D (unsigned long)&RIP_REL_REF(bringup_idt_table), - .size =3D sizeof(bringup_idt_table) - 1, - }; - struct idt_data data; - gate_desc idt_desc; - - /* @vc_handler is set only for a VMM Communication Exception */ - if (vc_handler) { - init_idt_data(&data, X86_TRAP_VC, vc_handler); - idt_init_desc(&idt_desc, &data); - native_write_idt_entry((gate_desc *)desc.address, X86_TRAP_VC, &idt_desc= ); - } - - native_load_idt(&desc); -} - -/* This is used when running on kernel addresses */ -void early_setup_idt(void) -{ - void *handler =3D NULL; - - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { - setup_ghcb(); - handler =3D vc_boot_ghcb; - } - - startup_64_load_idt(handler); -} - -/* - * Setup boot CPU state needed before kernel switches to virtual addresses. - */ -void __head startup_64_setup_gdt_idt(void) -{ - struct desc_struct *gdt =3D (void *)(__force unsigned long)gdt_page.gdt; - void *handler =3D NULL; - - struct desc_ptr startup_gdt_descr =3D { - .address =3D (unsigned long)&RIP_REL_REF(*gdt), - .size =3D GDT_SIZE - 1, - }; - - /* Load GDT */ - native_load_gdt(&startup_gdt_descr); - - /* New GDT is live - reload data segment registers */ - asm volatile("movl %%eax, %%ds\n" - "movl %%eax, %%ss\n" - "movl %%eax, %%es\n" : : "a"(__KERNEL_DS) : "memory"); - - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) - handler =3D &RIP_REL_REF(vc_no_ghcb); - - startup_64_load_idt(handler); -} --=20 2.49.0.472.ge94155a9ec-goog From nobody Thu Apr 10 17:54:14 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D04C5204690 for ; Tue, 1 Apr 2025 13:34:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514485; cv=none; b=cJj3YCcobaBudSjNdrrINdYOvdsF07H7XJIksRVeJmygRxr1EYlgaGzUM5cAbIjaYF4Fs4kHqQm9BgCHF/DAbaCLHYvHIj36/FXeDg4HMQRaeaffNt4Lyo+TltfD54JmYXYZyvjN24sFEZpbeT71n707iKEAcLdPP1mGJU0+IEU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514485; c=relaxed/simple; bh=N9vA4EpIqk8loSj9DhnOdzc82juHV1lnon569PilmDg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=KioqS2Te5Ee9HDwicZPqVC7RMP0ck5fE1hPYo7clSnmb0QfuZQe8eF+E4voXQVrQd2tHKcHcfWH9AEGcEinmYtqBamYZ5xZqd80WIdDGI65zZJAZAniTQDDIy8Q3w21PQiinbfrXGRkak9anamNbo+CxSxlLpD33btbopIMx2Pk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Sv+bkqfM; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Sv+bkqfM" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43d733063cdso47012435e9.0 for ; Tue, 01 Apr 2025 06:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743514481; x=1744119281; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=YATu9zXF0K3u0eA9uumphTz242jaGnKCjKZiC2CAoPc=; b=Sv+bkqfMz3PZo+prc7c6ldZlAqJpdi9Kw4KlSId+h0GancqAzo53Y79yG/ivnbLo9b CcKkXQ+L7iaLVBcVBStLPpRslLm5IGCh9snC1fKdVaH0kbJVsfAViXJKuF4nsbUI4T6O TNtz/kj1kz6tiZyyJLNZoAoOvDptox8Sc1NS1I3GG5jIe/8ANDtxUT/6zYXfHtftaDws cXgDcGQVS49NsDOa0wI9qdqDALqkaco+PLcM5E4WGpAlKygJubfuq2qv75KRvEdrCWWG mxa3v7gFas/77Vb0KuF6/OzBS0rMrMKFwC+rVkXcH0P482XKVYrAIbDw3YDhdGH8p9Ez OEYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743514481; x=1744119281; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YATu9zXF0K3u0eA9uumphTz242jaGnKCjKZiC2CAoPc=; b=T5J50JhtTIoNNBQruHsnll+iGU4nPGgyibZZWQePqovVYYCbZ7L0nmiaa57oKD+5R1 TK3+hhHXwlyuI23ldGJL9WjhMHU9Ua0ROA4BXb+slIZWxbdwc2B6QKVC9XxaxE2Ns5Dw xi8Rsnbvc44V9LojJM4sVxkKuUJENnbeqYu32KUCoGjwHbQi26sUB7bxkwXx5/hcdpv+ BeBjbui81nzJeKa3wYYi1mqtnf/CtiZPXYdrBe+ttt0fDO5D0nQByJPHBtBVbmzMtVry 8boR5iIjWlYiqtsudUUyli1/iu6jRXUMIESnH5VK82jbH0wcfo/nE1PRKhMtCcBqyUaA HnkQ== X-Gm-Message-State: AOJu0YzzVJU5IlCrnj61UEUXye0OYEmQv6Vj43CtB9tyovoSZ6vUeY3I iikqAJnDil6dwQSV/K+YcxkCpQhv36XeGbxXvyauH5ef1WZMsPUPIMJXHzfrtiC3botQhg== X-Google-Smtp-Source: AGHT+IHOHzfjMGFyukFYtPOemaeqC5O37H99HY0TU2Cv08Z8vwBJNdwTNIm9ON3XTcj+T/Ag710uHOdB X-Received: from wmoq17.prod.google.com ([2002:a05:600c:46d1:b0:43b:c336:7b29]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4e06:b0:43d:745a:5a50 with SMTP id 5b1f17b1804b1-43db62bfe4amr101859055e9.19.1743514481353; Tue, 01 Apr 2025 06:34:41 -0700 (PDT) Date: Tue, 1 Apr 2025 15:34:22 +0200 In-Reply-To: <20250401133416.1436741-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401133416.1436741-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=17256; i=ardb@kernel.org; h=from:subject; bh=a6z4ADsTbmwulvK5aAyxZaInG5zR2MdL4tUgODMxkTU=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3163jbmbO/ftt5oq1F14Hj8ff/4f9mXTz7cY3I/ffKv 4VU5oQad5SyMIhxMMiKKbIIzP77bufpiVK1zrNkYeawMoEMYeDiFICJbLRhZLjKuGt6SP1fL5Np uw6WXTxg+21xihf3z1p+9wfF1nydbzYy/M/vf3Ryxwphd5uPWWJnr79m3i7jXniyaH2mCNsO74U n1rIAAA== X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401133416.1436741-13-ardb+git@google.com> Subject: [RFC PATCH 5/6] x86/boot: Move early kernel mapping code into startup/ From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel The startup code that constructs the kernel virtual mapping runs from the 1:1 mapping of memory itself, and therefore, cannot use absolute symbol references. Move this code into a separate source file under arch/x86/boot/startup/ where all such code will be kept from now on. Since all code here is constructed in a manner that ensures that it tolerates running from the 1:1 mapping of memory, any uses of the RIP_REL_REF() macro can be dropped, along with __head annotations for placing this code in a dedicated startup section. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/startup/Makefile | 2 +- arch/x86/boot/startup/map_kernel.c | 232 ++++++++++++++++++++ arch/x86/kernel/head64.c | 228 +------------------ 3 files changed, 234 insertions(+), 228 deletions(-) diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 34b324cbd5a4..01423063fec2 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -15,7 +15,7 @@ KMSAN_SANITIZE :=3D n UBSAN_SANITIZE :=3D n KCOV_INSTRUMENT :=3D n =20 -obj-$(CONFIG_X86_64) +=3D gdt_idt.o +obj-$(CONFIG_X86_64) +=3D gdt_idt.o map_kernel.o =20 lib-$(CONFIG_X86_64) +=3D la57toggle.o lib-$(CONFIG_EFI_MIXED) +=3D efi-mixed.o diff --git a/arch/x86/boot/startup/map_kernel.c b/arch/x86/boot/startup/map= _kernel.c new file mode 100644 index 000000000000..ba856be92d10 --- /dev/null +++ b/arch/x86/boot/startup/map_kernel.c @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include +#include +#include + +#include +#include +#include + +extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD]; +extern unsigned int next_early_pgt; + +#ifdef CONFIG_X86_5LEVEL +unsigned int __pgtable_l5_enabled __ro_after_init; +unsigned int pgdir_shift __ro_after_init =3D 39; +EXPORT_SYMBOL(pgdir_shift); +unsigned int ptrs_per_p4d __ro_after_init =3D 1; +EXPORT_SYMBOL(ptrs_per_p4d); +#endif + +#ifdef CONFIG_DYNAMIC_MEMORY_LAYOUT +unsigned long page_offset_base __ro_after_init =3D __PAGE_OFFSET_BASE_L4; +EXPORT_SYMBOL(page_offset_base); +unsigned long vmalloc_base __ro_after_init =3D __VMALLOC_BASE_L4; +EXPORT_SYMBOL(vmalloc_base); +unsigned long vmemmap_base __ro_after_init =3D __VMEMMAP_BASE_L4; +EXPORT_SYMBOL(vmemmap_base); +#endif + +static inline bool check_la57_support(void) +{ + if (!IS_ENABLED(CONFIG_X86_5LEVEL)) + return false; + + /* + * 5-level paging is detected and enabled at kernel decompression + * stage. Only check if it has been enabled there. + */ + if (!(native_read_cr4() & X86_CR4_LA57)) + return false; + + __pgtable_l5_enabled =3D 1; + pgdir_shift =3D 48; + ptrs_per_p4d =3D 512; + page_offset_base =3D __PAGE_OFFSET_BASE_L5; + vmalloc_base =3D __VMALLOC_BASE_L5; + vmemmap_base =3D __VMEMMAP_BASE_L5; + + return true; +} + +static unsigned long sme_postprocess_startup(struct boot_params *bp, + pmdval_t *pmd, + unsigned long p2v_offset) +{ + unsigned long paddr, paddr_end; + int i; + + /* Encrypt the kernel and related (if SME is active) */ + sme_encrypt_kernel(bp); + + /* + * Clear the memory encryption mask from the .bss..decrypted section. + * The bss section will be memset to zero later in the initialization so + * there is no need to zero it after changing the memory encryption + * attribute. + */ + if (sme_get_me_mask()) { + paddr =3D (unsigned long)__start_bss_decrypted; + paddr_end =3D (unsigned long)__end_bss_decrypted; + + for (; paddr < paddr_end; paddr +=3D PMD_SIZE) { + /* + * On SNP, transition the page to shared in the RMP table so that + * it is consistent with the page table attribute change. + * + * __start_bss_decrypted has a virtual address in the high range + * mapping (kernel .text). PVALIDATE, by way of + * early_snp_set_memory_shared(), requires a valid virtual + * address but the kernel is currently running off of the identity + * mapping so use the PA to get a *currently* valid virtual address. + */ + early_snp_set_memory_shared(paddr, paddr, PTRS_PER_PMD); + + i =3D pmd_index(paddr - p2v_offset); + pmd[i] -=3D sme_get_me_mask(); + } + } + + /* + * Return the SME encryption mask (if SME is active) to be used as a + * modifier for the initial pgdir entry programmed into CR3. + */ + return sme_get_me_mask(); +} + +unsigned long __init __startup_64(unsigned long p2v_offset, + struct boot_params *bp) +{ + pmd_t (*early_pgts)[PTRS_PER_PMD] =3D early_dynamic_pgts; + unsigned long physaddr =3D (unsigned long)_text; + unsigned long va_text, va_end; + unsigned long pgtable_flags; + unsigned long load_delta; + pgdval_t *pgd; + p4dval_t *p4d; + pudval_t *pud; + pmdval_t *pmd, pmd_entry; + bool la57; + int i; + + la57 =3D check_la57_support(); + + /* Is the address too large? */ + if (physaddr >> MAX_PHYSMEM_BITS) + for (;;); + + /* + * Compute the delta between the address I am compiled to run at + * and the address I am actually running at. + */ + phys_base =3D load_delta =3D __START_KERNEL_map + p2v_offset; + + /* Is the address not 2M aligned? */ + if (load_delta & ~PMD_MASK) + for (;;); + + va_text =3D physaddr - p2v_offset; + va_end =3D (unsigned long)_end - p2v_offset; + + /* Include the SME encryption mask in the fixup value */ + load_delta +=3D sme_get_me_mask(); + + /* Fixup the physical addresses in the page table */ + + pgd =3D &early_top_pgt[0].pgd; + pgd[pgd_index(__START_KERNEL_map)] +=3D load_delta; + + if (IS_ENABLED(CONFIG_X86_5LEVEL) && la57) { + p4d =3D (p4dval_t *)level4_kernel_pgt; + p4d[MAX_PTRS_PER_P4D - 1] +=3D load_delta; + + pgd[pgd_index(__START_KERNEL_map)] =3D (pgdval_t)p4d | _PAGE_TABLE; + } + + level3_kernel_pgt[PTRS_PER_PUD - 2].pud +=3D load_delta; + level3_kernel_pgt[PTRS_PER_PUD - 1].pud +=3D load_delta; + + for (i =3D FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--) + level2_fixmap_pgt[i].pmd +=3D load_delta; + + /* + * Set up the identity mapping for the switchover. These + * entries should *NOT* have the global bit set! This also + * creates a bunch of nonsense entries but that is fine -- + * it avoids problems around wraparound. + */ + + pud =3D &early_pgts[0]->pmd; + pmd =3D &early_pgts[1]->pmd; + next_early_pgt =3D 2; + + pgtable_flags =3D _KERNPG_TABLE_NOENC + sme_get_me_mask(); + + if (la57) { + p4d =3D &early_pgts[next_early_pgt++]->pmd; + + i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; + pgd[i + 0] =3D (pgdval_t)p4d + pgtable_flags; + pgd[i + 1] =3D (pgdval_t)p4d + pgtable_flags; + + i =3D physaddr >> P4D_SHIFT; + p4d[(i + 0) % PTRS_PER_P4D] =3D (pgdval_t)pud + pgtable_flags; + p4d[(i + 1) % PTRS_PER_P4D] =3D (pgdval_t)pud + pgtable_flags; + } else { + i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; + pgd[i + 0] =3D (pgdval_t)pud + pgtable_flags; + pgd[i + 1] =3D (pgdval_t)pud + pgtable_flags; + } + + i =3D physaddr >> PUD_SHIFT; + pud[(i + 0) % PTRS_PER_PUD] =3D (pudval_t)pmd + pgtable_flags; + pud[(i + 1) % PTRS_PER_PUD] =3D (pudval_t)pmd + pgtable_flags; + + pmd_entry =3D __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL; + /* Filter out unsupported __PAGE_KERNEL_* bits: */ + pmd_entry &=3D __supported_pte_mask; + pmd_entry +=3D sme_get_me_mask(); + pmd_entry +=3D physaddr; + + for (i =3D 0; i < DIV_ROUND_UP(va_end - va_text, PMD_SIZE); i++) { + int idx =3D i + (physaddr >> PMD_SHIFT); + + pmd[idx % PTRS_PER_PMD] =3D pmd_entry + i * PMD_SIZE; + } + + /* + * Fixup the kernel text+data virtual addresses. Note that + * we might write invalid pmds, when the kernel is relocated + * cleanup_highmap() fixes this up along with the mappings + * beyond _end. + * + * Only the region occupied by the kernel image has so far + * been checked against the table of usable memory regions + * provided by the firmware, so invalidate pages outside that + * region. A page table entry that maps to a reserved area of + * memory would allow processor speculation into that area, + * and on some hardware (particularly the UV platform) even + * speculative access to some reserved areas is caught as an + * error, causing the BIOS to halt the system. + */ + + pmd =3D &level2_kernel_pgt[0].pmd; + + /* invalidate pages before the kernel image */ + for (i =3D 0; i < pmd_index(va_text); i++) + pmd[i] &=3D ~_PAGE_PRESENT; + + /* fixup pages that are part of the kernel image */ + for (; i <=3D pmd_index(va_end); i++) + if (pmd[i] & _PAGE_PRESENT) + pmd[i] +=3D load_delta; + + /* invalidate pages after the kernel image */ + for (; i < PTRS_PER_PMD; i++) + pmd[i] &=3D ~_PAGE_PRESENT; + + return sme_postprocess_startup(bp, pmd, p2v_offset); +} diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 5b993b545c7e..9afb123a8676 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -47,235 +47,9 @@ * Manage page tables very early on. */ extern pmd_t early_dynamic_pgts[EARLY_DYNAMIC_PAGE_TABLES][PTRS_PER_PMD]; -static unsigned int __initdata next_early_pgt; +unsigned int __initdata next_early_pgt; pmdval_t early_pmd_flags =3D __PAGE_KERNEL_LARGE & ~(_PAGE_GLOBAL | _PAGE_= NX); =20 -#ifdef CONFIG_X86_5LEVEL -unsigned int __pgtable_l5_enabled __ro_after_init; -unsigned int pgdir_shift __ro_after_init =3D 39; -EXPORT_SYMBOL(pgdir_shift); -unsigned int ptrs_per_p4d __ro_after_init =3D 1; -EXPORT_SYMBOL(ptrs_per_p4d); -#endif - -#ifdef CONFIG_DYNAMIC_MEMORY_LAYOUT -unsigned long page_offset_base __ro_after_init =3D __PAGE_OFFSET_BASE_L4; -EXPORT_SYMBOL(page_offset_base); -unsigned long vmalloc_base __ro_after_init =3D __VMALLOC_BASE_L4; -EXPORT_SYMBOL(vmalloc_base); -unsigned long vmemmap_base __ro_after_init =3D __VMEMMAP_BASE_L4; -EXPORT_SYMBOL(vmemmap_base); -#endif - -static inline bool check_la57_support(void) -{ - if (!IS_ENABLED(CONFIG_X86_5LEVEL)) - return false; - - /* - * 5-level paging is detected and enabled at kernel decompression - * stage. Only check if it has been enabled there. - */ - if (!(native_read_cr4() & X86_CR4_LA57)) - return false; - - RIP_REL_REF(__pgtable_l5_enabled) =3D 1; - RIP_REL_REF(pgdir_shift) =3D 48; - RIP_REL_REF(ptrs_per_p4d) =3D 512; - RIP_REL_REF(page_offset_base) =3D __PAGE_OFFSET_BASE_L5; - RIP_REL_REF(vmalloc_base) =3D __VMALLOC_BASE_L5; - RIP_REL_REF(vmemmap_base) =3D __VMEMMAP_BASE_L5; - - return true; -} - -static unsigned long __head sme_postprocess_startup(struct boot_params *bp, - pmdval_t *pmd, - unsigned long p2v_offset) -{ - unsigned long paddr, paddr_end; - int i; - - /* Encrypt the kernel and related (if SME is active) */ - sme_encrypt_kernel(bp); - - /* - * Clear the memory encryption mask from the .bss..decrypted section. - * The bss section will be memset to zero later in the initialization so - * there is no need to zero it after changing the memory encryption - * attribute. - */ - if (sme_get_me_mask()) { - paddr =3D (unsigned long)&RIP_REL_REF(__start_bss_decrypted); - paddr_end =3D (unsigned long)&RIP_REL_REF(__end_bss_decrypted); - - for (; paddr < paddr_end; paddr +=3D PMD_SIZE) { - /* - * On SNP, transition the page to shared in the RMP table so that - * it is consistent with the page table attribute change. - * - * __start_bss_decrypted has a virtual address in the high range - * mapping (kernel .text). PVALIDATE, by way of - * early_snp_set_memory_shared(), requires a valid virtual - * address but the kernel is currently running off of the identity - * mapping so use the PA to get a *currently* valid virtual address. - */ - early_snp_set_memory_shared(paddr, paddr, PTRS_PER_PMD); - - i =3D pmd_index(paddr - p2v_offset); - pmd[i] -=3D sme_get_me_mask(); - } - } - - /* - * Return the SME encryption mask (if SME is active) to be used as a - * modifier for the initial pgdir entry programmed into CR3. - */ - return sme_get_me_mask(); -} - -/* Code in __startup_64() can be relocated during execution, but the compi= ler - * doesn't have to generate PC-relative relocations when accessing globals= from - * that function. Clang actually does not generate them, which leads to - * boot-time crashes. To work around this problem, every global pointer mu= st - * be accessed using RIP_REL_REF(). Kernel virtual addresses can be determ= ined - * by subtracting p2v_offset from the RIP-relative address. - */ -unsigned long __head __startup_64(unsigned long p2v_offset, - struct boot_params *bp) -{ - pmd_t (*early_pgts)[PTRS_PER_PMD] =3D RIP_REL_REF(early_dynamic_pgts); - unsigned long physaddr =3D (unsigned long)&RIP_REL_REF(_text); - unsigned long va_text, va_end; - unsigned long pgtable_flags; - unsigned long load_delta; - pgdval_t *pgd; - p4dval_t *p4d; - pudval_t *pud; - pmdval_t *pmd, pmd_entry; - bool la57; - int i; - - la57 =3D check_la57_support(); - - /* Is the address too large? */ - if (physaddr >> MAX_PHYSMEM_BITS) - for (;;); - - /* - * Compute the delta between the address I am compiled to run at - * and the address I am actually running at. - */ - load_delta =3D __START_KERNEL_map + p2v_offset; - RIP_REL_REF(phys_base) =3D load_delta; - - /* Is the address not 2M aligned? */ - if (load_delta & ~PMD_MASK) - for (;;); - - va_text =3D physaddr - p2v_offset; - va_end =3D (unsigned long)&RIP_REL_REF(_end) - p2v_offset; - - /* Include the SME encryption mask in the fixup value */ - load_delta +=3D sme_get_me_mask(); - - /* Fixup the physical addresses in the page table */ - - pgd =3D &RIP_REL_REF(early_top_pgt)->pgd; - pgd[pgd_index(__START_KERNEL_map)] +=3D load_delta; - - if (IS_ENABLED(CONFIG_X86_5LEVEL) && la57) { - p4d =3D (p4dval_t *)&RIP_REL_REF(level4_kernel_pgt); - p4d[MAX_PTRS_PER_P4D - 1] +=3D load_delta; - - pgd[pgd_index(__START_KERNEL_map)] =3D (pgdval_t)p4d | _PAGE_TABLE; - } - - RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 2].pud +=3D load_delta; - RIP_REL_REF(level3_kernel_pgt)[PTRS_PER_PUD - 1].pud +=3D load_delta; - - for (i =3D FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--) - RIP_REL_REF(level2_fixmap_pgt)[i].pmd +=3D load_delta; - - /* - * Set up the identity mapping for the switchover. These - * entries should *NOT* have the global bit set! This also - * creates a bunch of nonsense entries but that is fine -- - * it avoids problems around wraparound. - */ - - pud =3D &early_pgts[0]->pmd; - pmd =3D &early_pgts[1]->pmd; - RIP_REL_REF(next_early_pgt) =3D 2; - - pgtable_flags =3D _KERNPG_TABLE_NOENC + sme_get_me_mask(); - - if (la57) { - p4d =3D &early_pgts[RIP_REL_REF(next_early_pgt)++]->pmd; - - i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; - pgd[i + 0] =3D (pgdval_t)p4d + pgtable_flags; - pgd[i + 1] =3D (pgdval_t)p4d + pgtable_flags; - - i =3D physaddr >> P4D_SHIFT; - p4d[(i + 0) % PTRS_PER_P4D] =3D (pgdval_t)pud + pgtable_flags; - p4d[(i + 1) % PTRS_PER_P4D] =3D (pgdval_t)pud + pgtable_flags; - } else { - i =3D (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; - pgd[i + 0] =3D (pgdval_t)pud + pgtable_flags; - pgd[i + 1] =3D (pgdval_t)pud + pgtable_flags; - } - - i =3D physaddr >> PUD_SHIFT; - pud[(i + 0) % PTRS_PER_PUD] =3D (pudval_t)pmd + pgtable_flags; - pud[(i + 1) % PTRS_PER_PUD] =3D (pudval_t)pmd + pgtable_flags; - - pmd_entry =3D __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL; - /* Filter out unsupported __PAGE_KERNEL_* bits: */ - pmd_entry &=3D RIP_REL_REF(__supported_pte_mask); - pmd_entry +=3D sme_get_me_mask(); - pmd_entry +=3D physaddr; - - for (i =3D 0; i < DIV_ROUND_UP(va_end - va_text, PMD_SIZE); i++) { - int idx =3D i + (physaddr >> PMD_SHIFT); - - pmd[idx % PTRS_PER_PMD] =3D pmd_entry + i * PMD_SIZE; - } - - /* - * Fixup the kernel text+data virtual addresses. Note that - * we might write invalid pmds, when the kernel is relocated - * cleanup_highmap() fixes this up along with the mappings - * beyond _end. - * - * Only the region occupied by the kernel image has so far - * been checked against the table of usable memory regions - * provided by the firmware, so invalidate pages outside that - * region. A page table entry that maps to a reserved area of - * memory would allow processor speculation into that area, - * and on some hardware (particularly the UV platform) even - * speculative access to some reserved areas is caught as an - * error, causing the BIOS to halt the system. - */ - - pmd =3D &RIP_REL_REF(level2_kernel_pgt)->pmd; - - /* invalidate pages before the kernel image */ - for (i =3D 0; i < pmd_index(va_text); i++) - pmd[i] &=3D ~_PAGE_PRESENT; - - /* fixup pages that are part of the kernel image */ - for (; i <=3D pmd_index(va_end); i++) - if (pmd[i] & _PAGE_PRESENT) - pmd[i] +=3D load_delta; - - /* invalidate pages after the kernel image */ - for (; i < PTRS_PER_PMD; i++) - pmd[i] &=3D ~_PAGE_PRESENT; - - return sme_postprocess_startup(bp, pmd, p2v_offset); -} - /* Wipe all early page tables except for the kernel symbol map */ static void __init reset_early_page_tables(void) { --=20 2.49.0.472.ge94155a9ec-goog From nobody Thu Apr 10 17:54:14 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5E22204C0D for ; Tue, 1 Apr 2025 13:34:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514487; cv=none; b=iBd4ElPAB+bbpR1nkygKCNt0nXqxxsY3dbEca2YvTPmAVIZ9wuyO47APNDkeGnbvGVkLeehsRsU/t7IScbnzOwFRK/jnN7AxLqmp3+aiaYsEtQnsLpw8swC0UOxoqZVDYKj/zhk5TBZGqZsD7jDCGUh3zB7E9f0UBtrJyLtr7vI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743514487; c=relaxed/simple; bh=gdyI7unVAqelFcsV1oQkRb/FpEZooiabpLxxOvJoVAE=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=NyK+HTjF2mIuLfSWJ3SUcqDQ1Y5BED7ErxUbVk0AI+XQC9c/TQ6/FfKXcSvhIBdpfncR3XUJNDwByKA1NEJE/JcFr2pSDLmcMVz92qzVJMZdANCVuq3VnvvJN7Y/CHf3278BRJVXp//0gj+L6kQn+O0PD95NvNREjaLvJGcbSFo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4ybXhcBI; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4ybXhcBI" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-43d51bd9b41so49798335e9.3 for ; Tue, 01 Apr 2025 06:34:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743514483; x=1744119283; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Y3WkZ4oiyNOu5L0IFpzb/JlaH4U0a2GEGnk/9vyxvlk=; b=4ybXhcBI2lzRiLirF7OjgcBj9nTZcUvui/+PZuDa/JwnK+Kb5+UetuyN9rj92jYCrv Oka1+Ls5x5DRH+kjYhKH8oTGuqiTE1J+PdnS38mU6Hm9nSq/Gq4oDhV0U+a61iurFTaR 5yX7+1spqhXoqyo+ApJxBzRt3mioZ60Q3fnC3Nu4+As1PfJnx9Qr+dbh4QYX2rcFMRRp v8m8v9Qi1KcVbhfj6VQoPUJHj5mFStitTJBfMzemBmiZn7qpwBn69ln8jZfj189SZAk+ 1mp6t93hVubCSH9wlbhfHg7Bc9U/Oj976tiu8IYJuI1VxcJQj3WVVw54gmxocqT3k4BI cvHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743514483; x=1744119283; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Y3WkZ4oiyNOu5L0IFpzb/JlaH4U0a2GEGnk/9vyxvlk=; b=QQdsGjnvP+XPcA7C92pNIbRmJdOEJSeuoBbKG8At5TgQRtNbmtV5KgPiOB0va9o5r1 cVtiM/lcslLLwpqx7XoMrk/8ngxWZsZfCv+ZKMgbsumIXSffXXEYQpL7n7s4QGf3spor SzvYyz3s/d+7tRVPvbOuwwFzZkKtiSrvFLPIboRdH5X1GoQrvEBEYQgreG49bNUbLBMn Eus3ea9b6sOeevgQ6dTlYTM94NqzWEG/BdatWLzdpCRPQw0dj8VParGRw73HNX7P9VpU Tpj++Q/ozDexi5e/GrQcT3NX0P5ebdrSSBkdFEh810i4iBBgCCXwgmTCW23h5ASXXNIF gnQw== X-Gm-Message-State: AOJu0YxqoTtWfA+WchwJtZ+NXeKpsEewbm/Q7gzE1Ze7nHZ4nArA5mYn AexCIvw5W4N62o1orZIVdLUeC6YI4eTQndkOKucZC807gkeIo8rbkyTrN76Wz5G3wsKX6Q== X-Google-Smtp-Source: AGHT+IGfPceNrKT6p4eafCCaO1Kg9NonHtCGA618XGANB6G80nVMJDaWbJqhrfqxJGWoYDWeOBLuDcQp X-Received: from wmqd10.prod.google.com ([2002:a05:600c:34ca:b0:43d:b71:a576]) (user=ardb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:1388:b0:43c:ec97:75db with SMTP id 5b1f17b1804b1-43db622a2a1mr87997065e9.11.1743514483360; Tue, 01 Apr 2025 06:34:43 -0700 (PDT) Date: Tue, 1 Apr 2025 15:34:23 +0200 In-Reply-To: <20250401133416.1436741-8-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250401133416.1436741-8-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=9644; i=ardb@kernel.org; h=from:subject; bh=Xn0NZVRSKnkBVNa4JliVxJ2z0L0pTwu/uZobfB4mEXk=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIf3164R5kqHHDzveeVQe+XVynVvWji9qr+o55RJ1zQJ69 dN+G7V0lLIwiHEwyIopsgjM/vtu5+mJUrXOs2Rh5rAygQxh4OIUgIlcvc3w31f52p/7ddNjX0c8 598R/Vry3Ktb03bGXC/mK/XhmMvHdZjhN5vJjohou+cFcRfEFB8UNASdl/fcn3tR0WGNWdcz+1c LmAA= X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250401133416.1436741-14-ardb+git@google.com> Subject: [RFC PATCH 6/6] x86/boot: Move early SME init code into startup/ From: Ard Biesheuvel To: linux-efi@vger.kernel.org Cc: linux-kernel@vger.kernel.org, x86@kernel.org, Ard Biesheuvel , Tom Lendacky , Dionna Amalie Glaze , Kevin Loughlin Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ard Biesheuvel Move the SME initialization code, which runs from the 1:1 mapping of memory as it operates on the kernel virtual mapping, into the new sub-directory arch/x86/boot/startup/ where all startup code will reside that needs to tolerate executing from the 1:1 mapping. This allows RIP_REL_REF() macro invocations and __head annotations to be dropped. Signed-off-by: Ard Biesheuvel --- arch/x86/boot/startup/Makefile | 1 + arch/x86/{mm/mem_encrypt_identity.c =3D> boot/startup/sme.c} | 45 ++++++++= +----------- arch/x86/include/asm/mem_encrypt.h | 2 +- arch/x86/mm/Makefile | 6 --- 4 files changed, 23 insertions(+), 31 deletions(-) diff --git a/arch/x86/boot/startup/Makefile b/arch/x86/boot/startup/Makefile index 01423063fec2..480c2d2063a0 100644 --- a/arch/x86/boot/startup/Makefile +++ b/arch/x86/boot/startup/Makefile @@ -16,6 +16,7 @@ UBSAN_SANITIZE :=3D n KCOV_INSTRUMENT :=3D n =20 obj-$(CONFIG_X86_64) +=3D gdt_idt.o map_kernel.o +obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D sme.o =20 lib-$(CONFIG_X86_64) +=3D la57toggle.o lib-$(CONFIG_EFI_MIXED) +=3D efi-mixed.o diff --git a/arch/x86/mm/mem_encrypt_identity.c b/arch/x86/boot/startup/sme= .c similarity index 92% rename from arch/x86/mm/mem_encrypt_identity.c rename to arch/x86/boot/startup/sme.c index 5eecdd92da10..85bd39652535 100644 --- a/arch/x86/mm/mem_encrypt_identity.c +++ b/arch/x86/boot/startup/sme.c @@ -45,8 +45,6 @@ #include #include =20 -#include "mm_internal.h" - #define PGD_FLAGS _KERNPG_TABLE_NOENC #define P4D_FLAGS _KERNPG_TABLE_NOENC #define PUD_FLAGS _KERNPG_TABLE_NOENC @@ -93,7 +91,7 @@ struct sme_populate_pgd_data { */ static char sme_workarea[2 * PMD_SIZE] __section(".init.scratch"); =20 -static void __head sme_clear_pgd(struct sme_populate_pgd_data *ppd) +static void __init sme_clear_pgd(struct sme_populate_pgd_data *ppd) { unsigned long pgd_start, pgd_end, pgd_size; pgd_t *pgd_p; @@ -108,7 +106,7 @@ static void __head sme_clear_pgd(struct sme_populate_pg= d_data *ppd) memset(pgd_p, 0, pgd_size); } =20 -static pud_t __head *sme_prepare_pgd(struct sme_populate_pgd_data *ppd) +static pud_t __init *sme_prepare_pgd(struct sme_populate_pgd_data *ppd) { pgd_t *pgd; p4d_t *p4d; @@ -145,7 +143,7 @@ static pud_t __head *sme_prepare_pgd(struct sme_populat= e_pgd_data *ppd) return pud; } =20 -static void __head sme_populate_pgd_large(struct sme_populate_pgd_data *pp= d) +static void __init sme_populate_pgd_large(struct sme_populate_pgd_data *pp= d) { pud_t *pud; pmd_t *pmd; @@ -161,7 +159,7 @@ static void __head sme_populate_pgd_large(struct sme_po= pulate_pgd_data *ppd) set_pmd(pmd, __pmd(ppd->paddr | ppd->pmd_flags)); } =20 -static void __head sme_populate_pgd(struct sme_populate_pgd_data *ppd) +static void __init sme_populate_pgd(struct sme_populate_pgd_data *ppd) { pud_t *pud; pmd_t *pmd; @@ -187,7 +185,7 @@ static void __head sme_populate_pgd(struct sme_populate= _pgd_data *ppd) set_pte(pte, __pte(ppd->paddr | ppd->pte_flags)); } =20 -static void __head __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) +static void __init __sme_map_range_pmd(struct sme_populate_pgd_data *ppd) { while (ppd->vaddr < ppd->vaddr_end) { sme_populate_pgd_large(ppd); @@ -197,7 +195,7 @@ static void __head __sme_map_range_pmd(struct sme_popul= ate_pgd_data *ppd) } } =20 -static void __head __sme_map_range_pte(struct sme_populate_pgd_data *ppd) +static void __init __sme_map_range_pte(struct sme_populate_pgd_data *ppd) { while (ppd->vaddr < ppd->vaddr_end) { sme_populate_pgd(ppd); @@ -207,7 +205,7 @@ static void __head __sme_map_range_pte(struct sme_popul= ate_pgd_data *ppd) } } =20 -static void __head __sme_map_range(struct sme_populate_pgd_data *ppd, +static void __init __sme_map_range(struct sme_populate_pgd_data *ppd, pmdval_t pmd_flags, pteval_t pte_flags) { unsigned long vaddr_end; @@ -231,22 +229,22 @@ static void __head __sme_map_range(struct sme_populat= e_pgd_data *ppd, __sme_map_range_pte(ppd); } =20 -static void __head sme_map_range_encrypted(struct sme_populate_pgd_data *p= pd) +static void __init sme_map_range_encrypted(struct sme_populate_pgd_data *p= pd) { __sme_map_range(ppd, PMD_FLAGS_ENC, PTE_FLAGS_ENC); } =20 -static void __head sme_map_range_decrypted(struct sme_populate_pgd_data *p= pd) +static void __init sme_map_range_decrypted(struct sme_populate_pgd_data *p= pd) { __sme_map_range(ppd, PMD_FLAGS_DEC, PTE_FLAGS_DEC); } =20 -static void __head sme_map_range_decrypted_wp(struct sme_populate_pgd_data= *ppd) +static void __init sme_map_range_decrypted_wp(struct sme_populate_pgd_data= *ppd) { __sme_map_range(ppd, PMD_FLAGS_DEC_WP, PTE_FLAGS_DEC_WP); } =20 -static unsigned long __head sme_pgtable_calc(unsigned long len) +static unsigned long __init sme_pgtable_calc(unsigned long len) { unsigned long entries =3D 0, tables =3D 0; =20 @@ -283,7 +281,7 @@ static unsigned long __head sme_pgtable_calc(unsigned l= ong len) return entries + tables; } =20 -void __head sme_encrypt_kernel(struct boot_params *bp) +void __init sme_encrypt_kernel(struct boot_params *bp) { unsigned long workarea_start, workarea_end, workarea_len; unsigned long execute_start, execute_end, execute_len; @@ -299,8 +297,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp) * instrumentation or checking boot_cpu_data in the cc_platform_has() * function. */ - if (!sme_get_me_mask() || - RIP_REL_REF(sev_status) & MSR_AMD64_SEV_ENABLED) + if (!sme_get_me_mask() || sev_status & MSR_AMD64_SEV_ENABLED) return; =20 /* @@ -318,8 +315,8 @@ void __head sme_encrypt_kernel(struct boot_params *bp) * memory from being cached. */ =20 - kernel_start =3D (unsigned long)RIP_REL_REF(_text); - kernel_end =3D ALIGN((unsigned long)RIP_REL_REF(_end), PMD_SIZE); + kernel_start =3D (unsigned long)_text; + kernel_end =3D ALIGN((unsigned long)_end, PMD_SIZE); kernel_len =3D kernel_end - kernel_start; =20 initrd_start =3D 0; @@ -345,7 +342,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp) * pagetable structures for the encryption of the kernel * pagetable structures for workarea (in case not currently mapped) */ - execute_start =3D workarea_start =3D (unsigned long)RIP_REL_REF(sme_worka= rea); + execute_start =3D workarea_start =3D (unsigned long)sme_workarea; execute_end =3D execute_start + (PAGE_SIZE * 2) + PMD_SIZE; execute_len =3D execute_end - execute_start; =20 @@ -488,7 +485,7 @@ void __head sme_encrypt_kernel(struct boot_params *bp) native_write_cr3(__native_read_cr3()); } =20 -void __head sme_enable(struct boot_params *bp) +void __init sme_enable(struct boot_params *bp) { unsigned int eax, ebx, ecx, edx; unsigned long feature_mask; @@ -526,7 +523,7 @@ void __head sme_enable(struct boot_params *bp) me_mask =3D 1UL << (ebx & 0x3f); =20 /* Check the SEV MSR whether SEV or SME is enabled */ - RIP_REL_REF(sev_status) =3D msr =3D __rdmsr(MSR_AMD64_SEV); + sev_status =3D msr =3D __rdmsr(MSR_AMD64_SEV); feature_mask =3D (msr & MSR_AMD64_SEV_ENABLED) ? AMD_SEV_BIT : AMD_SME_BI= T; =20 /* @@ -562,8 +559,8 @@ void __head sme_enable(struct boot_params *bp) return; } =20 - RIP_REL_REF(sme_me_mask) =3D me_mask; - RIP_REL_REF(physical_mask) &=3D ~me_mask; - RIP_REL_REF(cc_vendor) =3D CC_VENDOR_AMD; + sme_me_mask =3D me_mask; + physical_mask &=3D ~me_mask; + cc_vendor =3D CC_VENDOR_AMD; cc_set_mask(me_mask); } diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_= encrypt.h index 1530ee301dfe..ea6494628cb0 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -61,7 +61,7 @@ void __init sev_es_init_vc_handling(void); =20 static inline u64 sme_get_me_mask(void) { - return RIP_REL_REF(sme_me_mask); + return sme_me_mask; } =20 #define __bss_decrypted __section(".bss..decrypted") diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index 690fbf48e853..9cbb18c99adb 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -3,12 +3,10 @@ KCOV_INSTRUMENT_tlb.o :=3D n KCOV_INSTRUMENT_mem_encrypt.o :=3D n KCOV_INSTRUMENT_mem_encrypt_amd.o :=3D n -KCOV_INSTRUMENT_mem_encrypt_identity.o :=3D n KCOV_INSTRUMENT_pgprot.o :=3D n =20 KASAN_SANITIZE_mem_encrypt.o :=3D n KASAN_SANITIZE_mem_encrypt_amd.o :=3D n -KASAN_SANITIZE_mem_encrypt_identity.o :=3D n KASAN_SANITIZE_pgprot.o :=3D n =20 # Disable KCSAN entirely, because otherwise we get warnings that some func= tions @@ -16,12 +14,10 @@ KASAN_SANITIZE_pgprot.o :=3D n KCSAN_SANITIZE :=3D n # Avoid recursion by not calling KMSAN hooks for CEA code. KMSAN_SANITIZE_cpu_entry_area.o :=3D n -KMSAN_SANITIZE_mem_encrypt_identity.o :=3D n =20 ifdef CONFIG_FUNCTION_TRACER CFLAGS_REMOVE_mem_encrypt.o =3D -pg CFLAGS_REMOVE_mem_encrypt_amd.o =3D -pg -CFLAGS_REMOVE_mem_encrypt_identity.o =3D -pg CFLAGS_REMOVE_pgprot.o =3D -pg endif =20 @@ -32,7 +28,6 @@ obj-y +=3D pat/ =20 # Make sure __phys_addr has no stackprotector CFLAGS_physaddr.o :=3D -fno-stack-protector -CFLAGS_mem_encrypt_identity.o :=3D -fno-stack-protector =20 CFLAGS_fault.o :=3D -I $(src)/../include/asm/trace =20 @@ -65,5 +60,4 @@ obj-$(CONFIG_MITIGATION_PAGE_TABLE_ISOLATION) +=3D pti.o obj-$(CONFIG_X86_MEM_ENCRYPT) +=3D mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_amd.o =20 -obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_identity.o obj-$(CONFIG_AMD_MEM_ENCRYPT) +=3D mem_encrypt_boot.o --=20 2.49.0.472.ge94155a9ec-goog