From nobody Fri Apr 10 00:56:50 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47E61346E7E for ; Wed, 4 Mar 2026 16:06:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772640420; cv=none; b=PVsItUHCZVdFfNQANCZ7bv3xkkTA0KL2oRiVEZbeb15jAT+nHKnM0/h/K+ok3X2WEFeMpQGQ/6iB5LH+9GcZCVb+eXiBkHz2qA2EqOd5b5dhNj3LatPFLDpwA8ks/G4KT1cdKWo9TxaRL0/W0I9S6zDHE1peLeUpUlP1Lg/CoUo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772640420; c=relaxed/simple; bh=Ykb9jjbB+uS1J4WKedPBvfPmIvLojBB5UDfbaZqanN0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=eODvhF25YY/matlNOZFgbtaPQiduZlWBM5j4v3L4yaJKurJfA4egbykzIkH1bT+Z5jkhGjlpKYzMedYRPzmHJSW1oK9Pdo6sPIToUXhMhf8C5rTo55pdNS1XgHRAAr6f1X7FdwecR8WoTKn2nyZwC3WT65paYyy3yJH6FCD4m0E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=h+151r/5; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="h+151r/5" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B3B44C4CEF7; Wed, 4 Mar 2026 16:06:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1772640419; bh=Ykb9jjbB+uS1J4WKedPBvfPmIvLojBB5UDfbaZqanN0=; h=From:To:Cc:Subject:Date:From; b=h+151r/5hEqTMzDkwlLBtpI+LzxetjA9+NwL2f0wXQg3UJjFT0L9sEo6+KecV01mi flcqimcA1t00EY5REkfg82u6YCaM/TfDdrRRsKTml5/ZRIcQSnqhbiGxPRRHJ1IWX/ JYCxuhkCD7BdC9MHOS3cqdBOulRmSACixb7l7TCi9d6NaEdSCtg3zbMU0qvqzj+sgP 1kqQrJeRJpv8uK/jgRMVgUSn78lMfZyCtDte3+PsSU5K4CzLUjgg1LIiaKyogq7mh9 aA4/ZTAVQFdlmBKMg8EuNo+PoUs/qX452U1KiQLEGkerqZH6oKr0lZY9ClDHEXNoer lLCbkB2g90qYQ== From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Ard Biesheuvel , "H. Peter Anvin" , Kiryl Shutsemau Subject: [PATCH] x86/boot: Use 5-level paging trampoline in place if possible Date: Wed, 4 Mar 2026 17:05:50 +0100 Message-ID: <20260304160549.670066-2-ardb@kernel.org> X-Mailer: git-send-email 2.47.3 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5598; i=ardb@kernel.org; h=from:subject; bh=Ykb9jjbB+uS1J4WKedPBvfPmIvLojBB5UDfbaZqanN0=; b=owGbwMvMwCVmkMcZplerG8N4Wi2JIXNFROyHl0sk1r25afT4t5XSCbWUyze17y8zD3szr/BhS sxjEUXxjlIWBjEuBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRT7YM/0uYZmqFTf60b/P6 VdqpxSuPPjl+pfvoi84Z8zO3STOny3xkZDj9UHzT6m7utNsP96Y27FX7J+Aw9WtuWInbtQM2T57 /kOUAAA== X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The traditional x86 decompressor contains a rather nasty hack in find_trampoline_placement() to temporarily use some system RAM below 1M for a 32-bit addressable trampoline, which it needs in order to be able to switch from 4 levels of paging to 5 or vice versa. It does this even when running from 32-bit addressable memory itself, in which case none of this is needed - the trampoline code can be fixed up to execute correctly from anywhere below 4G, and so it can execute in place from the decompressor's initial placement. The additional root level page table allocation can be used directly if it resides below the 4G mark as well. The traditional decompressor is no longer used by the EFI stub, and is mostly relied upon by non-EFI bootloaders such as coreboot which run entirely in 32-bit mode, and are therefore guaranteed to place the decompressor in 32-bit addressable memory. In those cases, on CPUs that support it, 5-level paging will always get enabled, by invoking this code (unless it is explicitly disabled on the command line) but the trampoline allocation is never needed. kexec also relies on the traditional decompressor and does typically load the kernel above 4G, but in that case, the trampoline is only needed when either the first or the second kernel explicitly disables 5-level paging, otherwise it will just remain enabled and no trampoline is needed. So avoid the trampoline hack unless it is really required. While at it, bring the C prototype of trampoline_32bit_src() in line with the asm implementation, and drop a couple of redundant declarations. Cc: "H. Peter Anvin" Cc: Kiryl Shutsemau Signed-off-by: Ard Biesheuvel --- arch/x86/boot/compressed/pgtable_64.c | 40 +++++++++++--------- arch/x86/include/asm/boot.h | 2 +- drivers/firmware/efi/libstub/x86-stub.h | 3 -- 3 files changed, 24 insertions(+), 21 deletions(-) diff --git a/arch/x86/boot/compressed/pgtable_64.c b/arch/x86/boot/compress= ed/pgtable_64.c index 0e89e197e112..0f6e2ae5c1a1 100644 --- a/arch/x86/boot/compressed/pgtable_64.c +++ b/arch/x86/boot/compressed/pgtable_64.c @@ -102,7 +102,7 @@ static unsigned long find_trampoline_placement(void) =20 asmlinkage void configure_5level_paging(struct boot_params *bp, void *pgta= ble) { - void (*toggle_la57)(void *cr3); + void (*toggle_la57)(void *cr3) =3D trampoline_32bit_src; bool l5_required =3D false; =20 /* Initialize boot_params. Required for cmdline_find_option_bool(). */ @@ -135,18 +135,22 @@ asmlinkage void configure_5level_paging(struct boot_p= arams *bp, void *pgtable) if (l5_required =3D=3D !!(native_read_cr4() & X86_CR4_LA57)) return; =20 - trampoline_32bit =3D (unsigned long *)find_trampoline_placement(); + if ((unsigned long)pgtable > U32_MAX) { + trampoline_32bit =3D (unsigned long *)find_trampoline_placement(); =20 - /* Preserve trampoline memory */ - memcpy(trampoline_save, trampoline_32bit, TRAMPOLINE_32BIT_SIZE); + /* Preserve trampoline memory */ + memcpy(trampoline_save, trampoline_32bit, TRAMPOLINE_32BIT_SIZE); =20 - /* Clear trampoline memory first */ - memset(trampoline_32bit, 0, TRAMPOLINE_32BIT_SIZE); + /* Clear trampoline memory first */ + memset(trampoline_32bit, 0, TRAMPOLINE_32BIT_SIZE); =20 - /* Copy trampoline code in place */ - toggle_la57 =3D memcpy(trampoline_32bit + - TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long), - &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE); + /* Copy trampoline code in place */ + toggle_la57 =3D memcpy(trampoline_32bit + + TRAMPOLINE_32BIT_CODE_OFFSET / sizeof(unsigned long), + &trampoline_32bit_src, TRAMPOLINE_32BIT_CODE_SIZE); + } else { + trampoline_32bit =3D memset(pgtable, 0, PAGE_SIZE); + } =20 /* * Avoid the need for a stack in the 32-bit trampoline code, by using @@ -189,12 +193,14 @@ asmlinkage void configure_5level_paging(struct boot_p= arams *bp, void *pgtable) =20 toggle_la57(trampoline_32bit); =20 - /* - * Move the top level page table out of trampoline memory. - */ - memcpy(pgtable, trampoline_32bit, PAGE_SIZE); - native_write_cr3((unsigned long)pgtable); + if (pgtable !=3D trampoline_32bit) { + /* + * Move the top level page table out of trampoline memory. + */ + memcpy(pgtable, trampoline_32bit, PAGE_SIZE); + native_write_cr3((unsigned long)pgtable); =20 - /* Restore trampoline memory */ - memcpy(trampoline_32bit, trampoline_save, TRAMPOLINE_32BIT_SIZE); + /* Restore trampoline memory */ + memcpy(trampoline_32bit, trampoline_save, TRAMPOLINE_32BIT_SIZE); + } } diff --git a/arch/x86/include/asm/boot.h b/arch/x86/include/asm/boot.h index f7b67cb73915..bc9739a2633a 100644 --- a/arch/x86/include/asm/boot.h +++ b/arch/x86/include/asm/boot.h @@ -93,7 +93,7 @@ extern struct boot_params *boot_params_ptr; extern unsigned long *trampoline_32bit; extern const u16 trampoline_ljmp_imm_offset; =20 -void trampoline_32bit_src(void *trampoline, bool enable_5lvl); +void trampoline_32bit_src(void *cr3); =20 #endif =20 diff --git a/drivers/firmware/efi/libstub/x86-stub.h b/drivers/firmware/efi= /libstub/x86-stub.h index 1c20e99a6494..514560712610 100644 --- a/drivers/firmware/efi/libstub/x86-stub.h +++ b/drivers/firmware/efi/libstub/x86-stub.h @@ -2,9 +2,6 @@ =20 #include =20 -extern void trampoline_32bit_src(void *, bool); -extern const u16 trampoline_ljmp_imm_offset; - efi_status_t efi_adjust_memory_range_protection(unsigned long start, unsigned long size); =20 --=20 2.53.0.473.g4a7958ca14-goog