From: David Woodhouse <dwmw@amazon.co.uk>
Reduce the window during which exceptions are unhandled, by leaving the
GDT/IDT in place all the way into the relocate_kernel() function, until
the moment that %cr3 gets replaced.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
arch/x86/kernel/machine_kexec_64.c | 10 ++--------
arch/x86/kernel/relocate_kernel_64.S | 9 +++++++--
2 files changed, 9 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index ecb0da54abd1..949c9e4bfad2 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -434,16 +434,10 @@ void __nocfi machine_kexec(struct kimage *image)
* with from a table in memory. At no other time is the
* descriptor table in memory accessed.
*
- * I take advantage of this here by force loading the
- * segments, before I zap the gdt with an invalid value.
+ * Take advantage of this here by force loading the segments,
+ * before the GDT is zapped with an invalid value.
*/
load_segments();
- /*
- * The gdt & idt are now invalid.
- * If you want to load them you must set up your own idt & gdt.
- */
- native_idt_invalidate();
- native_gdt_invalidate();
/* now call it */
image->start = relocate_kernel_ptr((unsigned long)image->head,
diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocate_kernel_64.S
index 8808cfca6322..3062cb3efc44 100644
--- a/arch/x86/kernel/relocate_kernel_64.S
+++ b/arch/x86/kernel/relocate_kernel_64.S
@@ -79,8 +79,13 @@ SYM_CODE_START_NOALIGN(relocate_kernel)
pushq %r15
pushf
- /* zero out flags, and disable interrupts */
- pushq $0
+ /* Invalidate GDT/IDT, zero out flags */
+ pushq $0
+ pushq $0
+
+ lidt (%rsp)
+ lgdt (%rsp)
+ addq $8, %rsp
popfq
/* Switch to the identity mapped page tables */
--
2.48.1On Wed, 2025-03-26 at 14:16 +0000, David Woodhouse wrote: > From: David Woodhouse <dwmw@amazon.co.uk> > > Reduce the window during which exceptions are unhandled, by leaving the > GDT/IDT in place all the way into the relocate_kernel() function, until > the moment that %cr3 gets replaced. Seems the difference is only couple of instructions. Anyway ... > > Signed-off-by: David Woodhouse <dwmw@amazon.co.uk> Reviewed-by: Kai Huang <kai.huang@intel.com>
On Tue, 2025-04-01 at 09:48 +0000, Huang, Kai wrote: > On Wed, 2025-03-26 at 14:16 +0000, David Woodhouse wrote: > > From: David Woodhouse <dwmw@amazon.co.uk> > > > > Reduce the window during which exceptions are unhandled, by leaving > > the > > GDT/IDT in place all the way into the relocate_kernel() function, > > until > > the moment that %cr3 gets replaced. > > Seems the difference is only couple of instructions. Anyway ... It is, although that was important when I was trying to enable CET because one of those instructions was the indirect call. I've given up on CET now because it's largely pointless for that particular call. But still, putting the GDT/IDT invalidation into relocate_kernel.S means that we can further shrink the window if we want to look at leaving MC enabled during the handover, etc. > > > > Signed-off-by: David Woodhouse <dwmw@amazon.co.uk> > > Reviewed-by: Kai Huang <kai.huang@intel.com> Thanks.
The following commit has been merged into the x86/asm branch of tip:
Commit-ID: de085ddd493bccb77a3ec1b99ae7466133540f4d
Gitweb: https://git.kernel.org/tip/de085ddd493bccb77a3ec1b99ae7466133540f4d
Author: David Woodhouse <dwmw@amazon.co.uk>
AuthorDate: Wed, 26 Mar 2025 14:16:03
Committer: Ingo Molnar <mingo@kernel.org>
CommitterDate: Thu, 10 Apr 2025 12:17:14 +02:00
x86/kexec: Invalidate GDT/IDT from relocate_kernel() instead of earlier
Reduce the window during which exceptions are unhandled, by leaving the
GDT/IDT in place all the way into the relocate_kernel() function, until
the moment that %cr3 gets replaced.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20250326142404.256980-4-dwmw2@infradead.org
---
arch/x86/kernel/machine_kexec_64.c | 10 ++--------
arch/x86/kernel/relocate_kernel_64.S | 9 +++++++--
2 files changed, 9 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_kexec_64.c
index ecb0da5..949c9e4 100644
--- a/arch/x86/kernel/machine_kexec_64.c
+++ b/arch/x86/kernel/machine_kexec_64.c
@@ -434,16 +434,10 @@ void __nocfi machine_kexec(struct kimage *image)
* with from a table in memory. At no other time is the
* descriptor table in memory accessed.
*
- * I take advantage of this here by force loading the
- * segments, before I zap the gdt with an invalid value.
+ * Take advantage of this here by force loading the segments,
+ * before the GDT is zapped with an invalid value.
*/
load_segments();
- /*
- * The gdt & idt are now invalid.
- * If you want to load them you must set up your own idt & gdt.
- */
- native_idt_invalidate();
- native_gdt_invalidate();
/* now call it */
image->start = relocate_kernel_ptr((unsigned long)image->head,
diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocate_kernel_64.S
index 8808cfc..3062cb3 100644
--- a/arch/x86/kernel/relocate_kernel_64.S
+++ b/arch/x86/kernel/relocate_kernel_64.S
@@ -79,8 +79,13 @@ SYM_CODE_START_NOALIGN(relocate_kernel)
pushq %r15
pushf
- /* zero out flags, and disable interrupts */
- pushq $0
+ /* Invalidate GDT/IDT, zero out flags */
+ pushq $0
+ pushq $0
+
+ lidt (%rsp)
+ lgdt (%rsp)
+ addq $8, %rsp
popfq
/* Switch to the identity mapped page tables */© 2016 - 2025 Red Hat, Inc.