From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB4C81DFD98 for ; Fri, 22 Nov 2024 22:47:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315673; cv=none; b=iFV32rOmCSuOKsHLP3/Xt7U2wimPbyTc5wQYulZuCznsv7NY+hYVSpS3oJjn+O6M8+QcyU+CcnzsvMHZGTBmbcJTeEcTGFCiFBK+Dcp9T3x21wYahjps/kj87WXNohIrGoUBqrCLsqVAbuIMfFV1obD+azbWi2+5n8oQjjwGpBs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315673; c=relaxed/simple; bh=BIugvxcd1pueXS5wZLJ0OWYHQ5ZQ8FtsSi+ic4mE29o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AhlS8UuSs4T5+Ah2XtzBj5but18kjqhyWLYpZH5m45cw438aMNV1Bz4qrn7vzgvFe4yRvSHnAMMGkcE1IVgN07bRsPRGUh0b7cQ8j7T44P/fzNpVHBjpPQO6ZmbEOsfWn0buq87Ww/o4MdRp0SolGpy1+1M5YFn+wHo7pVBfg7Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=pIYDAZhH; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="pIYDAZhH" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=OXhWEFUeCm7JR33qPW8dR7lPOU0/xHsRBXJM331sN9E=; b=pIYDAZhHJqofFuySIRGkpP6r9f 39AT0s+xWsxB/WG5+oYrJLuir2dihEfE+1sRyItUvD+QWMoLnWG1AUyhCma3+jmtYZ3oYvHr36HMH Sx3907JoVZqIxCJe+OF5yJWQ/GLrTiDYk8ulp13NyjaohhQuvapLHJQFRrq5QJM3YOKEjrioKabbP EOKov4VsAmjr33GcqQffeu3mGeMIP137nadZT/fdRo6M/wpSA2kJyinX7EXeCs72U7ZtWslApBwxH pHc6AVzlbvT5172DbCcTg4OLiwoewZWNTvt4UKCX5a5EOrVE66M2sCoOb9kxvcuu4hZv/t9zjvkmA nPQAaTug==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQs-00000000ma5-01Cu; Fri, 22 Nov 2024 22:47:22 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQo-00000000ii3-3bC4; Fri, 22 Nov 2024 22:47:18 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 01/16] x86/kexec: Clean up and document register use in relocate_kernel_64.S Date: Fri, 22 Nov 2024 22:38:10 +0000 Message-ID: <20241122224715.171751-2-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Add more comments explaining what each register contains, and save the preserve_context flag to a non-clobbered register sooner, to keep things simpler. Signed-off-by: David Woodhouse Acked-by: Kai Huang --- arch/x86/kernel/relocate_kernel_64.S | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index e9e88c342f75..7ee32bcb6e01 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -100,6 +100,9 @@ SYM_CODE_START_NOALIGN(relocate_kernel) movq %r10, CP_PA_SWAP_PAGE(%r11) movq %rdi, CP_PA_BACKUP_PAGES_MAP(%r11) =20 + /* Save the preserve_context to %r11 as swap_pages clobbers %rcx. */ + movq %rcx, %r11 + /* Switch to the identity mapped page tables */ movq %r9, %cr3 =20 @@ -116,6 +119,14 @@ SYM_CODE_END(relocate_kernel) =20 SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) UNWIND_HINT_END_OF_STACK + /* + * %rdi indirection page + * %rdx start address + * %r11 preserve_context + * %r12 host_mem_enc_active + * %r13 original CR4 when relocate_kernel() was invoked + */ + /* set return address to 0 if not preserving context */ pushq $0 /* store the start address on the stack */ @@ -170,8 +181,6 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) wbinvd .Lsme_off: =20 - /* Save the preserve_context to %r11 as swap_pages clobbers %rcx. */ - movq %rcx, %r11 call swap_pages =20 /* @@ -183,13 +192,14 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) movq %cr3, %rax movq %rax, %cr3 =20 + testq %r11, %r11 /* preserve_context */ + jnz .Lrelocate + /* * set all of the registers to known values * leave %rsp alone */ =20 - testq %r11, %r11 - jnz .Lrelocate xorl %eax, %eax xorl %ebx, %ebx xorl %ecx, %ecx --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE428187332 for ; Fri, 22 Nov 2024 22:47:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315665; cv=none; b=ELLN/p2nOWzoJ4p7H8IDNfpfivG+Cd4wVRfqYy+oC+r4qbVJmM7LguC9Apy1MgeUzkuLu8n6nUNtzCbVdb4yMWH5amkF3UWzy2zzywHTIx2SL8I1Yaqxxdhlb0JdzVX73ajv8/WdGfwO+PpVl8kjRop/AJOyL3HzoXectd0IyRQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315665; c=relaxed/simple; bh=Tut/DVSgyPMceFwEIPGjIQ9A3QJx8b1IbX24tJaHN8o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tczk0TJulShFFzi0K4kILSrYTTkJDnjghO7K9XqLqGs1rAvFZKaPi2caLcnLIkqSERkkZvqROIl6T5ndV2M9qhdvK593Li4FzAxtX62E0yCbl4rUXp3eTm9C3zHEKHnPrcjVqcTiHVu80nhao3dCTf9zZKDlb8WmE2h+TMNeh/Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=O2h2pjVi; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="O2h2pjVi" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=1mMILGYOmOJ50kB2grC2kFU1xTOvUAHt2PqPYBJoz2c=; b=O2h2pjViyLSTCP7Ozn1mTqmoBC FMVFz+vDuC6SvC9JfPMfjrx8q0FEncgN2xLnnxqnydc6EE0khAuF167GLgAcy9KzaDBloSWusYkFn E6sEUzXgobVwZNC2eCr+BpJUMTQK+43XixU/QcI5Mpa5CccGadMlwS7SAdLcbwIP5zNmvHA32UeBZ kxR89qod79D233IRJZuqnvwnJHokol62+CpJUuU5trQ/XSaq89H4LxHByeDsnTDpYoH6BH1og22mY XYb/QyRHjRKsuWfL5pfDMp1XVKU+vT/kYhpMoano5j1ruJd2nCWP8gVikszKRwr0ZiFIlBusps2Dh aLpp4uJA==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000008PHR-2Caq; Fri, 22 Nov 2024 22:47:20 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQo-00000000ii6-3ky3; Fri, 22 Nov 2024 22:47:18 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 02/16] x86/kexec: Use named labels in swap_pages in relocate_kernel_64.S Date: Fri, 22 Nov 2024 22:38:11 +0000 Message-ID: <20241122224715.171751-3-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Make the code a little more readable. Signed-off-by: David Woodhouse Acked-by: Kai Huang --- arch/x86/kernel/relocate_kernel_64.S | 30 ++++++++++++++-------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 7ee32bcb6e01..ca01e3e2f097 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -272,31 +272,31 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) movq %rdi, %rcx /* Put the indirection_page in %rcx */ xorl %edi, %edi xorl %esi, %esi - jmp 1f + jmp .Lstart /* Should start with an indirection record */ =20 -0: /* top, read another word for the indirection page */ +.Lloop: /* top, read another word for the indirection page */ =20 movq (%rbx), %rcx addq $8, %rbx -1: +.Lstart: testb $0x1, %cl /* is it a destination page? */ - jz 2f + jz .Lnotdest movq %rcx, %rdi andq $0xfffffffffffff000, %rdi - jmp 0b -2: + jmp .Lloop +.Lnotdest: testb $0x2, %cl /* is it an indirection page? */ - jz 2f + jz .Lnotind movq %rcx, %rbx andq $0xfffffffffffff000, %rbx - jmp 0b -2: + jmp .Lloop +.Lnotind: testb $0x4, %cl /* is it the done indicator? */ - jz 2f - jmp 3f -2: + jz .Lnotdone + jmp .Ldone +.Lnotdone: testb $0x8, %cl /* is it the source indicator? */ - jz 0b /* Ignore it otherwise */ + jz .Lloop /* Ignore it otherwise */ movq %rcx, %rsi /* For ever source page do a copy */ andq $0xfffffffffffff000, %rsi =20 @@ -321,8 +321,8 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) rep ; movsq =20 lea PAGE_SIZE(%rax), %rsi - jmp 0b -3: + jmp .Lloop +.Ldone: ANNOTATE_UNRET_SAFE ret int3 --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBB62176FD2 for ; Fri, 22 Nov 2024 22:47:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; cv=none; b=iqnaTDmonwhl1nRxtozQqUYryIuI2urAij3RTSMWPLfmVw0LTrk6VvIlqOxVOoLopwzCBD/+kdJzFzish/T6FNKvpWTV9olgHvmelKrlzkaHMwEnxxvwEr7R9AvRcDfiwnaF5o7Z1LgkA8UHcTFdbI3boTr3rYKGnSUqgMigwLs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; c=relaxed/simple; bh=etXLrsfhpMzfNgr85hW8kZ0GD35Jbyx0gK4v/B6e5YI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=j2q5u5AoK0UyrN6x763o8d/Ah1ytFy/olVUdhS/+zqa/5jakjrjFc+Ulc7NTJsOtSXXRvwpMW0RTbiaP1MrEH3QwQmIm+12KcsPjzCMqF2t3Dg336fNxQUpnpyams/S60p5tAISKPKHbPy8dvHSbMo+fn844E/9Nk9WC8G76y+8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=j5eGqunu; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="j5eGqunu" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=YMX4hA6TET2DRs4mytpddGAuD8BkAw4cael/oyJ3CzM=; b=j5eGqunufmPhasYy2LK3CUUVYV FMuL2WPvLJfDxgVpWHdc6ZNz9HzCGNJ6GJ7aIvw7CZYCEF6EMCdja/ezLz4UbLm5d/bGjiH9QPWym ie+6CZewJaSFrjYaDhevFCk6HQqdjZ40crIiHTKq5x4AYfW/kWIqONSXADQ3yM7eJfe0qexsDRYnS DXML8O7v6WHh7lwwylGeWvWpjcpFLil0W8/i1fvWCnJbjK2/JLk00Ob+3cNwq44+KLoZ7PYoylsk+ GY9n4kJG71+jK2U+4P+UFEqPxo3dnJZalAzXi/eSb0BgnExDBkc9lK3nub4xKvJqqu4+lKNO+AD1A wCzb0c6A==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000008PHT-2E7k; Fri, 22 Nov 2024 22:47:21 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQo-00000000ii9-3zyd; Fri, 22 Nov 2024 22:47:18 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 03/16] x86/kexec: Restore GDT on return from preserve_context kexec Date: Fri, 22 Nov 2024 22:38:12 +0000 Message-ID: <20241122224715.171751-4-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse The restore_processor_state() function explicitly states that "the asm code that gets us here will have restored a usable GDT". That wasn't true in the case of returning from a preserve_context kexec. Make it so. Without this, the kernel was depending on the called function to reload an appropriate GDT. Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index ca01e3e2f097..ed2ae50535dd 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -252,6 +252,11 @@ SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped) movq CR0(%r8), %r8 movq %rax, %cr3 movq %r8, %cr0 + + /* Saved in save_processor_state. */ + movq $saved_context, %rax + lgdt saved_context_gdt_desc(%rax) + movq %rbp, %rax =20 popf --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECCA618A6A9 for ; Fri, 22 Nov 2024 22:47:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315665; cv=none; b=iAhU+N7i/t281xnJnR9H1pr9hmdgxz6qGu6yyTWUih6ocQWusP/KCA3Bt5OmjMjNXDPdGuv8Ig4gtpd47Nm5q+QDLp5xPdBDtVxz88oAA03DEuU6XwjJMU21P4eUl56tC0/voSTHDlLfP8Z5Kfj1i0uN9alrYkLNzvPC8/I9SJ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315665; c=relaxed/simple; bh=PsmoAGl847a9zQLItcMGgTXPyTmUHqs5pQtbYpPJSqo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nU+7R18Xo6Y5jkGBwsV57vxBUglAkMnhf80hb/oxERPDhU8g/KdySo1GWkQ90K1iMv+xBY5mSsswfXnX7ZWSyBYjHeSANyM7galdhnaoLm1JHqJ4jxQ3T0ui8QWA0u+g8UGqEWVXpB4c9PAuZsumX3sxctaYW1QCX3HhQMLaZvs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=jCMm6BNV; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="jCMm6BNV" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=S0QVtAVD7M7pFpqYQRYpOlRlIpsQlWjl3XEa+4AG3hg=; b=jCMm6BNVrfGOtot4P6MWH5QfVr Jaa98yGLe9SXtDBP7HNJm6Bj4EXEkUldoFDGZMjV2WsTkMKvcq0gEHh0kHy21FKLqaw7eb+munQpl 9w74HhVDAg4eQ7bZIWrVhQB+i6StbGjDlrAcusy/A22QCVNYoMQvD6UN40lqeyLXZnqNPRjhgsg3g OkmpX88v3OdYPkk6JkOqugHeJ41SGbOUeNxH5HV6XD5SlUUvtoTeDtXVDP/WyDDgpUe8+AHvklQQK /dgiDEx+D7xjXKej74Nk/9ZwPiJZic6d+rNmOWfVbTGXt/2uXS2v3KFnIvnKR5stFjjLb/Uu2CLXq LC7y+aYQ==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000008PHS-2CWY; Fri, 22 Nov 2024 22:47:20 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iiC-034R; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 04/16] x86/kexec: Only swap pages for preserve_context mode Date: Fri, 22 Nov 2024 22:38:13 +0000 Message-ID: <20241122224715.171751-5-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse There's no need to swap pages (which involves three memcopies for each page) in the plain kexec case. Just do a single copy from source to destination page. Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index ed2ae50535dd..92d5dbed3097 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -308,6 +308,9 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) movq %rdi, %rdx /* Save destination page to %rdx */ movq %rsi, %rax /* Save source page to %rax */ =20 + testq %r11, %r11 /* Only actually swap for preserve_context */ + jnz .Lnoswap + /* copy source page to swap page */ movq %r10, %rdi movl $512, %ecx @@ -322,6 +325,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) /* copy swap page to destination page */ movq %rdx, %rdi movq %r10, %rsi +.Lnoswap: movl $512, %ecx rep ; movsq =20 --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CB581DFE03 for ; Fri, 22 Nov 2024 22:47:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315673; cv=none; b=iMeTS/dTO8Kzh5IOr35KnhbHgkbAs/L9ZabKHsoIGV/i714J5kCbBHWkQlgdscg4CVLPrb7/SFynBKixduV6Qsm3ahKVWWWdZ03NXGBtxZTQ9vsAKY/t2Cxt+zQDDeF5k48ZMq0tYjC1xl0laiSvsUru8ZEZc+3uQiyrPPZjcTU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315673; c=relaxed/simple; bh=byQ6ivEm+vCp2AyLnaWi8o2SaVAhlRxQTOxWLvDIwIw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sR9s2rZNhRF01L00nNkhNuWvxWg7R8+ehGZGnm7Bi0DnFOlA+MTrW6x79lBU54XLp73LSrr4qM4/1LspXdmNtvdly1yOXjkjfEIp7lGwmXJ5wUz1Qc8Yj9IfUZmwADvc6Cxpxhu7TCQpv26Si8rlJOZCdvtjdsA8MSvkHjlSxfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=mn8XfcvF; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="mn8XfcvF" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=P2hLAbS9aCZfZzJ8RdHeHtH4P2lTFUxdp5gm5fCBSTI=; b=mn8XfcvFRp84epY7aROG7lolY5 H/CXZilSX26nd32rba3o+48Ck9TjbmqLQz/8S2OSEIeScjAJjmQf542irJeZphGexEzoJc841V/8+ ut3FQM0Us76nypopKkCpS6kKrapTYS235zIzr8KcrFGWCl7DylX6g0aQWgpye2xPmgMOaucQR4Lpv dwnbMkXCdf5wEJx42+9SNI/ZbUCWiGTq4PFb8cICxMnMgdVeG04QoFENphicizFqO2V3B+Bh6cSMq 5JZ7kSK/F7NpY4RsFwjQW6duIKh2XSoGnjv7PHI1zlQDlbcmL0ASY76Q719upyJ0oxfL5CrEavQcn rBZTfmvA==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQs-00000000ma6-01Ef; Fri, 22 Nov 2024 22:47:22 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iiF-0I92; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 05/16] x86/kexec: Invoke copy of relocate_kernel() instead of the original Date: Fri, 22 Nov 2024 22:38:14 +0000 Message-ID: <20241122224715.171751-6-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse This currently calls set_memory_x() from machine_kexec_prepare() just like the 32-bit version does. That's actually a bit earlier than I'd like, as it leaves the page RWX all the time the image is even *loaded*. Subsequent commits will eliminate all the writes to the page between the point it's marked executable in machine_kexec_prepare() the time that relocate_kernel() is running and has switched to the identmap %cr3, so that it can be ROX. But that can't happen until it's moved to the .data section of the kernel, and *that* can't happen until we start executing the copy instead of executing it in place in the kernel .text. So break the circular dependency in those commits by letting it be RWX for now. Signed-off-by: David Woodhouse --- arch/x86/kernel/machine_kexec_64.c | 28 +++++++++++++++++++++------- arch/x86/kernel/relocate_kernel_64.S | 5 ++++- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 9c9ac606893e..3aeb225a0b36 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -156,8 +156,8 @@ static int init_transition_pgtable(struct kimage *image= , pgd_t *pgd) pmd_t *pmd; pte_t *pte; =20 - vaddr =3D (unsigned long)relocate_kernel; - paddr =3D __pa(page_address(image->control_code_page)+PAGE_SIZE); + vaddr =3D (unsigned long)page_address(image->control_code_page) + PAGE_SI= ZE; + paddr =3D __pa(vaddr); pgd +=3D pgd_index(vaddr); if (!pgd_present(*pgd)) { p4d =3D (p4d_t *)get_zeroed_page(GFP_KERNEL); @@ -296,6 +296,7 @@ static void load_segments(void) =20 int machine_kexec_prepare(struct kimage *image) { + void *control_page =3D page_address(image->control_code_page) + PAGE_SIZE; unsigned long start_pgtable; int result; =20 @@ -307,11 +308,17 @@ int machine_kexec_prepare(struct kimage *image) if (result) return result; =20 + set_memory_x((unsigned long)control_page, 1); + return 0; } =20 void machine_kexec_cleanup(struct kimage *image) { + void *control_page =3D page_address(image->control_code_page) + PAGE_SIZE; + + set_memory_nx((unsigned long)control_page, 1); + free_transition_pgtable(image); } =20 @@ -321,6 +328,11 @@ void machine_kexec_cleanup(struct kimage *image) */ void machine_kexec(struct kimage *image) { + unsigned long (*relocate_kernel_ptr)(unsigned long indirection_page, + unsigned long page_list, + unsigned long start_address, + unsigned int preserve_context, + unsigned int host_mem_enc_active); unsigned long page_list[PAGES_NR]; unsigned int host_mem_enc_active; int save_ftrace_enabled; @@ -369,6 +381,8 @@ void machine_kexec(struct kimage *image) page_list[PA_SWAP_PAGE] =3D (page_to_pfn(image->swap_page) << PAGE_SHIFT); =20 + relocate_kernel_ptr =3D control_page; + /* * The segment registers are funny things, they have both a * visible and an invisible part. Whenever the visible part is @@ -388,11 +402,11 @@ void machine_kexec(struct kimage *image) native_gdt_invalidate(); =20 /* now call it */ - image->start =3D relocate_kernel((unsigned long)image->head, - (unsigned long)page_list, - image->start, - image->preserve_context, - host_mem_enc_active); + image->start =3D relocate_kernel_ptr((unsigned long)image->head, + (unsigned long)page_list, + image->start, + image->preserve_context, + host_mem_enc_active); =20 #ifdef CONFIG_KEXEC_JUMP if (image->preserve_context) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 92d5dbed3097..70539b1b9545 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -39,6 +39,7 @@ #define CP_PA_TABLE_PAGE DATA(0x20) #define CP_PA_SWAP_PAGE DATA(0x28) #define CP_PA_BACKUP_PAGES_MAP DATA(0x30) +#define CP_VA_CONTROL_PAGE DATA(0x38) =20 .text .align PAGE_SIZE @@ -99,6 +100,7 @@ SYM_CODE_START_NOALIGN(relocate_kernel) movq %r9, CP_PA_TABLE_PAGE(%r11) movq %r10, CP_PA_SWAP_PAGE(%r11) movq %rdi, CP_PA_BACKUP_PAGES_MAP(%r11) + movq %r11, CP_VA_CONTROL_PAGE(%r11) =20 /* Save the preserve_context to %r11 as swap_pages clobbers %rcx. */ movq %rcx, %r11 @@ -235,7 +237,8 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) movq %rax, %cr3 lea PAGE_SIZE(%r8), %rsp call swap_pages - movq $virtual_mapped, %rax + movq CP_VA_CONTROL_PAGE(%r8), %rax + addq $(virtual_mapped - relocate_kernel), %rax pushq %rax ANNOTATE_UNRET_SAFE ret --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE476189BBA for ; Fri, 22 Nov 2024 22:47:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; cv=none; b=qNVbNw2QLwlYzqtzWoUA2oFcEE2Eug4EuL+ZtStZ01zktw/uJEo7+BatD5jCdbH3m3/+swo8Jy+yysi5BGGpWrGPrZvnxaK5NGvJ8kNhgkQhu6/xjlup6j7Axy2qOkpD4gvouZHl75Qws+0KFtvbmTjZrbnzf9+wQjERSC9C6mw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; c=relaxed/simple; bh=ES94CY4oxmMLlOXPxmho4+dVHtK1sGih6Ia5cWMiqAE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=arXWvjI+r30Sr3EtjEfI6cKbOEomnqJlNwiSrBymi7Y005K73ZBUgIlBuPFJFvFsdvV6sjE44TJymtWCnQ0SglkgIbEno0g9KuOOQFdmT4KetgiUCvJpSHpdPmwMhOgxUqaQI4tSS2oMwgY2W8aepq9Ko96Vv7svsSvdpxqTLOc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=hKxKpX98; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="hKxKpX98" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=ubsMDj/bpOoQF9AOzyotTy2PSXk8cSR2Tvgh3iOVNio=; b=hKxKpX98/aMh88PkgdxNmX/Mgx a3PgEWqnnZCUJ2cg9LsTiZKxQ+3S0A+0NMeDH5D5JYpvJpeOBcwRYMUm+oiTk4b0KUQngIKgCsEgn IygEDtirKxYvjwqL49sJLUm5Bj7k8mHfOR7UF1kqryHBjl9HbMHjrj0fiasy1uXGROFTmf87mvkHf moicpez9/yJhvYiHO/fdMNAsmbTfbXSUjAV6xyhqPtxSWA2fQcJfzXVsE/bsCM7y7av21cH/qTVic 7M9aOe67xm8yYh7hjHvy5Uca5teizdmuLjR+jfWsH+rLzxGbSknn3ZNrdw9l7tHSwhxSXqgZvyu+w pkqHT1zg==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000008PHQ-2EVH; Fri, 22 Nov 2024 22:47:21 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iiI-0RYH; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 06/16] x86/kexec: Move relocate_kernel to kernel .data section Date: Fri, 22 Nov 2024 22:38:15 +0000 Message-ID: <20241122224715.171751-7-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Now that the copy is executed instead of the original, the relocate_kernel page can live in the kernel's .text section. This will allow subsequent commits to actually add real data to it and clean up the code somewhat as well as making the control page ROX. Signed-off-by: David Woodhouse --- arch/x86/include/asm/sections.h | 1 + arch/x86/kernel/machine_kexec_64.c | 4 +++- arch/x86/kernel/relocate_kernel_64.S | 6 +----- arch/x86/kernel/vmlinux.lds.S | 11 ++++++++++- 4 files changed, 15 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/sections.h b/arch/x86/include/asm/section= s.h index 3fa87e5e11ab..30e8ee7006f9 100644 --- a/arch/x86/include/asm/sections.h +++ b/arch/x86/include/asm/sections.h @@ -5,6 +5,7 @@ #include #include =20 +extern char __relocate_kernel_start[], __relocate_kernel_end[]; extern char __brk_base[], __brk_limit[]; extern char __end_rodata_aligned[]; =20 diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 3aeb225a0b36..048868d868ce 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -333,6 +333,8 @@ void machine_kexec(struct kimage *image) unsigned long start_address, unsigned int preserve_context, unsigned int host_mem_enc_active); + unsigned long reloc_start =3D (unsigned long)__relocate_kernel_start; + unsigned long reloc_end =3D (unsigned long)__relocate_kernel_end; unsigned long page_list[PAGES_NR]; unsigned int host_mem_enc_active; int save_ftrace_enabled; @@ -370,7 +372,7 @@ void machine_kexec(struct kimage *image) } =20 control_page =3D page_address(image->control_code_page) + PAGE_SIZE; - __memcpy(control_page, relocate_kernel, KEXEC_CONTROL_CODE_MAX_SIZE); + __memcpy(control_page, __relocate_kernel_start, reloc_end - reloc_start); =20 page_list[PA_CONTROL_PAGE] =3D virt_to_phys(control_page); page_list[VA_CONTROL_PAGE] =3D (unsigned long)control_page; diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 70539b1b9545..085dddf79476 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -41,10 +41,8 @@ #define CP_PA_BACKUP_PAGES_MAP DATA(0x30) #define CP_VA_CONTROL_PAGE DATA(0x38) =20 - .text - .align PAGE_SIZE + .section .text.relocate_kernel,"ax"; .code64 -SYM_CODE_START_NOALIGN(relocate_range) SYM_CODE_START_NOALIGN(relocate_kernel) UNWIND_HINT_END_OF_STACK ANNOTATE_NOENDBR @@ -340,5 +338,3 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) int3 SYM_CODE_END(swap_pages) =20 - .skip KEXEC_CONTROL_CODE_MAX_SIZE - (. - relocate_kernel), 0xcc -SYM_CODE_END(relocate_range); diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index b8c5741d2fb4..925a821134b5 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -95,7 +95,15 @@ const_pcpu_hot =3D pcpu_hot; #define BSS_DECRYPTED =20 #endif - +#if defined(CONFIG_X86_64) && defined(CONFIG_KEXEC_CORE) +#define KEXEC_RELOCATE_KERNEL \ + . =3D ALIGN(0x100); \ + __relocate_kernel_start =3D .; \ + *(.text.relocate_kernel); \ + __relocate_kernel_end =3D .; +#else +#define KEXEC_RELOCATE_KERNEL +#endif PHDRS { text PT_LOAD FLAGS(5); /* R_E */ data PT_LOAD FLAGS(6); /* RW_ */ @@ -181,6 +189,7 @@ SECTIONS =20 DATA_DATA CONSTRUCTORS + KEXEC_RELOCATE_KERNEL =20 /* rarely changed data like cpu maps */ READ_MOSTLY_DATA(INTERNODE_CACHE_BYTES) --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACD391DFD9C for ; Fri, 22 Nov 2024 22:47:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315672; cv=none; b=T7F7iplJ0wik6+Gj6nkDD6VeZdnkpQFCwi2sq87Scf4LqxChHagYBaxpFVvbdYQQhXDn2DzJX6L/aM1edXaBecfLrBc8+jhWxEZAFfkMvjFXa4qKzR5Uzh5N5tirXBZlwxoBGXnIIDPOWoGot62OuPJe/U1ar51bvNV4A8dozfM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315672; c=relaxed/simple; bh=1CQpF3bpcbIZMdH+g7V2kqts4I2IkYRuvHmPq1r0U4A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EvO/egxN7IugTiDhPrrXQhAX5gH4d1hXgwaILSZTPz8Sfstl+W+YvrCom/JNkkgRpBzu2rQWMOsfPtLE1CNHgruj513m4ZeFzdhHNEGBru/0z77USDX59tSORs44uLCkFi+Z0e8OHy09rxZaxrgDJDDuTGm3/+qK+tU8P3kfda8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=UsmPB4sb; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="UsmPB4sb" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=F1IMqtYJfaw0RknrZ4o5TbPrvhsDQP3o18Mfcsk2flo=; b=UsmPB4sblATqM+wzwEDUH9lkVB mJlOVBqgcjNqdKGVDae0KRt7YK2L5snkYDXeKmPdrSA1tJDvsPEdvAruBkUHcje5TelVaPyOps+6b +GoGK4U0MQRguLrsaP54+dJOtlev60+lKdm6DCSurBvzisgsiwJ6nxx9pVKsh6Yx+VkcMxHpXLBA/ Wj0QtCvCDfs8ChUEnvgkMoFXSzf+uhu74yRlv3fKIRhby9fev7408PutbBh8yk8JTz5fIvPV6YPI7 7CtydXLrkJZ5TM/iUHlbDFQSe0iIduXYNsVzXtrqwnfSBsf/ctektIKjxshur2I0GEfHO4Xzmdbzf IpxFLbIQ==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQs-00000000ma7-001i; Fri, 22 Nov 2024 22:47:22 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iiL-0aRO; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 07/16] x86/kexec: Add data section to relocate_kernel Date: Fri, 22 Nov 2024 22:38:16 +0000 Message-ID: <20241122224715.171751-8-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Now that the relocate_kernel page is handled sanely by a linker script we can have actual data, and just use %rip-relative addressing to access it. Signed-off-by: David Woodhouse --- arch/x86/kernel/machine_kexec_64.c | 7 +++- arch/x86/kernel/relocate_kernel_64.S | 62 ++++++++++++++-------------- arch/x86/kernel/vmlinux.lds.S | 1 + 3 files changed, 37 insertions(+), 33 deletions(-) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 048868d868ce..123e9544506b 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -383,7 +383,12 @@ void machine_kexec(struct kimage *image) page_list[PA_SWAP_PAGE] =3D (page_to_pfn(image->swap_page) << PAGE_SHIFT); =20 - relocate_kernel_ptr =3D control_page; + /* + * Allow for the possibility that relocate_kernel might not be at + * the very start of the page. + */ + relocate_kernel_ptr =3D control_page + (unsigned long)relocate_kernel - + reloc_start; =20 /* * The segment registers are funny things, they have both a diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 085dddf79476..445ca56dabbe 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -23,23 +23,21 @@ #define PAGE_ATTR (_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED | _PAGE_DIRTY) =20 /* - * control_page + KEXEC_CONTROL_CODE_MAX_SIZE - * ~ control_page + PAGE_SIZE are used as data storage and stack for - * jumping back + * The .text.relocate_kernel and .data.relocate_kernel sections are copied + * into the control page, and the remainder of the page is used as the sta= ck. */ -#define DATA(offset) (KEXEC_CONTROL_CODE_MAX_SIZE+(offset)) =20 + .section .data.relocate_kernel,"a"; /* Minimal CPU state */ -#define RSP DATA(0x0) -#define CR0 DATA(0x8) -#define CR3 DATA(0x10) -#define CR4 DATA(0x18) - -/* other data */ -#define CP_PA_TABLE_PAGE DATA(0x20) -#define CP_PA_SWAP_PAGE DATA(0x28) -#define CP_PA_BACKUP_PAGES_MAP DATA(0x30) -#define CP_VA_CONTROL_PAGE DATA(0x38) +SYM_DATA_LOCAL(saved_rsp, .quad 0) +SYM_DATA_LOCAL(saved_cr0, .quad 0) +SYM_DATA_LOCAL(saved_cr3, .quad 0) +SYM_DATA_LOCAL(saved_cr4, .quad 0) + /* other data */ +SYM_DATA_LOCAL(va_control_page, .quad 0) +SYM_DATA_LOCAL(pa_table_page, .quad 0) +SYM_DATA_LOCAL(pa_swap_page, .quad 0) +SYM_DATA_LOCAL(pa_backup_pages_map, .quad 0) =20 .section .text.relocate_kernel,"ax"; .code64 @@ -63,14 +61,13 @@ SYM_CODE_START_NOALIGN(relocate_kernel) pushq %r15 pushf =20 - movq PTR(VA_CONTROL_PAGE)(%rsi), %r11 - movq %rsp, RSP(%r11) + movq %rsp, saved_rsp(%rip) movq %cr0, %rax - movq %rax, CR0(%r11) + movq %rax, saved_cr0(%rip) movq %cr3, %rax - movq %rax, CR3(%r11) + movq %rax, saved_cr3(%rip) movq %cr4, %rax - movq %rax, CR4(%r11) + movq %rax, saved_cr4(%rip) =20 /* Save CR4. Required to enable the right paging mode later. */ movq %rax, %r13 @@ -83,10 +80,11 @@ SYM_CODE_START_NOALIGN(relocate_kernel) movq %r8, %r12 =20 /* - * get physical address of control page now + * get physical and virtual address of control page now * this is impossible after page table switch */ movq PTR(PA_CONTROL_PAGE)(%rsi), %r8 + movq PTR(VA_CONTROL_PAGE)(%rsi), %r11 =20 /* get physical address of page table now too */ movq PTR(PA_TABLE_PAGE)(%rsi), %r9 @@ -95,10 +93,10 @@ SYM_CODE_START_NOALIGN(relocate_kernel) movq PTR(PA_SWAP_PAGE)(%rsi), %r10 =20 /* save some information for jumping back */ - movq %r9, CP_PA_TABLE_PAGE(%r11) - movq %r10, CP_PA_SWAP_PAGE(%r11) - movq %rdi, CP_PA_BACKUP_PAGES_MAP(%r11) - movq %r11, CP_VA_CONTROL_PAGE(%r11) + movq %r9, pa_table_page(%rip) + movq %r10, pa_swap_page(%rip) + movq %rdi, pa_backup_pages_map(%rip) + movq %r11, va_control_page(%rip) =20 /* Save the preserve_context to %r11 as swap_pages clobbers %rcx. */ movq %rcx, %r11 @@ -229,13 +227,13 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) /* get the re-entry point of the peer system */ movq 0(%rsp), %rbp leaq relocate_kernel(%rip), %r8 - movq CP_PA_SWAP_PAGE(%r8), %r10 - movq CP_PA_BACKUP_PAGES_MAP(%r8), %rdi - movq CP_PA_TABLE_PAGE(%r8), %rax + movq pa_swap_page(%rip), %r10 + movq pa_backup_pages_map(%rip), %rdi + movq pa_table_page(%rip), %rax movq %rax, %cr3 lea PAGE_SIZE(%r8), %rsp call swap_pages - movq CP_VA_CONTROL_PAGE(%r8), %rax + movq va_control_page(%rip), %rax addq $(virtual_mapped - relocate_kernel), %rax pushq %rax ANNOTATE_UNRET_SAFE @@ -246,11 +244,11 @@ SYM_CODE_END(identity_mapped) SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped) UNWIND_HINT_END_OF_STACK ANNOTATE_NOENDBR // RET target, above - movq RSP(%r8), %rsp - movq CR4(%r8), %rax + movq saved_rsp(%rip), %rsp + movq saved_cr4(%rip), %rax movq %rax, %cr4 - movq CR3(%r8), %rax - movq CR0(%r8), %r8 + movq saved_cr3(%rip), %rax + movq saved_cr0(%rip), %r8 movq %rax, %cr3 movq %r8, %cr0 =20 diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 925a821134b5..324c1c42faae 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -100,6 +100,7 @@ const_pcpu_hot =3D pcpu_hot; . =3D ALIGN(0x100); \ __relocate_kernel_start =3D .; \ *(.text.relocate_kernel); \ + *(.data.relocate_kernel); \ __relocate_kernel_end =3D .; #else #define KEXEC_RELOCATE_KERNEL --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE4581DFD9D for ; Fri, 22 Nov 2024 22:47:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315672; cv=none; b=ukVMwrkQae3CvdrsdNO/SUGd5NqWRaUx9yVlaJ8Dzq46wN7qtHcYOVfcxVbbLy43uSTpDPB7pXHoasop8b9hj6AJnMFXLCEpdvrk5fxIjK1eu1FI3PVA28NbLYRawxmVDmxN4EJUA0AVyIdt5ldA2tAAIk6IrrIh2nFG4DuFX7Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315672; c=relaxed/simple; bh=oHixD4Rd4yh/kdDo4sDeeS4XxEV56OlFc1W88/QTwrI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pSTxbCPXWJPYgVluiwAn4+g4RTf//Cx5B5z+OzZjWQWJ7Df7GwptL0Drc4kSclNRTu9d6igipagxpJK4M23R58htc6LI89eFine0ZYYh9wSFItKv9BTd4KZUObkV7QeSxL2h6lJNqjIeHcXth5KmJggOHEUmBRwpRpgxvxS+IXU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=VtJlafzs; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="VtJlafzs" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=qYFRgzDbimC0blaDJOJhby5s1NuEeujIYbn61y3E6GM=; b=VtJlafzsjOxL1ciibmV+zNF2mD jJmOJnM3bMCLXyECAn33h6KmC7anHGDQhnxf54de4TgaQdixwGqRfU5m08YsRuxiONe/8qUh7NQG3 UUf+PVyb72aQW/l0c2+SfQreYi3jdmOrvLfINByqdzBzoqDXCrb9ezQQ6XlXPOlUivXD+13GDPDno DAUDeMuOYkyLG9GbdoQMlSMT5ejfxUOnmqIMMFTd+qcfGhZPgqIXu4hGWsFShlV3KVLkc9IWs2F3j GCEY9Rk7VTcQu1Uvv5Nnw8Eia1VP1DQipm1DJrKCkn+W7Djc6qw15WQgeMtielY3kqspM4AAwu+IW kq/Dek2g==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQs-00000000ma4-00Xq; Fri, 22 Nov 2024 22:47:22 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iiO-0jy2; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 08/16] x86/kexec: Copy control page into place in machine_kexec_prepare() Date: Fri, 22 Nov 2024 22:38:17 +0000 Message-ID: <20241122224715.171751-9-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse There's no need for this to wait until the actual machine_kexec() invocatio= n; a subsequent change will mark the control page ROX so all writes should be completed earlier. Signed-off-by: David Woodhouse --- arch/x86/kernel/machine_kexec_64.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 123e9544506b..60632a5a2a13 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -297,6 +297,8 @@ static void load_segments(void) int machine_kexec_prepare(struct kimage *image) { void *control_page =3D page_address(image->control_code_page) + PAGE_SIZE; + unsigned long reloc_start =3D (unsigned long)__relocate_kernel_start; + unsigned long reloc_end =3D (unsigned long)__relocate_kernel_end; unsigned long start_pgtable; int result; =20 @@ -308,6 +310,8 @@ int machine_kexec_prepare(struct kimage *image) if (result) return result; =20 + __memcpy(control_page, __relocate_kernel_start, reloc_end - reloc_start); + set_memory_x((unsigned long)control_page, 1); =20 return 0; @@ -334,7 +338,6 @@ void machine_kexec(struct kimage *image) unsigned int preserve_context, unsigned int host_mem_enc_active); unsigned long reloc_start =3D (unsigned long)__relocate_kernel_start; - unsigned long reloc_end =3D (unsigned long)__relocate_kernel_end; unsigned long page_list[PAGES_NR]; unsigned int host_mem_enc_active; int save_ftrace_enabled; @@ -372,7 +375,6 @@ void machine_kexec(struct kimage *image) } =20 control_page =3D page_address(image->control_code_page) + PAGE_SIZE; - __memcpy(control_page, __relocate_kernel_start, reloc_end - reloc_start); =20 page_list[PA_CONTROL_PAGE] =3D virt_to_phys(control_page); page_list[VA_CONTROL_PAGE] =3D (unsigned long)control_page; --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D15E61DFDA4 for ; Fri, 22 Nov 2024 22:47:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315672; cv=none; b=cDea9YBmrHiq53iI0Ofi3sPH8GhJ3/5eUYXdWifkt1bEVR9FosNMsIEEi2BN7kFXGsoQb35PTl5p49i75jyoz6u5pbZdToCMa75gNn59nO+RC5xLQl7xFVxHgcnfgZtqw7QZ0EeXtjZnnCj/u/hyVIQcyVtX5UPedtekfB6s8fY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315672; c=relaxed/simple; bh=uRUahdtDavNRTasiCR5XGsVM1KcJ5DDqEdAjGSbsx4o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dtl5XM4iH2UdkUYacZMNn2DNKqH6hKsYiYXRKxAEuWgXhH53Myjsf5TUTHBRYmv0r30mrGC+JZsifY6owdxMQprIQOMdyF6pixKflWoRQXpQl0Fhh2KszUi6qAIguX5vfl2XtbONKg5XUpM3Vca34Bf/lN5tX2sWZuPnbFxTWfw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=B/+vrUUN; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="B/+vrUUN" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=eNmI8QbdRntdJMuGHB3pdtes6ciQ8jd8QPd+JuE3tjw=; b=B/+vrUUNGVQSSZ80qj/dVWDO8b xH0sZ0Z/f2jXrKXQSO27b0NMruU6YSQ02ZyDNfo7WIMQLa0xOqJtqe6enamTcekvryT83TRiMFHL6 Bdg5tUiYmOsRjkE0+IcHd77tHkQmPu91VLjkqHug4BFRK61Z6AJ4grOVDgo7SNjTia2LT5aA/JmNc 9AtViSfYmDHrnnwArVF7ll9mgD7IpSYSuIqYjoLua41WniJ7cVZIKtaNASPAc426vvtuX2yG0W6UX vuaaGRG0MnS0C4gJYNH3Kz9xOuAV1iK3aMJ3NuX3rVvraC/B0tTukhbzcwcP9+RH5t11I8Con1GY2 +V4gJYfA==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQs-00000000ma8-00bk; Fri, 22 Nov 2024 22:47:22 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iiR-0ynt; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 09/16] x86/kexec: Drop page_list argument from relocate_kernel() Date: Fri, 22 Nov 2024 22:38:18 +0000 Message-ID: <20241122224715.171751-10-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse The kernel's virtual mapping of the relocate_kernel page currently needs to be RWX because it is written to before the %cr3 switch. Now that the relocate_kernel page has its own .data section and local variables, it can also have *global* variables. So eliminate the separate page_list argument, and write the same information directly to variables in the relocate_kernel page instead. This way, the relocate_kernel code itself doesn't need to copy it. Signed-off-by: David Woodhouse --- arch/x86/include/asm/kexec.h | 13 +++++----- arch/x86/kernel/machine_kexec_64.c | 21 +++++++--------- arch/x86/kernel/relocate_kernel_64.S | 36 ++++++++++------------------ 3 files changed, 27 insertions(+), 43 deletions(-) diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h index ae5482a2f0ca..9af54743de90 100644 --- a/arch/x86/include/asm/kexec.h +++ b/arch/x86/include/asm/kexec.h @@ -8,12 +8,6 @@ # define PA_PGD 2 # define PA_SWAP_PAGE 3 # define PAGES_NR 4 -#else -# define PA_CONTROL_PAGE 0 -# define VA_CONTROL_PAGE 1 -# define PA_TABLE_PAGE 2 -# define PA_SWAP_PAGE 3 -# define PAGES_NR 4 #endif =20 # define KEXEC_CONTROL_CODE_MAX_SIZE 2048 @@ -63,6 +57,11 @@ struct kimage; =20 /* The native architecture */ # define KEXEC_ARCH KEXEC_ARCH_X86_64 + +extern unsigned long kexec_pa_control_page; +extern unsigned long kexec_va_control_page; +extern unsigned long kexec_pa_table_page; +extern unsigned long kexec_pa_swap_page; #endif =20 /* @@ -125,7 +124,7 @@ relocate_kernel(unsigned long indirection_page, #else unsigned long relocate_kernel(unsigned long indirection_page, - unsigned long page_list, + unsigned long pa_control_page, unsigned long start_address, unsigned int preserve_context, unsigned int host_mem_enc_active); diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 60632a5a2a13..c653c2c22d63 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -309,6 +309,13 @@ int machine_kexec_prepare(struct kimage *image) result =3D init_pgtable(image, start_pgtable); if (result) return result; + kexec_va_control_page =3D (unsigned long)control_page; + kexec_pa_table_page =3D + (unsigned long)__pa(page_address(image->control_code_page)); + + if (image->type =3D=3D KEXEC_TYPE_DEFAULT) + kexec_pa_swap_page =3D (page_to_pfn(image->swap_page) + << PAGE_SHIFT); =20 __memcpy(control_page, __relocate_kernel_start, reloc_end - reloc_start); =20 @@ -333,12 +340,11 @@ void machine_kexec_cleanup(struct kimage *image) void machine_kexec(struct kimage *image) { unsigned long (*relocate_kernel_ptr)(unsigned long indirection_page, - unsigned long page_list, + unsigned long pa_control_page, unsigned long start_address, unsigned int preserve_context, unsigned int host_mem_enc_active); unsigned long reloc_start =3D (unsigned long)__relocate_kernel_start; - unsigned long page_list[PAGES_NR]; unsigned int host_mem_enc_active; int save_ftrace_enabled; void *control_page; @@ -376,15 +382,6 @@ void machine_kexec(struct kimage *image) =20 control_page =3D page_address(image->control_code_page) + PAGE_SIZE; =20 - page_list[PA_CONTROL_PAGE] =3D virt_to_phys(control_page); - page_list[VA_CONTROL_PAGE] =3D (unsigned long)control_page; - page_list[PA_TABLE_PAGE] =3D - (unsigned long)__pa(page_address(image->control_code_page)); - - if (image->type =3D=3D KEXEC_TYPE_DEFAULT) - page_list[PA_SWAP_PAGE] =3D (page_to_pfn(image->swap_page) - << PAGE_SHIFT); - /* * Allow for the possibility that relocate_kernel might not be at * the very start of the page. @@ -412,7 +409,7 @@ void machine_kexec(struct kimage *image) =20 /* now call it */ image->start =3D relocate_kernel_ptr((unsigned long)image->head, - (unsigned long)page_list, + virt_to_phys(control_page), image->start, image->preserve_context, host_mem_enc_active); diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 445ca56dabbe..b9ad3ef0b982 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -34,9 +34,9 @@ SYM_DATA_LOCAL(saved_cr0, .quad 0) SYM_DATA_LOCAL(saved_cr3, .quad 0) SYM_DATA_LOCAL(saved_cr4, .quad 0) /* other data */ -SYM_DATA_LOCAL(va_control_page, .quad 0) -SYM_DATA_LOCAL(pa_table_page, .quad 0) -SYM_DATA_LOCAL(pa_swap_page, .quad 0) +SYM_DATA(kexec_va_control_page, .quad 0) +SYM_DATA(kexec_pa_table_page, .quad 0) +SYM_DATA(kexec_pa_swap_page, .quad 0) SYM_DATA_LOCAL(pa_backup_pages_map, .quad 0) =20 .section .text.relocate_kernel,"ax"; @@ -46,7 +46,7 @@ SYM_CODE_START_NOALIGN(relocate_kernel) ANNOTATE_NOENDBR /* * %rdi indirection_page - * %rsi page_list + * %rsi pa_control_page * %rdx start address * %rcx preserve_context * %r8 host_mem_enc_active @@ -79,31 +79,19 @@ SYM_CODE_START_NOALIGN(relocate_kernel) /* Save SME active flag */ movq %r8, %r12 =20 - /* - * get physical and virtual address of control page now - * this is impossible after page table switch - */ - movq PTR(PA_CONTROL_PAGE)(%rsi), %r8 - movq PTR(VA_CONTROL_PAGE)(%rsi), %r11 - - /* get physical address of page table now too */ - movq PTR(PA_TABLE_PAGE)(%rsi), %r9 - - /* get physical address of swap page now */ - movq PTR(PA_SWAP_PAGE)(%rsi), %r10 - - /* save some information for jumping back */ - movq %r9, pa_table_page(%rip) - movq %r10, pa_swap_page(%rip) + /* save indirection list for jumping back */ movq %rdi, pa_backup_pages_map(%rip) - movq %r11, va_control_page(%rip) =20 /* Save the preserve_context to %r11 as swap_pages clobbers %rcx. */ movq %rcx, %r11 =20 /* Switch to the identity mapped page tables */ + movq kexec_pa_table_page(%rip), %r9 movq %r9, %cr3 =20 + /* Physical address of control page */ + movq %rsi, %r8 + /* setup a new stack at the end of the physical control page */ lea PAGE_SIZE(%r8), %rsp =20 @@ -227,13 +215,13 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) /* get the re-entry point of the peer system */ movq 0(%rsp), %rbp leaq relocate_kernel(%rip), %r8 - movq pa_swap_page(%rip), %r10 + movq kexec_pa_swap_page(%rip), %r10 movq pa_backup_pages_map(%rip), %rdi - movq pa_table_page(%rip), %rax + movq kexec_pa_table_page(%rip), %rax movq %rax, %cr3 lea PAGE_SIZE(%r8), %rsp call swap_pages - movq va_control_page(%rip), %rax + movq kexec_va_control_page(%rip), %rax addq $(virtual_mapped - relocate_kernel), %rax pushq %rax ANNOTATE_UNRET_SAFE --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE3D5184549 for ; Fri, 22 Nov 2024 22:47:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; cv=none; b=Y72Odh8oT7dLPM1icOM+F5fEYhvYqt7O/k0JhTf9GkC9jwOfRPz8xolo165lggD8jt4dfuVgyTPluDqixg+x0i8uyt8n0B0JIbL5zLsqLGZyI0I2F3zc0KdGwJRHfS67GafXxgxqq/oqWBfwIedAy53fqNw6H1XTzpRuK4YduCg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; c=relaxed/simple; bh=y3/a5/Ks2TnZdRoPI7F6nLHbCWNFoxgQiFOw1wppEkE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=M+vOYy7Klk1VWfMAaNbpGTrifIc7Wcg2q2OlNqDkPjBPRBFJKVMWcEBpHir7+JUNDtv07RDayYgOXgeK3jb1JjwthOrsQBlHSOmPj9m2ncdoU7pCcoLyGTEhr+cWCgUIA7nGgTq/ALOpMgObqGWV1lqi2Zwx45OsJwxwSXhfB1w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=qcrkkUW+; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="qcrkkUW+" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=i2Ix5g+A/s6rVJXMB+lf9O3jUtCCFkDMNyc2WtM36JM=; b=qcrkkUW+O+/LLv5VlEXxqoKLUq vqS/kjBYrGoaV3SjGC0ZScvV8oYy/boURhybxSSDQXD+Tp8FemmCIWF5XEzKcx6W4bFHN+UydTbml iWfPdQTa9cIDlbkR5SJ7MPCttKEufeYq6l/VjTS4OVEw7QlriyfOg68wB5UwlfKLcks39Xs5gnvOz IN20cSOlitZHRRd7x7Naq+yFXBgwefuXdIiTmarYgXcKtHGABNxW5ODj84J7V9AUiP01TFVMsLakU 7t9WNZ8qnvUdRwXJBaldpbwt87xi/+Jq3TP3bqPmMHEo98AEYsKZ33PJmdbG5p07douIckQr5hM7k BhkuZG0w==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQt-00000008PI5-15Yv; Fri, 22 Nov 2024 22:47:24 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iiU-18nA; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 10/16] x86/kexec: Eliminate writes through kernel mapping of relocate_kernel page Date: Fri, 22 Nov 2024 22:38:19 +0000 Message-ID: <20241122224715.171751-11-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse All writes to the relocate_kernel control page are now done *after* the %cr3 switch via simple %rip-relative addressing, which means the DATA() macro with its pointer arithmetic can also now be removed. Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 29 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index b9ad3ef0b982..5c6456467f08 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -61,21 +61,24 @@ SYM_CODE_START_NOALIGN(relocate_kernel) pushq %r15 pushf =20 - movq %rsp, saved_rsp(%rip) - movq %cr0, %rax - movq %rax, saved_cr0(%rip) - movq %cr3, %rax - movq %rax, saved_cr3(%rip) - movq %cr4, %rax - movq %rax, saved_cr4(%rip) - - /* Save CR4. Required to enable the right paging mode later. */ - movq %rax, %r13 - /* zero out flags, and disable interrupts */ pushq $0 popfq =20 + /* Switch to the identity mapped page tables */ + movq %cr3, %rax + movq kexec_pa_table_page(%rip), %r9 + movq %r9, %cr3 + + /* Save %rsp and CRs. */ + movq %rsp, saved_rsp(%rip) + movq %rax, saved_cr3(%rip) + movq %cr0, %rax + movq %rax, saved_cr0(%rip) + /* Leave CR4 in %r13 to enable the right paging mode later. */ + movq %cr4, %r13 + movq %r13, saved_cr4(%rip) + /* Save SME active flag */ movq %r8, %r12 =20 @@ -85,10 +88,6 @@ SYM_CODE_START_NOALIGN(relocate_kernel) /* Save the preserve_context to %r11 as swap_pages clobbers %rcx. */ movq %rcx, %r11 =20 - /* Switch to the identity mapped page tables */ - movq kexec_pa_table_page(%rip), %r9 - movq %r9, %cr3 - /* Physical address of control page */ movq %rsi, %r8 =20 --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A2771DFE0C for ; Fri, 22 Nov 2024 22:47:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315674; cv=none; b=U3zDDXIb70i9qDj20V7BWSmbepECJGEjnIbaV1kaLO+Jr0Bt4lit0wJFwbLY8VM4QSE2tKIBunlC7A8wzhOq89oLpWeXhMKxTLnZ/jgovUhJtN5tuVL5fzU3ceSBL8myReRrxFVdon9dNw2RReSS/cdGepSusQ+tIYoaDOpPLm8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315674; c=relaxed/simple; bh=HL3ux3eNef6SY3eR0/deQX8VN0VjBmggbUcVLonrZgs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VUNFeMAUYmB7S41UMuAS0vo2ZXlBMQ+8xg1FtLp15l0/tJTkYmjiEEAhmed38VXb6bDywgEuVwFRNcAzEZKpdZmBwaprFWJuxSNhwttJv+U3XkGLhfyi2pvyG+uOxDcRiMhHDwRYc5tt1XuMaVPrl4lw63NoPN+dtcKxF+6icw8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=UCav4PgT; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="UCav4PgT" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=j4VdY6af5RlPVPqmFUSdejWtC3j5qgVFRN++43ZJ7hw=; b=UCav4PgTGKePskAf9vMzi2epjA ROEgahlU9adV5GwqBDCw6RkMxLuvEL3NmwTdYDa6JYPraR6L7Vz/HDfQlFqya5WgP8X8Z0u+Sfg3R g1Nrw+YOE0NZrUeVwb6+ScNURx23Tg9T7PHCMSSVE+Zu25kzg08smKJAydcFDnoFEEc6t6fVtfydS uWcQjA+diU9kAdazzVvn5a6oqatNbJRPj1qJRx5PcVAMxypTiz64HOjtN7ohq/rAkT8emD3PpE4po TsCD2P6jE6+fVBIG+qkkvzqCJa9xzb5VfnjBH4i3vTPBMbepEimM1js5CaaP6t/3W04W2wv30rt2Z 5Lj4kMkA==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQv-00000000maK-3CSF; Fri, 22 Nov 2024 22:47:32 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iiX-1KwP; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 11/16] x86/kexec: Clean up register usage in relocate_kernel() Date: Fri, 22 Nov 2024 22:38:20 +0000 Message-ID: <20241122224715.171751-12-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse The memory encryption flag is passed in %r8 because that's where the calling convention puts it. Instead of moving it to %r12 and then using %r8 for other things, just leave it in %r8 and use other registers instead. Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 5c6456467f08..51dc55ac4395 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -79,24 +79,18 @@ SYM_CODE_START_NOALIGN(relocate_kernel) movq %cr4, %r13 movq %r13, saved_cr4(%rip) =20 - /* Save SME active flag */ - movq %r8, %r12 - /* save indirection list for jumping back */ movq %rdi, pa_backup_pages_map(%rip) =20 /* Save the preserve_context to %r11 as swap_pages clobbers %rcx. */ movq %rcx, %r11 =20 - /* Physical address of control page */ - movq %rsi, %r8 - /* setup a new stack at the end of the physical control page */ - lea PAGE_SIZE(%r8), %rsp + lea PAGE_SIZE(%rsi), %rsp =20 /* jump to identity mapped page */ - addq $(identity_mapped - relocate_kernel), %r8 - pushq %r8 + addq $(identity_mapped - relocate_kernel), %rsi + pushq %rsi ANNOTATE_UNRET_SAFE ret int3 @@ -107,8 +101,9 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) /* * %rdi indirection page * %rdx start address + * %r8 host_mem_enc_active + * %r9 page table page * %r11 preserve_context - * %r12 host_mem_enc_active * %r13 original CR4 when relocate_kernel() was invoked */ =20 @@ -161,7 +156,7 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) * entries that will conflict with the now unencrypted memory * used by kexec. Flush the caches before copying the kernel. */ - testq %r12, %r12 + testq %r8, %r8 jz .Lsme_off wbinvd .Lsme_off: --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 971B518B47C for ; Fri, 22 Nov 2024 22:47:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; cv=none; b=C8/v2w65gFGF56ZApLHclVRgqTPFomGgaEpqIJWbSWE/sSubudRpm7xgOtTu4Ut5L7P4IbCiD4ivYkNkge2m7c3iE44WN1YZ8IzVsDvepG74q+ZjZmVsj4iSp41KzT7+De2hl+zEB1tRb9t/6GL1T9aP+eBgNGzwLzevy/A/ong= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; c=relaxed/simple; bh=RGUKNDHGWOazvs/c7hrffgjfh6HEUXX7zQWnEARD2gs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h7SCOo78D5+YGGU7agpyr5cxPq4BaIF37UdMQuyvhhjg2l6Z5FJon3ecYlHoKrRB46+TJmsgFvHSCRMAQkN0DrZWA1f0QzRHVby8fbofnWbVZb8oyBesLALR2cislRi5NfjJP9AlO2x8sn4bTk4BiSKQjfDDF1gvKEIGiOj6K88= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=CZEHxI3q; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="CZEHxI3q" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=sLmOWyrmuAgxGhbmd1Cx2AL9gzQ7rBcfLnxnrcUkLFI=; b=CZEHxI3qag6H7vBnQzDyG84huZ 6QJ6owj1EX3+VI1h+UzfeQysyMxoa9CHS3YZfLFiQ+2fhmwV/ginQhEBpqB1b0AxDedvvD/kYpV2H jaFB0bXlJdSP0NFTO7UKSMgVIQetKhNWHawYDEZHKdOV7sNbKMZF5F/SKMKWlEd6zhkIEgQKxEILZ R6oUOzyCoN64mkI2D+nK9V/3FxC6ISvMvIduIBex7Y0bNMLVSU7r5rQAxfNNnEVzhnA8H2afkCtRM z53rRZIn6ppFhBO3Q4EMsyHwCLZsEn++faPY2fv5hxkk6pDRbZGOjel6/1MYbQpitZH1Odh3xf0Ey hu3/jr8w==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQt-00000008PI6-1TwR; Fri, 22 Nov 2024 22:47:24 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iia-1Wwr; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 12/16] x86/kexec: Mark relocate_kernel page as ROX instead of RWX Date: Fri, 22 Nov 2024 22:38:21 +0000 Message-ID: <20241122224715.171751-13-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse All writes to the page now happen before it gets marked as executable (or after it's already switched to the identmap page tables where it's OK to be RWX). Signed-off-by: David Woodhouse --- arch/x86/kernel/machine_kexec_64.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index c653c2c22d63..2a294daeeb1a 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -319,7 +319,7 @@ int machine_kexec_prepare(struct kimage *image) =20 __memcpy(control_page, __relocate_kernel_start, reloc_end - reloc_start); =20 - set_memory_x((unsigned long)control_page, 1); + set_memory_rox((unsigned long)control_page, 1); =20 return 0; } @@ -329,6 +329,7 @@ void machine_kexec_cleanup(struct kimage *image) void *control_page =3D page_address(image->control_code_page) + PAGE_SIZE; =20 set_memory_nx((unsigned long)control_page, 1); + set_memory_rw((unsigned long)control_page, 1); =20 free_transition_pgtable(image); } --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB3641DFE3C for ; Fri, 22 Nov 2024 22:47:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315675; cv=none; b=pje/MrAUwtY4sRxP251M6Nv8coImQBw/nL3nwvGHC0qDAeyglwIdmpGwQCE3wiHN2zGQD4xC1fSOhGG/gTDyYGupxF5NYfEqJR+Ixlo2fpecPsK6DnB01Xk2AoAC7oOKBsEkd8iRwGqTt//Z5SHVzNStLCV6wfOIRqZahIIfDok= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315675; c=relaxed/simple; bh=78bUkO/aNPFK2xJZsagiCKKRakQWzU9ImoWzrpNtIjE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MQzuk+gnip5mB7l0hzf3JmRGx/wHAQ+3/soloiv+6ifdtSfWDaK4XGfcIvKkjngTyMrSARGr2C0/i8WZwPim1ryv2kt4oyS7vxMCEwUm0P97BN9GjjBkSzcIV9cLuutqb370XecrFjWTM0XqPQ3Ngam7a4p8uIdC/66FYk0Rv/Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=rF3QGRXP; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="rF3QGRXP" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=PM8M+JIu8uZjpEQfel8bzCRRPJYH7fSk33eUzo/nnU0=; b=rF3QGRXPlEJmi4zzK2yeUu9q6o +MQhF7dc6fsRF6mRBksfr6ZvhnvFHLqOWro4HxxT9Hktex8e2Gln1L/BAKF3gBBgvhqyPCF8NdkZD rTi5X/0XG2g//sjIrvTY4H4+ma2j9wmoDBw7Kgvlfz1NXdneyys8+DIntjjw5n50wU83wapX1neYI CVvTFerkxoO86GxLLykO44W5BSwliMZWG8BijcOvUijggdAdJcpYRJI7cie73Meo0VV5f35wwwFh7 HjpwpSnPNO3TYvLqI1aQ34kgOF/xAqYvlDGif8EullkmJtpGzcR44F/Jt9spM5/p7NS9qaJZWGGr7 q7fa0QIQ==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQv-00000000maL-3C75; Fri, 22 Nov 2024 22:47:36 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iid-1guO; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 13/16] x86/kexec: Debugging support: load a GDT Date: Fri, 22 Nov 2024 22:38:22 +0000 Message-ID: <20241122224715.171751-14-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse There are some failure modes which lead to triple-faults in the relocate_kernel function, which is fairly much undebuggable for normal mortals. Adding a GDT in the relocate_kernel environment is step 1 towards being able to catch faults and do something more useful. Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 51dc55ac4395..5c174829f794 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -39,6 +39,18 @@ SYM_DATA(kexec_pa_table_page, .quad 0) SYM_DATA(kexec_pa_swap_page, .quad 0) SYM_DATA_LOCAL(pa_backup_pages_map, .quad 0) =20 +#ifdef DEBUG +SYM_DATA_START_LOCAL(reloc_kernel_gdt) + .balign 16 + .word reloc_kernel_gdt_end - reloc_kernel_gdt - 1 + .long 0 + .word 0 + .quad 0x00cf9a000000ffff /* __KERNEL32_CS */ + .quad 0x00af9a000000ffff /* __KERNEL_CS */ + .quad 0x00cf92000000ffff /* __KERNEL_DS */ +SYM_DATA_END_LABEL(reloc_kernel_gdt, SYM_L_LOCAL, reloc_kernel_gdt_end) +#endif /* DEBUG */ + .section .text.relocate_kernel,"ax"; .code64 SYM_CODE_START_NOALIGN(relocate_kernel) @@ -112,6 +124,21 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) /* store the start address on the stack */ pushq %rdx =20 +#ifdef DEBUG + /* Create a GDTR (16 bits limit, 64 bits addr) on stack */ + leaq reloc_kernel_gdt(%rip), %rax + pushq %rax + pushw (%rax) + + /* Load the GDT, put the stack back */ + lgdt (%rsp) + addq $10, %rsp + + /* Test that we can load segments */ + movq %ds, %rax + movq %rax, %ds +#endif /* DEBUG */ + /* * Clear X86_CR4_CET (if it was set) such that we can clear CR0_WP * below. --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B77A91A2C04 for ; Fri, 22 Nov 2024 22:47:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315674; cv=none; b=Xm1pWqgNiMjrxovy9IigSoYd/iJiPP3mxarG2RH97RJN1VX6w1uNk0TiJn4quSshRetoUNb8pTs+x4i/CMqHR4qjO2u5yxq+PniTqFuNakEkl0jHIjspZTMGxQvKwmoWx/q8PrhGkiu5IqGeCKgKuDoBjwvywYZLRU5d4koRiOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315674; c=relaxed/simple; bh=Bhs04KRfRQvGIUrz4dzn3/izuilfGY5/AIEzWkc0b4w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qL8yMuG369JDynDrCgz72tArfY4BDXEpiiYiO+C43wphQnjEDpLDmDWuylnbSHXrLeWNQw1Kz1JojdJ2G8bN7LHtAGK/3K3p0YGIq0WHOTOn2Broy6YWB7QMgm4ENxr+rZlkO0Cf2x6kjtr831o7D/8552AGXpmZ/OWOFqsNh3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=OUrawLPe; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="OUrawLPe" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=PEUnuyo3lv8xn/euDjI9tjOuOh4F6Q7bzHZN/CcSabk=; b=OUrawLPeNmRQKiVkHFQSboDgmt /DeTlAS1+leaw+52uRjWR6ujt9jDCzmuKKltyNXb4CQXaVJrX7GBcZyJoghGlPEsT2hTtFB/HdeQe yyUdd+E46ifHaJ2lZa9wCoA1UzDruhF+z0zvakY/i+NUJYkhw1AY35dyK3vX7qbWvhBZs3d3NB+5h pBMeW1r7IIBJDuAxJETD4uWt5BJhQI8VivET1q0P2jZQ7dCSMy9HwhKg5HPkeQ7ZEr68lvWuMHdEA 7PxGobCyvV2fHYqbbmbY2lgjMqlGgHObpbcUXuETQDQikofC10Fxyr0pLCkMs+cOx0icjVP1Psf8D Wh39c2sg==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQv-00000000maM-3Cpk; Fri, 22 Nov 2024 22:47:36 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iig-1wFO; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 14/16] x86/kexec: Debugging support: Load an IDT and basic exception entry points Date: Fri, 22 Nov 2024 22:38:23 +0000 Message-ID: <20241122224715.171751-15-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 114 +++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 5c174829f794..4ace2577afc6 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -40,6 +40,9 @@ SYM_DATA(kexec_pa_swap_page, .quad 0) SYM_DATA_LOCAL(pa_backup_pages_map, .quad 0) =20 #ifdef DEBUG + /* Size of each exception handler referenced by the IDT */ +#define EXC_HANDLER_SIZE 6 /* pushi, pushi, 2-byte jmp */ + SYM_DATA_START_LOCAL(reloc_kernel_gdt) .balign 16 .word reloc_kernel_gdt_end - reloc_kernel_gdt - 1 @@ -108,6 +111,11 @@ SYM_CODE_START_NOALIGN(relocate_kernel) int3 SYM_CODE_END(relocate_kernel) =20 +#ifdef DEBUG + UNWIND_HINT_UNDEFINED + .balign 0x100 /* relocate_kernel will be overwritten with an IDT */ +#endif + SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) UNWIND_HINT_END_OF_STACK /* @@ -137,6 +145,52 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) /* Test that we can load segments */ movq %ds, %rax movq %rax, %ds + + /* Load address of reloc_kernel, at start of this page, into %rsi */ + lea relocate_kernel(%rip), %rsi + + /* + * Build an IDT descriptor in %rax/%rbx. The address is in the low 16 + * and high 16 bits of %rax, and low 32 of %rbx. The niddle 32 bits + * of %rax hold the selector/ist/flags which are hard-coded below. + */ + movq %rsi, %rax // 1234567890abcdef + + andq $-0xFFFF, %rax // 1234567890ab.... + shlq $16, %rax // 567890ab........ + + movq $0x8F000010, %rcx // Present, DPL0, Interrupt Gate, __KERNEL_CS. + orq %rcx, %rax // 567890ab8F000010 + shlq $16, %rax // 90ab8F000010.... + + movq %rsi, %rcx + andq $0xffff, %rcx // ............cdef + orq %rcx, %rax // 90ab87000010cdef + + movq %rsi, %rbx + shrq $32, %rbx + + /* + * The descriptor was built using the address of relocate_kernel. Add + * the required offset to point to the actual entry points. + */ + addq $(exc_vectors - relocate_kernel), %rax + + /* Loop 16 times to handle exception 0-15 */ + movq $16, %rcx +1: + movq %rax, (%rsi) + movq %rbx, 8(%rsi) + addq $16, %rsi + addq $EXC_HANDLER_SIZE, %rax + loop 1b + + /* Now put an IDTR on the stack (temporarily) to load it */ + subq $0x100, %rsi + pushq %rsi + pushw $0xff + lidt (%rsp) + addq $10, %rsp #endif /* DEBUG */ =20 /* @@ -345,3 +399,63 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) int3 SYM_CODE_END(swap_pages) =20 +#ifdef DEBUG +SYM_CODE_START_LOCAL_NOALIGN(exc_vectors) + /* Each of these is 6 bytes. */ +.macro vec_err exc + UNWIND_HINT_ENTRY + . =3D exc_vectors + (\exc * EXC_HANDLER_SIZE) + nop + nop + pushq $\exc + jmp exc_handler +.endm + +.macro vec_noerr exc + UNWIND_HINT_ENTRY + . =3D exc_vectors + (\exc * EXC_HANDLER_SIZE) + pushq $0 + pushq $\exc + jmp exc_handler +.endm + + vec_noerr 0 // #DE + vec_noerr 1 // #DB + vec_noerr 2 // #NMI + vec_noerr 3 // #BP + vec_noerr 4 // #OF + vec_noerr 5 // #BR + vec_noerr 6 // #UD + vec_noerr 7 // #NM + vec_err 8 // #DF + vec_noerr 9 + vec_err 10 // #TS + vec_err 11 // #NP + vec_err 12 // #SS + vec_err 13 // #GP + vec_err 14 // #PF + vec_noerr 15 +SYM_CODE_END(exc_vectors) + +SYM_CODE_START_LOCAL_NOALIGN(exc_handler) + pushq %rax + pushq %rdx + movw $0x3f8, %dx + movb $'A', %al + outb %al, %dx + popq %rdx + popq %rax + + /* Only return from int3 */ + cmpq $3, (%rsp) + jne .Ldie + + addq $16, %rsp + iretq + +.Ldie: + hlt + jmp .Ldie + +SYM_CODE_END(exc_handler) +#endif /* DEBUG */ --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9722718C033 for ; Fri, 22 Nov 2024 22:47:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; cv=none; b=ZreZRoOOgJUrSeYb0nqOpMQexpz+Vkp5IOrUNbFzFeNhTtnwCuW5mRvcs7VggPdfYu3I2wQelIooidGeZV61RNdJFVEU7apRD8FZoqBKgfPjQp4Wc71ZjCc7b5NI6jvywqtj//JEVCClLpYhJkCy92EMCVk0qlfxZezlCZTvIp8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315666; c=relaxed/simple; bh=mm6W+60mWEXMtTcR54u9llsIS2mCxnvfy6OpRrFSG4o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jm68f7x4W21kAjbzwau853qmLAfUFvEDVKv/4HtNHW1ky9YzFGLlQezkzYi/zlbqtdta8oVzfvPSTb/19WQgzFj9bdoyK3607x++XiGthCgl9m06uxuIXcPx10mQJl9umgxI7EBdL0G4Nxqk5RMNmXOiA2viJCp3TlJYNhaEq10= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=qxKUp010; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="qxKUp010" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=p6nY0MqUdf1P9uopFxDVE2/tcFa/fWVkj9Ig2+H4ywM=; b=qxKUp010qnr2xUv2WEZk2EODFm jy1AblzHSPGfKfPB1FQlMN96+BoifoWUal7RbRHmHpTJ2SAOjo0tc0xe+G4//I7WQl+Jin1Y0ejP/ tGq2mrI1+zITB+m/EqVtTH0GfPrtq0yKGRFiKLWhxIRHIo94gNSgo+Bzsw3usdCidIp4qhvNP4GhZ QTSDNIo81E8i3YQxdCbd3R58ats/BDEfo56K0/ZtytISBcLLc3wk9Yw+Ner5cw1NLuM8OOxE6gun/ Tg3B+WVmHCDjBZ0Ka2SSHoLgd089Ob1hdwMeIbHEuesWEOBpZG+Cod+gybg8tx7zgRf0s5swZl5AO N41CjXFw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQt-00000008PI7-27yK; Fri, 22 Nov 2024 22:47:24 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iij-2Bxx; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 15/16] x86/kexec: Debugging support: Dump registers on exception Date: Fri, 22 Nov 2024 22:38:24 +0000 Message-ID: <20241122224715.171751-16-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 83 +++++++++++++++++++++++++++- 1 file changed, 80 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 4ace2577afc6..67f6853c7abe 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -400,6 +400,55 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) SYM_CODE_END(swap_pages) =20 #ifdef DEBUG +/* + * This allows other types of serial ports to be used. + * - %al: Character to be printed (no clobber %rax) + * - %rdx: MMIO address or port. + */ +.macro pr_char + outb %al, %dx +.endm + +/* Print the nybble in %bl, clobber %rax */ +SYM_CODE_START_LOCAL_NOALIGN(pr_nybble) + UNWIND_HINT_FUNC + movb %bl, %al + nop + andb $0x0f, %al + addb $0x30, %al + cmpb $0x3a, %al + jb 1f + addb $('a' - '0' - 10), %al +1: pr_char + ANNOTATE_UNRET_SAFE + ret +SYM_CODE_END(pr_nybble) + +SYM_CODE_START_LOCAL_NOALIGN(pr_qword) + UNWIND_HINT_FUNC + movq $16, %rcx +1: rolq $4, %rbx + call pr_nybble + loop 1b + movb $'\n', %al + pr_char + ANNOTATE_UNRET_SAFE + ret +SYM_CODE_END(pr_qword) + +.macro print_reg a, b, c, d, r + movb $\a, %al + pr_char + movb $\b, %al + pr_char + movb $\c, %al + pr_char + movb $\d, %al + pr_char + movq \r, %rbx + call pr_qword +.endm + SYM_CODE_START_LOCAL_NOALIGN(exc_vectors) /* Each of these is 6 bytes. */ .macro vec_err exc @@ -439,11 +488,39 @@ SYM_CODE_END(exc_vectors) =20 SYM_CODE_START_LOCAL_NOALIGN(exc_handler) pushq %rax + pushq %rbx + pushq %rcx pushq %rdx + movw $0x3f8, %dx - movb $'A', %al - outb %al, %dx + + /* rip and exception info */ + print_reg 'E', 'x', 'c', ':', 32(%rsp) + print_reg 'E', 'r', 'r', ':', 40(%rsp) + print_reg 'r', 'i', 'p', ':', 48(%rsp) + + /* We spilled these to the stack */ + print_reg 'r', 'a', 'x', ':', 24(%rsp) + print_reg 'r', 'b', 'x', ':', 16(%rsp) + print_reg 'r', 'c', 'x', ':', 8(%rsp) + print_reg 'r', 'd', 'x', ':', (%rsp) + + /* Other registers */ + print_reg 'r', 's', 'i', ':', %rsi + print_reg 'r', 'd', 'i', ':', %rdi + print_reg 'r', '8', ' ', ':', %r8 + print_reg 'r', '9', ' ', ':', %r9 + print_reg 'r', '1', '0', ':', %r10 + print_reg 'r', '1', '1', ':', %r11 + print_reg 'r', '1', '2', ':', %r12 + print_reg 'r', '1', '3', ':', %r13 + print_reg 'r', '1', '4', ':', %r14 + print_reg 'r', '1', '5', ':', %r15 + print_reg 'c', 'r', '2', ':', %cr2 + popq %rdx + popq %rcx + popq %rbx popq %rax =20 /* Only return from int3 */ @@ -456,6 +533,6 @@ SYM_CODE_START_LOCAL_NOALIGN(exc_handler) .Ldie: hlt jmp .Ldie - + int3 SYM_CODE_END(exc_handler) #endif /* DEBUG */ --=20 2.47.0 From nobody Mon Feb 9 01:35:32 2026 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B77FA1A3034 for ; Fri, 22 Nov 2024 22:47:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315674; cv=none; b=GM97HbBAHxl/sMTX9WAuWdoAC4/Wpc/0oyEZuBeESqDhhwLFVJV0OnqiH/AzeRcn88PuJ2dW/JfepAZoQcHFkqgqC0Ohm8wet6KxCdbUIb6RLnA8qUefhyYw4CJpvjplgoSgaWFtGbpKwS3MOHhyKlAz5TVb5obQeC2Q2iki6XA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732315674; c=relaxed/simple; bh=8Rz20baMO9chwr7g9P6Po0JLe92g35121XuVgdhLAP0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MYhuw9Www8Lr0m0UCKlaD9AebHPfZApAam6F5kdc5jhh1F3M6C0a9UnkUilJ1049Ar77QZBURCUZr08m8g9iolWZAhdvq9JJofGIW5I29EwZ1UFZG+wwVOqhdtW6IGizZ7M0MOInogDMmNgvUdmssw/B8P7b/EfcDltEi6UVFA4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=eqPlq+8P; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="eqPlq+8P" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=JN/n6oVdrchdFC0rEj9EsgKyog6/YyewaWd3kp8GrwA=; b=eqPlq+8Pcnn9Wm6QN/urM3oJlU /8Z/I/qLC3p8juCl8bKwHxS0W35gQExd/FT0b4Zapz6vZSPgBm0RbBshNbgiHmqgRw5Tc9Rq32qHi r1e5SwhtUrCnrtShNKCXaV8WKRjP35ewfXGpTEmSjRgvxpFERgCqphf3MyBMAGDH5LFG15A+iMw2B jozuJMz8kyg5YqdfoKYSobRNEEXhSS8e1kqL6SVjR62MXrXuVZciFl9Y2X4u4YoAd7ojZJliOqOOZ t0KqIuTOdhSm4s3zSJK1Phemaew0ngdCyw6Z4VvT/giAySZYoZJNYbtBSemSQ8tbQnpTRGFlK1fq0 M91XDnYw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQv-00000000maN-3CPb; Fri, 22 Nov 2024 22:47:36 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tEcQp-00000000iim-2L1u; Fri, 22 Nov 2024 22:47:19 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org Subject: [RFC PATCH v2 16/16] [DO NOT MERGE] x86/kexec: enable DEBUG Date: Fri, 22 Nov 2024 22:38:25 +0000 Message-ID: <20241122224715.171751-17-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241122224715.171751-1-dwmw2@infradead.org> References: <20241122224715.171751-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 67f6853c7abe..ebbd76c9a3e9 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -14,6 +14,8 @@ #include #include =20 +#define DEBUG + /* * Must be relocatable PIC code callable as a C function, in particular * there must be a plain RET and not jump to return thunk. @@ -191,6 +193,8 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) pushw $0xff lidt (%rsp) addq $10, %rsp + + int3 #endif /* DEBUG */ =20 /* --=20 2.47.0