From nobody Wed Dec 17 21:26:10 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6D3E219E93 for ; Thu, 9 Jan 2025 14:08:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431715; cv=none; b=bfhs6ToFIG+iQ2ZI7ZILdlzYuKmTUmFt+UvHPSKhye7NZCEY+inBpTP3xtqEeoA8XKL3GBP4VWBwR6pUsb50k7ASAT/ENoW8VeAZc8bhFfyN7MWHNb3zQMMC5YxVSwMgH9UTDSf1pQ9vytaUV352Kpx3ucwTbDRvXDtXjaIB1XI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431715; c=relaxed/simple; bh=dsUwMySuT8tHVtzcoztx43It5rTbBxKhZFdWvZoAD2o=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dVN25jTsuzuWOTGWpgTuhDFLe4F8tVEzu/BdqPsPTKsIJrDlntS3QTgnshmTprQllSrTsRq4jMhjhzqSS35LIJXhT6MmIIP3hc4N0xDx9n5eoRsbNGbOtbGjabvvM3tGS9T5Dca3JZDb3vbscTPpRQSYgNhpl/V6zpKIIhEmw0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=Ue0IbLkC; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Ue0IbLkC" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To: From:Reply-To:Cc:Content-ID:Content-Description; bh=VVE+d9HRCiUw5Gsyt81FEKKZcrEQhELhIN4TTfzz0EI=; b=Ue0IbLkC2r3iyYMn9RE/ADGrKh l8ZeSdSIhrzMBtk/TOqiXmmvf441QRFu7i3Ailv3DK/6axDmb2HHB0XvZI3pi2aCAvgHGKeIR/2fv 3uZIokMC8e0bPKkPb52gou9yxmfFi3pGMOFjQMFG5JFzx1r0pNmD6CYXCw70ya+ShltDA2vMlhTWA lFutfvEpofpnfZARlWN2grExuOA79YmlHxwduZItT8Vkg/JEa8D6QFU5i2zpXdFCDxC3q09mvVZxl hqh1AsmIBoQWBIV6zIvSMZhfCmdI6fGyuBoeUg7J6As4O2o+UmkW2VWCGECaM5Abi4lvO5WaJKtYg qIhQrUJw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCZ-00000006IQF-2Doj; Thu, 09 Jan 2025 14:07:59 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx5q-2Hvq; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 1/9] x86/kexec: Disable global pages before writing to control page Date: Thu, 9 Jan 2025 14:04:13 +0000 Message-ID: <20250109140757.2841269-2-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse The kernel switches to a new set of page tables during kexec. The global mappings (_PAGE_GLOBAL=3D=3D1) can remain in the TLB after this switch. This is generally not a problem because the new page tables use a different portion of the virtual address space than the normal kernel mappings. The critical exception to that generalisation (and the only mapping which isn't an identity mapping) is the kexec control page itself =E2=80=94 which was ROX in the original kernel mapping, but should be RWX in the new page tables. If there is a global TLB entry for that in its prior read-only state, it definitely needs to be flushed before attempting to write through that virtual mapping. It would be possible to just avoid writing to the virtual address of the page and defer all writes until they can be done through the identity mapping. But there's no good reason to keep the old TLB entries around, as they can cause nothing but trouble. Clear the PGE bit in %cr4 early, before storing data in the control page. Fixes: 5a82223e0743 ("x86/kexec: Mark relocate_kernel page as ROX instead o= f RWX") Co-authored-by: Dave Hansen Reported-by: Nathan Chancellor Reported-by: "Ning, Hongyu" Closes: https://bugzilla.kernel.org/show_bug.cgi?id=3D219592 Signed-off-by: Dave Hansen Signed-off-by: David Woodhouse Tested-by: Nathan Chancellor Tested-by: "Ning, Hongyu" --- arch/x86/kernel/relocate_kernel_64.S | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 8bc86a1e056a..9bd601dd8659 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -70,14 +70,20 @@ SYM_CODE_START_NOALIGN(relocate_kernel) movq kexec_pa_table_page(%rip), %r9 movq %r9, %cr3 =20 + /* Leave CR4 in %r13 to enable the right paging mode later. */ + movq %cr4, %r13 + + /* Disable global pages immediately to ensure this mapping is RWX */ + movq %r13, %r12 + andq $~(X86_CR4_PGE), %r12 + movq %r12, %cr4 + /* Save %rsp and CRs. */ + movq %r13, saved_cr4(%rip) movq %rsp, saved_rsp(%rip) movq %rax, saved_cr3(%rip) movq %cr0, %rax movq %rax, saved_cr0(%rip) - /* Leave CR4 in %r13 to enable the right paging mode later. */ - movq %cr4, %r13 - movq %r13, saved_cr4(%rip) =20 /* save indirection list for jumping back */ movq %rdi, pa_backup_pages_map(%rip) base-commit: 893930143440eb5e3ea8f69cb51ab2e61e15c4e1 --=20 2.47.0 From nobody Wed Dec 17 21:26:10 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 251C721B1A7 for ; Thu, 9 Jan 2025 14:08:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431717; cv=none; b=VAGtJgZyqnwhjBr2sC3C2HWVJ6Ml/rGaeAmwZYX7tuO05CCIIvFisi/vjDSYYSHwIwiO2O1sUtSaOiMIiI0pGq7wvszUhhEQIEV8BRFT4+JRK43KCikoyZ33ktulKSZ8xP2VIRr4uXw+GDYEdV5PhhD3ADBANTZRQB84CE5qeQU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431717; c=relaxed/simple; bh=g/Tb1scAluPn+1pgI2bFIEcG4fxvjCU1ZiUBIcliJgc=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SJcPgEhHpJkfwZyHnKG/aR+5IR0plt3qMgO01n5qCNqPzzv9a9mIO3bBirWOBGwt4eQazfCRjhrzwYj3gu6Gx7wviXUomGe0b/sievJohteJ6vPUUzc7Gixci3ZblVpfHoLK+5tE/pD25nxSIAvgB3Ml0rAzFV3tNPWu/VbU15Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=X10vdTEH; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="X10vdTEH" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=4GPX4uH5hPGTEsJKbCTX7Iw1UzlZ9qs8SxieW1Q8mSw=; b=X10vdTEH3d8k/EovQ7NgJsE1gp /eYVbxf/SQQRz1A+vrsYn6w9+3KdWDJsWoZ3UbUl17CWnLRKk1AUfECnEo9/hWsD0L4CxBHoaQHiz Ooh7xWg0e7BAgsTKQXg3cCmMz7dl2xaeDxYSYNYJgS46pjdMw/eaaJ80+Kxt4YFM8w8Uh4+R9FBBQ wAcV08AjhrsBWj5E1+bvokq5Nke3q4hjHOxSbBRJpo5/HLGARwDXMV+qlozDq0U7pu7LnyNPceJCL Av6pBMgufuj8o2vg5svMeqVFOU+t5S0gPekQZ/JY02Zz3k6RjPhlkuoB3VTns9eNRPUQv0bFz53pf D29t6BpQ==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCZ-00000006IQG-2Uet; Thu, 09 Jan 2025 14:07:59 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx5x-2Ygt; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 2/9] x86/kexec: Ensure preserve_context flag is set on return to kernel Date: Thu, 9 Jan 2025 14:04:14 +0000 Message-ID: <20250109140757.2841269-3-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse The swap_pages function will only actually *swap*, as its name implies, if the preserve_context flag in the %r11 register is non-zero. On the way back from a ::preserve_context kexec, ensure that the %r11 register is non-zero so that the pages get swapped back. Fixes: 9e5683e2d0b5 ("x86/kexec: Only swap pages for ::preserve_context mod= e") Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 9bd601dd8659..6fce4b459652 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -220,6 +220,7 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) movq kexec_pa_table_page(%rip), %rax movq %rax, %cr3 lea PAGE_SIZE(%r8), %rsp + movl $1, %r11d /* Ensure preserve_context flag is set */ call swap_pages movq kexec_va_control_page(%rip), %rax addq $(virtual_mapped - relocate_kernel), %rax --=20 2.47.0 From nobody Wed Dec 17 21:26:10 2025 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74FE4219A74 for ; Thu, 9 Jan 2025 14:08:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431715; cv=none; b=jUDPs789A3VbLORWIf9dkNpN2FRXBlVbysRfAOaTkgkNgzUe5QyE1EYa5IaZ4llGF5sj3+QFnKsT+n5eZoHAFRzj9ulCgRkQsyybo5fmkkcVRl48H4KLDh33m9Ps6/P2DO5xo3lknCrpjzRjKYH/Yu2FbtEHD3u+JCIyFA6MPEA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431715; c=relaxed/simple; bh=lFfg/xVgGCpkMthS9+K7x/uEqbtknHfNHHBj6VuQVSw=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cjd0WpwKLGTRv2Jz2GfdSIT8RX/2h3XiO68OGx5QuefSEP1tT05Bv8y7x/xd5CcRcB/3yXhQ20E58U1RX5lqeIP2mTDJF7wdEBBmijncPUFdYIG6r2uOrfU3p47X8hF+Ve3qLQr46cT9O30ssjtBBHaOqsFQdqy23U/oF0PzZIQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=hlBqLaT9; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="hlBqLaT9" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=Fe3DN3ndw9ctJtc0ND815AU97XruK260fddKsjsDX6w=; b=hlBqLaT9Cgu8uFFmIH0eBCfh1P IFq6xNRdw+UbHAw6WBfhV95Rzu4M9q1ZJvwUpy4jPXYdl3IomhJvcTSvksbAStshQrDi+16j3828+ 70WWTQ4JuWeT5wRSgphebXocZ8ee+ke+KxjXYT2BEzudFGHs1V/8WzL9TZCDDenwdMfztmw0REJYU 9j4xD/Sl6eHWeaa0yfzulf8XH+xzh+4lWW/3VueWoXCyvxebchOFIcEuZOW1YubiBOBWac8x7gRP3 jHk+YCxHaVV4imH+ZMW6taepLJ1z26wnvYqAGKjRSSTaXf8SzGF6PZBRMoqy1nPY4eWCXg9yh1JHP sGefLqmw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCa-00000009Zhf-3Dct; Thu, 09 Jan 2025 14:08:02 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx65-2p5c; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 3/9] x86/kexec: Use correct swap page in swap_pages function Date: Thu, 9 Jan 2025 14:04:15 +0000 Message-ID: <20250109140757.2841269-4-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse The swap_pages function expects the swap page to be in %r10, but there was no documentation to that effect. Once upon a time the setup code used to load its value from a kernel virtual address and save it to an address which is accessible in the identity-mapped page tables, and *happened* to use %r10 to do so, with no comment that it was left there on *purpose* instead of just being a scratch register. Once that was no longer necessary, %r10 just holds whatever the kernel happened to leave in it. Now that the original value passed by the kernel is accessible via %rip-relative addressing, load directly from there instead of using %r10 for it. But document the other parameters that the swap_pages function *does* expect in registers. Fixes: b3adabae8a96 ("x86/kexec: Drop page_list argument from relocate_kern= el()") Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 6fce4b459652..3ca3bf6b3f49 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -264,6 +264,10 @@ SYM_CODE_END(virtual_mapped) /* Do the copies */ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) UNWIND_HINT_END_OF_STACK + /* + * %rdi indirection page + * %r11 preserve_context + */ movq %rdi, %rcx /* Put the indirection_page in %rcx */ xorl %edi, %edi xorl %esi, %esi @@ -302,7 +306,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) jz .Lnoswap =20 /* copy source page to swap page */ - movq %r10, %rdi + movq kexec_pa_swap_page(%rip), %rdi movl $512, %ecx rep ; movsq =20 @@ -314,7 +318,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) =20 /* copy swap page to destination page */ movq %rdx, %rdi - movq %r10, %rsi + movq kexec_pa_swap_page(%rip), %rsi .Lnoswap: movl $512, %ecx rep ; movsq --=20 2.47.0 From nobody Wed Dec 17 21:26:10 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F26082165EA for ; Thu, 9 Jan 2025 14:08:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431713; cv=none; b=bmmluEtM7G4ZemHf4A1/lzeVhHgmiU74b/jGjY1bagQp4JzRPsPdM/Ov0nwZi5c1D1iTwprErOUgNdaVNcoXKqDSQwqH/3RXoQ5mF50Hidlq+zFL5vxIuzJniHE+SdY40xcebvT8aLSG3v0OUlPVrujGMWyFwkinpeZSx76wCGM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431713; c=relaxed/simple; bh=zDsIFhPAwITcTn1KoMkiuUhegSn2QXu70egLTjdASl8=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X9mzDCmGJSE1S5KaPcoqRfjG0nTNFS1mfFKsjMN4vbrT8c830ynrETbQLhCPt4Vj9B0Mtl/EpUSA9EdE5+DtVnQ60Kg7XO3EECj01is5kISAoAwhf3BX3TI9p+/CBRQGiR4pl4gbPSGhphYLdLZO0x3/sOJwudfpzOMcg6LH/NU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=eEewo2Dq; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="eEewo2Dq" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=UXXMy8NjgmXapI75A+lfqUhBwTskcOXo5TKctXbko48=; b=eEewo2DqTqYsZN0oqse9fjK050 Bkp6wVR+qVdk9ZRZSMunW1WYpzMyKt0yh2LCYqLhNpBiMAmQ5BXufoemNej9jBhlVSpklEAgHxuxU 0hZPf5RkIyhWuUqxzTjdZ8duCJA16X3u5Yi/Vvn0rd7ZpnhyRW7EdLJtAdx/HNg/l/S5wCrc2u4xv uQ1FFZWzIqiLIlaYIJ77sDhTe8rUa5l53vQlSPNoP7KD0ZQy6PN2P2EtjRGL7tlEcfGezlOOgehl0 3jFiKFQTgMNa/u58uh16wSpIbLNbghwn1TnE5Ty3XbboyMYnykVXQ0hkg9+hv0hXLHyCMwEOUCO5E UHlfmhow==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCZ-00000006IQH-30ko; Thu, 09 Jan 2025 14:07:59 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx6B-35Gp; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 4/9] x86/kexec: Fix stack and handling of re-entry point for ::preserve_context Date: Thu, 9 Jan 2025 14:04:16 +0000 Message-ID: <20250109140757.2841269-5-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse A ::preserve_context kimage can be invoked more than once, and the entry point can be different every time. When the callee returns to the kernel, it leaves the address of its entry point for next time on the stack. That being the case, one might reasonably assume that the caller would allocate space for it on the stack fram before actually performing the 'call' into the callee. Apparently not, though. Ever since the kjump code was first added in 2009, it has set up a *new* stack at the top of the swap_page scratch page, then just performed the 'call' without allocating any space for the re-entry address to be returned. It then reads the re-entry point for next time from 0(%rsp) which is actually the first qword of the page *after* the swap page, which might not exist at all! And if the callee has written to that, then it will have corrupted memory it doesn't own. Correct this by pushing the entry point of the callee onto the stack before calling it. The callee may then adjust it, or not, as it sees fit, and subsequent invocations should work correctly either way. Remove a stray push of zero to the *relocate_kernel* stack, which may have been intended for this purpose, but which was actually just noise. Also, loading the stack for the callee relied on the address of the swap page being in %r10 without ever documenting that fact. Recent code changes made that no longer true, so load it directly from the local kexec_pa_swap_page variable instead. Fixes: b3adabae8a96 ("x86/kexec: Drop page_list argument from relocate_kern= el()") Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 3ca3bf6b3f49..a95691b42c5c 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -113,8 +113,6 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) * %r13 original CR4 when relocate_kernel() was invoked */ =20 - /* set return address to 0 if not preserving context */ - pushq $0 /* store the start address on the stack */ pushq %rdx =20 @@ -208,12 +206,19 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) =20 .Lrelocate: popq %rdx + + /* Use the swap page for the callee's stack */ + movq kexec_pa_swap_page(%rip), %r10 leaq PAGE_SIZE(%r10), %rsp + + /* push the existing entry point onto the callee's stack */ + pushq %rdx + ANNOTATE_RETPOLINE_SAFE call *%rdx =20 /* get the re-entry point of the peer system */ - movq 0(%rsp), %rbp + popq %rbp leaq relocate_kernel(%rip), %r8 movq kexec_pa_swap_page(%rip), %r10 movq pa_backup_pages_map(%rip), %rdi @@ -247,6 +252,7 @@ SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped) lgdt saved_context_gdt_desc(%rax) #endif =20 + /* relocate_kernel() returns the re-entry point for next time */ movq %rbp, %rax =20 popf --=20 2.47.0 From nobody Wed Dec 17 21:26:10 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BCC7218EBF for ; Thu, 9 Jan 2025 14:08:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431714; cv=none; b=I/i46nudsjzxDwi0az1O7NpLp4Zmlu7MXiyg9mSBBabYuQiu6qvVKAusphiUX8BRDnLBiIWfyldoFWOhlTg46KzJBh5l++HLLPep8l/a5ds1hz7M8snvvUjZQtCc1mJiEL3q249qGnKDGD2DozlIXM5bIXrq+xQ/fwQsP4ScVYQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431714; c=relaxed/simple; bh=rpWT/GDc1n0O+aerKv/DMHw5wA2p1TibxIbW3iHqs7Q=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WHRiDLdXE544fYoIjhnZsIuhhESkzCqm3LBdfunD1CEoQaWhz68AmnuhsCzpWbkvhErbhFeSjj2fEjupgZ0ncX7i8F7JPmHufLZRlzLctQbT2FLBjdrrahHZ/3pcFmlKtqx8u24e+/jcYS1o7xTVmNOoON30VIlJiZKQPNIDZzA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=Qg7ot0dX; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Qg7ot0dX" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=lpLME9O4Hrgm0YpKuGQMslbI8cpLYo5TkEgKVoEpH/Y=; b=Qg7ot0dXIkUrzgylpdi1LK+NTp 4UQiZZu6ioGgls8rwTSSTJmJjZ6u5fqTryHrC+c0zH+aqinRTMxTU0nLmFMhrs6pCFCy8q1UxEc4q X2S2KTeijTnBan2P4L26rBVJM/VokS5Ap6i3OZ65qaSYYapabPf/x1b4bNgJVJf8M2/ifNklOKymF 6J6atdSGWLL4LtAYJZKhpdJb6lHv0p9018qxP9jR38r0LulxMCaz960dr9FatHXgAQ7ra1kzITO10 QhE0gWj4986pnc8tZwPrVZgbxYgNHLxyaMcREpHkLGc80HZxfgPuYsi7D7qFkBPOUuXtwdTTlrMng YE0vBURw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCZ-00000006IQI-3C1i; Thu, 09 Jan 2025 14:07:59 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx6F-3GY5; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 5/9] x86/kexec: Fix location of relocate_kernel with -ffunction-sections Date: Thu, 9 Jan 2025 14:04:17 +0000 Message-ID: <20250109140757.2841269-6-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: Nathan Chancellor After commit cb33ff9e063c ("x86/kexec: Move relocate_kernel to kernel .data section"), kernels configured with an option that uses -ffunction-sections, such as CONFIG_LTO_CLANG, crash when kexecing because the value of relocate_kernel does not match the value of __relocate_kernel_start so incorrect code gets copied via machine_kexec_prepare(). $ llvm-nm good-vmlinux &| rg relocate_kernel ffffffff83280d41 T __relocate_kernel_end ffffffff83280b00 T __relocate_kernel_start ffffffff83280b00 T relocate_kernel $ llvm-nm bad-vmlinux &| rg relocate_kernel ffffffff83266100 D __relocate_kernel_end ffffffff83266100 D __relocate_kernel_start ffffffff8120b0d8 T relocate_kernel When -ffunction-sections is enabled, TEXT_MAIN matches on '.text.[0-9a-zA-Z_]*' to coalesce the function specific functions back into .text during link time after they have been optimized. Due to the placement of TEXT_TEXT before KEXEC_RELOCATE_KERNEL in the x86 linker script, the .text.relocate_kernel section ends up in .text instead of .data. Use a second dot in the relocate_kernel section name to avoid matching on TEXT_MAIN, which matches a similar situation that happened in commit 79cd2a11224e ("x86/retpoline,kprobes: Fix position of thunk sections with CONFIG_LTO_CLANG"), which allows kexec to function properly. While .data.relocate_kernel still ends up in the .data section via DATA_MAIN -> DATA_DATA, ensure it is located with the .text.relocate_kernel section as intended by performing the same transformation. Fixes: cb33ff9e063c ("x86/kexec: Move relocate_kernel to kernel .data secti= on") Fixes: 8dbec5c77bc3 ("x86/kexec: Add data section to relocate_kernel") Signed-off-by: Nathan Chancellor Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 6 +++--- arch/x86/kernel/vmlinux.lds.S | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index a95691b42c5c..14ed40b7f322 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -23,11 +23,11 @@ #define PAGE_ATTR (_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED | _PAGE_DIRTY) =20 /* - * The .text.relocate_kernel and .data.relocate_kernel sections are copied + * The .text..relocate_kernel and .data..relocate_kernel sections are copi= ed * into the control page, and the remainder of the page is used as the sta= ck. */ =20 - .section .data.relocate_kernel,"a"; + .section .data..relocate_kernel,"a"; /* Minimal CPU state */ SYM_DATA_LOCAL(saved_rsp, .quad 0) SYM_DATA_LOCAL(saved_cr0, .quad 0) @@ -39,7 +39,7 @@ SYM_DATA(kexec_pa_table_page, .quad 0) SYM_DATA(kexec_pa_swap_page, .quad 0) SYM_DATA_LOCAL(pa_backup_pages_map, .quad 0) =20 - .section .text.relocate_kernel,"ax"; + .section .text..relocate_kernel,"ax"; .code64 SYM_CODE_START_NOALIGN(relocate_kernel) UNWIND_HINT_END_OF_STACK diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 0c893997f023..63ff60a11be5 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -100,8 +100,8 @@ const_pcpu_hot =3D pcpu_hot; #define KEXEC_RELOCATE_KERNEL \ . =3D ALIGN(0x100); \ __relocate_kernel_start =3D .; \ - *(.text.relocate_kernel); \ - *(.data.relocate_kernel); \ + *(.text..relocate_kernel); \ + *(.data..relocate_kernel); \ __relocate_kernel_end =3D .; =20 ASSERT(__relocate_kernel_end - __relocate_kernel_start <=3D KEXEC_CONTROL_= CODE_MAX_SIZE, --=20 2.47.0 From nobody Wed Dec 17 21:26:10 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F25B52165E7 for ; Thu, 9 Jan 2025 14:08:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431713; cv=none; b=Q9MrWiTYmKB2leVdv7bA7IjtHUi9+OQr2Z7cSKu0oyoP3fCbxsUsoIhdNM4G4Ca1XBPCc3NDbqJAe8MkyTu/GmdFJhmArQ3h7imoclae5EO0uwie8JSzcqXBrtTjLDtrEbSSny4X2bsKaRB/ssuoTzraOwNPUwf9RNziL9Z7qTk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431713; c=relaxed/simple; bh=pD/iM6D1Uc9Ku+aH6t9jBfBnAst+5WgG7HrJPvYhw58=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rKD/NIRPsk1pMGzUSQn15b5Tg+RhHJB6GeJ6aBToa1JPjfVYonfzvSDiy3GKemFCjDupCp/tURL4P+aZmmMY4LRHhgilG+e2N4HHVJaNLaNLHEE3FPZBuFyTg9d4j9GtX2raAZKfBMSWZuuhlj0ll1/rg1GGAG9bW6qWW18vi7Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=ayFO2bub; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ayFO2bub" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To: From:Reply-To:Cc:Content-ID:Content-Description; bh=KF9I8gMwuVGDi5vu6drdRG3X3dHP+mrVfyO8mKUOGvk=; b=ayFO2bubDxOj1KYg7nlMD/P8px C6a8e1XvolmwF+rYqh7lXuzDY+tB871wjnfQsny8+naV2xZNG3KYcmopHtIE2Z97GIfiP5A2Ek9iX 5RANKGBe4Q35241suVWSdTfz3Asr3XVBu9OmR/EEWmvcWa2n1KUyXho5BQWHJDjx6rFqbaWrW2c/P 5JeGZvcKiquuNPYEY9a5wJRKPKtQv4OY3OdEHIFmgwK4/BFahAf3djZ0KprECugu5xDZjIf/QKsWB oHRpScMQygCsTCOZCmUPYPESB7pUeiWc/kBOsQhoY6WF1t7HJyAPT+WuVFgPHUCtke2JPeJER7hMm 6cMdpH8g==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCZ-00000006IQJ-3NBC; Thu, 09 Jan 2025 14:07:59 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx6L-3Q4F; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 6/9] x86/kexec: Mark machine_kexec() with __nocfi Date: Thu, 9 Jan 2025 14:04:18 +0000 Message-ID: <20250109140757.2841269-7-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse A recent commit caused the relocate_kernel() function to be invoked through a function pointer, but it does not have CFI information. The resulting trap occurs after the IDT and GDT have been invalidated, leading to a triple-fault if CONFIG_CFI_CLANG is enabled. Using SYM_TYPED_FUNC_START() to provide the CFI information looks like it will require a prolonged battle with objtool. And is fairly pointless anyway, as the actual signature comes from a __kcfi_typeid_=E2=80=A6 symbol emitted from the C code based on the function prototype it thinks that relocate_kernel has, rendering the check somewhat tautological. The simple fix is just to mark machine_kexec() with __nocfi. Reported-by: Nathan Chancellor Suggested-by: Nathan Chancellor Fixes: eeebbde57113 ("x86/kexec: Invoke copy of relocate_kernel() instead of the original") Signed-off-by: David Woodhouse --- arch/x86/kernel/machine_kexec_64.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 9232ad1562c8..1440f792a86d 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -342,7 +342,7 @@ void machine_kexec_cleanup(struct kimage *image) * Do not allocate memory (or fail in any way) in machine_kexec(). * We are past the point of no return, committed to rebooting now. */ -void machine_kexec(struct kimage *image) +void __nocfi machine_kexec(struct kimage *image) { unsigned long (*relocate_kernel_ptr)(unsigned long indirection_page, unsigned long pa_control_page, --=20 2.47.0 From nobody Wed Dec 17 21:26:10 2025 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C057218ADA for ; Thu, 9 Jan 2025 14:08:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431715; cv=none; b=cQRJ+WIWfKqqkG7xA+aM7jDkxwrrdDliW2+5AK0uJyF0O1IEbB1ONCATksxrQbQF1FGLPGO4mAM84XZhZZKfv6HVG6OlbSZkuL4kgkkIShhjWkz5QotAKPeUyyWE791dZQn+q+HZnK6wpxSUpYIiIoZN/eRBUi+/vyXt0mefgus= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431715; c=relaxed/simple; bh=b6JswA7rfzdTMbDlCbvy9TcmyJgHcyKPKzbFliLK5Gs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aJjOUvG9JyPjpX8TSuhBDj6CsWEARomEB4omhWV0NJNyZTpAlYzTDZntz93YcoJeUdl/buI6xVYWZU0iCPsUsZL+Uh+85ScnuWMdBauytik2eByKxp9Nz8WprecRVoWTRUMc0MQRkCqsriL/Awq/kSsSMeQXnjk0S7wgKzgBqbw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=qh68UCXE; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="qh68UCXE" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=bFcppKCEUBjugJ7xwZiGVN+OuHVqmS8GOG0BIifw77s=; b=qh68UCXENO/RYJFF8kUFvYLW+Y 01DQVBz9CP2uynDSDCh7y+78zF7+fMfrmnOdrr9gMkpR2zAsy4p/Zfn/kaTkgM5WJmLL9yv0R2op7 T72cPT1au2Hz0HiQqTDp2ik0OQrpbYzTEoipTWQgvMP0n4oeQa3PwY7wC+y43mgPv37taurGILhj3 ZRNib6VCwnkDIEmkfd4MHTFHU9H4UNv6BOU3IfGAQAoI5c9blASIIvLfimYyCafBZOhHcLv/bYNfJ Gb2xdvlmK9CWpkOBTFSS49wPAvHdhnozIkve9D20xYbtOPC3qL7WqG/6j5Hq4gPBvcKdmRLffVgI6 a50uzDiA==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCa-00000009Zhg-3Cr9; Thu, 09 Jan 2025 14:08:01 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx6P-3fUv; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 7/9] kexec_core: Add and update comments regarding the KEXEC_JUMP flow Date: Thu, 9 Jan 2025 14:04:19 +0000 Message-ID: <20250109140757.2841269-8-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: "Rafael J. Wysocki" The KEXEC_JUMP flow is analogous to hibernation flows occurring before and after creating an image and before and after jumping from the restore kernel to the image one, which is why it uses the same device callbacks as those hibernation flows. Add comments explaining that to the code in question and update an existing comment in it which appears a bit out of context. No functional changes. Signed-off-by: Rafael J. Wysocki Signed-off-by: David Woodhouse --- kernel/kexec_core.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index c0caa14880c3..464ab059c58d 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -1001,6 +1001,12 @@ int kernel_kexec(void) =20 #ifdef CONFIG_KEXEC_JUMP if (kexec_image->preserve_context) { + /* + * This flow is analogous to hibernation flows that occur + * before creating an image and before jumping from the + * restore kernel to the image one, so it uses the same + * device callbacks as those two flows. + */ pm_prepare_console(); error =3D freeze_processes(); if (error) { @@ -1011,12 +1017,10 @@ int kernel_kexec(void) error =3D dpm_suspend_start(PMSG_FREEZE); if (error) goto Resume_console; - /* At this point, dpm_suspend_start() has been called, - * but *not* dpm_suspend_end(). We *must* call - * dpm_suspend_end() now. Otherwise, drivers for - * some devices (e.g. interrupt controllers) become - * desynchronized with the actual state of the - * hardware at resume time, and evil weirdness ensues. + /* + * dpm_suspend_end() must be called after dpm_suspend_start() + * to complete the transition, like in the hibernation flows + * mentioned above. */ error =3D dpm_suspend_end(PMSG_FREEZE); if (error) @@ -1052,6 +1056,13 @@ int kernel_kexec(void) =20 #ifdef CONFIG_KEXEC_JUMP if (kexec_image->preserve_context) { + /* + * This flow is analogous to hibernation flows that occur after + * creating an image and after the image hernel has got control + * back, and in case the devices have been reset or otherwise + * manipulated in the meantime, it uses the device callbacks + * used by the latter. + */ syscore_resume(); Enable_irqs: local_irq_enable(); --=20 2.47.0 From nobody Wed Dec 17 21:26:10 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8660B21C17E for ; Thu, 9 Jan 2025 14:08:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431719; cv=none; b=u5wsI6yO8eKjJWbaTPeEhZ0/hw4K8nD5hfI4u9c9dtfwgIcLeteyNanfxjeCFr/Lmgw3nMygxHLWc106LiWZS17iF44VR2mdunC2l+PtWaxvdyra/OABBaKoCdbg3vSiQDiCRbflqicAc03bnKDJDUaxZOlnpYeTNuJcWFxVC24= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431719; c=relaxed/simple; bh=42bLTThbrMk0UvpeRT4OhHC9vnfPgHml4XkxpkgpBhw=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jcJmFWFRo233y10InUo0ttsaz7+DlmKQowZ7cN3aSeqUbN6trmXBvYbtYff44HICyuDf+nN2l2YghyWdMnbzjQQtnD+Ej3+F7kkMzZdR71+VExxVKY51jRq7+mwLW8ODF+1kGO34jEKCznDRvIU+Y7oGN3pO4fQhVMN8hW7s+hs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=i9Jh/0QT; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="i9Jh/0QT" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=CgoaEkYIZX+dJ3COQN6/N0SHLTDEo7W5MwBROBJCJy8=; b=i9Jh/0QT/Ac0mjoxeiCq+bqKCM tdInaXsmQCUbQO/dA5zfAn4hyLKsMHi49QgNt9SsUsBJhKnFxEfEXlDcuaELkdeu9IXPT6Zz/XSOl RJlSOJOPfiN0Lul3Ay7LpweJRuCJSYq3lmEqFaiBouvLrZnD8aQlfJFSumItXZUmvP5BN2sLrwimW UMphoMTA7yu4R82OORrSb+LHC68gi7F5I2hmEeiT+GP8O8JmooJoy2YgAw/pLiLNsAzFamWvOrD8c ua/6X9kJNXIk0NiFToIFsmOEc43Ne22UbTxNRNSnnnjpfeK3Zpfz0N2H0u2fRpuIAMtpmoL9Ou8iR pYSaA3CA==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCZ-00000006IQN-3eVv; Thu, 09 Jan 2025 14:07:59 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx6V-3oum; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 8/9] x86/kexec: Cope with relocate_kernel() not being at the start of the page Date: Thu, 9 Jan 2025 14:04:20 +0000 Message-ID: <20250109140757.2841269-9-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse A few places in the kexec control code page make the assumption that the first instruction of relocate_kernel is at the very start of the page. To allow for Clang CFI information to be added to relocate_kernel(), as well as the general principle of removing unwarranted assumptions, fix them to use the external __relocate_kernel_start symbol that the linker adds. This means using a separate addq and subq for calculating offsets, as the assembler can no longer calculate the delta directly for itself and relocations aren't that versatile. But we can at least use values relative to a local label to avoid absolute relocations. Turn the jump from relocate_kernel() to identity_mapped() into a real indirect 'jmp *%rsi' too, while touching it. There was no real reason for it to be a push+ret in the first place, and adding Clang CFI info will also give objtool enough visibility to start complaining 'return with modified stack frame' about it. Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 14ed40b7f322..af2cd06ff318 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -95,11 +95,10 @@ SYM_CODE_START_NOALIGN(relocate_kernel) lea PAGE_SIZE(%rsi), %rsp =20 /* jump to identity mapped page */ - addq $(identity_mapped - relocate_kernel), %rsi - pushq %rsi - ANNOTATE_UNRET_SAFE - ret - int3 +0: addq $identity_mapped - 0b, %rsi + subq $__relocate_kernel_start - 0b, %rsi + ANNOTATE_RETPOLINE_SAFE + jmp *%rsi SYM_CODE_END(relocate_kernel) =20 SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) @@ -219,16 +218,21 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) =20 /* get the re-entry point of the peer system */ popq %rbp - leaq relocate_kernel(%rip), %r8 movq kexec_pa_swap_page(%rip), %r10 movq pa_backup_pages_map(%rip), %rdi movq kexec_pa_table_page(%rip), %rax movq %rax, %cr3 + + /* Find start (and end) of this physical mapping of control page */ + leaq (%rip), %r8 + ANNOTATE_NOENDBR + andq $PAGE_MASK, %r8 lea PAGE_SIZE(%r8), %rsp movl $1, %r11d /* Ensure preserve_context flag is set */ call swap_pages movq kexec_va_control_page(%rip), %rax - addq $(virtual_mapped - relocate_kernel), %rax +0: addq $virtual_mapped - 0b, %rax + subq $__relocate_kernel_start - 0b, %rax pushq %rax ANNOTATE_UNRET_SAFE ret --=20 2.47.0 From nobody Wed Dec 17 21:26:10 2025 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8FC5219E8F for ; Thu, 9 Jan 2025 14:08:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431715; cv=none; b=u8AhedY06y5u6AZjiVqQf8u58iQpyy5S94OiiqfsGkktrr4PO76lDf7sa0lA74JukjBpZcp5LKd07TIOuVET1+tsApPIim692P1coVf8SNyLXquBKXeJ7YORbJsdo4n78T+T+MiD75PsBziCIycMFuarqNIFxRVYSrp9zZj+HKY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736431715; c=relaxed/simple; bh=66bK3hS4MjiVy30c7KVE9tQcd7zIjJHoi4zEicwN8Po=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TB1bxbKYYEPetEi1gLP1RIgBmCVHQfqN7B81rqXE5VD50jKZNxKBVyNfH06+SzjgVO/TYS8D9kg0vSNDN7KLnQD0BPEKzKNHr/HO9uv3U89ZM0nZLXUOJAuafly7o4IyjYlQAyHXcaMROB0X5x4+sYrAHrQmjZq16meguxug6yU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=LheePo9B; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="LheePo9B" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=2QdnXzG6oBq2pMrwUNHKRzdcvGBvqjVeazTvAuBzw50=; b=LheePo9Br2Wd1F2sVHVFh9KY5F AH9CN9GC2iLeD4BI0v/rNS3oE5X704fVlk15HdgU/3Y8DpHdvAUa+6tmELwwau8hzBnWWwpHkFrSG z22xJb3HXUDtfOVR68jhLgFrKG8+d+YKOa+anod3bI1RgbV94rQAJxe7aE8xzso1tPFWZ8xy35mcu s9vdPpYmmUacgWxsrJimVlSx/WX0kR03pir/VKz0H3OiAUET1cvIb9PtYjfWxAd88U4KmAvpv5XF5 5c2c6VNLPhetGEogt+gyiW63xudikxjBS6LVAQKZD6VQIINM2p/lf87IogYyKocHu+1aWO1Uqq2AS GaJEjiXw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCa-00000009Zhh-3DJa; Thu, 09 Jan 2025 14:08:01 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tVtCY-0000000Bx6Y-41XW; Thu, 09 Jan 2025 14:07:58 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A . Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH v2 9/9] x86/kexec: Use typedef for relocate_kernel_fn function prototype Date: Thu, 9 Jan 2025 14:04:21 +0000 Message-ID: <20250109140757.2841269-10-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250109140757.2841269-1-dwmw2@infradead.org> References: <20250109140757.2841269-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Both i386 and x86_64 now copy the relocate_kernel function into the control page and execute it from there, using an open-coded function pointer. Use a typedef for it instead. Signed-off-by: David Woodhouse --- arch/x86/include/asm/kexec.h | 26 +++++++++++++------------- arch/x86/kernel/machine_kexec_32.c | 7 +------ arch/x86/kernel/machine_kexec_64.c | 6 +----- 3 files changed, 15 insertions(+), 24 deletions(-) diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h index 48e4f44f794f..8ad187462b68 100644 --- a/arch/x86/include/asm/kexec.h +++ b/arch/x86/include/asm/kexec.h @@ -111,21 +111,21 @@ static inline void crash_setup_regs(struct pt_regs *n= ewregs, } =20 #ifdef CONFIG_X86_32 -asmlinkage unsigned long -relocate_kernel(unsigned long indirection_page, - unsigned long control_page, - unsigned long start_address, - unsigned int has_pae, - unsigned int preserve_context); +typedef asmlinkage unsigned long +relocate_kernel_fn(unsigned long indirection_page, + unsigned long control_page, + unsigned long start_address, + unsigned int has_pae, + unsigned int preserve_context); #else -unsigned long -relocate_kernel(unsigned long indirection_page, - unsigned long pa_control_page, - unsigned long start_address, - unsigned int preserve_context, - unsigned int host_mem_enc_active); +typedef unsigned long +relocate_kernel_fn(unsigned long indirection_page, + unsigned long pa_control_page, + unsigned long start_address, + unsigned int preserve_context, + unsigned int host_mem_enc_active); #endif - +extern relocate_kernel_fn relocate_kernel; #define ARCH_HAS_KIMAGE_ARCH =20 #ifdef CONFIG_X86_32 diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_k= exec_32.c index 1b373d79cedc..80265162aeff 100644 --- a/arch/x86/kernel/machine_kexec_32.c +++ b/arch/x86/kernel/machine_kexec_32.c @@ -160,15 +160,10 @@ void machine_kexec_cleanup(struct kimage *image) */ void machine_kexec(struct kimage *image) { + relocate_kernel_fn *relocate_kernel_ptr; unsigned long page_list[PAGES_NR]; void *control_page; int save_ftrace_enabled; - asmlinkage unsigned long - (*relocate_kernel_ptr)(unsigned long indirection_page, - unsigned long control_page, - unsigned long start_address, - unsigned int has_pae, - unsigned int preserve_context); =20 #ifdef CONFIG_KEXEC_JUMP if (image->preserve_context) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 1440f792a86d..dd75a51463a2 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -344,12 +344,8 @@ void machine_kexec_cleanup(struct kimage *image) */ void __nocfi machine_kexec(struct kimage *image) { - unsigned long (*relocate_kernel_ptr)(unsigned long indirection_page, - unsigned long pa_control_page, - unsigned long start_address, - unsigned int preserve_context, - unsigned int host_mem_enc_active); unsigned long reloc_start =3D (unsigned long)__relocate_kernel_start; + relocate_kernel_fn *relocate_kernel_ptr; unsigned int host_mem_enc_active; int save_ftrace_enabled; void *control_page; --=20 2.47.0