From nobody Wed Dec 17 21:39:27 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 77BDA1DACBF for ; Mon, 16 Dec 2024 23:37:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392259; cv=none; b=V0pSr2BA38ww/Frz+FcRnqU8P02uphQ90ifDJ/1RBbzVmnqjrYyiMWTlb0c9nuEebKsk/TzdtR5vO+3EeQYaHrliS8hCzUuBjKs0S9SX7g0eif4goWV5wBlb+pQ6BkdNopiX1dziN8zYMvpMmeeEXxjOfyTPfGXl0aj/PNRvmKg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392259; c=relaxed/simple; bh=zkbYTmuZfQnBujK5vcetFookFFHLMepfdATcSMt8Pnk=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=q3WTR5o6RBD1rNgoKX3t0lJ8U6Rdh1iCsrzgmDf/LssPd9cMktBNwIkyEaK3+Jb3JnD6+vuUe/inNH2jMPgPkaaX29AU+NmT7RffY/EmsOU+yHHgUcBqEuuGEUeAtinfrBIXqs/D2l7nF9WjsKn0rrTrAz3Es32IUwZRpm/XASg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=qvaNRGwU; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="qvaNRGwU" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To: From:Reply-To:Cc:Content-ID:Content-Description; bh=siK05iL8Sl6AWnRvb8/748kKl/nChwb5EZxS4049r5c=; b=qvaNRGwUFC5FsBbeIs6VVrSnBe WbdfBwGzFPGSuxtAJlPzlK3kEAsmjb3fbEYVnEQPt0u2zYf9NRL/ttY09zBY2hMIHU4u3m6aGDKII jkkzC7rJlMsFboQaNYUbNnRpJmg5RWJZKBwCxmSqjXkpjFmLcDTAAWAV4JllqeMO6JJCh7ASYUMtE ezZGQz/5KZZeKbL2Rdr8m8AsmR4b/oAng2yu1xaUfCKil0/pWZ1qi/Dm1yb5IPeiiJP+eeiesmw82 /xpgrD+l9ZN+m0Y3mvxYmmArGtoVytSqpPNVbulOqP+Y0OpE5O98zHpUfoJbXBZBWYgJc78/fyGGn q9pBrmJA==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-000000034nU-1cjU; Mon, 16 Dec 2024 23:37:06 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeW3-00b8; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 1/9] x86/kexec: Disable global pages before writing to control page Date: Mon, 16 Dec 2024 23:24:08 +0000 Message-ID: <20241216233704.3208607-2-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse The kernel switches to a new set of page tables during kexec. The global mappings (_PAGE_GLOBAL=3D=3D1) can remain in the TLB after this switch. This is generally not a problem because the new page tables use a different portion of the virtual address space than the normal kernel mappings. The critical exception to that generalisation (and the only mapping which isn't an identity mapping) is the kexec control page itself =E2=80=94 which was ROX in the original kernel mapping, but should be RWX in the new page tables. If there is a global TLB entry for that in its prior read-only state, it definitely needs to be flushed before attempting to write through that virtual mapping. It would be possible to just avoid writing to the virtual address of the page and defer all writes until they can be done through the identity mapping. But there's no good reason to keep the old TLB entries around, as they can cause nothing but trouble. Clear the PGE bit in %cr4 early, before storing data in the control page. Fixes: 5a82223e0743 ("x86/kexec: Mark relocate_kernel page as ROX instead o= f RWX") Co-authored-by: Dave Hansen Reported-by: Nathan Chancellor Reported-by: "Ning, Hongyu" Closes: https://bugzilla.kernel.org/show_bug.cgi?id=3D219592 Signed-off-by: Dave Hansen Signed-off-by: David Woodhouse Tested-by: Nathan Chancellor Tested-by: "Ning, Hongyu" --- arch/x86/kernel/relocate_kernel_64.S | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 8bc86a1e056a..9bd601dd8659 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -70,14 +70,20 @@ SYM_CODE_START_NOALIGN(relocate_kernel) movq kexec_pa_table_page(%rip), %r9 movq %r9, %cr3 =20 + /* Leave CR4 in %r13 to enable the right paging mode later. */ + movq %cr4, %r13 + + /* Disable global pages immediately to ensure this mapping is RWX */ + movq %r13, %r12 + andq $~(X86_CR4_PGE), %r12 + movq %r12, %cr4 + /* Save %rsp and CRs. */ + movq %r13, saved_cr4(%rip) movq %rsp, saved_rsp(%rip) movq %rax, saved_cr3(%rip) movq %cr0, %rax movq %rax, saved_cr0(%rip) - /* Leave CR4 in %r13 to enable the right paging mode later. */ - movq %cr4, %r13 - movq %r13, saved_cr4(%rip) =20 /* save indirection list for jumping back */ movq %rdi, pa_backup_pages_map(%rip) base-commit: 35aafa1d41cee0d3d50164561bca34befc1d9ce3 --=20 2.47.0 From nobody Wed Dec 17 21:39:27 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E314F1D5AB6 for ; Mon, 16 Dec 2024 23:37:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392259; cv=none; b=QC8Datejon/4O+9awDw7qi4koVaxPP1fm4MdjuTpI+U71Co9VBt4mmWAiMwOc8ahiCH5kUJRHUy2/DvDJhM38k+AM69b+xEba9zKk9gznuhCye/GEO0NJdDH63hmoJksdEvwSzZqb4oQ0SQHPpD2nHnjf5yi2n2OGjCssiLUZeU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392259; c=relaxed/simple; bh=K1W4o2sZdF5lVYRGfeks7NhzGm/Jy2da4/Mpm9653ds=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eTtNlGioBLNjq9aBi8FRvXr4Io8nuwQFD6GRGvsY2LeZ5LbstzAgoARP9KHph5XBovbzVtT4ICYlCIqhxs/RW2IdEGcu5SN2ixT8kGZEtUhLskttip15qxCAZDcLDZCugVbob6niByTFn7Vc5e980RCqrGJFoCWoS7U1uTJP3q8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=vyMpdeUQ; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="vyMpdeUQ" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=CrhFZ3L2x07bub/40Vp59il27O/SizC8zFhP7HAHpi8=; b=vyMpdeUQOKLY34lxdul443TXiG SsMDdVpmb/GQZVDcjd8ZpZzYRiiaZy/H3uHDFcO/N+ZO9pYSVzqUDUMkCdfPv5vaADFjzdpBHrD4T J69AC1a41R3X6VWqPzkUEURqV6zbgPEIR0h5utxro0iKiHRxJ02vmxtSkrvm5J8MnYFU2vgflSgN7 Oz5wxizXbR0DgaEHbku7Z5p0Rvt0ltgt+Wg5vXD2v1rJZ2MHUonMgtiQTwVdB40gQAAuhsu7NUNPA fRjhmyBKf9F3PrTqCT6MtUvAKXyrCxKpjfMQJgWMBaCbyH9aX/HZ+TA7uL1dENbB/lntQxAHroFm0 eRvnKMrg==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-000000034nW-1rMG; Mon, 16 Dec 2024 23:37:06 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeW6-0Alp; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 2/9] x86/kexec: Ensure preserve_context flag is set on return to kernel Date: Mon, 16 Dec 2024 23:24:09 +0000 Message-ID: <20241216233704.3208607-3-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse The swap_pages function will only actually *swap*, as its name implies, if the preserve_context flag in the %r11 register is non-zero. On the way back from a ::preserve_context kexec, ensure that the %r11 register is non-zero so that the pages get swapped back. Fixes: 9e5683e2d0b5 ("x86/kexec: Only swap pages for ::preserve_context mod= e") Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 9bd601dd8659..1a52e4339c1d 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -220,6 +220,7 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) movq kexec_pa_table_page(%rip), %rax movq %rax, %cr3 lea PAGE_SIZE(%r8), %rsp + movq $1, %r11 /* Ensure preserve_context flag is set */ call swap_pages movq kexec_va_control_page(%rip), %rax addq $(virtual_mapped - relocate_kernel), %rax --=20 2.47.0 From nobody Wed Dec 17 21:39:27 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 521681DACAF for ; Mon, 16 Dec 2024 23:37:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392260; cv=none; b=gxPMqb7TU4eRm83AKdxz1OryDurt9Li3PKFZXWzKQVAWbFAAIj3GEYLk9V0/Jqqa1zoNbjWjx2TWOfUSsvuovzaLKkK0nbWKT5Ox+CSngFRxBD0Vz5YQ8/fwY8JC+sLZLMbQ3FL4s9qhPR+8qOcBVUxWnrFFQC/37qhvj5dmWZk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392260; c=relaxed/simple; bh=J15PTcLBO+YxoVKNHZq8FmvDQXS8n+Mg0vCpOx+RUxs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qTYWC2G87XmgKUrUjiqhebaggkti3elkZGKEKbTtj4REg1xeUNpJqpuAhcKfpFvXWZEtLb3yDock2AODhD1I8tspK4ndNZEP+vDV9O3hRmKuGOxzMW83eEEueBz+aH2sngPZ0IRQ4nd7ktybifrNeoouQYfAWGy6SKpp0+VS2wY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=gY/TwmhM; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="gY/TwmhM" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=zzCHIqIiQJDgySmzgmnKF8ZlcgBrOOsvhFm4kYphShs=; b=gY/TwmhMVrUX4ruTov+kQpAeXq vUof0P+v8tmNgZzieQgAPFNfKsdCHZ/fwo462vl2AG6mOxorwVgsneYQhjXjghSswbLIN6yyq2tG4 MLM5EBtf9WXI43ph0N7ap72TpygIWsaBhtBU+fFNKjt/VY3Rpmjp1TLnWE0uLjvSh15dTjVNS0Zqq xsxKda+QdQW9Z/ITvj5zEnY921zH0MMyZEtqdA6/StFGUzAoRPMx0NaN6IQY4fANl3FpmUJQlxLGP TX9n2KDYVOHgvF5DsVSzF60WeTB/u/TkkLqwmClvBnb+T2Wv3wqAtEWxJKwwFM/tQCqlMGEFmxN56 4l17Grwg==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-000000034nX-25us; Mon, 16 Dec 2024 23:37:06 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeW9-0Qg3; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 3/9] x86/kexec: Use correct swap page in swap_pages function Date: Mon, 16 Dec 2024 23:24:10 +0000 Message-ID: <20241216233704.3208607-4-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse The swap_pages function expects the swap page to be in %r10, but there was no documentation to that effect. Once upon a time the setup code used to load its value from a kernel virtual address and save it to an address which is accessible in the identity-mapped page tables, and *happened* to use %r10 to do so, with no comment that it was left there on *purpose* instead of just being a scratch register. Once that was no longer necessary, %r10 just holds whatever the kernel happened to leave in it. Now that the original value passed by the kernel is accessible via %rip-relative addressing, load directly from there instead of using %r10 for it. But document the other parameters that the swap_pages function *does* expect in registers. Fixes: b3adabae8a96 ("x86/kexec: Drop page_list argument from relocate_kern= el()") Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 1a52e4339c1d..0d6fce1e0a32 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -264,6 +264,10 @@ SYM_CODE_END(virtual_mapped) /* Do the copies */ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) UNWIND_HINT_END_OF_STACK + /* + * %rdi indirection page + * %r11 preserve_context + */ movq %rdi, %rcx /* Put the indirection_page in %rcx */ xorl %edi, %edi xorl %esi, %esi @@ -302,7 +306,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) jz .Lnoswap =20 /* copy source page to swap page */ - movq %r10, %rdi + movq kexec_pa_swap_page(%rip), %rdi movl $512, %ecx rep ; movsq =20 @@ -314,7 +318,7 @@ SYM_CODE_START_LOCAL_NOALIGN(swap_pages) =20 /* copy swap page to destination page */ movq %rdx, %rdi - movq %r10, %rsi + movq kexec_pa_swap_page(%rip), %rsi .Lnoswap: movl $512, %ecx rep ; movsq --=20 2.47.0 From nobody Wed Dec 17 21:39:27 2025 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CAB91D9592 for ; Mon, 16 Dec 2024 23:37:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392263; cv=none; b=Geowl0flhpq08bPYh+h4Efy3fUAovZjLY4wovshqeDC5ZZ9NZ64FiXUCE4SsL8b+5yUEuhHcMTDO6PGI2BZiQIqUOoa8IWek8+aqulG50iFRRHYpx3/K65KL8F2+kJQKRNJfqrjMBqSW8gTEtmMegEznHpF1cJq6HXieE/CMB1U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392263; c=relaxed/simple; bh=aaxElBxQe7VbZoqviico/VjmU+O3c6qs4deC5d6uovc=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QnBcgueod4LdvzSD3eq9N69YADIhpUJsZFHRA4+38thwwZ+ChAMANQA2r1NhLQokHIKA0xYehcPvYxQZn8L23JYYYfm1QrmQDHY/Xsm225fL0lhyterireCA5UifGJQhqVf8Ogkj72WrZp4Fg2uTDmKAQiCFJMwwwMWiwRv2kno= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=HZk53P+Q; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="HZk53P+Q" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=3ORzIP291/TaZUjjutyd5sQ2ZIP1eUhgRH/ydhloMEg=; b=HZk53P+QNu/mMYp9+pOCm1RQGW 0oTqkJukX/mf5SQigAHRX4z3odyjtFe+o7CnSBQUZsYHhuQsseTwukoJKAVW0FmKskEc0IPfA99yP 0+LVkk5hTdxxiK9TrUrtWR3ITPW3DEHSCHrkydpSN8T0Aq8+i/hs2fZN/bnSOTWR8vgDj8f92Yuuh 28SUD2RjFX537iXog2goRxBZ9MGlzB16AfxacU5J1txpykakwRSk8VtGBe76t+VXzccKo8sWbpdm+ 8YWQ4b4om5dIXdNqT33btYU5oQyudc9FcbE+/phWOWGa4uV7x/G1qh3Dxv/od73MLQlNVZpODG2KO F4OQdngA==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeB-00000004xiO-2A3d; Mon, 16 Dec 2024 23:37:08 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeWF-0frC; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 4/9] x86/kexec: Fix stack and handling of re-entry point for ::preserve_context Date: Mon, 16 Dec 2024 23:24:11 +0000 Message-ID: <20241216233704.3208607-5-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse A ::preserve_context kimage can be invoked more than once, and the entry point can be different every time. When the callee returns to the kernel, it leaves the address of its entry point for next time on the stack. That being the case, one might reasonably assume that the caller would allocate space for it on the stack fram before actually performing the 'call' into the callee. Apparently not, though. Ever since the kjump code was first added in 2009, it has set up a *new* stack at the top of the swap_page scratch page, then just performed the 'call' without allocating any space for the re-entry address to be returned. It then reads the re-entry point for next time from 0(%rsp) which is actually the first qword of the page *after* the swap page, which might not exist at all! And if the callee has written to that, then it will have corrupted memory it doesn't own. Correct this by pushing the entry point of the callee onto the stack before calling it. The callee may then adjust it, or not, as it sees fit, and subsequent invocations should work correctly either way. Remove a stray push of zero to the *relocate_kernel* stack, which may have been intended for this purpose, but which was actually just noise. Also, loading the stack for the callee relied on the address of the swap page being in %r10 without ever documenting that fact. Recent code changes made that no longer true, so load it directly from the local kexec_pa_swap_page variable instead. Fixes: b3adabae8a96 ("x86/kexec: Drop page_list argument from relocate_kern= el()") Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 0d6fce1e0a32..b680f24896b8 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -113,8 +113,6 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) * %r13 original CR4 when relocate_kernel() was invoked */ =20 - /* set return address to 0 if not preserving context */ - pushq $0 /* store the start address on the stack */ pushq %rdx =20 @@ -208,12 +206,19 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) =20 .Lrelocate: popq %rdx + + /* Use the swap page for the callee's stack */ + movq kexec_pa_swap_page(%rip), %r10 leaq PAGE_SIZE(%r10), %rsp + + /* push the existing entry point onto the callee's stack */ + pushq %rdx + ANNOTATE_RETPOLINE_SAFE call *%rdx =20 /* get the re-entry point of the peer system */ - movq 0(%rsp), %rbp + popq %rbp leaq relocate_kernel(%rip), %r8 movq kexec_pa_swap_page(%rip), %r10 movq pa_backup_pages_map(%rip), %rdi @@ -247,6 +252,7 @@ SYM_CODE_START_LOCAL_NOALIGN(virtual_mapped) lgdt saved_context_gdt_desc(%rax) #endif =20 + /* relocate_kernel() returns the re-entry point for next time */ movq %rbp, %rax =20 popf --=20 2.47.0 From nobody Wed Dec 17 21:39:27 2025 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB6561DE88A for ; Mon, 16 Dec 2024 23:37:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392265; cv=none; b=XatNe7N4OdrrdktyBKQMrsphFxkZHQ34ONaf1hETE02DhSj4mCpzy7ertUE1BtR9XcTu6R2SnihPwgA4N+eVNemIFZzuau/WD0Hqdp3IYykIfjBXUijNdOQ1fV7wmUApTQmUYS+HRIggwQQt6jJ/8XKD5quxqdbyh1u2t9OdL4E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392265; c=relaxed/simple; bh=VLUVdTyeV0P1JOB4I+p27vLpOo5Zr5seDRRWbSetvcY=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oqXMgx1z4CLZ26/buTnea47ccHf1ZSuGA3YYv6sEPUG1GnHKJwPcdUyZ4E/kzD+xODjhIS30EpWvmVi/5j1toDj1UeAOaTJNI96XUU0AGyNREWk5CnjxMHPK7Rt7r9DkWb26w5+o7dxS6x1LKl1djNGbbEfq5aIZ53/RC7q5i6E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=o6LkWnyX; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="o6LkWnyX" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=tC7qQ0TyIBftkKD3qsN4aGjl/EaVH0eOIxBQvILwXpc=; b=o6LkWnyX2Y1FY1IHBX8dWfzjEC Oargtl0xKp3FA9E7/BFmUs5BT4/GpkY4on/TIsvS/USNH3AoUt3RkeogGdJYl6O/l7rtdSKyDJDFd S0cSBYTp6INO+7XPmjwY4wJ5U7YAz11PGTWhYBITeopPfKbNvIw3u9bmsgAGH0stZ6cQ9vBYrem6q MsDSgyJ33S6VURHETBwQmcoNm6NGgX49KAnWOgVLYVXKfQ5djQw5eJuZcOkgAllFo2y1SzZVEV1hw g7Ph+DghE3+WIovQJ4XfLU/KbSAYj8SCD824VOVmna6FREWru7cy3RJU8d/z9vz5GmyXvoVWZthXC AuVzCnag==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeB-00000004xiP-29zs; Mon, 16 Dec 2024 23:37:08 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeWI-0qm5; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 5/9] x86/kexec: Fix location of relocate_kernel with -ffunction-sections Date: Mon, 16 Dec 2024 23:24:12 +0000 Message-ID: <20241216233704.3208607-6-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: Nathan Chancellor After commit cb33ff9e063c ("x86/kexec: Move relocate_kernel to kernel .data section"), kernels configured with an option that uses -ffunction-sections, such as CONFIG_LTO_CLANG, crash when kexecing because the value of relocate_kernel does not match the value of __relocate_kernel_start so incorrect code gets copied via machine_kexec_prepare(). $ llvm-nm good-vmlinux &| rg relocate_kernel ffffffff83280d41 T __relocate_kernel_end ffffffff83280b00 T __relocate_kernel_start ffffffff83280b00 T relocate_kernel $ llvm-nm bad-vmlinux &| rg relocate_kernel ffffffff83266100 D __relocate_kernel_end ffffffff83266100 D __relocate_kernel_start ffffffff8120b0d8 T relocate_kernel When -ffunction-sections is enabled, TEXT_MAIN matches on '.text.[0-9a-zA-Z_]*' to coalesce the function specific functions back into .text during link time after they have been optimized. Due to the placement of TEXT_TEXT before KEXEC_RELOCATE_KERNEL in the x86 linker script, the .text.relocate_kernel section ends up in .text instead of .data. Use a second dot in the relocate_kernel section name to avoid matching on TEXT_MAIN, which matches a similar situation that happened in commit 79cd2a11224e ("x86/retpoline,kprobes: Fix position of thunk sections with CONFIG_LTO_CLANG"), which allows kexec to function properly. While .data.relocate_kernel still ends up in the .data section via DATA_MAIN -> DATA_DATA, ensure it is located with the .text.relocate_kernel section as intended by performing the same transformation. Fixes: cb33ff9e063c ("x86/kexec: Move relocate_kernel to kernel .data secti= on") Fixes: 8dbec5c77bc3 ("x86/kexec: Add data section to relocate_kernel") Signed-off-by: Nathan Chancellor Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 6 +++--- arch/x86/kernel/vmlinux.lds.S | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index b680f24896b8..1996cea909ff 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -23,11 +23,11 @@ #define PAGE_ATTR (_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED | _PAGE_DIRTY) =20 /* - * The .text.relocate_kernel and .data.relocate_kernel sections are copied + * The .text..relocate_kernel and .data..relocate_kernel sections are copi= ed * into the control page, and the remainder of the page is used as the sta= ck. */ =20 - .section .data.relocate_kernel,"a"; + .section .data..relocate_kernel,"a"; /* Minimal CPU state */ SYM_DATA_LOCAL(saved_rsp, .quad 0) SYM_DATA_LOCAL(saved_cr0, .quad 0) @@ -39,7 +39,7 @@ SYM_DATA(kexec_pa_table_page, .quad 0) SYM_DATA(kexec_pa_swap_page, .quad 0) SYM_DATA_LOCAL(pa_backup_pages_map, .quad 0) =20 - .section .text.relocate_kernel,"ax"; + .section .text..relocate_kernel,"ax"; .code64 SYM_CODE_START_NOALIGN(relocate_kernel) UNWIND_HINT_END_OF_STACK diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 0c893997f023..63ff60a11be5 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -100,8 +100,8 @@ const_pcpu_hot =3D pcpu_hot; #define KEXEC_RELOCATE_KERNEL \ . =3D ALIGN(0x100); \ __relocate_kernel_start =3D .; \ - *(.text.relocate_kernel); \ - *(.data.relocate_kernel); \ + *(.text..relocate_kernel); \ + *(.data..relocate_kernel); \ __relocate_kernel_end =3D .; =20 ASSERT(__relocate_kernel_end - __relocate_kernel_start <=3D KEXEC_CONTROL_= CODE_MAX_SIZE, --=20 2.47.0 From nobody Wed Dec 17 21:39:27 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DAC77139CEF for ; Mon, 16 Dec 2024 23:37:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392258; cv=none; b=id9qc8UaFzVryJ0DSSOzVLAHmJpR4uKcIJiHyboge68J+OEV2wxg0fKG1EzG+4IbMf/gNIzzZHR9UtIgGNDC0c+p/jSScZ1YyGZbVShLRf7lFcIaRoqAjxug6kvG+cUpbdJ8lVC19WOj8AlXlGFBEcETMm5wzEcnK/KX+onuEVs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392258; c=relaxed/simple; bh=pD/iM6D1Uc9Ku+aH6t9jBfBnAst+5WgG7HrJPvYhw58=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GfDRTgF+9Uu1SbFaiO7FgQIP7u8TGKJK9SQFXmMfSEdE+ZfLb0CTiacCFNMyfG7sJyyvldyHDivDm+aOIMX3qNq/97odQXkMl4/I4eXp6qtplgQvOjONmBV25we81/8CRuuhGvcXMNGxcfCEz1WFT/MgZT5688h16l5RH68vjo4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=m94t7MaE; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="m94t7MaE" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To: From:Reply-To:Cc:Content-ID:Content-Description; bh=KF9I8gMwuVGDi5vu6drdRG3X3dHP+mrVfyO8mKUOGvk=; b=m94t7MaEEu9OBNPTDHoiuf70Fq Tp+6Tautcgn9+y4v+iOpPpRA7dlOW5BExNEZlyaWF6c5MembvO9E+FFAn0uqXpg5K1RzSXcq4jav2 0bPZNV6y+IFYVMhUtDQZqYbSCItpaxpTV1ihMtfL8DxE6H6VkEvxzqAqu4H8t5WnoBViQ3X5Z1Jra KCkt6QMGdR3bdWpiBS2RER03GULc8H1xzIA+fy/pEXdzwv5vxGdcmVMSrO6KJSnfRCNFZl5s+xqv7 W9sQtfqIFJO3Yz2pT5PTiGNtw22ueBKpuEkj9w2CSSaof8josM9C5BZT6ffocA8+uiijHgK8XXaqU YCQ/yO3w==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-000000034nY-2fG0; Mon, 16 Dec 2024 23:37:07 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeWN-17Kz; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 6/9] x86/kexec: Mark machine_kexec() with __nocfi Date: Mon, 16 Dec 2024 23:24:13 +0000 Message-ID: <20241216233704.3208607-7-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse A recent commit caused the relocate_kernel() function to be invoked through a function pointer, but it does not have CFI information. The resulting trap occurs after the IDT and GDT have been invalidated, leading to a triple-fault if CONFIG_CFI_CLANG is enabled. Using SYM_TYPED_FUNC_START() to provide the CFI information looks like it will require a prolonged battle with objtool. And is fairly pointless anyway, as the actual signature comes from a __kcfi_typeid_=E2=80=A6 symbol emitted from the C code based on the function prototype it thinks that relocate_kernel has, rendering the check somewhat tautological. The simple fix is just to mark machine_kexec() with __nocfi. Reported-by: Nathan Chancellor Suggested-by: Nathan Chancellor Fixes: eeebbde57113 ("x86/kexec: Invoke copy of relocate_kernel() instead of the original") Signed-off-by: David Woodhouse --- arch/x86/kernel/machine_kexec_64.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 9232ad1562c8..1440f792a86d 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -342,7 +342,7 @@ void machine_kexec_cleanup(struct kimage *image) * Do not allocate memory (or fail in any way) in machine_kexec(). * We are past the point of no return, committed to rebooting now. */ -void machine_kexec(struct kimage *image) +void __nocfi machine_kexec(struct kimage *image) { unsigned long (*relocate_kernel_ptr)(unsigned long indirection_page, unsigned long pa_control_page, --=20 2.47.0 From nobody Wed Dec 17 21:39:27 2025 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 06BD21DACBB for ; Mon, 16 Dec 2024 23:37:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392258; cv=none; b=eKhFvkyo+znz2X1voh6b6asdWY00oEDejKspo3j1hupCBOHhf8j1T5Tjn4QBG5MwJG98zUZYoqYjMgulFq8KJ55eGsyY+6Hrc+kd4XGUsu0jcQcFr/plEfV9S5KA7FcJsg+25TPn2vPL4TJKxbBQ23ahDPPBFEA1iAftkNRL9e4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392258; c=relaxed/simple; bh=h1wOv8MeWSS5bfN/J/5w9irQt2gV6YHa6Emr/0th+fI=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XkZJ89+eW9aInK0xQ2HN+ztuviWRTsj5fdU3ntW0xCpr7GKzT3Q5MMUNTCRjhrFJOS76sma/BdoTm+FmTOYEEdlUmRwv0rPXrA/oY9Ifs6R3xeG8ZQLFuMURJArEKCybDIAepN+9olWycfNFU4fijfpHZ8dmM3wDb909K5AW6LI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=Zx7gwPn0; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="Zx7gwPn0" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=xrUqJHk+DC7MJiWSLIDNfBawd31aIE7V5wzrcp1W6yM=; b=Zx7gwPn0rmONbPxIOvq+ePXTKW Zk15pJZDZDIkZLHFUp+nojiZHr5XeijgkL8l3gO/TpUea3CrhEhTjETXv6Y4iJY/jLacu/A1U5SEU UEVnOqaYEe6+B+OU8nCinx2QzHl/Vy0g5vpoMhcqyuNdibs++Cyh8pRrpIsbJX8Y3GpTO2XiWAozH 7539NDgNLojJl0KsYBe+6LAOwmS1NfJ3wguZtVUXGGlisG+9LJ1X+ndafT77hAKn5b3vsjqvv/Dv7 POPmXVwbmLF0IRf5vJKX/AIlVT86HwqM2md1pmSvQinWdKVQxuujVZUJdM1MHsHWoITKTDtxmzCVu nxh12/jQ==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-000000034na-2gcj; Mon, 16 Dec 2024 23:37:07 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeWQ-1HY4; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 7/9] kexec_core: Add and update comments regarding the KEXEC_JUMP flow Date: Mon, 16 Dec 2024 23:24:14 +0000 Message-ID: <20241216233704.3208607-8-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: "Rafael J. Wysocki" The KEXEC_JUMP flow is analogous to hibernation flows occurring before and after creating an image and before and after jumping from the restore kernel to the image one, which is why it uses the same device callbacks as those hibernation flows. Add comments explaining that to the code in question and update an existing comment in it which appears a bit out of context. No functional changes. Signed-off-by: Rafael J. Wysocki Signed-off-by: David Woodhouse --- kernel/kexec_core.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index c0caa14880c3..7cf8437e0f38 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -1001,6 +1001,12 @@ int kernel_kexec(void) =20 #ifdef CONFIG_KEXEC_JUMP if (kexec_image->preserve_context) { + /* + * This flow is analogous to hibernation flows that occur before + * creating an image and before jumping from the restore kernel + * to the image one, so it uses the same device device callbacks + * as those two flows. + */ pm_prepare_console(); error =3D freeze_processes(); if (error) { @@ -1011,12 +1017,10 @@ int kernel_kexec(void) error =3D dpm_suspend_start(PMSG_FREEZE); if (error) goto Resume_console; - /* At this point, dpm_suspend_start() has been called, - * but *not* dpm_suspend_end(). We *must* call - * dpm_suspend_end() now. Otherwise, drivers for - * some devices (e.g. interrupt controllers) become - * desynchronized with the actual state of the - * hardware at resume time, and evil weirdness ensues. + /* + * dpm_suspend_end() must be called after dpm_suspend_start() + * to complete the transition, like in the hibernation flows + * mentioned above. */ error =3D dpm_suspend_end(PMSG_FREEZE); if (error) @@ -1052,6 +1056,13 @@ int kernel_kexec(void) =20 #ifdef CONFIG_KEXEC_JUMP if (kexec_image->preserve_context) { + /* + * This flow is analogous to hibernation flows that occur after + * creating an image and after the image hernel has got control + * back, and in case the devices have been reset or otherwise + * manipulated in the meantime, it uses the device callbacks + * used by the latter. + */ syscore_resume(); Enable_irqs: local_irq_enable(); --=20 2.47.0 From nobody Wed Dec 17 21:39:27 2025 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4BFB1DED49 for ; Mon, 16 Dec 2024 23:37:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392266; cv=none; b=VsaPc5Dg33qGtFX12bMQNBdBCiKsuAr1HgPs49/442HntKFf4AB7dAvD16VWrDxjVVqrkutUcVrlifHiWtqjFQ7WTCUz0u0r34wKQiUH+phXoJfIcksQf72c2jM0sM1qLkJ4CQFSm1Ra9lThO3tjCBHl3yNeOeFWeaHsC+nysQk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392266; c=relaxed/simple; bh=ZnFGZgBd4aLQO4C/x8vIGcFWrrbQsg7gzuJ2gVC5XKg=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qK4Cv9DDyE05oT/3qZRQXEULCu58UCQZ11rhmC4nJ2pecsjZsPZg3tKbErS6xauj99F2QaKCwPm+bhOS2rj1SS+e+jgv/TOrTjNlF+9uh6PiAjVs76HSnzkNu8c9EXzIY/ALQtF0GRny5EitD+jAb9XZHxjXRr8SCiJ3Og61qGc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=C4BhlY9t; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="C4BhlY9t" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=qnDZGmQXXRBfaYMiHf8RcgZ8w0/B4+1DdFBULwwOb1o=; b=C4BhlY9tosLdIgFYuh5FbO/pL/ 7pNlWsSUXRZ1uO0qtK2mm/dZ0iRLw8BpfekA09Za0yudhv6GpxK5wD9LykP53btHBu8t84yPPLeJc hR4RYsxmPxIjQ+gIfBazGmf3BYNpEgmJuQVvPgQiBny2vTiCeytTB1OrzDrwiwjGLf9pLrFOKu+SA X3AS/tMDFA29VUlMJdqqTllWAqPSXjP6LTmuQ4oUP6EGhKLRleXwIBhpqe+eXYSsny4nZ/OhwQ95n R1pH7h1yp/HOhIEXY6kRe9vqzQKXt/+qGQlM1G3hWAZ3paIT2bnB3dX4UJrUOA8hfayYQ+LdkrPwn hPpVuuVw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeB-00000004xiQ-2AqD; Mon, 16 Dec 2024 23:37:09 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeWV-1XPO; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 8/9] x86/kexec: Cope with relocate_kernel() not being at the start of the page Date: Mon, 16 Dec 2024 23:24:15 +0000 Message-ID: <20241216233704.3208607-9-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse A few places in the kexec control code page make the assumption that the first instruction of relocate_kernel is at the very start of the page. To allow for Clang CFI information to be added to relocate_kernel(), as well as the general principle of removing unwarranted assumptions, fix them to use the external __relocate_kernel_start symbol that the linker adds. This means using a separate addq and subq for calculating offsets, as the assembler can no longer calculate the delta directly for itself and relocations aren't that versatile. Turn the jump from relocate_kernel() to identity_mapped() into a real indirect 'jmp *%rsi' too, while touching it. There was no real reason for it to be a push+ret in the first place, and adding Clang CFI info will also give objtool enough visibility to start complaining 'return with modified stack frame' about it. Signed-off-by: David Woodhouse --- arch/x86/kernel/relocate_kernel_64.S | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 1996cea909ff..d74798d78263 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -95,11 +95,10 @@ SYM_CODE_START_NOALIGN(relocate_kernel) lea PAGE_SIZE(%rsi), %rsp =20 /* jump to identity mapped page */ - addq $(identity_mapped - relocate_kernel), %rsi - pushq %rsi - ANNOTATE_UNRET_SAFE - ret - int3 + addq $identity_mapped, %rsi + subq $__relocate_kernel_start, %rsi + ANNOTATE_RETPOLINE_SAFE + jmp *%rsi SYM_CODE_END(relocate_kernel) =20 SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) @@ -219,16 +218,21 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) =20 /* get the re-entry point of the peer system */ popq %rbp - leaq relocate_kernel(%rip), %r8 movq kexec_pa_swap_page(%rip), %r10 movq pa_backup_pages_map(%rip), %rdi movq kexec_pa_table_page(%rip), %rax movq %rax, %cr3 + + /* Find start (and end) of this physical mapping of control page */ + leaq (%rip), %r8 + ANNOTATE_NOENDBR + andq $PAGE_MASK, %r8 lea PAGE_SIZE(%r8), %rsp movq $1, %r11 /* Ensure preserve_context flag is set */ call swap_pages movq kexec_va_control_page(%rip), %rax - addq $(virtual_mapped - relocate_kernel), %rax + addq $virtual_mapped, %rax + subq $__relocate_kernel_start, %rax pushq %rax ANNOTATE_UNRET_SAFE ret --=20 2.47.0 From nobody Wed Dec 17 21:39:27 2025 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 391ED1DDA31 for ; Mon, 16 Dec 2024 23:37:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.92.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392262; cv=none; b=ZBeAtQ4oL/eQ9tsIlsM1cTgNU7ebhKrAkNN0G7D/eytdKlRcb0IwGs3oGjw+o62wlSazPFuUS0FW3RsYuS6fCT3ws5Y+0lg77d9BH0JKi1aS2FufdQfn3Zal99Cvm8UB3SjxloW4UGnbTacC/TiaP3T+k3P7Uhs8nCtNGvZfpfI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734392262; c=relaxed/simple; bh=66bK3hS4MjiVy30c7KVE9tQcd7zIjJHoi4zEicwN8Po=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NEPVE/bR6yqGDGP4RlevVVZ02LtgTwSxWtPyeQGzmOKlCrEfllQQF/1TXIJiFN/VtUftoGgSxg7PfI+7sOsWHCOvS1VRFGl8KSIIOmjOrKeOGJvApQ83uswR70Uk61leNfhj5LFKuIRQR6kg6h97yhTFYT8Lhzg0SVoo35w091Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=desiato.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=YtrdpZV5; arc=none smtp.client-ip=90.155.92.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=desiato.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="YtrdpZV5" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:To:From:Reply-To: Cc:Content-Type:Content-ID:Content-Description; bh=2QdnXzG6oBq2pMrwUNHKRzdcvGBvqjVeazTvAuBzw50=; b=YtrdpZV5q7RwJLVF+vAMRjX/iH svvl4giWUH2iMjCT5ixTY4kEOHrtcZbHf0sb7EjpmCJgmE/sGAbZE5/TzDdZp41nJPLVbK0OwDsxK 7CRv9x10roEGFw+y3DjL/ybkPrACOs5l3dm3GetsvoZmvSWI9cVn2hEHt2XQqQnC9+/1zuG/pvOlh h8sKTArFfHWsPva6AJ2ueEWgthsJtT7pZtQuWLu7h916ye+6ENMw/Cy5CHBM85kTuKjE1/Qv2zxS7 l+1YD1+ry43OOQg5tRoITUxi0CR0Hs2zxOI2dNwgK02GphYE/IqWU2Qk8pggCLFvwNc7Ssv1ZbqI6 I8LqVnJQ==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by desiato.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeB-00000004xiR-2Ak7; Mon, 16 Dec 2024 23:37:08 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNKeA-0000000DeWZ-1mhY; Mon, 16 Dec 2024 23:37:06 +0000 From: David Woodhouse To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Eric Biederman , David Woodhouse , Sourabh Jain , Hari Bathini , Michael Ellerman , Thomas Zimmermann , Andrew Morton , Baoquan He , Yuntao Wang , David Kaplan , Tao Liu , "Kirill A. Shutemov" , Kai Huang , Ard Biesheuvel , Josh Poimboeuf , Breno Leitao , Wei Yang , Rong Xu , =?UTF-8?q?Thomas=20Wei=C3=9Fschuh?= , linux-kernel@vger.kernel.org, kexec@lists.infradead.org, Simon Horman , Dave Young , Peter Zijlstra , bsz@amazon.de, nathan@kernel.org Subject: [PATCH 9/9] x86/kexec: Use typedef for relocate_kernel_fn function prototype Date: Mon, 16 Dec 2024 23:24:16 +0000 Message-ID: <20241216233704.3208607-10-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241216233704.3208607-1-dwmw2@infradead.org> References: <20241216233704.3208607-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by desiato.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse Both i386 and x86_64 now copy the relocate_kernel function into the control page and execute it from there, using an open-coded function pointer. Use a typedef for it instead. Signed-off-by: David Woodhouse --- arch/x86/include/asm/kexec.h | 26 +++++++++++++------------- arch/x86/kernel/machine_kexec_32.c | 7 +------ arch/x86/kernel/machine_kexec_64.c | 6 +----- 3 files changed, 15 insertions(+), 24 deletions(-) diff --git a/arch/x86/include/asm/kexec.h b/arch/x86/include/asm/kexec.h index 48e4f44f794f..8ad187462b68 100644 --- a/arch/x86/include/asm/kexec.h +++ b/arch/x86/include/asm/kexec.h @@ -111,21 +111,21 @@ static inline void crash_setup_regs(struct pt_regs *n= ewregs, } =20 #ifdef CONFIG_X86_32 -asmlinkage unsigned long -relocate_kernel(unsigned long indirection_page, - unsigned long control_page, - unsigned long start_address, - unsigned int has_pae, - unsigned int preserve_context); +typedef asmlinkage unsigned long +relocate_kernel_fn(unsigned long indirection_page, + unsigned long control_page, + unsigned long start_address, + unsigned int has_pae, + unsigned int preserve_context); #else -unsigned long -relocate_kernel(unsigned long indirection_page, - unsigned long pa_control_page, - unsigned long start_address, - unsigned int preserve_context, - unsigned int host_mem_enc_active); +typedef unsigned long +relocate_kernel_fn(unsigned long indirection_page, + unsigned long pa_control_page, + unsigned long start_address, + unsigned int preserve_context, + unsigned int host_mem_enc_active); #endif - +extern relocate_kernel_fn relocate_kernel; #define ARCH_HAS_KIMAGE_ARCH =20 #ifdef CONFIG_X86_32 diff --git a/arch/x86/kernel/machine_kexec_32.c b/arch/x86/kernel/machine_k= exec_32.c index 1b373d79cedc..80265162aeff 100644 --- a/arch/x86/kernel/machine_kexec_32.c +++ b/arch/x86/kernel/machine_kexec_32.c @@ -160,15 +160,10 @@ void machine_kexec_cleanup(struct kimage *image) */ void machine_kexec(struct kimage *image) { + relocate_kernel_fn *relocate_kernel_ptr; unsigned long page_list[PAGES_NR]; void *control_page; int save_ftrace_enabled; - asmlinkage unsigned long - (*relocate_kernel_ptr)(unsigned long indirection_page, - unsigned long control_page, - unsigned long start_address, - unsigned int has_pae, - unsigned int preserve_context); =20 #ifdef CONFIG_KEXEC_JUMP if (image->preserve_context) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 1440f792a86d..dd75a51463a2 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -344,12 +344,8 @@ void machine_kexec_cleanup(struct kimage *image) */ void __nocfi machine_kexec(struct kimage *image) { - unsigned long (*relocate_kernel_ptr)(unsigned long indirection_page, - unsigned long pa_control_page, - unsigned long start_address, - unsigned int preserve_context, - unsigned int host_mem_enc_active); unsigned long reloc_start =3D (unsigned long)__relocate_kernel_start; + relocate_kernel_fn *relocate_kernel_ptr; unsigned int host_mem_enc_active; int save_ftrace_enabled; void *control_page; --=20 2.47.0