From nobody Wed Feb 11 01:26:04 2026 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B21E3191F83 for ; Mon, 25 Nov 2024 10:08:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732529315; cv=none; b=WmWdiNHPRzFnAu7kY7O2XU0cqGzOl+lz4BOD7/Hhdd/j6jPMvekwpqhkPVl+ZRtwc9ySVEwZ+Lrcbh5Xz7AOOk9iJpJcHG4pml08cNYvDNY2OFsC7gW917ikpiXsCLqK06V6ZT/gbu0u52lzAAo4iOxQrPSFAOJMf5LxT2DLGok= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732529315; c=relaxed/simple; bh=Stn6h+FvCrHxrLa6wU04/sPz7ZCYV2SfQ2zY+k0QTKs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mhLzHNmAxI7nv72J1tVr3VzWgJGTsw2c775jRHa12j5fBhwK55QTA4Ttok7OBqJh6xzlRMqMOU7bJsN98r2yUtRAyJMhoA+WJ/fCQBruvQrjT2CbdbxPCG3OkZ6hLpizmDvApPHo6LzQUaIMc7cEcvbK6QcL7alX5GXm1kzSxWE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org; spf=none smtp.mailfrom=casper.srs.infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=pSeW6BGA; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="pSeW6BGA" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=RnqUFSk9pQTD5s++rex+HrAEPnOx1bUxBJhNxxRb3UY=; b=pSeW6BGAlUKbiJqeZcmooz2VMr jeTuBUqmsK4tWhcR633gGedDTwpjiOjz0h+qT2HSNlYx+64YiUITDJiaozwMKAaSrY9Gzw0JsriJU YVf9rIIN6lJdrldKg8JvBmp+/DAraZjz9pFdhW0xM9lKYZ1/4a6oHl5WN0IxQQjwHK5cF17DqdACE Z0Zpd4FqdImTySXkJ8OxvZtBWvX7NhwjdcYcJQW4nVv4Ku63V7MsNxoGP39GQJqud2Pk+BwaGrTgW AzfWVTgFlzOyQVb/FRfGAD/cJ8CkNCn4hvtKm/YtZJahUyTupMpT/AREoLGaYok9Yq6WPuf7zdIoB qXr4Wbug==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1tFW0z-0000000Biqw-09mU; Mon, 25 Nov 2024 10:08:22 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tFW0z-000000000lB-2Nmr; Mon, 25 Nov 2024 10:08:21 +0000 From: David Woodhouse To: kexec@lists.infradead.org Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , David Woodhouse , "Kirill A. Shutemov" , Kai Huang , Nikolay Borisov , linux-kernel@vger.kernel.org, Simon Horman , Dave Young , Peter Zijlstra , jpoimboe@kernel.org, bsz@amazon.de Subject: [RFC PATCH v3 08/20] x86/kexec: Invoke copy of relocate_kernel() instead of the original Date: Mon, 25 Nov 2024 09:54:38 +0000 Message-ID: <20241125100815.2512-9-dwmw2@infradead.org> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241125100815.2512-1-dwmw2@infradead.org> References: <20241125100815.2512-1-dwmw2@infradead.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Sender: David Woodhouse X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html Content-Type: text/plain; charset="utf-8" From: David Woodhouse This currently calls set_memory_x() from machine_kexec_prepare() just like the 32-bit version does. That's actually a bit earlier than I'd like, as it leaves the page RWX all the time the image is even *loaded*. Subsequent commits will eliminate all the writes to the page between the point it's marked executable in machine_kexec_prepare() the time that relocate_kernel() is running and has switched to the identmap %cr3, so that it can be ROX. But that can't happen until it's moved to the .data section of the kernel, and *that* can't happen until we start executing the copy instead of executing it in place in the kernel .text. So break the circular dependency in those commits by letting it be RWX for now. Signed-off-by: David Woodhouse --- arch/x86/kernel/machine_kexec_64.c | 30 ++++++++++++++++++++++------ arch/x86/kernel/relocate_kernel_64.S | 5 ++++- 2 files changed, 28 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/machine_kexec_64.c b/arch/x86/kernel/machine_k= exec_64.c index 431a117f3fb3..6fcf54e87d44 100644 --- a/arch/x86/kernel/machine_kexec_64.c +++ b/arch/x86/kernel/machine_kexec_64.c @@ -157,7 +157,12 @@ static int init_transition_pgtable(struct kimage *imag= e, pgd_t *pgd, pmd_t *pmd; pte_t *pte; =20 - vaddr =3D (unsigned long)relocate_kernel; + /* + * For the transition to the identity mapped page tables, the control + * code page also needs to be mapped at the virtual address it starts + * off running from. + */ + vaddr =3D (unsigned long)__va(control_page); paddr =3D control_page; pgd +=3D pgd_index(vaddr); if (!pgd_present(*pgd)) { @@ -317,11 +322,17 @@ int machine_kexec_prepare(struct kimage *image) =20 __memcpy(control_page, relocate_kernel, KEXEC_CONTROL_CODE_MAX_SIZE); =20 + set_memory_x((unsigned long)control_page, 1); + return 0; } =20 void machine_kexec_cleanup(struct kimage *image) { + void *control_page =3D page_address(image->control_code_page); + + set_memory_nx((unsigned long)control_page, 1); + free_transition_pgtable(image); } =20 @@ -331,6 +342,11 @@ void machine_kexec_cleanup(struct kimage *image) */ void machine_kexec(struct kimage *image) { + unsigned long (*relocate_kernel_ptr)(unsigned long indirection_page, + unsigned long page_list, + unsigned long start_address, + unsigned int preserve_context, + unsigned int host_mem_enc_active); unsigned long page_list[PAGES_NR]; unsigned int host_mem_enc_active; int save_ftrace_enabled; @@ -377,6 +393,8 @@ void machine_kexec(struct kimage *image) page_list[PA_SWAP_PAGE] =3D (page_to_pfn(image->swap_page) << PAGE_SHIFT); =20 + relocate_kernel_ptr =3D control_page; + /* * The segment registers are funny things, they have both a * visible and an invisible part. Whenever the visible part is @@ -396,11 +414,11 @@ void machine_kexec(struct kimage *image) native_gdt_invalidate(); =20 /* now call it */ - image->start =3D relocate_kernel((unsigned long)image->head, - (unsigned long)page_list, - image->start, - image->preserve_context, - host_mem_enc_active); + image->start =3D relocate_kernel_ptr((unsigned long)image->head, + (unsigned long)page_list, + image->start, + image->preserve_context, + host_mem_enc_active); =20 #ifdef CONFIG_KEXEC_JUMP if (image->preserve_context) diff --git a/arch/x86/kernel/relocate_kernel_64.S b/arch/x86/kernel/relocat= e_kernel_64.S index 50cc33f2ecb7..b48bd82843fd 100644 --- a/arch/x86/kernel/relocate_kernel_64.S +++ b/arch/x86/kernel/relocate_kernel_64.S @@ -39,6 +39,7 @@ #define CP_PA_TABLE_PAGE DATA(0x20) #define CP_PA_SWAP_PAGE DATA(0x28) #define CP_PA_BACKUP_PAGES_MAP DATA(0x30) +#define CP_VA_CONTROL_PAGE DATA(0x38) =20 .text .align PAGE_SIZE @@ -99,6 +100,7 @@ SYM_CODE_START_NOALIGN(relocate_kernel) movq %r9, CP_PA_TABLE_PAGE(%r11) movq %r10, CP_PA_SWAP_PAGE(%r11) movq %rdi, CP_PA_BACKUP_PAGES_MAP(%r11) + movq %r11, CP_VA_CONTROL_PAGE(%r11) =20 /* Save the preserve_context to %r11 as swap_pages clobbers %rcx. */ movq %rcx, %r11 @@ -235,7 +237,8 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped) movq %rax, %cr3 lea PAGE_SIZE(%r8), %rsp call swap_pages - movq $virtual_mapped, %rax + movq CP_VA_CONTROL_PAGE(%r8), %rax + addq $(virtual_mapped - relocate_kernel), %rax pushq %rax ANNOTATE_UNRET_SAFE ret --=20 2.47.0