From nobody Wed Jun 17 05:27:56 2026 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F8FD3ACEFB; Mon, 27 Apr 2026 22:26:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777328803; cv=none; b=R82g8mVB0uT7kyfdVr7HeTJVbj05cR2ALwCHDnQUAj19Zbc0kL7POOr6E2Qfc1fQEP26Rklip7PFSn8l/rsvFyadezgv/yRfCOCmtUWwSHRWKWFhshIKxIu2QnsNOt1IkmSgIEl7Ba0WTut8uTrgRz3Zz3u9znd+w4JYpyBMUIM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777328803; c=relaxed/simple; bh=C7v+o3n6yrrgIe5zEXAwsLGU0z541FQ1555+zHt7Q20=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=KUiMRDkKi5S0UrB62Ty5QboBkGGkxlxZcviwGkGw1rkfd16DrgsXIrOZLt7duOtUOUgiS/0MnwUwXkdOH1LXUwjaCZzQVVMtdo4dOsDsyIlBC6cjUCNOKv5EG3cXOhznm30t8KrdYMQuQVRt0kTkF9RlBvOcJDheHag/fE6HsZA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=rw2t/P9a; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=Tn/o18kF; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="rw2t/P9a"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="Tn/o18kF" Date: Mon, 27 Apr 2026 22:26:38 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1777328800; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V7x2DMqpor/VnxfvCsMqVUBT6XTLX4/EcJodBz8y3q0=; b=rw2t/P9aUXziH98I5nJ6KrgR9QFgmVeDgmQKqHKb3+BG0T/OTZ7Kc6Jyp3Kkj6B6IRmbzj IoRBzRKnfmdAONjgeMrlfHGW5QHYmIWJbZeNbu38ITZjwp5AwUWZFnepEQk/HdhkNv1/UJ vc15eTD/91J3CvX3JdecC/lX0eSAHqynA7qlnH9CoYlWj1jlpECbK/3Oyo7159vXp0BPaV gexwQcr7Oyem2ZvLPtNTL9ICniu5VS52Q0kPJ6JUMCDWlh9iIpwb6U8rw35qVegUqk76ws h4QKF35gY8ADePQUoANdCmEO5UhKPG+KwMMxvYCvIFwdlvKfx6lGC0UtUUIqgQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1777328800; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V7x2DMqpor/VnxfvCsMqVUBT6XTLX4/EcJodBz8y3q0=; b=Tn/o18kFtzSuiqo/gQd3H2EGOlacnGfuyCyc7V2mJdGMTUGkMlgvmGvSYqDWNBezPnnXhC oOmm+/1Ekx32TDAA== From: "tip-bot2 for Rick Edgecombe" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: x86/tdx] x86/virt/tdx: Pull kexec cache flush logic into arch/x86 Cc: Rick Edgecombe , Vishal Verma , Dave Hansen , Chao Gao , Kai Huang , "Kiryl Shutsemau (Meta)" , Sean Christopherson , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20260402-fuller_tdx_kexec_support-v3-2-34438d7094bf@intel.com> References: <20260402-fuller_tdx_kexec_support-v3-2-34438d7094bf@intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <177732879886.3521451.13184879089319770924.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the x86/tdx branch of tip: Commit-ID: 597bdf6e068e2c8f6e93f24bf39a34cdf017481f Gitweb: https://git.kernel.org/tip/597bdf6e068e2c8f6e93f24bf39a34cdf= 017481f Author: Rick Edgecombe AuthorDate: Thu, 02 Apr 2026 00:32:02 -06:00 Committer: Dave Hansen CommitterDate: Mon, 27 Apr 2026 15:13:38 -07:00 x86/virt/tdx: Pull kexec cache flush logic into arch/x86 KVM tries to take care of some required cache flushing earlier in the kexec path in order to be kind to some long standing races that can occur later in the operation. Until recently, VMXOFF was handled within KVM. Since VMX being enabled is required to make a SEAMCALL, it had the best per-cpu scoped operation to plug the flushing into. So it is kicked off from there. This early kexec cache flushing in KVM happens via a syscore shutdown callback. Now that VMX enablement control has moved to arch/x86, which has grown its own syscore shutdown callback, it no longer make sense for it to live in KVM. It fits better with the TDX enablement managing code. In addition, future changes will add a SEAMCALL that happens immediately before VMXOFF, which means the cache flush in KVM will be too late to flush the cache before the last SEAMCALL. So move it to the newly added TDX arch/x86 syscore shutdown handler. Since tdx_cpu_flush_cache_for_kexec() is no longer needed by KVM, make it static and remove the export. Since it is also not part of an operation spread across disparate components, remove the redundant comments and verbose naming. In the existing KVM based code, CPU offline also funnels through tdx_cpu_flush_cache_for_kexec(). Add an explicit WBINVD in tdx_offline_cpu() as well, even though it may be redundant with WBINVD done elsewhere during CPU offline (e.g. hlt_play_dead()). This avoids relying on fragile code ordering for cache coherency safety. [Vishal: add explicit WBINVD in tdx_offline_cpu()] Signed-off-by: Rick Edgecombe Signed-off-by: Vishal Verma Signed-off-by: Dave Hansen Reviewed-by: Chao Gao Acked-by: Kai Huang Acked-by: Kiryl Shutsemau (Meta) Acked-by: Sean Christopherson Link: https://patch.msgid.link/20260402-fuller_tdx_kexec_support-v3-2-34438= d7094bf@intel.com --- arch/x86/include/asm/tdx.h | 6 +----- arch/x86/kvm/vmx/tdx.c | 10 +-------- arch/x86/virt/vmx/tdx/tdx.c | 46 +++++++++++++++++++++--------------- 3 files changed, 27 insertions(+), 35 deletions(-) diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index a149740..bf83a97 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -226,11 +226,5 @@ static inline const char *tdx_dump_mce_info(struct mce= *m) { return NULL; } static inline const struct tdx_sys_info *tdx_get_sysinfo(void) { return NU= LL; } #endif /* CONFIG_INTEL_TDX_HOST */ =20 -#ifdef CONFIG_KEXEC_CORE -void tdx_cpu_flush_cache_for_kexec(void); -#else -static inline void tdx_cpu_flush_cache_for_kexec(void) { } -#endif - #endif /* !__ASSEMBLER__ */ #endif /* _ASM_X86_TDX_H */ diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 04ce321..ed12805 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -440,16 +440,6 @@ void tdx_disable_virtualization_cpu(void) tdx_flush_vp(&arg); } local_irq_restore(flags); - - /* - * Flush cache now if kexec is possible: this is necessary to avoid - * having dirty private memory cachelines when the new kernel boots, - * but WBINVD is a relatively expensive operation and doing it during - * kexec can exacerbate races in native_stop_other_cpus(). Do it - * now, since this is a safe moment and there is going to be no more - * TDX activity on this CPU from this point on. - */ - tdx_cpu_flush_cache_for_kexec(); } =20 #define TDX_SEAMCALL_RETRIES 10000 diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index cb9b321..1b2d854 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -184,6 +184,17 @@ static int tdx_online_cpu(unsigned int cpu) return ret; } =20 +static void tdx_cpu_flush_cache(void) +{ + lockdep_assert_preemption_disabled(); + + if (!this_cpu_read(cache_state_incoherent)) + return; + + wbinvd(); + this_cpu_write(cache_state_incoherent, false); +} + static int tdx_offline_cpu(unsigned int cpu) { int i; @@ -220,12 +231,28 @@ static int tdx_offline_cpu(unsigned int cpu) return -EBUSY; =20 done: + /* + * Flush cache on the CPU going offline to ensure no dirty + * cachelines of TDX private memory remain. This may be + * redundant with WBINVD done elsewhere during CPU offline + * (e.g. hlt_play_dead()), but do it explicitly for safety. + */ + tdx_cpu_flush_cache(); x86_virt_put_ref(X86_FEATURE_VMX); return 0; } =20 static void tdx_shutdown_cpu(void *ign) { + /* + * Flush cache in preparation for kexec - this is necessary to avoid + * having dirty private memory cachelines when the new kernel boots, + * but WBINVD is a relatively expensive operation and doing it during + * kexec can exacerbate races in native_stop_other_cpus(). Do it + * now, since this is a safe moment and there is going to be no more + * TDX activity on this CPU from this point on. + */ + tdx_cpu_flush_cache(); x86_virt_put_ref(X86_FEATURE_VMX); } =20 @@ -1920,22 +1947,3 @@ u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, struct pag= e *page) return seamcall(TDH_PHYMEM_PAGE_WBINVD, &args); } EXPORT_SYMBOL_FOR_KVM(tdh_phymem_page_wbinvd_hkid); - -#ifdef CONFIG_KEXEC_CORE -void tdx_cpu_flush_cache_for_kexec(void) -{ - lockdep_assert_preemption_disabled(); - - if (!this_cpu_read(cache_state_incoherent)) - return; - - /* - * Private memory cachelines need to be clean at the time of - * kexec. Write them back now, as the caller promises that - * there should be no more SEAMCALLs on this CPU. - */ - wbinvd(); - this_cpu_write(cache_state_incoherent, false); -} -EXPORT_SYMBOL_FOR_KVM(tdx_cpu_flush_cache_for_kexec); -#endif