From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
TSA mitigation:
d8010d4ba43e ("x86/bugs: Add a Transient Scheduler Attacks mitigation")
introduced VM_CLEAR_CPU_BUFFERS for guests on AMD CPUs. Currently on Intel
CLEAR_CPU_BUFFERS is being used for guests which has a much broader scope
(kernel->user also).
Make mitigations on Intel consistent with TSA. This would help handling the
guest-only mitigations better in future.
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
[sean: make CLEAR_CPU_BUF_VM mutually exclusive with the MMIO mitigation]
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
arch/x86/kernel/cpu/bugs.c | 9 +++++++--
arch/x86/kvm/vmx/vmenter.S | 2 +-
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 6a526ae1fe99..723666a1357e 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -194,7 +194,7 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);
/*
* Controls CPU Fill buffer clear before VMenter. This is a subset of
- * X86_FEATURE_CLEAR_CPU_BUF, and should only be enabled when KVM-only
+ * X86_FEATURE_CLEAR_CPU_BUF_VM, and should only be enabled when KVM-only
* mitigation is required.
*/
DEFINE_STATIC_KEY_FALSE(cpu_buf_vm_clear);
@@ -536,6 +536,7 @@ static void __init mds_apply_mitigation(void)
if (mds_mitigation == MDS_MITIGATION_FULL ||
mds_mitigation == MDS_MITIGATION_VMWERV) {
setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
+ setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM);
if (!boot_cpu_has(X86_BUG_MSBDS_ONLY) &&
(mds_nosmt || smt_mitigations == SMT_MITIGATIONS_ON))
cpu_smt_disable(false);
@@ -647,6 +648,7 @@ static void __init taa_apply_mitigation(void)
* present on host, enable the mitigation for UCODE_NEEDED as well.
*/
setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
+ setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM);
if (taa_nosmt || smt_mitigations == SMT_MITIGATIONS_ON)
cpu_smt_disable(false);
@@ -748,6 +750,7 @@ static void __init mmio_apply_mitigation(void)
*/
if (verw_clear_cpu_buf_mitigation_selected) {
setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
+ setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM);
static_branch_disable(&cpu_buf_vm_clear);
} else {
static_branch_enable(&cpu_buf_vm_clear);
@@ -839,8 +842,10 @@ static void __init rfds_update_mitigation(void)
static void __init rfds_apply_mitigation(void)
{
- if (rfds_mitigation == RFDS_MITIGATION_VERW)
+ if (rfds_mitigation == RFDS_MITIGATION_VERW) {
setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
+ setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM);
+ }
}
static __init int rfds_parse_cmdline(char *str)
diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S
index bc255d709d8a..1f99a98a16a2 100644
--- a/arch/x86/kvm/vmx/vmenter.S
+++ b/arch/x86/kvm/vmx/vmenter.S
@@ -161,7 +161,7 @@ SYM_FUNC_START(__vmx_vcpu_run)
mov VCPU_RAX(%_ASM_AX), %_ASM_AX
/* Clobbers EFLAGS.ZF */
- CLEAR_CPU_BUFFERS
+ VM_CLEAR_CPU_BUFFERS
/* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */
jnc .Lvmlaunch
--
2.51.1.930.gacf6e81ea2-googOn Thu, Oct 30, 2025 at 05:30:33PM -0700, Sean Christopherson wrote:
> From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
>
> TSA mitigation:
>
> d8010d4ba43e ("x86/bugs: Add a Transient Scheduler Attacks mitigation")
>
> introduced VM_CLEAR_CPU_BUFFERS for guests on AMD CPUs. Currently on Intel
> CLEAR_CPU_BUFFERS is being used for guests which has a much broader scope
> (kernel->user also).
>
> Make mitigations on Intel consistent with TSA. This would help handling the
"consistent" as in "use the VM-specific buffer clearing variant in VMX too"?
In any case:
Acked-by: Borislav Petkov (AMD) <bp@alien8.de>
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On Fri, Nov 07, 2025 at 07:59:41PM +0100, Borislav Petkov wrote:
> On Thu, Oct 30, 2025 at 05:30:33PM -0700, Sean Christopherson wrote:
> > From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
> >
> > TSA mitigation:
> >
> > d8010d4ba43e ("x86/bugs: Add a Transient Scheduler Attacks mitigation")
> >
> > introduced VM_CLEAR_CPU_BUFFERS for guests on AMD CPUs. Currently on Intel
> > CLEAR_CPU_BUFFERS is being used for guests which has a much broader scope
> > (kernel->user also).
> >
> > Make mitigations on Intel consistent with TSA. This would help handling the
>
> "consistent" as in "use the VM-specific buffer clearing variant in VMX too"?
That's correct.
> In any case:
>
> Acked-by: Borislav Petkov (AMD) <bp@alien8.de>
Thanks.
On Thu, Oct 30, 2025 at 05:30:33PM -0700, Sean Christopherson wrote:
> From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
>
> TSA mitigation:
>
> d8010d4ba43e ("x86/bugs: Add a Transient Scheduler Attacks mitigation")
>
> introduced VM_CLEAR_CPU_BUFFERS for guests on AMD CPUs. Currently on Intel
> CLEAR_CPU_BUFFERS is being used for guests which has a much broader scope
> (kernel->user also).
>
> Make mitigations on Intel consistent with TSA. This would help handling the
> guest-only mitigations better in future.
>
> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
> [sean: make CLEAR_CPU_BUF_VM mutually exclusive with the MMIO mitigation]
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
> arch/x86/kernel/cpu/bugs.c | 9 +++++++--
> arch/x86/kvm/vmx/vmenter.S | 2 +-
> 2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index 6a526ae1fe99..723666a1357e 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
> @@ -194,7 +194,7 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);
>
> /*
> * Controls CPU Fill buffer clear before VMenter. This is a subset of
> - * X86_FEATURE_CLEAR_CPU_BUF, and should only be enabled when KVM-only
> + * X86_FEATURE_CLEAR_CPU_BUF_VM, and should only be enabled when KVM-only
> * mitigation is required.
> */
> DEFINE_STATIC_KEY_FALSE(cpu_buf_vm_clear);
> @@ -536,6 +536,7 @@ static void __init mds_apply_mitigation(void)
> if (mds_mitigation == MDS_MITIGATION_FULL ||
> mds_mitigation == MDS_MITIGATION_VMWERV) {
> setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
> + setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM);
> if (!boot_cpu_has(X86_BUG_MSBDS_ONLY) &&
> (mds_nosmt || smt_mitigations == SMT_MITIGATIONS_ON))
> cpu_smt_disable(false);
> @@ -647,6 +648,7 @@ static void __init taa_apply_mitigation(void)
> * present on host, enable the mitigation for UCODE_NEEDED as well.
> */
> setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
> + setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM);
>
> if (taa_nosmt || smt_mitigations == SMT_MITIGATIONS_ON)
> cpu_smt_disable(false);
> @@ -748,6 +750,7 @@ static void __init mmio_apply_mitigation(void)
> */
> if (verw_clear_cpu_buf_mitigation_selected) {
> setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
> + setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM);
> static_branch_disable(&cpu_buf_vm_clear);
> } else {
> static_branch_enable(&cpu_buf_vm_clear);
> @@ -839,8 +842,10 @@ static void __init rfds_update_mitigation(void)
>
> static void __init rfds_apply_mitigation(void)
> {
> - if (rfds_mitigation == RFDS_MITIGATION_VERW)
> + if (rfds_mitigation == RFDS_MITIGATION_VERW) {
> setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF);
> + setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM);
> + }
> }
>
> static __init int rfds_parse_cmdline(char *str)
> diff --git a/arch/x86/kvm/vmx/vmenter.S b/arch/x86/kvm/vmx/vmenter.S
> index bc255d709d8a..1f99a98a16a2 100644
> --- a/arch/x86/kvm/vmx/vmenter.S
> +++ b/arch/x86/kvm/vmx/vmenter.S
> @@ -161,7 +161,7 @@ SYM_FUNC_START(__vmx_vcpu_run)
> mov VCPU_RAX(%_ASM_AX), %_ASM_AX
>
> /* Clobbers EFLAGS.ZF */
> - CLEAR_CPU_BUFFERS
> + VM_CLEAR_CPU_BUFFERS
>
> /* Check EFLAGS.CF from the VMX_RUN_VMRESUME bit test above. */
> jnc .Lvmlaunch
> --
Sean, based on Brendan's feedback, below are the fixes to the comments on
top of this patch:
---
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 08ed5a2e46a5..2be9be782013 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -321,9 +321,11 @@
#endif
.endm
+/* Primarily used in exit-to-userspace path */
#define CLEAR_CPU_BUFFERS \
__CLEAR_CPU_BUFFERS X86_FEATURE_CLEAR_CPU_BUF
+/* For use in KVM */
#define VM_CLEAR_CPU_BUFFERS \
__CLEAR_CPU_BUFFERS X86_FEATURE_CLEAR_CPU_BUF_VM
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 723666a1357e..49d5797a2a42 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -490,7 +490,7 @@ static enum rfds_mitigations rfds_mitigation __ro_after_init =
/*
* Set if any of MDS/TAA/MMIO/RFDS are going to enable VERW clearing
- * through X86_FEATURE_CLEAR_CPU_BUF on kernel and guest entry.
+ * at userspace *and* guest entry.
*/
static bool verw_clear_cpu_buf_mitigation_selected __ro_after_init;
On Mon, Nov 03, 2025 at 10:18:40AM -0800, Pawan Gupta wrote:
> diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
> index 08ed5a2e46a5..2be9be782013 100644
> --- a/arch/x86/include/asm/nospec-branch.h
> +++ b/arch/x86/include/asm/nospec-branch.h
> @@ -321,9 +321,11 @@
> #endif
> .endm
>
> +/* Primarily used in exit-to-userspace path */
What does "primarily" mean here?
$ git grep -w CLEAR_CPU_BUFFERS
says *only* the kernel->user vector.
> #define CLEAR_CPU_BUFFERS \
> __CLEAR_CPU_BUFFERS X86_FEATURE_CLEAR_CPU_BUF
>
> +/* For use in KVM */
That's why the "VM_" prefix is there.
The comments in arch/x86/include/asm/cpufeatures.h actually already explain
that, you could make them more explicit but let's not sprinkle comments
willy-nilly.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On Fri, Nov 07, 2025 at 08:05:34PM +0100, Borislav Petkov wrote: > On Mon, Nov 03, 2025 at 10:18:40AM -0800, Pawan Gupta wrote: > > diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h > > index 08ed5a2e46a5..2be9be782013 100644 > > --- a/arch/x86/include/asm/nospec-branch.h > > +++ b/arch/x86/include/asm/nospec-branch.h > > @@ -321,9 +321,11 @@ > > #endif > > .endm > > > > +/* Primarily used in exit-to-userspace path */ > > What does "primarily" mean here? > $ git grep -w CLEAR_CPU_BUFFERS > > says *only* the kernel->user vector. By the end of this series, yes. At this patch this is used in VMX also: arch/x86/kvm/vmx/vmenter.S:164: CLEAR_CPU_BUFFERS "Primarily" can be dropped by the patch that replaces it in SVM/VMX. > > #define CLEAR_CPU_BUFFERS \ > > __CLEAR_CPU_BUFFERS X86_FEATURE_CLEAR_CPU_BUF > > > > +/* For use in KVM */ > > That's why the "VM_" prefix is there. > > The comments in arch/x86/include/asm/cpufeatures.h actually already explain > that, you could make them more explicit but let's not sprinkle comments > willy-nilly. As Sean pointed out, this goes away in later patches.
On Fri, Nov 07, 2025, Borislav Petkov wrote: > On Mon, Nov 03, 2025 at 10:18:40AM -0800, Pawan Gupta wrote: > > diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h > > index 08ed5a2e46a5..2be9be782013 100644 > > --- a/arch/x86/include/asm/nospec-branch.h > > +++ b/arch/x86/include/asm/nospec-branch.h > > @@ -321,9 +321,11 @@ > > #endif > > .endm > > > > +/* Primarily used in exit-to-userspace path */ > > What does "primarily" mean here? > > $ git grep -w CLEAR_CPU_BUFFERS > > says *only* the kernel->user vector. How about: /* If necessary, emit VERW on exit-to-userspace to clear CPU buffers. */ #define CLEAR_CPU_BUFFERS \ ALTERNATIVE "", __CLEAR_CPU_BUFFERS, X86_FEATURE_CLEAR_CPU_BUF > > > #define CLEAR_CPU_BUFFERS \ > > __CLEAR_CPU_BUFFERS X86_FEATURE_CLEAR_CPU_BUF > > > > +/* For use in KVM */ > > That's why the "VM_" prefix is there. Ya, and this goes away (moved into SVM) by the end of the series.
On Tue, Nov 11, 2025 at 02:03:40PM -0800, Sean Christopherson wrote:
> How about:
>
> /* If necessary, emit VERW on exit-to-userspace to clear CPU buffers. */
> #define CLEAR_CPU_BUFFERS \
> ALTERNATIVE "", __CLEAR_CPU_BUFFERS, X86_FEATURE_CLEAR_CPU_BUF
By the "If necessary" you mean whether X86_FEATURE_CLEAR_CPU_BUF is set or
not, I presume...
I was just wondering whether this macro is going to be used somewhere else
*except* on the kernel->user vector.
> Ya, and this goes away (moved into SVM) by the end of the series.
Aha, lemme look at the rest too then.
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
On Wed, Nov 12, 2025 at 11:23:36AM +0100, Borislav Petkov wrote: > On Tue, Nov 11, 2025 at 02:03:40PM -0800, Sean Christopherson wrote: > > How about: > > > > /* If necessary, emit VERW on exit-to-userspace to clear CPU buffers. */ > > #define CLEAR_CPU_BUFFERS \ > > ALTERNATIVE "", __CLEAR_CPU_BUFFERS, X86_FEATURE_CLEAR_CPU_BUF > > By the "If necessary" you mean whether X86_FEATURE_CLEAR_CPU_BUF is set or > not, I presume... > > I was just wondering whether this macro is going to be used somewhere else > *except* on the kernel->user vector. I believe the intent is to use it only at kernel->user transition.
Rewording my comments from:
https://lore.kernel.org/all/20251029-verw-vm-v1-1-babf9b961519@linux.intel.com/
On Fri Oct 31, 2025 at 12:30 AM UTC, Sean Christopherson wrote:
> From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
>
> TSA mitigation:
>
> d8010d4ba43e ("x86/bugs: Add a Transient Scheduler Attacks mitigation")
>
> introduced VM_CLEAR_CPU_BUFFERS for guests on AMD CPUs. Currently on Intel
> CLEAR_CPU_BUFFERS is being used for guests which has a much broader scope
> (kernel->user also).
>
> Make mitigations on Intel consistent with TSA. This would help handling the
> guest-only mitigations better in future.
>
> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
> [sean: make CLEAR_CPU_BUF_VM mutually exclusive with the MMIO mitigation]
> Signed-off-by: Sean Christopherson <seanjc@google.com>
I think this is a clear improvement. Now that X86_FEATURE_CLEAR_CPU_BUF
has a clear scope, can we also update the comment on its definition in
cpufeatures.h? I.e. say that it's specifically about exit to user.
This also seems like a good moment to update the comment on
verw_clear_cpu_buf_mitigation_selected to mention the _VM flag too.
Also, where we set vmx->disable_fb_clear in vmx_update_fb_clear_dis(),
it still refers to X86_FEATURE_CLEAR_CPU_BUF, is that wrong?
On Fri, Oct 31, 2025 at 11:30:54AM +0000, Brendan Jackman wrote:
> Rewording my comments from:
> https://lore.kernel.org/all/20251029-verw-vm-v1-1-babf9b961519@linux.intel.com/
>
> On Fri Oct 31, 2025 at 12:30 AM UTC, Sean Christopherson wrote:
> > From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
> >
> > TSA mitigation:
> >
> > d8010d4ba43e ("x86/bugs: Add a Transient Scheduler Attacks mitigation")
> >
> > introduced VM_CLEAR_CPU_BUFFERS for guests on AMD CPUs. Currently on Intel
> > CLEAR_CPU_BUFFERS is being used for guests which has a much broader scope
> > (kernel->user also).
> >
> > Make mitigations on Intel consistent with TSA. This would help handling the
> > guest-only mitigations better in future.
> >
> > Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
> > [sean: make CLEAR_CPU_BUF_VM mutually exclusive with the MMIO mitigation]
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
>
> I think this is a clear improvement. Now that X86_FEATURE_CLEAR_CPU_BUF
> has a clear scope, can we also update the comment on its definition in
> cpufeatures.h? I.e. say that it's specifically about exit to user.
Does this suffice?
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 08ed5a2e46a5..e842f27a1108 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -321,6 +321,7 @@
#endif
.endm
+/* Primarily used in exit-to-userspace path */
#define CLEAR_CPU_BUFFERS \
__CLEAR_CPU_BUFFERS X86_FEATURE_CLEAR_CPU_BUF
> This also seems like a good moment to update the comment on
> verw_clear_cpu_buf_mitigation_selected to mention the _VM flag too.
As we have 3 different flags, referring them with X86_FEATURE_CLEAR_CPU_*
should be okay?
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 723666a1357e..51dec95a9af5 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -490,7 +490,7 @@ static enum rfds_mitigations rfds_mitigation __ro_after_init =
/*
* Set if any of MDS/TAA/MMIO/RFDS are going to enable VERW clearing
- * through X86_FEATURE_CLEAR_CPU_BUF on kernel and guest entry.
+ * through X86_FEATURE_CLEAR_CPU_* on kernel and guest entry.
*/
static bool verw_clear_cpu_buf_mitigation_selected __ro_after_init;
> Also, where we set vmx->disable_fb_clear in vmx_update_fb_clear_dis(),
> it still refers to X86_FEATURE_CLEAR_CPU_BUF, is that wrong?
It looks correct to me. The only reason X86_FEATURE_CLEAR_CPU_BUF is used
in vmx_update_fb_clear_dis() is to check if host has enabled its
exit-to-userspace mitigation for some reason, and allow guest to also use
VERW.
© 2016 - 2026 Red Hat, Inc.