Explicitly filter out 64-bit exit codes when invoking exit handlers, as
svm_exit_handlers[] will never be sized with entries that use bits 63:32.

Processing the non-failing exit code as a 32-bit value will allow tracking
exit_code as a single 64-bit value (which it is, architecturally).  This
will also allow hardening KVM against Spectre-like attacks without needing
to do silly things to avoid build failures on 32-bit kernels
(array_index_nospec() rightly asserts that the index fits in an "unsigned
long").

Omit the check when running as a VM, as KVM has historically failed to set
bits 63:32 appropriately when synthesizing VM-Exits, i.e. KVM could get
false positives when running as a VM on an older, broken KVM/kernel.  From
a functional perspective, omitting the check is "fine", as any unwanted
collision between e.g. VMEXIT_INVALID and a 32-bit exit code will be
fatal to KVM-on-KVM regardless of what KVM-as-L1 does.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/svm/svm.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 202a4d8088a2..3b05476296d0 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -3433,8 +3433,22 @@ static void dump_vmcb(struct kvm_vcpu *vcpu)
 		sev_free_decrypted_vmsa(vcpu, save);
 }
 
-int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
+int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 __exit_code)
 {
+	u32 exit_code = __exit_code;
+
+	/*
+	 * SVM uses negative values, i.e. 64-bit values, to indicate that VMRUN
+	 * failed.  Report all such errors to userspace (note, VMEXIT_INVALID,
+	 * a.k.a. SVM_EXIT_ERR, is special cased by svm_handle_exit()).  Skip
+	 * the check when running as a VM, as KVM has historically left garbage
+	 * in bits 63:32, i.e. running KVM-on-KVM would hit false positives if
+	 * the underlying kernel is buggy.
+	 */
+	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR) &&
+	    (u64)exit_code != __exit_code)
+		goto unexpected_vmexit;
+
 #ifdef CONFIG_MITIGATION_RETPOLINE
 	if (exit_code == SVM_EXIT_MSR)
 		return msr_interception(vcpu);
@@ -3461,7 +3475,7 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
 
 unexpected_vmexit:
 	dump_vmcb(vcpu);
-	kvm_prepare_unexpected_reason_exit(vcpu, exit_code);
+	kvm_prepare_unexpected_reason_exit(vcpu, __exit_code);
 	return 0;
 }
 
-- 
2.52.0.rc1.455.g30608eb744-goog

On 11/13/25 23:56, Sean Christopherson wrote:
> Explicitly filter out 64-bit exit codes when invoking exit handlers, as
> svm_exit_handlers[] will never be sized with entries that use bits 63:32.
> 
> Processing the non-failing exit code as a 32-bit value will allow tracking
> exit_code as a single 64-bit value (which it is, architecturally).  This
> will also allow hardening KVM against Spectre-like attacks without needing
> to do silly things to avoid build failures on 32-bit kernels
> (array_index_nospec() rightly asserts that the index fits in an "unsigned
> long").
> 
> Omit the check when running as a VM, as KVM has historically failed to set
> bits 63:32 appropriately when synthesizing VM-Exits, i.e. KVM could get
> false positives when running as a VM on an older, broken KVM/kernel.  From
> a functional perspective, omitting the check is "fine", as any unwanted
> collision between e.g. VMEXIT_INVALID and a 32-bit exit code will be
> fatal to KVM-on-KVM regardless of what KVM-as-L1 does.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>   arch/x86/kvm/svm/svm.c | 18 ++++++++++++++++--
>   1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> index 202a4d8088a2..3b05476296d0 100644
> --- a/arch/x86/kvm/svm/svm.c
> +++ b/arch/x86/kvm/svm/svm.c
> @@ -3433,8 +3433,22 @@ static void dump_vmcb(struct kvm_vcpu *vcpu)
>   		sev_free_decrypted_vmsa(vcpu, save);
>   }
>   
> -int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
> +int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 __exit_code)
>   {
> +	u32 exit_code = __exit_code;
> +
> +	/*
> +	 * SVM uses negative values, i.e. 64-bit values, to indicate that VMRUN
> +	 * failed.  Report all such errors to userspace (note, VMEXIT_INVALID,
> +	 * a.k.a. SVM_EXIT_ERR, is special cased by svm_handle_exit()).  Skip
> +	 * the check when running as a VM, as KVM has historically left garbage
> +	 * in bits 63:32, i.e. running KVM-on-KVM would hit false positives if
> +	 * the underlying kernel is buggy.
> +	 */
> +	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR) &&
> +	    (u64)exit_code != __exit_code)
> +		goto unexpected_vmexit;

I reviewed the series and it looks good, but with respect to this patch 
and patch 8, is it really worth it?  While there is a possibility that 
code 0x00000000ffffffff is used, or that any high 32-bit values other 
than all-zeros or all-ones are used, they'd be presumably enabled by 
some control bits in the VMCB or some paravirt thing in the hypervisor.

What really matters is that SEV-ES's kvm_get_cached_sw_exit_code() is 
reading the full 64 bits and discarding invalid codes before reaching 
svm_invoke_exit_handler().

I totally agree, of course, with passing __exit_code as u64 and adding a 
comment explaining what's going on with "u32 exit_code == (u32)__exit_code".

Paolo

>   #ifdef CONFIG_MITIGATION_RETPOLINE
>   	if (exit_code == SVM_EXIT_MSR)
>   		return msr_interception(vcpu);
> @@ -3461,7 +3475,7 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
>   
>   unexpected_vmexit:
>   	dump_vmcb(vcpu);
> -	kvm_prepare_unexpected_reason_exit(vcpu, exit_code);
> +	kvm_prepare_unexpected_reason_exit(vcpu, __exit_code);
>   	return 0;
>   }
>

On Sat, Nov 15, 2025, Paolo Bonzini wrote:
> On 11/13/25 23:56, Sean Christopherson wrote:
> > Explicitly filter out 64-bit exit codes when invoking exit handlers, as
> > svm_exit_handlers[] will never be sized with entries that use bits 63:32.
> > 
> > Processing the non-failing exit code as a 32-bit value will allow tracking
> > exit_code as a single 64-bit value (which it is, architecturally).  This
> > will also allow hardening KVM against Spectre-like attacks without needing
> > to do silly things to avoid build failures on 32-bit kernels
> > (array_index_nospec() rightly asserts that the index fits in an "unsigned
> > long").
> > 
> > Omit the check when running as a VM, as KVM has historically failed to set
> > bits 63:32 appropriately when synthesizing VM-Exits, i.e. KVM could get
> > false positives when running as a VM on an older, broken KVM/kernel.  From
> > a functional perspective, omitting the check is "fine", as any unwanted
> > collision between e.g. VMEXIT_INVALID and a 32-bit exit code will be
> > fatal to KVM-on-KVM regardless of what KVM-as-L1 does.
> > 
> > Signed-off-by: Sean Christopherson <seanjc@google.com>
> > ---
> >   arch/x86/kvm/svm/svm.c | 18 ++++++++++++++++--
> >   1 file changed, 16 insertions(+), 2 deletions(-)
> > 
> > diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> > index 202a4d8088a2..3b05476296d0 100644
> > --- a/arch/x86/kvm/svm/svm.c
> > +++ b/arch/x86/kvm/svm/svm.c
> > @@ -3433,8 +3433,22 @@ static void dump_vmcb(struct kvm_vcpu *vcpu)
> >   		sev_free_decrypted_vmsa(vcpu, save);
> >   }
> > -int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
> > +int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 __exit_code)
> >   {
> > +	u32 exit_code = __exit_code;
> > +
> > +	/*
> > +	 * SVM uses negative values, i.e. 64-bit values, to indicate that VMRUN
> > +	 * failed.  Report all such errors to userspace (note, VMEXIT_INVALID,
> > +	 * a.k.a. SVM_EXIT_ERR, is special cased by svm_handle_exit()).  Skip
> > +	 * the check when running as a VM, as KVM has historically left garbage
> > +	 * in bits 63:32, i.e. running KVM-on-KVM would hit false positives if
> > +	 * the underlying kernel is buggy.
> > +	 */
> > +	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR) &&
> > +	    (u64)exit_code != __exit_code)
> > +		goto unexpected_vmexit;
> 
> I reviewed the series and it looks good, but with respect to this patch and
> patch 8, is it really worth it?  While there is a possibility that code
> 0x00000000ffffffff is used, or that any high 32-bit values other than
> all-zeros or all-ones are used, they'd be presumably enabled by some control
> bits in the VMCB or some paravirt thing in the hypervisor.

Maybe.  E.g. TDCALL and SEAMCALL VM-Exits on Intel show up without any enablement
in software (beyond VMXON).  I completely agree that it's extremely unlikely that
AMD will add a on-negative exit code with bits 63:32 != 0, i.e. that we could get
a false positive when truncating exit_code to a u32, but it also seems harmless
to be paranoid.

FWIW, I was assuming VMEXIT_INVALID_PMC (-4) was a generic vPMU thing, but it
looks like that one is also SEV-ES+ specific.

As for e57b84699534 ("KVM: SVM: Limit incorrect check on SVM_EXIT_ERR to running
as a VM"), I agree that being paranoid probably doesn't do anything in practice,
but I like being consistent. :-)

> What really matters is that SEV-ES's kvm_get_cached_sw_exit_code() is
> reading the full 64 bits and discarding invalid codes before reaching
> svm_invoke_exit_handler().

No?  sev_handle_vmgexit() only handles SVM_VMGEXIT_xxx exit codes, everything
else is punted to svm_invoke_exit_handler()

	exit_code = kvm_get_cached_sw_exit_code(control);
	switch (exit_code) {
	case SVM_VMGEXIT_<0>
	...
	case SVM_VMGEXIT_<N>
	default:
		ret = svm_invoke_exit_handler(vcpu, exit_code);
	}

And I don't see anything that filters/modifies exit_code_hi.

On Thu, Nov 13, 2025 at 02:56:18PM -0800, Sean Christopherson wrote:
> Explicitly filter out 64-bit exit codes when invoking exit handlers, as
> svm_exit_handlers[] will never be sized with entries that use bits 63:32.
> 
> Processing the non-failing exit code as a 32-bit value will allow tracking
> exit_code as a single 64-bit value (which it is, architecturally).  This
> will also allow hardening KVM against Spectre-like attacks without needing
> to do silly things to avoid build failures on 32-bit kernels
> (array_index_nospec() rightly asserts that the index fits in an "unsigned
> long").
> 
> Omit the check when running as a VM, as KVM has historically failed to set
> bits 63:32 appropriately when synthesizing VM-Exits, i.e. KVM could get
> false positives when running as a VM on an older, broken KVM/kernel.  From
> a functional perspective, omitting the check is "fine", as any unwanted
> collision between e.g. VMEXIT_INVALID and a 32-bit exit code will be
> fatal to KVM-on-KVM regardless of what KVM-as-L1 does.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>

Reviewed-by: Yosry Ahmed <yosry.ahmed@linux.dev>

> ---
>  arch/x86/kvm/svm/svm.c | 18 ++++++++++++++++--
>  1 file changed, 16 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
> index 202a4d8088a2..3b05476296d0 100644
> --- a/arch/x86/kvm/svm/svm.c
> +++ b/arch/x86/kvm/svm/svm.c
> @@ -3433,8 +3433,22 @@ static void dump_vmcb(struct kvm_vcpu *vcpu)
>  		sev_free_decrypted_vmsa(vcpu, save);
>  }
>  
> -int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
> +int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 __exit_code)
>  {
> +	u32 exit_code = __exit_code;
> +
> +	/*
> +	 * SVM uses negative values, i.e. 64-bit values, to indicate that VMRUN
> +	 * failed.  Report all such errors to userspace (note, VMEXIT_INVALID,
> +	 * a.k.a. SVM_EXIT_ERR, is special cased by svm_handle_exit()).  Skip
> +	 * the check when running as a VM, as KVM has historically left garbage
> +	 * in bits 63:32, i.e. running KVM-on-KVM would hit false positives if
> +	 * the underlying kernel is buggy.
> +	 */
> +	if (!cpu_feature_enabled(X86_FEATURE_HYPERVISOR) &&
> +	    (u64)exit_code != __exit_code)
> +		goto unexpected_vmexit;
> +
>  #ifdef CONFIG_MITIGATION_RETPOLINE
>  	if (exit_code == SVM_EXIT_MSR)
>  		return msr_interception(vcpu);
> @@ -3461,7 +3475,7 @@ int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code)
>  
>  unexpected_vmexit:
>  	dump_vmcb(vcpu);
> -	kvm_prepare_unexpected_reason_exit(vcpu, exit_code);
> +	kvm_prepare_unexpected_reason_exit(vcpu, __exit_code);
>  	return 0;
>  }
>  
> -- 
> 2.52.0.rc1.455.g30608eb744-goog
>