[PATCH v16 22/51] KVM: x86/mmu: Pretty print PK, SS, and SGX flags in MMU tracepoints

Sean Christopherson posted 51 patches 1 week, 5 days ago
[PATCH v16 22/51] KVM: x86/mmu: Pretty print PK, SS, and SGX flags in MMU tracepoints
Posted by Sean Christopherson 1 week, 5 days ago
Add PK (Protection Keys), SS (Shadow Stacks), and SGX (Software Guard
Extensions) to the set of #PF error flags handled via
kvm_mmu_trace_pferr_flags.  While KVM doesn't expect PK or SS #PFs in
particular, pretty print their names instead of the raw hex value saves
the user from having to go spelunking in the SDM to figure out what's
going on.

Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/kvm/mmu/mmutrace.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h
index f35a830ce469..764e3015d021 100644
--- a/arch/x86/kvm/mmu/mmutrace.h
+++ b/arch/x86/kvm/mmu/mmutrace.h
@@ -51,6 +51,9 @@
 	{ PFERR_PRESENT_MASK, "P" },	\
 	{ PFERR_WRITE_MASK, "W" },	\
 	{ PFERR_USER_MASK, "U" },	\
+	{ PFERR_PK_MASK, "PK" },	\
+	{ PFERR_SS_MASK, "SS" },	\
+	{ PFERR_SGX_MASK, "SGX" },	\
 	{ PFERR_RSVD_MASK, "RSVD" },	\
 	{ PFERR_FETCH_MASK, "F" }
 
-- 
2.51.0.470.ga7dc726c21-goog
Re: [PATCH v16 22/51] KVM: x86/mmu: Pretty print PK, SS, and SGX flags in MMU tracepoints
Posted by Xiaoyao Li 1 week, 1 day ago
On 9/20/2025 6:32 AM, Sean Christopherson wrote:
> Add PK (Protection Keys), SS (Shadow Stacks), and SGX (Software Guard
> Extensions) to the set of #PF error flags handled via
> kvm_mmu_trace_pferr_flags.  While KVM doesn't expect PK or SS #PFs in
> particular, pretty print their names instead of the raw hex value saves
> the user from having to go spelunking in the SDM to figure out what's
> going on.
> 
> Signed-off-by: Sean Christopherson <seanjc@google.com>

Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>

> ---
>   arch/x86/kvm/mmu/mmutrace.h | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h
> index f35a830ce469..764e3015d021 100644
> --- a/arch/x86/kvm/mmu/mmutrace.h
> +++ b/arch/x86/kvm/mmu/mmutrace.h
> @@ -51,6 +51,9 @@
>   	{ PFERR_PRESENT_MASK, "P" },	\
>   	{ PFERR_WRITE_MASK, "W" },	\
>   	{ PFERR_USER_MASK, "U" },	\
> +	{ PFERR_PK_MASK, "PK" },	\
> +	{ PFERR_SS_MASK, "SS" },	\
> +	{ PFERR_SGX_MASK, "SGX" },	\
>   	{ PFERR_RSVD_MASK, "RSVD" },	\
>   	{ PFERR_FETCH_MASK, "F" }
>
Re: [PATCH v16 22/51] KVM: x86/mmu: Pretty print PK, SS, and SGX flags in MMU tracepoints
Posted by Binbin Wu 1 week, 2 days ago

On 9/20/2025 6:32 AM, Sean Christopherson wrote:
> Add PK (Protection Keys), SS (Shadow Stacks), and SGX (Software Guard
> Extensions) to the set of #PF error flags handled via
> kvm_mmu_trace_pferr_flags.  While KVM doesn't expect PK or SS #PFs
Also SGX.

> in
> particular, pretty print their names instead of the raw hex value saves
> the user from having to go spelunking in the SDM to figure out what's
> going on.
>
> Signed-off-by: Sean Christopherson <seanjc@google.com>

Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>

> ---
>   arch/x86/kvm/mmu/mmutrace.h | 3 +++
>   1 file changed, 3 insertions(+)
>
> diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h
> index f35a830ce469..764e3015d021 100644
> --- a/arch/x86/kvm/mmu/mmutrace.h
> +++ b/arch/x86/kvm/mmu/mmutrace.h
> @@ -51,6 +51,9 @@
>   	{ PFERR_PRESENT_MASK, "P" },	\
>   	{ PFERR_WRITE_MASK, "W" },	\
>   	{ PFERR_USER_MASK, "U" },	\
> +	{ PFERR_PK_MASK, "PK" },	\
> +	{ PFERR_SS_MASK, "SS" },	\
> +	{ PFERR_SGX_MASK, "SGX" },	\
>   	{ PFERR_RSVD_MASK, "RSVD" },	\
>   	{ PFERR_FETCH_MASK, "F" }
>
Re: [PATCH v16 22/51] KVM: x86/mmu: Pretty print PK, SS, and SGX flags in MMU tracepoints
Posted by Sean Christopherson 1 week, 2 days ago
On Mon, Sep 22, 2025, Binbin Wu wrote:
> 
> 
> On 9/20/2025 6:32 AM, Sean Christopherson wrote:
> > Add PK (Protection Keys), SS (Shadow Stacks), and SGX (Software Guard
> > Extensions) to the set of #PF error flags handled via
> > kvm_mmu_trace_pferr_flags.  While KVM doesn't expect PK or SS #PFs
> Also SGX.

Huh.  I deliberately omitted SGX from this particular statement, as KVM supports
SGX virtualization with shadow paging.  I.e. KVM "expects" PFERR_SGX in the sense
that an EPCM violation on SGX2 hardware will show up in KVM.

Typing that out made me realize that, unless I'm forgetting/missing code, KVM
doesn't actually do the right thing with respect to intercepted #PFs with PFERR_SGX.
On SGX2 hardware, an EPCM permissions violation will trigger a #PF(SGX).  KVM isn't
aware that such exceptions effectively have nothing to do with software-visibile
page tables.  And so I'm pretty sure an EPCM violation on SGX2 hardware would put
the vCPU into an infinite loop due to KVM not realizing the #PF (ugh, or #GP if
the guest CPU model is only SGX1) should be injected into the guest, (KVM will
think the fault is spurious).

To fix that, we'd need something like the below (completely untested).  But for
this patch, the changelog is "correct", i.e. observing SGX #PFs shouldn't be
impossible.

diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 08845c1d7a62..99cc790615fd 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -5175,12 +5175,52 @@ static bool is_xfd_nm_fault(struct kvm_vcpu *vcpu)
               !kvm_is_cr0_bit_set(vcpu, X86_CR0_TS);
 }
 
+static int vmx_handle_page_fault(struct kvm_vcpu *vcpu, u32 error_code)
+{
+       unsigned long cr2 = vmx_get_exit_qual(vcpu);
+
+       if (vcpu->arch.apf.host_apf_flags)
+               goto handle_pf;
+
+       /* When using EPT, KVM intercepts #PF only to detect illegal GPAs. */
+       WARN_ON_ONCE(enable_ept && !allow_smaller_maxphyaddr);
+
+       /*
+        * On SGX2 hardware, EPCM violations are delivered as #PF with the SGX
+        * flag set in the error code (SGX1 harware generates #GP(0)).  EPCM
+        * violations have nothing to do with shadow paging and can never be
+        * resolved by KVM; always reflect them into the guest.
+        */
+       if (error_code & PFERR_SGX_MASK) {
+               WARN_ON_ONCE(!IS_ENABLED_(CONFIG_X86_SGX_KVM) ||
+                            !cpu_feature_enabled(X86_FEATURE_SGX2));
+               if (guest_cpu_cap_has(vcpu, X86_FEATURE_SGX2))
+                       kvm_fixup_and_inject_pf_error(vcpu, cr2, error_code);
+               else
+                       kvm_inject_gp(vcpu, 0);
+               return 1;
+       }
+
+       /*
+        * If EPT is enabled, fixup and inject the #PF.  KVM intercepts #PFs
+        * only to set PFERR_RSVD as appropriate (hardware won't set RSVD due
+        * to the GPA being legal with respect to host.MAXPHYADDR).
+        */
+       if (enable_ept) {
+               kvm_fixup_and_inject_pf_error(vcpu, cr2, error_code);
+               return 1;
+       }
+
+handle_pf:
+       return kvm_handle_page_fault(vcpu, error_code, cr2, NULL, 0);
+}
+
 static int handle_exception_nmi(struct kvm_vcpu *vcpu)
 {
        struct vcpu_vmx *vmx = to_vmx(vcpu);
        struct kvm_run *kvm_run = vcpu->run;
        u32 intr_info, ex_no, error_code;
-       unsigned long cr2, dr6;
+       unsigned long dr6;
        u32 vect_info;
 
        vect_info = vmx->idt_vectoring_info;
@@ -5255,19 +5295,8 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu)
                return 0;
        }
 
-       if (is_page_fault(intr_info)) {
-               cr2 = vmx_get_exit_qual(vcpu);
-               if (enable_ept && !vcpu->arch.apf.host_apf_flags) {
-                       /*
-                        * EPT will cause page fault only if we need to
-                        * detect illegal GPAs.
-                        */
-                       WARN_ON_ONCE(!allow_smaller_maxphyaddr);
-                       kvm_fixup_and_inject_pf_error(vcpu, cr2, error_code);
-                       return 1;
-               } else
-                       return kvm_handle_page_fault(vcpu, error_code, cr2, NULL, 0);
-       }
+       if (is_page_fault(intr_info))
+               return vmx_handle_page_fault(vcpu, error_code);
 
        ex_no = intr_info & INTR_INFO_VECTOR_MASK;