[RFC PATCH v5 089/104] KVM: TDX: Add a placeholder for handler of TDX hypercalls (TDG.VP.VMCALL)

isaku.yamahata@intel.com posted 104 patches 3 years, 9 months ago
There is a newer version of this series
[RFC PATCH v5 089/104] KVM: TDX: Add a placeholder for handler of TDX hypercalls (TDG.VP.VMCALL)
Posted by isaku.yamahata@intel.com 3 years, 9 months ago
From: Sean Christopherson <sean.j.christopherson@intel.com>

The TDX module specification defines TDG.VP.VMCALL API (TDVMCALL for short)
for the guest TD to call hypercall to VMM.  When the guest TD issues
TDG.VP.VMCALL, the guest TD exits to VMM with a new exit reason of
TDVMCALL.  The arguments from the guest TD and returned values from the VMM
are passed in the guest registers.  The guest RCX registers indicates which
registers are used.

Define the TDVMCALL exit reason, which is carved out from the VMX exit
reason namespace as the TDVMCALL exit from TDX guest to TDX-SEAM is really
just a VM-Exit.  Add a place holder to handle TDVMCALL exit.

Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/include/uapi/asm/vmx.h |  4 +++-
 arch/x86/kvm/vmx/tdx.c          | 27 ++++++++++++++++++++++++++-
 arch/x86/kvm/vmx/tdx.h          | 13 +++++++++++++
 3 files changed, 42 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/uapi/asm/vmx.h b/arch/x86/include/uapi/asm/vmx.h
index 3d9b4598e166..cb0a0565219a 100644
--- a/arch/x86/include/uapi/asm/vmx.h
+++ b/arch/x86/include/uapi/asm/vmx.h
@@ -92,6 +92,7 @@
 #define EXIT_REASON_UMWAIT              67
 #define EXIT_REASON_TPAUSE              68
 #define EXIT_REASON_BUS_LOCK            74
+#define EXIT_REASON_TDCALL              77
 
 #define VMX_EXIT_REASONS \
 	{ EXIT_REASON_EXCEPTION_NMI,         "EXCEPTION_NMI" }, \
@@ -154,7 +155,8 @@
 	{ EXIT_REASON_XRSTORS,               "XRSTORS" }, \
 	{ EXIT_REASON_UMWAIT,                "UMWAIT" }, \
 	{ EXIT_REASON_TPAUSE,                "TPAUSE" }, \
-	{ EXIT_REASON_BUS_LOCK,              "BUS_LOCK" }
+	{ EXIT_REASON_BUS_LOCK,              "BUS_LOCK" }, \
+	{ EXIT_REASON_TDCALL,                "TDCALL" }
 
 #define VMX_EXIT_REASON_FLAGS \
 	{ VMX_EXIT_REASONS_FAILED_VMENTRY,	"FAILED_VMENTRY" }
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 8695836ce796..86daafd9eec0 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -780,7 +780,8 @@ static noinstr void tdx_vcpu_enter_exit(struct kvm_vcpu *vcpu,
 					struct vcpu_tdx *tdx)
 {
 	guest_enter_irqoff();
-	tdx->exit_reason.full = __tdx_vcpu_run(tdx->tdvpr.pa, vcpu->arch.regs, 0);
+	tdx->exit_reason.full = __tdx_vcpu_run(tdx->tdvpr.pa, vcpu->arch.regs,
+					tdx->tdvmcall.regs_mask);
 	guest_exit_irqoff();
 }
 
@@ -815,6 +816,11 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu)
 
 	if (tdx->exit_reason.error || tdx->exit_reason.non_recoverable)
 		return EXIT_FASTPATH_NONE;
+
+	if (tdx->exit_reason.basic == EXIT_REASON_TDCALL)
+		tdx->tdvmcall.rcx = vcpu->arch.regs[VCPU_REGS_RCX];
+	else
+		tdx->tdvmcall.rcx = 0;
 	return EXIT_FASTPATH_NONE;
 }
 
@@ -859,6 +865,23 @@ static int tdx_handle_triple_fault(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
+static int handle_tdvmcall(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_tdx *tdx = to_tdx(vcpu);
+
+	if (unlikely(tdx->tdvmcall.xmm_mask))
+		goto unsupported;
+
+	switch (tdvmcall_exit_reason(vcpu)) {
+	default:
+		break;
+	}
+
+unsupported:
+	tdvmcall_set_return_code(vcpu, TDG_VP_VMCALL_INVALID_OPERAND);
+	return 1;
+}
+
 void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level)
 {
 	td_vmcs_write64(to_tdx(vcpu), SHARED_EPT_POINTER, root_hpa & PAGE_MASK);
@@ -1187,6 +1210,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath)
 		return tdx_handle_exception(vcpu);
 	case EXIT_REASON_EXTERNAL_INTERRUPT:
 		return tdx_handle_external_interrupt(vcpu);
+	case EXIT_REASON_TDCALL:
+		return handle_tdvmcall(vcpu);
 	case EXIT_REASON_EPT_VIOLATION:
 		return tdx_handle_ept_violation(vcpu);
 	case EXIT_REASON_EPT_MISCONFIG:
diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h
index 7cd81780f3fa..9e8ed9b3119e 100644
--- a/arch/x86/kvm/vmx/tdx.h
+++ b/arch/x86/kvm/vmx/tdx.h
@@ -86,6 +86,19 @@ struct vcpu_tdx {
 	/* Posted interrupt descriptor */
 	struct pi_desc pi_desc;
 
+	union {
+		struct {
+			union {
+				struct {
+					u16 gpr_mask;
+					u16 xmm_mask;
+				};
+				u32 regs_mask;
+			};
+			u32 reserved;
+		};
+		u64 rcx;
+	} tdvmcall;
 	union tdx_exit_reason exit_reason;
 
 	bool initialized;
-- 
2.25.1
Re: [RFC PATCH v5 089/104] KVM: TDX: Add a placeholder for handler of TDX hypercalls (TDG.VP.VMCALL)
Posted by Kai Huang 3 years, 8 months ago
On Fri, 2022-03-04 at 11:49 -0800, isaku.yamahata@intel.com wrote:
> From: Sean Christopherson <sean.j.christopherson@intel.com>
> 
> The TDX module specification defines TDG.VP.VMCALL API (TDVMCALL for short)
> for the guest TD to call hypercall to VMM.  When the guest TD issues
> TDG.VP.VMCALL, the guest TD exits to VMM with a new exit reason of
> TDVMCALL.  The arguments from the guest TD and returned values from the VMM
> are passed in the guest registers.  The guest RCX registers indicates which
> registers are used.
> 
> Define the TDVMCALL exit reason, which is carved out from the VMX exit
> reason namespace as the TDVMCALL exit from TDX guest to TDX-SEAM is really
> just a VM-Exit.  Add a place holder to handle TDVMCALL exit.
> 
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> ---
>  arch/x86/include/uapi/asm/vmx.h |  4 +++-
>  arch/x86/kvm/vmx/tdx.c          | 27 ++++++++++++++++++++++++++-
>  arch/x86/kvm/vmx/tdx.h          | 13 +++++++++++++
>  3 files changed, 42 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/include/uapi/asm/vmx.h b/arch/x86/include/uapi/asm/vmx.h
> index 3d9b4598e166..cb0a0565219a 100644
> --- a/arch/x86/include/uapi/asm/vmx.h
> +++ b/arch/x86/include/uapi/asm/vmx.h
> @@ -92,6 +92,7 @@
>  #define EXIT_REASON_UMWAIT              67
>  #define EXIT_REASON_TPAUSE              68
>  #define EXIT_REASON_BUS_LOCK            74
> +#define EXIT_REASON_TDCALL              77
>  
>  #define VMX_EXIT_REASONS \
>  	{ EXIT_REASON_EXCEPTION_NMI,         "EXCEPTION_NMI" }, \
> @@ -154,7 +155,8 @@
>  	{ EXIT_REASON_XRSTORS,               "XRSTORS" }, \
>  	{ EXIT_REASON_UMWAIT,                "UMWAIT" }, \
>  	{ EXIT_REASON_TPAUSE,                "TPAUSE" }, \
> -	{ EXIT_REASON_BUS_LOCK,              "BUS_LOCK" }
> +	{ EXIT_REASON_BUS_LOCK,              "BUS_LOCK" }, \
> +	{ EXIT_REASON_TDCALL,                "TDCALL" }
>  
>  #define VMX_EXIT_REASON_FLAGS \
>  	{ VMX_EXIT_REASONS_FAILED_VMENTRY,	"FAILED_VMENTRY" }
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index 8695836ce796..86daafd9eec0 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -780,7 +780,8 @@ static noinstr void tdx_vcpu_enter_exit(struct kvm_vcpu *vcpu,
>  					struct vcpu_tdx *tdx)
>  {
>  	guest_enter_irqoff();
> -	tdx->exit_reason.full = __tdx_vcpu_run(tdx->tdvpr.pa, vcpu->arch.regs, 0);
> +	tdx->exit_reason.full = __tdx_vcpu_run(tdx->tdvpr.pa, vcpu->arch.regs,
> +					tdx->tdvmcall.regs_mask);
>  	guest_exit_irqoff();
>  }
>  
> @@ -815,6 +816,11 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu)
>  
>  	if (tdx->exit_reason.error || tdx->exit_reason.non_recoverable)
>  		return EXIT_FASTPATH_NONE;
> +
> +	if (tdx->exit_reason.basic == EXIT_REASON_TDCALL)
> +		tdx->tdvmcall.rcx = vcpu->arch.regs[VCPU_REGS_RCX];
> +	else
> +		tdx->tdvmcall.rcx = 0;
>  	return EXIT_FASTPATH_NONE;
>  }
>  
> @@ -859,6 +865,23 @@ static int tdx_handle_triple_fault(struct kvm_vcpu *vcpu)
>  	return 0;
>  }
>  
> +static int handle_tdvmcall(struct kvm_vcpu *vcpu)
> +{
> +	struct vcpu_tdx *tdx = to_tdx(vcpu);
> +
> +	if (unlikely(tdx->tdvmcall.xmm_mask))
> +		goto unsupported;

Put a comment explaining this logic?

> +
> +	switch (tdvmcall_exit_reason(vcpu)) {

Could we rename tdxvmcall_exit_reason() to something like tdvmcall_leaf()?

Btw, why couldn't we merge previous patch to this one, so we don't have to look
back and forth to figure out exactly what do those functions do?


> +	default:
> +		break;
> +	}
> +
> +unsupported:
> +	tdvmcall_set_return_code(vcpu, TDG_VP_VMCALL_INVALID_OPERAND);
> +	return 1;
> +}
> +
>  void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level)
>  {
>  	td_vmcs_write64(to_tdx(vcpu), SHARED_EPT_POINTER, root_hpa & PAGE_MASK);
> @@ -1187,6 +1210,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath)
>  		return tdx_handle_exception(vcpu);
>  	case EXIT_REASON_EXTERNAL_INTERRUPT:
>  		return tdx_handle_external_interrupt(vcpu);
> +	case EXIT_REASON_TDCALL:
> +		return handle_tdvmcall(vcpu);
>  	case EXIT_REASON_EPT_VIOLATION:
>  		return tdx_handle_ept_violation(vcpu);
>  	case EXIT_REASON_EPT_MISCONFIG:
> diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h
> index 7cd81780f3fa..9e8ed9b3119e 100644
> --- a/arch/x86/kvm/vmx/tdx.h
> +++ b/arch/x86/kvm/vmx/tdx.h
> @@ -86,6 +86,19 @@ struct vcpu_tdx {
>  	/* Posted interrupt descriptor */
>  	struct pi_desc pi_desc;
>  
> +	union {
> +		struct {
> +			union {
> +				struct {
> +					u16 gpr_mask;
> +					u16 xmm_mask;
> +				};
> +				u32 regs_mask;
> +			};
> +			u32 reserved;
> +		};
> +		u64 rcx;
> +	} tdvmcall;
>  	union tdx_exit_reason exit_reason;
>  
>  	bool initialized;
Re: [RFC PATCH v5 089/104] KVM: TDX: Add a placeholder for handler of TDX hypercalls (TDG.VP.VMCALL)
Posted by Paolo Bonzini 3 years, 8 months ago
On 4/7/22 06:15, Kai Huang wrote:
>>   
>> +static int handle_tdvmcall(struct kvm_vcpu *vcpu)
>> +{
>> +	struct vcpu_tdx *tdx = to_tdx(vcpu);
>> +
>> +	if (unlikely(tdx->tdvmcall.xmm_mask))
>> +		goto unsupported;
> Put a comment explaining this logic?
> 

This only seems to be necessary for Hyper-V hypercalls, which however 
are not supported by this series in TDX guests (because the 
kvm_hv_hypercall still calls kvm_*_read, likewise for Xen).

So for now this conditional can be dropped.

Since
Re: [RFC PATCH v5 089/104] KVM: TDX: Add a placeholder for handler of TDX hypercalls (TDG.VP.VMCALL)
Posted by Sean Christopherson 3 years, 8 months ago
On Thu, Apr 07, 2022, Paolo Bonzini wrote:
> On 4/7/22 06:15, Kai Huang wrote:
> > > +static int handle_tdvmcall(struct kvm_vcpu *vcpu)
> > > +{
> > > +	struct vcpu_tdx *tdx = to_tdx(vcpu);
> > > +
> > > +	if (unlikely(tdx->tdvmcall.xmm_mask))
> > > +		goto unsupported;
> > Put a comment explaining this logic?
> > 
> 
> This only seems to be necessary for Hyper-V hypercalls, which however are
> not supported by this series in TDX guests (because the kvm_hv_hypercall
> still calls kvm_*_read, likewise for Xen).
> 
> So for now this conditional can be dropped.

I'd prefer to keep the sanity check, it's a cheap and easy way to detect a clear
cut guest bug.  E.g. KVM would be within its rights to write garbage the XMM
registers in this case.  Even though KVM isn't to be trusted, KVM can still be
nice to the guest.
Re: [RFC PATCH v5 089/104] KVM: TDX: Add a placeholder for handler of TDX hypercalls (TDG.VP.VMCALL)
Posted by Paolo Bonzini 3 years, 8 months ago
On 4/7/22 16:39, Sean Christopherson wrote:
> On Thu, Apr 07, 2022, Paolo Bonzini wrote:
>> On 4/7/22 06:15, Kai Huang wrote:
>>>> +static int handle_tdvmcall(struct kvm_vcpu *vcpu)
>>>> +{
>>>> +	struct vcpu_tdx *tdx = to_tdx(vcpu);
>>>> +
>>>> +	if (unlikely(tdx->tdvmcall.xmm_mask))
>>>> +		goto unsupported;
>>> Put a comment explaining this logic?
>>>
>>
>> This only seems to be necessary for Hyper-V hypercalls, which however are
>> not supported by this series in TDX guests (because the kvm_hv_hypercall
>> still calls kvm_*_read, likewise for Xen).
>>
>> So for now this conditional can be dropped.
> 
> I'd prefer to keep the sanity check, it's a cheap and easy way to detect a clear
> cut guest bug.

I don't think it's necessarily a guest bug, just silly but valid behavior.

Paolo

   E.g. KVM would be within its rights to write garbage the XMM
> registers in this case.  Even though KVM isn't to be trusted, KVM can still be
> nice to the guest.
Re: [RFC PATCH v5 089/104] KVM: TDX: Add a placeholder for handler of TDX hypercalls (TDG.VP.VMCALL)
Posted by Sean Christopherson 3 years, 8 months ago
On Thu, Apr 07, 2022, Paolo Bonzini wrote:
> On 4/7/22 16:39, Sean Christopherson wrote:
> > On Thu, Apr 07, 2022, Paolo Bonzini wrote:
> > > On 4/7/22 06:15, Kai Huang wrote:
> > > > > +static int handle_tdvmcall(struct kvm_vcpu *vcpu)
> > > > > +{
> > > > > +	struct vcpu_tdx *tdx = to_tdx(vcpu);
> > > > > +
> > > > > +	if (unlikely(tdx->tdvmcall.xmm_mask))
> > > > > +		goto unsupported;
> > > > Put a comment explaining this logic?
> > > > 
> > > 
> > > This only seems to be necessary for Hyper-V hypercalls, which however are
> > > not supported by this series in TDX guests (because the kvm_hv_hypercall
> > > still calls kvm_*_read, likewise for Xen).
> > > 
> > > So for now this conditional can be dropped.
> > 
> > I'd prefer to keep the sanity check, it's a cheap and easy way to detect a clear
> > cut guest bug.
> 
> I don't think it's necessarily a guest bug, just silly but valid behavior.

It's a bug from a security perspective given that letting the host unnecessarily
manipulate register state is an exploit waiting to happen.

Though for KVM to reject the TDVMCALLs, the GHCI should really be updated to state
that exposing more state than is required _may_ be considered invalid ("may" so
that KVM isn't required to check the mask on every exit, which IMO is beyond tedious).
Re: [RFC PATCH v5 089/104] KVM: TDX: Add a placeholder for handler of TDX hypercalls (TDG.VP.VMCALL)
Posted by Kai Huang 3 years, 8 months ago
On Thu, 2022-04-07 at 18:11 +0000, Sean Christopherson wrote:
> On Thu, Apr 07, 2022, Paolo Bonzini wrote:
> > On 4/7/22 16:39, Sean Christopherson wrote:
> > > On Thu, Apr 07, 2022, Paolo Bonzini wrote:
> > > > On 4/7/22 06:15, Kai Huang wrote:
> > > > > > +static int handle_tdvmcall(struct kvm_vcpu *vcpu)
> > > > > > +{
> > > > > > +	struct vcpu_tdx *tdx = to_tdx(vcpu);
> > > > > > +
> > > > > > +	if (unlikely(tdx->tdvmcall.xmm_mask))
> > > > > > +		goto unsupported;
> > > > > Put a comment explaining this logic?
> > > > > 
> > > > 
> > > > This only seems to be necessary for Hyper-V hypercalls, which however are
> > > > not supported by this series in TDX guests (because the kvm_hv_hypercall
> > > > still calls kvm_*_read, likewise for Xen).
> > > > 
> > > > So for now this conditional can be dropped.
> > > 
> > > I'd prefer to keep the sanity check, it's a cheap and easy way to detect a clear
> > > cut guest bug.
> > 
> > I don't think it's necessarily a guest bug, just silly but valid behavior.
> 
> It's a bug from a security perspective given that letting the host unnecessarily
> manipulate register state is an exploit waiting to happen.

Security perspective from guest's or host's respective?  It's guest's
responsibility to make sure itself doesn't expose unnecessarily security holes.
In this particular case, if guest does, then it's guest's business, but I don't
see how it can compromise host's security, or other VM's security?

As Paolo said it's a valid operation from guest, perhaps an alternative is KVM
can unconditionally clear XMM registers, instead of rejecting this VMCALL.

> 
> Though for KVM to reject the TDVMCALLs, the GHCI should really be updated to state
> that exposing more state than is required _may_ be considered invalid ("may" so
> that KVM isn't required to check the mask on every exit, which IMO is beyond tedious).

Independent from this issue,  I guess it's good for GHCI to have this anyway.