[PATCH 3/3] KVM: TDX: Exit to userspace for GetTdVmCallInfo

Paolo Bonzini posted 3 patches 3 months, 3 weeks ago
There is a newer version of this series
[PATCH 3/3] KVM: TDX: Exit to userspace for GetTdVmCallInfo
Posted by Paolo Bonzini 3 months, 3 weeks ago
From: Binbin Wu <binbin.wu@linux.intel.com>

Exit to userspace for TDG.VP.VMCALL<GetTdVmCallInfo> via KVM_EXIT_TDX,
to allow userspace to provide information about the support of
TDVMCALLs when r12 is 1 for the TDVMCALLs beyond the GHCI base API.

GHCI spec defines the GHCI base TDVMCALLs: <GetTdVmCallInfo>, <MapGPA>,
<ReportFatalError>, <Instruction.CPUID>, <#VE.RequestMMIO>,
<Instruction.HLT>, <Instruction.IO>, <Instruction.RDMSR> and
<Instruction.WRMSR>. They must be supported by VMM to support TDX guests.

For GetTdVmCallInfo
- When leaf (r12) to enumerate TDVMCALL functionality is set to 0,
  successful execution indicates all GHCI base TDVMCALLs listed above are
  supported.

  Update the KVM TDX document with the set of the GHCI base APIs.

- When leaf (r12) to enumerate TDVMCALL functionality is set to 1, it
  indicates the TDX guest is querying the supported TDVMCALLs beyond
  the GHCI base TDVMCALLs.
  Exit to userspace to let userspace set the TDVMCALL sub-function bit(s)
  accordingly to the leaf outputs.  KVM could set the TDVMCALL bit(s)
  supported by itself when the TDVMCALLs don't need support from userspace
  after returning from userspace and before entering guest. Currently, no
  such TDVMCALLs implemented, KVM just sets the values returned from
  userspace.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
[Adjust userspace API. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 Documentation/virt/kvm/api.rst | 15 +++++++++++++-
 arch/x86/kvm/vmx/tdx.c         | 38 ++++++++++++++++++++++++++++++----
 include/uapi/linux/kvm.h       |  5 +++++
 3 files changed, 53 insertions(+), 5 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 3643d853a634..2b1656907356 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -7190,6 +7190,11 @@ The valid value for 'flags' is:
 					u64 gpa;
 					u64 size;
 				} get_quote;
+				struct {
+					u64 ret;
+					u64 leaf;
+					u64 r11, r12, r13, r14;
+				} get_tdvmcall_info;
 			};
 		} tdx;

@@ -7216,9 +7221,17 @@ queued successfully, the TDX guest can poll the status field in the
 shared-memory area to check whether the Quote generation is completed or
 not. When completed, the generated Quote is returned via the same buffer.
 
+* ``TDVMCALL_GET_TD_VM_CALL_INFO``: the guest has requested the support
+status of TDVMCALLs.  The output values for the given leaf should be
+placed in fields from ``r11`` to ``r14`` of the ``get_tdvmcall_info``
+field of the union.  This TDVMCALL must succeed, therefore KVM leaves
+``ret`` equal to ``TDVMCALL_STATUS_SUCCESS`` and ``r11`` to ``r14``
+equal to zero on entry.
+
 KVM may add support for more values in the future that may cause a userspace
 exit, even without calls to ``KVM_ENABLE_CAP`` or similar.  In this case,
-it will enter with output fields already valid; in the common case, the
+it will enter with output fields already valid as mentioned for
+``TDVMCALL_GET_TD_VM_CALL_INFO`` above; in the common case, the
 ``unknown.ret`` field of the union will be ``TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED``.
 Userspace need not do anything if it does not wish to support a TDVMCALL.
 ::
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 6878a76069f8..5804d1b1ea0e 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1451,18 +1451,49 @@ static int tdx_emulate_mmio(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+static int tdx_complete_get_td_vm_call_info(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_tdx *tdx = to_tdx(vcpu);
+
+	tdvmcall_set_return_code(vcpu, vcpu->run->tdx.get_tdvmcall_info.ret);
+
+	/*
+	 * For now, there is no TDVMCALL beyond GHCI base API supported by KVM
+	 * directly without the support from userspace, just set the value
+	 * returned from userspace.
+	 */
+	tdx->vp_enter_args.r11 = vcpu->run->tdx.get_tdvmcall_info.r11;
+	tdx->vp_enter_args.r12 = vcpu->run->tdx.get_tdvmcall_info.r12;
+	tdx->vp_enter_args.r13 = vcpu->run->tdx.get_tdvmcall_info.r13;
+	tdx->vp_enter_args.r14 = vcpu->run->tdx.get_tdvmcall_info.r14;
+
+	return 1;
+}
+
 static int tdx_get_td_vm_call_info(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
 
-	if (tdx->vp_enter_args.r12)
-		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
-	else {
+	switch (tdx->vp_enter_args.r12) {
+	case 1:
+		vcpu->run->tdx.get_tdvmcall_info.leaf = tdx->vp_enter_args.r12;
+		vcpu->run->exit_reason = KVM_EXIT_TDX;
+		vcpu->run->tdx.flags = 0;
+		vcpu->run->tdx.nr = TDVMCALL_GET_TD_VM_CALL_INFO;
+		vcpu->run->tdx.get_tdvmcall_info.ret = TDVMCALL_STATUS_SUCCESS;
+		vcpu->run->tdx.get_tdvmcall_info.r11 = 0;
+		vcpu->run->tdx.get_tdvmcall_info.r12 = 0;
+		vcpu->run->tdx.get_tdvmcall_info.r13 = 0;
+		vcpu->run->tdx.get_tdvmcall_info.r14 = 0;
+		vcpu->arch.complete_userspace_io = tdx_complete_get_td_vm_call_info;
+		return 0;
+	default:
 		tdx->vp_enter_args.r11 = 0;
+		tdx->vp_enter_args.r12 = 0;
 		tdx->vp_enter_args.r13 = 0;
 		tdx->vp_enter_args.r14 = 0;
+		return 1;
 	}
-	return 1;
 }
 
 static int tdx_complete_simple(struct kvm_vcpu *vcpu)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 6708bc88ae69..fb3b4cd8d662 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -461,6 +461,11 @@ struct kvm_run {
 					__u64 gpa;
 					__u64 size;
 				} get_quote;
+				struct {
+					__u64 ret;
+					__u64 leaf;
+					__u64 r11, r12, r13, r14;
+				} get_tdvmcall_info;
 			};
 		} tdx;
 		/* Fix the size of the union. */
-- 
2.43.5
Re: [PATCH 3/3] KVM: TDX: Exit to userspace for GetTdVmCallInfo
Posted by Xiaoyao Li 3 months, 3 weeks ago
On 6/20/2025 2:01 AM, Paolo Bonzini wrote:
> From: Binbin Wu <binbin.wu@linux.intel.com>
> 
> Exit to userspace for TDG.VP.VMCALL<GetTdVmCallInfo> via KVM_EXIT_TDX,
> to allow userspace to provide information about the support of
> TDVMCALLs when r12 is 1 for the TDVMCALLs beyond the GHCI base API.
> 
> GHCI spec defines the GHCI base TDVMCALLs: <GetTdVmCallInfo>, <MapGPA>,
> <ReportFatalError>, <Instruction.CPUID>, <#VE.RequestMMIO>,
> <Instruction.HLT>, <Instruction.IO>, <Instruction.RDMSR> and
> <Instruction.WRMSR>. They must be supported by VMM to support TDX guests.
> 
> For GetTdVmCallInfo
> - When leaf (r12) to enumerate TDVMCALL functionality is set to 0,
>    successful execution indicates all GHCI base TDVMCALLs listed above are
>    supported.
> 
>    Update the KVM TDX document with the set of the GHCI base APIs.
> 
> - When leaf (r12) to enumerate TDVMCALL functionality is set to 1, it
>    indicates the TDX guest is querying the supported TDVMCALLs beyond
>    the GHCI base TDVMCALLs.
>    Exit to userspace to let userspace set the TDVMCALL sub-function bit(s)
>    accordingly to the leaf outputs.  KVM could set the TDVMCALL bit(s)
>    supported by itself when the TDVMCALLs don't need support from userspace
>    after returning from userspace and before entering guest. Currently, no
>    such TDVMCALLs implemented, KVM just sets the values returned from
>    userspace.
> 
> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
> [Adjust userspace API. - Paolo]
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>   Documentation/virt/kvm/api.rst | 15 +++++++++++++-
>   arch/x86/kvm/vmx/tdx.c         | 38 ++++++++++++++++++++++++++++++----
>   include/uapi/linux/kvm.h       |  5 +++++
>   3 files changed, 53 insertions(+), 5 deletions(-)
> 
> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> index 3643d853a634..2b1656907356 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -7190,6 +7190,11 @@ The valid value for 'flags' is:
>   					u64 gpa;
>   					u64 size;
>   				} get_quote;
> +				struct {
> +					u64 ret;
> +					u64 leaf;
> +					u64 r11, r12, r13, r14;
> +				} get_tdvmcall_info;
>   			};
>   		} tdx;
> 
> @@ -7216,9 +7221,17 @@ queued successfully, the TDX guest can poll the status field in the
>   shared-memory area to check whether the Quote generation is completed or
>   not. When completed, the generated Quote is returned via the same buffer.
>   
> +* ``TDVMCALL_GET_TD_VM_CALL_INFO``: the guest has requested the support
> +status of TDVMCALLs.  The output values for the given leaf should be
> +placed in fields from ``r11`` to ``r14`` of the ``get_tdvmcall_info``
> +field of the union.  This TDVMCALL must succeed, therefore KVM leaves
> +``ret`` equal to ``TDVMCALL_STATUS_SUCCESS`` and ``r11`` to ``r14``
> +equal to zero on entry.
> +
>   KVM may add support for more values in the future that may cause a userspace
>   exit, even without calls to ``KVM_ENABLE_CAP`` or similar.  In this case,
> -it will enter with output fields already valid; in the common case, the
> +it will enter with output fields already valid as mentioned for
> +``TDVMCALL_GET_TD_VM_CALL_INFO`` above; in the common case, the
>   ``unknown.ret`` field of the union will be ``TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED``.
>   Userspace need not do anything if it does not wish to support a TDVMCALL.
>   ::
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index 6878a76069f8..5804d1b1ea0e 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -1451,18 +1451,49 @@ static int tdx_emulate_mmio(struct kvm_vcpu *vcpu)
>   	return 1;
>   }
>   
> +static int tdx_complete_get_td_vm_call_info(struct kvm_vcpu *vcpu)
> +{
> +	struct vcpu_tdx *tdx = to_tdx(vcpu);
> +
> +	tdvmcall_set_return_code(vcpu, vcpu->run->tdx.get_tdvmcall_info.ret);
> +
> +	/*
> +	 * For now, there is no TDVMCALL beyond GHCI base API supported by KVM
> +	 * directly without the support from userspace, just set the value
> +	 * returned from userspace.
> +	 */
> +	tdx->vp_enter_args.r11 = vcpu->run->tdx.get_tdvmcall_info.r11;
> +	tdx->vp_enter_args.r12 = vcpu->run->tdx.get_tdvmcall_info.r12;
> +	tdx->vp_enter_args.r13 = vcpu->run->tdx.get_tdvmcall_info.r13;
> +	tdx->vp_enter_args.r14 = vcpu->run->tdx.get_tdvmcall_info.r14;
> +
> +	return 1;
> +}
> +
>   static int tdx_get_td_vm_call_info(struct kvm_vcpu *vcpu)
>   {
>   	struct vcpu_tdx *tdx = to_tdx(vcpu);
>   
> -	if (tdx->vp_enter_args.r12)
> -		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
> -	else {
> +	switch (tdx->vp_enter_args.r12) {
> +	case 1:
> +		vcpu->run->tdx.get_tdvmcall_info.leaf = tdx->vp_enter_args.r12;
> +		vcpu->run->exit_reason = KVM_EXIT_TDX;
> +		vcpu->run->tdx.flags = 0;
> +		vcpu->run->tdx.nr = TDVMCALL_GET_TD_VM_CALL_INFO;
> +		vcpu->run->tdx.get_tdvmcall_info.ret = TDVMCALL_STATUS_SUCCESS;
> +		vcpu->run->tdx.get_tdvmcall_info.r11 = 0;
> +		vcpu->run->tdx.get_tdvmcall_info.r12 = 0;
> +		vcpu->run->tdx.get_tdvmcall_info.r13 = 0;
> +		vcpu->run->tdx.get_tdvmcall_info.r14 = 0;
> +		vcpu->arch.complete_userspace_io = tdx_complete_get_td_vm_call_info;
> +		return 0;
> +	default:
>   		tdx->vp_enter_args.r11 = 0;
> +		tdx->vp_enter_args.r12 = 0;
>   		tdx->vp_enter_args.r13 = 0;
>   		tdx->vp_enter_args.r14 = 0;
> +		return 1;

Though it looks OK to return all-0 for r12 == 0 and undefined case of 
r12 > 1, I prefer returning TDVMCALL_STATUS_INVALID_OPERAND for 
undefined case.

So please make above "case 0:", and make the "default:" return 
TDVMCALL_STATUS_INVALID_OPERAND

>   	}
> -	return 1;
>   }
>   
>   static int tdx_complete_simple(struct kvm_vcpu *vcpu)
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 6708bc88ae69..fb3b4cd8d662 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -461,6 +461,11 @@ struct kvm_run {
>   					__u64 gpa;
>   					__u64 size;
>   				} get_quote;
> +				struct {
> +					__u64 ret;
> +					__u64 leaf;
> +					__u64 r11, r12, r13, r14;
> +				} get_tdvmcall_info;
>   			};
>   		} tdx;
>   		/* Fix the size of the union. */
Re: [PATCH 3/3] KVM: TDX: Exit to userspace for GetTdVmCallInfo
Posted by Paolo Bonzini 3 months, 2 weeks ago
Il ven 20 giu 2025, 03:21 Xiaoyao Li <xiaoyao.li@intel.com> ha scritto:
>
> >               tdx->vp_enter_args.r11 = 0;
> > +             tdx->vp_enter_args.r12 = 0;
> >               tdx->vp_enter_args.r13 = 0;
> >               tdx->vp_enter_args.r14 = 0;
> > +             return 1;
>
> Though it looks OK to return all-0 for r12 == 0 and undefined case of
> r12 > 1, I prefer returning TDVMCALL_STATUS_INVALID_OPERAND for
> undefined case.


From the GHCI I wasn't sure that TDVMCALL_STATUS_INVALID_OPERAND is a
valid result at all.

Paolo

>
> So please make above "case 0:", and make the "default:" return
> TDVMCALL_STATUS_INVALID_OPERAND
>
> >       }
> > -     return 1;
> >   }
> >
> >   static int tdx_complete_simple(struct kvm_vcpu *vcpu)
> > diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> > index 6708bc88ae69..fb3b4cd8d662 100644
> > --- a/include/uapi/linux/kvm.h
> > +++ b/include/uapi/linux/kvm.h
> > @@ -461,6 +461,11 @@ struct kvm_run {
> >                                       __u64 gpa;
> >                                       __u64 size;
> >                               } get_quote;
> > +                             struct {
> > +                                     __u64 ret;
> > +                                     __u64 leaf;
> > +                                     __u64 r11, r12, r13, r14;
> > +                             } get_tdvmcall_info;
> >                       };
> >               } tdx;
> >               /* Fix the size of the union. */
>
Re: [PATCH 3/3] KVM: TDX: Exit to userspace for GetTdVmCallInfo
Posted by Xiaoyao Li 3 months, 2 weeks ago
On 6/20/2025 8:03 PM, Paolo Bonzini wrote:
> Il ven 20 giu 2025, 03:21 Xiaoyao Li <xiaoyao.li@intel.com> ha scritto:
>>
>>>                tdx->vp_enter_args.r11 = 0;
>>> +             tdx->vp_enter_args.r12 = 0;
>>>                tdx->vp_enter_args.r13 = 0;
>>>                tdx->vp_enter_args.r14 = 0;
>>> +             return 1;
>>
>> Though it looks OK to return all-0 for r12 == 0 and undefined case of
>> r12 > 1, I prefer returning TDVMCALL_STATUS_INVALID_OPERAND for
>> undefined case.
> 
> 
>  From the GHCI I wasn't sure that TDVMCALL_STATUS_INVALID_OPERAND is a
> valid result at all.

It's part of the new GHCI change, which currently is still in draft 
state. (Sorry for not informing you)

The proposed GHCI update defines VMCALL_OPERAND_INVALID for the case of 
input R12 value is not supported. So for VMM that doesn't implement the 
enumeration for the optional leafs when r12 = 1 can return this status 
code. As well, VMM can return this status code for the case of input R12 
 >= 2, to avoid the VMM introduces its own defined behavior.

> Paolo
> 
>>
>> So please make above "case 0:", and make the "default:" return
>> TDVMCALL_STATUS_INVALID_OPERAND
>>
>>>        }
>>> -     return 1;
>>>    }
>>>
>>>    static int tdx_complete_simple(struct kvm_vcpu *vcpu)
>>> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>>> index 6708bc88ae69..fb3b4cd8d662 100644
>>> --- a/include/uapi/linux/kvm.h
>>> +++ b/include/uapi/linux/kvm.h
>>> @@ -461,6 +461,11 @@ struct kvm_run {
>>>                                        __u64 gpa;
>>>                                        __u64 size;
>>>                                } get_quote;
>>> +                             struct {
>>> +                                     __u64 ret;
>>> +                                     __u64 leaf;
>>> +                                     __u64 r11, r12, r13, r14;
>>> +                             } get_tdvmcall_info;
>>>                        };
>>>                } tdx;
>>>                /* Fix the size of the union. */
>>
> 
>