VMCALL and VMMCALL instructions are used by x86 guests to request services
from the host VMM. Both VMCALL and VMMCALL are not restricted to CPL 0.
This allows userspace software like open-vm-tools to communicate directly
with the VMM.

In the context of confidential VMs, direct communication with the host may
violate the security model. Existing binaries that make use of hypercalls
and are not hardened against malicious hypervisors can become a possible
attack surface. For this reason, user-level VMCALLs are not currently
forwarded to the host on TDX VMs. This breaks any user-level software that
use these instructions.

But if user-level software is aware of the risks and has been hardened to
address any known violations of the security model, then it seems
reasonable to allow hypercalls from this process to proceed.

This patchset introduces a new x86 process control flag to address this
concern. By setting the TIF_COCO_USER_HCALL thread information flag, the
process opts in to user-level hypercalls. When TDX is enabled, the VMCALL
will #VE and control will be transferred to a hypervisor-specific
hypercall handler (similar to how things work today for SEV with
sev_es_hcall_prepare/sev_es_hcall_finish). The flag has no effect on
non-TDX VMs. Other confidential computing technologies could use this flag
to provide limited access to user-level hypercalls.

Tim Merrifield (2):
  x86/tdx: Add prctl to allow userlevel TDX hypercalls
  x86/vmware: VMware support for TDX userspace hypercalls

 arch/x86/coco/tdx/tdx.c            | 18 +++++++++++
 arch/x86/include/asm/thread_info.h |  2 ++
 arch/x86/include/asm/x86_init.h    |  1 +
 arch/x86/include/uapi/asm/prctl.h  |  3 ++
 arch/x86/kernel/cpu/vmware.c       | 51 +++++++++++++++++++++++-------
 arch/x86/kernel/process.c          | 20 ++++++++++++
 6 files changed, 84 insertions(+), 11 deletions(-)

-- 
2.40.1

On Wed, Jul 03, 2024 at 11:35:59PM +0000, Tim Merrifield wrote:
> VMCALL and VMMCALL instructions are used by x86 guests to request services
> from the host VMM. Both VMCALL and VMMCALL are not restricted to CPL 0.
> This allows userspace software like open-vm-tools to communicate directly
> with the VMM.
> 
> In the context of confidential VMs, direct communication with the host may
> violate the security model. Existing binaries that make use of hypercalls
> and are not hardened against malicious hypervisors can become a possible
> attack surface. For this reason, user-level VMCALLs are not currently
> forwarded to the host on TDX VMs. This breaks any user-level software that
> use these instructions.
> 
> But if user-level software is aware of the risks and has been hardened to
> address any known violations of the security model, then it seems
> reasonable to allow hypercalls from this process to proceed.

And how are we to ascertain the software using these hooks is deemed
secure? What security risks are there for the kernel if a malicious
userspace process asks for these rights?

The kernel must assume malice on the part of userspace.

On Thu, Jul 04, 2024 at 03:05:05PM +0200, Peter Zijlstra wrote:
> And how are we to ascertain the software using these hooks is deemed
> secure? What security risks are there for the kernel if a malicious
> userspace process asks for these rights?
> 
> The kernel must assume malice on the part of userspace.

Thanks, Peter.
   
I don't believe there are any additional security risks for the kernel
itself being introduced here. The kernel is only responsible for
copying to and from userspace registers for the hypercall, and
executing the TDCALL. A similar approach already exists for AMD SEV
(see vc_handle_vmmcall), which does not restrict VMMCALL in the way
that TDX restricts VMCALL.

In the case of a malicious binary running in a TDX VM, if it wants to
communicate with the untrusted hypervisor or other software outside
of the TD, there are several existing mechanisms it could use, not
just a VMCALL. I guess the point here is that if the userspace
program is malicious, is anything gained by restricting VMCALL?

This patchset really only handles the case where a trusted guest
wants to limit access to VMCALL to binaries that self identify as
hardened against potential host attacks.

On 7/3/24 16:35, Tim Merrifield wrote:
> VMCALL and VMMCALL instructions are used by x86 guests to request services
> from the host VMM. Both VMCALL and VMMCALL are not restricted to CPL 0.
> This allows userspace software like open-vm-tools to communicate directly
> with the VMM.

Could we please be frank and transparent about what you actually want
here and how you expect this mechanism to be used?

...
> This patchset introduces a new x86 process control flag to address this
> concern. By setting the TIF_COCO_USER_HCALL thread information flag, the
> process opts in to user-level hypercalls.

The process, and anything it fork()s or execve()s, right?

This inheritance model seems more suited to wrapping a tiny helper app
around an existing binary, a la:

	prctl(ARCH_SET_COCO_USER_HCALL);
	execve("/existing/binary/that/i/surely/did/not/audit", ...);

... as opposed to something that you set in new versions of
open-vm-tools after an extensive audit and a bug fixing campaign to
clean up everything that the audit found.

Thanks for the response, Dave.

On Wed, Jul 03, 2024 at 05:18:22PM -0700, Dave Hansen wrote:
> 
> Could we please be frank and transparent about what you actually want
> here and how you expect this mechanism to be used?
>

Sorry for being unclear. open-vm-tools is currently broken on TDX and
the intent here is to fix that. The idea is that versions of open-vm-tools
that have been audited and restricted to certain hypercalls, would execute
prctl to mark the process as capable of executing hypercalls.
    
> This inheritance model seems more suited to wrapping a tiny helper app
> around an existing binary, a la:
> 
> 	prctl(ARCH_SET_COCO_USER_HCALL);
> 	execve("/existing/binary/that/i/surely/did/not/audit", ...);
> 
> ... as opposed to something that you set in new versions of
> open-vm-tools after an extensive audit and a bug fixing campaign to
> clean up everything that the audit found.

I understand the concern about inheritance. I chose prctl primarily
because of some existing options that seemed similar, mainly speculation
control. Is there an alternative approach that doesn't suffer from the
inheritance issue?