[v1] KVM: guest memory: Misc enhacnement

[RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by isaku.yamahata@intel.com 2 years, 7 months ago

From: Isaku Yamahata <isaku.yamahata@intel.com>

Hello. This is an RFC patch series based on KVM gmem [1] and [2] 
for the common use of TDX and SEV-SNP.  I'd like to discuss three items.

* Flags for kvm_gfn_range:
Roth, By adding flags to kvm_gfn_range, kvm_arch_gmem_invalidate() won't be
needed.  Maybe x86 gmem_invalidate callback is still needed, though.

* Page fault error code or how to figure out the private page fault
There is an issue to set kvm_page_fault.is_private. I came up with two new
error codes.  Is this a way or any other way?

* VM type: Now we have KVM_X86_PROTECTED_VM. How do we proceed?
  - Keep KVM_X86_PROTECTED_VM for its use. Introduce KVM_X86_TDX_VM
  - Use KVM_X86_PROTECTED_VM for TDX. (If necessary, introduce another type in
    the future)
  - any other way?

[1] KVM gmem patches
https://github.com/sean-jc/linux/tree/x86/kvm_gmem_solo

[2] Add AMD Secure Nested Paging (SEV-SNP) Hypervisor Support
https://lore.kernel.org/lkml/20230612042559.375660-1-michael.roth@amd.com/

Isaku Yamahata (6):
  KVM: selftests: Fix test_add_overlapping_private_memory_regions()
  KVM: selftests: Fix guest_memfd()
  KVM: x86/mmu: Pass round full 64-bit error code for the KVM page fault
  KVM: x86: Introduce PFERR_GUEST_ENC_MASK to indicate fault is private
  KVM: Add flags to struct kvm_gfn_range
  KVM: x86: Add is_vm_type_supported callback

 arch/x86/include/asm/kvm-x86-ops.h               |  1 +
 arch/x86/include/asm/kvm_host.h                  |  5 +++++
 arch/x86/include/uapi/asm/kvm.h                  |  1 +
 arch/x86/kvm/mmu/mmu.c                           | 14 +++++++++-----
 arch/x86/kvm/mmu/mmu_internal.h                  |  8 ++++----
 arch/x86/kvm/mmu/mmutrace.h                      |  2 +-
 arch/x86/kvm/mmu/paging_tmpl.h                   |  4 ++--
 arch/x86/kvm/svm/svm.c                           |  7 +++++++
 arch/x86/kvm/vmx/vmx.c                           |  6 ++++++
 arch/x86/kvm/x86.c                               | 10 +++++++++-
 arch/x86/kvm/x86.h                               |  2 ++
 include/linux/kvm_host.h                         | 11 ++++++++++-
 tools/testing/selftests/kvm/guest_memfd_test.c   |  4 ++--
 .../selftests/kvm/set_memory_region_test.c       | 16 ++++++++++++++--
 virt/kvm/guest_mem.c                             | 10 +++++++---
 virt/kvm/kvm_main.c                              |  4 +++-
 16 files changed, 83 insertions(+), 22 deletions(-)


base-commit: be8abcec83c87d4e15ae04816b685fe260c4bcfd
-- 
2.25.1

Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by Vishal Annapurve 2 years, 7 months ago

On Thu, Jun 15, 2023 at 1:12 PM <isaku.yamahata@intel.com> wrote:
> ...
>
> * VM type: Now we have KVM_X86_PROTECTED_VM. How do we proceed?
>   - Keep KVM_X86_PROTECTED_VM for its use. Introduce KVM_X86_TDX_VM
>   - Use KVM_X86_PROTECTED_VM for TDX. (If necessary, introduce another type in
>     the future)
>   - any other way?

There are selftests posted[1] in context of this work, which rely on
KVM_X86_PROTECTED_VM being just the software-only psuedo-confidential
VMs. In future there might be more work to expand this usecase to
full-scale VMs. So it would be better to treat protected VMs as a
separate type which can be used on any platform without the need of
enabling TDX/SEV functionality.

TDX VM type can possibly serve as a specialized type of protected VM
with additional arch specific capabilities enabled.

[1] - https://github.com/sean-jc/linux/commits/x86/kvm_gmem_solo

Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by Zhi Wang 2 years, 7 months ago

On Mon, 19 Jun 2023 12:11:50 -0700
Vishal Annapurve <vannapurve@google.com> wrote:

> On Thu, Jun 15, 2023 at 1:12___PM <isaku.yamahata@intel.com> wrote:
> > ...
> >
> > * VM type: Now we have KVM_X86_PROTECTED_VM. How do we proceed?
> >   - Keep KVM_X86_PROTECTED_VM for its use. Introduce KVM_X86_TDX_VM
> >   - Use KVM_X86_PROTECTED_VM for TDX. (If necessary, introduce another type in
> >     the future)
> >   - any other way?  
> 
> There are selftests posted[1] in context of this work, which rely on
> KVM_X86_PROTECTED_VM being just the software-only psuedo-confidential
> VMs. In future there might be more work to expand this usecase to
> full-scale VMs. So it would be better to treat protected VMs as a
> separate type which can be used on any platform without the need of
> enabling TDX/SEV functionality.
> 

Out of curiosity, is this really a valid case in practice except selftest?
It sounds to me whenever KVM_X86_PROTECTED_VM is used, it has to be tied
with a platform-specific CC type.

> TDX VM type can possibly serve as a specialized type of protected VM
> with additional arch specific capabilities enabled.
> 
> [1] - https://github.com/sean-jc/linux/commits/x86/kvm_gmem_solo

Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by Vishal Annapurve 2 years, 7 months ago

On Mon, Jun 19, 2023 at 1:11 PM Zhi Wang <zhi.wang.linux@gmail.com> wrote:
>
> On Mon, 19 Jun 2023 12:11:50 -0700
> Vishal Annapurve <vannapurve@google.com> wrote:
>
> > On Thu, Jun 15, 2023 at 1:12___PM <isaku.yamahata@intel.com> wrote:
> > > ...
> > >
> > > * VM type: Now we have KVM_X86_PROTECTED_VM. How do we proceed?
> > >   - Keep KVM_X86_PROTECTED_VM for its use. Introduce KVM_X86_TDX_VM
> > >   - Use KVM_X86_PROTECTED_VM for TDX. (If necessary, introduce another type in
> > >     the future)
> > >   - any other way?
> >
> > There are selftests posted[1] in context of this work, which rely on
> > KVM_X86_PROTECTED_VM being just the software-only psuedo-confidential
> > VMs. In future there might be more work to expand this usecase to
> > full-scale VMs. So it would be better to treat protected VMs as a
> > separate type which can be used on any platform without the need of
> > enabling TDX/SEV functionality.
> >
>
> Out of curiosity, is this really a valid case in practice except selftest?
> It sounds to me whenever KVM_X86_PROTECTED_VM is used, it has to be tied
> with a platform-specific CC type.

Protected VM effort is about being able to have guest memory ranges
not mapped into Userspace VMM and so are unreachable for most of the
cases from KVM as well. Non-CC VMs can use this support to mitigate
any unintended accesses from userspace VMM/KVM possibly using
enlightened kernels.

Exact implementation of such a support warrants more discussion but it
should be in the line of sight here as a future work item.




>
> > TDX VM type can possibly serve as a specialized type of protected VM
> > with additional arch specific capabilities enabled.
> >
> > [1] - https://github.com/sean-jc/linux/commits/x86/kvm_gmem_solo
>

RE: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by Dong, Eddie 2 years, 7 months ago


> -----Original Message-----
> From: Vishal Annapurve <vannapurve@google.com>
> Sent: Monday, June 19, 2023 2:55 PM
> To: Zhi Wang <zhi.wang.linux@gmail.com>
> Cc: Yamahata, Isaku <isaku.yamahata@intel.com>; kvm@vger.kernel.org;
> linux-kernel@vger.kernel.org; isaku.yamahata@gmail.com; Paolo Bonzini
> <pbonzini@redhat.com>; Aktas, Erdem <erdemaktas@google.com>;
> Christopherson,, Sean <seanjc@google.com>; Shahar, Sagi
> <sagis@google.com>; David Matlack <dmatlack@google.com>; Huang, Kai
> <kai.huang@intel.com>; Chen, Bo2 <chen.bo@intel.com>; linux-
> coco@lists.linux.dev; Chao Peng <chao.p.peng@linux.intel.com>; Ackerley
> Tng <ackerleytng@google.com>; Michael Roth <michael.roth@amd.com>
> Subject: Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement
> 
> On Mon, Jun 19, 2023 at 1:11 PM Zhi Wang <zhi.wang.linux@gmail.com>
> wrote:
> >
> > On Mon, 19 Jun 2023 12:11:50 -0700
> > Vishal Annapurve <vannapurve@google.com> wrote:
> >
> > > On Thu, Jun 15, 2023 at 1:12___PM <isaku.yamahata@intel.com> wrote:
> > > > ...
> > > >
> > > > * VM type: Now we have KVM_X86_PROTECTED_VM. How do we
> proceed?
> > > >   - Keep KVM_X86_PROTECTED_VM for its use. Introduce
> KVM_X86_TDX_VM
> > > >   - Use KVM_X86_PROTECTED_VM for TDX. (If necessary, introduce
> another type in
> > > >     the future)
> > > >   - any other way?
> > >
> > > There are selftests posted[1] in context of this work, which rely on
> > > KVM_X86_PROTECTED_VM being just the software-only
> > > psuedo-confidential VMs. In future there might be more work to
> > > expand this usecase to full-scale VMs. So it would be better to
> > > treat protected VMs as a separate type which can be used on any
> > > platform without the need of enabling TDX/SEV functionality.
> > >
> >
> > Out of curiosity, is this really a valid case in practice except selftest?
> > It sounds to me whenever KVM_X86_PROTECTED_VM is used, it has to be
> > tied with a platform-specific CC type.
> 
> Protected VM effort is about being able to have guest memory ranges not
> mapped into Userspace VMM and so are unreachable for most of the cases
> from KVM as well. Non-CC VMs can use this support to mitigate any
> unintended accesses from userspace VMM/KVM possibly using enlightened
> kernels.

"PROTECTED" seems to be not very close to what you mean here. "PROTECTED_MEM" ?
What case of non-CC VMs may use this feature in reality?  Or do you have any expected cases?

> 
> Exact implementation of such a support warrants more discussion but it
> should be in the line of sight here as a future work item.
> 
> 
> 
> 
> >
> > > TDX VM type can possibly serve as a specialized type of protected VM
> > > with additional arch specific capabilities enabled.
> > >
> > > [1] - https://github.com/sean-jc/linux/commits/x86/kvm_gmem_solo
> >

Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by Vishal Annapurve 2 years, 7 months ago

On Wed, Jun 21, 2023 at 11:20 AM Dong, Eddie <eddie.dong@intel.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Vishal Annapurve <vannapurve@google.com>
> > Sent: Monday, June 19, 2023 2:55 PM
> > To: Zhi Wang <zhi.wang.linux@gmail.com>
> > Cc: Yamahata, Isaku <isaku.yamahata@intel.com>; kvm@vger.kernel.org;
> > linux-kernel@vger.kernel.org; isaku.yamahata@gmail.com; Paolo Bonzini
> > <pbonzini@redhat.com>; Aktas, Erdem <erdemaktas@google.com>;
> > Christopherson,, Sean <seanjc@google.com>; Shahar, Sagi
> > <sagis@google.com>; David Matlack <dmatlack@google.com>; Huang, Kai
> > <kai.huang@intel.com>; Chen, Bo2 <chen.bo@intel.com>; linux-
> > coco@lists.linux.dev; Chao Peng <chao.p.peng@linux.intel.com>; Ackerley
> > Tng <ackerleytng@google.com>; Michael Roth <michael.roth@amd.com>
> > Subject: Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement
> >
> > On Mon, Jun 19, 2023 at 1:11 PM Zhi Wang <zhi.wang.linux@gmail.com>
> > wrote:
> > >
> > > On Mon, 19 Jun 2023 12:11:50 -0700
> ...
> >
> > Protected VM effort is about being able to have guest memory ranges not
> > mapped into Userspace VMM and so are unreachable for most of the cases
> > from KVM as well. Non-CC VMs can use this support to mitigate any
> > unintended accesses from userspace VMM/KVM possibly using enlightened
> > kernels.
>
> "PROTECTED" seems to be not very close to what you mean here. "PROTECTED_MEM" ?
> What case of non-CC VMs may use this feature in reality?  Or do you have any expected cases?
>

Similar to pKvm efforts [1], PROTECTED_VM functionality may be used to
unmap guest memory ranges from the host and userspace VMM on x86
platforms. If the KVM/host kernel and the guest VMs are enlightened
for this usecase, then it should be possible to deploy this feature
for normal VMs irrespective of the platforms they are running on.

Primary usecase here would be to prevent unintended accesses from
KVM/userspace VMM which would normally go undetected at runtime or are
hard to trace back to the original culprit.

[1] https://source.android.com/docs/core/virtualization/architecture#hypervisor

Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by Zhi Wang 2 years, 7 months ago

On Mon, 19 Jun 2023 14:55:09 -0700
Vishal Annapurve <vannapurve@google.com> wrote:

> On Mon, Jun 19, 2023 at 1:11___PM Zhi Wang <zhi.wang.linux@gmail.com> wrote:
> >
> > On Mon, 19 Jun 2023 12:11:50 -0700
> > Vishal Annapurve <vannapurve@google.com> wrote:
> >
> > > On Thu, Jun 15, 2023 at 1:12___PM <isaku.yamahata@intel.com> wrote:
> > > > ...
> > > >
> > > > * VM type: Now we have KVM_X86_PROTECTED_VM. How do we proceed?
> > > >   - Keep KVM_X86_PROTECTED_VM for its use. Introduce KVM_X86_TDX_VM
> > > >   - Use KVM_X86_PROTECTED_VM for TDX. (If necessary, introduce another type in
> > > >     the future)
> > > >   - any other way?
> > >
> > > There are selftests posted[1] in context of this work, which rely on
> > > KVM_X86_PROTECTED_VM being just the software-only psuedo-confidential
> > > VMs. In future there might be more work to expand this usecase to
> > > full-scale VMs. So it would be better to treat protected VMs as a
> > > separate type which can be used on any platform without the need of
> > > enabling TDX/SEV functionality.
> > >
> >
> > Out of curiosity, is this really a valid case in practice except selftest?
> > It sounds to me whenever KVM_X86_PROTECTED_VM is used, it has to be tied
> > with a platform-specific CC type.
> 
> Protected VM effort is about being able to have guest memory ranges
> not mapped into Userspace VMM and so are unreachable for most of the
> cases from KVM as well. Non-CC VMs can use this support to mitigate
> any unintended accesses from userspace VMM/KVM possibly using
> enlightened kernels.
> 
> Exact implementation of such a support warrants more discussion but it
> should be in the line of sight here as a future work item.
> 
>

IIUC, what you are saying is (KVM_X86_PROTECTED_VM == (VMs with UPM or GMEM))
&& (KVM_X86_PROTECTED_VM != KVM_X86_CC_VM) && KVM_X86_CC_VM requires
KVM_X86_PROTECTED_VM.

If we think KVM_X86_PROTECTED_VM as a dedicated feature, not tightly coupled
with CC techs, it seems we needs another defination of KVM_X86_CC_VM to represent
CC VM and CC platform types like KVM_X86_CC_TDX_VM to tell which CC tech sits
behind it?

I don't think it is good to mix the usages of KVM_X86_PROTECTED_VM and KVM_X86_CC_VM
together if we are sure KVM_X86_PROTECTED_VM is not going to represent CC VMs
in the code.

> 
> 
> >
> > > TDX VM type can possibly serve as a specialized type of protected VM
> > > with additional arch specific capabilities enabled.
> > >
> > > [1] - https://github.com/sean-jc/linux/commits/x86/kvm_gmem_solo
> >

Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by Sean Christopherson 2 years, 7 months ago

On Thu, Jun 15, 2023, isaku.yamahata@intel.com wrote:
> * VM type: Now we have KVM_X86_PROTECTED_VM. How do we proceed?
>   - Keep KVM_X86_PROTECTED_VM for its use. Introduce KVM_X86_TDX_VM
>   - Use KVM_X86_PROTECTED_VM for TDX. (If necessary, introduce another type in
>     the future)

How would KVM differentiate between software-protected VMs and TDX VMs if we go
with this option?

>   - any other way?

Re: [RFC PATCH 0/6] KVM: guest memory: Misc enhacnement

Posted by Isaku Yamahata 2 years, 7 months ago

On Fri, Jun 23, 2023 at 05:09:26PM -0700,
Sean Christopherson <seanjc@google.com> wrote:

> On Thu, Jun 15, 2023, isaku.yamahata@intel.com wrote:
> > * VM type: Now we have KVM_X86_PROTECTED_VM. How do we proceed?
> >   - Keep KVM_X86_PROTECTED_VM for its use. Introduce KVM_X86_TDX_VM
> >   - Use KVM_X86_PROTECTED_VM for TDX. (If necessary, introduce another type in
> >     the future)
> 
> How would KVM differentiate between software-protected VMs and TDX VMs if we go
> with this option?

Let's introduce new two VM type.  I'm fine with two new VM types.  I had KVM
capability in mind if SEV implementation doesn't like new VM types.
-- 
Isaku Yamahata <isaku.yamahata@gmail.com>