[PATCH v4 00/19] Enable FRED with KVM VMX

Xin Li (Intel) posted 19 patches 8 months, 3 weeks ago
There is a newer version of this series
Documentation/virt/kvm/api.rst            |  19 ++
Documentation/virt/kvm/x86/nested-vmx.rst |  19 ++
arch/x86/include/asm/kvm_host.h           |   8 +-
arch/x86/include/asm/msr-index.h          |   1 +
arch/x86/include/asm/vmx.h                |  48 ++++-
arch/x86/include/uapi/asm/kvm.h           |   4 +-
arch/x86/kvm/cpuid.c                      |   2 +
arch/x86/kvm/kvm_cache_regs.h             |  15 ++
arch/x86/kvm/svm/svm.c                    |   2 +-
arch/x86/kvm/vmx/capabilities.h           |  26 ++-
arch/x86/kvm/vmx/nested.c                 | 188 ++++++++++++++++-
arch/x86/kvm/vmx/nested.h                 |  22 ++
arch/x86/kvm/vmx/vmcs.h                   |   1 +
arch/x86/kvm/vmx/vmcs12.c                 |  19 ++
arch/x86/kvm/vmx/vmcs12.h                 |  38 ++++
arch/x86/kvm/vmx/vmcs_shadow_fields.h     |   4 +
arch/x86/kvm/vmx/vmx.c                    | 237 ++++++++++++++++++++--
arch/x86/kvm/vmx/vmx.h                    |  15 +-
arch/x86/kvm/x86.c                        |  74 ++++++-
arch/x86/kvm/x86.h                        |   8 +-
arch/x86/mm/cpu_entry_area.c              |   7 +
include/uapi/linux/kvm.h                  |   1 +
22 files changed, 727 insertions(+), 31 deletions(-)
[PATCH v4 00/19] Enable FRED with KVM VMX
Posted by Xin Li (Intel) 8 months, 3 weeks ago
This patch set enables the Intel flexible return and event delivery
(FRED) architecture with KVM VMX to allow guests to utilize FRED.

The FRED architecture defines simple new transitions that change
privilege level (ring transitions). The FRED architecture was
designed with the following goals:

1) Improve overall performance and response time by replacing event
   delivery through the interrupt descriptor table (IDT event
   delivery) and event return by the IRET instruction with lower
   latency transitions.

2) Improve software robustness by ensuring that event delivery
   establishes the full supervisor context and that event return
   establishes the full user context.

The new transitions defined by the FRED architecture are FRED event
delivery and, for returning from events, two FRED return instructions.
FRED event delivery can effect a transition from ring 3 to ring 0, but
it is used also to deliver events incident to ring 0. One FRED
instruction (ERETU) effects a return from ring 0 to ring 3, while the
other (ERETS) returns while remaining in ring 0. Collectively, FRED
event delivery and the FRED return instructions are FRED transitions.

Intel VMX architecture is extended to run FRED guests, and the major
changes are:

1) New VMCS fields for FRED context management, which includes two new
event data VMCS fields, eight new guest FRED context VMCS fields and
eight new host FRED context VMCS fields.

2) VMX nested-exception support for proper virtualization of stack
levels introduced with FRED architecture.

Search for the latest FRED spec in most search engines with this search
pattern:

  site:intel.com FRED (flexible return and event delivery) specification

Following is the link to the v3 of this patch set:
https://lore.kernel.org/lkml/20241001050110.3643764-1-xin@zytor.com/

Since several preparatory patches in v3 have been merged, and Sean
reiterated that it's NOT worth to precisely track which fields are/
aren't supported [1], v4 patch number is reduced to 19.

Although FRED and CET supervisor shadow stacks are independent CPU
features, FRED unconditionally includes FRED shadow stack pointer
MSRs IA32_FRED_SSP[0123], and IA32_FRED_SSP0 is just an alias of the
CET MSR IA32_PL0_SSP.  IOW, the state management of MSR IA32_PL0_SSP
becomes an overlap area, and Sean requested that FRED virtualization
to land after CET virtualization [2].

[1]: https://lore.kernel.org/lkml/Z73uK5IzVoBej3mi@google.com/
[2]: https://lore.kernel.org/kvm/ZvQaNRhrsSJTYji3@google.com/


Xin Li (17):
  KVM: VMX: Add support for the secondary VM exit controls
  KVM: VMX: Initialize VM entry/exit FRED controls in vmcs_config
  KVM: VMX: Disable FRED if FRED consistency checks fail
  KVM: VMX: Initialize VMCS FRED fields
  KVM: VMX: Set FRED MSR interception
  KVM: VMX: Save/restore guest FRED RSP0
  KVM: VMX: Add support for FRED context save/restore
  KVM: x86: Add a helper to detect if FRED is enabled for a vCPU
  KVM: VMX: Virtualize FRED event_data
  KVM: VMX: Virtualize FRED nested exception tracking
  KVM: x86: Mark CR4.FRED as not reserved
  KVM: VMX: Dump FRED context in dump_vmcs()
  KVM: x86: Allow FRED/LKGS to be advertised to guests
  KVM: nVMX: Add support for the secondary VM exit controls
  KVM: nVMX: Add FRED VMCS fields to nested VMX context management
  KVM: nVMX: Add VMCS FRED states checking
  KVM: nVMX: Allow VMX FRED controls

Xin Li (Intel) (2):
  x86/cea: Export per CPU array 'cea_exception_stacks' for KVM to use
  KVM: x86: Save/restore the nested flag of an exception

 Documentation/virt/kvm/api.rst            |  19 ++
 Documentation/virt/kvm/x86/nested-vmx.rst |  19 ++
 arch/x86/include/asm/kvm_host.h           |   8 +-
 arch/x86/include/asm/msr-index.h          |   1 +
 arch/x86/include/asm/vmx.h                |  48 ++++-
 arch/x86/include/uapi/asm/kvm.h           |   4 +-
 arch/x86/kvm/cpuid.c                      |   2 +
 arch/x86/kvm/kvm_cache_regs.h             |  15 ++
 arch/x86/kvm/svm/svm.c                    |   2 +-
 arch/x86/kvm/vmx/capabilities.h           |  26 ++-
 arch/x86/kvm/vmx/nested.c                 | 188 ++++++++++++++++-
 arch/x86/kvm/vmx/nested.h                 |  22 ++
 arch/x86/kvm/vmx/vmcs.h                   |   1 +
 arch/x86/kvm/vmx/vmcs12.c                 |  19 ++
 arch/x86/kvm/vmx/vmcs12.h                 |  38 ++++
 arch/x86/kvm/vmx/vmcs_shadow_fields.h     |   4 +
 arch/x86/kvm/vmx/vmx.c                    | 237 ++++++++++++++++++++--
 arch/x86/kvm/vmx/vmx.h                    |  15 +-
 arch/x86/kvm/x86.c                        |  74 ++++++-
 arch/x86/kvm/x86.h                        |   8 +-
 arch/x86/mm/cpu_entry_area.c              |   7 +
 include/uapi/linux/kvm.h                  |   1 +
 22 files changed, 727 insertions(+), 31 deletions(-)


base-commit: acb4f33713b9f6cadb6143f211714c343465411c
-- 
2.48.1
Re: [PATCH v4 00/19] Enable FRED with KVM VMX
Posted by Xin Li 8 months, 3 weeks ago
On 3/28/2025 10:11 AM, Xin Li (Intel) wrote:
> This patch set enables the Intel flexible return and event delivery
> (FRED) architecture with KVM VMX to allow guests to utilize FRED.
> 
> The FRED architecture defines simple new transitions that change
> privilege level (ring transitions). The FRED architecture was
> designed with the following goals:
> 
> 1) Improve overall performance and response time by replacing event
>     delivery through the interrupt descriptor table (IDT event
>     delivery) and event return by the IRET instruction with lower
>     latency transitions.
> 
> 2) Improve software robustness by ensuring that event delivery
>     establishes the full supervisor context and that event return
>     establishes the full user context.
> 
> The new transitions defined by the FRED architecture are FRED event
> delivery and, for returning from events, two FRED return instructions.
> FRED event delivery can effect a transition from ring 3 to ring 0, but
> it is used also to deliver events incident to ring 0. One FRED
> instruction (ERETU) effects a return from ring 0 to ring 3, while the
> other (ERETS) returns while remaining in ring 0. Collectively, FRED
> event delivery and the FRED return instructions are FRED transitions.
> 
> Intel VMX architecture is extended to run FRED guests, and the major
> changes are:
> 
> 1) New VMCS fields for FRED context management, which includes two new
> event data VMCS fields, eight new guest FRED context VMCS fields and
> eight new host FRED context VMCS fields.
> 
> 2) VMX nested-exception support for proper virtualization of stack
> levels introduced with FRED architecture.
> 
> Search for the latest FRED spec in most search engines with this search
> pattern:
> 
>    site:intel.com FRED (flexible return and event delivery) specification
> 
> Following is the link to the v3 of this patch set:
> https://lore.kernel.org/lkml/20241001050110.3643764-1-xin@zytor.com/
> 
> Since several preparatory patches in v3 have been merged, and Sean
> reiterated that it's NOT worth to precisely track which fields are/
> aren't supported [1], v4 patch number is reduced to 19.
> 
> Although FRED and CET supervisor shadow stacks are independent CPU
> features, FRED unconditionally includes FRED shadow stack pointer
> MSRs IA32_FRED_SSP[0123], and IA32_FRED_SSP0 is just an alias of the
> CET MSR IA32_PL0_SSP.  IOW, the state management of MSR IA32_PL0_SSP
> becomes an overlap area, and Sean requested that FRED virtualization
> to land after CET virtualization [2].

Hi Sean,

Any chance we could merge FRED ahead of CET?

Ofc with proper changes to FRED code.

Thanks!
     Xin

> 
> [1]: https://lore.kernel.org/lkml/Z73uK5IzVoBej3mi@google.com/
> [2]: https://lore.kernel.org/kvm/ZvQaNRhrsSJTYji3@google.com/
Re: [PATCH v4 00/19] Enable FRED with KVM VMX
Posted by Sean Christopherson 5 months, 3 weeks ago
On Fri, Mar 28, 2025, Xin Li wrote:
> Any chance we could merge FRED ahead of CET?

Probably not?  CET exists is publicly available CPUs.  AFAIK, FRED does not.
And CET is (/knock wood) hopefully pretty much ready?  FWIW, I'd really like to
get both CET and FRED virtualization landed by 6.18, i.e. in time for the next
LTS.
Re: [PATCH v4 00/19] Enable FRED with KVM VMX
Posted by Xin Li 5 months, 3 weeks ago
On 6/24/2025 10:06 AM, Sean Christopherson wrote:
> On Fri, Mar 28, 2025, Xin Li wrote:
>> Any chance we could merge FRED ahead of CET?
> 
> Probably not?  CET exists is publicly available CPUs.  AFAIK, FRED does not.

Better not, as you said it creates extra effort because FRED does lean a
bit on CET.

I was a bit worried that CET would take longer time than expected...

> And CET is (/knock wood) hopefully pretty much ready?  FWIW, I'd really like to

That is also my reading on CET.

> get both CET and FRED virtualization landed by 6.18, i.e. in time for the next
> LTS.

I love the plan!

FRED is my top priority.  I’ll address all your comments, rebase onto
kvm-x86/next (you have not updated yet :) ), and send out v5 at an
appropriate time.

Thanks!
      Xin

Re: [PATCH v4 00/19] Enable FRED with KVM VMX
Posted by H. Peter Anvin 5 months, 3 weeks ago
On June 24, 2025 10:43:26 AM PDT, Xin Li <xin@zytor.com> wrote:
>On 6/24/2025 10:06 AM, Sean Christopherson wrote:
>> On Fri, Mar 28, 2025, Xin Li wrote:
>>> Any chance we could merge FRED ahead of CET?
>> 
>> Probably not?  CET exists is publicly available CPUs.  AFAIK, FRED does not.
>
>Better not, as you said it creates extra effort because FRED does lean a
>bit on CET.
>
>I was a bit worried that CET would take longer time than expected...
>
>> And CET is (/knock wood) hopefully pretty much ready?  FWIW, I'd really like to
>
>That is also my reading on CET.
>
>> get both CET and FRED virtualization landed by 6.18, i.e. in time for the next
>> LTS.
>
>I love the plan!
>
>FRED is my top priority.  I’ll address all your comments, rebase onto
>kvm-x86/next (you have not updated yet :) ), and send out v5 at an
>appropriate time.
>
>Thanks!
>     Xin
>

FRED doesn't lean on CET... one could argue it leans on LASS, at least to a small extent, though.
Re: [PATCH v4 00/19] Enable FRED with KVM VMX
Posted by Xin Li 5 months, 3 weeks ago
On 6/24/2025 10:47 AM, H. Peter Anvin wrote:
> FRED doesn't lean on CET... one could argue it leans on LASS, at least to a small extent, though.

Probably I used a wrong verb "lean", "overlap" is better.
Re: [PATCH v4 00/19] Enable FRED with KVM VMX
Posted by H. Peter Anvin 5 months, 3 weeks ago
On June 24, 2025 11:02:41 AM PDT, Xin Li <xin@zytor.com> wrote:
>On 6/24/2025 10:47 AM, H. Peter Anvin wrote:
>> FRED doesn't lean on CET... one could argue it leans on LASS, at least to a small extent, though.
>
>Probably I used a wrong verb "lean", "overlap" is better.

I would personally say to the extent there is overlap it is the opposite direction (FRED helps enable kCET, but uCET is pretty much orthogonal.)