[PATCH v8 00/21] Enable FRED with KVM VMX

Xin Li (Intel) posted 21 patches 2 months ago
There is a newer version of this series
Documentation/virt/kvm/api.rst        |  21 +-
arch/x86/coco/sev/noinstr.c           |   4 +-
arch/x86/coco/sev/vc-handle.c         |   2 +-
arch/x86/include/asm/cpu_entry_area.h |  75 +++---
arch/x86/include/asm/kvm_host.h       |  13 +-
arch/x86/include/asm/msr-index.h      |   1 +
arch/x86/include/asm/vmx.h            |  48 +++-
arch/x86/include/uapi/asm/kvm.h       |   4 +-
arch/x86/kernel/cpu/common.c          |  10 +-
arch/x86/kernel/dumpstack_64.c        |  14 +-
arch/x86/kernel/fred.c                |   6 +-
arch/x86/kernel/traps.c               |   2 +-
arch/x86/kvm/cpuid.c                  |   1 +
arch/x86/kvm/kvm_cache_regs.h         |  15 ++
arch/x86/kvm/svm/svm.c                |   2 +-
arch/x86/kvm/vmx/capabilities.h       |  25 +-
arch/x86/kvm/vmx/nested.c             | 338 ++++++++++++++++++++++----
arch/x86/kvm/vmx/nested.h             |  22 ++
arch/x86/kvm/vmx/vmcs.h               |   1 +
arch/x86/kvm/vmx/vmcs12.c             |  19 ++
arch/x86/kvm/vmx/vmcs12.h             |  40 ++-
arch/x86/kvm/vmx/vmcs_shadow_fields.h |  37 ++-
arch/x86/kvm/vmx/vmx.c                | 247 +++++++++++++++++--
arch/x86/kvm/vmx/vmx.h                |  54 +++-
arch/x86/kvm/x86.c                    | 131 +++++++++-
arch/x86/kvm/x86.h                    |   8 +-
arch/x86/mm/cpu_entry_area.c          |  37 ++-
arch/x86/mm/fault.c                   |   2 +-
include/uapi/linux/kvm.h              |   1 +
29 files changed, 1036 insertions(+), 144 deletions(-)
[PATCH v8 00/21] Enable FRED with KVM VMX
Posted by Xin Li (Intel) 2 months ago
This patch set enables the Intel flexible return and event delivery
(FRED) architecture with KVM VMX to allow guests to utilize FRED.

The FRED architecture defines simple new transitions that change
privilege level (ring transitions). The FRED architecture was
designed with the following goals:

1) Improve overall performance and response time by replacing event
   delivery through the interrupt descriptor table (IDT event
   delivery) and event return by the IRET instruction with lower
   latency transitions.

2) Improve software robustness by ensuring that event delivery
   establishes the full supervisor context and that event return
   establishes the full user context.

The new transitions defined by the FRED architecture are FRED event
delivery and, for returning from events, two FRED return instructions.
FRED event delivery can effect a transition from ring 3 to ring 0, but
it is used also to deliver events incident to ring 0. One FRED
instruction (ERETU) effects a return from ring 0 to ring 3, while the
other (ERETS) returns while remaining in ring 0. Collectively, FRED
event delivery and the FRED return instructions are FRED transitions.


Intel VMX architecture is extended to run FRED guests, and the major
changes are:

1) New VMCS fields for FRED context management, which includes two new
event data VMCS fields, eight new guest FRED context VMCS fields and
eight new host FRED context VMCS fields.

2) VMX nested-exception support for proper virtualization of stack
levels introduced with FRED architecture.

Search for the latest FRED spec in most search engines with this search
pattern:

  site:intel.com FRED (flexible return and event delivery) specification


Although FRED and CET supervisor shadow stacks are independent CPU
features, FRED unconditionally includes FRED shadow stack pointer
MSRs IA32_FRED_SSP[0123], and IA32_FRED_SSP0 is just an alias of the
CET MSR IA32_PL0_SSP.  IOW, the state management of MSR IA32_PL0_SSP
becomes an overlap area, and Sean requested that FRED virtualization
to land after CET virtualization [1].

Since CET virtualization has landed with the release of v6.18-rc1, I'm
submitting v8 of FRED virtualization.

Changes in v8:
* Rebased on v6.18-rc1.
* Relocate secondary_vm_exit_controls to the last u64 padding field in
  vmcs12 (Isaku).
* Make the newly added FRED fields 64-bit aligned in vmcs12 (Isaku).
* Remove changes to Documentation/virt/kvm/x86/nested-vmx.rst.
* Update KVM_CAP_EXCEPTION_NESTED_FLAG, as the number used in v7 is
  occupied by another new cap.


Following is the link to v7 of this patch set:
https://lore.kernel.org/lkml/20250829153149.2871901-1-xin@zytor.com/


[1]: https://lore.kernel.org/kvm/ZvQaNRhrsSJTYji3@google.com/


Xin Li (18):
  KVM: VMX: Add support for the secondary VM exit controls
  KVM: VMX: Initialize VM entry/exit FRED controls in vmcs_config
  KVM: VMX: Disable FRED if FRED consistency checks fail
  KVM: VMX: Initialize VMCS FRED fields
  KVM: VMX: Set FRED MSR intercepts
  KVM: VMX: Save/restore guest FRED RSP0
  KVM: VMX: Add support for saving and restoring FRED MSRs
  KVM: x86: Add a helper to detect if FRED is enabled for a vCPU
  KVM: VMX: Virtualize FRED event_data
  KVM: VMX: Virtualize FRED nested exception tracking
  KVM: x86: Mark CR4.FRED as not reserved
  KVM: VMX: Dump FRED context in dump_vmcs()
  KVM: x86: Advertise support for FRED
  KVM: nVMX: Add support for the secondary VM exit controls
  KVM: nVMX: Add FRED VMCS fields to nested VMX context handling
  KVM: nVMX: Add FRED-related VMCS field checks
  KVM: nVMX: Add prerequisites to SHADOW_FIELD_R[OW] macros
  KVM: nVMX: Allow VMX FRED controls

Xin Li (Intel) (3):
  x86/cea: Prefix event stack names with ESTACK_
  x86/cea: Export API for per-CPU exception stacks for KVM
  KVM: x86: Save/restore the nested flag of an exception

 Documentation/virt/kvm/api.rst        |  21 +-
 arch/x86/coco/sev/noinstr.c           |   4 +-
 arch/x86/coco/sev/vc-handle.c         |   2 +-
 arch/x86/include/asm/cpu_entry_area.h |  75 +++---
 arch/x86/include/asm/kvm_host.h       |  13 +-
 arch/x86/include/asm/msr-index.h      |   1 +
 arch/x86/include/asm/vmx.h            |  48 +++-
 arch/x86/include/uapi/asm/kvm.h       |   4 +-
 arch/x86/kernel/cpu/common.c          |  10 +-
 arch/x86/kernel/dumpstack_64.c        |  14 +-
 arch/x86/kernel/fred.c                |   6 +-
 arch/x86/kernel/traps.c               |   2 +-
 arch/x86/kvm/cpuid.c                  |   1 +
 arch/x86/kvm/kvm_cache_regs.h         |  15 ++
 arch/x86/kvm/svm/svm.c                |   2 +-
 arch/x86/kvm/vmx/capabilities.h       |  25 +-
 arch/x86/kvm/vmx/nested.c             | 338 ++++++++++++++++++++++----
 arch/x86/kvm/vmx/nested.h             |  22 ++
 arch/x86/kvm/vmx/vmcs.h               |   1 +
 arch/x86/kvm/vmx/vmcs12.c             |  19 ++
 arch/x86/kvm/vmx/vmcs12.h             |  40 ++-
 arch/x86/kvm/vmx/vmcs_shadow_fields.h |  37 ++-
 arch/x86/kvm/vmx/vmx.c                | 247 +++++++++++++++++--
 arch/x86/kvm/vmx/vmx.h                |  54 +++-
 arch/x86/kvm/x86.c                    | 131 +++++++++-
 arch/x86/kvm/x86.h                    |   8 +-
 arch/x86/mm/cpu_entry_area.c          |  37 ++-
 arch/x86/mm/fault.c                   |   2 +-
 include/uapi/linux/kvm.h              |   1 +
 29 files changed, 1036 insertions(+), 144 deletions(-)


base-commit: 3a8660878839faadb4f1a6dd72c3179c1df56787
-- 
2.51.0
[syzbot ci] Re: Enable FRED with KVM VMX
Posted by syzbot ci 2 months ago
syzbot ci has tested the following series

[v8] Enable FRED with KVM VMX
https://lore.kernel.org/all/20251014010950.1568389-1-xin@zytor.com
* [PATCH v8 01/21] KVM: VMX: Add support for the secondary VM exit controls
* [PATCH v8 02/21] KVM: VMX: Initialize VM entry/exit FRED controls in vmcs_config
* [PATCH v8 03/21] KVM: VMX: Disable FRED if FRED consistency checks fail
* [PATCH v8 04/21] x86/cea: Prefix event stack names with ESTACK_
* [PATCH v8 05/21] x86/cea: Export API for per-CPU exception stacks for KVM
* [PATCH v8 06/21] KVM: VMX: Initialize VMCS FRED fields
* [PATCH v8 07/21] KVM: VMX: Set FRED MSR intercepts
* [PATCH v8 08/21] KVM: VMX: Save/restore guest FRED RSP0
* [PATCH v8 09/21] KVM: VMX: Add support for saving and restoring FRED MSRs
* [PATCH v8 10/21] KVM: x86: Add a helper to detect if FRED is enabled for a vCPU
* [PATCH v8 11/21] KVM: VMX: Virtualize FRED event_data
* [PATCH v8 12/21] KVM: VMX: Virtualize FRED nested exception tracking
* [PATCH v8 13/21] KVM: x86: Save/restore the nested flag of an exception
* [PATCH v8 14/21] KVM: x86: Mark CR4.FRED as not reserved
* [PATCH v8 15/21] KVM: VMX: Dump FRED context in dump_vmcs()
* [PATCH v8 16/21] KVM: x86: Advertise support for FRED
* [PATCH v8 17/21] KVM: nVMX: Add support for the secondary VM exit controls
* [PATCH v8 18/21] KVM: nVMX: Add FRED VMCS fields to nested VMX context handling
* [PATCH v8 19/21] KVM: nVMX: Add FRED-related VMCS field checks
* [PATCH v8 20/21] KVM: nVMX: Add prerequisites to SHADOW_FIELD_R[OW] macros
* [PATCH v8 21/21] KVM: nVMX: Allow VMX FRED controls

and found the following issue:
WARNING in vmread_error

Full report is available here:
https://ci.syzbot.org/series/706e7cbc-a7af-4357-af68-194e1c883968

***

WARNING in vmread_error

tree:      torvalds
URL:       https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux
base:      0d97f2067c166eb495771fede9f7b73999c67f66
arch:      amd64
compiler:  Debian clang version 20.1.8 (++20250708063551+0c9f909b7976-1~exp1~20250708183702.136), Debian LLD 20.1.8
config:    https://ci.syzbot.org/builds/0033ed68-ef22-4ef4-a630-0d101fb2cb8e/config
C repro:   https://ci.syzbot.org/findings/a19ef524-0065-4a83-abb5-c42a195ec916/c_repro
syz repro: https://ci.syzbot.org/findings/a19ef524-0065-4a83-abb5-c42a195ec916/syz_repro

------------[ cut here ]------------
vmread failed: field=281a
WARNING: CPU: 0 PID: 5954 at arch/x86/kvm/vmx/vmx.c:425 vmread_error+0x7e/0x90 arch/x86/kvm/vmx/vmx.c:425
Modules linked in:
CPU: 0 UID: 0 PID: 5954 Comm: syz.0.17 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
RIP: 0010:vmread_error+0x7e/0x90 arch/x86/kvm/vmx/vmx.c:425
Code: dc 63 8b 48 89 de 5b 5d e9 cf de cf ff e8 0a 62 68 00 c6 05 4c 67 30 0e 01 90 48 c7 c7 80 db 63 8b 48 89 de e8 03 7f 2b 00 90 <0f> 0b 90 90 eb 98 66 66 66 2e 0f 1f 84 00 00 00 00 00 90 90 90 90
RSP: 0018:ffffc90004d66f28 EFLAGS: 00010246
RAX: c305e84bb7041f00 RBX: 000000000000281a RCX: ffff888109e48000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000002
RBP: ffffc90004d674a8 R08: ffff888121024293 R09: 1ffff11024204852
R10: dffffc0000000000 R11: ffffed1024204853 R12: dffffc0000000000
R13: ffff8881152e0000 R14: ffff88811292a30f R15: ffff88811292a000
FS:  0000555576a3b500(0000) GS:ffff88818e70e000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 00000001bda0c000 CR4: 0000000000352ef0
Call Trace:
 <TASK>
 __vmcs_readl arch/x86/kvm/vmx/vmx_ops.h:111 [inline]
 vmcs_read64 arch/x86/kvm/vmx/vmx_ops.h:177 [inline]
 nested_vmx_enter_non_root_mode+0xa0ef/0xbb20 arch/x86/kvm/vmx/nested.c:3834
 nested_vmx_run+0x5f7/0xc40 arch/x86/kvm/vmx/nested.c:4053
 __vmx_handle_exit arch/x86/kvm/vmx/vmx.c:6821 [inline]
 vmx_handle_exit+0x10a4/0x18c0 arch/x86/kvm/vmx/vmx.c:6838
 vcpu_enter_guest arch/x86/kvm/x86.c:11575 [inline]
 vcpu_run+0x446f/0x6fb0 arch/x86/kvm/x86.c:11733
 kvm_arch_vcpu_ioctl_run+0xfc9/0x1940 arch/x86/kvm/x86.c:12072
 kvm_vcpu_ioctl+0x95c/0xe90 virt/kvm/kvm_main.c:4476
 vfs_ioctl fs/ioctl.c:51 [inline]
 __do_sys_ioctl fs/ioctl.c:597 [inline]
 __se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583
 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
 do_syscall_64+0xfa/0xfa0 arch/x86/entry/syscall_64.c:94
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f22ca18eec9
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fff1aacefa8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007f22ca3e5fa0 RCX: 00007f22ca18eec9
RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000005
RBP: 00007f22ca211f91 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007f22ca3e5fa0 R14: 00007f22ca3e5fa0 R15: 0000000000000003
 </TASK>


***

If these findings have caused you to resend the series or submit a
separate fix, please add the following tag to your commit message:
  Tested-by: syzbot@syzkaller.appspotmail.com

---
This report is generated by a bot. It may contain errors.
syzbot ci engineers can be reached at syzkaller@googlegroups.com.