[PATCH 00/18] KVM: x86: Fastpath cleanups and PMU prep work

Sean Christopherson posted 18 patches 2 months ago
arch/x86/include/asm/kvm_host.h |   3 +
arch/x86/kvm/lapic.c            |  59 ++++++++----
arch/x86/kvm/lapic.h            |   3 +-
arch/x86/kvm/pmu.c              | 155 +++++++++++++++++++++++++-------
arch/x86/kvm/pmu.h              |  60 ++-----------
arch/x86/kvm/svm/svm.c          |  14 ++-
arch/x86/kvm/vmx/nested.c       |   2 +-
arch/x86/kvm/vmx/pmu_intel.c    |   2 +-
arch/x86/kvm/vmx/vmx.c          |   2 +
arch/x86/kvm/x86.c              |  85 +++++-------------
arch/x86/kvm/x86.h              |   1 +
11 files changed, 218 insertions(+), 168 deletions(-)
[PATCH 00/18] KVM: x86: Fastpath cleanups and PMU prep work
Posted by Sean Christopherson 2 months ago
This is a prep series for the mediated PMU, and for Xin's series to add
support for the immediate forms of RDMSR and WRMSRNS (I'll post a v3 of
that series on top of this).

The first half cleans up a variety of warts and flaws in the VM-Exit fastpath
handlers.  The second half cleans up the PMU code related to "triggering"
instruction retired and branches retired events.  The end goal of the two
halves (other than general cleanup) is to be able bail from the fastpath when
using the mediated PMU and the guest is counting instructions retired, with
minimal overhead, e.g. without having to acquire SRCU.

Because the mediated PMU context switches PMU state _outside_ of the fastpath,
the mediated PMU won't be able to increment PMCs in the fastpath, and so won't
be able to skip emulated instructions in the fastpath if the vCPU is counting
instructions retired.

The last patch to handle INVD in the fastpath is a bit dubious.  It works just
fine, but it's dangerously close to "just because we can, doesn't mean we
should" territory.  I added INVD to the fastpath before I realized that
MSR_IA32_TSC_DEADLINE could be handled in the fastpath irrespective of the
VMX preemption timer, i.e. on AMD CPUs.  But being able to use INVD to test
the fastpath is still super convenient, as there are no side effects (unless
someone ran the test on bare metal :-D), no register constraints, and no
vCPU model requirements.  So, I kept it, because I couldn't come up with a
good reason not to.

Sean Christopherson (18):
  KVM: SVM: Skip fastpath emulation on VM-Exit if next RIP isn't valid
  KVM: x86: Add kvm_icr_to_lapic_irq() helper to allow for fastpath IPIs
  KVM: x86: Only allow "fast" IPIs in fastpath WRMSR(X2APIC_ICR) handler
  KVM: x86: Drop semi-arbitrary restrictions on IPI type in fastpath
  KVM: x86: Unconditionally handle MSR_IA32_TSC_DEADLINE in fastpath
    exits
  KVM: x86: Acquire SRCU in WRMSR fastpath iff instruction needs to be
    skipped
  KVM: x86: Unconditionally grab data from EDX:EAX in WRMSR fastpath
  KVM: x86: Fold WRMSR fastpath helpers into the main handler
  KVM: x86/pmu: Move kvm_init_pmu_capability() to pmu.c
  KVM: x86/pmu: Add wrappers for counting emulated instructions/branches
  KVM: x86/pmu: Calculate set of to-be-emulated PMCs at time of WRMSRs
  KVM: x86/pmu: Rename pmc_speculative_in_use() to
    pmc_is_locally_enabled()
  KVM: x86/pmu: Open code pmc_event_is_allowed() in its callers
  KVM: x86/pmu: Drop redundant check on PMC being globally enabled for
    emulation
  KVM: x86/pmu: Drop redundant check on PMC being locally enabled for
    emulation
  KVM: x86/pmu: Rename check_pmu_event_filter() to
    pmc_is_event_allowed()
  KVM: x86: Push acquisition of SRCU in fastpath into
    kvm_pmu_trigger_event()
  KVM: x86: Add a fastpath handler for INVD

 arch/x86/include/asm/kvm_host.h |   3 +
 arch/x86/kvm/lapic.c            |  59 ++++++++----
 arch/x86/kvm/lapic.h            |   3 +-
 arch/x86/kvm/pmu.c              | 155 +++++++++++++++++++++++++-------
 arch/x86/kvm/pmu.h              |  60 ++-----------
 arch/x86/kvm/svm/svm.c          |  14 ++-
 arch/x86/kvm/vmx/nested.c       |   2 +-
 arch/x86/kvm/vmx/pmu_intel.c    |   2 +-
 arch/x86/kvm/vmx/vmx.c          |   2 +
 arch/x86/kvm/x86.c              |  85 +++++-------------
 arch/x86/kvm/x86.h              |   1 +
 11 files changed, 218 insertions(+), 168 deletions(-)


base-commit: 196d9e72c4b0bd68b74a4ec7f52d248f37d0f030
-- 
2.50.1.565.gc32cd1483b-goog
Re: [PATCH 00/18] KVM: x86: Fastpath cleanups and PMU prep work
Posted by Mi, Dapeng 2 months ago
On 8/6/2025 3:05 AM, Sean Christopherson wrote:
> This is a prep series for the mediated PMU, and for Xin's series to add
> support for the immediate forms of RDMSR and WRMSRNS (I'll post a v3 of
> that series on top of this).
>
> The first half cleans up a variety of warts and flaws in the VM-Exit fastpath
> handlers.  The second half cleans up the PMU code related to "triggering"
> instruction retired and branches retired events.  The end goal of the two
> halves (other than general cleanup) is to be able bail from the fastpath when
> using the mediated PMU and the guest is counting instructions retired, with
> minimal overhead, e.g. without having to acquire SRCU.
>
> Because the mediated PMU context switches PMU state _outside_ of the fastpath,
> the mediated PMU won't be able to increment PMCs in the fastpath, and so won't
> be able to skip emulated instructions in the fastpath if the vCPU is counting
> instructions retired.
>
> The last patch to handle INVD in the fastpath is a bit dubious.  It works just
> fine, but it's dangerously close to "just because we can, doesn't mean we
> should" territory.  I added INVD to the fastpath before I realized that
> MSR_IA32_TSC_DEADLINE could be handled in the fastpath irrespective of the
> VMX preemption timer, i.e. on AMD CPUs.  But being able to use INVD to test
> the fastpath is still super convenient, as there are no side effects (unless
> someone ran the test on bare metal :-D), no register constraints, and no
> vCPU model requirements.  So, I kept it, because I couldn't come up with a
> good reason not to.
>
> Sean Christopherson (18):
>   KVM: SVM: Skip fastpath emulation on VM-Exit if next RIP isn't valid
>   KVM: x86: Add kvm_icr_to_lapic_irq() helper to allow for fastpath IPIs
>   KVM: x86: Only allow "fast" IPIs in fastpath WRMSR(X2APIC_ICR) handler
>   KVM: x86: Drop semi-arbitrary restrictions on IPI type in fastpath
>   KVM: x86: Unconditionally handle MSR_IA32_TSC_DEADLINE in fastpath
>     exits
>   KVM: x86: Acquire SRCU in WRMSR fastpath iff instruction needs to be
>     skipped
>   KVM: x86: Unconditionally grab data from EDX:EAX in WRMSR fastpath
>   KVM: x86: Fold WRMSR fastpath helpers into the main handler
>   KVM: x86/pmu: Move kvm_init_pmu_capability() to pmu.c
>   KVM: x86/pmu: Add wrappers for counting emulated instructions/branches
>   KVM: x86/pmu: Calculate set of to-be-emulated PMCs at time of WRMSRs
>   KVM: x86/pmu: Rename pmc_speculative_in_use() to
>     pmc_is_locally_enabled()
>   KVM: x86/pmu: Open code pmc_event_is_allowed() in its callers
>   KVM: x86/pmu: Drop redundant check on PMC being globally enabled for
>     emulation
>   KVM: x86/pmu: Drop redundant check on PMC being locally enabled for
>     emulation
>   KVM: x86/pmu: Rename check_pmu_event_filter() to
>     pmc_is_event_allowed()
>   KVM: x86: Push acquisition of SRCU in fastpath into
>     kvm_pmu_trigger_event()
>   KVM: x86: Add a fastpath handler for INVD
>
>  arch/x86/include/asm/kvm_host.h |   3 +
>  arch/x86/kvm/lapic.c            |  59 ++++++++----
>  arch/x86/kvm/lapic.h            |   3 +-
>  arch/x86/kvm/pmu.c              | 155 +++++++++++++++++++++++++-------
>  arch/x86/kvm/pmu.h              |  60 ++-----------
>  arch/x86/kvm/svm/svm.c          |  14 ++-
>  arch/x86/kvm/vmx/nested.c       |   2 +-
>  arch/x86/kvm/vmx/pmu_intel.c    |   2 +-
>  arch/x86/kvm/vmx/vmx.c          |   2 +
>  arch/x86/kvm/x86.c              |  85 +++++-------------
>  arch/x86/kvm/x86.h              |   1 +
>  11 files changed, 218 insertions(+), 168 deletions(-)
>
>
> base-commit: 196d9e72c4b0bd68b74a4ec7f52d248f37d0f030

Run PMU kselftests
(pmu_counters_test/pmu_event_filter_test/vmx_pmu_caps_test) on Sapphire
Rapids, no issue is found. Thanks.
Re: [PATCH 00/18] KVM: x86: Fastpath cleanups and PMU prep work
Posted by Sandipan Das 1 month, 4 weeks ago
On 06-08-2025 00:35, Sean Christopherson wrote:
> This is a prep series for the mediated PMU, and for Xin's series to add
> support for the immediate forms of RDMSR and WRMSRNS (I'll post a v3 of
> that series on top of this).
> 
> The first half cleans up a variety of warts and flaws in the VM-Exit fastpath
> handlers.  The second half cleans up the PMU code related to "triggering"
> instruction retired and branches retired events.  The end goal of the two
> halves (other than general cleanup) is to be able bail from the fastpath when
> using the mediated PMU and the guest is counting instructions retired, with
> minimal overhead, e.g. without having to acquire SRCU.
> 
> Because the mediated PMU context switches PMU state _outside_ of the fastpath,
> the mediated PMU won't be able to increment PMCs in the fastpath, and so won't
> be able to skip emulated instructions in the fastpath if the vCPU is counting
> instructions retired.
> 
> The last patch to handle INVD in the fastpath is a bit dubious.  It works just
> fine, but it's dangerously close to "just because we can, doesn't mean we
> should" territory.  I added INVD to the fastpath before I realized that
> MSR_IA32_TSC_DEADLINE could be handled in the fastpath irrespective of the
> VMX preemption timer, i.e. on AMD CPUs.  But being able to use INVD to test
> the fastpath is still super convenient, as there are no side effects (unless
> someone ran the test on bare metal :-D), no register constraints, and no
> vCPU model requirements.  So, I kept it, because I couldn't come up with a
> good reason not to.
> 
> Sean Christopherson (18):
>   KVM: SVM: Skip fastpath emulation on VM-Exit if next RIP isn't valid
>   KVM: x86: Add kvm_icr_to_lapic_irq() helper to allow for fastpath IPIs
>   KVM: x86: Only allow "fast" IPIs in fastpath WRMSR(X2APIC_ICR) handler
>   KVM: x86: Drop semi-arbitrary restrictions on IPI type in fastpath
>   KVM: x86: Unconditionally handle MSR_IA32_TSC_DEADLINE in fastpath
>     exits
>   KVM: x86: Acquire SRCU in WRMSR fastpath iff instruction needs to be
>     skipped
>   KVM: x86: Unconditionally grab data from EDX:EAX in WRMSR fastpath
>   KVM: x86: Fold WRMSR fastpath helpers into the main handler
>   KVM: x86/pmu: Move kvm_init_pmu_capability() to pmu.c
>   KVM: x86/pmu: Add wrappers for counting emulated instructions/branches
>   KVM: x86/pmu: Calculate set of to-be-emulated PMCs at time of WRMSRs
>   KVM: x86/pmu: Rename pmc_speculative_in_use() to
>     pmc_is_locally_enabled()
>   KVM: x86/pmu: Open code pmc_event_is_allowed() in its callers
>   KVM: x86/pmu: Drop redundant check on PMC being globally enabled for
>     emulation
>   KVM: x86/pmu: Drop redundant check on PMC being locally enabled for
>     emulation
>   KVM: x86/pmu: Rename check_pmu_event_filter() to
>     pmc_is_event_allowed()
>   KVM: x86: Push acquisition of SRCU in fastpath into
>     kvm_pmu_trigger_event()
>   KVM: x86: Add a fastpath handler for INVD
> 
>  arch/x86/include/asm/kvm_host.h |   3 +
>  arch/x86/kvm/lapic.c            |  59 ++++++++----
>  arch/x86/kvm/lapic.h            |   3 +-
>  arch/x86/kvm/pmu.c              | 155 +++++++++++++++++++++++++-------
>  arch/x86/kvm/pmu.h              |  60 ++-----------
>  arch/x86/kvm/svm/svm.c          |  14 ++-
>  arch/x86/kvm/vmx/nested.c       |   2 +-
>  arch/x86/kvm/vmx/pmu_intel.c    |   2 +-
>  arch/x86/kvm/vmx/vmx.c          |   2 +
>  arch/x86/kvm/x86.c              |  85 +++++-------------
>  arch/x86/kvm/x86.h              |   1 +
>  11 files changed, 218 insertions(+), 168 deletions(-)
> 
> 
> base-commit: 196d9e72c4b0bd68b74a4ec7f52d248f37d0f030

No issues observed with KVM Unit Tests on recent AMD platforms (Milan, Genoa and Turin).
Re: [PATCH 00/18] KVM: x86: Fastpath cleanups and PMU prep work
Posted by Sean Christopherson 1 month, 2 weeks ago
On Tue, 05 Aug 2025 12:05:08 -0700, Sean Christopherson wrote:
> This is a prep series for the mediated PMU, and for Xin's series to add
> support for the immediate forms of RDMSR and WRMSRNS (I'll post a v3 of
> that series on top of this).
> 
> The first half cleans up a variety of warts and flaws in the VM-Exit fastpath
> handlers.  The second half cleans up the PMU code related to "triggering"
> instruction retired and branches retired events.  The end goal of the two
> halves (other than general cleanup) is to be able bail from the fastpath when
> using the mediated PMU and the guest is counting instructions retired, with
> minimal overhead, e.g. without having to acquire SRCU.
> 
> [...]

Applied to kvm-x86 misc, thanks!

[01/18] KVM: SVM: Skip fastpath emulation on VM-Exit if next RIP isn't valid
        https://github.com/kvm-x86/linux/commit/0910dd7c9ad4
[02/18] KVM: x86: Add kvm_icr_to_lapic_irq() helper to allow for fastpath IPIs
        https://github.com/kvm-x86/linux/commit/15daa58e78ce
[03/18] KVM: x86: Only allow "fast" IPIs in fastpath WRMSR(X2APIC_ICR) handler
        https://github.com/kvm-x86/linux/commit/777414340085
[04/18] KVM: x86: Drop semi-arbitrary restrictions on IPI type in fastpath
        https://github.com/kvm-x86/linux/commit/aeeb4c7fff52
[05/18] KVM: x86: Unconditionally handle MSR_IA32_TSC_DEADLINE in fastpath exits
        https://github.com/kvm-x86/linux/commit/0a94b2042419
[06/18] KVM: x86: Acquire SRCU in WRMSR fastpath iff instruction needs to be skipped
        https://github.com/kvm-x86/linux/commit/aebcbb609773
[07/18] KVM: x86: Unconditionally grab data from EDX:EAX in WRMSR fastpath
        https://github.com/kvm-x86/linux/commit/aa2e4f029341
[08/18] KVM: x86: Fold WRMSR fastpath helpers into the main handler
        https://github.com/kvm-x86/linux/commit/d618fb4e43a0
[09/18] KVM: x86/pmu: Move kvm_init_pmu_capability() to pmu.c
        https://github.com/kvm-x86/linux/commit/a3e80bf73ee1
[10/18] KVM: x86/pmu: Add wrappers for counting emulated instructions/branches
        https://github.com/kvm-x86/linux/commit/43f5bea2639c
[11/18] KVM: x86/pmu: Calculate set of to-be-emulated PMCs at time of WRMSRs
        https://github.com/kvm-x86/linux/commit/5dfd498bad5f
[12/18] KVM: x86/pmu: Rename pmc_speculative_in_use() to pmc_is_locally_enabled()
        https://github.com/kvm-x86/linux/commit/6b6f1adc4332
[13/18] KVM: x86/pmu: Open code pmc_event_is_allowed() in its callers
        https://github.com/kvm-x86/linux/commit/e630bb52d27f
[14/18] KVM: x86/pmu: Drop redundant check on PMC being globally enabled for emulation
        https://github.com/kvm-x86/linux/commit/58baa649ea09
[15/18] KVM: x86/pmu: Drop redundant check on PMC being locally enabled for emulation
        https://github.com/kvm-x86/linux/commit/8709656514c1
[16/18] KVM: x86/pmu: Rename check_pmu_event_filter() to pmc_is_event_allowed()
        https://github.com/kvm-x86/linux/commit/3eced8b07bb9
[17/18] KVM: x86: Push acquisition of SRCU in fastpath into kvm_pmu_trigger_event()
        https://github.com/kvm-x86/linux/commit/8bb8b60c95c5
[18/18] KVM: x86: Add a fastpath handler for INVD
        https://github.com/kvm-x86/linux/commit/6c3d4b917995

--
https://github.com/kvm-x86/linux/tree/next