[PATCH v5 00/33] KVM: x86: AVIC and local APIC fixes+cleanups

Sean Christopherson posted 33 patches 1 year, 3 months ago
Documentation/virt/kvm/x86/errata.rst |  11 +
arch/x86/include/asm/kvm-x86-ops.h    |   1 -
arch/x86/include/asm/kvm_host.h       |  52 +++-
arch/x86/kvm/lapic.c                  | 319 +++++++++++++++++-----
arch/x86/kvm/lapic.h                  |   2 +
arch/x86/kvm/svm/avic.c               | 372 ++++++++++++--------------
arch/x86/kvm/svm/nested.c             |   2 +-
arch/x86/kvm/svm/svm.c                |   8 +-
arch/x86/kvm/svm/svm.h                |  27 +-
arch/x86/kvm/vmx/vmx.c                |  58 +---
arch/x86/kvm/x86.c                    |  29 +-
11 files changed, 524 insertions(+), 357 deletions(-)
[PATCH v5 00/33] KVM: x86: AVIC and local APIC fixes+cleanups
Posted by Sean Christopherson 1 year, 3 months ago
Paolo,

This is a wholesale (and hopefully a drop-in) replacement for the patches
in `kvm-lapic-fix-and-cleanup`.  The changes for " KVM: x86: Inhibit APIC
memslot if x2APIC and AVIC are enabled" in v5 are relatively minor, but it
led to multiple conflicts in later patches, i.e. a patch-to-be-squashed
wasn't going to work.  Let me know if you've already done a lot of
massaging on your side, shouldn't be too difficult to generate patches to
go on top.

The first half or so patches fix semi-urgent, real-world relevant APICv
and AVIC bugs.

The second half fixes a variety of AVIC and optimized APIC map bugs
where KVM doesn't play nice with various edge cases that are
architecturally legal(ish), but are unlikely to occur in most real world
scenarios

v5:
  - Fix even more bugs! [Greg]
  - Collect reviews. [Maxim]
  - Don't use an inhibit flag for hybrid AVIC. [Maxim]
  - Fix an LDR cluster calc goof in the AVIC code. [Maxim]
  - Drop a redundant "ldr == 0" check. [Maxim]
  - Add helpers for logical vs. physical optimized map calcs. [Maxim]

v4:
  - https://lore.kernel.org/all/20221001005915.2041642-1-seanjc@google.com
  - Fix more bugs! [Alejandro]
  - Delete APIC memslot to inhibit xAVIC acceleration when x2APIC is
    enabled on AMD/SVM instead of using a "partial" inihbit. [Maxim]

v3:
  - https://lore.kernel.org/all/20220920233134.940511-1-seanjc@google.com
  - Collect reviews. [Paolo]
  - Drop "partial" x2APIC inhibit and instead delete the memslot.
    [Maxim, Suravee]
  - Skip logical mode updates for x2APIC, which just reuses the
    phys_map with some clever logic. [Suravee]
  - Add a fix for "nodecode write" traps. [Alejandro]

v2:
  - https://lore.kernel.org/all/20220903002254.2411750-1-seanjc@google.com
  - Collect reviews. [Li, Maxim]
  - Disable only MMIO access when x2APIC is enabled (instead of disabling
    all of AVIC). [Maxim]
  - Inhibit AVIC when logical IDs are aliased. [Maxim]
  - Tweak name of set_virtual_apic_mode() hook. [Maxim]
  - Straight up revert logical ID fastpath mess. [Maxim]
  - Reword changelog about skipping vCPU during logical setup. [Maxim]
  - Fix LDR updates on AVIC. [Maxim?]
  - Fix a nasty ISR caching bug.
  - Flush TLB when activating AVIC.

v1: https://lore.kernel.org/all/20220831003506.4117148-1-seanjc@google.com

Greg Edwards (1):
  KVM: x86: Allow APICv APIC ID inhibit to be cleared

Sean Christopherson (31):
  KVM: x86: Blindly get current x2APIC reg value on "nodecode write"
    traps
  KVM: x86: Purge "highest ISR" cache when updating APICv state
  KVM: SVM: Flush the "current" TLB when activating AVIC
  KVM: SVM: Process ICR on AVIC IPI delivery failure due to invalid
    target
  KVM: x86: Don't inhibit APICv/AVIC on xAPIC ID "change" if APIC is
    disabled
  KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to
    32-bit ID
  KVM: SVM: Don't put/load AVIC when setting virtual APIC mode
  KVM: x86: Handle APICv updates for APIC "mode" changes via request
  KVM: x86: Move APIC access page helper to common x86 code
  KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled
  KVM: SVM: Replace "avic_mode" enum with "x2avic_enabled" boolean
  KVM: SVM: Compute dest based on sender's x2APIC status for AVIC kick
  Revert "KVM: SVM: Use target APIC ID to complete x2AVIC IRQs when
    possible"
  KVM: SVM: Document that vCPU ID == APIC ID in AVIC kick fastpatch
  KVM: SVM: Add helper to perform final AVIC "kick" of single vCPU
  KVM: x86: Explicitly skip optimized logical map setup if vCPU's LDR==0
  KVM: x86: Explicitly track all possibilities for APIC map's logical
    modes
  KVM: x86: Skip redundant x2APIC logical mode optimized cluster setup
  KVM: x86: Disable APIC logical map if logical ID covers multiple MDAs
  KVM: x86: Disable APIC logical map if vCPUs are aliased in logical
    mode
  KVM: x86: Honor architectural behavior for aliased 8-bit APIC IDs
  KVM: x86: Inhibit APICv/AVIC if the optimized physical map is disabled
  KVM: SVM: Inhibit AVIC if vCPUs are aliased in logical mode
  KVM: SVM: Always update local APIC on writes to logical dest register
  KVM: SVM: Update svm->ldr_reg cache even if LDR is "bad"
  KVM: SVM: Require logical ID to be power-of-2 for AVIC entry
  KVM: SVM: Handle multiple logical targets in AVIC kick fastpath
  KVM: SVM: Ignore writes to Remote Read Data on AVIC write traps
  Revert "KVM: SVM: Do not throw warning when calling avic_vcpu_load on
    a running vcpu"
  KVM: x86: Track required APICv inhibits with variable, not callback
  KVM: x86: Add helpers to recalc physical vs. logical optimized APIC
    maps

Suravee Suthikulpanit (1):
  KVM: SVM: Fix x2APIC Logical ID calculation for
    avic_kick_target_vcpus_fast

 Documentation/virt/kvm/x86/errata.rst |  11 +
 arch/x86/include/asm/kvm-x86-ops.h    |   1 -
 arch/x86/include/asm/kvm_host.h       |  52 +++-
 arch/x86/kvm/lapic.c                  | 319 +++++++++++++++++-----
 arch/x86/kvm/lapic.h                  |   2 +
 arch/x86/kvm/svm/avic.c               | 372 ++++++++++++--------------
 arch/x86/kvm/svm/nested.c             |   2 +-
 arch/x86/kvm/svm/svm.c                |   8 +-
 arch/x86/kvm/svm/svm.h                |  27 +-
 arch/x86/kvm/vmx/vmx.c                |  58 +---
 arch/x86/kvm/x86.c                    |  29 +-
 11 files changed, 524 insertions(+), 357 deletions(-)


base-commit: c04ec04c0d15a51aa33660be175ed978beb8de0c
-- 
2.39.0.314.g84b9a713c41-goog
Re: [PATCH v5 00/33] KVM: x86: AVIC and local APIC fixes+cleanups
Posted by Suthikulpanit, Suravee 1 year, 2 months ago

On 1/6/2023 8:12 AM, Sean Christopherson wrote:
> Paolo,
> 
> This is a wholesale (and hopefully a drop-in) replacement for the patches
> in `kvm-lapic-fix-and-cleanup`.  The changes for " KVM: x86: Inhibit APIC
> memslot if x2APIC and AVIC are enabled" in v5 are relatively minor, but it
> led to multiple conflicts in later patches, i.e. a patch-to-be-squashed
> wasn't going to work.  Let me know if you've already done a lot of
> massaging on your side, shouldn't be too difficult to generate patches to
> go on top.
> 
> The first half or so patches fix semi-urgent, real-world relevant APICv
> and AVIC bugs.
> 
> The second half fixes a variety of AVIC and optimized APIC map bugs
> where KVM doesn't play nice with various edge cases that are
> architecturally legal(ish), but are unlikely to occur in most real world
> scenarios
> 
> v5:
>    - Fix even more bugs! [Greg]
>    - Collect reviews. [Maxim]
>    - Don't use an inhibit flag for hybrid AVIC. [Maxim]
>    - Fix an LDR cluster calc goof in the AVIC code. [Maxim]
>    - Drop a redundant "ldr == 0" check. [Maxim]
>    - Add helpers for logical vs. physical optimized map calcs. [Maxim]
> 
> v4:
>    - https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20221001005915.2041642-1-seanjc%40google.com&data=05%7C01%7Csuravee.suthikulpanit%40amd.com%7C266a9838dbb74b5ac6fb08daef832f65%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085643969977723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BUOZ4e1Ov%2BwPkKTSe81P1tPUtxpEr2uddkhMn73kLhQ%3D&reserved=0
>    - Fix more bugs! [Alejandro]
>    - Delete APIC memslot to inhibit xAVIC acceleration when x2APIC is
>      enabled on AMD/SVM instead of using a "partial" inihbit. [Maxim]
> 
> v3:
>    - https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20220920233134.940511-1-seanjc%40google.com&data=05%7C01%7Csuravee.suthikulpanit%40amd.com%7C266a9838dbb74b5ac6fb08daef832f65%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085643969977723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=usZ0dheeVyhBJxLP2ahA1hUlN64gwAPooWR9jnQcRyQ%3D&reserved=0
>    - Collect reviews. [Paolo]
>    - Drop "partial" x2APIC inhibit and instead delete the memslot.
>      [Maxim, Suravee]
>    - Skip logical mode updates for x2APIC, which just reuses the
>      phys_map with some clever logic. [Suravee]
>    - Add a fix for "nodecode write" traps. [Alejandro]
> 
> v2:
>    - https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20220903002254.2411750-1-seanjc%40google.com&data=05%7C01%7Csuravee.suthikulpanit%40amd.com%7C266a9838dbb74b5ac6fb08daef832f65%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085643969977723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=olaieXGtEfg8K6wwNHjHRa0aRit0npXnoxTmnfR%2FhqQ%3D&reserved=0
>    - Collect reviews. [Li, Maxim]
>    - Disable only MMIO access when x2APIC is enabled (instead of disabling
>      all of AVIC). [Maxim]
>    - Inhibit AVIC when logical IDs are aliased. [Maxim]
>    - Tweak name of set_virtual_apic_mode() hook. [Maxim]
>    - Straight up revert logical ID fastpath mess. [Maxim]
>    - Reword changelog about skipping vCPU during logical setup. [Maxim]
>    - Fix LDR updates on AVIC. [Maxim?]
>    - Fix a nasty ISR caching bug.
>    - Flush TLB when activating AVIC.
> 
> v1: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20220831003506.4117148-1-seanjc%40google.com&data=05%7C01%7Csuravee.suthikulpanit%40amd.com%7C266a9838dbb74b5ac6fb08daef832f65%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638085643969977723%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=d9my%2BQJgnq3673JLC1YFgVSNz1zlSDi7x537abArwS0%3D&reserved=0
> 
> Greg Edwards (1):
>    KVM: x86: Allow APICv APIC ID inhibit to be cleared
> 
> Sean Christopherson (31):
>    KVM: x86: Blindly get current x2APIC reg value on "nodecode write"
>      traps
>    KVM: x86: Purge "highest ISR" cache when updating APICv state
>    KVM: SVM: Flush the "current" TLB when activating AVIC
>    KVM: SVM: Process ICR on AVIC IPI delivery failure due to invalid
>      target
>    KVM: x86: Don't inhibit APICv/AVIC on xAPIC ID "change" if APIC is
>      disabled
>    KVM: x86: Don't inhibit APICv/AVIC if xAPIC ID mismatch is due to
>      32-bit ID
>    KVM: SVM: Don't put/load AVIC when setting virtual APIC mode
>    KVM: x86: Handle APICv updates for APIC "mode" changes via request
>    KVM: x86: Move APIC access page helper to common x86 code
>    KVM: x86: Inhibit APIC memslot if x2APIC and AVIC are enabled
>    KVM: SVM: Replace "avic_mode" enum with "x2avic_enabled" boolean
>    KVM: SVM: Compute dest based on sender's x2APIC status for AVIC kick
>    Revert "KVM: SVM: Use target APIC ID to complete x2AVIC IRQs when
>      possible"
>    KVM: SVM: Document that vCPU ID == APIC ID in AVIC kick fastpatch
>    KVM: SVM: Add helper to perform final AVIC "kick" of single vCPU
>    KVM: x86: Explicitly skip optimized logical map setup if vCPU's LDR==0
>    KVM: x86: Explicitly track all possibilities for APIC map's logical
>      modes
>    KVM: x86: Skip redundant x2APIC logical mode optimized cluster setup
>    KVM: x86: Disable APIC logical map if logical ID covers multiple MDAs
>    KVM: x86: Disable APIC logical map if vCPUs are aliased in logical
>      mode
>    KVM: x86: Honor architectural behavior for aliased 8-bit APIC IDs
>    KVM: x86: Inhibit APICv/AVIC if the optimized physical map is disabled
>    KVM: SVM: Inhibit AVIC if vCPUs are aliased in logical mode
>    KVM: SVM: Always update local APIC on writes to logical dest register
>    KVM: SVM: Update svm->ldr_reg cache even if LDR is "bad"
>    KVM: SVM: Require logical ID to be power-of-2 for AVIC entry
>    KVM: SVM: Handle multiple logical targets in AVIC kick fastpath
>    KVM: SVM: Ignore writes to Remote Read Data on AVIC write traps
>    Revert "KVM: SVM: Do not throw warning when calling avic_vcpu_load on
>      a running vcpu"
>    KVM: x86: Track required APICv inhibits with variable, not callback
>    KVM: x86: Add helpers to recalc physical vs. logical optimized APIC
>      maps
> 
> Suravee Suthikulpanit (1):
>    KVM: SVM: Fix x2APIC Logical ID calculation for
>      avic_kick_target_vcpus_fast
> 
>   Documentation/virt/kvm/x86/errata.rst |  11 +
>   arch/x86/include/asm/kvm-x86-ops.h    |   1 -
>   arch/x86/include/asm/kvm_host.h       |  52 +++-
>   arch/x86/kvm/lapic.c                  | 319 +++++++++++++++++-----
>   arch/x86/kvm/lapic.h                  |   2 +
>   arch/x86/kvm/svm/avic.c               | 372 ++++++++++++--------------
>   arch/x86/kvm/svm/nested.c             |   2 +-
>   arch/x86/kvm/svm/svm.c                |   8 +-
>   arch/x86/kvm/svm/svm.h                |  27 +-
>   arch/x86/kvm/vmx/vmx.c                |  58 +---
>   arch/x86/kvm/x86.c                    |  29 +-
>   11 files changed, 524 insertions(+), 357 deletions(-)
> 
> 
> base-commit: c04ec04c0d15a51aa33660be175ed978beb8de0c

For AMD AVIC / x2AVIC:

Tested-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Tested-by: Kishon VijayAbraham <kishon.vijayabraham@amd.com>

Thanks,
Suravee