Series comparison

-[RFC v2 00/17] AMD: Add Secure AVIC Guest Support
+[PATCH v3 00/17] AMD: Add Secure AVIC Guest Support
 Introduction
 ------------
 Secure AVIC is a new hardware feature in the AMD64 architecture to
-allow SEV-SNP guests to prevent hypervisor from generating unexpected
+allow SEV-SNP guests to prevent the hypervisor from generating
-interrupts to a vCPU or otherwise violate architectural assumptions
+unexpected interrupts to a vCPU or otherwise violate architectural
-around APIC behavior.
+assumptions around APIC behavior.
 One of the significant differences from AVIC or emulated x2APIC is that
 Secure AVIC uses a guest-owned and managed APIC backing page. It also
 introduces additional fields in both the VMCB and the Secure AVIC backing
 page to aid the guest in limiting which interrupt vectors can be injected
 ...
 the guest APIC backing page which can be modified directly by the
 guest:
 a. ALLOWED_IRR
-ALLOWED_IRR vector indicates the interrupt vectors which the guest
+ALLOWED_IRR reg offset indicates the interrupt vectors which the guest
 allows the hypervisor to send. The combination of host-controlled
 REQUESTED_IRR vectors (part of VMCB) and ALLOWED_IRR is used by
 hardware to update the IRR vectors of the Guest APIC backing page.
 #Offset        #bits        Description
 ...
 b. NMI Request
 #Offset        #bits        Description
 h           0            Set by Guest to request Virtual NMI
+Guest need to set NMI Request register to allow the Hypervisor to
+inject vNMI to it.
 LAPIC Timer Support
 -------------------
-LAPIC timer is emulated by hypervisor. So, APIC_LVTT, APIC_TMICT and
+LAPIC timer is emulated by the hypervisor. So, APIC_LVTT, APIC_TMICT and
 APIC_TDCR, APIC_TMCCT APIC registers are not read/written to the guest
 APIC backing page and are communicated to the hypervisor using SVM_EXIT_MSR
 VMGEXIT.
 IPI Support
 ...
 -------------
 Secure AVIC enabled guest can kexec to another kernel which has Secure
 AVIC enabled, as the Hypervisor has Secure AVIC feature bit set in the
 sev_status.
-Driver Implementation Open Points
+Open Points
----------------------------------
+-----------
 The Secure AVIC driver only supports physical destination mode. If
 logical destination mode need to be supported, then a separate x2apic
 driver would be required for supporting logical destination mode.
-Setting of ALLOWED_IRR vectors is done from vector.c for IOAPIC and MSI
-interrupts. ALLOWED_IRR vector is not cleared when an interrupt vector
-migrates to different CPU. Using a cleaner approach to manage and
-configure allowed vectors needs more work.
 Testing
 -------
-This series is based on top of commit 0f966b199269 "Merge branch into tip/master:
+This series is based on top of commit 535bd326c565 "Merge branch into
-'x86/platform'" of tip/tip master branch.
+tip/master: 'x86/tdx'" of tip/tip master branch.
 Host Secure AVIC support patch series is at [1].
 Qemu support patch is at [2].
 ...
 ) Verified long run SCF TORTURE IPI test.
 [1] https://github.com/AMDESE/linux-kvm/tree/savic-host-latest
 [2] https://github.com/AMDESE/qemu/tree/secure-avic
-Change since v1
+Change since v2
-  - Added Kexec support.
+  - Removed RFC tag.
-  - Instead of doing a 2M aligned allocation for backing pages,
+  - Change config rule to not select AMD_SECURE_AVIC config if
-    allocate individual PAGE_SIZE pages for vCPUs.
+    AMD_MEM_ENCRYPT config is enabled.
-  - Instead of reading Extended Topology Enumeration CPUID, APIC_ID
+  - Fix broken backing page GFP_KERNEL allocation in setup_local_APIC().
-    value is read from Hv and updated in APIC backing page. Hv returned
+    Use alloc_percpu() for APIC backing pages allocation during Secure
-    ID is checked for any duplicates.
+    AVIC driver probe.
-  - Propagate all LVT* register reads and writes to Hv.
+  - Remove code to check for duplicate APIC_ID returned by the
-  - Check that Secure AVIC control MSR is not intercepted by Hv.
+    Hypervisor. Topology evaluation code already does that during boot.
-  - Fix EOI handling for level-triggered interrupts.
+  - Fix missing update_vector() callback invocation during vector
-  - Misc cleanups and commit log updates.
+    cleanup paths. Invoke update_vector() during setup and tearing down
     of a vector.
   - Reuse find_highest_vector() from kvm/lapic.c.
   - Change savic_register_gpa/savic_unregister_gpa() interface to be
     invoked only for the local CPU.
   - Misc cleanups.
-Kishon Vijay Abraham I (5):
+Kishon Vijay Abraham I (2):
   x86/apic: Support LAPIC timer for Secure AVIC
   x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC
-  x86/apic: Add support to send NMI IPI for Secure AVIC
   x86/sev: Enable NMI support for Secure AVIC
-  x86/sev: Indicate SEV-SNP guest supports Secure AVIC
-Neeraj Upadhyay (12):
+Neeraj Upadhyay (15):
   x86/apic: Add new driver for Secure AVIC
   x86/apic: Initialize Secure AVIC APIC backing page
   x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver
   x86/apic: Initialize APIC ID for Secure AVIC
   x86/apic: Add update_vector callback for Secure AVIC
   x86/apic: Add support to send IPI for Secure AVIC
+  x86/apic: Support LAPIC timer for Secure AVIC
+  x86/apic: Add support to send NMI IPI for Secure AVIC
   x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC
   x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests
   x86/apic: Handle EOI writes for SAVIC guests
   x86/apic: Add kexec support for Secure AVIC
   x86/apic: Enable Secure AVIC in Control MSR
   x86/sev: Prevent SECURE_AVIC_CONTROL MSR interception for Secure AVIC
     guests
+  x86/sev: Indicate SEV-SNP guest supports Secure AVIC
- arch/x86/Kconfig                    |  14 +
+ arch/x86/Kconfig                    |  13 +
- arch/x86/boot/compressed/sev.c      |   4 +-
+ arch/x86/boot/compressed/sev.c      |  10 +-
  arch/x86/coco/core.c                |   3 +
- arch/x86/coco/sev/core.c            | 145 +++++++-
+ arch/x86/coco/sev/core.c            | 131 +++++++-
- arch/x86/include/asm/apic.h         |   4 +
+ arch/x86/include/asm/apic-emul.h    |  28 ++
  arch/x86/include/asm/apic.h         |  12 +
  arch/x86/include/asm/apicdef.h      |   2 +
  arch/x86/include/asm/msr-index.h    |   9 +-
- arch/x86/include/asm/sev.h          |  10 +
+ arch/x86/include/asm/sev.h          |   8 +
  arch/x86/include/uapi/asm/svm.h     |   3 +
  arch/x86/kernel/apic/Makefile       |   1 +
- arch/x86/kernel/apic/apic.c         |   9 +
+ arch/x86/kernel/apic/apic.c         |   7 +
- arch/x86/kernel/apic/vector.c       |   8 +
+ arch/x86/kernel/apic/init.c         |   3 +
- arch/x86/kernel/apic/x2apic_savic.c | 530 ++++++++++++++++++++++++++++
+ arch/x86/kernel/apic/vector.c       |  53 +++-
  arch/x86/kernel/apic/x2apic_savic.c | 467 ++++++++++++++++++++++++++++
  arch/x86/kvm/lapic.c                |  23 +-
  include/linux/cc_platform.h         |   8 +
-files changed, 743 insertions(+), 7 deletions(-)
+files changed, 742 insertions(+), 39 deletions(-)
  create mode 100644 arch/x86/include/asm/apic-emul.h
  create mode 100644 arch/x86/kernel/apic/x2apic_savic.c
+base-commit: 535bd326c5657fe570f41b1f76941e449d9e2062
 base-commit: 0f966b1992694763de4dae6bdf817c5c1c6fc66d
 --
 .34.1

-[RFC v2 01/17] x86/apic: Add new driver for Secure AVIC
+[PATCH v3 01/17] x86/apic: Add new driver for Secure AVIC
 ...
 Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
+Changes since v2:
-Changes since v1:
+ - Do not autoselect AMD_SECURE_AVIC config when AMD_MEM_ENCRYPT config
- - Updated the commit log to highlight the behavior for the case when
+   is enabled. Make AMD_SECURE_AVIC depend on AMD_MEM_ENCRYPT.
-   guest SNP_FEATURES_PRESENT does not have SECURE AVIC set and
+ - Misc cleanups.
-   Hv has set the bit in SEV_STATUS.
+ arch/x86/Kconfig                    |  13 ++++
  - Select AMD_SECURE_AVIC config if AMD_MEM_ENCRYPT config is enabled.
  - Updated the config AMD_SECURE_AVIC description to highlight
    functional dependency on x2apic enablement.
  arch/x86/Kconfig                    |  14 ++++
  arch/x86/boot/compressed/sev.c      |   1 +
  arch/x86/coco/core.c                |   3 +
  arch/x86/include/asm/msr-index.h    |   4 +-
  arch/x86/kernel/apic/Makefile       |   1 +
- arch/x86/kernel/apic/x2apic_savic.c | 112 ++++++++++++++++++++++++++++
+ arch/x86/kernel/apic/x2apic_savic.c | 109 ++++++++++++++++++++++++++++
  include/linux/cc_platform.h         |   8 ++
-files changed, 142 insertions(+), 1 deletion(-)
+files changed, 138 insertions(+), 1 deletion(-)
  create mode 100644 arch/x86/kernel/apic/x2apic_savic.c
 diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/Kconfig
 +++ b/arch/x86/Kconfig
 @@ -XXX,XX +XXX,XX @@ config X86_X2APIC
-       If you don't know what to do here, say N.
+       If in doubt, say Y.
 +config AMD_SECURE_AVIC
 +    bool "AMD Secure AVIC"
-+    depends on X86_X2APIC
++    depends on AMD_MEM_ENCRYPT && X86_X2APIC
 +    help
-+      This enables AMD Secure AVIC support on guests that have this feature.
++      Enable this to get AMD Secure AVIC support on guests that have this feature.
 +
 +      AMD Secure AVIC provides hardware acceleration for performance sensitive
 +      APIC accesses and support for managing guest owned APIC state for SEV-SNP
 +      guests. Secure AVIC does not support xapic mode. It has functional
 +      dependency on x2apic being enabled in the guest.
 +
 +      If you don't know what to do here, say N.
 +
  config X86_POSTED_MSI
      bool "Enable MSI and MSI-x delivery by posted interrupts"
      depends on X86_64 && IRQ_REMAP
-@@ -XXX,XX +XXX,XX @@ config AMD_MEM_ENCRYPT
-     select X86_MEM_ENCRYPT
-     select UNACCEPTED_MEMORY
-     select CRYPTO_LIB_AESGCM
-+    select AMD_SECURE_AVIC
-     help
-       Say yes to enable support for the encryption of system memory.
-       This requires an AMD processor that supports Secure Memory
 diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/boot/compressed/sev.c
 +++ b/arch/x86/boot/compressed/sev.c
 @@ -XXX,XX +XXX,XX @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
 ...
  #define MSR_AMD64_SNP_VMSA_REG_PROT    BIT_ULL(MSR_AMD64_SNP_VMSA_REG_PROT_BIT)
  #define MSR_AMD64_SNP_SMT_PROT_BIT    17
  #define MSR_AMD64_SNP_SMT_PROT        BIT_ULL(MSR_AMD64_SNP_SMT_PROT_BIT)
 -#define MSR_AMD64_SNP_RESV_BIT        18
 +#define MSR_AMD64_SNP_SECURE_AVIC_BIT    18
-+#define MSR_AMD64_SNP_SECURE_AVIC     BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
++#define MSR_AMD64_SNP_SECURE_AVIC    BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
 +#define MSR_AMD64_SNP_RESV_BIT        19
  #define MSR_AMD64_SNP_RESERVED_MASK    GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
  #define MSR_AMD64_RMP_BASE        0xc0010132
  #define MSR_AMD64_RMP_END        0xc0010133
 diff --git a/arch/x86/kernel/apic/Makefile b/arch/x86/kernel/apic/Makefile
 ...
 +static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 +{
 +    return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
 +}
 +
-+static void x2apic_savic_send_IPI(int cpu, int vector)
++static void x2apic_savic_send_ipi(int cpu, int vector)
 +{
 +    u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
 +
 +    /* x2apic MSRs are special and need a special fence: */
 +    weak_wrmsr_fence();
 +    __x2apic_send_IPI_dest(dest, vector, APIC_DEST_PHYSICAL);
 +}
 +
-+static void
++static void __send_ipi_mask(const struct cpumask *mask, int vector, bool excl_self)
 +__send_IPI_mask(const struct cpumask *mask, int vector, int apic_dest)
 +{
 +    unsigned long query_cpu;
 +    unsigned long this_cpu;
 +    unsigned long flags;
 +
 ...
 +
 +    local_irq_save(flags);
 +
 +    this_cpu = smp_processor_id();
 +    for_each_cpu(query_cpu, mask) {
-+        if (apic_dest == APIC_DEST_ALLBUT && this_cpu == query_cpu)
++        if (excl_self && this_cpu == query_cpu)
 +            continue;
 +        __x2apic_send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu),
 +                       vector, APIC_DEST_PHYSICAL);
 +    }
 +    local_irq_restore(flags);
 +}
 +
-+static void x2apic_savic_send_IPI_mask(const struct cpumask *mask, int vector)
++static void x2apic_savic_send_ipi_mask(const struct cpumask *mask, int vector)
 +{
-+    __send_IPI_mask(mask, vector, APIC_DEST_ALLINC);
++    __send_ipi_mask(mask, vector, false);
 +}
 +
-+static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, int vector)
++static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, int vector)
 +{
-+    __send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
++    __send_ipi_mask(mask, vector, true);
 +}
 +
 +static int x2apic_savic_probe(void)
 +{
 +    if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
 ...
 +    if (!x2apic_mode) {
 +        pr_err("Secure AVIC enabled in non x2APIC mode\n");
 +        snp_abort();
 +    }
 +
-+    pr_info("Secure AVIC Enabled\n");
-+
 +    return 1;
 +}
 +
 +static struct apic apic_x2apic_savic __ro_after_init = {
 +
 ...
 +    .x2apic_set_max_apicid        = true,
 +    .get_apic_id            = x2apic_get_apic_id,
 +
 +    .calc_dest_apicid        = apic_default_calc_apicid,
 +
-+    .send_IPI            = x2apic_savic_send_IPI,
++    .send_IPI            = x2apic_savic_send_ipi,
-+    .send_IPI_mask            = x2apic_savic_send_IPI_mask,
++    .send_IPI_mask            = x2apic_savic_send_ipi_mask,
-+    .send_IPI_mask_allbutself    = x2apic_savic_send_IPI_mask_allbutself,
++    .send_IPI_mask_allbutself    = x2apic_savic_send_ipi_mask_allbutself,
 +    .send_IPI_allbutself        = x2apic_send_IPI_allbutself,
 +    .send_IPI_all            = x2apic_send_IPI_all,
 +    .send_IPI_self            = x2apic_send_IPI_self,
 +    .nmi_to_offline_cpu        = true,
 +
 ...

-[RFC v2 02/17] x86/apic: Initialize Secure AVIC APIC backing page
+[PATCH v3 02/17] x86/apic: Initialize Secure AVIC APIC backing page
 With Secure AVIC, the APIC backing page is owned and managed by guest.
 Allocate and initialize APIC backing page for all guest CPUs.
-The NPT entry for the vCPU's APIC backing page must always be present
+The NPT entry for a vCPU's APIC backing page must always be present
 when the vCPU is running in order for Secure AVIC to function. A
 VMEXIT_BUSY is returned on VMRUN and the vCPU cannot be resumed if
 the NPT entry for the APIC backing page is not present. Notify GPA of
 the vCPU's APIC backing page to the hypervisor by using the
 SVM_VMGEXIT_SECURE_AVIC GHCB protocol event. Before executing VMRUN,
 ...
 Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - Updated commit log.
+ - Fix broken AP bringup due to GFP_KERNEL allocation in setup()
- - Allocate APIC backing page for each CPU as a separate PAGE_SIZE
+   callback.
-   allocation with GFP_KERNEL flag.
+ - Define apic_page struct and allocate per CPU API backing pages
- - Update the GPA registeration API as per the latest GHCB spec updates
+   for all CPUs in Secure AVIC driver probe.
-   for Secure AVIC GHCB protocol event (yet to be published).
+ - Change savic_register_gpa() to only allow local CPU GPA
-   Corresponding KVM support is here:
+   registration.
-   https://github.com/AMDESE/linux-kvm/commit/5fbf231861207edf73bb31742f75e22cae18607b
+ - Misc cleanups.
- - Remove savic_setup_done variable.
- - Removed initialization of LVT* regs in backing page from Hv values.
+ arch/x86/coco/sev/core.c            | 27 +++++++++++++++++++
-   These regs will reads/writes will be propagated to Hv in subsequent
+ arch/x86/coco/sev/core.c            | 27 +++++++++++++++++++
    patches.
  - Move savic_ghcb_msr_read() definition to a later patch where it will
    be first used.
  arch/x86/coco/sev/core.c            | 32 +++++++++++++++++++++++++++
  arch/x86/include/asm/apic.h         |  1 +
- arch/x86/include/asm/sev.h          |  3 +++
+ arch/x86/include/asm/sev.h          |  2 ++
  arch/x86/include/uapi/asm/svm.h     |  3 +++
  arch/x86/kernel/apic/apic.c         |  2 ++
- arch/x86/kernel/apic/x2apic_savic.c | 34 +++++++++++++++++++++++++++++
+ arch/x86/kernel/apic/x2apic_savic.c | 42 +++++++++++++++++++++++++++++
-files changed, 75 insertions(+)
+files changed, 77 insertions(+)
 diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/coco/sev/core.c
 +++ b/arch/x86/coco/sev/core.c
 @@ -XXX,XX +XXX,XX @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
      return ret;
  }
-+/*
++enum es_result savic_register_gpa(u64 gpa)
 + * Register GPA of the Secure AVIC backing page.
 + *
 + * @apic_id: APIC ID of the vCPU. Use -1ULL for the current vCPU
 + *           doing the call.
 + * @gpa    : GPA of the Secure AVIC backing page.
 + */
 +enum es_result savic_register_gpa(u64 apic_id, u64 gpa)
 +{
 +    struct ghcb_state state;
 +    struct es_em_ctxt ctxt;
 +    unsigned long flags;
++    enum es_result res;
 +    struct ghcb *ghcb;
-+    int ret = 0;
 +
 +    local_irq_save(flags);
 +
 +    ghcb = __sev_get_ghcb(&state);
 +
 +    vc_ghcb_invalidate(ghcb);
 +
-+    ghcb_set_rax(ghcb, apic_id);
++    /* Register GPA for the local CPU */
 +    ghcb_set_rax(ghcb, -1ULL);
 +    ghcb_set_rbx(ghcb, gpa);
-+    ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SECURE_AVIC,
++    res = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SECURE_AVIC,
 +            SVM_VMGEXIT_SECURE_AVIC_REGISTER_GPA, 0);
 +
 +    __sev_put_ghcb(&state);
 +
 +    local_irq_restore(flags);
-+    return ret;
++
 +    return res;
 +}
 +
  static void snp_register_per_cpu_ghcb(void)
  {
      struct sev_es_runtime_data *data;
 ...
 +++ b/arch/x86/include/asm/sev.h
 @@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
  void __init snp_secure_tsc_prepare(void);
  void __init snp_secure_tsc_init(void);
-+enum es_result savic_register_gpa(u64 apic_id, u64 gpa);
++enum es_result savic_register_gpa(u64 gpa);
  #else    /* !CONFIG_AMD_MEM_ENCRYPT */
 @@ -XXX,XX +XXX,XX @@ static inline int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_
                       struct snp_guest_request_ioctl *rio) { return -ENODEV; }
  static inline void __init snp_secure_tsc_prepare(void) { }
  static inline void __init snp_secure_tsc_init(void) { }
-+static inline enum es_result savic_register_gpa(u64 apic_id,
++static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
 +                        u64 gpa) { return ES_UNSUPPORTED; }
  #endif    /* CONFIG_AMD_MEM_ENCRYPT */
 diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h
 index XXXXXXX..XXXXXXX 100644
 ...
  #include <asm/apic.h>
  #include <asm/sev.h>
  #include "local.h"
-+static DEFINE_PER_CPU(void *, apic_backing_page);
++/* APIC_EILVTn(3) is the last defined APIC register. */
 +#define NR_APIC_REGS    (APIC_EILVTn(4) >> 2)
 +
 +struct apic_page {
 +    union {
 +        u32    regs[NR_APIC_REGS];
 +        u8    bytes[PAGE_SIZE];
 +    };
 +} __aligned(PAGE_SIZE);
 +
 +static struct apic_page __percpu *apic_page __ro_after_init;
 +
  static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
  {
      return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
-@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, in
+@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, in
-     __send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
+     __send_ipi_mask(mask, vector, true);
  }
 +static void x2apic_savic_setup(void)
 +{
 +    void *backing_page;
 +    enum es_result ret;
 +    unsigned long gpa;
 +
-+    if (this_cpu_read(apic_backing_page))
++    backing_page = this_cpu_ptr(apic_page);
 +        return;
 +
 +    backing_page = kzalloc(PAGE_SIZE, GFP_KERNEL);
 +    if (!backing_page)
 +        snp_abort();
 +    this_cpu_write(apic_backing_page, backing_page);
 +    gpa = __pa(backing_page);
 +
 +    /*
-+     * The NPT entry for the vCPU's APIC backing page must always be
++     * The NPT entry for a vCPU's APIC backing page must always be
 +     * present when the vCPU is running in order for Secure AVIC to
 +     * function. A VMEXIT_BUSY is returned on VMRUN and the vCPU cannot
 +     * be resumed if the NPT entry for the APIC backing page is not
 +     * present. Notify GPA of the vCPU's APIC backing page to the
 +     * hypervisor by calling savic_register_gpa(). Before executing
 +     * VMRUN, the hypervisor makes use of this information to make sure
 +     * the APIC backing page is mapped in NPT.
 +     */
-+    ret = savic_register_gpa(-1ULL, gpa);
++    ret = savic_register_gpa(gpa);
 +    if (ret != ES_OK)
 +        snp_abort();
 +}
 +
  static int x2apic_savic_probe(void)
  {
      if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
+@@ -XXX,XX +XXX,XX @@ static int x2apic_savic_probe(void)
+         snp_abort();
+     }
++    apic_page = alloc_percpu(struct apic_page);
++    if (!apic_page)
++        snp_abort();
++
+     return 1;
+ }
 @@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
      .name                = "secure avic x2apic",
      .probe                = x2apic_savic_probe,
      .acpi_madt_oem_check        = x2apic_savic_acpi_madt_oem_check,
 +    .setup                = x2apic_savic_setup,
      .dest_mode_logical        = false,
 --
 .34.1

-[RFC v2 03/17] x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver
+[PATCH v3 03/17] x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver
+Add read() and write() APIC callback functions to read and write x2APIC
+registers directly from the guest APIC backing page of a vCPU.
 The x2APIC registers are mapped at an offset within the guest APIC
 backing page which is same as their x2APIC MMIO offset. Secure AVIC
 adds new registers such as ALLOWED_IRRs (which are at 4-byte offset
 within the IRR register offset range) and NMI_REQ to the APIC register
 space.
-Add read() and write() APIC callback functions to read and write x2APIC
+When Secure AVIC is enabled, guest's rdmsr/wrmsr of APIC registers
-registers directly from the guest APIC backing page.
+result in VC exception (for non-accelerated register accesses) with
+error code VMEXIT_AVIC_NOACCEL. The VC exception handler can read/write
-When Secure AVIC is enabled, rdmsr/wrmsr of APIC registers result in
+the x2APIC register in the guest APIC backing page to complete the
-VC exception (for non-accelerated register accesses). The #VC
+rdmsr/wrmsr. Since doing this would increase the latency of accessing
-exception handler can read/write the x2APIC register in the guest
+x2APIC registers, instead of doing rdmsr/wrmsr based reg accesses
-APIC backing page. Since doing this would increase the latency of
+and handling reads/writes in VC exception, directly read/write APIC
-accessing x2APIC registers, instead of doing rdmsr/wrmsr based
+registers from/to the guest APIC backing page of the vCPU in read()
-accesses and handling apic register reads/writes in VC
+and write() callbacks of the Secure AVIC APIC driver.
 VMEXIT_AVIC_NOACCEL error condition, the read() and write()
 callbacks of Secure AVIC driver directly read/write APIC register
 from/to the guest APIC backing page.
 Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
+ - Use this_cpu_ptr() instead of type casting in get_reg() and
- - APIC_ID reg write is not allowed.
+   set_reg().
  - Put information about not using #VC exception path for register
    reads/writes as comments.
  - So not read backing page if WARN_ONCE is triggered for misaligned
    reads.
  - Cleanups.
  arch/x86/include/asm/apicdef.h      |   2 +
- arch/x86/kernel/apic/x2apic_savic.c | 120 +++++++++++++++++++++++++++-
+ arch/x86/kernel/apic/x2apic_savic.c | 116 +++++++++++++++++++++++++++-
-files changed, 120 insertions(+), 2 deletions(-)
+files changed, 116 insertions(+), 2 deletions(-)
 diff --git a/arch/x86/include/asm/apicdef.h b/arch/x86/include/asm/apicdef.h
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/include/asm/apicdef.h
 +++ b/arch/x86/include/asm/apicdef.h
 ...
  #include <asm/sev.h>
 @@ -XXX,XX +XXX,XX @@ static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
      return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
  }
-+static inline u32 get_reg(char *page, int reg)
++static __always_inline u32 get_reg(unsigned int offset)
 +{
-+    return READ_ONCE(*((u32 *)(page + reg)));
++    return READ_ONCE(this_cpu_ptr(apic_page)->regs[offset >> 2]);
 +}
 +
-+static inline void set_reg(char *page, int reg, u32 val)
++static __always_inline void set_reg(unsigned int offset, u32 val)
 +{
-+    WRITE_ONCE(*((u32 *)(page + reg)), val);
++    WRITE_ONCE(this_cpu_ptr(apic_page)->regs[offset >> 2], val);
 +}
 +
-+#define SAVIC_ALLOWED_IRR_OFFSET    0x204
++#define SAVIC_ALLOWED_IRR    0x204
 +
 +static u32 x2apic_savic_read(u32 reg)
 +{
-+    void *backing_page = this_cpu_read(apic_backing_page);
-+
 +    /*
-+     * When Secure AVIC is enabled, rdmsr/wrmsr of APIC registers result in
++     * When Secure AVIC is enabled, rdmsr/wrmsr of APIC registers
-+     * #VC exception (for non-accelerated register accesses). The #VC
++     * result in VC exception (for non-accelerated register accesses)
-+     * exception handler can read/write the x2APIC register in the guest
++     * with VMEXIT_AVIC_NOACCEL error code. The VC exception handler
-+     * APIC backing page. Since doing this would increase the latency of
++     * can read/write the x2APIC register in the guest APIC backing page.
-+     * accessing x2APIC registers, instead of doing rdmsr/wrmsr based
++     * Since doing this would increase the latency of accessing x2APIC
-+     * accesses and handling apic register reads/writes in
++     * registers, instead of doing rdmsr/wrmsr based accesses and
-+     * #VC VMEXIT_AVIC_NOACCEL error condition, the read() and write()
++     * handling apic register reads/writes in VC exception, the read()
-+     * callbacks of Secure AVIC driver directly read/write APIC register
++     * and write() callbacks directly read/write APIC register from/to
-+     * from/to the guest APIC backing page.
++     * the vCPU APIC backing page.
 +     */
 +    switch (reg) {
 +    case APIC_LVTT:
 +    case APIC_TMICT:
 +    case APIC_TMCCT:
 ...
 +    case APIC_EFEAT:
 +    case APIC_ECTRL:
 +    case APIC_SEOI:
 +    case APIC_IER:
 +    case APIC_EILVTn(0) ... APIC_EILVTn(3):
-+        return get_reg(backing_page, reg);
++        return get_reg(reg);
 +    case APIC_ISR ... APIC_ISR + 0x70:
 +    case APIC_TMR ... APIC_TMR + 0x70:
 +        if (WARN_ONCE(!IS_ALIGNED(reg, 16),
-+                  "Reg offset 0x%x not aligned at 16 bytes", reg))
++                  "APIC reg read offset 0x%x not aligned at 16 bytes", reg))
 +            return 0;
-+        return get_reg(backing_page, reg);
++        return get_reg(reg);
 +    /* IRR and ALLOWED_IRR offset range */
 +    case APIC_IRR ... APIC_IRR + 0x74:
 +        /*
 +         * Either aligned at 16 bytes for valid IRR reg offset or a
 +         * valid Secure AVIC ALLOWED_IRR offset.
 +         */
 +        if (WARN_ONCE(!(IS_ALIGNED(reg, 16) ||
-+                IS_ALIGNED(reg - SAVIC_ALLOWED_IRR_OFFSET, 16)),
++                IS_ALIGNED(reg - SAVIC_ALLOWED_IRR, 16)),
-+                  "Misaligned IRR/ALLOWED_IRR reg offset 0x%x", reg))
++                  "Misaligned IRR/ALLOWED_IRR APIC reg read offset 0x%x", reg))
 +            return 0;
-+        return get_reg(backing_page, reg);
++        return get_reg(reg);
 +    default:
 +        pr_err("Permission denied: read of Secure AVIC reg offset 0x%x\n", reg);
 +        return 0;
 +    }
 +}
 +
-+#define SAVIC_NMI_REQ_OFFSET        0x278
++#define SAVIC_NMI_REQ        0x278
 +
 +static void x2apic_savic_write(u32 reg, u32 data)
 +{
-+    void *backing_page = this_cpu_read(apic_backing_page);
-+
 +    switch (reg) {
 +    case APIC_LVTT:
 +    case APIC_LVT0:
 +    case APIC_LVT1:
 +    case APIC_TMICT:
 +    case APIC_TDCR:
 +    case APIC_SELF_IPI:
 +    case APIC_TASKPRI:
 +    case APIC_EOI:
 +    case APIC_SPIV:
-+    case SAVIC_NMI_REQ_OFFSET:
++    case SAVIC_NMI_REQ:
 +    case APIC_ESR:
 +    case APIC_ICR:
 +    case APIC_LVTTHMR:
 +    case APIC_LVTPC:
 +    case APIC_LVTERR:
 +    case APIC_ECTRL:
 +    case APIC_SEOI:
 +    case APIC_IER:
 +    case APIC_EILVTn(0) ... APIC_EILVTn(3):
-+        set_reg(backing_page, reg, data);
++        set_reg(reg, data);
 +        break;
 +    /* ALLOWED_IRR offsets are writable */
-+    case SAVIC_ALLOWED_IRR_OFFSET ... SAVIC_ALLOWED_IRR_OFFSET + 0x70:
++    case SAVIC_ALLOWED_IRR ... SAVIC_ALLOWED_IRR + 0x70:
-+        if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR_OFFSET, 16)) {
++        if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR, 16)) {
-+            set_reg(backing_page, reg, data);
++            set_reg(reg, data);
 +            break;
 +        }
 +        fallthrough;
 +    default:
 +        pr_err("Permission denied: write to Secure AVIC reg offset 0x%x\n", reg);
 +    }
 +}
 +
- static void x2apic_savic_send_IPI(int cpu, int vector)
+ static void x2apic_savic_send_ipi(int cpu, int vector)
  {
      u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
 @@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
      .send_IPI_self            = x2apic_send_IPI_self,
      .nmi_to_offline_cpu        = true,
 ...

-[RFC v2 04/17] x86/apic: Initialize APIC ID for Secure AVIC
+[PATCH v3 04/17] x86/apic: Initialize APIC ID for Secure AVIC
 Initialize the APIC ID in the Secure AVIC APIC backing page with
-the APIC_ID msr value read from Hypervisor. Maintain a hashmap to
+the APIC_ID msr value read from Hypervisor. CPU topology evaluation
-check and report same APIC_ID value returned by Hypervisor for two
+later during boot would catch and report any duplicate APIC ID for
-different vCPUs.
+two CPUs.
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
  - Drop duplicate APIC ID checks.
- - Do not read APIC_ID from CPUID. Read APIC_ID from Hv and check for
+ arch/x86/kernel/apic/x2apic_savic.c | 13 +++++++++++++
-   duplicates.
+file changed, 13 insertions(+)
  - Add a more user-friendly log message on detecting duplicate APIC
    IDs.
  arch/x86/kernel/apic/x2apic_savic.c | 59 +++++++++++++++++++++++++++++
 file changed, 59 insertions(+)
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, in
- #include <linux/cc_platform.h>
+     __send_ipi_mask(mask, vector, true);
  #include <linux/percpu-defs.h>
  #include <linux/align.h>
 +#include <linux/sizes.h>
 +#include <linux/llist.h>
  #include <asm/apic.h>
  #include <asm/sev.h>
@@ -XXX,XX +XXX,XX @@
  static DEFINE_PER_CPU(void *, apic_backing_page);
 +struct apic_id_node {
 +     struct llist_node node;
 +     u32 apic_id;
 +     int cpu;
 +};
 +
 +static DEFINE_PER_CPU(struct apic_id_node, apic_id_node);
 +
 +static struct llist_head *apic_id_map;
 +
  static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
  {
      return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, in
      __send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
  }
-+static void init_backing_page(void *backing_page)
++static void init_apic_page(void)
 +{
-+    struct apic_id_node *next_node, *this_cpu_node;
-+    unsigned int apic_map_slot;
 +    u32 apic_id;
-+    int cpu;
 +
 +    /*
-+     * Before Secure AVIC is enabled, APIC msr reads are
++     * Before Secure AVIC is enabled, APIC msr reads are intercepted.
-+     * intercepted. APIC_ID msr read returns the value
++     * APIC_ID msr read returns the value from the Hypervisor.
 +     * from hv.
 +     */
 +    apic_id = native_apic_msr_read(APIC_ID);
-+    set_reg(backing_page, APIC_ID, apic_id);
++    set_reg(APIC_ID, apic_id);
 +
 +    if (!apic_id_map)
 +        return;
 +
 +    cpu = smp_processor_id();
 +    this_cpu_node = &per_cpu(apic_id_node, cpu);
 +    this_cpu_node->apic_id = apic_id;
 +    this_cpu_node->cpu = cpu;
 +    /*
 +     * In common case, apic_ids for CPUs are sequentially numbered.
 +     * So, each CPU should hash to a different slot in the apic id
 +     * map.
 +     */
 +    apic_map_slot = apic_id % nr_cpu_ids;
 +    llist_add(&this_cpu_node->node, &apic_id_map[apic_map_slot]);
 +    /* Each CPU checks only its next nodes for duplicates. */
 +    llist_for_each_entry(next_node, this_cpu_node->node.next, node) {
 +        if (WARN_ONCE(next_node->apic_id == apic_id,
 +                  "Duplicate APIC %u for cpu %d and cpu %d. IPI handling will suffer!",
 +                  apic_id, cpu, next_node->cpu))
 +            break;
 +    }
 +}
 +
  static void x2apic_savic_setup(void)
  {
      void *backing_page;
-@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
+     enum es_result ret;
-     if (!backing_page)
+     unsigned long gpa;
-         snp_abort();
-     this_cpu_write(apic_backing_page, backing_page);
++    init_apic_page();
-+    init_backing_page(backing_page);
+     backing_page = this_cpu_ptr(apic_page);
      gpa = __pa(backing_page);
-     /*
-@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
- static int x2apic_savic_probe(void)
- {
-+    int i;
-+
-     if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
-         return 0;
-@@ -XXX,XX +XXX,XX @@ static int x2apic_savic_probe(void)
-         snp_abort();
-     }
-+    apic_id_map = kvmalloc(nr_cpu_ids * sizeof(*apic_id_map), GFP_KERNEL);
-+
-+    if (apic_id_map)
-+        for (i = 0; i < nr_cpu_ids; i++)
-+            init_llist_head(&apic_id_map[i]);
-+
-     pr_info("Secure AVIC Enabled\n");
-     return 1;
 --
 .34.1

-[RFC v2 05/17] x86/apic: Add update_vector callback for Secure AVIC
+[PATCH v3 05/17] x86/apic: Add update_vector callback for Secure AVIC
 Add update_vector callback to set/clear ALLOWED_IRR field in
-the APIC backing page. The ALLOWED_IRR field indicates the
+a vCPU's APIC backing page for external vectors. The ALLOWED_IRR
-interrupt vectors which the guest allows the hypervisor to
+field indicates the interrupt vectors which the guest allows the
-send (typically for emulated devices). Interrupt vectors used
+hypervisor to send (typically for emulated devices). Interrupt
-exclusively by the guest itself (like IPI vectors) should not
+vectors used exclusively by the guest itself and the vectors which
-be allowed to be injected into the guest for security reasons.
+are not emulated by the hypervisor, such as IPI vectors, are part
-The update_vector callback is invoked from APIC vector domain
+of system vectors and are not set in the ALLOWED_IRR.
 whenever a vector is allocated, freed or moved.
 Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - No change.
+ - Associate update_vector() invocation with vector allocation/free
+   calls.
- arch/x86/include/asm/apic.h         |  2 ++
+ - Cleanup and simplify vector bitmap calculation for ALLOWED_IRR.
- arch/x86/kernel/apic/vector.c       |  8 ++++++++
- arch/x86/kernel/apic/x2apic_savic.c | 21 +++++++++++++++++++++
+ arch/x86/include/asm/apic.h         |  2 +
-files changed, 31 insertions(+)
+ arch/x86/include/asm/apic.h         |  2 +
  arch/x86/kernel/apic/vector.c       | 59 +++++++++++++++++++++++------
  arch/x86/kernel/apic/x2apic_savic.c | 20 ++++++++++
 files changed, 69 insertions(+), 12 deletions(-)
 diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/include/asm/apic.h
 +++ b/arch/x86/include/asm/apic.h
 ...
 diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/vector.c
 +++ b/arch/x86/kernel/apic/vector.c
+@@ -XXX,XX +XXX,XX @@ static void apic_update_irq_cfg(struct irq_data *irqd, unsigned int vector,
+                 apicd->hw_irq_cfg.dest_apicid);
+ }
+-static void apic_update_vector(struct irq_data *irqd, unsigned int newvec,
+-                   unsigned int newcpu)
++static inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set)
++{
++    if (apic->update_vector)
++        apic->update_vector(cpu, vector, set);
++}
++
++static int irq_alloc_vector(const struct cpumask *dest, bool resvd, unsigned int *cpu)
++{
++    int vector;
++
++    vector = irq_matrix_alloc(vector_matrix, dest, resvd, cpu);
++
++    if (vector >= 0)
++        apic_update_vector(*cpu, vector, true);
++
++    return vector;
++}
++
++static int irq_alloc_managed_vector(unsigned int *cpu)
++{
++    int vector;
++
++    vector = irq_matrix_alloc_managed(vector_matrix, vector_searchmask, cpu);
++
++    if (vector >= 0)
++        apic_update_vector(*cpu, vector, true);
++
++    return vector;
++}
++
++static void irq_free_vector(unsigned int cpu, unsigned int vector, bool managed)
++{
++    apic_update_vector(cpu, vector, false);
++    irq_matrix_free(vector_matrix, cpu, vector, managed);
++}
++
++static void apic_chipd_update_vector(struct irq_data *irqd, unsigned int newvec,
++                     unsigned int newcpu)
+ {
+     struct apic_chip_data *apicd = apic_chip_data(irqd);
+     struct irq_desc *desc = irq_data_to_desc(irqd);
 @@ -XXX,XX +XXX,XX @@ static void apic_update_vector(struct irq_data *irqd, unsigned int newvec,
          apicd->prev_cpu = apicd->cpu;
          WARN_ON_ONCE(apicd->cpu == newcpu);
      } else {
-+        if (apic->update_vector)
+-        irq_matrix_free(vector_matrix, apicd->cpu, apicd->vector,
-+            apic->update_vector(apicd->cpu, apicd->vector, false);
+-                managed);
-         irq_matrix_free(vector_matrix, apicd->cpu, apicd->vector,
++        irq_free_vector(apicd->cpu, apicd->vector, managed);
                  managed);
      }
-@@ -XXX,XX +XXX,XX @@ static void apic_update_vector(struct irq_data *irqd, unsigned int newvec,
-     apicd->cpu = newcpu;
+ setnew:
-     BUG_ON(!IS_ERR_OR_NULL(per_cpu(vector_irq, newcpu)[newvec]));
+@@ -XXX,XX +XXX,XX @@ assign_vector_locked(struct irq_data *irqd, const struct cpumask *dest)
-     per_cpu(vector_irq, newcpu)[newvec] = desc;
+     if (apicd->move_in_progress || !hlist_unhashed(&apicd->clist))
-+    if (apic->update_vector)
+         return -EBUSY;
-+        apic->update_vector(apicd->cpu, apicd->vector, true);
 -    vector = irq_matrix_alloc(vector_matrix, dest, resvd, &cpu);
 +    vector = irq_alloc_vector(dest, resvd, &cpu);
      trace_vector_alloc(irqd->irq, vector, resvd, vector);
      if (vector < 0)
          return vector;
 -    apic_update_vector(irqd, vector, cpu);
 +    apic_chipd_update_vector(irqd, vector, cpu);
      apic_update_irq_cfg(irqd, vector, cpu);
      return 0;
@@ -XXX,XX +XXX,XX @@ assign_managed_vector(struct irq_data *irqd, const struct cpumask *dest)
      /* set_affinity might call here for nothing */
      if (apicd->vector && cpumask_test_cpu(apicd->cpu, vector_searchmask))
          return 0;
 -    vector = irq_matrix_alloc_managed(vector_matrix, vector_searchmask,
 -                      &cpu);
 +    vector = irq_alloc_managed_vector(&cpu);
      trace_vector_alloc_managed(irqd->irq, vector, vector);
      if (vector < 0)
          return vector;
 -    apic_update_vector(irqd, vector, cpu);
 +    apic_chipd_update_vector(irqd, vector, cpu);
      apic_update_irq_cfg(irqd, vector, cpu);
      return 0;
  }
+@@ -XXX,XX +XXX,XX @@ static void clear_irq_vector(struct irq_data *irqd)
- static void vector_assign_managed_shutdown(struct irq_data *irqd)
+                apicd->prev_cpu);
      per_cpu(vector_irq, apicd->cpu)[vector] = VECTOR_SHUTDOWN;
 -    irq_matrix_free(vector_matrix, apicd->cpu, vector, managed);
 +    irq_free_vector(apicd->cpu, vector, managed);
      apicd->vector = 0;
      /* Clean up move in progress */
@@ -XXX,XX +XXX,XX @@ static void clear_irq_vector(struct irq_data *irqd)
          return;
      per_cpu(vector_irq, apicd->prev_cpu)[vector] = VECTOR_SHUTDOWN;
 -    irq_matrix_free(vector_matrix, apicd->prev_cpu, vector, managed);
 +    irq_free_vector(apicd->prev_cpu, vector, managed);
      apicd->prev_vector = 0;
      apicd->move_in_progress = 0;
      hlist_del_init(&apicd->clist);
 @@ -XXX,XX +XXX,XX @@ static bool vector_configure_legacy(unsigned int virq, struct irq_data *irqd,
      if (irqd_is_activated(irqd)) {
          trace_vector_setup(virq, true, 0);
          apic_update_irq_cfg(irqd, apicd->vector, apicd->cpu);
-+        if (apic->update_vector)
++        apic_update_vector(apicd->cpu, apicd->vector, true);
 +            apic->update_vector(apicd->cpu, apicd->vector, true);
      } else {
          /* Release the vector */
          apicd->can_reserve = true;
-         irqd_set_can_reserve(irqd);
+@@ -XXX,XX +XXX,XX @@ static void free_moved_vector(struct apic_chip_data *apicd)
-         clear_irq_vector(irqd);
+      *    affinity mask comes online.
-+        if (apic->update_vector)
+      */
-+            apic->update_vector(apicd->cpu, apicd->vector, false);
+     trace_vector_free_moved(apicd->irq, cpu, vector, managed);
-         realloc = true;
+-    irq_matrix_free(vector_matrix, cpu, vector, managed);
-     }
++    irq_free_vector(cpu, vector, managed);
-     raw_spin_unlock_irqrestore(&vector_lock, flags);
+     per_cpu(vector_irq, cpu)[vector] = VECTOR_UNUSED;
      hlist_del_init(&apicd->clist);
      apicd->prev_vector = 0;
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
-@@ -XXX,XX +XXX,XX @@
+@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, in
+     __send_ipi_mask(mask, vector, true);
  #include "local.h"
 +#define VEC_POS(v)    ((v) & (32 - 1))
 +#define REG_POS(v)    (((v) >> 5) << 4)
 +
  static DEFINE_PER_CPU(void *, apic_backing_page);
  struct apic_id_node {
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, in
      __send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
  }
 +static void x2apic_savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
 +{
-+    void *backing_page;
++    struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
-+    unsigned long *reg;
++    unsigned long *sirr = (unsigned long *) &ap->bytes[SAVIC_ALLOWED_IRR];
-+    int reg_off;
++    unsigned int bit;
 +
-+    backing_page = per_cpu(apic_backing_page, cpu);
++    /*
-+    reg_off = SAVIC_ALLOWED_IRR_OFFSET + REG_POS(vector);
++     * The registers are 32-bit wide and 16-byte aligned.
-+    reg = (unsigned long *)((char *)backing_page + reg_off);
++     * Compensate for the resulting bit number spacing.
 +     */
 +    bit = vector + 96 * (vector / 32);
 +
 +    if (set)
-+        test_and_set_bit(VEC_POS(vector), reg);
++        set_bit(bit, sirr);
 +    else
-+        test_and_clear_bit(VEC_POS(vector), reg);
++        clear_bit(bit, sirr);
 +}
 +
- static void init_backing_page(void *backing_page)
+ static void init_apic_page(void)
  {
-     struct apic_id_node *next_node, *this_cpu_node;
+     u32 apic_id;
 @@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
      .eoi                = native_apic_msr_eoi,
      .icr_read            = native_x2apic_icr_read,
      .icr_write            = native_x2apic_icr_write,
 +
 +    .update_vector            = x2apic_savic_update_vector,
  };
  apic_driver(apic_x2apic_savic);
 --
 .34.1

-[RFC v2 06/17] x86/apic: Add support to send IPI for Secure AVIC
+[PATCH v3 06/17] x86/apic: Add support to send IPI for Secure AVIC
 With Secure AVIC only Self-IPI is accelerated. To handle all the
 other IPIs, add new callbacks for sending IPI, which write to the
 IRR of the target guest vCPU's APIC backing page and then issue
 GHCB protocol MSR write event for the hypervisor to notify the
 target vCPU.
+Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
-Co-developed-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - Remove write_msr_to_hv() and define savic_ghcb_msr_write() in
+ - Simplify vector updates in bitmap.
-   sev/core.c.
+ - Cleanup icr_data parcelling and unparcelling.
  - Misc cleanups.
  - Fix warning reported by kernel test robot.
- arch/x86/coco/sev/core.c            |  40 +++++++-
+ arch/x86/coco/sev/core.c            |  40 ++++++-
  arch/x86/include/asm/sev.h          |   2 +
- arch/x86/kernel/apic/x2apic_savic.c | 138 +++++++++++++++++++++++++---
+ arch/x86/kernel/apic/x2apic_savic.c | 164 ++++++++++++++++++++++------
-files changed, 162 insertions(+), 18 deletions(-)
+files changed, 167 insertions(+), 39 deletions(-)
 diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/coco/sev/core.c
 +++ b/arch/x86/coco/sev/core.c
 ...
 +
 +    __sev_put_ghcb(&state);
 +    local_irq_restore(flags);
 +}
 +
- /*
+ enum es_result savic_register_gpa(u64 gpa)
-  * Register GPA of the Secure AVIC backing page.
+ {
-  *
+     struct ghcb_state state;
 diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/include/asm/sev.h
 +++ b/arch/x86/include/asm/sev.h
 @@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
  void __init snp_secure_tsc_prepare(void);
  void __init snp_secure_tsc_init(void);
- enum es_result savic_register_gpa(u64 apic_id, u64 gpa);
+ enum es_result savic_register_gpa(u64 gpa);
 +void savic_ghcb_msr_write(u32 reg, u64 value);
  #else    /* !CONFIG_AMD_MEM_ENCRYPT */
-@@ -XXX,XX +XXX,XX @@ static inline void __init snp_secure_tsc_prepare(void) { }
+@@ -XXX,XX +XXX,XX @@ static inline int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_
  static inline void __init snp_secure_tsc_prepare(void) { }
  static inline void __init snp_secure_tsc_init(void) { }
- static inline enum es_result savic_register_gpa(u64 apic_id,
+ static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
-                         u64 gpa) { return ES_UNSUPPORTED; }
++static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
 +static void savic_ghcb_msr_write(u32 reg, u64 value) { }
  #endif    /* CONFIG_AMD_MEM_ENCRYPT */
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
+@@ -XXX,XX +XXX,XX @@ static __always_inline void set_reg(unsigned int offset, u32 val)
+ #define SAVIC_ALLOWED_IRR    0x204
++static inline void update_vector(unsigned int cpu, unsigned int offset,
++                 unsigned int vector, bool set)
++{
++    struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
++    unsigned long *reg = (unsigned long *) &ap->bytes[offset];
++    unsigned int bit;
++
++    /*
++     * The registers are 32-bit wide and 16-byte aligned.
++     * Compensate for the resulting bit number spacing.
++     */
++    bit = vector + 96 * (vector / 32);
++
++    if (set)
++        set_bit(bit, reg);
++    else
++        clear_bit(bit, reg);
++}
++
+ static u32 x2apic_savic_read(u32 reg)
+ {
+     /*
 @@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
+ #define SAVIC_NMI_REQ        0x278
++static inline void self_ipi_reg_write(unsigned int vector)
++{
++    /*
++     * Secure AVIC hardware accelerates guest's MSR write to SELF_IPI
++     * register. It updates the IRR in the APIC backing page, evaluates
++     * the new IRR for interrupt injection and continues with guest
++     * code execution.
++     */
++    native_apic_msr_write(APIC_SELF_IPI, vector);
++}
++
  static void x2apic_savic_write(u32 reg, u32 data)
  {
-     void *backing_page = this_cpu_read(apic_backing_page);
-+    unsigned int cfg;
      switch (reg) {
-     case APIC_LVTT:
 @@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
      case APIC_LVT1:
      case APIC_TMICT:
      case APIC_TDCR:
 -    case APIC_SELF_IPI:
      case APIC_TASKPRI:
      case APIC_EOI:
      case APIC_SPIV:
 @@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
      case APIC_EILVTn(0) ... APIC_EILVTn(3):
-         set_reg(backing_page, reg, data);
+         set_reg(reg, data);
          break;
-+    /* Self IPIs are accelerated by hardware, use wrmsr */
 +    case APIC_SELF_IPI:
-+        cfg = __prepare_ICR(APIC_DEST_SELF, data, 0);
++        self_ipi_reg_write(data);
 +        native_x2apic_icr_write(cfg, 0);
 +        break;
      /* ALLOWED_IRR offsets are writable */
-     case SAVIC_ALLOWED_IRR_OFFSET ... SAVIC_ALLOWED_IRR_OFFSET + 0x70:
+     case SAVIC_ALLOWED_IRR ... SAVIC_ALLOWED_IRR + 0x70:
-         if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR_OFFSET, 16)) {
+         if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR, 16)) {
 @@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
      }
  }
-+static void send_ipi(int cpu, int vector)
++static inline void send_ipi_dest(unsigned int cpu, unsigned int vector)
 +{
-+    void *backing_page;
++    update_vector(cpu, APIC_IRR, vector, true);
-+    int reg_off;
++}
 +
-+    backing_page = per_cpu(apic_backing_page, cpu);
++static void send_ipi_allbut(unsigned int vector)
-+    reg_off = APIC_IRR + REG_POS(vector);
++{
-+    /*
++    unsigned int cpu, src_cpu;
-+     * Use test_and_set_bit() to ensure that IRR updates are atomic w.r.t. other
++    unsigned long flags;
-+     * IRR updates such as during VMRUN and during CPU interrupt handling flow.
++
-+     */
++    local_irq_save(flags);
-+    test_and_set_bit(VEC_POS(vector), (unsigned long *)((char *)backing_page + reg_off));
++
-+}
++    src_cpu = raw_smp_processor_id();
 +
-+static void send_ipi_dest(u64 icr_data)
++    for_each_cpu(cpu, cpu_online_mask) {
-+{
++        if (cpu == src_cpu)
-+    int vector, cpu;
++            continue;
-+
++        send_ipi_dest(cpu, vector);
 +    vector = icr_data & APIC_VECTOR_MASK;
 +    cpu = icr_data >> 32;
 +
 +    send_ipi(cpu, vector);
 +}
 +
 +static void send_ipi_target(u64 icr_data)
 +{
 +    if (icr_data & APIC_DEST_LOGICAL) {
 +        pr_err("IPI target should be of PHYSICAL type\n");
 +        return;
 +    }
 +
-+    send_ipi_dest(icr_data);
-+}
-+
-+static void send_ipi_allbut(u64 icr_data)
-+{
-+    const struct cpumask *self_cpu_mask = get_cpu_mask(smp_processor_id());
-+    unsigned long flags;
-+    int vector, cpu;
-+
-+    vector = icr_data & APIC_VECTOR_MASK;
-+    local_irq_save(flags);
-+    for_each_cpu_andnot(cpu, cpu_present_mask, self_cpu_mask)
-+        send_ipi(cpu, vector);
-+    savic_ghcb_msr_write(APIC_ICR, icr_data);
 +    local_irq_restore(flags);
 +}
 +
-+static void send_ipi_allinc(u64 icr_data)
++static inline void self_ipi(unsigned int vector)
 +{
-+    int vector;
++    u32 icr_low = APIC_SELF_IPI | vector;
 +
-+    send_ipi_allbut(icr_data);
++    native_x2apic_icr_write(icr_low, 0);
 +    vector = icr_data & APIC_VECTOR_MASK;
 +    native_x2apic_icr_write(APIC_DEST_SELF | vector, 0);
 +}
 +
 +static void x2apic_savic_icr_write(u32 icr_low, u32 icr_high)
 +{
-+    int dsh, vector;
++    unsigned int dsh, vector;
 +    u64 icr_data;
 +
-+    icr_data = ((u64)icr_high) << 32 | icr_low;
 +    dsh = icr_low & APIC_DEST_ALLBUT;
++    vector = icr_low & APIC_VECTOR_MASK;
 +
 +    switch (dsh) {
 +    case APIC_DEST_SELF:
-+        vector = icr_data & APIC_VECTOR_MASK;
++        self_ipi(vector);
 +        x2apic_savic_write(APIC_SELF_IPI, vector);
 +        break;
 +    case APIC_DEST_ALLINC:
-+        send_ipi_allinc(icr_data);
++        self_ipi(vector);
-+        break;
++        fallthrough;
 +    case APIC_DEST_ALLBUT:
-+        send_ipi_allbut(icr_data);
++        send_ipi_allbut(vector);
 +        break;
 +    default:
-+        send_ipi_target(icr_data);
++        send_ipi_dest(icr_high, vector);
 +        break;
 +    }
 +
 +    icr_data = ((u64)icr_high) << 32 | icr_low;
 +    if (dsh != APIC_DEST_SELF)
 +        savic_ghcb_msr_write(APIC_ICR, icr_data);
-+    }
++}
-+}
++
-+
++static void send_ipi(u32 dest, unsigned int vector, unsigned int dsh)
-+static void __send_IPI_dest(unsigned int apicid, int vector, unsigned int dest)
++{
-+{
++    unsigned int icr_low;
-+    unsigned int cfg = __prepare_ICR(0, vector, dest);
++
-+
++    icr_low = __prepare_ICR(dsh, vector, APIC_DEST_PHYSICAL);
-+    x2apic_savic_icr_write(cfg, apicid);
++    x2apic_savic_icr_write(icr_low, dest);
 +}
 +
- static void x2apic_savic_send_IPI(int cpu, int vector)
+ static void x2apic_savic_send_ipi(int cpu, int vector)
  {
      u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
 -    /* x2apic MSRs are special and need a special fence: */
 -    weak_wrmsr_fence();
 -    __x2apic_send_IPI_dest(dest, vector, APIC_DEST_PHYSICAL);
-+    __send_IPI_dest(dest, vector, APIC_DEST_PHYSICAL);
++    send_ipi(dest, vector, 0);
  }
- static void
+-static void __send_ipi_mask(const struct cpumask *mask, int vector, bool excl_self)
-@@ -XXX,XX +XXX,XX @@ __send_IPI_mask(const struct cpumask *mask, int vector, int apic_dest)
++static void send_ipi_mask(const struct cpumask *mask, unsigned int vector, bool excl_self)
-     unsigned long this_cpu;
+ {
 -    unsigned long query_cpu;
 -    unsigned long this_cpu;
 +    unsigned int this_cpu;
 +    unsigned int cpu;
      unsigned long flags;
 -    /* x2apic MSRs are special and need a special fence: */
 -    weak_wrmsr_fence();
 -
      local_irq_save(flags);
-     this_cpu = smp_processor_id();
+-    this_cpu = smp_processor_id();
-     for_each_cpu(query_cpu, mask) {
+-    for_each_cpu(query_cpu, mask) {
-         if (apic_dest == APIC_DEST_ALLBUT && this_cpu == query_cpu)
+-        if (excl_self && this_cpu == query_cpu)
 +    this_cpu = raw_smp_processor_id();
 +
 +    for_each_cpu(cpu, mask) {
 +        if (excl_self && cpu == this_cpu)
              continue;
 -        __x2apic_send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu),
 -                       vector, APIC_DEST_PHYSICAL);
-+        __send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu), vector,
++        send_ipi(per_cpu(x86_cpu_to_apicid, cpu), vector, 0);
 +                      APIC_DEST_PHYSICAL);
      }
 +
      local_irq_restore(flags);
  }
-@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, in
+ static void x2apic_savic_send_ipi_mask(const struct cpumask *mask, int vector)
-     __send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
+ {
- }
+-    __send_ipi_mask(mask, vector, false);
++    send_ipi_mask(mask, vector, false);
-+static void __send_IPI_shorthand(int vector, u32 which)
+ }
-+{
-+    unsigned int cfg = __prepare_ICR(which, vector, 0);
+ static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, int vector)
-+
+ {
-+    x2apic_savic_icr_write(cfg, 0);
+-    __send_ipi_mask(mask, vector, true);
-+}
++    send_ipi_mask(mask, vector, true);
-+
+ }
-+static void x2apic_savic_send_IPI_allbutself(int vector)
-+{
+-static void x2apic_savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
-+    __send_IPI_shorthand(vector, APIC_DEST_ALLBUT);
++static void x2apic_savic_send_ipi_allbutself(int vector)
-+}
+ {
-+
+-    struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
-+static void x2apic_savic_send_IPI_all(int vector)
+-    unsigned long *sirr = (unsigned long *) &ap->bytes[SAVIC_ALLOWED_IRR];
-+{
+-    unsigned int bit;
-+    __send_IPI_shorthand(vector, APIC_DEST_ALLINC);
++    send_ipi(0, vector, APIC_DEST_ALLBUT);
 +}
-+
-+static void x2apic_savic_send_IPI_self(int vector)
+-    /*
-+{
+-     * The registers are 32-bit wide and 16-byte aligned.
-+    __send_IPI_shorthand(vector, APIC_DEST_SELF);
+-     * Compensate for the resulting bit number spacing.
-+}
+-     */
-+
+-    bit = vector + 96 * (vector / 32);
- static void x2apic_savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
++static void x2apic_savic_send_ipi_all(int vector)
- {
++{
-     void *backing_page;
++    send_ipi(0, vector, APIC_DEST_ALLINC);
 +}
 -    if (set)
 -        set_bit(bit, sirr);
 -    else
 -        clear_bit(bit, sirr);
 +static void x2apic_savic_send_ipi_self(int vector)
 +{
 +    self_ipi_reg_write(vector);
 +}
 +
 +static void x2apic_savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
 +{
 +    update_vector(cpu, SAVIC_ALLOWED_IRR, vector, set);
  }
  static void init_apic_page(void)
 @@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
-     .send_IPI            = x2apic_savic_send_IPI,
+     .send_IPI            = x2apic_savic_send_ipi,
-     .send_IPI_mask            = x2apic_savic_send_IPI_mask,
+     .send_IPI_mask            = x2apic_savic_send_ipi_mask,
-     .send_IPI_mask_allbutself    = x2apic_savic_send_IPI_mask_allbutself,
+     .send_IPI_mask_allbutself    = x2apic_savic_send_ipi_mask_allbutself,
 -    .send_IPI_allbutself        = x2apic_send_IPI_allbutself,
 -    .send_IPI_all            = x2apic_send_IPI_all,
 -    .send_IPI_self            = x2apic_send_IPI_self,
-+    .send_IPI_allbutself        = x2apic_savic_send_IPI_allbutself,
++    .send_IPI_allbutself        = x2apic_savic_send_ipi_allbutself,
-+    .send_IPI_all            = x2apic_savic_send_IPI_all,
++    .send_IPI_all            = x2apic_savic_send_ipi_all,
-+    .send_IPI_self            = x2apic_savic_send_IPI_self,
++    .send_IPI_self            = x2apic_savic_send_ipi_self,
      .nmi_to_offline_cpu        = true,
      .read                = x2apic_savic_read,
      .write                = x2apic_savic_write,
      .eoi                = native_apic_msr_eoi,
 ...

-[RFC v2 07/17] x86/apic: Support LAPIC timer for Secure AVIC
+[PATCH v3 07/17] x86/apic: Support LAPIC timer for Secure AVIC
-From: Kishon Vijay Abraham I <kvijayab@amd.com>
+Secure AVIC requires LAPIC timer to be emulated by the hypervisor.
+KVM already supports emulating LAPIC timer using hrtimers. In order
 Secure AVIC requires LAPIC timer to be emulated by hypervisor. KVM
 already supports emulating LAPIC timer using hrtimers. In order
 to emulate LAPIC timer, APIC_LVTT, APIC_TMICT and APIC_TDCR register
 values need to be propagated to the hypervisor for arming the timer.
 APIC_TMCCT register value has to be read from the hypervisor, which
 is required for calibrating the APIC timer. So, read/write all APIC
 timer registers from/to the hypervisor.
-In addition, configure APIC_ALLOWED_IRR for the hypervisor to inject
+In addition, add a static call for apic's update_vector() callback,
-timer interrupt using LOCAL_TIMER_VECTOR.
+to configure ALLOWED_IRR for the hypervisor to inject timer interrupt
+using LOCAL_TIMER_VECTOR.
 Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
-Co-developed-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - Move savic_ghcb_msr_read() definition here.
+ - Add static call for apic_update_vector()
  - Call update_vector() callback only when it is initialized.
  arch/x86/coco/sev/core.c            | 27 +++++++++++++++++++++++++++
+ arch/x86/include/asm/apic.h         |  8 ++++++++
  arch/x86/include/asm/sev.h          |  2 ++
- arch/x86/kernel/apic/apic.c         |  4 ++++
+ arch/x86/kernel/apic/apic.c         |  2 ++
  arch/x86/kernel/apic/init.c         |  3 +++
  arch/x86/kernel/apic/vector.c       |  6 ------
  arch/x86/kernel/apic/x2apic_savic.c |  7 +++++--
-files changed, 38 insertions(+), 2 deletions(-)
+files changed, 47 insertions(+), 8 deletions(-)
 diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/coco/sev/core.c
 +++ b/arch/x86/coco/sev/core.c
 ...
 +}
 +
  void savic_ghcb_msr_write(u32 reg, u64 value)
  {
      u64 msr = APIC_BASE_MSR + (reg >> 4);
+diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
+index XXXXXXX..XXXXXXX 100644
+--- a/arch/x86/include/asm/apic.h
++++ b/arch/x86/include/asm/apic.h
+@@ -XXX,XX +XXX,XX @@ struct apic_override {
+     void    (*icr_write)(u32 low, u32 high);
+     int    (*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip);
+     int    (*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip);
++    void    (*update_vector)(unsigned int cpu, unsigned int vector, bool set);
+ };
+ /*
+@@ -XXX,XX +XXX,XX @@ DECLARE_APIC_CALL(wait_icr_idle);
+ DECLARE_APIC_CALL(wakeup_secondary_cpu);
+ DECLARE_APIC_CALL(wakeup_secondary_cpu_64);
+ DECLARE_APIC_CALL(write);
++DECLARE_APIC_CALL(update_vector);
+ static __always_inline u32 apic_read(u32 reg)
+ {
+@@ -XXX,XX +XXX,XX @@ static __always_inline bool apic_id_valid(u32 apic_id)
+     return apic_id <= apic->max_apic_id;
+ }
++static __always_inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set)
++{
++    static_call(apic_call_update_vector)(cpu, vector, set);
++}
++
+ #else /* CONFIG_X86_LOCAL_APIC */
+ static inline u32 apic_read(u32 reg) { return 0; }
+@@ -XXX,XX +XXX,XX @@ static inline void apic_wait_icr_idle(void) { }
+ static inline u32 safe_apic_wait_icr_idle(void) { return 0; }
+ static inline void apic_native_eoi(void) { WARN_ON_ONCE(1); }
+ static inline void apic_setup_apic_calls(void) { }
++static inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set) { }
+ #define apic_update_callback(_callback, _fn) do { } while (0)
 diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/include/asm/sev.h
 +++ b/arch/x86/include/asm/sev.h
 @@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
  void __init snp_secure_tsc_prepare(void);
  void __init snp_secure_tsc_init(void);
- enum es_result savic_register_gpa(u64 apic_id, u64 gpa);
+ enum es_result savic_register_gpa(u64 gpa);
 +u64 savic_ghcb_msr_read(u32 reg);
  void savic_ghcb_msr_write(u32 reg, u64 value);
  #else    /* !CONFIG_AMD_MEM_ENCRYPT */
 @@ -XXX,XX +XXX,XX @@ static inline void __init snp_secure_tsc_prepare(void) { }
  static inline void __init snp_secure_tsc_init(void) { }
- static inline enum es_result savic_register_gpa(u64 apic_id,
+ static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
-                         u64 gpa) { return ES_UNSUPPORTED; }
+ static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
 +static inline u64 savic_ghcb_msr_read(u32 reg) { return 0; }
- static void savic_ghcb_msr_write(u32 reg, u64 value) { }
  #endif    /* CONFIG_AMD_MEM_ENCRYPT */
 diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/apic.c
 +++ b/arch/x86/kernel/apic/apic.c
 @@ -XXX,XX +XXX,XX @@ static void setup_APIC_timer(void)
 xF, ~0UL);
      } else
          clockevents_register_device(levt);
 +
-+    if (apic->update_vector)
++    apic_update_vector(smp_processor_id(), LOCAL_TIMER_VECTOR, true);
-+        apic->update_vector(smp_processor_id(), LOCAL_TIMER_VECTOR,
+ }
 +                    true);
  }
  /*
+diff --git a/arch/x86/kernel/apic/init.c b/arch/x86/kernel/apic/init.c
+index XXXXXXX..XXXXXXX 100644
+--- a/arch/x86/kernel/apic/init.c
++++ b/arch/x86/kernel/apic/init.c
+@@ -XXX,XX +XXX,XX @@ DEFINE_APIC_CALL(wait_icr_idle);
+ DEFINE_APIC_CALL(wakeup_secondary_cpu);
+ DEFINE_APIC_CALL(wakeup_secondary_cpu_64);
+ DEFINE_APIC_CALL(write);
++DEFINE_APIC_CALL(update_vector);
+ EXPORT_STATIC_CALL_TRAMP_GPL(apic_call_send_IPI_mask);
+ EXPORT_STATIC_CALL_TRAMP_GPL(apic_call_send_IPI_self);
+@@ -XXX,XX +XXX,XX @@ static __init void restore_override_callbacks(void)
+     apply_override(icr_write);
+     apply_override(wakeup_secondary_cpu);
+     apply_override(wakeup_secondary_cpu_64);
++    apply_override(update_vector);
+ }
+ #define update_call(__cb)                    \
+@@ -XXX,XX +XXX,XX @@ static __init void update_static_calls(void)
+     update_call(wait_icr_idle);
+     update_call(wakeup_secondary_cpu);
+     update_call(wakeup_secondary_cpu_64);
++    update_call(update_vector);
+ }
+ void __init apic_setup_apic_calls(void)
+diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
+index XXXXXXX..XXXXXXX 100644
+--- a/arch/x86/kernel/apic/vector.c
++++ b/arch/x86/kernel/apic/vector.c
+@@ -XXX,XX +XXX,XX @@ static void apic_update_irq_cfg(struct irq_data *irqd, unsigned int vector,
+                 apicd->hw_irq_cfg.dest_apicid);
+ }
+-static inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set)
+-{
+-    if (apic->update_vector)
+-        apic->update_vector(cpu, vector, set);
+-}
+-
+ static int irq_alloc_vector(const struct cpumask *dest, bool resvd, unsigned int *cpu)
+ {
+     int vector;
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
 @@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 ...
 +        return savic_ghcb_msr_read(reg);
      case APIC_ID:
      case APIC_LVR:
      case APIC_TASKPRI:
 @@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
+ {
      switch (reg) {
      case APIC_LVTT:
 -    case APIC_LVT0:
 -    case APIC_LVT1:
      case APIC_TMICT:
 ...

-[RFC v2 08/17] x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC
+[PATCH v3 08/17] x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC
 From: Kishon Vijay Abraham I <kvijayab@amd.com>
 Secure AVIC requires VGIF to be configured in VMSA. Configure
-for secondary vCPUs (the configuration for boot CPU is done in
+for secondary vCPUs (the configuration for boot CPU is done by
-hypervisor).
+the hypervisor).
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
-Signed-off-by: Neeraj Upadhyay <neeraj.upadhyay@amd.com>
+Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
  - No change
  arch/x86/coco/sev/core.c | 3 +++
 file changed, 3 insertions(+)
 ...

-[RFC v2 09/17] x86/apic: Add support to send NMI IPI for Secure AVIC
+[PATCH v3 09/17] x86/apic: Add support to send NMI IPI for Secure AVIC
-From: Kishon Vijay Abraham I <kvijayab@amd.com>
 Secure AVIC has introduced a new field in the APIC backing page
-"NmiReq" that has to be set by the guest to request a NMI IPI.
+"NmiReq" that has to be set by the guest to request a NMI IPI
 through APIC_ICR write.
 Add support to set NmiReq appropriately to send NMI IPI.
 This also requires Virtual NMI feature to be enabled in VINTRL_CTRL
 field in the VMSA. However this would be added by a later commit
 after adding support for injecting NMI from the hypervisor.
+Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - Do not set APIC_IRR for NMI IPI.
+ - Updates to use per_cpu_ptr() on apic_page struct.
- arch/x86/kernel/apic/x2apic_savic.c | 24 ++++++++++++++++--------
+ arch/x86/kernel/apic/x2apic_savic.c | 28 ++++++++++++++++++++--------
-file changed, 16 insertions(+), 8 deletions(-)
+file changed, 20 insertions(+), 8 deletions(-)
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
 @@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
      }
  }
--static void send_ipi(int cpu, int vector)
+-static inline void send_ipi_dest(unsigned int cpu, unsigned int vector)
-+static void send_ipi(int cpu, int vector, bool nmi)
++static void send_ipi_dest(unsigned int cpu, unsigned int vector, bool nmi)
  {
-     void *backing_page;
++    if (nmi) {
-     int reg_off;
++        struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
++
-     backing_page = per_cpu(apic_backing_page, cpu);
++        WRITE_ONCE(ap->regs[SAVIC_NMI_REQ >> 2], 1);
-     reg_off = APIC_IRR + REG_POS(vector);
++        return;
--    /*
++    }
--     * Use test_and_set_bit() to ensure that IRR updates are atomic w.r.t. other
++
--     * IRR updates such as during VMRUN and during CPU interrupt handling flow.
+     update_vector(cpu, APIC_IRR, vector, true);
 -     */
 -    test_and_set_bit(VEC_POS(vector), (unsigned long *)((char *)backing_page + reg_off));
 +    if (!nmi)
 +        /*
 +         * Use test_and_set_bit() to ensure that IRR updates are atomic w.r.t. other
 +         * IRR updates such as during VMRUN and during CPU interrupt handling flow.
 +         * */
 +        test_and_set_bit(VEC_POS(vector),
 +                 (unsigned long *)((char *)backing_page + reg_off));
 +    else
 +        set_reg(backing_page, SAVIC_NMI_REQ_OFFSET, nmi);
  }
- static void send_ipi_dest(u64 icr_data)
+-static void send_ipi_allbut(unsigned int vector)
 +static void send_ipi_allbut(unsigned int vector, bool nmi)
  {
-     int vector, cpu;
+     unsigned int cpu, src_cpu;
 +    bool nmi;
      vector = icr_data & APIC_VECTOR_MASK;
      cpu = icr_data >> 32;
 +    nmi = ((icr_data & APIC_DM_FIXED_MASK) == APIC_DM_NMI);
 -    send_ipi(cpu, vector);
 +    send_ipi(cpu, vector, nmi);
  }
  static void send_ipi_target(u64 icr_data)
@@ -XXX,XX +XXX,XX @@ static void send_ipi_allbut(u64 icr_data)
      const struct cpumask *self_cpu_mask = get_cpu_mask(smp_processor_id());
      unsigned long flags;
-     int vector, cpu;
+@@ -XXX,XX +XXX,XX @@ static void send_ipi_allbut(unsigned int vector)
-+    bool nmi;
+     for_each_cpu(cpu, cpu_online_mask) {
+         if (cpu == src_cpu)
-     vector = icr_data & APIC_VECTOR_MASK;
+             continue;
-+    nmi = ((icr_data & APIC_DM_FIXED_MASK) == APIC_DM_NMI);
+-        send_ipi_dest(cpu, vector);
-     local_irq_save(flags);
++        send_ipi_dest(cpu, vector, nmi);
-     for_each_cpu_andnot(cpu, cpu_present_mask, self_cpu_mask)
+     }
--        send_ipi(cpu, vector);
 +        send_ipi(cpu, vector, nmi);
      savic_ghcb_msr_write(APIC_ICR, icr_data);
      local_irq_restore(flags);
  }
+-static inline void self_ipi(unsigned int vector)
++static inline void self_ipi(unsigned int vector, bool nmi)
+ {
+     u32 icr_low = APIC_SELF_IPI | vector;
++    if (nmi)
++        icr_low |= APIC_DM_NMI;
++
+     native_x2apic_icr_write(icr_low, 0);
+ }
+@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_icr_write(u32 icr_low, u32 icr_high)
+ {
+     unsigned int dsh, vector;
+     u64 icr_data;
++    bool nmi;
+     dsh = icr_low & APIC_DEST_ALLBUT;
+     vector = icr_low & APIC_VECTOR_MASK;
++    nmi = ((icr_low & APIC_DM_FIXED_MASK) == APIC_DM_NMI);
+     switch (dsh) {
+     case APIC_DEST_SELF:
+-        self_ipi(vector);
++        self_ipi(vector, nmi);
+         break;
+     case APIC_DEST_ALLINC:
+-        self_ipi(vector);
++        self_ipi(vector, nmi);
+         fallthrough;
+     case APIC_DEST_ALLBUT:
+-        send_ipi_allbut(vector);
++        send_ipi_allbut(vector, nmi);
+         break;
+     default:
+-        send_ipi_dest(icr_high, vector);
++        send_ipi_dest(icr_high, vector, nmi);
+         break;
+     }
 --
 .34.1

-[RFC v2 10/17] x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC
+[PATCH v3 10/17] x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC
 ...
 from hypervisor.
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - No change
+ - Remove MSR_AMD64_SECURE_AVIC_EN macros from this patch.
- arch/x86/include/asm/msr-index.h    | 5 +++++
+ arch/x86/include/asm/msr-index.h    | 3 +++
  arch/x86/kernel/apic/x2apic_savic.c | 6 ++++++
-files changed, 11 insertions(+)
+files changed, 9 insertions(+)
 diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/include/asm/msr-index.h
 +++ b/arch/x86/include/asm/msr-index.h
 @@ -XXX,XX +XXX,XX @@
- #define MSR_AMD64_SNP_SECURE_AVIC     BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
+ #define MSR_AMD64_SNP_SECURE_AVIC    BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
  #define MSR_AMD64_SNP_RESV_BIT        19
  #define MSR_AMD64_SNP_RESERVED_MASK    GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
 +#define MSR_AMD64_SECURE_AVIC_CONTROL    0xc0010138
-+#define MSR_AMD64_SECURE_AVIC_EN_BIT    0
-+#define MSR_AMD64_SECURE_AVIC_EN    BIT_ULL(MSR_AMD64_SECURE_AVIC_EN_BIT)
 +#define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT 1
 +#define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI BIT_ULL(MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT)
  #define MSR_AMD64_RMP_BASE        0xc0010132
  #define MSR_AMD64_RMP_END        0xc0010133
  #define MSR_AMD64_RMP_CFG        0xc0010136
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
-@@ -XXX,XX +XXX,XX @@ static DEFINE_PER_CPU(struct apic_id_node, apic_id_node);
+@@ -XXX,XX +XXX,XX @@ struct apic_page {
- static struct llist_head *apic_id_map;
+ static struct apic_page __percpu *apic_page __ro_after_init;
 +static inline void savic_wr_control_msr(u64 val)
 +{
 +    native_wrmsr(MSR_AMD64_SECURE_AVIC_CONTROL, lower_32_bits(val), upper_32_bits(val));
 +}
 +
  static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
  {
      return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
 @@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
-     ret = savic_register_gpa(-1ULL, gpa);
+     ret = savic_register_gpa(gpa);
      if (ret != ES_OK)
          snp_abort();
 +    savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
  }
  static int x2apic_savic_probe(void)
 --
 .34.1

-[RFC v2 11/17] x86/sev: Enable NMI support for Secure AVIC
+[PATCH v3 11/17] x86/sev: Enable NMI support for Secure AVIC
 From: Kishon Vijay Abraham I <kvijayab@amd.com>
 Now that support to send NMI IPI and support to inject NMI from
-hypervisor has been added, set V_NMI_ENABLE in VINTR_CTRL field of
+the hypervisor has been added, set V_NMI_ENABLE in VINTR_CTRL
-VMSA to enable NMI.
+field of VMSA to enable NMI for Secure AVIC guests.
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - No change
+ - No change.
  arch/x86/coco/sev/core.c | 2 +-
 file changed, 1 insertion(+), 1 deletion(-)
 diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
 ...

-[RFC v2 12/17] x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests
+[PATCH v3 12/17] x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests
 Hypervisor need information about the current state of LVT registers
 for device emulation and NMI. So, forward reads and write of these
 registers to the Hypervisor for Secure AVIC guests.
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - New change.
+ - No change.
  arch/x86/kernel/apic/x2apic_savic.c | 20 ++++++++++----------
 file changed, 10 insertions(+), 10 deletions(-)
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 ...
 +        savic_ghcb_msr_write(reg, data);
 +        break;
      case APIC_TASKPRI:
      case APIC_EOI:
      case APIC_SPIV:
-     case SAVIC_NMI_REQ_OFFSET:
+     case SAVIC_NMI_REQ:
      case APIC_ESR:
      case APIC_ICR:
 -    case APIC_LVTTHMR:
 -    case APIC_LVTPC:
 -    case APIC_LVTERR:
      case APIC_ECTRL:
      case APIC_SEOI:
      case APIC_IER:
 --
 .34.1

-[RFC v2 13/17] x86/apic: Handle EOI writes for SAVIC guests
+[PATCH v3 13/17] x86/apic: Handle EOI writes for SAVIC guests
-Secure AVIC accelerates EOI msr writes for edge-triggered interrupts.
+Secure AVIC accelerates guest's EOI msr writes for edge-triggered
-For level-triggered interrupts, EOI msr writes trigger #VC exception
+interrupts. For level-triggered interrupts, EOI msr writes trigger
-with SVM_EXIT_AVIC_UNACCELERATED_ACCESS error code. The #VC handler
+VC exception with SVM_EXIT_AVIC_UNACCELERATED_ACCESS error code. The
-would need to trigger a GHCB protocol MSR write event to the Hypervisor.
+VC handler would need to trigger a GHCB protocol MSR write event to
-As #VC handling adds extra overhead, directly do a GHCB protocol based
+to notify the Hypervisor about completion of the level-triggered
-EOI write from apic->eoi() callback for level-triggered interrupts.
+interrupt. This is required for cases like emulated IOAPIC. VC exception
-Use wrmsr for edge-triggered interrupts, so that hardware re-evaluates
+handling adds extra performance overhead for APIC register write. In
-any pending interrupt which can be delivered to guest vCPU. For level-
+addition, some unaccelerated APIC register msr writes are trapped,
-triggered interrupts, re-evaluation happens on return from VMGEXIT.
+whereas others are faulted. This results in additional complexity in
 VC exception handling for unacclerated accesses. So, directly do a GHCB
 protocol based EOI write from apic->eoi() callback for level-triggered
 interrupts. Use wrmsr for edge-triggered interrupts, so that hardware
 re-evaluates any pending interrupt which can be delivered to guest vCPU.
 For level-triggered interrupts, re-evaluation happens on return from
 VMGEXIT corresponding to the GHCB event for EOI msr write.
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - New change.
+ - Reuse find_highest_vector() from kvm/lapic.c
  - Misc cleanups.
- arch/x86/kernel/apic/x2apic_savic.c | 53 ++++++++++++++++++++++++++++-
+ arch/x86/include/asm/apic-emul.h    | 28 +++++++++++++
-file changed, 52 insertions(+), 1 deletion(-)
+ arch/x86/kernel/apic/x2apic_savic.c | 62 +++++++++++++++++++++++++----
  arch/x86/kvm/lapic.c                | 23 ++---------
 files changed, 85 insertions(+), 28 deletions(-)
  create mode 100644 arch/x86/include/asm/apic-emul.h
+diff --git a/arch/x86/include/asm/apic-emul.h b/arch/x86/include/asm/apic-emul.h
+new file mode 100644
+index XXXXXXX..XXXXXXX
+--- /dev/null
++++ b/arch/x86/include/asm/apic-emul.h
+@@ -XXX,XX +XXX,XX @@
++/* SPDX-License-Identifier: GPL-2.0-only */
++#ifndef _ASM_X86_APIC_EMUL_H
++#define _ASM_X86_APIC_EMUL_H
++
++#define MAX_APIC_VECTOR            256
++#define APIC_VECTORS_PER_REG        32
++
++static inline int apic_find_highest_vector(void *bitmap)
++{
++    unsigned int regno;
++    unsigned int vec;
++    u32 *reg;
++
++    /*
++     * The registers int the bitmap are 32-bit wide and 16-byte
++     * aligned. State of a vector is stored in a single bit.
++     */
++    for (regno = MAX_APIC_VECTOR / APIC_VECTORS_PER_REG - 1; regno >= 0; regno--) {
++        vec = regno * APIC_VECTORS_PER_REG;
++        reg = bitmap + regno * 16;
++        if (*reg)
++            return __fls(*reg) + vec;
++    }
++
++    return -1;
++}
++
++#endif /* _ASM_X86_APIC_EMUL_H */
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
+@@ -XXX,XX +XXX,XX @@
+ #include <linux/align.h>
+ #include <asm/apic.h>
++#include <asm/apic-emul.h>
+ #include <asm/sev.h>
+ #include "local.h"
+@@ -XXX,XX +XXX,XX @@ static __always_inline void set_reg(unsigned int offset, u32 val)
+     WRITE_ONCE(this_cpu_ptr(apic_page)->regs[offset >> 2], val);
+ }
+-#define SAVIC_ALLOWED_IRR    0x204
+-
+-static inline void update_vector(unsigned int cpu, unsigned int offset,
+-                 unsigned int vector, bool set)
++static inline unsigned long *get_reg_bitmap(unsigned int cpu, unsigned int offset)
+ {
+     struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
+-    unsigned long *reg = (unsigned long *) &ap->bytes[offset];
+-    unsigned int bit;
++    return (unsigned long *) &ap->bytes[offset];
++}
++
++static inline unsigned int get_vec_bit(unsigned int vector)
++{
+     /*
+      * The registers are 32-bit wide and 16-byte aligned.
+      * Compensate for the resulting bit number spacing.
+      */
+-    bit = vector + 96 * (vector / 32);
++    return vector + 96 * (vector / 32);
++}
++
++static inline void update_vector(unsigned int cpu, unsigned int offset,
++                 unsigned int vector, bool set)
++{
++    unsigned long *reg = get_reg_bitmap(cpu, offset);
++    unsigned int bit = get_vec_bit(vector);
+     if (set)
+         set_bit(bit, reg);
+@@ -XXX,XX +XXX,XX @@ static inline void update_vector(unsigned int cpu, unsigned int offset,
+         clear_bit(bit, reg);
+ }
++static inline bool test_vector(unsigned int cpu, unsigned int offset, unsigned int vector)
++{
++    unsigned long *reg = get_reg_bitmap(cpu, offset);
++    unsigned int bit = get_vec_bit(vector);
++
++    return test_bit(bit, reg);
++}
++
++#define SAVIC_ALLOWED_IRR    0x204
++
+ static u32 x2apic_savic_read(u32 reg)
+ {
+     /*
 @@ -XXX,XX +XXX,XX @@ static int x2apic_savic_probe(void)
      return 1;
  }
-+static int find_highest_isr(void *backing_page)
++static void x2apic_savic_eoi(void)
 +{
-+    int vec_per_reg = 32;
++    unsigned int cpu;
 +    int max_vec = 256;
 +    u32 reg;
 +    int vec;
 +
-+    for (vec = max_vec - 32; vec >= 0; vec -= vec_per_reg) {
++    cpu = raw_smp_processor_id();
-+        reg = get_reg(backing_page, APIC_ISR + REG_POS(vec));
++    vec = apic_find_highest_vector(get_reg_bitmap(cpu, APIC_ISR));
-+        if (reg)
++    if (WARN_ONCE(vec == -1, "EOI write while no active interrupt in APIC_ISR"))
 +            return __fls(reg) + vec;
 +    }
 +
 +    return -1;
 +}
 +
 +static void x2apic_savic_eoi(void)
 +{
 +    void *backing_page;
 +    int reg_off;
 +    int vec_pos;
 +    u32 tmr;
 +    int vec;
 +
 +    backing_page = this_cpu_read(apic_backing_page);
 +
 +    vec = find_highest_isr(backing_page);
 +    if (WARN_ONCE(vec == -1, "EOI write without any active interrupt in APIC_ISR"))
 +        return;
 +
-+    reg_off = REG_POS(vec);
++    if (test_vector(cpu, APIC_TMR, vec)) {
-+    vec_pos = VEC_POS(vec);
++        update_vector(cpu, APIC_ISR, vec, false);
 +    tmr = get_reg(backing_page, APIC_TMR + reg_off);
 +    if (tmr & BIT(vec_pos)) {
 +        clear_bit(vec_pos, backing_page + APIC_ISR + reg_off);
 +        /*
 +         * Propagate the EOI write to hv for level-triggered interrupts.
 +         * Return to guest from GHCB protocol event takes care of
 +         * re-evaluating interrupt state.
 +         */
 ...
 -    .eoi                = native_apic_msr_eoi,
 +    .eoi                = x2apic_savic_eoi,
      .icr_read            = native_x2apic_icr_read,
      .icr_write            = x2apic_savic_icr_write,
+diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
+index XXXXXXX..XXXXXXX 100644
+--- a/arch/x86/kvm/lapic.c
++++ b/arch/x86/kvm/lapic.c
+@@ -XXX,XX +XXX,XX @@
+ #include <linux/export.h>
+ #include <linux/math64.h>
+ #include <linux/slab.h>
++#include <asm/apic-emul.h>
+ #include <asm/processor.h>
+ #include <asm/mce.h>
+ #include <asm/msr.h>
+@@ -XXX,XX +XXX,XX @@
+ /* 14 is the version for Xeon and Pentium 8.4.8*/
+ #define APIC_VERSION            0x14UL
+ #define LAPIC_MMIO_LENGTH        (1 << 12)
+-/* followed define is not in apicdef.h */
+-#define MAX_APIC_VECTOR            256
+-#define APIC_VECTORS_PER_REG        32
+ /*
+  * Enable local APIC timer advancement (tscdeadline mode only) with adaptive
+@@ -XXX,XX +XXX,XX @@ static const unsigned int apic_lvt_mask[KVM_APIC_MAX_NR_LVT_ENTRIES] = {
+     [LVT_CMCI] = LVT_MASK | APIC_MODE_MASK
+ };
+-static int find_highest_vector(void *bitmap)
+-{
+-    int vec;
+-    u32 *reg;
+-
+-    for (vec = MAX_APIC_VECTOR - APIC_VECTORS_PER_REG;
+-         vec >= 0; vec -= APIC_VECTORS_PER_REG) {
+-        reg = bitmap + REG_POS(vec);
+-        if (*reg)
+-            return __fls(*reg) + vec;
+-    }
+-
+-    return -1;
+-}
+-
+ static u8 count_vectors(void *bitmap)
+ {
+     int vec;
+@@ -XXX,XX +XXX,XX @@ EXPORT_SYMBOL_GPL(kvm_apic_update_irr);
+ static inline int apic_search_irr(struct kvm_lapic *apic)
+ {
+-    return find_highest_vector(apic->regs + APIC_IRR);
++    return apic_find_highest_vector(apic->regs + APIC_IRR);
+ }
+ static inline int apic_find_highest_irr(struct kvm_lapic *apic)
+@@ -XXX,XX +XXX,XX @@ static inline int apic_find_highest_isr(struct kvm_lapic *apic)
+     if (likely(apic->highest_isr_cache != -1))
+         return apic->highest_isr_cache;
+-    result = find_highest_vector(apic->regs + APIC_ISR);
++    result = apic_find_highest_vector(apic->regs + APIC_ISR);
+     ASSERT(result == -1 || result >= 16);
+     return result;
 --
 .34.1

-[RFC v2 14/17] x86/apic: Add kexec support for Secure AVIC
+[PATCH v3 14/17] x86/apic: Add kexec support for Secure AVIC
-Add a ->teardown callback to disable Secure AVIC before
+Add a apic->teardown() callback to disable Secure AVIC before
 rebooting into the new kernel. This ensures that the new
 kernel does not access the old APIC backing page which was
 allocated by the previous kernel. Such accesses can happen
 if there are any APIC accesses done during guest boot before
-Secure AVIC driver probe is done by the new kernel (as
+Secure AVIC driver probe is done by the new kernel (as Secure
-Secure AVIC remains enabled in control msr).
+AVIC would have remained enabled in the Secure AVIC control
 msr).
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - New change.
+ - Change savic_unregister_gpa() interface to allow GPA unregistration
    only for local CPU.
- arch/x86/coco/sev/core.c            | 34 +++++++++++++++++++++++++++++
+ arch/x86/coco/sev/core.c            | 25 +++++++++++++++++++++++++
  arch/x86/include/asm/apic.h         |  1 +
- arch/x86/include/asm/sev.h          |  3 +++
+ arch/x86/include/asm/sev.h          |  2 ++
  arch/x86/kernel/apic/apic.c         |  3 +++
- arch/x86/kernel/apic/x2apic_savic.c |  8 +++++++
+ arch/x86/kernel/apic/x2apic_savic.c |  8 ++++++++
-files changed, 49 insertions(+)
+files changed, 39 insertions(+)
 diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/coco/sev/core.c
 +++ b/arch/x86/coco/sev/core.c
-@@ -XXX,XX +XXX,XX @@ enum es_result savic_register_gpa(u64 apic_id, u64 gpa)
+@@ -XXX,XX +XXX,XX @@ enum es_result savic_register_gpa(u64 gpa)
-     return ret;
+     return res;
  }
-+/*
++enum es_result savic_unregister_gpa(u64 *gpa)
 + * Unregister GPA of the Secure AVIC backing page.
 + *
 + * @apic_id: APIC ID of the vCPU. Use -1ULL for the current vCPU
 + *           doing the call.
 + *
 + * On success, returns previously registered GPA of the Secure AVIC
 + * backing page in gpa arg.
 + */
 +enum es_result savic_unregister_gpa(u64 apic_id, u64 *gpa)
 +{
 +    struct ghcb_state state;
 +    struct es_em_ctxt ctxt;
 +    unsigned long flags;
 +    struct ghcb *ghcb;
 ...
 +
 +    ghcb = __sev_get_ghcb(&state);
 +
 +    vc_ghcb_invalidate(ghcb);
 +
-+    ghcb_set_rax(ghcb, apic_id);
++    ghcb_set_rax(ghcb, -1ULL);
 +    ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SECURE_AVIC,
 +            SVM_VMGEXIT_SECURE_AVIC_UNREGISTER_GPA, 0);
 +    if (gpa && ret == ES_OK)
 +        *gpa = ghcb->save.rbx;
 +    __sev_put_ghcb(&state);
 ...
 --- a/arch/x86/include/asm/sev.h
 +++ b/arch/x86/include/asm/sev.h
 @@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
  void __init snp_secure_tsc_prepare(void);
  void __init snp_secure_tsc_init(void);
- enum es_result savic_register_gpa(u64 apic_id, u64 gpa);
+ enum es_result savic_register_gpa(u64 gpa);
-+enum es_result savic_unregister_gpa(u64 apic_id, u64 *gpa);
++enum es_result savic_unregister_gpa(u64 *gpa);
  u64 savic_ghcb_msr_read(u32 reg);
  void savic_ghcb_msr_write(u32 reg, u64 value);
-@@ -XXX,XX +XXX,XX @@ static inline void __init snp_secure_tsc_prepare(void) { }
+@@ -XXX,XX +XXX,XX @@ static inline int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_
  static inline void __init snp_secure_tsc_prepare(void) { }
  static inline void __init snp_secure_tsc_init(void) { }
- static inline enum es_result savic_register_gpa(u64 apic_id,
+ static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
-                         u64 gpa) { return ES_UNSUPPORTED; }
++static inline enum es_result savic_unregister_gpa(u64 *gpa) { return ES_UNSUPPORTED; }
-+static inline enum es_result savic_unregister_gpa(u64 apic_id,
+ static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
 +                          u64 *gpa) { return ES_UNSUPPORTED; }
  static inline u64 savic_ghcb_msr_read(u32 reg) { return 0; }
- static void savic_ghcb_msr_write(u32 reg, u64 value) { }
 diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/apic.c
 +++ b/arch/x86/kernel/apic/apic.c
 ...
  #ifdef CONFIG_X86_32
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
-@@ -XXX,XX +XXX,XX @@ static void init_backing_page(void *backing_page)
+@@ -XXX,XX +XXX,XX @@ static void init_apic_page(void)
-     }
+     set_reg(APIC_ID, apic_id);
  }
 +static void x2apic_savic_teardown(void)
 +{
 +    /* Disable Secure AVIC */
 +    native_wrmsr(MSR_AMD64_SECURE_AVIC_CONTROL, 0, 0);
-+    savic_unregister_gpa(-1ULL, NULL);
++    savic_unregister_gpa(NULL);
 +}
 +
  static void x2apic_savic_setup(void)
  {
      void *backing_page;
 ...

-[RFC v2 15/17] x86/apic: Enable Secure AVIC in Control MSR
+[PATCH v3 15/17] x86/apic: Enable Secure AVIC in Control MSR
 With all the pieces in place now, enable Secure AVIC in Secure
 AVIC Control MSR. Any access to x2APIC MSRs are emulated by
-hypervisor before Secure AVIC is enabled in the Control MSR.
+the hypervisor before Secure AVIC is enabled in the control MSR.
 Post Secure AVIC enablement, all x2APIC MSR accesses (whether
-accelerated by AVIC hardware or trapped as #VC exception) operate
+accelerated by AVIC hardware or trapped as VC exception) operate
-on guest APIC backing page.
+on vCPU's APIC backing page.
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - No change.
+ - Move MSR_AMD64_SECURE_AVIC_EN* macros to this patch.
+ arch/x86/include/asm/msr-index.h    | 2 ++
  arch/x86/kernel/apic/x2apic_savic.c | 2 +-
-file changed, 1 insertion(+), 1 deletion(-)
+files changed, 3 insertions(+), 1 deletion(-)
+diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
+index XXXXXXX..XXXXXXX 100644
+--- a/arch/x86/include/asm/msr-index.h
++++ b/arch/x86/include/asm/msr-index.h
+@@ -XXX,XX +XXX,XX @@
+ #define MSR_AMD64_SNP_RESV_BIT        19
+ #define MSR_AMD64_SNP_RESERVED_MASK    GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
+ #define MSR_AMD64_SECURE_AVIC_CONTROL    0xc0010138
++#define MSR_AMD64_SECURE_AVIC_EN_BIT    0
++#define MSR_AMD64_SECURE_AVIC_EN    BIT_ULL(MSR_AMD64_SECURE_AVIC_EN_BIT)
+ #define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT 1
+ #define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI BIT_ULL(MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT)
+ #define MSR_AMD64_RMP_BASE        0xc0010132
 diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/kernel/apic/x2apic_savic.c
 +++ b/arch/x86/kernel/apic/x2apic_savic.c
 @@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
-     ret = savic_register_gpa(-1ULL, gpa);
+     ret = savic_register_gpa(gpa);
      if (ret != ES_OK)
          snp_abort();
 -    savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
 +    savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_EN | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
  }
  static int x2apic_savic_probe(void)
 --
 .34.1

-[RFC v2 16/17] x86/sev: Prevent SECURE_AVIC_CONTROL MSR interception for Secure AVIC guests
+[PATCH v3 16/17] x86/sev: Prevent SECURE_AVIC_CONTROL MSR interception for Secure AVIC guests
-The SECURE_AVIC_CONTROL MSR (0xc0010138) holds the GPA of the guest
+The SECURE_AVIC_CONTROL MSR holds the GPA of the guest APIC backing
-APIC  backing page and bitfields to enable Secure AVIC and NMI.
+page and bitfields to control enablement of Secure AVIC and NMI by
-This MSR is  populated by the guest and the hypervisor should not
+guest vCPUs. This MSR is populated by the guest and the hypervisor
-intercept it. A #VC exception will be generated otherwise. If
+should not intercept it. A #VC exception will be generated otherwise.
-this should occur and Secure AVIC is enabled, terminate guest
+If this occurs and Secure AVIC is enabled, terminate guest execution.
 execution.
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - New change.
+ - No change
  arch/x86/coco/sev/core.c | 9 +++++++++
 file changed, 9 insertions(+)
 diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/coco/sev/core.c
 +++ b/arch/x86/coco/sev/core.c
 @@ -XXX,XX +XXX,XX @@ static enum es_result __vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt
+         if (sev_status & MSR_AMD64_SNP_SECURE_TSC)
              return __vc_handle_secure_tsc_msrs(regs, write);
-         else
+         break;
              break;
 +    case MSR_AMD64_SECURE_AVIC_CONTROL:
 +        /*
 +         * AMD64_SECURE_AVIC_CONTROL should not be intercepted when
 +         * Secure AVIC is enabled. Terminate the Secure AVIC guest
 +         * if the interception is enabled.
 ...

-[RFC v2 17/17] x86/sev: Indicate SEV-SNP guest supports Secure AVIC
+[PATCH v3 17/17] x86/sev: Indicate SEV-SNP guest supports Secure AVIC
-From: Kishon Vijay Abraham I <kvijayab@amd.com>
+Now that Secure AVIC support is added in the guest, indicate SEV-SNP
 guest supports Secure AVIC feature if CONFIG_AMD_SECURE_AVIC is
 enabled.
-Now that Secure AVIC support is added in the guest, indicate SEV-SNP
+Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 guest supports Secure AVIC.
 Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
 Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
 ---
-Changes since v1:
+Changes since v2:
- - No change.
+ - Set SNP_FEATURE_SECURE_AVIC in SNP_FEATURES_PRESENT only when
    CONFIG_AMD_SECURE_AVIC is enabled.
- arch/x86/boot/compressed/sev.c | 3 ++-
+ arch/x86/boot/compressed/sev.c | 9 ++++++++-
-file changed, 2 insertions(+), 1 deletion(-)
+file changed, 8 insertions(+), 1 deletion(-)
 diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
 index XXXXXXX..XXXXXXX 100644
 --- a/arch/x86/boot/compressed/sev.c
 +++ b/arch/x86/boot/compressed/sev.c
 @@ -XXX,XX +XXX,XX @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
+                  MSR_AMD64_SNP_SECURE_AVIC |        \
+                  MSR_AMD64_SNP_RESERVED_MASK)
++#ifdef CONFIG_AMD_SECURE_AVIC
++#define SNP_FEATURE_SECURE_AVIC        MSR_AMD64_SNP_SECURE_AVIC
++#else
++#define SNP_FEATURE_SECURE_AVIC        0
++#endif
++
+ /*
+  * SNP_FEATURES_PRESENT is the mask of SNP features that are implemented
+  * by the guest kernel. As and when a new feature is implemented in the
   * guest kernel, a corresponding bit should be added to the mask.
   */
  #define SNP_FEATURES_PRESENT    (MSR_AMD64_SNP_DEBUG_SWAP |    \
 -                 MSR_AMD64_SNP_SECURE_TSC)
 +                 MSR_AMD64_SNP_SECURE_TSC |    \
-+                 MSR_AMD64_SNP_SECURE_AVIC)
++                 SNP_FEATURE_SECURE_AVIC)
  u64 snp_get_unsupported_features(u64 status)
  {
 --
 .34.1

Introduction
------------

Secure AVIC is a new hardware feature in the AMD64 architecture to
allow SEV-SNP guests to prevent hypervisor from generating unexpected
interrupts to a vCPU or otherwise violate architectural assumptions
around APIC behavior.

One of the significant differences from AVIC or emulated x2APIC is that
Secure AVIC uses a guest-owned and managed APIC backing page. It also
introduces additional fields in both the VMCB and the Secure AVIC backing
page to aid the guest in limiting which interrupt vectors can be injected
into the guest.

Guest APIC Backing Page
-----------------------
Each vCPU has a guest-allocated APIC backing page of size 4K, which
maintains APIC state for that vCPU. The x2APIC MSRs are mapped at
their corresposing x2APIC MMIO offset within the guest APIC backing
page. All x2APIC accesses by guest or Secure AVIC hardware operate
on this backing page. The backing page should be pinned and NPT entry
for it should be always mapped while the corresponding vCPU is running.

MSR Accesses
------------
Secure AVIC only supports x2APIC MSR accesses. xAPIC MMIO offset based
accesses are not supported.

Some of the MSR accesses such as ICR writes (with shorthand equal to
self), SELF_IPI, EOI, TPR writes are accelerated by Secure AVIC
hardware. Other MSR accesses generate a #VC exception. The #VC
exception handler reads/writes to the guest APIC backing page.
As guest APIC backing page is accessible to the guest, the Secure
AVIC driver code optimizes APIC register access by directly
reading/writing to the guest APIC backing page (instead of taking
the #VC exception route).

In addition to the architected MSRs, following new fields are added to
the guest APIC backing page which can be modified directly by the
guest:

a. ALLOWED_IRR

ALLOWED_IRR vector indicates the interrupt vectors which the guest
allows the hypervisor to send. The combination of host-controlled
REQUESTED_IRR vectors (part of VMCB) and ALLOWED_IRR is used by
hardware to update the IRR vectors of the Guest APIC backing page.

#Offset        #bits        Description
204h           31:0         Guest allowed vectors 0-31
214h           31:0         Guest allowed vectors 32-63
...
274h           31:0         Guest allowed vectors 224-255

ALLOWED_IRR is meant to be used specifically for vectors that the
hypervisor is allowed to inject, such as device interrupts.  Interrupt
vectors used exclusively by the guest itself (like IPI vectors) should
not be allowed to be injected into the guest for security reasons.

b. NMI Request
 
#Offset        #bits        Description
278h           0            Set by Guest to request Virtual NMI

LAPIC Timer Support
-------------------
LAPIC timer is emulated by hypervisor. So, APIC_LVTT, APIC_TMICT and
APIC_TDCR, APIC_TMCCT APIC registers are not read/written to the guest
APIC backing page and are communicated to the hypervisor using SVM_EXIT_MSR
VMGEXIT.

IPI Support
-----------
Only SELF_IPI is accelerated by Secure AVIC hardware. Other IPIs require
writing (from the Secure AVIC driver) to the IRR vector of the target CPU
backing page and then issuing VMGEXIT for the hypervisor to notify the
target vCPU.

KEXEC Support
-------------
Secure AVIC enabled guest can kexec to another kernel which has Secure
AVIC enabled, as the Hypervisor has Secure AVIC feature bit set in the
sev_status.

Driver Implementation Open Points
---------------------------------

The Secure AVIC driver only supports physical destination mode. If
logical destination mode need to be supported, then a separate x2apic
driver would be required for supporting logical destination mode.

Setting of ALLOWED_IRR vectors is done from vector.c for IOAPIC and MSI
interrupts. ALLOWED_IRR vector is not cleared when an interrupt vector
migrates to different CPU. Using a cleaner approach to manage and
configure allowed vectors needs more work.

Testing
-------

This series is based on top of commit 0f966b199269 "Merge branch into tip/master:
'x86/platform'" of tip/tip master branch.

Host Secure AVIC support patch series is at [1].

Qemu support patch is at [2].

QEMU commandline for testing Secure AVIC enabled guest:

qemu-system-x86_64 <...> -object sev-snp-guest,id=sev0,policy=0xb0000,cbitpos=51,reduced-phys-bits=1,allowed-sev-features=true,secure-avic=true

Following tests are done:

1) Boot to Prompt using initramfs and ubuntu fs.
2) Verified timer and IPI as part of the guest bootup.
3) Verified long run SCF TORTURE IPI test.

[1] https://github.com/AMDESE/linux-kvm/tree/savic-host-latest
[2] https://github.com/AMDESE/qemu/tree/secure-avic

Change since v1

- Added Kexec support.
  - Instead of doing a 2M aligned allocation for backing pages,
    allocate individual PAGE_SIZE pages for vCPUs.
  - Instead of reading Extended Topology Enumeration CPUID, APIC_ID
    value is read from Hv and updated in APIC backing page. Hv returned
    ID is checked for any duplicates.
  - Propagate all LVT* register reads and writes to Hv.
  - Check that Secure AVIC control MSR is not intercepted by Hv.
  - Fix EOI handling for level-triggered interrupts.
  - Misc cleanups and commit log updates.

Kishon Vijay Abraham I (5):
  x86/apic: Support LAPIC timer for Secure AVIC
  x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC
  x86/apic: Add support to send NMI IPI for Secure AVIC
  x86/sev: Enable NMI support for Secure AVIC
  x86/sev: Indicate SEV-SNP guest supports Secure AVIC

Neeraj Upadhyay (12):
  x86/apic: Add new driver for Secure AVIC
  x86/apic: Initialize Secure AVIC APIC backing page
  x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver
  x86/apic: Initialize APIC ID for Secure AVIC
  x86/apic: Add update_vector callback for Secure AVIC
  x86/apic: Add support to send IPI for Secure AVIC
  x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC
  x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests
  x86/apic: Handle EOI writes for SAVIC guests
  x86/apic: Add kexec support for Secure AVIC
  x86/apic: Enable Secure AVIC in Control MSR
  x86/sev: Prevent SECURE_AVIC_CONTROL MSR interception for Secure AVIC
    guests

base-commit: 0f966b1992694763de4dae6bdf817c5c1c6fc66d
-- 
2.34.1

The Secure AVIC feature provides SEV-SNP guests hardware acceleration
for performance sensitive APIC accesses while securely managing the
guest-owned APIC state through the use of a private APIC backing page.
This helps prevent hypervisor from generating unexpected interrupts for
a vCPU or otherwise violate architectural assumptions around APIC
behavior.

Add a new x2APIC driver that will serve as the base of the Secure AVIC
support. It is initially the same as the x2APIC phys driver, but will be
modified as features of Secure AVIC are implemented.

If the hypervisor sets the Secure AVIC bit in SEV_STATUS and the bit is
not set in SNP_FEATURES_PRESENT, maintain the current behavior to
enforce the guest termination.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---

Changes since v1:

- Updated the commit log to highlight the behavior for the case when
   guest SNP_FEATURES_PRESENT does not have SECURE AVIC set and
   Hv has set the bit in SEV_STATUS.

- Select AMD_SECURE_AVIC config if AMD_MEM_ENCRYPT config is enabled.

- Updated the config AMD_SECURE_AVIC description to highlight
   functional dependency on x2apic enablement.

arch/x86/Kconfig                    |  14 ++++
 arch/x86/boot/compressed/sev.c      |   1 +
 arch/x86/coco/core.c                |   3 +
 arch/x86/include/asm/msr-index.h    |   4 +-
 arch/x86/kernel/apic/Makefile       |   1 +
 arch/x86/kernel/apic/x2apic_savic.c | 112 ++++++++++++++++++++++++++++
 include/linux/cc_platform.h         |   8 ++
 7 files changed, 142 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/apic/x2apic_savic.c

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -XXX,XX +XXX,XX @@ config X86_X2APIC
 
 	  If you don't know what to do here, say N.
 
+config AMD_SECURE_AVIC
+	bool "AMD Secure AVIC"
+	depends on X86_X2APIC
+	help
+	  This enables AMD Secure AVIC support on guests that have this feature.
+
+	  AMD Secure AVIC provides hardware acceleration for performance sensitive
+	  APIC accesses and support for managing guest owned APIC state for SEV-SNP
+	  guests. Secure AVIC does not support xapic mode. It has functional
+	  dependency on x2apic being enabled in the guest.
+
+	  If you don't know what to do here, say N.
+
 config X86_POSTED_MSI
 	bool "Enable MSI and MSI-x delivery by posted interrupts"
 	depends on X86_64 && IRQ_REMAP
@@ -XXX,XX +XXX,XX @@ config AMD_MEM_ENCRYPT
 	select X86_MEM_ENCRYPT
 	select UNACCEPTED_MEMORY
 	select CRYPTO_LIB_AESGCM
+	select AMD_SECURE_AVIC
 	help
 	  Say yes to enable support for the encryption of system memory.
 	  This requires an AMD processor that supports Secure Memory
diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/boot/compressed/sev.c
+++ b/arch/x86/boot/compressed/sev.c
@@ -XXX,XX +XXX,XX @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
 				 MSR_AMD64_SNP_VMSA_REG_PROT |		\
 				 MSR_AMD64_SNP_RESERVED_BIT13 |		\
 				 MSR_AMD64_SNP_RESERVED_BIT15 |		\
+				 MSR_AMD64_SNP_SECURE_AVIC |		\
 				 MSR_AMD64_SNP_RESERVED_MASK)
 
 /*
diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/core.c
+++ b/arch/x86/coco/core.c
@@ -XXX,XX +XXX,XX @@ static bool noinstr amd_cc_platform_has(enum cc_attr attr)
 	case CC_ATTR_HOST_SEV_SNP:
 		return cc_flags.host_sev_snp;
 
+	case CC_ATTR_SNP_SECURE_AVIC:
+		return sev_status & MSR_AMD64_SNP_SECURE_AVIC;
+
 	default:
 		return false;
 	}
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -XXX,XX +XXX,XX @@
 #define MSR_AMD64_SNP_VMSA_REG_PROT	BIT_ULL(MSR_AMD64_SNP_VMSA_REG_PROT_BIT)
 #define MSR_AMD64_SNP_SMT_PROT_BIT	17
 #define MSR_AMD64_SNP_SMT_PROT		BIT_ULL(MSR_AMD64_SNP_SMT_PROT_BIT)
-#define MSR_AMD64_SNP_RESV_BIT		18
+#define MSR_AMD64_SNP_SECURE_AVIC_BIT	18
+#define MSR_AMD64_SNP_SECURE_AVIC 	BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
+#define MSR_AMD64_SNP_RESV_BIT		19
 #define MSR_AMD64_SNP_RESERVED_MASK	GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
 #define MSR_AMD64_RMP_BASE		0xc0010132
 #define MSR_AMD64_RMP_END		0xc0010133
diff --git a/arch/x86/kernel/apic/Makefile b/arch/x86/kernel/apic/Makefile
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/Makefile
+++ b/arch/x86/kernel/apic/Makefile
@@ -XXX,XX +XXX,XX @@ ifeq ($(CONFIG_X86_64),y)
 # APIC probe will depend on the listing order here
 obj-$(CONFIG_X86_NUMACHIP)	+= apic_numachip.o
 obj-$(CONFIG_X86_UV)		+= x2apic_uv_x.o
+obj-$(CONFIG_AMD_SECURE_AVIC)	+= x2apic_savic.o
 obj-$(CONFIG_X86_X2APIC)	+= x2apic_phys.o
 obj-$(CONFIG_X86_X2APIC)	+= x2apic_cluster.o
 obj-y				+= apic_flat_64.o
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * AMD Secure AVIC Support (SEV-SNP Guests)
+ *
+ * Copyright (C) 2024 Advanced Micro Devices, Inc.
+ *
+ * Author: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
+ */
+
+#include <linux/cpumask.h>
+#include <linux/cc_platform.h>
+
+#include <asm/apic.h>
+#include <asm/sev.h>
+
+#include "local.h"
+
+static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
+{
+	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
+}
+
+static void x2apic_savic_send_IPI(int cpu, int vector)
+{
+	u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
+
+	/* x2apic MSRs are special and need a special fence: */
+	weak_wrmsr_fence();
+	__x2apic_send_IPI_dest(dest, vector, APIC_DEST_PHYSICAL);
+}
+
+static void
+__send_IPI_mask(const struct cpumask *mask, int vector, int apic_dest)
+{
+	unsigned long query_cpu;
+	unsigned long this_cpu;
+	unsigned long flags;
+
+	/* x2apic MSRs are special and need a special fence: */
+	weak_wrmsr_fence();
+
+	local_irq_save(flags);
+
+	this_cpu = smp_processor_id();
+	for_each_cpu(query_cpu, mask) {
+		if (apic_dest == APIC_DEST_ALLBUT && this_cpu == query_cpu)
+			continue;
+		__x2apic_send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu),
+				       vector, APIC_DEST_PHYSICAL);
+	}
+	local_irq_restore(flags);
+}
+
+static void x2apic_savic_send_IPI_mask(const struct cpumask *mask, int vector)
+{
+	__send_IPI_mask(mask, vector, APIC_DEST_ALLINC);
+}
+
+static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, int vector)
+{
+	__send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
+}
+
+static int x2apic_savic_probe(void)
+{
+	if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
+		return 0;
+
+	if (!x2apic_mode) {
+		pr_err("Secure AVIC enabled in non x2APIC mode\n");
+		snp_abort();
+	}
+
+	pr_info("Secure AVIC Enabled\n");
+
+	return 1;
+}
+
+static struct apic apic_x2apic_savic __ro_after_init = {
+
+	.name				= "secure avic x2apic",
+	.probe				= x2apic_savic_probe,
+	.acpi_madt_oem_check		= x2apic_savic_acpi_madt_oem_check,
+
+	.dest_mode_logical		= false,
+
+	.disable_esr			= 0,
+
+	.cpu_present_to_apicid		= default_cpu_present_to_apicid,
+
+	.max_apic_id			= UINT_MAX,
+	.x2apic_set_max_apicid		= true,
+	.get_apic_id			= x2apic_get_apic_id,
+
+	.calc_dest_apicid		= apic_default_calc_apicid,
+
+	.send_IPI			= x2apic_savic_send_IPI,
+	.send_IPI_mask			= x2apic_savic_send_IPI_mask,
+	.send_IPI_mask_allbutself	= x2apic_savic_send_IPI_mask_allbutself,
+	.send_IPI_allbutself		= x2apic_send_IPI_allbutself,
+	.send_IPI_all			= x2apic_send_IPI_all,
+	.send_IPI_self			= x2apic_send_IPI_self,
+	.nmi_to_offline_cpu		= true,
+
+	.read				= native_apic_msr_read,
+	.write				= native_apic_msr_write,
+	.eoi				= native_apic_msr_eoi,
+	.icr_read			= native_x2apic_icr_read,
+	.icr_write			= native_x2apic_icr_write,
+};
+
+apic_driver(apic_x2apic_savic);
diff --git a/include/linux/cc_platform.h b/include/linux/cc_platform.h
index XXXXXXX..XXXXXXX 100644
--- a/include/linux/cc_platform.h
+++ b/include/linux/cc_platform.h
@@ -XXX,XX +XXX,XX @@ enum cc_attr {
 	 * enabled to run SEV-SNP guests.
 	 */
 	CC_ATTR_HOST_SEV_SNP,
+
+	/**
+	 * @CC_ATTR_SNP_SECURE_AVIC: Secure AVIC mode is active.
+	 *
+	 * The host kernel is running with the necessary features enabled
+	 * to run SEV-SNP guests with full Secure AVIC capabilities.
+	 */
+	CC_ATTR_SNP_SECURE_AVIC,
 };
 
 #ifdef CONFIG_ARCH_HAS_CC_PLATFORM
-- 
2.34.1

With Secure AVIC, the APIC backing page is owned and managed by guest.
Allocate and initialize APIC backing page for all guest CPUs.

The NPT entry for the vCPU's APIC backing page must always be present
when the vCPU is running in order for Secure AVIC to function. A
VMEXIT_BUSY is returned on VMRUN and the vCPU cannot be resumed if
the NPT entry for the APIC backing page is not present. Notify GPA of
the vCPU's APIC backing page to the hypervisor by using the
SVM_VMGEXIT_SECURE_AVIC GHCB protocol event. Before executing VMRUN,
the hypervisor makes use of this information to make sure the APIC backing
page is mapped in NPT.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:

- Updated commit log.
 - Allocate APIC backing page for each CPU as a separate PAGE_SIZE
   allocation with GFP_KERNEL flag.
 - Update the GPA registeration API as per the latest GHCB spec updates
   for Secure AVIC GHCB protocol event (yet to be published).
   Corresponding KVM support is here:
   https://github.com/AMDESE/linux-kvm/commit/5fbf231861207edf73bb31742f75e22cae18607b
 - Remove savic_setup_done variable.
 - Removed initialization of LVT* regs in backing page from Hv values.
   These regs will reads/writes will be propagated to Hv in subsequent
   patches.
 - Move savic_ghcb_msr_read() definition to a later patch where it will
   be first used.

arch/x86/coco/sev/core.c            | 32 +++++++++++++++++++++++++++
 arch/x86/include/asm/apic.h         |  1 +
 arch/x86/include/asm/sev.h          |  3 +++
 arch/x86/include/uapi/asm/svm.h     |  3 +++
 arch/x86/kernel/apic/apic.c         |  2 ++
 arch/x86/kernel/apic/x2apic_savic.c | 34 +++++++++++++++++++++++++++++
 6 files changed, 75 insertions(+)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
 	return ret;
 }
 
+/*
+ * Register GPA of the Secure AVIC backing page.
+ *
+ * @apic_id: APIC ID of the vCPU. Use -1ULL for the current vCPU
+ *           doing the call.
+ * @gpa    : GPA of the Secure AVIC backing page.
+ */
+enum es_result savic_register_gpa(u64 apic_id, u64 gpa)
+{
+	struct ghcb_state state;
+	struct es_em_ctxt ctxt;
+	unsigned long flags;
+	struct ghcb *ghcb;
+	int ret = 0;
+
+	local_irq_save(flags);
+
+	ghcb = __sev_get_ghcb(&state);
+
+	vc_ghcb_invalidate(ghcb);
+
+	ghcb_set_rax(ghcb, apic_id);
+	ghcb_set_rbx(ghcb, gpa);
+	ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SECURE_AVIC,
+			SVM_VMGEXIT_SECURE_AVIC_REGISTER_GPA, 0);
+
+	__sev_put_ghcb(&state);
+
+	local_irq_restore(flags);
+	return ret;
+}
+
 static void snp_register_per_cpu_ghcb(void)
 {
 	struct sev_es_runtime_data *data;
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -XXX,XX +XXX,XX @@ struct apic {
 
 	/* Probe, setup and smpboot functions */
 	int	(*probe)(void);
+	void	(*setup)(void);
 	int	(*acpi_madt_oem_check)(char *oem_id, char *oem_table_id);
 
 	void	(*init_apic_ldr)(void);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
+enum es_result savic_register_gpa(u64 apic_id, u64 gpa);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
@@ -XXX,XX +XXX,XX @@ static inline int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_
 					 struct snp_guest_request_ioctl *rio) { return -ENODEV; }
 static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
+static inline enum es_result savic_register_gpa(u64 apic_id,
+						u64 gpa) { return ES_UNSUPPORTED; }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/uapi/asm/svm.h
+++ b/arch/x86/include/uapi/asm/svm.h
@@ -XXX,XX +XXX,XX @@
 #define SVM_VMGEXIT_AP_CREATE			1
 #define SVM_VMGEXIT_AP_DESTROY			2
 #define SVM_VMGEXIT_SNP_RUN_VMPL		0x80000018
+#define SVM_VMGEXIT_SECURE_AVIC			0x8000001a
+#define SVM_VMGEXIT_SECURE_AVIC_REGISTER_GPA	0
+#define SVM_VMGEXIT_SECURE_AVIC_UNREGISTER_GPA	1
 #define SVM_VMGEXIT_HV_FEATURES			0x8000fffd
 #define SVM_VMGEXIT_TERM_REQUEST		0x8000fffe
 #define SVM_VMGEXIT_TERM_REASON(reason_set, reason_code)	\
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -XXX,XX +XXX,XX @@ static void setup_local_APIC(void)
 		return;
 	}
 
+	if (apic->setup)
+		apic->setup();
 	/*
 	 * If this comes from kexec/kcrash the APIC might be enabled in
 	 * SPIV. Soft disable it before doing further initialization.
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@
 
 #include <linux/cpumask.h>
 #include <linux/cc_platform.h>
+#include <linux/percpu-defs.h>
 
 #include <asm/apic.h>
 #include <asm/sev.h>
 
 #include "local.h"
 
+static DEFINE_PER_CPU(void *, apic_backing_page);
+
 static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 {
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, in
 	__send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
 }
 
+static void x2apic_savic_setup(void)
+{
+	void *backing_page;
+	enum es_result ret;
+	unsigned long gpa;
+
+	if (this_cpu_read(apic_backing_page))
+		return;
+
+	backing_page = kzalloc(PAGE_SIZE, GFP_KERNEL);
+	if (!backing_page)
+		snp_abort();
+	this_cpu_write(apic_backing_page, backing_page);
+	gpa = __pa(backing_page);
+
+	/*
+	 * The NPT entry for the vCPU's APIC backing page must always be
+	 * present when the vCPU is running in order for Secure AVIC to
+	 * function. A VMEXIT_BUSY is returned on VMRUN and the vCPU cannot
+	 * be resumed if the NPT entry for the APIC backing page is not
+	 * present. Notify GPA of the vCPU's APIC backing page to the
+	 * hypervisor by calling savic_register_gpa(). Before executing
+	 * VMRUN, the hypervisor makes use of this information to make sure
+	 * the APIC backing page is mapped in NPT.
+	 */
+	ret = savic_register_gpa(-1ULL, gpa);
+	if (ret != ES_OK)
+		snp_abort();
+}
+
 static int x2apic_savic_probe(void)
 {
 	if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.name				= "secure avic x2apic",
 	.probe				= x2apic_savic_probe,
 	.acpi_madt_oem_check		= x2apic_savic_acpi_madt_oem_check,
+	.setup				= x2apic_savic_setup,
 
 	.dest_mode_logical		= false,
 
-- 
2.34.1

The x2APIC registers are mapped at an offset within the guest APIC
backing page which is same as their x2APIC MMIO offset. Secure AVIC
adds new registers such as ALLOWED_IRRs (which are at 4-byte offset
within the IRR register offset range) and NMI_REQ to the APIC register
space.

Add read() and write() APIC callback functions to read and write x2APIC
registers directly from the guest APIC backing page.

When Secure AVIC is enabled, rdmsr/wrmsr of APIC registers result in
VC exception (for non-accelerated register accesses). The #VC
exception handler can read/write the x2APIC register in the guest
APIC backing page. Since doing this would increase the latency of
accessing x2APIC registers, instead of doing rdmsr/wrmsr based
accesses and handling apic register reads/writes in VC
VMEXIT_AVIC_NOACCEL error condition, the read() and write()
callbacks of Secure AVIC driver directly read/write APIC register
from/to the guest APIC backing page.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:

- APIC_ID reg write is not allowed.
 - Put information about not using #VC exception path for register
   reads/writes as comments.
 - So not read backing page if WARN_ONCE is triggered for misaligned
   reads.
 - Cleanups.

arch/x86/include/asm/apicdef.h      |   2 +
 arch/x86/kernel/apic/x2apic_savic.c | 120 +++++++++++++++++++++++++++-
 2 files changed, 120 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/apicdef.h b/arch/x86/include/asm/apicdef.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -XXX,XX +XXX,XX @@
 #define		APIC_TDR_DIV_128	0xA
 #define	APIC_EFEAT	0x400
 #define	APIC_ECTRL	0x410
+#define APIC_SEOI	0x420
+#define APIC_IER	0x480
 #define APIC_EILVTn(n)	(0x500 + 0x10 * n)
 #define		APIC_EILVT_NR_AMD_K8	1	/* # of extended interrupts */
 #define		APIC_EILVT_NR_AMD_10H	4
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@
 #include <linux/cpumask.h>
 #include <linux/cc_platform.h>
 #include <linux/percpu-defs.h>
+#include <linux/align.h>
 
 #include <asm/apic.h>
 #include <asm/sev.h>
@@ -XXX,XX +XXX,XX @@ static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
 }
 
+static inline u32 get_reg(char *page, int reg)
+{
+	return READ_ONCE(*((u32 *)(page + reg)));
+}
+
+static inline void set_reg(char *page, int reg, u32 val)
+{
+	WRITE_ONCE(*((u32 *)(page + reg)), val);
+}
+
+#define SAVIC_ALLOWED_IRR_OFFSET	0x204
+
+static u32 x2apic_savic_read(u32 reg)
+{
+	void *backing_page = this_cpu_read(apic_backing_page);
+
+	/*
+	 * When Secure AVIC is enabled, rdmsr/wrmsr of APIC registers result in
+	 * #VC exception (for non-accelerated register accesses). The #VC
+	 * exception handler can read/write the x2APIC register in the guest
+	 * APIC backing page. Since doing this would increase the latency of
+	 * accessing x2APIC registers, instead of doing rdmsr/wrmsr based
+	 * accesses and handling apic register reads/writes in
+	 * #VC VMEXIT_AVIC_NOACCEL error condition, the read() and write()
+	 * callbacks of Secure AVIC driver directly read/write APIC register
+	 * from/to the guest APIC backing page.
+	 */
+	switch (reg) {
+	case APIC_LVTT:
+	case APIC_TMICT:
+	case APIC_TMCCT:
+	case APIC_TDCR:
+	case APIC_ID:
+	case APIC_LVR:
+	case APIC_TASKPRI:
+	case APIC_ARBPRI:
+	case APIC_PROCPRI:
+	case APIC_LDR:
+	case APIC_SPIV:
+	case APIC_ESR:
+	case APIC_ICR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_LVTERR:
+	case APIC_EFEAT:
+	case APIC_ECTRL:
+	case APIC_SEOI:
+	case APIC_IER:
+	case APIC_EILVTn(0) ... APIC_EILVTn(3):
+		return get_reg(backing_page, reg);
+	case APIC_ISR ... APIC_ISR + 0x70:
+	case APIC_TMR ... APIC_TMR + 0x70:
+		if (WARN_ONCE(!IS_ALIGNED(reg, 16),
+			      "Reg offset 0x%x not aligned at 16 bytes", reg))
+			return 0;
+		return get_reg(backing_page, reg);
+	/* IRR and ALLOWED_IRR offset range */
+	case APIC_IRR ... APIC_IRR + 0x74:
+		/*
+		 * Either aligned at 16 bytes for valid IRR reg offset or a
+		 * valid Secure AVIC ALLOWED_IRR offset.
+		 */
+		if (WARN_ONCE(!(IS_ALIGNED(reg, 16) ||
+				IS_ALIGNED(reg - SAVIC_ALLOWED_IRR_OFFSET, 16)),
+			      "Misaligned IRR/ALLOWED_IRR reg offset 0x%x", reg))
+			return 0;
+		return get_reg(backing_page, reg);
+	default:
+		pr_err("Permission denied: read of Secure AVIC reg offset 0x%x\n", reg);
+		return 0;
+	}
+}
+
+#define SAVIC_NMI_REQ_OFFSET		0x278
+
+static void x2apic_savic_write(u32 reg, u32 data)
+{
+	void *backing_page = this_cpu_read(apic_backing_page);
+
+	switch (reg) {
+	case APIC_LVTT:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_TMICT:
+	case APIC_TDCR:
+	case APIC_SELF_IPI:
+	case APIC_TASKPRI:
+	case APIC_EOI:
+	case APIC_SPIV:
+	case SAVIC_NMI_REQ_OFFSET:
+	case APIC_ESR:
+	case APIC_ICR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVTERR:
+	case APIC_ECTRL:
+	case APIC_SEOI:
+	case APIC_IER:
+	case APIC_EILVTn(0) ... APIC_EILVTn(3):
+		set_reg(backing_page, reg, data);
+		break;
+	/* ALLOWED_IRR offsets are writable */
+	case SAVIC_ALLOWED_IRR_OFFSET ... SAVIC_ALLOWED_IRR_OFFSET + 0x70:
+		if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR_OFFSET, 16)) {
+			set_reg(backing_page, reg, data);
+			break;
+		}
+		fallthrough;
+	default:
+		pr_err("Permission denied: write to Secure AVIC reg offset 0x%x\n", reg);
+	}
+}
+
 static void x2apic_savic_send_IPI(int cpu, int vector)
 {
 	u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.send_IPI_self			= x2apic_send_IPI_self,
 	.nmi_to_offline_cpu		= true,
 
-	.read				= native_apic_msr_read,
-	.write				= native_apic_msr_write,
+	.read				= x2apic_savic_read,
+	.write				= x2apic_savic_write,
 	.eoi				= native_apic_msr_eoi,
 	.icr_read			= native_x2apic_icr_read,
 	.icr_write			= native_x2apic_icr_write,
-- 
2.34.1

Initialize the APIC ID in the Secure AVIC APIC backing page with
the APIC_ID msr value read from Hypervisor. Maintain a hashmap to
check and report same APIC_ID value returned by Hypervisor for two
different vCPUs.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:

- Do not read APIC_ID from CPUID. Read APIC_ID from Hv and check for
   duplicates.
 - Add a more user-friendly log message on detecting duplicate APIC
   IDs.

arch/x86/kernel/apic/x2apic_savic.c | 59 +++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@
 #include <linux/cc_platform.h>
 #include <linux/percpu-defs.h>
 #include <linux/align.h>
+#include <linux/sizes.h>
+#include <linux/llist.h>
 
 #include <asm/apic.h>
 #include <asm/sev.h>
@@ -XXX,XX +XXX,XX @@
 
 static DEFINE_PER_CPU(void *, apic_backing_page);
 
+struct apic_id_node {
+	 struct llist_node node;
+	 u32 apic_id;
+	 int cpu;
+};
+
+static DEFINE_PER_CPU(struct apic_id_node, apic_id_node);
+
+static struct llist_head *apic_id_map;
+
 static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 {
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, in
 	__send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
 }
 
+static void init_backing_page(void *backing_page)
+{
+	struct apic_id_node *next_node, *this_cpu_node;
+	unsigned int apic_map_slot;
+	u32 apic_id;
+	int cpu;
+
+	/*
+	 * Before Secure AVIC is enabled, APIC msr reads are
+	 * intercepted. APIC_ID msr read returns the value
+	 * from hv.
+	 */
+	apic_id = native_apic_msr_read(APIC_ID);
+	set_reg(backing_page, APIC_ID, apic_id);
+
+	if (!apic_id_map)
+		return;
+
+	cpu = smp_processor_id();
+	this_cpu_node = &per_cpu(apic_id_node, cpu);
+	this_cpu_node->apic_id = apic_id;
+	this_cpu_node->cpu = cpu;
+	/*
+	 * In common case, apic_ids for CPUs are sequentially numbered.
+	 * So, each CPU should hash to a different slot in the apic id
+	 * map.
+	 */
+	apic_map_slot = apic_id % nr_cpu_ids;
+	llist_add(&this_cpu_node->node, &apic_id_map[apic_map_slot]);
+	/* Each CPU checks only its next nodes for duplicates. */
+	llist_for_each_entry(next_node, this_cpu_node->node.next, node) {
+		if (WARN_ONCE(next_node->apic_id == apic_id,
+			      "Duplicate APIC %u for cpu %d and cpu %d. IPI handling will suffer!",
+			      apic_id, cpu, next_node->cpu))
+			break;
+	}
+}
+
 static void x2apic_savic_setup(void)
 {
 	void *backing_page;
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
 	if (!backing_page)
 		snp_abort();
 	this_cpu_write(apic_backing_page, backing_page);
+	init_backing_page(backing_page);
 	gpa = __pa(backing_page);
 
 	/*
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
 
 static int x2apic_savic_probe(void)
 {
+	int i;
+
 	if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
 		return 0;
 
@@ -XXX,XX +XXX,XX @@ static int x2apic_savic_probe(void)
 		snp_abort();
 	}
 
+	apic_id_map = kvmalloc(nr_cpu_ids * sizeof(*apic_id_map), GFP_KERNEL);
+
+	if (apic_id_map)
+		for (i = 0; i < nr_cpu_ids; i++)
+			init_llist_head(&apic_id_map[i]);
+
 	pr_info("Secure AVIC Enabled\n");
 
 	return 1;
-- 
2.34.1

Add update_vector callback to set/clear ALLOWED_IRR field in
the APIC backing page. The ALLOWED_IRR field indicates the
interrupt vectors which the guest allows the hypervisor to
send (typically for emulated devices). Interrupt vectors used
exclusively by the guest itself (like IPI vectors) should not
be allowed to be injected into the guest for security reasons.
The update_vector callback is invoked from APIC vector domain
whenever a vector is allocated, freed or moved.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:

- No change.

arch/x86/include/asm/apic.h         |  2 ++
 arch/x86/kernel/apic/vector.c       |  8 ++++++++
 arch/x86/kernel/apic/x2apic_savic.c | 21 +++++++++++++++++++++
 3 files changed, 31 insertions(+)

With Secure AVIC only Self-IPI is accelerated. To handle all the
other IPIs, add new callbacks for sending IPI, which write to the
IRR of the target guest vCPU's APIC backing page and then issue
GHCB protocol MSR write event for the hypervisor to notify the
target vCPU.

Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Co-developed-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:
 - Remove write_msr_to_hv() and define savic_ghcb_msr_write() in
   sev/core.c.

arch/x86/coco/sev/core.c            |  40 +++++++-
 arch/x86/include/asm/sev.h          |   2 +
 arch/x86/kernel/apic/x2apic_savic.c | 138 +++++++++++++++++++++++++---
 3 files changed, 162 insertions(+), 18 deletions(-)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ static enum es_result __vc_handle_secure_tsc_msrs(struct pt_regs *regs, bool wri
 	return ES_OK;
 }
 
-static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
+static enum es_result __vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt, bool write)
 {
 	struct pt_regs *regs = ctxt->regs;
 	enum es_result ret;
-	bool write;
-
-	/* Is it a WRMSR? */
-	write = ctxt->insn.opcode.bytes[1] == 0x30;
 
 	switch (regs->cx) {
 	case MSR_SVSM_CAA:
@@ -XXX,XX +XXX,XX @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
 	return ret;
 }
 
+static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
+{
+	return __vc_handle_msr(ghcb, ctxt, ctxt->insn.opcode.bytes[1] == 0x30);
+}
+
+void savic_ghcb_msr_write(u32 reg, u64 value)
+{
+	u64 msr = APIC_BASE_MSR + (reg >> 4);
+	struct pt_regs regs = {
+		.cx = msr,
+		.ax = lower_32_bits(value),
+		.dx = upper_32_bits(value)
+	};
+	struct es_em_ctxt ctxt = { .regs = &regs };
+	struct ghcb_state state;
+	unsigned long flags;
+	enum es_result ret;
+	struct ghcb *ghcb;
+
+	local_irq_save(flags);
+	ghcb = __sev_get_ghcb(&state);
+	vc_ghcb_invalidate(ghcb);
+
+	ret = __vc_handle_msr(ghcb, &ctxt, true);
+	if (ret != ES_OK) {
+		pr_err("Secure AVIC msr (0x%llx) write returned error (%d)\n", msr, ret);
+		/* MSR writes should never fail. Any failure is fatal error for SNP guest */
+		snp_abort();
+	}
+
+	__sev_put_ghcb(&state);
+	local_irq_restore(flags);
+}
+
 /*
  * Register GPA of the Secure AVIC backing page.
  *
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 apic_id, u64 gpa);
+void savic_ghcb_msr_write(u32 reg, u64 value);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
@@ -XXX,XX +XXX,XX @@ static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 apic_id,
 						u64 gpa) { return ES_UNSUPPORTED; }
+static void savic_ghcb_msr_write(u32 reg, u64 value) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 static void x2apic_savic_write(u32 reg, u32 data)
 {
 	void *backing_page = this_cpu_read(apic_backing_page);
+	unsigned int cfg;
 
 	switch (reg) {
 	case APIC_LVTT:
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	case APIC_LVT1:
 	case APIC_TMICT:
 	case APIC_TDCR:
-	case APIC_SELF_IPI:
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	case APIC_EILVTn(0) ... APIC_EILVTn(3):
 		set_reg(backing_page, reg, data);
 		break;
+	/* Self IPIs are accelerated by hardware, use wrmsr */
+	case APIC_SELF_IPI:
+		cfg = __prepare_ICR(APIC_DEST_SELF, data, 0);
+		native_x2apic_icr_write(cfg, 0);
+		break;
 	/* ALLOWED_IRR offsets are writable */
 	case SAVIC_ALLOWED_IRR_OFFSET ... SAVIC_ALLOWED_IRR_OFFSET + 0x70:
 		if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR_OFFSET, 16)) {
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	}
 }
 
+static void send_ipi(int cpu, int vector)
+{
+	void *backing_page;
+	int reg_off;
+
+	backing_page = per_cpu(apic_backing_page, cpu);
+	reg_off = APIC_IRR + REG_POS(vector);
+	/*
+	 * Use test_and_set_bit() to ensure that IRR updates are atomic w.r.t. other
+	 * IRR updates such as during VMRUN and during CPU interrupt handling flow.
+	 */
+	test_and_set_bit(VEC_POS(vector), (unsigned long *)((char *)backing_page + reg_off));
+}
+
+static void send_ipi_dest(u64 icr_data)
+{
+	int vector, cpu;
+
+	vector = icr_data & APIC_VECTOR_MASK;
+	cpu = icr_data >> 32;
+
+	send_ipi(cpu, vector);
+}
+
+static void send_ipi_target(u64 icr_data)
+{
+	if (icr_data & APIC_DEST_LOGICAL) {
+		pr_err("IPI target should be of PHYSICAL type\n");
+		return;
+	}
+
+	send_ipi_dest(icr_data);
+}
+
+static void send_ipi_allbut(u64 icr_data)
+{
+	const struct cpumask *self_cpu_mask = get_cpu_mask(smp_processor_id());
+	unsigned long flags;
+	int vector, cpu;
+
+	vector = icr_data & APIC_VECTOR_MASK;
+	local_irq_save(flags);
+	for_each_cpu_andnot(cpu, cpu_present_mask, self_cpu_mask)
+		send_ipi(cpu, vector);
+	savic_ghcb_msr_write(APIC_ICR, icr_data);
+	local_irq_restore(flags);
+}
+
+static void send_ipi_allinc(u64 icr_data)
+{
+	int vector;
+
+	send_ipi_allbut(icr_data);
+	vector = icr_data & APIC_VECTOR_MASK;
+	native_x2apic_icr_write(APIC_DEST_SELF | vector, 0);
+}
+
+static void x2apic_savic_icr_write(u32 icr_low, u32 icr_high)
+{
+	int dsh, vector;
+	u64 icr_data;
+
+	icr_data = ((u64)icr_high) << 32 | icr_low;
+	dsh = icr_low & APIC_DEST_ALLBUT;
+
+	switch (dsh) {
+	case APIC_DEST_SELF:
+		vector = icr_data & APIC_VECTOR_MASK;
+		x2apic_savic_write(APIC_SELF_IPI, vector);
+		break;
+	case APIC_DEST_ALLINC:
+		send_ipi_allinc(icr_data);
+		break;
+	case APIC_DEST_ALLBUT:
+		send_ipi_allbut(icr_data);
+		break;
+	default:
+		send_ipi_target(icr_data);
+		savic_ghcb_msr_write(APIC_ICR, icr_data);
+	}
+}
+
+static void __send_IPI_dest(unsigned int apicid, int vector, unsigned int dest)
+{
+	unsigned int cfg = __prepare_ICR(0, vector, dest);
+
+	x2apic_savic_icr_write(cfg, apicid);
+}
+
 static void x2apic_savic_send_IPI(int cpu, int vector)
 {
 	u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
 
-	/* x2apic MSRs are special and need a special fence: */
-	weak_wrmsr_fence();
-	__x2apic_send_IPI_dest(dest, vector, APIC_DEST_PHYSICAL);
+	__send_IPI_dest(dest, vector, APIC_DEST_PHYSICAL);
 }
 
 static void
@@ -XXX,XX +XXX,XX @@ __send_IPI_mask(const struct cpumask *mask, int vector, int apic_dest)
 	unsigned long this_cpu;
 	unsigned long flags;
 
-	/* x2apic MSRs are special and need a special fence: */
-	weak_wrmsr_fence();
-
 	local_irq_save(flags);
 
 	this_cpu = smp_processor_id();
 	for_each_cpu(query_cpu, mask) {
 		if (apic_dest == APIC_DEST_ALLBUT && this_cpu == query_cpu)
 			continue;
-		__x2apic_send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu),
-				       vector, APIC_DEST_PHYSICAL);
+		__send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu), vector,
+				      APIC_DEST_PHYSICAL);
 	}
+
 	local_irq_restore(flags);
 }
 
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_IPI_mask_allbutself(const struct cpumask *mask, in
 	__send_IPI_mask(mask, vector, APIC_DEST_ALLBUT);
 }
 
+static void __send_IPI_shorthand(int vector, u32 which)
+{
+	unsigned int cfg = __prepare_ICR(which, vector, 0);
+
+	x2apic_savic_icr_write(cfg, 0);
+}
+
+static void x2apic_savic_send_IPI_allbutself(int vector)
+{
+	__send_IPI_shorthand(vector, APIC_DEST_ALLBUT);
+}
+
+static void x2apic_savic_send_IPI_all(int vector)
+{
+	__send_IPI_shorthand(vector, APIC_DEST_ALLINC);
+}
+
+static void x2apic_savic_send_IPI_self(int vector)
+{
+	__send_IPI_shorthand(vector, APIC_DEST_SELF);
+}
+
 static void x2apic_savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
 {
 	void *backing_page;
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.send_IPI			= x2apic_savic_send_IPI,
 	.send_IPI_mask			= x2apic_savic_send_IPI_mask,
 	.send_IPI_mask_allbutself	= x2apic_savic_send_IPI_mask_allbutself,
-	.send_IPI_allbutself		= x2apic_send_IPI_allbutself,
-	.send_IPI_all			= x2apic_send_IPI_all,
-	.send_IPI_self			= x2apic_send_IPI_self,
+	.send_IPI_allbutself		= x2apic_savic_send_IPI_allbutself,
+	.send_IPI_all			= x2apic_savic_send_IPI_all,
+	.send_IPI_self			= x2apic_savic_send_IPI_self,
 	.nmi_to_offline_cpu		= true,
 
 	.read				= x2apic_savic_read,
 	.write				= x2apic_savic_write,
 	.eoi				= native_apic_msr_eoi,
 	.icr_read			= native_x2apic_icr_read,
-	.icr_write			= native_x2apic_icr_write,
+	.icr_write			= x2apic_savic_icr_write,
 
 	.update_vector			= x2apic_savic_update_vector,
 };
-- 
2.34.1

From: Kishon Vijay Abraham I <kvijayab@amd.com>

Secure AVIC requires LAPIC timer to be emulated by hypervisor. KVM
already supports emulating LAPIC timer using hrtimers. In order
to emulate LAPIC timer, APIC_LVTT, APIC_TMICT and APIC_TDCR register
values need to be propagated to the hypervisor for arming the timer.
APIC_TMCCT register value has to be read from the hypervisor, which
is required for calibrating the APIC timer. So, read/write all APIC
timer registers from/to the hypervisor.

In addition, configure APIC_ALLOWED_IRR for the hypervisor to inject
timer interrupt using LOCAL_TIMER_VECTOR.

Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Co-developed-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:

- Move savic_ghcb_msr_read() definition here.
 - Call update_vector() callback only when it is initialized.

arch/x86/coco/sev/core.c            | 27 +++++++++++++++++++++++++++
 arch/x86/include/asm/sev.h          |  2 ++
 arch/x86/kernel/apic/apic.c         |  4 ++++
 arch/x86/kernel/apic/x2apic_savic.c |  7 +++++--
 4 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
 	return __vc_handle_msr(ghcb, ctxt, ctxt->insn.opcode.bytes[1] == 0x30);
 }
 
+u64 savic_ghcb_msr_read(u32 reg)
+{
+	u64 msr = APIC_BASE_MSR + (reg >> 4);
+	struct pt_regs regs = { .cx = msr };
+	struct es_em_ctxt ctxt = { .regs = &regs };
+	struct ghcb_state state;
+	unsigned long flags;
+	enum es_result ret;
+	struct ghcb *ghcb;
+
+	local_irq_save(flags);
+	ghcb = __sev_get_ghcb(&state);
+	vc_ghcb_invalidate(ghcb);
+
+	ret = __vc_handle_msr(ghcb, &ctxt, false);
+	if (ret != ES_OK) {
+		pr_err("Secure AVIC msr (0x%llx) read returned error (%d)\n", msr, ret);
+		/* MSR read failures are treated as fatal errors */
+		snp_abort();
+	}
+
+	__sev_put_ghcb(&state);
+	local_irq_restore(flags);
+
+	return regs.ax | regs.dx << 32;
+}
+
 void savic_ghcb_msr_write(u32 reg, u64 value)
 {
 	u64 msr = APIC_BASE_MSR + (reg >> 4);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 apic_id, u64 gpa);
+u64 savic_ghcb_msr_read(u32 reg);
 void savic_ghcb_msr_write(u32 reg, u64 value);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
@@ -XXX,XX +XXX,XX @@ static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 apic_id,
 						u64 gpa) { return ES_UNSUPPORTED; }
+static inline u64 savic_ghcb_msr_read(u32 reg) { return 0; }
 static void savic_ghcb_msr_write(u32 reg, u64 value) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -XXX,XX +XXX,XX @@ static void setup_APIC_timer(void)
 						0xF, ~0UL);
 	} else
 		clockevents_register_device(levt);
+
+	if (apic->update_vector)
+		apic->update_vector(smp_processor_id(), LOCAL_TIMER_VECTOR,
+				    true);
 }
 
 /*
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 	case APIC_TMICT:
 	case APIC_TMCCT:
 	case APIC_TDCR:
+		return savic_ghcb_msr_read(reg);
 	case APIC_ID:
 	case APIC_LVR:
 	case APIC_TASKPRI:
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 
 	switch (reg) {
 	case APIC_LVTT:
-	case APIC_LVT0:
-	case APIC_LVT1:
 	case APIC_TMICT:
 	case APIC_TDCR:
+		savic_ghcb_msr_write(reg, data);
+		break;
+	case APIC_LVT0:
+	case APIC_LVT1:
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
-- 
2.34.1

From: Kishon Vijay Abraham I <kvijayab@amd.com>

Secure AVIC has introduced a new field in the APIC backing page
"NmiReq" that has to be set by the guest to request a NMI IPI.

Add support to set NmiReq appropriately to send NMI IPI.

This also requires Virtual NMI feature to be enabled in VINTRL_CTRL
field in the VMSA. However this would be added by a later commit
after adding support for injecting NMI from the hypervisor.

Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:
 - Do not set APIC_IRR for NMI IPI.

arch/x86/kernel/apic/x2apic_savic.c | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	}
 }
 
-static void send_ipi(int cpu, int vector)
+static void send_ipi(int cpu, int vector, bool nmi)
 {
 	void *backing_page;
 	int reg_off;
 
 	backing_page = per_cpu(apic_backing_page, cpu);
 	reg_off = APIC_IRR + REG_POS(vector);
-	/*
-	 * Use test_and_set_bit() to ensure that IRR updates are atomic w.r.t. other
-	 * IRR updates such as during VMRUN and during CPU interrupt handling flow.
-	 */
-	test_and_set_bit(VEC_POS(vector), (unsigned long *)((char *)backing_page + reg_off));
+	if (!nmi)
+		/*
+		 * Use test_and_set_bit() to ensure that IRR updates are atomic w.r.t. other
+		 * IRR updates such as during VMRUN and during CPU interrupt handling flow.
+		 * */
+		test_and_set_bit(VEC_POS(vector),
+				 (unsigned long *)((char *)backing_page + reg_off));
+	else
+		set_reg(backing_page, SAVIC_NMI_REQ_OFFSET, nmi);
 }
 
 static void send_ipi_dest(u64 icr_data)
 {
 	int vector, cpu;
+	bool nmi;
 
 	vector = icr_data & APIC_VECTOR_MASK;
 	cpu = icr_data >> 32;
+	nmi = ((icr_data & APIC_DM_FIXED_MASK) == APIC_DM_NMI);
 
-	send_ipi(cpu, vector);
+	send_ipi(cpu, vector, nmi);
 }
 
 static void send_ipi_target(u64 icr_data)
@@ -XXX,XX +XXX,XX @@ static void send_ipi_allbut(u64 icr_data)
 	const struct cpumask *self_cpu_mask = get_cpu_mask(smp_processor_id());
 	unsigned long flags;
 	int vector, cpu;
+	bool nmi;
 
 	vector = icr_data & APIC_VECTOR_MASK;
+	nmi = ((icr_data & APIC_DM_FIXED_MASK) == APIC_DM_NMI);
 	local_irq_save(flags);
 	for_each_cpu_andnot(cpu, cpu_present_mask, self_cpu_mask)
-		send_ipi(cpu, vector);
+		send_ipi(cpu, vector, nmi);
 	savic_ghcb_msr_write(APIC_ICR, icr_data);
 	local_irq_restore(flags);
 }
-- 
2.34.1

Secure AVIC requires "AllowedNmi" bit in the Secure AVIC Control MSR
to be set for NMI to be injected from hypervisor. Set "AllowedNmi"
bit in Secure AVIC Control MSR to allow NMI interrupts to be injected
from hypervisor.

Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:
 - No change

arch/x86/include/asm/msr-index.h    | 5 +++++
 arch/x86/kernel/apic/x2apic_savic.c | 6 ++++++
 2 files changed, 11 insertions(+)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -XXX,XX +XXX,XX @@
 #define MSR_AMD64_SNP_SECURE_AVIC 	BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
 #define MSR_AMD64_SNP_RESV_BIT		19
 #define MSR_AMD64_SNP_RESERVED_MASK	GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
+#define MSR_AMD64_SECURE_AVIC_CONTROL	0xc0010138
+#define MSR_AMD64_SECURE_AVIC_EN_BIT	0
+#define MSR_AMD64_SECURE_AVIC_EN	BIT_ULL(MSR_AMD64_SECURE_AVIC_EN_BIT)
+#define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT 1
+#define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI BIT_ULL(MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT)
 #define MSR_AMD64_RMP_BASE		0xc0010132
 #define MSR_AMD64_RMP_END		0xc0010133
 #define MSR_AMD64_RMP_CFG		0xc0010136
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static DEFINE_PER_CPU(struct apic_id_node, apic_id_node);
 
 static struct llist_head *apic_id_map;
 
+static inline void savic_wr_control_msr(u64 val)
+{
+	native_wrmsr(MSR_AMD64_SECURE_AVIC_CONTROL, lower_32_bits(val), upper_32_bits(val));
+}
+
 static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 {
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
 	ret = savic_register_gpa(-1ULL, gpa);
 	if (ret != ES_OK)
 		snp_abort();
+	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
 }
 
 static int x2apic_savic_probe(void)
-- 
2.34.1

Hypervisor need information about the current state of LVT registers
for device emulation and NMI. So, forward reads and write of these
registers to the Hypervisor for Secure AVIC guests.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:
 - New change.

arch/x86/kernel/apic/x2apic_savic.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 	case APIC_TMICT:
 	case APIC_TMCCT:
 	case APIC_TDCR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_LVTERR:
 		return savic_ghcb_msr_read(reg);
 	case APIC_ID:
 	case APIC_LVR:
@@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 	case APIC_SPIV:
 	case APIC_ESR:
 	case APIC_ICR:
-	case APIC_LVTTHMR:
-	case APIC_LVTPC:
-	case APIC_LVT0:
-	case APIC_LVT1:
-	case APIC_LVTERR:
 	case APIC_EFEAT:
 	case APIC_ECTRL:
 	case APIC_SEOI:
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	case APIC_LVTT:
 	case APIC_TMICT:
 	case APIC_TDCR:
-		savic_ghcb_msr_write(reg, data);
-		break;
 	case APIC_LVT0:
 	case APIC_LVT1:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVTERR:
+		savic_ghcb_msr_write(reg, data);
+		break;
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
 	case SAVIC_NMI_REQ_OFFSET:
 	case APIC_ESR:
 	case APIC_ICR:
-	case APIC_LVTTHMR:
-	case APIC_LVTPC:
-	case APIC_LVTERR:
 	case APIC_ECTRL:
 	case APIC_SEOI:
 	case APIC_IER:
-- 
2.34.1

Secure AVIC accelerates EOI msr writes for edge-triggered interrupts.
For level-triggered interrupts, EOI msr writes trigger #VC exception
with SVM_EXIT_AVIC_UNACCELERATED_ACCESS error code. The #VC handler
would need to trigger a GHCB protocol MSR write event to the Hypervisor.
As #VC handling adds extra overhead, directly do a GHCB protocol based
EOI write from apic->eoi() callback for level-triggered interrupts.
Use wrmsr for edge-triggered interrupts, so that hardware re-evaluates
any pending interrupt which can be delivered to guest vCPU. For level-
triggered interrupts, re-evaluation happens on return from VMGEXIT.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:
 - New change.

arch/x86/kernel/apic/x2apic_savic.c | 53 ++++++++++++++++++++++++++++-
 1 file changed, 52 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static int x2apic_savic_probe(void)
 	return 1;
 }
 
+static int find_highest_isr(void *backing_page)
+{
+	int vec_per_reg = 32;
+	int max_vec = 256;
+	u32 reg;
+	int vec;
+
+	for (vec = max_vec - 32; vec >= 0; vec -= vec_per_reg) {
+		reg = get_reg(backing_page, APIC_ISR + REG_POS(vec));
+		if (reg)
+			return __fls(reg) + vec;
+	}
+
+	return -1;
+}
+
+static void x2apic_savic_eoi(void)
+{
+	void *backing_page;
+	int reg_off;
+	int vec_pos;
+	u32 tmr;
+	int vec;
+
+	backing_page = this_cpu_read(apic_backing_page);
+
+	vec = find_highest_isr(backing_page);
+	if (WARN_ONCE(vec == -1, "EOI write without any active interrupt in APIC_ISR"))
+		return;
+
+	reg_off = REG_POS(vec);
+	vec_pos = VEC_POS(vec);
+	tmr = get_reg(backing_page, APIC_TMR + reg_off);
+	if (tmr & BIT(vec_pos)) {
+		clear_bit(vec_pos, backing_page + APIC_ISR + reg_off);
+		/*
+		 * Propagate the EOI write to hv for level-triggered interrupts.
+		 * Return to guest from GHCB protocol event takes care of
+		 * re-evaluating interrupt state.
+		 */
+		savic_ghcb_msr_write(APIC_EOI, 0);
+	} else {
+		/*
+		 * Hardware clears APIC_ISR and re-evaluates the interrupt state
+		 * to determine if there is any pending interrupt which can be
+		 * delivered to CPU.
+		 */
+		native_apic_msr_eoi();
+	}
+}
+
 static struct apic apic_x2apic_savic __ro_after_init = {
 
 	.name				= "secure avic x2apic",
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 
 	.read				= x2apic_savic_read,
 	.write				= x2apic_savic_write,
-	.eoi				= native_apic_msr_eoi,
+	.eoi				= x2apic_savic_eoi,
 	.icr_read			= native_x2apic_icr_read,
 	.icr_write			= x2apic_savic_icr_write,
 
-- 
2.34.1

Add a ->teardown callback to disable Secure AVIC before
rebooting into the new kernel. This ensures that the new
kernel does not access the old APIC backing page which was
allocated by the previous kernel. Such accesses can happen
if there are any APIC accesses done during guest boot before
Secure AVIC driver probe is done by the new kernel (as
Secure AVIC remains enabled in control msr).

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:
 - New change.

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ enum es_result savic_register_gpa(u64 apic_id, u64 gpa)
 	return ret;
 }
 
+/*
+ * Unregister GPA of the Secure AVIC backing page.
+ *
+ * @apic_id: APIC ID of the vCPU. Use -1ULL for the current vCPU
+ *           doing the call.
+ *
+ * On success, returns previously registered GPA of the Secure AVIC
+ * backing page in gpa arg.
+ */
+enum es_result savic_unregister_gpa(u64 apic_id, u64 *gpa)
+{
+	struct ghcb_state state;
+	struct es_em_ctxt ctxt;
+	unsigned long flags;
+	struct ghcb *ghcb;
+	int ret = 0;
+
+	local_irq_save(flags);
+
+	ghcb = __sev_get_ghcb(&state);
+
+	vc_ghcb_invalidate(ghcb);
+
+	ghcb_set_rax(ghcb, apic_id);
+	ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SECURE_AVIC,
+			SVM_VMGEXIT_SECURE_AVIC_UNREGISTER_GPA, 0);
+	if (gpa && ret == ES_OK)
+		*gpa = ghcb->save.rbx;
+	__sev_put_ghcb(&state);
+
+	local_irq_restore(flags);
+	return ret;
+}
+
 static void snp_register_per_cpu_ghcb(void)
 {
 	struct sev_es_runtime_data *data;
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -XXX,XX +XXX,XX @@ struct apic {
 	/* Probe, setup and smpboot functions */
 	int	(*probe)(void);
 	void	(*setup)(void);
+	void	(*teardown)(void);
 	int	(*acpi_madt_oem_check)(char *oem_id, char *oem_table_id);
 
 	void	(*init_apic_ldr)(void);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 apic_id, u64 gpa);
+enum es_result savic_unregister_gpa(u64 apic_id, u64 *gpa);
 u64 savic_ghcb_msr_read(u32 reg);
 void savic_ghcb_msr_write(u32 reg, u64 value);
 
@@ -XXX,XX +XXX,XX @@ static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 apic_id,
 						u64 gpa) { return ES_UNSUPPORTED; }
+static inline enum es_result savic_unregister_gpa(u64 apic_id,
+						  u64 *gpa) { return ES_UNSUPPORTED; }
 static inline u64 savic_ghcb_msr_read(u32 reg) { return 0; }
 static void savic_ghcb_msr_write(u32 reg, u64 value) { }
 
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -XXX,XX +XXX,XX @@ void disable_local_APIC(void)
 	if (!apic_accessible())
 		return;
 
+	if (apic->teardown)
+		apic->teardown();
+
 	apic_soft_disable();
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static void init_backing_page(void *backing_page)
 	}
 }
 
+static void x2apic_savic_teardown(void)
+{
+	/* Disable Secure AVIC */
+	native_wrmsr(MSR_AMD64_SECURE_AVIC_CONTROL, 0, 0);
+	savic_unregister_gpa(-1ULL, NULL);
+}
+
 static void x2apic_savic_setup(void)
 {
 	void *backing_page;
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.probe				= x2apic_savic_probe,
 	.acpi_madt_oem_check		= x2apic_savic_acpi_madt_oem_check,
 	.setup				= x2apic_savic_setup,
+	.teardown			= x2apic_savic_teardown,
 
 	.dest_mode_logical		= false,
 
-- 
2.34.1

With all the pieces in place now, enable Secure AVIC in Secure
AVIC Control MSR. Any access to x2APIC MSRs are emulated by
hypervisor before Secure AVIC is enabled in the Control MSR.
Post Secure AVIC enablement, all x2APIC MSR accesses (whether
accelerated by AVIC hardware or trapped as #VC exception) operate
on guest APIC backing page.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:
 - No change.

arch/x86/kernel/apic/x2apic_savic.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
 	ret = savic_register_gpa(-1ULL, gpa);
 	if (ret != ES_OK)
 		snp_abort();
-	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
+	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_EN | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
 }
 
 static int x2apic_savic_probe(void)
-- 
2.34.1

The SECURE_AVIC_CONTROL MSR (0xc0010138) holds the GPA of the guest
APIC  backing page and bitfields to enable Secure AVIC and NMI.
This MSR is  populated by the guest and the hypervisor should not
intercept it. A #VC exception will be generated otherwise. If
this should occur and Secure AVIC is enabled, terminate guest
execution.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v1:
 - New change.

arch/x86/coco/sev/core.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ static enum es_result __vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt
 			return __vc_handle_secure_tsc_msrs(regs, write);
 		else
 			break;
+	case MSR_AMD64_SECURE_AVIC_CONTROL:
+		/*
+		 * AMD64_SECURE_AVIC_CONTROL should not be intercepted when
+		 * Secure AVIC is enabled. Terminate the Secure AVIC guest
+		 * if the interception is enabled.
+		 */
+		if (cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
+			return ES_VMM_ERROR;
+		fallthrough;
 	default:
 		break;
 	}
-- 
2.34.1

Introduction
------------

Secure AVIC is a new hardware feature in the AMD64 architecture to
allow SEV-SNP guests to prevent the hypervisor from generating
unexpected interrupts to a vCPU or otherwise violate architectural
assumptions around APIC behavior.

MSR Accesses
------------
Secure AVIC only supports x2APIC MSR accesses. xAPIC MMIO offset based
accesses are not supported.

In addition to the architected MSRs, following new fields are added to
the guest APIC backing page which can be modified directly by the
guest:

a. ALLOWED_IRR

ALLOWED_IRR reg offset indicates the interrupt vectors which the guest
allows the hypervisor to send. The combination of host-controlled
REQUESTED_IRR vectors (part of VMCB) and ALLOWED_IRR is used by
hardware to update the IRR vectors of the Guest APIC backing page.

b. NMI Request
 
#Offset        #bits        Description
278h           0            Set by Guest to request Virtual NMI

Guest need to set NMI Request register to allow the Hypervisor to
inject vNMI to it.

LAPIC Timer Support
-------------------
LAPIC timer is emulated by the hypervisor. So, APIC_LVTT, APIC_TMICT and
APIC_TDCR, APIC_TMCCT APIC registers are not read/written to the guest
APIC backing page and are communicated to the hypervisor using SVM_EXIT_MSR
VMGEXIT.

KEXEC Support
-------------
Secure AVIC enabled guest can kexec to another kernel which has Secure
AVIC enabled, as the Hypervisor has Secure AVIC feature bit set in the
sev_status.

Open Points
-----------

Testing
-------

This series is based on top of commit 535bd326c565 "Merge branch into
tip/master: 'x86/tdx'" of tip/tip master branch.

Host Secure AVIC support patch series is at [1].

Qemu support patch is at [2].

QEMU commandline for testing Secure AVIC enabled guest:

qemu-system-x86_64 <...> -object sev-snp-guest,id=sev0,policy=0xb0000,cbitpos=51,reduced-phys-bits=1,allowed-sev-features=true,secure-avic=true

Following tests are done:

1) Boot to Prompt using initramfs and ubuntu fs.
2) Verified timer and IPI as part of the guest bootup.
3) Verified long run SCF TORTURE IPI test.

[1] https://github.com/AMDESE/linux-kvm/tree/savic-host-latest
[2] https://github.com/AMDESE/qemu/tree/secure-avic

Change since v2

- Removed RFC tag.
  - Change config rule to not select AMD_SECURE_AVIC config if
    AMD_MEM_ENCRYPT config is enabled.
  - Fix broken backing page GFP_KERNEL allocation in setup_local_APIC().
    Use alloc_percpu() for APIC backing pages allocation during Secure
    AVIC driver probe.
  - Remove code to check for duplicate APIC_ID returned by the
    Hypervisor. Topology evaluation code already does that during boot.
  - Fix missing update_vector() callback invocation during vector
    cleanup paths. Invoke update_vector() during setup and tearing down
    of a vector.
  - Reuse find_highest_vector() from kvm/lapic.c.
  - Change savic_register_gpa/savic_unregister_gpa() interface to be
    invoked only for the local CPU.
  - Misc cleanups.

Kishon Vijay Abraham I (2):
  x86/sev: Initialize VGIF for secondary VCPUs for Secure AVIC
  x86/sev: Enable NMI support for Secure AVIC

Neeraj Upadhyay (15):
  x86/apic: Add new driver for Secure AVIC
  x86/apic: Initialize Secure AVIC APIC backing page
  x86/apic: Populate .read()/.write() callbacks of Secure AVIC driver
  x86/apic: Initialize APIC ID for Secure AVIC
  x86/apic: Add update_vector callback for Secure AVIC
  x86/apic: Add support to send IPI for Secure AVIC
  x86/apic: Support LAPIC timer for Secure AVIC
  x86/apic: Add support to send NMI IPI for Secure AVIC
  x86/apic: Allow NMI to be injected from hypervisor for Secure AVIC
  x86/apic: Read and write LVT* APIC registers from HV for SAVIC guests
  x86/apic: Handle EOI writes for SAVIC guests
  x86/apic: Add kexec support for Secure AVIC
  x86/apic: Enable Secure AVIC in Control MSR
  x86/sev: Prevent SECURE_AVIC_CONTROL MSR interception for Secure AVIC
    guests
  x86/sev: Indicate SEV-SNP guest supports Secure AVIC

arch/x86/Kconfig                    |  13 +
 arch/x86/boot/compressed/sev.c      |  10 +-
 arch/x86/coco/core.c                |   3 +
 arch/x86/coco/sev/core.c            | 131 +++++++-
 arch/x86/include/asm/apic-emul.h    |  28 ++
 arch/x86/include/asm/apic.h         |  12 +
 arch/x86/include/asm/apicdef.h      |   2 +
 arch/x86/include/asm/msr-index.h    |   9 +-
 arch/x86/include/asm/sev.h          |   8 +
 arch/x86/include/uapi/asm/svm.h     |   3 +
 arch/x86/kernel/apic/Makefile       |   1 +
 arch/x86/kernel/apic/apic.c         |   7 +
 arch/x86/kernel/apic/init.c         |   3 +
 arch/x86/kernel/apic/vector.c       |  53 +++-
 arch/x86/kernel/apic/x2apic_savic.c | 467 ++++++++++++++++++++++++++++
 arch/x86/kvm/lapic.c                |  23 +-
 include/linux/cc_platform.h         |   8 +
 17 files changed, 742 insertions(+), 39 deletions(-)
 create mode 100644 arch/x86/include/asm/apic-emul.h
 create mode 100644 arch/x86/kernel/apic/x2apic_savic.c

base-commit: 535bd326c5657fe570f41b1f76941e449d9e2062
-- 
2.34.1

Add a new x2APIC driver that will serve as the base of the Secure AVIC
support. It is initially the same as the x2APIC phys driver, but will be
modified as features of Secure AVIC are implemented.

If the hypervisor sets the Secure AVIC bit in SEV_STATUS and the bit is
not set in SNP_FEATURES_PRESENT, maintain the current behavior to
enforce the guest termination.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:

- Do not autoselect AMD_SECURE_AVIC config when AMD_MEM_ENCRYPT config
   is enabled. Make AMD_SECURE_AVIC depend on AMD_MEM_ENCRYPT.
 - Misc cleanups.

arch/x86/Kconfig                    |  13 ++++
 arch/x86/boot/compressed/sev.c      |   1 +
 arch/x86/coco/core.c                |   3 +
 arch/x86/include/asm/msr-index.h    |   4 +-
 arch/x86/kernel/apic/Makefile       |   1 +
 arch/x86/kernel/apic/x2apic_savic.c | 109 ++++++++++++++++++++++++++++
 include/linux/cc_platform.h         |   8 ++
 7 files changed, 138 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/kernel/apic/x2apic_savic.c

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -XXX,XX +XXX,XX @@ config X86_X2APIC
 
 	  If in doubt, say Y.
 
+config AMD_SECURE_AVIC
+	bool "AMD Secure AVIC"
+	depends on AMD_MEM_ENCRYPT && X86_X2APIC
+	help
+	  Enable this to get AMD Secure AVIC support on guests that have this feature.
+
+	  AMD Secure AVIC provides hardware acceleration for performance sensitive
+	  APIC accesses and support for managing guest owned APIC state for SEV-SNP
+	  guests. Secure AVIC does not support xapic mode. It has functional
+	  dependency on x2apic being enabled in the guest.
+
+	  If you don't know what to do here, say N.
+
 config X86_POSTED_MSI
 	bool "Enable MSI and MSI-x delivery by posted interrupts"
 	depends on X86_64 && IRQ_REMAP
diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/boot/compressed/sev.c
+++ b/arch/x86/boot/compressed/sev.c
@@ -XXX,XX +XXX,XX @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
 				 MSR_AMD64_SNP_VMSA_REG_PROT |		\
 				 MSR_AMD64_SNP_RESERVED_BIT13 |		\
 				 MSR_AMD64_SNP_RESERVED_BIT15 |		\
+				 MSR_AMD64_SNP_SECURE_AVIC |		\
 				 MSR_AMD64_SNP_RESERVED_MASK)
 
 /*
diff --git a/arch/x86/coco/core.c b/arch/x86/coco/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/core.c
+++ b/arch/x86/coco/core.c
@@ -XXX,XX +XXX,XX @@ static bool noinstr amd_cc_platform_has(enum cc_attr attr)
 	case CC_ATTR_HOST_SEV_SNP:
 		return cc_flags.host_sev_snp;
 
+	case CC_ATTR_SNP_SECURE_AVIC:
+		return sev_status & MSR_AMD64_SNP_SECURE_AVIC;
+
 	default:
 		return false;
 	}
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -XXX,XX +XXX,XX @@
 #define MSR_AMD64_SNP_VMSA_REG_PROT	BIT_ULL(MSR_AMD64_SNP_VMSA_REG_PROT_BIT)
 #define MSR_AMD64_SNP_SMT_PROT_BIT	17
 #define MSR_AMD64_SNP_SMT_PROT		BIT_ULL(MSR_AMD64_SNP_SMT_PROT_BIT)
-#define MSR_AMD64_SNP_RESV_BIT		18
+#define MSR_AMD64_SNP_SECURE_AVIC_BIT	18
+#define MSR_AMD64_SNP_SECURE_AVIC	BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
+#define MSR_AMD64_SNP_RESV_BIT		19
 #define MSR_AMD64_SNP_RESERVED_MASK	GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
 #define MSR_AMD64_RMP_BASE		0xc0010132
 #define MSR_AMD64_RMP_END		0xc0010133
diff --git a/arch/x86/kernel/apic/Makefile b/arch/x86/kernel/apic/Makefile
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/Makefile
+++ b/arch/x86/kernel/apic/Makefile
@@ -XXX,XX +XXX,XX @@ ifeq ($(CONFIG_X86_64),y)
 # APIC probe will depend on the listing order here
 obj-$(CONFIG_X86_NUMACHIP)	+= apic_numachip.o
 obj-$(CONFIG_X86_UV)		+= x2apic_uv_x.o
+obj-$(CONFIG_AMD_SECURE_AVIC)	+= x2apic_savic.o
 obj-$(CONFIG_X86_X2APIC)	+= x2apic_phys.o
 obj-$(CONFIG_X86_X2APIC)	+= x2apic_cluster.o
 obj-y				+= apic_flat_64.o
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * AMD Secure AVIC Support (SEV-SNP Guests)
+ *
+ * Copyright (C) 2024 Advanced Micro Devices, Inc.
+ *
+ * Author: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
+ */
+
+#include <linux/cpumask.h>
+#include <linux/cc_platform.h>
+
+#include <asm/apic.h>
+#include <asm/sev.h>
+
+#include "local.h"
+
+static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
+{
+	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
+}
+
+static void x2apic_savic_send_ipi(int cpu, int vector)
+{
+	u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
+
+	/* x2apic MSRs are special and need a special fence: */
+	weak_wrmsr_fence();
+	__x2apic_send_IPI_dest(dest, vector, APIC_DEST_PHYSICAL);
+}
+
+static void __send_ipi_mask(const struct cpumask *mask, int vector, bool excl_self)
+{
+	unsigned long query_cpu;
+	unsigned long this_cpu;
+	unsigned long flags;
+
+	/* x2apic MSRs are special and need a special fence: */
+	weak_wrmsr_fence();
+
+	local_irq_save(flags);
+
+	this_cpu = smp_processor_id();
+	for_each_cpu(query_cpu, mask) {
+		if (excl_self && this_cpu == query_cpu)
+			continue;
+		__x2apic_send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu),
+				       vector, APIC_DEST_PHYSICAL);
+	}
+	local_irq_restore(flags);
+}
+
+static void x2apic_savic_send_ipi_mask(const struct cpumask *mask, int vector)
+{
+	__send_ipi_mask(mask, vector, false);
+}
+
+static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, int vector)
+{
+	__send_ipi_mask(mask, vector, true);
+}
+
+static int x2apic_savic_probe(void)
+{
+	if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
+		return 0;
+
+	if (!x2apic_mode) {
+		pr_err("Secure AVIC enabled in non x2APIC mode\n");
+		snp_abort();
+	}
+
+	return 1;
+}
+
+static struct apic apic_x2apic_savic __ro_after_init = {
+
+	.name				= "secure avic x2apic",
+	.probe				= x2apic_savic_probe,
+	.acpi_madt_oem_check		= x2apic_savic_acpi_madt_oem_check,
+
+	.dest_mode_logical		= false,
+
+	.disable_esr			= 0,
+
+	.cpu_present_to_apicid		= default_cpu_present_to_apicid,
+
+	.max_apic_id			= UINT_MAX,
+	.x2apic_set_max_apicid		= true,
+	.get_apic_id			= x2apic_get_apic_id,
+
+	.calc_dest_apicid		= apic_default_calc_apicid,
+
+	.send_IPI			= x2apic_savic_send_ipi,
+	.send_IPI_mask			= x2apic_savic_send_ipi_mask,
+	.send_IPI_mask_allbutself	= x2apic_savic_send_ipi_mask_allbutself,
+	.send_IPI_allbutself		= x2apic_send_IPI_allbutself,
+	.send_IPI_all			= x2apic_send_IPI_all,
+	.send_IPI_self			= x2apic_send_IPI_self,
+	.nmi_to_offline_cpu		= true,
+
+	.read				= native_apic_msr_read,
+	.write				= native_apic_msr_write,
+	.eoi				= native_apic_msr_eoi,
+	.icr_read			= native_x2apic_icr_read,
+	.icr_write			= native_x2apic_icr_write,
+};
+
+apic_driver(apic_x2apic_savic);
diff --git a/include/linux/cc_platform.h b/include/linux/cc_platform.h
index XXXXXXX..XXXXXXX 100644
--- a/include/linux/cc_platform.h
+++ b/include/linux/cc_platform.h
@@ -XXX,XX +XXX,XX @@ enum cc_attr {
 	 * enabled to run SEV-SNP guests.
 	 */
 	CC_ATTR_HOST_SEV_SNP,
+
+	/**
+	 * @CC_ATTR_SNP_SECURE_AVIC: Secure AVIC mode is active.
+	 *
+	 * The host kernel is running with the necessary features enabled
+	 * to run SEV-SNP guests with full Secure AVIC capabilities.
+	 */
+	CC_ATTR_SNP_SECURE_AVIC,
 };
 
 #ifdef CONFIG_ARCH_HAS_CC_PLATFORM
-- 
2.34.1

With Secure AVIC, the APIC backing page is owned and managed by guest.
Allocate and initialize APIC backing page for all guest CPUs.

The NPT entry for a vCPU's APIC backing page must always be present
when the vCPU is running in order for Secure AVIC to function. A
VMEXIT_BUSY is returned on VMRUN and the vCPU cannot be resumed if
the NPT entry for the APIC backing page is not present. Notify GPA of
the vCPU's APIC backing page to the hypervisor by using the
SVM_VMGEXIT_SECURE_AVIC GHCB protocol event. Before executing VMRUN,
the hypervisor makes use of this information to make sure the APIC backing
page is mapped in NPT.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:

- Fix broken AP bringup due to GFP_KERNEL allocation in setup()
   callback.
 - Define apic_page struct and allocate per CPU API backing pages
   for all CPUs in Secure AVIC driver probe.
 - Change savic_register_gpa() to only allow local CPU GPA
   registration.
 - Misc cleanups.

arch/x86/coco/sev/core.c            | 27 +++++++++++++++++++
 arch/x86/coco/sev/core.c            | 27 +++++++++++++++++++
 arch/x86/include/asm/apic.h         |  1 +
 arch/x86/include/asm/sev.h          |  2 ++
 arch/x86/include/uapi/asm/svm.h     |  3 +++
 arch/x86/kernel/apic/apic.c         |  2 ++
 arch/x86/kernel/apic/x2apic_savic.c | 42 +++++++++++++++++++++++++++++
 6 files changed, 77 insertions(+)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
 	return ret;
 }
 
+enum es_result savic_register_gpa(u64 gpa)
+{
+	struct ghcb_state state;
+	struct es_em_ctxt ctxt;
+	unsigned long flags;
+	enum es_result res;
+	struct ghcb *ghcb;
+
+	local_irq_save(flags);
+
+	ghcb = __sev_get_ghcb(&state);
+
+	vc_ghcb_invalidate(ghcb);
+
+	/* Register GPA for the local CPU */
+	ghcb_set_rax(ghcb, -1ULL);
+	ghcb_set_rbx(ghcb, gpa);
+	res = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SECURE_AVIC,
+			SVM_VMGEXIT_SECURE_AVIC_REGISTER_GPA, 0);
+
+	__sev_put_ghcb(&state);
+
+	local_irq_restore(flags);
+
+	return res;
+}
+
 static void snp_register_per_cpu_ghcb(void)
 {
 	struct sev_es_runtime_data *data;
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -XXX,XX +XXX,XX @@ struct apic {
 
 	/* Probe, setup and smpboot functions */
 	int	(*probe)(void);
+	void	(*setup)(void);
 	int	(*acpi_madt_oem_check)(char *oem_id, char *oem_table_id);
 
 	void	(*init_apic_ldr)(void);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
+enum es_result savic_register_gpa(u64 gpa);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
@@ -XXX,XX +XXX,XX @@ static inline int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_
 					 struct snp_guest_request_ioctl *rio) { return -ENODEV; }
 static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
+static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/uapi/asm/svm.h
+++ b/arch/x86/include/uapi/asm/svm.h
@@ -XXX,XX +XXX,XX @@
 #define SVM_VMGEXIT_AP_CREATE			1
 #define SVM_VMGEXIT_AP_DESTROY			2
 #define SVM_VMGEXIT_SNP_RUN_VMPL		0x80000018
+#define SVM_VMGEXIT_SECURE_AVIC			0x8000001a
+#define SVM_VMGEXIT_SECURE_AVIC_REGISTER_GPA	0
+#define SVM_VMGEXIT_SECURE_AVIC_UNREGISTER_GPA	1
 #define SVM_VMGEXIT_HV_FEATURES			0x8000fffd
 #define SVM_VMGEXIT_TERM_REQUEST		0x8000fffe
 #define SVM_VMGEXIT_TERM_REASON(reason_set, reason_code)	\
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -XXX,XX +XXX,XX @@ static void setup_local_APIC(void)
 		return;
 	}
 
+	if (apic->setup)
+		apic->setup();
 	/*
 	 * If this comes from kexec/kcrash the APIC might be enabled in
 	 * SPIV. Soft disable it before doing further initialization.
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@
 
 #include <linux/cpumask.h>
 #include <linux/cc_platform.h>
+#include <linux/percpu-defs.h>
 
 #include <asm/apic.h>
 #include <asm/sev.h>
 
 #include "local.h"
 
+/* APIC_EILVTn(3) is the last defined APIC register. */
+#define NR_APIC_REGS	(APIC_EILVTn(4) >> 2)
+
+struct apic_page {
+	union {
+		u32	regs[NR_APIC_REGS];
+		u8	bytes[PAGE_SIZE];
+	};
+} __aligned(PAGE_SIZE);
+
+static struct apic_page __percpu *apic_page __ro_after_init;
+
 static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 {
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, in
 	__send_ipi_mask(mask, vector, true);
 }
 
+static void x2apic_savic_setup(void)
+{
+	void *backing_page;
+	enum es_result ret;
+	unsigned long gpa;
+
+	backing_page = this_cpu_ptr(apic_page);
+	gpa = __pa(backing_page);
+
+	/*
+	 * The NPT entry for a vCPU's APIC backing page must always be
+	 * present when the vCPU is running in order for Secure AVIC to
+	 * function. A VMEXIT_BUSY is returned on VMRUN and the vCPU cannot
+	 * be resumed if the NPT entry for the APIC backing page is not
+	 * present. Notify GPA of the vCPU's APIC backing page to the
+	 * hypervisor by calling savic_register_gpa(). Before executing
+	 * VMRUN, the hypervisor makes use of this information to make sure
+	 * the APIC backing page is mapped in NPT.
+	 */
+	ret = savic_register_gpa(gpa);
+	if (ret != ES_OK)
+		snp_abort();
+}
+
 static int x2apic_savic_probe(void)
 {
 	if (!cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
@@ -XXX,XX +XXX,XX @@ static int x2apic_savic_probe(void)
 		snp_abort();
 	}
 
+	apic_page = alloc_percpu(struct apic_page);
+	if (!apic_page)
+		snp_abort();
+
 	return 1;
 }
 
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.name				= "secure avic x2apic",
 	.probe				= x2apic_savic_probe,
 	.acpi_madt_oem_check		= x2apic_savic_acpi_madt_oem_check,
+	.setup				= x2apic_savic_setup,
 
 	.dest_mode_logical		= false,
 
-- 
2.34.1

Add read() and write() APIC callback functions to read and write x2APIC
registers directly from the guest APIC backing page of a vCPU.

When Secure AVIC is enabled, guest's rdmsr/wrmsr of APIC registers
result in VC exception (for non-accelerated register accesses) with
error code VMEXIT_AVIC_NOACCEL. The VC exception handler can read/write
the x2APIC register in the guest APIC backing page to complete the
rdmsr/wrmsr. Since doing this would increase the latency of accessing
x2APIC registers, instead of doing rdmsr/wrmsr based reg accesses
and handling reads/writes in VC exception, directly read/write APIC
registers from/to the guest APIC backing page of the vCPU in read()
and write() callbacks of the Secure AVIC APIC driver.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - Use this_cpu_ptr() instead of type casting in get_reg() and
   set_reg().

arch/x86/include/asm/apicdef.h      |   2 +
 arch/x86/kernel/apic/x2apic_savic.c | 116 +++++++++++++++++++++++++++-
 2 files changed, 116 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/apicdef.h b/arch/x86/include/asm/apicdef.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -XXX,XX +XXX,XX @@
 #define		APIC_TDR_DIV_128	0xA
 #define	APIC_EFEAT	0x400
 #define	APIC_ECTRL	0x410
+#define APIC_SEOI	0x420
+#define APIC_IER	0x480
 #define APIC_EILVTn(n)	(0x500 + 0x10 * n)
 #define		APIC_EILVT_NR_AMD_K8	1	/* # of extended interrupts */
 #define		APIC_EILVT_NR_AMD_10H	4
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@
 #include <linux/cpumask.h>
 #include <linux/cc_platform.h>
 #include <linux/percpu-defs.h>
+#include <linux/align.h>
 
 #include <asm/apic.h>
 #include <asm/sev.h>
@@ -XXX,XX +XXX,XX @@ static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
 }
 
+static __always_inline u32 get_reg(unsigned int offset)
+{
+	return READ_ONCE(this_cpu_ptr(apic_page)->regs[offset >> 2]);
+}
+
+static __always_inline void set_reg(unsigned int offset, u32 val)
+{
+	WRITE_ONCE(this_cpu_ptr(apic_page)->regs[offset >> 2], val);
+}
+
+#define SAVIC_ALLOWED_IRR	0x204
+
+static u32 x2apic_savic_read(u32 reg)
+{
+	/*
+	 * When Secure AVIC is enabled, rdmsr/wrmsr of APIC registers
+	 * result in VC exception (for non-accelerated register accesses)
+	 * with VMEXIT_AVIC_NOACCEL error code. The VC exception handler
+	 * can read/write the x2APIC register in the guest APIC backing page.
+	 * Since doing this would increase the latency of accessing x2APIC
+	 * registers, instead of doing rdmsr/wrmsr based accesses and
+	 * handling apic register reads/writes in VC exception, the read()
+	 * and write() callbacks directly read/write APIC register from/to
+	 * the vCPU APIC backing page.
+	 */
+	switch (reg) {
+	case APIC_LVTT:
+	case APIC_TMICT:
+	case APIC_TMCCT:
+	case APIC_TDCR:
+	case APIC_ID:
+	case APIC_LVR:
+	case APIC_TASKPRI:
+	case APIC_ARBPRI:
+	case APIC_PROCPRI:
+	case APIC_LDR:
+	case APIC_SPIV:
+	case APIC_ESR:
+	case APIC_ICR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_LVTERR:
+	case APIC_EFEAT:
+	case APIC_ECTRL:
+	case APIC_SEOI:
+	case APIC_IER:
+	case APIC_EILVTn(0) ... APIC_EILVTn(3):
+		return get_reg(reg);
+	case APIC_ISR ... APIC_ISR + 0x70:
+	case APIC_TMR ... APIC_TMR + 0x70:
+		if (WARN_ONCE(!IS_ALIGNED(reg, 16),
+			      "APIC reg read offset 0x%x not aligned at 16 bytes", reg))
+			return 0;
+		return get_reg(reg);
+	/* IRR and ALLOWED_IRR offset range */
+	case APIC_IRR ... APIC_IRR + 0x74:
+		/*
+		 * Either aligned at 16 bytes for valid IRR reg offset or a
+		 * valid Secure AVIC ALLOWED_IRR offset.
+		 */
+		if (WARN_ONCE(!(IS_ALIGNED(reg, 16) ||
+				IS_ALIGNED(reg - SAVIC_ALLOWED_IRR, 16)),
+			      "Misaligned IRR/ALLOWED_IRR APIC reg read offset 0x%x", reg))
+			return 0;
+		return get_reg(reg);
+	default:
+		pr_err("Permission denied: read of Secure AVIC reg offset 0x%x\n", reg);
+		return 0;
+	}
+}
+
+#define SAVIC_NMI_REQ		0x278
+
+static void x2apic_savic_write(u32 reg, u32 data)
+{
+	switch (reg) {
+	case APIC_LVTT:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_TMICT:
+	case APIC_TDCR:
+	case APIC_SELF_IPI:
+	case APIC_TASKPRI:
+	case APIC_EOI:
+	case APIC_SPIV:
+	case SAVIC_NMI_REQ:
+	case APIC_ESR:
+	case APIC_ICR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVTERR:
+	case APIC_ECTRL:
+	case APIC_SEOI:
+	case APIC_IER:
+	case APIC_EILVTn(0) ... APIC_EILVTn(3):
+		set_reg(reg, data);
+		break;
+	/* ALLOWED_IRR offsets are writable */
+	case SAVIC_ALLOWED_IRR ... SAVIC_ALLOWED_IRR + 0x70:
+		if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR, 16)) {
+			set_reg(reg, data);
+			break;
+		}
+		fallthrough;
+	default:
+		pr_err("Permission denied: write to Secure AVIC reg offset 0x%x\n", reg);
+	}
+}
+
 static void x2apic_savic_send_ipi(int cpu, int vector)
 {
 	u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.send_IPI_self			= x2apic_send_IPI_self,
 	.nmi_to_offline_cpu		= true,
 
-	.read				= native_apic_msr_read,
-	.write				= native_apic_msr_write,
+	.read				= x2apic_savic_read,
+	.write				= x2apic_savic_write,
 	.eoi				= native_apic_msr_eoi,
 	.icr_read			= native_x2apic_icr_read,
 	.icr_write			= native_x2apic_icr_write,
-- 
2.34.1

Initialize the APIC ID in the Secure AVIC APIC backing page with
the APIC_ID msr value read from Hypervisor. CPU topology evaluation
later during boot would catch and report any duplicate APIC ID for
two CPUs.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - Drop duplicate APIC ID checks.

arch/x86/kernel/apic/x2apic_savic.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, in
 	__send_ipi_mask(mask, vector, true);
 }
 
+static void init_apic_page(void)
+{
+	u32 apic_id;
+
+	/*
+	 * Before Secure AVIC is enabled, APIC msr reads are intercepted.
+	 * APIC_ID msr read returns the value from the Hypervisor.
+	 */
+	apic_id = native_apic_msr_read(APIC_ID);
+	set_reg(APIC_ID, apic_id);
+}
+
 static void x2apic_savic_setup(void)
 {
 	void *backing_page;
 	enum es_result ret;
 	unsigned long gpa;
 
+	init_apic_page();
 	backing_page = this_cpu_ptr(apic_page);
 	gpa = __pa(backing_page);
 
-- 
2.34.1

Add update_vector callback to set/clear ALLOWED_IRR field in
a vCPU's APIC backing page for external vectors. The ALLOWED_IRR
field indicates the interrupt vectors which the guest allows the
hypervisor to send (typically for emulated devices). Interrupt
vectors used exclusively by the guest itself and the vectors which
are not emulated by the hypervisor, such as IPI vectors, are part
of system vectors and are not set in the ALLOWED_IRR.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:

- Associate update_vector() invocation with vector allocation/free
   calls.
 - Cleanup and simplify vector bitmap calculation for ALLOWED_IRR.

arch/x86/include/asm/apic.h         |  2 +
 arch/x86/include/asm/apic.h         |  2 +
 arch/x86/kernel/apic/vector.c       | 59 +++++++++++++++++++++++------
 arch/x86/kernel/apic/x2apic_savic.c | 20 ++++++++++
 3 files changed, 69 insertions(+), 12 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -XXX,XX +XXX,XX @@ struct apic {
 	/* wakeup secondary CPU using 64-bit wakeup point */
 	int	(*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip);
 
+	void	(*update_vector)(unsigned int cpu, unsigned int vector, bool set);
+
 	char	*name;
 };
 
diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -XXX,XX +XXX,XX @@ static void apic_update_irq_cfg(struct irq_data *irqd, unsigned int vector,
 			    apicd->hw_irq_cfg.dest_apicid);
 }
 
-static void apic_update_vector(struct irq_data *irqd, unsigned int newvec,
-			       unsigned int newcpu)
+static inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set)
+{
+	if (apic->update_vector)
+		apic->update_vector(cpu, vector, set);
+}
+
+static int irq_alloc_vector(const struct cpumask *dest, bool resvd, unsigned int *cpu)
+{
+	int vector;
+
+	vector = irq_matrix_alloc(vector_matrix, dest, resvd, cpu);
+
+	if (vector >= 0)
+		apic_update_vector(*cpu, vector, true);
+
+	return vector;
+}
+
+static int irq_alloc_managed_vector(unsigned int *cpu)
+{
+	int vector;
+
+	vector = irq_matrix_alloc_managed(vector_matrix, vector_searchmask, cpu);
+
+	if (vector >= 0)
+		apic_update_vector(*cpu, vector, true);
+
+	return vector;
+}
+
+static void irq_free_vector(unsigned int cpu, unsigned int vector, bool managed)
+{
+	apic_update_vector(cpu, vector, false);
+	irq_matrix_free(vector_matrix, cpu, vector, managed);
+}
+
+static void apic_chipd_update_vector(struct irq_data *irqd, unsigned int newvec,
+				     unsigned int newcpu)
 {
 	struct apic_chip_data *apicd = apic_chip_data(irqd);
 	struct irq_desc *desc = irq_data_to_desc(irqd);
@@ -XXX,XX +XXX,XX @@ static void apic_update_vector(struct irq_data *irqd, unsigned int newvec,
 		apicd->prev_cpu = apicd->cpu;
 		WARN_ON_ONCE(apicd->cpu == newcpu);
 	} else {
-		irq_matrix_free(vector_matrix, apicd->cpu, apicd->vector,
-				managed);
+		irq_free_vector(apicd->cpu, apicd->vector, managed);
 	}
 
 setnew:
@@ -XXX,XX +XXX,XX @@ assign_vector_locked(struct irq_data *irqd, const struct cpumask *dest)
 	if (apicd->move_in_progress || !hlist_unhashed(&apicd->clist))
 		return -EBUSY;
 
-	vector = irq_matrix_alloc(vector_matrix, dest, resvd, &cpu);
+	vector = irq_alloc_vector(dest, resvd, &cpu);
 	trace_vector_alloc(irqd->irq, vector, resvd, vector);
 	if (vector < 0)
 		return vector;
-	apic_update_vector(irqd, vector, cpu);
+	apic_chipd_update_vector(irqd, vector, cpu);
 	apic_update_irq_cfg(irqd, vector, cpu);
 
 	return 0;
@@ -XXX,XX +XXX,XX @@ assign_managed_vector(struct irq_data *irqd, const struct cpumask *dest)
 	/* set_affinity might call here for nothing */
 	if (apicd->vector && cpumask_test_cpu(apicd->cpu, vector_searchmask))
 		return 0;
-	vector = irq_matrix_alloc_managed(vector_matrix, vector_searchmask,
-					  &cpu);
+	vector = irq_alloc_managed_vector(&cpu);
 	trace_vector_alloc_managed(irqd->irq, vector, vector);
 	if (vector < 0)
 		return vector;
-	apic_update_vector(irqd, vector, cpu);
+	apic_chipd_update_vector(irqd, vector, cpu);
 	apic_update_irq_cfg(irqd, vector, cpu);
 	return 0;
 }
@@ -XXX,XX +XXX,XX @@ static void clear_irq_vector(struct irq_data *irqd)
 			   apicd->prev_cpu);
 
 	per_cpu(vector_irq, apicd->cpu)[vector] = VECTOR_SHUTDOWN;
-	irq_matrix_free(vector_matrix, apicd->cpu, vector, managed);
+	irq_free_vector(apicd->cpu, vector, managed);
 	apicd->vector = 0;
 
 	/* Clean up move in progress */
@@ -XXX,XX +XXX,XX @@ static void clear_irq_vector(struct irq_data *irqd)
 		return;
 
 	per_cpu(vector_irq, apicd->prev_cpu)[vector] = VECTOR_SHUTDOWN;
-	irq_matrix_free(vector_matrix, apicd->prev_cpu, vector, managed);
+	irq_free_vector(apicd->prev_cpu, vector, managed);
 	apicd->prev_vector = 0;
 	apicd->move_in_progress = 0;
 	hlist_del_init(&apicd->clist);
@@ -XXX,XX +XXX,XX @@ static bool vector_configure_legacy(unsigned int virq, struct irq_data *irqd,
 	if (irqd_is_activated(irqd)) {
 		trace_vector_setup(virq, true, 0);
 		apic_update_irq_cfg(irqd, apicd->vector, apicd->cpu);
+		apic_update_vector(apicd->cpu, apicd->vector, true);
 	} else {
 		/* Release the vector */
 		apicd->can_reserve = true;
@@ -XXX,XX +XXX,XX @@ static void free_moved_vector(struct apic_chip_data *apicd)
 	 *    affinity mask comes online.
 	 */
 	trace_vector_free_moved(apicd->irq, cpu, vector, managed);
-	irq_matrix_free(vector_matrix, cpu, vector, managed);
+	irq_free_vector(cpu, vector, managed);
 	per_cpu(vector_irq, cpu)[vector] = VECTOR_UNUSED;
 	hlist_del_init(&apicd->clist);
 	apicd->prev_vector = 0;
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, in
 	__send_ipi_mask(mask, vector, true);
 }
 
+static void x2apic_savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
+{
+	struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
+	unsigned long *sirr = (unsigned long *) &ap->bytes[SAVIC_ALLOWED_IRR];
+	unsigned int bit;
+
+	/*
+	 * The registers are 32-bit wide and 16-byte aligned.
+	 * Compensate for the resulting bit number spacing.
+	 */
+	bit = vector + 96 * (vector / 32);
+
+	if (set)
+		set_bit(bit, sirr);
+	else
+		clear_bit(bit, sirr);
+}
+
 static void init_apic_page(void)
 {
 	u32 apic_id;
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.eoi				= native_apic_msr_eoi,
 	.icr_read			= native_x2apic_icr_read,
 	.icr_write			= native_x2apic_icr_write,
+
+	.update_vector			= x2apic_savic_update_vector,
 };
 
 apic_driver(apic_x2apic_savic);
-- 
2.34.1

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - Simplify vector updates in bitmap.
 - Cleanup icr_data parcelling and unparcelling.
 - Misc cleanups.
 - Fix warning reported by kernel test robot.

arch/x86/coco/sev/core.c            |  40 ++++++-
 arch/x86/include/asm/sev.h          |   2 +
 arch/x86/kernel/apic/x2apic_savic.c | 164 ++++++++++++++++++++++------
 3 files changed, 167 insertions(+), 39 deletions(-)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ static enum es_result __vc_handle_secure_tsc_msrs(struct pt_regs *regs, bool wri
 	return ES_OK;
 }
 
-static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
+static enum es_result __vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt, bool write)
 {
 	struct pt_regs *regs = ctxt->regs;
 	enum es_result ret;
-	bool write;
-
-	/* Is it a WRMSR? */
-	write = ctxt->insn.opcode.bytes[1] == 0x30;
 
 	switch (regs->cx) {
 	case MSR_SVSM_CAA:
@@ -XXX,XX +XXX,XX @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
 	return ret;
 }
 
+static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
+{
+	return __vc_handle_msr(ghcb, ctxt, ctxt->insn.opcode.bytes[1] == 0x30);
+}
+
+void savic_ghcb_msr_write(u32 reg, u64 value)
+{
+	u64 msr = APIC_BASE_MSR + (reg >> 4);
+	struct pt_regs regs = {
+		.cx = msr,
+		.ax = lower_32_bits(value),
+		.dx = upper_32_bits(value)
+	};
+	struct es_em_ctxt ctxt = { .regs = &regs };
+	struct ghcb_state state;
+	unsigned long flags;
+	enum es_result ret;
+	struct ghcb *ghcb;
+
+	local_irq_save(flags);
+	ghcb = __sev_get_ghcb(&state);
+	vc_ghcb_invalidate(ghcb);
+
+	ret = __vc_handle_msr(ghcb, &ctxt, true);
+	if (ret != ES_OK) {
+		pr_err("Secure AVIC msr (0x%llx) write returned error (%d)\n", msr, ret);
+		/* MSR writes should never fail. Any failure is fatal error for SNP guest */
+		snp_abort();
+	}
+
+	__sev_put_ghcb(&state);
+	local_irq_restore(flags);
+}
+
 enum es_result savic_register_gpa(u64 gpa)
 {
 	struct ghcb_state state;
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 gpa);
+void savic_ghcb_msr_write(u32 reg, u64 value);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
 
@@ -XXX,XX +XXX,XX @@ static inline int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_
 static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
+static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static __always_inline void set_reg(unsigned int offset, u32 val)
 
 #define SAVIC_ALLOWED_IRR	0x204
 
+static inline void update_vector(unsigned int cpu, unsigned int offset,
+				 unsigned int vector, bool set)
+{
+	struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
+	unsigned long *reg = (unsigned long *) &ap->bytes[offset];
+	unsigned int bit;
+
+	/*
+	 * The registers are 32-bit wide and 16-byte aligned.
+	 * Compensate for the resulting bit number spacing.
+	 */
+	bit = vector + 96 * (vector / 32);
+
+	if (set)
+		set_bit(bit, reg);
+	else
+		clear_bit(bit, reg);
+}
+
 static u32 x2apic_savic_read(u32 reg)
 {
 	/*
@@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 
 #define SAVIC_NMI_REQ		0x278
 
+static inline void self_ipi_reg_write(unsigned int vector)
+{
+	/*
+	 * Secure AVIC hardware accelerates guest's MSR write to SELF_IPI
+	 * register. It updates the IRR in the APIC backing page, evaluates
+	 * the new IRR for interrupt injection and continues with guest
+	 * code execution.
+	 */
+	native_apic_msr_write(APIC_SELF_IPI, vector);
+}
+
 static void x2apic_savic_write(u32 reg, u32 data)
 {
 	switch (reg) {
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	case APIC_LVT1:
 	case APIC_TMICT:
 	case APIC_TDCR:
-	case APIC_SELF_IPI:
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	case APIC_EILVTn(0) ... APIC_EILVTn(3):
 		set_reg(reg, data);
 		break;
+	case APIC_SELF_IPI:
+		self_ipi_reg_write(data);
+		break;
 	/* ALLOWED_IRR offsets are writable */
 	case SAVIC_ALLOWED_IRR ... SAVIC_ALLOWED_IRR + 0x70:
 		if (IS_ALIGNED(reg - SAVIC_ALLOWED_IRR, 16)) {
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	}
 }
 
+static inline void send_ipi_dest(unsigned int cpu, unsigned int vector)
+{
+	update_vector(cpu, APIC_IRR, vector, true);
+}
+
+static void send_ipi_allbut(unsigned int vector)
+{
+	unsigned int cpu, src_cpu;
+	unsigned long flags;
+
+	local_irq_save(flags);
+
+	src_cpu = raw_smp_processor_id();
+
+	for_each_cpu(cpu, cpu_online_mask) {
+		if (cpu == src_cpu)
+			continue;
+		send_ipi_dest(cpu, vector);
+	}
+
+	local_irq_restore(flags);
+}
+
+static inline void self_ipi(unsigned int vector)
+{
+	u32 icr_low = APIC_SELF_IPI | vector;
+
+	native_x2apic_icr_write(icr_low, 0);
+}
+
+static void x2apic_savic_icr_write(u32 icr_low, u32 icr_high)
+{
+	unsigned int dsh, vector;
+	u64 icr_data;
+
+	dsh = icr_low & APIC_DEST_ALLBUT;
+	vector = icr_low & APIC_VECTOR_MASK;
+
+	switch (dsh) {
+	case APIC_DEST_SELF:
+		self_ipi(vector);
+		break;
+	case APIC_DEST_ALLINC:
+		self_ipi(vector);
+		fallthrough;
+	case APIC_DEST_ALLBUT:
+		send_ipi_allbut(vector);
+		break;
+	default:
+		send_ipi_dest(icr_high, vector);
+		break;
+	}
+
+	icr_data = ((u64)icr_high) << 32 | icr_low;
+	if (dsh != APIC_DEST_SELF)
+		savic_ghcb_msr_write(APIC_ICR, icr_data);
+}
+
+static void send_ipi(u32 dest, unsigned int vector, unsigned int dsh)
+{
+	unsigned int icr_low;
+
+	icr_low = __prepare_ICR(dsh, vector, APIC_DEST_PHYSICAL);
+	x2apic_savic_icr_write(icr_low, dest);
+}
+
 static void x2apic_savic_send_ipi(int cpu, int vector)
 {
 	u32 dest = per_cpu(x86_cpu_to_apicid, cpu);
 
-	/* x2apic MSRs are special and need a special fence: */
-	weak_wrmsr_fence();
-	__x2apic_send_IPI_dest(dest, vector, APIC_DEST_PHYSICAL);
+	send_ipi(dest, vector, 0);
 }
 
-static void __send_ipi_mask(const struct cpumask *mask, int vector, bool excl_self)
+static void send_ipi_mask(const struct cpumask *mask, unsigned int vector, bool excl_self)
 {
-	unsigned long query_cpu;
-	unsigned long this_cpu;
+	unsigned int this_cpu;
+	unsigned int cpu;
 	unsigned long flags;
 
-	/* x2apic MSRs are special and need a special fence: */
-	weak_wrmsr_fence();
-
 	local_irq_save(flags);
 
-	this_cpu = smp_processor_id();
-	for_each_cpu(query_cpu, mask) {
-		if (excl_self && this_cpu == query_cpu)
+	this_cpu = raw_smp_processor_id();
+
+	for_each_cpu(cpu, mask) {
+		if (excl_self && cpu == this_cpu)
 			continue;
-		__x2apic_send_IPI_dest(per_cpu(x86_cpu_to_apicid, query_cpu),
-				       vector, APIC_DEST_PHYSICAL);
+		send_ipi(per_cpu(x86_cpu_to_apicid, cpu), vector, 0);
 	}
+
 	local_irq_restore(flags);
 }
 
 static void x2apic_savic_send_ipi_mask(const struct cpumask *mask, int vector)
 {
-	__send_ipi_mask(mask, vector, false);
+	send_ipi_mask(mask, vector, false);
 }
 
 static void x2apic_savic_send_ipi_mask_allbutself(const struct cpumask *mask, int vector)
 {
-	__send_ipi_mask(mask, vector, true);
+	send_ipi_mask(mask, vector, true);
 }
 
-static void x2apic_savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
+static void x2apic_savic_send_ipi_allbutself(int vector)
 {
-	struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
-	unsigned long *sirr = (unsigned long *) &ap->bytes[SAVIC_ALLOWED_IRR];
-	unsigned int bit;
+	send_ipi(0, vector, APIC_DEST_ALLBUT);
+}
 
-	/*
-	 * The registers are 32-bit wide and 16-byte aligned.
-	 * Compensate for the resulting bit number spacing.
-	 */
-	bit = vector + 96 * (vector / 32);
+static void x2apic_savic_send_ipi_all(int vector)
+{
+	send_ipi(0, vector, APIC_DEST_ALLINC);
+}
 
-	if (set)
-		set_bit(bit, sirr);
-	else
-		clear_bit(bit, sirr);
+static void x2apic_savic_send_ipi_self(int vector)
+{
+	self_ipi_reg_write(vector);
+}
+
+static void x2apic_savic_update_vector(unsigned int cpu, unsigned int vector, bool set)
+{
+	update_vector(cpu, SAVIC_ALLOWED_IRR, vector, set);
 }
 
 static void init_apic_page(void)
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.send_IPI			= x2apic_savic_send_ipi,
 	.send_IPI_mask			= x2apic_savic_send_ipi_mask,
 	.send_IPI_mask_allbutself	= x2apic_savic_send_ipi_mask_allbutself,
-	.send_IPI_allbutself		= x2apic_send_IPI_allbutself,
-	.send_IPI_all			= x2apic_send_IPI_all,
-	.send_IPI_self			= x2apic_send_IPI_self,
+	.send_IPI_allbutself		= x2apic_savic_send_ipi_allbutself,
+	.send_IPI_all			= x2apic_savic_send_ipi_all,
+	.send_IPI_self			= x2apic_savic_send_ipi_self,
 	.nmi_to_offline_cpu		= true,
 
 	.read				= x2apic_savic_read,
 	.write				= x2apic_savic_write,
 	.eoi				= native_apic_msr_eoi,
 	.icr_read			= native_x2apic_icr_read,
-	.icr_write			= native_x2apic_icr_write,
+	.icr_write			= x2apic_savic_icr_write,
 
 	.update_vector			= x2apic_savic_update_vector,
 };
-- 
2.34.1

Secure AVIC requires LAPIC timer to be emulated by the hypervisor.
KVM already supports emulating LAPIC timer using hrtimers. In order
to emulate LAPIC timer, APIC_LVTT, APIC_TMICT and APIC_TDCR register
values need to be propagated to the hypervisor for arming the timer.
APIC_TMCCT register value has to be read from the hypervisor, which
is required for calibrating the APIC timer. So, read/write all APIC
timer registers from/to the hypervisor.

In addition, add a static call for apic's update_vector() callback,
to configure ALLOWED_IRR for the hypervisor to inject timer interrupt
using LOCAL_TIMER_VECTOR.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:

- Add static call for apic_update_vector()

arch/x86/coco/sev/core.c            | 27 +++++++++++++++++++++++++++
 arch/x86/include/asm/apic.h         |  8 ++++++++
 arch/x86/include/asm/sev.h          |  2 ++
 arch/x86/kernel/apic/apic.c         |  2 ++
 arch/x86/kernel/apic/init.c         |  3 +++
 arch/x86/kernel/apic/vector.c       |  6 ------
 arch/x86/kernel/apic/x2apic_savic.c |  7 +++++--
 7 files changed, 47 insertions(+), 8 deletions(-)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ static enum es_result vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt)
 	return __vc_handle_msr(ghcb, ctxt, ctxt->insn.opcode.bytes[1] == 0x30);
 }
 
+u64 savic_ghcb_msr_read(u32 reg)
+{
+	u64 msr = APIC_BASE_MSR + (reg >> 4);
+	struct pt_regs regs = { .cx = msr };
+	struct es_em_ctxt ctxt = { .regs = &regs };
+	struct ghcb_state state;
+	unsigned long flags;
+	enum es_result ret;
+	struct ghcb *ghcb;
+
+	local_irq_save(flags);
+	ghcb = __sev_get_ghcb(&state);
+	vc_ghcb_invalidate(ghcb);
+
+	ret = __vc_handle_msr(ghcb, &ctxt, false);
+	if (ret != ES_OK) {
+		pr_err("Secure AVIC msr (0x%llx) read returned error (%d)\n", msr, ret);
+		/* MSR read failures are treated as fatal errors */
+		snp_abort();
+	}
+
+	__sev_put_ghcb(&state);
+	local_irq_restore(flags);
+
+	return regs.ax | regs.dx << 32;
+}
+
 void savic_ghcb_msr_write(u32 reg, u64 value)
 {
 	u64 msr = APIC_BASE_MSR + (reg >> 4);
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -XXX,XX +XXX,XX @@ struct apic_override {
 	void	(*icr_write)(u32 low, u32 high);
 	int	(*wakeup_secondary_cpu)(u32 apicid, unsigned long start_eip);
 	int	(*wakeup_secondary_cpu_64)(u32 apicid, unsigned long start_eip);
+	void	(*update_vector)(unsigned int cpu, unsigned int vector, bool set);
 };
 
 /*
@@ -XXX,XX +XXX,XX @@ DECLARE_APIC_CALL(wait_icr_idle);
 DECLARE_APIC_CALL(wakeup_secondary_cpu);
 DECLARE_APIC_CALL(wakeup_secondary_cpu_64);
 DECLARE_APIC_CALL(write);
+DECLARE_APIC_CALL(update_vector);
 
 static __always_inline u32 apic_read(u32 reg)
 {
@@ -XXX,XX +XXX,XX @@ static __always_inline bool apic_id_valid(u32 apic_id)
 	return apic_id <= apic->max_apic_id;
 }
 
+static __always_inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set)
+{
+	static_call(apic_call_update_vector)(cpu, vector, set);
+}
+
 #else /* CONFIG_X86_LOCAL_APIC */
 
 static inline u32 apic_read(u32 reg) { return 0; }
@@ -XXX,XX +XXX,XX @@ static inline void apic_wait_icr_idle(void) { }
 static inline u32 safe_apic_wait_icr_idle(void) { return 0; }
 static inline void apic_native_eoi(void) { WARN_ON_ONCE(1); }
 static inline void apic_setup_apic_calls(void) { }
+static inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set) { }
 
 #define apic_update_callback(_callback, _fn) do { } while (0)
 
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 gpa);
+u64 savic_ghcb_msr_read(u32 reg);
 void savic_ghcb_msr_write(u32 reg, u64 value);
 
 #else	/* !CONFIG_AMD_MEM_ENCRYPT */
@@ -XXX,XX +XXX,XX @@ static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
 static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
+static inline u64 savic_ghcb_msr_read(u32 reg) { return 0; }
 
 #endif	/* CONFIG_AMD_MEM_ENCRYPT */
 
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -XXX,XX +XXX,XX @@ static void setup_APIC_timer(void)
 						0xF, ~0UL);
 	} else
 		clockevents_register_device(levt);
+
+	apic_update_vector(smp_processor_id(), LOCAL_TIMER_VECTOR, true);
 }
 
 /*
diff --git a/arch/x86/kernel/apic/init.c b/arch/x86/kernel/apic/init.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/init.c
+++ b/arch/x86/kernel/apic/init.c
@@ -XXX,XX +XXX,XX @@ DEFINE_APIC_CALL(wait_icr_idle);
 DEFINE_APIC_CALL(wakeup_secondary_cpu);
 DEFINE_APIC_CALL(wakeup_secondary_cpu_64);
 DEFINE_APIC_CALL(write);
+DEFINE_APIC_CALL(update_vector);
 
 EXPORT_STATIC_CALL_TRAMP_GPL(apic_call_send_IPI_mask);
 EXPORT_STATIC_CALL_TRAMP_GPL(apic_call_send_IPI_self);
@@ -XXX,XX +XXX,XX @@ static __init void restore_override_callbacks(void)
 	apply_override(icr_write);
 	apply_override(wakeup_secondary_cpu);
 	apply_override(wakeup_secondary_cpu_64);
+	apply_override(update_vector);
 }
 
 #define update_call(__cb)					\
@@ -XXX,XX +XXX,XX @@ static __init void update_static_calls(void)
 	update_call(wait_icr_idle);
 	update_call(wakeup_secondary_cpu);
 	update_call(wakeup_secondary_cpu_64);
+	update_call(update_vector);
 }
 
 void __init apic_setup_apic_calls(void)
diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -XXX,XX +XXX,XX @@ static void apic_update_irq_cfg(struct irq_data *irqd, unsigned int vector,
 			    apicd->hw_irq_cfg.dest_apicid);
 }
 
-static inline void apic_update_vector(unsigned int cpu, unsigned int vector, bool set)
-{
-	if (apic->update_vector)
-		apic->update_vector(cpu, vector, set);
-}
-
 static int irq_alloc_vector(const struct cpumask *dest, bool resvd, unsigned int *cpu)
 {
 	int vector;
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 	case APIC_TMICT:
 	case APIC_TMCCT:
 	case APIC_TDCR:
+		return savic_ghcb_msr_read(reg);
 	case APIC_ID:
 	case APIC_LVR:
 	case APIC_TASKPRI:
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 {
 	switch (reg) {
 	case APIC_LVTT:
-	case APIC_LVT0:
-	case APIC_LVT1:
 	case APIC_TMICT:
 	case APIC_TDCR:
+		savic_ghcb_msr_write(reg, data);
+		break;
+	case APIC_LVT0:
+	case APIC_LVT1:
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
-- 
2.34.1

Secure AVIC has introduced a new field in the APIC backing page
"NmiReq" that has to be set by the guest to request a NMI IPI
through APIC_ICR write.

Add support to set NmiReq appropriately to send NMI IPI.

This also requires Virtual NMI feature to be enabled in VINTRL_CTRL
field in the VMSA. However this would be added by a later commit
after adding support for injecting NMI from the hypervisor.

arch/x86/kernel/apic/x2apic_savic.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	}
 }
 
-static inline void send_ipi_dest(unsigned int cpu, unsigned int vector)
+static void send_ipi_dest(unsigned int cpu, unsigned int vector, bool nmi)
 {
+	if (nmi) {
+		struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
+
+		WRITE_ONCE(ap->regs[SAVIC_NMI_REQ >> 2], 1);
+		return;
+	}
+
 	update_vector(cpu, APIC_IRR, vector, true);
 }
 
-static void send_ipi_allbut(unsigned int vector)
+static void send_ipi_allbut(unsigned int vector, bool nmi)
 {
 	unsigned int cpu, src_cpu;
 	unsigned long flags;
@@ -XXX,XX +XXX,XX @@ static void send_ipi_allbut(unsigned int vector)
 	for_each_cpu(cpu, cpu_online_mask) {
 		if (cpu == src_cpu)
 			continue;
-		send_ipi_dest(cpu, vector);
+		send_ipi_dest(cpu, vector, nmi);
 	}
 
 	local_irq_restore(flags);
 }
 
-static inline void self_ipi(unsigned int vector)
+static inline void self_ipi(unsigned int vector, bool nmi)
 {
 	u32 icr_low = APIC_SELF_IPI | vector;
 
+	if (nmi)
+		icr_low |= APIC_DM_NMI;
+
 	native_x2apic_icr_write(icr_low, 0);
 }
 
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_icr_write(u32 icr_low, u32 icr_high)
 {
 	unsigned int dsh, vector;
 	u64 icr_data;
+	bool nmi;
 
 	dsh = icr_low & APIC_DEST_ALLBUT;
 	vector = icr_low & APIC_VECTOR_MASK;
+	nmi = ((icr_low & APIC_DM_FIXED_MASK) == APIC_DM_NMI);
 
 	switch (dsh) {
 	case APIC_DEST_SELF:
-		self_ipi(vector);
+		self_ipi(vector, nmi);
 		break;
 	case APIC_DEST_ALLINC:
-		self_ipi(vector);
+		self_ipi(vector, nmi);
 		fallthrough;
 	case APIC_DEST_ALLBUT:
-		send_ipi_allbut(vector);
+		send_ipi_allbut(vector, nmi);
 		break;
 	default:
-		send_ipi_dest(icr_high, vector);
+		send_ipi_dest(icr_high, vector, nmi);
 		break;
 	}
 
-- 
2.34.1

Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - Remove MSR_AMD64_SECURE_AVIC_EN macros from this patch.

arch/x86/include/asm/msr-index.h    | 3 +++
 arch/x86/kernel/apic/x2apic_savic.c | 6 ++++++
 2 files changed, 9 insertions(+)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -XXX,XX +XXX,XX @@
 #define MSR_AMD64_SNP_SECURE_AVIC	BIT_ULL(MSR_AMD64_SNP_SECURE_AVIC_BIT)
 #define MSR_AMD64_SNP_RESV_BIT		19
 #define MSR_AMD64_SNP_RESERVED_MASK	GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
+#define MSR_AMD64_SECURE_AVIC_CONTROL	0xc0010138
+#define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT 1
+#define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI BIT_ULL(MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT)
 #define MSR_AMD64_RMP_BASE		0xc0010132
 #define MSR_AMD64_RMP_END		0xc0010133
 #define MSR_AMD64_RMP_CFG		0xc0010136
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ struct apic_page {
 
 static struct apic_page __percpu *apic_page __ro_after_init;
 
+static inline void savic_wr_control_msr(u64 val)
+{
+	native_wrmsr(MSR_AMD64_SECURE_AVIC_CONTROL, lower_32_bits(val), upper_32_bits(val));
+}
+
 static int x2apic_savic_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
 {
 	return x2apic_enabled() && cc_platform_has(CC_ATTR_SNP_SECURE_AVIC);
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
 	ret = savic_register_gpa(gpa);
 	if (ret != ES_OK)
 		snp_abort();
+	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
 }
 
 static int x2apic_savic_probe(void)
-- 
2.34.1

Hypervisor need information about the current state of LVT registers
for device emulation and NMI. So, forward reads and write of these
registers to the Hypervisor for Secure AVIC guests.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - No change.

arch/x86/kernel/apic/x2apic_savic.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 	case APIC_TMICT:
 	case APIC_TMCCT:
 	case APIC_TDCR:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVT0:
+	case APIC_LVT1:
+	case APIC_LVTERR:
 		return savic_ghcb_msr_read(reg);
 	case APIC_ID:
 	case APIC_LVR:
@@ -XXX,XX +XXX,XX @@ static u32 x2apic_savic_read(u32 reg)
 	case APIC_SPIV:
 	case APIC_ESR:
 	case APIC_ICR:
-	case APIC_LVTTHMR:
-	case APIC_LVTPC:
-	case APIC_LVT0:
-	case APIC_LVT1:
-	case APIC_LVTERR:
 	case APIC_EFEAT:
 	case APIC_ECTRL:
 	case APIC_SEOI:
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_write(u32 reg, u32 data)
 	case APIC_LVTT:
 	case APIC_TMICT:
 	case APIC_TDCR:
-		savic_ghcb_msr_write(reg, data);
-		break;
 	case APIC_LVT0:
 	case APIC_LVT1:
+	case APIC_LVTTHMR:
+	case APIC_LVTPC:
+	case APIC_LVTERR:
+		savic_ghcb_msr_write(reg, data);
+		break;
 	case APIC_TASKPRI:
 	case APIC_EOI:
 	case APIC_SPIV:
 	case SAVIC_NMI_REQ:
 	case APIC_ESR:
 	case APIC_ICR:
-	case APIC_LVTTHMR:
-	case APIC_LVTPC:
-	case APIC_LVTERR:
 	case APIC_ECTRL:
 	case APIC_SEOI:
 	case APIC_IER:
-- 
2.34.1

Secure AVIC accelerates guest's EOI msr writes for edge-triggered
interrupts. For level-triggered interrupts, EOI msr writes trigger
VC exception with SVM_EXIT_AVIC_UNACCELERATED_ACCESS error code. The
VC handler would need to trigger a GHCB protocol MSR write event to
to notify the Hypervisor about completion of the level-triggered
interrupt. This is required for cases like emulated IOAPIC. VC exception
handling adds extra performance overhead for APIC register write. In
addition, some unaccelerated APIC register msr writes are trapped,
whereas others are faulted. This results in additional complexity in
VC exception handling for unacclerated accesses. So, directly do a GHCB
protocol based EOI write from apic->eoi() callback for level-triggered
interrupts. Use wrmsr for edge-triggered interrupts, so that hardware
re-evaluates any pending interrupt which can be delivered to guest vCPU.
For level-triggered interrupts, re-evaluation happens on return from
VMGEXIT corresponding to the GHCB event for EOI msr write.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - Reuse find_highest_vector() from kvm/lapic.c
 - Misc cleanups.

arch/x86/include/asm/apic-emul.h    | 28 +++++++++++++
 arch/x86/kernel/apic/x2apic_savic.c | 62 +++++++++++++++++++++++++----
 arch/x86/kvm/lapic.c                | 23 ++---------
 3 files changed, 85 insertions(+), 28 deletions(-)
 create mode 100644 arch/x86/include/asm/apic-emul.h

diff --git a/arch/x86/include/asm/apic-emul.h b/arch/x86/include/asm/apic-emul.h
new file mode 100644
index XXXXXXX..XXXXXXX
--- /dev/null
+++ b/arch/x86/include/asm/apic-emul.h
@@ -XXX,XX +XXX,XX @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _ASM_X86_APIC_EMUL_H
+#define _ASM_X86_APIC_EMUL_H
+
+#define MAX_APIC_VECTOR			256
+#define APIC_VECTORS_PER_REG		32
+
+static inline int apic_find_highest_vector(void *bitmap)
+{
+	unsigned int regno;
+	unsigned int vec;
+	u32 *reg;
+
+	/*
+	 * The registers int the bitmap are 32-bit wide and 16-byte
+	 * aligned. State of a vector is stored in a single bit.
+	 */
+	for (regno = MAX_APIC_VECTOR / APIC_VECTORS_PER_REG - 1; regno >= 0; regno--) {
+		vec = regno * APIC_VECTORS_PER_REG;
+		reg = bitmap + regno * 16;
+		if (*reg)
+			return __fls(*reg) + vec;
+	}
+
+	return -1;
+}
+
+#endif /* _ASM_X86_APIC_EMUL_H */
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@
 #include <linux/align.h>
 
 #include <asm/apic.h>
+#include <asm/apic-emul.h>
 #include <asm/sev.h>
 
 #include "local.h"
@@ -XXX,XX +XXX,XX @@ static __always_inline void set_reg(unsigned int offset, u32 val)
 	WRITE_ONCE(this_cpu_ptr(apic_page)->regs[offset >> 2], val);
 }
 
-#define SAVIC_ALLOWED_IRR	0x204
-
-static inline void update_vector(unsigned int cpu, unsigned int offset,
-				 unsigned int vector, bool set)
+static inline unsigned long *get_reg_bitmap(unsigned int cpu, unsigned int offset)
 {
 	struct apic_page *ap = per_cpu_ptr(apic_page, cpu);
-	unsigned long *reg = (unsigned long *) &ap->bytes[offset];
-	unsigned int bit;
 
+	return (unsigned long *) &ap->bytes[offset];
+}
+
+static inline unsigned int get_vec_bit(unsigned int vector)
+{
 	/*
 	 * The registers are 32-bit wide and 16-byte aligned.
 	 * Compensate for the resulting bit number spacing.
 	 */
-	bit = vector + 96 * (vector / 32);
+	return vector + 96 * (vector / 32);
+}
+
+static inline void update_vector(unsigned int cpu, unsigned int offset,
+				 unsigned int vector, bool set)
+{
+	unsigned long *reg = get_reg_bitmap(cpu, offset);
+	unsigned int bit = get_vec_bit(vector);
 
 	if (set)
 		set_bit(bit, reg);
@@ -XXX,XX +XXX,XX @@ static inline void update_vector(unsigned int cpu, unsigned int offset,
 		clear_bit(bit, reg);
 }
 
+static inline bool test_vector(unsigned int cpu, unsigned int offset, unsigned int vector)
+{
+	unsigned long *reg = get_reg_bitmap(cpu, offset);
+	unsigned int bit = get_vec_bit(vector);
+
+	return test_bit(bit, reg);
+}
+
+#define SAVIC_ALLOWED_IRR	0x204
+
 static u32 x2apic_savic_read(u32 reg)
 {
 	/*
@@ -XXX,XX +XXX,XX @@ static int x2apic_savic_probe(void)
 	return 1;
 }
 
+static void x2apic_savic_eoi(void)
+{
+	unsigned int cpu;
+	int vec;
+
+	cpu = raw_smp_processor_id();
+	vec = apic_find_highest_vector(get_reg_bitmap(cpu, APIC_ISR));
+	if (WARN_ONCE(vec == -1, "EOI write while no active interrupt in APIC_ISR"))
+		return;
+
+	if (test_vector(cpu, APIC_TMR, vec)) {
+		update_vector(cpu, APIC_ISR, vec, false);
+		/*
+		 * Propagate the EOI write to hv for level-triggered interrupts.
+		 * Return to guest from GHCB protocol event takes care of
+		 * re-evaluating interrupt state.
+		 */
+		savic_ghcb_msr_write(APIC_EOI, 0);
+	} else {
+		/*
+		 * Hardware clears APIC_ISR and re-evaluates the interrupt state
+		 * to determine if there is any pending interrupt which can be
+		 * delivered to CPU.
+		 */
+		native_apic_msr_eoi();
+	}
+}
+
 static struct apic apic_x2apic_savic __ro_after_init = {
 
 	.name				= "secure avic x2apic",
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 
 	.read				= x2apic_savic_read,
 	.write				= x2apic_savic_write,
-	.eoi				= native_apic_msr_eoi,
+	.eoi				= x2apic_savic_eoi,
 	.icr_read			= native_x2apic_icr_read,
 	.icr_write			= x2apic_savic_icr_write,
 
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -XXX,XX +XXX,XX @@
 #include <linux/export.h>
 #include <linux/math64.h>
 #include <linux/slab.h>
+#include <asm/apic-emul.h>
 #include <asm/processor.h>
 #include <asm/mce.h>
 #include <asm/msr.h>
@@ -XXX,XX +XXX,XX @@
 /* 14 is the version for Xeon and Pentium 8.4.8*/
 #define APIC_VERSION			0x14UL
 #define LAPIC_MMIO_LENGTH		(1 << 12)
-/* followed define is not in apicdef.h */
-#define MAX_APIC_VECTOR			256
-#define APIC_VECTORS_PER_REG		32
 
 /*
  * Enable local APIC timer advancement (tscdeadline mode only) with adaptive
@@ -XXX,XX +XXX,XX @@ static const unsigned int apic_lvt_mask[KVM_APIC_MAX_NR_LVT_ENTRIES] = {
 	[LVT_CMCI] = LVT_MASK | APIC_MODE_MASK
 };
 
-static int find_highest_vector(void *bitmap)
-{
-	int vec;
-	u32 *reg;
-
-	for (vec = MAX_APIC_VECTOR - APIC_VECTORS_PER_REG;
-	     vec >= 0; vec -= APIC_VECTORS_PER_REG) {
-		reg = bitmap + REG_POS(vec);
-		if (*reg)
-			return __fls(*reg) + vec;
-	}
-
-	return -1;
-}
-
 static u8 count_vectors(void *bitmap)
 {
 	int vec;
@@ -XXX,XX +XXX,XX @@ EXPORT_SYMBOL_GPL(kvm_apic_update_irr);
 
 static inline int apic_search_irr(struct kvm_lapic *apic)
 {
-	return find_highest_vector(apic->regs + APIC_IRR);
+	return apic_find_highest_vector(apic->regs + APIC_IRR);
 }
 
 static inline int apic_find_highest_irr(struct kvm_lapic *apic)
@@ -XXX,XX +XXX,XX @@ static inline int apic_find_highest_isr(struct kvm_lapic *apic)
 	if (likely(apic->highest_isr_cache != -1))
 		return apic->highest_isr_cache;
 
-	result = find_highest_vector(apic->regs + APIC_ISR);
+	result = apic_find_highest_vector(apic->regs + APIC_ISR);
 	ASSERT(result == -1 || result >= 16);
 
 	return result;
-- 
2.34.1

Add a apic->teardown() callback to disable Secure AVIC before
rebooting into the new kernel. This ensures that the new
kernel does not access the old APIC backing page which was
allocated by the previous kernel. Such accesses can happen
if there are any APIC accesses done during guest boot before
Secure AVIC driver probe is done by the new kernel (as Secure
AVIC would have remained enabled in the Secure AVIC control
msr).

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - Change savic_unregister_gpa() interface to allow GPA unregistration
   only for local CPU.

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ enum es_result savic_register_gpa(u64 gpa)
 	return res;
 }
 
+enum es_result savic_unregister_gpa(u64 *gpa)
+{
+	struct ghcb_state state;
+	struct es_em_ctxt ctxt;
+	unsigned long flags;
+	struct ghcb *ghcb;
+	int ret = 0;
+
+	local_irq_save(flags);
+
+	ghcb = __sev_get_ghcb(&state);
+
+	vc_ghcb_invalidate(ghcb);
+
+	ghcb_set_rax(ghcb, -1ULL);
+	ret = sev_es_ghcb_hv_call(ghcb, &ctxt, SVM_VMGEXIT_SECURE_AVIC,
+			SVM_VMGEXIT_SECURE_AVIC_UNREGISTER_GPA, 0);
+	if (gpa && ret == ES_OK)
+		*gpa = ghcb->save.rbx;
+	__sev_put_ghcb(&state);
+
+	local_irq_restore(flags);
+	return ret;
+}
+
 static void snp_register_per_cpu_ghcb(void)
 {
 	struct sev_es_runtime_data *data;
diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -XXX,XX +XXX,XX @@ struct apic {
 	/* Probe, setup and smpboot functions */
 	int	(*probe)(void);
 	void	(*setup)(void);
+	void	(*teardown)(void);
 	int	(*acpi_madt_oem_check)(char *oem_id, char *oem_table_id);
 
 	void	(*init_apic_ldr)(void);
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -XXX,XX +XXX,XX @@ int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_guest_req *req
 void __init snp_secure_tsc_prepare(void);
 void __init snp_secure_tsc_init(void);
 enum es_result savic_register_gpa(u64 gpa);
+enum es_result savic_unregister_gpa(u64 *gpa);
 u64 savic_ghcb_msr_read(u32 reg);
 void savic_ghcb_msr_write(u32 reg, u64 value);
 
@@ -XXX,XX +XXX,XX @@ static inline int snp_send_guest_request(struct snp_msg_desc *mdesc, struct snp_
 static inline void __init snp_secure_tsc_prepare(void) { }
 static inline void __init snp_secure_tsc_init(void) { }
 static inline enum es_result savic_register_gpa(u64 gpa) { return ES_UNSUPPORTED; }
+static inline enum es_result savic_unregister_gpa(u64 *gpa) { return ES_UNSUPPORTED; }
 static inline void savic_ghcb_msr_write(u32 reg, u64 value) { }
 static inline u64 savic_ghcb_msr_read(u32 reg) { return 0; }
 
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -XXX,XX +XXX,XX @@ void disable_local_APIC(void)
 	if (!apic_accessible())
 		return;
 
+	if (apic->teardown)
+		apic->teardown();
+
 	apic_soft_disable();
 
 #ifdef CONFIG_X86_32
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static void init_apic_page(void)
 	set_reg(APIC_ID, apic_id);
 }
 
+static void x2apic_savic_teardown(void)
+{
+	/* Disable Secure AVIC */
+	native_wrmsr(MSR_AMD64_SECURE_AVIC_CONTROL, 0, 0);
+	savic_unregister_gpa(NULL);
+}
+
 static void x2apic_savic_setup(void)
 {
 	void *backing_page;
@@ -XXX,XX +XXX,XX @@ static struct apic apic_x2apic_savic __ro_after_init = {
 	.probe				= x2apic_savic_probe,
 	.acpi_madt_oem_check		= x2apic_savic_acpi_madt_oem_check,
 	.setup				= x2apic_savic_setup,
+	.teardown			= x2apic_savic_teardown,
 
 	.dest_mode_logical		= false,
 
-- 
2.34.1

With all the pieces in place now, enable Secure AVIC in Secure
AVIC Control MSR. Any access to x2APIC MSRs are emulated by
the hypervisor before Secure AVIC is enabled in the control MSR.
Post Secure AVIC enablement, all x2APIC MSR accesses (whether
accelerated by AVIC hardware or trapped as VC exception) operate
on vCPU's APIC backing page.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - Move MSR_AMD64_SECURE_AVIC_EN* macros to this patch.

arch/x86/include/asm/msr-index.h    | 2 ++
 arch/x86/kernel/apic/x2apic_savic.c | 2 +-
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -XXX,XX +XXX,XX @@
 #define MSR_AMD64_SNP_RESV_BIT		19
 #define MSR_AMD64_SNP_RESERVED_MASK	GENMASK_ULL(63, MSR_AMD64_SNP_RESV_BIT)
 #define MSR_AMD64_SECURE_AVIC_CONTROL	0xc0010138
+#define MSR_AMD64_SECURE_AVIC_EN_BIT	0
+#define MSR_AMD64_SECURE_AVIC_EN	BIT_ULL(MSR_AMD64_SECURE_AVIC_EN_BIT)
 #define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT 1
 #define MSR_AMD64_SECURE_AVIC_ALLOWEDNMI BIT_ULL(MSR_AMD64_SECURE_AVIC_ALLOWEDNMI_BIT)
 #define MSR_AMD64_RMP_BASE		0xc0010132
diff --git a/arch/x86/kernel/apic/x2apic_savic.c b/arch/x86/kernel/apic/x2apic_savic.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/kernel/apic/x2apic_savic.c
+++ b/arch/x86/kernel/apic/x2apic_savic.c
@@ -XXX,XX +XXX,XX @@ static void x2apic_savic_setup(void)
 	ret = savic_register_gpa(gpa);
 	if (ret != ES_OK)
 		snp_abort();
-	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
+	savic_wr_control_msr(gpa | MSR_AMD64_SECURE_AVIC_EN | MSR_AMD64_SECURE_AVIC_ALLOWEDNMI);
 }
 
 static int x2apic_savic_probe(void)
-- 
2.34.1

The SECURE_AVIC_CONTROL MSR holds the GPA of the guest APIC backing
page and bitfields to control enablement of Secure AVIC and NMI by
guest vCPUs. This MSR is populated by the guest and the hypervisor
should not intercept it. A #VC exception will be generated otherwise.
If this occurs and Secure AVIC is enabled, terminate guest execution.

Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - No change

arch/x86/coco/sev/core.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/coco/sev/core.c b/arch/x86/coco/sev/core.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/coco/sev/core.c
+++ b/arch/x86/coco/sev/core.c
@@ -XXX,XX +XXX,XX @@ static enum es_result __vc_handle_msr(struct ghcb *ghcb, struct es_em_ctxt *ctxt
 		if (sev_status & MSR_AMD64_SNP_SECURE_TSC)
 			return __vc_handle_secure_tsc_msrs(regs, write);
 		break;
+	case MSR_AMD64_SECURE_AVIC_CONTROL:
+		/*
+		 * AMD64_SECURE_AVIC_CONTROL should not be intercepted when
+		 * Secure AVIC is enabled. Terminate the Secure AVIC guest
+		 * if the interception is enabled.
+		 */
+		if (cc_platform_has(CC_ATTR_SNP_SECURE_AVIC))
+			return ES_VMM_ERROR;
+		fallthrough;
 	default:
 		break;
 	}
-- 
2.34.1

Now that Secure AVIC support is added in the guest, indicate SEV-SNP
guest supports Secure AVIC feature if CONFIG_AMD_SECURE_AVIC is
enabled.

Co-developed-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Kishon Vijay Abraham I <kvijayab@amd.com>
Signed-off-by: Neeraj Upadhyay <Neeraj.Upadhyay@amd.com>
---
Changes since v2:
 - Set SNP_FEATURE_SECURE_AVIC in SNP_FEATURES_PRESENT only when
   CONFIG_AMD_SECURE_AVIC is enabled.

arch/x86/boot/compressed/sev.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c
index XXXXXXX..XXXXXXX 100644
--- a/arch/x86/boot/compressed/sev.c
+++ b/arch/x86/boot/compressed/sev.c
@@ -XXX,XX +XXX,XX @@ void do_boot_stage2_vc(struct pt_regs *regs, unsigned long exit_code)
 				 MSR_AMD64_SNP_SECURE_AVIC |		\
 				 MSR_AMD64_SNP_RESERVED_MASK)
 
+#ifdef CONFIG_AMD_SECURE_AVIC
+#define SNP_FEATURE_SECURE_AVIC		MSR_AMD64_SNP_SECURE_AVIC
+#else
+#define SNP_FEATURE_SECURE_AVIC		0
+#endif
+
 /*
  * SNP_FEATURES_PRESENT is the mask of SNP features that are implemented
  * by the guest kernel. As and when a new feature is implemented in the
  * guest kernel, a corresponding bit should be added to the mask.
  */
 #define SNP_FEATURES_PRESENT	(MSR_AMD64_SNP_DEBUG_SWAP |	\
-				 MSR_AMD64_SNP_SECURE_TSC)
+				 MSR_AMD64_SNP_SECURE_TSC |	\
+				 SNP_FEATURE_SECURE_AVIC)
 
 u64 snp_get_unsupported_features(u64 status)
 {
-- 
2.34.1