On POWER9 systems, the XICS-on-XIVE and XIVE KVM devices currently
allocate a bunch of VPs in the XIVE HW to accomodate the highest
VCPU id that may be possibly used in a VM. This limits the number
of VMs that can run with an in-kernel interrupt controller to 63
per POWER9 chip, irrespectively of its number of HW threads, eg.
up to 96 on a POWER9 Nimbus socket. This is an unfortunate waste
of scarce HW resources since a typical VM doesn't need that much
VPs to run.
This series exploits new attributes of the XICS-on-XIVE and XIVE
KVM devices that allow userspace to tune the numbers of VPs it
really needs.
Patches 1 to 3 are preliminary work to teach the XICS and XIVE
backends about the range of needed VCPU ids, according to the
maximum number of VCPUs specified in the QEMU command line.
Patch 5 and 6 do the actual work of configuring the KVM devices,
based on new defines brought by a patch 4. RFC since the patches
for KVM are still being discussed on the kvm-ppc list:
https://patchwork.ozlabs.org/project/kvm-ppc/list/?series=132910
As a bonus, patch 7 allows the latest machine type to automatically
set int KVM the guest core stride (VSMT) to be equal to the number
of threads per core (-smp threads=N). This makes VCPU ids contiguous
and allows to reduce the VP consumption even more.
Both KVM and QEMU changes are available here:
https://github.com/gkurz/linux/commits/xive-nr-servers-5.3
https://github.com/gkurz/qemu/commits/xive-nr-servers-for-4.2
--
Greg
---
Greg Kurz (7):
spapr, xics: Get number of servers with a XICSFabricClass method
spapr, xive: Turn "nr-ends" property into "nr-servers" property
spapr, xics, xive: Drop nr_servers argument in DT-related functions
RFC linux-headers: Update against 5.3-rc2
spapr/xics: Configure number of servers in KVM
spapr/xive: Configure number of servers in KVM
spapr: Set VSMT to smp_threads by default
hw/intc/spapr_xive.c | 24 ++++++++++++++++------
hw/intc/spapr_xive_kvm.c | 22 +++++++++++++++++++-
hw/intc/xics.c | 7 +++++++
hw/intc/xics_kvm.c | 24 ++++++++++++++++++++--
hw/intc/xics_spapr.c | 6 +++---
hw/ppc/spapr.c | 18 ++++++++++++++---
hw/ppc/spapr_irq.c | 7 +++----
include/hw/ppc/spapr.h | 1 +
include/hw/ppc/spapr_irq.h | 3 +--
include/hw/ppc/spapr_xive.h | 4 ++--
include/hw/ppc/xics.h | 2 ++
include/hw/ppc/xics_spapr.h | 3 +--
include/standard-headers/asm-x86/bootparam.h | 2 ++
include/standard-headers/asm-x86/kvm_para.h | 1 +
include/standard-headers/linux/ethtool.h | 2 ++
include/standard-headers/linux/pci_regs.h | 4 ++++
include/standard-headers/linux/virtio_ids.h | 1 +
include/standard-headers/linux/virtio_pmem.h | 6 +++---
linux-headers/asm-arm/kvm.h | 12 +++++++++++
linux-headers/asm-arm/unistd-common.h | 2 ++
linux-headers/asm-arm64/kvm.h | 17 ++++++++++++++++
linux-headers/asm-generic/mman-common.h | 15 ++++++++------
linux-headers/asm-generic/mman.h | 10 ++++-----
linux-headers/asm-generic/unistd.h | 8 +++++++
linux-headers/asm-mips/unistd_n32.h | 1 +
linux-headers/asm-mips/unistd_n64.h | 1 +
linux-headers/asm-mips/unistd_o32.h | 1 +
linux-headers/asm-powerpc/kvm.h | 3 +++
linux-headers/asm-powerpc/mman.h | 6 +-----
linux-headers/asm-powerpc/unistd_32.h | 2 ++
linux-headers/asm-powerpc/unistd_64.h | 2 ++
linux-headers/asm-s390/unistd_32.h | 2 ++
linux-headers/asm-s390/unistd_64.h | 2 ++
linux-headers/asm-x86/kvm.h | 28 ++++++++++++++++++++------
linux-headers/asm-x86/unistd_32.h | 2 ++
linux-headers/asm-x86/unistd_64.h | 2 ++
linux-headers/asm-x86/unistd_x32.h | 2 ++
linux-headers/linux/kvm.h | 7 +++++--
linux-headers/linux/psp-sev.h | 5 +----
39 files changed, 206 insertions(+), 61 deletions(-)