On POWER9 systems, the XICS-on-XIVE and XIVE KVM devices currently
allocate a bunch of VPs in the XIVE HW to accomodate the highest
VCPU id that may be possibly used in a VM. This limits the number
of VMs that can run with an in-kernel interrupt controller to 63
per POWER9 chip, irrespectively of its number of HW threads, eg.
up to 96 on a POWER9 Nimbus socket. This is an unfortunate waste
of scarce HW resources since a typical VM doesn't need that much
VPs to run.

This series exploits new attributes of the XICS-on-XIVE and XIVE
KVM devices that allow userspace to tune the numbers of VPs it
really needs. Support for this attributes has just reached linux
mainline.

In combination with recent commit 29cb4187497d "spapr: Set VSMT to
smp_threads by default" which makes VCPU ids contiguous, it is
possible to start more than 200 VMs instead of 127 on an AC922
system (two sockets, 144 HW threads).

Changes in v2:
- add an nr_servers argument to the activate() handler instead of
  a QOM property for XIVE and a XICSFabric method for XICS-on-XIVE.
- even if the last two patches haven't changed much, the code base
  itself has, so I've deliberately dropped Cedric's R-b tags

--
Greg

---

Greg Kurz (4):
      linux-headers: Update
      spapr: Pass the maximum number of vCPUs to the KVM interrupt controller
      spapr/xics: Configure number of servers in KVM
      spapr/xive: Configure number of servers in KVM


 hw/intc/spapr_xive.c                         |    6 ++++--
 hw/intc/spapr_xive_kvm.c                     |   26 +++++++++++++++++++++++---
 hw/intc/xics_kvm.c                           |   24 +++++++++++++++++++++---
 hw/intc/xics_spapr.c                         |    5 +++--
 hw/ppc/spapr_irq.c                           |    8 +++++---
 include/hw/ppc/spapr_irq.h                   |   10 ++++++++--
 include/hw/ppc/spapr_xive.h                  |    3 ++-
 include/hw/ppc/xics_spapr.h                  |    3 ++-
 include/standard-headers/linux/ethtool.h     |    6 ++++++
 include/standard-headers/linux/virtio_ring.h |    2 +-
 linux-headers/asm-arm/kvm.h                  |    3 ++-
 linux-headers/asm-arm64/kvm.h                |    5 ++++-
 linux-headers/asm-mips/unistd_n32.h          |    1 +
 linux-headers/asm-mips/unistd_n64.h          |    1 +
 linux-headers/asm-mips/unistd_o32.h          |    1 +
 linux-headers/asm-powerpc/kvm.h              |    3 +++
 linux-headers/linux/kvm.h                    |   11 +++++++++++
 linux-headers/linux/psp-sev.h                |    3 +++
 18 files changed, 101 insertions(+), 20 deletions(-)

On Tue, Nov 26, 2019 at 05:46:12PM +0100, Greg Kurz wrote:
> On POWER9 systems, the XICS-on-XIVE and XIVE KVM devices currently
> allocate a bunch of VPs in the XIVE HW to accomodate the highest
> VCPU id that may be possibly used in a VM. This limits the number
> of VMs that can run with an in-kernel interrupt controller to 63
> per POWER9 chip, irrespectively of its number of HW threads, eg.
> up to 96 on a POWER9 Nimbus socket. This is an unfortunate waste
> of scarce HW resources since a typical VM doesn't need that much
> VPs to run.
> 
> This series exploits new attributes of the XICS-on-XIVE and XIVE
> KVM devices that allow userspace to tune the numbers of VPs it
> really needs. Support for this attributes has just reached linux
> mainline.
> 
> In combination with recent commit 29cb4187497d "spapr: Set VSMT to
> smp_threads by default" which makes VCPU ids contiguous, it is
> possible to start more than 200 VMs instead of 127 on an AC922
> system (two sockets, 144 HW threads).
> 
> Changes in v2:
> - add an nr_servers argument to the activate() handler instead of
>   a QOM property for XIVE and a XICSFabric method for XICS-on-XIVE.
> - even if the last two patches haven't changed much, the code base
>   itself has, so I've deliberately dropped Cedric's R-b tags

Applied to ppc-for-5.0, thanks.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson