docs/specs/ppc-spapr-xive.rst | 112 ++++++++++++++++++++++++++++++++-- docs/specs/ppc-xive.rst | 9 +-- 2 files changed, 111 insertions(+), 10 deletions(-)
This includes various small updates and a better description of the
chosen interrupt mode resulting from the combination of the 'ic-mode'
machine option, the 'kernel_irqchip' option, guest support and KVM
support.
Signed-off-by: Cédric Le Goater <clg@kaod.org>
---
David, please check the tables below. Thanks
docs/specs/ppc-spapr-xive.rst | 112 ++++++++++++++++++++++++++++++++--
docs/specs/ppc-xive.rst | 9 +--
2 files changed, 111 insertions(+), 10 deletions(-)
diff --git a/docs/specs/ppc-spapr-xive.rst b/docs/specs/ppc-spapr-xive.rst
index 539ce7ca4e90..7a64c9d04951 100644
--- a/docs/specs/ppc-spapr-xive.rst
+++ b/docs/specs/ppc-spapr-xive.rst
@@ -34,19 +34,118 @@ CAS Negotiation
---------------
QEMU advertises the supported interrupt modes in the device tree
-property "ibm,arch-vec-5-platform-support" in byte 23 and the OS
-Selection for XIVE is indicated in the "ibm,architecture-vec-5"
+property ``ibm,arch-vec-5-platform-support`` in byte 23 and the OS
+Selection for XIVE is indicated in the ``ibm,architecture-vec-5``
property byte 23.
The interrupt modes supported by the machine depend on the CPU type
(POWER9 is required for XIVE) but also on the machine property
``ic-mode`` which can be set on the command line. It can take the
-following values: ``xics``, ``xive``, ``dual`` and currently ``xics``
-is the default but it may change in the future.
+following values: ``xics``, ``xive``, and ``dual`` which is the
+default mode. ``dual`` means that both modes XICS **and** XIVE are
+supported and if the guest OS supports XIVE, this mode will be
+selected.
The choosen interrupt mode is activated after a reconfiguration done
in a machine reset.
+KVM negotiation
+---------------
+
+When the guest starts under KVM, the capabilities of the host kernel
+and QEMU are also negotiated. Depending on the version of the host
+kernel, KVM will advertise the XIVE capability to QEMU or not.
+
+Nevertheless, the available interrupt modes in the machine should not
+depend on the XIVE KVM capability of the host. On older kernels
+without XIVE KVM support, QEMU will use the emulated XIVE device as a
+fallback and on newer kernels (>=5.2), the KVM XIVE device.
+
+As a final refinement, the user can also switch the use of the KVM
+device with the machine option ``kernel_irqchip``.
+
+
+XIVE support in KVM
+~~~~~~~~~~~~~~~~~~~
+
+For guest OSes supporting XIVE, the resulting interrupt modes on host
+kernels with XIVE KVM support are the following:
+
+============== ============= ============= ================
+ic-mode kernel_irqchip
+-------------- ----------------------------------------------
+/ allowed off on
+ (default)
+============== ============= ============= ================
+dual (default) XIVE KVM XIVE emul. XIVE KVM
+xive XIVE KVM XIVE emul. XIVE KVM
+xics XICS KVM XICS emul. XICS KVM
+============== ============= ============= ================
+
+For legacy guest OSes without XIVE support, the resulting interrupt
+modes are the following:
+
+============== ============= ============= ================
+ic-mode kernel_irqchip
+-------------- ----------------------------------------------
+/ allowed off on
+ (default)
+============== ============= ============= ================
+dual (default) XICS KVM XICS emul. XICS KVM
+xive QEMU error(3) QEMU error(3) QEMU error(3)
+xics XICS KVM XICS emul. XICS KVM
+============== ============= ============= ================
+
+(3) QEMU fails at CAS with ``Guest requested unavailable interrupt
+ mode (XICS), either don't set the ic-mode machine property or try
+ ic-mode=xics or ic-mode=dual``
+
+
+No XIVE support in KVM
+~~~~~~~~~~~~~~~~~~~~~~
+
+For guest OSes supporting XIVE, the resulting interrupt modes on host
+kernels without XIVE KVM support are the following:
+
+============== ============= ============= ================
+ic-mode kernel_irqchip
+-------------- ----------------------------------------------
+/ allowed off on
+ (default)
+============== ============= ============= ================
+dual (default) XIVE emul.(1) XIVE emul. QEMU error (2)
+xive XIVE emul.(1) XIVE emul. QEMU error (2)
+xics XICS KVM XICS emul. XICS KVM
+============== ============= ============= ================
+
+
+(1) QEMU warns with ``warning: kernel_irqchip requested but unavailable:
+ IRQ_XIVE capability must be present for KVM``
+(2) QEMU fails with ``kernel_irqchip requested but unavailable:
+ IRQ_XIVE capability must be present for KVM``
+
+
+For legacy guest OSes without XIVE support, the resulting interrupt
+modes are the following:
+
+============== ============= ============= ================
+ic-mode kernel_irqchip
+-------------- ----------------------------------------------
+/ allowed off on
+ (default)
+============== ============= ============= ================
+dual (default) QEMU error(4) XICS emul. QEMU error(4)
+xive QEMU error(3) QEMU error(3) QEMU error(3)
+xics XICS KVM XICS emul. XICS KVM
+============== ============= ============= ================
+
+(3) QEMU fails at CAS with ``Guest requested unavailable interrupt
+ mode (XICS), either don't set the ic-mode machine property or try
+ ic-mode=xics or ic-mode=dual``
+(4) QEMU/KVM incompatibility due to device destruction in reset. This
+ needs to be addressed more cleanly with an error.
+
+
XIVE Device tree properties
---------------------------
@@ -92,10 +191,11 @@ for both interrupt mode. The different ranges are defined as follow :
- ``0x0000 .. 0x0FFF`` 4K CPU IPIs (only used under XIVE)
- ``0x1000 .. 0x1000`` 1 EPOW
- ``0x1001 .. 0x1001`` 1 HOTPLUG
+- ``0x1002 .. 0x10FF`` unused
- ``0x1100 .. 0x11FF`` 256 VIO devices
-- ``0x1200 .. 0x127F`` 32 PHBs devices
+- ``0x1200 .. 0x127F`` 32x4 LSIs for PHB devices
- ``0x1280 .. 0x12FF`` unused
-- ``0x1300 .. 0x1FFF`` PHB MSIs
+- ``0x1300 .. 0x1FFF`` PHB MSIs (dynamically allocated)
Monitoring XIVE
---------------
diff --git a/docs/specs/ppc-xive.rst b/docs/specs/ppc-xive.rst
index b997dc062910..148d57eb6ab2 100644
--- a/docs/specs/ppc-xive.rst
+++ b/docs/specs/ppc-xive.rst
@@ -20,10 +20,11 @@ The XIVE IC is composed of three sub-engines, each taking care of a
processing layer of external interrupts:
- Interrupt Virtualization Source Engine (IVSE), or Source Controller
- (SC). These are found in PCI PHBs, in the PSI host bridge
- controller, but also inside the main controller for the core IPIs
- and other sub-chips (NX, CAP, NPU) of the chip/processor. They are
- configured to feed the IVRE with events.
+ (SC). These are found in PCI PHBs, in the Processor Service
+ Interface (PSI) host bridge Controller, but also inside the main
+ controller for the core IPIs and other sub-chips (NX, CAP, NPU) of
+ the chip/processor. They are configured to feed the IVRE with
+ events.
- Interrupt Virtualization Routing Engine (IVRE) or Virtualization
Controller (VC). It handles event coalescing and perform interrupt
routing by matching an event source number with an Event
--
2.21.0
On Wed, Jun 12, 2019 at 06:04:25PM +0200, Cédric Le Goater wrote: > This includes various small updates and a better description of the > chosen interrupt mode resulting from the combination of the 'ic-mode' > machine option, the 'kernel_irqchip' option, guest support and KVM > support. > > Signed-off-by: Cédric Le Goater <clg@kaod.org> > --- > > David, please check the tables below. Thanks LGTM, applied. > > docs/specs/ppc-spapr-xive.rst | 112 ++++++++++++++++++++++++++++++++-- > docs/specs/ppc-xive.rst | 9 +-- > 2 files changed, 111 insertions(+), 10 deletions(-) > > diff --git a/docs/specs/ppc-spapr-xive.rst b/docs/specs/ppc-spapr-xive.rst > index 539ce7ca4e90..7a64c9d04951 100644 > --- a/docs/specs/ppc-spapr-xive.rst > +++ b/docs/specs/ppc-spapr-xive.rst > @@ -34,19 +34,118 @@ CAS Negotiation > --------------- > > QEMU advertises the supported interrupt modes in the device tree > -property "ibm,arch-vec-5-platform-support" in byte 23 and the OS > -Selection for XIVE is indicated in the "ibm,architecture-vec-5" > +property ``ibm,arch-vec-5-platform-support`` in byte 23 and the OS > +Selection for XIVE is indicated in the ``ibm,architecture-vec-5`` > property byte 23. > > The interrupt modes supported by the machine depend on the CPU type > (POWER9 is required for XIVE) but also on the machine property > ``ic-mode`` which can be set on the command line. It can take the > -following values: ``xics``, ``xive``, ``dual`` and currently ``xics`` > -is the default but it may change in the future. > +following values: ``xics``, ``xive``, and ``dual`` which is the > +default mode. ``dual`` means that both modes XICS **and** XIVE are > +supported and if the guest OS supports XIVE, this mode will be > +selected. > > The choosen interrupt mode is activated after a reconfiguration done > in a machine reset. > > +KVM negotiation > +--------------- > + > +When the guest starts under KVM, the capabilities of the host kernel > +and QEMU are also negotiated. Depending on the version of the host > +kernel, KVM will advertise the XIVE capability to QEMU or not. > + > +Nevertheless, the available interrupt modes in the machine should not > +depend on the XIVE KVM capability of the host. On older kernels > +without XIVE KVM support, QEMU will use the emulated XIVE device as a > +fallback and on newer kernels (>=5.2), the KVM XIVE device. > + > +As a final refinement, the user can also switch the use of the KVM > +device with the machine option ``kernel_irqchip``. > + > + > +XIVE support in KVM > +~~~~~~~~~~~~~~~~~~~ > + > +For guest OSes supporting XIVE, the resulting interrupt modes on host > +kernels with XIVE KVM support are the following: > + > +============== ============= ============= ================ > +ic-mode kernel_irqchip > +-------------- ---------------------------------------------- > +/ allowed off on > + (default) > +============== ============= ============= ================ > +dual (default) XIVE KVM XIVE emul. XIVE KVM > +xive XIVE KVM XIVE emul. XIVE KVM > +xics XICS KVM XICS emul. XICS KVM > +============== ============= ============= ================ > + > +For legacy guest OSes without XIVE support, the resulting interrupt > +modes are the following: > + > +============== ============= ============= ================ > +ic-mode kernel_irqchip > +-------------- ---------------------------------------------- > +/ allowed off on > + (default) > +============== ============= ============= ================ > +dual (default) XICS KVM XICS emul. XICS KVM > +xive QEMU error(3) QEMU error(3) QEMU error(3) > +xics XICS KVM XICS emul. XICS KVM > +============== ============= ============= ================ > + > +(3) QEMU fails at CAS with ``Guest requested unavailable interrupt > + mode (XICS), either don't set the ic-mode machine property or try > + ic-mode=xics or ic-mode=dual`` > + > + > +No XIVE support in KVM > +~~~~~~~~~~~~~~~~~~~~~~ > + > +For guest OSes supporting XIVE, the resulting interrupt modes on host > +kernels without XIVE KVM support are the following: > + > +============== ============= ============= ================ > +ic-mode kernel_irqchip > +-------------- ---------------------------------------------- > +/ allowed off on > + (default) > +============== ============= ============= ================ > +dual (default) XIVE emul.(1) XIVE emul. QEMU error (2) > +xive XIVE emul.(1) XIVE emul. QEMU error (2) > +xics XICS KVM XICS emul. XICS KVM > +============== ============= ============= ================ > + > + > +(1) QEMU warns with ``warning: kernel_irqchip requested but unavailable: > + IRQ_XIVE capability must be present for KVM`` > +(2) QEMU fails with ``kernel_irqchip requested but unavailable: > + IRQ_XIVE capability must be present for KVM`` > + > + > +For legacy guest OSes without XIVE support, the resulting interrupt > +modes are the following: > + > +============== ============= ============= ================ > +ic-mode kernel_irqchip > +-------------- ---------------------------------------------- > +/ allowed off on > + (default) > +============== ============= ============= ================ > +dual (default) QEMU error(4) XICS emul. QEMU error(4) > +xive QEMU error(3) QEMU error(3) QEMU error(3) > +xics XICS KVM XICS emul. XICS KVM > +============== ============= ============= ================ > + > +(3) QEMU fails at CAS with ``Guest requested unavailable interrupt > + mode (XICS), either don't set the ic-mode machine property or try > + ic-mode=xics or ic-mode=dual`` > +(4) QEMU/KVM incompatibility due to device destruction in reset. This > + needs to be addressed more cleanly with an error. > + > + > XIVE Device tree properties > --------------------------- > > @@ -92,10 +191,11 @@ for both interrupt mode. The different ranges are defined as follow : > - ``0x0000 .. 0x0FFF`` 4K CPU IPIs (only used under XIVE) > - ``0x1000 .. 0x1000`` 1 EPOW > - ``0x1001 .. 0x1001`` 1 HOTPLUG > +- ``0x1002 .. 0x10FF`` unused > - ``0x1100 .. 0x11FF`` 256 VIO devices > -- ``0x1200 .. 0x127F`` 32 PHBs devices > +- ``0x1200 .. 0x127F`` 32x4 LSIs for PHB devices > - ``0x1280 .. 0x12FF`` unused > -- ``0x1300 .. 0x1FFF`` PHB MSIs > +- ``0x1300 .. 0x1FFF`` PHB MSIs (dynamically allocated) > > Monitoring XIVE > --------------- > diff --git a/docs/specs/ppc-xive.rst b/docs/specs/ppc-xive.rst > index b997dc062910..148d57eb6ab2 100644 > --- a/docs/specs/ppc-xive.rst > +++ b/docs/specs/ppc-xive.rst > @@ -20,10 +20,11 @@ The XIVE IC is composed of three sub-engines, each taking care of a > processing layer of external interrupts: > > - Interrupt Virtualization Source Engine (IVSE), or Source Controller > - (SC). These are found in PCI PHBs, in the PSI host bridge > - controller, but also inside the main controller for the core IPIs > - and other sub-chips (NX, CAP, NPU) of the chip/processor. They are > - configured to feed the IVRE with events. > + (SC). These are found in PCI PHBs, in the Processor Service > + Interface (PSI) host bridge Controller, but also inside the main > + controller for the core IPIs and other sub-chips (NX, CAP, NPU) of > + the chip/processor. They are configured to feed the IVRE with > + events. > - Interrupt Virtualization Routing Engine (IVRE) or Virtualization > Controller (VC). It handles event coalescing and perform interrupt > routing by matching an event source number with an Event -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
© 2016 - 2024 Red Hat, Inc.