docs/system/ppc/pseries.rst | 226 ++++++++++++++++++++++++++++++++++++ 1 file changed, 226 insertions(+)
From: Leonardo Garcia <lagarcia@br.ibm.com>
The purpose of this document is to substitute the content currently
available in the QEMU wiki at [0]. This initial version does contain
some additional content as well. Whenever this documentation gets
upstream and is reflected in [1], the QEMU wiki will be edited to point
to this documentation, so that we only need to keep it updated in one
place.
0. https://wiki.qemu.org/Documentation/Platforms/POWER
1. https://qemu.readthedocs.io/en/latest/system/ppc/pseries.html
Signed-off-by: Leonardo Garcia <lagarcia@br.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
Changelog:
v2->v3:
- Updating LoPAR link.
- Minor clarifications in the text.
v1->v2:
- Addressing David Gibson and Cédric's comments. Thanks!
docs/system/ppc/pseries.rst | 226 ++++++++++++++++++++++++++++++++++++
1 file changed, 226 insertions(+)
diff --git a/docs/system/ppc/pseries.rst b/docs/system/ppc/pseries.rst
index 932d4dd17d..f072f4a935 100644
--- a/docs/system/ppc/pseries.rst
+++ b/docs/system/ppc/pseries.rst
@@ -1,12 +1,238 @@
pSeries family boards (``pseries``)
===================================
+The Power machine para-virtualized environment described by the `Linux on Power
+Architecture Reference document (LoPAR)
+<https://cdn.openpowerfoundation.org/wp-content/uploads/2020/07/LoPAR-20200812.pdf>`_
+is called pSeries. This environment is also known as sPAPR, System p guests, or
+simply Power Linux guests (although it is capable of running other operating
+systems, such as AIX).
+
+Even though pSeries is designed to behave as a guest environment, it is also
+capable of acting as a hypervisor OS, providing, on that role, nested
+virtualization capabilities.
+
Supported devices
-----------------
+ * Multi processor support for many Power processors generations: POWER7,
+ POWER7+, POWER8, POWER8NVL, POWER9, and Power10. Support for POWER5+ exists,
+ but its state is unknown.
+ * Interrupt Controller, XICS (POWER8) and XIVE (POWER9 and Power10)
+ * vPHB PCIe Host bridge.
+ * vscsi and vnet devices, compatible with the same devices available on a
+ PowerVM hypervisor with VIOS managing LPARs.
+ * Virtio based devices.
+ * PCIe device pass through.
+
Missing devices
---------------
+ * SPICE support.
Firmware
--------
+
+`SLOF <https://github.com/aik/SLOF>`_ (Slimline Open Firmware) is an
+implementation of the `IEEE 1275-1994, Standard for Boot (Initialization
+Configuration) Firmware: Core Requirements and Practices
+<https://standards.ieee.org/standard/1275-1994.html>`_.
+
+QEMU includes a prebuilt image of SLOF which is updated when a more recent
+version is required.
+
+Build directions
+----------------
+
+.. code-block:: bash
+
+ ./configure --target-list=ppc64-softmmu && make
+
+Running instructions
+--------------------
+
+Someone can select the pSeries machine type by running QEMU with the following
+options:
+
+.. code-block:: bash
+
+ qemu-system-ppc64 -M pseries <other QEMU arguments>
+
+sPAPR devices
+-------------
+
+The sPAPR specification defines a set of para-virtualized devices, which are
+also supported by the pSeries machine in QEMU and can be instantiated with the
+``-device`` option:
+
+* ``spapr-vlan`` : a virtual network interface.
+* ``spapr-vscsi`` : a virtual SCSI disk interface.
+* ``spapr-rng`` : a pseudo-device for passing random number generator data to the
+ guest (see the `H_RANDOM hypercall feature
+ <https://wiki.qemu.org/Features/HRandomHypercall>`_ for details).
+* ``spapr-vty``: a virtual teletype.
+* ``spapr-pci-host-bridge``: a PCI host bridge.
+* ``tpm-spapr``: a Trusted Platform Module (TPM).
+* ``spapr-tpm-proxy``: a TPM proxy.
+
+These are compatible with the devices historically available for use when
+running the IBM PowerVM hypervisor with LPARs.
+
+However, since these devices have originally been specified with another
+hypervisor and non-Linux guests in mind, you should use the virtio counterparts
+(virtio-net, virtio-blk/scsi and virtio-rng for instance) if possible instead,
+since they will most probably give you better performance with Linux guests in a
+QEMU environment.
+
+The pSeries machine in QEMU is always instantiated with the following devices:
+
+* A NVRAM device (``spapr-nvram``).
+* A virtual teletype (``spapr-vty``).
+* A PCI host bridge (``spapr-pci-host-bridge``).
+
+Hence, it is not needed to add them manually, unless you use the ``-nodefaults``
+command line option in QEMU.
+
+In the case of the default ``spapr-nvram`` device, if someone wants to make the
+contents of the NVRAM device persistent, they will need to specify a PFLASH
+device when starting QEMU, i.e. either use
+``-drive if=pflash,file=<filename>,format=raw`` to set the default PFLASH
+device, or specify one with an ID
+(``-drive if=none,file=<filename>,format=raw,id=pfid``) and pass that ID to the
+NVRAM device with ``-global spapr-nvram.drive=pfid``.
+
+sPAPR specification
+^^^^^^^^^^^^^^^^^^^
+
+The main source of documentation on the sPAPR standard is the `Linux on Power
+Architecture Reference document (LoPAR)
+<https://openpowerfoundation.org/wp-content/uploads/2020/07/LoPAR-20200611.pdf>`_.
+However, documentation specific to QEMU's implementation of the specification
+can also be found in QEMU documentation:
+
+.. toctree::
+ :maxdepth: 1
+
+ ../../specs/ppc-spapr-numa.rst
+ ../../specs/ppc-spapr-xive.rst
+
+Other documentation available in QEMU docs directory:
+
+* Hypervisor calls (a.k.a. hcalls) (``docs/specs/ppc-spapr-hcalls.txt``).
+* Hot plug (``/docs/specs/ppc-spapr-hotplug.txt``).
+* Hypervisor calls needed by the Ultravisor
+ (``/docs/specs/ppc-spapr-uv-hcalls.txt``).
+
+Switching between the KVM-PR and KVM-HV kernel module
+-----------------------------------------------------
+
+Currently, there are two implementations of KVM on Power, ``kvm_hv.ko`` and
+``kvm_pr.ko``.
+
+
+If a host supports both KVM modes, and both KVM kernel modules are loaded, it is
+possible to switch between the two modes with the ``kvm-type`` parameter:
+
+* Use ``qemu-system-ppc64 -M pseries,accel=kvm,kvm-type=PR`` to use the
+ ``kvm_pr.ko`` kernel module.
+* Use ``qemu-system-ppc64 -M pseries,accel=kvm,kvm-type=HV`` to use ``kvm_hv.ko``
+ instead.
+
+KVM-PR
+^^^^^^
+
+KVM-PR uses the so-called **PR**\ oblem state of the PPC CPUs to run the guests,
+i.e. the virtual machine is run in user mode and all privileged instructions
+trap and have to be emulated by the host. That means you can run KVM-PR inside
+a pSeries guest (or a PowerVM LPAR for that matter), and that is where it has
+originated, as historically (prior to POWER7) it was not possible to run Linux
+on hypervisor mode on a Power processor (this function was restricted to
+PowerVM, the IBM proprietary hypervisor).
+
+Because all privileged instructions are trapped, guests that use a lot of
+privileged instructions run quite slow with KVM-PR. On the other hand, because
+of that, this kernel module can run on pretty much every PPC hardware, and is
+able to emulate a lot of guests CPUs. This module can even be used to run other
+PowerPC guests like an emulated PowerMac.
+
+As KVM-PR can be run inside a pSeries guest, it can also provide nested
+virtualization capabilities (i.e. running a guest from within a guest).
+
+It is important to notice that, as KVM-HV provides a much better execution
+performance, maintenance work has been much more focused on it in the past
+years. Maintenance for KVM-PR has been minimal.
+
+In order to run KVM-PR guests with POWER9 processors, someone will need to start
+QEMU with ``kernel_irqchip=off`` command line option.
+
+KVM-HV
+^^^^^^
+
+KVM-HV uses the hypervisor mode of more recent Power processors, that allow
+access to the bare metal hardware directly. Although POWER7 had this capability,
+it was only starting with POWER8 that this was officially supported by IBM.
+
+Originally, KVM-HV was only available when running on a PowerNV platform (a.k.a.
+Power bare metal). Although it runs on a PowerNV platform, it can only be used
+to start pSeries guests. As the pSeries guest doesn't have access to the
+hypervisor mode of the Power CPU, it wasn't possible to run KVM-HV on a guest.
+This limitation has been lifted, and now it is possible to run KVM-HV inside
+pSeries guests as well, making nested virtualization possible with KVM-HV.
+
+As KVM-HV has access to privileged instructions, guests that use a lot of these
+can run much faster than with KVM-PR. On the other hand, the guest CPU has to be
+of the same type as the host CPU this way, e.g. it is not possible to specify an
+embedded PPC CPU for the guest with KVM-HV. However, there is at least the
+possibility to run the guest in a backward-compatibility mode of the previous
+CPUs generations, e.g. you can run a POWER7 guest on a POWER8 host by using
+``-cpu POWER8,compat=power7`` as parameter to QEMU.
+
+Modules support
+---------------
+
+As noticed in the sections above, each module can run in a different
+environment. The following table shows with which environment each module can
+run. As long as you are in a supported environment, you can run KVM-PR or KVM-HV
+nested. Combinations not shown in the table are not available.
+
++--------------+------------+------+-------------------+----------+--------+
+| Platform | Host type | Bits | Page table format | KVM-HV | KVM-PR |
++==============+============+======+===================+==========+========+
+| PowerNV | bare metal | 32 | hash | no | yes |
+| | | +-------------------+----------+--------+
+| | | | radix | N/A | N/A |
+| | +------+-------------------+----------+--------+
+| | | 64 | hash | yes | yes |
+| | | +-------------------+----------+--------+
+| | | | radix | yes | no |
++--------------+------------+------+-------------------+----------+--------+
+| pSeries [1]_ | PowerNV | 32 | hash | no | yes |
+| | | +-------------------+----------+--------+
+| | | | radix | N/A | N/A |
+| | +------+-------------------+----------+--------+
+| | | 64 | hash | no | yes |
+| | | +-------------------+----------+--------+
+| | | | radix | yes [2]_ | no |
+| +------------+------+-------------------+----------+--------+
+| | PowerVM | 32 | hash | no | yes |
+| | | +-------------------+----------+--------+
+| | | | radix | N/A | N/A |
+| | +------+-------------------+----------+--------+
+| | | 64 | hash | no | yes |
+| | | +-------------------+----------+--------+
+| | | | radix [3]_ | no | yes |
++--------------+------------+------+-------------------+----------+--------+
+
+.. [1] On POWER9 DD2.1 processors, the page table format on the host and guest
+ must be the same.
+
+.. [2] KVM-HV cannot run nested on POWER8 machines.
+
+.. [3] Introduced on Power10 machines.
+
+Maintainer contact information
+------------------------------
+
+Cédric Le Goater <clg@kaod.org>
+
+Daniel Henrique Barboza <danielhb413@gmail.com>
\ No newline at end of file
--
2.33.1
Argh! Disregard this patch. It fails to build. I'll send v4 fixing this. On 12/8/21 12:47, lagarcia@linux.ibm.com wrote: > From: Leonardo Garcia <lagarcia@br.ibm.com> > > The purpose of this document is to substitute the content currently > available in the QEMU wiki at [0]. This initial version does contain > some additional content as well. Whenever this documentation gets > upstream and is reflected in [1], the QEMU wiki will be edited to point > to this documentation, so that we only need to keep it updated in one > place. > > 0. https://wiki.qemu.org/Documentation/Platforms/POWER > 1. https://qemu.readthedocs.io/en/latest/system/ppc/pseries.html > > Signed-off-by: Leonardo Garcia <lagarcia@br.ibm.com> > Reviewed-by: David Gibson <david@gibson.dropbear.id.au> > --- > > Changelog: > v2->v3: > - Updating LoPAR link. > - Minor clarifications in the text. > > v1->v2: > - Addressing David Gibson and Cédric's comments. Thanks! > > docs/system/ppc/pseries.rst | 226 ++++++++++++++++++++++++++++++++++++ > 1 file changed, 226 insertions(+) > > diff --git a/docs/system/ppc/pseries.rst b/docs/system/ppc/pseries.rst > index 932d4dd17d..f072f4a935 100644 > --- a/docs/system/ppc/pseries.rst > +++ b/docs/system/ppc/pseries.rst > @@ -1,12 +1,238 @@ > pSeries family boards (``pseries``) > =================================== > > +The Power machine para-virtualized environment described by the `Linux on Power > +Architecture Reference document (LoPAR) > +<https://cdn.openpowerfoundation.org/wp-content/uploads/2020/07/LoPAR-20200812.pdf>`_ > +is called pSeries. This environment is also known as sPAPR, System p guests, or > +simply Power Linux guests (although it is capable of running other operating > +systems, such as AIX). > + > +Even though pSeries is designed to behave as a guest environment, it is also > +capable of acting as a hypervisor OS, providing, on that role, nested > +virtualization capabilities. > + > Supported devices > ----------------- > > + * Multi processor support for many Power processors generations: POWER7, > + POWER7+, POWER8, POWER8NVL, POWER9, and Power10. Support for POWER5+ exists, > + but its state is unknown. > + * Interrupt Controller, XICS (POWER8) and XIVE (POWER9 and Power10) > + * vPHB PCIe Host bridge. > + * vscsi and vnet devices, compatible with the same devices available on a > + PowerVM hypervisor with VIOS managing LPARs. > + * Virtio based devices. > + * PCIe device pass through. > + > Missing devices > --------------- > > + * SPICE support. > > Firmware > -------- > + > +`SLOF <https://github.com/aik/SLOF>`_ (Slimline Open Firmware) is an > +implementation of the `IEEE 1275-1994, Standard for Boot (Initialization > +Configuration) Firmware: Core Requirements and Practices > +<https://standards.ieee.org/standard/1275-1994.html>`_. > + > +QEMU includes a prebuilt image of SLOF which is updated when a more recent > +version is required. > + > +Build directions > +---------------- > + > +.. code-block:: bash > + > + ./configure --target-list=ppc64-softmmu && make > + > +Running instructions > +-------------------- > + > +Someone can select the pSeries machine type by running QEMU with the following > +options: > + > +.. code-block:: bash > + > + qemu-system-ppc64 -M pseries <other QEMU arguments> > + > +sPAPR devices > +------------- > + > +The sPAPR specification defines a set of para-virtualized devices, which are > +also supported by the pSeries machine in QEMU and can be instantiated with the > +``-device`` option: > + > +* ``spapr-vlan`` : a virtual network interface. > +* ``spapr-vscsi`` : a virtual SCSI disk interface. > +* ``spapr-rng`` : a pseudo-device for passing random number generator data to the > + guest (see the `H_RANDOM hypercall feature > + <https://wiki.qemu.org/Features/HRandomHypercall>`_ for details). > +* ``spapr-vty``: a virtual teletype. > +* ``spapr-pci-host-bridge``: a PCI host bridge. > +* ``tpm-spapr``: a Trusted Platform Module (TPM). > +* ``spapr-tpm-proxy``: a TPM proxy. > + > +These are compatible with the devices historically available for use when > +running the IBM PowerVM hypervisor with LPARs. > + > +However, since these devices have originally been specified with another > +hypervisor and non-Linux guests in mind, you should use the virtio counterparts > +(virtio-net, virtio-blk/scsi and virtio-rng for instance) if possible instead, > +since they will most probably give you better performance with Linux guests in a > +QEMU environment. > + > +The pSeries machine in QEMU is always instantiated with the following devices: > + > +* A NVRAM device (``spapr-nvram``). > +* A virtual teletype (``spapr-vty``). > +* A PCI host bridge (``spapr-pci-host-bridge``). > + > +Hence, it is not needed to add them manually, unless you use the ``-nodefaults`` > +command line option in QEMU. > + > +In the case of the default ``spapr-nvram`` device, if someone wants to make the > +contents of the NVRAM device persistent, they will need to specify a PFLASH > +device when starting QEMU, i.e. either use > +``-drive if=pflash,file=<filename>,format=raw`` to set the default PFLASH > +device, or specify one with an ID > +(``-drive if=none,file=<filename>,format=raw,id=pfid``) and pass that ID to the > +NVRAM device with ``-global spapr-nvram.drive=pfid``. > + > +sPAPR specification > +^^^^^^^^^^^^^^^^^^^ > + > +The main source of documentation on the sPAPR standard is the `Linux on Power > +Architecture Reference document (LoPAR) > +<https://openpowerfoundation.org/wp-content/uploads/2020/07/LoPAR-20200611.pdf>`_. > +However, documentation specific to QEMU's implementation of the specification > +can also be found in QEMU documentation: > + > +.. toctree:: > + :maxdepth: 1 > + > + ../../specs/ppc-spapr-numa.rst > + ../../specs/ppc-spapr-xive.rst > + > +Other documentation available in QEMU docs directory: > + > +* Hypervisor calls (a.k.a. hcalls) (``docs/specs/ppc-spapr-hcalls.txt``). > +* Hot plug (``/docs/specs/ppc-spapr-hotplug.txt``). > +* Hypervisor calls needed by the Ultravisor > + (``/docs/specs/ppc-spapr-uv-hcalls.txt``). > + > +Switching between the KVM-PR and KVM-HV kernel module > +----------------------------------------------------- > + > +Currently, there are two implementations of KVM on Power, ``kvm_hv.ko`` and > +``kvm_pr.ko``. > + > + > +If a host supports both KVM modes, and both KVM kernel modules are loaded, it is > +possible to switch between the two modes with the ``kvm-type`` parameter: > + > +* Use ``qemu-system-ppc64 -M pseries,accel=kvm,kvm-type=PR`` to use the > + ``kvm_pr.ko`` kernel module. > +* Use ``qemu-system-ppc64 -M pseries,accel=kvm,kvm-type=HV`` to use ``kvm_hv.ko`` > + instead. > + > +KVM-PR > +^^^^^^ > + > +KVM-PR uses the so-called **PR**\ oblem state of the PPC CPUs to run the guests, > +i.e. the virtual machine is run in user mode and all privileged instructions > +trap and have to be emulated by the host. That means you can run KVM-PR inside > +a pSeries guest (or a PowerVM LPAR for that matter), and that is where it has > +originated, as historically (prior to POWER7) it was not possible to run Linux > +on hypervisor mode on a Power processor (this function was restricted to > +PowerVM, the IBM proprietary hypervisor). > + > +Because all privileged instructions are trapped, guests that use a lot of > +privileged instructions run quite slow with KVM-PR. On the other hand, because > +of that, this kernel module can run on pretty much every PPC hardware, and is > +able to emulate a lot of guests CPUs. This module can even be used to run other > +PowerPC guests like an emulated PowerMac. > + > +As KVM-PR can be run inside a pSeries guest, it can also provide nested > +virtualization capabilities (i.e. running a guest from within a guest). > + > +It is important to notice that, as KVM-HV provides a much better execution > +performance, maintenance work has been much more focused on it in the past > +years. Maintenance for KVM-PR has been minimal. > + > +In order to run KVM-PR guests with POWER9 processors, someone will need to start > +QEMU with ``kernel_irqchip=off`` command line option. > + > +KVM-HV > +^^^^^^ > + > +KVM-HV uses the hypervisor mode of more recent Power processors, that allow > +access to the bare metal hardware directly. Although POWER7 had this capability, > +it was only starting with POWER8 that this was officially supported by IBM. > + > +Originally, KVM-HV was only available when running on a PowerNV platform (a.k.a. > +Power bare metal). Although it runs on a PowerNV platform, it can only be used > +to start pSeries guests. As the pSeries guest doesn't have access to the > +hypervisor mode of the Power CPU, it wasn't possible to run KVM-HV on a guest. > +This limitation has been lifted, and now it is possible to run KVM-HV inside > +pSeries guests as well, making nested virtualization possible with KVM-HV. > + > +As KVM-HV has access to privileged instructions, guests that use a lot of these > +can run much faster than with KVM-PR. On the other hand, the guest CPU has to be > +of the same type as the host CPU this way, e.g. it is not possible to specify an > +embedded PPC CPU for the guest with KVM-HV. However, there is at least the > +possibility to run the guest in a backward-compatibility mode of the previous > +CPUs generations, e.g. you can run a POWER7 guest on a POWER8 host by using > +``-cpu POWER8,compat=power7`` as parameter to QEMU. > + > +Modules support > +--------------- > + > +As noticed in the sections above, each module can run in a different > +environment. The following table shows with which environment each module can > +run. As long as you are in a supported environment, you can run KVM-PR or KVM-HV > +nested. Combinations not shown in the table are not available. > + > ++--------------+------------+------+-------------------+----------+--------+ > +| Platform | Host type | Bits | Page table format | KVM-HV | KVM-PR | > ++==============+============+======+===================+==========+========+ > +| PowerNV | bare metal | 32 | hash | no | yes | > +| | | +-------------------+----------+--------+ > +| | | | radix | N/A | N/A | > +| | +------+-------------------+----------+--------+ > +| | | 64 | hash | yes | yes | > +| | | +-------------------+----------+--------+ > +| | | | radix | yes | no | > ++--------------+------------+------+-------------------+----------+--------+ > +| pSeries [1]_ | PowerNV | 32 | hash | no | yes | > +| | | +-------------------+----------+--------+ > +| | | | radix | N/A | N/A | > +| | +------+-------------------+----------+--------+ > +| | | 64 | hash | no | yes | > +| | | +-------------------+----------+--------+ > +| | | | radix | yes [2]_ | no | > +| +------------+------+-------------------+----------+--------+ > +| | PowerVM | 32 | hash | no | yes | > +| | | +-------------------+----------+--------+ > +| | | | radix | N/A | N/A | > +| | +------+-------------------+----------+--------+ > +| | | 64 | hash | no | yes | > +| | | +-------------------+----------+--------+ > +| | | | radix [3]_ | no | yes | > ++--------------+------------+------+-------------------+----------+--------+ > + > +.. [1] On POWER9 DD2.1 processors, the page table format on the host and guest > + must be the same. > + > +.. [2] KVM-HV cannot run nested on POWER8 machines. > + > +.. [3] Introduced on Power10 machines. > + > +Maintainer contact information > +------------------------------ > + > +Cédric Le Goater <clg@kaod.org> > + > +Daniel Henrique Barboza <danielhb413@gmail.com> > \ No newline at end of file
© 2016 - 2024 Red Hat, Inc.