Add some basic examples for the definition of cpu topology
in s390x.

Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
---
 docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++
 1 file changed, 88 insertions(+)
 create mode 100644 docs/system/s390x/cpu_topology.rst

diff --git a/docs/system/s390x/cpu_topology.rst b/docs/system/s390x/cpu_topology.rst
new file mode 100644
index 0000000000..00977d4319
--- /dev/null
+++ b/docs/system/s390x/cpu_topology.rst
@@ -0,0 +1,88 @@
+CPU Topology on s390x
+=====================
+
+CPU Topology on S390x provides up to 4 levels of topology containers:
+drawers, books, sockets and CPUs.
+While the three higher level containers, Containers Topology List Entries,
+(Containers TLE) define a tree hierarchy, the lowest level of topology
+definition, the CPU Topology List Entry (CPU TLE), provides the placement
+of the CPUs inside the last container.
+
+Prerequisites
+-------------
+
+To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility
+(STFLE bit 11) is required.
+
+However, since this facility has been enabled by default in an early version,
+the capability ``KVM_CAP_S390_CPU_TOPOLOGY`` is needed to indicate to KVM
+that QEMU support CPU Topology.
+
+Indicating the CPU topology to the Virtual Machine
+--------------------------------------------------
+
+The CPU Topology, number of drawers, number of books per drawers, number of
+sockets per book and number of cores per sockets is specified with the
+``-smp`` qemu command arguments.
+
+Like in :
+
+.. code-block:: sh
+    -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192
+
+If drawers or books are not specified, their default to 1.
+
+New CPUs can be plugged using the device_add hmp command like in:
+
+.. code-block:: sh
+   (qemu) device_add host-s390x-cpu,core-id=9
+
+The core-id defines the placement of the core in the topology by
+starting with core 0 in socket 0, book 0 and drawer 0 up to the maximum
+core number of the last socket of the last book in the last drawer.
+
+In the example above:
+
+* the core with ID 9 will be placed in container (0,0,1), as core 9
+  of CPU TLE 0 of socket 1 in book 0 from drawer 0.
+* the core ID 0 is defined by the -smp cpus=1 command and will be
+  placed as core 0 in CPU TLE 0 of container (0,0,0)
+
+Note that the core ID is machine wide and the CPU TLE masks provided
+by the STSI instruction will be:
+
+* in socket 0: 0x80000000 (core id 0)
+* in socket 1: 0x00400000 (core id 9)
+
+Indicating the CPU topology to the Guest
+----------------------------------------
+
+The guest can query for topology changes using the PTF instruction.
+In case of a topology change it can request the new topology by issuing
+STSI instructions specifying the level of detail required, drawer with
+STSI(15.1.4) or books STSI(15.1.3).
+
+The virtual machine will fill the provided buffer with the count of
+drawers (MAG4), books per drawer (MAG3), sockets per book (MAG2) and
+cores per socket (MAG1).
+
+Note that the STSI(15.1.2) is special in two ways:
+
+* When the firmware detect a change in the values calculated for STSI(15.1.2)
+  it will trigger the report of the topology change for the PTF instruction.
+
+Migration
+---------
+
+For virtio-ccw machines older than s390-virtio-ccw-7.2, CPU Topoogy is
+by default disabled.
+
+CPU Topoogy is by default enabled for s390-virtio-ccw-7.2 and newer machines.
+
+Enabling the CPU topology on older Machine is done by setting the global
+option ''topology-disable'' to false before enabling cpu topology with the
+cpu feature "ctop" like in:
+
+.. code-block:: sh
+   -machine s390-ccw-virtio-3.0,accel=kvm,topology-disable=false
+   -cpu z14,ctop=on
-- 
2.31.1

On Fri, 2022-09-02 at 09:55 +0200, Pierre Morel wrote:
> Add some basic examples for the definition of cpu topology
> in s390x.
> 
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
>  docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++
>  1 file changed, 88 insertions(+)
>  create mode 100644 docs/system/s390x/cpu_topology.rst
> 
> diff --git a/docs/system/s390x/cpu_topology.rst b/docs/system/s390x/cpu_topology.rst
> new file mode 100644
> index 0000000000..00977d4319
> --- /dev/null
> +++ b/docs/system/s390x/cpu_topology.rst
> @@ -0,0 +1,88 @@

[...]

> +Indicating the CPU topology to the Virtual Machine
> +--------------------------------------------------
> +
> +The CPU Topology, number of drawers, number of books per drawers, number of
> +sockets per book and number of cores per sockets is specified with the
> +``-smp`` qemu command arguments.
> +
> +Like in :
> +
> +.. code-block:: sh
> +    -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192
> +
> +If drawers or books are not specified, their default to 1.

Forgot this:
s/their default/they default/


[...]

On Fri, 2022-09-02 at 09:55 +0200, Pierre Morel wrote:
> Add some basic examples for the definition of cpu topology
> in s390x.
> 
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
>  docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++
>  1 file changed, 88 insertions(+)
>  create mode 100644 docs/system/s390x/cpu_topology.rst
> 
> diff --git a/docs/system/s390x/cpu_topology.rst b/docs/system/s390x/cpu_topology.rst
> new file mode 100644
> index 0000000000..00977d4319
> --- /dev/null
> +++ b/docs/system/s390x/cpu_topology.rst
> @@ -0,0 +1,88 @@
> +CPU Topology on s390x
> +=====================
> +
> +CPU Topology on S390x provides up to 4 levels of topology containers:
> +drawers, books, sockets and CPUs.
> +While the three higher level containers, Containers Topology List Entries,
> +(Containers TLE) define a tree hierarchy, the lowest level of topology
> +definition, the CPU Topology List Entry (CPU TLE), provides the placement
> +of the CPUs inside the last container.

inside the parent container
> +
> +Prerequisites
> +-------------
> +
> +To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility
> +(STFLE bit 11) is required.
> +
> +However, since this facility has been enabled by default in an early version,
> +the capability ``KVM_CAP_S390_CPU_TOPOLOGY`` is needed to indicate to KVM
> +that QEMU support CPU Topology.

I don't understand this paragraph. Early version of what?
> +
> +Indicating the CPU topology to the Virtual Machine
> +--------------------------------------------------
> +
> +The CPU Topology, number of drawers, number of books per drawers, number of
> +sockets per book and number of cores per sockets is specified with the
> +``-smp`` qemu command arguments.
> +
> +Like in :
> +
> +.. code-block:: sh
> +    -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192
> +
> +If drawers or books are not specified, their default to 1.
> +
> +New CPUs can be plugged using the device_add hmp command like in:
> +
> +.. code-block:: sh
> +   (qemu) device_add host-s390x-cpu,core-id=9
> +
> +The core-id defines the placement of the core in the topology by
> +starting with core 0 in socket 0, book 0 and drawer 0 up to the maximum
> +core number of the last socket of the last book in the last drawer.
> +
> +In the example above:
> +
> +* the core with ID 9 will be placed in container (0,0,1), as core 9
> +  of CPU TLE 0 of socket 1 in book 0 from drawer 0.
> +* the core ID 0 is defined by the -smp cpus=1 command and will be
> +  placed as core 0 in CPU TLE 0 of container (0,0,0)
> +
> +Note that the core ID is machine wide and the CPU TLE masks provided
> +by the STSI instruction will be:
> +
> +* in socket 0: 0x80000000 (core id 0)
> +* in socket 1: 0x00400000 (core id 9)
> +
> +Indicating the CPU topology to the Guest
> +----------------------------------------
> +
> +The guest can query for topology changes using the PTF instruction.
> +In case of a topology change it can request the new topology by issuing
> +STSI instructions specifying the level of detail required, drawer with
> +STSI(15.1.4) or books STSI(15.1.3).
> +
> +The virtual machine will fill the provided buffer with the count of
> +drawers (MAG4), books per drawer (MAG3), sockets per book (MAG2) and
> +cores per socket (MAG1).
> +
> +Note that the STSI(15.1.2) is special in two ways:
> +
> +* When the firmware detect a change in the values calculated for STSI(15.1.2)
> +  it will trigger the report of the topology change for the PTF instruction.

I don't know if we need this section, after all documenting this is the
job of the principles of operation. You could just refer to the
relevant sections.
> +
> +Migration
> +---------
> +
> +For virtio-ccw machines older than s390-virtio-ccw-7.2, CPU Topoogy is
> +by default disabled.
> +
> +CPU Topoogy is by default enabled for s390-virtio-ccw-7.2 and newer machines.
> +
> +Enabling the CPU topology on older Machine is done by setting the global
> +option ''topology-disable'' to false before enabling cpu topology with the
> +cpu feature "ctop" like in:
> +
> +.. code-block:: sh
> +   -machine s390-ccw-virtio-3.0,accel=kvm,topology-disable=false
> +   -cpu z14,ctop=on

On 9/12/22 15:41, Janis Schoetterl-Glausch wrote:
> On Fri, 2022-09-02 at 09:55 +0200, Pierre Morel wrote:
>> Add some basic examples for the definition of cpu topology
>> in s390x.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>>   docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++
>>   1 file changed, 88 insertions(+)
>>   create mode 100644 docs/system/s390x/cpu_topology.rst
>>
>> diff --git a/docs/system/s390x/cpu_topology.rst b/docs/system/s390x/cpu_topology.rst
>> new file mode 100644
>> index 0000000000..00977d4319
>> --- /dev/null
>> +++ b/docs/system/s390x/cpu_topology.rst
>> @@ -0,0 +1,88 @@
>> +CPU Topology on s390x
>> +=====================
>> +
>> +CPU Topology on S390x provides up to 4 levels of topology containers:
>> +drawers, books, sockets and CPUs.
>> +While the three higher level containers, Containers Topology List Entries,
>> +(Containers TLE) define a tree hierarchy, the lowest level of topology
>> +definition, the CPU Topology List Entry (CPU TLE), provides the placement
>> +of the CPUs inside the last container.
> 
> inside the parent container

OK

>> +
>> +Prerequisites
>> +-------------
>> +
>> +To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility
>> +(STFLE bit 11) is required.
>> +
>> +However, since this facility has been enabled by default in an early version,
>> +the capability ``KVM_CAP_S390_CPU_TOPOLOGY`` is needed to indicate to KVM
>> +that QEMU support CPU Topology.
> 
> I don't understand this paragraph. Early version of what?

of QEMU, I add this

>> +
>> +Indicating the CPU topology to the Virtual Machine
>> +--------------------------------------------------
>> +
>> +The CPU Topology, number of drawers, number of books per drawers, number of
>> +sockets per book and number of cores per sockets is specified with the
>> +``-smp`` qemu command arguments.
>> +
>> +Like in :
>> +
>> +.. code-block:: sh
>> +    -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192
>> +
>> +If drawers or books are not specified, their default to 1.
>> +
>> +New CPUs can be plugged using the device_add hmp command like in:
>> +
>> +.. code-block:: sh
>> +   (qemu) device_add host-s390x-cpu,core-id=9
>> +
>> +The core-id defines the placement of the core in the topology by
>> +starting with core 0 in socket 0, book 0 and drawer 0 up to the maximum
>> +core number of the last socket of the last book in the last drawer.
>> +
>> +In the example above:
>> +
>> +* the core with ID 9 will be placed in container (0,0,1), as core 9
>> +  of CPU TLE 0 of socket 1 in book 0 from drawer 0.
>> +* the core ID 0 is defined by the -smp cpus=1 command and will be
>> +  placed as core 0 in CPU TLE 0 of container (0,0,0)
>> +
>> +Note that the core ID is machine wide and the CPU TLE masks provided
>> +by the STSI instruction will be:
>> +
>> +* in socket 0: 0x80000000 (core id 0)
>> +* in socket 1: 0x00400000 (core id 9)
>> +
>> +Indicating the CPU topology to the Guest
>> +----------------------------------------
>> +
>> +The guest can query for topology changes using the PTF instruction.
>> +In case of a topology change it can request the new topology by issuing
>> +STSI instructions specifying the level of detail required, drawer with
>> +STSI(15.1.4) or books STSI(15.1.3).
>> +
>> +The virtual machine will fill the provided buffer with the count of
>> +drawers (MAG4), books per drawer (MAG3), sockets per book (MAG2) and
>> +cores per socket (MAG1).
>> +
>> +Note that the STSI(15.1.2) is special in two ways:
>> +
>> +* When the firmware detect a change in the values calculated for STSI(15.1.2)
>> +  it will trigger the report of the topology change for the PTF instruction.
> 
> I don't know if we need this section, after all documenting this is the
> job of the principles of operation. You could just refer to the
> relevant sections.

OK, I will make it shorter

Thanks,
Pierre


-- 
Pierre Morel
IBM Lab Boeblingen