[v2] pseries NUMA distance rework

[PATCH v2 0/7] pseries NUMA distance rework

Posted by Daniel Henrique Barboza 3 years, 8 months ago

Hi,

Following the reviews of the first version [1], specially this
reply from David [2], I decided to take a step back and refactor
all the code in hw/ppc/spapr* that operates with ibm,associativity,
ibm,associativity-reference-points and ibm,max-associativity-domains.

A new file named 'spapr_numa.c' was created to gather all the
associativity related code into helpers that write NUMA/associativity
related info to the FDT. These helpers are then used in other
spapr_* files. This allows us to change NUMA related code in a
single location, instead of searching every file to see where is
associativity being written and how, and all the soon to get
more complex logic can be contained in spapr_numa.c. I consider
the end result to be better than what I ended up doing in v1.

Unlike v1, there is no NUMA distance change being done in this series.
Later on, the hub of the new NUMA distance calculation will be
spapr_numa_associativity_init(), where we'll take into consideration
user input from numa_states, handle sizes to what the PAPR kernel
understands and establish assoaciativity domains between the NUMA nodes.


Changes from v1:
- all the patches that did guest visible changes were removed. They
will be re-submitted in a follow-up series after this one.
- patch 02 from v1 will be reworked and reposted in the follow-up
series as well.
- version 1 link:
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg03169.html



These patches were rebased using David's ppc-for-5.2 tree. Github
repo with the patches applied:

https://github.com/danielhb/qemu/tree/spapr_numa_v2


[1] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg03169.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg04661.html
 
Daniel Henrique Barboza (7):
  ppc: introducing spapr_numa.c NUMA code helper
  ppc/spapr_nvdimm: turn spapr_dt_nvdimm() static
  spapr: introduce SpaprMachineClass::numa_assoc_array
  spapr, spapr_numa: handle vcpu ibm,associativity
  spapr, spapr_numa: move lookup-arrays handling to spapr_numa.c
  spapr_numa: move NVLink2 associativity handling to spapr_numa.c
  spapr_hcall: h_home_node_associativity now reads numa_assoc_array

 hw/ppc/meson.build            |   3 +-
 hw/ppc/spapr.c                |  91 +++---------------
 hw/ppc/spapr_hcall.c          |  16 +++-
 hw/ppc/spapr_numa.c           | 172 ++++++++++++++++++++++++++++++++++
 hw/ppc/spapr_nvdimm.c         |  37 ++++----
 hw/ppc/spapr_pci.c            |   9 +-
 hw/ppc/spapr_pci_nvlink2.c    |  19 +---
 include/hw/ppc/spapr.h        |  13 ++-
 include/hw/ppc/spapr_numa.h   |  32 +++++++
 include/hw/ppc/spapr_nvdimm.h |   3 +-
 10 files changed, 268 insertions(+), 127 deletions(-)
 create mode 100644 hw/ppc/spapr_numa.c
 create mode 100644 include/hw/ppc/spapr_numa.h

-- 
2.26.2

Re: [PATCH v2 0/7] pseries NUMA distance rework

Posted by David Gibson 3 years, 8 months ago

On Tue, Sep 01, 2020 at 09:56:38AM -0300, Daniel Henrique Barboza wrote:
> Hi,
> 
> Following the reviews of the first version [1], specially this
> reply from David [2], I decided to take a step back and refactor
> all the code in hw/ppc/spapr* that operates with ibm,associativity,
> ibm,associativity-reference-points and ibm,max-associativity-domains.
> 
> A new file named 'spapr_numa.c' was created to gather all the
> associativity related code into helpers that write NUMA/associativity
> related info to the FDT. These helpers are then used in other
> spapr_* files. This allows us to change NUMA related code in a
> single location, instead of searching every file to see where is
> associativity being written and how, and all the soon to get
> more complex logic can be contained in spapr_numa.c. I consider
> the end result to be better than what I ended up doing in v1.
> 
> Unlike v1, there is no NUMA distance change being done in this series.
> Later on, the hub of the new NUMA distance calculation will be
> spapr_numa_associativity_init(), where we'll take into consideration
> user input from numa_states, handle sizes to what the PAPR kernel
> understands and establish assoaciativity domains between the NUMA
> nodes.

Patches 1..4 applied to ppc-for-5.2.  Patch 5 has some nits I've
commented on.

> 
> 
> Changes from v1:
> - all the patches that did guest visible changes were removed. They
> will be re-submitted in a follow-up series after this one.
> - patch 02 from v1 will be reworked and reposted in the follow-up
> series as well.
> - version 1 link:
> https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg03169.html
> 
> 
> 
> These patches were rebased using David's ppc-for-5.2 tree. Github
> repo with the patches applied:
> 
> https://github.com/danielhb/qemu/tree/spapr_numa_v2
> 
> 
> [1] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg03169.html
> [2] https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg04661.html
>  
> Daniel Henrique Barboza (7):
>   ppc: introducing spapr_numa.c NUMA code helper
>   ppc/spapr_nvdimm: turn spapr_dt_nvdimm() static
>   spapr: introduce SpaprMachineClass::numa_assoc_array
>   spapr, spapr_numa: handle vcpu ibm,associativity
>   spapr, spapr_numa: move lookup-arrays handling to spapr_numa.c
>   spapr_numa: move NVLink2 associativity handling to spapr_numa.c
>   spapr_hcall: h_home_node_associativity now reads numa_assoc_array
> 
>  hw/ppc/meson.build            |   3 +-
>  hw/ppc/spapr.c                |  91 +++---------------
>  hw/ppc/spapr_hcall.c          |  16 +++-
>  hw/ppc/spapr_numa.c           | 172 ++++++++++++++++++++++++++++++++++
>  hw/ppc/spapr_nvdimm.c         |  37 ++++----
>  hw/ppc/spapr_pci.c            |   9 +-
>  hw/ppc/spapr_pci_nvlink2.c    |  19 +---
>  include/hw/ppc/spapr.h        |  13 ++-
>  include/hw/ppc/spapr_numa.h   |  32 +++++++
>  include/hw/ppc/spapr_nvdimm.h |   3 +-
>  10 files changed, 268 insertions(+), 127 deletions(-)
>  create mode 100644 hw/ppc/spapr_numa.c
>  create mode 100644 include/hw/ppc/spapr_numa.h
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: [PATCH v2 0/7] pseries NUMA distance rework

Posted by David Gibson 3 years, 8 months ago

On Thu, Sep 03, 2020 at 11:35:39AM +1000, David Gibson wrote:
> On Tue, Sep 01, 2020 at 09:56:38AM -0300, Daniel Henrique Barboza wrote:
> > Hi,
> > 
> > Following the reviews of the first version [1], specially this
> > reply from David [2], I decided to take a step back and refactor
> > all the code in hw/ppc/spapr* that operates with ibm,associativity,
> > ibm,associativity-reference-points and ibm,max-associativity-domains.
> > 
> > A new file named 'spapr_numa.c' was created to gather all the
> > associativity related code into helpers that write NUMA/associativity
> > related info to the FDT. These helpers are then used in other
> > spapr_* files. This allows us to change NUMA related code in a
> > single location, instead of searching every file to see where is
> > associativity being written and how, and all the soon to get
> > more complex logic can be contained in spapr_numa.c. I consider
> > the end result to be better than what I ended up doing in v1.
> > 
> > Unlike v1, there is no NUMA distance change being done in this series.
> > Later on, the hub of the new NUMA distance calculation will be
> > spapr_numa_associativity_init(), where we'll take into consideration
> > user input from numa_states, handle sizes to what the PAPR kernel
> > understands and establish assoaciativity domains between the NUMA
> > nodes.
> 
> Patches 1..4 applied to ppc-for-5.2.  Patch 5 has some nits I've
> commented on.

Ah, sorry, I realised I missed something.  Patches 1..2 are still
applied, but patch 3 has a nit large enough to call for a respin.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson