Hi,
I am sorry, I had added a wrong address--to=xen-devel@lists.xenproject.org
in this series' recipient address by mistake. When you reply to this series,
please remember to remove it. I'm sorry for the inconvenience again!
Cheers,
Wei Chen
> -----Original Message-----
> From: Wei Chen <wei.chen@arm.com>
> Sent: 2022年4月18日 17:07
> To: --to=xen-devel@lists.xenproject.org; xen-devel@lists.xenproject.org
> Cc: nd <nd@arm.com>; Wei Chen <Wei.Chen@arm.com>; Stefano Stabellini
> <sstabellini@kernel.org>; Julien Grall <julien@xen.org>; Jan Beulich
> <jbeulich@suse.com>
> Subject: [PATCH v2 00/10] Device tree based NUMA support for Arm - Part#1
>
> (The Arm device tree based NUMA support patch set contains 35
> patches. In order to make stuff easier for reviewers, I split
> them into 3 parts:
> 1. Preparation. I have re-sorted the patch series. And moved
> independent patches to the head of the series.
> 2. Move generically usable code from x86 to common.
> 3. Add new code to support Arm.
>
> This series only contains the first part patches.
> The full patch list can be found in this cover letter.)
>
> Xen memory allocation and scheduler modules are NUMA aware.
> But actually, on x86 has implemented the architecture APIs
> to support NUMA. Arm was providing a set of fake architecture
> APIs to make it compatible with NUMA awared memory allocation
> and scheduler.
>
> Arm system was working well as a single node NUMA system with
> these fake APIs, because we didn't have multiple nodes NUMA
> system on Arm. But in recent years, more and more Arm devices
> support multiple nodes NUMA system.
>
> So now we have a new problem. When Xen is running on these Arm
> devices, Xen still treat them as single node SMP systems. The
> NUMA affinity capability of Xen memory allocation and scheduler
> becomes meaningless. Because they rely on input data that does
> not reflect real NUMA layout.
>
> Xen still think the access time for all of the memory is the
> same for all CPUs. However, Xen may allocate memory to a VM
> from different NUMA nodes with different access speeds. This
> difference can be amplified in workloads inside VM, causing
> performance instability and timeouts.
>
> So in this patch series, we implement a set of NUMA API to use
> device tree to describe the NUMA layout. We reuse most of the
> code of x86 NUMA to create and maintain the mapping between
> memory and CPU, create the matrix between any two NUMA nodes.
> Except ACPI and some x86 specified code, we have moved other
> code to common. In next stage, when we implement ACPI based
> NUMA for Arm64, we may move the ACPI NUMA code to common too,
> but in current stage, we keep it as x86 only.
>
> This patch serires has been tested and booted well on one
> Arm64 NUMA machine and one HPE x86 NUMA machine.
>
> ---
> Part1 v1->v2:
> 1. Move independent patches from later to early of this series.
> 2. Drop the copy of EFI stub.c from Arm. Share common codes of
> x86 EFI stub for Arm.
> 3. Use CONFIG_ARM_EFI to replace CONFIG_EFI and remove help text
> and make CONFIG_ARM_EFI invisible.
> 4. Use ASSERT to replace VIRTUAL_BUG_ON in phys_to_nid.
> 5. Move MAX_NUMNODES from xen/numa.h to asm/numa.h for x86.
> 6. Extend the description of Arm's workaround for reserve DMA
> allocations to avoid the same discussion every time for
> arch_have_default_dmazone.
> 7. Update commit messages.
>
> Wei Chen (10):
> xen/arm: Print a 64-bit number in hex from early uart
> xen/x86: move reusable EFI stub functions from x86 to common
> xen/arm: add CONFIG_ARM_EFI to stub EFI API
> xen/arm: Keep memory nodes in device tree when Xen boots from EFI
> xen/x86: Use ASSERT instead of VIRTUAL_BUG_ON for phys_to_nid
> xen: introduce an arch helper for default dma zone status
> xen: decouple NUMA from ACPI in Kconfig
> xen/arm: use !CONFIG_NUMA to keep fake NUMA API
> xen/x86: use paddr_t for addresses in NUMA node structure
> xen/x86: add detection of memory interleaves for different nodes
>
> PART2:
> xen/x86: introduce a helper to update memory hotplug end
> xen/x86: Use enumerations to indicate NUMA status
> xen/x86: move generically usable NUMA code from x86 to common
> xen/x86: use arch_get_memory_map to get information from E820 map
> xen/x86: move NUMA scan nodes codes from x86 to common
> xen/x86: rename bad_srat to numa_bad
>
> PART3:
> xen: rename acpi_scan_nodes to numa_scan_nodes
> xen: introduce a Kconfig option to configure NUMA nodes number
> xen/arm: use NR_MEM_BANKS to override default NR_NODE_MEMBLKS
> xen/arm: implement helpers to get and update NUMA status
> xen/arm: implement node distance helpers for Arm
> xen/arm: use arch_get_memory_map to memory bank from bootinfo
> xen/arm: build NUMA cpu_to_node map in dt_smp_init_cpus
> xen/arm: Add boot and secondary CPU to NUMA system
> xen/arm: stub mem_hotplug_update_boundary for Arm
> xen/arm: introduce a helper to parse device tree processor node
> xen/arm: introduce a helper to parse device tree memory node
> xen/arm: introduce a helper to parse device tree NUMA distance map
> xen/arm: unified entry to parse all NUMA data from device tree
> xen/arm: keep guest still be NUMA unware
> xen/arm: enable device tree based NUMA in system init
> xen/arm: implement a dummy 1:1 node to pxm mapping
> xen/arm: use CONFIG_NUMA to gate node_online_map in smpboot
> xen/arm: Provide Kconfig options for Arm to enable NUMA
> docs: update numa command line to support Arm
>
> xen/arch/arm/Kconfig | 5 ++
> xen/arch/arm/Makefile | 2 +-
> xen/arch/arm/arm64/head.S | 12 ++--
> xen/arch/arm/bootfdt.c | 8 ++-
> xen/arch/arm/efi/Makefile | 5 ++
> xen/arch/arm/efi/efi-boot.h | 25 ---------
> xen/arch/arm/include/asm/numa.h | 6 ++
> xen/arch/x86/Kconfig | 2 +-
> xen/arch/x86/efi/Makefile | 4 +-
> xen/arch/x86/efi/{stub.c => stub-x86.c} | 37 -------------
> xen/arch/x86/include/asm/config.h | 1 -
> xen/arch/x86/include/asm/numa.h | 15 ++---
> xen/arch/x86/numa.c | 30 +++++-----
> xen/arch/x86/srat.c | 74 +++++++++++++++++++------
> xen/common/Kconfig | 3 +
> xen/common/efi/stub.c | 38 +++++++++++++
> xen/common/page_alloc.c | 2 +-
> xen/drivers/acpi/Kconfig | 3 +-
> xen/drivers/acpi/Makefile | 2 +-
> xen/include/xen/numa.h | 2 +
> 20 files changed, 162 insertions(+), 114 deletions(-)
> rename xen/arch/x86/efi/{stub.c => stub-x86.c} (71%)
> create mode 100644 xen/common/efi/stub.c
>
> --
> 2.25.1