[v3] Introduce and use NUMA distance metrics

[PATCH net-next V3 0/3] Introduce and use NUMA distance metrics

Posted by Tariq Toukan 3 years, 9 months ago

Hi,

Implement and expose CPU spread API based on the scheduler's
sched_numa_find_closest().  Use it in mlx5 and enic device drivers.  This
replaces the binary NUMA preference (local / remote) with an improved one
that minds the actual distances, so that remote NUMAs with short distance
are preferred over farther ones.

This has significant performance implications when using NUMA-aware
memory allocations, improving the throughput and CPU utilization.

Regards,
Tariq

v3:
- Introduce the logic as a common API instead of being mlx5 specific.
- Add implementation to enic device driver.
- Use non-atomic version of __cpumask_clear_cpu.

v2:
- Replace EXPORT_SYMBOL with EXPORT_SYMBOL_GPL, per Peter's comment.
- Separate the set_cpu operation into two functions, per Saeed's suggestion.
- Add Saeed's Acked-by signature.


Tariq Toukan (3):
  sched/topology: Add NUMA-based CPUs spread API
  net/mlx5e: Improve remote NUMA preferences used for the IRQ affinity
    hints
  enic: Use NUMA distances logic when setting affinity hints

 drivers/net/ethernet/cisco/enic/enic_main.c  | 10 +++-
 drivers/net/ethernet/mellanox/mlx5/core/eq.c |  5 +-
 include/linux/sched/topology.h               |  4 ++
 kernel/sched/topology.c                      | 49 ++++++++++++++++++++
 4 files changed, 64 insertions(+), 4 deletions(-)

-- 
2.21.0

Re: [PATCH net-next V3 0/3] Introduce and use NUMA distance metrics

Posted by Tariq Toukan 3 years, 8 months ago


On 7/19/2022 7:23 PM, Tariq Toukan wrote:
> Hi,
> 
> Implement and expose CPU spread API based on the scheduler's
> sched_numa_find_closest().  Use it in mlx5 and enic device drivers.  This
> replaces the binary NUMA preference (local / remote) with an improved one
> that minds the actual distances, so that remote NUMAs with short distance
> are preferred over farther ones.
> 
> This has significant performance implications when using NUMA-aware
> memory allocations, improving the throughput and CPU utilization.
> 
> Regards,
> Tariq
> 
> v3:
> - Introduce the logic as a common API instead of being mlx5 specific.
> - Add implementation to enic device driver.
> - Use non-atomic version of __cpumask_clear_cpu.
> 

Comments on V2 were addressed.
Please let me now of any other comments on this V3.