Hi Gregory,

Thank you for the v3 patch.

Gregory Price <gourry.memverge@gmail.com> write:

[snip]

> =====================================================================
> Performance tests - MLC
> From Ravi Jonnalagadda <ravis.opensrc@micron.com>
> 
> Workload:                               W2
> Data Signature:                         2:1 read:write
> DRAM only bandwidth (GBps):             298.8
> DRAM + CXL (default interleave) (GBps): 113.04
> DRAM + CXL (weighted interleave)(GBps): 412.5
> Gain over DRAM only:                    1.38x
> 
> Workload:                               W5
> Data Signature:                         1:1 read:write
> DRAM only bandwidth (GBps):             273.2
> DRAM + CXL (default interleave) (GBps): 117.23
> DRAM + CXL (weighted interleave)(GBps): 382.7
> Gain over DRAM only:                    1.4x

I've run XSBench based on the v3 patch and got numbers below. I used
your sample numactl extension from here:
Link: https://github.com/gmprice/numactl/tree/weighted_interleave_master

Performance tests – XSBench
NUMA node 0: 56 logical cores, 128 GB memory
NUMA node 2: 96 GB CXL memory

  1. dram only
  $ numactl -membind 0 ./XSBench -s XL –p 5000000
  Threads:     56
  Runtime:     36.235 seconds
  Lookups:     170,000,000
  Lookups/s:   4,691,618
 
  2. default interleave
  $ numactl –-interleave 0,2 ./XSBench –s XL –p 5000000
  Threads:     56
  Runtime:     55.243 seconds
  Lookups:     170,000,000
  Lookups/s:   3,077,293

  3. weighted interleave
  $ numactl --weighted --interleave 0,2 ./XSBench –s XL –p 5000000
  Threads:     56
  Runtime:     29.262 seconds
  Lookups:     170,000,000
  Lookups/s:   5,809,513

In terms of runtime, weighted-interleaving shows 1.19x improvement
compared to dram only, and 1.47x compared to default interleave.  I’ve
repeatedly run XSBench and have not observed any significant variations
across the runs.

Kind regards,
Hyeongtak

On Mon, Dec 18, 2023 at 04:07:48PM +0900, Hyeongtak Ji wrote:
> Hi Gregory,
> 
> Thank you for the v3 patch.
> 
> Gregory Price <gourry.memverge@gmail.com> write:
> 
> [snip]
> 

Hi Hyeongtak!

Thanks for the tests! Added test notes to the v4 cover.

> Performance tests – XSBench
> NUMA node 0: 56 logical cores, 128 GB memory
> NUMA node 2: 96 GB CXL memory
> 
>   1. dram only
>   $ numactl -membind 0 ./XSBench -s XL –p 5000000
>   Threads:     56
>   Runtime:     36.235 seconds
>   Lookups:     170,000,000
>   Lookups/s:   4,691,618
>  
>   2. default interleave
>   $ numactl –-interleave 0,2 ./XSBench –s XL –p 5000000
>   Threads:     56
>   Runtime:     55.243 seconds
>   Lookups:     170,000,000
>   Lookups/s:   3,077,293
> 
>   3. weighted interleave
>   $ numactl --weighted --interleave 0,2 ./XSBench –s XL –p 5000000
>   Threads:     56
>   Runtime:     29.262 seconds
>   Lookups:     170,000,000
>   Lookups/s:   5,809,513
>