mm: numa_memblks: Identify the accurate NUMA ID of CFMW

[PATCH] mm: numa_memblks: Identify the accurate NUMA ID of CFMW

Posted by Cui Chao 1 month, 1 week ago

In some physical memory layout designs, the address space of CFMW
resides between multiple segments of system memory belonging to
the same NUMA node. In numa_cleanup_meminfo, these multiple segments
of system memory are merged into a larger numa_memblk. When
identifying which NUMA node the CFMW belongs to, it may be incorrectly
assigned to the NUMA node of the merged system memory. To address this
scenario, accurately identifying the correct NUMA node can be achieved
by checking whether the region belongs to both numa_meminfo and
numa_reserved_meminfo.

Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn>
---
 mm/numa_memblks.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/numa_memblks.c b/mm/numa_memblks.c
index 5b009a9cd8b4..1ef037f0e0e0 100644
--- a/mm/numa_memblks.c
+++ b/mm/numa_memblks.c
@@ -573,7 +573,8 @@ int phys_to_target_node(u64 start)
 	 * Prefer online nodes, but if reserved memory might be
 	 * hot-added continue the search with reserved ranges.
 	 */
-	if (nid != NUMA_NO_NODE)
+	if (nid != NUMA_NO_NODE &&
+		meminfo_to_nid(&numa_reserved_meminfo, start) == NUMA_NO_NODE)
 		return nid;
 
 	return meminfo_to_nid(&numa_reserved_meminfo, start);
-- 
2.33.0

Re: [PATCH] mm: numa_memblks: Identify the accurate NUMA ID of CFMW

Posted by Mike Rapoport 1 month, 1 week ago

Hi,

On Tue, Dec 30, 2025 at 05:27:50PM +0800, Cui Chao wrote:
> In some physical memory layout designs, the address space of CFMW
> resides between multiple segments of system memory belonging to
> the same NUMA node. In numa_cleanup_meminfo, these multiple segments
> of system memory are merged into a larger numa_memblk. When
> identifying which NUMA node the CFMW belongs to, it may be incorrectly
> assigned to the NUMA node of the merged system memory. To address this

Can you please provide an example of such memory layout?

> scenario, accurately identifying the correct NUMA node can be achieved
> by checking whether the region belongs to both numa_meminfo and
> numa_reserved_meminfo.
> 
> Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn>
> ---
>  mm/numa_memblks.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/numa_memblks.c b/mm/numa_memblks.c
> index 5b009a9cd8b4..1ef037f0e0e0 100644
> --- a/mm/numa_memblks.c
> +++ b/mm/numa_memblks.c
> @@ -573,7 +573,8 @@ int phys_to_target_node(u64 start)
>  	 * Prefer online nodes, but if reserved memory might be
>  	 * hot-added continue the search with reserved ranges.
>  	 */
> -	if (nid != NUMA_NO_NODE)
> +	if (nid != NUMA_NO_NODE &&
> +		meminfo_to_nid(&numa_reserved_meminfo, start) == NUMA_NO_NODE)

I'd suggest assigning the result of meminfo_to_nid(&numa_reserved_meminfo,
start) to a local variable and using that in if and return statements.

>  		return nid;
>  
>  	return meminfo_to_nid(&numa_reserved_meminfo, start);
> -- 
> 2.33.0
> 
> 

-- 
Sincerely yours,
Mike.

Re: [PATCH] mm: numa_memblks: Identify the accurate NUMA ID of CFMW

Posted by Cui Chao 1 month ago

Hi,

Thank you for your review.

On 12/30/2025 11:18 PM, Mike Rapoport wrote:
> Hi,
>
> On Tue, Dec 30, 2025 at 05:27:50PM +0800, Cui Chao wrote:
>> In some physical memory layout designs, the address space of CFMW
>> resides between multiple segments of system memory belonging to
>> the same NUMA node. In numa_cleanup_meminfo, these multiple segments
>> of system memory are merged into a larger numa_memblk. When
>> identifying which NUMA node the CFMW belongs to, it may be incorrectly
>> assigned to the NUMA node of the merged system memory. To address this
> Can you please provide an example of such memory layout?


Example memory layout:

Physical address space:
     0x00000000 - 0x1FFFFFFF  System RAM (node0)
     0x20000000 - 0x2FFFFFFF  CXL CFMW (node2)
     0x40000000 - 0x5FFFFFFF  System RAM (node0)
     0x60000000 - 0x7FFFFFFF  System RAM (node1)

After numa_cleanup_meminfo, the two node0 segments are merged into one:
     0x00000000 - 0x5FFFFFFF  System RAM (node0)  // CFMW is inside this 
range
     0x60000000 - 0x7FFFFFFF  System RAM (node1)

So the CFMW (0x20000000-0x2FFFFFFF) will be incorrectly assigned to node0.


>> scenario, accurately identifying the correct NUMA node can be achieved
>> by checking whether the region belongs to both numa_meminfo and
>> numa_reserved_meminfo.
>>
>> Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn>
>> ---
>>   mm/numa_memblks.c | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/numa_memblks.c b/mm/numa_memblks.c
>> index 5b009a9cd8b4..1ef037f0e0e0 100644
>> --- a/mm/numa_memblks.c
>> +++ b/mm/numa_memblks.c
>> @@ -573,7 +573,8 @@ int phys_to_target_node(u64 start)
>>   	 * Prefer online nodes, but if reserved memory might be
>>   	 * hot-added continue the search with reserved ranges.
>>   	 */
>> -	if (nid != NUMA_NO_NODE)
>> +	if (nid != NUMA_NO_NODE &&
>> +		meminfo_to_nid(&numa_reserved_meminfo, start) == NUMA_NO_NODE)
> I'd suggest assigning the result of meminfo_to_nid(&numa_reserved_meminfo,
> start) to a local variable and using that in if and return statements.


I will use a local variable named reserved_nid.


>>   		return nid;
>>   
>>   	return meminfo_to_nid(&numa_reserved_meminfo, start);
>> -- 
>> 2.33.0
>>
>>
-- 
Best regards,
Cui Chao.

Re: [PATCH] mm: numa_memblks: Identify the accurate NUMA ID of CFMW

Posted by Mike Rapoport 1 month ago

On Mon, Jan 05, 2026 at 10:38:30AM +0800, Cui Chao wrote:
> Hi,
> 
> Thank you for your review.
> 
> On 12/30/2025 11:18 PM, Mike Rapoport wrote:
> > Hi,
> > 
> > On Tue, Dec 30, 2025 at 05:27:50PM +0800, Cui Chao wrote:
> > > In some physical memory layout designs, the address space of CFMW
> > > resides between multiple segments of system memory belonging to
> > > the same NUMA node. In numa_cleanup_meminfo, these multiple segments
> > > of system memory are merged into a larger numa_memblk. When
> > > identifying which NUMA node the CFMW belongs to, it may be incorrectly
> > > assigned to the NUMA node of the merged system memory. To address this
> > Can you please provide an example of such memory layout?
> 
> Example memory layout:
> 
> Physical address space:
>     0x00000000 - 0x1FFFFFFF  System RAM (node0)
>     0x20000000 - 0x2FFFFFFF  CXL CFMW (node2)
>     0x40000000 - 0x5FFFFFFF  System RAM (node0)
>     0x60000000 - 0x7FFFFFFF  System RAM (node1)
> 
> After numa_cleanup_meminfo, the two node0 segments are merged into one:
>     0x00000000 - 0x5FFFFFFF  System RAM (node0)  // CFMW is inside this
> range
>     0x60000000 - 0x7FFFFFFF  System RAM (node1)
> 
> So the CFMW (0x20000000-0x2FFFFFFF) will be incorrectly assigned to node0.

Can you please add this example to the changelog? 
 
> > > scenario, accurately identifying the correct NUMA node can be achieved
> > > by checking whether the region belongs to both numa_meminfo and
> > > numa_reserved_meminfo.
> > > 
> > > Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn>

-- 
Sincerely yours,
Mike.

Re: [PATCH] mm: numa_memblks: Identify the accurate NUMA ID of CFMW

Posted by Jonathan Cameron 1 month ago

On Mon, 5 Jan 2026 11:34:55 +0200
Mike Rapoport <rppt@kernel.org> wrote:

> On Mon, Jan 05, 2026 at 10:38:30AM +0800, Cui Chao wrote:
> > Hi,
> > 
> > Thank you for your review.
> > 
> > On 12/30/2025 11:18 PM, Mike Rapoport wrote:  
> > > Hi,
> > > 
> > > On Tue, Dec 30, 2025 at 05:27:50PM +0800, Cui Chao wrote:  
> > > > In some physical memory layout designs, the address space of CFMW
> > > > resides between multiple segments of system memory belonging to
> > > > the same NUMA node. In numa_cleanup_meminfo, these multiple segments
> > > > of system memory are merged into a larger numa_memblk. When
> > > > identifying which NUMA node the CFMW belongs to, it may be incorrectly
> > > > assigned to the NUMA node of the merged system memory. To address this  
> > > Can you please provide an example of such memory layout?  
> > 
> > Example memory layout:
> > 
> > Physical address space:
> >     0x00000000 - 0x1FFFFFFF  System RAM (node0)
> >     0x20000000 - 0x2FFFFFFF  CXL CFMW (node2)
> >     0x40000000 - 0x5FFFFFFF  System RAM (node0)
> >     0x60000000 - 0x7FFFFFFF  System RAM (node1)
> > 
> > After numa_cleanup_meminfo, the two node0 segments are merged into one:
> >     0x00000000 - 0x5FFFFFFF  System RAM (node0)  // CFMW is inside this
> > range
> >     0x60000000 - 0x7FFFFFFF  System RAM (node1)
> > 
> > So the CFMW (0x20000000-0x2FFFFFFF) will be incorrectly assigned to node0.  
> 
> Can you please add this example to the changelog? #
Also, please +CC linux-cxl@vger.kernel.org given this is all about how
we handle the CXL fixed memory windows.

>  
> > > > scenario, accurately identifying the correct NUMA node can be achieved
> > > > by checking whether the region belongs to both numa_meminfo and
> > > > numa_reserved_meminfo.
> > > > 
> > > > Signed-off-by: Cui Chao <cuichao1753@phytium.com.cn>  
>