mm: hugetlb: avoid fallback for specific node allocation of 1G pages

[PATCH] mm: hugetlb: avoid fallback for specific node allocation of 1G pages

Posted by Luiz Capitulino 10 months, 1 week ago

When using the HugeTLB kernel command-line to allocate 1G pages from
a specific node, such as:

   default_hugepagesz=1G hugepages=1:1

If node 1 happens to not have enough memory for the requested number of
1G pages, the allocation falls back to other nodes. A quick way to
reproduce this is by creating a KVM guest with a memory-less node and
trying to allocate 1 1G page from it. Instead of failing, the allocation
will fallback to other nodes.

This defeats the purpose of node specific allocation. Also, specific
node allocation for 2M pages don't have this behavior: the allocation
will just fail for the pages it can't satisfy.

This issue happens because HugeTLB calls memblock_alloc_try_nid_raw()
for 1G boot-time allocation as this function falls back to other nodes
if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw()
instead, which ensures that the allocation will only be satisfied from
the specified node.

Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")

Signed-off-by: Luiz Capitulino <luizcap@redhat.com>
---
 mm/hugetlb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 65068671e460..163190e89ea1 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -3145,7 +3145,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid)
 
 	/* do node specific alloc */
 	if (nid != NUMA_NO_NODE) {
-		m = memblock_alloc_try_nid_raw(huge_page_size(h), huge_page_size(h),
+		m = memblock_alloc_exact_nid_raw(huge_page_size(h), huge_page_size(h),
 				0, MEMBLOCK_ALLOC_ACCESSIBLE, nid);
 		if (!m)
 			return 0;
-- 
2.48.1

Re: [PATCH] mm: hugetlb: avoid fallback for specific node allocation of 1G pages

Posted by David Hildenbrand 10 months, 1 week ago

On 11.02.25 04:48, Luiz Capitulino wrote:
> When using the HugeTLB kernel command-line to allocate 1G pages from
> a specific node, such as:
> 
>     default_hugepagesz=1G hugepages=1:1
> 
> If node 1 happens to not have enough memory for the requested number of
> 1G pages, the allocation falls back to other nodes. A quick way to
> reproduce this is by creating a KVM guest with a memory-less node and
> trying to allocate 1 1G page from it. Instead of failing, the allocation
> will fallback to other nodes.
> 
> This defeats the purpose of node specific allocation. Also, specific
> node allocation for 2M pages don't have this behavior: the allocation
> will just fail for the pages it can't satisfy.
> 
> This issue happens because HugeTLB calls memblock_alloc_try_nid_raw()
> for 1G boot-time allocation as this function falls back to other nodes
> if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw()
> instead, which ensures that the allocation will only be satisfied from
> the specified node.
> 
> Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")
> 
> Signed-off-by: Luiz Capitulino <luizcap@redhat.com>
> ---
>   mm/hugetlb.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 65068671e460..163190e89ea1 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -3145,7 +3145,7 @@ int __alloc_bootmem_huge_page(struct hstate *h, int nid)
>   
>   	/* do node specific alloc */
>   	if (nid != NUMA_NO_NODE) {
> -		m = memblock_alloc_try_nid_raw(huge_page_size(h), huge_page_size(h),
> +		m = memblock_alloc_exact_nid_raw(huge_page_size(h), huge_page_size(h),
>   				0, MEMBLOCK_ALLOC_ACCESSIBLE, nid);
>   		if (!m)
>   			return 0;

Yeah, documentation says "The node format specifies the number of huge 
pages to allocate on specific nodes."

Likely the patch simply copied the memblock_alloc_try_nid_raw() call; 
memblock_alloc_exact_nid_raw() seems to be the right thing to do

Acked-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb

Re: [PATCH] mm: hugetlb: avoid fallback for specific node allocation of 1G pages

Posted by Oscar Salvador 10 months, 1 week ago

On Mon, Feb 10, 2025 at 10:48:56PM -0500, Luiz Capitulino wrote:
> When using the HugeTLB kernel command-line to allocate 1G pages from
> a specific node, such as:
> 
>    default_hugepagesz=1G hugepages=1:1
> 
> If node 1 happens to not have enough memory for the requested number of
> 1G pages, the allocation falls back to other nodes. A quick way to
> reproduce this is by creating a KVM guest with a memory-less node and
> trying to allocate 1 1G page from it. Instead of failing, the allocation
> will fallback to other nodes.
> 
> This defeats the purpose of node specific allocation. Also, specific
> node allocation for 2M pages don't have this behavior: the allocation
> will just fail for the pages it can't satisfy.
> 
> This issue happens because HugeTLB calls memblock_alloc_try_nid_raw()
> for 1G boot-time allocation as this function falls back to other nodes
> if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw()
> instead, which ensures that the allocation will only be satisfied from
> the specified node.
> 
> Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")
> 
> Signed-off-by: Luiz Capitulino <luizcap@redhat.com>

Acked-by: Oscar Salvador <osalvador@suse.de>

This was discussed yesterday in [1], ccing Frank for awareness.

[1] https://patchwork.kernel.org/project/linux-mm/patch/20250206185109.1210657-6-fvdl@google.com/


-- 
Oscar Salvador
SUSE Labs

Re: [PATCH] mm: hugetlb: avoid fallback for specific node allocation of 1G pages

Posted by Luiz Capitulino 10 months, 1 week ago

On 2025-02-11 04:06, Oscar Salvador wrote:
> On Mon, Feb 10, 2025 at 10:48:56PM -0500, Luiz Capitulino wrote:
>> When using the HugeTLB kernel command-line to allocate 1G pages from
>> a specific node, such as:
>>
>>     default_hugepagesz=1G hugepages=1:1
>>
>> If node 1 happens to not have enough memory for the requested number of
>> 1G pages, the allocation falls back to other nodes. A quick way to
>> reproduce this is by creating a KVM guest with a memory-less node and
>> trying to allocate 1 1G page from it. Instead of failing, the allocation
>> will fallback to other nodes.
>>
>> This defeats the purpose of node specific allocation. Also, specific
>> node allocation for 2M pages don't have this behavior: the allocation
>> will just fail for the pages it can't satisfy.
>>
>> This issue happens because HugeTLB calls memblock_alloc_try_nid_raw()
>> for 1G boot-time allocation as this function falls back to other nodes
>> if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw()
>> instead, which ensures that the allocation will only be satisfied from
>> the specified node.
>>
>> Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")
>>
>> Signed-off-by: Luiz Capitulino <luizcap@redhat.com>
> 
> Acked-by: Oscar Salvador <osalvador@suse.de>
> 
> This was discussed yesterday in [1], ccing Frank for awareness.
> 
> [1] https://patchwork.kernel.org/project/linux-mm/patch/20250206185109.1210657-6-fvdl@google.com/

Interesting, thanks for the reference.

I stumbled over this issue back in December when debugging a HugeTLB issue
at Red Hat (David knows it ;) ) and had this patch pending for more than a
week now...

Re: [PATCH] mm: hugetlb: avoid fallback for specific node allocation of 1G pages

Posted by Frank van der Linden 10 months, 1 week ago

On Tue, Feb 11, 2025 at 6:51 AM Luiz Capitulino <luizcap@redhat.com> wrote:
>
> On 2025-02-11 04:06, Oscar Salvador wrote:
> > On Mon, Feb 10, 2025 at 10:48:56PM -0500, Luiz Capitulino wrote:
> >> When using the HugeTLB kernel command-line to allocate 1G pages from
> >> a specific node, such as:
> >>
> >>     default_hugepagesz=1G hugepages=1:1
> >>
> >> If node 1 happens to not have enough memory for the requested number of
> >> 1G pages, the allocation falls back to other nodes. A quick way to
> >> reproduce this is by creating a KVM guest with a memory-less node and
> >> trying to allocate 1 1G page from it. Instead of failing, the allocation
> >> will fallback to other nodes.
> >>
> >> This defeats the purpose of node specific allocation. Also, specific
> >> node allocation for 2M pages don't have this behavior: the allocation
> >> will just fail for the pages it can't satisfy.
> >>
> >> This issue happens because HugeTLB calls memblock_alloc_try_nid_raw()
> >> for 1G boot-time allocation as this function falls back to other nodes
> >> if the allocation can't be satisfied. Use memblock_alloc_exact_nid_raw()
> >> instead, which ensures that the allocation will only be satisfied from
> >> the specified node.
> >>
> >> Fixes: b5389086ad7b ("hugetlbfs: extend the definition of hugepages parameter to support node allocation")
> >>
> >> Signed-off-by: Luiz Capitulino <luizcap@redhat.com>
> >
> > Acked-by: Oscar Salvador <osalvador@suse.de>
> >
> > This was discussed yesterday in [1], ccing Frank for awareness.
> >
> > [1] https://patchwork.kernel.org/project/linux-mm/patch/20250206185109.1210657-6-fvdl@google.com/
>
> Interesting, thanks for the reference.
>
> I stumbled over this issue back in December when debugging a HugeTLB issue
> at Red Hat (David knows it ;) ) and had this patch pending for more than a
> week now...
>

Looks good, I'll drop the same change from my upcoming v4 series. This
will create a contextual dependency, but that's ok, this one will go
in first in any case.

Reviewed-by: Frank van der Linden <fvdl@google.com>