mm/page_alloc: optimize the loop in find_suitable_fallback()

[PATCH] mm/page_alloc: optimize the loop in find_suitable_fallback()

Posted by Yajun Deng 2 years, 7 months ago

There is no need to execute the next loop if it not return in the first
loop. So add a break at the end of the loop.

There are only three rows in fallbacks, so reduce the first index size
from MIGRATE_TYPES to MIGRATE_PCPTYPES.

Signed-off-by: Yajun Deng <yajun.deng@linux.dev>
---
 mm/page_alloc.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 1113483fa6c5..536e8d838fb5 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2603,7 +2603,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
  *
  * The other migratetypes do not have fallbacks.
  */
-static int fallbacks[MIGRATE_TYPES][MIGRATE_PCPTYPES - 1] = {
+static int fallbacks[MIGRATE_PCPTYPES][MIGRATE_PCPTYPES - 1] = {
 	[MIGRATE_UNMOVABLE]   = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE   },
 	[MIGRATE_MOVABLE]     = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE },
 	[MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE,   MIGRATE_MOVABLE   },
@@ -2861,7 +2861,7 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
 	int i;
 	int fallback_mt;
 
-	if (area->nr_free == 0)
+	if (area->nr_free == 0 || !migratetype_is_mergeable(migratetype))
 		return -1;
 
 	*can_steal = false;
@@ -2873,11 +2873,10 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
 		if (can_steal_fallback(order, migratetype))
 			*can_steal = true;
 
-		if (!only_stealable)
-			return fallback_mt;
-
-		if (*can_steal)
+		if (!only_stealable || *can_steal)
 			return fallback_mt;
+		else
+			break;
 	}
 
 	return -1;
-- 
2.25.1

Re: [PATCH] mm/page_alloc: optimize the loop in find_suitable_fallback()

Posted by Vlastimil Babka 2 years, 7 months ago

On 2/9/23 03:44, Yajun Deng wrote:
> There is no need to execute the next loop if it not return in the first
> loop. So add a break at the end of the loop.
> 
> There are only three rows in fallbacks, so reduce the first index size
> from MIGRATE_TYPES to MIGRATE_PCPTYPES.
> 
> Signed-off-by: Yajun Deng <yajun.deng@linux.dev>

Acked-by: Vlastimil Babka <vbabka@suse.cz>

> ---
>  mm/page_alloc.c | 11 +++++------
>  1 file changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 1113483fa6c5..536e8d838fb5 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2603,7 +2603,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
>   *
>   * The other migratetypes do not have fallbacks.
>   */
> -static int fallbacks[MIGRATE_TYPES][MIGRATE_PCPTYPES - 1] = {
> +static int fallbacks[MIGRATE_PCPTYPES][MIGRATE_PCPTYPES - 1] = {
>  	[MIGRATE_UNMOVABLE]   = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE   },
>  	[MIGRATE_MOVABLE]     = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE },
>  	[MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE,   MIGRATE_MOVABLE   },
> @@ -2861,7 +2861,7 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>  	int i;
>  	int fallback_mt;
>  
> -	if (area->nr_free == 0)
> +	if (area->nr_free == 0 || !migratetype_is_mergeable(migratetype))

Just curious, did you the check for extra safety or did you find (by running
or code inspection) that this can be indeed called with a non-mergeable
migratetype, and cause out of bounds access of the shrinked fallbacks array?

BTW, I noticed the commment on migratetype_is_mergeable() contains:

"See fallbacks[MIGRATE_TYPES][3] in page_alloc.c. "

Should probably change it to e.g. "See fallbacks[][] array ..." so we don't
have to keep it in exact sync...

>  		return -1;
>  
>  	*can_steal = false;
> @@ -2873,11 +2873,10 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>  		if (can_steal_fallback(order, migratetype))
>  			*can_steal = true;
>  
> -		if (!only_stealable)
> -			return fallback_mt;
> -
> -		if (*can_steal)
> +		if (!only_stealable || *can_steal)
>  			return fallback_mt;
> +		else
> +			break;
>  	}
>  
>  	return -1;

Re: [PATCH] mm/page_alloc: optimize the loop in find_suitable_fallback()

Posted by Yajun Deng 2 years, 7 months ago

February 9, 2023 4:12 PM, "Vlastimil Babka" <vbabka@suse.cz> wrote:

> On 2/9/23 03:44, Yajun Deng wrote:
> 
>> There is no need to execute the next loop if it not return in the first
>> loop. So add a break at the end of the loop.
>> 
>> There are only three rows in fallbacks, so reduce the first index size
>> from MIGRATE_TYPES to MIGRATE_PCPTYPES.
>> 
>> Signed-off-by: Yajun Deng <yajun.deng@linux.dev>
> 
> Acked-by: Vlastimil Babka <vbabka@suse.cz>
> 
>> ---
>> mm/page_alloc.c | 11 +++++------
>> 1 file changed, 5 insertions(+), 6 deletions(-)
>> 
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 1113483fa6c5..536e8d838fb5 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -2603,7 +2603,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
>> *
>> * The other migratetypes do not have fallbacks.
>> */
>> -static int fallbacks[MIGRATE_TYPES][MIGRATE_PCPTYPES - 1] = {
>> +static int fallbacks[MIGRATE_PCPTYPES][MIGRATE_PCPTYPES - 1] = {
>> [MIGRATE_UNMOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE },
>> [MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE },
>> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE },
>> @@ -2861,7 +2861,7 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>> int i;
>> int fallback_mt;
>> 
>> - if (area->nr_free == 0)
>> + if (area->nr_free == 0 || !migratetype_is_mergeable(migratetype))
> 
> Just curious, did you the check for extra safety or did you find (by running
> or code inspection) that this can be indeed called with a non-mergeable
> migratetype, and cause out of bounds access of the shrinked fallbacks array?
> 

No, I'm not sure if it is called with a non-mergeable migratetype.
It is just for extra safety.

> BTW, I noticed the commment on migratetype_is_mergeable() contains:
> 
> "See fallbacks[MIGRATE_TYPES][3] in page_alloc.c. "
> 
> Should probably change it to e.g. "See fallbacks[][] array ..." so we don't
> have to keep it in exact sync...
> 

Yes, this comment should be changed.
So do I need to submit a v2 patch?

>> return -1;
>> 
>> *can_steal = false;
>> @@ -2873,11 +2873,10 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>> if (can_steal_fallback(order, migratetype))
>> *can_steal = true;
>> 
>> - if (!only_stealable)
>> - return fallback_mt;
>> -
>> - if (*can_steal)
>> + if (!only_stealable || *can_steal)
>> return fallback_mt;
>> + else
>> + break;
>> }
>> 
>> return -1;

Re: [PATCH] mm/page_alloc: optimize the loop in find_suitable_fallback()

Posted by Vlastimil Babka 2 years, 7 months ago

On 2/9/23 09:44, Yajun Deng wrote:
> February 9, 2023 4:12 PM, "Vlastimil Babka" <vbabka@suse.cz> wrote:
> 
>> On 2/9/23 03:44, Yajun Deng wrote:
>> 
>>> There is no need to execute the next loop if it not return in the first
>>> loop. So add a break at the end of the loop.
>>> 
>>> There are only three rows in fallbacks, so reduce the first index size
>>> from MIGRATE_TYPES to MIGRATE_PCPTYPES.
>>> 
>>> Signed-off-by: Yajun Deng <yajun.deng@linux.dev>
>> 
>> Acked-by: Vlastimil Babka <vbabka@suse.cz>
>> 
>>> ---
>>> mm/page_alloc.c | 11 +++++------
>>> 1 file changed, 5 insertions(+), 6 deletions(-)
>>> 
>>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>>> index 1113483fa6c5..536e8d838fb5 100644
>>> --- a/mm/page_alloc.c
>>> +++ b/mm/page_alloc.c
>>> @@ -2603,7 +2603,7 @@ struct page *__rmqueue_smallest(struct zone *zone, unsigned int order,
>>> *
>>> * The other migratetypes do not have fallbacks.
>>> */
>>> -static int fallbacks[MIGRATE_TYPES][MIGRATE_PCPTYPES - 1] = {
>>> +static int fallbacks[MIGRATE_PCPTYPES][MIGRATE_PCPTYPES - 1] = {
>>> [MIGRATE_UNMOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE },
>>> [MIGRATE_MOVABLE] = { MIGRATE_RECLAIMABLE, MIGRATE_UNMOVABLE },
>>> [MIGRATE_RECLAIMABLE] = { MIGRATE_UNMOVABLE, MIGRATE_MOVABLE },
>>> @@ -2861,7 +2861,7 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>>> int i;
>>> int fallback_mt;
>>> 
>>> - if (area->nr_free == 0)
>>> + if (area->nr_free == 0 || !migratetype_is_mergeable(migratetype))
>> 
>> Just curious, did you the check for extra safety or did you find (by running
>> or code inspection) that this can be indeed called with a non-mergeable
>> migratetype, and cause out of bounds access of the shrinked fallbacks array?
>> 
> 
> No, I'm not sure if it is called with a non-mergeable migratetype.
> It is just for extra safety.

OK, I agree with that.

>> BTW, I noticed the commment on migratetype_is_mergeable() contains:
>> 
>> "See fallbacks[MIGRATE_TYPES][3] in page_alloc.c. "
>> 
>> Should probably change it to e.g. "See fallbacks[][] array ..." so we don't
>> have to keep it in exact sync...
>> 
> 
> Yes, this comment should be changed.
> So do I need to submit a v2 patch?

Please do, with my acked-by.

>>> return -1;
>>> 
>>> *can_steal = false;
>>> @@ -2873,11 +2873,10 @@ int find_suitable_fallback(struct free_area *area, unsigned int order,
>>> if (can_steal_fallback(order, migratetype))
>>> *can_steal = true;
>>> 
>>> - if (!only_stealable)
>>> - return fallback_mt;
>>> -
>>> - if (*can_steal)
>>> + if (!only_stealable || *can_steal)
>>> return fallback_mt;
>>> + else
>>> + break;
>>> }
>>> 
>>> return -1;