The softlockup still occurs in get_swap_pages() under memory pressure.
64 CPU cores, 64GB memory, and 28 zram devices, the disksize of each
zram device is 50MB with same priority as si. Use the stress-ng tool
to increase memory pressure, causing the system to oom frequently.

The plist_for_each_entry_safe() loops in get_swap_pages() could reach
tens of thousands of times to find available space (extreme case:
cond_resched() is not called in scan_swap_map_slots()). Let's add
cond_resched() into get_swap_pages() when failed to find available
space to avoid softlockup.

Signed-off-by: Longlong Xia <xialonglong1@huawei.com>
---
 mm/swapfile.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/swapfile.c b/mm/swapfile.c
index 908a529bca12..4fa440e87cd6 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -1100,6 +1100,7 @@ int get_swap_pages(int n_goal, swp_entry_t swp_entries[], int entry_size)
 			goto check_out;
 		pr_debug("scan_swap_map of si %d failed to find offset\n",
 			si->type);
+		cond_resched();
 
 		spin_lock(&swap_avail_lock);
 nextsi:
-- 
2.25.1

On Sat, 28 Jan 2023 09:47:57 +0000 Longlong Xia <xialonglong1@huawei.com> wrote:

> The softlockup still occurs in get_swap_pages() under memory pressure.
> 64 CPU cores, 64GB memory, and 28 zram devices, the disksize of each
> zram device is 50MB with same priority as si. Use the stress-ng tool
> to increase memory pressure, causing the system to oom frequently.
> 
> The plist_for_each_entry_safe() loops in get_swap_pages() could reach
> tens of thousands of times to find available space (extreme case:
> cond_resched() is not called in scan_swap_map_slots()). Let's add
> cond_resched() into get_swap_pages() when failed to find available
> space to avoid softlockup.
> 
> ...
>
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -1100,6 +1100,7 @@ int get_swap_pages(int n_goal, swp_entry_t swp_entries[], int entry_size)
>  			goto check_out;
>  		pr_debug("scan_swap_map of si %d failed to find offset\n",
>  			si->type);
> +		cond_resched();
>  
>  		spin_lock(&swap_avail_lock);
>  nextsi:

This must be pretty rare?  My googling for "scan_swap_map of si %d
failed to find offset" turns up zero reports, but I guess few people
enable pr_debug.

I wonder if we should remove that pr_debug().  I mean, it's known that
this happens, what value does the printk add?

I'm thinking this fix should be backported into -stable kernels.

Andrew Morton <akpm@linux-foundation.org> writes:

> On Sat, 28 Jan 2023 09:47:57 +0000 Longlong Xia <xialonglong1@huawei.com> wrote:
>
>> The softlockup still occurs in get_swap_pages() under memory pressure.
>> 64 CPU cores, 64GB memory, and 28 zram devices, the disksize of each
>> zram device is 50MB with same priority as si. Use the stress-ng tool
>> to increase memory pressure, causing the system to oom frequently.
>> 
>> The plist_for_each_entry_safe() loops in get_swap_pages() could reach
>> tens of thousands of times to find available space (extreme case:
>> cond_resched() is not called in scan_swap_map_slots()). Let's add
>> cond_resched() into get_swap_pages() when failed to find available
>> space to avoid softlockup.
>> 
>> ...
>>
>> --- a/mm/swapfile.c
>> +++ b/mm/swapfile.c
>> @@ -1100,6 +1100,7 @@ int get_swap_pages(int n_goal, swp_entry_t swp_entries[], int entry_size)
>>  			goto check_out;
>>  		pr_debug("scan_swap_map of si %d failed to find offset\n",
>>  			si->type);
>> +		cond_resched();
>>  
>>  		spin_lock(&swap_avail_lock);
>>  nextsi:
>
> This must be pretty rare?  My googling for "scan_swap_map of si %d
> failed to find offset" turns up zero reports, but I guess few people
> enable pr_debug.
>
> I wonder if we should remove that pr_debug().  I mean, it's known that
> this happens, what value does the printk add?

Sounds reasonable to me.  And if we want to debug, we can use bpf too.

> I'm thinking this fix should be backported into -stable kernels.

Best Regards,
Huang, Ying

It's known that get_swap_pages() may fail to find available space
under some extreme case, but pr_debug() provides useless information.
Let's remove it.

Signed-off-by: Longlong Xia <xialonglong1@huawei.com>
---
 mm/swapfile.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/mm/swapfile.c b/mm/swapfile.c
index 6202a6668a63..99143875d6f0 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -1098,8 +1098,6 @@ int get_swap_pages(int n_goal, swp_entry_t swp_entries[], int entry_size)
 		spin_unlock(&si->lock);
 		if (n_ret || size == SWAPFILE_CLUSTER)
 			goto check_out;
-		pr_debug("scan_swap_map of si %d failed to find offset\n",
-			si->type);
 		cond_resched();
 
 		spin_lock(&swap_avail_lock);
-- 
2.25.1

Longlong Xia <xialonglong1@huawei.com> writes:

> It's known that get_swap_pages() may fail to find available space
> under some extreme case, but pr_debug() provides useless information.
> Let's remove it.
>
> Signed-off-by: Longlong Xia <xialonglong1@huawei.com>

Thanks!

Reviewed-by: "Huang, Ying" <ying.huang@intel.com>

> ---
>  mm/swapfile.c | 2 --
>  1 file changed, 2 deletions(-)
>
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index 6202a6668a63..99143875d6f0 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -1098,8 +1098,6 @@ int get_swap_pages(int n_goal, swp_entry_t swp_entries[], int entry_size)
>  		spin_unlock(&si->lock);
>  		if (n_ret || size == SWAPFILE_CLUSTER)
>  			goto check_out;
> -		pr_debug("scan_swap_map of si %d failed to find offset\n",
> -			si->type);
>  		cond_resched();
>  
>  		spin_lock(&swap_avail_lock);

Longlong Xia <xialonglong1@huawei.com> writes:

> The softlockup still occurs in get_swap_pages() under memory pressure.
> 64 CPU cores, 64GB memory, and 28 zram devices, the disksize of each
> zram device is 50MB with same priority as si. Use the stress-ng tool
> to increase memory pressure, causing the system to oom frequently.
>
> The plist_for_each_entry_safe() loops in get_swap_pages() could reach
> tens of thousands of times to find available space (extreme case:
> cond_resched() is not called in scan_swap_map_slots()). Let's add
> cond_resched() into get_swap_pages() when failed to find available
> space to avoid softlockup.
>
> Signed-off-by: Longlong Xia <xialonglong1@huawei.com>

Looks good to me. Thanks!

Reviewed-by: "Huang, Ying" <ying.huang@intel.com>

> ---
>  mm/swapfile.c | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index 908a529bca12..4fa440e87cd6 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -1100,6 +1100,7 @@ int get_swap_pages(int n_goal, swp_entry_t swp_entries[], int entry_size)
>  			goto check_out;
>  		pr_debug("scan_swap_map of si %d failed to find offset\n",
>  			si->type);
> +		cond_resched();
>  
>  		spin_lock(&swap_avail_lock);
>  nextsi: