[PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()

Jiucheng Xu via B4 Relay posted 1 patch 1 month, 2 weeks ago
block/bio.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
[PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()
Posted by Jiucheng Xu via B4 Relay 1 month, 2 weeks ago
From: Jiucheng Xu <jiucheng.xu@amlogic.com>

When current->bio_list is non-NULL in submit_bio_wait(),
submit_bio_noacct_nocheck appends bio to bio_list but skips IO
submission, causing submit_bio_wait() to hang indefinitely.

Fix this by temporarily backup bio_list, setting bio_list to
NULL before calling submit_bio(), then restoring bio_list
after submit_bio() returns.

I've trimmed down the call stack, as follows:

f2fs_submit_read_io
  submit_bio
    mmc_blk_mq_recovery
      z_erofs_endio
        vm_map_ram
          __pte_alloc_kernel
            __alloc_pages_direct_reclaim
              shrink_folio_list
                __swap_writepage
                  submit_bio_wait  hang!!!

Signed-off-by: Jiucheng Xu <jiucheng.xu@amlogic.com>
---
 block/bio.c | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/block/bio.c b/block/bio.c
index d80d5d26804e32944bcfe4506ca190033308844f..22c8769722cc89620c239310a0f3d4924de68cf9 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1505,8 +1505,17 @@ int submit_bio_wait(struct bio *bio)
 	bio->bi_private = &done;
 	bio->bi_end_io = submit_bio_wait_endio;
 	bio->bi_opf |= REQ_SYNC;
-	submit_bio(bio);
-	blk_wait_io(&done);
+	if (!current->bio_list) {
+		submit_bio(bio);
+		blk_wait_io(&done);
+	} else {
+		struct bio_list *tmp = current->bio_list;
+
+		current->bio_list = NULL;
+		submit_bio(bio);
+		blk_wait_io(&done);
+		current->bio_list = tmp;
+	}
 
 	return blk_status_to_errno(bio->bi_status);
 }

---
base-commit: 8c5f40a3ba43ae9a26991f0e4a01a3a06e8958fc
change-id: 20260224-for-next-df6f02c3694d

Best regards,
-- 
Jiucheng Xu <jiucheng.xu@amlogic.com>
Re: [PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()
Posted by Christoph Hellwig 1 month, 2 weeks ago
On Mon, Mar 02, 2026 at 10:51:03AM +0800, Jiucheng Xu via B4 Relay wrote:
> From: Jiucheng Xu <jiucheng.xu@amlogic.com>
> 
> When current->bio_list is non-NULL in submit_bio_wait(),
> submit_bio_noacct_nocheck appends bio to bio_list but skips IO
> submission, causing submit_bio_wait() to hang indefinitely.
> 
> Fix this by temporarily backup bio_list, setting bio_list to
> NULL before calling submit_bio(), then restoring bio_list
> after submit_bio() returns.

No.  Fix this by not doing something that is a bad idea.

> I've trimmed down the call stack, as follows:
> 
> f2fs_submit_read_io
>   submit_bio
>     mmc_blk_mq_recovery
>       z_erofs_endio
>         vm_map_ram

->bi_end_io code really should not be having random in_atomic()
checks that make it completely different, but even if they have
that need to use GFP_NOIO.
Re: [PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()
Posted by Gao Xiang 1 month, 2 weeks ago
Hi Christoph,

On 2026/3/2 21:50, Christoph Hellwig wrote:
> On Mon, Mar 02, 2026 at 10:51:03AM +0800, Jiucheng Xu via B4 Relay wrote:
>> From: Jiucheng Xu <jiucheng.xu@amlogic.com>
>>
>> When current->bio_list is non-NULL in submit_bio_wait(),
>> submit_bio_noacct_nocheck appends bio to bio_list but skips IO
>> submission, causing submit_bio_wait() to hang indefinitely.
>>
>> Fix this by temporarily backup bio_list, setting bio_list to
>> NULL before calling submit_bio(), then restoring bio_list
>> after submit_bio() returns.
> 
> No.  Fix this by not doing something that is a bad idea.
> 
>> I've trimmed down the call stack, as follows:
>>
>> f2fs_submit_read_io
>>    submit_bio
>>      mmc_blk_mq_recovery
>>        z_erofs_endio
>>          vm_map_ram
> 
> ->bi_end_io code really should not be having random in_atomic()
> checks that make it completely different, but even if they have

Thanks for the head-up.

For this part, I'm pretty sure we need this particular one
otherwise the scheduling performance (latency sensitive)
is unacceptable for all Android phone users.

> that need to use GFP_NOIO.

Yes, it should make vm_map_ram() in the end_io path use
GFP_NOIO instead.

Jiucheng, could you add memalloc_noio_{save,restore}() to
wrap up this path?

Thanks,
Gao Xiang
Re: [PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()
Posted by Jiucheng Xu 1 month, 2 weeks ago

On 3/2/2026 10:23 PM, Gao Xiang wrote:
> [Some people who received this message don't often get email from 
> hsiangkao@linux.alibaba.com. Learn why this is important at https:// 
> aka.ms/LearnAboutSenderIdentification ]
> 
> [ EXTERNAL EMAIL ]
> 
> Hi Christoph,
> 
> On 2026/3/2 21:50, Christoph Hellwig wrote:
>> On Mon, Mar 02, 2026 at 10:51:03AM +0800, Jiucheng Xu via B4 Relay wrote:
>>> From: Jiucheng Xu <jiucheng.xu@amlogic.com>
>>>
>>> When current->bio_list is non-NULL in submit_bio_wait(),
>>> submit_bio_noacct_nocheck appends bio to bio_list but skips IO
>>> submission, causing submit_bio_wait() to hang indefinitely.
>>>
>>> Fix this by temporarily backup bio_list, setting bio_list to
>>> NULL before calling submit_bio(), then restoring bio_list
>>> after submit_bio() returns.
>>
>> No.  Fix this by not doing something that is a bad idea.
>>
>>> I've trimmed down the call stack, as follows:
>>>
>>> f2fs_submit_read_io
>>>    submit_bio
>>>      mmc_blk_mq_recovery
>>>        z_erofs_endio
>>>          vm_map_ram
>>
>> ->bi_end_io code really should not be having random in_atomic()
>> checks that make it completely different, but even if they have
> 
> Thanks for the head-up.
> 
> For this part, I'm pretty sure we need this particular one
> otherwise the scheduling performance (latency sensitive)
> is unacceptable for all Android phone users.
> 
>> that need to use GFP_NOIO.
> 
> Yes, it should make vm_map_ram() in the end_io path use
> GFP_NOIO instead.
> 
> Jiucheng, could you add memalloc_noio_{save,restore}() to
> wrap up this path?

Thanks for Christoph's and Xiang's comments, I will try it. Thanks!

Best Regards,
Jiucheng
Re: [PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()
Posted by Gao Xiang 1 month, 2 weeks ago

On 2026/3/3 10:03, Jiucheng Xu wrote:
> 
> 

...

>>
>>> that need to use GFP_NOIO.
>>
>> Yes, it should make vm_map_ram() in the end_io path use
>> GFP_NOIO instead.
>>
>> Jiucheng, could you add memalloc_noio_{save,restore}() to
>> wrap up this path?
> 
> Thanks for Christoph's and Xiang's comments, I will try it. Thanks!

Just one more note: just wrap up z_erofs_decompressqueue_work() in
z_erofs_decompress_kickoff() with memalloc_noio_{save,restore}() is
enough.

  ...
  memalloc_noio_save()
  z_erofs_decompressqueue_work()
  memalloc_noio_restore()

> 
> Best Regards,
> Jiucheng
Re: [PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()
Posted by Jiucheng Xu 1 month, 2 weeks ago

On 3/3/2026 10:11 AM, Gao Xiang wrote:
> [Some people who received this message don't often get email from 
> hsiangkao@linux.alibaba.com. Learn why this is important at https:// 
> aka.ms/LearnAboutSenderIdentification ]
> 
> [ EXTERNAL EMAIL ]
> 
> On 2026/3/3 10:03, Jiucheng Xu wrote:
>>
>>
> 
> ...
> 
>>>
>>>> that need to use GFP_NOIO.
>>>
>>> Yes, it should make vm_map_ram() in the end_io path use
>>> GFP_NOIO instead.
>>>
>>> Jiucheng, could you add memalloc_noio_{save,restore}() to
>>> wrap up this path?
>>
>> Thanks for Christoph's and Xiang's comments, I will try it. Thanks!
> 
> Just one more note: just wrap up z_erofs_decompressqueue_work() in
> z_erofs_decompress_kickoff() with memalloc_noio_{save,restore}() is
> enough.
> 
>   ...
>   memalloc_noio_save()
>   z_erofs_decompressqueue_work()
>   memalloc_noio_restore()
Got it, thanks for the details!


Re: [PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()
Posted by Christoph Hellwig 1 month, 2 weeks ago
On Mon, Mar 02, 2026 at 10:23:04PM +0800, Gao Xiang wrote:
> > > I've trimmed down the call stack, as follows:
> > > 
> > > f2fs_submit_read_io
> > >    submit_bio
> > >      mmc_blk_mq_recovery
> > >        z_erofs_endio
> > >          vm_map_ram
> > 
> > ->bi_end_io code really should not be having random in_atomic()
> > checks that make it completely different, but even if they have
> 
> Thanks for the head-up.
> 
> For this part, I'm pretty sure we need this particular one
> otherwise the scheduling performance (latency sensitive)
> is unacceptable for all Android phone users.

Where do you regularly get user context calls to ->bi_end_io?
Re: [PATCH] block: avoild hang when bio_list is non-NULL in submit_bio_wait()
Posted by Gao Xiang 1 month, 2 weeks ago

On 2026/3/2 22:29, Christoph Hellwig wrote:
> On Mon, Mar 02, 2026 at 10:23:04PM +0800, Gao Xiang wrote:
>>>> I've trimmed down the call stack, as follows:
>>>>
>>>> f2fs_submit_read_io
>>>>     submit_bio
>>>>       mmc_blk_mq_recovery
>>>>         z_erofs_endio
>>>>           vm_map_ram
>>>
>>> ->bi_end_io code really should not be having random in_atomic()
>>> checks that make it completely different, but even if they have
>>
>> Thanks for the head-up.
>>
>> For this part, I'm pretty sure we need this particular one
>> otherwise the scheduling performance (latency sensitive)
>> is unacceptable for all Android phone users.
> 
> Where do you regularly get user context calls to ->bi_end_io?

The obvious one is that dm-verity, it's actually in
the workqueue context.

Thanks,
Gao Xiang