[PATCH 04/10] blk-mq: serialize updating nr_requests with update_nr_hwq_lock

Yu Kuai posted 10 patches 1 month, 2 weeks ago
[PATCH 04/10] blk-mq: serialize updating nr_requests with update_nr_hwq_lock
Posted by Yu Kuai 1 month, 2 weeks ago
From: Yu Kuai <yukuai3@huawei.com>

request_queue->nr_requests can be changed by:

a) switching elevator by update nr_hw_queues
b) switching elevator by elevator sysfs attribute
c) configue queue sysfs attribute nr_requests

Current lock order is:

1) update_nr_hwq_lock, case a,b
- allocate elevator tags
2) freeze_queue
3) elevator_lock, cas a,b,c
- update nr_requests

The update_nr_hwq_lock is not held by case c for now, and we need to check
nr_requests while elevator_lock is held, however, we can't allocate
elevator_tags whih queue freezed.

Hence also require update_nr_hwq_lock and check nr_requests, make it
possible to allocate new elevator_tags without queue freezed, also
prevent nr_requests to be changed concurrently by other contexts.

Signed-off-by: Yu Kuai <yukuai3@huawei.com>
---
 block/blk-sysfs.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index f3d08edcc34f..37eae7927b45 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -68,13 +68,14 @@ queue_requests_store(struct gendisk *disk, const char *page, size_t count)
 	int ret, err;
 	unsigned int memflags;
 	struct request_queue *q = disk->queue;
+	struct blk_mq_tag_set *set = q->tag_set;
 
 	ret = queue_var_store(&nr, page, count);
 	if (ret < 0)
 		return ret;
 
-	memflags = blk_mq_freeze_queue(q);
-	mutex_lock(&q->elevator_lock);
+	/* serialize with switching elevator */
+	down_write(&set->update_nr_hwq_lock);
 
 	if (q->nr_requests == nr)
 		goto unlock;
@@ -89,13 +90,17 @@ queue_requests_store(struct gendisk *disk, const char *page, size_t count)
 		goto unlock;
 	}
 
+	memflags = blk_mq_freeze_queue(q);
+	mutex_lock(&q->elevator_lock);
 	err = blk_mq_update_nr_requests(disk->queue, nr);
 	if (err)
 		ret = err;
 
-unlock:
 	mutex_unlock(&q->elevator_lock);
 	blk_mq_unfreeze_queue(q, memflags);
+
+unlock:
+	up_write(&set->update_nr_hwq_lock);
 	return ret;
 }
 
-- 
2.39.2
Re: [PATCH 04/10] blk-mq: serialize updating nr_requests with update_nr_hwq_lock
Posted by Ming Lei 1 month, 2 weeks ago
On Fri, Aug 15, 2025 at 04:02:10PM +0800, Yu Kuai wrote:
> From: Yu Kuai <yukuai3@huawei.com>
> 
> request_queue->nr_requests can be changed by:
> 
> a) switching elevator by update nr_hw_queues
> b) switching elevator by elevator sysfs attribute
> c) configue queue sysfs attribute nr_requests

 ->elevator_lock is grabbed for updating ->nr_requests except for queue
initialization, so what is the real problem you are trying to solve?


Thanks,
Ming
Re: [PATCH 04/10] blk-mq: serialize updating nr_requests with update_nr_hwq_lock
Posted by Yu Kuai 1 month, 2 weeks ago
Hi,

在 2025/8/15 22:47, Ming Lei 写道:
> On Fri, Aug 15, 2025 at 04:02:10PM +0800, Yu Kuai wrote:
>> From: Yu Kuai <yukuai3@huawei.com>
>>
>> request_queue->nr_requests can be changed by:
>>
>> a) switching elevator by update nr_hw_queues
>> b) switching elevator by elevator sysfs attribute
>> c) configue queue sysfs attribute nr_requests
>   ->elevator_lock is grabbed for updating ->nr_requests except for queue
> initialization, so what is the real problem you are trying to solve?
>
In order to fix the regression and prevent deadlock, we'll have to 
allocate memory

in the case bitmap tags grow before freezing the queue, also prevent 
nr_requests

to be updated by case a and b concurrently.


Thanks,

Kuai

> Thanks,
> Ming
>
>
Re: [PATCH 04/10] blk-mq: serialize updating nr_requests with update_nr_hwq_lock
Posted by Ming Lei 1 month, 2 weeks ago
On Sat, Aug 16, 2025 at 08:49:41AM +0800, Yu Kuai wrote:
> Hi,
> 
> 在 2025/8/15 22:47, Ming Lei 写道:
> > On Fri, Aug 15, 2025 at 04:02:10PM +0800, Yu Kuai wrote:
> > > From: Yu Kuai <yukuai3@huawei.com>
> > > 
> > > request_queue->nr_requests can be changed by:
> > > 
> > > a) switching elevator by update nr_hw_queues
> > > b) switching elevator by elevator sysfs attribute
> > > c) configue queue sysfs attribute nr_requests
> >   ->elevator_lock is grabbed for updating ->nr_requests except for queue
> > initialization, so what is the real problem you are trying to solve?
> > 
> In order to fix the regression and prevent deadlock, we'll have to allocate
> memory
> 
> in the case bitmap tags grow before freezing the queue, also prevent
> nr_requests
> 
> to be updated by case a and b concurrently.

Yeah, that is potential deadlock issue in updating nr_request.

I feel the patch title is a bit confusing because q->nr_requests updating
is already serialized.


Thanks,
Ming