From nobody Sat Oct 4 17:31:12 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD15026CE1A; Thu, 14 Aug 2025 03:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142999; cv=none; b=sXxxOTU4l+grqC4nomtDpRBVpHpWN1u09TjD1r9yhBp9mRgWfEZ+q7BccM7AkrGjnQQ9wcbuZSOliWCCy15UahwznVdgADQscUmrqrY0fEXmuXok1elNUqaYtYJRt5637T/i88jRYsF8U0+kYXw8YW16oTWsqEVZE6t2vA1m8aw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142999; c=relaxed/simple; bh=49dd3k2uOhNsqUa1AxTb+jCQ1g7RZ+Wk2EQCin/MhFU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ez126COYaSEXY3/BeUEyZTIcbhXHTExG8+hcKXfE9KiJeeztOgPEcnul0CksgOw+GX7rtyWOzKZ2vdpFAIteG0JXoG3xQwgYIXM6kRKBdhfKmmnD9IMdBM/k9tebhwkx33UZt7DwwWuRitHWP7PauPcFoaD3PJl88ivNKEOYOqI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKp0szQzYQv4t; Thu, 14 Aug 2025 11:43:10 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id B8D101A1A7C; Thu, 14 Aug 2025 11:43:08 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S5; Thu, 14 Aug 2025 11:43:08 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 01/16] blk-mq-sched: add new parameter nr_requests in blk_mq_alloc_sched_tags() Date: Thu, 14 Aug 2025 11:35:07 +0800 Message-Id: <20250814033522.770575-2-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S5 X-Coremail-Antispam: 1UD129KBjvJXoWxJw4UtFWUAw45Jr1rJF18Xwb_yoW5AFWDpF s8Xa1qk3yYqr4kXay7A3yxZw17Kws29FyxJrWft3yYyr1vqws3u3W09r43XrW0yrs5AFsI yr1Dtay5XrsFq3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUm014x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWU JVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67 kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY 6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0x vEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVj vjDU0xZFpf9x0JU4OJ5UUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai This helper only support to iallocate the default number of requests, add a new parameter to support specific number of requests. Prepare to fix tags double free problem if nr_requests is grown by queue sysfs attribute nr_requests. Signed-off-by: Yu Kuai --- block/blk-mq-sched.c | 11 +++++++---- block/blk-mq-sched.h | 2 +- block/elevator.c | 2 +- 3 files changed, 9 insertions(+), 6 deletions(-) diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index e2ce4a28e6c9..9a8a0b5e04a9 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -454,7 +454,7 @@ void blk_mq_free_sched_tags_batch(struct xarray *et_tab= le, } =20 struct elevator_tags *blk_mq_alloc_sched_tags(struct blk_mq_tag_set *set, - unsigned int nr_hw_queues) + unsigned int nr_hw_queues, unsigned int nr_requests) { unsigned int nr_tags; int i; @@ -475,8 +475,11 @@ struct elevator_tags *blk_mq_alloc_sched_tags(struct b= lk_mq_tag_set *set, * 128, since we don't split into sync/async like the old code * did. Additionally, this is a per-hw queue depth. */ - et->nr_requests =3D 2 * min_t(unsigned int, set->queue_depth, - BLKDEV_DEFAULT_RQ); + if (nr_requests) + et->nr_requests =3D nr_requests; + else + et->nr_requests =3D 2 * min_t(unsigned int, set->queue_depth, + BLKDEV_DEFAULT_RQ); et->nr_hw_queues =3D nr_hw_queues; =20 if (blk_mq_is_shared_tags(set->flags)) { @@ -521,7 +524,7 @@ int blk_mq_alloc_sched_tags_batch(struct xarray *et_tab= le, * concurrently. */ if (q->elevator) { - et =3D blk_mq_alloc_sched_tags(set, nr_hw_queues); + et =3D blk_mq_alloc_sched_tags(set, nr_hw_queues, 0); if (!et) goto out_unwind; if (xa_insert(et_table, q->id, et, gfp)) diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index b554e1d55950..0582d4bc3987 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -24,7 +24,7 @@ void blk_mq_exit_sched(struct request_queue *q, struct el= evator_queue *e); void blk_mq_sched_free_rqs(struct request_queue *q); =20 struct elevator_tags *blk_mq_alloc_sched_tags(struct blk_mq_tag_set *set, - unsigned int nr_hw_queues); + unsigned int nr_hw_queues, unsigned int nr_requests); int blk_mq_alloc_sched_tags_batch(struct xarray *et_table, struct blk_mq_tag_set *set, unsigned int nr_hw_queues); void blk_mq_free_sched_tags(struct elevator_tags *et, diff --git a/block/elevator.c b/block/elevator.c index fe96c6f4753c..f8a04f32cbcf 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -669,7 +669,7 @@ static int elevator_change(struct request_queue *q, str= uct elv_change_ctx *ctx) lockdep_assert_held(&set->update_nr_hwq_lock); =20 if (strncmp(ctx->name, "none", 4)) { - ctx->et =3D blk_mq_alloc_sched_tags(set, set->nr_hw_queues); + ctx->et =3D blk_mq_alloc_sched_tags(set, set->nr_hw_queues, 0); if (!ctx->et) return -ENOMEM; } --=20 2.39.2 From nobody Sat Oct 4 17:31:12 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AFA011CA0; Thu, 14 Aug 2025 03:43:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142993; cv=none; b=ZVf4YwUeqY6+N0CQjUiag6npzRcFz+R/9a0VNoJDKlj2fB5k35BjD6YDULmcXfpXc/2LOXbcyQxJq5etWvBirMyuBhzNZX1nB3zlMYgN/60xwEFxnfIkkD9Xvh3XLeZ9gYMn7zLgotKKoi2b//nyFRG1QO10CtS7huZcmQIT7uQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142993; c=relaxed/simple; bh=n6vReAJVtwzVAOe+TgrKwM2LSZah4lupyvFThnDLfB4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tOTVogZBFsjXCEGD+utV5L0h0OoegmmXrppeQx8nMT/0oTTjtQD5fPRGnUYntG+2vUbp1+SztzwoM9HkmKUSk8VTLOzkcaqpLZwvtC+BaA7U7dbJpI9cOsSP4d6v6rI4EYxmxkV8Z6PSKAdhlgSR2KV5KuS1J2WmTtcCNOvURQU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=none smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKn74HMzKHMdf; Thu, 14 Aug 2025 11:43:09 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 481121A1A84; Thu, 14 Aug 2025 11:43:09 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S6; Thu, 14 Aug 2025 11:43:09 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 02/16] blk-mq: remove useless checking from queue_requests_store() Date: Thu, 14 Aug 2025 11:35:08 +0800 Message-Id: <20250814033522.770575-3-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S6 X-Coremail-Antispam: 1UD129KBjvdXoW7JrWxurW3KF13Aw47ZFy8Grg_yoWxtFg_GF y8Kry2qrsakF48Zr47Ar10qF4Ikw1rGF45WFWDJas5AFyfJa4fKrWrXryrAr43ua9rGa1r C3WxG34xGr40vjkaLaAFLSUrUUUUjb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUIcSsGvfJTRUUUbkxFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k26cxKx2IYs7xG 6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUXwA2048vs2IY02 0Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM2 8EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AI xVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20x vE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xv r2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2IY04 v7MxkF7I0En4kS14v26r1q6r43MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j 6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7 AF67AKxVWUtVW8ZwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE 2Ix0cI8IcVCY1x0267AKxVW8JVWxJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcV C2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2Kfnx nUUI43ZEXa7VUbH5lUUUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai blk_mq_queue_attr_visible() already checked queue_is_mq(), no need to check this again in queue_requests_store(). Signed-off-by: Yu Kuai --- block/blk-sysfs.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index c5cf79a20842..1086f7b9da28 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -69,9 +69,6 @@ queue_requests_store(struct gendisk *disk, const char *pa= ge, size_t count) unsigned int memflags; struct request_queue *q =3D disk->queue; =20 - if (!queue_is_mq(q)) - return -EINVAL; - ret =3D queue_var_store(&nr, page, count); if (ret < 0) return ret; --=20 2.39.2 From nobody Sat Oct 4 17:31:12 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4E35270551; Thu, 14 Aug 2025 03:43:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142999; cv=none; b=nRkfM/94iGhuIzeU4g7RMqg8GYgwFDCYujfX7Uya0uDz2gbzEjyuxLpTTvAFfobc4bz2wea2wzl6OzT710T78w/Hz4/GoA4ruJFtjB33VEVLcynbyLAnKhycKNGcHBraO22dHrBmhKiLK1/hckSzTl6a3QgVXiznmL3hoLlz5ew= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142999; c=relaxed/simple; bh=85gZwIK6x546CVb930ksOXuGAXPnVm8Y2M/um/VlP5Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XUb+jM3+In10Km+lYADSsgHYxnKCiniAoIb4IybcZKgNBhUU1SBEYGICvSarEERBYudbG5ldeS2W1eriAcDyShvY/OduJG/q17C/xyaWG9mfUOCnNvuvxmhagudI+qgDaoH5Z7CGljGuZiCDx0a5nMxV6Nxp44wNe+4urOq/AoI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKq19bZzYQv3s; Thu, 14 Aug 2025 11:43:11 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id BE01C1A018D; Thu, 14 Aug 2025 11:43:09 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S7; Thu, 14 Aug 2025 11:43:09 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 03/16] blk-mq: remove useless checkings from blk_mq_update_nr_requests() Date: Thu, 14 Aug 2025 11:35:09 +0800 Message-Id: <20250814033522.770575-4-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S7 X-Coremail-Antispam: 1UD129KBjvJXoW7AFy7Wr18CryxtrWkurW3Wrg_yoW8GF4fpF W3Ga1ak340gr18Aas7tan7Xa47Gr4DCr1Ikr4aqr1ayry5KFsFvr1rKr4jqr1IyrZ3Aanr KFWDJ397Zr1kZa7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUm014x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JrWl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWU JVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67 kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY 6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0x vEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVj vjDU0xZFpf9x0JUHWlkUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai 1) queue_requests_store() is the only caller of blk_mq_update_nr_requests(), where queue is already freezed, no need to check mq_freeze_depth; 2) q->tag_set must be set for request_based device, and queue_is_mq() is already checked in blk_mq_queue_attr_visible(), no need to check q->tag_set. 3) During initialization, hctx->tags in initialized before queue kobject, and during del_gendisk, queue kobject is deleted before exiting hctx, hence checking hctx->tags is useless. Signed-off-by: Yu Kuai --- block/blk-mq.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index b67d6c02eceb..3a219b7b3688 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4921,24 +4921,15 @@ int blk_mq_update_nr_requests(struct request_queue = *q, unsigned int nr) { struct blk_mq_tag_set *set =3D q->tag_set; struct blk_mq_hw_ctx *hctx; - int ret; + int ret =3D 0; unsigned long i; =20 - if (WARN_ON_ONCE(!q->mq_freeze_depth)) - return -EINVAL; - - if (!set) - return -EINVAL; - if (q->nr_requests =3D=3D nr) return 0; =20 blk_mq_quiesce_queue(q); =20 - ret =3D 0; queue_for_each_hw_ctx(q, hctx, i) { - if (!hctx->tags) - continue; /* * If we're using an MQ scheduler, just update the scheduler * queue depth. This is similar to what the old code would do. --=20 2.39.2 From nobody Sat Oct 4 17:31:12 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 377EC14A62B; Thu, 14 Aug 2025 03:43:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142994; cv=none; b=ZriNKPNfUc9AjTVKPlTtLgcPZLS0mQVZ+8YFcBObQ5+3xo4+soo7o9chIS2WV1Lb7KP66zS5RwNoNI1GXHhJnBYzUpoUJGsi6rrgckY1K2UdeHB2685V7crOhkRyRb/qWxKET6Oss0vfGs/S7D/JOWCmcsYMWlnU1C+or+GDumc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142994; c=relaxed/simple; bh=saM+n0D5pmnodx9T6iE8cyyLYX/dVjqVxc4FoBTKkVw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=u23aR4T9ZPD1WCMGSouRp6Gfq8fWAVYAuY/HUTC7s9Ke9ViyW1ymIxg62+Cd4f5jB/Rlw+uHSqIUSpQUewYKjnqml+kIvaNZixP6uY9moUID3+hAS8Ir/bsG/bhnL76A2SfribvXWbgAilLff+fybUdpWKgTLcnchpDnGASQpNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKp6pdXzKHMhB; Thu, 14 Aug 2025 11:43:10 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 3D31F1A07BD; Thu, 14 Aug 2025 11:43:10 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S8; Thu, 14 Aug 2025 11:43:10 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 04/16] blk-mq: check invalid nr_requests in queue_requests_store() Date: Thu, 14 Aug 2025 11:35:10 +0800 Message-Id: <20250814033522.770575-5-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S8 X-Coremail-Antispam: 1UD129KBjvJXoWxCF4ruw1rXr45Kr17urWfXwb_yoWrCFyxpF W5Ka12k340qr10vFWIyay7Xw1Fkws3Kr1xGFsaqa4avr1jkr4xXF1FqryrZrW0yrWkCF47 Zr4DtrW5JF1kGrDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCw CI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnI WIevJa73UjIFyTuYvjfUOyIUUUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai queue_requests_store() is the only caller of blk_mq_update_nr_requests(), and blk_mq_update_nr_requests() is the only caller of blk_mq_tag_update_depth(), however, they all have checkings for nr_requests input by user. Make code cleaner by moving all the checkings to the top function: 1) nr_requests > reserved tags; 2) if there is elevator, 4 <=3D nr_requests <=3D 2048; 3) if elevator is none, 4 <=3D nr_requests < tag_set->queue_depth; Meanwhile, case 2 is the only case tags can grow and -ENOMEM might be returned. Signed-off-by: Yu Kuai --- block/blk-mq-tag.c | 16 +--------------- block/blk-mq.c | 14 +++++--------- block/blk-mq.h | 2 +- block/blk-sysfs.c | 13 +++++++++++++ 4 files changed, 20 insertions(+), 25 deletions(-) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index d880c50629d6..7613a9889eb1 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -584,14 +584,10 @@ void blk_mq_free_tags(struct blk_mq_tags *tags) } =20 int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx, - struct blk_mq_tags **tagsptr, unsigned int tdepth, - bool can_grow) + struct blk_mq_tags **tagsptr, unsigned int tdepth) { struct blk_mq_tags *tags =3D *tagsptr; =20 - if (tdepth <=3D tags->nr_reserved_tags) - return -EINVAL; - /* * If we are allowed to grow beyond the original size, allocate * a new set of tags before freeing the old one. @@ -600,16 +596,6 @@ int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx, struct blk_mq_tag_set *set =3D hctx->queue->tag_set; struct blk_mq_tags *new; =20 - if (!can_grow) - return -EINVAL; - - /* - * We need some sort of upper limit, set it high enough that - * no valid use cases should require more. - */ - if (tdepth > MAX_SCHED_RQ) - return -EINVAL; - /* * Only the sbitmap needs resizing since we allocated the max * initially. diff --git a/block/blk-mq.c b/block/blk-mq.c index 3a219b7b3688..e86cab125a2d 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4924,9 +4924,6 @@ int blk_mq_update_nr_requests(struct request_queue *q= , unsigned int nr) int ret =3D 0; unsigned long i; =20 - if (q->nr_requests =3D=3D nr) - return 0; - blk_mq_quiesce_queue(q); =20 queue_for_each_hw_ctx(q, hctx, i) { @@ -4934,13 +4931,12 @@ int blk_mq_update_nr_requests(struct request_queue = *q, unsigned int nr) * If we're using an MQ scheduler, just update the scheduler * queue depth. This is similar to what the old code would do. */ - if (hctx->sched_tags) { + if (hctx->sched_tags) ret =3D blk_mq_tag_update_depth(hctx, &hctx->sched_tags, - nr, true); - } else { - ret =3D blk_mq_tag_update_depth(hctx, &hctx->tags, nr, - false); - } + nr); + else + ret =3D blk_mq_tag_update_depth(hctx, &hctx->tags, nr); + if (ret) break; if (q->elevator && q->elevator->type->ops.depth_updated) diff --git a/block/blk-mq.h b/block/blk-mq.h index affb2e14b56e..2b3ade60c90b 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -171,7 +171,7 @@ void blk_mq_put_tag(struct blk_mq_tags *tags, struct bl= k_mq_ctx *ctx, unsigned int tag); void blk_mq_put_tags(struct blk_mq_tags *tags, int *tag_array, int nr_tags= ); int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx, - struct blk_mq_tags **tags, unsigned int depth, bool can_grow); + struct blk_mq_tags **tags, unsigned int depth); void blk_mq_tag_resize_shared_tags(struct blk_mq_tag_set *set, unsigned int size); void blk_mq_tag_update_sched_shared_tags(struct request_queue *q); diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 1086f7b9da28..f3d08edcc34f 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -75,12 +75,25 @@ queue_requests_store(struct gendisk *disk, const char *= page, size_t count) =20 memflags =3D blk_mq_freeze_queue(q); mutex_lock(&q->elevator_lock); + + if (q->nr_requests =3D=3D nr) + goto unlock; + if (nr < BLKDEV_MIN_RQ) nr =3D BLKDEV_MIN_RQ; =20 + if (nr <=3D q->tag_set->reserved_tags || + (q->elevator && nr > MAX_SCHED_RQ) || + (!q->elevator && nr > q->tag_set->queue_depth)) { + ret =3D -EINVAL; + goto unlock; + } + err =3D blk_mq_update_nr_requests(disk->queue, nr); if (err) ret =3D err; + +unlock: mutex_unlock(&q->elevator_lock); blk_mq_unfreeze_queue(q, memflags); return ret; --=20 2.39.2 From nobody Sat Oct 4 17:31:12 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C9A6B1C1AAA; Thu, 14 Aug 2025 03:43:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142995; cv=none; b=bL+MwjVWMDEcooAUYMm4ES8X/GhrF5KxNU+79K4+wUGktCvwWil/LUqwj14R3+9P8ovVAPynazBFbR47isR5DrnWT6+tJJl2C+YdlVuIUG17ukWpWF/TcC0mA5KuZ8b45vgcecOEPfh429CbBZY2vcXSAIM9YkovKPwSUj6aTwk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142995; c=relaxed/simple; bh=9HCgRc+TAQUdTQUM95EmCDP3h6calstWWkZjnBPOJ3g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ePd9xRsFgYUOStSe6CP0TlgGnd4xU922AQJY2RKo3d/NITmI9j+VEP3fW2CmMupx/YawsUmfFxxkWrAh7sKiZUhcjc2T/tAbuC8Fz1QPtSBD0nB/lMcAE8sV4codi0NazpG1AFTLAZoFXYKC6jdizb+emxreOJB7Uj64XPM5sl8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKq3HB9zKHMg4; Thu, 14 Aug 2025 11:43:11 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id B7A581A0BD4; Thu, 14 Aug 2025 11:43:10 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S9; Thu, 14 Aug 2025 11:43:10 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 05/16] blk-mq: fix elevator depth_updated method Date: Thu, 14 Aug 2025 11:35:11 +0800 Message-Id: <20250814033522.770575-6-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S9 X-Coremail-Antispam: 1UD129KBjvJXoW3AF4xWFyfGF4fZFyUuw17ZFb_yoWfJF47pF W5ta17KryrtF4UuFWjy39xXw43K39agry2yFs3t34rKrZrKFs3XF1rGFyxXFWIqrZ5CFsF vr4Dta4DXr1Iq3JanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCw CI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnI WIevJa73UjIFyTuYvjfUOyIUUUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai Current depth_updated has some problems: 1) depth_updated() will be called for each hctx, while all elevators will update async_depth for the disk level, this is not related to hctx; 2) In blk_mq_update_nr_requests(), if previous hctx update succeed and this hctx update failed, q->nr_requests will not be updated, while async_depth is already updated with new nr_reqeuests in previous depth_updated(); 3) All elevators are using q->nr_requests to calculate async_depth now, however, q->nr_requests is still the old value when depth_updated() is called from blk_mq_update_nr_requests(); Fix those problems by: - pass in request_queue instead of hctx; - move depth_updated() after q->nr_requests is updated in blk_mq_update_nr_requests(); - add depth_updated() call in blk_mq_init_sched(); - remove init_hctx() method for mq-deadline and bfq that is useless now; Fixes: 77f1e0a52d26 ("bfq: update internal depth state when queue depth cha= nges") Fixes: 39823b47bbd4 ("block/mq-deadline: Fix the tag reservation code") Fixes: 42e6c6ce03fd ("lib/sbitmap: convert shallow_depth from one word to t= he whole sbitmap") Signed-off-by: Yu Kuai --- block/bfq-iosched.c | 21 ++++----------------- block/blk-mq-sched.c | 3 +++ block/blk-mq-sched.h | 11 +++++++++++ block/blk-mq.c | 24 ++++++++++++------------ block/elevator.h | 2 +- block/kyber-iosched.c | 10 ++++------ block/mq-deadline.c | 15 ++------------- 7 files changed, 37 insertions(+), 49 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index 50e51047e1fe..c0c398998aa1 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -7109,9 +7109,10 @@ void bfq_put_async_queues(struct bfq_data *bfqd, str= uct bfq_group *bfqg) * See the comments on bfq_limit_depth for the purpose of * the depths set in the function. Return minimum shallow depth we'll use. */ -static void bfq_update_depths(struct bfq_data *bfqd, struct sbitmap_queue = *bt) +static void bfq_depth_updated(struct request_queue *q) { - unsigned int nr_requests =3D bfqd->queue->nr_requests; + struct bfq_data *bfqd =3D q->elevator->elevator_data; + unsigned int nr_requests =3D q->nr_requests; =20 /* * In-word depths if no bfq_queue is being weight-raised: @@ -7143,21 +7144,8 @@ static void bfq_update_depths(struct bfq_data *bfqd,= struct sbitmap_queue *bt) bfqd->async_depths[1][0] =3D max((nr_requests * 3) >> 4, 1U); /* no more than ~37% of tags for sync writes (~20% extra tags) */ bfqd->async_depths[1][1] =3D max((nr_requests * 6) >> 4, 1U); -} - -static void bfq_depth_updated(struct blk_mq_hw_ctx *hctx) -{ - struct bfq_data *bfqd =3D hctx->queue->elevator->elevator_data; - struct blk_mq_tags *tags =3D hctx->sched_tags; =20 - bfq_update_depths(bfqd, &tags->bitmap_tags); - sbitmap_queue_min_shallow_depth(&tags->bitmap_tags, 1); -} - -static int bfq_init_hctx(struct blk_mq_hw_ctx *hctx, unsigned int index) -{ - bfq_depth_updated(hctx); - return 0; + blk_mq_set_min_shallow_depth(q, 1); } =20 static void bfq_exit_queue(struct elevator_queue *e) @@ -7628,7 +7616,6 @@ static struct elevator_type iosched_bfq_mq =3D { .request_merged =3D bfq_request_merged, .has_work =3D bfq_has_work, .depth_updated =3D bfq_depth_updated, - .init_hctx =3D bfq_init_hctx, .init_sched =3D bfq_init_queue, .exit_sched =3D bfq_exit_queue, }, diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index 9a8a0b5e04a9..c6a1e5a25fec 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -588,6 +588,9 @@ int blk_mq_init_sched(struct request_queue *q, struct e= levator_type *e, } } } + + if (e->ops.depth_updated) + e->ops.depth_updated(q); return 0; =20 out: diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index 0582d4bc3987..8e21a6b1415d 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -92,4 +92,15 @@ static inline bool blk_mq_sched_needs_restart(struct blk= _mq_hw_ctx *hctx) return test_bit(BLK_MQ_S_SCHED_RESTART, &hctx->state); } =20 +static inline void blk_mq_set_min_shallow_depth(struct request_queue *q, + unsigned int depth) +{ + struct blk_mq_hw_ctx *hctx; + unsigned long i; + + queue_for_each_hw_ctx(q, hctx, i) + sbitmap_queue_min_shallow_depth(&hctx->sched_tags->bitmap_tags, + depth); +} + #endif diff --git a/block/blk-mq.c b/block/blk-mq.c index e86cab125a2d..14446d3927dd 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4938,22 +4938,22 @@ int blk_mq_update_nr_requests(struct request_queue = *q, unsigned int nr) ret =3D blk_mq_tag_update_depth(hctx, &hctx->tags, nr); =20 if (ret) - break; - if (q->elevator && q->elevator->type->ops.depth_updated) - q->elevator->type->ops.depth_updated(hctx); + goto out; } - if (!ret) { - q->nr_requests =3D nr; - if (blk_mq_is_shared_tags(set->flags)) { - if (q->elevator) - blk_mq_tag_update_sched_shared_tags(q); - else - blk_mq_tag_resize_shared_tags(set, nr); - } + + q->nr_requests =3D nr; + if (q->elevator && q->elevator->type->ops.depth_updated) + q->elevator->type->ops.depth_updated(q); + + if (blk_mq_is_shared_tags(set->flags)) { + if (q->elevator) + blk_mq_tag_update_sched_shared_tags(q); + else + blk_mq_tag_resize_shared_tags(set, nr); } =20 +out: blk_mq_unquiesce_queue(q); - return ret; } =20 diff --git a/block/elevator.h b/block/elevator.h index adc5c157e17e..c4d20155065e 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -37,7 +37,7 @@ struct elevator_mq_ops { void (*exit_sched)(struct elevator_queue *); int (*init_hctx)(struct blk_mq_hw_ctx *, unsigned int); void (*exit_hctx)(struct blk_mq_hw_ctx *, unsigned int); - void (*depth_updated)(struct blk_mq_hw_ctx *); + void (*depth_updated)(struct request_queue *); =20 bool (*allow_merge)(struct request_queue *, struct request *, struct bio = *); bool (*bio_merge)(struct request_queue *, struct bio *, unsigned int); diff --git a/block/kyber-iosched.c b/block/kyber-iosched.c index 70cbc7b2deb4..49ae52aa20d9 100644 --- a/block/kyber-iosched.c +++ b/block/kyber-iosched.c @@ -440,13 +440,12 @@ static void kyber_ctx_queue_init(struct kyber_ctx_que= ue *kcq) INIT_LIST_HEAD(&kcq->rq_list[i]); } =20 -static void kyber_depth_updated(struct blk_mq_hw_ctx *hctx) +static void kyber_depth_updated(struct request_queue *q) { - struct kyber_queue_data *kqd =3D hctx->queue->elevator->elevator_data; - struct blk_mq_tags *tags =3D hctx->sched_tags; + struct kyber_queue_data *kqd =3D q->elevator->elevator_data; =20 - kqd->async_depth =3D hctx->queue->nr_requests * KYBER_ASYNC_PERCENT / 100= U; - sbitmap_queue_min_shallow_depth(&tags->bitmap_tags, kqd->async_depth); + kqd->async_depth =3D q->nr_requests * KYBER_ASYNC_PERCENT / 100U; + blk_mq_set_min_shallow_depth(q, kqd->async_depth); } =20 static int kyber_init_hctx(struct blk_mq_hw_ctx *hctx, unsigned int hctx_i= dx) @@ -493,7 +492,6 @@ static int kyber_init_hctx(struct blk_mq_hw_ctx *hctx, = unsigned int hctx_idx) khd->batching =3D 0; =20 hctx->sched_data =3D khd; - kyber_depth_updated(hctx); =20 return 0; =20 diff --git a/block/mq-deadline.c b/block/mq-deadline.c index b9b7cdf1d3c9..578bc79c5654 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -507,22 +507,12 @@ static void dd_limit_depth(blk_opf_t opf, struct blk_= mq_alloc_data *data) } =20 /* Called by blk_mq_update_nr_requests(). */ -static void dd_depth_updated(struct blk_mq_hw_ctx *hctx) +static void dd_depth_updated(struct request_queue *q) { - struct request_queue *q =3D hctx->queue; struct deadline_data *dd =3D q->elevator->elevator_data; - struct blk_mq_tags *tags =3D hctx->sched_tags; =20 dd->async_depth =3D q->nr_requests; - - sbitmap_queue_min_shallow_depth(&tags->bitmap_tags, 1); -} - -/* Called by blk_mq_init_hctx() and blk_mq_init_sched(). */ -static int dd_init_hctx(struct blk_mq_hw_ctx *hctx, unsigned int hctx_idx) -{ - dd_depth_updated(hctx); - return 0; + blk_mq_set_min_shallow_depth(q, 1); } =20 static void dd_exit_sched(struct elevator_queue *e) @@ -1048,7 +1038,6 @@ static struct elevator_type mq_deadline =3D { .has_work =3D dd_has_work, .init_sched =3D dd_init_sched, .exit_sched =3D dd_exit_sched, - .init_hctx =3D dd_init_hctx, }, =20 #ifdef CONFIG_BLK_DEBUG_FS --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1DDF273D6D; Thu, 14 Aug 2025 03:43:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143000; cv=none; b=pvSLsGauY6OAo2nDhhGFXgGcosmcmIruxchlkchOeu/3ZubXsgBj+R9ZSDPag3gnfvYYIsKVkV8tDmCCLtaoJXV1YeJQNsunT6G6BF23tbG+VmN0oEyQHpBdkXqFA2050O8DEJGs7p/6vHliysgyPKeqVkobnQGW3JupRjrLD2o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143000; c=relaxed/simple; bh=mRbI3tmkA/TYRsZDHdbsy7279ICniBUzF0QF8ydpZ9c=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=F9G58ORc7r4s3E9mqzXpxMkrz8CUPLVjlR231R6w3eFmulD9hgXZq6GESzx1dkWuQ0JhbIBP4sl7UukJycqyOa653DORoXCTU4KHCFsI/erm8NaonXMAH7teIVf0FOOUDsQVnpjJ/mz+Zk1a8nT3ftb17qhA8kJd4vCBP/JtDQM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKr4QhWzYQv7q; Thu, 14 Aug 2025 11:43:12 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 3AA271A0BD8; Thu, 14 Aug 2025 11:43:11 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S10; Thu, 14 Aug 2025 11:43:11 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 06/16] blk-mq: cleanup shared tags case in blk_mq_update_nr_requests() Date: Thu, 14 Aug 2025 11:35:12 +0800 Message-Id: <20250814033522.770575-7-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S10 X-Coremail-Antispam: 1UD129KBjvJXoW7Cr1rWrWUuF1xCr17tF45Awb_yoW5JrW8pr W3Ka13K34Fqr1SvFWavw43Xw4Ygws3Wr1IkFsxJryFqr12kF4xGr4rWrn5XFW8trZ5AFsI 9F4DJFWUXr1kW37anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmI14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCw CI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnI WIevJa73UjIFyTuYvjfUOyIUUUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai For shared tags case, all hctx->shared_tags/tags are the same, it doesn't make sense to call blk_mq_tag_update_depth() multiple times for the same tags. Signed-off-by: Yu Kuai --- block/blk-mq-tag.c | 7 ------- block/blk-mq.c | 40 +++++++++++++++++++++------------------- 2 files changed, 21 insertions(+), 26 deletions(-) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index 7613a9889eb1..84a452e708e4 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -596,13 +596,6 @@ int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx, struct blk_mq_tag_set *set =3D hctx->queue->tag_set; struct blk_mq_tags *new; =20 - /* - * Only the sbitmap needs resizing since we allocated the max - * initially. - */ - if (blk_mq_is_shared_tags(set->flags)) - return 0; - new =3D blk_mq_alloc_map_and_rqs(set, hctx->queue_num, tdepth); if (!new) return -ENOMEM; diff --git a/block/blk-mq.c b/block/blk-mq.c index 14446d3927dd..c29f98ec283d 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4926,32 +4926,34 @@ int blk_mq_update_nr_requests(struct request_queue = *q, unsigned int nr) =20 blk_mq_quiesce_queue(q); =20 - queue_for_each_hw_ctx(q, hctx, i) { - /* - * If we're using an MQ scheduler, just update the scheduler - * queue depth. This is similar to what the old code would do. - */ - if (hctx->sched_tags) - ret =3D blk_mq_tag_update_depth(hctx, &hctx->sched_tags, - nr); - else - ret =3D blk_mq_tag_update_depth(hctx, &hctx->tags, nr); - - if (ret) - goto out; - } - - q->nr_requests =3D nr; - if (q->elevator && q->elevator->type->ops.depth_updated) - q->elevator->type->ops.depth_updated(q); - if (blk_mq_is_shared_tags(set->flags)) { if (q->elevator) blk_mq_tag_update_sched_shared_tags(q); else blk_mq_tag_resize_shared_tags(set, nr); + } else { + queue_for_each_hw_ctx(q, hctx, i) { + /* + * If we're using an MQ scheduler, just update the + * scheduler queue depth. This is similar to what the + * old code would do. + */ + if (hctx->sched_tags) + ret =3D blk_mq_tag_update_depth(hctx, + &hctx->sched_tags, nr); + else + ret =3D blk_mq_tag_update_depth(hctx, + &hctx->tags, nr); + + if (ret) + goto out; + } } =20 + q->nr_requests =3D nr; + if (q->elevator && q->elevator->type->ops.depth_updated) + q->elevator->type->ops.depth_updated(q); + out: blk_mq_unquiesce_queue(q); return ret; --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1709258CF6; Thu, 14 Aug 2025 03:43:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142995; cv=none; b=r4U+gMhgUJef1gVqESDuEnQUYvxG/7fBCLaxhK5KA3hd3Z+HwT3HHpAh5SwqUjAiob08aVfGleNYJ3MTMcj7Ro6GVRRG3J0bOhN8GDmYPAkGuIxPmt2PbDNbGwABqZ5BMIefYrN419Jzbo7hwUpHGQsD8qjAs8D9jsbklqw59L4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142995; c=relaxed/simple; bh=YPDyAuwKKOPXX/Tm4QMGe7PRqZNCYTsJRqSGPk1MFhA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MiGF7jnVikUv7ztph01oGoGvpGYza7D4aEmKeTh2ED2F07Guylf9Ka7RvwMO1+s+qMoTypL4glstC8NY2JT4OmVyVw3hdgnaYz5FaLlyrtpWpojnF453m1IM4dYlc2u8ayaMsrnVhUlH2KswyO9nlWNAl11eFsnaZSwHglpLVyc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKr356dzKHMSx; Thu, 14 Aug 2025 11:43:12 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id B07411A0CF1; Thu, 14 Aug 2025 11:43:11 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S11; Thu, 14 Aug 2025 11:43:11 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 07/16] blk-mq: split bitmap grow and resize case in blk_mq_update_nr_requests() Date: Thu, 14 Aug 2025 11:35:13 +0800 Message-Id: <20250814033522.770575-8-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S11 X-Coremail-Antispam: 1UD129KBjvJXoW7ZrWkXF18Ww1rZr17tFykGrg_yoW8XryrpF Waga17K34Fqr1FvFWIqw4UX3WFgFsaqF1xtFs3tFyFgr1jgr4Sgr4Sgry5XFy8Jrs3CayS 9r4DtryUAr17XrUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUma14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCw CI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsG vfC2KfnxnUUI43ZEXa7VUbPC7UUUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai No functional changes are intended, make code cleaner and prepare to fix the grow case in the next patch. Signed-off-by: Yu Kuai --- block/blk-mq.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index c29f98ec283d..a7d6a20c1524 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4931,6 +4931,14 @@ int blk_mq_update_nr_requests(struct request_queue *= q, unsigned int nr) blk_mq_tag_update_sched_shared_tags(q); else blk_mq_tag_resize_shared_tags(set, nr); + } else if (!q->elevator) { + queue_for_each_hw_ctx(q, hctx, i) + sbitmap_queue_resize(&hctx->tags->bitmap_tags, + nr - hctx->tags->nr_reserved_tags); + } else if (nr <=3D q->elevator->et->nr_requests) { + queue_for_each_hw_ctx(q, hctx, i) + sbitmap_queue_resize(&hctx->sched_tags->bitmap_tags, + nr - hctx->sched_tags->nr_reserved_tags); } else { queue_for_each_hw_ctx(q, hctx, i) { /* @@ -4938,13 +4946,8 @@ int blk_mq_update_nr_requests(struct request_queue *= q, unsigned int nr) * scheduler queue depth. This is similar to what the * old code would do. */ - if (hctx->sched_tags) - ret =3D blk_mq_tag_update_depth(hctx, - &hctx->sched_tags, nr); - else - ret =3D blk_mq_tag_update_depth(hctx, - &hctx->tags, nr); - + ret =3D blk_mq_tag_update_depth(hctx, + &hctx->sched_tags, nr); if (ret) goto out; } --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A3352750E6; Thu, 14 Aug 2025 03:43:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143001; cv=none; b=mo4TDCRkRVPwS3jv3V0tP6JoQzio0a6eIsZ8Gs5Y6cX677n3SjD8KD64AQjEFlr1d1j18roxa12ycDLnFoqM9PuqlY+RIYbh9COQvgGX79LsSpXrp1u8SiysLnbkmjFdBYzxdi/EKGhBhKSZuN08uiifnz5fQ/UAnOL054IJn4Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143001; c=relaxed/simple; bh=Jvoii4nVwMwLtCBE+IEICnuYYTqxC1rF9PACWcI0ppY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mjokNTeVfqJYDVy4W+V8BL4rNFu0CkSOugmW68HKm7oSRjnSaF3v0frcpVa7p+YP62f3OfL7L11glJIBCxu30fJ1k5gr/IrgS0Zp4IcZx2Z/C5Gq5VEkOIJQe5W6m8vq7lMS0zUZweZCwwaQe0/ZLiI6Z4X+68mcmu3c9oMYgok= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKs4GZNzYQv3p; Thu, 14 Aug 2025 11:43:13 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 342551A07BD; Thu, 14 Aug 2025 11:43:12 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S12; Thu, 14 Aug 2025 11:43:11 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 08/16] blk-mq: fix blk_mq_tags double free while nr_requests grown Date: Thu, 14 Aug 2025 11:35:14 +0800 Message-Id: <20250814033522.770575-9-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S12 X-Coremail-Antispam: 1UD129KBjvJXoWxur13Xr1DtF47ZFWfXw48Crg_yoW5Gr4xpF W5Wa13K34Fqr1xZFZ2y3y7Xas0gws2qw1xKFs3tryrtr1UKrs7ur48Wr4FqrWIyrWkCFZ2 krZ8GF9rXryqqaDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0JUQFxUUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai In the case user trigger tags grow by queue sysfs attribute nr_requests, hctx->sched_tags will be freed directly and replaced with a new allocated tags, see blk_mq_tag_update_depth(). The problem is that hctx->sched_tags is from elevator->et->tags, while et->tags is still the freed tags, hence later elevator exist will try to free the tags again, causing kernel panic. Fix this problem by using new halper blk_mq_alloc_sched_tags() to allocate a new sched_tags. Meanwhile, there is a longterm problem can be fixed as well: If blk_mq_tag_update_depth() succeed for previous hctx, then bitmap depth is updated, however, if following hctx failed, q->nr_requests is not updated and the previous hctx->sched_tags endup bigger than q->nr_requests. Fixes: f5a6604f7a44 ("block: fix lockdep warning caused by lock dependency = in elv_iosched_store") Fixes: e3a2b3f931f5 ("blk-mq: allow changing of queue depth through sysfs") Signed-off-by: Yu Kuai Reviewed-by: Ming Lei --- block/blk-mq.c | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index a7d6a20c1524..f1c11f591c27 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -4917,6 +4917,23 @@ void blk_mq_free_tag_set(struct blk_mq_tag_set *set) } EXPORT_SYMBOL(blk_mq_free_tag_set); =20 +static int blk_mq_sched_grow_tags(struct request_queue *q, unsigned int nr) +{ + struct elevator_tags *et =3D + blk_mq_alloc_sched_tags(q->tag_set, q->nr_hw_queues, nr); + struct blk_mq_hw_ctx *hctx; + unsigned long i; + + if (!et) + return -ENOMEM; + + blk_mq_free_sched_tags(q->elevator->et, q->tag_set); + queue_for_each_hw_ctx(q, hctx, i) + hctx->sched_tags =3D et->tags[i]; + q->elevator->et =3D et; + return 0; +} + int blk_mq_update_nr_requests(struct request_queue *q, unsigned int nr) { struct blk_mq_tag_set *set =3D q->tag_set; @@ -4940,17 +4957,9 @@ int blk_mq_update_nr_requests(struct request_queue *= q, unsigned int nr) sbitmap_queue_resize(&hctx->sched_tags->bitmap_tags, nr - hctx->sched_tags->nr_reserved_tags); } else { - queue_for_each_hw_ctx(q, hctx, i) { - /* - * If we're using an MQ scheduler, just update the - * scheduler queue depth. This is similar to what the - * old code would do. - */ - ret =3D blk_mq_tag_update_depth(hctx, - &hctx->sched_tags, nr); - if (ret) - goto out; - } + ret =3D blk_mq_sched_grow_tags(q, nr); + if (ret) + goto out; } =20 q->nr_requests =3D nr; --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD144262FC0; Thu, 14 Aug 2025 03:43:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142997; cv=none; b=CQgcpHMzmyveDpBgcOeSx+BCJ3IxjPLuZQ1NCCcFyJ3iXQknkT9PqXF0hGy+6Qs7bB/1LpkTyc3vz941A00+G/A0FKM/h9q9QsSFUa06ORpoKpfJSXCrWGpqnahEijRalDy50WAFXef6Ka0fsPxzZri6HtiwmcagWHw10R4oF1w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142997; c=relaxed/simple; bh=+ZYFiXz6XEMMB/z859pfgQTuxR1Xl5UD4q07I/9BRGc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bKomIQQfMaAft2zIgO9pW7QbT/cIdX8LSREUrElZeh6RFIh5DUGLZRLMD0zf5X2V36ntHLrF1mpNuDkMOhbgg9/6S8l4cFO/3LWedFe6sl1T8Y7woWJ10JrkhFs5dPkLv/F0xp/GZb3c7MOZicdecgD1ugl/+ZGqMHKlvx+f2sA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKs3CyvzKHLwq; Thu, 14 Aug 2025 11:43:13 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id B50C61A0CED; Thu, 14 Aug 2025 11:43:12 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S13; Thu, 14 Aug 2025 11:43:12 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 09/16] block: convert nr_requests to unsigned int Date: Thu, 14 Aug 2025 11:35:15 +0800 Message-Id: <20250814033522.770575-10-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S13 X-Coremail-Antispam: 1UD129KBjvdXoWrZw4DJFW7Cw18ur17CrWxJFb_yoW3ZrgEka 9FkFs2gw4fJF1Iqw4UtFs7tFyqgwn3JFZ7u3Z3JF1jqan8WanYqa1kGF9rGF42gFW7CF97 Xan5W39Yvr1YgjkaLaAFLSUrUUUUjb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUIcSsGvfJTRUUUbvAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k26cxKx2IYs7xG 6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2 IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28E F7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr 1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxkF7I0En4kS14v26r1q6r43MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY 6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17 CEb7AF67AKxVWUtVW8ZwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1I6r4UMIIF 0xvE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCw CI42IY6I8E87Iv67AKxVW8JVWxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsG vfC2KfnxnUUI43ZEXa7VUbPC7UUUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai This value represents the number of requests for elevator tags, or drivers tags if elevator is none. The max value for elevator tags is 2048, and in drivers at most 16 bits is used for tag. Signed-off-by: Yu Kuai --- include/linux/blkdev.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 95886b404b16..ad5087d5cade 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -541,7 +541,7 @@ struct request_queue { /* * queue settings */ - unsigned long nr_requests; /* Max # of requests */ + unsigned int nr_requests; /* Max # of requests */ =20 #ifdef CONFIG_BLK_INLINE_ENCRYPTION struct blk_crypto_profile *crypto_profile; --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 96D0C27935A; Thu, 14 Aug 2025 03:43:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143002; cv=none; b=L+vwiIZL3UCdRHAe04b48G79nEWo9iHXZWfbA0UBtIfvAz+SFHt1/UrkLsPKCLIx5DBcrzZ+UUC0whQ9mncM+2QncWpxnwHNZfGYxen354DV/5pWMDF9TpoGKCTt+I0v4Ih+ZTBFhBDFrmumNUdZ6KJOOrOYpiQ1mgZPPb7qqjU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143002; c=relaxed/simple; bh=tncOV2F2tmgK5wdW3Q9QnEkK6Fac19U7mrM7QyNy2H0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZSjUmPvvy0AEPPT860XQIDSlK0KFsweRWqUCnqZ+fs17zFjLOJcSQdHmr8I6fpFJkGcXDXTop4nvfxVwOfcNxciKPQfI31qVeWjdySr00rTvt/lGFhX5+jM3tegqu6QZ9E9MKo1AXJmYX2v3grYsyAErVcIj5GHZu0Z7D8E4f8k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKt4Rv0zYQtLx; Thu, 14 Aug 2025 11:43:14 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 38EB11A0CED; Thu, 14 Aug 2025 11:43:13 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S14; Thu, 14 Aug 2025 11:43:12 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 10/16] blk-mq-sched: unify elevators checking for async requests Date: Thu, 14 Aug 2025 11:35:16 +0800 Message-Id: <20250814033522.770575-11-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S14 X-Coremail-Antispam: 1UD129KBjvJXoWxXw4fWFyrWry5ZFW8uFWktFb_yoW5XF4Dpr ZxWay5GryUKF4kuFW8Z3W7Xr1rJ3ZxWry7Cr4rJw4fKF9I9FsruF1rtr4fXFWIvrWfAFW7 Zr1qqa43XrnY93DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0JUQFxUUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai bfq and mq-deadline consider sync writes as async requests and only resver tags for sync reads by async_depth, however, kyber doesn't consider sync writes as async requests. Consider the case there are lots of dirty pages, and user do fsync, in this case sched_tags can be exhausted by sync writes and sync reads can stuck waiting for tag. Hence let kyber follow what mq-deadline and bfq did, and unify async requests checking for all elevators. Signed-off-by: Yu Kuai --- block/bfq-iosched.c | 2 +- block/blk-mq-sched.h | 5 +++++ block/kyber-iosched.c | 2 +- block/mq-deadline.c | 2 +- 4 files changed, 8 insertions(+), 3 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index c0c398998aa1..de0dee255ccf 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -697,7 +697,7 @@ static void bfq_limit_depth(blk_opf_t opf, struct blk_m= q_alloc_data *data) unsigned int limit, act_idx; =20 /* Sync reads have full depth available */ - if (op_is_sync(opf) && !op_is_write(opf)) + if (blk_mq_sched_sync_request(opf)) limit =3D data->q->nr_requests; else limit =3D bfqd->async_depths[!!bfqd->wr_busy_queues][op_is_sync(opf)]; diff --git a/block/blk-mq-sched.h b/block/blk-mq-sched.h index 8e21a6b1415d..ae747f9053c7 100644 --- a/block/blk-mq-sched.h +++ b/block/blk-mq-sched.h @@ -103,4 +103,9 @@ static inline void blk_mq_set_min_shallow_depth(struct = request_queue *q, depth); } =20 +static inline bool blk_mq_sched_sync_request(blk_opf_t opf) +{ + return op_is_sync(opf) && !op_is_write(opf); +} + #endif diff --git a/block/kyber-iosched.c b/block/kyber-iosched.c index 49ae52aa20d9..b3df807044c3 100644 --- a/block/kyber-iosched.c +++ b/block/kyber-iosched.c @@ -543,7 +543,7 @@ static void kyber_limit_depth(blk_opf_t opf, struct blk= _mq_alloc_data *data) * We use the scheduler tags as per-hardware queue queueing tokens. * Async requests can be limited at this stage. */ - if (!op_is_sync(opf)) { + if (!blk_mq_sched_sync_request(opf)) { struct kyber_queue_data *kqd =3D data->q->elevator->elevator_data; =20 data->shallow_depth =3D kqd->async_depth; diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 578bc79c5654..1825173d82a6 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -496,7 +496,7 @@ static void dd_limit_depth(blk_opf_t opf, struct blk_mq= _alloc_data *data) struct deadline_data *dd =3D data->q->elevator->elevator_data; =20 /* Do not throttle synchronous reads. */ - if (op_is_sync(opf) && !op_is_write(opf)) + if (blk_mq_sched_sync_request(opf)) return; =20 /* --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 83A4D2609FD; Thu, 14 Aug 2025 03:43:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142997; cv=none; b=bKRsjxJwoMgBGQVHV4LRpe6O7vMR79G0d4FBbjYL+HhSHdBUReSXejvZFtCoXfvsmtX3JGr8S9NqzuQAOQKItJiJX09E38PIimCANNaw7UXiEUvyKc7922i+DrdNHTBigfXdUs8xphQ7uaSAtc0SBSrgHSARJIGUnXaMca0Tljw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142997; c=relaxed/simple; bh=5Z3kjnyI5tTIJkojp9LQTJHdQ/0EqGDz+asxv2zVa90=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=CxTdkC6rUy6JhMBEKvSm0fQcGXQGZYouYgeUMLm7SOHcXKfF4FKUMnn372B5OkkChqkEX2UOgLFoSgTqDBAlyld+Aqyfxeppf1AvlfKpPMD+eN0M2hZvvx3RzIi8hFVEsiyMdDEzMYGLhrCKSxIFBf0UlvHvTR+37oAZH4CK/o4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKt2qH8zKHLwq; Thu, 14 Aug 2025 11:43:14 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id A997C1A01A3; Thu, 14 Aug 2025 11:43:13 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S15; Thu, 14 Aug 2025 11:43:13 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 11/16] blk-mq: add a new queue sysfs attribute async_depth Date: Thu, 14 Aug 2025 11:35:17 +0800 Message-Id: <20250814033522.770575-12-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S15 X-Coremail-Antispam: 1UD129KBjvJXoW3JFy5KFWrCF4DKw4UCF1rWFg_yoW7Zr43pr 45Ja1Ykw10qF4xW34ftw13Xw43J3sagr1xJF4ay34akry7trs7XF1rCFyUXFZ2yrZ5CFsr Wr4DJFZ8uFy2q3DanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0JUQFxUUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai Add a new field async_depth to request_queue and related APIs, this is currently not used, following patches will convert elevators to use this instead of internal async_depth. Signed-off-by: Yu Kuai --- block/blk-core.c | 1 + block/blk-mq.c | 24 ++++++++++++++++++--- block/blk-sysfs.c | 47 ++++++++++++++++++++++++++++++++++++++++++ block/elevator.c | 1 + include/linux/blkdev.h | 1 + 5 files changed, 71 insertions(+), 3 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index fdac48aec5ef..443056be1c4c 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -463,6 +463,7 @@ struct request_queue *blk_alloc_queue(struct queue_limi= ts *lim, int node_id) fs_reclaim_release(GFP_KERNEL); =20 q->nr_requests =3D BLKDEV_DEFAULT_RQ; + q->async_depth =3D BLKDEV_DEFAULT_RQ; =20 return q; =20 diff --git a/block/blk-mq.c b/block/blk-mq.c index f1c11f591c27..699f7a2a36e5 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -520,6 +520,8 @@ static struct request *__blk_mq_alloc_requests(struct b= lk_mq_alloc_data *data) data->rq_flags |=3D RQF_USE_SCHED; if (ops->limit_depth) ops->limit_depth(data->cmd_flags, data); + else if (!blk_mq_sched_sync_request(data->cmd_flags)) + data->shallow_depth =3D q->async_depth; } } else { blk_mq_tag_busy(data->hctx); @@ -4606,6 +4608,7 @@ int blk_mq_init_allocated_queue(struct blk_mq_tag_set= *set, spin_lock_init(&q->requeue_lock); =20 q->nr_requests =3D set->queue_depth; + q->async_depth =3D set->queue_depth; =20 blk_mq_init_cpu_queues(q, set->nr_hw_queues); blk_mq_map_swqueue(q); @@ -4934,6 +4937,23 @@ static int blk_mq_sched_grow_tags(struct request_que= ue *q, unsigned int nr) return 0; } =20 +static void __blk_mq_update_nr_requests(struct request_queue *q, + unsigned int nr) +{ + unsigned int old_nr =3D q->nr_requests; + + q->nr_requests =3D nr; + if (!q->elevator) { + q->async_depth =3D nr; + return; + } + + /* keep the percentage of async requests */ + q->async_depth =3D max(q->async_depth * nr / old_nr, 1); + if (q->elevator->type->ops.depth_updated) + q->elevator->type->ops.depth_updated(q); +} + int blk_mq_update_nr_requests(struct request_queue *q, unsigned int nr) { struct blk_mq_tag_set *set =3D q->tag_set; @@ -4962,9 +4982,7 @@ int blk_mq_update_nr_requests(struct request_queue *q= , unsigned int nr) goto out; } =20 - q->nr_requests =3D nr; - if (q->elevator && q->elevator->type->ops.depth_updated) - q->elevator->type->ops.depth_updated(q); + __blk_mq_update_nr_requests(q, nr); =20 out: blk_mq_unquiesce_queue(q); diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index f3d08edcc34f..8f55730f06c6 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -99,6 +99,51 @@ queue_requests_store(struct gendisk *disk, const char *p= age, size_t count) return ret; } =20 +static ssize_t queue_async_depth_show(struct gendisk *disk, char *page) +{ + ssize_t ret; + + mutex_lock(&disk->queue->elevator_lock); + ret =3D queue_var_show(disk->queue->async_depth, page); + mutex_unlock(&disk->queue->elevator_lock); + return ret; +} + +static ssize_t +queue_async_depth_store(struct gendisk *disk, const char *page, size_t cou= nt) +{ + struct request_queue *q =3D disk->queue; + unsigned int memflags; + unsigned long nr; + int ret; + + if (!queue_is_mq(q)) + return -EINVAL; + + ret =3D queue_var_store(&nr, page, count); + if (ret < 0) + return ret; + + if (nr =3D=3D 0) + return -EINVAL; + + memflags =3D blk_mq_freeze_queue(q); + mutex_lock(&q->elevator_lock); + + if (q->elevator) { + q->async_depth =3D min(q->nr_requests, nr); + if (q->elevator->type->ops.depth_updated) + q->elevator->type->ops.depth_updated(q); + } else { + ret =3D -EINVAL; + } + + mutex_unlock(&q->elevator_lock); + blk_mq_unfreeze_queue(q, memflags); + + return ret; +} + static ssize_t queue_ra_show(struct gendisk *disk, char *page) { ssize_t ret; @@ -514,6 +559,7 @@ static struct queue_sysfs_entry _prefix##_entry =3D { \ } =20 QUEUE_RW_ENTRY(queue_requests, "nr_requests"); +QUEUE_RW_ENTRY(queue_async_depth, "async_depth"); QUEUE_RW_ENTRY(queue_ra, "read_ahead_kb"); QUEUE_LIM_RW_ENTRY(queue_max_sectors, "max_sectors_kb"); QUEUE_LIM_RO_ENTRY(queue_max_hw_sectors, "max_hw_sectors_kb"); @@ -736,6 +782,7 @@ static struct attribute *blk_mq_queue_attrs[] =3D { */ &elv_iosched_entry.attr, &queue_requests_entry.attr, + &queue_async_depth_entry.attr, #ifdef CONFIG_BLK_WBT &queue_wb_lat_entry.attr, #endif diff --git a/block/elevator.c b/block/elevator.c index f8a04f32cbcf..6bdb05d2500d 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -601,6 +601,7 @@ static int elevator_switch(struct request_queue *q, str= uct elv_change_ctx *ctx) blk_queue_flag_clear(QUEUE_FLAG_SQ_SCHED, q); q->elevator =3D NULL; q->nr_requests =3D q->tag_set->queue_depth; + q->async_depth =3D q->tag_set->queue_depth; } blk_add_trace_msg(q, "elv switch: %s", ctx->name); =20 diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index ad5087d5cade..2e33298fcc15 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -542,6 +542,7 @@ struct request_queue { * queue settings */ unsigned int nr_requests; /* Max # of requests */ + unsigned int async_depth; /* Max # of async requests */ =20 #ifdef CONFIG_BLK_INLINE_ENCRYPTION struct blk_crypto_profile *crypto_profile; --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1428A2798EF; Thu, 14 Aug 2025 03:43:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143003; cv=none; b=YRHgKpVIsay1dbPUIcjGCoN8JrGAQBEpPRTbA2amu50yofwk4MCPO/odPVpMf1cDUJTYf8c71hHD33jO64El+zhgO1Uiacxs0hISFkvH/0hmMTFPIXWBv3bDRrEl2YBQZIPiHPMgR0fZLiZq9iRJCpbnB6mpZfP3fdx4ASC15J8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143003; c=relaxed/simple; bh=x8b5vVzsArBONueQXu3yg1g74YDP24Pvc4pmd7gaF0I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gj4mnAYJ56CbONuFKvxrN/bMjmUDAzPKe5fFFIAq7NN1Lqp0AdM2oaTYZJno7HtWImOSS6UgyaWPBPaX8yozDMXDVh2LlsAO/94zX2TUSth92KLwVef6XXIh3Xhu1zAPfWiTP9k/ysho1HA+qEfVGPTLSfLbIv0jCslY9SVSIEI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKv3rp7zYQv8x; Thu, 14 Aug 2025 11:43:15 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 2AEBA1A01A3; Thu, 14 Aug 2025 11:43:14 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S16; Thu, 14 Aug 2025 11:43:13 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 12/16] kyber: covert to use request_queue->async_depth Date: Thu, 14 Aug 2025 11:35:18 +0800 Message-Id: <20250814033522.770575-13-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S16 X-Coremail-Antispam: 1UD129KBjvJXoWxXF17Xw43XFyDGr48XF1fWFg_yoW5KF1rpF ZYvanIyF1UtF4j9rykA3yxZw1fXr4agr13AFsYyw4xKryDt3Z3XF18tFy0qFWIvrWrArsF 9rWqvasrXrn7tF7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0JUQFxUUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai Instead of the internal async_depth, remove kqd->async_depth and related helpers, also remove limit_depth() method that is useless now. Signed-off-by: Yu Kuai --- block/kyber-iosched.c | 36 +++--------------------------------- 1 file changed, 3 insertions(+), 33 deletions(-) diff --git a/block/kyber-iosched.c b/block/kyber-iosched.c index b3df807044c3..2b38ee46140f 100644 --- a/block/kyber-iosched.c +++ b/block/kyber-iosched.c @@ -47,9 +47,8 @@ enum { * asynchronous requests, we reserve 25% of requests for synchronous * operations. */ - KYBER_ASYNC_PERCENT =3D 75, + KYBER_DEFAULT_ASYNC_PERCENT =3D 75, }; - /* * Maximum device-wide depth for each scheduling domain. * @@ -157,9 +156,6 @@ struct kyber_queue_data { */ struct sbitmap_queue domain_tokens[KYBER_NUM_DOMAINS]; =20 - /* Number of allowed async requests. */ - unsigned int async_depth; - struct kyber_cpu_latency __percpu *cpu_latency; =20 /* Timer for stats aggregation and adjusting domain tokens. */ @@ -413,6 +409,7 @@ static int kyber_init_sched(struct request_queue *q, st= ruct elevator_queue *eq) =20 eq->elevator_data =3D kqd; q->elevator =3D eq; + q->async_depth =3D q->nr_requests * KYBER_DEFAULT_ASYNC_PERCENT / 100; =20 return 0; } @@ -442,10 +439,7 @@ static void kyber_ctx_queue_init(struct kyber_ctx_queu= e *kcq) =20 static void kyber_depth_updated(struct request_queue *q) { - struct kyber_queue_data *kqd =3D q->elevator->elevator_data; - - kqd->async_depth =3D q->nr_requests * KYBER_ASYNC_PERCENT / 100U; - blk_mq_set_min_shallow_depth(q, kqd->async_depth); + blk_mq_set_min_shallow_depth(q, q->async_depth); } =20 static int kyber_init_hctx(struct blk_mq_hw_ctx *hctx, unsigned int hctx_i= dx) @@ -537,19 +531,6 @@ static void rq_clear_domain_token(struct kyber_queue_d= ata *kqd, } } =20 -static void kyber_limit_depth(blk_opf_t opf, struct blk_mq_alloc_data *dat= a) -{ - /* - * We use the scheduler tags as per-hardware queue queueing tokens. - * Async requests can be limited at this stage. - */ - if (!blk_mq_sched_sync_request(opf)) { - struct kyber_queue_data *kqd =3D data->q->elevator->elevator_data; - - data->shallow_depth =3D kqd->async_depth; - } -} - static bool kyber_bio_merge(struct request_queue *q, struct bio *bio, unsigned int nr_segs) { @@ -943,15 +924,6 @@ KYBER_DEBUGFS_DOMAIN_ATTRS(KYBER_DISCARD, discard) KYBER_DEBUGFS_DOMAIN_ATTRS(KYBER_OTHER, other) #undef KYBER_DEBUGFS_DOMAIN_ATTRS =20 -static int kyber_async_depth_show(void *data, struct seq_file *m) -{ - struct request_queue *q =3D data; - struct kyber_queue_data *kqd =3D q->elevator->elevator_data; - - seq_printf(m, "%u\n", kqd->async_depth); - return 0; -} - static int kyber_cur_domain_show(void *data, struct seq_file *m) { struct blk_mq_hw_ctx *hctx =3D data; @@ -977,7 +949,6 @@ static const struct blk_mq_debugfs_attr kyber_queue_deb= ugfs_attrs[] =3D { KYBER_QUEUE_DOMAIN_ATTRS(write), KYBER_QUEUE_DOMAIN_ATTRS(discard), KYBER_QUEUE_DOMAIN_ATTRS(other), - {"async_depth", 0400, kyber_async_depth_show}, {}, }; #undef KYBER_QUEUE_DOMAIN_ATTRS @@ -1003,7 +974,6 @@ static struct elevator_type kyber_sched =3D { .exit_sched =3D kyber_exit_sched, .init_hctx =3D kyber_init_hctx, .exit_hctx =3D kyber_exit_hctx, - .limit_depth =3D kyber_limit_depth, .bio_merge =3D kyber_bio_merge, .prepare_request =3D kyber_prepare_request, .insert_requests =3D kyber_insert_requests, --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1420D238C0D; Thu, 14 Aug 2025 03:43:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143003; cv=none; b=jgplzgqwceWqLsTMWkfH1t/Xj0Sa2APlZcsTvw8FrZzYGJshfRFDTODDVCSblCnLYqYZWUFq6ha9dZhYa1XanZ4QcGj2Dxp8gHyudsF2UYaepU0EYhEIRE7UwwGuMtCAZyvS61RK6HN4pRzdQ06ekAzK3AWyHo0Jw5yoQxMqq90= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143003; c=relaxed/simple; bh=cC14FkZGZjFZZFqnFpTKNL9aUyjZkBSlQU3VZfQUWqg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lDeg5eMLJsS7IUngFKI5lwoQO1KbTIdXILLxdVSY9QTKfP1mpMDCFS8l2xBVATPlINU7xUl1yGBk3tpgcJNtfCuR5lY80agUJ4LgDe9uiEqqkWW6XKrfbiTVlQ27EQ/VR1b6RLiQGnv77EO/PksZVbr6a1/kKphj8jwMiXuJZtU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=none smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKw0M5SzYQtwj; Thu, 14 Aug 2025 11:43:16 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id A3D8A1A1672; Thu, 14 Aug 2025 11:43:14 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S17; Thu, 14 Aug 2025 11:43:14 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 13/16] mq-deadline: covert to use request_queue->async_depth Date: Thu, 14 Aug 2025 11:35:19 +0800 Message-Id: <20250814033522.770575-14-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S17 X-Coremail-Antispam: 1UD129KBjvJXoW3JF1UJw47CF15JF1fCr48JFb_yoW7uFWUpF W3Janrtr1UtF4UurW8Aw43Zr1xWw4fC3sxKFyrKw4fGFn0yFsxtFyFyF1fZF93JrWfCw47 Kr1ktas5X347taDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0JUQFxUUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai In downstream kernel, we test with mq-deadline with many fio workloads, and we found a performance regression after commit 39823b47bbd4 ("block/mq-deadline: Fix the tag reservation code") with following test: [global] rw=3Drandread direct=3D1 ramp_time=3D1 ioengine=3Dlibaio iodepth=3D1024 numjobs=3D24 bs=3D1024k group_reporting=3D1 runtime=3D60 [job1] filename=3D/dev/sda Root cause is that mq-deadline now support configuring async_depth, although the default value is nr_request, however the minimal value is 1, hence min_shallow_depth is set to 1, causing wake_batch to be 1. For consequence, sbitmap_queue will be waken up after each IO instead of 8 IO. In this test case, sda is HDD and max_sectors is 128k, hence each submitted 1M io will be splited into 8 sequential 128k requests, however due to there are 24 jobs and total tags are exhausted, the 8 requests are unlikely to be dispatched sequentially, and changing wake_batch to 1 will make this much worse, accounting blktrace D stage, the percentage of sequential io is decreased from 8% to 0.8%. Fix this problem by converting to request_queue->async_depth, where min_shallow_depth is set each time async_depth is updated. Fixes: 39823b47bbd4 ("block/mq-deadline: Fix the tag reservation code") Signed-off-by: Yu Kuai --- block/mq-deadline.c | 42 +++--------------------------------------- 1 file changed, 3 insertions(+), 39 deletions(-) diff --git a/block/mq-deadline.c b/block/mq-deadline.c index 1825173d82a6..970e1541b7b7 100644 --- a/block/mq-deadline.c +++ b/block/mq-deadline.c @@ -98,7 +98,6 @@ struct deadline_data { int fifo_batch; int writes_starved; int front_merges; - u32 async_depth; int prio_aging_expire; =20 spinlock_t lock; @@ -487,32 +486,10 @@ static struct request *dd_dispatch_request(struct blk= _mq_hw_ctx *hctx) return rq; } =20 -/* - * Called by __blk_mq_alloc_request(). The shallow_depth value set by this - * function is used by __blk_mq_get_tag(). - */ -static void dd_limit_depth(blk_opf_t opf, struct blk_mq_alloc_data *data) -{ - struct deadline_data *dd =3D data->q->elevator->elevator_data; - - /* Do not throttle synchronous reads. */ - if (blk_mq_sched_sync_request(opf)) - return; - - /* - * Throttle asynchronous requests and writes such that these requests - * do not block the allocation of synchronous requests. - */ - data->shallow_depth =3D dd->async_depth; -} - -/* Called by blk_mq_update_nr_requests(). */ +/* Called by blk_mq_init_sched() and blk_mq_update_nr_requests(). */ static void dd_depth_updated(struct request_queue *q) { - struct deadline_data *dd =3D q->elevator->elevator_data; - - dd->async_depth =3D q->nr_requests; - blk_mq_set_min_shallow_depth(q, 1); + blk_mq_set_min_shallow_depth(q, q->async_depth); } =20 static void dd_exit_sched(struct elevator_queue *e) @@ -577,6 +554,7 @@ static int dd_init_sched(struct request_queue *q, struc= t elevator_queue *eq) blk_queue_flag_set(QUEUE_FLAG_SQ_SCHED, q); =20 q->elevator =3D eq; + q->async_depth =3D q->nr_requests; return 0; } =20 @@ -761,7 +739,6 @@ SHOW_JIFFIES(deadline_write_expire_show, dd->fifo_expir= e[DD_WRITE]); SHOW_JIFFIES(deadline_prio_aging_expire_show, dd->prio_aging_expire); SHOW_INT(deadline_writes_starved_show, dd->writes_starved); SHOW_INT(deadline_front_merges_show, dd->front_merges); -SHOW_INT(deadline_async_depth_show, dd->async_depth); SHOW_INT(deadline_fifo_batch_show, dd->fifo_batch); #undef SHOW_INT #undef SHOW_JIFFIES @@ -791,7 +768,6 @@ STORE_JIFFIES(deadline_write_expire_store, &dd->fifo_ex= pire[DD_WRITE], 0, INT_MA STORE_JIFFIES(deadline_prio_aging_expire_store, &dd->prio_aging_expire, 0,= INT_MAX); STORE_INT(deadline_writes_starved_store, &dd->writes_starved, INT_MIN, INT= _MAX); STORE_INT(deadline_front_merges_store, &dd->front_merges, 0, 1); -STORE_INT(deadline_async_depth_store, &dd->async_depth, 1, INT_MAX); STORE_INT(deadline_fifo_batch_store, &dd->fifo_batch, 0, INT_MAX); #undef STORE_FUNCTION #undef STORE_INT @@ -805,7 +781,6 @@ static const struct elv_fs_entry deadline_attrs[] =3D { DD_ATTR(write_expire), DD_ATTR(writes_starved), DD_ATTR(front_merges), - DD_ATTR(async_depth), DD_ATTR(fifo_batch), DD_ATTR(prio_aging_expire), __ATTR_NULL @@ -892,15 +867,6 @@ static int deadline_starved_show(void *data, struct se= q_file *m) return 0; } =20 -static int dd_async_depth_show(void *data, struct seq_file *m) -{ - struct request_queue *q =3D data; - struct deadline_data *dd =3D q->elevator->elevator_data; - - seq_printf(m, "%u\n", dd->async_depth); - return 0; -} - static int dd_queued_show(void *data, struct seq_file *m) { struct request_queue *q =3D data; @@ -1010,7 +976,6 @@ static const struct blk_mq_debugfs_attr deadline_queue= _debugfs_attrs[] =3D { DEADLINE_NEXT_RQ_ATTR(write2), {"batching", 0400, deadline_batching_show}, {"starved", 0400, deadline_starved_show}, - {"async_depth", 0400, dd_async_depth_show}, {"dispatch0", 0400, .seq_ops =3D &deadline_dispatch0_seq_ops}, {"dispatch1", 0400, .seq_ops =3D &deadline_dispatch1_seq_ops}, {"dispatch2", 0400, .seq_ops =3D &deadline_dispatch2_seq_ops}, @@ -1024,7 +989,6 @@ static const struct blk_mq_debugfs_attr deadline_queue= _debugfs_attrs[] =3D { static struct elevator_type mq_deadline =3D { .ops =3D { .depth_updated =3D dd_depth_updated, - .limit_depth =3D dd_limit_depth, .insert_requests =3D dd_insert_requests, .dispatch_request =3D dd_dispatch_request, .prepare_request =3D dd_prepare_request, --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C434B26CE0A; Thu, 14 Aug 2025 03:43:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142998; cv=none; b=nSkb5xShdyXhjVE8IxPTQRX9qlIZxtZtC65sbV6ZI4sfHGij87gko7TaCkK8UFGDFOhXPRcrkr41kokoxyvkH/mEunen+XD6pfH2AAHcI3GuOacgY7h4pQ7RBrl859OPY/rwOZftoxf5UdmeCjYs+hR+zeKnSyM2Ic0Hejai7pw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142998; c=relaxed/simple; bh=24T5+g8WMOUYYSAoco/LftJ3y+twWAskhfuDyz4/5jI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dW2l2kEjpTyoIUMx1J5JcLI+nP1dvcXzQ+EbnXtgs4P4D7qKwi00+a80Efr0Q9vr5i/bBSNB8BPEZpQ9lnY++AauOd+z+WGf2tRjo9Xng9TYIBnWKUjHxJyPab+VXeMhunBKSeF8kN0Kc9ZNvmE994MnH2F+DAo7zwiVEHfbQJs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKv63ywzKHMhx; Thu, 14 Aug 2025 11:43:15 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 24C601A01A3; Thu, 14 Aug 2025 11:43:15 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S18; Thu, 14 Aug 2025 11:43:14 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 14/16] block, bfq: convert to use request_queue->async_depth Date: Thu, 14 Aug 2025 11:35:20 +0800 Message-Id: <20250814033522.770575-15-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S18 X-Coremail-Antispam: 1UD129KBjvJXoWxJF48trWDGF4UtFW8Cw45KFg_yoW5Ar4xp3 y3tanxKr18JF429w1rAw1a9ryrKwn5uryfJF1ftryxJrZ8ZFn7ZF1YqFyrXr97Xr93Aw4U XrWDZa95W3WIqF7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0JUQFxUUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai The default limits is unchanged, and user can configure async_depth now. Signed-off-by: Yu Kuai --- block/bfq-iosched.c | 43 +++++++++++++++++-------------------------- 1 file changed, 17 insertions(+), 26 deletions(-) diff --git a/block/bfq-iosched.c b/block/bfq-iosched.c index de0dee255ccf..46520e50f584 100644 --- a/block/bfq-iosched.c +++ b/block/bfq-iosched.c @@ -7112,39 +7112,29 @@ void bfq_put_async_queues(struct bfq_data *bfqd, st= ruct bfq_group *bfqg) static void bfq_depth_updated(struct request_queue *q) { struct bfq_data *bfqd =3D q->elevator->elevator_data; - unsigned int nr_requests =3D q->nr_requests; + unsigned int async_depth =3D q->async_depth; =20 /* - * In-word depths if no bfq_queue is being weight-raised: - * leaving 25% of tags only for sync reads. + * By default: + * - sync reads are not limited + * If bfqq is not being weight-raised: + * - sync writes are limited to 75%(async depth default value) + * - async IO are limited to 50% + * If bfqq is being weight-raised: + * - sync writes are limited to ~37% + * - async IO are limited to ~18 * - * In next formulas, right-shift the value - * (1U<sb.shift), instead of computing directly - * (1U<<(bt->sb.shift - something)), to be robust against - * any possible value of bt->sb.shift, without having to - * limit 'something'. + * If request_queue->async_depth is updated by user, all limit are + * updated relatively. */ - /* no more than 50% of tags for async I/O */ - bfqd->async_depths[0][0] =3D max(nr_requests >> 1, 1U); - /* - * no more than 75% of tags for sync writes (25% extra tags - * w.r.t. async I/O, to prevent async I/O from starving sync - * writes) - */ - bfqd->async_depths[0][1] =3D max((nr_requests * 3) >> 2, 1U); + bfqd->async_depths[0][1] =3D async_depth; + bfqd->async_depths[0][0] =3D max(async_depth * 2 / 3, 1U); + bfqd->async_depths[1][1] =3D max(async_depth >> 1, 1U); + bfqd->async_depths[1][0] =3D max(async_depth >> 2, 1U); =20 /* - * In-word depths in case some bfq_queue is being weight- - * raised: leaving ~63% of tags for sync reads. This is the - * highest percentage for which, in our tests, application - * start-up times didn't suffer from any regression due to tag - * shortage. + * Due to cgroup qos, the allowed request for bfqq might be 1 */ - /* no more than ~18% of tags for async I/O */ - bfqd->async_depths[1][0] =3D max((nr_requests * 3) >> 4, 1U); - /* no more than ~37% of tags for sync writes (~20% extra tags) */ - bfqd->async_depths[1][1] =3D max((nr_requests * 6) >> 4, 1U); - blk_mq_set_min_shallow_depth(q, 1); } =20 @@ -7364,6 +7354,7 @@ static int bfq_init_queue(struct request_queue *q, st= ruct elevator_queue *eq) blk_queue_flag_set(QUEUE_FLAG_DISABLE_WBT_DEF, q); wbt_disable_default(q->disk); blk_stat_enable_accounting(q); + q->async_depth =3D (q->nr_requests * 3) >> 2; =20 return 0; =20 --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5615270572; Thu, 14 Aug 2025 03:43:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143001; cv=none; b=IIT9JB/THUAs1TQm47lTI5YnjPvL3hSzlmiiA/YJaw0dIKj/Dl6ZL/WTwI8HJlHApUj6GJegTGPinKC2ZnXvGXa2YA5U8I6hy77gvlYmy+ZjfH7ZfES15x2H4N0p5tWgGKv1Dgbl/UCobceh9wlxSnsTqNl3lX88UE56ShAM4Ys= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755143001; c=relaxed/simple; bh=SJDQlr8kYnUVXlqjG3XTlvRhDl2Ax+wgpIYELK54IoE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=k1MXgQBbBt0mEGTeD0ZgDv2TGErxtyN9zFYi2fhBwjuxmq1JNF5yl2gTARyZUdYo2BSCYwqObrjqcEG+ec53HrKjFmnXVZOX44O97dnBky24cueLbaQ/GTCwzCHbFx1Zi4T116oPh013ME9iNHu1nqP3j+KcM1hJEzJpkedJzVc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKw2FbVzKHMST; Thu, 14 Aug 2025 11:43:16 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 963951A1679; Thu, 14 Aug 2025 11:43:15 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S19; Thu, 14 Aug 2025 11:43:15 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 15/16] blk-mq: fix stale nr_requests documentation Date: Thu, 14 Aug 2025 11:35:21 +0800 Message-Id: <20250814033522.770575-16-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S19 X-Coremail-Antispam: 1UD129KBjvJXoW7KFWDCryfKr15Zw1xKFWfZrb_yoW8Xw1Dp3 yft39Fgwn5Zw18Wr10yay8tF13Aa95Aw43Jr4DKF1rtr98Awn29Fs2qr1rXF4xZrZ7AFWU urZ29r98Aa1qva7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVWUtVW8ZwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVW8JVW5JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0JUQFxUUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai The nr_requests documentation is still the removed single queue, remove it and update to current blk-mq. Signed-off-by: Yu Kuai --- Documentation/ABI/stable/sysfs-block | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stabl= e/sysfs-block index 0ddffc9133d0..0ed10aeff86b 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -603,16 +603,10 @@ Date: July 2003 Contact: linux-block@vger.kernel.org Description: [RW] This controls how many requests may be allocated in the - block layer for read or write requests. Note that the total - allocated number may be twice this amount, since it applies only - to reads or writes (not the accumulated sum). - - To avoid priority inversion through request starvation, a - request queue maintains a separate request pool per each cgroup - when CONFIG_BLK_CGROUP is enabled, and this parameter applies to - each such per-block-cgroup request pool. IOW, if there are N - block cgroups, each request queue may have up to N request - pools, each independently regulated by nr_requests. + block layer. Noted this value only represents the quantity for a + single blk_mq_tags instance. The actual number for the entire + device depends on the hardware queue count, whether elevator is + enabled, and whether tags are shared. =20 =20 What: /sys/block//queue/nr_zones --=20 2.39.2 From nobody Sat Oct 4 17:31:13 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D7CBF270EBB; Thu, 14 Aug 2025 03:43:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142999; cv=none; b=lXyGEQSwsDUuyLTdn+VIOgHN+vWJZgdgD6b17aSgWGSutNAa3yxLtgVNA6ApuhAVOG5dYJ095PyMPR2FvGwJIwJyRTvS9gi3v2EPLWV7mgR14IvEBjQjyKE0lmgH4/vjfHvkcydvWK3U2yd1KN14zB1mdK8YxOWPyXKFyPhGwio= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755142999; c=relaxed/simple; bh=EknLAK2gLRyPv6Yw7SLp+eYJNTIbHKFqWvdA/PDvUlU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=k85lfF7u75m9nTjvrI5rLCSw/fv/Ai1lCoe5Rw+taf3kEJwmjNgqWcBaAleQO3gjMtUh4BfUljxVeKX2JJyv9bfI1J4LiCpaL6yZKChWv6S8B3k3pS/jSMfQS08MREE06gQJY26zIRbVpjg29i20RQ0skoaY/TjWmb4J1Yq946Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c2WKw5rF7zKHMST; Thu, 14 Aug 2025 11:43:16 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 1BCCE1A0BDE; Thu, 14 Aug 2025 11:43:16 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCnIxRKW51ogs93Dg--.15627S20; Thu, 14 Aug 2025 11:43:15 +0800 (CST) From: Yu Kuai To: axboe@kernel.dk, yukuai3@huawei.com, bvanassche@acm.org, nilay@linux.ibm.com, hare@suse.de, ming.lei@redhat.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH 16/16] blk-mq: add documentation for new queue attribute async_dpeth Date: Thu, 14 Aug 2025 11:35:22 +0800 Message-Id: <20250814033522.770575-17-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250814033522.770575-1-yukuai1@huaweicloud.com> References: <20250814033522.770575-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCnIxRKW51ogs93Dg--.15627S20 X-Coremail-Antispam: 1UD129KBjvdXoWruF1rAFWrCryktr4fuw47CFg_yoWDXFbEgF WDKFnYg3ykAF1YvFWjyF4kAF12g3y5KFy8G3WkAry5Zry3taySka95XrZ8CFZrXFs29rn3 uws5XrWUKrnaqjkaLaAFLSUrUUUUjb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUIcSsGvfJTRUUUbvAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k26cxKx2IYs7xG 6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2 IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28E F7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr 1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxkF7I0En4kS14v26r1q6r43MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY 6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17 CEb7AF67AKxVWUtVW8ZwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r4j6ryUMIIF 0xvE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCw CI42IY6I8E87Iv67AKxVW8JVWxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsG vfC2KfnxnUUI43ZEXa7VUbPC7UUUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai Signed-off-by: Yu Kuai --- Documentation/ABI/stable/sysfs-block | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stabl= e/sysfs-block index 0ed10aeff86b..09b9b3db9a1f 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -609,6 +609,16 @@ Description: enabled, and whether tags are shared. =20 =20 +What: /sys/block//queue/async_depth +Date: August 2025 +Contact: linux-block@vger.kernel.org +Description: + [RW] This controls how many async requests may be allocated in the + block layer. If elevator is none, then this value is nr_requests. + By default, this value is 75% of nr_requests for bfq and kyber, + abd nr_requests for mq-deadline. + + What: /sys/block//queue/nr_zones Date: November 2018 Contact: Damien Le Moal --=20 2.39.2