From nobody Fri Oct 3 21:00:02 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55B9A2C0F89; Mon, 25 Aug 2025 09:45:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115132; cv=none; b=k154C41kvzT+2Z4HN625HC5wrhWLJlFZR2MfWxstttmUHUfVBKesClwRfRTllD4wmWLIrAmT++21AA3yCwtF/CjZYl67odsQ8fuGhXkyfPJ7Pj34WSQFqJKx0mdJ8zoFx6shSiouD+7a4xdHLgkyRdPx1vdj/oTP+2mxOlIYjq4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115132; c=relaxed/simple; bh=V8UqlYonaFpjyDrOre12ZnZAcqcI7ueaBnFbHt1JfFU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mTF63E+FXIPCigSKth4ZbU2tfvTmZ11/E8OlJ/N/q7dX/qi+1nGSI2SGNUjwacxQ3ipVVU+8KBZIrgDHz+5eR9UUfvW2XhJo+pd8y/9O/w4EJejEjI0vZ4m8Vgs5WF0/rNzxs9Ea53dJavyu8c45Hjgm3s5DwV0MO4AHgKyKkxc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c9Qrm10FgzKHNKY; Mon, 25 Aug 2025 17:45:28 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id B47261A1C37; Mon, 25 Aug 2025 17:45:27 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAXYIy1MKxonJnxAA--.44975S5; Mon, 25 Aug 2025 17:45:27 +0800 (CST) From: Yu Kuai To: hch@infradead.org, colyli@kernel.org, hare@suse.de, tieren@fnnas.com, axboe@kernel.dk, tj@kernel.org, josef@toxicpanda.com, song@kernel.org, yukuai3@huawei.com, akpm@linux-foundation.org, neil@brown.name Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH RFC 1/7] block: export helper bio_submit_split() Date: Mon, 25 Aug 2025 17:36:54 +0800 Message-Id: <20250825093700.3731633-2-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250825093700.3731633-1-yukuai1@huaweicloud.com> References: <20250825093700.3731633-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgAXYIy1MKxonJnxAA--.44975S5 X-Coremail-Antispam: 1UD129KBjvJXoWxCF48Kr1rGF47Kry3Kw1Dtrb_yoWrtF1rpr 4j9wn3JrZ5JFs2gwnxXa1UK3Zakay5ZrW5C3yxX39rArnxtwnrKF4xWr1rZFyFkrW5C345 Jw1vkFW5C34UCFDanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmY14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r4j6ryUM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUGVWUXwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCY1x0262kKe7AKxVW8ZVWrXwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWU JVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67 kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY 6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42 IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIev Ja73UjIFyTuYvjTRAb10DUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai No functional changes are intended, some drivers like mdraid will split bio by internal processing, prepare to unify bio split codes. Signed-off-by: Yu Kuai --- block/blk-merge.c | 60 +++++++++++++++++++++++++++++---------------- include/linux/bio.h | 2 ++ 2 files changed, 41 insertions(+), 21 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index 70d704615be5..c45d5e43e172 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -104,35 +104,49 @@ static unsigned int bio_allowed_max_sectors(const str= uct queue_limits *lim) return round_down(UINT_MAX, lim->logical_block_size) >> SECTOR_SHIFT; } =20 -static struct bio *bio_submit_split(struct bio *bio, int split_sectors) +/** + * bio_submit_split - Submit a bio, splitting it at a designated sector + * @bio: the original bio to be submitted and split + * @split_sectors: the sector count (from the start of @bio) at which to s= plit + * @bs: the bio set used for allocating the new split bio + * + * The original bio is modified to contain the remaining sectors and submi= tted. + * The caller is responsible for submitting the returned bio. + * + * If succeed, the newly allocated bio representing the initial part will = be + * returned, on failure NULL will be returned and original bio will fail. + */ +struct bio *bio_submit_split(struct bio *bio, int split_sectors, + struct bio_set *bs) { + struct bio *split; + if (unlikely(split_sectors < 0)) goto error; =20 - if (split_sectors) { - struct bio *split; + if (!split_sectors) + return bio; =20 - split =3D bio_split(bio, split_sectors, GFP_NOIO, - &bio->bi_bdev->bd_disk->bio_split); - if (IS_ERR(split)) { - split_sectors =3D PTR_ERR(split); - goto error; - } - split->bi_opf |=3D REQ_NOMERGE; - blkcg_bio_issue_init(split); - bio_chain(split, bio); - trace_block_split(split, bio->bi_iter.bi_sector); - WARN_ON_ONCE(bio_zone_write_plugging(bio)); - submit_bio_noacct(bio); - return split; + split =3D bio_split(bio, split_sectors, GFP_NOIO, bs); + if (IS_ERR(split)) { + split_sectors =3D PTR_ERR(split); + goto error; } =20 - return bio; + split->bi_opf |=3D REQ_NOMERGE; + blkcg_bio_issue_init(split); + bio_chain(split, bio); + trace_block_split(split, bio->bi_iter.bi_sector); + WARN_ON_ONCE(bio_zone_write_plugging(bio)); + submit_bio_noacct(bio); + return split; + error: bio->bi_status =3D errno_to_blk_status(split_sectors); bio_endio(bio); return NULL; } +EXPORT_SYMBOL_GPL(bio_submit_split); =20 struct bio *bio_split_discard(struct bio *bio, const struct queue_limits *= lim, unsigned *nsegs) @@ -167,7 +181,8 @@ struct bio *bio_split_discard(struct bio *bio, const st= ruct queue_limits *lim, if (split_sectors > tmp) split_sectors -=3D tmp; =20 - return bio_submit_split(bio, split_sectors); + return bio_submit_split(bio, split_sectors, + &bio->bi_bdev->bd_disk->bio_split); } =20 static inline unsigned int blk_boundary_sectors(const struct queue_limits = *lim, @@ -357,7 +372,8 @@ struct bio *bio_split_rw(struct bio *bio, const struct = queue_limits *lim, { return bio_submit_split(bio, bio_split_rw_at(bio, lim, nr_segs, - get_max_io_size(bio, lim) << SECTOR_SHIFT)); + get_max_io_size(bio, lim) << SECTOR_SHIFT), + &bio->bi_bdev->bd_disk->bio_split); } =20 /* @@ -376,7 +392,8 @@ struct bio *bio_split_zone_append(struct bio *bio, lim->max_zone_append_sectors << SECTOR_SHIFT); if (WARN_ON_ONCE(split_sectors > 0)) split_sectors =3D -EINVAL; - return bio_submit_split(bio, split_sectors); + return bio_submit_split(bio, split_sectors, + &bio->bi_bdev->bd_disk->bio_split); } =20 struct bio *bio_split_write_zeroes(struct bio *bio, @@ -396,7 +413,8 @@ struct bio *bio_split_write_zeroes(struct bio *bio, return bio; if (bio_sectors(bio) <=3D max_sectors) return bio; - return bio_submit_split(bio, max_sectors); + return bio_submit_split(bio, max_sectors, + &bio->bi_bdev->bd_disk->bio_split); } =20 /** diff --git a/include/linux/bio.h b/include/linux/bio.h index 46ffac5caab7..2233261be5e8 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -324,6 +324,8 @@ extern struct bio *bio_split(struct bio *bio, int secto= rs, gfp_t gfp, struct bio_set *bs); int bio_split_rw_at(struct bio *bio, const struct queue_limits *lim, unsigned *segs, unsigned max_bytes); +struct bio *bio_submit_split(struct bio *bio, int split_sectors, + struct bio_set *bs); =20 /** * bio_next_split - get next @sectors from a bio, splitting if necessary --=20 2.39.2 From nobody Fri Oct 3 21:00:02 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9125A2C15A5; Mon, 25 Aug 2025 09:45:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115132; cv=none; b=LurdT1rc9T2cDR5pRSgM1V2I6eYKmGumMaEJGXrQ914xcpBJpV73rsSLjh8WRYGzKJGoMTKp0Srp/d419Qq6f0jCDSgPT/aI1rNffDuFDSO+GeEGBiQY+aYKElPITCoZRHimMzzFXcqPZ0zc6KjbjBGNzhh32D/sWmGAhOF3+uc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115132; c=relaxed/simple; bh=fJYF4Nf5ATfwWZdatndbzlCs3ZbeJ7Yr612QdgFXfLY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=T3U3mmcSQ8uhQazYkaXdM3pQERP+vhpGdvyp9odbzQ3A7ZuMqmzrVPmYVxP18Rz6PsvynszY5r7OwiwaJqvr3ktKjFFuDNIDEL1c5IDslePtV6nN2+KTfwJx0UzcU8DrO/DEspy3Yy8Ur/Y2SEXsr+JCrZFeXDVfmedz8o0b2lY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c9Qrn6QTYzYQvgT; Mon, 25 Aug 2025 17:45:29 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 6DF0E1A1C47; Mon, 25 Aug 2025 17:45:28 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAXYIy1MKxonJnxAA--.44975S6; Mon, 25 Aug 2025 17:45:28 +0800 (CST) From: Yu Kuai To: hch@infradead.org, colyli@kernel.org, hare@suse.de, tieren@fnnas.com, axboe@kernel.dk, tj@kernel.org, josef@toxicpanda.com, song@kernel.org, yukuai3@huawei.com, akpm@linux-foundation.org, neil@brown.name Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH RFC 2/7] md/raid0: convert raid0_handle_discard() to use bio_submit_split() Date: Mon, 25 Aug 2025 17:36:55 +0800 Message-Id: <20250825093700.3731633-3-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250825093700.3731633-1-yukuai1@huaweicloud.com> References: <20250825093700.3731633-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgAXYIy1MKxonJnxAA--.44975S6 X-Coremail-Antispam: 1UD129KBjvJXoW7AFWUury8Aw4rJw1xCF45ZFb_yoW8Ar15p3 9xWa4avrW8Grs09wsrAa42gas5Ja48WrW7GFZrJ393JF98Zryqkr45Kry8XFyUCryIka4D J3WkAasxCryDJrUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmF14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jryl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUGVWUXwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCY1x0262kKe7AKxVW8ZVWrXwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWU JVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67 kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY 6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42 IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2 KfnxnUUI43ZEXa7sRREfO5UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai On the one hand unify bio split code, prepare to fix disordered split IO; On the other hand fix missing blkcg_bio_issue_init() and trace_block_split() for split IO. Noted raid0_make_request() already fix disordered split IO by 319ff40a5427 ("md/raid0: Fix performance regression for large sequential writes"), by convert bio to underlying disks before submit_bio_noacct(), with the respect md_submit_bio() already split by sectors, and raid0_make_request() will split at most once for unaligned IO. This is a bit hacky and we'll convert this to solution in general later. Signed-off-by: Yu Kuai --- drivers/md/raid0.c | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c index f1d8811a542a..19b5faf238b7 100644 --- a/drivers/md/raid0.c +++ b/drivers/md/raid0.c @@ -463,21 +463,17 @@ static void raid0_handle_discard(struct mddev *mddev,= struct bio *bio) zone =3D find_zone(conf, &start); =20 if (bio_end_sector(bio) > zone->zone_end) { - struct bio *split =3D bio_split(bio, - zone->zone_end - bio->bi_iter.bi_sector, GFP_NOIO, - &mddev->bio_set); - - if (IS_ERR(split)) { - bio->bi_status =3D errno_to_blk_status(PTR_ERR(split)); - bio_endio(bio); + bio =3D bio_submit_split(bio, + zone->zone_end - bio->bi_iter.bi_sector, + &mddev->bio_set); + if (!bio) return; - } - bio_chain(split, bio); - submit_bio_noacct(bio); - bio =3D split; + + bio->bi_opf &=3D ~REQ_NOMERGE; end =3D zone->zone_end; - } else + } else { end =3D bio_end_sector(bio); + } =20 orig_end =3D end; if (zone !=3D conf->strip_zone) --=20 2.39.2 From nobody Fri Oct 3 21:00:02 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DFF42C3254; Mon, 25 Aug 2025 09:45:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115133; cv=none; b=I4WoXg160p9vQpW0HAffgjy7LKChkoBU2zzBvBy6I8mlrhaHRAxSBy079Jw4UAJAjq0yIf1mB5QfoR7s+u38Xyd6vkvblJNbFlDPemxiZ0+1u2fWESmYTIAqpQHB7DGFjnx5t3EYu0wzCg3uxOpWE6cLY+CZoJS2Tw479LRj0Y4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115133; c=relaxed/simple; bh=X22YaUjiEkOMtkXKh2Lb2TGbzXLathpEPi3Ls3O9JP4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QKzJ3cr0/fiblewvfpcG4YzRT1dIHZt0mr/uspRdJmaZ8Jf5d3cCLP/nbZ1ys1XSsd/1n4Q1RBnsbLBipGtpHwhq9cBlafW0OV4Ykvswjt9kIlteyzuYaTBClD6TmRiKRn4NBDJnpLoCcELwTycH+w2Or26jL8y2QogBXyn2Idc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=none smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c9Qrp4PdqzYQvj9; Mon, 25 Aug 2025 17:45:30 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 24BE11A1088; Mon, 25 Aug 2025 17:45:29 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAXYIy1MKxonJnxAA--.44975S7; Mon, 25 Aug 2025 17:45:28 +0800 (CST) From: Yu Kuai To: hch@infradead.org, colyli@kernel.org, hare@suse.de, tieren@fnnas.com, axboe@kernel.dk, tj@kernel.org, josef@toxicpanda.com, song@kernel.org, yukuai3@huawei.com, akpm@linux-foundation.org, neil@brown.name Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH RFC 3/7] md/raid1: convert to use bio_submit_split() Date: Mon, 25 Aug 2025 17:36:56 +0800 Message-Id: <20250825093700.3731633-4-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250825093700.3731633-1-yukuai1@huaweicloud.com> References: <20250825093700.3731633-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgAXYIy1MKxonJnxAA--.44975S7 X-Coremail-Antispam: 1UD129KBjvJXoWxGr4DuFWUKw48Zr15tr4DCFg_yoW5Cr1fpr 4Yga1IgrWUJFZYgws8Jayq9as5AF1qgrW7ArWxJ3s7WFnFv3sxKF4UWryFqryruFy5u3sr Aw1kAr4DCa17tFUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmF14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JrWl82xGYIkIc2 x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUGVWUXwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2 xKxwCY1x0262kKe7AKxVW8ZVWrXwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWU JVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67 kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI42IY 6xIIjxv20xvEc7CjxVAFwI0_Cr0_Gr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42 IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsGvfC2 KfnxnUUI43ZEXa7VUUbAw7UUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai On the one hand unify bio split code, prepare to fix disordered split IO; On the other hand fix missing blkcg_bio_issue_init() and trace_block_split() for split IO. Signed-off-by: Yu Kuai --- drivers/md/raid1.c | 35 ++++++++++++++--------------------- 1 file changed, 14 insertions(+), 21 deletions(-) diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 408c26398321..95196c8749f9 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -1317,7 +1317,7 @@ static void raid1_read_request(struct mddev *mddev, s= truct bio *bio, struct raid1_info *mirror; struct bio *read_bio; int max_sectors; - int rdisk, error; + int rdisk; bool r1bio_existed =3D !!r1_bio; =20 /* @@ -1376,16 +1376,13 @@ static void raid1_read_request(struct mddev *mddev,= struct bio *bio, } =20 if (max_sectors < bio_sectors(bio)) { - struct bio *split =3D bio_split(bio, max_sectors, - gfp, &conf->bio_split); - - if (IS_ERR(split)) { - error =3D PTR_ERR(split); + bio =3D bio_submit_split(bio, max_sectors,&conf->bio_split); + if (!bio) { + set_bit(R1BIO_Returned, &r1_bio->state); goto err_handle; } - bio_chain(split, bio); - submit_bio_noacct(bio); - bio =3D split; + + bio->bi_opf &=3D ~REQ_NOMERGE; r1_bio->master_bio =3D bio; r1_bio->sectors =3D max_sectors; } @@ -1413,7 +1410,6 @@ static void raid1_read_request(struct mddev *mddev, s= truct bio *bio, =20 err_handle: atomic_dec(&mirror->rdev->nr_pending); - bio->bi_status =3D errno_to_blk_status(error); set_bit(R1BIO_Uptodate, &r1_bio->state); raid_end_bio_io(r1_bio); } @@ -1457,7 +1453,7 @@ static void raid1_write_request(struct mddev *mddev, = struct bio *bio, { struct r1conf *conf =3D mddev->private; struct r1bio *r1_bio; - int i, disks, k, error; + int i, disks, k; unsigned long flags; int first_clone; int max_sectors; @@ -1562,7 +1558,8 @@ static void raid1_write_request(struct mddev *mddev, = struct bio *bio, * the benefit. */ if (bio->bi_opf & REQ_ATOMIC) { - error =3D -EIO; + bio->bi_status =3D + errno_to_blk_status(-EIO); goto err_handle; } =20 @@ -1584,16 +1581,13 @@ static void raid1_write_request(struct mddev *mddev= , struct bio *bio, max_sectors =3D min_t(int, max_sectors, BIO_MAX_VECS * (PAGE_SIZE >> 9)); if (max_sectors < bio_sectors(bio)) { - struct bio *split =3D bio_split(bio, max_sectors, - GFP_NOIO, &conf->bio_split); - - if (IS_ERR(split)) { - error =3D PTR_ERR(split); + bio =3D bio_submit_split(bio, max_sectors, &conf->bio_split); + if (!bio) { + set_bit(R1BIO_Returned, &r1_bio->state); goto err_handle; } - bio_chain(split, bio); - submit_bio_noacct(bio); - bio =3D split; + + bio->bi_opf &=3D ~REQ_NOMERGE; r1_bio->master_bio =3D bio; r1_bio->sectors =3D max_sectors; } @@ -1683,7 +1677,6 @@ static void raid1_write_request(struct mddev *mddev, = struct bio *bio, } } =20 - bio->bi_status =3D errno_to_blk_status(error); set_bit(R1BIO_Uptodate, &r1_bio->state); raid_end_bio_io(r1_bio); } --=20 2.39.2 From nobody Fri Oct 3 21:00:02 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22EA82C0F7E; Mon, 25 Aug 2025 09:45:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115134; cv=none; b=tGeZ+zmjCVaZafnqRYJfMgFZI06vplGH2cbaB0CHrxEoe2GquweBbL/MvH/2Mz4i9W4jI3Yo5eny9fV6zaowsFUgAiO8DYWU6bqPjirqVr/FSEw7polNhnTD9whqz/HrDfXMQ4xx0+lab5npGv/dmXXaJ6CorPhMlWhfHlAWWpg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115134; c=relaxed/simple; bh=+Zd0GaFHRFBuWMcYQJ9Xfws2YiVhvqbjue6U79Groy0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AWVMjumsrjsKrw4IpTtBvFxUgV1EJwziTgGBA6wzYwsE4n55QV+5mt0keIKsC0H8M5vCDzGwnj47RCYcGjTnRxl5US4Am9iCD3npDN4kKk2iGp/ouvroj1xWQqxUrWr28ZV3JFTGtrL7nXqtJXbsCGimBoEkCAmfJTHxJ6d7aag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c9Qrp1pp2zKHNL2; Mon, 25 Aug 2025 17:45:30 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id CE8761A1C95; Mon, 25 Aug 2025 17:45:29 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAXYIy1MKxonJnxAA--.44975S8; Mon, 25 Aug 2025 17:45:29 +0800 (CST) From: Yu Kuai To: hch@infradead.org, colyli@kernel.org, hare@suse.de, tieren@fnnas.com, axboe@kernel.dk, tj@kernel.org, josef@toxicpanda.com, song@kernel.org, yukuai3@huawei.com, akpm@linux-foundation.org, neil@brown.name Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH RFC 4/7] md/raid10: convert read/write to use bio_submit_split() Date: Mon, 25 Aug 2025 17:36:57 +0800 Message-Id: <20250825093700.3731633-5-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250825093700.3731633-1-yukuai1@huaweicloud.com> References: <20250825093700.3731633-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgAXYIy1MKxonJnxAA--.44975S8 X-Coremail-Antispam: 1UD129KBjvJXoWxGr4kWr1fWw1DAr45ZFy7Wrg_yoWrur4Up3 yag3WSkrW5JFsa9w1DJayDuasYy340grW7A3yxJwn7W3ZIvrs8KF1UX3yFqr98uFy5ur9r X3WkZr4UCayqqF7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUGVWUXwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVW8ZVWrXwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0pRiF4iUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai On the one hand unify bio split code, prepare to fix disordered split IO; On the other hand fix missing blkcg_bio_issue_init() and trace_block_split() for split IO. Noted discard is not handled, because discard is only splited for unaligned head and tail. Signed-off-by: Yu Kuai --- drivers/md/raid10.c | 53 ++++++++++++++++++++------------------------- drivers/md/raid10.h | 1 + 2 files changed, 24 insertions(+), 30 deletions(-) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index b60c30bfb6c7..b8777661307b 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -322,10 +322,12 @@ static void raid_end_bio_io(struct r10bio *r10_bio) struct bio *bio =3D r10_bio->master_bio; struct r10conf *conf =3D r10_bio->mddev->private; =20 - if (!test_bit(R10BIO_Uptodate, &r10_bio->state)) - bio->bi_status =3D BLK_STS_IOERR; + if (!test_and_set_bit(R10BIO_Returned, &r10_bio->state)) { + if (!test_bit(R10BIO_Uptodate, &r10_bio->state)) + bio->bi_status =3D BLK_STS_IOERR; + bio_endio(bio); + } =20 - bio_endio(bio); /* * Wake up any possible resync thread that waits for the device * to go idle. @@ -1154,7 +1156,6 @@ static void raid10_read_request(struct mddev *mddev, = struct bio *bio, int slot =3D r10_bio->read_slot; struct md_rdev *err_rdev =3D NULL; gfp_t gfp =3D GFP_NOIO; - int error; =20 if (slot >=3D 0 && r10_bio->devs[slot].rdev) { /* @@ -1203,17 +1204,16 @@ static void raid10_read_request(struct mddev *mddev= , struct bio *bio, rdev->bdev, (unsigned long long)r10_bio->sector); if (max_sectors < bio_sectors(bio)) { - struct bio *split =3D bio_split(bio, max_sectors, - gfp, &conf->bio_split); - if (IS_ERR(split)) { - error =3D PTR_ERR(split); - goto err_handle; - } - bio_chain(split, bio); allow_barrier(conf); - submit_bio_noacct(bio); + bio =3D bio_submit_split(bio, max_sectors, &conf->bio_split); wait_barrier(conf, false); - bio =3D split; + + if (!bio) { + set_bit(R10BIO_Returned, &r10_bio->state); + goto err_handle; + } + + bio->bi_opf &=3D ~REQ_NOMERGE; r10_bio->master_bio =3D bio; r10_bio->sectors =3D max_sectors; } @@ -1239,10 +1239,9 @@ static void raid10_read_request(struct mddev *mddev,= struct bio *bio, mddev_trace_remap(mddev, read_bio, r10_bio->sector); submit_bio_noacct(read_bio); return; + err_handle: atomic_dec(&rdev->nr_pending); - bio->bi_status =3D errno_to_blk_status(error); - set_bit(R10BIO_Uptodate, &r10_bio->state); raid_end_bio_io(r10_bio); } =20 @@ -1351,7 +1350,6 @@ static void raid10_write_request(struct mddev *mddev,= struct bio *bio, int i, k; sector_t sectors; int max_sectors; - int error; =20 if ((mddev_is_clustered(mddev) && mddev->cluster_ops->area_resyncing(mddev, WRITE, @@ -1465,10 +1463,8 @@ static void raid10_write_request(struct mddev *mddev= , struct bio *bio, * complexity of supporting that is not worth * the benefit. */ - if (bio->bi_opf & REQ_ATOMIC) { - error =3D -EIO; + if (bio->bi_opf & REQ_ATOMIC) goto err_handle; - } =20 good_sectors =3D first_bad - dev_sector; if (good_sectors < max_sectors) @@ -1489,17 +1485,16 @@ static void raid10_write_request(struct mddev *mdde= v, struct bio *bio, r10_bio->sectors =3D max_sectors; =20 if (r10_bio->sectors < bio_sectors(bio)) { - struct bio *split =3D bio_split(bio, r10_bio->sectors, - GFP_NOIO, &conf->bio_split); - if (IS_ERR(split)) { - error =3D PTR_ERR(split); - goto err_handle; - } - bio_chain(split, bio); allow_barrier(conf); - submit_bio_noacct(bio); + bio =3D bio_submit_split(bio, r10_bio->sectors, &conf->bio_split); wait_barrier(conf, false); - bio =3D split; + + if (!bio) { + set_bit(R10BIO_Returned, &r10_bio->state); + goto err_handle; + } + + bio->bi_opf &=3D ~REQ_NOMERGE; r10_bio->master_bio =3D bio; } =20 @@ -1531,8 +1526,6 @@ static void raid10_write_request(struct mddev *mddev,= struct bio *bio, } } =20 - bio->bi_status =3D errno_to_blk_status(error); - set_bit(R10BIO_Uptodate, &r10_bio->state); raid_end_bio_io(r10_bio); } =20 diff --git a/drivers/md/raid10.h b/drivers/md/raid10.h index 3f16ad6904a9..cc167e708125 100644 --- a/drivers/md/raid10.h +++ b/drivers/md/raid10.h @@ -165,6 +165,7 @@ enum r10bio_state { * so that raid10d knows what to do with them. */ R10BIO_ReadError, + R10BIO_Returned, /* If a write for this request means we can clear some * known-bad-block records, we set this flag. */ --=20 2.39.2 From nobody Fri Oct 3 21:00:02 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D7D82D0C92; Mon, 25 Aug 2025 09:45:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115134; cv=none; b=scA+x5YLV7VbwmQ8AnQst+i6TtqKYQ1wpHAYFXhpgYh8Eaf/txjtQV9QiQeQAwhx7FSmLTRp1JxjttiVgYFXUobx6RCdPvAeFZ8jMVtFx5S2gq09wSg6oN399KK8VXzALs3ibwSrP3ur9XPlV7JBV4pTehAasdkHMD5M+qltVXk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115134; c=relaxed/simple; bh=IrSpQPTclCR963Ewq+DqKgmHRDJaySLVds86rjTX8Sc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mM7UaNRsvqURsJcRTcL35jDIDMCMm+/FQyy6OConbfClLxrMuumGjFScjDHXR4xstEHD8eKP9UzDdEVi9YWiIWhd6Slkz5JovtaVM9wn5ggObmbMVh8RWdStUe8jFyEak58UubgFIql760bim5lWxEmZYlD3zByZFd1ufFsP4MQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c9Qrp6c2bzKHNKx; Mon, 25 Aug 2025 17:45:30 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 7F9121A12C9; Mon, 25 Aug 2025 17:45:30 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAXYIy1MKxonJnxAA--.44975S9; Mon, 25 Aug 2025 17:45:30 +0800 (CST) From: Yu Kuai To: hch@infradead.org, colyli@kernel.org, hare@suse.de, tieren@fnnas.com, axboe@kernel.dk, tj@kernel.org, josef@toxicpanda.com, song@kernel.org, yukuai3@huawei.com, akpm@linux-foundation.org, neil@brown.name Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH RFC 5/7] md/raid5: convert to use bio_submit_split() Date: Mon, 25 Aug 2025 17:36:58 +0800 Message-Id: <20250825093700.3731633-6-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250825093700.3731633-1-yukuai1@huaweicloud.com> References: <20250825093700.3731633-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgAXYIy1MKxonJnxAA--.44975S9 X-Coremail-Antispam: 1UD129KBjvJXoW7AFykKrWkJw1UKrWDtw1fZwb_yoW8Gr13pw sxury3XrW5Jrs09wsIqw4DKas5tFyDWrZ7trWxX34xGa92vFnrKay5XryYqF15CFWrKry2 yw1kJr1YkF18KFJanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUGVWUXwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVW8ZVWrXwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0pRiF4iUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai On the one hand unify bio split code, prepare to fix disordered split IO; On the other hand fix missing blkcg_bio_issue_init() and trace_block_split() for split IO. Signed-off-by: Yu Kuai --- drivers/md/raid5.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 023649fe2476..9ae749e66e9d 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -5468,17 +5468,19 @@ static int raid5_read_one_chunk(struct mddev *mddev= , struct bio *raid_bio) =20 static struct bio *chunk_aligned_read(struct mddev *mddev, struct bio *rai= d_bio) { - struct bio *split; sector_t sector =3D raid_bio->bi_iter.bi_sector; unsigned chunk_sects =3D mddev->chunk_sectors; unsigned sectors =3D chunk_sects - (sector & (chunk_sects-1)); =20 if (sectors < bio_sectors(raid_bio)) { struct r5conf *conf =3D mddev->private; - split =3D bio_split(raid_bio, sectors, GFP_NOIO, &conf->bio_split); - bio_chain(split, raid_bio); - submit_bio_noacct(raid_bio); - raid_bio =3D split; + + raid_bio =3D bio_submit_split(raid_bio, sectors, + &conf->bio_split); + if (!raid_bio) + return NULL; + + raid_bio->bi_opf &=3D ~REQ_NOMERGE; } =20 if (!raid5_read_one_chunk(mddev, raid_bio)) --=20 2.39.2 From nobody Fri Oct 3 21:00:02 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BAFF2D1F64; Mon, 25 Aug 2025 09:45:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115135; cv=none; b=kDrc/EKfkvoCD5e6IkbOXp/EI2wfPmCY0mG2ewVqeLLT8Q902eoxOWGzrcARo1IhsykV+VAeX/dHtw3rcCAEx0zo07S3bWTqNgGCkVv25yhBaM5j2ohPBEl5AhQv1hsJdhYkdpF64SBYZmnCOgvrSgUKhkRSA4R+wu99QIQJXxI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115135; c=relaxed/simple; bh=dZPCWnk0AIU0IXgTTO+yfv52hus4m7mfMmMGyfeJWSE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=oHBG9abDEOw/31OQAKqb+hPwTEkeGlJuQnNziVuBPBYd97M0UFeWLf2dxWB24fGuf5hO00M2dJJJ8DUIUmqRUlotzixbmz4K/UmmV3y1wUG3r9Jqfd+TyPYFooUEOVcQRunmAq/z/jgLeANDkOF74mG4CsWdyjMci4VTKhV0hrc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTPS id 4c9Qrr51VRzYQvhL; Mon, 25 Aug 2025 17:45:32 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 3BB131A15CA; Mon, 25 Aug 2025 17:45:31 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAXYIy1MKxonJnxAA--.44975S10; Mon, 25 Aug 2025 17:45:30 +0800 (CST) From: Yu Kuai To: hch@infradead.org, colyli@kernel.org, hare@suse.de, tieren@fnnas.com, axboe@kernel.dk, tj@kernel.org, josef@toxicpanda.com, song@kernel.org, yukuai3@huawei.com, akpm@linux-foundation.org, neil@brown.name Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH RFC 6/7] md/md-linear: convert to use bio_submit_split() Date: Mon, 25 Aug 2025 17:36:59 +0800 Message-Id: <20250825093700.3731633-7-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250825093700.3731633-1-yukuai1@huaweicloud.com> References: <20250825093700.3731633-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgAXYIy1MKxonJnxAA--.44975S10 X-Coremail-Antispam: 1UD129KBjvdXoWrtw43GFWfuFWDZF1fuw4fuFg_yoWkZwb_Wa 4ruFykWr1UK393Kr1YqwsIv3y5Wrn5WrykuFyjqa13A3WfZwn7Cr95Gw45X3yxXFWfXa45 Jr42y34ftr1UJjkaLaAFLSUrUUUUjb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUIcSsGvfJTRUUUbvAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k26cxKx2IYs7xG 6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2 IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28E F7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr 1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0D M2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjx v20xvE14v26r106r15McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1l F7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E8cxan2 IY04v7MxkF7I0En4kS14v26r4a6rW5MxAIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY 6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17 CEb7AF67AKxVW8ZVWrXwCIc40Y0x0EwIxGrwCI42IY6xIIjxv20xvE14v26r1I6r4UMIIF 0xvE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCw CI42IY6I8E87Iv67AKxVW8JVWxJwCI42IY6I8E87Iv6xkF7I0E14v26r4UJVWxJrUvcSsG vfC2KfnxnUUI43ZEXa7sRRtCztUUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai On the one hand unify bio split code, prepare to fix disordered split IO; On the other hand fix missing blkcg_bio_issue_init() and trace_block_split() for split IO. Signed-off-by: Yu Kuai --- drivers/md/md-linear.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/drivers/md/md-linear.c b/drivers/md/md-linear.c index 5d9b08115375..61375c61e4aa 100644 --- a/drivers/md/md-linear.c +++ b/drivers/md/md-linear.c @@ -256,18 +256,10 @@ static bool linear_make_request(struct mddev *mddev, = struct bio *bio) =20 if (unlikely(bio_end_sector(bio) > end_sector)) { /* This bio crosses a device boundary, so we have to split it */ - struct bio *split =3D bio_split(bio, end_sector - bio_sector, - GFP_NOIO, &mddev->bio_set); - - if (IS_ERR(split)) { - bio->bi_status =3D errno_to_blk_status(PTR_ERR(split)); - bio_endio(bio); + bio =3D bio_submit_split(bio, end_sector - bio_sector, + &mddev->bio_set); + if (!bio) return true; - } - - bio_chain(split, bio); - submit_bio_noacct(bio); - bio =3D split; } =20 md_account_bio(mddev, &bio); --=20 2.39.2 From nobody Fri Oct 3 21:00:02 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0619A2D29CE; Mon, 25 Aug 2025 09:45:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115137; cv=none; b=IUzeGOCitqrU6w5svB1lTDlY0OlFTtNwD+VqeGF4xUOxvyJM9rvBgdvvugeWiquPB26FaN6CoczhZvTnYJBRHfChe4c05I3lngizdSE9ZcP68ukCMx5MwYI+qONpjhHAf8aJCVbMcAOuo7WfgXNspzad2xEjHoUXyAmYh2Wh03A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756115137; c=relaxed/simple; bh=/NBSlyyO/P3lBhNkPHs4fzj3VCj2TiugBxOZZh4+1oM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NziQtVNs3/vWk4Z++7/KB7K7PN1XnW2HfKRCxMc+25fJq2BrPZR37HiUP/B7aFHpKtx5CkdyszsJYzCEHY3wBy054Oq9qCz4f0VCOpDfdcWVz6zdcqCTpp9cA166BfveNMilUJ366maTFwKENb/ut/Hkau+dUzhAvjlNstr7ALU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTPS id 4c9Qrr2X3fzKHNJ3; Mon, 25 Aug 2025 17:45:32 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id E70011A08F9; Mon, 25 Aug 2025 17:45:31 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgAXYIy1MKxonJnxAA--.44975S11; Mon, 25 Aug 2025 17:45:31 +0800 (CST) From: Yu Kuai To: hch@infradead.org, colyli@kernel.org, hare@suse.de, tieren@fnnas.com, axboe@kernel.dk, tj@kernel.org, josef@toxicpanda.com, song@kernel.org, yukuai3@huawei.com, akpm@linux-foundation.org, neil@brown.name Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-raid@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com, johnny.chenyi@huawei.com Subject: [PATCH RFC 7/7] block: fix disordered IO in the case recursive split Date: Mon, 25 Aug 2025 17:37:00 +0800 Message-Id: <20250825093700.3731633-8-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250825093700.3731633-1-yukuai1@huaweicloud.com> References: <20250825093700.3731633-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgAXYIy1MKxonJnxAA--.44975S11 X-Coremail-Antispam: 1UD129KBjvJXoW3AryrZr13CF1UGryrCrWfKrg_yoWfWw47pr Wagw1akr4DGF4Sgws7JF4293Wft3WkCr4UCay5C393JFsY9r1Dt3WDAa40vas8ArWrGrWU XF1kKry7Ww42va7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUmS14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_tr0E3s1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUGVWUXwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2 kIc2xKxwCY1x0262kKe7AKxVW8ZVWrXwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkE bVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67 AF67kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUCVW8JwCI 42IY6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF 4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x0267AKxVW8Jr0_Cr1UYxBI daVFxhVjvjDU0xZFpf9x0pRiF4iUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai Currently, split bio will be chained to original bio, and original bio will be resubmitted to the tail of current->bio_list, waiting for split bio to be issued. However, if split bio get split again, the IO order will be messed up, for example, in raid456 IO will first be split by max_sector from md_submit_bio(), and then later be split again by chunksize for internal handling: For example, assume max_sectors is 1M, and chunksize is 512k 1) issue a 2M IO: bio issuing: 0+2M current->bio_list: NULL 2) md_submit_bio() split by max_sector: bio issuing: 0+1M current->bio_list: 1M+1M 3) chunk_aligned_read() split by chunksize: bio issuing: 0+512k current->bio_list: 1M+1M -> 512k+512k 4) after first bio issued, __submit_bio_noacct() will contuine issuing next bio: bio issuing: 1M+1M current->bio_list: 512k+512k bio issued: 0+512k 5) chunk_aligned_read() split by chunksize: bio issuing: 1M+512k current->bio_list: 512k+512k -> 1536k+512k bio issued: 0+512k 6) no split afterwards, finally the issue order is: 0+512k -> 1M+512k -> 512k+512k -> 1536k+512k This behaviour will cause large IO read on raid456 endup to be small discontinuous IO in underlying disks. Fix this problem by placing split bio to the head of current->bio_list. Test script: test on 8 disk raid5 with 64k chunksize dd if=3D/dev/md0 of=3D/dev/null bs=3D4480k iflag=3Ddirect Test results: Before this patch 1) iostat results: Device r/s rMB/s rrqm/s %rrqm r_await rareq-sz aqu-sz %= util md0 52430.00 3276.87 0.00 0.00 0.62 64.00 32.60 = 80.10 sd* 4487.00 409.00 2054.00 31.40 0.82 93.34 3.68 7= 1.20 2) blktrace G stage: 8,0 0 486445 11.357392936 843 G R 14071424 + 128 [dd] 8,0 0 486451 11.357466360 843 G R 14071168 + 128 [dd] 8,0 0 486454 11.357515868 843 G R 14071296 + 128 [dd] 8,0 0 486468 11.357968099 843 G R 14072192 + 128 [dd] 8,0 0 486474 11.358031320 843 G R 14071936 + 128 [dd] 8,0 0 486480 11.358096298 843 G R 14071552 + 128 [dd] 8,0 0 486490 11.358303858 843 G R 14071808 + 128 [dd] 3) io seek for sdx: Noted io seek is the result from blktrace D stage, statistic of: ABS((offset of next IO) - (offset + len of previous IO)) Read|Write seek cnt 55175, zero cnt 25079 >=3D(KB) .. <(KB) : count ratio |distribution = | 0 .. 1 : 25079 45.5% |##############################= ##########| 1 .. 2 : 0 0.0% | = | 2 .. 4 : 0 0.0% | = | 4 .. 8 : 0 0.0% | = | 8 .. 16 : 0 0.0% | = | 16 .. 32 : 0 0.0% | = | 32 .. 64 : 12540 22.7% |##################### = | 64 .. 128 : 2508 4.5% |##### = | 128 .. 256 : 0 0.0% | = | 256 .. 512 : 10032 18.2% |################# = | 512 .. 1024 : 5016 9.1% |######### = | After this patch: 1) iostat results: Device r/s rMB/s rrqm/s %rrqm r_await rareq-sz aqu-sz %= util md0 87965.00 5271.88 0.00 0.00 0.16 61.37 14.03 = 90.60 sd* 6020.00 658.44 5117.00 45.95 0.44 112.00 2.68 8= 6.50 2) blktrace G stage: 8,0 0 206296 5.354894072 664 G R 7156992 + 128 [dd] 8,0 0 206305 5.355018179 664 G R 7157248 + 128 [dd] 8,0 0 206316 5.355204438 664 G R 7157504 + 128 [dd] 8,0 0 206319 5.355241048 664 G R 7157760 + 128 [dd] 8,0 0 206333 5.355500923 664 G R 7158016 + 128 [dd] 8,0 0 206344 5.355837806 664 G R 7158272 + 128 [dd] 8,0 0 206353 5.355960395 664 G R 7158528 + 128 [dd] 8,0 0 206357 5.356020772 664 G R 7158784 + 128 [dd] 2) io seek for sdx Read|Write seek cnt 28644, zero cnt 21483 >=3D(KB) .. <(KB) : count ratio |distribution = | 0 .. 1 : 21483 75.0% |##############################= ##########| 1 .. 2 : 0 0.0% | = | 2 .. 4 : 0 0.0% | = | 4 .. 8 : 0 0.0% | = | 8 .. 16 : 0 0.0% | = | 16 .. 32 : 0 0.0% | = | 32 .. 64 : 7161 25.0% |############## = | BTW, this looks like a long term problem from day one, and large sequential IO read is pretty common case like video playing. And even with this patch, in this test case IO is merged to at most 128k is due to block layer plug limit BLK_PLUG_FLUSH_SIZE, increase such limit and cat get even better performance. However, we'll figure out how to do this properly later. Fixes: d89d87965dcb ("When stacked block devices are in-use (e.g. md or dm)= , the recursive calls") Signed-off-by: Yu Kuai --- block/blk-core.c | 54 ++++++++++++++++++++++++++++---------------- block/blk-merge.c | 2 +- block/blk-throttle.c | 2 +- block/blk.h | 3 ++- 4 files changed, 39 insertions(+), 22 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 4201504158a1..cfb2179cc91e 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -725,7 +725,7 @@ static void __submit_bio_noacct_mq(struct bio *bio) current->bio_list =3D NULL; } =20 -void submit_bio_noacct_nocheck(struct bio *bio) +void submit_bio_noacct_nocheck(struct bio *bio, bool split) { blk_cgroup_bio_start(bio); blkcg_bio_issue_init(bio); @@ -745,12 +745,16 @@ void submit_bio_noacct_nocheck(struct bio *bio) * to collect a list of requests submited by a ->submit_bio method while * it is active, and then process them after it returned. */ - if (current->bio_list) - bio_list_add(¤t->bio_list[0], bio); - else if (!bdev_test_flag(bio->bi_bdev, BD_HAS_SUBMIT_BIO)) + if (current->bio_list) { + if (split) + bio_list_add_head(¤t->bio_list[0], bio); + else + bio_list_add(¤t->bio_list[0], bio); + } else if (!bdev_test_flag(bio->bi_bdev, BD_HAS_SUBMIT_BIO)) { __submit_bio_noacct_mq(bio); - else + } else { __submit_bio_noacct(bio); + } } =20 static blk_status_t blk_validate_atomic_write_op_size(struct request_queue= *q, @@ -765,16 +769,7 @@ static blk_status_t blk_validate_atomic_write_op_size(= struct request_queue *q, return BLK_STS_OK; } =20 -/** - * submit_bio_noacct - re-submit a bio to the block device layer for I/O - * @bio: The bio describing the location in memory and on the device. - * - * This is a version of submit_bio() that shall only be used for I/O that = is - * resubmitted to lower level drivers by stacking block drivers. All file - * systems and other upper level users of the block layer should use - * submit_bio() instead. - */ -void submit_bio_noacct(struct bio *bio) +static bool submit_bio_check(struct bio *bio) { struct block_device *bdev =3D bio->bi_bdev; struct request_queue *q =3D bdev_get_queue(bdev); @@ -869,19 +864,40 @@ void submit_bio_noacct(struct bio *bio) goto not_supported; } =20 - if (blk_throtl_bio(bio)) - return; - submit_bio_noacct_nocheck(bio); - return; + return !blk_throtl_bio(bio); =20 not_supported: status =3D BLK_STS_NOTSUPP; end_io: bio->bi_status =3D status; bio_endio(bio); + return false; +} + +/** + * submit_bio_noacct - re-submit a bio to the block device layer for I/O + * @bio: The bio describing the location in memory and on the device. + * + * This is a version of submit_bio() that shall only be used for I/O that = is + * resubmitted to lower level drivers by stacking block drivers. All file + * systems and other upper level users of the block layer should use + * submit_bio() instead. + */ +void submit_bio_noacct(struct bio *bio) +{ + if (submit_bio_check(bio)) + submit_bio_noacct_nocheck(bio, false); } EXPORT_SYMBOL(submit_bio_noacct); =20 +void submit_split_bio_noacct(struct bio *bio) +{ + WARN_ON_ONCE(!current->bio_list); + + if (submit_bio_check(bio)) + submit_bio_noacct_nocheck(bio, true); +} + static void bio_set_ioprio(struct bio *bio) { /* Nobody set ioprio so far? Initialize it based on task's nice value */ diff --git a/block/blk-merge.c b/block/blk-merge.c index c45d5e43e172..934bbafe0462 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -138,7 +138,7 @@ struct bio *bio_submit_split(struct bio *bio, int split= _sectors, bio_chain(split, bio); trace_block_split(split, bio->bi_iter.bi_sector); WARN_ON_ONCE(bio_zone_write_plugging(bio)); - submit_bio_noacct(bio); + submit_split_bio_noacct(bio); return split; =20 error: diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 397b6a410f9e..ead7b0eb4846 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -1224,7 +1224,7 @@ static void blk_throtl_dispatch_work_fn(struct work_s= truct *work) if (!bio_list_empty(&bio_list_on_stack)) { blk_start_plug(&plug); while ((bio =3D bio_list_pop(&bio_list_on_stack))) - submit_bio_noacct_nocheck(bio); + submit_bio_noacct_nocheck(bio, false); blk_finish_plug(&plug); } } diff --git a/block/blk.h b/block/blk.h index 46f566f9b126..d804a49c6313 100644 --- a/block/blk.h +++ b/block/blk.h @@ -54,7 +54,8 @@ bool blk_queue_start_drain(struct request_queue *q); bool __blk_freeze_queue_start(struct request_queue *q, struct task_struct *owner); int __bio_queue_enter(struct request_queue *q, struct bio *bio); -void submit_bio_noacct_nocheck(struct bio *bio); +void submit_bio_noacct_nocheck(struct bio *bio, bool split); +void submit_split_bio_noacct(struct bio *bio); void bio_await_chain(struct bio *bio); =20 static inline bool blk_try_enter_queue(struct request_queue *q, bool pm) --=20 2.39.2