From nobody Thu Jan 30 18:49:49 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 413391AAC9; Thu, 9 Jan 2025 01:56:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387808; cv=none; b=o8b2kiIYuW25Ew2pzGCXk/l8qEucoVx6ExxrWrUnwx4SNvi4QEc3yXSIZmQtxhY1EUehEZeNxurELUm7WEZ+vrYxEueEnuRmbOBMxQc8Ka6NwfRs2UStO9zujW5wGxc+cGrJ0TmEKjkpFYVm7VwS2WciOvX4ouFiD56Epd+j+Ps= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387808; c=relaxed/simple; bh=dCZcrVt+NhAlOI3TWoncE2IGE8o0mXFXpSd76oXb/fQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kqJO6UHa0j2OR3RcZlOZfI6nx4WQPWZGLCtZ8qADu81GgoI/m3XT4uf8B+j45Y6EoOSRW2JvSwrnbofzYgRBSSzytxQTuKXih6X/rKAcp+NPg91u0sU+Z0uhfsCjvyOWzkI2/i5BW2wP976HvOGTfAtp/0+R64IASinfLBf9PfA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4YT7Dj2HSqz4f3kvp; Thu, 9 Jan 2025 09:56:21 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id B21521A14BB; Thu, 9 Jan 2025 09:56:42 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCH61_YLH9nk+G_AQ--.844S5; Thu, 09 Jan 2025 09:56:42 +0800 (CST) From: Yu Kuai To: song@kernel.org, xni@redhat.com, yukuai3@huawei.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v2 md-6.14 1/5] md/md-bitmap: factor behind write counters out from bitmap_{start/end}write() Date: Thu, 9 Jan 2025 09:51:41 +0800 Message-Id: <20250109015145.158868-2-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109015145.158868-1-yukuai1@huaweicloud.com> References: <20250109015145.158868-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCH61_YLH9nk+G_AQ--.844S5 X-Coremail-Antispam: 1UD129KBjvJXoW3Jw4kCFW8Ar48JF48KFWDXFb_yoWfCF1Dpa 9FqFyakrW5trWaqw1DAFyDuF1Fvw1ktr9rtrWfW3sYg3Wqyr90gF4rWFWrKrn8AFy3ZFW3 Zan8trWUCrW2qFUanT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBG14x267AKxVW5JVWrJwAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_Jr4l82xGYIkIc2 x26xkF7I0E14v26r1I6r4UM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2z4x0 Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F4UJw A2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS 0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2 IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0 Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwCY1x0262kKe7AKxVWUAVWUtw CF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j 6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2Ij64 vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_ Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0x vEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUfDGrUUUUU= X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai behind_write is only used in raid1, prepare to refactor bitmap_{start/end}write(), there are no functional changes. Signed-off-by: Yu Kuai Reviewed-by: Xiao Ni --- drivers/md/md-bitmap.c | 57 +++++++++++++++++++++++++--------------- drivers/md/md-bitmap.h | 7 +++-- drivers/md/raid1.c | 12 +++++---- drivers/md/raid10.c | 6 ++--- drivers/md/raid5-cache.c | 3 +-- drivers/md/raid5.c | 11 ++++---- 6 files changed, 56 insertions(+), 40 deletions(-) diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c index c3a42dd66ce5..84fb4cc67d5e 100644 --- a/drivers/md/md-bitmap.c +++ b/drivers/md/md-bitmap.c @@ -1671,24 +1671,13 @@ __acquires(bitmap->lock) } =20 static int bitmap_startwrite(struct mddev *mddev, sector_t offset, - unsigned long sectors, bool behind) + unsigned long sectors) { struct bitmap *bitmap =3D mddev->bitmap; =20 if (!bitmap) return 0; =20 - if (behind) { - int bw; - atomic_inc(&bitmap->behind_writes); - bw =3D atomic_read(&bitmap->behind_writes); - if (bw > bitmap->behind_writes_used) - bitmap->behind_writes_used =3D bw; - - pr_debug("inc write-behind count %d/%lu\n", - bw, bitmap->mddev->bitmap_info.max_write_behind); - } - while (sectors) { sector_t blocks; bitmap_counter_t *bmc; @@ -1737,21 +1726,13 @@ static int bitmap_startwrite(struct mddev *mddev, s= ector_t offset, } =20 static void bitmap_endwrite(struct mddev *mddev, sector_t offset, - unsigned long sectors, bool success, bool behind) + unsigned long sectors, bool success) { struct bitmap *bitmap =3D mddev->bitmap; =20 if (!bitmap) return; =20 - if (behind) { - if (atomic_dec_and_test(&bitmap->behind_writes)) - wake_up(&bitmap->behind_wait); - pr_debug("dec write-behind count %d/%lu\n", - atomic_read(&bitmap->behind_writes), - bitmap->mddev->bitmap_info.max_write_behind); - } - while (sectors) { sector_t blocks; unsigned long flags; @@ -2062,6 +2043,37 @@ static void md_bitmap_free(void *data) kfree(bitmap); } =20 +static void bitmap_start_behind_write(struct mddev *mddev) +{ + struct bitmap *bitmap =3D mddev->bitmap; + int bw; + + if (!bitmap) + return; + + atomic_inc(&bitmap->behind_writes); + bw =3D atomic_read(&bitmap->behind_writes); + if (bw > bitmap->behind_writes_used) + bitmap->behind_writes_used =3D bw; + + pr_debug("inc write-behind count %d/%lu\n", + bw, bitmap->mddev->bitmap_info.max_write_behind); +} + +static void bitmap_end_behind_write(struct mddev *mddev) +{ + struct bitmap *bitmap =3D mddev->bitmap; + + if (!bitmap) + return; + + if (atomic_dec_and_test(&bitmap->behind_writes)) + wake_up(&bitmap->behind_wait); + pr_debug("dec write-behind count %d/%lu\n", + atomic_read(&bitmap->behind_writes), + bitmap->mddev->bitmap_info.max_write_behind); +} + static void bitmap_wait_behind_writes(struct mddev *mddev) { struct bitmap *bitmap =3D mddev->bitmap; @@ -2981,6 +2993,9 @@ static struct bitmap_operations bitmap_ops =3D { .dirty_bits =3D bitmap_dirty_bits, .unplug =3D bitmap_unplug, .daemon_work =3D bitmap_daemon_work, + + .start_behind_write =3D bitmap_start_behind_write, + .end_behind_write =3D bitmap_end_behind_write, .wait_behind_writes =3D bitmap_wait_behind_writes, =20 .startwrite =3D bitmap_startwrite, diff --git a/drivers/md/md-bitmap.h b/drivers/md/md-bitmap.h index 662e6fc141a7..e87a1f493d3c 100644 --- a/drivers/md/md-bitmap.h +++ b/drivers/md/md-bitmap.h @@ -84,12 +84,15 @@ struct bitmap_operations { unsigned long e); void (*unplug)(struct mddev *mddev, bool sync); void (*daemon_work)(struct mddev *mddev); + + void (*start_behind_write)(struct mddev *mddev); + void (*end_behind_write)(struct mddev *mddev); void (*wait_behind_writes)(struct mddev *mddev); =20 int (*startwrite)(struct mddev *mddev, sector_t offset, - unsigned long sectors, bool behind); + unsigned long sectors); void (*endwrite)(struct mddev *mddev, sector_t offset, - unsigned long sectors, bool success, bool behind); + unsigned long sectors, bool success); bool (*start_sync)(struct mddev *mddev, sector_t offset, sector_t *blocks, bool degraded); void (*end_sync)(struct mddev *mddev, sector_t offset, sector_t *blocks); diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 519c56f0ee3d..15ba7a001f30 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -420,10 +420,11 @@ static void close_write(struct r1bio *r1_bio) r1_bio->behind_master_bio =3D NULL; } =20 + if (test_bit(R1BIO_BehindIO, &r1_bio->state)) + mddev->bitmap_ops->end_behind_write(mddev); /* clear the bitmap if all writes complete successfully */ mddev->bitmap_ops->endwrite(mddev, r1_bio->sector, r1_bio->sectors, - !test_bit(R1BIO_Degraded, &r1_bio->state), - test_bit(R1BIO_BehindIO, &r1_bio->state)); + !test_bit(R1BIO_Degraded, &r1_bio->state)); md_write_end(mddev); } =20 @@ -1645,9 +1646,10 @@ static void raid1_write_request(struct mddev *mddev,= struct bio *bio, stats.behind_writes < max_write_behind) alloc_behind_master_bio(r1_bio, bio); =20 - mddev->bitmap_ops->startwrite( - mddev, r1_bio->sector, r1_bio->sectors, - test_bit(R1BIO_BehindIO, &r1_bio->state)); + if (test_bit(R1BIO_BehindIO, &r1_bio->state)) + mddev->bitmap_ops->start_behind_write(mddev); + mddev->bitmap_ops->startwrite(mddev, r1_bio->sector, + r1_bio->sectors); first_clone =3D 0; } =20 diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 7d7a8a2524dc..c3a93b2a26a6 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -430,8 +430,7 @@ static void close_write(struct r10bio *r10_bio) =20 /* clear the bitmap if all writes complete successfully */ mddev->bitmap_ops->endwrite(mddev, r10_bio->sector, r10_bio->sectors, - !test_bit(R10BIO_Degraded, &r10_bio->state), - false); + !test_bit(R10BIO_Degraded, &r10_bio->state)); md_write_end(mddev); } =20 @@ -1519,8 +1518,7 @@ static void raid10_write_request(struct mddev *mddev,= struct bio *bio, md_account_bio(mddev, &bio); r10_bio->master_bio =3D bio; atomic_set(&r10_bio->remaining, 1); - mddev->bitmap_ops->startwrite(mddev, r10_bio->sector, r10_bio->sectors, - false); + mddev->bitmap_ops->startwrite(mddev, r10_bio->sector, r10_bio->sectors); =20 for (i =3D 0; i < conf->copies; i++) { if (r10_bio->devs[i].bio) diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c index b4f7b79fd187..4c7ecdd5c1f3 100644 --- a/drivers/md/raid5-cache.c +++ b/drivers/md/raid5-cache.c @@ -315,8 +315,7 @@ void r5c_handle_cached_data_endio(struct r5conf *conf, r5c_return_dev_pending_writes(conf, &sh->dev[i]); conf->mddev->bitmap_ops->endwrite(conf->mddev, sh->sector, RAID5_STRIPE_SECTORS(conf), - !test_bit(STRIPE_DEGRADED, &sh->state), - false); + !test_bit(STRIPE_DEGRADED, &sh->state)); } } } diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index f09e7677ee9f..93cc7e252dd4 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -3564,7 +3564,7 @@ static void __add_stripe_bio(struct stripe_head *sh, = struct bio *bi, set_bit(STRIPE_BITMAP_PENDING, &sh->state); spin_unlock_irq(&sh->stripe_lock); conf->mddev->bitmap_ops->startwrite(conf->mddev, sh->sector, - RAID5_STRIPE_SECTORS(conf), false); + RAID5_STRIPE_SECTORS(conf)); spin_lock_irq(&sh->stripe_lock); clear_bit(STRIPE_BITMAP_PENDING, &sh->state); if (!sh->batch_head) { @@ -3665,7 +3665,7 @@ handle_failed_stripe(struct r5conf *conf, struct stri= pe_head *sh, if (bitmap_end) conf->mddev->bitmap_ops->endwrite(conf->mddev, sh->sector, RAID5_STRIPE_SECTORS(conf), - false, false); + false); bitmap_end =3D 0; /* and fail all 'written' */ bi =3D sh->dev[i].written; @@ -3712,7 +3712,7 @@ handle_failed_stripe(struct r5conf *conf, struct stri= pe_head *sh, if (bitmap_end) conf->mddev->bitmap_ops->endwrite(conf->mddev, sh->sector, RAID5_STRIPE_SECTORS(conf), - false, false); + false); /* If we were in the middle of a write the parity block might * still be locked - so just clear all R5_LOCKED flags */ @@ -4063,8 +4063,7 @@ static void handle_stripe_clean_event(struct r5conf *= conf, } conf->mddev->bitmap_ops->endwrite(conf->mddev, sh->sector, RAID5_STRIPE_SECTORS(conf), - !test_bit(STRIPE_DEGRADED, &sh->state), - false); + !test_bit(STRIPE_DEGRADED, &sh->state)); if (head_sh->batch_head) { sh =3D list_first_entry(&sh->batch_list, struct stripe_head, @@ -5787,7 +5786,7 @@ static void make_discard_request(struct mddev *mddev,= struct bio *bi) for (d =3D 0; d < conf->raid_disks - conf->max_degraded; d++) mddev->bitmap_ops->startwrite(mddev, sh->sector, - RAID5_STRIPE_SECTORS(conf), false); + RAID5_STRIPE_SECTORS(conf)); sh->bm_seq =3D conf->seq_flush + 1; set_bit(STRIPE_BIT_DELAY, &sh->state); } --=20 2.39.2 From nobody Thu Jan 30 18:49:49 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7B9F33A8D2; Thu, 9 Jan 2025 01:56:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387810; cv=none; b=Mo8e0KxGEDU7zELF3Skyi/rGHwAKJjht6ofF9jaFz2KN3pv2UBqCfYsg0whOx2iHyP0qAEKl5uH+GSDtZhOTzPPS3X32a2eF8KnKF1MewYykGGqC4wqF8aGHk5Y2OaN4TF5IGSFOtRC5dHzyhrAgN1wKKXCxcpAESsNIxui0o0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387810; c=relaxed/simple; bh=zCQsLVI4kZusdW7H/k+X3HrCdigvlylVBlt+ocaX4tU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=B41WYtCljWv2e9CrO14IPj3NF6PszV40c/Mn3aEbS0V4+SL0+I4FDzPQfScypNZ97Fgda/SphpQ3hKyL9H9tBzWf35rTA4I3ZIwR6uIh8igR4PXfCHEavhLsytzmZ7dk7FXT7X0Yt6wc4DeWcJ+WFfSwYSA9h5RpriqLOMm116o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4YT7Dr0YD5z4f3jqr; Thu, 9 Jan 2025 09:56:28 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 1A9521A08FC; Thu, 9 Jan 2025 09:56:43 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCH61_YLH9nk+G_AQ--.844S6; Thu, 09 Jan 2025 09:56:42 +0800 (CST) From: Yu Kuai To: song@kernel.org, xni@redhat.com, yukuai3@huawei.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v2 md-6.14 2/5] md/md-bitmap: remove the last parameter for bimtap_ops->endwrite() Date: Thu, 9 Jan 2025 09:51:42 +0800 Message-Id: <20250109015145.158868-3-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109015145.158868-1-yukuai1@huaweicloud.com> References: <20250109015145.158868-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCH61_YLH9nk+G_AQ--.844S6 X-Coremail-Antispam: 1UD129KBjvAXoW3ur17KFyrAF18Zw48KF4xXrb_yoW8JryUuo Z5ZwnI9a1Fqw1rXFyDtr1ftFW3Was0k3ySy3WfWrs8WrZrXa4Fqr1xGrWfG3yUtryfZFW3 Zr9Fqr48JF45Aw17n29KB7ZKAUJUUUU8529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UjIYCTnIWjp_UUUY_7AC8VAFwI0_Wr0E3s1l1xkIjI8I6I8E6xAIw20EY4v20xva j40_Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l82xGYIkIc2x26280x7IE14v26r15M28IrcIa0x kI8VCY1x0267AKxVW8JVW5JwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK021l84AC jcxK6xIIjxv20xvE14v26F1j6w1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr 1l84ACjcxK6I8E87Iv67AKxVW0oVCq3wA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE3s1l e2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI 8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwAC jcxG0xvY0x0EwIxGrwACjI8F5VA0II8E6IAqYI8I648v4I1lc7CjxVAaw2AFwI0_JF0_Jw 1l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWU JVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7V AKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4j 6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42 IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfU1c_TUUUUU X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai For the case that IO failed for one rdev, the bit will be mark as NEEDED in following cases: 1) If badblocks is set and rdev is not faulty; 2) If rdev is faulty; Case 1) is useless because synchronize data to badblocks make no sense. Case 2) can be replaced with mddev->degraded. Also remove R1BIO_Degraded, R10BIO_Degraded and STRIPE_DEGRADED since case 2) no longer use them. Signed-off-by: Yu Kuai --- drivers/md/md-bitmap.c | 19 ++++++++++--------- drivers/md/md-bitmap.h | 2 +- drivers/md/raid1.c | 26 +++----------------------- drivers/md/raid1.h | 1 - drivers/md/raid10.c | 23 +++-------------------- drivers/md/raid10.h | 1 - drivers/md/raid5-cache.c | 3 +-- drivers/md/raid5.c | 15 +++------------ drivers/md/raid5.h | 1 - 9 files changed, 21 insertions(+), 70 deletions(-) diff --git a/drivers/md/md-bitmap.c b/drivers/md/md-bitmap.c index 84fb4cc67d5e..b40a84b01085 100644 --- a/drivers/md/md-bitmap.c +++ b/drivers/md/md-bitmap.c @@ -1726,7 +1726,7 @@ static int bitmap_startwrite(struct mddev *mddev, sec= tor_t offset, } =20 static void bitmap_endwrite(struct mddev *mddev, sector_t offset, - unsigned long sectors, bool success) + unsigned long sectors) { struct bitmap *bitmap =3D mddev->bitmap; =20 @@ -1745,15 +1745,16 @@ static void bitmap_endwrite(struct mddev *mddev, se= ctor_t offset, return; } =20 - if (success && !bitmap->mddev->degraded && - bitmap->events_cleared < bitmap->mddev->events) { - bitmap->events_cleared =3D bitmap->mddev->events; - bitmap->need_sync =3D 1; - sysfs_notify_dirent_safe(bitmap->sysfs_can_clear); - } - - if (!success && !NEEDED(*bmc)) + if (!bitmap->mddev->degraded) { + if (bitmap->events_cleared < bitmap->mddev->events) { + bitmap->events_cleared =3D bitmap->mddev->events; + bitmap->need_sync =3D 1; + sysfs_notify_dirent_safe( + bitmap->sysfs_can_clear); + } + } else if (!NEEDED(*bmc)) { *bmc |=3D NEEDED_MASK; + } =20 if (COUNTER(*bmc) =3D=3D COUNTER_MAX) wake_up(&bitmap->overflow_wait); diff --git a/drivers/md/md-bitmap.h b/drivers/md/md-bitmap.h index e87a1f493d3c..31c93019c76b 100644 --- a/drivers/md/md-bitmap.h +++ b/drivers/md/md-bitmap.h @@ -92,7 +92,7 @@ struct bitmap_operations { int (*startwrite)(struct mddev *mddev, sector_t offset, unsigned long sectors); void (*endwrite)(struct mddev *mddev, sector_t offset, - unsigned long sectors, bool success); + unsigned long sectors); bool (*start_sync)(struct mddev *mddev, sector_t offset, sector_t *blocks, bool degraded); void (*end_sync)(struct mddev *mddev, sector_t offset, sector_t *blocks); diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 15ba7a001f30..4d09d85321b6 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -423,8 +423,7 @@ static void close_write(struct r1bio *r1_bio) if (test_bit(R1BIO_BehindIO, &r1_bio->state)) mddev->bitmap_ops->end_behind_write(mddev); /* clear the bitmap if all writes complete successfully */ - mddev->bitmap_ops->endwrite(mddev, r1_bio->sector, r1_bio->sectors, - !test_bit(R1BIO_Degraded, &r1_bio->state)); + mddev->bitmap_ops->endwrite(mddev, r1_bio->sector, r1_bio->sectors); md_write_end(mddev); } =20 @@ -481,8 +480,6 @@ static void raid1_end_write_request(struct bio *bio) if (!test_bit(Faulty, &rdev->flags)) set_bit(R1BIO_WriteError, &r1_bio->state); else { - /* Fail the request */ - set_bit(R1BIO_Degraded, &r1_bio->state); /* Finished with this branch */ r1_bio->bios[mirror] =3D NULL; to_put =3D bio; @@ -1536,11 +1533,8 @@ static void raid1_write_request(struct mddev *mddev,= struct bio *bio, write_behind =3D true; =20 r1_bio->bios[i] =3D NULL; - if (!rdev || test_bit(Faulty, &rdev->flags)) { - if (i < conf->raid_disks) - set_bit(R1BIO_Degraded, &r1_bio->state); + if (!rdev || test_bit(Faulty, &rdev->flags)) continue; - } =20 atomic_inc(&rdev->nr_pending); if (test_bit(WriteErrorSeen, &rdev->flags)) { @@ -1559,16 +1553,6 @@ static void raid1_write_request(struct mddev *mddev,= struct bio *bio, */ max_sectors =3D bad_sectors; rdev_dec_pending(rdev, mddev); - /* We don't set R1BIO_Degraded as that - * only applies if the disk is - * missing, so it might be re-added, - * and we want to know to recover this - * chunk. - * In this case the device is here, - * and the fact that this chunk is not - * in-sync is recorded in the bad - * block log - */ continue; } if (is_bad) { @@ -2616,12 +2600,10 @@ static void handle_write_finished(struct r1conf *co= nf, struct r1bio *r1_bio) * errors. */ fail =3D true; - if (!narrow_write_error(r1_bio, m)) { + if (!narrow_write_error(r1_bio, m)) md_error(conf->mddev, conf->mirrors[m].rdev); /* an I/O failed, we can't clear the bitmap */ - set_bit(R1BIO_Degraded, &r1_bio->state); - } rdev_dec_pending(conf->mirrors[m].rdev, conf->mddev); } @@ -2712,8 +2694,6 @@ static void raid1d(struct md_thread *thread) list_del(&r1_bio->retry_list); idx =3D sector_to_idx(r1_bio->sector); atomic_dec(&conf->nr_queued[idx]); - if (mddev->degraded) - set_bit(R1BIO_Degraded, &r1_bio->state); if (test_bit(R1BIO_WriteError, &r1_bio->state)) close_write(r1_bio); raid_end_bio_io(r1_bio); diff --git a/drivers/md/raid1.h b/drivers/md/raid1.h index 5300cbaa58a4..33f318fcc268 100644 --- a/drivers/md/raid1.h +++ b/drivers/md/raid1.h @@ -188,7 +188,6 @@ struct r1bio { enum r1bio_state { R1BIO_Uptodate, R1BIO_IsSync, - R1BIO_Degraded, R1BIO_BehindIO, /* Set ReadError on bios that experience a readerror so that * raid1d knows what to do with them. diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index c3a93b2a26a6..340a4710c222 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -429,8 +429,7 @@ static void close_write(struct r10bio *r10_bio) struct mddev *mddev =3D r10_bio->mddev; =20 /* clear the bitmap if all writes complete successfully */ - mddev->bitmap_ops->endwrite(mddev, r10_bio->sector, r10_bio->sectors, - !test_bit(R10BIO_Degraded, &r10_bio->state)); + mddev->bitmap_ops->endwrite(mddev, r10_bio->sector, r10_bio->sectors); md_write_end(mddev); } =20 @@ -500,7 +499,6 @@ static void raid10_end_write_request(struct bio *bio) set_bit(R10BIO_WriteError, &r10_bio->state); else { /* Fail the request */ - set_bit(R10BIO_Degraded, &r10_bio->state); r10_bio->devs[slot].bio =3D NULL; to_put =3D bio; dec_rdev =3D 1; @@ -1437,10 +1435,8 @@ static void raid10_write_request(struct mddev *mddev= , struct bio *bio, r10_bio->devs[i].bio =3D NULL; r10_bio->devs[i].repl_bio =3D NULL; =20 - if (!rdev && !rrdev) { - set_bit(R10BIO_Degraded, &r10_bio->state); + if (!rdev && !rrdev) continue; - } if (rdev && test_bit(WriteErrorSeen, &rdev->flags)) { sector_t first_bad; sector_t dev_sector =3D r10_bio->devs[i].addr; @@ -1457,14 +1453,6 @@ static void raid10_write_request(struct mddev *mddev= , struct bio *bio, * to other devices yet */ max_sectors =3D bad_sectors; - /* We don't set R10BIO_Degraded as that - * only applies if the disk is missing, - * so it might be re-added, and we want to - * know to recover this chunk. - * In this case the device is here, and the - * fact that this chunk is not in-sync is - * recorded in the bad block log. - */ continue; } if (is_bad) { @@ -2964,11 +2952,8 @@ static void handle_write_completed(struct r10conf *c= onf, struct r10bio *r10_bio) rdev_dec_pending(rdev, conf->mddev); } else if (bio !=3D NULL && bio->bi_status) { fail =3D true; - if (!narrow_write_error(r10_bio, m)) { + if (!narrow_write_error(r10_bio, m)) md_error(conf->mddev, rdev); - set_bit(R10BIO_Degraded, - &r10_bio->state); - } rdev_dec_pending(rdev, conf->mddev); } bio =3D r10_bio->devs[m].repl_bio; @@ -3027,8 +3012,6 @@ static void raid10d(struct md_thread *thread) r10_bio =3D list_first_entry(&tmp, struct r10bio, retry_list); list_del(&r10_bio->retry_list); - if (mddev->degraded) - set_bit(R10BIO_Degraded, &r10_bio->state); =20 if (test_bit(R10BIO_WriteError, &r10_bio->state)) diff --git a/drivers/md/raid10.h b/drivers/md/raid10.h index 2e75e88d0802..3f16ad6904a9 100644 --- a/drivers/md/raid10.h +++ b/drivers/md/raid10.h @@ -161,7 +161,6 @@ enum r10bio_state { R10BIO_IsSync, R10BIO_IsRecover, R10BIO_IsReshape, - R10BIO_Degraded, /* Set ReadError on bios that experience a read error * so that raid10d knows what to do with them. */ diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c index 4c7ecdd5c1f3..ba4f9577c737 100644 --- a/drivers/md/raid5-cache.c +++ b/drivers/md/raid5-cache.c @@ -314,8 +314,7 @@ void r5c_handle_cached_data_endio(struct r5conf *conf, set_bit(R5_UPTODATE, &sh->dev[i].flags); r5c_return_dev_pending_writes(conf, &sh->dev[i]); conf->mddev->bitmap_ops->endwrite(conf->mddev, - sh->sector, RAID5_STRIPE_SECTORS(conf), - !test_bit(STRIPE_DEGRADED, &sh->state)); + sh->sector, RAID5_STRIPE_SECTORS(conf)); } } } diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 93cc7e252dd4..a5a619400d8f 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -1345,8 +1345,6 @@ static void ops_run_io(struct stripe_head *sh, struct= stripe_head_state *s) submit_bio_noacct(rbi); } if (!rdev && !rrdev) { - if (op_is_write(op)) - set_bit(STRIPE_DEGRADED, &sh->state); pr_debug("skip op %d on disc %d for sector %llu\n", bi->bi_opf, i, (unsigned long long)sh->sector); clear_bit(R5_LOCKED, &sh->dev[i].flags); @@ -2884,7 +2882,6 @@ static void raid5_end_write_request(struct bio *bi) set_bit(R5_MadeGoodRepl, &sh->dev[i].flags); } else { if (bi->bi_status) { - set_bit(STRIPE_DEGRADED, &sh->state); set_bit(WriteErrorSeen, &rdev->flags); set_bit(R5_WriteError, &sh->dev[i].flags); if (!test_and_set_bit(WantReplacement, &rdev->flags)) @@ -3664,8 +3661,7 @@ handle_failed_stripe(struct r5conf *conf, struct stri= pe_head *sh, } if (bitmap_end) conf->mddev->bitmap_ops->endwrite(conf->mddev, - sh->sector, RAID5_STRIPE_SECTORS(conf), - false); + sh->sector, RAID5_STRIPE_SECTORS(conf)); bitmap_end =3D 0; /* and fail all 'written' */ bi =3D sh->dev[i].written; @@ -3711,8 +3707,7 @@ handle_failed_stripe(struct r5conf *conf, struct stri= pe_head *sh, } if (bitmap_end) conf->mddev->bitmap_ops->endwrite(conf->mddev, - sh->sector, RAID5_STRIPE_SECTORS(conf), - false); + sh->sector, RAID5_STRIPE_SECTORS(conf)); /* If we were in the middle of a write the parity block might * still be locked - so just clear all R5_LOCKED flags */ @@ -4062,8 +4057,7 @@ static void handle_stripe_clean_event(struct r5conf *= conf, wbi =3D wbi2; } conf->mddev->bitmap_ops->endwrite(conf->mddev, - sh->sector, RAID5_STRIPE_SECTORS(conf), - !test_bit(STRIPE_DEGRADED, &sh->state)); + sh->sector, RAID5_STRIPE_SECTORS(conf)); if (head_sh->batch_head) { sh =3D list_first_entry(&sh->batch_list, struct stripe_head, @@ -4340,7 +4334,6 @@ static void handle_parity_checks5(struct r5conf *conf= , struct stripe_head *sh, s->locked++; set_bit(R5_Wantwrite, &dev->flags); =20 - clear_bit(STRIPE_DEGRADED, &sh->state); set_bit(STRIPE_INSYNC, &sh->state); break; case check_state_run: @@ -4497,7 +4490,6 @@ static void handle_parity_checks6(struct r5conf *conf= , struct stripe_head *sh, clear_bit(R5_Wantwrite, &dev->flags); s->locked--; } - clear_bit(STRIPE_DEGRADED, &sh->state); =20 set_bit(STRIPE_INSYNC, &sh->state); break; @@ -4899,7 +4891,6 @@ static void break_stripe_batch_list(struct stripe_hea= d *head_sh, =20 set_mask_bits(&sh->state, ~(STRIPE_EXPAND_SYNC_FLAGS | (1 << STRIPE_PREREAD_ACTIVE) | - (1 << STRIPE_DEGRADED) | (1 << STRIPE_ON_UNPLUG_LIST)), head_sh->state & (1 << STRIPE_INSYNC)); =20 diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h index d174e586698f..69000fb90bd5 100644 --- a/drivers/md/raid5.h +++ b/drivers/md/raid5.h @@ -358,7 +358,6 @@ enum { STRIPE_REPLACED, STRIPE_PREREAD_ACTIVE, STRIPE_DELAYED, - STRIPE_DEGRADED, STRIPE_BIT_DELAY, STRIPE_EXPANDING, STRIPE_EXPAND_SOURCE, --=20 2.39.2 From nobody Thu Jan 30 18:49:49 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EB7D1BF37; Thu, 9 Jan 2025 01:56:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387808; cv=none; b=CzLVk5KNcBP1nmu43Q3olMAUsvHu/cNKMQrET2dNHWYblcs6/1S9P7Ci09dFE8FQRtwyv9O74TB+qOhMM1kL6tYC9MnOR24hdQfBMLU/DoPFvLkD819y2OWdeHRJ/g9OaUbvGsayoR46w3Hi184GshWEJ7WR+ZCrjKAu4xs7biU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387808; c=relaxed/simple; bh=qCBiIYWOdYrECwI01ejDTEQKuhKzGqr7m4XdSxpZWr0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LdfNtpClra5g8C4T3SFBW6IAhfDP8M07/l0xjXvSehlCZo+19CjjBcrDmuj2/8HE60Kx9wzGR3bUiBU0E0/F2Uru0toZzmO5QaXVgF43sybrmWsnBT2taWKwNiJuZpyz8higXYf2FYECsGhflzcYHF7NwEdlXt3BCtYc6ZRqDgg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4YT7Dk0bktz4f3kvl; Thu, 9 Jan 2025 09:56:22 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id 7854E1A14BB; Thu, 9 Jan 2025 09:56:43 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCH61_YLH9nk+G_AQ--.844S7; Thu, 09 Jan 2025 09:56:43 +0800 (CST) From: Yu Kuai To: song@kernel.org, xni@redhat.com, yukuai3@huawei.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v2 md-6.14 3/5] md: add a new callback pers->bitmap_sector() Date: Thu, 9 Jan 2025 09:51:43 +0800 Message-Id: <20250109015145.158868-4-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109015145.158868-1-yukuai1@huaweicloud.com> References: <20250109015145.158868-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCH61_YLH9nk+G_AQ--.844S7 X-Coremail-Antispam: 1UD129KBjvdXoWrZry3Kw1Dury7Gr4rAw15XFb_yoW3urb_Cr 9Iv3s3WFnxKr95tF1Ivr4SvrWqvwn3Wr4DuFy7KFyfZF9YqrySvrWv9r1xJw1xZF95CasI yryDXryrArnxtjkaLaAFLSUrUUUUjb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUIcSsGvfJTRUUUb68FF20E14v26rWj6s0DM7CY07I20VC2zVCF04k26cxKx2IYs7xG 6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUWwA2048vs2IY02 0Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xv wVC0I7IYx2IY67AKxVW7JVWDJwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM2 8EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AI xVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20x vE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xv r2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7MxkF7I0En4kS14v26r126r1DMx AIw28IcxkI7VAKI48JMxC20s026xCaFVCjc4AY6r1j6r4UMI8I3I0E5I8CrVAFwI0_Jr0_ Jr4lx2IqxVCjr7xvwVAFwI0_JrI_JrWlx4CE17CEb7AF67AKxVWUtVW8ZwCIc40Y0x0EwI xGrwCI42IY6xIIjxv20xvE14v26r1j6r1xMIIF0xvE2Ix0cI8IcVCY1x0267AKxVW8JVWx JwCI42IY6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcV C2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7VUbTUDJUUUUU== X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai This callback will be used in raid5 to convert io ranges from array to bitmap. Signed-off-by: Yu Kuai Reviewed-by: Xiao Ni --- drivers/md/md.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/md/md.h b/drivers/md/md.h index 4ba93af36126..de6dadb9a40b 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -746,6 +746,9 @@ struct md_personality void *(*takeover) (struct mddev *mddev); /* Changes the consistency policy of an active array. */ int (*change_consistency_policy)(struct mddev *mddev, const char *buf); + /* convert io ranges from array to bitmap */ + void (*bitmap_sector)(struct mddev *mddev, sector_t *offset, + unsigned long *sectors); }; =20 struct md_sysfs_entry { --=20 2.39.2 From nobody Thu Jan 30 18:49:49 2025 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB52221345; Thu, 9 Jan 2025 01:56:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387809; cv=none; b=arIl8LrMESNBkcdLR7hJevXt/qG6jT3yZ/KmXQjlwiq2StEMOIOY+YZjk04N4hOJKktiLRv+LL/zl3YeIREZWaL535q06nO9FSQZisnaQhV0faxKQpTyE4LU4uPS2Vg1B44z90pN1Cj0TDT4wlwFvPkP1cu8PK2+o98ogu9Iq/s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387809; c=relaxed/simple; bh=/qcelbv0tg3mBSh4E9DZULPHKc+YIKxZXWIPO3yDEu8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=US8Ap5o8mtni2xBOsxKBakrdRc97B886c+AMXN2zfQ2IXOnOeS7zOHgu8QKEsTc9kYEODBIoSNYvJZIEzO11YpQbdfs1i9iTHIsYXXfpjnzkx92zWcu97S6jXzKovVJ5nmtqcF1GfuVShV4tE5jI92yVrkhPA+LVs6DLgmBgAZc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4YT7Dk3BStz4f3kvv; Thu, 9 Jan 2025 09:56:22 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id D20761A09F1; Thu, 9 Jan 2025 09:56:43 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCH61_YLH9nk+G_AQ--.844S8; Thu, 09 Jan 2025 09:56:43 +0800 (CST) From: Yu Kuai To: song@kernel.org, xni@redhat.com, yukuai3@huawei.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v2 md-6.14 4/5] md/raid5: implement pers->bitmap_sector() Date: Thu, 9 Jan 2025 09:51:44 +0800 Message-Id: <20250109015145.158868-5-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109015145.158868-1-yukuai1@huaweicloud.com> References: <20250109015145.158868-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCH61_YLH9nk+G_AQ--.844S8 X-Coremail-Antispam: 1UD129KBjvJXoW3GFy5AFWfury7JF1DZr17GFg_yoWxKFW8pa yayFy2grWqqrn0gwsxJw1vgFyFkrZFkrW5tasrW34Ikw1fGr97Z3WUGwn0gF1UCFy3Jr45 tw1UAFW8Cr4qga7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBE14x267AKxVWrJVCq3wAFc2x0x2IEx4CE42xK8VAvwI8IcIk0 rVWrJVCq3wAFIxvE14AKwVWUJVWUGwA2048vs2IY020E87I2jVAFwI0_JF0E3s1l82xGYI kIc2x26xkF7I0E14v26ryj6s0DM28lY4IEw2IIxxk0rwA2F7IY1VAKz4vEj48ve4kI8wA2 z4x0Y4vE2Ix0cI8IcVAFwI0_Ar0_tr1l84ACjcxK6xIIjxv20xvEc7CjxVAFwI0_Gr1j6F 4UJwA2z4x0Y4vEx4A2jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq 3wAS0I0E0xvYzxvE52x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7 IYx2IY67AKxVWUJVWUGwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4U M4x0Y48IcxkI7VAKI48JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwCY1x0262kKe7AKxVWUAV WUtwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v2 6r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_Jw0_GFylIxkGc2 Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_ Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMI IF0xvEx4A2jsIEc7CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x0JUvYLPUUUUU = X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ Content-Type: text/plain; charset="utf-8" From: Yu Kuai Bitmap is used for the whole array for raid1/raid10, hence IO for the array can be used directly for bitmap. However, bitmap is used for underlying disks for raid5, hence IO for the array can't be used directly for bitmap. Implement pers->bitmap_sector() for raid5 to convert IO ranges from the array to the underlying disks. Signed-off-by: Yu Kuai --- drivers/md/raid5.c | 145 ++++++++++++++++++++++++++++++--------------- 1 file changed, 98 insertions(+), 47 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index a5a619400d8f..5377f4c3fffc 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -63,6 +63,13 @@ =20 #define RAID5_MAX_REQ_STRIPES 256 =20 +enum reshape_loc { + LOC_NO_RESHAPE, + LOC_AHEAD_OF_RESHAPE, + LOC_INSIDE_RESHAPE, + LOC_BEHIND_RESHAPE, +}; + static bool devices_handle_discard_safely =3D false; module_param(devices_handle_discard_safely, bool, 0644); MODULE_PARM_DESC(devices_handle_discard_safely, @@ -2947,6 +2954,94 @@ static void raid5_error(struct mddev *mddev, struct = md_rdev *rdev) r5c_update_on_rdev_error(mddev, rdev); } =20 +static bool ahead_of_reshape(struct mddev *mddev, sector_t sector, + sector_t reshape_sector) +{ + return mddev->reshape_backwards ? sector < reshape_sector : + sector >=3D reshape_sector; +} + +static bool range_ahead_of_reshape(struct mddev *mddev, sector_t min, + sector_t max, sector_t reshape_sector) +{ + return mddev->reshape_backwards ? max < reshape_sector : + min >=3D reshape_sector; +} + +static enum reshape_loc get_reshape_loc(struct mddev *mddev, + struct r5conf *conf, sector_t logical_sector) +{ + sector_t reshape_progress, reshape_safe; + /* + * Spinlock is needed as reshape_progress may be + * 64bit on a 32bit platform, and so it might be + * possible to see a half-updated value + * Of course reshape_progress could change after + * the lock is dropped, so once we get a reference + * to the stripe that we think it is, we will have + * to check again. + */ + spin_lock_irq(&conf->device_lock); + reshape_progress =3D conf->reshape_progress; + reshape_safe =3D conf->reshape_safe; + spin_unlock_irq(&conf->device_lock); + if (reshape_progress =3D=3D MaxSector) + return LOC_NO_RESHAPE; + if (ahead_of_reshape(mddev, logical_sector, reshape_progress)) + return LOC_AHEAD_OF_RESHAPE; + if (ahead_of_reshape(mddev, logical_sector, reshape_safe)) + return LOC_INSIDE_RESHAPE; + return LOC_BEHIND_RESHAPE; +} + +static void raid5_bitmap_sector(struct mddev *mddev, sector_t *offset, + unsigned long *sectors) +{ + struct r5conf *conf =3D mddev->private; + sector_t start =3D *offset; + sector_t end =3D start + *sectors; + sector_t prev_start =3D start; + sector_t prev_end =3D end; + int sectors_per_chunk; + enum reshape_loc loc; + int dd_idx; + + sectors_per_chunk =3D conf->chunk_sectors * + (conf->raid_disks - conf->max_degraded); + start =3D round_down(start, sectors_per_chunk); + end =3D round_up(end, sectors_per_chunk); + + start =3D raid5_compute_sector(conf, start, 0, &dd_idx, NULL); + end =3D raid5_compute_sector(conf, end, 0, &dd_idx, NULL); + + /* + * For LOC_INSIDE_RESHAPE, this IO will wait for reshape to make + * progress, hence it's the same as LOC_BEHIND_RESHAPE. + */ + loc =3D get_reshape_loc(mddev, conf, prev_start); + if (likely(loc !=3D LOC_AHEAD_OF_RESHAPE)) { + *offset =3D start; + *sectors =3D end - start; + return; + } + + sectors_per_chunk =3D conf->prev_chunk_sectors * + (conf->previous_raid_disks - conf->max_degraded); + prev_start =3D round_down(prev_start, sectors_per_chunk); + prev_end =3D round_down(prev_end, sectors_per_chunk); + + prev_start =3D raid5_compute_sector(conf, prev_start, 1, &dd_idx, NULL); + prev_end =3D raid5_compute_sector(conf, prev_end, 1, &dd_idx, NULL); + + /* + * for LOC_AHEAD_OF_RESHAPE, reshape can make progress before this IO + * is handled in make_stripe_request(), we can't know this here hence + * we set bits for both. + */ + *offset =3D min(start, prev_start); + *sectors =3D max(end, prev_end) - *offset; +} + /* * Input: a 'big' sector number, * Output: index of the data and parity disk, and the sector # in them. @@ -5792,20 +5887,6 @@ static void make_discard_request(struct mddev *mddev= , struct bio *bi) bio_endio(bi); } =20 -static bool ahead_of_reshape(struct mddev *mddev, sector_t sector, - sector_t reshape_sector) -{ - return mddev->reshape_backwards ? sector < reshape_sector : - sector >=3D reshape_sector; -} - -static bool range_ahead_of_reshape(struct mddev *mddev, sector_t min, - sector_t max, sector_t reshape_sector) -{ - return mddev->reshape_backwards ? max < reshape_sector : - min >=3D reshape_sector; -} - static bool stripe_ahead_of_reshape(struct mddev *mddev, struct r5conf *co= nf, struct stripe_head *sh) { @@ -5885,39 +5966,6 @@ static int add_all_stripe_bios(struct r5conf *conf, return 1; } =20 -enum reshape_loc { - LOC_NO_RESHAPE, - LOC_AHEAD_OF_RESHAPE, - LOC_INSIDE_RESHAPE, - LOC_BEHIND_RESHAPE, -}; - -static enum reshape_loc get_reshape_loc(struct mddev *mddev, - struct r5conf *conf, sector_t logical_sector) -{ - sector_t reshape_progress, reshape_safe; - /* - * Spinlock is needed as reshape_progress may be - * 64bit on a 32bit platform, and so it might be - * possible to see a half-updated value - * Of course reshape_progress could change after - * the lock is dropped, so once we get a reference - * to the stripe that we think it is, we will have - * to check again. - */ - spin_lock_irq(&conf->device_lock); - reshape_progress =3D conf->reshape_progress; - reshape_safe =3D conf->reshape_safe; - spin_unlock_irq(&conf->device_lock); - if (reshape_progress =3D=3D MaxSector) - return LOC_NO_RESHAPE; - if (ahead_of_reshape(mddev, logical_sector, reshape_progress)) - return LOC_AHEAD_OF_RESHAPE; - if (ahead_of_reshape(mddev, logical_sector, reshape_safe)) - return LOC_INSIDE_RESHAPE; - return LOC_BEHIND_RESHAPE; -} - static enum stripe_result make_stripe_request(struct mddev *mddev, struct r5conf *conf, struct stripe_request_ctx *ctx, sector_t logical_sector, struct bio *bi) @@ -8966,6 +9014,7 @@ static struct md_personality raid6_personality =3D .takeover =3D raid6_takeover, .change_consistency_policy =3D raid5_change_consistency_policy, .prepare_suspend =3D raid5_prepare_suspend, + .bitmap_sector =3D raid5_bitmap_sector, }; static struct md_personality raid5_personality =3D { @@ -8991,6 +9040,7 @@ static struct md_personality raid5_personality =3D .takeover =3D raid5_takeover, .change_consistency_policy =3D raid5_change_consistency_policy, .prepare_suspend =3D raid5_prepare_suspend, + .bitmap_sector =3D raid5_bitmap_sector, }; =20 static struct md_personality raid4_personality =3D @@ -9017,6 +9067,7 @@ static struct md_personality raid4_personality =3D .takeover =3D raid4_takeover, .change_consistency_policy =3D raid5_change_consistency_policy, .prepare_suspend =3D raid5_prepare_suspend, + .bitmap_sector =3D raid5_bitmap_sector, }; =20 static int __init raid5_init(void) --=20 2.39.2 From nobody Thu Jan 30 18:49:49 2025 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0A41824A3; Thu, 9 Jan 2025 01:56:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387811; cv=none; b=RFDup487iiI0qz+JWHsAd1NIFBnCxT6dWJVrTdcGZvkKKdMRoWUeBCeDe413EZUCrQb2BVKU9dKt6i+BAEJD+CdUy+5E4I3mg6/96uYp+asZ072WO4mJuS3Pcz+wrTYWTZvB8OWmeulYjaNj9tfEz1plyhIo9lf1pVKwwjM6s2c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736387811; c=relaxed/simple; bh=BTUcyrUsGVSkKAibdOuSMvFBpmc8aDfhXq5SoTrlVBs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=iC5f5F/0j6kyaNE+mQgHegpPKENWB85MCOk+rurnZcW0Dv389MVNu8shY2kjGeCXogJ4Vhfy6qQJ/WwxXeP9UPxaDAUKV9WHqJGB/HBh05yAxmdBAWXbilWqpgiyD1m4X9lpT96XUIQVT2dvl4vuTY3+ihsxlz4qRkTdezvmGjs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4YT7Dn5Lpmz4f3kFM; Thu, 9 Jan 2025 09:56:25 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.128]) by mail.maildlp.com (Postfix) with ESMTP id E92481A10B9; Thu, 9 Jan 2025 09:56:45 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.104.67]) by APP4 (Coremail) with SMTP id gCh0CgCH61_YLH9nk+G_AQ--.844S9; Thu, 09 Jan 2025 09:56:44 +0800 (CST) From: Yu Kuai To: song@kernel.org, xni@redhat.com, yukuai3@huawei.com Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org, yukuai1@huaweicloud.com, yi.zhang@huawei.com, yangerkun@huawei.com Subject: [PATCH v2 md-6.14 5/5] md/md-bitmap: move bitmap_{start, end}write to md upper layer Date: Thu, 9 Jan 2025 09:51:45 +0800 Message-Id: <20250109015145.158868-6-yukuai1@huaweicloud.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250109015145.158868-1-yukuai1@huaweicloud.com> References: <20250109015145.158868-1-yukuai1@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgCH61_YLH9nk+G_AQ--.844S9 X-Coremail-Antispam: 1UD129KBjvAXoW3ZrWrtr15XF48Gr4Utr4fXwb_yoW8Jw13Ao Z7Zry5Xrn5Gr48X34rAr43JFW3WryDJry5A345GryDXF9rXrnYqw1IkrW3Jr1jqr13XF43 Zry7J3WUJr1UJr1Dn29KB7ZKAUJUUUU8529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UjIYCTnIWjp_UUUYR7AC8VAFwI0_Wr0E3s1l1xkIjI8I6I8E6xAIw20EY4v20xva j40_Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l82xGYIkIc2x26280x7IE14v26r126s0DM28Irc Ia0xkI8VCY1x0267AKxVW5JVCq3wA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK021l 84ACjcxK6xIIjxv20xvE14v26F1j6w1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26r4UJV WxJr1l84ACjcxK6I8E87Iv67AKxVW0oVCq3wA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE 3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2I x0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8 JwACjcxG0xvY0x0EwIxGrwACjI8F5VA0II8E6IAqYI8I648v4I1lc7CjxVAaw2AFwI0_JF 0_Jw1l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AK xVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r1q6r43MIIYrx kI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_JFI_Gr1lIxAIcVC0I7IYx2IY6xkF7I0E14v2 6r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY6I8E87Iv67AKxVWUJVW8Jw CI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa73UjIFyTuYvjfUYcTQUUUU U X-CM-SenderInfo: 51xn3trlr6x35dzhxuhorxvhhfrp/ From: Yu Kuai There are two BUG reports that raid5 will hang at bitmap_startwrite([1],[2]), root cause is that bitmap start write and end write is unbalanced, it's not quite clear where, and while reviewing raid5 code, it's found that bitmap operations can be optimized. For example, for a 4 disks raid5, with chunksize=3D8k, if user issue a IO (0 + 48k) to the array: =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=90 =E2=94=82chunk 0 =E2=94= =82 =E2=94=82 =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=AC=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=BC =E2=94=82 sh0 =E2=94=82A0: 0 + 4k =E2=94=82A1: 8k + 4k =E2=94=82A2: 16k = + 4k =E2=94=82A3: P =E2=94=82 =E2=94=82 =E2=94=BC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=BC=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=BC=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=BC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=BC =E2=94=82 sh1 =E2=94=82B0: 4k + 4k =E2=94=82B1: 12k + 4k =E2=94=82B2: 20k = + 4k =E2=94=82B3: P =E2=94=82 =E2=94=BC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=B4=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=B4=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=B4=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=B4=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=BC =E2=94=82chunk 1 =E2=94= =82 =E2=94=82 =E2=94=8C=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=AC=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=AC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=A4 =E2=94=82 sh2 =E2=94=82C0: 24k + 4k=E2=94=82C1: 32k + 4k =E2=94=82C2: P = =E2=94=82C3: 40k + 4k=E2=94=82 =E2=94=82 =E2=94=BC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=BC=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=BC=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=BC=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=BC =E2=94=82 sh3 =E2=94=82D0: 28k + 4k=E2=94=82D1: 36k + 4k =E2=94=82D2: P = =E2=94=82D3: 44k + 4k=E2=94=82 =E2=94=94=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=B4=E2= =94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=B4=E2=94=80=E2=94=80=E2=94=80=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=B4=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94= =80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=B4=E2=94=80= =E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2=94=80=E2= =94=80=E2=94=80=E2=94=80=E2=94=98 Before this patch, 4 stripe head will be used, and each sh will attach bio for 3 disks, and each attached bio will trigger bitmap_startwrite() once, which means total 12 times. - 3 times (0 + 4k), for (A0, A1 and A2) - 3 times (4 + 4k), for (B0, B1 and B2) - 3 times (8 + 4k), for (C0, C1 and C3) - 3 times (12 + 4k), for (D0, D1 and D3) After this patch, md upper layer will calculate that IO range (0 + 48k) is corresponding to the bitmap (0 + 16k), and call bitmap_startwrite() just once. Noted that this patch will align bitmap ranges to the chunks, for example, if user issue a IO (0 + 4k) to array: - Before this patch, 1 time (0 + 4k), for A0; - After this patch, 1 time (0 + 8k) for chunk 0; Usually, one bitmap bit will represent more than one disk chunk, and this doesn't have any difference. And even if user really created a array that one chunk contain multiple bits, the overhead is that more data will be recovered after power failure. Also remove STRIPE_BITMAP_PENDING since it's not used anymore. [1] https://lore.kernel.org/all/CAJpMwyjmHQLvm6zg1cmQErttNNQPDAAXPKM3xgTjMh= bfts986Q@mail.gmail.com/ [2] https://lore.kernel.org/all/ADF7D720-5764-4AF3-B68E-1845988737AA@flying= circus.io/ Signed-off-by: Yu Kuai --- drivers/md/md.c | 29 +++++++++++++++++++++++ drivers/md/md.h | 2 ++ drivers/md/raid1.c | 4 ---- drivers/md/raid10.c | 3 --- drivers/md/raid5-cache.c | 2 -- drivers/md/raid5.c | 50 +++++----------------------------------- drivers/md/raid5.h | 3 --- 7 files changed, 37 insertions(+), 56 deletions(-) diff --git a/drivers/md/md.c b/drivers/md/md.c index 3dd013f82e26..866015b681af 100644 --- a/drivers/md/md.c +++ b/drivers/md/md.c @@ -8745,12 +8745,32 @@ void md_submit_discard_bio(struct mddev *mddev, str= uct md_rdev *rdev, } EXPORT_SYMBOL_GPL(md_submit_discard_bio); =20 +static void md_bitmap_start(struct mddev *mddev, + struct md_io_clone *md_io_clone) +{ + if (mddev->pers->bitmap_sector) + mddev->pers->bitmap_sector(mddev, &md_io_clone->offset, + &md_io_clone->sectors); + + mddev->bitmap_ops->startwrite(mddev, md_io_clone->offset, + md_io_clone->sectors); +} + +static void md_bitmap_end(struct mddev *mddev, struct md_io_clone *md_io_c= lone) +{ + mddev->bitmap_ops->endwrite(mddev, md_io_clone->offset, + md_io_clone->sectors); +} + static void md_end_clone_io(struct bio *bio) { struct md_io_clone *md_io_clone =3D bio->bi_private; struct bio *orig_bio =3D md_io_clone->orig_bio; struct mddev *mddev =3D md_io_clone->mddev; =20 + if (bio_data_dir(orig_bio) =3D=3D WRITE && mddev->bitmap) + md_bitmap_end(mddev, md_io_clone); + if (bio->bi_status && !orig_bio->bi_status) orig_bio->bi_status =3D bio->bi_status; =20 @@ -8775,6 +8795,12 @@ static void md_clone_bio(struct mddev *mddev, struct= bio **bio) if (blk_queue_io_stat(bdev->bd_disk->queue)) md_io_clone->start_time =3D bio_start_io_acct(*bio); =20 + if (bio_data_dir(*bio) =3D=3D WRITE && mddev->bitmap) { + md_io_clone->offset =3D (*bio)->bi_iter.bi_sector; + md_io_clone->sectors =3D bio_sectors(*bio); + md_bitmap_start(mddev, md_io_clone); + } + clone->bi_end_io =3D md_end_clone_io; clone->bi_private =3D md_io_clone; *bio =3D clone; @@ -8793,6 +8819,9 @@ void md_free_cloned_bio(struct bio *bio) struct bio *orig_bio =3D md_io_clone->orig_bio; struct mddev *mddev =3D md_io_clone->mddev; =20 + if (bio_data_dir(orig_bio) =3D=3D WRITE && mddev->bitmap) + md_bitmap_end(mddev, md_io_clone); + if (bio->bi_status && !orig_bio->bi_status) orig_bio->bi_status =3D bio->bi_status; =20 diff --git a/drivers/md/md.h b/drivers/md/md.h index de6dadb9a40b..def808064ad8 100644 --- a/drivers/md/md.h +++ b/drivers/md/md.h @@ -831,6 +831,8 @@ struct md_io_clone { struct mddev *mddev; struct bio *orig_bio; unsigned long start_time; + sector_t offset; + unsigned long sectors; struct bio bio_clone; }; =20 diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c index 4d09d85321b6..a5cd6522fc2d 100644 --- a/drivers/md/raid1.c +++ b/drivers/md/raid1.c @@ -422,8 +422,6 @@ static void close_write(struct r1bio *r1_bio) =20 if (test_bit(R1BIO_BehindIO, &r1_bio->state)) mddev->bitmap_ops->end_behind_write(mddev); - /* clear the bitmap if all writes complete successfully */ - mddev->bitmap_ops->endwrite(mddev, r1_bio->sector, r1_bio->sectors); md_write_end(mddev); } =20 @@ -1632,8 +1630,6 @@ static void raid1_write_request(struct mddev *mddev, = struct bio *bio, =20 if (test_bit(R1BIO_BehindIO, &r1_bio->state)) mddev->bitmap_ops->start_behind_write(mddev); - mddev->bitmap_ops->startwrite(mddev, r1_bio->sector, - r1_bio->sectors); first_clone =3D 0; } =20 diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 340a4710c222..e1e6cd7fb125 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -428,8 +428,6 @@ static void close_write(struct r10bio *r10_bio) { struct mddev *mddev =3D r10_bio->mddev; =20 - /* clear the bitmap if all writes complete successfully */ - mddev->bitmap_ops->endwrite(mddev, r10_bio->sector, r10_bio->sectors); md_write_end(mddev); } =20 @@ -1506,7 +1504,6 @@ static void raid10_write_request(struct mddev *mddev,= struct bio *bio, md_account_bio(mddev, &bio); r10_bio->master_bio =3D bio; atomic_set(&r10_bio->remaining, 1); - mddev->bitmap_ops->startwrite(mddev, r10_bio->sector, r10_bio->sectors); =20 for (i =3D 0; i < conf->copies; i++) { if (r10_bio->devs[i].bio) diff --git a/drivers/md/raid5-cache.c b/drivers/md/raid5-cache.c index ba4f9577c737..011246e16a99 100644 --- a/drivers/md/raid5-cache.c +++ b/drivers/md/raid5-cache.c @@ -313,8 +313,6 @@ void r5c_handle_cached_data_endio(struct r5conf *conf, if (sh->dev[i].written) { set_bit(R5_UPTODATE, &sh->dev[i].flags); r5c_return_dev_pending_writes(conf, &sh->dev[i]); - conf->mddev->bitmap_ops->endwrite(conf->mddev, - sh->sector, RAID5_STRIPE_SECTORS(conf)); } } } diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 5377f4c3fffc..c97df8acf0ba 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -913,8 +913,7 @@ static bool stripe_can_batch(struct stripe_head *sh) if (raid5_has_log(conf) || raid5_has_ppl(conf)) return false; return test_bit(STRIPE_BATCH_READY, &sh->state) && - !test_bit(STRIPE_BITMAP_PENDING, &sh->state) && - is_full_stripe_write(sh); + is_full_stripe_write(sh); } =20 /* we only do back search */ @@ -3640,29 +3639,9 @@ static void __add_stripe_bio(struct stripe_head *sh,= struct bio *bi, (*bip)->bi_iter.bi_sector, sh->sector, dd_idx, sh->dev[dd_idx].sector); =20 - if (conf->mddev->bitmap && firstwrite) { - /* Cannot hold spinlock over bitmap_startwrite, - * but must ensure this isn't added to a batch until - * we have added to the bitmap and set bm_seq. - * So set STRIPE_BITMAP_PENDING to prevent - * batching. - * If multiple __add_stripe_bio() calls race here they - * much all set STRIPE_BITMAP_PENDING. So only the first one - * to complete "bitmap_startwrite" gets to set - * STRIPE_BIT_DELAY. This is important as once a stripe - * is added to a batch, STRIPE_BIT_DELAY cannot be changed - * any more. - */ - set_bit(STRIPE_BITMAP_PENDING, &sh->state); - spin_unlock_irq(&sh->stripe_lock); - conf->mddev->bitmap_ops->startwrite(conf->mddev, sh->sector, - RAID5_STRIPE_SECTORS(conf)); - spin_lock_irq(&sh->stripe_lock); - clear_bit(STRIPE_BITMAP_PENDING, &sh->state); - if (!sh->batch_head) { - sh->bm_seq =3D conf->seq_flush+1; - set_bit(STRIPE_BIT_DELAY, &sh->state); - } + if (conf->mddev->bitmap && firstwrite && !sh->batch_head) { + sh->bm_seq =3D conf->seq_flush+1; + set_bit(STRIPE_BIT_DELAY, &sh->state); } } =20 @@ -3713,7 +3692,6 @@ handle_failed_stripe(struct r5conf *conf, struct stri= pe_head *sh, BUG_ON(sh->batch_head); for (i =3D disks; i--; ) { struct bio *bi; - int bitmap_end =3D 0; =20 if (test_bit(R5_ReadError, &sh->dev[i].flags)) { struct md_rdev *rdev =3D conf->disks[i].rdev; @@ -3738,8 +3716,6 @@ handle_failed_stripe(struct r5conf *conf, struct stri= pe_head *sh, sh->dev[i].towrite =3D NULL; sh->overwrite_disks =3D 0; spin_unlock_irq(&sh->stripe_lock); - if (bi) - bitmap_end =3D 1; =20 log_stripe_write_finished(sh); =20 @@ -3754,10 +3730,6 @@ handle_failed_stripe(struct r5conf *conf, struct str= ipe_head *sh, bio_io_error(bi); bi =3D nextbi; } - if (bitmap_end) - conf->mddev->bitmap_ops->endwrite(conf->mddev, - sh->sector, RAID5_STRIPE_SECTORS(conf)); - bitmap_end =3D 0; /* and fail all 'written' */ bi =3D sh->dev[i].written; sh->dev[i].written =3D NULL; @@ -3766,7 +3738,6 @@ handle_failed_stripe(struct r5conf *conf, struct stri= pe_head *sh, sh->dev[i].page =3D sh->dev[i].orig_page; } =20 - if (bi) bitmap_end =3D 1; while (bi && bi->bi_iter.bi_sector < sh->dev[i].sector + RAID5_STRIPE_SECTORS(conf)) { struct bio *bi2 =3D r5_next_bio(conf, bi, sh->dev[i].sector); @@ -3800,9 +3771,6 @@ handle_failed_stripe(struct r5conf *conf, struct stri= pe_head *sh, bi =3D nextbi; } } - if (bitmap_end) - conf->mddev->bitmap_ops->endwrite(conf->mddev, - sh->sector, RAID5_STRIPE_SECTORS(conf)); /* If we were in the middle of a write the parity block might * still be locked - so just clear all R5_LOCKED flags */ @@ -4151,8 +4119,7 @@ static void handle_stripe_clean_event(struct r5conf *= conf, bio_endio(wbi); wbi =3D wbi2; } - conf->mddev->bitmap_ops->endwrite(conf->mddev, - sh->sector, RAID5_STRIPE_SECTORS(conf)); + if (head_sh->batch_head) { sh =3D list_first_entry(&sh->batch_list, struct stripe_head, @@ -4977,8 +4944,7 @@ static void break_stripe_batch_list(struct stripe_hea= d *head_sh, (1 << STRIPE_COMPUTE_RUN) | (1 << STRIPE_DISCARD) | (1 << STRIPE_BATCH_READY) | - (1 << STRIPE_BATCH_ERR) | - (1 << STRIPE_BITMAP_PENDING)), + (1 << STRIPE_BATCH_ERR)), "stripe state: %lx\n", sh->state); WARN_ONCE(head_sh->state & ((1 << STRIPE_DISCARD) | (1 << STRIPE_REPLACED)), @@ -5869,10 +5835,6 @@ static void make_discard_request(struct mddev *mddev= , struct bio *bi) } spin_unlock_irq(&sh->stripe_lock); if (conf->mddev->bitmap) { - for (d =3D 0; d < conf->raid_disks - conf->max_degraded; - d++) - mddev->bitmap_ops->startwrite(mddev, sh->sector, - RAID5_STRIPE_SECTORS(conf)); sh->bm_seq =3D conf->seq_flush + 1; set_bit(STRIPE_BIT_DELAY, &sh->state); } diff --git a/drivers/md/raid5.h b/drivers/md/raid5.h index 69000fb90bd5..eafc6e9ed6ee 100644 --- a/drivers/md/raid5.h +++ b/drivers/md/raid5.h @@ -371,9 +371,6 @@ enum { STRIPE_ON_RELEASE_LIST, STRIPE_BATCH_READY, STRIPE_BATCH_ERR, - STRIPE_BITMAP_PENDING, /* Being added to bitmap, don't add - * to batch yet. - */ STRIPE_LOG_TRAPPED, /* trapped into log (see raid5-cache.c) * this bit is used in two scenarios: * --=20 2.39.2