From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FB6CCD5BA6 for ; Tue, 19 Sep 2023 12:16:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231781AbjISMQY (ORCPT ); Tue, 19 Sep 2023 08:16:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231564AbjISMQT (ORCPT ); Tue, 19 Sep 2023 08:16:19 -0400 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CC47F2; Tue, 19 Sep 2023 05:16:11 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RqgcM5SfNz4f4Pks; Tue, 19 Sep 2023 20:16:03 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S3; Tue, 19 Sep 2023 20:16:07 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 01/12] ext4: make state in ext4_mb_mark_bb to be bool Date: Wed, 20 Sep 2023 04:15:21 +0800 Message-Id: <20230919201532.310085-2-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S3 X-Coremail-Antispam: 1UD129KBjvJXoWxWw4DJFWUWF1UAFWUWrWDCFg_yoW5Gryrpa sxCF1rCFs5XF1Dua17Gw18Jw1jg34kKF18tryru34F9rZ3tryIqan8tFyjy3W5trWku3W3 XF4jkryUArWfGa7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBjb4IE77IF4wAFF20E14v26r4j6ryUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUGwA2048vs2IY020Ec7CjxVAFwI0_JFI_Gr1l8cAvFVAK 0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW5JVW7JwA2z4 x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVW0oVCq3wA2 z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4 xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v2 6r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x0EwI xGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480 Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7 IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k2 6cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxV AFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07j-Oz3UUUUU= X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" As state could only be either 0 or 1, just make it bool. Signed-off-by: Kemeng Shi --- fs/ext4/ext4.h | 2 +- fs/ext4/fast_commit.c | 8 ++++---- fs/ext4/mballoc.c | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 84618c46f239..88186ea2322f 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -2918,7 +2918,7 @@ extern int ext4_group_add_blocks(handle_t *handle, st= ruct super_block *sb, extern int ext4_trim_fs(struct super_block *, struct fstrim_range *); extern void ext4_process_freed_data(struct super_block *sb, tid_t commit_t= id); extern void ext4_mb_mark_bb(struct super_block *sb, ext4_fsblk_t block, - int len, int state); + int len, bool state); static inline bool ext4_mb_cr_expensive(enum criteria cr) { return cr >=3D CR_GOAL_LEN_SLOW; diff --git a/fs/ext4/fast_commit.c b/fs/ext4/fast_commit.c index b06de728b3b6..87c009e0c59a 100644 --- a/fs/ext4/fast_commit.c +++ b/fs/ext4/fast_commit.c @@ -1806,7 +1806,7 @@ static int ext4_fc_replay_add_range(struct super_bloc= k *sb, * at the end of the FC replay using our array of * modified inodes. */ - ext4_mb_mark_bb(inode->i_sb, map.m_pblk, map.m_len, 0); + ext4_mb_mark_bb(inode->i_sb, map.m_pblk, map.m_len, false); goto next; } =20 @@ -1875,7 +1875,7 @@ ext4_fc_replay_del_range(struct super_block *sb, if (ret > 0) { remaining -=3D ret; cur +=3D ret; - ext4_mb_mark_bb(inode->i_sb, map.m_pblk, map.m_len, 0); + ext4_mb_mark_bb(inode->i_sb, map.m_pblk, map.m_len, false); } else { remaining -=3D map.m_len; cur +=3D map.m_len; @@ -1934,12 +1934,12 @@ static void ext4_fc_set_bitmaps_and_counters(struct= super_block *sb) if (!IS_ERR(path)) { for (j =3D 0; j < path->p_depth; j++) ext4_mb_mark_bb(inode->i_sb, - path[j].p_block, 1, 1); + path[j].p_block, 1, true); ext4_free_ext_path(path); } cur +=3D ret; ext4_mb_mark_bb(inode->i_sb, map.m_pblk, - map.m_len, 1); + map.m_len, true); } else { cur =3D cur + (map.m_len ? map.m_len : 1); } diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 1e599305d85f..cf09adfbaf11 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -4077,7 +4077,7 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_co= ntext *ac, * blocks in bitmaps and update counters. */ void ext4_mb_mark_bb(struct super_block *sb, ext4_fsblk_t block, - int len, int state) + int len, bool state) { struct buffer_head *bitmap_bh =3D NULL; struct ext4_group_desc *gdp; --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D700CCD5BA0 for ; Tue, 19 Sep 2023 12:16:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231933AbjISMQp (ORCPT ); Tue, 19 Sep 2023 08:16:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59124 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231684AbjISMQU (ORCPT ); Tue, 19 Sep 2023 08:16:20 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 195B0FB; Tue, 19 Sep 2023 05:16:12 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RqgcR43PLz4f3xsY; Tue, 19 Sep 2023 20:16:07 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S4; Tue, 19 Sep 2023 20:16:08 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 02/12] ext4: factor out codes to update block bitmap and group descriptor on disk from ext4_mb_mark_bb Date: Wed, 20 Sep 2023 04:15:22 +0800 Message-Id: <20230919201532.310085-3-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S4 X-Coremail-Antispam: 1UD129KBjvJXoWxtrWrZFy7WFW5Kw1DKr4fuFg_yoW7uFWUpF 9IyF1DCr43JrnruF43W34jq3WrGw109w1kGryfC345CF4ayr95WF1xKFyS9as0yFZrZFnF qF1Yya1xur1rWrDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBjb4IE77IF4wAFF20E14v26ryj6rWUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUXwA2048vs2IY020Ec7CjxVAFwI0_Gr0_Xr1l8cAvFVAK 0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW5JVW7JwA2z4 x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVW0oVCq3wA2 z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4 xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v2 6r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x0EwI xGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480 Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7 IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k2 6cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxV AFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jy9aPUUUUU= X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" There are several reasons to add a general function ext4_mb_mark_context to update block bitmap and group descriptor on disk: 1. pair behavior of alloc/free bits. For example, ext4_mb_new_blocks_simple will update free_clusters in struct flex_groups in ext4_mb_mark_bb while ext4_free_blocks_simple forgets this. 2. remove repeat code to read from disk, update and write back to disk. 3. reduce future unit test mocks to catch real IO to update structure on disk. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 147 ++++++++++++++++++++++++---------------------- 1 file changed, 77 insertions(+), 70 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index cf09adfbaf11..e1320eea46e9 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -3953,6 +3953,80 @@ void ext4_exit_mballoc(void) ext4_groupinfo_destroy_slabs(); } =20 +static int +ext4_mb_mark_context(struct super_block *sb, bool state, ext4_group_t grou= p, + ext4_grpblk_t blkoff, ext4_grpblk_t len) +{ + struct ext4_sb_info *sbi =3D EXT4_SB(sb); + struct buffer_head *bitmap_bh =3D NULL; + struct ext4_group_desc *gdp; + struct buffer_head *gdp_bh; + int err; + unsigned int i, already, changed; + + bitmap_bh =3D ext4_read_block_bitmap(sb, group); + if (IS_ERR(bitmap_bh)) + return PTR_ERR(bitmap_bh); + + err =3D -EIO; + gdp =3D ext4_get_group_desc(sb, group, &gdp_bh); + if (!gdp) + goto out_err; + + ext4_lock_group(sb, group); + if (ext4_has_group_desc_csum(sb) && + (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT))) { + gdp->bg_flags &=3D cpu_to_le16(~EXT4_BG_BLOCK_UNINIT); + ext4_free_group_clusters_set(sb, gdp, + ext4_free_clusters_after_init(sb, group, gdp)); + } + + already =3D 0; + for (i =3D 0; i < len; i++) + if (mb_test_bit(blkoff + i, bitmap_bh->b_data) =3D=3D + state) + already++; + changed =3D len - already; + + if (state) { + mb_set_bits(bitmap_bh->b_data, blkoff, len); + ext4_free_group_clusters_set(sb, gdp, + ext4_free_group_clusters(sb, gdp) - changed); + } else { + mb_clear_bits(bitmap_bh->b_data, blkoff, len); + ext4_free_group_clusters_set(sb, gdp, + ext4_free_group_clusters(sb, gdp) + changed); + } + + ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); + ext4_group_desc_csum_set(sb, group, gdp); + ext4_unlock_group(sb, group); + + if (sbi->s_log_groups_per_flex) { + ext4_group_t flex_group =3D ext4_flex_group(sbi, group); + struct flex_groups *fg =3D sbi_array_rcu_deref(sbi, + s_flex_groups, flex_group); + + if (state) + atomic64_sub(changed, &fg->free_clusters); + else + atomic64_add(changed, &fg->free_clusters); + } + + err =3D ext4_handle_dirty_metadata(NULL, NULL, bitmap_bh); + if (err) + goto out_err; + err =3D ext4_handle_dirty_metadata(NULL, NULL, gdp_bh); + if (err) + goto out_err; + + sync_dirty_buffer(bitmap_bh); + sync_dirty_buffer(gdp_bh); + +out_err: + brelse(bitmap_bh); + return err; +} =20 /* * Check quota and mark chosen space (ac->ac_b_ex) non-free in bitmaps @@ -4079,15 +4153,11 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_= context *ac, void ext4_mb_mark_bb(struct super_block *sb, ext4_fsblk_t block, int len, bool state) { - struct buffer_head *bitmap_bh =3D NULL; - struct ext4_group_desc *gdp; - struct buffer_head *gdp_bh; struct ext4_sb_info *sbi =3D EXT4_SB(sb); ext4_group_t group; ext4_grpblk_t blkoff; - int i, err =3D 0; - int already; - unsigned int clen, clen_changed, thisgrp_len; + int err =3D 0; + unsigned int clen, thisgrp_len; =20 while (len > 0) { ext4_get_group_no_and_offset(sb, block, &group, &blkoff); @@ -4108,80 +4178,17 @@ void ext4_mb_mark_bb(struct super_block *sb, ext4_f= sblk_t block, ext4_error(sb, "Marking blocks in system zone - " "Block =3D %llu, len =3D %u", block, thisgrp_len); - bitmap_bh =3D NULL; break; } =20 - bitmap_bh =3D ext4_read_block_bitmap(sb, group); - if (IS_ERR(bitmap_bh)) { - err =3D PTR_ERR(bitmap_bh); - bitmap_bh =3D NULL; - break; - } - - err =3D -EIO; - gdp =3D ext4_get_group_desc(sb, group, &gdp_bh); - if (!gdp) - break; - - ext4_lock_group(sb, group); - already =3D 0; - for (i =3D 0; i < clen; i++) - if (!mb_test_bit(blkoff + i, bitmap_bh->b_data) =3D=3D - !state) - already++; - - clen_changed =3D clen - already; - if (state) - mb_set_bits(bitmap_bh->b_data, blkoff, clen); - else - mb_clear_bits(bitmap_bh->b_data, blkoff, clen); - if (ext4_has_group_desc_csum(sb) && - (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT))) { - gdp->bg_flags &=3D cpu_to_le16(~EXT4_BG_BLOCK_UNINIT); - ext4_free_group_clusters_set(sb, gdp, - ext4_free_clusters_after_init(sb, group, gdp)); - } - if (state) - clen =3D ext4_free_group_clusters(sb, gdp) - clen_changed; - else - clen =3D ext4_free_group_clusters(sb, gdp) + clen_changed; - - ext4_free_group_clusters_set(sb, gdp, clen); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, group, gdp); - - ext4_unlock_group(sb, group); - - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, group); - struct flex_groups *fg =3D sbi_array_rcu_deref(sbi, - s_flex_groups, flex_group); - - if (state) - atomic64_sub(clen_changed, &fg->free_clusters); - else - atomic64_add(clen_changed, &fg->free_clusters); - - } - - err =3D ext4_handle_dirty_metadata(NULL, NULL, bitmap_bh); - if (err) - break; - sync_dirty_buffer(bitmap_bh); - err =3D ext4_handle_dirty_metadata(NULL, NULL, gdp_bh); - sync_dirty_buffer(gdp_bh); + err =3D ext4_mb_mark_context(sb, state, group, blkoff, clen); if (err) break; =20 block +=3D thisgrp_len; len -=3D thisgrp_len; - brelse(bitmap_bh); BUG_ON(len < 0); } - - if (err) - brelse(bitmap_bh); } =20 /* --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F1EDCD5BA8 for ; Tue, 19 Sep 2023 12:16:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231842AbjISMQ3 (ORCPT ); Tue, 19 Sep 2023 08:16:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231660AbjISMQT (ORCPT ); Tue, 19 Sep 2023 08:16:19 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 15F61F7; Tue, 19 Sep 2023 05:16:12 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RqgcQ5kwHz4f41SW; Tue, 19 Sep 2023 20:16:06 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S5; Tue, 19 Sep 2023 20:16:08 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 03/12] ext4: call ext4_mb_mark_context in ext4_free_blocks_simple Date: Wed, 20 Sep 2023 04:15:23 +0800 Message-Id: <20230919201532.310085-4-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S5 X-Coremail-Antispam: 1UD129KBjvJXoW7ury5trWDKFWkWF15GF13CFg_yoW8Zr4kpr ZxAr1xCr1fGr1q9F4xJ34xX3W8Kw18W3WDGrWfW3s5CFW3Ar9agF48KF1Fva4YkFZrZ3ZI vF1Y93y7Zr15W37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBjb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUWwA2048vs2IY020Ec7CjxVAFwI0_Gr0_Xr1l8cAvFVAK 0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJwA2z4 x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVW0oVCq3wA2 z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8wAqx4 xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE14v2 6r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x0EwI xGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480 Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcVC0I7 IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k2 6cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7CjxV AFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jgsqAUUUUU= X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" call ext4_mb_mark_context in ext4_free_blocks_simple to: 1. remove repeat code 2. pair update of free_clusters in ext4_mb_new_blocks_simple. 3. add missing ext4_lock_group/ext4_unlock_group protection. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 33 +-------------------------------- 1 file changed, 1 insertion(+), 32 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index e1320eea46e9..cd2fd5dbfcdd 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6393,43 +6393,12 @@ ext4_mb_free_metadata(handle_t *handle, struct ext4= _buddy *e4b, static void ext4_free_blocks_simple(struct inode *inode, ext4_fsblk_t bloc= k, unsigned long count) { - struct buffer_head *bitmap_bh; struct super_block *sb =3D inode->i_sb; - struct ext4_group_desc *gdp; - struct buffer_head *gdp_bh; ext4_group_t group; ext4_grpblk_t blkoff; - int already_freed =3D 0, err, i; =20 ext4_get_group_no_and_offset(sb, block, &group, &blkoff); - bitmap_bh =3D ext4_read_block_bitmap(sb, group); - if (IS_ERR(bitmap_bh)) { - pr_warn("Failed to read block bitmap\n"); - return; - } - gdp =3D ext4_get_group_desc(sb, group, &gdp_bh); - if (!gdp) - goto err_out; - - for (i =3D 0; i < count; i++) { - if (!mb_test_bit(blkoff + i, bitmap_bh->b_data)) - already_freed++; - } - mb_clear_bits(bitmap_bh->b_data, blkoff, count); - err =3D ext4_handle_dirty_metadata(NULL, NULL, bitmap_bh); - if (err) - goto err_out; - ext4_free_group_clusters_set( - sb, gdp, ext4_free_group_clusters(sb, gdp) + - count - already_freed); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, group, gdp); - ext4_handle_dirty_metadata(NULL, NULL, gdp_bh); - sync_dirty_buffer(bitmap_bh); - sync_dirty_buffer(gdp_bh); - -err_out: - brelse(bitmap_bh); + ext4_mb_mark_context(sb, false, group, blkoff, count); } =20 /** --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D6D8CD5BA6 for ; Tue, 19 Sep 2023 12:16:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231714AbjISMQV (ORCPT ); Tue, 19 Sep 2023 08:16:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229552AbjISMQS (ORCPT ); Tue, 19 Sep 2023 08:16:18 -0400 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C6C6E3; Tue, 19 Sep 2023 05:16:11 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RqgcN4kGSz4f3nqj; Tue, 19 Sep 2023 20:16:04 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S6; Tue, 19 Sep 2023 20:16:08 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 04/12] ext4: extend ext4_mb_mark_context to support allocation under journal Date: Wed, 20 Sep 2023 04:15:24 +0800 Message-Id: <20230919201532.310085-5-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S6 X-Coremail-Antispam: 1UD129KBjvJXoW3Jr4DtF4rAFWDJF1ktF1DWrg_yoW7GF1Dpr WDAr1qkr43KrnI9FsrA342q3WfGw1Ikw1DGrWfG3s5CrWxtr95XF4fKFyFyF4YyF47Z3Z8 XF1Yy34xGr4rG37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBFb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVW0oVCq 3wA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8w Aqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE 14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x 0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E 7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcV C0I7IYx2IY67AKxVWUJVWUCwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF 04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7 CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jstxDUUUUU= X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Previously, ext4_mb_mark_context is only called under fast commit replay path, so there is no valid handle when we update block bitmap and group descriptor. This patch try to extend ext4_mb_mark_context to be used by code under journal. There are several improvement: 1. add "handle_t *handle" to struct ext4_mark_context to journal block bitmap and group descriptor update inside ext4_mb_mark_context (the added journal code is based on ext4_mb_mark_diskspace_used where ext4_mb_mark_context is going to be used.) 2. add EXT4_MB_BITMAP_MARKED_CHECK flag to control check if bits in block bitmap are already marked as allocation code under journal asserts that all bits to be changed are not marked before. 3. add "ext4_grpblk_t changed" to struct ext4_mark_context to notify number of bits in block bitmap has changed. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 64 ++++++++++++++++++++++++++++++++++++----------- 1 file changed, 49 insertions(+), 15 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index cd2fd5dbfcdd..bdf07b2904ca 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -3953,26 +3953,47 @@ void ext4_exit_mballoc(void) ext4_groupinfo_destroy_slabs(); } =20 +#define EXT4_MB_BITMAP_MARKED_CHECK 0x0001 +#define EXT4_MB_SYNC_UPDATE 0x0002 static int -ext4_mb_mark_context(struct super_block *sb, bool state, ext4_group_t grou= p, - ext4_grpblk_t blkoff, ext4_grpblk_t len) +ext4_mb_mark_context(handle_t *handle, struct super_block *sb, bool state, + ext4_group_t group, ext4_grpblk_t blkoff, + ext4_grpblk_t len, int flags, ext4_grpblk_t *ret_changed) { struct ext4_sb_info *sbi =3D EXT4_SB(sb); struct buffer_head *bitmap_bh =3D NULL; struct ext4_group_desc *gdp; struct buffer_head *gdp_bh; int err; - unsigned int i, already, changed; + unsigned int i, already, changed =3D len; =20 + if (ret_changed) + *ret_changed =3D 0; bitmap_bh =3D ext4_read_block_bitmap(sb, group); if (IS_ERR(bitmap_bh)) return PTR_ERR(bitmap_bh); =20 + if (handle) { + BUFFER_TRACE(bitmap_bh, "getting write access"); + err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, + EXT4_JTR_NONE); + if (err) + goto out_err; + } + err =3D -EIO; gdp =3D ext4_get_group_desc(sb, group, &gdp_bh); if (!gdp) goto out_err; =20 + if (handle) { + BUFFER_TRACE(gdp_bh, "get_write_access"); + err =3D ext4_journal_get_write_access(handle, sb, gdp_bh, + EXT4_JTR_NONE); + if (err) + goto out_err; + } + ext4_lock_group(sb, group); if (ext4_has_group_desc_csum(sb) && (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT))) { @@ -3981,12 +4002,14 @@ ext4_mb_mark_context(struct super_block *sb, bool s= tate, ext4_group_t group, ext4_free_clusters_after_init(sb, group, gdp)); } =20 - already =3D 0; - for (i =3D 0; i < len; i++) - if (mb_test_bit(blkoff + i, bitmap_bh->b_data) =3D=3D - state) - already++; - changed =3D len - already; + if (flags & EXT4_MB_BITMAP_MARKED_CHECK) { + already =3D 0; + for (i =3D 0; i < len; i++) + if (mb_test_bit(blkoff + i, bitmap_bh->b_data) =3D=3D + state) + already++; + changed =3D len - already; + } =20 if (state) { mb_set_bits(bitmap_bh->b_data, blkoff, len); @@ -4001,6 +4024,8 @@ ext4_mb_mark_context(struct super_block *sb, bool sta= te, ext4_group_t group, ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); ext4_group_desc_csum_set(sb, group, gdp); ext4_unlock_group(sb, group); + if (ret_changed) + *ret_changed =3D changed; =20 if (sbi->s_log_groups_per_flex) { ext4_group_t flex_group =3D ext4_flex_group(sbi, group); @@ -4013,15 +4038,17 @@ ext4_mb_mark_context(struct super_block *sb, bool s= tate, ext4_group_t group, atomic64_add(changed, &fg->free_clusters); } =20 - err =3D ext4_handle_dirty_metadata(NULL, NULL, bitmap_bh); + err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); if (err) goto out_err; - err =3D ext4_handle_dirty_metadata(NULL, NULL, gdp_bh); + err =3D ext4_handle_dirty_metadata(handle, NULL, gdp_bh); if (err) goto out_err; =20 - sync_dirty_buffer(bitmap_bh); - sync_dirty_buffer(gdp_bh); + if (flags & EXT4_MB_SYNC_UPDATE) { + sync_dirty_buffer(bitmap_bh); + sync_dirty_buffer(gdp_bh); + } =20 out_err: brelse(bitmap_bh); @@ -4181,7 +4208,11 @@ void ext4_mb_mark_bb(struct super_block *sb, ext4_fs= blk_t block, break; } =20 - err =3D ext4_mb_mark_context(sb, state, group, blkoff, clen); + err =3D ext4_mb_mark_context(NULL, sb, state, + group, blkoff, clen, + EXT4_MB_BITMAP_MARKED_CHECK | + EXT4_MB_SYNC_UPDATE, + NULL); if (err) break; =20 @@ -6398,7 +6429,10 @@ static void ext4_free_blocks_simple(struct inode *in= ode, ext4_fsblk_t block, ext4_grpblk_t blkoff; =20 ext4_get_group_no_and_offset(sb, block, &group, &blkoff); - ext4_mb_mark_context(sb, false, group, blkoff, count); + ext4_mb_mark_context(NULL, sb, false, group, blkoff, count, + EXT4_MB_BITMAP_MARKED_CHECK | + EXT4_MB_SYNC_UPDATE, + NULL); } =20 /** --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A006CD5BA6 for ; Tue, 19 Sep 2023 12:16:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231796AbjISMQk (ORCPT ); Tue, 19 Sep 2023 08:16:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231669AbjISMQT (ORCPT ); Tue, 19 Sep 2023 08:16:19 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16E75F9; Tue, 19 Sep 2023 05:16:12 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RqgcS3G09z4f3q3m; Tue, 19 Sep 2023 20:16:08 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S7; Tue, 19 Sep 2023 20:16:09 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 05/12] ext4: call ext4_mb_mark_context in ext4_mb_mark_diskspace_used Date: Wed, 20 Sep 2023 04:15:25 +0800 Message-Id: <20230919201532.310085-6-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S7 X-Coremail-Antispam: 1UD129KBjvJXoW3XF4rWryrWF4rZFW7Zr43Wrg_yoW7Gw1kpF nIyF1DCr1fWr1DuFWI934DXF1rKw48G3WrG34xGr1fCF12krZ8Aay8ta40ya9rKFW7A3Wq vF4Yya4UCr47WrDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBFb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVW0oVCq 3wA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8w Aqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE 14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x 0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E 7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcV C0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF 04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7 CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jstxDUUUUU= X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" call ext4_mb_mark_context in ext4_mb_mark_diskspace_used to: 1. remove repeat code to normally update bitmap and group descriptor on disk. 2. call ext4_mb_mark_context instead of only setting bits in block bitmap to fix the bitmap. Function ext4_mb_mark_context will also update checksum of bitmap and other counter along with the bit change to keep the cosistent with bit change or block bitmap will be marked corrupted as checksum of bitmap is in inconsistent state. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 86 +++++++++++------------------------------------ 1 file changed, 20 insertions(+), 66 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index bdf07b2904ca..87e5922997fb 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -4063,13 +4063,13 @@ static noinline_for_stack int ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac, handle_t *handle, unsigned int reserv_clstrs) { - struct buffer_head *bitmap_bh =3D NULL; struct ext4_group_desc *gdp; - struct buffer_head *gdp_bh; struct ext4_sb_info *sbi; struct super_block *sb; ext4_fsblk_t block; int err, len; + int flags =3D 0; + ext4_grpblk_t changed; =20 BUG_ON(ac->ac_status !=3D AC_STATUS_FOUND); BUG_ON(ac->ac_b_ex.fe_len <=3D 0); @@ -4077,32 +4077,13 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_= context *ac, sb =3D ac->ac_sb; sbi =3D EXT4_SB(sb); =20 - bitmap_bh =3D ext4_read_block_bitmap(sb, ac->ac_b_ex.fe_group); - if (IS_ERR(bitmap_bh)) { - return PTR_ERR(bitmap_bh); - } - - BUFFER_TRACE(bitmap_bh, "getting write access"); - err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, - EXT4_JTR_NONE); - if (err) - goto out_err; - - err =3D -EIO; - gdp =3D ext4_get_group_desc(sb, ac->ac_b_ex.fe_group, &gdp_bh); + gdp =3D ext4_get_group_desc(sb, ac->ac_b_ex.fe_group, NULL); if (!gdp) - goto out_err; - + return -EIO; ext4_debug("using block group %u(%d)\n", ac->ac_b_ex.fe_group, ext4_free_group_clusters(sb, gdp)); =20 - BUFFER_TRACE(gdp_bh, "get_write_access"); - err =3D ext4_journal_get_write_access(handle, sb, gdp_bh, EXT4_JTR_NONE); - if (err) - goto out_err; - block =3D ext4_grp_offs_to_block(sb, &ac->ac_b_ex); - len =3D EXT4_C2B(sbi, ac->ac_b_ex.fe_len); if (!ext4_inode_block_valid(ac->ac_inode, block, len)) { ext4_error(sb, "Allocating blocks %llu-%llu which overlap " @@ -4111,41 +4092,29 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_= context *ac, * Fix the bitmap and return EFSCORRUPTED * We leak some of the blocks here. */ - ext4_lock_group(sb, ac->ac_b_ex.fe_group); - mb_set_bits(bitmap_bh->b_data, ac->ac_b_ex.fe_start, - ac->ac_b_ex.fe_len); - ext4_unlock_group(sb, ac->ac_b_ex.fe_group); - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); + err =3D ext4_mb_mark_context(handle, sb, true, + ac->ac_b_ex.fe_group, + ac->ac_b_ex.fe_start, + ac->ac_b_ex.fe_len, + 0, NULL); if (!err) err =3D -EFSCORRUPTED; - goto out_err; + return err; } =20 - ext4_lock_group(sb, ac->ac_b_ex.fe_group); #ifdef AGGRESSIVE_CHECK - { - int i; - for (i =3D 0; i < ac->ac_b_ex.fe_len; i++) { - BUG_ON(mb_test_bit(ac->ac_b_ex.fe_start + i, - bitmap_bh->b_data)); - } - } + flags |=3D EXT4_MB_BITMAP_MARKED_CHECK; #endif - mb_set_bits(bitmap_bh->b_data, ac->ac_b_ex.fe_start, - ac->ac_b_ex.fe_len); - if (ext4_has_group_desc_csum(sb) && - (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT))) { - gdp->bg_flags &=3D cpu_to_le16(~EXT4_BG_BLOCK_UNINIT); - ext4_free_group_clusters_set(sb, gdp, - ext4_free_clusters_after_init(sb, - ac->ac_b_ex.fe_group, gdp)); - } - len =3D ext4_free_group_clusters(sb, gdp) - ac->ac_b_ex.fe_len; - ext4_free_group_clusters_set(sb, gdp, len); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, ac->ac_b_ex.fe_group, gdp); + err =3D ext4_mb_mark_context(handle, sb, true, ac->ac_b_ex.fe_group, + ac->ac_b_ex.fe_start, ac->ac_b_ex.fe_len, + flags, &changed); + + if (err && changed =3D=3D 0) + return err; =20 - ext4_unlock_group(sb, ac->ac_b_ex.fe_group); +#ifdef AGGRESSIVE_CHECK + BUG_ON(changed !=3D ac->ac_b_ex.fe_len); +#endif percpu_counter_sub(&sbi->s_freeclusters_counter, ac->ac_b_ex.fe_len); /* * Now reduce the dirty block count also. Should not go negative @@ -4155,21 +4124,6 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_c= ontext *ac, percpu_counter_sub(&sbi->s_dirtyclusters_counter, reserv_clstrs); =20 - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, - ac->ac_b_ex.fe_group); - atomic64_sub(ac->ac_b_ex.fe_len, - &sbi_array_rcu_deref(sbi, s_flex_groups, - flex_group)->free_clusters); - } - - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); - if (err) - goto out_err; - err =3D ext4_handle_dirty_metadata(handle, NULL, gdp_bh); - -out_err: - brelse(bitmap_bh); return err; } =20 --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA05FCD5BA0 for ; Tue, 19 Sep 2023 12:16:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231948AbjISMQs (ORCPT ); Tue, 19 Sep 2023 08:16:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59092 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231702AbjISMQU (ORCPT ); Tue, 19 Sep 2023 08:16:20 -0400 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46564FC; Tue, 19 Sep 2023 05:16:12 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RqgcP27C7z4f3s6B; Tue, 19 Sep 2023 20:16:05 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S8; Tue, 19 Sep 2023 20:16:09 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 06/12] ext4: Separate block bitmap and buddy bitmap freeing in ext4_mb_clear_bb() Date: Wed, 20 Sep 2023 04:15:26 +0800 Message-Id: <20230919201532.310085-7-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S8 X-Coremail-Antispam: 1UD129KBjvJXoW3WFWDAF4fCry7JFWftry3CFg_yoW3Jr45pr 9FyFnrCrn5GrnF9F40k34jq3WSkw48ua1DGrW3uryxCr1ayr9akFZ7tF93uFWUtFZ7X3Wq qr1Y93y8Cr4ag37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBFb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVW0oVCq 3wA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8w Aqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE 14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x 0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E 7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcV C0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF 04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7 CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jstxDUUUUU= X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch separates block bitmap and buddy bitmap freeing in order to update block bitmap with ext4_mb_mark_context in following patch. Separated freeing is safe with concurrent allocation as long as: 1. Firstly allocate block in buddy bitmap, and then in block bitmap. 2. Firstly free block in block bitmap, and then buddy bitmap. Then freed block will only be available to allocation when both buddy bitmap and block bitmap are updated by freeing. Allocation obeys rule 1 already, just do sperated freeing with rule 2. Separated freeing has no race with generate_buddy as: Once ext4_mb_load_buddy_gfp is executed successfully, the update-to-date buddy page can be found in sbi->s_buddy_cache and no more buddy initialization of the buddy page will be executed concurrently until buddy page is unloaded. As we always do free in "load buddy, free, unload buddy" sequence, separated freeing has no race with generate_buddy. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 98 +++++++++++++++++++++++------------------------ 1 file changed, 49 insertions(+), 49 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 87e5922997fb..8c1127aa7f59 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6423,7 +6423,7 @@ static void ext4_mb_clear_bb(handle_t *handle, struct= inode *inode, ext4_error(sb, "Freeing blocks in system zone - " "Block =3D %llu, count =3D %lu", block, count); /* err =3D 0. ext4_std_error should be a no op */ - goto error_return; + goto error_out; } flags |=3D EXT4_FREE_BLOCKS_VALIDATED; =20 @@ -6447,31 +6447,39 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, flags &=3D ~EXT4_FREE_BLOCKS_VALIDATED; } count_clusters =3D EXT4_NUM_B2C(sbi, count); + trace_ext4_mballoc_free(sb, inode, block_group, bit, count_clusters); + + /* __GFP_NOFAIL: retry infinitely, ignore TIF_MEMDIE and memcg limit. */ + err =3D ext4_mb_load_buddy_gfp(sb, block_group, &e4b, + GFP_NOFS|__GFP_NOFAIL); + if (err) + goto error_out; + + if (!(flags & EXT4_FREE_BLOCKS_VALIDATED) && + !ext4_inode_block_valid(inode, block, count)) { + ext4_error(sb, "Freeing blocks in system zone - " + "Block =3D %llu, count =3D %lu", block, count); + /* err =3D 0. ext4_std_error should be a no op */ + goto error_clean; + } + bitmap_bh =3D ext4_read_block_bitmap(sb, block_group); if (IS_ERR(bitmap_bh)) { err =3D PTR_ERR(bitmap_bh); bitmap_bh =3D NULL; - goto error_return; + goto error_clean; } gdp =3D ext4_get_group_desc(sb, block_group, &gd_bh); if (!gdp) { err =3D -EIO; - goto error_return; - } - - if (!(flags & EXT4_FREE_BLOCKS_VALIDATED) && - !ext4_inode_block_valid(inode, block, count)) { - ext4_error(sb, "Freeing blocks in system zone - " - "Block =3D %llu, count =3D %lu", block, count); - /* err =3D 0. ext4_std_error should be a no op */ - goto error_return; + goto error_clean; } =20 BUFFER_TRACE(bitmap_bh, "getting write access"); err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, EXT4_JTR_NONE); if (err) - goto error_return; + goto error_clean; =20 /* * We are about to modify some metadata. Call the journal APIs @@ -6481,7 +6489,7 @@ static void ext4_mb_clear_bb(handle_t *handle, struct= inode *inode, BUFFER_TRACE(gd_bh, "get_write_access"); err =3D ext4_journal_get_write_access(handle, sb, gd_bh, EXT4_JTR_NONE); if (err) - goto error_return; + goto error_clean; #ifdef AGGRESSIVE_CHECK { int i; @@ -6489,13 +6497,30 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, BUG_ON(!mb_test_bit(bit + i, bitmap_bh->b_data)); } #endif - trace_ext4_mballoc_free(sb, inode, block_group, bit, count_clusters); + ext4_lock_group(sb, block_group); + mb_clear_bits(bitmap_bh->b_data, bit, count_clusters); + ret =3D ext4_free_group_clusters(sb, gdp) + count_clusters; + ext4_free_group_clusters_set(sb, gdp, ret); + ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); + ext4_group_desc_csum_set(sb, block_group, gdp); + ext4_unlock_group(sb, block_group); =20 - /* __GFP_NOFAIL: retry infinitely, ignore TIF_MEMDIE and memcg limit. */ - err =3D ext4_mb_load_buddy_gfp(sb, block_group, &e4b, - GFP_NOFS|__GFP_NOFAIL); - if (err) - goto error_return; + if (sbi->s_log_groups_per_flex) { + ext4_group_t flex_group =3D ext4_flex_group(sbi, block_group); + atomic64_add(count_clusters, + &sbi_array_rcu_deref(sbi, s_flex_groups, + flex_group)->free_clusters); + } + + /* We dirtied the bitmap block */ + BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); + err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); + + /* And the group descriptor block */ + BUFFER_TRACE(gd_bh, "dirtied group descriptor block"); + ret =3D ext4_handle_dirty_metadata(handle, NULL, gd_bh); + if (!err) + err =3D ret; =20 /* * We need to make sure we don't reuse the freed block until after the @@ -6519,13 +6544,8 @@ static void ext4_mb_clear_bb(handle_t *handle, struc= t inode *inode, new_entry->efd_tid =3D handle->h_transaction->t_tid; =20 ext4_lock_group(sb, block_group); - mb_clear_bits(bitmap_bh->b_data, bit, count_clusters); ext4_mb_free_metadata(handle, &e4b, new_entry); } else { - /* need to update group_info->bb_free and bitmap - * with group lock held. generate_buddy look at - * them with group lock_held - */ if (test_opt(sb, DISCARD)) { err =3D ext4_issue_discard(sb, block_group, bit, count_clusters, NULL); @@ -6538,23 +6558,11 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, EXT4_MB_GRP_CLEAR_TRIMMED(e4b.bd_info); =20 ext4_lock_group(sb, block_group); - mb_clear_bits(bitmap_bh->b_data, bit, count_clusters); mb_free_blocks(inode, &e4b, bit, count_clusters); } =20 - ret =3D ext4_free_group_clusters(sb, gdp) + count_clusters; - ext4_free_group_clusters_set(sb, gdp, ret); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, block_group, gdp); ext4_unlock_group(sb, block_group); =20 - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, block_group); - atomic64_add(count_clusters, - &sbi_array_rcu_deref(sbi, s_flex_groups, - flex_group)->free_clusters); - } - /* * on a bigalloc file system, defer the s_freeclusters_counter * update to the caller (ext4_remove_space and friends) so they @@ -6567,28 +6575,20 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, count_clusters); } =20 - ext4_mb_unload_buddy(&e4b); - - /* We dirtied the bitmap block */ - BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); - - /* And the group descriptor block */ - BUFFER_TRACE(gd_bh, "dirtied group descriptor block"); - ret =3D ext4_handle_dirty_metadata(handle, NULL, gd_bh); - if (!err) - err =3D ret; - if (overflow && !err) { block +=3D count; count =3D overflow; + ext4_mb_unload_buddy(&e4b); put_bh(bitmap_bh); /* The range changed so it's no longer validated */ flags &=3D ~EXT4_FREE_BLOCKS_VALIDATED; goto do_more; } -error_return: + +error_clean: + ext4_mb_unload_buddy(&e4b); brelse(bitmap_bh); +error_out: ext4_std_error(sb, err); } =20 --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16739CD5BA6 for ; Tue, 19 Sep 2023 12:16:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231921AbjISMQn (ORCPT ); Tue, 19 Sep 2023 08:16:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231683AbjISMQT (ORCPT ); Tue, 19 Sep 2023 08:16:19 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A99B1F4; Tue, 19 Sep 2023 05:16:12 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RqgcR6wgmz4f41Sr; Tue, 19 Sep 2023 20:16:07 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S9; Tue, 19 Sep 2023 20:16:09 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 07/12] ext4: call ext4_mb_mark_context in ext4_mb_clear_bb Date: Wed, 20 Sep 2023 04:15:27 +0800 Message-Id: <20230919201532.310085-8-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S9 X-Coremail-Antispam: 1UD129KBjvJXoWxJw4fJw13Xr1DJr4DCF15Jwb_yoWrGF1xpr nxtFnrCr45GFnruF40k34UXF1fKw18W3WkJrW3CryfCrW2yr93KFyxtF1FvFW5tFZrX3Za qF1Y9348ur47W37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBFb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVWxJVW8Jr1l84ACjcxK6I8E87Iv67AKxVW0oVCq 3wA2z4x0Y4vEx4A2jsIEc7CjxVAFwI0_GcCE3s1le2I262IYc4CY6c8Ij28IcVAaY2xG8w Aqx4xG64xvF2IEw4CE5I8CrVC2j2WlYx0E2Ix0cI8IcVAFwI0_Jr0_Jr4lYx0Ex4A2jsIE 14v26r1j6r4UMcvjeVCFs4IE7xkEbVWUJVW8JwACjcxG0xvY0x0EwIxGrwCF04k20xvY0x 0EwIxGrwCFx2IqxVCFs4IE7xkEbVWUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E 7480Y4vE14v26r106r1rMI8E67AF67kF1VAFwI0_JF0_Jw1lIxkGc2Ij64vIr41lIxAIcV C0I7IYx2IY67AKxVWUCVW8JwCI42IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF 04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r1j6r4UMIIF0xvEx4A2jsIEc7 CjxVAFwI0_Gr0_Gr1UYxBIdaVFxhVjvjDU0xZFpf9x07jstxDUUUUU= X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Call ext4_mb_mark_context in ext4_mb_clear_bb to remove repeat code. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 69 +++++++---------------------------------------- 1 file changed, 10 insertions(+), 59 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 8c1127aa7f59..730397af6975 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6402,19 +6402,17 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, ext4_fsblk_t block, unsigned long count, int flags) { - struct buffer_head *bitmap_bh =3D NULL; struct super_block *sb =3D inode->i_sb; - struct ext4_group_desc *gdp; struct ext4_group_info *grp; unsigned int overflow; ext4_grpblk_t bit; - struct buffer_head *gd_bh; ext4_group_t block_group; struct ext4_sb_info *sbi; struct ext4_buddy e4b; unsigned int count_clusters; int err =3D 0; - int ret; + int mark_flags =3D 0; + ext4_grpblk_t changed; =20 sbi =3D EXT4_SB(sb); =20 @@ -6463,64 +6461,19 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, goto error_clean; } =20 - bitmap_bh =3D ext4_read_block_bitmap(sb, block_group); - if (IS_ERR(bitmap_bh)) { - err =3D PTR_ERR(bitmap_bh); - bitmap_bh =3D NULL; - goto error_clean; - } - gdp =3D ext4_get_group_desc(sb, block_group, &gd_bh); - if (!gdp) { - err =3D -EIO; - goto error_clean; - } - - BUFFER_TRACE(bitmap_bh, "getting write access"); - err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, - EXT4_JTR_NONE); - if (err) - goto error_clean; - - /* - * We are about to modify some metadata. Call the journal APIs - * to unshare ->b_data if a currently-committing transaction is - * using it - */ - BUFFER_TRACE(gd_bh, "get_write_access"); - err =3D ext4_journal_get_write_access(handle, sb, gd_bh, EXT4_JTR_NONE); - if (err) - goto error_clean; #ifdef AGGRESSIVE_CHECK - { - int i; - for (i =3D 0; i < count_clusters; i++) - BUG_ON(!mb_test_bit(bit + i, bitmap_bh->b_data)); - } + mark_flags |=3D EXT4_MB_BITMAP_MARKED_CHECK; #endif - ext4_lock_group(sb, block_group); - mb_clear_bits(bitmap_bh->b_data, bit, count_clusters); - ret =3D ext4_free_group_clusters(sb, gdp) + count_clusters; - ext4_free_group_clusters_set(sb, gdp, ret); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, block_group, gdp); - ext4_unlock_group(sb, block_group); + err =3D ext4_mb_mark_context(handle, sb, false, block_group, bit, + count_clusters, mark_flags, &changed); =20 - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, block_group); - atomic64_add(count_clusters, - &sbi_array_rcu_deref(sbi, s_flex_groups, - flex_group)->free_clusters); - } =20 - /* We dirtied the bitmap block */ - BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); + if (err && changed =3D=3D 0) + goto error_clean; =20 - /* And the group descriptor block */ - BUFFER_TRACE(gd_bh, "dirtied group descriptor block"); - ret =3D ext4_handle_dirty_metadata(handle, NULL, gd_bh); - if (!err) - err =3D ret; +#ifdef AGGRESSIVE_CHECK + BUG_ON(changed !=3D count_clusters); +#endif =20 /* * We need to make sure we don't reuse the freed block until after the @@ -6579,7 +6532,6 @@ static void ext4_mb_clear_bb(handle_t *handle, struct= inode *inode, block +=3D count; count =3D overflow; ext4_mb_unload_buddy(&e4b); - put_bh(bitmap_bh); /* The range changed so it's no longer validated */ flags &=3D ~EXT4_FREE_BLOCKS_VALIDATED; goto do_more; @@ -6587,7 +6539,6 @@ static void ext4_mb_clear_bb(handle_t *handle, struct= inode *inode, =20 error_clean: ext4_mb_unload_buddy(&e4b); - brelse(bitmap_bh); error_out: ext4_std_error(sb, err); } --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1493FCD5BA6 for ; Tue, 19 Sep 2023 12:16:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231775AbjISMQz (ORCPT ); Tue, 19 Sep 2023 08:16:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231687AbjISMQU (ORCPT ); Tue, 19 Sep 2023 08:16:20 -0400 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67C57100; Tue, 19 Sep 2023 05:16:13 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RqgcP6JkXz4f3s63; Tue, 19 Sep 2023 20:16:05 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S10; Tue, 19 Sep 2023 20:16:10 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 08/12] ext4: Separate block bitmap and buddy bitmap freeing in ext4_group_add_blocks() Date: Wed, 20 Sep 2023 04:15:28 +0800 Message-Id: <20230919201532.310085-9-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S10 X-Coremail-Antispam: 1UD129KBjvJXoWxCr1fZrWfWr17KryrZw47Jwb_yoW7Jr4rpr y2krn5Cr1rGrnrZF4xA34jq3W0kw48Ca1UKrWfur1xCrZFyFnakFyktFn5ZFWUtFs7A3Zr XryY9348ur12gr7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBIb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_JFI_Gr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY 6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aV CY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU0TqcUUUUUU== X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch separates block bitmap and buddy bitmap freeing in order to udpate block bitmap with ext4_mb_mark_context in following patch. The reason why this can be sperated is explained in previous submit. Put the explanation here to simplify the code archeology to ext4_group_add_blocks(): Separated freeing is safe with concurrent allocation as long as: 1. Firstly allocate block in buddy bitmap, and then in block bitmap. 2. Firstly free block in block bitmap, and then buddy bitmap. Then freed block will only be available to allocation when both buddy bitmap and block bitmap are updated by freeing. Allocation obeys rule 1 already, just do sperated freeing with rule 2. Separated freeing has no race with generate_buddy as: Once ext4_mb_load_buddy_gfp is executed successfully, the update-to-date buddy page can be found in sbi->s_buddy_cache and no more buddy initialization of the buddy page will be executed concurrently until buddy page is unloaded. As we always do free in "load buddy, free, unload buddy" sequence, separated freeing has no race with generate_buddy. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 54 +++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 730397af6975..404121445fc3 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6685,35 +6685,39 @@ int ext4_group_add_blocks(handle_t *handle, struct = super_block *sb, ext4_warning(sb, "too many blocks added to group %u", block_group); err =3D -EINVAL; - goto error_return; + goto error_out; + } + + err =3D ext4_mb_load_buddy(sb, block_group, &e4b); + if (err) + goto error_out; + + if (!ext4_sb_block_valid(sb, NULL, block, count)) { + ext4_error(sb, "Adding blocks in system zones - " + "Block =3D %llu, count =3D %lu", + block, count); + err =3D -EINVAL; + goto error_clean; } =20 bitmap_bh =3D ext4_read_block_bitmap(sb, block_group); if (IS_ERR(bitmap_bh)) { err =3D PTR_ERR(bitmap_bh); bitmap_bh =3D NULL; - goto error_return; + goto error_clean; } =20 desc =3D ext4_get_group_desc(sb, block_group, &gd_bh); if (!desc) { err =3D -EIO; - goto error_return; - } - - if (!ext4_sb_block_valid(sb, NULL, block, count)) { - ext4_error(sb, "Adding blocks in system zones - " - "Block =3D %llu, count =3D %lu", - block, count); - err =3D -EINVAL; - goto error_return; + goto error_clean; } =20 BUFFER_TRACE(bitmap_bh, "getting write access"); err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, EXT4_JTR_NONE); if (err) - goto error_return; + goto error_clean; =20 /* * We are about to modify some metadata. Call the journal APIs @@ -6723,7 +6727,7 @@ int ext4_group_add_blocks(handle_t *handle, struct su= per_block *sb, BUFFER_TRACE(gd_bh, "get_write_access"); err =3D ext4_journal_get_write_access(handle, sb, gd_bh, EXT4_JTR_NONE); if (err) - goto error_return; + goto error_clean; =20 for (i =3D 0, clusters_freed =3D 0; i < cluster_count; i++) { BUFFER_TRACE(bitmap_bh, "clear bit"); @@ -6736,26 +6740,14 @@ int ext4_group_add_blocks(handle_t *handle, struct = super_block *sb, } } =20 - err =3D ext4_mb_load_buddy(sb, block_group, &e4b); - if (err) - goto error_return; - - /* - * need to update group_info->bb_free and bitmap - * with group lock held. generate_buddy look at - * them with group lock_held - */ ext4_lock_group(sb, block_group); mb_clear_bits(bitmap_bh->b_data, bit, cluster_count); - mb_free_blocks(NULL, &e4b, bit, cluster_count); free_clusters_count =3D clusters_freed + ext4_free_group_clusters(sb, desc); ext4_free_group_clusters_set(sb, desc, free_clusters_count); ext4_block_bitmap_csum_set(sb, desc, bitmap_bh); ext4_group_desc_csum_set(sb, block_group, desc); ext4_unlock_group(sb, block_group); - percpu_counter_add(&sbi->s_freeclusters_counter, - clusters_freed); =20 if (sbi->s_log_groups_per_flex) { ext4_group_t flex_group =3D ext4_flex_group(sbi, block_group); @@ -6764,8 +6756,6 @@ int ext4_group_add_blocks(handle_t *handle, struct su= per_block *sb, flex_group)->free_clusters); } =20 - ext4_mb_unload_buddy(&e4b); - /* We dirtied the bitmap block */ BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); @@ -6776,8 +6766,16 @@ int ext4_group_add_blocks(handle_t *handle, struct s= uper_block *sb, if (!err) err =3D ret; =20 -error_return: + ext4_lock_group(sb, block_group); + mb_free_blocks(NULL, &e4b, bit, cluster_count); + ext4_unlock_group(sb, block_group); + percpu_counter_add(&sbi->s_freeclusters_counter, + clusters_freed); + +error_clean: brelse(bitmap_bh); + ext4_mb_unload_buddy(&e4b); +error_out: ext4_std_error(sb, err); return err; } --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17BDDCD5BA7 for ; Tue, 19 Sep 2023 12:16:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231971AbjISMQ5 (ORCPT ); Tue, 19 Sep 2023 08:16:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231736AbjISMQV (ORCPT ); Tue, 19 Sep 2023 08:16:21 -0400 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8225711C; Tue, 19 Sep 2023 05:16:15 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RqgcQ1GYsz4f3tCf; Tue, 19 Sep 2023 20:16:06 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S11; Tue, 19 Sep 2023 20:16:10 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 09/12] ext4: call ext4_mb_mark_context in ext4_group_add_blocks() Date: Wed, 20 Sep 2023 04:15:29 +0800 Message-Id: <20230919201532.310085-10-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S11 X-Coremail-Antispam: 1UD129KBjvJXoWxCFWrGF13KFWfKFWUJF1UWrg_yoWrAF17pr nIyFnFkryfGrnrWF48J34jq3WrKw48C3W7GrW3CryfCF9Fyrn3CF97tFnY9FWDtFZ7ZFnr XF1Y9348ursxW37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBIb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_JFI_Gr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY 6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aV CY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU0TqcUUUUUU== X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Call ext4_mb_mark_context in ext4_group_add_blocks() to remove repeat code. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 82 ++++++----------------------------------------- 1 file changed, 10 insertions(+), 72 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 404121445fc3..db0b15098a80 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6657,23 +6657,19 @@ void ext4_free_blocks(handle_t *handle, struct inod= e *inode, int ext4_group_add_blocks(handle_t *handle, struct super_block *sb, ext4_fsblk_t block, unsigned long count) { - struct buffer_head *bitmap_bh =3D NULL; - struct buffer_head *gd_bh; ext4_group_t block_group; ext4_grpblk_t bit; - unsigned int i; - struct ext4_group_desc *desc; struct ext4_sb_info *sbi =3D EXT4_SB(sb); struct ext4_buddy e4b; - int err =3D 0, ret, free_clusters_count; - ext4_grpblk_t clusters_freed; + int err =3D 0; ext4_fsblk_t first_cluster =3D EXT4_B2C(sbi, block); ext4_fsblk_t last_cluster =3D EXT4_B2C(sbi, block + count - 1); unsigned long cluster_count =3D last_cluster - first_cluster + 1; + ext4_grpblk_t changed; =20 ext4_debug("Adding block(s) %llu-%llu\n", block, block + count - 1); =20 - if (count =3D=3D 0) + if (cluster_count =3D=3D 0) return 0; =20 ext4_get_group_no_and_offset(sb, block, &block_group, &bit); @@ -6700,80 +6696,22 @@ int ext4_group_add_blocks(handle_t *handle, struct = super_block *sb, goto error_clean; } =20 - bitmap_bh =3D ext4_read_block_bitmap(sb, block_group); - if (IS_ERR(bitmap_bh)) { - err =3D PTR_ERR(bitmap_bh); - bitmap_bh =3D NULL; - goto error_clean; - } - - desc =3D ext4_get_group_desc(sb, block_group, &gd_bh); - if (!desc) { - err =3D -EIO; - goto error_clean; - } - - BUFFER_TRACE(bitmap_bh, "getting write access"); - err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, - EXT4_JTR_NONE); - if (err) - goto error_clean; - - /* - * We are about to modify some metadata. Call the journal APIs - * to unshare ->b_data if a currently-committing transaction is - * using it - */ - BUFFER_TRACE(gd_bh, "get_write_access"); - err =3D ext4_journal_get_write_access(handle, sb, gd_bh, EXT4_JTR_NONE); - if (err) + err =3D ext4_mb_mark_context(handle, sb, false, block_group, bit, + cluster_count, EXT4_MB_BITMAP_MARKED_CHECK, + &changed); + if (err && changed =3D=3D 0) goto error_clean; =20 - for (i =3D 0, clusters_freed =3D 0; i < cluster_count; i++) { - BUFFER_TRACE(bitmap_bh, "clear bit"); - if (!mb_test_bit(bit + i, bitmap_bh->b_data)) { - ext4_error(sb, "bit already cleared for block %llu", - (ext4_fsblk_t)(block + i)); - BUFFER_TRACE(bitmap_bh, "bit already cleared"); - } else { - clusters_freed++; - } - } - - ext4_lock_group(sb, block_group); - mb_clear_bits(bitmap_bh->b_data, bit, cluster_count); - free_clusters_count =3D clusters_freed + - ext4_free_group_clusters(sb, desc); - ext4_free_group_clusters_set(sb, desc, free_clusters_count); - ext4_block_bitmap_csum_set(sb, desc, bitmap_bh); - ext4_group_desc_csum_set(sb, block_group, desc); - ext4_unlock_group(sb, block_group); - - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, block_group); - atomic64_add(clusters_freed, - &sbi_array_rcu_deref(sbi, s_flex_groups, - flex_group)->free_clusters); - } - - /* We dirtied the bitmap block */ - BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); - - /* And the group descriptor block */ - BUFFER_TRACE(gd_bh, "dirtied group descriptor block"); - ret =3D ext4_handle_dirty_metadata(handle, NULL, gd_bh); - if (!err) - err =3D ret; + if (changed !=3D cluster_count) + ext4_error(sb, "bit already cleared in group %u", block_group); =20 ext4_lock_group(sb, block_group); mb_free_blocks(NULL, &e4b, bit, cluster_count); ext4_unlock_group(sb, block_group); percpu_counter_add(&sbi->s_freeclusters_counter, - clusters_freed); + changed); =20 error_clean: - brelse(bitmap_bh); ext4_mb_unload_buddy(&e4b); error_out: ext4_std_error(sb, err); --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C326FCD5BA0 for ; Tue, 19 Sep 2023 12:16:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231707AbjISMQi (ORCPT ); Tue, 19 Sep 2023 08:16:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231674AbjISMQT (ORCPT ); Tue, 19 Sep 2023 08:16:19 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CF6D102; Tue, 19 Sep 2023 05:16:13 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RqgcT6xHZz4f3yyV; Tue, 19 Sep 2023 20:16:09 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S12; Tue, 19 Sep 2023 20:16:10 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 10/12] ext4: add some kunit stub for mballoc kunit test Date: Wed, 20 Sep 2023 04:15:30 +0800 Message-Id: <20230919201532.310085-11-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S12 X-Coremail-Antispam: 1UD129KBjvJXoWxWFW8uF47Kr1fGw48Cr4rZrb_yoW5GFW5p3 ZFyF1UGr43Ww1q9F4Ika40q3WxWw1xKF1UJryfury3uF9xJFn3ZFZ7JF90ya1YvFZIvFsr X3W5Zry3Cr1fW3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBIb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY 6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aV CY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU0TqcUUUUUU== X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Multiblocks allocation will read and write block bitmap and group descriptor which reside on disk. Add kunit stub to function ext4_get_group_desc, ext4_read_block_bitmap_nowait, ext4_wait_block_bitmap and ext4_mb_mark_context to avoid real IO to disk. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/balloc.c | 10 ++++++++++ fs/ext4/mballoc.c | 5 +++++ 2 files changed, 15 insertions(+) diff --git a/fs/ext4/balloc.c b/fs/ext4/balloc.c index 79b20d6ae39e..b0bf255b159f 100644 --- a/fs/ext4/balloc.c +++ b/fs/ext4/balloc.c @@ -22,6 +22,7 @@ #include "mballoc.h" =20 #include +#include =20 static unsigned ext4_num_base_meta_clusters(struct super_block *sb, ext4_group_t block_group); @@ -274,6 +275,9 @@ struct ext4_group_desc * ext4_get_group_desc(struct sup= er_block *sb, struct ext4_sb_info *sbi =3D EXT4_SB(sb); struct buffer_head *bh_p; =20 + KUNIT_STATIC_STUB_REDIRECT(ext4_get_group_desc, + sb, block_group, bh); + if (block_group >=3D ngroups) { ext4_error(sb, "block_group >=3D groups_count - block_group =3D %u," " groups_count =3D %u", block_group, ngroups); @@ -468,6 +472,9 @@ ext4_read_block_bitmap_nowait(struct super_block *sb, e= xt4_group_t block_group, ext4_fsblk_t bitmap_blk; int err; =20 + KUNIT_STATIC_STUB_REDIRECT(ext4_read_block_bitmap_nowait, + sb, block_group, ignore_locked); + desc =3D ext4_get_group_desc(sb, block_group, NULL); if (!desc) return ERR_PTR(-EFSCORRUPTED); @@ -563,6 +570,9 @@ int ext4_wait_block_bitmap(struct super_block *sb, ext4= _group_t block_group, { struct ext4_group_desc *desc; =20 + KUNIT_STATIC_STUB_REDIRECT(ext4_wait_block_bitmap, + sb, block_group, bh); + if (!buffer_new(bh)) return 0; desc =3D ext4_get_group_desc(sb, block_group, NULL); diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index db0b15098a80..70dc446996a0 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 /* * MUSTDO: @@ -3967,6 +3968,10 @@ ext4_mb_mark_context(handle_t *handle, struct super_= block *sb, bool state, int err; unsigned int i, already, changed =3D len; =20 + KUNIT_STATIC_STUB_REDIRECT(ext4_mb_mark_context, + handle, sb, state, group, blkoff, len, + flags, ret_changed); + if (ret_changed) *ret_changed =3D 0; bitmap_bh =3D ext4_read_block_bitmap(sb, group); --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C034DCD5BA6 for ; Tue, 19 Sep 2023 12:16:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231991AbjISMRD (ORCPT ); Tue, 19 Sep 2023 08:17:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59154 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231744AbjISMQW (ORCPT ); Tue, 19 Sep 2023 08:16:22 -0400 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 817DAF2; Tue, 19 Sep 2023 05:16:15 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RqgcQ5XRRz4f3tD0; Tue, 19 Sep 2023 20:16:06 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S13; Tue, 19 Sep 2023 20:16:10 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 11/12] ext4: add first unit test for ext4_mb_new_blocks_simple in mballoc Date: Wed, 20 Sep 2023 04:15:31 +0800 Message-Id: <20230919201532.310085-12-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S13 X-Coremail-Antispam: 1UD129KBjvJXoW3CFWftw45WrWUuF4xGr4UArb_yoWDur1xpa n3AF1F9r43WFsrWayxtryFq3WSgw18AryUtryfu34YkFyxAr93J3Z7tF10yF4F9FWxJFnx Xa1YkryUCrWxGa7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBIb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY 6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aV CY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU0TqcUUUUUU== X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Here are prepared work: 1. Include mballoc-test.c to mballoc.c to be able test static function in mballoc.c. 2. Implement static stub to avoid read IO to disk. 3. Construct fake super_block. Only partial members are set, more members will be set when more functions are tested. Then unit test for ext4_mb_new_blocks_simple is added. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc-test.c | 325 +++++++++++++++++++++++++++++++++++++++++ fs/ext4/mballoc.c | 4 + 2 files changed, 329 insertions(+) create mode 100644 fs/ext4/mballoc-test.c diff --git a/fs/ext4/mballoc-test.c b/fs/ext4/mballoc-test.c new file mode 100644 index 000000000000..120c4944d2e1 --- /dev/null +++ b/fs/ext4/mballoc-test.c @@ -0,0 +1,325 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test of ext4 multiblocks allocation. + */ + +#include +#include + +#include "ext4.h" + +struct mbt_grp_ctx { + struct buffer_head bitmap_bh; + /* desc and gd_bh are just the place holders for now */ + struct ext4_group_desc desc; + struct buffer_head gd_bh; +}; + +struct mbt_ctx { + struct mbt_grp_ctx *grp_ctx; +}; + +struct mbt_ext4_super_block { + struct super_block sb; + struct mbt_ctx mbt_ctx; +}; + +#define MBT_CTX(_sb) (&(container_of((_sb), struct mbt_ext4_super_block, s= b)->mbt_ctx)) +#define MBT_GRP_CTX(_sb, _group) (&MBT_CTX(_sb)->grp_ctx[_group]) + +static struct super_block *mbt_ext4_alloc_super_block(void) +{ + struct ext4_super_block *es =3D kzalloc(sizeof(*es), GFP_KERNEL); + struct ext4_sb_info *sbi =3D kzalloc(sizeof(*sbi), GFP_KERNEL); + struct mbt_ext4_super_block *fsb =3D kzalloc(sizeof(*fsb), GFP_KERNEL); + + if (fsb =3D=3D NULL || sbi =3D=3D NULL || es =3D=3D NULL) + goto out; + + sbi->s_es =3D es; + fsb->sb.s_fs_info =3D sbi; + return &fsb->sb; + +out: + kfree(fsb); + kfree(sbi); + kfree(es); + return NULL; +} + +static void mbt_ext4_free_super_block(struct super_block *sb) +{ + struct mbt_ext4_super_block *fsb =3D + container_of(sb, struct mbt_ext4_super_block, sb); + struct ext4_sb_info *sbi =3D EXT4_SB(sb); + + kfree(sbi->s_es); + kfree(sbi); + kfree(fsb); +} + +struct mbt_ext4_block_layout { + unsigned char blocksize_bits; + unsigned int cluster_bits; + uint32_t blocks_per_group; + ext4_group_t group_count; + uint16_t desc_size; +}; + +static void mbt_init_sb_layout(struct super_block *sb, + struct mbt_ext4_block_layout *layout) +{ + struct ext4_sb_info *sbi =3D EXT4_SB(sb); + struct ext4_super_block *es =3D sbi->s_es; + + sb->s_blocksize =3D 1UL << layout->blocksize_bits; + sb->s_blocksize_bits =3D layout->blocksize_bits; + + sbi->s_groups_count =3D layout->group_count; + sbi->s_blocks_per_group =3D layout->blocks_per_group; + sbi->s_cluster_bits =3D layout->cluster_bits; + sbi->s_cluster_ratio =3D 1U << layout->cluster_bits; + sbi->s_clusters_per_group =3D layout->blocks_per_group >> + layout->cluster_bits; + sbi->s_desc_size =3D layout->desc_size; + + es->s_first_data_block =3D cpu_to_le32(0); + es->s_blocks_count_lo =3D cpu_to_le32(layout->blocks_per_group * + layout->group_count); +} + +static int mbt_grp_ctx_init(struct super_block *sb, + struct mbt_grp_ctx *grp_ctx) +{ + grp_ctx->bitmap_bh.b_data =3D kzalloc(EXT4_BLOCK_SIZE(sb), GFP_KERNEL); + if (grp_ctx->bitmap_bh.b_data =3D=3D NULL) + return -ENOMEM; + + return 0; +} + +static void mbt_grp_ctx_release(struct mbt_grp_ctx *grp_ctx) +{ + kfree(grp_ctx->bitmap_bh.b_data); + grp_ctx->bitmap_bh.b_data =3D NULL; +} + +static void mbt_ctx_mark_used(struct super_block *sb, ext4_group_t group, + unsigned int start, unsigned int len) +{ + struct mbt_grp_ctx *grp_ctx =3D MBT_GRP_CTX(sb, group); + + mb_set_bits(grp_ctx->bitmap_bh.b_data, start, len); +} + +/* called after mbt_init_sb_layout */ +static int mbt_ctx_init(struct super_block *sb) +{ + struct mbt_ctx *ctx =3D MBT_CTX(sb); + ext4_group_t i, ngroups =3D ext4_get_groups_count(sb); + + ctx->grp_ctx =3D kcalloc(ngroups, sizeof(struct mbt_grp_ctx), + GFP_KERNEL); + if (ctx->grp_ctx =3D=3D NULL) + return -ENOMEM; + + for (i =3D 0; i < ngroups; i++) + if (mbt_grp_ctx_init(sb, &ctx->grp_ctx[i])) + goto out; + + /* + * first data block(first cluster in first group) is used by + * metadata, mark it used to avoid to alloc data block at first + * block which will fail ext4_sb_block_valid check. + */ + mb_set_bits(ctx->grp_ctx[0].bitmap_bh.b_data, 0, 1); + + return 0; +out: + while (i-- > 0) + mbt_grp_ctx_release(&ctx->grp_ctx[i]); + kfree(ctx->grp_ctx); + return -ENOMEM; +} + +static void mbt_ctx_release(struct super_block *sb) +{ + struct mbt_ctx *ctx =3D MBT_CTX(sb); + ext4_group_t i, ngroups =3D ext4_get_groups_count(sb); + + for (i =3D 0; i < ngroups; i++) + mbt_grp_ctx_release(&ctx->grp_ctx[i]); + kfree(ctx->grp_ctx); +} + +static struct buffer_head * +ext4_read_block_bitmap_nowait_stub(struct super_block *sb, ext4_group_t bl= ock_group, + bool ignore_locked) +{ + struct mbt_grp_ctx *grp_ctx =3D MBT_GRP_CTX(sb, block_group); + + /* paired with brelse from caller of ext4_read_block_bitmap_nowait */ + get_bh(&grp_ctx->bitmap_bh); + return &grp_ctx->bitmap_bh; +} + +static int ext4_wait_block_bitmap_stub(struct super_block *sb, + ext4_group_t block_group, + struct buffer_head *bh) +{ + return 0; +} + +static struct ext4_group_desc * +ext4_get_group_desc_stub(struct super_block *sb, ext4_group_t block_group, + struct buffer_head **bh) +{ + struct mbt_grp_ctx *grp_ctx =3D MBT_GRP_CTX(sb, block_group); + + if (bh !=3D NULL) + *bh =3D &grp_ctx->gd_bh; + + return &grp_ctx->desc; +} + +static int +ext4_mb_mark_context_stub(handle_t *handle, struct super_block *sb, bool s= tate, + ext4_group_t group, ext4_grpblk_t blkoff, + ext4_grpblk_t len, int flags, + ext4_grpblk_t *ret_changed) +{ + struct mbt_grp_ctx *grp_ctx =3D MBT_GRP_CTX(sb, group); + struct buffer_head *bitmap_bh =3D &grp_ctx->bitmap_bh; + + if (state) + mb_set_bits(bitmap_bh->b_data, blkoff, len); + else + mb_clear_bits(bitmap_bh->b_data, blkoff, len); + + return 0; +} + +#define TEST_BLOCKSIZE_BITS 10 +#define TEST_CLUSTER_BITS 3 +#define TEST_BLOCKS_PER_GROUP 8192 +#define TEST_GROUP_COUNT 4 +#define TEST_DESC_SIZE 64 +#define TEST_GOAL_GROUP 1 +static int mbt_kunit_init(struct kunit *test) +{ + struct mbt_ext4_block_layout layout =3D { + .blocksize_bits =3D TEST_BLOCKSIZE_BITS, + .cluster_bits =3D TEST_CLUSTER_BITS, + .blocks_per_group =3D TEST_BLOCKS_PER_GROUP, + .group_count =3D TEST_GROUP_COUNT, + .desc_size =3D TEST_DESC_SIZE, + }; + struct super_block *sb; + int ret; + + sb =3D mbt_ext4_alloc_super_block(); + if (sb =3D=3D NULL) + return -ENOMEM; + + mbt_init_sb_layout(sb, &layout); + + ret =3D mbt_ctx_init(sb); + if (ret !=3D 0) { + mbt_ext4_free_super_block(sb); + return ret; + } + + test->priv =3D sb; + kunit_activate_static_stub(test, + ext4_read_block_bitmap_nowait, + ext4_read_block_bitmap_nowait_stub); + kunit_activate_static_stub(test, + ext4_wait_block_bitmap, + ext4_wait_block_bitmap_stub); + kunit_activate_static_stub(test, + ext4_get_group_desc, + ext4_get_group_desc_stub); + kunit_activate_static_stub(test, + ext4_mb_mark_context, + ext4_mb_mark_context_stub); + return 0; +} + +static void mbt_kunit_exit(struct kunit *test) +{ + struct super_block *sb =3D (struct super_block *)test->priv; + + mbt_ctx_release(sb); + mbt_ext4_free_super_block(sb); +} + +static void test_new_blocks_simple(struct kunit *test) +{ + struct super_block *sb =3D (struct super_block *)test->priv; + struct inode inode =3D { .i_sb =3D sb, }; + struct ext4_allocation_request ar; + ext4_group_t i, goal_group =3D TEST_GOAL_GROUP; + int err =3D 0; + ext4_fsblk_t found; + struct ext4_sb_info *sbi =3D EXT4_SB(sb); + + ar.inode =3D &inode; + + /* get block at goal */ + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_EQ_MSG(test, ar.goal, found, + "failed to alloc block at goal, expected %llu found %llu", + ar.goal, found); + + /* get block after goal in goal group */ + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_EQ_MSG(test, ar.goal + EXT4_C2B(sbi, 1), found, + "failed to alloc block after goal in goal group, expected %llu found %ll= u", + ar.goal + 1, found); + + /* get block after goal group */ + mbt_ctx_mark_used(sb, goal_group, 0, EXT4_CLUSTERS_PER_GROUP(sb)); + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_EQ_MSG(test, + ext4_group_first_block_no(sb, goal_group + 1), found, + "failed to alloc block after goal group, expected %llu found %llu", + ext4_group_first_block_no(sb, goal_group + 1), found); + + /* get block before goal group */ + for (i =3D goal_group; i < ext4_get_groups_count(sb); i++) + mbt_ctx_mark_used(sb, i, 0, EXT4_CLUSTERS_PER_GROUP(sb)); + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_EQ_MSG(test, + ext4_group_first_block_no(sb, 0) + EXT4_C2B(sbi, 1), found, + "failed to alloc block before goal group, expected %llu found %llu", + ext4_group_first_block_no(sb, 0 + EXT4_C2B(sbi, 1)), found); + + /* no block available, fail to allocate block */ + for (i =3D 0; i < ext4_get_groups_count(sb); i++) + mbt_ctx_mark_used(sb, i, 0, EXT4_CLUSTERS_PER_GROUP(sb)); + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_NE_MSG(test, err, 0, + "unexpectedly get block when no block is available"); +} + + +static struct kunit_case mbt_test_cases[] =3D { + KUNIT_CASE(test_new_blocks_simple), + {} +}; + +static struct kunit_suite mbt_test_suite =3D { + .name =3D "ext4_mballoc_test", + .init =3D mbt_kunit_init, + .exit =3D mbt_kunit_exit, + .test_cases =3D mbt_test_cases, +}; + +kunit_test_suites(&mbt_test_suite); + +MODULE_LICENSE("GPL"); diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 70dc446996a0..95125711468e 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -7026,3 +7026,7 @@ ext4_mballoc_query_range( =20 return error; } + +#ifdef CONFIG_EXT4_KUNIT_TESTS +#include "mballoc-test.c" +#endif --=20 2.30.0 From nobody Sat Sep 7 23:51:39 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 164F1CD5BA6 for ; Tue, 19 Sep 2023 12:16:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231819AbjISMQv (ORCPT ); Tue, 19 Sep 2023 08:16:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59090 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231724AbjISMQV (ORCPT ); Tue, 19 Sep 2023 08:16:21 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8598E3; Tue, 19 Sep 2023 05:16:14 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.143]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RqgcV43Knz4f400Y; Tue, 19 Sep 2023 20:16:10 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP2 (Coremail) with SMTP id Syh0CgAnOA0DkQllO8FpAw--.40065S14; Tue, 19 Sep 2023 20:16:11 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com Cc: ojaswin@linux.ibm.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v7 12/12] ext4: run mballoc test with different layouts setting Date: Wed, 20 Sep 2023 04:15:32 +0800 Message-Id: <20230919201532.310085-13-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230919201532.310085-1-shikemeng@huaweicloud.com> References: <20230919201532.310085-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: Syh0CgAnOA0DkQllO8FpAw--.40065S14 X-Coremail-Antispam: 1UD129KBjvJXoWxWF4xZrWkCryrGw4rXrWrKrg_yoW5Xryfpa nIkF1Fkr15WFsF93W3K3s7Zw1agw1kur18Jry7W34FyFn7Aws7JFsrtryYya40qrWkXFn0 vFn09r17C3y8Cw7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBIb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVW7JVWDJw A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY 6xAIw20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aV CY1x0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU0TqcUUUUUU== X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use KUNIT_CASE_PARAM to run mballoc test with different layouts setting. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc-test.c | 52 ++++++++++++++++++++++++++++++------------ 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/fs/ext4/mballoc-test.c b/fs/ext4/mballoc-test.c index 120c4944d2e1..f94901fd3835 100644 --- a/fs/ext4/mballoc-test.c +++ b/fs/ext4/mballoc-test.c @@ -199,21 +199,11 @@ ext4_mb_mark_context_stub(handle_t *handle, struct su= per_block *sb, bool state, return 0; } =20 -#define TEST_BLOCKSIZE_BITS 10 -#define TEST_CLUSTER_BITS 3 -#define TEST_BLOCKS_PER_GROUP 8192 -#define TEST_GROUP_COUNT 4 -#define TEST_DESC_SIZE 64 #define TEST_GOAL_GROUP 1 static int mbt_kunit_init(struct kunit *test) { - struct mbt_ext4_block_layout layout =3D { - .blocksize_bits =3D TEST_BLOCKSIZE_BITS, - .cluster_bits =3D TEST_CLUSTER_BITS, - .blocks_per_group =3D TEST_BLOCKS_PER_GROUP, - .group_count =3D TEST_GROUP_COUNT, - .desc_size =3D TEST_DESC_SIZE, - }; + struct mbt_ext4_block_layout *layout =3D + (struct mbt_ext4_block_layout *)(test->param_value); struct super_block *sb; int ret; =20 @@ -221,7 +211,7 @@ static int mbt_kunit_init(struct kunit *test) if (sb =3D=3D NULL) return -ENOMEM; =20 - mbt_init_sb_layout(sb, &layout); + mbt_init_sb_layout(sb, layout); =20 ret =3D mbt_ctx_init(sb); if (ret !=3D 0) { @@ -307,9 +297,43 @@ static void test_new_blocks_simple(struct kunit *test) "unexpectedly get block when no block is available"); } =20 +static const struct mbt_ext4_block_layout mbt_test_layouts[] =3D { + { + .blocksize_bits =3D 10, + .cluster_bits =3D 3, + .blocks_per_group =3D 8192, + .group_count =3D 4, + .desc_size =3D 64, + }, + { + .blocksize_bits =3D 12, + .cluster_bits =3D 3, + .blocks_per_group =3D 8192, + .group_count =3D 4, + .desc_size =3D 64, + }, + { + .blocksize_bits =3D 16, + .cluster_bits =3D 3, + .blocks_per_group =3D 8192, + .group_count =3D 4, + .desc_size =3D 64, + }, +}; + +static void mbt_show_layout(const struct mbt_ext4_block_layout *layout, + char *desc) +{ + snprintf(desc, KUNIT_PARAM_DESC_SIZE, "block_bits=3D%d cluster_bits=3D%d " + "blocks_per_group=3D%d group_count=3D%d desc_size=3D%d\n", + layout->blocksize_bits, layout->cluster_bits, + layout->blocks_per_group, layout->group_count, + layout->desc_size); +} +KUNIT_ARRAY_PARAM(mbt_layouts, mbt_test_layouts, mbt_show_layout); =20 static struct kunit_case mbt_test_cases[] =3D { - KUNIT_CASE(test_new_blocks_simple), + KUNIT_CASE_PARAM(test_new_blocks_simple, mbt_layouts_gen_params), {} }; =20 --=20 2.30.0