From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7F26C83F25 for ; Sat, 26 Aug 2023 07:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232114AbjHZHva (ORCPT ); Sat, 26 Aug 2023 03:51:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229908AbjHZHur (ORCPT ); Sat, 26 Aug 2023 03:50:47 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 14A111FD7; Sat, 26 Aug 2023 00:50:45 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RXpsD5KHfz4f3jM9; Sat, 26 Aug 2023 15:50:40 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S3; Sat, 26 Aug 2023 15:50:41 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 01/11] ext4: factor out codes to update block bitmap and group descriptor on disk from ext4_mb_mark_bb Date: Sat, 26 Aug 2023 23:50:18 +0800 Message-Id: <20230826155028.4019470-2-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S3 X-Coremail-Antispam: 1UD129KBjvJXoW3Gw45Zw18Wry3ZFy3JrWxXrb_yoWxWryfpF 90yF1DCr43JrnruFsxW342qF1rKw1093WkGryfG345CF43tr95WF1xKFyF9asIyFZrXFnF qF1YyrWxur4ruFDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBYb4IE77IF4wAFF20E14v26r4j6ryUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUGwA2048vs2IY020Ec7CjxVAFwI0_Jrv_JF4l8cAvFVAK 0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4 x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l 84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I 8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv67AK xVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64 vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8G jcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2I x0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwCI42IY6xAI w20EY4v20xvaj40_Jr0_JF4lIxAIcVC2z280aVAFwI0_Gr0_Cr1lIxAIcVC2z280aVCY1x 0267AKxVW8JVW8JrUvcSsGvfC2KfnxnUUI43ZEXa7IU04SoJUUUUU== X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" There are several reasons to add a general function to update block bitmap and group descriptor on disk: 1. pair behavior of alloc/free bits. For example, ext4_mb_new_blocks_simple will update free_clusters in struct flex_groups in ext4_mb_mark_bb while ext4_free_blocks_simple forgets this. 2. remove repeat code to read from disk, update and write back to disk. 3. reduce future unit test mocks to catch real IO to update structure on disk. Signed-off-by: Kemeng Shi Reviewed-by: Ojaswin Mujoo Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 169 +++++++++++++++++++++++++++------------------- 1 file changed, 99 insertions(+), 70 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index c91db9f57524..e2be572deb75 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -3952,6 +3952,100 @@ void ext4_exit_mballoc(void) ext4_groupinfo_destroy_slabs(); } =20 +/* + * Collect global setting to reduce the number of variable passing to + * ext4_mb_mark_context. Pass target group blocks range directly to + * reuse the prepared global setting for multiple block ranges and + * to show clearly the specific block range will be marked. + */ +struct ext4_mark_context { + struct super_block *sb; + int state; +}; + +static inline void ext4_mb_prepare_mark_context(struct ext4_mark_context *= mc, + struct super_block *sb, + int state) +{ + mc->sb =3D sb; + mc->state =3D state; +} + +static int +ext4_mb_mark_context(struct ext4_mark_context *mc, ext4_group_t group, + ext4_grpblk_t blkoff, ext4_grpblk_t len) +{ + struct super_block *sb =3D mc->sb; + struct ext4_sb_info *sbi =3D EXT4_SB(sb); + struct buffer_head *bitmap_bh =3D NULL; + struct ext4_group_desc *gdp; + struct buffer_head *gdp_bh; + int err; + unsigned int i, already, changed; + + bitmap_bh =3D ext4_read_block_bitmap(sb, group); + if (IS_ERR(bitmap_bh)) + return PTR_ERR(bitmap_bh); + + err =3D -EIO; + gdp =3D ext4_get_group_desc(sb, group, &gdp_bh); + if (!gdp) + goto out_err; + + ext4_lock_group(sb, group); + if (ext4_has_group_desc_csum(sb) && + (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT))) { + gdp->bg_flags &=3D cpu_to_le16(~EXT4_BG_BLOCK_UNINIT); + ext4_free_group_clusters_set(sb, gdp, + ext4_free_clusters_after_init(sb, group, gdp)); + } + + already =3D 0; + for (i =3D 0; i < len; i++) + if (mb_test_bit(blkoff + i, bitmap_bh->b_data) =3D=3D + mc->state) + already++; + changed =3D len - already; + + if (mc->state) { + mb_set_bits(bitmap_bh->b_data, blkoff, len); + ext4_free_group_clusters_set(sb, gdp, + ext4_free_group_clusters(sb, gdp) - changed); + } else { + mb_clear_bits(bitmap_bh->b_data, blkoff, len); + ext4_free_group_clusters_set(sb, gdp, + ext4_free_group_clusters(sb, gdp) + changed); + } + + ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); + ext4_group_desc_csum_set(sb, group, gdp); + ext4_unlock_group(sb, group); + + if (sbi->s_log_groups_per_flex) { + ext4_group_t flex_group =3D ext4_flex_group(sbi, group); + struct flex_groups *fg =3D sbi_array_rcu_deref(sbi, + s_flex_groups, flex_group); + + if (mc->state) + atomic64_sub(changed, &fg->free_clusters); + else + atomic64_add(changed, &fg->free_clusters); + } + + err =3D ext4_handle_dirty_metadata(NULL, NULL, bitmap_bh); + if (err) + goto out_err; + err =3D ext4_handle_dirty_metadata(NULL, NULL, gdp_bh); + if (err) + goto out_err; + + sync_dirty_buffer(bitmap_bh); + sync_dirty_buffer(gdp_bh); + +out_err: + brelse(bitmap_bh); + return err; +} =20 /* * Check quota and mark chosen space (ac->ac_b_ex) non-free in bitmaps @@ -4078,16 +4172,14 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_= context *ac, void ext4_mb_mark_bb(struct super_block *sb, ext4_fsblk_t block, int len, int state) { - struct buffer_head *bitmap_bh =3D NULL; - struct ext4_group_desc *gdp; - struct buffer_head *gdp_bh; + struct ext4_mark_context mc; struct ext4_sb_info *sbi =3D EXT4_SB(sb); ext4_group_t group; ext4_grpblk_t blkoff; - int i, err =3D 0; - int already; - unsigned int clen, clen_changed, thisgrp_len; + int err =3D 0; + unsigned int clen, thisgrp_len; =20 + ext4_mb_prepare_mark_context(&mc, sb, state); while (len > 0) { ext4_get_group_no_and_offset(sb, block, &group, &blkoff); =20 @@ -4107,80 +4199,17 @@ void ext4_mb_mark_bb(struct super_block *sb, ext4_f= sblk_t block, ext4_error(sb, "Marking blocks in system zone - " "Block =3D %llu, len =3D %u", block, thisgrp_len); - bitmap_bh =3D NULL; - break; - } - - bitmap_bh =3D ext4_read_block_bitmap(sb, group); - if (IS_ERR(bitmap_bh)) { - err =3D PTR_ERR(bitmap_bh); - bitmap_bh =3D NULL; break; } =20 - err =3D -EIO; - gdp =3D ext4_get_group_desc(sb, group, &gdp_bh); - if (!gdp) - break; - - ext4_lock_group(sb, group); - already =3D 0; - for (i =3D 0; i < clen; i++) - if (!mb_test_bit(blkoff + i, bitmap_bh->b_data) =3D=3D - !state) - already++; - - clen_changed =3D clen - already; - if (state) - mb_set_bits(bitmap_bh->b_data, blkoff, clen); - else - mb_clear_bits(bitmap_bh->b_data, blkoff, clen); - if (ext4_has_group_desc_csum(sb) && - (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT))) { - gdp->bg_flags &=3D cpu_to_le16(~EXT4_BG_BLOCK_UNINIT); - ext4_free_group_clusters_set(sb, gdp, - ext4_free_clusters_after_init(sb, group, gdp)); - } - if (state) - clen =3D ext4_free_group_clusters(sb, gdp) - clen_changed; - else - clen =3D ext4_free_group_clusters(sb, gdp) + clen_changed; - - ext4_free_group_clusters_set(sb, gdp, clen); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, group, gdp); - - ext4_unlock_group(sb, group); - - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, group); - struct flex_groups *fg =3D sbi_array_rcu_deref(sbi, - s_flex_groups, flex_group); - - if (state) - atomic64_sub(clen_changed, &fg->free_clusters); - else - atomic64_add(clen_changed, &fg->free_clusters); - - } - - err =3D ext4_handle_dirty_metadata(NULL, NULL, bitmap_bh); - if (err) - break; - sync_dirty_buffer(bitmap_bh); - err =3D ext4_handle_dirty_metadata(NULL, NULL, gdp_bh); - sync_dirty_buffer(gdp_bh); + err =3D ext4_mb_mark_context(&mc, group, blkoff, clen); if (err) break; =20 block +=3D thisgrp_len; len -=3D thisgrp_len; - brelse(bitmap_bh); BUG_ON(len < 0); } - - if (err) - brelse(bitmap_bh); } =20 /* --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7E51C83F28 for ; Sat, 26 Aug 2023 07:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232120AbjHZHvc (ORCPT ); Sat, 26 Aug 2023 03:51:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230136AbjHZHus (ORCPT ); Sat, 26 Aug 2023 03:50:48 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EC0F1FF0; Sat, 26 Aug 2023 00:50:45 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RXpsB6KsTz4f3jMP; Sat, 26 Aug 2023 15:50:38 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S4; Sat, 26 Aug 2023 15:50:42 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 02/11] ext4: call ext4_mb_mark_context in ext4_free_blocks_simple Date: Sat, 26 Aug 2023 23:50:19 +0800 Message-Id: <20230826155028.4019470-3-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S4 X-Coremail-Antispam: 1UD129KBjvJXoW7ury5trWDKFy5CF47Zry8AFb_yoW8Zw17pr ZxAr1xCr1fGr1DuF4xJ342q3W8Kw18W3WDGryfW34rCFW3Ar93WF48KF1FvFyYkFZrZ3ZI vr1Y9rW7Zw15W37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUB2b4IE77IF4wAFF20E14v26ryj6rWUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUXwA2048vs2IY020Ec7CjxVAFwI0_JFI_Gr1l8cAvFVAK 0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4 x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l 84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I 8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv67AK xVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64 vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8G jcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2I x0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAIcVCF 04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2jsIEc7 CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxUsknYUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" call ext4_mb_mark_context in ext4_free_blocks_simple to: 1. remove repeat code 2. pair update of free_clusters in ext4_mb_new_blocks_simple. 3. add missing ext4_lock_group/ext4_unlock_group protection. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 35 +++-------------------------------- 1 file changed, 3 insertions(+), 32 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index e2be572deb75..c803f74aaf63 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6414,43 +6414,14 @@ ext4_mb_free_metadata(handle_t *handle, struct ext4= _buddy *e4b, static void ext4_free_blocks_simple(struct inode *inode, ext4_fsblk_t bloc= k, unsigned long count) { - struct buffer_head *bitmap_bh; + struct ext4_mark_context mc; struct super_block *sb =3D inode->i_sb; - struct ext4_group_desc *gdp; - struct buffer_head *gdp_bh; ext4_group_t group; ext4_grpblk_t blkoff; - int already_freed =3D 0, err, i; =20 + ext4_mb_prepare_mark_context(&mc, sb, 0); ext4_get_group_no_and_offset(sb, block, &group, &blkoff); - bitmap_bh =3D ext4_read_block_bitmap(sb, group); - if (IS_ERR(bitmap_bh)) { - pr_warn("Failed to read block bitmap\n"); - return; - } - gdp =3D ext4_get_group_desc(sb, group, &gdp_bh); - if (!gdp) - goto err_out; - - for (i =3D 0; i < count; i++) { - if (!mb_test_bit(blkoff + i, bitmap_bh->b_data)) - already_freed++; - } - mb_clear_bits(bitmap_bh->b_data, blkoff, count); - err =3D ext4_handle_dirty_metadata(NULL, NULL, bitmap_bh); - if (err) - goto err_out; - ext4_free_group_clusters_set( - sb, gdp, ext4_free_group_clusters(sb, gdp) + - count - already_freed); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, group, gdp); - ext4_handle_dirty_metadata(NULL, NULL, gdp_bh); - sync_dirty_buffer(bitmap_bh); - sync_dirty_buffer(gdp_bh); - -err_out: - brelse(bitmap_bh); + ext4_mb_mark_context(&mc, group, blkoff, count); } =20 /** --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 207BBC83F20 for ; Sat, 26 Aug 2023 07:51:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232073AbjHZHvX (ORCPT ); Sat, 26 Aug 2023 03:51:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231454AbjHZHut (ORCPT ); Sat, 26 Aug 2023 03:50:49 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 527E42137; Sat, 26 Aug 2023 00:50:45 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RXpsC11T5z4f3jMZ; Sat, 26 Aug 2023 15:50:39 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S5; Sat, 26 Aug 2023 15:50:42 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 03/11] ext4: extent ext4_mb_mark_context to support allocation under journal Date: Sat, 26 Aug 2023 23:50:20 +0800 Message-Id: <20230826155028.4019470-4-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S5 X-Coremail-Antispam: 1UD129KBjvJXoW3Jr4DtF4rZw4xtF4DAFyxXwb_yoW7KrykpF yDAr1qkr4fJFnI9FsFk342qF1fKw18C3WUGrWfGryFkryxtryrZF4xKFyFyFWYyF47ZFn8 Zr1Yy3y7Wa1fK37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUB2b4IE77IF4wAFF20E14v26ryj6rWUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUWwA2048vs2IY020Ec7CjxVAFwI0_Gr0_Xr1l8cAvFVAK 0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4 x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l 84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I 8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv67AK xVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64 vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8G jcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2I x0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAIcVCF 04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2jsIEc7 CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxU3-txDUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Previously, ext4_mb_mark_context is only called under fast commit replay path, so there is no valid handle when we update block bitmap and group descriptor. This patch try to extent ext4_mb_mark_context to be used by code under journal. There are several improves: 1. add "handle_t *handle" to struct ext4_mark_context to accept handle to journal block bitmap and group descriptor update inside ext4_mb_mark_context (the added journal caode is based on journal code in ext4_mb_mark_diskspace_used where ext4_mb_mark_context is going to be used.) 2. add EXT4_MB_BITMAP_MARKED_CHECK flag to control check if bits in block bitmap are already marked as allocation code under journal asserts that all bits to be changed are not marked before. 3. add "ext4_grpblk_t changed" to struct ext4_mark_context to notify number of bits in block bitmap has changed. Signed-off-by: Kemeng Shi Reviewed-by: Ojaswin Mujoo Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 65 +++++++++++++++++++++++++++++++++++------------ 1 file changed, 49 insertions(+), 16 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index c803f74aaf63..b066ee018cdb 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -3952,6 +3952,8 @@ void ext4_exit_mballoc(void) ext4_groupinfo_destroy_slabs(); } =20 +#define EXT4_MB_BITMAP_MARKED_CHECK 0x0001 +#define EXT4_MB_SYNC_UPDATE 0x0002 /* * Collect global setting to reduce the number of variable passing to * ext4_mb_mark_context. Pass target group blocks range directly to @@ -3959,39 +3961,61 @@ void ext4_exit_mballoc(void) * to show clearly the specific block range will be marked. */ struct ext4_mark_context { + handle_t *handle; struct super_block *sb; int state; + ext4_grpblk_t changed; }; =20 static inline void ext4_mb_prepare_mark_context(struct ext4_mark_context *= mc, + handle_t *handle, struct super_block *sb, int state) { + mc->handle =3D handle; mc->sb =3D sb; mc->state =3D state; } =20 static int ext4_mb_mark_context(struct ext4_mark_context *mc, ext4_group_t group, - ext4_grpblk_t blkoff, ext4_grpblk_t len) + ext4_grpblk_t blkoff, ext4_grpblk_t len, int flags) { + handle_t *handle =3D mc->handle; struct super_block *sb =3D mc->sb; struct ext4_sb_info *sbi =3D EXT4_SB(sb); struct buffer_head *bitmap_bh =3D NULL; struct ext4_group_desc *gdp; struct buffer_head *gdp_bh; int err; - unsigned int i, already, changed; + unsigned int i, already, changed =3D len; =20 + mc->changed =3D 0; bitmap_bh =3D ext4_read_block_bitmap(sb, group); if (IS_ERR(bitmap_bh)) return PTR_ERR(bitmap_bh); =20 + if (handle) { + BUFFER_TRACE(bitmap_bh, "getting write access"); + err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, + EXT4_JTR_NONE); + if (err) + goto out_err; + } + err =3D -EIO; gdp =3D ext4_get_group_desc(sb, group, &gdp_bh); if (!gdp) goto out_err; =20 + if (handle) { + BUFFER_TRACE(gdp_bh, "get_write_access"); + err =3D ext4_journal_get_write_access(handle, sb, gdp_bh, + EXT4_JTR_NONE); + if (err) + goto out_err; + } + ext4_lock_group(sb, group); if (ext4_has_group_desc_csum(sb) && (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT))) { @@ -4000,12 +4024,14 @@ ext4_mb_mark_context(struct ext4_mark_context *mc, = ext4_group_t group, ext4_free_clusters_after_init(sb, group, gdp)); } =20 - already =3D 0; - for (i =3D 0; i < len; i++) - if (mb_test_bit(blkoff + i, bitmap_bh->b_data) =3D=3D - mc->state) - already++; - changed =3D len - already; + if (flags & EXT4_MB_BITMAP_MARKED_CHECK) { + already =3D 0; + for (i =3D 0; i < len; i++) + if (mb_test_bit(blkoff + i, bitmap_bh->b_data) =3D=3D + mc->state) + already++; + changed =3D len - already; + } =20 if (mc->state) { mb_set_bits(bitmap_bh->b_data, blkoff, len); @@ -4020,6 +4046,7 @@ ext4_mb_mark_context(struct ext4_mark_context *mc, ex= t4_group_t group, ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); ext4_group_desc_csum_set(sb, group, gdp); ext4_unlock_group(sb, group); + mc->changed =3D changed; =20 if (sbi->s_log_groups_per_flex) { ext4_group_t flex_group =3D ext4_flex_group(sbi, group); @@ -4032,15 +4059,17 @@ ext4_mb_mark_context(struct ext4_mark_context *mc, = ext4_group_t group, atomic64_add(changed, &fg->free_clusters); } =20 - err =3D ext4_handle_dirty_metadata(NULL, NULL, bitmap_bh); + err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); if (err) goto out_err; - err =3D ext4_handle_dirty_metadata(NULL, NULL, gdp_bh); + err =3D ext4_handle_dirty_metadata(handle, NULL, gdp_bh); if (err) goto out_err; =20 - sync_dirty_buffer(bitmap_bh); - sync_dirty_buffer(gdp_bh); + if (flags & EXT4_MB_SYNC_UPDATE) { + sync_dirty_buffer(bitmap_bh); + sync_dirty_buffer(gdp_bh); + } =20 out_err: brelse(bitmap_bh); @@ -4179,7 +4208,7 @@ void ext4_mb_mark_bb(struct super_block *sb, ext4_fsb= lk_t block, int err =3D 0; unsigned int clen, thisgrp_len; =20 - ext4_mb_prepare_mark_context(&mc, sb, state); + ext4_mb_prepare_mark_context(&mc, NULL, sb, state); while (len > 0) { ext4_get_group_no_and_offset(sb, block, &group, &blkoff); =20 @@ -4202,7 +4231,9 @@ void ext4_mb_mark_bb(struct super_block *sb, ext4_fsb= lk_t block, break; } =20 - err =3D ext4_mb_mark_context(&mc, group, blkoff, clen); + err =3D ext4_mb_mark_context(&mc, group, blkoff, clen, + EXT4_MB_BITMAP_MARKED_CHECK | + EXT4_MB_SYNC_UPDATE); if (err) break; =20 @@ -6419,9 +6450,11 @@ static void ext4_free_blocks_simple(struct inode *in= ode, ext4_fsblk_t block, ext4_group_t group; ext4_grpblk_t blkoff; =20 - ext4_mb_prepare_mark_context(&mc, sb, 0); + ext4_mb_prepare_mark_context(&mc, NULL, sb, 0); ext4_get_group_no_and_offset(sb, block, &group, &blkoff); - ext4_mb_mark_context(&mc, group, blkoff, count); + ext4_mb_mark_context(&mc, group, blkoff, count, + EXT4_MB_BITMAP_MARKED_CHECK | + EXT4_MB_SYNC_UPDATE); } =20 /** --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9209BC83F26 for ; Sat, 26 Aug 2023 07:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232108AbjHZHv2 (ORCPT ); Sat, 26 Aug 2023 03:51:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231231AbjHZHut (ORCPT ); Sat, 26 Aug 2023 03:50:49 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87BE3213C; Sat, 26 Aug 2023 00:50:45 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RXpsC2qlkz4f3jN0; Sat, 26 Aug 2023 15:50:39 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S6; Sat, 26 Aug 2023 15:50:42 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 04/11] ext4: call ext4_mb_mark_context in ext4_mb_mark_diskspace_used Date: Sat, 26 Aug 2023 23:50:21 +0800 Message-Id: <20230826155028.4019470-5-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S6 X-Coremail-Antispam: 1UD129KBjvJXoW3XF47WF43tw47Zr17uryxuFg_yoW7GFW3pF nIyF1DCr1fWr1DuFWIk34qqF1rKw48Gw1rG34xGr1rCF12krZ8Aay8ta40vasrKFW7A3Z0 vr4YyayUCr47W37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Gr0_Xr1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3w A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAI cVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2js IEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxU3TKuUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" call ext4_mb_mark_context in ext4_mb_mark_diskspace_used to: 1. remove repeat code to normally update bitmap and group descriptor on disk. 2. call ext4_mb_mark_context instead of only setting bits in block bitmap to fix the bitmap. Function ext4_mb_mark_context will also update checksum of bitmap and other counter along with the bit change to keep the cosistent with bit change or block bitmap will be marked corrupted as checksum of bitmap is in inconsistent state. Signed-off-by: Kemeng Shi Reviewed-by: Ojaswin Mujoo Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc.c | 86 +++++++++++------------------------------------ 1 file changed, 20 insertions(+), 66 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index b066ee018cdb..e650eac22237 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -4084,46 +4084,28 @@ static noinline_for_stack int ext4_mb_mark_diskspace_used(struct ext4_allocation_context *ac, handle_t *handle, unsigned int reserv_clstrs) { - struct buffer_head *bitmap_bh =3D NULL; + struct ext4_mark_context mc; struct ext4_group_desc *gdp; - struct buffer_head *gdp_bh; struct ext4_sb_info *sbi; struct super_block *sb; ext4_fsblk_t block; int err, len; + int flags =3D 0; =20 BUG_ON(ac->ac_status !=3D AC_STATUS_FOUND); BUG_ON(ac->ac_b_ex.fe_len <=3D 0); =20 sb =3D ac->ac_sb; sbi =3D EXT4_SB(sb); + ext4_mb_prepare_mark_context(&mc, handle, sb, 1); =20 - bitmap_bh =3D ext4_read_block_bitmap(sb, ac->ac_b_ex.fe_group); - if (IS_ERR(bitmap_bh)) { - return PTR_ERR(bitmap_bh); - } - - BUFFER_TRACE(bitmap_bh, "getting write access"); - err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, - EXT4_JTR_NONE); - if (err) - goto out_err; - - err =3D -EIO; - gdp =3D ext4_get_group_desc(sb, ac->ac_b_ex.fe_group, &gdp_bh); + gdp =3D ext4_get_group_desc(sb, ac->ac_b_ex.fe_group, NULL); if (!gdp) - goto out_err; - + return -EIO; ext4_debug("using block group %u(%d)\n", ac->ac_b_ex.fe_group, ext4_free_group_clusters(sb, gdp)); =20 - BUFFER_TRACE(gdp_bh, "get_write_access"); - err =3D ext4_journal_get_write_access(handle, sb, gdp_bh, EXT4_JTR_NONE); - if (err) - goto out_err; - block =3D ext4_grp_offs_to_block(sb, &ac->ac_b_ex); - len =3D EXT4_C2B(sbi, ac->ac_b_ex.fe_len); if (!ext4_inode_block_valid(ac->ac_inode, block, len)) { ext4_error(sb, "Allocating blocks %llu-%llu which overlap " @@ -4132,41 +4114,28 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_= context *ac, * Fix the bitmap and return EFSCORRUPTED * We leak some of the blocks here. */ - ext4_lock_group(sb, ac->ac_b_ex.fe_group); - mb_set_bits(bitmap_bh->b_data, ac->ac_b_ex.fe_start, - ac->ac_b_ex.fe_len); - ext4_unlock_group(sb, ac->ac_b_ex.fe_group); - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); + err =3D ext4_mb_mark_context(&mc, ac->ac_b_ex.fe_group, + ac->ac_b_ex.fe_start, + ac->ac_b_ex.fe_len, + 0); if (!err) err =3D -EFSCORRUPTED; - goto out_err; + return err; } =20 - ext4_lock_group(sb, ac->ac_b_ex.fe_group); #ifdef AGGRESSIVE_CHECK - { - int i; - for (i =3D 0; i < ac->ac_b_ex.fe_len; i++) { - BUG_ON(mb_test_bit(ac->ac_b_ex.fe_start + i, - bitmap_bh->b_data)); - } - } + flags |=3D EXT4_MB_BITMAP_MARKED_CHECK; #endif - mb_set_bits(bitmap_bh->b_data, ac->ac_b_ex.fe_start, - ac->ac_b_ex.fe_len); - if (ext4_has_group_desc_csum(sb) && - (gdp->bg_flags & cpu_to_le16(EXT4_BG_BLOCK_UNINIT))) { - gdp->bg_flags &=3D cpu_to_le16(~EXT4_BG_BLOCK_UNINIT); - ext4_free_group_clusters_set(sb, gdp, - ext4_free_clusters_after_init(sb, - ac->ac_b_ex.fe_group, gdp)); - } - len =3D ext4_free_group_clusters(sb, gdp) - ac->ac_b_ex.fe_len; - ext4_free_group_clusters_set(sb, gdp, len); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, ac->ac_b_ex.fe_group, gdp); + err =3D ext4_mb_mark_context(&mc, ac->ac_b_ex.fe_group, + ac->ac_b_ex.fe_start, ac->ac_b_ex.fe_len, + flags); + + if (err && mc.changed =3D=3D 0) + return err; =20 - ext4_unlock_group(sb, ac->ac_b_ex.fe_group); +#ifdef AGGRESSIVE_CHECK + BUG_ON(mc.changed !=3D ac->ac_b_ex.fe_len); +#endif percpu_counter_sub(&sbi->s_freeclusters_counter, ac->ac_b_ex.fe_len); /* * Now reduce the dirty block count also. Should not go negative @@ -4176,21 +4145,6 @@ ext4_mb_mark_diskspace_used(struct ext4_allocation_c= ontext *ac, percpu_counter_sub(&sbi->s_dirtyclusters_counter, reserv_clstrs); =20 - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, - ac->ac_b_ex.fe_group); - atomic64_sub(ac->ac_b_ex.fe_len, - &sbi_array_rcu_deref(sbi, s_flex_groups, - flex_group)->free_clusters); - } - - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); - if (err) - goto out_err; - err =3D ext4_handle_dirty_metadata(handle, NULL, gdp_bh); - -out_err: - brelse(bitmap_bh); return err; } =20 --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46DCEC83F11 for ; Sat, 26 Aug 2023 07:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232089AbjHZHvY (ORCPT ); Sat, 26 Aug 2023 03:51:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231352AbjHZHut (ORCPT ); Sat, 26 Aug 2023 03:50:49 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C837213D; Sat, 26 Aug 2023 00:50:45 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RXpsG3gz8z4f3jYp; Sat, 26 Aug 2023 15:50:42 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S7; Sat, 26 Aug 2023 15:50:43 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 05/11] ext4: Separate block bitmap and buddy bitmap freeing in ext4_mb_clear_bb() Date: Sat, 26 Aug 2023 23:50:22 +0800 Message-Id: <20230826155028.4019470-6-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S7 X-Coremail-Antispam: 1UD129KBjvJXoWxCF45ZF15XFy8XF4UWr4kWFg_yoW5CF18pr yqkr1UCrn5GF4v9F4I934jq3WfKw48Wa1DGrW3ur18CFy7Xr9akFn7tFy3AF1DtFs7Janr Xw4Y93yUur4fWa7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3w A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAI cVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2js IEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxUIVyIUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch separates block bitmap and buddy bitmap freeing in order to udpate block bitmap with ext4_mb_mark_context in following patch. Separated freeing is safe with concurrent allocation as long as: 1. Firstly allocate block in buddy bitmap, and then in block bitmap. 2. Firstly free block in block bitmap, and then buddy bitmap. Then freed block will only be available to allocation when both buddy bitmap and block bitmap are updated by freeing. Allocation obeys rule 1 already, just do sperated freeing with rule 2. Separated freeing has no race with generate_buddy as: Once ext4_mb_load_buddy_gfp is executed successfully, the update-to-date buddy page can be found in sbi->s_buddy_cache and no more buddy initialization of the buddy page will be executed concurrently until buddy page is unloaded. As we always do free in "load buddy, free, unload buddy" sequence, separated freeing has no race with generate_buddy. Signed-off-by: Kemeng Shi --- fs/ext4/mballoc.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index e650eac22237..01ad36a1cc96 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6519,6 +6519,14 @@ static void ext4_mb_clear_bb(handle_t *handle, struc= t inode *inode, if (err) goto error_return; =20 + ext4_lock_group(sb, block_group); + mb_clear_bits(bitmap_bh->b_data, bit, count_clusters); + ret =3D ext4_free_group_clusters(sb, gdp) + count_clusters; + ext4_free_group_clusters_set(sb, gdp, ret); + ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); + ext4_group_desc_csum_set(sb, block_group, gdp); + ext4_unlock_group(sb, block_group); + /* * We need to make sure we don't reuse the freed block until after the * transaction is committed. We make an exception if the inode is to be @@ -6541,13 +6549,8 @@ static void ext4_mb_clear_bb(handle_t *handle, struc= t inode *inode, new_entry->efd_tid =3D handle->h_transaction->t_tid; =20 ext4_lock_group(sb, block_group); - mb_clear_bits(bitmap_bh->b_data, bit, count_clusters); ext4_mb_free_metadata(handle, &e4b, new_entry); } else { - /* need to update group_info->bb_free and bitmap - * with group lock held. generate_buddy look at - * them with group lock_held - */ if (test_opt(sb, DISCARD)) { err =3D ext4_issue_discard(sb, block_group, bit, count_clusters, NULL); @@ -6560,14 +6563,9 @@ static void ext4_mb_clear_bb(handle_t *handle, struc= t inode *inode, EXT4_MB_GRP_CLEAR_TRIMMED(e4b.bd_info); =20 ext4_lock_group(sb, block_group); - mb_clear_bits(bitmap_bh->b_data, bit, count_clusters); mb_free_blocks(inode, &e4b, bit, count_clusters); } =20 - ret =3D ext4_free_group_clusters(sb, gdp) + count_clusters; - ext4_free_group_clusters_set(sb, gdp, ret); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, block_group, gdp); ext4_unlock_group(sb, block_group); =20 if (sbi->s_log_groups_per_flex) { --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9E97C83F1B for ; Sat, 26 Aug 2023 07:51:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232059AbjHZHvW (ORCPT ); Sat, 26 Aug 2023 03:51:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58822 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231778AbjHZHut (ORCPT ); Sat, 26 Aug 2023 03:50:49 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D45972683; Sat, 26 Aug 2023 00:50:45 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RXpsC6QdTz4f3jRD; Sat, 26 Aug 2023 15:50:39 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S8; Sat, 26 Aug 2023 15:50:43 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 06/11] ext4: call ext4_mb_mark_context in ext4_mb_clear_bb Date: Sat, 26 Aug 2023 23:50:23 +0800 Message-Id: <20230826155028.4019470-7-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S8 X-Coremail-Antispam: 1UD129KBjvJXoWxCFWxCw4rZF4DKr43ArW3Awb_yoW7JryrpF n8JFnrCr4rGFnF9F40k34UXF1fKw18W347JrWfCr93Cr1ayr9aka97tFnYvFW5tFZ7X3Wq qr1Y93yUur4xW37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3w A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAI cVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2js IEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxUIVyIUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Call ext4_mb_mark_context in ext4_mb_clear_bb to remove repeat code. Signed-off-by: Kemeng Shi --- fs/ext4/mballoc.c | 83 +++++++++++------------------------------------ 1 file changed, 19 insertions(+), 64 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 01ad36a1cc96..25265531cb6a 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6424,21 +6424,20 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, ext4_fsblk_t block, unsigned long count, int flags) { - struct buffer_head *bitmap_bh =3D NULL; + struct ext4_mark_context mc; struct super_block *sb =3D inode->i_sb; - struct ext4_group_desc *gdp; struct ext4_group_info *grp; unsigned int overflow; ext4_grpblk_t bit; - struct buffer_head *gd_bh; ext4_group_t block_group; struct ext4_sb_info *sbi; struct ext4_buddy e4b; unsigned int count_clusters; int err =3D 0; - int ret; + int mark_flags =3D 0; =20 sbi =3D EXT4_SB(sb); + ext4_mb_prepare_mark_context(&mc, handle, sb, 0); =20 if (!(flags & EXT4_FREE_BLOCKS_VALIDATED) && !ext4_inode_block_valid(inode, block, count)) { @@ -6468,18 +6467,6 @@ static void ext4_mb_clear_bb(handle_t *handle, struc= t inode *inode, /* The range changed so it's no longer validated */ flags &=3D ~EXT4_FREE_BLOCKS_VALIDATED; } - count_clusters =3D EXT4_NUM_B2C(sbi, count); - bitmap_bh =3D ext4_read_block_bitmap(sb, block_group); - if (IS_ERR(bitmap_bh)) { - err =3D PTR_ERR(bitmap_bh); - bitmap_bh =3D NULL; - goto error_return; - } - gdp =3D ext4_get_group_desc(sb, block_group, &gd_bh); - if (!gdp) { - err =3D -EIO; - goto error_return; - } =20 if (!(flags & EXT4_FREE_BLOCKS_VALIDATED) && !ext4_inode_block_valid(inode, block, count)) { @@ -6489,28 +6476,7 @@ static void ext4_mb_clear_bb(handle_t *handle, struc= t inode *inode, goto error_return; } =20 - BUFFER_TRACE(bitmap_bh, "getting write access"); - err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, - EXT4_JTR_NONE); - if (err) - goto error_return; - - /* - * We are about to modify some metadata. Call the journal APIs - * to unshare ->b_data if a currently-committing transaction is - * using it - */ - BUFFER_TRACE(gd_bh, "get_write_access"); - err =3D ext4_journal_get_write_access(handle, sb, gd_bh, EXT4_JTR_NONE); - if (err) - goto error_return; -#ifdef AGGRESSIVE_CHECK - { - int i; - for (i =3D 0; i < count_clusters; i++) - BUG_ON(!mb_test_bit(bit + i, bitmap_bh->b_data)); - } -#endif + count_clusters =3D EXT4_NUM_B2C(sbi, count); trace_ext4_mballoc_free(sb, inode, block_group, bit, count_clusters); =20 /* __GFP_NOFAIL: retry infinitely, ignore TIF_MEMDIE and memcg limit. */ @@ -6519,13 +6485,21 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, if (err) goto error_return; =20 - ext4_lock_group(sb, block_group); - mb_clear_bits(bitmap_bh->b_data, bit, count_clusters); - ret =3D ext4_free_group_clusters(sb, gdp) + count_clusters; - ext4_free_group_clusters_set(sb, gdp, ret); - ext4_block_bitmap_csum_set(sb, gdp, bitmap_bh); - ext4_group_desc_csum_set(sb, block_group, gdp); - ext4_unlock_group(sb, block_group); +#ifdef AGGRESSIVE_CHECK + mark_flags |=3D EXT4_MB_BITMAP_MARKED_CHECK; +#endif + err =3D ext4_mb_mark_context(&mc, block_group, bit, count_clusters, + mark_flags); + + + if (err && mc.changed =3D=3D 0) { + ext4_mb_unload_buddy(&e4b); + goto error_return; + } + +#ifdef AGGRESSIVE_CHECK + BUG_ON(mc.changed !=3D count_clusters); +#endif =20 /* * We need to make sure we don't reuse the freed block until after the @@ -6568,13 +6542,6 @@ static void ext4_mb_clear_bb(handle_t *handle, struc= t inode *inode, =20 ext4_unlock_group(sb, block_group); =20 - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, block_group); - atomic64_add(count_clusters, - &sbi_array_rcu_deref(sbi, s_flex_groups, - flex_group)->free_clusters); - } - /* * on a bigalloc file system, defer the s_freeclusters_counter * update to the caller (ext4_remove_space and friends) so they @@ -6589,26 +6556,14 @@ static void ext4_mb_clear_bb(handle_t *handle, stru= ct inode *inode, =20 ext4_mb_unload_buddy(&e4b); =20 - /* We dirtied the bitmap block */ - BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); - - /* And the group descriptor block */ - BUFFER_TRACE(gd_bh, "dirtied group descriptor block"); - ret =3D ext4_handle_dirty_metadata(handle, NULL, gd_bh); - if (!err) - err =3D ret; - if (overflow && !err) { block +=3D count; count =3D overflow; - put_bh(bitmap_bh); /* The range changed so it's no longer validated */ flags &=3D ~EXT4_FREE_BLOCKS_VALIDATED; goto do_more; } error_return: - brelse(bitmap_bh); ext4_std_error(sb, err); } =20 --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BEC2C83F23 for ; Sat, 26 Aug 2023 07:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232099AbjHZHv1 (ORCPT ); Sat, 26 Aug 2023 03:51:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230386AbjHZHut (ORCPT ); Sat, 26 Aug 2023 03:50:49 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3C1226A1; Sat, 26 Aug 2023 00:50:45 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RXpsG1xvFz4f3lVF; Sat, 26 Aug 2023 15:50:42 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S9; Sat, 26 Aug 2023 15:50:43 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 07/11] ext4: Separate block bitmap and buddy bitmap freeing in ext4_group_add_blocks() Date: Sat, 26 Aug 2023 23:50:24 +0800 Message-Id: <20230826155028.4019470-8-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S9 X-Coremail-Antispam: 1UD129KBjvJXoW7tryDJw4fuw43WFWkKry8Krg_yoW8KF4Upr 9Fkr15Crn5GrnruFs7C34jq3W8K3ykCF45GrWrZr1xCF9xXF9ak3WvyF15Ja1UtFs3AFsF q3yY934xuw129aDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3w A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Gr0_Xr1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAI cVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2js IEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxUIVyIUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This patch separates block bitmap and buddy bitmap freeing in order to udpate block bitmap with ext4_mb_mark_context in following patch. The reason why this can be sperated is explained in previous submit. Put the explanation here to simplify the code archeology to ext4_group_add_blocks(): Separated freeing is safe with concurrent allocation as long as: 1. Firstly allocate block in buddy bitmap, and then in block bitmap. 2. Firstly free block in block bitmap, and then buddy bitmap. Then freed block will only be available to allocation when both buddy bitmap and block bitmap are updated by freeing. Allocation obeys rule 1 already, just do sperated freeing with rule 2. Separated freeing has no race with generate_buddy as: Once ext4_mb_load_buddy_gfp is executed successfully, the update-to-date buddy page can be found in sbi->s_buddy_cache and no more buddy initialization of the buddy page will be executed concurrently until buddy page is unloaded. As we always do free in "load buddy, free, unload buddy" sequence, separated freeing has no race with generate_buddy. Signed-off-by: Kemeng Shi --- fs/ext4/mballoc.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 25265531cb6a..bb08a71a6e61 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6764,20 +6764,19 @@ int ext4_group_add_blocks(handle_t *handle, struct = super_block *sb, if (err) goto error_return; =20 - /* - * need to update group_info->bb_free and bitmap - * with group lock held. generate_buddy look at - * them with group lock_held - */ ext4_lock_group(sb, block_group); mb_clear_bits(bitmap_bh->b_data, bit, cluster_count); - mb_free_blocks(NULL, &e4b, bit, cluster_count); free_clusters_count =3D clusters_freed + ext4_free_group_clusters(sb, desc); ext4_free_group_clusters_set(sb, desc, free_clusters_count); ext4_block_bitmap_csum_set(sb, desc, bitmap_bh); ext4_group_desc_csum_set(sb, block_group, desc); ext4_unlock_group(sb, block_group); + + ext4_lock_group(sb, block_group); + mb_free_blocks(NULL, &e4b, bit, cluster_count); + ext4_unlock_group(sb, block_group); + percpu_counter_add(&sbi->s_freeclusters_counter, clusters_freed); =20 --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED7CDC83F29 for ; Sat, 26 Aug 2023 07:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232140AbjHZHvf (ORCPT ); Sat, 26 Aug 2023 03:51:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232021AbjHZHuv (ORCPT ); Sat, 26 Aug 2023 03:50:51 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 083971FF0; Sat, 26 Aug 2023 00:50:48 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RXpsD2xJzz4f3jXr; Sat, 26 Aug 2023 15:50:40 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S10; Sat, 26 Aug 2023 15:50:43 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 08/11] ext4: call ext4_mb_mark_context in ext4_group_add_blocks() Date: Sat, 26 Aug 2023 23:50:25 +0800 Message-Id: <20230826155028.4019470-9-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S10 X-Coremail-Antispam: 1UD129KBjvJXoWxGry5Cry5uw43Xr15tw45trb_yoWrtw1fpr nIyFnrCryrGrnruF48Aa4jq3WrKw48C3W7WrW3CryfCFy2yFn3CF97tFnY9FWDtFZ7ZFnr Xw1Y9348ursxW37anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3w A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Xr0_Ar1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAI cVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2js IEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxUIVyIUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Call ext4_mb_mark_context in ext4_group_add_blocks() to remove repeat code. Signed-off-by: Kemeng Shi --- fs/ext4/mballoc.c | 84 +++++++---------------------------------------- 1 file changed, 12 insertions(+), 72 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index bb08a71a6e61..fdffa3b40bcd 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -6681,25 +6681,22 @@ void ext4_free_blocks(handle_t *handle, struct inod= e *inode, int ext4_group_add_blocks(handle_t *handle, struct super_block *sb, ext4_fsblk_t block, unsigned long count) { - struct buffer_head *bitmap_bh =3D NULL; - struct buffer_head *gd_bh; + struct ext4_mark_context mc; ext4_group_t block_group; ext4_grpblk_t bit; - unsigned int i; - struct ext4_group_desc *desc; struct ext4_sb_info *sbi =3D EXT4_SB(sb); struct ext4_buddy e4b; - int err =3D 0, ret, free_clusters_count; - ext4_grpblk_t clusters_freed; + int err =3D 0; ext4_fsblk_t first_cluster =3D EXT4_B2C(sbi, block); ext4_fsblk_t last_cluster =3D EXT4_B2C(sbi, block + count - 1); unsigned long cluster_count =3D last_cluster - first_cluster + 1; =20 ext4_debug("Adding block(s) %llu-%llu\n", block, block + count - 1); =20 - if (count =3D=3D 0) + if (cluster_count =3D=3D 0) return 0; =20 + ext4_mb_prepare_mark_context(&mc, handle, sb, 0); ext4_get_group_no_and_offset(sb, block, &block_group, &bit); /* * Check to see if we are freeing blocks across a group @@ -6712,19 +6709,6 @@ int ext4_group_add_blocks(handle_t *handle, struct s= uper_block *sb, goto error_return; } =20 - bitmap_bh =3D ext4_read_block_bitmap(sb, block_group); - if (IS_ERR(bitmap_bh)) { - err =3D PTR_ERR(bitmap_bh); - bitmap_bh =3D NULL; - goto error_return; - } - - desc =3D ext4_get_group_desc(sb, block_group, &gd_bh); - if (!desc) { - err =3D -EIO; - goto error_return; - } - if (!ext4_sb_block_valid(sb, NULL, block, count)) { ext4_error(sb, "Adding blocks in system zones - " "Block =3D %llu, count =3D %lu", @@ -6733,74 +6717,30 @@ int ext4_group_add_blocks(handle_t *handle, struct = super_block *sb, goto error_return; } =20 - BUFFER_TRACE(bitmap_bh, "getting write access"); - err =3D ext4_journal_get_write_access(handle, sb, bitmap_bh, - EXT4_JTR_NONE); + err =3D ext4_mb_load_buddy(sb, block_group, &e4b); if (err) goto error_return; =20 - /* - * We are about to modify some metadata. Call the journal APIs - * to unshare ->b_data if a currently-committing transaction is - * using it - */ - BUFFER_TRACE(gd_bh, "get_write_access"); - err =3D ext4_journal_get_write_access(handle, sb, gd_bh, EXT4_JTR_NONE); - if (err) + err =3D ext4_mb_mark_context(&mc, block_group, bit, cluster_count, + EXT4_MB_BITMAP_MARKED_CHECK); + if (err && mc.changed =3D=3D 0) { + ext4_mb_unload_buddy(&e4b); goto error_return; - - for (i =3D 0, clusters_freed =3D 0; i < cluster_count; i++) { - BUFFER_TRACE(bitmap_bh, "clear bit"); - if (!mb_test_bit(bit + i, bitmap_bh->b_data)) { - ext4_error(sb, "bit already cleared for block %llu", - (ext4_fsblk_t)(block + i)); - BUFFER_TRACE(bitmap_bh, "bit already cleared"); - } else { - clusters_freed++; - } } =20 - err =3D ext4_mb_load_buddy(sb, block_group, &e4b); - if (err) - goto error_return; - - ext4_lock_group(sb, block_group); - mb_clear_bits(bitmap_bh->b_data, bit, cluster_count); - free_clusters_count =3D clusters_freed + - ext4_free_group_clusters(sb, desc); - ext4_free_group_clusters_set(sb, desc, free_clusters_count); - ext4_block_bitmap_csum_set(sb, desc, bitmap_bh); - ext4_group_desc_csum_set(sb, block_group, desc); - ext4_unlock_group(sb, block_group); + if (mc.changed !=3D cluster_count) + ext4_error(sb, "bit already cleared in group %u", block_group); =20 ext4_lock_group(sb, block_group); mb_free_blocks(NULL, &e4b, bit, cluster_count); ext4_unlock_group(sb, block_group); =20 percpu_counter_add(&sbi->s_freeclusters_counter, - clusters_freed); - - if (sbi->s_log_groups_per_flex) { - ext4_group_t flex_group =3D ext4_flex_group(sbi, block_group); - atomic64_add(clusters_freed, - &sbi_array_rcu_deref(sbi, s_flex_groups, - flex_group)->free_clusters); - } + mc.changed); =20 ext4_mb_unload_buddy(&e4b); =20 - /* We dirtied the bitmap block */ - BUFFER_TRACE(bitmap_bh, "dirtied bitmap block"); - err =3D ext4_handle_dirty_metadata(handle, NULL, bitmap_bh); - - /* And the group descriptor block */ - BUFFER_TRACE(gd_bh, "dirtied group descriptor block"); - ret =3D ext4_handle_dirty_metadata(handle, NULL, gd_bh); - if (!err) - err =3D ret; - error_return: - brelse(bitmap_bh); ext4_std_error(sb, err); return err; } --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B13BC83F18 for ; Sat, 26 Aug 2023 07:51:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232046AbjHZHvU (ORCPT ); Sat, 26 Aug 2023 03:51:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58838 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232018AbjHZHut (ORCPT ); Sat, 26 Aug 2023 03:50:49 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 809A226A6; Sat, 26 Aug 2023 00:50:46 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RXpsH3r2Jz4f3jq3; Sat, 26 Aug 2023 15:50:43 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S11; Sat, 26 Aug 2023 15:50:44 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 09/11] ext4: add some kunit stub for mballoc kunit test Date: Sat, 26 Aug 2023 23:50:26 +0800 Message-Id: <20230826155028.4019470-10-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S11 X-Coremail-Antispam: 1UD129KBjvJXoWxWFW8Xw15JFWDZryUGrW3KFg_yoW5Jw4rp3 ZrAF1DGr43Ww1q9F4Ika40q3WxGw1xKF1UJryfur13uFW7JrnxuFWktFn0yF4YvFsxZFsr X3W5Zry3Cr1fW3DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3w A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Xr0_Ar1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAI cVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2js IEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxUIVyIUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Multiblocks allocation will read and write block bitmap and group descriptor which reside on disk. Add kunit stub to function ext4_get_group_desc, ext4_read_block_bitmap_nowait, ext4_wait_block_bitmap and ext4_mb_mark_context to avoid real IO to disk. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/balloc.c | 10 ++++++++++ fs/ext4/mballoc.c | 4 ++++ 2 files changed, 14 insertions(+) diff --git a/fs/ext4/balloc.c b/fs/ext4/balloc.c index 79b20d6ae39e..b0bf255b159f 100644 --- a/fs/ext4/balloc.c +++ b/fs/ext4/balloc.c @@ -22,6 +22,7 @@ #include "mballoc.h" =20 #include +#include =20 static unsigned ext4_num_base_meta_clusters(struct super_block *sb, ext4_group_t block_group); @@ -274,6 +275,9 @@ struct ext4_group_desc * ext4_get_group_desc(struct sup= er_block *sb, struct ext4_sb_info *sbi =3D EXT4_SB(sb); struct buffer_head *bh_p; =20 + KUNIT_STATIC_STUB_REDIRECT(ext4_get_group_desc, + sb, block_group, bh); + if (block_group >=3D ngroups) { ext4_error(sb, "block_group >=3D groups_count - block_group =3D %u," " groups_count =3D %u", block_group, ngroups); @@ -468,6 +472,9 @@ ext4_read_block_bitmap_nowait(struct super_block *sb, e= xt4_group_t block_group, ext4_fsblk_t bitmap_blk; int err; =20 + KUNIT_STATIC_STUB_REDIRECT(ext4_read_block_bitmap_nowait, + sb, block_group, ignore_locked); + desc =3D ext4_get_group_desc(sb, block_group, NULL); if (!desc) return ERR_PTR(-EFSCORRUPTED); @@ -563,6 +570,9 @@ int ext4_wait_block_bitmap(struct super_block *sb, ext4= _group_t block_group, { struct ext4_group_desc *desc; =20 + KUNIT_STATIC_STUB_REDIRECT(ext4_wait_block_bitmap, + sb, block_group, bh); + if (!buffer_new(bh)) return 0; desc =3D ext4_get_group_desc(sb, block_group, NULL); diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index fdffa3b40bcd..5961b08ae7df 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -17,6 +17,7 @@ #include #include #include +#include =20 /* * MUSTDO: @@ -3990,6 +3991,9 @@ ext4_mb_mark_context(struct ext4_mark_context *mc, ex= t4_group_t group, int err; unsigned int i, already, changed =3D len; =20 + KUNIT_STATIC_STUB_REDIRECT(ext4_mb_mark_context, + mc, group, blkoff, len, flags); + mc->changed =3D 0; bitmap_bh =3D ext4_read_block_bitmap(sb, group); if (IS_ERR(bitmap_bh)) --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE375C83F2A for ; Sat, 26 Aug 2023 07:51:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232134AbjHZHve (ORCPT ); Sat, 26 Aug 2023 03:51:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58882 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232024AbjHZHuv (ORCPT ); Sat, 26 Aug 2023 03:50:51 -0400 Received: from dggsgout12.his.huawei.com (dggsgout12.his.huawei.com [45.249.212.56]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E83A31991; Sat, 26 Aug 2023 00:50:47 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4RXpsD6bSPz4f3kFP; Sat, 26 Aug 2023 15:50:40 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S12; Sat, 26 Aug 2023 15:50:44 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 10/11] ext4: add first unit test for ext4_mb_new_blocks_simple in mballoc Date: Sat, 26 Aug 2023 23:50:27 +0800 Message-Id: <20230826155028.4019470-11-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S12 X-Coremail-Antispam: 1UD129KBjvJXoW3CFWftw45WrWUuF4xGr4UArb_yoWDZFy7pa n3AF1rur43WFsrWayxKryjq3WSgw18AryUtryfuw1YkFyxAr93JFn7tF10yF4F9FWxJFnx Xa1YkryUCrW7Ga7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3w A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Xr0_Ar1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAI cVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2js IEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxUIVyIUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Here are prepared work: 1. Include mballoc-test.c to mballoc.c to be able test static function in mballoc.c. 2. Implement static stub to avoid read IO to disk. 3. Construct fake super_block. Only partial members are set, more members will be set when more functions are tested. Then unit test for ext4_mb_new_blocks_simple is added. Signed-off-by: Kemeng Shi --- fs/ext4/mballoc-test.c | 322 +++++++++++++++++++++++++++++++++++++++++ fs/ext4/mballoc.c | 4 + 2 files changed, 326 insertions(+) create mode 100644 fs/ext4/mballoc-test.c diff --git a/fs/ext4/mballoc-test.c b/fs/ext4/mballoc-test.c new file mode 100644 index 000000000000..d643c56ac003 --- /dev/null +++ b/fs/ext4/mballoc-test.c @@ -0,0 +1,322 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * KUnit test of ext4 multiblocks allocation. + */ + +#include +#include + +#include "ext4.h" + +struct mbt_grp_ctx { + struct buffer_head bitmap_bh; + /* desc and gd_bh are just the place holders for now */ + struct ext4_group_desc desc; + struct buffer_head gd_bh; +}; + +struct mbt_ctx { + struct mbt_grp_ctx *grp_ctx; +}; + +struct mbt_ext4_super_block { + struct super_block sb; + struct mbt_ctx mbt_ctx; +}; + +#define MBT_CTX(_sb) (&(container_of((_sb), struct mbt_ext4_super_block, s= b)->mbt_ctx)) +#define MBT_GRP_CTX(_sb, _group) (&MBT_CTX(_sb)->grp_ctx[_group]) + +static struct super_block *mbt_ext4_alloc_super_block(void) +{ + struct ext4_super_block *es =3D kzalloc(sizeof(*es), GFP_KERNEL); + struct ext4_sb_info *sbi =3D kzalloc(sizeof(*sbi), GFP_KERNEL); + struct mbt_ext4_super_block *fsb =3D kzalloc(sizeof(*fsb), GFP_KERNEL); + + if (fsb =3D=3D NULL || sbi =3D=3D NULL || es =3D=3D NULL) + goto out; + + sbi->s_es =3D es; + fsb->sb.s_fs_info =3D sbi; + return &fsb->sb; + +out: + kfree(fsb); + kfree(sbi); + kfree(es); + return NULL; +} + +static void mbt_ext4_free_super_block(struct super_block *sb) +{ + struct mbt_ext4_super_block *fsb =3D container_of(sb, struct mbt_ext4_sup= er_block, sb); + struct ext4_sb_info *sbi =3D EXT4_SB(sb); + + kfree(sbi->s_es); + kfree(sbi); + kfree(fsb); +} + +struct mbt_ext4_block_layout { + unsigned char blocksize_bits; + unsigned int cluster_bits; + uint32_t blocks_per_group; + ext4_group_t group_count; + uint16_t desc_size; +}; + +static void mbt_init_sb_layout(struct super_block *sb, + struct mbt_ext4_block_layout *layout) +{ + struct ext4_sb_info *sbi =3D EXT4_SB(sb); + struct ext4_super_block *es =3D sbi->s_es; + + sb->s_blocksize =3D 1UL << layout->blocksize_bits; + sb->s_blocksize_bits =3D layout->blocksize_bits; + + sbi->s_groups_count =3D layout->group_count; + sbi->s_blocks_per_group =3D layout->blocks_per_group; + sbi->s_cluster_bits =3D layout->cluster_bits; + sbi->s_cluster_ratio =3D 1U << layout->cluster_bits; + sbi->s_clusters_per_group =3D layout->blocks_per_group >> + layout->cluster_bits; + sbi->s_desc_size =3D layout->desc_size; + + es->s_first_data_block =3D cpu_to_le32(0); + es->s_blocks_count_lo =3D cpu_to_le32(layout->blocks_per_group * + layout->group_count); +} + +static int mbt_grp_ctx_init(struct super_block *sb, + struct mbt_grp_ctx *grp_ctx) +{ + grp_ctx->bitmap_bh.b_data =3D kzalloc(EXT4_BLOCK_SIZE(sb), GFP_KERNEL); + if (grp_ctx->bitmap_bh.b_data =3D=3D NULL) + return -ENOMEM; + + return 0; +} + +static void mbt_grp_ctx_release(struct mbt_grp_ctx *grp_ctx) +{ + kfree(grp_ctx->bitmap_bh.b_data); + grp_ctx->bitmap_bh.b_data =3D NULL; +} + +static void mbt_ctx_mark_used(struct super_block *sb, ext4_group_t group, + unsigned int start, unsigned int len) +{ + struct mbt_grp_ctx *grp_ctx =3D MBT_GRP_CTX(sb, group); + + mb_set_bits(grp_ctx->bitmap_bh.b_data, start, len); +} + +/* called after mbt_init_sb_layout */ +static int mbt_ctx_init(struct super_block *sb) +{ + struct mbt_ctx *ctx =3D MBT_CTX(sb); + ext4_group_t i, ngroups =3D ext4_get_groups_count(sb); + + ctx->grp_ctx =3D kcalloc(ngroups, sizeof(struct mbt_grp_ctx), + GFP_KERNEL); + if (ctx->grp_ctx =3D=3D NULL) + return -ENOMEM; + + for (i =3D 0; i < ngroups; i++) + if (mbt_grp_ctx_init(sb, &ctx->grp_ctx[i])) + goto out; + + /* + * first data block(first cluster in first group) is used by + * metadata, mark it used to avoid to alloc data block at first + * block which will fail ext4_sb_block_valid check. + */ + mb_set_bits(ctx->grp_ctx[0].bitmap_bh.b_data, 0, 1); + + return 0; +out: + while (i-- > 0) + mbt_grp_ctx_release(&ctx->grp_ctx[i]); + kfree(ctx->grp_ctx); + return -ENOMEM; +} + +static void mbt_ctx_release(struct super_block *sb) +{ + struct mbt_ctx *ctx =3D MBT_CTX(sb); + ext4_group_t i, ngroups =3D ext4_get_groups_count(sb); + + for (i =3D 0; i < ngroups; i++) + mbt_grp_ctx_release(&ctx->grp_ctx[i]); + kfree(ctx->grp_ctx); +} + +static struct buffer_head * +ext4_read_block_bitmap_nowait_stub(struct super_block *sb, ext4_group_t bl= ock_group, + bool ignore_locked) +{ + struct mbt_grp_ctx *grp_ctx =3D MBT_GRP_CTX(sb, block_group); + + /* paired with brelse from caller of ext4_read_block_bitmap_nowait */ + get_bh(&grp_ctx->bitmap_bh); + return &grp_ctx->bitmap_bh; +} + +static int ext4_wait_block_bitmap_stub(struct super_block *sb, + ext4_group_t block_group, + struct buffer_head *bh) +{ + return 0; +} + +static struct ext4_group_desc * +ext4_get_group_desc_stub(struct super_block *sb, ext4_group_t block_group, + struct buffer_head **bh) +{ + struct mbt_grp_ctx *grp_ctx =3D MBT_GRP_CTX(sb, block_group); + + if (bh !=3D NULL) + *bh =3D &grp_ctx->gd_bh; + + return &grp_ctx->desc; +} + +static int ext4_mb_mark_context_stub(struct ext4_mark_context *mc, + ext4_group_t group, ext4_grpblk_t blkoff, + ext4_grpblk_t len, int flags) +{ + struct mbt_grp_ctx *grp_ctx =3D MBT_GRP_CTX(mc->sb, group); + struct buffer_head *bitmap_bh =3D &grp_ctx->bitmap_bh; + + if (mc->state) + mb_set_bits(bitmap_bh->b_data, blkoff, len); + else + mb_clear_bits(bitmap_bh->b_data, blkoff, len); + + return 0; +} + +#define TEST_BLOCKSIZE_BITS 10 +#define TEST_CLUSTER_BITS 3 +#define TEST_BLOCKS_PER_GROUP 8192 +#define TEST_GROUP_COUNT 4 +#define TEST_DESC_SIZE 64 +#define TEST_GOAL_GROUP 1 +static int mbt_kunit_init(struct kunit *test) +{ + struct mbt_ext4_block_layout layout =3D { + .blocksize_bits =3D TEST_BLOCKSIZE_BITS, + .cluster_bits =3D TEST_CLUSTER_BITS, + .blocks_per_group =3D TEST_BLOCKS_PER_GROUP, + .group_count =3D TEST_GROUP_COUNT, + .desc_size =3D TEST_DESC_SIZE, + }; + struct super_block *sb; + int ret; + + sb =3D mbt_ext4_alloc_super_block(); + if (sb =3D=3D NULL) + return -ENOMEM; + + mbt_init_sb_layout(sb, &layout); + + ret =3D mbt_ctx_init(sb); + if (ret !=3D 0) { + mbt_ext4_free_super_block(sb); + return ret; + } + + test->priv =3D sb; + kunit_activate_static_stub(test, + ext4_read_block_bitmap_nowait, + ext4_read_block_bitmap_nowait_stub); + kunit_activate_static_stub(test, + ext4_wait_block_bitmap, + ext4_wait_block_bitmap_stub); + kunit_activate_static_stub(test, + ext4_get_group_desc, + ext4_get_group_desc_stub); + kunit_activate_static_stub(test, + ext4_mb_mark_context, + ext4_mb_mark_context_stub); + return 0; +} + +static void mbt_kunit_exit(struct kunit *test) +{ + struct super_block *sb =3D (struct super_block *)test->priv; + + mbt_ctx_release(sb); + mbt_ext4_free_super_block(sb); +} + +static void test_new_blocks_simple(struct kunit *test) +{ + struct super_block *sb =3D (struct super_block *)test->priv; + struct inode inode =3D { .i_sb =3D sb, }; + struct ext4_allocation_request ar; + ext4_group_t i, goal_group =3D TEST_GOAL_GROUP; + int err =3D 0; + ext4_fsblk_t found; + struct ext4_sb_info *sbi =3D EXT4_SB(sb); + + ar.inode =3D &inode; + + /* get block at goal */ + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_EQ_MSG(test, ar.goal, found, + "failed to alloc block at goal, expected %llu found %llu", + ar.goal, found); + + /* get block after goal in goal group */ + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_EQ_MSG(test, ar.goal + EXT4_C2B(sbi, 1), found, + "failed to alloc block after goal in goal group, expected %llu found %ll= u", + ar.goal + 1, found); + + /* get block after goal group */ + mbt_ctx_mark_used(sb, goal_group, 0, EXT4_CLUSTERS_PER_GROUP(sb)); + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_EQ_MSG(test, + ext4_group_first_block_no(sb, goal_group + 1), found, + "failed to alloc block after goal group, expected %llu found %llu", + ext4_group_first_block_no(sb, goal_group + 1), found); + + /* get block before goal group */ + for (i =3D goal_group; i < ext4_get_groups_count(sb); i++) + mbt_ctx_mark_used(sb, i, 0, EXT4_CLUSTERS_PER_GROUP(sb)); + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_EQ_MSG(test, + ext4_group_first_block_no(sb, 0) + EXT4_C2B(sbi, 1), found, + "failed to alloc block before goal group, expected %llu found %llu", + ext4_group_first_block_no(sb, 0 + EXT4_C2B(sbi, 1)), found); + + /* no block available, fail to allocate block */ + for (i =3D 0; i < ext4_get_groups_count(sb); i++) + mbt_ctx_mark_used(sb, i, 0, EXT4_CLUSTERS_PER_GROUP(sb)); + ar.goal =3D ext4_group_first_block_no(sb, goal_group); + found =3D ext4_mb_new_blocks_simple(&ar, &err); + KUNIT_ASSERT_NE_MSG(test, err, 0, + "unexpectedly get block when no block is available"); +} + + +static struct kunit_case mbt_test_cases[] =3D { + KUNIT_CASE(test_new_blocks_simple), + {} +}; + +static struct kunit_suite mbt_test_suite =3D { + .name =3D "ext4_mballoc_test", + .init =3D mbt_kunit_init, + .exit =3D mbt_kunit_exit, + .test_cases =3D mbt_test_cases, +}; + +kunit_test_suites(&mbt_test_suite); + +MODULE_LICENSE("GPL"); diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 5961b08ae7df..7d230d6317e6 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -7041,3 +7041,7 @@ ext4_mballoc_query_range( =20 return error; } + +#ifdef CONFIG_EXT4_KUNIT_TESTS +#include "mballoc-test.c" +#endif --=20 2.30.0 From nobody Fri Dec 19 11:14:12 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DAD0C83F17 for ; Sat, 26 Aug 2023 07:51:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232035AbjHZHvT (ORCPT ); Sat, 26 Aug 2023 03:51:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58854 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231920AbjHZHut (ORCPT ); Sat, 26 Aug 2023 03:50:49 -0400 Received: from dggsgout11.his.huawei.com (unknown [45.249.212.51]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0BACF1FD7; Sat, 26 Aug 2023 00:50:47 -0700 (PDT) Received: from mail02.huawei.com (unknown [172.30.67.169]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4RXpsH25Prz4f3lDW; Sat, 26 Aug 2023 15:50:43 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP4 (Coremail) with SMTP id gCh0CgDXZajPrulkChN1Bg--.43571S13; Sat, 26 Aug 2023 15:50:44 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, ritesh.list@gmail.com, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v6 11/11] ext4: run mballoc test with different layouts setting Date: Sat, 26 Aug 2023 23:50:28 +0800 Message-Id: <20230826155028.4019470-12-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20230826155028.4019470-1-shikemeng@huaweicloud.com> References: <20230826155028.4019470-1-shikemeng@huaweicloud.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: gCh0CgDXZajPrulkChN1Bg--.43571S13 X-Coremail-Antispam: 1UD129KBjvJXoWxAF43KFWUCr47tr4xWw1DGFg_yoW5GFy7pa nIkF1Fkr1UWFsF9w4fK397Zw13Kw1kur48JryxW34FkF1xAws7GFs7tryYya4qqrWkZFn0 vF1Y9w17u3y8Cw7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUBSb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M280x2IEY4vEnII2IxkI6r1a6r45M2 8IrcIa0xkI8VA2jI8067AKxVWUAVCq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAv FVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3w A2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE 3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr2 1l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj6xIIjxv20xvE14v26r1Y6r17McIj6I8E87Iv 67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2 Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s02 6x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0x vE2Ix0cI8IcVAFwI0_Xr0_Ar1lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4UJVWxJr1lIxAI cVCF04k26cxKx2IYs7xG6r1j6r1xMIIF0xvEx4A2jsIE14v26r4j6F4UMIIF0xvEx4A2js IEc7CjxVAFwI0_Gr1j6F4UJbIYCTnIWIevJa73UjIFyTuYvjxUIVyIUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Use KUNIT_CASE_PARAM to run mbalaloc test with different layouts setting. Signed-off-by: Kemeng Shi Reviewed-by: Ritesh Harjani (IBM) --- fs/ext4/mballoc-test.c | 52 ++++++++++++++++++++++++++++++------------ 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/fs/ext4/mballoc-test.c b/fs/ext4/mballoc-test.c index d643c56ac003..af48a39c8ba2 100644 --- a/fs/ext4/mballoc-test.c +++ b/fs/ext4/mballoc-test.c @@ -196,21 +196,11 @@ static int ext4_mb_mark_context_stub(struct ext4_mark= _context *mc, return 0; } =20 -#define TEST_BLOCKSIZE_BITS 10 -#define TEST_CLUSTER_BITS 3 -#define TEST_BLOCKS_PER_GROUP 8192 -#define TEST_GROUP_COUNT 4 -#define TEST_DESC_SIZE 64 #define TEST_GOAL_GROUP 1 static int mbt_kunit_init(struct kunit *test) { - struct mbt_ext4_block_layout layout =3D { - .blocksize_bits =3D TEST_BLOCKSIZE_BITS, - .cluster_bits =3D TEST_CLUSTER_BITS, - .blocks_per_group =3D TEST_BLOCKS_PER_GROUP, - .group_count =3D TEST_GROUP_COUNT, - .desc_size =3D TEST_DESC_SIZE, - }; + struct mbt_ext4_block_layout *layout =3D + (struct mbt_ext4_block_layout *)(test->param_value); struct super_block *sb; int ret; =20 @@ -218,7 +208,7 @@ static int mbt_kunit_init(struct kunit *test) if (sb =3D=3D NULL) return -ENOMEM; =20 - mbt_init_sb_layout(sb, &layout); + mbt_init_sb_layout(sb, layout); =20 ret =3D mbt_ctx_init(sb); if (ret !=3D 0) { @@ -304,9 +294,43 @@ static void test_new_blocks_simple(struct kunit *test) "unexpectedly get block when no block is available"); } =20 +static const struct mbt_ext4_block_layout mbt_test_layouts[] =3D { + { + .blocksize_bits =3D 10, + .cluster_bits =3D 3, + .blocks_per_group =3D 8192, + .group_count =3D 4, + .desc_size =3D 64, + }, + { + .blocksize_bits =3D 12, + .cluster_bits =3D 3, + .blocks_per_group =3D 8192, + .group_count =3D 4, + .desc_size =3D 64, + }, + { + .blocksize_bits =3D 18, + .cluster_bits =3D 3, + .blocks_per_group =3D 8192, + .group_count =3D 4, + .desc_size =3D 64, + }, +}; + +static void mbt_show_layout(const struct mbt_ext4_block_layout *layout, + char *desc) +{ + snprintf(desc, KUNIT_PARAM_DESC_SIZE, "block_bits=3D%d cluster_bits=3D%d " + "blocks_per_group=3D%d group_count=3D%d desc_size=3D%d\n", + layout->blocksize_bits, layout->cluster_bits, + layout->blocks_per_group, layout->group_count, + layout->desc_size); +} +KUNIT_ARRAY_PARAM(mbt_layouts, mbt_test_layouts, mbt_show_layout); =20 static struct kunit_case mbt_test_cases[] =3D { - KUNIT_CASE(test_new_blocks_simple), + KUNIT_CASE_PARAM(test_new_blocks_simple, mbt_layouts_gen_params), {} }; =20 --=20 2.30.0