From nobody Sun May 19 04:56:00 2024 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB4D0156C4C; Wed, 24 Apr 2024 06:19:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939553; cv=none; b=jxf9h2lKgnhsGMAjwvGpHpJ6QyF2NfG60Mv19AR01/xO9Rc+YpOGcZkSrrnFeM2suA+ZGOvifLN2WBwW76TebIQd+4xioZg7bu9qrbDELsVN2ava1kb4OH8bDF2IcQ1RiatzcO65YW8swRHe9CEJwU1vf2IDIgq5pFg1BaBrW5I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939553; c=relaxed/simple; bh=t1U/InzqxiUqGU+e34imVaTcWHfcqLRThUU4nLPa18Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MxlgcW8dq2tZj37oeWJdSeckQnnXIHWRQzQ3CupFDHMiKH85fG4gOHjS9soz/CVeJQLiPjPGaVvRlZo7UfLGfeu7KPMZtKdEpecUCXyGfuOE+4y7wUTbiZS5/HyRT7p8LXsqdP6CtmO1OfDHijQP/t8mf2wRIYahM2qz+sRN0VY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4VPTMk34J7z4f3n6F; Wed, 24 Apr 2024 14:18:58 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 9C8551A0572; Wed, 24 Apr 2024 14:19:07 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP1 (Coremail) with SMTP id cCh0CgBHGRJZpChmpfZBKw--.32510S3; Wed, 24 Apr 2024 14:19:07 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Cc: jack@suse.cz, ojaswin@linux.ibm.com, ritesh.list@gmail.com Subject: [PATCH v3 1/5] ext4: keep "prefetch_grp" and "nr" consistent Date: Wed, 24 Apr 2024 14:19:00 +0800 Message-Id: <20240424061904.987525-2-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20240424061904.987525-1-shikemeng@huaweicloud.com> References: <20240424061904.987525-1-shikemeng@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: cCh0CgBHGRJZpChmpfZBKw--.32510S3 X-Coremail-Antispam: 1UD129KBjvJXoW7Cr1Uur48urWfuFyfWw1rJFb_yoW8GrWUpF nak3y7Cr4rXr1UAayUG3Z0gw18Kw40gw12qFyIkryqq347ZryfJrsrtr4ruFWUZF47AFy3 uFnIy3W8WF1UGa7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvGb4IE77IF4wAFF20E14v26r4j6ryUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUGw A2048vs2IY020Ec7CjxVAFwI0_JFI_Gr1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxS w2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxV W8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v2 6rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMc Ij6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_ Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr 0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY 17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcV C0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY 6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa 73UjIFyTuYvjxUzoGQUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ Content-Type: text/plain; charset="utf-8" Keep "prefetch_grp" and "nr" consistent to avoid to call ext4_mb_prefetch_fini with non-prefetched groups. When we step into next criteria, "prefetch_grp" is set to prefetch start of new criteria while "nr" is number of the prefetched group in previous criteria. If previous criteria and next criteria are both inexpensive (< CR_GOAL_LEN_SLOW) and prefetch_ios reachs sbi->s_mb_prefetch_limit in previous criteria, "prefetch_grp" and "nr" will be inconsistent and may introduce unexpected cost to do ext4_mb_init_group for non-prefetched groups. Reset "nr" to 0 when we reset "prefetch_grp" to goal group to keep them consistent. Signed-off-by: Kemeng Shi Reviewed-by: Ojaswin Mujoo Reviewed-by: Jan Kara --- fs/ext4/mballoc.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 12b3f196010b..a61fc52956b2 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -2856,6 +2856,7 @@ ext4_mb_regular_allocator(struct ext4_allocation_cont= ext *ac) group =3D ac->ac_g_ex.fe_group; ac->ac_groups_linear_remaining =3D sbi->s_mb_max_linear_groups; prefetch_grp =3D group; + nr =3D 0; =20 for (i =3D 0, new_cr =3D cr; i < ngroups; i++, ext4_mb_choose_next_group(ac, &new_cr, &group, ngroups)) { --=20 2.30.0 From nobody Sun May 19 04:56:00 2024 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 140E1156C62; Wed, 24 Apr 2024 06:19:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939554; cv=none; b=SyzNVS5lmdJ5ag2+/KqTGaN3ZYI5vlph7n9ceS5nFdJSYWjLGiPNHKoR/fjD3bpD85jgMd33z+zpZwV6+0rqeJQ9gnK9oz49qrkKy5jKWgWscT2Qarwkrxom9OOSxfScKGvyM6LXmjNunDXj4uPbZm8yWdUKKyMJCd3IslsbKg4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939554; c=relaxed/simple; bh=DRLKy6PWpcyAW3cnassTtwoi1nM6lNE0qZRpoZdKvMk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YcxHTHmIVckzv2CRabi8uuxs7K7tj+Szzg5EEaqItrYhpylHj1XNf9fd+XYn19Sm1FrvuZ0+pLb52cw5xVKJxMtC7OLpppZvwrpHY/njKGOhslSXh79PCzqYNepmddLJPyKDOcvRv7EEwzBZBHukXD3+h5ctvpC86cpShvTil90= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.216]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4VPTMm30K6z4f3jHV; Wed, 24 Apr 2024 14:19:00 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id ECC381A115C; Wed, 24 Apr 2024 14:19:07 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP1 (Coremail) with SMTP id cCh0CgBHGRJZpChmpfZBKw--.32510S4; Wed, 24 Apr 2024 14:19:07 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Cc: jack@suse.cz, ojaswin@linux.ibm.com, ritesh.list@gmail.com Subject: [PATCH v3 2/5] ext4: add test_mb_mark_used_cost to estimate cost of mb_mark_used Date: Wed, 24 Apr 2024 14:19:01 +0800 Message-Id: <20240424061904.987525-3-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20240424061904.987525-1-shikemeng@huaweicloud.com> References: <20240424061904.987525-1-shikemeng@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: cCh0CgBHGRJZpChmpfZBKw--.32510S4 X-Coremail-Antispam: 1UD129KBjvJXoW7uF4fJr1xury7Aw48WryxAFb_yoW5Jr4kpa yjkF1Ykr4rWws7uw4fGws7Ww1rKw4kZr48GryxWr9YqF47AFyfC3Z7tFn8Gr48trs7Xrn8 uFyqqFy7G397CFJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvGb4IE77IF4wAFF20E14v26ryj6rWUM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUXw A2048vs2IY020Ec7CjxVAFwI0_Gr0_Xr1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxS w2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxV W8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v2 6rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMc Ij6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_ Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr 0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY 17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcV C0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY 6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa 73UjIFyTuYvjxU2_MaUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ Content-Type: text/plain; charset="utf-8" Add test_mb_mark_used_cost to estimate cost of mb_mark_used Signed-off-by: Kemeng Shi --- fs/ext4/mballoc-test.c | 52 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/fs/ext4/mballoc-test.c b/fs/ext4/mballoc-test.c index 044ca5238f41..df482782d695 100644 --- a/fs/ext4/mballoc-test.c +++ b/fs/ext4/mballoc-test.c @@ -859,6 +859,56 @@ static void test_mb_free_blocks(struct kunit *test) ext4_mb_unload_buddy(&e4b); } =20 +#define COUNT_FOR_ESTIMATE 100000 +static void test_mb_mark_used_cost(struct kunit *test) +{ + struct ext4_buddy e4b; + struct super_block *sb =3D (struct super_block *)test->priv; + struct ext4_free_extent ex; + int ret; + struct test_range ranges[TEST_RANGE_COUNT]; + int i, j; + unsigned long start, end, all =3D 0; + + /* buddy cache assumes that each page contains at least one block */ + if (sb->s_blocksize > PAGE_SIZE) + kunit_skip(test, "blocksize exceeds pagesize"); + + ret =3D ext4_mb_load_buddy(sb, TEST_GOAL_GROUP, &e4b); + KUNIT_ASSERT_EQ(test, ret, 0); + + ex.fe_group =3D TEST_GOAL_GROUP; + for (j =3D 0; j < COUNT_FOR_ESTIMATE; j++) { + mbt_generate_test_ranges(sb, ranges, TEST_RANGE_COUNT); + start =3D jiffies; + for (i =3D 0; i < TEST_RANGE_COUNT; i++) { + if (ranges[i].len =3D=3D 0) + continue; + + ex.fe_start =3D ranges[i].start; + ex.fe_len =3D ranges[i].len; + ext4_lock_group(sb, TEST_GOAL_GROUP); + mb_mark_used(&e4b, &ex); + ext4_unlock_group(sb, TEST_GOAL_GROUP); + } + end =3D jiffies; + all +=3D (end - start); + + for (i =3D 0; i < TEST_RANGE_COUNT; i++) { + if (ranges[i].len =3D=3D 0) + continue; + + ext4_lock_group(sb, TEST_GOAL_GROUP); + mb_free_blocks(NULL, &e4b, ranges[i].start, + ranges[i].len); + ext4_unlock_group(sb, TEST_GOAL_GROUP); + } + } + + kunit_info(test, "costed jiffies %lu\n", all); + ext4_mb_unload_buddy(&e4b); +} + static const struct mbt_ext4_block_layout mbt_test_layouts[] =3D { { .blocksize_bits =3D 10, @@ -901,6 +951,8 @@ static struct kunit_case mbt_test_cases[] =3D { KUNIT_CASE_PARAM(test_mb_mark_used, mbt_layouts_gen_params), KUNIT_CASE_PARAM(test_mb_free_blocks, mbt_layouts_gen_params), KUNIT_CASE_PARAM(test_mark_diskspace_used, mbt_layouts_gen_params), + KUNIT_CASE_PARAM_ATTR(test_mb_mark_used_cost, mbt_layouts_gen_params, + { .speed =3D KUNIT_SPEED_SLOW }), {} }; =20 --=20 2.30.0 From nobody Sun May 19 04:56:00 2024 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BC4E156C5F; Wed, 24 Apr 2024 06:19:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939553; cv=none; b=AJuAG0OsPD3LbAqRLeJ6A9GAOWseVlPQ3zWCTgccWMkybwWaGf6DJli9C8Q9Yjtp735WaSxwrVMTRLLyrNN5nwaBJ1ShxA8opNcoSzbE82rBNdbLce5hO6cdCalVHeQs2X82HmN+mQy3fgPcOksI3FD10d62RsvoXjbESXDMlg4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939553; c=relaxed/simple; bh=xrcKKyueyRmWYlrLSjxKDPS0MeWA/AvpTFX4TJN6LYU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LALr25LtyaKchOU74LbtgtJ4Ygv97rUA5mqFkX7PuPJjajfnh82TnkZKAa88SqNJGroIwXHa1eJps7JSt8FUXFIo52GyvPAm0fEUGC3hRiUmu/9djSfReEH0QjjFYxK243R5f5HO7Xmta1uVCbQ+xvZWxHjNayCvmhzINV6A6og= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4VPTMl0n15z4f3n6M; Wed, 24 Apr 2024 14:18:59 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 4D3CF1A0572; Wed, 24 Apr 2024 14:19:08 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP1 (Coremail) with SMTP id cCh0CgBHGRJZpChmpfZBKw--.32510S5; Wed, 24 Apr 2024 14:19:08 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Cc: jack@suse.cz, ojaswin@linux.ibm.com, ritesh.list@gmail.com Subject: [PATCH v3 3/5] ext4: call ext4_mb_mark_free_simple to free continuous bits in found chunk Date: Wed, 24 Apr 2024 14:19:02 +0800 Message-Id: <20240424061904.987525-4-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20240424061904.987525-1-shikemeng@huaweicloud.com> References: <20240424061904.987525-1-shikemeng@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: cCh0CgBHGRJZpChmpfZBKw--.32510S5 X-Coremail-Antispam: 1UD129KBjvJXoWxAFy8Jr4rtr15tr17Cr1UAwb_yoW5Zr48pF 43GrW3Gr4rJr929FsrCr1qg34rKw4vyFy5GayfWw1rKFnxAr9Yvr95GrnxZ34kJrWfJ3W0 ya1UurW5Gw47Ka7anT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvGb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUWw A2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxS w2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxV W8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v2 6rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMc Ij6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_ Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7v_Jr 0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF1VAY 17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIxAIcV C0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI42IY 6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWIevJa 73UjIFyTuYvjxUFYFCUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ Content-Type: text/plain; charset="utf-8" In mb_mark_used, we will find free chunk and mark it inuse. For chunk in mid of passed range, we could simply mark whole chunk inuse. For chunk at end of range, we may need to mark a continuous bits at end of part of chunk inuse and keep rest part of chunk free. To only mark a part of chunk inuse, we firstly mark whole chunk inuse and then mark a continuous range at end of chunk free. Function mb_mark_used does several times of "mb_find_buddy; mb_clear_bit; ..." to mark a continuous range free which can be done by simply calling ext4_mb_mark_free_simple which free continuous bits in a more effective way. Just call ext4_mb_mark_free_simple in mb_mark_used to use existing and effective code to free continuous blocks in chunk at end of passed range. Signed-off-by: Kemeng Shi Reviewed-by: Jan Kara --- fs/ext4/mballoc.c | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index a61fc52956b2..5acf413808a2 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -2040,13 +2040,12 @@ static int mb_mark_used(struct ext4_buddy *e4b, str= uct ext4_free_extent *ex) int ord; int mlen =3D 0; int max =3D 0; - int cur; int start =3D ex->fe_start; int len =3D ex->fe_len; unsigned ret =3D 0; int len0 =3D len; void *buddy; - bool split =3D false; + int ord_start, ord_end; =20 BUG_ON(start + len > (e4b->bd_sb->s_blocksize << 3)); BUG_ON(e4b->bd_group !=3D ex->fe_group); @@ -2071,16 +2070,12 @@ static int mb_mark_used(struct ext4_buddy *e4b, str= uct ext4_free_extent *ex) =20 /* let's maintain buddy itself */ while (len) { - if (!split) - ord =3D mb_find_order_for_block(e4b, start); + ord =3D mb_find_order_for_block(e4b, start); =20 if (((start >> ord) << ord) =3D=3D start && len >=3D (1 << ord)) { /* the whole chunk may be allocated at once! */ mlen =3D 1 << ord; - if (!split) - buddy =3D mb_find_buddy(e4b, ord, &max); - else - split =3D false; + buddy =3D mb_find_buddy(e4b, ord, &max); BUG_ON((start >> ord) >=3D max); mb_set_bit(start >> ord, buddy); e4b->bd_info->bb_counters[ord]--; @@ -2094,20 +2089,29 @@ static int mb_mark_used(struct ext4_buddy *e4b, str= uct ext4_free_extent *ex) if (ret =3D=3D 0) ret =3D len | (ord << 16); =20 - /* we have to split large buddy */ BUG_ON(ord <=3D 0); buddy =3D mb_find_buddy(e4b, ord, &max); mb_set_bit(start >> ord, buddy); e4b->bd_info->bb_counters[ord]--; =20 - ord--; - cur =3D (start >> ord) & ~1U; - buddy =3D mb_find_buddy(e4b, ord, &max); - mb_clear_bit(cur, buddy); - mb_clear_bit(cur + 1, buddy); - e4b->bd_info->bb_counters[ord]++; - e4b->bd_info->bb_counters[ord]++; - split =3D true; + ord_start =3D (start >> ord) << ord; + ord_end =3D ord_start + (1 << ord); + /* first chunk */ + if (start > ord_start) + ext4_mb_mark_free_simple(e4b->bd_sb, e4b->bd_buddy, + ord_start, start - ord_start, + e4b->bd_info); + + /* last chunk */ + if (start + len < ord_end) { + ext4_mb_mark_free_simple(e4b->bd_sb, e4b->bd_buddy, + start + len, + ord_end - (start + len), + e4b->bd_info); + break; + } + len =3D start + len - ord_end; + start =3D ord_end; } mb_set_largest_free_order(e4b->bd_sb, e4b->bd_info); =20 --=20 2.30.0 From nobody Sun May 19 04:56:00 2024 Received: from dggsgout11.his.huawei.com (dggsgout11.his.huawei.com [45.249.212.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BC05156C5D; Wed, 24 Apr 2024 06:19:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939554; cv=none; b=RVPaaeDmDqwn1WLIpZX96hn6TMbMBYuVoJW/pXrscDmVy/bqVjL6qdwOGDYe7YSVeVN1prqYKZohchZgTfyqvpb4gebuRihx8+DwCHha3pwS7zRoLtflwBChaXkTJiCsVXSr10OYcnlFp4dwjz1+JedGht4pQ+RJVE2yYh6EvSg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939554; c=relaxed/simple; bh=zW/J/B5sp0BaTPxRGgaqDVSl1Ae1DCfweYILs40wZbU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eC3yIRERdQNn7Q9TXdSAQH/2No/OLCewDXDiJTB6T7E2hl97QiDX5NcDccdjjpRzFcYIIqG4f1Bs6fPilD3c/7DN4xY0r1I1v83LaQW0Jqp6TN/zgPMIelEmbl7dzUUYg2m8Dai/7lvFqsMgAr8Q0t2/JEeEVIiKJ+qa/Ms7en0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.93.142]) by dggsgout11.his.huawei.com (SkyGuard) with ESMTP id 4VPTMq5hG1z4f3kkV; Wed, 24 Apr 2024 14:19:03 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id 9DD441A017C; Wed, 24 Apr 2024 14:19:08 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP1 (Coremail) with SMTP id cCh0CgBHGRJZpChmpfZBKw--.32510S6; Wed, 24 Apr 2024 14:19:08 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Cc: jack@suse.cz, ojaswin@linux.ibm.com, ritesh.list@gmail.com Subject: [PATCH v3 4/5] ext4: use correct criteria name instead stale integer number in comment Date: Wed, 24 Apr 2024 14:19:03 +0800 Message-Id: <20240424061904.987525-5-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20240424061904.987525-1-shikemeng@huaweicloud.com> References: <20240424061904.987525-1-shikemeng@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: cCh0CgBHGRJZpChmpfZBKw--.32510S6 X-Coremail-Antispam: 1UD129KBjvJXoWxZFWrZFyktFWfZFWkCr43GFg_yoWrGryrp3 93GF18Kw4SgFy5urZ7Wa1jg3W8Kw409FyUXryF9w1xuFyaqr93tr9xKFW0yFy09r48A3Wr XF12qr48CF1S93DanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvEb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUAV Cq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0 rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267 AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E 14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7 xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Y z7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7 v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF 1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr0_JF4lIx AIcVC0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI 42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWI evJa73UjIFyTuYvjxUFgAwUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ Content-Type: text/plain; charset="utf-8" Use correct criteria name instead stale integer number in comment Signed-off-by: Kemeng Shi Reviewed-by: Ojaswin Mujoo Reviewed-by: Jan Kara --- fs/ext4/ext4.h | 9 ++++++--- fs/ext4/mballoc.c | 16 +++++++++------- fs/ext4/mballoc.h | 4 ++-- 3 files changed, 17 insertions(+), 12 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 023571f8dd1b..9bd3764d1121 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -213,11 +213,14 @@ enum criteria { #define EXT4_MB_USE_RESERVED 0x2000 /* Do strict check for free blocks while retrying block allocation */ #define EXT4_MB_STRICT_CHECK 0x4000 -/* Large fragment size list lookup succeeded at least once for cr =3D 0 */ +/* Large fragment size list lookup succeeded at least once for + * CR_POWER2_ALIGNED */ #define EXT4_MB_CR_POWER2_ALIGNED_OPTIMIZED 0x8000 -/* Avg fragment size rb tree lookup succeeded at least once for cr =3D 1 */ +/* Avg fragment size rb tree lookup succeeded at least once for + * CR_GOAL_LEN_FAST */ #define EXT4_MB_CR_GOAL_LEN_FAST_OPTIMIZED 0x00010000 -/* Avg fragment size rb tree lookup succeeded at least once for cr =3D 1.5= */ +/* Avg fragment size rb tree lookup succeeded at least once for + * CR_BEST_AVAIL_LEN */ #define EXT4_MB_CR_BEST_AVAIL_LEN_OPTIMIZED 0x00020000 =20 struct ext4_allocation_request { diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 5acf413808a2..3d344a21b7f0 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -1131,8 +1131,9 @@ static void ext4_mb_choose_next_group(struct ext4_all= ocation_context *ac, ext4_mb_choose_next_group_best_avail(ac, new_cr, group); } else { /* - * TODO: For CR=3D2, we can arrange groups in an rb tree sorted by - * bb_free. But until that happens, we should never come here. + * TODO: For CR_GOAL_LEN_SLOW, we can arrange groups in an + * rb tree sorted by bb_free. But until that happens, we should + * never come here. */ WARN_ON(1); } @@ -2679,7 +2680,7 @@ static int ext4_mb_good_group_nolock(struct ext4_allo= cation_context *ac, int ret; =20 /* - * cr=3DCR_POWER2_ALIGNED/CR_GOAL_LEN_FAST is a very optimistic + * CR_POWER2_ALIGNED/CR_GOAL_LEN_FAST is a very optimistic * search to find large good chunks almost for free. If buddy * data is not ready, then this optimization makes no sense. But * we never skip the first block group in a flex_bg, since this @@ -3445,10 +3446,11 @@ static int ext4_mb_init_backend(struct super_block = *sb) } if (sbi->s_mb_prefetch > ext4_get_groups_count(sb)) sbi->s_mb_prefetch =3D ext4_get_groups_count(sb); - /* now many real IOs to prefetch within a single allocation at cr=3D0 - * given cr=3D0 is an CPU-related optimization we shouldn't try to - * load too many groups, at some point we should start to use what - * we've got in memory. + /* + * now many real IOs to prefetch within a single allocation at + * CR_POWER2_ALIGNED. Given CR_POWER2_ALIGNED is an CPU-related + * optimization we shouldn't try to load too many groups, at some point + * we should start to use what we've got in memory. * with an average random access time 5ms, it'd take a second to get * 200 groups (* N with flex_bg), so let's make this limit 4 */ diff --git a/fs/ext4/mballoc.h b/fs/ext4/mballoc.h index 56938532b4ce..042437d8860f 100644 --- a/fs/ext4/mballoc.h +++ b/fs/ext4/mballoc.h @@ -187,8 +187,8 @@ struct ext4_allocation_context { struct ext4_free_extent ac_f_ex; =20 /* - * goal len can change in CR1.5, so save the original len. This is - * used while adjusting the PA window and for accounting. + * goal len can change in CR_BEST_AVAIL_LEN, so save the original len. + * This is used while adjusting the PA window and for accounting. */ ext4_grpblk_t ac_orig_goal_len; =20 --=20 2.30.0 From nobody Sun May 19 04:56:00 2024 Received: from dggsgout12.his.huawei.com (unknown [45.249.212.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14138156C64; Wed, 24 Apr 2024 06:19:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.249.212.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939554; cv=none; b=Jf2fTYS+x37Z2BONWOO2wpB8iOdOdVLAy5O1FygqvQjBCaAQnLe1H59QclzMYAOEavT7/Ruuv1hQ4xjC6or2xTf5f3nrf9KZsk/7frm0w1t5cKpbbWexg1U6GatEqQD47Vt43prng2w2l+OHXmyLM0xNWfNEP/hvHb/3tR3Xtu0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713939554; c=relaxed/simple; bh=Q2xaLIEV8BtfJUj+p2EXVQqVrRzAUXZJ9zUs7OeqwAI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AX4rXY2yRxH6LBTiT44m+pSidXPRc9q8dgfp3cx1Uv44t4tnzrZ4uOWwOz6FwhI/8YkVoB876gQuvg33zNmbfESxh+TkXAQCm6E+5Gz3zNGadsuK61xAvyZsgmAxl0DzWoAV333yuXlB2g9w+JsPOSImEUVuLXwDkEPkRRphTXQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com; spf=pass smtp.mailfrom=huaweicloud.com; arc=none smtp.client-ip=45.249.212.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=huaweicloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huaweicloud.com Received: from mail.maildlp.com (unknown [172.19.163.235]) by dggsgout12.his.huawei.com (SkyGuard) with ESMTP id 4VPTMn2w1Zz4f3khg; Wed, 24 Apr 2024 14:19:01 +0800 (CST) Received: from mail02.huawei.com (unknown [10.116.40.112]) by mail.maildlp.com (Postfix) with ESMTP id E97D51A058D; Wed, 24 Apr 2024 14:19:08 +0800 (CST) Received: from huaweicloud.com (unknown [10.175.124.27]) by APP1 (Coremail) with SMTP id cCh0CgBHGRJZpChmpfZBKw--.32510S7; Wed, 24 Apr 2024 14:19:08 +0800 (CST) From: Kemeng Shi To: tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org Cc: jack@suse.cz, ojaswin@linux.ibm.com, ritesh.list@gmail.com Subject: [PATCH v3 5/5] ext4: open coding repeated check in next_linear_group Date: Wed, 24 Apr 2024 14:19:04 +0800 Message-Id: <20240424061904.987525-6-shikemeng@huaweicloud.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20240424061904.987525-1-shikemeng@huaweicloud.com> References: <20240424061904.987525-1-shikemeng@huaweicloud.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-CM-TRANSID: cCh0CgBHGRJZpChmpfZBKw--.32510S7 X-Coremail-Antispam: 1UD129KBjvJXoW7uF1UWrykAF15tw4fCFW7Arb_yoW8Zr18pF sxAry7Wr42gr1DCa9xGayvq3WSgw48Ga1UJryfWr93tF9xCrn8GFW3tr10vFy7XFWDAr13 Xr43AF4Uu3Z3CaDanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDU0xBIdaVrnRJUUUvEb4IE77IF4wAFF20E14v26rWj6s0DM7CY07I20VC2zVCF04k2 6cxKx2IYs7xG6rWj6s0DM7CIcVAFz4kK6r1j6r18M28IrcIa0xkI8VA2jI8067AKxVWUAV Cq3wA2048vs2IY020Ec7CjxVAFwI0_Xr0E3s1l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0 rcxSw2x7M28EF7xvwVC0I7IYx2IY67AKxVWDJVCq3wA2z4x0Y4vE2Ix0cI8IcVCY1x0267 AKxVW8Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E 14v26rxl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7 xfMcIj6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Y z7v_Jr0_Gr1lF7xvr2IYc2Ij64vIr41l42xK82IYc2Ij64vIr41l4I8I3I0E4IkC6x0Yz7 v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWUWwC2zVAF 1VAY17CE14v26r126r1DMIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_JFI_Gr1lIx AIcVC0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWUJVWUCwCI 42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJbIYCTnIWI evJa73UjIFyTuYvjxUFgAwUUUUU X-CM-SenderInfo: 5vklyvpphqwq5kxd4v5lfo033gof0z/ Content-Type: text/plain; charset="utf-8" Open coding repeated check in next_linear_group. Signed-off-by: Kemeng Shi Reviewed-by: Jan Kara --- fs/ext4/mballoc.c | 31 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/fs/ext4/mballoc.c b/fs/ext4/mballoc.c index 3d344a21b7f0..9f73a76d679a 100644 --- a/fs/ext4/mballoc.c +++ b/fs/ext4/mballoc.c @@ -1076,23 +1076,11 @@ static inline int should_optimize_scan(struct ext4_= allocation_context *ac) } =20 /* - * Return next linear group for allocation. If linear traversal should not= be - * performed, this function just returns the same group + * Return next linear group for allocation. */ static ext4_group_t -next_linear_group(struct ext4_allocation_context *ac, ext4_group_t group, - ext4_group_t ngroups) +next_linear_group(ext4_group_t group, ext4_group_t ngroups) { - if (!should_optimize_scan(ac)) - goto inc_and_return; - - if (ac->ac_groups_linear_remaining) { - ac->ac_groups_linear_remaining--; - goto inc_and_return; - } - - return group; -inc_and_return: /* * Artificially restricted ngroups for non-extent * files makes group > ngroups possible on first loop. @@ -1118,8 +1106,19 @@ static void ext4_mb_choose_next_group(struct ext4_al= location_context *ac, { *new_cr =3D ac->ac_criteria; =20 - if (!should_optimize_scan(ac) || ac->ac_groups_linear_remaining) { - *group =3D next_linear_group(ac, *group, ngroups); + if (!should_optimize_scan(ac)) { + *group =3D next_linear_group(*group, ngroups); + return; + } + + /* + * Optimized scanning can return non adjacent groups which can cause + * seek overhead for rotational disks. So try few linear groups before + * trying optimized scan. + */ + if (ac->ac_groups_linear_remaining) { + *group =3D next_linear_group(*group, ngroups); + ac->ac_groups_linear_remaining--; return; } =20 --=20 2.30.0