From nobody Sat Feb 7 20:57:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCBCBC77B7A for ; Thu, 25 May 2023 12:54:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239689AbjEYMyT (ORCPT ); Thu, 25 May 2023 08:54:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37130 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231263AbjEYMyS (ORCPT ); Thu, 25 May 2023 08:54:18 -0400 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2EB7B135 for ; Thu, 25 May 2023 05:54:16 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R101e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046050;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSNUk1_1685019251; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSNUk1_1685019251) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:12 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/6] mm: compaction: drop the redundant page validation in update_pageblock_skip() Date: Thu, 25 May 2023 20:53:56 +0800 Message-Id: <5142e15b9295fe8c447dbb39b7907a20177a1413.1685018752.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The caller has validated the page before calling pdate_pageblock_skip(), thus drop the redundant page validation in update_pageblock_skip(). Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 163e2ec70aff..426bb6ce070b 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -436,9 +436,6 @@ static void update_pageblock_skip(struct compact_contro= l *cc, if (cc->no_set_skip_hint) return; =20 - if (!page) - return; - set_pageblock_skip(page); =20 /* Update where async and sync compaction should restart */ --=20 2.27.0 From nobody Sat Feb 7 20:57:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC48CC7EE2F for ; Thu, 25 May 2023 12:54:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234832AbjEYMyV (ORCPT ); Thu, 25 May 2023 08:54:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231895AbjEYMyS (ORCPT ); Thu, 25 May 2023 08:54:18 -0400 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D241187 for ; Thu, 25 May 2023 05:54:16 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R621e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSJ3fp_1685019252; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSJ3fp_1685019252) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:13 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/6] mm: compaction: change fast_isolate_freepages() to void type Date: Thu, 25 May 2023 20:53:57 +0800 Message-Id: <759fca20b22ebf4c81afa30496837b9e0fb2e53b.1685018752.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" No caller cares about the return value of fast_isolate_freepages(), void it. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 426bb6ce070b..3737c6591bfb 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1435,8 +1435,7 @@ static int next_search_order(struct compact_control *= cc, int order) return order; } =20 -static unsigned long -fast_isolate_freepages(struct compact_control *cc) +static void fast_isolate_freepages(struct compact_control *cc) { unsigned int limit =3D max(1U, freelist_scan_limit(cc) >> 1); unsigned int nr_scanned =3D 0; @@ -1449,7 +1448,7 @@ fast_isolate_freepages(struct compact_control *cc) =20 /* Full compaction passes in a negative order */ if (cc->order <=3D 0) - return cc->free_pfn; + return; =20 /* * If starting the scan, use a deeper search and use the highest @@ -1588,11 +1587,10 @@ fast_isolate_freepages(struct compact_control *cc) =20 cc->total_free_scanned +=3D nr_scanned; if (!page) - return cc->free_pfn; + return; =20 low_pfn =3D page_to_pfn(page); fast_isolate_around(cc, low_pfn); - return low_pfn; } =20 /* --=20 2.27.0 From nobody Sat Feb 7 20:57:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD4EAC77B7E for ; Thu, 25 May 2023 12:54:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240874AbjEYMy0 (ORCPT ); Thu, 25 May 2023 08:54:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236065AbjEYMyT (ORCPT ); Thu, 25 May 2023 08:54:19 -0400 Received: from out30-130.freemail.mail.aliyun.com (out30-130.freemail.mail.aliyun.com [115.124.30.130]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B46D183 for ; Thu, 25 May 2023 05:54:18 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R861e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046049;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSJ3gf_1685019253; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSJ3gf_1685019253) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:14 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/6] mm: compaction: skip more fully scanned pageblock Date: Thu, 25 May 2023 20:53:58 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" In fast_isolate_around(), it assumes the pageblock is fully scanned if cc->nr_freepages < cc->nr_migratepages after trying to isolate some free pages, and will set skip flag to avoid scanning in future. However this can miss setting the skip flag for a fully scanned pageblock (returned 'start_pfn' is equal to 'end_pfn') in the case where cc->nr_freepages is larger than cc->nr_migratepages. So using the returned 'start_pfn' from isolate_freepages_block() and 'end_pfn' to decide if a pageblock is fully scanned makes more sense. It can also cover the case where cc->nr_freepages < cc->nr_migratepages, which means the 'start_pfn' is usually equal to 'end_pfn' except some uncommon fatal error occurs after non-strict mode isolation. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/compaction.c b/mm/compaction.c index 3737c6591bfb..1e5183f39ca9 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1411,7 +1411,7 @@ fast_isolate_around(struct compact_control *cc, unsig= ned long pfn) isolate_freepages_block(cc, &start_pfn, end_pfn, &cc->freepages, 1, false= ); =20 /* Skip this pageblock in the future as it's full or nearly full */ - if (cc->nr_freepages < cc->nr_migratepages) + if (start_pfn =3D=3D end_pfn) set_pageblock_skip(page); =20 return; --=20 2.27.0 From nobody Sat Feb 7 20:57:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD75DC77B7E for ; Thu, 25 May 2023 12:54:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241021AbjEYMy2 (ORCPT ); Thu, 25 May 2023 08:54:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239813AbjEYMyU (ORCPT ); Thu, 25 May 2023 08:54:20 -0400 Received: from out30-100.freemail.mail.aliyun.com (out30-100.freemail.mail.aliyun.com [115.124.30.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C0A44183 for ; Thu, 25 May 2023 05:54:19 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045170;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSJ3hf_1685019254; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSJ3hf_1685019254) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:15 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/6] mm: compaction: only set skip flag if cc->no_set_skip_hint is false Date: Thu, 25 May 2023 20:53:59 +0800 Message-Id: <0eb2cd2407ffb259ae6e3071e10f70f2d41d0f3e.1685018752.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" To keep the same logic as test_and_set_skip(), only set the skip flag if cc->no_set_skip_hint is false, which makes code more reasonable. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/compaction.c b/mm/compaction.c index 1e5183f39ca9..65d8d9223acc 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1223,7 +1223,7 @@ isolate_migratepages_block(struct compact_control *cc= , unsigned long low_pfn, * rescanned twice in a row. */ if (low_pfn =3D=3D end_pfn && (!nr_isolated || cc->finish_pageblock)) { - if (valid_page && !skip_updated) + if (!cc->no_set_skip_hint && valid_page && !skip_updated) set_pageblock_skip(valid_page); update_cached_migrate(cc, low_pfn); } --=20 2.27.0 From nobody Sat Feb 7 20:57:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42E09C7EE2D for ; Thu, 25 May 2023 12:54:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241091AbjEYMya (ORCPT ); Thu, 25 May 2023 08:54:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231895AbjEYMyV (ORCPT ); Thu, 25 May 2023 08:54:21 -0400 Received: from out30-124.freemail.mail.aliyun.com (out30-124.freemail.mail.aliyun.com [115.124.30.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97C5D187 for ; Thu, 25 May 2023 05:54:20 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R781e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018045168;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSMoYJ_1685019255; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSMoYJ_1685019255) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:16 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/6] mm: compaction: add trace event for fast freepages isolation Date: Thu, 25 May 2023 20:54:00 +0800 Message-Id: <78d2932d0160d122c15372aceb3f2c45460a17fc.1685018752.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The fast_isolate_freepages() can also isolate freepages, but we can not know the fast isolation efficiency to understand the fast isolation pressur= e. So add a trace event to show some numbers to help to understand the efficie= ncy for fast freepages isolation. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- include/trace/events/compaction.h | 11 +++++++++++ mm/compaction.c | 6 +++++- 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/include/trace/events/compaction.h b/include/trace/events/compa= ction.h index 3313eb83c117..2b2a975efd20 100644 --- a/include/trace/events/compaction.h +++ b/include/trace/events/compaction.h @@ -64,6 +64,17 @@ DEFINE_EVENT(mm_compaction_isolate_template, mm_compacti= on_isolate_freepages, TP_ARGS(start_pfn, end_pfn, nr_scanned, nr_taken) ); =20 +DEFINE_EVENT(mm_compaction_isolate_template, mm_compaction_fast_isolate_fr= eepages, + + TP_PROTO( + unsigned long start_pfn, + unsigned long end_pfn, + unsigned long nr_scanned, + unsigned long nr_taken), + + TP_ARGS(start_pfn, end_pfn, nr_scanned, nr_taken) +); + #ifdef CONFIG_COMPACTION TRACE_EVENT(mm_compaction_migratepages, =20 diff --git a/mm/compaction.c b/mm/compaction.c index 65d8d9223acc..eccec84dae82 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1438,7 +1438,7 @@ static int next_search_order(struct compact_control *= cc, int order) static void fast_isolate_freepages(struct compact_control *cc) { unsigned int limit =3D max(1U, freelist_scan_limit(cc) >> 1); - unsigned int nr_scanned =3D 0; + unsigned int nr_scanned =3D 0, total_isolated =3D 0; unsigned long low_pfn, min_pfn, highest =3D 0; unsigned long nr_isolated =3D 0; unsigned long distance; @@ -1537,6 +1537,7 @@ static void fast_isolate_freepages(struct compact_con= trol *cc) set_page_private(page, order); nr_isolated =3D 1 << order; nr_scanned +=3D nr_isolated - 1; + total_isolated +=3D nr_isolated; cc->nr_freepages +=3D nr_isolated; list_add_tail(&page->lru, &cc->freepages); count_compact_events(COMPACTISOLATED, nr_isolated); @@ -1557,6 +1558,9 @@ static void fast_isolate_freepages(struct compact_con= trol *cc) limit =3D max(1U, limit >> 1); } =20 + trace_mm_compaction_fast_isolate_freepages(min_pfn, cc->free_pfn, + nr_scanned, total_isolated); + if (!page) { cc->fast_search_fail++; if (scan_start) { --=20 2.27.0 From nobody Sat Feb 7 20:57:14 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64090C77B7A for ; Thu, 25 May 2023 12:54:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240626AbjEYMyc (ORCPT ); Thu, 25 May 2023 08:54:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240536AbjEYMyW (ORCPT ); Thu, 25 May 2023 08:54:22 -0400 Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E6FB3189 for ; Thu, 25 May 2023 05:54:20 -0700 (PDT) X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R711e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=baolin.wang@linux.alibaba.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---0VjSJ3iS_1685019256; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VjSJ3iS_1685019256) by smtp.aliyun-inc.com; Thu, 25 May 2023 20:54:17 +0800 From: Baolin Wang To: akpm@linux-foundation.org Cc: mgorman@techsingularity.net, vbabka@suse.cz, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/6] mm: compaction: skip fast freepages isolation if enough freepages are isolated Date: Thu, 25 May 2023 20:54:01 +0800 Message-Id: X-Mailer: git-send-email 2.27.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" I've observed that fast isolation often isolates more pages than cc->migratepages, and the excess freepages will be released back to the buddy system. So skip fast freepages isolation if enough freepages are isolated to save some CPU cycles. Signed-off-by: Baolin Wang Acked-by: Vlastimil Babka --- mm/compaction.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/compaction.c b/mm/compaction.c index eccec84dae82..3ade4c095ed2 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1550,6 +1550,10 @@ static void fast_isolate_freepages(struct compact_co= ntrol *cc) =20 spin_unlock_irqrestore(&cc->zone->lock, flags); =20 + /* Skip fast search if enough freepages isolated */ + if (cc->nr_freepages >=3D cc->nr_migratepages) + break; + /* * Smaller scan on next order so the total scan is related * to freelist_scan_limit. --=20 2.27.0