From nobody Fri Oct 2 13:03:27 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABFFE3D9027 for ; Fri, 31 Jul 2026 08:39:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487146; cv=none; b=hKO8hZzVbPZxqaakTrOnsDsi9yMu4LJt8J8n3iITmESyE0WQUGzM4wLHSNDrQ7+oWF30MOC7FETNKNgTz/cNMfXi1BrxAq/GUml6AXEejgDTQMpdDosF99wBRVn+OJH4SHU3hfuob0C8dMjqhMFf+v3FviLJCfpZSv+hMF0zjRI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487146; c=relaxed/simple; bh=gn0RARJRHyZuicJRTWIyBy/C/GTI+d5HNDMDlQw2Mbs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BIvo98nj1/C98ZbPUaG+EC3k/q7X1jmlPPUn2KEHsZmS1NKV1ZD/FZVuFYQnI+rVRQnzu1SSkqFWCsHbLLUKxMGM3YVlZQUitIyzDhTHZ0bu/DTlGlBNS6DzOmRZrlePSh1IWQdrymxddKzUsuAb4JMqra3ppBb5CvuKFKkfrMg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AGKCawgR; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AGKCawgR" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3A621F00A3A; Fri, 31 Jul 2026 08:39:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785487145; bh=l1RvYFIQMTPrfL3d/VmiupnZ2Z7loXdewoH+NnJ+qiA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=AGKCawgRDVa9JVQhwQohtBa09MGFFkDREph4vYnF3B+JfGvStvZ0PIkk+2L+zYZI+ L9ZBTUCEKx9UEHkhSZrBXrg1Db0FY90P/1ohEhd/s50Q78YGgNQorq6K8oNKHZtPUI E39RZeQAsP7nJ2WzIE63IzQDh6x/us8qAqKrraimS3yw84mTnyUR8sbAmhlx9XP92C WHLpAAHxPod6IEYSXCYRSQu5WdcyqMGiEcwVtAzEtwLwZayVlaogdnYfuRkDdhyd/c Lz38uMsMMSUUKZt4BesUUrdHvksZEKHjzo5pTCdjwmoSVKBvT0QsNHqpcVFI74pyRD pkm+VKIALvOZw== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, weixugc@google.com, yuanchu@google.com, chenridong@xiaomi.com, zhangbo56@xiaomi.com, wangzicheng@honor.com, lianux.mm@gmail.com, "Barry Song (Xiaomi)" Subject: [RFC PATCH v3 1/6] mm: mglru: prevent min_seq[type] from pointing to an empty generation Date: Fri, 31 Jul 2026 16:38:38 +0800 Message-Id: <20260731083843.37811-2-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260731083843.37811-1-baohua@kernel.org> References: <20260731083843.37811-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In try_to_inc_min_seq(), min_seq[LRU_GEN_ANON] and min_seq[LRU_GEN_FILE] can be adjusted to point to empty generations to keep their gap within one. As a result, scan_folios() may repeatedly scan empty generations because it assumes the min_seq generation still contains folios. Likewise, should_run_aging() has no way to tell that the min_seq generation has already been drained. This is quite confusing. I observed scan_folios() returning 0 even though the following check in scan_folios() doesn't take effect: if (get_nr_gens(lruvec, type) =3D=3D MIN_NR_GENS) return 0; There is no need to adjust min_seq[], since the reclaim logic already triggers aging when the number of generations reaches MIN_NR_GENS, and reclaim never reduces it below MIN_NR_GENS. Signed-off-by: Barry Song (Xiaomi) --- include/linux/mmzone.h | 6 ++---- mm/vmscan.c | 10 ---------- 2 files changed, 2 insertions(+), 14 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index a26c8b855222..233d2006a541 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -552,10 +552,8 @@ enum { * The youngest generation number is stored in max_seq for both anon and f= ile * types as they are aged on an equal footing. The oldest generation numbe= rs are * stored in min_seq[] separately for anon and file types so that they can= be - * incremented independently. Ideally min_seq[] are kept in sync when both= anon - * and file types are evictable. However, to adapt to situations like extr= eme - * swappiness, they are allowed to be out of sync by at most - * MAX_NR_GENS-MIN_NR_GENS-1. + * incremented independently. For both file and anonymous memory, the mini= mum + * generation must be at least MIN_NR_GENS. * * The number of pages in each generation is eventually consistent and the= refore * can be transiently negative when reset_batch_size() is pending. diff --git a/mm/vmscan.c b/mm/vmscan.c index 566c4e837c7d..ce027c271e9b 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3972,16 +3972,6 @@ static void try_to_inc_min_seq(struct lruvec *lruvec= , int swappiness) if (!seq_inc_flag) return; =20 - /* see the comment on lru_gen_folio */ - if (swappiness && swappiness <=3D MAX_SWAPPINESS) { - unsigned long seq =3D lrugen->max_seq - MIN_NR_GENS; - - if (min_seq[LRU_GEN_ANON] > seq && min_seq[LRU_GEN_FILE] < seq) - min_seq[LRU_GEN_ANON] =3D seq; - else if (min_seq[LRU_GEN_FILE] > seq && min_seq[LRU_GEN_ANON] < seq) - min_seq[LRU_GEN_FILE] =3D seq; - } - for_each_evictable_type(type, swappiness) { if (min_seq[type] <=3D lrugen->min_seq[type]) continue; --=20 2.34.1 From nobody Fri Oct 2 13:03:27 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 791743D953A for ; Fri, 31 Jul 2026 08:39:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487150; cv=none; b=mB19hGdkkuEbXpSdhW5lBKoXvtFGCh0RHFvxOSlGAjlTnmmrGA+Sj7Hjy0MWMITyTOS64Y9W03LsSgt9eofGpcSekoVxQvZI59DLtF1Dlx4veQhfCMrFQARgCkbxCGKRjgz1a2MuMtSfDEWSZk4qZMeqElU5PmQr/KzoaAzljbo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487150; c=relaxed/simple; bh=wTTSHv8Qeo6B1VR3hq5t1c9kMRMIFdcbHs+/cJaaJW4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Zu0I7V4YPQgt5oWio+0hCSCrtAYkhxkjkU4KkCq6KgNJZFjtxiDX8KEE17RjD8H7hlc4ISstz3+trdadrq0jWh8fTpAeE0cnewnn/uRSRfF3RNdqKT923cKZCOLqWlUBqzKtnbcZg2P7YV967wU3eecMQvuNG71FBG4gq7IFxv8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Vesdli7t; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Vesdli7t" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A763C1F000E9; Fri, 31 Jul 2026 08:39:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785487149; bh=eGS47ujxTvtlq11ZCiBl4aLoPuUh0trV8PXbmmGJxG0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=Vesdli7txUOZi8YTM10BgeOKsRiQZoXphJj+BNdln3V6vmaSIL625HyEuf7jV+/X9 FQWWXB2la+2QgphgyByXKbHDJLR9Fp9z5oB4Jdu2/xNmFGtFXsP4VKALXo5AEsV/U+ Ww7jaHzQg9vj7Oz4VOFtwm9/kzfETkxnuqZsKaHSlknJsD/K+u2ULirYN2DW3ew1bs YRgcbdMpugr3k/TH05t/1fZcL3ekneYjW/DoqV225cjyRatdqafO5hKl4jNNq8lwow TSb6PQzHJf6qC2Vj533l7IRPkffo02gyh2DsRIKp+lLEcqPMMSWJs76Le1LXkLXbn/ YY1dgcyZxN1dw== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, weixugc@google.com, yuanchu@google.com, chenridong@xiaomi.com, zhangbo56@xiaomi.com, wangzicheng@honor.com, lianux.mm@gmail.com, "Barry Song (Xiaomi)" Subject: [RFC PATCH v3 2/6] mm: mglru: let scan_folios() scan both reclaimable generations Date: Fri, 31 Jul 2026 16:38:39 +0800 Message-Id: <20260731083843.37811-3-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260731083843.37811-1-baohua@kernel.org> References: <20260731083843.37811-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When we have four generations, and scan_folios() exhausts the oldest one while the second-oldest still contains reclaimable folios, the current implementation doesn't move on to scan the second-oldest generation. Instead, it returns, leaving that generation with no chance to be scanned at the current sc->priority. This doesn't seem right. Rather than breaking out and starting a larger next iteration, let's let scan_folios() continue scanning the second-oldest generation directly. Signed-off-by: Barry Song (Xiaomi) --- mm/vmscan.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index ce027c271e9b..31947fa60f18 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4718,6 +4718,7 @@ static int scan_folios(unsigned long nr_to_scan, stru= ct lruvec *lruvec, int skipped =3D 0; unsigned long remaining =3D nr_to_scan; struct lru_gen_folio *lrugen =3D &lruvec->lrugen; + unsigned long min_seq =3D lrugen->min_seq[type]; =20 VM_WARN_ON_ONCE(nr_to_scan > MAX_LRU_BATCH); VM_WARN_ON_ONCE(!list_empty(list)); @@ -4725,8 +4726,8 @@ static int scan_folios(unsigned long nr_to_scan, stru= ct lruvec *lruvec, if (get_nr_gens(lruvec, type) =3D=3D MIN_NR_GENS) return 0; =20 - gen =3D lru_gen_from_seq(lrugen->min_seq[type]); - +next_gen: + gen =3D lru_gen_from_seq(min_seq); for (i =3D MAX_NR_ZONES; i > 0; i--) { LIST_HEAD(moved); int skipped_zone =3D 0; @@ -4768,6 +4769,14 @@ static int scan_folios(unsigned long nr_to_scan, str= uct lruvec *lruvec, break; } =20 + /* + * This generation is exhausted across all zones, but the scan + * target has not been reached yet. Continue with the next + * reclaimable generation. + */ + if (i =3D=3D 0 && ++min_seq + MIN_NR_GENS <=3D lrugen->max_seq) + goto next_gen; + item =3D PGSCAN_KSWAPD + reclaimer_offset(sc); mod_lruvec_state(lruvec, item, isolated); mod_lruvec_state(lruvec, PGREFILL, sorted); --=20 2.34.1 From nobody Fri Oct 2 13:03:27 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F31423D9DBF for ; Fri, 31 Jul 2026 08:39:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487154; cv=none; b=OhlZj2V7R5nxs/wS0adHiO+KP6VyMU79GSqKYzywpQTzbTPGtY+iWLneW3KB+GFcpKc8Csf1ryI2Hcikq4OSl1rdvuwSiXOEq1/jmxWs/ZVA8ohVVMz1Qba6VDMrjJnhWyyqhk1wZArl2LV28Ve1wqkioNuqCE2FXvPCu1Au3Qw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487154; c=relaxed/simple; bh=qJM6K3gfhp9lqgQ1bjrKznW2mi5DScFFjm/436rkddk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=u+MyQtyzMPxZoxJFBgF9RaY5XxLAVHAudjQJcz7ZW7FMJAUgYz7yAD/pr0801y6/rc+9UVq0nVgUM+t5O+nCu6PCeq/tcLepnXxPo3e+BWpR60Qp5OL10OJi2o5ush9LDQslFTn5WVnQEaxFZUbtC9jW2c18t5oT9JdOA6yqdak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=MRUp7+N8; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="MRUp7+N8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 778D31F00A3A; Fri, 31 Jul 2026 08:39:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785487152; bh=YV+vGlkWwAAqTC1IWyggngTy7GK63iEPW6UUQSpPO7o=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=MRUp7+N8XwNJdsu5G0MjXW8uZSR7JgKcxsOQe35smP5Ce/TgkBCUhrkLBy5roxTwX wsNkeHlivzLYQ8qDRhUw8i5Ah+2z2t8t+3fF2yPrsBWMhPosqSaMQXlIZHNrjHltjD HNp9iOUMD9bPFzOx5ApnpLUBmkHv3dwvZlEI43vunfEqC+mlnIfF6Gij6+G8Sh+/iD 2hhEPoC2hKyKC2XbPDfwtErhBYGsFGyhYoe/MNMPYl0tUto6xJV77kQJWB+C/uNAAv 6vOmIvJs6DAdvL8rp9qpdP8/V6fYwsjx5HVqQWkZpydRCjGk8fcQsma3njfqqS1mm3 GJVPgIVmCADXw== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, weixugc@google.com, yuanchu@google.com, chenridong@xiaomi.com, zhangbo56@xiaomi.com, wangzicheng@honor.com, lianux.mm@gmail.com, Barry Song Subject: [RFC PATCH v3 3/6] mm/mglru: improve readability of isolate_folios() Date: Fri, 31 Jul 2026 16:38:40 +0800 Message-Id: <20260731083843.37811-4-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260731083843.37811-1-baohua@kernel.org> References: <20260731083843.37811-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Ridong Chen The for_each_evictable_type() loop in isolate_folios() is misleading: it does not actually iterate over each evictable type. Instead, get_type_to_scan() selects the type to scan, while the iterator `i` merely bounds the number of attempts. Signed-off-by: Ridong Chen Co-developed-by: Barry Song (Xiaomi) Signed-off-by: Barry Song (Xiaomi) --- mm/vmscan.c | 47 +++++++++++++++++++++++++++-------------------- 1 file changed, 27 insertions(+), 20 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 31947fa60f18..0038f33aa318 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4828,35 +4828,42 @@ static int get_type_to_scan(struct lruvec *lruvec, = int swappiness) return positive_ctrl_err(&sp, &pv); } =20 +static inline bool is_single_type_reclaim(int swappiness) +{ + return swappiness =3D=3D MIN_SWAPPINESS || + swappiness =3D=3D SWAPPINESS_ANON_ONLY; +} + static int isolate_folios(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, int swappiness, struct list_head *list, int *isolated, int *isolate_type, int *isolate_scanned) { - int i; - int total_scanned =3D 0; + bool single_type =3D is_single_type_reclaim(swappiness); int type =3D get_type_to_scan(lruvec, swappiness); + int total_scanned =3D 0, scanned, tier; + bool tried =3D false; =20 - for_each_evictable_type(i, swappiness) { - int scanned; - int tier =3D get_tier_idx(lruvec, type); +retry: + tier =3D get_tier_idx(lruvec, type); + scanned =3D scan_folios(nr_to_scan, lruvec, sc, + type, tier, list, isolated); =20 - scanned =3D scan_folios(nr_to_scan, lruvec, sc, - type, tier, list, isolated); + total_scanned +=3D scanned; + if (*isolated) { + *isolate_type =3D type; + *isolate_scanned =3D scanned; + return total_scanned; + } =20 - total_scanned +=3D scanned; - if (*isolated) { - *isolate_type =3D type; - *isolate_scanned =3D scanned; - break; - } - /* - * If scanned > 0 and isolated =3D=3D 0, avoid falling back to the - * other type, as this type remains sufficient. Falling back - * too readily can disrupt the positive_ctrl_err() bias. - */ - if (!scanned) - type =3D !type; + /* + * We are running out of the current reclaim type. Fall back to + * the other type if allowed. + */ + if (!tried && !scanned && !single_type) { + type =3D !type; + tried =3D true; + goto retry; } =20 return total_scanned; --=20 2.34.1 From nobody Fri Oct 2 13:03:27 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C292F3D953A for ; Fri, 31 Jul 2026 08:39:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487157; cv=none; b=HLFlplZ7uPoLf43BHSeWUJpViYw2HoKH1nNZMZk6yMZje9S+IM7GOMUuoo4nTlyRV9PQUJciBRk3xsN+Pug0TI/lRDAm8pohfwnGtjTdbA8ZAMdJi+NlUGK2DoHSw1uyE9XOsqtJHJGZ93NqeDhe1qWzlxw7sf0MwREicI6fuQ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487157; c=relaxed/simple; bh=cPE/fBdUE/ox4jo0Z4xuOssFz/5s5fgX8oI+mSYk5Fw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UeBl8AGJaIQ8Y0eBoptRm4OUbV9W1n6eB1JCM9Oeo7QUCGBEyf0iRfQeW/N1+9t12iv7MrzkGN1RDtfB4TR8PuqDmCv/rMD4xUxh54zpOtTmoCD1bZJKGciGsNqz+8hCsxl+JQZULTxv6dgl7kmTAxC+2IFuIdQAxdulFtIUkDk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oB5R0jJl; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oB5R0jJl" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 491851F00ACA; Fri, 31 Jul 2026 08:39:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785487156; bh=AvSIyHGbFQ1xRyWKcwWNl/S/3zQ4GNtnzqsqU1kijnI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=oB5R0jJl9rpHQNnSXw+f7IDGKtQYOmfWfdnJgVy38zEsNN8R5zf0M71p8oteLlYZD oMGUm2b87+pPjeAkFYkPI/iRVCG39Kp/tGMwtnxEfBDR0wQ5xyP1tCfuKD6PuASCRc LklG8eCQrx/rPxMfVcHtcUTG2k3fM5uGIKuHE6dWrt0dMMySoVK/SdhpvgTV7xM03M EUJMERFNO4b7wctVfbVTZJEopBnlQIckPKI+yzMdYTczrhplEKDmbnQgVE1D3w+SiR 0lL2r4ntXf4c9xn4ORaVdh67DAhwOSpV5/02RmB1ANDVSPqdlQpiJpxsqhHCyfLiH4 6qEF87Nx8PhgA== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, weixugc@google.com, yuanchu@google.com, chenridong@xiaomi.com, zhangbo56@xiaomi.com, wangzicheng@honor.com, lianux.mm@gmail.com, "Barry Song (Xiaomi)" Subject: [RFC PATCH v3 4/6] mm: mglru: improve scan_folios() exhaustion detection Date: Fri, 31 Jul 2026 16:38:41 +0800 Message-Id: <20260731083843.37811-5-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260731083843.37811-1-baohua@kernel.org> References: <20260731083843.37811-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Commit 16b475d2ac3c ("mm/mglru: avoid reclaim type fall back when isolation makes no progress") uses scanned =3D=3D 0 to determine whether scan_folios() has exhausted a reclaim type. However, this is not always sufficient. It is possible for scanned > 0, while the oldest reclaimable generation is exhausted after the first scan_folios() call. Add an exhausted output argument to scan_folios() so it can explicitly report whether the reclaimable lists for the current type have been exhausted. Signed-off-by: Barry Song (Xiaomi) --- mm/vmscan.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 0038f33aa318..2e7fef6975b6 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4707,7 +4707,8 @@ static bool isolate_folio(struct lruvec *lruvec, stru= ct folio *folio, struct sca =20 static int scan_folios(unsigned long nr_to_scan, struct lruvec *lruvec, struct scan_control *sc, int type, int tier, - struct list_head *list, int *isolatedp) + struct list_head *list, int *isolatedp, + bool *exhausted) { int i; int gen; @@ -4723,8 +4724,10 @@ static int scan_folios(unsigned long nr_to_scan, str= uct lruvec *lruvec, VM_WARN_ON_ONCE(nr_to_scan > MAX_LRU_BATCH); VM_WARN_ON_ONCE(!list_empty(list)); =20 - if (get_nr_gens(lruvec, type) =3D=3D MIN_NR_GENS) + if (get_nr_gens(lruvec, type) =3D=3D MIN_NR_GENS) { + *exhausted =3D true; return 0; + } =20 next_gen: gen =3D lru_gen_from_seq(min_seq); @@ -4785,6 +4788,11 @@ static int scan_folios(unsigned long nr_to_scan, str= uct lruvec *lruvec, scanned, skipped, isolated, type ? LRU_INACTIVE_FILE : LRU_INACTIVE_ANON); =20 + /* + * This scan exhausted the reclaimable lists before reaching the + * scan target or accumulating enough isolated folios. + */ + *exhausted =3D remaining > 0 && isolated < MIN_LRU_BATCH; *isolatedp =3D isolated; return scanned; } @@ -4842,12 +4850,12 @@ static int isolate_folios(unsigned long nr_to_scan,= struct lruvec *lruvec, bool single_type =3D is_single_type_reclaim(swappiness); int type =3D get_type_to_scan(lruvec, swappiness); int total_scanned =3D 0, scanned, tier; - bool tried =3D false; + bool tried =3D false, exhausted; =20 retry: tier =3D get_tier_idx(lruvec, type); scanned =3D scan_folios(nr_to_scan, lruvec, sc, - type, tier, list, isolated); + type, tier, list, isolated, &exhausted); =20 total_scanned +=3D scanned; if (*isolated) { @@ -4860,7 +4868,7 @@ static int isolate_folios(unsigned long nr_to_scan, s= truct lruvec *lruvec, * We are running out of the current reclaim type. Fall back to * the other type if allowed. */ - if (!tried && !scanned && !single_type) { + if (!tried && exhausted && !single_type) { type =3D !type; tried =3D true; goto retry; --=20 2.34.1 From nobody Fri Oct 2 13:03:27 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8E693D7D6C for ; Fri, 31 Jul 2026 08:39:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487162; cv=none; b=CQL6Wh5vMyNs6HdMv3aAc56IDSimF/Go6GyqzsrbGXTa8O/58yLkaW8n46SqzAfMzWgbCd6jNCwZAX1TBI/rQGfzcjtA0XKtz3tTf9qald35MvCinf/lmkdGaHWRqkKoKx7hLKW9mwddXaFdsJEmFacWyHmIyociRUrPmoBVCxw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487162; c=relaxed/simple; bh=lZ94UIn6/gawFF/AaRI0gt80BUeBtN2i3lSBeKDIi+g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=W/BlAL1yjgOPMV/T+QyqpUH0X1dWEtha7bR+/ZnTZv1GeyladojAkVagi0Nup7Thb1hbxV8Khq85x9AOjxaiWBjhF/eAO21Qp9QcpsXY/utLI1n3Hnm9q/D0U0NJK3fT6hxFqe1uamhD5jEksGyHCapnJMpf8Y8SfHw3EMNCOMM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BqpESvbN; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BqpESvbN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 17D4F1F00A3D; Fri, 31 Jul 2026 08:39:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785487160; bh=ihmxc/uju4l+NYhonEwKFariUb9t7YQeSVnFbp6tBNE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=BqpESvbNDFygmGxUmbtGwn2yI5In9LieGYe0mz4gyWc2pGlqVEWaKwrz5Gs1RxeHM t31hQRkpRcT+6o28KD7Mdoy4X+yKaRjtBY5HgINATIjTQ43ZfmTShoXd7cQzvyunOj u0DxEQX04D0c1Vhfqdhx/MFqPDWSuqvjHrSn+udwR68kPtHDXlFXZ9ieTMy26marnc HlySCWb7HtS3B9LPqpE2+3MyFwZV8ljPVo/e3Jz9DVhf0ELc7VFdVj89OsgVhS24Cl YyQEUWylSNGKWnPD5/an3hDnqTzRXmRmV+tnriR4ObTxIZ2X/2tq+yweasg8BajPn/ XeDbpfhkNDcdQ== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, weixugc@google.com, yuanchu@google.com, chenridong@xiaomi.com, zhangbo56@xiaomi.com, wangzicheng@honor.com, lianux.mm@gmail.com, Barry Song Subject: [RFC PATCH v3 5/6] mm: mglru: run aging when pages are severely imbalanced across gens Date: Fri, 31 Jul 2026 16:38:42 +0800 Message-Id: <20260731083843.37811-6-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260731083843.37811-1-baohua@kernel.org> References: <20260731083843.37811-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: lyugaofei This partially restores the reclaim behavior introduced in Yu Zhao's initial MGLRU commit, ac35a4902370 ("mm: multi-gen LRU: minimal implementation"): /* * It's also ideal to spread pages out evenly, i.e., 1/(MIN_NR_GENS+1) * of the total number of pages for each generation. A reasonable range * for this average portion is [1/MIN_NR_GENS, 1/(MIN_NR_GENS+2)]. The * aging cares about the upper bound of hot pages, while the eviction * cares about the lower bound of cold pages. */ if (young * MIN_NR_GENS > total) return true; if (old * (MIN_NR_GENS + 2) < total) return true; But with a stricter condition: the younger generations must contain at least old_ratio times as many folios as the older generations. The old_ratio is derived from the lruvec size, so larger lruvecs use a higher old_ratio. This allows aging to keep folios of the preferred type distributed across the reclaimable generations. Also, imbalanced aging is only applied to non-balanced swappiness values (for example, < 60 or > 140). For balanced swappiness values near the middle, the swappiness bias is less of a concern, while the slightly increased aging overhead might be unacceptable. Signed-off-by: lyugaofei Co-developed-by: Barry Song (Xiaomi) Signed-off-by: Barry Song (Xiaomi) --- mm/vmscan.c | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 58 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 2e7fef6975b6..98fd2fac90ac 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4973,9 +4973,62 @@ static int evict_folios(unsigned long nr_to_scan, st= ruct lruvec *lruvec, return scanned; } =20 +static inline bool swappiness_is_balanced(int swappiness) +{ + int range =3D MAX_SWAPPINESS - MIN_SWAPPINESS; + int middle =3D MIN_SWAPPINESS + range / 2; + + return swappiness >=3D middle - range / 5 && + swappiness <=3D middle + range / 5; +} + +static bool lru_gen_imbalanced(struct lruvec *lruvec, int type, + unsigned long max_seq, unsigned long min_seq, + struct scan_control *sc, int swappiness) +{ + struct lru_gen_folio *lrugen =3D &lruvec->lrugen; + unsigned long young =3D 0, old =3D 0, seq; + unsigned long old_ratio, gb; + + /* Skip swappiness bias for single-type reclaim or balanced swappiness */ + if (is_single_type_reclaim(swappiness) || swappiness_is_balanced(swappine= ss)) + return false; + + /* More than two generations remain to reclaim */ + if (min_seq + MIN_NR_GENS < max_seq) + return false; + + /* + * Run aging if the only remaining reclaimable generation + * has few folios. + */ + for (seq =3D min_seq; seq <=3D max_seq; seq++) { + int gen =3D lru_gen_from_seq(seq); + unsigned long size =3D 0; + int zone; + + for (zone =3D 0; zone < MAX_NR_ZONES; zone++) + size +=3D max(READ_ONCE(lrugen->nr_pages[gen][type][zone]), 0L); + + if (seq + MIN_NR_GENS > max_seq) + young +=3D size; + else + old +=3D size; + } + /* + * Copied from inactive_is_low(), but uses a higher old_ratio to + * make aging less aggressive. + */ + gb =3D (young + old) >> (30 - PAGE_SHIFT); + old_ratio =3D gb ? int_sqrt(10 * gb) : 1; + old_ratio *=3D MAX_NR_GENS; + return young > old * old_ratio; +} + static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq, struct scan_control *sc, int swappiness) { + int type =3D get_type_to_scan(lruvec, swappiness); DEFINE_MIN_SEQ(lruvec); =20 /* have to run aging, since eviction is not possible anymore */ @@ -4987,7 +5040,11 @@ static bool should_run_aging(struct lruvec *lruvec, = unsigned long max_seq, return false; =20 /* better to run aging even though eviction is still possible */ - return evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS =3D=3D max_se= q; + if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS =3D=3D max_seq) + return true; + + /* Run aging if the preferred type is severely imbalanced across gens */ + return lru_gen_imbalanced(lruvec, type, max_seq, min_seq[type], sc, swapp= iness); } =20 static long get_nr_to_scan(struct lruvec *lruvec, struct scan_control *sc, --=20 2.34.1 From nobody Fri Oct 2 13:03:27 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F8763DDB03 for ; Fri, 31 Jul 2026 08:39:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487165; cv=none; b=oQ9gEgm9nr7T9wq8pSVvScJWpFTY2LY7QEHvPYYwS7pYXEBFQE8foc6bg0PUFHsEmAcnGcjEAIuKRwaNl5OCkO5X+wKUWfvBp6xVvVImtYAiwo3T0T5DZSPGoLhcjYDmY6GZlnjt/s5CNYn03qI2bCQKzrDO2XFJ9OT6mjBLRTQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1785487165; c=relaxed/simple; bh=NulH51rAVc/tFPbSMgpYSVV7eFFI3S0tmZ6J3neDFPg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kApIez5bbNyDUapxppYpmu4Sv9Qk4cfNF4e1I51CMxCbSfXTEdm5lE4dqEsONzG25zLv9WCt51K0NNp9n8rvECr7pmtJTzcgkjclWr3xMegJd41eyqOddCgbvVE/9LwQdgthS0Pyzd6IUgL95IQY4xca7gyTsh9oWmtqbNH6TuY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UxGQuFi+; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UxGQuFi+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DBC061F000E9; Fri, 31 Jul 2026 08:39:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1785487164; bh=As8NSXV13y2cGJLUhiwyYLHOUPaX+4YofgJJNeeFOg4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=UxGQuFi+56gUrbDHVKPXs43VOxdya1Luqh1HeFxSpNC80syWtNsjRod7vsh0Wo2WX HUv1IU7w3c4TuyxRhs9tvfOXiMAJWSEoNn+PbFJcnRfH9TO56WvA/9w/Ig3mKrmpjg O13c6Vo82grMUU9qJkIX81Wi6O9zRStciEOObMIE5Kkyrp20NaI0HRJoMSgx36aEXa D3vUuTOQleM1h5p1/Qw+gO7ZOk9W2agKzkzPYqScmpFj2u0jQTkOKUdicqf+qHrjXZ mpM1NJlXFJWX8T3EJKGAs6kgovzm34R7Y8dmAiWAlY0km6mnaPtal8w/1p6e3kbHF+ DFoISSHaeyFmQ== From: "Barry Song (Xiaomi)" To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: axelrasmussen@google.com, david@kernel.org, hannes@cmpxchg.org, kasong@tencent.com, linux-kernel@vger.kernel.org, ljs@kernel.org, lyugaofei@xiaomi.com, mhocko@kernel.org, qi.zheng@linux.dev, shakeel.butt@linux.dev, stevensd@chromium.org, weixugc@google.com, yuanchu@google.com, chenridong@xiaomi.com, zhangbo56@xiaomi.com, wangzicheng@honor.com, lianux.mm@gmail.com, "Barry Song (Xiaomi)" Subject: [RFC PATCH v3 6/6] mm: mglru: run aging if the preferred type has no folios in reclaimable gens Date: Fri, 31 Jul 2026 16:38:43 +0800 Message-Id: <20260731083843.37811-7-baohua@kernel.org> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20260731083843.37811-1-baohua@kernel.org> References: <20260731083843.37811-1-baohua@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Respect the type selected by positive_ctrl_err(). If there are no reclaimable generations left for that type, run aging. For balanced swappiness values near the middle, still allow gentle reclaim since the swappiness bias is less of a concern, while the slightly increased aging overhead might become a problem. Signed-off-by: Barry Song (Xiaomi) --- mm/vmscan.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index 98fd2fac90ac..902d85afc5f6 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -5035,6 +5035,10 @@ static bool should_run_aging(struct lruvec *lruvec, = unsigned long max_seq, if (evictable_min_seq(min_seq, swappiness) + MIN_NR_GENS > max_seq) return true; =20 + /* run aging if the preferred type is exhausted for extreme swappiness */ + if (min_seq[type] + MIN_NR_GENS > max_seq && !swappiness_is_balanced(swap= piness)) + return true; + /* try to avoid aging, do gentle reclaim at the default priority */ if (sc->priority =3D=3D DEF_PRIORITY) return false; --=20 2.34.1