From nobody Sun Feb 8 08:36:42 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B7F9C7EE26 for ; Tue, 23 May 2023 20:59:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230009AbjEWU7b (ORCPT ); Tue, 23 May 2023 16:59:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229717AbjEWU72 (ORCPT ); Tue, 23 May 2023 16:59:28 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2ED28F for ; Tue, 23 May 2023 13:59:27 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-babb52dbb00so380175276.1 for ; Tue, 23 May 2023 13:59:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684875567; x=1687467567; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=oqzY22DU8hsochaAkiR/zou3nxJkIdpTHuQYEZ4euM0=; b=00x6wRrXLaQc4E3HL5TfDI5wMyAGjsMlESwMz+Ip4SzPcVjGJi/DmFI7yqcmjpIDXU pPCYsg34Ok7Qu3IWjFmttSP2QWAvyZvKR2As9PcQkIbMrWSB0Pqf43q9Fv05dgCWXA90 8L/zBV0+LK7j5OKIqtev3NQqSaoWx7+MDW/HMzhoLZWiADmByxFVuQRAUjGTQJ8yZ72u 07YFpJeOKKk+Y8azY6PMmooahKyJQF48a6TKVwhqgHWmHi5brLqbLRdX7yLsDCfoplvy gMa+0HxIHAn+ddNOkeS8dOuCqzS4SY/rdq3vouSURnGek+KmjC84DuvNK8RgrObbCYXG r2VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684875567; x=1687467567; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=oqzY22DU8hsochaAkiR/zou3nxJkIdpTHuQYEZ4euM0=; b=iK8C8Ksx5060226Hsc9uDOu/qQGaIBe1JUJuOn2yFoVvTWDE3wle/0E27Hr4BgpPpN rep3x4pt4hcq2axKKjxGmI9rcAq9pcc7rJpZythlhC7YB/wnPwgFJC8hle5bupWlivNb jPhecmX5r36G6UQuW+mM4C0vIP9Ovw6mnAj7oVCsXuaeI2F3BONLXCJqOYyNuvNTcZco J6Toa95PzPsN1p0uIuZydeYPg15rEOCqp7mw8mhxGWzkzhx6U30bfCpObfMqvM+1ib1D N0UEgM83vpCNseGO3jpbCswMTi8Jf/n+1alJ+2VUWsV04ynZNqkHATplqOIYvnJ+OS8V Qq6Q== X-Gm-Message-State: AC+VfDxX/9oXr7GYpye9Y6nRnLp+bvLXn9gepI1oleDi79c5uFgpo72N Va1C99MCWaoiMhKbFlylcnXyrcatOVH7pKySwQ== X-Google-Smtp-Source: ACHHUZ6CXRzlSLLUEil2mIC1MEhVd9KbB1k8BRZTxoZezM54UzgFmX5aoe8mWHGRqKL8xAx7PpFCLEoULxy3kxBITA== X-Received: from kalesh.mtv.corp.google.com ([2620:15c:211:201:1c60:f27:f83c:ef25]) (user=kaleshsingh job=sendgmr) by 2002:a25:9786:0:b0:ba8:939b:3e04 with SMTP id i6-20020a259786000000b00ba8939b3e04mr6271389ybo.5.1684875566909; Tue, 23 May 2023 13:59:26 -0700 (PDT) Date: Tue, 23 May 2023 13:59:21 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog Message-ID: <20230523205922.3852731-1-kaleshsingh@google.com> Subject: [PATCH] mm-unstable: Multi-gen LRU: Fix workingset accounting From: Kalesh Singh To: yuzhao@google.com, akpm@linux-foundation.org Cc: surenb@google.com, android-mm@google.com, kernel-team@android.com, Kalesh Singh , Charan Teja Kalla , Oleksandr Natalenko , "Jan Alexander Steffens (heftig)" , Brian Geffon , linux-mm@kvack.org, linux-kernel@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" On Android app cycle workloads, MGLRU showed a significant reduction in workingset refaults although pgpgin/pswpin remained relatively unchanged. This indicated MGLRU may be undercounting workingset refaults. This has impact on userspace programs, like Android's LMKD, that monitor workingset refault statistics to detect thrashing. It was found that refaults were only accounted if the MGLRU shadow entry was for a recently evicted folio. However, recently evicted folios should be accounted as workingset activation, and refaults should be accounted regardless of recency. Fix MGLRU's workingset refault and activation accounting to more closely match that of the conventional active/inactive LRU. Fixes: ac35a4902374 ("mm: multi-gen LRU: minimal implementation") Cc: Yu Zhao Cc: Andrew Morton =20 Reported-by: Charan Teja Kalla Signed-off-by: Kalesh Singh Acked-by: Yu Zhao --- mm/vmscan.c | 1 - mm/workingset.c | 10 +++++++--- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index eeca83e28c9b..ef687f9be13c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4925,7 +4925,6 @@ static bool sort_folio(struct lruvec *lruvec, struct = folio *folio, int tier_idx) =20 WRITE_ONCE(lrugen->protected[hist][type][tier - 1], lrugen->protected[hist][type][tier - 1] + delta); - __mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta); return true; } =20 diff --git a/mm/workingset.c b/mm/workingset.c index 5796e927e6d7..4686ae363000 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -278,6 +278,7 @@ static bool lru_gen_test_recent(void *shadow, bool file= , struct lruvec **lruvec, =20 static void lru_gen_refault(struct folio *folio, void *shadow) { + bool recent; int hist, tier, refs; bool workingset; unsigned long token; @@ -288,10 +289,13 @@ static void lru_gen_refault(struct folio *folio, void= *shadow) =20 rcu_read_lock(); =20 - if (!lru_gen_test_recent(shadow, type, &lruvec, &token, &workingset)) + recent =3D lru_gen_test_recent(shadow, type, &lruvec, &token, &workingset= ); + if (lruvec !=3D folio_lruvec(folio)) goto unlock; =20 - if (lruvec !=3D folio_lruvec(folio)) + mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + type, delta); + + if (!recent) goto unlock; =20 lrugen =3D &lruvec->lrugen; @@ -302,7 +306,7 @@ static void lru_gen_refault(struct folio *folio, void *= shadow) tier =3D lru_tier_from_refs(refs); =20 atomic_long_add(delta, &lrugen->refaulted[hist][type][tier]); - mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + type, delta); + mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta); =20 /* * Count the following two cases as stalls: base-commit: cd255dae35f51ea33cb85d29720d6b4e5fa16365 --=20 2.40.1.698.g37aff9b760-goog