From nobody Sat Apr 18 10:53:40 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F135BC43334 for ; Thu, 14 Jul 2022 06:49:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234463AbiGNGt0 (ORCPT ); Thu, 14 Jul 2022 02:49:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232322AbiGNGtY (ORCPT ); Thu, 14 Jul 2022 02:49:24 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F2C223BF7 for ; Wed, 13 Jul 2022 23:49:23 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-31c9d560435so7737197b3.21 for ; Wed, 13 Jul 2022 23:49:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=OWO7azjUaMs5aX9KKa+l/Q7z/45AbzWJxqiXiv/5VIU=; b=haGNeiKz2kRNng6AUosqHR6JF78x1FLSdCVHlrlH10qcMkjhiIPZF9S02ryBUY7FXP OTVSF3ccuWZJQS8Rxj/TdSCex2uRmB7BwCCZjAz39OAJ9VW4YkMjnPhyubNmaKqh2HbN pgp65382e8ExXlsqcajmFZBt8S7u5uF4QgFIA97FHGu4g9jVxZx0RavJCpARNopbcKHM sOrIGrmktyTV2eCBOyIhOcYreiwz4dUW22a1q/8kBSNMbNF4AyiNqXQYLcfh6Ujt1pvt CjB5UNYRhx/eG96BkxJ6MsUCrLcD1UHunMNawVGeShicrq9n5m/KPg95Z3yXvXcn+yGQ Nsdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=OWO7azjUaMs5aX9KKa+l/Q7z/45AbzWJxqiXiv/5VIU=; b=d61kolYUQq7XXM/hWYB1VMEUg6F9EyA0/ZNIz+r603u0yEMo34oit2vAj5JkO1FMyv mIhNURDtGrJVNrXthbAt01l0xGOXj1fyW6nNwQZIRFRkoKcICWimfsizXdgsRratq6/e lPfN5myjN/s6aSyn3TuewqDhpAPdEvA3GQAV4xMOkesWvJ7f0DD5fFrk8A5nomQrfo7Y jQLit7ljP03PUyq2dd5I9StgljH9pCtK1rybl6LoGdShiGOZDRSKZIWuv4U2kmC5Tcnb PrnjXBdk3kctCy3O6cHRXHKIfhA+zVhxZQIrbES/RwZNL6S2X91fuwHgUuClXEjQ0XH0 ug1A== X-Gm-Message-State: AJIora/rZ5K8cww98f/gMBq0f6kZwFxlc2lBB3fyMg3r92+QFbUvFBHD ORLxkLh/fDqVaxWP8JDPPZWu6YtBtIPRK/Q4 X-Google-Smtp-Source: AGRyM1u6B/wHfl+IQRGW5yeE9E+fRV09NHiNSnERTgi/Lyj3e6bPqXEu9iK84kwS0wwLfm65Xnvc0vlUeR+yFgUB X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a0d:e882:0:b0:31d:5aa3:2cbd with SMTP id r124-20020a0de882000000b0031d5aa32cbdmr8449855ywe.52.1657781362316; Wed, 13 Jul 2022 23:49:22 -0700 (PDT) Date: Thu, 14 Jul 2022 06:49:18 +0000 Message-Id: <20220714064918.2576464-1-yosryahmed@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.37.0.144.g8ac04bfd2-goog Subject: [PATCH v4] mm: vmpressure: don't count proactive reclaim in vmpressure From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: Matthew Wilcox , Vlastimil Babka , David Hildenbrand , Miaohe Lin , NeilBrown , Alistair Popple , Suren Baghdasaryan , Peter Xu , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" vmpressure is used in cgroup v1 to notify userspace of reclaim efficiency events, and is also used in both cgroup v1 and v2 as a signal for memory pressure for networking, see mem_cgroup_under_socket_pressure(). Proactive reclaim intends to probe memcgs for cold memory, without affecting their performance. Hence, reclaim caused by writing to memory.reclaim should not trigger vmpressure. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt --- Changes in v4: - Removed unneeded reclaim_options local variables (Andrew). Changes in v3: - Limited the vmpressure change to memory.reclaim, dropped psi changes, updated changelog to reflect new behavior (Michal, Shakeel) Changes in v2: - Removed unnecessary initializations to zero (Andrew). - Separate declarations and initializations when it causes line wrapping (Andrew). --- include/linux/swap.h | 5 ++++- mm/memcontrol.c | 24 ++++++++++++++---------- mm/vmscan.c | 27 +++++++++++++++++---------- 3 files changed, 35 insertions(+), 21 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 0c0fed1b348f..f6e9eaa2339f 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -411,10 +411,13 @@ extern void lru_cache_add_inactive_or_unevictable(str= uct page *page, extern unsigned long zone_reclaimable_pages(struct zone *zone); extern unsigned long try_to_free_pages(struct zonelist *zonelist, int orde= r, gfp_t gfp_mask, nodemask_t *mask); + +#define MEMCG_RECLAIM_MAY_SWAP (1 << 1) +#define MEMCG_RECLAIM_PROACTIVE (1 << 2) extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - bool may_swap); + unsigned int reclaim_options); extern unsigned long mem_cgroup_shrink_node(struct mem_cgroup *mem, gfp_t gfp_mask, bool noswap, pg_data_t *pgdat, diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a550042d88c3..b668224142c7 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2331,7 +2331,8 @@ static unsigned long reclaim_high(struct mem_cgroup *= memcg, =20 psi_memstall_enter(&pflags); nr_reclaimed +=3D try_to_free_mem_cgroup_pages(memcg, nr_pages, - gfp_mask, true); + gfp_mask, + MEMCG_RECLAIM_MAY_SWAP); psi_memstall_leave(&pflags); } while ((memcg =3D parent_mem_cgroup(memcg)) && !mem_cgroup_is_root(memcg)); @@ -2576,7 +2577,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg,= gfp_t gfp_mask, struct page_counter *counter; unsigned long nr_reclaimed; bool passed_oom =3D false; - bool may_swap =3D true; + unsigned int reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP; bool drained =3D false; unsigned long pflags; =20 @@ -2593,7 +2594,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg,= gfp_t gfp_mask, mem_over_limit =3D mem_cgroup_from_counter(counter, memory); } else { mem_over_limit =3D mem_cgroup_from_counter(counter, memsw); - may_swap =3D false; + reclaim_options &=3D ~MEMCG_RECLAIM_MAY_SWAP; } =20 if (batch > nr_pages) { @@ -2620,7 +2621,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg,= gfp_t gfp_mask, =20 psi_memstall_enter(&pflags); nr_reclaimed =3D try_to_free_mem_cgroup_pages(mem_over_limit, nr_pages, - gfp_mask, may_swap); + gfp_mask, reclaim_options); psi_memstall_leave(&pflags); =20 if (mem_cgroup_margin(mem_over_limit) >=3D nr_pages) @@ -3402,8 +3403,8 @@ static int mem_cgroup_resize_max(struct mem_cgroup *m= emcg, continue; } =20 - if (!try_to_free_mem_cgroup_pages(memcg, 1, - GFP_KERNEL, !memsw)) { + if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, + memsw ? 0 : MEMCG_RECLAIM_MAY_SWAP)) { ret =3D -EBUSY; break; } @@ -3513,7 +3514,8 @@ static int mem_cgroup_force_empty(struct mem_cgroup *= memcg) if (signal_pending(current)) return -EINTR; =20 - if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, true)) + if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, + MEMCG_RECLAIM_MAY_SWAP)) nr_retries--; } =20 @@ -6248,7 +6250,7 @@ static ssize_t memory_high_write(struct kernfs_open_f= ile *of, } =20 reclaimed =3D try_to_free_mem_cgroup_pages(memcg, nr_pages - high, - GFP_KERNEL, true); + GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP); =20 if (!reclaimed && !nr_retries--) break; @@ -6297,7 +6299,7 @@ static ssize_t memory_max_write(struct kernfs_open_fi= le *of, =20 if (nr_reclaims) { if (!try_to_free_mem_cgroup_pages(memcg, nr_pages - max, - GFP_KERNEL, true)) + GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP)) nr_reclaims--; continue; } @@ -6426,6 +6428,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, struct mem_cgroup *memcg =3D mem_cgroup_from_css(of_css(of)); unsigned int nr_retries =3D MAX_RECLAIM_RETRIES; unsigned long nr_to_reclaim, nr_reclaimed =3D 0; + unsigned int reclaim_options; int err; =20 buf =3D strstrip(buf); @@ -6433,6 +6436,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, if (err) return err; =20 + reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP | MEMCG_RECLAIM_PROACTIVE; while (nr_reclaimed < nr_to_reclaim) { unsigned long reclaimed; =20 @@ -6449,7 +6453,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, =20 reclaimed =3D try_to_free_mem_cgroup_pages(memcg, nr_to_reclaim - nr_reclaimed, - GFP_KERNEL, true); + GFP_KERNEL, reclaim_options); =20 if (!reclaimed && !nr_retries--) return -EAGAIN; diff --git a/mm/vmscan.c b/mm/vmscan.c index f7d9a683e3a7..0969e6408a53 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -102,6 +102,9 @@ struct scan_control { /* Can pages be swapped as part of reclaim? */ unsigned int may_swap:1; =20 + /* Proactive reclaim invoked by userspace through memory.reclaim */ + unsigned int proactive:1; + /* * Cgroup memory below memory.low is protected as long as we * don't threaten to OOM. If any cgroup is reclaimed at @@ -3125,9 +3128,10 @@ static void shrink_node_memcgs(pg_data_t *pgdat, str= uct scan_control *sc) sc->priority); =20 /* Record the group's reclaim efficiency */ - vmpressure(sc->gfp_mask, memcg, false, - sc->nr_scanned - scanned, - sc->nr_reclaimed - reclaimed); + if (!sc->proactive) + vmpressure(sc->gfp_mask, memcg, false, + sc->nr_scanned - scanned, + sc->nr_reclaimed - reclaimed); =20 } while ((memcg =3D mem_cgroup_iter(target_memcg, memcg, NULL))); } @@ -3250,9 +3254,10 @@ static void shrink_node(pg_data_t *pgdat, struct sca= n_control *sc) } =20 /* Record the subtree's reclaim efficiency */ - vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, - sc->nr_scanned - nr_scanned, - sc->nr_reclaimed - nr_reclaimed); + if (!sc->proactive) + vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, + sc->nr_scanned - nr_scanned, + sc->nr_reclaimed - nr_reclaimed); =20 if (sc->nr_reclaimed - nr_reclaimed) reclaimable =3D true; @@ -3534,8 +3539,9 @@ static unsigned long do_try_to_free_pages(struct zone= list *zonelist, __count_zid_vm_events(ALLOCSTALL, sc->reclaim_idx, 1); =20 do { - vmpressure_prio(sc->gfp_mask, sc->target_mem_cgroup, - sc->priority); + if (!sc->proactive) + vmpressure_prio(sc->gfp_mask, sc->target_mem_cgroup, + sc->priority); sc->nr_scanned =3D 0; shrink_zones(zonelist, sc); =20 @@ -3825,7 +3831,7 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgrou= p *memcg, unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - bool may_swap) + unsigned int reclaim_options) { unsigned long nr_reclaimed; unsigned int noreclaim_flag; @@ -3838,7 +3844,8 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem= _cgroup *memcg, .priority =3D DEF_PRIORITY, .may_writepage =3D !laptop_mode, .may_unmap =3D 1, - .may_swap =3D may_swap, + .may_swap =3D !!(reclaim_options & MEMCG_RECLAIM_MAY_SWAP), + .proactive =3D !!(reclaim_options & MEMCG_RECLAIM_PROACTIVE), }; /* * Traverse the ZONELIST_FALLBACK zonelist of the current node to put --=20 2.37.0.144.g8ac04bfd2-goog