From nobody Sun Apr 19 13:43:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E8C4C433EF for ; Thu, 30 Jun 2022 08:31:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233986AbiF3IbB (ORCPT ); Thu, 30 Jun 2022 04:31:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34484 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230160AbiF3Iau (ORCPT ); Thu, 30 Jun 2022 04:30:50 -0400 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 57491175A0 for ; Thu, 30 Jun 2022 01:30:49 -0700 (PDT) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-2dc7bdd666fso147199957b3.7 for ; Thu, 30 Jun 2022 01:30:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=7HQituB3ydtZlLOBUfMwLNbC40NYxK5d+DcsRkBqWVw=; b=qvYugF4LGa9HNkaUfzlCATYgsT4KFJted/Z9XADqaC/MwyWLYNNFnLTNo2l+99eEpa lbf86B2fa2VUfXIazwlLRNTH5NI9s/e/ltuIPIsFxNqhvEwkDmt6zK57w1qFmFb0Vmgw 9zTXjXP/P6Ml+0jYtnYufTU1yCWNRs8EoKD36nQz1QbzBYoTIiyQ+lOFN3xYExI5V2SN 8fJfNqPkMrReaepHmvGHAY35gqvEucW7I1BdDoJrgbaJQ2MOnDnu8oqHjUsTmK+/m1mM 2QTuCxNnzztReKYN1JxkqHzhvQZ6MaFunq58+5kZYgA2gBehwTT8wsqbN9jnBLU9cPMj 6jOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=7HQituB3ydtZlLOBUfMwLNbC40NYxK5d+DcsRkBqWVw=; b=BgnTNj/WvZUKjb3pJHMHyr/qf3NfxhmbUeXdbmLKmj8jRCcufR/I6qxpqgPx3jaLZ1 m4vndlwis0I3PL84jkSbNXcyFhFLf44PpHxIDe/QqnkryqQvHGaIGWmIZpzG4eEoZksR LAfXaZGpTTYApJSnGO6ts4gvCgtnGjTKjn3yG+JwpZydOm8PiajA/Taf6zZo9NBKlfma Mqp+d5bHBE2nM+BNPCFiDsCkrSQkdoSpLJL38+ZZDQWpvGqtNYthIg3P7V+PUdzocRDg S0Zyb3IdIPSjv27XVL+hvQbrYorxL91t15ZxOcXnorAmAIDi8SlUlmqkqBHkdOICrhDS iZiQ== X-Gm-Message-State: AJIora8c59Kc9qxEwG09TYdLR3RY8IcQwsxU2v1Iu+VT/RqGC1siC6d3 uwzwfUFb7NAG/tNVI66oxVEIg61RctH/vw9s X-Google-Smtp-Source: AGRyM1vdFStgKd8mzDiN3GzzklDeJQtLdXUlA8XSsBQYP+u201j5Qau6OzQmviGHrBZOcwUh87d6Mwl74erD1XxU X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a81:52d7:0:b0:317:b3bf:5c39 with SMTP id g206-20020a8152d7000000b00317b3bf5c39mr9197060ywb.99.1656577848620; Thu, 30 Jun 2022 01:30:48 -0700 (PDT) Date: Thu, 30 Jun 2022 08:30:44 +0000 Message-Id: <20220630083044.997474-1-yosryahmed@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v3] mm: vmpressure: don't count proactive reclaim in vmpressure From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: Matthew Wilcox , Vlastimil Babka , David Hildenbrand , Miaohe Lin , NeilBrown , Alistair Popple , Suren Baghdasaryan , Peter Xu , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" vmpressure is used in cgroup v1 to notify userspace of reclaim efficiency events, and is also used in both cgroup v1 and v2 as a signal for memory pressure for networking, see mem_cgroup_under_socket_pressure(). Proactive reclaim intends to probe memcgs for cold memory, without affecting their performance. Hence, reclaim caused by writing to memory.reclaim should not trigger vmpressure. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt --- Changes in v3: - Limited the vmpressure change to memory.reclaim, dropped psi changes, updated changelog to reflect new behavior (Michal, Shakeel) Changes in v2: - Removed unnecessary initializations to zero (Andrew). - Separate declarations and initializations when it causes line wrapping (Andrew). --- include/linux/swap.h | 5 ++++- mm/memcontrol.c | 27 ++++++++++++++++++--------- mm/vmscan.c | 27 +++++++++++++++++---------- 3 files changed, 39 insertions(+), 20 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 0c0fed1b348f2..f6e9eaa2339fe 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -411,10 +411,13 @@ extern void lru_cache_add_inactive_or_unevictable(str= uct page *page, extern unsigned long zone_reclaimable_pages(struct zone *zone); extern unsigned long try_to_free_pages(struct zonelist *zonelist, int orde= r, gfp_t gfp_mask, nodemask_t *mask); + +#define MEMCG_RECLAIM_MAY_SWAP (1 << 1) +#define MEMCG_RECLAIM_PROACTIVE (1 << 2) extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - bool may_swap); + unsigned int reclaim_options); extern unsigned long mem_cgroup_shrink_node(struct mem_cgroup *mem, gfp_t gfp_mask, bool noswap, pg_data_t *pgdat, diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 9f2731d62eef2..8dc29cbb5fbf7 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2319,6 +2319,7 @@ static unsigned long reclaim_high(struct mem_cgroup *= memcg, gfp_t gfp_mask) { unsigned long nr_reclaimed =3D 0; + unsigned int reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP; =20 do { unsigned long pflags; @@ -2331,7 +2332,8 @@ static unsigned long reclaim_high(struct mem_cgroup *= memcg, =20 psi_memstall_enter(&pflags); nr_reclaimed +=3D try_to_free_mem_cgroup_pages(memcg, nr_pages, - gfp_mask, true); + gfp_mask, + reclaim_options); psi_memstall_leave(&pflags); } while ((memcg =3D parent_mem_cgroup(memcg)) && !mem_cgroup_is_root(memcg)); @@ -2576,7 +2578,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg,= gfp_t gfp_mask, struct page_counter *counter; unsigned long nr_reclaimed; bool passed_oom =3D false; - bool may_swap =3D true; + unsigned int reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP; bool drained =3D false; unsigned long pflags; =20 @@ -2593,7 +2595,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg,= gfp_t gfp_mask, mem_over_limit =3D mem_cgroup_from_counter(counter, memory); } else { mem_over_limit =3D mem_cgroup_from_counter(counter, memsw); - may_swap =3D false; + reclaim_options &=3D ~MEMCG_RECLAIM_MAY_SWAP; } =20 if (batch > nr_pages) { @@ -2620,7 +2622,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg,= gfp_t gfp_mask, =20 psi_memstall_enter(&pflags); nr_reclaimed =3D try_to_free_mem_cgroup_pages(mem_over_limit, nr_pages, - gfp_mask, may_swap); + gfp_mask, reclaim_options); psi_memstall_leave(&pflags); =20 if (mem_cgroup_margin(mem_over_limit) >=3D nr_pages) @@ -3369,6 +3371,7 @@ static int mem_cgroup_resize_max(struct mem_cgroup *m= emcg, int ret; bool limits_invariant; struct page_counter *counter =3D memsw ? &memcg->memsw : &memcg->memory; + unsigned int reclaim_options =3D memsw ? 0 : MEMCG_RECLAIM_MAY_SWAP; =20 do { if (signal_pending(current)) { @@ -3403,7 +3406,7 @@ static int mem_cgroup_resize_max(struct mem_cgroup *m= emcg, } =20 if (!try_to_free_mem_cgroup_pages(memcg, 1, - GFP_KERNEL, !memsw)) { + GFP_KERNEL, reclaim_options)) { ret =3D -EBUSY; break; } @@ -3502,6 +3505,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t= *pgdat, int order, static int mem_cgroup_force_empty(struct mem_cgroup *memcg) { int nr_retries =3D MAX_RECLAIM_RETRIES; + unsigned int reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP; =20 /* we call try-to-free pages for make this cgroup empty */ lru_add_drain_all(); @@ -3513,7 +3517,8 @@ static int mem_cgroup_force_empty(struct mem_cgroup *= memcg) if (signal_pending(current)) return -EINTR; =20 - if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, true)) + if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, + reclaim_options)) nr_retries--; } =20 @@ -6222,6 +6227,7 @@ static ssize_t memory_high_write(struct kernfs_open_f= ile *of, unsigned int nr_retries =3D MAX_RECLAIM_RETRIES; bool drained =3D false; unsigned long high; + unsigned int reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP; int err; =20 buf =3D strstrip(buf); @@ -6248,7 +6254,7 @@ static ssize_t memory_high_write(struct kernfs_open_f= ile *of, } =20 reclaimed =3D try_to_free_mem_cgroup_pages(memcg, nr_pages - high, - GFP_KERNEL, true); + GFP_KERNEL, reclaim_options); =20 if (!reclaimed && !nr_retries--) break; @@ -6271,6 +6277,7 @@ static ssize_t memory_max_write(struct kernfs_open_fi= le *of, unsigned int nr_reclaims =3D MAX_RECLAIM_RETRIES; bool drained =3D false; unsigned long max; + unsigned int reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP; int err; =20 buf =3D strstrip(buf); @@ -6297,7 +6304,7 @@ static ssize_t memory_max_write(struct kernfs_open_fi= le *of, =20 if (nr_reclaims) { if (!try_to_free_mem_cgroup_pages(memcg, nr_pages - max, - GFP_KERNEL, true)) + GFP_KERNEL, reclaim_options)) nr_reclaims--; continue; } @@ -6426,6 +6433,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, struct mem_cgroup *memcg =3D mem_cgroup_from_css(of_css(of)); unsigned int nr_retries =3D MAX_RECLAIM_RETRIES; unsigned long nr_to_reclaim, nr_reclaimed =3D 0; + unsigned int reclaim_options; int err; =20 buf =3D strstrip(buf); @@ -6433,6 +6441,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, if (err) return err; =20 + reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP | MEMCG_RECLAIM_PROACTIVE; while (nr_reclaimed < nr_to_reclaim) { unsigned long reclaimed; =20 @@ -6449,7 +6458,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, =20 reclaimed =3D try_to_free_mem_cgroup_pages(memcg, nr_to_reclaim - nr_reclaimed, - GFP_KERNEL, true); + GFP_KERNEL, reclaim_options); =20 if (!reclaimed && !nr_retries--) return -EAGAIN; diff --git a/mm/vmscan.c b/mm/vmscan.c index f7d9a683e3a7d..0969e6408a539 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -102,6 +102,9 @@ struct scan_control { /* Can pages be swapped as part of reclaim? */ unsigned int may_swap:1; =20 + /* Proactive reclaim invoked by userspace through memory.reclaim */ + unsigned int proactive:1; + /* * Cgroup memory below memory.low is protected as long as we * don't threaten to OOM. If any cgroup is reclaimed at @@ -3125,9 +3128,10 @@ static void shrink_node_memcgs(pg_data_t *pgdat, str= uct scan_control *sc) sc->priority); =20 /* Record the group's reclaim efficiency */ - vmpressure(sc->gfp_mask, memcg, false, - sc->nr_scanned - scanned, - sc->nr_reclaimed - reclaimed); + if (!sc->proactive) + vmpressure(sc->gfp_mask, memcg, false, + sc->nr_scanned - scanned, + sc->nr_reclaimed - reclaimed); =20 } while ((memcg =3D mem_cgroup_iter(target_memcg, memcg, NULL))); } @@ -3250,9 +3254,10 @@ static void shrink_node(pg_data_t *pgdat, struct sca= n_control *sc) } =20 /* Record the subtree's reclaim efficiency */ - vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, - sc->nr_scanned - nr_scanned, - sc->nr_reclaimed - nr_reclaimed); + if (!sc->proactive) + vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, + sc->nr_scanned - nr_scanned, + sc->nr_reclaimed - nr_reclaimed); =20 if (sc->nr_reclaimed - nr_reclaimed) reclaimable =3D true; @@ -3534,8 +3539,9 @@ static unsigned long do_try_to_free_pages(struct zone= list *zonelist, __count_zid_vm_events(ALLOCSTALL, sc->reclaim_idx, 1); =20 do { - vmpressure_prio(sc->gfp_mask, sc->target_mem_cgroup, - sc->priority); + if (!sc->proactive) + vmpressure_prio(sc->gfp_mask, sc->target_mem_cgroup, + sc->priority); sc->nr_scanned =3D 0; shrink_zones(zonelist, sc); =20 @@ -3825,7 +3831,7 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgrou= p *memcg, unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - bool may_swap) + unsigned int reclaim_options) { unsigned long nr_reclaimed; unsigned int noreclaim_flag; @@ -3838,7 +3844,8 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem= _cgroup *memcg, .priority =3D DEF_PRIORITY, .may_writepage =3D !laptop_mode, .may_unmap =3D 1, - .may_swap =3D may_swap, + .may_swap =3D !!(reclaim_options & MEMCG_RECLAIM_MAY_SWAP), + .proactive =3D !!(reclaim_options & MEMCG_RECLAIM_PROACTIVE), }; /* * Traverse the ZONELIST_FALLBACK zonelist of the current node to put --=20 2.37.0.rc0.161.g10f37bed90-goog