From nobody Fri Dec 19 14:13:29 2025 Received: from mail-oi1-f178.google.com (mail-oi1-f178.google.com [209.85.167.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1862B4175F; Wed, 20 Dec 2023 15:27:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ykz4Zk4Y" Received: by mail-oi1-f178.google.com with SMTP id 5614622812f47-3bb7344a0e1so348800b6e.0; Wed, 20 Dec 2023 07:27:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703086028; x=1703690828; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=59LZCmYd65g0ypkperKKK/kJ8MVLAq87LEnhd0/0rRY=; b=Ykz4Zk4YQCBbpgt+DJNn1KXxw74vJqW17rwIKaS2Ws5AXcEFPvuKkKA0iHAQDoPDxA 35cgr2MFrOCMmQfmLUVb1WZfLg24rg9cOOOCU7vg8Oc9F5/nlM7zJOrrUay1OZmLcT1R Z76Rv4OicwSD2r94CEY/uJiXx3AVPueBsUcUyBacmwF06Q1gz9O83zZCYoq7FJrCtcx/ N6nfIIPqEa4SRYaOcllQMP/SIvQsyWzkHZXSXkIBK+C7EW8a/omF7OJ1Ci0YAK5cEkcG xgif4TmwxscsLWlxlaq9oDUSpuT5Z8vmosiJ/qZa3HyNc+cA8FymStl8ERBdm8OHuFGU 9rMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703086028; x=1703690828; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=59LZCmYd65g0ypkperKKK/kJ8MVLAq87LEnhd0/0rRY=; b=TH6a/mkriBF3A5LboMpQDERUgaVtKM2dJjpuihPr78tcCJ18AFiy4ad6/EvHlsFn66 d7rPy9JpNkEUU3d15nI9/nXDaMrVoy27Y1GwKubgOsKjxzb+KQCke9ZW/NfCiFRX28t8 HTGdmh6nb6s7A8K7wPJjvrG64wMRbyDZ9oDBdY+zxlDW8DS3gGPJcv6NLqEyDq2utlzg fNs2PVWir+ZJAfQbpI7WtgO+bMSqwIsdLxgO/+d1g6ML5Szf2YjJ9T9ZrGFMEbOPW1V3 CI1d4FTwmPBvQW0rf3rctSLt/YIMlKS/e0BUB7NSgKJks1IVPWUNuhERqpoRlqG0PNaA oQUw== X-Gm-Message-State: AOJu0YyZXx5i6p5O6xJuW3EsEPWJ3sHHQnKY/mxMYOBs5crtif0XvFPJ 7E7AAeAaGu6DUdqJ6RBm7Byss4VHs4Eb5Q== X-Google-Smtp-Source: AGHT+IH48x/lOXdg5JUxZV/lLg8CrWcbuR3qeufJmbXflf5jebTrUVeALmaxLuoHMFc2j402sC1Y/A== X-Received: by 2002:a05:6808:16a0:b0:3bb:535d:27ed with SMTP id bb32-20020a05680816a000b003bb535d27edmr4751625oib.86.1703086027982; Wed, 20 Dec 2023 07:27:07 -0800 (PST) Received: from localhost ([2620:10d:c091:400::5:1124]) by smtp.gmail.com with ESMTPSA id k4-20020ad44504000000b0067f6a9a446esm743093qvu.136.2023.12.20.07.27.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Dec 2023 07:27:07 -0800 (PST) From: Dan Schatzberg To: Johannes Weiner , Roman Gushchin , Yosry Ahmed , Huan Yang Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Tejun Heo , Zefan Li , Jonathan Corbet , Michal Hocko , Shakeel Butt , Muchun Song , Andrew Morton , Kefeng Wang , SeongJae Park , "Vishal Moola (Oracle)" , Nhat Pham , Yue Zhao Subject: [PATCH v5 1/2] mm: add defines for min/max swappiness Date: Wed, 20 Dec 2023 07:26:50 -0800 Message-Id: <20231220152653.3273778-2-schatzberg.dan@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20231220152653.3273778-1-schatzberg.dan@gmail.com> References: <20231220152653.3273778-1-schatzberg.dan@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We use the constants 0 and 200 in a few places in the mm code when referring to the min and max swappiness. This patch adds MIN_SWAPPINESS and MAX_SWAPPINESS #defines to improve clarity. There are no functional changes. Signed-off-by: Dan Schatzberg Acked-by: Chris Li Acked-by: David Rientjes Reviewed-by: Nhat Pham --- include/linux/swap.h | 2 ++ mm/memcontrol.c | 2 +- mm/vmscan.c | 14 +++++++------- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index f6dd6575b905..e2ab76c25b4a 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -407,6 +407,8 @@ extern unsigned long try_to_free_pages(struct zonelist = *zonelist, int order, =20 #define MEMCG_RECLAIM_MAY_SWAP (1 << 1) #define MEMCG_RECLAIM_PROACTIVE (1 << 2) +#define MIN_SWAPPINESS 0 +#define MAX_SWAPPINESS 200 extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b226090fd906..fbe9f02dd206 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4337,7 +4337,7 @@ static int mem_cgroup_swappiness_write(struct cgroup_= subsys_state *css, { struct mem_cgroup *memcg =3D mem_cgroup_from_css(css); =20 - if (val > 200) + if (val > MAX_SWAPPINESS) return -EINVAL; =20 if (!mem_cgroup_is_root(memcg)) diff --git a/mm/vmscan.c b/mm/vmscan.c index 9dd8977de5a2..d91963e2d47f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -183,7 +183,7 @@ struct scan_control { #endif =20 /* - * From 0 .. 200. Higher means more swappy. + * From 0 .. MAX_SWAPPINESS. Higher means more swappy. */ int vm_swappiness =3D 60; =20 @@ -2403,7 +2403,7 @@ static void get_scan_count(struct lruvec *lruvec, str= uct scan_control *sc, ap =3D swappiness * (total_cost + 1); ap /=3D anon_cost + 1; =20 - fp =3D (200 - swappiness) * (total_cost + 1); + fp =3D (MAX_SWAPPINESS - swappiness) * (total_cost + 1); fp /=3D file_cost + 1; =20 fraction[0] =3D ap; @@ -4400,7 +4400,7 @@ static int get_type_to_scan(struct lruvec *lruvec, in= t swappiness, int *tier_idx { int type, tier; struct ctrl_pos sp, pv; - int gain[ANON_AND_FILE] =3D { swappiness, 200 - swappiness }; + int gain[ANON_AND_FILE] =3D { swappiness, MAX_SWAPPINESS - swappiness }; =20 /* * Compare the first tier of anon with that of file to determine which @@ -4436,7 +4436,7 @@ static int isolate_folios(struct lruvec *lruvec, stru= ct scan_control *sc, int sw /* * Try to make the obvious choice first. When anon and file are both * available from the same generation, interpret swappiness 1 as file - * first and 200 as anon first. + * first and MAX_SWAPPINESS as anon first. */ if (!swappiness) type =3D LRU_GEN_FILE; @@ -4444,7 +4444,7 @@ static int isolate_folios(struct lruvec *lruvec, stru= ct scan_control *sc, int sw type =3D LRU_GEN_ANON; else if (swappiness =3D=3D 1) type =3D LRU_GEN_FILE; - else if (swappiness =3D=3D 200) + else if (swappiness =3D=3D MAX_SWAPPINESS) type =3D LRU_GEN_ANON; else type =3D get_type_to_scan(lruvec, swappiness, &tier); @@ -5398,9 +5398,9 @@ static int run_cmd(char cmd, int memcg_id, int nid, u= nsigned long seq, =20 lruvec =3D get_lruvec(memcg, nid); =20 - if (swappiness < 0) + if (swappiness < MIN_SWAPPINESS) swappiness =3D get_swappiness(lruvec, sc); - else if (swappiness > 200) + else if (swappiness > MAX_SWAPPINESS) goto done; =20 switch (cmd) { --=20 2.39.3 From nobody Fri Dec 19 14:13:29 2025 Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A583445C00; Wed, 20 Dec 2023 15:27:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B6CZhDbZ" Received: by mail-qv1-f51.google.com with SMTP id 6a1803df08f44-67f6739d7acso6682886d6.1; Wed, 20 Dec 2023 07:27:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1703086032; x=1703690832; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GGeG3BQJs8KXUJq1DB8Qp/yk33MHWE5qrkTbzfRxpso=; b=B6CZhDbZTssf4dr4OT2r7iR0IoEynblvXqPPLCFPYid+uXotPw+21bK2ekzxE9t84N hCouCGQgwMQmObDMfRJX5JZDLy7KG8jp5HAjNrHFRuyclnaFqAmInY20wJoW+NsMqOLM WBQ+0m7Vhkf+PbRkHKjAxh7xjNmEaSq3nWnLHk2SvIy4CCtk2Ws3oYGkL01/Iq4k8Bg9 rMAH8Ppy03a5fErrIGuZCdQslSBQdwdXaXmt8ja+m0kwONhtZsopZ2bJoaUh2VVtsVUj peaQ1scaOXkw4Y+9zu/tR14AK6YKXe2FuTHk0QMl24XDxyketBvwb43isBS3BjvszBGG a4lQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703086032; x=1703690832; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GGeG3BQJs8KXUJq1DB8Qp/yk33MHWE5qrkTbzfRxpso=; b=QDEoBJaAvmkjSac7DWCKiOoHaq+iwfd/4EwF1pSr2UFjVJZxmaBhh9Um8tzfF9gf0J n71tbPkYZK7y2mEBCI1Xw4BpXqYEx7YJ4iE3F9XZNG1XqIhQqFL1gNGcSkGvsD24OC12 dXnKld947SaWFGqLygOFgHFeM06Mn676ps4wAUVmgG0S/MVz5YRhc5VF9EfSlBaIYn1p eoSlpWxR4DXP/t1qVrxVH4KqjPdGeXr3C8lTwIrgHkFVs2Eb79CMtC7iFmO3q8QkJTx3 1A3IQeNlAs6ub+cxt/da3j1Mv+ZMOU+wFj1GZ0hvRrU3JDzo4SLzGlygn8jTyWbH0+Hk Jimw== X-Gm-Message-State: AOJu0YzvCyri1FzOYAXBwgw7mKWNzyuJjlXwdlHF8dhs5mSqXPp5jv8W 0bjKebZu8iPID8QjaFWSTgQ= X-Google-Smtp-Source: AGHT+IEzkCtuXKrCGnKuBmzDm+mPo6Gt/cgqwgYPQEwzS5DMjQVntJCFa2a9HakWaD1ykKsbxKhg2A== X-Received: by 2002:a05:6214:1d0a:b0:67f:4c93:6f16 with SMTP id e10-20020a0562141d0a00b0067f4c936f16mr4682609qvd.27.1703086032331; Wed, 20 Dec 2023 07:27:12 -0800 (PST) Received: from localhost ([2620:10d:c091:400::5:1124]) by smtp.gmail.com with ESMTPSA id m14-20020a0ce8ce000000b0067f19f17629sm4991299qvo.82.2023.12.20.07.27.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Dec 2023 07:27:12 -0800 (PST) From: Dan Schatzberg To: Johannes Weiner , Roman Gushchin , Yosry Ahmed , Huan Yang Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Tejun Heo , Zefan Li , Jonathan Corbet , Michal Hocko , Shakeel Butt , Muchun Song , Andrew Morton , Kefeng Wang , SeongJae Park , "Vishal Moola (Oracle)" , Nhat Pham , Yue Zhao Subject: [PATCH v5 2/2] mm: add swapiness= arg to memory.reclaim Date: Wed, 20 Dec 2023 07:26:51 -0800 Message-Id: <20231220152653.3273778-3-schatzberg.dan@gmail.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20231220152653.3273778-1-schatzberg.dan@gmail.com> References: <20231220152653.3273778-1-schatzberg.dan@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allow proactive reclaimers to submit an additional swappiness=3D argument to memory.reclaim. This overrides the global or per-memcg swappiness setting for that reclaim attempt. For example: echo "2M swappiness=3D0" > /sys/fs/cgroup/memory.reclaim will perform reclaim on the rootcg with a swappiness setting of 0 (no swap) regardless of the vm.swappiness sysctl setting. Userspace proactive reclaimers use the memory.reclaim interface to trigger reclaim. The memory.reclaim interface does not allow for any way to effect the balance of file vs anon during proactive reclaim. The only approach is to adjust the vm.swappiness setting. However, there are a few reasons we look to control the balance of file vs anon during proactive reclaim, separately from reactive reclaim: * Swapout should be limited to manage SSD write endurance. In near-OOM situations we are fine with lots of swap-out to avoid OOMs. As these are typically rare events, they have relatively little impact on write endurance. However, proactive reclaim runs continuously and so its impact on SSD write endurance is more significant. Therefore it is desireable to control swap-out for proactive reclaim separately from reactive reclaim * Some userspace OOM killers like systemd-oomd[1] support OOM killing on swap exhaustion. This makes sense if the swap exhaustion is triggered due to reactive reclaim but less so if it is triggered due to proactive reclaim (e.g. one could see OOMs when free memory is ample but anon is just particularly cold). Therefore, it's desireable to have proactive reclaim reduce or stop swap-out before the threshold at which OOM killing occurs. In the case of Meta's Senpai proactive reclaimer, we adjust vm.swappiness before writes to memory.reclaim[2]. This has been in production for nearly two years and has addressed our needs to control proactive vs reactive reclaim behavior but is still not ideal for a number of reasons: * vm.swappiness is a global setting, adjusting it can race/interfere with other system administration that wishes to control vm.swappiness. In our case, we need to disable Senpai before adjusting vm.swappiness. * vm.swappiness is stateful - so a crash or restart of Senpai can leave a misconfigured setting. This requires some additional management to record the "desired" setting and ensure Senpai always adjusts to it. With this patch, we avoid these downsides of adjusting vm.swappiness globally. [1]https://www.freedesktop.org/software/systemd/man/latest/systemd-oomd.ser= vice.html [2]https://github.com/facebookincubator/oomd/blob/main/src/oomd/plugins/Sen= pai.cpp#L585-L598 Signed-off-by: Dan Schatzberg Acked-by: Chris Li Acked-by: David Rientjes Acked-by: Michal Hocko --- Documentation/admin-guide/cgroup-v2.rst | 18 ++++---- include/linux/swap.h | 3 +- mm/memcontrol.c | 56 ++++++++++++++++++++----- mm/vmscan.c | 13 +++++- 4 files changed, 69 insertions(+), 21 deletions(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-= guide/cgroup-v2.rst index 3f85254f3cef..ee42f74e0765 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1282,17 +1282,10 @@ PAGE_SIZE multiple when read back. This is a simple interface to trigger memory reclaim in the target cgroup. =20 - This file accepts a single key, the number of bytes to reclaim. - No nested keys are currently supported. - Example:: =20 echo "1G" > memory.reclaim =20 - The interface can be later extended with nested keys to - configure the reclaim behavior. For example, specify the - type of memory to reclaim from (anon, file, ..). - Please note that the kernel can over or under reclaim from the target cgroup. If less bytes are reclaimed than the specified amount, -EAGAIN is returned. @@ -1304,6 +1297,17 @@ PAGE_SIZE multiple when read back. This means that the networking layer will not adapt based on reclaim induced by memory.reclaim. =20 +The following nested keys are defined. + + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + swappiness Swappiness value to reclaim with + =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + + Specifying a swappiness value instructs the kernel to perform + the reclaim with that swappiness value. Note that this has the + same semantics as vm.swappiness applied to memcg reclaim with + all the existing limitations and potential future extensions. + memory.peak A read-only single value file which exists on non-root cgroups. diff --git a/include/linux/swap.h b/include/linux/swap.h index e2ab76c25b4a..8afdec40efe3 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -412,7 +412,8 @@ extern unsigned long try_to_free_pages(struct zonelist = *zonelist, int order, extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - unsigned int reclaim_options); + unsigned int reclaim_options, + int *swappiness); extern unsigned long mem_cgroup_shrink_node(struct mem_cgroup *mem, gfp_t gfp_mask, bool noswap, pg_data_t *pgdat, diff --git a/mm/memcontrol.c b/mm/memcontrol.c index fbe9f02dd206..6d627a754851 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include @@ -2449,7 +2450,8 @@ static unsigned long reclaim_high(struct mem_cgroup *= memcg, psi_memstall_enter(&pflags); nr_reclaimed +=3D try_to_free_mem_cgroup_pages(memcg, nr_pages, gfp_mask, - MEMCG_RECLAIM_MAY_SWAP); + MEMCG_RECLAIM_MAY_SWAP, + NULL); psi_memstall_leave(&pflags); } while ((memcg =3D parent_mem_cgroup(memcg)) && !mem_cgroup_is_root(memcg)); @@ -2740,7 +2742,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg,= gfp_t gfp_mask, =20 psi_memstall_enter(&pflags); nr_reclaimed =3D try_to_free_mem_cgroup_pages(mem_over_limit, nr_pages, - gfp_mask, reclaim_options); + gfp_mask, reclaim_options, NULL); psi_memstall_leave(&pflags); =20 if (mem_cgroup_margin(mem_over_limit) >=3D nr_pages) @@ -3660,7 +3662,7 @@ static int mem_cgroup_resize_max(struct mem_cgroup *m= emcg, } =20 if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, - memsw ? 0 : MEMCG_RECLAIM_MAY_SWAP)) { + memsw ? 0 : MEMCG_RECLAIM_MAY_SWAP, NULL)) { ret =3D -EBUSY; break; } @@ -3774,7 +3776,7 @@ static int mem_cgroup_force_empty(struct mem_cgroup *= memcg) return -EINTR; =20 if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, - MEMCG_RECLAIM_MAY_SWAP)) + MEMCG_RECLAIM_MAY_SWAP, NULL)) nr_retries--; } =20 @@ -6720,7 +6722,7 @@ static ssize_t memory_high_write(struct kernfs_open_f= ile *of, } =20 reclaimed =3D try_to_free_mem_cgroup_pages(memcg, nr_pages - high, - GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP); + GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP, NULL); =20 if (!reclaimed && !nr_retries--) break; @@ -6769,7 +6771,7 @@ static ssize_t memory_max_write(struct kernfs_open_fi= le *of, =20 if (nr_reclaims) { if (!try_to_free_mem_cgroup_pages(memcg, nr_pages - max, - GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP)) + GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP, NULL)) nr_reclaims--; continue; } @@ -6895,19 +6897,50 @@ static ssize_t memory_oom_group_write(struct kernfs= _open_file *of, return nbytes; } =20 +enum { + MEMORY_RECLAIM_SWAPPINESS =3D 0, + MEMORY_RECLAIM_NULL, +}; + +static const match_table_t tokens =3D { + { MEMORY_RECLAIM_SWAPPINESS, "swappiness=3D%d"}, + { MEMORY_RECLAIM_NULL, NULL }, +}; + static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off) { struct mem_cgroup *memcg =3D mem_cgroup_from_css(of_css(of)); unsigned int nr_retries =3D MAX_RECLAIM_RETRIES; unsigned long nr_to_reclaim, nr_reclaimed =3D 0; + int swappiness =3D -1; unsigned int reclaim_options; - int err; + char *old_buf, *start; + substring_t args[MAX_OPT_ARGS]; =20 buf =3D strstrip(buf); - err =3D page_counter_memparse(buf, "", &nr_to_reclaim); - if (err) - return err; + + old_buf =3D buf; + nr_to_reclaim =3D memparse(buf, &buf) / PAGE_SIZE; + if (buf =3D=3D old_buf) + return -EINVAL; + + buf =3D strstrip(buf); + + while ((start =3D strsep(&buf, " ")) !=3D NULL) { + if (!strlen(start)) + continue; + switch (match_token(start, tokens, args)) { + case MEMORY_RECLAIM_SWAPPINESS: + if (match_int(&args[0], &swappiness)) + return -EINVAL; + if (swappiness < MIN_SWAPPINESS || swappiness > MAX_SWAPPINESS) + return -EINVAL; + break; + default: + return -EINVAL; + } + } =20 reclaim_options =3D MEMCG_RECLAIM_MAY_SWAP | MEMCG_RECLAIM_PROACTIVE; while (nr_reclaimed < nr_to_reclaim) { @@ -6926,7 +6959,8 @@ static ssize_t memory_reclaim(struct kernfs_open_file= *of, char *buf, =20 reclaimed =3D try_to_free_mem_cgroup_pages(memcg, min(nr_to_reclaim - nr_reclaimed, SWAP_CLUSTER_MAX), - GFP_KERNEL, reclaim_options); + GFP_KERNEL, reclaim_options, + swappiness =3D=3D -1 ? NULL : &swappiness); =20 if (!reclaimed && !nr_retries--) return -EAGAIN; diff --git a/mm/vmscan.c b/mm/vmscan.c index d91963e2d47f..aa5666842c49 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -92,6 +92,9 @@ struct scan_control { unsigned long anon_cost; unsigned long file_cost; =20 + /* Swappiness value for reclaim. NULL will fall back to per-memcg/global = value */ + int *swappiness; + /* Can active folios be deactivated as part of reclaim? */ #define DEACTIVATE_ANON 1 #define DEACTIVATE_FILE 2 @@ -2327,7 +2330,8 @@ static void get_scan_count(struct lruvec *lruvec, str= uct scan_control *sc, struct pglist_data *pgdat =3D lruvec_pgdat(lruvec); struct mem_cgroup *memcg =3D lruvec_memcg(lruvec); unsigned long anon_cost, file_cost, total_cost; - int swappiness =3D mem_cgroup_swappiness(memcg); + int swappiness =3D sc->swappiness ? + *sc->swappiness : mem_cgroup_swappiness(memcg); u64 fraction[ANON_AND_FILE]; u64 denominator =3D 0; /* gcc */ enum scan_balance scan_balance; @@ -2608,6 +2612,9 @@ static int get_swappiness(struct lruvec *lruvec, stru= ct scan_control *sc) mem_cgroup_get_nr_swap_pages(memcg) < MIN_LRU_BATCH) return 0; =20 + if (sc->swappiness) + return *sc->swappiness; + return mem_cgroup_swappiness(memcg); } =20 @@ -6463,12 +6470,14 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgr= oup *memcg, unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - unsigned int reclaim_options) + unsigned int reclaim_options, + int *swappiness) { unsigned long nr_reclaimed; unsigned int noreclaim_flag; struct scan_control sc =3D { .nr_to_reclaim =3D max(nr_pages, SWAP_CLUSTER_MAX), + .swappiness =3D swappiness, .gfp_mask =3D (current_gfp_context(gfp_mask) & GFP_RECLAIM_MASK) | (GFP_HIGHUSER_MOVABLE & ~GFP_RECLAIM_MASK), .reclaim_idx =3D MAX_NR_ZONES - 1, --=20 2.39.3