From nobody Fri Dec 19 09:13:32 2025 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2063.outbound.protection.outlook.com [40.107.117.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB70014883E; Tue, 2 Jul 2024 08:46:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.117.63 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719909971; cv=fail; b=nwmr57bicxgZdQ2b6fZvVMwTD29N8N+Sbh3/e5y8f+tDZshMnBZQwUFG0ymVzV51zAmLzTeRjnNqE9o9BU0JOTWhIYwrOoym/GtA5WThHxPdMpP1h46/q+eYAnGhFq1/rEr4gKKGQ54z7hxJX4tCwcqr8YCvFTFhhVjkTAJfgOs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719909971; c=relaxed/simple; bh=z1SOsXXLrTMLBIFQ74lUffXP3Mn/1wGvSqo3YZPCBp0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=mNoACLTIsWVvgGDKL5DzWHRVxEq8uCB3bUHkR8Ma8i6IUWr97dljEyb6r/aWgA6bL6Ec4mZBOmUBJ870vCY4w6CLdZUMeB4qTDna+8/2aib1rycUlKwuvEqw12KoBKqDVO57hHtwvKgYBHtyxTm/UYDP96pE5ti3PIX15wxY5UE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=Uleg6qOv; arc=fail smtp.client-ip=40.107.117.63 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="Uleg6qOv" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PArtcsHgRAOSoH4wlQV0BTnJwYMYtPxo9TNyjMxlu+Io0MD6KyPqi640mghyMjGo2MhaESfmWTiFGAP3zFvPZvVhnhSbxOszrRKQpJ+WoUFnm+DmLJcwqc0Ut3wxe5+6B/SXC6WqZAvAan4tnspWxMiznXHbwWr23D/f3LimnOU9gHMwlyEp92zD+ZnkcM6+YbbuVtloeAB7KmO6e3uGVpnkWdyLuPMBzmq93mckW11oLse4pdXgIjLz5OTbzYugTaWqaisa205A2VSG2GERD/xMqJABcTnTyyYA1mD6T1FUYeiX+yeJ9IIwyZCk4DjOsYJdb4E7lkDiYMFMHeeaAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HdNNIIt5i57KrDH2G5inTTWNfKhy1MKf1/9NPo25L0U=; b=ZyE/rjfdNJvt1dGaNUOndoRB4Z3p7ZGQPywyq5lCS1muakE8fnop1g5OJpxtlWwFjlAOT5k1OwAq3jCXTuvCZlD4lxntLbfe/0gU+Y9F/R9u7+L9H6NcP+mHmG5HT3MPvLMGOnZE69lXo9kni0LaXA5quJScAwHPP0wr3qLDAiqs/94/mKc5z979BNV8CRY7Zbs77D6uWFJmfYiNRaC7LkJr7UFDcQUa1auIaBqDMgxvK+cUBU1u1mK5iRKKh2fNinbrvQpMNSR27Z7rYxlts9IVm/legA9w4M51x1Ry//O5C9X1lW9Hp4mRf+1CdKC+WbYZwuDKgzWTVbfVbHDurg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HdNNIIt5i57KrDH2G5inTTWNfKhy1MKf1/9NPo25L0U=; b=Uleg6qOvvtBmqW/bniZ1yZvAX/tErIBcHdw7gJV6hKmLW1fQrnwPboNr2XAIdhL35QqJoJ6d0G6wHcQunSRsrZdclBX7MPPO6lQOyvT65Y1ox07rY69CkadkOuzUXL+BhhVac08YoHfq5L2KeCWDQVGLP4lMAaQjAX5FVECqxui+/2vqkkQuSfP3mNmEjhldD/RELjPfjYchPRh2SFugas4ZQWrtbqCXHV2iTSnWGcNWJ4e3qYhiCkIdZfLeaINF2cFOc+PzJiKJ6QLmL40by/5aL/Gck3j3UPOr7NOg2aEw9mv0H/+04uOQlimQamPRNbyAc29U0eulwOXbv0A0BQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEYPR06MB5790.apcprd06.prod.outlook.com (2603:1096:101:b9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.23; Tue, 2 Jul 2024 08:46:05 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7719.028; Tue, 2 Jul 2024 08:46:05 +0000 From: Huan Yang To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , Ryan Roberts , Chris Li , Dan Schatzberg , Huan Yang , Kairui Song , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christian Brauner Cc: opensource.kernel@vivo.com Subject: [RFC PATCH 1/4] mm: memcg: pmc framework Date: Tue, 2 Jul 2024 16:44:04 +0800 Message-ID: <20240702084423.1717904-2-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240702084423.1717904-1-link@vivo.com> References: <20240702084423.1717904-1-link@vivo.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR01CA0148.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::28) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEYPR06MB5790:EE_ X-MS-Office365-Filtering-Correlation-Id: 84119b26-1c3e-469d-ac40-08dc9a736905 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|52116014|7416014|376014|366016|921020|38350700014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?YMOCAppgicIGMw9JCerPdCkLRRDdwAiX/jpU7zGN5fSPeyM2dr7gvAh2gb+r?= =?us-ascii?Q?JQUTogBf8Qu4Lg58CjBNUiZqFjjhZ9wsRHzLo4a2GaEiZW/aW/THCANTGtHd?= =?us-ascii?Q?UyuQPOoQ54TNsRLXci3Ysv0vJjZuNY4nz+wNlBIwo9JXBYBoJ71xORkQwRzd?= =?us-ascii?Q?Z/7y4Z0BhkbCcFTHoprPE0bjnBnjKGTZcVntGQSAwemHESM6zQxQm1F8QS/A?= =?us-ascii?Q?dHbObv6J4oujjaHQC81XiURMmR6O3tw0/yszKQ2aTFUTKUMWTLXX9pC5RX3h?= =?us-ascii?Q?AG+zyJQBWjIUqkhu9AFmubFIRPPArBuiz7b24Z1xiXzVX9/K5iTCV9Tq6dSR?= =?us-ascii?Q?v55chyoW8F+3WRRVheZaMY5hE38enCer91bEipc1jYHDwpFTZKYvzNF0V0YU?= =?us-ascii?Q?isOUI54XiTkKYo/q7wOmzl6bvNHC61bBIyO3PAEssYiRZkHvaKoSVDaW3tao?= =?us-ascii?Q?Loqlf7FexSV5jWFE8q9lBhjJvNlyrXccPs4eXxZouAzgVG6wiVHEJK+EKsEl?= =?us-ascii?Q?eVmXBsWH9AzytpQScUI1qkaDzGXZ6mtz5aVQgzyN2P188R0bsyv1pjSQvBkd?= =?us-ascii?Q?S8xQnfqddxkCQd/TsNtBwW43zbytHnTWFwC6xH2CPCs8fQSJeuWZsAuf2G/N?= =?us-ascii?Q?VtU1sgyCEk9Cjcekm+TLWDcq8SAth/D3SBP3YKoeOE13nO1zwZho66vUzbBG?= =?us-ascii?Q?c/VfUGABv0RkQix06lckJ1Whc6cfGCchZBRDBrRhh6q3qn60r4Z34EvdXvYH?= =?us-ascii?Q?UnokPiXC/WLKq1O/mbT2YbYVa0iolF4CyNxLaQGEa69Jb47EFJfwlw7q/ffz?= =?us-ascii?Q?OQMwkEONygG5JBPuRPsV/1R2vTRWsLI/s4hyOH/l6VTE/zqUwmQUe1W6yR8X?= =?us-ascii?Q?2xVW8EsnvQBLHS/G9CU7z321yWW/lZl4JBxKCNDdKejdDWNAK8YfHAmalIe4?= =?us-ascii?Q?8HAY5CYaiSI2W17Vk8bu4sU3o0IBKkxidMlWZTAo5uuQNKkc5o4pOXdWM4Mq?= =?us-ascii?Q?uGDLNaxep8xLCBvRajDZTNTBGZwH1fj7PBlDUwwiAXPdZCiNlSuP0UBZwEPM?= =?us-ascii?Q?s/wkdSxWUK/DAw7bQ4URDjJqlpVyrY0uXUj/05MopO0w5XN8vfIrMZdEcvHO?= =?us-ascii?Q?yUZPU1uU9451UmcuoYeCxvBgILQ5k2glpu3j1IWr/xsnbzamrKDtrGWMscbO?= =?us-ascii?Q?k0GlmXIHH3jgu6GmtC1GLpnxjjsxiCxw4YQas1HhazlbB8LtU3M3lJqz9Ius?= =?us-ascii?Q?59+x0D0ZeQ1UQ9BSyAyO4nRI4ZRxbzqMjzI5H7HeuCVY5u67xCbGl1tUvO+J?= =?us-ascii?Q?Btb2f+2mou5lJO0qi6V5ezQ4ppZPBbJPd66NjRPlSaLYvgNs3hrBYNLLB4GK?= =?us-ascii?Q?qHqjV/dPSPzHb56H4w2EwUsX2k6vnpM95HdMGQzvQ6B4OwDCgE+R/gf2p7b6?= =?us-ascii?Q?6qznO7v0dyI=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(52116014)(7416014)(376014)(366016)(921020)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rMuw7FBY94xkJq2Gp8p50HP7cnuz+T1pVbvGAQeSpi/dSPjeGNCWmnUa9kp4?= =?us-ascii?Q?YAoBdGOCXnqcQECybIG/4nLJmki/AgeEny7/DmHre4+0E/OUYNxcxG0sbQkF?= =?us-ascii?Q?RDmN4xHmWHICms4rdYQX4opsULKFiCAAy8HntHF6ERbxCbvaAHZeFKQO9GVW?= =?us-ascii?Q?QxToq7pXfp37+KvcTS5PJTrbblJ79Qopi+LTf4Hf8bfjKRG2ugNIOVXK1E8u?= =?us-ascii?Q?3YIY53iXdXQcFeW97grJRqXemV3b1F4sVEBY5arw+iywX6zLNXEghOcp1mqk?= =?us-ascii?Q?gNGWzMBUaG2ARKsxUw3f9/NBvqkpZq2nhOmyIpApK3jcBB56Tmr/+avQY/Bo?= =?us-ascii?Q?Q5kZY+aZQ5MaOfNLAnFoQ1gegXZcQ1RhHVGRMRQuT2CMPRMxCyJIurIhPSeJ?= =?us-ascii?Q?UNKuOxZqzDc8E5kSPW47poBzOf086PLkJvvfRQ5hIhMpzcX7VlF+Tv0R16bg?= =?us-ascii?Q?GHzCLvq0/gHRLvOb0rK94CA2ummYLW4QQr11fqEB42ZmoIS/vT7nLfZCBKP6?= =?us-ascii?Q?bNb6cLcTQuG1IEmtZcNx462srUyqZq5pEuOtbCaDoJuh7+3DzkaOyyIZS3Kb?= =?us-ascii?Q?235fTJ6ALHnvFf3u72bnjKoO/8hrq50xMyCuOSiqE8WDqgWEpv2gz9sSMqXu?= =?us-ascii?Q?zqel5GG2TCRALWIDGDqlyE70wr/W05V2uG37F7w/xMbI9HxzmQMktU/aazSn?= =?us-ascii?Q?0kCnUyRgV6m/p5LbBuADxhUs2MC+T8Ow8XfMYHxwImky1JElXXOHoIDrmyvx?= =?us-ascii?Q?n4pw8Q1eYg5orXFTm6avKxb4KX0mxzf34g2zBevOI3hO2AnTXH2TcwPLV9q6?= =?us-ascii?Q?ZhkffIsKc88Ny9r4OZ3sNWQ+wIHNGxoLLLkKrxXEmSgwDTqxlgcQes64yMyZ?= =?us-ascii?Q?puBRKNy+WICLJpvvUqKpSe3Xx8rAEFViGPzfZSoZ62LwXkPJTxZkCRzvhRrC?= =?us-ascii?Q?jkCBcuFP09gl90WwJiRAW2nMMdYIHxpxdSJ4s/ecS3oDsVhJfTHf1IHZLU2/?= =?us-ascii?Q?gMaXJv9ni9lVURo1kCjin6lUjZjBkuITyR7LYWK7A0+r8/aMBJNQ/aaulWKB?= =?us-ascii?Q?bT3lgjJjud+NBdidOPK1ryjXjmqrofvihhgItMZCpHPjh7RCwS3917pE8lLM?= =?us-ascii?Q?WLwttgZaiar9L5jGGErl6Rzdm1u3QuH/iRI9+ZO4bJP/h3Ti9tFNBADvSTen?= =?us-ascii?Q?yvBD2mMaNoQzE65vU4osPACP7jWOa0hg+2ifDFg03wJ36PxIXn9mF3pepXKL?= =?us-ascii?Q?ytCYf/snEvh9HNJD55Et58dveKPSzSO8dHb7qRjgDbtdYh+fdbxPuJvHwoCw?= =?us-ascii?Q?HSiB516nTriR+S8mx9Iwsn8bzrYTkUAl72FzgbE44VYBu7fKpYhgbpAQeheT?= =?us-ascii?Q?5+Maym+lYBwLJRtmvvjgL/sGvfxdsNlEgR4MQ3yRnlma5X4uARkB1hAZFKsi?= =?us-ascii?Q?KzN4KtVjthzYqgYyvZ90VmGaFSXTN8j6ucnayRJmMtLhvoIpdKKrAnc2Un51?= =?us-ascii?Q?6jIy3faEKkBpFeGXk0p2I1mfeZcbtNkxmwh8zWcVcSJi3KNAD9rw008N6xUV?= =?us-ascii?Q?Bhuig6T8YTvLhkmIdcpZte0SlEfdOZOD/T/a+cE2?= X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 84119b26-1c3e-469d-ac40-08dc9a736905 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jul 2024 08:46:05.5322 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: E1kgVzoS6lowAWmEcHfN2rn34YWoi5y3uowseeJVne2cYk0TPp806UqtaQ7TgD+uM1NR3Kp7oeBa6MM+OXaptg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5790 Content-Type: text/plain; charset="utf-8" pmc - per memcg cache This patch add a feature pmc in each memcg unless root memcg. User can enable pmc in a target memcg, so all task in this memcg will share a cache pool, the alloc/free order 0 page will high priority turn in this cache pool. Signed-off-by: Huan Yang --- include/linux/memcontrol.h | 41 +++++++ include/linux/mmzone.h | 25 ++++ include/linux/swap.h | 1 + mm/memcontrol.c | 237 +++++++++++++++++++++++++++++++++++++ mm/page_alloc.c | 146 +++++++++++++++++++++++ 5 files changed, 450 insertions(+) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 8f332b4ae84c..5ec4c64bc515 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -130,6 +130,7 @@ struct mem_cgroup_per_node { bool on_tree; struct mem_cgroup *memcg; /* Back pointer, we cannot */ /* use container_of */ + struct mem_cgroup_per_node_cache *cachep; }; =20 struct mem_cgroup_threshold { @@ -336,6 +337,8 @@ struct mem_cgroup { struct lru_gen_mm_list mm_list; #endif =20 + bool cache_enabled; + struct mem_cgroup_per_node *nodeinfo[]; }; =20 @@ -557,6 +560,8 @@ static inline struct mem_cgroup *get_mem_cgroup_from_ob= jcg(struct obj_cgroup *ob return memcg; } =20 +extern struct static_key_true pmc_key; + #ifdef CONFIG_MEMCG_KMEM /* * folio_memcg_kmem - Check if the folio has the memcg_kmem flag set. @@ -1185,6 +1190,25 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_= t *pgdat, int order, gfp_t gfp_mask, unsigned long *total_scanned); =20 +static inline bool pmc_disabled(void) +{ + return static_branch_likely(&pmc_key); +} + +static inline bool mem_cgroup_cache_disabled(struct mem_cgroup *memcg) +{ + return !READ_ONCE(memcg->cache_enabled); +} + + +static inline struct mem_cgroup_per_node_cache * +mem_cgroup_get_node_cachep(struct mem_cgroup *memcg, int nid) +{ + struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; + + return nodeinfo->cachep; +} + #else /* CONFIG_MEMCG */ =20 #define MEM_CGROUP_ID_SHIFT 0 @@ -1648,6 +1672,23 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_= t *pgdat, int order, { return 0; } + +static inline bool pmc_disabled(void) +{ + return true; +} + +static inline bool mem_cgroup_cache_disabled(struct mem_cgroup *memcg) +{ + return true; +} + + +static inline struct mem_cgroup_per_node_cache * +mem_cgroup_get_node_cachep(struct mem_cgroup *memcg, int nid) +{ + return NULL; +} #endif /* CONFIG_MEMCG */ =20 /* diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index c11b7cde81ef..773b89e214c9 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -603,6 +603,31 @@ static inline void lru_gen_soft_reclaim(struct mem_cgr= oup *memcg, int nid) =20 #endif /* CONFIG_LRU_GEN */ =20 +struct mem_cgroup_zone_cache { + /* cache pages, current only hold order 0 */ + struct list_head pages; + spinlock_t pages_lock; + atomic_t nr_pages; + atomic_t nr_alloced; +}; + +struct mem_cgroup_per_node_cache { + /* per zone cache */ + struct mem_cgroup_zone_cache zone_cachep[MAX_NR_ZONES]; + struct mem_cgroup *memcg; + + /* max number to hold page, unit page, default 100MB */ +#define DEFAULT_PMC_HOLD_LIMIX ((100 << 20) >> PAGE_SHIFT) + unsigned int hold_limit; + +#define DEFAULT_PMC_GAP_WATERMARK ((50 << 20) >> PAGE_SHIFT) + /* + * Only when zone free pages above high+allow watermark, can hold cache, + * unit page, default 50MB + */ + unsigned int allow_watermark; +}; + struct lruvec { struct list_head lists[NR_LRU_LISTS]; /* per lruvec lru_lock for memcg */ diff --git a/include/linux/swap.h b/include/linux/swap.h index 11c53692f65f..d7b5e0a8317c 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -420,6 +420,7 @@ extern unsigned long mem_cgroup_shrink_node(struct mem_= cgroup *mem, extern unsigned long shrink_all_memory(unsigned long nr_pages); extern int vm_swappiness; long remove_mapping(struct address_space *mapping, struct folio *folio); +extern int mem_cgroup_release_cache(struct mem_cgroup_per_node_cache *fc); =20 #ifdef CONFIG_NUMA extern int node_reclaim_mode; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 1b3c3394a2ba..404fcb96bf68 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -95,6 +95,15 @@ static bool cgroup_memory_nokmem __ro_after_init; /* BPF memory accounting disabled? */ static bool cgroup_memory_nobpf __ro_after_init; =20 +/* + * How many memcg enabled cache? If none, static branch will enable + * so none task free/alloc will into PMC path. + * Else, hold/free cache in target memcg, disable static branch. + */ +static atomic_t pmc_nr_enabled; +DEFINE_STATIC_KEY_TRUE(pmc_key); + + #ifdef CONFIG_CGROUP_WRITEBACK static DECLARE_WAIT_QUEUE_HEAD(memcg_cgwb_frn_waitq); #endif @@ -5738,6 +5747,8 @@ static void mem_cgroup_css_released(struct cgroup_sub= sys_state *css) lru_gen_release_memcg(memcg); } =20 +static int __disable_mem_cgroup_cache(struct mem_cgroup *memcg); + static void mem_cgroup_css_free(struct cgroup_subsys_state *css) { struct mem_cgroup *memcg =3D mem_cgroup_from_css(css); @@ -5762,6 +5773,8 @@ static void mem_cgroup_css_free(struct cgroup_subsys_= state *css) cancel_work_sync(&memcg->high_work); mem_cgroup_remove_from_trees(memcg); free_shrinker_info(memcg); + if (READ_ONCE(memcg->cache_enabled)) + __disable_mem_cgroup_cache(memcg); mem_cgroup_free(memcg); } =20 @@ -7088,6 +7101,223 @@ static ssize_t memory_reclaim(struct kernfs_open_fi= le *of, char *buf, return nbytes; } =20 +static int __enable_mem_cgroup_cache(struct mem_cgroup *memcg) +{ + int nid, idx; + + if (!mem_cgroup_cache_disabled(memcg)) + return -EINVAL; + + for_each_node(nid) { + struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; + struct mem_cgroup_per_node_cache *p =3D kvzalloc_node( + sizeof(struct mem_cgroup_per_node_cache), + GFP_KERNEL, nid); + + if (unlikely(!p)) + goto fail; + + nodeinfo->cachep =3D p; + } + + for_each_node(nid) { + struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; + pg_data_t *pgdat =3D NODE_DATA(nid); + struct mem_cgroup_per_node_cache *p =3D nodeinfo->cachep; + + for (idx =3D 0; idx < MAX_NR_ZONES; idx++) { + struct zone *z =3D &pgdat->node_zones[idx]; + struct mem_cgroup_zone_cache *zc; + + if (!populated_zone(z)) + continue; + + zc =3D &p->zone_cachep[idx]; + + INIT_LIST_HEAD(&zc->pages); + spin_lock_init(&zc->pages_lock); + } + + p->memcg =3D memcg; + p->hold_limit =3D DEFAULT_PMC_HOLD_LIMIX; + p->allow_watermark =3D DEFAULT_PMC_GAP_WATERMARK; + + atomic_inc(&pmc_nr_enabled); + } + + if (static_branch_likely(&pmc_key)) + static_branch_disable(&pmc_key); + + //online + smp_wmb(); + WRITE_ONCE(memcg->cache_enabled, true); + atomic_inc(&pmc_nr_enabled); + + return 0; + +fail: + for_each_node(nid) { + struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; + + if (nodeinfo->cachep) { + kvfree(nodeinfo->cachep); + nodeinfo->cachep =3D NULL; + } + } + + return -ENOMEM; +} + +static int __disable_mem_cgroup_cache(struct mem_cgroup *memcg) +{ + int nid; + + if (unlikely(mem_cgroup_cache_disabled(memcg))) + return -EINVAL; + + //offline + WRITE_ONCE(memcg->cache_enabled, false); + + for_each_node(nid) { + struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; + struct mem_cgroup_per_node_cache *p; + + p =3D nodeinfo->cachep; + + mem_cgroup_release_cache(p); + + kfree(p); + } + + if (atomic_dec_and_test(&pmc_nr_enabled)) + static_branch_enable(&pmc_key); + + return 0; +} + +static int mem_cgroup_cache_show(struct seq_file *m, void *v) +{ + struct mem_cgroup *memcg; + int nid; + + if (static_branch_likely(&pmc_key)) + return -EINVAL; + + memcg =3D mem_cgroup_from_seq(m); + if (!READ_ONCE(memcg->cache_enabled)) + return -EINVAL; + + seq_printf(m, "%4s %16s %16s\n", "NODE", "WATERMARK", "HOLD_LIMIT"); + for_each_online_node(nid) { + struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; + struct mem_cgroup_per_node_cache *p; + + p =3D nodeinfo->cachep; + if (!p) + continue; + + seq_printf(m, "%4d %14uKB %14uKB\n", nid, + (READ_ONCE(p->allow_watermark) << (PAGE_SHIFT - 10)), + (READ_ONCE(p->hold_limit) << (PAGE_SHIFT - 10))); + } + + seq_puts(m, "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D\n"); + seq_printf(m, "%4s %16s %16s %16s\n", "NODE", "ZONE", "CACHE", "HIT"); + + for_each_online_node(nid) { + struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; + struct mem_cgroup_per_node_cache *p; + pg_data_t *pgdat =3D NODE_DATA(nid); + int idx; + + p =3D nodeinfo->cachep; + if (!p) + continue; + + for (idx =3D 0; idx < MAX_NR_ZONES; idx++) { + struct mem_cgroup_zone_cache *zc; + struct zone *z =3D &pgdat->node_zones[idx]; + + if (!populated_zone(z)) + continue; + + zc =3D &p->zone_cachep[idx]; + seq_printf(m, "%4d %16s %14dKB %14dKB\n", nid, z->name, + (atomic_read(&zc->nr_pages) + << (PAGE_SHIFT - 10)), + (atomic_read(&zc->nr_alloced) + << (PAGE_SHIFT - 10))); + } + } + + return 0; +} + +enum { + OPT_CTRL_ENABLE, + OPT_CTRL_ERR, + OPR_CTRL_NR =3D OPT_CTRL_ERR, +}; + +static const match_table_t ctrl_tokens =3D { + { OPT_CTRL_ENABLE, "enable=3D%s" }, + { OPT_CTRL_ERR, NULL } }; + +/** + * This function can control target memcg's cache. include enable\keys se= t. + * To enable\disable this cache, by `echo enable=3D[y|n] > memory.cace` + * in target memcg. + */ +static ssize_t mem_cgroup_cache_control(struct kernfs_open_file *of, char = *buf, + size_t nbytes, loff_t off) +{ + bool enable; + bool opt_enable_set =3D false; + int err =3D 0; + char *sub; + struct mem_cgroup *memcg =3D mem_cgroup_from_css(of_css(of)); + + buf =3D strstrip(buf); + if (!strlen(buf)) + return -EINVAL; + + while ((sub =3D strsep(&buf, " ")) !=3D NULL) { + int token; + substring_t args[MAX_OPT_ARGS]; + char tbuf[256]; + + sub =3D strstrip(sub); + + token =3D match_token(sub, ctrl_tokens, args); + switch (token) { + case OPT_CTRL_ENABLE: + if (match_strlcpy(tbuf, &args[0], sizeof(tbuf)) >=3D + sizeof(tbuf)) + return -EINVAL; + + err =3D kstrtobool(tbuf, &enable); + if (err) + return -EINVAL; + opt_enable_set =3D true; + break; + case OPT_CTRL_ERR: + default: + return -EINVAL; + } + } + + if (opt_enable_set) { + if (enable) { + __enable_mem_cgroup_cache(memcg); + } else { + __disable_mem_cgroup_cache(memcg); + return nbytes; + } + } + + return err ? err : nbytes; +} + static struct cftype memory_files[] =3D { { .name =3D "current", @@ -7156,6 +7386,13 @@ static struct cftype memory_files[] =3D { .flags =3D CFTYPE_NS_DELEGATABLE, .write =3D memory_reclaim, }, + /* free cache field */ + { + .name =3D "cache", + .flags =3D CFTYPE_NOT_ON_ROOT, + .write =3D mem_cgroup_cache_control, + .seq_show =3D mem_cgroup_cache_show, + }, { } /* terminate */ }; =20 diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1beb56f75319..54c4d00c2506 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -530,6 +530,14 @@ static inline int pindex_to_order(unsigned int pindex) return order; } =20 +/** + * Per memcg cache currently only allow order 0. + */ +static inline bool pmc_allow_order(unsigned int order) +{ + return !order; +} + static inline bool pcp_allowed_order(unsigned int order) { if (order <=3D PAGE_ALLOC_COSTLY_ORDER) @@ -1271,6 +1279,43 @@ void __free_pages_core(struct page *page, unsigned i= nt order) __free_pages_ok(page, order, FPI_TO_TAIL); } =20 +int mem_cgroup_release_cache(struct mem_cgroup_per_node_cache *nodep) +{ + LIST_HEAD(temp_list); + int zid, num =3D 0; + + for (zid =3D 0; zid < MAX_NR_ZONES; ++zid) { + struct mem_cgroup_zone_cache *zc =3D &nodep->zone_cachep[zid]; + int i =3D 0; + + if (!atomic_read(&zc->nr_pages)) + continue; + + spin_lock(&zc->pages_lock); + list_splice_init(&zc->pages, &temp_list); + spin_unlock(&zc->pages_lock); + + while (!list_empty(&temp_list)) { + struct page *page =3D + list_first_entry(&temp_list, struct page, lru); + struct zone *zone =3D page_zone(page); + unsigned long pfn =3D page_to_pfn(page); + + list_del(&page->lru); + + + // is good to put into pcp? + free_one_page(zone, page, pfn, 0, FPI_NONE); + ++i; + } + + num +=3D i; + atomic_sub(i, &zc->nr_pages); + } + + return num; +} + /* * Check that the whole (or subset of) a pageblock given by the interval of * [start_pfn, end_pfn) is valid and within the same zone, before scanning= it @@ -2603,6 +2648,41 @@ static void free_unref_page_commit(struct zone *zone= , struct per_cpu_pages *pcp, } } =20 +static bool free_unref_page_to_pmc(struct page *page, struct zone *zone, + int order) +{ + struct mem_cgroup *memcg; + struct mem_cgroup_per_node_cache *cachp; + struct mem_cgroup_zone_cache *zc; + unsigned long flags; + bool ret =3D false; + + if (pmc_disabled()) + return false; + + memcg =3D get_mem_cgroup_from_current(); + if (!memcg || mem_cgroup_is_root(memcg) || + mem_cgroup_cache_disabled(memcg)) + goto out; + + cachp =3D mem_cgroup_get_node_cachep(memcg, page_to_nid(page)); + zc =3D &cachp->zone_cachep[page_zonenum(page)]; + + if (high_wmark_pages(zone) + READ_ONCE(cachp->allow_watermark) >=3D + zone_page_state(zone, NR_FREE_PAGES)) + goto out; + + spin_lock_irqsave(&zc->pages_lock, flags); + list_add(&page->lru, &zc->pages); + spin_unlock_irqrestore(&zc->pages_lock, flags); + atomic_inc(&zc->nr_pages); + + ret =3D true; +out: + mem_cgroup_put(memcg); + return ret; +} + /* * Free a pcp page */ @@ -2634,6 +2714,17 @@ void free_unref_page(struct page *page, unsigned int= order) } =20 zone =3D page_zone(page); + + /** + * This function can cache release page before free into pcp if current + * memcg enabled cache feature. + * Compared to PCP, PMC is unique, only processes in PMC can access it. + * So, if the conditions are met, it should be prioritized to be + * released to PMC before being released to the public CPU cache. + */ + if (pmc_allow_order(order) && free_unref_page_to_pmc(page, zone, order)) + return; + pcp_trylock_prepare(UP_flags); pcp =3D pcp_spin_trylock(zone->per_cpu_pageset); if (pcp) { @@ -3012,6 +3103,49 @@ static struct page *rmqueue_pcplist(struct zone *pre= ferred_zone, return page; } =20 +static struct page *rmqueue_mem_cgroup_cache(struct zone *preferred_zone, + struct zone *zone, + unsigned int order, + int migratetype) +{ + struct mem_cgroup *memcg; + struct mem_cgroup_per_node_cache *cachp; + struct mem_cgroup_zone_cache *zc; + unsigned long flags; + int nid =3D zone->zone_pgdat->node_id; + struct page *page =3D NULL; + + if (pmc_disabled()) + return NULL; + + memcg =3D get_mem_cgroup_from_current(); + if (!memcg || mem_cgroup_is_root(memcg) || + mem_cgroup_cache_disabled(memcg)) + goto out; + + cachp =3D mem_cgroup_get_node_cachep(memcg, nid); + + zc =3D &cachp->zone_cachep[zone_idx(zone)]; + if (!atomic_read(&zc->nr_pages)) + goto out; + + spin_lock_irqsave(&zc->pages_lock, flags); + if (list_empty(&zc->pages)) { + spin_unlock_irqrestore(&zc->pages_lock, flags); + goto out; + } + page =3D list_first_entry(&zc->pages, struct page, lru); + list_del(&page->lru); + spin_unlock_irqrestore(&zc->pages_lock, flags); + + atomic_dec(&zc->nr_pages); + atomic_inc(&zc->nr_alloced); + +out: + mem_cgroup_put(memcg); + return page; +} + /* * Allocate a page from the given zone. * Use pcplists for THP or "cheap" high-order allocations. @@ -3038,6 +3172,18 @@ struct page *rmqueue(struct zone *preferred_zone, */ WARN_ON_ONCE((gfp_flags & __GFP_NOFAIL) && (order > 1)); =20 + /* + * Before disturb public pcp or buddy, current may in a memcg + * which already enabled cache feature. + * If that's true, first get page from private pool can boost alloc. + */ + if (pmc_allow_order(order)) { + page =3D rmqueue_mem_cgroup_cache(preferred_zone, zone, order, + migratetype); + if (page) + goto out; + } + if (likely(pcp_allowed_order(order))) { page =3D rmqueue_pcplist(preferred_zone, zone, order, migratetype, alloc_flags); --=20 2.45.2 From nobody Fri Dec 19 09:13:32 2025 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2057.outbound.protection.outlook.com [40.107.117.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 138591514F3; Tue, 2 Jul 2024 08:46:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.117.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719909975; cv=fail; b=o7TXt5K4h0oN5unmm4INWOaN9ZuYLE3DxiRYzM6ott541bFUW+RCtdM5FvK1Dne6Fm+HQuTjGY21IAJi4jxhXVQa7lw5GkuBdmXEq+l9xmZHqHXKFJGL3179ylCxETWB6C74sABJkUSWnZ8vyGTGGE+wvTJFbMZ3P1LbNfAOC6A= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719909975; c=relaxed/simple; bh=7hOPhFx3f/nMSHHYU+/vf5XJQiMJcB3McLZO5wd0SUk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=LVwcrKVTqlKUheUGalSf7GlsjbA2MTgkH5DSBRY/eEEt05yvHZy/t2qzgE/MomNWnLPjY8yfdFL1CdzgzupFjTREiIrQB/TCbrYmKdJecrdJ4qWuWL/6KLfqYFdu6srENff0wqAyMFI8IO5QEZdiEaR2q5vVqH2pvEv9PJwu22E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=qTZi65VU; arc=fail smtp.client-ip=40.107.117.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="qTZi65VU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iRFhxV3/yKnBSTgIU5u9jy0ZCM28mdMIAlHoeieG/eFg6NLSUQXT3pZzpQAlHY7U2ccj0Z2X7boPS3Jf42v9+vA6e2Uw/ybaLAdI04QwJFRJBNmru4aKZij6wQ2u9EqjI0Jj49DokNl/G0jd6np/g299T/QrMBwDpPSXBHMtYAaw4WRmrK4ATSkvyanvjhFhwEFb14bEaabKdh+mJW1cU8IGmdl3YhV/MDYgC8ML9ZIb0bULSoFmW32RAhvPrWs1l1xBmVRUIKIDgG4iyI1SuHECnal+L1AoIQWn7G7H45acgSF8qOa6TnJHuK+WQGuTwrMtHCXUQuh+vtuV3TgM+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2MV7Q2+/9roovDyuynNKu6Tlm1HAL+DluQZMl+IyKQU=; b=i0h9VaPXUyBTcOb9X4EXP6HaKFAtkPo2UXDaVSZEZBdd8nuTwAsntkM8zXVXL2t+4KJ227PUHFKfBnBaGPcihgJZZ28LzmHS2Qoyrw+mN07MOUL+NTn/mfQ7oQhfyfP2sZdqEKc7NUd6traLZmvaF3fvC2ngZlWoiiCOnk4vFg26fL1yx3vXKkyxeMQ+liECpID0iXARg77u/vZesCkJKjNc/h9NH25zp+2Bzajdy3faK/J2PpqrmxRRZ2OX7BCrEny8ejl6JNbgbzKIdZzNXbHtoF2OWjz5AeOSzEzZ8NyMvq84xfJxffm+K2JGQCXd5kwJKQ7wtIiYELYJV/7TOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2MV7Q2+/9roovDyuynNKu6Tlm1HAL+DluQZMl+IyKQU=; b=qTZi65VUKb02QN2AQ5YlH4npS6tSgw//8x+xsr3mqdz6FHeEqqgSyqKMhDT6BeAvSnvQFAqM6lgOy7ZY9NX8G7LwgOnoyxl2IhDIKFbonz2pkX0dFC3s56cSnlRsBxpTcNpkNDAttqyGrnkNt0lzJkdt/aSUQVeGWBepvH4SZ//Dla+jqJ9TeNxLJb70Yfxt4sGTqKTt0ED1mat+e3ylzwO1XDiakBCpRPXEd8AonMBV3u7ETUqkqzyY3n0lG1cFRfssAngV82QF/3cRbHwjJfxaG6cDpn2UXw2O+wsc4CDFy1RhG6xkp2+gVklN+OSk1CoQI3/ZBfqmehZGBoYV/Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEYPR06MB5790.apcprd06.prod.outlook.com (2603:1096:101:b9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.23; Tue, 2 Jul 2024 08:46:10 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7719.028; Tue, 2 Jul 2024 08:46:10 +0000 From: Huan Yang To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , Ryan Roberts , Chris Li , Dan Schatzberg , Huan Yang , Kairui Song , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christian Brauner Cc: opensource.kernel@vivo.com Subject: [RFC PATCH 2/4] mm: memcg: pmc support change attribute Date: Tue, 2 Jul 2024 16:44:05 +0800 Message-ID: <20240702084423.1717904-3-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240702084423.1717904-1-link@vivo.com> References: <20240702084423.1717904-1-link@vivo.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR01CA0148.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::28) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEYPR06MB5790:EE_ X-MS-Office365-Filtering-Correlation-Id: 2a2b0f9d-f637-4db7-7d21-08dc9a736c00 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|52116014|7416014|376014|366016|921020|38350700014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?49AJizRIli6T8+XohSyCV4OOS/8am/tL8wB9DywapHJf/AkidjaKey7NaYYc?= =?us-ascii?Q?czjOUUBTzfcH1oTB7PlJkowYz/9Ff4UAjmdztuz9BUMcfOCrrjF+CaV0YFuM?= =?us-ascii?Q?mMjLp0qIsEb3/sCBqpETEfNpJOVqgR0/Grry21JRoEqaFrPG0EDZwnL4UGQM?= =?us-ascii?Q?VpaVwXF3np9oBBP8DwXeqLzeGLC+87+CRxXiuZbq1RmKXhZDM0fyz7mcdZHC?= =?us-ascii?Q?zGheHC1jsu8+0F0su5Lx+SP4SocdaqyVLm/ePC5B7YS15CAli1ND07eALBea?= =?us-ascii?Q?yVWdhsKzvZWbyzdY5hs+fDh1OyXrszlQYbpyRfJFkAYLheQ9SA2SNOVQ1tZs?= =?us-ascii?Q?+ISq/4882UBqJdZL0mWRTxdrVE6AnYbrkEasDU4qCCJa2vNdE8Ky9eOAzQhm?= =?us-ascii?Q?0KZ5sufcMS/d8Pwc/E3MWdG0nyQNNVoWcbb87B6C1qfSZ4YrKsXq0e8CVlLC?= =?us-ascii?Q?SHGin/3oQ1AirPNzc7z4Jq+M/qTh6EzzMo0/+EsBUWjJXHJF7uvKLHqGAbXM?= =?us-ascii?Q?wBFlC7VS4BwWHjp2Ya3FHZTFzQzQxizuIiuftj/2T1mZGfEhSzdJpfFQRAlE?= =?us-ascii?Q?aGDXk5kcq+IC7btwXdPqP51yILqkGmz/L+yXui35Flp0EEllvg+veXMrQRBg?= =?us-ascii?Q?Yf5lBYLdQyUSjXV96pgLq5JMCm1ZWsMZ0JApCByrjEuBOxYgDreUjNzeSdJP?= =?us-ascii?Q?nMo4/zUDvnGNzBfp2MRk2dbEG1tFKoXlNXhjD9w73ddtY0QYFiyewp2MnvWO?= =?us-ascii?Q?82QeJr3nHK5ww6uOgMYBMdPm64vK8WfTcl8DkQNePT0k5mikiHBwje3SCfB+?= =?us-ascii?Q?xvajcDFSC3z4vhGRwPLt3doUoXVvebUG5H2A60Xtvfj2pX9T0CCdG+dH4j/N?= =?us-ascii?Q?Akoltub8BAH9R/vpdjhMTqLjpCFTfHR2LeoTHhfsAAYvFfP7+kOozNcTJwQG?= =?us-ascii?Q?OzRUKaTUMXz8hNzpUCeuCvG/iT/dErlo+ML+xkKdAPLgtKWqhbdHB6UwA30a?= =?us-ascii?Q?Qq73yT7edJKu0SB3SedH2qkKJFoyQ9+1lijS/5d7tym6JKRUiKuyf4yLBWL2?= =?us-ascii?Q?SKOVOIOVgYa8jnVM5tmdOi9Lq9eQ6wPMjEjCXHJLtZfooOfzubvedj8Ld6mv?= =?us-ascii?Q?8XIet1zDuW5+fNjQ31um0cMbQXNRTl+S0kKmxPfsgwtq+n0fB2vEh5ZEZx68?= =?us-ascii?Q?2XBlBoLb/pxpBr3BYb1QZtWaVe0GoqdFQQLh/gRfqXGQrzqZWLM1oZ00RrNB?= =?us-ascii?Q?g1GuXKwPIX+4weoCD2qY4pOBo2ZcjH6veJFOKggC1xwH0RtGXFf1IXoSZMGy?= =?us-ascii?Q?IZQE/jeUeOUySMQDmHm87k6x6hynoeOycc0qs0i62fcz7uKEbjVCYNXQivj5?= =?us-ascii?Q?/Ouxcg0vQCcaGAMvuMdpHZpRveXPxEhy811/C9RTDrApHcqSb11IkzJR95Rf?= =?us-ascii?Q?tcJ7iPQeSzU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(52116014)(7416014)(376014)(366016)(921020)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?x6kusNf0ShESwMV1tB9QubYUsfGsHTzqJOKZFnzJmUmaqMNTMklNeVr0WJ8B?= =?us-ascii?Q?OB4oyJGmfCgs0c59lPtrYHQqPmjnEgLd/a/EhJiK+8e0NQC9mOEXlmN9lAhi?= =?us-ascii?Q?hpYw+NyTMlNjhvbSrqfTEwfSC3mNka4v83ZGumy4qL6kVnXJYhBnWmuSNiik?= =?us-ascii?Q?4RKuXODYMQUxZB8W4oMu9rOb4J7mi4Rp8Pqj3krfDBT/tepTCdHEKNV58UM3?= =?us-ascii?Q?CNz4dxbI6BqhUjhGimm3ZCRfq/WmzxNQFcMfEutXNo7U6OIkJVfH4mfTFRZb?= =?us-ascii?Q?Cbzdu4+/yvu2tYHLHUy8bDsTo8bJ/CqidfwKtGWPN0g4IDHop1sfiaGJJc0N?= =?us-ascii?Q?CkkwS/M6+gWJU47D5R+PWdMzAyJgTIqFmqXUkyqaSHGiPOrWKhHFtABDFu6r?= =?us-ascii?Q?FuGd7iv/NRDVCBnMcJSAr+396Mle//z0Bgu8hIxj7d5eZhDoXVhmbm9oDN0k?= =?us-ascii?Q?A0S+hOQ+J5JGtqoxfd3VEgXT+6kdgkPDEKZLawF9mPw5wbhOd8FlCjHYeYW4?= =?us-ascii?Q?wuDcEzRDamGANGW/4stFjaoZb2dk8oi5g7HblcfZNiaFqfRRlh3iKkebuE8Z?= =?us-ascii?Q?ptRJY3Uk7pAWdxLRPCITN7owK9kuCAWQLE0sNCTMy4j+kdp7aZ8rXSNmaDSQ?= =?us-ascii?Q?WpNZbJ9UACrukz1o7HhHmvwGZDr9xrcRukhjNGaQtkbDHP1NkGGJt379ymGz?= =?us-ascii?Q?nesXx8nRKTLR0Th1rjLGww9jpS6sUHvu4NS4pP+TSSIC1TVJcGhsb4HmNdcY?= =?us-ascii?Q?pW+DjyoAYXw9oexlbAi4X3zJBrMmZWzAOLqgnE8ncXNK9xxy2w5sxZ7EEqSF?= =?us-ascii?Q?XLEDvdcUUlo73fBnCzmhRgF3DZC/17J+EUafdFg1bfk+QJcjHtzJo+n1QSFn?= =?us-ascii?Q?YhnL3FbJqxr7YzXpxoteiLRdtlVfV5MmkYkjUFppOleWzB/W8TR16a0lgy00?= =?us-ascii?Q?NcNp0dYvM6ME0twpLtd9qcSS+EJxGyKJJSicsnC7q1ByD2Nv5ZMd/iYCjXxJ?= =?us-ascii?Q?rkm/Pk4q9Mgk7O9GtJYRTa/U0n5ehADKAjHLxDYgED8mJzd76xq67SC51c6c?= =?us-ascii?Q?qXY6fb2T8+ylAIj/khtYPr23Ei2vQeAeuQYC79lv9sVoBHt89RbfXqsicEBZ?= =?us-ascii?Q?+Lk9I15N/Z8iqzjrcUXdql/Ai/lBZ79GVC6zbPNry1o8/J7D765vlK3ZbvYG?= =?us-ascii?Q?bw1nQIsbqxGGOiTfQ/mi2JW7Y5oJjcyFgS0XFu9vBztjrMHiOsEcg5iiYvVV?= =?us-ascii?Q?JHmNALDg2IIl0aDd5/LhXHnI4HbgLSSr50iJRHK6TmVcX0uADEihtdxqYPpm?= =?us-ascii?Q?H46sAf9ZmkJakYUQT28Uv7zXB8Uyoy1/nX+qlbkfOTsHgy4tsWIZiGbh1fi9?= =?us-ascii?Q?pq1Z4WjkSYjk7/5U8JfFxt6g5Erc6Rz6M1WTomY+v5Xch1v6/kvmyDDIukpt?= =?us-ascii?Q?MO6CVwDWghjrYdvHG+I2GI3M5utcQWbPy9kajfO7FmJfWXa6UQgFBLGHyERY?= =?us-ascii?Q?dekKgIHWsy2xbs9cmCRAomShpfiwZrGHHexPcumIPi4sZHKIC5CgkiO4RFWt?= =?us-ascii?Q?zfQWfJ/1mDrQ59VPjfG3TU25Q0jyWxHE2uBLyEep?= X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2a2b0f9d-f637-4db7-7d21-08dc9a736c00 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jul 2024 08:46:10.5289 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6KY6lu++Iu/ILojwIxMOEIrWhiVYpDb2UC6OKonul2hy/y4jSDyVr4WZSMr3g10NKUvkiaYmE33qbMpwZAkQJw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5790 Content-Type: text/plain; charset="utf-8" pmc have below attribute: watermark: only when zone free pages above high+watermark can cache pages limit: max memory it can cached. This patch let user can change each attribute by `memory.cache`. To change attribute, can type `keys=3Dattribute=3Dvaule` into memcg's `memory.cache` if it enabled cache. For example: echo keys=3Dwatermark=3D157286400,limit=3D209715200 > memory.cache This changed memcg's only when free pages above high+150MB can cache pages, and can cache up to a maximum of 200MB . Signed-off-by: Huan Yang --- mm/memcontrol.c | 152 +++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 151 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 404fcb96bf68..9db5bbe63b34 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -7253,29 +7253,168 @@ static int mem_cgroup_cache_show(struct seq_file *= m, void *v) return 0; } =20 +#define STATUS_UNSET_DEFAULT_VALUE -1 +enum { + OPT_KEY_NID, + OPT_KEY_WATERMARK, + OPT_KEY_HOLD_LIMIT, + OPT_KEY_ERR, + NR_PMC_KEY_OPTS =3D OPT_KEY_ERR +}; + +static const match_table_t fc_tokens =3D { + { OPT_KEY_NID, "nid=3D%d" }, + { OPT_KEY_WATERMARK, "watermark=3D%u" }, + { OPT_KEY_HOLD_LIMIT, "limit=3D%u" }, + { OPT_KEY_ERR, NULL} +}; + +static void +__apply_status_for_mem_cgroup_cache(struct mem_cgroup_per_node_cache *p, + unsigned int opts[]) +{ + int i; + + for (i =3D OPT_KEY_WATERMARK; i < NR_PMC_KEY_OPTS; ++i) { + switch (i) { + case OPT_KEY_WATERMARK: + if (opts[OPT_KEY_WATERMARK] !=3D + STATUS_UNSET_DEFAULT_VALUE) + p->allow_watermark =3D opts[OPT_KEY_WATERMARK]; + break; + case OPT_KEY_HOLD_LIMIT: + if (opts[OPT_KEY_HOLD_LIMIT] !=3D + STATUS_UNSET_DEFAULT_VALUE) + p->hold_limit =3D opts[OPT_KEY_HOLD_LIMIT]; + break; + default: + break; + } + } +} + +static __always_inline int +mem_cgroup_apply_cache_status(struct mem_cgroup *memcg, + unsigned int opts[]) +{ + struct mem_cgroup_per_node_cache *p; + unsigned int nid =3D opts[OPT_KEY_NID]; + + if (nid !=3D STATUS_UNSET_DEFAULT_VALUE) { + p =3D memcg->nodeinfo[nid]->cachep; + if (unlikely(!p)) + return -EINVAL; + __apply_status_for_mem_cgroup_cache(p, opts); + return 0; + } + + for_each_node(nid) { + p =3D memcg->nodeinfo[nid]->cachep; + if (!p) + continue; + __apply_status_for_mem_cgroup_cache(p, opts); + } + + return 0; +} + +/** + * Support nid=3Dx,watermark=3Dbytes,limit=3Dbytes args + */ +static int __mem_cgroup_cache_control_key(char *buf, + struct mem_cgroup *memcg) +{ + char *p; + unsigned int opts[NR_PMC_KEY_OPTS]; + + memset(opts, STATUS_UNSET_DEFAULT_VALUE, sizeof(opts)); + + if (!READ_ONCE(memcg->cache_enabled)) + return -EINVAL; + + if (!buf) + return -EINVAL; + + while ((p =3D strsep(&buf, ",")) !=3D NULL) { + int token; + u32 v; + substring_t args[MAX_OPT_ARGS]; + + p =3D strstrip(p); + + if (!*p) + continue; + + token =3D match_token(p, fc_tokens, args); + switch (token) { + case OPT_KEY_NID: + if (match_uint(&args[0], &v) || v >=3D MAX_NUMNODES) + return -EINVAL; + opts[OPT_KEY_NID] =3D v; + break; + case OPT_KEY_WATERMARK: +#define MIN_WATERMARK_LIMIT ((10 << 20) >> PAGE_SHIFT) + if (match_uint(&args[0], &v)) + return -EINVAL; + v >>=3D PAGE_SHIFT; + if (v < MIN_WATERMARK_LIMIT) + return -EINVAL; + opts[OPT_KEY_WATERMARK] =3D v; + break; + case OPT_KEY_HOLD_LIMIT: + if (match_uint(&args[0], &v)) + return -EINVAL; + v >>=3D PAGE_SHIFT; +#define MAX_CACHE_LIMIT_NR ((500 << 20) >> PAGE_SHIFT) + if (v > MAX_CACHE_LIMIT_NR) + return -EINVAL; + opts[OPT_KEY_HOLD_LIMIT] =3D v; + break; + case OPT_KEY_ERR: + default: + break; + } + } + + if (mem_cgroup_apply_cache_status(memcg, opts)) + return -EINVAL; + + return 0; +} + enum { OPT_CTRL_ENABLE, + OPT_CTRL_KEYS, OPT_CTRL_ERR, OPR_CTRL_NR =3D OPT_CTRL_ERR, }; =20 static const match_table_t ctrl_tokens =3D { { OPT_CTRL_ENABLE, "enable=3D%s" }, + { OPT_CTRL_KEYS, "keys=3D%s" }, { OPT_CTRL_ERR, NULL } }; =20 /** * This function can control target memcg's cache. include enable\keys se= t. * To enable\disable this cache, by `echo enable=3D[y|n] > memory.cace` * in target memcg. + * To set keys, by `echo keys=3D[key=3Dargs;..] > memory.cache`, current s= upport keys: + * 1. nid=3Dx, if input, will only change target NODE's cache status. El= se, all. + * 2. watermark=3Dbytes, change cache hold behavior, only zone free page= s above + * high watermark+watermark, can hold. + * 3. limit=3Dbytes, change max pages can cache. Max can change to 500MB + * Enable and keys can both input, split by space, so can set args after e= nable, + * if cache not enable, can't set keys. */ static ssize_t mem_cgroup_cache_control(struct kernfs_open_file *of, char = *buf, size_t nbytes, loff_t off) { bool enable; - bool opt_enable_set =3D false; + bool opt_enable_set =3D false, opt_key_set =3D false; int err =3D 0; char *sub; struct mem_cgroup *memcg =3D mem_cgroup_from_css(of_css(of)); + char keybuf[256]; =20 buf =3D strstrip(buf); if (!strlen(buf)) @@ -7300,6 +7439,14 @@ static ssize_t mem_cgroup_cache_control(struct kernf= s_open_file *of, char *buf, return -EINVAL; opt_enable_set =3D true; break; + case OPT_CTRL_KEYS: + if (match_strlcpy(tbuf, &args[0], sizeof(tbuf)) >=3D + sizeof(tbuf)) + return -EINVAL; + + memcpy(keybuf, tbuf, sizeof(keybuf)); + opt_key_set =3D true; + break; case OPT_CTRL_ERR: default: return -EINVAL; @@ -7315,6 +7462,9 @@ static ssize_t mem_cgroup_cache_control(struct kernfs= _open_file *of, char *buf, } } =20 + if (opt_key_set) + err =3D __mem_cgroup_cache_control_key(keybuf, memcg); + return err ? err : nbytes; } =20 --=20 2.45.2 From nobody Fri Dec 19 09:13:32 2025 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2080.outbound.protection.outlook.com [40.107.117.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19C8F152DF1; Tue, 2 Jul 2024 08:46:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.117.80 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719909979; cv=fail; b=nv82G5ej2kcaXI4uoksgAyaG4HPD+8Tsl0nvUyytZlxMB4mnXU6DiKTcZXK8tJTLccW7CjRc4j7+rxL9zaHGx333OL1lFzRQgmiWQbwa4iQ9bFpi/SNBsR8Pkkj4H+FTboyih9CRGIMSXXSCRUwLhveuLU8hmGvROr6juyUxWRc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719909979; c=relaxed/simple; bh=umZjQkmPExVF9mjm+W4YEm9OcS17G4xwfLsC3/18sgk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=sR3+1ZKBjr7ha/dllq59eFHaeMdYc4rUWq0OIM0I9yyGoETmJ8imk/G/Rz3rjh5Xskd1ShpMwhTXtdH+3ICY4+3kzcVgZQzma/UloKV4/qlQWDn7jYoFl3sSiveRdgOHvrbxbFRv+o6ZEOhGuD7zvngIvUD2FfhgPZ+kjmGpWFk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=EXvsFRi1; arc=fail smtp.client-ip=40.107.117.80 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="EXvsFRi1" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NG0QT6WaQyDMRNhuKpwKm4IjyN6ujj4mv9OjwCybe07stJvxBXnODj5XQo4R6+aGR1LFplJoRWF1eSM9nuazxdi0oO8QlzchB14CR7t2xPm8Mp3d5V0rZKwLRQRQ+bBvt/LgrMcNN+9nBYscp92Z14c43nA2ch78kyODkZaqEt97YZ8ELfc3yPESpAglE55NNWGg4L0XfFPH2fJJKJSeZ8UzX956EDiqglbXuS09osPj7Av5qsrAPPJhhHTJz5KQseffF2SRUvbCu1IUxwp6yKb7ejrZoex+9guTWCpCKr0Xg/SflpwsWufTnNppbV5w+YUYQ7jZ75vxIeNOglCbiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JHncJhtTb2CVCgR6KyprnXFeC/NMYgOPEuy1IeHa92A=; b=gWM7HRpHHNti1miCfZWpwf4Er+XP5/Z0vnPPDJqfzCyDiBeGajzkFZS+FanDXCUTBSmZR5WacOMC8LVwcX+2AYBVlTbTC2kiIvEK9l3hT1xIGQo4nb78M92FLRVBcJRsvFtMCRaZajwQYFUVI49Dcb6IjMxTQA0ItjGyqqh1C8nCxkRL/7eou/icHfrOYugVadL3udXI6kEacox4OFTDcRM7orZt6dXhvEf4nUBuMQqRBBlCpQxJEW5maMVDGvJxv9+X0gcmfd86dPSsRiSP+MNeoBKNxX3ZTPLEaj4LbtFJDi0EPKmbRrGP18f3IUGQ9/zV6+XcSbFp/Il6K6P8Ig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JHncJhtTb2CVCgR6KyprnXFeC/NMYgOPEuy1IeHa92A=; b=EXvsFRi1MNmtpSI2shGOY8GWVHvzvVZNozT6or9XUJHiM5GIYo/fKOl8hRUt7nW9s/hJWbln3yw+OikN5mC7fp9Q0Ttbm//1MIIQJ8qzRB9V0omtK5CEVi/TumwHze603NHGVAqB34zT41OlcZ4qog+xAJkN3tI8AFhNDedKbXo/8WELMroCD+hP2emNMVfGbT4SNBCsIDJ8E7jE+eyR6Fqpuwx7kwgTRe23qn4iGoT8vs6Z/6oBcDm/PXWJJESM9Vrm24EBY3WwmG71+qafy7n/WzZVVZ0kln54An/ha8rR6aJMybtp7u8MxkgldIRTkeTsVfUvp68wuuaMtbHSMA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEYPR06MB5790.apcprd06.prod.outlook.com (2603:1096:101:b9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.23; Tue, 2 Jul 2024 08:46:15 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7719.028; Tue, 2 Jul 2024 08:46:15 +0000 From: Huan Yang To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , Ryan Roberts , Chris Li , Dan Schatzberg , Huan Yang , Kairui Song , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christian Brauner Cc: opensource.kernel@vivo.com Subject: [RFC PATCH 3/4] mm: memcg: pmc: support reaper Date: Tue, 2 Jul 2024 16:44:06 +0800 Message-ID: <20240702084423.1717904-4-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240702084423.1717904-1-link@vivo.com> References: <20240702084423.1717904-1-link@vivo.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR01CA0148.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::28) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEYPR06MB5790:EE_ X-MS-Office365-Filtering-Correlation-Id: e9c9a2ec-00b6-4829-04f8-08dc9a736ef7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|52116014|7416014|376014|366016|921020|38350700014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?R+ALswNfFrf7uwYXx6CNHD6PVJXXWXMzyIzOMBrmf6WIpIEbzYtLvmBLFD9v?= =?us-ascii?Q?+0eCUNthFpaiSC9Gigy4S+GSUyxKpyAiXRw28OV9k2w9LTGmwnk1CFz5Ld0+?= =?us-ascii?Q?rUn3AZpMsR1/j79+pp9Ok87mEW206l2K0A0RuGyuoG3lqkKix9T/Fk7lEQXm?= =?us-ascii?Q?HwJzMOF9kakAEpI3BAGFqCiaIf80+hlTIBdekNoQvxQNACAyyv8V18ypMD0h?= =?us-ascii?Q?Y93bTS/0+xhnS8kAm9vRSQT8x5VX+DZNGVLcwZBBfiqqOdjOyG4lOgIGSxXW?= =?us-ascii?Q?V4ig7ivvBfbxU0dylMuRZLE7yE9kt/ejp08mcsu98zZBJTCWYfyl3akI2vfY?= =?us-ascii?Q?GHtaYEYWl1SjiIoBeDwI9ZK3jt4vkMlXwxqQ9Vy/WFAoiuUqnVV0WosJwHgt?= =?us-ascii?Q?tx5o3UDJmXZW5QJCzsXhWcDc2MDvk4oYkaGK7CnaWWnfWuxU5foTowNGMiB4?= =?us-ascii?Q?FBpLyKa2q6wYFVJMpPUcKuGSYByIb2WkIS2RKEBDM15V2IJdhn/KwS2a8AhU?= =?us-ascii?Q?ygXngxxlwdrFTUnXjRrkaY9DUTETw3c6OKw3SBsXPBqMytLefR/t/skwpNy1?= =?us-ascii?Q?gWa2RX++wL5Xk+32XbRdgZupSE73qJJbcwGfLSFvRwMXMuBD20veYtypoQob?= =?us-ascii?Q?T3sn/5SXcjdB/z6oOd45TNIaa5o2xXg9TkqlYrZ0v3dARE5WKAlWvX5dPaKR?= =?us-ascii?Q?U3EMhtzhOkv1NBX0KXW6+q2xp0MvdWhanKrP9alMN/MmqkFqyXZ5Gw33V55C?= =?us-ascii?Q?z0dbZgYRrewqEjfyn8eKWqTxnM/NkmfngMcwnNvV51bmbt+SxF8Tb7aq+PWS?= =?us-ascii?Q?pQSPXVTFFIyXuKm7LQUAvDe+Ve6aXmm88W4cdWSToovYtP7NtMq23yOvOiGp?= =?us-ascii?Q?u7I+0unOMaoju6LfO869Z0FImcy61AYt/vnQpI/sQQW+lxGqu4QEsVLJIV0l?= =?us-ascii?Q?il/cUwIj3FrkNq2bLaPQqB5HGTd+uhtIV2Weja5B96WX3DCck5J0+Ci2MIoC?= =?us-ascii?Q?Etqh/OOh393KNHbqyOHxKRr3nXoz00P6lBu6cP+t+y4Etn1Ujd6F8Vty64kx?= =?us-ascii?Q?qJlmSebZwQT87VXm205iwdpfqJZ4d/chUJP6sOlE15zsVbBGfEiYhEUkzXNk?= =?us-ascii?Q?pLThqXPdfxLBqPoOrsxxDGh807r4LU9cmdPu02Xk8mJize1FWKHZ3ch3lYcz?= =?us-ascii?Q?MHyIR9Dsn4CoiMybHIpkxas2U8PICwR0+q+CGAJRpRAdyPLrlWVXU39nmPFa?= =?us-ascii?Q?2CcJ7HiGrWn/ExEZUz9+v/qpLWJj2mEAZwbRytA8s8lA4vRn7oxBXKwtL1xB?= =?us-ascii?Q?smGAljwHbzQQogMHb3+vBBFQeeoGbiuezULJMeRQkeVlsTtteihdbpG7jq/O?= =?us-ascii?Q?tSaL3J6RTaDd6fsntly+K54Igcz+ia5eJy9C4iBEdtxE7uOEUr1hkmAkyH9U?= =?us-ascii?Q?q2Ag3N1cgqo=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(52116014)(7416014)(376014)(366016)(921020)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?r45KEwGVvi1egnDNkWTzmCZ0Dq3RfYHlV9fz3IMCmCE+tjCO1/SRF+yW+Yf+?= =?us-ascii?Q?sU9C3Fd1wfn/vBpIDkVCzhkmIGkO49J+N6mgsjrye7e06+izDoLlaFFHVXGS?= =?us-ascii?Q?WJ7eMG9Q+RPa1U7Pq9FAhf+mzu02y5L5VMMqPxu1llfbLijAPzQVnVRv1lys?= =?us-ascii?Q?2shyonsFRvQ1NmX2OgWU6QYGoymV96VWZGjNX27xTyGUYCOTXnTYFBXVb3CR?= =?us-ascii?Q?/FzV3QNn5DIhouVPs/q1sB9nWLUlK6CrFd3itd3lwnW95HJOAt5ANZOPIEb6?= =?us-ascii?Q?qgnzCRVLzsvWhVj9n3OgK86HO7p/tOycorKw+0svgBOhJiyPqDpik/q48ZQ/?= =?us-ascii?Q?H1cjguNRb/J0RtNYI9i1LGerUTPXjxXkEEGzXIkF8B2LYecxB4Yo0bcwfMrA?= =?us-ascii?Q?7SKCceqVtOBWm6wjOA0YYs8lIowhPs78sVJcss8pM9TwbEEs4foE9HjKfFs0?= =?us-ascii?Q?giJuVxezPIzlD9ZLJvmy2pgR7rxNwD4vp0BWLhAkJygx0bqxA7edaY+l0MFE?= =?us-ascii?Q?S3H1K9uRr5fT4fjHVKa7a4SVkDVUXxfTwvk0EWQgPJlG1DL9JtgBDjHkE+vm?= =?us-ascii?Q?CAnJUkAstvYUpKD/xbMo7Zm1luGWdfgrOJt+kjP6s1Z21i5IAVpXnd6JyJMK?= =?us-ascii?Q?suRxYcwZwUyqjAqoPl8kFphfotD2NNa+3DU42kOU7a0muEjvUbTYN/jTittf?= =?us-ascii?Q?5gQyZ5SN3P8KhyVnYvbnX5pjcQB5e/mZiXyRUhfylKFhprJvSH7PawEFbU88?= =?us-ascii?Q?kbIv8XXpcPvEMiVEYM2vuxyn3y2LYnOwNY7m08gjqAUjR8ftDpp3Qg1hWQZO?= =?us-ascii?Q?AfwbZ6ToLJArYQEKPr4VjIQ/8KM6LwiPhnYk4iEitCNDKE6qOB4yM3ZAVmsz?= =?us-ascii?Q?c9cZnsDM632zftROOtJRgYFv9Q5bvREee6MIZ01e7JUlSpMf4WjT3TIllWYM?= =?us-ascii?Q?SxFCIncKT8fJAUbQhjwBlyEXrLW4bbMQlCSCXTrIr1S9bO09sqBCtHZUd4fT?= =?us-ascii?Q?GsjKMYG65gqyt86BUjLqB9eVeSUlw44FK2hZcUxfdDoh0tR3Tpn9Gqd2EWXL?= =?us-ascii?Q?cPR+zMf0KQ7n5AWynaWyZeKrb5ylqSzkG8tnvZMvKl6OjCl65y73BQ/YlOwp?= =?us-ascii?Q?XYJFvzJXYS5Gi1LT8VIC94zybgxXB5NLx4TUiZ0K8VmZI6nmQtpwX7qnt/0Y?= =?us-ascii?Q?lXCCkXbgT+w9PZZqTiBjVG8QMUFn4HGhmWAfJTKYgGRab86PtrcohDzMshXk?= =?us-ascii?Q?DAfhQlh2m21ebf9OBhSj9NnEqaAAk4UwvmWzXrJonR17O3zzb1ZUynJFiVJ0?= =?us-ascii?Q?XfA2Qhx7hFLIwhoOVym5cgIpUFRx0RsGF9Q8DzIDrNUnNVrwXxM8fbO6ybL0?= =?us-ascii?Q?PVW7gANQTIAO/M6eL0g/ZQWFnKRL8TQMT7fNYuG6Tq6T4fgeDFmDdvZSnSST?= =?us-ascii?Q?iQ+gvV5kO/kmZR9xiZNTWgGyj8hqGLEhFcxVZZZgu5vNVDOpKveYmpos60LF?= =?us-ascii?Q?aMhtFMtj9zHCfaRJRLg+ZskpO1uIxBmkpWUB9jEqwFGlAiUggtRtThI0vlqB?= =?us-ascii?Q?j3CQz9HWTaLUk2akv3MQkIuKEY7Vy5tdr4IOoCUS?= X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: e9c9a2ec-00b6-4829-04f8-08dc9a736ef7 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jul 2024 08:46:15.4736 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IQlY2azfKDb1kg7oQIMdIERcJVFFW+6CsslydbbvgCpB/EyythP7qZTREJ9qZsJ2abYPA7x8j4htXKNs8EJ+2w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5790 Content-Type: text/plain; charset="utf-8" If memcg enables pmc, it will cache some pages. However, if all processes in memcg exit and there are some remaining pages in the cache, those pages will not be used unless memcg is deleted. To avoid this situation, a periodic reaping job has been added to each memcg when pmc enabled, which will reclaim all the cached memory in the memcg at regular intervals(default 5s). User also can change reaper interval time like below: echo keys=3Dreaper_time=3D8000000 > memory.cache This memcg will reaper cache each 8s(type unit is us) Signed-off-by: Huan Yang --- include/linux/mmzone.h | 6 ++++ mm/memcontrol.c | 77 ++++++++++++++++++++++++++++++++++++++---- mm/page_alloc.c | 1 + 3 files changed, 77 insertions(+), 7 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 773b89e214c9..b56dd462232b 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -608,6 +608,7 @@ struct mem_cgroup_zone_cache { struct list_head pages; spinlock_t pages_lock; atomic_t nr_pages; + atomic_t nr_reapered; atomic_t nr_alloced; }; =20 @@ -616,6 +617,11 @@ struct mem_cgroup_per_node_cache { struct mem_cgroup_zone_cache zone_cachep[MAX_NR_ZONES]; struct mem_cgroup *memcg; =20 + /* cycle cache reclaim time unit, us, default 5s, 0 means disable reaper = */ +#define DEFAULT_PMC_REAPER_TIME ((5 * 1000 * 1000)) + unsigned int reaper_wait; + struct delayed_work reaper_work; + /* max number to hold page, unit page, default 100MB */ #define DEFAULT_PMC_HOLD_LIMIX ((100 << 20) >> PAGE_SHIFT) unsigned int hold_limit; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 9db5bbe63b34..ae6917de91cc 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -7101,6 +7101,39 @@ static ssize_t memory_reclaim(struct kernfs_open_fil= e *of, char *buf, return nbytes; } =20 +/** + * This function use to reaper all cache pages by cycling scan. + * The scan interval time depends on @reaper_wait which can set by `keys` = nest + * key. + * Default, each memcg which enabled cache will be reapered every 5s. + */ +static void pmc_reaper(struct work_struct *worker) +{ + struct mem_cgroup_per_node_cache *node_cachep =3D container_of( + to_delayed_work(worker), struct mem_cgroup_per_node_cache, + reaper_work); + struct mem_cgroup *memcg; + int num; + + if (!READ_ONCE(node_cachep->reaper_wait)) + return; + + memcg =3D node_cachep->memcg; + rcu_read_lock(); + if (!css_tryget(&memcg->css)) { + rcu_read_unlock(); + return; + } + rcu_read_unlock(); + + num =3D mem_cgroup_release_cache(node_cachep); + + css_put(&memcg->css); + + schedule_delayed_work(&node_cachep->reaper_work, + usecs_to_jiffies(node_cachep->reaper_wait)); +} + static int __enable_mem_cgroup_cache(struct mem_cgroup *memcg) { int nid, idx; @@ -7141,8 +7174,13 @@ static int __enable_mem_cgroup_cache(struct mem_cgro= up *memcg) p->memcg =3D memcg; p->hold_limit =3D DEFAULT_PMC_HOLD_LIMIX; p->allow_watermark =3D DEFAULT_PMC_GAP_WATERMARK; + p->reaper_wait =3D DEFAULT_PMC_REAPER_TIME; =20 atomic_inc(&pmc_nr_enabled); + + INIT_DELAYED_WORK(&p->reaper_work, pmc_reaper); + schedule_delayed_work(&p->reaper_work, + usecs_to_jiffies(p->reaper_wait)); } =20 if (static_branch_likely(&pmc_key)) @@ -7184,6 +7222,7 @@ static int __disable_mem_cgroup_cache(struct mem_cgro= up *memcg) =20 p =3D nodeinfo->cachep; =20 + cancel_delayed_work_sync(&p->reaper_work); mem_cgroup_release_cache(p); =20 kfree(p); @@ -7207,7 +7246,8 @@ static int mem_cgroup_cache_show(struct seq_file *m, = void *v) if (!READ_ONCE(memcg->cache_enabled)) return -EINVAL; =20 - seq_printf(m, "%4s %16s %16s\n", "NODE", "WATERMARK", "HOLD_LIMIT"); + seq_printf(m, "%4s %16s %16s %16s\n", "NODE", "WATERMARK", + "HOLD_LIMIT", "REAPER_TIME"); for_each_online_node(nid) { struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; struct mem_cgroup_per_node_cache *p; @@ -7216,13 +7256,15 @@ static int mem_cgroup_cache_show(struct seq_file *m= , void *v) if (!p) continue; =20 - seq_printf(m, "%4d %14uKB %14uKB\n", nid, + seq_printf(m, "%4d %14uKB %14uKB %16u\n", nid, (READ_ONCE(p->allow_watermark) << (PAGE_SHIFT - 10)), - (READ_ONCE(p->hold_limit) << (PAGE_SHIFT - 10))); + (READ_ONCE(p->hold_limit) << (PAGE_SHIFT - 10)), + READ_ONCE(p->reaper_wait)); } =20 seq_puts(m, "=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D\n"); - seq_printf(m, "%4s %16s %16s %16s\n", "NODE", "ZONE", "CACHE", "HIT"); + seq_printf(m, "%4s %16s %16s %16s %16s\n", "NODE", "ZONE", "CACHE", + "REAPER", "HIT"); =20 for_each_online_node(nid) { struct mem_cgroup_per_node *nodeinfo =3D memcg->nodeinfo[nid]; @@ -7242,9 +7284,12 @@ static int mem_cgroup_cache_show(struct seq_file *m,= void *v) continue; =20 zc =3D &p->zone_cachep[idx]; - seq_printf(m, "%4d %16s %14dKB %14dKB\n", nid, z->name, + seq_printf(m, "%4d %16s %14dKB %14dKB %14dKB\n", nid, + z->name, (atomic_read(&zc->nr_pages) << (PAGE_SHIFT - 10)), + (atomic_read(&zc->nr_reapered) + << (PAGE_SHIFT - 10)), (atomic_read(&zc->nr_alloced) << (PAGE_SHIFT - 10))); } @@ -7257,6 +7302,7 @@ static int mem_cgroup_cache_show(struct seq_file *m, = void *v) enum { OPT_KEY_NID, OPT_KEY_WATERMARK, + OPT_KEY_REAPER_TIME, OPT_KEY_HOLD_LIMIT, OPT_KEY_ERR, NR_PMC_KEY_OPTS =3D OPT_KEY_ERR @@ -7265,6 +7311,7 @@ enum { static const match_table_t fc_tokens =3D { { OPT_KEY_NID, "nid=3D%d" }, { OPT_KEY_WATERMARK, "watermark=3D%u" }, + { OPT_KEY_REAPER_TIME, "reaper_time=3D%u" }, { OPT_KEY_HOLD_LIMIT, "limit=3D%u" }, { OPT_KEY_ERR, NULL} }; @@ -7282,6 +7329,12 @@ __apply_status_for_mem_cgroup_cache(struct mem_cgrou= p_per_node_cache *p, STATUS_UNSET_DEFAULT_VALUE) p->allow_watermark =3D opts[OPT_KEY_WATERMARK]; break; + case OPT_KEY_REAPER_TIME: + if (opts[OPT_KEY_REAPER_TIME] !=3D + STATUS_UNSET_DEFAULT_VALUE) + WRITE_ONCE(p->reaper_wait, + opts[OPT_KEY_REAPER_TIME]); + break; case OPT_KEY_HOLD_LIMIT: if (opts[OPT_KEY_HOLD_LIMIT] !=3D STATUS_UNSET_DEFAULT_VALUE) @@ -7319,7 +7372,7 @@ mem_cgroup_apply_cache_status(struct mem_cgroup *memc= g, } =20 /** - * Support nid=3Dx,watermark=3Dbytes,limit=3Dbytes args + * Support nid=3Dx,watermark=3Dbytes,limit=3Dbytes,reaper=3Dus args */ static int __mem_cgroup_cache_control_key(char *buf, struct mem_cgroup *memcg) @@ -7361,6 +7414,14 @@ static int __mem_cgroup_cache_control_key(char *buf, return -EINVAL; opts[OPT_KEY_WATERMARK] =3D v; break; + case OPT_KEY_REAPER_TIME: + if (match_uint(&args[0], &v)) + return -EINVAL; +#define MAX_REAPER_TIME ((10 * 1000 * 1000)) + if (v > MAX_REAPER_TIME) + return -EINVAL; + opts[OPT_KEY_REAPER_TIME] =3D v; + break; case OPT_KEY_HOLD_LIMIT: if (match_uint(&args[0], &v)) return -EINVAL; @@ -7402,7 +7463,9 @@ static const match_table_t ctrl_tokens =3D { * 1. nid=3Dx, if input, will only change target NODE's cache status. El= se, all. * 2. watermark=3Dbytes, change cache hold behavior, only zone free page= s above * high watermark+watermark, can hold. - * 3. limit=3Dbytes, change max pages can cache. Max can change to 500MB + * 3. reaper_time=3Dus, change reaper time, default is 10s. Set 0 can di= sable, + * max can change to 10s + * 4. limit=3Dbytes, change max pages can cache. Max can change to 500MB * Enable and keys can both input, split by space, so can set args after e= nable, * if cache not enable, can't set keys. */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 54c4d00c2506..1fe02f4f3b33 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1310,6 +1310,7 @@ int mem_cgroup_release_cache(struct mem_cgroup_per_no= de_cache *nodep) } =20 num +=3D i; + atomic_add(i, &zc->nr_reapered); atomic_sub(i, &zc->nr_pages); } =20 --=20 2.45.2 From nobody Fri Dec 19 09:13:32 2025 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2065.outbound.protection.outlook.com [40.107.117.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CEF21552E7; Tue, 2 Jul 2024 08:46:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.117.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719909984; cv=fail; b=frpM+RepfRX7fly2iDgaiWrwF/FIlGLzbfwUrc1zXf7aT9zsxnfOO/l/h4Cp8e+z4OK/bkHqfUmZkxatkWggzcqqqF6O6xHv6H8g+QZ6cCJ3M6y3BlwojmE/1kVNpBk0BpNqnRJ6UtkONqNstrSeOnR6Cu8Dcv4DA0+rHkC+QhQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719909984; c=relaxed/simple; bh=Tx3p0Zwb2La6K0c1322gFWoGWJDZIThUbTNXsuKF4ds=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=GoisjWWDpbkF6TRE9f96eXc/4cVJCrt15hIS9DXZudICP9T7XUSpfw7FwfD/+5yhONoRDSxmua1rtPSYSBqW2W5MAS57H1Dfomeqdc69BMgzIt/T8jzIvm/wS3QAgE5bAXgJJOmDD/daAqjvwx8zFMUjR00/0h5RcVbQYqbWKJY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com; spf=pass smtp.mailfrom=vivo.com; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b=jbDyxQC+; arc=fail smtp.client-ip=40.107.117.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=vivo.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=vivo.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=vivo.com header.i=@vivo.com header.b="jbDyxQC+" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J89eKsVjJVafsKba0ue95qSrgWkHY/NfjpKu5FkaidCshw4qSUuviuh7NGddXzA6s3pqsFFohWrmDaz97YYjOPF7hai/RVyTD7r+VRsKximo/GksF5p/ZZaFaTtp7Fkj1tzJfRTckefZOcELfRGVSg5upPtbr/YQBPgUJlDpgoSt/VqqJj4FWqQFJAJpYpJlQQXSQtgaY0a9JZ2oOZbVH2LIgwL1DWeqNtkaBA5dun2j88+SIqcgV3MVI1rn7bhWGo+/h3/q4mxmIzwZDd6FRLOo7vhxLAs5jCEOG+iw+R3oIrxuNTQjQCBbt3dAd7pPk5g1b+H1/Z3IoPZtftuHbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wayXX6if5q49kSH+e7BH2GqjBW2PQyGEKDlKzSgTM/M=; b=GzE6U8GDjkpeKwdzvJMgMAelLrT+IiKm9e5HzN4LAzseTAM9Z1BJ5oZ+Y7h7dyzk3+zz9569X++cOtMj1OD7G/M/1YwqeeivUj9NL2oYnyHMBgtBGrLH6loniuFZiSMMz/q9A33QRo3GJ8Md6MAxS3cUlXdsEaA8o29WRtS50g321bzgjrSDC3fbehKJ5HqVJubl2fx151BqcRcyfsxJF3Nmk4Q3zKzxgho4fAEVyM8oz0zJrDUcLJVAfeFQakV2HykxiFgFNOqQvNQ65y1pguPOD9/NbgYMF5mpK64dDBwi+dM7Eon2uFMiy3gl/4ZMSDN5H5ZEwWv/nn6lUg2PTQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wayXX6if5q49kSH+e7BH2GqjBW2PQyGEKDlKzSgTM/M=; b=jbDyxQC+prZBsQ+Bb1fY+q59ALKp3VoiOI12VUaQGpE75r52B/+htLdIVeCnQNUgKqaacaF/6HhWB1I2TuHd2l5XoxiozliXXp/A643ztyUPlbhEYVs5wJRIpx9Kplx1/PM7dwQjdD/Gvpqe3DCVbY9RFCk49t1URN6/QJl0qXys7jG2CGOLXwyX9zzykd5bHbYCgNcANGWVTg/MIFONn9UHKcYASqBpfgOn+D8iYiYb5hKfNS73H42QaQIycvEF7iuGXlV/Ce4s9bkCLbXLKkZAmaP2fzOL2895x1tBmUMhC7uVG46COfWqf887BvTvv4beX6mcl5+x3WRsuoYbwQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=vivo.com; Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEYPR06MB5790.apcprd06.prod.outlook.com (2603:1096:101:b9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.23; Tue, 2 Jul 2024 08:46:20 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7719.028; Tue, 2 Jul 2024 08:46:20 +0000 From: Huan Yang To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , Ryan Roberts , Chris Li , Dan Schatzberg , Huan Yang , Kairui Song , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christian Brauner , Jan Kara Cc: opensource.kernel@vivo.com Subject: [RFC PATCH 4/4] mm: memcg: pmc: support oom release Date: Tue, 2 Jul 2024 16:44:07 +0800 Message-ID: <20240702084423.1717904-5-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240702084423.1717904-1-link@vivo.com> References: <20240702084423.1717904-1-link@vivo.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: SG2PR01CA0148.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::28) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEYPR06MB5790:EE_ X-MS-Office365-Filtering-Correlation-Id: 8d1a0e68-ab3e-4099-7ed0-08dc9a7371fb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|52116014|7416014|376014|366016|921020|38350700014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?vsqpTpPI6N2WGgg0iSkF+In/Pxw7vcXP7rull1Detp3BnAezK6peCymBMaYE?= =?us-ascii?Q?4BTFC6VUgTVrFrfSdpv3j3OnmhBlaItao8IymvyMiZta04sqX5geHlLQrPei?= =?us-ascii?Q?CvpfB9BZxggphQxFj0B13CyQ1IcW/6n8Rr3dmscfIRVkKfxwc1Qm/uCaQ8JA?= =?us-ascii?Q?uPP1fmtxqkUW7kKqOM1rPGhanwlCUIVSKXEwu42ZIywJdRAh0CqACi7cnsMw?= =?us-ascii?Q?0J72BSgDFph+0otu3EnaBqT8IAJp1bgcGzu6XgewjGax77qA7XONpmp7r1GL?= =?us-ascii?Q?SQ+sc4gHSexyEMdxYmPj3fjUF6Ftjx9hBPY+YmhRnQpLIicquovcuTcMVFSw?= =?us-ascii?Q?M+6i6UYQj5uxuI2FqThOeuvRXOmFRcSgel6nWqrtHyRJ+k93lm4bJj+JiLfU?= =?us-ascii?Q?vQgJAuyaRETxJVoAyq987wSmodwxYIkUuL9VqRilkh7sPn8hfTIkvDa41pgj?= =?us-ascii?Q?zS7TslaCVrVgzSVBfwI+t+CskecOyWG42lcABFw4l16ai+RxNS/CI/xyePVe?= =?us-ascii?Q?/dByfkzw4uc0TM2CoRiGFFPMs/QYEhbxSKbR1ja+t4kbUYqWE4CXbPoAG4o+?= =?us-ascii?Q?n4GEmyiNfQyVjqjZV5smuauptrRJfJzW7xIi4rYrZLLKi7ddAPKbxhA1crli?= =?us-ascii?Q?10kkLViiKmQ5WRA68a7ANoGaJNl+Bdz487InmmjR7fWscyO3zsLdICv9+Wz3?= =?us-ascii?Q?rTYATn3cFobDUgXlHQ93KRVUgKcD9EhRvIKmTpoU5q1QXS8C51vlN0L8kW9L?= =?us-ascii?Q?O8iwVNR+b0H36stJdheAOBLr6A4g34DllIgz2LxrGLa+gCuFXXRRng3FpPIM?= =?us-ascii?Q?crkSsh/Z9NNgUP3B7d4zDgzBKrHNFOPDrVho8uAQR5cT7VBpmIiN+2otuTj3?= =?us-ascii?Q?xye7GJcty7PbxYb3eOk7GeaOsJoDc9XroVRhMixenYQt+2RPiUspsRJS0DGw?= =?us-ascii?Q?cc9lqCryrG/E2SfRN/DktqjhbjyPEU16mrc3HCCcZvYBnUKgYa/Rx4Z7s4Dh?= =?us-ascii?Q?w4kB3fBblsQvAs6p57QxcsXQYwnjKsrhQP76LH8E4OhfuCI73EidLSOTdac5?= =?us-ascii?Q?D37a7InU1rU+MHaHDGx1iencOn7C9QuDSlCCstDJKl5pSZJde33CrlyVi+BZ?= =?us-ascii?Q?swztEX2ymA2J+oNWBj7YaYpmtQQPd/AIxJZkW/ZL1mWq4LRwzv0lyYL0iKXL?= =?us-ascii?Q?NSk1ZrW9G1TXl9Kr/VbZzmN/UbDrJeqyxi7oLTf80kuAlKGr757JnHMvNoOg?= =?us-ascii?Q?dARgEh3+UJzWeTIh9IZtvYzce2SndrehSFj/QqC9rsg8ph/CnGYJ+82MRxiH?= =?us-ascii?Q?Jqn0czHyDVIq2Qewtukhq9+lFDhjrk+vXCo/Nhw9WmnfvrB3huyslvturCbf?= =?us-ascii?Q?dYC83RR+zRrio16D5qzzcY3h0rR5+zQrWDRgDEq37NgJxjmchkF/UEvDNUOQ?= =?us-ascii?Q?DrvMsxCbcho=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(52116014)(7416014)(376014)(366016)(921020)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?G00yIZH0KEXv5OXfIs8l6NEO6NXL/A61iR75yv9JrQ4l9AbLuHyffUj7a5J3?= =?us-ascii?Q?d2gY4FFKiG50arGFekXXHwmcwiP1TVEfJ5+X8k8JaqIJxEpkrn+2MQnmtIKb?= =?us-ascii?Q?EiG26ITnRSNlhBjhggQHSl+hdj8WUR7tKjC2qAx/m7RoM1rsrs8KFH86Vb0t?= =?us-ascii?Q?6tdSk7W+AAowq/LfC/ktSMvcNnKQa1+L+qUsM1iqyXCII1obprL072SPiPYS?= =?us-ascii?Q?ARlvA6iLrLKNYhEMSbOUzT7wA5DcqCLga3rSXvefwjQeowPJQbbmdt8SnEk9?= =?us-ascii?Q?S6EdbcKT0AzGya17bOURMeoo7WaeYAWOA6UCBCKpDXZhY0fToSzqbwqO1hSb?= =?us-ascii?Q?SxF4paYKYxZ8Ys3QguAz+K2dBqqC7OrC8DujIUwT4Y0m485VLXTvVtqH3YQ8?= =?us-ascii?Q?f7gCa2QAN0Nk0jtHCnZw+pqUdnISEoAp2Kocpz32oDfyFPBNJvPfCZfvh5iI?= =?us-ascii?Q?zSO2U15w5fgcPMIndPcVppPhD1a5625W2n5hn/uaKN02jA+97hvrCttA5QoU?= =?us-ascii?Q?ft2OyxyAwND0cOPUwSyrf754LWRHAvGz6vswU0/eqmeFpbRpmEWcbqX/vLB4?= =?us-ascii?Q?xAHjOQCy7dz9OMMDD+hh4RVeMk++XHoB7pk4U+bo0ud7e0ktqiGtK0FCTYKb?= =?us-ascii?Q?ZYg/a3XhHMypbn8xtHyJZaE366zSpIc9t9o8o9n0nL+kjzfLxx1ILNwQ6oLo?= =?us-ascii?Q?BgV8WwmcppeJZmriG45r3+UL8iPsJxqcmXDDKYCVDh6MrgpnNMpvNRs9clGl?= =?us-ascii?Q?ZWRTuJrHlPlARJWIOYaoQAuhMZGkCAHQaaKGhinKXekj2++hFYrm3lq1ayAu?= =?us-ascii?Q?2iEbNgjprj9jYb1PwPEUDjWU4YOex29tH1k5T+/fyLvLBaPvyuCsbmUy3zwJ?= =?us-ascii?Q?Z7cy9x2f+pODrkiCD3PIntCpSMwihFOytiOuBnW2aZARey2Onmy7AfZnoRPl?= =?us-ascii?Q?iiFz6cuyjlg1dePCg4pvROMjZulm9WvB69IaVNruS6JhxEe179k0jrg2KJwp?= =?us-ascii?Q?/Jv4m+Ucu+OYTD4v4PeRJCUMb8yeqU5ZZVCxDGexGNRxry9ris3Cd2GlkRPX?= =?us-ascii?Q?zJKL43BgbsXOHsHj8IFLskUiCy/Xu5GkA3WX4kHKSDb4sdCnMvUEFSh6DGqI?= =?us-ascii?Q?74aqL8ZM4q5khl66VaGGiKE5yNzpD+kznrzOnrIOFOkGeJVpAMRFVfj6DXfD?= =?us-ascii?Q?/AkVvDq2oRY8DXP5jkks9u8pdmy16PSnd9KIpuNAzG7TpJrdhBt1BiVzXRrq?= =?us-ascii?Q?de6vaDcRgo9/M8QkXYsDXU2x9cdvzCUDUr1nJyEDj2tq7h54ffeXv7asqoP7?= =?us-ascii?Q?gSJz+YLRsGUFyKVSVqpw6ydEAttdptNFTxm3j86kN63QsiqTD9KKSP/X8DlD?= =?us-ascii?Q?H39+/lAdnGHAv7HtB8HkfFSOsiOX4Z/AU3lTUzLMKjQl18ZSuWiRGhPRgoQH?= =?us-ascii?Q?jkHcdVXzhyy0rTAjCzuthXBXTgkGEtKfm4gH/A+AyZpT2qAqUnCDuaNCB6ZB?= =?us-ascii?Q?6w8PSH2DZuhMK75cmRRK3CtpfXHVNBG3tmeP3Ns9pYY3m6vE6hKCBO8ZqBDE?= =?us-ascii?Q?qdwA+zWHWkPl98DkrvkqkTvMPWBgApWyp+O0YKGP?= X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8d1a0e68-ab3e-4099-7ed0-08dc9a7371fb X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jul 2024 08:46:20.5457 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: NzEupW0V4BOAb11XvdtEl5741crWW/za4brGf8wfDzJOF1VhuEWE95CWnRRZ0d9S8GmZWXDn1bEoyugO4NVe2g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5790 Content-Type: text/plain; charset="utf-8" This patch let each enabled pmc's memcg register a oom listener, so if oom will trigger, release all hold pages. Signed-off-by: Huan Yang --- include/linux/mmzone.h | 3 +++ mm/memcontrol.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index b56dd462232b..640a9cf51791 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -622,6 +622,9 @@ struct mem_cgroup_per_node_cache { unsigned int reaper_wait; struct delayed_work reaper_work; =20 + /* listen oom event, release hold cache */ + struct notifier_block oom_nb; + /* max number to hold page, unit page, default 100MB */ #define DEFAULT_PMC_HOLD_LIMIX ((100 << 20) >> PAGE_SHIFT) unsigned int hold_limit; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ae6917de91cc..3dfb2a17c1fd 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -7101,6 +7101,33 @@ static ssize_t memory_reclaim(struct kernfs_open_fil= e *of, char *buf, return nbytes; } =20 +/** + * This function listen to oom event, if oom will trigger, check and relea= se + * all holded pages. + */ +static int pmc_oom_notify(struct notifier_block *self, + unsigned long notused, void *nfreed) +{ + struct mem_cgroup_per_node_cache *node_cachep =3D + container_of(self, struct mem_cgroup_per_node_cache, oom_nb); + struct mem_cgroup *memcg =3D node_cachep->memcg; + + unsigned long *nf =3D (unsigned long *)nfreed; + + rcu_read_lock(); + if (!css_tryget(&memcg->css)) { + rcu_read_unlock(); + return NOTIFY_STOP; + } + rcu_read_unlock(); + + nf +=3D mem_cgroup_release_cache(node_cachep); + + css_put(&memcg->css); + + return NOTIFY_OK; +} + /** * This function use to reaper all cache pages by cycling scan. * The scan interval time depends on @reaper_wait which can set by `keys` = nest @@ -7176,6 +7203,9 @@ static int __enable_mem_cgroup_cache(struct mem_cgrou= p *memcg) p->allow_watermark =3D DEFAULT_PMC_GAP_WATERMARK; p->reaper_wait =3D DEFAULT_PMC_REAPER_TIME; =20 + p->oom_nb.notifier_call =3D pmc_oom_notify; + register_oom_notifier(&p->oom_nb); + atomic_inc(&pmc_nr_enabled); =20 INIT_DELAYED_WORK(&p->reaper_work, pmc_reaper); @@ -7222,6 +7252,7 @@ static int __disable_mem_cgroup_cache(struct mem_cgro= up *memcg) =20 p =3D nodeinfo->cachep; =20 + unregister_oom_notifier(&p->oom_nb); cancel_delayed_work_sync(&p->reaper_work); mem_cgroup_release_cache(p); =20 --=20 2.45.2