From nobody Thu Apr 2 20:22:02 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35BEF274B5C for ; Fri, 13 Feb 2026 10:03:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770977031; cv=none; b=NyTGyydc5EE2efOhqZu5CFnpfvCmRwvFw+YIUi/9keH9EZvCas4Ew9PJndYMtMHXc+EJP3RHd0B4DmI5oexDL2hk8oGhBaUWXJwRMoIGdVGM35VJ95ky7Bfaa9NWmIRVNvLQVy3dUh32eKdf5N+vgDb4I6uL8Bu/J6ehbeaMNiA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770977031; c=relaxed/simple; bh=Zs7QNHNf3ZiH09poIh8XMpwvw4NFTa+WNpDRkmMCPPg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=gWF0lflHamycEJOuEVQfttEf/yMYegVBh/LjxmbHl4U1Ae+rg7PPB1GF2vFlTmCksagYWBi+a5PgBBZ8i4hJZOuuz2Bu7a7q4yxxT9NapP3DBbZy3C+pvcG6ZfR+BnVi87CVK7JtTWxBvm0tjOr21qBmtcfIOmOVz5yoqlw+iLs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bFPGMLyr; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bFPGMLyr" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-824ba8f0acaso475092b3a.1 for ; Fri, 13 Feb 2026 02:03:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1770977029; x=1771581829; darn=vger.kernel.org; h=cc:to:message-id:content-transfer-encoding:mime-version:subject :date:from:from:to:cc:subject:date:message-id:reply-to; bh=Ua28z24qUIwhHYuR3RexH1TybxsEtCTG1vVnJWGtDUc=; b=bFPGMLyrp8+kvBoAibO6aGdYwHXyce7Vv02mCGqZZ0lwW3VhXc+Dx+3MSDCLUdJ/bN zUKoZa3O7Qb69ePcXOl+G3vkXJ/HNIMlS2EkIVh4k97mwyyzvpnGIoE0hj+l1sjcU572 uJbpXwfQtIHmvaNdGTZwMXkZsmUUdE7JXE4jpPNkYVxRanxW338Bky3/3EHxwFqeedAn 3PMi5Snlf6hTa8CkdsdLqBMmB/OkBoG45Z5jm57GUVw5kh8ss/rxjGNlFag3dATpXZ/l /dZtGe7l1E04eF0odj8sAVxQuaLp/7/96NgCWjnLzMqPNHEXPIockO9puSSY6plWpb8O msUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1770977029; x=1771581829; h=cc:to:message-id:content-transfer-encoding:mime-version:subject :date:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Ua28z24qUIwhHYuR3RexH1TybxsEtCTG1vVnJWGtDUc=; b=XY9VQCq9xmKCAPh0pd1r8/hirzCsNIR8ChDq9ZBfCsyec956i2Vl/sg/MdjS2NIZOr rLF6fWmUPQvrd5FlBOwLEZGtEBhoju8/0l/8lDx1T9RuDcHO8YdeXWU6Qq2/okUijGuu lxl+54AXHz/UBznNj+kaeVo5TDZt9hp3GMaZuPgd2UzBfOUWq7VYgg6apcdNYbkTmb8f GavyS3Mj9uxnlZGFfOvXzaoZK0r6+znXd+eofQbTmmKTJR68JWm4ms8hCYzA2vF3J1tM tW/RW56Gz4r4eS1fpYPQsXZnj4sAbNE/tn2TEKNbzNzRFKr+YYdwb9bWhpP3bWLuL2bM DSOg== X-Forwarded-Encrypted: i=1; AJvYcCVsHD5Jl8CL/t58pYOltmue0J54HBPxpqNqWuIi74j7H1wuFlvjjxCi2LCmQv3zXow/kfkC5EQSVnSMd2I=@vger.kernel.org X-Gm-Message-State: AOJu0YxPj8sjQw+PhlIIKlUiFl1TAW9CGKG+dA2UQV/8TVOG05Hij+Pa WJgKxsW27of4nsgmHYfks0mMLjP9BPEOFgzdljih4DzxAtWmss8GIJB7 X-Gm-Gg: AZuq6aLzqcgP3KowEl7S6gjL8XipZID8bCUmevJ6sninl6diarL1V3xFEC9/TSXKJy9 LD4TBNlKSK52qgrlgDRx9Ohh3Zx1DAtii5Q0LiWyzvbw3JRokA7Ko+3M8J9TXNuM1C8Qh7IX3f7 Q9wCzy6RDk2uDSCqwVcmfFSaSkCLDNZQPrvTBYfGFSM7gwvJr4R9cB+dnkAlfl+n+0k1XsbNYHy 2XVyf1yUxl0etAfSGW9Y7P5VTXdn/w5GvRjBkIQpTYn3MLcaX4JIKX2pE9yxd9afvEtDMm3M7zM IVaWWP9HE8hfC5DlGVWD9HJMLLKzN0uvgOvVbaeRBYt3FvB7qsfullTIlb90DYSF7DAY6PLcvkK 2d+QZoXvL71bHMpBhb+DO4JcndqzXbg5cenvd1LFys63jErJedaIayrayR97X3m8UgvuS0LHfhH 3ppWBfFneOafA971RD0aDYfNRq9qElGLPdb+u90CzYLwGh6Ly+N5m1h0vg7xj9emZ6GPzjnA== X-Received: by 2002:a05:6a00:9288:b0:7b9:8142:96f4 with SMTP id d2e1a72fcca58-824c94ab2eamr1348552b3a.21.1770977029363; Fri, 13 Feb 2026 02:03:49 -0800 (PST) Received: from [127.0.0.1] ([43.132.141.25]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-824c6a627a5sm2473663b3a.27.2026.02.13.02.03.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Feb 2026 02:03:48 -0800 (PST) From: Kairui Song Date: Fri, 13 Feb 2026 18:03:32 +0800 Subject: [PATCH] memcg: consolidate private id refcount get/put helpers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260213-memcg-privid-v1-1-d8cb7afcf831@tencent.com> X-B4-Tracking: v=1; b=H4sIAAAAAAAC/6tWKk4tykwtVrJSqFYqSi3LLM7MzwNyDHUUlJIzE vPSU3UzU4B8JSMDIzMDI0Nj3dzU3OR03YKizLLMFF2zpEQjc3PjJNPkRCMloJaCotS0zAqwcdG xtbUAcS/uOl4AAAA= X-Change-ID: 20260213-memcg-privid-6ba2773b5ca2 To: cgroups@vger.kernel.org, linux-mm@kvack.org Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , linux-kernel@vger.kernel.org, Kairui Song X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1770977026; l=5327; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=aaFj5J9A/PnGFEo2q9D27aps5NSieS+BOnpHHXp5Eqk=; b=NCr+8KXvWVTribPSok34ty4hUhLU2/EfjCEL+VahbxmDCkbQ5IkaoayDDK1+gfER+QaxYvFZJ gftXwyIzRAYCv2mzXrGBZ1ysQkQw9seQLFFImWE0V6EUqmzB4tcT1lI X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= From: Kairui Song We currently have two different sets of helpers for getting or putting the private IDs' refcount for order 0 and large folios. This is redundant. Just use one and always acquire the refcount of the swapout folio size unless it's zero, and put the refcount using the folio size if the charge failed, since the folio size can't change. Then there is no need to update the refcount for tail pages. Same for freeing, then only one pair of get/put helper is needed now. The performance might be slightly better, too: both "inc unless zero" and "add unless zero" use the same cmpxchg implementation. For large folios, we saved an atomic operation. And for both order 0 and large folios, we saved a branch. Signed-off-by: Kairui Song Acked-by: Johannes Weiner Acked-by: Shakeel Butt Reviewed-by: Chen Ridong --- mm/memcontrol-v1.c | 5 +---- mm/memcontrol-v1.h | 4 ++-- mm/memcontrol.c | 29 +++++++---------------------- 3 files changed, 10 insertions(+), 28 deletions(-) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 0e3d972fad33..c28a060abc64 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -635,11 +635,8 @@ void memcg1_swapout(struct folio *folio, swp_entry_t e= ntry) * have an ID allocated to it anymore, charge the closest online * ancestor for the swap instead and transfer the memory+swap charge. */ - swap_memcg =3D mem_cgroup_private_id_get_online(memcg); nr_entries =3D folio_nr_pages(folio); - /* Get references for the tail pages, too */ - if (nr_entries > 1) - mem_cgroup_private_id_get_many(swap_memcg, nr_entries - 1); + swap_memcg =3D mem_cgroup_private_id_get_online(memcg, nr_entries); mod_memcg_state(swap_memcg, MEMCG_SWAP, nr_entries); =20 swap_cgroup_record(folio, mem_cgroup_private_id(swap_memcg), entry); diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index 49933925b4ba..dbbd0e13d4ff 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -28,8 +28,8 @@ unsigned long memcg_events(struct mem_cgroup *memcg, int = event); unsigned long memcg_page_state_output(struct mem_cgroup *memcg, int item); int memory_stat_show(struct seq_file *m, void *v); =20 -void mem_cgroup_private_id_get_many(struct mem_cgroup *memcg, unsigned int= n); -struct mem_cgroup *mem_cgroup_private_id_get_online(struct mem_cgroup *mem= cg); +struct mem_cgroup *mem_cgroup_private_id_get_online(struct mem_cgroup *mem= cg, + unsigned int n); =20 /* Cgroup v1-specific declarations */ #ifdef CONFIG_MEMCG_V1 diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 007413a53b45..4425ef51feae 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3564,13 +3564,7 @@ static void mem_cgroup_private_id_remove(struct mem_= cgroup *memcg) } } =20 -void __maybe_unused mem_cgroup_private_id_get_many(struct mem_cgroup *memc= g, - unsigned int n) -{ - refcount_add(n, &memcg->id.ref); -} - -static void mem_cgroup_private_id_put_many(struct mem_cgroup *memcg, unsig= ned int n) +static inline void mem_cgroup_private_id_put(struct mem_cgroup *memcg, uns= igned int n) { if (refcount_sub_and_test(n, &memcg->id.ref)) { mem_cgroup_private_id_remove(memcg); @@ -3580,14 +3574,9 @@ static void mem_cgroup_private_id_put_many(struct me= m_cgroup *memcg, unsigned in } } =20 -static inline void mem_cgroup_private_id_put(struct mem_cgroup *memcg) +struct mem_cgroup *mem_cgroup_private_id_get_online(struct mem_cgroup *mem= cg, unsigned int n) { - mem_cgroup_private_id_put_many(memcg, 1); -} - -struct mem_cgroup *mem_cgroup_private_id_get_online(struct mem_cgroup *mem= cg) -{ - while (!refcount_inc_not_zero(&memcg->id.ref)) { + while (!refcount_add_not_zero(n, &memcg->id.ref)) { /* * The root cgroup cannot be destroyed, so it's refcount must * always be >=3D 1. @@ -3888,7 +3877,7 @@ static void mem_cgroup_css_offline(struct cgroup_subs= ys_state *css) =20 drain_all_stock(memcg); =20 - mem_cgroup_private_id_put(memcg); + mem_cgroup_private_id_put(memcg, 1); } =20 static void mem_cgroup_css_released(struct cgroup_subsys_state *css) @@ -5170,19 +5159,15 @@ int __mem_cgroup_try_charge_swap(struct folio *foli= o, swp_entry_t entry) return 0; } =20 - memcg =3D mem_cgroup_private_id_get_online(memcg); + memcg =3D mem_cgroup_private_id_get_online(memcg, nr_pages); =20 if (!mem_cgroup_is_root(memcg) && !page_counter_try_charge(&memcg->swap, nr_pages, &counter)) { memcg_memory_event(memcg, MEMCG_SWAP_MAX); memcg_memory_event(memcg, MEMCG_SWAP_FAIL); - mem_cgroup_private_id_put(memcg); + mem_cgroup_private_id_put(memcg, nr_pages); return -ENOMEM; } - - /* Get references for the tail pages, too */ - if (nr_pages > 1) - mem_cgroup_private_id_get_many(memcg, nr_pages - 1); mod_memcg_state(memcg, MEMCG_SWAP, nr_pages); =20 swap_cgroup_record(folio, mem_cgroup_private_id(memcg), entry); @@ -5211,7 +5196,7 @@ void __mem_cgroup_uncharge_swap(swp_entry_t entry, un= signed int nr_pages) page_counter_uncharge(&memcg->swap, nr_pages); } mod_memcg_state(memcg, MEMCG_SWAP, -nr_pages); - mem_cgroup_private_id_put_many(memcg, nr_pages); + mem_cgroup_private_id_put(memcg, nr_pages); } rcu_read_unlock(); } --- base-commit: 9fff1ab283e0982c2b8e73f1d2246fd38caf40c8 change-id: 20260213-memcg-privid-6ba2773b5ca2 Best regards, --=20 Kairui Song