From nobody Thu Sep 24 20:04:34 2026 Received: from mta0.migadu.com (out-109.mta0.migadu.com [91.218.175.109]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AD4245D93E for ; Mon, 21 Sep 2026 09:13:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.109 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789981994; cv=none; b=CVxQSo98z+x3ccP2I3qrBTLEEHWLunjuCIjcZfWYTIiPxwQzbfcwDjWQYfYuDFmlsaPVsRmrN4PfdSdQtc8TvLwBPXEFQUfPjOCtdIfwktelJ3NaEHbRAiewrTRIWBzsabdyK434p0cIkr2zidBrvjqp7dPP1ADlxLXjtQ9pgMc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789981994; c=relaxed/simple; bh=Cu2ExFzZ0wPeXHlo/H4RTvXS1GKs5wBwOLrGXeuqp+k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=mjQpQde58SmrxLBtRRgF/fhG4Wr1S/iQssuXRUgWscalhItJ/zkk83oPRvsXNCfOV48tgsoxAbeJ4UacOWLhK+SzGYhJrz2IlzoaRMz8CUG7AWFNg0fxlRzvkjw+nnTHrxgJ8Ru5a3cl7LdpDAhehYtKiWSR+br76P3LSeWFNaI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=Miis2Cv4; arc=none smtp.client-ip=91.218.175.109 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="Miis2Cv4" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=Cu2ExFzZ0wPeXHlo/H4RTvXS1GKs5wBwOLrGXeuqp+k=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1789981990; v=1; x=1790586790; b=Miis2Cv4MjcWjIMZzm2CtRBTMulDYVqwjxWv0zqNylRVpysSxP27oY6zsOOCZnraLLmrFyv/ XVV1D2tJlPWCMeYMVBuI7M7Q14Cre+b7zjaeHfavZYFtRtBt/AeWhLMNC1AMwz0POKmoOnrOJ+F /17o2PHquQeDW9GpjHZ/m2aA= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id fd58ae7357b5b993; Mon, 21 Sep 2026 09:13:10 +0000 X-Mizu-Trace-ID: fd58ae7357b5b993 X-Migadu-Flow: FLOW_OUT From: Hongfu Li Date: Mon, 21 Sep 2026 17:12:34 +0800 Subject: [PATCH 1/2] mm/hugetlb: account migration target folio in per-node NR_HUGETLB vmstat Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260921-for-hugetlb_state-v1-1-8a6eec92661f@kylinos.cn> References: <20260921-for-hugetlb_state-v1-0-8a6eec92661f@kylinos.cn> In-Reply-To: <20260921-for-hugetlb_state-v1-0-8a6eec92661f@kylinos.cn> To: Muchun Song , Oscar Salvador , David Hildenbrand , Andrew Morton , Shakeel Butt , Michal Hocko , Roman Gushchin , Nhat Pham , Chris Down , Johannes Weiner , Michal Hocko Cc: hongfu.li@linux.dev, Joshua Hahn , linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Hongfu Li , stable@vger.kernel.org X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1789981974; l=3203; i=lihongfu@kylinos.cn; s=20260915; h=from:subject:message-id; bh=qFQURuvqIWZRduSFy2tOfQ/YWYqYpSChKzoedQvCG4U=; b=alqTdmVgU+bX1/P6i4lpe9eRPyzHnbFGO5R5WIL/PQfB4BeOJYPmPMpV6DQ7XbSba/uZrVhJW ifPbV5agVEODv7ZRdaHq4bMs4Yc+A/BUxoZMSwYZei0bAwbxGooDdMp X-Developer-Key: i=lihongfu@kylinos.cn; a=ed25519; pk=rUpDSpB3neoNZRlGG6dHlFb3RepiJakcev+ucyqaZzw= From: Hongfu Li The NR_HUGETLB vmstat counter is maintained per folio's node: incremented when a huge page is handed to a user via hugetlb_alloc_folio() and decremented when it is returned to the pool via free_huge_folio(). A folio obtained by alloc_hugetlb_folio_nodemask() never goes through hugetlb_alloc_folio(), so it is never accounted, while its free always is. For a migration target this means the target node gets no matching increment for the decrement on the old node, so the global nr_hugetlb in /proc/vmstat drops by nr_pages for each migration. The same asymmetry affects the failed migration path, which frees the target again right away, and the temporary folio hugetlb_mfill_atomic_pte() takes from the same helper. alloc_hugetlb_folio_reserve(), used to preallocate the memfd page cache folios, has the same asymmetry: the folio is handed to a user without being accounted, while its free is accounted through free_huge_folio(). Account the folio where it is obtained, in alloc_hugetlb_folio_nodemask() and alloc_hugetlb_folio_reserve(), so that the increment pairs with the decrement in free_huge_folio(): a successful migration hands the folio to a user, a failed one frees it again. Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg") Cc: stable@vger.kernel.org Signed-off-by: Hongfu Li Acked-by: Muchun Song Acked-by: Oscar Salvador Reviewed-by: Joshua Hahn Tested-by: Joshua Hahn --- mm/hugetlb.c | 35 +++++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 12 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1b53ba991d36..8ccc769bf48c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2206,6 +2206,11 @@ struct folio *alloc_hugetlb_folio_reserve(struct hst= ate *h, int preferred_nid, } =20 spin_unlock_irq(&hugetlb_lock); + + if (folio) + lruvec_stat_mod_folio(folio, NR_HUGETLB, + folio_nr_pages(folio)); + return folio; } =20 @@ -2213,24 +2218,30 @@ struct folio *alloc_hugetlb_folio_reserve(struct hs= tate *h, int preferred_nid, struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred= _nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback) { - spin_lock_irq(&hugetlb_lock); - if (available_huge_pages(h)) { - struct folio *folio; + struct folio *folio =3D NULL; =20 + spin_lock_irq(&hugetlb_lock); + if (available_huge_pages(h)) folio =3D dequeue_hugetlb_folio_nodemask(h, gfp_mask, preferred_nid, nmask); - if (folio) { - spin_unlock_irq(&hugetlb_lock); - return folio; - } - } spin_unlock_irq(&hugetlb_lock); =20 - /* We cannot fallback to other nodes, as we could break the per-node pool= . */ - if (!allow_alloc_fallback) - gfp_mask |=3D __GFP_THISNODE; + if (!folio) { + /* + * We cannot fallback to other nodes, as we could break the + * per-node pool. + */ + if (!allow_alloc_fallback) + gfp_mask |=3D __GFP_THISNODE; =20 - return alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid, nmask); + folio =3D alloc_migrate_hugetlb_folio(h, gfp_mask, preferred_nid, + nmask); + } + + if (folio) + lruvec_stat_mod_folio(folio, NR_HUGETLB, folio_nr_pages(folio)); + + return folio; } =20 static nodemask_t *policy_mbind_nodemask(gfp_t gfp) --=20 2.54.0 From nobody Thu Sep 24 20:04:34 2026 Received: from mta1.migadu.com (out-18.mta1.migadu.com [95.215.58.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7911445FFA6 for ; Mon, 21 Sep 2026 09:13:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789982002; cv=none; b=Gsk5m4CAPeQszMS4/WAoqbOKeZGqWB2xbhfzn8dQ7OyYap7r69DtXRlYfWnwJQQrg68Rx0bTtlsC8I/fIgSfdnI7zASWAPNtLm6yz9+1djfu/EeXu+19tF8pQvSx2fl29w7Ct1d/qX+ndH2jafwbA2WRObKybTXn+r0MlNHdF1Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789982002; c=relaxed/simple; bh=IDs/EyWNmqJ898UTthMq8jY9Q1siEUrx/p114gn5Wns=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=niK4UfHf78BOOFzI4hdRF+ZcOC5RA4GlqneWhTCvgdzz5H9ApM4iO7Ni9pHNwBx8WwQTGnYhue83rwXMC9+VNSITPMeEpD5mF/YPtOpGnPeSLkypCpmoSKxasUzOzIDA+V2gj0U0khMePm7A/TQ/jd0tzCOUtP6Q2FUXploG4XQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=k9INECDi; arc=none smtp.client-ip=95.215.58.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="k9INECDi" X-Envelope-To: linux-kernel@vger.kernel.org DKIM-Signature: a=rsa-sha256; bh=IDs/EyWNmqJ898UTthMq8jY9Q1siEUrx/p114gn5Wns=; c=simple/simple; d=linux.dev; h=from:to:subject:date:message-id:mime-version:content-type; s=key1; t=1789981998; v=1; x=1790586798; b=k9INECDiU2azQYNr44eOEUMVpgjRuGHtZgw3KW/S0rCYFh5svYkyAudRiOe0RlFhOo71ZKV3 JxstsVrQiNl/pNTeqE7qH5XfSaNomzF2QxPN3CZndi23nQYTYLmNGOFDSOlaSdlIaHSujntjOla SVe6LkR/PhUwmmH/pmR6Wk4I= X-Envelope-To: linux-kernel@vger.kernel.org Received: by smtp.migadu.com with ESMTPS id c126fa98069b27b1; Mon, 21 Sep 2026 09:13:18 +0000 X-Mizu-Trace-ID: c126fa98069b27b1 X-Migadu-Flow: FLOW_OUT From: Hongfu Li Date: Mon, 21 Sep 2026 17:12:35 +0800 Subject: [PATCH 2/2] mm/memcg: migrate per-node hugetlb lruvec stat together with hugetlb folio Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260921-for-hugetlb_state-v1-2-8a6eec92661f@kylinos.cn> References: <20260921-for-hugetlb_state-v1-0-8a6eec92661f@kylinos.cn> In-Reply-To: <20260921-for-hugetlb_state-v1-0-8a6eec92661f@kylinos.cn> To: Muchun Song , Oscar Salvador , David Hildenbrand , Andrew Morton , Shakeel Butt , Michal Hocko , Roman Gushchin , Nhat Pham , Chris Down , Johannes Weiner , Michal Hocko Cc: hongfu.li@linux.dev, Joshua Hahn , linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Hongfu Li , stable@vger.kernel.org X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1789981974; l=2700; i=lihongfu@kylinos.cn; s=20260915; h=from:subject:message-id; bh=7maGt7gBIU3c+GPCYRQ3OHOtV+Op/E4Dxf2YD4KKGv0=; b=sYVTVa5VHI0QpIl9gKOtFeVmq6k72AX6PouymGlK4tZgfU/sDP/xUMQEYtPFsuetgF5HnkyHp Wol77pUzrqMA5+yMrquCywDFXnag9IZ0lTqCj5u9dbSqJzQakpxSiK4 X-Developer-Key: i=lihongfu@kylinos.cn; a=ed25519; pk=rUpDSpB3neoNZRlGG6dHlFb3RepiJakcev+ucyqaZzw= From: Hongfu Li memory.numa_stat exposes per-node hugetlb counters from per-node lruvec stats. These stats are accounted against folio_nid(): incremented on the folio's node when handed to a user, decremented when the folio is returned to the pool. During hugetlb folio migration, mem_cgroup_migrate() moves the charge to the new folio and drops the memcg data of the old one, so the free of the old folio right after the migration skips the memcg per-node lruvec decrement. The hugetlb count stays attributed to the old node for the rest of the life of the charge, while the target folio gets no increment on the new node; its later free decrements a counter that was never incremented. Migrate the per-node lruvec accounting alongside migration. Global memcg totals remain balanced because they track resource consumption, not node placement. Fixes: 05d4532b60e3 ("memcg/hugetlb: add hugeTLB counters to memcg") Cc: stable@vger.kernel.org Signed-off-by: Hongfu Li Reviewed-by: Joshua Hahn Reviewed-by: Oscar Salvador Tested-by: Joshua Hahn --- mm/memcontrol.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 1460cba53588..9c96ebd5436f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5598,6 +5598,34 @@ void mem_cgroup_replace_folio(struct folio *old, str= uct folio *new) rcu_read_unlock(); } =20 +#ifdef CONFIG_HUGETLB_PAGE +static void move_hugetlb_lruvec_stat(struct obj_cgroup *objcg, + struct folio *old, struct folio *new) +{ + long nr_pages =3D folio_nr_pages(old); + struct mem_cgroup *memcg; + int old_nid =3D folio_nid(old); + int new_nid =3D folio_nid(new); + + if (old_nid =3D=3D new_nid) + return; + + rcu_read_lock(); + memcg =3D obj_cgroup_memcg(objcg); + mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(old_nid)), + NR_HUGETLB, -nr_pages); + mod_memcg_lruvec_state(mem_cgroup_lruvec(memcg, NODE_DATA(new_nid)), + NR_HUGETLB, nr_pages); + rcu_read_unlock(); +} +#else /* CONFIG_HUGETLB_PAGE */ +static inline void move_hugetlb_lruvec_stat(struct obj_cgroup *objcg, + struct folio *old, + struct folio *new) +{ +} +#endif /* CONFIG_HUGETLB_PAGE */ + /** * mem_cgroup_migrate - Transfer the memcg data from the old to the new fo= lio. * @old: Currently circulating folio. @@ -5635,6 +5663,9 @@ void mem_cgroup_migrate(struct folio *old, struct fol= io *new) =20 new_objcg =3D get_migration_objcg(old, new); =20 + if (folio_test_hugetlb(old)) + move_hugetlb_lruvec_stat(new_objcg, old, new); + /* * @old was charged through a non-root objcg, so its charge is in the * page counters. If the re-derivation walked up to the root objcg - --=20 2.54.0