From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 502E641F357 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=vCx061CLLWsyeGk4NUPGuwwzLdICF5gBU8Dc+DN2OB9ztioKngJIFAF6HAXr3yY1X4P7g5nxfCQPpwIf6suVAG+N7HZaRU5/KzRh8CUjNQ6YY9S0ojsgyQKf+oFuTCMAn5n5sGhKCBYpC6uF8SI9T8j4kR42KKd21xJnyun52aU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=q6d8QZgJ9uVNO04KnNDgtygNJ+rHQadMULxGSKbRmEg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=UTObcLixZIjyH8Yzp7F76Tdxrq1UecKHsWBKeah4BcNoCQ2dGCe7uBhMFvIdFaCye6y+zC4BOZlvyLJihz7XxcBQEr5OtQZHQ+dsMKoITkOZOGrRUEiHV3KLEaCMyT/sCxXJcycMS0lSxX8ldRLEWS3S+cOLJKEygiaj1Rtj6Uk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AfS2126m; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AfS2126m" Received: by smtp.kernel.org (Postfix) with ESMTPS id DCB0DC2BCF5; Wed, 23 Sep 2026 21:32:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199162; bh=q6d8QZgJ9uVNO04KnNDgtygNJ+rHQadMULxGSKbRmEg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=AfS2126mBOIZCNW8Yw33eyefuLmORQ/mOw93lQcSGdTsNuzuTSLEB0E/9tbrI+jYd 9JCZ+veWdEvWcCQ9aw9Le41pPftAFYBeJuxgPscvf9oJ87qq6a/TvRWcgjgqWfsU9y dqsHb/nw9nEmb9hTtT5bvl7kfaIPCVbw4iDQ+ruBAZeN6yIqbiekpCOzofaMgJztY3 naHRERim1HgDEyLhBo5It/i9+DRw6gGhVor1QeZCU7JJ/Gr4DuRCAnCU0fEOxhAH8/ yF4EoFU2BGmtmWW+4SJHridvuYctKdI8GDc1Bypk3phbMlU36TM9HlqMZaLCjIPCcc 14wRAE4itHp7w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA5C7C98304; Wed, 23 Sep 2026 21:32:42 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:32 +0200 Subject: [PATCH v6 01/17] mm/swap: fix off-by-one in swap cache replace sanity check Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-1-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=1650; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=810n4do4aOKMWihCxqnhsHXg2hcES1SAm+rRly1NOZc=; b=vI2SJv+2aeuKdY+dWJee/PWf/aWyUl9ypGDpgl7FiReWrKyzgi6HVzlApGTKJIbXcZeUL12EC CimIwkHeUSPBmNnfoHAvke7AZXvA0B0jzBZHX0CHs91bgDmImjxMDGJ X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song The DEBUG_VM sanity check in __swap_cache_replace_folio() iterates the old folio's range with "while (ci_off++ < ci_end)", so the loop body runs on the already-incremented offset: the first entry is skipped and one entry past the range is read. For a folio split that entry belongs to the first after-split folio and was just repointed by the replacement loop above, so the check would warn spuriously whenever sub-folio orders differ from the head folio's. Currently we don't support non-uniform swapcache split, but this still needs a fix to clean it up and prepare for non-uniform swap cache split. Use the same do-while pattern as the replacement loop. Fixes: 8578e0c00dcf ("mm, swap: use the swap table for the swap cache and s= witch API") Acked-by: Zi Yan Reviewed-by: Barry Song Acked-by: David Hildenbrand (Arm) Reviewed-by: Yeoreum Yun Acked-by: Kiryl Shutsemau (Meta) Signed-off-by: Kairui Song --- mm/swap_state.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/swap_state.c b/mm/swap_state.c index 625c185a1ca4..cef44aadee61 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -396,8 +396,9 @@ void __swap_cache_replace_folio(struct swap_cluster_inf= o *ci, folio_order(old) !=3D folio_order(new)) { ci_off =3D swp_cluster_offset(old->swap); ci_end =3D ci_off + folio_nr_pages(old); - while (ci_off++ < ci_end) + do { WARN_ON_ONCE(swp_tb_to_folio(__swap_table_get(ci, ci_off)) !=3D old); + } while (++ci_off < ci_end); } } =20 --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50259408023 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=gwy9X95YU/p4NYtOOTuaTtPOUB/s3cTIHaJp7g4H6FsDHpH26N5S9YrTxm8K77XhWoKHJjATsivVThZtxE9tB02CzwJcEWFYpSWiA+pVlHlbsvga+bzzRoJF2V/Jm9fUQYt+qjAZI2jILVypFdtWl82um8fkMFpgJmcD0S94XxU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=AlG9b9X0RGUOUnaV4lCSu0L+6PtqGMzvhv4TLKbM208=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gRR33bT7FZ9NzHpTMoNG87tbP2zlTUsMFFRsL84boz2IUiblv8WywRbYBZlmWXG9zTjFrmgN9oI4RQ4D8MEoJJcCAhzdFtln/yXOT7fdTZv8hiytq1ChQqDHVA3uKZ008RpdyRIiCN9VzGnW1r60Ms6lpqPgMOQC7zbbc3JMD5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=pMj/ApG3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="pMj/ApG3" Received: by smtp.kernel.org (Postfix) with ESMTPS id F16ABC2BCFA; Wed, 23 Sep 2026 21:32:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=AlG9b9X0RGUOUnaV4lCSu0L+6PtqGMzvhv4TLKbM208=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=pMj/ApG3ZDXO4DysHJStGRu52DpPLCKbuSj+eGCfgzqlVsx4ajFfzvpo+acLJkGbV jvnYYq6LddJT7OilFCG++BLICd0ynoxEV0drCb0bz0yFwcVJYvC3ScENCxMwHdgZ4M lLh7v8GDy2YKN7CDXnCsEKhkdf/YNGJlizOsEnn8N6YffTp/iB823ujnSqtrgd6pm/ xD0Hz2dVUvqJhHzJ95cbX/ynC5l4FRIIj0WvvwbdX4SkmrjuYHgA7GQTPNGiqytbhb pGeWrD2Y9YVk+ttw3pvtlOI7hQnA1wOhpu+8E9oa+T6VVBFEQaDuQUlUyYRho8xEQP w9Mu4SdDhHpnQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF98DC9830D; Wed, 23 Sep 2026 21:32:42 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:33 +0200 Subject: [PATCH v6 02/17] mm/huge_memory: fix rejection of swap cache folios with a mapping Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-2-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=3844; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=r8O7IxC0Y3pay3mFjqjlnT64x3uz6sA8DPmkIPqd0RE=; b=eGkwoI5fC+OmgprGowmfNfI6eQgDDCAuA8gmp8oDZZv08ehu9OY+GY0DCCIpio4jRxVa7WegH csogviooyBkA9cHBdKwQGIYK3IuJoMGlUmjgs/1OM+9P8/KJxmMPSKO X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song A folio in the swap cache cannot be split if it has a mapping (shmem). The split code does a defensive check for this in __folio_freeze_and_split_unmapped, after the folio ref has been frozen and the NR_SHMEM_THPS/NR_FILE_THPS counters have been decremented. It rejects the split and returns -EINVAL without unfreezing the folio or restoring the counters. That error path is buggy, if it is ever taken. It leaves the folio frozen and stuck, skews the counters, and fires the VM_WARN_ON_ONCE_FOLIO for a state that is actually legitimate. Check for this case up front in folio_check_splittable and return -EBUSY before any state is modified, so the split routine always backs out cleanly. Also fix a bracket style issue that checkpatch.pl keeps complaining about. Fixes: 00527733d0dc ("mm/huge_memory: add two new (not yet used) functions = for folio_split()") Fixes: 714b056c8321 ("mm/huge_memory: convert VM_BUG* to VM_WARN* in __foli= o_split") Reviewed-by: Zi Yan Reviewed-by: Barry Song Acked-by: David Hildenbrand (Arm) Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Signed-off-by: Kairui Song --- mm/huge_memory.c | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index a4984e14909e..2500fbd9197a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3933,6 +3933,9 @@ static int __split_unmapped_folio(struct folio *folio= , int new_order, int folio_check_splittable(struct folio *folio, unsigned int new_order, enum split_type split_type) { + const bool is_anon =3D folio_test_anon(folio); + const bool is_swapcache =3D folio_test_swapcache(folio); + VM_WARN_ON_FOLIO(!folio_test_locked(folio), folio); /* * Folios that just got truncated cannot get split. Signal to the @@ -3941,11 +3944,11 @@ int folio_check_splittable(struct folio *folio, uns= igned int new_order, * TODO: this will also currently refuse folios without a mapping in the * swapcache (shmem or to-be-anon folios). */ - if (!folio->mapping && !folio_test_anon(folio)) + if (!folio->mapping && !is_anon) return -EBUSY; =20 /* order-1 is not supported for anonymous THP. */ - if (folio_test_anon(folio) && new_order =3D=3D 1) + if (is_anon && new_order =3D=3D 1) return -EINVAL; =20 /* @@ -3956,7 +3959,7 @@ int folio_check_splittable(struct folio *folio, unsig= ned int new_order, * swapcache folio split. Only uniform split to order-0 can be used * here. */ - if ((split_type =3D=3D SPLIT_TYPE_NON_UNIFORM || new_order) && folio_test= _swapcache(folio)) + if ((split_type =3D=3D SPLIT_TYPE_NON_UNIFORM || new_order) && is_swapcac= he) return -EINVAL; =20 if (is_huge_zero_folio(folio)) @@ -3965,6 +3968,15 @@ int folio_check_splittable(struct folio *folio, unsi= gned int new_order, if (folio_test_writeback(folio)) return -EBUSY; =20 + /* + * A non-anon swapcache folio that still has a mapping can only be a + * shmem folio under SWAP IO, it's removed from either swap cache or + * shmem mapping afterward. There is little benefit in splitting them + * hence reject it here up front before touching anything. + */ + if (!is_anon && is_swapcache && folio->mapping) + return -EBUSY; + return 0; } =20 @@ -4037,14 +4049,8 @@ static int __folio_freeze_and_split_unmapped(struct = folio *folio, unsigned int n } } =20 - if (folio_test_swapcache(folio)) { - if (mapping) { - VM_WARN_ON_ONCE_FOLIO(mapping, folio); - return -EINVAL; - } - + if (folio_test_swapcache(folio)) ci =3D swap_cluster_get_and_lock(folio); - } =20 /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ if (do_lru) --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 503B6423EB7 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=XjcA08NYXrMqji/KotCmvwF5TdEtKgloUIy9eabe1dPnw6hLVEwyaiY3maKYCXbnDImWPuuWAuxA1Sl3l1XwkFW/XTcFSV3ZdBVN8GCbZaoWYghXU9QnPbtdYLguwtHibpxLhUJKDFY6nuSq3t4cg2YuHkTWVMhQheazMsVQrsY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=lxvk+d2PTRziyGkFuhfUQJlCysK6pDw0nqV5ypb7+8k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ZGhJYcf0oKURRhiEiqMg+9wxn73l9BsxIRD3Hu5oNRJwlevUjv0h+MUcOAGNINvbQ8EccWZkidHGe2aDXlTR5a3zLksGjPAOBZgAQZZy0m1FfH3QHFEPPcG8i9091lpgumzW53TXd9QOGnKyASA7wE328ObofZV6fjB7Pj2uaCA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=kfuYGiQZ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="kfuYGiQZ" Received: by smtp.kernel.org (Postfix) with ESMTPS id 0BF11C2BD01; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=lxvk+d2PTRziyGkFuhfUQJlCysK6pDw0nqV5ypb7+8k=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=kfuYGiQZu0R3xO/YmQAV4LCsfb60HWXvEXlJugCurZLhpJ6GG7rMRzW1fSTEf2p17 upe6ykci2dT2f1Zzu4pd3taJKAmUy/0tExEPiWlqppY8IB/M8cZg6029XyxV9Qeavw K4gc/TsLBWBKiRTo1cm/f2Ocvqql7cgkMc4lH3hPS0O953hSkxyDd4wZRH8NaxxY9K +yj+AMJPcAf5iW2gQG8nM07aJzuKcgqhPOvofNDKzOw2tphJ3hfZ+osGaULDhZZWly zlWFuJWonnY+K7nBVPUP1JJqyxJ6vQ+aD+NuiiIkAgI4lIxozXZk2X8rzXWLlu+dxl 56N3jU+U6ZA7A== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4077C98312; Wed, 23 Sep 2026 21:32:42 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:34 +0200 Subject: [PATCH v6 03/17] mm/huge_memory: invert folio_ref_freeze() check to reduce indentation Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-3-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=7784; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=tyKyhC3HjxOEHD6oTdGuiWz4W7o2CnGS6AQaMVDEp6A=; b=/2Uieid6+ENpSAQHNc5R2oOJ0WDr+q1oj3vrg/vQy397s6ljQneeMAyvfZ4RjnXUJEpS5vvuN vtR8//qXmOTBcv/z10vRtx7nzlZrg+D0IJderA2w2iB2FNpUa35vQVO X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song Invert the folio_ref_freeze() success check in __folio_freeze_and_split_unmapped() to return early on failure, which removes one level of indentation from the entire success path. This is a pure refactoring with no functional change. It prepares the function to be split into separate helpers for anonymous and file-backed folios in a later patch. Reviewed-by: Zi Yan Reviewed-by: Barry Song Acked-by: David Hildenbrand (Arm) Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Signed-off-by: Kairui Song --- mm/huge_memory.c | 178 +++++++++++++++++++++++++++------------------------= ---- 1 file changed, 88 insertions(+), 90 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2500fbd9197a..a475278e1e67 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4014,121 +4014,119 @@ static int __folio_freeze_and_split_unmapped(stru= ct folio *folio, unsigned int n pgoff_t end, int *nr_shmem_dropped) { struct folio *end_folio =3D folio_next(folio); + struct swap_cluster_info *ci =3D NULL; struct folio *new_folio, *next; + struct lruvec *lruvec; int ret =3D 0; =20 VM_WARN_ON_ONCE(!mapping && end); =20 - if (folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) { - struct swap_cluster_info *ci =3D NULL; - struct lruvec *lruvec; + if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) + return -EAGAIN; =20 - /* Take off the deferred split queue while frozen and memcg set */ - folio_unqueue_deferred_split(folio); + /* Take off the deferred split queue while frozen and memcg set */ + folio_unqueue_deferred_split(folio); =20 - /* - * deferred_split_scan() takes the folio off the queue before it - * splits it, so the unqueue above finds an empty list and - * leaves PG_partially_mapped set. - * Clear it here: the flag does not survive the split. - */ - folio_reset_partially_mapped(folio); + /* + * deferred_split_scan() takes the folio off the queue before it + * splits it, so the unqueue above finds an empty list and + * leaves PG_partially_mapped set. + * Clear it here: the flag does not survive the split. + */ + folio_reset_partially_mapped(folio); =20 - if (mapping) { - int nr =3D folio_nr_pages(folio); - - if (folio_test_pmd_mappable(folio) && - new_order < HPAGE_PMD_ORDER) { - if (folio_test_swapbacked(folio)) { - lruvec_stat_mod_folio(folio, - NR_SHMEM_THPS, -nr); - } else { - lruvec_stat_mod_folio(folio, - NR_FILE_THPS, -nr); - } + if (mapping) { + int nr =3D folio_nr_pages(folio); + + if (folio_test_pmd_mappable(folio) && + new_order < HPAGE_PMD_ORDER) { + if (folio_test_swapbacked(folio)) { + lruvec_stat_mod_folio(folio, + NR_SHMEM_THPS, -nr); + } else { + lruvec_stat_mod_folio(folio, + NR_FILE_THPS, -nr); } } + } =20 - if (folio_test_swapcache(folio)) - ci =3D swap_cluster_get_and_lock(folio); - - /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ - if (do_lru) - lruvec =3D folio_lruvec_lock(folio); + if (folio_test_swapcache(folio)) + ci =3D swap_cluster_get_and_lock(folio); =20 - ret =3D __split_unmapped_folio(folio, new_order, split_at, xas, - mapping, split_type); + /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ + if (do_lru) + lruvec =3D folio_lruvec_lock(folio); =20 - /* - * Unfreeze after-split folios and put them back to the right - * list. @folio should be kept frozon until page cache - * entries are updated with all the other after-split folios - * to prevent others seeing stale page cache entries. - * As a result, new_folio starts from the next folio of - * @folio. - */ - for (new_folio =3D folio_next(folio); new_folio !=3D end_folio; - new_folio =3D next) { - unsigned long nr_pages =3D folio_nr_pages(new_folio); + ret =3D __split_unmapped_folio(folio, new_order, split_at, xas, + mapping, split_type); =20 - next =3D folio_next(new_folio); + /* + * Unfreeze after-split folios and put them back to the right + * list. @folio should be kept frozon until page cache + * entries are updated with all the other after-split folios + * to prevent others seeing stale page cache entries. + * As a result, new_folio starts from the next folio of + * @folio. + */ + for (new_folio =3D folio_next(folio); new_folio !=3D end_folio; + new_folio =3D next) { + unsigned long nr_pages =3D folio_nr_pages(new_folio); =20 - zone_device_private_split_cb(folio, new_folio); + next =3D folio_next(new_folio); =20 - folio_ref_unfreeze(new_folio, - folio_cache_ref_count(new_folio) + 1); + zone_device_private_split_cb(folio, new_folio); =20 - if (do_lru) - lru_add_split_folio(folio, new_folio, lruvec, list); + folio_ref_unfreeze(new_folio, + folio_cache_ref_count(new_folio) + 1); =20 - /* - * Anonymous folio with swap cache. - * NOTE: shmem in swap cache is not supported yet. - */ - if (ci) { - __swap_cache_replace_folio(ci, folio, new_folio); - continue; - } + if (do_lru) + lru_add_split_folio(folio, new_folio, lruvec, list); =20 - /* Anonymous folio without swap cache */ - if (!mapping) - continue; + /* + * Anonymous folio with swap cache. + * NOTE: shmem in swap cache is not supported yet. + */ + if (ci) { + __swap_cache_replace_folio(ci, folio, new_folio); + continue; + } =20 - /* Add the new folio to the page cache. */ - if (new_folio->index < end) { - __xa_store(&mapping->i_pages, new_folio->index, - new_folio, 0); - continue; - } + /* Anonymous folio without swap cache */ + if (!mapping) + continue; =20 - VM_WARN_ON_ONCE(!nr_shmem_dropped); - /* Drop folio beyond EOF: ->index >=3D end */ - if (shmem_mapping(mapping) && nr_shmem_dropped) - *nr_shmem_dropped +=3D nr_pages; - else if (folio_test_clear_dirty(new_folio)) - folio_account_cleaned( - new_folio, inode_to_wb(mapping->host)); - __filemap_remove_folio(new_folio, NULL); - folio_put_refs(new_folio, nr_pages); + /* Add the new folio to the page cache. */ + if (new_folio->index < end) { + __xa_store(&mapping->i_pages, new_folio->index, + new_folio, 0); + continue; } =20 - zone_device_private_split_cb(folio, NULL); - /* - * Unfreeze @folio only after all page cache entries, which - * used to point to it, have been updated with new folios. - * Otherwise, a parallel folio_try_get() can grab @folio - * and its caller can see stale page cache entries. - */ - folio_ref_unfreeze(folio, folio_cache_ref_count(folio) + 1); + VM_WARN_ON_ONCE(!nr_shmem_dropped); + /* Drop folio beyond EOF: ->index >=3D end */ + if (shmem_mapping(mapping) && nr_shmem_dropped) + *nr_shmem_dropped +=3D nr_pages; + else if (folio_test_clear_dirty(new_folio)) + folio_account_cleaned( + new_folio, inode_to_wb(mapping->host)); + __filemap_remove_folio(new_folio, NULL); + folio_put_refs(new_folio, nr_pages); + } =20 - if (do_lru) - lruvec_unlock(lruvec); + zone_device_private_split_cb(folio, NULL); + /* + * Unfreeze @folio only after all page cache entries, which + * used to point to it, have been updated with new folios. + * Otherwise, a parallel folio_try_get() can grab @folio + * and its caller can see stale page cache entries. + */ + folio_ref_unfreeze(folio, folio_cache_ref_count(folio) + 1); =20 - if (ci) - swap_cluster_unlock(ci); - } else { - return -EAGAIN; - } + if (do_lru) + lruvec_unlock(lruvec); + + if (ci) + swap_cluster_unlock(ci); =20 return ret; } --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B4711DF980 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=K72oZAm559B8pFRfADMNVeBAEXuPOQVjk6RmIw2jrQGS5C5ta69jxJzJ/FMnB4F8u0IRu1W+zET4RLzTuDJd4OaonIRxSiR34n2gHsC78O13OL0hB3byuERJie0c0TEYYy/Sk6eQlxqNuaoRmz1JeIS/crxaFhkdmh2BL8KCLsE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=F7Ybs+UnoH15Z+C6t0pZPct+6mxob0eiqLWChRfhCk8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=IQ4PK0d77y5REh8KZgZOTIrP6Z5RrTgmY60/+XAWU306TZq84QMO+LnR3ZCDj3/1yqQhl3kZ4wmWoO62CMluRjYfRiUDUgiXmsuNQxSpx5D7NB442UsHQzafvr4mwZG+fOaQm+vK6oQCqEYHafyqFFGnKr2SwK9zAbH7UBnIzfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=E8NxxRmK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="E8NxxRmK" Received: by smtp.kernel.org (Postfix) with ESMTPS id 1DDCBC2BCF7; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=F7Ybs+UnoH15Z+C6t0pZPct+6mxob0eiqLWChRfhCk8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=E8NxxRmKYmu7XXWNo5GkQfRwnZr05kwfJwhDk6ICROdKJTTRUx0aq/zY1Ur3M33Aj RfxsqVIzpNEi/gyiBTs+uL9XZ+cb2X6VxiyRFjAvq6WwsoahcLVC+SYg/Is04vKD72 C3KyF9f5u3KIe3m5l5MY9ar/oBu7wpbXE5cBbh+faoKvQ66v9GK2+C92Q/UnfTpmrR HKc3BTuwsXfSPx24vzJr42OK9et1N60ZDoJ8rFH1NI77JkaEAB462i9HTbGLCE1SEw pCSevPNuzp/lSfy8gz+l2L6YP7VVK/+QONeL23ckUKPxqpX7kOY6OFehhl0vdtmD4i 3gcDTKex0a3SA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06A41C98313; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:35 +0200 Subject: [PATCH v6 04/17] mm/huge_memory: split the routine for splitting anon and file folio Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-4-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=8306; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=dOJyL8IG9RVvH6KLP68TxljYyrB1bKWLEdWsFRfNmxo=; b=M5xVhC3qKALj78ArBjYIYb/7GECUSEhvM2BPhegOukcy2eV+2StCN8BjY4OIycRYa4ItGPY1C 75w9fK8MwOFAKDjyfn4uG6sblpHqJY2BNdBOxncZG5PF5ogNfvHri6P X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song No functional change intended. Before adding more logic, split __folio_freeze_and_split_unmapped() into an anon and a file variant so each path can evolve independently. The two paths shared little beyond the folio freeze call, the LRU locking, and the unfreeze skeleton, but differed in all other per-folio bookkeeping and routines. While splitting, some cleanups become easy to apply, and helped drop a few now-redundant checks. The zone_device_private_split_cb() calls are only kept in the anon variant, as device private folios can only back anonymous memory, and add a VM_WARN_ON_ONCE_FOLIO() at the entry of the file variant. Acked-by: David Hildenbrand (Arm) Reviewed-by: Zi Yan Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Signed-off-by: Kairui Song --- mm/huge_memory.c | 124 ++++++++++++++++++++++++++++++++++-----------------= ---- 1 file changed, 78 insertions(+), 46 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index a475278e1e67..1efd858892c1 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4007,11 +4007,9 @@ static void folio_reset_partially_mapped(struct foli= o *folio) MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); } =20 -static int __folio_freeze_and_split_unmapped(struct folio *folio, unsigned= int new_order, - struct page *split_at, struct xa_state *xas, - struct address_space *mapping, bool do_lru, - struct list_head *list, enum split_type split_type, - pgoff_t end, int *nr_shmem_dropped) +static int __folio_freeze_split_anon(struct folio *folio, + unsigned int new_order, struct page *split_at, bool do_lru, + struct list_head *list, enum split_type split_type) { struct folio *end_folio =3D folio_next(folio); struct swap_cluster_info *ci =3D NULL; @@ -4019,8 +4017,6 @@ static int __folio_freeze_and_split_unmapped(struct f= olio *folio, unsigned int n struct lruvec *lruvec; int ret =3D 0; =20 - VM_WARN_ON_ONCE(!mapping && end); - if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) return -EAGAIN; =20 @@ -4035,24 +4031,75 @@ static int __folio_freeze_and_split_unmapped(struct= folio *folio, unsigned int n */ folio_reset_partially_mapped(folio); =20 - if (mapping) { + if (folio_test_swapcache(folio)) + ci =3D swap_cluster_get_and_lock(folio); + + if (do_lru) + lruvec =3D folio_lruvec_lock(folio); + + ret =3D __split_unmapped_folio(folio, new_order, split_at, NULL, + NULL, split_type); + + /* + * Unfreeze the after-split folios and put them back to the right + * place. Keep the head @folio frozen until the end: sub entries + * in swap cache must be updated first, so a concurrent + * swap_cache_get_folio() cannot return the head folio for a sub + * entry (folio_try_get() will fail on the head @folio until unfreeze). + */ + for (new_folio =3D folio_next(folio); new_folio !=3D end_folio; + new_folio =3D next) { + next =3D folio_next(new_folio); + zone_device_private_split_cb(folio, new_folio); + folio_ref_unfreeze(new_folio, + folio_cache_ref_count(new_folio) + 1); + if (do_lru) + lru_add_split_folio(folio, new_folio, lruvec, list); + if (ci) + __swap_cache_replace_folio(ci, folio, new_folio); + } + + zone_device_private_split_cb(folio, NULL); + folio_ref_unfreeze(folio, folio_cache_ref_count(folio) + 1); + + if (do_lru) + lruvec_unlock(lruvec); + if (ci) + swap_cluster_unlock(ci); + + return ret; +} + +static int __folio_freeze_split_file(struct folio *folio, + unsigned int new_order, struct page *split_at, + struct xa_state *xas, struct address_space *mapping, + bool do_lru, struct list_head *list, + enum split_type split_type, pgoff_t end, int *nr_shmem_dropped) +{ + struct folio *end_folio =3D folio_next(folio); + struct folio *new_folio, *next; + struct lruvec *lruvec; + int ret; + + /* Currently device private folios can only back anonymous memory. */ + VM_WARN_ON_ONCE_FOLIO(folio_is_device_private(folio), folio); + + if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) + return -EAGAIN; + + if (folio_test_pmd_mappable(folio) && + new_order < HPAGE_PMD_ORDER) { int nr =3D folio_nr_pages(folio); =20 - if (folio_test_pmd_mappable(folio) && - new_order < HPAGE_PMD_ORDER) { - if (folio_test_swapbacked(folio)) { - lruvec_stat_mod_folio(folio, - NR_SHMEM_THPS, -nr); - } else { - lruvec_stat_mod_folio(folio, - NR_FILE_THPS, -nr); - } + if (folio_test_swapbacked(folio)) { + lruvec_stat_mod_folio(folio, + NR_SHMEM_THPS, -nr); + } else { + lruvec_stat_mod_folio(folio, + NR_FILE_THPS, -nr); } } =20 - if (folio_test_swapcache(folio)) - ci =3D swap_cluster_get_and_lock(folio); - /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ if (do_lru) lruvec =3D folio_lruvec_lock(folio); @@ -4062,7 +4109,7 @@ static int __folio_freeze_and_split_unmapped(struct f= olio *folio, unsigned int n =20 /* * Unfreeze after-split folios and put them back to the right - * list. @folio should be kept frozon until page cache + * list. @folio should be kept frozen until page cache * entries are updated with all the other after-split folios * to prevent others seeing stale page cache entries. * As a result, new_folio starts from the next folio of @@ -4072,29 +4119,15 @@ static int __folio_freeze_and_split_unmapped(struct= folio *folio, unsigned int n new_folio =3D next) { unsigned long nr_pages =3D folio_nr_pages(new_folio); =20 + /* compute next before the folio can be freed below */ next =3D folio_next(new_folio); =20 - zone_device_private_split_cb(folio, new_folio); - folio_ref_unfreeze(new_folio, folio_cache_ref_count(new_folio) + 1); =20 if (do_lru) lru_add_split_folio(folio, new_folio, lruvec, list); =20 - /* - * Anonymous folio with swap cache. - * NOTE: shmem in swap cache is not supported yet. - */ - if (ci) { - __swap_cache_replace_folio(ci, folio, new_folio); - continue; - } - - /* Anonymous folio without swap cache */ - if (!mapping) - continue; - /* Add the new folio to the page cache. */ if (new_folio->index < end) { __xa_store(&mapping->i_pages, new_folio->index, @@ -4113,7 +4146,6 @@ static int __folio_freeze_and_split_unmapped(struct f= olio *folio, unsigned int n folio_put_refs(new_folio, nr_pages); } =20 - zone_device_private_split_cb(folio, NULL); /* * Unfreeze @folio only after all page cache entries, which * used to point to it, have been updated with new folios. @@ -4125,9 +4157,6 @@ static int __folio_freeze_and_split_unmapped(struct f= olio *folio, unsigned int n if (do_lru) lruvec_unlock(lruvec); =20 - if (ci) - swap_cluster_unlock(ci); - return ret; } =20 @@ -4269,7 +4298,10 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, =20 /* block interrupt reentry in xa_lock and spinlock */ local_irq_disable(); - if (mapping) { + if (is_anon) { + ret =3D __folio_freeze_split_anon(folio, new_order, split_at, + true, list, split_type); + } else { /* * Check if the folio is present in page cache. * We assume all tail are present too, if folio is there. @@ -4280,10 +4312,11 @@ static int __folio_split(struct folio *folio, unsig= ned int new_order, ret =3D -EAGAIN; goto fail; } + ret =3D __folio_freeze_split_file(folio, new_order, split_at, &xas, mapp= ing, + true, list, split_type, end, + &nr_shmem_dropped); } =20 - ret =3D __folio_freeze_and_split_unmapped(folio, new_order, split_at, &xa= s, mapping, - true, list, split_type, end, &nr_shmem_dropped); fail: if (mapping) xas_unlock(&xas); @@ -4383,9 +4416,8 @@ int folio_split_unmapped(struct folio *folio, unsigne= d int new_order) return -EAGAIN; =20 local_irq_disable(); - ret =3D __folio_freeze_and_split_unmapped(folio, new_order, &folio->page,= NULL, - NULL, false, NULL, SPLIT_TYPE_UNIFORM, - 0, NULL); + ret =3D __folio_freeze_split_anon(folio, new_order, &folio->page, + false, NULL, SPLIT_TYPE_UNIFORM); local_irq_enable(); return ret; } --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 81A22472522 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=H8pcYL+j7QEHN1BLKRFBFRiYlQcBXULVh2Zuov3fOpBWLZ6r/Uap1g4iap6iAH8pnh2ZSYLn/JU3b2AyOVVLrauWMgT2yfccEwfDWwwPcrNRGPSWLhS9Y6HBJIZ+phxy7VK2R7bdsgt2qgK6G/toMVZD01qVciKorLjk8Amr7gY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=lCnOjWIjAtP1yg4fNlobLAfUGV6MCr4aJdathae1sTw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=i/DJ873XqhYXc+EzSzWUpUZR7yV9uEUKGHDLpd4YXx9iSuaFqf+Kx7tHWml0LcfehYI2KVR7+HCx8yKGjrjlRfDJaLbbtvRktq+CiNw7oRxWqN2MabucUf6nigDzQ5AmVlxvdL+yl6k+jL3e45WXE9wIsawEaDI1uBIlVlixsAU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VQZ7swZ1; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VQZ7swZ1" Received: by smtp.kernel.org (Postfix) with ESMTPS id 35BC8C4AF12; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=lCnOjWIjAtP1yg4fNlobLAfUGV6MCr4aJdathae1sTw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=VQZ7swZ1RYYjOjBaVVU9PmAPrKsbC+Lkty1FobDdeVBCiT2TZPcDFR4Frisg5OS8h LpAX8GUOEMcGDrvSPKWWNxffAV2AweVArIWzIZntWvjHthyv1WIgJTt8MLHaSkmxFl a5RmJhY8fynEsneN15D5ei53S8uIbORzE4S8PtVEPtc5Z5BrlnEmt4+tU+HqPxb9sZ x59cwBKRfFlg4a7Id92WFYDmkCmjTQzNvFxitSctYpVPUTqDlGc4byFrxiWipXvKA7 O5uZd5tJn055798OweROyIuRtS5V8upj2ypopgk7Af70Sl7HgA3NiB/+fHqgdQmlVL O3oJyfZGWcw1Q== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C0C4C98311; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:36 +0200 Subject: [PATCH v6 05/17] mm/huge_memory: rename __split_unmapped_folio() to __split_frozen_folio() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-5-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=4280; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=0tFQGQYoeM+DEYcm38D1h/sxX+0NJVFyUi8PLZkhh8o=; b=HzFKI9Us9gEuTHBixDaUISCMUSTKiCmB0vtgkZgPKMJuArJ62XMZbcI4yupzzVIPE+Z+5MQns gTKTgOeI9CCBwI2yaWdoH2NHSWm0Y8Ca3OYTOY4NlbNEFnCJXmqXGZS X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song The helper splits a folio whose refcount is frozen: the frozen refcount is the state it relies on, while unmapping is arranged by the caller beforehand. The old name caused confusion and people may try to call the helper on non-frozen folios. Also add a VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio)) to self document that frozen implies unmapped. Suggested-by: Zi Yan Reviewed-by: Zi Yan Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1efd858892c1..5f54a3559e4e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3810,8 +3810,8 @@ static void __split_folio_to_order(struct folio *foli= o, int old_order, } =20 /** - * __split_unmapped_folio() - splits an unmapped @folio to lower order fol= ios in - * two ways: uniform split or non-uniform split. + * __split_frozen_folio() - splits a frozen @folio to lower order folios + * in two ways: uniform split or non-uniform split. * @folio: the to-be-split folio * @new_order: the smallest order of the after split folios (since buddy * allocator like split generates folios with orders from @fol= io's @@ -3850,7 +3850,7 @@ static void __split_folio_to_order(struct folio *foli= o, int old_order, * Return: 0 - successful, <0 - failed (if -ENOMEM is returned, @folio mig= ht be * split but not to @new_order, the caller needs to check) */ -static int __split_unmapped_folio(struct folio *folio, int new_order, +static int __split_frozen_folio(struct folio *folio, int new_order, struct page *split_at, struct xa_state *xas, struct address_space *mapping, enum split_type split_type) { @@ -3860,6 +3860,9 @@ static int __split_unmapped_folio(struct folio *folio= , int new_order, struct folio *old_folio =3D folio; int split_order; =20 + /* Frozen implies unmapped, callers unmap before splitting. */ + VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio); + /* * split to new_order one order at a time. For uniform split, * folio is split to new_order directly. @@ -4037,8 +4040,8 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, if (do_lru) lruvec =3D folio_lruvec_lock(folio); =20 - ret =3D __split_unmapped_folio(folio, new_order, split_at, NULL, - NULL, split_type); + ret =3D __split_frozen_folio(folio, new_order, split_at, NULL, + NULL, split_type); =20 /* * Unfreeze the after-split folios and put them back to the right @@ -4104,8 +4107,8 @@ static int __folio_freeze_split_file(struct folio *fo= lio, if (do_lru) lruvec =3D folio_lruvec_lock(folio); =20 - ret =3D __split_unmapped_folio(folio, new_order, split_at, xas, - mapping, split_type); + ret =3D __split_frozen_folio(folio, new_order, split_at, xas, + mapping, split_type); =20 /* * Unfreeze after-split folios and put them back to the right @@ -4169,9 +4172,9 @@ static int __folio_freeze_split_file(struct folio *fo= lio, * @list: after-split folios will be put on it if non NULL * @split_type: perform uniform split or not (non-uniform split) * - * It calls __split_unmapped_folio() to perform uniform and non-uniform sp= lit. + * It calls __split_frozen_folio() to perform uniform and non-uniform spli= t. * It is in charge of checking whether the split is supported or not and - * preparing @folio for __split_unmapped_folio(). + * preparing @folio for __split_frozen_folio(). * * After splitting, the after-split folio containing @lock_at remains lock= ed * and others are unlocked: @@ -4274,7 +4277,7 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, i_mmap_lock_read(mapping); =20 /* - *__split_unmapped_folio() may need to trim off pages beyond + * __split_frozen_folio() may need to trim off pages beyond * EOF: but on 32-bit, i_size_read() takes an irq-unsafe * seqlock, which cannot be nested inside the page tree lock. * So note end now: i_size itself may be changed at any moment, --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 81922468C16 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=nO1qYEXolPthbtiN1+8Xlx93VKxOU8f1eMZUaIma9PFs3c10Nrl82mBWYEgNg6yc/4M01nWplpCxQq6KmqHT6+SBqFF2xzPIEfvY7JuDoR25JK5K212UThfUOWMg3ccoBB0vUcA7wbR4tixR9ivkkBDnBqdjq7KIx/y4VciW+4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=kpGjokCilLiDdPLTyYRNKpK0lLQYtt0pXI1RqARDLw8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=mn4iDwaad1qDuLZ4RmTq4ndpZCv0I/lPCC0aLOQL2VFveCoaD47zsNPz1zz5ZpUZO6HhFf8o358mlrIiRZM17P0EfaswJF7klKGBp3vh/I60HZph8Cnf5OcQjwAtz1nGdRn0PgGifqArOcEz6wDpHZFKfYggaYU5ypW1joz/Wfs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=b2joWg8Z; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="b2joWg8Z" Received: by smtp.kernel.org (Postfix) with ESMTPS id 4D18AC2BCFB; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=kpGjokCilLiDdPLTyYRNKpK0lLQYtt0pXI1RqARDLw8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=b2joWg8ZXjX02ni3/gODQx15ScarpcWMqA4h0MRuFakU6R9+fcmT1VUFyeRsJitxB t1NFpP8tE6ukgf2Lw1tnm6ps55/Az6s5jaO3Upn4arkUtpEy5l1bbUUkpw+fFBjT3q VSXeFipnE/BbhqMG8KbaVdqstZhDXLfusrn40JKFWv6CLLRogNswv7rg2WzLRO7Nl+ ocOaBGRTL5IpcMYLAR6VDpUlHgyIvzZe22tCqBRr/vjSxoeA8Sc1ukyuDVw57ESb4o G1y+hu2u/HSHfCIi3OO6YspJQBmbhjOX800i7jTN45uYSTzfDPYqNbeQf9tj85JwQh p6aUrXrq6Hfug== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 320F8C98302; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:37 +0200 Subject: [PATCH v6 06/17] mm/huge_memory: consolidate irq and locking for folio split Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-6-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=4898; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=9801dxSFmr5iYbSNFCm6NGwAwyCbNb/aE79GzNHnX1k=; b=oYvZJrEg40ByRDiz932WhVRb/QVu57xS3KHFRX0EZQm0Q8hhaT4jdmyyS68GO78DUmvQofMt8 3G05ProET/VDBtsgYaqPJ4gWp9RnxTol4/AHwHcIV8HqvaOtvG1j3Mx X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song Let each split helper handle its own locking instead of relying on the caller, so both helpers manage their own irq and locking state. This lets __folio_split() drop its local irq handling and fail label, preparing for further cleanup. The file path now uses xas_lock_irq() instead of local_irq_disable() with xas_lock(). The two are equivalent on non-RT, and TRANSPARENT_HUGEPAGE cannot be enabled on RT anyway. This conversion also buys consistency: every other place in mm/ that freezes a folio while it is still reachable through the page cache already takes the lock this way. This was actually the last plain xas_lock() on mapping->i_pages left in mm. If we are going to support RT, spinning on frozen folio refs could be a problem, but it already exists in many places and should be fixed generically. The anon helper keeps a single local_irq_disable() as before, because it has to cover several plain spinlocks at once. The dropped xas_reset() was a no-op as the xa_state is not walked before the xas_load() under the lock. Reviewed-by: Zi Yan Reviewed-by: Kiryl Shutsemau (Meta) Reviewed-by: Yeoreum Yun Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 58 ++++++++++++++++++++++++++--------------------------= ---- 1 file changed, 27 insertions(+), 31 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 5f54a3559e4e..46712d3700cf 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4020,8 +4020,12 @@ static int __folio_freeze_split_anon(struct folio *f= olio, struct lruvec *lruvec; int ret =3D 0; =20 - if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) + local_irq_disable(); + + if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) { + local_irq_enable(); return -EAGAIN; + } =20 /* Take off the deferred split queue while frozen and memcg set */ folio_unqueue_deferred_split(folio); @@ -4069,6 +4073,7 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, lruvec_unlock(lruvec); if (ci) swap_cluster_unlock(ci); + local_irq_enable(); =20 return ret; } @@ -4087,8 +4092,21 @@ static int __folio_freeze_split_file(struct folio *f= olio, /* Currently device private folios can only back anonymous memory. */ VM_WARN_ON_ONCE_FOLIO(folio_is_device_private(folio), folio); =20 - if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) - return -EAGAIN; + xas_lock_irq(xas); + + /* + * Check if the folio is present in page cache. + * We assume all tail are present too, if folio is there. + */ + if (xas_load(xas) !=3D folio) { + ret =3D -EAGAIN; + goto fail; + } + + if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) { + ret =3D -EAGAIN; + goto fail; + } =20 if (folio_test_pmd_mappable(folio) && new_order < HPAGE_PMD_ORDER) { @@ -4160,6 +4178,8 @@ static int __folio_freeze_split_file(struct folio *fo= lio, if (do_lru) lruvec_unlock(lruvec); =20 +fail: + xas_unlock_irq(xas); return ret; } =20 @@ -4299,32 +4319,13 @@ static int __folio_split(struct folio *folio, unsig= ned int new_order, =20 unmap_folio(folio); =20 - /* block interrupt reentry in xa_lock and spinlock */ - local_irq_disable(); - if (is_anon) { + if (is_anon) ret =3D __folio_freeze_split_anon(folio, new_order, split_at, true, list, split_type); - } else { - /* - * Check if the folio is present in page cache. - * We assume all tail are present too, if folio is there. - */ - xas_lock(&xas); - xas_reset(&xas); - if (xas_load(&xas) !=3D folio) { - ret =3D -EAGAIN; - goto fail; - } + else ret =3D __folio_freeze_split_file(folio, new_order, split_at, &xas, mapp= ing, true, list, split_type, end, &nr_shmem_dropped); - } - -fail: - if (mapping) - xas_unlock(&xas); - - local_irq_enable(); =20 if (nr_shmem_dropped) shmem_uncharge(mapping->host, nr_shmem_dropped); @@ -4408,8 +4409,6 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, */ int folio_split_unmapped(struct folio *folio, unsigned int new_order) { - int ret =3D 0; - VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio); VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio); VM_WARN_ON_ONCE_FOLIO(!folio_test_large(folio), folio); @@ -4418,11 +4417,8 @@ int folio_split_unmapped(struct folio *folio, unsign= ed int new_order) if (folio_expected_ref_count(folio) !=3D folio_ref_count(folio) - 1) return -EAGAIN; =20 - local_irq_disable(); - ret =3D __folio_freeze_split_anon(folio, new_order, &folio->page, - false, NULL, SPLIT_TYPE_UNIFORM); - local_irq_enable(); - return ret; + return __folio_freeze_split_anon(folio, new_order, &folio->page, + false, NULL, SPLIT_TYPE_UNIFORM); } =20 /* --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 916B74734C7 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=VwJIliwyRGMK3T/peKb2QF6iZXynS8qzu5sYmzqjV8eZzJ6Rsm9wAOQRRX1wQGTU1BZsrQiA0SeP8+rSY2AR+v0r4DuzKNtYtycusCmXJjJwq28moOpQkwaqYUujKbgDqf/gul3Tx2fN1f6TwLL/TUDCctargYRIlhope46o3bU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=iTLZGVjgoHpMRdSYonjQeKB/M7uNxAVPULkag3MrBFQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rm/6XWmx6jNH21oe8/kFFKoCDIxRCaVgJriUWtfy/U0yTfuBIKOv5kgamu/K7MYfRcJn9mQmdhWXqETeC9r8i9q2D16FFWfF37RedGU1l0tbljJJZQg8rw/oIe0S1KAYP29OJtMK+YzI9hBIbzycV6P6+8Zgff0BJQUHjiPrdrA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ev/OJyw7; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ev/OJyw7" Received: by smtp.kernel.org (Postfix) with ESMTPS id 64A17C2BD00; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=iTLZGVjgoHpMRdSYonjQeKB/M7uNxAVPULkag3MrBFQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Ev/OJyw7/DMacbSTtZWNQHhKTwrXYmZPZ0IjruuD9qh9cDkOX9Xm1BBvkP5NOqq5U 2cAoLJSsz7IzLQjxuqmKe3qA+libAhhTX3ANujwZzeVGZ+X5EHH5FIfwdaKLrgcCDU gWdLhP2I1GI6l9tx7Ij+5X2s/ZJkpdkjjJ998n0Subs2PeXrkYKWosC/g/FISuOu91 RSkkl6VR5tTqLmWYWGO3bSyaMDjGAAgg88lbsoDI4PshSEocm21fESbbY6esxHJa6K hAt682SZIbRSH9j9/jeh1d6QNAAD3ZRsmv8LcOmTkXQ76peNLo4aZslyF8ftk6F0qz CJEID+Dbkp+kg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DAF3C9830D; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:38 +0200 Subject: [PATCH v6 07/17] mm/huge_memory: move EOF trimming into the file split helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-7-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=4114; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=Kmu1++mV7nc4Tr+6jZYhD8yAKOx3Sk1bhpFOnueBvVc=; b=zHrF5I+OHAffKORsRYYD2Fi/tvD9Tx8ghkvc9fkakSvwsaRzAUVK5dbIz/+pWxzosu1pKyhG6 Vdc0slLOwPRB0W0JdWLkiLyBvXIbQPXa3NYQM9O96MZJ/A9cPrQlgQq X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song Instead of receiving @end and @nr_shmem_dropped from the caller, the file split helper now computes the EOF boundary and trims pages beyond it itself, as this is only needed for file split. This drops the redundant parameter passing and sanity check. Reviewed-by: Zi Yan Acked-by: David Hildenbrand (Arm) Reviewed-by: Kiryl Shutsemau (Meta) Reviewed-by: Yeoreum Yun Signed-off-by: Kairui Song --- mm/huge_memory.c | 41 +++++++++++++++++++---------------------- 1 file changed, 19 insertions(+), 22 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 46712d3700cf..625e7987ff40 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4082,16 +4082,29 @@ static int __folio_freeze_split_file(struct folio *= folio, unsigned int new_order, struct page *split_at, struct xa_state *xas, struct address_space *mapping, bool do_lru, struct list_head *list, - enum split_type split_type, pgoff_t end, int *nr_shmem_dropped) + enum split_type split_type) { struct folio *end_folio =3D folio_next(folio); struct folio *new_folio, *next; + int nr_shmem_dropped =3D 0; struct lruvec *lruvec; + pgoff_t end; int ret; =20 /* Currently device private folios can only back anonymous memory. */ VM_WARN_ON_ONCE_FOLIO(folio_is_device_private(folio), folio); =20 + /* + * The loop below may need to trim off pages beyond + * EOF: but on 32-bit, i_size_read() takes an irq-unsafe + * seqlock, which cannot be nested inside the page tree lock. + * So note end now: i_size itself may be changed at any moment, + * but folio lock is good enough to serialize the trimming. + */ + end =3D DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); + if (shmem_mapping(mapping)) + end =3D shmem_fallocend(mapping->host, end); + xas_lock_irq(xas); =20 /* @@ -4156,10 +4169,9 @@ static int __folio_freeze_split_file(struct folio *f= olio, continue; } =20 - VM_WARN_ON_ONCE(!nr_shmem_dropped); /* Drop folio beyond EOF: ->index >=3D end */ - if (shmem_mapping(mapping) && nr_shmem_dropped) - *nr_shmem_dropped +=3D nr_pages; + if (shmem_mapping(mapping)) + nr_shmem_dropped +=3D nr_pages; else if (folio_test_clear_dirty(new_folio)) folio_account_cleaned( new_folio, inode_to_wb(mapping->host)); @@ -4180,6 +4192,8 @@ static int __folio_freeze_split_file(struct folio *fo= lio, =20 fail: xas_unlock_irq(xas); + if (nr_shmem_dropped) + shmem_uncharge(mapping->host, nr_shmem_dropped); return ret; } =20 @@ -4216,9 +4230,7 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, struct anon_vma *anon_vma =3D NULL; int old_order =3D folio_order(folio); struct folio *new_folio, *next; - int nr_shmem_dropped =3D 0; enum ttu_flags ttu_flags =3D 0; - pgoff_t end =3D 0; int ret; =20 VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio); @@ -4295,17 +4307,6 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, =20 anon_vma =3D NULL; i_mmap_lock_read(mapping); - - /* - * __split_frozen_folio() may need to trim off pages beyond - * EOF: but on 32-bit, i_size_read() takes an irq-unsafe - * seqlock, which cannot be nested inside the page tree lock. - * So note end now: i_size itself may be changed at any moment, - * but folio lock is good enough to serialize the trimming. - */ - end =3D DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); - if (shmem_mapping(mapping)) - end =3D shmem_fallocend(mapping->host, end); } =20 /* @@ -4324,11 +4325,7 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, true, list, split_type); else ret =3D __folio_freeze_split_file(folio, new_order, split_at, &xas, mapp= ing, - true, list, split_type, end, - &nr_shmem_dropped); - - if (nr_shmem_dropped) - shmem_uncharge(mapping->host, nr_shmem_dropped); + true, list, split_type); =20 if (!ret && is_anon && !folio_is_device_private(folio)) ttu_flags =3D TTU_USE_SHARED_ZEROPAGE; --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A39644746A4 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=l+pTglRfBSeySnRovqVrx9UnF3njq9mzZ8itJAH4+E8We/N+NYU/ske2Pi1hzEs8W4wcjMdp3KYlMS2HwHgwXVJWikLEv+FqeoBegq64i9hMjpDLDmMq7qM0hgp2EFpbjpx+dLkS+VZ7tvwVtLlktnm7zdeOXBbJksAXpdqmLew= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=cBInc2Y150gnR9QShVRFXkTGJvi1jKLY1kMb8gVdedk=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=SGCsMwh2nuci+TRZhm10Lnqv11oA6Xdli0jSAb7rWF/gLZUi0tCwkFPXc7Hveqq1Ru7jW31CcsoG/tbYHnLa/QWAqvMGZ7arjryFZy0HQJ4YzeU5JahYVjhewD3IADx0LnqsSSocDzbqZtxMX8bcM5iJ5AhxQdWKhanIVT+81PM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZXhznrxP; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZXhznrxP" Received: by smtp.kernel.org (Postfix) with ESMTPS id 79EB7C2BCFF; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=cBInc2Y150gnR9QShVRFXkTGJvi1jKLY1kMb8gVdedk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=ZXhznrxPcf2EstKnav+Pa3DT7m31Go+xL3r5SL3BQBI4zv27ActRYMFiHs0dNP3cZ xfj2BtdiwGLa/ppCC/SRKujeCXImEXws+q8CRtAQsO4PWmcTI+PJcB0huqugQ5nu+t xkwzZU0mFjb0NXumMxDItUfq52YnPJnItGYmPXRDce7hk/eV6Nv34IxMpp9rN6c4c7 E81F1kHF/JQPffVdRMmVWMdhI9HNjk9Zp4cSOuD3Gmyb43DinchPrCyNzDL9Lx6YJh xF6Nti8agQnvRzAr8lDwdvfFfXI1uVHdX8ZTAuIwO3g433FDUL+PROlGTw4dL/xpr9 vQPL3CplyeXzw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65D5CC98312; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:39 +0200 Subject: [PATCH v6 08/17] mm/huge_memory: move unmap and remap into the split helpers Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-8-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=3700; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=bG37iOhPck6tDgOBAYXQonkZfp1LV5/lDeTBdSogHrU=; b=8K1WXT22OIwKUAR2XaU3sTnYjG39/waxHTBKElGxAkw++akKW5L/02mUG2pKRHVLdsftyZjOA cog7IfhNcd+DQDhBoo/MyGV4KGyGgowBxJoahQDy6No6z8kLowN2raZ X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song To prepare for further cleanup, move the unmap/remap handling from __folio_split() into the split helpers. Only anon folios need to be remapped, so remap_page() is now only called for anon splits and the anon check in remap_page() is redundant and can be removed. Reviewed-by: Zi Yan Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 625e7987ff40..7ee9eb2e1e93 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3640,9 +3640,6 @@ static void remap_page(struct folio *folio, unsigned = long nr, int flags) { int i =3D 0; =20 - /* If unmap_folio() uses try_to_migrate() on file, remove this check */ - if (!folio_test_anon(folio)) - return; for (;;) { remove_migration_ptes(folio, folio, TTU_RMAP_LOCKED | flags); i +=3D folio_nr_pages(folio); @@ -4016,15 +4013,23 @@ static int __folio_freeze_split_anon(struct folio *= folio, { struct folio *end_folio =3D folio_next(folio); struct swap_cluster_info *ci =3D NULL; + const int old_order =3D folio_order(folio); struct folio *new_folio, *next; + enum ttu_flags ttu_flags =3D 0; struct lruvec *lruvec; + bool need_remap =3D false; int ret =3D 0; =20 + if (folio_mapped(folio)) { + need_remap =3D true; + unmap_folio(folio); + } + local_irq_disable(); =20 if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) { - local_irq_enable(); - return -EAGAIN; + ret =3D -EAGAIN; + goto out_no_split; } =20 /* Take off the deferred split queue while frozen and memcg set */ @@ -4073,7 +4078,13 @@ static int __folio_freeze_split_anon(struct folio *f= olio, lruvec_unlock(lruvec); if (ci) swap_cluster_unlock(ci); +out_no_split: local_irq_enable(); + if (need_remap) { + if (!ret && !folio_is_device_private(folio)) + ttu_flags =3D TTU_USE_SHARED_ZEROPAGE; + remap_page(folio, 1 << old_order, ttu_flags); + } =20 return ret; } @@ -4105,6 +4116,8 @@ static int __folio_freeze_split_file(struct folio *fo= lio, if (shmem_mapping(mapping)) end =3D shmem_fallocend(mapping->host, end); =20 + unmap_folio(folio); + xas_lock_irq(xas); =20 /* @@ -4189,7 +4202,6 @@ static int __folio_freeze_split_file(struct folio *fo= lio, =20 if (do_lru) lruvec_unlock(lruvec); - fail: xas_unlock_irq(xas); if (nr_shmem_dropped) @@ -4230,7 +4242,6 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, struct anon_vma *anon_vma =3D NULL; int old_order =3D folio_order(folio); struct folio *new_folio, *next; - enum ttu_flags ttu_flags =3D 0; int ret; =20 VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio); @@ -4318,8 +4329,6 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, goto out_unlock; } =20 - unmap_folio(folio); - if (is_anon) ret =3D __folio_freeze_split_anon(folio, new_order, split_at, true, list, split_type); @@ -4327,11 +4336,6 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, ret =3D __folio_freeze_split_file(folio, new_order, split_at, &xas, mapp= ing, true, list, split_type); =20 - if (!ret && is_anon && !folio_is_device_private(folio)) - ttu_flags =3D TTU_USE_SHARED_ZEROPAGE; - - remap_page(folio, 1 << old_order, ttu_flags); - /* * Drop the mapping while the inode is still pinned. @folio stays * locked and present in the page cache until the loop below, so --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8F5F47533E for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=LAhvxPQl4rjDM8qrUqfg8VGy8GkX7FDe/fy/NooTrFgGfhZsaz/+VG4vta75CgMGc5J872X7whS+GjSBU7z0VH7W5hvtsM4PUUIUGX7YrKRPg3usnic2XQweJLIOTMlr+86FuvEcMJbVTCiDqpA4CDIBdr1BMdNyWpD+uJ8Aopc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=HryFPDO+o0DPrLt3FJ0uwMdDvPOPxQxH9aBcZB4iPP0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=UrxEbkwPu+HkvM1ISIOi3T8PFfjH6xwJX7pc4PLpTliVQx9wv+ANGMcQOnU+oj85aFaQx7M7aqUbEWyS276rV7zWBWiTE0u2sMoofh6fmh/NyCJ71Z4rUJFbCtla6jKbUKpszG31HjZmublbuS/+8fgUJ57XBpEDPZ6KJLNYRoU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UO6CmGIq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UO6CmGIq" Received: by smtp.kernel.org (Postfix) with ESMTPS id 99064C2BCB8; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=HryFPDO+o0DPrLt3FJ0uwMdDvPOPxQxH9aBcZB4iPP0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=UO6CmGIq+mt4WHgZKtR/B01p7dLt6r9MdFjKo8cir3xX4AcYUekqg474uIIfqDvyr d2sRPPIt40vOapgBLnLJMBN0vE+EeNb4CevOP9qqFO5Tqe56aLdK8rq6k+Z7KBFngO H+06bdg5lG41pKPG5XCd3t2A0AfB/WgzKmAoiNql/vgTu95RZ/C682abqnP2czTVVs brrlKgOtO14oLfjrA260m+6cI6uiISgH3A+J2YcyrZxV8MBKk3IRlbslW38uX9fzR7 dkEas21DAcP3NvHbJ4D2bYaqCxN0L7ObLv95v1HXoY5UFp/+emJ/1ZdKlXlOd7meDq seCWS0PeX935w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CF58C98304; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:40 +0200 Subject: [PATCH v6 09/17] mm/huge_memory: rename remap_page() to remap_anon_folio() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-9-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=2828; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=gWQY+D0TFANvAu4vbrZhmhS3l7U0TOtTOCxVNTsk234=; b=BnJau7sxut6tkcndu+7uN8tKe/gVNjBYe4mrrOlSP5w/jkUSgTJAegMGiutUDwlxnWLVU6V5U OY05PM3iqbGBXLe00rOUGnTFxtoYX7RbhZihp8PqlzsUYUkuIN+KmbM X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song remap_page() now only has one caller, __folio_freeze_split_anon(), and is only ever called for anon folios: unmap_folio() currently leaves file folios unmapped after the split, so they need no remapping. Rename it to remap_anon_folio() to make that explicit, and add a VM_WARN_ON_FOLIO() documenting it. Reviewed-by: Zi Yan Reviewed-by: Kiryl Shutsemau (Meta) Reviewed-by: Yeoreum Yun Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 7ee9eb2e1e93..fa74fada0659 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3551,7 +3551,6 @@ static void unmap_folio(struct folio *folio) /* * Anon pages need migration entries to preserve them, but file * pages can simply be left unmapped, then faulted back on demand. - * If that is ever changed (perhaps for mlock), update remap_page(). */ if (folio_test_anon(folio)) try_to_migrate(folio, ttu_flags); @@ -3636,10 +3635,17 @@ bool unmap_huge_pmd_locked(struct vm_area_struct *v= ma, unsigned long addr, return __discard_anon_folio_pmd_locked(vma, addr, pmdp, folio); } =20 -static void remap_page(struct folio *folio, unsigned long nr, int flags) +static void remap_anon_folio(struct folio *folio, unsigned long nr, int fl= ags) { int i =3D 0; =20 + /* + * unmap_folio() installs migration entries only for anon folios, + * so currently only anon folios need to be remapped. File folios + * stay unmapped after the split and are faulted back on demand. + */ + VM_WARN_ON_FOLIO(!folio_test_anon(folio), folio); + for (;;) { remove_migration_ptes(folio, folio, TTU_RMAP_LOCKED | flags); i +=3D folio_nr_pages(folio); @@ -3723,7 +3729,7 @@ static void __split_folio_to_order(struct folio *foli= o, int old_order, * * Note that for mapped sub-pages of an anonymous THP, * PG_anon_exclusive has been cleared in unmap_folio() and is stored in - * the migration entry instead from where remap_page() will restore it. + * the migration entry instead from where remap_anon_folio() will restor= e it. * We can still have PG_anon_exclusive set on effectively unmapped and * unreferenced sub-pages of an anonymous THP: we can simply drop * PG_anon_exclusive (-> PG_mappedtodisk) for these here. @@ -4083,7 +4089,7 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, if (need_remap) { if (!ret && !folio_is_device_private(folio)) ttu_flags =3D TTU_USE_SHARED_ZEROPAGE; - remap_page(folio, 1 << old_order, ttu_flags); + remap_anon_folio(folio, 1 << old_order, ttu_flags); } =20 return ret; --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CD589476041 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; cv=none; b=pKMdfHc502t5b4yFNzvRfriuRefsS2Uhc6xYXung333RatE9cw7HaZmLnnLNaNA/s2e8sUWpTCfRU0gknMWEd4642o2u8Lrb5NhCQ9EYOizc1NsFUbsuOlDcwYL7ksWI8Y8EM7wHmWNt5VQy2Plgiug9HavcB2i2n0wZtA+Ypd4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199163; c=relaxed/simple; bh=3Q/WGYCD2V4+Spm/P1lmcpsO7ecYPfZ+PAIr13xEzSw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=rd1143H+0d2rqGXYIkuh388Lx6Rwvcfip8MB1NlxM0M04hMz3x2FVMRYk+v21jDn5wNybUs01ZqaLznf+Z/eDRxOoQnJpUCCBOCy7fFVi1jr3mG05ZaN3Q393s3vMDLKimPsKU4ogxEzOc0C/DXG6dOlvEUkaClEb9i+g7E4CIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oGvKKika; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oGvKKika" Received: by smtp.kernel.org (Postfix) with ESMTPS id ADDA8C2BCF7; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=3Q/WGYCD2V4+Spm/P1lmcpsO7ecYPfZ+PAIr13xEzSw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=oGvKKikaOYFs5Ma4mmzfDRa6aQCJwUQIQjkmaR3o6FUuI3i45nQ765hIpbn3IN8PJ eNE6tN6gmIoqW8QX8ihVMjLwVuEPVmrPnHqm38qo7EBhGnKw7YkSbFpoWtwtbLhBbi XNF8eh9RpCYbXwuxF6SFiPqlkYrvaJoxoSEdDkFodEvPBIdR0EinxO8PyZql2Wk3kY 7u/R2AFxoJLkkwa9gvNWHPNmkRHRbuw743QF4QLz2WBryMXHkeIqvkPhqSs6rSPFuh tomOPbgkrKqFaOJM2AWQ/jwVsBKun2PnC0uuJCnxZL0NO+WOyw0lJZpWfnV/f1rvUi uETfjkMhZZs+g== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98FD1C98311; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:41 +0200 Subject: [PATCH v6 10/17] mm/huge_memory: move the racy refcount check into unmap_folio() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-10-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=3793; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=h+PSQR1ffInIFUefUIt17YL1Yscv/2dT4X172Kq2ITc=; b=xrN0J9iwjrwqLhGirP8N5AQfDTl3N6APTqFL2dMa1Und10j1hlQbs5DqMisUFV8QHMDpllxhF OxyT4E+/2+IC3b9IhRgkeLAqnlYnSVM/x410yWjo+ygQPZSgbqM1irA X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song The check only exists to avoid the expensive PMD-splitting unmap of a folio that cannot be split anyway. Move it from __folio_split() and folio_split_unmapped() into one check in unmap_folio(), right before the PMD split. All split helpers get the same early check without repeating it. unmap_folio() now returns -EAGAIN if the check fails and the split helpers propagate the error. folio_split_unmapped() drops its own copy of the check: it works on already unmapped folios and the definitive folio_ref_freeze() in __folio_freeze_split_anon() still catches unexpected references. Reviewed-by: Zi Yan Reviewed-by: Kiryl Shutsemau (Meta) Reviewed-by: Yeoreum Yun Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index fa74fada0659..d738b25f50dc 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3538,13 +3538,22 @@ void vma_adjust_trans_huge(struct vm_area_struct *v= ma, split_huge_pmd_if_needed(next, end); } =20 -static void unmap_folio(struct folio *folio) +/* + * A return value of 0 does not mean that unmapping succeeded. It might + * still have failed, but remap_anon_folio() must be called afterwards, + * for anon folios. + */ +static int unmap_folio(struct folio *folio) { enum ttu_flags ttu_flags =3D TTU_RMAP_LOCKED | TTU_SYNC | TTU_BATCH_FLUSH; =20 VM_BUG_ON_FOLIO(!folio_test_large(folio), folio); =20 + /* Racy check if we can split the page, before we split PMDs */ + if (folio_expected_ref_count(folio) !=3D folio_ref_count(folio) - 1) + return -EAGAIN; + if (folio_test_pmd_mappable(folio)) ttu_flags |=3D TTU_SPLIT_HUGE_PMD; =20 @@ -3558,6 +3567,8 @@ static void unmap_folio(struct folio *folio) try_to_unmap(folio, ttu_flags | TTU_IGNORE_MLOCK); =20 try_to_unmap_flush(); + + return 0; } =20 static bool __discard_anon_folio_pmd_locked(struct vm_area_struct *vma, @@ -4028,7 +4039,9 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, =20 if (folio_mapped(folio)) { need_remap =3D true; - unmap_folio(folio); + ret =3D unmap_folio(folio); + if (ret) + return ret; } =20 local_irq_disable(); @@ -4122,7 +4135,9 @@ static int __folio_freeze_split_file(struct folio *fo= lio, if (shmem_mapping(mapping)) end =3D shmem_fallocend(mapping->host, end); =20 - unmap_folio(folio); + ret =3D unmap_folio(folio); + if (ret) + return ret; =20 xas_lock_irq(xas); =20 @@ -4326,15 +4341,6 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, i_mmap_lock_read(mapping); } =20 - /* - * Racy check if we can split the page, before unmap_folio() will - * split PMDs - */ - if (folio_expected_ref_count(folio) !=3D folio_ref_count(folio) - 1) { - ret =3D -EAGAIN; - goto out_unlock; - } - if (is_anon) ret =3D __folio_freeze_split_anon(folio, new_order, split_at, true, list, split_type); @@ -4373,7 +4379,6 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, free_folio_and_swap_cache(new_folio); } =20 -out_unlock: if (anon_vma) { anon_vma_unlock_write(anon_vma); put_anon_vma(anon_vma); @@ -4421,9 +4426,6 @@ int folio_split_unmapped(struct folio *folio, unsigne= d int new_order) VM_WARN_ON_ONCE_FOLIO(!folio_test_large(folio), folio); VM_WARN_ON_ONCE_FOLIO(!folio_test_anon(folio), folio); =20 - if (folio_expected_ref_count(folio) !=3D folio_ref_count(folio) - 1) - return -EAGAIN; - return __folio_freeze_split_anon(folio, new_order, &folio->page, false, NULL, SPLIT_TYPE_UNIFORM); } --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E422747605E for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; cv=none; b=tN9RNgy6Aw7c1VSpyKwzYJJ0hnoMRgWCuz5hivOJBkyfuxZbCXCBi1OQ+TGLtirgYJdddBwerZnT6alv+sPaxpw7Kn3zZ5XXo9EeW5S60SpGSaH0BWLY1ZvfKl9h/Ntj3W/sibrLvqW5Hx525lHV9K+rsXQvXj8ZcEuuuexQPvI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; c=relaxed/simple; bh=tCuZEPVVgjW53lTmDy3GfqdEs5Dr1bwfAnf2kMd3J38=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Yw1bsCXNa2LbmQA2KQW+uA4Ojq57fdoY9kuvytXeVnj2NMm5MS+Xs4N3bY0r7ZqPldiIlL1344XStIl6MvxYyvLrC9vQEIjHtgffNPZV4HHqWo/Sg6dluNZIQxuYttf/Cc7MWwi8UwwaWAL+C9VhA9sAOz5PL/UVSdYCNwlOALU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=PBNDBe/6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="PBNDBe/6" Received: by smtp.kernel.org (Postfix) with ESMTPS id C4670C2BCF5; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=tCuZEPVVgjW53lTmDy3GfqdEs5Dr1bwfAnf2kMd3J38=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=PBNDBe/6RctRVoEiJGVpt1aOYVYLIp7QAPU41UMqtaocb2nihwcFPnLVAWUwr14x8 UxrDielLUYnzDWmp1gTL0D+wpe+mx9hdrt1hpt/55OE1ai2WDYZIG8qoAlKeWk152u +blQVHRZe4Era/HGUzMCzvi4SUIvB1ftodyxZWb7XpvazxtUIbTNgKlsY0un+TeOd9 uktF2eDk7Ky/JXtsdHzdVcMMhAzavwFzFV+CdiT9kpiqXCi0s/KpMg7QcGJlAWZ+xv MxpGRO2w5R7XtR79nzFjCEj4KS9eF9UkkpvX1t23PCMf+lbnj8Sf2YzvA49SeSw+ps haIVcD0YzkdFA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD923C9830D; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:42 +0200 Subject: [PATCH v6 11/17] mm/huge_memory: move filemap management into the file split helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-11-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=6383; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=8pTRgHH/mO2XdrGT65s0vaWhb44uyEw/ZQxVpZoD/iU=; b=ltWPhfqdVU33z+Cs4byEFOCxy5U7xMheOpjHCVpuxXGwYl6C2Qgpte23bfCIWbL+UWfuBU2K8 NptI7VZ2y8SD8Ef9v7Ey4NMv7me6sC4VBcOvfVg6sRAB548Ssydq3Od X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song Only file split needs the filemap and xarray handling and related variables. Move them out of __folio_split() into the file helper so the helper is self-contained, and simplify the parameters. No functional change. Reviewed-by: Zi Yan Reviewed-by: Kiryl Shutsemau (Meta) Reviewed-by: Yeoreum Yun Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 102 ++++++++++++++++++++++++---------------------------= ---- 1 file changed, 44 insertions(+), 58 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d738b25f50dc..24073f75e87b 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4110,16 +4110,42 @@ static int __folio_freeze_split_anon(struct folio *= folio, =20 static int __folio_freeze_split_file(struct folio *folio, unsigned int new_order, struct page *split_at, - struct xa_state *xas, struct address_space *mapping, bool do_lru, struct list_head *list, enum split_type split_type) { + struct address_space *mapping =3D folio->mapping; + XA_STATE(xas, &mapping->i_pages, folio->index); struct folio *end_folio =3D folio_next(folio); struct folio *new_folio, *next; int nr_shmem_dropped =3D 0; + unsigned int min_order; struct lruvec *lruvec; pgoff_t end; - int ret; + gfp_t gfp; + int ret =3D 0; + + min_order =3D mapping_min_folio_order(mapping); + if (new_order < min_order) + return -EINVAL; + + gfp =3D current_gfp_context(mapping_gfp_mask(mapping) & GFP_RECLAIM_MASK); + if (!filemap_release_folio(folio, gfp)) + return -EBUSY; + + mapping_set_update(&xas, mapping); + + if (split_type =3D=3D SPLIT_TYPE_UNIFORM) { + const int old_order =3D folio_order(folio); + + xas_set_order(&xas, folio->index, new_order); + xas_split_alloc(&xas, folio, old_order, gfp); + if (xas_error(&xas)) { + ret =3D xas_error(&xas); + goto fail_free; + } + } + + i_mmap_lock_read(mapping); =20 /* Currently device private folios can only back anonymous memory. */ VM_WARN_ON_ONCE_FOLIO(folio_is_device_private(folio), folio); @@ -4137,15 +4163,15 @@ static int __folio_freeze_split_file(struct folio *= folio, =20 ret =3D unmap_folio(folio); if (ret) - return ret; + goto fail_mmap_unlock; =20 - xas_lock_irq(xas); + xas_lock_irq(&xas); =20 /* * Check if the folio is present in page cache. * We assume all tail are present too, if folio is there. */ - if (xas_load(xas) !=3D folio) { + if (xas_load(&xas) !=3D folio) { ret =3D -EAGAIN; goto fail; } @@ -4172,7 +4198,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, if (do_lru) lruvec =3D folio_lruvec_lock(folio); =20 - ret =3D __split_frozen_folio(folio, new_order, split_at, xas, + ret =3D __split_frozen_folio(folio, new_order, split_at, &xas, mapping, split_type); =20 /* @@ -4224,9 +4250,19 @@ static int __folio_freeze_split_file(struct folio *f= olio, if (do_lru) lruvec_unlock(lruvec); fail: - xas_unlock_irq(xas); + xas_unlock_irq(&xas); +fail_mmap_unlock: if (nr_shmem_dropped) shmem_uncharge(mapping->host, nr_shmem_dropped); + /* + * Drop the mapping while the inode is still pinned. @folio stays + * locked and present in the page cache, so eviction cannot free + * the inode yet, nothing past this point may touch the inode or + * the mapping. + */ + i_mmap_unlock_read(mapping); +fail_free: + xas_destroy(&xas); return ret; } =20 @@ -4255,11 +4291,9 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, struct page *split_at, struct page *lock_at, struct list_head *list, enum split_type split_type) { - XA_STATE(xas, &folio->mapping->i_pages, folio->index); struct folio *end_folio =3D folio_next(folio); bool is_anon =3D folio_test_anon(folio); struct mem_cgroup *memcg, *old_memcg; - struct address_space *mapping =3D NULL; struct anon_vma *anon_vma =3D NULL; int old_order =3D folio_order(folio); struct folio *new_folio, *next; @@ -4306,60 +4340,15 @@ static int __folio_split(struct folio *folio, unsig= ned int new_order, goto out; } anon_vma_lock_write(anon_vma); - mapping =3D NULL; - } else { - unsigned int min_order; - gfp_t gfp; - - mapping =3D folio->mapping; - min_order =3D mapping_min_folio_order(mapping); - if (new_order < min_order) { - ret =3D -EINVAL; - goto out; - } - - gfp =3D current_gfp_context(mapping_gfp_mask(mapping) & - GFP_RECLAIM_MASK); - - if (!filemap_release_folio(folio, gfp)) { - ret =3D -EBUSY; - goto out; - } - - mapping_set_update(&xas, mapping); - - if (split_type =3D=3D SPLIT_TYPE_UNIFORM) { - xas_set_order(&xas, folio->index, new_order); - xas_split_alloc(&xas, folio, old_order, gfp); - if (xas_error(&xas)) { - ret =3D xas_error(&xas); - goto out; - } - } - - anon_vma =3D NULL; - i_mmap_lock_read(mapping); } =20 if (is_anon) ret =3D __folio_freeze_split_anon(folio, new_order, split_at, true, list, split_type); else - ret =3D __folio_freeze_split_file(folio, new_order, split_at, &xas, mapp= ing, + ret =3D __folio_freeze_split_file(folio, new_order, split_at, true, list, split_type); =20 - /* - * Drop the mapping while the inode is still pinned. @folio stays - * locked and present in the page cache until the loop below, so - * eviction cannot free the inode yet; @lock_at is not enough, it may - * be a tail beyond EOF that the split already dropped from the page - * cache. Nothing past this point may touch the inode or the mapping. - */ - if (mapping) { - i_mmap_unlock_read(mapping); - mapping =3D NULL; - } - /* * Unlock all after-split folios except the one containing * @lock_at page. If @folio is not split, it will be kept locked. @@ -4383,14 +4372,11 @@ static int __folio_split(struct folio *folio, unsig= ned int new_order, anon_vma_unlock_write(anon_vma); put_anon_vma(anon_vma); } - if (mapping) - i_mmap_unlock_read(mapping); out: /* restore to caller's old_memcg */ set_active_memcg(old_memcg); mem_cgroup_put(memcg); out_no_memcg: - xas_destroy(&xas); if (is_pmd_order(old_order)) count_vm_event(!ret ? THP_SPLIT_PAGE : THP_SPLIT_PAGE_FAILED); count_mthp_stat(old_order, !ret ? MTHP_STAT_SPLIT : MTHP_STAT_SPLIT_FAILE= D); --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F36644766B1 for ; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; cv=none; b=R9sC7pAB5GDCCNnaE2OMsE42KFrnMRCCTab4Ojh4UjsbyHpLhYMDtPNSwG9PN8+kLWLBFwMn25kOJEi/IRaT5tfc3djL8wk34SogTboOFgsXSP4vNf5Yq7qj4a1idNn74cWcl69zAAdDMMe7zCBG6P+c1X4dluxkBvuB6/sB+Lw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; c=relaxed/simple; bh=G6lzysDx7CRWXFJdUsrK5RErF2sIGfTamezuD6h7XWY=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dvU7hopbncC5I2Jq7rVrWR4hB+qzB8TRBM/MqKzMo8w/dRv/XPAiZlxZyEbmtH0OUkBLEBBwubO5m56SM7NPbg+2btvnYtzip4Zjyy5KKt5Tx/VROD5HdHgGkkkX2A+QPN8OIe3ZAL9CIm4ebXBi3yGvgmC+RFcIq8fDo1MogG8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=L4RrsQ8w; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="L4RrsQ8w" Received: by smtp.kernel.org (Postfix) with ESMTPS id D5F31C2BCFA; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199163; bh=G6lzysDx7CRWXFJdUsrK5RErF2sIGfTamezuD6h7XWY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=L4RrsQ8wCChdTjex3TrlsGudelzPqwIV3UjNqe+KE9EP1qMjvdkb9j8ybccP5VMCe GW1Dq3/7j+stFXqG+2rX6D/5l+R5AqCOLSRwCnY1M9xEv9fnAtGDya6HhnjJbT234c SRIc2YS44WmrrEWveozkHCD8TDRv7kixxBKQ4zH58/q4Mlary/DuzQNpSOhS3+etXw 7vitEDUpJGbRQotXu6f2TNt4ig5oOMKFVUbKqn328Fu0gr9/GUCuC87HqGg7zOguDD 5PlDEzXTRr5CQJ90ZsyZLsnpD23Ks4Rb2f9zHCmKyprw0hjjyFS97wnoiIrhfr7ZRR TFwnjc51LpxiA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1EDEC98312; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:43 +0200 Subject: [PATCH v6 12/17] mm/huge_memory: move anon_vma handling into the anon split helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-12-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=5852; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=LJ+5qeAIckMWthAZegpdl6LRllp3JqAkUk0Cnq/gqes=; b=qnfjZNR9a2xlXxe0wVn/8SYFjU6JKiNDw8GL2w4GPXE6B/0t3thsDsK7G474P2HAB1uwXJDJl 3bonPrBDXeWDMvoWV+UDBd9CwDxKjoLK8i4C9TU39HmFZDct4GVCDVV X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song Only anon split needs the anon_vma, and it only needs it to unmap and remap. Move the folio_get_anon_vma()/anon_vma_lock_write() pair out of __folio_split() into the anon helper next to the folio_mapped() check that already gates unmap_folio(). This makes the anon_vma conditional on folio_mapped(), which is a behaviour change but should be fine. folio_get_anon_vma() returns NULL whenever !folio_mapped(), so an anon folio with folio_mapcount() =3D=3D 0 used to get -EBUSY from split_huge_page() and is now split instead. A realistic case is a THP that has been fully swapped out and is still in the swap cache: swap PTEs do not contribute mapcount, so it is !folio_mapped() but still alive. That should be safe and right to have because: - folio_ref_freeze() below still rejects a folio that picked up any reference, a mapping or a GUP pin, in the meantime. - A parallel split is excluded by the folio lock. The anon_vma write lock was added to serialize split in commit 062f1af2170a ("mm: thp: acquire the anon_vma rwsem for write during split"), when split_huge_page() did not hold the folio lock throughout. commit e9b61f19858a ("thp: reintroduce split_huge_page()") later made the folio lock a caller requirement and added the folio_ref_freeze() scheme, so that has been covered ever since. - Unmapped path is already exercised by folio_split_unmapped(), and the swap cache split already runs well for a partially swapped-out mapped THP. - A !folio_mapped() folio cannot become mapped meanwhile: mapping it requires the folio lock. For mapped folios the anon_vma write lock is now released before __folio_split() unlocks the after-split sub-folios, where previously it was held across that loop; that window is harmless as the sub-folios stay folio-locked and referenced until it. Reviewed-by: Zi Yan Reviewed-by: Yeoreum Yun Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 57 +++++++++++++++++++++++++++++-----------------------= ---- 1 file changed, 30 insertions(+), 27 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 24073f75e87b..62e2879cd3b4 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4032,16 +4032,37 @@ static int __folio_freeze_split_anon(struct folio *= folio, struct swap_cluster_info *ci =3D NULL; const int old_order =3D folio_order(folio); struct folio *new_folio, *next; + struct anon_vma *anon_vma =3D NULL; enum ttu_flags ttu_flags =3D 0; struct lruvec *lruvec; - bool need_remap =3D false; int ret =3D 0; =20 + /* + * Unmap/remap needs the anon_vma, so we first take a reference on + * it to prevent it from disappearing, and lock it for write here, + * letting unmap_folio() walk the rmap with TTU_RMAP_LOCKED. + * + * folio_mapped() is not stable here, but it can only change in + * one direction while the folio is locked. The mapcount can drop + * to zero at any time, zap_pte_range() takes no folio lock. It + * cannot go up: swapin, migration and uffd move all lock the folio + * before mapping it, and fork only copies PTEs that already exist. + * + * So if we see the folio mapped, the worst case is an empty rmap + * walk. If we see it unmapped, it stays unmapped and needs neither + * the reference nor the lock. Anything else needs a reference + * first and folio_ref_freeze() below catches it. + * + * Note that entirely swapped-out THPs are unmapped but can be split. + */ if (folio_mapped(folio)) { - need_remap =3D true; + anon_vma =3D folio_get_anon_vma(folio); + if (!anon_vma) + return -EBUSY; + anon_vma_lock_write(anon_vma); ret =3D unmap_folio(folio); if (ret) - return ret; + goto out_unlock; } =20 local_irq_disable(); @@ -4099,11 +4120,16 @@ static int __folio_freeze_split_anon(struct folio *= folio, swap_cluster_unlock(ci); out_no_split: local_irq_enable(); - if (need_remap) { + if (anon_vma) { if (!ret && !folio_is_device_private(folio)) ttu_flags =3D TTU_USE_SHARED_ZEROPAGE; remap_anon_folio(folio, 1 << old_order, ttu_flags); } +out_unlock: + if (anon_vma) { + anon_vma_unlock_write(anon_vma); + put_anon_vma(anon_vma); + } =20 return ret; } @@ -4294,7 +4320,6 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, struct folio *end_folio =3D folio_next(folio); bool is_anon =3D folio_test_anon(folio); struct mem_cgroup *memcg, *old_memcg; - struct anon_vma *anon_vma =3D NULL; int old_order =3D folio_order(folio); struct folio *new_folio, *next; int ret; @@ -4325,23 +4350,6 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, memcg =3D get_mem_cgroup_from_folio(folio); old_memcg =3D set_active_memcg(memcg); =20 - if (is_anon) { - /* - * The caller does not necessarily hold an mmap_lock that would - * prevent the anon_vma disappearing so we first we take a - * reference to it and then lock the anon_vma for write. This - * is similar to folio_lock_anon_vma_read except the write lock - * is taken to serialise against parallel split or collapse - * operations. - */ - anon_vma =3D folio_get_anon_vma(folio); - if (!anon_vma) { - ret =3D -EBUSY; - goto out; - } - anon_vma_lock_write(anon_vma); - } - if (is_anon) ret =3D __folio_freeze_split_anon(folio, new_order, split_at, true, list, split_type); @@ -4368,11 +4376,6 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, free_folio_and_swap_cache(new_folio); } =20 - if (anon_vma) { - anon_vma_unlock_write(anon_vma); - put_anon_vma(anon_vma); - } -out: /* restore to caller's old_memcg */ set_active_memcg(old_memcg); mem_cgroup_put(memcg); --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14B8B476CE9 for ; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; cv=none; b=iLdDjDB10KcUAF81QNIVxQD11UtGrs+n2ScnyVhEwZbVALuTE27KtiRoX1dm78ZWXKgUy4jZBobIX9WgnRBsHKGg0Wu5zlLoRWKWcbT1sRczPO+R5I+A4fDG7JBHq+C+bkgNXfbvaIeHlPmuIA2J35ZkiHZXlWuTXLMhw0wFSzg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; c=relaxed/simple; bh=ZDf6KklUL1VJiJN35meAQReHtETf/ilTmtshGQq/oto=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GII7w0a/fPx3sCTzJ1tCKBDJYp73/zaDpGytID0y0scqZ5j6/xm4cRfS6trYY2G1WOv3OqKk0Ag/BFWR1IW6XeWgNL34iG6W53vUVNNW57jxF4Aqiuve5byQ1ZgYCaLYcQjYhjY8T3KT/9JCQGkgR9NK3pVVXXizsLTn9s2/DRg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=i9xNylgK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="i9xNylgK" Received: by smtp.kernel.org (Postfix) with ESMTPS id EA119C2BCFC; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199164; bh=ZDf6KklUL1VJiJN35meAQReHtETf/ilTmtshGQq/oto=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=i9xNylgKls7bTuzU/eo6aSagb2pvXGi/fIn63P5z4zUY4YYVjwS5uZhTHjb/eQJ9V qlCcVezEyplfnMUQCcyNjYVUM6htc+3yeYTg1AJVQ9HnC9fZb4ZBenynVSS04hLwpr qjt1IYZT8NR7tg578FnJg+e7sgJNxbjk8gnyZZFTIbLOaIUqVwkHK8plHkWsXbW98G gp6yNsyV2DyNyLti5ntQeNfA8g6Aoe2co9v+NkJJKdglCBLEbDYQSjX9gQXevDMW+t 9Id3oWmCGCbpKilFl5izs6E844k9QYG0IIe8MyfvXcn6W7to3hV55WNn4dtvF5IUoE fmjLWwk1yOEPw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5E6EC98304; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:44 +0200 Subject: [PATCH v6 13/17] mm/huge_memory: move memcg switch into the file split helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-13-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=3709; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=yE5eu4lNBeWc96ndYU7jpzh4N9fRuY9dY8qKr7QXzUs=; b=IcJ3bl74clci/tOWp+x2uWCjpK7cQMm1RWbh4/TQ1A1LxhoxWtE6pwFJFItArBlnSms/Fm8R1 77uxr2R3aH6Bs9C4b4uad7QTI2Zn+4yCPdU4GPZx/wWtw+S01ccME06 X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song The xarray node allocations in __folio_freeze_split_file() need to be charged to the folio's memcg, so move the memcg switch from __folio_split() into the helper. The anon split helper and the after-split folio freeing perform no chargeable allocations, so no memcg handling is left in __folio_split(). Rename its out_no_memcg label to out. Acked-by: Zi Yan Acked-by: David Hildenbrand (Arm) Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Signed-off-by: Kairui Song --- mm/huge_memory.c | 36 +++++++++++++++++++----------------- 1 file changed, 19 insertions(+), 17 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 62e2879cd3b4..8d43ee243b27 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4142,6 +4142,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, struct address_space *mapping =3D folio->mapping; XA_STATE(xas, &mapping->i_pages, folio->index); struct folio *end_folio =3D folio_next(folio); + struct mem_cgroup *memcg, *old_memcg; struct folio *new_folio, *next; int nr_shmem_dropped =3D 0; unsigned int min_order; @@ -4154,9 +4155,18 @@ static int __folio_freeze_split_file(struct folio *f= olio, if (new_order < min_order) return -EINVAL; =20 + /* + * Switch to folio's memcg as xarray node allocation can happen and + * needs to charge to it. + */ + memcg =3D get_mem_cgroup_from_folio(folio); + old_memcg =3D set_active_memcg(memcg); + gfp =3D current_gfp_context(mapping_gfp_mask(mapping) & GFP_RECLAIM_MASK); - if (!filemap_release_folio(folio, gfp)) - return -EBUSY; + if (!filemap_release_folio(folio, gfp)) { + ret =3D -EBUSY; + goto fail_free; + } =20 mapping_set_update(&xas, mapping); =20 @@ -4288,6 +4298,9 @@ static int __folio_freeze_split_file(struct folio *fo= lio, */ i_mmap_unlock_read(mapping); fail_free: + /* Restore the previously active memcg */ + set_active_memcg(old_memcg); + mem_cgroup_put(memcg); xas_destroy(&xas); return ret; } @@ -4319,7 +4332,6 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, { struct folio *end_folio =3D folio_next(folio); bool is_anon =3D folio_test_anon(folio); - struct mem_cgroup *memcg, *old_memcg; int old_order =3D folio_order(folio); struct folio *new_folio, *next; int ret; @@ -4329,27 +4341,20 @@ static int __folio_split(struct folio *folio, unsig= ned int new_order, =20 if (folio !=3D page_folio(split_at) || folio !=3D page_folio(lock_at)) { ret =3D -EINVAL; - goto out_no_memcg; + goto out; } =20 if (new_order >=3D old_order) { ret =3D -EINVAL; - goto out_no_memcg; + goto out; } =20 ret =3D folio_check_splittable(folio, new_order, split_type); if (ret) { VM_WARN_ONCE(ret =3D=3D -EINVAL, "Tried to split an unsplittable folio"); - goto out_no_memcg; + goto out; } =20 - /* - * switch to folio's memcg as xarray node allocation can happen and - * needs to charge to it. - */ - memcg =3D get_mem_cgroup_from_folio(folio); - old_memcg =3D set_active_memcg(memcg); - if (is_anon) ret =3D __folio_freeze_split_anon(folio, new_order, split_at, true, list, split_type); @@ -4376,10 +4381,7 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, free_folio_and_swap_cache(new_folio); } =20 - /* restore to caller's old_memcg */ - set_active_memcg(old_memcg); - mem_cgroup_put(memcg); -out_no_memcg: +out: if (is_pmd_order(old_order)) count_vm_event(!ret ? THP_SPLIT_PAGE : THP_SPLIT_PAGE_FAILED); count_mthp_stat(old_order, !ret ? MTHP_STAT_SPLIT : MTHP_STAT_SPLIT_FAILE= D); --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24DDB47728E for ; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; cv=none; b=eUwvwoh3pLW2MKu0OVMusY2A1Kvl2ifworvnL0SHC1iabZKNN1am6PeOo3yjyavUELOBaLK8PXslObRrahzs9u7gGiDJ4Xrplgnuaw8QDamBXq2vgEEmiZzIFxyMvyuEb6jDLPUnYCi6V8xTxrx1yIpaDt7EOZNNqafrFSewHZM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; c=relaxed/simple; bh=sOp8u6+5I5kr8seW5fU3M5ogXFq82vrw4VOqm9lx8z0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=WlmXloP2UReyWv2MtEo9pGXoe7+Q4099vXYvYUHVbzHQUy8d5+HrzSK2jRM/xTHWhX6FF/v7xav9d6+UqZjCXDOnyG93GJr0EgTA1zf5/KOoPrQU9SDTX3G1d3l3vLa1jOdogZo478FWZL6x9b9aD7gtQG4/uUeCp7153UdrjoU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=U9Y8ZvQ3; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="U9Y8ZvQ3" Received: by smtp.kernel.org (Postfix) with ESMTPS id 09362C2BCB8; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199164; bh=sOp8u6+5I5kr8seW5fU3M5ogXFq82vrw4VOqm9lx8z0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=U9Y8ZvQ3t8OG3KjhwXTUtRrGU1BzTZNinBYYaxRN14jp88gBZArAUQUavuUKo9WMx oJOFNChjLUsyzXhp9X+J1lwsMq5RyKBzcEkxF1NUMYFtzPNNBavbB9fZf98sKTGtDx quTZW7p+WIVhDkxZ6Pmr+kX9kSc6ufChnRCEhkQbr9jhnPFnhSgMU/HH5s/zrltIRh KleDfi5yd11JLF5Z+Laphc3f54S0viePX9PTApED2TZpWgJZgex3gZLxIZ9qZdEbtw OHGznJZE61MncNjUTP/c0xwdqbTSvmk/wsQHnb3Eu85QC+6R4TGAeg+tSbWgHaEAEU XAHkc0qyYwgvQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9E31C98313; Wed, 23 Sep 2026 21:32:43 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:45 +0200 Subject: [PATCH v6 14/17] mm/huge_memory: drop the unused do_lru argument of the file split helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-14-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=2467; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=xmffsm1SQoen6wnNoHnLCjDOebcsOHMJxuynkP1cNFQ=; b=BWZ2N3ClOOOgEbeQScIyuu2ZTz7UVZMZ8G6SWaNqCCOoFcNJmQX5pBttoyawp/k/M9HGCp/vi jNP0aNEeJsqB6/OPrJVBTzDozEblsDSMu5XdCvGraqT2eWMQjfnj20a X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song The only caller of __folio_freeze_split_file() always passes do_lru as true, so the argument and the branches gated on it are dead code. Drop it. Reviewed-by: Zi Yan Acked-by: David Hildenbrand (Arm) Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Signed-off-by: Kairui Song --- mm/huge_memory.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8d43ee243b27..86583bf1b148 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4136,8 +4136,7 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, =20 static int __folio_freeze_split_file(struct folio *folio, unsigned int new_order, struct page *split_at, - bool do_lru, struct list_head *list, - enum split_type split_type) + struct list_head *list, enum split_type split_type) { struct address_space *mapping =3D folio->mapping; XA_STATE(xas, &mapping->i_pages, folio->index); @@ -4231,9 +4230,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, } =20 /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ - if (do_lru) - lruvec =3D folio_lruvec_lock(folio); - + lruvec =3D folio_lruvec_lock(folio); ret =3D __split_frozen_folio(folio, new_order, split_at, &xas, mapping, split_type); =20 @@ -4255,8 +4252,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, folio_ref_unfreeze(new_folio, folio_cache_ref_count(new_folio) + 1); =20 - if (do_lru) - lru_add_split_folio(folio, new_folio, lruvec, list); + lru_add_split_folio(folio, new_folio, lruvec, list); =20 /* Add the new folio to the page cache. */ if (new_folio->index < end) { @@ -4282,9 +4278,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, * and its caller can see stale page cache entries. */ folio_ref_unfreeze(folio, folio_cache_ref_count(folio) + 1); - - if (do_lru) - lruvec_unlock(lruvec); + lruvec_unlock(lruvec); fail: xas_unlock_irq(&xas); fail_mmap_unlock: @@ -4360,7 +4354,7 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, true, list, split_type); else ret =3D __folio_freeze_split_file(folio, new_order, split_at, - true, list, split_type); + list, split_type); =20 /* * Unlock all after-split folios except the one containing --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A5F3477295 for ; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; cv=none; b=ZxjZOt4GGaNVhwbhuA6OEvCrJ5ZWX/OiTtqFjl17Ar8Ca69DlGrJxNJuLh3ln1VORZKVJy5+vciZo4JDFIOHNlyv30P3jInQyD2jpW7F7ZO4VLfY9YfelkcPBrMi5umOrPzUlW96DCx10Iv3mNTwp6sFIEd79YYOsPTgSPl4kfY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; c=relaxed/simple; bh=1HjEP9h01m2os52VFtLg0w0k8h9JkhcfR+8oAefYrS0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=U7YAF6mj80KKb4SC/t+baZb+h3Sn9Glpc7ffNkJ5hlZDAac390Np2+SluSoeUk/blBWvtBuMWmC4kpLrRzaoqSpbt584th/BeJ7D7zJ4Z3KEriddFhqrJBf7ZHCwpvN4dClJZhytK1h+e3ZI2Cja+GozUBniNRBWkDsu5xyJ9T8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Iq3SdsXE; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Iq3SdsXE" Received: by smtp.kernel.org (Postfix) with ESMTPS id 1E058C2BCF5; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199164; bh=1HjEP9h01m2os52VFtLg0w0k8h9JkhcfR+8oAefYrS0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Iq3SdsXEQ00dkvVhZPrugL6rHpldJisSBZePiddEKfdn7c9bKS4FYwE5U3XYzbnzo ZIK/wNtGf+G5DAXqVwC2Gj9FJh4IuwApuB7uSan3rFc08k4Ah/kmmFmUCzrP5Uxp4I Spij0Bc3DpnTMn30vwVOU/odrNz/Wabnta+rmDbLcDJfOVmO50J8rePXyWG33V/TsW U9F9TnLxUM7gfJioGql2jSF3R6fIZ/5gmLZXe3SesmMw3FrwmJ1BQHkatyvRXTT5gB dAqu4OXdYxDmLsBqCsrC1HutgWfncPcXJmBmcl565cA8Nrr4972SZ1XIYg+MYtBqgH mS69YZjuBklkQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09C2BC98311; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:46 +0200 Subject: [PATCH v6 15/17] mm/huge_memory: clean up after-split folio freeing in __folio_split Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-15-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=2665; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=AQb8VGHmCtACgG3BJvB2LYY1R1HEBncd5BHWOwnHe8M=; b=F06ZdTYwgfJVEsPAfkgvPL/eTORiKnaXRd6FY2R/9M89Xtg7PMRDlyTl2fjhQ/0e97jYMuLBm bmbM7a7dTJTAjsXllyw2lbB1vjd82/tlqbPT3RSz4CIb88a6/Wu0Bzu X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song Replace free_folio_and_swap_cache() with an explicit folio_free_swap() and folio_put() in the after-split loop. free_folio_and_swap_cache() must trylock it again and re-check folio_mapped() before freeing the swap cache entries. If the trylock loses a race, the entries are left behind even though the folio reference is dropped. The sub folios are still locked here, so just directly call folio_free_swap() under the lock if it's unmapped, then unlock and drop the reference. This makes the swap cache freeing deterministic and the reference drop explicit. Reviewed-by: Zi Yan Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 86583bf1b148..a98f447ae78a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -4325,7 +4325,8 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, struct list_head *list, enum split_type split_type) { struct folio *end_folio =3D folio_next(folio); - bool is_anon =3D folio_test_anon(folio); + const bool is_anon =3D folio_test_anon(folio); + const bool is_swapcache =3D folio_test_swapcache(folio); int old_order =3D folio_order(folio); struct folio *new_folio, *next; int ret; @@ -4365,14 +4366,16 @@ static int __folio_split(struct folio *folio, unsig= ned int new_order, if (new_folio =3D=3D page_folio(lock_at)) continue; =20 - folio_unlock(new_folio); /* * Subpages whose mapping has been zapped may be freed * earlier, but freeing them requires taking the - * lru_lock, so we defer put_page() on tail pages until + * lru_lock, so we defer folio_put() on tail pages until * after the split completes. */ - free_folio_and_swap_cache(new_folio); + if (is_swapcache && !folio_mapped(new_folio)) + folio_free_swap(new_folio); + folio_unlock(new_folio); + folio_put(new_folio); } =20 out: @@ -4399,7 +4402,7 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, * isolated from LRU (if applicable) * * Upon return, the folio is not remapped, split folios are not added to L= RU, - * free_folio_and_swap_cache() is not called, and new folios remain locked. + * folio_free_swap() is not called, and new folios remain locked. * * Return: 0 on success, -EAGAIN if the folio cannot be split (e.g., due to * insufficient reference count or extra pins). --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D8A04772B4 for ; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; cv=none; b=an9NN210dMKM5TKM2WVdHHyxYkI6PkFjBFNbAgJE62Py5FQclY+sLbeYkqnja7fENOMEYjHWCBWiBLrCVG2hS5bLe4WI/7VyeRwFRWvWKSy2ymyMfItmGavAPDxXMKTjj+g3QKTIZb/+xtf8roGp/fZg38gMnGgK4CUgzxDTUUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; c=relaxed/simple; bh=YBEVbbweHYfcoloA7VJ7yXPyt9Ai7rrUfs3rNGNaTv8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=s5hQOBqh6MbXDOhj8Eq2pmG/Dk8IBeFFaAwnF5UmzjMYykTPNGZyvTJcLTQBCvMdkdnEPyY5kxMxsQvxacQ70I4mKVkCAxkslBwjK/DsnPjDzcY+ytraYXqioyh39zdC1zBKt0kWYPtApLfEiDpjtzFIpwA/MPRA05kvaDCKSlY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=WOapASXf; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="WOapASXf" Received: by smtp.kernel.org (Postfix) with ESMTPS id 30CC1C2BCFF; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199164; bh=YBEVbbweHYfcoloA7VJ7yXPyt9Ai7rrUfs3rNGNaTv8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=WOapASXfKBBRREDikSE4WTSa2o0hMgvZbcTfUaa3EndgpzHW1rHJx9zIoF645FM3b mhuLhyFpnzwqPOmUfQuxMJkBtIiDexiB4p95un25TiJg2JNCS02z8xGyM9jPx6rwym l55KZzT4ulwSrg9N+zXPk/zRi10Rfhlw03iVvOvDxfEDk/EyzYnTc8+Ozsmo0R2oad DeFWX/H7TF5ruyGOYnwBhJq1k4jDmnk55lktJEp57MEQLaEPqto2lkESCNRI8nLQW4 q1xZaPtuD5tf15XQNi5nNquRibA2eBZYeJf2vubkgWLrifGT1YBMESM6C+xR3sZo1Y aeB87J9ndtY4Q== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D012C9830D; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:47 +0200 Subject: [PATCH v6 16/17] mm/huge_memory: count only swap cache refs in anon folio split Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-16-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=4430; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=bHk44bYsClavSrL0S+ty/yMpowjsaEA3StLe+r4XKwY=; b=adVtrlDV113daGntojhWEIZR8gAlsfZuiqul4V9XPviQqWQ9DSI+VL0Hz5+JkAEMIkQQng25i /kyStc8n+qFDd4H8Y+olbsnYpS0MeClcqmmEsaj8hkNIvq+wsc06lg1 X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song Only __folio_freeze_split_anon() sees anon folios and swap cache folios now. The file split helper only handles page cache folios, which hold exactly folio_nr_pages() references. Rename folio_cache_ref_count() to folio_swapcache_ref_count() and drop the anon check so the helper counts what its name says. The file split helper now uses folio_nr_pages() directly. No feature change. Reviewed-by: Zi Yan Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 35 +++++++++++++++-------------------- 1 file changed, 15 insertions(+), 20 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index a98f447ae78a..d31c1d3d1f8e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3997,10 +3997,10 @@ int folio_check_splittable(struct folio *folio, uns= igned int new_order, return 0; } =20 -/* Number of folio references from the pagecache or the swapcache. */ -static unsigned int folio_cache_ref_count(const struct folio *folio) +/* Number of folio references from the swapcache. */ +static unsigned int folio_swapcache_ref_count(const struct folio *folio) { - if (folio_test_anon(folio) && !folio_test_swapcache(folio)) + if (!folio_test_swapcache(folio)) return 0; return folio_nr_pages(folio); } @@ -4067,7 +4067,7 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, =20 local_irq_disable(); =20 - if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) { + if (!folio_ref_freeze(folio, folio_swapcache_ref_count(folio) + 1)) { ret =3D -EAGAIN; goto out_no_split; } @@ -4104,7 +4104,7 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, next =3D folio_next(new_folio); zone_device_private_split_cb(folio, new_folio); folio_ref_unfreeze(new_folio, - folio_cache_ref_count(new_folio) + 1); + folio_swapcache_ref_count(new_folio) + 1); if (do_lru) lru_add_split_folio(folio, new_folio, lruvec, list); if (ci) @@ -4112,7 +4112,7 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, } =20 zone_device_private_split_cb(folio, NULL); - folio_ref_unfreeze(folio, folio_cache_ref_count(folio) + 1); + folio_ref_unfreeze(folio, folio_swapcache_ref_count(folio) + 1); =20 if (do_lru) lruvec_unlock(lruvec); @@ -4138,6 +4138,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, unsigned int new_order, struct page *split_at, struct list_head *list, enum split_type split_type) { + const long old_nr_pages =3D folio_nr_pages(folio); struct address_space *mapping =3D folio->mapping; XA_STATE(xas, &mapping->i_pages, folio->index); struct folio *end_folio =3D folio_next(folio); @@ -4211,22 +4212,16 @@ static int __folio_freeze_split_file(struct folio *= folio, goto fail; } =20 - if (!folio_ref_freeze(folio, folio_cache_ref_count(folio) + 1)) { + if (!folio_ref_freeze(folio, old_nr_pages + 1)) { ret =3D -EAGAIN; goto fail; } =20 - if (folio_test_pmd_mappable(folio) && - new_order < HPAGE_PMD_ORDER) { - int nr =3D folio_nr_pages(folio); - - if (folio_test_swapbacked(folio)) { - lruvec_stat_mod_folio(folio, - NR_SHMEM_THPS, -nr); - } else { - lruvec_stat_mod_folio(folio, - NR_FILE_THPS, -nr); - } + if (folio_test_pmd_mappable(folio) && new_order < HPAGE_PMD_ORDER) { + if (folio_test_swapbacked(folio)) + lruvec_stat_mod_folio(folio, NR_SHMEM_THPS, -old_nr_pages); + else + lruvec_stat_mod_folio(folio, NR_FILE_THPS, -old_nr_pages); } =20 /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ @@ -4250,7 +4245,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, next =3D folio_next(new_folio); =20 folio_ref_unfreeze(new_folio, - folio_cache_ref_count(new_folio) + 1); + folio_nr_pages(new_folio) + 1); =20 lru_add_split_folio(folio, new_folio, lruvec, list); =20 @@ -4277,7 +4272,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, * Otherwise, a parallel folio_try_get() can grab @folio * and its caller can see stale page cache entries. */ - folio_ref_unfreeze(folio, folio_cache_ref_count(folio) + 1); + folio_ref_unfreeze(folio, folio_nr_pages(folio) + 1); lruvec_unlock(lruvec); fail: xas_unlock_irq(&xas); --=20 2.55.0 From nobody Thu Sep 24 14:27:13 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5AB847887E for ; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; cv=none; b=NjiA9/JHbx+MPAvaQsTQNJTis5MffMXXWcoRsxH1NcZC8iQAczN3955CxZClexek3Q3vXmI+goPRrlPrgFms5WKRyQU1iA8NQNNFfuIP14J30GmuO6yU5+bZOQD+xqmNTHasNdUbc0qxHdzS89elYdTrnPCaxvmNDGMGRTP+qZg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790199164; c=relaxed/simple; bh=VCcgX9Zt0Ud9YgxqYFqd3Iy4Vbv14vrCAHPv6HfCCTs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RdxJTjOpqigx9gcw+cMV0IJaaUEW2F1bhreic08CuhzMO1pYQRaoAA5b2r06+sx8vf0Biwn8HiSmnq+KAW3rM/63eomgp2FOQy0nSyvZqX+A8hPIprN9BS2rGW2Nzwf2wJIWbdyRYFiwKOpTfQpC2hHhW0HokqDTuETO1WBKA0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=YxAeVBrR; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="YxAeVBrR" Received: by smtp.kernel.org (Postfix) with ESMTPS id 686D5C2BCF5; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1790199164; bh=VCcgX9Zt0Ud9YgxqYFqd3Iy4Vbv14vrCAHPv6HfCCTs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=YxAeVBrRanLAGfXVedme9BHiS2iAavj3ObSVBg6J8RXgs4wOE5zNKgxlGhbYEZNLW j0rfVwXgZGSZlqoFJuKG/YA6nm8wi2Xs95muRvOZ4d3zCH8wFoUZHZhldMCUws5wEX H1jfzNSwYpqYYTiag7kWS9hhUvtIP8t1imTfmxD+Q6tCrHLIWpcrSHUiYGDJtdFHQJ hFbDbYNPxNJAuXgvdiBl73oK/1eeyEg6eT5ZxEVrSOL93VhwbheqUIEVPqjw0g6y7u 6YGDOIG88dS9BFbgqMZgVUil1tNE6cg1GZBfYvtmJf1LxmvOJZd50TOHp3uVJBI3GZ m7C71j20OD6Zw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52973C9830D; Wed, 23 Sep 2026 21:32:44 +0000 (UTC) From: Kairui Song via B4 Relay Date: Wed, 23 Sep 2026 23:32:48 +0200 Subject: [PATCH v6 17/17] mm/huge_memory: drop the redundant mapping argument of __split_frozen_folio Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260923-swap-thp-cleanup-v6-17-ba1b4ba72c6f@tencent.com> References: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> In-Reply-To: <20260923-swap-thp-cleanup-v6-0-ba1b4ba72c6f@tencent.com> To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Lance Yang , Usama Arif , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Youngjun Park , Yeoreum Yun , "Kiryl Shutsemau (Meta)" , Shivam Kalra , Kairui Song , Kairui Song X-Mailer: b4 0.16.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1790199160; l=2842; i=kasong@tencent.com; s=kasong-sign-tencent; h=from:subject:message-id; bh=oJuA6piE3d7PEesJP/+7GrdrNMc/XlosPlQGEaGrc3Y=; b=qsTJuWfxYYH4Yerg2qYGzB6xSXXLGUpTM1l3XsXyLjIutCI2Os1FniHyi8KQqPuloo8hkH7zF 4bHfDxkVElLCEJtHufe36HFJzFEk2gsuIO21CqpgJmP7yRZRyk7wCmT X-Developer-Key: i=kasong@tencent.com; a=ed25519; pk=kCdoBuwrYph+KrkJnrr7Sm1pwwhGDdZKcKrqiK8Y1mI= X-Endpoint-Received: by B4 Relay for kasong@tencent.com/kasong-sign-tencent with auth_id=562 X-Original-From: Kairui Song Reply-To: kasong@tencent.com From: Kairui Song The mapping parameter only served as a non-NULL check to detect whether page cache entries need updating. The xa_state pointer conveys exactly the same information: the anon split helper passes NULL and the file split helper passes &xas, which is non-NULL iff the folio is in the page cache. Use the xas pointer instead and drop the parameter, along with its kerneldoc entry. Reviewed-by: Zi Yan Reviewed-by: Yeoreum Yun Reviewed-by: Kiryl Shutsemau (Meta) Acked-by: David Hildenbrand (Arm) Signed-off-by: Kairui Song --- mm/huge_memory.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d31c1d3d1f8e..16b142de5008 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3833,7 +3833,6 @@ static void __split_folio_to_order(struct folio *foli= o, int old_order, * @split_at: in buddy allocator like split, the folio containing @split_at * will be split until its order becomes @new_order. * @xas: xa_state pointing to folio->mapping->i_pages and locked by caller - * @mapping: @folio->mapping * @split_type: if the split is uniform or not (buddy allocator like split) * * @@ -3866,7 +3865,7 @@ static void __split_folio_to_order(struct folio *foli= o, int old_order, */ static int __split_frozen_folio(struct folio *folio, int new_order, struct page *split_at, struct xa_state *xas, - struct address_space *mapping, enum split_type split_type) + enum split_type split_type) { const bool is_anon =3D folio_test_anon(folio); int old_order =3D folio_order(folio); @@ -3890,7 +3889,7 @@ static int __split_frozen_folio(struct folio *folio, = int new_order, if (is_anon && split_order =3D=3D 1) continue; =20 - if (mapping) { + if (xas) { /* * uniform split has xas_split_alloc() called before * irq is disabled to allocate enough memory, whereas @@ -4089,8 +4088,7 @@ static int __folio_freeze_split_anon(struct folio *fo= lio, if (do_lru) lruvec =3D folio_lruvec_lock(folio); =20 - ret =3D __split_frozen_folio(folio, new_order, split_at, NULL, - NULL, split_type); + ret =3D __split_frozen_folio(folio, new_order, split_at, NULL, split_type= ); =20 /* * Unfreeze the after-split folios and put them back to the right @@ -4226,8 +4224,7 @@ static int __folio_freeze_split_file(struct folio *fo= lio, =20 /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ lruvec =3D folio_lruvec_lock(folio); - ret =3D __split_frozen_folio(folio, new_order, split_at, &xas, - mapping, split_type); + ret =3D __split_frozen_folio(folio, new_order, split_at, &xas, split_type= ); =20 /* * Unfreeze after-split folios and put them back to the right --=20 2.55.0