From nobody Tue Oct 7 07:23:05 2025 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2050.outbound.protection.outlook.com [40.107.96.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 629142F3626 for ; Fri, 11 Jul 2025 18:24:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752258245; cv=fail; b=XqTfX5NCmZdvhAk6Xf7ZZUlpHi/JkzgjIhsDNsbYomy7OII1xwbwrHu8iSo4dZDkGEPMg1BgXLrJghQ9q2mClFPXh4o7r60Mi349A0u3FPhBr26qMhXwTLIwZvrdAVt7DzN6MptmdX/plv7cD7p7XoeyXGeXOjMp9ia79QxLWkQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752258245; c=relaxed/simple; bh=9MDJAgTLM0w56yeQulLQLIEwCmgMQEfCrMmX3g2rI/M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=u7CiunKIL0YPOzCDkAQJwGdHcpTTbHautkAIXYhLJAPVsW7K0gMhP8zxLSRezMiilavssTem/oMU/Ak/l0hSMhHJba7LuMzx33yy1uqlw7ZEBQOklev07sBlnjFsbKBBBlbG1hRD0+6FGyW4bpwSiQVrmhUuALgxhy89i20yx+E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=YtQUq5jg; arc=fail smtp.client-ip=40.107.96.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="YtQUq5jg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=b3DEygysrdpcJdGbqT0x/62V/Zhiv7CMAdBoo3MD+6SFKgbtKzzzl2PSpZJ/+DSULBcBZILnIIzUYUzzcqAVe7AhRSQ+K8C2wcUqdGrpKW0gJ1WF6FziN1gqb1uIlnX9d43Df224t0E4qH0YMPilOpiVs+HT56NVA5cEisbSaXnfHZpioNn2lXSKNCSYFAS1yZGdQ1VSQQFmSyGFD4bCMzYSf0FfZIm5YaI2J5mWXILLCMTz2vTXdJmm2ic2S9fG9v4JCX564MXS9OIWMxAj2/BKtNeFfaN9E/b141Gjex/IDQhwZUGVlUKROrjoB9Jt6G26BXKv5l4blStThkDnSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HGnCv3ktad15mv+kb9Ocrbj3h3Td/GQT/iWaRDQCLRE=; b=xmBlONtVB5t5GraSwTvHrA+7mN2Ye5gGnyWfqpY8oA2FQXy0/Vzqy1tLWnK7zRWyV/e3CaQ96s3TetSkc465qaccOX/Q8odfaJ2H57Tz91/wPsJvkCrGC4gX9l2keD15S4RJVLjltb/NHhMnOa/cvHLlNgesNlesM2a4e2tvlk3XESpNYXkM+3dUTQ8FVe9zgXyx3+ug07i4vZah+wOzrtcu0q+Ts1hORt6RiLd4Oc77MS2S4mg82P7curmxNLqO4xyLFCjm91vPSBcYBDoQU/MixHbKcgWCK7RmuTSbUaHSfCri09+EqZlsLbXF4KnwsJw5DpR/S0uqHeP0pnaAqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HGnCv3ktad15mv+kb9Ocrbj3h3Td/GQT/iWaRDQCLRE=; b=YtQUq5jgfoWBkTeIGlGcHDvgQ1Cdz3n/i2x0hOZdgmE00Ty1wwDHCt37RQCSmmwiLjWGDCWyrWIfjiAmuzy6TGlEhUwmkTUQu0c/hGRcyNujurJpn6hsPIntQ/1TkiV5knFL+KfkIOsrqTJ0Kblk/hX23VucDFgBSDoFHSAQZ8h+lqwLDTviIwAhEn2XxSd7TZk16W53HsWix4ZDSeXObZrM+5HeE9ysNPpHW2GH0C2QVkEHDnqn5eTqKdrCdLNuA723NfN/Dg8MDEOmu+vbhRVi4vGPtN5bGOkS1udBdGjstcv2Le8ueZGntI5SCd0Saebbi7hiY4bhQy8IdQBkwA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CYYPR12MB9016.namprd12.prod.outlook.com (2603:10b6:930:c4::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8922.25; Fri, 11 Jul 2025 18:24:00 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%6]) with mapi id 15.20.8922.023; Fri, 11 Jul 2025 18:24:00 +0000 From: Zi Yan To: Balbir Singh , David Hildenbrand , linux-mm@kvack.org Cc: Andrew Morton , Hugh Dickins , Kirill Shutemov , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , linux-kernel@vger.kernel.org Subject: [PATCH v2 1/2] mm/huge_memory: move unrelated code out of __split_unmapped_folio() Date: Fri, 11 Jul 2025 14:23:54 -0400 Message-ID: <20250711182355.3592618-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250711182355.3592618-1-ziy@nvidia.com> References: <20250711182355.3592618-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0P221CA0015.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:52a::31) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CYYPR12MB9016:EE_ X-MS-Office365-Filtering-Correlation-Id: 06517d7c-37c8-4682-a05b-08ddc0a81b3d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?fjFWStd0VVycNOfbXQAsVuznjY1dLfB9z3udpCX5i+3fZowD7HhYIHMSf1hc?= =?us-ascii?Q?NuK7/QBqyZNiQgKfuQFZRt95EoVVZF0R0YgIi5/MJttYqGrYMjAgr6chSVgn?= =?us-ascii?Q?aMeiLSXlGF7XQqC47yZ3rCrXtl5OZ9Mb8Q0xnBrug2GONFaXrvu4xuvyl3sL?= =?us-ascii?Q?oQU6rqJ75LQpaKpNAkGlZEE/g9SpI616cRdYu+j/6DHPttHlHll2QBQQdzap?= =?us-ascii?Q?aHtX06XL2ZnxRMGHwBQKqYdcnHNYzZxkfLDnhuAtcCJZiBsO+sLYok8rjpOX?= =?us-ascii?Q?Nr9y6ZbyCFBU+PxLrREoFpCAGM+wB4CKKldvwWEuXn2mhPCWKozwk/mr03JU?= =?us-ascii?Q?QEAbsNZxcDfNAQ4T5F6GCu7r/YwUiPCW2bVn05hqg+mf143FcsyZeBuTT6Bm?= =?us-ascii?Q?E/NNpTo1BQsIBCi7VpxVHvv+2UjVSgQQEEZX9e5Mrn32D/gfWkWFJ0SkAHwS?= =?us-ascii?Q?FrYsBUmb2IerxgLuD0IThuTurt96CPYPTfmNg+6HlMHfwNDNtCLUaqTeA1x1?= =?us-ascii?Q?NRkZY1gagSKSmx9OUiS6kUZGxDRidb9cU+roCpZjgdH8jz16iz45qUjm6qPh?= =?us-ascii?Q?x659mpUsjVBluX2fKgIelIsGHM6m2mDWw+APQH+M01NqnwzJpXZT+Nj5HqEL?= =?us-ascii?Q?UdS4ULavxJHMbwDsJbX5XDo1SZ0/oTAPfdXHJx1X0YSC2D5GHrtVuTtqGvuc?= =?us-ascii?Q?ZiXGzQVnj3Momz2P+w4BBGMynT5/0BkoTke+7aJZ8VUvxuW64kvQUQvbjNuh?= =?us-ascii?Q?GXqPP4YLoH5A7oHwghxsnH/MuT8xhY9j9ZWnwU2HEwvEZb0EtlZ6cbamXhxe?= =?us-ascii?Q?sVRl1XLhF+u2FjzG8iwCA11dmAVYNKu0u0BmffHCOGrsVU9iU9U+RyzI+aM4?= =?us-ascii?Q?8bYRmShejYlOnAXCl6VQ02kr2wqY88VB8wLHlu+d+NFjtutMXTfrDst223wo?= =?us-ascii?Q?0VOiillzuzSIr4N3cU+gQ+AwbnLouQQPd9YeVITRlcDB/9myIAtwJfxz5LRB?= =?us-ascii?Q?PmwT3lbae28NwzJ6Iv5eQajVXxRbsduxWdr0dpCJE4w+XyFJe79xUlQeSqzj?= =?us-ascii?Q?bSOOU9JzOQafK6CjTSVglOwYLYkzQyd5pruzIgB5oNP+lkNukU71LbGiOm4/?= =?us-ascii?Q?8M8db3amIUI0wZyGt9LD1y4O44ckRiG+He4KizKqWd3Y+S74JcuqPgzruBLU?= =?us-ascii?Q?URzy6hcoVmFFgHkDeJtw/I/PDltBBlqhqu8z3mJ2ukQxr3n2skwIeXd54y4I?= =?us-ascii?Q?IR+9qx8MTDfBqsJw6TCm8DQvSUVPFnNlCNvhXp8kk/mI9XxKpN7HO/B0mjW0?= =?us-ascii?Q?e8CWba2clI7cVGuYjr6O9CZ9SEKgnpWWH0ylVtSr/vbA+QwDaB9+8dnX+qkg?= =?us-ascii?Q?DxEiyDy4PyzC5ICo+CHvgAPt8mQtIjNY6CMREUs2cAj7fe2E3hY5O7pca6AR?= =?us-ascii?Q?VRLtkw9pmsU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?66w6R0bEC3rQcuzXVEet+Pwr8L0I4iNX2UqHtDI23NZj845cNGc4rBV3x7AA?= =?us-ascii?Q?WYN/Y7lwGBMvVNtAEk8oWlSrziAyC6FgnUD1bYAhnuWsf/o+A5f2O6mOAr8a?= =?us-ascii?Q?SCTCgxNJAdHS7DhAiPajXISfpkVvutAS2EfR9B+ysGJMfLOByHVhGLKDNPgn?= =?us-ascii?Q?F35B/wsHV5ljfcd3bffXXBcGp4oZySzucH6jC4iFJ2/UEG+8CHoeGNwwkJu4?= =?us-ascii?Q?/sJZhHWEkuZkmaancyEqf1H9f7Ezvm4PjYKUVhAwhel+7yblUGvRYcYw8y3u?= =?us-ascii?Q?+EVtR2Tn3saB3j6R9ZReEtyoSFup3gFl8c7DT1IgU4Pb2NKcRzK1jAlmuemI?= =?us-ascii?Q?yCYAdFOxKfmHGutYPmX73RHhOJfUYBkB0s3dCN+s4Pn5HWa29VCImiwmgmcP?= =?us-ascii?Q?2wd2eUynTZu01pDtBKju1o0rRTF8FPCz2nVMgI07gU1I6/IJga1U/aDARvwL?= =?us-ascii?Q?T2i8/DzSleAqi+Z998tVHw+bWgqVHhcZ766p/IzcPvu2LJ1vpG131sQ+bZsi?= =?us-ascii?Q?E6vWxcCjhMc3K1ZF8GJ0Ddlsc4DKf5jnQppO2jXgItAV/BF/4KOWwOulF4Uw?= =?us-ascii?Q?HcykLn0zAtDfduBgJ3rGTLfArp+WBi2HbrD+LqEY6LGE8W6l4ktwzSDGivsg?= =?us-ascii?Q?SQwdWCC1CFDq2VEYF70WAgHLqFAkHI2SOuHIEF/Y4V5w6nReuWckVoYV/2Nj?= =?us-ascii?Q?1Lkf1if7WRE+Eb6dCbM7asqvuQu+059d29Y9Wi/5Cis/X++8Awijjju7Aedl?= =?us-ascii?Q?BZ7cqUGd7GxCs/NpQcY8RkrEvw6vbzOeKtldAinVKR9NA5owdAN84uqmJvDH?= =?us-ascii?Q?QdDBbd+MgnZbz8lMEz5C7jWYCZVVVogNbBlmOiTNHucR1aD7pKEOx61oX8Yy?= =?us-ascii?Q?DWWWzQyKIGNKUzEpyf2sRQweqM6PTpXxdgJc7vtx99/5iWS9wLIWDucDRtiQ?= =?us-ascii?Q?JkK8lfjj40gRjwCdMw0wvZekFRP2/UsWWi+wSOiDrzUZJYgBX16sujd9BgGk?= =?us-ascii?Q?ATecTgDO3+8gan7QcCH2FUwP1/8TNTh6flKq57NyYtBbmuXkuAzuBzJWxosq?= =?us-ascii?Q?Gd/nABATcOWNfE8gn3FZ2mVLB7SSYlcFXyEsWKxTcdrl1109v8wlf8851k9N?= =?us-ascii?Q?kKTSfZRgqJYX1ckWt4khvaTBsnLBUyGj0bnsoRc7qDufvxxm3dddVaNMGF+9?= =?us-ascii?Q?yp4jDxmWoYDtgQorY1Q13TqAk+QfGDw02X4S6kLI5uT4Hd284MefEMpafQ7t?= =?us-ascii?Q?O3kQhFuJgcMm8oHmkWYR1RdhIuOW1qo81/uWP8Px6tYdvCyK3KHJrCOvmOGg?= =?us-ascii?Q?AMYOjNwajB/Cl3ymRBPh3toD1gQKgdSMLMmY2DGxqFHko5DXUhevAacj+8Wr?= =?us-ascii?Q?2Vmgl+SVVbmyDtKYc0KRRbyMSaVX/QDufZUPWIxVL4MAgpxZt/vcQQ+sBeAD?= =?us-ascii?Q?PxVC9WagA7UxMbt99OeKsw/1JmzTafiCvb8TX6IdJjHNNV5lnYBqpiMgb/1E?= =?us-ascii?Q?mH7fOE0SN89sYZw53sZqrJ3GD+FvEfOuLhhkkI2KzSrx8l8DVpKyC0rhjLhS?= =?us-ascii?Q?dvMRHfg+grN2qNNJkxEyJJ/OGVIE2Cs6MfeuICXR?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 06517d7c-37c8-4682-a05b-08ddc0a81b3d X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jul 2025 18:24:00.1688 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /fuK+jOoQnlHB51WYVSJk2bxCIrcMzUqwuaFlYrUZb2wMl7/eocT9erlPShwLNr8 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB9016 Content-Type: text/plain; charset="utf-8" remap(), folio_ref_unfreeze(), lru_add_split_folio() are not relevant to splitting unmapped folio operations. Move them out to the caller so that __split_unmapped_folio() only handles unmapped folio splits. This makes __split_unmapped_folio() reusable. Convert VM_BUG_ON(mapping) to use VM_WARN_ON_ONCE_FOLIO(). Signed-off-by: Zi Yan Acked-by: Balbir Singh --- mm/huge_memory.c | 273 ++++++++++++++++++++++++----------------------- 1 file changed, 137 insertions(+), 136 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 3eb1c34be601..818a6bd9f0d4 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3396,10 +3396,6 @@ static void __split_folio_to_order(struct folio *fol= io, int old_order, * order - 1 to new_order). * @split_at: in buddy allocator like split, the folio containing @split_at * will be split until its order becomes @new_order. - * @lock_at: the folio containing @lock_at is left locked for caller. - * @list: the after split folios will be added to @list if it is not NULL, - * otherwise to LRU lists. - * @end: the end of the file @folio maps to. -1 if @folio is anonymous mem= ory. * @xas: xa_state pointing to folio->mapping->i_pages and locked by caller * @mapping: @folio->mapping * @uniform_split: if the split is uniform or not (buddy allocator like sp= lit) @@ -3425,51 +3421,26 @@ static void __split_folio_to_order(struct folio *fo= lio, int old_order, * @page, which is split in next for loop. * * After splitting, the caller's folio reference will be transferred to the - * folio containing @page. The other folios may be freed if they are not m= apped. - * - * In terms of locking, after splitting, - * 1. uniform split leaves @page (or the folio contains it) locked; - * 2. buddy allocator like (non-uniform) split leaves @folio locked. - * + * folio containing @page. The caller needs to unlock and/or free after-sp= lit + * folios if necessary. * * For !uniform_split, when -ENOMEM is returned, the original folio might = be * split. The caller needs to check the input folio. */ static int __split_unmapped_folio(struct folio *folio, int new_order, - struct page *split_at, struct page *lock_at, - struct list_head *list, pgoff_t end, - struct xa_state *xas, struct address_space *mapping, - bool uniform_split) + struct page *split_at, struct xa_state *xas, + struct address_space *mapping, bool uniform_split) { - struct lruvec *lruvec; - struct address_space *swap_cache =3D NULL; - struct folio *origin_folio =3D folio; - struct folio *next_folio =3D folio_next(folio); - struct folio *new_folio; struct folio *next; int order =3D folio_order(folio); int split_order; int start_order =3D uniform_split ? new_order : order - 1; - int nr_dropped =3D 0; int ret =3D 0; bool stop_split =3D false; =20 - if (folio_test_swapcache(folio)) { - VM_BUG_ON(mapping); - - /* a swapcache folio can only be uniformly split to order-0 */ - if (!uniform_split || new_order !=3D 0) - return -EINVAL; - - swap_cache =3D swap_address_space(folio->swap); - xa_lock(&swap_cache->i_pages); - } - if (folio_test_anon(folio)) mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); =20 - /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ - lruvec =3D folio_lruvec_lock(folio); =20 folio_clear_has_hwpoisoned(folio); =20 @@ -3480,9 +3451,9 @@ static int __split_unmapped_folio(struct folio *folio= , int new_order, for (split_order =3D start_order; split_order >=3D new_order && !stop_split; split_order--) { - int old_order =3D folio_order(folio); - struct folio *release; struct folio *end_folio =3D folio_next(folio); + int old_order =3D folio_order(folio); + struct folio *new_folio; =20 /* order-1 anonymous folio is not supported */ if (folio_test_anon(folio) && split_order =3D=3D 1) @@ -3517,113 +3488,34 @@ static int __split_unmapped_folio(struct folio *fo= lio, int new_order, =20 after_split: /* - * Iterate through after-split folios and perform related - * operations. But in buddy allocator like split, the folio + * Iterate through after-split folios and update folio stats. + * But in buddy allocator like split, the folio * containing the specified page is skipped until its order * is new_order, since the folio will be worked on in next * iteration. */ - for (release =3D folio; release !=3D end_folio; release =3D next) { - next =3D folio_next(release); + for (new_folio =3D folio; new_folio !=3D end_folio; new_folio =3D next) { + next =3D folio_next(new_folio); /* - * for buddy allocator like split, the folio containing - * page will be split next and should not be released, - * until the folio's order is new_order or stop_split - * is set to true by the above xas_split() failure. + * for buddy allocator like split, new_folio containing + * page could be split again, thus do not change stats + * yet. Wait until new_folio's order is new_order or + * stop_split is set to true by the above xas_split() + * failure. */ - if (release =3D=3D page_folio(split_at)) { - folio =3D release; + if (new_folio =3D=3D page_folio(split_at)) { + folio =3D new_folio; if (split_order !=3D new_order && !stop_split) continue; } - if (folio_test_anon(release)) { - mod_mthp_stat(folio_order(release), + if (folio_test_anon(new_folio)) { + mod_mthp_stat(folio_order(new_folio), MTHP_STAT_NR_ANON, 1); } =20 - /* - * origin_folio should be kept frozon until page cache - * entries are updated with all the other after-split - * folios to prevent others seeing stale page cache - * entries. - */ - if (release =3D=3D origin_folio) - continue; - - folio_ref_unfreeze(release, 1 + - ((mapping || swap_cache) ? - folio_nr_pages(release) : 0)); - - lru_add_split_folio(origin_folio, release, lruvec, - list); - - /* Some pages can be beyond EOF: drop them from cache */ - if (release->index >=3D end) { - if (shmem_mapping(mapping)) - nr_dropped +=3D folio_nr_pages(release); - else if (folio_test_clear_dirty(release)) - folio_account_cleaned(release, - inode_to_wb(mapping->host)); - __filemap_remove_folio(release, NULL); - folio_put_refs(release, folio_nr_pages(release)); - } else if (mapping) { - __xa_store(&mapping->i_pages, - release->index, release, 0); - } else if (swap_cache) { - __xa_store(&swap_cache->i_pages, - swap_cache_index(release->swap), - release, 0); - } } } =20 - /* - * Unfreeze origin_folio only after all page cache entries, which used - * to point to it, have been updated with new folios. Otherwise, - * a parallel folio_try_get() can grab origin_folio and its caller can - * see stale page cache entries. - */ - folio_ref_unfreeze(origin_folio, 1 + - ((mapping || swap_cache) ? folio_nr_pages(origin_folio) : 0)); - - unlock_page_lruvec(lruvec); - - if (swap_cache) - xa_unlock(&swap_cache->i_pages); - if (mapping) - xa_unlock(&mapping->i_pages); - - /* Caller disabled irqs, so they are still disabled here */ - local_irq_enable(); - - if (nr_dropped) - shmem_uncharge(mapping->host, nr_dropped); - - remap_page(origin_folio, 1 << order, - folio_test_anon(origin_folio) ? - RMP_USE_SHARED_ZEROPAGE : 0); - - /* - * At this point, folio should contain the specified page. - * For uniform split, it is left for caller to unlock. - * For buddy allocator like split, the first after-split folio is left - * for caller to unlock. - */ - for (new_folio =3D origin_folio; new_folio !=3D next_folio; new_folio =3D= next) { - next =3D folio_next(new_folio); - if (new_folio =3D=3D page_folio(lock_at)) - continue; - - folio_unlock(new_folio); - /* - * Subpages may be freed if there wasn't any mapping - * like if add_to_swap() is running on a lru page that - * had its mapping zapped. And freeing these pages - * requires taking the lru_lock so we do the put_page - * of the tail pages after the split is complete. - */ - free_folio_and_swap_cache(new_folio); - } return ret; } =20 @@ -3706,10 +3598,13 @@ static int __folio_split(struct folio *folio, unsig= ned int new_order, { struct deferred_split *ds_queue =3D get_deferred_split_queue(folio); XA_STATE(xas, &folio->mapping->i_pages, folio->index); + struct folio *next_folio =3D folio_next(folio); bool is_anon =3D folio_test_anon(folio); struct address_space *mapping =3D NULL; struct anon_vma *anon_vma =3D NULL; int order =3D folio_order(folio); + struct folio *new_folio, *next; + int nr_shmem_dropped =3D 0; int extra_pins, ret; pgoff_t end; bool is_hzp; @@ -3833,13 +3728,18 @@ static int __folio_split(struct folio *folio, unsig= ned int new_order, */ xas_lock(&xas); xas_reset(&xas); - if (xas_load(&xas) !=3D folio) + if (xas_load(&xas) !=3D folio) { + ret =3D -EAGAIN; goto fail; + } } =20 /* Prevent deferred_split_scan() touching ->_refcount */ spin_lock(&ds_queue->split_queue_lock); if (folio_ref_freeze(folio, 1 + extra_pins)) { + struct address_space *swap_cache =3D NULL; + struct lruvec *lruvec; + if (folio_order(folio) > 1 && !list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; @@ -3873,18 +3773,119 @@ static int __folio_split(struct folio *folio, unsi= gned int new_order, } } =20 - ret =3D __split_unmapped_folio(folio, new_order, - split_at, lock_at, list, end, &xas, mapping, - uniform_split); + if (folio_test_swapcache(folio)) { + if (mapping) { + VM_WARN_ON_ONCE_FOLIO(mapping, folio); + ret =3D -EINVAL; + goto fail; + } + + /* + * a swapcache folio can only be uniformly split to + * order-0 + */ + if (!uniform_split || new_order !=3D 0) { + ret =3D -EINVAL; + goto fail; + } + + swap_cache =3D swap_address_space(folio->swap); + xa_lock(&swap_cache->i_pages); + } + + /* lock lru list/PageCompound, ref frozen by page_ref_freeze */ + lruvec =3D folio_lruvec_lock(folio); + + ret =3D __split_unmapped_folio(folio, new_order, split_at, &xas, + mapping, uniform_split); + + /* + * Unfreeze after-split folios and put them back to the right + * list. @folio should be kept frozon until page cache entries + * are updated with all the other after-split folios to prevent + * others seeing stale page cache entries. + */ + for (new_folio =3D folio_next(folio); new_folio !=3D next_folio; + new_folio =3D next) { + next =3D folio_next(new_folio); + + folio_ref_unfreeze( + new_folio, + 1 + ((mapping || swap_cache) ? + folio_nr_pages(new_folio) : + 0)); + + lru_add_split_folio(folio, new_folio, lruvec, list); + + /* Some pages can be beyond EOF: drop them from cache */ + if (new_folio->index >=3D end) { + if (shmem_mapping(mapping)) + nr_shmem_dropped +=3D folio_nr_pages(new_folio); + else if (folio_test_clear_dirty(new_folio)) + folio_account_cleaned( + new_folio, + inode_to_wb(mapping->host)); + __filemap_remove_folio(new_folio, NULL); + folio_put_refs(new_folio, + folio_nr_pages(new_folio)); + } else if (mapping) { + __xa_store(&mapping->i_pages, new_folio->index, + new_folio, 0); + } else if (swap_cache) { + __xa_store(&swap_cache->i_pages, + swap_cache_index(new_folio->swap), + new_folio, 0); + } + } + /* + * Unfreeze @folio only after all page cache entries, which + * used to point to it, have been updated with new folios. + * Otherwise, a parallel folio_try_get() can grab origin_folio + * and its caller can see stale page cache entries. + */ + folio_ref_unfreeze(folio, 1 + + ((mapping || swap_cache) ? folio_nr_pages(folio) : 0)); + + unlock_page_lruvec(lruvec); + + if (swap_cache) + xa_unlock(&swap_cache->i_pages); } else { spin_unlock(&ds_queue->split_queue_lock); -fail: - if (mapping) - xas_unlock(&xas); - local_irq_enable(); - remap_page(folio, folio_nr_pages(folio), 0); ret =3D -EAGAIN; } +fail: + if (mapping) + xas_unlock(&xas); + + local_irq_enable(); + + if (nr_shmem_dropped) + shmem_uncharge(mapping->host, nr_shmem_dropped); + + remap_page(folio, 1 << order, + !ret && folio_test_anon(folio) ? RMP_USE_SHARED_ZEROPAGE : + 0); + + /* + * Unlock all after-split folios except the one containing @lock_at + * page. If @folio is not split, it will be kept locked. + */ + for (new_folio =3D folio; new_folio !=3D next_folio; new_folio =3D next) { + next =3D folio_next(new_folio); + if (new_folio =3D=3D page_folio(lock_at)) + continue; + + folio_unlock(new_folio); + /* + * Subpages may be freed if there wasn't any mapping + * like if add_to_swap() is running on a lru page that + * had its mapping zapped. And freeing these pages + * requires taking the lru_lock so we do the put_page + * of the tail pages after the split is complete. + */ + free_folio_and_swap_cache(new_folio); + } =20 out_unlock: if (anon_vma) { --=20 2.47.2 From nobody Tue Oct 7 07:23:05 2025 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2085.outbound.protection.outlook.com [40.107.96.85]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 32D722F3C1C for ; Fri, 11 Jul 2025 18:24:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.85 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752258251; cv=fail; b=GGx4GtucB1VzNJ9z+VGQJpL2GMxEOYcsDsjum8TGCeBC2oQe8I47R/TeNVeprPI0bXWF+vBiDeEbCTka47eYwMR5uA/HsQ9OsDbvBFfYodCR//LXIZCXS8gtx9IITRsu8ZvNxRBwlB4N5jpJlKGTtvSC1tieI7oWqag1JiJqqCg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1752258251; c=relaxed/simple; bh=87KjgNpRzkDPfbBFvsCUfqzzQTgeoqN6hnAiMZGaZDY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=RISXkYezypc40Ysy/P+4POTW1SX64qZedOymzOc42Vy1W7CHDvHSWwc9HBAhxidD6Q3Vs8C9xGb5L6QlyRMl1aXK15ZkIbstvhNt2GuqkMs+/nQVyUS8V8AxhY8apcsUcgG+BSrOwhntVSz4jeuxj4bRgOyuSUxTPPxsrNKHqIw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=KNg6S3v8; arc=fail smtp.client-ip=40.107.96.85 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="KNg6S3v8" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=jTJbMa8fd+ZTh5Fyp1Gj2PND98i4RNYUiHmUGvRfmIZN3YD9dPOPsdpr0wJTAAczwV0a7njCPDux+O0dvPxutJrOltosE4e4B9vpDe+R5n8j0HuNXCu/suWnDagZQzef6aYUI/uRela8djkMyE/D1oIqawwiuKVIix9kHaoH5aWd9Ti8zkSIJrC3IzlBxBcrl1NmHUWpzC7MyLx5E1L7Z9lTYL9YeiW22Zhec7UnxVpfThYNALU9nwfz0bpRAwKlwPcajxWwmmS2uApZUMH6BSVlvLjMSKcHXKc3zQPq3DxgMeFCeWocr+f72MgjgxUCDD1po/3KIYsqm6nbCPqEWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gW7f5sOkljICYDVinO7KC+cA90y2qzq2kZI78FujKe8=; b=E+4uumKkOrvQlU3ctsaKvbPgiB8jD5yjL08RYK4XZjzas2orwOoS13bBeeQdcduc1K5QboIU3nNGlZeCXopzNHoiZ3qEzvPaaTFGoMfi/H9ntuJvTZUm9iB5nh+3Xo9Ec/IbWV4A41EE3PANQPjotbKlmARVK3cmqMFng4iIyhLfICTTj8Bh7DAUmiQQHhlKlyQ2MLwPSwCqUM7LTEZTZM7rnaE3r7CCu2gDf98/BplhSJZw7bU/U1yIMjYBo7YUB+HIwoPiTA+vA0BDsi3HyYPOXE8/Mlqanxa5jOQ1V7DHo41IaySJXc7Sy2DxMeVNu36LxAXAiag43t1U8WjwCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gW7f5sOkljICYDVinO7KC+cA90y2qzq2kZI78FujKe8=; b=KNg6S3v8EUx6J49KAPo+Xn2vynSXC9UOWs0SORgIn0PiXNBpbsV+66B7H6kkyub3YV0UV4Ql+2vjGKgiGnuotIUgAdNXns/SOueVbYdm728HaTXrQd3u4pwJPNsc36piOoKUuUILu0LMUu9XxY6evOugSZByLbUODNDCes94i1w4XV/P7d9XlcD/EU/dodexDi/9/xX3NgS1WG28Du7VxIfqVA+3fJ9kbxsVKw7a3Z1dQW1DhZ7NheCGeMWZuucvociybWTpebkDEC77jyJHGSlt7AQE27CTF7jmopz5EoGcBVgPL8eNjyc1tbD1l1UMwV4uPAfUdAuWc8U9q4UYEQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CYYPR12MB9016.namprd12.prod.outlook.com (2603:10b6:930:c4::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8922.25; Fri, 11 Jul 2025 18:24:06 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%6]) with mapi id 15.20.8922.023; Fri, 11 Jul 2025 18:24:06 +0000 From: Zi Yan To: Balbir Singh , David Hildenbrand , linux-mm@kvack.org Cc: Andrew Morton , Hugh Dickins , Kirill Shutemov , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , linux-kernel@vger.kernel.org Subject: [PATCH v2 2/2] mm/huge_memory: use folio_expected_ref_count() to calculate ref_count. Date: Fri, 11 Jul 2025 14:23:55 -0400 Message-ID: <20250711182355.3592618-3-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250711182355.3592618-1-ziy@nvidia.com> References: <20250711182355.3592618-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0P221CA0027.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:52a::21) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CYYPR12MB9016:EE_ X-MS-Office365-Filtering-Correlation-Id: 5c5b6111-119d-4b2d-a234-08ddc0a81f39 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?clwESH9m632w73DAlL/WeeyKPOyIgPrSeqcl5ewbFqRa8/xKGn1eSHNyTxBz?= =?us-ascii?Q?euuF5Z5Qqy6rj8zBVO64ZOuyYNKRlei3hTKpftAOKr7cA5w/MABHml8yZm5C?= =?us-ascii?Q?t/fXm/OH1YkWy8U+sPq31fFDbDBU5u5GTvVfABZ1MI9MUobDyvoYfMFKkgAb?= =?us-ascii?Q?7yt6S+ZKhVb/pkgdVjxVz9+prZ2A2C52+xTTLZdwGZagz3am/kJFqSM1hsEn?= =?us-ascii?Q?EH+TSctUIG5b9G54sh5WmvEAgTPgZ3n9vJRMIeH6tuiJRSemDhLjDO5NZAgY?= =?us-ascii?Q?kKwfZnEewlYaEP1lwq8lg1M6xgnOnjELkQWk3fKs8X5MX9+uPb2KtIg2H3QI?= =?us-ascii?Q?Qj44U3qEae93gtxhZzV3WKQaYhAk7DvGRsTPeVElY9yy+tRCHzNATivKZFaD?= =?us-ascii?Q?0+eVG0noWY0PZp4FMOxtANoAD2ueGkjgyIuSsHzalFctd39MrQcvAuIhhE1W?= =?us-ascii?Q?8DpSStGjBDSXzuFKyVlyXgwHNjIji/Z3W/vW3ifLMez3U5T2Tb+WESw07yna?= =?us-ascii?Q?eS7Q+kQF+tsV6UEchIw0EkgR27oej+bXT9tteNoOeEgiCbXn9lzIRK7EYkQT?= =?us-ascii?Q?5A4OWaqofmz2bE25E8uN8TDB6Fyy/Lb0ALLSFnyiyqQgcW0/+LqLaUzE4l+D?= =?us-ascii?Q?YpL5ahb9EPgMZHM1HM2XOvmsCrRla1KowR0lwhmi6EZHeit53joIYW3pk0e5?= =?us-ascii?Q?M2GitwfKV/5INuzwp9BGaR5nH2s+jTq9POluF6uJdDoS0nqqgpe496DbloE7?= =?us-ascii?Q?00ZBsjqdqNMo/19vIrGRiuIHpunRHhAcGPmnIHCLQqqY9omkCaml4Rfg13F8?= =?us-ascii?Q?xgULk49zSlEFPA/cyTgfkeF2PvhtyZNo30S6IyIpO2ndbpumaFikSHJFrflD?= =?us-ascii?Q?CpP2aJIDFDM0vr0s24D8zd8wU5RqFRp7fCjU32nNg77ZzJu8CvpMtXX2PICl?= =?us-ascii?Q?5bJnC7JWABemOUkEfiP0VPOtAqqGF1MglxeosJbRPjClO0Nxgx1VGlbVkHOD?= =?us-ascii?Q?hcavaypIa/Hb4KUSmffY8k2J3PKLgfg5ltMg0z/fx+dyUN32GwpemrC8l05o?= =?us-ascii?Q?XAD71LDfu52CSAx4WDKGzARZsfEUc/Fn6kdmbxL6IOkH3VAeK2enab4JcJN4?= =?us-ascii?Q?lqoE65Lg59zhcp26iIpiMMkeiMGxe8W1S9ZQ33fNYZIB1nMSENpkYw9lhWQq?= =?us-ascii?Q?oqohY68ilu6yMjAUeZs8Nz9UAMzV1C+bTWrszVvf18NjZrIUHcqIHQcmWqFS?= =?us-ascii?Q?8aUUXyu4VNyOPFOQSQVK/vwBVxBL4y/zRS4gAp6cm1whDzKp8g2Y0pg2sQRt?= =?us-ascii?Q?+xcz9UpN2HZKyySJz9uQ7ugNat9Hw7eLAI7GgmDIMfnYPMowYvuK+WzBW4k8?= =?us-ascii?Q?SEBUJ33apQDxjQzjZtT96JpDtPuMVyiN1zFAGk1kDMuLpnYSpMikVKCjTMXw?= =?us-ascii?Q?eRdCd4H74cg=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?b9aNEFJBrV33ZuKmeN8Gwy59XG7M5iDU7ibJsnRHdJP8nvZNynf3mGKAIKsf?= =?us-ascii?Q?I2DjkGdeIVldWSiNNNCSIWiwbnCslot7RQcfXT2Ixwc/fajlkFFBWaTiDsJQ?= =?us-ascii?Q?XkVw3+0s8XHLEeRde0vt/a9xdTYjIXbnrLRkaD+B9S3Suj3/bizfumZJw+06?= =?us-ascii?Q?a2O4JhvuHnHnVdjv061RqyQdOu8mmdMEugEJrzwaWbtBD2y4fm+t3sO6ZXh6?= =?us-ascii?Q?x0B9soJwSF50Rmmv8Ke7zVfrR8IydCO2Q5oEYb8btJR7oGRgFNDtp8BsnOLB?= =?us-ascii?Q?QsGWu0nfTOlbOCRXmZ8vn4UEpJ/sXxFRCR3w9mSvJVMXfGhUQ71XHVgjQ7Wx?= =?us-ascii?Q?OK2c70n1u//aJ1zZNL+bjPkHE6Kp8P1ZleMfoAY510hBYTX6Oad115csOxEF?= =?us-ascii?Q?wTbl/k1ourRM2QNnwbFDm0daRRhXpUtmm8z83/WbCvp91tV3wt7Y1stNd1iZ?= =?us-ascii?Q?71dDfmYeOkkJeJVbAmWwQKVoAyq5JohG9jwHx7DCJJJrda2Awo6WIiNMYv1q?= =?us-ascii?Q?gkuqEVswgVQUgRKxtskpn4GZODIsf710jOK71KQr4BNSIzgTIImJ/iDkUKaa?= =?us-ascii?Q?fV4vq+uFJAwWEiw0CpiXBklBX6JRqQ12UlkbyRc/2P/IGwlNWiSauQFWgBUx?= =?us-ascii?Q?W4dOZ+v5SEPhfjv1uH1TsG5zoMigp+yqRuUpi+YeCGY4N7I1AU7ch54Az4lW?= =?us-ascii?Q?/nc15yHTYNEbwp7S2vow/Xp87O02MzIk/An9FIpgRdkKN/rBtIWQLTLKXWyv?= =?us-ascii?Q?NhKzsn4wqUB72dQCkJBBQdnKJHK1y/8eXOqyAMXGLt539aQ2zxlDDte1iONW?= =?us-ascii?Q?XQLoo1tlwgYnbBSDHzM0z6MihnhEIl6irhgDqmykIUypNSn4i0ePbMc3FaWl?= =?us-ascii?Q?+eW5VZ5dbK+dazwXvj9p5trrpBdf3GVyyjIBDYevqyn56nWFmiSKd7q3i4+p?= =?us-ascii?Q?Dzss5lhFCld0XGLYBrMQfQ3EkpoBCbJT6Ici+pu1Rz+KmL4OVMU75Qh1kNl8?= =?us-ascii?Q?dbM+kb5uFC5nAOjFMclqKYFUukYL5/ZoqbVYAwMPamB8xTGMAGGteaAkm0Wj?= =?us-ascii?Q?LIakFPdqhK3UWa0PWVNbOeKGXwO66pr24hssuU1CkNT3npKPpAQUd63Sgv4c?= =?us-ascii?Q?VkBTTqMN24879DnCYc8xDLn8B3ist7i8xHLhcBlMTurQb5iYkWn+/B7A16mi?= =?us-ascii?Q?VHaVf0mTtyvfq2BpZL3+CVN4dxuVBVVVrChpa9H9PtaaNfS+M+HBVGDPA3Tx?= =?us-ascii?Q?Cf9Ah5eoScsbaKeA89kX1mBi7C3DajK4v/MjYXxsM7JOauVd3qpLPoakdwxb?= =?us-ascii?Q?TgcWYP516plIUsY4+t3/vLgHGGIsTNQ/T/CebriAA1B+y+IEIetjDmScTI8o?= =?us-ascii?Q?d4zAq8ULstv4NGLkQjDKqS9TRLbMmJH/l67pceycGlbd7sLWpJgO2o6urgwd?= =?us-ascii?Q?mwlFu5hAQWebtkCHVJjE/6NmNcd5w7X07NywUqXWwIbhLDeyxL6rwVGIXeD7?= =?us-ascii?Q?29bm1WsphtBZgAw6Cwnlcvyhjuttx4uIBKERiUn4VpCsFECUn6RM5ebPOHPK?= =?us-ascii?Q?JHDL6CKl7CrVdJpGKMvevXqCUH9IaLI9Hhlm001W?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5c5b6111-119d-4b2d-a234-08ddc0a81f39 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jul 2025 18:24:06.8666 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6jdiFRs5mdDQxIOpvEZafQaiySA9I0iAbRXmLUMmGkD20/paMYsp6iEf+y/Xw3Nr X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB9016 Content-Type: text/plain; charset="utf-8" Instead of open coding the ref_count calculation, use folio_expected_ref_count(). Suggested-by: David Hildenbrand Signed-off-by: Zi Yan Acked-by: Balbir Singh Acked-by: David Hildenbrand --- mm/huge_memory.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 818a6bd9f0d4..57e5699cf638 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3739,6 +3739,7 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, if (folio_ref_freeze(folio, 1 + extra_pins)) { struct address_space *swap_cache =3D NULL; struct lruvec *lruvec; + int expected_refs; =20 if (folio_order(folio) > 1 && !list_empty(&folio->_deferred_list)) { @@ -3809,11 +3810,8 @@ static int __folio_split(struct folio *folio, unsign= ed int new_order, new_folio =3D next) { next =3D folio_next(new_folio); =20 - folio_ref_unfreeze( - new_folio, - 1 + ((mapping || swap_cache) ? - folio_nr_pages(new_folio) : - 0)); + expected_refs =3D folio_expected_ref_count(new_folio) + 1; + folio_ref_unfreeze(new_folio, expected_refs); =20 lru_add_split_folio(folio, new_folio, lruvec, list); =20 @@ -3843,8 +3841,8 @@ static int __folio_split(struct folio *folio, unsigne= d int new_order, * Otherwise, a parallel folio_try_get() can grab origin_folio * and its caller can see stale page cache entries. */ - folio_ref_unfreeze(folio, 1 + - ((mapping || swap_cache) ? folio_nr_pages(folio) : 0)); + expected_refs =3D folio_expected_ref_count(folio) + 1; + folio_ref_unfreeze(folio, expected_refs); =20 unlock_page_lruvec(lruvec); =20 --=20 2.47.2