From nobody Mon Feb 9 04:03:54 2026 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2076.outbound.protection.outlook.com [40.107.96.76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CB0B1DE8AB for ; Mon, 6 Jan 2025 16:55:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.76 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736182542; cv=fail; b=OyDScWgUFIQkA5kS3f8mBRX9VhNOjR4EVMQybLe7LTJ/TJCD/fqbFzuq+lXXMAzi2zyArD4AfSbAK17bG8TBmlmoMmaZ/L1e0QZsxdTzYn3xZ2YFHsFG91FH/b9LpKruxECmMjDCnPnbddytwD4G23u96XT2VpQi8fwz9xNzNpA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736182542; c=relaxed/simple; bh=xmhSi88GmLWc3bvmwFfTKPJTjKy72cJX5JMaKiO4k6E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=hhhnjQU13vrm7VVA3JoqGtCL/sz00ZLm0H01izXZCHuUL2PcKcNo4tLJZfkq+jYAnOzlji5nqEsSNa13ptmwi7kfi+yJZ8s2oh3g/3wBJTwUkZHRePPr7tlh6h4lO4zhpiVdRJKPlvZgmG5QlC5fi4n6UrfsMsYbFYswXVCTZXQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=SZuQG6rh; arc=fail smtp.client-ip=40.107.96.76 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="SZuQG6rh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=FtUorw4fnz7gfZIeiP17bNOg9tqBoV9Aq+GPXxl+Hiut3HLB1CADXzHw7H0c1L9NeF2bce0NvvDgZ3IS6HOY21khS4wpc87kabSpFtpdqbIBB1/8j4TLBpyUmhymEt9l7R2uPYpG2P9B6R+wXCNXa5RgI759isMgyggYBa35N17WNQITcfKcXSEeUN4UwizEDMpqz0TIRcVBu7u8ZZLKZ3gHfKvmC2nqiWBOqbOC83tyrTeN+9X5y9qco6NAURywQEu/vTW7R4PUF0GCEiC12XH7NkZO/cWn0oAm08j4Jo8s9kPA4ijVdyiIGfdaZh13T00ccORlybkwYn4THl0Tbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SPvVoJvT4mBBzU5IHfOt+qRzccUAAB1+CNSz4Sg4YUY=; b=J8MzkTQTeqzfa15TpUiYMyapCuwNajPvyXm3yanQNnS3Q/EWrT8c+PdZiiIP1Hb4HAIRV3In6yl8ItlQQj+36bZnLonYXK+d0O/qjY6sEENfGkGCIzucm+dANbNdz7D7euzKx5wKu3Hu+bJBf/KdwhIUKEc7gtPnIBqphDvoiY77tJPGaARLXRO+ZZM5Ij1QOjV9RrWNKUuFPhbM9Hb/zMAyNGvSvyABLywjpUHiNVyiRhm7kIa2Vn0wD0SWg904qERsK0Yup3LIpwcI2ktL4RX7LYCMGn6CALvN7eESZmnBLtSOjBk8mwoZVQcn2R68SWghYVGOVfO1wBJajpQweA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SPvVoJvT4mBBzU5IHfOt+qRzccUAAB1+CNSz4Sg4YUY=; b=SZuQG6rh6hqu+FEmnrS0uHer/Z/LSQS4dQtNEq0j+M2haqwC+Eb1BAhJOqvzTa7J5n+WjRAKLJziwBF3zaN9UJ2yBXb9+a4aoBr9MCU1VvShU+MhSULzpvp7jb759HO8EdBeswGHobao/PZ0J1WQPVYK15nbg+lX69UUDACj4K95ULL2Wp9q/+D1QBTpBMa1zAnrSFPVb1BJOVTpFVrb2NowipOhEpDu+t0VjvJRBiFIIN5nESBnikjKscxTyEGXm/mlxwY2lKggF60CQkhmhcNhQ2BXhZHnbqoBHFXy4Exyz3HZqHoF7t1WICyjaTEi5TtTmra/U8Vq62ncPanBZA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by PH7PR12MB6634.namprd12.prod.outlook.com (2603:10b6:510:211::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8314.17; Mon, 6 Jan 2025 16:55:27 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%3]) with mapi id 15.20.8314.018; Mon, 6 Jan 2025 16:55:27 +0000 From: Zi Yan To: linux-mm@kvack.org, "Kirill A . Shutemov" , "Matthew Wilcox (Oracle)" Cc: Ryan Roberts , Hugh Dickins , David Hildenbrand , Yang Shi , Miaohe Lin , Kefeng Wang , Yu Zhao , John Hubbard , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v4 05/10] mm/huge_memory: move folio split common code to __folio_split() Date: Mon, 6 Jan 2025 11:55:08 -0500 Message-ID: <20250106165513.104899-6-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250106165513.104899-1-ziy@nvidia.com> References: <20250106165513.104899-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR08CA0022.namprd08.prod.outlook.com (2603:10b6:208:239::27) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|PH7PR12MB6634:EE_ X-MS-Office365-Filtering-Correlation-Id: 88cb1e27-229b-4cc7-01dd-08dd2e72ebdf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?DqEU+yVDVswf/iqKrnHlgBDcHIFARiocXPoBx2D6Ugj4W3Pp48J3iVil9XNe?= =?us-ascii?Q?jJTaDEpYRbs9n+yh/EzpdEIj0/VnJqf6dQ2Qzqft/cn0umbXvc6bI5gj455Y?= =?us-ascii?Q?L/HgGhDHpDjBooZINh6uokeE3X3Y7fXHJCI8dERHBFD+VZ1Y2W31nOOsJzkV?= =?us-ascii?Q?ksSxEHvoq9xEhJrctYwe8smMNqj+I3aA9MJk9Tyz3QssuwIV5PwYxvObb55o?= =?us-ascii?Q?/l6zWA6VwgKgE0bCBNObgJcb5YflqPAkIFz4k2RmawQ0qbnqsGc1+YQqwOsE?= =?us-ascii?Q?rfvNaIt/d8PwqzI+7filzsDtIqhQy2uaocXJhj+CqSx84HDtOnluD2o5mElG?= =?us-ascii?Q?Vc/VA0kmesLVc14pI7v2YT8vNss9pdEnD9jWU67ihLl+DP1kLK1O2jg3V8vz?= =?us-ascii?Q?PszGuS+PaKx3/lgelbqz9SILNwD/gFZ8N+Y2O6pTycP9ZrNNPba6bpu/r4XU?= =?us-ascii?Q?sxiHAnOzantPzyQns8S5sLHdMlkycDKianHUiiMJrZn/8wbzMEUA51x9aqdt?= =?us-ascii?Q?yU6Nw00nTrVG43mB6XXQkTXJH9+ygb1OtiNHCVez5kXfxdHiA2QN37t9/0qs?= =?us-ascii?Q?NY+xWQtuTaB4IGfs4CvBdT5mxuVZWRTeNO2xG49u43ebueBX0WkC6gnLY2hE?= =?us-ascii?Q?Mr97heStK3CoZnZVGqZK6YHOLNryQYNqJ3RkaYODJz7L6vpdffcZiAi1TL78?= =?us-ascii?Q?4FuV2Mt4Kdf4EqF/81UVRegLDyFAilKLYZRhyIqFDLSE3YsTNuE1qxRAT3bS?= =?us-ascii?Q?MQz2dwkwsk30v8+fJkAFU+XFLKfeJzX044w0xO1kHTTzEiLANfwaYwZq0Ktz?= =?us-ascii?Q?FJaSVqh9lRzxt8Zh5xCkALRgOk4dx7gy+NimrJUNFpG2Hye4nayQQ9tJElAo?= =?us-ascii?Q?DVCdjiQeqDg9f0AF5FjA8qhQ2FQTWjSAbV0fQ5Oo192sSDbfVt5CP5KlazLr?= =?us-ascii?Q?EG4ikvARn/EfkjCzJCM58Beur6MnnlbfVZWGD6shyJZVLXyyMpwH6G3aSYsz?= =?us-ascii?Q?T4Ou+jkp9dc9NikMyUJB05XgKDexxFOXYQGuYKOa/3c7a7vsrjLwYYltMkA5?= =?us-ascii?Q?okczu5KKq2FQ9YTVPdyvsbdGDnJGqz/X+YFS4IJll9YsXQAbnsmoeNGa0l+c?= =?us-ascii?Q?K+UNWgpP/jntIo6bodDs2lCA7s15ZAkC4WowxTFBngwq8kXCqP1nfA1ZkoVA?= =?us-ascii?Q?YThHK2G0FS2x34XD1knXs1V0nFl3Z3xaiEXKJKyCSKHLDKaj6Ne/Qr2TpPzi?= =?us-ascii?Q?fwAxMrHnhfrOkgCwG0GNURoM495Fwd2gPLo0uMwIdbi+VpQiogIhLAfjFMNZ?= =?us-ascii?Q?i1fEDPqKNRdfoBfvvusLLdFlbUOjBEZSlt+IfteCWRi/Zv5pjaB8PXTVSJX7?= =?us-ascii?Q?cNtHT1kzyFizBu4a0ISse8QCeQTY?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?NQ3X1QnXhu920HW/Ke2kCZAiyZKTYX2gOT7Zzs3WbxbE9oMdLb4S6YGGrOHh?= =?us-ascii?Q?Cbp2+hZNwCaPGkZlRCyv5tNkvzuJCakazBMonD3E9AsrIKR7pSLZD6ftn9cu?= =?us-ascii?Q?/PsZuqj1Vm1KUjhEVICxPNrMr/Ws4cfmYqfRQjqHRnVnY99snkScBRFJ8jiN?= =?us-ascii?Q?c1N8yEqq7uxoK9g+4kNZ0ddcNICoqBhxlXdynDaSXl7YchCnx7Anj2ryLyIi?= =?us-ascii?Q?pZv3KB9gR+g7L0aPgX2JtYQPAfwWtgasXiO5v3JgNoRahKqDqq7RMwWAUK4I?= =?us-ascii?Q?tXCEdFGsvKny0loFgzyYd66MCmnHBtUN363ExtgREj0cONWRty1GxAR8NtTQ?= =?us-ascii?Q?AxJ35HFiH7hjv4yibG1425NrbMTHZYWBah4WyoJtMFlKGsWLZWglUdJbeS6f?= =?us-ascii?Q?yfkn5xiGwRWz+XY4jdm/a51sBjJrPtMHfIhe3SCLOE2L8hkFfK/OjdBLN2zo?= =?us-ascii?Q?qCN6aJF6qDA23h17pJvcSGFDA/qf7FRaorDHeIdwy/ZqqblyZC3HBjCfUpT9?= =?us-ascii?Q?2MEuGQYTuoAW1we9Wfp/8RGpvQjILfX9ptys3F9L1vq8WIsnQ/X4e6EPgO2y?= =?us-ascii?Q?FPq8B2+wT4J5MgK3tQH0GjVzUmoocXnCiKif0RII/f8azGxJvklCQtxxEIl2?= =?us-ascii?Q?xCJfH/Iu+y5dhQ9E1xcRsvutQORMh/bC/e7BpSinHvTH54mCX4AIbQ3YRkJv?= =?us-ascii?Q?YSO7xZsPFLElNFjQSr5vznZzWsKklWSPH9qGpg/wW1fvns1q2YNNi2taeTRn?= =?us-ascii?Q?tzEwLtE5hPJLE/VEm34stdUe2W+y+4LM8SuRxy8zQD/cWoghGcYANHn4Ku4Y?= =?us-ascii?Q?bd0E7k8aP3PaCrrSZIk+GVIEcS7FX+WPPeP3DnoWRDVPgaClyaGKL0XvRoHY?= =?us-ascii?Q?DkUHMlXCQkJm1HQwblF3uGp939+pbxxFx6TDRYrAb7l8K58SNnX/OwWb31Lm?= =?us-ascii?Q?PtQGImoRlPRw5t7tqDWGprN2V90B6p4RJVrxHPuzULKVrZwMhbZJxoOIZuTL?= =?us-ascii?Q?kYAjQrXZHYsv24bfYnEEQ5ZjAu7+hvjQ5rf+V3ZgQMy9RVm70bUBd/MPdjzn?= =?us-ascii?Q?yn+IEVS2si0FM1QtwiXBtC4lpjMfZiO5c4SjTF5LY2tRE7nrYO2x/xz6PgXx?= =?us-ascii?Q?RLNfgmHGHKFIfPweJaAiNeqamuwuKo7E2m34tFNrjpHHhYByG10OM+Jw6ruG?= =?us-ascii?Q?0fR45ECmz4T+4W77nbBhWxv0o2OnavPLWpSA/RFI2b+PmaxY5HUHl1s4zttu?= =?us-ascii?Q?1qynWiAhdFr0zd5cnzfn4fOkcG4e7fL7NWoyh7dlo233wA2Xtpa6ltSr/D5V?= =?us-ascii?Q?eLRzWCW8GZVmZIGAOI/Ke3u+1++lq8FfEvIy4e6ss+Q6/3Un3MfjsbYEIW8B?= =?us-ascii?Q?5UinQae+R1HIY2/Rh4/pUm3sbGjMKSEpJpyexrueb9cQ6etqsLj/wjNGhAy1?= =?us-ascii?Q?FlKlahmafSyX4e5alSWFGyG5rui1SS2m5YhHj0mJTayBRW6/bTnLmiIGYF1w?= =?us-ascii?Q?/zbAubUKPhiR3HMnbvFi7ZE2iB2IHWye+WMvc65E5bTuK/kTrn3b6CTJbHhR?= =?us-ascii?Q?RU81E0HacpkcOPtZrprKdtKnhHsWd9ShIm3+hXVs?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 88cb1e27-229b-4cc7-01dd-08dd2e72ebdf X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jan 2025 16:55:27.6119 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: eHS5XiGVcKiTcYLOQ9lOclptGe2xgJOHMFq4iy11H9ikwkX/7Wls+Rs6EMyXd0jn X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6634 Content-Type: text/plain; charset="utf-8" This is a preparation patch for folio_split(). In the upcoming patch folio_split() will share folio unmapping and remapping code with split_huge_page_to_list_to_order(), so move the code to a common function __folio_split() first. Signed-off-by: Zi Yan --- mm/huge_memory.c | 107 +++++++++++++++++++++++++---------------------- 1 file changed, 57 insertions(+), 50 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d8e743f81e76..586870e60003 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3724,57 +3724,9 @@ static int __split_unmapped_folio(struct folio *foli= o, int new_order, return ret; } =20 -/* - * This function splits a large folio into smaller folios of order @new_or= der. - * @page can point to any page of the large folio to split. The split oper= ation - * does not change the position of @page. - * - * Prerequisites: - * - * 1) The caller must hold a reference on the @page's owning folio, also k= nown - * as the large folio. - * - * 2) The large folio must be locked. - * - * 3) The folio must not be pinned. Any unexpected folio references, inclu= ding - * GUP pins, will result in the folio not getting split; instead, the c= aller - * will receive an -EAGAIN. - * - * 4) @new_order > 1, usually. Splitting to order-1 anonymous folios is not - * supported for non-file-backed folios, because folio->_deferred_list,= which - * is used by partially mapped folios, is stored in subpage 2, but an o= rder-1 - * folio only has subpages 0 and 1. File-backed order-1 folios are supp= orted, - * since they do not use _deferred_list. - * - * After splitting, the caller's folio reference will be transferred to @p= age, - * resulting in a raised refcount of @page after this call. The other page= s may - * be freed if they are not mapped. - * - * If @list is null, tail pages will be added to LRU list, otherwise, to @= list. - * - * Pages in @new_order will inherit the mapping, flags, and so on from the - * huge page. - * - * Returns 0 if the huge page was split successfully. - * - * Returns -EAGAIN if the folio has unexpected reference (e.g., GUP) or if - * the folio was concurrently removed from the page cache. - * - * Returns -EBUSY when trying to split the huge zeropage, if the folio is - * under writeback, if fs-specific folio metadata cannot currently be - * released, or if some unexpected race happened (e.g., anon VMA disappear= ed, - * truncation). - * - * Callers should ensure that the order respects the address space mapping - * min-order if one is set for non-anonymous folios. - * - * Returns -EINVAL when trying to split to an order that is incompatible - * with the folio. Splitting to order 0 is compatible with all folios. - */ -int split_huge_page_to_list_to_order(struct page *page, struct list_head *= list, - unsigned int new_order) +static int __folio_split(struct folio *folio, unsigned int new_order, + struct page *page, struct list_head *list) { - struct folio *folio =3D page_folio(page); struct deferred_split *ds_queue =3D get_deferred_split_queue(folio); /* reset xarray order to new order after split */ XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); @@ -3984,6 +3936,61 @@ int split_huge_page_to_list_to_order(struct page *pa= ge, struct list_head *list, return ret; } =20 +/* + * This function splits a large folio into smaller folios of order @new_or= der. + * @page can point to any page of the large folio to split. The split oper= ation + * does not change the position of @page. + * + * Prerequisites: + * + * 1) The caller must hold a reference on the @page's owning folio, also k= nown + * as the large folio. + * + * 2) The large folio must be locked. + * + * 3) The folio must not be pinned. Any unexpected folio references, inclu= ding + * GUP pins, will result in the folio not getting split; instead, the c= aller + * will receive an -EAGAIN. + * + * 4) @new_order > 1, usually. Splitting to order-1 anonymous folios is not + * supported for non-file-backed folios, because folio->_deferred_list,= which + * is used by partially mapped folios, is stored in subpage 2, but an o= rder-1 + * folio only has subpages 0 and 1. File-backed order-1 folios are supp= orted, + * since they do not use _deferred_list. + * + * After splitting, the caller's folio reference will be transferred to @p= age, + * resulting in a raised refcount of @page after this call. The other page= s may + * be freed if they are not mapped. + * + * If @list is null, tail pages will be added to LRU list, otherwise, to @= list. + * + * Pages in @new_order will inherit the mapping, flags, and so on from the + * huge page. + * + * Returns 0 if the huge page was split successfully. + * + * Returns -EAGAIN if the folio has unexpected reference (e.g., GUP) or if + * the folio was concurrently removed from the page cache. + * + * Returns -EBUSY when trying to split the huge zeropage, if the folio is + * under writeback, if fs-specific folio metadata cannot currently be + * released, or if some unexpected race happened (e.g., anon VMA disappear= ed, + * truncation). + * + * Callers should ensure that the order respects the address space mapping + * min-order if one is set for non-anonymous folios. + * + * Returns -EINVAL when trying to split to an order that is incompatible + * with the folio. Splitting to order 0 is compatible with all folios. + */ +int split_huge_page_to_list_to_order(struct page *page, struct list_head *= list, + unsigned int new_order) +{ + struct folio *folio =3D page_folio(page); + + return __folio_split(folio, new_order, page, list); +} + int min_order_for_split(struct folio *folio) { if (folio_test_anon(folio)) --=20 2.45.2