From nobody Tue Dec 16 22:30:09 2025 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2061.outbound.protection.outlook.com [40.107.236.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 556F422D7BF for ; Fri, 30 May 2025 16:22:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.61 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622167; cv=fail; b=eDqYQ61vUDzzUmM26Y6q9WCN7oOn+E2sxWgigNtLBpFeyqVsWcWAPYqTxTXbDHI7JV40uWjmbdviFPhiJM/MvWSOkXtEn/AxtVP0I93eqjPMC+N5y88dJSBJXfvf6BJB1Zs+QRLpMhRbVMw0yBAcUZTlKWnou6bVSx/0b6Pfhtc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622167; c=relaxed/simple; bh=VBOven1z9DwX0i2cJZT3s0wMPPXL1hFYLuID8diWdxg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=L0lWwPXcZceNEyoKhcNJpjXmR7HOs+bZPYx5pRmDtNkeVLKle3rcl6F2rN3Puwup96S8uyLHP4Oba6OEJmY22Kprj2OTtutVuAWVM070fXryxGp/rzOx68MBZfXELMt0ZRLFvEmiBRmzLb8ZxZdUxWImcTXQdGgjp0QPnA7kx2A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=VV2xd/0E; arc=fail smtp.client-ip=40.107.236.61 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="VV2xd/0E" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Pha4Y7MSWD945GBcwbSdbZD+n4YYx+GbmwNItJjsK141eAdiT5Q6iuyUANdxKIA8ed+aQMUFh3YjXWgSfvB/fMMtyIbzr/hGzIH026jLyc/io9t7tNQobKhs6S/Aa4HHRi/YhY0Z452fvScWD5kagn43zAtLSYkrfhkwqudD4D92FE4CcqFbfDdeggYBGsNaTxQt4BHZ2lxYz6HWaZXI4oSpmdcumkBRbWSyEyoNLR7LlTOOPtAKGKWxtmhXhClU6d7uVGQU885JPpL+ozCOVhii9Tb+R6UdIiynyAKs5vqfOsQPVTPpY1JBTiqoolAbWL8y0F87T4+VxB/XZTET3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fz1Zb+kfv+xcRFBOLl0PvJfG59HBvmSkJQMvhf887Q4=; b=ZseOclNutLBTdbm6G3sCJoNgqzF7/yGOxj2C799lq+VDmxb/h5+6pP3u0Jp3oag5QsxPeClharUN1S8dKw2NbrsexFT/du+Fbz6sDQHdU6fqWWYNp8P/Gp2XvrPO23FwY9aT6VVrtpLhsLL9kOIYVfI+SWDhh8dG1H9g0y28dHimV9maVB8intI4S2eiBLTDtBMslwm2bedSqFrNgIsbuQR8UV8xMQtaNA96r06uA3rlolZS9gfGQf8zRrYeKxYSvniJDN/QFv6LkO5JBfXatgpGz2DOzad4OxkoA2Wrd54eoqqVaFpkWblFsZ8CeLTltiqi/Mfn/Dz3joKVLc+NPw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fz1Zb+kfv+xcRFBOLl0PvJfG59HBvmSkJQMvhf887Q4=; b=VV2xd/0Epph3FqYO+d7Ye++lQriSrkutI1Kn6TLGWrG6GTN3QVKyRreNSnyOS4igSkIWZV/1zG9nkaGCauexPlf9GmkI9Dd4rox0d7EMWWikRWVPeUyxjagXhplJI2HiWMi8Ea8BY5+bcsQ4U42ujGUEWofIiWGJP5Xp819o9eUUJr3TH7uyeKk5G64/Nki1SBeq8KreXEZK6hjLhqafokEW6Pux50A3ruyteQ0h0Rx7yp/6xO3vq6uCAXV6rzowW2YCA0RmZbTfbHx5W3gP6ufEidzr+yKBsmDuDVExCIlY6zUTWM27iG1RuyKiSvXEqnl7cE39GzjFbM1s7zX+FA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CH3PR12MB8187.namprd12.prod.outlook.com (2603:10b6:610:125::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.29; Fri, 30 May 2025 16:22:39 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%4]) with mapi id 15.20.8769.025; Fri, 30 May 2025 16:22:39 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v6 1/6] mm/page_alloc: pageblock flags functions clean up. Date: Fri, 30 May 2025 12:22:22 -0400 Message-ID: <20250530162227.715551-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250530162227.715551-1-ziy@nvidia.com> References: <20250530162227.715551-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR15CA0012.namprd15.prod.outlook.com (2603:10b6:208:1b4::25) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CH3PR12MB8187:EE_ X-MS-Office365-Filtering-Correlation-Id: 6d38e52c-d3f7-454e-0beb-08dd9f963239 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?eqJdAkfvxFWJiRCffkWtdwLb0587uMQj0F24YuecC4gt3VdJ/Xc46gJOq4J0?= =?us-ascii?Q?ZPkCVQ7xZunzlnSA4T5PUGXvYbN41HPes1abcXJ9ea0LTW12h5pw/xt3eg0O?= =?us-ascii?Q?ZaHSDJwg+GfIX/Tqubq0Dm2kG8yguk52ZKlWrRO+wcuq4a5LdXRA8fEebNK8?= =?us-ascii?Q?vQNTx7xwfKDPQpxrhmPBunw8rSxvd8U0iQeQIqRJO0W1LRL7s5mNBOVN7NOR?= =?us-ascii?Q?JtF4sfn6hOiNiRpWTkCeBdeBiOaLTrjtvGMWWfF5Du8Wo67ZFGm9+Qn4KI3a?= =?us-ascii?Q?q8tL9jvQf+bbCYij7Fwn2obReXIFHCXLr5g0lVM1YxgQMVq1hpdyBpacW6xE?= =?us-ascii?Q?7sj9h0mrfmXbHcsu9vQ8HNOO5nBDRLGnhFQ3C1zbRAnqnDdFPl96GGQLICNG?= =?us-ascii?Q?ebz9WSh5xtHSMEY6PihXWVbKn6tR2lzlkLvcm3RLP5lXM67T9YYq3tJrDP3e?= =?us-ascii?Q?zrrTQcjcw/Qgiz1DgOj4GzmQ8zZVghcnTUnQ9MJvHOWz4UMHttG1msmBl0CR?= =?us-ascii?Q?aVnRya/PyE68Z/8T0Jc91//65+tzY52JdX1SC8rZjfywGxExXruAAxZQbQGb?= =?us-ascii?Q?CSKLAp7xED90qzt6vsIa0XZnESqr1mTSS4W8BrYiZEYErZFstCqWwJGouuan?= =?us-ascii?Q?HhoWaQKjN215O65ENkgMAL72mMwY/Gvj+eEM5FXWok8UTztrO3M0BewTn4pG?= =?us-ascii?Q?iOH3wxmkhfJulKuYWjnokszjRYaMPhsUUaOtmlP1D7ucF6M8PfKoqJI9vnyv?= =?us-ascii?Q?0MOkRggIMDlPQ+NZ0dD8KM27UzoQY9xEYJIrVDEwt70JCkzlsINDpyU2jqvQ?= =?us-ascii?Q?K6Kx6E8IOONQzkQrgVozKfDYptj66Q3G89V45t8Ax7N2mhOAne1kShLEsugy?= =?us-ascii?Q?Lr9NHKVQM/mhbvk21d0FfnS2eGRH2Lc6nknvClXduS4N0MACpgW2z2h2miMN?= =?us-ascii?Q?81SIA2a+pkwAl4cevbv/ovyak55kp7Z0A126JKargiP6yn1NEU/kiSl8Mut0?= =?us-ascii?Q?FEqCdrhqZdNIAOABq3z6aSkQuWUjB9DNQRc+Dxn4C9A/vCfQ72957RN7NSPj?= =?us-ascii?Q?nG76Sw6DUYy+JRapxjdWJW4dpaQqDJYfGSi7vAYE0b0HhhnPy19CGO/l+313?= =?us-ascii?Q?m7jwmX12Z3d3ArkcyNXowzPq9Lat7NmwUKAkcpMlu3C+hAGS9yXXw3BywSp9?= =?us-ascii?Q?/6J266fCaC8h2PvmPxkfnXcvnbz+AOPZcZ5ZRWXDNI8NsAHF0i6afy2pw3gj?= =?us-ascii?Q?oSyd6TTo9BnfRQkE2KT9BhwPbQ1Yg5p8FIcnBoAfJ47P6TZRZjrFpcRUjVQs?= =?us-ascii?Q?1AEVwEiee/71k/SCAAh+977OFAZAntMuvjMLaaOmOEOjWvYuT+CrPWQFxEgX?= =?us-ascii?Q?gq5+1iQ+Nx2wPVVo+mtW8ZmH//xWm9dagp+qBPnWpetFsX10ZPHgi1xAOvxN?= =?us-ascii?Q?bHnI/uUfJMI=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?j5OReM2x3fMALRlxAWFxaWx+KLd9m/dtzAXEnRKJP6L42RHvUSrlayj9cL2h?= =?us-ascii?Q?jRMwDRZWGnIswoNQ8lBQzLioLmGPOgAMsbZShdn7zkalWRhqp6XxOMorVVFj?= =?us-ascii?Q?Bdrk1r8Z8Ic9lWGavJ4MUTDjCt+MU82Rn6Jb5A6w3U0CFiLLbLqn8DAo5QAn?= =?us-ascii?Q?Yqp15j/iNnMoEnCNEIRHrZDpu/loswwFSWTdbrVKvlt5lSGT22pWKesClOS4?= =?us-ascii?Q?WAdArJL9ATLK5sUv1fZ686Bgdi0BL1uXifJuvOXnsA7RTBeZxcct4TNTgbHV?= =?us-ascii?Q?GX4JXCsTPEgXx3o7djtKjW7tT5rARR+GymsuTtDwcNoGzCnWsCX/VfiHeaKX?= =?us-ascii?Q?1D7AV8p3UJ/oLQ8tEURF4vC2Jsdt2yL6UQRwmNlZ/I9pn9PYqD2NKII3nP+Q?= =?us-ascii?Q?00PJ55Oc7nUXWOKbXWn6aK/3gyeOAiMJN03soNg+p3EwNll6f/p2qTPkpYRf?= =?us-ascii?Q?GQEf5kxUd30idOlzKVvbPgyV3AzXd2Fm9eHVYamnUs/Pl8EUPKiIUmVo8ieA?= =?us-ascii?Q?Jm5QbXNssg55C1cv0tyAN9kpQZEgkuQxUqfAxialRGdm14VcTwQirPvdlZss?= =?us-ascii?Q?dAOGpMJ0Qvbg+lGxWdMmIQ1DdWowsO1F/etd0kSIrrPPSdoS+XrvOi5MAa2+?= =?us-ascii?Q?v18HEjMdiQOmT5lBnTRRzC6sLRB09lpywCoTkT+8aiB2/J+0Kyyw4SqAywBk?= =?us-ascii?Q?Ob7Hhc4XhPKqE96QqZcDCGJ/tD2XA+MbYiXby85bPfs5I7NfS/6YxA02Nxut?= =?us-ascii?Q?LwdMjySVKXrvpV4XqkPbUR92fPhfnfEfYtgbj+UfateOQlIG7fu6DaLKZyeX?= =?us-ascii?Q?q0eP+L5m2T0JKkzfZn26OuR9mouv4FSAFSduErVPrqqfw7niNBmBofQi+pg3?= =?us-ascii?Q?U5Xo5UTdjFnZpKxa6SkIzQkbFi8BGvmVzG1rMH4T0AFsQi4IaA5GndMcUjwJ?= =?us-ascii?Q?6rA3atIekuialrR/zqF0UYAw6FwIJgDyiW1YLQ9lx1pfzKt84H2uhB5Ax8nR?= =?us-ascii?Q?hhJRZQpgc5j8CZ/6w2SSZImxcyxlKR0tIol5abq84ux89xU4a09nz2M1fJ/w?= =?us-ascii?Q?3M299/BI4GuHzYknlQZan2UWmDiDnn/yhI1LOB0+j6xYBpk4CGfRQ9wiYESG?= =?us-ascii?Q?NDkUEgq080e8OXj+eP3VvFCAHBnOliL8+aCeUlCyvcOpI3MIZmsoXQYHnDD5?= =?us-ascii?Q?P5zxUDIc1KK+1R0/fIDzIk+b6NUQ601cDMlxJLeHy/oEx+LxxEBIsH/FqTtd?= =?us-ascii?Q?TgEwm9zTBQSkb/Zw1LBacgnBmBJV8suBODiDqnHYxaTQrKDBnDSgplH2Xnjy?= =?us-ascii?Q?amAoRVHiwexmEA5XiXSV29z8lxPizlcmLyqvFEEUcuF9aC/89cBuUXy16Yh/?= =?us-ascii?Q?OxvAQdFBOdUR7hG98yuCncrcST5ZCbkEOoSSd+AMjl0WwPsN3EA2fH4r0xoX?= =?us-ascii?Q?brrG7TdLHvsFL1TgWKLeuio0XYSrELv14w30BSF4sQ88mjsHguLaUQgI84Wb?= =?us-ascii?Q?5AWC2NhfrN2WDFsEkZLqRuSmMyeRERTx3284cvKYeUORh0pmoeqFIf6i0ATR?= =?us-ascii?Q?C/7Bk0FWQ/V9Gyc7qcsBSSlkJdnhdy6SBdfGJglO?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6d38e52c-d3f7-454e-0beb-08dd9f963239 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2025 16:22:39.4416 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: X0j0beXrkRLg4WeP+PkzI+k4pptfZHCaXhyEh8SDPlUTofXKTkbgRjw01c8Vkglu X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8187 Content-Type: text/plain; charset="utf-8" No functional change is intended. 1. Add __NR_PAGEBLOCK_BITS for the number of pageblock flag bits and use roundup_pow_of_two(__NR_PAGEBLOCK_BITS) as NR_PAGEBLOCK_BITS to take right amount of bits for pageblock flags. 2. Rename PB_migrate_skip to PB_compact_skip. 3. Add {get,set,clear}_pfnblock_bit() to operate one a standalone bit, like PB_compact_skip. 3. Make {get,set}_pfnblock_flags_mask() internal functions and use {get,set}_pfnblock_migratetype() for pageblock migratetype operations. 4. Move pageblock flags common code to get_pfnblock_bitmap_bitidx(). 3. Use MIGRATETYPE_MASK to get the migratetype of a pageblock from its flags. 4. Use PB_migrate_end in the definition of MIGRATETYPE_MASK instead of PB_migrate_bits. 5. Add a comment on is_migrate_cma_folio() to prevent one from changing it to use get_pageblock_migratetype() and causing issues. Signed-off-by: Zi Yan Acked-by: David Hildenbrand Reviewed-by: Vlastimil Babka --- Documentation/mm/physical_memory.rst | 2 +- include/linux/mmzone.h | 18 +-- include/linux/page-isolation.h | 2 +- include/linux/pageblock-flags.h | 34 +++--- mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 171 +++++++++++++++++++++------ 6 files changed, 162 insertions(+), 67 deletions(-) diff --git a/Documentation/mm/physical_memory.rst b/Documentation/mm/physic= al_memory.rst index d3ac106e6b14..9af11b5bd145 100644 --- a/Documentation/mm/physical_memory.rst +++ b/Documentation/mm/physical_memory.rst @@ -584,7 +584,7 @@ Compaction control =20 ``compact_blockskip_flush`` Set to true when compaction migration scanner and free scanner meet, whi= ch - means the ``PB_migrate_skip`` bits should be cleared. + means the ``PB_compact_skip`` bits should be cleared. =20 ``contiguous`` Set to true when the zone is contiguous (in other words, no hole). diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 87a667533d6d..392a03e37610 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -92,8 +92,12 @@ extern const char * const migratetype_names[MIGRATE_TYPE= S]; #ifdef CONFIG_CMA # define is_migrate_cma(migratetype) unlikely((migratetype) =3D=3D MIGRAT= E_CMA) # define is_migrate_cma_page(_page) (get_pageblock_migratetype(_page) =3D= =3D MIGRATE_CMA) -# define is_migrate_cma_folio(folio, pfn) (MIGRATE_CMA =3D=3D \ - get_pfnblock_flags_mask(&folio->page, pfn, MIGRATETYPE_MASK)) +/* + * __dump_folio() in mm/debug.c passes a folio pointer to on-stack struct = folio, + * so folio_pfn() cannot be used and pfn is needed. + */ +# define is_migrate_cma_folio(folio, pfn) \ + (get_pfnblock_migratetype(&folio->page, pfn) =3D=3D MIGRATE_CMA) #else # define is_migrate_cma(migratetype) false # define is_migrate_cma_page(_page) false @@ -122,14 +126,12 @@ static inline bool migratetype_is_mergeable(int mt) =20 extern int page_group_by_mobility_disabled; =20 -#define MIGRATETYPE_MASK ((1UL << PB_migratetype_bits) - 1) +#define get_pageblock_migratetype(page) \ + get_pfnblock_migratetype(page, page_to_pfn(page)) =20 -#define get_pageblock_migratetype(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), MIGRATETYPE_MASK) +#define folio_migratetype(folio) \ + get_pageblock_migratetype(&folio->page) =20 -#define folio_migratetype(folio) \ - get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ - MIGRATETYPE_MASK) struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 898bb788243b..277d8d92980c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -25,7 +25,7 @@ static inline bool is_migrate_isolate(int migratetype) #define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 =20 -void set_pageblock_migratetype(struct page *page, int migratetype); +void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 bool move_freepages_block_isolate(struct zone *zone, struct page *page, int migratetype); diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flag= s.h index e73a4292ef02..451b351c689e 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -19,15 +19,19 @@ enum pageblock_bits { PB_migrate, PB_migrate_end =3D PB_migrate + PB_migratetype_bits - 1, /* 3 bits required for migrate types */ - PB_migrate_skip,/* If set the block is skipped by compaction */ + PB_compact_skip,/* If set the block is skipped by compaction */ =20 /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. */ - NR_PAGEBLOCK_BITS + __NR_PAGEBLOCK_BITS }; =20 +#define NR_PAGEBLOCK_BITS (roundup_pow_of_two(__NR_PAGEBLOCK_BITS)) + +#define MIGRATETYPE_MASK ((1UL << (PB_migrate_end + 1)) - 1) + #if defined(CONFIG_HUGETLB_PAGE) =20 #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -65,27 +69,23 @@ extern unsigned int pageblock_order; /* Forward declaration */ struct page; =20 -unsigned long get_pfnblock_flags_mask(const struct page *page, - unsigned long pfn, - unsigned long mask); - -void set_pfnblock_flags_mask(struct page *page, - unsigned long flags, - unsigned long pfn, - unsigned long mask); +enum migratetype get_pfnblock_migratetype(const struct page *page, + unsigned long pfn); +bool get_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); +void set_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); +void clear_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); =20 /* Declarations for getting and setting flags. See mm/page_alloc.c */ #ifdef CONFIG_COMPACTION #define get_pageblock_skip(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), \ - (1 << (PB_migrate_skip))) + get_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #define clear_pageblock_skip(page) \ - set_pfnblock_flags_mask(page, 0, page_to_pfn(page), \ - (1 << PB_migrate_skip)) + clear_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #define set_pageblock_skip(page) \ - set_pfnblock_flags_mask(page, (1 << PB_migrate_skip), \ - page_to_pfn(page), \ - (1 << PB_migrate_skip)) + set_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #else static inline bool get_pageblock_skip(struct page *page) { diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index b1caedbade5b..4ce5210ea56e 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -797,7 +797,7 @@ void move_pfn_range_to_zone(struct zone *zone, unsigned= long start_pfn, =20 /* * TODO now we have a visible range of pages which are not associated - * with their zone properly. Not nice but set_pfnblock_flags_mask + * with their zone properly. Not nice but set_pfnblock_migratetype() * expects the zone spans the pfn range. All the pages in the range * are reserved so nobody should be touching them so we should be safe */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cc9577a27ec4..74cb7696e527 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -353,81 +353,174 @@ static inline int pfn_to_bitidx(const struct page *p= age, unsigned long pfn) return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS; } =20 +static __always_inline bool is_standalone_pb_bit(enum pageblock_bits pb_bi= t) +{ + return pb_bit > PB_migrate_end && pb_bit < __NR_PAGEBLOCK_BITS; +} + +static __always_inline void +get_pfnblock_bitmap_bitidx(const struct page *page, unsigned long pfn, + unsigned long **bitmap_word, unsigned long *bitidx) +{ + unsigned long *bitmap; + unsigned long word_bitidx; + + BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); + BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); + VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); + + bitmap =3D get_pageblock_bitmap(page, pfn); + *bitidx =3D pfn_to_bitidx(page, pfn); + word_bitidx =3D *bitidx / BITS_PER_LONG; + *bitidx &=3D (BITS_PER_LONG - 1); + *bitmap_word =3D &bitmap[word_bitidx]; +} + + /** - * get_pfnblock_flags_mask - Return the requested group of flags for the p= ageblock_nr_pages block of pages + * __get_pfnblock_flags_mask - Return the requested group of flags for + * a pageblock_nr_pages block of pages * @page: The page within the block of interest * @pfn: The target page frame number * @mask: mask of bits that the caller is interested in * * Return: pageblock_bits flags */ -unsigned long get_pfnblock_flags_mask(const struct page *page, - unsigned long pfn, unsigned long mask) +static unsigned long __get_pfnblock_flags_mask(const struct page *page, + unsigned long pfn, + unsigned long mask) { - unsigned long *bitmap; - unsigned long bitidx, word_bitidx; + unsigned long *bitmap_word; + unsigned long bitidx; unsigned long word; =20 - bitmap =3D get_pageblock_bitmap(page, pfn); - bitidx =3D pfn_to_bitidx(page, pfn); - word_bitidx =3D bitidx / BITS_PER_LONG; - bitidx &=3D (BITS_PER_LONG-1); + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); /* - * This races, without locks, with set_pfnblock_flags_mask(). Ensure + * This races, without locks, with set_pfnblock_migratetype(). Ensure * a consistent read of the memory array, so that results, even though * racy, are not corrupted. */ - word =3D READ_ONCE(bitmap[word_bitidx]); + word =3D READ_ONCE(*bitmap_word); return (word >> bitidx) & mask; } =20 -static __always_inline int get_pfnblock_migratetype(const struct page *pag= e, - unsigned long pfn) +/** + * get_pfnblock_bit - Check if a standalone bit of a pageblock is set + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to check + * + * Return: true if the bit is set, otherwise false + */ +bool get_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) { - return get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return false; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + return test_bit(bitidx + pb_bit, bitmap_word); } =20 /** - * set_pfnblock_flags_mask - Set the requested group of flags for a pagebl= ock_nr_pages block of pages + * get_pfnblock_migratetype - Return the migratetype of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * + * Return: The migratetype of the pageblock + * + * Use get_pfnblock_migratetype() if caller already has both @page and @pfn + * to save a call to page_to_pfn(). + */ +__always_inline enum migratetype +get_pfnblock_migratetype(const struct page *page, unsigned long pfn) +{ + return __get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); +} + +/** + * __set_pfnblock_flags_mask - Set the requested group of flags for + * a pageblock_nr_pages block of pages * @page: The page within the block of interest - * @flags: The flags to set * @pfn: The target page frame number + * @flags: The flags to set * @mask: mask of bits that the caller is interested in */ -void set_pfnblock_flags_mask(struct page *page, unsigned long flags, - unsigned long pfn, - unsigned long mask) +static void __set_pfnblock_flags_mask(struct page *page, unsigned long pfn, + unsigned long flags, unsigned long mask) { - unsigned long *bitmap; - unsigned long bitidx, word_bitidx; + unsigned long *bitmap_word; + unsigned long bitidx; unsigned long word; =20 - BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); - BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); - - bitmap =3D get_pageblock_bitmap(page, pfn); - bitidx =3D pfn_to_bitidx(page, pfn); - word_bitidx =3D bitidx / BITS_PER_LONG; - bitidx &=3D (BITS_PER_LONG-1); - - VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); =20 mask <<=3D bitidx; flags <<=3D bitidx; =20 - word =3D READ_ONCE(bitmap[word_bitidx]); + word =3D READ_ONCE(*bitmap_word); do { - } while (!try_cmpxchg(&bitmap[word_bitidx], &word, (word & ~mask) | flags= )); + } while (!try_cmpxchg(bitmap_word, &word, (word & ~mask) | flags)); +} + +/** + * set_pfnblock_bit - Set a standalone bit of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to set + */ +void set_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) +{ + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + set_bit(bitidx + pb_bit, bitmap_word); } =20 -void set_pageblock_migratetype(struct page *page, int migratetype) +/** + * clear_pfnblock_bit - Clear a standalone bit of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to clear + */ +void clear_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) +{ + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + clear_bit(bitidx + pb_bit, bitmap_word); +} + +/** + * set_pageblock_migratetype - Set the migratetype of a pageblock + * @page: The page within the block of interest + * @migratetype: migratetype to set + */ +__always_inline void set_pageblock_migratetype(struct page *page, + enum migratetype migratetype) { if (unlikely(page_group_by_mobility_disabled && migratetype < MIGRATE_PCPTYPES)) migratetype =3D MIGRATE_UNMOVABLE; =20 - set_pfnblock_flags_mask(page, (unsigned long)migratetype, - page_to_pfn(page), MIGRATETYPE_MASK); + __set_pfnblock_flags_mask(page, page_to_pfn(page), + (unsigned long)migratetype, MIGRATETYPE_MASK); } =20 #ifdef CONFIG_DEBUG_VM @@ -667,7 +760,7 @@ static inline void __add_to_free_list(struct page *page= , struct zone *zone, int nr_pages =3D 1 << order; =20 VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D migratetype, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), migratetype, nr_pages); =20 if (tail) @@ -693,7 +786,7 @@ static inline void move_to_free_list(struct page *page,= struct zone *zone, =20 /* Free page moving can fail, so it happens before the type update */ VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D old_mt, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), old_mt, nr_pages); =20 list_move_tail(&page->buddy_list, &area->free_list[new_mt]); @@ -715,7 +808,7 @@ static inline void __del_page_from_free_list(struct pag= e *page, struct zone *zon int nr_pages =3D 1 << order; =20 VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D migratetype, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), migratetype, nr_pages); =20 /* clear reported state and update reported page count */ @@ -3127,7 +3220,7 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, =20 /* * Do not instrument rmqueue() with KMSAN. This function may call - * __msan_poison_alloca() through a call to set_pfnblock_flags_mask(). + * __msan_poison_alloca() through a call to set_pfnblock_migratetype(). * If __msan_poison_alloca() attempts to allocate pages for the stack depo= t, it * may call rmqueue() again, which will result in a deadlock. */ --=20 2.47.2 From nobody Tue Dec 16 22:30:09 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2067.outbound.protection.outlook.com [40.107.94.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FA041A5B85 for ; Fri, 30 May 2025 16:22:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622173; cv=fail; b=TqgOKx3rMDuoqSIOyfQPwqc6DhJW1bdFe72YmmxsP0ulNscNTExQrLeigw9wzkwz8+IMoAstK+am9SM6vFynrSkJQGaIhEyb8SOsP8IZ5aRxdGxuifuaZyUst3Ju+ihWzyXiTKLi8oC6+0Zxbmtv1/m05+xW8lQmKWtMpP5ukUo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622173; c=relaxed/simple; bh=uZcrBxC33RZOU3DBKeTbakPVeujulD65cdi6z4n8XVc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=upsu5c4eIx8+orWAFE/faUoWcvmBhmpVXBULzDZ7/uxeN6odvxtLmQBSD48sCkmCuDTVcf1hK5vG3uvdibZubOdEhazIhtUvRKJMa2bAXko4wY6aXDGpFTcS7IK8nCAs7ibvPhZ25A+ZYa+QJVfnYoCcxF9xDkS3AqDZfqdt8TE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=fEDcZAVS; arc=fail smtp.client-ip=40.107.94.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="fEDcZAVS" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=totBCeueVE5UrwBG+NjYw/V7CRDJmJyjdNXD6sCAxeodubEmfM8NXQkWoeG6L6i1XInj0YH4ny2xvdsEPWuEIBga7NSJMD0IpumASQeW53Hl2mbCk8c20Idkfaztd8ms+aMfohBOaINWXFUIaQrYFRYOXoFHBdij/sBm8j6oQlvV+ue3SQZ94HTTTUe/vbvK/HRdqGIMeBIrxQl1yq1/JtRndogzYXikylsLUKSsVBAeh7bOfMXyGbEEPbb0t0WEJ9LFKgSYHdb8SJcDJ7nuH63nI8w1P/0gIsGRjIVN9rI6dk8OUP0fb8jCBdNJGC5+YQm9KyXMD+Wa3ciQjU/+Yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0hiK3kLot4o7xS1s5Cj3Ucc8b+4P6Dh6DmiQNaN80VM=; b=SNUlPWVzl4zBZWpOm1YpRkCvjOaZs4QXxjskO/mvee6X41q2yRGbCCn+hcQmAX/xpJfGDmrWYkDBWVPHGDQIfgwE0Co3l0GQm+7nbraJxIJQdpeXl9w9PZL9ZbvJnvmiecrepJwb+GWTQU3EDrOyoLJWFMhLkDGQrMbO1WI3LhHtrWhRHUiN3eKqyOWGz/ed4G1lafkRDTuA7IxpznIwo7NzUxkSCmT/N+Cp3zihKIMi+jXdBDqxZ/dRg6B323DHfpoKpT+nBjQsWncSlN8vSXJ4dbRV1AXTMedwkVYi2BeTE0bisjKcaRWZg+NUZgNGSUVf79hVIF5ECWNYJM2v8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0hiK3kLot4o7xS1s5Cj3Ucc8b+4P6Dh6DmiQNaN80VM=; b=fEDcZAVS+A9zoXcGu3eGSfUh8NHJwPncH+DVmA8DvLkL9qrTpN3kzB6HKUNpLa2sSvZGBKWX2WgJE9SE/sZW/VTDfXvR3gw6mqTIDX9Pq29CFbS3CYCmf/KeVKk1LQmWTa2CipuMAZUNSjVe1JXLs5H5H5Wq32+hC8ebxL3laxyFa2Selq47LFH5qh84ymjbolNapLjFBcgrLxl2UCKrq4ZSmwLJxplS7dmvZ6nYqhHb8dA1MzDJSkY+80Uhgjp2dpSdLsKaF2tUr90jy+IVVFnCw58rP1jtXziIkU3PQQ2WFFzBwSMdVkzXlsynyp0mBJLNa8ed0gd0MWKRFWT5GQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by DM3PR12MB9352.namprd12.prod.outlook.com (2603:10b6:0:4a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.27; Fri, 30 May 2025 16:22:49 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%4]) with mapi id 15.20.8769.025; Fri, 30 May 2025 16:22:49 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v6 2/6] mm/page_isolation: make page isolation a standalone bit. Date: Fri, 30 May 2025 12:22:23 -0400 Message-ID: <20250530162227.715551-3-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250530162227.715551-1-ziy@nvidia.com> References: <20250530162227.715551-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR15CA0021.namprd15.prod.outlook.com (2603:10b6:208:1b4::34) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|DM3PR12MB9352:EE_ X-MS-Office365-Filtering-Correlation-Id: 835ec09a-6d87-487a-50d4-08dd9f963844 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Hvylag61y7I4lrJ2XyXSgkjQK8+T5mUQq7kdWGKImhvGLsXzEjDu7j8sGBSw?= =?us-ascii?Q?QZaIEkmz7dDej41YLVBHyD4W7TiQsHzk/M/Q/ap5wi9pLQ7ZNemFOPvVvlqS?= =?us-ascii?Q?IQvHLECwGKxrzXwDoClaFb/wgTlLA+JvDaAEDwGpjp1DyglDpJ1EMs15KwTk?= =?us-ascii?Q?DsDJie+edqoLESwBRcjZUDGfDphEx4oBibUa7NNmNYsKxGHIokfwlszyIVx9?= =?us-ascii?Q?7AshqRgwzzw1SGvQv+Vrii3HUq/Oe/F+lRBkCG/N5R9AKZZoeOqH1V1k1uNF?= =?us-ascii?Q?Q1XzNF0SQtvXUMHYRroBzsudyD5jV9jT8JtAlTqlZHyKC+m7TXRhDkJFsQ6S?= =?us-ascii?Q?9m0Pf4lNBo7+qs9pGhMrea4BpmO9yVI/ThOuO9JTIDPIn937cfSv7VQTq/ib?= =?us-ascii?Q?NfTKs/mCoSO5+darkCmQH4mWdprqEtgDF2Psfy60EaaDu1sLWq19Pq4Vd88M?= =?us-ascii?Q?KFFb1IfGXAIZfYu9qUoPV8gL1xcU/h15+ymLx1f7sAkuq/kgGWBNiV0+cLbY?= =?us-ascii?Q?MRDnDKK56R4xUHjffjx29AFEoAml67z0UTRuimIXc/gNabqDlr5bkYmshodI?= =?us-ascii?Q?2XRBEvQQ3N2AafuLpQ/5gyXdbiI5LS8F1iBwBys4LQZH34iAFAEQiOwsW02u?= =?us-ascii?Q?TsJd6aXn5j/1yH+dGYPZBe5a5VQ/uR+qX9iigyiMQNS0NIa7czHdMLmfQxWY?= =?us-ascii?Q?0BgCXl7eN2KFCb/AHsFgc/7WvFV4PlKBQamgnepOxTNO2VbRnyEpjpvm5Hfd?= =?us-ascii?Q?m4b+YlQM4CUW/e0/P3kZXz/T6qtlJUQGMbQkl7veaT1HpLY3ZZDDeX2zxs8r?= =?us-ascii?Q?op2kLCODXzhvrV9bEkqedrQmayQtt8qT+KWZiq+Pe5hG8h6j2GHI+0GGDXRV?= =?us-ascii?Q?o5mTqY8Bq5FTHaOsRmtT3BzaTfohgjroRZnZ5hARzn6Ql4Izqm2/ErJvGqKK?= =?us-ascii?Q?Hg5aCqab1K0UgTuMhRjvwzfAPr4CTjQnfvV6y5x7Et9f3tcGpDGvN8GIMT5v?= =?us-ascii?Q?LSToRYNgoOy/mbdL0kJ/AQ/h2mPi0BnzjQmWZi9mYH1UJ0+ljcPZ2PFwqXpa?= =?us-ascii?Q?lyTxXT7fvGnxwT022mQagTglEk5UrwcD72kzVMtOCK6UBKknLSgqbyclYDyL?= =?us-ascii?Q?vcZGLNKuISDbkEcO8jNWvjWTnLrUuEOJmH3ZsjlAeY1csdDxx2GVS9jzT4Q6?= =?us-ascii?Q?rQART2VO4hdaRgSDAXEdawhgl1Fg8po0qPzCA0TWLQ59m8lTuojzvccvb/ti?= =?us-ascii?Q?HN5E/IiFqc2LWu/7BSlucIzKjoBMazzcRsOOoUB1yHErt+sy8MRs046Cbf2m?= =?us-ascii?Q?bNcXWNtQmV4g6Kr1naBzvBbcjsCrWPIiMQvTAG1lONcNrw+JG4jkHsCxZ+jW?= =?us-ascii?Q?XBKrRVbhgb8DJbQWQHpwnOLjS8fhlEYpQ5aVtmT6rI9/efBkTA=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sOwNr2Av/G+I4GW1QsK0KNfA9HZespTFH2eTw/j92wuU1pUFnhINS2vSKJNR?= =?us-ascii?Q?pEdTUvRJ+/w0z3/9W18a4PJjsw0ntkdeKiYU56zZlJ7tHb6HF/lupfEzh4J3?= =?us-ascii?Q?vnerrTH43tVBFr3hLHhhuhri4ySLZ23y2iQTnf/bayJCOOlkgLDtyIO/lSu2?= =?us-ascii?Q?rsY1DvkuVFZq1/31wHWT3v2uPfEV/rSeq1UQYGTRuVY8dwg8WKctFxmmHPJi?= =?us-ascii?Q?ZsaFRhGUTRg8y2webEnGPLJpRvBZbm2QP5nTxoWiagc5ZdCC2b3lhyAOOiiw?= =?us-ascii?Q?cB4/kr8NpMin/JrEXWWI3QK21huAoz8/aTJzPLjLDR4cH0YRCPOtoB3KKsI8?= =?us-ascii?Q?FYxsubzrUPAKuO0gfkYLFYohxW4aTfrhCra7hZN4S44FwkQUvZEwDwqz2i8x?= =?us-ascii?Q?l5kQOtt1DFoPnfeT+IVLUMEab9iup9CJC2QJw6kaenrK5rbDLLh1tury50Mq?= =?us-ascii?Q?gcrACSRMRWNBpz8Ol/eRps+VOgfE9iibGuq8t1q8wecut+nH9Oh5RjBwGl1L?= =?us-ascii?Q?O95aCdSuffskzRp7QQDCUZrqYJ+F8qauX4GFhDBpKFTSC0d73IOM03hiNmBF?= =?us-ascii?Q?pbKmAAe0xJ9QL77A6pl/c+mcVPk6N+gIYufuiudwRahdGvZ8Bf9N+sub1Gfc?= =?us-ascii?Q?ho82fWcbA024KupujaM9aMeiDr+yj7JV0uv0h6jlx/oTyk1KQ9viJo2iGU3O?= =?us-ascii?Q?wC0DSs1kqaBvQ5AnAk/F1CZP2051GgcbVxLiSXfFXIF5/+7SCib0Yfbk7LA4?= =?us-ascii?Q?ZtiEHJ7aNUQ0jOwfeg3pbv2x7Fo8zlXYkP1MFp7FjJN9Kv3Pg+edocsYPkfe?= =?us-ascii?Q?rvdkP6SaI/AYrxYr0yWnowzRNwvAeOoTR3y/vjSViYN6Rhxi8RnBHzuk1wvX?= =?us-ascii?Q?4IQ5rOLMr4PQ39oeSqqteEweAUmKu78s6cXaCkuBoFKSTME+n6995Gc/KYF8?= =?us-ascii?Q?zV+MUsJGsGXZIhtm9fUMqM8YdnpluyxqagbrbSH0X6Mb2kQChjeE/aiNuyTx?= =?us-ascii?Q?xSqSUa0asF7BcWJwjkPAbDzS6JBL2E4xerw/5m8qE9KI1do+bETgDDEzWn1w?= =?us-ascii?Q?mKclE7A2sL1jy7AsIWR152Bcw5Ml9NRTB7TvTa+pYHbxj1fObT/aPLAw1KTl?= =?us-ascii?Q?iNDKcimQxu1hMO0KRSvMsaL+HOTmyByaHIlbyVdjP4WWxx1Gy8sun8DDGFCG?= =?us-ascii?Q?Ss9lZkbPo7dQ4dmncXEH6tmlJ4Mdfi7oxLrfZ1HUI9RZ1JEXiCQ2Kd649Ks/?= =?us-ascii?Q?Yv0wxvlSvkJsjzfi4Ffx+f7YbBssocqtdWzEg6sghms1pW6imn8HRUlyLIXw?= =?us-ascii?Q?aMWeNPI0gps08SYyJuaDi9ylFmU8RmKmY/yUyWTPCgdfeB7EFTn+s5Llb3tu?= =?us-ascii?Q?JX8grPpreBNTky7BfEZiu4/jMYmcoLH6hDHPE2cBtgrpA9Wf1nwbQ3bQfIj4?= =?us-ascii?Q?PWQPygTaWN/SNPkC5yKPipnrR3LgptzmUIMpAwdtgjhg0BfHjh1wlot/2YG4?= =?us-ascii?Q?nUCqGHW4Np9i1JCPEKLmiFaC4z532eRAGqugcsz6cnEK0aU9hvvhRUOhFz2z?= =?us-ascii?Q?mVsoJO0P1wSAFRIpj+OErQg9HmKRAxHBlKmad+Tf?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 835ec09a-6d87-487a-50d4-08dd9f963844 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2025 16:22:49.5639 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: e3qLjEezvwlWmPhmxu1c8GX5CazhWVUY7rU4d/iRnhSCAgltXcahur8rH4PH1YRf X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9352 Content-Type: text/plain; charset="utf-8" During page isolation, the original migratetype is overwritten, since MIGRATE_* are enums and stored in pageblock bitmaps. Change MIGRATE_ISOLATE to be stored a standalone bit, PB_migrate_isolate, like PB_compact_skip, so that migratetype is not lost during pageblock isolation. Signed-off-by: Zi Yan Acked-by: David Hildenbrand Reviewed-by: Vlastimil Babka --- include/linux/mmzone.h | 3 +++ include/linux/page-isolation.h | 16 ++++++++++++++++ include/linux/pageblock-flags.h | 14 ++++++++++++++ mm/page_alloc.c | 27 ++++++++++++++++++++++++--- 4 files changed, 57 insertions(+), 3 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 392a03e37610..0a5cdc52b405 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -79,6 +79,9 @@ enum migratetype { * __free_pageblock_cma() function. */ MIGRATE_CMA, + __MIGRATE_TYPE_END =3D MIGRATE_CMA, +#else + __MIGRATE_TYPE_END =3D MIGRATE_HIGHATOMIC, #endif #ifdef CONFIG_MEMORY_ISOLATION MIGRATE_ISOLATE, /* can't allocate from here */ diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 277d8d92980c..fc021d3f95ca 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -11,6 +11,12 @@ static inline bool is_migrate_isolate(int migratetype) { return migratetype =3D=3D MIGRATE_ISOLATE; } +#define get_pageblock_isolate(page) \ + get_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) +#define clear_pageblock_isolate(page) \ + clear_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) +#define set_pageblock_isolate(page) \ + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) #else static inline bool is_migrate_isolate_page(struct page *page) { @@ -20,6 +26,16 @@ static inline bool is_migrate_isolate(int migratetype) { return false; } +static inline bool get_pageblock_isolate(struct page *page) +{ + return false; +} +static inline void clear_pageblock_isolate(struct page *page) +{ +} +static inline void set_pageblock_isolate(struct page *page) +{ +} #endif =20 #define MEMORY_OFFLINE 0x1 diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flag= s.h index 451b351c689e..1cf5f0fbd627 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -21,6 +21,13 @@ enum pageblock_bits { /* 3 bits required for migrate types */ PB_compact_skip,/* If set the block is skipped by compaction */ =20 +#ifdef CONFIG_MEMORY_ISOLATION + /* + * Pageblock isolation is represented with a separate bit, so that + * the migratetype of a block is not overwritten by isolation. + */ + PB_migrate_isolate, /* If set the block is isolated */ +#endif /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. @@ -32,6 +39,13 @@ enum pageblock_bits { =20 #define MIGRATETYPE_MASK ((1UL << (PB_migrate_end + 1)) - 1) =20 +#ifdef CONFIG_MEMORY_ISOLATION +#define MIGRATETYPE_AND_ISO_MASK \ + (((1UL << (PB_migrate_end + 1)) - 1) | BIT(PB_migrate_isolate)) +#else +#define MIGRATETYPE_AND_ISO_MASK MIGRATETYPE_MASK +#endif + #if defined(CONFIG_HUGETLB_PAGE) =20 #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 74cb7696e527..5de23eba0db8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -365,8 +365,12 @@ get_pfnblock_bitmap_bitidx(const struct page *page, un= signed long pfn, unsigned long *bitmap; unsigned long word_bitidx; =20 +#ifdef CONFIG_MEMORY_ISOLATION + BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 8); +#else BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); - BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); +#endif + BUILD_BUG_ON(__MIGRATE_TYPE_END >=3D (1 << PB_migratetype_bits)); VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); =20 bitmap =3D get_pageblock_bitmap(page, pfn); @@ -439,7 +443,16 @@ bool get_pfnblock_bit(const struct page *page, unsigne= d long pfn, __always_inline enum migratetype get_pfnblock_migratetype(const struct page *page, unsigned long pfn) { - return __get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); + unsigned long mask =3D MIGRATETYPE_AND_ISO_MASK; + unsigned long flags; + + flags =3D __get_pfnblock_flags_mask(page, pfn, mask); + +#ifdef CONFIG_MEMORY_ISOLATION + if (flags & BIT(PB_migrate_isolate)) + return MIGRATE_ISOLATE; +#endif + return flags & MIGRATETYPE_MASK; } =20 /** @@ -519,8 +532,16 @@ __always_inline void set_pageblock_migratetype(struct = page *page, migratetype < MIGRATE_PCPTYPES)) migratetype =3D MIGRATE_UNMOVABLE; =20 +#ifdef CONFIG_MEMORY_ISOLATION + if (migratetype =3D=3D MIGRATE_ISOLATE) { + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + return; + } + /* MIGRATETYPE_AND_ISO_MASK clears PB_migrate_isolate if it is set */ +#endif __set_pfnblock_flags_mask(page, page_to_pfn(page), - (unsigned long)migratetype, MIGRATETYPE_MASK); + (unsigned long)migratetype, + MIGRATETYPE_AND_ISO_MASK); } =20 #ifdef CONFIG_DEBUG_VM --=20 2.47.2 From nobody Tue Dec 16 22:30:09 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2067.outbound.protection.outlook.com [40.107.94.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5D7C23537B for ; Fri, 30 May 2025 16:22:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622176; cv=fail; b=ZEqLkmVM6ZnIsKPwzeSXZ9fuMu3FpcQQG4S0YZX1MOz3Y/SHZ5CZv4jiNI69EhwD1aMb0i+pYKncDy4KAcUmcc8cZ9/xsi2Cg5L8Su6CdyirTzCHs2f8w5WbYyWttQBap8122oZiSXjgJZgh0Uv4z1VbzFkTIe3tBeIlUQt97ZM= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622176; c=relaxed/simple; bh=B8HBxEoKXnWhpqSBNXkNRCip/JGvhgw7uFeSYz2Uegk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=gfZsr9LT+fIoMpCkFMzFTAm4ydrWeTUFhTKfYxTo4BWFW6OnYFRAgUtg2VAoI7pGBAGSf+tfI0UlsMib12WHtKWjDVtAGf+Gi790smzqsLBksp5dsmImPWw6sbH2UJdhSHbIbWmGyrqfJQQdjSKJJlPgmbT7TO2bafn4C6YumoI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=n9tu83gg; arc=fail smtp.client-ip=40.107.94.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="n9tu83gg" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GmPHvuAqPM8CKmwM0wsko1vn3dUbDwZNCc4Ssk5nsAovri3m+R2Pwn/tU7A4/84bXJaVdW9tIzQWo2QimeRMc3I1BVOvnd59j9aF36QZJpUtMfKZCLZMLyNDV1RvDBm6UtIgIWR7ieB8A1/925DZapcyzJNFFISzPxv6GMgVIbNu1UFbfn8AQeD+Esaz6uOxLY/fBAivG4i8yt+V8YRDJ3ociEr13GMUaioqYfs0djZlwo3T2OBlTpOktsB5aKZRQUJhja+SeRu2DMZHW21TD4mrZZSBqsSx5YVRdJLbr9FnWHNjclbnYQ/VbNmL5DSVJxXMcLGFwbiCNT42aIQ5uQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ixF44OA4L7wU4LPtCvDuK+emluTWtTBTmb32LZwOjT4=; b=yXxWMdSuDeCqjjx958dL5INaokm9Qfpr+i6Ho/uURJ+tdN18ReDinovHI9VEoeu8pgSw6k3M1oPEtUCyMeD2Jv3jlVpkwZRT3cuWVu0P2DVzCYuhBH5wNGRCGNcHHos0eoBwqQWWCSiG7nbo9OtNdTCLig+3Val+Yb4ve7KYq11m+tXVo3GCWCQxwmDn6R/lYV9nnFG5E4rzdp0CCwqM/6u32U8BB7RValQzb29pmmjeOR7M3RMDolrKUPT5pxzbpFXiKqDaX7mcMQnPyxf0mR9HVdObF44Ef59gzR864PSJR+i94CmB9BMpZJSEfnZkTF75X2nWAx0DptUauUEPtA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ixF44OA4L7wU4LPtCvDuK+emluTWtTBTmb32LZwOjT4=; b=n9tu83ggfN9Z5MezWtcve2HW/zPErgR/pK8N6P1bCDKxhNkgii2QICDZ77QvLysvZnVQShjdLmbNCvLUxgMMTdYnIfM8Qr79vk52xX1it6JUJadWnQDkNM7aw+QAoXHZDOOydkUK++DDHYwosmx8wGYzNKYm4fX0Ln/fhA4YpAR7rGJzq+N9c2XB8QPVGymRyrzO+ZJ6un1f5tt7C2rc7D2LuERkiwmZaz60tmchM2mpFkOvlevA01VHdRSNGwQfFhlKeYGDcMao0pwYkORlz/OlDbHLsJluHE40mvmZS9gHTI3pioYsD2kegBr37OQN7LNeO5m3DzOJb90dSHYWWQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by DM3PR12MB9352.namprd12.prod.outlook.com (2603:10b6:0:4a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.27; Fri, 30 May 2025 16:22:50 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%4]) with mapi id 15.20.8769.025; Fri, 30 May 2025 16:22:50 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v6 3/6] mm/page_alloc: add support for initializing pageblock as isolated. Date: Fri, 30 May 2025 12:22:24 -0400 Message-ID: <20250530162227.715551-4-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250530162227.715551-1-ziy@nvidia.com> References: <20250530162227.715551-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN9PR03CA0706.namprd03.prod.outlook.com (2603:10b6:408:ef::21) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|DM3PR12MB9352:EE_ X-MS-Office365-Filtering-Correlation-Id: ad25eab5-8be5-4b8e-73b3-08dd9f96390b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?0T+g6stCLhU5h4slPhgdswiKGzhhYlHzF+VVMpjmMsi+ZhoQ1WlGC3gWXZt6?= =?us-ascii?Q?WHRd2Dy4ju+Su7vzuzsJxXgvdHtEK6Py3AxZ09PRrr5HHgMXchf1bt4McwyE?= =?us-ascii?Q?c6JCHZzGVit5CpKmKYLeczYL4IzmBMWKZMP4gnq7kDMXrcr8/uBJrIMJscKi?= =?us-ascii?Q?WS6xC2wJDLB8sOd0VQBl+LELOxlz13lQm9wPnKA+qOPfQDHEJC3FoynRk329?= =?us-ascii?Q?zyNn65QdBv2/HCI+Y1eDngVLiKIGUvEmk+FvbSWA0VVQcEsEU8HX28cHDkxm?= =?us-ascii?Q?ZgnyDBSlRdP3pHCVgSMTv+/C86UQge/0omW/2spORu7k7hnCmxBTU+g60+sT?= =?us-ascii?Q?lvr04sBgu+3M7yTpSPNQ6SNaE4hF2hFZ/2sjRDPjonk69ZQqty6p2s3uI10k?= =?us-ascii?Q?bhwx9+fwpaNCrT21qnZ8JffZNIAMQ/ChBWqAjUUcEffC6RYZOsVdHn4iFnLb?= =?us-ascii?Q?NQ/1os252DFwwAdPwi85ukC+9ScoZetvAZXmFvsj0vK7bL6+dEp2E5hgDCW/?= =?us-ascii?Q?7vbMPTDEJY+wNW1B2lQl6wAyxglVxW9a7vjSNECFFL0J+VAPAqfNhOwcBax0?= =?us-ascii?Q?7BkoSdWjVfYCKPJ7l0sIUh2p1mg1+AgTDzUho9fFonfq2GDThK9SKERLJpfA?= =?us-ascii?Q?WFEoe50GdooBkZaxW/QWflm9YuMs5ds2w61n3bhbecLanYNxORPul/dggN7O?= =?us-ascii?Q?uA/9nWvQ+sX4L/rSUi5TDNRYzZFS+mXt5sV4msMRxoow9IZUYXGzFHnJx7xV?= =?us-ascii?Q?sXVI7ZIfvni8ichtr84/AmVwfEXDBpNpgHZbhCiRxxT/EOJ+rf1YUjo6gnvH?= =?us-ascii?Q?Kjp0jGpSuLK3AoWUK/mJUH6sRJEON7YMVmNYDCFM7PeAEmQmVWiFu4+HSOYk?= =?us-ascii?Q?iRGGPTmx+ZR5Ii6st5TXNSAtPc/9KNqkeglIOsexLgqblMJqFatUAuU8/m5R?= =?us-ascii?Q?IpkMuSYnRkEf4F6EJRE9OO0pwfy1JWEzyDoYUVZZEzcgdQz+PX2A0Zj3YrQY?= =?us-ascii?Q?ZktPWQHRnAzKbfUkxilT71fSQPFzMUEw/7Vqv2JdDd54NhAAISETPqQFJ5Un?= =?us-ascii?Q?e6t3WV5NGQHCY3thFb44dHnG3iBEyb8KUE7Kpp2/Fyl063uxLk16RdOZdYhP?= =?us-ascii?Q?cwX1g+Jt3afjn0kJJEyL18o22MxWKp5e1WqGKJkvpYOUGlVaaRw4P0C+DVm5?= =?us-ascii?Q?eFxxItldBxR0qz7uNDG6fImEFzH4QyCE69L/BhAhkcpFwTvfCefuJCmqbcO3?= =?us-ascii?Q?+5A+0+3sjHGj+mg5QI4pkpQo1F8M2zdcEJH4pRjQplESlbU1Icq70Hij/A/i?= =?us-ascii?Q?wdT7XCGqF7IaMl5+XzzQCwl8XFeJfhxlgujFMKyioEpYT9iM1B0wwBGjoN0J?= =?us-ascii?Q?ZnMBAKzsd7387GsdjFm2Q5Cy/vncR4C/mi3ZSIPJS+XgARojBjts8R0kSxCg?= =?us-ascii?Q?DLWrJdcoye8=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?KXZ6oI69a/VggIJpl4PBJlVguU22FSEBGDqkGiTmXd17krTPBfqKtXRh0DDp?= =?us-ascii?Q?Te6s+ca6we9jd321z9hRG7vdxDcwyomMvhW7qDilycdjKm6qpPtt5mNkbs4e?= =?us-ascii?Q?rhHve8CD667BLfGzjS69HuTafd1VLMeo6YKkdmBdEuLPVc7mhhjIB62jPDfX?= =?us-ascii?Q?71DEVGuR9mR5idIrYq3lyWe01HqqAbKSmbNwCTPB7Ahxlf/MN/bVpzFmJgqR?= =?us-ascii?Q?pkrKS453gM8XXA8YcFaHZl6LK1d5NgKETFXET8Kk9K94HoNclzJaLagrkXDm?= =?us-ascii?Q?hpJRGBaiAgefCG0aE1aie69JHb5zbdJEhaWB0JyxllKdjeP6ymyBxiMqTqKs?= =?us-ascii?Q?Q2cJVsC0ZGMa5k9TUVcDYrNixxs04pgFHWVZL3HgnxXapkHQ2khE54+aXbzO?= =?us-ascii?Q?miXYEouZ6Sn24CvRMYz1ORPYrVj3IXws43NWvCdBDhnX1uT72GVCuShx/Xmx?= =?us-ascii?Q?BTP4KOAy2WCptCCLx6l95dq+u8jRaGUWWMXkXmM6udATKjlqbCk+o0kwPupW?= =?us-ascii?Q?P7sUZC3frvYZKEo833buvqU6xdiITDN5RfCXpYMzvjvN14viNiEut0YlbRTh?= =?us-ascii?Q?LT5GZLo4MT/pCGSu9tPUVzWYdltV72Gng3otEOKKHbqwX2KUPMWn+R/tJPhc?= =?us-ascii?Q?l+CnlbTg7NaP1VIE7yWn6/3Bf5qdxzi6ZWXJtlFjVMAhiEex3GiwMbelM8+8?= =?us-ascii?Q?7UJLt8vFkuwIpTL2aa1pcQ/1RTZyYYruFGiTTJvMT6ys/W9D8U7INR7c8ys+?= =?us-ascii?Q?4AQJJy1rRSKqWhMxLEDcRa8gDuuUHczLarwBUFZBgx+NV+uRdZewvH4GgmMJ?= =?us-ascii?Q?7hYCsU7phSkL2y9YZ7Uc7DxgvTCQoAgTfH07b312bHu6f1/g/o0/JLi+Kfv2?= =?us-ascii?Q?u+PsUL2QPrGLAbuOFO+pbpeQ6n4CH5wbEaIsMk2Mke9uJxCZ8Vc2l2JBF/4T?= =?us-ascii?Q?WqxtEsqiOU9wnXBI1Q0qjhI811TbcLUohLD0SXInhx3r9my+vCqDoCAvOEHQ?= =?us-ascii?Q?H8UAcgQbk+zrw6sp2I6oIyCEYvb6LvGV4Z2E9gXN9qDL/R57RElFYZqLVXi3?= =?us-ascii?Q?ptsrpZh4HkSc/pKyCEBrPb81xeW9uhg0Jbzu13R1WASbfQJIWdtgxVmmcspI?= =?us-ascii?Q?Al4Hjv792nh2gdNNZ04j9KawUKxiqlxdJLaByFJi2+wjeNiEnjy2ZuyQEoGc?= =?us-ascii?Q?lKqR7IJsVF5DZVqBBSeta5Q5abJ3ftiBXXtrsKMDjFeaLSi2ksWF/oHTqe7x?= =?us-ascii?Q?8CDHtK/mqRamNzil7WePlSivqK4LawxHeHBsw8SfLJ1vLmiqrWi0ugiJbQat?= =?us-ascii?Q?j1mZxD4UctcYfF+nRwnwAdRiFAuFffnCigidfeZtTD/tuPpyVUGRdoTxRO2S?= =?us-ascii?Q?12o5KsKri7M4NyezNWgr0tL/NcPqL4GznnttRcFsdSTX3sWFukqxO89UGUZg?= =?us-ascii?Q?piA6aerG8pXR8WrGzOzMFCjMVoM5AfloF7xoRQwk9xGOaT9ZyGw8KL7VlNKl?= =?us-ascii?Q?a0BaLXEVAoAH+XgjoaXmBLgeMlRZYHW6ffEXbJly17dMC+ffaKz9f//uGrHx?= =?us-ascii?Q?1Ab5qQnUVb+SDdJ381iMkE8c55zEO8TRefECks9c?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ad25eab5-8be5-4b8e-73b3-08dd9f96390b X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2025 16:22:50.8825 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uK8BK37x8771qsY91bQdtg4wBLxrJHagqp1HWSjR489BKowFiTGyRE5J4rjj4uPB X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9352 Content-Type: text/plain; charset="utf-8" MIGRATE_ISOLATE is a standalone bit, so a pageblock cannot be initialized to just MIGRATE_ISOLATE. Add init_pageblock_migratetype() to enable initialize a pageblock with a migratetype and isolated. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/memory_hotplug.h | 3 ++- include/linux/page-isolation.h | 3 +++ mm/hugetlb.c | 4 ++-- mm/internal.h | 3 ++- mm/memory_hotplug.c | 12 ++++++++---- mm/memremap.c | 2 +- mm/mm_init.c | 24 +++++++++++++++--------- mm/page_alloc.c | 26 ++++++++++++++++++++++++++ 8 files changed, 59 insertions(+), 18 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index eaac5ae8c05c..23f038a16231 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -314,7 +314,8 @@ extern int add_memory_driver_managed(int nid, u64 start= , u64 size, mhp_t mhp_flags); extern void move_pfn_range_to_zone(struct zone *zone, unsigned long start_= pfn, unsigned long nr_pages, - struct vmem_altmap *altmap, int migratetype); + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock); extern void remove_pfn_range_from_zone(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages); diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index fc021d3f95ca..14c6a5f691c2 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -41,6 +41,9 @@ static inline void set_pageblock_isolate(struct page *pag= e) #define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 =20 +void __meminit init_pageblock_migratetype(struct page *page, + enum migratetype migratetype, + bool isolate); void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 bool move_freepages_block_isolate(struct zone *zone, struct page *page, diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 8746ed2fec13..afeae59b29e1 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3319,8 +3319,8 @@ static void __init hugetlb_bootmem_init_migratetype(s= truct folio *folio, if (folio_test_hugetlb_cma(folio)) init_cma_pageblock(folio_page(folio, i)); else - set_pageblock_migratetype(folio_page(folio, i), - MIGRATE_MOVABLE); + init_pageblock_migratetype(folio_page(folio, i), + MIGRATE_MOVABLE, false); } } =20 diff --git a/mm/internal.h b/mm/internal.h index 6b8ed2017743..c43180bea6b4 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -821,7 +821,8 @@ extern void *memmap_alloc(phys_addr_t size, phys_addr_t= align, int nid, bool exact_nid); =20 void memmap_init_range(unsigned long, int, unsigned long, unsigned long, - unsigned long, enum meminit_context, struct vmem_altmap *, int); + unsigned long, enum meminit_context, struct vmem_altmap *, int, + bool); =20 #if defined CONFIG_COMPACTION || defined CONFIG_CMA =20 diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 4ce5210ea56e..43ac34ee8d2e 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -770,7 +770,8 @@ static inline void section_taint_zone_device(unsigned l= ong pfn) */ void move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages, - struct vmem_altmap *altmap, int migratetype) + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock) { struct pglist_data *pgdat =3D zone->zone_pgdat; int nid =3D pgdat->node_id; @@ -802,7 +803,8 @@ void move_pfn_range_to_zone(struct zone *zone, unsigned= long start_pfn, * are reserved so nobody should be touching them so we should be safe */ memmap_init_range(nr_pages, nid, zone_idx(zone), start_pfn, 0, - MEMINIT_HOTPLUG, altmap, migratetype); + MEMINIT_HOTPLUG, altmap, migratetype, + isolate_pageblock); =20 set_zone_contiguous(zone); } @@ -1127,7 +1129,8 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsi= gned long nr_pages, if (mhp_off_inaccessible) page_init_poison(pfn_to_page(pfn), sizeof(struct page) * nr_pages); =20 - move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE); + move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE, + false); =20 for (i =3D 0; i < nr_pages; i++) { struct page *page =3D pfn_to_page(pfn + i); @@ -1192,7 +1195,8 @@ int online_pages(unsigned long pfn, unsigned long nr_= pages, =20 =20 /* associate pfn range with the zone */ - move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_ISOLATE); + move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_MOVABLE, + true); =20 arg.start_pfn =3D pfn; arg.nr_pages =3D nr_pages; diff --git a/mm/memremap.c b/mm/memremap.c index c417c843e9b1..3319e7cc2898 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -254,7 +254,7 @@ static int pagemap_range(struct dev_pagemap *pgmap, str= uct mhp_params *params, zone =3D &NODE_DATA(nid)->node_zones[ZONE_DEVICE]; move_pfn_range_to_zone(zone, PHYS_PFN(range->start), PHYS_PFN(range_len(range)), params->altmap, - MIGRATE_MOVABLE); + MIGRATE_MOVABLE, false); } =20 mem_hotplug_done(); diff --git a/mm/mm_init.c b/mm/mm_init.c index 8684fa851b84..6e753ca2c338 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -685,7 +685,8 @@ void __meminit __init_page_from_nid(unsigned long pfn, = int nid) __init_single_page(pfn_to_page(pfn), pfn, zid, nid); =20 if (pageblock_aligned(pfn)) - set_pageblock_migratetype(pfn_to_page(pfn), MIGRATE_MOVABLE); + init_pageblock_migratetype(pfn_to_page(pfn), MIGRATE_MOVABLE, + false); } =20 #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT @@ -874,7 +875,8 @@ static void __init init_unavailable_range(unsigned long= spfn, void __meminit memmap_init_range(unsigned long size, int nid, unsigned lon= g zone, unsigned long start_pfn, unsigned long zone_end_pfn, enum meminit_context context, - struct vmem_altmap *altmap, int migratetype) + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock) { unsigned long pfn, end_pfn =3D start_pfn + size; struct page *page; @@ -931,7 +933,8 @@ void __meminit memmap_init_range(unsigned long size, in= t nid, unsigned long zone * over the place during system boot. */ if (pageblock_aligned(pfn)) { - set_pageblock_migratetype(page, migratetype); + init_pageblock_migratetype(page, migratetype, + isolate_pageblock); cond_resched(); } pfn++; @@ -954,7 +957,8 @@ static void __init memmap_init_zone_range(struct zone *= zone, return; =20 memmap_init_range(end_pfn - start_pfn, nid, zone_id, start_pfn, - zone_end_pfn, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); + zone_end_pfn, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE, + false); =20 if (*hole_pfn < start_pfn) init_unavailable_range(*hole_pfn, start_pfn, zone_id, nid); @@ -1035,7 +1039,7 @@ static void __ref __init_zone_device_page(struct page= *page, unsigned long pfn, * because this is done early in section_activate() */ if (pageblock_aligned(pfn)) { - set_pageblock_migratetype(page, MIGRATE_MOVABLE); + init_pageblock_migratetype(page, MIGRATE_MOVABLE, false); cond_resched(); } =20 @@ -1996,7 +2000,8 @@ static void __init deferred_free_pages(unsigned long = pfn, /* Free a large naturally-aligned chunk if possible */ if (nr_pages =3D=3D MAX_ORDER_NR_PAGES && IS_MAX_ORDER_ALIGNED(pfn)) { for (i =3D 0; i < nr_pages; i +=3D pageblock_nr_pages) - set_pageblock_migratetype(page + i, MIGRATE_MOVABLE); + init_pageblock_migratetype(page + i, MIGRATE_MOVABLE, + false); __free_pages_core(page, MAX_PAGE_ORDER, MEMINIT_EARLY); return; } @@ -2006,7 +2011,8 @@ static void __init deferred_free_pages(unsigned long = pfn, =20 for (i =3D 0; i < nr_pages; i++, page++, pfn++) { if (pageblock_aligned(pfn)) - set_pageblock_migratetype(page, MIGRATE_MOVABLE); + init_pageblock_migratetype(page, MIGRATE_MOVABLE, + false); __free_pages_core(page, 0, MEMINIT_EARLY); } } @@ -2305,7 +2311,7 @@ void __init init_cma_reserved_pageblock(struct page *= page) set_page_count(p, 0); } while (++p, --i); =20 - set_pageblock_migratetype(page, MIGRATE_CMA); + init_pageblock_migratetype(page, MIGRATE_CMA, false); set_page_refcounted(page); /* pages were reserved and not allocated */ clear_page_tag_ref(page); @@ -2319,7 +2325,7 @@ void __init init_cma_reserved_pageblock(struct page *= page) */ void __init init_cma_pageblock(struct page *page) { - set_pageblock_migratetype(page, MIGRATE_CMA); + init_pageblock_migratetype(page, MIGRATE_CMA, false); adjust_managed_page_count(page, pageblock_nr_pages); page_zone(page)->cma_pages +=3D pageblock_nr_pages; } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5de23eba0db8..0284d74b6d8e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -544,6 +544,32 @@ __always_inline void set_pageblock_migratetype(struct = page *page, MIGRATETYPE_AND_ISO_MASK); } =20 +void __meminit init_pageblock_migratetype(struct page *page, + enum migratetype migratetype, + bool isolate) +{ + unsigned long mask =3D MIGRATETYPE_MASK; + unsigned long flags =3D migratetype; + + if (unlikely(page_group_by_mobility_disabled && + migratetype < MIGRATE_PCPTYPES)) + migratetype =3D MIGRATE_UNMOVABLE; + +#ifdef CONFIG_MEMORY_ISOLATION + if (migratetype =3D=3D MIGRATE_ISOLATE) { + VM_WARN_ONCE( + 1, + "Set isolate=3Dtrue to isolate pageblock with a migratetype"); + return; + } + if (isolate) { + mask =3D MIGRATETYPE_AND_ISO_MASK; + flags |=3D BIT(PB_migrate_isolate); + } +#endif + __set_pfnblock_flags_mask(page, page_to_pfn(page), flags, mask); +} + #ifdef CONFIG_DEBUG_VM static int page_outside_zone_boundaries(struct zone *zone, struct page *pa= ge) { --=20 2.47.2 From nobody Tue Dec 16 22:30:09 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2067.outbound.protection.outlook.com [40.107.94.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE3CE1DD525 for ; Fri, 30 May 2025 16:22:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622178; cv=fail; b=VTYqbVYEHtArEuD8ofGIgEx0gui6lEhBLIF6uuISI0v5BfbcIH+kAdQmlmYP1aN77ieIJJhQxYcgYbj8yP/8fpC9VHPz1OkiWXsH378tOM0LlV0wEdUXh9ktoX30cq3KbZhG4dZRAhj9LblMJwk35+9S+vs5Wa/tiHQH+np8Jwc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622178; c=relaxed/simple; bh=U81CblCcrWScAypmQ6acEpxdQZEO2JLgGOfam4mL1oY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=PblGqPnZCG77YZ6Hn90GpC1SYSfVWPgPA8DNCavWLWoaSUPR4JLas2xIz4KJSvu2lGGQhoNeTjhDaUFYdxgyqQBqozrGKYpDq5jI6bdOMAHTWvVZd/eY/86fvgdgBv08rWbDOa+ZK+ZROLXOh7ACjHA1FS4SOPyCsO2fXjBaJE4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=XHq1bb1D; arc=fail smtp.client-ip=40.107.94.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="XHq1bb1D" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wUghLF3iYaXcno0r9NI9i1v8NYhENIopJC1NvcmIU8WidXMoMT5hyGulrIp0rLYOjdUN5P0ADh1WNLZJR09bei3y6RKiiMw8DwimShCfXPMQv/1ho82kaOb3WaH+F9EDp/ehUnva4azEQHjuUvKSfc7Wwlhb+7MZflQuNnHdaaYivRwA6S9YSNMXa9taGJ/jEmh0Gz5Tbz4OR0IsAI3swRfrMZA0Oc4udkeTuNx7A9kZYpbqo/23jKO+zZ+YtoKoyKCk5DFtkEi0Xc7XZpgokYhwt9P4f5iIWOx7buzv6JdNwSwh3Y4Td5CTW31ght0r3bVr9yT01df98nC5cNEjfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yQkMhK2uz65pnQJTJn2PE8FmaCLhqJsQihDE4G47oAs=; b=QTbPKVBliZjags5M6nbmYEkmOnixCZZhA/hxNL4ROIi6vsjC61lQN0FKDG1zKdzDehnmyZZzoros7nodW23dM5yc64sUnpLdSYZcxqX9vOUPZXE9GeRRAwt49aUmsfi7EH4VxUOQXS4fQ7o18NPzoki8gzzufxwUFmdpYj9lOr80agxhSYvoULZpzo2Nb7y4YDmK5RQmHPGBL3TATC+rQpqxv7bKaP6WMp6nliC/ADQAqmubQfUPsT3N53ZG+zWZZ9le6L45Fx45eyeW1FM98vRsourgBmouOJzc3vZPre3PKLDD5A9sO1QviMYbXXbs2Slmx5QWloh8hL4XWUU8XA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yQkMhK2uz65pnQJTJn2PE8FmaCLhqJsQihDE4G47oAs=; b=XHq1bb1DMylO/QGXBKcK2STA9pn8agJCex9thhdMiljdLTdAjd/59YJpOkH4eVhFH1mP1qCEkJYzDKKQsuu4njQ8PG0xqcJzLP1gMioPvld1Tg977oWhsYWyVdEz8F0rFVpB6rbaClo/xx5ZHlW1fs4GQNYZW5BiLaA5Lfe3mxLin+q9IULFRzI4gvk7ptSWcwzoDiWDPA9ErrmdnYzwMTh7yuIY1ANBh7HK4QTvyBL0YQjpO9gAeg9xvjaK7R5kJ39s6vGZkUYi5hpyDsJhfL11o0zFkH5YO7AEGIbpZBflrFH+koNETwWsZ3YL6ykG8PJBGiXjn0b+ZbrrmEYW4A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by DM3PR12MB9352.namprd12.prod.outlook.com (2603:10b6:0:4a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.27; Fri, 30 May 2025 16:22:52 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%4]) with mapi id 15.20.8769.025; Fri, 30 May 2025 16:22:52 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v6 4/6] mm/page_isolation: remove migratetype from move_freepages_block_isolate() Date: Fri, 30 May 2025 12:22:25 -0400 Message-ID: <20250530162227.715551-5-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250530162227.715551-1-ziy@nvidia.com> References: <20250530162227.715551-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN9PR03CA0649.namprd03.prod.outlook.com (2603:10b6:408:13b::24) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|DM3PR12MB9352:EE_ X-MS-Office365-Filtering-Correlation-Id: 699389eb-c9b5-45cf-2f3b-08dd9f9639e0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?gX73YE8Ft5zF02Ud5a26kmzyXgXrc35gW4av/ZZe8UXb0oaLF6h4hSIh4uOF?= =?us-ascii?Q?UG0DGIM1gFZmIbrppi4MkDBMn9IIxKDA9pWxm/zZ3uihlKgTl3xejU8STIaE?= =?us-ascii?Q?tI+meswbqTReF2YT6VM0bbmuJj/dRmFUTZeqHq2YHClBN77aar4Cy3mTYB/D?= =?us-ascii?Q?OYssU1fy6WryLXzfKJj5FGxl3KTPmCEnE6WV7Ikcw6n1Ai+vgOx0d+sQfbxQ?= =?us-ascii?Q?/Xbvd137ISAU2eTXANgYPKcfEoDWd2skPs/BIjJ5+WEgCxBt3tAyVVYDt8YQ?= =?us-ascii?Q?rTT7aUxlXVsRRg+DDTQMsyJOpej0USg8S6FHRKNj8cJ0CobP+1ITOAQpTeur?= =?us-ascii?Q?B1hkpepOb+jv/Vs5Bq/Mv/XAnYVC/iyjUTZ6fvEn2inQSWtrSOARlQxfI/YS?= =?us-ascii?Q?tO+B14X38dbGq0lkYtGJUQRmQAxouC+1DlHzIsYmzAQWSEAO0QMeHRqQgO+J?= =?us-ascii?Q?LyiMEkmIXzIXYbnSbuu9TOjXSr7Wu5H5sOSKy9ppTiCoFXJH78vFJIZE41tw?= =?us-ascii?Q?DE6cqGc8j4DpP1N8iCruH/a8Lfx+xEgdqbMjstNoz0zEP0/P7jclWkFcIKYb?= =?us-ascii?Q?7dzLobcu1oXMn7SDIObS+NYadSWR1ux1KNjKVCgTxQOFM/sUd20GYgHvfGv+?= =?us-ascii?Q?FHRi91Vc7VsJ7Tx8JKyBL876hqCduhdUF9UGagphbMdZv8/h03RbAaapza5j?= =?us-ascii?Q?N5/XpGU5D2+u+l+V+yrNYro3mHEilrF2d9f+5BWynO9Ut12jdITJYYmn9H2y?= =?us-ascii?Q?paaL9l2idxLdOJQmqFKPeK5A3yo+TbU6W/vpWXkG1AcRY70h/RhmKP2WoWep?= =?us-ascii?Q?H7WRIWENlJCNTbmU285fEJuo0XXq9S3G2jeJBjLq0BN6AkD3JoeepXZkeTiG?= =?us-ascii?Q?QuUFFvO8Koav3YcPpiY++WMkx3cXO7V5wB1cg9/MbcbPNTeKTRXoyiBMT7SX?= =?us-ascii?Q?7WhhLKZ1X4244MxnWHMCJa6KlM4iIlEgV+URAT4+TLWU5JK4Hqjz82egkyiv?= =?us-ascii?Q?X3grAh10nI+Nza0lXxsW4fzjoh7eQDkxayxqWsQ7Zd/fP+xbjqzVwr7hqddF?= =?us-ascii?Q?QqoJqU7WkIm8fOKSQbGCNOm0p2y6/5KDwpztnkhwHcUsXJHiyWeiTvlpKpps?= =?us-ascii?Q?czK2UYtZLKEDJ478DsnOQqWHUp+fe1W3YaFLMw1YK1rQT83YIJoTtMQeKd9V?= =?us-ascii?Q?I1uEK/b1qEki3bMFAqumgOHZM86g10MZyGs3OCm0w9fBZavoI2KixTDNfm/W?= =?us-ascii?Q?vigm3gLvL+RwY/F1FVXk2PfyEZAgJf7aa/ZOfDEUwlwIQN/4kuiUaZKgx1oJ?= =?us-ascii?Q?M0TNcuPTybqn56rH8uENfkI/6hTVPMYckQ3sCcQ+31wF4nASRuzHVCzIt4g9?= =?us-ascii?Q?iF3MuGsq+J0mI0LIxGwIUddpoV11ZJrWt3qjiKo0K2sHmiqp2ApHuSLBsxAd?= =?us-ascii?Q?rCNz3YFpWOI=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?tOwvJ8vsPrUHgnj/YiZbyOrjgpxf9DRgWOlWg0Nb1Crg5w8+Lt0f3mZ3PREr?= =?us-ascii?Q?p+KgQzBMcb1uZ9bqXPFEVishZy2O6p367J2Zr1qvEJ+L/N7Q9vmc3+Nf4mr4?= =?us-ascii?Q?sKxpHfP3hao0ibz5Ntab3k/Maqx4KBzhWJryxD+W1eg39/i7baJ+pwIfsMHZ?= =?us-ascii?Q?tqAKVAQkrPfxjqB1LygzUCWBIpLcWnsr6O1P3PZoTxQlJs5GVQviNMmB6bDH?= =?us-ascii?Q?B7tjA6ub3OqMMasnA4PlK3hm+zqcemq/xZ+jcdRLlr9aM6VudDp1cJtJAoLN?= =?us-ascii?Q?2kI0OO46rC5DxUWDmos6fnQ16PPv1opuaph3FQur9tBp4tssEzcqOk2AEjlV?= =?us-ascii?Q?tmDrw2VRe/VAH5q/uu0VlchZEFr1Rt9gwDWTNcHiqyz80NycToQXOHIv3k6q?= =?us-ascii?Q?X6x+FvSgO+7+hCm7iePrq8LCO8YLQwIAC/vGA/IqSkhFJ7JunycS5QxrqTar?= =?us-ascii?Q?kzaWO0VDlcgAVf6a2UYGaV3cdwgVWthstoDei6l+v46FqITbOsaXF0m9V6bO?= =?us-ascii?Q?hTf2xVcIVTpgk0jwZnKmKrL9b8BbFjEf6tzkuJcAMfTIdacv5clCqJduUK4s?= =?us-ascii?Q?oeHaD9ucBwwi17Z9ET5JcH/jzA50ANqu0uFfdn3czjHMADCIUO/4sFyFWA/p?= =?us-ascii?Q?DExhpM/JB8JNeBBFYiUmXqOg5uBMnYjqrozJ6nz2QIquxWUOS0nDPC6y+Rv5?= =?us-ascii?Q?2YfCt/1HGaHxAUPVAzuoiwgYLPAPcb36RlIm9eLaiN7j95aIvVtMzjfEc3q4?= =?us-ascii?Q?lFdyTTauwWQRVd5fXrY1PoRM1/4oxxlguK3+1nmTZq+LHv7Nb+0GNqdZeB2z?= =?us-ascii?Q?WmruJWxUljdv0xBWyo8cK+nGSMTuyfw8YTskkyU93ORViErdjiwpjZvFJmyH?= =?us-ascii?Q?pS43+9h59ECKgxeJ9Dq+vNYoF+SL3AZhupyioUb6WNOiPPeQ/kuO34uIlCQ7?= =?us-ascii?Q?Yik4bM+ayss9/yX3LEQK2eq4GdehI1S8rGoYJcHuDn6qDSz+GrdsS9XIpaIG?= =?us-ascii?Q?Nyo++k/RXZZSvgcjozC6GLPx7bGD5znfAoUAY0ZvwXwo10L8wATFfiuKdAhR?= =?us-ascii?Q?nLE0Z6S2hdQEJZBOyOxvghqo2cwJSmAJ5Pbm7nbE6Hl6bn9WdkBBz28JoO5i?= =?us-ascii?Q?sMVz9TxbbvPmMvJk/9g5ySAb+lFTBhrM5/3XtHtIJkbp5byIJi5CyAPDD9MJ?= =?us-ascii?Q?dSr1wgIfs7Jr40nfJr+sglejqy6P1ItQXMPE4WtvWJjc8SSL1zKbrVlRhqIP?= =?us-ascii?Q?Xo3mMNG9MSStW9ZreFAFn+CtHfF37f+THwNFAsVOEZWVtuwceVXndDdBG54M?= =?us-ascii?Q?IXkGfvRPoYvux3sxo99FCI6suedWFONRjAD6btaLbfSjQfMonJkV9QLWpSEB?= =?us-ascii?Q?4RSuZYtYBq+s+kFM1r5e9/c2BKKIRag1NR4EXOxVbB16MuYg6wHzqccvMYeP?= =?us-ascii?Q?zTF6cLQycAT/42ABTWwZICT8BltDbiv0TE44sNxq5xx4nP4YOweArdp9vItI?= =?us-ascii?Q?/YQRqwtmnLXtykmqgtjH7GSgoF0ty7aAzSsaThESUZHRyCkPAaUe+4ULFFwG?= =?us-ascii?Q?ppZFUf2wyZTDnGr7tie9zUiXntisWPd6NwgtJo4k?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 699389eb-c9b5-45cf-2f3b-08dd9f9639e0 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2025 16:22:52.2714 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9MnpOJpUSOiXFVmlY7GfdtnzLSYAMU/4yMfVq515/EI2MCsPBcJ8ysd+gcJ8hmIp X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9352 Content-Type: text/plain; charset="utf-8" Since migratetype is no longer overwritten during pageblock isolation, moving a pageblock out of MIGRATE_ISOLATE no longer needs a new migratetype. Add pageblock_isolate_and_move_free_pages() and pageblock_unisolate_and_move_free_pages() to be explicit about the page isolation operations. Both share the common code in __move_freepages_block_isolate(), which is renamed from move_freepages_block_isolate(). Add toggle_pageblock_isolate() to flip pageblock isolation bit in __move_freepages_block_isolate(). Make set_pageblock_migratetype() only accept non MIGRATE_ISOLATE types, so that one should use set_pageblock_isolate() to isolate pageblocks. As a result, move pageblock migratetype code out of __move_freepages_block(). Signed-off-by: Zi Yan Acked-by: David Hildenbrand Reviewed-by: Vlastimil Babka --- include/linux/page-isolation.h | 5 +-- mm/page_alloc.c | 80 ++++++++++++++++++++++++++-------- mm/page_isolation.c | 21 +++++---- 3 files changed, 75 insertions(+), 31 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 14c6a5f691c2..7241a6719618 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -44,10 +44,9 @@ static inline void set_pageblock_isolate(struct page *pa= ge) void __meminit init_pageblock_migratetype(struct page *page, enum migratetype migratetype, bool isolate); -void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 -bool move_freepages_block_isolate(struct zone *zone, struct page *page, - int migratetype); +bool pageblock_isolate_and_move_free_pages(struct zone *zone, struct page = *page); +bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page); =20 int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, int migratetype, int flags); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0284d74b6d8e..bab114022829 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -525,8 +525,8 @@ void clear_pfnblock_bit(const struct page *page, unsign= ed long pfn, * @page: The page within the block of interest * @migratetype: migratetype to set */ -__always_inline void set_pageblock_migratetype(struct page *page, - enum migratetype migratetype) +static void set_pageblock_migratetype(struct page *page, + enum migratetype migratetype) { if (unlikely(page_group_by_mobility_disabled && migratetype < MIGRATE_PCPTYPES)) @@ -534,9 +534,13 @@ __always_inline void set_pageblock_migratetype(struct = page *page, =20 #ifdef CONFIG_MEMORY_ISOLATION if (migratetype =3D=3D MIGRATE_ISOLATE) { - set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + VM_WARN_ONCE(1, + "Use set_pageblock_isolate() for pageblock isolation"); return; } + VM_WARN_ONCE(get_pfnblock_bit(page, page_to_pfn(page), + PB_migrate_isolate), + "Use clear_pageblock_isolate() to unisolate pageblock"); /* MIGRATETYPE_AND_ISO_MASK clears PB_migrate_isolate if it is set */ #endif __set_pfnblock_flags_mask(page, page_to_pfn(page), @@ -1925,8 +1929,8 @@ static inline struct page *__rmqueue_cma_fallback(str= uct zone *zone, #endif =20 /* - * Change the type of a block and move all its free pages to that - * type's freelist. + * Move all free pages of a block to new type's freelist. Caller needs to + * change the block type. */ static int __move_freepages_block(struct zone *zone, unsigned long start_p= fn, int old_mt, int new_mt) @@ -1958,8 +1962,6 @@ static int __move_freepages_block(struct zone *zone, = unsigned long start_pfn, pages_moved +=3D 1 << order; } =20 - set_pageblock_migratetype(pfn_to_page(start_pfn), new_mt); - return pages_moved; } =20 @@ -2017,11 +2019,16 @@ static int move_freepages_block(struct zone *zone, = struct page *page, int old_mt, int new_mt) { unsigned long start_pfn; + int res; =20 if (!prep_move_freepages_block(zone, page, &start_pfn, NULL, NULL)) return -1; =20 - return __move_freepages_block(zone, start_pfn, old_mt, new_mt); + res =3D __move_freepages_block(zone, start_pfn, old_mt, new_mt); + set_pageblock_migratetype(pfn_to_page(start_pfn), new_mt); + + return res; + } =20 #ifdef CONFIG_MEMORY_ISOLATION @@ -2049,11 +2056,19 @@ static unsigned long find_large_buddy(unsigned long= start_pfn) return start_pfn; } =20 +static inline void toggle_pageblock_isolate(struct page *page, bool isolat= e) +{ + if (isolate) + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + else + clear_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); +} + /** - * move_freepages_block_isolate - move free pages in block for page isolat= ion + * __move_freepages_block_isolate - move free pages in block for page isol= ation * @zone: the zone * @page: the pageblock page - * @migratetype: migratetype to set on the pageblock + * @isolate: to isolate the given pageblock or unisolate it * * This is similar to move_freepages_block(), but handles the special * case encountered in page isolation, where the block of interest @@ -2068,10 +2083,18 @@ static unsigned long find_large_buddy(unsigned long= start_pfn) * * Returns %true if pages could be moved, %false otherwise. */ -bool move_freepages_block_isolate(struct zone *zone, struct page *page, - int migratetype) +static bool __move_freepages_block_isolate(struct zone *zone, + struct page *page, bool isolate) { unsigned long start_pfn, pfn; + int from_mt; + int to_mt; + + if (isolate =3D=3D get_pageblock_isolate(page)) { + VM_WARN_ONCE(1, "%s a pageblock that is already in that state", + isolate ? "Isolate" : "Unisolate"); + return false; + } =20 if (!prep_move_freepages_block(zone, page, &start_pfn, NULL, NULL)) return false; @@ -2088,7 +2111,7 @@ bool move_freepages_block_isolate(struct zone *zone, = struct page *page, =20 del_page_from_free_list(buddy, zone, order, get_pfnblock_migratetype(buddy, pfn)); - set_pageblock_migratetype(page, migratetype); + toggle_pageblock_isolate(page, isolate); split_large_buddy(zone, buddy, pfn, order, FPI_NONE); return true; } @@ -2099,16 +2122,38 @@ bool move_freepages_block_isolate(struct zone *zone= , struct page *page, =20 del_page_from_free_list(page, zone, order, get_pfnblock_migratetype(page, pfn)); - set_pageblock_migratetype(page, migratetype); + toggle_pageblock_isolate(page, isolate); split_large_buddy(zone, page, pfn, order, FPI_NONE); return true; } move: - __move_freepages_block(zone, start_pfn, - get_pfnblock_migratetype(page, start_pfn), - migratetype); + /* Use MIGRATETYPE_MASK to get non-isolate migratetype */ + if (isolate) { + from_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_MASK); + to_mt =3D MIGRATE_ISOLATE; + } else { + from_mt =3D MIGRATE_ISOLATE; + to_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_MASK); + } + + __move_freepages_block(zone, start_pfn, from_mt, to_mt); + toggle_pageblock_isolate(pfn_to_page(start_pfn), isolate); + return true; } + +bool pageblock_isolate_and_move_free_pages(struct zone *zone, struct page = *page) +{ + return __move_freepages_block_isolate(zone, page, true); +} + +bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page) +{ + return __move_freepages_block_isolate(zone, page, false); +} + #endif /* CONFIG_MEMORY_ISOLATION */ =20 static void change_pageblock_range(struct page *pageblock_page, @@ -2300,6 +2345,7 @@ try_to_claim_block(struct zone *zone, struct page *pa= ge, if (free_pages + alike_pages >=3D (1 << (pageblock_order-1)) || page_group_by_mobility_disabled) { __move_freepages_block(zone, start_pfn, block_type, start_type); + set_pageblock_migratetype(pfn_to_page(start_pfn), start_type); return __rmqueue_smallest(zone, order, start_type); } =20 diff --git a/mm/page_isolation.c b/mm/page_isolation.c index b2fc5266e3d2..08f627a5032f 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -188,7 +188,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ unmovable =3D has_unmovable_pages(check_unmovable_start, check_unmovable_= end, migratetype, isol_flags); if (!unmovable) { - if (!move_freepages_block_isolate(zone, page, MIGRATE_ISOLATE)) { + if (!pageblock_isolate_and_move_free_pages(zone, page)) { spin_unlock_irqrestore(&zone->lock, flags); return -EBUSY; } @@ -209,7 +209,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ return -EBUSY; } =20 -static void unset_migratetype_isolate(struct page *page, int migratetype) +static void unset_migratetype_isolate(struct page *page) { struct zone *zone; unsigned long flags; @@ -262,10 +262,10 @@ static void unset_migratetype_isolate(struct page *pa= ge, int migratetype) * Isolating this block already succeeded, so this * should not fail on zone boundaries. */ - WARN_ON_ONCE(!move_freepages_block_isolate(zone, page, migratetype)); + WARN_ON_ONCE(!pageblock_unisolate_and_move_free_pages(zone, page)); } else { - set_pageblock_migratetype(page, migratetype); - __putback_isolated_page(page, order, migratetype); + clear_pageblock_isolate(page); + __putback_isolated_page(page, order, get_pageblock_migratetype(page)); } zone->nr_isolate_pageblock--; out: @@ -383,7 +383,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, if (PageBuddy(page)) { int order =3D buddy_order(page); =20 - /* move_freepages_block_isolate() handled this */ + /* pageblock_isolate_and_move_free_pages() handled this */ VM_WARN_ON_ONCE(pfn + (1 << order) > boundary_pfn); =20 pfn +=3D 1UL << order; @@ -433,7 +433,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, failed: /* restore the original migratetype */ if (!skip_isolation) - unset_migratetype_isolate(pfn_to_page(isolate_pageblock), migratetype); + unset_migratetype_isolate(pfn_to_page(isolate_pageblock)); return -EBUSY; } =20 @@ -504,7 +504,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, ret =3D isolate_single_pageblock(isolate_end, flags, true, skip_isolation, migratetype); if (ret) { - unset_migratetype_isolate(pfn_to_page(isolate_start), migratetype); + unset_migratetype_isolate(pfn_to_page(isolate_start)); return ret; } =20 @@ -517,8 +517,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, start_pfn, end_pfn)) { undo_isolate_page_range(isolate_start, pfn, migratetype); unset_migratetype_isolate( - pfn_to_page(isolate_end - pageblock_nr_pages), - migratetype); + pfn_to_page(isolate_end - pageblock_nr_pages)); return -EBUSY; } } @@ -548,7 +547,7 @@ void undo_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, page =3D __first_valid_page(pfn, pageblock_nr_pages); if (!page || !is_migrate_isolate_page(page)) continue; - unset_migratetype_isolate(page, migratetype); + unset_migratetype_isolate(page); } } /* --=20 2.47.2 From nobody Tue Dec 16 22:30:09 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2067.outbound.protection.outlook.com [40.107.94.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B395623642E for ; Fri, 30 May 2025 16:22:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622180; cv=fail; b=qXnz3fVQfrmjzpWN8ZN9XGq2W4E3CsA41NUXAFxljGN5yYvcEdBjn7py2CpL+dxITY/HIefcUrqWO5x4shwbdL+1vmCSHhbDQ3NfMbGz2Q30+Rp+3Tz/BWfORAvlEKwhsSGTbE9gn/Q3Saiw2f7ju5cxaPIZWHCT/lcxbsMNto8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622180; c=relaxed/simple; bh=LvS/1JIskFMUktwfKAtqmyL0OtX/8Q1n6Puzx7+gHPI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=jrKpRQHvB1KjN3RRPFQeUuE8BoP3E3q+dsOeARG8P88dBT29okJLAdoyhzzUxuepGyhNaQQ8XAeQ1GkzX6PikRfC/D8bujLla8eqp6/xxTaf+yFqyzG3WDxvhU9F102qevMBNrRT7E5KpVXs3ScquBLMJqdUx34vuD34qy7HVuY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=CEGd9d2k; arc=fail smtp.client-ip=40.107.94.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="CEGd9d2k" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lunTq+O1fDrWYfZ0zQk9RaAiIHEgd0rUQbP78/j0nwa1q+y/IhEaA+ZBw+iMSksVDnt9HzB6EFzbxft0QKKY1yjQuDGVVPePnERdnyMuMSrvhgZ1Ve+NiuIyS5eeXtTQCxnylNAkIrVTrI/iLa0A9xOn7kVf/wiH668ay0QSbvMZRNxyjOqEC7IIV+g2enVirdkKCAWZy+0dDj2uf2x9gNO+nuq99NMpO2LL2Tr/kYmEvS3d8SVHQCBh5Hh6uR0j/naZmW3ZG/AMlAKrsPn79EXexjAFJ/c75Fet6nSuUR0VbEXfzeav3Vg9+JsF23DUnnU4xR1n/YQU/GpaAe4hfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=b/6F2REMxyWAtpRnlSvivqg8w0x0yFmFzOF1and/A0A=; b=BTnPa9T/26PRucQI0fy1AVyo86O5P/GHVLa2XsD88Bsy9xijzMjREB8axa7UKqFYF0e3pr+UIBb6d9c9DR6UYnD8ao31vJr1nsVx/oIXzlVMwgyp9qDQFQ/ReAL2tE8tYnH8Y3WSFuXobXAhO1WPsZQKmEA/NYpkUN5FXjJBZa5l0hKIofGsugTBAAnVQnn+z7YGJcaLNyjQV1YnZRV9nxoN8WBu8rSZDTeCCXKN5KuyuAlKGNsaHUPzokUaGXTE9HD9R7+PN3vAjGHlt7mSa4ZOW+9pzFM03nX3MhucWUciRtPQJHo/ZEee7kYK2br0Kvq4pXg6vnR5fRQJy9Wzjg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b/6F2REMxyWAtpRnlSvivqg8w0x0yFmFzOF1and/A0A=; b=CEGd9d2k1tbY10FJ1Xos0HVAXwdXFigGAQqR0pKd8rkE2kXWmXNSR4SqxgLFgBDCrevdUCblktwD7epp/40r5Ln1Kmq9FXYdy6xLGYXUXadhcGjjS3Dxn4tVB6y5aFkBtExKj2X/FMdKYWeN67MsWZKUb2kqkr96Km0OZMGe1w2UlqKu1RX0+wXzS4yqD/U8IbEh0kLtYfREJ4sz8hSuer6Hb1UtnZKme022cboAFwcpp7dk2ev0oRQkOCWlq0swKcuTp/lgPb1jEZsCbsm+Zg+jVRcNUP+cJCtxeBogBbWZH7M7xm5hAUCQTUyXQ46ITmjSGAshSwDdrgr2nvQ7TA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by DM3PR12MB9352.namprd12.prod.outlook.com (2603:10b6:0:4a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.27; Fri, 30 May 2025 16:22:54 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%4]) with mapi id 15.20.8769.025; Fri, 30 May 2025 16:22:54 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v6 5/6] mm/page_isolation: remove migratetype from undo_isolate_page_range() Date: Fri, 30 May 2025 12:22:26 -0400 Message-ID: <20250530162227.715551-6-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250530162227.715551-1-ziy@nvidia.com> References: <20250530162227.715551-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN9PR03CA0646.namprd03.prod.outlook.com (2603:10b6:408:13b::21) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|DM3PR12MB9352:EE_ X-MS-Office365-Filtering-Correlation-Id: e264d5b1-533e-4488-b969-08dd9f963ae8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?H8z8n2nb9AXIN01GBJfCG62kbgY79G00+8zUKhJ8RdLMODK1WYv9IQmoPJqS?= =?us-ascii?Q?XZlbfj8GfqO9uvhYJBybGiDXpVaYUudb7mlwaxdu0Ak+PQfK19ZZe7qRc5eI?= =?us-ascii?Q?kEo2GP5pTLP7eV5XiCqiuajTUZxVqDuaYESCJchXL6P76dukiFWO7WtYTS30?= =?us-ascii?Q?tgeLYhiI18mchc7BIolD564gMe9O6mZ3HtFQPTix7GxXBzBywybIBtWsnK/z?= =?us-ascii?Q?JSE1ixGHKp4c/PlB4JfAD9/KMVQ8FQwT/ufg7cpavp66vb6E1AI1ykG1lGWf?= =?us-ascii?Q?d6+qXA+iDoy8Rep0axIacuq1ObBkMW+M12Jx26Of9pK0filg8lDhMlF+LZiU?= =?us-ascii?Q?On6CmlAMII4CtN6REHx2Qt67pVjLdJKoBDe0cBJdDRmWjBn6WLK8w8zr9fw6?= =?us-ascii?Q?Pl/2eX8gS18uYRRx+bcVdeMxw30SnZV9B8jSlJRQN49amgNl5kwspXDkFOeh?= =?us-ascii?Q?F8BbU33iVD11JRdv/+/JpbQG4ZQMLSE0neRoj5ixbO9LrUJqpyfOKXGZtEOa?= =?us-ascii?Q?N6A8gbqCDNB9e3uzDCzWP4B/ghgCPpthjYSqAgrYWiRxzma+UBJix3zWRJNI?= =?us-ascii?Q?3jEwl/fXW4K6WSDOrgWtWAQSyHzXC+kQcBHLlkxWfbTDViz9pPKmDi3Ee9xH?= =?us-ascii?Q?kt36gbQgVyM2MYDB9uQJ1ctUSKYGKjgCjRnDwPNGezTF0kOqDoS58MRIrW4O?= =?us-ascii?Q?1vdeAtHpreySoZJnYC5zkusMEGehgIC8izefGK1lcyReoFpJQ6Az+UHKg0T1?= =?us-ascii?Q?WWsz1AkhNepYA62gx1AHZUrPXbyWD6ToXHKet1/Np/P8PhqfnkeKqdFOLNCn?= =?us-ascii?Q?YXlHW8/R7LFCCOT4D2uzhhKkMLuRtfez478v4snChHwKVsysI70Lf7V7a9bK?= =?us-ascii?Q?x97wIMCuHmKHie33zdh+kMECclUgSLINXdkejKCVcWYJmMpkTl1tjcssXFCO?= =?us-ascii?Q?h6Awik8WEd17LgdFmfYoGhFeglEAvQ0K6S6afre5VWoBazk0OUljTDQhMIc7?= =?us-ascii?Q?s5AnqgslT/o4xosUbyCcHcYVU1VbOraUFtpQx+k+X1E9X0uVYuRfoVh69OC3?= =?us-ascii?Q?X7zRnzeLOnDFpAb7/1FnzTl77NXH159gZS8/Dxp0Li0Hqx0YICVjl4SNSVmY?= =?us-ascii?Q?hx8YuoWYze0MLcDY0Vqxb99mNU6g6oN8izm6JUS3KjELPoGPLGWqfAtqBBuV?= =?us-ascii?Q?MXsv4qIJxHZEo0Qk1mGHc8ZC66coh/U6kjZBKAgL5o3CR/gGl8rzNk6usQuz?= =?us-ascii?Q?cJOU7b4x/v7sDhAtFWqfthB7uS7D93Ojya6ML4MWZkw0P1C4LN+KJgtQkHEV?= =?us-ascii?Q?7JxTXkco7ocrUbxSUT6yJR9I/PU0ITLrByA7muliCWRuZm+eRsH17cShC0hq?= =?us-ascii?Q?LO4Dt071EZcLCOt95SmPpEi92wXl3FEIT0SJbLU3RCE1PTOcTknXM9GoHLd+?= =?us-ascii?Q?ptNOoD9Izsw=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?VHHhTuvSS0ngB/KCnieMegRUYXUAZeQ/y9PxVpJMMWtNc2bLLGcM9X8cyApD?= =?us-ascii?Q?x2zrObXcEB4GoRJACfcOL99Qbwu5rxxllEZ0GQ2G29I3IbW5CrUDneP2/UIc?= =?us-ascii?Q?++TMD0LAX2t9tOXwlNrN2Ai5m8qmXMjXyzQ/oIX1D9AM6IKfawWudTSZRhdI?= =?us-ascii?Q?iFTd7aweEWKdem+Uhc37VKSBZXeqqJhnWhXGAWLMrWsVgtcgjdZ0WKvnhqPd?= =?us-ascii?Q?hwKW/PNX1/aGooYV2g365kGNhIwer+83DsX6FUFeRiU5XC2cvk/DCk2xwtOM?= =?us-ascii?Q?JGDFNFUo5AKPmxFe8BLeNHHp9dshxJvyLLlpZa24aro5K8g1Z6fw94AQjyBj?= =?us-ascii?Q?XzwzMLZnNb37URMa61pruuBbpWvBSEk4It2SsSfUuHWRevVYGJYhnaU1XNcp?= =?us-ascii?Q?WEVg5c9y/VZZW9T4n1ijPwzu8KBUwVVgb7G2EurrJrTBhzBFaEZEf3hyDD/V?= =?us-ascii?Q?KLgu6VyL02CPdvl+hDkCPPv9rB4N5E2R87KTXae0LdKkmF1jKYt9WW3FbZKd?= =?us-ascii?Q?aTOeRqmwdfpSEs//MlrusiNIefQ20Ht+xPesp5QgGrj9LVdI0FpCayaRnQHo?= =?us-ascii?Q?HKF7ftg3/6RNjOGeBBh0CbV0ol8SRMXcIai+5YowPw6Df3keyAb8zcp2mRwE?= =?us-ascii?Q?pqhTyGZVTwaZmmGaHD3Byg+7G97sit7Nju5wlZ8QI9kW6dFiVHJ5BGKD3hCp?= =?us-ascii?Q?OHKOcV78XY+PGiMPxF8IrSeQetCPM5KqUY1btUFJteAG7R8XyHK1M5Vm6oOV?= =?us-ascii?Q?J5qmLbkFA5/K5O1O5qayTn3ONxdJOG43DALtzJgCqIxFR4w/0Qo78+9oJfwS?= =?us-ascii?Q?2JrWKHi6vb3TQ2Itv6wOvV6CLY8Ij3VqikB7ylWLTrB2n8hLz8XRJfGOnreh?= =?us-ascii?Q?yJbhvWV/8DIw+tGIWxdev7z7UMwgMUAfh8GmgEmCIMYPuwgL0QO7qmC3Qbww?= =?us-ascii?Q?sdbWstQttj3FjHGEx6H2Ah8x/uNzRgxQZpNyJpOL/q7JbVPIZO83MKK5pT6T?= =?us-ascii?Q?l9NWYARRVX3HlAyQE37zQIDwZOqsJ1Hq0/CbSRiCehh7d5s4fmwkOVx1MqQT?= =?us-ascii?Q?dHtNvbL5lagk7x8OGxU8UwToeM5qVBvn6DVw05LC8EYGvtJETsnNXjHnVr/3?= =?us-ascii?Q?EkWIiIdzQ0EUl4qRGSqCa+3JrwaGyn3lfw4wuKmoCgVKsMQ7+gsJ7DzgqJTH?= =?us-ascii?Q?0+7wtGJpZNteWGFV1JdNx6QS90tR5VyGdWdgbBfkiRdTxKqLUqq70r6AvnDk?= =?us-ascii?Q?BKaP9kk5lUasrQMseDU6ErrbNjKE9cx7ickdYRTDaEN1NwTSBrN9uO9T5kjO?= =?us-ascii?Q?8mujtmHTTsa/hevts26jnmfStEtJNRHH2+kT8xCTvhEV/8G8YCJIHqvSX0l0?= =?us-ascii?Q?mH4AtbeXR4CvR/6kuYONOsJaCmAGXz6/gR7sTDN7SNHdkg/BUqUj+HuBMtDb?= =?us-ascii?Q?H183ZiyV9HGl5v638S1QHhJWlGUerEw1oTPQhgIVHRfwmSHAc+Pl6IaX7FGl?= =?us-ascii?Q?9HmUOEeEYVukupWYvwU970A3bbUdv5rFkMVc9AM+8ZmEMuKnR2eE7lY4UD+X?= =?us-ascii?Q?H9UVXXyXUzrpKt6EVHEcqModPMYMtNRM0samh9BW?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e264d5b1-533e-4488-b969-08dd9f963ae8 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2025 16:22:53.9666 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BpIkbaROXQ5wlmdRtEFdoDjJuxPl5/2egpq2mNAYvrevy4Tlskz7b/S0iR6OiB/B X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9352 Content-Type: text/plain; charset="utf-8" Since migratetype is no longer overwritten during pageblock isolation, undoing pageblock isolation no longer needs which migratetype to restore. Signed-off-by: Zi Yan Acked-by: David Hildenbrand Reviewed-by: Vlastimil Babka --- include/linux/page-isolation.h | 3 +-- mm/memory_hotplug.c | 4 ++-- mm/page_alloc.c | 2 +- mm/page_isolation.c | 9 +++------ 4 files changed, 7 insertions(+), 11 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 7241a6719618..7a681a49e73c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -51,8 +51,7 @@ bool pageblock_unisolate_and_move_free_pages(struct zone = *zone, struct page *pag int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, int migratetype, int flags); =20 -void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype); +void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n); =20 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, int isol_flags); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 43ac34ee8d2e..ab66acd3e6b3 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1233,7 +1233,7 @@ int online_pages(unsigned long pfn, unsigned long nr_= pages, build_all_zonelists(NULL); =20 /* Basic onlining is complete, allow allocation of onlined pages. */ - undo_isolate_page_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE); + undo_isolate_page_range(pfn, pfn + nr_pages); =20 /* * Freshly onlined pages aren't shuffled (e.g., all pages are placed to @@ -2119,7 +2119,7 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, =20 failed_removal_isolated: /* pushback to free area */ - undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE); + undo_isolate_page_range(start_pfn, end_pfn); memory_notify(MEM_CANCEL_OFFLINE, &arg); failed_removal_pcplists_disabled: lru_cache_enable(); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index bab114022829..a248faf30ee0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6988,7 +6988,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, start, end, outer_start, outer_end); } done: - undo_isolate_page_range(start, end, migratetype); + undo_isolate_page_range(start, end); return ret; } EXPORT_SYMBOL(alloc_contig_range_noprof); diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 08f627a5032f..1edfef408faf 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -515,7 +515,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, page =3D __first_valid_page(pfn, pageblock_nr_pages); if (page && set_migratetype_isolate(page, migratetype, flags, start_pfn, end_pfn)) { - undo_isolate_page_range(isolate_start, pfn, migratetype); + undo_isolate_page_range(isolate_start, pfn); unset_migratetype_isolate( pfn_to_page(isolate_end - pageblock_nr_pages)); return -EBUSY; @@ -528,13 +528,10 @@ int start_isolate_page_range(unsigned long start_pfn,= unsigned long end_pfn, * undo_isolate_page_range - undo effects of start_isolate_page_range() * @start_pfn: The first PFN of the isolated range * @end_pfn: The last PFN of the isolated range - * @migratetype: New migrate type to set on the range * - * This finds every MIGRATE_ISOLATE page block in the given range - * and switches it to @migratetype. + * This finds and unsets every MIGRATE_ISOLATE page block in the given ran= ge */ -void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype) +void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n) { unsigned long pfn; struct page *page; --=20 2.47.2 From nobody Tue Dec 16 22:30:09 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2067.outbound.protection.outlook.com [40.107.94.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72DAB2367A0 for ; Fri, 30 May 2025 16:23:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.67 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622182; cv=fail; b=lHoKwOOJAk6t9btNIkbThSErrZZmaqmtGjLPs15q2MgF8jMIeAfRudX0DG2vnOXBNrj2KWATAGgW0jqLtp8hG0GhSPirxYuq15fBPT3iRcyhznKgOMuZQWidOYutsG0zx5yas2fn9EduFhRGTjvTUEkwUMuZ1e9UZonLUQmWk+Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748622182; c=relaxed/simple; bh=sFmvWoG8UWI82YJ/HyVQD4m849oUTB2wyhnSXUCWp+Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=mn2MqtuOrIGvro2NmRJqDDGPYJuqNb290L+gDT3O7QPm0pmqFWMl4mqnWpJtunQOhn8RbV20FjrgJo+HJyhlKCqA4D9EDtFJut/4gueRJAKpMaCj2FezCOVRiArpn3QgRD0at2Viy7yZOkAx68xc1C5YJ0VfpHZ7938uOHINMMI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=SQGEdxb5; arc=fail smtp.client-ip=40.107.94.67 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="SQGEdxb5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wDd16J2m784gQKYYgA+nG9YDZdkQ98vRkmubi0T1XCKBYGIzpCZc1ggrR/9XOMPofLesq9W4nEZoCBqs+aJr6CvS2ukulTu2davJzpn6L5Yu25KX4IzxZblJnG9kHGb1TNZjzNZ8lc5zY6C8ka725sXeXjGUIDmzsxuMkmhYyssDM2h6DChlefDJUzByVSK1X3RZv7TK4RC5/nhM8rxxe6owoq8Zx7+aHTj98DxyTvRnp4tQzwsU+b45Hxa/UBz7ro3qBrCl4XTTUu3HLlZ2iNr97LSUCJYlAyRBODRyWS7hyiN7NL9aWfU5k4rwafzLGMSkyeielF8XuQu5XZgBdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wb/Fp+uRMUu3LWYISpfCe6+ZoJXs2ED2L45GhzmoFkw=; b=VrU1OvYf+bP4HfW+jNWX8IBpVTs1vjY8z/8imJ41up0q1fWg00er3WlRDUZPvd/UF7o3WnABSraSrGm2TKG5JX+zz/4VapwavsB7Mb3sBslbPJOws1gaw8n1XcMxfBt1qTsZ+urX96OkcZLSfQrr+NPtkO9C6MMyQH+GF+2TXtUWGl/D0Np/hfuUNyXU9Stccp3dPtuAJ+zmrLvng7GowKT5/RkVOMTMOsbDCamA8/Oweu9iOKN+uPPCYFFo4G25oN7FF61Uo80RYIVRx+h/2lQFHKsoHdgqozld2CNSFA84Vsj5dbJSIGfD2cCB6fakkc/beyLNBGV9VbTCSo4yng== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wb/Fp+uRMUu3LWYISpfCe6+ZoJXs2ED2L45GhzmoFkw=; b=SQGEdxb5wOZl/U+JqwQKWscf1bT9tq61mJQEi0Ie3MTlxjisoIbufdMEZIESVMssuB9x87R+Bo7FdfirXKMa3Kzs3ByvG8I0/zq8sRyD+6/Dqm6pmVXtmBCyL1/c+6urEtiM82x/gE7ZBs34qpREKNsP3FRQIeSENROwwmJRFMIp/3yZJykw2seoCZyTLjyL5KgDNZ0UYjOixTpLf8C7zbKytc1D+g8tFTC0W5xwAKrgSBE+cCvmqOuWpetw8X9RzWX5LXv9n29FH53vQAPZG2lzS/X/8ZaN8FJOY5yvJyp5e9obO0g0/2CNDJ6Wnom9XLfOFdJi/b9+MxuO/qbM8g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by DM3PR12MB9352.namprd12.prod.outlook.com (2603:10b6:0:4a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.27; Fri, 30 May 2025 16:22:55 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%4]) with mapi id 15.20.8769.025; Fri, 30 May 2025 16:22:55 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v6 6/6] mm/page_isolation: remove migratetype parameter from more functions. Date: Fri, 30 May 2025 12:22:27 -0400 Message-ID: <20250530162227.715551-7-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250530162227.715551-1-ziy@nvidia.com> References: <20250530162227.715551-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN9PR03CA0649.namprd03.prod.outlook.com (2603:10b6:408:13b::24) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|DM3PR12MB9352:EE_ X-MS-Office365-Filtering-Correlation-Id: eea8a2c8-ea08-4ffd-f0e2-08dd9f963ba6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?QXIOsZ6AE/Zqw+XGItAxgmCkDxDFsCkfuB1tOrj7kW9QqxHZJ86EA/bja9fH?= =?us-ascii?Q?/sZWGLY9wAB7TnWy9kE7JZeFGeva9WyuG90D8Vsv2hOej8phKrYESC9PFS7q?= =?us-ascii?Q?WeRF7gCAHWUAirSVthyZEBkcC/wWhp10Fpdfco9my/Hlz0Raybfqexwc4Wtq?= =?us-ascii?Q?EOmUjyjjf5KXqoT8KBuFcSCGY685Rfsh5XS0Kcwzc//cW+KaLj2mLtTlnkxY?= =?us-ascii?Q?GDPZctINoCule3wqVZommZpAaOljc8af8mtcJhuQgIiQSxZqN+VjUn64Ng10?= =?us-ascii?Q?58LofmSeSv7+uiZbgyCEsRh03+VCA1rC33cHE2/vN1xnjsv9p4Vqrj5NeuXB?= =?us-ascii?Q?QHrHkabX+APM1duG0BkeVabtwy17VdfFLhCOYo5iUGWpGf9XBsvNpq5nYoff?= =?us-ascii?Q?CFiW/jDT3eGLLqmuTRCDdYTh+TbirSdazGTpI2tM0OMqcPB4brlpBQ/oFZpt?= =?us-ascii?Q?KlwtMHJ1b7w7SkVUVcT8fYnLVAurRcl2Dcs/W6BI2yA9b7XcZPntlvnu0yS0?= =?us-ascii?Q?uZ0SxM3wktAR8DuZopytcsL9NEeV5fxU4IucJqiSRRYxhKhjS+iEu02Gg6Ue?= =?us-ascii?Q?Mgqzh5isS/lpDWjuJDssS5tkSVgTTXRYXth7cPEVybk2cYlyg0XanLtWDbt0?= =?us-ascii?Q?MV+agk4dqdqlxz8FvDYMBS87dPT84cMREe12zMOU9ThKIjKZ+U5TJtldbkQ5?= =?us-ascii?Q?ECtX+Js4z3g02PX4VPW1qPHX/8RyyuKgGf/GUzu3FlAZPbj6+I49yoOIRuT7?= =?us-ascii?Q?icc3BvPp43TzI6wPFYO82K0t0U3YrOU4oMgyjUkqmSYR2lwNPd00//oTst7T?= =?us-ascii?Q?EwndOW3PSVz09dkp7OLSf+grihVsdTRQOrumJkSweU54P6UtufrQ3JupjHi5?= =?us-ascii?Q?emV/RFm0Noh0l9urm+nKdVAwMNYzoTgpwM9wgATGWY9/CJzFdLtFK1/KlVRz?= =?us-ascii?Q?PdrUaHR5gHSYxWKw+V0nZYmr+0GLv6tLORscfFP38xm8fhKR4vIGVeKO3L8J?= =?us-ascii?Q?+rKM3uXy+iec7Y/cHqWWNQD2gUtmUqTimILLrLuTxpH9DicBnoEqgF1xjTAe?= =?us-ascii?Q?vvTBKp54Rx3j+3sDqpHmh4eQkY22g0iSsKoktxNJnx5tytGMJIhCpcVdlOKc?= =?us-ascii?Q?lYV6JoQNHJgw0s/wrRNM6NpHMK1dGr1Lo15ActcLjRFSNTRgjpjNeJgO0lXP?= =?us-ascii?Q?wVAq1/fAwClcXM7wYCKi+WXq4iwn3AsifGeLFDiayzUB/UVFkBj2NmAObDvm?= =?us-ascii?Q?8UXyckafYlGnAhUPPLn67SixTGpX6pV5oDaHMOW+dPs0ikTWsgMOSanK2/ip?= =?us-ascii?Q?X7yZ9YwRCSJwJCLsSISlTAmid9XEcgPgdTTkykReXYLwNXFLT53703tU7fF5?= =?us-ascii?Q?fzkqQ9PvJqMQOVltnQGfWN00xiigsPS+Ff/0tFNNmUXsiZ1oV5D4s7Qwccen?= =?us-ascii?Q?lG8L/xLhcIw=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4VfCvKWpcCzq7IFJVfXPiIqZ2gI4Z9f+MT2qAJCnx5gh3XiHViItRDV1cGjl?= =?us-ascii?Q?0Koxag2o4Mu0ky4SnXiV5vVEutwLgURCsdzKHp3TsObvcQvxpBWaQ3k7O0GV?= =?us-ascii?Q?23svcUH4xmj3+q1dNqiZ/5EfnUTEsnMSh1n02bUfpGMvCadMP55BQNiBI3aU?= =?us-ascii?Q?kF29EbnuCNBY/MR8u2ChTrHIbNgS1vb/6ABCgOkP3GpwD70xE3Pz99ZFEsGh?= =?us-ascii?Q?1eDhSIZTlFAaqVVEUVeXcWzSkrbMqSRVEQAZfTkH2i5FYZsee3dViKhi44y+?= =?us-ascii?Q?2tNSfRcjjyT3HngKVXqF0yN5Ene356T4ZTah/asE59s8xmxxaZkdB5lAlsRN?= =?us-ascii?Q?NMqsoMfUv6k7NhiaiDtShCww4GcsMbildSfphVzj6KY/iOkLUep1yjPgkcGj?= =?us-ascii?Q?jSA2VgePbnQe9JA7OLjOMLcq8cGBhRG1Dm1wDNwXVk3AyOCqyLa5EPlKnBy5?= =?us-ascii?Q?nyEBp+eXVLOgFZvwcD6Y9IE7XGG0YU0aGLMjyfV285dfcCbtXT0sksIklmN5?= =?us-ascii?Q?Bddqi6KGljzirjuIEcwHZ0AI/tGE1uyUh3iX8xpdPD9lAVT6oev3cuTfKXRn?= =?us-ascii?Q?KnIhvSp1ijsDrOHY20/aTFXZ3H+nPJQoXyJX/E6h2BK8vwyC9qdv6zPAyiZk?= =?us-ascii?Q?LGUVPY4Tepu+hK8U66qmm+blWYQ989nKeBCx5XqCZEUtilBpBaQydtTJsqiJ?= =?us-ascii?Q?8U/ybGK7ZVOZIGB6g3uOytPLGpsZ0rmp0yuAuhuL4LrzSXzymyGZPOXEGzOz?= =?us-ascii?Q?O6yeZYZ+VPRr0vQG9skndlGHRB/reZnXpxt80NFm0s0x3AVHRIMfQt1+GnfF?= =?us-ascii?Q?VekKAfHpHk3C9eiWkk7cBvoIht7LtxOMQCgVBMObypcPfkxHGPIHm8xwXFMY?= =?us-ascii?Q?lzOn6mP+1qUZ3ttVIV8DW2oXD7JzLIPgzbzC7ZEcfd2/lRVSRMmyB3MmETQP?= =?us-ascii?Q?fPz2jq4XU/U8zhTHK7xmsacyykuDg3H4Z/kyBNgYPjt3JhcPfDrbStd99+mJ?= =?us-ascii?Q?HOY740MGQTMRepgfOOlLCh5mIg1ITyL1nRTJZbmd3tntp7lRKv1n8y8wr9xs?= =?us-ascii?Q?p9nr81WUIUSKmjntcNS1d7/EAIeUQ5syBo+ePmAsxDLgu0KmweNmw+R+4s7p?= =?us-ascii?Q?+J1oS/Kuj7aJql6DBx707MSKxMAwdnbHocNu6C+FZoHlMJ6JNlh+iGv6DXoN?= =?us-ascii?Q?anDmTJ5NUXQyibCGSekKoJvxOZa7ZNxebQJKfI8Ksutk5Zoe7UmXcdI9pLMy?= =?us-ascii?Q?Jaa0VTn+qDrN+oNQd/eFXarjWn0IIK4EgDnwqNCLJmaqUwBjwhdNlM7xpHzt?= =?us-ascii?Q?e1iqJ5Kbbjb3czUrxOSTnjkqzqRim6ZBPNj0UAd7y/XQszwveA6+NlLFx6A4?= =?us-ascii?Q?Z+C/a3k+OmJK019PehGvW5y9ifpz04i01AWzGI0AXQui9TqT9d8t7subifOa?= =?us-ascii?Q?Vuesgf827pELqQy0hQ+kbqbEXHzzAkhKZPN/dMk7KeeRQ6vp8qaUJrWZ4xIp?= =?us-ascii?Q?HYZ4griR7fukzHvs//XrrJGSor8QXuFsaJgHdhxy0In49xzAMeW76RY/vIfU?= =?us-ascii?Q?bZ7b4ct0fp+FbPd3HL3PjgHC0iJ788D9o/qbQ/RY?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: eea8a2c8-ea08-4ffd-f0e2-08dd9f963ba6 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2025 16:22:55.3079 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: t1LCcF+GHO/46pB1yTSsy3/ii2MDy9SHOydL0MFwSh8s9AJqtJkLCmS3g4Bpoa5s X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR12MB9352 Content-Type: text/plain; charset="utf-8" migratetype is no longer overwritten during pageblock isolation, start_isolate_page_range(), has_unmovable_pages(), and set_migratetype_isolate() no longer need which migratetype to restore during isolation failure. For has_unmoable_pages(), it needs to know if the isolation is for CMA allocation, so adding CMA_ALLOCATION to provide the information. At the same time change isolation flags to enum pb_isolate_mode (PB_ISOLATE_MODE_MEM_OFFLINE, PB_ISOLATE_MODE_CMA_ALLOC, PB_ISOLATE_MODE_OTHER). Remove REPORT_FAILURE and check MEMORY_OFFLINE instead, since only PB_ISOLATE_MODE_MEM_OFFLINE reports isolation failures. alloc_contig_range() no longer needs migratetype. Replace it with enum acr_flags_t to tell if an allocation is for CMA. So does __alloc_contig_migrate_range(). Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka --- drivers/virtio/virtio_mem.c | 2 +- include/linux/gfp.h | 9 ++++- include/linux/page-isolation.h | 20 ++++++++-- include/trace/events/kmem.h | 14 ++++--- mm/cma.c | 2 +- mm/memory_hotplug.c | 6 +-- mm/page_alloc.c | 27 ++++++------- mm/page_isolation.c | 70 +++++++++++++++------------------- 8 files changed, 82 insertions(+), 68 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 56d0dbe62163..6bce70b139b2 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -1243,7 +1243,7 @@ static int virtio_mem_fake_offline(struct virtio_mem = *vm, unsigned long pfn, if (atomic_read(&vm->config_changed)) return -EAGAIN; =20 - rc =3D alloc_contig_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE, + rc =3D alloc_contig_range(pfn, pfn + nr_pages, ACR_OTHER, GFP_KERNEL); if (rc =3D=3D -ENOMEM) /* whoops, out of memory */ diff --git a/include/linux/gfp.h b/include/linux/gfp.h index be160e8d8bcb..51990d571e3e 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -423,9 +423,16 @@ static inline bool gfp_compaction_allowed(gfp_t gfp_ma= sk) extern gfp_t vma_thp_gfp_mask(struct vm_area_struct *vma); =20 #ifdef CONFIG_CONTIG_ALLOC + +enum acr_flags_t { + ACR_CMA, // CMA allocation + ACR_OTHER, // other allocation +}; + /* The below functions must be run on a range from a single zone. */ extern int alloc_contig_range_noprof(unsigned long start, unsigned long en= d, - unsigned migratetype, gfp_t gfp_mask); + enum acr_flags_t alloc_flags, + gfp_t gfp_mask); #define alloc_contig_range(...) alloc_hooks(alloc_contig_range_noprof(__= VA_ARGS__)) =20 extern struct page *alloc_contig_pages_noprof(unsigned long nr_pages, gfp_= t gfp_mask, diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 7a681a49e73c..3e2f960e166c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -38,8 +38,20 @@ static inline void set_pageblock_isolate(struct page *pa= ge) } #endif =20 -#define MEMORY_OFFLINE 0x1 -#define REPORT_FAILURE 0x2 +/* + * Pageblock isolation modes: + * PB_ISOLATE_MODE_MEM_OFFLINE - isolate to offline (!allocate) memory + * e.g., skip over PageHWPoison() pages and + * PageOffline() pages. Unmovable pages will be + * reported in this mode. + * PB_ISOLATE_MODE_CMA_ALLOC - isolate for CMA allocations + * PB_ISOLATE_MODE_OTHER - isolate for other purposes + */ +enum pb_isolate_mode { + PB_ISOLATE_MODE_MEM_OFFLINE, + PB_ISOLATE_MODE_CMA_ALLOC, + PB_ISOLATE_MODE_OTHER, +}; =20 void __meminit init_pageblock_migratetype(struct page *page, enum migratetype migratetype, @@ -49,10 +61,10 @@ bool pageblock_isolate_and_move_free_pages(struct zone = *zone, struct page *page) bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page); =20 int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype, int flags); + enum pb_isolate_mode mode); =20 void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n); =20 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, - int isol_flags); + enum pb_isolate_mode mode); #endif diff --git a/include/trace/events/kmem.h b/include/trace/events/kmem.h index f74925a6cf69..7c4e2e703a23 100644 --- a/include/trace/events/kmem.h +++ b/include/trace/events/kmem.h @@ -304,6 +304,7 @@ TRACE_EVENT(mm_page_alloc_extfrag, __entry->change_ownership) ); =20 +#ifdef CONFIG_CONTIG_ALLOC TRACE_EVENT(mm_alloc_contig_migrate_range_info, =20 TP_PROTO(unsigned long start, @@ -311,9 +312,9 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, unsigned long nr_migrated, unsigned long nr_reclaimed, unsigned long nr_mapped, - int migratetype), + enum acr_flags_t alloc_flags), =20 - TP_ARGS(start, end, nr_migrated, nr_reclaimed, nr_mapped, migratetype), + TP_ARGS(start, end, nr_migrated, nr_reclaimed, nr_mapped, alloc_flags), =20 TP_STRUCT__entry( __field(unsigned long, start) @@ -321,7 +322,7 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, __field(unsigned long, nr_migrated) __field(unsigned long, nr_reclaimed) __field(unsigned long, nr_mapped) - __field(int, migratetype) + __field(enum acr_flags_t, alloc_flags) ), =20 TP_fast_assign( @@ -330,17 +331,18 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, __entry->nr_migrated =3D nr_migrated; __entry->nr_reclaimed =3D nr_reclaimed; __entry->nr_mapped =3D nr_mapped; - __entry->migratetype =3D migratetype; + __entry->alloc_flags =3D alloc_flags; ), =20 - TP_printk("start=3D0x%lx end=3D0x%lx migratetype=3D%d nr_migrated=3D%lu n= r_reclaimed=3D%lu nr_mapped=3D%lu", + TP_printk("start=3D0x%lx end=3D0x%lx alloc_flags=3D%d nr_migrated=3D%lu n= r_reclaimed=3D%lu nr_mapped=3D%lu", __entry->start, __entry->end, - __entry->migratetype, + __entry->alloc_flags, __entry->nr_migrated, __entry->nr_reclaimed, __entry->nr_mapped) ); +#endif =20 TRACE_EVENT(mm_setup_per_zone_wmarks, =20 diff --git a/mm/cma.c b/mm/cma.c index 397567883a10..9ee8fad797bc 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -822,7 +822,7 @@ static int cma_range_alloc(struct cma *cma, struct cma_= memrange *cmr, =20 pfn =3D cmr->base_pfn + (bitmap_no << cma->order_per_bit); mutex_lock(&cma->alloc_mutex); - ret =3D alloc_contig_range(pfn, pfn + count, MIGRATE_CMA, gfp); + ret =3D alloc_contig_range(pfn, pfn + count, ACR_CMA, gfp); mutex_unlock(&cma->alloc_mutex); if (ret =3D=3D 0) { page =3D pfn_to_page(pfn); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index ab66acd3e6b3..19cad4460cee 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -2009,8 +2009,7 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, =20 /* set above range as isolated */ ret =3D start_isolate_page_range(start_pfn, end_pfn, - MIGRATE_MOVABLE, - MEMORY_OFFLINE | REPORT_FAILURE); + PB_ISOLATE_MODE_MEM_OFFLINE); if (ret) { reason =3D "failure to isolate range"; goto failed_removal_pcplists_disabled; @@ -2069,7 +2068,8 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, goto failed_removal_isolated; } =20 - ret =3D test_pages_isolated(start_pfn, end_pfn, MEMORY_OFFLINE); + ret =3D test_pages_isolated(start_pfn, end_pfn, + PB_ISOLATE_MODE_MEM_OFFLINE); =20 } while (ret); =20 diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a248faf30ee0..c28e5c2105e5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6697,11 +6697,12 @@ static void alloc_contig_dump_pages(struct list_hea= d *page_list) =20 /* * [start, end) must belong to a single zone. - * @migratetype: using migratetype to filter the type of migration in + * @alloc_flags: using acr_flags_t to filter the type of migration in * trace_mm_alloc_contig_migrate_range_info. */ static int __alloc_contig_migrate_range(struct compact_control *cc, - unsigned long start, unsigned long end, int migratetype) + unsigned long start, unsigned long end, + enum acr_flags_t alloc_flags) { /* This function is based on compact_zone() from compaction.c. */ unsigned int nr_reclaimed; @@ -6773,7 +6774,7 @@ static int __alloc_contig_migrate_range(struct compac= t_control *cc, putback_movable_pages(&cc->migratepages); } =20 - trace_mm_alloc_contig_migrate_range_info(start, end, migratetype, + trace_mm_alloc_contig_migrate_range_info(start, end, alloc_flags, total_migrated, total_reclaimed, total_mapped); @@ -6844,10 +6845,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_= mask, gfp_t *gfp_cc_mask) * alloc_contig_range() -- tries to allocate given range of pages * @start: start PFN to allocate * @end: one-past-the-last PFN to allocate - * @migratetype: migratetype of the underlying pageblocks (either - * #MIGRATE_MOVABLE or #MIGRATE_CMA). All pageblocks - * in range must have the same migratetype and it must - * be either of the two. + * @alloc_flags: allocation information * @gfp_mask: GFP mask. Node/zone/placement hints are ignored; only some * action and reclaim modifiers are supported. Reclaim modifiers * control allocation behavior during compaction/migration/reclaim. @@ -6864,7 +6862,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_m= ask, gfp_t *gfp_cc_mask) * need to be freed with free_contig_range(). */ int alloc_contig_range_noprof(unsigned long start, unsigned long end, - unsigned migratetype, gfp_t gfp_mask) + enum acr_flags_t alloc_flags, gfp_t gfp_mask) { unsigned long outer_start, outer_end; int ret =3D 0; @@ -6879,6 +6877,9 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, .alloc_contig =3D true, }; INIT_LIST_HEAD(&cc.migratepages); + enum pb_isolate_mode mode =3D (alloc_flags =3D=3D ACR_CMA) ? + PB_ISOLATE_MODE_CMA_ALLOC : + PB_ISOLATE_MODE_OTHER; =20 gfp_mask =3D current_gfp_context(gfp_mask); if (__alloc_contig_verify_gfp_mask(gfp_mask, (gfp_t *)&cc.gfp_mask)) @@ -6905,7 +6906,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, * put back to page allocator so that buddy can use them. */ =20 - ret =3D start_isolate_page_range(start, end, migratetype, 0); + ret =3D start_isolate_page_range(start, end, mode); if (ret) goto done; =20 @@ -6921,7 +6922,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, * allocated. So, if we fall through be sure to clear ret so that * -EBUSY is not accidentally used or returned to caller. */ - ret =3D __alloc_contig_migrate_range(&cc, start, end, migratetype); + ret =3D __alloc_contig_migrate_range(&cc, start, end, alloc_flags); if (ret && ret !=3D -EBUSY) goto done; =20 @@ -6955,7 +6956,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, outer_start =3D find_large_buddy(start); =20 /* Make sure the range is really isolated. */ - if (test_pages_isolated(outer_start, end, 0)) { + if (test_pages_isolated(outer_start, end, mode)) { ret =3D -EBUSY; goto done; } @@ -6998,8 +6999,8 @@ static int __alloc_contig_pages(unsigned long start_p= fn, { unsigned long end_pfn =3D start_pfn + nr_pages; =20 - return alloc_contig_range_noprof(start_pfn, end_pfn, MIGRATE_MOVABLE, - gfp_mask); + return alloc_contig_range_noprof(start_pfn, end_pfn, ACR_OTHER, + gfp_mask); } =20 static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn, diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 1edfef408faf..ece3bfc56bcd 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -31,7 +31,7 @@ * */ static struct page *has_unmovable_pages(unsigned long start_pfn, unsigned = long end_pfn, - int migratetype, int flags) + enum pb_isolate_mode mode) { struct page *page =3D pfn_to_page(start_pfn); struct zone *zone =3D page_zone(page); @@ -46,7 +46,7 @@ static struct page *has_unmovable_pages(unsigned long sta= rt_pfn, unsigned long e * isolate CMA pageblocks even when they are not movable in fact * so consider them movable here. */ - if (is_migrate_cma(migratetype)) + if (mode =3D=3D PB_ISOLATE_MODE_CMA_ALLOC) return NULL; =20 return page; @@ -117,7 +117,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * The HWPoisoned page may be not in buddy system, and * page_count() is not 0. */ - if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) + if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && PageHWPoison(page)) continue; =20 /* @@ -130,7 +130,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * move these pages that still have a reference count > 0. * (false negatives in this function only) */ - if ((flags & MEMORY_OFFLINE) && PageOffline(page)) + if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && PageOffline(page)) continue; =20 if (__PageMovable(page) || PageLRU(page)) @@ -151,7 +151,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * present in [start_pfn, end_pfn). The pageblock must intersect with * [start_pfn, end_pfn). */ -static int set_migratetype_isolate(struct page *page, int migratetype, int= isol_flags, +static int set_migratetype_isolate(struct page *page, enum pb_isolate_mode= mode, unsigned long start_pfn, unsigned long end_pfn) { struct zone *zone =3D page_zone(page); @@ -186,7 +186,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ end_pfn); =20 unmovable =3D has_unmovable_pages(check_unmovable_start, check_unmovable_= end, - migratetype, isol_flags); + mode); if (!unmovable) { if (!pageblock_isolate_and_move_free_pages(zone, page)) { spin_unlock_irqrestore(&zone->lock, flags); @@ -198,7 +198,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ } =20 spin_unlock_irqrestore(&zone->lock, flags); - if (isol_flags & REPORT_FAILURE) { + if (mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) { /* * printk() with zone->lock held will likely trigger a * lockdep splat, so defer it here. @@ -292,11 +292,10 @@ __first_valid_page(unsigned long pfn, unsigned long n= r_pages) * isolate_single_pageblock() -- tries to isolate a pageblock that might be * within a free or in-use page. * @boundary_pfn: pageblock-aligned pfn that a page might cross - * @flags: isolation flags + * @mode: isolation mode * @isolate_before: isolate the pageblock before the boundary_pfn * @skip_isolation: the flag to skip the pageblock isolation in second * isolate_single_pageblock() - * @migratetype: migrate type to set in error recovery. * * Free and in-use pages can be as big as MAX_PAGE_ORDER and contain more = than one * pageblock. When not all pageblocks within a page are isolated at the sa= me @@ -311,8 +310,9 @@ __first_valid_page(unsigned long pfn, unsigned long nr_= pages) * either. The function handles this by splitting the free page or migrati= ng * the in-use page then splitting the free page. */ -static int isolate_single_pageblock(unsigned long boundary_pfn, int flags, - bool isolate_before, bool skip_isolation, int migratetype) +static int isolate_single_pageblock(unsigned long boundary_pfn, + enum pb_isolate_mode mode, bool isolate_before, + bool skip_isolation) { unsigned long start_pfn; unsigned long isolate_pageblock; @@ -338,12 +338,11 @@ static int isolate_single_pageblock(unsigned long bou= ndary_pfn, int flags, zone->zone_start_pfn); =20 if (skip_isolation) { - int mt __maybe_unused =3D get_pageblock_migratetype(pfn_to_page(isolate_= pageblock)); - - VM_BUG_ON(!is_migrate_isolate(mt)); + VM_BUG_ON(!get_pageblock_isolate(pfn_to_page(isolate_pageblock))); } else { - ret =3D set_migratetype_isolate(pfn_to_page(isolate_pageblock), migratet= ype, - flags, isolate_pageblock, isolate_pageblock + pageblock_nr_pages); + ret =3D set_migratetype_isolate(pfn_to_page(isolate_pageblock), + mode, isolate_pageblock, + isolate_pageblock + pageblock_nr_pages); =20 if (ret) return ret; @@ -441,14 +440,7 @@ static int isolate_single_pageblock(unsigned long boun= dary_pfn, int flags, * start_isolate_page_range() - mark page range MIGRATE_ISOLATE * @start_pfn: The first PFN of the range to be isolated. * @end_pfn: The last PFN of the range to be isolated. - * @migratetype: Migrate type to set in error recovery. - * @flags: The following flags are allowed (they can be combined in - * a bit mask) - * MEMORY_OFFLINE - isolate to offline (!allocate) memory - * e.g., skip over PageHWPoison() pages - * and PageOffline() pages. - * REPORT_FAILURE - report details about the failure to - * isolate the range + * @mode: isolation mode * * Making page-allocation-type to be MIGRATE_ISOLATE means free pages in * the range will never be allocated. Any free pages and pages freed in the @@ -481,7 +473,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, * Return: 0 on success and -EBUSY if any part of range cannot be isolated. */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype, int flags) + enum pb_isolate_mode mode) { unsigned long pfn; struct page *page; @@ -492,8 +484,8 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, bool skip_isolation =3D false; =20 /* isolate [isolate_start, isolate_start + pageblock_nr_pages) pageblock = */ - ret =3D isolate_single_pageblock(isolate_start, flags, false, - skip_isolation, migratetype); + ret =3D isolate_single_pageblock(isolate_start, mode, false, + skip_isolation); if (ret) return ret; =20 @@ -501,8 +493,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, skip_isolation =3D true; =20 /* isolate [isolate_end - pageblock_nr_pages, isolate_end) pageblock */ - ret =3D isolate_single_pageblock(isolate_end, flags, true, - skip_isolation, migratetype); + ret =3D isolate_single_pageblock(isolate_end, mode, true, skip_isolation); if (ret) { unset_migratetype_isolate(pfn_to_page(isolate_start)); return ret; @@ -513,8 +504,8 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, pfn < isolate_end - pageblock_nr_pages; pfn +=3D pageblock_nr_pages) { page =3D __first_valid_page(pfn, pageblock_nr_pages); - if (page && set_migratetype_isolate(page, migratetype, flags, - start_pfn, end_pfn)) { + if (page && set_migratetype_isolate(page, mode, start_pfn, + end_pfn)) { undo_isolate_page_range(isolate_start, pfn); unset_migratetype_isolate( pfn_to_page(isolate_end - pageblock_nr_pages)); @@ -556,7 +547,7 @@ void undo_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn) */ static unsigned long __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn, - int flags) + enum pb_isolate_mode mode) { struct page *page; =20 @@ -569,11 +560,12 @@ __test_page_isolated_in_pageblock(unsigned long pfn, = unsigned long end_pfn, * simple way to verify that as VM_BUG_ON(), though. */ pfn +=3D 1 << buddy_order(page); - else if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) + else if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && + PageHWPoison(page)) /* A HWPoisoned page cannot be also PageBuddy */ pfn++; - else if ((flags & MEMORY_OFFLINE) && PageOffline(page) && - !page_count(page)) + else if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && + PageOffline(page) && !page_count(page)) /* * The responsible driver agreed to skip PageOffline() * pages when offlining memory by dropping its @@ -591,11 +583,11 @@ __test_page_isolated_in_pageblock(unsigned long pfn, = unsigned long end_pfn, * test_pages_isolated - check if pageblocks in range are isolated * @start_pfn: The first PFN of the isolated range * @end_pfn: The first PFN *after* the isolated range - * @isol_flags: Testing mode flags + * @mode: Testing mode * * This tests if all in the specified range are free. * - * If %MEMORY_OFFLINE is specified in @flags, it will consider + * If %PB_ISOLATE_MODE_MEM_OFFLINE specified in @mode, it will consider * poisoned and offlined pages free as well. * * Caller must ensure the requested range doesn't span zones. @@ -603,7 +595,7 @@ __test_page_isolated_in_pageblock(unsigned long pfn, un= signed long end_pfn, * Returns 0 if true, -EBUSY if one or more pages are in use. */ int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, - int isol_flags) + enum pb_isolate_mode mode) { unsigned long pfn, flags; struct page *page; @@ -639,7 +631,7 @@ int test_pages_isolated(unsigned long start_pfn, unsign= ed long end_pfn, /* Check all pages are free or marked as ISOLATED */ zone =3D page_zone(page); spin_lock_irqsave(&zone->lock, flags); - pfn =3D __test_page_isolated_in_pageblock(start_pfn, end_pfn, isol_flags); + pfn =3D __test_page_isolated_in_pageblock(start_pfn, end_pfn, mode); spin_unlock_irqrestore(&zone->lock, flags); =20 ret =3D pfn < end_pfn ? -EBUSY : 0; --=20 2.47.2