From nobody Thu Oct 9 20:24:37 2025 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2051.outbound.protection.outlook.com [40.107.237.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA40A285C9B for ; Mon, 16 Jun 2025 12:10:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075832; cv=fail; b=uCBsIqUKYJQn+JFjlJDDM2TeKm4tgkM0zlWvQiQzttPXFtWaE58OU7xRrf6CC8x0RLflzRVBSFE4fFlW5VdmBNDTxZ3ZdEuYXTBf1JXWNZXZpbaHzsa3zL/gKveH+cuf50IQBXJOqzlKykMBZUtIbRk/bADBTAS67CD/ZSIoPSo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075832; c=relaxed/simple; bh=N7UO7wHQXxuCm5PacS6BHJrNgpM2js1V5V2lHd94PiM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=FOxluhdoSiMfrKENVGneBQUtfCIqw55OvBIMWWza3W6a82h4KBv3UqQ7caBlWJurMCZD3elJ8NA+PGF1b4ZXY0O3WX5RNjMoqW+LS8P8s35I35j/JfVCBzegheEEs2ynK3wzExjUH9L01fvtpTljlgqG9p/59OCTJoOW7EGMMA0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=loEnV0Q6; arc=fail smtp.client-ip=40.107.237.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="loEnV0Q6" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Z5TmCDawQgqBHgYy8msBvVC7lp5S37qNWYIwWdpmW2WmsQ3c+/QdcV/4Ve6SgNmlcgWCgmavT2Qm8g1C3tFKy7Tf6+YXdmpXTWMNQqYSW3YOnvyo7QqRNCW1P7Szd9JfpcZxYzn3vC++Cf4hMALnKSNMMHSPgKpQPzsdTVbjUKaGQOWmKBohqoLMS5tyRH0j/nPh13800tfwHuyYHmXpD7DXfhLDlFe0cQ0h3bongzYNdEKyrBwP2fAJGgTXjJeut29GIzsmLTGeUYH6ZaeErq4+UUyyICy33t1wWKWT/0q0GMEnrf0iXlM7xp8DRS9C6Mq9A/6ewxiT1G5RGrKM6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wE8sLxzQaz8zGgNGZIBWr6Ap7mgkheIObVmJrbM19qM=; b=kTHcpnX8FxFGf5KU+ltZnk6eXsmXetd6VuFDh5+wEeRiXYpJCmJwNOlzDG3ZdwzbPBZRfF2+mqQWuKgP+GeThURaM8h7ab9vnoj76sSjjhR4qTTEIxJUtzPrNyasVmtvKAN58Yh2aDHmpvSSAejQOwWgDtkAVHutM6fx+Mh6hwbOWWUVt/I0ANomLMoLDJOwJNncBCDN/YBgZyRT1ldEL0Une1IxzoLerjb+jNi1vOeV2QXjxMVPe70Yv310l7KJrflfr1dYTf67VYwdneHJybnZCJ6dExtkpWuzXfrmoIxCP8xymh9+Ut22qSf3us1mPBCLj7TYPM5poAoZBx2kMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wE8sLxzQaz8zGgNGZIBWr6Ap7mgkheIObVmJrbM19qM=; b=loEnV0Q6nQNjjuD7coUxJKgsv/81QzQSlYsqVCPT4jyyuSumHgjZ8uPrjnw4XnCBbzm3wHOZ+s2XsNHxiF+lBrhlkDwEGe2euKZCV9sU2iD2TD+8KrlqRj1ViipprK1Fql2FUv+/9cLxMCWrJxUT2U5F3tG5lZiS3eKZL+lBLysPJeq/X8a8VTH0pcX/ZvRsSLNjMR4PxOrBQihzAxPn8W4wehJGAEwE8yqvVJ9iQVHWNmdZpfQfYAxZlbKRVvwKXHVy2dyaysJ4YMs9Yuhhgw4Z3l1zVDbK5SPxSLSYmKpLP0vc4cIZvsWyJChlQRQ04NlDuKw+iz6nBWy5mHRhlg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY1PR12MB9560.namprd12.prod.outlook.com (2603:10b6:930:fd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.29; Mon, 16 Jun 2025 12:10:26 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8835.027; Mon, 16 Jun 2025 12:10:26 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Johannes Weiner , Vlastimil Babka , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v9 1/6] mm/page_alloc: pageblock flags functions clean up. Date: Mon, 16 Jun 2025 08:10:14 -0400 Message-ID: <20250616121019.1925851-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250616121019.1925851-1-ziy@nvidia.com> References: <20250616121019.1925851-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR07CA0017.namprd07.prod.outlook.com (2603:10b6:208:1a0::27) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY1PR12MB9560:EE_ X-MS-Office365-Filtering-Correlation-Id: 8249c89d-6892-4534-5b86-08ddaccec6f8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?58Pe9qBnmKmDNVYQ8l/uUqU9k81aJD9QonwoQaXGHVCCPbaKhr/LUnDJif1+?= =?us-ascii?Q?sxirNVJYH7IfC6GlOAwvtPUSRcGB+a3tZUL2geeDbEL1MgAzYUqkfdlHcTBD?= =?us-ascii?Q?vkOHrva5bWWENc13qX+AQdZXj/hdiYC33FVlfTZW++RxZZ80kH7qCKZF/bk4?= =?us-ascii?Q?JZ6r6jqecOJzdtzqAF8gRmDdG3Z9+fEoiQ4gJnzAVilikkc+qtamIu3iQI9n?= =?us-ascii?Q?VLovPK/CapOZ7b91LQoZpjf2r6hA65lAElCCPjtJ7y0iQPGYyxqsNgKmsoJ7?= =?us-ascii?Q?k7diYRncSwHnFbbapshxV9MPT3/lHx8IR41Dh0a1m6OUP7MKl6Jv42J9eNZL?= =?us-ascii?Q?jqMXTMwTicgtDT1aiJz3Go+2m0v6ibMFkopQE97zZG8XVHPdnrRmuSEw2TVD?= =?us-ascii?Q?t77mlceRWMTTIku16j+k8fvyCLhQpmIBUfngJGOCPkmir0GRs6qphrffQWVP?= =?us-ascii?Q?ke6SZSt2tqvmjGAuD+euJcy5fuO34IUB162sydpEZVd6ZyjTalBYaZtZ42ng?= =?us-ascii?Q?bRQXOIZ90++GoxCLRDlid1ZL13ICTmO/nK1TqW4owd6H8ItFEAGJqTNKbGgR?= =?us-ascii?Q?CtzpneTSHSg5oTp0gS9m8RIz8izfmP7c6Ej/PpH+1uQIzS+X6dexfkFANXnR?= =?us-ascii?Q?CoA4Ad24HXSn+dR5twu2qI28dgESJYs6B56drTSrsmYLnZUZ1waIYHHLG6EO?= =?us-ascii?Q?2cpff8/Zk1ucIn08z8b/metu7IpQYrFTkwERzRy/is+C8iamAjzDtcluG7Ry?= =?us-ascii?Q?YVT1KSjoMHOQanDVZk62EUGNYp9VizP2NzPTmd53nHIc7euQj2ru3T7SR1M3?= =?us-ascii?Q?Q4dmrOIPTA4goLNn4nJDRi3HS7cHF581gTOagW0UVdhA3oJMCsWd6fB77r9i?= =?us-ascii?Q?Oif1Mj38PfbV7D9oheV14mgLzW5H6SQFeIsEsln8alLCVCnKMCuSHW2zkjGa?= =?us-ascii?Q?tBBFKWaTJDSXrXfIJzRnxle/WLhTzI7eNtcH+FoD0VyKnCDVKI7q4IPkmzH2?= =?us-ascii?Q?HkvbazMxQwldFxXsKLdxXtGJcsDq1hoVgWpn85zsu9u5eB20GsN7NzMiCd5g?= =?us-ascii?Q?W+DC8c7fJNLAz1O8Tr/fs0gJNekV1gXBDba3ao/BzpTZNr9oshxxjY5+HbFb?= =?us-ascii?Q?8xNqx4MV9SlZNfgB+Gj0Iui+Xje2Z17dNkVTkU98YQOj/Tv94w9/YgOwLmzI?= =?us-ascii?Q?HOGVwBEVpX6TwOGTcmmb+NRAqled1l1X+p7qf4fMa7QIykijngxwuvQdqJHk?= =?us-ascii?Q?MlJRDmoom0QD/YBKDZ3oemQE09YjvDf4pUF2mQwCu8By24mWbAxtgmFvyAdX?= =?us-ascii?Q?PQuAUXCDXIrXrU6cv3YdT+yrDWxne6ohis5hyFNEvbCTy3Pd7Z7CK8DM+eYc?= =?us-ascii?Q?U/tb+MugHnNWfiTxmE/Jal9ZVL6n6gu9d+muD7ONs0VTf3/TX/ODMw7h8Bao?= =?us-ascii?Q?CW/l/fSt5dU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?fHGuKXsf4f6LJDHwaGkalJ8pZmaL4/IcqxUlfKSyHCTk1N5myJaHJg8/goF4?= =?us-ascii?Q?U8Iu2tKm/l0B8gVDi3KJ5zZPcZfnmNYv1pFeb/TQLmmKyFTfcalCshKWh1aD?= =?us-ascii?Q?vktBQjpdV8hrJs2l3WCKVzF1rKapBYl2dcMhpDhGqZ46VbzwtJr2/t7wBb2N?= =?us-ascii?Q?S9pbsSvvqXcB1Ne64Gd393Ex8D+QcRQC3aXXdwvlxg78cjE5Th+8JO2Yk2F4?= =?us-ascii?Q?TQFWdyF1Uv1/9/CHYHOtkXfWz0XVnugcXB/9LxV8AcNLVRn9wBA+bPGN1Jp+?= =?us-ascii?Q?1f0xniBjxsNZYSY4hMlu3V7UwJ4oZSbOCHCNDgcotKdsPESUwqnVXHLVrLlu?= =?us-ascii?Q?whipN3CdyFkt7P+Jnni26v0Meh8GXskp5OmGN9xHbkHypG/L6lzopce3kyru?= =?us-ascii?Q?WOk5fMLIW/e8HTrB3zwNHzQ4ewz8nW257xbjdPN6U5SK2ZqONESVm4GpKFZy?= =?us-ascii?Q?J1tkpruKmfxVlPAbdSa30h+NHJRo1u6jPY41758/a35gZgDXAhfa0Ujydh5j?= =?us-ascii?Q?57qZKJCkHpXCPVAOSuaBQt3U+/qxmip11W7sWONcXn8zmI8OkkNHQcmxoLWz?= =?us-ascii?Q?Ndi2SJmx1m72BvkXuYWSt9dw032a2Hg8Wf6VgJsQKr0x2Nfd3RPArrpBHgq6?= =?us-ascii?Q?nB7gufacj0xX/8SOD4zPQ1HiwMiT0J8bd+Y/rlQdbZ0UfLlYu/2Yr7Lfj4BX?= =?us-ascii?Q?NWgUFDnQq2dGgrsopkmj17VfsP5qEpIGqbGREzz6dMmNJmypd1Vt1hxm1OJt?= =?us-ascii?Q?Y/kJ6F4F2dfLXbrrWh5YuNNZ9yZpdjuACMW6/qtUaKdlWAGF6ymxxlASeAui?= =?us-ascii?Q?RFVzzUv4IXK2BH8eslkydATP+KxmfjpYmzwqBHl4G4Rv4ro0syX5+FQWd4+V?= =?us-ascii?Q?M2VpPAosRUZWOW9JAbi14vW6bcEgu1quYGAXg4US5kNPgf6yHT9RTyTCY9pG?= =?us-ascii?Q?bVT9Zr5LuLyoKqmVDMCkmv1+QceFZ9wS/4lbFTe71boGqHoOLoreAaQOClyi?= =?us-ascii?Q?FHxqpkh7D5wlHlpXeqeOL2l2aAep8JsUCMvOHgAnmIKEKn04lpGoK5aKjpHT?= =?us-ascii?Q?pSwIR5D2N/iApEK+cQR1y3G/0D4IZrRtrLn52dqbOr9EOkDbxViE+qyQMSn7?= =?us-ascii?Q?UVKAMK4S9tcExKROJDzepFeEREqyL/FMnjVpaVXZMaeXR9BhCJHjk31q9z0B?= =?us-ascii?Q?NRYa7QfGvCy5VmUVXHsCzb1Wm7Addr+IcDJ/G2jeNwQmULSYMUm6HiCbz32d?= =?us-ascii?Q?g32ZYkdLa58eUHfZrCkV9lWcvM9hqledhB+936eZzZbF3Gm/Phwi1nIR77tP?= =?us-ascii?Q?fPriMr+LZQNOf5QTCrd/y3hjRzH5jh3XW6t93W0ATGJEvKTa6zkFIzXLny15?= =?us-ascii?Q?jUjm7cnqPmWcmeROOflye/eiugoWHgBhcv7dKI5SGplAe0hHqgrLcqkV2NFu?= =?us-ascii?Q?/SRO7kj52massK6Zl2Wx8fXYQtLOEcdqTiaGrhtD5ZRATc/gDX7GsCSf4s83?= =?us-ascii?Q?3Y/delVG1HzLdsCigTWHkZIX+t7JJsTSHoghhOYG4RfEcpn8UmdfDgy8ffK4?= =?us-ascii?Q?JJ5+P1wp0ZG7hGW48/p8+ffz0SXAfIXnh0q0F4Q+?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8249c89d-6892-4534-5b86-08ddaccec6f8 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2025 12:10:26.0853 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +Y/vAvEFfMZpVMhQhqzyuF9p+s6Qu7lnZjL21wiQ/rOWFhrnnuLuOxr3SF7on1uH X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB9560 Content-Type: text/plain; charset="utf-8" No functional change is intended. 1. Add __NR_PAGEBLOCK_BITS for the number of pageblock flag bits and use roundup_pow_of_two(__NR_PAGEBLOCK_BITS) as NR_PAGEBLOCK_BITS to take right amount of bits for pageblock flags. 2. Rename PB_migrate_skip to PB_compact_skip. 3. Add {get,set,clear}_pfnblock_bit() to operate one a standalone bit, like PB_compact_skip. 3. Make {get,set}_pfnblock_flags_mask() internal functions and use {get,set}_pfnblock_migratetype() for pageblock migratetype operations. 4. Move pageblock flags common code to get_pfnblock_bitmap_bitidx(). 3. Use MIGRATETYPE_MASK to get the migratetype of a pageblock from its flags. 4. Use PB_migrate_end in the definition of MIGRATETYPE_MASK instead of PB_migrate_bits. 5. Add a comment on is_migrate_cma_folio() to prevent one from changing it to use get_pageblock_migratetype() and causing issues. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- Documentation/mm/physical_memory.rst | 2 +- include/linux/mmzone.h | 18 +-- include/linux/page-isolation.h | 2 +- include/linux/pageblock-flags.h | 34 +++--- mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 171 +++++++++++++++++++++------ 6 files changed, 162 insertions(+), 67 deletions(-) diff --git a/Documentation/mm/physical_memory.rst b/Documentation/mm/physic= al_memory.rst index d3ac106e6b14..9af11b5bd145 100644 --- a/Documentation/mm/physical_memory.rst +++ b/Documentation/mm/physical_memory.rst @@ -584,7 +584,7 @@ Compaction control =20 ``compact_blockskip_flush`` Set to true when compaction migration scanner and free scanner meet, whi= ch - means the ``PB_migrate_skip`` bits should be cleared. + means the ``PB_compact_skip`` bits should be cleared. =20 ``contiguous`` Set to true when the zone is contiguous (in other words, no hole). diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 5bec8b1d0e66..76d66c07b673 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -92,8 +92,12 @@ extern const char * const migratetype_names[MIGRATE_TYPE= S]; #ifdef CONFIG_CMA # define is_migrate_cma(migratetype) unlikely((migratetype) =3D=3D MIGRAT= E_CMA) # define is_migrate_cma_page(_page) (get_pageblock_migratetype(_page) =3D= =3D MIGRATE_CMA) -# define is_migrate_cma_folio(folio, pfn) (MIGRATE_CMA =3D=3D \ - get_pfnblock_flags_mask(&folio->page, pfn, MIGRATETYPE_MASK)) +/* + * __dump_folio() in mm/debug.c passes a folio pointer to on-stack struct = folio, + * so folio_pfn() cannot be used and pfn is needed. + */ +# define is_migrate_cma_folio(folio, pfn) \ + (get_pfnblock_migratetype(&folio->page, pfn) =3D=3D MIGRATE_CMA) #else # define is_migrate_cma(migratetype) false # define is_migrate_cma_page(_page) false @@ -122,14 +126,12 @@ static inline bool migratetype_is_mergeable(int mt) =20 extern int page_group_by_mobility_disabled; =20 -#define MIGRATETYPE_MASK ((1UL << PB_migratetype_bits) - 1) +#define get_pageblock_migratetype(page) \ + get_pfnblock_migratetype(page, page_to_pfn(page)) =20 -#define get_pageblock_migratetype(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), MIGRATETYPE_MASK) +#define folio_migratetype(folio) \ + get_pageblock_migratetype(&folio->page) =20 -#define folio_migratetype(folio) \ - get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ - MIGRATETYPE_MASK) struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 898bb788243b..277d8d92980c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -25,7 +25,7 @@ static inline bool is_migrate_isolate(int migratetype) #define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 =20 -void set_pageblock_migratetype(struct page *page, int migratetype); +void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 bool move_freepages_block_isolate(struct zone *zone, struct page *page, int migratetype); diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flag= s.h index 6297c6343c55..c240c7a1fb03 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -19,15 +19,19 @@ enum pageblock_bits { PB_migrate, PB_migrate_end =3D PB_migrate + PB_migratetype_bits - 1, /* 3 bits required for migrate types */ - PB_migrate_skip,/* If set the block is skipped by compaction */ + PB_compact_skip,/* If set the block is skipped by compaction */ =20 /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. */ - NR_PAGEBLOCK_BITS + __NR_PAGEBLOCK_BITS }; =20 +#define NR_PAGEBLOCK_BITS (roundup_pow_of_two(__NR_PAGEBLOCK_BITS)) + +#define MIGRATETYPE_MASK ((1UL << (PB_migrate_end + 1)) - 1) + #if defined(CONFIG_HUGETLB_PAGE) =20 #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -65,27 +69,23 @@ extern unsigned int pageblock_order; /* Forward declaration */ struct page; =20 -unsigned long get_pfnblock_flags_mask(const struct page *page, - unsigned long pfn, - unsigned long mask); - -void set_pfnblock_flags_mask(struct page *page, - unsigned long flags, - unsigned long pfn, - unsigned long mask); +enum migratetype get_pfnblock_migratetype(const struct page *page, + unsigned long pfn); +bool get_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); +void set_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); +void clear_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); =20 /* Declarations for getting and setting flags. See mm/page_alloc.c */ #ifdef CONFIG_COMPACTION #define get_pageblock_skip(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), \ - (1 << (PB_migrate_skip))) + get_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #define clear_pageblock_skip(page) \ - set_pfnblock_flags_mask(page, 0, page_to_pfn(page), \ - (1 << PB_migrate_skip)) + clear_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #define set_pageblock_skip(page) \ - set_pfnblock_flags_mask(page, (1 << PB_migrate_skip), \ - page_to_pfn(page), \ - (1 << PB_migrate_skip)) + set_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #else static inline bool get_pageblock_skip(struct page *page) { diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index dd1c4332347c..ddc6c6c63a30 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -774,7 +774,7 @@ void move_pfn_range_to_zone(struct zone *zone, unsigned= long start_pfn, =20 /* * TODO now we have a visible range of pages which are not associated - * with their zone properly. Not nice but set_pfnblock_flags_mask + * with their zone properly. Not nice but set_pfnblock_migratetype() * expects the zone spans the pfn range. All the pages in the range * are reserved so nobody should be touching them so we should be safe */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1d46d0fb1f61..b303f60b6ed1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -353,81 +353,174 @@ static inline int pfn_to_bitidx(const struct page *p= age, unsigned long pfn) return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS; } =20 +static __always_inline bool is_standalone_pb_bit(enum pageblock_bits pb_bi= t) +{ + return pb_bit > PB_migrate_end && pb_bit < __NR_PAGEBLOCK_BITS; +} + +static __always_inline void +get_pfnblock_bitmap_bitidx(const struct page *page, unsigned long pfn, + unsigned long **bitmap_word, unsigned long *bitidx) +{ + unsigned long *bitmap; + unsigned long word_bitidx; + + BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); + BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); + VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); + + bitmap =3D get_pageblock_bitmap(page, pfn); + *bitidx =3D pfn_to_bitidx(page, pfn); + word_bitidx =3D *bitidx / BITS_PER_LONG; + *bitidx &=3D (BITS_PER_LONG - 1); + *bitmap_word =3D &bitmap[word_bitidx]; +} + + /** - * get_pfnblock_flags_mask - Return the requested group of flags for the p= ageblock_nr_pages block of pages + * __get_pfnblock_flags_mask - Return the requested group of flags for + * a pageblock_nr_pages block of pages * @page: The page within the block of interest * @pfn: The target page frame number * @mask: mask of bits that the caller is interested in * * Return: pageblock_bits flags */ -unsigned long get_pfnblock_flags_mask(const struct page *page, - unsigned long pfn, unsigned long mask) +static unsigned long __get_pfnblock_flags_mask(const struct page *page, + unsigned long pfn, + unsigned long mask) { - unsigned long *bitmap; - unsigned long bitidx, word_bitidx; + unsigned long *bitmap_word; + unsigned long bitidx; unsigned long word; =20 - bitmap =3D get_pageblock_bitmap(page, pfn); - bitidx =3D pfn_to_bitidx(page, pfn); - word_bitidx =3D bitidx / BITS_PER_LONG; - bitidx &=3D (BITS_PER_LONG-1); + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); /* - * This races, without locks, with set_pfnblock_flags_mask(). Ensure + * This races, without locks, with set_pfnblock_migratetype(). Ensure * a consistent read of the memory array, so that results, even though * racy, are not corrupted. */ - word =3D READ_ONCE(bitmap[word_bitidx]); + word =3D READ_ONCE(*bitmap_word); return (word >> bitidx) & mask; } =20 -static __always_inline int get_pfnblock_migratetype(const struct page *pag= e, - unsigned long pfn) +/** + * get_pfnblock_bit - Check if a standalone bit of a pageblock is set + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to check + * + * Return: true if the bit is set, otherwise false + */ +bool get_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) { - return get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return false; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + return test_bit(bitidx + pb_bit, bitmap_word); } =20 /** - * set_pfnblock_flags_mask - Set the requested group of flags for a pagebl= ock_nr_pages block of pages + * get_pfnblock_migratetype - Return the migratetype of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * + * Return: The migratetype of the pageblock + * + * Use get_pfnblock_migratetype() if caller already has both @page and @pfn + * to save a call to page_to_pfn(). + */ +__always_inline enum migratetype +get_pfnblock_migratetype(const struct page *page, unsigned long pfn) +{ + return __get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); +} + +/** + * __set_pfnblock_flags_mask - Set the requested group of flags for + * a pageblock_nr_pages block of pages * @page: The page within the block of interest - * @flags: The flags to set * @pfn: The target page frame number + * @flags: The flags to set * @mask: mask of bits that the caller is interested in */ -void set_pfnblock_flags_mask(struct page *page, unsigned long flags, - unsigned long pfn, - unsigned long mask) +static void __set_pfnblock_flags_mask(struct page *page, unsigned long pfn, + unsigned long flags, unsigned long mask) { - unsigned long *bitmap; - unsigned long bitidx, word_bitidx; + unsigned long *bitmap_word; + unsigned long bitidx; unsigned long word; =20 - BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); - BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); - - bitmap =3D get_pageblock_bitmap(page, pfn); - bitidx =3D pfn_to_bitidx(page, pfn); - word_bitidx =3D bitidx / BITS_PER_LONG; - bitidx &=3D (BITS_PER_LONG-1); - - VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); =20 mask <<=3D bitidx; flags <<=3D bitidx; =20 - word =3D READ_ONCE(bitmap[word_bitidx]); + word =3D READ_ONCE(*bitmap_word); do { - } while (!try_cmpxchg(&bitmap[word_bitidx], &word, (word & ~mask) | flags= )); + } while (!try_cmpxchg(bitmap_word, &word, (word & ~mask) | flags)); +} + +/** + * set_pfnblock_bit - Set a standalone bit of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to set + */ +void set_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) +{ + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + set_bit(bitidx + pb_bit, bitmap_word); } =20 -void set_pageblock_migratetype(struct page *page, int migratetype) +/** + * clear_pfnblock_bit - Clear a standalone bit of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to clear + */ +void clear_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) +{ + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + clear_bit(bitidx + pb_bit, bitmap_word); +} + +/** + * set_pageblock_migratetype - Set the migratetype of a pageblock + * @page: The page within the block of interest + * @migratetype: migratetype to set + */ +__always_inline void set_pageblock_migratetype(struct page *page, + enum migratetype migratetype) { if (unlikely(page_group_by_mobility_disabled && migratetype < MIGRATE_PCPTYPES)) migratetype =3D MIGRATE_UNMOVABLE; =20 - set_pfnblock_flags_mask(page, (unsigned long)migratetype, - page_to_pfn(page), MIGRATETYPE_MASK); + __set_pfnblock_flags_mask(page, page_to_pfn(page), + (unsigned long)migratetype, MIGRATETYPE_MASK); } =20 #ifdef CONFIG_DEBUG_VM @@ -667,7 +760,7 @@ static inline void __add_to_free_list(struct page *page= , struct zone *zone, int nr_pages =3D 1 << order; =20 VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D migratetype, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), migratetype, nr_pages); =20 if (tail) @@ -693,7 +786,7 @@ static inline void move_to_free_list(struct page *page,= struct zone *zone, =20 /* Free page moving can fail, so it happens before the type update */ VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D old_mt, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), old_mt, nr_pages); =20 list_move_tail(&page->buddy_list, &area->free_list[new_mt]); @@ -715,7 +808,7 @@ static inline void __del_page_from_free_list(struct pag= e *page, struct zone *zon int nr_pages =3D 1 << order; =20 VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D migratetype, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), migratetype, nr_pages); =20 /* clear reported state and update reported page count */ @@ -3123,7 +3216,7 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, =20 /* * Do not instrument rmqueue() with KMSAN. This function may call - * __msan_poison_alloca() through a call to set_pfnblock_flags_mask(). + * __msan_poison_alloca() through a call to set_pfnblock_migratetype(). * If __msan_poison_alloca() attempts to allocate pages for the stack depo= t, it * may call rmqueue() again, which will result in a deadlock. */ --=20 2.47.2 From nobody Thu Oct 9 20:24:37 2025 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2051.outbound.protection.outlook.com [40.107.237.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B98A285C9F for ; Mon, 16 Jun 2025 12:10:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075834; cv=fail; b=C1Xrti+5eGYELRyAB3OZ5nrCOqE+fkSO+GFrdLEGZdRGFmot57yCloZiA5vGNWBMtmoNM8Oml0O3SpP5+tM41y2RaOZizVaguw/+XjquOEcPEB5Xo1pzSLWUn+g+Oi/SCr8lfHg30GXg7vAuGV2VvrwW6wBZYu1sCGUMbteQb3Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075834; c=relaxed/simple; bh=xMNOBFptsatJunn4GVsHm8dUqDAS7dh3Q2xY9FmORKs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ngQDyL6JnPqqryNiZNjKvHl4BbPVQuA49jzir9hg9vVmRoPUsK7mJ+8U/1vDWsEkEQwYu4RlHSgwh5wpbrtCn6p8jWQPPFF25SDGARetN6E8/7rgLoQlzKCkLYR05eijVSZWTp4FMGjHsMi5XMbsr5m/hGvO4ZP4aCUJ4OPciiU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=FSMgsS+o; arc=fail smtp.client-ip=40.107.237.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="FSMgsS+o" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ULTkU+FFDfryWARvvKNPYjEMprx0/13gtVt6cLTspg4c/lWbxEOxKoCIuxlr5KC7yyHWbPqV5mCfidNOy/4FOLJNdiSurOSdQxoCMZOaawCimxcxn2kaWrY9riUbSBsMXVH/ef+zGr2QlOS5hNzDAB9hQKJuYc7P0BimLZYl1d/YJ/fbAgxJsUCMc3ejHmuu5viz5A2Jxhv2/Ky4JKBAkyzsqOeIhVVDWkWIaUXdg0BPIjlgB+GjTgs8FETQwbr+gBKuIa0tF0xVYRxkITDGmwBdc3q4U74eS2kSXB0tbglS3zIUBILlX8pWfeEpzxCQaeSrCNcXEXcy7G/ynKSk+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3aWlbvxfbraMWybz7/Pfg4gx+wiq3tJBlLsue4a5SvI=; b=WKLkdo9cN4GsHf5Pxi+5Azfm3ZP3wDD/Pe9zBRkEjg/OK4pUsuB7jdAjzkmz30VVCg9WzCXfIWA489aa0BxPf0x6FnpfbRAukrXpoRTYKwo+Jsb6s8FjwPbLmhLYgY8F0/j7ZYtxM0Y1ebeCfUDkbXXgJ8q0aeKaoMAmq1GkhjAplxeYOonZWE0nkdKwEr41jYcHUSBFeGi62+97AgX2tdafHytShhLwhpfHbl/t6EQcqFAYvzCU0Nng/lueUEgQ/ebTYNaVlkco1svW2Oj6lTvWXKGyl0KKF9tkVe1PRttJfYkjBiIIzZB00ZvnM+3LCb1AP9i+iYSBptRlOAycyQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3aWlbvxfbraMWybz7/Pfg4gx+wiq3tJBlLsue4a5SvI=; b=FSMgsS+olqZTyrJiikt1k6KdLp0FoKTbj87KGytJE8K1JzFfjGBmbALtQY1RASuPYdN28mFYZ54ysZqVyM4O0yaKc0P8PBwLjqR5YklrUDKiw/b+e+LTGBHad9e01nJlZD0Z/texCpOLOjth3p45pbzRtOEZh+0aZu4jVD6QfkmVlE0zfzk6HrO0YXp/dijQT0xwkHvZdLM1QBAxi/gotdKwuaGZQTBj8VM2uhNHPa1TKpPRAQEiSY53xeeNzb7jctJ4Pp9Ysi738CDQd8lAp76pk0JtUWz4l+mVcbTwkNad983wjuBa4bWRZRZ7YLS04/uyz6HrwPqD8+rzzIsLpg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY1PR12MB9560.namprd12.prod.outlook.com (2603:10b6:930:fd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.29; Mon, 16 Jun 2025 12:10:27 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8835.027; Mon, 16 Jun 2025 12:10:27 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Johannes Weiner , Vlastimil Babka , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v9 2/6] mm/page_isolation: make page isolation a standalone bit. Date: Mon, 16 Jun 2025 08:10:15 -0400 Message-ID: <20250616121019.1925851-3-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250616121019.1925851-1-ziy@nvidia.com> References: <20250616121019.1925851-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN0PR10CA0004.namprd10.prod.outlook.com (2603:10b6:408:143::18) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY1PR12MB9560:EE_ X-MS-Office365-Filtering-Correlation-Id: d320346b-d1a5-4ecf-9613-08ddaccec7f6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?CIGYqcKruEyT58my7RSOE8FVc+og/EJQFvDQMhFm4SQfL8nzYI7h77RypRQ6?= =?us-ascii?Q?V3lz+vbcvtpRIVa+aWTYQHOPQKp4tgU3eTJntIyoh1hjpwKKIcWIxsRJpDdw?= =?us-ascii?Q?6C38yvY30j1OMGPiOdMOkWXKt3jM2xZ8ixLdDYFtSHoRj0dMTrCV+qPfqe0T?= =?us-ascii?Q?OfgdsN2K81Ptcic/MR3Jj9tdwq+UwH2SA4lRmIQ8UnCZoFy/jj7MehsMaoP0?= =?us-ascii?Q?kqd/RSnsKLwkZkphwWr8wWPW1/s9oiGYzNtNxUPbkR6y5O3wmUWbFf2jV+eu?= =?us-ascii?Q?wLKS4c5RXKi5psoZNq667UbbtGk2izUub7z+bEKGj4HwkOMFqe0Kk5BHOvKB?= =?us-ascii?Q?Lde5JZ+Du7XBzCXt9Nt60+rCb9T3V0bwrmqBeaaFDmsdqLaDXAElGpQfw/Uh?= =?us-ascii?Q?Riq72hguzYMFjgGD8smEj4UiD93LmoFB4TSbruiMTr9sUEO532c+EfF4yljx?= =?us-ascii?Q?Zp+0xZPdeLU0O92QNIP86V5hGVAbxOc2n0SOVpw73EjSnTjTpEFK/pD8/XMO?= =?us-ascii?Q?x2NfIq74rEZ5/B+FrTpMujEyzcfWAQ84kt29DZ9KivFnwD17/Qnc9dbtXHDE?= =?us-ascii?Q?ZLVj1kvkf34zAuHYKAqWNvj8lp4UjXmhvbgt190jQpCHWReMdwWIGOSs3i7d?= =?us-ascii?Q?63tBfYX1A9BoL8+DrTuc3LXqOKd53y8nBOmT5e0WklILRNFLRsAYUhHoBcc3?= =?us-ascii?Q?9JXaXAFA/O/fWqUKfI2zi0x1uZUb+1B1BNGsNiXAA2T2sxa4PGRA1FEQqxak?= =?us-ascii?Q?OuJiZ/iOw0AHmo6nmEL9phCLGAI2oesgswP6fK9IpnmyvPktC2lZIqp2+Wp8?= =?us-ascii?Q?bRGHTjwe2CHxUtNVz+bt0snp8A/XB4m0Vw/fgAxC2gAQ1M8xMQM06/5z9eGM?= =?us-ascii?Q?cSQL66KTVWuS7DcqwwQJewUpAYCHlmN3e0zMS3JNfQFSd5mi2OXz3O2HdNjn?= =?us-ascii?Q?y1yV40cWeblk1f++s4tPKsik3qjeSLiUEgCJkL46rBOS5X78QyYBK3hM7/8C?= =?us-ascii?Q?RmaWEsatFSiBazhblNV8GNuIIZlhuiITJNsOVyxtckDKabRqlFQP+VvWJldA?= =?us-ascii?Q?4FcaghUpMGdqgZ8dladQGpv+v74WM1ir2xnLF5ET30Z0WRkNrADABhaCEVBM?= =?us-ascii?Q?cK/wKQDTYQDw42eZryLyDVVif4qcK7MdqxZzPxSZBbWgUQnHZwht7w1HzOv5?= =?us-ascii?Q?vpyz+nZSqJHPGqFp0Wpzc7Fp75ltjprXb1cCFwN8vwYZmKDUiA2fx8pr33kR?= =?us-ascii?Q?6b7WP4nKCH4zxZrPd3L7MiWh5O1NTLrTtnyX5tMk8fQGcKYgM+VCTFTa9hwp?= =?us-ascii?Q?kSrKh5OhcGsstj7zevbDb2dfjO4xa2CEupMqYWW1kLXOecuRRw/uPE105VA+?= =?us-ascii?Q?hGNsq7tFsVcuDqRoettHgkfnhkzkthma0d1gfcHygoEY81CAoA=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ZAD6u6Oec3QkkZV4uQlm+vLP/zXAUPQ+M6/IoFcqU9lTuh4BcDw4MCZS3BPf?= =?us-ascii?Q?gMzYqFwK14W2QSZMfUkEVdD46liVIwAZyR3VzSjSy0ivvDWrfyQk7Hgg9JGl?= =?us-ascii?Q?RmfFmi/0g30Mn4WdZ2+nr/0JHPurF26ywXLPBIodSJaNgq9CogpDSca8SQZf?= =?us-ascii?Q?3NxSKejCmEtT6/QDsqWb2wxgXgP+L+Knyoe/y9dz94nva7rjNnIra4mLKX1s?= =?us-ascii?Q?u1X0/DDDsGWWO2lfYNSRQuUhZzyWDbI2cW/vGSxLLaTW+GhCF0tzzAP5Zj8b?= =?us-ascii?Q?ryB7AUCy6zVaz9yxtM2LlwxOlS9UsboKHGqYM57C+OV4AGob4TJ6VI9poJPW?= =?us-ascii?Q?aeoyXgD5mWmbCtULpA99MctShSXfX/OwUoJVT013xKv77tguWSnBY/UnVnnm?= =?us-ascii?Q?mSm6b2lZFdJ6JDzoFoVW4t1rC+Ewz4gAxwRDmvIuYebL4h8N8J/skH/sJ6Lo?= =?us-ascii?Q?q7dz+pHipE9bDpLDLCPH14c/ncJAGO2CpviafA9ELNQx05zAA+2lc76hftmN?= =?us-ascii?Q?UUZVdk4XeKM5p5C2Uew85/ODfZvp/U5bkNJ1sH56ya2g0IQp8BLBjfmsYsR1?= =?us-ascii?Q?XpF78jqCIMrccEvwHr3diAt8+o63eAlx5d1n7bjxVcDuDvxWaX6+3jApUJ81?= =?us-ascii?Q?i2/OC6okPzvjY9dSsgtn12bC77YK4bFLRk8iwkq9UQwRj8DrQQS8g8z+p3QI?= =?us-ascii?Q?JML1ALsuSJi3HlT36QraEr12wb6fUk6R5erBqfZS3MX4wooUd4CooyLwQilr?= =?us-ascii?Q?i75bjODjcUVjkpOXq8YOmY72OIWRzRBAyNXmh2C+ZdmBmlXAXDZzoYdhu8sO?= =?us-ascii?Q?nBq4u8x7c6eBBZjcs0ArGk00ZxhZmRkmI7QaUOu7mBP8C9NTTf1lm/jaYNNI?= =?us-ascii?Q?PV1uteWPi7cey6zOzaqqPPzXjZzWSWER8nUC5LA6KQXbOk0LxTa9D7zqmHO/?= =?us-ascii?Q?YDm+ugu0FpMgjW6d0uNxOJ0+IItSsiC66rpByKc/Y5z0f38kO4Alanx3Fuvo?= =?us-ascii?Q?jLOUHr9WjDrxnr46YPrhfYB21QAbaKsOQbx+G2nnG2fbQuyMcuiw0/HEl77W?= =?us-ascii?Q?7AYoF6qBJFU1MesKY2skGImPR8imjNM4azi8IGgRJc9D8Fb2Hp5jtgQrGsZW?= =?us-ascii?Q?nxSZzajj4yu6KCO1vgPVuAwF95QaViRy0o5EOXsvr4ZxZTiRrIvSXT91aDbM?= =?us-ascii?Q?zh6IOZZ5j7/WKZFjp0caVuQm3DJmz226/a7BqpiXwEE6PaKeHwFpLdAdiu0r?= =?us-ascii?Q?zWLeFwftK/gSIdqbUeTL4ooheyfKXxvM0ZY1aumAxLrbYy04TRlocwI7V/Nf?= =?us-ascii?Q?6ZlasivKmUHdLdP7XqtWB8Mhn0hUz2PdAR/8XCYVd2bX8UX9IH7MrOdf5NPN?= =?us-ascii?Q?z6auQuVCNH0f9WMDCCcV42pwTeh50I5i2IX6UrR6YayqSOp+ncqICDsBYWOt?= =?us-ascii?Q?nQK3WCddcYufNlaPIviQhJQuYvFX6NzwPxbViD8STT9D9X7zjPstaiUW9GP5?= =?us-ascii?Q?wcr4oE3fTGxISaXZlEC0D7kOEthit45KBBMOobOtxVblE1NvVNbloNU5G6af?= =?us-ascii?Q?9m1u3npPlLrdJGG+9Y5ZPaE63ssmdyxEFRtD3YDO?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d320346b-d1a5-4ecf-9613-08ddaccec7f6 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2025 12:10:27.7185 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TuCg1h+dwvGX0x5NEe/rHFLMKpMUTAtRldeWP7tZ3f1PgvLh5cxRRbJaKxGshHR2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB9560 Content-Type: text/plain; charset="utf-8" During page isolation, the original migratetype is overwritten, since MIGRATE_* are enums and stored in pageblock bitmaps. Change MIGRATE_ISOLATE to be stored a standalone bit, PB_migrate_isolate, like PB_compact_skip, so that migratetype is not lost during pageblock isolation. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/mmzone.h | 3 +++ include/linux/page-isolation.h | 16 ++++++++++++++++ include/linux/pageblock-flags.h | 14 ++++++++++++++ mm/page_alloc.c | 27 ++++++++++++++++++++++++--- 4 files changed, 57 insertions(+), 3 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 76d66c07b673..1d1bb2b7f40d 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -79,6 +79,9 @@ enum migratetype { * __free_pageblock_cma() function. */ MIGRATE_CMA, + __MIGRATE_TYPE_END =3D MIGRATE_CMA, +#else + __MIGRATE_TYPE_END =3D MIGRATE_HIGHATOMIC, #endif #ifdef CONFIG_MEMORY_ISOLATION MIGRATE_ISOLATE, /* can't allocate from here */ diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 277d8d92980c..fc021d3f95ca 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -11,6 +11,12 @@ static inline bool is_migrate_isolate(int migratetype) { return migratetype =3D=3D MIGRATE_ISOLATE; } +#define get_pageblock_isolate(page) \ + get_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) +#define clear_pageblock_isolate(page) \ + clear_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) +#define set_pageblock_isolate(page) \ + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) #else static inline bool is_migrate_isolate_page(struct page *page) { @@ -20,6 +26,16 @@ static inline bool is_migrate_isolate(int migratetype) { return false; } +static inline bool get_pageblock_isolate(struct page *page) +{ + return false; +} +static inline void clear_pageblock_isolate(struct page *page) +{ +} +static inline void set_pageblock_isolate(struct page *page) +{ +} #endif =20 #define MEMORY_OFFLINE 0x1 diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flag= s.h index c240c7a1fb03..6a44be0f39f4 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -21,6 +21,13 @@ enum pageblock_bits { /* 3 bits required for migrate types */ PB_compact_skip,/* If set the block is skipped by compaction */ =20 +#ifdef CONFIG_MEMORY_ISOLATION + /* + * Pageblock isolation is represented with a separate bit, so that + * the migratetype of a block is not overwritten by isolation. + */ + PB_migrate_isolate, /* If set the block is isolated */ +#endif /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. @@ -32,6 +39,13 @@ enum pageblock_bits { =20 #define MIGRATETYPE_MASK ((1UL << (PB_migrate_end + 1)) - 1) =20 +#ifdef CONFIG_MEMORY_ISOLATION +#define MIGRATETYPE_AND_ISO_MASK \ + (((1UL << (PB_migrate_end + 1)) - 1) | BIT(PB_migrate_isolate)) +#else +#define MIGRATETYPE_AND_ISO_MASK MIGRATETYPE_MASK +#endif + #if defined(CONFIG_HUGETLB_PAGE) =20 #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b303f60b6ed1..a6e191f86ea3 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -365,8 +365,12 @@ get_pfnblock_bitmap_bitidx(const struct page *page, un= signed long pfn, unsigned long *bitmap; unsigned long word_bitidx; =20 +#ifdef CONFIG_MEMORY_ISOLATION + BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 8); +#else BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); - BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); +#endif + BUILD_BUG_ON(__MIGRATE_TYPE_END >=3D (1 << PB_migratetype_bits)); VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); =20 bitmap =3D get_pageblock_bitmap(page, pfn); @@ -439,7 +443,16 @@ bool get_pfnblock_bit(const struct page *page, unsigne= d long pfn, __always_inline enum migratetype get_pfnblock_migratetype(const struct page *page, unsigned long pfn) { - return __get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); + unsigned long mask =3D MIGRATETYPE_AND_ISO_MASK; + unsigned long flags; + + flags =3D __get_pfnblock_flags_mask(page, pfn, mask); + +#ifdef CONFIG_MEMORY_ISOLATION + if (flags & BIT(PB_migrate_isolate)) + return MIGRATE_ISOLATE; +#endif + return flags & MIGRATETYPE_MASK; } =20 /** @@ -519,8 +532,16 @@ __always_inline void set_pageblock_migratetype(struct = page *page, migratetype < MIGRATE_PCPTYPES)) migratetype =3D MIGRATE_UNMOVABLE; =20 +#ifdef CONFIG_MEMORY_ISOLATION + if (migratetype =3D=3D MIGRATE_ISOLATE) { + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + return; + } + /* MIGRATETYPE_AND_ISO_MASK clears PB_migrate_isolate if it is set */ +#endif __set_pfnblock_flags_mask(page, page_to_pfn(page), - (unsigned long)migratetype, MIGRATETYPE_MASK); + (unsigned long)migratetype, + MIGRATETYPE_AND_ISO_MASK); } =20 #ifdef CONFIG_DEBUG_VM --=20 2.47.2 From nobody Thu Oct 9 20:24:37 2025 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2051.outbound.protection.outlook.com [40.107.237.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F08CF2E426A for ; Mon, 16 Jun 2025 12:10:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075836; cv=fail; b=F8w2CbkGDYJVdUVJD1VbdhaGJLrJ7ZHZBiaiwy2KkRu2P1bM8bd7X4KKJ2YxTJTWkQecVayHTw03PJWRfku09UDFX6QZi6otk+qvHVeHU+uTot89m9isdK94EKmHpYsW63bKG6NNdfsXGAMsukswk1PPmzikJwfDUw3tT+dEnc8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075836; c=relaxed/simple; bh=R3FnvYprIf8vL5sGqzgIS2s6TTfd0SOtyt3sQdeM5jg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=LRR1bHWvHzrzeFr8VsgHN2CKT308QNeSglxrVl35u8UmGEAufnNI+Fu0F+5QxQNRRfIvc+CIamUXOtDj5iRL9GLhV6eAwvVNL3haQOuH1WhME4n0zmrcVbdWehGdKIyMlt0xGmrDzXlE+mYaeQuJxdjTnhFXEGj9FCbxisUNBgo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=RypWRML7; arc=fail smtp.client-ip=40.107.237.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="RypWRML7" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=J8ptj2vZgajF5awI+l0rdF+wM0pzcqkV6V3Cgq91OrWROt3wvcM5DAuGOxLvm0ciEATYbQixWBb7gxO+sDB3OEwfilOoFWyKJF1DMYOL4o4blc6KLLp/uN6cUZph+DXPVc9K36EkSyhE2ErgCqALbfZ7aSdS++TUdtkPiP1cSyeH769U8LKWb70lvT5jRprG7s1uU7fmdyEivuXP1XjV5JY3PWSNYxNn+rvi6nMZrEzaWZTabr/0eYGIR3pclF+H21iXxGt15Q7UC9SVMdgBVmlxZghwlARHay+sEA6doEUFTNb8g/fPr8qpjrFUBWHAcR2uvRjI9xdh8kE8VYe1Fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gS5g8DEOfQZrx4Ln+5mZoGBlbyOzV3zhfgR2vmKq4k8=; b=b/GYgNfWeWs77WIHhlXOQkJxOIGvXYmABpUC/bo6Je68mOV7bYMOOgkna6vry+DwOWiymVkTN5sZOC9WQIqMYldfDCc698vUqeLwEYR1DR+h3FBUtjny+pR50e8CZCF7U4dzeadc0jKdvfhmLc+1peLLgwky1l29rW9FUOWptICiszikcGexVY8kIPOY5M+cWfA1R9ILHGBumgpu7SqCgaiE4HWFXYlMPAk+gNWiVCxqf+T1LvcaQAxtioBQNOhlR4YYc2XZTdCmYU9mz2582RjzNxys6msRbM9hFYM98Ngpkqp3opj465XXCO5lwhiqmHcaYv39BnpjoT5Zj6FwiQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gS5g8DEOfQZrx4Ln+5mZoGBlbyOzV3zhfgR2vmKq4k8=; b=RypWRML7ntwwj3Sso2lizkhhITlx2RJk9hG7UToBnsWAE1dhxoTHMwBGzKNiFbmGp/XMHiWMdAnNq9wZ+xztxaOG68DxXSvEP+ib9OjAqrATo0reerGKRPSSM0FlpgGWnWvzdeiv/xRFpET7q6P4zqluJWIZ3ms/oXEdUpQH6rP3/02j6IwA4GClCC81Kf0KYAjFBzXi7AYGz8s8VQlGwtLv+jR2R6r5dPnS6yffXc434cv9dToZgoyDGXJDbMtRpHctv/qjmukbiOd94cboLzWV5coe/2IGFw6TT9eRmqJfb1IH4O1PVIlL2R3Ff4lIPU4aSDFsA/T91O3ljcreQQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY1PR12MB9560.namprd12.prod.outlook.com (2603:10b6:930:fd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.29; Mon, 16 Jun 2025 12:10:29 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8835.027; Mon, 16 Jun 2025 12:10:29 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Johannes Weiner , Vlastimil Babka , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v9 3/6] mm/page_alloc: add support for initializing pageblock as isolated. Date: Mon, 16 Jun 2025 08:10:16 -0400 Message-ID: <20250616121019.1925851-4-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250616121019.1925851-1-ziy@nvidia.com> References: <20250616121019.1925851-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR15CA0035.namprd15.prod.outlook.com (2603:10b6:208:1b4::48) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY1PR12MB9560:EE_ X-MS-Office365-Filtering-Correlation-Id: 1caa833d-2ab4-4edb-0378-08ddaccec8fb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?sthWU6t4z4/QtaiauBAUc/Fjk193FOywkHwi4dgMP1d6A7ARgkakIELssjnF?= =?us-ascii?Q?KEnTScUX00vJYeG5vvn262MtNIu4vgAP0myWrjrODxcblnyqjzfaFqZhH8Pi?= =?us-ascii?Q?/QNZq+RVsJNSdxp/AySugPyVNx4qxAvyK1D4eRAsF6Ak6ozjwcoz4GNyvC08?= =?us-ascii?Q?qN1qYGdlRl3Z3XtuKqKf81XiXRDVy2RX+8C1jS6A0/G+zZVkN3OEmRHLOw9f?= =?us-ascii?Q?Td60Y/0qfZGtj4P7xFWVx20cT3aO7E9D+uIqUSjLnoV1luWm3WZRcAGkTGL7?= =?us-ascii?Q?k5M9AFyxV7ifIEc4pGsISY7D0TkekX8FUz1dKv13wpEvo2RbBCAGbDKEpZVp?= =?us-ascii?Q?M2qPuKoEik5F1SZuJ4DD366puHQ/ZsWeWCVYmoJ/hrwyB+BI5PFHFZ/+2WgW?= =?us-ascii?Q?VX965/RXUPYfv52YO1P5d5e15lpyF5hqFRwceFChRFoNy8S8mJOB+RLoY7oh?= =?us-ascii?Q?QU3TSqUDHAhUXfcb9BVJ5ng0DcM+D3Zo5ZsM53FI9LRNlpnnWNt0C7WzlUAA?= =?us-ascii?Q?tW8240Wnxzqbh/N/PLgu7p3+V2/iCM2Iu0LzCfydKrbZkmIO79xmhycpkgnr?= =?us-ascii?Q?nZkVDjVn7P6mMQtGN2/Gn5mbC8dQzFD9r3vPmLsxKfTpog+HaB1i4ZaN0t74?= =?us-ascii?Q?PZY1vKt80jGSBp5rZgd050zUbRd6gbsz2qZmb2RTVmNAWpU+DO1occ4QW3/1?= =?us-ascii?Q?89MnuLP6vdgmw1C5V+W+vwCjqByRcZ2/pBVz4LfCxQoe5bmyMjwFRIKXos//?= =?us-ascii?Q?qWco6Wt9MXtXGOB0qc0Tn5aBuhf7IuL7egmhwIACJfviS/WG/dEv+8AhgGPJ?= =?us-ascii?Q?aCWYIX7hAU7tbaR99zMVD6q0/TT3Jnm+YKneFj1PhOXYdEXx3XsfGGocaYa9?= =?us-ascii?Q?lT9QWAEbAkITsQqfeg81GUWh6Fp+EOlOuXQPlwBnZoMM7bPxYbD19cGsJFd4?= =?us-ascii?Q?5cdAYdgWl0/Q49BfUO62Wt8n8UAbWRrPCq4iunrhXP20cH0/XlnECuO+aDAh?= =?us-ascii?Q?bmdy+FvzHktNU39dwh5AmKlDbAws1YCMTByS9Q2UUkpgwyNjVMM2QF3qAh7m?= =?us-ascii?Q?3a+5y8QIeeMZFWu4/Mki+ifpo9k4rhrTsZOxSBOJmzibulP1VZ4hypkOezv4?= =?us-ascii?Q?KOI1lI0pxZWQPQhEqx+N52IqBz8qSlaFXptqLrQj6pK5x8GK8MLpYMm2gRRL?= =?us-ascii?Q?8SjUivcPehKuyGUCxlQ/DC/h1O5L3ByoFnkUKZBl45n5lkhP3ysfjD+AuT4O?= =?us-ascii?Q?DSZl+3WnwNGTce6WEhNUveEFvrx/g/I3WdNMNpq1TXpiMBUq1Fiz8eRpkDwI?= =?us-ascii?Q?gVl+GnbUR0UF7VFPNIk7u6SV2M4F87BI4R1Xg9GEl/rqpl7bwb28TNxC3R2z?= =?us-ascii?Q?e1A4/95HzIbNz+Pjb8hW4Opb4A20exGhutUNW3NU1yhZW3IE16AwloZCxy14?= =?us-ascii?Q?54M1dxXGOLQ=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BxINXdqXDk65yVd7wdNphblqH67X39KzlRPnL0hLVn9Abz/4qAdXZ+m+3N4h?= =?us-ascii?Q?WB7Odo0AVTgYbTqZDsuzze+mKpu0ZfdZ+8S4HR6agWdveQLYSnotpSN8RtkT?= =?us-ascii?Q?6sRVoCZsx8xBWcbCcoA6wCj9AE+3PtMTxNU7vAdEzZJOAugo+l2h4tpXQrme?= =?us-ascii?Q?vUieEpT81YR0Lk6dJwJIJ7TjVBv97B9jiU1F/8MddDBoVRzRC0OW0XyYig1j?= =?us-ascii?Q?TAiqqtYPH0+iTH50YQtlRGa/MwlKPb/zUvao37AZGufqBMbEYrpwIdmo5J1D?= =?us-ascii?Q?Ak3IS0SoPf6EqNa5lCRPUSqBd2Bv13tr8PSzwQNefyGIsaAOnQpG5dVNR0Fu?= =?us-ascii?Q?maF8pHSuUZe1uoveXPMtux6CJ6JbuYZBt/QDPq9/7qaR3EO8v7TuhCGA0XUA?= =?us-ascii?Q?E8IvZZK4w6g+4zS5SGoxDesIuSs4+DaddirFlwn1B3t6LvbCD+7iCXBCdiZc?= =?us-ascii?Q?p27+2XnezPIpqifh4ePTS7Qpafo6InnFrQeBqjK3fMAOwwUrSLlg8XUnTVAB?= =?us-ascii?Q?Re3rkJtoynqJ7Z11Uekhsle0KvpOv5LNr4bUqTyBNSzPmkp9GScB0xhom5zy?= =?us-ascii?Q?dZ6j1YYswbSV/0YsHaNrfJ0hH8dhVqLiT8lRy9xbaLcq9ZUlFSWouG3s7D9r?= =?us-ascii?Q?TxfY0hqr7v20ysqvW5LPWC0JXzOCbprYzI0H3uSEdmx7og9rLtleH/2JCIcN?= =?us-ascii?Q?4SrG9oBjE+teQEMb2JZZq8tFRkdMJxohL4vb8MhDNsR/9sQcSA9Bx1DYqaHz?= =?us-ascii?Q?vuVppwBWatPcodeDuw/PaNdnIYVaMAUidjTguyaIJrCtVWiH9sbN6dck5odA?= =?us-ascii?Q?a+L7EnMuud9vNhQu0i2LH4DhnKoVrAwr95t4F9yoV+FKe1fwC3lLOkfbVjrR?= =?us-ascii?Q?pHXyBcjdcwU9VAtib/IgYQZIJsNjX+m06Yck5Uhxhe21RX3YLjeVs+EdOmIL?= =?us-ascii?Q?umBmN67IDEFIZKFJCSO4B7D0cBpiUDdoVyJ9S0f7f4gU69+jmDvBK2KuNZU2?= =?us-ascii?Q?WT+qfwRrV+b1Ql5BNZ1TlNyqHBP3aZEIaFo3ngFhFGRxd+R+5RXV0Cmary+O?= =?us-ascii?Q?LJcz7a5GsEqVL5oIEo5cmsaN5VImZ7s8swjrLFZipEFCyIVum75DSed/5wdg?= =?us-ascii?Q?GOSZS5P44S0hdrFnuVdlOVJdiFwVWqkSuG6XVkx8TWG+lOlwdf8J+asOZsiM?= =?us-ascii?Q?2IJdikTVWiNM9OfcPVV4oXBZ0dzuf+UG6cXKdY7Vw4qZokPTbOdbvU2CHEwV?= =?us-ascii?Q?w3vbFIIzDlvrlXiS+Kl/KKqKnXfDgoK4E5qBszbp0DHITFVeaDqII9dqlkSQ?= =?us-ascii?Q?he2DKySHJA5vlPgRyu4X8kXbqJxEbW809ddEs1ZVRMNeafCpLDvV6/ZqKF5Z?= =?us-ascii?Q?0N8hMjt/kckAgiDDoQBiB43eYKJcZKT8TPuyMk1PH4CMX8GbvRuIBJfdOeXD?= =?us-ascii?Q?IuKbarh3SXpTJMj1c1iNS22uTKzwP6137Bg5Dx0pPh75ImxtIDNWx+lmGTYp?= =?us-ascii?Q?Qp2/1LcsPm9No1hH9mxCl/eaE0WJQRUYMu6HtIcJ33GuW4KqaGrN++gl8pQY?= =?us-ascii?Q?9ggOwSOcHOOkz53VmTSKNOzH9vbaKKDfm8IncOuQ?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1caa833d-2ab4-4edb-0378-08ddaccec8fb X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2025 12:10:29.4373 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: vXEF05qpzbnz0kCTBnLa5c+q7eahddAlOAmc4gpSjf1gxU3Pp5U/zwRy/dap7axs X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB9560 Content-Type: text/plain; charset="utf-8" MIGRATE_ISOLATE is a standalone bit, so a pageblock cannot be initialized to just MIGRATE_ISOLATE. Add init_pageblock_migratetype() to enable initialize a pageblock with a migratetype and isolated. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/memory_hotplug.h | 3 ++- include/linux/page-isolation.h | 3 +++ mm/hugetlb.c | 4 ++-- mm/internal.h | 3 ++- mm/memory_hotplug.c | 12 ++++++++---- mm/memremap.c | 2 +- mm/mm_init.c | 24 +++++++++++++++--------- mm/page_alloc.c | 26 ++++++++++++++++++++++++++ 8 files changed, 59 insertions(+), 18 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index eaac5ae8c05c..23f038a16231 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -314,7 +314,8 @@ extern int add_memory_driver_managed(int nid, u64 start= , u64 size, mhp_t mhp_flags); extern void move_pfn_range_to_zone(struct zone *zone, unsigned long start_= pfn, unsigned long nr_pages, - struct vmem_altmap *altmap, int migratetype); + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock); extern void remove_pfn_range_from_zone(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages); diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index fc021d3f95ca..14c6a5f691c2 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -41,6 +41,9 @@ static inline void set_pageblock_isolate(struct page *pag= e) #define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 =20 +void __meminit init_pageblock_migratetype(struct page *page, + enum migratetype migratetype, + bool isolate); void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 bool move_freepages_block_isolate(struct zone *zone, struct page *page, diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 7a7df0b2a561..3d61ec17c15a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3299,8 +3299,8 @@ static void __init hugetlb_bootmem_init_migratetype(s= truct folio *folio, if (folio_test_hugetlb_cma(folio)) init_cma_pageblock(folio_page(folio, i)); else - set_pageblock_migratetype(folio_page(folio, i), - MIGRATE_MOVABLE); + init_pageblock_migratetype(folio_page(folio, i), + MIGRATE_MOVABLE, false); } } =20 diff --git a/mm/internal.h b/mm/internal.h index 3823fb356d3b..79955ed9ac0a 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -821,7 +821,8 @@ extern void *memmap_alloc(phys_addr_t size, phys_addr_t= align, int nid, bool exact_nid); =20 void memmap_init_range(unsigned long, int, unsigned long, unsigned long, - unsigned long, enum meminit_context, struct vmem_altmap *, int); + unsigned long, enum meminit_context, struct vmem_altmap *, int, + bool); =20 #if defined CONFIG_COMPACTION || defined CONFIG_CMA =20 diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index ddc6c6c63a30..35f008364698 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -747,7 +747,8 @@ static inline void section_taint_zone_device(unsigned l= ong pfn) */ void move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages, - struct vmem_altmap *altmap, int migratetype) + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock) { struct pglist_data *pgdat =3D zone->zone_pgdat; int nid =3D pgdat->node_id; @@ -779,7 +780,8 @@ void move_pfn_range_to_zone(struct zone *zone, unsigned= long start_pfn, * are reserved so nobody should be touching them so we should be safe */ memmap_init_range(nr_pages, nid, zone_idx(zone), start_pfn, 0, - MEMINIT_HOTPLUG, altmap, migratetype); + MEMINIT_HOTPLUG, altmap, migratetype, + isolate_pageblock); =20 set_zone_contiguous(zone); } @@ -1104,7 +1106,8 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsi= gned long nr_pages, if (mhp_off_inaccessible) page_init_poison(pfn_to_page(pfn), sizeof(struct page) * nr_pages); =20 - move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE); + move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE, + false); =20 for (i =3D 0; i < nr_pages; i++) { struct page *page =3D pfn_to_page(pfn + i); @@ -1171,7 +1174,8 @@ int online_pages(unsigned long pfn, unsigned long nr_= pages, =20 =20 /* associate pfn range with the zone */ - move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_ISOLATE); + move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_MOVABLE, + true); =20 node_arg.nid =3D NUMA_NO_NODE; if (!node_state(nid, N_MEMORY)) { diff --git a/mm/memremap.c b/mm/memremap.c index c417c843e9b1..3319e7cc2898 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -254,7 +254,7 @@ static int pagemap_range(struct dev_pagemap *pgmap, str= uct mhp_params *params, zone =3D &NODE_DATA(nid)->node_zones[ZONE_DEVICE]; move_pfn_range_to_zone(zone, PHYS_PFN(range->start), PHYS_PFN(range_len(range)), params->altmap, - MIGRATE_MOVABLE); + MIGRATE_MOVABLE, false); } =20 mem_hotplug_done(); diff --git a/mm/mm_init.c b/mm/mm_init.c index 02f41e2bdf60..5c21b3af216b 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -685,7 +685,8 @@ void __meminit __init_page_from_nid(unsigned long pfn, = int nid) __init_single_page(pfn_to_page(pfn), pfn, zid, nid); =20 if (pageblock_aligned(pfn)) - set_pageblock_migratetype(pfn_to_page(pfn), MIGRATE_MOVABLE); + init_pageblock_migratetype(pfn_to_page(pfn), MIGRATE_MOVABLE, + false); } =20 #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT @@ -874,7 +875,8 @@ static void __init init_unavailable_range(unsigned long= spfn, void __meminit memmap_init_range(unsigned long size, int nid, unsigned lon= g zone, unsigned long start_pfn, unsigned long zone_end_pfn, enum meminit_context context, - struct vmem_altmap *altmap, int migratetype) + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock) { unsigned long pfn, end_pfn =3D start_pfn + size; struct page *page; @@ -931,7 +933,8 @@ void __meminit memmap_init_range(unsigned long size, in= t nid, unsigned long zone * over the place during system boot. */ if (pageblock_aligned(pfn)) { - set_pageblock_migratetype(page, migratetype); + init_pageblock_migratetype(page, migratetype, + isolate_pageblock); cond_resched(); } pfn++; @@ -954,7 +957,8 @@ static void __init memmap_init_zone_range(struct zone *= zone, return; =20 memmap_init_range(end_pfn - start_pfn, nid, zone_id, start_pfn, - zone_end_pfn, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); + zone_end_pfn, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE, + false); =20 if (*hole_pfn < start_pfn) init_unavailable_range(*hole_pfn, start_pfn, zone_id, nid); @@ -1035,7 +1039,7 @@ static void __ref __init_zone_device_page(struct page= *page, unsigned long pfn, * because this is done early in section_activate() */ if (pageblock_aligned(pfn)) { - set_pageblock_migratetype(page, MIGRATE_MOVABLE); + init_pageblock_migratetype(page, MIGRATE_MOVABLE, false); cond_resched(); } =20 @@ -1996,7 +2000,8 @@ static void __init deferred_free_pages(unsigned long = pfn, /* Free a large naturally-aligned chunk if possible */ if (nr_pages =3D=3D MAX_ORDER_NR_PAGES && IS_MAX_ORDER_ALIGNED(pfn)) { for (i =3D 0; i < nr_pages; i +=3D pageblock_nr_pages) - set_pageblock_migratetype(page + i, MIGRATE_MOVABLE); + init_pageblock_migratetype(page + i, MIGRATE_MOVABLE, + false); __free_pages_core(page, MAX_PAGE_ORDER, MEMINIT_EARLY); return; } @@ -2006,7 +2011,8 @@ static void __init deferred_free_pages(unsigned long = pfn, =20 for (i =3D 0; i < nr_pages; i++, page++, pfn++) { if (pageblock_aligned(pfn)) - set_pageblock_migratetype(page, MIGRATE_MOVABLE); + init_pageblock_migratetype(page, MIGRATE_MOVABLE, + false); __free_pages_core(page, 0, MEMINIT_EARLY); } } @@ -2305,7 +2311,7 @@ void __init init_cma_reserved_pageblock(struct page *= page) set_page_count(p, 0); } while (++p, --i); =20 - set_pageblock_migratetype(page, MIGRATE_CMA); + init_pageblock_migratetype(page, MIGRATE_CMA, false); set_page_refcounted(page); /* pages were reserved and not allocated */ clear_page_tag_ref(page); @@ -2319,7 +2325,7 @@ void __init init_cma_reserved_pageblock(struct page *= page) */ void __init init_cma_pageblock(struct page *page) { - set_pageblock_migratetype(page, MIGRATE_CMA); + init_pageblock_migratetype(page, MIGRATE_CMA, false); adjust_managed_page_count(page, pageblock_nr_pages); page_zone(page)->cma_pages +=3D pageblock_nr_pages; } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a6e191f86ea3..3fae0694aaaa 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -544,6 +544,32 @@ __always_inline void set_pageblock_migratetype(struct = page *page, MIGRATETYPE_AND_ISO_MASK); } =20 +void __meminit init_pageblock_migratetype(struct page *page, + enum migratetype migratetype, + bool isolate) +{ + unsigned long flags; + + if (unlikely(page_group_by_mobility_disabled && + migratetype < MIGRATE_PCPTYPES)) + migratetype =3D MIGRATE_UNMOVABLE; + + flags =3D migratetype; + +#ifdef CONFIG_MEMORY_ISOLATION + if (migratetype =3D=3D MIGRATE_ISOLATE) { + VM_WARN_ONCE( + 1, + "Set isolate=3Dtrue to isolate pageblock with a migratetype"); + return; + } + if (isolate) + flags |=3D BIT(PB_migrate_isolate); +#endif + __set_pfnblock_flags_mask(page, page_to_pfn(page), flags, + MIGRATETYPE_AND_ISO_MASK); +} + #ifdef CONFIG_DEBUG_VM static int page_outside_zone_boundaries(struct zone *zone, struct page *pa= ge) { --=20 2.47.2 From nobody Thu Oct 9 20:24:37 2025 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2051.outbound.protection.outlook.com [40.107.237.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10B4B285CBB for ; Mon, 16 Jun 2025 12:10:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075838; cv=fail; b=lKUPHpUHHBTzjZETTHHznMZbZVgERce3fRSuS5sB4j329qQ3gwsCvxgwz4VPe3hcwSWHm76VxfnModW6T3zO0azBhxlkLH+/JzX4Vlf8bwvVUzzvEE4t7OS6g+TkrsEkOcYC6OfGwSSOLb5ZiTMPIPhty1gtVg/CaJRXBX/A7SQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075838; c=relaxed/simple; bh=EGWpb/EQeiWMZrEmUAFVy8Ia/PUL5CuUPfNWpAowAFU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=S+qHzDE2P+C+H3FkIqB47OxioW7Xa4jmOC0ZBsZYIkPXX+3hOFDiR2CJPDurCmB45Yke18/YllPelLS7Wy6gzRApyUN6lWRSCyVpo/vOE6uk74c63NeiYhzxG1rym6JJ1PR4//mXPxlTWAXmArMzMowhFLoosoSmmXAKk/K9kvU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=I391AgzB; arc=fail smtp.client-ip=40.107.237.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="I391AgzB" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mOQ73l06nGu0tEJq82MjHNPiWjNgtef5V8P5ToDp21Oy3G7wtpLS4dt8UhgiXO6jdWugAtinylN8prCTQXNg/Fm7ZnGqK8emi8K+X+qZbgAoVrVq5IYTK7DvcT4BIYbAcEnQ6VkpGicH/LGK54Zba7qAbWWcxnEvu7ga4R+MkQRfWs5DMu34p0KhLHY+n0ChCdNfYMDxt4QaIIi0tp60BFMSXIiX51NLVLe9VpUjSbn59SGoSH0VGxSbGjUZmP2H3xE5VpmPF26uVYsAkuGLXHk7DTpBI3oc4IU1NfLsICb4P62fXpaeFMRvmjwitrelmWehbJDohTGKc4HnqSiaAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NxI0/1wZBlRnZRVuH9jl4pcmlwXpkQJNlOlcwtgHQek=; b=jIzLuaj4RxJ+6A/otBUCLYnY1xeXQ3E63QyNiFaPNpy7oKuy98gpxyBkB8W9pDfgsrkT9sw6xO83PhQZZxBs1HSv92bHutVxB2HHuriRQkhy0s+GMdmftqIiP7VtplScsoHpOkP+wL4QnYbg8nJlkiWduwM4gXtjo1YBAQKMDDc0iG7KLzDvvAx1NFFoiSL1LpvdD49sioVYVtQtn5UcIw5YcIaIFYMYLZ12Gc0a+wxLPBWVcGuJ0P0gCPYJshKiZEiwd4e8u8OgcggWMsvvsTbYBGbn6yCZX+aB18FG2SEkf7SixNHAHll/Bo307Z1RXA1YJmc5o9oBgGwp6QwqlA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NxI0/1wZBlRnZRVuH9jl4pcmlwXpkQJNlOlcwtgHQek=; b=I391AgzB9XHKGZrhzf6SftXcvB479drarna1zgOMFS6Q5vq9j8tqj/X9rzC10NBLkmYu5CUp+iIXHDqYMdLHtOPhxWLUYX+0LaltPufCCSVFgcGX7MSA87HvsFZBlyB07vG5VMhcZjFex7Z+kycO3Yg1xo3G70bX8zVAPQMWdpP3wJ6hUD4AakwznRQrUQQAn8z/g7g/oonqfAqJoUlZXuGl3oYKEu28812iJS6pqn5tUHDNLC7XTjuVRAvtnNzg/HNt4soAehzi+gHsIccy1agT+In9As0Z+Tio0isXD/09Nw3cgq8izsFZvvZJc9dGHDdCLnQ8AkqVQSV5vauPtA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY1PR12MB9560.namprd12.prod.outlook.com (2603:10b6:930:fd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.29; Mon, 16 Jun 2025 12:10:31 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8835.027; Mon, 16 Jun 2025 12:10:31 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Johannes Weiner , Vlastimil Babka , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v9 4/6] mm/page_isolation: remove migratetype from move_freepages_block_isolate() Date: Mon, 16 Jun 2025 08:10:17 -0400 Message-ID: <20250616121019.1925851-5-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250616121019.1925851-1-ziy@nvidia.com> References: <20250616121019.1925851-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR15CA0029.namprd15.prod.outlook.com (2603:10b6:208:1b4::42) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY1PR12MB9560:EE_ X-MS-Office365-Filtering-Correlation-Id: 75f944d1-017c-4e20-2279-08ddacceca25 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?DV1mx1JMAUHROQ/0Hru4TgdIj8cBI69b+Gu8wuDmBjuCubA2Lej48uD3iyJi?= =?us-ascii?Q?1xTuy76mi68cLZgP+FbzvzDnfRkotru/j21MZChAJVqg6Icvz/+zOa3FiH70?= =?us-ascii?Q?ZFVnSuVFko5Lr/ACjX8tS9JhTj3lkMKoqddCoBtIdD6m3zGmLI01z8tTuDCY?= =?us-ascii?Q?tjqTMBKSkTBAg4rdHbKki8jXp8+VUBhV2/Wuddi0NwNsC1LDzpXKcYIFSxu1?= =?us-ascii?Q?9UdwKcKKp4SOCK0EXNgejkIWtWQmptrifBraqZLcF1NxjoOnktyizsMexqRj?= =?us-ascii?Q?u7106rfO89/z50yMgxtYHkNePmfdQmWrHxHzZFNgMZ2bNuFrh4tnOLRUi1t3?= =?us-ascii?Q?U2VIWtsmr2q/vaZaJffEOC6NwE0JYq1yAQl05uVZRSb0TXuPzjLeq7vY8VQ7?= =?us-ascii?Q?Yf6jKLG2MhTYkaLPRjSol3s2GC1fOew2P0S4f7L7SMbuUNQQ8MZLBLRFShB2?= =?us-ascii?Q?Bi/laiW30EXVlqUPmeco9KYyfg8xSLvpeHgSGk7ik+HGN80/H7VGa1XTp656?= =?us-ascii?Q?UJ7QN8VESLcYNHD5QxmMcmEOJXMydx/mR0sKRGcyJC9zTiMzRwoL9h7t/9vK?= =?us-ascii?Q?Ou64IfkDIH0OtdVf7eWLmvkyJHPb4+Q+dEA3wywP/S2FMLAjbe8PWlnJKRjO?= =?us-ascii?Q?BySQ/jZKcdwFqxdwXMutGw2o478J7KWAwVjFifoWi+IAUE9l+iO1Jmwa9Ww/?= =?us-ascii?Q?HNWpB+XRdDfrAhymqIZdy7O9j/q7j2G+Ng5NMW1N+wBcw9Dbi64/xaeHS6a8?= =?us-ascii?Q?ULthrNu2C41744hNXVo1kKkD9ktsBxcFyOKzs7rdYJtBcRUkHmUrTbqwSFpn?= =?us-ascii?Q?RHOPvMuOUNCKnSL4QUze3uYSbvCKcUgjKhJCYEZSAGvHmYL6c0xb8eeAUvb9?= =?us-ascii?Q?S+Yn01P2YdSpfMWEYhqfM+3MM1FIv6kvOp8u6azHQ1sTDTJu8TBycI3ohSvH?= =?us-ascii?Q?e4r71q3bvRf2ThbMOaBtbTZMtZbWs5hPNw8JCJa9DteyERlPf2N3jtftg1d+?= =?us-ascii?Q?HZdwzQVp6WUedKh/cnOdstuE3OdwYngwXQgFA0UfCs6+zI+z0yiOUHJq8fXQ?= =?us-ascii?Q?L3HVxO/MA61v8paH23V5+LASvyAkE0c6sRYsXYVNFfIaVb1poBs0x+HCdsze?= =?us-ascii?Q?rLGXduUAvevEW0JsFLay/BD4fJ+i9g6HFtRKN7MMeTcXeDHUEx/4LXfQB4Lo?= =?us-ascii?Q?7uqC77x5sE7L8+jEZLyi4qyDsHyUGnXW82Sy8mPykzkPuW76q0AgOZMWlMkp?= =?us-ascii?Q?RaL8acWia+4wVYfQE/QRvh6ucal6QeAXwnESBqkj3egjt+Tk6TAVnV2ohGcL?= =?us-ascii?Q?wAc9Ep0pqyIXs+dV3c4JTyUruKcr0GFjiZDjySsn4nwXPa/8NL5Ptb1eipOB?= =?us-ascii?Q?ksLj7qC083VhrtoDjA6Jx+MqGnd7GAoRUVJt6UJsDM5VFhdtRTPuQFDjBYth?= =?us-ascii?Q?dhEhPeWFvzE=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?wR62WxbylBebbqwMSFvXlO3aLwOn9DzFe+IhiTM/MhJ53S4gtTGc3MAriY3+?= =?us-ascii?Q?LftPMTD1fRrlLZjvrzhOyOW7u2xqrtIHeWFUXHfbbvciP0tTF8lnZODSC/Zv?= =?us-ascii?Q?PxSqiChxsS9L6Mpp1+dh9y62kxBKDaiO9a5jxenp4dg+05q5XXmnnJDCfYOw?= =?us-ascii?Q?WpDt8NxoItwapL00SVCi8jdSgzf3ANI1PeLNZgxigCkHqiUZMdBxtb+ovVwH?= =?us-ascii?Q?8CUry5A7aoI8mPLYADkvkaWmPmRk2PPrYBKn/2eqamVpVy8zJVUdaWVvQAeG?= =?us-ascii?Q?CYPTvEypPGUXJTieeKTNTHvpNpliQXxaLD8j2rgutMcO8L4n8SQvQCuMkOqK?= =?us-ascii?Q?0RbKCCrQ/Iem721FvegF+XUO/UXdHC+xBGghKZwDXmNRuq0aGNM1OPpSHo3v?= =?us-ascii?Q?qnt5LdL/Kz4XnSR0ZuC4ZQzQgRcDzafTTD511gtBYJeDYdPJUAAM1EwGtj8B?= =?us-ascii?Q?MJwM7R+3tXxICy4adHt+nM3eNpsz6bXLvjEuWRb1SUfSHEuKI1pfReowax7G?= =?us-ascii?Q?fo6OcUizDPr9wfm77sjcE6OANRXYZX0GOVBroYr6iGFAZ/Qhq6g19L0egKy3?= =?us-ascii?Q?/1d7Dp/QTluvroiBrh5jGtKVlCoj2zylh0OhOxLEPYvtHejUECYbFc/4SFvL?= =?us-ascii?Q?VHFutXl+up//jF0cI9D6tH/0aXtEz8/+xdFhkTS8JdNUu5z1YmvkfxvctvjL?= =?us-ascii?Q?EWK80EM35c+FkkZKjsH23fl94v/Ua3dSq28UQszfrJ2DYTOA5VSIQqkTERII?= =?us-ascii?Q?EeAItSdPaeL/teEeqKlM3oWJ/APYYz+HXSKVDkZUlRoeD6mFGIgEH0iJNRTG?= =?us-ascii?Q?0EXbvh5BRrisjJgil4kbfA9A3L9BJlZupMOaMbiXNPk5g6a0va6NuBK4J+zq?= =?us-ascii?Q?mS4Lc9I1tALJrO0onuJMbRHWoOTvJ3aVh14Ouy+jxzRL0T5zSbBHoTHBnwMV?= =?us-ascii?Q?8LOGWc5BOvA5BSXKw0eWxIfhbew9vg6xlVI0niqRsgq2m+pWI/p3YZ3YfJK/?= =?us-ascii?Q?LNpNiew1nZGrrOtUPhNskP7EnkD0exreCHhHksgYYY++Ktd7rUoXXiqTOHNy?= =?us-ascii?Q?C8Dh30SQ5c2dXDmOpdVaiIs8RCsK2iGdMpZW1rQ6Ws3rSnjDaqvlKxAdhZr5?= =?us-ascii?Q?3QtxC1KrN4SSE1YS358eQhrZA5trjKCuXYyZyTk1u5BpKwiGw4iaoH2tHsCP?= =?us-ascii?Q?2hUTOhBP96wTU5f+hFkPBRh9ivlmLZKpZ70m0gtzpU+CxruqJFQzgjdyKDcP?= =?us-ascii?Q?kqLfjSUadL6XD0q3hskpbTsIcUcP5WitTVOWYO1LsXJbNQJJMsw8O4Krz9iS?= =?us-ascii?Q?KLnpaHEcLqNRuM7L15JrFUQ9XCgF141SqEw0ztZbU1nPCH6Kg4TcxlL3gvtX?= =?us-ascii?Q?RdeLTAX/CzF/j8vbXn35idO/IiQxkmrflqCst/sMTzoekWcc9TwVzj/o8YIY?= =?us-ascii?Q?pVtXRYf/ylsBH2OhBTXvPnahConG4frMFzRpGCoLASaQ8OY3ZYnWz2c9D0AV?= =?us-ascii?Q?ygNARtsxLWTgygY0T0RQvRqlkCV6yL6fFYBRJuDO+s8oUwTCG4Dx/y1sOZBX?= =?us-ascii?Q?urii+QwgA+z92XwdQcaIUNx8GN5SMchnw8RfiZJe?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 75f944d1-017c-4e20-2279-08ddacceca25 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2025 12:10:31.3608 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XeSYlheHw9kSz078akL3CvtxjqrnDBdCL2EvjXXo7fX8TOPaVJurVSFbaY7dWGIY X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB9560 Content-Type: text/plain; charset="utf-8" Since migratetype is no longer overwritten during pageblock isolation, moving a pageblock out of MIGRATE_ISOLATE no longer needs a new migratetype. Add pageblock_isolate_and_move_free_pages() and pageblock_unisolate_and_move_free_pages() to be explicit about the page isolation operations. Both share the common code in __move_freepages_block_isolate(), which is renamed from move_freepages_block_isolate(). Add toggle_pageblock_isolate() to flip pageblock isolation bit in __move_freepages_block_isolate(). Make set_pageblock_migratetype() only accept non MIGRATE_ISOLATE types, so that one should use set_pageblock_isolate() to isolate pageblocks. As a result, move pageblock migratetype code out of __move_freepages_block(). Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/page-isolation.h | 5 +-- mm/page_alloc.c | 80 ++++++++++++++++++++++++++-------- mm/page_isolation.c | 21 +++++---- 3 files changed, 75 insertions(+), 31 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 14c6a5f691c2..7241a6719618 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -44,10 +44,9 @@ static inline void set_pageblock_isolate(struct page *pa= ge) void __meminit init_pageblock_migratetype(struct page *page, enum migratetype migratetype, bool isolate); -void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 -bool move_freepages_block_isolate(struct zone *zone, struct page *page, - int migratetype); +bool pageblock_isolate_and_move_free_pages(struct zone *zone, struct page = *page); +bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page); =20 int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, int migratetype, int flags); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3fae0694aaaa..06cc990e2a62 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -525,8 +525,8 @@ void clear_pfnblock_bit(const struct page *page, unsign= ed long pfn, * @page: The page within the block of interest * @migratetype: migratetype to set */ -__always_inline void set_pageblock_migratetype(struct page *page, - enum migratetype migratetype) +static void set_pageblock_migratetype(struct page *page, + enum migratetype migratetype) { if (unlikely(page_group_by_mobility_disabled && migratetype < MIGRATE_PCPTYPES)) @@ -534,9 +534,13 @@ __always_inline void set_pageblock_migratetype(struct = page *page, =20 #ifdef CONFIG_MEMORY_ISOLATION if (migratetype =3D=3D MIGRATE_ISOLATE) { - set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + VM_WARN_ONCE(1, + "Use set_pageblock_isolate() for pageblock isolation"); return; } + VM_WARN_ONCE(get_pfnblock_bit(page, page_to_pfn(page), + PB_migrate_isolate), + "Use clear_pageblock_isolate() to unisolate pageblock"); /* MIGRATETYPE_AND_ISO_MASK clears PB_migrate_isolate if it is set */ #endif __set_pfnblock_flags_mask(page, page_to_pfn(page), @@ -1921,8 +1925,8 @@ static inline struct page *__rmqueue_cma_fallback(str= uct zone *zone, #endif =20 /* - * Change the type of a block and move all its free pages to that - * type's freelist. + * Move all free pages of a block to new type's freelist. Caller needs to + * change the block type. */ static int __move_freepages_block(struct zone *zone, unsigned long start_p= fn, int old_mt, int new_mt) @@ -1954,8 +1958,6 @@ static int __move_freepages_block(struct zone *zone, = unsigned long start_pfn, pages_moved +=3D 1 << order; } =20 - set_pageblock_migratetype(pfn_to_page(start_pfn), new_mt); - return pages_moved; } =20 @@ -2013,11 +2015,16 @@ static int move_freepages_block(struct zone *zone, = struct page *page, int old_mt, int new_mt) { unsigned long start_pfn; + int res; =20 if (!prep_move_freepages_block(zone, page, &start_pfn, NULL, NULL)) return -1; =20 - return __move_freepages_block(zone, start_pfn, old_mt, new_mt); + res =3D __move_freepages_block(zone, start_pfn, old_mt, new_mt); + set_pageblock_migratetype(pfn_to_page(start_pfn), new_mt); + + return res; + } =20 #ifdef CONFIG_MEMORY_ISOLATION @@ -2045,11 +2052,19 @@ static unsigned long find_large_buddy(unsigned long= start_pfn) return start_pfn; } =20 +static inline void toggle_pageblock_isolate(struct page *page, bool isolat= e) +{ + if (isolate) + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + else + clear_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); +} + /** - * move_freepages_block_isolate - move free pages in block for page isolat= ion + * __move_freepages_block_isolate - move free pages in block for page isol= ation * @zone: the zone * @page: the pageblock page - * @migratetype: migratetype to set on the pageblock + * @isolate: to isolate the given pageblock or unisolate it * * This is similar to move_freepages_block(), but handles the special * case encountered in page isolation, where the block of interest @@ -2064,10 +2079,18 @@ static unsigned long find_large_buddy(unsigned long= start_pfn) * * Returns %true if pages could be moved, %false otherwise. */ -bool move_freepages_block_isolate(struct zone *zone, struct page *page, - int migratetype) +static bool __move_freepages_block_isolate(struct zone *zone, + struct page *page, bool isolate) { unsigned long start_pfn, pfn; + int from_mt; + int to_mt; + + if (isolate =3D=3D get_pageblock_isolate(page)) { + VM_WARN_ONCE(1, "%s a pageblock that is already in that state", + isolate ? "Isolate" : "Unisolate"); + return false; + } =20 if (!prep_move_freepages_block(zone, page, &start_pfn, NULL, NULL)) return false; @@ -2084,7 +2107,7 @@ bool move_freepages_block_isolate(struct zone *zone, = struct page *page, =20 del_page_from_free_list(buddy, zone, order, get_pfnblock_migratetype(buddy, pfn)); - set_pageblock_migratetype(page, migratetype); + toggle_pageblock_isolate(page, isolate); split_large_buddy(zone, buddy, pfn, order, FPI_NONE); return true; } @@ -2095,16 +2118,38 @@ bool move_freepages_block_isolate(struct zone *zone= , struct page *page, =20 del_page_from_free_list(page, zone, order, get_pfnblock_migratetype(page, pfn)); - set_pageblock_migratetype(page, migratetype); + toggle_pageblock_isolate(page, isolate); split_large_buddy(zone, page, pfn, order, FPI_NONE); return true; } move: - __move_freepages_block(zone, start_pfn, - get_pfnblock_migratetype(page, start_pfn), - migratetype); + /* Use MIGRATETYPE_MASK to get non-isolate migratetype */ + if (isolate) { + from_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_MASK); + to_mt =3D MIGRATE_ISOLATE; + } else { + from_mt =3D MIGRATE_ISOLATE; + to_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_MASK); + } + + __move_freepages_block(zone, start_pfn, from_mt, to_mt); + toggle_pageblock_isolate(pfn_to_page(start_pfn), isolate); + return true; } + +bool pageblock_isolate_and_move_free_pages(struct zone *zone, struct page = *page) +{ + return __move_freepages_block_isolate(zone, page, true); +} + +bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page) +{ + return __move_freepages_block_isolate(zone, page, false); +} + #endif /* CONFIG_MEMORY_ISOLATION */ =20 static void change_pageblock_range(struct page *pageblock_page, @@ -2296,6 +2341,7 @@ try_to_claim_block(struct zone *zone, struct page *pa= ge, if (free_pages + alike_pages >=3D (1 << (pageblock_order-1)) || page_group_by_mobility_disabled) { __move_freepages_block(zone, start_pfn, block_type, start_type); + set_pageblock_migratetype(pfn_to_page(start_pfn), start_type); return __rmqueue_smallest(zone, order, start_type); } =20 diff --git a/mm/page_isolation.c b/mm/page_isolation.c index b2fc5266e3d2..08f627a5032f 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -188,7 +188,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ unmovable =3D has_unmovable_pages(check_unmovable_start, check_unmovable_= end, migratetype, isol_flags); if (!unmovable) { - if (!move_freepages_block_isolate(zone, page, MIGRATE_ISOLATE)) { + if (!pageblock_isolate_and_move_free_pages(zone, page)) { spin_unlock_irqrestore(&zone->lock, flags); return -EBUSY; } @@ -209,7 +209,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ return -EBUSY; } =20 -static void unset_migratetype_isolate(struct page *page, int migratetype) +static void unset_migratetype_isolate(struct page *page) { struct zone *zone; unsigned long flags; @@ -262,10 +262,10 @@ static void unset_migratetype_isolate(struct page *pa= ge, int migratetype) * Isolating this block already succeeded, so this * should not fail on zone boundaries. */ - WARN_ON_ONCE(!move_freepages_block_isolate(zone, page, migratetype)); + WARN_ON_ONCE(!pageblock_unisolate_and_move_free_pages(zone, page)); } else { - set_pageblock_migratetype(page, migratetype); - __putback_isolated_page(page, order, migratetype); + clear_pageblock_isolate(page); + __putback_isolated_page(page, order, get_pageblock_migratetype(page)); } zone->nr_isolate_pageblock--; out: @@ -383,7 +383,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, if (PageBuddy(page)) { int order =3D buddy_order(page); =20 - /* move_freepages_block_isolate() handled this */ + /* pageblock_isolate_and_move_free_pages() handled this */ VM_WARN_ON_ONCE(pfn + (1 << order) > boundary_pfn); =20 pfn +=3D 1UL << order; @@ -433,7 +433,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, failed: /* restore the original migratetype */ if (!skip_isolation) - unset_migratetype_isolate(pfn_to_page(isolate_pageblock), migratetype); + unset_migratetype_isolate(pfn_to_page(isolate_pageblock)); return -EBUSY; } =20 @@ -504,7 +504,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, ret =3D isolate_single_pageblock(isolate_end, flags, true, skip_isolation, migratetype); if (ret) { - unset_migratetype_isolate(pfn_to_page(isolate_start), migratetype); + unset_migratetype_isolate(pfn_to_page(isolate_start)); return ret; } =20 @@ -517,8 +517,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, start_pfn, end_pfn)) { undo_isolate_page_range(isolate_start, pfn, migratetype); unset_migratetype_isolate( - pfn_to_page(isolate_end - pageblock_nr_pages), - migratetype); + pfn_to_page(isolate_end - pageblock_nr_pages)); return -EBUSY; } } @@ -548,7 +547,7 @@ void undo_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, page =3D __first_valid_page(pfn, pageblock_nr_pages); if (!page || !is_migrate_isolate_page(page)) continue; - unset_migratetype_isolate(page, migratetype); + unset_migratetype_isolate(page); } } /* --=20 2.47.2 From nobody Thu Oct 9 20:24:37 2025 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2051.outbound.protection.outlook.com [40.107.237.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 302CD292B33 for ; Mon, 16 Jun 2025 12:10:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075841; cv=fail; b=fHaMvthuXYf5v5IXT48uWdYYH9C0G9iAX9W7t+fjD7O51dV22KrUz4LQuvwLrVLoz1ZKkkiW7HS3v1tWnpS42X4ePrTY6NKeNk5uwI8/IQmLggGNVRLuNw+KOLuXdHNsvXwmjM7SQKz9tDKQMQddNCMxB5UZJ0yopaBXXhmOD5Y= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075841; c=relaxed/simple; bh=Vaw4uKXFktsK2QkoVycqkyBrM2YsK1BHmChTsh1BKI0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=K+qbFTHrX5xUj7HxVf9yszxF9q7NNvSTutl2YAnYkLCXRcjG5c6soockg2np9pJ6ulse51ilSQSg8gAT6+kX1bjXm0qaWbaC9ZstolMBEaXYNAdc5yy4CbvDsTaA4t/n9DAZb6O/wxPftP5prwKhVRXsz9i2/wPtfF+d6jL3Y9s= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Ji+accCI; arc=fail smtp.client-ip=40.107.237.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Ji+accCI" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VTDyp7XHpnHyE4vg/vMPXpaXHP3FWsBx2Rne/lUhg7dKqkgSZ+BBDsix5abtPuH8J6ksRd+iLCxHFNyG5XUkJIRWJWb4tfpuDvBXjFp10rZmehY+Ei16dBPtgKbrR2JtmpZ+mw8RhDvcDghS/64r7s/mI43G4xDR/GNfa2VS0w12SPCq/QIN1p4/Q7nf6J/VSkEMaXVditdfG2DArMZBNXOls+pjiFEgUlbNqKr8w+d1+RF+L1Bo2vZ3Uige0onolbt8sx/vxkWOVyh+LUfcRUnCKIFJVF8cvy4oYUKIYTfiDJcvgQQd86kg5O42I7uDtYH5+pNtcQX4vJopJZQv7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cjP6KZAdN18Ayzcw+VYjsBh+lBRJyJFtgmNXEjUOicY=; b=nyYyOOw+5bvquB9wCi3BWZubYEFLmgJnLzE6e/LI/DyfbnOT0u1ALaed76jAjz8WqbfEJfmEonsRn0jlhhM0L5aWGQv813lWQ7eYXn2pU1pC6zLT4Av68R0DGe3uMMdMYVcv9alko8Udqp7UMl37OUTnRnE7LpP2ZyLA+N3yFspyqnhw7SmJuNAvlnj8pW1CE1sToMEm471tG0xSvcGA0Vosn1fUIjblKPBuSKA5JPXe/QlFkFVg6ThFjkYbNW3m1e61295YVaj/b4DGDiURMDLR3xbFIiOEz2mS+AuN0Bs7iKtTG9Kwceljko1m5iQNr9NgVWJgPkZpw3tLGSkESA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cjP6KZAdN18Ayzcw+VYjsBh+lBRJyJFtgmNXEjUOicY=; b=Ji+accCIGrkX7wAfOq9ElQPIy+JX/fEZq3jffuudXZM4cGz2v9911+smCdaDr3zLn7p65LqcmxLU+wS3cBlqTlPqZlEf94sYfUoqIZS5Yp9dt+ZXEYkuvRxDp4Hst7b+ow9qTbowTIPvET0Vf/PQAeG0aeJoP/JvzFsDzeeqnIFJsiowrzKDr0IYkd8SxXPF7+/sFUPbHac9kSsiFxBrw7VqMcMPgy7y72B8CvZ/4ctP1ibe1e11Dlz/o243cyLkB0sNMGe5R97KQy60Kwr9IR/PyMe6oUdnvtTWoLNa3ptzX2bpxQl8D1D+3eKkA/HKT+S2FbchYPw2ykKkPQdfuw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY1PR12MB9560.namprd12.prod.outlook.com (2603:10b6:930:fd::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.29; Mon, 16 Jun 2025 12:10:33 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8835.027; Mon, 16 Jun 2025 12:10:33 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Johannes Weiner , Vlastimil Babka , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v9 5/6] mm/page_isolation: remove migratetype from undo_isolate_page_range() Date: Mon, 16 Jun 2025 08:10:18 -0400 Message-ID: <20250616121019.1925851-6-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250616121019.1925851-1-ziy@nvidia.com> References: <20250616121019.1925851-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR15CA0014.namprd15.prod.outlook.com (2603:10b6:208:1b4::27) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY1PR12MB9560:EE_ X-MS-Office365-Filtering-Correlation-Id: 33a1246f-959b-4391-b9ff-08ddaccecb27 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?mwXp8W4A+oSStAr+drINPqhp/Wjv/mEwkq6BZXzTRECoxadp/r70bbk5blDg?= =?us-ascii?Q?uXZDTTn2A+kgKfaTFHc3hzHXTw5jsWsYwGBl2ZvvQO0viEAwjpn0M35ktQvi?= =?us-ascii?Q?Lq4cQtavYYM2+lasui9KxzTUE0nt7e8rw7LSAS3RnEB9LEm72VkmKYoaiGfL?= =?us-ascii?Q?SXtBROUow+NnpFxuC5iBqRPZy3yD1+VA9Ongf+bx3lsRUPP+YpdhVCWZMIpN?= =?us-ascii?Q?OEzqQnIcCvbND2qtcpgiR9l2Gc1NUtTaUxgSiSCwwb/V5Vpqx0iOqkBIEHik?= =?us-ascii?Q?DxC9ME/rCNrFTEkZRAm9CdJ5mTkg083w5l0Zz6buxYwVqHZhzcNStUWI9Hjy?= =?us-ascii?Q?JnLHZ54WnZn/q6UiMEY1RPd2G1wgj74K7PYUmyLqCsiHasw/0cx07EcDf2WJ?= =?us-ascii?Q?MScxiuGPWGVKWyrb1u+TTT35bUfvpXR3q5BQ08E8bL6lMBXSAnicfX+iScEc?= =?us-ascii?Q?EfWyUt728GzKI7ffbUM5lqyMI2F0dJqrOjgGfLhsuLOvmg8QPaL9ZUujv6TE?= =?us-ascii?Q?g3eiQPoDS+RwgEBkVZ0catISXl00DmSna9+o3CSaLwqO7ruk0I/6+iiWLnTN?= =?us-ascii?Q?hwjU1Nv0G315syvRfcTM3F6SD9MJqRryW9MHDLgnTnX0kswa+I31+zd/67nd?= =?us-ascii?Q?w5neykwcNZNqM4a0FuoyzgPYkRxa6EAcmXFBlozpDBlmXAX8t5Sy63tCYi1g?= =?us-ascii?Q?R/GOHXDzx+uNFjHnVVPfdVJKuHRyJ+RSm0XT6/Sl8UiNbP++hBO8hr6gP2S6?= =?us-ascii?Q?XBe+6FtpEhGoSal5wJJQb6zjOlduiZ3+AEZmEgXPp5M4gioLDx/BTMeojD86?= =?us-ascii?Q?RYRS3I7VnMuvhmdbO2cIhlVeXjMUooYLYvvgzkiEkOAuuAn8i4ynU7VimyOI?= =?us-ascii?Q?sa4irUvsUNOzZg5mJQVhZ2hfR6sYub34NRYDypBQy8KlgRsWI3QahL4pobHZ?= =?us-ascii?Q?S+XJPUnOK2XVSrF+bUt1gRFmQOU+I6FvLEm41Kz3q6YSwEorsRcFjrL3bhR1?= =?us-ascii?Q?l//mMAycoz6g5RQd9yBe+UiSqXG7PhVg5VHh/dP0w2EsFHfWBl6gHAykWM3y?= =?us-ascii?Q?n+wNBr69qc0Wnsf0+9HjYhtD0GrTPD+7iLXtHwMq/ZG+b02HNGJbc1kaflH6?= =?us-ascii?Q?Nv1HyKasahLCYs7GdnJaZsTF2HUf1yYPubojBxtf/JPPBWRKQifWKLEcLdQv?= =?us-ascii?Q?VBQO8beMPRKI6RDVzsoX5MOlMRW74MsU2K3vNjlcT5Z6Xqp+iBVuPZPCCvqt?= =?us-ascii?Q?jGLy8wYI3UPAfoKwoIpgS5PsIQzQ3UHOMBzXA3nw8xpCTS6zQXuNWCNoWbC0?= =?us-ascii?Q?TE1w8Ik8HNWbF/bHL/aEH4t6S8MJVc6HX/y3Pox7rB+7qxG0wuVV1G+2HKJ6?= =?us-ascii?Q?KDNjp01yuDtaOFS4la62bwZ3QvoMZfHXwmZ9C31NdyVPdBMpB4X1oBu+x3EJ?= =?us-ascii?Q?GNGirH/H2+o=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?6tMcEpDl+q3QxhqNNMwZWsS2thyhzQ+Ky0RYLsOkiBHiaQ7pQXQ/2jFMyf/F?= =?us-ascii?Q?9uxLJnY8vTRlQjWbv9QwNVYx3XZdxve1WbanZTAD1X9dX+ojPH6QSoTppf5m?= =?us-ascii?Q?PGB4Idd7JwEyyEehF2RgHbVT0U9ZIziOQ+UbaXiyhX0LxGFLsAlaG98BYxxG?= =?us-ascii?Q?kqN8cKszhTUaRlDl20NwwpO0VlbW7RuRLZLbuE8SXKg78++nWnkPbzo/KLqc?= =?us-ascii?Q?Qf36SCDJCadUiJtgr4ZPgVhDNHoDsGuKcMI4MBt/jlB9npqQ2Fr/uB0ne8QD?= =?us-ascii?Q?opfKicPYXv7uz9ObImHUxETIx87rM6+r9T2eNXcukVcWbOWLRepX8+vJxA2m?= =?us-ascii?Q?r9lNsRNqir2iYMCy4vT0q+k3CVGJk8mb0MQc2467VV08dEf0mvUZWhniDav/?= =?us-ascii?Q?JMHMkWpavLRN99EGk6gLZdhfv7YePRzUQhep7MzMRIDSvVFZBlC+tN2R/x2H?= =?us-ascii?Q?cCRZw75lrtCyG9NL2HIdm+iBYHmqv0EjljlsgS55Nv2A50oHt4Lo5ZDAfJA5?= =?us-ascii?Q?2cKD1/uDWH5IWGZWZou+0bJR0CfmLiRqwZ1HHcND1YAPnK3jJTDxCPqMg6BF?= =?us-ascii?Q?2bshrh8EIQeyYqU2m9Miz/AAgj2sRkCWlYdhXpAN33UsuHDqHBYR7EICyrI5?= =?us-ascii?Q?rjVbm+XT/bnbq6gKtCCBV3A2l3XqqkRwfL8rnzzuaPmW1INBgvYrDswlHXwQ?= =?us-ascii?Q?zbA0kScH+wf9qvxm1O6ytxsIKLxoB5SExDv5f7InZkdBNm1rfD2Tft2YHkl/?= =?us-ascii?Q?Xxlz2wDRkx3BA/UPxI4sE6X8qppMCuRSnz4ZMkRhS5Uav/Px/P0TM/zHZgOI?= =?us-ascii?Q?LruHty1koQPBX7sIxzu5wXKQB1lbhmdSE6WubKvAXUW7vRgoFr/cemmk5gg8?= =?us-ascii?Q?7k7GoKtbB0ezhf5+AtT6plOeJWs3nvsj6BQ37QrzZ8i4W+DC0xUFQZipco21?= =?us-ascii?Q?uNHA7Pm3CYgYL1+w4wL5mVOMXKNYLC6Eb0FwGy1t4ScqJPaFVfi+sVP8Vj+K?= =?us-ascii?Q?xHWhcCz0X/2hpKVcsap8IReQ8PQ3jj30PKQJf4aysPkgk6VEQI8tI3C1yeKD?= =?us-ascii?Q?fU5QCgZFZN+ccJIhs8SGDYIZTeFxCX4QVPt7xWZFXQH9tzaLVZoqeFbnwJ66?= =?us-ascii?Q?PA1U73aoH9wsRl47q+hzw15fBswc17IoAfEIYOE40aXaH/79H50N/ERNeOnl?= =?us-ascii?Q?CIVYyWLoMytgcBv+B6nuowfnjMfAnBB0qFFrAy+1ier55S3yzeGpc3OhBHT8?= =?us-ascii?Q?bI8a9dDKTVm+81pnPR946xn/Ac28mK6jEU9PBv9btvE9hvazyCJ3ZDzEtFz5?= =?us-ascii?Q?S3NJeYmMheBLKdUz0/9ajVsoJkSBT7RPBjhwLOQ9nWiroMhFpmeXKd0TCWVC?= =?us-ascii?Q?gcmN4u4O4Zi/cDrZSPOmLzvwu9qUtDB+XBE9zM6nTCujtdhLmO+uTmfVLIKu?= =?us-ascii?Q?O0ghiGvsg/AjkdH4eZUIjJBtKe/Kfut5gDWJFWoj6YUtZ03qmVsmWT8tDaWF?= =?us-ascii?Q?MV1wYH+zKRQ87W/24Q0CQy9Ft8IGkGOwGZMeXjym9qKATw3/SvxZb9PBEheY?= =?us-ascii?Q?abN2FV78vctXVZEjcjnnlZg1CCATyMai9lltKWrM?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 33a1246f-959b-4391-b9ff-08ddaccecb27 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2025 12:10:33.0830 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: utYqdVgtuAi2FGsokGvTcBGea7VtWftFsZK0G02IvRahL08oZVJM8Jk73ztTPhZe X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR12MB9560 Content-Type: text/plain; charset="utf-8" Since migratetype is no longer overwritten during pageblock isolation, undoing pageblock isolation no longer needs which migratetype to restore. Signed-off-by: Zi Yan Acked-by: David Hildenbrand Reviewed-by: Vlastimil Babka --- include/linux/page-isolation.h | 3 +-- mm/memory_hotplug.c | 4 ++-- mm/page_alloc.c | 2 +- mm/page_isolation.c | 9 +++------ 4 files changed, 7 insertions(+), 11 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 7241a6719618..7a681a49e73c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -51,8 +51,7 @@ bool pageblock_unisolate_and_move_free_pages(struct zone = *zone, struct page *pag int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, int migratetype, int flags); =20 -void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype); +void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n); =20 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, int isol_flags); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 35f008364698..224987824a84 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1223,7 +1223,7 @@ int online_pages(unsigned long pfn, unsigned long nr_= pages, build_all_zonelists(NULL); =20 /* Basic onlining is complete, allow allocation of onlined pages. */ - undo_isolate_page_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE); + undo_isolate_page_range(pfn, pfn + nr_pages); =20 /* * Freshly onlined pages aren't shuffled (e.g., all pages are placed to @@ -2092,7 +2092,7 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, =20 failed_removal_isolated: /* pushback to free area */ - undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE); + undo_isolate_page_range(start_pfn, end_pfn); if (cancel_mem_notifier_on_err) memory_notify(MEM_CANCEL_OFFLINE, &mem_arg); if (cancel_node_notifier_on_err) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 06cc990e2a62..64321f6b714f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6984,7 +6984,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, start, end, outer_start, outer_end); } done: - undo_isolate_page_range(start, end, migratetype); + undo_isolate_page_range(start, end); return ret; } EXPORT_SYMBOL(alloc_contig_range_noprof); diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 08f627a5032f..1edfef408faf 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -515,7 +515,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, page =3D __first_valid_page(pfn, pageblock_nr_pages); if (page && set_migratetype_isolate(page, migratetype, flags, start_pfn, end_pfn)) { - undo_isolate_page_range(isolate_start, pfn, migratetype); + undo_isolate_page_range(isolate_start, pfn); unset_migratetype_isolate( pfn_to_page(isolate_end - pageblock_nr_pages)); return -EBUSY; @@ -528,13 +528,10 @@ int start_isolate_page_range(unsigned long start_pfn,= unsigned long end_pfn, * undo_isolate_page_range - undo effects of start_isolate_page_range() * @start_pfn: The first PFN of the isolated range * @end_pfn: The last PFN of the isolated range - * @migratetype: New migrate type to set on the range * - * This finds every MIGRATE_ISOLATE page block in the given range - * and switches it to @migratetype. + * This finds and unsets every MIGRATE_ISOLATE page block in the given ran= ge */ -void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype) +void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n) { unsigned long pfn; struct page *page; --=20 2.47.2 From nobody Thu Oct 9 20:24:37 2025 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2053.outbound.protection.outlook.com [40.107.220.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1D65292B5E for ; Mon, 16 Jun 2025 12:10:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075845; cv=fail; b=mlSCjXjAv6IL0ZQ/91/aEcnxBBczdPUWr2TzFpoO8fa14Lqgof+aFTDkFaTYPeIXzXymbdkEPNHaOwbukKMI8ClUWde+PP90gUCHsM/OHIqBhSJkbjIq0+t9sNKHXO/tVQxxUPEEBtP+P4pt63t+Fk3JIpwv2VCmoHuqtN5HGXg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1750075845; c=relaxed/simple; bh=TWLr1IJU4hz3dpukg3JsvLk3Xeq3laYv9YFOlk9NsvU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ODRrlxQRYOBKnPPh3V7FkhvMw2bQcxYJMYj0iUEyJZrJ6YxbUfInh+uzFHi7vH8RllUgJjSqzZncidcCv/bTz0wHl4ur29562OaFBlMzie1gsgi4nmoVpzXb+fPdPI+156t3ym6za+Ud7lNn64qwodrutv7udy13jV9YKKcon3E= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=WAXZCh2w; arc=fail smtp.client-ip=40.107.220.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="WAXZCh2w" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wnYDIBU6n4HTR3ZYFsbpUV+Bzg6s23FLXlNWgbXPxoE4cOnzmcHZdAgT6zILODMVQfMMlj4k5kCBfWGmoAdgTjm1hT8yA/QVP8ojwUVzQW5jcf1lgLoxvhHxO10tqUBMDXBRu6MQyimM8Vf04oRoTf8Wf0STtYMYB5/Vyj3aXUN5Merk80vmYGendxQv1bxFbjHmt+FYMO48Vgly8FLTuQITdiFOQzNPwSOIsR1b9Rrkc5ZYQepFESfA0Eh6mBmsPP2c6fnz+Z/xGMxK6YKG+klny9RbOqyYx7tgetO6WCScKvTsfYh/V6BWikAlN6P34qId/w7tdDeM0wp8K2Mc1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xk1w4mfxIjRWH63CBLYERyUbuHzBZIgXXm6bHwmDfmc=; b=jMYE+remwFDrh/zHfhcklqcKn4nWPCw8GAQ+urLhEvWA8TxCp+acMftft6zRLdTuF6BXg7UvFk5gXg5TvdpzeS7tnJHBUVrQCTXo3hgJKfD2123wqS57uf2iCdpoqFMjvxknPyvErol6ojmvlSNa0SBRsDa0bwJTDZLiwDZZ7DBq3W9xWenDDZHOR+Dhzdkxcmxr09DT6ipagad3lIFWZBSau/gVehT+yRDp2znZDLTRw7uEEfuTW7bEQFPcORq/pt1GDIDvCLYO589zlOMo75bcmjAUuPUDEAANP7uSzUKZBVPjSMN0hItdVU6+MVZRCttkRZ+c13LJoGo2qAsZfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xk1w4mfxIjRWH63CBLYERyUbuHzBZIgXXm6bHwmDfmc=; b=WAXZCh2wolcCV2ijRbKPSYsnaUjpb1FAeGC4TpfzNzU/3NdrT5l0vM30htq2wU3M0Tdho0GjMdlVQTwj/89MLeYNm35OTk9iER+ultu673PMEQ+fP0VMLTCjrsOkm4xK6K1pdzXeuak4qrG8t1bjaT3ytAHVlZg30048P8c2z0gOW0ZBmxb2kPP5Gxdw8aZxt46xJDfN8KDvMGeFoZ1tycP95tHBhWEkU3mkyaxHQw0i0Z0UxJWKIm9+VT0VE1EXJW7nnxW46UGaj0OlLzVN799ol/meCBcH+ybzOM/wDSsdNj6fJzhmb/anFwlv4SJXgSeH+OiOZsZ17ChQL++Ijg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by SJ1PR12MB6363.namprd12.prod.outlook.com (2603:10b6:a03:453::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.19; Mon, 16 Jun 2025 12:10:35 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8835.027; Mon, 16 Jun 2025 12:10:34 +0000 From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: David Hildenbrand , Johannes Weiner , Vlastimil Babka , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v9 6/6] mm/page_isolation: remove migratetype parameter from more functions. Date: Mon, 16 Jun 2025 08:10:19 -0400 Message-ID: <20250616121019.1925851-7-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250616121019.1925851-1-ziy@nvidia.com> References: <20250616121019.1925851-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR15CA0021.namprd15.prod.outlook.com (2603:10b6:208:1b4::34) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|SJ1PR12MB6363:EE_ X-MS-Office365-Filtering-Correlation-Id: 38c164cc-98e0-4dab-82e0-08ddaccecc36 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?bqWRK1CE4Q8tJgjnbi603SOmRpvpijLzs/AIwkjm+yY21ubInV5NSLAUx6Iw?= =?us-ascii?Q?yHTGh8vX6jnxLTh69CK6yycbh0M9+lKlRyOLc3+lCFaTkMij7B5HVIA43DMv?= =?us-ascii?Q?zVQ9juOHf9WT5C6SZOGn/fyc2B81klf66WoRy9zYjCNgD+lFoNx8OQO71D50?= =?us-ascii?Q?mka11kju6O2UaBU3HYAic0ukgMos68DALj62V63ZE1LdTsbTBCwL+18yiuhf?= =?us-ascii?Q?X8qmuG7i+M5wGB7rx0LLYcYUrxWVE1Ln1NDlQX3oVZNFfqWTqFoLkVIxFI1K?= =?us-ascii?Q?d3n1Wpv4M51HPtRpOBPSEieJeu8e8ZrQkIm2YesUvt4VLemb9lIIpQK4zw2M?= =?us-ascii?Q?aQ5VWJdRWsUH3R7AZURsXtDNEF9kFaNQFflbCwQl62nzRjtu5oTOBxvvL/ou?= =?us-ascii?Q?Nr9Z2egIL8V374tVYcN6c9ufompgDX+e2UA3/iJCdZWz6aETWMcfkXVpVfce?= =?us-ascii?Q?Cn75pApDzN5CNWmI7coN47KAh4RZhdWlfKv4z+EiFvQsJ71nCEAnwShlJoBn?= =?us-ascii?Q?CM+cARmTEI9+LO8e46uoTjNmZrHxVLSEOJrlI50VtTpWoQBGTKqrS5O0GySy?= =?us-ascii?Q?J7ooRHag3suPT//+br1yQqe4X6QN2Dp8aN66TIJ84FHZu155ZFEsFjs0p2gx?= =?us-ascii?Q?9jzjLgR51zaNZ/8KdD870sdzTG3oQI3IQxXGvaP4Phu7AL7vA4NYG+EaZKL7?= =?us-ascii?Q?TBZPcupfi5xU55hfb4PStsItWoTEg6tdVc70EqYfSoRGvV9h139E7PvENUA2?= =?us-ascii?Q?wsKF3P7XjF73PcTpZDO3aHZ7L3wzjsiX6tOyNmmGadViQo68rOfMAR+uexnH?= =?us-ascii?Q?oNo4b48x8weKylPYv/j0AJpgOaYqNDEeX1f4KU95HSoNXWVQ4iVo0MgIMkZD?= =?us-ascii?Q?t68IxiKQSSPNS7u66G8G9R7FiCTAguaqkrb5fI//ftUXjeg5mGIlQXqIJvtw?= =?us-ascii?Q?OpXDL8P8HdvIx4FwICQzm5885x0ru3/AsF+gTKMZvP6DVqQB9qQdDgthfaP/?= =?us-ascii?Q?iowXZoEfGuycozLAEcyrIOtyvHDx5ZDGS9zfjIzz6g9T+gdQautkDSDbFVff?= =?us-ascii?Q?3omFSCy/UF3Wv9F39XXl0xUqWLcDGyyJZKYzMuILkhxQfSn6MN+7ApvpLWno?= =?us-ascii?Q?aq4kDfR/cL3XY1tschKpHErJMQ8RxeVVklr2CKNXjZy7Py3alJ33vOOKYquZ?= =?us-ascii?Q?7cPaQOqpF9AkDqFXBE/73KdxMVyi/F3OTHeLefvp0fT0glz8N6b9DOXajv+x?= =?us-ascii?Q?Fpw0nl5ZVz8QuRvTVTr4IvAriA4o/50qzTvnezKRW1CDfrse1iWmajpJR8Mk?= =?us-ascii?Q?hQ8jSqA3D3Ckk6Q1EruwHiJK2xRS7tNsggi9YrtlVrJrSnpr6wZFuzAilIM6?= =?us-ascii?Q?mFjI9EBFMLJS+VzTAKzYzF/Fd0MTGJPDx05yPktMg2vusczbm/GsD2MWolrv?= =?us-ascii?Q?04r17bOELuo=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(7416014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?+haXXs0hsfJQvkl1ez0nZ06VGTivU0YRUnB6hP4ptGuoAl7BAeHTwLO6qqd7?= =?us-ascii?Q?6Lp5CUVOboJ0+oV80krSxvSOa0PVv7plzAajKx1giqSrvAf/uipu9H1hPW/9?= =?us-ascii?Q?u8tLQhZLqrePG7PmLLa8YViO4mqMqRx2K9CgBmapr3cXMsKkOqat+Rumaq1K?= =?us-ascii?Q?TBYkg++KzYiD0/MVRg4DX+0okLV6GInZvFNSmXMIXfGsr5DMOp4asz+GqeeS?= =?us-ascii?Q?9A5H23onM2N7eF/OiBtiBlxXTFNQzefPQdup5uD82m9KC5FkGpXyQaLSy+qE?= =?us-ascii?Q?fJRfg9YTGJoksrsyFDVPve/IvUx+GUDM2W/rNkSmtUI910jAMSKWSIY/WtTn?= =?us-ascii?Q?Qy44jXbydjRxfpBgwCAfGZgAszN+W7yJrbs1EBd6siUW7b7juCSesvSici0s?= =?us-ascii?Q?ovMzZWNo5+f0neGwXJ0SAXZ9dZ+mug8d7ElL62l4ejDd+DD87dAGb3x6LM0T?= =?us-ascii?Q?4elaOcFETXHIY+bcGs4FTbiWGkh7Z3orGMDzDas0VEiqQVxlX8UGWo8+X02Z?= =?us-ascii?Q?DYYbg4TLHnSrGIAghOwMnmw1QZKqWdr8lisSLpj0L7OtCiE0kvVqQnxkMQXs?= =?us-ascii?Q?3JvpIRfwhf91GebJ0B5P/1Z/MAbMp90Lsk3+cMZf9Iue8uGXjXeNZgbsd6qX?= =?us-ascii?Q?mMyCyEP6ZyGJVaqSgKAeLimGqIQXi05a7W97kfP+FU0btr58Pp5vC6nBPECt?= =?us-ascii?Q?Oq2upTVvtBxL51RNPFlXkEffa8bpDoxegV5Hj/pvIbweLJv9fAdOrROfVDEd?= =?us-ascii?Q?XE1sxlQcLjd4PU1NCT/feOhNXUW/RUJq1BgJnvnVOGeyGQqg6QciE0b5RSup?= =?us-ascii?Q?PIFdz4kNoyH/MTUQXT9Nk9foWMSE0diDtjfOt95RnYI7kW94x/OI7GlczJ2D?= =?us-ascii?Q?ara1pol7Z9C3on7PN6QkYjJuxlZagwSYI/t4Vk4Qo7GEe/C3vSXZ4bnzONYR?= =?us-ascii?Q?cjr0F8ikA3AGeh2CZAlau5yHeRy7LbHB1UnDXRZ6iIhwKnF6dEKyfuVq86K5?= =?us-ascii?Q?tXQ9k9pZiK5Mrp9a3zQoq0mbWmqHUMF4roKWzHEAUIx0ViYIRcLsSttl5CAr?= =?us-ascii?Q?FsobUJ7CMuEAdqx8IlUo5xsxAonBM920FkmLeIcEpeF/CV6D7kWLYmSY7TnQ?= =?us-ascii?Q?abWd1QdWlZFiS2l6fYkMc6VEBC5+pO3met61yE/iiaheOWPm7bxXcVrUEg/i?= =?us-ascii?Q?sd0M1Bie5rvjbU0QnFq+ZPwuOrC+MIoEG/xnJoPtnB3w7ITxivpTYQfaMANF?= =?us-ascii?Q?BV4aKh1uaDeXR8J9+lWwW4nAGPjOpjQjWxGoIDoJmfFOkEW/vzUD+Lf/289U?= =?us-ascii?Q?jh8SA7nqyjZdO1Gxf8Z+2zbr3Qzp/XWvAD6Hlx4hnIRUltgaI5+eRaY07xY4?= =?us-ascii?Q?RIo2+Z6YuIpTC69LpJ37O2Uk+u7nlLNkjya6vK89w5C8iULGwcTu6eAfs9pP?= =?us-ascii?Q?BcSVzCb6rzyJH+D6TyDINvKoUPeEmBaUh0ZNquRTval182rlgJgRQVADsgbS?= =?us-ascii?Q?zyrg/GRE6jbrkqdPxNd6kHCbbmcijt7qktJ5HIOd9UmDolBAwmeLfq70xXRp?= =?us-ascii?Q?CXFpwVHhAnG21jRYUl1Zr+FGprEt3bO1kzFnFqpy?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 38c164cc-98e0-4dab-82e0-08ddaccecc36 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2025 12:10:34.8858 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TpBG+xmki6+hXSK2DCxRvBETIZYUYZlwfCGRQGMBjV4E96sgB/s1IgYAQdw+dP/a X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6363 Content-Type: text/plain; charset="utf-8" migratetype is no longer overwritten during pageblock isolation, start_isolate_page_range(), has_unmovable_pages(), and set_migratetype_isolate() no longer need which migratetype to restore during isolation failure. For has_unmoable_pages(), it needs to know if the isolation is for CMA allocation, so adding PB_ISOLATE_MODE_CMA_ALLOC provide the information. At the same time change isolation flags to enum pb_isolate_mode (PB_ISOLATE_MODE_MEM_OFFLINE, PB_ISOLATE_MODE_CMA_ALLOC, PB_ISOLATE_MODE_OTHER). Remove REPORT_FAILURE and check PB_ISOLATE_MODE_MEM_OFFLINE, since only PB_ISOLATE_MODE_MEM_OFFLINE reports isolation failures. alloc_contig_range() no longer needs migratetype. Replace it with a newly defined acr_flags_t to tell if an allocation is for CMA. So does __alloc_contig_migrate_range(). Add ACR_NONE (set to 0) to indicate ordinary allocations. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 2 +- include/linux/gfp.h | 7 +++- include/linux/page-isolation.h | 20 ++++++++-- include/trace/events/kmem.h | 14 ++++--- mm/cma.c | 2 +- mm/memory_hotplug.c | 6 +-- mm/page_alloc.c | 27 ++++++------- mm/page_isolation.c | 70 +++++++++++++++------------------- 8 files changed, 80 insertions(+), 68 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 56d0dbe62163..42ebaafb9591 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -1243,7 +1243,7 @@ static int virtio_mem_fake_offline(struct virtio_mem = *vm, unsigned long pfn, if (atomic_read(&vm->config_changed)) return -EAGAIN; =20 - rc =3D alloc_contig_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE, + rc =3D alloc_contig_range(pfn, pfn + nr_pages, ACR_NONE, GFP_KERNEL); if (rc =3D=3D -ENOMEM) /* whoops, out of memory */ diff --git a/include/linux/gfp.h b/include/linux/gfp.h index be160e8d8bcb..ccf35cc351ff 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -423,9 +423,14 @@ static inline bool gfp_compaction_allowed(gfp_t gfp_ma= sk) extern gfp_t vma_thp_gfp_mask(struct vm_area_struct *vma); =20 #ifdef CONFIG_CONTIG_ALLOC + +typedef unsigned int __bitwise acr_flags_t; +#define ACR_NONE ((__force acr_flags_t)0) // ordinary allocation request +#define ACR_CMA ((__force acr_flags_t)BIT(0)) // allocate for CMA + /* The below functions must be run on a range from a single zone. */ extern int alloc_contig_range_noprof(unsigned long start, unsigned long en= d, - unsigned migratetype, gfp_t gfp_mask); + acr_flags_t alloc_flags, gfp_t gfp_mask); #define alloc_contig_range(...) alloc_hooks(alloc_contig_range_noprof(__= VA_ARGS__)) =20 extern struct page *alloc_contig_pages_noprof(unsigned long nr_pages, gfp_= t gfp_mask, diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 7a681a49e73c..3e2f960e166c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -38,8 +38,20 @@ static inline void set_pageblock_isolate(struct page *pa= ge) } #endif =20 -#define MEMORY_OFFLINE 0x1 -#define REPORT_FAILURE 0x2 +/* + * Pageblock isolation modes: + * PB_ISOLATE_MODE_MEM_OFFLINE - isolate to offline (!allocate) memory + * e.g., skip over PageHWPoison() pages and + * PageOffline() pages. Unmovable pages will be + * reported in this mode. + * PB_ISOLATE_MODE_CMA_ALLOC - isolate for CMA allocations + * PB_ISOLATE_MODE_OTHER - isolate for other purposes + */ +enum pb_isolate_mode { + PB_ISOLATE_MODE_MEM_OFFLINE, + PB_ISOLATE_MODE_CMA_ALLOC, + PB_ISOLATE_MODE_OTHER, +}; =20 void __meminit init_pageblock_migratetype(struct page *page, enum migratetype migratetype, @@ -49,10 +61,10 @@ bool pageblock_isolate_and_move_free_pages(struct zone = *zone, struct page *page) bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page); =20 int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype, int flags); + enum pb_isolate_mode mode); =20 void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n); =20 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, - int isol_flags); + enum pb_isolate_mode mode); #endif diff --git a/include/trace/events/kmem.h b/include/trace/events/kmem.h index f74925a6cf69..efffcf578217 100644 --- a/include/trace/events/kmem.h +++ b/include/trace/events/kmem.h @@ -304,6 +304,7 @@ TRACE_EVENT(mm_page_alloc_extfrag, __entry->change_ownership) ); =20 +#ifdef CONFIG_CONTIG_ALLOC TRACE_EVENT(mm_alloc_contig_migrate_range_info, =20 TP_PROTO(unsigned long start, @@ -311,9 +312,9 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, unsigned long nr_migrated, unsigned long nr_reclaimed, unsigned long nr_mapped, - int migratetype), + acr_flags_t alloc_flags), =20 - TP_ARGS(start, end, nr_migrated, nr_reclaimed, nr_mapped, migratetype), + TP_ARGS(start, end, nr_migrated, nr_reclaimed, nr_mapped, alloc_flags), =20 TP_STRUCT__entry( __field(unsigned long, start) @@ -321,7 +322,7 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, __field(unsigned long, nr_migrated) __field(unsigned long, nr_reclaimed) __field(unsigned long, nr_mapped) - __field(int, migratetype) + __field(acr_flags_t, alloc_flags) ), =20 TP_fast_assign( @@ -330,17 +331,18 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, __entry->nr_migrated =3D nr_migrated; __entry->nr_reclaimed =3D nr_reclaimed; __entry->nr_mapped =3D nr_mapped; - __entry->migratetype =3D migratetype; + __entry->alloc_flags =3D alloc_flags; ), =20 - TP_printk("start=3D0x%lx end=3D0x%lx migratetype=3D%d nr_migrated=3D%lu n= r_reclaimed=3D%lu nr_mapped=3D%lu", + TP_printk("start=3D0x%lx end=3D0x%lx alloc_flags=3D%d nr_migrated=3D%lu n= r_reclaimed=3D%lu nr_mapped=3D%lu", __entry->start, __entry->end, - __entry->migratetype, + __entry->alloc_flags, __entry->nr_migrated, __entry->nr_reclaimed, __entry->nr_mapped) ); +#endif =20 TRACE_EVENT(mm_setup_per_zone_wmarks, =20 diff --git a/mm/cma.c b/mm/cma.c index bd3772773736..acabc00d1278 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -822,7 +822,7 @@ static int cma_range_alloc(struct cma *cma, struct cma_= memrange *cmr, =20 pfn =3D cmr->base_pfn + (bitmap_no << cma->order_per_bit); mutex_lock(&cma->alloc_mutex); - ret =3D alloc_contig_range(pfn, pfn + count, MIGRATE_CMA, gfp); + ret =3D alloc_contig_range(pfn, pfn + count, ACR_CMA, gfp); mutex_unlock(&cma->alloc_mutex); if (ret =3D=3D 0) { page =3D pfn_to_page(pfn); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 224987824a84..b4ac226795f2 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1961,8 +1961,7 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, =20 /* set above range as isolated */ ret =3D start_isolate_page_range(start_pfn, end_pfn, - MIGRATE_MOVABLE, - MEMORY_OFFLINE | REPORT_FAILURE); + PB_ISOLATE_MODE_MEM_OFFLINE); if (ret) { reason =3D "failure to isolate range"; goto failed_removal_pcplists_disabled; @@ -2039,7 +2038,8 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, goto failed_removal_isolated; } =20 - ret =3D test_pages_isolated(start_pfn, end_pfn, MEMORY_OFFLINE); + ret =3D test_pages_isolated(start_pfn, end_pfn, + PB_ISOLATE_MODE_MEM_OFFLINE); =20 } while (ret); =20 diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 64321f6b714f..8eb6f93aaf2c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6693,11 +6693,12 @@ static void alloc_contig_dump_pages(struct list_hea= d *page_list) =20 /* * [start, end) must belong to a single zone. - * @migratetype: using migratetype to filter the type of migration in + * @alloc_flags: using acr_flags_t to filter the type of migration in * trace_mm_alloc_contig_migrate_range_info. */ static int __alloc_contig_migrate_range(struct compact_control *cc, - unsigned long start, unsigned long end, int migratetype) + unsigned long start, unsigned long end, + acr_flags_t alloc_flags) { /* This function is based on compact_zone() from compaction.c. */ unsigned int nr_reclaimed; @@ -6769,7 +6770,7 @@ static int __alloc_contig_migrate_range(struct compac= t_control *cc, putback_movable_pages(&cc->migratepages); } =20 - trace_mm_alloc_contig_migrate_range_info(start, end, migratetype, + trace_mm_alloc_contig_migrate_range_info(start, end, alloc_flags, total_migrated, total_reclaimed, total_mapped); @@ -6840,10 +6841,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_= mask, gfp_t *gfp_cc_mask) * alloc_contig_range() -- tries to allocate given range of pages * @start: start PFN to allocate * @end: one-past-the-last PFN to allocate - * @migratetype: migratetype of the underlying pageblocks (either - * #MIGRATE_MOVABLE or #MIGRATE_CMA). All pageblocks - * in range must have the same migratetype and it must - * be either of the two. + * @alloc_flags: allocation information * @gfp_mask: GFP mask. Node/zone/placement hints are ignored; only some * action and reclaim modifiers are supported. Reclaim modifiers * control allocation behavior during compaction/migration/reclaim. @@ -6860,7 +6858,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_m= ask, gfp_t *gfp_cc_mask) * need to be freed with free_contig_range(). */ int alloc_contig_range_noprof(unsigned long start, unsigned long end, - unsigned migratetype, gfp_t gfp_mask) + acr_flags_t alloc_flags, gfp_t gfp_mask) { unsigned long outer_start, outer_end; int ret =3D 0; @@ -6875,6 +6873,9 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, .alloc_contig =3D true, }; INIT_LIST_HEAD(&cc.migratepages); + enum pb_isolate_mode mode =3D (alloc_flags & ACR_CMA) ? + PB_ISOLATE_MODE_CMA_ALLOC : + PB_ISOLATE_MODE_OTHER; =20 gfp_mask =3D current_gfp_context(gfp_mask); if (__alloc_contig_verify_gfp_mask(gfp_mask, (gfp_t *)&cc.gfp_mask)) @@ -6901,7 +6902,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, * put back to page allocator so that buddy can use them. */ =20 - ret =3D start_isolate_page_range(start, end, migratetype, 0); + ret =3D start_isolate_page_range(start, end, mode); if (ret) goto done; =20 @@ -6917,7 +6918,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, * allocated. So, if we fall through be sure to clear ret so that * -EBUSY is not accidentally used or returned to caller. */ - ret =3D __alloc_contig_migrate_range(&cc, start, end, migratetype); + ret =3D __alloc_contig_migrate_range(&cc, start, end, alloc_flags); if (ret && ret !=3D -EBUSY) goto done; =20 @@ -6951,7 +6952,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, outer_start =3D find_large_buddy(start); =20 /* Make sure the range is really isolated. */ - if (test_pages_isolated(outer_start, end, 0)) { + if (test_pages_isolated(outer_start, end, mode)) { ret =3D -EBUSY; goto done; } @@ -6994,8 +6995,8 @@ static int __alloc_contig_pages(unsigned long start_p= fn, { unsigned long end_pfn =3D start_pfn + nr_pages; =20 - return alloc_contig_range_noprof(start_pfn, end_pfn, MIGRATE_MOVABLE, - gfp_mask); + return alloc_contig_range_noprof(start_pfn, end_pfn, ACR_NONE, + gfp_mask); } =20 static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn, diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 1edfef408faf..ece3bfc56bcd 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -31,7 +31,7 @@ * */ static struct page *has_unmovable_pages(unsigned long start_pfn, unsigned = long end_pfn, - int migratetype, int flags) + enum pb_isolate_mode mode) { struct page *page =3D pfn_to_page(start_pfn); struct zone *zone =3D page_zone(page); @@ -46,7 +46,7 @@ static struct page *has_unmovable_pages(unsigned long sta= rt_pfn, unsigned long e * isolate CMA pageblocks even when they are not movable in fact * so consider them movable here. */ - if (is_migrate_cma(migratetype)) + if (mode =3D=3D PB_ISOLATE_MODE_CMA_ALLOC) return NULL; =20 return page; @@ -117,7 +117,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * The HWPoisoned page may be not in buddy system, and * page_count() is not 0. */ - if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) + if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && PageHWPoison(page)) continue; =20 /* @@ -130,7 +130,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * move these pages that still have a reference count > 0. * (false negatives in this function only) */ - if ((flags & MEMORY_OFFLINE) && PageOffline(page)) + if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && PageOffline(page)) continue; =20 if (__PageMovable(page) || PageLRU(page)) @@ -151,7 +151,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * present in [start_pfn, end_pfn). The pageblock must intersect with * [start_pfn, end_pfn). */ -static int set_migratetype_isolate(struct page *page, int migratetype, int= isol_flags, +static int set_migratetype_isolate(struct page *page, enum pb_isolate_mode= mode, unsigned long start_pfn, unsigned long end_pfn) { struct zone *zone =3D page_zone(page); @@ -186,7 +186,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ end_pfn); =20 unmovable =3D has_unmovable_pages(check_unmovable_start, check_unmovable_= end, - migratetype, isol_flags); + mode); if (!unmovable) { if (!pageblock_isolate_and_move_free_pages(zone, page)) { spin_unlock_irqrestore(&zone->lock, flags); @@ -198,7 +198,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ } =20 spin_unlock_irqrestore(&zone->lock, flags); - if (isol_flags & REPORT_FAILURE) { + if (mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) { /* * printk() with zone->lock held will likely trigger a * lockdep splat, so defer it here. @@ -292,11 +292,10 @@ __first_valid_page(unsigned long pfn, unsigned long n= r_pages) * isolate_single_pageblock() -- tries to isolate a pageblock that might be * within a free or in-use page. * @boundary_pfn: pageblock-aligned pfn that a page might cross - * @flags: isolation flags + * @mode: isolation mode * @isolate_before: isolate the pageblock before the boundary_pfn * @skip_isolation: the flag to skip the pageblock isolation in second * isolate_single_pageblock() - * @migratetype: migrate type to set in error recovery. * * Free and in-use pages can be as big as MAX_PAGE_ORDER and contain more = than one * pageblock. When not all pageblocks within a page are isolated at the sa= me @@ -311,8 +310,9 @@ __first_valid_page(unsigned long pfn, unsigned long nr_= pages) * either. The function handles this by splitting the free page or migrati= ng * the in-use page then splitting the free page. */ -static int isolate_single_pageblock(unsigned long boundary_pfn, int flags, - bool isolate_before, bool skip_isolation, int migratetype) +static int isolate_single_pageblock(unsigned long boundary_pfn, + enum pb_isolate_mode mode, bool isolate_before, + bool skip_isolation) { unsigned long start_pfn; unsigned long isolate_pageblock; @@ -338,12 +338,11 @@ static int isolate_single_pageblock(unsigned long bou= ndary_pfn, int flags, zone->zone_start_pfn); =20 if (skip_isolation) { - int mt __maybe_unused =3D get_pageblock_migratetype(pfn_to_page(isolate_= pageblock)); - - VM_BUG_ON(!is_migrate_isolate(mt)); + VM_BUG_ON(!get_pageblock_isolate(pfn_to_page(isolate_pageblock))); } else { - ret =3D set_migratetype_isolate(pfn_to_page(isolate_pageblock), migratet= ype, - flags, isolate_pageblock, isolate_pageblock + pageblock_nr_pages); + ret =3D set_migratetype_isolate(pfn_to_page(isolate_pageblock), + mode, isolate_pageblock, + isolate_pageblock + pageblock_nr_pages); =20 if (ret) return ret; @@ -441,14 +440,7 @@ static int isolate_single_pageblock(unsigned long boun= dary_pfn, int flags, * start_isolate_page_range() - mark page range MIGRATE_ISOLATE * @start_pfn: The first PFN of the range to be isolated. * @end_pfn: The last PFN of the range to be isolated. - * @migratetype: Migrate type to set in error recovery. - * @flags: The following flags are allowed (they can be combined in - * a bit mask) - * MEMORY_OFFLINE - isolate to offline (!allocate) memory - * e.g., skip over PageHWPoison() pages - * and PageOffline() pages. - * REPORT_FAILURE - report details about the failure to - * isolate the range + * @mode: isolation mode * * Making page-allocation-type to be MIGRATE_ISOLATE means free pages in * the range will never be allocated. Any free pages and pages freed in the @@ -481,7 +473,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, * Return: 0 on success and -EBUSY if any part of range cannot be isolated. */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype, int flags) + enum pb_isolate_mode mode) { unsigned long pfn; struct page *page; @@ -492,8 +484,8 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, bool skip_isolation =3D false; =20 /* isolate [isolate_start, isolate_start + pageblock_nr_pages) pageblock = */ - ret =3D isolate_single_pageblock(isolate_start, flags, false, - skip_isolation, migratetype); + ret =3D isolate_single_pageblock(isolate_start, mode, false, + skip_isolation); if (ret) return ret; =20 @@ -501,8 +493,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, skip_isolation =3D true; =20 /* isolate [isolate_end - pageblock_nr_pages, isolate_end) pageblock */ - ret =3D isolate_single_pageblock(isolate_end, flags, true, - skip_isolation, migratetype); + ret =3D isolate_single_pageblock(isolate_end, mode, true, skip_isolation); if (ret) { unset_migratetype_isolate(pfn_to_page(isolate_start)); return ret; @@ -513,8 +504,8 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, pfn < isolate_end - pageblock_nr_pages; pfn +=3D pageblock_nr_pages) { page =3D __first_valid_page(pfn, pageblock_nr_pages); - if (page && set_migratetype_isolate(page, migratetype, flags, - start_pfn, end_pfn)) { + if (page && set_migratetype_isolate(page, mode, start_pfn, + end_pfn)) { undo_isolate_page_range(isolate_start, pfn); unset_migratetype_isolate( pfn_to_page(isolate_end - pageblock_nr_pages)); @@ -556,7 +547,7 @@ void undo_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn) */ static unsigned long __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn, - int flags) + enum pb_isolate_mode mode) { struct page *page; =20 @@ -569,11 +560,12 @@ __test_page_isolated_in_pageblock(unsigned long pfn, = unsigned long end_pfn, * simple way to verify that as VM_BUG_ON(), though. */ pfn +=3D 1 << buddy_order(page); - else if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) + else if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && + PageHWPoison(page)) /* A HWPoisoned page cannot be also PageBuddy */ pfn++; - else if ((flags & MEMORY_OFFLINE) && PageOffline(page) && - !page_count(page)) + else if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && + PageOffline(page) && !page_count(page)) /* * The responsible driver agreed to skip PageOffline() * pages when offlining memory by dropping its @@ -591,11 +583,11 @@ __test_page_isolated_in_pageblock(unsigned long pfn, = unsigned long end_pfn, * test_pages_isolated - check if pageblocks in range are isolated * @start_pfn: The first PFN of the isolated range * @end_pfn: The first PFN *after* the isolated range - * @isol_flags: Testing mode flags + * @mode: Testing mode * * This tests if all in the specified range are free. * - * If %MEMORY_OFFLINE is specified in @flags, it will consider + * If %PB_ISOLATE_MODE_MEM_OFFLINE specified in @mode, it will consider * poisoned and offlined pages free as well. * * Caller must ensure the requested range doesn't span zones. @@ -603,7 +595,7 @@ __test_page_isolated_in_pageblock(unsigned long pfn, un= signed long end_pfn, * Returns 0 if true, -EBUSY if one or more pages are in use. */ int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, - int isol_flags) + enum pb_isolate_mode mode) { unsigned long pfn, flags; struct page *page; @@ -639,7 +631,7 @@ int test_pages_isolated(unsigned long start_pfn, unsign= ed long end_pfn, /* Check all pages are free or marked as ISOLATED */ zone =3D page_zone(page); spin_lock_irqsave(&zone->lock, flags); - pfn =3D __test_page_isolated_in_pageblock(start_pfn, end_pfn, isol_flags); + pfn =3D __test_page_isolated_in_pageblock(start_pfn, end_pfn, mode); spin_unlock_irqrestore(&zone->lock, flags); =20 ret =3D pfn < end_pfn ? -EBUSY : 0; --=20 2.47.2