From nobody Sun Feb 8 06:22:35 2026 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2070.outbound.protection.outlook.com [40.107.236.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE5F21991DD for ; Mon, 2 Jun 2025 23:53:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.70 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908384; cv=fail; b=aNnjJFV4KFv8aLZoJY19Rz77WXvESMFSUDQ2CgEkGmXzAKlJuaHf7raIB9vbm6qACrkfB35mIP2rHDmHfLxcx0N7HDYA99n4YdwuX0LjjQXRPog2mAR/2fal/ktEzUZewVZOFJrrfZPrkFoTrqQZF8RvloMY44YoRlwPaFDDBPU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908384; c=relaxed/simple; bh=6U9w1dKsYh/DEQW00DsMHKBMT+TqlZMfENr8pBytKIA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=R9djQqMOzDairkBX4kKBfdgHDh+Qqy8NjokZAT1VpouTk3wG7IPDMOdZqXQs/nZtnmV4nxFh8m3kFFvWaiSkpikjMHti7+991D4bqcGizAbI/1Dk18rwfE7Z5y9LI4J9cE8wNtgbTlNEh0EIJoFLo1RXglWg13JhlHbspUw6/hI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=jQFvogZ3; arc=fail smtp.client-ip=40.107.236.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="jQFvogZ3" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NZykmlmmQfYtZngGQ9XDFDz3awu+VwIrBcwmHi6x03BNjQWbdCa1eAmUYYtQcoUvx8hhjRbAlXbOVA9AmOUfl4vigT+YPHKaSjTJ3nM4v7AJ+D23/nv2dVD7G6OadQln7RXD2eZhx8XVBjxls7aCN0z0ulPEsker5ksMtNAfmHGcmrhpS709CgbmstR6U0dAc25zOujPdgU3leKeOKyxbiWrmJ6vlJp5erDMxq7iH1YgoU1hTZKZ0W6F9Zd+M0KjNBRWN2OeUqA1l+XnG7aZFCVRFcf+1Ofg5kQBCO+MPxF24XFpYPObq6/hOf5GaUJczlXoYTf6mg39QBICDlxNkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S32XQEizl/6DUk/6LC99drW7fwx9X9K2aG6VfQn+4xo=; b=fLFhAmuzCdA1nI4mWWu+wyioWfPH9q871WCiJMpmgRgdW9es7oHEp/VKezOU31y9yR3tx+h5nCezUsrbyHZba6SIXChiTnYi8/aSJ5chJV/NS4KTSCW4iqbma+HNUYSdrfj46pI4COT2VkQD/G1ddmmB34oojJ5/PaT46K7Dj6z2FnNMyCE6jXrIWItO+xRaHDYa4YY+kABfZ1zJ1adnqhJXtRAHmhtLVepUp9du8OeX8TjjxwLF1J/bWvpMhcoCmfUBBrdBty5mweQ5H7x/TUiKee5m4eRg9xkLDd4bjt6uUT3/IwIyU7DIHcy1Ey9yfc5gDmzgFayxcEAcWofdsg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S32XQEizl/6DUk/6LC99drW7fwx9X9K2aG6VfQn+4xo=; b=jQFvogZ3Xbzi+de188kxcLH4j1hK6cMXTHHZO3AFB0sogXd6o10GNqin4fuJ78dV8K80PTwurUHhr0jFH5szTk5o0PbR2lgBn2T+PSxUq/LkfbTbp9RPWmoJRIBkw+pO2adhzJZ6c/X0HoRh3AWKKUN4dI89jrGjioSMNPS3PN5izU7PYfR/iibIkAzvGNY33UZzD2k9dC1nBxt5MPxmqLZ5RgxMNOPALBpQDPAMm9cuj5QnDmysT8UEbykS6QIGOHNKd6qj+57pQ6Sxm3lE3pUB3dHIl/kFxk5b2vrSLDYnIBvyyf582qmm6hacunFS93kKNbywwPXhyUCsvpO4kA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by LV2PR12MB5726.namprd12.prod.outlook.com (2603:10b6:408:17e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.37; Mon, 2 Jun 2025 23:52:57 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8792.034; Mon, 2 Jun 2025 23:52:57 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v8 1/6] mm/page_alloc: pageblock flags functions clean up. Date: Mon, 2 Jun 2025 19:52:42 -0400 Message-ID: <20250602235247.1219983-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250602235247.1219983-1-ziy@nvidia.com> References: <20250602235247.1219983-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BYAPR02CA0018.namprd02.prod.outlook.com (2603:10b6:a02:ee::31) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|LV2PR12MB5726:EE_ X-MS-Office365-Filtering-Correlation-Id: a3fceb6e-0fa1-4a29-aa3f-08dda230996f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?RMsYOiIQMkpV50fBvmQGSGB6L+RoR7QSV3s4tXzgVBHeajz0jCVF9P4k7OYl?= =?us-ascii?Q?2RTSwvtaFeYovJCJUHeiImJ7tENpabZqwGKz5utu0tzJ7Zn8Y5tckHi3ndYH?= =?us-ascii?Q?TvB2sxOOsEP5RCjuGzDqVCFtw9Y28mMS9EX2m8p7EdRHAbl+/V1bHrtkXFYZ?= =?us-ascii?Q?Qu66OHrGN+T5gmeN7lNNbTJD9V1LSKFoRiPLP8nKejBmI6A62azbYlKxiWeW?= =?us-ascii?Q?6YkYuXgJ3oN1WO9Pacp+su4E9nHly7k4Gu3T3L9y1jWB4Y5caKeQQHQyOLoJ?= =?us-ascii?Q?sPWDniPbKOWVSX9LkfSR6bBwbh/eXD+CgOHT5p5MerMUkmY0UivCjfCoXOKj?= =?us-ascii?Q?CuKFesY1j4v7OJtOQefVehp2SWMbgJ4IAKgXryqg3Db3Fq2D+B11WlpJftBw?= =?us-ascii?Q?X8mPSiMRe7Z5+2HBcxgKW84USPyr0gwn2vuNF/eyX0q9bg3Z4K03ud/3aziI?= =?us-ascii?Q?t9J+KmhwVVNaz9OTrVZ0ZRcKz2QHDX6iqVT8lttgCX+ghpfwKtCrPpDf+j2P?= =?us-ascii?Q?OvAvXIUQSGPB2M5xt3n3ui+xKYo960DS1XITuZ5FjAPdomCBGJ5w1szZUy23?= =?us-ascii?Q?OPOec+lbyNs/tQKTSl5y9uDcsbhO5B//lA0BxUSCza+Nq8wihvs5Kj/nvSrt?= =?us-ascii?Q?AImUi/fq0bFhUdPVkWWMVU7QRME4+bN+s9v9ySEGNCyqD+oNuLQelwfTxXEb?= =?us-ascii?Q?VlMe2A755qWBS2MuVUIRe6ZSVoaiCiLRFtcsNgEN0s7w9P7L6Ok23fbKrr0q?= =?us-ascii?Q?k/pnTDkYXzbTv4DeW41xlAvMCgDrJvF0o77Un8o/8H5tA3qy6RJEx6xSiOv3?= =?us-ascii?Q?nOCKyH4Bs7bLifjKUMG+4QsQiKVcqgfh4oFfEpiITJkNtGC2utx0nCH+CPre?= =?us-ascii?Q?i/asV73iJMxCS2mkrjuVw0mjDz9OUNWlwxMWekFN6j0IH0AAreNnVySM9Q5r?= =?us-ascii?Q?rOCRw6ASvKh9cpOUdpxHge1QC0eDKbdsqkc3WK8dUZ++bMEL7SHB2pVkHRPB?= =?us-ascii?Q?BRI29/W2Y5nxFkqJWhUsBkiRYlyliMndCSIUw70t5lm6opHrT2C4zmc/Qu3k?= =?us-ascii?Q?zSE8wr/FEiyYNK30hJgoDYs2YJESFdwr0dHHe9KqPqYZgD3zRqkDdersGOnO?= =?us-ascii?Q?DD09gthP77FuC6+cPyf3Xbht6A0WgaszfKKKb8EnUzv8YSLYTsCY85H6tTFn?= =?us-ascii?Q?RGBiR9qw0ebMaCj2E1ptp2/p0lNEL5zRDS/qyaV0YbEPkqXqWEtGeN7PWjYT?= =?us-ascii?Q?4jkq+ohWZTP2IXkhz4gbMatdPk2q4lsMUQQf8zNHscowW8nXPGehy0WflpKT?= =?us-ascii?Q?sMS/i/N6UD6RUmZNJmKb6PrJzQPeV9C1gVFZH+mg3vmftSO7ThWD3Myx+txO?= =?us-ascii?Q?lVAYUJd1QY/VmoKOSX/nYy0cHq5w+hQ2fFhcTKr/NgFkpwZMhZZcSSWIUFTI?= =?us-ascii?Q?+PN9FyhPDjE=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?jPpUAvmDCene9t9O7Qlo30Ad6km3kp9bGDtcXlGD9psz5Xc6mj62+IHgvwXQ?= =?us-ascii?Q?PyJVDRAjzlLqwkkTCeOGFjIIFfSmaozaWYPiWQppvJQpSwNfmhZGmzQ4ZNrY?= =?us-ascii?Q?ZZhpVgcxFbZgrj+n7Q7uKBkr4K0hmqLQz/efWqnqiJMlIM+e8Q8f4aJCDYaA?= =?us-ascii?Q?PFM44kRSKcjPN5oXAkf/yV+f+wPuUC4T8UVVkM0FTppnRLhNJ6He1xxKwH9I?= =?us-ascii?Q?MAigLm76a+q4SrqNdaAh9pdPTkbslTHcj5qxvZbvEVQUmSUvd1yy4jK6M5Gj?= =?us-ascii?Q?wqLRnhPiGjYUMqN+n30OAjPvzFhI7wTIJ1yqXHjsf601Aj7ZIzTlqRfgVPEf?= =?us-ascii?Q?KUTcGEKKCDqde/cW7ssI0gwXTtbgXUZaaOWF8pitCbEhNxyxw8GGB2EdgGeT?= =?us-ascii?Q?/zpJcwGJiDRsJOJ5CO7PvSibqjY6Zkoi57vQDnm1Pt2LNvcLdJdvutRC4pgT?= =?us-ascii?Q?1/jFjIzG2XnPiRPid6ejvY/11Xj3k4dzIOrqSfMk3h9c/LaPFoL4ww+VljIy?= =?us-ascii?Q?nE6NI2XzCFCYODvYI+bdov/0Xea4DbRlb++B3Ed6VUxrpi/quyjTaMzLCG6v?= =?us-ascii?Q?6GiF0O+1U6vFiZDOzG18xYIgpKQLG/xUIsQzEVbAUS7r+YQ2yIxZu9f7TK5s?= =?us-ascii?Q?5vc+72SfQMT8HtFSK2g2YqcSWczxxB8FvWr7c90OsZVnsYM5EA0OnuN6G0og?= =?us-ascii?Q?ntgDjSxi2jOSnZCt9VIFtoMF3p09fe3zwzmnuHwkTAEz1XIvVvYmBK6Ho5ky?= =?us-ascii?Q?PY0ln77SOX12W4+ZnuUbi+Z9NND8VE4ZDcYH3we/2flbZp02QEOprOxKt+FV?= =?us-ascii?Q?U6nMKsQhC4K09+6aRF0r5xl692BsnLP3H2/d/dfLKx5honJaKdMv1jY9DGWG?= =?us-ascii?Q?LWH94UP5V8gyi/Zinqrdbpvt7floD9WR7fdHSeqeKHpNq7dKf3zd6n0UpKgb?= =?us-ascii?Q?qAeLB8wrtOXJh/aFMDlfsxeL/Jl5usRAYWanukESbvQKuE2kJcOHgAS5gsdr?= =?us-ascii?Q?Kc1PAJXBtljvcfMKIhtiFO8JXLpvAdSgGBQt2A80QBPiSaenH3T96OHrnvOc?= =?us-ascii?Q?i6Iqdevl3rMfNJWC6KgjaDMKmyrzFZvKOtDYrP6FASdm9lNf0xB8ydquiaTZ?= =?us-ascii?Q?yha63HymYJiMJvR/hDf9eHv3m8szY2v0EKKls9zotPMeYno9/8IThAJPMVaJ?= =?us-ascii?Q?YlYz99CYk1ohaTjSa3weUeARqmL8kAPuheDWpe33shjd4lORUsIEdY8qWVmh?= =?us-ascii?Q?99xMnI2Jmy0RFFvB78wpmKuilJnA9qQxbkjiYMBzSybqFwHy9rghtfos5Djo?= =?us-ascii?Q?7PTLWMvwfh6GND69WdXaq/uS69QNjPzfCVOt+depQzvlHdz4VJ+QWV4yq1J9?= =?us-ascii?Q?h4Yri/csuuW3lqjaFxpKD/RWul0xKx9pwz0ZaDbGkKRqIOlq8inWUyWf5eg0?= =?us-ascii?Q?QtvZ4KT0QaXVdE989oVHQUi2llW4ovgo1jAooxqMCOb3lLP2X3OXDzROyfW1?= =?us-ascii?Q?Lgu4Qxd81vrbQOIJQLwIX84MIsIXSVWe3Sy6brDWbtIZEVuSbP5IxtHhNrdq?= =?us-ascii?Q?/NaEEmqkxe2BiTzw4Fe2EdyFpky0i7CJiDxwYM0c?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a3fceb6e-0fa1-4a29-aa3f-08dda230996f X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2025 23:52:57.5102 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 23624mqMf0/CMhrHs8SjuTNpvJGBIwiAlK5Zkjpg6/SJrd/YfdWBckxqF1bAcFeb X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5726 Content-Type: text/plain; charset="utf-8" No functional change is intended. 1. Add __NR_PAGEBLOCK_BITS for the number of pageblock flag bits and use roundup_pow_of_two(__NR_PAGEBLOCK_BITS) as NR_PAGEBLOCK_BITS to take right amount of bits for pageblock flags. 2. Rename PB_migrate_skip to PB_compact_skip. 3. Add {get,set,clear}_pfnblock_bit() to operate one a standalone bit, like PB_compact_skip. 3. Make {get,set}_pfnblock_flags_mask() internal functions and use {get,set}_pfnblock_migratetype() for pageblock migratetype operations. 4. Move pageblock flags common code to get_pfnblock_bitmap_bitidx(). 3. Use MIGRATETYPE_MASK to get the migratetype of a pageblock from its flags. 4. Use PB_migrate_end in the definition of MIGRATETYPE_MASK instead of PB_migrate_bits. 5. Add a comment on is_migrate_cma_folio() to prevent one from changing it to use get_pageblock_migratetype() and causing issues. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- Documentation/mm/physical_memory.rst | 2 +- include/linux/mmzone.h | 18 +-- include/linux/page-isolation.h | 2 +- include/linux/pageblock-flags.h | 34 +++--- mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 171 +++++++++++++++++++++------ 6 files changed, 162 insertions(+), 67 deletions(-) diff --git a/Documentation/mm/physical_memory.rst b/Documentation/mm/physic= al_memory.rst index d3ac106e6b14..9af11b5bd145 100644 --- a/Documentation/mm/physical_memory.rst +++ b/Documentation/mm/physical_memory.rst @@ -584,7 +584,7 @@ Compaction control =20 ``compact_blockskip_flush`` Set to true when compaction migration scanner and free scanner meet, whi= ch - means the ``PB_migrate_skip`` bits should be cleared. + means the ``PB_compact_skip`` bits should be cleared. =20 ``contiguous`` Set to true when the zone is contiguous (in other words, no hole). diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 87a667533d6d..392a03e37610 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -92,8 +92,12 @@ extern const char * const migratetype_names[MIGRATE_TYPE= S]; #ifdef CONFIG_CMA # define is_migrate_cma(migratetype) unlikely((migratetype) =3D=3D MIGRAT= E_CMA) # define is_migrate_cma_page(_page) (get_pageblock_migratetype(_page) =3D= =3D MIGRATE_CMA) -# define is_migrate_cma_folio(folio, pfn) (MIGRATE_CMA =3D=3D \ - get_pfnblock_flags_mask(&folio->page, pfn, MIGRATETYPE_MASK)) +/* + * __dump_folio() in mm/debug.c passes a folio pointer to on-stack struct = folio, + * so folio_pfn() cannot be used and pfn is needed. + */ +# define is_migrate_cma_folio(folio, pfn) \ + (get_pfnblock_migratetype(&folio->page, pfn) =3D=3D MIGRATE_CMA) #else # define is_migrate_cma(migratetype) false # define is_migrate_cma_page(_page) false @@ -122,14 +126,12 @@ static inline bool migratetype_is_mergeable(int mt) =20 extern int page_group_by_mobility_disabled; =20 -#define MIGRATETYPE_MASK ((1UL << PB_migratetype_bits) - 1) +#define get_pageblock_migratetype(page) \ + get_pfnblock_migratetype(page, page_to_pfn(page)) =20 -#define get_pageblock_migratetype(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), MIGRATETYPE_MASK) +#define folio_migratetype(folio) \ + get_pageblock_migratetype(&folio->page) =20 -#define folio_migratetype(folio) \ - get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ - MIGRATETYPE_MASK) struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 898bb788243b..277d8d92980c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -25,7 +25,7 @@ static inline bool is_migrate_isolate(int migratetype) #define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 =20 -void set_pageblock_migratetype(struct page *page, int migratetype); +void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 bool move_freepages_block_isolate(struct zone *zone, struct page *page, int migratetype); diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flag= s.h index e73a4292ef02..451b351c689e 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -19,15 +19,19 @@ enum pageblock_bits { PB_migrate, PB_migrate_end =3D PB_migrate + PB_migratetype_bits - 1, /* 3 bits required for migrate types */ - PB_migrate_skip,/* If set the block is skipped by compaction */ + PB_compact_skip,/* If set the block is skipped by compaction */ =20 /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. */ - NR_PAGEBLOCK_BITS + __NR_PAGEBLOCK_BITS }; =20 +#define NR_PAGEBLOCK_BITS (roundup_pow_of_two(__NR_PAGEBLOCK_BITS)) + +#define MIGRATETYPE_MASK ((1UL << (PB_migrate_end + 1)) - 1) + #if defined(CONFIG_HUGETLB_PAGE) =20 #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -65,27 +69,23 @@ extern unsigned int pageblock_order; /* Forward declaration */ struct page; =20 -unsigned long get_pfnblock_flags_mask(const struct page *page, - unsigned long pfn, - unsigned long mask); - -void set_pfnblock_flags_mask(struct page *page, - unsigned long flags, - unsigned long pfn, - unsigned long mask); +enum migratetype get_pfnblock_migratetype(const struct page *page, + unsigned long pfn); +bool get_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); +void set_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); +void clear_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit); =20 /* Declarations for getting and setting flags. See mm/page_alloc.c */ #ifdef CONFIG_COMPACTION #define get_pageblock_skip(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), \ - (1 << (PB_migrate_skip))) + get_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #define clear_pageblock_skip(page) \ - set_pfnblock_flags_mask(page, 0, page_to_pfn(page), \ - (1 << PB_migrate_skip)) + clear_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #define set_pageblock_skip(page) \ - set_pfnblock_flags_mask(page, (1 << PB_migrate_skip), \ - page_to_pfn(page), \ - (1 << PB_migrate_skip)) + set_pfnblock_bit(page, page_to_pfn(page), PB_compact_skip) #else static inline bool get_pageblock_skip(struct page *page) { diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index b1caedbade5b..4ce5210ea56e 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -797,7 +797,7 @@ void move_pfn_range_to_zone(struct zone *zone, unsigned= long start_pfn, =20 /* * TODO now we have a visible range of pages which are not associated - * with their zone properly. Not nice but set_pfnblock_flags_mask + * with their zone properly. Not nice but set_pfnblock_migratetype() * expects the zone spans the pfn range. All the pages in the range * are reserved so nobody should be touching them so we should be safe */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cc9577a27ec4..74cb7696e527 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -353,81 +353,174 @@ static inline int pfn_to_bitidx(const struct page *p= age, unsigned long pfn) return (pfn >> pageblock_order) * NR_PAGEBLOCK_BITS; } =20 +static __always_inline bool is_standalone_pb_bit(enum pageblock_bits pb_bi= t) +{ + return pb_bit > PB_migrate_end && pb_bit < __NR_PAGEBLOCK_BITS; +} + +static __always_inline void +get_pfnblock_bitmap_bitidx(const struct page *page, unsigned long pfn, + unsigned long **bitmap_word, unsigned long *bitidx) +{ + unsigned long *bitmap; + unsigned long word_bitidx; + + BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); + BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); + VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); + + bitmap =3D get_pageblock_bitmap(page, pfn); + *bitidx =3D pfn_to_bitidx(page, pfn); + word_bitidx =3D *bitidx / BITS_PER_LONG; + *bitidx &=3D (BITS_PER_LONG - 1); + *bitmap_word =3D &bitmap[word_bitidx]; +} + + /** - * get_pfnblock_flags_mask - Return the requested group of flags for the p= ageblock_nr_pages block of pages + * __get_pfnblock_flags_mask - Return the requested group of flags for + * a pageblock_nr_pages block of pages * @page: The page within the block of interest * @pfn: The target page frame number * @mask: mask of bits that the caller is interested in * * Return: pageblock_bits flags */ -unsigned long get_pfnblock_flags_mask(const struct page *page, - unsigned long pfn, unsigned long mask) +static unsigned long __get_pfnblock_flags_mask(const struct page *page, + unsigned long pfn, + unsigned long mask) { - unsigned long *bitmap; - unsigned long bitidx, word_bitidx; + unsigned long *bitmap_word; + unsigned long bitidx; unsigned long word; =20 - bitmap =3D get_pageblock_bitmap(page, pfn); - bitidx =3D pfn_to_bitidx(page, pfn); - word_bitidx =3D bitidx / BITS_PER_LONG; - bitidx &=3D (BITS_PER_LONG-1); + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); /* - * This races, without locks, with set_pfnblock_flags_mask(). Ensure + * This races, without locks, with set_pfnblock_migratetype(). Ensure * a consistent read of the memory array, so that results, even though * racy, are not corrupted. */ - word =3D READ_ONCE(bitmap[word_bitidx]); + word =3D READ_ONCE(*bitmap_word); return (word >> bitidx) & mask; } =20 -static __always_inline int get_pfnblock_migratetype(const struct page *pag= e, - unsigned long pfn) +/** + * get_pfnblock_bit - Check if a standalone bit of a pageblock is set + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to check + * + * Return: true if the bit is set, otherwise false + */ +bool get_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) { - return get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return false; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + return test_bit(bitidx + pb_bit, bitmap_word); } =20 /** - * set_pfnblock_flags_mask - Set the requested group of flags for a pagebl= ock_nr_pages block of pages + * get_pfnblock_migratetype - Return the migratetype of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * + * Return: The migratetype of the pageblock + * + * Use get_pfnblock_migratetype() if caller already has both @page and @pfn + * to save a call to page_to_pfn(). + */ +__always_inline enum migratetype +get_pfnblock_migratetype(const struct page *page, unsigned long pfn) +{ + return __get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); +} + +/** + * __set_pfnblock_flags_mask - Set the requested group of flags for + * a pageblock_nr_pages block of pages * @page: The page within the block of interest - * @flags: The flags to set * @pfn: The target page frame number + * @flags: The flags to set * @mask: mask of bits that the caller is interested in */ -void set_pfnblock_flags_mask(struct page *page, unsigned long flags, - unsigned long pfn, - unsigned long mask) +static void __set_pfnblock_flags_mask(struct page *page, unsigned long pfn, + unsigned long flags, unsigned long mask) { - unsigned long *bitmap; - unsigned long bitidx, word_bitidx; + unsigned long *bitmap_word; + unsigned long bitidx; unsigned long word; =20 - BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); - BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); - - bitmap =3D get_pageblock_bitmap(page, pfn); - bitidx =3D pfn_to_bitidx(page, pfn); - word_bitidx =3D bitidx / BITS_PER_LONG; - bitidx &=3D (BITS_PER_LONG-1); - - VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); =20 mask <<=3D bitidx; flags <<=3D bitidx; =20 - word =3D READ_ONCE(bitmap[word_bitidx]); + word =3D READ_ONCE(*bitmap_word); do { - } while (!try_cmpxchg(&bitmap[word_bitidx], &word, (word & ~mask) | flags= )); + } while (!try_cmpxchg(bitmap_word, &word, (word & ~mask) | flags)); +} + +/** + * set_pfnblock_bit - Set a standalone bit of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to set + */ +void set_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) +{ + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + set_bit(bitidx + pb_bit, bitmap_word); } =20 -void set_pageblock_migratetype(struct page *page, int migratetype) +/** + * clear_pfnblock_bit - Clear a standalone bit of a pageblock + * @page: The page within the block of interest + * @pfn: The target page frame number + * @pb_bit: pageblock bit to clear + */ +void clear_pfnblock_bit(const struct page *page, unsigned long pfn, + enum pageblock_bits pb_bit) +{ + unsigned long *bitmap_word; + unsigned long bitidx; + + if (WARN_ON_ONCE(!is_standalone_pb_bit(pb_bit))) + return; + + get_pfnblock_bitmap_bitidx(page, pfn, &bitmap_word, &bitidx); + + clear_bit(bitidx + pb_bit, bitmap_word); +} + +/** + * set_pageblock_migratetype - Set the migratetype of a pageblock + * @page: The page within the block of interest + * @migratetype: migratetype to set + */ +__always_inline void set_pageblock_migratetype(struct page *page, + enum migratetype migratetype) { if (unlikely(page_group_by_mobility_disabled && migratetype < MIGRATE_PCPTYPES)) migratetype =3D MIGRATE_UNMOVABLE; =20 - set_pfnblock_flags_mask(page, (unsigned long)migratetype, - page_to_pfn(page), MIGRATETYPE_MASK); + __set_pfnblock_flags_mask(page, page_to_pfn(page), + (unsigned long)migratetype, MIGRATETYPE_MASK); } =20 #ifdef CONFIG_DEBUG_VM @@ -667,7 +760,7 @@ static inline void __add_to_free_list(struct page *page= , struct zone *zone, int nr_pages =3D 1 << order; =20 VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D migratetype, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), migratetype, nr_pages); =20 if (tail) @@ -693,7 +786,7 @@ static inline void move_to_free_list(struct page *page,= struct zone *zone, =20 /* Free page moving can fail, so it happens before the type update */ VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D old_mt, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), old_mt, nr_pages); =20 list_move_tail(&page->buddy_list, &area->free_list[new_mt]); @@ -715,7 +808,7 @@ static inline void __del_page_from_free_list(struct pag= e *page, struct zone *zon int nr_pages =3D 1 << order; =20 VM_WARN_ONCE(get_pageblock_migratetype(page) !=3D migratetype, - "page type is %lu, passed migratetype is %d (nr=3D%d)\n", + "page type is %d, passed migratetype is %d (nr=3D%d)\n", get_pageblock_migratetype(page), migratetype, nr_pages); =20 /* clear reported state and update reported page count */ @@ -3127,7 +3220,7 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, =20 /* * Do not instrument rmqueue() with KMSAN. This function may call - * __msan_poison_alloca() through a call to set_pfnblock_flags_mask(). + * __msan_poison_alloca() through a call to set_pfnblock_migratetype(). * If __msan_poison_alloca() attempts to allocate pages for the stack depo= t, it * may call rmqueue() again, which will result in a deadlock. */ --=20 2.47.2 From nobody Sun Feb 8 06:22:35 2026 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2070.outbound.protection.outlook.com [40.107.236.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05D221C1F13 for ; Mon, 2 Jun 2025 23:53:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.70 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908386; cv=fail; b=ME4sEY19FDqgKmt1pW/IgOV07BNAHOvb40X6ST+uP6XeUwn4MAhr+d0vi61umZPacQC08NX1+Fj4l9w9EiiXAFqs2n+GpFLz6nscB4dCu+xgYybe6zQfs93LQIC2MzKUPGegnxPaBsIXozVydF9e44U88XRR/2OoZU4MHltAxTY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908386; c=relaxed/simple; bh=CTAG4sYZwfXNpQc1Ac1m55h4u4WrybEEqbXezJHBoFc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=dfCgLXwaVVD2+a+g9ooQ2OGDh8rvFgujhPaWXXOgFua8xdEX+BrojDgt4admVGece5ExeAvhgeFPzqR4gU2jgc8vxnNEfolh/Tt+StFEbtMTub7Ft+xif4ojq4vANcsmnGzyleavAoIojK9pcQE0+Q9xYThD8PhmWtQygnWVKf0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=oL5knMP6; arc=fail smtp.client-ip=40.107.236.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="oL5knMP6" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rMb1p6CcF3y8Lu25hewdhrGhkD/iK9eLMVez/KN37367BUIUi1j3mlGv30rNEsWmEw5oz6Dci3XoZE2TzhUdcb3sqYrdJ2Ij/jUVFor85Dxxm5JrtScsxswcfxjzMKj6SIoOvjlMHTRrMI1apw9QGkOwLt7+Y32B1ASpoy2kT3KsRRCYVDfz0+NSnn5wQkq9Zsb6Lhy4Ubn0Zvi265Cvtz++VpP4sgR0Tfd0yKYe6DuGQXJ60qvag4DEPdJ1OCgMG+eJVPqQZr7mm/JPwFPMq6sw7UsnHCMPGgdEEZM32XnVghmRzy40/BMvv+73hZhBFHgs3IDxfeYfLqcZUjpPew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fwi20KomilEllyctI1+J40cz0JDU1TIzlORODcSer18=; b=ZblF6L8sGwofwgvQpCVCQtcAeBpK4iFeUsPB+v1LHx7XoIRyImhBKEqV4JTUBPF4RbUGTi2DfETgSjJhJrZxMK5c+Ti46fpOCL9AXcvdo9TmHOr4L+9lupSqPAoJP5Y6J1Ql+j+sdIs/zxjo+l7vx1TuHtXJbylE8NRPosoYFPwaEuPHSUHsm65mSCtLpD8R9Z5UyzjwTJ3lXybIvisgtGrYrmI3hveQ7tmhFuEElrfv8d3z45pJ9Un5g8eHRXxC1VOibKRt7av5XbuJ3n9i+b3fr08xeQJ3CgS13WGbe2geEENuLwyMLZXJMHNMHZCVrnrKs89m0KKfUCnFVhYfGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fwi20KomilEllyctI1+J40cz0JDU1TIzlORODcSer18=; b=oL5knMP67IfmTV3sXs7UmoopVtQs8YmjxIe+1TH7pPFwpG48HPkYuBitsLD1EcXj+QiIvU7AZiHIVSdiHCDzupLit7YGPOSm8oELuGbFI7OLgwa+1s/ptbRUBaf5ErgFMfQTkgszgCSt9SZUABlxK4aaQiB0ZFMdZsyleISxDXORhIsUc7OQKLF32zF/Oert4JF2iqOIq15La7OqKrNYXmEcsLVYkWBRyf0snlWcleJoV+f7jnYFO53id7GpmWlvaOJvXrA1mIaJMSTwCxNBvdFEbAvHpviWr/qy+L8zPn/MCyjfTmnDwckwyEV7Blca9+fdFa8CFQnCOBqEvCL10g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by LV2PR12MB5726.namprd12.prod.outlook.com (2603:10b6:408:17e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.37; Mon, 2 Jun 2025 23:52:59 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8792.034; Mon, 2 Jun 2025 23:52:59 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v8 2/6] mm/page_isolation: make page isolation a standalone bit. Date: Mon, 2 Jun 2025 19:52:43 -0400 Message-ID: <20250602235247.1219983-3-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250602235247.1219983-1-ziy@nvidia.com> References: <20250602235247.1219983-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN0PR02CA0055.namprd02.prod.outlook.com (2603:10b6:408:e5::30) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|LV2PR12MB5726:EE_ X-MS-Office365-Filtering-Correlation-Id: 57f0f769-40d3-4bf8-bace-08dda2309a96 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?niof8WsMIp6BDCL86KcuWYgnD/+Svt+V4Y6PDm3rOFAozP0DmUAV9/hrE73z?= =?us-ascii?Q?DtNLDe8CVW9TDBZP3cUupNuennTnrU2sMYWkN8h21noS0UKgS/rm/rxUC+cI?= =?us-ascii?Q?iRTJhXpF8ZVgaNF//gli8UNoeBiyVDnmck8R8Hdvd8eTEIXZcaJaXHi6asEA?= =?us-ascii?Q?GUjPNKkDewQlDPvbUwOA7j3YujLvIptfP9xBH5j4oN0lGsLznw9vsaqjSetV?= =?us-ascii?Q?kxeeu0LaYMbLZ1yqkIQTIwfrdHIx/BvkzfLgfa8jG50u+8veEMLoCMj98w9V?= =?us-ascii?Q?XPOhVh1PDwbVJoDeMjjLLzMJBmz1JPx4zmRShglYp3tZw8DAX/bUfZAibJHo?= =?us-ascii?Q?Gy0WxmVsgjzfrgXzL5qEqzfLT5vp7hfgfR4BMhwYDCHE0C9DMTugV/KoVgfb?= =?us-ascii?Q?FzlaLVsZ+ic75GyYrH68egcWc9GJiwpkW/O/ZF2Wm92dJV7PzK7fCpbFWrpX?= =?us-ascii?Q?S9A7qjMJ4ro/+AuHJX15C7eJ3G9T0kw5+H2dA/Dvb0olUY6bVeN6S+5gVwfJ?= =?us-ascii?Q?1A1yOo1KxTCiKWA4clIAkU1WBfC0EXjn4QTbOc/Q1TvvZkSUlfzhGIEDBDvN?= =?us-ascii?Q?bNVQvmFisQ1BCGFhaybjSQwb55VS+hGhNgVoFXlDqe49wWTXVJVSDhqFeRpK?= =?us-ascii?Q?dWDKuc4+xVRF/NAnYE4SNwppRG2TQ6v91Yq+zW2paUISWHUFBLuGWWwNj7T0?= =?us-ascii?Q?oud35jMhmrtpgKSAPTyL3lLKHAQngrDksnIU2O8PeyXEHVrMhQU/vYGbkAOC?= =?us-ascii?Q?+puGm2QJtWv5QZZuhW5/+atYVdYqNA/9Ah7M889mJYFDvOmwrbEjfTkc1IN7?= =?us-ascii?Q?NT3ElQgAexz4Dlih4FfqmHF1Rx2Vojf/NwxU7+H9OVkQFiykFTpv1VGNsNWZ?= =?us-ascii?Q?CZ2N5wE+pBPKr/+H3pJDH+ua5/Yid9l9GIiAiWMzzJUlhOPpw8v6dYvdMOQM?= =?us-ascii?Q?RM7nyn18hYNOLDZ80ajORx/vZ4b1n2n86n9DF6JQIzCqwwW75yHTp0Jd1/Ab?= =?us-ascii?Q?9lQxAwFo7wJzmuurJwPoftJYu5Nar3p70RLADlETHJZmO9gd7y4+sv6VKhp7?= =?us-ascii?Q?W77o4kiPja7StrJgAgM27eG/1hCUgSfosc6J01z6jVWBpQIaPgKahZAhwaU3?= =?us-ascii?Q?DEUY+5NSOq/NZNyND1WCl/0xPiuLngv6J2u0370hq56RRMjBnwuqou3KGjFL?= =?us-ascii?Q?ccyA/TtSqQG6ftObsj1gZ3jXapCkAVMJ1cLdp/I6UvuiKj4+TzaG9wz9qahT?= =?us-ascii?Q?7aW4wWnOEcwbZwuSF4TLSKVN8kndQVxkVaXpDromZlVO4PFEYDMHYd3p0D98?= =?us-ascii?Q?UyoVvHuz4Y4MFH/Nk3/hGUD79jLCACveZSwYy7OXMiIW7rZp4pUHCISVu5Vj?= =?us-ascii?Q?ud8yc0Y4Uk50DVkxvkRDR2J0WHv/PIHg2E/exI4miY+++AXfRw=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?BFHkjTNj+WdpHHJLWIzELgwjxy3f3oJH4EfBThy79X/kWyRIW87lSdp/ImSF?= =?us-ascii?Q?90whpgK1q3KjL5WRvjSPWQDSdzEVdyhzBF79vfb9g5zRN3YPt9+Uy12rFC2G?= =?us-ascii?Q?Iz1SFYPSPCH1nZS5gCtYKXVEaOsQd5rTNd3y8/Rx3LSNyO7woxQDJGrLm2fV?= =?us-ascii?Q?jwDRV4D9OFLw4eDH6l/G9yEQYS0KVaNW3twXIbBwNWDTxpXu9B3ZH0/Rx2HX?= =?us-ascii?Q?qSN7gm8v2dk2luLKY+93Y/Qr2X+DXlIza8c3Ii44tSwkjNPvv7jNGRq2B0a3?= =?us-ascii?Q?1FhzBeAz1kaRE/wR86rk5P+zp554iSzq36ba9hTEPn0wJdS1WgIRawivQOG8?= =?us-ascii?Q?bvtefwzP/f3fgu/FMpGzM7HZE8rNaC8yxftLsD7z9ktK9x9Q9LLDF+RR7fls?= =?us-ascii?Q?ca6dswvByiVuQOFUeH2nxLtkbe0KdiL5krxEAIJWi4p3S/2O/OSnf9qdgjeE?= =?us-ascii?Q?cy5P/W9Lz6J5hZE16mEgPy507By2sJ7nPwhlSFbv3FQRN0NwVSoq+BSUquHx?= =?us-ascii?Q?VodRBKb03/fkgRlVFORn4RhaJ6poEcnbiD6UG+rExZcg+4zBDaspl+xaN2fF?= =?us-ascii?Q?j9EkIwKPZgxA6QQALgX6hOKBXrXKWQiEs2TdxNIVx/I5g5nj32bpXzNQ2ICP?= =?us-ascii?Q?vfGsjGeKfEuJKhZy4va0SvIdnYpuOjfl91dFmJVnSKQyFAZu5tgi1LnwPA2Y?= =?us-ascii?Q?d3JRWtulRIONd6CyEa2E2Xyeh8NShXg7NL5uaW44NQXtFwHyr6XxmIy0RbYj?= =?us-ascii?Q?a8BlXw02RjkZ8nNm438IJNiGNfWPMkY60K3jA3Bd/Dxnhro53UZoX9DgnMeu?= =?us-ascii?Q?UrvZPCkWb9ynRG7w2g6rgAy6OVre6d5yUZQM2LfXs52ZexUCma211VwM5g3Z?= =?us-ascii?Q?NJPtJ3SUP2KXWNMIyE8eOURJrHlTaebLGYUEErlEsBX4vzKvBs3keNaBQg7i?= =?us-ascii?Q?AEP7dithbIyjW+Hidh5BgIOUvws9l/XvTa5bvBBbYg6UUHngilD03Wogq8kn?= =?us-ascii?Q?l2hVCChKOAa1D8szflkZ8oJ4QAfhaEZqDQTL3ziA1JaxGv9owdO3J6mkwg4H?= =?us-ascii?Q?9JKtsJ2ezvmZNaE/qgzdg9zS2eY5LYe+M3hfmVQ7ny6RE/1DqF1fSGEms8/T?= =?us-ascii?Q?W0WGNk0op3oA/F6P5CKkgaNMx9Rx7V+sgVHDHPCQHH6yDqaEgBDCq/3PvzSU?= =?us-ascii?Q?tFRlcO5EcEVWdBi1c8Pi4kyAaph0Nfly7l4Nkey7Y9rpASp9UUzEYCBwA0Qu?= =?us-ascii?Q?2s3D7n3Dje6BgtpZHIz2325CqiJAccvad/4U2YUjs9Z2Ne/P7lR5Ywj3XfLK?= =?us-ascii?Q?CD9TDpThkFJ3qQvp4LygPDLi9pp3HOKOoWQQhNahjAQOXGivNWZvKsMRpNIa?= =?us-ascii?Q?QSCMk8EGnibXr6x2nvBLSCUqay3/gNc+150M8SAF5jLonNR8dqpZBhItEMsW?= =?us-ascii?Q?2OQUsPkw7Z2HZsImR2wE0JLewVOnhlRpaT1s/C6pFb/N7pZrmppYya87TGAu?= =?us-ascii?Q?EUMhF0z+4Z++HAK+JfJVgOae5X+Fz1YsAX7pX87TLy19KMTn6LxE6Otalr6d?= =?us-ascii?Q?gA4CVp0PVDQUptSezERS04H93oM0Ni8+q6L1283i?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 57f0f769-40d3-4bf8-bace-08dda2309a96 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2025 23:52:59.3585 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: F8QNBvSYdKtxrcc/MGnbykWd6+YDturH1/5qGm0/J75DxeWE1t/7TmxHlHoUM7x4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5726 Content-Type: text/plain; charset="utf-8" During page isolation, the original migratetype is overwritten, since MIGRATE_* are enums and stored in pageblock bitmaps. Change MIGRATE_ISOLATE to be stored a standalone bit, PB_migrate_isolate, like PB_compact_skip, so that migratetype is not lost during pageblock isolation. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/mmzone.h | 3 +++ include/linux/page-isolation.h | 16 ++++++++++++++++ include/linux/pageblock-flags.h | 14 ++++++++++++++ mm/page_alloc.c | 27 ++++++++++++++++++++++++--- 4 files changed, 57 insertions(+), 3 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 392a03e37610..0a5cdc52b405 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -79,6 +79,9 @@ enum migratetype { * __free_pageblock_cma() function. */ MIGRATE_CMA, + __MIGRATE_TYPE_END =3D MIGRATE_CMA, +#else + __MIGRATE_TYPE_END =3D MIGRATE_HIGHATOMIC, #endif #ifdef CONFIG_MEMORY_ISOLATION MIGRATE_ISOLATE, /* can't allocate from here */ diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 277d8d92980c..fc021d3f95ca 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -11,6 +11,12 @@ static inline bool is_migrate_isolate(int migratetype) { return migratetype =3D=3D MIGRATE_ISOLATE; } +#define get_pageblock_isolate(page) \ + get_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) +#define clear_pageblock_isolate(page) \ + clear_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) +#define set_pageblock_isolate(page) \ + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate) #else static inline bool is_migrate_isolate_page(struct page *page) { @@ -20,6 +26,16 @@ static inline bool is_migrate_isolate(int migratetype) { return false; } +static inline bool get_pageblock_isolate(struct page *page) +{ + return false; +} +static inline void clear_pageblock_isolate(struct page *page) +{ +} +static inline void set_pageblock_isolate(struct page *page) +{ +} #endif =20 #define MEMORY_OFFLINE 0x1 diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flag= s.h index 451b351c689e..1cf5f0fbd627 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -21,6 +21,13 @@ enum pageblock_bits { /* 3 bits required for migrate types */ PB_compact_skip,/* If set the block is skipped by compaction */ =20 +#ifdef CONFIG_MEMORY_ISOLATION + /* + * Pageblock isolation is represented with a separate bit, so that + * the migratetype of a block is not overwritten by isolation. + */ + PB_migrate_isolate, /* If set the block is isolated */ +#endif /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. @@ -32,6 +39,13 @@ enum pageblock_bits { =20 #define MIGRATETYPE_MASK ((1UL << (PB_migrate_end + 1)) - 1) =20 +#ifdef CONFIG_MEMORY_ISOLATION +#define MIGRATETYPE_AND_ISO_MASK \ + (((1UL << (PB_migrate_end + 1)) - 1) | BIT(PB_migrate_isolate)) +#else +#define MIGRATETYPE_AND_ISO_MASK MIGRATETYPE_MASK +#endif + #if defined(CONFIG_HUGETLB_PAGE) =20 #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 74cb7696e527..5de23eba0db8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -365,8 +365,12 @@ get_pfnblock_bitmap_bitidx(const struct page *page, un= signed long pfn, unsigned long *bitmap; unsigned long word_bitidx; =20 +#ifdef CONFIG_MEMORY_ISOLATION + BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 8); +#else BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); - BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); +#endif + BUILD_BUG_ON(__MIGRATE_TYPE_END >=3D (1 << PB_migratetype_bits)); VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page); =20 bitmap =3D get_pageblock_bitmap(page, pfn); @@ -439,7 +443,16 @@ bool get_pfnblock_bit(const struct page *page, unsigne= d long pfn, __always_inline enum migratetype get_pfnblock_migratetype(const struct page *page, unsigned long pfn) { - return __get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); + unsigned long mask =3D MIGRATETYPE_AND_ISO_MASK; + unsigned long flags; + + flags =3D __get_pfnblock_flags_mask(page, pfn, mask); + +#ifdef CONFIG_MEMORY_ISOLATION + if (flags & BIT(PB_migrate_isolate)) + return MIGRATE_ISOLATE; +#endif + return flags & MIGRATETYPE_MASK; } =20 /** @@ -519,8 +532,16 @@ __always_inline void set_pageblock_migratetype(struct = page *page, migratetype < MIGRATE_PCPTYPES)) migratetype =3D MIGRATE_UNMOVABLE; =20 +#ifdef CONFIG_MEMORY_ISOLATION + if (migratetype =3D=3D MIGRATE_ISOLATE) { + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + return; + } + /* MIGRATETYPE_AND_ISO_MASK clears PB_migrate_isolate if it is set */ +#endif __set_pfnblock_flags_mask(page, page_to_pfn(page), - (unsigned long)migratetype, MIGRATETYPE_MASK); + (unsigned long)migratetype, + MIGRATETYPE_AND_ISO_MASK); } =20 #ifdef CONFIG_DEBUG_VM --=20 2.47.2 From nobody Sun Feb 8 06:22:35 2026 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2070.outbound.protection.outlook.com [40.107.236.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE8711E885A for ; Mon, 2 Jun 2025 23:53:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.70 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908388; cv=fail; b=AaBZ2P8QvApmcPfhLs+kHb0sMyHPsJdwyzSv1FwSy/nxGwCuVWmHm+Uo9Pn7zBJOCkwqvRK0AxkbDaliHwjwHWEXF7KG8GfVZu9fUjQ52tooMXj4PX83/lWiS7kXK2z/oRW83I5VMq6Vqbbh3m1tOfzrERYG5t0i+wIJXv7n/Xg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908388; c=relaxed/simple; bh=axbLy5kfGy7ERP+dTcLFkmbbXM7Moc9wuXGtzPhq7Xs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=JS9ycUQoGGMoWXk6UuM1Cak8bMK5baN51rohOgwpoRWHXJ6NltM2HO66/dbw49VR+H/SeGW4n1EOP8q0xiNDHiiReups7wrfmSaLALWUvxB0wASygQ0OKa7c2Ce3ax9Q2YYIRx8y/jIEUF3O+cXk1QdP4N7frLs32O2qb0Izmh4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=A0X53oOM; arc=fail smtp.client-ip=40.107.236.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="A0X53oOM" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=O3YHMmwS4h73+yCk9Ie1opRy+5O4z2LXRz73yLiNInrME7GQKqhU7DnOjk5v1wAJA/QYxPYG2Q1uDhS8EMkchouqv9NqgBrrCgjVS3dLs4dNBVAepVj6j9x2yPDk9INbzbA4DoTE4u/Jo+R5UlW7M0E/DIRKjO3Yk+ez6DMxbnxvOfjggIxTaRusk9D+FrftbqonrxD37sNDOF0mj84wP9zWNk0Ddj5NOt+CRWsx3QUl6X3f1n9Hqkk20Qvc/uRQAJpEp9lbFwH1CFRXLwhnRfLzV6JO+ww2L0cC4TrFZkIjfDyBwvFUVGbMA4mKMCQ8nuh2KDk5QrIP11sMhe2HEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cHKcI/NcdlxqGUi8fUIpIqJzLD2YbZ+F2c4iWBCvJbw=; b=ugrvlJ9i8nQvbhAYJOgoRxXSKzsOh6pzTgZuaA7OW/+0Z8ZJEeQ1mLVC3osL5BfVBVqsjH2EVYk+jt5woRzULnkBNyyzQE7fAz0J+tKKrz08kSQP0SHAVe1FUQv85W4yhHpiYNvTenjqR3WjQXT3mybsLMaQiZtIjvjlxoNCMdpoJxi3+vi3mN7FUNcUyEJPdxNFiwUlmFWccHpGpkdMV3oLaqQtO2ngbfJbaECAPU6+Fh68A6fL7fN9JTNjEBNowUNBFtFRyatjBAFh5O/k3BqkYAjBkjQcL+69GTGDWCZbLik8m5COPOvgeLkZTu4xHVAoDPkTr3vaIwidTskBYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cHKcI/NcdlxqGUi8fUIpIqJzLD2YbZ+F2c4iWBCvJbw=; b=A0X53oOM+Y4En2tqgxuspEHqXhuoY1Nu7JaygRypdBIhAUTtP7Jo/1g90Mv7KGg5wgcsdpKYhXCBuhICfN3gVqTSRPMxCK9qsDQ8ruuIZJANJT/J1/1RjfNWZ3ZohBdu99C/h1QurJ04sAIuYmmwkJAXI04fjhkkvEV821xFepnyrzSZYL0RNk462i3MTVur0bsa8aJyf/kxqSZcb5yvnYkzP3ETKCWzRlm20dpOQm38nLAnDoAMSWBfgeMfLhEXqZGuy1u+6lU686a4xcXT3MvsQZ2Ss3fnniuYLN9t88R+TXNOA64TeOicYTVpdYGLofSzOCDQOTcTOkIQsI7zxQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by LV2PR12MB5726.namprd12.prod.outlook.com (2603:10b6:408:17e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.37; Mon, 2 Jun 2025 23:53:01 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8792.034; Mon, 2 Jun 2025 23:53:01 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v8 3/6] mm/page_alloc: add support for initializing pageblock as isolated. Date: Mon, 2 Jun 2025 19:52:44 -0400 Message-ID: <20250602235247.1219983-4-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250602235247.1219983-1-ziy@nvidia.com> References: <20250602235247.1219983-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR13CA0014.namprd13.prod.outlook.com (2603:10b6:208:160::27) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|LV2PR12MB5726:EE_ X-MS-Office365-Filtering-Correlation-Id: 3b61c700-3913-429e-90a9-08dda2309b98 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?tvx3+cjhEN35LHZ/rDFJv7f0X325kIe1pxRilQUBiI+MPI7lGLvAId1cN3+1?= =?us-ascii?Q?SPQQZSmHGfhyZHSzGjnVI6YQZLApb72d5j7HuK1M8S7O5NYrSx908A/xOYeS?= =?us-ascii?Q?dRVp5HhMbo6/YBQig56Q5HdbN9iKc1krC/DUmI22V1cReX36KR9D922QHGho?= =?us-ascii?Q?oA1FH8QwSW76uewACWe1zXiWUVz2saMYxcDELvqxywYNs3/QaAO9PwMQHsu9?= =?us-ascii?Q?n8CLb79li7BjFxDjY2IDVzpAE3y6p1qW++aVqVCSPW7W41Vk+FWTu9qB3H4P?= =?us-ascii?Q?pW6AKy/klSmB40lQoLbQ266bsL159lwOs/jIAf23OGTakD0uF2DBu4i9oPrI?= =?us-ascii?Q?4o2effsjjMv5YVRy6H6UPLoOPu1od+5WQGgUXJGljp0MLCgTV7cP3N18W6YG?= =?us-ascii?Q?LwT2pqBwszyf+KnCZua5iMK/W59YeTF9uzQIdiEcm64QsRbkdDuZbth4uRnT?= =?us-ascii?Q?0f2XYHiCwZiDN+V+zLzGv/QNmuLmV9iQ/ZS250bHZU+Gp1+WBg+AjCmR/B3h?= =?us-ascii?Q?SF3vtp0C2VxdmSzR4Em70Np3gP8o9k2wLSqHwcTuDaIiWYFAE10aiVBtqG7o?= =?us-ascii?Q?y1tCV32UVW6D9+KSURcMU0ol4a7qUFF6e/nb9c60sqwJBSYPFk6PyOmQVwsP?= =?us-ascii?Q?9QruKYyQ+8aPT3b+hGzfstOyd5O7EEQeoI8ijYirC/Um0a8mkl+q8xkIk0hn?= =?us-ascii?Q?9IvAZwRac/sOTnXGASjcUyo8exoonGopjqxd1aNXyiqai1DAs6zstpSemQsu?= =?us-ascii?Q?jDRy9a+Wgc4M5FpR2Au4kP7S/5UOEiC0WhmBtGG98d4F6AkN0H6MFy6TEl94?= =?us-ascii?Q?pQM+601VzYq+MezijjZ1dB1MeRfYk/MbFEuV0lY4pLKbNgbyYrJKQVZHp7s4?= =?us-ascii?Q?NNknW2iATjmaDxx+arMjcOTGYC95BO4NLZUTXzS0OCsyMSNdd73iOz7F3duH?= =?us-ascii?Q?q8ye6iXgqeRuzZ0UWcUYlBvIDFh9ljr6BdDpu+QF+jsSp4QZPoRcYkhLgEcg?= =?us-ascii?Q?+3RAfem2msy6iSnebhqiNTuC1ec5ugUCylnAMxTnFUHToJnuIC6cry3pjH3I?= =?us-ascii?Q?ABx/EoMMe6oiZPcY84r3Y5/KzaBVBcQ/bxdQq49qlGm7cWYDxRfeXt71oJM6?= =?us-ascii?Q?HmYj5JDqVD3RVx9y5TTzTnutIrdEAjYNTuRSPJw5AnU2WxqzPqU0Ks8v5OQm?= =?us-ascii?Q?OjuAng6kTXoLJGTn86ix3DfWycOzAHXzKrxOXiIqTMBbLmAyhsvtlC5CEnsY?= =?us-ascii?Q?IW9v5LlbxYSIhprSPS0nVmfjLFRZzcVVrI95nwb46DphdmTDS152gFI0Aps/?= =?us-ascii?Q?wEKTmwO8K5KaNvae0G89vwtbpggid4OjpjJVlbnjyaNMkbbJRAf+aiVxk6Qo?= =?us-ascii?Q?RJ/278mnFxfHROuvbsbWi/NVDIlozcJtXUPnR7D6rLYkTjpWUhUIuAos2xQB?= =?us-ascii?Q?eEK9zXUA6tU=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?LMupQps2pYIDYxJQzgs1iYe+2eqyvhjZQJyph4o62QVPNsTPvN4gSJffc2Nu?= =?us-ascii?Q?vYHeeRlsfjZT9EARwXm94ut6zrTQQ11rKkiMt7KSjQ8bePLHDMUK3aT8A2x2?= =?us-ascii?Q?hyP/HIC4VnNEsbXZk5zYODvglgPaqC8RIZJtoDe/unCO+g0zCHQ0Cz508R+A?= =?us-ascii?Q?1fHaWymCbyFkmWQqN8vc0CrZa11rZAYOvmVsokfsD1OCQXjycATkdV8Qmsbj?= =?us-ascii?Q?mYLDeMyLB120pFnDXrje/D8r4cBQh5mAvrC3KiVN9cEfeb7hLeBY2BZZDTEN?= =?us-ascii?Q?a+5+Gmn9gnefUlRcbW7rCXiqGveT5i63VD+EP43V7j0NDcdvKKWiGVuK0A2q?= =?us-ascii?Q?BX+wrPos4liFFedodsxf0spqecZ2fcyuhV1bdqwHQbvuKNm2/TXbAF/84p7U?= =?us-ascii?Q?8hFjx0Vq+brgu72/lDG0hAogqItL0jwe1K2yxYXK/ytF91pu/oqsnlv4FUZT?= =?us-ascii?Q?8oFnsP7KNvUHtKkKEJyAjWbP6Q2feDB9B+13aAVnWow4QtZdIU8j0PlqBuFl?= =?us-ascii?Q?9hELufyjAu6mXUnjOnHi46ZOEaqoLbB6sr2ehSX7rDq8CY9qwUbcNqoulDJ/?= =?us-ascii?Q?GbjqP9nmH0YP+llRMOPg4TquPJrW3L07swWYmGOstZAsxdMfhrPhYFPnIiq/?= =?us-ascii?Q?KDiYjTbk+7Qz4Yg6HRxHhh8OyUIWdq9vHQjOJ8dQBj4Si7QXzzB4Qfh21fEn?= =?us-ascii?Q?JlftoyKwFsfG5INwdEJH8UL4TjdFb0vs8pIXUbMhHDQdrM3WerkfXlu8tJEJ?= =?us-ascii?Q?qcw5hJf2YGGotJ5x8DpJahch4CMHxkyBksJYgVohe8n36pSZ8K0VHEqVyzj8?= =?us-ascii?Q?T9xBmelcGlnJADEXoL8/r/+0qIhPnKmhz1Zl6V2HVM3tA5R9usCQAxQba2hm?= =?us-ascii?Q?3fXcuE+HH+zLRJemeZnTRS/OuokTZEyc6NcUy2Ccg7j06Q26xzIrEffr2SHg?= =?us-ascii?Q?9xRhEAH7UlgHfC6jVPB98w50fKjKqSiGUcepBfCJpFM5CcC2+uapmxDzjlA9?= =?us-ascii?Q?Wr1frVFMY/dFu+VYRa/oRfyIDZuaHtojJi9zUP4dewQIsMDA5MRiyn0W9rsJ?= =?us-ascii?Q?A6kJd0+Go9ZkYhSwTbfeAvRkiWbqACt/c2ljyubZ7jkPmvCUDrSPS8tAUH3+?= =?us-ascii?Q?DgOD86uJcuFZ39lvIPqnaHWYHgp/4MOOOeDEpl7+Yf+qNiFIbHR9lZts3I7Q?= =?us-ascii?Q?nfwcBgwgC+21M8e0BBWJK8PoB0ZpFKgeSkAMAhPErB06pZ+7GIJG1nOuLVGZ?= =?us-ascii?Q?0dvuzPJXaqgQ47Bs+6zkQ/IsQPmNNQDcEXdDAgFZ1KTC25I8giyi7g7CVmK0?= =?us-ascii?Q?l8/MKtSVST7IWLJSmvil/WFZjVMVIopyfNuu5W+FCR6XqE7r+Cr0YBn3aQ+9?= =?us-ascii?Q?fAH9vjlCHrgiF/Q95AuDIVHZTsRMQrJPdMkUAasPWvpDyOZpZ1Ieg6jnSRzj?= =?us-ascii?Q?ZOBdXYqvW7SMFr4IebBbQkK0+/FoyvJFwpJhciXtIa/sgsLuJ/JsKW7eIlW8?= =?us-ascii?Q?yAYVKhEeg4jkOAuTMt42yKqmHUt1wlhaARuQUNwQ19x3mxM7qCnD2B4hsTzm?= =?us-ascii?Q?iCq1KTrYftalnL77VjtDtxCv+VDTt3AxXXT5npfF?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3b61c700-3913-429e-90a9-08dda2309b98 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2025 23:53:01.0308 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tYY8zZsaWgcEXf6U+aWoATq/2ZYv7FyqMx/oxTMjONSRkktK2IuZoMgLxBnoq32g X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5726 Content-Type: text/plain; charset="utf-8" MIGRATE_ISOLATE is a standalone bit, so a pageblock cannot be initialized to just MIGRATE_ISOLATE. Add init_pageblock_migratetype() to enable initialize a pageblock with a migratetype and isolated. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/memory_hotplug.h | 3 ++- include/linux/page-isolation.h | 3 +++ mm/hugetlb.c | 4 ++-- mm/internal.h | 3 ++- mm/memory_hotplug.c | 12 ++++++++---- mm/memremap.c | 2 +- mm/mm_init.c | 24 +++++++++++++++--------- mm/page_alloc.c | 24 ++++++++++++++++++++++++ 8 files changed, 57 insertions(+), 18 deletions(-) diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h index eaac5ae8c05c..23f038a16231 100644 --- a/include/linux/memory_hotplug.h +++ b/include/linux/memory_hotplug.h @@ -314,7 +314,8 @@ extern int add_memory_driver_managed(int nid, u64 start= , u64 size, mhp_t mhp_flags); extern void move_pfn_range_to_zone(struct zone *zone, unsigned long start_= pfn, unsigned long nr_pages, - struct vmem_altmap *altmap, int migratetype); + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock); extern void remove_pfn_range_from_zone(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages); diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index fc021d3f95ca..14c6a5f691c2 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -41,6 +41,9 @@ static inline void set_pageblock_isolate(struct page *pag= e) #define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 =20 +void __meminit init_pageblock_migratetype(struct page *page, + enum migratetype migratetype, + bool isolate); void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 bool move_freepages_block_isolate(struct zone *zone, struct page *page, diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 8746ed2fec13..afeae59b29e1 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3319,8 +3319,8 @@ static void __init hugetlb_bootmem_init_migratetype(s= truct folio *folio, if (folio_test_hugetlb_cma(folio)) init_cma_pageblock(folio_page(folio, i)); else - set_pageblock_migratetype(folio_page(folio, i), - MIGRATE_MOVABLE); + init_pageblock_migratetype(folio_page(folio, i), + MIGRATE_MOVABLE, false); } } =20 diff --git a/mm/internal.h b/mm/internal.h index 6b8ed2017743..c43180bea6b4 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -821,7 +821,8 @@ extern void *memmap_alloc(phys_addr_t size, phys_addr_t= align, int nid, bool exact_nid); =20 void memmap_init_range(unsigned long, int, unsigned long, unsigned long, - unsigned long, enum meminit_context, struct vmem_altmap *, int); + unsigned long, enum meminit_context, struct vmem_altmap *, int, + bool); =20 #if defined CONFIG_COMPACTION || defined CONFIG_CMA =20 diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 4ce5210ea56e..16e3ad874144 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -770,7 +770,8 @@ static inline void section_taint_zone_device(unsigned l= ong pfn) */ void move_pfn_range_to_zone(struct zone *zone, unsigned long start_pfn, unsigned long nr_pages, - struct vmem_altmap *altmap, int migratetype) + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock) { struct pglist_data *pgdat =3D zone->zone_pgdat; int nid =3D pgdat->node_id; @@ -802,7 +803,8 @@ void move_pfn_range_to_zone(struct zone *zone, unsigned= long start_pfn, * are reserved so nobody should be touching them so we should be safe */ memmap_init_range(nr_pages, nid, zone_idx(zone), start_pfn, 0, - MEMINIT_HOTPLUG, altmap, migratetype); + MEMINIT_HOTPLUG, altmap, migratetype, + isolate_pageblock); =20 set_zone_contiguous(zone); } @@ -1127,7 +1129,8 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsi= gned long nr_pages, if (mhp_off_inaccessible) page_init_poison(pfn_to_page(pfn), sizeof(struct page) * nr_pages); =20 - move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE); + move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_UNMOVABLE, + false); =20 for (i =3D 0; i < nr_pages; i++) { struct page *page =3D pfn_to_page(pfn + i); @@ -1192,7 +1195,8 @@ int online_pages(unsigned long pfn, unsigned long nr_= pages, =20 =20 /* associate pfn range with the zone */ - move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_ISOLATE); + move_pfn_range_to_zone(zone, pfn, nr_pages, NULL, MIGRATE_MOVABLE, + true); =20 arg.start_pfn =3D pfn; arg.nr_pages =3D nr_pages; diff --git a/mm/memremap.c b/mm/memremap.c index c417c843e9b1..3319e7cc2898 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -254,7 +254,7 @@ static int pagemap_range(struct dev_pagemap *pgmap, str= uct mhp_params *params, zone =3D &NODE_DATA(nid)->node_zones[ZONE_DEVICE]; move_pfn_range_to_zone(zone, PHYS_PFN(range->start), PHYS_PFN(range_len(range)), params->altmap, - MIGRATE_MOVABLE); + MIGRATE_MOVABLE, false); } =20 mem_hotplug_done(); diff --git a/mm/mm_init.c b/mm/mm_init.c index 8684fa851b84..6e753ca2c338 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -685,7 +685,8 @@ void __meminit __init_page_from_nid(unsigned long pfn, = int nid) __init_single_page(pfn_to_page(pfn), pfn, zid, nid); =20 if (pageblock_aligned(pfn)) - set_pageblock_migratetype(pfn_to_page(pfn), MIGRATE_MOVABLE); + init_pageblock_migratetype(pfn_to_page(pfn), MIGRATE_MOVABLE, + false); } =20 #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT @@ -874,7 +875,8 @@ static void __init init_unavailable_range(unsigned long= spfn, void __meminit memmap_init_range(unsigned long size, int nid, unsigned lon= g zone, unsigned long start_pfn, unsigned long zone_end_pfn, enum meminit_context context, - struct vmem_altmap *altmap, int migratetype) + struct vmem_altmap *altmap, int migratetype, + bool isolate_pageblock) { unsigned long pfn, end_pfn =3D start_pfn + size; struct page *page; @@ -931,7 +933,8 @@ void __meminit memmap_init_range(unsigned long size, in= t nid, unsigned long zone * over the place during system boot. */ if (pageblock_aligned(pfn)) { - set_pageblock_migratetype(page, migratetype); + init_pageblock_migratetype(page, migratetype, + isolate_pageblock); cond_resched(); } pfn++; @@ -954,7 +957,8 @@ static void __init memmap_init_zone_range(struct zone *= zone, return; =20 memmap_init_range(end_pfn - start_pfn, nid, zone_id, start_pfn, - zone_end_pfn, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); + zone_end_pfn, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE, + false); =20 if (*hole_pfn < start_pfn) init_unavailable_range(*hole_pfn, start_pfn, zone_id, nid); @@ -1035,7 +1039,7 @@ static void __ref __init_zone_device_page(struct page= *page, unsigned long pfn, * because this is done early in section_activate() */ if (pageblock_aligned(pfn)) { - set_pageblock_migratetype(page, MIGRATE_MOVABLE); + init_pageblock_migratetype(page, MIGRATE_MOVABLE, false); cond_resched(); } =20 @@ -1996,7 +2000,8 @@ static void __init deferred_free_pages(unsigned long = pfn, /* Free a large naturally-aligned chunk if possible */ if (nr_pages =3D=3D MAX_ORDER_NR_PAGES && IS_MAX_ORDER_ALIGNED(pfn)) { for (i =3D 0; i < nr_pages; i +=3D pageblock_nr_pages) - set_pageblock_migratetype(page + i, MIGRATE_MOVABLE); + init_pageblock_migratetype(page + i, MIGRATE_MOVABLE, + false); __free_pages_core(page, MAX_PAGE_ORDER, MEMINIT_EARLY); return; } @@ -2006,7 +2011,8 @@ static void __init deferred_free_pages(unsigned long = pfn, =20 for (i =3D 0; i < nr_pages; i++, page++, pfn++) { if (pageblock_aligned(pfn)) - set_pageblock_migratetype(page, MIGRATE_MOVABLE); + init_pageblock_migratetype(page, MIGRATE_MOVABLE, + false); __free_pages_core(page, 0, MEMINIT_EARLY); } } @@ -2305,7 +2311,7 @@ void __init init_cma_reserved_pageblock(struct page *= page) set_page_count(p, 0); } while (++p, --i); =20 - set_pageblock_migratetype(page, MIGRATE_CMA); + init_pageblock_migratetype(page, MIGRATE_CMA, false); set_page_refcounted(page); /* pages were reserved and not allocated */ clear_page_tag_ref(page); @@ -2319,7 +2325,7 @@ void __init init_cma_reserved_pageblock(struct page *= page) */ void __init init_cma_pageblock(struct page *page) { - set_pageblock_migratetype(page, MIGRATE_CMA); + init_pageblock_migratetype(page, MIGRATE_CMA, false); adjust_managed_page_count(page, pageblock_nr_pages); page_zone(page)->cma_pages +=3D pageblock_nr_pages; } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5de23eba0db8..92a368d38d77 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -544,6 +544,30 @@ __always_inline void set_pageblock_migratetype(struct = page *page, MIGRATETYPE_AND_ISO_MASK); } =20 +void __meminit init_pageblock_migratetype(struct page *page, + enum migratetype migratetype, + bool isolate) +{ + unsigned long flags =3D migratetype; + + if (unlikely(page_group_by_mobility_disabled && + migratetype < MIGRATE_PCPTYPES)) + migratetype =3D MIGRATE_UNMOVABLE; + +#ifdef CONFIG_MEMORY_ISOLATION + if (migratetype =3D=3D MIGRATE_ISOLATE) { + VM_WARN_ONCE( + 1, + "Set isolate=3Dtrue to isolate pageblock with a migratetype"); + return; + } + if (isolate) + flags |=3D BIT(PB_migrate_isolate); +#endif + __set_pfnblock_flags_mask(page, page_to_pfn(page), flags, + MIGRATETYPE_AND_ISO_MASK); +} + #ifdef CONFIG_DEBUG_VM static int page_outside_zone_boundaries(struct zone *zone, struct page *pa= ge) { --=20 2.47.2 From nobody Sun Feb 8 06:22:35 2026 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2070.outbound.protection.outlook.com [40.107.236.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFA2F253F22 for ; Mon, 2 Jun 2025 23:53:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.70 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908390; cv=fail; b=j3X1GY+d9xnH7o47vusvyKvRJXZXwTGbhVNTXICdLET3mCvoJCpbXfFwW76iM0+/QhzcqAsSg6bgKj8CViHeIGncv+LaQ8yWmKKwhe3GaMgRyB6f58J5jdGvg6xm4rJe6+ycVz3Z9D2JyJnxeso33TQXBfsaN26Gqm57zPlu/P4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908390; c=relaxed/simple; bh=LhxcHub0OtdzSA45orzN+aD1TFgllLctkqv00XizHAs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=RgrtDuNCOh2Cp8sbBP7qAFz0rVNxvLj11TBtVBWwOk0hgkhEfjhQ1HxCpYR7wguq9y4OWjIKAsApZNbyyO3zLVekOr9evqDs7Hlkk26Ie2/GnHZbATGKtn4Ej9dkd/oEZYkZzuHd3tVRAxd4G6c8JcsJLr9mqZjhk+F2dybQwak= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=usErkH+k; arc=fail smtp.client-ip=40.107.236.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="usErkH+k" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ooiCkTZeUX+sFEa2DjRirXhFxT8r+ZlpnV5XWbe+7rwktiEgw1coW6Tl49BFkQ9gHxYjPLUr/rQHXoou6TEABF5CtlDcMtYeNYtyX3OTP7HXfC4a6IfPB4570UlpkU8diYKITkRSnIXrNQ5jFnOkpJ3k8ZhtUNUbDQtYLbBIEdq5IYJ3AK6Yr+xu+LIjq/V+t68Z5w23zn9Nb1Z5omEFVCf/UKssi6WvvKh1wrBbzN5cbQE6cp8ou8mT96wz01hU9P59ubzBFgsgM6xM1ajI9/azNOFwCwZCVhrT2OBIZ7IEfZdRx4O32R1U/BVXwDOAXQWppLZ36yFjybydC5MVNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nHSzkPLbidH6Z9sxeXvVezSWHaVdGptkKez5GwKTgDg=; b=HBkbEiESmTnt1QvUEnvr8CZY6/rXkfcGlZhm+xvrvtCEjEDq78rr7yIBV2/bxGrFctTQ8uvfNhP9qnasXmqM7RBxdhZ+vmyc4G7HJ1pgHpFV9t/beZVDHg+/g/PjZ20Oj4cmb7pKJD6/xGPsgIhKJr+0ycDfqu8flYaXQsyXQA8tBRIxIPS/vkcX3VUCTFxugmbx7gmWnw3R/EggDipAP5OjKMh+vLPyLB51kO83hKRZxx7xOvtMWJdPDK2cNWgIqJtxJBdRQmnJicBP66iUA7bvwUBy90bq45p+duCvC48CQZ4Zy7g7cPJSuZ0JyhDU4rWdaiP63OtIr88FxMqr7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nHSzkPLbidH6Z9sxeXvVezSWHaVdGptkKez5GwKTgDg=; b=usErkH+kHSryLJAtaDXsSUJqyzr1ssI7kw7MXe0Gv31dP87vomalaeRzU3bnbW2SxNXH5lHC+rjs73XZuiQeZNZzwZt2HJObL/6V93+knBfXTVh6dLJrHlnQA5zrDz7XnVtSenEUHQgZfobK+ZiKivRDfFSD67sTXvze87B+HZCRIpn6YsGW4143sLbu97/+jLhf0HJBDcpXpCaJ+0VrrrRc5OZg+zEQHg6YanF39PmNtf7QgK5patGvBiffElFfFyCnQRdwkjzxsdqQciZ6vTvVWamOgdCkh7+BroF1uVvr8nnqgbXrygGNUtU/UVTXw5gVf7jyp1v0zKD8v1O8jw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by LV2PR12MB5726.namprd12.prod.outlook.com (2603:10b6:408:17e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.37; Mon, 2 Jun 2025 23:53:02 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8792.034; Mon, 2 Jun 2025 23:53:02 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v8 4/6] mm/page_isolation: remove migratetype from move_freepages_block_isolate() Date: Mon, 2 Jun 2025 19:52:45 -0400 Message-ID: <20250602235247.1219983-5-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250602235247.1219983-1-ziy@nvidia.com> References: <20250602235247.1219983-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR13CA0025.namprd13.prod.outlook.com (2603:10b6:208:160::38) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|LV2PR12MB5726:EE_ X-MS-Office365-Filtering-Correlation-Id: d4c3092f-cc22-48ad-59e2-08dda2309c78 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?eM7G8o089aIuU/pv2HhbL14uSMPFa57/94K9Ufvk/LKoHv9lLBCs+sqlw1NM?= =?us-ascii?Q?U4YAJP6rYPCi1mPejHIEEZCP88Mb4RFcyqtLFbf1owVI87WIcPPIuvCOk4R0?= =?us-ascii?Q?3jydk+A1pxIm2QfZt7WWvi827RkN0NKslxy6NxXb/5AFlmUe3K9nVoJreSNY?= =?us-ascii?Q?oQEhnuiIuIr+mCWfOlIcMEEoAVKZ0Kmnx//dc6pn/lWy+m3g3ZlDvc/plp3m?= =?us-ascii?Q?6DKHYeUTsUfVCN0KadLd8U8FCjO1hoDgLRaUGyhwMa1o2DqdriHnU9+cZS75?= =?us-ascii?Q?frVd/oQV5gwoM7MOYnwEtIhpduNOajFSZC3y0Jd9wJc6J6ZsThuuJP/ZRl9m?= =?us-ascii?Q?uDK3L+fX27zbbTrA1ZHPZw2ZVMP+wbWiDSYlU/gJwISFW78ETWSo3STHqC6d?= =?us-ascii?Q?SZTI7E667xIuG1+EXVJ+lzRkgeyC0b8d9Q6GXcOpO3j8tozcakJ/fTUvh+Qj?= =?us-ascii?Q?3W+h7BtZKRN7nNsRxwgidgvxigi3sYfdf9nZDZ+2yktfHEEwwxTKpIRqElIQ?= =?us-ascii?Q?EFX9Vl/QmWHVC4RYslaeg819VVaHNLRaQDH6ZgincYaBjKKUK6fWfH7Gj/hD?= =?us-ascii?Q?zFhdjs5Se844+bCuVu1x92CCtN/D7qHrtW9hPelpq4ze1nklImjBM+8cz/vn?= =?us-ascii?Q?+4O0GWyQbW8fu8RjatnBx/tMGg4fONSuJzRKICOdLYrGY4r2yFZv0cc8HypM?= =?us-ascii?Q?FJoLvN5KHRNslMg8BG4tnd88mvxdlEkUrDORlJo/AbBKicwm+YhjINexGe/r?= =?us-ascii?Q?eZPnCrrnWkyxXUfpQyW+iCrsT+UgWzvRmKfzDDrtZjEvqrwEOYBthkw2Q97k?= =?us-ascii?Q?04nzZ7CevfWRBtJsMYlc1hm0xmttWojmjBpJkoervXuaF3chCOBoBOzN9qdj?= =?us-ascii?Q?IfJrp71mCngqeTUCSOh7uCV5PS7/yUBHdso0q8emG0c0PL2rP/glEwaain+D?= =?us-ascii?Q?g3fUBsiPWedzB/qQ1o/eAgZRHtKcJY7rdlVvSXjFl+z3+mquB9efzUNCV1c3?= =?us-ascii?Q?TRWshQWsFd2/V1/C4cfydk0jxOljEvbQ7BUftV5l5a/ezIKghBYmKk+dqwxW?= =?us-ascii?Q?oZI4bRpw58a4e+GEu2EmZmrz7p+E2LF+OwOMZKm65Zn8tD1hFb4YFznBJUAk?= =?us-ascii?Q?TIeBi/BU2qdUqzVVsJSVb0gFyIrdFjyitKGq8CJe4I4B/85dk+TwSIQdsGqn?= =?us-ascii?Q?/SamisnDAD5j36PzBp/rUeSxCsq59Z64f0YT80+mDCdiUDtIFeK7rIEP8sNT?= =?us-ascii?Q?vYsnKw/Ru7ROMm7NQhjJ1VGETBvIc2w1UzVl042FxrhkTfW86GCYi5esP4RJ?= =?us-ascii?Q?zX8a6NjlE+Hh19Rb1YixFprV+kidh723Hns4VNVl3ajk6F7Feo54RknazP52?= =?us-ascii?Q?hKGtWn25sE2NmIMOVN1YbiHKDT/jPG6J57L6Lwl3PmkFnRPNIefrOcOz1nt4?= =?us-ascii?Q?NDO7poxmZ78=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Q+nmbqXrbl76CxBRJnZw1eZLhzhwFz6WwXhnjGHgsrPvFXLaMdDPQiQT1anh?= =?us-ascii?Q?EavOLaXeg7BJkDHmVLIQnwEFjTYhrwJ09tn+EzaNPakkuncaIV0vfh9h0Is1?= =?us-ascii?Q?2W+bcmsdFK5cPT8086ggrMkPiJavKg7snusATvgGZRVqwX3wybauKPXXOLHw?= =?us-ascii?Q?J3MSVu8vnN7MHoWlgumVxoYFlWYExv44lYrFrAke1RJVMN7czHCmzhuNtAaC?= =?us-ascii?Q?Md4AmGXC8PdKfgcaDTTGlgo08eYvhhdnndTP6RmuxNpdEPab9oO+8x8uh/UA?= =?us-ascii?Q?JVbLet/Rf71bV8OMFFmrJxee3/KzF9iJSifZt3H5JFHOnibr3Xop1r7q/UXx?= =?us-ascii?Q?qvUiwVPbr5Qjf1n3cw8ISH0CW8R8lhosp6pzHAfOV+3ai44iVKkZYeue/naq?= =?us-ascii?Q?RnFcp756MZSV1UrcLnS1RavhoNzyPEWNNao1IyrM4SMXSyt0N64YuaBHOKLR?= =?us-ascii?Q?n9H+tXZifSI0Rfi+6mb47S6JZnKUhEj7Z1A0USssQ0mv51PxLzFhtwe1zumg?= =?us-ascii?Q?I5PCOYLaWJPqXELuZNE6yWKNlbZh8GwWdE+NWFh5njpFwd3mHusDUCFVP3GY?= =?us-ascii?Q?SJcysVrDjOloWCSzoIGvqX4zYqYCpHn4QI/a0NP+8D90Ni80T5XHri5OjADQ?= =?us-ascii?Q?IVlQNQoSIIuXE6jwpP9D84VHnNXZqmsDDDbaMAxyFtWJyR7R3UVbv/NRXpha?= =?us-ascii?Q?rPBcH2QBRsNm2AgNePG60c6XRDjRvPizNvOrJrRgDfJ3MAHAzs+84d6WHEBj?= =?us-ascii?Q?eIhzfN/ES8Rmcgh0FzaDXyLPFuypx7I0jKpYH2qWQftzYNv0I5sorYaXqNe8?= =?us-ascii?Q?nMQF5pSgl5HuWqOICzJWs+X436C7n3+2jppY10jtFNTgnf9eNcdC6SNsb/gR?= =?us-ascii?Q?84wFKcseBcBqBUW59YcdokGtGUly8lZSkFC6xmiAhZcANZzx78VuyfA86sio?= =?us-ascii?Q?5E7ico2AIjrXnFuF08YOdDA2Ppo2gwpAezNYy9hSq22biYiDoW5sKKNq+0ja?= =?us-ascii?Q?yOopp880LAB24HlYscYQPEXhhNlhoHqV0TeP5pPL8V4qW2a0mfPemQ1kINWA?= =?us-ascii?Q?q7u7ZL+3cw6DhMkykIN7PHi4TwzeIuBsS1fpgUMTGU0enLnb06Z7GeOUuU9r?= =?us-ascii?Q?2hba9Q4i+qEV4jMoXe+QabIOetjz1ENq6SdV2VaAvVM3k+h+ZXh12NZS92pv?= =?us-ascii?Q?3s1+Xc08lLYGPDXUQgUh0aSIdVmoH2D2njdPeKry0c1HbxJv9H+8RAvx4B8s?= =?us-ascii?Q?9s/kr1UCX4Y1HqXwf5bLZFqyx1yjBKQMFPcq9qeddk4cWx3CZhtk7HcwPD4M?= =?us-ascii?Q?Eu3MLhN1GI4YWbYS0Xq91flSAivp2xg6yBACuD3qCAe1ZAnSlRhd6SqkWxfv?= =?us-ascii?Q?AJ/526H61+22mmby3orCubN6l3pLf+/+OEE+vA0BSsTezSDEIBgbsPz5bSxv?= =?us-ascii?Q?f9adw60ZykcPilVluZzTB0UZytvQlDEoStASY9XPKP/zH50w+7Xc5ETMsdTu?= =?us-ascii?Q?q649JQvOJIxxoXHtgS6t7JBagsyi52pIwsni5Jb4MrcH0OSoQZ7yAnXpTlly?= =?us-ascii?Q?c7/yA+6Kyn/6Z+0yEOb7sdwdD5NCX8xZBYf8O55f?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d4c3092f-cc22-48ad-59e2-08dda2309c78 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2025 23:53:02.7141 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 16d7uXZMtcZ+uZ/7XKzIAwduX1VkM7VcfRCH9DnP1VpvMatZyBN7hlfijlOZ5xfY X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5726 Content-Type: text/plain; charset="utf-8" Since migratetype is no longer overwritten during pageblock isolation, moving a pageblock out of MIGRATE_ISOLATE no longer needs a new migratetype. Add pageblock_isolate_and_move_free_pages() and pageblock_unisolate_and_move_free_pages() to be explicit about the page isolation operations. Both share the common code in __move_freepages_block_isolate(), which is renamed from move_freepages_block_isolate(). Add toggle_pageblock_isolate() to flip pageblock isolation bit in __move_freepages_block_isolate(). Make set_pageblock_migratetype() only accept non MIGRATE_ISOLATE types, so that one should use set_pageblock_isolate() to isolate pageblocks. As a result, move pageblock migratetype code out of __move_freepages_block(). Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- include/linux/page-isolation.h | 5 +-- mm/page_alloc.c | 80 ++++++++++++++++++++++++++-------- mm/page_isolation.c | 21 +++++---- 3 files changed, 75 insertions(+), 31 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 14c6a5f691c2..7241a6719618 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -44,10 +44,9 @@ static inline void set_pageblock_isolate(struct page *pa= ge) void __meminit init_pageblock_migratetype(struct page *page, enum migratetype migratetype, bool isolate); -void set_pageblock_migratetype(struct page *page, enum migratetype migrate= type); =20 -bool move_freepages_block_isolate(struct zone *zone, struct page *page, - int migratetype); +bool pageblock_isolate_and_move_free_pages(struct zone *zone, struct page = *page); +bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page); =20 int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, int migratetype, int flags); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 92a368d38d77..6ddf4fd6b2bd 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -525,8 +525,8 @@ void clear_pfnblock_bit(const struct page *page, unsign= ed long pfn, * @page: The page within the block of interest * @migratetype: migratetype to set */ -__always_inline void set_pageblock_migratetype(struct page *page, - enum migratetype migratetype) +static void set_pageblock_migratetype(struct page *page, + enum migratetype migratetype) { if (unlikely(page_group_by_mobility_disabled && migratetype < MIGRATE_PCPTYPES)) @@ -534,9 +534,13 @@ __always_inline void set_pageblock_migratetype(struct = page *page, =20 #ifdef CONFIG_MEMORY_ISOLATION if (migratetype =3D=3D MIGRATE_ISOLATE) { - set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + VM_WARN_ONCE(1, + "Use set_pageblock_isolate() for pageblock isolation"); return; } + VM_WARN_ONCE(get_pfnblock_bit(page, page_to_pfn(page), + PB_migrate_isolate), + "Use clear_pageblock_isolate() to unisolate pageblock"); /* MIGRATETYPE_AND_ISO_MASK clears PB_migrate_isolate if it is set */ #endif __set_pfnblock_flags_mask(page, page_to_pfn(page), @@ -1923,8 +1927,8 @@ static inline struct page *__rmqueue_cma_fallback(str= uct zone *zone, #endif =20 /* - * Change the type of a block and move all its free pages to that - * type's freelist. + * Move all free pages of a block to new type's freelist. Caller needs to + * change the block type. */ static int __move_freepages_block(struct zone *zone, unsigned long start_p= fn, int old_mt, int new_mt) @@ -1956,8 +1960,6 @@ static int __move_freepages_block(struct zone *zone, = unsigned long start_pfn, pages_moved +=3D 1 << order; } =20 - set_pageblock_migratetype(pfn_to_page(start_pfn), new_mt); - return pages_moved; } =20 @@ -2015,11 +2017,16 @@ static int move_freepages_block(struct zone *zone, = struct page *page, int old_mt, int new_mt) { unsigned long start_pfn; + int res; =20 if (!prep_move_freepages_block(zone, page, &start_pfn, NULL, NULL)) return -1; =20 - return __move_freepages_block(zone, start_pfn, old_mt, new_mt); + res =3D __move_freepages_block(zone, start_pfn, old_mt, new_mt); + set_pageblock_migratetype(pfn_to_page(start_pfn), new_mt); + + return res; + } =20 #ifdef CONFIG_MEMORY_ISOLATION @@ -2047,11 +2054,19 @@ static unsigned long find_large_buddy(unsigned long= start_pfn) return start_pfn; } =20 +static inline void toggle_pageblock_isolate(struct page *page, bool isolat= e) +{ + if (isolate) + set_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); + else + clear_pfnblock_bit(page, page_to_pfn(page), PB_migrate_isolate); +} + /** - * move_freepages_block_isolate - move free pages in block for page isolat= ion + * __move_freepages_block_isolate - move free pages in block for page isol= ation * @zone: the zone * @page: the pageblock page - * @migratetype: migratetype to set on the pageblock + * @isolate: to isolate the given pageblock or unisolate it * * This is similar to move_freepages_block(), but handles the special * case encountered in page isolation, where the block of interest @@ -2066,10 +2081,18 @@ static unsigned long find_large_buddy(unsigned long= start_pfn) * * Returns %true if pages could be moved, %false otherwise. */ -bool move_freepages_block_isolate(struct zone *zone, struct page *page, - int migratetype) +static bool __move_freepages_block_isolate(struct zone *zone, + struct page *page, bool isolate) { unsigned long start_pfn, pfn; + int from_mt; + int to_mt; + + if (isolate =3D=3D get_pageblock_isolate(page)) { + VM_WARN_ONCE(1, "%s a pageblock that is already in that state", + isolate ? "Isolate" : "Unisolate"); + return false; + } =20 if (!prep_move_freepages_block(zone, page, &start_pfn, NULL, NULL)) return false; @@ -2086,7 +2109,7 @@ bool move_freepages_block_isolate(struct zone *zone, = struct page *page, =20 del_page_from_free_list(buddy, zone, order, get_pfnblock_migratetype(buddy, pfn)); - set_pageblock_migratetype(page, migratetype); + toggle_pageblock_isolate(page, isolate); split_large_buddy(zone, buddy, pfn, order, FPI_NONE); return true; } @@ -2097,16 +2120,38 @@ bool move_freepages_block_isolate(struct zone *zone= , struct page *page, =20 del_page_from_free_list(page, zone, order, get_pfnblock_migratetype(page, pfn)); - set_pageblock_migratetype(page, migratetype); + toggle_pageblock_isolate(page, isolate); split_large_buddy(zone, page, pfn, order, FPI_NONE); return true; } move: - __move_freepages_block(zone, start_pfn, - get_pfnblock_migratetype(page, start_pfn), - migratetype); + /* Use MIGRATETYPE_MASK to get non-isolate migratetype */ + if (isolate) { + from_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_MASK); + to_mt =3D MIGRATE_ISOLATE; + } else { + from_mt =3D MIGRATE_ISOLATE; + to_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_MASK); + } + + __move_freepages_block(zone, start_pfn, from_mt, to_mt); + toggle_pageblock_isolate(pfn_to_page(start_pfn), isolate); + return true; } + +bool pageblock_isolate_and_move_free_pages(struct zone *zone, struct page = *page) +{ + return __move_freepages_block_isolate(zone, page, true); +} + +bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page) +{ + return __move_freepages_block_isolate(zone, page, false); +} + #endif /* CONFIG_MEMORY_ISOLATION */ =20 static void change_pageblock_range(struct page *pageblock_page, @@ -2298,6 +2343,7 @@ try_to_claim_block(struct zone *zone, struct page *pa= ge, if (free_pages + alike_pages >=3D (1 << (pageblock_order-1)) || page_group_by_mobility_disabled) { __move_freepages_block(zone, start_pfn, block_type, start_type); + set_pageblock_migratetype(pfn_to_page(start_pfn), start_type); return __rmqueue_smallest(zone, order, start_type); } =20 diff --git a/mm/page_isolation.c b/mm/page_isolation.c index b2fc5266e3d2..08f627a5032f 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -188,7 +188,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ unmovable =3D has_unmovable_pages(check_unmovable_start, check_unmovable_= end, migratetype, isol_flags); if (!unmovable) { - if (!move_freepages_block_isolate(zone, page, MIGRATE_ISOLATE)) { + if (!pageblock_isolate_and_move_free_pages(zone, page)) { spin_unlock_irqrestore(&zone->lock, flags); return -EBUSY; } @@ -209,7 +209,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ return -EBUSY; } =20 -static void unset_migratetype_isolate(struct page *page, int migratetype) +static void unset_migratetype_isolate(struct page *page) { struct zone *zone; unsigned long flags; @@ -262,10 +262,10 @@ static void unset_migratetype_isolate(struct page *pa= ge, int migratetype) * Isolating this block already succeeded, so this * should not fail on zone boundaries. */ - WARN_ON_ONCE(!move_freepages_block_isolate(zone, page, migratetype)); + WARN_ON_ONCE(!pageblock_unisolate_and_move_free_pages(zone, page)); } else { - set_pageblock_migratetype(page, migratetype); - __putback_isolated_page(page, order, migratetype); + clear_pageblock_isolate(page); + __putback_isolated_page(page, order, get_pageblock_migratetype(page)); } zone->nr_isolate_pageblock--; out: @@ -383,7 +383,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, if (PageBuddy(page)) { int order =3D buddy_order(page); =20 - /* move_freepages_block_isolate() handled this */ + /* pageblock_isolate_and_move_free_pages() handled this */ VM_WARN_ON_ONCE(pfn + (1 << order) > boundary_pfn); =20 pfn +=3D 1UL << order; @@ -433,7 +433,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, failed: /* restore the original migratetype */ if (!skip_isolation) - unset_migratetype_isolate(pfn_to_page(isolate_pageblock), migratetype); + unset_migratetype_isolate(pfn_to_page(isolate_pageblock)); return -EBUSY; } =20 @@ -504,7 +504,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, ret =3D isolate_single_pageblock(isolate_end, flags, true, skip_isolation, migratetype); if (ret) { - unset_migratetype_isolate(pfn_to_page(isolate_start), migratetype); + unset_migratetype_isolate(pfn_to_page(isolate_start)); return ret; } =20 @@ -517,8 +517,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, start_pfn, end_pfn)) { undo_isolate_page_range(isolate_start, pfn, migratetype); unset_migratetype_isolate( - pfn_to_page(isolate_end - pageblock_nr_pages), - migratetype); + pfn_to_page(isolate_end - pageblock_nr_pages)); return -EBUSY; } } @@ -548,7 +547,7 @@ void undo_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, page =3D __first_valid_page(pfn, pageblock_nr_pages); if (!page || !is_migrate_isolate_page(page)) continue; - unset_migratetype_isolate(page, migratetype); + unset_migratetype_isolate(page); } } /* --=20 2.47.2 From nobody Sun Feb 8 06:22:35 2026 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2070.outbound.protection.outlook.com [40.107.236.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C5A62254877 for ; Mon, 2 Jun 2025 23:53:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.70 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908392; cv=fail; b=njWFuedYFrzENZX/9/dMPB93BaCgZZQ/eZtbK+APX0QUwNZfNO1RV/QMUZ19DkOvWQD1LlJ2uOP3go1htFQgW2MKafkrjeieGFFAIFd433P95lBu1RdORyRRauNP65saHd2UQSHoMTTEWQ7n9r7HX0nmm6LYVAyZ7i45W1A/zxY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908392; c=relaxed/simple; bh=mGHan1f3pvsQ3eX/Uo2gFvcOfPdp8iVlcI8dL4nzZHc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=dDr9e83n2f3afkbOCAtwo6rHbO6Sm2lAcKi3aUdLTbzKrRMqW1rwnd+asJa+0OP6mO+A25VmJkdb9fBgbukjn+HC7rzGj6d9zIeoxRvGkU9JN+6phOveO27WArXfmASAetdi9gfis8q1nn4KKtxDFAP4b75fK2y96pgWwYz1QXk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=BgixA/0b; arc=fail smtp.client-ip=40.107.236.70 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="BgixA/0b" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GEryMbA3bFzBPn/MXAaoGUwAeGQstNUdQno88q9kr/bX7e7D7eyABdJi/hVC5GWGpNdlAurp5wcqLITHP/+D627DdXyGl7I9zIdHr19RkwAfAWpFWfBi0PBs9HV/k5ImP7/14lPPzXAb/x8ahAy8JoGXYmJxLD2EPaGTh7ZkSuyK9eD6YaBcWu+DHfUugRA88bHPWG8zu02dw7PQUPz0DGM2atNmT8Vf9pLrw7uYW2gC4wwtm2cSZlfHbmcsSNeoq4GZuPChMgugMuoV49eqf8g9357iJTIFkyCVOzmC9Obn4iCrXdVWaFD1F/dxfSPhqJhYb8KoS2s5tCrT20w6Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6Ca2y4Ll6GhldlIIPPkaafx8RWllJVTRpRpAU6TyVdY=; b=p9SXYJAXwWny08uIQn7v2pN6wC2s/UtZEFmudIjvXACxlbnm5eu/u3VOtsWI68YfSaTR9Wr/pzCS/JFlyVaehrFRHF7NyzkeDv3lrX6ing49BwKi2RQTTXDbhHBimnHTQT/ybM7hX9lJCWXu6DnnV7tYYd4qnL0nTSUGI9MNsLVtz+A8KX5yVk7dHzeGkv6YMAHCtjWUnDUCn3hclrP/azf2i5OCF+feZJOpllJTDdWfGE7s2CG0skulUCrmJtWu+EJld78dLGtn/99bcuMHSS0kLD2MbG09pAXj14hJkcZ1T7FG72X7YAjt2xTy5A90QFc+4+TBOMo9T7o/FcrjJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6Ca2y4Ll6GhldlIIPPkaafx8RWllJVTRpRpAU6TyVdY=; b=BgixA/0bLrhWgblthHMjAyLwEjNdfQDs1SBEySHOsyiVmBJvt/hGRZtYBpiaxk5NVpM8CWM/DcgGrvy80hGoFaCKNkOX+/wqPU60++bjUkYF5OmP3Yit2SPtKP8SuD8UFkT6+WbQo368+CdZDSSSDBqRDe8HH6L63A9+MhHT9WAI6ht6NawXiZr/HKQFypbufjJtnyOfaGe9/6vQORM769d2MauyctXkMKGhAks9amydsztW/G/X1ZsEUdQ9ozBWoA10mbfNOv+egOC2Kt4XW5HSaHmvYy00M7gp4lNd5/a862pG0Rp2gAOzyUTdoo6VuEogF2h7ecguX6Hh4Lp5LQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by LV2PR12MB5726.namprd12.prod.outlook.com (2603:10b6:408:17e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8769.37; Mon, 2 Jun 2025 23:53:04 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8792.034; Mon, 2 Jun 2025 23:53:03 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v8 5/6] mm/page_isolation: remove migratetype from undo_isolate_page_range() Date: Mon, 2 Jun 2025 19:52:46 -0400 Message-ID: <20250602235247.1219983-6-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250602235247.1219983-1-ziy@nvidia.com> References: <20250602235247.1219983-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR13CA0025.namprd13.prod.outlook.com (2603:10b6:208:160::38) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|LV2PR12MB5726:EE_ X-MS-Office365-Filtering-Correlation-Id: 7890dfa1-65a6-4a16-2159-08dda2309d48 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ZfXnRBnj7zMaO4WIDvlWEjGgY20euaYjOl9eOBzdQj3bP/cP1Gy7/NNLbsY8?= =?us-ascii?Q?DiqBDg1YZXydCzdOfdMgqnjS+EesPOcrOqQWiqfq+w/wQkQ4etI1IwQJC5LM?= =?us-ascii?Q?sgY6NYwxHgJeLh3LWCgSdaJxKTjMP6o9wRVxndSz5BdwdDU9diTHmOmkOT7d?= =?us-ascii?Q?6cEd0aTjZueHZMON853Hsxq293SPp0+5cPV37liqDQFBmukJUA0bfYEARoIJ?= =?us-ascii?Q?UoFSvOHM9QzdA3DocGQictdDh2ILRLtgbYHISJIPTdt71fNXtovzpjedmkFW?= =?us-ascii?Q?zCx1KRGTEhvCEaXGlrnbrWgkFyKESxaWhjaokmfhKueyrMI+pnJe/Ygto6Yh?= =?us-ascii?Q?KHAOAjCtwmZD7QFzGyTdfsIvn5ekeEFbVlS05/vCRET6GqYQdMm35gxqS+t5?= =?us-ascii?Q?D/xI++3obZmRwSP9b9vfSCJrHruwI8Sduy0smidlKhguJSipaAWGujiuC1jK?= =?us-ascii?Q?JjohPZKdryJBkdZl3po9CnPhy4b9Tr2nTY9PLqmPINgD2bJJiJRklpmJ4xh3?= =?us-ascii?Q?F/9GuAJYdbTFshPKdJyNshMxxxeBAHt9voLQtJWJTd0yKiIeTjt9SEmc3Qpm?= =?us-ascii?Q?jdwtQI9lO/+KqksdwmOu/4+WWS8eY9esPOiF+E/cuaxD5c8OY9phIafYXazr?= =?us-ascii?Q?DVBIwQwZpY904dR3fFosJOtBOgWdsV18OYAyq1Rfj/s2HZiK44+LTCNa944Y?= =?us-ascii?Q?9WGGVWAt5WZIz9Ig3GY5r9Mr+Od6Bt/d+k14AyxW4UBhMDJ1Jejvf6+Ay9mD?= =?us-ascii?Q?eGaboMz3eSIr3NFOIgaVJPQwVa7CK6LqniWK3ORo5ZktIrl0BXppy+wIg3QU?= =?us-ascii?Q?0hy9unLuX77etouolxOukVacMo0J3NXYKWbQVCkiIhGBBILbM24hA1eYDKEM?= =?us-ascii?Q?f+c+S9dTVdbHVXgsFKa8FYn86tJhQyN3N3dmcW9TWobx6ECsrfkqAVtgTBS1?= =?us-ascii?Q?ml2c4/p7nfFk8Dc58PXf3eJamZiKy/g1806iUKHmjsguByII4Rpcr561X56n?= =?us-ascii?Q?wfaelFQXpvxZ9J4esSpxKCVQ85sSFyEvKDhTigPLDFcelS2/E2VHitTDE1hJ?= =?us-ascii?Q?QhjiMm6leERcAYCJ7bSsDqf8Knw+DXfaAr/qFPOVw0EgiCRq+1s8OUNs0RCe?= =?us-ascii?Q?REvQaDh36TkvXLrCngo/XmrvVgOoX/xatSX//xR0BkQzY8sIX3jvmqDINPXv?= =?us-ascii?Q?yetyvz5dGHrYq0RYbk+U0LxXLF5QTRxFi/sKHotsvJE/FHSjD+QPtfv+Du45?= =?us-ascii?Q?xPsY9dBwjzAzhziWGgT6AqLcy/nVpb3tSBNKB/xDWE4PBULJUVGgiCq9dL3v?= =?us-ascii?Q?/DToiW0ppCKyj1iCuAf0NZlVbut40yz9z2NQvyrpF89eyM5bkVJD3A+rQmC8?= =?us-ascii?Q?iK+meJeHqAt/VpXxvP3Zbb/kB34qpgSH2HxAX3NvtdleDFu/qGIGzwymYWYV?= =?us-ascii?Q?5rdgaYi02lA=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?FF7Rbg+InB35HvmCmg9K2gA9kAwIpTuLTh0zQja1pcBXq3X+b+tZ+re1esWO?= =?us-ascii?Q?29+uKbIPBYhUNj36y4z9XUHQm1P3fRhSjHtwhZA5PyoiiJDLwycXdn7SgjJz?= =?us-ascii?Q?Yu6n8uPbjJIDfdjzLU7LoG6dyTsSNG6/+teyY7wdWSYZGXycpJU9yrjaoyxQ?= =?us-ascii?Q?uR5BQ5VOeFf3iAZVCvCXbOfvRqai70m2uo7tq0OT5aNCs1PsUl83avJ52JHE?= =?us-ascii?Q?RYq6aYNi5p62hG14pQGfPOBp3Hnerdtk/6RSYkCF3h1X5p/oS8JtOKdRjH4z?= =?us-ascii?Q?garw28BNE6P2WMO+GR6nnEZk2vk06evo2nr6Do4NYxGjeLEDMn+x8HrkEPN1?= =?us-ascii?Q?j/tSSux3oP3TPy5WxXzEV+ekJeuyryH6RERlJzH1Mitxe3cvNjTl/NBd/3QJ?= =?us-ascii?Q?iLHAswuukIXGuWdronc/8e3uwZaTeaVI3oOMgWazVzVUkjudUPebpSryg486?= =?us-ascii?Q?ZR6/S6kpmYVIL5IF8J8oKF57CHrIvuSwkRBpNwTWGygSneM1viRzhlYjgHK3?= =?us-ascii?Q?HyM5Ug2YwExyHsvbqvy95i4+/U+OI4XoAPi3YZI6G7WJruTSqvbR4avTTWNq?= =?us-ascii?Q?1tkKLEWjT9kCw3Y5rlsvkwvkOAPnk+UHx5lvyUNfiz8uvS2Kf9ToFUui4R1h?= =?us-ascii?Q?DavS/jTYJ46TmBAol6g9VAK8a0vYZQWUheEF0pQ/0t23dzyUD3bsuWnC/Vd6?= =?us-ascii?Q?6SpIaNhAvP5wqJpLc4qU+OuW7pbbZo98EslyDSobaXtLF65ucYGnqoGhkbSL?= =?us-ascii?Q?iJjJ4LKmaMc0bxAyrz3SD+dWXascqxhsauy32YlcnyRM1O494HV1pjPECow6?= =?us-ascii?Q?Wngo99n3PxCZ/QhtGz7kB+c25VVFG101bgZSZEDNsWUZGjOgqJaxCisL8LCq?= =?us-ascii?Q?uRKenRH1FVH1eAGY4U9aw3vGGtT/QRn1AgDIcWKdGr/Yke+V3Sa8HTq0MOY0?= =?us-ascii?Q?aV0KZWL9PEZpQvz9y3i8Y4OCtoMvOqkqpdfP7zK7rks4RShc9kvnyxDCThXA?= =?us-ascii?Q?CIVLf6Qczb3ORe2Y7r4F5FUQsu2KcsNcravrQJaIF22Twl8Gb45NN5EaEwpk?= =?us-ascii?Q?AKhg1MSyc3ukVyQRbK//I2bX6YzIrzovRSurVIa5a5mtROPLArzHKFjYXjrp?= =?us-ascii?Q?OkHpe6HxTPLfMXJfeJ8PzkJYv1scWAEp4wvknL9UnMUSCpFbAzx5XWzbP/WU?= =?us-ascii?Q?0grUtxF7ZghIWUO0OG4xKFNgc9Rvm/jLSwxu9ShSPQoNdTDklfmBM3D7s5r+?= =?us-ascii?Q?UG+MVqaA9VNq22sztqg/U5UjMoN1kCD0+Vl1XYA1nD8+yNGTchumwld4F9CY?= =?us-ascii?Q?4ZYTfsxm+rgMm7NrTwHToRWFwra8/VwFwYXLZvl6cuVYBApyfjWhEnaGRBFt?= =?us-ascii?Q?NqfVOuhkAa+6PIfLt4ATbe7NDp/E7WXDJR4cSPxILLI8PPvgt8WuBptfoyhS?= =?us-ascii?Q?VX2zuP7SDVBW0/h6IaR5BrxDVop8u1mQ1ngiqYcsDbqjNosmRjA4SBr2COoP?= =?us-ascii?Q?I05r5bsbn8ZoO1ZRfdv5G86ycwcmW/4bFVmbAgf2U1gvhdWLuMo+H9ia2ohp?= =?us-ascii?Q?gfOD41Uz/vkgCWmVMJrfEWtix2tq2c540q63SPOO?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7890dfa1-65a6-4a16-2159-08dda2309d48 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2025 23:53:03.8621 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CjB8QaowE56MSoeBI7GDoVh+654go96YEska1vgTQd/UvfNE05EBb8t1DvNulq/m X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5726 Content-Type: text/plain; charset="utf-8" Since migratetype is no longer overwritten during pageblock isolation, undoing pageblock isolation no longer needs which migratetype to restore. Signed-off-by: Zi Yan Acked-by: David Hildenbrand Reviewed-by: Vlastimil Babka --- include/linux/page-isolation.h | 3 +-- mm/memory_hotplug.c | 4 ++-- mm/page_alloc.c | 2 +- mm/page_isolation.c | 9 +++------ 4 files changed, 7 insertions(+), 11 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 7241a6719618..7a681a49e73c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -51,8 +51,7 @@ bool pageblock_unisolate_and_move_free_pages(struct zone = *zone, struct page *pag int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, int migratetype, int flags); =20 -void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype); +void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n); =20 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, int isol_flags); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 16e3ad874144..4626064705ac 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1233,7 +1233,7 @@ int online_pages(unsigned long pfn, unsigned long nr_= pages, build_all_zonelists(NULL); =20 /* Basic onlining is complete, allow allocation of onlined pages. */ - undo_isolate_page_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE); + undo_isolate_page_range(pfn, pfn + nr_pages); =20 /* * Freshly onlined pages aren't shuffled (e.g., all pages are placed to @@ -2119,7 +2119,7 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, =20 failed_removal_isolated: /* pushback to free area */ - undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE); + undo_isolate_page_range(start_pfn, end_pfn); memory_notify(MEM_CANCEL_OFFLINE, &arg); failed_removal_pcplists_disabled: lru_cache_enable(); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6ddf4fd6b2bd..ccb21af002b0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6986,7 +6986,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, start, end, outer_start, outer_end); } done: - undo_isolate_page_range(start, end, migratetype); + undo_isolate_page_range(start, end); return ret; } EXPORT_SYMBOL(alloc_contig_range_noprof); diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 08f627a5032f..1edfef408faf 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -515,7 +515,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, page =3D __first_valid_page(pfn, pageblock_nr_pages); if (page && set_migratetype_isolate(page, migratetype, flags, start_pfn, end_pfn)) { - undo_isolate_page_range(isolate_start, pfn, migratetype); + undo_isolate_page_range(isolate_start, pfn); unset_migratetype_isolate( pfn_to_page(isolate_end - pageblock_nr_pages)); return -EBUSY; @@ -528,13 +528,10 @@ int start_isolate_page_range(unsigned long start_pfn,= unsigned long end_pfn, * undo_isolate_page_range - undo effects of start_isolate_page_range() * @start_pfn: The first PFN of the isolated range * @end_pfn: The last PFN of the isolated range - * @migratetype: New migrate type to set on the range * - * This finds every MIGRATE_ISOLATE page block in the given range - * and switches it to @migratetype. + * This finds and unsets every MIGRATE_ISOLATE page block in the given ran= ge */ -void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype) +void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n) { unsigned long pfn; struct page *page; --=20 2.47.2 From nobody Sun Feb 8 06:22:35 2026 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2050.outbound.protection.outlook.com [40.107.101.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F1922561D4 for ; Mon, 2 Jun 2025 23:53:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908398; cv=fail; b=KqDKuFwn7zOmcWGGjh4mCfwsF1LD9GQXPRa12NyRw6vsC4JivNhwsPAIyQ3NtY1wHIdU201TJzHNFBFocSd3RU9Os9DhnMI9Fbtg8oFzYr1lXABaYCFAvI31SSGdnn3ZiGSJmw90+t1mo37SpHiL4+HfGgq6BLh82RqaEiFOMH8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1748908398; c=relaxed/simple; bh=yHB2O6VMIgqQmi1VpWZtYnlDCehi2qTXVzHzj8x0YGs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=sXAEc73NKllssn8hs3t3LsQE4KbKZMmBEm57wkWjp/flRC+yz9yZJQCjCbL66NZJUtLM/dbE6tyrX9RF9MGhqegwnqaAQI7iZl1Gbib3rHYz0QBRUiSeA2b0EKa8jU0DUIR0LmW0HvPn4TKpwZHjUhXGGBxhW/y63ttkWyHJEFM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=gMG7q1R5; arc=fail smtp.client-ip=40.107.101.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="gMG7q1R5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cOq3dI4Tws2+dC9gFPRVW/mV3NQ5mI/Ior0lUfrbV1FUPz0TMxOC9Wv80EjfIedyFgPfvmOUfqRTS/7Z7W/KC0vz1A+NodA/9e4px3Uu0fiAo7jG5HBHWXWN34j/q2jOV5Y7SImaP8tgnxBqDY45D6sY4aHEKicKNh8/2EgadY6VOzd13iBgjJ93OxCCH9UcAX25p2vZi37/DLvHEFiVS7+vrj2qPa3fE5Z8vAqe1QDqiJxKzkQR5swSoC+ArxmnuNiEQ2oQ387wkN9TOmDPjvreEG6BXw/k9TtYhBFCUahzJeYTErr2XUdq+FotsUlkNUR8D7AZ73GvG+PMP4RF+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XPYeAlQll8sVHoT4JWslf9XEsn6UipUm64tBSCQSXQ4=; b=Qlb6162kv3Gq3TdYi/H/vUBre+288euo/Xr9NLXRhzyQR8aYNf31tkFTfdLZAsIfKq2N52snkoQrFrCYZbrqfoszZRwO859s+J6aRpLz2HcKgslPYNexgK2MibpxAM3cTSLr1xejIDDW2qDoRQfb5xb8eadBKpC7bU6bCXvAmBqyDakTDEC4lbSo1m2DHqYh6KiBcvLfXeqqv6dlV/pSvtP3rawfMSwr3anMB8CMoh2l5R4ndP8/W6huHcvB2rgIOMsV6CIh5EQfpkCjxvboOI/SRns7LRDykMPe2GNtDbyar67Xq8SV0XDp99Nja6T9PcscWZeU1oSxhzqf9nSOlA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XPYeAlQll8sVHoT4JWslf9XEsn6UipUm64tBSCQSXQ4=; b=gMG7q1R5yk1FT5HupK8DqKC/t52CSyVNtxSp/VYCCknmoH+tPdTE2iNtVVpO0Z5UbN51OZzqHOLHpFbHSFBXHIZGYGibAydjTguFc+4rsDhqWyvYC9kcmuS0flQGeH1dlmr6FRb5qDB82yNdiuzvLemxQMJ5WGD9pw2SyPy6yNsqdkJ6e+mZ+Ej31wLSmvAhWFKYqp10NTUii9Y33cjLfvyZu4KUTcqnde6u/0YvLukUYxz4k96uS4Yw9c9yhdvh2628oODJu4XQ1P02UIwJvMYnDBNnPUDnWtXQuObnZMkMrP8dtZ+e88z6CJvJbRXnNm9lfd1a+u+/FfTCL35lXA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by DS4PR12MB9658.namprd12.prod.outlook.com (2603:10b6:8:280::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8792.34; Mon, 2 Jun 2025 23:53:05 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8792.034; Mon, 2 Jun 2025 23:53:05 +0000 From: Zi Yan To: David Hildenbrand , Johannes Weiner , Vlastimil Babka , linux-mm@kvack.org Cc: Andrew Morton , Oscar Salvador , Baolin Wang , "Kirill A . Shutemov" , Mel Gorman , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Richard Chang , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v8 6/6] mm/page_isolation: remove migratetype parameter from more functions. Date: Mon, 2 Jun 2025 19:52:47 -0400 Message-ID: <20250602235247.1219983-7-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250602235247.1219983-1-ziy@nvidia.com> References: <20250602235247.1219983-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN2PR06CA0027.namprd06.prod.outlook.com (2603:10b6:208:23d::32) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|DS4PR12MB9658:EE_ X-MS-Office365-Filtering-Correlation-Id: 1d637cfb-94bc-4faa-8f86-08dda2309e39 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?SLmOAg8y3uzigg9pEu2eWVA9ClGZ496vTf4ggaMXhRk8H5E8S6UUoFyUbEfv?= =?us-ascii?Q?oPQ6QhL7X994Fsx2+xJ6SsU599cSYhn80jhrEsdsurnnL9Ig5lzAc+r8haVE?= =?us-ascii?Q?RLr4y0/WLJh/MC1vchcw6dgZIxU97wD/JEffPSktCgtEfG5uCD8q/ntVqQ1I?= =?us-ascii?Q?d2msTuSYaPQVX7IXrLc1iLK5KzUnNvhrjC2M7a8Rgi0mczxpHmGjdJOpkshY?= =?us-ascii?Q?EIgdPIWyt60AkoKn493aMB0ABpl6gYFef43WKP8et25cWQPotfkZk/JL0fy5?= =?us-ascii?Q?eI906F2ZA6paqnm7ZJJSTZtnjlIqkLKbHEp2ATp7nM07Bn1pM7vbbKq42gDu?= =?us-ascii?Q?YuAqa1eB+6Wxr3F4v2dmmMdBpOHxh6Vu0bUEasHsCIIVp2HMnQXcSO6mn9ac?= =?us-ascii?Q?URU0yGAlR8I0zC8RSFNwh4rwfORnSIsSt3HJq2G4lq1NbV8LpjLNdlNyMU/7?= =?us-ascii?Q?FWwdrKznYyujZKchD1iJduf/Pop+Ya8PwYxhTJBO9AuqF2gOofQCYW5s4fzC?= =?us-ascii?Q?mHYjX6y+v2UopcUK+XnrFDFSTr24YkbqAjta7QrbQhquMLm98VynTviYF5VO?= =?us-ascii?Q?VYldsfIxa2olvDt99uWHVNWrl0GmDYTO8iMpkr2y5eS+TNyLIKcDrWiidnhe?= =?us-ascii?Q?thj7SEZzUxkMqwqrDioHH8AWRiw+imISJlfLnkbnknv2RtfbAHl9N4JdvUhq?= =?us-ascii?Q?ijGM/FBydVDL5f7pauoaZJP+ekbP71sfJ4U0rE6nRNw5V146mOZscR21Rr6R?= =?us-ascii?Q?g+FsRFYUwv+4d4yD0+XjYGjlhoohOFw/TrsgZUiTyUfcPrMPckjNSepWCQ90?= =?us-ascii?Q?5mrPmfgiLaRo2dB+2FdUSrM09Jdpfxd/Z9GwPRAfl6Hr3WrVh2gW8Vl4T2sw?= =?us-ascii?Q?KXa0CgalyTJ0xucxYKeVmvan7W98TPy1sDikn5PhHbNiQyO5VhwPIOkMzfry?= =?us-ascii?Q?zEBMYHGNEErq7LILUWzt8OjwUphWtg6rQA58f2mOz2+t2ekcRs6YCPotnDKZ?= =?us-ascii?Q?fOJpVAnQ0r2l9fv1BW6XM1GkYzPEIe1FUskaOeB1BZeQ/ndulw73R5H/9Mi0?= =?us-ascii?Q?qnvQ+2zKi/z6JMoSGsZSJFI9KDQMHglS30/yr45nhEy5Qmw0YsBRdQkwNhHB?= =?us-ascii?Q?RijOL87SVWLFK35i1bCke5LsvgVBM5C7NFRisDbzroChrxoMTDnRwFu96yxJ?= =?us-ascii?Q?rS/JIEVgTBJxLasGA13uZora9C+CpLp3IhX1gGiEJkfGMwExVaWJOgSkvmi0?= =?us-ascii?Q?qiaGl7yFLne87ywGr5dnRLde5d5P6W3k/OC175TgfQkrYxKSxvUDdPrNW0Ft?= =?us-ascii?Q?0t5L56MRmvqxlgCMluKTLNqs+q3QPtHxVQt8M2zXfSe9fdfRlHG1Tyv+0IXr?= =?us-ascii?Q?2LjXq886lxxHlczzsMi9tYg5V5lzqt2s/bucMUrt29xs92HuJf49ukaTRzCo?= =?us-ascii?Q?htN01u39GNw=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?l/OUKTaBvgZfUND/OkNQ6zbT3xEM7HqEECKfIbauPykwzPv2lRuapWwKrx7W?= =?us-ascii?Q?ubG+NpNaLAE+08IxtVtBQ+E97LzukAkuDcgUxO0qibmumfocswzkGlX54ZYk?= =?us-ascii?Q?R48tX80cdaSliMnHPFnofWY1z1USOepo3CxRZFDJAmOFHSKAOqPVKudHY4U9?= =?us-ascii?Q?YA6kEiAkvXiCeAJSGaCTB+rczWR9w++qMAGqq8piF6pbhAVfeHpVZa1uopjg?= =?us-ascii?Q?LRmZj2rmQEcturgWvdXav7TNY9eVbKBOcfT4IxtcEnliu/Y8v27JjiEVdBnk?= =?us-ascii?Q?g43Mb0v/eZXu31e+tqgsTg40dPC31fGNMkIsb01TXtjwRsgPY+Godf5fdRjA?= =?us-ascii?Q?rvOpARW3EOXqB60QnNwLLviNYznDdLzdW2KZ12quP+53U+QHnfjguLVWy6le?= =?us-ascii?Q?qQSNZx/BY6NQcrhKbMq4Gbc/smaDugmUPciFCmAyNuQVPtm/JDppVHo7FSDi?= =?us-ascii?Q?IyM8w7AiPE75dR/qo/PM98WAyT8yqHnF9Le9V0lMRlTTFNxhOJ2GYhpmVrJu?= =?us-ascii?Q?ua0WCZzjC5zq0fnlx6yt8b01aV8eBilR1wXC8xb/iXD9JMyTOuIN9r1SRLYj?= =?us-ascii?Q?TGhdDGXYSvFKigCxMdC5sqEYhS8We7eEa6N5sOAvJcg1IFWIVy9VlB/LG3kQ?= =?us-ascii?Q?9vLzMLxKw6h/WrdtLA444t0KQpflPCAMFmoDGdZsuXuljJ15y3f15ZkGLVTh?= =?us-ascii?Q?o1l2/CkvoOlT02/QS2Py0BGs4s/4lvT1R7kbEB4xFFA2tOj9+tySx3S5mX1l?= =?us-ascii?Q?Eshy3QKkmLKUUuSeZzK5xDzrbJT+W/CTgem81eX+c3qCWm+UWp/b99K1Xhv4?= =?us-ascii?Q?Rbvg1gKKNt9rWKW0fn9+Q6TQN/AC8Wzd3WnhwvSWeP3/sfK9lLuN6cuneVWK?= =?us-ascii?Q?2XoyxQKQIXgwukw2ugvdVVI8DlNY2z0vn/FYQqTrjBkn4Z3/TXBC4Sxr8wsk?= =?us-ascii?Q?852XHVE38ZzXxz4GdPJfe9lUlwUC7SnjSSsgGIijvxEqHTIH4JRh6ct/IjZK?= =?us-ascii?Q?E5SReAUnGDoI6NFrbVB/15QBqPPgYXhXsmDyUeBX8tAJPxZqG40WZZrwbDAI?= =?us-ascii?Q?MoVBzyATg3kqptrl68j+GY0b+lS2HFOWS93IxybukQKFJuVCbi8gjVkWXr4A?= =?us-ascii?Q?6N5RCFvedN4qLoC9NDi4/KGGj7Z7HgRES22MLJ4YLTy8EQkt0W8LgTFW7ZjV?= =?us-ascii?Q?XnsEceTNTd1Z+j+JWHMZ83CXhT5yTRLPnHz+lU2FkeQY9nk65VYWVsubidLw?= =?us-ascii?Q?YcQHHOgiKeCWnLi+Ro2RxN+Z0izZFSnSO2T4nTG/kcRujksOe6bq+B7mgi9C?= =?us-ascii?Q?pn0LTER0d7MDJVsbVh/XE8fUQ7G2aVm08LPoJgvSzBnrvVqNZAY3LmwQbnKs?= =?us-ascii?Q?VlZY+FVAU3gj58Gcow7thFjaZ+xpMEvmY7jTKKmu9B2NlUUn5BntJ2/LLyej?= =?us-ascii?Q?V4ro2lOQMuXfZR8FCETwOf97QNFAiTzJyyMzRzpr8pabfwUlYqafzOl4QsFV?= =?us-ascii?Q?uS8aXiMDbj/RcPn9I7LBs00imMCeTmgnqdB++fD6K7Qt4Tgm3Q2c+0alNLl5?= =?us-ascii?Q?ugQV4tAhHyXiQYDUR3e00dERGEs+Isy20VrnLL2Z?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1d637cfb-94bc-4faa-8f86-08dda2309e39 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2025 23:53:05.4534 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8ZEvo2I7htdTQRnIsUuUt9KCBGAL31SjHGqyVoyWunCW0GAPfmDijNZrxUbirn/0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS4PR12MB9658 Content-Type: text/plain; charset="utf-8" migratetype is no longer overwritten during pageblock isolation, start_isolate_page_range(), has_unmovable_pages(), and set_migratetype_isolate() no longer need which migratetype to restore during isolation failure. For has_unmoable_pages(), it needs to know if the isolation is for CMA allocation, so adding PB_ISOLATE_MODE_CMA_ALLOC provide the information. At the same time change isolation flags to enum pb_isolate_mode (PB_ISOLATE_MODE_MEM_OFFLINE, PB_ISOLATE_MODE_CMA_ALLOC, PB_ISOLATE_MODE_OTHER). Remove REPORT_FAILURE and check PB_ISOLATE_MODE_MEM_OFFLINE, since only PB_ISOLATE_MODE_MEM_OFFLINE reports isolation failures. alloc_contig_range() no longer needs migratetype. Replace it with a newly defined acr_flags_t to tell if an allocation is for CMA. So does __alloc_contig_migrate_range(). Add ACR_NONE (set to 0) to indicate ordinary allocations. Signed-off-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 2 +- include/linux/gfp.h | 7 +++- include/linux/page-isolation.h | 20 ++++++++-- include/trace/events/kmem.h | 14 ++++--- mm/cma.c | 2 +- mm/memory_hotplug.c | 6 +-- mm/page_alloc.c | 27 ++++++------- mm/page_isolation.c | 70 +++++++++++++++------------------- 8 files changed, 80 insertions(+), 68 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 56d0dbe62163..42ebaafb9591 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -1243,7 +1243,7 @@ static int virtio_mem_fake_offline(struct virtio_mem = *vm, unsigned long pfn, if (atomic_read(&vm->config_changed)) return -EAGAIN; =20 - rc =3D alloc_contig_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE, + rc =3D alloc_contig_range(pfn, pfn + nr_pages, ACR_NONE, GFP_KERNEL); if (rc =3D=3D -ENOMEM) /* whoops, out of memory */ diff --git a/include/linux/gfp.h b/include/linux/gfp.h index be160e8d8bcb..ccf35cc351ff 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -423,9 +423,14 @@ static inline bool gfp_compaction_allowed(gfp_t gfp_ma= sk) extern gfp_t vma_thp_gfp_mask(struct vm_area_struct *vma); =20 #ifdef CONFIG_CONTIG_ALLOC + +typedef unsigned int __bitwise acr_flags_t; +#define ACR_NONE ((__force acr_flags_t)0) // ordinary allocation request +#define ACR_CMA ((__force acr_flags_t)BIT(0)) // allocate for CMA + /* The below functions must be run on a range from a single zone. */ extern int alloc_contig_range_noprof(unsigned long start, unsigned long en= d, - unsigned migratetype, gfp_t gfp_mask); + acr_flags_t alloc_flags, gfp_t gfp_mask); #define alloc_contig_range(...) alloc_hooks(alloc_contig_range_noprof(__= VA_ARGS__)) =20 extern struct page *alloc_contig_pages_noprof(unsigned long nr_pages, gfp_= t gfp_mask, diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 7a681a49e73c..3e2f960e166c 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -38,8 +38,20 @@ static inline void set_pageblock_isolate(struct page *pa= ge) } #endif =20 -#define MEMORY_OFFLINE 0x1 -#define REPORT_FAILURE 0x2 +/* + * Pageblock isolation modes: + * PB_ISOLATE_MODE_MEM_OFFLINE - isolate to offline (!allocate) memory + * e.g., skip over PageHWPoison() pages and + * PageOffline() pages. Unmovable pages will be + * reported in this mode. + * PB_ISOLATE_MODE_CMA_ALLOC - isolate for CMA allocations + * PB_ISOLATE_MODE_OTHER - isolate for other purposes + */ +enum pb_isolate_mode { + PB_ISOLATE_MODE_MEM_OFFLINE, + PB_ISOLATE_MODE_CMA_ALLOC, + PB_ISOLATE_MODE_OTHER, +}; =20 void __meminit init_pageblock_migratetype(struct page *page, enum migratetype migratetype, @@ -49,10 +61,10 @@ bool pageblock_isolate_and_move_free_pages(struct zone = *zone, struct page *page) bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page); =20 int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype, int flags); + enum pb_isolate_mode mode); =20 void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n); =20 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, - int isol_flags); + enum pb_isolate_mode mode); #endif diff --git a/include/trace/events/kmem.h b/include/trace/events/kmem.h index f74925a6cf69..efffcf578217 100644 --- a/include/trace/events/kmem.h +++ b/include/trace/events/kmem.h @@ -304,6 +304,7 @@ TRACE_EVENT(mm_page_alloc_extfrag, __entry->change_ownership) ); =20 +#ifdef CONFIG_CONTIG_ALLOC TRACE_EVENT(mm_alloc_contig_migrate_range_info, =20 TP_PROTO(unsigned long start, @@ -311,9 +312,9 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, unsigned long nr_migrated, unsigned long nr_reclaimed, unsigned long nr_mapped, - int migratetype), + acr_flags_t alloc_flags), =20 - TP_ARGS(start, end, nr_migrated, nr_reclaimed, nr_mapped, migratetype), + TP_ARGS(start, end, nr_migrated, nr_reclaimed, nr_mapped, alloc_flags), =20 TP_STRUCT__entry( __field(unsigned long, start) @@ -321,7 +322,7 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, __field(unsigned long, nr_migrated) __field(unsigned long, nr_reclaimed) __field(unsigned long, nr_mapped) - __field(int, migratetype) + __field(acr_flags_t, alloc_flags) ), =20 TP_fast_assign( @@ -330,17 +331,18 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, __entry->nr_migrated =3D nr_migrated; __entry->nr_reclaimed =3D nr_reclaimed; __entry->nr_mapped =3D nr_mapped; - __entry->migratetype =3D migratetype; + __entry->alloc_flags =3D alloc_flags; ), =20 - TP_printk("start=3D0x%lx end=3D0x%lx migratetype=3D%d nr_migrated=3D%lu n= r_reclaimed=3D%lu nr_mapped=3D%lu", + TP_printk("start=3D0x%lx end=3D0x%lx alloc_flags=3D%d nr_migrated=3D%lu n= r_reclaimed=3D%lu nr_mapped=3D%lu", __entry->start, __entry->end, - __entry->migratetype, + __entry->alloc_flags, __entry->nr_migrated, __entry->nr_reclaimed, __entry->nr_mapped) ); +#endif =20 TRACE_EVENT(mm_setup_per_zone_wmarks, =20 diff --git a/mm/cma.c b/mm/cma.c index 397567883a10..9ee8fad797bc 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -822,7 +822,7 @@ static int cma_range_alloc(struct cma *cma, struct cma_= memrange *cmr, =20 pfn =3D cmr->base_pfn + (bitmap_no << cma->order_per_bit); mutex_lock(&cma->alloc_mutex); - ret =3D alloc_contig_range(pfn, pfn + count, MIGRATE_CMA, gfp); + ret =3D alloc_contig_range(pfn, pfn + count, ACR_CMA, gfp); mutex_unlock(&cma->alloc_mutex); if (ret =3D=3D 0) { page =3D pfn_to_page(pfn); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 4626064705ac..3eea3008727f 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -2009,8 +2009,7 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, =20 /* set above range as isolated */ ret =3D start_isolate_page_range(start_pfn, end_pfn, - MIGRATE_MOVABLE, - MEMORY_OFFLINE | REPORT_FAILURE); + PB_ISOLATE_MODE_MEM_OFFLINE); if (ret) { reason =3D "failure to isolate range"; goto failed_removal_pcplists_disabled; @@ -2069,7 +2068,8 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, goto failed_removal_isolated; } =20 - ret =3D test_pages_isolated(start_pfn, end_pfn, MEMORY_OFFLINE); + ret =3D test_pages_isolated(start_pfn, end_pfn, + PB_ISOLATE_MODE_MEM_OFFLINE); =20 } while (ret); =20 diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ccb21af002b0..0867e2b2e187 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6695,11 +6695,12 @@ static void alloc_contig_dump_pages(struct list_hea= d *page_list) =20 /* * [start, end) must belong to a single zone. - * @migratetype: using migratetype to filter the type of migration in + * @alloc_flags: using acr_flags_t to filter the type of migration in * trace_mm_alloc_contig_migrate_range_info. */ static int __alloc_contig_migrate_range(struct compact_control *cc, - unsigned long start, unsigned long end, int migratetype) + unsigned long start, unsigned long end, + acr_flags_t alloc_flags) { /* This function is based on compact_zone() from compaction.c. */ unsigned int nr_reclaimed; @@ -6771,7 +6772,7 @@ static int __alloc_contig_migrate_range(struct compac= t_control *cc, putback_movable_pages(&cc->migratepages); } =20 - trace_mm_alloc_contig_migrate_range_info(start, end, migratetype, + trace_mm_alloc_contig_migrate_range_info(start, end, alloc_flags, total_migrated, total_reclaimed, total_mapped); @@ -6842,10 +6843,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_= mask, gfp_t *gfp_cc_mask) * alloc_contig_range() -- tries to allocate given range of pages * @start: start PFN to allocate * @end: one-past-the-last PFN to allocate - * @migratetype: migratetype of the underlying pageblocks (either - * #MIGRATE_MOVABLE or #MIGRATE_CMA). All pageblocks - * in range must have the same migratetype and it must - * be either of the two. + * @alloc_flags: allocation information * @gfp_mask: GFP mask. Node/zone/placement hints are ignored; only some * action and reclaim modifiers are supported. Reclaim modifiers * control allocation behavior during compaction/migration/reclaim. @@ -6862,7 +6860,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_m= ask, gfp_t *gfp_cc_mask) * need to be freed with free_contig_range(). */ int alloc_contig_range_noprof(unsigned long start, unsigned long end, - unsigned migratetype, gfp_t gfp_mask) + acr_flags_t alloc_flags, gfp_t gfp_mask) { unsigned long outer_start, outer_end; int ret =3D 0; @@ -6877,6 +6875,9 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, .alloc_contig =3D true, }; INIT_LIST_HEAD(&cc.migratepages); + enum pb_isolate_mode mode =3D (alloc_flags & ACR_CMA) ? + PB_ISOLATE_MODE_CMA_ALLOC : + PB_ISOLATE_MODE_OTHER; =20 gfp_mask =3D current_gfp_context(gfp_mask); if (__alloc_contig_verify_gfp_mask(gfp_mask, (gfp_t *)&cc.gfp_mask)) @@ -6903,7 +6904,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, * put back to page allocator so that buddy can use them. */ =20 - ret =3D start_isolate_page_range(start, end, migratetype, 0); + ret =3D start_isolate_page_range(start, end, mode); if (ret) goto done; =20 @@ -6919,7 +6920,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, * allocated. So, if we fall through be sure to clear ret so that * -EBUSY is not accidentally used or returned to caller. */ - ret =3D __alloc_contig_migrate_range(&cc, start, end, migratetype); + ret =3D __alloc_contig_migrate_range(&cc, start, end, alloc_flags); if (ret && ret !=3D -EBUSY) goto done; =20 @@ -6953,7 +6954,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, outer_start =3D find_large_buddy(start); =20 /* Make sure the range is really isolated. */ - if (test_pages_isolated(outer_start, end, 0)) { + if (test_pages_isolated(outer_start, end, mode)) { ret =3D -EBUSY; goto done; } @@ -6996,8 +6997,8 @@ static int __alloc_contig_pages(unsigned long start_p= fn, { unsigned long end_pfn =3D start_pfn + nr_pages; =20 - return alloc_contig_range_noprof(start_pfn, end_pfn, MIGRATE_MOVABLE, - gfp_mask); + return alloc_contig_range_noprof(start_pfn, end_pfn, ACR_NONE, + gfp_mask); } =20 static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn, diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 1edfef408faf..ece3bfc56bcd 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -31,7 +31,7 @@ * */ static struct page *has_unmovable_pages(unsigned long start_pfn, unsigned = long end_pfn, - int migratetype, int flags) + enum pb_isolate_mode mode) { struct page *page =3D pfn_to_page(start_pfn); struct zone *zone =3D page_zone(page); @@ -46,7 +46,7 @@ static struct page *has_unmovable_pages(unsigned long sta= rt_pfn, unsigned long e * isolate CMA pageblocks even when they are not movable in fact * so consider them movable here. */ - if (is_migrate_cma(migratetype)) + if (mode =3D=3D PB_ISOLATE_MODE_CMA_ALLOC) return NULL; =20 return page; @@ -117,7 +117,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * The HWPoisoned page may be not in buddy system, and * page_count() is not 0. */ - if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) + if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && PageHWPoison(page)) continue; =20 /* @@ -130,7 +130,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * move these pages that still have a reference count > 0. * (false negatives in this function only) */ - if ((flags & MEMORY_OFFLINE) && PageOffline(page)) + if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && PageOffline(page)) continue; =20 if (__PageMovable(page) || PageLRU(page)) @@ -151,7 +151,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * present in [start_pfn, end_pfn). The pageblock must intersect with * [start_pfn, end_pfn). */ -static int set_migratetype_isolate(struct page *page, int migratetype, int= isol_flags, +static int set_migratetype_isolate(struct page *page, enum pb_isolate_mode= mode, unsigned long start_pfn, unsigned long end_pfn) { struct zone *zone =3D page_zone(page); @@ -186,7 +186,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ end_pfn); =20 unmovable =3D has_unmovable_pages(check_unmovable_start, check_unmovable_= end, - migratetype, isol_flags); + mode); if (!unmovable) { if (!pageblock_isolate_and_move_free_pages(zone, page)) { spin_unlock_irqrestore(&zone->lock, flags); @@ -198,7 +198,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ } =20 spin_unlock_irqrestore(&zone->lock, flags); - if (isol_flags & REPORT_FAILURE) { + if (mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) { /* * printk() with zone->lock held will likely trigger a * lockdep splat, so defer it here. @@ -292,11 +292,10 @@ __first_valid_page(unsigned long pfn, unsigned long n= r_pages) * isolate_single_pageblock() -- tries to isolate a pageblock that might be * within a free or in-use page. * @boundary_pfn: pageblock-aligned pfn that a page might cross - * @flags: isolation flags + * @mode: isolation mode * @isolate_before: isolate the pageblock before the boundary_pfn * @skip_isolation: the flag to skip the pageblock isolation in second * isolate_single_pageblock() - * @migratetype: migrate type to set in error recovery. * * Free and in-use pages can be as big as MAX_PAGE_ORDER and contain more = than one * pageblock. When not all pageblocks within a page are isolated at the sa= me @@ -311,8 +310,9 @@ __first_valid_page(unsigned long pfn, unsigned long nr_= pages) * either. The function handles this by splitting the free page or migrati= ng * the in-use page then splitting the free page. */ -static int isolate_single_pageblock(unsigned long boundary_pfn, int flags, - bool isolate_before, bool skip_isolation, int migratetype) +static int isolate_single_pageblock(unsigned long boundary_pfn, + enum pb_isolate_mode mode, bool isolate_before, + bool skip_isolation) { unsigned long start_pfn; unsigned long isolate_pageblock; @@ -338,12 +338,11 @@ static int isolate_single_pageblock(unsigned long bou= ndary_pfn, int flags, zone->zone_start_pfn); =20 if (skip_isolation) { - int mt __maybe_unused =3D get_pageblock_migratetype(pfn_to_page(isolate_= pageblock)); - - VM_BUG_ON(!is_migrate_isolate(mt)); + VM_BUG_ON(!get_pageblock_isolate(pfn_to_page(isolate_pageblock))); } else { - ret =3D set_migratetype_isolate(pfn_to_page(isolate_pageblock), migratet= ype, - flags, isolate_pageblock, isolate_pageblock + pageblock_nr_pages); + ret =3D set_migratetype_isolate(pfn_to_page(isolate_pageblock), + mode, isolate_pageblock, + isolate_pageblock + pageblock_nr_pages); =20 if (ret) return ret; @@ -441,14 +440,7 @@ static int isolate_single_pageblock(unsigned long boun= dary_pfn, int flags, * start_isolate_page_range() - mark page range MIGRATE_ISOLATE * @start_pfn: The first PFN of the range to be isolated. * @end_pfn: The last PFN of the range to be isolated. - * @migratetype: Migrate type to set in error recovery. - * @flags: The following flags are allowed (they can be combined in - * a bit mask) - * MEMORY_OFFLINE - isolate to offline (!allocate) memory - * e.g., skip over PageHWPoison() pages - * and PageOffline() pages. - * REPORT_FAILURE - report details about the failure to - * isolate the range + * @mode: isolation mode * * Making page-allocation-type to be MIGRATE_ISOLATE means free pages in * the range will never be allocated. Any free pages and pages freed in the @@ -481,7 +473,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, * Return: 0 on success and -EBUSY if any part of range cannot be isolated. */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype, int flags) + enum pb_isolate_mode mode) { unsigned long pfn; struct page *page; @@ -492,8 +484,8 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, bool skip_isolation =3D false; =20 /* isolate [isolate_start, isolate_start + pageblock_nr_pages) pageblock = */ - ret =3D isolate_single_pageblock(isolate_start, flags, false, - skip_isolation, migratetype); + ret =3D isolate_single_pageblock(isolate_start, mode, false, + skip_isolation); if (ret) return ret; =20 @@ -501,8 +493,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, skip_isolation =3D true; =20 /* isolate [isolate_end - pageblock_nr_pages, isolate_end) pageblock */ - ret =3D isolate_single_pageblock(isolate_end, flags, true, - skip_isolation, migratetype); + ret =3D isolate_single_pageblock(isolate_end, mode, true, skip_isolation); if (ret) { unset_migratetype_isolate(pfn_to_page(isolate_start)); return ret; @@ -513,8 +504,8 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, pfn < isolate_end - pageblock_nr_pages; pfn +=3D pageblock_nr_pages) { page =3D __first_valid_page(pfn, pageblock_nr_pages); - if (page && set_migratetype_isolate(page, migratetype, flags, - start_pfn, end_pfn)) { + if (page && set_migratetype_isolate(page, mode, start_pfn, + end_pfn)) { undo_isolate_page_range(isolate_start, pfn); unset_migratetype_isolate( pfn_to_page(isolate_end - pageblock_nr_pages)); @@ -556,7 +547,7 @@ void undo_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn) */ static unsigned long __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn, - int flags) + enum pb_isolate_mode mode) { struct page *page; =20 @@ -569,11 +560,12 @@ __test_page_isolated_in_pageblock(unsigned long pfn, = unsigned long end_pfn, * simple way to verify that as VM_BUG_ON(), though. */ pfn +=3D 1 << buddy_order(page); - else if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) + else if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && + PageHWPoison(page)) /* A HWPoisoned page cannot be also PageBuddy */ pfn++; - else if ((flags & MEMORY_OFFLINE) && PageOffline(page) && - !page_count(page)) + else if ((mode =3D=3D PB_ISOLATE_MODE_MEM_OFFLINE) && + PageOffline(page) && !page_count(page)) /* * The responsible driver agreed to skip PageOffline() * pages when offlining memory by dropping its @@ -591,11 +583,11 @@ __test_page_isolated_in_pageblock(unsigned long pfn, = unsigned long end_pfn, * test_pages_isolated - check if pageblocks in range are isolated * @start_pfn: The first PFN of the isolated range * @end_pfn: The first PFN *after* the isolated range - * @isol_flags: Testing mode flags + * @mode: Testing mode * * This tests if all in the specified range are free. * - * If %MEMORY_OFFLINE is specified in @flags, it will consider + * If %PB_ISOLATE_MODE_MEM_OFFLINE specified in @mode, it will consider * poisoned and offlined pages free as well. * * Caller must ensure the requested range doesn't span zones. @@ -603,7 +595,7 @@ __test_page_isolated_in_pageblock(unsigned long pfn, un= signed long end_pfn, * Returns 0 if true, -EBUSY if one or more pages are in use. */ int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, - int isol_flags) + enum pb_isolate_mode mode) { unsigned long pfn, flags; struct page *page; @@ -639,7 +631,7 @@ int test_pages_isolated(unsigned long start_pfn, unsign= ed long end_pfn, /* Check all pages are free or marked as ISOLATED */ zone =3D page_zone(page); spin_lock_irqsave(&zone->lock, flags); - pfn =3D __test_page_isolated_in_pageblock(start_pfn, end_pfn, isol_flags); + pfn =3D __test_page_isolated_in_pageblock(start_pfn, end_pfn, mode); spin_unlock_irqrestore(&zone->lock, flags); =20 ret =3D pfn < end_pfn ? -EBUSY : 0; --=20 2.47.2