From nobody Fri Dec 19 04:54:03 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2043.outbound.protection.outlook.com [40.107.94.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 448DF266EFE for ; Fri, 14 Feb 2025 15:42:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739547750; cv=fail; b=XUivfRk8cXMbcknhnhTbmFpNWaz3VvP/ocIiKdvLaIyQDe43puHQIQoPGVgW7k0tulkrHnIS/JTUExxCdlVipluIiAXgkVbd/zInzlRraal1IOPaAHNFatCDHuzCtS45LH2iYKxaMejJTxB8TUiC1h2bscdGPo/i98AL7eIIpGQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739547750; c=relaxed/simple; bh=t+ojR14eNEZc6uxP0HjYSPTvSlOZcN+q/vemDcmLcRY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=uws6RPIGjPEBVQsgkuAWvtf0cPvYmYgKhabgsBbeWwHQXJU6GRAdpsyOm0Jg0psd7dFxGoi17xOi7nSbCBlmTP61d2EBXzT/jFnaBnoZvVkjYktJtIn3MA78Dge/hLD54RwTymTRFRpW7ZQ97jI/QurFkbeIbpSG557EWv2Tw/0= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ndcCe9LN; arc=fail smtp.client-ip=40.107.94.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ndcCe9LN" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JPHg5r7ppzyYSV2+BCHWue17bD8SpU+9hKde0Mpvzd2hQ4Gcgiqlshh7r+x+/sZ6/cbw7661/j0jwek6f/+lGvEOPKvR9XmLwC9III1DeZU4R/8SA5ZA/XGnhEq7AoEj0vghcXRlH6RAuJFm8fkrRhzPTS6JNtHYmr2KdK0g12hrX1bW/5R8iUhza66p6MwRwGMsO9OTurjSZ1+WB2iy1WdilS3yYFxOs3a/fpXhp/Gual5bZXLAq2NQiT6LNKoxdMrjtONKQPBbThdrIIoUqLA0dVgypK66u3euhJ7/ATxouuCc4CuwzoDIVv4KldDoZLXEN4niYHzt+1k6nFYF8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AebmBmJkIAuQ7h16HolSR4JegMPa7EHSV1KqsaK63Ow=; b=oQeKuA88qfEU3vyO/V7GMhWE1JZRGfEM8qvwh+kBGYW/VJ4Mc0RpPYxoLuQwW04ReFDx1r9TisuxKwvlfJxlOFqYXxUR+SKTSt8fIe2FR0nzPTg2IRvikILA5ONGGR0BTXpJj6rgP19UKMnS1gmjGrfgfP/iRyb/feFvLmHiRgyYnU/mbAF5ZXKUMW3RK3e7mnWdD1XCmuRqHxR5xUCSsvb0PTy/nbfd32xVZV7U+k5KCzcTdBEFvxKoFI4zA08nFSs0fJMiNwh4hnSvnWNz8jcEeRuueRMRr8396afBsCkk1wMX3OqU7e+RCMoBHTPBSbbW4KnQKsEIb3+Lwc4ueQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AebmBmJkIAuQ7h16HolSR4JegMPa7EHSV1KqsaK63Ow=; b=ndcCe9LNbuiXBcEVO7xZfNKtkrXt4mzQP3JP1rfh8uOlitgLwpFb6E4KiiljeiMsspM7jtG9a8Vo0HA4c9yJ41oaZPgR7WPiSVZw6rKByhoFlkpY5EKj1C8UVT1qHh82g5XzR3w2jH2M0UiLtwR1lohbU1qm2RwSr+hdW+yjgEjRgzSwaR3M5nU86RTIkIcOOg5mRZZVOsXM9cw2tVytWBKbBWxchHBtqvQNV9z5p7yijYGJV5tMD4j7BnBVJaXFyvKF6npWhnfMudb377dPO4APcFCPnpc+lGgW4YU1hpP5W8clVWcRe/EVL7yrLGZUjILrgjbFKpW4qfZS5HgzJQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY8PR12MB8339.namprd12.prod.outlook.com (2603:10b6:930:7e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.14; Fri, 14 Feb 2025 15:42:24 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8445.016; Fri, 14 Feb 2025 15:42:24 +0000 From: Zi Yan To: linux-mm@kvack.org, David Hildenbrand Cc: Oscar Salvador , Vlastimil Babka , Johannes Weiner , Baolin Wang , "Kirill A. Shutemov" , Mel Gorman , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 1/4] mm/page_isolation: make page isolation a standalone bit. Date: Fri, 14 Feb 2025 10:42:12 -0500 Message-ID: <20250214154215.717537-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250214154215.717537-1-ziy@nvidia.com> References: <20250214154215.717537-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0PR04CA0009.namprd04.prod.outlook.com (2603:10b6:208:52d::30) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY8PR12MB8339:EE_ X-MS-Office365-Filtering-Correlation-Id: f2ba1d9f-37c0-476f-09b7-08dd4d0e2d56 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?lxmdowwLmZYoXQfU1A24v2y++AMfGflHKUgGPAM9v4ZygP7eLkowhmD/YW12?= =?us-ascii?Q?YMHDA8pW4zh0GsNWxQ8SlMiRBR2LgiIruSNPEPCunDsOgNNNdUE2D2o89ab4?= =?us-ascii?Q?G5ZVKDQ3tG0bLQTBUaayFvYR9klgfcV9BaCmS8z++AYavghnKHWVvdBOztXL?= =?us-ascii?Q?7kwFepXhf5t5PBHtM5d5rQKy+FJoOY+YYxRA/ObJ8Gw8xQpxqJSad4yx5oWj?= =?us-ascii?Q?kW0sQoK4bHnnv/ihF4KbCIlEq54dQg6yjLbLJtfvQhJNc6gXGhkYXxuAMJmH?= =?us-ascii?Q?5Faa5rsHEOhdM/LXTLIVxZcKneb/4FPy0Z7i0Ac8cSRg5go0Wh82+UevSesd?= =?us-ascii?Q?hKrMhZdUGcH1H3XzIr6rypHFaCcRGMMaSaC8yx6XiGzyaE6TET69mmc0xIAk?= =?us-ascii?Q?rnGtrJCxcb6TclgkioQ5Dd4mj9KJlr2xWKpdPpDfdELurA2Y1m8eplMYo2M5?= =?us-ascii?Q?il1i39taUlTZWZW/NZEU/svFJZbI31v1+Vwq9MF81+jI4z1lZ90XUxxUq8Zm?= =?us-ascii?Q?XwjyhWPfWq1fen4vIWgIjNJ+Nc5ggI96gBZGN4k2EiAruBFaebGIlbrcIsr8?= =?us-ascii?Q?dHl7KHTChLfKicWH5ZPYt+aVEL8GdvY8zQ+R7U9HA3JLWl7HPj1Cl0q0uQqN?= =?us-ascii?Q?RY6eStQ7TVPn9flVsB4GrPRVZUEzymV0tE+lI8GwXfbG2V+S5mWIwkSpoNDv?= =?us-ascii?Q?rJfgij723XBt0MibxR9fiXYkGtyDrdeq94B+AHsexsFWZLGG2/IeSPixCaOq?= =?us-ascii?Q?LZHSaJmrJNfuHmq5uWVMYj8z2zTNKUBEgbV/INaI6edHDF3CCQ98dWn06fHM?= =?us-ascii?Q?RRdCrNpZozvbIIZLNofP1WSrzyMKoxWirWd004fywG7hCpJjzvA9VTFxPIZj?= =?us-ascii?Q?lFrXDH3jqKOTXtMCh7/7suVCD2srisOSfV9S4VijU9488PYZqErDbT0QNwiA?= =?us-ascii?Q?F6tDj25XYFe22jzXpEaVGsP2B6f6YX5lDFBQNj6lB5idPI8hhO7/Xosub/hn?= =?us-ascii?Q?T1mO91yUOwnfrGGeGAb24NuuPWRPdS/zIDOJ9S4K/oU3UBuT4m39+igy0vFT?= =?us-ascii?Q?jSdbaucgUm+oCsKtT6zm7lVT1un3+y9TLPDT3mGktu/DvF1DlTwatwlNruFW?= =?us-ascii?Q?mzricjdZBEfem0y8AmZ1NqmIazkzS6GxeuoIHBjr/cWA1/SWDrYpaAFpJmkL?= =?us-ascii?Q?dl77nZK2j4a8f0Kz2zyoKSt94BiRNsUDaeC2TfMXqpo1Ocr8YM51xeIwgHvk?= =?us-ascii?Q?dTrYsG7i7IHCMTevqTApqLkOMH8Oxla7bG0+Vc98qLSCLwVu/TUIr1UrHtrA?= =?us-ascii?Q?IqCRBxSjAsYTiB8pcvsda5z92MhzghqCnASqKtiGH0v3VAByvEea1nq73B7/?= =?us-ascii?Q?TzUwwOnl1DmLnmgVWLyLI9viTKEq?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8UW9o0egujEIDx9Hat6VaHAeAqpnhtDpooDZCaodjDnI2E3ZAIHQrDuvIMXj?= =?us-ascii?Q?vgUzDi6kIJoaFSC/5gAUsoTjE8G6U6ay33sJM/w0BxyCK8+2t9YRDgmFOacX?= =?us-ascii?Q?A5ZT8vmujNYWCf1d3Ey1SaDrvjCKsntT/gLJVFla18VqUoFQHn5emHjEVqeS?= =?us-ascii?Q?icpXcTMfTrmfT41rnceGRW7FeA5N091+qt+i6/vS2tfyjrKQ4Nl3pUhDvDxF?= =?us-ascii?Q?f2JRHFBgSpolnU+N0iOAMnppSvUgzdDRrtWmrdSWyvrXB/mkNtjntz+OC/eA?= =?us-ascii?Q?u+d3+jmJUyJaeXr64uxA9yGK32GPg8WaCC10deMthawcwLoXpPGXaBqwURUT?= =?us-ascii?Q?pT5Kbz0kfwsBx/bNBXWLVWaK5gcc9/ozLZtrjIno/ABS+l4MQg2T/atwRmPb?= =?us-ascii?Q?hiCETU4f45WUeQKr+kZKdZj4SZfp0wotfKuhthavBliXGmudW9ZGKjme86fF?= =?us-ascii?Q?fqU2Rxau4N3CThtma/J4SqcTCtcFHct/jxExGt4XnwGLtLvqpCHlelZy/+sC?= =?us-ascii?Q?YBJfmohC0HornGHjW6XKjLR01DffN4iLYkfWl/SzUoxFfaDY7Rgdlamh+BxG?= =?us-ascii?Q?RJF7CcZyz/Oaa9gnGVjrxnVd3hH7lquGRpSJ7KiwdnyJYdfgyvi4in1l1tk7?= =?us-ascii?Q?ab9t+U+wMiZGQFjpWVgNQQtnAVqkG063Oe6JAgkG12z29zKyaDJOFVf+jUQ1?= =?us-ascii?Q?ocJWQ5pTgVxMJMiAHudM3JPHXcktYR0W88+Bbk5IK4QtQihH92qIKCCuY6mU?= =?us-ascii?Q?RuQBFiDdjcKThhGLZudWa3VgKDS3+2vq3sk3gU4TftK1PcEvSl3QRZ09RrVJ?= =?us-ascii?Q?GW6eVeHYyuWiRpxl9MTiRi587ahX9UKcEYZUf1q88ZQOxQTMuqEjomUJKQjO?= =?us-ascii?Q?35ykwgKQXmCOkpg62SOqDtQuW1oEJJ3FB4VL+B3kyUmJ7YcxO55hjNp/mHde?= =?us-ascii?Q?IzbMGQWoipgO9NjDnUWXnrxxWAmNIshw4T9/AJ8/yTql/XWVT1c2Aq5WWuMu?= =?us-ascii?Q?zY8+6AmR7znT+OP47DviGjpPxTxac8tduq179N7EJL3q2zr7+9mmSGMu9eGQ?= =?us-ascii?Q?DWvLtTP0GBmUlmCIIXF9wrdL2GWGnUMtbhacXg+a1KGOiJONSt6Jml+fzXky?= =?us-ascii?Q?wwO/ONOTbu6NSoKP0C0vcHM2ye/bh+23fcd98EWSmrcJcjE2eHSBGjNN7vdz?= =?us-ascii?Q?bWyiYFTYvXP4YyLURBH7gWWjq6B+TKkwxwOYSdBZ7n22sVm+88HguY/97AtT?= =?us-ascii?Q?Qu/l8OHgkWgKd5bKIZRPb4a5WVi4rwaarva7t/kdyljmWrdNi0iR0DzAVgMh?= =?us-ascii?Q?YXX4xnT1y63STu8jX3nlbORnAlJbqO5wgwuYTZFBPA5UN4lcxCXhKDTQEbI0?= =?us-ascii?Q?imIvsxiMp81KCOmYvr+pGOIe7RqLwKX8dvY60WWWZeN/X/+vOrfgnYA+3Hu6?= =?us-ascii?Q?2v8gyS63PdvdOeyhVO9kuaN200bYXEYDCiZFY97DNu4uPRDvAb5DJFUf59Ik?= =?us-ascii?Q?Se+QrPrsSpugTXr9rNaMsMefm8PTQDBwG9cieMKIqSnpjw41lOIV9dL0fnA+?= =?us-ascii?Q?rXA53ttkAwWuqqVRBj8Jb4T4l7LJssyvjXUfkq14?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f2ba1d9f-37c0-476f-09b7-08dd4d0e2d56 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:42:24.3016 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9BQljV9wq+4Fgs4DmRO1GFk0C538OvTrgMr2IS2dZ4JqZoJEyyOoHzLamhDfD2QK X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8339 Content-Type: text/plain; charset="utf-8" During page isolation, the original migratetype is overwritten, since MIGRATE_* are enums. Change MIGRATE_ISOLATE to be a standalone bit like PB_migrate_skip. pageblock bits needs to be word aligned, so expand the number of pageblock bits from 4 to 8 and make migrate isolate bit 7. Signed-off-by: Zi Yan --- include/linux/mmzone.h | 18 +++++++++++++----- include/linux/page-isolation.h | 2 +- include/linux/pageblock-flags.h | 33 ++++++++++++++++++++++++++++++++- mm/page_alloc.c | 21 +++++++++++++++++++-- 4 files changed, 65 insertions(+), 9 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 8aecbbb0b685..3c7d3f22ccb2 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -106,14 +106,22 @@ static inline bool migratetype_is_mergeable(int mt) =20 extern int page_group_by_mobility_disabled; =20 -#define MIGRATETYPE_MASK ((1UL << PB_migratetype_bits) - 1) +#ifdef CONFIG_MEMORY_ISOLATION +#define MIGRATETYPE_NO_ISO_MASK (BIT(PB_migratetype_bits) - 1) +#define MIGRATETYPE_MASK (MIGRATETYPE_NO_ISO_MASK | PB_migrate_isolate_bit) +#else +#define MIGRATETYPE_NO_ISO_MASK (BIT(PB_migratetype_bits) - 1) +#define MIGRATETYPE_MASK (BIT(PB_migratetype_bits) - 1) +#endif =20 -#define get_pageblock_migratetype(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), MIGRATETYPE_MASK) +#define get_pageblock_migratetype(page) \ + get_pfnblock_flags_mask(page, page_to_pfn(page), \ + MIGRATETYPE_MASK) =20 -#define folio_migratetype(folio) \ - get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ +#define folio_migratetype(folio) \ + get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ MIGRATETYPE_MASK) + struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 898bb788243b..51797dc39cbc 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -5,7 +5,7 @@ #ifdef CONFIG_MEMORY_ISOLATION static inline bool is_migrate_isolate_page(struct page *page) { - return get_pageblock_migratetype(page) =3D=3D MIGRATE_ISOLATE; + return get_pageblock_isolate(page); } static inline bool is_migrate_isolate(int migratetype) { diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flag= s.h index fc6b9c87cb0a..d6fe17bf0c9b 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -20,7 +20,10 @@ enum pageblock_bits { PB_migrate_end =3D PB_migrate + PB_migratetype_bits - 1, /* 3 bits required for migrate types */ PB_migrate_skip,/* If set the block is skipped by compaction */ - +#ifdef CONFIG_MEMORY_ISOLATION + PB_migrate_isolate =3D 7, /* If set the block is isolated */ + /* set it to 7 to make pageblock bit word aligned */ +#endif /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. @@ -28,6 +31,10 @@ enum pageblock_bits { NR_PAGEBLOCK_BITS }; =20 +#ifdef CONFIG_MEMORY_ISOLATION +#define PB_migrate_isolate_bit BIT(PB_migrate_isolate) +#endif + #if defined(CONFIG_HUGETLB_PAGE) =20 #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -99,4 +106,28 @@ static inline void set_pageblock_skip(struct page *page) } #endif /* CONFIG_COMPACTION */ =20 +#ifdef CONFIG_MEMORY_ISOLATION +#define get_pageblock_isolate(page) \ + get_pfnblock_flags_mask(page, page_to_pfn(page), \ + (1 << (PB_migrate_isolate))) +#define clear_pageblock_isolate(page) \ + set_pfnblock_flags_mask(page, 0, page_to_pfn(page), \ + (1 << PB_migrate_isolate)) +#define set_pageblock_isolate(page) \ + set_pfnblock_flags_mask(page, (1 << PB_migrate_isolate), \ + page_to_pfn(page), \ + (1 << PB_migrate_isolate)) +#else +static inline bool get_pageblock_isolate(struct page *page) +{ + return false; +} +static inline void clear_pageblock_isolate(struct page *page) +{ +} +static inline void set_pageblock_isolate(struct page *page) +{ +} +#endif /* CONFIG_MEMORY_ISOLATION */ + #endif /* PAGEBLOCK_FLAGS_H */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 16dfcf7ade74..f17f4acc38c6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -362,6 +362,7 @@ unsigned long get_pfnblock_flags_mask(const struct page= *page, unsigned long *bitmap; unsigned long bitidx, word_bitidx; unsigned long word; + unsigned long flags; =20 bitmap =3D get_pageblock_bitmap(page, pfn); bitidx =3D pfn_to_bitidx(page, pfn); @@ -373,7 +374,13 @@ unsigned long get_pfnblock_flags_mask(const struct pag= e *page, * racy, are not corrupted. */ word =3D READ_ONCE(bitmap[word_bitidx]); - return (word >> bitidx) & mask; + flags =3D (word >> bitidx) & mask; + +#ifdef CONFIG_MEMORY_ISOLATION + if (flags & PB_migrate_isolate_bit) + return MIGRATE_ISOLATE; +#endif + return flags; } =20 static __always_inline int get_pfnblock_migratetype(const struct page *pag= e, @@ -397,8 +404,18 @@ void set_pfnblock_flags_mask(struct page *page, unsign= ed long flags, unsigned long bitidx, word_bitidx; unsigned long word; =20 +#ifdef CONFIG_MEMORY_ISOLATION + BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 8); + /* keep other migratetype bits if MIGRATE_ISOLATE is set */ + if (flags =3D=3D MIGRATE_ISOLATE) { + mask &=3D ~((1UL << PB_migratetype_bits) - 1); + flags =3D PB_migrate_isolate_bit; + } +#else BUILD_BUG_ON(NR_PAGEBLOCK_BITS !=3D 4); - BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); +#endif + /* extra one for MIGRATE_ISOLATE */ + BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits) + 1); =20 bitmap =3D get_pageblock_bitmap(page, pfn); bitidx =3D pfn_to_bitidx(page, pfn); --=20 2.47.2 From nobody Fri Dec 19 04:54:03 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2043.outbound.protection.outlook.com [40.107.94.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08800267388 for ; Fri, 14 Feb 2025 15:42:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739547755; cv=fail; b=msH86jSAWSo16wVP8EkwcgFEdPbyb7DNxT/sXvOHKm+Qh2PDdV3cBKFEqNHDhQ9EE0QidTwZ8fSQkG4Xmja4aaSKyCCphcp7F/a+bBbY61MZwfxMQIIXNXNuYwiNmQWt0yYr1LCHcTqe7HN3AvMBWx3gLkstFPP6dN1QbyEv4WA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739547755; c=relaxed/simple; bh=i0n8816nYOn1RWCrOl2agolwcEC5FmBLHq0MwRkyGjU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=KiqdUmzw7v8qC/V5SxZhnWETTIiS8pTPS845bP+AWbGP+kDQtld9BMIqZfbZC084fG7X7Toc95+pFcfqnT8m/+eMG/TFYsqze1vjs4yNhzvTxgIOrmCdCT8q+avx5dC/4jk7TnB7ggC8AlyCMVA6euAxuO9y0XF6X57ywFaDpME= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=D3daoQ5+; arc=fail smtp.client-ip=40.107.94.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="D3daoQ5+" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JcKR+gSlOFJs2dCdD8is4SJYkklmNF4pRtJkH/akNSY0uGlSOW21pDp4OFryFEMzWrDeLGfERRPNHHgfnhNdLHHG9PXj9FewY7gUSSzpAnboMjGUd80bbU9UFrLqk4RNCbSDQHfgRGVrrcrG4BO+zEmo2/ZPGCf/dn13WYJJZCbrlCPX0aS3X+AoOYGKDGC+3QUG3EG3W/Couj36UFuykx+80OE3kfzAQTdCplWDSqmwvqt/farX0bCggfant9FEu1VI0Yx1tyDThPjcCu3MPppyEb+q2z8bMfpzpsciANtyT3uoxW1qkLEwlzMiLAh6LHEerEKSIPDWXT9AzczQzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/uEVNm36daEAQedWM1fmv32brDf7DLmvj6nt8IZy6Dw=; b=lrJYJ9Gwxm6Ri1Z22QGQS+ZV7EBpz4cG9AQyxrp+u7QQC8oZMazWkwEUu1Z3QkFEkZ0r+e7ivXv3cV3Kz5XhIiaAIpDoDhCOvJfVf5uFCBH4JQAioeHLRJmw+Jj722fHNQy0VWCFTHq+AaPsn4J7XjXodE+MpB+TXcDJ+pD1hxcjg/DvpsAXUTx7+MehefAvBlo03MENImluhObIe9wweFH0izqpwvJLORuIFVTD431vY21R7hcVG1xiXY/0sBvPtJZcU2xm42mUBTYT2NfvCq5gQ/6LgNvJlfiXQzthBmXjwDFc9prX8junu8xpcI1QID5rhotfzhuaiJJ+u3AZFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/uEVNm36daEAQedWM1fmv32brDf7DLmvj6nt8IZy6Dw=; b=D3daoQ5+em1PgiHIu5ibfHx0vgBxPl158qqlRuDSzBE0fKPY89tiyhNZKcwyNHVDSF/fx/fzq5y7uqGqddgEbZ1o9/HwKd0GUL9iWRNvPID7hAfObv6qtTpw5XOwlvTep/h9fA2lAkrPqEa911O6pM1WUhh1EBal3XNagwZlOyjGSOs1gBzgPk268L7HkYJyq9LqnwKY65nsuDBVy3lY/O5E6WwUW/jCoIsoaWxH65n0l6CIsDHU/PxHYZ2BbjrbxAP5NoWa3nypxusOvHEuH2yhPq7b/TYVBblDA0zYDr/RxJBlPRQOaNhJVUcAqKqN7s9bzVhxanmzO6Y5+caQCQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY8PR12MB8339.namprd12.prod.outlook.com (2603:10b6:930:7e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.14; Fri, 14 Feb 2025 15:42:25 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8445.016; Fri, 14 Feb 2025 15:42:25 +0000 From: Zi Yan To: linux-mm@kvack.org, David Hildenbrand Cc: Oscar Salvador , Vlastimil Babka , Johannes Weiner , Baolin Wang , "Kirill A. Shutemov" , Mel Gorman , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 2/4] mm/page_isolation: remove migratetype from move_freepages_block_isolate() Date: Fri, 14 Feb 2025 10:42:13 -0500 Message-ID: <20250214154215.717537-3-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250214154215.717537-1-ziy@nvidia.com> References: <20250214154215.717537-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: MN0PR04CA0005.namprd04.prod.outlook.com (2603:10b6:208:52d::9) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY8PR12MB8339:EE_ X-MS-Office365-Filtering-Correlation-Id: af76635c-b321-4b72-a20b-08dd4d0e2e41 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?00NuQBAPdZNCJgjPosGQu+5+M/ivtm6R/nfQFYVhIjZQSnfpsJ1PFqsg8ROB?= =?us-ascii?Q?G1jw0bdRcQVZSqU0tw9jmkXYTexMeGRizvItU32/j6Y8CbQPgcgBJxMjMf4Q?= =?us-ascii?Q?0p/XLp/M7cUKD95XoMg7/Pp2YqffAcmntHLAz2o19UHy8on4yX2QAjt2rmt3?= =?us-ascii?Q?Z29sEizzd66JmS4ac4267VvTfipkh54sUTA70UG0y9qZ1sXe6Xr5vNGzrfuH?= =?us-ascii?Q?eLPtz8Bsn0zzePB2Lnq14nm07958uTYmQVdOf9Z7s9FEKH5LdXtyT1LMkqjs?= =?us-ascii?Q?MNwEltwGXH5KEcjR0TmXKjU3hMBLuQUKOtjGBnFHXcEQC3cw/Z+Q5+Umn8Bk?= =?us-ascii?Q?aiG22xuQ7jv3PSF4gDD5wQrxehHgfdtmwH+FwGq0hLc85MCV1PVqh//EME6l?= =?us-ascii?Q?HDTMG35daLe4rngvXPMJquZFTSWUIO6MxM9amgbMzPOXx1MteJRDfnoquX5R?= =?us-ascii?Q?NqXWe5GQAuJOlg4RXtDu4QlH2/ceDhhI9yaOgot3uswN6c4D/caAFzp9IwCA?= =?us-ascii?Q?zXyx4+XoUu5FGscqHcmugcaSoMvaOCSNCb8RPbFpB/di3b2Wo0JGduh6hzZc?= =?us-ascii?Q?KhSGKXsPMsEhXDMKIN3jYoyGpwT3apdsS1uC2OzI6QHWOAnjuTWeeYE1IxKy?= =?us-ascii?Q?xr+VB+TQ8NuqrDZJdf+PrVNm15kzoTvCp7JFOY7DYY1Jl2vMShrt5Zvbrp3w?= =?us-ascii?Q?zW2+ksErHUvylqemb855jNUpA7Y6RbE0be5ml+ll7aQqJmXeckoK24ILNMMV?= =?us-ascii?Q?9fNkSojyo6RIXjwfwvAsNfLDzpoCG4+OqWryB5jMtOiuEWVVAfGxljd5M7To?= =?us-ascii?Q?LsTe0t+OgRPh11w/ho11TVh+JMjZQHDJdEYxDXajXVD7YRWc4AcbiCNiVxuV?= =?us-ascii?Q?kPOAHFgcQPWDaSoyl7ljxlfRRQ/Wtgn9eXWx4VDLUTMo3K/L5GQj3bq2emIK?= =?us-ascii?Q?pUT6wUy7uD+4sLJO+VlHFbb9s639hGSScgyqZhAKMHCH/Vb9EyhL5ZnTwhK/?= =?us-ascii?Q?m21nV4XCZnvKvH4QtEPFwOB8PQQ3gxPBAfvxTJhLeaHgRyDJVue51CZnvjIX?= =?us-ascii?Q?HjGzC+vYcmFKWWcb8xaB0C8jG8hkDELdNVB6mFWQWJkTFvny+tYBqRoWVw1r?= =?us-ascii?Q?aGlgzVqWw4YBni6642O+QqSW0H5yVC4V1rRD81R9r/i4vxzuKocGxuNsCjEf?= =?us-ascii?Q?Sc71lSvvHj8P43k9ydtPe5lUN8lf2WRDhZIgF0M6tm2OqJNvFzD0bpc4rAI9?= =?us-ascii?Q?yN6E95EpSTy+DJGAKtYIsngSFPtQ8lfrwZq3ARFImLMyMkxaOERPQbYy4k8o?= =?us-ascii?Q?ex0Fc5HJIMb8ufN2tzaDOg66eOUDXqj3vM1DJ6a6to7+Ru3Hr2f9Vt0RYh3j?= =?us-ascii?Q?/f9KVNvx4I0f/R664UDA6dypFo4L?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/KTufJfOzhmERdxbqVaH09hSAfDLI5zd2qYNxso/H7o8fju0ew/uhruv6v4S?= =?us-ascii?Q?zV/ThRhksDrrmY4az7qA27Df0vn2/FMis099TpOhgnbR8fT4WGAB4zwpk+mL?= =?us-ascii?Q?5Ph1EhuV9QhVStjQci2T4t08D+yUTNAADYzC0ELrHQT3ur6XDH9dm4VfZvM3?= =?us-ascii?Q?VzdcXswOBWOPtTHizN6NOS0cZdfyyiE6ToJOw8OY5OVXoxf+7GrIkPDe8Pnp?= =?us-ascii?Q?9j81OjakWzxfymbbpXdD9CRxTKKU5tL+miYpJLzedukI76YCioqxksCtPI2L?= =?us-ascii?Q?r380peshEeC38rsw6zPp0UHUJ3tr00pGq/Bi/Vb+yilorTYfUCLb01g9KI9o?= =?us-ascii?Q?08f7eu8tuYpfdnzdLJrc/hA2wPPkodYK2N0+4Q/MQlFTYvy2h9RjOf8P3Vky?= =?us-ascii?Q?G9Rq3MhNg+ZpcKgqWig0lJqKzqx8nwQZvYMCO+SUn1CZ5F78tCjHjGkN6xMw?= =?us-ascii?Q?/QqKUkEAulw/Q2PZoVNT7FpLTNfymz5NcjvJo9/9/2PK6iHt/BPs46o140oJ?= =?us-ascii?Q?qn/JDV8JN0RD5A0Q/w9sISiIDonoVUskEa0QTcQcgVctilI8/ZxVNl+D6uCv?= =?us-ascii?Q?jelrQUlwKIkamWaGL57fpEvdLi1PHbzP5RM8k+zcmvNLFFSwWYX8FF33/BFH?= =?us-ascii?Q?YjoruyhCiS2BpuuxqFnHB/J/IPODSjpA8eQtlCd35L3c0BziAbcDV8yhzw7E?= =?us-ascii?Q?ZnZAX4dWvQ0tZVzyKQHbkfjymoE13av9Zz1U64be5tPslXAmkIfZdOpwd7bx?= =?us-ascii?Q?KfTW9VRrlICy05aQTrIMyCYlnDrtAGzByEcqtd2hqYrbdXKEU2lY5fNGXpCm?= =?us-ascii?Q?2pYF0SkV21q+EpYIL3rhPStiD8/8VIeKliKWrgRNZ5ltPiqCO6MYjLzGR9UX?= =?us-ascii?Q?rBjhUyUmFuIusoH2egwuKo3UkfOHlRlVOv8w9TVxIWDgu3IeOO1D/D7TvarW?= =?us-ascii?Q?zas5svddpNZBREm/vhBftEAS0nAzTzGRSDy4yzdNu6HrS88upmHTH8P9gzP9?= =?us-ascii?Q?COgVtWr6YKt1uyOoelOVU75+vqNqlagQgxwv4qH/gBsqAGOlPV630AxAR7rS?= =?us-ascii?Q?xuHjnjsIKF1kAzzwrWwWAzu7830iruI8vKDQBL8Byn4qq67AdUWV0oeQyjyP?= =?us-ascii?Q?NYv/jPykS/zs4ENBJED9Z5t9uRLRPLCKi+W6qswkEyzRpMWGd6M5kTtT22ZP?= =?us-ascii?Q?oGV5TDvoKCeGFrW8ojTFzOG5A/oY2wa/LcO+dJJy2VYSXsf4gjGnEMaYOSi0?= =?us-ascii?Q?K/o5lEvlOCi5CIQNsyXWTTA/myexFTwBSzAu7qMVqXR/GvYfWt2NOVBJYYuh?= =?us-ascii?Q?/G5+TVjIR5ckVFrC7ZE+q5QLF8heRNcljRPebV8hdHFBaUCjB4YeYyG4wZBH?= =?us-ascii?Q?e3IT5LcTebql7VwvgrAftMpGDn0yVm1g7o8k836Fg7CJatts3voxeZRNsb4m?= =?us-ascii?Q?CYpvcsGgc8cbVnfsIoOTC4ZenYg5NnJ9mi88RQBrXt0iabh91ZLwf/B87cTC?= =?us-ascii?Q?+6QvZ/ZL9gmJiuregJT6au397fk8P+x2lf8HobO1C2zLBUAfX9iJIb0UvlCt?= =?us-ascii?Q?HVG/rBqen0lt3EfmWZ3FR1jl1gmZQEaZt0MIym2b?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: af76635c-b321-4b72-a20b-08dd4d0e2e41 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:42:25.8483 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Idb0lS7AmnEH0ThnxLALp2rXBzg8+dmQAnz8ES71r6UPULUzgRLFU+YJx9of+E9d X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8339 Content-Type: text/plain; charset="utf-8" Since migratetype is no longer overwritten during pageblock isolation, moving pageblocks to and from MIGRATE_ISOLATE do not need migratetype. Rename move_freepages_block_isolate() to share common code and add pageblock_isolate_and_move_free_pages() and pageblock_unisolate_and_move_free_pages() to be explicit about the page isolation operations. Signed-off-by: Zi Yan --- include/linux/page-isolation.h | 4 +-- mm/page_alloc.c | 48 +++++++++++++++++++++++++++------- mm/page_isolation.c | 21 +++++++-------- 3 files changed, 51 insertions(+), 22 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 51797dc39cbc..28c56f423e34 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -27,8 +27,8 @@ static inline bool is_migrate_isolate(int migratetype) =20 void set_pageblock_migratetype(struct page *page, int migratetype); =20 -bool move_freepages_block_isolate(struct zone *zone, struct page *page, - int migratetype); +bool pageblock_isolate_and_move_free_pages(struct zone *zone, struct page = *page); +bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page); =20 int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, int migratetype, int flags); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index f17f4acc38c6..9bba5b1c4f1d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1848,10 +1848,10 @@ static unsigned long find_large_buddy(unsigned long= start_pfn) } =20 /** - * move_freepages_block_isolate - move free pages in block for page isolat= ion + * __move_freepages_for_page_isolation - move free pages in block for page= isolation * @zone: the zone * @page: the pageblock page - * @migratetype: migratetype to set on the pageblock + * @isolate_pageblock to isolate the given pageblock or unisolate it * * This is similar to move_freepages_block(), but handles the special * case encountered in page isolation, where the block of interest @@ -1866,10 +1866,15 @@ static unsigned long find_large_buddy(unsigned long= start_pfn) * * Returns %true if pages could be moved, %false otherwise. */ -bool move_freepages_block_isolate(struct zone *zone, struct page *page, - int migratetype) +static bool __move_freepages_for_page_isolation(struct zone *zone, + struct page *page, bool isolate_pageblock) { unsigned long start_pfn, pfn; + int from_mt; + int to_mt; + + if (isolate_pageblock =3D=3D get_pageblock_isolate(page)) + return false; =20 if (!prep_move_freepages_block(zone, page, &start_pfn, NULL, NULL)) return false; @@ -1886,7 +1891,10 @@ bool move_freepages_block_isolate(struct zone *zone,= struct page *page, =20 del_page_from_free_list(buddy, zone, order, get_pfnblock_migratetype(buddy, pfn)); - set_pageblock_migratetype(page, migratetype); + if (isolate_pageblock) + set_pageblock_isolate(page); + else + clear_pageblock_isolate(page); split_large_buddy(zone, buddy, pfn, order, FPI_NONE); return true; } @@ -1897,16 +1905,38 @@ bool move_freepages_block_isolate(struct zone *zone= , struct page *page, =20 del_page_from_free_list(page, zone, order, get_pfnblock_migratetype(page, pfn)); - set_pageblock_migratetype(page, migratetype); + if (isolate_pageblock) + set_pageblock_isolate(page); + else + clear_pageblock_isolate(page); split_large_buddy(zone, page, pfn, order, FPI_NONE); return true; } move: - __move_freepages_block(zone, start_pfn, - get_pfnblock_migratetype(page, start_pfn), - migratetype); + if (isolate_pageblock) { + from_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_MASK); + to_mt =3D MIGRATE_ISOLATE; + } else { + from_mt =3D MIGRATE_ISOLATE; + to_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_MASK); + } + + __move_freepages_block(zone, start_pfn, from_mt, to_mt); return true; } + +bool pageblock_isolate_and_move_free_pages(struct zone *zone, struct page = *page) +{ + return __move_freepages_for_page_isolation(zone, page, true); +} + +bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page) +{ + return __move_freepages_for_page_isolation(zone, page, false); +} + #endif /* CONFIG_MEMORY_ISOLATION */ =20 static void change_pageblock_range(struct page *pageblock_page, diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 8ed53ee00f2a..01d9a4eace7a 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -188,7 +188,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ unmovable =3D has_unmovable_pages(check_unmovable_start, check_unmovable_= end, migratetype, isol_flags); if (!unmovable) { - if (!move_freepages_block_isolate(zone, page, MIGRATE_ISOLATE)) { + if (!pageblock_isolate_and_move_free_pages(zone, page)) { spin_unlock_irqrestore(&zone->lock, flags); return -EBUSY; } @@ -209,7 +209,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ return -EBUSY; } =20 -static void unset_migratetype_isolate(struct page *page, int migratetype) +static void unset_migratetype_isolate(struct page *page) { struct zone *zone; unsigned long flags; @@ -262,10 +262,10 @@ static void unset_migratetype_isolate(struct page *pa= ge, int migratetype) * Isolating this block already succeeded, so this * should not fail on zone boundaries. */ - WARN_ON_ONCE(!move_freepages_block_isolate(zone, page, migratetype)); + WARN_ON_ONCE(!pageblock_unisolate_and_move_free_pages(zone, page)); } else { - set_pageblock_migratetype(page, migratetype); - __putback_isolated_page(page, order, migratetype); + clear_pageblock_isolate(page); + __putback_isolated_page(page, order, get_pageblock_migratetype(page)); } zone->nr_isolate_pageblock--; out: @@ -383,7 +383,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, if (PageBuddy(page)) { int order =3D buddy_order(page); =20 - /* move_freepages_block_isolate() handled this */ + /* pageblock_isolate_and_move_free_pages() handled this */ VM_WARN_ON_ONCE(pfn + (1 << order) > boundary_pfn); =20 pfn +=3D 1UL << order; @@ -433,7 +433,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, failed: /* restore the original migratetype */ if (!skip_isolation) - unset_migratetype_isolate(pfn_to_page(isolate_pageblock), migratetype); + unset_migratetype_isolate(pfn_to_page(isolate_pageblock)); return -EBUSY; } =20 @@ -504,7 +504,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, ret =3D isolate_single_pageblock(isolate_end, flags, true, skip_isolation, migratetype); if (ret) { - unset_migratetype_isolate(pfn_to_page(isolate_start), migratetype); + unset_migratetype_isolate(pfn_to_page(isolate_start)); return ret; } =20 @@ -517,8 +517,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, start_pfn, end_pfn)) { undo_isolate_page_range(isolate_start, pfn, migratetype); unset_migratetype_isolate( - pfn_to_page(isolate_end - pageblock_nr_pages), - migratetype); + pfn_to_page(isolate_end - pageblock_nr_pages)); return -EBUSY; } } @@ -548,7 +547,7 @@ void undo_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, page =3D __first_valid_page(pfn, pageblock_nr_pages); if (!page || !is_migrate_isolate_page(page)) continue; - unset_migratetype_isolate(page, migratetype); + unset_migratetype_isolate(page); } } /* --=20 2.47.2 From nobody Fri Dec 19 04:54:03 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2043.outbound.protection.outlook.com [40.107.94.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B61C267384 for ; Fri, 14 Feb 2025 15:42:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739547758; cv=fail; b=KiOV0mYiQztQsUIAzOoovimvWSoEtkKc6rw0ZWH8OprUDMFwQT3Nm/KlbeRt8km+vmX6yiDB5Dlw9bliL5ZTH7IZUPLUcuL1W8mIOA5H+jkoAIQ2e/cz/5REoshJWM9DPnt+rXZt+dcs+o1Ztnil5k8LJCrCUSOpTl4kjPM7nC4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739547758; c=relaxed/simple; bh=95TAqSnbMUxiycANJHOWr0E+7GMDlT/YmTXghrjOSNk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=jxJmTDJQ7Tyz0pwfEIW/27sskzm60Qs30IJ86dwueebAM+9S/Eysv9LBjDj+6mfDBbrAehamXmwPhZ46ED8xrchaQpPYCMRy0IU8raDBKmT4YkHVUY+sGhBsmylh8sK/rZm+XIh1LjzHsQ8LlVPB6RKrDMyQLVtEwK1lUuyRrFY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=WokOkzZC; arc=fail smtp.client-ip=40.107.94.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="WokOkzZC" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UnWSmqMgt0aUV+4oTTJHrsv/9u6AajD7JjuqhWoB8k/KWUbINnc+xBrsh6mKoWJZFkIsAK7OJ2hIGTboLQ5xcZLvJUppaTk8J6edIIhtzg+zc68OrOBz7XCjSmDZB8+tYBewHh1+o8vn3h6GtzAFjFAyobi/MGnTpXwAxo/fXr+7l3c8/ky9oSgpwILIfC64Dl2pNrhY2na7wUcGHeU0a8SFLtrYof/ypF1K9dwteGsf6MnS4GjWBC2Wpp3JvZy5H5i6uQqe+3XGdzB765idaj9jkxAiNIu9n6rwTUejYjgPexPzgueM/ZoFyx2Ry8hQ8WIp1q0O0PMwrwBmD7f2Tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3sizwI47ZLkUjZJnH3dOI3jZ9Fu2HYfFAi247w0Nv8A=; b=LvCJlymzbnE71AlQYRKW5Orjhpm/xiMmJ5vEX9XmT0nh1wGJ1EYnHvLhqXsHjLqnW2QzRGzud9dquxLYVvXwIIhJZeehd62P/uFXuiCoWV/Xc/jtNm+DJ8aqphIgY7MNUs8WmLCNxVs6vr7tesKMXzQNI93cOKL53LuPtvnLsykpe0TLYsVJN+2ZERzbIFwzPklGzOyWrEbIzJL5ZTP/8IAgq0cEdwJFOVZmNd2Q0kvQgk0PgwoRr8Vk6DvUM2LtJU0naRF2Cp/d3mHF8hQ69OXuaxSJ2t2WRaSTlU6r5atIxH47TgDq9DZrxYHVPKAo89iA1lY8awRYZMycA8v3mw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3sizwI47ZLkUjZJnH3dOI3jZ9Fu2HYfFAi247w0Nv8A=; b=WokOkzZC0h/qwkvM9waNljhg1R8kdCPm+BEnJKuPRwcxYEZ/XSaTiEDHM7eZz2Se9MzyuguvDX06K9mukSnr36Ej8EaXzUex2d8m+S84K0VI6wbAJSEE3w2xyK55OCN+G/ase7cm8pZR7Du5ea7InltgzWdUWNbm5ujXTKe5uKheBNj9ybCG+12NTcIrY6JTJQw+hyZeL0sC9DsszrV9PcVMuJul0s0GEewZHu9kHkXwxIkL2pKmH4FknhBnOoLdZpnon+sTOIdghpC0rSWabkR7Lm2VyuutLjVOxyirTzxl58rVUgbcKOgGm+qw5XfmAI2QqO6Hy4JZ9yhVf2nAyg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY8PR12MB8339.namprd12.prod.outlook.com (2603:10b6:930:7e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.14; Fri, 14 Feb 2025 15:42:27 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8445.016; Fri, 14 Feb 2025 15:42:27 +0000 From: Zi Yan To: linux-mm@kvack.org, David Hildenbrand Cc: Oscar Salvador , Vlastimil Babka , Johannes Weiner , Baolin Wang , "Kirill A. Shutemov" , Mel Gorman , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 3/4] mm/page_isolation: remove migratetype from undo_isolate_page_range() Date: Fri, 14 Feb 2025 10:42:14 -0500 Message-ID: <20250214154215.717537-4-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250214154215.717537-1-ziy@nvidia.com> References: <20250214154215.717537-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN9P220CA0004.NAMP220.PROD.OUTLOOK.COM (2603:10b6:408:13e::9) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY8PR12MB8339:EE_ X-MS-Office365-Filtering-Correlation-Id: 47e1e1be-5606-4a10-e9a2-08dd4d0e2f2e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?lsT9itf52KMQqJXn776MRUdHBzrpBCOVWqkPGmJrV2FuvwQtEZ7chIrtF61C?= =?us-ascii?Q?wsr714Rf63FiT+8WoWo2pC1zdA9pwOyoXpr/Ov3dYBhun9Kcre2/2z2Ghb6H?= =?us-ascii?Q?bWHJBIHJdEmYe+l53U4r77/fITgR31nY1xSu3dbx4lARUU3rmNBell/5Jek9?= =?us-ascii?Q?yCA8lRebZEx2liXo/qm960ly7jS1InGNITXMFtY44haKxFSUtmnWzmxur1RU?= =?us-ascii?Q?55G0cEpPXkIZVU7tQPY1BS0Iz3La51RjkoMp922ICwvozxvIDFM75ifLb0W+?= =?us-ascii?Q?CYpZlMgu9kpMnPrYY6dBbHhGAduJfZ4P+YJeUb8HfDsOCwdU9FPcAgwvg5hj?= =?us-ascii?Q?dK5nsy4qLp2qZWWmRpbwRABdy//dPyoopOHDj7hwZXvDXKg8VykG3fHURuAF?= =?us-ascii?Q?QdYsGOO5Riida+HUrfQP9SO4kFL0mHeY6vgdZZzA9m+GGsWFCjHBxI7yp+8F?= =?us-ascii?Q?WJ+N5m23iWFPULHsWt978UOwRmcpN118RpNNDpkE2/PuQIQg1owD9L60VbXx?= =?us-ascii?Q?NkesEj60WM8Ye8/QF2NZcW6SEcWvF1RgaWACBLwg6WMauB3yn2tXopVZju9l?= =?us-ascii?Q?nGSy+ccxJcgPwiBVtpGHqUdABAqboMUdpaTAsB9Tpss7O39BbcubQ6pw5PC8?= =?us-ascii?Q?FJuGmSi31wxyeq0suc5W7cLYJxikFkSZRO25Em/4PaEiHM5Wdtm7+Cn3PoXN?= =?us-ascii?Q?93ISrQuP3WpwZJHxg/8RCu/0uNNArwZDJQCtDx3PGum/ZpjtBr1exvjgO8V1?= =?us-ascii?Q?RMPn9YQlmHthhybjX7bvAh+rMmGOjwZDaLU0wxyoR5MeSWOIeLxw83x5YnoT?= =?us-ascii?Q?bI2uhqWyiIx9uXM7HLRSUgy8Zu37kmiQ6SYj7xad7z6nJzPzeYJFATidGMqn?= =?us-ascii?Q?T49xyI6jQGRW6YX6+62ciE13/fewdsTowtbHxOFHXhcYwkNFm7/5/+Zfsr4E?= =?us-ascii?Q?fI/qGoszNEvTJkxCvljGabRO6CBfCIc+YftxBlPpAvk70SA0lETyl8qTtGGt?= =?us-ascii?Q?lyExYkTNwDEWLl2lVv28Xd0ZDRv/S4ZCTCFiK9Bz39DYmjH0E1/RFvDQcKqg?= =?us-ascii?Q?Jjl77toL/EwkP0FbMNUbcFytkjGXnvlyqoZpYTG0+G2Fa6jXRceJPdBkuQCB?= =?us-ascii?Q?afeXt5IvLVepuFaQF3ubdGW5K2QGWFVrjaLxG1+XAtykf3Z1o2h237PmjdB9?= =?us-ascii?Q?+BkqJlMKd6iYPvsyIL7hC311oLs4ZF/MM7Grd3gU1dQL5BheFQA78PwRJ2xZ?= =?us-ascii?Q?WnX20FwOngexY9gUrGkivLh8YTIgoGVYAKZRRunWOF6MF/7tdBJ4DFqRqYmA?= =?us-ascii?Q?g9rwBkBg0f3r/T2cFszbNKj9Fkk69vzVjSj34XQcLrU4RTV4HT3J3MTqmJ4I?= =?us-ascii?Q?pWPQvwW0mOQ3cvTmBs/Kqo26S+mq?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dSBupQgC4fo4tAGvaYdU2BylL075qwT6WmahjUrMTos/lFdFQyP4TT09cSwS?= =?us-ascii?Q?jXTY1AapQClXYYttefdDqZ4DpGlBoHeT5qJnJQ47qgZ4yibCwKA2z0OXAe1O?= =?us-ascii?Q?70aSRaRd0WUqjehQVOPf2J/fSBAD7ldr2KAwDAiCyZjZHnQZHHIZNtVZFfE3?= =?us-ascii?Q?mBYXd5HrSlSOA5u1FssIWzjZF0yJKu1wRXHavJSrZw5w0J30bRVGAwMuEkks?= =?us-ascii?Q?EH5DV0Znj26b774rx1CxeN440vwJNO/7j1bpmYKgtTjkCySaD3UP4RRYhUSu?= =?us-ascii?Q?vK/YR6/kz9nGkw6eZUJjUPhMSDHSPhCAZO+f+4FavQdkCv04/eRgYRu8wxEe?= =?us-ascii?Q?htiK110DW9U9L4uUakfrGDEXg39cURlgGRFnptqJzpVjXCiP9fFekVFfvL7h?= =?us-ascii?Q?zbF8vuJEU4hD7NkSnBy+sTDmZHcJOPqtoJCNTfxt6n+AyPOTGmTXOLnPXZjV?= =?us-ascii?Q?9AD6hChwd+N5b+NZIMJY3xn9CN9nkgkhUUc1ZGvKLBOGPl6828XIqdcInrUX?= =?us-ascii?Q?EyZgQAWqf0k0WPRf2f/Hxdn+uDyBW2KyxdoGs+qPypjyJjPn6Uvpun9F0Pa3?= =?us-ascii?Q?iY+E5UcSfM8hOa2Hx24uUIWjbFHJ78NskO3VZhb2U2F9Zl/xCZsBi9kGdGBK?= =?us-ascii?Q?1MMTNJQ41JtHuN/+eb1OnweVlqKaHvTFpRONxNgdMhMJKz4rbMA78hgOn9Oc?= =?us-ascii?Q?EtMBlaV1ZxmdA0FEZiM7WdtOqE5fKpT4M/i6KEFJg6TZS/kuOKkWB2uteu6C?= =?us-ascii?Q?PvZVLeZcWEreQDau024q2BvDnbJ7ge1EZEi1a6dEaI30IPjGWMt5zmX6b1R8?= =?us-ascii?Q?BfvL4Q4DFsiASJDWq5E5Y865qP08FbXPsM7w0akckwZwV0FjpLUw9ffJdrKl?= =?us-ascii?Q?eCJeIzNcCSDeGJg43uUGxWJ6pT0YlHMEf0GUyVjnhXlNjtjF7/HOaBS1Y2aN?= =?us-ascii?Q?vqoAvddGhWPvuVULWucsKbzeg/ZurMJmPuGzIZMgkFjuDAvMcIR7r5GGMt/u?= =?us-ascii?Q?lhU5sGCxbQFvQ9V+a6GSHzB1v16GCWIuRFm2YmK3V0Bf7OX7Ig4bompPlgq1?= =?us-ascii?Q?lFa6GZLfZnk0c2hev3BvhVCNRwE37K74L4IxvVBvHr2v0KcvykLPaN6LFEIY?= =?us-ascii?Q?P3HCIbgQJUAEHiJ5lPB4D+/6+Sx2FhcsTv5jSWTbqY/oezM2TiNGY74k+uxJ?= =?us-ascii?Q?auKNNNfArk/t+H1wn6aWY0eZy9z2Gw4XVq6t5bS5YM278WpiubkLndE3fspH?= =?us-ascii?Q?wchR6cduein3p2y5P9FyLcOPdNLFpr2Z6oGhaEppeYnMycnoqpETNCbIP9E1?= =?us-ascii?Q?LmcTph3UhrtQMgEDX5fr2QA6fNqwcNM9MERqoazjiEWtLpX7HRvJXvii4ix2?= =?us-ascii?Q?Oi0JQAcxa/oiCNrCQfHs/lrC2z9Bxkd/46kzpjMquEd8zxXNSKUxG9nrgFQH?= =?us-ascii?Q?VSkOd3+9mBZSr1IeHvbM5S5wUmzL7M5EN2nrddn+tm9zFdPo8GAW+XGTRLR5?= =?us-ascii?Q?rbVQ1XsXh6v3toTWhVNOQyxMpMjFFjUGx9+gWOqUq/RXNDRISMfSFHHCqYgQ?= =?us-ascii?Q?DACZeOe/UImfWWRNVosHm0kBIFSBaqlDb3jpXCK4?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 47e1e1be-5606-4a10-e9a2-08dd4d0e2f2e X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:42:27.4232 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: O/fWrmggcDwZdEVLzkBY8y9r8/ewd/G4KM0zljlgY6SC0U4YsfG8dcD9OaolWriP X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8339 Content-Type: text/plain; charset="utf-8" Since migratetype is no longer overwritten during pageblock isolation, undoing pageblock isolation no longer needs which migratetype to restore. Signed-off-by: Zi Yan Acked-by: David Hildenbrand --- include/linux/page-isolation.h | 3 +-- mm/memory_hotplug.c | 4 ++-- mm/page_alloc.c | 2 +- mm/page_isolation.c | 9 +++------ 4 files changed, 7 insertions(+), 11 deletions(-) diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index 28c56f423e34..b8b44d3aacd4 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -33,8 +33,7 @@ bool pageblock_unisolate_and_move_free_pages(struct zone = *zone, struct page *pag int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, int migratetype, int flags); =20 -void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype); +void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n); =20 int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, int isol_flags); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index e3655f07dd6e..fb2216f267d8 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1229,7 +1229,7 @@ int online_pages(unsigned long pfn, unsigned long nr_= pages, build_all_zonelists(NULL); =20 /* Basic onlining is complete, allow allocation of onlined pages. */ - undo_isolate_page_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE); + undo_isolate_page_range(pfn, pfn + nr_pages); =20 /* * Freshly onlined pages aren't shuffled (e.g., all pages are placed to @@ -2128,7 +2128,7 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, =20 failed_removal_isolated: /* pushback to free area */ - undo_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE); + undo_isolate_page_range(start_pfn, end_pfn); memory_notify(MEM_CANCEL_OFFLINE, &arg); failed_removal_pcplists_disabled: lru_cache_enable(); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 9bba5b1c4f1d..4b251aa35b73 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6673,7 +6673,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, start, end, outer_start, outer_end); } done: - undo_isolate_page_range(start, end, migratetype); + undo_isolate_page_range(start, end); return ret; } EXPORT_SYMBOL(alloc_contig_range_noprof); diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 01d9a4eace7a..095cb2152fae 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -515,7 +515,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, page =3D __first_valid_page(pfn, pageblock_nr_pages); if (page && set_migratetype_isolate(page, migratetype, flags, start_pfn, end_pfn)) { - undo_isolate_page_range(isolate_start, pfn, migratetype); + undo_isolate_page_range(isolate_start, pfn); unset_migratetype_isolate( pfn_to_page(isolate_end - pageblock_nr_pages)); return -EBUSY; @@ -528,13 +528,10 @@ int start_isolate_page_range(unsigned long start_pfn,= unsigned long end_pfn, * undo_isolate_page_range - undo effects of start_isolate_page_range() * @start_pfn: The first PFN of the isolated range * @end_pfn: The last PFN of the isolated range - * @migratetype: New migrate type to set on the range * - * This finds every MIGRATE_ISOLATE page block in the given range - * and switches it to @migratetype. + * This finds and unsets every MIGRATE_ISOLATE page block in the given ran= ge */ -void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype) +void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n) { unsigned long pfn; struct page *page; --=20 2.47.2 From nobody Fri Dec 19 04:54:03 2025 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2043.outbound.protection.outlook.com [40.107.94.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B415E2676C4 for ; Fri, 14 Feb 2025 15:42:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739547760; cv=fail; b=j+HYMFBwxBAB2UNEUQMS9rhRcYXwpW5hIfLpgpEq0WTVvFcBFXosJABK4yMTnc0rbLSefp857OMyMYp4kSpDFkQpBOZcj7BaEDsp6L5vfCixagIXhGZFZcBFLYZJ3OA+JV/Nsm5SvWvynhYDe6KGNB+zbJ/HLZtzNMq/IuOxCos= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739547760; c=relaxed/simple; bh=lql8Nei02dbfEeakg5TomEwC5pnQFgdTeOZLMQYZoJ4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=bxABmj2OPHTXcSxll9doFRvVuYP1IG7yjQA/dFp70SUzjD2Gof2H9fgveq+u20gruE3i6HJk2AMrtVPt28zh+JPdoLDVoGEZ8wIYlJhNTHvgdyiBDts55+QxtxyETbVxK3IUm2Elx9NR+gGhvXfQbdTIwAHl0m3Rwp0UPHCvNaw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=ci7u2Hfi; arc=fail smtp.client-ip=40.107.94.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="ci7u2Hfi" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=F219jN1nSLrc1ewgQmsWreXnS0qN07dXRH19/5kh3heFFKcHHZoXLOU9dIUn8c1716TgZ4zO+hmF1PZrmit4C/8N4Wef6x2jhYQBIVDGhumSMxDDGTyCCj1+3RnVSOHF6T93RiREU3E9LZzCbQGkfzd8keAFFj2f4Zx67/A9ZNMxoTkulrPzRnStZvGSskQmWBNoQg5//DSpG5z4TXKAc/MkLqar4LwZV9wv6eyIoy1YmcvkAhYjmEVD72RvPzWqQa6t7vwXxWl12mmmOaSFrSf/bWoKZnPum2PIv8DmIUfWi7sZMw/i9POjR2e5n5N4NqatQF9EliVCRDGBwtHWog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ntTrZU1dkPj7e+9hvcpJN+E38Z6MlqNiYO8Xs1Kokdo=; b=ow1DoD1FFbHMtagz00BMUYfOLOi3uItDcVFXjJsidRllqdNM3O9LSLtTM+2Y49VCbUr6u+0SbkJ4/oe32nnztTC23PtbS/HHxvoRS8hmGswAQzxqbjHnSpgnAzAxpL4EQbTdbuhQencXisrV5zZhO0mymzj5mvmyaTipb+e0ibKJFtK4vvnmY//vc0DAU/PnEOCVlS+DYQbTacUIs44msoEU2WpxqXuAE51m1rRS3Ns2EetJcyQ3G8oe60I37jbZS79b6AQpUsHIUjvGTnATrLwuXcCWsQ+6TcDepHk/7Rz7EhwYYvvjyRTSu3h6GW2Nd37C+pS9Nk+6DY4jszWJgA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ntTrZU1dkPj7e+9hvcpJN+E38Z6MlqNiYO8Xs1Kokdo=; b=ci7u2HfiNm2fxTyUfcpBLspb4NMwuFfn53FmMoOa3cVp9Wur2WB+k91jjAKhcAIFRp+TIYm90NOcAVIJeMPFuisz9ocVquRxUTYSzgAcw+7oaZkF7ksteoHMSVhTu5ro2C0Lrc/wkpVlXLttrapKeiiFKxr4cr3X9MAvvpVSgVMhyLYuoNWssP6ekaXbBk8gGEUyWM0kxjhkZACfW0vV0oUX9g8vpoxTtxo5XdDajp+AxZFqgw+QM0GeecAQNp3Nu0wvPxucb7R0bqjv7w7wED2GXgjD75YvHQy5HW1wq1viQVMAJVP00k3ZX5QS+5tguC+EWItVYsTbqT3eOdo7xw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CY8PR12MB8339.namprd12.prod.outlook.com (2603:10b6:930:7e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.14; Fri, 14 Feb 2025 15:42:28 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8445.016; Fri, 14 Feb 2025 15:42:28 +0000 From: Zi Yan To: linux-mm@kvack.org, David Hildenbrand Cc: Oscar Salvador , Vlastimil Babka , Johannes Weiner , Baolin Wang , "Kirill A. Shutemov" , Mel Gorman , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 4/4] mm/page_isolation: remove migratetype parameter from more functions. Date: Fri, 14 Feb 2025 10:42:15 -0500 Message-ID: <20250214154215.717537-5-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250214154215.717537-1-ziy@nvidia.com> References: <20250214154215.717537-1-ziy@nvidia.com> Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: BN9P220CA0011.NAMP220.PROD.OUTLOOK.COM (2603:10b6:408:13e::16) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CY8PR12MB8339:EE_ X-MS-Office365-Filtering-Correlation-Id: 78ddc0d3-dbc7-415b-9a3f-08dd4d0e3003 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?yKiLafgzEcxGasXX4WJ+T58N+Vkq1lE/8UsAx/yRrlTL3dl4e1FnTOmIX3Nt?= =?us-ascii?Q?pO5DbrkBls1yDKhahmdZllkP4+RhFU1abYTqx30hOGhUeYORA7h12cqWCmIy?= =?us-ascii?Q?YKeZ0zkIHlSUOW4Df9aMB/sDMiY4yQPvCl0F/CwAshegbB0mmcF5UfoersUe?= =?us-ascii?Q?64TxE2NjBkC8CcvWL/9rD/czkt2Pslc6rK5+eAOYL0C14otlnjERpN/oDs0y?= =?us-ascii?Q?pRQgppqgzWlja63ibMV0TRjpDt/hITcD+AmOHUFoZ+bJ0FRC3oRlOWYiQZar?= =?us-ascii?Q?3U4ZqACq8c15b83cRVZhDJ4i1Lyv/h3kBrQN9bWhovE2tN6CFDTDM+1r9L0/?= =?us-ascii?Q?M4R0FsxaUZyW9maEcxzaXmRfEWQQOEU2bP/tZ388Sgj0rLACuzSXvVwFbOrC?= =?us-ascii?Q?BBWd6CrtBvZEZFHRZcT67Oe+v/1XocY6GfehvdRm9Z+9bg1sUEFkAnlho1FS?= =?us-ascii?Q?jhfIe5FzSWNPfIF86474h5enYI86yECDRMafqJen2V1L0nH91af2zemTKrG1?= =?us-ascii?Q?1x6Agl2cSOvxfTq4oadML7k7seBWsEchfZjHC4lNhgZoBbaVrQDnvEREtglC?= =?us-ascii?Q?Nbmf174H8nYCR8609K8Q13CqAmEHJBV9K5yLBYavHU9NhzT94NQzeHW0XcAP?= =?us-ascii?Q?hzY8VQvCTUO5GZoHDgosiSCq88XcVEl0IbcKAeNFrBNxDm++/PW+ongBqTSQ?= =?us-ascii?Q?nx5fmmrVksKIwMuXvE93qOByb7WFNwt/XlAq96bPvXova1Y1Q242SRrwa+HR?= =?us-ascii?Q?froSSeCvhRmERsv7vqljmVSIqp05U28uHDn8Qmx0atMcF3+8wV/KUsy1msO2?= =?us-ascii?Q?Ng5im03tHIqVDVRMHC23iHpbmMTJjzzEeP4CJPaeHRH5HZ28mxjZ54sXZ0MM?= =?us-ascii?Q?C+13anoUM+o+9dZjbj9Uo87aTpNF4AqGGWtBgB/wZpscSOwmd4tpPwHxl0CS?= =?us-ascii?Q?G9m1Etl9mcvhmPTweSTSBLlF0nLv9MKTt9JmGhj+wbfKazEqTga/os4za6hN?= =?us-ascii?Q?BN1PntAcItESMgwcVt7YdzOBtq4AIXBD3tvSW79TxQiV23kdwrtnZOe8i8gX?= =?us-ascii?Q?EF/D7CwtSxPnpPuFcMQbHqQNCpPtYPQnnrZ8hZQtJZTbHotj954w0pC+o3uw?= =?us-ascii?Q?2OSijBGgeoLQwLFAkyk/S7IAdq08+2+za0hL2K/6+ZtgDVLaPPGYC4SvaxJX?= =?us-ascii?Q?Lmgi2Z69zHk9iUETFSRt6KizAPNHbanWcMgO+r3vzHr6miBJfWfe2gSWo4WQ?= =?us-ascii?Q?90FI5CepQUlMEhwkUxjWVuS5T70cd7ghEnRF664hWeEvzb99ooaCAb6n2/hk?= =?us-ascii?Q?T7Of3CXRWKyx9C9ZTM5GOluEn7mX/BgeLygKLGKK1ladu2sb6WNICG6lcQBV?= =?us-ascii?Q?QIdXWFxNF3es8V+tCNL2U1CLqrt8?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?83bCZOOujte+2mhOpmxOhjB2UFeIGjxK4z1r2PtjFBIromFRQssIdZn4YpEt?= =?us-ascii?Q?qKEPwTw4DHHRHy/d2n29feDVlegCM9j0Q1pbZEqAcx5uAuLWDIrYQBAqQaSk?= =?us-ascii?Q?/vKJbJbuMDmkXQgVc0Y7fqYzVRz0y4FJIP3zXjTGtQl8YgVuKGcvLPlPyuLd?= =?us-ascii?Q?62558SEzLfXtz+j2DuL68vAR6V/ESq7wy+i3zAE98zRIjDka+0LtQY3B1z9g?= =?us-ascii?Q?qXsmz/URC3l8elrkHFwhysreC1c4zFgUuyEeugibK/4ueDrsWVGdy9zQ6TUt?= =?us-ascii?Q?V7RDnSvWzQtnBtmRgcKeKpA4gIlsUN0O1jKgB935LDfptuH/jNOX/dzGktp/?= =?us-ascii?Q?0Y5dRCzpEskp1EH2b56+5akm9TjU6g99cfZRkKDkLh0RIQDbAuh9a8srYpxh?= =?us-ascii?Q?jO+9egpLG0Hlkl5uuAjWRRjey97qANQDI2C4XvXafxipS4kRnH0zGA2xf+xk?= =?us-ascii?Q?/W2a+j6+ZzPoVP5bEglisoMKSdXCohC+4pqN5YUqIV00IAGh+BC/FnQnkz7y?= =?us-ascii?Q?848v0gNe0RvJE3Bs0MmFrKRdj/Jwlhwpv6h//uUqe5z9scpBEwki9NI9mru2?= =?us-ascii?Q?0x7LGCY0c0ZeCXEwoUwAtUblh89Q/cpjZy4heQlNMZVmFGaFQZjjpe+N8w0q?= =?us-ascii?Q?AU8BmC66YXksSo9t4v3vcXmJzyg0TAIWvMIda/pg3YvqIgvoqrTKDkz43oJi?= =?us-ascii?Q?ZnWJqnNCxCv7phF7uF1gPkKGcn+O1o4TGSSAR/c0NNE2geR3o28TrUE0Gcpk?= =?us-ascii?Q?AUuLa4Q215gn772OxrCIcQMppApPKSaVs5WmXUcc8DOW7TU0ulzCMilJwmbf?= =?us-ascii?Q?aJQZUloTBwJVelTaICg1EvWeXEXRgz7gh5PtHBgSHed4+suSHsBdxxY9vam9?= =?us-ascii?Q?hml/E8z003iXPBJakkh56X6EnYkDuIJnQtjr34j4iGUvR8/xmaygWJfA+kro?= =?us-ascii?Q?vk2A23VzlNMqpNZuq2J2N1IPIBSM8V3556rcURSWb+qpm80XvXg6FCeZXnm/?= =?us-ascii?Q?uxyr6VHSuE5j2+6jBlOdhXyUH1VOfCl87DW4P+STuyvLrUeFRdAcynFXPhNA?= =?us-ascii?Q?BeVARPB+mv5nRoVQzuoC2HHx/Yz42M5oJrKsu9ux4XjUhedvHi7VS5s30Exx?= =?us-ascii?Q?Y+/JRigRholeE6kbqSGjmdAQDJEdI/UBjM1GIvg9cj0A3+Xc12vhZh7QAiwT?= =?us-ascii?Q?5q9Tls2jLFAbBICwizLCVim+sLF0xd7n2wCuLc+P6/UrHP8OWuB8o1rAflN6?= =?us-ascii?Q?GJilRwTl0LlqcM5bfooGfQfDfNKLz+QzpTplHD8MBU9JWrOpplF3Jg8erUfW?= =?us-ascii?Q?SRCuLazV1OSubFE2Rmq0WxOzUNDjrYCbWpNQ8NKU+aVsUOpfkhp58rel1KEc?= =?us-ascii?Q?D23opXai2gPFly6rMsEMleKjhUih49ig5V667THV1Gex/Mb3p9pE4LwywXjW?= =?us-ascii?Q?fpzjwD1u1FomJ5/3uqA/6XNmGReH5jy2UAha+kYUt+gHLxTLb7Ti9tNPm+Jn?= =?us-ascii?Q?D3lA6deuJlsFQM7FeDDApFqq6ajO1bwMneoMZ0TLnuNJ6ZVJkH5k604bcVt/?= =?us-ascii?Q?deTpiDVO6oAd56nD2CeHMT3WYyoJcj/XwnZkXoha?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 78ddc0d3-dbc7-415b-9a3f-08dd4d0e3003 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2025 15:42:28.8236 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hGNxs7rnmp4Sv+oUppsNiziBhZjeVp+CAmRh0pVL/LDCn0yEJFfHlCJlJsja8EVA X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8339 Content-Type: text/plain; charset="utf-8" migratetype is no longer overwritten during pageblock isolation, start_isolate_page_range(), has_unmovable_pages(), and set_migratetype_isolate() no longer need which migratetype to restore during isolation failure. For has_unmoable_pages(), it needs to know if the isolation is for CMA allocation, so adding CMA_ALLOCATION to isolation flags to provide the information. alloc_contig_range() no longer needs migratetype. Replace it with a newly defined acr_flags_t to tell if an allocation is for CMA. So does __alloc_contig_migrate_range(). Signed-off-by: Zi Yan --- drivers/virtio/virtio_mem.c | 3 +-- include/linux/gfp.h | 6 +++++- include/linux/page-isolation.h | 15 +++++++++++--- include/trace/events/kmem.h | 14 +++++++------ mm/cma.c | 2 +- mm/memory_hotplug.c | 1 - mm/page_alloc.c | 33 +++++++++++++++---------------- mm/page_isolation.c | 36 ++++++++++++---------------------- 8 files changed, 56 insertions(+), 54 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 8a294b9cbcf6..160c681028d2 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -1243,8 +1243,7 @@ static int virtio_mem_fake_offline(struct virtio_mem = *vm, unsigned long pfn, if (atomic_read(&vm->config_changed)) return -EAGAIN; =20 - rc =3D alloc_contig_range(pfn, pfn + nr_pages, MIGRATE_MOVABLE, - GFP_KERNEL); + rc =3D alloc_contig_range(pfn, pfn + nr_pages, 0, GFP_KERNEL); if (rc =3D=3D -ENOMEM) /* whoops, out of memory */ return rc; diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 6bb1a5a7a4ae..ff1965561d48 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -400,9 +400,13 @@ static inline bool gfp_compaction_allowed(gfp_t gfp_ma= sk) extern gfp_t vma_thp_gfp_mask(struct vm_area_struct *vma); =20 #ifdef CONFIG_CONTIG_ALLOC + +typedef unsigned int __bitwise acr_flags_t; +#define ACR_CMA ((__force acr_flags_t)BIT(0)) // allocate for CMA + /* The below functions must be run on a range from a single zone. */ extern int alloc_contig_range_noprof(unsigned long start, unsigned long en= d, - unsigned migratetype, gfp_t gfp_mask); + acr_flags_t alloc_flags, gfp_t gfp_mask); #define alloc_contig_range(...) alloc_hooks(alloc_contig_range_noprof(__= VA_ARGS__)) =20 extern struct page *alloc_contig_pages_noprof(unsigned long nr_pages, gfp_= t gfp_mask, diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index b8b44d3aacd4..709a807202e9 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -22,8 +22,17 @@ static inline bool is_migrate_isolate(int migratetype) } #endif =20 -#define MEMORY_OFFLINE 0x1 -#define REPORT_FAILURE 0x2 +/* + * Isolation flags: + * MEMORY_OFFLINE - isolate to offline (!allocate) memory e.g., skip over + * PageHWPoison() pages and PageOffline() pages. + * REPORT_FAILURE - report details about the failure to isolate the range + * CMA_ALLOCATION - isolate for CMA allocations + */ +typedef unsigned int __bitwise isol_flags_t; +#define MEMORY_OFFLINE ((__force isol_flags_t)BIT(0)) +#define REPORT_FAILURE ((__force isol_flags_t)BIT(1)) +#define CMA_ALLOCATION ((__force isol_flags_t)BIT(2)) =20 void set_pageblock_migratetype(struct page *page, int migratetype); =20 @@ -31,7 +40,7 @@ bool pageblock_isolate_and_move_free_pages(struct zone *z= one, struct page *page) bool pageblock_unisolate_and_move_free_pages(struct zone *zone, struct pag= e *page); =20 int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype, int flags); + isol_flags_t flags); =20 void undo_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n); =20 diff --git a/include/trace/events/kmem.h b/include/trace/events/kmem.h index b37eb0a7060f..a29471d33566 100644 --- a/include/trace/events/kmem.h +++ b/include/trace/events/kmem.h @@ -304,6 +304,7 @@ TRACE_EVENT(mm_page_alloc_extfrag, __entry->change_ownership) ); =20 +#ifdef CONFIG_CONTIG_ALLOC TRACE_EVENT(mm_alloc_contig_migrate_range_info, =20 TP_PROTO(unsigned long start, @@ -311,9 +312,9 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, unsigned long nr_migrated, unsigned long nr_reclaimed, unsigned long nr_mapped, - int migratetype), + acr_flags_t alloc_flags), =20 - TP_ARGS(start, end, nr_migrated, nr_reclaimed, nr_mapped, migratetype), + TP_ARGS(start, end, nr_migrated, nr_reclaimed, nr_mapped, alloc_flags), =20 TP_STRUCT__entry( __field(unsigned long, start) @@ -321,7 +322,7 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, __field(unsigned long, nr_migrated) __field(unsigned long, nr_reclaimed) __field(unsigned long, nr_mapped) - __field(int, migratetype) + __field(acr_flags_t, alloc_flags) ), =20 TP_fast_assign( @@ -330,17 +331,18 @@ TRACE_EVENT(mm_alloc_contig_migrate_range_info, __entry->nr_migrated =3D nr_migrated; __entry->nr_reclaimed =3D nr_reclaimed; __entry->nr_mapped =3D nr_mapped; - __entry->migratetype =3D migratetype; + __entry->alloc_flags =3D alloc_flags; ), =20 - TP_printk("start=3D0x%lx end=3D0x%lx migratetype=3D%d nr_migrated=3D%lu n= r_reclaimed=3D%lu nr_mapped=3D%lu", + TP_printk("start=3D0x%lx end=3D0x%lx alloc_flags=3D%d nr_migrated=3D%lu n= r_reclaimed=3D%lu nr_mapped=3D%lu", __entry->start, __entry->end, - __entry->migratetype, + __entry->alloc_flags, __entry->nr_migrated, __entry->nr_reclaimed, __entry->nr_mapped) ); +#endif =20 /* * Required for uniquely and securely identifying mm in rss_stat tracepoin= t. diff --git a/mm/cma.c b/mm/cma.c index de5bc0c81fc2..e4ad8a5f8bde 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -461,7 +461,7 @@ static struct page *__cma_alloc(struct cma *cma, unsign= ed long count, =20 pfn =3D cma->base_pfn + (bitmap_no << cma->order_per_bit); mutex_lock(&cma_mutex); - ret =3D alloc_contig_range(pfn, pfn + count, MIGRATE_CMA, gfp); + ret =3D alloc_contig_range(pfn, pfn + count, ACR_CMA, gfp); mutex_unlock(&cma_mutex); if (ret =3D=3D 0) { page =3D pfn_to_page(pfn); diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index fb2216f267d8..b69fb43c5ac7 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -2018,7 +2018,6 @@ int offline_pages(unsigned long start_pfn, unsigned l= ong nr_pages, =20 /* set above range as isolated */ ret =3D start_isolate_page_range(start_pfn, end_pfn, - MIGRATE_MOVABLE, MEMORY_OFFLINE | REPORT_FAILURE); if (ret) { reason =3D "failure to isolate range"; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 4b251aa35b73..3a9328f7b2e7 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1913,14 +1913,15 @@ static bool __move_freepages_for_page_isolation(str= uct zone *zone, return true; } move: + /* use MIGRATETYPE_NO_ISO_MASK to get the non-isolate migratetype */ if (isolate_pageblock) { - from_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), - MIGRATETYPE_MASK); + from_mt =3D get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_NO_ISO_MASK); to_mt =3D MIGRATE_ISOLATE; } else { from_mt =3D MIGRATE_ISOLATE; - to_mt =3D __get_pfnblock_flags_mask(page, page_to_pfn(page), - MIGRATETYPE_MASK); + to_mt =3D get_pfnblock_flags_mask(page, page_to_pfn(page), + MIGRATETYPE_NO_ISO_MASK); } =20 __move_freepages_block(zone, start_pfn, from_mt, to_mt); @@ -6382,11 +6383,12 @@ static void alloc_contig_dump_pages(struct list_hea= d *page_list) =20 /* * [start, end) must belong to a single zone. - * @migratetype: using migratetype to filter the type of migration in + * @alloc_flags: using acr_flags_t to filter the type of migration in * trace_mm_alloc_contig_migrate_range_info. */ -static int __alloc_contig_migrate_range(struct compact_control *cc, - unsigned long start, unsigned long end, int migratetype) +int __alloc_contig_migrate_range(struct compact_control *cc, + unsigned long start, unsigned long end, + acr_flags_t alloc_flags) { /* This function is based on compact_zone() from compaction.c. */ unsigned int nr_reclaimed; @@ -6458,7 +6460,7 @@ static int __alloc_contig_migrate_range(struct compac= t_control *cc, putback_movable_pages(&cc->migratepages); } =20 - trace_mm_alloc_contig_migrate_range_info(start, end, migratetype, + trace_mm_alloc_contig_migrate_range_info(start, end, alloc_flags, total_migrated, total_reclaimed, total_mapped); @@ -6529,10 +6531,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_= mask, gfp_t *gfp_cc_mask) * alloc_contig_range() -- tries to allocate given range of pages * @start: start PFN to allocate * @end: one-past-the-last PFN to allocate - * @migratetype: migratetype of the underlying pageblocks (either - * #MIGRATE_MOVABLE or #MIGRATE_CMA). All pageblocks - * in range must have the same migratetype and it must - * be either of the two. + * @alloc_flags: allocation information * @gfp_mask: GFP mask. Node/zone/placement hints are ignored; only some * action and reclaim modifiers are supported. Reclaim modifiers * control allocation behavior during compaction/migration/reclaim. @@ -6549,7 +6548,7 @@ static int __alloc_contig_verify_gfp_mask(gfp_t gfp_m= ask, gfp_t *gfp_cc_mask) * need to be freed with free_contig_range(). */ int alloc_contig_range_noprof(unsigned long start, unsigned long end, - unsigned migratetype, gfp_t gfp_mask) + acr_flags_t alloc_flags, gfp_t gfp_mask) { unsigned long outer_start, outer_end; int ret =3D 0; @@ -6590,7 +6589,8 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, * put back to page allocator so that buddy can use them. */ =20 - ret =3D start_isolate_page_range(start, end, migratetype, 0); + ret =3D start_isolate_page_range(start, end, + (alloc_flags & ACR_CMA) ? CMA_ALLOCATION : 0); if (ret) goto done; =20 @@ -6606,7 +6606,7 @@ int alloc_contig_range_noprof(unsigned long start, un= signed long end, * allocated. So, if we fall through be sure to clear ret so that * -EBUSY is not accidentally used or returned to caller. */ - ret =3D __alloc_contig_migrate_range(&cc, start, end, migratetype); + ret =3D __alloc_contig_migrate_range(&cc, start, end, alloc_flags); if (ret && ret !=3D -EBUSY) goto done; =20 @@ -6683,8 +6683,7 @@ static int __alloc_contig_pages(unsigned long start_p= fn, { unsigned long end_pfn =3D start_pfn + nr_pages; =20 - return alloc_contig_range_noprof(start_pfn, end_pfn, MIGRATE_MOVABLE, - gfp_mask); + return alloc_contig_range_noprof(start_pfn, end_pfn, 0, gfp_mask); } =20 static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn, diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 095cb2152fae..c39b51e96186 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -31,7 +31,7 @@ * */ static struct page *has_unmovable_pages(unsigned long start_pfn, unsigned = long end_pfn, - int migratetype, int flags) + isol_flags_t flags) { struct page *page =3D pfn_to_page(start_pfn); struct zone *zone =3D page_zone(page); @@ -46,7 +46,7 @@ static struct page *has_unmovable_pages(unsigned long sta= rt_pfn, unsigned long e * isolate CMA pageblocks even when they are not movable in fact * so consider them movable here. */ - if (is_migrate_cma(migratetype)) + if (flags & CMA_ALLOCATION) return NULL; =20 return page; @@ -151,7 +151,7 @@ static struct page *has_unmovable_pages(unsigned long s= tart_pfn, unsigned long e * present in [start_pfn, end_pfn). The pageblock must intersect with * [start_pfn, end_pfn). */ -static int set_migratetype_isolate(struct page *page, int migratetype, int= isol_flags, +static int set_migratetype_isolate(struct page *page, isol_flags_t isol_fl= ags, unsigned long start_pfn, unsigned long end_pfn) { struct zone *zone =3D page_zone(page); @@ -186,7 +186,7 @@ static int set_migratetype_isolate(struct page *page, i= nt migratetype, int isol_ end_pfn); =20 unmovable =3D has_unmovable_pages(check_unmovable_start, check_unmovable_= end, - migratetype, isol_flags); + isol_flags); if (!unmovable) { if (!pageblock_isolate_and_move_free_pages(zone, page)) { spin_unlock_irqrestore(&zone->lock, flags); @@ -296,7 +296,6 @@ __first_valid_page(unsigned long pfn, unsigned long nr_= pages) * @isolate_before: isolate the pageblock before the boundary_pfn * @skip_isolation: the flag to skip the pageblock isolation in second * isolate_single_pageblock() - * @migratetype: migrate type to set in error recovery. * * Free and in-use pages can be as big as MAX_PAGE_ORDER and contain more = than one * pageblock. When not all pageblocks within a page are isolated at the sa= me @@ -311,8 +310,8 @@ __first_valid_page(unsigned long pfn, unsigned long nr_= pages) * either. The function handles this by splitting the free page or migrati= ng * the in-use page then splitting the free page. */ -static int isolate_single_pageblock(unsigned long boundary_pfn, int flags, - bool isolate_before, bool skip_isolation, int migratetype) +static int isolate_single_pageblock(unsigned long boundary_pfn, isol_flags= _t flags, + bool isolate_before, bool skip_isolation) { unsigned long start_pfn; unsigned long isolate_pageblock; @@ -338,11 +337,9 @@ static int isolate_single_pageblock(unsigned long boun= dary_pfn, int flags, zone->zone_start_pfn); =20 if (skip_isolation) { - int mt __maybe_unused =3D get_pageblock_migratetype(pfn_to_page(isolate_= pageblock)); - - VM_BUG_ON(!is_migrate_isolate(mt)); + VM_BUG_ON(!get_pageblock_isolate(pfn_to_page(isolate_pageblock))); } else { - ret =3D set_migratetype_isolate(pfn_to_page(isolate_pageblock), migratet= ype, + ret =3D set_migratetype_isolate(pfn_to_page(isolate_pageblock), flags, isolate_pageblock, isolate_pageblock + pageblock_nr_pages); =20 if (ret) @@ -441,14 +438,7 @@ static int isolate_single_pageblock(unsigned long boun= dary_pfn, int flags, * start_isolate_page_range() - mark page range MIGRATE_ISOLATE * @start_pfn: The first PFN of the range to be isolated. * @end_pfn: The last PFN of the range to be isolated. - * @migratetype: Migrate type to set in error recovery. - * @flags: The following flags are allowed (they can be combined in - * a bit mask) - * MEMORY_OFFLINE - isolate to offline (!allocate) memory - * e.g., skip over PageHWPoison() pages - * and PageOffline() pages. - * REPORT_FAILURE - report details about the failure to - * isolate the range + * @flags: isolation flags * * Making page-allocation-type to be MIGRATE_ISOLATE means free pages in * the range will never be allocated. Any free pages and pages freed in the @@ -481,7 +471,7 @@ static int isolate_single_pageblock(unsigned long bound= ary_pfn, int flags, * Return: 0 on success and -EBUSY if any part of range cannot be isolated. */ int start_isolate_page_range(unsigned long start_pfn, unsigned long end_pf= n, - int migratetype, int flags) + isol_flags_t flags) { unsigned long pfn; struct page *page; @@ -493,7 +483,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, =20 /* isolate [isolate_start, isolate_start + pageblock_nr_pages) pageblock = */ ret =3D isolate_single_pageblock(isolate_start, flags, false, - skip_isolation, migratetype); + skip_isolation); if (ret) return ret; =20 @@ -502,7 +492,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, =20 /* isolate [isolate_end - pageblock_nr_pages, isolate_end) pageblock */ ret =3D isolate_single_pageblock(isolate_end, flags, true, - skip_isolation, migratetype); + skip_isolation); if (ret) { unset_migratetype_isolate(pfn_to_page(isolate_start)); return ret; @@ -513,7 +503,7 @@ int start_isolate_page_range(unsigned long start_pfn, u= nsigned long end_pfn, pfn < isolate_end - pageblock_nr_pages; pfn +=3D pageblock_nr_pages) { page =3D __first_valid_page(pfn, pageblock_nr_pages); - if (page && set_migratetype_isolate(page, migratetype, flags, + if (page && set_migratetype_isolate(page, flags, start_pfn, end_pfn)) { undo_isolate_page_range(isolate_start, pfn); unset_migratetype_isolate( --=20 2.47.2