From nobody Wed Jun 17 01:50:58 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010050.outbound.protection.outlook.com [52.101.85.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D8DA22EDD62 for ; Tue, 28 Apr 2026 15:52:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391539; cv=fail; b=thudLMIknBUNOt2Hehu+O858rMoGoJODc4v8IojURvKzMyWd5TWuvf3eQpnkB2iDm6qPe6Zs75j5822IeWLQjY9gkJ/d8lG3EYcp3hQa10qk3B0M2CT0Q5eDorGAb2Uj4hDzIDrH3+j3bd9WU5XFOmV+xlcFd+/s/1sN1Gkhg9s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391539; c=relaxed/simple; bh=l1WY1F2y/bwB6TDN0jxNgIU1J6lgcVfHgYQgquONt6E=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=DN2ywtPRTQdkFURw08MHbHPcGqed7GLwEYm4hAR+H7A7OSf2IkA/ejCJVJdhVE8By0jGwu8VLQKuHKF1iG+yKFNkf4lLiZ+mOpvaOfP85Zhh1noSrLFCZeiW0JpOUghkqGIWrJQUD76LHXv28+okYs17ij7tDv/h8wNic/pANDE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=IQWRcdOD; arc=fail smtp.client-ip=52.101.85.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="IQWRcdOD" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=w114zZToBU0thxIhAy+NTk8PH7N6SQTMYWEdXKWGO5K5V/siFeZDhVVybXX0XMAF1zRJHK8KR1pZLVTeUWxXJZXMIIIt7dZov6xsNWGauBrRAx2PghvhH/d94YWZD+cJyR02tanzWqGNHyrpljK/sypNqEvn8OLT5lsTs5kt8ZUE7PvHG7PsxIasrD7Nzyn9nXmsb0VDmUu+QmhUARCVOF904Z2pNQLSa8+ivt82vCcNu+OmN7WT9/jxohi1VRs+I6T/XR7OecswNkxa1uDwJLVjnY/A8ogWDUitgQ+5hWeI5lzhNXwNbrOIq2W2Q86RpTvLez9jZl7xHUTy0ghtHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i4xms1cijBzKJu8O/pAsQ0BL/RdmQNHdZRvXeM9xbLE=; b=yLaBPTaM7IIbLtY3Py9LP+HcUJSF+EPussQUxqwc1UhxYGUcqXMgoQPXnNMsi5fII1VhbHg0K5MksM1fuvf/Qs+s2Rg7lKkKu4DkHr9jH3MrYDLY2OZi3ByR/fWtwE+hXotbrwyrC8pOGlDHG9qFfydOcrwexqi6gAxcJtRhWKr5g8Suti/afNQibnUU4DpwMh0WvZbdPhHfWIZIq/GRel04vNEk3GQ0gOQgjmYE7CBmhC5457AqbRMYq1hvLeag6OGBsPK71qQ2UrimpkS12gBM7VmieIUQDcKN6VaNVEUHNqXm70UW1WgWltose6YvZH93h/tu8aVo1GG+ezSImA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i4xms1cijBzKJu8O/pAsQ0BL/RdmQNHdZRvXeM9xbLE=; b=IQWRcdODnXUjN23DsyVNJx6UTD5YftYo5Ml20kheheyw2w05PYcaAJ4t6zba31kGRujsMyOtC0IowelWyDjt35VNQcxM5ga0RY3kh4/kL7r+J9RgeuHMYQlAswHRbYCtA0xKJJS3CaBDgzpURSfo2E0l3/jlQYQ9Sk4yqPE50nI= Received: from BYAPR06CA0005.namprd06.prod.outlook.com (2603:10b6:a03:d4::18) by CYYPR12MB8704.namprd12.prod.outlook.com (2603:10b6:930:c2::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.16; Tue, 28 Apr 2026 15:52:12 +0000 Received: from SJ1PEPF000026C8.namprd04.prod.outlook.com (2603:10b6:a03:d4:cafe::d5) by BYAPR06CA0005.outlook.office365.com (2603:10b6:a03:d4::18) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.26 via Frontend Transport; Tue, 28 Apr 2026 15:52:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SJ1PEPF000026C8.mail.protection.outlook.com (10.167.244.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Tue, 28 Apr 2026 15:52:11 +0000 Received: from kaveri.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 28 Apr 2026 10:51:53 -0500 From: Shivank Garg To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Shivank Garg , Baolin Wang , Lance Yang Subject: [PATCH 1/7] mm/migrate: rename PAGE_ migration flags to FOLIO_ Date: Tue, 28 Apr 2026 15:50:39 +0000 Message-ID: <20260428155043.39251-4-shivankg@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260428155043.39251-2-shivankg@amd.com> References: <20260428155043.39251-2-shivankg@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000026C8:EE_|CYYPR12MB8704:EE_ X-MS-Office365-Filtering-Correlation-Id: 55f4a7ed-b4e2-438c-9088-08dea53e1ca1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|7416014|376014|36860700016|18002099003|22082099003|56012099003; X-Microsoft-Antispam-Message-Info: gQEmw4aapZepwWUEtr+Q4Q6GKSu2ui8iE9/RoXXo/AlIB4XdupG0U0RtjcBoMaX0WY9OlAroROAvdeSeZ2lfWIKpl36gHF2TNmN/KXFnh2K6NTDdQqjmrT6dre/1BvCs+qbfwsAENdtohukczvVk1H0TbgiEgubfS0r/FaHpriBz8n/tbCy3ZCBYuTmkbC6aiTVaVgy4yYMpvIZX2ywt4nrhItnJvhdpz11LpPUeNZLkw12U+vPTtmI7zfi5pvxITFCHqTnPRrsAQOYernDtHnigp4kegc9aWJft6V1CVfnTTEQY4x1389NqiVj6+HoooNeomQzmEMGGBZF2MVxeiU19Glf8ZrJSZZV8ZJocu9iaz2HXi++K0YOqMCAjHj9OXmrTeLfLherjhjQiBCRVdJ5KGdXQDJ/CBCSKCb/8YikNMZyjL9AeyofkS6buTasEQrz8tL60eFbsre2PFKrrK9MjDjbXrRKq5Ns4daAQSXiZOaWbd6RldthZLfZBxVa7OQFmn25Nb2JjIu5zxKumUpJgNAlJmypfI/ci09czX19SoYR/ITv/NGEnIxAqoXrwwPg/P6bOZ/2zXiIxQonOv2DuyDhFJVlbKItYcQ/54yDkD+UA5I76pOds1lPdpZmMIMF+gEk8CbC3BPqX3OXLT5ukfOsmitFdPlTl9NZZQu8e/aeyFEwyBF/q13xq9LWzcbeLOWjuQ0qMIhJYDcWMWZLfI/ASz/GRs/BxQO1eKEfDBrs40wKzDXOW18rCrULNIHNc9iuNFjjXpKiARdeggA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(7416014)(376014)(36860700016)(18002099003)(22082099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: okaJK36rdwdAR3rOjMRvQSNDQsazaVV3EofYMBqatQmgKJa0oZrKYlRp0A6j9ZWtxw6cSzLHmWBXOrtoLYbUSfEg+mfz5G9AzcCS0Y62QxCDkx8pZx9rxqKGHLPPI0pf4iW3AvqPJNrhHMRsLw1W3w67ws/mA8yMTuy9aQiZCSMlZq6OeVCeGwy/h5XabT/PTD+l00U1/871/Cy4yjo7MIvEHW6+AC97iZROvxayGDBUY/UvXS+fPR+E24Q8MZi3jf6PL2eLIr1yQTpvGKU3n1sBFzErWbQsVNxbytLuPHzvfMAGvE3Tmu+0xIiV5enrVCAohD11+bpkEW0vZnLv+7gkJF6cqyQtZCSy1N5HfDvInZgS3gUggDgRYd0n1vDjGzbZ3k/0MmQsCKQVQpCafePWI8zFGjNa9l8GyfB+MVrYI3BKXHG4qJ9NJ3hlgmQi X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2026 15:52:11.8759 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 55f4a7ed-b4e2-438c-9088-08dea53e1ca1 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000026C8.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR12MB8704 Content-Type: text/plain; charset="utf-8" These flags only track folio-specific state during migration and are not used for movable_ops pages. Rename the enum values and the old_page_state variable to match. No functional change. Suggested-by: David Hildenbrand Acked-by: David Hildenbrand (Arm) Reviewed-by: Zi Yan Reviewed-by: Baolin Wang Reviewed-by: Lance Yang Signed-off-by: Shivank Garg Reviewed-by: Huang Ying --- mm/migrate.c | 48 +++++++++++++++++++++++------------------------- 1 file changed, 23 insertions(+), 25 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 8a64291ab5b4..0c6a0ab6ecce 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1135,26 +1135,24 @@ static int move_to_new_folio(struct folio *dst, str= uct folio *src, * This is safe because nobody is using it except us. */ enum { - PAGE_WAS_MAPPED =3D BIT(0), - PAGE_WAS_MLOCKED =3D BIT(1), - PAGE_OLD_STATES =3D PAGE_WAS_MAPPED | PAGE_WAS_MLOCKED, + FOLIO_WAS_MAPPED =3D BIT(0), + FOLIO_WAS_MLOCKED =3D BIT(1), + FOLIO_OLD_STATES =3D FOLIO_WAS_MAPPED | FOLIO_WAS_MLOCKED, }; =20 static void __migrate_folio_record(struct folio *dst, - int old_page_state, - struct anon_vma *anon_vma) + int old_folio_state, struct anon_vma *anon_vma) { - dst->private =3D (void *)anon_vma + old_page_state; + dst->private =3D (void *)anon_vma + old_folio_state; } =20 static void __migrate_folio_extract(struct folio *dst, - int *old_page_state, - struct anon_vma **anon_vmap) + int *old_folio_state, struct anon_vma **anon_vmap) { unsigned long private =3D (unsigned long)dst->private; =20 - *anon_vmap =3D (struct anon_vma *)(private & ~PAGE_OLD_STATES); - *old_page_state =3D private & PAGE_OLD_STATES; + *anon_vmap =3D (struct anon_vma *)(private & ~FOLIO_OLD_STATES); + *old_folio_state =3D private & FOLIO_OLD_STATES; dst->private =3D NULL; } =20 @@ -1209,7 +1207,7 @@ static int migrate_folio_unmap(new_folio_t get_new_fo= lio, { struct folio *dst; int rc =3D -EAGAIN; - int old_page_state =3D 0; + int old_folio_state =3D 0; struct anon_vma *anon_vma =3D NULL; bool locked =3D false; bool dst_locked =3D false; @@ -1253,7 +1251,7 @@ static int migrate_folio_unmap(new_folio_t get_new_fo= lio, } locked =3D true; if (folio_test_mlocked(src)) - old_page_state |=3D PAGE_WAS_MLOCKED; + old_folio_state |=3D FOLIO_WAS_MLOCKED; =20 if (folio_test_writeback(src)) { /* @@ -1302,7 +1300,7 @@ static int migrate_folio_unmap(new_folio_t get_new_fo= lio, dst_locked =3D true; =20 if (unlikely(page_has_movable_ops(&src->page))) { - __migrate_folio_record(dst, old_page_state, anon_vma); + __migrate_folio_record(dst, old_folio_state, anon_vma); return 0; } =20 @@ -1328,11 +1326,11 @@ static int migrate_folio_unmap(new_folio_t get_new_= folio, VM_BUG_ON_FOLIO(folio_test_anon(src) && !folio_test_ksm(src) && !anon_vma, src); try_to_migrate(src, mode =3D=3D MIGRATE_ASYNC ? TTU_BATCH_FLUSH : 0); - old_page_state |=3D PAGE_WAS_MAPPED; + old_folio_state |=3D FOLIO_WAS_MAPPED; } =20 if (!folio_mapped(src)) { - __migrate_folio_record(dst, old_page_state, anon_vma); + __migrate_folio_record(dst, old_folio_state, anon_vma); return 0; } =20 @@ -1344,7 +1342,7 @@ static int migrate_folio_unmap(new_folio_t get_new_fo= lio, if (rc =3D=3D -EAGAIN) ret =3D NULL; =20 - migrate_folio_undo_src(src, old_page_state & PAGE_WAS_MAPPED, + migrate_folio_undo_src(src, old_folio_state & FOLIO_WAS_MAPPED, anon_vma, locked, ret); migrate_folio_undo_dst(dst, dst_locked, put_new_folio, private); =20 @@ -1358,13 +1356,13 @@ static int migrate_folio_move(free_folio_t put_new_= folio, unsigned long private, struct list_head *ret) { int rc; - int old_page_state =3D 0; + int old_folio_state =3D 0; struct anon_vma *anon_vma =3D NULL; bool src_deferred_split =3D false; bool src_partially_mapped =3D false; struct list_head *prev; =20 - __migrate_folio_extract(dst, &old_page_state, &anon_vma); + __migrate_folio_extract(dst, &old_folio_state, &anon_vma); prev =3D dst->lru.prev; list_del(&dst->lru); =20 @@ -1404,10 +1402,10 @@ static int migrate_folio_move(free_folio_t put_new_= folio, unsigned long private, * isolated from the unevictable LRU: but this case is the easiest. */ folio_add_lru(dst); - if (old_page_state & PAGE_WAS_MLOCKED) + if (old_folio_state & FOLIO_WAS_MLOCKED) lru_add_drain(); =20 - if (old_page_state & PAGE_WAS_MAPPED) + if (old_folio_state & FOLIO_WAS_MAPPED) remove_migration_ptes(src, dst, 0); =20 out_unlock_both: @@ -1439,11 +1437,11 @@ static int migrate_folio_move(free_folio_t put_new_= folio, unsigned long private, */ if (rc =3D=3D -EAGAIN) { list_add(&dst->lru, prev); - __migrate_folio_record(dst, old_page_state, anon_vma); + __migrate_folio_record(dst, old_folio_state, anon_vma); return rc; } =20 - migrate_folio_undo_src(src, old_page_state & PAGE_WAS_MAPPED, + migrate_folio_undo_src(src, old_folio_state & FOLIO_WAS_MAPPED, anon_vma, true, ret); migrate_folio_undo_dst(dst, true, put_new_folio, private); =20 @@ -1777,11 +1775,11 @@ static void migrate_folios_undo(struct list_head *s= rc_folios, dst =3D list_first_entry(dst_folios, struct folio, lru); dst2 =3D list_next_entry(dst, lru); list_for_each_entry_safe(folio, folio2, src_folios, lru) { - int old_page_state =3D 0; + int old_folio_state =3D 0; struct anon_vma *anon_vma =3D NULL; =20 - __migrate_folio_extract(dst, &old_page_state, &anon_vma); - migrate_folio_undo_src(folio, old_page_state & PAGE_WAS_MAPPED, + __migrate_folio_extract(dst, &old_folio_state, &anon_vma); + migrate_folio_undo_src(folio, old_folio_state & FOLIO_WAS_MAPPED, anon_vma, true, ret_folios); list_del(&dst->lru); migrate_folio_undo_dst(dst, true, put_new_folio, private); --=20 2.43.0 From nobody Wed Jun 17 01:50:58 2026 Received: from DM5PR21CU001.outbound.protection.outlook.com (mail-centralusazon11011026.outbound.protection.outlook.com [52.101.62.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71D161FBEA6 for ; Tue, 28 Apr 2026 15:52:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.62.26 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391569; cv=fail; b=YUr3PK6cTPNutoABLCwNr1+pABYAkjZwzx4zr70qn8om8uLb1MFgqxkpOMh0R+4WiYQR5zE2q1439FB/BNRq2HjyB8I7eMgTtrf4DlIuesDXJjce3j/SI5UX6i8GM/u3XzljVJl3km8jXmXWdkaf2OJmBRLf8lPbCHmMAe+YoF0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391569; c=relaxed/simple; bh=I38WGPg+3IGROGukiRkPHe1K9GwD1FSW1QuFMSuFxmE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gzUuDkxhxsqjsYyDAtkK+owpa3txi1D7aLytT2P3+EGQ2/ABom0JfUCYMWTCXBBE/ngtSnzhr3yPPEGFf4xQr1GJ65QH04hOHoNhDeBtSb7RSxIKxJf02JL+br/sqt/q1TRnf9Tz1JuGqRTuQEwauRPqe80J21Zi5l/9nXncT74= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=IVMDBok5; arc=fail smtp.client-ip=52.101.62.26 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="IVMDBok5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GNPamv+jQDkYBwdPBnYqvJ57p/8/bBj3/jMyohc47z390zoKVKHae58OesGV36seFj+97cQT5RyYIcOtVsv6ZYsXgXhaqtuU2xA9R3e11HkU2w2sxI5Zg6iL8RiGwYTwnse4YBAyKU6qYlU9/ZUNYCiJilGRKGQ0KTIYdZOWj7So2KFasgzn94HoR3t4bMoS+wH2XyEqqvWVPPiPsPdZC9OCWCNYdpVwHX+HYJvAg5+8G9+iipAUKgjbQpU527E7ptxo0rhfZHhbHuBxLcXiyS7Cavw9XZu5Rcxj6VnX6IWhOn1g11aHWX/behE67t6CX8/XESFrO6iGDK/4EN8R6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dPEBg8P1WzTi/oSWfHVMqGKa/e05YHT1uXS9J9s75CY=; b=khf3WUiix1aE2Zp5ue7TICtmobqwuiLg2jqOvnb+hHEP/ZY8F8WIE0prpKkUO8cjX2/niFdjbS+Fr71eGORALejhwjAZouDknAGsb1Jt4+eIHymApVlJQblvf+gz5/Ce9sF2FF+mpJP4HCYpIOJ0yrlS06+E5rq8Zram08LPf6+g5OmxcPrkcjjTLn62xRZhotAYEBKRKeJ9b6Ef3L8Rr+Ikd1FCR8QvWaWP0yo+HUtwWIfm7d+gZjUuzTga3jiX7eBd3TQJLIPLwIhwEFab4TM3Jgb/NYwZQzXQQzRFO99PXr7i3M5ARkYflxoDn1B22F1sxOloyjfwNB3E4JzqsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dPEBg8P1WzTi/oSWfHVMqGKa/e05YHT1uXS9J9s75CY=; b=IVMDBok50ynmLJZ43lNVjsDC7FH00gLhULZwb9PZUQChYQ7J41vAUBfoMYtZxlozcQGOXZWrY8lgma+w4T/Zwc2630/3bHu0TWfDiqv9joSjR5Om3zD1hgfBYnCAy5lXP7wKlfVEp13fxQDkCbchAmDqieCHDxP/eiYzTOfw3pw= Received: from BY1P220CA0021.NAMP220.PROD.OUTLOOK.COM (2603:10b6:a03:5c3::16) by BL4PR12MB9483.namprd12.prod.outlook.com (2603:10b6:208:590::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.22; Tue, 28 Apr 2026 15:52:36 +0000 Received: from SJ1PEPF000026C7.namprd04.prod.outlook.com (2603:10b6:a03:5c3:cafe::62) by BY1P220CA0021.outlook.office365.com (2603:10b6:a03:5c3::16) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.28 via Frontend Transport; Tue, 28 Apr 2026 15:52:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SJ1PEPF000026C7.mail.protection.outlook.com (10.167.244.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Tue, 28 Apr 2026 15:52:35 +0000 Received: from kaveri.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 28 Apr 2026 10:52:24 -0500 From: Shivank Garg To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Shivank Garg Subject: [PATCH 2/7] mm/migrate: use migrate_info field instead of private Date: Tue, 28 Apr 2026 15:50:41 +0000 Message-ID: <20260428155043.39251-6-shivankg@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260428155043.39251-2-shivankg@amd.com> References: <20260428155043.39251-2-shivankg@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000026C7:EE_|BL4PR12MB9483:EE_ X-MS-Office365-Filtering-Correlation-Id: e86c1940-6492-4b3b-74a6-08dea53e2aec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|1800799024|36860700016|82310400026|376014|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: Kk/oovJeNUg3Y+ALijfkmoRu99Mo154uNHUdTLCB3e6WBVsUD18VsjQoYuO+vmHP1g6EUQy8rp0BC5gScCBnYeqg0Hrip+fE9LlTkWZ7nLJK2kXvNRRaxBbNvfXpWXPAm6dR0NVTGDVdgkaD4GyBeukA/kDmkIgaACO7MQ0bOg/ElbhgBTSh+wRYMnhy1R4pSF53mjXV9osBVH4RhC79oGNbs2eSQP35mHdEoIvG5L8zFeSYlyn+h9P10QW3VHkh1v266S8Bfy45ELPqj93U275xXr88cXOn0jvl8MOKAuWndHie5mn/UUInWjaHm67f73myArFo+RUat7SRh2x7yFHmDtiHnfyNCRzoHEuDwkiX+uOvOMztq5v4V3aoIZdHPaKRKoWfgwF3gnP2K/hgFcZw8KqgtKJjWHIFMS5QM+RGTsG6muIdMKve9T3lJhiYN8Ts/J6Dz6KL6W/WaXBNoltUH96bVitWSjvgI3/mg7xRdBJDxtH5g7skJGlyYb999rOiF25qvC8tjJ+9PiVTRUnE9A2waazz07IQuyvjvQdPycsLUbM3ik0gcJ6f7U3pSCY4FvF7KnES8ncXOqiHGd35O515TPBizcV9s+imeGuRoIesN3e1ARfNi4HNmVHke0py0WCTHtLw0RCWQV8UoNx1YvBkiTPiabgyd7X5Ik0JMlBPwP6MXi8uQe+Nq4pHUI7qNM5m0gx5+L/hNsa+XyOnliEYOpXXvEE4c6CnK6cIwTuhw1L6SWRp9bOaXiVpM4/Hj7vUcGc5sj9v34Rgpw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(1800799024)(36860700016)(82310400026)(376014)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: TurHvNUM5LwQw8rVwsUM5wE/Grlto9UpDBogP/c21/stf/zYte62N3806+b6WAxjezALYtnc4lCo/54+aX7K1mVKpJuqMg3SW62u6AFzU/ZAyJxKggI+Ke0mkdXE+lYQ1liFZy0C2zYvhJirFA5kXXy8i61Nza1WGGTULa8/gdAgAonb4yXJryDqpueKJeEqzmzpSAcpFSKQ3xECxf/OLWP6qEDfRvc9f/VtJWnZPB+UOclwGYMqdpguMFLuwRfOjAiBath4yphGw9KCnQ/ezbSAVmkb3raa4ZpJJFUljzwZRpO32xM1dLURPMrXtHJF5FuDq1ceLgh6xRnwZLYBAi4/EzHNZ68VyI4FHPNZG/ILapdPslULFYzct+huOBYt1w+Y7Xmg8eZBYknfMlIWE7fLymrHFqffsXoesEm10XvgP5BYVeCByTyx4ON+KZPv X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2026 15:52:35.8475 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e86c1940-6492-4b3b-74a6-08dea53e2aec X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000026C7.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL4PR12MB9483 Content-Type: text/plain; charset="utf-8" Add an unsigned long migrate_info member to the struct folio union and use it to store migration state (anon_vma pointer and FOLIO_WAS_* flags) instead of using folio->private. No functional change. Suggested-by: David Hildenbrand Signed-off-by: Shivank Garg Acked-by: David Hildenbrand (Arm) Reviewed-by: Huang Ying Reviewed-by: Jonathan Cameron --- include/linux/mm_types.h | 1 + mm/migrate.c | 14 +++++++------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index a308e2c23b82..f52818dcf4d2 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -426,6 +426,7 @@ struct folio { union { void *private; swp_entry_t swap; + unsigned long migrate_info; }; atomic_t _mapcount; atomic_t _refcount; diff --git a/mm/migrate.c b/mm/migrate.c index 0c6a0ab6ecce..03c2a6f7e5e4 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1130,7 +1130,7 @@ static int move_to_new_folio(struct folio *dst, struc= t folio *src, } =20 /* - * To record some information during migration, we use unused private + * To record some information during migration, we use the migrate_info * field of struct folio of the newly allocated destination folio. * This is safe because nobody is using it except us. */ @@ -1143,17 +1143,17 @@ enum { static void __migrate_folio_record(struct folio *dst, int old_folio_state, struct anon_vma *anon_vma) { - dst->private =3D (void *)anon_vma + old_folio_state; + dst->migrate_info =3D (unsigned long)anon_vma | old_folio_state; } =20 static void __migrate_folio_extract(struct folio *dst, int *old_folio_state, struct anon_vma **anon_vmap) { - unsigned long private =3D (unsigned long)dst->private; + unsigned long info =3D dst->migrate_info; =20 - *anon_vmap =3D (struct anon_vma *)(private & ~FOLIO_OLD_STATES); - *old_folio_state =3D private & FOLIO_OLD_STATES; - dst->private =3D NULL; + *anon_vmap =3D (struct anon_vma *)(info & ~FOLIO_OLD_STATES); + *old_folio_state =3D info & FOLIO_OLD_STATES; + dst->migrate_info =3D 0; } =20 /* Restore the source folio to the original state upon failure */ @@ -1217,7 +1217,7 @@ static int migrate_folio_unmap(new_folio_t get_new_fo= lio, return -ENOMEM; *dstp =3D dst; =20 - dst->private =3D NULL; + dst->migrate_info =3D 0; =20 if (!folio_trylock(src)) { if (mode =3D=3D MIGRATE_ASYNC) --=20 2.43.0 From nobody Wed Jun 17 01:50:58 2026 Received: from CH1PR05CU001.outbound.protection.outlook.com (mail-northcentralusazon11010062.outbound.protection.outlook.com [52.101.193.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2015E3CA4A3 for ; Tue, 28 Apr 2026 15:53:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.193.62 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391598; cv=fail; b=Py1kckK2BGZpGPhcjcfONbJjx71un4dZn8EplTDxfptqeyNd4CJZ+r5V2tQZV7G8VH2Vs8fnj1DxVl/Y4U7ezRJC7huBkhLpg54/zeYFcffFVtk5LfUQmPFmI+uMN7EwjsSuuiNGV4dIOspLkSqKkxRdKfgVkKGigL4Pdn0/bX4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391598; c=relaxed/simple; bh=fT536E9fdileuXnyedNekPspqpGJVWHb5qcLWwgYA7Y=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=RlePEh0GbLVFETnW4EP6KJOrZX35zXV+SIswvuV0m/OCp4wgYpR+ouNFIf5ZSb0GIrbYgnUHiZYv2VLMw/epZV6+Rv98IGa/Apgs+PZzSKAqBR4QD2xg/H6+2WAJbPp7EH0jg5GiF3JDXHBN8eAFnf0mPi92l6X2mvdYXWW70UY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=TWKdWPvP; arc=fail smtp.client-ip=52.101.193.62 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="TWKdWPvP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vi+92PYisdE+z+r3TabivKsF77UnyCIuynjkquaWCJIpkFvhhsU+Ul3wN4hAVJ+D99jM/edBwXGhlSlUka3vg/rkY7LhBt5+1Ekqp9mpeC6jinY6waQ+w/aC6ZZslcKm2GR5vtRsNK0QraBAv9vd+xNjmG2wtO1lIoousicAr7O4gyhxYx+zmo9K0/xBMKpaPmOnxsEPSreTAcdAbrXn8Pf2AmnnxCWQ7/GxWAHLVJHEcLllhI6WAChshFpfm91q3wWp2oxotOS3wRFDHboy0tZ+raj5nN5DXzie6J6uwoVPknVxDmN/xJi/H9LsC9IMyrg1Bg0jkqLF5ALS/06tzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8KL+QP1qDwV2Jdt/D3PbFBW6EvKUJ05LbyqoktAEQxA=; b=r9s66Guv/Hm649GJxfD09nn0KgtAYXjIJ5hC98lzugyhpVZuaBK3Jo0qnDPBL2gtAhcyf6l/ddoNRqUojRPIA8irp+9Ijckk9tM7MWXrYANPOKKuO9Hv7Ylp0J55653qIg8O+OpBA9qULLwk9yDSk94mToQZ7i5qdMNfCWVDmQrNKY+j3lh779nJQ2cBMiME4lLD1sP9P9VIPAWjP3GsOWYeSLPL8qp0iBCXh1hxj+cF5Rxk8Ky69361GisxmpuMMGdBJkRqtZSKLw2NzGVrhJN0/2VQuzGjv0yZNEGno+KgG7E1p8pBwFAPiyMTMsGaM0pkcV99/9kAEdMeqxWgUg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8KL+QP1qDwV2Jdt/D3PbFBW6EvKUJ05LbyqoktAEQxA=; b=TWKdWPvPnkio/LrDs2vzf9JVZ10zj3LeOuR6G7mWL2XDqxnhIC27BP2K34gXOtLngl0XzLZTaGQs2P+5TWXmm7BRt5EkWbBi2R5XcvHqHwEknBoETLftrepYSz1MAa7GwZUApqApXEl6BI6E8YT+cbvJF1XT28rq3j9vlQ5ThpU= Received: from MW4P221CA0005.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::10) by DM6PR12MB4465.namprd12.prod.outlook.com (2603:10b6:5:28f::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.17; Tue, 28 Apr 2026 15:53:06 +0000 Received: from SJ1PEPF000026C5.namprd04.prod.outlook.com (2603:10b6:303:8b:cafe::cf) by MW4P221CA0005.outlook.office365.com (2603:10b6:303:8b::10) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.28 via Frontend Transport; Tue, 28 Apr 2026 15:53:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by SJ1PEPF000026C5.mail.protection.outlook.com (10.167.244.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Tue, 28 Apr 2026 15:53:06 +0000 Received: from kaveri.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 28 Apr 2026 10:52:54 -0500 From: Shivank Garg To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Shivank Garg Subject: [PATCH 3/7] mm/migrate: skip data copy for already-copied folios Date: Tue, 28 Apr 2026 15:50:43 +0000 Message-ID: <20260428155043.39251-8-shivankg@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260428155043.39251-2-shivankg@amd.com> References: <20260428155043.39251-2-shivankg@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000026C5:EE_|DM6PR12MB4465:EE_ X-MS-Office365-Filtering-Correlation-Id: 195495c4-9261-4a2a-ec13-08dea53e3cf3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|376014|7416014|1800799024|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: 4Umv66NnVeJoHfKG+jQOjGkqwkFR4nihm0Mc2nOa2wvXfkrL136aQQLKyMGhNnEBbJ3NP9wu6o0Bi81oIJ78Ap0WgiEQTGj3lYxOMe4/Edbx7wsase9bcvUXAHEX76LfPzWuljVtEy5Ms6PtHMvsxJ8uJomVDEH9foLkxLxDg5JkTbqYN2Z6/QaQfLa0wP6UVgRCN1xIaIBMO2CYmkLNOLFuQTDx+dIw62QFKhY3B1ec+EWwZvtdNE6w32FC3pbjjrqZy5sxDPMA40vw1pIo/ll71Og6/4lJbqgDLA7rlvSD2aSi7cijFZ3TCSwfiKiBA6s3CcSmSnZwFIyXOo9hbSNHIeXSvJzNsv32Cuw/X/e7TCPoIphFIpUB5lS+3lxXmalxV+MRvyl6Rx8Wlroq0ChE+onh2eg2ZKayoFDj8DMxE+a8/hQaMdW5Fqlf/qfd0ORpIDuiUoUOjeZ+Ut4TRvhGoDycBlrJ9l6SEBpN+h6eZW3spQztZrX+gMeNY0xnDiEnd030c+HMyMRqew1Y5WwSqCdy8/xdOfhXFx135DVVDs3E+9YyXsziSC/xexLOFAn7WCuKu6ZsW2nkfnJ9Hn+XG0OUbkVhU3z8HywQ3I0e8xojXc5+FNG5srfO3Qk+r57d9RcYv7bn+A0trKG6KbiFVtAOuj3Ovlihlth7dSoTHuUAuNUe61HWKC7xYaqoSRa8JhyZCtq2e4dX7CuT/Acc4bJ5S8UNNKA2Vb2dOV7/fXbZOjydiJdEYe1J0+ZZuneqTqcBWTmGwO5EGyGAwA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(376014)(7416014)(1800799024)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QRIMIhXAh5wSW3vBCc/qAwF4h+UpOe4U8O/NAHXQW1bR00jFdlVMzY6KD3fsH+GBAzPGlQBU8Sbq7Oht3udynIsiIqYjmBpYayq67YarX5aC8coq1mb0dgpqZemAjr7OqNBZ/Z3qsbkk0VxH1jCTqnPdWW/LsRA3nAAY7+xsRtLgq1brd9rN8kUbDbcIFdFOZXn+LavQ4TlF7J76fVYE5wjUeJrEnSRHp/gSQtWvvLFyKKIeYOI7ijB8v62eOKK5ZU/LngXpGLPLRekm49oCRsVw5WdF0N+vIB3ypP+U+0cvltQHc0zohhBrHBMXxSXt2rJQACKyEyM2olsyxbjJ8ilc72KHCWrje1qDMIMUZCxDQkxOfssUGUrT7FH0t5teqQ1HgN3O6UM7CraOrWsNPu+3iyAyVF4VIdOyLHrAcJixPgNys7OD4ugvccwHCtMW X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2026 15:53:06.0968 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 195495c4-9261-4a2a-ec13-08dea53e3cf3 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000026C5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4465 Content-Type: text/plain; charset="utf-8" Add a FOLIO_ALREADY_COPIED flag to the dst->migrate_info migration state. When set, __migrate_folio() skips folio_mc_copy() and performs metadata-only migration. All callers currently pass already_copied=3Dfalse. The batch-copy path enables it later in a subsequent patch. Move the dst->migrate_info state enum earlier in the file so __migrate_folio() and move_to_new_folio() can see FOLIO_ALREADY_COPIED. Signed-off-by: Shivank Garg --- mm/migrate.c | 53 +++++++++++++++++++++++++++++++--------------------- 1 file changed, 32 insertions(+), 21 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 03c2a6f7e5e4..c493e67e359d 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -850,6 +850,19 @@ void folio_migrate_flags(struct folio *newfolio, struc= t folio *folio) } EXPORT_SYMBOL(folio_migrate_flags); =20 +/* + * To record some information during migration, we use the migrate_info + * field of struct folio of the newly allocated destination folio. + * This is safe because nobody is using it except us. + */ +enum { + FOLIO_WAS_MAPPED =3D BIT(0), + FOLIO_WAS_MLOCKED =3D BIT(1), + FOLIO_ALREADY_COPIED =3D BIT(2), + FOLIO_OLD_STATES =3D FOLIO_WAS_MAPPED | FOLIO_WAS_MLOCKED | + FOLIO_ALREADY_COPIED, +}; + /************************************************************ * Migration functions ***********************************************************/ @@ -859,14 +872,20 @@ static int __migrate_folio(struct address_space *mapp= ing, struct folio *dst, enum migrate_mode mode) { int rc, expected_count =3D folio_expected_ref_count(src) + 1; + bool already_copied =3D (dst->migrate_info & FOLIO_ALREADY_COPIED); + + if (already_copied) + dst->migrate_info =3D 0; =20 /* Check whether src does not have extra refs before we do more work */ if (folio_ref_count(src) !=3D expected_count) return -EAGAIN; =20 - rc =3D folio_mc_copy(dst, src); - if (unlikely(rc)) - return rc; + if (!already_copied) { + rc =3D folio_mc_copy(dst, src); + if (unlikely(rc)) + return rc; + } =20 rc =3D __folio_migrate_mapping(mapping, dst, src, expected_count); if (rc) @@ -1090,7 +1109,7 @@ static int fallback_migrate_folio(struct address_spac= e *mapping, * 0 - success */ static int move_to_new_folio(struct folio *dst, struct folio *src, - enum migrate_mode mode) + enum migrate_mode mode, bool already_copied) { struct address_space *mapping =3D folio_mapping(src); int rc =3D -EAGAIN; @@ -1098,6 +1117,9 @@ static int move_to_new_folio(struct folio *dst, struc= t folio *src, VM_BUG_ON_FOLIO(!folio_test_locked(src), src); VM_BUG_ON_FOLIO(!folio_test_locked(dst), dst); =20 + if (already_copied) + dst->migrate_info =3D FOLIO_ALREADY_COPIED; + if (!mapping) rc =3D migrate_folio(mapping, dst, src, mode); else if (mapping_inaccessible(mapping)) @@ -1129,17 +1151,6 @@ static int move_to_new_folio(struct folio *dst, stru= ct folio *src, return rc; } =20 -/* - * To record some information during migration, we use the migrate_info - * field of struct folio of the newly allocated destination folio. - * This is safe because nobody is using it except us. - */ -enum { - FOLIO_WAS_MAPPED =3D BIT(0), - FOLIO_WAS_MLOCKED =3D BIT(1), - FOLIO_OLD_STATES =3D FOLIO_WAS_MAPPED | FOLIO_WAS_MLOCKED, -}; - static void __migrate_folio_record(struct folio *dst, int old_folio_state, struct anon_vma *anon_vma) { @@ -1353,7 +1364,7 @@ static int migrate_folio_unmap(new_folio_t get_new_fo= lio, static int migrate_folio_move(free_folio_t put_new_folio, unsigned long pr= ivate, struct folio *src, struct folio *dst, enum migrate_mode mode, enum migrate_reason reason, - struct list_head *ret) + struct list_head *ret, bool already_copied) { int rc; int old_folio_state =3D 0; @@ -1379,7 +1390,7 @@ static int migrate_folio_move(free_folio_t put_new_fo= lio, unsigned long private, src_partially_mapped =3D folio_test_partially_mapped(src); } =20 - rc =3D move_to_new_folio(dst, src, mode); + rc =3D move_to_new_folio(dst, src, mode, already_copied); if (rc) goto out; =20 @@ -1536,7 +1547,7 @@ static int unmap_and_move_huge_page(new_folio_t get_n= ew_folio, } =20 if (!folio_mapped(src)) - rc =3D move_to_new_folio(dst, src, mode); + rc =3D move_to_new_folio(dst, src, mode, false); =20 if (page_was_mapped) remove_migration_ptes(src, !rc ? dst : src, ttu); @@ -1720,7 +1731,7 @@ static void migrate_folios_move(struct list_head *src= _folios, struct list_head *ret_folios, struct migrate_pages_stats *stats, int *retry, int *thp_retry, int *nr_failed, - int *nr_retry_pages) + int *nr_retry_pages, bool already_copied) { struct folio *folio, *folio2, *dst, *dst2; bool is_thp; @@ -1737,7 +1748,7 @@ static void migrate_folios_move(struct list_head *src= _folios, =20 rc =3D migrate_folio_move(put_new_folio, private, folio, dst, mode, - reason, ret_folios); + reason, ret_folios, already_copied); /* * The rules are: * 0: folio will be freed @@ -1994,7 +2005,7 @@ static int migrate_pages_batch(struct list_head *from, migrate_folios_move(&unmap_folios, &dst_folios, put_new_folio, private, mode, reason, ret_folios, stats, &retry, &thp_retry, - &nr_failed, &nr_retry_pages); + &nr_failed, &nr_retry_pages, false); } nr_failed +=3D retry; stats->nr_thp_failed +=3D thp_retry; --=20 2.43.0 From nobody Wed Jun 17 01:50:58 2026 Received: from BN8PR05CU002.outbound.protection.outlook.com (mail-eastus2azon11011058.outbound.protection.outlook.com [52.101.57.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6CBAF38F247 for ; Tue, 28 Apr 2026 15:53:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.57.58 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391621; cv=fail; b=DfK2QJ+oEDVlojEyzYfIFwM0n67K4jAfGicrzJvv4AESSuS6KsMU0hRCNmRRlKEfSO7huDIY6J1+bGXK44acKeoJorAXsisc5Nq8f16o4G01nd0vqV0ckY6P/O6GsIs48MqjjF7qX2oFPziKPnecoT66/LIQF2mebtVc+qSaxoY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391621; c=relaxed/simple; bh=FRwY6VcepLEgLasA3Ok97t5lVBUMpsCeaL6KgvE6uPw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=jc4VzStwzV7Kthvou/YG2p4ZAutWtQX+uXByiABPt+Pq9srwOMsXwNLN0b82ic+wuvlKXf2HMbVQ11rZyiH8QqSLFOFVc4RBUV10/zCPNM1NFcF1qUvQ4D0OKemTZzi8FluQdeIs5FGWoVI3CwO7c6tk7J2jZqKuoseaq6F0fxI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=FHWBosiJ; arc=fail smtp.client-ip=52.101.57.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="FHWBosiJ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=d/CipOQbAiKjAjrC42R3MM3lJqUAjabVka6hXTxbtOymTLYT1D2nhcGbAbAQ3ffwaTUq5dnySc5kVdDaGjvfTu+QjbBs3OaunMrWTHkuQ2ZY/7+JJEa3ueylPAbgcNLeVSG1hmzEDw14qybBeDREJTmEa4sHjSewZUEoXNzANgCBlIdQGzeNwW/h4t7vX3SpSwvjvjdVXDBZFKecwYZKs34KuV/N0frgSeb0FW+p2fwaeJE813YdIfiWz50FK/7Le31mtdW3dmr+wnDiBVuWLyZJQpDql5oE8FisCKivGevWgOr0fOdemRjmvA31vsN0mzKFWfCGK1NjX0ueSdzDcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tRnnMzgr6c+7+Zgh/jXAbYOfZG0nWIYRW9JUzke/+x4=; b=RpoBjcitGSH1EMIBl/vxtoWo7akhIpJz8JoBYq1KLClLbohOCVWx0nlCjfKEkiRi/FZyfLnWQWrvjcMjY3+XwdtytYCvrMc603VQRw3zdSomYvGPjBC610DoSRGhL7vLNqVtBxLxBLrZitzTJGMpqPMEqkhUoveaISSRYQuFiMbP5eLu1mD0U5CFYySGvQ2RNtOSq8tDUsd0m0aVWV11uZrfhDRpxIyalWP6f2nF5rv4WHgsnu/wiAnpCpTdAZ+pF6oohZDlwXj2zMpbXBBsPRCc8JG6kVan/GV/g0uOQgZxVsxpY1E9MWfOooDaImHHRPj7a0dkmvaBRU37Mb2+vg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tRnnMzgr6c+7+Zgh/jXAbYOfZG0nWIYRW9JUzke/+x4=; b=FHWBosiJLzdD7lyRy1C1aG8u52+2F/KxTdzCXbAxvUXRcdJAWCLcsC7RV11Q+Q1fu6xsTjCY6phQ3YfClfsZ9ZmCQtk74trAmnMc0e6MP129P+GWaRuESQ2hR6N0kj19d7Nswh2XYmWOVNCm+gFqY7n1AwnRihjlc8EJXxOvFbI= Received: from MN2PR01CA0029.prod.exchangelabs.com (2603:10b6:208:10c::42) by DS0PR12MB6557.namprd12.prod.outlook.com (2603:10b6:8:d3::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.18; Tue, 28 Apr 2026 15:53:34 +0000 Received: from MN1PEPF0000F0E1.namprd04.prod.outlook.com (2603:10b6:208:10c:cafe::f6) by MN2PR01CA0029.outlook.office365.com (2603:10b6:208:10c::42) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.26 via Frontend Transport; Tue, 28 Apr 2026 15:53:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MN1PEPF0000F0E1.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Tue, 28 Apr 2026 15:53:33 +0000 Received: from kaveri.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 28 Apr 2026 10:53:23 -0500 From: Shivank Garg To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Shivank Garg Subject: [PATCH 4/7] mm/migrate: add batch-copy path in migrate_pages_batch Date: Tue, 28 Apr 2026 15:50:45 +0000 Message-ID: <20260428155043.39251-10-shivankg@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260428155043.39251-2-shivankg@amd.com> References: <20260428155043.39251-2-shivankg@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E1:EE_|DS0PR12MB6557:EE_ X-MS-Office365-Filtering-Correlation-Id: 13d2c0be-c7eb-4ec3-cb28-08dea53e4d7e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|7416014|376014|1800799024|22082099003|18002099003|18096099003|56012099003; X-Microsoft-Antispam-Message-Info: guZFqto26DkCopFTcRxpiX8nKE8tsxqJIqAF8YdhA/jQ8oJl7LHMb+grZmT495BuX7KsueqGl2pUhp1lgmWAVzlviG2Bz9KCNSAte8C15jaOxC3U6SJYlib1HgQhT+IbJGPIja0pydRo1/L1y+kU9iQb7Yd9iIczf318xr/d8JnRkzRk3thXz4xbSN3oVAMTXYiTRnhgCoWX7f0G4EbmBLcg+XMjOAqcRKEPSBnFm3M5MYUE9EGIvtuSeJnA5G/osGCDJb/U97MHpKqsxZ9WZqr6tcDBNkAY0DCaAWj6Bm7okM98k8kh7MS/i3yW6H09fjIVuhf+UUzRe/IzN4IqRg1YVN7uGwi3WJ64K7/9aXzeBa3oTcyWcpaMqjZ/MYk5LoAD+B5qO5qFdR6WQcWrJSHrtYsnv/uI5e1sF9usxbVTImDpgxX0a6PW+gxsF3btl/H+OGAsCK/32EzYWmreeGStVBVoMb9BAAPUyNzOQUZo5QWrWzOC7nz2ppeeVFCZ85H6+TgGc5tDuGtefp8CP5DYuI5STyUnoYOdMC5iB9hX9/rawndG6I6tTJ/5UiyseginIUOD+oLhP5He97DhGVz3hzrWp+PMxgH4IaEmOrUarU/w31sa7XF8esto7pvY0jD5JZOoCDaT41WIH4BRm1YN9+k/SNDT+YtxHcB/epRrn4Zlg1j5poC3khioUuJaNwoaRspvb/quw9vYM3ZOblv6Yudmzq61XfVCNdHL9ZMLsOLLZoPnbov8hdUPyohrjW0Gd4GsX6TefDyxrDvYZQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(7416014)(376014)(1800799024)(22082099003)(18002099003)(18096099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: YN/O/J4/5agNb9fdu8/2uSC8mfB9MojehMKblSn/+203uKPWmF1nyUzp8miu3mVhTmh8v9Rq5RStsPyGCL1MjeSR2O9pKF/C3FQxxvGWgm+UO+B01eG92pxMzNLfljiNW8kzM88mBbSbpi5hLO2eeyAOY2xHLt5HK6tnIPfpNA1I7EGwMxsR9KNsWLtwatThG2RN9plhqk5cDSYXR+s9GwgXjsHb0IQd3M+1xJyEqBDYnCNfCaBfNgg15CSnSDSzn6tCyiplrGqSP46YMReuXiYAqO/zvYIBHAAWVB49gx1FLuG7v2MrfAyiCCSYY5HZnbT5gwulR6RTF3NzdQ6AaQr68aEthC48jRGHYWL9LqSY5cKLFwDYtQEQNunP3bO4IwzxavStlwKcNXYuv9b0yFeKHjuoMDC1GdGuZgyKR4kb5UqISwCdgNyLnJZDKgmS X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2026 15:53:33.9458 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 13d2c0be-c7eb-4ec3-cb28-08dea53e4d7e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E1.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB6557 Content-Type: text/plain; charset="utf-8" Add folios_mc_copy() which walks list of src and dst folios in lockstep, and copies folio content via folio_mc_copy(). folios_cnt parameter is unused here, but is part of the offload_copy callback signature used by later patches in the series. Split unmapped folios into batch-eligible (unmap_batch/dst_batch) and standard (unmap_single/dst_single) lists, gated by the migrate_offload_enabled which is off by default. So, when no offload driver is active, the branch is never taken and everything goes through the standard path. After TLB flush, batch copy the eligible folios via folios_mc_copy() and pass already_copied=3Dtrue into migrate_folios_move() so __migrate_folio() skips the per-folio copy. On batch copy failure, already_copied flag stays false and each folio fall back to individual copy. Signed-off-by: Shivank Garg --- include/linux/mm.h | 2 ++ mm/migrate.c | 61 +++++++++++++++++++++++++++++++++++----------- mm/util.c | 30 +++++++++++++++++++++++ 3 files changed, 79 insertions(+), 14 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 0b776907152e..e6ab9bc3de8f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1917,6 +1917,8 @@ void __folio_put(struct folio *folio); void split_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); int folio_mc_copy(struct folio *dst, struct folio *src); +int folios_mc_copy(struct list_head *dst_list, struct list_head *src_list, + unsigned int __always_unused folios_cnt); =20 unsigned long nr_free_buffer_pages(void); =20 diff --git a/mm/migrate.c b/mm/migrate.c index c493e67e359d..6c2f1cb66f96 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -43,6 +43,7 @@ #include #include #include +#include =20 #include =20 @@ -51,6 +52,8 @@ #include "internal.h" #include "swap.h" =20 +DEFINE_STATIC_KEY_FALSE(migrate_offload_enabled); + static const struct movable_operations *offline_movable_ops; static const struct movable_operations *zsmalloc_movable_ops; =20 @@ -1724,6 +1727,12 @@ static int migrate_hugetlbs(struct list_head *from, = new_folio_t get_new_folio, return nr_failed; } =20 +/* movable_ops folios have their own migrate path */ +static bool folio_supports_batch_copy(struct folio *folio) +{ + return likely(!page_has_movable_ops(&folio->page)); +} + static void migrate_folios_move(struct list_head *src_folios, struct list_head *dst_folios, free_folio_t put_new_folio, unsigned long private, @@ -1752,7 +1761,7 @@ static void migrate_folios_move(struct list_head *src= _folios, /* * The rules are: * 0: folio will be freed - * -EAGAIN: stay on the unmap_folios list + * -EAGAIN: stay on the src_folios list * Other errno: put on ret_folios list */ switch (rc) { @@ -1823,8 +1832,12 @@ static int migrate_pages_batch(struct list_head *fro= m, bool is_large =3D false; struct folio *folio, *folio2, *dst =3D NULL; int rc, rc_saved =3D 0, nr_pages; - LIST_HEAD(unmap_folios); - LIST_HEAD(dst_folios); + unsigned int nr_batch =3D 0; + bool batch_copied =3D false; + LIST_HEAD(unmap_batch); + LIST_HEAD(dst_batch); + LIST_HEAD(unmap_single); + LIST_HEAD(dst_single); bool nosplit =3D (reason =3D=3D MR_NUMA_MISPLACED); =20 VM_WARN_ON_ONCE(mode !=3D MIGRATE_ASYNC && @@ -1919,8 +1932,8 @@ static int migrate_pages_batch(struct list_head *from, private, folio, &dst, mode, ret_folios); /* * The rules are: - * 0: folio will be put on unmap_folios list, - * dst folio put on dst_folios list + * 0: folio put on unmap_batch or unmap_single, + * dst folio put on dst_batch or dst_single * -EAGAIN: stay on the from list * -ENOMEM: stay on the from list * Other errno: put on ret_folios list @@ -1961,7 +1974,7 @@ static int migrate_pages_batch(struct list_head *from, /* nr_failed isn't updated for not used */ stats->nr_thp_failed +=3D thp_retry; rc_saved =3D rc; - if (list_empty(&unmap_folios)) + if (list_empty(&unmap_batch) && list_empty(&unmap_single)) goto out; else goto move; @@ -1971,8 +1984,15 @@ static int migrate_pages_batch(struct list_head *fro= m, nr_retry_pages +=3D nr_pages; break; case 0: - list_move_tail(&folio->lru, &unmap_folios); - list_add_tail(&dst->lru, &dst_folios); + if (static_branch_unlikely(&migrate_offload_enabled) && + folio_supports_batch_copy(folio)) { + list_move_tail(&folio->lru, &unmap_batch); + list_add_tail(&dst->lru, &dst_batch); + nr_batch++; + } else { + list_move_tail(&folio->lru, &unmap_single); + list_add_tail(&dst->lru, &dst_single); + } break; default: /* @@ -1995,17 +2015,28 @@ static int migrate_pages_batch(struct list_head *fr= om, /* Flush TLBs for all unmapped folios */ try_to_unmap_flush(); =20 + /* Batch-copy eligible folios before the move phase */ + if (!list_empty(&unmap_batch)) { + rc =3D folios_mc_copy(&dst_batch, &unmap_batch, nr_batch); + batch_copied =3D (rc =3D=3D 0); + } + retry =3D 1; for (pass =3D 0; pass < nr_pass && retry; pass++) { retry =3D 0; thp_retry =3D 0; nr_retry_pages =3D 0; =20 - /* Move the unmapped folios */ - migrate_folios_move(&unmap_folios, &dst_folios, - put_new_folio, private, mode, reason, - ret_folios, stats, &retry, &thp_retry, - &nr_failed, &nr_retry_pages, false); + if (!list_empty(&unmap_batch)) + migrate_folios_move(&unmap_batch, &dst_batch, put_new_folio, + private, mode, reason, ret_folios, stats, + &retry, &thp_retry, &nr_failed, + &nr_retry_pages, batch_copied); + if (!list_empty(&unmap_single)) + migrate_folios_move(&unmap_single, &dst_single, put_new_folio, + private, mode, reason, ret_folios, stats, + &retry, &thp_retry, &nr_failed, + &nr_retry_pages, false); } nr_failed +=3D retry; stats->nr_thp_failed +=3D thp_retry; @@ -2014,7 +2045,9 @@ static int migrate_pages_batch(struct list_head *from, rc =3D rc_saved ? : nr_failed; out: /* Cleanup remaining folios */ - migrate_folios_undo(&unmap_folios, &dst_folios, + migrate_folios_undo(&unmap_batch, &dst_batch, + put_new_folio, private, ret_folios); + migrate_folios_undo(&unmap_single, &dst_single, put_new_folio, private, ret_folios); =20 return rc; diff --git a/mm/util.c b/mm/util.c index 232c3930a662..77eeb285def1 100644 --- a/mm/util.c +++ b/mm/util.c @@ -778,6 +778,36 @@ int folio_mc_copy(struct folio *dst, struct folio *src) } EXPORT_SYMBOL(folio_mc_copy); =20 +/** + * folios_mc_copy - Copy the contents of list of folios. + * @dst_list: destination folio list. + * @src_list: source folio list. + * @folios_cnt: unused here, present for callback signature compatibility. + * + * Walks list of src and dst folios in lockstep and copies folio + * content via folio_mc_copy(). The caller must ensure both lists have + * the same number of entries. This may sleep. + * + * Return: 0 on success, negative errno on failure. + */ +int folios_mc_copy(struct list_head *dst_list, struct list_head *src_list, + unsigned int __always_unused folios_cnt) +{ + struct folio *src, *dst; + int ret; + + dst =3D list_first_entry(dst_list, struct folio, lru); + list_for_each_entry(src, src_list, lru) { + ret =3D folio_mc_copy(dst, src); + if (ret) + return ret; + dst =3D list_next_entry(dst, lru); + } + + return 0; +} +EXPORT_SYMBOL_GPL(folios_mc_copy); + int sysctl_overcommit_memory __read_mostly =3D OVERCOMMIT_GUESS; static int sysctl_overcommit_ratio __read_mostly =3D 50; static unsigned long sysctl_overcommit_kbytes __read_mostly; --=20 2.43.0 From nobody Wed Jun 17 01:50:58 2026 Received: from PH0PR06CU001.outbound.protection.outlook.com (mail-westus3azon11011050.outbound.protection.outlook.com [40.107.208.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7096C3C1973 for ; Tue, 28 Apr 2026 15:54:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.208.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391666; cv=fail; b=XrAiO2pcI/Wq31AjVbNJnMXNFUB7zNFdAXQI3fjtQ2p2CbRYmKWJ7sXWulgX8pQKU312fZ1b19xgIdvYmD9C51bu7bHchDc57XLdgbuKkhD/UiuPwUOBnvRzwZHDT5uM+uEJu27wvWsJElRohkNCcWFE6qqP28gWqwJhneT7wUg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391666; c=relaxed/simple; bh=nUsTCoeyfbKl1XTS5ddQErKxBMek4VAW+NuyJ43vSbE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=AosWs7FpC9qUra9IhfOXQJ9dwub44kbTwAs3SafUTZjSGVR+fjen42w/8XdkypLbdmyQDhEZv/5dgJnmLjlTBKJSy4R+HTTJa+QSDbKO4C11svhTdMMFK3YN8AYlech9Qch4n98NVFK81nRcrdHBgppAI2H74Zyyy1H4LZZN20g= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=RJDWg4Xq; arc=fail smtp.client-ip=40.107.208.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="RJDWg4Xq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tk2N2YPRRJkeyLINY0Dgw+YRVDCnH0oZi5J9Ul6QhsYTjl+GPeNw2HCDiNa755+kNausuR3MM7+BH7LQTzfRtRXU9JMpH/ULCjl3OCQMeQr3nXQA/D4YETBYfrTPssP3hMKNWdns2+1JkknN5GKQrcg3PjwO3bc4L75EgVLMRP5QqBbbd0wLNq6wOxDjefNA79n+4alt9vpa99ODj6zihxUCX3BStVWR5Bzm2xABxSwn/eefjfvtSKg4BiITWyLqwjUP2deZqhjXurJ87ll6jp8bc6BbiNpPJKqqFVZOewgQhDtrsypa5VyEvlPWez6rdSPwr8rnS8CmXJBlJDISKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nljV61LSXhzqhBC1urFr4hQnCE/si+5O1JBTyx/owO8=; b=mlcBbRRCnWaH+AzveqUGMyev+PCfjzwu15Ek+JYC+stwfq0sSdyN0aYPc2u01pyFYl5UEyuIRS2C5S6H32+ACY7f06CDRwVl+nGxyOodY+UHVuywvQJIguuqBcvWLtVgTFUIOWJ2g706xwitSXJaNtwRXplQkMIncdGbDcIDjzFZRAoBgO+D9NCNCXafcGpCF6SSHSLEjTP2eRZ/zRvxNu0G5hfNjjKY0wkdhFYZ5jsoITRJavGpAk5WB0rHh6DUiNqMAzCKOXfJWhHps6f+KiJZnzxWHIGWQVHdjuFpua3HC0SlPWNs0BzR/JrJrPUzDxAonj99uZ5z5j1hYWDPRQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nljV61LSXhzqhBC1urFr4hQnCE/si+5O1JBTyx/owO8=; b=RJDWg4Xqs2jAUTs5to12Fhb+8wnOHduzJMusemPjtLYmQCuLvtBL4KRndvt79/ABt3Vt8Eg2/BPVZTgUaVJlixQ327B78Cp41aIx144QbYL5dIylZolwH5hnEH9BtTtjaw891ct+1YD5v92YBxvYPjmGX+bYDRmfGEMIC6K9SNk= Received: from BL1PR13CA0071.namprd13.prod.outlook.com (2603:10b6:208:2b8::16) by CH3PR12MB7715.namprd12.prod.outlook.com (2603:10b6:610:151::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.16; Tue, 28 Apr 2026 15:54:10 +0000 Received: from MN1PEPF0000F0DF.namprd04.prod.outlook.com (2603:10b6:208:2b8:cafe::ec) by BL1PR13CA0071.outlook.office365.com (2603:10b6:208:2b8::16) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.26 via Frontend Transport; Tue, 28 Apr 2026 15:54:10 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MN1PEPF0000F0DF.mail.protection.outlook.com (10.167.242.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Tue, 28 Apr 2026 15:54:09 +0000 Received: from kaveri.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 28 Apr 2026 10:53:56 -0500 From: Shivank Garg To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Shivank Garg , Mike Day Subject: [PATCH 5/7] mm/migrate: add copy offload registration infrastructure Date: Tue, 28 Apr 2026 15:50:47 +0000 Message-ID: <20260428155043.39251-12-shivankg@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260428155043.39251-2-shivankg@amd.com> References: <20260428155043.39251-2-shivankg@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0DF:EE_|CH3PR12MB7715:EE_ X-MS-Office365-Filtering-Correlation-Id: e7e449f8-45ac-431d-565c-08dea53e62f3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|7416014|82310400026|36860700016|376014|56012099003|22082099003|18002099003; X-Microsoft-Antispam-Message-Info: XH6ktHeLnbIFMXhfQn36RfXHuma+Xr7Mf59sdHcUlKE4XvKV6ZJY+vYE7ZgF9Bal6jl/Nu08IjX+4AHAaeFgQn6zmSoUtynBMHgEmgPfLgnJjzYOYInOoiDC481DQABG7OuvJFgxJx9WaP7FZjAnKcvHgcSdKogqMFbVF/jYgUZXQA3BZZZOCUdM9uJzuKONeuqgQasYuOFK2aRvPehWDREVK1dz6fqkGgrRiA9OVVkkeBdVkSkyr6OZ21qKbFyixRugpS+/CD6YUhnhdD+Li2C/EQKz4bWlQ6MMN56cwI8lOVTXEhQ1/qnMEEQJ08pyxEhlE5MklaS4RqaxTiFJ6+SrW/CcJt+PZ8rXUJeZLrIjPV4dh/mVAz71DNBeRUXjQM8EbWQt2Ns7A/6X5EXDFddcJJnOAYl5YvZn8aUUcbyUVNLXDKsbdJemj7L/MI2dZayKyD/YMd/4JNpiE/8+GkzN1sW31pRaMzhtbBKzPg/NbJDwbSFCMC4L7Hd2jrekOpr8VjM5VSsvHjtppLadwahne+qxECVk3RAZzJL9OcvGbbjtm+Ih8Cf8BtDyw3w6ADII82QKp+plCPjDCjENPdGB5cBB+Hr9o4yGs4zH1mIhZjZWB24i9pfG0WJ2Cu77D5T5LvWA0We4rRcti9zswD7QAvR6LrqFICWebENl53etw6u7rhV5VizdfO6itj4lGtloRwTxb+f0NWWsBkcC23WiAePfFFoqKjZKs4IfqvP+61Q7o7g+LcyHku7ahesd6qg2JXAhCYVfWK/MempoEw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(7416014)(82310400026)(36860700016)(376014)(56012099003)(22082099003)(18002099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: qZZDn8hzIxRrMr6Jzp7YyBRcFOi0STRPK8HPTUVamIFOQkkqYxnHI9e0+v92FMrn4vx8hi5YNrVi5PDZjxCrkDH4/UXcpilBiEWkNJTOh6eUpZez3s1OpLggrfj0VquPTH8CDgyKzTdk8JyWqLpF4Q6xxkT1+DI77ehtqHpIyr9gfeamPbO+tFXqqoWZ2ri+CzMzdAkpmiqmlPZpXwXnMHhbPgvT/q9y8i5WXLkvKx+gDva7J/k5mcEpR/I8e/N/6bSPEcrf/uB3fFGs81YOa5Qxu47dy1TAdd7zIzmPrrWkShbAvroIEwI87DhMmSiuOqcJX3mJ1EgJ87YBsSC/fmK8yHYZZpZzJOlLtRKBR/ggkxncH71qhUAS0ThjfOsG9GgE7VuQTNcVnl4Jm3VMcgGZqDzBO8fYT36s1XMq64IgVZivNhMo2Zbm7O2fr+fs X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2026 15:54:09.9422 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e7e449f8-45ac-431d-565c-08dea53e62f3 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0DF.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7715 Content-Type: text/plain; charset="utf-8" Add a registration interface that lets a single offload provider (DMA, multi-threaded CPU copy, etc) take over the batch folio copy performed by migrate_pages_batch(). The provider fills in a struct migrator with an offload_copy() callback and calls migrate_offload_register(). Registration patches the migrate_offload_copy() static_call and flips the migrate_offload_enabled static branch. The migrate_offload_unregister() reverts both. Whether a migration reason is batch-copy eligible is decided by the core in migrate_offload_do_batch(). A migrator may decline a particular batch (e.g. when nr_batch is too small to amortize setup) by returning -EOPNOTSUPP, and the move phase falls back to per-folio CPU copy. Only one migrator can be active at a time. A second registration returns -EBUSY, and only the active migrator can unregister itself. The static_call dispatch is protected by SRCU so that the synchronize_srcu() in unregister waits for all in-flight copy before the module reference is dropped. Co-developed-by: Mike Day Signed-off-by: Mike Day Signed-off-by: Shivank Garg --- include/linux/migrate_copy_offload.h | 44 +++++++++++++ mm/Kconfig | 6 ++ mm/Makefile | 1 + mm/migrate.c | 57 +++++++++++++++-- mm/migrate_copy_offload.c | 94 ++++++++++++++++++++++++++++ 5 files changed, 198 insertions(+), 4 deletions(-) create mode 100644 include/linux/migrate_copy_offload.h create mode 100644 mm/migrate_copy_offload.c diff --git a/include/linux/migrate_copy_offload.h b/include/linux/migrate_c= opy_offload.h new file mode 100644 index 000000000000..d68b10a84743 --- /dev/null +++ b/include/linux/migrate_copy_offload.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_MIGRATE_COPY_OFFLOAD_H +#define _LINUX_MIGRATE_COPY_OFFLOAD_H + +#include +#include +#include +#include + +struct list_head; +struct module; + +#define MIGRATOR_NAME_LEN 32 + +/** + * struct migrator - batch-copy provider for page migration. + * @name: name of the provider. + * @offload_copy: copy @folio_cnt folios from @src_list to @dst_list. + * + * The migrator may inspect @folio_cnt to decide whether the batch + * is worth offloading, e.g. skip when the batch is too small to + * amortize setup cost. If returns error, the core falls back to CPU copy. + * + * @owner: module providing the migrator. + */ +struct migrator { + char name[MIGRATOR_NAME_LEN]; + int (*offload_copy)(struct list_head *dst_list, + struct list_head *src_list, + unsigned int folio_cnt); + struct module *owner; +}; + +#ifdef CONFIG_MIGRATION_COPY_OFFLOAD +extern struct static_key_false migrate_offload_enabled; +extern struct srcu_struct migrate_offload_srcu; +int migrate_offload_register(struct migrator *m); +int migrate_offload_unregister(struct migrator *m); +#else +static inline int migrate_offload_register(struct migrator *m) { return -E= OPNOTSUPP; } +static inline int migrate_offload_unregister(struct migrator *m) { return = -EOPNOTSUPP; } +#endif + +#endif /* _LINUX_MIGRATE_COPY_OFFLOAD_H */ diff --git a/mm/Kconfig b/mm/Kconfig index e8bf1e9e6ad9..325d79619680 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -647,6 +647,12 @@ config MIGRATION config DEVICE_MIGRATION def_bool MIGRATION && ZONE_DEVICE =20 +# Page-migration batch-copy offload infrastructure. +# Selected by migrator drivers (e.g. CONFIG_DCBM_DMA). +config MIGRATION_COPY_OFFLOAD + bool + depends on MIGRATION + config ARCH_ENABLE_HUGEPAGE_MIGRATION bool =20 diff --git a/mm/Makefile b/mm/Makefile index 8ad2ab08244e..db1ac8097089 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -96,6 +96,7 @@ obj-$(CONFIG_FAILSLAB) +=3D failslab.o obj-$(CONFIG_FAIL_PAGE_ALLOC) +=3D fail_page_alloc.o obj-$(CONFIG_MEMTEST) +=3D memtest.o obj-$(CONFIG_MIGRATION) +=3D migrate.o +obj-$(CONFIG_MIGRATION_COPY_OFFLOAD) +=3D migrate_copy_offload.o obj-$(CONFIG_NUMA) +=3D memory-tiers.o obj-$(CONFIG_DEVICE_MIGRATION) +=3D migrate_device.o obj-$(CONFIG_TRANSPARENT_HUGEPAGE) +=3D huge_memory.o khugepaged.o diff --git a/mm/migrate.c b/mm/migrate.c index 6c2f1cb66f96..9af070f9a1f2 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -44,6 +44,8 @@ #include #include #include +#include +#include =20 #include =20 @@ -54,6 +56,51 @@ =20 DEFINE_STATIC_KEY_FALSE(migrate_offload_enabled); =20 +#ifdef CONFIG_MIGRATION_COPY_OFFLOAD +DEFINE_SRCU(migrate_offload_srcu); +DEFINE_STATIC_CALL(migrate_offload_copy, folios_mc_copy); + +static bool migrate_offload_do_batch(int reason) +{ + if (!static_branch_unlikely(&migrate_offload_enabled)) + return false; + + switch (reason) { + case MR_COMPACTION: + case MR_SYSCALL: + case MR_DEMOTION: + case MR_NUMA_MISPLACED: + return true; + default: + return false; + } +} + +static int migrate_offload_batch_copy(struct list_head *dst_batch, + struct list_head *src_batch, + unsigned int nr_batch) +{ + int idx, rc; + + idx =3D srcu_read_lock(&migrate_offload_srcu); + rc =3D static_call(migrate_offload_copy)(dst_batch, src_batch, nr_batch); + srcu_read_unlock(&migrate_offload_srcu, idx); + return rc; +} +#else +static bool migrate_offload_do_batch(int reason) +{ + return false; +} + +static int migrate_offload_batch_copy(struct list_head *dst_batch, + struct list_head *src_batch, + unsigned int nr_batch) +{ + return -EOPNOTSUPP; +} +#endif + static const struct movable_operations *offline_movable_ops; static const struct movable_operations *zsmalloc_movable_ops; =20 @@ -1833,7 +1880,7 @@ static int migrate_pages_batch(struct list_head *from, struct folio *folio, *folio2, *dst =3D NULL; int rc, rc_saved =3D 0, nr_pages; unsigned int nr_batch =3D 0; - bool batch_copied =3D false; + bool do_batch =3D false, batch_copied =3D false; LIST_HEAD(unmap_batch); LIST_HEAD(dst_batch); LIST_HEAD(unmap_single); @@ -1843,6 +1890,8 @@ static int migrate_pages_batch(struct list_head *from, VM_WARN_ON_ONCE(mode !=3D MIGRATE_ASYNC && !list_empty(from) && !list_is_singular(from)); =20 + do_batch =3D migrate_offload_do_batch(reason); + for (pass =3D 0; pass < nr_pass && retry; pass++) { retry =3D 0; thp_retry =3D 0; @@ -1984,8 +2033,7 @@ static int migrate_pages_batch(struct list_head *from, nr_retry_pages +=3D nr_pages; break; case 0: - if (static_branch_unlikely(&migrate_offload_enabled) && - folio_supports_batch_copy(folio)) { + if (do_batch && folio_supports_batch_copy(folio)) { list_move_tail(&folio->lru, &unmap_batch); list_add_tail(&dst->lru, &dst_batch); nr_batch++; @@ -2017,7 +2065,8 @@ static int migrate_pages_batch(struct list_head *from, =20 /* Batch-copy eligible folios before the move phase */ if (!list_empty(&unmap_batch)) { - rc =3D folios_mc_copy(&dst_batch, &unmap_batch, nr_batch); + rc =3D migrate_offload_batch_copy(&dst_batch, &unmap_batch, + nr_batch); batch_copied =3D (rc =3D=3D 0); } =20 diff --git a/mm/migrate_copy_offload.c b/mm/migrate_copy_offload.c new file mode 100644 index 000000000000..6f837c725239 --- /dev/null +++ b/mm/migrate_copy_offload.c @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include +#include +#include + +static DEFINE_MUTEX(migrator_mutex); +static struct migrator *active_migrator; + +DECLARE_STATIC_CALL(migrate_offload_copy, folios_mc_copy); + +/** + * migrate_offload_register - register a batch-copy provider for page migr= ation. + * @m: migrator to install. + * + * Only one provider can be active at a time, returns -EBUSY if another mi= grator + * is already registered. + * + * Return: 0 on success, negative errno on failure. + */ +int migrate_offload_register(struct migrator *m) +{ + int ret =3D 0; + + if (!m || !m->offload_copy || !m->owner) + return -EINVAL; + + mutex_lock(&migrator_mutex); + if (active_migrator) { + ret =3D -EBUSY; + goto unlock; + } + + if (!try_module_get(m->owner)) { + ret =3D -ENODEV; + goto unlock; + } + + static_call_update(migrate_offload_copy, m->offload_copy); + active_migrator =3D m; + static_branch_enable(&migrate_offload_enabled); + +unlock: + mutex_unlock(&migrator_mutex); + + if (ret) + pr_err("migrate_offload: %s: failed to register (%d)\n", + m->name, ret); + else + pr_info("migrate_offload: enabled by %s\n", m->name); + return ret; +} +EXPORT_SYMBOL_GPL(migrate_offload_register); + +/** + * migrate_offload_unregister - unregister the active batch-copy provider. + * @m: migrator to remove (must be the currently active one). + * + * Reverts static_call targets and waits for SRCU grace period so that + * no in-flight migration is still calling the driver functions before + * releasing the module. + * + * Return: 0 on success, negative errno on failure. + */ +int migrate_offload_unregister(struct migrator *m) +{ + struct module *owner; + + mutex_lock(&migrator_mutex); + if (active_migrator !=3D m) { + mutex_unlock(&migrator_mutex); + return -EINVAL; + } + + /* + * Disable the static branch first so new migrate_pages_batch calls + * won't enter the batch copy path. + */ + static_branch_disable(&migrate_offload_enabled); + static_call_update(migrate_offload_copy, folios_mc_copy); + owner =3D active_migrator->owner; + active_migrator =3D NULL; + mutex_unlock(&migrator_mutex); + + /* Wait for all in-flight callers to finish before module_put(). */ + synchronize_srcu(&migrate_offload_srcu); + module_put(owner); + + pr_info("migrate_offload: disabled by %s\n", m->name); + return 0; +} +EXPORT_SYMBOL_GPL(migrate_offload_unregister); --=20 2.43.0 From nobody Wed Jun 17 01:50:58 2026 Received: from BYAPR05CU005.outbound.protection.outlook.com (mail-westusazon11010008.outbound.protection.outlook.com [52.101.85.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2CFAA35A384 for ; Tue, 28 Apr 2026 15:55:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=52.101.85.8 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391718; cv=fail; b=k9sRAvgQwWvxRIt0Y7zUOvnnHH/uuCuPngnMH/E1p4/IPLpBr8pSuGNPDB9ZAB2lFDGHMP2aJgbWuBJvkJ7AYOr7WHbaUdOzimD3rFunUk5MeEVyoCuuiKJ9ZjnD6lWH6hI8Vdkzj2pghwQIsDMHzoIZoF41rpfhOUsKCCaaSPo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391718; c=relaxed/simple; bh=R/d1yJMGz3tEbJD24ysQSQlM0RHJwLW2R2xcW5WNZkA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=E+7Ya5OvIDIdJIX1kiFyRsaanQ4qnmTUIjj92hpnsn0G7CM5Pk7J7Iakqf6107AXPXL6VtkG2QOFral8FQ6/oboK1rH4UfsIjpe6jtYk++DpdIumWLgW9z34qvtFdX6a+qvQife8tSMIanEt6VXY2MZLHcxzrhhXRmQToYc1bGw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=2BLsKEUU; arc=fail smtp.client-ip=52.101.85.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="2BLsKEUU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RMU2mzX1eNkjXFsCRTNvfZ2cJ+LLa2wBpXZmm371hYaBkfHzrX0YtUb9Za9tWlkRUdNDjXFSoJhYbwZgtlWLoHb02vbPaK0pOutej2mx8omfeJXPxlnP1VTuhlhd4iZeImO6kyM3ULykylvglXREIos4LkSznd9IVWw/cKsXLvJuVA14I9IAtHSDmnTgxjBtrl3OJtDXpDFXZ6pdRZWU2UV8SalL/FoF/SkUU6hyj3Az/ZHwqziuP7aAtcyIu8BmW6gBcoxTvbf9qFRNbZXLM/R0JmTiBsiSM83MsXUz7E+vxrhsBAjmh6TpJCtRqfMlIhYhagHyPgXAuL+Fb6jaXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KLV+myZqiJhgtZuZ2oncD5LtNGMPg1yB0LDSwCOUPqQ=; b=AzkWfdAWcGScq3ye+XEm/7+lospG4loAj+ys08l6ubbtk9y3kZ8HYGZXCqTiYgvx9uPTcpW5E8BTkFjgzogLHIqqeVcXsMeXjykWKgEsJQLwVS1Q43fjZSHRV8v7QAvkif7U0DYNihLv/cl0gmo07yJG7yMJKX7FFW/+yyIJhWuLSOm0Nnkc/y4EAtSjvQNVcNmzrGlYyb2/GpTk/34zX4XMY9iVjIZakdDtqyIXkiOQHSghfPKIDdeJuKywLRKeeOFSbhvBvnQnRkpFFCSupj1UnM2hZThxnYvSLiLqIkUDR4RZ2Q7NiHWlxS6l33yUkTBniwVwHz0d2zfq84MG+A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KLV+myZqiJhgtZuZ2oncD5LtNGMPg1yB0LDSwCOUPqQ=; b=2BLsKEUUoJ5MXIc1rJyKuWm5oAsBaYK/RnmqNWzv3kzmWv+4NONKLgD1Z8B9sXnfpDx4FHKf6Ttbk4t5UhH1OdjoB8pltDmtwQyirtyzYUYDK2Pu/i7AY14Qnqv3W++CpSo4Xm8y8eWNIdn/HwSoN/zLYaKeasy4+H1tCXhjFpg= Received: from BL1PR13CA0174.namprd13.prod.outlook.com (2603:10b6:208:2bd::29) by DM6PR12MB4250.namprd12.prod.outlook.com (2603:10b6:5:21a::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.16; Tue, 28 Apr 2026 15:55:09 +0000 Received: from MN1PEPF0000F0E0.namprd04.prod.outlook.com (2603:10b6:208:2bd:cafe::ed) by BL1PR13CA0174.outlook.office365.com (2603:10b6:208:2bd::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.26 via Frontend Transport; Tue, 28 Apr 2026 15:55:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MN1PEPF0000F0E0.mail.protection.outlook.com (10.167.242.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Tue, 28 Apr 2026 15:55:09 +0000 Received: from kaveri.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 28 Apr 2026 10:54:58 -0500 From: Shivank Garg To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Shivank Garg Subject: [PATCH 6/7] drivers/migrate_offload: add DMA batch copy driver (dcbm) Date: Tue, 28 Apr 2026 15:50:49 +0000 Message-ID: <20260428155043.39251-14-shivankg@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260428155043.39251-2-shivankg@amd.com> References: <20260428155043.39251-2-shivankg@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E0:EE_|DM6PR12MB4250:EE_ X-MS-Office365-Filtering-Correlation-Id: a29eb70a-b7fa-4a2d-298a-08dea53e866c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|36860700016|1800799024|82310400026|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: Z/+rMGUAOuO+Iwitu6KmMdkwiAEkLkIagKFfeGq3PfaXmEmBpdnRl4RetNerDY9XCwbr2S8fK2g1O3KmCTeM8RQO9J8LoALaeXDkiQ3jea9EbVyMAlttYUccow9PI4I+oeQ0RAaYHr0O6kDSpXJ1rAMQSO++J96piAjLuEx+73GERLCJUUdsjm4GMcERFoUj0F45Z7h3zSDHIlF3QwiW+zwxk5+vSYx3CKLpuyOFC000/gihTaUyDZn3GrJb+/sj4lw5pQFhHS39QNDHd3e0fb9V57YXx203yWQOPSZ/Jt5RsNl9t5u4EIrGFeMTk+72D65c5kZjI3t7d4/ZD83CVIsX5GWbl84FhpbTJeYJ5T4E94ngiZ6LozDDbhgbqCL1ecF8pY/IFo9E0jSr4iB76ad1QrdyILcpALxjMdqGB+9Mt9qQPwYSwEJ/0yj17bEa2IfMrTYanRH036z04/8vkAW/nkYOj5iopBX1Utt6rPPZy7ENrr9VGjUaFq9qZmLEUVDlbWwX4s/i0rzDB/FEhrYMeWPLRfVI9Ys+ShMA0WUiqX/cv9FxbDGLB+9+wzA6P2G2QeJ0fAWY0oMZsioz+RxIa5EkKIzN778rRYqT0o+yy3J1TaQuSjXr4TVHUmTQd8hfzXY9Sz/oGaDBGhvgJIrGFxuKDOX/z2yOIX+7y4sxIqSSLlHyeueTXsaidzgvcToPIZjywXJCsvNzz6OeC7W0umGAc27y9eljZ+V7rx102yOn3r+C0TKRqjagIHqWKVQ5pU1UqxQ9HSrndfGYtw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(7416014)(376014)(36860700016)(1800799024)(82310400026)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: dggU02yw0L+0mjuzLyf3BqW0Fnc6caot1XUuaag1Js5C0Zm95fz+EOUtlsAMqKHkRJsiMtE1MIQt6G5wixVW2H9Wi3oUBsplP3ZXkN03hFZnaDV4hVNK/FBQO+QrQv5dIdAvunHSn068/dj8jrWbHy6JQmIKqmM9dxrZupQap8mswfXMCUUxKwG7uOrQGeJnEnKJ2REWB8YrnKHWxY13cdloTQfRuyN9EsXMB4CJniA8WNluTOUgVOjjtNq1T1uH3nzlwc8usSkn//p0yARMm9tnqJa/5oY0P5ytHSq8jyWBo5jmgLiNaL2JMUfYZaS80nHhBkwCF3+LciReUZ/JkILxGLuKcb7/2+ddZIUPEmgZ3576giBgoLStNBdRWT+Rb2kLt5cOZHe4IWtAUNYctaBnftHYmi6MTZeFr7sydF7VCUeBONjn+KHXI3HAybas X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2026 15:55:09.4542 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a29eb70a-b7fa-4a2d-298a-08dea53e866c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E0.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4250 Content-Type: text/plain; charset="utf-8" Simple DMAEngine-based migrator that plugs into the page migration copy offload infrastructure to batch-copy folios via DMA memcpy channels. It is intended for testing the offload plumbing and as a template for future migrators (SDXI, multi-threaded CPU copy, etc.). When DMA fails, the callback returns an error and the migration path falls back to per-folio CPU copy. Loading the module exposes attributes under /sys/module/dcbm/: offloading - enable/disable DMA offload nr_dma_chan - max DMA channels to use folios_migrated - folios copied via DMA folios_failures - fallback count CONFIG_DCBM_DMA selects MIGRATION_COPY_OFFLOAD so enabling the driver pulls in the infrastructure automatically. Channel acquisition uses dma_request_chan_by_mask(DMA_MEMCPY), which works for providers that set DMA_PRIVATE (e.g. AMD PTDMA). Generic mem-to-mem engines that do not set DMA_PRIVATE (e.g. SDXI) should acquire channels via dma_find_channel(DMA_MEMCPY) or the async_tx APIs, which can be added in a follow-up. Signed-off-by: Shivank Garg Reviewed-by: Karim Manaouil --- drivers/Kconfig | 2 + drivers/Makefile | 2 + drivers/migrate_offload/Kconfig | 9 + drivers/migrate_offload/Makefile | 1 + drivers/migrate_offload/dcbm/Makefile | 1 + drivers/migrate_offload/dcbm/dcbm.c | 440 ++++++++++++++++++++++++++ 6 files changed, 455 insertions(+) create mode 100644 drivers/migrate_offload/Kconfig create mode 100644 drivers/migrate_offload/Makefile create mode 100644 drivers/migrate_offload/dcbm/Makefile create mode 100644 drivers/migrate_offload/dcbm/dcbm.c diff --git a/drivers/Kconfig b/drivers/Kconfig index f2bed2ddeb66..3e83a1475cbc 100644 --- a/drivers/Kconfig +++ b/drivers/Kconfig @@ -253,4 +253,6 @@ source "drivers/cdx/Kconfig" =20 source "drivers/resctrl/Kconfig" =20 +source "drivers/migrate_offload/Kconfig" + endmenu diff --git a/drivers/Makefile b/drivers/Makefile index 0841ea851847..88cb8e3e88df 100644 --- a/drivers/Makefile +++ b/drivers/Makefile @@ -42,6 +42,8 @@ obj-y +=3D clk/ # really early. obj-$(CONFIG_DMADEVICES) +=3D dma/ =20 +obj-$(CONFIG_MIGRATION_COPY_OFFLOAD) +=3D migrate_offload/ + # SOC specific infrastructure drivers. obj-y +=3D soc/ obj-$(CONFIG_PM_GENERIC_DOMAINS) +=3D pmdomain/ diff --git a/drivers/migrate_offload/Kconfig b/drivers/migrate_offload/Kcon= fig new file mode 100644 index 000000000000..930d8605c15d --- /dev/null +++ b/drivers/migrate_offload/Kconfig @@ -0,0 +1,9 @@ +config DCBM_DMA + tristate "DMA Core Batch Migrator" + depends on MIGRATION && DMA_ENGINE + select MIGRATION_COPY_OFFLOAD + help + DMA-based batch copy engine for page migration. Uses + DMAEngine memcpy channels to offload folio data copies + during migration. Primarily intended for testing the copy + offload infrastructure. diff --git a/drivers/migrate_offload/Makefile b/drivers/migrate_offload/Mak= efile new file mode 100644 index 000000000000..9e16018beb15 --- /dev/null +++ b/drivers/migrate_offload/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_DCBM_DMA) +=3D dcbm/ diff --git a/drivers/migrate_offload/dcbm/Makefile b/drivers/migrate_offloa= d/dcbm/Makefile new file mode 100644 index 000000000000..56ba47cce0f1 --- /dev/null +++ b/drivers/migrate_offload/dcbm/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_DCBM_DMA) +=3D dcbm.o diff --git a/drivers/migrate_offload/dcbm/dcbm.c b/drivers/migrate_offload/= dcbm/dcbm.c new file mode 100644 index 000000000000..893580cb9fac --- /dev/null +++ b/drivers/migrate_offload/dcbm/dcbm.c @@ -0,0 +1,440 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * DMA Core Batch Migrator (DCBM) + * + * Uses DMAEngine memcpy channels to offload batch folio copies during + * page migration. Reference driver meant for testing the offload + * infrastructure. + * + * Copyright (C) 2024-26 Advanced Micro Devices, Inc. + */ + +#include +#include +#include +#include +#include + +#define MAX_DMA_CHANNELS 16 + +static atomic_long_t folios_migrated; +static atomic_long_t folios_failures; + +static bool offloading_enabled; +static unsigned int nr_dma_channels =3D 1; +static DEFINE_MUTEX(dcbm_mutex); + +struct dma_work { + struct dma_chan *chan; + struct completion done; + atomic_t pending; + struct sg_table *src_sgt; + struct sg_table *dst_sgt; + bool mapped; +}; + +static void dma_completion_callback(void *data) +{ + struct dma_work *work =3D data; + + if (atomic_dec_and_test(&work->pending)) + complete(&work->done); +} + +static int setup_sg_tables(struct dma_work *work, struct list_head **src_p= os, + struct list_head **dst_pos, int nr) +{ + struct scatterlist *sg_src, *sg_dst; + struct device *dev; + int i, ret; + + work->src_sgt =3D kmalloc_obj(*work->src_sgt, GFP_KERNEL); + if (!work->src_sgt) + return -ENOMEM; + work->dst_sgt =3D kmalloc_obj(*work->dst_sgt, GFP_KERNEL); + if (!work->dst_sgt) { + ret =3D -ENOMEM; + goto err_free_src; + } + + ret =3D sg_alloc_table(work->src_sgt, nr, GFP_KERNEL); + if (ret) + goto err_free_dst; + ret =3D sg_alloc_table(work->dst_sgt, nr, GFP_KERNEL); + if (ret) + goto err_free_src_table; + + sg_src =3D work->src_sgt->sgl; + sg_dst =3D work->dst_sgt->sgl; + for (i =3D 0; i < nr; i++) { + struct folio *src =3D list_entry(*src_pos, struct folio, lru); + struct folio *dst =3D list_entry(*dst_pos, struct folio, lru); + + sg_set_folio(sg_src, src, folio_size(src), 0); + sg_set_folio(sg_dst, dst, folio_size(dst), 0); + + *src_pos =3D (*src_pos)->next; + *dst_pos =3D (*dst_pos)->next; + + if (i < nr - 1) { + sg_src =3D sg_next(sg_src); + sg_dst =3D sg_next(sg_dst); + } + } + + dev =3D dmaengine_get_dma_device(work->chan); + if (!dev) { + ret =3D -ENODEV; + goto err_free_dst_table; + } + ret =3D dma_map_sgtable(dev, work->src_sgt, DMA_TO_DEVICE, + DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_NO_KERNEL_MAPPING); + if (ret) + goto err_free_dst_table; + ret =3D dma_map_sgtable(dev, work->dst_sgt, DMA_FROM_DEVICE, + DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_NO_KERNEL_MAPPING); + if (ret) + goto err_unmap_src; + + /* + * TODO: IOMMU may merge segments unevenly on the two sides, fall back + * bail to CPU copy. In practice, I have not observed merging in tests. + * Handling unequal nents is left for follow-up. + */ + if (work->src_sgt->nents !=3D work->dst_sgt->nents) { + ret =3D -EINVAL; + goto err_unmap_dst; + } + work->mapped =3D true; + return 0; + +err_unmap_dst: + dma_unmap_sgtable(dev, work->dst_sgt, DMA_FROM_DEVICE, + DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_NO_KERNEL_MAPPING); +err_unmap_src: + dma_unmap_sgtable(dev, work->src_sgt, DMA_TO_DEVICE, + DMA_ATTR_SKIP_CPU_SYNC | DMA_ATTR_NO_KERNEL_MAPPING); +err_free_dst_table: + sg_free_table(work->dst_sgt); +err_free_src_table: + sg_free_table(work->src_sgt); +err_free_dst: + kfree(work->dst_sgt); + work->dst_sgt =3D NULL; +err_free_src: + kfree(work->src_sgt); + work->src_sgt =3D NULL; + return ret; +} + +static void cleanup_dma_work(struct dma_work *works, int actual_channels) +{ + struct device *dev; + int i; + + if (!works) + return; + + for (i =3D 0; i < actual_channels; i++) { + if (!works[i].chan) + continue; + + dev =3D dmaengine_get_dma_device(works[i].chan); + + if (works[i].mapped) + dmaengine_terminate_sync(works[i].chan); + + if (dev && works[i].mapped) { + if (works[i].src_sgt) { + dma_unmap_sgtable(dev, works[i].src_sgt, + DMA_TO_DEVICE, + DMA_ATTR_SKIP_CPU_SYNC | + DMA_ATTR_NO_KERNEL_MAPPING); + sg_free_table(works[i].src_sgt); + kfree(works[i].src_sgt); + } + if (works[i].dst_sgt) { + dma_unmap_sgtable(dev, works[i].dst_sgt, + DMA_FROM_DEVICE, + DMA_ATTR_SKIP_CPU_SYNC | + DMA_ATTR_NO_KERNEL_MAPPING); + sg_free_table(works[i].dst_sgt); + kfree(works[i].dst_sgt); + } + } + dma_release_channel(works[i].chan); + } + kfree(works); +} + +static int submit_dma_transfers(struct dma_work *work) +{ + struct scatterlist *sg_src, *sg_dst; + struct dma_async_tx_descriptor *tx; + unsigned long flags =3D DMA_CTRL_ACK; + dma_cookie_t cookie; + int i; + + atomic_set(&work->pending, 1); + + sg_src =3D work->src_sgt->sgl; + sg_dst =3D work->dst_sgt->sgl; + for_each_sgtable_dma_sg(work->src_sgt, sg_src, i) { + if (i =3D=3D work->src_sgt->nents - 1) + flags |=3D DMA_PREP_INTERRUPT; + + tx =3D dmaengine_prep_dma_memcpy(work->chan, + sg_dma_address(sg_dst), + sg_dma_address(sg_src), + sg_dma_len(sg_src), flags); + if (!tx) { + atomic_set(&work->pending, 0); + return -EIO; + } + + if (i =3D=3D work->src_sgt->nents - 1) { + tx->callback =3D dma_completion_callback; + tx->callback_param =3D work; + } + + cookie =3D dmaengine_submit(tx); + if (dma_submit_error(cookie)) { + atomic_set(&work->pending, 0); + return -EIO; + } + sg_dst =3D sg_next(sg_dst); + } + return 0; +} + +/** + * folios_copy_dma - copy a batch of folios via DMA memcpy + * @dst_list: destination folio list + * @src_list: source folio list + * @nr_folios: number of folios in each list + * + * Return: 0 on success, negative errno on failure. + */ +static int folios_copy_dma(struct list_head *dst_list, + struct list_head *src_list, unsigned int nr_folios) +{ + struct dma_work *works; + struct list_head *src_pos =3D src_list->next; + struct list_head *dst_pos =3D dst_list->next; + int i, folios_per_chan, ret; + dma_cap_mask_t mask; + int actual_channels =3D 0; + unsigned int max_channels; + + max_channels =3D min3(nr_dma_channels, nr_folios, + (unsigned int)MAX_DMA_CHANNELS); + + works =3D kcalloc(max_channels, sizeof(*works), GFP_KERNEL); + if (!works) + return -ENOMEM; + + dma_cap_zero(mask); + dma_cap_set(DMA_MEMCPY, mask); + + for (i =3D 0; i < max_channels; i++) { + works[actual_channels].chan =3D dma_request_chan_by_mask(&mask); + if (IS_ERR(works[actual_channels].chan)) + break; + init_completion(&works[actual_channels].done); + actual_channels++; + } + + if (actual_channels =3D=3D 0) { + kfree(works); + return -ENODEV; + } + + for (i =3D 0; i < actual_channels; i++) { + folios_per_chan =3D nr_folios * (i + 1) / actual_channels - + (nr_folios * i) / actual_channels; + if (folios_per_chan =3D=3D 0) + continue; + + ret =3D setup_sg_tables(&works[i], &src_pos, &dst_pos, + folios_per_chan); + if (ret) + goto err_cleanup; + } + + for (i =3D 0; i < actual_channels; i++) { + ret =3D submit_dma_transfers(&works[i]); + if (ret) + goto err_cleanup; + } + + for (i =3D 0; i < actual_channels; i++) { + if (atomic_read(&works[i].pending) > 0) + dma_async_issue_pending(works[i].chan); + } + + for (i =3D 0; i < actual_channels; i++) { + if (atomic_read(&works[i].pending) =3D=3D 0) + continue; + if (!wait_for_completion_timeout(&works[i].done, + msecs_to_jiffies(10000))) { + ret =3D -ETIMEDOUT; + goto err_cleanup; + } + } + + cleanup_dma_work(works, actual_channels); + + atomic_long_add(nr_folios, &folios_migrated); + return 0; + +err_cleanup: + pr_warn_ratelimited("dcbm: DMA copy failed (%d), falling back to CPU\n", + ret); + cleanup_dma_work(works, actual_channels); + + atomic_long_add(nr_folios, &folios_failures); + return ret; +} + +static struct migrator dma_migrator =3D { + .name =3D "DCBM", + .offload_copy =3D folios_copy_dma, + .owner =3D THIS_MODULE, +}; + +static ssize_t offloading_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%d\n", offloading_enabled); +} + +static ssize_t offloading_store(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + bool enable; + int ret; + + ret =3D kstrtobool(buf, &enable); + if (ret) + return ret; + + mutex_lock(&dcbm_mutex); + + if (enable =3D=3D offloading_enabled) + goto out; + + if (enable) { + ret =3D migrate_offload_register(&dma_migrator); + if (ret) { + mutex_unlock(&dcbm_mutex); + return ret; + } + offloading_enabled =3D true; + } else { + migrate_offload_unregister(&dma_migrator); + offloading_enabled =3D false; + } +out: + mutex_unlock(&dcbm_mutex); + return count; +} + +static ssize_t folios_migrated_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%lu\n", atomic_long_read(&folios_migrated)); +} + +static ssize_t folios_migrated_store(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + atomic_long_set(&folios_migrated, 0); + return count; +} + +static ssize_t folios_failures_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%lu\n", atomic_long_read(&folios_failures)); +} + +static ssize_t folios_failures_store(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + atomic_long_set(&folios_failures, 0); + return count; +} + +static ssize_t nr_dma_chan_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%u\n", nr_dma_channels); +} + +static ssize_t nr_dma_chan_store(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + unsigned int val; + int ret; + + ret =3D kstrtouint(buf, 0, &val); + if (ret) + return ret; + + if (val < 1 || val > MAX_DMA_CHANNELS) + return -EINVAL; + + mutex_lock(&dcbm_mutex); + nr_dma_channels =3D val; + mutex_unlock(&dcbm_mutex); + return count; +} + +static struct kobj_attribute offloading_attr =3D __ATTR_RW(offloading); +static struct kobj_attribute nr_dma_chan_attr =3D __ATTR_RW(nr_dma_chan); +static struct kobj_attribute folios_migrated_attr =3D __ATTR_RW(folios_mig= rated); +static struct kobj_attribute folios_failures_attr =3D __ATTR_RW(folios_fai= lures); + +static struct attribute *dcbm_attrs[] =3D { + &offloading_attr.attr, + &nr_dma_chan_attr.attr, + &folios_migrated_attr.attr, + &folios_failures_attr.attr, + NULL +}; + +static const struct attribute_group dcbm_attr_group =3D { + .attrs =3D dcbm_attrs, +}; + +static int __init dcbm_init(void) +{ + int ret; + + ret =3D sysfs_create_group(&THIS_MODULE->mkobj.kobj, &dcbm_attr_group); + if (ret) + return ret; + + pr_info("dcbm: DMA Core Batch Migrator initialized\n"); + return 0; +} + +static void __exit dcbm_exit(void) +{ + mutex_lock(&dcbm_mutex); + if (offloading_enabled) { + migrate_offload_unregister(&dma_migrator); + offloading_enabled =3D false; + } + mutex_unlock(&dcbm_mutex); + + sysfs_remove_group(&THIS_MODULE->mkobj.kobj, &dcbm_attr_group); + pr_info("dcbm: DMA Core Batch Migrator unloaded\n"); +} + +module_init(dcbm_init); +module_exit(dcbm_exit); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Shivank Garg"); +MODULE_DESCRIPTION("DMA Core Batch Migrator"); --=20 2.43.0 From nobody Wed Jun 17 01:50:58 2026 Received: from SN4PR2101CU001.outbound.protection.outlook.com (mail-southcentralusazon11012040.outbound.protection.outlook.com [40.93.195.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BB513C6A57 for ; Tue, 28 Apr 2026 15:55:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.93.195.40 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391736; cv=fail; b=AEHxhfdc1UD83B6HFKoUwSyV9vac7Byy7GMxxK1ef1NN6GoIb0VDSqFwbd74pgjeMzggquJPgmVdaGZ7Ynd3b8HOc3r1l2kPfKhDWUV7D1turwrTXtCxzVj71YtfErO+IHQbCpDHDbCqHPXKQZCuQG6xAs9c2901Qcl4hVtw+tw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777391736; c=relaxed/simple; bh=9CYHwJr2SPOR0l/SUzVi3eaNvp4Nq8ll+KdjmrjiJas=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QNCUIRGIHeyyFK2m3XiOqX5B2R7KSCLnv+jw5KMd34ZAngTVkAkGdNdgwRvakum9qdAPnhnmT6QXowWgOWX2DsxKbHu7zo9/G1ox/Ca4CTfaXhKD9pYYDM+WX4rF2jWy6xNQuCVYq+RpMYWWBPXUAI674qbDUGn06xzB1trNWJQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=3nAXM7Ge; arc=fail smtp.client-ip=40.93.195.40 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="3nAXM7Ge" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=xScbdTWVHDdGOJpAdZtMPJLq7ZH2W3EUZartltLkc5VtvLOoC1rC1KE6KZ6qV6EmMknc3eeOFeWQCKvCDW3Jz90YILXkkk52z2b9NjN+kBoDtFJGgOwuzbddEXLAef4pIPzklkDOqu1tHMVTYIvnGe3DikoeSm8Ag4oDo+eFa1ptLC1pbmdZN1msH00ibTfYLa9m60VWPnwFpUIStSygkwHalmKwlJxj4BrGme7Y/QY49twmSpBpCQrTya1QdfOL75iCT8SkJneWOQLOsgKCztO5euYfem+Zq1lDJOr50syxYXC4CIgBBeg75Xb2kY+BwlGY4OImN0/6QwBs4Vgyng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aZQWCWPN4P6SLdJrwctFxsaENiyxGoZvFwaflD3cq98=; b=Q7dV3ZWMNGDz8f0PneNYpmzgdYqDoc5vcOFoI0QiDTdJ5qJBSlGNzVWvAsSDIU+ehwnky0msuDfuo1pIxWZy/3E92oQMXfwLFSgFbX884CazIfez2ZfAoawCfE2VZjB5h8sNjKHv//2NKxUZNXKyf0il7L37HQi5LegNfkPkeIsgZwYHsG0+Pj8OfcCWHXFCiYsDBN7tKfzC79/PUGbXWShfibALI1PkOSWZaCW4ah9XF4tLDEB3+rXLKVui9ZFx0Fdn/TIWDUz6fi+AQT2ulQVlMI6xWh5xr8vK7hvWHR29n22lom6qzcBUM7laqx9OU5FIQpUWQhYGG6/G593ekg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=linux-foundation.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aZQWCWPN4P6SLdJrwctFxsaENiyxGoZvFwaflD3cq98=; b=3nAXM7GegHcLZT1cItCp8hRau6kRPEudXhPfKKD8Rz6jR8+u5fdchZYWXsr0mquKVd9Es7Lbkvvo02UOgWpNIm6w1Ce3b/ioWzemmweR6lULhb/QkPG3IrZax4zfIuAl+FdbPEWLyDB2LKst0hf+Dub8o89VO7jPCEIWDTj8zt0= Received: from BL1PR13CA0362.namprd13.prod.outlook.com (2603:10b6:208:2c0::7) by LV2PR12MB5992.namprd12.prod.outlook.com (2603:10b6:408:14e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9870.17; Tue, 28 Apr 2026 15:55:29 +0000 Received: from MN1PEPF0000F0E2.namprd04.prod.outlook.com (2603:10b6:208:2c0:cafe::21) by BL1PR13CA0362.outlook.office365.com (2603:10b6:208:2c0::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9846.26 via Frontend Transport; Tue, 28 Apr 2026 15:55:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=satlexmb07.amd.com; pr=C Received: from satlexmb07.amd.com (165.204.84.17) by MN1PEPF0000F0E2.mail.protection.outlook.com (10.167.242.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9846.18 via Frontend Transport; Tue, 28 Apr 2026 15:55:28 +0000 Received: from kaveri.amd.com (10.180.168.240) by satlexmb07.amd.com (10.181.42.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.17; Tue, 28 Apr 2026 10:55:17 -0500 From: Shivank Garg To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Shivank Garg Subject: [PATCH 7/7] mm/migrate: adjust NR_MAX_BATCHED_MIGRATION for testing Date: Tue, 28 Apr 2026 15:50:51 +0000 Message-ID: <20260428155043.39251-16-shivankg@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260428155043.39251-2-shivankg@amd.com> References: <20260428155043.39251-2-shivankg@amd.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: satlexmb07.amd.com (10.181.42.216) To satlexmb07.amd.com (10.181.42.216) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E2:EE_|LV2PR12MB5992:EE_ X-MS-Office365-Filtering-Correlation-Id: f7e80da6-571d-4c74-1c41-08dea53e9201 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|376014|7416014|1800799024|22082099003|18002099003|56012099003; X-Microsoft-Antispam-Message-Info: wckO9DmWPkYv4D7vDFx4QbziyWlkpD1gHiB5R6GuDTi5lG6VOVfTFyJI3AmXUKFQXWo9jcpIT++C4NAtdQvfnfKmZ24kiNvwRg0V5/QEWFMCzf3xKs32QZ8+DVn84lalPJBRNpIk4zWsQJJHyIe3WnKdGDHAQ2mSNDjQk6mL5GFlnpZVm+X5sFHwu3rLdat61EhhXyjWnMJb5mNoRDHdrFimyR7GIZC4QJMjtnv29JFoUYZGycaEXLphguL97fCGVq0tup7NsB27fGxsAUfmjGNzOqZUF6OcEc49j0E2XivN39L5GviBF9ww1ItIiOkKtriPAMGzBTGZpMYaIOrUmNkisCIqjJ6kwUCPJlyBiv4BqeUjk23V/yB2HnQLXdHYPDBrwC4GKjWKxIuHpxeSYtcbRfMI287EnCKlH2LP6jR5F/b2Lst30bqUpZxIKfFylQ5GM6XikvhliiX/4Sz6akyl2fmWF4B4wYs8Y+61QUqjF3u5spFvKrJnjN6SQgxGFrAtLPFSK5q/nUikHOZ7sBKlfYAmpzm9B85WTLPknBt3SzvywDWYn5CrDtKiaZqSd8QnS0Se5xNgHIV5gv6xI4mo5kV0Ko7x89Qdb97vtGtaW2z1a96L3tdh22lVHr0qufnKkin3Xi15Hx0g7HVvCl+qClRbTR4qVjhpgyj8vdjeATmwWUte/5FikjNuKGXqIl5ySaSCYK8i1rkTXboEBEJfcVcjzlQZBAGEYutROhyndGsX9lbREIPc8/QYHrcGca5yH4dJGR7afWHvfo3Xgg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:satlexmb07.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(376014)(7416014)(1800799024)(22082099003)(18002099003)(56012099003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /eKYAjQselOYYrgSw1Vnp1CtTVZcg4btuLcgIti7GVqC/jQykYQNt/ftwFxpMMMJMkJWmo0Fu+qHikH5Lx7EogXd1xSoyb5TFdF/nk72oGpfY/50PRo0pFlLE9HxNLyDAgFijIILa1NF2voY5mXwfyLpwoqYbEjDZcaSkxXZLOrcjkOPAiZwAds5+5gnRY6guqg/hHrUuoB+af6CLlbVcJHzUFRc0kUUs/Uj1MnJeaXFixwg3rFIEQGdclB4z1a++vF20EtPyGN1PmhO6vvYeWDdPXjSVvhir6exSpb5uYS0Z+AM4QPE30XhxXW0e66uhe+bz9K2pxgYs1GXMJwSUGoMMnuITdoMlkk1OI0UZoso8Hlt2g/Bq19r8/0aW7/Z8ix6s1nXCt2Vzk/DAaN+o9VnjBhZTpnXX1zNCLVHLJK1sinIEaVn0ntRYsH/X0ln X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Apr 2026 15:55:28.7581 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f7e80da6-571d-4c74-1c41-08dea53e9201 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[satlexmb07.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E2.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5992 Content-Type: text/plain; charset="utf-8" From: Zi Yan Change NR_MAX_BATCHED_MIGRATION to HPAGE_PUD_NR to allow batching THP copies. These are for testing purpose only. Signed-off-by: Zi Yan Signed-off-by: Shivank Garg --- mm/migrate.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index 9af070f9a1f2..a16c009d31d0 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1658,7 +1658,7 @@ static inline int try_split_folio(struct folio *folio= , struct list_head *split_f } =20 #ifdef CONFIG_TRANSPARENT_HUGEPAGE -#define NR_MAX_BATCHED_MIGRATION HPAGE_PMD_NR +#define NR_MAX_BATCHED_MIGRATION HPAGE_PUD_NR #else #define NR_MAX_BATCHED_MIGRATION 512 #endif --=20 2.43.0