From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF8D92F39B1; Mon, 10 Nov 2025 22:23:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813415; cv=fail; b=N5I1KD+8QSUW+8gKo8iQS9lS+bst5npBJ6sKowGXwUIwKkDY76PIDFYLt4qXCcgJkfoOfMr9Gu7d5zJJLTiBJ0uaG7MMCPkOo+hWgjhHVp+dhCsrhS3JS2DCXgVtm/UN1A0D6L777Q1Q1A35vh1o+j937dXQhf7K3CKaR3UCwts= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813415; c=relaxed/simple; bh=EpKr6G0Rk11ui8D+6dz9SCRpM5mYLzPC+hQiqugBw3s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=PnfbLpvOW8GI7+lDUEEfmLld1wOHgxqGeSrVhUILAMgz/pvP1GZEWHML+CDai35gC+Ryf/5TfI4nHpS/bSAI5UuK9M7FvBPKDF+IjI6QioqBILC9ki3FdSrmEZohh2WxApecxmBS7H0IPhdh8LTrpUNGEa/c5Oo/TBE065pszUE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=TvbJWKj/; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=upOpG8zb; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="TvbJWKj/"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="upOpG8zb" Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAM0dN1019647; Mon, 10 Nov 2025 22:21:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=+OmHm5+r2RgMhhHMJNazyAU9W9VnLQDJDS1/tF1j8VM=; b= TvbJWKj/h79+yuoeK3imNHSpV6NX1RwxvvumQjz7nVULCH9j33U03inWQL+FHkQA tzcpPMhhNNQwI449ujp/B7jaFRYPEGGvTDwsRXnedv+yGHJeI3izFK1L615FYIEu AN/oYVk+5FDOJtYUMnSJ060/ZazOGJYNWJjdzvgJOET4dQDiSKtVWNrdIDomEqk8 vqO0sYk2ZtfyAzKE/pNUfdeOHtGRRBKtwVZBZPmPUtL9s0h0J8+65fVxYY612Vsk FxpAGJwKZ9LUViN7Lif8uck5sN7Uz5En/5i1jUjZq0W7SA/O8lpviIuRbqXI0kzB 5pb4nf3b75NWXbPfybTifQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abqvfg49p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:44 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAMBBYe020907; Mon, 10 Nov 2025 22:21:43 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010007.outbound.protection.outlook.com [52.101.201.7]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8rhn4-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=awXxLhJrRa1RMLuYBg8y4+CNnH3tPd1hrqXWCSAr2OuW7zv/q9VhZHGOrpUFWkw73fb5RCz4xeb2+Vd9oMKyvTm6HpDJVIu10x3A/y5cCJ0sk6LiMZQWTsCD4+MmtMhMdu78ZL9zyQMGG6joGSJ2+ri3JikuyzN4J4WXp/Ex3lqCtuG3S5WRVptqb4YBkx7Lq0u9Qng0Yr82tP9XlwcbQvv0eQofkTI54FpO06dVReKJNmPPPjgXWS9+3/LXlL4dk/Rmuuh54AQ8R6H+xWO8ae42RrLMQegTwt2JtUWWkMN5cQ8lcuuqWBKsWgOVSq7hez1YWD3GLxOYAdAOKGsPUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+OmHm5+r2RgMhhHMJNazyAU9W9VnLQDJDS1/tF1j8VM=; b=FdZNvEzrZLAX+h1/0jaxT15gdVrCU2tNDOM5CW2rNzrM+O9GCumZ2hvltBenPbRzZXcBXICATUlsDGN4qSANMoQKrMYassA2cqLOvnSn/084UDAXu2lNSnksH4cpRSVjtPXXh82k76FVoiRt5idhoA3r3Df9vn9XIt5VrlL6zAsGqmuZzheaBEYeY6BIVb4hTmFECRav8LZtJiPMl5VhtgCzLAhg8Q56dqI8OQz/hu9ZwR/W1YRUsFC3fL6IFeOjlXc6koyjAfInkw3vwry39w7z70b61njoR+1oGV+lM8cMiqYfFS710pNxWekKmRvvPSMVr0vkB/TQXFqE1eCcHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+OmHm5+r2RgMhhHMJNazyAU9W9VnLQDJDS1/tF1j8VM=; b=upOpG8zbj6u1O23jmsrI2WmLlOPX69DRB4sj9PujgmlqOm2a35YuJbnxm1zSmMsaRLH1x9FBWNmPDPSUrxDXHjCJWsFurDrs/KBeyJEBapxbKBKC2NgLzEb4WYRMjrRiMyI2d7lPc11Z5uXF0B4iWpS/GxMosBuqYStF9PIYKX4= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:39 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:39 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 01/16] mm: correctly handle UFFD PTE markers Date: Mon, 10 Nov 2025 22:21:19 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0149.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2c7::13) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: f22be284-8da9-41c4-b9d9-08de20a784fe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Ct3ZLE9Kvzyb/OuUli0zfxYfXgDosMU5XGllEVAftG7J5jTOW0aeIs1Qb+UX?= =?us-ascii?Q?h5QRJVTCIRTFff2oJTwe3P4oLtR0ddT/Y6eDDTyxUO2gQefKOFQbVV4awy7C?= =?us-ascii?Q?5SjjfxBhZMM+hhMqXoJPM7OfaMeRXh80MGAU3MMv2PcykunoawQCrcKRkIA+?= =?us-ascii?Q?dRi8U0DOWmq8CDLWHQV+WVRvWsRGZfHxd0CgqX73zRSAj4Yk5qofY/sxr2ok?= =?us-ascii?Q?9xevoZ0pWPrILNlQ1UhUaOZ9AhDfUgwZhAcEkUSnV89eLQXTmpEX8aQwGtDv?= =?us-ascii?Q?DMrp+kO5TVuYzVbMpJAC6Rx3M9TCC5uKVB0Ulp1k3I41R7ckfEEcbEFRbF9w?= =?us-ascii?Q?m1KDj2lQXOnWaCGfUp+05gecaf3/41JqCtBxi2lS5Ik8SiplSa0REfJERNPd?= =?us-ascii?Q?EVkq1J+YX0vI7S1l+Zkx5mzmuH1kSZscPOnhHUNasffvkzQimVv0RUW06Slf?= =?us-ascii?Q?RgROPjkFAimZyTJnhc2KEHY0zgq6EWbyoVu/iD4oJFToiPl5b6kdTqhfBRoP?= =?us-ascii?Q?mGnu8qHJClCEukaH+xpQgUtfV0C/Nc/WisC8yoZInlDoRXgZrJDLZywCP6m7?= =?us-ascii?Q?ncmi0h9k7DClUSNdYpmNDEon4xprHyzlMVk+MtAdReFy7zj8GS1fWuSzUcpw?= =?us-ascii?Q?di/b/HlE8YyWR1SvPpc9by369w5eAxCaTW66EZmNhO3QmWbLIStMx45xSmcu?= =?us-ascii?Q?LuQPdYvehdstb0dQC9TUUxln5FYEA2L5x3wZ0Iy4PgmwM31TwxQmDG2adpw8?= =?us-ascii?Q?huRXrkxhZ4QKoPaV618w6LrT0qYEy7BvjHtTraRB7j4yd3QlM2DiHanJ8gLU?= =?us-ascii?Q?zCCKnSC5Re3GpI0wQHt4i7xB/NBDsG/9X8J8U6F1v/FXQi5b4MvA0HOLcbvP?= =?us-ascii?Q?yTa3tL7+BtqzgUHxB7zcj54Nzdt+DBetxWr9v8nEZoYVTkgSsvYdRYiWNd0I?= =?us-ascii?Q?MI+Raz1dzOvj5d7/YHavnkUNH2lPmc3oUd4YrE7L8Wkyfbv6sWJTHmTiBABz?= =?us-ascii?Q?8TKrHV5iMQRyDUr3T4SoqRv5bnj8OokoonhwDf7RwIPe9vPL9Xo5bUAthLSh?= =?us-ascii?Q?TKmHQQkDcE7vhobR8NIH86F4Dibwiqu7iR6AHCucM/eSCsIG6k3z+IX5a9vm?= =?us-ascii?Q?/YYwDjm/XPMvWiMftCOKG7EH7qPnxRK8WIHbaKbOh4mMwp1DG/3WpFb+BrNE?= =?us-ascii?Q?nxcFxTO1TRQ/uAHfy7PAYcnuasHdHSqZ+LRD+v/Gx7bwWqJRMzMfcZLINaez?= =?us-ascii?Q?pVoSGYcbHNO/7TbzoJEMtIPG4ZeCVtg1qKAkSfVejW5TlLkhvOgBRqRwdCUW?= =?us-ascii?Q?Cy+8/Dnf00Rk7eF4fJdtxO2Hxy32hc+k/calehym1XcUNHXk1eluazEN7WQ5?= =?us-ascii?Q?y00tSYxie1ziDt7/N1pG1oQ+d9loI8f8J4QEJf3hbKEQdEnCqQibOPsYF/CX?= =?us-ascii?Q?63JaMjY/WhQKOxi07O17rLmRdPIJORih?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?zlYJKFRdM4M9RS9kbZB/nR7z5TWEF1gRhM9rQfHn4OhD49VtNt54IQKmAUsq?= =?us-ascii?Q?gJMXyg6oCQ+6rVD/6J5Zn5nUln+eWU893JPG6VkECkPqL9VRbw5bu3yiV2Du?= =?us-ascii?Q?PPi+/qpUgq/LWXA+GI3QF5RBwztAijD4lT1esPOwxGkmrb1ZOAxqVX13Qwh8?= =?us-ascii?Q?+qwqsBQ/jnv5xxRiGKnQDszL4L7SJeFg534dLr2MTadtZrEjCH5iz9PAZP7B?= =?us-ascii?Q?CfQuafxI15qMWGRBpmGBZ20y+P8tihKpoyM6TNhjkMs6EAtc2+EXg9bRuTYj?= =?us-ascii?Q?wtVYZDDGOmRht7TDbgv/wzaEsDHVLEL1XYEe4/hWXWyPhfytpaxxRgAvj4R1?= =?us-ascii?Q?hbvxb5C2UzPh9MExiRaAOWYz7FJfb+EPfsYnKJ2ddsAJMDY0QSEJnvPwnZWN?= =?us-ascii?Q?oYXwhUcgnHm+cDChRLpEf6ahBLXHta94jSLGGNiYQvJ1Sw+JeIdfV95n4nsN?= =?us-ascii?Q?96BMRlkf0o/PXy+zdgDf0QpwF6rPcQLtY8pSXVMv/aCr/NWCHEMzJcL+ibfV?= =?us-ascii?Q?5Koosf2RIVlgW1hxuKQsID2fdc4PRgdrtdVA2EnH/X+U2m9FZyh5DYEKxq0o?= =?us-ascii?Q?2ZbLTOurhc22l5PPn06T2o3KnZSqkfDomIMTzvYZcqEznqpfaaUYLuBcIFvq?= =?us-ascii?Q?f0sjBEozRxmyrLlPcgTy5zte/MNo7CssuQMwTHlgEdTdGZ4v9viKXbl5l6HN?= =?us-ascii?Q?jEk8YaUYDqkItNBEDyO7YO26sGQt4wZ1p5UizQjmgH3D/DbirU5h3Ye5Dgzv?= =?us-ascii?Q?jBUo95aTobU8qOhkr/1mjm8yoeTadNprPvyu5+ZltNHSURWVLu3ZuY/GROHc?= =?us-ascii?Q?raHNCcYLFoPTGnopiCPLLyTdhROXq86I4W/08icU32oIsLvBbVavOb2DY4aF?= =?us-ascii?Q?PoinoX5MgLE1rYfqH9WsakvAvp8NRK0WOcXlsk0rrpftRwYvah4BWHWiqrKu?= =?us-ascii?Q?oiEzX9Fv/iCtx+lxg0Yyi/JLb4rC+Zx+YyAFDsuGOIUrZ2kd10GJLCUThpY7?= =?us-ascii?Q?MCPAIUp2gUnmCMOi/RPmFw2BXKbnPjJZTjzrOwIpUHy2h6TWkKI8IcyT3M51?= =?us-ascii?Q?axb2YLdYUhmWZ5+5apcVCuSUXuaCExPDfwAabab9FW9l/mcRt0GInRmgIvXq?= =?us-ascii?Q?lvkjElyf4FktLMmnbFmR5od4K43LHowU6pHc62Vicj+JXy4IIkLI3GbtBOe9?= =?us-ascii?Q?+43Eg/ndtu0dC9mJqLGUA01zp6MXr5t33wYWS0z6rtlwODOx6KGHP2UoYq0m?= =?us-ascii?Q?HxvkKHuvyOSYAaT9olc7/QQRrGbPvm7cU1sv2lXWbdmZ8j8A/MjkVz6yOxwr?= =?us-ascii?Q?7C+lFzTMkgTfpxEuMYqtltGEhqsClYMRZcNVaT9xx/EF/R4wBRoffA7a6eIr?= =?us-ascii?Q?wKBNoqoqld7fl7X8S9+2W5lbYHeVgdkuA4gC94Bhks3hQn9RZMYGYBYxDZJu?= =?us-ascii?Q?q1LCEY11ad53hweYwEQG6nb1oNUtzX5kNsjtnVoUjcMycsiUKMVznh6i9MM6?= =?us-ascii?Q?6771dtMV58nmFizYsL5FvQKo2TK1xKFmGeMoDg/FBBZw1UslysLV8JICq2X0?= =?us-ascii?Q?n5N73UQXIeLyTd9qWMMEAfoAyWb8hcyUHinvLH2SDcGvdlm20g6dc0UIUY3r?= =?us-ascii?Q?kQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: rykGXGR76JQFQncI+dM6UUO/+aO6DCcGWcKVGiamjA3uiLrFBAdKrNpK5He+TASKiko8R/SC0RvFW//W7xb20nfPknW4ZaUC/ioWcLvShl2ND5YSrReFp7MGzHGD/B+qMg9dfhUiIQzaT6HH3kZA19FyY7xF9zZCVcZMCjm0Yzudafw4kotlf11sL7NKvyI1l57Bk3VjkdAbOrapOWVICOfB3djJKz0CEqX4PywMw8w/TiaLEV5EDLMNvI3HjIJ0MJmnEsoc+aA56DKerYbTmnSNOm/GbDOz4dttEXCv5OBcSKAbwLkyTSioMe9Bk9tBeXCCToubFghfD9mj7db1M7qn0WK4W4ZeyyacQ9AihYL/adQDU6tKJpEr07DjSAH3M2W/z/z8tl9R7yIHPoPbdgehKolCwyWl+vwXNHQiDvasxAbLWwiAB7747I1U8gJVq/tCw/WopfzFZp8+Phlg2GR+sphdLux7ap+ooQ1d2VJWMZFfUc4nu86/ukHLcx1MlSqUvyIMi29SHzRuVTOLFmRX+u6JVbWBd/PNxwLhsSVzAZGhzJG9e0sjOnwu9FZlfVXaXL0RtAcbt5WMLod9A2B7SDDvEy1QyhBWFDQmQqA= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f22be284-8da9-41c4-b9d9-08de20a784fe X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:39.7765 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xVAzVTIU8NrrlZPFfMppJd9nQZ8+PR0+z6luZuaMsTXLXfwTDS+ssnQVEk3jDCScHLtOCOlPLq2Nll15x3s1HLFeHSdVxcqxeVvLa6QeRSo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 adultscore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: nO88I5oKXZ4LCqrAVZJVNDjRF_dy7Lf9 X-Authority-Analysis: v=2.4 cv=FaY6BZ+6 c=1 sm=1 tr=0 ts=69126578 cx=c_pps a=OOZaFjgC48PWsiFpTAqLcw==:117 a=OOZaFjgC48PWsiFpTAqLcw==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=g27RGp_RAGdlA0bXL-AA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-ORIG-GUID: nO88I5oKXZ4LCqrAVZJVNDjRF_dy7Lf9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4MiBTYWx0ZWRfX6n+r3Lwjnh5r sipCq5XAXMGGpwddOhWJ6yjh9qS8w2qc7TC/6y2uD6SLoH1sOR33rmBvhywgt56oVDfjXJW84Ba 86XQ7+Ig7Bm3/hd58RHkUYp2y3mkoRO7/9CKAK/rM/hyc4UgefLGHlbUNyh4puZTH7hFpX3d2IV GC4mHQ60eyRVMLQ9Wv71/AEbsF/J7vE7/23gppjnp9OZ7/VUM38QkWokeXTTWsww4kbBrSgIJfN fXx3zMIqpgEh/TU125ZDL/thM78KfgEJ74yk9sNuubdWUeNaXrejWP83AZKRBJdOXMV1YSWmWlX 6RGdez9WXfRzVw/vq/gsKVkiY9Qf3bWRhu+a0VOdceaxTchVFo+7BEqmw1VOnVguFisbzsjQLad eF4aq4sIPYDNhGf/5TLjgIk2CGk/XQ== Content-Type: text/plain; charset="utf-8" PTE markers were previously only concerned with UFFD-specific logic - that is, PTE entries with the UFFD WP marker set or those marked via UFFDIO_POISON. However since the introduction of guard markers in commit 7c53dfbdb024 ("mm: add PTE_MARKER_GUARD PTE marker"), this has no longer been the case. Issues have been avoided as guard regions are not permitted in conjunction with UFFD, but it still leaves very confusing logic in place, most notably the misleading and poorly named pte_none_mostly() and huge_pte_none_mostly(). This predicate returns true for PTE entries that ought to be treated as none, but only in certain circumstances, and on the assumption we are dealing with H/W poison markers or UFFD WP markers. This patch removes these functions and makes each invocation of these functions instead explicitly check what it needs to check. As part of this effort it introduces is_uffd_pte_marker() to explicitly determine if a marker in fact is used as part of UFFD or not. In the HMM logic we note that the only time we would need to check for a fault is in the case of a UFFD WP marker, otherwise we simply encounter a fault error (VM_FAULT_HWPOISON for H/W poisoned marker, VM_FAULT_SIGSEGV for a guard marker), so only check for the UFFD WP case. While we're here we also refactor code to make it easier to understand. Reviewed-by: Vlastimil Babka Signed-off-by: Lorenzo Stoakes Reviewed-by: Mike Rapoport (Microsoft) --- fs/userfaultfd.c | 93 +++++++++++++++++++++-------------- include/asm-generic/hugetlb.h | 8 --- include/linux/swapops.h | 18 ------- include/linux/userfaultfd_k.h | 21 ++++++++ mm/hmm.c | 7 ++- mm/hugetlb.c | 47 +++++++++--------- mm/mincore.c | 17 +++++-- mm/userfaultfd.c | 27 ++++++---- 8 files changed, 138 insertions(+), 100 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 54c6cc7fe9c6..375494309182 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -233,40 +233,48 @@ static inline bool userfaultfd_huge_must_wait(struct = userfaultfd_ctx *ctx, { struct vm_area_struct *vma =3D vmf->vma; pte_t *ptep, pte; - bool ret =3D true; =20 assert_fault_locked(vmf); =20 ptep =3D hugetlb_walk(vma, vmf->address, vma_mmu_pagesize(vma)); if (!ptep) - goto out; + return true; =20 - ret =3D false; pte =3D huge_ptep_get(vma->vm_mm, vmf->address, ptep); =20 /* * Lockless access: we're in a wait_event so it's ok if it - * changes under us. PTE markers should be handled the same as none - * ptes here. + * changes under us. + */ + + /* Entry is still missing, wait for userspace to resolve the fault. */ + if (huge_pte_none(pte)) + return true; + /* UFFD PTE markers require userspace to resolve the fault. */ + if (is_uffd_pte_marker(pte)) + return true; + /* + * If VMA has UFFD WP faults enabled and WP fault, wait for userspace to + * resolve the fault. */ - if (huge_pte_none_mostly(pte)) - ret =3D true; if (!huge_pte_write(pte) && (reason & VM_UFFD_WP)) - ret =3D true; -out: - return ret; + return true; + + return false; } #else static inline bool userfaultfd_huge_must_wait(struct userfaultfd_ctx *ctx, struct vm_fault *vmf, unsigned long reason) { - return false; /* should never get here */ + /* Should never get here. */ + VM_WARN_ON_ONCE(1); + return false; } #endif /* CONFIG_HUGETLB_PAGE */ =20 /* - * Verify the pagetables are still not ok after having reigstered into + * Verify the pagetables are still not ok after having registered into * the fault_pending_wqh to avoid userland having to UFFDIO_WAKE any * userfault that has already been resolved, if userfaultfd_read_iter and * UFFDIO_COPY|ZEROPAGE are being run simultaneously on two different @@ -284,53 +292,63 @@ static inline bool userfaultfd_must_wait(struct userf= aultfd_ctx *ctx, pmd_t *pmd, _pmd; pte_t *pte; pte_t ptent; - bool ret =3D true; + bool ret; =20 assert_fault_locked(vmf); =20 pgd =3D pgd_offset(mm, address); if (!pgd_present(*pgd)) - goto out; + return true; p4d =3D p4d_offset(pgd, address); if (!p4d_present(*p4d)) - goto out; + return true; pud =3D pud_offset(p4d, address); if (!pud_present(*pud)) - goto out; + return true; pmd =3D pmd_offset(pud, address); again: _pmd =3D pmdp_get_lockless(pmd); if (pmd_none(_pmd)) - goto out; + return true; =20 - ret =3D false; + /* + * A race could arise which would result in a softleaf entry such a + * migration entry unexpectedly being present in the PMD, so explicitly + * check for this and bail out if so. + */ if (!pmd_present(_pmd)) - goto out; + return false; =20 - if (pmd_trans_huge(_pmd)) { - if (!pmd_write(_pmd) && (reason & VM_UFFD_WP)) - ret =3D true; - goto out; - } + if (pmd_trans_huge(_pmd)) + return !pmd_write(_pmd) && (reason & VM_UFFD_WP); =20 pte =3D pte_offset_map(pmd, address); - if (!pte) { - ret =3D true; + if (!pte) goto again; - } + /* * Lockless access: we're in a wait_event so it's ok if it - * changes under us. PTE markers should be handled the same as none - * ptes here. + * changes under us. */ ptent =3D ptep_get(pte); - if (pte_none_mostly(ptent)) - ret =3D true; + + ret =3D true; + /* Entry is still missing, wait for userspace to resolve the fault. */ + if (pte_none(ptent)) + goto out; + /* UFFD PTE markers require userspace to resolve the fault. */ + if (is_uffd_pte_marker(ptent)) + goto out; + /* + * If VMA has UFFD WP faults enabled and WP fault, wait for userspace to + * resolve the fault. + */ if (!pte_write(ptent) && (reason & VM_UFFD_WP)) - ret =3D true; - pte_unmap(pte); + goto out; =20 + ret =3D false; out: + pte_unmap(pte); return ret; } =20 @@ -490,12 +508,13 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, uns= igned long reason) set_current_state(blocking_state); spin_unlock_irq(&ctx->fault_pending_wqh.lock); =20 - if (!is_vm_hugetlb_page(vma)) - must_wait =3D userfaultfd_must_wait(ctx, vmf, reason); - else + if (is_vm_hugetlb_page(vma)) { must_wait =3D userfaultfd_huge_must_wait(ctx, vmf, reason); - if (is_vm_hugetlb_page(vma)) hugetlb_vma_unlock_read(vma); + } else { + must_wait =3D userfaultfd_must_wait(ctx, vmf, reason); + } + release_fault_lock(vmf); =20 if (likely(must_wait && !READ_ONCE(ctx->released))) { diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h index dcb8727f2b82..e1a2e1b7c8e7 100644 --- a/include/asm-generic/hugetlb.h +++ b/include/asm-generic/hugetlb.h @@ -97,14 +97,6 @@ static inline int huge_pte_none(pte_t pte) } #endif =20 -/* Please refer to comments above pte_none_mostly() for the usage */ -#ifndef __HAVE_ARCH_HUGE_PTE_NONE_MOSTLY -static inline int huge_pte_none_mostly(pte_t pte) -{ - return huge_pte_none(pte) || is_pte_marker(pte); -} -#endif - #ifndef __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 2687928a8146..d1f665935cfc 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -469,24 +469,6 @@ static inline int is_guard_swp_entry(swp_entry_t entry) (pte_marker_get(entry) & PTE_MARKER_GUARD); } =20 -/* - * This is a special version to check pte_none() just to cover the case wh= en - * the pte is a pte marker. It existed because in many cases the pte mark= er - * should be seen as a none pte; it's just that we have stored some inform= ation - * onto the none pte so it becomes not-none any more. - * - * It should be used when the pte is file-backed, ram-based and backing - * userspace pages, like shmem. It is not needed upon pgtables that do not - * support pte markers at all. For example, it's not needed on anonymous - * memory, kernel-only memory (including when the system is during-boot), - * non-ram based generic file-system. It's fine to be used even there, bu= t the - * extra pte marker check will be pure overhead. - */ -static inline int pte_none_mostly(pte_t pte) -{ - return pte_none(pte) || is_pte_marker(pte); -} - static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) { struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index c0e716aec26a..da0b4fcc566f 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -479,4 +479,25 @@ static inline bool pte_swp_uffd_wp_any(pte_t pte) return false; } =20 + +static inline bool is_uffd_pte_marker(pte_t pte) +{ + swp_entry_t entry; + + if (pte_present(pte)) + return false; + + entry =3D pte_to_swp_entry(pte); + if (!is_pte_marker_entry(entry)) + return false; + + /* UFFD WP, poisoned swap entries are UFFD handled. */ + if (pte_marker_entry_uffd_wp(entry)) + return true; + if (is_poisoned_swp_entry(entry)) + return true; + + return false; +} + #endif /* _LINUX_USERFAULTFD_K_H */ diff --git a/mm/hmm.c b/mm/hmm.c index a56081d67ad6..387a38bbaf6a 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -244,7 +244,12 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, un= signed long addr, uint64_t pfn_req_flags =3D *hmm_pfn; uint64_t new_pfn_flags =3D 0; =20 - if (pte_none_mostly(pte)) { + /* + * Any other marker than a UFFD WP marker will result in a fault error + * that will be correctly handled, so we need only check for UFFD WP + * here. + */ + if (pte_none(pte) || pte_marker_uffd_wp(pte)) { required_fault =3D hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, 0); if (required_fault) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1ea459723cce..01c784547d1e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6743,29 +6743,28 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, stru= ct vm_area_struct *vma, } =20 vmf.orig_pte =3D huge_ptep_get(mm, vmf.address, vmf.pte); - if (huge_pte_none_mostly(vmf.orig_pte)) { - if (is_pte_marker(vmf.orig_pte)) { - pte_marker marker =3D - pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); - - if (marker & PTE_MARKER_POISONED) { - ret =3D VM_FAULT_HWPOISON_LARGE | - VM_FAULT_SET_HINDEX(hstate_index(h)); - goto out_mutex; - } else if (WARN_ON_ONCE(marker & PTE_MARKER_GUARD)) { - /* This isn't supported in hugetlb. */ - ret =3D VM_FAULT_SIGSEGV; - goto out_mutex; - } - } - + if (huge_pte_none(vmf.orig_pte)) /* - * Other PTE markers should be handled the same way as none PTE. - * * hugetlb_no_page will drop vma lock and hugetlb fault * mutex internally, which make us return immediately. */ return hugetlb_no_page(mapping, &vmf); + + if (is_pte_marker(vmf.orig_pte)) { + const pte_marker marker =3D + pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); + + if (marker & PTE_MARKER_POISONED) { + ret =3D VM_FAULT_HWPOISON_LARGE | + VM_FAULT_SET_HINDEX(hstate_index(h)); + goto out_mutex; + } else if (WARN_ON_ONCE(marker & PTE_MARKER_GUARD)) { + /* This isn't supported in hugetlb. */ + ret =3D VM_FAULT_SIGSEGV; + goto out_mutex; + } + + return hugetlb_no_page(mapping, &vmf); } =20 ret =3D 0; @@ -6934,6 +6933,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, int ret =3D -ENOMEM; struct folio *folio; bool folio_in_pagecache =3D false; + pte_t dst_ptep; =20 if (uffd_flags_mode_is(flags, MFILL_ATOMIC_POISON)) { ptl =3D huge_pte_lock(h, dst_mm, dst_pte); @@ -7073,13 +7073,14 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, if (folio_test_hwpoison(folio)) goto out_release_unlock; =20 + ret =3D -EEXIST; + + dst_ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); /* - * We allow to overwrite a pte marker: consider when both MISSING|WP - * registered, we firstly wr-protect a none pte which has no page cache - * page backing it, then access the page. + * See comment about UFFD marker overwriting in + * mfill_atomic_install_pte(). */ - ret =3D -EEXIST; - if (!huge_pte_none_mostly(huge_ptep_get(dst_mm, dst_addr, dst_pte))) + if (!huge_pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) goto out_release_unlock; =20 if (folio_in_pagecache) diff --git a/mm/mincore.c b/mm/mincore.c index 8ec4719370e1..fb80becd6119 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -32,11 +32,22 @@ static int mincore_hugetlb(pte_t *pte, unsigned long hm= ask, unsigned long addr, spinlock_t *ptl; =20 ptl =3D huge_pte_lock(hstate_vma(walk->vma), walk->mm, pte); + /* * Hugepages under user process are always in RAM and never * swapped out, but theoretically it needs to be checked. */ - present =3D pte && !huge_pte_none_mostly(huge_ptep_get(walk->mm, addr, pt= e)); + if (!pte) { + present =3D 0; + } else { + const pte_t ptep =3D huge_ptep_get(walk->mm, addr, pte); + + if (huge_pte_none(ptep) || is_pte_marker(ptep)) + present =3D 0; + else + present =3D 1; + } + for (; addr !=3D end; vec++, addr +=3D PAGE_SIZE) *vec =3D present; walk->private =3D vec; @@ -175,8 +186,8 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, pte_t pte =3D ptep_get(ptep); =20 step =3D 1; - /* We need to do cache lookup too for pte markers */ - if (pte_none_mostly(pte)) + /* We need to do cache lookup too for markers */ + if (pte_none(pte) || is_pte_marker(pte)) __mincore_unmapped_range(addr, addr + PAGE_SIZE, vma, vec); else if (pte_present(pte)) { diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 00122f42718c..cc4ce205bbec 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -178,6 +178,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, spinlock_t *ptl; struct folio *folio =3D page_folio(page); bool page_in_cache =3D folio_mapping(folio); + pte_t dst_ptep; =20 _dst_pte =3D mk_pte(page, dst_vma->vm_page_prot); _dst_pte =3D pte_mkdirty(_dst_pte); @@ -199,12 +200,15 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, } =20 ret =3D -EEXIST; + + dst_ptep =3D ptep_get(dst_pte); + /* - * We allow to overwrite a pte marker: consider when both MISSING|WP - * registered, we firstly wr-protect a none pte which has no page cache - * page backing it, then access the page. + * We are allowed to overwrite a UFFD pte marker: consider when both + * MISSING|WP registered, we firstly wr-protect a none pte which has no + * page cache page backing it, then access the page. */ - if (!pte_none_mostly(ptep_get(dst_pte))) + if (!pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) goto out_unlock; =20 if (page_in_cache) { @@ -583,12 +587,15 @@ static __always_inline ssize_t mfill_atomic_hugetlb( goto out_unlock; } =20 - if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE) && - !huge_pte_none_mostly(huge_ptep_get(dst_mm, dst_addr, dst_pte))) { - err =3D -EEXIST; - hugetlb_vma_unlock_read(dst_vma); - mutex_unlock(&hugetlb_fault_mutex_table[hash]); - goto out_unlock; + if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) { + const pte_t ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); + + if (!huge_pte_none(ptep) && !is_uffd_pte_marker(ptep)) { + err =3D -EEXIST; + hugetlb_vma_unlock_read(dst_vma); + mutex_unlock(&hugetlb_fault_mutex_table[hash]); + goto out_unlock; + } } =20 err =3D hugetlb_mfill_atomic_pte(dst_pte, dst_vma, dst_addr, --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3ECC02F39B1; Mon, 10 Nov 2025 22:23:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813433; cv=fail; b=ZX994XRraXBqd8HZwfYYpigVVpxCjWbZB8OXkKrpSuPiKNpiFS+N+WLeMYDv9rsTd09Oxhdv22znWQVl2r4iHRBGnJqUbNrQGyXAzPBf1l10IHu/NFzqN+R7ceurcR1ZzYE2NC2PrIY4fxmYooKPwLvvqefoZiagbQNC2KGUc9w= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813433; c=relaxed/simple; bh=LR8bW9tolqmeduLigpt8Nj1oQNGKo+zRMTEydm9J4Y8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=D1YCY7xqdU0Nqh6sQKx+sxqyW9HYcED/gNa69ngyyxudERUXsAke8qemEOrNDBZRlp5ZlXq0WkJBHVw214KelbWLKMzgv73AgiQMXpNhGVLQ/Os8NXMq5PleYZ87kzH29uUqSiG75ktd+rMO50t1QN2S1mejpp+1lc66U77SxVs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=GsKN8DfQ; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=zxJAos3N; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="GsKN8DfQ"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="zxJAos3N" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAL8d3E009183; Mon, 10 Nov 2025 22:21:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=aJGhxDX8TnplLLuvZd+Xijrhn30dBPb6U+yF189PK+Y=; b= GsKN8DfQc+20/DFRuL0HauXgy2rE351oHLUu2mDGv1EcaNEjxU/e1exIN3oPa2vs NR8XFurSY5vMkyt8j3qHr+F+x9qty9LQgKvGBdSQHZO6NofjatQ7f9Doyl8Ls4fh sAy6Na4BpDG387byxIqUw26DVPUiPus6cj9fPsdEAkN5XXGKUYF+FoNq9TFZ1UyC 4KYNIAU5zCr4hsX44THmmQQAxPaZG4t1Xttlw/N03+AkjpCeZ8eh96Aqn06knz94 hKbnhKllst74JQPfiNXkosGKP6NahWZZ9n7DmrB2Fz+CUq/UuWQ5DXaHtGnq+BGh byi0FxP1zsaRnMN3a5oRPA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abqpug5rr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:48 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAL8Wxl012637; Mon, 10 Nov 2025 22:21:47 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010031.outbound.protection.outlook.com [52.101.201.31]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vac8sq7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:47 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=seozOJGx99g10UNfG3OyT5LPL68iqccywVpmX019zT6bz2XMSh82+5Ke5/tWT7ZbcaBlpOuqdUWBwl+9+2/CZ74lqg8nlaoXNzi7QJ19Gaa1lLIc/UQW+y82XqjQqE7Z98kUlDycVVjax99eY8T2/I+FR0HERjpydWzEuQb6+VnPWv/keXVc/N5hiPLMhbC8HWe5xphDbb92mU6KF1hy183991npMDpTPSBdwxdTXHFM+RmMv4KisRw/WUGThc529kXcqxULMuH5mH2JACoPSmg5J5oPIEN5KEALHG/UAMy15lfhED3M2HY4vHYkdXKVOotprlSTjIgVtRE6jUyXCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aJGhxDX8TnplLLuvZd+Xijrhn30dBPb6U+yF189PK+Y=; b=Ls6+uAYDOBuEoH6A/eezYl6GKvE4ZbDIUg1/izU5v4OQicbDit8VhQhYrf9T7U9LWE3wGfPUEJvgg9F17suyuOfrivoFoWSM6WQZxWpJtbkzwIuCXl+OLtyGL4VK2oYuLu5adoW36FsEhI3qKckEa5y5ahIOD0W091Trse1jHlqWe5kyshU8CW0hKJmyK7atv5fvay9RInrA/50mGsdYP/WQVOI22C0/TRck0vQ2oObNh8XRxRySyjv4wQxDXdxwA6UdOWtaYcCs9lY3QWwviR0i8k2+ey6xTIb1l6NdOePh2rYgSmDC1FivTfrXOecQ4xOclSNjHbP8ACiFgu21Uw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aJGhxDX8TnplLLuvZd+Xijrhn30dBPb6U+yF189PK+Y=; b=zxJAos3NZiQmgLYrGNR+LRR8glzpAkCfVKy8kTD7e5E593Lspyiu+v463auiZYF30KsUrIZ0QcZSeLnw4t9PbFUSeIF1fyzntHUKgl7hE4S3qUCqVJ1jjAZd3KmcUzKZbDg9zKL1ZtM53I7LhdNDNdCV3Xllepx/v4fRH/SJGoM= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:42 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:42 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 02/16] mm: introduce leaf entry type and use to simplify leaf entry logic Date: Mon, 10 Nov 2025 22:21:20 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0672.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:351::10) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: fba25159-23d0-4a56-78fd-08de20a7864e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?cF0knPop6w0+G4OfbIsM1tVmwVx+sv9v2q+TfN9R3eNGNkWRDKBDBv1GIRS5?= =?us-ascii?Q?pJM/QAik6nJ8zq6Uf29QRLW+VZ34GZOMBoAEBNT1mui/P4Op0JYZBv1guhQc?= =?us-ascii?Q?S8RPEcKxoU2tzpM+e7/gjlqxMdMFMDtvedYggW1JtXFgQTa82TSFymp8MCsf?= =?us-ascii?Q?xXCuSw7evgnaWDqmgAFNw1Hl5WVu3azrr2vOesoPAfGzbCg1T798C+X/3R31?= =?us-ascii?Q?UGteaWjp11UBODyw3+adsWSCfHTJY7mcG2kXSrGcbNRapNIQLHl0ZCllp98f?= =?us-ascii?Q?O7Xdq8j5/QwYtoW4Lnqrb7ealrSiirp1nzMBhOCx4AIT5oXlw0U7/Mdr0U2u?= =?us-ascii?Q?j11dng3mPZBcWIQSEZWo89Xg/yp7eNQy/YZYiMUcaZcixykwbBTOBY4gTVmH?= =?us-ascii?Q?b9yFyX1FEW5D7hR0cYi3lCvY7Ec0M3SaFIjN6fQJ+bltz5xeDnPmtUGTHltn?= =?us-ascii?Q?HWF9aGltDizJDpOtw/rJP8ncmRqgc/CQHlv0GlCXhJvsJ7M70Z37E1Ew84VE?= =?us-ascii?Q?X0ZqjW1DtxG/7v73OxkkMlhEzRNF8wA0HsDP3c6vXmN0ntIIJG/0YVEHcxAW?= =?us-ascii?Q?slGAechhEXBEyLpqfZI1u7+n3Zr2TSoCQ1H4LCPOO4pcY9TwK5g9vE/025pr?= =?us-ascii?Q?yXWvylGSsCmwO2bZAigAH4UuQPVx8X/ZTefh6SRMAVIFp21fA2hD/K8TRFc2?= =?us-ascii?Q?I+xSCd/OYLKIvYLa6nzjqLX/O2lcmBur+Man4iGzWfFHWqs2tab2ctFOWazh?= =?us-ascii?Q?exa9SQnsFicoDhTtTrblvOjSvi2CQDYvOEVxsI56Nq7wJxTHXPsovJVgI+A1?= =?us-ascii?Q?PdoWv6sTqQiTcX5LOW7+7o3XOw2bW8PHfB52agb4506YdKhjfiZqeADGS70P?= =?us-ascii?Q?eQyudaY/Gf3pKJSD39Qe15ECF+YsNUHiG+tU6BjRQVhrliPkZCBdN9FgUgyJ?= =?us-ascii?Q?RBe8e4+Np3vwtpWICLebWDZj2bT0dn7F06ESqeq17tuzh7zvUXuJJC0q/Ro8?= =?us-ascii?Q?WmFBAJ0qaBwz5dRaByXZp33+O6MjXc64CXeyGlpH0oZut0n2VM4vSUz3WjSW?= =?us-ascii?Q?tc2v5IfYyGnPApD4kADNOVfRF9ZmLpfIsgjd94xEhHmvKl1dSvhf6iwz2z4I?= =?us-ascii?Q?kv59UVWZShpBcLSp7f7EV1kuYzmXmM4iakpJs7rTvZOoByPkokv5n5or8vmI?= =?us-ascii?Q?Dalk931cZZF/u3oJMwlsx6/8A8Ee3ZWcUHu9aLjuNFlLhiupRKZkv++bAIgp?= =?us-ascii?Q?mG77lGhsRw0q5703kyHQPScg7GqVAQf8xTIDVxc9c0peX6U2Gq1aouHNm693?= =?us-ascii?Q?w7A+nZl9q2nX2z91WmmNCLgofAPRknPBb1XZk4y+cw/VPlmCC5YElsbibDvY?= =?us-ascii?Q?LblaxKVNnLEX8MCsIz93Ne66FtlRMjVNLgzeA3xXE2S56lsXBtg7emWH/HhH?= =?us-ascii?Q?EoI242XCwdD7JLEHwNxxAubuKlJp6nlz?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?U9gDGMke1y+o+8ehZJlNZg1QNQ3KWFbZ2z/d2IRtH2gmDzncQX+LIrLxNmAd?= =?us-ascii?Q?ZT5nTSrtn1wwyJ908wwW0xc87mspvnBSyXw1ZMCqxJGDMDGNmEhXlxfjxiSd?= =?us-ascii?Q?ETEvrgSp3th41k0ARgOmGSNwF0nu2CB0rLx9qs67XYpd/KCHiKcprVPGZLXl?= =?us-ascii?Q?Alm7p7XyF04TstBFpv2QIy1SBn25OU03pAmHlhwqGl2Qz4B+OFwfpjJgn4nD?= =?us-ascii?Q?5X8ym/WMY6ra4gakWcZd2AmmLEdvNVh+b4PbJ3XT6PNTIFyH6vEu2AOseGR+?= =?us-ascii?Q?HsTgi1TOhjOaDRrB0vqQJp9pvrXg1hKpBbku/0EcY81/QpD5X3UWUsRvAAWo?= =?us-ascii?Q?5c308OV4o2wFCMUlrHxq/usghNFXxuRe51smNoL6rcWQFlr4jh9hTFvjBIiA?= =?us-ascii?Q?0xkDHyS19JsBi1QE8KCWxn3Egx+Dlitspum8XbQtahFCmm2xR+8R269WLykI?= =?us-ascii?Q?oYypfDXUUoLBQZcN87nJ9tZPGWqv2g7/hWjQdc4A2QwNnx9DuPsNH6Ma8lRP?= =?us-ascii?Q?QPzS+9SZte1FwToSKW9wjjNhrTAtXCArLh2i4GHwpPxihKlXtLOU7lloi8QV?= =?us-ascii?Q?TwvJuAWPVR/44oG/WwdGWkY/pxVc639XW91i5zuYjwOJ2CPLgKgH/R9GAQDm?= =?us-ascii?Q?vtlPdHWKFfp8nEeJ7UpUpLtzufg5d1iPhtYnvlXPAAUuvV99SYQcJDPr2CLY?= =?us-ascii?Q?JPlvJgU8HWdWKM5V8HTwmsQKPlbJtLwY80+VuwvI6Qgea++5koLaGoz9T/w6?= =?us-ascii?Q?o4xm25u7oIGQ3NcfxmqmHQd7rvke3aVvhE7hhAngxmDNU5TN65RlcZ8OEpRS?= =?us-ascii?Q?qvWovL08lxl3gBo4nTdbJjEBJtGZJEjwyKdn/I+gPqCTGDAbLw5MKBhbe73x?= =?us-ascii?Q?UPigTikmciFDDR3TuklkaN0BNVKA1Ejut3FpmdDwL92b/iNyEkAfXkG/L0iZ?= =?us-ascii?Q?oOh0Oa1QnTZfIOrDrtCvbgQ0/lQ6FmMltgWcO+DJxLfZlqWvmaYbKBMS4PUN?= =?us-ascii?Q?DRimYTOoLUX58uSZbxzCjt0flP6WZOPYEimVt6i01u9Af8pecd5rSXykFa9u?= =?us-ascii?Q?hc6WinEXJihNm+/Q5MnATmXUBMcWJKj7yX4CqQroCgaMlIaSjQs3+3XErPqD?= =?us-ascii?Q?P0bbzq8Vj3GW/y4veuzHrDzUZehHezTwHOKTvqb7vhQlH3/XjicMqJsuyXl0?= =?us-ascii?Q?tHJzBfbviHyXR16Qh6FLczt5OVXym8nmrS+POhEU4KLFhoUVrUBTKl2EVjGo?= =?us-ascii?Q?mAGLUyj0F1kYxgxD9nrnPls/9c19L6osBMAQrXJX67xZ5Nc0IBWVYnZMW6S1?= =?us-ascii?Q?uMxsSv5bVJQmbuWpQVH0dr00+XQ9f4IJaipxIElfcDWgaAONecMiQS3QWfix?= =?us-ascii?Q?9AYdV5JO7NlJOQcmdKjWR+sSSxliQS2zp4xI48rjYhP1M2k3MR0UgwtDINuM?= =?us-ascii?Q?dXiKY+bxPBqpEkOCLxxGx3zBSfQrty7qBNxB2ZYSCVMqr7uVdj4yo8VuYF8C?= =?us-ascii?Q?6Vve00tN514X28YPVMkjcd+ProCHxO8v0ilOkuJa4TG2rIOiTipfdH1/u7GO?= =?us-ascii?Q?TyVBRYgz4KOdQ+ep85PN+QBgxBDTaK9KkprJ6opDSseemZlpfzIuHuW99lHT?= =?us-ascii?Q?gA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: a83lY7lg2bBU0yXLb1tPUWNZO1WC7/ttMmYkaFRolP/RbQZscUYESSFC3fi7rMnWXH6AXNfsU5iQxVZgEQ7bQAfS8GrkQMx00q2wKBTDY6Hw+ErBgFrc0Nc4DTxi5XLqsCMsVEUcc770PmDJyOPY1LElBmH+el9nbL5G3WS1fPQF9rnGiARU58WVEwOhzlYGdYAstYTazFp9m6Rdfp7XLoLP41Txucowdjm7D2+LzHC57QiBjAGD5Y2kP/C3TjU7e4myLCrAZUqIqCOZY77tYyjjx1w9k1Ui0LmaVWePgEyc2u6d+GGj09ymcCkVluGHaYm2EKGqAjo9trv6briF+IJU23axmyYcuJfT13bzDvyzY1LPcvw2qzpbVunQSyUTif9kwLo+aSb/kAH9nRwMdlRNj7NDfqKDE+OgBUwJBupU9Gzc9TEHSW3qyf2uFmrwoYTKmjlCwguydRfTeErBSRelEDrISy83YhiyZb0U8zNckCGrDVRLpCGyrpdYFyPlCt4KYlkYADq1LpA4Y1M81LI/AL0duSMneo92hpEoSX983u3m2AX2BFpQXJy4OdIGvaAIX+mQiyyfDpbpAhDFYhB01FanvtvZqHNENJ17PMA= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: fba25159-23d0-4a56-78fd-08de20a7864e X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:41.9862 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /mwE71dHn/FGr80XWZR4e7RDjLYRgvw22cZ0G5gxVTxfJjdU3+59SDLetjH0VwQsvbg1ZbzbR3aXFIuiOr/YBsgasW0O+1HArMeKBomI+4Y= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 adultscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE3OSBTYWx0ZWRfX318+Cr6K7lhj OnV8Y7Lsy1ygnkrTuKzvYyJgFD4csyBqyNieEipjw9XWMKbYk4I9cCc6iVN4ElXfwcTikYNUAZz A9JP4kGdC+53lX5OSDSAx3BVT92P7Jh8olG/6xjI/VMZFqvgfFMrDGBpbAEMU1vhX0z2+EeGJsc gz3YlVdDZKtfiTG7HPFvjXyn/myuDbSFDq1a/MyfgiRlMzLSrWpUU2ep5/joxlF/TKpsLh4CpzA j4DC+OiLxZo8GjdkR48WNQCjOnSL8o3UZlTR/Hgme2SXBZ5ADG9GKFIlVVGeT7o3q8hPXaxE/Ze sF6dDKdIgM+2N99lYkLOCURbg1MHeFNOJ9T3nG6dcGNVTXlBxuVV01JPB2OXhLz5tgPdfl75yxK TJjQZf3QP6Ys3XdzthvfL5QniGY4cwoH+3Q5xLhV8lnSCloze98= X-Authority-Analysis: v=2.4 cv=H5rWAuYi c=1 sm=1 tr=0 ts=6912657c b=1 cx=c_pps a=zPCbziy225d3KhSqZt3L1A==:117 a=zPCbziy225d3KhSqZt3L1A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=VwQbUJbxAAAA:8 a=-kTjvzIDFa8eMlyDQDQA:9 cc=ntf awl=host:12099 X-Proofpoint-ORIG-GUID: Pk7w4ZrsTjLpWtVW8RmeqMNzfU6txZYd X-Proofpoint-GUID: Pk7w4ZrsTjLpWtVW8RmeqMNzfU6txZYd Content-Type: text/plain; charset="utf-8" The kernel maintains leaf page table entries which contain either: - Nothing ('none' entries) - Present entries (that is stuff the hardware can navigate without fault) - Everything else that will cause a fault which the kernel handles In the 'everything else' group we include swap entries, but we also include a number of other things such as migration entries, device private entries and marker entries. Unfortunately this 'everything else' group expresses everything through a swp_entry_t type, and these entries are referred to swap entries even though they may well not contain a... swap entry. This is compounded by the rather mind-boggling concept of a non-swap swap entry (checked via non_swap_entry()) and the means by which we twist and turn to satisfy this. This patch lays the foundation for reducing this confusion. We refer to 'everything else' as a 'software-define leaf entry' or 'softleaf'. for short And in fact we scoop up the 'none' entries into this concept also so we are left with: - Present entries. - Softleaf entries (which may be empty). This allows for radical simplification across the board - one can simply convert any leaf page table entry to a leaf entry via softleaf_from_pte(). If the entry is present, we return an empty leaf entry, so it is assumed the caller is aware that they must differentiate between the two categories of page table entries, checking for the former via pte_present(). As a result, we can eliminate a number of places where we would otherwise need to use predicates to see if we can proceed with leaf page table entry conversion and instead just go ahead and do it unconditionally. We do so where we can, adjusting surrounding logic as necessary to integrate the new softleaf_t logic as far as seems reasonable at this stage. We typedef swp_entry_t to softleaf_t for the time being until the conversion can be complete, meaning everything remains compatible regardless of which type is used. We will eventually remove swp_entry_t when the conversion is complete. We introduce a new header file to keep things clear - leafops.h - this imports swapops.h so can direct replace swapops imports without issue, and we do so in all the files that require it. Additionally, add new leafops.h file to core mm maintainers entry. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- MAINTAINERS | 1 + fs/proc/task_mmu.c | 26 +-- fs/userfaultfd.c | 6 +- include/linux/leafops.h | 387 ++++++++++++++++++++++++++++++++++ include/linux/mm_inline.h | 6 +- include/linux/mm_types.h | 25 +++ include/linux/swapops.h | 28 --- include/linux/userfaultfd_k.h | 51 +---- mm/hmm.c | 2 +- mm/hugetlb.c | 37 ++-- mm/madvise.c | 16 +- mm/memory.c | 41 ++-- mm/mincore.c | 6 +- mm/mprotect.c | 6 +- mm/mremap.c | 4 +- mm/page_vma_mapped.c | 11 +- mm/shmem.c | 7 +- mm/userfaultfd.c | 6 +- 18 files changed, 502 insertions(+), 164 deletions(-) create mode 100644 include/linux/leafops.h diff --git a/MAINTAINERS b/MAINTAINERS index 2628431dcdfe..314910a70bbf 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -16257,6 +16257,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/gi= t/akpm/mm F: include/linux/gfp.h F: include/linux/gfp_types.h F: include/linux/highmem.h +F: include/linux/leafops.h F: include/linux/memory.h F: include/linux/mm.h F: include/linux/mm_*.h diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index fc35a0543f01..24d26b49d870 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -14,7 +14,7 @@ #include #include #include -#include +#include #include #include #include @@ -1230,11 +1230,11 @@ static int smaps_hugetlb_range(pte_t *pte, unsigned= long hmask, if (pte_present(ptent)) { folio =3D page_folio(pte_page(ptent)); present =3D true; - } else if (is_swap_pte(ptent)) { - swp_entry_t swpent =3D pte_to_swp_entry(ptent); + } else { + const softleaf_t entry =3D softleaf_from_pte(ptent); =20 - if (is_pfn_swap_entry(swpent)) - folio =3D pfn_swap_entry_folio(swpent); + if (softleaf_has_pfn(entry)) + folio =3D softleaf_to_folio(entry); } =20 if (folio) { @@ -1955,9 +1955,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, flags |=3D PM_SWAP; if (is_pfn_swap_entry(entry)) page =3D pfn_swap_entry_to_page(entry); - if (pte_marker_entry_uffd_wp(entry)) + if (softleaf_is_uffd_wp_marker(entry)) flags |=3D PM_UFFD_WP; - if (is_guard_swp_entry(entry)) + if (softleaf_is_guard_marker(entry)) flags |=3D PM_GUARD_REGION; } =20 @@ -2330,18 +2330,18 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; } else if (is_swap_pte(pte)) { - swp_entry_t swp; + softleaf_t entry; =20 categories |=3D PAGE_IS_SWAPPED; if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; =20 - swp =3D pte_to_swp_entry(pte); - if (is_guard_swp_entry(swp)) + entry =3D softleaf_from_pte(pte); + if (softleaf_is_guard_marker(entry)) categories |=3D PAGE_IS_GUARD; else if ((p->masks_of_interest & PAGE_IS_FILE) && - is_pfn_swap_entry(swp) && - !folio_test_anon(pfn_swap_entry_folio(swp))) + softleaf_has_pfn(entry) && + !folio_test_anon(softleaf_to_folio(entry))) categories |=3D PAGE_IS_FILE; =20 if (pte_swp_soft_dirty(pte)) @@ -2466,7 +2466,7 @@ static void make_uffd_wp_huge_pte(struct vm_area_stru= ct *vma, { unsigned long psize; =20 - if (is_hugetlb_entry_hwpoisoned(ptent) || is_pte_marker(ptent)) + if (is_hugetlb_entry_hwpoisoned(ptent) || pte_is_marker(ptent)) return; =20 psize =3D huge_page_size(hstate_vma(vma)); diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 375494309182..4e900091849b 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include =20 @@ -251,7 +251,7 @@ static inline bool userfaultfd_huge_must_wait(struct us= erfaultfd_ctx *ctx, if (huge_pte_none(pte)) return true; /* UFFD PTE markers require userspace to resolve the fault. */ - if (is_uffd_pte_marker(pte)) + if (pte_is_uffd_marker(pte)) return true; /* * If VMA has UFFD WP faults enabled and WP fault, wait for userspace to @@ -337,7 +337,7 @@ static inline bool userfaultfd_must_wait(struct userfau= ltfd_ctx *ctx, if (pte_none(ptent)) goto out; /* UFFD PTE markers require userspace to resolve the fault. */ - if (is_uffd_pte_marker(ptent)) + if (pte_is_uffd_marker(ptent)) goto out; /* * If VMA has UFFD WP faults enabled and WP fault, wait for userspace to diff --git a/include/linux/leafops.h b/include/linux/leafops.h new file mode 100644 index 000000000000..cff9d94fd5d1 --- /dev/null +++ b/include/linux/leafops.h @@ -0,0 +1,387 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Describes operations that can be performed on software-defined page tab= le + * leaf entries. These are abstracted from the hardware page table entries + * themselves by the softleaf_t type, see mm_types.h. + */ +#ifndef _LINUX_LEAFOPS_H +#define _LINUX_LEAFOPS_H + +#include +#include +#include + +#ifdef CONFIG_MMU + +/* Temporary until swp_entry_t eliminated. */ +#define LEAF_TYPE_SHIFT SWP_TYPE_SHIFT + +enum softleaf_type { + /* Fundamental types. */ + SOFTLEAF_NONE, + SOFTLEAF_SWAP, + /* Migration types. */ + SOFTLEAF_MIGRATION_READ, + SOFTLEAF_MIGRATION_READ_EXCLUSIVE, + SOFTLEAF_MIGRATION_WRITE, + /* Device types. */ + SOFTLEAF_DEVICE_PRIVATE_READ, + SOFTLEAF_DEVICE_PRIVATE_WRITE, + SOFTLEAF_DEVICE_EXCLUSIVE, + /* H/W posion types. */ + SOFTLEAF_HWPOISON, + /* Marker types. */ + SOFTLEAF_MARKER, +}; + +/** + * softleaf_mk_none() - Create an empty ('none') leaf entry. + * Returns: empty leaf entry. + */ +static inline softleaf_t softleaf_mk_none(void) +{ + return ((softleaf_t) { 0 }); +} + +/** + * softleaf_from_pte() - Obtain a leaf entry from a PTE entry. + * @pte: PTE entry. + * + * If @pte is present (therefore not a leaf entry) the function returns an= empty + * leaf entry. Otherwise, it returns a leaf entry. + * + * Returns: Leaf entry. + */ +static inline softleaf_t softleaf_from_pte(pte_t pte) +{ + if (pte_present(pte) || pte_none(pte)) + return softleaf_mk_none(); + + /* Temporary until swp_entry_t eliminated. */ + return pte_to_swp_entry(pte); +} + +/** + * softleaf_is_none() - Is the leaf entry empty? + * @entry: Leaf entry. + * + * Empty entries are typically the result of a 'none' page table leaf entry + * being converted to a leaf entry. + * + * Returns: true if the entry is empty, false otherwise. + */ +static inline bool softleaf_is_none(softleaf_t entry) +{ + return entry.val =3D=3D 0; +} + +/** + * softleaf_type() - Identify the type of leaf entry. + * @enntry: Leaf entry. + * + * Returns: the leaf entry type associated with @entry. + */ +static inline enum softleaf_type softleaf_type(softleaf_t entry) +{ + unsigned int type_num; + + if (softleaf_is_none(entry)) + return SOFTLEAF_NONE; + + type_num =3D entry.val >> LEAF_TYPE_SHIFT; + + if (type_num < MAX_SWAPFILES) + return SOFTLEAF_SWAP; + + switch (type_num) { +#ifdef CONFIG_MIGRATION + case SWP_MIGRATION_READ: + return SOFTLEAF_MIGRATION_READ; + case SWP_MIGRATION_READ_EXCLUSIVE: + return SOFTLEAF_MIGRATION_READ_EXCLUSIVE; + case SWP_MIGRATION_WRITE: + return SOFTLEAF_MIGRATION_WRITE; +#endif +#ifdef CONFIG_DEVICE_PRIVATE + case SWP_DEVICE_WRITE: + return SOFTLEAF_DEVICE_PRIVATE_WRITE; + case SWP_DEVICE_READ: + return SOFTLEAF_DEVICE_PRIVATE_READ; + case SWP_DEVICE_EXCLUSIVE: + return SOFTLEAF_DEVICE_EXCLUSIVE; +#endif +#ifdef CONFIG_MEMORY_FAILURE + case SWP_HWPOISON: + return SOFTLEAF_HWPOISON; +#endif + case SWP_PTE_MARKER: + return SOFTLEAF_MARKER; + } + + /* Unknown entry type. */ + VM_WARN_ON_ONCE(1); + return SOFTLEAF_NONE; +} + +/** + * softleaf_is_swap() - Is this leaf entry a swap entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a swap entry, otherwise false. + */ +static inline bool softleaf_is_swap(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_SWAP; +} + +/** + * softleaf_is_migration() - Is this leaf entry a migration entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a migration entry, otherwise false. + */ +static inline bool softleaf_is_migration(softleaf_t entry) +{ + switch (softleaf_type(entry)) { + case SOFTLEAF_MIGRATION_READ: + case SOFTLEAF_MIGRATION_READ_EXCLUSIVE: + case SOFTLEAF_MIGRATION_WRITE: + return true; + default: + return false; + } +} + +/** + * softleaf_is_device_private() - Is this leaf entry a device private entr= y? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device private entry, otherwise fa= lse. + */ +static inline bool softleaf_is_device_private(softleaf_t entry) +{ + switch (softleaf_type(entry)) { + case SOFTLEAF_DEVICE_PRIVATE_WRITE: + case SOFTLEAF_DEVICE_PRIVATE_READ: + return true; + default: + return false; + } +} + +/** + * softleaf_is_device_exclusive() - Is this leaf entry a device exclusive = entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device exclusive entry, otherwise = false. + */ +static inline bool softleaf_is_device_exclusive(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_DEVICE_EXCLUSIVE; +} + +/** + * softleaf_is_hwpoison() - Is this leaf entry a hardware poison entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a hardware poison entry, otherwise f= alse. + */ +static inline bool softleaf_is_hwpoison(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_HWPOISON; +} + +/** + * softleaf_is_marker() - Is this leaf entry a marker? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a marker entry, otherwise false. + */ +static inline bool softleaf_is_marker(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_MARKER; +} + +/** + * softleaf_to_marker() - Obtain marker associated with leaf entry. + * @entry: Leaf entry, softleaf_is_marker(@entry) must return true. + * + * Returns: Marker associated with the leaf entry. + */ +static inline pte_marker softleaf_to_marker(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_is_marker(entry)); + + return swp_offset(entry) & PTE_MARKER_MASK; +} + +/** + * softleaf_has_pfn() - Does this leaf entry encode a valid PFN number? + * @entry: Leaf entry. + * + * A pfn swap entry is a special type of swap entry that always has a pfn = stored + * in the swap offset. They can either be used to represent unaddressable = device + * memory, to restrict access to a page undergoing migration or to represe= nt a + * pfn which has been hwpoisoned and unmapped. + * + * Returns: true if the leaf entry encodes a PFN, otherwise false. + */ +static inline bool softleaf_has_pfn(softleaf_t entry) +{ + /* Make sure the swp offset can always store the needed fields. */ + BUILD_BUG_ON(SWP_TYPE_SHIFT < SWP_PFN_BITS); + + if (softleaf_is_migration(entry)) + return true; + if (softleaf_is_device_private(entry)) + return true; + if (softleaf_is_device_exclusive(entry)) + return true; + if (softleaf_is_hwpoison(entry)) + return true; + + return false; +} + +/** + * softleaf_to_pfn() - Obtain PFN encoded within leaf entry. + * @entry: Leaf entry, softleaf_has_pfn(@entry) must return true. + * + * Returns: The PFN associated with the leaf entry. + */ +static inline unsigned long softleaf_to_pfn(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return swp_offset_pfn(entry); +} + +/** + * softleaf_to_page() - Obtains struct page for PFN encoded within leaf en= try. + * @entry: Leaf entry, softleaf_has_pfn(@entry) must return true. + * + * Returns: Pointer to the struct page associated with the leaf entry's PF= N. + */ +static inline struct page *softleaf_to_page(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return pfn_swap_entry_to_page(entry); +} + +/** + * softleaf_to_folio() - Obtains struct folio for PFN encoded within leaf = entry. + * @entry: Leaf entry, softleaf_has_pfn(@entry) must return true. + * + * Returns: Pointer to the struct folio associated with the leaf entry's P= FN. + */ +static inline struct folio *softleaf_to_folio(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + + /* Temporary until swp_entry_t eliminated. */ + return pfn_swap_entry_folio(entry); +} + +/** + * softleaf_is_poison_marker() - Is this leaf entry a poison marker? + * @entry: Leaf entry. + * + * The poison marker is set via UFFDIO_POISON. Userfaultfd-specific. + * + * Returns: true if the leaf entry is a poison marker, otherwise false. + */ +static inline bool softleaf_is_poison_marker(softleaf_t entry) +{ + if (!softleaf_is_marker(entry)) + return false; + + return softleaf_to_marker(entry) & PTE_MARKER_POISONED; +} + +/** + * softleaf_is_guard_marker() - Is this leaf entry a guard region marker? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a guard marker, otherwise false. + */ +static inline bool softleaf_is_guard_marker(softleaf_t entry) +{ + if (!softleaf_is_marker(entry)) + return false; + + return softleaf_to_marker(entry) & PTE_MARKER_GUARD; +} + +/** + * softleaf_is_uffd_wp_marker() - Is this leaf entry a userfautlfd write p= rotect + * marker? + * @entry: Leaf entry. + * + * Userfaultfd-specific. + * + * Returns: true if the leaf entry is a UFFD WP marker, otherwise false. + */ +static inline bool softleaf_is_uffd_wp_marker(softleaf_t entry) +{ + if (!softleaf_is_marker(entry)) + return false; + + return softleaf_to_marker(entry) & PTE_MARKER_UFFD_WP; +} + +/** + * pte_is_marker() - Does the PTE entry encode a marker leaf entry? + * @pte: PTE entry. + * + * Returns: true if this PTE is a marker leaf entry, otherwise false. + */ +static inline bool pte_is_marker(pte_t pte) +{ + return softleaf_is_marker(softleaf_from_pte(pte)); +} + +/** + * pte_is_uffd_wp_marker() - Does this PTE entry encode a userfaultfd write + * protect marker leaf entry? + * @pte: PTE entry. + * + * Returns: true if this PTE is a UFFD WP marker leaf entry, otherwise fal= se. + */ +static inline bool pte_is_uffd_wp_marker(pte_t pte) +{ + const softleaf_t entry =3D softleaf_from_pte(pte); + + return softleaf_is_uffd_wp_marker(entry); +} + +/** + * pte_is_uffd_marker() - Does this PTE entry encode a userfault-specific = marker + * leaf entry? + * @entry: Leaf entry. + * + * It's useful to be able to determine which leaf entries encode UFFD-spec= ific + * markers so we can handle these correctly. + * + * Returns: true if this PTE entry is a UFFD-specific marker, otherwise fa= lse. + */ +static inline bool pte_is_uffd_marker(pte_t pte) +{ + const softleaf_t entry =3D softleaf_from_pte(pte); + + if (!softleaf_is_marker(entry)) + return false; + + /* UFFD WP, poisoned swap entries are UFFD-handled. */ + if (softleaf_is_uffd_wp_marker(entry)) + return true; + if (softleaf_is_poison_marker(entry)) + return true; + + return false; +} + +#endif /* CONFIG_MMU */ +#endif /* _LINUX_LEAFOPS_H */ diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f6a2b2d20016..ca7a18351797 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -8,7 +8,7 @@ #include #include #include -#include +#include =20 /** * folio_is_file_lru - Should the folio be on a file LRU or anon LRU? @@ -541,9 +541,9 @@ static inline bool mm_tlb_flush_nested(const struct mm_= struct *mm) * The caller should insert a new pte created with make_pte_marker(). */ static inline pte_marker copy_pte_marker( - swp_entry_t entry, struct vm_area_struct *dst_vma) + softleaf_t entry, struct vm_area_struct *dst_vma) { - pte_marker srcm =3D pte_marker_get(entry); + const pte_marker srcm =3D softleaf_to_marker(entry); /* Always copy error entries. */ pte_marker dstm =3D srcm & (PTE_MARKER_POISONED | PTE_MARKER_GUARD); =20 diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 5021047485a9..4f66a3206a63 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -285,6 +285,31 @@ typedef struct { unsigned long val; } swp_entry_t; =20 +/** + * typedef softleaf_t - Describes a page table software leaf entry, abstra= cted + * from its architecture-specific encoding. + * + * Page table leaf entries are those which do not reference any descendent= page + * tables but rather either reference a data page, are an empty (or 'none' + * entry), or contain a non-present entry. + * + * If referencing another page table or a data page then the page table en= try is + * pertinent to hardware - that is it tells the hardware how to decode the= page + * table entry. + * + * Otherwise it is a software-defined leaf page table entry, which this ty= pe + * describes. See leafops.h and specifically @softleaf_type for a list of = all + * possible kinds of software leaf entry. + * + * A softleaf_t entry is abstracted from the hardware page table entry, so= is + * not architecture-specific. + * + * NOTE: While we transition from the confusing swp_entry_t type used for = this + * purpose, we simply alias this type. This will be removed once the + * transition is complete. + */ +typedef swp_entry_t softleaf_t; + #if defined(CONFIG_MEMCG) || defined(CONFIG_SLAB_OBJ_EXT) /* We have some extra room after the refcount in tail pages. */ #define NR_PAGES_IN_LARGE_FOLIO diff --git a/include/linux/swapops.h b/include/linux/swapops.h index d1f665935cfc..0a4b3f51ecf5 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -426,21 +426,6 @@ static inline swp_entry_t make_pte_marker_entry(pte_ma= rker marker) return swp_entry(SWP_PTE_MARKER, marker); } =20 -static inline bool is_pte_marker_entry(swp_entry_t entry) -{ - return swp_type(entry) =3D=3D SWP_PTE_MARKER; -} - -static inline pte_marker pte_marker_get(swp_entry_t entry) -{ - return swp_offset(entry) & PTE_MARKER_MASK; -} - -static inline bool is_pte_marker(pte_t pte) -{ - return is_swap_pte(pte) && is_pte_marker_entry(pte_to_swp_entry(pte)); -} - static inline pte_t make_pte_marker(pte_marker marker) { return swp_entry_to_pte(make_pte_marker_entry(marker)); @@ -451,24 +436,11 @@ static inline swp_entry_t make_poisoned_swp_entry(voi= d) return make_pte_marker_entry(PTE_MARKER_POISONED); } =20 -static inline int is_poisoned_swp_entry(swp_entry_t entry) -{ - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_POISONED); - -} - static inline swp_entry_t make_guard_swp_entry(void) { return make_pte_marker_entry(PTE_MARKER_GUARD); } =20 -static inline int is_guard_swp_entry(swp_entry_t entry) -{ - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_GUARD); -} - static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) { struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index da0b4fcc566f..983c860a00f1 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include =20 @@ -434,32 +434,6 @@ static inline bool userfaultfd_wp_use_markers(struct v= m_area_struct *vma) return userfaultfd_wp_unpopulated(vma); } =20 -static inline bool pte_marker_entry_uffd_wp(swp_entry_t entry) -{ -#ifdef CONFIG_PTE_MARKER_UFFD_WP - return is_pte_marker_entry(entry) && - (pte_marker_get(entry) & PTE_MARKER_UFFD_WP); -#else - return false; -#endif -} - -static inline bool pte_marker_uffd_wp(pte_t pte) -{ -#ifdef CONFIG_PTE_MARKER_UFFD_WP - swp_entry_t entry; - - if (!is_swap_pte(pte)) - return false; - - entry =3D pte_to_swp_entry(pte); - - return pte_marker_entry_uffd_wp(entry); -#else - return false; -#endif -} - /* * Returns true if this is a swap pte and was uffd-wp wr-protected in eith= er * forms (pte marker or a normal swap pte), false otherwise. @@ -473,31 +447,10 @@ static inline bool pte_swp_uffd_wp_any(pte_t pte) if (pte_swp_uffd_wp(pte)) return true; =20 - if (pte_marker_uffd_wp(pte)) + if (pte_is_uffd_wp_marker(pte)) return true; #endif return false; } =20 - -static inline bool is_uffd_pte_marker(pte_t pte) -{ - swp_entry_t entry; - - if (pte_present(pte)) - return false; - - entry =3D pte_to_swp_entry(pte); - if (!is_pte_marker_entry(entry)) - return false; - - /* UFFD WP, poisoned swap entries are UFFD handled. */ - if (pte_marker_entry_uffd_wp(entry)) - return true; - if (is_poisoned_swp_entry(entry)) - return true; - - return false; -} - #endif /* _LINUX_USERFAULTFD_K_H */ diff --git a/mm/hmm.c b/mm/hmm.c index 387a38bbaf6a..e350d0cc9d41 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -249,7 +249,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, * that will be correctly handled, so we need only check for UFFD WP * here. */ - if (pte_none(pte) || pte_marker_uffd_wp(pte)) { + if (pte_none(pte) || pte_is_uffd_wp_marker(pte)) { required_fault =3D hmm_pte_need_fault(hmm_vma_walk, pfn_req_flags, 0); if (required_fault) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 01c784547d1e..a05edefec1ca 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -28,7 +28,7 @@ #include #include #include -#include +#include #include #include #include @@ -5662,17 +5662,17 @@ int copy_hugetlb_page_range(struct mm_struct *dst, = struct mm_struct *src, entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(is_hugetlb_entry_migration(entry))) { - swp_entry_t swp_entry =3D pte_to_swp_entry(entry); + softleaf_t softleaf =3D softleaf_from_pte(entry); bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 - if (!is_readable_migration_entry(swp_entry) && cow) { + if (!is_readable_migration_entry(softleaf) && cow) { /* * COW mappings require pages in both * parent and child to be set to read. */ - swp_entry =3D make_readable_migration_entry( - swp_offset(swp_entry)); - entry =3D swp_entry_to_pte(swp_entry); + softleaf =3D make_readable_migration_entry( + swp_offset(softleaf)); + entry =3D swp_entry_to_pte(softleaf); if (userfaultfd_wp(src_vma) && uffd_wp) entry =3D pte_swp_mkuffd_wp(entry); set_huge_pte_at(src, addr, src_pte, entry, sz); @@ -5680,9 +5680,9 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, if (!userfaultfd_wp(dst_vma)) entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); - } else if (unlikely(is_pte_marker(entry))) { - pte_marker marker =3D copy_pte_marker( - pte_to_swp_entry(entry), dst_vma); + } else if (unlikely(pte_is_marker(entry))) { + const softleaf_t softleaf =3D softleaf_from_pte(entry); + const pte_marker marker =3D copy_pte_marker(softleaf, dst_vma); =20 if (marker) set_huge_pte_at(dst, addr, dst_pte, @@ -5798,7 +5798,7 @@ static void move_huge_pte(struct vm_area_struct *vma,= unsigned long old_addr, =20 pte =3D huge_ptep_get_and_clear(mm, old_addr, src_pte, sz); =20 - if (need_clear_uffd_wp && pte_marker_uffd_wp(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) huge_pte_clear(mm, new_addr, dst_pte, sz); else { if (need_clear_uffd_wp) { @@ -6617,7 +6617,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, * If this pte was previously wr-protected, keep it wr-protected even * if populated. */ - if (unlikely(pte_marker_uffd_wp(vmf->orig_pte))) + if (unlikely(pte_is_uffd_wp_marker(vmf->orig_pte))) new_pte =3D huge_pte_mkuffd_wp(new_pte); set_huge_pte_at(mm, vmf->address, vmf->pte, new_pte, huge_page_size(h)); =20 @@ -6750,9 +6750,9 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct= vm_area_struct *vma, */ return hugetlb_no_page(mapping, &vmf); =20 - if (is_pte_marker(vmf.orig_pte)) { + if (pte_is_marker(vmf.orig_pte)) { const pte_marker marker =3D - pte_marker_get(pte_to_swp_entry(vmf.orig_pte)); + softleaf_to_marker(softleaf_from_pte(vmf.orig_pte)); =20 if (marker & PTE_MARKER_POISONED) { ret =3D VM_FAULT_HWPOISON_LARGE | @@ -7080,7 +7080,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, * See comment about UFFD marker overwriting in * mfill_atomic_install_pte(). */ - if (!huge_pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) + if (!huge_pte_none(dst_ptep) && !pte_is_uffd_marker(dst_ptep)) goto out_release_unlock; =20 if (folio_in_pagecache) @@ -7201,8 +7201,9 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, if (unlikely(is_hugetlb_entry_hwpoisoned(pte))) { /* Nothing to do. */ } else if (unlikely(is_hugetlb_entry_migration(pte))) { - swp_entry_t entry =3D pte_to_swp_entry(pte); - struct folio *folio =3D pfn_swap_entry_folio(entry); + softleaf_t entry =3D softleaf_from_pte(pte); + + struct folio *folio =3D softleaf_to_folio(entry); pte_t newpte =3D pte; =20 if (is_writable_migration_entry(entry)) { @@ -7222,14 +7223,14 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, newpte =3D pte_swp_clear_uffd_wp(newpte); if (!pte_same(pte, newpte)) set_huge_pte_at(mm, address, ptep, newpte, psize); - } else if (unlikely(is_pte_marker(pte))) { + } else if (unlikely(pte_is_marker(pte))) { /* * Do nothing on a poison marker; page is * corrupted, permissions do not apply. Here * pte_marker_uffd_wp()=3D=3Dtrue implies !poison * because they're mutual exclusive. */ - if (pte_marker_uffd_wp(pte) && uffd_wp_resolve) + if (pte_is_uffd_wp_marker(pte) && uffd_wp_resolve) /* Safe to modify directly (non-present->none). */ huge_pte_clear(mm, address, ptep, psize); } else if (!huge_pte_none(pte)) { diff --git a/mm/madvise.c b/mm/madvise.c index fb1c86e630b6..2d5ad3cb37bb 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include =20 @@ -690,17 +690,16 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigne= d long addr, * (page allocation + zeroing). */ if (!pte_present(ptent)) { - swp_entry_t entry; + softleaf_t entry =3D softleaf_from_pte(ptent); =20 - entry =3D pte_to_swp_entry(ptent); - if (!non_swap_entry(entry)) { + if (softleaf_is_swap(entry)) { max_nr =3D (end - addr) / PAGE_SIZE; nr =3D swap_pte_batch(pte, max_nr, ptent); nr_swap -=3D nr; free_swap_and_cache_nr(entry, nr); clear_not_present_full_ptes(mm, addr, pte, nr, tlb->fullmm); - } else if (is_hwpoison_entry(entry) || - is_poisoned_swp_entry(entry)) { + } else if (softleaf_is_hwpoison(entry) || + softleaf_is_poison_marker(entry)) { pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); } continue; @@ -1071,8 +1070,9 @@ static bool is_valid_guard_vma(struct vm_area_struct = *vma, bool allow_locked) =20 static bool is_guard_pte_marker(pte_t ptent) { - return is_swap_pte(ptent) && - is_guard_swp_entry(pte_to_swp_entry(ptent)); + const softleaf_t entry =3D softleaf_from_pte(ptent); + + return softleaf_is_guard_marker(entry); } =20 static int guard_install_pud_entry(pud_t *pud, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index 4c3a7e09a159..7493ed084b99 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -60,7 +60,7 @@ #include #include #include -#include +#include #include #include #include @@ -109,7 +109,7 @@ static __always_inline bool vmf_orig_pte_uffd_wp(struct= vm_fault *vmf) if (!(vmf->flags & FAULT_FLAG_ORIG_PTE_VALID)) return false; =20 - return pte_marker_uffd_wp(vmf->orig_pte); + return pte_is_uffd_wp_marker(vmf->orig_pte); } =20 /* @@ -927,10 +927,10 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, { vm_flags_t vm_flags =3D dst_vma->vm_flags; pte_t orig_pte =3D ptep_get(src_pte); + softleaf_t entry =3D softleaf_from_pte(orig_pte); pte_t pte =3D orig_pte; struct folio *folio; struct page *page; - swp_entry_t entry =3D pte_to_swp_entry(orig_pte); =20 if (likely(!non_swap_entry(entry))) { if (swap_duplicate(entry) < 0) @@ -1016,7 +1016,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, if (try_restore_exclusive_pte(src_vma, addr, src_pte, orig_pte)) return -EBUSY; return -ENOENT; - } else if (is_pte_marker_entry(entry)) { + } else if (softleaf_is_marker(entry)) { pte_marker marker =3D copy_pte_marker(entry, dst_vma); =20 if (marker) @@ -1717,14 +1717,14 @@ static inline int zap_nonpresent_ptes(struct mmu_ga= ther *tlb, unsigned int max_nr, unsigned long addr, struct zap_details *details, int *rss, bool *any_skipped) { - swp_entry_t entry; + softleaf_t entry; int nr =3D 1; =20 *any_skipped =3D true; - entry =3D pte_to_swp_entry(ptent); - if (is_device_private_entry(entry) || - is_device_exclusive_entry(entry)) { - struct page *page =3D pfn_swap_entry_to_page(entry); + entry =3D softleaf_from_pte(ptent); + if (softleaf_is_device_private(entry) || + softleaf_is_device_exclusive(entry)) { + struct page *page =3D softleaf_to_page(entry); struct folio *folio =3D page_folio(page); =20 if (unlikely(!should_zap_folio(details, folio))) @@ -1739,7 +1739,7 @@ static inline int zap_nonpresent_ptes(struct mmu_gath= er *tlb, rss[mm_counter(folio)]--; folio_remove_rmap_pte(folio, page, vma); folio_put(folio); - } else if (!non_swap_entry(entry)) { + } else if (softleaf_is_swap(entry)) { /* Genuine swap entries, hence a private anon pages */ if (!should_zap_cows(details)) return 1; @@ -1747,20 +1747,20 @@ static inline int zap_nonpresent_ptes(struct mmu_ga= ther *tlb, nr =3D swap_pte_batch(pte, max_nr, ptent); rss[MM_SWAPENTS] -=3D nr; free_swap_and_cache_nr(entry, nr); - } else if (is_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + } else if (softleaf_is_migration(entry)) { + struct folio *folio =3D softleaf_to_folio(entry); =20 if (!should_zap_folio(details, folio)) return 1; rss[mm_counter(folio)]--; - } else if (pte_marker_entry_uffd_wp(entry)) { + } else if (softleaf_is_uffd_wp_marker(entry)) { /* * For anon: always drop the marker; for file: only * drop the marker if explicitly requested. */ if (!vma_is_anonymous(vma) && !zap_drop_markers(details)) return 1; - } else if (is_guard_swp_entry(entry)) { + } else if (softleaf_is_guard_marker(entry)) { /* * Ordinary zapping should not remove guard PTE * markers. Only do so if we should remove PTE markers @@ -1768,7 +1768,8 @@ static inline int zap_nonpresent_ptes(struct mmu_gath= er *tlb, */ if (!zap_drop_markers(details)) return 1; - } else if (is_hwpoison_entry(entry) || is_poisoned_swp_entry(entry)) { + } else if (softleaf_is_hwpoison(entry) || + softleaf_is_poison_marker(entry)) { if (!should_zap_cows(details)) return 1; } else { @@ -4390,7 +4391,7 @@ static vm_fault_t pte_marker_clear(struct vm_fault *v= mf) * * This should also cover the case where e.g. the pte changed * quickly from a PTE_MARKER_UFFD_WP into PTE_MARKER_POISONED. - * So is_pte_marker() check is not enough to safely drop the pte. + * So pte_is_marker() check is not enough to safely drop the pte. */ if (pte_same(vmf->orig_pte, ptep_get(vmf->pte))) pte_clear(vmf->vma->vm_mm, vmf->address, vmf->pte); @@ -4424,8 +4425,8 @@ static vm_fault_t pte_marker_handle_uffd_wp(struct vm= _fault *vmf) =20 static vm_fault_t handle_pte_marker(struct vm_fault *vmf) { - swp_entry_t entry =3D pte_to_swp_entry(vmf->orig_pte); - unsigned long marker =3D pte_marker_get(entry); + const softleaf_t entry =3D softleaf_from_pte(vmf->orig_pte); + const pte_marker marker =3D softleaf_to_marker(entry); =20 /* * PTE markers should never be empty. If anything weird happened, @@ -4442,7 +4443,7 @@ static vm_fault_t handle_pte_marker(struct vm_fault *= vmf) if (marker & PTE_MARKER_GUARD) return VM_FAULT_SIGSEGV; =20 - if (pte_marker_entry_uffd_wp(entry)) + if (softleaf_is_uffd_wp_marker(entry)) return pte_marker_handle_uffd_wp(vmf); =20 /* This is an unknown pte marker */ @@ -4690,7 +4691,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } } else if (is_hwpoison_entry(entry)) { ret =3D VM_FAULT_HWPOISON; - } else if (is_pte_marker_entry(entry)) { + } else if (softleaf_is_marker(entry)) { ret =3D handle_pte_marker(vmf); } else { print_bad_pte(vma, vmf->address, vmf->orig_pte, NULL); diff --git a/mm/mincore.c b/mm/mincore.c index fb80becd6119..b3682488a65d 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -14,7 +14,7 @@ #include #include #include -#include +#include #include #include #include @@ -42,7 +42,7 @@ static int mincore_hugetlb(pte_t *pte, unsigned long hmas= k, unsigned long addr, } else { const pte_t ptep =3D huge_ptep_get(walk->mm, addr, pte); =20 - if (huge_pte_none(ptep) || is_pte_marker(ptep)) + if (huge_pte_none(ptep) || pte_is_marker(ptep)) present =3D 0; else present =3D 1; @@ -187,7 +187,7 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, =20 step =3D 1; /* We need to do cache lookup too for markers */ - if (pte_none(pte) || is_pte_marker(pte)) + if (pte_none(pte) || pte_is_marker(pte)) __mincore_unmapped_range(addr, addr + PAGE_SIZE, vma, vec); else if (pte_present(pte)) { diff --git a/mm/mprotect.c b/mm/mprotect.c index ab4e06cd9a69..0bae241eb7aa 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -326,14 +326,14 @@ static long change_pte_range(struct mmu_gather *tlb, newpte =3D swp_entry_to_pte(entry); if (pte_swp_uffd_wp(oldpte)) newpte =3D pte_swp_mkuffd_wp(newpte); - } else if (is_pte_marker_entry(entry)) { + } else if (softleaf_is_marker(entry)) { /* * Ignore error swap entries unconditionally, * because any access should sigbus/sigsegv * anyway. */ - if (is_poisoned_swp_entry(entry) || - is_guard_swp_entry(entry)) + if (softleaf_is_poison_marker(entry) || + softleaf_is_guard_marker(entry)) continue; /* * If this is uffd-wp pte marker and we'd like diff --git a/mm/mremap.c b/mm/mremap.c index 8ad06cf50783..7c21b2ad13f6 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -17,7 +17,7 @@ #include #include #include -#include +#include #include #include #include @@ -288,7 +288,7 @@ static int move_ptes(struct pagetable_move_control *pmc, pte =3D move_pte(pte, old_addr, new_addr); pte =3D move_soft_dirty_pte(pte); =20 - if (need_clear_uffd_wp && pte_marker_uffd_wp(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) pte_clear(mm, new_addr, new_ptep); else { if (need_clear_uffd_wp) { diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 137ce27ff68c..be20468fb5a9 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -3,7 +3,7 @@ #include #include #include -#include +#include =20 #include "internal.h" =20 @@ -107,15 +107,12 @@ static bool check_pte(struct page_vma_mapped_walk *pv= mw, unsigned long pte_nr) pte_t ptent =3D ptep_get(pvmw->pte); =20 if (pvmw->flags & PVMW_MIGRATION) { - swp_entry_t entry; - if (!is_swap_pte(ptent)) - return false; - entry =3D pte_to_swp_entry(ptent); + const softleaf_t entry =3D softleaf_from_pte(ptent); =20 - if (!is_migration_entry(entry)) + if (!softleaf_is_migration(entry)) return false; =20 - pfn =3D swp_offset_pfn(entry); + pfn =3D softleaf_to_pfn(entry); } else if (is_swap_pte(ptent)) { swp_entry_t entry; =20 diff --git a/mm/shmem.c b/mm/shmem.c index 6580f3cd24bb..395ca58ac4a5 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -66,7 +66,7 @@ static struct vfsmount *shm_mnt __ro_after_init; #include #include #include -#include +#include #include #include #include @@ -2286,7 +2286,8 @@ static int shmem_swapin_folio(struct inode *inode, pg= off_t index, struct address_space *mapping =3D inode->i_mapping; struct mm_struct *fault_mm =3D vma ? vma->vm_mm : NULL; struct shmem_inode_info *info =3D SHMEM_I(inode); - swp_entry_t swap, index_entry; + swp_entry_t swap; + softleaf_t index_entry; struct swap_info_struct *si; struct folio *folio =3D NULL; bool skip_swapcache =3D false; @@ -2298,7 +2299,7 @@ static int shmem_swapin_folio(struct inode *inode, pg= off_t index, swap =3D index_entry; *foliop =3D NULL; =20 - if (is_poisoned_swp_entry(index_entry)) + if (softleaf_is_poison_marker(index_entry)) return -EIO; =20 si =3D get_swap_device(index_entry); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index cc4ce205bbec..055ec1050776 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -10,7 +10,7 @@ #include #include #include -#include +#include #include #include #include @@ -208,7 +208,7 @@ int mfill_atomic_install_pte(pmd_t *dst_pmd, * MISSING|WP registered, we firstly wr-protect a none pte which has no * page cache page backing it, then access the page. */ - if (!pte_none(dst_ptep) && !is_uffd_pte_marker(dst_ptep)) + if (!pte_none(dst_ptep) && !pte_is_uffd_marker(dst_ptep)) goto out_unlock; =20 if (page_in_cache) { @@ -590,7 +590,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb( if (!uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) { const pte_t ptep =3D huge_ptep_get(dst_mm, dst_addr, dst_pte); =20 - if (!huge_pte_none(ptep) && !is_uffd_pte_marker(ptep)) { + if (!huge_pte_none(ptep) && !pte_is_uffd_marker(ptep)) { err =3D -EEXIST; hugetlb_vma_unlock_read(dst_vma); mutex_unlock(&hugetlb_fault_mutex_table[hash]); --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35813337BA6; Mon, 10 Nov 2025 22:24:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813459; cv=fail; b=ttCxfWSX6aoj3Va07bVX6ZnBjdpAaFwKzBsbgj+vcjudxM+WFdrgPrFMDKAAxUNwPLCwpWR+0Ahf9DeloTQw7FvNG6x+QU+wRU7Hu9gs/ZRMtzy79zwLhBp45dLewbWJs8BIvfxeVvC0G/1jYcQhTz0cx0W9xMq5+PJqN19Gl6I= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813459; c=relaxed/simple; bh=T68qemNrU9aXNg9BwcHhuUskC8JepUns48gZSOBTgIY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=JsFUjnLHwEh7yoLb4dWa9ZyVb5o5EM2kklDHTz0ZUG+gKtO+LsE+s6kPfmCTrPoM4Usuyt2kJqvEc49jMjji7XOyQOPl4UXQrARR1+2K1xLMbwlEg6NRpfzpJdnmVm5wLnnJ6NdhMYtu2wGLgMimBxuZ14uVQMPZ1seCyS0XvGk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=IFudrTzO; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=NTZ8zVvx; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="IFudrTzO"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="NTZ8zVvx" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAL8hmP009313; Mon, 10 Nov 2025 22:21:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=nYYuTMwRkRPS0ue5pIa/JHKJQGWQ3ZtYL9FYx6anL54=; b= IFudrTzOO+hppA+LjraInZ7zQ97TLrUBezN/8ShnlXnecLP8kvAwE+TyFCNli+Pw RTC2Zs42I/ACdUS6xbQSosJh2qN8oi+TyPI0YKl8/KPSeO+1rXjXONFMHwQWKSpm iiZ9GT0FhPDAAKJv/RyyvUsGenBl0124/rlC72pDtxh/ou8trY2DvUKYr/ZBAa7K byKwy81tyuP7QkgHlRplyTKPGVMGNtu78mqsT3Over9wMD2pGXQqIqo2ohJ02UBS D44qw9Ox0kCPsy+TCMkbgsynl7xaRNBHU5wIFjO4cf52eLR1ulofr7uNgXsiDd90 K7tMWV7N+y4/3x9bik8dRQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abqpug5ru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:50 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAL8Wxn012637; Mon, 10 Nov 2025 22:21:48 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010031.outbound.protection.outlook.com [52.101.201.31]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vac8sq7-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:48 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=l1XOQVn/GIpJWuiyP7pD2Q/i7FuS5BF/mKA5S3pCJNOL0SkxsDAB2HXeu9LkoNY7DZQHgXr2AtDV+2KH0Z/o8++WXt3mvzW1MDcpQuPrm7O4K8H8VHDdtRtv0HkmCkQOXg1HDYgq00vWQcAnmbLPrJ3u5GXC9K/CN0JHbk+Ob0ARiiIO/rL43MInUYnWprnW+rQuX4pRZgPbB/nJ3iVe17eR6n4gnXrDv0dRS2zNBrwe+Yyo5E+pOqa/v1XkGygrEIKBhZ1pzvFSx+j9Eba6HLevs59nN3Y9/7WlRZfSCs8eCA/+l3h9m0gj6Ru2l/HMWAQmppwAOUoeJk0BrK0mbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nYYuTMwRkRPS0ue5pIa/JHKJQGWQ3ZtYL9FYx6anL54=; b=f+NxEZ8fqd1tV9LvT3rXM7aF6tzHFwR48dIkhQR6L8iWOBQW2wFwiwnR6/Rjb+l7GBuRr/h/CPxb2mheS1rec9r1g/l3NKzzjMp1Se0NKt684PmEIEAisIvXoVRdRosWz/U5dF68xB8/G3+G2xuoAjMwIE8OLkhYsYl+pKiQ+evg2+I6xwlapQksOz8QCV8L0+q9jM4hy0YZA0PkxbVpUXVDhkhzai9jYAJ65ouyb8lbEGQIB7vkI4qY7mn6USVQti2yBKqsAeX4zFXoel8usVrG/m1A5mUxf/X5r6k0kXyJXB4IFhGjx/lqH3hNtcVwElHU6+R/7SeQjU4qcDnLXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nYYuTMwRkRPS0ue5pIa/JHKJQGWQ3ZtYL9FYx6anL54=; b=NTZ8zVvx4sYo3cVCuHrErKCiMUT1twVR1SL6gdGQPj6+tQTl1kg/Hd4g/OCLbSO0NMsjAooCiy6dQvZOnsWrOMxoqadjFTBChUeCDUpSu3Nc5TQzIsvtQhGZoPm/36BN+hwmobH1EZ95A2mi66WkayUO9hx3xdXn9/qmOEb+gHc= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:44 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:44 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 03/16] mm: avoid unnecessary uses of is_swap_pte() Date: Mon, 10 Nov 2025 22:21:21 +0000 Message-ID: <17fd6d7f46a846517fd455fadd640af47fcd7c55.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0016.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::28) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: dd7ab494-5256-4354-8a85-08de20a7878c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ztwg7vIpxGWz4ZdCpAeDrPTVfl+5VT8htDKDKnB0G8lUvCtc9mnc7PfBSViQ?= =?us-ascii?Q?01ww2bDJ+ayjccG6Qw3HewvTlADwrTHfdD0zkswFmhiyT9xxkaJTrAUA7IBU?= =?us-ascii?Q?Z0ZNxsYO/heGVxVi6pW2cdYXjOuL7GImUW3RmU2ABwr8jC/T1fS+sPUwzhUr?= =?us-ascii?Q?Fli+EQ4SQSIjIrhVOzdO3au1KZpWx0Sgs9gnYOcBVBTcRyKTJVmmCxrdrT7M?= =?us-ascii?Q?7oKBYHOEZ/I4WWjhFrxEcLUEOd0ml8kEOvras29DClvEjE/d+peLKHTev3c2?= =?us-ascii?Q?YXQctnOGADUvaARC/B//0eXdvm2eT00z5uKym75neciWrep7ze2h1khWrEL+?= =?us-ascii?Q?a9rpfnxR5fuLHHe6i2AaHdH/cHbL2+BbRZUwB8pp0Vsr8Yl9AyK2umYp1qMa?= =?us-ascii?Q?zXpSUimzWSUubeQa/GFBQJJ4PQwcmCE7VuTUbDWsSb4cPP8qHXb4AWREA/pb?= =?us-ascii?Q?IXzMtjJ5Na8mh/gUNrs2YVnc8vCDPLiUP8eOwyIcEpMN6T1Ez0XC3XwG1BP/?= =?us-ascii?Q?mtnL3rKLR9/FLHYcoL7myPlMZFOd1zPGPryFgRfxg66CdBSr+NsH49FmWZ/c?= =?us-ascii?Q?+U4K01yHQkS3nSTtyGZueMuFEJ/xpOSyJb6Lnq0S4smlsryep4tYChWHleSH?= =?us-ascii?Q?Ac9MwCHcHn05psHWlHPdXW2EU84WW3hpYURPpKGkd8ncwTR+LeNVOaiciXYF?= =?us-ascii?Q?cnvw6reMXJCLFoRVaAzT1ZEwNbWcDnEXQEb2BddCbEU2AdnpJ4hJlk1DUfy8?= =?us-ascii?Q?7NZxoOXAerGRHvA12DivwPQSX/2ocVLBctDvvBnjfOrr2dWer4qebdCdj2rX?= =?us-ascii?Q?isjY0Fr5AJVHlkxYuAI8e5yNBedLsllzfGnTAcmt/KuRYtWJlaJx9TizGm58?= =?us-ascii?Q?q+Yn4/h9zXrHV9rDQVJEs9PpJlAy+VUrZj/hTttpYJSFb94+DRhGEBmcIEPA?= =?us-ascii?Q?riw+6aZ0P/fQ41yNgVppG8MtBSg6MIk2L/cUwXnmJsBzf2KrvElfFZnQhNzN?= =?us-ascii?Q?triWfW2uvo5BL++lJe/hdFy2kJ4fovAHMftJWiJ7t+eqwNY0n3VU0FCo7mQE?= =?us-ascii?Q?GA+eJMd85gqn1p7RF/wFiqqttgdSBJuVfsNLyX5HANFhvYB5tcCfoZrVTg0Q?= =?us-ascii?Q?/p8VKyrx3WgKFEK5RgEkTlH0DHhKflTbjs24BpacX9MKWa2EsDJzBJQ0ArGi?= =?us-ascii?Q?tAT3I6C2CVpdWeiK4tKDuGqsnn8Iuq35sW/atp45LUvpXtMFAMtybaNkZmwa?= =?us-ascii?Q?QXzOKAHYjTM0KwsFPj2Hq1aznb07n8mm6hJhcwjyIBtByqXpSqe+yAKjM+Sr?= =?us-ascii?Q?Lhtn1hkfml7fTqIO8JBr0Tj7yPk0eQ3PyfgVvk8/LQLglyhPJ07zuQemVKCw?= =?us-ascii?Q?0saaSNndhhMFtphjDXuLqQ7OnVecq/4R2+1tB58d3JeH9Ah5C0F/eKoRtxr2?= =?us-ascii?Q?NDU5XH7UcT76oAY84mCJwestjO6PzCPQ?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?OGC8Vavjvpncuh2ZhCGTHS+XVtuIWK61bC08dfhvo2MhfmomfQMv2kzLeVF/?= =?us-ascii?Q?YTuJLXfIscZgw433mQjbaGOQpfyPaKj3E6kCu5gVPl8wUNLKr0x/yhObpVDo?= =?us-ascii?Q?2v61FksTHasxZunTknYm/j1UqtLSa9pkClrZJjwr9hNaVPrBicYHMh0sqqb0?= =?us-ascii?Q?pid9og2rymJtLmg68DK1CjmkHGuW0mRh1ForXHetzNtgTRphCQ2BLFNar4ow?= =?us-ascii?Q?6QbnC71JuRmQKfr9dq3UgW7UBTvq5e6v1fUUo3obCIwesQN3LLvLtkIvHE0i?= =?us-ascii?Q?3DJFjldvkbcweD+Ec0a4WyHjm48W7KekmlZozEx5DMBHDwIPiC0DQfLxcorg?= =?us-ascii?Q?1/2FTXgh9EAj2NQqAG4VNtzncoInNKhDhYtKUJkfKhyx+GifNejJBHCGSYf9?= =?us-ascii?Q?J9i/W7/QF+85I3kM5Q+mOU+wKJF8p6oCYBYziynRBCyvIxIA6pM4yZgGURqb?= =?us-ascii?Q?JAsPVDK9MNwE8+tUIWnbF5ta2OH1SBsbxFxtRjdHvxMJDELDGELJkIPsXYuR?= =?us-ascii?Q?tan5KBeoDneIxLL/NMBPFqgmy1WWe6y1gP1OG3BlDSOF+VJG6Who74ihJtw2?= =?us-ascii?Q?IDYTLM2DeXRRR5qsS2lI077/vO4fP9Y0t5PqWoju/gNKkOarH3XkDOvYOxUV?= =?us-ascii?Q?YopcYRP3jzEjqMsg1jDsrBXBXXNCnHUHjF+ShdeL1PvAQm4N/zUd7gu7pl1A?= =?us-ascii?Q?5JZoBJ7o9xAi967zxsxgRVaPusew5rU5Jrzc4HvP+nA0ByzwbXwbBtT9Ddw3?= =?us-ascii?Q?2jPMEn0KWcn/YnE0pB+pT+Nc2nAtez6FlEBbohhCUWLEjto/RM9k4NW+Yaoe?= =?us-ascii?Q?Xk9WAoPCqsT3Ob+dG1Rko28XtRlQKoVVcaMmZcsA6rgpwUvuDI3uWpJNgXs6?= =?us-ascii?Q?5yddI/6Vn/SZfpWld5oVGqM+Y+/M20hCptCsduygyrfoUFpT0h0SSRV8tBSw?= =?us-ascii?Q?ltyrh8SKvNzPvKBDeEQYCYm/2UJJDsqXaditzQG5lmBjIduSHUIZeh/Zx3G9?= =?us-ascii?Q?GV9jgiRlb9oJo5oMDA16HOnPxdaj04L8jQMVC5nOQ7P062frXNLdgt0Pgfbd?= =?us-ascii?Q?KIP2VV2c06Zr/+CLTPvQTDJ9M5zrA9FmwR/TLyCw+IAWz0TPGQ29KtHwjUEh?= =?us-ascii?Q?VXXSjK/dRZjiFgvEDlmbkHBZXlQylPNQBWcu6z+RI9vL4Dh8N9SZo8W1ZxQD?= =?us-ascii?Q?1XyZ1rpol8LhkYheUHTu1szSXwCRaDTx2WX5u9uJeRrSUPPj2+RAjlgqWSLL?= =?us-ascii?Q?zjGVwMHftyUPOjKPQhXREJ40yo/HbJXcB8xsEGnnzm1xYAt9mNFAFrsVcNuC?= =?us-ascii?Q?v0xOmiQeh0U9Dapv96r8T5MZF056arpFVnGIZyzYql2tqU5QcWDxffNWZWWn?= =?us-ascii?Q?UumFMA9CKPyB1FlYSOQ9cuzFIUM6RwOmRtPchfZTlq6233kX6692oA2emLqA?= =?us-ascii?Q?ec7g8EnBKpZuW3gUavdLl8NlFyC41RgVufdZPTYyDPwVBk8XADj1roGtui7K?= =?us-ascii?Q?B55PCjYlHFQg0NAmyFjXsVAN1bKF+FsPKGLromnNFl0zUyPtdZvyaQ0Tb4oH?= =?us-ascii?Q?xfShORIOr3+aenPbx2FqHsdqPnNF6WBqQDmGBXcL67G252oubsRT35VGBeoz?= =?us-ascii?Q?xg=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: EVky+rKXSMgX6LVljAeq+xD7AK1h3xfxz8sDDDTk/AUD6PX0GYOheVgxTK0CP3JOx6IqAQVOMVjQyAkoiaWL9EVaRxyVT7hA5SAPpk+DidYYLHNArP/PV1QkD89Nn8t6pDI0gaqbFBxHfgYib0LOtwWnM8WqoNduI/A6yT0gPC4RMjxvQCuzomht4NfZhK3NMq79PzT5OG4nQHRtVWf1v2G4fD2G5e1jIvJOYEdzHxPOeuMlnSoRa5vPlQYSSRAcmF1AU58++a27NKHyEQ3jPfpzVYUQBs/CjI4kh55/xlv5z8iH80yLUDGcIMNrCG8krwk3VK/dxfIj8742SyZPm4SwEcYjj64DF+E4hpXkd1lj13MWFJaOhVb1h+VKxTzrOxH7KF0He3hXKhi3wBGGz9OHP2dT0fRWmyHCx6XMiQLWKu3cF77JXQcKFWmEOSYPvliMULfmRvTqlmNvJiXEQ50hjL1AUZs17Iw+WmutytDpW8Dpwuf3WbGw6F1sWRwQ6jA/p4eZVAEfot777UeNaspKwTeXlHj0S0+IdZ857dtdE64NvTBe7opeAtRASz6gVTx+/c/WznC6tWFqFOpFN4J8nQ5ubD3UqEWD++IAAoo= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd7ab494-5256-4354-8a85-08de20a7878c X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:44.0659 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: CjS2tF6VKyKW/RTZ/T39Dq1I1BcrWm9bp9ypKoUXbLEgoEY4h2/J0D5t1Hj0yljVCPhGpAg/pUnMcuSG/cuOGcSxSiRdM2HBnMGvmIt9ZiQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0 spamscore=0 phishscore=0 suspectscore=0 adultscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE3OSBTYWx0ZWRfX2vgdA1HuF8UP mdn/USE6yqSXesFmcRSlc3CWupQy6ndMlnv/t5O7VBjSombWqyMVbBepZ/LJ4nuylDBp/Oe8AzH /tDNEOgljosh7RtPXVc2poGLq6Y0MIFmJknTeKwFc6WfZ+Kv45g2vHE3ANQD+RXJK4ZA51JpPM8 7+oQ9cyJBbmDI+tokAJLupu2QSbDykS17qxqKYgtDoJyc4DjI2X/5ADCOc/cWvjWz67sGRxhETr I60o/sghrHDhbVHnnyQnLynvkYGXl2FBpkhbSlJn0a0j9qJ1h0p4FNo12hfLAhq3a+vaMFErH3g 5aKvRcMPv6vjhva9BbFH+C9ROBfjAtNhzfCgzSgf84hGwrJfAg16vmoF8pTYVtz5i4e8TNzfkrm lNF3BmFDRcyb2IsrKr+7Ek+3cTHjPkigkGlZepQuo3yjZIN6I9w= X-Authority-Analysis: v=2.4 cv=H5rWAuYi c=1 sm=1 tr=0 ts=6912657e b=1 cx=c_pps a=zPCbziy225d3KhSqZt3L1A==:117 a=zPCbziy225d3KhSqZt3L1A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=360PBzQsUBXz8B3xgqMA:9 cc=ntf awl=host:12099 X-Proofpoint-ORIG-GUID: DhBPEGfWLUdPm6e88BTve8SJ9KafYEr8 X-Proofpoint-GUID: DhBPEGfWLUdPm6e88BTve8SJ9KafYEr8 Content-Type: text/plain; charset="utf-8" There's an established convention in the kernel that we treat PTEs as containing swap entries (and the unfortunately named non-swap swap entries) should they be neither empty (i.e. pte_none() evaluating true) nor present (i.e. pte_present() evaluating true). However, there is some inconsistency in how this is applied, as we also have the is_swap_pte() helper which explicitly performs this check: /* check whether a pte points to a swap entry */ static inline int is_swap_pte(pte_t pte) { return !pte_none(pte) && !pte_present(pte); } As this represents a predicate, and it's logical to assume that in order to establish that a PTE entry can correctly be manipulated as a swap/non-swap entry, this predicate seems as if it must first be checked. But we instead, we far more often utilise the established convention of checking pte_none() / pte_present() before operating on entries as if they were swap/non-swap. This patch works towards correcting this inconsistency by removing all uses of is_swap_pte() where we are already in a position where we perform pte_none()/pte_present() checks anyway or otherwise it is clearly logical to do so. We also take advantage of the fact that pte_swp_uffd_wp() is only set on swap entries. Additionally, update comments referencing to is_swap_pte() and non_swap_entry(). No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- fs/proc/task_mmu.c | 49 ++++++++++++++++++++++++----------- include/linux/userfaultfd_k.h | 3 +-- mm/hugetlb.c | 6 ++--- mm/internal.h | 6 ++--- mm/khugepaged.c | 29 +++++++++++---------- mm/migrate.c | 2 +- mm/mprotect.c | 43 ++++++++++++++---------------- mm/mremap.c | 7 +++-- mm/page_table_check.c | 13 ++++++---- mm/page_vma_mapped.c | 31 +++++++++++----------- 10 files changed, 104 insertions(+), 85 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 24d26b49d870..ddbf177ecc45 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1017,7 +1017,9 @@ static void smaps_pte_entry(pte_t *pte, unsigned long= addr, young =3D pte_young(ptent); dirty =3D pte_dirty(ptent); present =3D true; - } else if (is_swap_pte(ptent)) { + } else if (pte_none(ptent)) { + smaps_pte_hole_lookup(addr, walk); + } else { swp_entry_t swpent =3D pte_to_swp_entry(ptent); =20 if (!non_swap_entry(swpent)) { @@ -1038,9 +1040,6 @@ static void smaps_pte_entry(pte_t *pte, unsigned long= addr, present =3D true; page =3D pfn_swap_entry_to_page(swpent); } - } else { - smaps_pte_hole_lookup(addr, walk); - return; } =20 if (!page) @@ -1611,6 +1610,9 @@ static inline void clear_soft_dirty(struct vm_area_st= ruct *vma, */ pte_t ptent =3D ptep_get(pte); =20 + if (pte_none(ptent)) + return; + if (pte_present(ptent)) { pte_t old_pte; =20 @@ -1620,7 +1622,7 @@ static inline void clear_soft_dirty(struct vm_area_st= ruct *vma, ptent =3D pte_wrprotect(old_pte); ptent =3D pte_clear_soft_dirty(ptent); ptep_modify_prot_commit(vma, addr, pte, old_pte, ptent); - } else if (is_swap_pte(ptent)) { + } else { ptent =3D pte_swp_clear_soft_dirty(ptent); set_pte_at(vma->vm_mm, addr, pte, ptent); } @@ -1923,6 +1925,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, struct page *page =3D NULL; struct folio *folio; =20 + if (pte_none(pte)) + goto out; + if (pte_present(pte)) { if (pm->show_pfn) frame =3D pte_pfn(pte); @@ -1932,8 +1937,9 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, flags |=3D PM_SOFT_DIRTY; if (pte_uffd_wp(pte)) flags |=3D PM_UFFD_WP; - } else if (is_swap_pte(pte)) { + } else { swp_entry_t entry; + if (pte_swp_soft_dirty(pte)) flags |=3D PM_SOFT_DIRTY; if (pte_swp_uffd_wp(pte)) @@ -1941,6 +1947,7 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, entry =3D pte_to_swp_entry(pte); if (pm->show_pfn) { pgoff_t offset; + /* * For PFN swap offsets, keeping the offset field * to be PFN only to be compatible with old smaps. @@ -1969,6 +1976,8 @@ static pagemap_entry_t pte_to_pagemap_entry(struct pa= gemapread *pm, __folio_page_mapped_exclusively(folio, page)) flags |=3D PM_MMAP_EXCLUSIVE; } + +out: if (vma->vm_flags & VM_SOFTDIRTY) flags |=3D PM_SOFT_DIRTY; =20 @@ -2310,12 +2319,16 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, struct vm_area_struct *vma, unsigned long addr, pte_t pte) { - unsigned long categories =3D 0; + unsigned long categories; + + if (pte_none(pte)) + return 0; =20 if (pte_present(pte)) { struct page *page; =20 - categories |=3D PAGE_IS_PRESENT; + categories =3D PAGE_IS_PRESENT; + if (!pte_uffd_wp(pte)) categories |=3D PAGE_IS_WRITTEN; =20 @@ -2329,10 +2342,11 @@ static unsigned long pagemap_page_category(struct p= agemap_scan_private *p, categories |=3D PAGE_IS_PFNZERO; if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pte(pte)) { + } else { softleaf_t entry; =20 - categories |=3D PAGE_IS_SWAPPED; + categories =3D PAGE_IS_SWAPPED; + if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; =20 @@ -2360,12 +2374,12 @@ static void make_uffd_wp_pte(struct vm_area_struct = *vma, old_pte =3D ptep_modify_prot_start(vma, addr, pte); ptent =3D pte_mkuffd_wp(old_pte); ptep_modify_prot_commit(vma, addr, pte, old_pte, ptent); - } else if (is_swap_pte(ptent)) { - ptent =3D pte_swp_mkuffd_wp(ptent); - set_pte_at(vma->vm_mm, addr, pte, ptent); - } else { + } else if (pte_none(ptent)) { set_pte_at(vma->vm_mm, addr, pte, make_pte_marker(PTE_MARKER_UFFD_WP)); + } else { + ptent =3D pte_swp_mkuffd_wp(ptent); + set_pte_at(vma->vm_mm, addr, pte, ptent); } } =20 @@ -2434,6 +2448,9 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) { unsigned long categories =3D PAGE_IS_HUGE; =20 + if (pte_none(pte)) + return categories; + /* * According to pagemap_hugetlb_range(), file-backed HugeTLB * page cannot be swapped. So PAGE_IS_FILE is not checked for @@ -2441,6 +2458,7 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) */ if (pte_present(pte)) { categories |=3D PAGE_IS_PRESENT; + if (!huge_pte_uffd_wp(pte)) categories |=3D PAGE_IS_WRITTEN; if (!PageAnon(pte_page(pte))) @@ -2449,8 +2467,9 @@ static unsigned long pagemap_hugetlb_category(pte_t p= te) categories |=3D PAGE_IS_PFNZERO; if (pte_soft_dirty(pte)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pte(pte)) { + } else { categories |=3D PAGE_IS_SWAPPED; + if (!pte_swp_uffd_wp_any(pte)) categories |=3D PAGE_IS_WRITTEN; if (pte_swp_soft_dirty(pte)) diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index 983c860a00f1..96b089dff4ef 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -441,9 +441,8 @@ static inline bool userfaultfd_wp_use_markers(struct vm= _area_struct *vma) static inline bool pte_swp_uffd_wp_any(pte_t pte) { #ifdef CONFIG_PTE_MARKER_UFFD_WP - if (!is_swap_pte(pte)) + if (pte_present(pte)) return false; - if (pte_swp_uffd_wp(pte)) return true; =20 diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a05edefec1ca..a74cde267c2a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5798,13 +5798,13 @@ static void move_huge_pte(struct vm_area_struct *vm= a, unsigned long old_addr, =20 pte =3D huge_ptep_get_and_clear(mm, old_addr, src_pte, sz); =20 - if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) + if (need_clear_uffd_wp && pte_is_uffd_wp_marker(pte)) { huge_pte_clear(mm, new_addr, dst_pte, sz); - else { + } else { if (need_clear_uffd_wp) { if (pte_present(pte)) pte =3D huge_pte_clear_uffd_wp(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_clear_uffd_wp(pte); } set_huge_pte_at(mm, new_addr, dst_pte, pte, sz); diff --git a/mm/internal.h b/mm/internal.h index 116a1ba85e66..9465129367a4 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -325,8 +325,7 @@ unsigned int folio_pte_batch(struct folio *folio, pte_t= *ptep, pte_t pte, /** * pte_move_swp_offset - Move the swap entry offset field of a swap pte * forward or backward by delta - * @pte: The initial pte state; is_swap_pte(pte) must be true and - * non_swap_entry() must be false. + * @pte: The initial pte state; must be a swap entry * @delta: The direction and the offset we are moving; forward if delta * is positive; backward if delta is negative * @@ -352,8 +351,7 @@ static inline pte_t pte_move_swp_offset(pte_t pte, long= delta) =20 /** * pte_next_swp_offset - Increment the swap entry offset field of a swap p= te. - * @pte: The initial pte state; is_swap_pte(pte) must be true and - * non_swap_entry() must be false. + * @pte: The initial pte state; must be a swap entry. * * Increments the swap offset, while maintaining all other fields, includi= ng * swap type, and any swp pte bits. The resulting pte is returned. diff --git a/mm/khugepaged.c b/mm/khugepaged.c index f6ed1072ed6e..a97ff7bcb232 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1019,7 +1019,8 @@ static int __collapse_huge_page_swapin(struct mm_stru= ct *mm, } =20 vmf.orig_pte =3D ptep_get_lockless(pte); - if (!is_swap_pte(vmf.orig_pte)) + if (pte_none(vmf.orig_pte) || + pte_present(vmf.orig_pte)) continue; =20 vmf.pte =3D pte; @@ -1276,7 +1277,19 @@ static int hpage_collapse_scan_pmd(struct mm_struct = *mm, for (addr =3D start_addr, _pte =3D pte; _pte < pte + HPAGE_PMD_NR; _pte++, addr +=3D PAGE_SIZE) { pte_t pteval =3D ptep_get(_pte); - if (is_swap_pte(pteval)) { + if (pte_none_or_zero(pteval)) { + ++none_or_zero; + if (!userfaultfd_armed(vma) && + (!cc->is_khugepaged || + none_or_zero <=3D khugepaged_max_ptes_none)) { + continue; + } else { + result =3D SCAN_EXCEED_NONE_PTE; + count_vm_event(THP_SCAN_EXCEED_NONE_PTE); + goto out_unmap; + } + } + if (!pte_present(pteval)) { ++unmapped; if (!cc->is_khugepaged || unmapped <=3D khugepaged_max_ptes_swap) { @@ -1296,18 +1309,6 @@ static int hpage_collapse_scan_pmd(struct mm_struct = *mm, goto out_unmap; } } - if (pte_none_or_zero(pteval)) { - ++none_or_zero; - if (!userfaultfd_armed(vma) && - (!cc->is_khugepaged || - none_or_zero <=3D khugepaged_max_ptes_none)) { - continue; - } else { - result =3D SCAN_EXCEED_NONE_PTE; - count_vm_event(THP_SCAN_EXCEED_NONE_PTE); - goto out_unmap; - } - } if (pte_uffd_wp(pteval)) { /* * Don't collapse the page if any of the small diff --git a/mm/migrate.c b/mm/migrate.c index ceee354ef215..862b2e261cf9 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -492,7 +492,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, pte =3D ptep_get(ptep); pte_unmap(ptep); =20 - if (!is_swap_pte(pte)) + if (pte_none(pte) || pte_present(pte)) goto out; =20 entry =3D pte_to_swp_entry(pte); diff --git a/mm/mprotect.c b/mm/mprotect.c index 0bae241eb7aa..a3e360a8cdec 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -297,7 +297,26 @@ static long change_pte_range(struct mmu_gather *tlb, prot_commit_flush_ptes(vma, addr, pte, oldpte, ptent, nr_ptes, /* idx =3D */ 0, /* set_write =3D */ false, tlb); pages +=3D nr_ptes; - } else if (is_swap_pte(oldpte)) { + } else if (pte_none(oldpte)) { + /* + * Nobody plays with any none ptes besides + * userfaultfd when applying the protections. + */ + if (likely(!uffd_wp)) + continue; + + if (userfaultfd_wp_use_markers(vma)) { + /* + * For file-backed mem, we need to be able to + * wr-protect a none pte, because even if the + * pte is none, the page/swap cache could + * exist. Doing that by install a marker. + */ + set_pte_at(vma->vm_mm, addr, pte, + make_pte_marker(PTE_MARKER_UFFD_WP)); + pages++; + } + } else { swp_entry_t entry =3D pte_to_swp_entry(oldpte); pte_t newpte; =20 @@ -358,28 +377,6 @@ static long change_pte_range(struct mmu_gather *tlb, set_pte_at(vma->vm_mm, addr, pte, newpte); pages++; } - } else { - /* It must be an none page, or what else?.. */ - WARN_ON_ONCE(!pte_none(oldpte)); - - /* - * Nobody plays with any none ptes besides - * userfaultfd when applying the protections. - */ - if (likely(!uffd_wp)) - continue; - - if (userfaultfd_wp_use_markers(vma)) { - /* - * For file-backed mem, we need to be able to - * wr-protect a none pte, because even if the - * pte is none, the page/swap cache could - * exist. Doing that by install a marker. - */ - set_pte_at(vma->vm_mm, addr, pte, - make_pte_marker(PTE_MARKER_UFFD_WP)); - pages++; - } } } while (pte +=3D nr_ptes, addr +=3D nr_ptes * PAGE_SIZE, addr !=3D end); arch_leave_lazy_mmu_mode(); diff --git a/mm/mremap.c b/mm/mremap.c index 7c21b2ad13f6..62b6827abacf 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -158,6 +158,9 @@ static void drop_rmap_locks(struct vm_area_struct *vma) =20 static pte_t move_soft_dirty_pte(pte_t pte) { + if (pte_none(pte)) + return pte; + /* * Set soft dirty bit so we can notice * in userspace the ptes were moved. @@ -165,7 +168,7 @@ static pte_t move_soft_dirty_pte(pte_t pte) #ifdef CONFIG_MEM_SOFT_DIRTY if (pte_present(pte)) pte =3D pte_mksoft_dirty(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_mksoft_dirty(pte); #endif return pte; @@ -294,7 +297,7 @@ static int move_ptes(struct pagetable_move_control *pmc, if (need_clear_uffd_wp) { if (pte_present(pte)) pte =3D pte_clear_uffd_wp(pte); - else if (is_swap_pte(pte)) + else pte =3D pte_swp_clear_uffd_wp(pte); } set_ptes(mm, new_addr, new_ptep, pte, nr_ptes); diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 4eeca782b888..43f75d2f7c36 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -185,12 +185,15 @@ static inline bool swap_cached_writable(swp_entry_t e= ntry) is_writable_migration_entry(entry); } =20 -static inline void page_table_check_pte_flags(pte_t pte) +static void page_table_check_pte_flags(pte_t pte) { - if (pte_present(pte) && pte_uffd_wp(pte)) - WARN_ON_ONCE(pte_write(pte)); - else if (is_swap_pte(pte) && pte_swp_uffd_wp(pte)) - WARN_ON_ONCE(swap_cached_writable(pte_to_swp_entry(pte))); + if (pte_present(pte)) { + WARN_ON_ONCE(pte_uffd_wp(pte) && pte_write(pte)); + } else if (pte_swp_uffd_wp(pte)) { + const swp_entry_t entry =3D pte_to_swp_entry(pte); + + WARN_ON_ONCE(swap_cached_writable(entry)); + } } =20 void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t = pte, diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index be20468fb5a9..a4e23818f37f 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -16,6 +16,7 @@ static inline bool not_found(struct page_vma_mapped_walk = *pvmw) static bool map_pte(struct page_vma_mapped_walk *pvmw, pmd_t *pmdvalp, spinlock_t **ptlp) { + bool is_migration; pte_t ptent; =20 if (pvmw->flags & PVMW_SYNC) { @@ -26,6 +27,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, return !!pvmw->pte; } =20 + is_migration =3D pvmw->flags & PVMW_MIGRATION; again: /* * It is important to return the ptl corresponding to pte, @@ -41,11 +43,14 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, = pmd_t *pmdvalp, =20 ptent =3D ptep_get(pvmw->pte); =20 - if (pvmw->flags & PVMW_MIGRATION) { - if (!is_swap_pte(ptent)) + if (pte_none(ptent)) { + return false; + } else if (pte_present(ptent)) { + if (is_migration) return false; - } else if (is_swap_pte(ptent)) { + } else if (!is_migration) { swp_entry_t entry; + /* * Handle un-addressable ZONE_DEVICE memory. * @@ -66,8 +71,6 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, if (!is_device_private_entry(entry) && !is_device_exclusive_entry(entry)) return false; - } else if (!pte_present(ptent)) { - return false; } spin_lock(*ptlp); if (unlikely(!pmd_same(*pmdvalp, pmdp_get_lockless(pvmw->pmd)))) { @@ -113,21 +116,17 @@ static bool check_pte(struct page_vma_mapped_walk *pv= mw, unsigned long pte_nr) return false; =20 pfn =3D softleaf_to_pfn(entry); - } else if (is_swap_pte(ptent)) { - swp_entry_t entry; + } else if (pte_present(ptent)) { + pfn =3D pte_pfn(ptent); + } else { + const softleaf_t entry =3D softleaf_from_pte(ptent); =20 /* Handle un-addressable ZONE_DEVICE memory */ - entry =3D pte_to_swp_entry(ptent); - if (!is_device_private_entry(entry) && - !is_device_exclusive_entry(entry)) - return false; - - pfn =3D swp_offset_pfn(entry); - } else { - if (!pte_present(ptent)) + if (!softleaf_is_device_private(entry) && + !softleaf_is_device_exclusive(entry)) return false; =20 - pfn =3D pte_pfn(ptent); + pfn =3D softleaf_to_pfn(entry); } =20 if ((pfn + pte_nr - 1) < pvmw->pfn) --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8CDC2F549C; Mon, 10 Nov 2025 22:23:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813430; cv=fail; b=Xn6GGN7bOW77EvufXZu+EmwoqytlU2BcmARgT0KWHGJ//B8aItk8una5zXfSs28DhU84/OT0e8ZvK0Ya4OBxBAOt17lW026UpS7SyTyXUwKNfO9Oizah0c9fix4al1KcOdgW4B5938jIk9oeOjdvCfAfBEAXYizU4/b+iCpn3NA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813430; c=relaxed/simple; bh=7XmOOpaf5TmZLupavCzYibr5KX6ZsU6IglQlUo1EsoE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=o7jyMa4AbMNwn+/i4NfqPetnMZ+zNmLlkQeKVrfDzgTADFPYRp8/7g+LgHXCUUwc8xZKfUXchDiu/FnJN3wTkx9t51A1B9n4wicQT1ZJJgcaRQshxnsTDKA+iW8TYTx6TvvZqAOp3+vhiVcwiHkUsaZJavUsimkLQIGCmeatjMM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=aYTMltRg; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=HQUsxgcC; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="aYTMltRg"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="HQUsxgcC" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAMIKUT025943; Mon, 10 Nov 2025 22:21:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=VW1rVEDry31QDVbfRseXtsv01MYxQ7aH996ZkSCtb9U=; b= aYTMltRgp9YiSPPyaqMTEQM/Ky4RuxrMO7JRoeLoTQPvxNP7c2LaIb3xdgNkWaer XrczuDiUc7akV0qF6cqdgXxO7A+7B6DrJJAqabNeTwktrdpJbh8DiwVAoObva43Q Uer0c6X93MZ/xeuCZrTT2TRppd0cwG9mEU7OVIxlLtpfJKgf5PqkBFeFAOB5sL5a u4nZkPf695KjLUcWtONu51bn4dI3YzFuH+sVuDpUz0ia6L/NJKe1vqBypk7NgOJO 7BpSTKcHwKRJcM7AyaQDLXVyISghN7LQVJIuTsBP/R0MyzZhWu+ZzSHBv3X+n4lY 8BgBxMKj1jw51f2hcV9rCQ== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abr5vr2ap-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:52 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AALo2I4040096; Mon, 10 Nov 2025 22:21:51 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010061.outbound.protection.outlook.com [52.101.201.61]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8rypg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=OtOj2C6mvDuqLVGdAMue/IFR8VS/CGw2pwyIGWMm41KwTWCGaWAgo10Vb+6R4fa3HRsyCjLr/c2Mm/VNhFjA88pMfqgPLkQ5va382WOODkPdDAiVYsVEUFan9ilNw3HrUb3pamTl75Zkp5/E9F64ppaAXV0I8qAz58yQjpfnAy7NYI17CzApUYZ6Gp3HjUR130yBT8PQoeKtPKB77rERn3XO/lcEKwFazQlmMnMJv6tAans/CMUpqfSg/x12kHGidCGMl0DTVHFGO0Yt3oEfzKQOI3Bx9vU4g72IQSSe4kn35JiC/PCmWpHKJyd5lQ0tikEUxQj0zdCNZwaF96K4zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VW1rVEDry31QDVbfRseXtsv01MYxQ7aH996ZkSCtb9U=; b=T48HDB3ho49W6PqbQf4D2rgveWw/WRMIdqiBwXdJwxcS2VXTR6f3BoLCTG54mYVJEezcqQVtH+j5Mp2P+WzulHrFGEGn4jrtS1D1BbsDKkRH0Hy1pGOAEGdwrlEa4qtKno6KkSDkSZPsA1YijoN1Qfji8eOGDTMKnkSHgn+u5pRg+W41iLOBGc7N95KSfVEEdUNFFNamTeaq7jr6VXYcFPm5A0hIjI1CEy2Yy2IrF305n3CWigPVhackwF/6KWvXN3qqWoxSQPVR6KctBEaNkxBR1lShm9+95y2Z5jGXjISByrudN0CacfA0OBkiPqNUMF7AfdA58xifXEoMLQ7NVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VW1rVEDry31QDVbfRseXtsv01MYxQ7aH996ZkSCtb9U=; b=HQUsxgcCxq9WkLSgOuY/F9vJHoMhaQYpgenb/b1IFsZ1p47jAhVQG7K11oI00rCGkGytrYD+Yj0LPSqQJ7SDyu+4Zo7Xlc8sBgS2H7JFB2Ba6zOPw+nCtOo801WqI7JA4U53qHsAMYDheg5ayrJVdHfPyEQMdCTVWxa6d3vo6S4= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:46 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:46 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 04/16] mm: eliminate is_swap_pte() when softleaf_from_pte() suffices Date: Mon, 10 Nov 2025 22:21:22 +0000 Message-ID: <92ebab9567978155116804c67babc3c64636c403.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LNXP265CA0032.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:5c::20) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 68971286-5a6b-4a2b-76d7-08de20a788ec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?6vwbNIFUjPKEana/ji5krkpAvAEnR77MNInpoPvsS3dyvkW7V6n/WlLG5RfM?= =?us-ascii?Q?4vYN3IVabWPRa480RohncOhot19NK0g21CCQAtaAmMcaPc2dh2DjGWynlkbH?= =?us-ascii?Q?C4cDx+Gc/CsckluY+n8+gvGXCZsD2fGFaBO9EJK+zKgzaiSXp/YX/Mf4Dtjs?= =?us-ascii?Q?wRG2147YO3JoDNnO4lbjZz5g8KRV76lc3HKUUHj4S+Dup4CZ5uwAoyWd+nQs?= =?us-ascii?Q?ctxVpTKEnB6gZ2m63eEs65NqyxF+4VGSX1wrKMM9GOtoqzgK7LLN/WkKUFGn?= =?us-ascii?Q?3IqmIXhNSf04NEGwPaA9P4MM9g3vRKMyioodAvFbmJ3BsodpU3xSnCJWDgrd?= =?us-ascii?Q?iP7FJbyn/sqBliEwb+nhmK1GiBd+If1WI0lqGgBhDpW4Ly1QVh2k+G/dIXK4?= =?us-ascii?Q?934Yrkb+PrDUxL+SQj9lIuWrbyvSg1com8+saADsI2sd0zyo+pZU+/yhyDaS?= =?us-ascii?Q?4DDUniKsjBn3dJLsoxl1zhoMBT9hOMDhXFwqvWqMHrrf/7mOqCMygqcTRvxA?= =?us-ascii?Q?1+undE/nsdfb6on4PGoH4Xx1EKYP9AlTRwuvxHOpLxyZzLeq/0rttPrFLAME?= =?us-ascii?Q?d3uULipmCOdOhlB2pNYRneFbOC97cLMFTdR048m9lLvEpnt4yC0023yNi86A?= =?us-ascii?Q?meCO4uXtX8WlHrLt312dp8EyEZxoe77cIMv+7ELSgl297O3Tt4YY1kq4jmDw?= =?us-ascii?Q?Yjxi7NNarXhANxUG9eN5sW9jKD5kEEV0ysZ3FmxJTKq2wYrHv/xfMVbahcpa?= =?us-ascii?Q?g7vU6e7m+PKjrRk4WJCc10ON6HAz5oVG4AU07E69I2eFmlAmSNmbdt+4l5Yx?= =?us-ascii?Q?z4stW/H7BeJbVbraQnt15nXWEaGqZ7f2lkRVyYhiW+yGYdRRDoWGheXct6jx?= =?us-ascii?Q?fEdBQbf2BeTpIFnujoxQXnZigFw4/QxLaAh6Q0V5b4fq2yDT0vzPHSUrEP5o?= =?us-ascii?Q?AZw7EpvG7JtCiCQVYEYTXYFEqSpA+qITQ9COXQd0ys72t5fmEuohYBm/HlBo?= =?us-ascii?Q?whOa3Puw2XSzzxHWeGaYh6Y9F0IiO7xXbg1xAMN8FFUblUhLW6NuaHzkINak?= =?us-ascii?Q?SDwFgkw2CyJlN9nOC6/zrzZra0KZlt015XMfCfUACsfceDuSB/m6i1hdTkQ/?= =?us-ascii?Q?Q6tlEU0gt0mdcaGljwc4vC4YL89yWOIbf3A4xwFaqQ1EU968eLa2j3OzmpVJ?= =?us-ascii?Q?jV37L8KES1F4JPEaerCRLUsVA3Y1aRVQQBgpMUoGus63cR48xnshBLserWad?= =?us-ascii?Q?O78FmS0gbBTjKcGQrISXsp7lOV4IVqSY36xrCBzCnYODUMyq+c8RxB54kMAK?= =?us-ascii?Q?KNv7Ft7a/uY6LkOglQxxtN5p2Dlj4iEGIrJ2MlEqumMqlAqGSJPXMgiln28O?= =?us-ascii?Q?cDA2Br1ciTHZ5YoukJCyT4oL3G9oCDJc3zg/vFyMM0XOIa1JOUySuCWBg/XR?= =?us-ascii?Q?LWs2CGjLzBk9tkRh2GiRtS5UZ/MJIZVI?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kUxkDVzqZ7dCROhieK8dmfw5/fO9vo2DntD9n6BBU+vcMGjXhsF8/VyjC8vO?= =?us-ascii?Q?+La8p0OYDcvq5lHOE6QwXZYAYNUDV0BlAsMFNI6lSAb3LRp3zZt1u/xaNr1Q?= =?us-ascii?Q?+421gfGSPSgCbxAKTqCoKuJdTrmzwaypks5evOmyOgjx52/W9Y6nZgZiXB+5?= =?us-ascii?Q?Gji3BY6vhZdecV6x874zXk2kod+62SAS8Bk8ZXrzOlTAAdzN4LLpZIMLuF2w?= =?us-ascii?Q?vNin7h4+FZqt5vsnCaf/s81hZ+MYbzQiW9C3CNFrThMu1sAyI/RXZ4gHCLrM?= =?us-ascii?Q?7dCyI7c+14IgD9fxSmJtbQCuWN2KVK5Pylu5QhlGiQQGfWdGsht0QZbWoAtu?= =?us-ascii?Q?FwYEhL67Egwzlfyn7Stz0fePvLLbAgF9dtSQ+t6bDaWg+zEtv9wJ8lNHiFoD?= =?us-ascii?Q?X2Caca+ZK2rlixV1Nk5D9TFhy/lVd6VvYUMh5nzKbubQJrs9ENBud93RPgH0?= =?us-ascii?Q?9QIz7uJ1/fQcdMl1NgQ8wzEx7xSDrdMbH8aCoKPfUjXOoPvh6XRsiS+3FXF0?= =?us-ascii?Q?TA5KKmO6ye+Rft8juGU63KgO2+pU/bYFDk+6ZvoA3CAMoqPYytvxq7niYEbl?= =?us-ascii?Q?vl7wPFMd9ShSBlDhTtD+6BikjQgoZ5cFsL4PAml0VmepkhU6WHPM3gHLd5EN?= =?us-ascii?Q?vSMqNE0QUDXT9IU3HdmOS61pj95vovvV/mvkbmZHjawcMD7jIzrdMpGm7hnd?= =?us-ascii?Q?oqN56f00Zjg5uD/HAxiUXbdhykF4c+9VdDC+dg9PvZAnxUUh3dd4QiCLPigq?= =?us-ascii?Q?ZZCl3T9XbdGDJ6cR7jFdXh4KP68HsthwCsM1WIcWZXuPvXSU46xozL4Tqv7x?= =?us-ascii?Q?/P7e04sMd28S6r5LMHydVJzdYSy69sI7zw40brSnyGGK41/4TObLQBlz3CWL?= =?us-ascii?Q?XLJ8WIxN5KCRLFGCRD50UnAYyEWEr1cWETLUmEoIfXOd1arKz1d9eq1ha8kq?= =?us-ascii?Q?ZqK4PzAYEwcEdncXWZQwkZklStgmR0i7doW8L77SFfM/8J/NNQox6V4aLKX7?= =?us-ascii?Q?tG5Zv2AYO0X3XEMBeKemNCCtiLplVDsNyDrVesYdfUNf0uGbaqnrGeRLDgl1?= =?us-ascii?Q?i7FBPDxouXxcaoZaH0IE8owxfC680HF74egcWKZba2jIVvNNjh9ftumra00m?= =?us-ascii?Q?BsIzMNr6onSNxumuqzl28nBQyFp44jIz23UFwu5ZCSJstDlfAbJc6h76Su1V?= =?us-ascii?Q?oupbePaxBVzoJ7MQWUnbHNRQmojwMqk74YNY7q/hFzbOxHyeqJ5Dim+XZJf/?= =?us-ascii?Q?NANOOnK00yipYhyFvbuBtk0X9PnCOL4OosS8/2AajTRbKWGpblt9KAPadlTv?= =?us-ascii?Q?RvSnXLeYpKs0Phg7v9Gsp7v8WxLKqcBmX+340dBcL0GRL872bZjZainaWJsJ?= =?us-ascii?Q?jD1SfuFunvlAesyWaRl2ZSEHeBJFEomea0sXBjmQVnChTqsJzvqepRoe67So?= =?us-ascii?Q?W5gRWrcdpqOzaFYvnW08s05peYqQmSFOux9b5LFWCo5n/179A0leGfBF0mJ1?= =?us-ascii?Q?FYDgILfnvNWU7v4pixATm2Wanqd0mBfr0km4giH0dJat759c9WEZ3yjwbLrG?= =?us-ascii?Q?t0tJkeX+2T9fvb9MsSwxDJCtNsuXvJrMeCSnIRpB11VTRrTAYZXAwXB+2QQ/?= =?us-ascii?Q?2A=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: nZQt3VfhrTKWOoDiFM0XJqOWCOx/NP8F72kZqu8j3AFlUeT+d6IlmgmytyjzNL+Es+3fe2tcJOqaqFwEjKi8sbAZAKWopB7rU97qUKrrVc7nQ9+KgrhtH79wcVLzhXbA8cdJLbDofiV6fHd8FOAklK+nNeJ5NTGMz/B0xy5gvs5T4O72s6h0cxHiUfxGggZUMiBCMII2l8RaIImX2z1e/9bJbad2njsUzNnv3bU4GagZfwz11zVBqYxVc+r3uLYjcACFXx1g4vrSl8JxheabHzG5Jx3mOqLDN9HtQMYafyHsZUKVBzkQhXhpItiYwyGj/wljnSowi/EvmCLhduI1f1dt50QLz1zL0b0CYeomS5Th/rgcyZIerdOBvpR7E7VP/jKjUg55ccW84NZAperSqI1to/NaVniYtEwrXzpwbsjlvQMyTxnCsRLjQ4SH6vLWMU7kUjbqCn093+ybTwmjeG+OyUQvMiiZz3jDJ1WnH1plz3uGptLMYLYDk5ZijGMXMf0ZJ5epSfy01yBuaTGpYcZxLcooGg867k+9O2XSg+Wu8IvF+kGvBpDX3EobpYaclin/pHk5AMxfQ8EWLhuPbBwQEJywBghWS/tDPu2bxxo= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 68971286-5a6b-4a2b-76d7-08de20a788ec X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:46.2600 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /vfYouWY6f1CIC6VUBUr6kKvD4AdFvvt2GNGwXF9eXj1Gr++pL4qj/VLWWaLLe1ucN89/k9iRb5JXUecMlrVvwAee/5J+O0NKMiCz74T65g= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: J6K0JQeCzL56HZuvwCHTmCCcY15dATRM X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4NCBTYWx0ZWRfX4F+W3hWa6N2m Fe6VV2zNOWuTmjo6eoRBxMX8JIM8jdXtE47onYwpj3nTRumWvFT29NaMyQwmn1k7VQbFhrbV2MK T/Tb3kwYe287RpfeWDmSSNkf6oJMlNUcKs/QKSn3ICy3dxdw0dFvNGJMlCedG93KF8x0xUMvQ8y NbnuV3uL/EanplSr5zr8MI9ey5JU8JF/s+BrqVw5SC9UjzepEIvnBkWB1cb19aLOkY6/XvjbpF9 +oIY50fXdg8t4pSmq7NhC2nJAqy9uEvX+w0PaXVgPXLK+6iiow4F6WNyDW9yoAwMnprQQmOT0RJ SknmTlAQprRgk1LYpRd0Fb4yL46DVcgWj+ygjbQsKqrZF9bcwZv0RiI2qgsK09RaW+t6opoZngO LZFcv7x0paAoVDW9NvUH5bUK8rCmvw== X-Proofpoint-ORIG-GUID: J6K0JQeCzL56HZuvwCHTmCCcY15dATRM X-Authority-Analysis: v=2.4 cv=YN+SCBGx c=1 sm=1 tr=0 ts=69126580 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=x3qbZFVQRPWiRpXx7eIA:9 Content-Type: text/plain; charset="utf-8" In cases where we can simply utilise the fact that softleaf_from_pte() treats present entries as if they were none entries and thus eliminate spurious uses of is_swap_pte(), do so. No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/internal.h | 7 +++---- mm/madvise.c | 8 +++----- mm/swap_state.c | 12 ++++++------ mm/swapfile.c | 9 ++++----- 4 files changed, 16 insertions(+), 20 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 9465129367a4..f0c7461bb02c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -15,7 +15,7 @@ #include #include #include -#include +#include #include #include =20 @@ -380,13 +380,12 @@ static inline int swap_pte_batch(pte_t *start_ptep, i= nt max_nr, pte_t pte) { pte_t expected_pte =3D pte_next_swp_offset(pte); const pte_t *end_ptep =3D start_ptep + max_nr; - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); pte_t *ptep =3D start_ptep + 1; unsigned short cgroup_id; =20 VM_WARN_ON(max_nr < 1); - VM_WARN_ON(!is_swap_pte(pte)); - VM_WARN_ON(non_swap_entry(entry)); + VM_WARN_ON(!softleaf_is_swap(entry)); =20 cgroup_id =3D lookup_swap_cgroup_id(entry); while (ptep < end_ptep) { diff --git a/mm/madvise.c b/mm/madvise.c index 2d5ad3cb37bb..58d82495b6c6 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -195,7 +195,7 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned l= ong start, =20 for (addr =3D start; addr < end; addr +=3D PAGE_SIZE) { pte_t pte; - swp_entry_t entry; + softleaf_t entry; struct folio *folio; =20 if (!ptep++) { @@ -205,10 +205,8 @@ static int swapin_walk_pmd_entry(pmd_t *pmd, unsigned = long start, } =20 pte =3D ptep_get(ptep); - if (!is_swap_pte(pte)) - continue; - entry =3D pte_to_swp_entry(pte); - if (unlikely(non_swap_entry(entry))) + entry =3D softleaf_from_pte(pte); + if (unlikely(!softleaf_is_swap(entry))) continue; =20 pte_unmap_unlock(ptep, ptl); diff --git a/mm/swap_state.c b/mm/swap_state.c index d20d238109f9..8881a79f200c 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -12,7 +12,7 @@ #include #include #include -#include +#include #include #include #include @@ -732,7 +732,6 @@ static struct folio *swap_vma_readahead(swp_entry_t tar= g_entry, gfp_t gfp_mask, pte_t *pte =3D NULL, pentry; int win; unsigned long start, end, addr; - swp_entry_t entry; pgoff_t ilx; bool page_allocated; =20 @@ -744,16 +743,17 @@ static struct folio *swap_vma_readahead(swp_entry_t t= arg_entry, gfp_t gfp_mask, =20 blk_start_plug(&plug); for (addr =3D start; addr < end; ilx++, addr +=3D PAGE_SIZE) { + softleaf_t entry; + if (!pte++) { pte =3D pte_offset_map(vmf->pmd, addr); if (!pte) break; } pentry =3D ptep_get_lockless(pte); - if (!is_swap_pte(pentry)) - continue; - entry =3D pte_to_swp_entry(pentry); - if (unlikely(non_swap_entry(entry))) + entry =3D softleaf_from_pte(pentry); + + if (!softleaf_is_swap(entry)) continue; pte_unmap(pte); pte =3D NULL; diff --git a/mm/swapfile.c b/mm/swapfile.c index 543f303f101d..684f78cd7dd1 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -44,7 +44,7 @@ #include =20 #include -#include +#include #include #include "swap_table.h" #include "internal.h" @@ -2256,7 +2256,7 @@ static int unuse_pte_range(struct vm_area_struct *vma= , pmd_t *pmd, struct folio *folio; unsigned long offset; unsigned char swp_count; - swp_entry_t entry; + softleaf_t entry; int ret; pte_t ptent; =20 @@ -2267,11 +2267,10 @@ static int unuse_pte_range(struct vm_area_struct *v= ma, pmd_t *pmd, } =20 ptent =3D ptep_get_lockless(pte); + entry =3D softleaf_from_pte(ptent); =20 - if (!is_swap_pte(ptent)) + if (!softleaf_is_swap(entry)) continue; - - entry =3D pte_to_swp_entry(ptent); if (swp_type(entry) !=3D type) continue; =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9E272ED16D; Mon, 10 Nov 2025 22:23:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813400; cv=fail; b=Y0epPGOHd7aH7c3K8TC2m4ZrY6RrdFIuXq5AM8wu+3riftXgp51i+HYQiFcY/qoSsBd5ttFktt9V1rVJOEu9LX/g/Jb/+1TN5j9BwW/Nukhmm2WMJUQergtmmzuuI4NWG0JU0gn1WxD3yoGW6SBK8wyGeda0b/OQu2Dh76p/qs0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813400; c=relaxed/simple; bh=Jc7k15MXVqD+LodAv3sGe3ypLfG9f5hKPjGmdnE6+KA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=TDNT/Yt0hB3X/yVMuDzPIL4IJDfTDJ61xsrinw0RGpKH0RJaXvktF4EIppQTThI7ChFqBTwVfExRmcGeObQ8l2xEvcfyq44zBu7inMF7XJWolKbBzTRKMtnLHyUkiMm2C2Ag6btgnRpS+hf1iF2KEMUpAy0ur8WXnowRACeztVQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=OFPbnqOZ; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=yZd9Ozkr; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="OFPbnqOZ"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="yZd9Ozkr" Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAM0dN4019647; Mon, 10 Nov 2025 22:21:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=8WOrh/xELmUWEKs0VQab4I3hnqf3OQgLiTzmuIkkR3k=; b= OFPbnqOZyE2j3zc4eG5Cvub3C1KpAD7c2ow6ojvnYpaWpkG9qQz8oLi+hb8Nn3OO KYu69x4TJyNTIVJZcEwGWc9DCMXmx+OKautBSIDy38no93QfnUB20bVOBtwxabSM 0X96UiZW20rDQN4GI0hk2cihfM06zomSEFMR5J8ObWgAs5nOcLwhgh2e6D41hzsT SBDqBuN6OV/75D0hzm4gHTIKUKKu0nrIRnaHm24ItiQo2tSW2pH0ARowOStEx89I E3oBk629eyQN5r5ytKcQ514I6zI+ep1olky9ET/vRH80MXsb5gwHFFpFNdeBTxyM DHVOhxcXrhmuSY2jX9lYQw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abqvfg4ac-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:54 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AALo2I6040096; Mon, 10 Nov 2025 22:21:53 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010061.outbound.protection.outlook.com [52.101.201.61]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8rypg-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:53 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IiwFLZePMaogNavHlN5GZDGAeOJALIzNTxI+X73+CpmsGya4rHtBr4b4IcwC1u7e6ayDTl/OaBRHvIBBvgAht/3mRaOaBYBAJqU36DzyxB+ns09Bf8EaR0pNZBjsJS7ImAG8UFDZjiqKvyWfWCGe7Uz2ffcCfL/De0b1wjN+XT0+eFgU7BTtfXXqom4yWmfey4AbOE2ro+4kKSMLgDFYGFxtiBVoYkcP9ufu+Ujgz18KPQcFSTb+k+5qK1/fnFvglLjh8fGqwqQfa6h5x+aTkLYtk2iCMzWH+XNZF7iDsY09LG1nNUXF6emy4C8HYWdAwvf204jHOTU+nk6+2wMADA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8WOrh/xELmUWEKs0VQab4I3hnqf3OQgLiTzmuIkkR3k=; b=DfmuLtim4P6jZSM88S8MocJkMcueduQ1coKN1YS81ZVCHxlmb1zSfHK7cAWQInja/M39nmQRdnkko0vES57tBsynvRVbGBK3krP/Pyr8qZ7emFdVdjbGY1NlKMqikFbnKCQlBrt60N+G6EcQQA8vTz4WjVVe1jZaSuLNAuVUBkOhVeq0yqptut8O9VQWqmRP40LQElQSNF8YSNvE/M4KCdUqqqtByq3EvItwHMNZNfI60W+hZ2PAbxRVciBnhru+mA9aEvKH16m+Smx7B1Gfe+lTr2VKYB/vOT9VkLoQnIEWBQRxY6DFxm3DWv5HkjhimYO+p61k+8adwZpxYivBDQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8WOrh/xELmUWEKs0VQab4I3hnqf3OQgLiTzmuIkkR3k=; b=yZd9Ozkr22tb3gd8HpFpvml9tIzkWF2LAiGOaP2uWdg8Md0ky9+sAUIXOQXcuF+Gf+mirXdXHr4OUuwzpbQgPTBiBnrbF+Lm6fv84pQoBBZ2ZQzOOz11iHdu4pKpCX08STLrL2ZGhwbW0yq4rRRbqK50IRTHSAE+RabuZRi1haI= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:48 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:48 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 05/16] mm: use leaf entries in debug pgtable + remove is_swap_pte() Date: Mon, 10 Nov 2025 22:21:23 +0000 Message-ID: <222f352e7a99191b4bdfa77e835f2fc0dd83fa72.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0009.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:62::21) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: a5cda8e4-a770-47c6-7492-08de20a78a24 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?bryGD8yEFNwvoM3Z+8m0FTs+qB1OEVSsUz0X6yMktJI9BKV/V923G6n/1hD8?= =?us-ascii?Q?ianZKd/5Mfi9T8oGwCL976/IdrTFmq7CY+hIbdOsF1fck0hPWTcLHrD2ZDPF?= =?us-ascii?Q?mDvGFa0qc6PWDpskH1//VTjQT1a85mhTV3qyuGXBN35DdBMsFycDAkUB1jXb?= =?us-ascii?Q?mT0QpfjO4uwxuW+hAyfRf+coEi3UdkehusFxGUJNIXaxLl3CenyncJW0tjOz?= =?us-ascii?Q?ZSg0PpfAuuHiZSYhx39NUXuCg5Sqha/7EUxRpmPQG9F8kdM0iebtZBJMrmOh?= =?us-ascii?Q?qOKolo12pzBZL1QsiWXKcFeDdFcBjo3f+mcmbH9kkvktGdiyI+N7NxsEKwjm?= =?us-ascii?Q?7d8wmrno2ab4X4pgK0GRiwpoq5HfOV4Bw7gUlYZBhik/hXhd26ckW34gZ3Un?= =?us-ascii?Q?9GmFzHwMX/SgJ//ADQy/+u6Bi9aSD+TDml/8vh1C+1fc+5Pm+oR5ybM29HlB?= =?us-ascii?Q?fBSHNEjr5dytPdENGrwYe8CX/48yGRjtcMdIGq+NUA9WvGVLh2tOmXEetj/l?= =?us-ascii?Q?zaqZR1veraM8CtNkjPg7oYo0MYlybR61KpfuJLi0rL3v37dp2g8DNcPvgnrj?= =?us-ascii?Q?F6XIHLIyG6wIkVRCINz0Q3Cmd3yWebFHkB8Lk/Lyv8+/TdMVNF8JbbtYbXMl?= =?us-ascii?Q?fG9fulzFLYD74kfDPnFn1HiwEe8brXhG1eR+bzgQG1MT7xPXPVudBxdIuNXP?= =?us-ascii?Q?HCVQram08KkAfb2Ks9r7dsLC7ooxKto5Z/Na5nDh9ryqc9CC7qkEiew8jE/n?= =?us-ascii?Q?j/fB9zmiqx1fxFooGfouL1Ys0S2iKxFnhRYzlhoe/JuyVgeGFmQ8M2Qavpj9?= =?us-ascii?Q?g+mhBu/rF+POSMqp0NXtaf1pZuQfbaqYbaqgjZhFhRltex4l8HVSwSWxwK9E?= =?us-ascii?Q?6O9YzyrQHzXoa5+GIi+bcKKJURYDo8rQFwr0YuC1BmmdUqMisePK5Hf98/nB?= =?us-ascii?Q?K8UikFMvdnL8/oRA0S57ONk/NiXlmlJNF6FiL7emeATmkD6ozmjcndRuK4fk?= =?us-ascii?Q?14eP+ZVOU7b1mCbBC9KhakJrwBGdfHu1ew3Zdyh/d5WwptXHehujUhnPK5ak?= =?us-ascii?Q?aN3QmSiLD+o/pc2r11+KZBn8D+ixDa6CwyLVeodenJNDbi8lBmuNwrvLlu7v?= =?us-ascii?Q?uktA2nuAAABUqpE7gbfD1bsiYLIexRp5ESX+R/LqQI+dCOfk8jVkQ4RuZ10R?= =?us-ascii?Q?fysylVeG+gtb03rxz/AKe5K1ePKnWxkWXq9LrDmgzKxcDn2sgO/KxurH1K5W?= =?us-ascii?Q?XpaUDwEBuQc99kxN4JPScpkOGTkazUvyF7rsC8kY4ixXAYJ3t43UC7TABdKF?= =?us-ascii?Q?1/qq1HsWeKH02ph+DgCfCVo2OJEeo1IEOvMpW2Z/6SBoCsmShby1Oa3AprhD?= =?us-ascii?Q?BDw0Qs4MP1IdKxh8AzxScTEhmyQyqCSp7PNPTxJU734uOe5HIIlp65XLGg4m?= =?us-ascii?Q?1Zf6XhAd/0IEDW+kKRhLVbrQdiC+zRn9?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?RZtgJLd3yXAgfwltltU3wbe3CnP6cgBU0eM/qE1mSsrALTy04YDSrDC/U/vb?= =?us-ascii?Q?Aj/PpVy+WnROgq2mcvBzoWtpoYyKeiUvAJvPxYspsQzwYr5iICM1tO/Q5Bqv?= =?us-ascii?Q?JI0yxXM47Ucz+0qfMiXjoBfQ+KGwMVxI9+sFahfsGCe97DPAjk4IFHBnUpiO?= =?us-ascii?Q?23MssIz04uYxO2q5CxN1JxX/ZZhIyK7mkqNLp73n1cuZRNH1JrbZu0KFtLbH?= =?us-ascii?Q?U/VH8x6uPIspIelvfio+WZSPedYWen8HKU0dxGMO0e30A6W0W9QkLZu1aUF2?= =?us-ascii?Q?5i8b3ApoPwvkh2h/49iUc207Hy6/ior+geconcZfrOkZ4l5w1DpyjjFyakOo?= =?us-ascii?Q?xlJzBlbBuz0b/zQ2KcuGy6frImgI2i6B0VObb1iAGEKFJzggde8NILUyHFKZ?= =?us-ascii?Q?a3eETrHXNREw6ANtg75lcY3rGwsPWgsKrpXKMfVbylZJ4ydfPjGfHhzryspR?= =?us-ascii?Q?3ikeXaSLVdFbMeZi2KcLX2K8oIGoUYy0/mbO2jbUjMMlgoCrhb34DHRHoK7M?= =?us-ascii?Q?p96ao1IQEw7yTINnwJMk80BAu3V3mBktc+lsTxVJVQFoXzpFg+Q6Hcf5PLpJ?= =?us-ascii?Q?SrOv0NB1o0q90hGUtNnSXG852Hydo/TmJfX6AOjPdm719FZOWzQTvdB+LmuX?= =?us-ascii?Q?9Y0OJSwTHlICybzUKxo/g8dEnvKuk0l54bFzK+G4la59zJ1Beo8ahuNWWs3t?= =?us-ascii?Q?nSXPIXTU9R6Mj9yHOoQhv/5EU/Xy4p2cp/6uf4aafXOwPHtVgvux7Tm9r/fC?= =?us-ascii?Q?QwEqXUAdcNRB8HTBpDiOjhvAIg4VuyYxDzcoexstkiI6+3waCU2RqnkclZR0?= =?us-ascii?Q?Ec7YQYx2KIaIyArrQoFp92VklSXORG64BxQw+EECNdv2xuqUWiRpSaos4O6+?= =?us-ascii?Q?dlUkFXdMUTv3boD+QmgI1lecfWoNb/UfVTph/3l816waAR6MyDFiA4f0C2wB?= =?us-ascii?Q?NRZ+xO6hZwcYqimnUCX8l8m4kyenZRNmS5i7opEi5jqIE89Tb2kQXWf/Jddm?= =?us-ascii?Q?tMWvKvWJXw2gal9ZygAgMDUTjeNMRLLe7dGnjc9k/FpLLL+5gDv7tnTmXJHv?= =?us-ascii?Q?lVSoXu9qD0Y/S8mO6ISqDXVcVIwJLlYp2SOnCaz8lxOqy3etAp59hX0saCO9?= =?us-ascii?Q?pWGlSXVsUIRmuU2SAyKdHjAb5kOLM4NJQAyoI2PHkVDDa9i/bEHr7RMn98Q/?= =?us-ascii?Q?Pk3SzYdPcfFN+EF/6U2nT7uH9gTWh/tcftjDKuK3uS+gBytDRTwBbvOjFGof?= =?us-ascii?Q?8wbdLJO6I1sHrm6TuakL9dT4Pr+BQrNGAXOUF7kpgxCQAvNWfjpykUC0pejS?= =?us-ascii?Q?SQEgeQAPN1wMC3XBKUa/ufouVAAJxg1Qk9QuLiNZe/EdwC/HlBsrZ3yNrgqd?= =?us-ascii?Q?C19FYTGf7XNsNqSA/35w2YRBeh6xb3y4Chb+uWeyNIRo+zEBbk8PK66xvTQM?= =?us-ascii?Q?Z5lklImKlcT2LyyJ4mB0D6OJjam0Xz7kqt0gB5ud9qRIFBR/Gpbjf3TTJJPd?= =?us-ascii?Q?6LCVz/HpretuphnzmdR5ui5bDeFZqsf3rwzcD9JmfOzj9+K5Eh13sc8EEuiJ?= =?us-ascii?Q?ot7+DKcvRH8AGi/TSmy1nuoysv85jFVowzOU+tbyxha7DC3GEF8njECTy8NL?= =?us-ascii?Q?5w=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: wKeta1OM0CZ3mLA8RigGHHwrZexLLFuMMeVowF63E3Q/VEYZgx1h5pyMuC1/GHIbUIad0EDUt9eQt9v2LZjrkEZ71QBGFvkZddQp9vQjrJMjhExFFEfW7j/ggT8p4/18263N62TMBuJgbeMNSu3/STfzXAUufZmhAojApG8uUR1SxhjpJXjtCXp+3TYHeVFkUxwBqe4suPcApaBefAyQXYvqNy8c0+rlTuuurrOwZguJrSbuBmwXwrlNQ4gjadjxvg6wTmjIs2fqKF0IfmtRoKnChE0aARoijtbxyFgmlDXuxKpLLtMEhGHxEMwUbYdF5RBc+0k6tTODqQyM8kLgozZE04xLVNjmjdZYzGe8rZOAKOEzwM6oJHThCmaPtMP6DX5r14gGbiKP5Ehh0ruAZzS/k4uMyaC7ARailAisb3PMiJIHIJfycvv9ypPDof0xmS7PghjVyCoIZNy6srgtEHfZrBe1CAvshcAhu766LCTw7SyruYkUT+DFaOrb1A4YtzEGNblV4mCl8DasPHddVtVhMGN82eWBEKMo8kcvWRhS5K0kEoK+Nf7bU+NXm+E4ugAtYTkOtbg4Ov9dpk7SZXZ2jog8QwIUOMF8jyLo8gM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: a5cda8e4-a770-47c6-7492-08de20a78a24 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:48.4038 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IC6/3MiK+ygU5KpHP/7WlkS+aQ4jK5bQsqEEGl7xjoidQBLJm6yzjCd9MKSV24ShmggfzYyHMF9rRBu8S0l3zwz0WVPWGtJbwlrI/lFQpos= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: V0ENqnUuisHv8T6Mx46qZqiSHkNZf-TI X-Authority-Analysis: v=2.4 cv=FaY6BZ+6 c=1 sm=1 tr=0 ts=69126582 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=JfhMdzgkenBfAv4L-EYA:9 X-Proofpoint-ORIG-GUID: V0ENqnUuisHv8T6Mx46qZqiSHkNZf-TI X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4MiBTYWx0ZWRfX2RxC8/wkmbD3 5zRrvGYTsaK1CaWHOVcrmTDdM3X8nNlffmBdKaDZzIUDYHfZdlqIiHxK77E6DwKC7hvj+lrQLFe heXoK/FaKbFDaxdWq2yZphfa2cQlEdMcjxsa+P0LpprF6AUIxUZ4Pss4u/XtpT9BZ55kN+6VO8P IKTf1kplZG93G+780nbro2AGAqlhp328xmYvaPGL/y8zcipAZrzuyEu34u79NoU5XnH9aV2qUWC GiwevrGcU9lQHtl6rockmt8gHpDyI/U9jzMutULD8DyPSF8SyHNnLwcPssOQ6RZkFZevmkXSznF sYaUOqfZ4NczYK+QOqMg+j94LUqbGO0SdwBP+GvhOORFgioxu6cB1P+cKQaqhN/rcwbD4xJl6ym AhCqvePMSvYusA7UnN7o2q/PLkI78A== Content-Type: text/plain; charset="utf-8" Remove invocations of is_swap_pte() in mm/debug_vm_pgtable.c and use softleaf_from_pte() and softleaf_is_swap() as necessary to replace this usage. We update the test code to use a 'true' swap entry throughout so we are guaranteed this is not a non-swap entry, so all asserts continue to operate correctly. With this change in place, we no longer use is_swap_pte() anywhere, so remove it. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- include/linux/swapops.h | 6 ------ mm/debug_vm_pgtable.c | 39 ++++++++++++++++++++++++--------------- 2 files changed, 24 insertions(+), 21 deletions(-) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 0a4b3f51ecf5..a66ac4f2105c 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -120,12 +120,6 @@ static inline unsigned long swp_offset_pfn(swp_entry_t= entry) return swp_offset(entry) & SWP_PFN_MASK; } =20 -/* check whether a pte points to a swap entry */ -static inline int is_swap_pte(pte_t pte) -{ - return !pte_none(pte) && !pte_present(pte); -} - /* * Convert the arch-dependent pte representation of a swp_entry_t into an * arch-independent swp_entry_t. diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 055e0e025b42..fff311830959 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -25,7 +25,7 @@ #include #include #include -#include +#include #include #include #include @@ -714,14 +714,16 @@ static void __init pte_soft_dirty_tests(struct pgtabl= e_debug_args *args) static void __init pte_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { pte_t pte; + softleaf_t entry; =20 if (!IS_ENABLED(CONFIG_MEM_SOFT_DIRTY)) return; =20 pr_debug("Validating PTE swap soft dirty\n"); pte =3D swp_entry_to_pte(args->swp_entry); - WARN_ON(!is_swap_pte(pte)); + entry =3D softleaf_from_pte(pte); =20 + WARN_ON(!softleaf_is_swap(entry)); WARN_ON(!pte_swp_soft_dirty(pte_swp_mksoft_dirty(pte))); WARN_ON(pte_swp_soft_dirty(pte_swp_clear_soft_dirty(pte))); } @@ -768,40 +770,47 @@ static void __init pmd_swap_soft_dirty_tests(struct p= gtable_debug_args *args) { =20 static void __init pte_swap_exclusive_tests(struct pgtable_debug_args *arg= s) { - swp_entry_t entry, entry2; + swp_entry_t entry; + softleaf_t softleaf; pte_t pte; =20 pr_debug("Validating PTE swap exclusive\n"); entry =3D args->swp_entry; =20 pte =3D swp_entry_to_pte(entry); + softleaf =3D softleaf_from_pte(pte); + WARN_ON(pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(!softleaf_is_swap(softleaf)); + WARN_ON(memcmp(&entry, &softleaf, sizeof(entry))); =20 pte =3D pte_swp_mkexclusive(pte); + softleaf =3D softleaf_from_pte(pte); + WARN_ON(!pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); + WARN_ON(!softleaf_is_swap(softleaf)); WARN_ON(pte_swp_soft_dirty(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(memcmp(&entry, &softleaf, sizeof(entry))); =20 pte =3D pte_swp_clear_exclusive(pte); + softleaf =3D softleaf_from_pte(pte); + WARN_ON(pte_swp_exclusive(pte)); - WARN_ON(!is_swap_pte(pte)); - entry2 =3D pte_to_swp_entry(pte); - WARN_ON(memcmp(&entry, &entry2, sizeof(entry))); + WARN_ON(!softleaf_is_swap(softleaf)); + WARN_ON(memcmp(&entry, &softleaf, sizeof(entry))); } =20 static void __init pte_swap_tests(struct pgtable_debug_args *args) { swp_entry_t arch_entry; + softleaf_t entry; pte_t pte1, pte2; =20 pr_debug("Validating PTE swap\n"); pte1 =3D swp_entry_to_pte(args->swp_entry); - WARN_ON(!is_swap_pte(pte1)); + entry =3D softleaf_from_pte(pte1); + + WARN_ON(!softleaf_is_swap(entry)); =20 arch_entry =3D __pte_to_swp_entry(pte1); pte2 =3D __swp_entry_to_pte(arch_entry); @@ -1218,8 +1227,8 @@ static int __init init_args(struct pgtable_debug_args= *args) =20 /* See generic_max_swapfile_size(): probe the maximum offset */ max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); - /* Create a swp entry with all possible bits set */ - args->swp_entry =3D swp_entry((1 << MAX_SWAPFILES_SHIFT) - 1, max_swap_of= fset); + /* Create a swp entry with all possible bits set while still being swap. = */ + args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); =20 /* * Allocate (huge) pages because some of the tests need to access --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AAFF7337BAF; Mon, 10 Nov 2025 22:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813466; cv=fail; b=KOqqHtBVTsc3+xa0Szqfz4Q0zlodfdzaBJ4fGWEvtMjX/SvbUh3dToxjt3HadJAIXUPzXmBElbZFLMyJ8A0zfiOHAqirU6Tb0JFALQBBHcXqwRCtvH7S3+2sPAl6V4kocOzfhtOYg2Tr4/M7tMwQedsP+mCOepn8LRb5MalbD4g= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813466; c=relaxed/simple; bh=oxEdzxGrJWd6pCTi5NwurPb3nyfqg6tRupFAuDaPM60=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Z4K3+QukiSKMmIB3euudTKOBv5/E5G42urAJxB5BlfcLSEMHT/D/hHIb3m/i+sgc4aTCHF590EPlsByP28L/ZQ8AYNJMf3t8QeR2MguSq7422+QkCzYQBYwChdwBbkk43YZN4m1x6u1R34OfDX4rjtrApXQbFq34uyztNby9thw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=ZGz0UZWF; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=0F4kRLid; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="ZGz0UZWF"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="0F4kRLid" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAL8d3I009183; Mon, 10 Nov 2025 22:21:55 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=GuY2tVtboAEPardNoS8eL6m9Qb5SaAi7gKx5ePxzR3o=; b= ZGz0UZWFJM0VU4Wgfm5nAnVKsMJOlrZ0skPcSHmWSsX+RRNl0h0j/oYNJCdZXh+q n73El4+2ljdk7iK/x86xrmfjBX5CTcyRqiEfnjvPgS0bFtS5x6PKGGzcIcYrJmnd 48mc7eUfVo8Sdc1XQYmgctDrLNLTyoGN5PDC7KviQplc6qR9VYFfGdvYvt/iM9ut 7bb6WQQ6LbHDjdZQLD4ws7sjhjsFB+86Z+K3WhYa6QrztGlbzoot8E0WM+WzOY7c Dmf2ol4Vzf6Ut+mdLDsDODGPf2rJSSkGwYjwtBre7GU+wwweGsGTcBnv/F1s9pCH XNr7ygzX20852cCzWs+Cjw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abqpug5s3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:55 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AALo2I8040096; Mon, 10 Nov 2025 22:21:54 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010061.outbound.protection.outlook.com [52.101.201.61]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8rypg-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:54 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=nRc3he5gR/IEnRvew4hf3qj2/HiJ7fVdHe2AoECXTlDASHsjcbbIrqT6OarxYc8Fm2NkoXJwekcpy+0raZgECg+a/VuV6Tr52fNnWHQSDXtLIQq32/nJEvc/n2xz+9UHWEdHXP7y+ln13Q3V33ED5b+0npPcsOoG9q0nT5Ct1Ti35lL0lUZNYyECFLAeWt9rrL0lqBTG40d8Kd54AoK8rVce2RAidQe7BFHps5i3yf14zMuij4Dug+hJaZH46tzrzo2QfpO2prAmBcQGQiG0xSpXyUNQbBVDIjAADexlriCO09gNUud4N85WItTpuqFsr8K+vSCIIi/PUyi0AUOJEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GuY2tVtboAEPardNoS8eL6m9Qb5SaAi7gKx5ePxzR3o=; b=SHXHy0OrRij7NyO2ewNia5LCEwPaPeRVUPdK/7nzcLBN+L7pKYGDj89uCHZ3GS5vM5dQ5bNCuRD7v5u04F+hMdbsgETqlfiwTKHPtEodIb1d0WiaMwLC9c2WV3OG+qbqsvqUJfkizY+iKAvxSh12wpt3ukNol0FRRnha2kMPWwcpsb8vo0g622ohysjS9TRwHIxZx28VtbRHlvKi8VI6Rfgs+qQftBgEt+fElcA7DFsxwTyHxB17ZF6r4/DtvU0c/u16wvlA3cFLnazk8BzqMcnaNLHeeBL2l42HhHxfwpPV3uCaYy6oIChXP6jfa5K1wbgKpzBz4m8nDApM1amrog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GuY2tVtboAEPardNoS8eL6m9Qb5SaAi7gKx5ePxzR3o=; b=0F4kRLid8UsSzQc0RPYnEiJJbqeu/S39y6Tf/nl4UZg8+QULS0lZelpgu0ezrUD1KhXYc03RlwdeUp4iLcXj1jL+29/It5nLNN/iVgTDM07zfVj6AGD0JgKsg/xTRdkJZBwInzsJv2qWGxKa72KHM8bpCWacrkM5pmrm5ZvQJCE= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:50 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:50 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 06/16] fs/proc/task_mmu: refactor pagemap_pmd_range() Date: Mon, 10 Nov 2025 22:21:24 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0236.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:b::32) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: aa9510b9-0c33-424a-5667-08de20a78b6e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?p920Y/VbKyjo4rWlfSlNEd45F56UeZ0T9BoSyiMfPC36y1J/0LMCn54eYt5U?= =?us-ascii?Q?JOZyPnwhnitrwFLshRHRtiGfBoDJk08irHQ0gFITvlB4tAPKVGejn+zQF2Vd?= =?us-ascii?Q?X5ppfAjq+1UTjrcjwnfYIDvd3qA6MhWvJeeHTaMMdTn3dGRZvvnYqXE/4rea?= =?us-ascii?Q?uCrJyeGehTCUQ1ISC4tGfKYj+5LxsVjedYzv9zcxTSkhriiHHDwr97CqgdLg?= =?us-ascii?Q?smPfUhvQH/Rxr7qeHN2WhtOSvb1ZQp1NsglLbJhCStCSaH9+nsDouhaoPcx4?= =?us-ascii?Q?xynZQtEKVGU92Oo17tgGr78RUEjHn25l4I3qtJ+Sc6p4T1ukG1Lm5g8yrtmR?= =?us-ascii?Q?B65mZYxN/+njmgfQO8nHPCRkCFEYB4zHJvhgHfw36DisF7Zrt9YkZODjSSaQ?= =?us-ascii?Q?w7BJiPrccYDIUJoN8HbionRdl+vGbyfKcHUkO55HtNsHFeroSAsA6BWTTkij?= =?us-ascii?Q?0FH3TqoLqJVkrzmMTn222OgMvWblMq6ODk7NXGEfhfOvS5iikRaKHhdJPaRv?= =?us-ascii?Q?YS988OdCwY2AouymNCXIo7K0jVyl/Dmt9ioc/dnkYj5Om7kj9DtqETdBq9kw?= =?us-ascii?Q?oErczpHei1Oz8Kj7PiQGk3Jdu2NZhsGJcnzYsGYW6KZGRNbumKtiN5yiWwje?= =?us-ascii?Q?AtP7NwusfytLz9DKkN4r0TQpbtKkEjWYklOMNFCNjQypZFlwf3dpjuqo5bOM?= =?us-ascii?Q?7Q9HP6izSF9NS8QU75SrkuPezJKCCJWkSnwrcnyD5w7wjWtYh8bx7RnC+K3I?= =?us-ascii?Q?FvuCQhmT6QmKhErj/JmDO0mXeXq+ne+jV5jD/n08aboX0rYXoyR0FUh0f8zG?= =?us-ascii?Q?4ZpDfbQ4xV7X5XU4F5egunfn6x0VFTfC5VYNjplSvU+MYJLHMaC0OwLOBp81?= =?us-ascii?Q?5tvi1O1fmT+eSRXK89DXbvuN//GF59kMSuyTjV/xDCVC2EuIvzBmpH30NpgF?= =?us-ascii?Q?NMEHJ0LLt8EuV0RniV284OPmIpa6gcYBRszRlq/2JKFkJA6ZNWwvTJ9zJojA?= =?us-ascii?Q?P0CjgiaGi/OJptZlzp8eAshA9T6bzOV3TwXn0apfF6ZEAjY64hh9SOQeTkHx?= =?us-ascii?Q?rAVy2cRFOKsA42OCsdKnpTdIjmbp4ImEQM55cLdLnxAxbh1uPsQU+qPtPbeE?= =?us-ascii?Q?GXnKkfXam1zKqb5wtvyeen9y/DJ+exqpfyv6Z6lx5D+6kyVy0TFMHbYcNbxs?= =?us-ascii?Q?kFd9woQ94W49eyTXh8kBfS1qnmuE/lJuakRwWOELLWRG0wAgSUtp833aGo7e?= =?us-ascii?Q?l5YXJIH/35VSX4xtEL9xtLQP0h13ePuCP5vuV3GFcvhcPzSURJ9IwRQ29KfI?= =?us-ascii?Q?btaT6zilfDeeaD/dwOuV/ZMm/XhsRUtf/23mWquPKr+3QwrwtJhp465Aywaf?= =?us-ascii?Q?UQ4bE8soYWoMx52OhFPfXO+QAuFgNGizr+SuMHmG9uMwDtHsgcfo3TDzijpb?= =?us-ascii?Q?AUpBexuUirkAX5Ar1zaitaQMAn3TcqO3?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?JOmedkeX/n+YM7fJh7RVABnEpuEJq+s2ePUGaGbYoWbt7IZwkUxgPP8E4c/+?= =?us-ascii?Q?4T8tQsTaRpz2iIrxdFjUyOZdUyUzY/PIHAhcTk2VD5xCmY4+eA84ExE56nQa?= =?us-ascii?Q?AlP2ewLC1m3bTfhu2YstQwauTzAZ/DiRYlVYNnfqx6ALHAmRpvZEeWCVHLsi?= =?us-ascii?Q?0WXrGehY26aygOmBAiB1ELHX1dL433l8V+O3hsnzC+O785ikSHatJxnqJhTQ?= =?us-ascii?Q?nS0J0Ev6aaS+JNqXqfXAjyJAleUisMSrLNxpyCHABVrbJ7w/M55AklHT5njC?= =?us-ascii?Q?cx3BGaI+3bA6Fq7PV+Kgr8YC8cF7/ZqEr4AWCZMdPilWN+VUOyHnVEQdpuOo?= =?us-ascii?Q?bnUAJSrhWnNSuuGt8ua9RBaz0WPquYfWE5NEZhF0yBMzz9SXCq1whFl1GGby?= =?us-ascii?Q?IW3MLqBGjDgby2zdMeSjuMvmbWDobfwwjSK9sOG4VkPrg+tshakW64AwbiEC?= =?us-ascii?Q?mhqyPiPHQT56lB4ukQ4yz/NiaDk38ofnhXbO+cnTs8By+LxszqK6mv0eCCOI?= =?us-ascii?Q?Pobu+mTYZ4ChEpA1IDPxYV2cYZA4RDZMIyxq9ANJ/VMC9L/NwNiqDYgF8BjW?= =?us-ascii?Q?/D5RscGyC7dqDu63EsWCEgxYW7MKU1d/aceRIJLIRGU0SUHadIMEy7s1na9L?= =?us-ascii?Q?7qx2s485idZyt1QhPpL0lXNIfnsBJfxzSCpK3o/sI0hqKQRdkM2htP7L4pZc?= =?us-ascii?Q?lyQRQUMHo2Afu4g6mILGnaYlOwRAg7KAze+lHobZLO4J3ibx4C9eL1BfFq77?= =?us-ascii?Q?zDUGnInbHtr28fFOo5QvbDxFMBun5kyd3dtbY7empFnfvqHJ6peQ2oGQkfxf?= =?us-ascii?Q?Xnxzud9JfbcoG0aOmIcnuMEMGZlxLGPNP+pib69JjzDOtLOPY2A5+hPmCDPN?= =?us-ascii?Q?uDkf3xn4uJ9LAAEvcGcHdlgRyHDY/EcFiAjuYpYRIHB+SG5BDYoMOXZTDTwH?= =?us-ascii?Q?SCebwQ1nPDRwBlmJHj7bFLNRLCVDJH3J7uuNlHEIK1RKbr1MSm/rm1JUUvE5?= =?us-ascii?Q?UqDTPbVvE5qD203gobJfJ7q4/nzfoPCZScijy/NgwKfyfAsv0Vg08a4zYKX4?= =?us-ascii?Q?QE9eRB0eVevDEkGmc0UDf7PI/wMIvRirwdykPX9Y1xxjTk5ylGtdouJ2qS44?= =?us-ascii?Q?wiSmtlVw9GqldpHKY3PTnCMTCWE7yFXhy2M52n5QVFKmVXP5ahu3Euhz3WK6?= =?us-ascii?Q?xXGlNHTxZf1Egj8rOLhK8iClMjovh1rgeGY9gBK/nSCgGKKDlsaF7N9tzo3C?= =?us-ascii?Q?sQxgyQF7uzGmbYZVjLfuwNGVSuJwBt0xtYLjyrGwbcirpnqCP1Rvlic3nOUx?= =?us-ascii?Q?2BAxYPuBPSalKBsidYv/rjyzytXsZct6G4LKfsAtE5s46Qto4eU5vp59bTtY?= =?us-ascii?Q?a4goYBcH0KB0qkH3iUG1hiNJw0K4Y3oqx1PSMwYgV0143Lq61QG/yQ0bzbrB?= =?us-ascii?Q?JigsjbeM2E3ldol+hMXO0HuBFhf1xs2gvMb2xLxo8ENUI3mjBVj7laer+umu?= =?us-ascii?Q?1VFEZK/dWsgHSSiCdEo0DnuBpjyE0ItZS/PMBVuZL6YR2wY1qtpWcbamAWhf?= =?us-ascii?Q?5BH+iOE1Bh3xw6XP4doLvgEcgnUlE9Jh+fUOrhV6morBecCN8dIIwD4QznA2?= =?us-ascii?Q?XA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 6wvzntT0VGo5cIrmfs457FdgMfq2Emazd7ngC03DCes6VVLLVVf2OZ6a6NCdFhTF8RuoFCaLJEXa2b1nKHtop86S9zyF961rXiO+8Hmpi3j4y7i9LZP+m5xSR0BzXA1lSJTVFdmTFpv7cHLFl4tHBNta55ZjUzvimV1Uques+4VXSO1qT3z5bc2e+oTQ8iP19uLXMg2L2g9CUx5dKxqEJnKpwn0xyQFbLGsQy8C8yWLF+DgEvEhHUnMzhXFR+I8OZJRoXcaXKIL+aZeBokxRyYiJ5q4OkYc8eIlUQmSSXojzy1+8x1uUC3RAjUBlS7OW/060Pf2NyBuUXZQg4R5C+fymTzYnQieTSEolZRwn3132hR84GU6dOepE9VQreJ7wgkYbLv7ov+7OQ2urToDIiW50MUOONKMYoTyDa37qWT1wtC4tCHMedhqCAUlD3ZszP6IkXnswr6jU57Gl6k0dcVNTdENGjanU8XDMc5lEXCgbza59M/SzjlotOOrDJglUAeN48TUtJLbl9r1CEQJsODaAw1zKy6ifQTJGhSTBUO8aX6J7A3pm6UJGH3wXoNrKkfeSlwYJ8skfRiDZdi1IPy1R/XzRVSEX6QJ5Xqad/yY= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa9510b9-0c33-424a-5667-08de20a78b6e X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:50.4669 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mFp76T0bpmz4L5PsfzGaKrIvjfiliksINl2hjs+/j0Hxgt1miSAnGYFMLMWljl36mwK2axA2ZvVyazn1XYJ9T6cz69gXf0/JNAv0UJTGoVY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE3OSBTYWx0ZWRfXze5O4FXLhfkl 5Lv6/Lgo+/hophBFGpJoT3m3aQLgVuhPjC6HQV1kcpM6q1V5RLGY2YrH4KCC6p6IgNexivqHiVZ Z36rH+uCbinNwXDDxis91J0Ml0cuNXkMYata3PerEhaReSeza6/yN9++xaMdtLsFyHkP3Qengcx nLzb0Q+sHagzzhdmW8paAw5DxBBkYQwtnp5vS/zRPguF7Y71rj+JBUuAmbmFP+eEccSyXzPmOmX 38dSj9AT6+QEdGX80IS+nmIrVUIUDdmvuMdxMih8UEBZDP+ypPOTDsNXsv1is1e07kU+yVHIY7+ +zedOUPtKaslDGzjBW8yBNwiAWmlYTvzxJRradM1rfTqB/p9bpBCgMbMoC4TnqLIGF32KEg6KU3 3vpaEjdWE33uStbO3QHkLOfZAmhb0A== X-Authority-Analysis: v=2.4 cv=H5rWAuYi c=1 sm=1 tr=0 ts=69126583 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=r4PJAH3pRi85-HfG15IA:9 X-Proofpoint-ORIG-GUID: cvej-drcshnieka4Fs_gMFCcwA7JLDWR X-Proofpoint-GUID: cvej-drcshnieka4Fs_gMFCcwA7JLDWR Content-Type: text/plain; charset="utf-8" Separate out THP logic so we can drop an indentation level and reduce the amount of noise in this function. We add pagemap_pmd_range_thp() for this purpose. While we're here, convert the VM_BUG_ON() to a VM_WARN_ON_ONCE() at the same time. No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- fs/proc/task_mmu.c | 146 ++++++++++++++++++++++++--------------------- 1 file changed, 77 insertions(+), 69 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index ddbf177ecc45..5ca18bd3b2d0 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1984,90 +1984,98 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, return make_pme(frame, flags); } =20 -static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned lon= g end, - struct mm_walk *walk) +#ifdef CONFIG_TRANSPARENT_HUGEPAGE +static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigned long addr, + unsigned long end, struct vm_area_struct *vma, + struct pagemapread *pm) { - struct vm_area_struct *vma =3D walk->vma; - struct pagemapread *pm =3D walk->private; - spinlock_t *ptl; - pte_t *pte, *orig_pte; + unsigned int idx =3D (addr & ~PMD_MASK) >> PAGE_SHIFT; + u64 flags =3D 0, frame =3D 0; + pmd_t pmd =3D *pmdp; + struct page *page =3D NULL; + struct folio *folio =3D NULL; int err =3D 0; -#ifdef CONFIG_TRANSPARENT_HUGEPAGE =20 - ptl =3D pmd_trans_huge_lock(pmdp, vma); - if (ptl) { - unsigned int idx =3D (addr & ~PMD_MASK) >> PAGE_SHIFT; - u64 flags =3D 0, frame =3D 0; - pmd_t pmd =3D *pmdp; - struct page *page =3D NULL; - struct folio *folio =3D NULL; + if (vma->vm_flags & VM_SOFTDIRTY) + flags |=3D PM_SOFT_DIRTY; =20 - if (vma->vm_flags & VM_SOFTDIRTY) - flags |=3D PM_SOFT_DIRTY; + if (pmd_present(pmd)) { + page =3D pmd_page(pmd); =20 - if (pmd_present(pmd)) { - page =3D pmd_page(pmd); + flags |=3D PM_PRESENT; + if (pmd_soft_dirty(pmd)) + flags |=3D PM_SOFT_DIRTY; + if (pmd_uffd_wp(pmd)) + flags |=3D PM_UFFD_WP; + if (pm->show_pfn) + frame =3D pmd_pfn(pmd) + idx; + } else if (thp_migration_supported() && is_swap_pmd(pmd)) { + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + unsigned long offset; =20 - flags |=3D PM_PRESENT; - if (pmd_soft_dirty(pmd)) - flags |=3D PM_SOFT_DIRTY; - if (pmd_uffd_wp(pmd)) - flags |=3D PM_UFFD_WP; - if (pm->show_pfn) - frame =3D pmd_pfn(pmd) + idx; - } -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - else if (is_swap_pmd(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); - unsigned long offset; - - if (pm->show_pfn) { - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry) + idx; - else - offset =3D swp_offset(entry) + idx; - frame =3D swp_type(entry) | - (offset << MAX_SWAPFILES_SHIFT); - } - flags |=3D PM_SWAP; - if (pmd_swp_soft_dirty(pmd)) - flags |=3D PM_SOFT_DIRTY; - if (pmd_swp_uffd_wp(pmd)) - flags |=3D PM_UFFD_WP; - VM_BUG_ON(!is_pmd_migration_entry(pmd)); - page =3D pfn_swap_entry_to_page(entry); + if (pm->show_pfn) { + if (is_pfn_swap_entry(entry)) + offset =3D swp_offset_pfn(entry) + idx; + else + offset =3D swp_offset(entry) + idx; + frame =3D swp_type(entry) | + (offset << MAX_SWAPFILES_SHIFT); } -#endif + flags |=3D PM_SWAP; + if (pmd_swp_soft_dirty(pmd)) + flags |=3D PM_SOFT_DIRTY; + if (pmd_swp_uffd_wp(pmd)) + flags |=3D PM_UFFD_WP; + VM_WARN_ON_ONCE(!is_pmd_migration_entry(pmd)); + page =3D pfn_swap_entry_to_page(entry); + } =20 - if (page) { - folio =3D page_folio(page); - if (!folio_test_anon(folio)) - flags |=3D PM_FILE; - } + if (page) { + folio =3D page_folio(page); + if (!folio_test_anon(folio)) + flags |=3D PM_FILE; + } =20 - for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { - u64 cur_flags =3D flags; - pagemap_entry_t pme; + for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { + u64 cur_flags =3D flags; + pagemap_entry_t pme; =20 - if (folio && (flags & PM_PRESENT) && - __folio_page_mapped_exclusively(folio, page)) - cur_flags |=3D PM_MMAP_EXCLUSIVE; + if (folio && (flags & PM_PRESENT) && + __folio_page_mapped_exclusively(folio, page)) + cur_flags |=3D PM_MMAP_EXCLUSIVE; =20 - pme =3D make_pme(frame, cur_flags); - err =3D add_to_pagemap(&pme, pm); - if (err) - break; - if (pm->show_pfn) { - if (flags & PM_PRESENT) - frame++; - else if (flags & PM_SWAP) - frame +=3D (1 << MAX_SWAPFILES_SHIFT); - } + pme =3D make_pme(frame, cur_flags); + err =3D add_to_pagemap(&pme, pm); + if (err) + break; + if (pm->show_pfn) { + if (flags & PM_PRESENT) + frame++; + else if (flags & PM_SWAP) + frame +=3D (1 << MAX_SWAPFILES_SHIFT); } + } + return err; +} +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + +static int pagemap_pmd_range(pmd_t *pmdp, unsigned long addr, unsigned lon= g end, + struct mm_walk *walk) +{ + struct vm_area_struct *vma =3D walk->vma; + struct pagemapread *pm =3D walk->private; + spinlock_t *ptl; + pte_t *pte, *orig_pte; + int err =3D 0; + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + ptl =3D pmd_trans_huge_lock(pmdp, vma); + if (ptl) { + err =3D pagemap_pmd_range_thp(pmdp, addr, end, vma, pm); spin_unlock(ptl); return err; } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#endif =20 /* * We can assume that @vma always points to a valid one and @end never --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DC573446A6; Mon, 10 Nov 2025 22:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813466; cv=fail; b=HViEHrVwMyTEYIP8ffqLlqxCpaUaY6vlGDaCs+VljsFE6A/0oPJnePYFFYmFuu/A/CRU400dYZxMQLypdf19gXtvTTjWVdbflZk9YD1uslbhe9lJF7NQ0n98freJzUd+aaFMhezzQI26UHe2/zjCrJn82cHXJRjkEtDHkig2DQA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813466; c=relaxed/simple; bh=gJ8zRazqzQ9hDxhtbg0qAlApDfcn449VuQ4f4oPaLcc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Fg79oaQKxq+iTnElp6TvM3ng0jKPAjgQ+1IimoBOtAGY/Cvfwe+dQGHEMa0tBpq17wi8soTZFCTWZXbWV48SYSvHfxTlfvI85TbXlldN4XZpnF4iiGKox403GITQG6QaKcmo3j2mfnhro7EPxb0yzU68s1vn0z2rhmiTyuWxcSM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=nTjQ4+NP; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=fl7EL6/7; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="nTjQ4+NP"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="fl7EL6/7" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAMINsW025996; Mon, 10 Nov 2025 22:21:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=0JzEYXLZhk+XPgEC4WpeH2fagEy4AWaKQdESXVSnsms=; b= nTjQ4+NPSG4FKJmkSoPAiLvYB59zj/DTIOe3O4rbQSW3HO203zSRFiVTkfOK3XtI Wvmy702pTkoLwE1s72fE6SSuRbRDWnjEtEwa1yJp97Po3esismy4HsXEd4fbr/DE 1pgWB1l4tDu/liSTEmlqV97IuslXSgc3m+GmP1l160WnWNYpJ7/S8z3+6zBmjg8s ZNTIelO9r+46K3MfFRCc/rNzJX3bAQ+tEqPdJ3s3ZsvacScKtFdz5A5b2Uu6jv7F mPLcMb2bvBqWQzpyqeE+H5cRO59CvfU80e34gFfoCNXuLcxAWSadLUf2YeGN7OFP FS81UA3eNTAdAeXO6H2mPw== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abr5vr2b1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:57 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AALo2IA040096; Mon, 10 Nov 2025 22:21:56 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010061.outbound.protection.outlook.com [52.101.201.61]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8rypg-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:56 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fGXyEQvgcjlpneja9tZkWei1iGykF+SqSpAcLv6BQhughEBf5reKUVsAAX1+CXE/s7nQqD9K+FH7rSWkO5kkRxIoynNkmymnvDb75Pus6xCXDTObHdCDeZk7apORNscICWkfB6OQ2BlNWB8B5+uoxShIQP6I4zgeOLlX7DFjXVrxwh2orbTO3zdzSQ+e6nDILOyYw6otPzJ4/G3EKg+kOEtz6CEm19tTTEq56FQ34xLHanrtuxAZsGf45voQyN1Bk1q+fobpTuI9Orpnq+mNio6hH7E/Wg7D3V575K9s/a55i71eOTf76U29GTXw0WyDmXxnQOx1derr2ujRVhMfug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0JzEYXLZhk+XPgEC4WpeH2fagEy4AWaKQdESXVSnsms=; b=sm9RxcmpmbfpdOYzDXGNVr4m1K21egRlNzGorb3tqs987lHUzjivB1WgSt2vY4ECJ+fPOVIOHRUjOv6bKDnfAPHRpUHuxhY9aKk2G2ov7r2WkJ7uMKT0Im8LjjP4sM50Dq+xXQVVWghUhfvo+FbjHPzERD6BzM6pO8OInwMfpCY8MFSbml36WiqF1SC9zG9ipOCKP9psa22GghU145JXBNIGZ56GE/10i3WAGrBNcmMrFdUj8GO/IfcNoTtDS4/z32nqaSYbaQ1Vz16sJdSuw4VUKxzj1pcGrH9BgXB6L4Lbh3fD9RGkYznCtatoc5FYA+TsdwTN+Ml8K0mUuMR+WA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0JzEYXLZhk+XPgEC4WpeH2fagEy4AWaKQdESXVSnsms=; b=fl7EL6/7z+lhm9oTUH6kVdYzUsWgia2sox8RKUok5boDWMea4LBvtmHbmAhqZxV7MsV/9z/+SXhi8aX3lcixOBJVbL0j1wg+s4UCYHafYF4K43ugLkJHtDlVQp+nuhmy/GyRmNlnEF/gz9Ms1N2FZjnT1ibLGVPU7qZxlL1IZ3Y= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:52 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:52 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 07/16] mm: avoid unnecessary use of is_swap_pmd() Date: Mon, 10 Nov 2025 22:21:25 +0000 Message-ID: <8a1704b36a009c18032d5bea4cb68e71448fbbe5.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0100.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2bc::12) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 1249e555-dc60-46be-801f-08de20a78c97 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Q2aVa2f19MTK2T5KBmtVoeDLOYTKxanPoS5Md5EIXTcdsrEspf+WQxwM3uLl?= =?us-ascii?Q?VL16kfgtPxRYtPCoJUP9w5riu94xDEGeL2VhFZYY09PKnFJ645FaBXU+5gg/?= =?us-ascii?Q?ODXjvR1Zdi4AYc+PoQ9S3OEs7oueh12quF2PkNFhzhggyeEg2r/U+Co26zys?= =?us-ascii?Q?2ItzE13wuy4YYTSPtZjF0e3ztYetYG/ACwP6Wc2aasDpsp//fI4iGGCuG2gy?= =?us-ascii?Q?sVNdO6U5eRqHrKaVoRbMOJp561ziuHZlMzz77fNOWQ8F14cozUPV5JpVryNb?= =?us-ascii?Q?rS+omEo9fzn+mCJlFLYZc33zzLWwL4K7pRZea8b8eiuxSLx31606R4HdP4DS?= =?us-ascii?Q?baqOdwJHpwUiCdLH7xDeJ1aq9vnIEZMIKK4n4OstKGamgJQg/uyg2tvPVg3n?= =?us-ascii?Q?bTjw1J1KQ8WwdzO22VVPskpUOe5aM0oY2GYu1BM5KkZyDDKku9lBjuyb6Ej3?= =?us-ascii?Q?FVgVl6xrnY6CBzLAcsowMWHTocbEOTIzi4dzHt6kpEtL/vZEgubhzbsrBAsC?= =?us-ascii?Q?tKspO8qw4wsJvRdpJknYQDylIsfSpR5028TBEE3vHJlm77GALmWRs0RSwep/?= =?us-ascii?Q?o6VDjW9oFgNRlljanJ0m399srk3wAUDkrzhn5ldW+bQmVUMrWo1L8m/PvEj7?= =?us-ascii?Q?yfcAxETnHk2a40yg78ICT6VJMUbRx2iNiRsnH8FGTueTYo0t20T29Gn3aNir?= =?us-ascii?Q?JFixp1/jpKYOnfcrbkbE+k3gNpCuQYH1nIcjVVTU+BfUFWxStrfKaXu2x19J?= =?us-ascii?Q?Qr+voQXk8n0tqp1ObAllJXNjMisKMbVo7VDypuHfS/RMU0ShjazJBRSux4er?= =?us-ascii?Q?8y0e5d22P+2X3gyeWgQkFJenS6wnujZeDWouCVhePMkn6lYUgfIXMTgOgJOQ?= =?us-ascii?Q?4pyZHUotbbr7n9qJKXZOEvwriKNjYY57X7W9+r2c9nXRWmfegHUv3yq6KeW/?= =?us-ascii?Q?ZHm1Arr7l2l9SMFf7fCdXG+ADijDmeygT7tyiOXldNG72K5/fciMiHLIeC8W?= =?us-ascii?Q?GLguLFRcXrE5ztN+yp+qlBh3IYtz2NLHbSFozmeGMWs+3GDW9osIE0ki12wk?= =?us-ascii?Q?w3i0g35g82JBapLZCMt1PKfYPTuaUDnoy17GxDKmuxLm3S8OqDro5ibUzH3W?= =?us-ascii?Q?GGXQylTexTYLBkEWZO+mmWp70K19M0ZghleEfDPIrN8Ab3Zx5K4q7S1UDGX7?= =?us-ascii?Q?uN7TEy1j7r3Pv0qduPrxzhdDJZoNsmcVnLVQnv3aA9krJ0iv2w/b1fOHgdp5?= =?us-ascii?Q?IIN8KuWKJeqK1FJyWnu2t7Ci7Fu+pnCV6gmFnPnbKt9SCBIfHUuR/Hi8Zx+E?= =?us-ascii?Q?udskwWGgT5GqeNUjr43lsC4oymv855NyMLlcM3605rdLssTN8qUSKAFKjDM0?= =?us-ascii?Q?4XzM6uZ0nDmmtzPlqv7urttQfNxg7j+GL1F/xj+4sveuc5dp8HKrSILkfXwK?= =?us-ascii?Q?MbqkI6qUvrKQ1WdMsMWOJSNdvYKnXyAx?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?9AKIXyBxKjFnva7hFW832AzOupn4so/6ZbGNfjh46JazzGP3w+z5o98J/cKN?= =?us-ascii?Q?NKn4hV8CurBbcFRmqcGR+/ikbTMkW7ERRO1zGvDEk03fYOrsBfgOxgRVa/S9?= =?us-ascii?Q?Bi6juJW/hnDnIy62WRCFKlH+13FnhRx455v/d3IoPVbxC6jygoEniOhvHYCO?= =?us-ascii?Q?mJ0tAyyLCD/BrMypcUr6CPfCypD76TxrxG4styEAscUo2Tqq40GbK7kzbwrZ?= =?us-ascii?Q?/HL/sPu11+THBVgY+FFnFUl7JtPIq1w5FzFe949yH2D1wXs7L3vu7IwhefAC?= =?us-ascii?Q?iLYjSvfVTsmkNGC6+tPKUrlyIX5TwrSmEE78bc+hC5SZWMnH6C/vJBZf3b79?= =?us-ascii?Q?RJJK7IO21IIDpkL0XJ0U+njXefmWKeIF+bDXa2edsItzlFLYMN5Mr5rCkyej?= =?us-ascii?Q?aYvYtPuyS6ueNNL6PtY7QvcFJhZUxNCySfEgZZoMj2a6iu+/ul50ohWymvgz?= =?us-ascii?Q?hV8YVy3d8eRqxd2A6ujAhx6Bwd9891ppApoPW+V0m3U4o677xnyPYHMpoTCZ?= =?us-ascii?Q?d6125NF87O0hdKZ3bDm6f/wnyZ9sIBwANYXGeUNXUAxcg8Qx/QQWZyHmpb17?= =?us-ascii?Q?i++QQEbIZr/GpN5brDId/CpWwkIA2OJ7wpvd1IaOa6V01uNA2wYRntrMnMgK?= =?us-ascii?Q?xDb6JXsgPzeKI7FLA9kj5q5Y1+e2Q2janKUXhrKWXICNdG5fu9qlBWUmhA7v?= =?us-ascii?Q?3EOtrHSEJ7y1jLguRKoUu4s3OA3yRzpa5z84y5exsZfF18ZwD90s60VkrpEt?= =?us-ascii?Q?/eZXQZ8+O58+45XO1HHeWynQAjDaZFbvcMslPGlhjGYYWWgl2S/zV9kR12SL?= =?us-ascii?Q?IdYOWm10ZJXuK+IZq+jVb4XELGD5UWWklRxwvJtOPIMuaE7xJFYRcS4R5GJM?= =?us-ascii?Q?bAcHlZ1WiIYyc+QdEH7Wky5JDDDJ5kFuAaj/F5/8MPErTjWK7NO95+QjDKdL?= =?us-ascii?Q?4fjfqjwC7rR7zF9ANdxesU5Z/OWPeQUy10fkUEQYJUVnEdtV++ICiC9moMrQ?= =?us-ascii?Q?rYVg5ZXR6JKjmcM7FbYr6zYGezi73/lMIhZ+efE9YIAzkmBT4t3uP03Fcdzy?= =?us-ascii?Q?RQ1/0ZtMrfSKfvVQj54zI6p2kYr/BDV64AyOn+BjdKBUWO1fQvxqYtc3yJY9?= =?us-ascii?Q?nHzr+//zfwMOyX6CTpeBUly+E5cmu2z5nFVaJEWRW1SzggO13qjwbECKzz4a?= =?us-ascii?Q?Z+zNwQAB8FDqyRbmZNjHLuBPyGFiWrVz80CQLHz1Gf3kpP6CgzSH11+UOOwp?= =?us-ascii?Q?+Us8nckpQlZhWYef1GxFqU+Xi6Wg1ArQZm8XDOfdfQjD1t/OmnhtkE3iAkxP?= =?us-ascii?Q?C+L2xr+LKo9wyqF2CDw8GFISucvsBFZSklGWf5DOhEmXXnTq6EKJvXCtzD7Z?= =?us-ascii?Q?qvJBEdU/IEK7dJu2dZ6wB8mDAzfi18tr/xY5kiASdiJo6Ny3wQUH13QD0ykM?= =?us-ascii?Q?nMWiNlOKTDJtoEGe6M1kPFakmda0LPxog91X/sHVqcwsOFrLbeY3Zg//zRK0?= =?us-ascii?Q?z0ndTMQ5YsPeTs3dD2lyWnh33i9loUfIZksW97HWE46StLdKsk8lBluO2riP?= =?us-ascii?Q?+u7o9+bgkVpFf6CJpeojH/zcobXyaVz4mzkuyA34XpUA/oLxcM2b2qsZTWlL?= =?us-ascii?Q?bA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: bMTjKeP1uxsQq7y7exb/GvRleSVqvq9xQQR7ZEy/02aCc8Qg/5eVv4QpQ93ZaoZ2tF6EiMHg3/F3AkwtChyEt82TjZgaVj1cnvywGC+lFaULg2dBVDsHybWnwYsK7wZEvXS4ktuf+99WLhNgcB5Gr8w3Z52CJ4k1V7pNhWjfI4tdTLec7vEoKn/f58bzfU1x1mrCqsTOYoZa8zoGEjFYd3S2ACvEkbgja/0O+qGDHXFeIZRicgpLIwP/zPM3w1JuFzkxJhzgEVe9G2mvFHhbZpoC7UK5Kdve/IxdSaDIevQhQjGc4w+PyVoLaaA4+pNbd5Cl9ngpm9ylyx/rhka5FWd3j6rkCpHpw8l32LgvvyFnm4JNAfjtY3jnx4PX++OljtMpIneI9TV5iSS1AGb/RqrLglGtZoOrWjAQoh/y/K18PXNy048paFjIebA8vnIKFIztP6x/6aKf6E/q0VSL3zWwD9F46q6o9RAwk6W3b7BtzjOfMW6pCSHKdYMHQ6IrHJRUApEJaTf3zQj6s/2AcVGlsS8vWV5jEKa1GHiXMRtK9ouDrE2lHEUTr1StPmCKm/yEESu9zDTSPjwLmVnOq2WVRhMmljhcIMY0Q5riOQE= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1249e555-dc60-46be-801f-08de20a78c97 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:52.5096 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OLcDEvyKUzvykc3IiIDLzqTATS3/+FGIEmc3vMVbmzQ0tRESghbM+XsP5wsOBbu1Gz2vHTaYkI1+nvHkvMyHCqUcvQKUOU9ExoAyGLT3/a8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: -of_CjyfhY9ReBR_qkHI1eLsRWcfXJKy X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4NCBTYWx0ZWRfX5fIXtJ9DfdVW OjNQvMfIjzxzA/13zv594RBTWTDowtbxV6uwBTx3vq/vEqtYjJXocUnGsx57aAK5oONsAWzQiV2 LGmypprLcEmZif/I0ecNdxG8K8OqGy9kMI51b5E+Yek8Yv6mhWqCJXiNG/RDch41xlQXuAYGkvA kqJVqwQpjzM9fE4SZSGGTaaRzOqub1N2TVLkk16UxDcWrxj+rNT3yqw0xXZYnRSXTI9lF9NM/jz 0jjGL/fLEvWUcy/n3RCCI5PPfHeWanzj2v+2/MHSMP/jfsB9qIFYoobyCSW5RKawd1RuoIB02eQ qCX98SNaWHN/VK6Z9aH+kwa+oej/CsjtyplJ8B49Js3cObkRDpYN1dtu6j2YvmoXGLqldaBC+tP gnPJSc/F44F9RuQ/cm7lsWXZu2o8Zw== X-Proofpoint-ORIG-GUID: -of_CjyfhY9ReBR_qkHI1eLsRWcfXJKy X-Authority-Analysis: v=2.4 cv=YN+SCBGx c=1 sm=1 tr=0 ts=69126585 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=96_WKPqrBKIKR3zHGkMA:9 Content-Type: text/plain; charset="utf-8" PMD 'non-swap' swap entries are currently used for PMD-level migration entries and device private entries. To add to the confusion in this terminology we use is_swap_pmd() in an inconsistent way similar to how is_swap_pte() was being used - sometimes adopting the convention that pmd_none(), !pmd_present() implies PMD 'swap' entry, sometimes not. This patch handles the low-hanging fruit of cases where we can simply substitute other predicates for is_swap_pmd(). No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- fs/proc/task_mmu.c | 15 ++++++++++--- include/linux/swapops.h | 16 +++++++++++-- mm/huge_memory.c | 4 +++- mm/memory.c | 50 +++++++++++++++++++++++------------------ mm/page_table_check.c | 12 ++++++---- 5 files changed, 65 insertions(+), 32 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 5ca18bd3b2d0..b68eabb26f29 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1059,10 +1059,12 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned lo= ng addr, bool present =3D false; struct folio *folio; =20 + if (pmd_none(*pmd)) + return; if (pmd_present(*pmd)) { page =3D vm_normal_page_pmd(vma, addr, *pmd); present =3D true; - } else if (unlikely(thp_migration_supported() && is_swap_pmd(*pmd))) { + } else if (unlikely(thp_migration_supported())) { swp_entry_t entry =3D pmd_to_swp_entry(*pmd); =20 if (is_pfn_swap_entry(entry)) @@ -1999,6 +2001,9 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, if (vma->vm_flags & VM_SOFTDIRTY) flags |=3D PM_SOFT_DIRTY; =20 + if (pmd_none(pmd)) + goto populate_pagemap; + if (pmd_present(pmd)) { page =3D pmd_page(pmd); =20 @@ -2009,7 +2014,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_UFFD_WP; if (pm->show_pfn) frame =3D pmd_pfn(pmd) + idx; - } else if (thp_migration_supported() && is_swap_pmd(pmd)) { + } else if (thp_migration_supported()) { swp_entry_t entry =3D pmd_to_swp_entry(pmd); unsigned long offset; =20 @@ -2036,6 +2041,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_FILE; } =20 +populate_pagemap: for (; addr !=3D end; addr +=3D PAGE_SIZE, idx++) { u64 cur_flags =3D flags; pagemap_entry_t pme; @@ -2398,6 +2404,9 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, { unsigned long categories =3D PAGE_IS_HUGE; =20 + if (pmd_none(pmd)) + return categories; + if (pmd_present(pmd)) { struct page *page; =20 @@ -2415,7 +2424,7 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, categories |=3D PAGE_IS_PFNZERO; if (pmd_soft_dirty(pmd)) categories |=3D PAGE_IS_SOFT_DIRTY; - } else if (is_swap_pmd(pmd)) { + } else { swp_entry_t swp; =20 categories |=3D PAGE_IS_SWAPPED; diff --git a/include/linux/swapops.h b/include/linux/swapops.h index a66ac4f2105c..3e8dd6ea94dd 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -509,7 +509,13 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) =20 static inline int is_pmd_migration_entry(pmd_t pmd) { - return is_swap_pmd(pmd) && is_migration_entry(pmd_to_swp_entry(pmd)); + swp_entry_t entry; + + if (pmd_present(pmd)) + return 0; + + entry =3D pmd_to_swp_entry(pmd); + return is_migration_entry(entry); } #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, @@ -557,7 +563,13 @@ static inline int is_pmd_migration_entry(pmd_t pmd) */ static inline int is_pmd_device_private_entry(pmd_t pmd) { - return is_swap_pmd(pmd) && is_device_private_entry(pmd_to_swp_entry(pmd)); + swp_entry_t entry; + + if (pmd_present(pmd)) + return 0; + + entry =3D pmd_to_swp_entry(pmd); + return is_device_private_entry(entry); } =20 #else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f6c353a8d7bd..2e5196a68f14 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2429,9 +2429,11 @@ static pmd_t move_soft_dirty_pmd(pmd_t pmd) =20 static pmd_t clear_uffd_wp_pmd(pmd_t pmd) { + if (pmd_none(pmd)) + return pmd; if (pmd_present(pmd)) pmd =3D pmd_clear_uffd_wp(pmd); - else if (is_swap_pmd(pmd)) + else pmd =3D pmd_swp_clear_uffd_wp(pmd); =20 return pmd; diff --git a/mm/memory.c b/mm/memory.c index 7493ed084b99..fea079e5fb90 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1376,6 +1376,7 @@ copy_pmd_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, next =3D pmd_addr_end(addr, end); if (is_swap_pmd(*src_pmd) || pmd_trans_huge(*src_pmd)) { int err; + VM_BUG_ON_VMA(next-addr !=3D HPAGE_PMD_SIZE, src_vma); err =3D copy_huge_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, dst_vma, src_vma); @@ -6350,35 +6351,40 @@ static vm_fault_t __handle_mm_fault(struct vm_area_= struct *vma, if (pmd_none(*vmf.pmd) && thp_vma_allowable_order(vma, vm_flags, TVA_PAGEFAULT, PMD_ORDER)) { ret =3D create_huge_pmd(&vmf); - if (!(ret & VM_FAULT_FALLBACK)) + if (ret & VM_FAULT_FALLBACK) + goto fallback; + else return ret; - } else { - vmf.orig_pmd =3D pmdp_get_lockless(vmf.pmd); + } =20 - if (unlikely(is_swap_pmd(vmf.orig_pmd))) { - if (is_pmd_device_private_entry(vmf.orig_pmd)) - return do_huge_pmd_device_private(&vmf); + vmf.orig_pmd =3D pmdp_get_lockless(vmf.pmd); + if (pmd_none(vmf.orig_pmd)) + goto fallback; =20 - if (is_pmd_migration_entry(vmf.orig_pmd)) - pmd_migration_entry_wait(mm, vmf.pmd); - return 0; - } - if (pmd_trans_huge(vmf.orig_pmd)) { - if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma)) - return do_huge_pmd_numa_page(&vmf); + if (unlikely(!pmd_present(vmf.orig_pmd))) { + if (is_pmd_device_private_entry(vmf.orig_pmd)) + return do_huge_pmd_device_private(&vmf); =20 - if ((flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) && - !pmd_write(vmf.orig_pmd)) { - ret =3D wp_huge_pmd(&vmf); - if (!(ret & VM_FAULT_FALLBACK)) - return ret; - } else { - huge_pmd_set_accessed(&vmf); - return 0; - } + if (is_pmd_migration_entry(vmf.orig_pmd)) + pmd_migration_entry_wait(mm, vmf.pmd); + return 0; + } + if (pmd_trans_huge(vmf.orig_pmd)) { + if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma)) + return do_huge_pmd_numa_page(&vmf); + + if ((flags & (FAULT_FLAG_WRITE|FAULT_FLAG_UNSHARE)) && + !pmd_write(vmf.orig_pmd)) { + ret =3D wp_huge_pmd(&vmf); + if (!(ret & VM_FAULT_FALLBACK)) + return ret; + } else { + huge_pmd_set_accessed(&vmf); + return 0; } } =20 +fallback: return handle_pte_fault(&vmf); } =20 diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 43f75d2f7c36..f5f25e120f69 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -215,10 +215,14 @@ EXPORT_SYMBOL(__page_table_check_ptes_set); =20 static inline void page_table_check_pmd_flags(pmd_t pmd) { - if (pmd_present(pmd) && pmd_uffd_wp(pmd)) - WARN_ON_ONCE(pmd_write(pmd)); - else if (is_swap_pmd(pmd) && pmd_swp_uffd_wp(pmd)) - WARN_ON_ONCE(swap_cached_writable(pmd_to_swp_entry(pmd))); + if (pmd_present(pmd)) { + if (pmd_uffd_wp(pmd)) + WARN_ON_ONCE(pmd_write(pmd)); + } else if (pmd_swp_uffd_wp(pmd)) { + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + + WARN_ON_ONCE(swap_cached_writable(entry)); + } } =20 void __page_table_check_pmds_set(struct mm_struct *mm, pmd_t *pmdp, pmd_t = pmd, --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3CB83446B3; Mon, 10 Nov 2025 22:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813467; cv=fail; b=S4pLN9U0TYYPrIHPmHlKw2dyiQvlDWm7938FfZ7iJIwqWpHi/xIYUsEzt7pv76A8c9LoZwnpmtzy8B46Fc68qbRyKe5nJmjX0cZQ6P/T1AaXma32EXv72LG78xYAtnM5aMMaoWlnxBm7E+o/JPCshYcptbQGEFTba3siC9Uhkyg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813467; c=relaxed/simple; bh=AZ3s0B+uhePQZXTS6i9X9ezX6E6ciUS6QFxCmlxX9io=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=QFXwg67KorsbO42ZLFpuZ1eM4MWi+SfWUmRSuGcyDkOnvKbzAJP+dDl+x+mPT8GUeQMGRbDHG9KQMWfPJyH9Gt/62mZJh+NCeOFUHHa+yHuF6xj9VSkVBdrYz0KnW/kIkFwmZ3hVAmWqzxqTw/OyvIjQ0V+6M10/Jit631N71hU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=lKIB3P1N; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=IHNo8oJR; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="lKIB3P1N"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="IHNo8oJR" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAMIJlW025921; Mon, 10 Nov 2025 22:21:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b= lKIB3P1NgLu2xw3DsELqn6NY/aTTRtqqm2OzAO0gH7H08cRyaX0uCeOsYS2JI8nN uNb2czrI5gbTmd4Fh248hX+qJZT03GRgr6An8BHAv54CPTpSHsC7WzKMuG0rWmgO uxtuJoL5JQd5WGL7FM4Lm1bz9iB1HY5GY9+o7TD9Ufy6tdw95LPWPXX5hUcmZrgu gWdv+JSP2N6NxcZzomD1gBfHTtfZEL37liQ0Dg6M6LHT/lSAE3qJyMPmkFdoHlb6 uah+8WyB8EgmWjHkJa8XuKFtjxAqxjuywpdBHtWXoAsNU+CbKOxYVNpw+M/I0P3K tx1t9WlBaJW7sRTWSFhmuA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abr5vr2b5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:58 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AALo2IC040096; Mon, 10 Nov 2025 22:21:58 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010061.outbound.protection.outlook.com [52.101.201.61]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8rypg-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:21:57 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=NcvmxQz+GOpi/pnsG0ZKUrck1ATEMfp8LqpOW92I4mFZS3laQzVC3IHjjLFZHgZCW2tz3qbXvQP+sm3vjJH4o18gL2aBK38pPHb5CQTuGJX3nijQq95BdTDuwEss+tBZQ9TE4X9WeoKJiNxtjedYXwB7OQ36g73bSvuCTW9CxtTzKRj03Z6SXs2vRJKwJtbRA7YKIQZsTGA8oKgAAm1VXUdCgVN87nmWf19JmKpQVbUVNi3fcknKipnTNHgEB0f15hR0P+WOAbqT2uLvzNWRVZkJubIL71x/PLQMi+aLHB6L6TjFFZUsBwaIr0kglfWxtz5r/JoBDemY+vQudhq3Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b=tFmKtEOrr0i41E/h3ud3S/fAgIY5HNQhiak20WmFgDf9zO4BIJReHgAuUwLNRwKvGOoJiA/jlyPBHQdxGFhJjoXIGIp6+U+S6El7F87xpMADG/NOGEb/Litqlns5cbSWR9NkIrwHyFmcVb3337WpsAT282KvgdvBxpEG552MvHaseUNeBrfKiDnQzdBs3Rfqhk/VS0hhSek61c3ZwdoCE4F3bzRUl9yfXBb5W7O1l6O2bPBunTTA9O6OctS5RipzJwi+646qoU9OAc8+V8NU86lVpFwo4TeHyHyo3iHw6r2G3rtQLrqYIwpbZAYPY6qrPADihelJ7PBG/1l045KPrw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vzCGIt3sJ5ONOndZRuqYi7OVSaXyL8N0gC2xsfawbow=; b=IHNo8oJRSHFSgBTWEkk2tUqcDmp/xIc9GYfpWeKljdhhq+bfkl2HXucijS3Ftp3FJHuxcjX6aoHxobBl9UFfZp1m9uMU6Tw22X1SSUsM8yCxIkH1H+8OEQDgproTBXmtMdPcgcaT0LsgAHrw+LAk8seWDBXpCmyLXArW7rWV0sU= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:54 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:54 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 08/16] mm/huge_memory: refactor copy_huge_pmd() non-present logic Date: Mon, 10 Nov 2025 22:21:26 +0000 Message-ID: <6eaadc23ed512d370ede65561e34e96241c54b9d.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0230.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:b::26) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 3555e853-ddb6-429c-e67c-08de20a78de5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?zjXli5ingarQUx/RCxDJ3Icq+PTM9nO6jH/I6Q8j+jYuKiP3YGfq9chr46LU?= =?us-ascii?Q?A7O4EHXgqa2TD8FDWxdwoQsLvGrS8zbyK2z1JDoXkfsexbBnF04srZG2Pc9+?= =?us-ascii?Q?7LBTQ4K5e0fvZPL/dlzmmqlhbwicTOrtM2dVwDSY+riCUk1/6WBQXTmS3GVH?= =?us-ascii?Q?3Dym2An4bbtjM8O/QablGqDpqKVqUTzSsBJYMKxUxVyrCj8moVkZiAH/U6FL?= =?us-ascii?Q?y6x3fljTVciJfvXIp03r/a3NRh8hXDWmzHQFKDq7L+xzxSMRkFT9B6sgl5T8?= =?us-ascii?Q?ayuM8SbcWy3/a78PVp0r0i9ugqNkY4a7lJzh/4/345ccxX8Dbe41STGz8jKH?= =?us-ascii?Q?4JXG/S+Hkw3M7jL7HzHXa1gkNWtbiivMuTiYEoyfd1fF16zCzZQBwrVQadB1?= =?us-ascii?Q?fsN702IzuRedT0GkInhBZIFnfnW9zAncKLwg955Z9PbMcDl04Q0UGT+ApocU?= =?us-ascii?Q?jwibyGKBWveBdReCbSrwDGEUwmFrV0xgk+DiNUEQ/XBbvZocLE0HJYXYyuDw?= =?us-ascii?Q?eSLoIiEXUir9wxe/NrEHyDmV1X4cbOmUMsLzxx0cc8nZxOj2ctMoHFMXrVAL?= =?us-ascii?Q?Y7aRReXyVGmvC5txOfUxFoohm1WCFhMhHiNCtniSpHUAcFfiNHRHXjoPR4Og?= =?us-ascii?Q?Hq4Gq44KW3AGg3l59bgGqYRcTwG4aQ8vUkYQRe4OlY1PQLYjhl4euXKk4RVN?= =?us-ascii?Q?e+qge3zvGp6uJbAnB0Xvk8l/zB3vskdH3veStYvwjIsejtUo26FS+c2ogI2V?= =?us-ascii?Q?fepewA/CdF3rxAeQMS0vKq7xuEJSk+qdKJZrhcFrv6ljPe9jBPcr2ltVQRSl?= =?us-ascii?Q?wz10NYDXaIP9mxZmccR3KF7ofj1HD5Q6cDJ0PcjrOpvehpyxEb1mWbTn3OOX?= =?us-ascii?Q?U2VhxbqdLubAz64fKPk/vttmMzl97uRNF7NLM8Z19fGWCMPqkkdHHFksNnFl?= =?us-ascii?Q?4p4QNsI5DPO2w+UCrb1jEM6fs3zFRgt9F+FDuHCInbZ1iD2977TYqvlQ2tAZ?= =?us-ascii?Q?udA+FKP1d4dfVTOX+HLDRzIjdF7Vu6S6j2Zyyu4cdc7pKp8HB1QIXIkJRgmu?= =?us-ascii?Q?OPpdUz8Wmojr3QGW5EjqS6PtGBN/eLNApTdqmBWTjlns/rW8QJcnp/FNSujo?= =?us-ascii?Q?ooKAmIGxOlmpOByIvvPf2KRQO64iCkFgQZ3sVYh1VwNvyOd2CSqARyQG+IrJ?= =?us-ascii?Q?yN5L2/0w/e3g3WjUs4nSqzGqtvLBreYgazX802ZYbFAOZH0YuAnSPzTFj8gc?= =?us-ascii?Q?+YeFb6EYPKmsKXTy2Cqzc+jZDa8ovG8NZSwW7eni6viXdwwvzdDXn1g1p6UA?= =?us-ascii?Q?ffhrmt+FESsehoT8NPyFhagD6W4ettgg9NnNZrp+TKZnojqm7ytP3qT8dbu4?= =?us-ascii?Q?FmAekgfkXnfkZyH78DTxx5yM7sPm+hkw1v/cvyvseXFzjQjZOcEECiiiXlfC?= =?us-ascii?Q?OVTNp9W2pkAaiMI/KRJ6t2kWXyUtKam9?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?YNbBrA+rVGacWyh81xEsgkoduswr/j7D9WJLD0Ktvd0xWmuBZ6erim3gNWNg?= =?us-ascii?Q?MDb6t0LKC5KrZjJCDG2ZBasXGpdZuYwlMhDzI9sZOXk2wLVirez/O7B55E/b?= =?us-ascii?Q?2BoRc5i4OCYsydOn4Dqc62iy9IDzTZOXQuQUgzy7YtGjZc2zB8aG9Jpc7WBw?= =?us-ascii?Q?MnTR1nIKUtVx7M0LBUtGIEha2kSMePddeii6QftPFyxnoB8z2tuFRRS87GHy?= =?us-ascii?Q?SJTkjvob2NZChg8kcc+9RoPZlVTWxDEXdPtnoMpJPl3K+r09bgEI0hThMOYG?= =?us-ascii?Q?+R1duUMfIEYNNu8yKXJ21+MMZ9Y3o/r73yTNvFMVgW6ORvWdA6k3YpUnWxM0?= =?us-ascii?Q?PVOxlSbH+KT/Qy1TJzSo0PivjioMIwYVeUAIVWw3BU82fV3btlTJBSZztlGm?= =?us-ascii?Q?ZqBS7hNDXLU4aGAEeEbgb3xffdw0lYnLani2i5U0acpnvTD7Y6SRGkjVcfgH?= =?us-ascii?Q?1OQIyNTJKl3Pt9HhCwIlv4uG9zrsA2vdrm+ftaaAvSyo7dhxK4LzSnUQHoSy?= =?us-ascii?Q?Fo5hrZnCqttBb7voAma3Ee6gwBlNa1ktFDbuKz/JpnlPStqp0sSMWF1Jf/uJ?= =?us-ascii?Q?Rtn9qqIoO9tz4P3jfaSu86FLQFNQ019+wn2Ch9lW23xO8WZ5GExcbF1E9xi0?= =?us-ascii?Q?UClTcflkbVzj1rpnTGY/hhn8XqVxrJDHmSVeQpmKbH3r+EN1GKVdsVuf7JUB?= =?us-ascii?Q?IYbS6RGP2C+rlWsC3umOq1HHAsNM+ykyIe8Ic88EeziIfcqzJj9z6MMlBPB/?= =?us-ascii?Q?sTdtzr4pKH3Pz2cMuJEXiwagDzbl08YmP7lmscbUg8UqZG+m3sVOzTXDWwQp?= =?us-ascii?Q?yZJv4wHS1iVWPJbD0fGx3D5k+B3b0zxOYDJB/Rcy1Dd078iL2dWEjHkBusNg?= =?us-ascii?Q?HQgpZT6xoEXJz8yJqK2N/z3LKQekx+2VSMDrMHnLrNf9g8wSsZXaGhLkUorT?= =?us-ascii?Q?G6oyFszka8NdRXoBcFVX6A/qIQTZffyKPXAALV96fPipupBPHEZXNkxfXqBz?= =?us-ascii?Q?i0aZMPrM7yvvDJHygwQ5wDHpd+9GaNz1pEVf66jsl+oHo7LNgXyzR6Rdc2J2?= =?us-ascii?Q?6F8YYuFuJfzTfG4iZWG9KZiULhPOJFbEHLtVvDfpXstliVxIQZ+Kp3ZUyKHM?= =?us-ascii?Q?SCAQNZydZZYPKO8xgp12WveAV0qd/ICekPZMnJSFNv7TnNEJ6It4zbH0s3YE?= =?us-ascii?Q?CsqaplGjkZLmhE0SGUAoBIhADQLUN15d5MjXkD0HohrsPeuSoMVNI4iCVgC+?= =?us-ascii?Q?RNn9kEaQSoSim0uVzT2FaOv5tng2uHmYip3zHDgffOnQkPf+gT3iDt/cYiq0?= =?us-ascii?Q?poN1qkz1ELy7JChKC/99pX1OaowquirmQyCv+c87I5RkionlQkrCoHPUaRhn?= =?us-ascii?Q?9eHBEjzu5qgHvd5Mb+V+Js2BOtQF9ocpeTo4gYuJuw46htfFoE+9LmRXzIBE?= =?us-ascii?Q?YSMg4MspI8jX8MDtEWP1iyAOp1ao1t3FXg7rT+WqttZ0rYHYFg2K7uzClple?= =?us-ascii?Q?ERPI08FF0aT1u8HsZZSk8SlUUVS3NtpUvg/aECNNdrnotXKV4IWelNSY2nh9?= =?us-ascii?Q?lDDlhd4vXEpPpE2jsSCaSF3C6o9Ca3NKzbk4n7nPDy51xznm+QaA5ro9Rm5G?= =?us-ascii?Q?VA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: jz66qQejytzq11cNl8Ks1WjNIuW3Bg4FNP7bzMinpaOBz3fXvFQaWhrRgA6bcuVyekc64bTrzqc3aTchiL2YCk/m513yFz/GucbsBB2QbDUvwItKhLCDOzgdsTy3LwNxecFtBr3TSWN0gnhel9Ni6/cjX2bNRG+UJn0gd+q9TOu0xcmFCpUcVNaSpGd9BpEEcoIcKfkoU4t/t//SgeQYuhWWpceZ4Bsn/tsFPw5LKqOFcpGfh6Cojd17wK8g3JrWUe2lMyBiAZ44PstLKOz8VRFnVbZe0idp63w6yXBXERUb0tXigih73VfQKcJCxgcWkjvdtLlRi1yvDWqSooPTHyNlDUV5s5xoYJe3/1/HEgehxeztmJ7UTAjE1Wx+z8pdF05rotVejJtvtegq548WwzuIbcmDCNqVMntEEHFE/GoUq6yeWcioOb0NLyEleCYMArSCNoPZF1C+NKwLBazDX9ZQZRNpDQWaZlFrtnmBKYSVA/A17YzblyVjEUXeuCNAGJzckeNBkwE1VZPPo7wGD++ySBE/FgLMbr0B7x/JFgJBpCno0k0IwCJdVivz6b+p5owVFMlPl4U3WzHWwPknCf+gWFdhanVfUbjM75FdUOE= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3555e853-ddb6-429c-e67c-08de20a78de5 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:54.6125 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: EUSPxbgSsoMdbPzxtsWpJfHWiEKPPcOqg6RMn8WGWmX3Kte0Kr6SYUOiqySeRo6/OIJpmxpHQpHa6xw4SLBiLB2rri+Md0Vj/9XDGikTonA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: hlUy9_HQyJtD8Fw90kSVH5r59-9SCpJH X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4NCBTYWx0ZWRfXxojIbNOOp7yW 3NfYLwQL+CY8QXGKeplVwn4TR/4hzYzilUeKU7oaihwkyBbmPFmmH8wntRL0pTHC99Sq9HnhQUP ipPIBn7RXQQ8K5b3PpBqnvae8N92tzA0Jr/zuLyvxlt7BU998AUdVrQnGJVRt4qkrfY3/Q+Fm7y jbMJ7H+sgJoRahHMVqEZBTCo5I40RBPJGptj7ZcTRzoLGeF7KHNx4I81FSDtkLf/q9CeIJecBLL z127iv1V5QyF6Vtwtdz4ZhHJwU2FwQgcdjiXdcXktswIGXVzz5fnkq8s0qJXagaQFLBGXNtwQgz b1KBviYm2ILp+i4vLwpegHkdvj/Pf4Qi/tIcOuw7AzT21KhpXrI6A+DLfU+45wfL42L+ZddWV5q +k4Ll2k83BEuTncBSGpRHZZ60NFpUw== X-Proofpoint-ORIG-GUID: hlUy9_HQyJtD8Fw90kSVH5r59-9SCpJH X-Authority-Analysis: v=2.4 cv=YN+SCBGx c=1 sm=1 tr=0 ts=69126586 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=9UmCEYkwnbpDqGD1g-YA:9 Content-Type: text/plain; charset="utf-8" Right now we are inconsistent in our use of thp_migration_supported(): static inline bool thp_migration_supported(void) { return IS_ENABLED(CONFIG_ARCH_ENABLE_THP_MIGRATION); } And simply having arbitrary and ugly #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION blocks in code. This is exhibited in copy_huge_pmd(), which inserts a large #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION block and an if-branch which is difficult to follow It's difficult to follow the logic of such a large function and the non-present PMD logic is clearly separate as it sits in a giant if-branch. Therefore this patch both separates out the logic and utilises thp_migration_supported(). No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/huge_memory.c | 109 +++++++++++++++++++++++++---------------------- 1 file changed, 59 insertions(+), 50 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2e5196a68f14..31116d69e289 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1774,6 +1774,62 @@ void touch_pmd(struct vm_area_struct *vma, unsigned = long addr, update_mmu_cache_pmd(vma, addr, pmd); } =20 +static void copy_huge_non_present_pmd( + struct mm_struct *dst_mm, struct mm_struct *src_mm, + pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, + struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, + pmd_t pmd, pgtable_t pgtable) +{ + swp_entry_t entry =3D pmd_to_swp_entry(pmd); + struct folio *src_folio; + + VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); + + if (is_writable_migration_entry(entry) || + is_readable_exclusive_migration_entry(entry)) { + entry =3D make_readable_migration_entry(swp_offset(entry)); + pmd =3D swp_entry_to_pmd(entry); + if (pmd_swp_soft_dirty(*src_pmd)) + pmd =3D pmd_swp_mksoft_dirty(pmd); + if (pmd_swp_uffd_wp(*src_pmd)) + pmd =3D pmd_swp_mkuffd_wp(pmd); + set_pmd_at(src_mm, addr, src_pmd, pmd); + } else if (is_device_private_entry(entry)) { + /* + * For device private entries, since there are no + * read exclusive entries, writable =3D !readable + */ + if (is_writable_device_private_entry(entry)) { + entry =3D make_readable_device_private_entry(swp_offset(entry)); + pmd =3D swp_entry_to_pmd(entry); + + if (pmd_swp_soft_dirty(*src_pmd)) + pmd =3D pmd_swp_mksoft_dirty(pmd); + if (pmd_swp_uffd_wp(*src_pmd)) + pmd =3D pmd_swp_mkuffd_wp(pmd); + set_pmd_at(src_mm, addr, src_pmd, pmd); + } + + src_folio =3D pfn_swap_entry_folio(entry); + VM_WARN_ON(!folio_test_large(src_folio)); + + folio_get(src_folio); + /* + * folio_try_dup_anon_rmap_pmd does not fail for + * device private entries. + */ + folio_try_dup_anon_rmap_pmd(src_folio, &src_folio->page, + dst_vma, src_vma); + } + + add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); + mm_inc_nr_ptes(dst_mm); + pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); + if (!userfaultfd_wp(dst_vma)) + pmd =3D pmd_swp_clear_uffd_wp(pmd); + set_pmd_at(dst_mm, addr, dst_pmd, pmd); +} + int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma) @@ -1819,59 +1875,12 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct = mm_struct *src_mm, ret =3D -EAGAIN; pmd =3D *src_pmd; =20 -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - if (unlikely(is_swap_pmd(pmd))) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); - - VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); - - if (is_writable_migration_entry(entry) || - is_readable_exclusive_migration_entry(entry)) { - entry =3D make_readable_migration_entry(swp_offset(entry)); - pmd =3D swp_entry_to_pmd(entry); - if (pmd_swp_soft_dirty(*src_pmd)) - pmd =3D pmd_swp_mksoft_dirty(pmd); - if (pmd_swp_uffd_wp(*src_pmd)) - pmd =3D pmd_swp_mkuffd_wp(pmd); - set_pmd_at(src_mm, addr, src_pmd, pmd); - } else if (is_device_private_entry(entry)) { - /* - * For device private entries, since there are no - * read exclusive entries, writable =3D !readable - */ - if (is_writable_device_private_entry(entry)) { - entry =3D make_readable_device_private_entry(swp_offset(entry)); - pmd =3D swp_entry_to_pmd(entry); - - if (pmd_swp_soft_dirty(*src_pmd)) - pmd =3D pmd_swp_mksoft_dirty(pmd); - if (pmd_swp_uffd_wp(*src_pmd)) - pmd =3D pmd_swp_mkuffd_wp(pmd); - set_pmd_at(src_mm, addr, src_pmd, pmd); - } - - src_folio =3D pfn_swap_entry_folio(entry); - VM_WARN_ON(!folio_test_large(src_folio)); - - folio_get(src_folio); - /* - * folio_try_dup_anon_rmap_pmd does not fail for - * device private entries. - */ - folio_try_dup_anon_rmap_pmd(src_folio, &src_folio->page, - dst_vma, src_vma); - } - - add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); - mm_inc_nr_ptes(dst_mm); - pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); - if (!userfaultfd_wp(dst_vma)) - pmd =3D pmd_swp_clear_uffd_wp(pmd); - set_pmd_at(dst_mm, addr, dst_pmd, pmd); + if (unlikely(thp_migration_supported() && is_swap_pmd(pmd))) { + copy_huge_non_present_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, + dst_vma, src_vma, pmd, pgtable); ret =3D 0; goto out_unlock; } -#endif =20 if (unlikely(!pmd_trans_huge(pmd))) { pte_free(dst_mm, pgtable); --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84C183328F3; Mon, 10 Nov 2025 22:24:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813446; cv=fail; b=eQo2abhU1oKcpSTyk2wp/emCTWMHZbORv0/SVYwzPGqnWieD9fWPRJHNYqeEBiywp+xrOweHIv7c3+BOG6eWzXaU0QjME9TSUhp8Jm6u6PKBMR5ZW9vgRqsOu9tnY3yHeWpv+CvPL4jnPZg90b7RXjp0IKsExZqTKeHGF8um4wU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813446; c=relaxed/simple; bh=wEswhEPB/UrhDyPK9C9sI1IHFlT5+kgMCKmd18ehtZM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=KWDsaoefe7GKHuTDxyeIt68KR70E/xP8NU8F0f2RbdxO7qnLxvVN6Yf+d7zmq85oOG8CMe73SFtLIiU+tNCr0vUAFyud3UUBdD2t8ydGY1AkCgMSSSmdHexdDPHJGg0GwpV6BaSa/tkfLm3evf+O+Oia+SFyMHtmyWYhDqytQeA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=dqwmpGMK; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=ypIXZ00B; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="dqwmpGMK"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="ypIXZ00B" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAMIQ7h026148; Mon, 10 Nov 2025 22:22:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b= dqwmpGMKqq7TC0Gp5PVypaXdQk79olLrCSoCMCdr/Athit7oT7E37UDbeCfm/CCU XNoWiaDJ4W/WgQkdfxcIj1rzNkSLSMpGzn9V3eo5EjUrZ6v4rB1ns4Wfip1LtxUU PTQwLVliBsNsZMZmlQNPxowxPFPg782bL8+ExALI+VTLJufwva5NPI7q0eM4TI1X wj+IMY25Av/jLby9RxZJu77wyV4DNluJ8boBWp/fVawQhDC87y+mnqYTZ+5pOtYz xmt7ArI+/+W6sFOLAoAN6+AYHo2+IXPF9BHQNwtLTEA33CGMLq/IeuzYHD1z0ANV 41TMYgBoun7v8TQKtGk9BQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abr5vr2be-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:02 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAKGUhq007458; Mon, 10 Nov 2025 22:22:01 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010062.outbound.protection.outlook.com [52.101.201.62]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va997sw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=UIWGDC65w4y5JL4jaW+u5n2XlnQ5pbKK85/mSObPgLZumSXSbhrUYxDwAgLOmwk2v81AuvtwFBroum3lpKKTf+EdOQuKBZcZDWZp37DDjDRthjku8Hp/5Lkb91llLu861Nht3/fAVrdxO/1mtzM85EDO+0da50QDPwL7ngy7h+xm7J7MErMVg6zVYdVCyi0Q+ErT9tIpYN/K7DK4gf3awyQLKGSzEnVntGgwe+/qQTQyx1xJ3mjqwU2y+wG6JZNIzQAdLX+HF/RuGmwHJ00SO/DOEs0awE3e9e+4mEt0/zhGiEAUCaXwFOTytbTZktm73sHkV+PvHAa40ig3VvaxNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b=S8JdnjJPEPv3yq2LB+dnUMZrAxsESpUAkie6ABTPx1pcvhnQHPZP9uAs8g2wliXl/JoZ34G5MvgiYpDX4S65hp3AXF7Y8BT4d7ovB4giQ63tJvQ4MtWGuDeACsj/ZSEcNBRzTb+FDrr8kJnARQWzcvlfEMSK7gCJzY5LZZdTmj9B/AjqmHnRykSYMZ8rsHlyAxh1CIkP+lrRhTOaiL3Ro5gCoEr3uKhd8mnWNz3sDkMGmMw/Gei3D2ZJRiouykD+xv61iK9vNPpxBWn7AQqg9UypatiEBtfM0khFnRc6C4t7mXO3XL7WKCsEyE8az/bpMyDjYNnJ2EQLaD64ylVrgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ES+ODnUpx6m2BxJg2i8m0/ywBdmg7etQq70a9R7KC9k=; b=ypIXZ00BgNSEAk0T+Dg+1XXPRejAue+B4ZoINvAA9eGRSojFiV0OA2Dxit8mpa5f3dtV79jp2t21/WlKR1qCoVGDw2u4vhlkv6rkdUUNHINquo8mnadVPSLz2M/OYKxtd2TpUPc4i2yH1u84jpvRNkT5MtfeTbZQsWpltfedMZA= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:56 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:56 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 09/16] mm/huge_memory: refactor change_huge_pmd() non-present logic Date: Mon, 10 Nov 2025 22:21:27 +0000 Message-ID: <451b85636ad711e307fdfbff19af699fdab4d05f.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0237.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:315::20) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 29b15133-3027-46e1-a6cb-08de20a78f21 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?oXPkxBv3HFziTBzFTfWnJbiT71uYyaAXegJkcKLCiSCCejcQTOiCLiABSP2B?= =?us-ascii?Q?DHJJ3yzQdouwCLjupaMoeCPlSIIkAf98az463VjpDLy9RhrIxoK1Ky3ozzb2?= =?us-ascii?Q?HQhGDknLvOqLEAFJS8PiBWImZJjw/OiaEaXQDqJPzqTDOu6uqC3rHNYeIev6?= =?us-ascii?Q?qFmPasUDNKEk26JDz8mPjjlTJdnoPfGCZARXxTV0TkadYxx4GAYXg0pk98HX?= =?us-ascii?Q?O6DHUTUmflTtvpRBnKpZ/knP+C6orY7WJjhiAEqsDo8uj2xG3VqjNY/GwbkT?= =?us-ascii?Q?F1vkjad5XJk1EC96cXMvF8N0G6g2oErAJPSYmFmYQy/20FWbX/6qncNakDF3?= =?us-ascii?Q?4V9Kv3PXkMKtKC303EK5WFcatdeTLQPhkku+8VIz9HtSO54gZX4Oz1UO5adc?= =?us-ascii?Q?DCixi860mGwYS52OGzyjh+7opmyKExTQTEaFkShIuPQNZG82xyuLkFa127/C?= =?us-ascii?Q?JKnB0ACKJV+EqXTLYrUOfxAQdXFMOYDlx/uaBqaTS7EltQgzwiWEG6NaNd4d?= =?us-ascii?Q?lhGCMuiWFwEbCpLAlUoklNIEhhTZjTZoCCS4DtpcbbZBhFRgrVEePAZw/rPy?= =?us-ascii?Q?+iq1QlPhwSO0wQxjFfNAKUb6UNhIzlXXKqrfl7vCy2aiCtOBJE6pY7r/hRZO?= =?us-ascii?Q?FmLqH6rcp33Y9pGV9q3ZZITKYqEEygfRk58pzYa40S013yRSHIBeOljBRfXT?= =?us-ascii?Q?Vm/DlUMpfAtldzORLaHPrfMqqLKUGD6UG07LdoLGpFw/PFZEwKjrAcA7hu4Z?= =?us-ascii?Q?U6AAaWNAxpya28O4eCssj06AXtCfQhsrUrXaNiBWPinTUsKU9q14xSMPvdnr?= =?us-ascii?Q?/PF+01HIzD4yOVTHKiIN2Jgia2JGf7EhlVlmSLqHPxR1fZgU2ebhOS4CMYkg?= =?us-ascii?Q?ZwW5qzIUDdwBEBM031zRMmxA7qLt9I6jjcADfMVahG98GVMGFgIZiDDgbSLP?= =?us-ascii?Q?3uUVugeH25W/jQMqLC38eudopK0E3N1eamAQGMW/UrZeQPxol8ol4QZbqX42?= =?us-ascii?Q?yAPyS5wHFpVWKdhe3OWYNXUePoWWSE9BVuKfvzQMVR0Oh6qSPOFkmpmZIQFF?= =?us-ascii?Q?SjuwTyuCBi/H7mO7xkG+5/4Ivxbf74v10dgUoscrhOrwCoRwCt1zc8AVK4x7?= =?us-ascii?Q?exr1paNr2+ju8flpuzVgnFuxWDamkiLUzvDeM6ZPllyTwanfJkz670EFQAW9?= =?us-ascii?Q?njLGPKgFMbwR1eigBZ8rAGwAohRkmSEJpQnEwc1H4WjAlFqQFmE+Lf0Q09xA?= =?us-ascii?Q?/dEMW8xyZ6Cxtur9oVmw0zE9aU2kQjFRaiq/HkGmGRAWo4jRNaIjfWZQjh0T?= =?us-ascii?Q?sxF6ixgldcA2uzBRY9+UswL4SQPY0EFsOGBgWjd/KohG+3Sl7/M9ztdlHqhB?= =?us-ascii?Q?T42ITggzfQ3og+c8iyopP2PKfrCQ7LNTyR7SeHoHPe3n6safVgrc2BkFzVJW?= =?us-ascii?Q?Fpe3B/0mqB8s8REvVYurLxxkrzaA9/k+?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?aajk5WwGh4Zkx1IsmIz2Nvzgqkgvv8sDUoa6JV8PWy/nKtmnCYet9BxOLFHT?= =?us-ascii?Q?gDOUzEQ2OSyahZOKGQiUUL80AXS7aTfvvUAQ7CE5fPEeoNLGFdGbHSF5VSkW?= =?us-ascii?Q?hK7yj1AkqEni9TGIgknX/PqWGZZlM+41qHBigleVOtVBPAwcHSuFzq1QP/wi?= =?us-ascii?Q?4iUSpEDt3rHOCWZk8eKXsC4rQGbVgtRM0itHw+HIudQX9nzic8l0Mb3BcwDq?= =?us-ascii?Q?mcuARdzQTObOqoPl4XSpi3RjzDWCDONDk6vuuaUVhae+MY+qvJSHV0rVL93M?= =?us-ascii?Q?FmNW/EwaRxiSe5GWc2Tat2u0mhxgXVk+p/oBCRLWt1iCG/4vbXtsbr7bYUBi?= =?us-ascii?Q?+dJ2nN6Tp9jSt7eCD8KgVsb2Pmw5np7Vk4GpJefoY1PMDoFWVG61B9lEaRBN?= =?us-ascii?Q?l/bMuVX86KVraqNUTEOpyMpXzVzVFH08024Ulwzut0OArVstwmQKfCKacMxL?= =?us-ascii?Q?ekMvMtiaG/Qzl5L/mC5SsCaZNh8Np9c0jY1CiUKj0cP1wte9leGNHW87Xx9v?= =?us-ascii?Q?KfgaBxndSgerXcmN7+E+k7vTMn/HIRKctNh82hYDf6JwFWmykZsFRPQ2XMqG?= =?us-ascii?Q?EB8FroGbsMybWeaMf8CUIFknnG35/2p8FVXccnm+zYgiSCLlY+FZiS7k9git?= =?us-ascii?Q?zTTAEWlZKVwRdibsAxDarLpNVrAqm9WHM7XOifiMKmu6vdw9FE/Oy+PbnVyT?= =?us-ascii?Q?YSJA28cs2U8YKKAaGiY5vlF3dA8hKgvk5bKaDNcOQ1ObbVoYS11ch/SYRni2?= =?us-ascii?Q?ordyTRlEntYkoRrf5hCEyNaHLG/qr3gzRSRvvkYXHfSXfzlVWbQz3xvCHLaT?= =?us-ascii?Q?1bEE5QXtlAYJsT+W+oeDArNgaUoC3XIPE0S7E4IxuWgWnAaAiWFiVTADJGd5?= =?us-ascii?Q?hlI277twf2SlIRoGORx+wNvc2RjnxPJXFzKcl9ZZDekQ+omAxapholuq5Fn1?= =?us-ascii?Q?6jST1Z2wNb78HJywh0gp8PGc9oXkuCZeEJdGO06PoAb3Af4qUCyK/+Uws9NY?= =?us-ascii?Q?KTzhWThycjyjGd1SA8WUmWku+cvEGZ0fDNEcv7498J7fIhpHM29N55tZfIZL?= =?us-ascii?Q?ztPkhPnw35BK6s9g9ILUU01l9ljzCm6gn/Yb+NcuNo2CD1QqKWtE+qtAIvmS?= =?us-ascii?Q?0HC5Zf/2OmBIV95xkq9+jHssLWAzs/Pwc1Jv9vrLIa55SqepzKekC4UNrdLG?= =?us-ascii?Q?dah9W+bsOuzXMHNk+5aXYb9SygCX6hFXUhLhFjThH6F5zGzhAjEm8POxrG91?= =?us-ascii?Q?SpxSegbKnA7rGOlas5bu38EO2C6p2JXFdtr1Zjfh9uDcQ5kg5b2miLqUnjP5?= =?us-ascii?Q?4Jm26tF/j1mkp4IMhoJu6QOvdLxAl/2tkCHsaqgh+iErXZZ8i4L2b2YpnBcs?= =?us-ascii?Q?O5GCMxWUfBsAtrct6w1TU+W/p1n0OU6r9D26eBDD6I1mIPWTGbnRb+pe1kM4?= =?us-ascii?Q?sU+StkNrhbFMmKd/9Xyzv93kWqyFbWgBOINNh/LAsRDN5+pHuvL90UOK3Hfg?= =?us-ascii?Q?4WZxfKVqaRBX139CqeruBGUOsUs1K50pgeQH2DviATO5hD7q9hFctMXI7NTP?= =?us-ascii?Q?IuM0hesOb2j8p8izdFNj3GuD58X82PnID4tVz1sXnvdrByJ/hIfwRdGP8MUg?= =?us-ascii?Q?vA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: BaQfCbPSqaTsAEj/f3f4JKLaaYGWLZj9nQ9C6giBR389nxwmLSDPl2f32NoH03a2RqTXPG+pazlKKEfm4c36olguUivMM9qimtlajLl8GNUQylAB+SOleK2YwR6f+L5HDH6nm7Sd/CMhaXyUhzNxUK00YSX2wgpmmQVHa5YWAd/8hSfVGicwSIlZc6MTYUJ0t5WXx3T6s0vY5GG07C6uHN72yYhFPlbRvoBRvkQ1Z8uhzH5tyQ4EJiXgXvXtYRFr+7zDHm9jsvIQsD767FSE+F7yX+YI85p0cHFwAoB6Q5dLjDEeFMElt2Q2qGiEyv5I1AD1G+5ntmjaI0wzd2En3mesfZsvk03cxo4+W8g01Lb9QbJuS95NGPOwZoPEj6f8PdvQixkiq5AIif8SI5puPQytkpGhXps0FI4AOEKTDz7f7Mv1x85mbASt2f0uhiigWGxgHPBe6b4EHD+NDmGmdA2Ly0WR4Syl6n9ZNrRf+9XphCUDe64N/4npUWwTJumr6O468K55U6lUAeBYYirfjXeBjIxnEgGtOidYcMAK2sYHVOcaaSPtVmfnJjZuJ1Z9hTZSt+LNwsnCtEjXgJc6Fs7hZnK/iHjpQd7MxTcqiSc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 29b15133-3027-46e1-a6cb-08de20a78f21 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:56.6767 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +xkJo1Q7ZUxf8hzrWsCqQOzdb+BRnrVzBS1AnFrO8Y8rMqwl7++shby8vzAgcctsx+WF18DA5WsRaQXG0selbt/964TLCenkod8mAovvLpw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: fU10awHxOl59HnSL3uH8zrC9pdc2Jx88 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4NCBTYWx0ZWRfX3C5liiu5Iplg jdDIAfhdYhfVEBcpjnMRCsZXOuRycEz6UqfzSN49DUiMPBBCLGsxnDmijJlBsy1tnqfoxE/8rlt bWfcA3O6UJyi4qMxCbU18+YPAFBhbMINtYHxhdpF6YXPReb5BgOXh07pjZcIqynTmeTnW776o+3 sCYI4igbxMQtN6wwLJExLx+59xYUSeDnkwpqNvlkEz+JrxzHS6Ryi78jCfEb6tV+vAtJnZnf2B2 OWkfcDB7U3UiyeRhH+/wFhQHr5n0J1R2HEbB42b6uDUGtK+Pbebi08kB5qihnwN96Wc7aDfDvRw CcvFrJgYVyZRgCzq0QoNMhsPQs1BehzxhekjOavLUR57mIf4AHdNeZiD0WeeTvnoVMFnGG0/Z+C sn/DpEHXLETQKaDuinqUzCZ6GVQAOA== X-Proofpoint-ORIG-GUID: fU10awHxOl59HnSL3uH8zrC9pdc2Jx88 X-Authority-Analysis: v=2.4 cv=YN+SCBGx c=1 sm=1 tr=0 ts=6912658a cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=gAL6uhOEssAPNySJDCYA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 Content-Type: text/plain; charset="utf-8" Similar to copy_huge_pmd(), there is a large mass of open-coded logic for the CONFIG_ARCH_ENABLE_THP_MIGRATION non-present entry case that does not use thp_migration_supported() consistently. Resolve this by separating out this logic and introduce change_non_present_huge_pmd(). No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/huge_memory.c | 72 ++++++++++++++++++++++++++---------------------- 1 file changed, 39 insertions(+), 33 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 31116d69e289..40a8a2c1e080 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2499,6 +2499,42 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsig= ned long old_addr, return false; } =20 +static void change_non_present_huge_pmd(struct mm_struct *mm, + unsigned long addr, pmd_t *pmd, bool uffd_wp, + bool uffd_wp_resolve) +{ + swp_entry_t entry =3D pmd_to_swp_entry(*pmd); + struct folio *folio =3D pfn_swap_entry_folio(entry); + pmd_t newpmd; + + VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); + if (is_writable_migration_entry(entry)) { + /* + * A protection check is difficult so + * just be safe and disable write + */ + if (folio_test_anon(folio)) + entry =3D make_readable_exclusive_migration_entry(swp_offset(entry)); + else + entry =3D make_readable_migration_entry(swp_offset(entry)); + newpmd =3D swp_entry_to_pmd(entry); + if (pmd_swp_soft_dirty(*pmd)) + newpmd =3D pmd_swp_mksoft_dirty(newpmd); + } else if (is_writable_device_private_entry(entry)) { + entry =3D make_readable_device_private_entry(swp_offset(entry)); + newpmd =3D swp_entry_to_pmd(entry); + } else { + newpmd =3D *pmd; + } + + if (uffd_wp) + newpmd =3D pmd_swp_mkuffd_wp(newpmd); + else if (uffd_wp_resolve) + newpmd =3D pmd_swp_clear_uffd_wp(newpmd); + if (!pmd_same(*pmd, newpmd)) + set_pmd_at(mm, addr, pmd, newpmd); +} + /* * Returns * - 0 if PMD could not be locked @@ -2527,41 +2563,11 @@ int change_huge_pmd(struct mmu_gather *tlb, struct = vm_area_struct *vma, if (!ptl) return 0; =20 -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - if (is_swap_pmd(*pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); - struct folio *folio =3D pfn_swap_entry_folio(entry); - pmd_t newpmd; - - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); - if (is_writable_migration_entry(entry)) { - /* - * A protection check is difficult so - * just be safe and disable write - */ - if (folio_test_anon(folio)) - entry =3D make_readable_exclusive_migration_entry(swp_offset(entry)); - else - entry =3D make_readable_migration_entry(swp_offset(entry)); - newpmd =3D swp_entry_to_pmd(entry); - if (pmd_swp_soft_dirty(*pmd)) - newpmd =3D pmd_swp_mksoft_dirty(newpmd); - } else if (is_writable_device_private_entry(entry)) { - entry =3D make_readable_device_private_entry(swp_offset(entry)); - newpmd =3D swp_entry_to_pmd(entry); - } else { - newpmd =3D *pmd; - } - - if (uffd_wp) - newpmd =3D pmd_swp_mkuffd_wp(newpmd); - else if (uffd_wp_resolve) - newpmd =3D pmd_swp_clear_uffd_wp(newpmd); - if (!pmd_same(*pmd, newpmd)) - set_pmd_at(mm, addr, pmd, newpmd); + if (thp_migration_supported() && is_swap_pmd(*pmd)) { + change_non_present_huge_pmd(mm, addr, pmd, uffd_wp, + uffd_wp_resolve); goto unlock; } -#endif =20 if (prot_numa) { =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9BC2E343216; Mon, 10 Nov 2025 22:24:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813462; cv=fail; b=DWQJT/fRJvqhU6mapTKyrp4bYGRvZ0qPNi0iNpW4zIbCUkJdEmkXGcZLRya72PW6njr3BD0+2+SVo15fgaU5arjHmgKAb8kCSr9T1pXOtDUp33xLEQegeAkDvD3jM64JKQA/kXOysiuCjDNgwlaLfUqF648sH+uFpfjX6Sp9dRI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813462; c=relaxed/simple; bh=MeIGKmBE/AJArO8ad0y5oWnwuP1Z4vu62nuxzrDgL3g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=FBHFRGTWm+Dyniyhe+JRKAGDAdL8wTA+MHJvIJerLsXHhYWjsg5lO02EPIG/kyT51B6u9GLNRxGhqhbbt0DIaCyv+z/xiVLSSy6VGgcLKht6DVsq7V+/tkuU/Cdp+rZLbQSHLcJJOdeLJA+9tYpz4s7yyADljPgW9WqNzj+4BL8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=jqi6T2W3; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=qbgLc5LQ; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="jqi6T2W3"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="qbgLc5LQ" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAL8d3K009183; Mon, 10 Nov 2025 22:22:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=tqyjXIjTe/WH4TE0OwmTqG8WBXO7P04SnotToIeqRqA=; b= jqi6T2W3amwK5ypTmVVFRysSpIeihvuB8JFYV2y+rrSzIZhryR7HoldfYOAuCTNX yF3UAWWSEURb/5JvizLc2u6i8k4PUVP/uCW84wNAU5oelgoVSrWbDLQ+PhiiHjkh 5gKR912tFKjtnp4nOe29laXD8AhxzwnugXY69tfpus3P/U3jcnhYjre3FMV6PYnn Dl+KAo2TJGFh7iciuwvkA6/heJXq6URPH3R+lu9t9JkgsvCzegMP5h671IKMebVK A0KehLw+zDsLsKa3Ogf1hgTtA1lB2QdwSQDOEVjkXSydCFfOU16KQtbQ3FtvFdJK hyWZQ+ykfXDJqA7VU/tehg== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abqpug5sg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:04 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAKGUhs007458; Mon, 10 Nov 2025 22:22:03 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010062.outbound.protection.outlook.com [52.101.201.62]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va997sw-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=MBJzXQ5EDN0DY7nQDNcXKYMXpcZmh3ovONyG8kK4l5gZwbunlyTysND8cuSkzCfmO/Ac+9WkMVXu9I8US5109V8KD7j+mPCZLqpwqjDRYejGOKvwtVZLnB642y7jc/qGK7S7UMqfTOUcGBbwnReipS1W8UFXbTlUk4/KzWlO58N4ZLXp0hIgW1PHlVq6r0hUZKLkW1rD7mApx1FUwrBZ2q4gknZ90DOwX/IVZL0wFjcSX1LcSRGu0+JgXYQEfQuA82oxvp5bxyVPrmLMLKLOalGM4QVpFTnV0kmPCcF7yRuq3G00U2GeCyzNiLhACwtJ4e1P+9HExbuBRSQuwZIeYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tqyjXIjTe/WH4TE0OwmTqG8WBXO7P04SnotToIeqRqA=; b=wKPn8tZ5VvSHxK1FBG2Gg8Y7bgHnYUILbsy9TFC8FxY/wtV7LGDp4I31AlzF0ez1shINhEXI70P9rMnxTC9oKOgb7wQ6N5jUvGinggLi7FA8A1hBU3GTh01mYV7+3OuiqWc7rxbUHWua43eRkMIVsgqjQbR1NLDouWEUBzmhrDo4Sh6WSzPD8J5UrD/lnuoPL4oaFPoqMY3YskdzXMwtimZWkGONZFo8ftyGtFdkPdlz1yDGEpqZeaBrM4zV/23Q9IX3PgW0LDP+ez9mxzfq45y5TWTN1dXdW21H2HNhS3h1aciQi1cTfgQjte/3PC3pI9VOFvlZa+qsvNfliCy9Jg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tqyjXIjTe/WH4TE0OwmTqG8WBXO7P04SnotToIeqRqA=; b=qbgLc5LQcQHJCGlhhIjZEiu2aW7NbaPI7h9Muos/x9wspEieOXdFErfCGRfOhM60Y3p0yT/Ux5rZz0fnPV8hl2dkSaMsxdV3XBdbT6XOI9FEYOAp+xTG67HHzOymLGVBTHtad053XQSbnAffkSLxnVsYVbUdeBjp4gCyEhfSUt0= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:21:58 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:21:58 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 10/16] mm: replace pmd_to_swp_entry() with softleaf_from_pmd() Date: Mon, 10 Nov 2025 22:21:28 +0000 Message-ID: <3fb431699639ded8fdc63d2210aa77a38c8891f1.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0169.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:312::6) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 0c00907b-f3c1-4be6-bc73-08de20a79052 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?qrDG4MwVkrX75OFrUKUuTXPczyXn0qV2WX3euYCP17T9JBItnlXM+7jvKGI7?= =?us-ascii?Q?fRoUkeRk4QtZTxavaN4dOJ2weSvMTA51hIUtvZys8LdIVzSqbaBFN5ec4s45?= =?us-ascii?Q?AzCn5rGyy0Sqvjfvz/ccjyuNGlZ2ND/PlbFqG7D/17o9UjfJvqTCDCDaYdaD?= =?us-ascii?Q?USRyMVW0bWzt2YLpKsiTCbboyyVuBjQlDsnnca282+xPBJJ2RXhbJdvnsYEk?= =?us-ascii?Q?dZlX1Vwwr3P/s3KMrnwmz2dvHMHuoMaX4zJ1usse7T/dmf0r0hauOS25QI25?= =?us-ascii?Q?I8f9v830rZbUmBkgq8KycohQcBUotOPUAwWOEFFHtwNOkC+wVc2Jfs8TBGe2?= =?us-ascii?Q?tO5Q22vOdDM0f+6ltrXFLbw+fXasF5xSqEBT8kRtTq/PrOrufTYMUDmOhiwF?= =?us-ascii?Q?Nf7WzgVfv6X1E+ZCWlAc7o8/np7falm5X18x6b0EyaOrCUzUdi/7g/1xYb4E?= =?us-ascii?Q?kucyxEPt8hko9wrvBvIguYdeFc3MDunCYJhFiDlF7iiK6hQt5ksZGzg3h+bQ?= =?us-ascii?Q?gFhX5gGH7ZkNhAwrKHd6m6Ommn16J6UoVkRxriGeCB7GtQ6ABrc+x/+/8/vM?= =?us-ascii?Q?ZkFk5SWzUwfkQCHcyxA5b1OmfLEd4+CY1URNv1LKG6H66UU7z+VRaLHaGJbr?= =?us-ascii?Q?whor1tymFAixwUwclne34EHd39Eig9XiS85kZttGxwMeD3+8pDlqIGMiuSbI?= =?us-ascii?Q?fa/iKZwRQcQABjRWoNPEDe1MLR85yF6JvjQCPPkOl3Uezn/OF5Gn8ZrhFHs7?= =?us-ascii?Q?p6CUsbuHGcJ1iBrMK7Yxa5IyYG1U/IbjeGoL8tEkwQhvKnctP+UH0S5DHlw+?= =?us-ascii?Q?PZWzrlBT6CFkVY9/IgeFrF59q/kLt9sFIo4y1daq64iSs60faHFwBadHLNho?= =?us-ascii?Q?vfUeAS+rJ/E0XGbY5OGQ4y942xGdP5gi8u5BhojLhRm3bhi7wjcHchi15HLq?= =?us-ascii?Q?SVl8TuxUEY3/w4339T1b3srj7ppTqDk+0B0hRfj71sqT+lQYl5veWwWGK77w?= =?us-ascii?Q?U9OGpHpXArhRuvoACBng70HC5O2zYwKgMULbUDyzn3El4Y0rdSAFkgFt234U?= =?us-ascii?Q?wjBg7YORcQ6Zk2/rnRUwAfXOSDx6Wcj6EUfIfEt/85Fzflslb7uxZTftU2qq?= =?us-ascii?Q?G67zFuzHX9BSpOaA6p4x6poPWSQo+cXvd+Zdbe63Vrbl+x6nWr66J09t7PfH?= =?us-ascii?Q?sZ/nrK9ahMfybJzuDG5T614SusdnE070zwYPskthA1BWUEAH5QQ9nPCA03fr?= =?us-ascii?Q?nWkMqHFDa8Tj7Ew7XRxTlHB8vGkaSwygAu5yDYg6OJ8VQwFVX+oeXyX6W00Z?= =?us-ascii?Q?Oa10unoKNQm0RS9XpRoY/H0kjCbGDa+VmALvrkkVdw7RldTvgZHXGZ3E0Ido?= =?us-ascii?Q?bl41v8dW58/UXoodqcsZ3/zTRQYrjMSyLp1qO/tT2uy3QEId3KLQzRnjGSDT?= =?us-ascii?Q?UlvqjN4B0xoYzA9gNO8eR1v5QwbCMcC0?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rk3XYiMA4qRCQbIzanROfd7FmxtFLK8V0BZ/kMKDGLqKvCdqRjz1mwZp3WU/?= =?us-ascii?Q?U8zcGWSft0W7JNaOMmLjCy4UX7Ro+9qnuWpcnXhvR6YvMw2UpFkbjM9uaSG8?= =?us-ascii?Q?QbRtCGJUotnnxfqzTL9fYgCevKNMv8Njs4TQ2g/iB7yxxV4xGixkuBp0qjlD?= =?us-ascii?Q?RVKT/u/Q4x2YmKukjmcmcZrY5znn/7p1pEbKjAP2avysMSSNJshDJVx6QTqm?= =?us-ascii?Q?ZKPz5Q/XdPE8DPnRnZzYkwzhVwEGbZiAH8JiaqBh8ZDYV3cjuNN8mhmCHbej?= =?us-ascii?Q?40GDk2SMxhTtjJwhg4pclmYi4J4sCXxp9iLN7vR8Xetw+UXf22jvv67VBQad?= =?us-ascii?Q?+qnluBYhHcKyMx1SYK9ALMl2GLBf0SZexgBzmKrf8+f0hc252LQ3goPGN4/8?= =?us-ascii?Q?Gf3QCLlfsTWaydLBh67UXsCBvFxI1PIt6g/wTE98hcXOgBJGGqoQY+PJLjpD?= =?us-ascii?Q?EPwCT80x0PDbWdaTKmJ6oKYrHiLKezdBPejcBx9Kmb46smIRBiWVUa+qrAGw?= =?us-ascii?Q?rBM8eLP0N7HrrrJJMAoGUkhSRglcegqdfDcvasEQ21PV/1Xn/4Snp85lPHDL?= =?us-ascii?Q?CZAIAgnmbihNaBJ0RScBrUmks1Citb8/sNG1avlC/csxmYA+C1Uj2D26j0BB?= =?us-ascii?Q?Xz1hNZFFiErNJair2QkIrDnHdFeiT6OO3t8BfBTO7ol4Q+W8hspE2fw7DLaf?= =?us-ascii?Q?Ty+YWmzCjBz9kxWzr3uJrR1QLPQBPlKrg1unXd+yvZvlkAfmtoqzqJnN2l3F?= =?us-ascii?Q?U2lh4jLm85EEjwUGpl52yaxlArShJxoW2P7B3bVPxGfKQT3QqWlHNAgajAJI?= =?us-ascii?Q?Zcx6oUNBbQ3BIyKCtudtMsjjdJk2bR3YNbdXr0hxnMm2ZkkPeNbTMjwl3CNt?= =?us-ascii?Q?LPdArXjTgA2Fwh1ozEXBvVT3aNFigpjKadqtjrrDwvR8pAm5dyxlGcDXhq+e?= =?us-ascii?Q?k/TXHEn7qv+j3XFauLFEIsOjDz85gqxhmzHptt03BIBr5twFunTcr9djvT77?= =?us-ascii?Q?0cuhsEBoUzneEBkNvoAa/9TnJqSJQpi+BczvZ2i+EKazjsOI+fhSVXufrgMD?= =?us-ascii?Q?R/47QtCDJfQHEaQ6y5QS8A++38qGQqYEMcmYE8nM54jNnyGF0Sp+GBG3Q8ih?= =?us-ascii?Q?R1ieYn4In7rr9G71/+6Zj7Ak6A4FutsTNNFZg13u58VutH3r+BMFpSmNZ5O9?= =?us-ascii?Q?yb6rYtr+VunM0xOfHAcF7BrhaY8abEJixgo9SMTrkBulWThX+pDWAcRHLq0X?= =?us-ascii?Q?mxHR+gzTTI5CYh/sZY3dL8b/zd9PIecckc0ms1qPbJIXtx1fh0EUVPq1ce5y?= =?us-ascii?Q?OBAjn6+B2ji2koOaYbJPyf3UTShrYx9+qB5qK/Y7qLDQDyfWWOeEB4zR2mxn?= =?us-ascii?Q?b38xXoooiw3dYfyo4nVzn5B0IVUupEygofORlWHeKJ1EjkzbWZOxjrkDbiqx?= =?us-ascii?Q?8voOoihiow0WgV4ha8ZrT/7az0NbNdKXlbNTsBXV8pblSg/WbMv2yVUD+QPg?= =?us-ascii?Q?uZEC3LKYhZFmlPLooSNe9YvG7ySXhoEMdLCDw1mTYaanEnuF4nKNs+4xMh70?= =?us-ascii?Q?RO9k39JsbFjUBrf/3crqIOCAOfaEz6G0rnyBHYZrYQr4Q9qBV+ff6eBZlZ5h?= =?us-ascii?Q?ng=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Ce6JZumWx3ypSnBJI0mGNrlyIF9DiC08g4gsiNZ4XbKaTmOvByQ0oVPH82hKdxX+SvTLkUo7FJf/8uzPOsl5UYpfHcW6qhhKoI6QXK+CoV7xjnrWe77PXhB9ytAen5OPXTIflDRgz1OL0zaBKtblKcTOhMk0Pw9LswYgz3qOBp+BrbIgw2o/2kK77K5fv1PcJldZe97VwXdLFUK1OBcXRFsQF7sg6lwOOW/gLNtodSEoZj7tBnv6+YOnulz9cDD+jHr0GdK4QE6tmUS4sfnM3t/g7swGJvITksbS83SlFs20kc7pHjn4QK0CTG/o4kIyGIZrQlJSkZ3yTegojo3HC31sj7EYkGBTn9ZVaF5FkVpEaBSwLWEyADO4dm6NftU1A2V1s3fWs5V7foRmDE4/ocIzMk2VuWbKPeeNnkeBwHONF6q/S35GhRP+nml7gpKLzUxxyPMI/W8n1c8eEIs2q7jBIGygLqloWK7h60KfJ8c87cqexhOj9SgL9wXltJDWZmJHGpV73VuZZgJmWJHw1zVV5suuNAKZHe+3oFcr7kdKkke7LJdZiEf+kq/59Ii9rlr1cyGvx0aaQgrJ13tiPdEebpSYQHNmIX7RqW9jJyo= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0c00907b-f3c1-4be6-bc73-08de20a79052 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:21:58.8058 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: yxPF0Z77Bn13itgv4qnb4iRIacJm2Pd6Qgjfm4l3IYg756RRLX/sbUFh/MoNUR7BGtJuU1SIixzr/LohRp2/j9Yg3hXTS0i3J5s6hyAybcU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE3OSBTYWx0ZWRfX3Syre37iH4Xg 79ijIrJhQpP4DcHiW3y5CTwTqQ8qZthQRI1i2DfqC19Fsx+H9qtCxKNeus1Jyv4Xz41/UaaT9UX OgAE3zrWBnEPOEvD8VUR8bZrv8lvQkpkjB7OIsmxOcflwJRAtwp/aME6rfn7pTajjFgFkK+DSMB 9khYFMRjvsTqe8JX1Yz76oYGrGEOOt5ltft1bcd7CXUOEX6+jrFlnoZl65vf1KRgHoE0TixQPdu f2MT/FjBaUK9fnSSZysqtOe0MGh4OHfLyyBUYbqxvgeXBoh0qiQys7cC/Ckn86nqEAo/K1QNxRr 5VRA93Fb9/fwjULFY6fHvEFT4vg3z3oXJtMb0NOaJjAWjyJktA81hXR1LXaMvZbUVGq4CqQS1pQ KreJo5zQEaAP8IKjk4W+O36EL9n5+g== X-Authority-Analysis: v=2.4 cv=H5rWAuYi c=1 sm=1 tr=0 ts=6912658c cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=VwQbUJbxAAAA:8 a=yPCof4ZbAAAA:8 a=1B2o6Llj8-7n3TOjnuQA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 X-Proofpoint-ORIG-GUID: VvwX3RcCNMOFNSnDmD20JaddGCRjsrYG X-Proofpoint-GUID: VvwX3RcCNMOFNSnDmD20JaddGCRjsrYG Content-Type: text/plain; charset="utf-8" Introduce softleaf_from_pmd() to do the equivalent operation for PMDs that softleaf_from_pte() fulfils, and cascade changes through code base accordingly, introducing helpers as necessary. We are then able to eliminate pmd_to_swp_entry(), is_pmd_migration_entry(), is_pmd_device_private_entry() and is_pmd_non_present_folio_entry(). This further establishes the use of leaf operations throughout the code base and further establishes the foundations for eliminating is_swap_pmd(). No functional change intended. Reviewed-by: SeongJae Park Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- fs/proc/task_mmu.c | 27 +++-- include/linux/leafops.h | 218 +++++++++++++++++++++++++++++++++++++++- include/linux/migrate.h | 2 +- include/linux/swapops.h | 100 ------------------ mm/damon/ops-common.c | 6 +- mm/filemap.c | 6 +- mm/hmm.c | 16 +-- mm/huge_memory.c | 98 +++++++++--------- mm/khugepaged.c | 4 +- mm/madvise.c | 2 +- mm/memory.c | 4 +- mm/mempolicy.c | 4 +- mm/migrate.c | 20 ++-- mm/migrate_device.c | 14 +-- mm/page_table_check.c | 16 +-- mm/page_vma_mapped.c | 15 +-- mm/pagewalk.c | 8 +- mm/rmap.c | 4 +- 18 files changed, 339 insertions(+), 225 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index b68eabb26f29..d982fdfcf057 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1065,10 +1065,10 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned lo= ng addr, page =3D vm_normal_page_pmd(vma, addr, *pmd); present =3D true; } else if (unlikely(thp_migration_supported())) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); + const softleaf_t entry =3D softleaf_from_pmd(*pmd); =20 - if (is_pfn_swap_entry(entry)) - page =3D pfn_swap_entry_to_page(entry); + if (softleaf_has_pfn(entry)) + page =3D softleaf_to_page(entry); } if (IS_ERR_OR_NULL(page)) return; @@ -1654,7 +1654,7 @@ static inline void clear_soft_dirty_pmd(struct vm_are= a_struct *vma, pmd =3D pmd_clear_soft_dirty(pmd); =20 set_pmd_at(vma->vm_mm, addr, pmdp, pmd); - } else if (is_migration_entry(pmd_to_swp_entry(pmd))) { + } else if (pmd_is_migration_entry(pmd)) { pmd =3D pmd_swp_clear_soft_dirty(pmd); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); } @@ -2015,12 +2015,12 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsig= ned long addr, if (pm->show_pfn) frame =3D pmd_pfn(pmd) + idx; } else if (thp_migration_supported()) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + const softleaf_t entry =3D softleaf_from_pmd(pmd); unsigned long offset; =20 if (pm->show_pfn) { - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry) + idx; + if (softleaf_has_pfn(entry)) + offset =3D softleaf_to_pfn(entry) + idx; else offset =3D swp_offset(entry) + idx; frame =3D swp_type(entry) | @@ -2031,7 +2031,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, flags |=3D PM_SOFT_DIRTY; if (pmd_swp_uffd_wp(pmd)) flags |=3D PM_UFFD_WP; - VM_WARN_ON_ONCE(!is_pmd_migration_entry(pmd)); + VM_WARN_ON_ONCE(!pmd_is_migration_entry(pmd)); page =3D pfn_swap_entry_to_page(entry); } =20 @@ -2425,8 +2425,6 @@ static unsigned long pagemap_thp_category(struct page= map_scan_private *p, if (pmd_soft_dirty(pmd)) categories |=3D PAGE_IS_SOFT_DIRTY; } else { - swp_entry_t swp; - categories |=3D PAGE_IS_SWAPPED; if (!pmd_swp_uffd_wp(pmd)) categories |=3D PAGE_IS_WRITTEN; @@ -2434,9 +2432,10 @@ static unsigned long pagemap_thp_category(struct pag= emap_scan_private *p, categories |=3D PAGE_IS_SOFT_DIRTY; =20 if (p->masks_of_interest & PAGE_IS_FILE) { - swp =3D pmd_to_swp_entry(pmd); - if (is_pfn_swap_entry(swp) && - !folio_test_anon(pfn_swap_entry_folio(swp))) + const softleaf_t entry =3D softleaf_from_pmd(pmd); + + if (softleaf_has_pfn(entry) && + !folio_test_anon(softleaf_to_folio(entry))) categories |=3D PAGE_IS_FILE; } } @@ -2453,7 +2452,7 @@ static void make_uffd_wp_pmd(struct vm_area_struct *v= ma, old =3D pmdp_invalidate_ad(vma, addr, pmdp); pmd =3D pmd_mkuffd_wp(old); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); - } else if (is_migration_entry(pmd_to_swp_entry(pmd))) { + } else if (pmd_is_migration_entry(pmd)) { pmd =3D pmd_swp_mkuffd_wp(pmd); set_pmd_at(vma->vm_mm, addr, pmdp, pmd); } diff --git a/include/linux/leafops.h b/include/linux/leafops.h index cff9d94fd5d1..f5ea9b0385ff 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -61,6 +61,57 @@ static inline softleaf_t softleaf_from_pte(pte_t pte) return pte_to_swp_entry(pte); } =20 +/** + * softleaf_to_pte() - Obtain a PTE entry from a leaf entry. + * @entry: Leaf entry. + * + * This generates an architecture-specific PTE entry that can be utilised = to + * encode the metadata the leaf entry encodes. + * + * Returns: Architecture-specific PTE entry encoding leaf entry. + */ +static inline pte_t softleaf_to_pte(softleaf_t entry) +{ + /* Temporary until swp_entry_t eliminated. */ + return swp_entry_to_pte(entry); +} + +#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION +/** + * softleaf_from_pmd() - Obtain a leaf entry from a PMD entry. + * @pmd: PMD entry. + * + * If @pmd is present (therefore not a leaf entry) the function returns an= empty + * leaf entry. Otherwise, it returns a leaf entry. + * + * Returns: Leaf entry. + */ +static inline softleaf_t softleaf_from_pmd(pmd_t pmd) +{ + softleaf_t arch_entry; + + if (pmd_present(pmd) || pmd_none(pmd)) + return softleaf_mk_none(); + + if (pmd_swp_soft_dirty(pmd)) + pmd =3D pmd_swp_clear_soft_dirty(pmd); + if (pmd_swp_uffd_wp(pmd)) + pmd =3D pmd_swp_clear_uffd_wp(pmd); + arch_entry =3D __pmd_to_swp_entry(pmd); + + /* Temporary until swp_entry_t eliminated. */ + return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); +} + +#else + +static inline softleaf_t softleaf_from_pmd(pmd_t pmd) +{ + return softleaf_mk_none(); +} + +#endif + /** * softleaf_is_none() - Is the leaf entry empty? * @entry: Leaf entry. @@ -134,6 +185,43 @@ static inline bool softleaf_is_swap(softleaf_t entry) return softleaf_type(entry) =3D=3D SOFTLEAF_SWAP; } =20 +/** + * softleaf_is_migration_write() - Is this leaf entry a writable migration= entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a writable migration entry, otherwise + * false. + */ +static inline bool softleaf_is_migration_write(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_MIGRATION_WRITE; +} + +/** + * softleaf_is_migration_read() - Is this leaf entry a readable migration = entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a readable migration entry, otherwise + * false. + */ +static inline bool softleaf_is_migration_read(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_MIGRATION_READ; +} + +/** + * softleaf_is_migration_read_exclusive() - Is this leaf entry an exclusive + * readable migration entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is an exclusive readable migration entr= y, + * otherwise false. + */ +static inline bool softleaf_is_migration_read_exclusive(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_MIGRATION_READ_EXCLUSIVE; +} + /** * softleaf_is_migration() - Is this leaf entry a migration entry? * @entry: Leaf entry. @@ -152,6 +240,19 @@ static inline bool softleaf_is_migration(softleaf_t en= try) } } =20 +/** + * softleaf_is_device_private_write() - Is this leaf entry a device private + * writable entry? + * @entry: Leaf entry. + * + * Returns: true if the leaf entry is a device private writable entry, oth= erwise + * false. + */ +static inline bool softleaf_is_device_private_write(softleaf_t entry) +{ + return softleaf_type(entry) =3D=3D SOFTLEAF_DEVICE_PRIVATE_WRITE; +} + /** * softleaf_is_device_private() - Is this leaf entry a device private entr= y? * @entry: Leaf entry. @@ -170,10 +271,10 @@ static inline bool softleaf_is_device_private(softlea= f_t entry) } =20 /** - * softleaf_is_device_exclusive() - Is this leaf entry a device exclusive = entry? + * softleaf_is_device_exclusive() - Is this leaf entry a device-exclusive = entry? * @entry: Leaf entry. * - * Returns: true if the leaf entry is a device exclusive entry, otherwise = false. + * Returns: true if the leaf entry is a device-exclusive entry, otherwise = false. */ static inline bool softleaf_is_device_exclusive(softleaf_t entry) { @@ -332,6 +433,61 @@ static inline bool softleaf_is_uffd_wp_marker(softleaf= _t entry) return softleaf_to_marker(entry) & PTE_MARKER_UFFD_WP; } =20 +#ifdef CONFIG_MIGRATION + +/** + * softleaf_is_migration_young() - Does this migration entry contain an ac= cessed + * bit? + * @entry: Leaf entry. + * + * If the architecture can support storing A/D bits in migration entries, = this + * determines whether the accessed (or 'young') bit was set on the migrate= d page + * table entry. + * + * Returns: true if the entry contains an accessed bit, otherwise false. + */ +static inline bool softleaf_is_migration_young(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_is_migration(entry)); + + if (migration_entry_supports_ad()) + return swp_offset(entry) & SWP_MIG_YOUNG; + /* Keep the old behavior of aging page after migration */ + return false; +} + +/** + * softleaf_is_migration_dirty() - Does this migration entry contain a dir= ty bit? + * @entry: Leaf entry. + * + * If the architecture can support storing A/D bits in migration entries, = this + * determines whether the dirty bit was set on the migrated page table ent= ry. + * + * Returns: true if the entry contains a dirty bit, otherwise false. + */ +static inline bool softleaf_is_migration_dirty(softleaf_t entry) +{ + VM_WARN_ON_ONCE(!softleaf_is_migration(entry)); + + if (migration_entry_supports_ad()) + return swp_offset(entry) & SWP_MIG_DIRTY; + /* Keep the old behavior of clean page after migration */ + return false; +} + +#else /* CONFIG_MIGRATION */ + +static inline bool softleaf_is_migration_young(softleaf_t entry) +{ + return false; +} + +static inline bool softleaf_is_migration_dirty(softleaf_t entry) +{ + return false; +} +#endif /* CONFIG_MIGRATION */ + /** * pte_is_marker() - Does the PTE entry encode a marker leaf entry? * @pte: PTE entry. @@ -383,5 +539,63 @@ static inline bool pte_is_uffd_marker(pte_t pte) return false; } =20 +#if defined(CONFIG_ZONE_DEVICE) && defined(CONFIG_ARCH_ENABLE_THP_MIGRATIO= N) + +/** + * pmd_is_device_private_entry() - Check if PMD contains a device private = swap + * entry. + * @pmd: The PMD to check. + * + * Returns true if the PMD contains a swap entry that represents a device = private + * page mapping. This is used for zone device private pages that have been + * swapped out but still need special handling during various memory manag= ement + * operations. + * + * Return: true if PMD contains device private entry, false otherwise + */ +static inline bool pmd_is_device_private_entry(pmd_t pmd) +{ + return softleaf_is_device_private(softleaf_from_pmd(pmd)); +} + +#else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ + +static inline bool pmd_is_device_private_entry(pmd_t pmd) +{ + return false; +} + +#endif /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ + +/** + * pmd_is_migration_entry() - Does this PMD entry encode a migration entry? + * @pmd: PMD entry. + * + * Returns: true if the PMD encodes a migration entry, otherwise false. + */ +static inline bool pmd_is_migration_entry(pmd_t pmd) +{ + return softleaf_is_migration(softleaf_from_pmd(pmd)); +} + +/** + * pmd_is_valid_softleaf() - Is this PMD entry a valid leaf entry? + * @pmd: PMD entry. + * + * PMD leaf entries are valid only if they are device private or migration + * entries. This function asserts that a PMD leaf entry is valid in this + * respect. + * + * Returns: true if the PMD entry is a valid leaf entry, otherwise false. + */ +static inline bool pmd_is_valid_softleaf(pmd_t pmd) +{ + const softleaf_t entry =3D softleaf_from_pmd(pmd); + + /* Only device private, migration entries valid for PMD. */ + return softleaf_is_device_private(entry) || + softleaf_is_migration(entry); +} + #endif /* CONFIG_MMU */ #endif /* _LINUX_LEAFOPS_H */ diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 41b4cc05a450..26ca00c325d9 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -65,7 +65,7 @@ bool isolate_folio_to_list(struct folio *folio, struct li= st_head *list); =20 int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src); -void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) +void migration_entry_wait_on_locked(softleaf_t entry, spinlock_t *ptl) __releases(ptl); void folio_migrate_flags(struct folio *newfolio, struct folio *folio); int folio_migrate_mapping(struct address_space *mapping, diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 3e8dd6ea94dd..f1277647262d 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -283,14 +283,6 @@ static inline swp_entry_t make_migration_entry_young(s= wp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_young(swp_entry_t entry) -{ - if (migration_entry_supports_ad()) - return swp_offset(entry) & SWP_MIG_YOUNG; - /* Keep the old behavior of aging page after migration */ - return false; -} - static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry) { if (migration_entry_supports_ad()) @@ -299,14 +291,6 @@ static inline swp_entry_t make_migration_entry_dirty(s= wp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_dirty(swp_entry_t entry) -{ - if (migration_entry_supports_ad()) - return swp_offset(entry) & SWP_MIG_DIRTY; - /* Keep the old behavior of clean page after migration */ - return false; -} - extern void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address); extern void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned= long addr, pte_t *pte); @@ -349,20 +333,11 @@ static inline swp_entry_t make_migration_entry_young(= swp_entry_t entry) return entry; } =20 -static inline bool is_migration_entry_young(swp_entry_t entry) -{ - return false; -} - static inline swp_entry_t make_migration_entry_dirty(swp_entry_t entry) { return entry; } =20 -static inline bool is_migration_entry_dirty(swp_entry_t entry) -{ - return false; -} #endif /* CONFIG_MIGRATION */ =20 #ifdef CONFIG_MEMORY_FAILURE @@ -487,18 +462,6 @@ extern void remove_migration_pmd(struct page_vma_mappe= d_walk *pvmw, =20 extern void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd); =20 -static inline swp_entry_t pmd_to_swp_entry(pmd_t pmd) -{ - swp_entry_t arch_entry; - - if (pmd_swp_soft_dirty(pmd)) - pmd =3D pmd_swp_clear_soft_dirty(pmd); - if (pmd_swp_uffd_wp(pmd)) - pmd =3D pmd_swp_clear_uffd_wp(pmd); - arch_entry =3D __pmd_to_swp_entry(pmd); - return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); -} - static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) { swp_entry_t arch_entry; @@ -507,23 +470,7 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) return __swp_entry_to_pmd(arch_entry); } =20 -static inline int is_pmd_migration_entry(pmd_t pmd) -{ - swp_entry_t entry; - - if (pmd_present(pmd)) - return 0; - - entry =3D pmd_to_swp_entry(pmd); - return is_migration_entry(entry); -} #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ -static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, - struct page *page) -{ - BUILD_BUG(); -} - static inline void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) { @@ -532,64 +479,17 @@ static inline void remove_migration_pmd(struct page_v= ma_mapped_walk *pvmw, =20 static inline void pmd_migration_entry_wait(struct mm_struct *m, pmd_t *p)= { } =20 -static inline swp_entry_t pmd_to_swp_entry(pmd_t pmd) -{ - return swp_entry(0, 0); -} - static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) { return __pmd(0); } =20 -static inline int is_pmd_migration_entry(pmd_t pmd) -{ - return 0; -} #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 -#if defined(CONFIG_ZONE_DEVICE) && defined(CONFIG_ARCH_ENABLE_THP_MIGRATIO= N) - -/** - * is_pmd_device_private_entry() - Check if PMD contains a device private = swap entry - * @pmd: The PMD to check - * - * Returns true if the PMD contains a swap entry that represents a device = private - * page mapping. This is used for zone device private pages that have been - * swapped out but still need special handling during various memory manag= ement - * operations. - * - * Return: 1 if PMD contains device private entry, 0 otherwise - */ -static inline int is_pmd_device_private_entry(pmd_t pmd) -{ - swp_entry_t entry; - - if (pmd_present(pmd)) - return 0; - - entry =3D pmd_to_swp_entry(pmd); - return is_device_private_entry(entry); -} - -#else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ - -static inline int is_pmd_device_private_entry(pmd_t pmd) -{ - return 0; -} - -#endif /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */ - static inline int non_swap_entry(swp_entry_t entry) { return swp_type(entry) >=3D MAX_SWAPFILES; } =20 -static inline int is_pmd_non_present_folio_entry(pmd_t pmd) -{ - return is_pmd_migration_entry(pmd) || is_pmd_device_private_entry(pmd); -} - #endif /* CONFIG_MMU */ #endif /* _LINUX_SWAPOPS_H */ diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index 971df8a16ba4..a218d9922234 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -11,7 +11,7 @@ #include #include #include -#include +#include =20 #include "../internal.h" #include "ops-common.h" @@ -51,7 +51,7 @@ void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *= vma, unsigned long addr if (likely(pte_present(pteval))) pfn =3D pte_pfn(pteval); else - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D softleaf_to_pfn(softleaf_from_pte(pteval)); =20 folio =3D damon_get_folio(pfn); if (!folio) @@ -83,7 +83,7 @@ void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *= vma, unsigned long addr if (likely(pmd_present(pmdval))) pfn =3D pmd_pfn(pmdval); else - pfn =3D swp_offset_pfn(pmd_to_swp_entry(pmdval)); + pfn =3D softleaf_to_pfn(softleaf_from_pmd(pmdval)); =20 folio =3D damon_get_folio(pfn); if (!folio) diff --git a/mm/filemap.c b/mm/filemap.c index ff75bd89b68c..950d93885e38 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -21,7 +21,7 @@ #include #include #include -#include +#include #include #include #include @@ -1402,7 +1402,7 @@ static inline int folio_wait_bit_common(struct folio = *folio, int bit_nr, * This follows the same logic as folio_wait_bit_common() so see the comme= nts * there. */ -void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) +void migration_entry_wait_on_locked(softleaf_t entry, spinlock_t *ptl) __releases(ptl) { struct wait_page_queue wait_page; @@ -1411,7 +1411,7 @@ void migration_entry_wait_on_locked(swp_entry_t entry= , spinlock_t *ptl) unsigned long pflags; bool in_thrashing; wait_queue_head_t *q; - struct folio *folio =3D pfn_swap_entry_folio(entry); + struct folio *folio =3D softleaf_to_folio(entry); =20 q =3D folio_waitqueue(folio); if (!folio_test_uptodate(folio) && folio_test_workingset(folio)) { diff --git a/mm/hmm.c b/mm/hmm.c index e350d0cc9d41..e9735a9b6102 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -18,7 +18,7 @@ #include #include #include -#include +#include #include #include #include @@ -339,19 +339,19 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *= walk, unsigned long start, struct hmm_vma_walk *hmm_vma_walk =3D walk->private; struct hmm_range *range =3D hmm_vma_walk->range; unsigned long npages =3D (end - start) >> PAGE_SHIFT; + const softleaf_t entry =3D softleaf_from_pmd(pmd); unsigned long addr =3D start; - swp_entry_t entry =3D pmd_to_swp_entry(pmd); unsigned int required_fault; =20 - if (is_device_private_entry(entry) && - pfn_swap_entry_folio(entry)->pgmap->owner =3D=3D + if (softleaf_is_device_private(entry) && + softleaf_to_folio(entry)->pgmap->owner =3D=3D range->dev_private_owner) { unsigned long cpu_flags =3D HMM_PFN_VALID | hmm_pfn_flags_order(PMD_SHIFT - PAGE_SHIFT); - unsigned long pfn =3D swp_offset_pfn(entry); + unsigned long pfn =3D softleaf_to_pfn(entry); unsigned long i; =20 - if (is_writable_device_private_entry(entry)) + if (softleaf_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; =20 /* @@ -370,7 +370,7 @@ static int hmm_vma_handle_absent_pmd(struct mm_walk *wa= lk, unsigned long start, required_fault =3D hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, 0); if (required_fault) { - if (is_device_private_entry(entry)) + if (softleaf_is_device_private(entry)) return hmm_vma_fault(addr, end, required_fault, walk); else return -EFAULT; @@ -412,7 +412,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, if (pmd_none(pmd)) return hmm_vma_walk_hole(start, end, -1, walk); =20 - if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { + if (thp_migration_supported() && pmd_is_migration_entry(pmd)) { if (hmm_range_need_fault(hmm_vma_walk, hmm_pfns, npages, 0)) { hmm_vma_walk->last =3D addr; pmd_migration_entry_wait(walk->mm, pmdp); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 40a8a2c1e080..5876595b00d5 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1374,7 +1374,7 @@ vm_fault_t do_huge_pmd_device_private(struct vm_fault= *vmf) struct vm_area_struct *vma =3D vmf->vma; vm_fault_t ret =3D 0; spinlock_t *ptl; - swp_entry_t swp_entry; + softleaf_t entry; struct page *page; struct folio *folio; =20 @@ -1389,8 +1389,8 @@ vm_fault_t do_huge_pmd_device_private(struct vm_fault= *vmf) return 0; } =20 - swp_entry =3D pmd_to_swp_entry(vmf->orig_pmd); - page =3D pfn_swap_entry_to_page(swp_entry); + entry =3D softleaf_from_pmd(vmf->orig_pmd); + page =3D softleaf_to_page(entry); folio =3D page_folio(page); vmf->page =3D page; vmf->pte =3D NULL; @@ -1780,13 +1780,13 @@ static void copy_huge_non_present_pmd( struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, pmd_t pmd, pgtable_t pgtable) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + softleaf_t entry =3D softleaf_from_pmd(pmd); struct folio *src_folio; =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(pmd)); + VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(pmd)); =20 - if (is_writable_migration_entry(entry) || - is_readable_exclusive_migration_entry(entry)) { + if (softleaf_is_migration_write(entry) || + softleaf_is_migration_read_exclusive(entry)) { entry =3D make_readable_migration_entry(swp_offset(entry)); pmd =3D swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*src_pmd)) @@ -1794,12 +1794,12 @@ static void copy_huge_non_present_pmd( if (pmd_swp_uffd_wp(*src_pmd)) pmd =3D pmd_swp_mkuffd_wp(pmd); set_pmd_at(src_mm, addr, src_pmd, pmd); - } else if (is_device_private_entry(entry)) { + } else if (softleaf_is_device_private(entry)) { /* * For device private entries, since there are no * read exclusive entries, writable =3D !readable */ - if (is_writable_device_private_entry(entry)) { + if (softleaf_is_device_private_write(entry)) { entry =3D make_readable_device_private_entry(swp_offset(entry)); pmd =3D swp_entry_to_pmd(entry); =20 @@ -1810,7 +1810,7 @@ static void copy_huge_non_present_pmd( set_pmd_at(src_mm, addr, src_pmd, pmd); } =20 - src_folio =3D pfn_swap_entry_folio(entry); + src_folio =3D softleaf_to_folio(entry); VM_WARN_ON(!folio_test_large(src_folio)); =20 folio_get(src_folio); @@ -2270,7 +2270,7 @@ bool madvise_free_huge_pmd(struct mmu_gather *tlb, st= ruct vm_area_struct *vma, =20 if (unlikely(!pmd_present(orig_pmd))) { VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); + !pmd_is_migration_entry(orig_pmd)); goto out; } =20 @@ -2368,11 +2368,10 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_= area_struct *vma, folio_remove_rmap_pmd(folio, page, vma); WARN_ON_ONCE(folio_mapcount(folio) < 0); VM_BUG_ON_PAGE(!PageHead(page), page); - } else if (is_pmd_non_present_folio_entry(orig_pmd)) { - swp_entry_t entry; + } else if (pmd_is_valid_softleaf(orig_pmd)) { + const softleaf_t entry =3D softleaf_from_pmd(orig_pmd); =20 - entry =3D pmd_to_swp_entry(orig_pmd); - folio =3D pfn_swap_entry_folio(entry); + folio =3D softleaf_to_folio(entry); flush_needed =3D 0; =20 if (!thp_migration_supported()) @@ -2428,7 +2427,7 @@ static inline int pmd_move_must_withdraw(spinlock_t *= new_pmd_ptl, static pmd_t move_soft_dirty_pmd(pmd_t pmd) { #ifdef CONFIG_MEM_SOFT_DIRTY - if (unlikely(is_pmd_migration_entry(pmd))) + if (unlikely(pmd_is_migration_entry(pmd))) pmd =3D pmd_swp_mksoft_dirty(pmd); else if (pmd_present(pmd)) pmd =3D pmd_mksoft_dirty(pmd); @@ -2503,12 +2502,12 @@ static void change_non_present_huge_pmd(struct mm_s= truct *mm, unsigned long addr, pmd_t *pmd, bool uffd_wp, bool uffd_wp_resolve) { - swp_entry_t entry =3D pmd_to_swp_entry(*pmd); - struct folio *folio =3D pfn_swap_entry_folio(entry); + softleaf_t entry =3D softleaf_from_pmd(*pmd); + const struct folio *folio =3D softleaf_to_folio(entry); pmd_t newpmd; =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd)); - if (is_writable_migration_entry(entry)) { + VM_WARN_ON(!pmd_is_valid_softleaf(*pmd)); + if (softleaf_is_migration_write(entry)) { /* * A protection check is difficult so * just be safe and disable write @@ -2520,7 +2519,7 @@ static void change_non_present_huge_pmd(struct mm_str= uct *mm, newpmd =3D swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*pmd)) newpmd =3D pmd_swp_mksoft_dirty(newpmd); - } else if (is_writable_device_private_entry(entry)) { + } else if (softleaf_is_device_private_write(entry)) { entry =3D make_readable_device_private_entry(swp_offset(entry)); newpmd =3D swp_entry_to_pmd(entry); } else { @@ -2718,7 +2717,7 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *= dst_pmd, pmd_t *src_pmd, pm =20 if (!pmd_trans_huge(src_pmdval)) { spin_unlock(src_ptl); - if (is_pmd_migration_entry(src_pmdval)) { + if (pmd_is_migration_entry(src_pmdval)) { pmd_migration_entry_wait(mm, &src_pmdval); return -EAGAIN; } @@ -2983,13 +2982,12 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, unsigned long addr; pte_t *pte; int i; - swp_entry_t entry; =20 VM_BUG_ON(haddr & ~HPAGE_PMD_MASK); VM_BUG_ON_VMA(vma->vm_start > haddr, vma); VM_BUG_ON_VMA(vma->vm_end < haddr + HPAGE_PMD_SIZE, vma); =20 - VM_WARN_ON(!is_pmd_non_present_folio_entry(*pmd) && !pmd_trans_huge(*pmd)= ); + VM_WARN_ON_ONCE(!pmd_is_valid_softleaf(*pmd) && !pmd_trans_huge(*pmd)); =20 count_vm_event(THP_SPLIT_PMD); =20 @@ -3003,11 +3001,10 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, zap_deposited_table(mm, pmd); if (!vma_is_dax(vma) && vma_is_special_huge(vma)) return; - if (unlikely(is_pmd_migration_entry(old_pmd))) { - swp_entry_t entry; + if (unlikely(pmd_is_migration_entry(old_pmd))) { + const softleaf_t old_entry =3D softleaf_from_pmd(old_pmd); =20 - entry =3D pmd_to_swp_entry(old_pmd); - folio =3D pfn_swap_entry_folio(entry); + folio =3D softleaf_to_folio(old_entry); } else if (is_huge_zero_pmd(old_pmd)) { return; } else { @@ -3037,31 +3034,34 @@ static void __split_huge_pmd_locked(struct vm_area_= struct *vma, pmd_t *pmd, return __split_huge_zero_page_pmd(vma, haddr, pmd); } =20 + if (pmd_is_migration_entry(*pmd)) { + softleaf_t entry; =20 - if (is_pmd_migration_entry(*pmd)) { old_pmd =3D *pmd; - entry =3D pmd_to_swp_entry(old_pmd); - page =3D pfn_swap_entry_to_page(entry); + entry =3D softleaf_from_pmd(old_pmd); + page =3D softleaf_to_page(entry); folio =3D page_folio(page); =20 soft_dirty =3D pmd_swp_soft_dirty(old_pmd); uffd_wp =3D pmd_swp_uffd_wp(old_pmd); =20 - write =3D is_writable_migration_entry(entry); + write =3D softleaf_is_migration_write(entry); if (PageAnon(page)) - anon_exclusive =3D is_readable_exclusive_migration_entry(entry); - young =3D is_migration_entry_young(entry); - dirty =3D is_migration_entry_dirty(entry); - } else if (is_pmd_device_private_entry(*pmd)) { + anon_exclusive =3D softleaf_is_migration_read_exclusive(entry); + young =3D softleaf_is_migration_young(entry); + dirty =3D softleaf_is_migration_dirty(entry); + } else if (pmd_is_device_private_entry(*pmd)) { + softleaf_t entry; + old_pmd =3D *pmd; - entry =3D pmd_to_swp_entry(old_pmd); - page =3D pfn_swap_entry_to_page(entry); + entry =3D softleaf_from_pmd(old_pmd); + page =3D softleaf_to_page(entry); folio =3D page_folio(page); =20 soft_dirty =3D pmd_swp_soft_dirty(old_pmd); uffd_wp =3D pmd_swp_uffd_wp(old_pmd); =20 - write =3D is_writable_device_private_entry(entry); + write =3D softleaf_is_device_private_write(entry); anon_exclusive =3D PageAnonExclusive(page); =20 /* @@ -3165,7 +3165,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, * Note that NUMA hinting access restrictions are not transferred to * avoid any possibility of altering permissions across VMAs. */ - if (freeze || is_pmd_migration_entry(old_pmd)) { + if (freeze || pmd_is_migration_entry(old_pmd)) { pte_t entry; swp_entry_t swp_entry; =20 @@ -3191,7 +3191,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, VM_WARN_ON(!pte_none(ptep_get(pte + i))); set_pte_at(mm, addr, pte + i, entry); } - } else if (is_pmd_device_private_entry(old_pmd)) { + } else if (pmd_is_device_private_entry(old_pmd)) { pte_t entry; swp_entry_t swp_entry; =20 @@ -3241,7 +3241,7 @@ static void __split_huge_pmd_locked(struct vm_area_st= ruct *vma, pmd_t *pmd, } pte_unmap(pte); =20 - if (!is_pmd_migration_entry(*pmd)) + if (!pmd_is_migration_entry(*pmd)) folio_remove_rmap_pmd(folio, page, vma); if (freeze) put_page(page); @@ -3254,7 +3254,7 @@ void split_huge_pmd_locked(struct vm_area_struct *vma= , unsigned long address, pmd_t *pmd, bool freeze) { VM_WARN_ON_ONCE(!IS_ALIGNED(address, HPAGE_PMD_SIZE)); - if (pmd_trans_huge(*pmd) || is_pmd_non_present_folio_entry(*pmd)) + if (pmd_trans_huge(*pmd) || pmd_is_valid_softleaf(*pmd)) __split_huge_pmd_locked(vma, pmd, address, freeze); } =20 @@ -4855,12 +4855,12 @@ void remove_migration_pmd(struct page_vma_mapped_wa= lk *pvmw, struct page *new) unsigned long address =3D pvmw->address; unsigned long haddr =3D address & HPAGE_PMD_MASK; pmd_t pmde; - swp_entry_t entry; + softleaf_t entry; =20 if (!(pvmw->pmd && !pvmw->pte)) return; =20 - entry =3D pmd_to_swp_entry(*pvmw->pmd); + entry =3D softleaf_from_pmd(*pvmw->pmd); folio_get(folio); pmde =3D folio_mk_pmd(folio, READ_ONCE(vma->vm_page_prot)); =20 @@ -4876,20 +4876,20 @@ void remove_migration_pmd(struct page_vma_mapped_wa= lk *pvmw, struct page *new) =20 if (pmd_swp_soft_dirty(*pvmw->pmd)) pmde =3D pmd_mksoft_dirty(pmde); - if (is_writable_migration_entry(entry)) + if (softleaf_is_migration_write(entry)) pmde =3D pmd_mkwrite(pmde, vma); if (pmd_swp_uffd_wp(*pvmw->pmd)) pmde =3D pmd_mkuffd_wp(pmde); - if (!is_migration_entry_young(entry)) + if (!softleaf_is_migration_young(entry)) pmde =3D pmd_mkold(pmde); /* NOTE: this may contain setting soft-dirty on some archs */ - if (folio_test_dirty(folio) && is_migration_entry_dirty(entry)) + if (folio_test_dirty(folio) && softleaf_is_migration_dirty(entry)) pmde =3D pmd_mkdirty(pmde); =20 if (folio_test_anon(folio)) { rmap_t rmap_flags =3D RMAP_NONE; =20 - if (!is_readable_migration_entry(entry)) + if (!softleaf_is_migration_read(entry)) rmap_flags |=3D RMAP_EXCLUSIVE; =20 folio_add_anon_rmap_pmd(folio, new, vma, haddr, rmap_flags); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a97ff7bcb232..1a08673b0d8b 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -17,7 +17,7 @@ #include #include #include -#include +#include #include #include #include @@ -941,7 +941,7 @@ static inline int check_pmd_state(pmd_t *pmd) * collapse it. Migration success or failure will eventually end * up with a present PMD mapping a folio again. */ - if (is_pmd_migration_entry(pmde)) + if (pmd_is_migration_entry(pmde)) return SCAN_PMD_MAPPED; if (!pmd_present(pmde)) return SCAN_PMD_NULL; diff --git a/mm/madvise.c b/mm/madvise.c index 58d82495b6c6..ffae3b566dc1 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -390,7 +390,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, =20 if (unlikely(!pmd_present(orig_pmd))) { VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); + !pmd_is_migration_entry(orig_pmd)); goto huge_unlock; } =20 diff --git a/mm/memory.c b/mm/memory.c index fea079e5fb90..bf2bbd0dbc97 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6362,10 +6362,10 @@ static vm_fault_t __handle_mm_fault(struct vm_area_= struct *vma, goto fallback; =20 if (unlikely(!pmd_present(vmf.orig_pmd))) { - if (is_pmd_device_private_entry(vmf.orig_pmd)) + if (pmd_is_device_private_entry(vmf.orig_pmd)) return do_huge_pmd_device_private(&vmf); =20 - if (is_pmd_migration_entry(vmf.orig_pmd)) + if (pmd_is_migration_entry(vmf.orig_pmd)) pmd_migration_entry_wait(mm, vmf.pmd); return 0; } diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 7ae3f5e2dee6..01c3b98f87a6 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -110,7 +110,7 @@ #include #include #include -#include +#include #include =20 #include @@ -647,7 +647,7 @@ static void queue_folios_pmd(pmd_t *pmd, struct mm_walk= *walk) struct folio *folio; struct queue_pages *qp =3D walk->private; =20 - if (unlikely(is_pmd_migration_entry(*pmd))) { + if (unlikely(pmd_is_migration_entry(*pmd))) { qp->nr_failed++; return; } diff --git a/mm/migrate.c b/mm/migrate.c index 862b2e261cf9..3b6bd374157d 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include #include @@ -353,7 +353,7 @@ static bool remove_migration_pte(struct folio *folio, rmap_t rmap_flags =3D RMAP_NONE; pte_t old_pte; pte_t pte; - swp_entry_t entry; + softleaf_t entry; struct page *new; unsigned long idx =3D 0; =20 @@ -379,22 +379,22 @@ static bool remove_migration_pte(struct folio *folio, folio_get(folio); pte =3D mk_pte(new, READ_ONCE(vma->vm_page_prot)); =20 - entry =3D pte_to_swp_entry(old_pte); - if (!is_migration_entry_young(entry)) + entry =3D softleaf_from_pte(old_pte); + if (!softleaf_is_migration_young(entry)) pte =3D pte_mkold(pte); - if (folio_test_dirty(folio) && is_migration_entry_dirty(entry)) + if (folio_test_dirty(folio) && softleaf_is_migration_dirty(entry)) pte =3D pte_mkdirty(pte); if (pte_swp_soft_dirty(old_pte)) pte =3D pte_mksoft_dirty(pte); else pte =3D pte_clear_soft_dirty(pte); =20 - if (is_writable_migration_entry(entry)) + if (softleaf_is_migration_write(entry)) pte =3D pte_mkwrite(pte, vma); else if (pte_swp_uffd_wp(old_pte)) pte =3D pte_mkuffd_wp(pte); =20 - if (folio_test_anon(folio) && !is_readable_migration_entry(entry)) + if (folio_test_anon(folio) && !softleaf_is_migration_read(entry)) rmap_flags |=3D RMAP_EXCLUSIVE; =20 if (unlikely(is_device_private_page(new))) { @@ -404,7 +404,7 @@ static bool remove_migration_pte(struct folio *folio, else entry =3D make_readable_device_private_entry( page_to_pfn(new)); - pte =3D swp_entry_to_pte(entry); + pte =3D softleaf_to_pte(entry); if (pte_swp_soft_dirty(old_pte)) pte =3D pte_swp_mksoft_dirty(pte); if (pte_swp_uffd_wp(old_pte)) @@ -543,9 +543,9 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd= _t *pmd) spinlock_t *ptl; =20 ptl =3D pmd_lock(mm, pmd); - if (!is_pmd_migration_entry(*pmd)) + if (!pmd_is_migration_entry(*pmd)) goto unlock; - migration_entry_wait_on_locked(pmd_to_swp_entry(*pmd), ptl); + migration_entry_wait_on_locked(softleaf_from_pmd(*pmd), ptl); return; unlock: spin_unlock(ptl); diff --git a/mm/migrate_device.c b/mm/migrate_device.c index c869b272e85a..880f26a316f8 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -13,7 +13,7 @@ #include #include #include -#include +#include #include #include #include "internal.h" @@ -145,7 +145,6 @@ static int migrate_vma_collect_huge_pmd(pmd_t *pmdp, un= signed long start, struct folio *folio; struct migrate_vma *migrate =3D walk->private; spinlock_t *ptl; - swp_entry_t entry; int ret; unsigned long write =3D 0; =20 @@ -169,23 +168,24 @@ static int migrate_vma_collect_huge_pmd(pmd_t *pmdp, = unsigned long start, if (pmd_write(*pmdp)) write =3D MIGRATE_PFN_WRITE; } else if (!pmd_present(*pmdp)) { - entry =3D pmd_to_swp_entry(*pmdp); - folio =3D pfn_swap_entry_folio(entry); + const softleaf_t entry =3D softleaf_from_pmd(*pmdp); =20 - if (!is_device_private_entry(entry) || + folio =3D softleaf_to_folio(entry); + + if (!softleaf_is_device_private(entry) || !(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_PRIVATE) || (folio->pgmap->owner !=3D migrate->pgmap_owner)) { spin_unlock(ptl); return migrate_vma_collect_skip(start, end, walk); } =20 - if (is_migration_entry(entry)) { + if (softleaf_is_migration(entry)) { migration_entry_wait_on_locked(entry, ptl); spin_unlock(ptl); return -EAGAIN; } =20 - if (is_writable_device_private_entry(entry)) + if (softleaf_is_device_private_write(entry)) write =3D MIGRATE_PFN_WRITE; } else { spin_unlock(ptl); diff --git a/mm/page_table_check.c b/mm/page_table_check.c index f5f25e120f69..9af1ecff5221 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -8,7 +8,7 @@ #include #include #include -#include +#include =20 #undef pr_fmt #define pr_fmt(fmt) "page_table_check: " fmt @@ -179,10 +179,10 @@ void __page_table_check_pud_clear(struct mm_struct *m= m, pud_t pud) EXPORT_SYMBOL(__page_table_check_pud_clear); =20 /* Whether the swap entry cached writable information */ -static inline bool swap_cached_writable(swp_entry_t entry) +static inline bool softleaf_cached_writable(softleaf_t entry) { - return is_writable_device_private_entry(entry) || - is_writable_migration_entry(entry); + return softleaf_is_device_private(entry) || + softleaf_is_migration_write(entry); } =20 static void page_table_check_pte_flags(pte_t pte) @@ -190,9 +190,9 @@ static void page_table_check_pte_flags(pte_t pte) if (pte_present(pte)) { WARN_ON_ONCE(pte_uffd_wp(pte) && pte_write(pte)); } else if (pte_swp_uffd_wp(pte)) { - const swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 - WARN_ON_ONCE(swap_cached_writable(entry)); + WARN_ON_ONCE(softleaf_cached_writable(entry)); } } =20 @@ -219,9 +219,9 @@ static inline void page_table_check_pmd_flags(pmd_t pmd) if (pmd_uffd_wp(pmd)) WARN_ON_ONCE(pmd_write(pmd)); } else if (pmd_swp_uffd_wp(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + const softleaf_t entry =3D softleaf_from_pmd(pmd); =20 - WARN_ON_ONCE(swap_cached_writable(entry)); + WARN_ON_ONCE(softleaf_cached_writable(entry)); } } =20 diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index a4e23818f37f..8137d2366722 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -242,18 +242,19 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk= *pvmw) */ pmde =3D pmdp_get_lockless(pvmw->pmd); =20 - if (pmd_trans_huge(pmde) || is_pmd_migration_entry(pmde)) { + if (pmd_trans_huge(pmde) || pmd_is_migration_entry(pmde)) { pvmw->ptl =3D pmd_lock(mm, pvmw->pmd); pmde =3D *pvmw->pmd; if (!pmd_present(pmde)) { - swp_entry_t entry; + softleaf_t entry; =20 if (!thp_migration_supported() || !(pvmw->flags & PVMW_MIGRATION)) return not_found(pvmw); - entry =3D pmd_to_swp_entry(pmde); - if (!is_migration_entry(entry) || - !check_pmd(swp_offset_pfn(entry), pvmw)) + entry =3D softleaf_from_pmd(pmde); + + if (!softleaf_is_migration(entry) || + !check_pmd(softleaf_to_pfn(entry), pvmw)) return not_found(pvmw); return true; } @@ -273,9 +274,9 @@ bool page_vma_mapped_walk(struct page_vma_mapped_walk *= pvmw) * cannot return prematurely, while zap_huge_pmd() has * cleared *pmd but not decremented compound_mapcount(). */ - swp_entry_t entry =3D pmd_to_swp_entry(pmde); + const softleaf_t entry =3D softleaf_from_pmd(pmde); =20 - if (is_device_private_entry(entry)) { + if (softleaf_is_device_private(entry)) { pvmw->ptl =3D pmd_lock(mm, pvmw->pmd); return true; } diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 9f91cf85a5be..3067feb970d1 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -5,7 +5,7 @@ #include #include #include -#include +#include =20 #include =20 @@ -966,10 +966,10 @@ struct folio *folio_walk_start(struct folio_walk *fw, goto found; } } else if ((flags & FW_MIGRATION) && - is_pmd_migration_entry(pmd)) { - swp_entry_t entry =3D pmd_to_swp_entry(pmd); + pmd_is_migration_entry(pmd)) { + const softleaf_t entry =3D softleaf_from_pmd(pmd); =20 - page =3D pfn_swap_entry_to_page(entry); + page =3D softleaf_to_page(entry); expose_page =3D false; goto found; } diff --git a/mm/rmap.c b/mm/rmap.c index 1954c538a991..775710115a41 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -57,7 +57,7 @@ #include #include #include -#include +#include #include #include #include @@ -2341,7 +2341,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pmd_present(pmdval))) pfn =3D pmd_pfn(pmdval); else - pfn =3D swp_offset_pfn(pmd_to_swp_entry(pmdval)); + pfn =3D softleaf_to_pfn(softleaf_from_pmd(pmdval)); =20 subpage =3D folio_page(folio, pfn - folio_pfn(folio)); =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A1917341AC7; Mon, 10 Nov 2025 22:24:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813457; cv=fail; b=kQ950h1gSvAL5m50AkJ8TdDHHuZ14GbiqfY0Htm4L+NZnO3oj0etLyCjkw4e8PZNKZxWh7Vl34GInbFsvk2opr4NV+sKXQ2Y+xOndGmcXkVFtDoEzt/uvbzxWn7PIJjQ/PFacdQJNkXXx5OWrN0iLMlLF+KDG+Xc+xZn3Nz+AFw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813457; c=relaxed/simple; bh=POgjLibdleyIT97aLcMp4rGu38r3SsMhgSVUA/LXdP4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=fGPRZ+uPPbmGmomCKK2zMa9bC2Uz3JZNtLqNxsDkkqV2oAN8PhDvD8R9Acd+HDOPv79miLzIMoKPpiEl3vLJRlthlLTt947FH8QE5s1FI5lEklPnSUBAOHZnexNbkd7lSD+JzTTXh2GuWSuM5Bsx2czNMElexrY9wVwAqwpdZuc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=Oy7z0hDr; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=IO3AxVl1; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Oy7z0hDr"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="IO3AxVl1" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAMIJlZ025921; Mon, 10 Nov 2025 22:22:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=hnulhczD3XQkqRXiu7lNvxRJz88HMl5Uk5p8sHNK2Ms=; b= Oy7z0hDr3XHPzWEPjqbW+Yk92OenLk3UC1NakElwLrewB/7LkPQ4ChtYpJHKVWG8 7cS0XMS7+6um5jXJNHzBu6xkNGYUEepQetyZrTtBo4f6OJtjTe9HaF4jO+FN6Z15 LhvTgbZoB38+arHl+J+ay4vwVODKOVOqlkESATwQCQA9Tgqp+fZHrpUKhIvb6EgQ +NcaCBEd2l/IR/gsg3mJjg1RGxdUzZlWHP914ma/TghOau51QMmQeQAY5KyyKFHh 7p8esP50MDq7EH1q7PxIdvbGt4zz2FuxwBxjwtso5RhLng9utOAnd+/mff11mH0u hVqSZkd9N7Bpe41dTgR7RA== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abr5vr2bx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:06 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAKGUhu007458; Mon, 10 Nov 2025 22:22:05 GMT Received: from ph7pr06cu001.outbound.protection.outlook.com (mail-westus3azon11010062.outbound.protection.outlook.com [52.101.201.62]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va997sw-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:05 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IFqyPLgx95pbSD2ctrRxvg45m/5GNnRaJTMwwyzFIHye+Jnc98tzDCmbtolQ8YmKPgjQWpXy1Yewhqrs4jVqQguYra3u8lFvN2TsdlCqMCgd6f/qdXJ28jmzPxSIbdkpskEQYDGzyNQV4rDiKVpY1iIQ99Jyg9Zj3iVSq+Wv0PkBzO/QqBy9GtpSF0JILc7Bz35tsPW4ebDcUudE/gz44ZWjhDFv6ipjZJtlCKh3XbSWfXsXfpwr1o9m+UxSg6tHWQKgHCp0eQXIFgenK+w4ShhOdG8QfWB8aIQMfFCqcZiz3/VrQ7iu7a/t+1cz1sSMg5ZkcwFePiNqxFldKJ4Z5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hnulhczD3XQkqRXiu7lNvxRJz88HMl5Uk5p8sHNK2Ms=; b=lBlIq+42vMIaXIc9DCxRNo0NEByGPGcFjh0hgN5Hh69QQ+BcCiS55AraBunkmjcDIQIYABICToT4D5/lxRxWTh2u9i6wPhVhYBcCNpi7x/lF2uJPBFXZdfIxrit182SJ3pR0l+sljQoqSzLwzDT5yq1XFj5mYRnc0C13aiS7C/Ffu5M0fFIwfPZbs/FOWDXLTrPNtlHTjFzKdBp3cWjCZICOIM+gXJWLUMyeiB9DjWhOcAUgE8yOm3VgZretDu+2Z2ntePBo2Y8rtdusReFKnJ0L4uYOJXWE8PcserVUYa4qRPYsmgpa8zNqNy7BqWELhhN9Fg6pPCQDN+Bxq2k4pg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hnulhczD3XQkqRXiu7lNvxRJz88HMl5Uk5p8sHNK2Ms=; b=IO3AxVl1aEM+cFwwGNW7tJPs2aoKSAD7JbHAsCzmbtJ+O6bTrSTV+vOurYsTe3blZJ67L7LXLMly7c7vlx2e4x8R24pEKK1+4vG7dSJ0rZm6sVyYzbFkpneNI4WnIzcqvhZdFMMIFE95m1pXtqN0ueArhk3sN8Qh5FgVKCO6I/k= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:22:01 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:22:01 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 11/16] mm: introduce pmd_is_huge() and use where appropriate Date: Mon, 10 Nov 2025 22:21:29 +0000 Message-ID: <00f79db3b15293cac8f7040a48d69c52d00117e4.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0115.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:192::12) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: dfcd31a3-d48f-4b0d-ecb2-08de20a791b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?wZaFF6rxV07BFmPjbRMPHioNOMZzSW4k8M7YRnwi4x7PtVaI8xwWqDyNGBNV?= =?us-ascii?Q?j12IFX2KsFS9bGYPJdb9qN85zPhG+iPDgba4jXm26lt6bVA+dU8+3/L6pvd2?= =?us-ascii?Q?v8kUBP396ShYU12+J/V5sRhUNsCM7tcKBfy/5VURP2YSiAMt1uSlKYmAVd3K?= =?us-ascii?Q?GIEYDrfHSMGpQn/m5upvJdm7L5b+4w+rk5M9i1L1tvr6vyBAurZUoquiZE9R?= =?us-ascii?Q?HfZHGT5fOdiX2ulDuvTgQ2rVX8mvF9rVV5vFIwhqxvB1wdHM08RauH9AgPXg?= =?us-ascii?Q?o+slt+vzK5sNg9W146LmPkGET75XQGBnOCCe88WxdRk3ARaPW15/JldpXodu?= =?us-ascii?Q?cC4h/GmCYuBN5C4gbd5Yf8y3ecGd5vXSfu+1RkedFFP2/JE2ERAVFV9ljLLE?= =?us-ascii?Q?dv4yMzFNb9ljdV8xC+/BZSOwzPIy6s0Hym8OLlhWFjGIivoJylZrfg48Mw62?= =?us-ascii?Q?N3SVgJCqmtT8taWVzI6IlRt0PcFN2GKEIFopd08+WA6Any/1djbmq1aJvFLo?= =?us-ascii?Q?4ZvFoH7hD/uD4LBa+TNy2KLkv+yqKYdIRXXa+HRTDyTF5vlkCiAWfMSiLvoN?= =?us-ascii?Q?GE6oYJQzUKuAAZbz0N4j6u3bv6MeDUXOCsQ0YIhcF0g1zhyZ8IhzXLGS5gR4?= =?us-ascii?Q?OqVo5JAmMkpZMHXDxyD/ZELb2OG+HW5kNk81TuW81Sg6do2UgxeSrVnWn/LS?= =?us-ascii?Q?woHnkTApEk5lo4uOpzH0nudt/zJ/lhJfXrRzjcE+eSF0COcw1i10RB9ViQ3M?= =?us-ascii?Q?j8gKxYYfhcDM4BmoEptG7Qa1bKBM7u30wP0RECRAkzNyF/7xb2ue9aOgtL35?= =?us-ascii?Q?dMtgJ44NURdJsmYtg7YlKJFqDFaC1FLQx7acFqtBhi8XsdKQyZ+4jMT7zAAT?= =?us-ascii?Q?0NuvPCe92VCwwTBWQ1gNUG4i0lN3rdc1qsLZWbeS4Ejacf/emhvSI35vRbxo?= =?us-ascii?Q?Qb2NIsRLHPe9TkBBuw3Sg8oPmMiw4xi2VuObwyJFzLCdyAQGCmV7AdGq5APr?= =?us-ascii?Q?HuJYaXFp+lugUuCS9KEsZ55l79X21iZWT5lCOuus7QffFk4c8H1ln97wXGn6?= =?us-ascii?Q?DLaGZEGnAU0q3IzJGwEhvlhWnJy8CGZUxEsJ1Le3T82W1yPW+tAGponry3L3?= =?us-ascii?Q?+Hz99WwWIdVKIbgxnyv6m/fYnNXx6BevAtyYPSFN71RtEnnI3CYaZvtCLN1n?= =?us-ascii?Q?qlnQ6euLNstZPVjJ/LsHsKwrMv36oXbauS6DhyRQAHOTNFH6WWdf67XLXsLO?= =?us-ascii?Q?/TmrFJ595GRKVUYEopAP22CEQcA45UBk3UdIbFV2iZGMbe3iI6+YqD7ceZoi?= =?us-ascii?Q?5f5jJ07Mv6q3HGaY86+Sl58PqdtSsOHZgMAczBTx1zNZCBAJgXPAgYKOVVPc?= =?us-ascii?Q?7CL8mAnmi2OhF3Uut2PJ43qGf1rB/rosPZ8I/fs3uAEXf/EVrWoSIrmYTryA?= =?us-ascii?Q?kEp7YGCFnQ9h/XqNNl4xheMFN5h35dGA?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/mQF1nqniR/Ay2d2OyZSCZeDqmFPqBdswkxXbsZ4OAh/tiEGAr/HOjmphJrv?= =?us-ascii?Q?Wf7+LB/PRTzYNhoS9n0rerj1d22aS0F/R1gpQzOXvMrCCNOkgWpoMUpI9yM1?= =?us-ascii?Q?kEEq8qbdMQVFKRj+yoiVhGTyP6KbLBPIbyGZ8/fwMs/BpITsOW0d2zLTbATT?= =?us-ascii?Q?qO6AhWaYB81i84cPxHpdPKnna6/r/l75T8GrFCr+IHphsWryWcN6XBNBKL+0?= =?us-ascii?Q?ZX98btx+tKF1Qb6pqNgtxGsd1G5f9tAVQy13OxiQqUF+3+hNngUEI3GbqlUf?= =?us-ascii?Q?d302NqbQIvuvRDiH1ltJzaRUfXwJl9Mttk46eeNExXy4dwZpBIFcMKhxuISQ?= =?us-ascii?Q?8F5IL8JPnxLzSxBEboGVQ58SPqeZDeiYRFNR9JiYMW98DjVXSUh0cow8zPrH?= =?us-ascii?Q?Le33KjkkmHtGMC9gKdpWhdQFDEl5Tcfkd3s4vIiDMrqtsuh8yAt/n0Qozppn?= =?us-ascii?Q?r9voehW+TnVDcPGtcEedD5hgs4uL2GzN/95Cc2jEysXPpu+E/HfILg5orGY1?= =?us-ascii?Q?PzbGM2hub+Vch+KH2oC9zxSwXhKPQ58CEyG/cCuFYGbVEDiNcPZ1hdU6QaWz?= =?us-ascii?Q?enHTeBux8AQPPSi/KmYH4/P4luAluQjCepyxcyxPg9+dwhxipfHx8oWsOUqO?= =?us-ascii?Q?Y/vu/bcI+vsRXNKzNm+HWzZvLw9BM5vxW4gx5iu0X1R+kcNS5cTtqlBZ9ShJ?= =?us-ascii?Q?zmsNwAUpKs0FJvU8N6gPei1951Co4G0umerPWd6J2JAJeuPD5AvNWRELsyBC?= =?us-ascii?Q?9HvBWwje0vnOmWcqiWB74ihN1fw5QP/Fjk5zqFOYQUivGBlJLXSgJ69vgSI7?= =?us-ascii?Q?8ZQa+7JAaoak5ytuhd5bu6hXOKiwTBJuri0Y77yBxcZ4VZ3Rey0UMURZjXYv?= =?us-ascii?Q?y6XMcQHblAMHQQpfPW9Vevr6fnbLzDekb4+YsfT3toqSPLVD4NecgZSAK6NI?= =?us-ascii?Q?v1PmxTy8F1dsKSi4KHknhnYkanjvJgdHbKpL/ER5ZN+0xLyM0NZgqbLU6Uvh?= =?us-ascii?Q?H3lFO1fwV60+QgQoPMGz9OtpS1i6oOsnX5tPFffhL3gxsiSofu7opQhg2uBX?= =?us-ascii?Q?fBFjCp2CoFX449dNRsatuTScwWHZEmYcviCQE1yHD5nuO8WuEQnUUpsNSC0B?= =?us-ascii?Q?egiO+eI1xdeQR/NAQk2jC6OAmrG7VCbx39uDfsXgdVu2EJPTXVJ3PZi5YgKy?= =?us-ascii?Q?gv0SgDDC5haajiS2KdYGdn68+PxLaGjEhV3hGGtjnCJYQ6T6UIUogl1bYRNu?= =?us-ascii?Q?jzyTLQEXUgHEOUN97A9dBENPTQ4ZQi5IrJKY1jql/phI8DjRYJ+h+HyV1hJn?= =?us-ascii?Q?EwOPo+uHs01l4rXqx9NqNheutr/AYc3hVbMO701RKGjiUatdMX3WvoqKcgIO?= =?us-ascii?Q?E827j16UN9O2Hr5Zobiuvyo3K70k2V8ahtEs1rps3guD/wh/Aq+fqDb+Dgq1?= =?us-ascii?Q?HHvq2PY6xXE5nxg1y5ZwAl+V8+tBKPAOP5aJSANMpo+Z8Liob9g7AkeUDwzJ?= =?us-ascii?Q?Y2Irnt6C+Yqb7x16DmBiB1D+21B2B+KpJh6+qvvXhbfePQtbaFv59jcihOqz?= =?us-ascii?Q?HlQ5yfAms9g40q6JWfvfilvojyd69sTKtiPaZN8vMOKObFN9PiR0cf6SLePj?= =?us-ascii?Q?Kw=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: xe4QvGLoDPVLyWHqEPVjlJ6P0gu9/LbDshR+PjD3Cr/UJD0lEECav8+EFw1ZbeBpobpX1WWjz22K36kMTc7C2vLMKtdcx5PGi7BOU2oyIgoNELSqrciplv+cpwG04XkJN/Z9CJVjizSKnSBHTtGoCiznsBNHhHlwfS0WhrF4SkCA6DIWMOlYwiO6tnaS7t2GkVttEDKdJrcUu6qPl/1knEXhWq6zQT8JlHrZlNU9K/6JR7yVghA8bifhVJfs6CK7xf29uhN3CNQJzm9smFVFEvkkw6oTzaGQrG2rIbxmD+H8sJYlgIhBA+Bk/YDvrZv0IhZY9E9eT9cWLDhl85BDUwCzKlb7xRu9QtZYvA9tOc4WAsyIarpZildUpyKgn7DKoCfbClQlggOTUw0o+NuVsuxa9uFJ+o9XGdZlwrqT2QdG5EsOtEb+sgGufvVn2opHJypAvzW+M1Iopeb5d4YJlxbUq91ERMK+ufJPG+nTE37jXyXrmLApmrpWiZqFfxk3OuoCEnbixv9SlYwJCDRFv4Yt3yTSwQyuj5o7XSqUGNAY707PT19SXeOO/QAlyiUnpTopBMOjfvXDRJ7GCfjBDVDDZVrSdayII0DF2FSMsOw= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: dfcd31a3-d48f-4b0d-ecb2-08de20a791b8 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:22:01.0363 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9B80JOquNDtnXzLO5Ct+WIB8oPYga66SeERHfNKmgDIa3qG4My8qCsB3IitlimCD0J8IqqxSaD4qpiIddCf8I/PW8TzbjQhysQLolHWF6Hw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 spamscore=0 mlxscore=0 bulkscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: XoHRleAVpnHArodDsuyc0euT4uEL_Xnd X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4NCBTYWx0ZWRfX9Ygyk9By6Zun I4Vs0do95KbyJpDw6k9qF5b7Ch5qIfz69egadn/6woEATDydOQl6DirA9uHMbnPvMZ+sl32e+sG mVzhjqvYg0RWQamNQDUO1RkIHdcUPHxAtXFWenwIvfCWzhr9Uvz56GCmPhjTKEzWvRx2DBijrJH dhoV1grtUWG6fAu1KwfbKyY0HtRofMAy3VO2knoWNRgIC9Z1yWdVwrQh2/LANO4adVpq1sthDSP uwaDkP/JBmx699ggvL9+YeN3bOjp3aR4paBQlYZfVlVmMtxcWqFGIvu7IZV0k/OQHvxsKmb2Gfb Y6Dt37BRBpPY9kW+wPgptnIafrH1Bs7MaZtgvpfpwg7WKZTf19yFwFFxGTzru03uaCLLF/QLh10 5N4SuZnwof1dMMBEeLKMGSpUE0SgDg== X-Proofpoint-ORIG-GUID: XoHRleAVpnHArodDsuyc0euT4uEL_Xnd X-Authority-Analysis: v=2.4 cv=YN+SCBGx c=1 sm=1 tr=0 ts=6912658e cx=c_pps a=XiAAW1AwiKB2Y8Wsi+sD2Q==:117 a=XiAAW1AwiKB2Y8Wsi+sD2Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=Z_ABK6pF82JkGIUUY4QA:9 a=UhEZJTgQB8St2RibIkdl:22 a=Z5ABNNGmrOfJ6cZ5bIyy:22 a=QOGEsqRv6VhmHaoFNykA:22 Content-Type: text/plain; charset="utf-8" The leaf entry PMD case is confusing as only migration entries and device private entries are valid at PMD level, not true swap entries. We repeatedly perform checks of the form is_swap_pmd() || pmd_trans_huge() which is itself confusing - it implies that leaf entries at PMD level exist and are different from huge entries. Address this confusion by introduced pmd_is_huge() which checks for either case. Sadly due to header dependency issues (huge_mm.h is included very early on in headers and cannot really rely on much else) we cannot use pmd_is_valid_softleaf() here. However since these are the only valid, handled cases the function is still achieving what it intends to do. We then replace all instances of is_swap_pmd() || pmd_trans_huge() with pmd_is_huge() invocations and adjust logic accordingly to accommodate this. No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- include/linux/huge_mm.h | 39 +++++++++++++++++++++++++++++++++++---- include/linux/swapops.h | 6 ++++++ mm/huge_memory.c | 3 ++- mm/memory.c | 4 ++-- mm/mprotect.c | 2 +- mm/mremap.c | 2 +- 6 files changed, 47 insertions(+), 9 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index cbb2243f8e56..403e13009631 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -419,10 +419,36 @@ void reparent_deferred_split_queue(struct mem_cgroup = *memcg); void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, unsigned long address, bool freeze); =20 +/** + * pmd_is_huge() - Is this PMD either a huge PMD entry or a software leaf = entry? + * @pmd: The PMD to check. + * + * A huge PMD entry is a non-empty entry which is present and marked huge = or a + * software leaf entry. This check be performed without the appropriate lo= cks + * held, in which case the condition should be rechecked after they are + * acquired. + * + * Returns: true if this PMD is huge, false otherwise. + */ +static inline bool pmd_is_huge(pmd_t pmd) +{ + if (pmd_present(pmd)) { + return pmd_trans_huge(pmd); + } else if (!pmd_none(pmd)) { + /* + * Non-present PMDs must be valid huge non-present entries. We + * cannot assert that here due to header dependency issues. + */ + return true; + } + + return false; +} + #define split_huge_pmd(__vma, __pmd, __address) \ do { \ pmd_t *____pmd =3D (__pmd); \ - if (is_swap_pmd(*____pmd) || pmd_trans_huge(*____pmd)) \ + if (pmd_is_huge(*____pmd)) \ __split_huge_pmd(__vma, __pmd, __address, \ false); \ } while (0) @@ -469,10 +495,10 @@ static inline int is_swap_pmd(pmd_t pmd) static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { - if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd)) + if (pmd_is_huge(*pmd)) return __pmd_trans_huge_lock(pmd, vma); - else - return NULL; + + return NULL; } static inline spinlock_t *pud_trans_huge_lock(pud_t *pud, struct vm_area_struct *vma) @@ -744,6 +770,11 @@ static inline struct folio *get_persistent_huge_zero_f= olio(void) { return NULL; } + +static inline bool pmd_is_huge(pmd_t pmd) +{ + return false; +} #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ =20 static inline int split_folio_to_list_to_order(struct folio *folio, diff --git a/include/linux/swapops.h b/include/linux/swapops.h index f1277647262d..41cfc6d59054 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -471,6 +471,12 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) } =20 #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ +static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvm= w, + struct page *page) +{ + BUILD_BUG(); +} + static inline void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 5876595b00d5..2f0bdc987596 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2810,8 +2810,9 @@ int move_pages_huge_pmd(struct mm_struct *mm, pmd_t *= dst_pmd, pmd_t *src_pmd, pm spinlock_t *__pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { spinlock_t *ptl; + ptl =3D pmd_lock(vma->vm_mm, pmd); - if (likely(is_swap_pmd(*pmd) || pmd_trans_huge(*pmd))) + if (likely(pmd_is_huge(*pmd))) return ptl; spin_unlock(ptl); return NULL; diff --git a/mm/memory.c b/mm/memory.c index bf2bbd0dbc97..087f31a291b4 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1374,7 +1374,7 @@ copy_pmd_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, src_pmd =3D pmd_offset(src_pud, addr); do { next =3D pmd_addr_end(addr, end); - if (is_swap_pmd(*src_pmd) || pmd_trans_huge(*src_pmd)) { + if (pmd_is_huge(*src_pmd)) { int err; =20 VM_BUG_ON_VMA(next-addr !=3D HPAGE_PMD_SIZE, src_vma); @@ -1923,7 +1923,7 @@ static inline unsigned long zap_pmd_range(struct mmu_= gather *tlb, pmd =3D pmd_offset(pud, addr); do { next =3D pmd_addr_end(addr, end); - if (is_swap_pmd(*pmd) || pmd_trans_huge(*pmd)) { + if (pmd_is_huge(*pmd)) { if (next - addr !=3D HPAGE_PMD_SIZE) __split_huge_pmd(vma, pmd, addr, false); else if (zap_huge_pmd(tlb, vma, pmd, addr)) { diff --git a/mm/mprotect.c b/mm/mprotect.c index a3e360a8cdec..ab014ce17f9c 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -474,7 +474,7 @@ static inline long change_pmd_range(struct mmu_gather *= tlb, goto next; =20 _pmd =3D pmdp_get_lockless(pmd); - if (is_swap_pmd(_pmd) || pmd_trans_huge(_pmd)) { + if (pmd_is_huge(_pmd)) { if ((next - addr !=3D HPAGE_PMD_SIZE) || pgtable_split_needed(vma, cp_flags)) { __split_huge_pmd(vma, pmd, addr, false); diff --git a/mm/mremap.c b/mm/mremap.c index 62b6827abacf..fdb0485ede74 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -850,7 +850,7 @@ unsigned long move_page_tables(struct pagetable_move_co= ntrol *pmc) if (!new_pmd) break; again: - if (is_swap_pmd(*old_pmd) || pmd_trans_huge(*old_pmd)) { + if (pmd_is_huge(*old_pmd)) { if (extent =3D=3D HPAGE_PMD_SIZE && move_pgt_entry(pmc, HPAGE_PMD, old_pmd, new_pmd)) continue; --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCCAB3328F3; Mon, 10 Nov 2025 22:24:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813455; cv=fail; b=X7Ht/YFHYa+ZsslYaLqZPWJxqsvUrFWod/kFoIbn10F2lUo58sUc9kD6RUJsfIHJdBg7wCsmgJscZkorJlxRM9oE/Ct5j0QpuKsdrpUiEwZTQliIdCYskkR9TzYuNKY9WyG9CuD1RlGHRgsmWHXTs95K+PQe4ptjIKM7lYXzL7Y= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813455; c=relaxed/simple; bh=dJwfRNiI3WXPCdC9hAF5POoUcmBmxlnAw611TSol11U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=KFUBkzPFgYdBn4UjXoAWZJIZq8TqK5rEudPwtwtjzl3ZWcOUn2AjioclURgKTW09VPvreKvCh/SAvqr3L/nXDkayYq3X7MiADR3egst+Dv/LbRKOPSHPnxjpy+BV4QbqGW6j9kETDFPOwQYiGyaxkQY6+lxjzBLo9ooDuph4anc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=NWbbg7Hc; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=ddh4QU5+; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="NWbbg7Hc"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="ddh4QU5+" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAL8hmT009313; Mon, 10 Nov 2025 22:22:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=3biM3jPk/T84wrFDEkrtg6nQEHWQ6QNNYccvVkfwbPk=; b= NWbbg7HcPQWOnT6F5UoUgKu89jQEyVhbCpp8h3tksM9jUms5OjqPfu+KWJ4txLTG 3rxCaU4BVMY9uSVbq7QO/5fgnes/r1a+pUbHFKE5+k+jgBT6T7cEXfaGtAb29Yy6 pOJc937cT5HC5HFX1kiiTjMp8+hsjd+780cUa3xkJNKOBannkMX8nf6ECNK/OClo slvOH3qNWf3HOpcyfGsxpu0QeuXXEgTF4CWmefAEqsGAK0xNVss5me7BDsE/zC1c tOsK4lruhRc2bBvuOEzeoldIVAaSI9u2UmE/WbiYvk39tJFV5rQcocECq4oDdtcx Poq7x34mUpXZvTlx9JzmUA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abqpug5sv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:09 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAK9m0m040038; Mon, 10 Nov 2025 22:22:08 GMT Received: from co1pr03cu002.outbound.protection.outlook.com (mail-westus2azon11010055.outbound.protection.outlook.com [52.101.46.55]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8ryxq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XF4D6ItagaXh9hOyDOtHnI3p3Bx6g4qclhnZw0GN6BdZaSDPPrYSnc3VTy+vUjarIelIiQB6sDWGChamd8UyKmH2GMfm+1wNFFYuUwBPZiM7NxKATaBFi3qIndDhdCd8Abo3pTy4HeAmYeBTJb9PxozCr7JkfG2uVWiQ/YRMnJnoOV8WBXq4FW/goKQENIKyfg7oSEQA2uHFRQ51NTLEmsrRJIA64D02aWPfXGQs81MAK8NQYVr9CVZ0D5RNRR15o7wAtn4yt5ehjR/Ss7HO23+jPVaHCfw+wIevNb0i5j9ryLOXY9URURYVQ8MfbXDiFS1fha3bJX7402tlXmUQjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3biM3jPk/T84wrFDEkrtg6nQEHWQ6QNNYccvVkfwbPk=; b=WvXEEKfrau6M5CGN/WT1NgTbfu8wt5dEjqLRK+dlQ1tGXq+QqaVPuuPt6w+4dT4+ZYtVxkIsY/vpSPO2pGJmI3EZYMeYLbL/JpTPPjkUMJENqODo6W/e0W6SZH2Wmipo2y11Zpy+uwoypNUSrbVVzJg45jyCcjcTJwMrTL11bjzMcxBCpUs+RXtkiBKhRYdFKxKThAei+XDbuwJvuc2HUgoPCNGCOL3l2nmv+n5r7t5uv8z6rlI6aw/uw30SUDjXbVP+CIM/dpcDNMviO/2TlJtsxHfXp+lpG02U+BpEuGWOY0ZLbd5BMnJWhwc41lf5e1Un5Ld2R9yPd4IB7Cq1mA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3biM3jPk/T84wrFDEkrtg6nQEHWQ6QNNYccvVkfwbPk=; b=ddh4QU5+R2plnbhbRvAcWCYP1M/UEjLylOt770/VQ/o30ja8HOZAzfdMkpgC1RM2QAFFHVWOjj5fE/cngpQy47YjSwde3AJAfzbQy6OqEeiaa7OPFAJx2JYKsmYrph3XuLG5W5+HJl1fPbalUPUy5AFMsB75ZER736uavUYXnDE= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:22:03 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:22:03 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 12/16] mm: remove remaining is_swap_pmd() users and is_swap_pmd() Date: Mon, 10 Nov 2025 22:21:30 +0000 Message-ID: <1628b00b00c8498bbd2c20b82117ee87845fb738.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0486.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:13a::11) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d2b8b58-206a-495b-86df-08de20a792fb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?1xttGCUXK3m7os1+4FS61OFZittB+ieABCNCUBTh1oR2kEUVEZwVRJG6fHAZ?= =?us-ascii?Q?/43xTY6+YPHZKnWK8QW+GHa9iCxpyEWOSjzQEcV0p6dtNbNIXrjnwJeIRkjL?= =?us-ascii?Q?+ao59JqhX7Y0pS44d5mtc6W6XH+74YuIJCC8RUB/mMmoMoj6BGoRMLTWCMme?= =?us-ascii?Q?Pi5ulVGloMO97ojsQcUQCwzgG/FvWy7M0dTuvHFFg29xnFXa/txvVivWSwN9?= =?us-ascii?Q?UtVEU+RHcGK4SPYrbuRcOEyieQGNupL/N5KjekCESuivywTHjUuVg0jD9lQT?= =?us-ascii?Q?ofpg71Va3sj3GUtY2HU9IqpiD4Zgn9EpWnoEGONJJpi1guYbwMV8RbLXEWbm?= =?us-ascii?Q?wlBlVlK7YVEIEUwaRL/5A9HsNXa0zylQpzAtwShzXjt2P4HtbwNgjAriPipe?= =?us-ascii?Q?XNbV0ZEyCJh6H+lFSCCGcxGTz0ugwmRr9JzAMRdLRQiw854dtUwBvboQGW6X?= =?us-ascii?Q?nRWZrfwbxBfyMbWdN6dS24kXXAlWc0nPa9jNmUodfbnZKEF8JHXUgVRoXSV6?= =?us-ascii?Q?MAgXtor472EKXZ5wt5UMzuT6gGa5gi0UM9M7xsUYlrfYWgiTYZnZxAlf0unS?= =?us-ascii?Q?uzG9PvM1Ou7qLUp3RDhblkw4fQ9lAf5Wl3ZwP2Kz4R0bccE5zOq817RC0XVm?= =?us-ascii?Q?vv1hMqIaOQPa9I4i9T0yek37NGAohDxPyPSBT3z5ezP7TQVGR1AxY+PxDZkz?= =?us-ascii?Q?7XG/PfzknyJ6C/xgWGWWARoDcujRlcSET3LbYzc6D8HvlA8oPPBKcz7C4v4H?= =?us-ascii?Q?/RGJgyQjK96/iZey0phtuaZ2tTRwX2xKQ/Ek4glbly243PYuSQcBc9rVoe/3?= =?us-ascii?Q?iH+Z54xfOvk4JGvKrle3vOTHq+4V12TACL2mSGbwUtFR9XJIoBzYS1U/Hx9Q?= =?us-ascii?Q?Ywc47gIn8KWxIeuGhYkYoI1s45s0LghmZtAKcoaVmVdaT8PFbnuaEjCVus7e?= =?us-ascii?Q?FPROEXcXDK+KJWyZZaVuwqjA8rmH89xPD0MrNI5Yazk3K6/j1qXBEtoHMLpA?= =?us-ascii?Q?x7LOXoGPDoGaMlus922O+08KrMoVRMTB4BoNV3i38vPHj96BnHOK6sUKadD4?= =?us-ascii?Q?9M6ElMqncr1H8F9leb+tQ5GBgwVClSuZKchW5orz3z1t6TRnQfo6t9UCFBS7?= =?us-ascii?Q?K291tQBETs4Az69TqQzWCLa3yCRLlAq+xh4j6FvaX2dZMQEDhGw+cSwgmx1i?= =?us-ascii?Q?JGA0wVrtnrojmNaDqIiwS+dPoz0qQtwgL7F5Tixvxd1gND2sBZ0/HfFwf2Aa?= =?us-ascii?Q?1jOAOAyzMFKBE7oheGwmP4QYZ/TSmZ73b4lN3pmjTMZHzLctX9TsGWylY64+?= =?us-ascii?Q?xZx5pSsNRZUc0Xc6g3OvfCh43Mm/GpF9PUC6vdQKaksEcNa2PzwxtFHTM6vs?= =?us-ascii?Q?lk4tW2prTgJOQz+TfqeKZ1UPuRfaKtaKPesWSsSfpaagtcmliEfoQomMIW7r?= =?us-ascii?Q?5skT/LRqDcgeVwBIWivCp8jkup4Tjcff?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?gZzCXysvSXuLNysxKYiELlczyrpxDiyAQyuSi9/FjgFBymkqHcBbPJ6vcAf/?= =?us-ascii?Q?8FoZZfSBNQGOkEstcJNfFqZHhpa0P7UU0PXQLPJ9AFLkhEj5+h/GUTv3Yobk?= =?us-ascii?Q?Af8T7mylmlswGlyLT8/mx92Yl5QYzwy7nnGYs/HUgzMybZXShyt9Hd91/c5X?= =?us-ascii?Q?89fxOWf5nq5hImboo/SMiH+gdvt2oaTvvT3u/PjVNO3sHS46sA2dnkRV0VVN?= =?us-ascii?Q?D57gqi7OKAjmR0DvKLEYecXktvG+P1jS6wH7o3+YHGFIuiUMMBn4fNXZmcxD?= =?us-ascii?Q?kk2P7wMLiXDrtE5bl90FXmE+ALSXQEc3U/+Y8mPqoV7xLvUZFXTbcYR1rIzb?= =?us-ascii?Q?GaPVmTxj90XvuOmoYpCOjU0H21tVo4dXvJo0ZsKK4AwHIFl1Z5j7JNqx7pYh?= =?us-ascii?Q?KoBx4DZ8/rTFuU3LkvEKWL/4sidk0pFYUkhVaKN4phqXhhFH4lITKWgnzYg2?= =?us-ascii?Q?0QTNhCCciCeqvdOQzA5eY9gbU9icuz+71NolVhO+je1ONTObkHJmLSec3ZUB?= =?us-ascii?Q?xZ+YPSA0/0928TF885NLe+ElZK1dheCTEWmOw7B/kbwufmYOeDm5ZcyZEOcj?= =?us-ascii?Q?4/9PZzOJo+oIX0CwcOV7ecC6/ZXUAMQ3QYT1/SYHs+Bw1o3xNIECYunz+D0t?= =?us-ascii?Q?UmTWr2gKJoBqWU5aQN+MFhvq5IQLyOcB6JdpSSUJ+Fybv1rPKUG5HMSeBhKy?= =?us-ascii?Q?t9XAKtyQRSgwf0tgICOUWeOicuzHuuUxV3hhXBxduBAj0zLhAnQ8vxusEWOs?= =?us-ascii?Q?3TZgs532xXS4A/v2eoAVYB1p4x7JqN2MJfQkI5+vc/VguQHVyHZN58Mi1Yi6?= =?us-ascii?Q?o0dAgA5N1YNrTFxwmfBwKgL4HTw3x57QJsHV3MFBFKMZODpJk8Fr+PXyrNZ0?= =?us-ascii?Q?69xf2Ts7kkTxL6cmBj2i1uCLWvgiJhWvW6EnSkSeueAD3OfbLEvnbOjJ1gB5?= =?us-ascii?Q?l71/Y59bNKVQ2XjD4St5eHqRdC06xVl3BUDZGAmAqMqhwTt19bh8Z9iINhpd?= =?us-ascii?Q?YMVFqSfGAseQDjh5sX/esR1T//wilS9aPm/H0FUUqENl8awzY8i0wC6EdLiP?= =?us-ascii?Q?412r0ea0GNq6NhCMrIU60j8sYTHuyHPJgc9zhH639acl8LAYls07UXsMCM85?= =?us-ascii?Q?olU8o2Tg2n5HVp8BnXkyr4f1qF0CoPJ2IHtVxLwzqVA334RiY+u9ryG1HZpK?= =?us-ascii?Q?cTWh3hQBrpljAfoktqlypvIBR7+D9w6fpWDqtXpy8biMu64zy2qZV9nRfpAV?= =?us-ascii?Q?nt80y1m1dQBlrrsnTifGVrKf3Y0i3ryjTYMQpuaASoaxZko82RhLRv6lTqR2?= =?us-ascii?Q?7KJP7xOKPKs9VwZNxz23ROlQExEMxciJYr6tsI9BucaJNG9M13QVV5D8yZGz?= =?us-ascii?Q?VHGj40Ujs1JMzoEc3YpqO1ynR5onVkwa7QPyHbQJKnbSb+BNJYtnkH5/YTdX?= =?us-ascii?Q?O3EhuwpL/JXoSjRPoA1jNGDfapEULL+zuHoUmhWA6X8/QB++8M35G5pAx7LE?= =?us-ascii?Q?TbQHsTD5HoTmvUKVSGtiwPGMuV/KrANga+kBuP/gsLfCdBp6IjplwriUz+BW?= =?us-ascii?Q?tAtMGcfYC7wkh6TgYVabuBhN8ZwV3dAHlStiHi9uDgmSPO5H0hOYQOR+tctU?= =?us-ascii?Q?FA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 2/WlQcONr/M7bedLoCeUwG0hPDlT3tzV6DEctmr4C16INEOsTJXuR6tYWpUQ6a/RvyXAlBnn0xjjhzPxur9QxijonXWp6oumuoTCIlYM5YqGZTIibfMnGXaIbkgJOR7PCecYiOtLOckpLpYtJkti71SZzZb0NgthHmmGSa/fc3NilWwg+711vbqVUroAvIk+W4QA/oylAaWVsDpXaQnPAwg33RE/oasuR2YCS0GarYkV67/EruVbQs1QJFhF+3AuAGo3M3pX87hyitHzfdmgmxVtDnk18FVduFir880juoQNVtYcCs5bgaEg5EVHxya2oSBZhJxUs+RbqdtBhNYTC2vtBd8X0G+pmm282M+mGX6pRvlG9AUyUpiePxN0Zuu/cV/SiHp44/t8FhjMhbSVnJy8x53rY8sZM3SUxcGD7X9SdwgwizmUYRC+4/7Y/LMF6jSdSqp6YVrC4F5IaM5Is/S3gXSMpGBOMY2fe/XKkaBXbtVQEYN14WM7W+VhR4HzR7uXtI87cUYY53j/uUQE85XCqjYtH9z24j4TDUlmg5sam3WT7QZuDCE6jyaCFhELWVkgvbYRMHdRG7+FwLSFyGGKGL5nBjW97KCepWv/Axw= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5d2b8b58-206a-495b-86df-08de20a792fb X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:22:03.1397 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IVXll7YBZ2zdv8jjWxLPYF3EcqbyKJl40EnLvFHhynYR4FmTJpVZS3bXxCZgf8a30clRfub1F1mdYNLzZuSmN+bUM0ePHC1gHfjkIToPsUQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE3OSBTYWx0ZWRfX/gt0jNyXG1g9 +ycAB6ZwYLy5UYCwy7EQ8+qN4JyuofDBIPQBiyxR4yOsDD0ip9fjvP7aM1kfeIlJuf0V1zs/o2R QBRO/M2naEghIzzE3XmF2TMskFHvG/jjOm63CqpAqgGZ0cNmHG4z2qrw64nOARhI1kw3MvTcV6j XPhn/c127LQ5thafUux2yBPLawLoVBV6ieTKwXYA9e5df+YNZTME15ybq3rWDwGzWuj51I1lW5g cmlOoA4SLFSOiY+qduWZLSePYPIlOZn69+da69W62GBDf0KSQO7/euXJJyJeZM0TKdfvgrq0X2E W4INggvTWCD4vGqw7u+QwWNJIbGQo7ptplxTWUu52sKnivivF57AnFUz7I9RKgJzZvP3GDxktTG cUS3eX0iZuGKZeIdisDkjJmu2G0j6A== X-Authority-Analysis: v=2.4 cv=H5rWAuYi c=1 sm=1 tr=0 ts=69126591 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=xbP9_UvFYPnkkC6NLJ8A:9 X-Proofpoint-ORIG-GUID: doCR_rOvSkfIp61ccpOnZMngjov1HWev X-Proofpoint-GUID: doCR_rOvSkfIp61ccpOnZMngjov1HWev Content-Type: text/plain; charset="utf-8" Update copy_huge_pmd() and change_huge_pmd() to use pmd_is_valid_softleaf() - as this checks for the only valid non-present huge PMD states. Also update mm/debug_vm_pgtable.c to explicitly test for a valid leaf PMD entry (which it was not before, which was incorrect), and have it test against pmd_is_huge() and pmd_is_valid_softleaf() rather than is_swap_pmd(). With these changes done there are no further users of is_swap_pmd(), so remove it. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- include/linux/huge_mm.h | 9 --------- mm/debug_vm_pgtable.c | 25 +++++++++++++++---------- mm/huge_memory.c | 5 +++-- 3 files changed, 18 insertions(+), 21 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 403e13009631..79f16b5aa5f0 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -486,11 +486,6 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,= unsigned long start, spinlock_t *__pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma); spinlock_t *__pud_trans_huge_lock(pud_t *pud, struct vm_area_struct *vma); =20 -static inline int is_swap_pmd(pmd_t pmd) -{ - return !pmd_none(pmd) && !pmd_present(pmd); -} - /* mmap_lock must be held on entry */ static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) @@ -693,10 +688,6 @@ static inline void vma_adjust_trans_huge(struct vm_are= a_struct *vma, struct vm_area_struct *next) { } -static inline int is_swap_pmd(pmd_t pmd) -{ - return 0; -} static inline spinlock_t *pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) { diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index fff311830959..608d1011ce03 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -74,6 +74,7 @@ struct pgtable_debug_args { unsigned long fixed_pte_pfn; =20 swp_entry_t swp_entry; + swp_entry_t leaf_entry; }; =20 static void __init pte_basic_tests(struct pgtable_debug_args *args, int id= x) @@ -745,7 +746,7 @@ static void __init pmd_soft_dirty_tests(struct pgtable_= debug_args *args) WARN_ON(pmd_soft_dirty(pmd_clear_soft_dirty(pmd))); } =20 -static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) +static void __init pmd_leaf_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { pmd_t pmd; =20 @@ -757,15 +758,16 @@ static void __init pmd_swap_soft_dirty_tests(struct p= gtable_debug_args *args) return; =20 pr_debug("Validating PMD swap soft dirty\n"); - pmd =3D swp_entry_to_pmd(args->swp_entry); - WARN_ON(!is_swap_pmd(pmd)); + pmd =3D swp_entry_to_pmd(args->leaf_entry); + WARN_ON(!pmd_is_huge(pmd)); + WARN_ON(!pmd_is_valid_softleaf(pmd)); =20 WARN_ON(!pmd_swp_soft_dirty(pmd_swp_mksoft_dirty(pmd))); WARN_ON(pmd_swp_soft_dirty(pmd_swp_clear_soft_dirty(pmd))); } #else /* !CONFIG_TRANSPARENT_HUGEPAGE */ static void __init pmd_soft_dirty_tests(struct pgtable_debug_args *args) {= } -static void __init pmd_swap_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { } +static void __init pmd_leaf_soft_dirty_tests(struct pgtable_debug_args *ar= gs) { } #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ =20 static void __init pte_swap_exclusive_tests(struct pgtable_debug_args *arg= s) @@ -818,7 +820,7 @@ static void __init pte_swap_tests(struct pgtable_debug_= args *args) } =20 #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION -static void __init pmd_swap_tests(struct pgtable_debug_args *args) +static void __init pmd_softleaf_tests(struct pgtable_debug_args *args) { swp_entry_t arch_entry; pmd_t pmd1, pmd2; @@ -827,15 +829,16 @@ static void __init pmd_swap_tests(struct pgtable_debu= g_args *args) return; =20 pr_debug("Validating PMD swap\n"); - pmd1 =3D swp_entry_to_pmd(args->swp_entry); - WARN_ON(!is_swap_pmd(pmd1)); + pmd1 =3D swp_entry_to_pmd(args->leaf_entry); + WARN_ON(!pmd_is_huge(pmd1)); + WARN_ON(!pmd_is_valid_softleaf(pmd1)); =20 arch_entry =3D __pmd_to_swp_entry(pmd1); pmd2 =3D __swp_entry_to_pmd(arch_entry); WARN_ON(memcmp(&pmd1, &pmd2, sizeof(pmd1))); } #else /* !CONFIG_ARCH_ENABLE_THP_MIGRATION */ -static void __init pmd_swap_tests(struct pgtable_debug_args *args) { } +static void __init pmd_softleaf_tests(struct pgtable_debug_args *args) { } #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 static void __init swap_migration_tests(struct pgtable_debug_args *args) @@ -1229,6 +1232,8 @@ static int __init init_args(struct pgtable_debug_args= *args) max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); /* Create a swp entry with all possible bits set while still being swap. = */ args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); + /* Create a non-present migration entry. */ + args->leaf_entry =3D make_writable_migration_entry(~0UL); =20 /* * Allocate (huge) pages because some of the tests need to access @@ -1318,12 +1323,12 @@ static int __init debug_vm_pgtable(void) pte_soft_dirty_tests(&args); pmd_soft_dirty_tests(&args); pte_swap_soft_dirty_tests(&args); - pmd_swap_soft_dirty_tests(&args); + pmd_leaf_soft_dirty_tests(&args); =20 pte_swap_exclusive_tests(&args); =20 pte_swap_tests(&args); - pmd_swap_tests(&args); + pmd_softleaf_tests(&args); =20 swap_migration_tests(&args); =20 diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2f0bdc987596..d1a5c5f01d94 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1875,7 +1875,8 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm= _struct *src_mm, ret =3D -EAGAIN; pmd =3D *src_pmd; =20 - if (unlikely(thp_migration_supported() && is_swap_pmd(pmd))) { + if (unlikely(thp_migration_supported() && + pmd_is_valid_softleaf(pmd))) { copy_huge_non_present_pmd(dst_mm, src_mm, dst_pmd, src_pmd, addr, dst_vma, src_vma, pmd, pgtable); ret =3D 0; @@ -2562,7 +2563,7 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm= _area_struct *vma, if (!ptl) return 0; =20 - if (thp_migration_supported() && is_swap_pmd(*pmd)) { + if (thp_migration_supported() && pmd_is_valid_softleaf(*pmd)) { change_non_present_huge_pmd(mm, addr, pmd, uffd_wp, uffd_wp_resolve); goto unlock; --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 584743446AD; Mon, 10 Nov 2025 22:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813466; cv=fail; b=VwTb/RUbBzdDyO7WubrEAUrDViw4Iq5oNmE2UzvsarecFvH/q9YMD9YX3KDbZWBhBJz4c5l3K9ruvEYFMxWeqliLpwIkcvMYwgJki+HAt0g21kRMKHj/BluV/Zb0F7Ph6bT73xl5PNdVqiICVbTL8Ck16hxu3W2VRzFhJMHeYDY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813466; c=relaxed/simple; bh=UVzI0162fEkE8W4HuwqsQyiwBgaQfv8gaKgbWm5zR54=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=UX1PDPzIHui1GnxQ+yOZ4QOc1wppooeRbclez1IDjzzCqDFXHVXLCwRsk55De2r1RNtNNPdh1I1OqZEvGsXG12iyArhiiOLsWi/NLg0/wZAkQ547cHWIrB0vjKNBuz2VCTTgge2JxkXOrynU/gXsN8oQwEOaYR4Oq7xLhU1cVmQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=pqAskCi1; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=S8yk0cUk; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="pqAskCi1"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="S8yk0cUk" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAMIJlg025921; Mon, 10 Nov 2025 22:22:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=qu4T7fD6pb7n42lUuHwrGUGNaZkm34HScexd2cq7FIE=; b= pqAskCi18J4tEzwKO6fuGmoFPYKuMgdyuUjIE65WoRcaTrH/rVCVmJ7UxFSdZNWB tUbBmZ5nts9H5CCtE95toGMwCe5p5hmnQCf/pzqd+gGV/p6I85Q7MH04ql/xnpAB F18n0Wkxs+UzKFQ3wS/vg2MWHdwZgEXWUYKZA5WfucWqMU1o8OuzejahpHSocVC8 VpGeIaUPzAxvTGLekdX/z1mTRaAHZb4Cy3KcSzoSzQebgq8NYCxhGBpPR6Dr3yNL 6Ri8ljzjc3iV3cf2y1mUTMB80eD4EldmLRSpXXSu0tYvtYa4UR/BvPVCPz3TTJFh 8G/hPVKnIFXlREP7r7aI8Q== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abr5vr2ct-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:11 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAK9m0o040038; Mon, 10 Nov 2025 22:22:10 GMT Received: from co1pr03cu002.outbound.protection.outlook.com (mail-westus2azon11010055.outbound.protection.outlook.com [52.101.46.55]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8ryxq-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=d4IcOk+hxmWxzb7Rs69/u0WFeLySRcUs9zdWfJdyb/UT0rUtVACJjW/YK1fgdXpyoWywuxU4M2/u2LzT/bbaq+Q0Df76tJOE59p3gSFiZQxq1oiRGItTmigXHpjxV2H1xfTBKuHGbbH8sAxpwiHv1Tc6IGGAw4Gp08u2XLAXJ5Rn0UBoPyd0cl/+UciRsaEPe56RrirZUxksJmOIpZzi77FRwJ+9HNdEpZwaXE7VPRaLYY1MNrOLxcWhqXATDul6XnN0D8U/pIQzowT+hgGiPuplNHgGd1bEVr1SnW5xgLE59Plvs03TD5F/BMa+mqdPiraC6w+IImT95GCpDEuc9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qu4T7fD6pb7n42lUuHwrGUGNaZkm34HScexd2cq7FIE=; b=V8h3YKsI3JZZ/sTD4sXzcli9yRTcORngBG6KLX2P9fliQhvpZ1yWVbRZ9Q5yii7kGjPxSyLHl0JmrsB15RpodlvIM7VipirQh1ZzMUWA8Ja6ru8iC63qM74+i3GQ8FdPy4uEUuUjv5om0sCvhHkJEAefYyPZOZUFFHg0HHnmFEqnS+tPA2e6wiq5UN8GWw3XjYF3ZOBngzroTlBS4G9JyIq5PqzW7neQOFvhcm/TjJKxJ/6jrpvtiRmp2PuByVP9eeQdAGxQJ1bdRIW+T5iNFT4S/Ov1xdu412w1+3pxF7yU7xptew2x+wR6dDF2l+7P8GBOSC8lGN+bYw4Gcx/x1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qu4T7fD6pb7n42lUuHwrGUGNaZkm34HScexd2cq7FIE=; b=S8yk0cUkesD5yXgQEt/Mh68R4eq3DQXPshVh556a7zf5EFqxkRPoTZG44/qK5MMoCKcYHUi8n8AVcOeKnca2PPIhixHsED85q++VDw0Bi1l1ImV4QWBh/1o63hVMLVFszXeK9mAUyyUZtBq5zE5bLXoT23WGPQLmHOsznotYhhs= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:22:05 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:22:05 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 13/16] mm: remove non_swap_entry() and use softleaf helpers instead Date: Mon, 10 Nov 2025 22:21:31 +0000 Message-ID: <2562093f37f4a9cffea0447058014485eb50aaaf.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P123CA0118.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:192::15) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 67da5d85-357e-4f90-6a07-08de20a79465 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?boSUb5z4irPYLA7tF4WpXiTvTvU6rUr5W7nxQ4Cd21LuBk/YO4Wnb1t6mrob?= =?us-ascii?Q?MnvmWaVuumgmpK8o0zE+nLO6CKrkswfx1BDQb8j2KS5xL2CZcYmCjBQWIGWy?= =?us-ascii?Q?KivdyjV1kTeEh00kcAeO7tB31rNAetHDn8azO7EMu3flcbaWj6/ll/HaXe7h?= =?us-ascii?Q?KOm5Hb/2ZtKhhgSetWz+otDGqeXWyc/TRn1R1JkQ8Pn96u5RnJPoFnqTznWF?= =?us-ascii?Q?LqrqIhI/DrH9/WXOkvMJRcyO9HZpUAXdhVRXnJHWOzIGA4dvV7NHzLaYPOPV?= =?us-ascii?Q?ubNmSxYLgx56sILfGhzPQncGzxkvvfZJRmNFamq03vjAhM2HGPpCg6Na5TFm?= =?us-ascii?Q?Da0Xpmhjl6H2RKc/uwsD3/3Bc8WMvC6H5AE1n0LVErsw5oZzlakTeFXK3Dtd?= =?us-ascii?Q?aaK2mfdbssxnHJac5umeGsu//cjeZYzr64zfXd869Zb9pK/5g96XCnKrnCH5?= =?us-ascii?Q?NpBU59XeBXMbXsVT4t30MOJRCelp/Id9dmHeD7HdTutX2OceQI1nrhAkCEsF?= =?us-ascii?Q?SFfYzK3JhNYZjUNLPfxceokcFmMU+ue/FBH0MrXa+6YVxUeSc0BNS3km1dua?= =?us-ascii?Q?gWQ+k/ZwU7kteb9JAhhuGjb4ss6aXMwA1qe+pYXkruLhmYjCnjeTGSz20B2a?= =?us-ascii?Q?yCimYlrkq1lFNvXn+q7cTiVCUOmGaq1ZA1qEPN7XynsIfSe/ffuDSKd2Epw/?= =?us-ascii?Q?XJeA6byMeEY1HfC8hTdQzsBql448Rz/EtI6Nch8wCEZp74BKrGVTe3hQsJ3A?= =?us-ascii?Q?U4mC+bHJeUJLfvvayVNM8fFQCdVFQIVy1/JhCvXCC7ODgK2uAJYTQqG2YI84?= =?us-ascii?Q?Gc7w2xO2tqe/D0SqrjMAr8/5PzqCLAmXu7X67Cx9b1K3k3GdvxZM3oCxe7wk?= =?us-ascii?Q?ONrOv02vY9nUCjSjny8Mkyo1qLjMHA8dzwr+p2a4D6mvlDyMIYQFOUbtgiSj?= =?us-ascii?Q?ojSJDbPQ/NAkH1oxp1rOlVcEsbo561yd6kQ+Qxxc7xow5XRrrVYxeOKV78OS?= =?us-ascii?Q?fI7sMHgK46z7PhdXqN7kO6l7q3Ac1kSC5KPk79h6gkWJZTVYFFYicynqQxTg?= =?us-ascii?Q?DTf/ORcmfPSX1e9P0z8XzdRuXDuLI1zSaOQDKdWO/jIVCdo74/giL3lor5Y5?= =?us-ascii?Q?FI41z2fQ4SvgM6u0jAaVi6ioMmzH1ODXVIO6aBhH9ll/hZS4ScO26JjS5dk5?= =?us-ascii?Q?Y11t/zb9nUNy9NNl6qWxEEb/65BIBKB5OuE/I+MYpI85BKp0MKqByl/gQQzQ?= =?us-ascii?Q?rmpRwr+tRHWw31/23c/rmT+jgnsAIEgTJkdF9cSUvMV5U4X3tkQRkEp96bpv?= =?us-ascii?Q?760oWg32ZXr5uHvvpHFb8FLXjo5ImHrs4mLcqbmKzRS+xLtkL4Va+K0pdkep?= =?us-ascii?Q?Aw9uGs1ASK9jN70nDWadK4mW5AcAVt20TVSkscl5X2qHkIfudUplf5KwTKL7?= =?us-ascii?Q?Of2CyZ/Bsvm1GzOScdAHzl4JPIFCii+K?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?7zswWGx/t35SEJIln2nJ459NeQZINV67g5qfsiruh+k0O4GvcZxEXQaWjWp5?= =?us-ascii?Q?FxEEJ+6P7iKsdcpeBIY+0pWiwwRQVcvn1IyBrJvTqomeMz7J+DYn0rAzHuJT?= =?us-ascii?Q?h764muAJBxGJou7Od+Fkyyfacglgsuk1KzVH3T1YH88Gk3iU3gl9fKgSOg6W?= =?us-ascii?Q?yhtBVcQ9DJOfQQvdZPaP4dFJJ8uEN4CDynkvkdJB4e+QUit6bH6uHjI4VgFr?= =?us-ascii?Q?aPcmTRb8/4WN8kQSGQUbZViwvg1bUau4NznxD21fOsPLLkb1Hw93J3L6Mh80?= =?us-ascii?Q?pSKOZ92qmB2AL+8+7I1RtZe1Fmzxzujmp2FikouZYjf9P6NwBor+aR9f0WwT?= =?us-ascii?Q?p5kK2U0wC4xID+8kNcPfUX3zXBHVHwa5YvPqPnofWVZerR16qEyDtlGc/awO?= =?us-ascii?Q?l/wuTwZfKlFGEW8OTebDcMKPNLb+NUFEANk51nt9EmkQLFvR44gbMlGxAHnj?= =?us-ascii?Q?5x8PHOy2Qmnx2zFNtXz4KE0EOGU56zwVAsnt3svTTwHOJzpdInh6D64S2wM7?= =?us-ascii?Q?f59SzDGTISz2eIrcYl8tkVkUO3JgX/DWCqWSataa6LlUEQ0BfLaBWyJNDbkB?= =?us-ascii?Q?CODfxvy1A0xs605wZ5LvoZnLHGap+jXDnfUYkjjNpyiIpS3ZypEJy7SMfC2J?= =?us-ascii?Q?eYxa29lycl/3bZ8H96jDVZEy/BxcYx7nBoMMLnkrARWbqQBFVSRTl38d8rHn?= =?us-ascii?Q?r5YXb06nK1kc0HOuy/wzYQ8y/VmYgNRgtR6g8bSdYgyBUU6KyTg2XL9WR/w3?= =?us-ascii?Q?gsdPqLnSsYuV9TwQLoGDCllxPh4PNWEjcjoKxH2BsUY7qr4kGdbgARfTn2BN?= =?us-ascii?Q?jld9Onnxj7xJGhTBpiuLbVukG7m+JOi2kw4F8BxDpwaw09XHQGaDleykV75K?= =?us-ascii?Q?vE+7je+RXAijqh+aY0mIhZNFIsbwcKudWkli8IvJ3txxfrjxrdVH3NYKXZRl?= =?us-ascii?Q?VMqztdLAV6yne17a6jYjQUa1EoyORRbPXvU6P2NQX2ogC9GwlUE7KdW/HYYF?= =?us-ascii?Q?JXJz3J2M+sy8+zkaLSavKd/EBPORHcl+cGtkisfuLgw2yyhaZSVA6hMJOPPg?= =?us-ascii?Q?RWEEx9ZX7tZCAgBNKeFLteg0J4ftAu2zU0kDIllyFAII9B+sw0F82U+fMDgT?= =?us-ascii?Q?deRkVIcg44uOA6yRksvL7PKNKVMdqN4oUNKTn653qxSt9+cKzwlaLidW9aAH?= =?us-ascii?Q?5Cuz8y8JdFg5aalMiJgRTNO4DPNxzS3MFHJSVs4XeE/LIsnhf5soIDN19uxF?= =?us-ascii?Q?PZO3S1ablhPEAzcoHXr9eDhx1EFiw7b6GoRJJquCnz9E7QAZknMS/8UIFPdT?= =?us-ascii?Q?RGXSrmqozC9/w55P+0mlh5Z7gmmKOcovpHuvovH70/mebnm22rVAnzciSkbL?= =?us-ascii?Q?dLZV4oblltfmKCPZQ6YtJh79cOeQTVDYY5LaRI/cuC/hIu/utGD4LGqeN4PG?= =?us-ascii?Q?qtsmAPHMLyU8gBrLW12twovkQRh+lBjNy6rvvp73+T5zH0vaU1xGQQ1JSKmI?= =?us-ascii?Q?XF4BwOJF/PmMwgwTxIaTpDmNqKrDdkys8DBm/yLkkcrBojmV5JocKLJDq7rY?= =?us-ascii?Q?8TV9xtjOQKrWKmrSDwm456bywFQZO815EJGwjcf/UMQz/F3ocn1Y+JMXnZIi?= =?us-ascii?Q?dQ=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: vkfTuEkQLA1aKJ2TB2cs92F9xWSa5t8zhHsFgoAPNdNh8lVGGJlJam8HGKC/ApbMc4AlW9gX/BUswVMgESKhL7ZXRf9rkkSmPK8E4Z+ixox/Edg0R53+JHTp4aaZ2vN5iY4xZvBG8oS2i+IbJ/nkWrXGkt6vzqR0GSsqlKuVbk3Q7DeJPQTgCHWcNUzhamJFVL4hcGylDlJek1K2B4/llYRDaWvU7tocUxK4ja2r7//cuGvLd2LdZHdRGUwaiI5mZ4dBczWQBBpnJn1Olek+cDksXUAe9cSiMRaz4k19I6Yw+VkNWbr+xk8UPLfrIuIkpc62/vHVCGGnnwrGCjVc/Kp+QYI/1i6Ol0Hgr+z/zbfaczmbLTptVtLDnYVcVa9qDtWRLNJm7Jn/XUHczFYH5kg7CMkBqs6WKfM7daUVBw4j0kVbj8xax72JrGAb6w6ofvrHlGG5toi0yJlvR8Ihjc1LSuaslgp/UhBSRt4481eDcIUAXp5s7byqCZx1NXgascpbH+/7mZDxCUQvgNBkDz0wmAsmruQwBzoHZab/jmQhwFU3qWhkeuDcbj2ZCnskMYn/Vk/cLzpU2FwPup07YSCMfM/08W7HSgh4lk6MVvM= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 67da5d85-357e-4f90-6a07-08de20a79465 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:22:05.6176 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: obMNyq0AF0WRMFGUnL43cJHSa7KzixaM4E1rZybthmKf9mWWOMzHX1CqaMzI7FHDRoklkLrf4QphqirFVC2MIGXe02xyqw+sKa60VSCE12U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: MFc6-x_c5hFzvlMUHHVLlKBF-KZAU5ld X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4NCBTYWx0ZWRfX2IQgTHv6iTuD cLdFTHENm3uE5UpQzpBxlIwsn6c3IbTkXYyyXRKFixWGod6fOgQ8BlMFRJPR0Jhi1wlj/YeZ35b 4GsO8xWHMpadpvKHFKR4OMseT/HTFKmFK5cRFPw6g5ueVJHeNbZRMrfqXtAzEyon5y6zI+RUapY rjIAXnBdjLV4LptJs6V9G8a3WgGCDpDd+ulk2sQAuW7TCmrkPVpaNTm4vHneJMzSeYiBKzvl3dW c1IjybOtg3wUzs427A7ECarbIvLiOxzYWKcjaRQcIA6qWGeKMKPJJj85vzd0RxsPm8FBX0M6In0 33n443zzNtU/HfyvGSbjPTDH5sK1SCIJrs76jKHZAKkTQ3zULFCqahO6zhIhxZFt6g11kLK/Atf ApmpDORn0noZSwm4yZdHkIRPhFDPbw== X-Proofpoint-ORIG-GUID: MFc6-x_c5hFzvlMUHHVLlKBF-KZAU5ld X-Authority-Analysis: v=2.4 cv=YN+SCBGx c=1 sm=1 tr=0 ts=69126593 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=jXjnEuVU91_iq225JjgA:9 Content-Type: text/plain; charset="utf-8" There is simply no need for the hugely confusing concept of 'non-swap' swap entries now we have the concept of softleaf entries and relevant softleaf_xxx() helpers. Adjust all callers to use these instead and remove non_swap_entry() altogether. No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- arch/s390/mm/gmap_helpers.c | 20 ++++++++++---------- arch/s390/mm/pgtable.c | 12 ++++++------ fs/proc/task_mmu.c | 12 ++++++------ include/linux/swapops.h | 5 ----- mm/filemap.c | 2 +- mm/hmm.c | 16 ++++++++-------- mm/madvise.c | 2 +- mm/memory.c | 36 ++++++++++++++++++------------------ mm/mincore.c | 2 +- mm/userfaultfd.c | 24 ++++++++++++------------ 10 files changed, 63 insertions(+), 68 deletions(-) diff --git a/arch/s390/mm/gmap_helpers.c b/arch/s390/mm/gmap_helpers.c index d4c3c36855e2..549f14ad08af 100644 --- a/arch/s390/mm/gmap_helpers.c +++ b/arch/s390/mm/gmap_helpers.c @@ -11,27 +11,27 @@ #include #include #include -#include +#include #include #include #include #include =20 /** - * ptep_zap_swap_entry() - discard a swap entry. + * ptep_zap_softleaf_entry() - discard a software leaf entry. * @mm: the mm - * @entry: the swap entry that needs to be zapped + * @entry: the software leaf entry that needs to be zapped * - * Discards the given swap entry. If the swap entry was an actual swap - * entry (and not a migration entry, for example), the actual swapped + * Discards the given software leaf entry. If the leaf entry was an actual + * swap entry (and not a migration entry, for example), the actual swapped * page is also discarded from swap. */ -static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) +static void ptep_zap_softleaf_entry(struct mm_struct *mm, softleaf_t entry) { - if (!non_swap_entry(entry)) + if (softleaf_is_swap(entry)) dec_mm_counter(mm, MM_SWAPENTS); - else if (is_migration_entry(entry)) - dec_mm_counter(mm, mm_counter(pfn_swap_entry_folio(entry))); + else if (softleaf_is_migration(entry)) + dec_mm_counter(mm, mm_counter(softleaf_to_folio(entry))); free_swap_and_cache(entry); } =20 @@ -66,7 +66,7 @@ void gmap_helper_zap_one_page(struct mm_struct *mm, unsig= ned long vmaddr) preempt_disable(); pgste =3D pgste_get_lock(ptep); =20 - ptep_zap_swap_entry(mm, pte_to_swp_entry(*ptep)); + ptep_zap_softleaf_entry(mm, softleaf_from_pte(*ptep)); pte_clear(mm, vmaddr, ptep); =20 pgste_set_unlock(ptep, pgste); diff --git a/arch/s390/mm/pgtable.c b/arch/s390/mm/pgtable.c index 0fde20bbc50b..d670bfb47d9b 100644 --- a/arch/s390/mm/pgtable.c +++ b/arch/s390/mm/pgtable.c @@ -16,7 +16,7 @@ #include #include #include -#include +#include #include #include #include @@ -683,12 +683,12 @@ void ptep_unshadow_pte(struct mm_struct *mm, unsigned= long saddr, pte_t *ptep) pgste_set_unlock(ptep, pgste); } =20 -static void ptep_zap_swap_entry(struct mm_struct *mm, swp_entry_t entry) +static void ptep_zap_softleaf_entry(struct mm_struct *mm, softleaf_t entry) { - if (!non_swap_entry(entry)) + if (softleaf_is_swap(entry)) dec_mm_counter(mm, MM_SWAPENTS); - else if (is_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + else if (softleaf_is_migration(entry)) { + struct folio *folio =3D softleaf_to_folio(entry); =20 dec_mm_counter(mm, mm_counter(folio)); } @@ -710,7 +710,7 @@ void ptep_zap_unused(struct mm_struct *mm, unsigned lon= g addr, if (!reset && pte_swap(pte) && ((pgstev & _PGSTE_GPS_USAGE_MASK) =3D=3D _PGSTE_GPS_USAGE_UNUSED || (pgstev & _PGSTE_GPS_ZERO))) { - ptep_zap_swap_entry(mm, pte_to_swp_entry(pte)); + ptep_zap_softleaf_entry(mm, softleaf_from_pte(pte)); pte_clear(mm, addr, ptep); } if (reset) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index d982fdfcf057..6cb9e1691e18 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1020,13 +1020,13 @@ static void smaps_pte_entry(pte_t *pte, unsigned lo= ng addr, } else if (pte_none(ptent)) { smaps_pte_hole_lookup(addr, walk); } else { - swp_entry_t swpent =3D pte_to_swp_entry(ptent); + const softleaf_t entry =3D softleaf_from_pte(ptent); =20 - if (!non_swap_entry(swpent)) { + if (softleaf_is_swap(entry)) { int mapcount; =20 mss->swap +=3D PAGE_SIZE; - mapcount =3D swp_swapcount(swpent); + mapcount =3D swp_swapcount(entry); if (mapcount >=3D 2) { u64 pss_delta =3D (u64)PAGE_SIZE << PSS_SHIFT; =20 @@ -1035,10 +1035,10 @@ static void smaps_pte_entry(pte_t *pte, unsigned lo= ng addr, } else { mss->swap_pss +=3D (u64)PAGE_SIZE << PSS_SHIFT; } - } else if (is_pfn_swap_entry(swpent)) { - if (is_device_private_entry(swpent)) + } else if (softleaf_has_pfn(entry)) { + if (softleaf_is_device_private(entry)) present =3D true; - page =3D pfn_swap_entry_to_page(swpent); + page =3D softleaf_to_page(entry); } } =20 diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 41cfc6d59054..c8e6f927da48 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -492,10 +492,5 @@ static inline pmd_t swp_entry_to_pmd(swp_entry_t entry) =20 #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */ =20 -static inline int non_swap_entry(swp_entry_t entry) -{ - return swp_type(entry) >=3D MAX_SWAPFILES; -} - #endif /* CONFIG_MMU */ #endif /* _LINUX_SWAPOPS_H */ diff --git a/mm/filemap.c b/mm/filemap.c index 950d93885e38..ab8ff5b2fc3b 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -4566,7 +4566,7 @@ static void filemap_cachestat(struct address_space *m= apping, swp_entry_t swp =3D radix_to_swp_entry(folio); =20 /* swapin error results in poisoned entry */ - if (non_swap_entry(swp)) + if (!softleaf_is_swap(swp)) goto resched; =20 /* diff --git a/mm/hmm.c b/mm/hmm.c index e9735a9b6102..0158f2d1e027 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -258,17 +258,17 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, u= nsigned long addr, } =20 if (!pte_present(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 /* * Don't fault in device private pages owned by the caller, * just report the PFN. */ - if (is_device_private_entry(entry) && - page_pgmap(pfn_swap_entry_to_page(entry))->owner =3D=3D + if (softleaf_is_device_private(entry) && + page_pgmap(softleaf_to_page(entry))->owner =3D=3D range->dev_private_owner) { cpu_flags =3D HMM_PFN_VALID; - if (is_writable_device_private_entry(entry)) + if (softleaf_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; new_pfn_flags =3D swp_offset_pfn(entry) | cpu_flags; goto out; @@ -279,16 +279,16 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, u= nsigned long addr, if (!required_fault) goto out; =20 - if (!non_swap_entry(entry)) + if (softleaf_is_swap(entry)) goto fault; =20 - if (is_device_private_entry(entry)) + if (softleaf_is_device_private(entry)) goto fault; =20 - if (is_device_exclusive_entry(entry)) + if (softleaf_is_device_exclusive(entry)) goto fault; =20 - if (is_migration_entry(entry)) { + if (softleaf_is_migration(entry)) { pte_unmap(ptep); hmm_vma_walk->last =3D addr; migration_entry_wait(walk->mm, pmdp, addr); diff --git a/mm/madvise.c b/mm/madvise.c index ffae3b566dc1..234178685793 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -249,7 +249,7 @@ static void shmem_swapin_range(struct vm_area_struct *v= ma, continue; entry =3D radix_to_swp_entry(folio); /* There might be swapin error entries in shmem mapping. */ - if (non_swap_entry(entry)) + if (!softleaf_is_swap(entry)) continue; =20 addr =3D vma->vm_start + diff --git a/mm/memory.c b/mm/memory.c index 087f31a291b4..ad336cbf1d88 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -932,7 +932,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, struct folio *folio; struct page *page; =20 - if (likely(!non_swap_entry(entry))) { + if (likely(softleaf_is_swap(entry))) { if (swap_duplicate(entry) < 0) return -EIO; =20 @@ -950,12 +950,12 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, set_pte_at(src_mm, addr, src_pte, pte); } rss[MM_SWAPENTS]++; - } else if (is_migration_entry(entry)) { - folio =3D pfn_swap_entry_folio(entry); + } else if (softleaf_is_migration(entry)) { + folio =3D softleaf_to_folio(entry); =20 rss[mm_counter(folio)]++; =20 - if (!is_readable_migration_entry(entry) && + if (!softleaf_is_migration_read(entry) && is_cow_mapping(vm_flags)) { /* * COW mappings require pages in both parent and child @@ -964,15 +964,15 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, */ entry =3D make_readable_migration_entry( swp_offset(entry)); - pte =3D swp_entry_to_pte(entry); + pte =3D softleaf_to_pte(entry); if (pte_swp_soft_dirty(orig_pte)) pte =3D pte_swp_mksoft_dirty(pte); if (pte_swp_uffd_wp(orig_pte)) pte =3D pte_swp_mkuffd_wp(pte); set_pte_at(src_mm, addr, src_pte, pte); } - } else if (is_device_private_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + } else if (softleaf_is_device_private(entry)) { + page =3D softleaf_to_page(entry); folio =3D page_folio(page); =20 /* @@ -996,7 +996,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm= _struct *src_mm, * when a device driver is involved (you cannot easily * save and restore device driver state). */ - if (is_writable_device_private_entry(entry) && + if (softleaf_is_device_private_write(entry) && is_cow_mapping(vm_flags)) { entry =3D make_readable_device_private_entry( swp_offset(entry)); @@ -1005,7 +1005,7 @@ copy_nonpresent_pte(struct mm_struct *dst_mm, struct = mm_struct *src_mm, pte =3D pte_swp_mkuffd_wp(pte); set_pte_at(src_mm, addr, src_pte, pte); } - } else if (is_device_exclusive_entry(entry)) { + } else if (softleaf_is_device_exclusive(entry)) { /* * Make device exclusive entries present by restoring the * original entry then copying as for a present pte. Device @@ -4635,7 +4635,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) rmap_t rmap_flags =3D RMAP_NONE; bool need_clear_cache =3D false; bool exclusive =3D false; - swp_entry_t entry; + softleaf_t entry; pte_t pte; vm_fault_t ret =3D 0; void *shadow =3D NULL; @@ -4647,15 +4647,15 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (!pte_unmap_same(vmf)) goto out; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); - if (unlikely(non_swap_entry(entry))) { - if (is_migration_entry(entry)) { + entry =3D softleaf_from_pte(vmf->orig_pte); + if (unlikely(!softleaf_is_swap(entry))) { + if (softleaf_is_migration(entry)) { migration_entry_wait(vma->vm_mm, vmf->pmd, vmf->address); - } else if (is_device_exclusive_entry(entry)) { - vmf->page =3D pfn_swap_entry_to_page(entry); + } else if (softleaf_is_device_exclusive(entry)) { + vmf->page =3D softleaf_to_page(entry); ret =3D remove_device_exclusive_entry(vmf); - } else if (is_device_private_entry(entry)) { + } else if (softleaf_is_device_private(entry)) { if (vmf->flags & FAULT_FLAG_VMA_LOCK) { /* * migrate_to_ram is not yet ready to operate @@ -4666,7 +4666,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) goto out; } =20 - vmf->page =3D pfn_swap_entry_to_page(entry); + vmf->page =3D softleaf_to_page(entry); vmf->pte =3D pte_offset_map_lock(vma->vm_mm, vmf->pmd, vmf->address, &vmf->ptl); if (unlikely(!vmf->pte || @@ -4690,7 +4690,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } else { pte_unmap_unlock(vmf->pte, vmf->ptl); } - } else if (is_hwpoison_entry(entry)) { + } else if (softleaf_is_hwpoison(entry)) { ret =3D VM_FAULT_HWPOISON; } else if (softleaf_is_marker(entry)) { ret =3D handle_pte_marker(vmf); diff --git a/mm/mincore.c b/mm/mincore.c index b3682488a65d..9a908d8bb706 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -74,7 +74,7 @@ static unsigned char mincore_swap(swp_entry_t entry, bool= shmem) * absent. Page table may contain migration or hwpoison * entries which are always uptodate. */ - if (non_swap_entry(entry)) + if (!softleaf_is_swap(entry)) return !shmem; =20 /* diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 055ec1050776..bd1f74a7a5ac 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -1256,7 +1256,6 @@ static long move_pages_ptes(struct mm_struct *mm, pmd= _t *dst_pmd, pmd_t *src_pmd unsigned long dst_addr, unsigned long src_addr, unsigned long len, __u64 mode) { - swp_entry_t entry; struct swap_info_struct *si =3D NULL; pte_t orig_src_pte, orig_dst_pte; pte_t src_folio_pte; @@ -1430,19 +1429,20 @@ static long move_pages_ptes(struct mm_struct *mm, p= md_t *dst_pmd, pmd_t *src_pmd orig_dst_pte, orig_src_pte, dst_pmd, dst_pmdval, dst_ptl, src_ptl, &src_folio, len); - } else { + } else { /* !pte_present() */ struct folio *folio =3D NULL; + const softleaf_t entry =3D softleaf_from_pte(orig_src_pte); =20 - entry =3D pte_to_swp_entry(orig_src_pte); - if (non_swap_entry(entry)) { - if (is_migration_entry(entry)) { - pte_unmap(src_pte); - pte_unmap(dst_pte); - src_pte =3D dst_pte =3D NULL; - migration_entry_wait(mm, src_pmd, src_addr); - ret =3D -EAGAIN; - } else - ret =3D -EFAULT; + if (softleaf_is_migration(entry)) { + pte_unmap(src_pte); + pte_unmap(dst_pte); + src_pte =3D dst_pte =3D NULL; + migration_entry_wait(mm, src_pmd, src_addr); + + ret =3D -EAGAIN; + goto out; + } else if (!softleaf_is_swap(entry)) { + ret =3D -EFAULT; goto out; } =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5F2F340A72; Mon, 10 Nov 2025 22:24:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813451; cv=fail; b=L2TSNV58XHOW45Jlzq/rs0T5i4DN2wAgiqaBtiQAa9bpfkPDuils9BZB/5s9BcCUpYF2MGhZNmPEkdum5Feem1KTYH/IQVL01P2gsWhtI5CQ89GnQgwYbZPphDDBIZ/kSm0JjbMH3zYFZg49fn3Ww6wyW5fss2q8ypgFnczzO6o= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813451; c=relaxed/simple; bh=Ywwk8BgU1vvgEYMYYm12WFq+m5B/cdbVU9dI9FbCrwo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=CxT3NySbDgZPYc7Gd6sCX01Yi4BmDrD9AJA14Qh3yLyqI7+6j1Fg2C0fcxJhAc8+zonbCRYvOzcuWmwIYYmQo5BuwEcvLwB/nC6TocIpr+nZEo6tpGGBLs11uyo6OSo53e7bYes1H01Tl3KtLqGHtmXHpo7gwITViTUDPdQyUJg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=dcILQqPI; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=fFqGGErw; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="dcILQqPI"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="fFqGGErw" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAMIQ7q026148; Mon, 10 Nov 2025 22:22:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=8Mj4dZ2xkCZUyYXA/0EAZ/Vac+RE+RRWNsVC2E5wqGE=; b= dcILQqPI2I1lrj658IeiVrC4i8Jomk6wt6KP2UScMg/tbpEx06SOIJa7kohsZPEi zz/xvpNKuJyJiS7jW8AXeMiGMbhrKh/9U6ZVdM8/2Xj2W5C613mi8VRcr2QvwQbF PoYmmgI91koc+uWQQSCFkWpHk676E5do8TfniDj//F4phDjFI4E5W+MGmdv9waVz ESCA+UEvZzgkFaZR3SsSOIIzlamzJ7zU9HuUzYPUaslYnuE/r+mKD5Yf2d3xta1R gWD5dV6084Wn8aCh5FxbK/G//H5hDXkFQk+r93Zkll9zckv+lqZ0hYCv9P1KfUEO MlHlH1x3X9bdk2L2ApWuiA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abr5vr2d3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:13 +0000 (GMT) Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAK9m0q040038; Mon, 10 Nov 2025 22:22:12 GMT Received: from co1pr03cu002.outbound.protection.outlook.com (mail-westus2azon11010055.outbound.protection.outlook.com [52.101.46.55]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 4a9va8ryxq-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:12 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=uC3FAylCYUXSQ5gzHebJLUFsjJUzWYqz2gj+S+AV6XsePnw2X0yrqUNDn93baK1W9o+O1bqDcFOLMQ8Dir6/DQqhgKxmDSk3Uyfe2KZfggvN56CJDSUACl98jAgEj494GAEEsMRxEvN1ZEjxH+/TOM+qtb106PMkeQ8cJDzSHY4w77hTHh3D2EOYoJ3i/u3AuZE/Jc/LdY6zTQhs0n2A5GdWd6pyAkGvIJJDgxx2ZMNXw0bRfDHZN/UdauIAqceLb7m+Hmo+VXWeg8xajkUCOGkFisYo8FNzk8ftmCGBqDCVAmINaDbWBii2gdxs9D9SsGNoRKj/9WMYHTK96VKVlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8Mj4dZ2xkCZUyYXA/0EAZ/Vac+RE+RRWNsVC2E5wqGE=; b=q9wIVwNpz5GMrs5NmhnCnpfkk91pSLesAIlpliPzHxRCsWVWomx7RR6VJTd2DMFT9Gzv5UdPyVqu7moEEbzmwpgA/Vh5bHL+loyXzLd+ziFEAAiMQ0jpV2mbDaOcfIkDKcI2PbeyljjrNXk8599T9sXJh3ShvtazQ47oTZ5yONaO0OzVGJlDY6YEGJQDmlSbDSjweiFnCIgcO4jCfkbT+/5vuanjnFDDWReEbaJjW4otCfAV30p7fWx7FnH7379l5zS9yvE5hwugVt78GJt/GTon9K/G9uqUF+s/RI9Rq8uiIoQiCDGc7J8gPtWQc33KL5lmR2ZGqCdgmUbWIaBmLg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8Mj4dZ2xkCZUyYXA/0EAZ/Vac+RE+RRWNsVC2E5wqGE=; b=fFqGGErwjNLLc6UthQbkgaFPsT+NTqk5uflg58UJSENXkpDKt8AEBZg1p0Nn7gkLb5VZRakeSji2p67E4XaOobSlfD7ORMp1ayFyIq75OdyhU3srKTKqdkyABPaL8BqMyvl90WfDz5G4T9BvQoMAwiVZFcis88HotnWnIyYML00= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:22:08 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:22:08 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 14/16] mm: remove is_hugetlb_entry_[migration, hwpoisoned]() Date: Mon, 10 Nov 2025 22:21:32 +0000 Message-ID: <0e92d6924d3de88cd014ce1c53e20edc08fc152e.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0224.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:315::8) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: f633ecb7-41dd-4db0-aee9-08de20a79601 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?7aP65Pna5gqFhGWAn9I6t7YVq81tD5uw/qii4Iird9ZZJBF+Kwi8MPIwhuF8?= =?us-ascii?Q?Ft6hINN5c258Q4BguAIhnbyuJDF8ZQHhhWapNqJRn5iBvUmQ8JlynCjpXQq4?= =?us-ascii?Q?xkjLUh7pG2EYaUIP5C+FYdVt/FcLRB6+S476EtOmWaOG7ei7vXbc7C50N2+G?= =?us-ascii?Q?HlcDRMNk5TQgJmlidMuQ/oaHcVNCzaf7mO3Q9spkhu1kgqjz5r6FndWcyH6o?= =?us-ascii?Q?hZX0/qAPSnRsJvNjnVv1TCP7XWHNA5hsK8c2PzEtsyRvI9b0tEeaml0MgXkl?= =?us-ascii?Q?FzIXBQTpZJht5f0B+Xi658N7e4BNS0m5TQaSovnMTaKcoR7pa93PpFnyRlUR?= =?us-ascii?Q?VQw14gaYo0Qxki07zmLe4Yr4C94eRsS+PeiyI2S54gl3Zn4UTsDepDd01U5j?= =?us-ascii?Q?dnXVRCRFoQzcrv56Y1tE/rceyTKj82h//b7mbCi0jPnwK8OlpGshNKghKZ0/?= =?us-ascii?Q?UDxDOBr9wKAS6OtcGefP4U4NiEdYpC+31MrRy/wb4izV1sQquFLqeltLUk22?= =?us-ascii?Q?C9ApDM9ALy2/faOPmNGhLPnLQ6hTVwDwcqRnKETmgTuqtl405f8gcl8yf+lO?= =?us-ascii?Q?qdIMtlM/ilbRbGbLcI1dIqPk1Ncu4Ku0Ym2nYWi77JlN9YwvTePfLSbxgy8w?= =?us-ascii?Q?1c894CvfsbVKOZqnIhw0fEGmYH4pflSNmreHTZ7Pnw2/LLC595okrbUj7USa?= =?us-ascii?Q?w3r15eDuZuEphKs3PEz//w559fExcoaOTLSUUpOUhxNFtqtnuT+tbVgTVWwc?= =?us-ascii?Q?+TvucSqqiP7JwpKnYKiiTzf7qzT6uxASEbx1TglRT7fGqOp+ZTviSzDX0xG0?= =?us-ascii?Q?eRLYYppOf9HODN1wBybsAvR7ybncGZR7VzWs6/y2BHlJLwA2jwAajJ0k6tcX?= =?us-ascii?Q?bcnurJAAMUSlpj30hahSoPe/xQEw08C/WYgBceIS+fQILJEYCkt8xru7xekE?= =?us-ascii?Q?3mLOthdy/CEJoe7fdCJ/JLokoG9PmN6Msag6yqqAKnNKAQ0uWoxx5udODZg2?= =?us-ascii?Q?u7HCgp80WTcZmJcDpeAiKS2WbF2YLbVWJP5b6C7jS2MIYRDIbMuWF8gGRn1T?= =?us-ascii?Q?q8cb1J0Nef9eTNFPkFNRQ3ObsUwm+9Za5bWJLZHXjsoTNGJsBL1NZ2qDGav4?= =?us-ascii?Q?GlWAUy7LC3NEz8Z9JsAaBbAhCKtD18ZKzUH5xgnz+6NprQ3RCW14EvuwbnpS?= =?us-ascii?Q?MvS2O+mE1hgzWsbVWutpiKkIuLBMa6OAyHodYCpfi5182qNnl6DuxKomf1Cv?= =?us-ascii?Q?DfcBYPbFiGtJrb0UuOhk3vV6jtyTrqImW9MwED7Xb3HtoMDWK06wC8x1biL4?= =?us-ascii?Q?TBSXMWXAhOs6CVPgDAE77U+n6A7LR+rJXn/DVlfUDAkoGi8VoFv2FLz6HBM4?= =?us-ascii?Q?L7uuEQPhcfaNaW4MhYHC7RMfA+F8pexwxh3dJ/vFzLyqhzRSwMFNxGFGjzkt?= =?us-ascii?Q?wb2GIGnA0XzBCSY2tl/pGYwyvPrCVKDE?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?KKigt8Xc8ZcGerH1XoxB0WDTS5RIES+2Zl6knURH9qIuEtOpAaOWijHFDRL2?= =?us-ascii?Q?DP5G/oGm37OY5HhUZAgAclnzrOSqS6XViqEUQMUTEQfTaa4Uphcb6jn1fQvb?= =?us-ascii?Q?4T+RhjWa+JfSiTcKpeYLVGfTTuMGUxL/lfqlnpO15d7pf0WTq/szEqZp/SGK?= =?us-ascii?Q?UG7oymB74SGsokXlNmFrAP7w9d+dZronQcNwoaH2kf2/S064w2TJcoD3ijzA?= =?us-ascii?Q?Hxk4RzbNl0BABnOxUPfPcuHFZOpWq4FCO0hAsC7HHqerCnN5YdtaHfd1i6sN?= =?us-ascii?Q?0ScdEjoYA7CGlkOB4kHUAI5NFapaf0mEzFdKqsUvCO/Nqk7xN7VXg7cjm5Qo?= =?us-ascii?Q?3nz+9VhY7yA/ON2d1l29RZti2xCoOqRXhSgRXVhOBpr+s+Sj4Fu5GrhUapbB?= =?us-ascii?Q?SplDl7/JiFCUoctOVfu0qdSr4gLSQLB0+azTeJOpVoeGs+SmJSvg+HIv3Kci?= =?us-ascii?Q?SSFM7N1Q4jnirLT/wVUTPfKzapgvOAIhMCH953D4+Qfdy1jUiYvU+43x28ZQ?= =?us-ascii?Q?YsXQNtcpp27kezTAUvLyBoq/E455VolSjoGv0QLyKqaTVMumNrwu0bOEWOdT?= =?us-ascii?Q?iOBzCdMQUL7uO95mkKC7SlyBNyDmclkTLlHVVqmK9vCwxkP3fXJP7WhIykmU?= =?us-ascii?Q?ydBKc29pK2JuGNLU40rBr2xn+mPgDw53QsYG84PmT8YccrNUjf5Q3spuRKoA?= =?us-ascii?Q?wyhBGvYS7l8pA6ibYVv7xUsRvsCYbIG7nHqyOHUErz32lbC9g+Qomi0hJbjz?= =?us-ascii?Q?OqGxoMoRxadQkx0oO4MF9I6fRc2G7jZHeKaRnrQKUvnIypaoTHD1dkc/r1ru?= =?us-ascii?Q?ZxS5/PvZwUUuZrqz+GPxjY7xD4nQcnIFjnjw/+PQagNYfMm53JWAUN7K1x4d?= =?us-ascii?Q?9HVyfyFJHMDFinu67BkRRFkCP7w6fNMjeK1JYBldFBoSvhbJHlG3tGjBVUgJ?= =?us-ascii?Q?2lj163EPEEB2uyA5CsEojmyLY36qg7tZJhe6K44ZWCGchNJ0QDn83MItjDCF?= =?us-ascii?Q?FnfoD52QaWPoaFztctnVrbrs+h7rGvm0tam++UBUgnPH1osMvs2PgNCXXAvj?= =?us-ascii?Q?cNuJNzcqOH3X8iGi1eSchSH5FyDMwp2GWCB6sdPj9vnGjXZ7TrfT/4pX4pWV?= =?us-ascii?Q?h9nvB1TZ6EIi6EKD8v+QRwm4giGIX4MBfTrF+gKclP3OUMdxuTmQks5AWse4?= =?us-ascii?Q?0PAIgyfXG5Ve9aXeW85gUF2UlUuectqyB/tKpUAen661cgQVOcHB/EvwLMRP?= =?us-ascii?Q?dl+c5wYGQb39vcyW63QUWJBYj+3mhOXqCasTqJ1ZN2nVmVOdkCS85NCfsPnk?= =?us-ascii?Q?f5jrpjOTV0Rb8v13lKbOrn9OU4TK/SDwfEXF26DF1IL+zBehUVJc3qhKSHdK?= =?us-ascii?Q?Jw8BjSZiG7fPFGyMuWZzWy7xnDI2BKGmYgu2zSg4L5AbD/U0MFfCkSAKZJXO?= =?us-ascii?Q?W5ndV2RSRCFLyql2VuHwGr9BVPl/hSsjtosErJdmYJ0P2cyS8ZBq/wWljeA/?= =?us-ascii?Q?+1jMMuMiaycbwTyGPLNWaZ8f2U0cRRu0hrsFSf0UkWoZa+ZWQ5EiZ2MR2MnW?= =?us-ascii?Q?r18CuOiJydFYwFldpKIFBFgfxfgbqqYd0ta90PaPU7SrOR32EGW5oOtPZn/n?= =?us-ascii?Q?cw=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 0uqd/jHsf3JigH/yUUCJnPMx8egpmgwrtPsg8IgIgbTrwb7z+KVSExWE8QbL5+JIvKBN6nVdB5+oaJg4hJ0ix0wnPhMOyxb0Y9P1dg0oPuOIrSSUleeYrYUhP78PfjAoLGV5VPgNTSKafVg1pBG5bjMLXcLsNeNhKWEsfkFYDk6ZGTF8XwZFA9msD5DsccBG0Yau7S9kQpANcbqHMfWLaVe8oW/oD9RvAm4M4ZfIP48e85gHd8FFktRE9p2x+A9IXnEiY5vJL2CTUMu9qHxl0O0bAJjfBVDa5nlSmf9xbRUx2FsoDe+IDZQrSiRhKVctIsE21ZBPQjdKJW0tC1gNNVbAIURIWsTfAIxRErHmI+R3Y/j2BZFd9NK8/3mR5DaYFqmR+1YX74DBqs2znR7AleLH5UMf25GfPqSNYUoo/v4hPFU4Z8XgorGSCmbXwhorDwE26VOjhASShQdFwrePRBhtxhfto/jPxy+HIx+AKKD2+u7J8Rm+aMLR6tvfNtDCkIqh7AmJf4R7YHhFxkmiYIBNMT9Fhhpr2HbwINtVRT7xCXwZoF142VmUslVa5sMRQXexEkmCqqeLRfbx8nhFypWpJ7ikA3m0DccqVnzxTA8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f633ecb7-41dd-4db0-aee9-08de20a79601 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:22:08.3093 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BVBQqjNQYPrJaCJwiKGpVfwDX36+rinJPxZyGYNDvgt4AauO9qsTFsVoXeoQ9jfhAPY6faS2XeckGMjQuMA49Rxxi9UWUNbKu3iGb5UGKGM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-GUID: yFQd-S0gGg-n_MuzvJcyUsR4PscbTUef X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE4NCBTYWx0ZWRfX1YrPK1+eH6U9 BP/t63kMORTT27AD6xIyUgYltNoH9TGtab6dmWZnq7F9WDjSxmGqH4K4eLcJrpewiVQUwiXjM+b FgDp1Pv97uZhqfzw1cnuOeerjsmHpzFyLANpeSLKPOKmig40PyqQgBDpLjxm4SVCNopcJ/zJdXc 3+nRQOETAVGKDevpRvKEyYYt5BkaDL+wjeI5Lpuj11vH77IyJnZv/bnpTfUfAu9zfL2dpSQPds7 1khTT9VS5z4lCO7oEojjKfbiZm5t4h314useS/OmTVMvb7rYPKVVhAPebFD0w/aIkpAb1LvBkf/ YUQal/YaU0gRVvxPIB39fweoA24CQbwhHxomiGj0qfK71L1km6QFYGkYCcqNNFMQBgnxqmY7FNr H+wIixj/eGG2CIJKHksa1z+Twde3mw== X-Proofpoint-ORIG-GUID: yFQd-S0gGg-n_MuzvJcyUsR4PscbTUef X-Authority-Analysis: v=2.4 cv=YN+SCBGx c=1 sm=1 tr=0 ts=69126595 b=1 cx=c_pps a=WeWmnZmh0fydH62SvGsd2A==:117 a=WeWmnZmh0fydH62SvGsd2A==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=gaN7zjf0O2erDMybN6AA:9 Content-Type: text/plain; charset="utf-8" We do not need to have explicit helper functions for these, it adds a level of confusion and indirection when we can simply use software leaf entry logic here instead and spell out the special huge_pte_none() case we must consider. No functional change intended. Signed-off-by: Lorenzo Stoakes Reported-by: Vlastimil Babka Reviewed-by: Vlastimil Babka --- fs/proc/task_mmu.c | 19 +++++---- include/linux/hugetlb.h | 2 - mm/hugetlb.c | 91 +++++++++++++++++------------------------ mm/mempolicy.c | 17 +++++--- mm/migrate.c | 15 +++++-- 5 files changed, 69 insertions(+), 75 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 6cb9e1691e18..3cdefa7546db 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -2499,22 +2499,23 @@ static void make_uffd_wp_huge_pte(struct vm_area_st= ruct *vma, unsigned long addr, pte_t *ptep, pte_t ptent) { - unsigned long psize; + const unsigned long psize =3D huge_page_size(hstate_vma(vma)); + softleaf_t entry; =20 - if (is_hugetlb_entry_hwpoisoned(ptent) || pte_is_marker(ptent)) - return; + if (huge_pte_none(ptent)) + set_huge_pte_at(vma->vm_mm, addr, ptep, + make_pte_marker(PTE_MARKER_UFFD_WP), psize); =20 - psize =3D huge_page_size(hstate_vma(vma)); + entry =3D softleaf_from_pte(ptent); + if (softleaf_is_hwpoison(entry) || softleaf_is_marker(entry)) + return; =20 - if (is_hugetlb_entry_migration(ptent)) + if (softleaf_is_migration(entry)) set_huge_pte_at(vma->vm_mm, addr, ptep, pte_swp_mkuffd_wp(ptent), psize); - else if (!huge_pte_none(ptent)) + else huge_ptep_modify_prot_commit(vma, addr, ptep, ptent, huge_pte_mkuffd_wp(ptent)); - else - set_huge_pte_at(vma->vm_mm, addr, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP), psize); } #endif /* CONFIG_HUGETLB_PAGE */ =20 diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 2387513d6ae5..457d48ac7bcd 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -274,8 +274,6 @@ void hugetlb_vma_lock_release(struct kref *kref); long hugetlb_change_protection(struct vm_area_struct *vma, unsigned long address, unsigned long end, pgprot_t newprot, unsigned long cp_flags); -bool is_hugetlb_entry_migration(pte_t pte); -bool is_hugetlb_entry_hwpoisoned(pte_t pte); void hugetlb_unshare_all_pmds(struct vm_area_struct *vma); void fixup_hugetlb_reservations(struct vm_area_struct *vma); void hugetlb_split(struct vm_area_struct *vma, unsigned long addr); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a74cde267c2a..b702b161ab35 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5552,32 +5552,6 @@ static void set_huge_ptep_maybe_writable(struct vm_a= rea_struct *vma, set_huge_ptep_writable(vma, address, ptep); } =20 -bool is_hugetlb_entry_migration(pte_t pte) -{ - swp_entry_t swp; - - if (huge_pte_none(pte) || pte_present(pte)) - return false; - swp =3D pte_to_swp_entry(pte); - if (is_migration_entry(swp)) - return true; - else - return false; -} - -bool is_hugetlb_entry_hwpoisoned(pte_t pte) -{ - swp_entry_t swp; - - if (huge_pte_none(pte) || pte_present(pte)) - return false; - swp =3D pte_to_swp_entry(pte); - if (is_hwpoison_entry(swp)) - return true; - else - return false; -} - static void hugetlb_install_folio(struct vm_area_struct *vma, pte_t *ptep, unsigned lo= ng addr, struct folio *new_folio, pte_t old, unsigned long sz) @@ -5606,6 +5580,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, unsigned long npages =3D pages_per_huge_page(h); struct mmu_notifier_range range; unsigned long last_addr_mask; + softleaf_t softleaf; int ret =3D 0; =20 if (cow) { @@ -5653,16 +5628,16 @@ int copy_hugetlb_page_range(struct mm_struct *dst, = struct mm_struct *src, entry =3D huge_ptep_get(src_vma->vm_mm, addr, src_pte); again: if (huge_pte_none(entry)) { - /* - * Skip if src entry none. - */ - ; - } else if (unlikely(is_hugetlb_entry_hwpoisoned(entry))) { + /* Skip if src entry none. */ + goto next; + } + + softleaf =3D softleaf_from_pte(entry); + if (unlikely(softleaf_is_hwpoison(softleaf))) { if (!userfaultfd_wp(dst_vma)) entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); - } else if (unlikely(is_hugetlb_entry_migration(entry))) { - softleaf_t softleaf =3D softleaf_from_pte(entry); + } else if (unlikely(softleaf_is_migration(softleaf))) { bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 if (!is_readable_migration_entry(softleaf) && cow) { @@ -5681,7 +5656,6 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, entry =3D huge_pte_clear_uffd_wp(entry); set_huge_pte_at(dst, addr, dst_pte, entry, sz); } else if (unlikely(pte_is_marker(entry))) { - const softleaf_t softleaf =3D softleaf_from_pte(entry); const pte_marker marker =3D copy_pte_marker(softleaf, dst_vma); =20 if (marker) @@ -5739,9 +5713,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, } hugetlb_install_folio(dst_vma, dst_pte, addr, new_folio, src_pte_old, sz); - spin_unlock(src_ptl); - spin_unlock(dst_ptl); - continue; + goto next; } =20 if (cow) { @@ -5762,6 +5734,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, set_huge_pte_at(dst, addr, dst_pte, entry, sz); hugetlb_count_add(npages, dst); } + +next: spin_unlock(src_ptl); spin_unlock(dst_ptl); } @@ -6770,8 +6744,10 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, ret =3D 0; =20 /* Not present, either a migration or a hwpoisoned entry */ - if (!pte_present(vmf.orig_pte)) { - if (is_hugetlb_entry_migration(vmf.orig_pte)) { + if (!pte_present(vmf.orig_pte) && !huge_pte_none(vmf.orig_pte)) { + const softleaf_t softleaf =3D softleaf_from_pte(vmf.orig_pte); + + if (softleaf_is_migration(softleaf)) { /* * Release the hugetlb fault lock now, but retain * the vma lock, because it is needed to guard the @@ -6782,9 +6758,12 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struc= t vm_area_struct *vma, mutex_unlock(&hugetlb_fault_mutex_table[hash]); migration_entry_wait_huge(vma, vmf.address, vmf.pte); return 0; - } else if (is_hugetlb_entry_hwpoisoned(vmf.orig_pte)) + } + if (softleaf_is_hwpoison(softleaf)) { ret =3D VM_FAULT_HWPOISON_LARGE | VM_FAULT_SET_HINDEX(hstate_index(h)); + } + goto out_mutex; } =20 @@ -7166,7 +7145,9 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, i_mmap_lock_write(vma->vm_file->f_mapping); last_addr_mask =3D hugetlb_mask_last_page(h); for (; address < end; address +=3D psize) { + softleaf_t entry; spinlock_t *ptl; + ptep =3D hugetlb_walk(vma, address, psize); if (!ptep) { if (!uffd_wp) { @@ -7198,15 +7179,23 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, continue; } pte =3D huge_ptep_get(mm, address, ptep); - if (unlikely(is_hugetlb_entry_hwpoisoned(pte))) { - /* Nothing to do. */ - } else if (unlikely(is_hugetlb_entry_migration(pte))) { - softleaf_t entry =3D softleaf_from_pte(pte); + if (huge_pte_none(pte)) { + if (unlikely(uffd_wp)) + /* Safe to modify directly (none->non-present). */ + set_huge_pte_at(mm, address, ptep, + make_pte_marker(PTE_MARKER_UFFD_WP), + psize); + goto next; + } =20 + entry =3D softleaf_from_pte(pte); + if (unlikely(softleaf_is_hwpoison(entry))) { + /* Nothing to do. */ + } else if (unlikely(softleaf_is_migration(entry))) { struct folio *folio =3D softleaf_to_folio(entry); pte_t newpte =3D pte; =20 - if (is_writable_migration_entry(entry)) { + if (softleaf_is_migration_write(entry)) { if (folio_test_anon(folio)) entry =3D make_readable_exclusive_migration_entry( swp_offset(entry)); @@ -7233,7 +7222,7 @@ long hugetlb_change_protection(struct vm_area_struct = *vma, if (pte_is_uffd_wp_marker(pte) && uffd_wp_resolve) /* Safe to modify directly (non-present->none). */ huge_pte_clear(mm, address, ptep, psize); - } else if (!huge_pte_none(pte)) { + } else { pte_t old_pte; unsigned int shift =3D huge_page_shift(hstate_vma(vma)); =20 @@ -7246,16 +7235,10 @@ long hugetlb_change_protection(struct vm_area_struc= t *vma, pte =3D huge_pte_clear_uffd_wp(pte); huge_ptep_modify_prot_commit(vma, address, ptep, old_pte, pte); pages++; - } else { - /* None pte */ - if (unlikely(uffd_wp)) - /* Safe to modify directly (none->non-present). */ - set_huge_pte_at(mm, address, ptep, - make_pte_marker(PTE_MARKER_UFFD_WP), - psize); } - spin_unlock(ptl); =20 +next: + spin_unlock(ptl); cond_resched(); } /* diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 01c3b98f87a6..dee95d5ecfd4 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -768,16 +768,21 @@ static int queue_folios_hugetlb(pte_t *pte, unsigned = long hmask, unsigned long flags =3D qp->flags; struct folio *folio; spinlock_t *ptl; - pte_t entry; + pte_t ptep; =20 ptl =3D huge_pte_lock(hstate_vma(walk->vma), walk->mm, pte); - entry =3D huge_ptep_get(walk->mm, addr, pte); - if (!pte_present(entry)) { - if (unlikely(is_hugetlb_entry_migration(entry))) - qp->nr_failed++; + ptep =3D huge_ptep_get(walk->mm, addr, pte); + if (!pte_present(ptep)) { + if (!huge_pte_none(ptep)) { + const softleaf_t entry =3D softleaf_from_pte(ptep); + + if (unlikely(softleaf_is_migration(entry))) + qp->nr_failed++; + } + goto unlock; } - folio =3D pfn_folio(pte_pfn(entry)); + folio =3D pfn_folio(pte_pfn(ptep)); if (!queue_folio_required(folio, qp)) goto unlock; if (!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) || diff --git a/mm/migrate.c b/mm/migrate.c index 3b6bd374157d..48f98a6c1ad2 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -515,16 +515,18 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t= *pmd, void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long a= ddr, pte_t *ptep) { spinlock_t *ptl =3D huge_pte_lockptr(hstate_vma(vma), vma->vm_mm, ptep); + softleaf_t entry; pte_t pte; =20 hugetlb_vma_assert_locked(vma); spin_lock(ptl); pte =3D huge_ptep_get(vma->vm_mm, addr, ptep); =20 - if (unlikely(!is_hugetlb_entry_migration(pte))) { - spin_unlock(ptl); - hugetlb_vma_unlock_read(vma); - } else { + if (huge_pte_none(pte)) + goto fail; + + entry =3D softleaf_from_pte(pte); + if (softleaf_is_migration(entry)) { /* * If migration entry existed, safe to release vma lock * here because the pgtable page won't be freed without the @@ -533,7 +535,12 @@ void migration_entry_wait_huge(struct vm_area_struct *= vma, unsigned long addr, p */ hugetlb_vma_unlock_read(vma); migration_entry_wait_on_locked(pte_to_swp_entry(pte), ptl); + return; } + +fail: + spin_unlock(ptl); + hugetlb_vma_unlock_read(vma); } #endif =20 --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F4313446BB; Mon, 10 Nov 2025 22:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813468; cv=fail; b=ipLWIbGyCGl/6n0Pw8MN0/hbBfgZS88OruhNVcpFO6J3Hp9ec023RqdQoh66UbqSaziWWbczRLtxj2/t+jPrDh4XEzyUhWIHmOmLjPEJaCGFv65Cjoo9Z5E1Tbf40SHlt8DsC3THga6SgUe3aeupL6ghcIgXJq8vXvhyDhizaTA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813468; c=relaxed/simple; bh=SOKdtiMkUX1wfXKL3vCMqT58JDUIorhnArOoC54ZFes=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=kzxiADtWsigEfWCaPDInsaODU6hToGbhhveaJ0xflvnPEPv2PXNFhlGzBqq8uTO4SlpHcbJYhSYn3opZZI9BaOOaexb3DayKyQZdRKnF9x+2f0FSS5j94bY8buaoC0vugtfyUPCcGooUMkvMh8uIg5IJue2Y5k+AmZ7K+Q4snZ4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=fo8OalgA; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=YSeTAazj; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="fo8OalgA"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="YSeTAazj" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAL8kOf009329; Mon, 10 Nov 2025 22:22:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=OHeAr/1/JvOCPFjeSf6O3LGZncrh/u/f9P6dTRZkvgE=; b= fo8OalgArR7+giFNItfG3QK0SuB0xrxpRG+vXq/ttg/j7CvYyWCC2zWDkVSQFj9L CdMyNJnI6YeKitsjO+opaoPU2nAV8KGVKMwXB9JuMkwfjO6TKKQADmsMQ9nCEP1M Lwfs/xXFUxj/skE6klQyoW+NhlF0ns8sDIkOQZQKO3THTIgP9adyhSB4CJIwJUlh FYYtGvOfx62wxNupbQ/PufStAUT58x6NKtMAHTFKmqI8MljsXvn3h2VxfoO5PzL4 eZFqSp4eX9JDKJSTtQYe48T4GOheu2PkFp48pgGLryRsQgmV/2/ounmtLQVuT6+M YsbMHFt+40mtazIQ0R1OjQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abqpug5tq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:17 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AALsCKb000873; Mon, 10 Nov 2025 22:22:15 GMT Received: from co1pr03cu002.outbound.protection.outlook.com (mail-westus2azon11010031.outbound.protection.outlook.com [52.101.46.31]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vacs4p9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:15 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QleGP/ScfbxlGdnC6G+wg6Ul5AkMEjO5saRGhphWHq3R1onCzsE9pSnQpwsBj3Nzvf6Lv08H73xYCOh5zEGF8sRFpu6IEYnqUDdauSd+S+NgSKAMXv43rDWrdggmiBsOWlRGtXVVaAYI4QE68u9AjwVPLIIuRpdL5A+2bEJo/6W+ZA1pPmp/gW6EwNxxpC7lf1FbM3KkFWU+EH0MHdH0uaSHAovG4/Y19wsV627nVfI8IM345gttTtzrY4+/0dzUvGLeKIoi7y4wUwOust5PuNgq6c6DjX9OQbG9lVRVEkYX9mYggo6MMLLTGnyYei6sCN0j8Z3Mx+HA/ONAjA6tLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OHeAr/1/JvOCPFjeSf6O3LGZncrh/u/f9P6dTRZkvgE=; b=v6oV6QirklPlWoPhgbtqFZ+tfZ0qRPk2ITEsAt48tVgeO+OGn2uAPGo1C2ZkkJe8asU8XeZsOsC4lPNMhteBliqLrLi5WjZ7fr8mBSMZrAI0J1OpAbengysu/LlEW7QE6q5/9545aGUaV6yEb00YjyAeJZFyxENxYhdu6Z+wQJeUQvZh5OYrTl3yf0bDXRKs1jiceCNQR+CRtTE9w6oI47ODskZXitKSH009jHC9688me8p/5bNoAAMQiXVK1giGqzqjpC2wbAcQsijLmVpEVQjVVD2bUJG65bgzzDM82UqqxkBXqj5RcGNq6eg01dCUDlwMBxpupAwDG+GCe8siFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OHeAr/1/JvOCPFjeSf6O3LGZncrh/u/f9P6dTRZkvgE=; b=YSeTAazjtj5T6rLrG6K/0t1LG7c4JwJuMHzIdka55EbEn2yMjys44QffxmdZRPxEYdpdNdCKc75M5nXiwq1DcXJUpJLyW/p0xomFFnj2Y1SiWwZqTtkbhRrehvZGp8NaeTh97VaWaBGs/SG5698QRFjrDku1OPCVzRqHmyOKPD4= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:22:10 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:22:10 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 15/16] mm: eliminate further swapops predicates Date: Mon, 10 Nov 2025 22:21:33 +0000 Message-ID: <956bc9c031604811c0070d2f4bf2f1373f230213.1762812360.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO4P265CA0261.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:37c::14) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: e3185bdc-bdc1-46c9-a811-08de20a79756 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?100XBnxb3COr2qwyFsUaPCUgUDqUQYmiw3KqThTKVPTvqZmYEyIFOEuF5t9P?= =?us-ascii?Q?bEiRNhcfMiBKFGnxH587uuDISDjzDgY1Hupmd1vGt7ruUvyglpSM4ve6BeiG?= =?us-ascii?Q?DrqZRcqtQwdSUYNiQufnmOgqqZaYKjlEES/JV/Nrc0Ni1cftO+UdGMAB08eT?= =?us-ascii?Q?MWtvACtK1rXstXFGKYzQ1etYPBx20IzifZQ49qs1bQ6nUJXG5Y4Rq07aysBq?= =?us-ascii?Q?4sgxeLPgcPlYUUMDCVug0LniaDt5701HTqar0P3OjedbVu0LSbZFy28a3BWQ?= =?us-ascii?Q?dWvfPZs8ooGdP7OjFOvD1llrhzsbK/wo2vMcwqDRm2TU5UvYdTOHT/qwJ8Zh?= =?us-ascii?Q?mE73uMLqur0lblSXfvTlTci20/SIC1B5txrSc3W/7I7jKFNvs0kcdQ5IkR5h?= =?us-ascii?Q?EoLDvrBx1GC2FloQWwpqAfIRLZRIWIfxvnYgZgRUSGoEboYat+6x9pSVRovG?= =?us-ascii?Q?UVVMFtc8R0L/4gxoRgjInIvZx1K/aDdh7JAgZHVMKRJoLbLtgi/e4xOwfycc?= =?us-ascii?Q?I0uImybcZBioJU55efbSmRJKZ390addQZYfar//bYcjhkOqhswOLbbXMGBmo?= =?us-ascii?Q?Tz4+H3LcJoAwOtZ86/+8H8WlIZMmTXKvQSA7PAimdO+alRXRALbTWaI6xCwF?= =?us-ascii?Q?jt0o57pnkvD4e64Ca32QuaBNJSXtKyxsNQkas+sr87Myo954NcxxRVWjJpbT?= =?us-ascii?Q?+LnMjPAO62WPNWNu+ndri4a3rwtpraEOoyVRsBfMT+FJrscTYJU+AmdL6BL3?= =?us-ascii?Q?JjTEujioufqr7flwnr7tnr14hFMjQ6foE5a2MWwcpOEbuWUeuU77K/G9rp/W?= =?us-ascii?Q?KhRg09VWHesxikzZvBM2R4RKuWrJlsoieSc4DIyMdIMdsJJgfx+5L0/ehnl9?= =?us-ascii?Q?MgsDEtrl7sDTcSdQw1GwGUHPLYsUUH2AD5FfCcWWsRozHvXOvsyNbZi08K2M?= =?us-ascii?Q?4YNJxCbXP4EKJOpat9yXUzW7A9eM6OavTdjAmCCw2MzcXVzY47+XloWkuo+I?= =?us-ascii?Q?Wp1b1mrhESBw7WRlna0SsZMYNTcIV/MXfcbVQMW7hiAo/KwUn1LrKPx88/az?= =?us-ascii?Q?9KdV3WJL6HNkkoR2JknNAlXCzsirq7+UFa78WYXc3+XI4ui5sPCjz96Ypznw?= =?us-ascii?Q?Hd94+XLcxu8oPvNX6xcKBZiUZovV5ZCSVnNCEUND+WZtgLgJk+N20bz9bKvT?= =?us-ascii?Q?S7CFGep827+2yYWt42p/MYV4F9Zj2GLboY2fBg4hr6WK3a5YTsCuXwTcsK5h?= =?us-ascii?Q?kRGrhmY+BgacxVEX60CNE6BD8FmcEAG/UbolkeJ1fc9UdYFeSpuYArKzIR4R?= =?us-ascii?Q?MrNE5R7WxaeWy/6YeTkVg4C1CpvzKZeCpUBAN7YIDkYGz6ODXpN4JGnlsslc?= =?us-ascii?Q?PeMNeAMRLEUbcUQiWs7VEVOMfXVcW+UwUe8C7SJHeSdY7093k2Ki7edf/Hw+?= =?us-ascii?Q?jNAwn1xX0ilD0DMcp4JTtHii55yzt1a0?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ZWwngl24VxRXIZoW2qqVX2O822Mp+GqJ5YAUCL5ryz7b/vX7tn3tIrRf1dM9?= =?us-ascii?Q?xPD2wKV3DzoQdv4B4HS1X1dqDyyE4gGb36pVQ9n0tiMK+sQLA4QtYvBN1ZGg?= =?us-ascii?Q?i0halEwoISw8il4YtrWGRsbZWvRU9/JSPyyvqlMndFjBMkeS5VFKvPY9Fob/?= =?us-ascii?Q?nmufKgeROACJQuaHNvj6ZZsNIGmxwcWi1XSCvQ+xkdeCVQ1QDpQvtW8xDmFe?= =?us-ascii?Q?1aRwDeLjdcbaiou5c0t8Hnf91m/ZRwgYuD/XtxmPCO5HycopaquY/IbC8NUL?= =?us-ascii?Q?telwTH+cWg2ZEskitI9T00pq2r/LFh7KLvg1/HGUvDfFyYIM/UVGZpc2mZEr?= =?us-ascii?Q?ToL1d+zrcK/OVMc0l//bpSjwcio/WVaWhy3NsB8txtJwxitcj66rgd8bxWsH?= =?us-ascii?Q?Vu/iBliXGrAZw0wS9tFhUbkhOejFcfA+NQ3Fmg5s8WCoWnT/AMpjJRV9XKcQ?= =?us-ascii?Q?oP5ZAdkZBvWYCVjCJshtrHVIw2ZS5cGe8YvXPk9AcBmHKeK8qERXDWoT5W2R?= =?us-ascii?Q?2EhRHlWZMFWia4A7L/LUSPtwbCNFINlgfG/7geNCuJY+jE96CH3lk4izjWNZ?= =?us-ascii?Q?feeVw/ZXXhtJYcjQc7vnq78P5ZlfX119FHol8yR7LLo2vcz+uHdo0tAkHzG4?= =?us-ascii?Q?36T5H8QXBSeBpLcRI257t77yBeS9GU0IMzc1OtjuI+MI3QV+nGWSEC9nX68f?= =?us-ascii?Q?aQrVrHVMZ+hKdiMfBLrbH56LeOX4iyVyT0uyuNtT10fbtm3uPoknkkzi84kH?= =?us-ascii?Q?Pamqes2+Y/qqrNvNiTnETDWCrfeGPrprAkEb0lTyf2M8gGN8qDfE2FGSbROw?= =?us-ascii?Q?W2vm9tiw+4bUNIwFKSEIio1vJHWs5IrQ15JMtH/cq4QgPJ8mNSg394+LQnGv?= =?us-ascii?Q?xZKAVXhajolEUlBm0XIl+ewG73E4Gizvu/0JOPaJJMRkOIUMBbxFeCmV1wAS?= =?us-ascii?Q?bxHmHE0dJxza00pJvUXHCvzhzxvODPgdKhkbyi/xDgGTWAfMNXOsQ16DvcgN?= =?us-ascii?Q?jqfvTd2I4oqNLLzUkLaBV3tKG7L4jLSA8pSdW+v+91G+mJoed/ZIrTXLFLsE?= =?us-ascii?Q?Re1zykltFlG2BCYKX1VLYx50o7pnHxd58hG6UQm/Cjo6ywBEe5AJmdDcu+7g?= =?us-ascii?Q?/Qk3MNT9PNG/KJ5TFxYOl1b1SYliSp19b30oGlyvRt5ZJzOdxkr1SMI9MVFY?= =?us-ascii?Q?LyWOF4mlbVXwoZpupnXW+8vAyp5Y9tdm18SIfHVGiyQU21Vtmzldtw8psOAV?= =?us-ascii?Q?B+ZxXr4+arc7/w5MptY1lqKwH3zB4QwajbH2u8Rp2TcDgoS2+2xxQk3OdpP3?= =?us-ascii?Q?7w975IRK6j5xLhIIeAykV0qpnhTIBItJvu2fgvxh0cvMfJ6DnLotzbsf8WFf?= =?us-ascii?Q?vM3MBIEy5YgGNWbhsuGhBvdQlEWDqlvhNLTS/MWxbOZotlyMo6H+OhqAX9ua?= =?us-ascii?Q?NkusI1g2KhrKuL55D7Fbo+7l7biVyUYT9L2WhGV909V+53nkLIZVomE2YAUL?= =?us-ascii?Q?+YvIl3+Dx+o3WzxWia04Iy6g5Tm5mMEjRMFgVXssOKaI8ResCC+6NiCkge7o?= =?us-ascii?Q?+e/Y+zlV1fCyayLdYC+zv+PulnUz6rEEjYiNlZYBg8a/cI5XlJacb7hBLTPB?= =?us-ascii?Q?xA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: rlwtzLa4it8zbi16vyzsXcviQsAugjm8rzOv5Uu5HSa7xIZCfXupfEbjGYLIVwdgsBotloaNhpIbuYsGSBLmYcAq5VMeroO0py0g62LMBxARNQQbJSxiT0EQY39OaPHEovISDRYaV2nAsZyjzdRdIYAGLCJSsSetwtomxNCukz+85L8GPN8xXuWVoMiHGcyCm0RHQ7dwmnriPaYHTO0rRoteDPVAE40sTbEB7S/sg9I79M71EmPAC6RXuCCV12/vc6+mofPXPVGyd/axf7qQiU5er29jeqNs559Py8VOwB/WDklASaVS9PzeH4+yXvbhJlKxotHfQTg2vu4XAG/dIOyPTUtBwYMRntDXr8meRFW0CH8DPWN+dMZxbMGpTSrkEgYb25xR/HCVTobnMsM8WsBHaRHSRuDBp6lsb3m/Lz7svEmQcYjE0QGX2sUKjimgPZ0aTNdapXz3Aj3VwNPsbbKO19X+Dl8KG0wBebBAK2RV9hT+1+bszxjQ54LpcGmNLor0KofAVms2v210vkjU6wDXHguB4DE3agRx5/Ph2MD0Jv1XSkuVk0NTI+yV8aKVPCg1ulNZj/4JFMZdX569igW/ufDjNa1+n/Uswh62BM0= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e3185bdc-bdc1-46c9-a811-08de20a79756 X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:22:10.5456 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uwgpAIP5UFfan02YOewxZrccqPz81mFW25ddch98VqUM4F+VUk1GvmXLL1G/WQ3BfNQ3dr1u0Oc9yNI/avR+JldRalazWkNyDHowHGUjpPw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 adultscore=0 malwarescore=0 spamscore=0 suspectscore=0 bulkscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE3OSBTYWx0ZWRfX5krCvsOrQdyT p6WLJFoKWvscAcHAGZ+7/wYGJPlCidMilQxrVNga+E8Xwzfl8DCkuFxOQH7XqXE+Jl/33XxKi/e eMw//nk6njp22D2AQw3kcZIndlrrTl24F0/+DhSed1+iNqFPNCxlYNlvQKCv7hxD1kLenVEVZxk SgIEHXPbp4hN3TrzY/YFnVnZhOuDgGUJ3KUM5vTmLnaJpALKEEksMzAG40KYwhtYv/TQtcmKdQ+ 0K1r/QIO8GKONcYTlyT7tPZtnC3boPGZnlPOy1kg2xTdiwHoDxGsZy+S6e4ObEBE/nlcBBy4jqi U0CHXlN0N/U9FmCCaP7jYCd6KBR47w7vIarnNQDW58oFv/ljaWyNXYFLLR0U02W0q0iRmCowC06 Mekje+f8iPIcURYnGz3KwMIwBcXe1htQ467kCr2YTzRtlPlhJN0= X-Authority-Analysis: v=2.4 cv=H5rWAuYi c=1 sm=1 tr=0 ts=69126599 b=1 cx=c_pps a=e1sVV491RgrpLwSTMOnk8w==:117 a=e1sVV491RgrpLwSTMOnk8w==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=12TsxHtCQ5kFwp23Kd0A:9 cc=ntf awl=host:13634 X-Proofpoint-ORIG-GUID: n25A8vz2vEqTzeAjjkp9rrzYWGQRZrsj X-Proofpoint-GUID: n25A8vz2vEqTzeAjjkp9rrzYWGQRZrsj Content-Type: text/plain; charset="utf-8" Having converted so much of the code base to software leaf entries, we can mop up some remaining cases. We replace is_pfn_swap_entry(), pfn_swap_entry_to_page(), is_writable_device_private_entry(), is_device_exclusive_entry(), is_migration_entry(), is_writable_migration_entry(), is_readable_migration_entry(), swp_offset_pfn() and pfn_swap_entry_folio() with softleaf equivalents. No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- fs/proc/task_mmu.c | 14 ++--- include/linux/leafops.h | 25 +++++++-- include/linux/swapops.h | 121 +--------------------------------------- mm/debug_vm_pgtable.c | 20 +++---- mm/hmm.c | 2 +- mm/hugetlb.c | 2 +- mm/ksm.c | 6 +- mm/memory-failure.c | 6 +- mm/memory.c | 3 +- mm/mempolicy.c | 4 +- mm/migrate.c | 6 +- mm/migrate_device.c | 10 ++-- mm/mprotect.c | 8 +-- mm/page_vma_mapped.c | 8 +-- mm/pagewalk.c | 7 +-- mm/rmap.c | 9 ++- 16 files changed, 75 insertions(+), 176 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 3cdefa7546db..4deded872c46 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1940,13 +1940,13 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, if (pte_uffd_wp(pte)) flags |=3D PM_UFFD_WP; } else { - swp_entry_t entry; + softleaf_t entry; =20 if (pte_swp_soft_dirty(pte)) flags |=3D PM_SOFT_DIRTY; if (pte_swp_uffd_wp(pte)) flags |=3D PM_UFFD_WP; - entry =3D pte_to_swp_entry(pte); + entry =3D softleaf_from_pte(pte); if (pm->show_pfn) { pgoff_t offset; =20 @@ -1954,16 +1954,16 @@ static pagemap_entry_t pte_to_pagemap_entry(struct = pagemapread *pm, * For PFN swap offsets, keeping the offset field * to be PFN only to be compatible with old smaps. */ - if (is_pfn_swap_entry(entry)) - offset =3D swp_offset_pfn(entry); + if (softleaf_has_pfn(entry)) + offset =3D softleaf_to_pfn(entry); else offset =3D swp_offset(entry); frame =3D swp_type(entry) | (offset << MAX_SWAPFILES_SHIFT); } flags |=3D PM_SWAP; - if (is_pfn_swap_entry(entry)) - page =3D pfn_swap_entry_to_page(entry); + if (softleaf_has_pfn(entry)) + page =3D softleaf_to_page(entry); if (softleaf_is_uffd_wp_marker(entry)) flags |=3D PM_UFFD_WP; if (softleaf_is_guard_marker(entry)) @@ -2032,7 +2032,7 @@ static int pagemap_pmd_range_thp(pmd_t *pmdp, unsigne= d long addr, if (pmd_swp_uffd_wp(pmd)) flags |=3D PM_UFFD_WP; VM_WARN_ON_ONCE(!pmd_is_migration_entry(pmd)); - page =3D pfn_swap_entry_to_page(entry); + page =3D softleaf_to_page(entry); } =20 if (page) { diff --git a/include/linux/leafops.h b/include/linux/leafops.h index f5ea9b0385ff..d282fab866a1 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -355,7 +355,7 @@ static inline unsigned long softleaf_to_pfn(softleaf_t = entry) VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); =20 /* Temporary until swp_entry_t eliminated. */ - return swp_offset_pfn(entry); + return swp_offset(entry) & SWP_PFN_MASK; } =20 /** @@ -366,10 +366,16 @@ static inline unsigned long softleaf_to_pfn(softleaf_= t entry) */ static inline struct page *softleaf_to_page(softleaf_t entry) { + struct page *page =3D pfn_to_page(softleaf_to_pfn(entry)); + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + /* + * Any use of migration entries may only occur while the + * corresponding page is locked + */ + VM_WARN_ON_ONCE(softleaf_is_migration(entry) && !PageLocked(page)); =20 - /* Temporary until swp_entry_t eliminated. */ - return pfn_swap_entry_to_page(entry); + return page; } =20 /** @@ -380,10 +386,17 @@ static inline struct page *softleaf_to_page(softleaf_= t entry) */ static inline struct folio *softleaf_to_folio(softleaf_t entry) { - VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + struct folio *folio =3D pfn_folio(softleaf_to_pfn(entry)); =20 - /* Temporary until swp_entry_t eliminated. */ - return pfn_swap_entry_folio(entry); + VM_WARN_ON_ONCE(!softleaf_has_pfn(entry)); + /* + * Any use of migration entries may only occur while the + * corresponding folio is locked. + */ + VM_WARN_ON_ONCE(softleaf_is_migration(entry) && + !folio_test_locked(folio)); + + return folio; } =20 /** diff --git a/include/linux/swapops.h b/include/linux/swapops.h index c8e6f927da48..3d02b288c15e 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -28,7 +28,7 @@ #define SWP_OFFSET_MASK ((1UL << SWP_TYPE_SHIFT) - 1) =20 /* - * Definitions only for PFN swap entries (see is_pfn_swap_entry()). To + * Definitions only for PFN swap entries (see leafeant_has_pfn()). To * store PFN, we only need SWP_PFN_BITS bits. Each of the pfn swap entries * can use the extra bits to store other information besides PFN. */ @@ -66,8 +66,6 @@ #define SWP_MIG_YOUNG BIT(SWP_MIG_YOUNG_BIT) #define SWP_MIG_DIRTY BIT(SWP_MIG_DIRTY_BIT) =20 -static inline bool is_pfn_swap_entry(swp_entry_t entry); - /* Clear all flags but only keep swp_entry_t related information */ static inline pte_t pte_swp_clear_flags(pte_t pte) { @@ -109,17 +107,6 @@ static inline pgoff_t swp_offset(swp_entry_t entry) return entry.val & SWP_OFFSET_MASK; } =20 -/* - * This should only be called upon a pfn swap entry to get the PFN stored - * in the swap entry. Please refers to is_pfn_swap_entry() for definition - * of pfn swap entry. - */ -static inline unsigned long swp_offset_pfn(swp_entry_t entry) -{ - VM_BUG_ON(!is_pfn_swap_entry(entry)); - return swp_offset(entry) & SWP_PFN_MASK; -} - /* * Convert the arch-dependent pte representation of a swp_entry_t into an * arch-independent swp_entry_t. @@ -169,27 +156,11 @@ static inline swp_entry_t make_writable_device_privat= e_entry(pgoff_t offset) return swp_entry(SWP_DEVICE_WRITE, offset); } =20 -static inline bool is_device_private_entry(swp_entry_t entry) -{ - int type =3D swp_type(entry); - return type =3D=3D SWP_DEVICE_READ || type =3D=3D SWP_DEVICE_WRITE; -} - -static inline bool is_writable_device_private_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_DEVICE_WRITE); -} - static inline swp_entry_t make_device_exclusive_entry(pgoff_t offset) { return swp_entry(SWP_DEVICE_EXCLUSIVE, offset); } =20 -static inline bool is_device_exclusive_entry(swp_entry_t entry) -{ - return swp_type(entry) =3D=3D SWP_DEVICE_EXCLUSIVE; -} - #else /* CONFIG_DEVICE_PRIVATE */ static inline swp_entry_t make_readable_device_private_entry(pgoff_t offse= t) { @@ -201,50 +172,14 @@ static inline swp_entry_t make_writable_device_privat= e_entry(pgoff_t offset) return swp_entry(0, 0); } =20 -static inline bool is_device_private_entry(swp_entry_t entry) -{ - return false; -} - -static inline bool is_writable_device_private_entry(swp_entry_t entry) -{ - return false; -} - static inline swp_entry_t make_device_exclusive_entry(pgoff_t offset) { return swp_entry(0, 0); } =20 -static inline bool is_device_exclusive_entry(swp_entry_t entry) -{ - return false; -} - #endif /* CONFIG_DEVICE_PRIVATE */ =20 #ifdef CONFIG_MIGRATION -static inline int is_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ || - swp_type(entry) =3D=3D SWP_MIGRATION_READ_EXCLUSIVE || - swp_type(entry) =3D=3D SWP_MIGRATION_WRITE); -} - -static inline int is_writable_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_WRITE); -} - -static inline int is_readable_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ); -} - -static inline int is_readable_exclusive_migration_entry(swp_entry_t entry) -{ - return unlikely(swp_type(entry) =3D=3D SWP_MIGRATION_READ_EXCLUSIVE); -} =20 static inline swp_entry_t make_readable_migration_entry(pgoff_t offset) { @@ -310,23 +245,10 @@ static inline swp_entry_t make_writable_migration_ent= ry(pgoff_t offset) return swp_entry(0, 0); } =20 -static inline int is_migration_entry(swp_entry_t swp) -{ - return 0; -} - static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address) { } static inline void migration_entry_wait_huge(struct vm_area_struct *vma, unsigned long addr, pte_t *pte) { } -static inline int is_writable_migration_entry(swp_entry_t entry) -{ - return 0; -} -static inline int is_readable_migration_entry(swp_entry_t entry) -{ - return 0; -} =20 static inline swp_entry_t make_migration_entry_young(swp_entry_t entry) { @@ -410,47 +332,6 @@ static inline swp_entry_t make_guard_swp_entry(void) return make_pte_marker_entry(PTE_MARKER_GUARD); } =20 -static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) -{ - struct page *p =3D pfn_to_page(swp_offset_pfn(entry)); - - /* - * Any use of migration entries may only occur while the - * corresponding page is locked - */ - BUG_ON(is_migration_entry(entry) && !PageLocked(p)); - - return p; -} - -static inline struct folio *pfn_swap_entry_folio(swp_entry_t entry) -{ - struct folio *folio =3D pfn_folio(swp_offset_pfn(entry)); - - /* - * Any use of migration entries may only occur while the - * corresponding folio is locked - */ - BUG_ON(is_migration_entry(entry) && !folio_test_locked(folio)); - - return folio; -} - -/* - * A pfn swap entry is a special type of swap entry that always has a pfn = stored - * in the swap offset. They can either be used to represent unaddressable = device - * memory, to restrict access to a page undergoing migration or to represe= nt a - * pfn which has been hwpoisoned and unmapped. - */ -static inline bool is_pfn_swap_entry(swp_entry_t entry) -{ - /* Make sure the swp offset can always store the needed fields */ - BUILD_BUG_ON(SWP_TYPE_SHIFT < SWP_PFN_BITS); - - return is_migration_entry(entry) || is_device_private_entry(entry) || - is_device_exclusive_entry(entry) || is_hwpoison_entry(entry); -} - struct page_vma_mapped_walk; =20 #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 608d1011ce03..64db85a80558 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -844,7 +844,7 @@ static void __init pmd_softleaf_tests(struct pgtable_de= bug_args *args) { } static void __init swap_migration_tests(struct pgtable_debug_args *args) { struct page *page; - swp_entry_t swp; + softleaf_t entry; =20 if (!IS_ENABLED(CONFIG_MIGRATION)) return; @@ -867,17 +867,17 @@ static void __init swap_migration_tests(struct pgtabl= e_debug_args *args) * be locked, otherwise it stumbles upon a BUG_ON(). */ __SetPageLocked(page); - swp =3D make_writable_migration_entry(page_to_pfn(page)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(!is_writable_migration_entry(swp)); + entry =3D make_writable_migration_entry(page_to_pfn(page)); + WARN_ON(!softleaf_is_migration(entry)); + WARN_ON(!softleaf_is_migration_write(entry)); =20 - swp =3D make_readable_migration_entry(swp_offset(swp)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(is_writable_migration_entry(swp)); + entry =3D make_readable_migration_entry(swp_offset(entry)); + WARN_ON(!softleaf_is_migration(entry)); + WARN_ON(softleaf_is_migration_write(entry)); =20 - swp =3D make_readable_migration_entry(page_to_pfn(page)); - WARN_ON(!is_migration_entry(swp)); - WARN_ON(is_writable_migration_entry(swp)); + entry =3D make_readable_migration_entry(page_to_pfn(page)); + WARN_ON(!softleaf_is_migration(entry)); + WARN_ON(softleaf_is_migration_write(entry)); __ClearPageLocked(page); } =20 diff --git a/mm/hmm.c b/mm/hmm.c index 0158f2d1e027..3912d92a2b9a 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -270,7 +270,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, uns= igned long addr, cpu_flags =3D HMM_PFN_VALID; if (softleaf_is_device_private_write(entry)) cpu_flags |=3D HMM_PFN_WRITE; - new_pfn_flags =3D swp_offset_pfn(entry) | cpu_flags; + new_pfn_flags =3D softleaf_to_pfn(entry) | cpu_flags; goto out; } =20 diff --git a/mm/hugetlb.c b/mm/hugetlb.c index b702b161ab35..f7f18a3ea495 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5640,7 +5640,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, } else if (unlikely(softleaf_is_migration(softleaf))) { bool uffd_wp =3D pte_swp_uffd_wp(entry); =20 - if (!is_readable_migration_entry(softleaf) && cow) { + if (!softleaf_is_migration_read(softleaf) && cow) { /* * COW mappings require pages in both * parent and child to be set to read. diff --git a/mm/ksm.c b/mm/ksm.c index 7cd19a6ce45f..b911df37f04e 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -637,14 +637,14 @@ static int break_ksm_pmd_entry(pmd_t *pmdp, unsigned = long addr, unsigned long en if (pte_present(pte)) { folio =3D vm_normal_folio(walk->vma, addr, pte); } else if (!pte_none(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 /* * As KSM pages remain KSM pages until freed, no need to wait * here for migration to end. */ - if (is_migration_entry(entry)) - folio =3D pfn_swap_entry_folio(entry); + if (softleaf_is_migration(entry)) + folio =3D softleaf_to_folio(entry); } /* return 1 if the page is an normal ksm page or KSM-placed zero page */ found =3D (folio && folio_test_ksm(folio)) || is_ksm_zero_pte(pte); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index acc35c881547..6e79da3de221 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -691,10 +691,10 @@ static int check_hwpoisoned_entry(pte_t pte, unsigned= long addr, short shift, if (pte_present(pte)) { pfn =3D pte_pfn(pte); } else { - swp_entry_t swp =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 - if (is_hwpoison_entry(swp)) - pfn =3D swp_offset_pfn(swp); + if (softleaf_is_hwpoison(entry)) + pfn =3D softleaf_to_pfn(entry); } =20 if (!pfn || pfn !=3D poisoned_pfn) diff --git a/mm/memory.c b/mm/memory.c index ad336cbf1d88..accd275cd651 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -902,7 +902,8 @@ static void restore_exclusive_pte(struct vm_area_struct= *vma, static int try_restore_exclusive_pte(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t orig_pte) { - struct page *page =3D pfn_swap_entry_to_page(pte_to_swp_entry(orig_pte)); + const softleaf_t entry =3D softleaf_from_pte(orig_pte); + struct page *page =3D softleaf_to_page(entry); struct folio *folio =3D page_folio(page); =20 if (folio_trylock(folio)) { diff --git a/mm/mempolicy.c b/mm/mempolicy.c index dee95d5ecfd4..acb9bf89f619 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -705,7 +705,9 @@ static int queue_folios_pte_range(pmd_t *pmd, unsigned = long addr, if (pte_none(ptent)) continue; if (!pte_present(ptent)) { - if (is_migration_entry(pte_to_swp_entry(ptent))) + const softleaf_t entry =3D softleaf_from_pte(ptent); + + if (softleaf_is_migration(entry)) qp->nr_failed++; continue; } diff --git a/mm/migrate.c b/mm/migrate.c index 48f98a6c1ad2..182a5b7b2ead 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -483,7 +483,7 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, spinlock_t *ptl; pte_t *ptep; pte_t pte; - swp_entry_t entry; + softleaf_t entry; =20 ptep =3D pte_offset_map_lock(mm, pmd, address, &ptl); if (!ptep) @@ -495,8 +495,8 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *= pmd, if (pte_none(pte) || pte_present(pte)) goto out; =20 - entry =3D pte_to_swp_entry(pte); - if (!is_migration_entry(entry)) + entry =3D softleaf_from_pte(pte); + if (!softleaf_is_migration(entry)) goto out; =20 migration_entry_wait_on_locked(entry, ptl); diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 880f26a316f8..c50abbd32f21 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -282,7 +282,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, unsigned long mpfn =3D 0, pfn; struct folio *folio; struct page *page; - swp_entry_t entry; + softleaf_t entry; pte_t pte; =20 pte =3D ptep_get(ptep); @@ -301,11 +301,11 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, * page table entry. Other special swap entries are not * migratable, and we ignore regular swapped page. */ - entry =3D pte_to_swp_entry(pte); - if (!is_device_private_entry(entry)) + entry =3D softleaf_from_pte(pte); + if (!softleaf_is_device_private(entry)) goto next; =20 - page =3D pfn_swap_entry_to_page(entry); + page =3D softleaf_to_page(entry); pgmap =3D page_pgmap(page); if (!(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_PRIVATE) || @@ -331,7 +331,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, =20 mpfn =3D migrate_pfn(page_to_pfn(page)) | MIGRATE_PFN_MIGRATE; - if (is_writable_device_private_entry(entry)) + if (softleaf_is_device_private_write(entry)) mpfn |=3D MIGRATE_PFN_WRITE; } else { pfn =3D pte_pfn(pte); diff --git a/mm/mprotect.c b/mm/mprotect.c index ab014ce17f9c..476a29cc89bf 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -317,11 +317,11 @@ static long change_pte_range(struct mmu_gather *tlb, pages++; } } else { - swp_entry_t entry =3D pte_to_swp_entry(oldpte); + softleaf_t entry =3D softleaf_from_pte(oldpte); pte_t newpte; =20 - if (is_writable_migration_entry(entry)) { - struct folio *folio =3D pfn_swap_entry_folio(entry); + if (softleaf_is_migration_write(entry)) { + const struct folio *folio =3D softleaf_to_folio(entry); =20 /* * A protection check is difficult so @@ -335,7 +335,7 @@ static long change_pte_range(struct mmu_gather *tlb, newpte =3D swp_entry_to_pte(entry); if (pte_swp_soft_dirty(oldpte)) newpte =3D pte_swp_mksoft_dirty(newpte); - } else if (is_writable_device_private_entry(entry)) { + } else if (softleaf_is_device_private_write(entry)) { /* * We do not preserve soft-dirtiness. See * copy_nonpresent_pte() for explanation. diff --git a/mm/page_vma_mapped.c b/mm/page_vma_mapped.c index 8137d2366722..b38a1d00c971 100644 --- a/mm/page_vma_mapped.c +++ b/mm/page_vma_mapped.c @@ -49,7 +49,7 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, if (is_migration) return false; } else if (!is_migration) { - swp_entry_t entry; + softleaf_t entry; =20 /* * Handle un-addressable ZONE_DEVICE memory. @@ -67,9 +67,9 @@ static bool map_pte(struct page_vma_mapped_walk *pvmw, pm= d_t *pmdvalp, * For more details on device private memory see HMM * (include/linux/hmm.h or mm/hmm.c). */ - entry =3D pte_to_swp_entry(ptent); - if (!is_device_private_entry(entry) && - !is_device_exclusive_entry(entry)) + entry =3D softleaf_from_pte(ptent); + if (!softleaf_is_device_private(entry) && + !softleaf_is_device_exclusive(entry)) return false; } spin_lock(*ptlp); diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 3067feb970d1..d6e29da60d09 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -1000,11 +1000,10 @@ struct folio *folio_walk_start(struct folio_walk *f= w, goto found; } } else if (!pte_none(pte)) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); =20 - if ((flags & FW_MIGRATION) && - is_migration_entry(entry)) { - page =3D pfn_swap_entry_to_page(entry); + if ((flags & FW_MIGRATION) && softleaf_is_migration(entry)) { + page =3D softleaf_to_page(entry); expose_page =3D false; goto found; } diff --git a/mm/rmap.c b/mm/rmap.c index 775710115a41..345466ad396b 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1969,7 +1969,7 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D softleaf_to_pfn(pte_to_swp_entry(pteval)); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2368,7 +2368,7 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D swp_offset_pfn(pte_to_swp_entry(pteval)); + pfn =3D softleaf_to_pfn(pte_to_swp_entry(pteval)); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2453,8 +2453,11 @@ static bool try_to_migrate_one(struct folio *folio, = struct vm_area_struct *vma, folio_mark_dirty(folio); writable =3D pte_write(pteval); } else { + const softleaf_t entry =3D softleaf_from_pte(pteval); + pte_clear(mm, address, pvmw.pte); - writable =3D is_writable_device_private_entry(pte_to_swp_entry(pteval)); + + writable =3D softleaf_is_device_private_write(entry); } =20 VM_WARN_ON_FOLIO(writable && folio_test_anon(folio) && --=20 2.51.0 From nobody Fri Dec 19 21:47:08 2025 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E4002E62C4; Mon, 10 Nov 2025 22:23:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813413; cv=fail; b=rDwVPZASW8hgW3v5qjrjFdBSrsRkg+T6j8YelLCxV61gOu4c2Hm+ONa8MF6ZhDsnrvdrRlFMtomhTb2s5kk9uMRYjFwet1jAI81EohkFWxFiyc3HtYT+PMSlxwCNxPx3aUnLDo5ZZG//EESTqihpLkNsjt2MQWrJf8XuT+N1cjQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762813413; c=relaxed/simple; bh=BwtqsbOZ/zTXiFh3bRPTEI8r0LJSVuqNgZCQjdJzkIU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=hiHcVAxHuAzH0tIrktud0MXciaGdxOKOMapp1X4Eaoaz7tE8vYd1f/9wnZ/X+qQXjDGfMv1Afi2YhPgOXe/MA1O93vTK5ysIehtRRcd9vDQDa+m+rGXririoQ10VKkTFDB41naenW0n/nVieIb6YQMS2Umvq+nNWR4+KPIK9tXI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=UYitXxCx; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=PGbtcf8U; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="UYitXxCx"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="PGbtcf8U" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5AAKcAFO002920; Mon, 10 Nov 2025 22:22:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2025-04-25; bh=IZbC4u1/g233rEO38PGwbP9yIpeb5PXNM3qTia8WOz0=; b= UYitXxCx6rCxQpu1kf6MpkYcDbP0Mtzq3Mqix6xKJ1a/Mf2usZ9Flpn6SBH/qGns IOy8p41oI8JQKAIWySh06oB3dKUi9EZzJn3i+9MQsg0jwBhjRIbsuQ+w7Dyd1B10 cBjUunwScmL8cpkpvUYi2Wx8WcY4xjr2IeMTVuMO5z0HQxMXpUKfXPbNaCM/u9P2 PvI8aCbEDKRcmfyMPoHnIWkocEkDb7PwK0CXxEwjd20+IxNg7gJgOyvxe7Vm0EMC GF+f8Y2e2mv/RX9sN+f1ixoSvny1eM3QEPWYSxZ0hLikBkeU218KdYpo0WJ4F2P0 cMeBzJb7KFPPLm4z0bvDLw== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 4abpmn0b8n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:18 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 5AALsCKe000873; Mon, 10 Nov 2025 22:22:17 GMT Received: from co1pr03cu002.outbound.protection.outlook.com (mail-westus2azon11010031.outbound.protection.outlook.com [52.101.46.31]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 4a9vacs4p9-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Nov 2025 22:22:17 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=M9LaLXlRrSSsqCVmlMam8Z8KyCSncfGlnQ4+5hFZWw1aq4jI/Xt5yOCisLRp5Zpx/fnctCRlYq2pc/wpQjMp08yn84DITu6AAXH8yK9OYBM2fhas2LorRxnYPFYwVyWarhL1ozKOhSG0ASbempW5x9gmo7Qq+zqOnJNgP8LtIeLTwy+4xcEHLxsJW7v+klTTK1N65kcR8BKMTSNAlX7IXztIFUxj4q/4CjaESGfQAabP12muCetm4871pOeP2Uy+FNgCTA3XwDJxAsHOkXyK03TdG9s22xvy+mh6iYSxf4S0Rh7nUdTNBMt9zihxQgzxrtEaFuxdp8HSmuNVQyFezw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IZbC4u1/g233rEO38PGwbP9yIpeb5PXNM3qTia8WOz0=; b=vgVy3oqmePPT8ZKmc7g4hv12nsw4peXX82ATW1lBdiDniwUuxlT3osVNwWhZ7+PGVvRc7/4SJySBvZjmdyUGp6my5qNZGW3+Kn7ljN8AYeOEibkA3qSeJ1+nglQaVE3NNfbZzdUK662P7tlpAza7Yx28r6PAgWb4gAdXfDX7lvANQc0Dmx9b56YPv4SHb6YUDHQCMEo5aoCDUT5Jl9m3XWWgnD9vdlDtwRxRw6kEmARrWS5axUkh2dNEIg8/uUt08Jqktbg6yzgZlocnatIKOP7kZyq6GwT1U0yCNBoU/28ubi6sLYdbFjnHWlRrCRYXw5W7dxKSl4odKsEGkP/EbQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IZbC4u1/g233rEO38PGwbP9yIpeb5PXNM3qTia8WOz0=; b=PGbtcf8UhL2ejNSgT6MJDo9X8erDGtDXTfdxTk7UoRYVCVuLe+FiTakpAlbKRCWxG8Mq70LZyaqo7CZfrREZxEymo12KKyKzL70aqAUwrBkwU3zg62c5vQOtOjYe11NJqhtg36JuXPuEjeRywMZGb00KWhSO5WJhdeIBgIwH3pY= Received: from DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) by IA0PR10MB7622.namprd10.prod.outlook.com (2603:10b6:208:483::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9298.16; Mon, 10 Nov 2025 22:22:12 +0000 Received: from DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2]) by DM4PR10MB8218.namprd10.prod.outlook.com ([fe80::2650:55cf:2816:5f2%7]) with mapi id 15.20.9298.010; Mon, 10 Nov 2025 22:22:12 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: Christian Borntraeger , Janosch Frank , Claudio Imbrenda , David Hildenbrand , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Sven Schnelle , Peter Xu , Alexander Viro , Christian Brauner , Jan Kara , Arnd Bergmann , Zi Yan , Baolin Wang , "Liam R . Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Muchun Song , Oscar Salvador , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Axel Rasmussen , Yuanchu Xie , Wei Xu , Kemeng Shi , Kairui Song , Nhat Pham , Baoquan He , Chris Li , SeongJae Park , Matthew Wilcox , Jason Gunthorpe , Leon Romanovsky , Xu Xin , Chengming Zhou , Jann Horn , Miaohe Lin , Naoya Horiguchi , Pedro Falcato , Pasha Tatashin , Rik van Riel , Harry Yoo , Hugh Dickins , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, damon@lists.linux.dev Subject: [PATCH v3 16/16] mm: replace remaining pte_to_swp_entry() with softleaf_from_pte() Date: Mon, 10 Nov 2025 22:21:34 +0000 Message-ID: X-Mailer: git-send-email 2.51.0 In-Reply-To: References: Content-Transfer-Encoding: quoted-printable X-ClientProxiedBy: LO2P265CA0310.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a5::34) To DM4PR10MB8218.namprd10.prod.outlook.com (2603:10b6:8:1cc::16) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM4PR10MB8218:EE_|IA0PR10MB7622:EE_ X-MS-Office365-Filtering-Correlation-Id: 25aa71ee-38fa-4312-2a1c-08de20a7988a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?E37ctphzNvedEdhPq1SNKxdKYndnwxvbFk5Sb5HhFBukiOB3IiQ3KIwKf0B5?= =?us-ascii?Q?PbjuYOlCOxrUiOnYxSX9Y/Jp9m7Mb/hgrhFef02SWckL0pqpmgsFUjHD3E7C?= =?us-ascii?Q?GE3EkF/qPr1JOoATL4HNstjGkAuWRfFbTLSQplEPLoKI7ltej6k2hqGO2MVi?= =?us-ascii?Q?r8g3hR4j8AXDCzzaRy6sLs4kU8wi5M17nLGnQt44vEK4hnY+qDjUJR+BMSEm?= =?us-ascii?Q?yGWg+JbkKwPHGE3ij8BjXS2nYh6rde8HcpxvqZuNXYGoXewEqpQCpvXkJcY+?= =?us-ascii?Q?7YV3geK3tqN+44LplNZbZ7pn9U+IC2r21/6x5L+0KG/RJJ4UcZsHLrK2GSA+?= =?us-ascii?Q?4oSjIZAkewWy+Dw845QlUTtO6JQrEkaXUIwLXzFxWtEoeffxi/qKBk+7COFi?= =?us-ascii?Q?HkacMam0OBFxaNgFuUvOtLVnLBfvkxkl6HYBDeB5XNHVm0RiWm6CiSOnKXU8?= =?us-ascii?Q?3p+EbNck8YwhfK9Jm+3Qs+QcNSAEtftVwb0WLJkkvtZKb2Yn7+B8/NOJLyRo?= =?us-ascii?Q?za/QNzO+EOdIrSNjQbMYeTWFbFAAMMjtqHYmxJUqXfm/yQVZUezu8l/z0t1P?= =?us-ascii?Q?yO7Dl2/SLPYDFfVgXWSbp9AV9LQvJdYYOzsTvcgXzFnTqF3pqPn6jCNXPow/?= =?us-ascii?Q?1qfRa4hGf8C9fBpm6LMI6zwatrN5ArKCZkfwR4iO9WS7yeSavBGDTqT3lchB?= =?us-ascii?Q?IYuy+ufQKHo8sa9oLklAm/Gpfp8fvlTIxKlCKKlNDVrAJKgeTmW1hYPymuvO?= =?us-ascii?Q?x42GzHL3zYr/fKnjv5AYAk4i9mAH9p+bKJeuCrhfpu4UpamecElT78zS/cbv?= =?us-ascii?Q?eaFsULagvgZkEd+qbSHjpzghg2kA+k23/7dXwiNj9i6yy5Mh4CeIAdzLD7j7?= =?us-ascii?Q?/vOYCZglhNsybGlezHHQPlP1KBWS72qDyQZJu3xxIpvohdK4QUecNFXrNrzs?= =?us-ascii?Q?ZcYG9D2k71XI1+gRwdYPOMPoQZS4UWoycCSL9JJtMUIMcHD7rvbpHzwSmnuu?= =?us-ascii?Q?r00AQJctbUfpReDEMZA6q1y8+GkYW3WgwbdGS8npmnR8zHD/Dex/IzDDAoaU?= =?us-ascii?Q?kS3wTM3DfMtUdUZfRubaU0bc4FwDbbq86ifX8+PGu4PmfsvB7A3DpLRW76AM?= =?us-ascii?Q?thFXc+qdxR4Lz0BNMnp3qo5Rg7sDQgVs1F0e/mTqsszolmHfEZpbhjkH0MIb?= =?us-ascii?Q?NMmmHbIa5GLleJW6JfDiog+tB7YhLobM4g6pF5M/XI8AtkyiI2pMduJkNFyX?= =?us-ascii?Q?9vqRTkuIufoytGrkjio4Aczx2XlFU+sBA+4LQi7iqCvFlqToA2P3HK48ukRN?= =?us-ascii?Q?2WmmEuvV8613hKDRNUN2KF1x4zO5iS86v8wppVWi0BrlbTz2owY0fJxeodrh?= =?us-ascii?Q?9aZclfv7eyJEM6FhB27HwnbR1U2t4ti/8IxcUUDbEZKwRqzPSlAe8kdxOR58?= =?us-ascii?Q?PLGxmUgWdhAebTL9rIjhO9qKQoi0OzN5?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR10MB8218.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?VmqL8Rz1uDsGgqrASahg5QW/1MjG+KMrurFseiRkrQdl0B7wC+LVaddZpCiY?= =?us-ascii?Q?MLDRhvt/ErKuaw5fMY91NAnOo2Rb+ekosNRKcZtXz6kiqV4uKovIE3n1m7fz?= =?us-ascii?Q?OdcPysFcK9+OEJryKy5e7w8MhskzeRBWz+iiuGaIB0eSB3SpGUFUNPYKRHRa?= =?us-ascii?Q?rpf7sij9PPl0TyJhBzTeAZGy9kAlUDPt1jGKZBNfl6O/l8vzU6gxB9tJu0Z6?= =?us-ascii?Q?BM3vpVJJi1h7ULxLuJ6Xe2IfXDIUXTU6DDPQZ7WgP6NJeEACr4MgLqPP7F59?= =?us-ascii?Q?ZcDaugV8acAKIiDKJzcxquF/M7CjmYoIn9NxGK8NH0CFfo6fwLMxsDQZsaxx?= =?us-ascii?Q?eoJMCFjr9WuXG9tp6IUmZDG++RuwPmEHwcRyf+Y8yK4umOp5TsIOXBMBHRlO?= =?us-ascii?Q?pzQy6tBt/4i2JSIee3rXwRtJbC6wAB5oygUVjCBmmdK50U0skQRlU6dBj8Kj?= =?us-ascii?Q?ANlWeTFRdXQenp4bysbxOTjxsPkEjgsFdFjS7Y4H4u9KOlT2HlGIZjqxA35g?= =?us-ascii?Q?UlW5A23aIyHqsyYBkcBQMhzd3BQB/AkIQZ6tz/+0MyFfAvxKjBWMhvpi8ZtT?= =?us-ascii?Q?oSfwqR4EOCMoM1nKWHoC2+q/R8Lrd74AnpHChCFPdAPsi9fUJxgWQrL4Mie9?= =?us-ascii?Q?TNlRLFwdVGg9VQXTcpI5Y1+EbicN8rU7i7wRgvex3cPhNCgpgUb1gnBCcV5Q?= =?us-ascii?Q?gusj+39HyciDnjy6saTjgj6yvpR2Hx9fCKX/NEiUVpWbBMaY3Wa7N1T3wpM9?= =?us-ascii?Q?nYRr7ibgcjK9LFSp+Um4MbS0JCw0DplqOLZtTVwevFSmFW4w5yawoc3fc4jw?= =?us-ascii?Q?17/UrvsdzY8vsOg50i9iAYPuJgdtiQffDZwtDPnYphAx8Pt98csJ7ljuKR/b?= =?us-ascii?Q?6LW9TQUsc9P6l14V8YkqJ4EJL2SQkqKh9AIwv6+32w3BxpkejT8Y3jUUn0P5?= =?us-ascii?Q?OlESR4oHQ7XtDQzHQlxXF6Z8y1aEoWIXvIwCtM4KH86uUj5P/CbgolTq3fTT?= =?us-ascii?Q?/o8/S8zzMfgd7PoAy1kBzXhjiY0qn3us8cmRUOfnUSFBJx4W2BamIl0nsLOK?= =?us-ascii?Q?m4I7nYHExlnLhTyCKJu3IExQcqhbldqM0rCZe3HE8Gw5jSK/tu21Btt/W6U/?= =?us-ascii?Q?25wau6SP3s3i2VeU3QRYnPC48e0XbekEFDlOUeQPVivd8g2YoU+v2MvwRQA0?= =?us-ascii?Q?REY8gR5EJMjCXvFa2JS7BGTTy8yuHrNKei0c442K3ZtcDs97zz9Lv+oc/G4T?= =?us-ascii?Q?TmL+nagMoWTUC8JjBS6nAAJexr+vmcrDLNPDqPmaxhFt4bcsknv67tx0UjVR?= =?us-ascii?Q?lQlnWQPjKG+PLEH7rGYJ0quccfxR8iic9TjdHP7Lu0r7Adxvyiv2vVllfCHY?= =?us-ascii?Q?zqFROlQIdP7JKrMXIcqCvq6VLR/DlKSYNgsX86Y/oON/P694V7WkVXaITiXH?= =?us-ascii?Q?vviG7OGdUEH6Tt15mXSjU4+o3rmLuve2bX6qtOUfWzQjFIBae7Vwcgi5qQeV?= =?us-ascii?Q?0mHiotVSV69gUeamOM6dLKKPZXvtsOYm/U/PtJ2UxEVpVnvWpbbX1z16W9D/?= =?us-ascii?Q?Kv5eP3KPAUsYt8QnrdeGB+Yun+pgixVWc1D9+9lQxNSgYZKMBBc4hu/3s0V+?= =?us-ascii?Q?JA=3D=3D?= X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: nfkXx3GZM2wQiT1hHV37Xi/IbWxCCni1RBSUy9XoLhdVcD249Umyf21Q9tKJYeFgpf3vuVw3C9oGJig4cQgG1BtNQC9tA0QPCEI5gfOvsGCJkt71TFTInnG8aAIvZ7j95zSy/nunKon2MCqKHItLMKlNr2yCvuczVON9i5tVfvYlbYPSuXPPSbpiHHto7ayBBBrPOgT/QzcOz3M5caW0B6s+Ah0O80rwu2uTYL7idlTNc4OeFJgxghq6SR9PZFLHKx6OCViXYe8wpt3eNg71D7ETv4m5Q/wWN+WXK4P821tAWJSjM8UJ+UdinX0BzbW8J0k6Wucaj1Ed59ZAMmR6FDl1cQ/U++SoymGC2NwBN0d+ouSj7VzbpdDpqZmVBXj7KFj7+bqCH4g6jvgzArDXa0erNuSFnpjhwjMf7+3SKsIUP4QcbL1DENYcyDrnihcrAVfVEvxVf70iOgg9vhkQ7gGB6gOpQPtVWPbXv6ue1JtlaBaR7FMbjzNMJLmODZG2iv/qsDasHXYPPVJd96Mkn/7Yovn8u99m5bK8YHklR/4CqZvrtBh0yvC/nXYHRxya0l7/NSebfPV3/iQd98LMNVjZCQveLx9VBmGenmmh/qQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 25aa71ee-38fa-4312-2a1c-08de20a7988a X-MS-Exchange-CrossTenant-AuthSource: DM4PR10MB8218.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2025 22:22:12.5575 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 3LsFGTbK8iOJwZxQlzOOHyFhD59/p6LzkeZydDhnSXTfXE24LWza/Iv8C3Xl3ZyjoVwEw5v6bjLoAWWsaUj4E0tXrPlZ+qeuNCHxcSmURko= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR10MB7622 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1121,Hydra:6.1.9,FMLib:17.12.100.49 definitions=2025-11-10_07,2025-11-10_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 adultscore=0 malwarescore=0 spamscore=0 suspectscore=0 bulkscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2510240000 definitions=main-2511100190 X-Authority-Analysis: v=2.4 cv=WP5yn3sR c=1 sm=1 tr=0 ts=6912659a b=1 cx=c_pps a=e1sVV491RgrpLwSTMOnk8w==:117 a=e1sVV491RgrpLwSTMOnk8w==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=z/mQ4Ysz8XfWz/Q5cLBRGdckG28=:19 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=xqWC_Br6kY4A:10 a=6UeiqGixMTsA:10 a=GoEa3M9JfhUA:10 a=VkNPw1HP01LnGYTKEx00:22 a=yPCof4ZbAAAA:8 a=0t1AzniXCxXm8NuIC04A:9 cc=ntf awl=host:13634 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUxMTEwMDE2OCBTYWx0ZWRfX5HjwD9LYsd1N gAPx02txgluoX9MuH3ztbHnHnqzbGP1/fYHForf9mof/+yVBqejnEbUjOejsVTBxjmC+zhUN+ma PYwKKbSPBk5sKDHWrbL6qqahswT7Pt3rlXpKbRA0DSaLH5KmZMtxdQVXpe2QuHs9OhFBBakpEd0 LOWx3Aj6kZ7jRBRNZLr4dA3HMIRqgYFU8BG54AwhUeGqpjaNEIcXdWkUD4wKbYhrFPTj3PLkLBY 5hT2pCjVwh1tuRsHpQtozTKp/vEcAv7N4ZvKK7bOsJAtVZDJ6RaJ+P3WLOMcKQ6QzzzQiCYM7Nv XRuhsY9b7uNDm+qFzy0kRYK2Impk1kybaHSUqJ45euvceXd1j9qEfKmUYn5gKEnlGGaFgyLrk74 VgiUp0VNHoS509nB11YthjFIAG9a4G8LzUkbJEPAwTflc1aPaCg= X-Proofpoint-GUID: e4aJF2VH7aYZEPurwjyDyH_BLue2Wlcl X-Proofpoint-ORIG-GUID: e4aJF2VH7aYZEPurwjyDyH_BLue2Wlcl Content-Type: text/plain; charset="utf-8" There are straggler invocations of pte_to_swp_entry() lying around, replace all of these with the software leaf entry equivalent - softleaf_from_pte(). With those removed, eliminate pte_to_swp_entry() altogether. No functional change intended. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- include/linux/leafops.h | 7 ++++++- include/linux/swapops.h | 13 ------------- mm/debug_vm_pgtable.c | 2 +- mm/internal.h | 7 +++++-- mm/memory-failure.c | 2 +- mm/memory.c | 16 ++++++++-------- mm/migrate.c | 2 +- mm/mincore.c | 4 +++- mm/rmap.c | 8 ++++++-- mm/swapfile.c | 13 +++++++++++-- 10 files changed, 42 insertions(+), 32 deletions(-) diff --git a/include/linux/leafops.h b/include/linux/leafops.h index d282fab866a1..cfafe7a5e7b1 100644 --- a/include/linux/leafops.h +++ b/include/linux/leafops.h @@ -54,11 +54,16 @@ static inline softleaf_t softleaf_mk_none(void) */ static inline softleaf_t softleaf_from_pte(pte_t pte) { + softleaf_t arch_entry; + if (pte_present(pte) || pte_none(pte)) return softleaf_mk_none(); =20 + pte =3D pte_swp_clear_flags(pte); + arch_entry =3D __pte_to_swp_entry(pte); + /* Temporary until swp_entry_t eliminated. */ - return pte_to_swp_entry(pte); + return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); } =20 /** diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 3d02b288c15e..8cfc966eae48 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -107,19 +107,6 @@ static inline pgoff_t swp_offset(swp_entry_t entry) return entry.val & SWP_OFFSET_MASK; } =20 -/* - * Convert the arch-dependent pte representation of a swp_entry_t into an - * arch-independent swp_entry_t. - */ -static inline swp_entry_t pte_to_swp_entry(pte_t pte) -{ - swp_entry_t arch_entry; - - pte =3D pte_swp_clear_flags(pte); - arch_entry =3D __pte_to_swp_entry(pte); - return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); -} - /* * Convert the arch-independent representation of a swp_entry_t into the * arch-dependent pte representation. diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c index 64db85a80558..1eae87dbef73 100644 --- a/mm/debug_vm_pgtable.c +++ b/mm/debug_vm_pgtable.c @@ -1229,7 +1229,7 @@ static int __init init_args(struct pgtable_debug_args= *args) init_fixed_pfns(args); =20 /* See generic_max_swapfile_size(): probe the maximum offset */ - max_swap_offset =3D swp_offset(pte_to_swp_entry(swp_entry_to_pte(swp_entr= y(0, ~0UL)))); + max_swap_offset =3D swp_offset(softleaf_from_pte(softleaf_to_pte(swp_entr= y(0, ~0UL)))); /* Create a swp entry with all possible bits set while still being swap. = */ args->swp_entry =3D swp_entry(MAX_SWAPFILES - 1, max_swap_offset); /* Create a non-present migration entry. */ diff --git a/mm/internal.h b/mm/internal.h index f0c7461bb02c..985605ba3364 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -334,7 +334,7 @@ unsigned int folio_pte_batch(struct folio *folio, pte_t= *ptep, pte_t pte, */ static inline pte_t pte_move_swp_offset(pte_t pte, long delta) { - swp_entry_t entry =3D pte_to_swp_entry(pte); + const softleaf_t entry =3D softleaf_from_pte(pte); pte_t new =3D __swp_entry_to_pte(__swp_entry(swp_type(entry), (swp_offset(entry) + delta))); =20 @@ -389,11 +389,14 @@ static inline int swap_pte_batch(pte_t *start_ptep, i= nt max_nr, pte_t pte) =20 cgroup_id =3D lookup_swap_cgroup_id(entry); while (ptep < end_ptep) { + softleaf_t entry; + pte =3D ptep_get(ptep); =20 if (!pte_same(pte, expected_pte)) break; - if (lookup_swap_cgroup_id(pte_to_swp_entry(pte)) !=3D cgroup_id) + entry =3D softleaf_from_pte(pte); + if (lookup_swap_cgroup_id(entry) !=3D cgroup_id) break; expected_pte =3D pte_next_swp_offset(expected_pte); ptep++; diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 6e79da3de221..ca2204c4647e 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -50,7 +50,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/mm/memory.c b/mm/memory.c index accd275cd651..f9a2c608aff9 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1218,7 +1218,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, spinlock_t *src_ptl, *dst_ptl; int progress, max_nr, ret =3D 0; int rss[NR_MM_COUNTERS]; - swp_entry_t entry =3D (swp_entry_t){0}; + softleaf_t entry =3D softleaf_mk_none(); struct folio *prealloc =3D NULL; int nr; =20 @@ -1282,7 +1282,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct= vm_area_struct *src_vma, dst_vma, src_vma, addr, rss); if (ret =3D=3D -EIO) { - entry =3D pte_to_swp_entry(ptep_get(src_pte)); + entry =3D softleaf_from_pte(ptep_get(src_pte)); break; } else if (ret =3D=3D -EBUSY) { break; @@ -4456,13 +4456,13 @@ static struct folio *__alloc_swap_folio(struct vm_f= ault *vmf) { struct vm_area_struct *vma =3D vmf->vma; struct folio *folio; - swp_entry_t entry; + softleaf_t entry; =20 folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vmf->address); if (!folio) return NULL; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); + entry =3D softleaf_from_pte(vmf->orig_pte); if (mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, GFP_KERNEL, entry)) { folio_put(folio); @@ -4480,7 +4480,7 @@ static struct folio *__alloc_swap_folio(struct vm_fau= lt *vmf) static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages) { unsigned long addr; - swp_entry_t entry; + softleaf_t entry; int idx; pte_t pte; =20 @@ -4490,7 +4490,7 @@ static bool can_swapin_thp(struct vm_fault *vmf, pte_= t *ptep, int nr_pages) =20 if (!pte_same(pte, pte_move_swp_offset(vmf->orig_pte, -idx))) return false; - entry =3D pte_to_swp_entry(pte); + entry =3D softleaf_from_pte(pte); if (swap_pte_batch(ptep, nr_pages, pte) !=3D nr_pages) return false; =20 @@ -4536,7 +4536,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) unsigned long orders; struct folio *folio; unsigned long addr; - swp_entry_t entry; + softleaf_t entry; spinlock_t *ptl; pte_t *pte; gfp_t gfp; @@ -4557,7 +4557,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) if (!zswap_never_enabled()) goto fallback; =20 - entry =3D pte_to_swp_entry(vmf->orig_pte); + entry =3D softleaf_from_pte(vmf->orig_pte); /* * Get a list of all the (large) orders below PMD_ORDER that are enabled * and suitable for swapping THP. diff --git a/mm/migrate.c b/mm/migrate.c index 182a5b7b2ead..c01bc0ddf819 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -534,7 +534,7 @@ void migration_entry_wait_huge(struct vm_area_struct *v= ma, unsigned long addr, p * lock release in migration_entry_wait_on_locked(). */ hugetlb_vma_unlock_read(vma); - migration_entry_wait_on_locked(pte_to_swp_entry(pte), ptl); + migration_entry_wait_on_locked(entry, ptl); return; } =20 diff --git a/mm/mincore.c b/mm/mincore.c index 9a908d8bb706..e5d13eea9234 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -202,7 +202,9 @@ static int mincore_pte_range(pmd_t *pmd, unsigned long = addr, unsigned long end, for (i =3D 0; i < step; i++) vec[i] =3D 1; } else { /* pte is a swap entry */ - *vec =3D mincore_swap(pte_to_swp_entry(pte), false); + const softleaf_t entry =3D softleaf_from_pte(pte); + + *vec =3D mincore_swap(entry, false); } vec +=3D step; } diff --git a/mm/rmap.c b/mm/rmap.c index 345466ad396b..d871f2eb821c 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1969,7 +1969,9 @@ static bool try_to_unmap_one(struct folio *folio, str= uct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D softleaf_to_pfn(pte_to_swp_entry(pteval)); + const softleaf_t entry =3D softleaf_from_pte(pteval); + + pfn =3D softleaf_to_pfn(entry); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 @@ -2368,7 +2370,9 @@ static bool try_to_migrate_one(struct folio *folio, s= truct vm_area_struct *vma, if (likely(pte_present(pteval))) { pfn =3D pte_pfn(pteval); } else { - pfn =3D softleaf_to_pfn(pte_to_swp_entry(pteval)); + const softleaf_t entry =3D softleaf_from_pte(pteval); + + pfn =3D softleaf_to_pfn(entry); VM_WARN_ON_FOLIO(folio_test_hugetlb(folio), folio); } =20 diff --git a/mm/swapfile.c b/mm/swapfile.c index 684f78cd7dd1..e5667a31be9f 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -3201,8 +3201,17 @@ static int claim_swapfile(struct swap_info_struct *s= i, struct inode *inode) */ unsigned long generic_max_swapfile_size(void) { - return swp_offset(pte_to_swp_entry( - swp_entry_to_pte(swp_entry(0, ~0UL)))) + 1; + swp_entry_t entry =3D swp_entry(0, ~0UL); + const pte_t pte =3D softleaf_to_pte(entry); + + /* + * Since the PTE can be an invalid softleaf entry (e.g. the none PTE), + * we need to do this manually. + */ + entry =3D __pte_to_swp_entry(pte); + entry =3D swp_entry(__swp_type(entry), __swp_offset(entry)); + + return swp_offset(entry) + 1; } =20 /* Can be overridden by an architecture for additional checks. */ --=20 2.51.0